2024-04-24T23:51:45.481 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:51:45.481 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/scrub} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227 -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/orig.config.yaml 2024-04-24T23:51:45.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:51:45.626 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/scrub} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672227' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBh1yKy/w+Iuqtvo+V0dp6Y+nFJ9GfXW5D9HFtNVGyeSrmyOoTLyd/wLzSApMmmq5ZKJ2jQpdugIRG7pEO8eeCo= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyvInHSGUjooPxRpFHT0j3UX5iibjgkDiFlCFtO9PxVViY67e+DsfT8wviUYK9897jnLhLR3r73t4hUBpsPD7k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:51:45.626 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:51:45.627 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:51:45.627 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:51:45.629 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:51:45.653 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:51:45.653 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:51:45.653 INFO:teuthology.packaging:ref: None 2024-04-24T23:51:45.653 INFO:teuthology.packaging:tag: None 2024-04-24T23:51:45.653 INFO:teuthology.packaging:branch: squid 2024-04-24T23:51:45.653 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:51:45.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T23:51:45.963 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T23:51:45.965 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:51:45.971 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:51:45.971 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:51:45.977 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:51:45.994 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:51:46.000 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:51:46.025 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:46:13.755401', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBh1yKy/w+Iuqtvo+V0dp6Y+nFJ9GfXW5D9HFtNVGyeSrmyOoTLyd/wLzSApMmmq5ZKJ2jQpdugIRG7pEO8eeCo='} 2024-04-24T23:51:46.043 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:46:13.757335', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyvInHSGUjooPxRpFHT0j3UX5iibjgkDiFlCFtO9PxVViY67e+DsfT8wviUYK9897jnLhLR3r73t4hUBpsPD7k='} 2024-04-24T23:51:46.043 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:51:46.050 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-24T23:51:46.050 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-24T23:51:46.050 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:51:46.115 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb0ba8490d0>, signals=[15]) 2024-04-24T23:51:46.115 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:51:46.122 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:51:46.122 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:51:46.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:51:46.203 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:51:46.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:51:46.280 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:51:46.286 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-04-24T23:51:46.303 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-04-24T23:51:46.303 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:51:46.360 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:51:46.361 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:51:46.361 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:51:46.361 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:51:46.361 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:51:46.361 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-04-24T23:51:46.389 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-04-24T23:51:46.406 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-04-24T23:51:46.407 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:51:46.464 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-04-24T23:51:46.465 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:51:46.465 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:51:46.465 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:51:46.465 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:51:46.465 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:51:46.465 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:51:46.465 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-04-24T23:51:46.479 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:51:46.492 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:51:46.498 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:51:46.498 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:51:46.501 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:51:46.520 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:51:46.526 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:51:46.526 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:51:46.557 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:51:46.647 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:51:46.691 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:51:46.737 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:51:46.738 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-24T23:51:46.738 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-24T23:51:46.738 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:51:46.739 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:51:46.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:51:46.755 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-04-24T23:51:46.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:51:46.757 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-04-24T23:51:46.813 INFO:teuthology.orchestra.run.smithi113.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:51:46.813 INFO:teuthology.task.kernel:Running kernel on smithi113: 5.14.0-437.el9.x86_64 2024-04-24T23:51:46.814 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum install -y kernel 2024-04-24T23:51:46.815 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:51:46.815 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.14.0-437.el9.x86_64 2024-04-24T23:51:46.815 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2024-04-24T23:51:49.191 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-24T23:51:49.373 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-24T23:51:51.009 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T23:51:51.383 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-24T23:51:55.234 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-24T23:51:55.575 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-24T23:51:56.626 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-04-24T23:51:57.225 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-24T23:51:58.261 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-24T23:51:58.618 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T23:52:03.519 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 6.2 kB/s | 1.7 kB 00:00 2024-04-24T23:52:03.718 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-24T23:52:05.452 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:52:05.452 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:52:05.544 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-04-24T23:52:05.545 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-04-24T23:52:05.546 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-04-24T23:52:05.624 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:52:05.624 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:52:05.685 DEBUG:teuthology.orchestra.run.smithi178:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:52:05.717 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-24T23:52:05.718 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-24T23:52:05.718 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-24T23:52:05.848 DEBUG:teuthology.orchestra.run.smithi113:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:52:06.181 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:52:03 PM UTC. 2024-04-24T23:52:06.304 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-04-24T23:52:06.318 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-24T23:52:06.318 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-04-24T23:52:06.318 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-24T23:52:06.318 INFO:teuthology.orchestra.run.smithi178.stdout:Reinstalling: 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 255 k 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 0 2024-04-24T23:52:06.319 INFO:teuthology.orchestra.run.smithi178.stderr:Operation aborted. 2024-04-24T23:52:06.343 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:52:03 PM UTC. 2024-04-24T23:52:06.375 INFO:teuthology.orchestra.run.smithi178.stdout:Is this ok [y/N]: 2024-04-24T23:52:06.375 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum reinstall -y kernel || true 2024-04-24T23:52:06.479 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-24T23:52:06.491 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-24T23:52:06.491 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-24T23:52:06.491 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-24T23:52:06.491 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalling: 2024-04-24T23:52:06.491 INFO:teuthology.orchestra.run.smithi113.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:52:06.491 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:52:06.491 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-24T23:52:06.492 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-24T23:52:06.492 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:52:06.492 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 255 k 2024-04-24T23:52:06.492 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 0 2024-04-24T23:52:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:Operation aborted. 2024-04-24T23:52:06.551 INFO:teuthology.orchestra.run.smithi113.stdout:Is this ok [y/N]: 2024-04-24T23:52:06.551 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum reinstall -y kernel || true 2024-04-24T23:52:06.869 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:52:03 PM UTC. 2024-04-24T23:52:06.986 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-04-24T23:52:06.986 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout:Reinstalling: 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 255 k 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 0 2024-04-24T23:52:06.987 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-04-24T23:52:07.042 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:04 ago on Wed 24 Apr 2024 11:52:03 PM UTC. 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalling: 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-24T23:52:07.158 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-24T23:52:07.159 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:52:07.159 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 255 k 2024-04-24T23:52:07.159 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 0 2024-04-24T23:52:07.159 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-24T23:52:07.670 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-24T23:52:07.670 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:52:07.670 INFO:teuthology.orchestra.run.smithi113.stdout:Total 498 kB/s | 255 kB 00:00 2024-04-24T23:52:07.685 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-24T23:52:07.713 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-437.el9.x86_64.rpm 554 kB/s | 255 kB 00:00 2024-04-24T23:52:07.713 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:52:07.713 INFO:teuthology.orchestra.run.smithi178.stdout:Total 351 kB/s | 255 kB 00:00 2024-04-24T23:52:07.731 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-04-24T23:52:07.858 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-24T23:52:07.858 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-24T23:52:07.863 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-24T23:52:07.863 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-24T23:52:07.940 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-04-24T23:52:07.940 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-04-24T23:52:07.945 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-04-24T23:52:07.946 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-04-24T23:52:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-24T23:52:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-04-24T23:52:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:52:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:52:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:52:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:52:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:52:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:52:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:52:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:52:09.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:52:09.570 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:52:09.570 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalled: 2024-04-24T23:52:09.570 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:52:09.571 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:52:09.571 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-24T23:52:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:52:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:52:09.577 INFO:teuthology.orchestra.run.smithi178.stdout:Reinstalled: 2024-04-24T23:52:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:52:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:52:09.577 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-04-24T23:52:09.732 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:52:09.741 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:52:09.787 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:52:09.787 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:52:09.788 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:52:09.788 DEBUG:teuthology.parallel:result is None 2024-04-24T23:52:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:52:09.819 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:52:09.819 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:52:09.819 DEBUG:teuthology.parallel:result is None 2024-04-24T23:52:09.820 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:52:09.827 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:52:09.827 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:52:09.829 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:52:09.844 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:52:09.885 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:52:09.956 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:52:09.956 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:52:09.958 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:52:09.986 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:52:09.993 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:52:09.993 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:52:10.021 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:52:10.046 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:52:10.060 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:52:10.088 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:52:10.104 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:52:10.105 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:52:10.113 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:52:10.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:52:10.116 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:52:10.139 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:52:10.231 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:52:10.231 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:52:10.234 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:52:10.248 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:52:10.328 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:52:10.377 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:52:10.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:52:10.497 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:52:10.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:52:10.563 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:52:10.607 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:52:10.656 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:52:10.742 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:52:10.779 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:52:10.779 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:52:10.843 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-04-24T23:52:10.845 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-04-24T23:52:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:52:10.914 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:52:11.387 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:52:11.395 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:52:11.396 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:52:11.559 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:52:11.610 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T23:52:11.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-04-24T23:52:11.664 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-04-24T23:52:11.665 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-04-24T23:52:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-04-24T23:52:11.704 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:52:11.704 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-04-24T23:52:11.737 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-04-24T23:52:11.737 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-04-24T23:52:11.767 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-04-24T23:52:11.767 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi113.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-04-24T23:52:11.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:52:11.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:52:11.802 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:52:11.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:52:11.832 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:52:11.832 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:52:11.871 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:52:11.897 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:52:11.907 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T23:52:11.914 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T23:52:12.060 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:52:12.077 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:52:12.078 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi113.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T23:55:39.686 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi113.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-04-24T23:55:39.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-24T23:55:39.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:55:39.763 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-04-24T23:55:39.842 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-24T23:55:39.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-24T23:55:39.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:55:39.918 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-04-24T23:55:39.998 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-24T23:55:39.998 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T23:55:40.008 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T23:55:40.008 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:55:40.009 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:55:40.011 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:55:40.011 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:55:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:55:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:55:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:55:40.074 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-04-24T23:55:40.082 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:55:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-04-24T23:55:40.097 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:55:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-04-24T23:55:40.110 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:55:40.116 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-04-24T23:55:40.129 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:55:40.142 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:55:40.163 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:55:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:55:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-24T23:55:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.199 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:55:40.203 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:55:40.203 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-04-24T23:55:40.203 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.203 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.203 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.203 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:55:40.203 INFO:teuthology.run_tasks:Running task install... 2024-04-24T23:55:40.212 DEBUG:teuthology.task.install:project ceph 2024-04-24T23:55:40.212 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T23:55:40.212 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:55:40.212 INFO:teuthology.task.install:Using flavor: default 2024-04-24T23:55:40.218 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T23:55:40.218 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T23:55:40.218 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:55:40.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:55:40.220 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:55:40.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:55:40.367 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:55:40.367 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:55:40.371 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:55:40.371 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:55:40.534 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:55:40.534 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:55:40.534 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:55:40.539 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:55:40.539 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:55:40.539 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:55:40.559 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:55:40.560 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:55:40.564 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:55:40.564 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:55:40.623 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:55:40.628 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:55:40.696 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:55:40.700 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:55:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-04-24T23:55:40.760 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-04-24T23:55:40.763 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-04-24T23:55:40.764 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-04-24T23:55:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:48 files removed 2024-04-24T23:55:40.990 INFO:teuthology.orchestra.run.smithi178.stdout:48 files removed 2024-04-24T23:55:41.013 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:55:41.017 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:55:41.667 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 212 kB/s | 83 kB 00:00 2024-04-24T23:55:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-24T23:55:42.059 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-04-24T23:55:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-04-24T23:55:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-24T23:55:42.476 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-24T23:55:43.093 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-24T23:55:44.786 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-04-24T23:55:47.439 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.0 MB 00:04 2024-04-24T23:55:48.945 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-24T23:55:50.207 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-24T23:55:54.633 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-04-24T23:55:55.387 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 3.1 kB/s | 16 kB 00:05 2024-04-24T23:55:56.075 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-24T23:55:56.563 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-24T23:55:57.202 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-24T23:56:01.547 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:56:02.393 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:56:03.647 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:56:03.647 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:56:03.658 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:56:03.659 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:56:03.660 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:56:03.660 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:56:03.662 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:56:03.664 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:56:03.665 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:56:03.726 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:56:03.735 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:56:03.736 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:56:03.737 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:56:03.738 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:56:03.739 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:56:03.740 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:56:03.741 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:56:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:56:03.743 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:56:03.744 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:56:03.745 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout:Install 148 Packages 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 4 Packages 2024-04-24T23:56:03.746 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:03.747 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 220 M 2024-04-24T23:56:03.747 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-04-24T23:56:04.511 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:56:04.511 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:56:04.522 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:56:04.523 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:56:04.523 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:56:04.524 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:56:04.526 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:56:04.528 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:56:04.529 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:56:04.589 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-24T23:56:04.598 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-24T23:56:04.598 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-24T23:56:04.598 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:56:04.599 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:56:04.600 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:56:04.601 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:56:04.602 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:56:04.603 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:56:04.604 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:56:04.605 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:56:04.606 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:56:04.607 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:56:04.608 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:56:04.609 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout:Install 148 Packages 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 4 Packages 2024-04-24T23:56:04.610 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:04.611 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 220 M 2024-04-24T23:56:04.611 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-24T23:56:04.648 INFO:teuthology.orchestra.run.smithi178.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-04-24T23:56:04.956 INFO:teuthology.orchestra.run.smithi178.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.8 MB/s | 890 kB 00:00 2024-04-24T23:56:05.031 INFO:teuthology.orchestra.run.smithi178.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.9 MB/s | 145 kB 00:00 2024-04-24T23:56:05.098 INFO:teuthology.orchestra.run.smithi178.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 8.7 MB/s | 5.4 MB 00:00 2024-04-24T23:56:05.248 INFO:teuthology.orchestra.run.smithi178.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 1.5 MB 00:00 2024-04-24T23:56:05.286 INFO:teuthology.orchestra.run.smithi113.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-24T23:56:05.632 INFO:teuthology.orchestra.run.smithi178.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 3.9 MB/s | 2.4 MB 00:00 2024-04-24T23:56:05.699 INFO:teuthology.orchestra.run.smithi178.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 4.7 MB 00:00 2024-04-24T23:56:06.245 INFO:teuthology.orchestra.run.smithi113.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 929 kB/s | 890 kB 00:00 2024-04-24T23:56:06.453 INFO:teuthology.orchestra.run.smithi113.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 695 kB/s | 145 kB 00:00 2024-04-24T23:56:06.949 INFO:teuthology.orchestra.run.smithi178.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 8.5 MB/s | 21 MB 00:02 2024-04-24T23:56:07.037 INFO:teuthology.orchestra.run.smithi113.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-24T23:56:07.049 INFO:teuthology.orchestra.run.smithi178.stdout:(9/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 7.6 MB/s | 10 MB 00:01 2024-04-24T23:56:07.066 INFO:teuthology.orchestra.run.smithi178.stdout:(10/152): ceph-selinux-19.0.0-2445.gd2c45d97.el 215 kB/s | 25 kB 00:00 2024-04-24T23:56:07.133 INFO:teuthology.orchestra.run.smithi178.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.3 MB/s | 222 kB 00:00 2024-04-24T23:56:07.200 INFO:teuthology.orchestra.run.smithi178.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 406 kB/s | 27 kB 00:00 2024-04-24T23:56:07.275 INFO:teuthology.orchestra.run.smithi178.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 424 kB/s | 32 kB 00:00 2024-04-24T23:56:07.304 INFO:teuthology.orchestra.run.smithi113.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-24T23:56:07.426 INFO:teuthology.orchestra.run.smithi178.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 4.9 MB/s | 745 kB 00:00 2024-04-24T23:56:07.509 INFO:teuthology.orchestra.run.smithi178.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.9 MB/s | 163 kB 00:00 2024-04-24T23:56:07.529 INFO:teuthology.orchestra.run.smithi113.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-04-24T23:56:07.593 INFO:teuthology.orchestra.run.smithi178.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.5 MB/s | 127 kB 00:00 2024-04-24T23:56:07.693 INFO:teuthology.orchestra.run.smithi178.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 4.9 MB/s | 501 kB 00:00 2024-04-24T23:56:08.146 INFO:teuthology.orchestra.run.smithi113.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.6 MB/s | 4.7 MB 00:00 2024-04-24T23:56:08.560 INFO:teuthology.orchestra.run.smithi178.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 6.1 MB/s | 5.3 MB 00:00 2024-04-24T23:56:08.652 INFO:teuthology.orchestra.run.smithi178.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 494 kB/s | 45 kB 00:00 2024-04-24T23:56:08.744 INFO:teuthology.orchestra.run.smithi178.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.4 MB/s | 133 kB 00:00 2024-04-24T23:56:08.836 INFO:teuthology.orchestra.run.smithi178.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.7 MB/s | 162 kB 00:00 2024-04-24T23:56:08.936 INFO:teuthology.orchestra.run.smithi178.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.1 MB/s | 321 kB 00:00 2024-04-24T23:56:09.162 INFO:teuthology.orchestra.run.smithi178.stdout:(23/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 4.8 MB/s | 17 MB 00:03 2024-04-24T23:56:09.189 INFO:teuthology.orchestra.run.smithi178.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 299 kB 00:00 2024-04-24T23:56:09.231 INFO:teuthology.orchestra.run.smithi178.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.4 MB/s | 99 kB 00:00 2024-04-24T23:56:09.264 INFO:teuthology.orchestra.run.smithi178.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T23:56:09.356 INFO:teuthology.orchestra.run.smithi178.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-24T23:56:09.440 INFO:teuthology.orchestra.run.smithi178.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 381 kB/s | 32 kB 00:00 2024-04-24T23:56:09.540 INFO:teuthology.orchestra.run.smithi178.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.4 MB/s | 141 kB 00:00 2024-04-24T23:56:09.982 INFO:teuthology.orchestra.run.smithi178.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.1 MB/s | 3.1 MB 00:00 2024-04-24T23:56:10.137 INFO:teuthology.orchestra.run.smithi113.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 5.1 MB/s | 10 MB 00:01 2024-04-24T23:56:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 5.2 MB/s | 3.5 MB 00:00 2024-04-24T23:56:10.238 INFO:teuthology.orchestra.run.smithi113.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 252 kB/s | 25 kB 00:00 2024-04-24T23:56:10.300 INFO:teuthology.orchestra.run.smithi178.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.9 MB/s | 249 kB 00:00 2024-04-24T23:56:10.383 INFO:teuthology.orchestra.run.smithi178.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 594 kB/s | 49 kB 00:00 2024-04-24T23:56:10.467 INFO:teuthology.orchestra.run.smithi178.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 184 kB/s | 15 kB 00:00 2024-04-24T23:56:10.559 INFO:teuthology.orchestra.run.smithi178.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.8 MB/s | 264 kB 00:00 2024-04-24T23:56:10.792 INFO:teuthology.orchestra.run.smithi178.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.2 MB/s | 762 kB 00:00 2024-04-24T23:56:10.859 INFO:teuthology.orchestra.run.smithi178.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 274 kB/s | 18 kB 00:00 2024-04-24T23:56:10.959 INFO:teuthology.orchestra.run.smithi178.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 804 kB/s | 80 kB 00:00 2024-04-24T23:56:10.971 INFO:teuthology.orchestra.run.smithi113.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 4.9 MB/s | 17 MB 00:03 2024-04-24T23:56:11.060 INFO:teuthology.orchestra.run.smithi178.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2024-04-24T23:56:11.071 INFO:teuthology.orchestra.run.smithi113.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.2 MB/s | 222 kB 00:00 2024-04-24T23:56:11.101 INFO:teuthology.orchestra.run.smithi178.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-24T23:56:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-24T23:56:11.172 INFO:teuthology.orchestra.run.smithi113.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 269 kB/s | 27 kB 00:00 2024-04-24T23:56:11.219 INFO:teuthology.orchestra.run.smithi178.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-24T23:56:11.256 INFO:teuthology.orchestra.run.smithi113.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 381 kB/s | 32 kB 00:00 2024-04-24T23:56:11.260 INFO:teuthology.orchestra.run.smithi178.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-24T23:56:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-24T23:56:11.345 INFO:teuthology.orchestra.run.smithi178.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-24T23:56:11.387 INFO:teuthology.orchestra.run.smithi178.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-24T23:56:11.414 INFO:teuthology.orchestra.run.smithi113.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 4.6 MB/s | 745 kB 00:00 2024-04-24T23:56:11.487 INFO:teuthology.orchestra.run.smithi178.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-04-24T23:56:11.515 INFO:teuthology.orchestra.run.smithi113.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-24T23:56:11.596 INFO:teuthology.orchestra.run.smithi178.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-04-24T23:56:11.607 INFO:teuthology.orchestra.run.smithi113.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.4 MB/s | 127 kB 00:00 2024-04-24T23:56:11.679 INFO:teuthology.orchestra.run.smithi178.stdout:(49/152): ceph-mgr-diskprediction-local-19.0.0- 4.4 MB/s | 7.4 MB 00:01 2024-04-24T23:56:11.690 INFO:teuthology.orchestra.run.smithi113.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.9 MB/s | 501 kB 00:00 2024-04-24T23:56:12.113 INFO:teuthology.orchestra.run.smithi178.stdout:(50/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 9.5 MB/s | 48 MB 00:05 2024-04-24T23:56:12.154 INFO:teuthology.orchestra.run.smithi178.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 191 kB/s | 107 kB 00:00 2024-04-24T23:56:12.180 INFO:teuthology.orchestra.run.smithi178.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 43 kB/s | 21 kB 00:00 2024-04-24T23:56:12.207 INFO:teuthology.orchestra.run.smithi113.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 5.3 MB 00:00 2024-04-24T23:56:12.222 INFO:teuthology.orchestra.run.smithi178.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-04-24T23:56:12.247 INFO:teuthology.orchestra.run.smithi178.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-24T23:56:12.274 INFO:teuthology.orchestra.run.smithi113.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 681 kB/s | 45 kB 00:00 2024-04-24T23:56:12.281 INFO:teuthology.orchestra.run.smithi178.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-24T23:56:12.298 INFO:teuthology.orchestra.run.smithi178.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-24T23:56:12.341 INFO:teuthology.orchestra.run.smithi113.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.0 MB/s | 133 kB 00:00 2024-04-24T23:56:12.373 INFO:teuthology.orchestra.run.smithi178.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-24T23:56:12.424 INFO:teuthology.orchestra.run.smithi113.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.9 MB/s | 162 kB 00:00 2024-04-24T23:56:12.457 INFO:teuthology.orchestra.run.smithi178.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 285 kB/s | 45 kB 00:00 2024-04-24T23:56:12.482 INFO:teuthology.orchestra.run.smithi178.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 798 kB/s | 160 kB 00:00 2024-04-24T23:56:12.525 INFO:teuthology.orchestra.run.smithi113.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.1 MB/s | 321 kB 00:00 2024-04-24T23:56:12.532 INFO:teuthology.orchestra.run.smithi178.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 510 kB/s | 25 kB 00:00 2024-04-24T23:56:12.558 INFO:teuthology.orchestra.run.smithi178.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-24T23:56:12.599 INFO:teuthology.orchestra.run.smithi178.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-24T23:56:12.600 INFO:teuthology.orchestra.run.smithi113.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 299 kB 00:00 2024-04-24T23:56:12.625 INFO:teuthology.orchestra.run.smithi178.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-04-24T23:56:12.675 INFO:teuthology.orchestra.run.smithi178.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-24T23:56:12.676 INFO:teuthology.orchestra.run.smithi113.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.3 MB/s | 99 kB 00:00 2024-04-24T23:56:12.700 INFO:teuthology.orchestra.run.smithi178.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-04-24T23:56:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-24T23:56:12.742 INFO:teuthology.orchestra.run.smithi113.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-24T23:56:12.859 INFO:teuthology.orchestra.run.smithi178.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-24T23:56:12.959 INFO:teuthology.orchestra.run.smithi178.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-04-24T23:56:13.026 INFO:teuthology.orchestra.run.smithi178.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-24T23:56:13.076 INFO:teuthology.orchestra.run.smithi113.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 9.3 MB/s | 3.1 MB 00:00 2024-04-24T23:56:13.085 INFO:teuthology.orchestra.run.smithi178.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-24T23:56:13.118 INFO:teuthology.orchestra.run.smithi178.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-04-24T23:56:13.152 INFO:teuthology.orchestra.run.smithi113.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-24T23:56:13.185 INFO:teuthology.orchestra.run.smithi178.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-24T23:56:13.219 INFO:teuthology.orchestra.run.smithi113.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 478 kB/s | 32 kB 00:00 2024-04-24T23:56:13.235 INFO:teuthology.orchestra.run.smithi178.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-24T23:56:13.294 INFO:teuthology.orchestra.run.smithi113.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.8 MB/s | 141 kB 00:00 2024-04-24T23:56:13.344 INFO:teuthology.orchestra.run.smithi178.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-24T23:56:13.386 INFO:teuthology.orchestra.run.smithi178.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-04-24T23:56:13.453 INFO:teuthology.orchestra.run.smithi178.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-24T23:56:13.478 INFO:teuthology.orchestra.run.smithi178.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 278 kB/s | 26 kB 00:00 2024-04-24T23:56:13.520 INFO:teuthology.orchestra.run.smithi178.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-04-24T23:56:13.562 INFO:teuthology.orchestra.run.smithi178.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-24T23:56:13.588 INFO:teuthology.orchestra.run.smithi178.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-04-24T23:56:13.652 INFO:teuthology.orchestra.run.smithi113.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.8 MB/s | 3.5 MB 00:00 2024-04-24T23:56:13.671 INFO:teuthology.orchestra.run.smithi178.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-24T23:56:13.755 INFO:teuthology.orchestra.run.smithi178.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-24T23:56:13.830 INFO:teuthology.orchestra.run.smithi178.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-24T23:56:13.855 INFO:teuthology.orchestra.run.smithi178.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-24T23:56:13.922 INFO:teuthology.orchestra.run.smithi178.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-24T23:56:13.948 INFO:teuthology.orchestra.run.smithi178.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 360 kB/s | 42 kB 00:00 2024-04-24T23:56:14.006 INFO:teuthology.orchestra.run.smithi178.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-04-24T23:56:14.031 INFO:teuthology.orchestra.run.smithi178.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-24T23:56:14.057 INFO:teuthology.orchestra.run.smithi178.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-24T23:56:14.082 INFO:teuthology.orchestra.run.smithi178.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-04-24T23:56:14.107 INFO:teuthology.orchestra.run.smithi178.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-24T23:56:14.149 INFO:teuthology.orchestra.run.smithi178.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-04-24T23:56:14.174 INFO:teuthology.orchestra.run.smithi178.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-04-24T23:56:14.224 INFO:teuthology.orchestra.run.smithi178.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-04-24T23:56:14.250 INFO:teuthology.orchestra.run.smithi178.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-24T23:56:14.292 INFO:teuthology.orchestra.run.smithi178.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-04-24T23:56:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-24T23:56:14.342 INFO:teuthology.orchestra.run.smithi178.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-24T23:56:14.359 INFO:teuthology.orchestra.run.smithi178.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-04-24T23:56:14.403 INFO:teuthology.orchestra.run.smithi113.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.3 MB/s | 249 kB 00:00 2024-04-24T23:56:14.426 INFO:teuthology.orchestra.run.smithi178.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-24T23:56:14.443 INFO:teuthology.orchestra.run.smithi178.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-04-24T23:56:14.478 INFO:teuthology.orchestra.run.smithi113.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 658 kB/s | 49 kB 00:00 2024-04-24T23:56:14.493 INFO:teuthology.orchestra.run.smithi178.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-04-24T23:56:14.510 INFO:teuthology.orchestra.run.smithi178.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-04-24T23:56:14.687 INFO:teuthology.orchestra.run.smithi113.stdout:(34/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 21 MB 00:09 2024-04-24T23:56:14.712 INFO:teuthology.orchestra.run.smithi113.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 66 kB/s | 15 kB 00:00 2024-04-24T23:56:14.744 INFO:teuthology.orchestra.run.smithi178.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-24T23:56:14.763 INFO:teuthology.orchestra.run.smithi113.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.4 MB/s | 264 kB 00:00 2024-04-24T23:56:14.829 INFO:teuthology.orchestra.run.smithi113.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 275 kB/s | 18 kB 00:00 2024-04-24T23:56:14.977 INFO:teuthology.orchestra.run.smithi178.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-24T23:56:15.086 INFO:teuthology.orchestra.run.smithi178.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 952 kB/s | 548 kB 00:00 2024-04-24T23:56:15.169 INFO:teuthology.orchestra.run.smithi178.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 728 kB/s | 309 kB 00:00 2024-04-24T23:56:15.211 INFO:teuthology.orchestra.run.smithi178.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 83 kB/s | 19 kB 00:00 2024-04-24T23:56:15.245 INFO:teuthology.orchestra.run.smithi178.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-04-24T23:56:15.255 INFO:teuthology.orchestra.run.smithi113.stdout:(38/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 9.6 MB/s | 48 MB 00:05 2024-04-24T23:56:15.280 INFO:teuthology.orchestra.run.smithi113.stdout:(39/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 1.3 MB/s | 762 kB 00:00 2024-04-24T23:56:15.329 INFO:teuthology.orchestra.run.smithi178.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-04-24T23:56:15.354 INFO:teuthology.orchestra.run.smithi178.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 341 kB/s | 49 kB 00:00 2024-04-24T23:56:15.397 INFO:teuthology.orchestra.run.smithi113.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 142 kB/s | 80 kB 00:00 2024-04-24T23:56:15.421 INFO:teuthology.orchestra.run.smithi178.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-24T23:56:15.446 INFO:teuthology.orchestra.run.smithi178.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-24T23:56:15.472 INFO:teuthology.orchestra.run.smithi113.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-24T23:56:15.496 INFO:teuthology.orchestra.run.smithi178.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-24T23:56:15.497 INFO:teuthology.orchestra.run.smithi113.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-24T23:56:15.530 INFO:teuthology.orchestra.run.smithi178.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 518 kB/s | 43 kB 00:00 2024-04-24T23:56:15.564 INFO:teuthology.orchestra.run.smithi178.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-04-24T23:56:15.581 INFO:teuthology.orchestra.run.smithi113.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-04-24T23:56:15.597 INFO:teuthology.orchestra.run.smithi178.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-24T23:56:15.622 INFO:teuthology.orchestra.run.smithi178.stdout:(117/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-24T23:56:15.648 INFO:teuthology.orchestra.run.smithi178.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-24T23:56:15.648 INFO:teuthology.orchestra.run.smithi113.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 845 kB/s | 106 kB 00:00 2024-04-24T23:56:15.682 INFO:teuthology.orchestra.run.smithi113.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-04-24T23:56:15.706 INFO:teuthology.orchestra.run.smithi178.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-24T23:56:15.732 INFO:teuthology.orchestra.run.smithi178.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-24T23:56:15.732 INFO:teuthology.orchestra.run.smithi113.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 903 kB/s | 135 kB 00:00 2024-04-24T23:56:15.757 INFO:teuthology.orchestra.run.smithi178.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-24T23:56:15.782 INFO:teuthology.orchestra.run.smithi178.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-24T23:56:15.799 INFO:teuthology.orchestra.run.smithi113.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 830 kB/s | 125 kB 00:00 2024-04-24T23:56:15.841 INFO:teuthology.orchestra.run.smithi178.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-24T23:56:15.857 INFO:teuthology.orchestra.run.smithi113.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-04-24T23:56:15.866 INFO:teuthology.orchestra.run.smithi178.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-24T23:56:15.908 INFO:teuthology.orchestra.run.smithi178.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-24T23:56:15.933 INFO:teuthology.orchestra.run.smithi178.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-04-24T23:56:15.984 INFO:teuthology.orchestra.run.smithi178.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-24T23:56:16.009 INFO:teuthology.orchestra.run.smithi178.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-24T23:56:16.024 INFO:teuthology.orchestra.run.smithi113.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-04-24T23:56:16.034 INFO:teuthology.orchestra.run.smithi178.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-24T23:56:16.050 INFO:teuthology.orchestra.run.smithi113.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 557 kB/s | 107 kB 00:00 2024-04-24T23:56:16.083 INFO:teuthology.orchestra.run.smithi113.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 365 kB/s | 21 kB 00:00 2024-04-24T23:56:16.084 INFO:teuthology.orchestra.run.smithi178.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-24T23:56:16.109 INFO:teuthology.orchestra.run.smithi178.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-04-24T23:56:16.143 INFO:teuthology.orchestra.run.smithi178.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-04-24T23:56:16.151 INFO:teuthology.orchestra.run.smithi113.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-24T23:56:16.168 INFO:teuthology.orchestra.run.smithi178.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-04-24T23:56:16.176 INFO:teuthology.orchestra.run.smithi113.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-24T23:56:16.194 INFO:teuthology.orchestra.run.smithi178.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-24T23:56:16.201 INFO:teuthology.orchestra.run.smithi113.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-04-24T23:56:16.219 INFO:teuthology.orchestra.run.smithi178.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-24T23:56:16.235 INFO:teuthology.orchestra.run.smithi113.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-24T23:56:16.244 INFO:teuthology.orchestra.run.smithi178.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-24T23:56:16.260 INFO:teuthology.orchestra.run.smithi113.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-24T23:56:16.269 INFO:teuthology.orchestra.run.smithi178.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-24T23:56:16.294 INFO:teuthology.orchestra.run.smithi113.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-24T23:56:16.295 INFO:teuthology.orchestra.run.smithi178.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-24T23:56:16.319 INFO:teuthology.orchestra.run.smithi113.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-04-24T23:56:16.320 INFO:teuthology.orchestra.run.smithi178.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-24T23:56:16.345 INFO:teuthology.orchestra.run.smithi178.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-04-24T23:56:16.370 INFO:teuthology.orchestra.run.smithi178.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-24T23:56:16.394 INFO:teuthology.orchestra.run.smithi113.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-24T23:56:16.395 INFO:teuthology.orchestra.run.smithi178.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-24T23:56:16.421 INFO:teuthology.orchestra.run.smithi178.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-24T23:56:16.428 INFO:teuthology.orchestra.run.smithi113.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-24T23:56:16.446 INFO:teuthology.orchestra.run.smithi178.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T23:56:16.453 INFO:teuthology.orchestra.run.smithi113.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 433 kB/s | 25 kB 00:00 2024-04-24T23:56:16.480 INFO:teuthology.orchestra.run.smithi178.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-24T23:56:16.503 INFO:teuthology.orchestra.run.smithi113.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-24T23:56:16.514 INFO:teuthology.orchestra.run.smithi178.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-04-24T23:56:16.529 INFO:teuthology.orchestra.run.smithi113.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-04-24T23:56:16.539 INFO:teuthology.orchestra.run.smithi178.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-24T23:56:16.554 INFO:teuthology.orchestra.run.smithi113.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-24T23:56:16.579 INFO:teuthology.orchestra.run.smithi113.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-24T23:56:16.597 INFO:teuthology.orchestra.run.smithi178.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-24T23:56:16.604 INFO:teuthology.orchestra.run.smithi113.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-04-24T23:56:16.756 INFO:teuthology.orchestra.run.smithi178.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 1.3 MB 00:00 2024-04-24T23:56:16.780 INFO:teuthology.orchestra.run.smithi113.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-24T23:56:16.857 INFO:teuthology.orchestra.run.smithi178.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-24T23:56:16.906 INFO:teuthology.orchestra.run.smithi178.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 8.5 MB/s | 3.1 MB 00:00 2024-04-24T23:56:16.930 INFO:teuthology.orchestra.run.smithi113.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-24T23:56:16.955 INFO:teuthology.orchestra.run.smithi113.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-04-24T23:56:17.022 INFO:teuthology.orchestra.run.smithi113.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-24T23:56:17.032 INFO:teuthology.orchestra.run.smithi178.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 6.5 MB/s | 3.4 MB 00:00 2024-04-24T23:56:17.038 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:56:17.038 INFO:teuthology.orchestra.run.smithi178.stdout:Total 17 MB/s | 220 MB 00:13 2024-04-24T23:56:17.048 INFO:teuthology.orchestra.run.smithi113.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-24T23:56:17.106 INFO:teuthology.orchestra.run.smithi113.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-24T23:56:17.140 INFO:teuthology.orchestra.run.smithi113.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-04-24T23:56:17.207 INFO:teuthology.orchestra.run.smithi113.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-24T23:56:17.282 INFO:teuthology.orchestra.run.smithi113.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-24T23:56:17.358 INFO:teuthology.orchestra.run.smithi113.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-04-24T23:56:17.483 INFO:teuthology.orchestra.run.smithi113.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-24T23:56:17.508 INFO:teuthology.orchestra.run.smithi113.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 127 kB/s | 19 kB 00:00 2024-04-24T23:56:17.559 INFO:teuthology.orchestra.run.smithi113.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-24T23:56:17.592 INFO:teuthology.orchestra.run.smithi113.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 414 kB/s | 35 kB 00:00 2024-04-24T23:56:17.668 INFO:teuthology.orchestra.run.smithi113.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-24T23:56:17.693 INFO:teuthology.orchestra.run.smithi113.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-24T23:56:17.760 INFO:teuthology.orchestra.run.smithi113.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-24T23:56:17.793 INFO:teuthology.orchestra.run.smithi113.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 769 kB/s | 77 kB 00:00 2024-04-24T23:56:17.827 INFO:teuthology.orchestra.run.smithi113.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-04-24T23:56:17.877 INFO:teuthology.orchestra.run.smithi113.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-04-24T23:56:17.902 INFO:teuthology.orchestra.run.smithi113.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-24T23:56:17.986 INFO:teuthology.orchestra.run.smithi113.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T23:56:18.011 INFO:teuthology.orchestra.run.smithi113.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-04-24T23:56:18.037 INFO:teuthology.orchestra.run.smithi113.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-04-24T23:56:18.062 INFO:teuthology.orchestra.run.smithi113.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-04-24T23:56:18.104 INFO:teuthology.orchestra.run.smithi113.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-04-24T23:56:18.129 INFO:teuthology.orchestra.run.smithi113.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 658 kB/s | 44 kB 00:00 2024-04-24T23:56:18.159 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-04-24T23:56:18.171 INFO:teuthology.orchestra.run.smithi113.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-24T23:56:18.197 INFO:teuthology.orchestra.run.smithi113.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-04-24T23:56:18.230 INFO:teuthology.orchestra.run.smithi113.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-24T23:56:18.264 INFO:teuthology.orchestra.run.smithi113.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-24T23:56:18.298 INFO:teuthology.orchestra.run.smithi113.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 860 kB/s | 58 kB 00:00 2024-04-24T23:56:18.302 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-04-24T23:56:18.302 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-04-24T23:56:18.331 INFO:teuthology.orchestra.run.smithi113.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-24T23:56:18.365 INFO:teuthology.orchestra.run.smithi113.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-04-24T23:56:18.398 INFO:teuthology.orchestra.run.smithi113.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-04-24T23:56:18.740 INFO:teuthology.orchestra.run.smithi113.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-24T23:56:18.765 INFO:teuthology.orchestra.run.smithi113.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 56 kB/s | 22 kB 00:00 2024-04-24T23:56:19.182 INFO:teuthology.orchestra.run.smithi113.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-24T23:56:19.391 INFO:teuthology.orchestra.run.smithi113.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 494 kB/s | 309 kB 00:00 2024-04-24T23:56:19.474 INFO:teuthology.orchestra.run.smithi113.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 747 kB/s | 548 kB 00:00 2024-04-24T23:56:19.533 INFO:teuthology.orchestra.run.smithi113.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 55 kB/s | 19 kB 00:00 2024-04-24T23:56:19.567 INFO:teuthology.orchestra.run.smithi113.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 281 kB/s | 26 kB 00:00 2024-04-24T23:56:19.667 INFO:teuthology.orchestra.run.smithi113.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-04-24T23:56:19.709 INFO:teuthology.orchestra.run.smithi113.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2024-04-24T23:56:19.776 INFO:teuthology.orchestra.run.smithi113.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-24T23:56:19.868 INFO:teuthology.orchestra.run.smithi113.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-24T23:56:19.935 INFO:teuthology.orchestra.run.smithi113.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-24T23:56:19.969 INFO:teuthology.orchestra.run.smithi113.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-04-24T23:56:20.036 INFO:teuthology.orchestra.run.smithi113.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-24T23:56:20.044 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-04-24T23:56:20.044 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-04-24T23:56:20.069 INFO:teuthology.orchestra.run.smithi113.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-24T23:56:20.103 INFO:teuthology.orchestra.run.smithi113.stdout:(117/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 838 kB 00:00 2024-04-24T23:56:20.145 INFO:teuthology.orchestra.run.smithi113.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-24T23:56:20.170 INFO:teuthology.orchestra.run.smithi113.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-24T23:56:20.220 INFO:teuthology.orchestra.run.smithi113.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-24T23:56:20.246 INFO:teuthology.orchestra.run.smithi113.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-04-24T23:56:20.271 INFO:teuthology.orchestra.run.smithi113.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-24T23:56:20.355 INFO:teuthology.orchestra.run.smithi113.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-24T23:56:20.380 INFO:teuthology.orchestra.run.smithi113.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-24T23:56:20.455 INFO:teuthology.orchestra.run.smithi113.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-24T23:56:20.480 INFO:teuthology.orchestra.run.smithi113.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-24T23:56:20.531 INFO:teuthology.orchestra.run.smithi113.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-04-24T23:56:20.556 INFO:teuthology.orchestra.run.smithi113.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-24T23:56:20.581 INFO:teuthology.orchestra.run.smithi113.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-04-24T23:56:20.665 INFO:teuthology.orchestra.run.smithi113.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-24T23:56:20.690 INFO:teuthology.orchestra.run.smithi113.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-04-24T23:56:20.716 INFO:teuthology.orchestra.run.smithi113.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 589 kB/s | 79 kB 00:00 2024-04-24T23:56:20.766 INFO:teuthology.orchestra.run.smithi113.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-04-24T23:56:20.808 INFO:teuthology.orchestra.run.smithi113.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-24T23:56:20.833 INFO:teuthology.orchestra.run.smithi113.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-24T23:56:20.884 INFO:teuthology.orchestra.run.smithi113.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 765 kB/s | 90 kB 00:00 2024-04-24T23:56:20.910 INFO:teuthology.orchestra.run.smithi113.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-04-24T23:56:20.934 INFO:teuthology.orchestra.run.smithi113.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-04-24T23:56:20.984 INFO:teuthology.orchestra.run.smithi113.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-24T23:56:21.010 INFO:teuthology.orchestra.run.smithi113.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-04-24T23:56:21.035 INFO:teuthology.orchestra.run.smithi113.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-24T23:56:21.085 INFO:teuthology.orchestra.run.smithi113.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-04-24T23:56:21.127 INFO:teuthology.orchestra.run.smithi113.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-24T23:56:21.161 INFO:teuthology.orchestra.run.smithi113.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-04-24T23:56:21.194 INFO:teuthology.orchestra.run.smithi113.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-24T23:56:21.228 INFO:teuthology.orchestra.run.smithi113.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-24T23:56:21.262 INFO:teuthology.orchestra.run.smithi113.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-24T23:56:21.345 INFO:teuthology.orchestra.run.smithi113.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-24T23:56:21.496 INFO:teuthology.orchestra.run.smithi113.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-04-24T23:56:21.638 INFO:teuthology.orchestra.run.smithi113.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 8.3 MB/s | 3.1 MB 00:00 2024-04-24T23:56:21.713 INFO:teuthology.orchestra.run.smithi113.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 7.0 MB/s | 3.4 MB 00:00 2024-04-24T23:56:21.763 INFO:teuthology.orchestra.run.smithi113.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-04-24T23:56:21.768 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:56:21.769 INFO:teuthology.orchestra.run.smithi113.stdout:Total 13 MB/s | 220 MB 00:17 2024-04-24T23:56:22.880 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-24T23:56:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-24T23:56:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-24T23:56:24.777 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-24T23:56:24.778 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-24T23:56:26.003 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-04-24T23:56:26.127 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:56:26.282 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:56:26.336 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:56:26.370 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:56:26.395 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:56:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:56:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:56:26.888 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:56:27.009 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:56:27.038 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:56:27.114 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:56:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:56:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:56:27.339 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:56:27.367 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:56:27.454 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:56:27.476 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:56:27.539 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:56:27.561 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:56:27.590 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:56:27.786 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:56:27.819 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:56:27.928 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:56:28.021 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:56:28.062 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:56:28.094 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:56:28.158 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:56:28.193 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:56:28.237 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:56:28.269 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:56:28.297 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:56:28.365 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:56:28.419 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:56:28.450 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:56:28.486 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:56:28.561 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:56:28.597 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:56:28.642 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:56:28.668 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:56:28.736 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:56:28.783 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:56:28.856 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:56:29.001 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:56:29.059 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:56:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:56:29.673 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:56:29.701 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:56:29.744 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:56:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:56:29.824 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:56:29.976 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:56:30.043 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:56:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:56:30.161 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:56:30.193 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:56:30.238 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:56:30.285 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:56:30.321 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:56:30.364 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:56:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:56:30.551 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:56:30.603 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:56:30.634 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:56:30.675 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:56:30.680 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-24T23:56:30.766 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:56:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:56:30.800 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:56:30.804 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:56:30.840 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:56:30.858 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:56:30.866 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:56:30.891 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:56:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:56:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:56:30.965 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:56:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:56:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:56:31.123 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:56:31.151 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:56:31.185 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:56:31.224 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:56:31.264 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:56:31.294 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:56:31.307 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:56:31.427 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:56:31.429 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:56:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:56:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:56:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:56:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:56:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:56:31.678 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:56:31.693 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:56:31.715 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:56:31.735 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:56:31.766 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:56:31.813 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:56:31.867 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:56:31.911 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:56:31.921 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:56:31.958 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:56:31.959 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:56:32.060 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:56:32.083 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:56:32.138 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:56:32.159 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:56:32.183 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:56:32.384 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:56:32.426 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:56:32.532 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:56:32.629 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:56:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:56:32.683 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:56:32.718 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:56:32.731 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:56:32.764 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:56:32.784 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:56:32.834 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:56:32.854 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:56:32.886 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:56:32.914 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:56:32.918 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:56:32.946 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:56:33.002 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:56:33.017 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:56:33.033 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:56:33.069 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:56:33.100 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:56:33.143 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:56:33.212 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:56:33.227 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:56:33.239 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:56:33.270 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:56:33.314 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:56:33.317 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:56:33.343 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:56:33.412 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:56:33.474 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:56:33.548 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:56:33.682 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:56:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:56:33.951 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:56:34.071 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:56:34.274 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:56:34.355 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:56:34.384 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:56:34.435 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:56:34.505 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:56:34.557 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:56:34.719 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:56:34.777 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:56:34.831 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:56:34.904 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:56:34.935 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:56:34.972 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:56:35.027 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:56:35.071 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:56:35.114 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:56:35.157 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:56:35.291 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:56:35.299 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:56:35.346 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:56:35.376 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:56:35.393 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:56:35.417 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:56:35.422 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:56:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:56:35.511 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:56:35.550 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:56:35.590 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:56:35.616 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:56:35.641 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:56:35.716 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:56:35.766 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:56:35.809 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:56:35.842 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:56:35.874 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:56:35.899 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:56:35.901 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:56:35.930 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:56:35.935 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:56:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:56:35.983 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:56:36.023 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:56:36.053 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:56:36.087 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:56:36.121 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:56:36.162 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:56:36.197 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:56:36.284 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:56:36.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:56:36.470 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:56:36.470 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:56:36.509 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:56:36.541 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:56:36.565 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:56:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:56:36.663 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:56:36.710 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:56:37.434 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:56:37.482 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:56:37.516 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:56:37.625 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:56:37.682 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:56:37.771 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:56:38.034 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:56:38.329 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:56:38.382 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:56:38.459 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:56:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:56:38.893 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:56:39.098 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:56:39.215 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:56:39.435 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:56:39.471 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:56:39.561 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:56:39.561 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:56:39.562 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:56:39.562 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:39.589 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:56:40.473 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:56:40.562 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:56:40.591 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:56:40.624 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:56:40.935 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:56:40.986 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:56:41.078 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:56:41.108 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:56:41.144 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:56:41.617 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:56:41.648 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:56:41.719 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:56:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:56:44.020 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:56:44.056 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:56:44.598 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:56:44.660 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:56:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:56:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:56:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:56:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-04-24T23:56:50.963 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:51.134 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:56:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:56:51.149 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:51.150 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:56:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:56:52.655 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:56:52.656 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:52.656 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:56:52.656 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:56:52.656 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:56:52.656 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:52.879 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:56:52.902 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:56:52.902 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:52.902 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:56:52.902 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:56:52.903 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:56:52.903 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:53.338 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:56:53.362 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:56:53.362 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:53.362 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:56:53.362 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:56:53.362 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:56:53.362 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:53.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:56:53.525 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:56:53.580 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:56:53.765 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:56:53.814 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:56:53.891 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:56:53.980 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:56:54.128 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:56:54.172 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:56:54.204 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:56:54.240 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:56:54.606 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:56:54.639 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:56:54.670 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:56:54.731 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:56:54.770 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:56:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:56:55.838 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:56:56.294 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-04-24T23:56:56.295 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:56.464 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:56:56.480 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:56:56.480 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:56.480 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:56:56.481 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:56.804 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:56:56.830 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:56:56.932 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:56:57.094 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:56:57.124 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:56:57.147 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:56:57.148 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:57.148 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:56:57.148 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:56:57.148 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:56:57.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:57.191 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:56:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:56:57.301 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:56:57.356 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:56:57.926 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:56:57.951 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:56:57.951 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:57.951 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:56:57.951 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:56:57.952 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:56:57.952 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:58.177 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:56:58.199 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:56:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:56:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:56:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:56:58.199 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:58.254 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:56:58.285 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:56:58.308 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:56:58.308 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:58.308 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:56:58.308 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:56:58.308 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:56:58.308 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:58.345 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:56:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:56:58.367 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:58.367 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:56:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:56:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:56:58.435 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:58.435 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:56:58.435 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:56:58.435 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:56:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:58.636 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:56:58.658 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:56:58.658 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:58.658 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:56:58.658 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:56:58.658 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:56:58.659 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:56:58.738 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:56:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:56:58.761 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:56:58.761 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:56:58.761 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:56:58.761 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:56:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:56:58.774 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:56:58.831 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:56:58.886 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:56:59.073 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:56:59.120 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:56:59.198 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:56:59.287 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:56:59.428 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:56:59.469 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:56:59.502 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:56:59.529 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:56:59.911 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:56:59.928 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:56:59.951 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:57:00.005 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:57:00.034 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:57:01.072 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:57:01.111 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:57:02.076 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:57:02.111 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:57:02.213 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:57:02.384 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:57:02.413 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:57:02.436 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:57:02.436 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:57:02.436 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:57:02.437 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:57:02.437 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:57:02.437 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:02.481 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:57:02.493 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:57:02.589 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:57:02.637 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:57:03.299 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:57:03.340 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:57:03.381 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:57:03.413 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:57:03.449 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:57:03.536 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:57:03.567 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:57:03.590 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:57:03.591 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:57:03.591 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:57:03.591 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:57:03.591 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:57:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:03.631 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:57:03.634 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:57:03.655 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:57:03.656 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:57:03.656 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:57:03.656 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:03.670 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:57:03.700 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:57:03.701 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:57:03.723 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:57:03.724 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:57:03.724 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:57:03.724 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:57:03.724 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:57:03.724 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:57:04.050 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:57:04.050 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:57:04.050 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:57:04.050 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:57:04.050 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:57:04.050 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:04.304 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:57:04.339 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:57:04.359 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:57:05.177 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:57:05.268 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:57:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:57:05.398 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:57:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:57:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:57:05.515 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:57:05.539 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:57:05.540 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:57:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:57:05.687 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:57:08.566 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:57:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:57:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:57:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:57:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:57:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:57:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:57:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:57:09.576 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:57:09.616 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:57:09.636 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:57:10.466 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:57:10.555 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:57:10.596 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:57:10.679 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:57:10.718 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:57:10.718 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:57:10.802 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:57:10.835 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:57:10.835 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:57:10.925 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:57:10.992 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:57:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:57:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:57:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:57:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:57:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:57:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:57:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:57:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:57:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:57:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:57:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:57:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:57:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:57:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:57:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:57:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:57:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:57:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:57:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:57:13.577 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.579 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:57:13.580 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:57:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:57:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:57:13.583 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:57:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:57:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:57:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:57:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:57:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:57:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:57:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:57:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:57:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:13.590 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-04-24T23:57:14.012 DEBUG:teuthology.parallel:result is None 2024-04-24T23:57:17.073 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:57:17.073 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:57:17.074 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:57:17.074 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:57:17.074 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:57:17.074 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:57:17.074 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:57:17.074 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:57:17.074 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:57:17.075 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:57:17.076 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:57:17.076 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:57:17.076 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:57:17.076 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:57:17.076 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:57:17.087 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:57:17.088 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:57:17.089 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:57:17.090 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:57:17.091 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:57:17.092 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:57:17.093 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:57:17.094 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:57:17.094 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:57:17.094 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:57:17.094 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:57:17.095 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:57:17.096 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:57:17.097 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:57:17.098 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:57:17.098 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:57:17.098 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:57:17.098 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:57:17.098 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:57:17.098 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:57:19.134 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:57:19.135 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.136 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:57:19.137 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:57:19.138 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:57:19.139 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:57:19.140 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:57:19.141 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:57:19.142 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:57:19.143 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:57:19.144 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:19.146 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-24T23:57:19.485 DEBUG:teuthology.parallel:result is None 2024-04-24T23:57:19.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:57:19.658 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:57:19.679 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:57:19.679 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:57:19.679 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:57:19.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:57:19.828 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:57:19.853 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:57:19.853 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:57:19.853 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:57:19.855 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:57:19.855 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:19.855 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:57:19.880 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:19.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:57:19.923 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:57:19.923 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:19.924 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:57:19.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:57:20.012 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:20.012 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:57:20.039 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:57:20.106 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:57:20.107 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:20.107 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:57:20.131 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:57:20.195 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:20.195 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:57:20.221 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:57:20.287 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:57:20.287 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:20.287 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:57:20.310 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:57:20.373 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:20.373 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:57:20.398 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:57:20.463 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T23:57:20.544 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T23:57:20.544 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-04-24T23:57:20.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2024-04-24T23:57:20.570 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T23:57:20.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T23:57:20.609 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T23:57:20.634 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T23:57:20.634 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T23:57:20.672 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T23:57:20.704 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-24T23:57:20.705 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-24T23:57:20.705 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-24T23:57:20.705 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:20.705 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T23:57:20.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T23:57:20.850 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T23:57:20.893 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:20.893 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T23:57:20.907 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T23:57:21.015 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T23:57:21.059 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T23:57:21.059 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T23:57:21.061 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/scrub}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672227', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBh1yKy/w+Iuqtvo+V0dp6Y+nFJ9GfXW5D9HFtNVGyeSrmyOoTLyd/wLzSApMmmq5ZKJ2jQpdugIRG7pEO8eeCo=', 'smithi178.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyvInHSGUjooPxRpFHT0j3UX5iibjgkDiFlCFtO9PxVViY67e+DsfT8wviUYK9897jnLhLR3r73t4hUBpsPD7k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T23:57:21.062 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T23:57:21.064 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T23:57:21.108 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T23:57:21.119 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T23:57:21.172 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:21.172 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:57:21.194 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:57:21.195 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 23:57:11.352351637 +0000 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 23:55:20.141689091 +0000 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 23:55:20.141689091 +0000 2024-04-24T23:57:21.249 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 23:55:20.141689091 +0000 2024-04-24T23:57:21.250 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:57:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T23:57:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T23:57:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000218634 s, 2.3 MB/s 2024-04-24T23:57:21.312 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:57:21.368 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-04-24T23:57:21.423 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:57:21.423 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:21.423 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-24T23:57:21.423 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:21.423 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:21.423 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 23:57:11.353351615 +0000 2024-04-24T23:57:21.424 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 23:55:20.507681169 +0000 2024-04-24T23:57:21.424 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 23:55:20.507681169 +0000 2024-04-24T23:57:21.424 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 23:55:20.507681169 +0000 2024-04-24T23:57:21.424 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:57:21.485 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T23:57:21.485 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T23:57:21.485 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000199964 s, 2.6 MB/s 2024-04-24T23:57:21.486 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:57:21.542 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-04-24T23:57:21.597 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:57:21.597 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:21.598 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-24T23:57:21.598 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:21.598 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:21.598 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 23:57:11.353351615 +0000 2024-04-24T23:57:21.598 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 23:55:20.854673659 +0000 2024-04-24T23:57:21.598 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 23:55:20.854673659 +0000 2024-04-24T23:57:21.598 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 23:55:20.854673659 +0000 2024-04-24T23:57:21.598 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:57:21.659 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T23:57:21.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T23:57:21.660 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000198107 s, 2.6 MB/s 2024-04-24T23:57:21.661 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:57:21.716 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 23:57:11.353351615 +0000 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 23:55:21.202666128 +0000 2024-04-24T23:57:21.770 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 23:55:21.202666128 +0000 2024-04-24T23:57:21.771 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 23:55:21.202666128 +0000 2024-04-24T23:57:21.771 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:57:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T23:57:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T23:57:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000272326 s, 1.9 MB/s 2024-04-24T23:57:21.834 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:57:21.889 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:21.890 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:21.890 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:57:21.906 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:57:21.906 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-04-24T23:57:21.962 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:57:21.962 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:21.962 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-24T23:57:21.962 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:21.963 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:21.963 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-24 23:57:06.041255479 +0000 2024-04-24T23:57:21.963 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-24 23:55:20.127412067 +0000 2024-04-24T23:57:21.963 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-24 23:55:20.127412067 +0000 2024-04-24T23:57:21.963 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-24 23:55:20.127412067 +0000 2024-04-24T23:57:21.963 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:57:22.026 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-24T23:57:22.026 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-24T23:57:22.027 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 8.8337e-05 s, 5.8 MB/s 2024-04-24T23:57:22.028 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:57:22.083 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-04-24T23:57:22.138 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-24 23:57:06.041255479 +0000 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-24 23:55:20.456406055 +0000 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-24 23:55:20.456406055 +0000 2024-04-24T23:57:22.139 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-24 23:55:20.456406055 +0000 2024-04-24T23:57:22.142 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:57:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-24T23:57:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-24T23:57:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000107448 s, 4.8 MB/s 2024-04-24T23:57:22.202 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:57:22.258 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-04-24T23:57:22.313 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-24 23:57:06.042255462 +0000 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-24 23:55:20.782400097 +0000 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-24 23:55:20.782400097 +0000 2024-04-24T23:57:22.314 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-24 23:55:20.782400097 +0000 2024-04-24T23:57:22.314 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:57:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-24T23:57:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-24T23:57:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000106688 s, 4.8 MB/s 2024-04-24T23:57:22.377 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:57:22.434 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-24 23:57:06.042255462 +0000 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-24 23:55:21.107394158 +0000 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-24 23:55:21.107394158 +0000 2024-04-24T23:57:22.489 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-24 23:55:21.107394158 +0000 2024-04-24T23:57:22.490 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:57:22.551 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-24T23:57:22.551 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-24T23:57:22.551 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000102265 s, 5.0 MB/s 2024-04-24T23:57:22.552 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:57:22.608 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:22.608 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T23:57:22.608 INFO:tasks.ceph:Generating config... 2024-04-24T23:57:22.609 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-24T23:57:22.610 INFO:tasks.ceph:[client] debug client = 20 2024-04-24T23:57:22.610 INFO:tasks.ceph:[client] debug ms = 1 2024-04-24T23:57:22.610 INFO:tasks.ceph:[client] pid file = 2024-04-24T23:57:22.610 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-24T23:57:22.610 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-24T23:57:22.610 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-24T23:57:22.610 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-24T23:57:22.610 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-24T23:57:22.610 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-24T23:57:22.610 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-24T23:57:22.610 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-24T23:57:22.610 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-24T23:57:22.610 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-24T23:57:22.611 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-24T23:57:22.611 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T23:57:22.612 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-24T23:57:22.613 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T23:57:22.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T23:57:22.672 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T23:57:22.674 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T23:57:22.714 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:57:22.777 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.113'), ('mon.b', '172.21.15.178'), ('mon.c', '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]')] 2024-04-24T23:57:22.777 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-24T23:57:22.778 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:22.778 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T23:57:22.831 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.113 --add b 172.21.15.178 --addv c '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:22.915 INFO:teuthology.orchestra.run.smithi113.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:22.915 INFO:teuthology.orchestra.run.smithi113.stdout:monmaptool: generated fsid c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:setting min_mon_release = quincy 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:epoch 0 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:fsid c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:last_changed 2024-04-24T23:57:22.916905+0000 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:created 2024-04-24T23:57:22.916905+0000 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:min_mon_release 17 (quincy) 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:election_strategy: 1 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-04-24T23:57:22.916 INFO:teuthology.orchestra.run.smithi113.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T23:57:22.918 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T23:57:22.932 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c4eaaf81-6120-4e3f-a50c-97aa81453142... 2024-04-24T23:57:22.933 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T23:57:22.975 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T23:57:23.011 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-24T23:57:23.011 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-24T23:57:23.011 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T23:57:23.011 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-24T23:57:23.011 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.011 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-24T23:57:23.011 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T23:57:23.012 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-24T23:57:23.013 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-24T23:57:23.014 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-24T23:57:23.015 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-24T23:57:23.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T23:57:23.017 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-24T23:57:23.018 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T23:57:23.019 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: mon down mkfs grace = 300 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-04-24T23:57:23.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-24T23:57:23.021 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout: mds cache max size = 1000 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2024-04-24T23:57:23.022 INFO:teuthology.orchestra.run.smithi113.stdout: mds log max segments = 1 2024-04-24T23:57:23.023 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-04-24T23:57:23.023 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2024-04-24T23:57:23.023 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-24T23:57:23.023 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-24T23:57:23.023 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-24T23:57:23.023 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-24T23:57:23.023 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.024 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-24T23:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-24T23:57:23.026 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-24T23:57:23.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-24T23:57:23.028 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T23:57:23.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-24T23:57:23.030 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T23:57:23.031 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-24T23:57:23.032 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-24T23:57:23.033 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: mds cache max size = 1000 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = True 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: mds log max segments = 1 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = True 2024-04-24T23:57:23.034 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-24T23:57:23.035 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-24T23:57:23.035 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-24T23:57:23.036 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T23:57:23.036 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T23:57:23.127 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T23:57:23.128 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:23.128 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T23:57:23.142 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:23.142 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T23:57:23.197 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:57:23.197 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:23.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T23:57:23.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:57:23.270 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:23.271 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:23.325 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:57:23.325 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:23.325 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T23:57:23.325 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T23:57:23.359 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:23.359 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:23.414 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T23:57:23.415 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T23:57:23.415 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T23:57:23.463 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T23:57:23.465 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T23:57:23.525 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T23:57:23.527 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T23:57:23.528 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T23:57:23.575 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T23:57:23.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-24T23:57:23.601 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T23:57:23.688 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T23:57:23.691 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-24T23:57:23.713 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T23:57:23.762 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T23:57:23.764 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-24T23:57:23.788 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T23:57:23.876 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T23:57:23.879 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-24T23:57:23.902 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T23:57:23.902 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:57:23.941 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T23:57:23.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-24T23:57:24.031 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-24T23:57:24.043 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T23:57:24.081 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T23:57:24.093 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-24T23:57:24.172 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-24T23:57:24.184 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T23:57:24.184 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T23:57:24.185 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T23:57:24.208 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:24.208 INFO:tasks.ceph:role: osd.0 2024-04-24T23:57:24.208 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:57:24.208 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:24.279 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:24.280 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:24.280 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:24.818 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T23:57:24.820 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T23:57:24.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T23:57:24.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T23:57:24.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T23:57:25.012 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:25.012 INFO:tasks.ceph:role: osd.1 2024-04-24T23:57:25.013 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:57:25.013 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:25.083 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:25.084 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:25.546 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T23:57:25.548 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T23:57:25.548 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T23:57:25.634 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T23:57:25.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T23:57:25.727 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:25.727 INFO:tasks.ceph:role: osd.2 2024-04-24T23:57:25.727 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:57:25.727 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T23:57:25.799 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:25.799 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:25.799 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:25.800 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:25.800 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:25.800 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:25.800 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:25.800 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:25.800 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:25.800 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:26.313 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T23:57:26.315 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T23:57:26.315 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T23:57:26.427 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T23:57:26.455 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T23:57:26.520 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:26.520 INFO:tasks.ceph:role: osd.3 2024-04-24T23:57:26.521 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:57:26.521 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T23:57:26.591 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:26.591 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:26.591 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:26.591 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:26.591 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:26.592 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:26.592 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:26.592 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:26.592 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:26.592 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:27.118 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T23:57:27.120 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T23:57:27.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T23:57:27.231 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T23:57:27.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:27.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:27.350+0000 7f73eba3d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T23:57:27.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:27.352+0000 7f73eba3d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T23:57:27.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:27.352+0000 7f73eba3d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:57:27.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:27.352+0000 7f73eba3d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:57:27.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:27.352+0000 7f73eba3d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T23:57:27.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:27.357+0000 7f73eba3d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T23:57:30.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T23:57:30.229 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:30.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:30.313+0000 7f4e3f7c4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T23:57:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:30.313+0000 7f4e3f7c4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T23:57:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:30.313+0000 7f4e3f7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:57:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:30.313+0000 7f4e3f7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:57:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:30.313+0000 7f4e3f7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T23:57:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:30.314+0000 7f4e3f7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T23:57:33.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T23:57:33.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:33.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:33.284+0000 7f6011814640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T23:57:33.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:33.285+0000 7f6011814640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T23:57:33.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:33.285+0000 7f6011814640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:57:33.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:33.285+0000 7f6011814640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:57:33.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:33.285+0000 7f6011814640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T23:57:33.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:33.286+0000 7f6011814640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T23:57:36.139 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T23:57:36.167 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:36.255+0000 7f956cddc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T23:57:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:36.256+0000 7f956cddc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T23:57:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:36.256+0000 7f956cddc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:57:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:36.256+0000 7f956cddc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:57:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:36.256+0000 7f956cddc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T23:57:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:36.256+0000 7f956cddc640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T23:57:39.108 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T23:57:39.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T23:57:39.163 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:39.163 INFO:tasks.ceph:role: osd.4 2024-04-24T23:57:39.164 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:57:39.164 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:39.232 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:39.264 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-24T23:57:39.265 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-24T23:57:39.266 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T23:57:39.335 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T23:57:39.404 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T23:57:39.470 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:39.470 INFO:tasks.ceph:role: osd.5 2024-04-24T23:57:39.470 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:57:39.470 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T23:57:39.533 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:39.533 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:39.533 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:39.533 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:39.533 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:39.533 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:39.533 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:39.534 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:39.534 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:39.534 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:39.565 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-24T23:57:39.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-24T23:57:39.567 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T23:57:39.635 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T23:57:39.702 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T23:57:39.767 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:39.767 INFO:tasks.ceph:role: osd.6 2024-04-24T23:57:39.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:57:39.767 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:39.831 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:39.864 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-24T23:57:39.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-24T23:57:39.866 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T23:57:39.935 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T23:57:40.003 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T23:57:40.067 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T23:57:40.067 INFO:tasks.ceph:role: osd.7 2024-04-24T23:57:40.067 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:57:40.067 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T23:57:40.130 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T23:57:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T23:57:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T23:57:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T23:57:40.131 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T23:57:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-24T23:57:40.131 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T23:57:40.131 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T23:57:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T23:57:40.131 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T23:57:40.163 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-24T23:57:40.165 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-24T23:57:40.165 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T23:57:40.234 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T23:57:40.301 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:40.388+0000 7fd718019640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T23:57:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:40.388+0000 7fd718019640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T23:57:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:40.388+0000 7fd718019640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:57:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:40.388+0000 7fd718019640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:57:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:40.388+0000 7fd718019640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T23:57:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:40.389+0000 7fd718019640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T23:57:43.243 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T23:57:43.271 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:43.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:43.360+0000 7efdc8c0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T23:57:43.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:43.360+0000 7efdc8c0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T23:57:43.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:43.360+0000 7efdc8c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:57:43.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:43.360+0000 7efdc8c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:57:43.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:43.360+0000 7efdc8c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T23:57:43.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:43.360+0000 7efdc8c0e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T23:57:46.209 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T23:57:46.237 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:46.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:46.325+0000 7f208e8ee640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T23:57:46.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:46.325+0000 7f208e8ee640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T23:57:46.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:46.325+0000 7f208e8ee640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:57:46.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:46.325+0000 7f208e8ee640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:57:46.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:46.325+0000 7f208e8ee640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T23:57:46.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:46.326+0000 7f208e8ee640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T23:57:49.173 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T23:57:49.201 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:49.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:49.289+0000 7fa946ba1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T23:57:49.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:49.289+0000 7fa946ba1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T23:57:49.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:49.289+0000 7fa946ba1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:57:49.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:49.289+0000 7fa946ba1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:57:49.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:49.289+0000 7fa946ba1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T23:57:49.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:49.289+0000 7fa946ba1640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T23:57:51.063 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:57:51.066 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:57:52.137 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T23:57:52.164 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T23:57:52.165 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:52.165 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T23:57:52.193 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:52.193 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-24T23:57:52.259 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:52.260 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-24T23:57:52.327 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:52.327 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T23:57:52.392 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:52.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T23:57:52.456 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:52.456 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T23:57:52.520 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:52.520 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T23:57:52.584 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:52.584 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T23:57:52.614 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:52.614 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-24T23:57:52.682 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:52.682 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-24T23:57:52.748 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:52.748 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T23:57:52.814 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:52.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T23:57:52.879 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:52.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T23:57:52.944 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:52.944 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T23:57:53.008 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:53.009 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T23:57:53.025 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:53.025 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-24T23:57:53.080 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:53.081 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T23:57:53.095 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:53.095 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-24T23:57:53.150 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T23:57:53.150 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T23:57:53.152 DEBUG:teuthology.orchestra.run.smithi178:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA== 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw== 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg== 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw== 2024-04-24T23:57:53.194 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA== 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A== 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg== 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA== 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw== 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q== 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB0nClmw983FxAAp7st675AydbXKmej07HmSA== 2024-04-24T23:57:53.195 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w== 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA== 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g== 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout:[client.2] 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog== 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw== 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout:[client.3] 2024-04-24T23:57:53.196 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ== 2024-04-24T23:57:53.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.y] 2024-04-24T23:57:53.212 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA== 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw== 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg== 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.0] 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw== 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.1] 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA== 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.2] 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A== 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.3] 2024-04-24T23:57:53.213 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg== 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.x] 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA== 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw== 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q== 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.4] 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB0nClmw983FxAAp7st675AydbXKmej07HmSA== 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.5] 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w== 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.6] 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA== 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.7] 2024-04-24T23:57:53.214 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g== 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:[client.2] 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog== 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:[client.1] 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw== 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:[client.3] 2024-04-24T23:57:53.215 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ== 2024-04-24T23:57:53.216 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:57:53.240 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:57:53.305 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.307 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.358 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.360 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.412 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.414 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.462 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.464 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.512 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.514 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.562 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.564 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:57:53.615 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T23:57:53.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.665 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.719 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T23:57:53.771 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.773 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.821 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.823 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.871 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.874 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.924 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T23:57:53.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:53.975 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:54.026 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:54.028 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:54.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:54.081 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:54.133 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:54.135 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T23:57:54.186 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T23:57:54.187 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T23:57:54.210 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:57:54.305 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T23:57:54.328 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T23:57:54.351 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:57:54.444 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T23:57:54.467 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T23:57:54.531 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T23:57:54.624 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T23:57:54.647 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:54.649 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T23:57:54.702 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T23:57:54.702 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T23:57:54.702 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T23:57:54.704 INFO:tasks.ceph.mon.a:Started 2024-04-24T23:57:54.704 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T23:57:54.704 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T23:57:54.744 INFO:tasks.ceph.mon.b:Started 2024-04-24T23:57:54.744 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T23:57:54.744 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T23:57:54.746 INFO:tasks.ceph.mon.c:Started 2024-04-24T23:57:54.746 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T23:57:54.746 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T23:57:54.747 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T23:57:54.749 INFO:tasks.ceph.mgr.y:Started 2024-04-24T23:57:54.749 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T23:57:54.749 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T23:57:54.751 INFO:tasks.ceph.mgr.x:Started 2024-04-24T23:57:54.751 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T23:57:54.751 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T23:57:54.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.848+0000 7f7b4b24d640 1 Processor -- start 2024-04-24T23:57:54.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.848+0000 7f7b4b24d640 1 -- start start 2024-04-24T23:57:54.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b4b24d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 0x7f7b440c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:54.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b4b24d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b441416a0 con 0x7f7b441440d0 2024-04-24T23:57:54.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b4b24d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b440708d0 con 0x7f7b441412c0 2024-04-24T23:57:54.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b4b24d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7b440c7300 con 0x7f7b4415f400 2024-04-24T23:57:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b497ea640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 0x7f7b440c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b43fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7b441440d0 0x7f7b4406f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46138/0 (socket says 172.21.15.113:46138) 2024-04-24T23:57:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b43fff640 1 -- 172.21.15.113:0/589382753 learned_addr learned my addr 172.21.15.113:0/589382753 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:57:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b497ea640 1 -- 172.21.15.113:0/589382753 >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 msgr2=0x7f7b440c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T23:57:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b497ea640 1 -- 172.21.15.113:0/589382753 >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 msgr2=0x7f7b440c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T23:57:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b497ea640 1 --2- 172.21.15.113:0/589382753 >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 0x7f7b440c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:57:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:54.849+0000 7f7b497ea640 1 --2- 172.21.15.113:0/589382753 >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 0x7f7b440c6dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:57:55.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.003+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4050440079 0 0) 0x7f7b38001110 con 0x7f7b441440d0 2024-04-24T23:57:55.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.003+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b2c003520 con 0x7f7b441440d0 2024-04-24T23:57:55.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.003+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168659209 0 0) 0x7f7b440708d0 con 0x7f7b441412c0 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.003+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b38001110 con 0x7f7b441412c0 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 300464973 0 0) 0x7f7b38001110 con 0x7f7b441412c0 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 msgr2=0x7f7b440c6dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 --2- 172.21.15.113:0/589382753 >> v2:172.21.15.113:3300/0 conn(0x7f7b4415f400 0x7f7b440c6dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 >> v1:172.21.15.113:6789/0 conn(0x7f7b441440d0 legacy=0x7f7b4406f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b44070bb0 con 0x7f7b441412c0 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f7b34001670 con 0x7f7b441412c0 2024-04-24T23:57:55.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 >> v1:172.21.15.178:6789/0 conn(0x7f7b441412c0 legacy=0x7f7b441439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b2c003b40 0x7f7b2c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c004800 0x7f7b2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c0051c0 0x7f7b2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b440c7300 con 0x7f7b2c003b40 2024-04-24T23:57:55.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c00c0f0 con 0x7f7b2c004800 2024-04-24T23:57:55.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2c00c2f0 con 0x7f7b2c0051c0 2024-04-24T23:57:55.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b48fe9640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b2c003b40 0x7f7b2c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b43fff640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c004800 0x7f7b2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b497ea640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c0051c0 0x7f7b2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b48fe9640 1 -- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c0051c0 msgr2=0x7f7b2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:55.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b48fe9640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c0051c0 0x7f7b2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b48fe9640 1 -- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c004800 msgr2=0x7f7b2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:55.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b48fe9640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c004800 0x7f7b2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b48fe9640 1 -- 172.21.15.113:0/589382753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b38001110 con 0x7f7b2c003b40 2024-04-24T23:57:55.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.004+0000 7f7b497ea640 1 -- 172.21.15.113:0/589382753 reap_dead start 2024-04-24T23:57:55.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.005+0000 7f7b48fe9640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b2c003b40 0x7f7b2c003f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b34007f50 tx=0x7f7b2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360829b24bb31fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.006+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34036070 con 0x7f7b2c003b40 2024-04-24T23:57:55.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.006+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34031620 con 0x7f7b2c003b40 2024-04-24T23:57:55.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.006+0000 7f7b437fe640 1 -- 172.21.15.113:0/589382753 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34007bb0 con 0x7f7b2c003b40 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.007+0000 7f7b4b24d640 1 -- 172.21.15.113:0/589382753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b2c003b40 msgr2=0x7f7b2c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.007+0000 7f7b4b24d640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b2c003b40 0x7f7b2c003f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b34007f50 tx=0x7f7b2c005830 comp rx=0 tx=0).stop 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.007+0000 7f7b4b24d640 1 -- 172.21.15.113:0/589382753 shutdown_connections 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.007+0000 7f7b4b24d640 1 --2- 172.21.15.113:0/589382753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b2c003b40 0x7f7b2c003f40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.007+0000 7f7b4b24d640 1 -- 172.21.15.113:0/589382753 >> 172.21.15.113:0/589382753 conn(0x7f7b440d1f20 msgr2=0x7f7b440d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.007+0000 7f7b4b24d640 1 -- 172.21.15.113:0/589382753 shutdown_connections 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.007+0000 7f7b4b24d640 1 -- 172.21.15.113:0/589382753 wait complete. 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.008+0000 7f7b4b24d640 1 Processor -- start 2024-04-24T23:57:55.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.008+0000 7f7b4b24d640 1 -- start start 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b4b24d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c003b40 0x7f7b44146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b4b24d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b44140790 0x7f7b4415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b4b24d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b4415b4b0 0x7f7b44146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b4b24d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b440c7540 con 0x7f7b44140790 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b4b24d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b44071170 con 0x7f7b4415b4b0 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b4b24d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b44071690 con 0x7f7b2c003b40 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b44140790 0x7f7b4415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b44140790 0x7f7b4415af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45852/0 (socket says 172.21.15.113:45852) 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 -- 172.21.15.113:0/951611764 learned_addr learned my addr 172.21.15.113:0/951611764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b497ea640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b4415b4b0 0x7f7b44146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b48fe9640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c003b40 0x7f7b44146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 -- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c003b40 msgr2=0x7f7b44146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c003b40 0x7f7b44146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 -- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b4415b4b0 msgr2=0x7f7b44146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b4415b4b0 0x7f7b44146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.009+0000 7f7b43fff640 1 -- 172.21.15.113:0/951611764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b441454e0 con 0x7f7b44140790 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b43fff640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b44140790 0x7f7b4415af70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b38007340 tx=0x7f7b38007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b417fa640 1 -- 172.21.15.113:0/951611764 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800f9c0 con 0x7f7b44140790 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b417fa640 1 -- 172.21.15.113:0/951611764 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b38018070 con 0x7f7b44140790 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b44145780 con 0x7f7b44140790 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b417fa640 1 -- 172.21.15.113:0/951611764 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800fda0 con 0x7f7b44140790 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b44145d40 con 0x7f7b44140790 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b417fa640 1 -- 172.21.15.113:0/951611764 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b3801d460 con 0x7f7b44140790 2024-04-24T23:57:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.010+0000 7f7b417fa640 1 -- 172.21.15.113:0/951611764 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b38013790 con 0x7f7b44140790 2024-04-24T23:57:55.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.011+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b44068220 con 0x7f7b44140790 2024-04-24T23:57:55.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.013+0000 7f7b417fa640 1 -- 172.21.15.113:0/951611764 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7b441416a0 con 0x7f7b44140790 2024-04-24T23:57:55.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.079+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7b440d1d00 con 0x7f7b44140790 2024-04-24T23:57:55.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.080+0000 7f7b417fa640 1 -- 172.21.15.113:0/951611764 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f7b380139e0 con 0x7f7b44140790 2024-04-24T23:57:55.080 INFO:teuthology.orchestra.run.smithi113.stderr:adjusted tunables profile to default 2024-04-24T23:57:55.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b44140790 msgr2=0x7f7b4415af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b44140790 0x7f7b4415af70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b38007340 tx=0x7f7b38007800 comp rx=0 tx=0).stop 2024-04-24T23:57:55.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 shutdown_connections 2024-04-24T23:57:55.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b4415b4b0 0x7f7b44146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b44140790 0x7f7b4415af70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 --2- 172.21.15.113:0/951611764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b2c003b40 0x7f7b44146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 >> 172.21.15.113:0/951611764 conn(0x7f7b440d1f20 msgr2=0x7f7b441582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:55.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 shutdown_connections 2024-04-24T23:57:55.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:55.081+0000 7f7b4b24d640 1 -- 172.21.15.113:0/951611764 wait complete. 2024-04-24T23:57:55.092 INFO:tasks.ceph:check_enable_crimson: False 2024-04-24T23:57:55.092 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T23:57:55.093 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:55.093 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T23:57:55.110 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:55.110+0000 7f08b4320040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:57:55.116 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:55.116 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T23:57:55.130 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:55.130+0000 7f037dd23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:57:55.153 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:55.153 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T23:57:55.217 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:57:55.217 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T23:57:55.254 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:55.254+0000 7f08b4320040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:57:55.279 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:55.280 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T23:57:55.303 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:55.303 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T23:57:55.308 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:55.309+0000 7f08b4320040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:57:55.314 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:55.314+0000 7f037dd23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:57:55.331 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:55.331 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T23:57:55.367 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:55.368+0000 7f037dd23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:57:55.395 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:57:55.395 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T23:57:55.458 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 69805b28-aeb7-471f-b7ac-7181bae022b0 0 2024-04-24T23:57:55.564 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:55.564+0000 7f08b4320040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:57:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.596+0000 7fd3dd509640 1 Processor -- start 2024-04-24T23:57:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.596+0000 7fd3dd509640 1 -- start start 2024-04-24T23:57:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3dd509640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3d815c140 0x7fd3d80b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3dd509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3dd509640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d8152390 con 0x7fd3d8151fb0 2024-04-24T23:57:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3dd509640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d80c5830 con 0x7fd3d815c140 2024-04-24T23:57:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3dd509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d80c5ad0 con 0x7fd3d80716e0 2024-04-24T23:57:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3d67fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3d815c140 0x7fd3d80b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3d77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3d6ffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd3d8151fb0 0x7fd3d815f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:38804/0 (socket says 172.21.15.178:38804) 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.597+0000 7fd3d6ffd640 1 -- 172.21.15.178:0/3303166117 learned_addr learned my addr 172.21.15.178:0/3303166117 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d67fc640 1 -- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 msgr2=0x7fd3d8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d67fc640 1 --2- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d8071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d67fc640 1 -- 172.21.15.178:0/3303166117 >> v1:172.21.15.178:6789/0 conn(0x7fd3d8151fb0 legacy=0x7fd3d815f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d5ffb640 1 -- 172.21.15.178:0/3303166117 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283065124 0 0) 0x7fd3d8152390 con 0x7fd3d8151fb0 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d67fc640 1 -- 172.21.15.178:0/3303166117 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d815ec90 con 0x7fd3d815c140 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d77fe640 1 --2- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d67fc640 1 --2- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d815c140 0x7fd3d80b2d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80097a0 tx=0x7fd3c802c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54d8dbc17f4f381 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d5ffb640 1 -- 172.21.15.178:0/3303166117 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8039070 con 0x7fd3d815c140 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.598+0000 7fd3d5ffb640 1 -- 172.21.15.178:0/3303166117 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c8034640 con 0x7fd3d815c140 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3d5ffb640 1 -- 172.21.15.178:0/3303166117 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c802ce60 con 0x7fd3d815c140 2024-04-24T23:57:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 -- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d815c140 msgr2=0x7fd3d80b2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 --2- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d815c140 0x7fd3d80b2d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80097a0 tx=0x7fd3c802c310 comp rx=0 tx=0).stop 2024-04-24T23:57:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 -- 172.21.15.178:0/3303166117 shutdown_connections 2024-04-24T23:57:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 --2- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 --2- 172.21.15.178:0/3303166117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d815c140 0x7fd3d80b2d00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 -- 172.21.15.178:0/3303166117 >> 172.21.15.178:0/3303166117 conn(0x7fd3d8061f70 msgr2=0x7fd3d80b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 -- 172.21.15.178:0/3303166117 shutdown_connections 2024-04-24T23:57:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.599+0000 7fd3dd509640 1 -- 172.21.15.178:0/3303166117 wait complete. 2024-04-24T23:57:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.600+0000 7fd3dd509640 1 Processor -- start 2024-04-24T23:57:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3dd509640 1 -- start start 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3dd509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d81f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3dd509640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d8151fb0 0x7fd3d80ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3dd509640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d815c140 0x7fd3d81f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3dd509640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d806d7b0 con 0x7fd3d815c140 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3dd509640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d80b3240 con 0x7fd3d8151fb0 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3dd509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d806d930 con 0x7fd3d80716e0 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3d67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d8151fb0 0x7fd3d80ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3d6ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d81f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3d67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d8151fb0 0x7fd3d80ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39850/0 (socket says 172.21.15.178:39850) 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3d67fc640 1 -- 172.21.15.178:0/3791214854 learned_addr learned my addr 172.21.15.178:0/3791214854 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.601+0000 7fd3d77fe640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d815c140 0x7fd3d81f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3d67fc640 1 -- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 msgr2=0x7fd3d81f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3d67fc640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d81f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3d67fc640 1 -- 172.21.15.178:0/3791214854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d815c140 msgr2=0x7fd3d81f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3d67fc640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d815c140 0x7fd3d81f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3d67fc640 1 -- 172.21.15.178:0/3791214854 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d81f7be0 con 0x7fd3d8151fb0 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3d6ffd640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d81f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3d67fc640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d8151fb0 0x7fd3d80ca2b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80098c0 tx=0x7fd3c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3bf7fe640 1 -- 172.21.15.178:0/3791214854 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8044070 con 0x7fd3d8151fb0 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3bf7fe640 1 -- 172.21.15.178:0/3791214854 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c8032980 con 0x7fd3d8151fb0 2024-04-24T23:57:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3bf7fe640 1 -- 172.21.15.178:0/3791214854 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8039070 con 0x7fd3d8151fb0 2024-04-24T23:57:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d81f3b40 con 0x7fd3d8151fb0 2024-04-24T23:57:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d81f7ec0 con 0x7fd3d8151fb0 2024-04-24T23:57:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3bf7fe640 1 -- 172.21.15.178:0/3791214854 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd3c8047070 con 0x7fd3d8151fb0 2024-04-24T23:57:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.602+0000 7fd3bf7fe640 1 -- 172.21.15.178:0/3791214854 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fd3c8049440 con 0x7fd3d8151fb0 2024-04-24T23:57:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.603+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d8068220 con 0x7fd3d8151fb0 2024-04-24T23:57:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.605+0000 7fd3bf7fe640 1 -- 172.21.15.178:0/3791214854 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd3c8032d50 con 0x7fd3d8151fb0 2024-04-24T23:57:55.612 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:55.613+0000 7f08b4320040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:57:55.617 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:55.618+0000 7f037dd23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:57:55.662 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:55.662+0000 7f037dd23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:57:55.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.676+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "69805b28-aeb7-471f-b7ac-7181bae022b0", "id": 0} v 0) -- 0x7fd3d80cfb90 con 0x7fd3d8151fb0 2024-04-24T23:57:55.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.678+0000 7fd3bf7fe640 1 -- 172.21.15.178:0/3791214854 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69805b28-aeb7-471f-b7ac-7181bae022b0", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fd3c803f460 con 0x7fd3d8151fb0 2024-04-24T23:57:55.678 INFO:teuthology.orchestra.run.smithi178.stdout:0 2024-04-24T23:57:55.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.679+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d8151fb0 msgr2=0x7fd3d80ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.679+0000 7fd3dd509640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d8151fb0 0x7fd3d80ca2b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80098c0 tx=0x7fd3c80046c0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.679+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 shutdown_connections 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.679+0000 7fd3dd509640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3d8151fb0 0x7fd3d80ca2b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.679+0000 7fd3dd509640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d815c140 0x7fd3d81f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.679+0000 7fd3dd509640 1 --2- 172.21.15.178:0/3791214854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3d80716e0 0x7fd3d81f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.679+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 >> 172.21.15.178:0/3791214854 conn(0x7fd3d8061f70 msgr2=0x7fd3d80b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.680+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 shutdown_connections 2024-04-24T23:57:55.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.680+0000 7fd3dd509640 1 -- 172.21.15.178:0/3791214854 wait complete. 2024-04-24T23:57:55.690 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 57db595e-9575-4f51-b81c-078f498b956a 1 2024-04-24T23:57:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.787+0000 7ff4d20ee640 1 Processor -- start 2024-04-24T23:57:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.788+0000 7ff4d20ee640 1 -- start start 2024-04-24T23:57:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.788+0000 7ff4d20ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4cc162720 0x7ff4cc166cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.788+0000 7ff4d20ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc06c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.788+0000 7ff4d20ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4cc061b80 0x7ff4cc062010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.788+0000 7ff4d20ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc06caa0 con 0x7ff4cc162720 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.788+0000 7ff4d20ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc06dfc0 con 0x7ff4cc061b80 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.788+0000 7ff4d20ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc167210 con 0x7ff4cc06c100 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4cc061b80 0x7ff4cc062010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4cc061b80 0x7ff4cc062010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39864/0 (socket says 172.21.15.178:39864) 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 -- 172.21.15.178:0/4238172201 learned_addr learned my addr 172.21.15.178:0/4238172201 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cb7fe640 1 --2- 172.21.15.178:0/4238172201 >> v2:172.21.15.113:3300/0 conn(0x7ff4cc162720 0x7ff4cc166cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 -- 172.21.15.178:0/4238172201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 msgr2=0x7ff4cc06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 --2- 172.21.15.178:0/4238172201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc06c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 -- 172.21.15.178:0/4238172201 >> v2:172.21.15.113:3300/0 conn(0x7ff4cc162720 msgr2=0x7ff4cc166cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 --2- 172.21.15.178:0/4238172201 >> v2:172.21.15.113:3300/0 conn(0x7ff4cc162720 0x7ff4cc166cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 -- 172.21.15.178:0/4238172201 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc162b00 con 0x7ff4cc061b80 2024-04-24T23:57:55.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.789+0000 7ff4cbfff640 1 --2- 172.21.15.178:0/4238172201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc061b80 0x7ff4cc062010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09f0b0 tx=0x7ff4c402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5e5889e6949297 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4caffd640 1 -- 172.21.15.178:0/4238172201 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4036070 con 0x7ff4cc061b80 2024-04-24T23:57:55.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4caffd640 1 -- 172.21.15.178:0/4238172201 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c4031600 con 0x7ff4cc061b80 2024-04-24T23:57:55.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4caffd640 1 -- 172.21.15.178:0/4238172201 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40319d0 con 0x7ff4cc061b80 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 -- 172.21.15.178:0/4238172201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc061b80 msgr2=0x7ff4cc062010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/4238172201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc061b80 0x7ff4cc062010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09f0b0 tx=0x7ff4c402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 -- 172.21.15.178:0/4238172201 shutdown_connections 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/4238172201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc061b80 0x7ff4cc062010 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/4238172201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc06c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/4238172201 >> v2:172.21.15.113:3300/0 conn(0x7ff4cc162720 0x7ff4cc166cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 -- 172.21.15.178:0/4238172201 >> 172.21.15.178:0/4238172201 conn(0x7ff4cc071ec0 msgr2=0x7ff4cc0722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:55.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 -- 172.21.15.178:0/4238172201 shutdown_connections 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.790+0000 7ff4d20ee640 1 -- 172.21.15.178:0/4238172201 wait complete. 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.791+0000 7ff4d20ee640 1 Processor -- start 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4d20ee640 1 -- start start 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4d20ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc061b80 0x7ff4cc163d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4d20ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4d20ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc162720 0x7ff4cc061f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4d20ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b5fb0 con 0x7ff4cc061b80 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4d20ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b4420 con 0x7ff4cc162720 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4d20ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b4700 con 0x7ff4cc06c100 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc162720 0x7ff4cc061f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37680/0 (socket says 172.21.15.178:37680) 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4c3fff640 1 -- 172.21.15.178:0/2884893379 learned_addr learned my addr 172.21.15.178:0/2884893379 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.792+0000 7ff4cb7fe640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc061b80 0x7ff4cc163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4cbfff640 1 -- 172.21.15.178:0/2884893379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 msgr2=0x7ff4cc150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4cbfff640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4cbfff640 1 -- 172.21.15.178:0/2884893379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc061b80 msgr2=0x7ff4cc163d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4cbfff640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc061b80 0x7ff4cc163d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4cbfff640 1 -- 172.21.15.178:0/2884893379 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc164260 con 0x7ff4cc162720 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4cbfff640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc162720 0x7ff4cc061f60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4030000 tx=0x7ff4c4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4c8ff9640 1 -- 172.21.15.178:0/2884893379 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4036050 con 0x7ff4cc162720 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4c8ff9640 1 -- 172.21.15.178:0/2884893379 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c4042070 con 0x7ff4cc162720 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4c8ff9640 1 -- 172.21.15.178:0/2884893379 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4009510 con 0x7ff4cc162720 2024-04-24T23:57:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc164500 con 0x7ff4cc162720 2024-04-24T23:57:55.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc164ac0 con 0x7ff4cc162720 2024-04-24T23:57:55.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4c8ff9640 1 -- 172.21.15.178:0/2884893379 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4c4007bf0 con 0x7ff4cc162720 2024-04-24T23:57:55.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.793+0000 7ff4c8ff9640 1 -- 172.21.15.178:0/2884893379 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7ff4c403d450 con 0x7ff4cc162720 2024-04-24T23:57:55.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.794+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc167030 con 0x7ff4cc162720 2024-04-24T23:57:55.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.796+0000 7ff4c8ff9640 1 -- 172.21.15.178:0/2884893379 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff4cc167030 con 0x7ff4cc162720 2024-04-24T23:57:55.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.869+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "57db595e-9575-4f51-b81c-078f498b956a", "id": 1} v 0) -- 0x7ff4cc068220 con 0x7ff4cc162720 2024-04-24T23:57:55.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.872+0000 7ff4c8ff9640 1 -- 172.21.15.178:0/2884893379 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "57db595e-9575-4f51-b81c-078f498b956a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7ff4c4047460 con 0x7ff4cc162720 2024-04-24T23:57:55.872 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-04-24T23:57:55.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc162720 msgr2=0x7ff4cc061f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc162720 0x7ff4cc061f60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4030000 tx=0x7ff4c4004a40 comp rx=0 tx=0).stop 2024-04-24T23:57:55.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 shutdown_connections 2024-04-24T23:57:55.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc162720 0x7ff4cc061f60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc06c100 0x7ff4cc150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 --2- 172.21.15.178:0/2884893379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc061b80 0x7ff4cc163d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 >> 172.21.15.178:0/2884893379 conn(0x7ff4cc071ec0 msgr2=0x7ff4cc0b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:55.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 shutdown_connections 2024-04-24T23:57:55.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.873+0000 7ff4d20ee640 1 -- 172.21.15.178:0/2884893379 wait complete. 2024-04-24T23:57:55.884 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535 2 2024-04-24T23:57:55.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.982+0000 7f14c4845640 1 Processor -- start 2024-04-24T23:57:55.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.982+0000 7f14c4845640 1 -- start start 2024-04-24T23:57:55.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14c4845640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f14c015e510 0x7f14c00d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14c4845640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c00b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14c4845640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14c00b61e0 0x7f14c0071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14c4845640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f14c00d2e00 con 0x7f14c00b61e0 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14c4845640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f14c00d30e0 con 0x7f14c015e510 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14c4845640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f14c00d33c0 con 0x7f14c00d28c0 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14bdd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c00b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14be59c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f14c015e510 0x7f14c00d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14bdd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c00b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37690/0 (socket says 172.21.15.178:37690) 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14bdd9b640 1 -- 172.21.15.178:0/82891206 learned_addr learned my addr 172.21.15.178:0/82891206 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14be59c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f14c015e510 0x7f14c00d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39884/0 (socket says 172.21.15.178:39884) 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14bed9d640 1 --2- 172.21.15.178:0/82891206 >> v2:172.21.15.113:3300/0 conn(0x7f14c00b61e0 0x7f14c0071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14be59c640 1 -- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 msgr2=0x7f14c00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14be59c640 1 --2- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c00b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14be59c640 1 -- 172.21.15.178:0/82891206 >> v2:172.21.15.113:3300/0 conn(0x7f14c00b61e0 msgr2=0x7f14c0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14be59c640 1 --2- 172.21.15.178:0/82891206 >> v2:172.21.15.113:3300/0 conn(0x7f14c00b61e0 0x7f14c0071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14be59c640 1 -- 172.21.15.178:0/82891206 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c00d36a0 con 0x7f14c015e510 2024-04-24T23:57:55.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.983+0000 7f14bdd9b640 1 --2- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:55.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.984+0000 7f14be59c640 1 --2- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c015e510 0x7f14c00d2380 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14c00d52a0 tx=0x7f14b002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a4430147c79448 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.984+0000 7f14bd59a640 1 -- 172.21.15.178:0/82891206 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0037070 con 0x7f14c015e510 2024-04-24T23:57:55.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.984+0000 7f14bd59a640 1 -- 172.21.15.178:0/82891206 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b002add0 con 0x7f14c015e510 2024-04-24T23:57:55.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.984+0000 7f14bd59a640 1 -- 172.21.15.178:0/82891206 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0032660 con 0x7f14c015e510 2024-04-24T23:57:55.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 -- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c015e510 msgr2=0x7f14c00d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 --2- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c015e510 0x7f14c00d2380 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14c00d52a0 tx=0x7f14b002a080 comp rx=0 tx=0).stop 2024-04-24T23:57:55.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 -- 172.21.15.178:0/82891206 shutdown_connections 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 --2- 172.21.15.178:0/82891206 >> v2:172.21.15.113:3300/0 conn(0x7f14c00b61e0 0x7f14c0071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 --2- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 --2- 172.21.15.178:0/82891206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c015e510 0x7f14c00d2380 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 -- 172.21.15.178:0/82891206 >> 172.21.15.178:0/82891206 conn(0x7f14c006b720 msgr2=0x7f14c00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 -- 172.21.15.178:0/82891206 shutdown_connections 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.985+0000 7f14c4845640 1 -- 172.21.15.178:0/82891206 wait complete. 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.986+0000 7f14c4845640 1 Processor -- start 2024-04-24T23:57:55.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.986+0000 7f14c4845640 1 -- start start 2024-04-24T23:57:55.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14c4845640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c00b61e0 0x7f14c00b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14c4845640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14c4845640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14c015e510 0x7f14c00ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:55.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14c4845640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00b9dd0 con 0x7f14c015e510 2024-04-24T23:57:55.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14c4845640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00d3ea0 con 0x7f14c00b61e0 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14c4845640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f14c0071c50 con 0x7f14c00d28c0 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14be59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c00b61e0 0x7f14c00b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37704/0 (socket says 172.21.15.178:37704) 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 -- 172.21.15.178:0/3422864917 learned_addr learned my addr 172.21.15.178:0/3422864917 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bed9d640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14c015e510 0x7f14c00ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 -- 172.21.15.178:0/3422864917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c00b61e0 msgr2=0x7f14c00b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c00b61e0 0x7f14c00b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 -- 172.21.15.178:0/3422864917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14c015e510 msgr2=0x7f14c00ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14c015e510 0x7f14c00ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 -- 172.21.15.178:0/3422864917 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c00ce370 con 0x7f14c00d28c0 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.987+0000 7f14bdd9b640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c015bee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f14b400e9d0 tx=0x7f14b400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:55.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.988+0000 7f14aeffd640 1 -- 172.21.15.178:0/3422864917 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4017070 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.988+0000 7f14aeffd640 1 -- 172.21.15.178:0/3422864917 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b400ca60 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.988+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c00cd2d0 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.988+0000 7f14aeffd640 1 -- 172.21.15.178:0/3422864917 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b400ce10 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.988+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c00cd890 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.988+0000 7f14aeffd640 1 -- 172.21.15.178:0/3422864917 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f14b401a070 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.988+0000 7f14aeffd640 1 -- 172.21.15.178:0/3422864917 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f14b401c870 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.989+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c00691a0 con 0x7f14c00d28c0 2024-04-24T23:57:55.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:55.991+0000 7f14aeffd640 1 -- 172.21.15.178:0/3422864917 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f14b401c450 con 0x7f14c00d28c0 2024-04-24T23:57:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.093+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535", "id": 2} v 0) -- 0x7f14c00b9620 con 0x7f14c00d28c0 2024-04-24T23:57:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.096+0000 7f14aeffd640 1 -- 172.21.15.178:0/3422864917 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f14b401cb90 con 0x7f14c00d28c0 2024-04-24T23:57:56.096 INFO:teuthology.orchestra.run.smithi178.stdout:2 2024-04-24T23:57:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 msgr2=0x7f14c015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c015bee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f14b400e9d0 tx=0x7f14b400ee90 comp rx=0 tx=0).stop 2024-04-24T23:57:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 shutdown_connections 2024-04-24T23:57:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14c015e510 0x7f14c00ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14c00d28c0 0x7f14c015bee0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 --2- 172.21.15.178:0/3422864917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c00b61e0 0x7f14c00b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 >> 172.21.15.178:0/3422864917 conn(0x7f14c006b720 msgr2=0x7f14c00b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 shutdown_connections 2024-04-24T23:57:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.097+0000 7f14c4845640 1 -- 172.21.15.178:0/3422864917 wait complete. 2024-04-24T23:57:56.108 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new a878c827-2212-4027-84a3-c426552c4fce 3 2024-04-24T23:57:56.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.206+0000 7f8fba8ee640 1 Processor -- start 2024-04-24T23:57:56.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.206+0000 7f8fba8ee640 1 -- start start 2024-04-24T23:57:56.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fba8ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb4141d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fba8ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb415cd30 con 0x7f8fb40d1a10 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fba8ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb415d5a0 con 0x7f8fb413f2a0 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fba8ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb4142250 con 0x7f8fb41418a0 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb8e8b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb4141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb3fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8fb413f2a0 0x7f8fb40710b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:38810/0 (socket says 172.21.15.178:38810) 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb3fff640 1 -- 172.21.15.178:0/4183868708 learned_addr learned my addr 172.21.15.178:0/4183868708 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb2ffd640 1 -- 172.21.15.178:0/4183868708 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972848721 0 0) 0x7f8fb415d5a0 con 0x7f8fb413f2a0 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb2ffd640 1 -- 172.21.15.178:0/4183868708 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f9c003520 con 0x7f8fb413f2a0 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb8e8b640 1 -- 172.21.15.178:0/4183868708 >> v1:172.21.15.178:6789/0 conn(0x7f8fb413f2a0 legacy=0x7f8fb40710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb8e8b640 1 -- 172.21.15.178:0/4183868708 >> v1:172.21.15.113:6789/0 conn(0x7f8fb40d1a10 legacy=0x7f8fb415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.207+0000 7f8fb8e8b640 1 -- 172.21.15.178:0/4183868708 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb415cfe0 con 0x7f8fb41418a0 2024-04-24T23:57:56.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fb8e8b640 1 --2- 172.21.15.178:0/4183868708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb4141d10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009dc0 tx=0x7f8fac02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732e6b2e0c4543b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fb2ffd640 1 -- 172.21.15.178:0/4183868708 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac02ae60 con 0x7f8fb41418a0 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fb2ffd640 1 -- 172.21.15.178:0/4183868708 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac007740 con 0x7f8fb41418a0 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fb2ffd640 1 -- 172.21.15.178:0/4183868708 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac007b10 con 0x7f8fb41418a0 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4183868708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 msgr2=0x7f8fb4141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fba8ee640 1 --2- 172.21.15.178:0/4183868708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb4141d10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009dc0 tx=0x7f8fac02a640 comp rx=0 tx=0).stop 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4183868708 shutdown_connections 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fba8ee640 1 --2- 172.21.15.178:0/4183868708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb4141d10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.208+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4183868708 >> 172.21.15.178:0/4183868708 conn(0x7f8fb40d1f20 msgr2=0x7f8fb40d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.209+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4183868708 shutdown_connections 2024-04-24T23:57:56.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.209+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4183868708 wait complete. 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.210+0000 7f8fba8ee640 1 Processor -- start 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.210+0000 7f8fba8ee640 1 -- start start 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.210+0000 7f8fba8ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fb40d1a10 0x7f8fb413e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.210+0000 7f8fba8ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fb413f2a0 0x7f8fb41580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fba8ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb40c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fba8ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40717c0 con 0x7f8fb40d1a10 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fba8ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40ce370 con 0x7f8fb413f2a0 2024-04-24T23:57:56.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fba8ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb40708b0 con 0x7f8fb41418a0 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fb413f2a0 0x7f8fb41580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb40c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb40c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37722/0 (socket says 172.21.15.178:37722) 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fb413f2a0 0x7f8fb41580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39900/0 (socket says 172.21.15.178:39900) 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 -- 172.21.15.178:0/4001280503 learned_addr learned my addr 172.21.15.178:0/4001280503 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb3fff640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fb40d1a10 0x7f8fb413e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 -- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fb413f2a0 msgr2=0x7f8fb41580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fb413f2a0 0x7f8fb41580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 -- 172.21.15.178:0/4001280503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fb40d1a10 msgr2=0x7f8fb413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fb40d1a10 0x7f8fb413e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 -- 172.21.15.178:0/4001280503 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb415a370 con 0x7f8fb41418a0 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb37fe640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fb413f2a0 0x7f8fb41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb8e8b640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb40c7e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009ee0 tx=0x7f8fac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb0ff9640 1 -- 172.21.15.178:0/4001280503 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac03f070 con 0x7f8fb41418a0 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb0ff9640 1 -- 172.21.15.178:0/4001280503 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac02ae60 con 0x7f8fb41418a0 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb415a610 con 0x7f8fb41418a0 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fb0ff9640 1 -- 172.21.15.178:0/4001280503 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac031be0 con 0x7f8fb41418a0 2024-04-24T23:57:56.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.211+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb415abd0 con 0x7f8fb41418a0 2024-04-24T23:57:56.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.212+0000 7f8fb0ff9640 1 -- 172.21.15.178:0/4001280503 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8fac003990 con 0x7f8fb41418a0 2024-04-24T23:57:56.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.212+0000 7f8fb0ff9640 1 -- 172.21.15.178:0/4001280503 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f8fac044460 con 0x7f8fb41418a0 2024-04-24T23:57:56.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.213+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb415b3b0 con 0x7f8fb41418a0 2024-04-24T23:57:56.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.215+0000 7f8fb0ff9640 1 -- 172.21.15.178:0/4001280503 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8fac049060 con 0x7f8fb41418a0 2024-04-24T23:57:56.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.287+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a878c827-2212-4027-84a3-c426552c4fce", "id": 3} v 0) -- 0x7f8fb4071a20 con 0x7f8fb41418a0 2024-04-24T23:57:56.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.290+0000 7f8fb0ff9640 1 -- 172.21.15.178:0/4001280503 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a878c827-2212-4027-84a3-c426552c4fce", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f8fac031d80 con 0x7f8fb41418a0 2024-04-24T23:57:56.290 INFO:teuthology.orchestra.run.smithi178.stdout:3 2024-04-24T23:57:56.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 msgr2=0x7f8fb40c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb40c7e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009ee0 tx=0x7f8fac004a40 comp rx=0 tx=0).stop 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 shutdown_connections 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fb413f2a0 0x7f8fb41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fb41418a0 0x7f8fb40c7e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 --2- 172.21.15.178:0/4001280503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fb40d1a10 0x7f8fb413e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 >> 172.21.15.178:0/4001280503 conn(0x7f8fb40d1f20 msgr2=0x7f8fb4159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 shutdown_connections 2024-04-24T23:57:56.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.291+0000 7f8fba8ee640 1 -- 172.21.15.178:0/4001280503 wait complete. 2024-04-24T23:57:56.302 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 4c8d4c12-d33b-44fa-8393-a4628959cccb 4 2024-04-24T23:57:56.358 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:56.358+0000 7f037dd23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:57:56.369 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:56.370+0000 7f08b4320040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:57:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.400+0000 7f67307ce640 1 Processor -- start 2024-04-24T23:57:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.400+0000 7f67307ce640 1 -- start start 2024-04-24T23:57:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.400+0000 7f67307ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672815e480 0x7f67280c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f67307ce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6728062290 0x7f672806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f67307ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672806faf0 con 0x7f672806f610 2024-04-24T23:57:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f67307ce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f67280c7360 con 0x7f6728062290 2024-04-24T23:57:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f67307ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f672806fdd0 con 0x7f672815e480 2024-04-24T23:57:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672e56a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672815e480 0x7f67280c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672ed6b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6728062290 0x7f672806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672ed6b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6728062290 0x7f672806b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39904/0 (socket says 172.21.15.178:39904) 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672e56a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672815e480 0x7f67280c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37724/0 (socket says 172.21.15.178:37724) 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672ed6b640 1 -- 172.21.15.178:0/2791459493 learned_addr learned my addr 172.21.15.178:0/2791459493 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672ed6b640 1 -- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672815e480 msgr2=0x7f67280c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672ed6b640 1 --2- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672815e480 0x7f67280c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672ed6b640 1 -- 172.21.15.178:0/2791459493 >> v1:172.21.15.113:6789/0 conn(0x7f672806f610 legacy=0x7f6728061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672ed6b640 1 -- 172.21.15.178:0/2791459493 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67280c6880 con 0x7f6728062290 2024-04-24T23:57:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.401+0000 7f672e56a640 1 --2- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672815e480 0x7f67280c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.402+0000 7f672ed6b640 1 --2- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f672806b0f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6720009e50 tx=0x7f672002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b0be080c55300d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.402+0000 7f672d568640 1 -- 172.21.15.178:0/2791459493 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720036070 con 0x7f6728062290 2024-04-24T23:57:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.402+0000 7f672d568640 1 -- 172.21.15.178:0/2791459493 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6720031600 con 0x7f6728062290 2024-04-24T23:57:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.402+0000 7f672d568640 1 -- 172.21.15.178:0/2791459493 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200319d0 con 0x7f6728062290 2024-04-24T23:57:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 -- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 msgr2=0x7f672806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 --2- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f672806b0f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6720009e50 tx=0x7f672002a710 comp rx=0 tx=0).stop 2024-04-24T23:57:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 -- 172.21.15.178:0/2791459493 shutdown_connections 2024-04-24T23:57:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 --2- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f672806b0f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 --2- 172.21.15.178:0/2791459493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672815e480 0x7f67280c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 -- 172.21.15.178:0/2791459493 >> 172.21.15.178:0/2791459493 conn(0x7f67280cd9b0 msgr2=0x7f67280cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 -- 172.21.15.178:0/2791459493 shutdown_connections 2024-04-24T23:57:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.403+0000 7f67307ce640 1 -- 172.21.15.178:0/2791459493 wait complete. 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.405+0000 7f67307ce640 1 Processor -- start 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.405+0000 7f67307ce640 1 -- start start 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f67307ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f67280c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f67307ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672806f610 0x7f672814b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f67307ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672815e480 0x7f67280c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f67307ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67280c7c10 con 0x7f672815e480 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f67307ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67280c6e40 con 0x7f6728062290 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f67307ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f67280c7d90 con 0x7f672806f610 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f67280c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f67280c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39916/0 (socket says 172.21.15.178:39916) 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 -- 172.21.15.178:0/1149731044 learned_addr learned my addr 172.21.15.178:0/1149731044 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672dd69640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672806f610 0x7f672814b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672ed6b640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672815e480 0x7f67280c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 -- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672806f610 msgr2=0x7f672814b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672806f610 0x7f672814b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 -- 172.21.15.178:0/1149731044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672815e480 msgr2=0x7f67280c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672815e480 0x7f67280c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 -- 172.21.15.178:0/1149731044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67280b1530 con 0x7f6728062290 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.406+0000 7f672e56a640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f67280c8000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f671c00e990 tx=0x7f671c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.407+0000 7f671affd640 1 -- 172.21.15.178:0/1149731044 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c017070 con 0x7f6728062290 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.407+0000 7f671affd640 1 -- 172.21.15.178:0/1149731044 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c00c9e0 con 0x7f6728062290 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.407+0000 7f671affd640 1 -- 172.21.15.178:0/1149731044 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c00cdb0 con 0x7f6728062290 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.407+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67280b1810 con 0x7f6728062290 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.407+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67280b1dd0 con 0x7f6728062290 2024-04-24T23:57:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.407+0000 7f671affd640 1 -- 172.21.15.178:0/1149731044 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f671c01a070 con 0x7f6728062290 2024-04-24T23:57:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.407+0000 7f671affd640 1 -- 172.21.15.178:0/1149731044 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f671c01c960 con 0x7f6728062290 2024-04-24T23:57:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.408+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6728068220 con 0x7f6728062290 2024-04-24T23:57:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.410+0000 7f671affd640 1 -- 172.21.15.178:0/1149731044 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f671c01cc40 con 0x7f6728062290 2024-04-24T23:57:56.445 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:56.445+0000 7f037dd23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:57:56.461 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:56.461+0000 7f08b4320040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:57:56.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.481+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4c8d4c12-d33b-44fa-8393-a4628959cccb", "id": 4} v 0) -- 0x7f67280c8540 con 0x7f6728062290 2024-04-24T23:57:56.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.484+0000 7f671affd640 1 -- 172.21.15.178:0/1149731044 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4c8d4c12-d33b-44fa-8393-a4628959cccb", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f671c0093e0 con 0x7f6728062290 2024-04-24T23:57:56.484 INFO:teuthology.orchestra.run.smithi178.stdout:4 2024-04-24T23:57:56.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 msgr2=0x7f67280c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f67280c8000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f671c00e990 tx=0x7f671c00ee50 comp rx=0 tx=0).stop 2024-04-24T23:57:56.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 shutdown_connections 2024-04-24T23:57:56.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f672815e480 0x7f67280c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f672806f610 0x7f672814b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 --2- 172.21.15.178:0/1149731044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728062290 0x7f67280c8000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 >> 172.21.15.178:0/1149731044 conn(0x7f67280cd9b0 msgr2=0x7f672815fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 shutdown_connections 2024-04-24T23:57:56.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.485+0000 7f67307ce640 1 -- 172.21.15.178:0/1149731044 wait complete. 2024-04-24T23:57:56.487 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:56.487+0000 7f037dd23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:57:56.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 4df70f87-c9a1-4614-928d-b71a7a4fc6a9 5 2024-04-24T23:57:56.506 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:56.506+0000 7f08b4320040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:57:56.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.592+0000 7ff8ef3ce640 1 Processor -- start 2024-04-24T23:57:56.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.592+0000 7ff8ef3ce640 1 -- start start 2024-04-24T23:57:56.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ef3ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8e80b5ca0 0x7ff8e80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ef3ce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8e80b73c0 0x7ff8e80b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ef3ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8e80d2e00 con 0x7ff8e815e510 2024-04-24T23:57:56.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ef3ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff8e80b6640 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ef3ce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff8e80d30e0 con 0x7ff8e80b73c0 2024-04-24T23:57:56.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ed96b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8e80b73c0 0x7ff8e80b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ed96b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8e80b73c0 0x7ff8e80b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39924/0 (socket says 172.21.15.178:39924) 2024-04-24T23:57:56.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ed96b640 1 -- 172.21.15.178:0/3307701477 learned_addr learned my addr 172.21.15.178:0/3307701477 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:56.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ec969640 1 --2- 172.21.15.178:0/3307701477 >> v2:172.21.15.113:3300/0 conn(0x7ff8e80b5ca0 0x7ff8e80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ed96b640 1 -- 172.21.15.178:0/3307701477 >> v2:172.21.15.113:3300/0 conn(0x7ff8e80b5ca0 msgr2=0x7ff8e80b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:56.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ed96b640 1 --2- 172.21.15.178:0/3307701477 >> v2:172.21.15.113:3300/0 conn(0x7ff8e80b5ca0 0x7ff8e80b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ed96b640 1 -- 172.21.15.178:0/3307701477 >> v1:172.21.15.113:6789/0 conn(0x7ff8e815e510 legacy=0x7ff8e80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.593+0000 7ff8ed96b640 1 -- 172.21.15.178:0/3307701477 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e80d33c0 con 0x7ff8e80b73c0 2024-04-24T23:57:56.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.594+0000 7ff8ed96b640 1 --2- 172.21.15.178:0/3307701477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b73c0 0x7ff8e80b77d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80d1d80 tx=0x7ff8e002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab40dd73b235ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.594+0000 7ff8dffff640 1 -- 172.21.15.178:0/3307701477 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0036070 con 0x7ff8e80b73c0 2024-04-24T23:57:56.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.594+0000 7ff8dffff640 1 -- 172.21.15.178:0/3307701477 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e0031600 con 0x7ff8e80b73c0 2024-04-24T23:57:56.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.594+0000 7ff8dffff640 1 -- 172.21.15.178:0/3307701477 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00319d0 con 0x7ff8e80b73c0 2024-04-24T23:57:56.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/3307701477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b73c0 msgr2=0x7ff8e80b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 --2- 172.21.15.178:0/3307701477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b73c0 0x7ff8e80b77d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80d1d80 tx=0x7ff8e002a6a0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/3307701477 shutdown_connections 2024-04-24T23:57:56.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 --2- 172.21.15.178:0/3307701477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b73c0 0x7ff8e80b77d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 --2- 172.21.15.178:0/3307701477 >> v2:172.21.15.113:3300/0 conn(0x7ff8e80b5ca0 0x7ff8e80b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/3307701477 >> 172.21.15.178:0/3307701477 conn(0x7ff8e806b720 msgr2=0x7ff8e80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/3307701477 shutdown_connections 2024-04-24T23:57:56.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.595+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/3307701477 wait complete. 2024-04-24T23:57:56.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.596+0000 7ff8ef3ce640 1 Processor -- start 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.596+0000 7ff8ef3ce640 1 -- start start 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ef3ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b5ca0 0x7ff8e80c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ef3ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e80b73c0 0x7ff8e8160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ef3ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e815e510 0x7ff8e80b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ef3ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e80d3ea0 con 0x7ff8e815e510 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ef3ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e80d3980 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ef3ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e80d2a90 con 0x7ff8e80b73c0 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b5ca0 0x7ff8e80c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ec969640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e80b73c0 0x7ff8e8160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b5ca0 0x7ff8e80c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39932/0 (socket says 172.21.15.178:39932) 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 -- 172.21.15.178:0/2595727288 learned_addr learned my addr 172.21.15.178:0/2595727288 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed96b640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e815e510 0x7ff8e80b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 -- 172.21.15.178:0/2595727288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e80b73c0 msgr2=0x7ff8e8160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e80b73c0 0x7ff8e8160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 -- 172.21.15.178:0/2595727288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e815e510 msgr2=0x7ff8e80b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e815e510 0x7ff8e80b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 -- 172.21.15.178:0/2595727288 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e80c8710 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ed16a640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b5ca0 0x7ff8e80c91b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c400 tx=0x7ff8d800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ddffb640 1 -- 172.21.15.178:0/2595727288 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80052d0 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ddffb640 1 -- 172.21.15.178:0/2595727288 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d8005490 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.597+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e8160490 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.598+0000 7ff8ddffb640 1 -- 172.21.15.178:0/2595727288 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8007260 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.598+0000 7ff8ddffb640 1 -- 172.21.15.178:0/2595727288 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff8d8007400 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.598+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e80c89b0 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.598+0000 7ff8ddffb640 1 -- 172.21.15.178:0/2595727288 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ff8d8006da0 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.599+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e80691a0 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.601+0000 7ff8ddffb640 1 -- 172.21.15.178:0/2595727288 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff8d8020070 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.651 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:56.651+0000 7f037dd23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:57:56.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.673+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4df70f87-c9a1-4614-928d-b71a7a4fc6a9", "id": 5} v 0) -- 0x7ff8e80b61a0 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.676+0000 7ff8ddffb640 1 -- 172.21.15.178:0/2595727288 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4df70f87-c9a1-4614-928d-b71a7a4fc6a9", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ff8d8003650 con 0x7ff8e80b5ca0 2024-04-24T23:57:56.676 INFO:teuthology.orchestra.run.smithi178.stdout:5 2024-04-24T23:57:56.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b5ca0 msgr2=0x7ff8e80c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b5ca0 0x7ff8e80c91b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c400 tx=0x7ff8d800c8c0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 shutdown_connections 2024-04-24T23:57:56.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e815e510 0x7ff8e80b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e80b73c0 0x7ff8e8160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 --2- 172.21.15.178:0/2595727288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8e80b5ca0 0x7ff8e80c91b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 >> 172.21.15.178:0/2595727288 conn(0x7ff8e806b720 msgr2=0x7ff8e80d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 shutdown_connections 2024-04-24T23:57:56.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.677+0000 7ff8ef3ce640 1 -- 172.21.15.178:0/2595727288 wait complete. 2024-04-24T23:57:56.679 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:56.680+0000 7f08b4320040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:57:56.688 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new dc93ad16-c47e-4936-b4e4-1ee139eff53d 6 2024-04-24T23:57:56.708 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:56.708+0000 7f037dd23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:57:56.728 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:56.729+0000 7f08b4320040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:57:56.760 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:56.760+0000 7f037dd23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:57:56.784 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:56.784+0000 7f08b4320040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:57:56.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.826+0000 7fcc1d116640 1 Processor -- start 2024-04-24T23:57:56.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.826+0000 7fcc1d116640 1 -- start start 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc1d116640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc180cdd20 0x7fcc180cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc1d116640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcc18141f20 0x7fcc1806f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc1d116640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc180d0030 con 0x7fcc1815d2c0 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc1d116640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcc18071290 con 0x7fcc18141f20 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc1d116640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcc18070960 con 0x7fcc180cdd20 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc180cdd20 0x7fcc180cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc180cdd20 0x7fcc180cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37744/0 (socket says 172.21.15.178:37744) 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc1759e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcc18141f20 0x7fcc1806f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc16d9d640 1 -- 172.21.15.178:0/2120366048 learned_addr learned my addr 172.21.15.178:0/2120366048 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:56.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc16d9d640 1 -- 172.21.15.178:0/2120366048 >> v2:172.21.15.178:3300/0 conn(0x7fcc18141f20 msgr2=0x7fcc1806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc16d9d640 1 --2- 172.21.15.178:0/2120366048 >> v2:172.21.15.178:3300/0 conn(0x7fcc18141f20 0x7fcc1806f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc16d9d640 1 -- 172.21.15.178:0/2120366048 >> v1:172.21.15.113:6789/0 conn(0x7fcc1815d2c0 legacy=0x7fcc18141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.827+0000 7fcc16d9d640 1 -- 172.21.15.178:0/2120366048 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc18070c40 con 0x7fcc180cdd20 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc16d9d640 1 --2- 172.21.15.178:0/2120366048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc180cdd20 0x7fcc180cfaf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009870 tx=0x7fcc0802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8f20c46924b855 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1659c640 1 -- 172.21.15.178:0/2120366048 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08037070 con 0x7fcc180cdd20 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1659c640 1 -- 172.21.15.178:0/2120366048 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc08032440 con 0x7fcc180cdd20 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1659c640 1 -- 172.21.15.178:0/2120366048 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08032810 con 0x7fcc180cdd20 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1d116640 1 -- 172.21.15.178:0/2120366048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc180cdd20 msgr2=0x7fcc180cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1d116640 1 --2- 172.21.15.178:0/2120366048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc180cdd20 0x7fcc180cfaf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009870 tx=0x7fcc0802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1d116640 1 -- 172.21.15.178:0/2120366048 shutdown_connections 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1d116640 1 --2- 172.21.15.178:0/2120366048 >> v2:172.21.15.178:3300/0 conn(0x7fcc18141f20 0x7fcc1806f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1d116640 1 --2- 172.21.15.178:0/2120366048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc180cdd20 0x7fcc180cfaf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.828+0000 7fcc1d116640 1 -- 172.21.15.178:0/2120366048 >> 172.21.15.178:0/2120366048 conn(0x7fcc180d1f20 msgr2=0x7fcc180d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.829+0000 7fcc1d116640 1 -- 172.21.15.178:0/2120366048 shutdown_connections 2024-04-24T23:57:56.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.829+0000 7fcc1d116640 1 -- 172.21.15.178:0/2120366048 wait complete. 2024-04-24T23:57:56.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.830+0000 7fcc1d116640 1 Processor -- start 2024-04-24T23:57:56.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.830+0000 7fcc1d116640 1 -- start start 2024-04-24T23:57:56.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc180cdd20 0x7fcc180b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc18141f20 0x7fcc181e6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc1815d2c0 0x7fcc1813dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18071720 con 0x7fcc1815d2c0 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcc180d0300 con 0x7fcc180cdd20 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcc180ce100 con 0x7fcc18141f20 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc18141f20 0x7fcc181e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc180cdd20 0x7fcc180b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc18141f20 0x7fcc181e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37754/0 (socket says 172.21.15.178:37754) 2024-04-24T23:57:56.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 -- 172.21.15.178:0/208772539 learned_addr learned my addr 172.21.15.178:0/208772539 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1759e640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc1815d2c0 0x7fcc1813dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 -- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc180cdd20 msgr2=0x7fcc180b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc180cdd20 0x7fcc180b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 -- 172.21.15.178:0/208772539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc1815d2c0 msgr2=0x7fcc1813dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc1815d2c0 0x7fcc1813dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 -- 172.21.15.178:0/208772539 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc181e6cb0 con 0x7fcc18141f20 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc16d9d640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc180cdd20 0x7fcc180b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0ffff640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc18141f20 0x7fcc181e6560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c430 tx=0x7fcc0000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0f7fe640 1 -- 172.21.15.178:0/208772539 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc000052d0 con 0x7fcc18141f20 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0f7fe640 1 -- 172.21.15.178:0/208772539 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc00005490 con 0x7fcc18141f20 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc181e6f90 con 0x7fcc18141f20 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc0f7fe640 1 -- 172.21.15.178:0/208772539 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00007260 con 0x7fcc18141f20 2024-04-24T23:57:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.831+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc181e7550 con 0x7fcc18141f20 2024-04-24T23:57:56.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.832+0000 7fcc0f7fe640 1 -- 172.21.15.178:0/208772539 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc00007400 con 0x7fcc18141f20 2024-04-24T23:57:56.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.832+0000 7fcc0f7fe640 1 -- 172.21.15.178:0/208772539 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fcc000035f0 con 0x7fcc18141f20 2024-04-24T23:57:56.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.832+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc18068220 con 0x7fcc18141f20 2024-04-24T23:57:56.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.834+0000 7fcc0f7fe640 1 -- 172.21.15.178:0/208772539 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcc00003a50 con 0x7fcc18141f20 2024-04-24T23:57:56.872 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:56.872+0000 7f037dd23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:57:56.902 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:56.902+0000 7f08b4320040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:57:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.906+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dc93ad16-c47e-4936-b4e4-1ee139eff53d", "id": 6} v 0) -- 0x7fcc18071a40 con 0x7fcc18141f20 2024-04-24T23:57:56.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.910+0000 7fcc0f7fe640 1 -- 172.21.15.178:0/208772539 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dc93ad16-c47e-4936-b4e4-1ee139eff53d", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fcc00003c30 con 0x7fcc18141f20 2024-04-24T23:57:56.910 INFO:teuthology.orchestra.run.smithi178.stdout:6 2024-04-24T23:57:56.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc18141f20 msgr2=0x7fcc181e6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc18141f20 0x7fcc181e6560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c430 tx=0x7fcc0000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 shutdown_connections 2024-04-24T23:57:56.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc1815d2c0 0x7fcc1813dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc18141f20 0x7fcc181e6560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 --2- 172.21.15.178:0/208772539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc180cdd20 0x7fcc180b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 >> 172.21.15.178:0/208772539 conn(0x7fcc180d1f20 msgr2=0x7fcc180c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 shutdown_connections 2024-04-24T23:57:56.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:56.911+0000 7fcc1d116640 1 -- 172.21.15.178:0/208772539 wait complete. 2024-04-24T23:57:56.921 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new c237b717-8504-49fd-a49b-d9a5ac74c922 7 2024-04-24T23:57:57.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.019+0000 7f304c5a5640 1 Processor -- start 2024-04-24T23:57:57.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.019+0000 7f304c5a5640 1 -- start start 2024-04-24T23:57:57.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304c5a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f3044061b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304c5a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f30440620c0 0x7f304406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304c5a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3044160ca0 con 0x7f304415c140 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304c5a5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f30440ceb00 con 0x7f30440620c0 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304c5a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f304406e090 con 0x7f3044160860 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304ab42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f30440620c0 0x7f304406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f3049b40640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f3044061b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304ab42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f30440620c0 0x7f304406e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39946/0 (socket says 172.21.15.178:39946) 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f304ab42640 1 -- 172.21.15.178:0/402200859 learned_addr learned my addr 172.21.15.178:0/402200859 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f3049b40640 1 -- 172.21.15.178:0/402200859 >> v2:172.21.15.178:3300/0 conn(0x7f30440620c0 msgr2=0x7f304406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f3049b40640 1 --2- 172.21.15.178:0/402200859 >> v2:172.21.15.178:3300/0 conn(0x7f30440620c0 0x7f304406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f3049b40640 1 -- 172.21.15.178:0/402200859 >> v1:172.21.15.113:6789/0 conn(0x7f304415c140 legacy=0x7f30440d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.020+0000 7f3049b40640 1 -- 172.21.15.178:0/402200859 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30440ced50 con 0x7f3044160860 2024-04-24T23:57:57.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.021+0000 7f304ab42640 1 --2- 172.21.15.178:0/402200859 >> v2:172.21.15.178:3300/0 conn(0x7f30440620c0 0x7f304406e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:57.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.021+0000 7f3049b40640 1 --2- 172.21.15.178:0/402200859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f3044061b80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f30400097a0 tx=0x7f304002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e301f775a039fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:57.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.021+0000 7f304933f640 1 -- 172.21.15.178:0/402200859 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040037070 con 0x7f3044160860 2024-04-24T23:57:57.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.021+0000 7f304933f640 1 -- 172.21.15.178:0/402200859 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3040032440 con 0x7f3044160860 2024-04-24T23:57:57.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.021+0000 7f304933f640 1 -- 172.21.15.178:0/402200859 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040032810 con 0x7f3044160860 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 -- 172.21.15.178:0/402200859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 msgr2=0x7f3044061b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 --2- 172.21.15.178:0/402200859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f3044061b80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f30400097a0 tx=0x7f304002a180 comp rx=0 tx=0).stop 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 -- 172.21.15.178:0/402200859 shutdown_connections 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 --2- 172.21.15.178:0/402200859 >> v2:172.21.15.178:3300/0 conn(0x7f30440620c0 0x7f304406e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 --2- 172.21.15.178:0/402200859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f3044061b80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 -- 172.21.15.178:0/402200859 >> 172.21.15.178:0/402200859 conn(0x7f30440cd9b0 msgr2=0x7f30440cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 -- 172.21.15.178:0/402200859 shutdown_connections 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.022+0000 7f304c5a5640 1 -- 172.21.15.178:0/402200859 wait complete. 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.023+0000 7f304c5a5640 1 Processor -- start 2024-04-24T23:57:57.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.023+0000 7f304c5a5640 1 -- start start 2024-04-24T23:57:57.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304c5a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30440620c0 0x7f3044153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304c5a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f304415c140 0x7f30440b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304c5a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f30440b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304c5a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f304415c520 con 0x7f30440620c0 2024-04-24T23:57:57.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304c5a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f304406dd20 con 0x7f304415c140 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304c5a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f304415c6a0 con 0x7f3044160860 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f304415c140 0x7f30440b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f304415c140 0x7f30440b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39956/0 (socket says 172.21.15.178:39956) 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 -- 172.21.15.178:0/253794344 learned_addr learned my addr 172.21.15.178:0/253794344 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304ab42640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f30440b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304a341640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30440620c0 0x7f3044153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 -- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 msgr2=0x7f30440b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f30440b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 -- 172.21.15.178:0/253794344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30440620c0 msgr2=0x7f3044153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30440620c0 0x7f3044153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 -- 172.21.15.178:0/253794344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3044153fa0 con 0x7f304415c140 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f304a341640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30440620c0 0x7f3044153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.024+0000 7f3049b40640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f304415c140 0x7f30440b10e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30400098c0 tx=0x7f30400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.025+0000 7f303affd640 1 -- 172.21.15.178:0/253794344 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040037040 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.025+0000 7f303affd640 1 -- 172.21.15.178:0/253794344 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3040030500 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.025+0000 7f303affd640 1 -- 172.21.15.178:0/253794344 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30400308d0 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.025+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30441f80d0 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.025+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30441f8690 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.025+0000 7f303affd640 1 -- 172.21.15.178:0/253794344 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3040044020 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.025+0000 7f303affd640 1 -- 172.21.15.178:0/253794344 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f304003bbb0 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.026+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3044061b80 con 0x7f304415c140 2024-04-24T23:57:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.028+0000 7f303affd640 1 -- 172.21.15.178:0/253794344 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3044061b80 con 0x7f304415c140 2024-04-24T23:57:57.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.099+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c237b717-8504-49fd-a49b-d9a5ac74c922", "id": 7} v 0) -- 0x7f3044068220 con 0x7f304415c140 2024-04-24T23:57:57.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.103+0000 7f303affd640 1 -- 172.21.15.178:0/253794344 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c237b717-8504-49fd-a49b-d9a5ac74c922", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f304003b430 con 0x7f304415c140 2024-04-24T23:57:57.103 INFO:teuthology.orchestra.run.smithi178.stdout:7 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f304415c140 msgr2=0x7f30440b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f304415c140 0x7f30440b10e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30400098c0 tx=0x7f30400046c0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 shutdown_connections 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f304415c140 0x7f30440b10e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3044160860 0x7f30440b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 --2- 172.21.15.178:0/253794344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30440620c0 0x7f3044153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 >> 172.21.15.178:0/253794344 conn(0x7f30440cd9b0 msgr2=0x7f30440cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 shutdown_connections 2024-04-24T23:57:57.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:57:57.104+0000 7f304c5a5640 1 -- 172.21.15.178:0/253794344 wait complete. 2024-04-24T23:57:57.114 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T23:57:57.114 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T23:57:57.116 INFO:tasks.ceph.osd.0:Started 2024-04-24T23:57:57.116 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T23:57:57.117 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T23:57:57.119 INFO:tasks.ceph.osd.1:Started 2024-04-24T23:57:57.119 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T23:57:57.119 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T23:57:57.122 INFO:tasks.ceph.osd.2:Started 2024-04-24T23:57:57.122 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T23:57:57.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T23:57:57.124 INFO:tasks.ceph.osd.3:Started 2024-04-24T23:57:57.124 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T23:57:57.124 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T23:57:57.126 INFO:tasks.ceph.osd.4:Started 2024-04-24T23:57:57.127 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T23:57:57.127 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T23:57:57.129 INFO:tasks.ceph.osd.5:Started 2024-04-24T23:57:57.129 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T23:57:57.129 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T23:57:57.133 INFO:tasks.ceph.osd.6:Started 2024-04-24T23:57:57.134 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T23:57:57.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T23:57:57.138 INFO:tasks.ceph.osd.7:Started 2024-04-24T23:57:57.138 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:57:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.265+0000 7fb5bdde7640 1 Processor -- start 2024-04-24T23:57:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.265+0000 7fb5bdde7640 1 -- start start 2024-04-24T23:57:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.265+0000 7fb5bdde7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb5b80cea50 0x7fb5b806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.265+0000 7fb5bdde7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5b806e950 0x7fb5b806bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.265+0000 7fb5bdde7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5b806ee00 con 0x7fb5b80cb4d0 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.265+0000 7fb5bdde7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b806b0f0 con 0x7fb5b806e950 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.265+0000 7fb5bdde7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b806f8e0 con 0x7fb5b80cea50 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b6ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb5b80cb4d0 0x7fb5b80cee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46176/0 (socket says 172.21.15.113:46176) 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b6ffd640 1 -- 172.21.15.113:0/2279895653 learned_addr learned my addr 172.21.15.113:0/2279895653 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b7fff640 1 --2- 172.21.15.113:0/2279895653 >> v2:172.21.15.113:3300/0 conn(0x7fb5b806e950 0x7fb5b806bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b77fe640 1 --2- 172.21.15.113:0/2279895653 >> v2:172.21.15.178:3300/0 conn(0x7fb5b80cea50 0x7fb5b806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b67fc640 1 -- 172.21.15.113:0/2279895653 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2014003802 0 0) 0x7fb5b806ee00 con 0x7fb5b80cb4d0 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b67fc640 1 -- 172.21.15.113:0/2279895653 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5a0003520 con 0x7fb5b80cb4d0 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b7fff640 1 -- 172.21.15.113:0/2279895653 >> v2:172.21.15.178:3300/0 conn(0x7fb5b80cea50 msgr2=0x7fb5b806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b7fff640 1 --2- 172.21.15.113:0/2279895653 >> v2:172.21.15.178:3300/0 conn(0x7fb5b80cea50 0x7fb5b806f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b7fff640 1 -- 172.21.15.113:0/2279895653 >> v1:172.21.15.113:6789/0 conn(0x7fb5b80cb4d0 legacy=0x7fb5b80cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.266+0000 7fb5b7fff640 1 -- 172.21.15.113:0/2279895653 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b806fbc0 con 0x7fb5b806e950 2024-04-24T23:57:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.267+0000 7fb5b7fff640 1 --2- 172.21.15.113:0/2279895653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b806bb40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80504b0 tx=0x7fb5b002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b853ec72ba43fc82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:57.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.267+0000 7fb5b67fc640 1 -- 172.21.15.113:0/2279895653 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0007740 con 0x7fb5b806e950 2024-04-24T23:57:57.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.267+0000 7fb5b67fc640 1 -- 172.21.15.113:0/2279895653 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b00078e0 con 0x7fb5b806e950 2024-04-24T23:57:57.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.267+0000 7fb5b67fc640 1 -- 172.21.15.113:0/2279895653 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0007cb0 con 0x7fb5b806e950 2024-04-24T23:57:57.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 -- 172.21.15.113:0/2279895653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 msgr2=0x7fb5b806bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 --2- 172.21.15.113:0/2279895653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b806bb40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80504b0 tx=0x7fb5b002a5f0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 -- 172.21.15.113:0/2279895653 shutdown_connections 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 --2- 172.21.15.113:0/2279895653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b806bb40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 --2- 172.21.15.113:0/2279895653 >> v2:172.21.15.178:3300/0 conn(0x7fb5b80cea50 0x7fb5b806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 -- 172.21.15.113:0/2279895653 >> 172.21.15.113:0/2279895653 conn(0x7fb5b80cd900 msgr2=0x7fb5b80cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 -- 172.21.15.113:0/2279895653 shutdown_connections 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.268+0000 7fb5bdde7640 1 -- 172.21.15.113:0/2279895653 wait complete. 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.269+0000 7fb5bdde7640 1 Processor -- start 2024-04-24T23:57:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.269+0000 7fb5bdde7640 1 -- start start 2024-04-24T23:57:57.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5bdde7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b81549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5bdde7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5b80cb4d0 0x7fb5b809f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5bdde7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5b80cea50 0x7fb5b809fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:57.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5bdde7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b815e6e0 con 0x7fb5b806e950 2024-04-24T23:57:57.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5bdde7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b806b620 con 0x7fb5b80cb4d0 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5bdde7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b815e860 con 0x7fb5b80cea50 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b81549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b81549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45940/0 (socket says 172.21.15.113:45940) 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5b6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5b80cb4d0 0x7fb5b809f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5b77fe640 1 -- 172.21.15.113:0/262080715 learned_addr learned my addr 172.21.15.113:0/262080715 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.270+0000 7fb5b7fff640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5b80cea50 0x7fb5b809fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5b77fe640 1 -- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5b80cea50 msgr2=0x7fb5b809fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5b77fe640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5b80cea50 0x7fb5b809fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5b77fe640 1 -- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5b80cb4d0 msgr2=0x7fb5b809f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5b77fe640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5b80cb4d0 0x7fb5b809f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5b77fe640 1 -- 172.21.15.113:0/262080715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b809f2f0 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5b6ffd640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5b80cb4d0 0x7fb5b809f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5b77fe640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b81549b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800e8a0 tx=0x7fb5a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb59ffff640 1 -- 172.21.15.113:0/262080715 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8017070 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb59ffff640 1 -- 172.21.15.113:0/262080715 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a800c940 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b80ccde0 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb59ffff640 1 -- 172.21.15.113:0/262080715 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a800cd30 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.271+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b80c5680 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.272+0000 7fb59ffff640 1 -- 172.21.15.113:0/262080715 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb5a801a070 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.272+0000 7fb59ffff640 1 -- 172.21.15.113:0/262080715 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fb5a80065e0 con 0x7fb5b806e950 2024-04-24T23:57:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.272+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b8074e20 con 0x7fb5b806e950 2024-04-24T23:57:57.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.274+0000 7fb59ffff640 1 -- 172.21.15.113:0/262080715 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb5a8006830 con 0x7fb5b806e950 2024-04-24T23:57:57.291 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:57.291+0000 7f037dd23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:57:57.317 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:57.317+0000 7f08b4320040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:57:57.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.336+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5b8068220 con 0x7fb5b806e950 2024-04-24T23:57:57.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.336+0000 7fb59ffff640 1 -- 172.21.15.113:0/262080715 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fb5a8005990 con 0x7fb5b806e950 2024-04-24T23:57:57.337 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:57:57.337 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":10,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:57:57.101501+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:57:57.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.337+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 msgr2=0x7fb5b81549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.337+0000 7fb5bdde7640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b81549b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800e8a0 tx=0x7fb5a800ed60 comp rx=0 tx=0).stop 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.337+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 shutdown_connections 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.337+0000 7fb5bdde7640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5b80cb4d0 0x7fb5b809f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.337+0000 7fb5bdde7640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5b80cea50 0x7fb5b809fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.337+0000 7fb5bdde7640 1 --2- 172.21.15.113:0/262080715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5b806e950 0x7fb5b81549b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.337+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 >> 172.21.15.113:0/262080715 conn(0x7fb5b80cd900 msgr2=0x7fb5b8071e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.338+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 shutdown_connections 2024-04-24T23:57:57.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:57:57.338+0000 7fb5bdde7640 1 -- 172.21.15.113:0/262080715 wait complete. 2024-04-24T23:57:57.351 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T23:57:57.351 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T23:57:57.553 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:57.553+0000 7f037dd23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:57:57.575 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:57.575+0000 7f08b4320040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:57:57.605 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:57.606+0000 7f037dd23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:57:57.627 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:57.627+0000 7f08b4320040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:57:57.721 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:57.721+0000 7f037dd23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:57:57.723 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-04-24T23:57:57.723+0000 7f1e0a415640 -1 Falling back to public interface 2024-04-24T23:57:57.726 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-24T23:57:57.726+0000 7f790392c640 -1 Falling back to public interface 2024-04-24T23:57:57.731 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-24T23:57:57.731+0000 7fae591dc640 -1 Falling back to public interface 2024-04-24T23:57:57.734 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-24T23:57:57.734+0000 7f6db2a1a640 -1 Falling back to public interface 2024-04-24T23:57:57.736 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-04-24T23:57:57.736+0000 7f96f03a7640 -1 Falling back to public interface 2024-04-24T23:57:57.742 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-24T23:57:57.742+0000 7f1fdd41e640 -1 Falling back to public interface 2024-04-24T23:57:57.745 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:57.744+0000 7f08b4320040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:57:57.751 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-24T23:57:57.751+0000 7f89b341d640 -1 Falling back to public interface 2024-04-24T23:57:57.757 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-04-24T23:57:57.757+0000 7fc5218b1640 -1 Falling back to public interface 2024-04-24T23:57:58.000 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.000+0000 7f037dd23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:57:58.023 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.023+0000 7f08b4320040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:57:58.049 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.049+0000 7f037dd23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:57:58.071 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.071+0000 7f08b4320040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:57:58.157 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:57:58.208 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.208+0000 7f037dd23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:57:58.225 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.225+0000 7f08b4320040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:57:58.247 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.247+0000 7f846b970640 1 Processor -- start 2024-04-24T23:57:58.248 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.248+0000 7f846b970640 1 -- start start 2024-04-24T23:57:58.251 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.248+0000 7f846b970640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 0x7f846406f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.251 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.248+0000 7f846b970640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f846406f9a0 0x7f84640b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.248+0000 7f846b970640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84640d10f0 con 0x7f8464145ad0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.248+0000 7f846b970640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f846406fe10 con 0x7f846406f9a0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.248+0000 7f846b970640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84640b1a50 con 0x7f84640709a0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f846970c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8464145ad0 0x7f84640d08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55716/0 (socket says 172.21.15.113:55716) 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f846970c640 1 -- 172.21.15.113:0/2318233466 learned_addr learned my addr 172.21.15.113:0/2318233466 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f8468f0b640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 0x7f846406f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f8469f0d640 1 --2- 172.21.15.113:0/2318233466 >> v2:172.21.15.178:3300/0 conn(0x7f846406f9a0 0x7f84640b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971365129 0 0) 0x7f84640d10f0 con 0x7f8464145ad0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8450003520 con 0x7f8464145ad0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735430147 0 0) 0x7f8450003520 con 0x7f8464145ad0 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 msgr2=0x7f846406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:58.252 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 0x7f846406f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 >> v2:172.21.15.178:3300/0 conn(0x7f846406f9a0 msgr2=0x7f84640b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 --2- 172.21.15.113:0/2318233466 >> v2:172.21.15.178:3300/0 conn(0x7f846406f9a0 0x7f84640b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84640d13d0 con 0x7f8464145ad0 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8460001670 con 0x7f8464145ad0 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 >> v1:172.21.15.113:6789/0 conn(0x7f8464145ad0 legacy=0x7f84640d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500039b0 0x7f8450003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450004800 0x7f8450004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 0x7f84500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f846406fe10 con 0x7f84500039b0 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f845000b9c0 con 0x7f8450004800 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f845000bb40 con 0x7f84640709a0 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f846970c640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500039b0 0x7f8450003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.253 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f8468f0b640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450004800 0x7f8450004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.249+0000 7f8469f0d640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 0x7f84500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.250+0000 7f846970c640 1 -- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 msgr2=0x7f84500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.250+0000 7f846970c640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84640709a0 0x7f84500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.250+0000 7f846970c640 1 -- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450004800 msgr2=0x7f8450004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.250+0000 7f846970c640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450004800 0x7f8450004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.250+0000 7f846970c640 1 -- 172.21.15.113:0/2318233466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8450003520 con 0x7f84500039b0 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.250+0000 7f8468f0b640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450004800 0x7f8450004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f846970c640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500039b0 0x7f8450003dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f846000c290 tx=0x7f8450005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f84ceb7522c6bff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460007d50 con 0x7f84500039b0 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8460005af0 con 0x7f84500039b0 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f845bfff640 1 -- 172.21.15.113:0/2318233466 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460007070 con 0x7f84500039b0 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f846b970640 1 -- 172.21.15.113:0/2318233466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500039b0 msgr2=0x7f8450003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:58.254 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f846b970640 1 --2- 172.21.15.113:0/2318233466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84500039b0 0x7f8450003dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f846000c290 tx=0x7f8450005830 comp rx=0 tx=0).stop 2024-04-24T23:57:58.255 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f8469f0d640 1 -- 172.21.15.113:0/2318233466 reap_dead start 2024-04-24T23:57:58.255 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f846b970640 1 -- 172.21.15.113:0/2318233466 shutdown_connections 2024-04-24T23:57:58.255 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.251+0000 7f846b970640 1 -- 172.21.15.113:0/2318233466 >> 172.21.15.113:0/2318233466 conn(0x7f84640b3c70 msgr2=0x7f84640b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:58.255 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.252+0000 7f846b970640 1 -- 172.21.15.113:0/2318233466 shutdown_connections 2024-04-24T23:57:58.255 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.252+0000 7f846b970640 1 -- 172.21.15.113:0/2318233466 wait complete. 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 Processor -- start 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 -- start start 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84500039b0 0x7f846415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84641f1bc0 0x7f84641420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84641f2040 0x7f84640aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f84640d1990 con 0x7f84641f1bc0 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84640d1d00 con 0x7f84641f2040 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f846b970640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f846406f460 con 0x7f84500039b0 2024-04-24T23:57:58.258 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.253+0000 7f8468f0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84641f1bc0 0x7f84641420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84641f1bc0 0x7f84641420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40048/0 (socket says 172.21.15.113:40048) 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 -- 172.21.15.113:0/1629846057 learned_addr learned my addr 172.21.15.113:0/1629846057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f846970c640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84500039b0 0x7f846415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8469f0d640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84641f2040 0x7f84640aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 -- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84500039b0 msgr2=0x7f846415bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84500039b0 0x7f846415bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 -- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84641f2040 msgr2=0x7f84640aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84641f2040 0x7f84640aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 -- 172.21.15.113:0/1629846057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846413e9f0 con 0x7f84641f1bc0 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8468f0b640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84641f1bc0 0x7f84641420f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8454011a70 tx=0x7f8454011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8459ffb640 1 -- 172.21.15.113:0/1629846057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845400fbb0 con 0x7f84641f1bc0 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8459ffb640 1 -- 172.21.15.113:0/1629846057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845400fd50 con 0x7f84641f1bc0 2024-04-24T23:57:58.259 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846413ecd0 con 0x7f84641f1bc0 2024-04-24T23:57:58.260 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8459ffb640 1 -- 172.21.15.113:0/1629846057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454014640 con 0x7f84641f1bc0 2024-04-24T23:57:58.260 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846413f290 con 0x7f84641f1bc0 2024-04-24T23:57:58.260 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8459ffb640 1 -- 172.21.15.113:0/1629846057 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8454012040 con 0x7f84641f1bc0 2024-04-24T23:57:58.260 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.254+0000 7f8459ffb640 1 -- 172.21.15.113:0/1629846057 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f84540263f0 con 0x7f84641f1bc0 2024-04-24T23:57:58.260 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.255+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846404f570 con 0x7f84641f1bc0 2024-04-24T23:57:58.260 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.257+0000 7f8459ffb640 1 -- 172.21.15.113:0/1629846057 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8454019030 con 0x7f84641f1bc0 2024-04-24T23:57:58.319 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.319+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f846413c0e0 con 0x7f84641f1bc0 2024-04-24T23:57:58.319 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.319+0000 7f8459ffb640 1 -- 172.21.15.113:0/1629846057 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f84540266b0 con 0x7f84641f1bc0 2024-04-24T23:57:58.320 INFO:teuthology.misc.health.smithi113.stdout: 2024-04-24T23:57:58.320 INFO:teuthology.misc.health.smithi113.stdout:{"epoch":10,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:57:57.101501+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84641f1bc0 msgr2=0x7f84641420f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84641f1bc0 0x7f84641420f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8454011a70 tx=0x7f8454011f30 comp rx=0 tx=0).stop 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 shutdown_connections 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84641f2040 0x7f84640aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84641f1bc0 0x7f84641420f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 --2- 172.21.15.113:0/1629846057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84500039b0 0x7f846415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 >> 172.21.15.113:0/1629846057 conn(0x7f84640b3c70 msgr2=0x7f8464143a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 shutdown_connections 2024-04-24T23:57:58.321 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:57:58.320+0000 7f846b970640 1 -- 172.21.15.113:0/1629846057 wait complete. 2024-04-24T23:57:58.329 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T23:57:58.351 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.351+0000 7f037dd23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:57:58.362 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.362+0000 7f08b4320040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:57:58.400 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.400+0000 7f037dd23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:57:58.411 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.412+0000 7f08b4320040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:57:58.469 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.469+0000 7f037dd23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:57:58.480 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.480+0000 7f08b4320040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:57:58.513 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.513+0000 7f037dd23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:57:58.524 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.524+0000 7f08b4320040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:57:58.983 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:58.984+0000 7f037dd23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:57:58.996 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:58.997+0000 7f08b4320040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:57:59.099 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:57:59.099+0000 7f037dd23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:57:59.113 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-24T23:57:59.114+0000 7f08b4320040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:58:00.114 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:58:00.114+0000 7f031e8aa640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T23:58:00.115 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-04-24T23:58:00.114+0000 7f031e8aa640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T23:58:01.234 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-24T23:58:01.234+0000 7f790392c640 -1 osd.1 0 log_to_monitors true 2024-04-24T23:58:01.257 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-24T23:58:01.257+0000 7fae591dc640 -1 osd.6 0 log_to_monitors true 2024-04-24T23:58:01.265 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-04-24T23:58:01.265+0000 7f96f03a7640 -1 osd.2 0 log_to_monitors true 2024-04-24T23:58:01.267 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-24T23:58:01.268+0000 7f1fdd41e640 -1 osd.5 0 log_to_monitors true 2024-04-24T23:58:01.268 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-24T23:58:01.268+0000 7f89b341d640 -1 osd.7 0 log_to_monitors true 2024-04-24T23:58:01.269 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-24T23:58:01.269+0000 7f6db2a1a640 -1 osd.4 0 log_to_monitors true 2024-04-24T23:58:01.271 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-04-24T23:58:01.271+0000 7f1e0a415640 -1 osd.0 0 log_to_monitors true 2024-04-24T23:58:01.277 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-04-24T23:58:01.277+0000 7fc5218b1640 -1 osd.3 0 log_to_monitors true 2024-04-24T23:58:04.462 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-24T23:58:04.462+0000 7f1fd8647640 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:58:04.481 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-24T23:58:04.482+0000 7f1fd0c79640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:04.520 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-24T23:58:04.520+0000 7fae5494f640 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:58:04.534 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-24T23:58:04.534+0000 7f6daf3fc640 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:58:04.536 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-24T23:58:04.537+0000 7fae4cf81640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:04.543 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-24T23:58:04.544+0000 7f6da61f6640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:04.567 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-24T23:58:04.567+0000 7f89aeb90640 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:58:04.581 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-24T23:58:04.582+0000 7f89a71c2640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:04.609 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-24T23:58:04.609+0000 7f78ff09f640 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:58:04.619 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-24T23:58:04.620+0000 7f78f76d1640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:04.626 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-04-24T23:58:04.626+0000 7f1e054d8640 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:58:04.633 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-04-24T23:58:04.633+0000 7f96ebb1a640 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:58:04.633 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-04-24T23:58:04.633+0000 7fc51e85c640 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:58:04.638 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-04-24T23:58:04.639+0000 7f1dfdb0a640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:04.642 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-04-24T23:58:04.643+0000 7f96e414c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:04.643 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-04-24T23:58:04.643+0000 7fc515656640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:05.136 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:05.229 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.229+0000 7fcfbf361640 1 Processor -- start 2024-04-24T23:58:05.229 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.229+0000 7fcfbf361640 1 -- start start 2024-04-24T23:58:05.230 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbf361640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:05.230 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbf361640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcfb80d0560 0x7fcfb80720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbf361640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb8072720 con 0x7fcfb8071c60 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbf361640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb80b30f0 con 0x7fcfb80d0560 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbf361640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb80b33d0 con 0x7fcfb815a8d0 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd0fd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd0fd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:58172/0 (socket says 172.21.15.113:58172) 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/3867789955 learned_addr learned my addr 172.21.15.113:0/3867789955 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd8fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcfb80d0560 0x7fcfb80720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfb7fff640 1 -- 172.21.15.113:0/3867789955 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676399560 0 0) 0x7fcfb8072720 con 0x7fcfb8071c60 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfb7fff640 1 -- 172.21.15.113:0/3867789955 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfa0004630 con 0x7fcfb8071c60 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/3867789955 >> v2:172.21.15.178:3300/0 conn(0x7fcfb80d0560 msgr2=0x7fcfb80720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd0fd640 1 --2- 172.21.15.113:0/3867789955 >> v2:172.21.15.178:3300/0 conn(0x7fcfb80d0560 0x7fcfb80720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/3867789955 >> v1:172.21.15.178:6789/0 conn(0x7fcfb8071c60 legacy=0x7fcfb80cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:05.231 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/3867789955 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb80b36b0 con 0x7fcfb815a8d0 2024-04-24T23:58:05.232 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.230+0000 7fcfbd8fe640 1 --2- 172.21.15.113:0/3867789955 >> v2:172.21.15.178:3300/0 conn(0x7fcfb80d0560 0x7fcfb80720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.231+0000 7fcfbd0fd640 1 --2- 172.21.15.113:0/3867789955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80b3bd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009870 tx=0x7fcfb002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292407c1d462efee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.231+0000 7fcfb7fff640 1 -- 172.21.15.113:0/3867789955 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0037070 con 0x7fcfb815a8d0 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.231+0000 7fcfb7fff640 1 -- 172.21.15.113:0/3867789955 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb0032440 con 0x7fcfb815a8d0 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.231+0000 7fcfb7fff640 1 -- 172.21.15.113:0/3867789955 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0032810 con 0x7fcfb815a8d0 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 -- 172.21.15.113:0/3867789955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 msgr2=0x7fcfb80b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 --2- 172.21.15.113:0/3867789955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80b3bd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009870 tx=0x7fcfb002a250 comp rx=0 tx=0).stop 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 -- 172.21.15.113:0/3867789955 shutdown_connections 2024-04-24T23:58:05.235 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 --2- 172.21.15.113:0/3867789955 >> v2:172.21.15.178:3300/0 conn(0x7fcfb80d0560 0x7fcfb80720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 --2- 172.21.15.113:0/3867789955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80b3bd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 -- 172.21.15.113:0/3867789955 >> 172.21.15.113:0/3867789955 conn(0x7fcfb806ea80 msgr2=0x7fcfb806b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 -- 172.21.15.113:0/3867789955 shutdown_connections 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.232+0000 7fcfbf361640 1 -- 172.21.15.113:0/3867789955 wait complete. 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.233+0000 7fcfbf361640 1 Processor -- start 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.233+0000 7fcfbf361640 1 -- start start 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbf361640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb8071c60 0x7fcfb81f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbf361640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb80d0560 0x7fcfb814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbf361640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbf361640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb815ae30 con 0x7fcfb8071c60 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbf361640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb815acb0 con 0x7fcfb80d0560 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbf361640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb80b1c50 con 0x7fcfb815a8d0 2024-04-24T23:58:05.236 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb8071c60 0x7fcfb81f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb8071c60 0x7fcfb81f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40168/0 (socket says 172.21.15.113:40168) 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/2283256591 learned_addr learned my addr 172.21.15.113:0/2283256591 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd8fe640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbc8fc640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb80d0560 0x7fcfb814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 msgr2=0x7fcfb80ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80ab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb80d0560 msgr2=0x7fcfb814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb80d0560 0x7fcfb814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 -- 172.21.15.113:0/2283256591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb80ac200 con 0x7fcfb8071c60 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbd0fd640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb8071c60 0x7fcfb81f1170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcfb002a290 tx=0x7fcfb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfb5ffb640 1 -- 172.21.15.113:0/2283256591 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0041020 con 0x7fcfb8071c60 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfb5ffb640 1 -- 172.21.15.113:0/2283256591 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb003d620 con 0x7fcfb8071c60 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfb5ffb640 1 -- 172.21.15.113:0/2283256591 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0037040 con 0x7fcfb8071c60 2024-04-24T23:58:05.237 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.234+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb80ac4e0 con 0x7fcfb8071c60 2024-04-24T23:58:05.238 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.235+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb80ad4a0 con 0x7fcfb8071c60 2024-04-24T23:58:05.238 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.235+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb80acfd0 con 0x7fcfb8071c60 2024-04-24T23:58:05.238 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.236+0000 7fcfb5ffb640 1 -- 172.21.15.113:0/2283256591 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fcfb0044020 con 0x7fcfb8071c60 2024-04-24T23:58:05.238 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.236+0000 7fcfb5ffb640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfa406bda0 0x7fcfa406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:05.238 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.236+0000 7fcfbc8fc640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfa406bda0 0x7fcfa406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:05.239 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.236+0000 7fcfb5ffb640 1 -- 172.21.15.113:0/2283256591 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fcfb00aa540 con 0x7fcfb8071c60 2024-04-24T23:58:05.239 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.237+0000 7fcfbc8fc640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfa406bda0 0x7fcfa406e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcfac007fd0 tx=0x7fcfac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:05.240 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.241+0000 7fcfb5ffb640 1 -- 172.21.15.113:0/2283256591 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcfb004b030 con 0x7fcfb8071c60 2024-04-24T23:58:05.413 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.413+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfb804f570 con 0x7fcfb8071c60 2024-04-24T23:58:05.414 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.414+0000 7fcfb5ffb640 1 -- 172.21.15.113:0/2283256591 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fcfb0079d60 con 0x7fcfb8071c60 2024-04-24T23:58:05.414 INFO:teuthology.misc.health.smithi113.stdout: 2024-04-24T23:58:05.414 INFO:teuthology.misc.health.smithi113.stdout:{"epoch":12,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:03.119042+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:05.416 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfa406bda0 msgr2=0x7fcfa406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:05.416 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfa406bda0 0x7fcfa406e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcfac007fd0 tx=0x7fcfac009040 comp rx=0 tx=0).stop 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb8071c60 msgr2=0x7fcfb81f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb8071c60 0x7fcfb81f1170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcfb002a290 tx=0x7fcfb00046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 shutdown_connections 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfa406bda0 0x7fcfa406e260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb80d0560 0x7fcfb814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfb815a8d0 0x7fcfb80ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 --2- 172.21.15.113:0/2283256591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb8071c60 0x7fcfb81f1170 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 >> 172.21.15.113:0/2283256591 conn(0x7fcfb806ea80 msgr2=0x7fcfb806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 shutdown_connections 2024-04-24T23:58:05.417 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:05.416+0000 7fcfbf361640 1 -- 172.21.15.113:0/2283256591 wait complete. 2024-04-24T23:58:05.426 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T23:58:12.232 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:12.333 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.333+0000 7f4f75ebd640 1 Processor -- start 2024-04-24T23:58:12.333 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.333+0000 7f4f75ebd640 1 -- start start 2024-04-24T23:58:12.334 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f75ebd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4f7013f260 0x7f4f7015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.334 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f75ebd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f700cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.334 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f75ebd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7013f640 con 0x7f4f7015d510 2024-04-24T23:58:12.334 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f75ebd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4f70141cb0 con 0x7f4f7013f260 2024-04-24T23:58:12.334 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f75ebd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4f70141f90 con 0x7f4f700b22f0 2024-04-24T23:58:12.335 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f6f7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4f7013f260 0x7f4f7015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.335 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f6ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f700cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.335 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f6ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f700cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:44018/0 (socket says 172.21.15.113:44018) 2024-04-24T23:58:12.335 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.334+0000 7f4f6ffff640 1 -- 172.21.15.113:0/2372799079 learned_addr learned my addr 172.21.15.113:0/2372799079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.335+0000 7f4f6ffff640 1 -- 172.21.15.113:0/2372799079 >> v2:172.21.15.178:3300/0 conn(0x7f4f7013f260 msgr2=0x7f4f7015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.335+0000 7f4f6ffff640 1 --2- 172.21.15.113:0/2372799079 >> v2:172.21.15.178:3300/0 conn(0x7f4f7013f260 0x7f4f7015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.335+0000 7f4f6ffff640 1 -- 172.21.15.113:0/2372799079 >> v1:172.21.15.178:6789/0 conn(0x7f4f7015d510 legacy=0x7f4f700b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.335+0000 7f4f6ffff640 1 -- 172.21.15.113:0/2372799079 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f70142270 con 0x7f4f700b22f0 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.335+0000 7f4f6f7fe640 1 --2- 172.21.15.113:0/2372799079 >> v2:172.21.15.178:3300/0 conn(0x7f4f7013f260 0x7f4f7015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.336+0000 7f4f6ffff640 1 --2- 172.21.15.113:0/2372799079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f700cea00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009e80 tx=0x7f4f6802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe8c0246b9e0c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.336+0000 7f4f6e7fc640 1 -- 172.21.15.113:0/2372799079 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68036070 con 0x7f4f700b22f0 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.336+0000 7f4f6e7fc640 1 -- 172.21.15.113:0/2372799079 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f68031600 con 0x7f4f700b22f0 2024-04-24T23:58:12.336 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.336+0000 7f4f6e7fc640 1 -- 172.21.15.113:0/2372799079 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680319d0 con 0x7f4f700b22f0 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 -- 172.21.15.113:0/2372799079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 msgr2=0x7f4f700cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/2372799079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f700cea00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009e80 tx=0x7f4f6802a750 comp rx=0 tx=0).stop 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 -- 172.21.15.113:0/2372799079 shutdown_connections 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/2372799079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f700cea00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/2372799079 >> v2:172.21.15.178:3300/0 conn(0x7f4f7013f260 0x7f4f7015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 -- 172.21.15.113:0/2372799079 >> 172.21.15.113:0/2372799079 conn(0x7f4f7006ea80 msgr2=0x7f4f7006b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 -- 172.21.15.113:0/2372799079 shutdown_connections 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.337+0000 7f4f75ebd640 1 -- 172.21.15.113:0/2372799079 wait complete. 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.338+0000 7f4f75ebd640 1 Processor -- start 2024-04-24T23:58:12.338 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.338+0000 7f4f75ebd640 1 -- start start 2024-04-24T23:58:12.340 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f75ebd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f70138c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.340 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f75ebd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f7013f260 0x7f4f700d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.340 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f75ebd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f7015d510 0x7f4f701f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.340 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f75ebd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4f70141a50 con 0x7f4f7013f260 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f75ebd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701418d0 con 0x7f4f7015d510 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f75ebd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7015cb40 con 0x7f4f700b22f0 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f7013f260 0x7f4f700d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f7013f260 0x7f4f700d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34482/0 (socket says 172.21.15.113:34482) 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 -- 172.21.15.113:0/125071955 learned_addr learned my addr 172.21.15.113:0/125071955 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6f7fe640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f70138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6ffff640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f7015d510 0x7f4f701f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 -- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 msgr2=0x7f4f70138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f70138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 -- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f7015d510 msgr2=0x7f4f701f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f7015d510 0x7f4f701f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 -- 172.21.15.113:0/125071955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700b3580 con 0x7f4f7013f260 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.339+0000 7f4f6effd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f7013f260 0x7f4f700d0580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c3c0 tx=0x7f4f6400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.341 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.340+0000 7f4f57fff640 1 -- 172.21.15.113:0/125071955 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f640052d0 con 0x7f4f7013f260 2024-04-24T23:58:12.342 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.340+0000 7f4f57fff640 1 -- 172.21.15.113:0/125071955 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f64005490 con 0x7f4f7013f260 2024-04-24T23:58:12.342 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.340+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f700b3860 con 0x7f4f7013f260 2024-04-24T23:58:12.342 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.340+0000 7f4f57fff640 1 -- 172.21.15.113:0/125071955 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64007260 con 0x7f4f7013f260 2024-04-24T23:58:12.342 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.340+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f700b3e20 con 0x7f4f7013f260 2024-04-24T23:58:12.342 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.341+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7004f570 con 0x7f4f7013f260 2024-04-24T23:58:12.343 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.341+0000 7f4f57fff640 1 -- 172.21.15.113:0/125071955 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f64007400 con 0x7f4f7013f260 2024-04-24T23:58:12.343 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.342+0000 7f4f57fff640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4f5c06be30 0x7f4f5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.343 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.342+0000 7f4f6f7fe640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4f5c06be30 0x7f4f5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.343 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.342+0000 7f4f57fff640 1 -- 172.21.15.113:0/125071955 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4f64089f40 con 0x7f4f7013f260 2024-04-24T23:58:12.343 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.342+0000 7f4f6f7fe640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4f5c06be30 0x7f4f5c06e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009560 tx=0x7f4f60009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.346 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.346+0000 7f4f57fff640 1 -- 172.21.15.113:0/125071955 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f640585f0 con 0x7f4f7013f260 2024-04-24T23:58:12.520 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.520+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f700735b0 con 0x7f4f7013f260 2024-04-24T23:58:12.521 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.521+0000 7f4f57fff640 1 -- 172.21.15.113:0/125071955 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f4f6405c8f0 con 0x7f4f7013f260 2024-04-24T23:58:12.521 INFO:teuthology.misc.health.smithi113.stdout: 2024-04-24T23:58:12.521 INFO:teuthology.misc.health.smithi113.stdout:{"epoch":16,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:09.159828+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:12.523 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4f5c06be30 msgr2=0x7f4f5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.523 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4f5c06be30 0x7f4f5c06e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009560 tx=0x7f4f60009190 comp rx=0 tx=0).stop 2024-04-24T23:58:12.523 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f7013f260 msgr2=0x7f4f700d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.523 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f7013f260 0x7f4f700d0580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c3c0 tx=0x7f4f6400c880 comp rx=0 tx=0).stop 2024-04-24T23:58:12.523 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 shutdown_connections 2024-04-24T23:58:12.523 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4f5c06be30 0x7f4f5c06e2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.523 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f7015d510 0x7f4f701f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.524 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f7013f260 0x7f4f700d0580 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.524 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 --2- 172.21.15.113:0/125071955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f700b22f0 0x7f4f70138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.524 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 >> 172.21.15.113:0/125071955 conn(0x7f4f7006ea80 msgr2=0x7f4f7013f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:12.524 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 shutdown_connections 2024-04-24T23:58:12.524 INFO:teuthology.misc.health.smithi113.stderr:2024-04-24T23:58:12.523+0000 7f4f75ebd640 1 -- 172.21.15.113:0/125071955 wait complete. 2024-04-24T23:58:12.533 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T23:58:12.533 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T23:58:12.533 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:58:12.533 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:58:12.535 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:58:12.536 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:58:12.536 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:58:12.538 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:58:12.538 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:58:12.538 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:58:12.540 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:58:12.541 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:58:12.541 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:58:12.543 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:58:12.543 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-24T23:58:12.543 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:12.609 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-24T23:58:12.609 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-24T23:58:12.624 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-24T23:58:12.625 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-24T23:58:12.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.644+0000 7f6232c3f640 1 Processor -- start 2024-04-24T23:58:12.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.644+0000 7f6232c3f640 1 -- start start 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6232c3f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f622c0cea50 0x7f622c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6232c3f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f622c0cb4d0 0x7f622c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6232c3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cee30 0x7f622c15ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6232c3f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f622c06ee90 con 0x7f622c0cb4d0 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6232c3f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f622c06b0f0 con 0x7f622c0cea50 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6232c3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f622c06f8e0 con 0x7f622c0cee30 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f622c0cb4d0 0x7f622c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f622c0cb4d0 0x7f622c06e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34520/0 (socket says 172.21.15.113:34520) 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 -- 172.21.15.113:0/210272083 learned_addr learned my addr 172.21.15.113:0/210272083 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f62309db640 1 --2- 172.21.15.113:0/210272083 >> v2:172.21.15.178:3300/0 conn(0x7f622c0cea50 0x7f622c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 -- 172.21.15.113:0/210272083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cee30 msgr2=0x7f622c15ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f62311dc640 1 --2- 172.21.15.113:0/210272083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cee30 0x7f622c15ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 --2- 172.21.15.113:0/210272083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cee30 0x7f622c15ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 -- 172.21.15.113:0/210272083 >> v2:172.21.15.178:3300/0 conn(0x7f622c0cea50 msgr2=0x7f622c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 --2- 172.21.15.113:0/210272083 >> v2:172.21.15.178:3300/0 conn(0x7f622c0cea50 0x7f622c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f6223fff640 1 -- 172.21.15.113:0/210272083 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c06fbc0 con 0x7f622c0cb4d0 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.645+0000 7f62309db640 1 --2- 172.21.15.113:0/210272083 >> v2:172.21.15.178:3300/0 conn(0x7f622c0cea50 0x7f622c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.646+0000 7f6223fff640 1 --2- 172.21.15.113:0/210272083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c06e950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f622c0504b0 tx=0x7f621c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5861198abc24e46f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.646+0000 7f62237fe640 1 -- 172.21.15.113:0/210272083 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c037070 con 0x7f622c0cb4d0 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.646+0000 7f62237fe640 1 -- 172.21.15.113:0/210272083 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c032440 con 0x7f622c0cb4d0 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.646+0000 7f62237fe640 1 -- 172.21.15.113:0/210272083 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c032810 con 0x7f622c0cb4d0 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 -- 172.21.15.113:0/210272083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 msgr2=0x7f622c06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 --2- 172.21.15.113:0/210272083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c06e950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f622c0504b0 tx=0x7f621c02a140 comp rx=0 tx=0).stop 2024-04-24T23:58:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 -- 172.21.15.113:0/210272083 shutdown_connections 2024-04-24T23:58:12.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 --2- 172.21.15.113:0/210272083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cee30 0x7f622c15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 --2- 172.21.15.113:0/210272083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c06e950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 --2- 172.21.15.113:0/210272083 >> v2:172.21.15.178:3300/0 conn(0x7f622c0cea50 0x7f622c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 -- 172.21.15.113:0/210272083 >> 172.21.15.113:0/210272083 conn(0x7f622c0cd900 msgr2=0x7f622c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:12.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 -- 172.21.15.113:0/210272083 shutdown_connections 2024-04-24T23:58:12.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.647+0000 7f6232c3f640 1 -- 172.21.15.113:0/210272083 wait complete. 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.648+0000 7f6232c3f640 1 Processor -- start 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.648+0000 7f6232c3f640 1 -- start start 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6232c3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c0c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6232c3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cea50 0x7f622c0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6232c3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0cee30 0x7f622c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6232c3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f622c0cf2e0 con 0x7f622c0cb4d0 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6232c3f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f622c06b620 con 0x7f622c0cee30 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6232c3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f622c15e6e0 con 0x7f622c0cea50 2024-04-24T23:58:12.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c0c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c0c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34522/0 (socket says 172.21.15.113:34522) 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 -- 172.21.15.113:0/3152039028 learned_addr learned my addr 172.21.15.113:0/3152039028 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6223fff640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cea50 0x7f622c0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62311dc640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0cee30 0x7f622c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 -- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cea50 msgr2=0x7f622c0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cea50 0x7f622c0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 -- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0cee30 msgr2=0x7f622c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0cee30 0x7f622c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 -- 172.21.15.113:0/3152039028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c0c5910 con 0x7f622c0cb4d0 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6223fff640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cea50 0x7f622c0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62309db640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c0c74b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f622800c7f0 tx=0x7f622800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62217fa640 1 -- 172.21.15.113:0/3152039028 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228015070 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62217fa640 1 -- 172.21.15.113:0/3152039028 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6228007a30 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f62217fa640 1 -- 172.21.15.113:0/3152039028 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228007e00 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.649+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622c0c5bf0 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.650+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622c0c61b0 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.650+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622c068220 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.651+0000 7f62217fa640 1 -- 172.21.15.113:0/3152039028 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6228018070 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.651+0000 7f62217fa640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f621806be30 0x7f621806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.651+0000 7f62217fa640 1 -- 172.21.15.113:0/3152039028 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f622808aa70 con 0x7f622c0cb4d0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.651+0000 7f6223fff640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f621806be30 0x7f621806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.652+0000 7f6223fff640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f621806be30 0x7f621806e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f621c034000 tx=0x7f621c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.656+0000 7f62217fa640 1 -- 172.21.15.113:0/3152039028 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6228059800 con 0x7f622c0cb4d0 2024-04-24T23:58:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.828+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f622c0ce560 con 0x7f622c0cb4d0 2024-04-24T23:58:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.829+0000 7f62217fa640 1 -- 172.21.15.113:0/3152039028 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f6228059100 con 0x7f622c0cb4d0 2024-04-24T23:58:12.829 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:12.829 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":16,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:09.159828+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f621806be30 msgr2=0x7f621806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f621806be30 0x7f621806e2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f621c034000 tx=0x7f621c02e000 comp rx=0 tx=0).stop 2024-04-24T23:58:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 msgr2=0x7f622c0c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c0c74b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f622800c7f0 tx=0x7f622800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 shutdown_connections 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f621806be30 0x7f621806e2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0cee30 0x7f622c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622c0cea50 0x7f622c0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 --2- 172.21.15.113:0/3152039028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c0cb4d0 0x7f622c0c74b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 >> 172.21.15.113:0/3152039028 conn(0x7f622c0cd900 msgr2=0x7f622c0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 shutdown_connections 2024-04-24T23:58:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.831+0000 7f6232c3f640 1 -- 172.21.15.113:0/3152039028 wait complete. 2024-04-24T23:58:12.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:12.842 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:12.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.940+0000 7fd2b7e1c640 1 Processor -- start 2024-04-24T23:58:12.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.941+0000 7fd2b7e1c640 1 -- start start 2024-04-24T23:58:12.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.941+0000 7fd2b7e1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 0x7fd2b00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.941+0000 7fd2b7e1c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b015faf0 con 0x7fd2b015e430 2024-04-24T23:58:12.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.941+0000 7fd2b7e1c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b015fdd0 con 0x7fd2b00c9b10 2024-04-24T23:58:12.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.941+0000 7fd2b7e1c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b01600b0 con 0x7fd2b0071df0 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b5bb8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd2b015e430 0x7fd2b00716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58208/0 (socket says 172.21.15.113:58208) 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b5bb8640 1 -- 172.21.15.113:0/4007901403 learned_addr learned my addr 172.21.15.113:0/4007901403 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b53b7640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 0x7fd2b00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607790709 0 0) 0x7fd2b015faf0 con 0x7fd2b015e430 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd29c003520 con 0x7fd2b015e430 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 283820932 0 0) 0x7fd2b015fdd0 con 0x7fd2b00c9b10 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2b015faf0 con 0x7fd2b00c9b10 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3257868497 0 0) 0x7fd29c003520 con 0x7fd2b015e430 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 msgr2=0x7fd2b00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 0x7fd2b00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 >> v1:172.21.15.178:6789/0 conn(0x7fd2b00c9b10 legacy=0x7fd2b0160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b0160390 con 0x7fd2b015e430 2024-04-24T23:58:12.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fd2a40015f0 con 0x7fd2b015e430 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b53b7640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 0x7fd2b00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 >> v1:172.21.15.113:6789/0 conn(0x7fd2b015e430 legacy=0x7fd2b00716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd29c003b70 0x7fd29c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 0x7fd29c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c004d40 0x7fd29c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01600b0 con 0x7fd29c003b70 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c00ba70 con 0x7fd29c004d40 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd29c00bc70 con 0x7fd2b0071df0 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b5bb8640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd29c003b70 0x7fd29c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b53b7640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 0x7fd29c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.942+0000 7fd2b63b9640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c004d40 0x7fd29c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b5bb8640 1 -- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 msgr2=0x7fd29c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b5bb8640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2b0071df0 0x7fd29c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b5bb8640 1 -- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c004d40 msgr2=0x7fd29c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b5bb8640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c004d40 0x7fd29c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b5bb8640 1 -- 172.21.15.113:0/4007901403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c003520 con 0x7fd29c003b70 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b63b9640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c004d40 0x7fd29c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b5bb8640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd29c003b70 0x7fd29c003f70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4007f40 tx=0x7fd29c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea759593f2d649 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4007b60 con 0x7fd29c003b70 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a4005bb0 con 0x7fd29c003b70 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.943+0000 7fd2b4bb6640 1 -- 172.21.15.113:0/4007901403 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4007070 con 0x7fd29c003b70 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.944+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/4007901403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd29c003b70 msgr2=0x7fd29c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.944+0000 7fd2b7e1c640 1 --2- 172.21.15.113:0/4007901403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd29c003b70 0x7fd29c003f70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4007f40 tx=0x7fd29c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.944+0000 7fd2b63b9640 1 -- 172.21.15.113:0/4007901403 reap_dead start 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.944+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/4007901403 shutdown_connections 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.944+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/4007901403 >> 172.21.15.113:0/4007901403 conn(0x7fd2b006fb60 msgr2=0x7fd2b006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:12.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.944+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/4007901403 shutdown_connections 2024-04-24T23:58:12.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.944+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/4007901403 wait complete. 2024-04-24T23:58:12.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.945+0000 7fd2b7e1c640 1 Processor -- start 2024-04-24T23:58:12.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.945+0000 7fd2b7e1c640 1 -- start start 2024-04-24T23:58:12.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b7e1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd29c003b70 0x7fd2b0146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b7e1c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b015bdc0 0x7fd2b015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b7e1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b01f4570 0x7fd2b01f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b7e1c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b00d00e0 con 0x7fd2b015bdc0 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b7e1c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b0160950 con 0x7fd2b01f4570 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b7e1c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b00d0260 con 0x7fd29c003b70 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b015bdc0 0x7fd2b015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b015bdc0 0x7fd2b015c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34538/0 (socket says 172.21.15.113:34538) 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 -- 172.21.15.113:0/1825480416 learned_addr learned my addr 172.21.15.113:0/1825480416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b5bb8640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd29c003b70 0x7fd2b0146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 -- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd29c003b70 msgr2=0x7fd2b0146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b63b9640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b01f4570 0x7fd2b01f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd29c003b70 0x7fd2b0146190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 -- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b01f4570 msgr2=0x7fd2b01f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:12.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b01f4570 0x7fd2b01f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 -- 172.21.15.113:0/1825480416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b01466d0 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b53b7640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b015bdc0 0x7fd2b015c240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac011a00 tx=0x7fd2ac011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2a27fc640 1 -- 172.21.15.113:0/1825480416 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00faf0 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2a27fc640 1 -- 172.21.15.113:0/1825480416 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac00fc90 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2a27fc640 1 -- 172.21.15.113:0/1825480416 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac004030 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.946+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b0146950 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.947+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b0146f10 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.947+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b015c780 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.948+0000 7fd2a27fc640 1 -- 172.21.15.113:0/1825480416 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2ac012040 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.948+0000 7fd2a27fc640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd29406be30 0x7fd29406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.948+0000 7fd2a27fc640 1 -- 172.21.15.113:0/1825480416 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd2ac08ae90 con 0x7fd2b015bdc0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.948+0000 7fd2b5bb8640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd29406be30 0x7fd29406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:12.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.949+0000 7fd2b5bb8640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd29406be30 0x7fd29406e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4032000 tx=0x7fd2a4009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:12.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:12.952+0000 7fd2a27fc640 1 -- 172.21.15.113:0/1825480416 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2b01466d0 con 0x7fd2b015bdc0 2024-04-24T23:58:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.126+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2b00691a0 con 0x7fd2b015bdc0 2024-04-24T23:58:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.126+0000 7fd2a27fc640 1 -- 172.21.15.113:0/1825480416 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fd2ac019070 con 0x7fd2b015bdc0 2024-04-24T23:58:13.126 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd29406be30 msgr2=0x7fd29406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd29406be30 0x7fd29406e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4032000 tx=0x7fd2a4009030 comp rx=0 tx=0).stop 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b015bdc0 msgr2=0x7fd2b015c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b015bdc0 0x7fd2b015c240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac011a00 tx=0x7fd2ac011ec0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 shutdown_connections 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd29406be30 0x7fd29406e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b01f4570 0x7fd2b01f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b015bdc0 0x7fd2b015c240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 --2- 172.21.15.113:0/1825480416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd29c003b70 0x7fd2b0146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 >> 172.21.15.113:0/1825480416 conn(0x7fd2b006fb60 msgr2=0x7fd2b00c9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 shutdown_connections 2024-04-24T23:58:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.128+0000 7fd2b7e1c640 1 -- 172.21.15.113:0/1825480416 wait complete. 2024-04-24T23:58:13.139 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:13.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.235+0000 7fd81095a640 1 Processor -- start 2024-04-24T23:58:13.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.235+0000 7fd81095a640 1 -- start start 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.236+0000 7fd81095a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.236+0000 7fd81095a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd80c0cf360 0x7fd80c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.236+0000 7fd81095a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd80c06b8a0 con 0x7fd80c0b3290 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.236+0000 7fd81095a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd80c06b670 con 0x7fd80c0cf360 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.236+0000 7fd81095a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0d0f20 con 0x7fd80c0d2390 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd80ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd80c0cf360 0x7fd80c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd809d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd80a59c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd80c0b3290 0x7fd80c0d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:51116/0 (socket says 172.21.15.113:51116) 2024-04-24T23:58:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd80a59c640 1 -- 172.21.15.113:0/2127490054 learned_addr learned my addr 172.21.15.113:0/2127490054 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd80ad9d640 1 -- 172.21.15.113:0/2127490054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 msgr2=0x7fd80c06c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd80ad9d640 1 --2- 172.21.15.113:0/2127490054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c06c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd80ad9d640 1 -- 172.21.15.113:0/2127490054 >> v1:172.21.15.178:6789/0 conn(0x7fd80c0b3290 legacy=0x7fd80c0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.237+0000 7fd80ad9d640 1 -- 172.21.15.113:0/2127490054 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c06bb80 con 0x7fd80c0cf360 2024-04-24T23:58:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd80ad9d640 1 --2- 172.21.15.113:0/2127490054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0cf360 0x7fd80c0d0820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8040097a0 tx=0x7fd80402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d684788bcbfde900 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd80959a640 1 -- 172.21.15.113:0/2127490054 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804037070 con 0x7fd80c0cf360 2024-04-24T23:58:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd80959a640 1 -- 172.21.15.113:0/2127490054 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd804032440 con 0x7fd80c0cf360 2024-04-24T23:58:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd80959a640 1 -- 172.21.15.113:0/2127490054 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804032810 con 0x7fd80c0cf360 2024-04-24T23:58:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd81095a640 1 -- 172.21.15.113:0/2127490054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0cf360 msgr2=0x7fd80c0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd81095a640 1 --2- 172.21.15.113:0/2127490054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0cf360 0x7fd80c0d0820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8040097a0 tx=0x7fd80402a180 comp rx=0 tx=0).stop 2024-04-24T23:58:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd81095a640 1 -- 172.21.15.113:0/2127490054 shutdown_connections 2024-04-24T23:58:13.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd81095a640 1 --2- 172.21.15.113:0/2127490054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0cf360 0x7fd80c0d0820 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd81095a640 1 --2- 172.21.15.113:0/2127490054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c06c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd81095a640 1 -- 172.21.15.113:0/2127490054 >> 172.21.15.113:0/2127490054 conn(0x7fd80c09aa20 msgr2=0x7fd80c09ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:13.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.238+0000 7fd81095a640 1 -- 172.21.15.113:0/2127490054 shutdown_connections 2024-04-24T23:58:13.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.239+0000 7fd81095a640 1 -- 172.21.15.113:0/2127490054 wait complete. 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 Processor -- start 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 -- start start 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0b3290 0x7fd80c0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c0cf360 0x7fd80c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c0b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0b3670 con 0x7fd80c0cf360 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c06c1d0 con 0x7fd80c0b3290 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.240+0000 7fd81095a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0b37f0 con 0x7fd80c0d2390 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c0cf360 0x7fd80c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c0cf360 0x7fd80c0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34552/0 (socket says 172.21.15.113:34552) 2024-04-24T23:58:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 -- 172.21.15.113:0/302632270 learned_addr learned my addr 172.21.15.113:0/302632270 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd80a59c640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0b3290 0x7fd80c0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd80ad9d640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c0b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 -- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 msgr2=0x7fd80c0b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c0b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 -- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0b3290 msgr2=0x7fd80c0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0b3290 0x7fd80c0b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 -- 172.21.15.113:0/302632270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0b6c60 con 0x7fd80c0cf360 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd809d9b640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c0cf360 0x7fd80c0cb860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e9b0 tx=0x7fd80000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd7faffd640 1 -- 172.21.15.113:0/302632270 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800017070 con 0x7fd80c0cf360 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd7faffd640 1 -- 172.21.15.113:0/302632270 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80000c9a0 con 0x7fd80c0cf360 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c0b6f40 con 0x7fd80c0cf360 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd7faffd640 1 -- 172.21.15.113:0/302632270 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000cd90 con 0x7fd80c0cf360 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.241+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c0b7500 con 0x7fd80c0cf360 2024-04-24T23:58:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.242+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c068220 con 0x7fd80c0cf360 2024-04-24T23:58:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.243+0000 7fd7faffd640 1 -- 172.21.15.113:0/302632270 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd80001a070 con 0x7fd80c0cf360 2024-04-24T23:58:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.243+0000 7fd7faffd640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.243+0000 7fd7faffd640 1 -- 172.21.15.113:0/302632270 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd80008ae50 con 0x7fd80c0cf360 2024-04-24T23:58:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.243+0000 7fd80a59c640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.243+0000 7fd80a59c640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0012e0 tx=0x7fd7fc001990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:13.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.247+0000 7fd7faffd640 1 -- 172.21.15.113:0/302632270 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd800059500 con 0x7fd80c0cf360 2024-04-24T23:58:13.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.458+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd80c0d2dc0 con 0x7fd80c0cf360 2024-04-24T23:58:13.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.458+0000 7fd7faffd640 1 -- 172.21.15.113:0/302632270 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fd80005d800 con 0x7fd80c0cf360 2024-04-24T23:58:13.459 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:13.459 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:58:13.459 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 2 2024-04-24T23:58:13.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7ec06be30 msgr2=0x7fd7ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0012e0 tx=0x7fd7fc001990 comp rx=0 tx=0).stop 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c0cf360 msgr2=0x7fd80c0cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c0cf360 0x7fd80c0cb860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e9b0 tx=0x7fd80000ee70 comp rx=0 tx=0).stop 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 shutdown_connections 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd80c0d2390 0x7fd80c0b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c0cf360 0x7fd80c0cb860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 --2- 172.21.15.113:0/302632270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd80c0b3290 0x7fd80c0b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 >> 172.21.15.113:0/302632270 conn(0x7fd80c09aa20 msgr2=0x7fd80c0ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 shutdown_connections 2024-04-24T23:58:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:13.460+0000 7fd81095a640 1 -- 172.21.15.113:0/302632270 wait complete. 2024-04-24T23:58:15.472 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:15.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.579+0000 7f05caa09640 1 Processor -- start 2024-04-24T23:58:15.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.579+0000 7f05caa09640 1 -- start start 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05caa09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05c406e6e0 0x7f05c4169160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05caa09640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c40b3c20 con 0x7f05c40b5620 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05caa09640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c415e620 con 0x7f05c415e240 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05caa09640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f05c4072d90 con 0x7f05c406e6e0 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05c8fa6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05c406e6e0 0x7f05c4169160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05c37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f05c40b5620 0x7f05c406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58216/0 (socket says 172.21.15.113:58216) 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05c37fe640 1 -- 172.21.15.113:0/745696973 learned_addr learned my addr 172.21.15.113:0/745696973 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05c8fa6640 1 -- 172.21.15.113:0/745696973 >> v1:172.21.15.178:6789/0 conn(0x7f05c415e240 legacy=0x7f05c41627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05c8fa6640 1 -- 172.21.15.113:0/745696973 >> v1:172.21.15.113:6789/0 conn(0x7f05c40b5620 legacy=0x7f05c406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.580+0000 7f05c8fa6640 1 -- 172.21.15.113:0/745696973 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c4072210 con 0x7f05c406e6e0 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05c8fa6640 1 --2- 172.21.15.113:0/745696973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c4169160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009d90 tx=0x7f05bc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c007e4e98c8b747 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05c2ffd640 1 -- 172.21.15.113:0/745696973 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc007740 con 0x7f05c406e6e0 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05c2ffd640 1 -- 172.21.15.113:0/745696973 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05bc0078e0 con 0x7f05c406e6e0 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05c2ffd640 1 -- 172.21.15.113:0/745696973 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc007cb0 con 0x7f05c406e6e0 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05caa09640 1 -- 172.21.15.113:0/745696973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 msgr2=0x7f05c4169160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05caa09640 1 --2- 172.21.15.113:0/745696973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c4169160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009d90 tx=0x7f05bc02a6a0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05caa09640 1 -- 172.21.15.113:0/745696973 shutdown_connections 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05caa09640 1 --2- 172.21.15.113:0/745696973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c4169160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05caa09640 1 -- 172.21.15.113:0/745696973 >> 172.21.15.113:0/745696973 conn(0x7f05c40b1a50 msgr2=0x7f05c40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05caa09640 1 -- 172.21.15.113:0/745696973 shutdown_connections 2024-04-24T23:58:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.581+0000 7f05caa09640 1 -- 172.21.15.113:0/745696973 wait complete. 2024-04-24T23:58:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 Processor -- start 2024-04-24T23:58:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 -- start start 2024-04-24T23:58:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c40b5620 0x7f05c4150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05c415e240 0x7f05c41609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f05c4072ab0 con 0x7f05c406e6e0 2024-04-24T23:58:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05c40727d0 con 0x7f05c40b5620 2024-04-24T23:58:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.583+0000 7f05caa09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f05c40ad890 con 0x7f05c415e240 2024-04-24T23:58:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c414d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34568/0 (socket says 172.21.15.113:34568) 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 -- 172.21.15.113:0/2266075261 learned_addr learned my addr 172.21.15.113:0/2266075261 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c37fe640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c40b5620 0x7f05c4150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c8fa6640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05c415e240 0x7f05c41609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 -- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05c415e240 msgr2=0x7f05c41609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05c415e240 0x7f05c41609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 -- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c40b5620 msgr2=0x7f05c4150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c40b5620 0x7f05c4150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 -- 172.21.15.113:0/2266075261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c415f940 con 0x7f05c406e6e0 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c8fa6640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05c415e240 0x7f05c41609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c37fe640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c40b5620 0x7f05c4150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c3fff640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c414d9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05b400b410 tx=0x7f05b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c0ff9640 1 -- 172.21.15.113:0/2266075261 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4015020 con 0x7f05c406e6e0 2024-04-24T23:58:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c0ff9640 1 -- 172.21.15.113:0/2266075261 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b4011830 con 0x7f05c406e6e0 2024-04-24T23:58:15.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05c0ff9640 1 -- 172.21.15.113:0/2266075261 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4011c20 con 0x7f05c406e6e0 2024-04-24T23:58:15.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c415fbe0 con 0x7f05c406e6e0 2024-04-24T23:58:15.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.584+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c4160480 con 0x7f05c406e6e0 2024-04-24T23:58:15.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.585+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c415fdb0 con 0x7f05c406e6e0 2024-04-24T23:58:15.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.586+0000 7f05c0ff9640 1 -- 172.21.15.113:0/2266075261 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05b4018020 con 0x7f05c406e6e0 2024-04-24T23:58:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.586+0000 7f05c0ff9640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f05b006be30 0x7f05b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.586+0000 7f05c37fe640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f05b006be30 0x7f05b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.586+0000 7f05c0ff9640 1 -- 172.21.15.113:0/2266075261 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f05b408a9e0 con 0x7f05c406e6e0 2024-04-24T23:58:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.587+0000 7f05c37fe640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f05b006be30 0x7f05b006e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05c415d9f0 tx=0x7f05b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.591+0000 7f05c0ff9640 1 -- 172.21.15.113:0/2266075261 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05b401f020 con 0x7f05c406e6e0 2024-04-24T23:58:15.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.799+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05c406f080 con 0x7f05c406e6e0 2024-04-24T23:58:15.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.799+0000 7f05c0ff9640 1 -- 172.21.15.113:0/2266075261 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7f05b40172c0 con 0x7f05c406e6e0 2024-04-24T23:58:15.800 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:15.800 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:58:15.801 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 3 2024-04-24T23:58:15.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f05b006be30 msgr2=0x7f05b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f05b006be30 0x7f05b006e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05c415d9f0 tx=0x7f05b8009040 comp rx=0 tx=0).stop 2024-04-24T23:58:15.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 msgr2=0x7f05c414d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c414d9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05b400b410 tx=0x7f05b400b8d0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 shutdown_connections 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f05b006be30 0x7f05b006e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c40b5620 0x7f05c4150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05c415e240 0x7f05c41609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 --2- 172.21.15.113:0/2266075261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c406e6e0 0x7f05c414d9a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 >> 172.21.15.113:0/2266075261 conn(0x7f05c40b1a50 msgr2=0x7f05c415b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 shutdown_connections 2024-04-24T23:58:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.801+0000 7f05caa09640 1 -- 172.21.15.113:0/2266075261 wait complete. 2024-04-24T23:58:15.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.907+0000 7fdd97396640 1 Processor -- start 2024-04-24T23:58:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.907+0000 7fdd97396640 1 -- start start 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd97396640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdd900b3fd0 0x7fdd9006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd97396640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd90144cf0 con 0x7fdd9006e510 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd97396640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd900b2f30 con 0x7fdd90144910 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd97396640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdd9006e980 con 0x7fdd900b3fd0 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd95933640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdd9006e510 0x7fdd900cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58220/0 (socket says 172.21.15.113:58220) 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd95933640 1 -- 172.21.15.113:0/4282537411 learned_addr learned my addr 172.21.15.113:0/4282537411 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd94931640 1 --2- 172.21.15.113:0/4282537411 >> v2:172.21.15.178:3300/0 conn(0x7fdd900b3fd0 0x7fdd9006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3833081876 0 0) 0x7fdd90144cf0 con 0x7fdd9006e510 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd7c005730 con 0x7fdd9006e510 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3223780590 0 0) 0x7fdd7c005730 con 0x7fdd9006e510 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 >> v2:172.21.15.178:3300/0 conn(0x7fdd900b3fd0 msgr2=0x7fdd9006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 --2- 172.21.15.113:0/4282537411 >> v2:172.21.15.178:3300/0 conn(0x7fdd900b3fd0 0x7fdd9006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 >> v1:172.21.15.178:6789/0 conn(0x7fdd90144910 legacy=0x7fdd900b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd900b3210 con 0x7fdd9006e510 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fdd880019f0 con 0x7fdd9006e510 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 >> v1:172.21.15.113:6789/0 conn(0x7fdd9006e510 legacy=0x7fdd900cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c005bf0 0x7fdd7c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd7c006a10 0x7fdd7c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0073d0 0x7fdd7c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9006e980 con 0x7fdd7c0073d0 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c00e2a0 con 0x7fdd7c005bf0 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c00e4a0 con 0x7fdd7c006a10 2024-04-24T23:58:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd95933640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0073d0 0x7fdd7c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.908+0000 7fdd95132640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c005bf0 0x7fdd7c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd94931640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd7c006a10 0x7fdd7c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd95933640 1 -- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd7c006a10 msgr2=0x7fdd7c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd95933640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd7c006a10 0x7fdd7c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd95933640 1 -- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c005bf0 msgr2=0x7fdd7c006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd95933640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c005bf0 0x7fdd7c006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd95933640 1 -- 172.21.15.113:0/4282537411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c005730 con 0x7fdd7c0073d0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd95933640 1 -- 172.21.15.113:0/4282537411 reap_dead start 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd95933640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0073d0 0x7fdd7c00dbd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800a840 tx=0x7fdd7c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54cbdb72d4684b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8800a460 con 0x7fdd7c0073d0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8802f070 con 0x7fdd7c0073d0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.909+0000 7fdd87fff640 1 -- 172.21.15.113:0/4282537411 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8802bcb0 con 0x7fdd7c0073d0 2024-04-24T23:58:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.910+0000 7fdd97396640 1 -- 172.21.15.113:0/4282537411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0073d0 msgr2=0x7fdd7c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.910+0000 7fdd97396640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0073d0 0x7fdd7c00dbd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800a840 tx=0x7fdd7c007a40 comp rx=0 tx=0).stop 2024-04-24T23:58:15.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.910+0000 7fdd97396640 1 -- 172.21.15.113:0/4282537411 shutdown_connections 2024-04-24T23:58:15.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.910+0000 7fdd97396640 1 --2- 172.21.15.113:0/4282537411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0073d0 0x7fdd7c00dbd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.910+0000 7fdd97396640 1 -- 172.21.15.113:0/4282537411 >> 172.21.15.113:0/4282537411 conn(0x7fdd900b1a50 msgr2=0x7fdd900b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:15.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.910+0000 7fdd97396640 1 -- 172.21.15.113:0/4282537411 shutdown_connections 2024-04-24T23:58:15.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.910+0000 7fdd97396640 1 -- 172.21.15.113:0/4282537411 wait complete. 2024-04-24T23:58:15.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.911+0000 7fdd97396640 1 Processor -- start 2024-04-24T23:58:15.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.911+0000 7fdd97396640 1 -- start start 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd97396640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c0073d0 0x7fdd9015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd97396640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90138fc0 0x7fdd90140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd97396640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd901394a0 0x7fdd90139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd97396640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd900b4440 con 0x7fdd90138fc0 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd97396640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd900b37d0 con 0x7fdd7c0073d0 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd97396640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd900b4da0 con 0x7fdd901394a0 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90138fc0 0x7fdd90140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90138fc0 0x7fdd90140410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34594/0 (socket says 172.21.15.113:34594) 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 -- 172.21.15.113:0/4260870563 learned_addr learned my addr 172.21.15.113:0/4260870563 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd95933640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd901394a0 0x7fdd90139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd95132640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c0073d0 0x7fdd9015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 -- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd901394a0 msgr2=0x7fdd90139930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd901394a0 0x7fdd90139930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 -- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c0073d0 msgr2=0x7fdd9015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c0073d0 0x7fdd9015d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 -- 172.21.15.113:0/4260870563 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd90140d60 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd94931640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90138fc0 0x7fdd90140410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd80011910 tx=0x7fdd80011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd85ffb640 1 -- 172.21.15.113:0/4260870563 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8000faf0 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.912+0000 7fdd85ffb640 1 -- 172.21.15.113:0/4260870563 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8000fc90 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.913+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd90141040 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.913+0000 7fdd85ffb640 1 -- 172.21.15.113:0/4260870563 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80014440 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.913+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd900adcf0 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.914+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd900691a0 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.914+0000 7fdd85ffb640 1 -- 172.21.15.113:0/4260870563 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd80012040 con 0x7fdd90138fc0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.914+0000 7fdd85ffb640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd7406be30 0x7fdd7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.915+0000 7fdd95132640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd7406be30 0x7fdd7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:15.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.915+0000 7fdd85ffb640 1 -- 172.21.15.113:0/4260870563 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdd8008b0d0 con 0x7fdd90138fc0 2024-04-24T23:58:15.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.915+0000 7fdd95132640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd7406be30 0x7fdd7406e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000c00 tx=0x7fdd8c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:15.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:15.919+0000 7fdd85ffb640 1 -- 172.21.15.113:0/4260870563 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd80019030 con 0x7fdd90138fc0 2024-04-24T23:58:16.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.090+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd900731a0 con 0x7fdd90138fc0 2024-04-24T23:58:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.091+0000 7fdd85ffb640 1 -- 172.21.15.113:0/4260870563 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7fdd80059e60 con 0x7fdd90138fc0 2024-04-24T23:58:16.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:16.092 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":16,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:09.159828+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.093+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd7406be30 msgr2=0x7fdd7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.093+0000 7fdd97396640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd7406be30 0x7fdd7406e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000c00 tx=0x7fdd8c009040 comp rx=0 tx=0).stop 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.093+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90138fc0 msgr2=0x7fdd90140410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.093+0000 7fdd97396640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90138fc0 0x7fdd90140410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd80011910 tx=0x7fdd80011dd0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 shutdown_connections 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd7406be30 0x7fdd7406e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd901394a0 0x7fdd90139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90138fc0 0x7fdd90140410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 --2- 172.21.15.113:0/4260870563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd7c0073d0 0x7fdd9015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 >> 172.21.15.113:0/4260870563 conn(0x7fdd900b1a50 msgr2=0x7fdd90071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:16.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 shutdown_connections 2024-04-24T23:58:16.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.094+0000 7fdd97396640 1 -- 172.21.15.113:0/4260870563 wait complete. 2024-04-24T23:58:16.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:16.105 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.204+0000 7fe74829e640 1 Processor -- start 2024-04-24T23:58:16.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.204+0000 7fe74829e640 1 -- start start 2024-04-24T23:58:16.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.205+0000 7fe74829e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe74015ba50 0x7fe74015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.205+0000 7fe74829e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74006cea0 0x7fe74006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.205+0000 7fe74829e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe74006d7e0 con 0x7fe74015b670 2024-04-24T23:58:16.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.205+0000 7fe74829e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe740071170 con 0x7fe74015ba50 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.205+0000 7fe74829e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe740071450 con 0x7fe74006cea0 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe745839640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe74015ba50 0x7fe74015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe745839640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe74015ba50 0x7fe74015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34602/0 (socket says 172.21.15.113:34602) 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe745839640 1 -- 172.21.15.113:0/1815362883 learned_addr learned my addr 172.21.15.113:0/1815362883 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe74683b640 1 --2- 172.21.15.113:0/1815362883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74006cea0 0x7fe74006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe745839640 1 -- 172.21.15.113:0/1815362883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74006cea0 msgr2=0x7fe74006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe745839640 1 --2- 172.21.15.113:0/1815362883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74006cea0 0x7fe74006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe745839640 1 -- 172.21.15.113:0/1815362883 >> v1:172.21.15.178:6789/0 conn(0x7fe74015b670 legacy=0x7fe7400cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe745839640 1 -- 172.21.15.113:0/1815362883 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe740071730 con 0x7fe74015ba50 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.206+0000 7fe74683b640 1 --2- 172.21.15.113:0/1815362883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74006cea0 0x7fe74006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe745839640 1 --2- 172.21.15.113:0/1815362883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015ba50 0x7fe74015bff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009870 tx=0x7fe73c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fb4bd7c873dc06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe745038640 1 -- 172.21.15.113:0/1815362883 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c037070 con 0x7fe74015ba50 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe745038640 1 -- 172.21.15.113:0/1815362883 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73c032440 con 0x7fe74015ba50 2024-04-24T23:58:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe745038640 1 -- 172.21.15.113:0/1815362883 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c032810 con 0x7fe74015ba50 2024-04-24T23:58:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 -- 172.21.15.113:0/1815362883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015ba50 msgr2=0x7fe74015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 --2- 172.21.15.113:0/1815362883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015ba50 0x7fe74015bff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009870 tx=0x7fe73c02a130 comp rx=0 tx=0).stop 2024-04-24T23:58:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 -- 172.21.15.113:0/1815362883 shutdown_connections 2024-04-24T23:58:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 --2- 172.21.15.113:0/1815362883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74006cea0 0x7fe74006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 --2- 172.21.15.113:0/1815362883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015ba50 0x7fe74015bff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 -- 172.21.15.113:0/1815362883 >> 172.21.15.113:0/1815362883 conn(0x7fe74006b0f0 msgr2=0x7fe74006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 -- 172.21.15.113:0/1815362883 shutdown_connections 2024-04-24T23:58:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.207+0000 7fe74829e640 1 -- 172.21.15.113:0/1815362883 wait complete. 2024-04-24T23:58:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.208+0000 7fe74829e640 1 Processor -- start 2024-04-24T23:58:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.208+0000 7fe74829e640 1 -- start start 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe74829e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe74006cea0 0x7fe740144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe74829e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015b670 0x7fe740154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe74829e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74015ba50 0x7fe74014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe74829e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7400cca90 con 0x7fe74015b670 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe74829e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7400cc910 con 0x7fe74006cea0 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe74829e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe7400d0080 con 0x7fe74015ba50 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015b670 0x7fe740154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015b670 0x7fe740154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34616/0 (socket says 172.21.15.113:34616) 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 -- 172.21.15.113:0/1189848082 learned_addr learned my addr 172.21.15.113:0/1189848082 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 -- 172.21.15.113:0/1189848082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74015ba50 msgr2=0x7fe74014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74015ba50 0x7fe74014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 -- 172.21.15.113:0/1189848082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe74006cea0 msgr2=0x7fe740144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe74006cea0 0x7fe740144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.209+0000 7fe745839640 1 -- 172.21.15.113:0/1189848082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe740155670 con 0x7fe74015b670 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.210+0000 7fe745839640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015b670 0x7fe740154780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000c00 tx=0x7fe73c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.210+0000 7fe736ffd640 1 -- 172.21.15.113:0/1189848082 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c041020 con 0x7fe74015b670 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.210+0000 7fe736ffd640 1 -- 172.21.15.113:0/1189848082 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73c0306c0 con 0x7fe74015b670 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.210+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe740155950 con 0x7fe74015b670 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.210+0000 7fe736ffd640 1 -- 172.21.15.113:0/1189848082 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c037040 con 0x7fe74015b670 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.210+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe740155f10 con 0x7fe74015b670 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.211+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7400691a0 con 0x7fe74015b670 2024-04-24T23:58:16.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.211+0000 7fe736ffd640 1 -- 172.21.15.113:0/1189848082 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe73c044020 con 0x7fe74015b670 2024-04-24T23:58:16.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.211+0000 7fe736ffd640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe72406be30 0x7fe72406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.212+0000 7fe736ffd640 1 -- 172.21.15.113:0/1189848082 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe73c0ab300 con 0x7fe74015b670 2024-04-24T23:58:16.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.212+0000 7fe74603a640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe72406be30 0x7fe72406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.212+0000 7fe74603a640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe72406be30 0x7fe72406e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe730000f60 tx=0x7fe7300079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.216+0000 7fe736ffd640 1 -- 172.21.15.113:0/1189848082 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe73c04b030 con 0x7fe74015b670 2024-04-24T23:58:16.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.389+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe7400d0830 con 0x7fe74015b670 2024-04-24T23:58:16.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.389+0000 7fe736ffd640 1 -- 172.21.15.113:0/1189848082 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fe73c07a090 con 0x7fe74015b670 2024-04-24T23:58:16.391 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:16.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe72406be30 msgr2=0x7fe72406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe72406be30 0x7fe72406e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe730000f60 tx=0x7fe7300079d0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015b670 msgr2=0x7fe740154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015b670 0x7fe740154780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000c00 tx=0x7fe73c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 shutdown_connections 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe72406be30 0x7fe72406e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe74015ba50 0x7fe74014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe74015b670 0x7fe740154780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 --2- 172.21.15.113:0/1189848082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe74006cea0 0x7fe740144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 >> 172.21.15.113:0/1189848082 conn(0x7fe74006b0f0 msgr2=0x7fe74015a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 shutdown_connections 2024-04-24T23:58:16.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.391+0000 7fe74829e640 1 -- 172.21.15.113:0/1189848082 wait complete. 2024-04-24T23:58:16.403 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:58:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.497+0000 7ff15cf4f640 1 Processor -- start 2024-04-24T23:58:16.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.497+0000 7ff15cf4f640 1 -- start start 2024-04-24T23:58:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.498+0000 7ff15cf4f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff15806e6e0 0x7ff1580cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.498+0000 7ff15cf4f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff158144cf0 con 0x7ff158144910 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.498+0000 7ff15cf4f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1580b2f30 con 0x7ff1580b3fd0 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.498+0000 7ff15cf4f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff1580b3210 con 0x7ff15806e6e0 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff158144910 0x7ff1580b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58230/0 (socket says 172.21.15.113:58230) 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15659c640 1 -- 172.21.15.113:0/217539657 learned_addr learned my addr 172.21.15.113:0/217539657 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352515947 0 0) 0x7ff158144cf0 con 0x7ff158144910 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff156d9d640 1 --2- 172.21.15.113:0/217539657 >> v2:172.21.15.178:3300/0 conn(0x7ff15806e6e0 0x7ff1580cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff140003520 con 0x7ff158144910 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2137235497 0 0) 0x7ff140003520 con 0x7ff158144910 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 >> v2:172.21.15.178:3300/0 conn(0x7ff15806e6e0 msgr2=0x7ff1580cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 --2- 172.21.15.113:0/217539657 >> v2:172.21.15.178:3300/0 conn(0x7ff15806e6e0 0x7ff1580cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 >> v1:172.21.15.178:6789/0 conn(0x7ff1580b3fd0 legacy=0x7ff15806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1580b34f0 con 0x7ff158144910 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff14c001670 con 0x7ff158144910 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 >> v1:172.21.15.113:6789/0 conn(0x7ff158144910 legacy=0x7ff1580b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1400039b0 0x7ff140003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff140004800 0x7ff140004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1400051c0 0x7ff14000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580b3210 con 0x7ff1400039b0 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff14000c090 con 0x7ff140004800 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff14000c290 con 0x7ff1400051c0 2024-04-24T23:58:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff15659c640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1400039b0 0x7ff140003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff155d9b640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff140004800 0x7ff140004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.499+0000 7ff156d9d640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1400051c0 0x7ff14000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15659c640 1 -- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1400051c0 msgr2=0x7ff14000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15659c640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1400051c0 0x7ff14000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15659c640 1 -- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff140004800 msgr2=0x7ff140004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15659c640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff140004800 0x7ff140004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15659c640 1 -- 172.21.15.113:0/217539657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff140003520 con 0x7ff1400039b0 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff156d9d640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1400051c0 0x7ff14000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff156d9d640 1 -- 172.21.15.113:0/217539657 reap_dead start 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15659c640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1400039b0 0x7ff140003dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c190 tx=0x7ff140005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e8d169c0784078 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c007ce0 con 0x7ff1400039b0 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14c005ad0 con 0x7ff1400039b0 2024-04-24T23:58:16.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.500+0000 7ff15559a640 1 -- 172.21.15.113:0/217539657 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c007070 con 0x7ff1400039b0 2024-04-24T23:58:16.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.501+0000 7ff15cf4f640 1 -- 172.21.15.113:0/217539657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1400039b0 msgr2=0x7ff140003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.501+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1400039b0 0x7ff140003dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c190 tx=0x7ff140005830 comp rx=0 tx=0).stop 2024-04-24T23:58:16.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.501+0000 7ff15cf4f640 1 -- 172.21.15.113:0/217539657 shutdown_connections 2024-04-24T23:58:16.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.501+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/217539657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1400039b0 0x7ff140003dd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.501+0000 7ff15cf4f640 1 -- 172.21.15.113:0/217539657 >> 172.21.15.113:0/217539657 conn(0x7ff1580b1a50 msgr2=0x7ff1580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:16.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.501+0000 7ff15cf4f640 1 -- 172.21.15.113:0/217539657 shutdown_connections 2024-04-24T23:58:16.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.501+0000 7ff15cf4f640 1 -- 172.21.15.113:0/217539657 wait complete. 2024-04-24T23:58:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.502+0000 7ff15cf4f640 1 Processor -- start 2024-04-24T23:58:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.502+0000 7ff15cf4f640 1 -- start start 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff15cf4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1400039b0 0x7ff15815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff15cf4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff15813ce00 0x7ff1580711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff15cf4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1580716e0 0x7ff15813eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff15cf4f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580b4da0 con 0x7ff1580716e0 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff15cf4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580b4440 con 0x7ff1400039b0 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff15cf4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff1580b4f20 con 0x7ff15813ce00 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1580716e0 0x7ff15813eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1580716e0 0x7ff15813eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34630/0 (socket says 172.21.15.113:34630) 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 -- 172.21.15.113:0/4060628383 learned_addr learned my addr 172.21.15.113:0/4060628383 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff155d9b640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff15813ce00 0x7ff1580711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff15659c640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1400039b0 0x7ff15815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 -- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff15813ce00 msgr2=0x7ff1580711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff15813ce00 0x7ff1580711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 -- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1400039b0 msgr2=0x7ff15815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1400039b0 0x7ff15815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 -- 172.21.15.113:0/4060628383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff158159530 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.503+0000 7ff156d9d640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1580716e0 0x7ff15813eea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff15806ed60 tx=0x7ff1500115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.504+0000 7ff146ffd640 1 -- 172.21.15.113:0/4060628383 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150011db0 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.504+0000 7ff146ffd640 1 -- 172.21.15.113:0/4060628383 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15001c070 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.504+0000 7ff146ffd640 1 -- 172.21.15.113:0/4060628383 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1500166b0 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.504+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff158159810 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.504+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff158159dd0 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.505+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1580691a0 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.505+0000 7ff146ffd640 1 -- 172.21.15.113:0/4060628383 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff150016850 con 0x7ff1580716e0 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.505+0000 7ff146ffd640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff13806be60 0x7ff13806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.505+0000 7ff146ffd640 1 -- 172.21.15.113:0/4060628383 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff15008b4f0 con 0x7ff1580716e0 2024-04-24T23:58:16.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.505+0000 7ff15659c640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff13806be60 0x7ff13806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.506+0000 7ff15659c640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff13806be60 0x7ff13806e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff14c032000 tx=0x7ff14c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.510+0000 7ff146ffd640 1 -- 172.21.15.113:0/4060628383 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff15005a1f0 con 0x7ff1580716e0 2024-04-24T23:58:16.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.684+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff1580731a0 con 0x7ff1580716e0 2024-04-24T23:58:16.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.684+0000 7ff146ffd640 1 -- 172.21.15.113:0/4060628383 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7ff150059af0 con 0x7ff1580716e0 2024-04-24T23:58:16.686 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:16.686 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-24T23:58:16.686 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:58:16.686 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-24T23:58:16.686 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-24T23:58:16.686 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:58:16.686 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.686+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff13806be60 msgr2=0x7ff13806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.686+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff13806be60 0x7ff13806e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff14c032000 tx=0x7ff14c009030 comp rx=0 tx=0).stop 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.686+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1580716e0 msgr2=0x7ff15813eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.686+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1580716e0 0x7ff15813eea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff15806ed60 tx=0x7ff1500115a0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 shutdown_connections 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff13806be60 0x7ff13806e320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1580716e0 0x7ff15813eea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff15813ce00 0x7ff1580711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 --2- 172.21.15.113:0/4060628383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1400039b0 0x7ff15815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 >> 172.21.15.113:0/4060628383 conn(0x7ff1580b1a50 msgr2=0x7ff1580cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:16.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 shutdown_connections 2024-04-24T23:58:16.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.687+0000 7ff15cf4f640 1 -- 172.21.15.113:0/4060628383 wait complete. 2024-04-24T23:58:16.698 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:58:16.698 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:58:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.797+0000 7fa8175a9640 1 Processor -- start 2024-04-24T23:58:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.797+0000 7fa8175a9640 1 -- start start 2024-04-24T23:58:16.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa8175a9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8100b9bb0 0x7fa8100b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa8175a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8100b61e0 0x7fa8100d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa8175a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8101626f0 con 0x7fa81015e510 2024-04-24T23:58:16.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa8175a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa8101629d0 con 0x7fa8100b61e0 2024-04-24T23:58:16.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa8175a9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa810162cb0 con 0x7fa8100b9bb0 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa815345640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa81015e510 0x7fa810161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58234/0 (socket says 172.21.15.113:58234) 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa815345640 1 -- 172.21.15.113:0/4181962383 learned_addr learned my addr 172.21.15.113:0/4181962383 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.798+0000 7fa815b46640 1 --2- 172.21.15.113:0/4181962383 >> v2:172.21.15.113:3300/0 conn(0x7fa8100b61e0 0x7fa8100d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa814b44640 1 --2- 172.21.15.113:0/4181962383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8100b9bb0 0x7fa8100b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa807fff640 1 -- 172.21.15.113:0/4181962383 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 981302990 0 0) 0x7fa8101626f0 con 0x7fa81015e510 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa807fff640 1 -- 172.21.15.113:0/4181962383 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7fc003520 con 0x7fa81015e510 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa815b46640 1 -- 172.21.15.113:0/4181962383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8100b9bb0 msgr2=0x7fa8100b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa815b46640 1 --2- 172.21.15.113:0/4181962383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8100b9bb0 0x7fa8100b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa815b46640 1 -- 172.21.15.113:0/4181962383 >> v1:172.21.15.113:6789/0 conn(0x7fa81015e510 legacy=0x7fa810161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa815b46640 1 -- 172.21.15.113:0/4181962383 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8100d2660 con 0x7fa8100b61e0 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa815b46640 1 --2- 172.21.15.113:0/4181962383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b61e0 0x7fa8100d2e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa8080097a0 tx=0x7fa80802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221d416ed4f15d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa807fff640 1 -- 172.21.15.113:0/4181962383 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808037070 con 0x7fa8100b61e0 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa807fff640 1 -- 172.21.15.113:0/4181962383 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80802ae50 con 0x7fa8100b61e0 2024-04-24T23:58:16.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.799+0000 7fa807fff640 1 -- 172.21.15.113:0/4181962383 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808032660 con 0x7fa8100b61e0 2024-04-24T23:58:16.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 -- 172.21.15.113:0/4181962383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b61e0 msgr2=0x7fa8100d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 --2- 172.21.15.113:0/4181962383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b61e0 0x7fa8100d2e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa8080097a0 tx=0x7fa80802a100 comp rx=0 tx=0).stop 2024-04-24T23:58:16.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 -- 172.21.15.113:0/4181962383 shutdown_connections 2024-04-24T23:58:16.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 --2- 172.21.15.113:0/4181962383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b61e0 0x7fa8100d2e60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 --2- 172.21.15.113:0/4181962383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8100b9bb0 0x7fa8100b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 -- 172.21.15.113:0/4181962383 >> 172.21.15.113:0/4181962383 conn(0x7fa81006b720 msgr2=0x7fa8100d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 -- 172.21.15.113:0/4181962383 shutdown_connections 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.800+0000 7fa8175a9640 1 -- 172.21.15.113:0/4181962383 wait complete. 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.801+0000 7fa8175a9640 1 Processor -- start 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.801+0000 7fa8175a9640 1 -- start start 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa8175a9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8100b61e0 0x7fa810153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa8175a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b9bb0 0x7fa8101f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa8175a9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81015e510 0x7fa8101f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa8175a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100b7540 con 0x7fa8100b9bb0 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa8175a9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100b73c0 con 0x7fa8100b61e0 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa8175a9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa8100d33a0 con 0x7fa81015e510 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b9bb0 0x7fa8101f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b9bb0 0x7fa8101f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34656/0 (socket says 172.21.15.113:34656) 2024-04-24T23:58:16.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 -- 172.21.15.113:0/3847866915 learned_addr learned my addr 172.21.15.113:0/3847866915 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 -- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81015e510 msgr2=0x7fa8101f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa815345640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8100b61e0 0x7fa810153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa815b46640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81015e510 0x7fa8101f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81015e510 0x7fa8101f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 -- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8100b61e0 msgr2=0x7fa810153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8100b61e0 0x7fa810153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 -- 172.21.15.113:0/3847866915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa810154050 con 0x7fa8100b9bb0 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa815b46640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81015e510 0x7fa8101f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa815345640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8100b61e0 0x7fa810153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.802+0000 7fa814b44640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b9bb0 0x7fa8101f4340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa80000d7d0 tx=0x7fa80000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.803+0000 7fa805ffb640 1 -- 172.21.15.113:0/3847866915 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800004020 con 0x7fa8100b9bb0 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.803+0000 7fa805ffb640 1 -- 172.21.15.113:0/3847866915 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8000041e0 con 0x7fa8100b9bb0 2024-04-24T23:58:16.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.803+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8100c8ad0 con 0x7fa8100b9bb0 2024-04-24T23:58:16.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.803+0000 7fa805ffb640 1 -- 172.21.15.113:0/3847866915 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000043d0 con 0x7fa8100b9bb0 2024-04-24T23:58:16.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.803+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8100c9090 con 0x7fa8100b9bb0 2024-04-24T23:58:16.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.804+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8100691a0 con 0x7fa8100b9bb0 2024-04-24T23:58:16.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.804+0000 7fa805ffb640 1 -- 172.21.15.113:0/3847866915 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa80001b020 con 0x7fa8100b9bb0 2024-04-24T23:58:16.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.805+0000 7fa805ffb640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7f406be30 0x7fa7f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:16.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.805+0000 7fa815345640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7f406be30 0x7fa7f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:16.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.805+0000 7fa805ffb640 1 -- 172.21.15.113:0/3847866915 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa80008a960 con 0x7fa8100b9bb0 2024-04-24T23:58:16.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.805+0000 7fa815345640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7f406be30 0x7fa7f406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005c00 tx=0x7fa80c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:16.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.809+0000 7fa805ffb640 1 -- 172.21.15.113:0/3847866915 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa810154050 con 0x7fa8100b9bb0 2024-04-24T23:58:16.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:16.981+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa81015b8c0 con 0x7fa8100b9bb0 2024-04-24T23:58:17.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.225+0000 7fa805ffb640 1 -- 172.21.15.113:0/3847866915 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fa800059660 con 0x7fa8100b9bb0 2024-04-24T23:58:17.226 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7f406be30 msgr2=0x7fa7f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7f406be30 0x7fa7f406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005c00 tx=0x7fa80c009040 comp rx=0 tx=0).stop 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b9bb0 msgr2=0x7fa8101f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b9bb0 0x7fa8101f4340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa80000d7d0 tx=0x7fa80000dc90 comp rx=0 tx=0).stop 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 shutdown_connections 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7f406be30 0x7fa7f406e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81015e510 0x7fa8101f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8100b9bb0 0x7fa8101f4340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 --2- 172.21.15.113:0/3847866915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8100b61e0 0x7fa810153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 >> 172.21.15.113:0/3847866915 conn(0x7fa81006b720 msgr2=0x7fa810154870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:17.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 shutdown_connections 2024-04-24T23:58:17.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.228+0000 7fa8175a9640 1 -- 172.21.15.113:0/3847866915 wait complete. 2024-04-24T23:58:17.242 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:58:17.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9dfd258640 1 Processor -- start 2024-04-24T23:58:17.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9dfd258640 1 -- start start 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9dfd258640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9dfd258640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9df80b4a40 0x7f9df80ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9dfd258640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9df815f4f0 con 0x7f9df80b3080 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9dfd258640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9df80cde00 con 0x7f9df80b4a40 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9dfd258640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9df815f7d0 con 0x7f9df815e2d0 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9df80b4a40 0x7f9df80ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9df80b4a40 0x7f9df80ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34670/0 (socket says 172.21.15.113:34670) 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df759e640 1 -- 172.21.15.113:0/3665887230 learned_addr learned my addr 172.21.15.113:0/3665887230 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df6d9d640 1 --2- 172.21.15.113:0/3665887230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df759e640 1 -- 172.21.15.113:0/3665887230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 msgr2=0x7f9df815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:17.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df759e640 1 --2- 172.21.15.113:0/3665887230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df759e640 1 -- 172.21.15.113:0/3665887230 >> v1:172.21.15.178:6789/0 conn(0x7f9df80b3080 legacy=0x7f9df80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.345+0000 7f9df759e640 1 -- 172.21.15.113:0/3665887230 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df815fab0 con 0x7f9df80b4a40 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9df759e640 1 --2- 172.21.15.113:0/3665887230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b4a40 0x7f9df80ce290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009e50 tx=0x7f9df002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a0bb3bf4482a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9df5d9b640 1 -- 172.21.15.113:0/3665887230 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0036070 con 0x7f9df80b4a40 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9df5d9b640 1 -- 172.21.15.113:0/3665887230 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9df0031600 con 0x7f9df80b4a40 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9df5d9b640 1 -- 172.21.15.113:0/3665887230 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df00319d0 con 0x7f9df80b4a40 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 -- 172.21.15.113:0/3665887230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b4a40 msgr2=0x7f9df80ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 --2- 172.21.15.113:0/3665887230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b4a40 0x7f9df80ce290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009e50 tx=0x7f9df002a710 comp rx=0 tx=0).stop 2024-04-24T23:58:17.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 -- 172.21.15.113:0/3665887230 shutdown_connections 2024-04-24T23:58:17.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 --2- 172.21.15.113:0/3665887230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b4a40 0x7f9df80ce290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 --2- 172.21.15.113:0/3665887230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 -- 172.21.15.113:0/3665887230 >> 172.21.15.113:0/3665887230 conn(0x7f9df806b0f0 msgr2=0x7f9df806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:17.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 -- 172.21.15.113:0/3665887230 shutdown_connections 2024-04-24T23:58:17.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.346+0000 7f9dfd258640 1 -- 172.21.15.113:0/3665887230 wait complete. 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.348+0000 7f9dfd258640 1 Processor -- start 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.348+0000 7f9dfd258640 1 -- start start 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b3080 0x7f9df806d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df80b4a40 0x7f9df81f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df8153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9df815f2b0 con 0x7f9df80b3080 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9df815f130 con 0x7f9df80b4a40 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9df815e6b0 con 0x7f9df815e2d0 2024-04-24T23:58:17.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b3080 0x7f9df806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b3080 0x7f9df806d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34680/0 (socket says 172.21.15.113:34680) 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 -- 172.21.15.113:0/474009870 learned_addr learned my addr 172.21.15.113:0/474009870 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df659c640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df80b4a40 0x7f9df81f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df759e640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df8153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 -- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 msgr2=0x7f9df8153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df8153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 -- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df80b4a40 msgr2=0x7f9df81f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df80b4a40 0x7f9df81f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 -- 172.21.15.113:0/474009870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df81f2b40 con 0x7f9df80b3080 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9df6d9d640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b3080 0x7f9df806d820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d5f0 tx=0x7f9de800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9ddf7fe640 1 -- 172.21.15.113:0/474009870 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de80125f0 con 0x7f9df80b3080 2024-04-24T23:58:17.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9ddf7fe640 1 -- 172.21.15.113:0/474009870 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de8019070 con 0x7f9df80b3080 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df81f2e20 con 0x7f9df80b3080 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9ddf7fe640 1 -- 172.21.15.113:0/474009870 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de80129d0 con 0x7f9df80b3080 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.349+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df81f36c0 con 0x7f9df80b3080 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.350+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df80691a0 con 0x7f9df80b3080 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.351+0000 7f9ddf7fe640 1 -- 172.21.15.113:0/474009870 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9de8012b70 con 0x7f9df80b3080 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.351+0000 7f9ddf7fe640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9de406be30 0x7f9de406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.351+0000 7f9ddf7fe640 1 -- 172.21.15.113:0/474009870 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f9de808aed0 con 0x7f9df80b3080 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.351+0000 7f9df659c640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9de406be30 0x7f9de406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:17.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.352+0000 7f9df659c640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9de406be30 0x7f9de406e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9df80b34c0 tx=0x7f9dec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:17.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.357+0000 7f9ddf7fe640 1 -- 172.21.15.113:0/474009870 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9de8059410 con 0x7f9df80b3080 2024-04-24T23:58:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:17.529+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9df8072050 con 0x7f9df80b3080 2024-04-24T23:58:18.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.228+0000 7f9ddf7fe640 1 -- 172.21.15.113:0/474009870 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f9de805d710 con 0x7f9df80b3080 2024-04-24T23:58:18.228 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-24T23:58:18.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9de406be30 msgr2=0x7f9de406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:18.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9de406be30 0x7f9de406e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9df80b34c0 tx=0x7f9dec009040 comp rx=0 tx=0).stop 2024-04-24T23:58:18.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b3080 msgr2=0x7f9df806d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:18.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b3080 0x7f9df806d820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d5f0 tx=0x7f9de800dab0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 shutdown_connections 2024-04-24T23:58:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9de406be30 0x7f9de406e2f0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9df80b34c0 tx=0x7f9dec009040 comp rx=0 tx=0).stop 2024-04-24T23:58:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df80b4a40 0x7f9df81f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9df815e2d0 0x7f9df8153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 --2- 172.21.15.113:0/474009870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9df80b3080 0x7f9df806d820 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d5f0 tx=0x7f9de800dab0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.231+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 >> 172.21.15.113:0/474009870 conn(0x7f9df806b0f0 msgr2=0x7f9df8158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.233+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 shutdown_connections 2024-04-24T23:58:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.233+0000 7f9dfd258640 1 -- 172.21.15.113:0/474009870 wait complete. 2024-04-24T23:58:18.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:58:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.358+0000 7ff576faf640 1 Processor -- start 2024-04-24T23:58:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.358+0000 7ff576faf640 1 -- start start 2024-04-24T23:58:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff576faf640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff57015e310 0x7ff5700d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff576faf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff570071df0 0x7ff57015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff576faf640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57006d900 con 0x7ff5700b1e00 2024-04-24T23:58:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff576faf640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff57015fc40 con 0x7ff57015e310 2024-04-24T23:58:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff576faf640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff57006e9a0 con 0x7ff570071df0 2024-04-24T23:58:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff574d4b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff57015e310 0x7ff5700d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff57554c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff570071df0 0x7ff57015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff56ffff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff5700b1e00 0x7ff5700716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:49778/0 (socket says 172.21.15.113:49778) 2024-04-24T23:58:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff56ffff640 1 -- 172.21.15.113:0/2026375233 learned_addr learned my addr 172.21.15.113:0/2026375233 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.359+0000 7ff574d4b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff57015e310 0x7ff5700d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:56664/0 (socket says 172.21.15.113:56664) 2024-04-24T23:58:18.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.360+0000 7ff574d4b640 1 -- 172.21.15.113:0/2026375233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff570071df0 msgr2=0x7ff57015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:18.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.360+0000 7ff574d4b640 1 --2- 172.21.15.113:0/2026375233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff570071df0 0x7ff57015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.360+0000 7ff574d4b640 1 -- 172.21.15.113:0/2026375233 >> v1:172.21.15.178:6789/0 conn(0x7ff5700b1e00 legacy=0x7ff5700716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:18.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.360+0000 7ff56f7fe640 1 -- 172.21.15.113:0/2026375233 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3620461477 0 0) 0x7ff57006d900 con 0x7ff5700b1e00 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.360+0000 7ff574d4b640 1 -- 172.21.15.113:0/2026375233 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57006dbe0 con 0x7ff57015e310 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.360+0000 7ff574d4b640 1 --2- 172.21.15.113:0/2026375233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff57015e310 0x7ff5700d00e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5600097a0 tx=0x7ff56002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb51dae788b19158 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff56f7fe640 1 -- 172.21.15.113:0/2026375233 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560037070 con 0x7ff57015e310 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff56f7fe640 1 -- 172.21.15.113:0/2026375233 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56002adb0 con 0x7ff57015e310 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff56f7fe640 1 -- 172.21.15.113:0/2026375233 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560032660 con 0x7ff57015e310 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 -- 172.21.15.113:0/2026375233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff57015e310 msgr2=0x7ff5700d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 --2- 172.21.15.113:0/2026375233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff57015e310 0x7ff5700d00e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5600097a0 tx=0x7ff56002a060 comp rx=0 tx=0).stop 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 -- 172.21.15.113:0/2026375233 shutdown_connections 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 --2- 172.21.15.113:0/2026375233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff570071df0 0x7ff57015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 --2- 172.21.15.113:0/2026375233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff57015e310 0x7ff5700d00e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 -- 172.21.15.113:0/2026375233 >> 172.21.15.113:0/2026375233 conn(0x7ff57006fb60 msgr2=0x7ff57006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 -- 172.21.15.113:0/2026375233 shutdown_connections 2024-04-24T23:58:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.361+0000 7ff576faf640 1 -- 172.21.15.113:0/2026375233 wait complete. 2024-04-24T23:58:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.362+0000 7ff576faf640 1 Processor -- start 2024-04-24T23:58:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.362+0000 7ff576faf640 1 -- start start 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff576faf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570071df0 0x7ff57013eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff576faf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5700b1e00 0x7ff570152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff576faf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015e310 0x7ff5700b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff576faf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff57006e1a0 con 0x7ff5700b1e00 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff576faf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff57006e510 con 0x7ff570071df0 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff576faf640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5700ceb20 con 0x7ff57015e310 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5700b1e00 0x7ff570152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5700b1e00 0x7ff570152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48968/0 (socket says 172.21.15.113:48968) 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.363+0000 7ff56ffff640 1 -- 172.21.15.113:0/2760308051 learned_addr learned my addr 172.21.15.113:0/2760308051 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff574d4b640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570071df0 0x7ff57013eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff57554c640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015e310 0x7ff5700b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56ffff640 1 -- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015e310 msgr2=0x7ff5700b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56ffff640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015e310 0x7ff5700b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56ffff640 1 -- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570071df0 msgr2=0x7ff57013eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56ffff640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570071df0 0x7ff57013eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56ffff640 1 -- 172.21.15.113:0/2760308051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5701555e0 con 0x7ff5700b1e00 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff574d4b640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570071df0 0x7ff57013eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56ffff640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5700b1e00 0x7ff570152d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c4d0 tx=0x7ff56400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56d7fa640 1 -- 172.21.15.113:0/2760308051 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564018070 con 0x7ff5700b1e00 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56d7fa640 1 -- 172.21.15.113:0/2760308051 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5640074e0 con 0x7ff5700b1e00 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff570155bf0 con 0x7ff5700b1e00 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff56d7fa640 1 -- 172.21.15.113:0/2760308051 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564004a20 con 0x7ff5700b1e00 2024-04-24T23:58:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.364+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff570155880 con 0x7ff5700b1e00 2024-04-24T23:58:18.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.365+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5700691a0 con 0x7ff5700b1e00 2024-04-24T23:58:18.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.366+0000 7ff56d7fa640 1 -- 172.21.15.113:0/2760308051 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff564004bc0 con 0x7ff5700b1e00 2024-04-24T23:58:18.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.366+0000 7ff56d7fa640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff55c06be30 0x7ff55c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:18.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.366+0000 7ff574d4b640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff55c06be30 0x7ff55c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.366+0000 7ff56d7fa640 1 -- 172.21.15.113:0/2760308051 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7ff56408a160 con 0x7ff5700b1e00 2024-04-24T23:58:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.366+0000 7ff574d4b640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff55c06be30 0x7ff55c06e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff560034000 tx=0x7ff56002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.370+0000 7ff56d7fa640 1 -- 172.21.15.113:0/2760308051 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff564058b80 con 0x7ff5700b1e00 2024-04-24T23:58:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:18.583+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff5700b3200 con 0x7ff5700b1e00 2024-04-24T23:58:19.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.248+0000 7ff56d7fa640 1 -- 172.21.15.113:0/2760308051 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ff564058480 con 0x7ff5700b1e00 2024-04-24T23:58:19.251 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T23:58:19.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.262+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff55c06be30 msgr2=0x7ff55c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.262+0000 7ff576faf640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff55c06be30 0x7ff55c06e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff560034000 tx=0x7ff56002e000 comp rx=0 tx=0).stop 2024-04-24T23:58:19.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.262+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5700b1e00 msgr2=0x7ff570152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.262+0000 7ff576faf640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5700b1e00 0x7ff570152d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff56400c4d0 tx=0x7ff56400c990 comp rx=0 tx=0).stop 2024-04-24T23:58:19.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.265+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 shutdown_connections 2024-04-24T23:58:19.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.265+0000 7ff576faf640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff55c06be30 0x7ff55c06e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.265+0000 7ff576faf640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015e310 0x7ff5700b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.265+0000 7ff576faf640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5700b1e00 0x7ff570152d30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.265+0000 7ff576faf640 1 --2- 172.21.15.113:0/2760308051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570071df0 0x7ff57013eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.265+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 >> 172.21.15.113:0/2760308051 conn(0x7ff57006fb60 msgr2=0x7ff5700716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:19.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.266+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 shutdown_connections 2024-04-24T23:58:19.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.266+0000 7ff576faf640 1 -- 172.21.15.113:0/2760308051 wait complete. 2024-04-24T23:58:19.280 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:19.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.377+0000 7f1266a37640 1 Processor -- start 2024-04-24T23:58:19.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.377+0000 7f1266a37640 1 -- start start 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f1266a37640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f126015e240 0x7f12601627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f1266a37640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 0x7f12600b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f1266a37640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1260162d20 con 0x7f12600b5620 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f1266a37640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1260072d90 con 0x7f126015e240 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f1266a37640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12600b3c20 con 0x7f126006e6e0 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f125ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f126015e240 0x7f12601627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f125ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f126015e240 0x7f12601627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48976/0 (socket says 172.21.15.113:48976) 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f125ffff640 1 -- 172.21.15.113:0/2547052618 learned_addr learned my addr 172.21.15.113:0/2547052618 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f1264fd4640 1 --2- 172.21.15.113:0/2547052618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 0x7f12600b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f125ffff640 1 -- 172.21.15.113:0/2547052618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 msgr2=0x7f12600b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f125ffff640 1 --2- 172.21.15.113:0/2547052618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 0x7f12600b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f125ffff640 1 -- 172.21.15.113:0/2547052618 >> v1:172.21.15.178:6789/0 conn(0x7f12600b5620 legacy=0x7f126006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.378+0000 7f125ffff640 1 -- 172.21.15.113:0/2547052618 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126015e620 con 0x7f126015e240 2024-04-24T23:58:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f125ffff640 1 --2- 172.21.15.113:0/2547052618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126015e240 0x7f12601627e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1250009870 tx=0x7f125002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdabf8ca541eeae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f125effd640 1 -- 172.21.15.113:0/2547052618 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250037070 con 0x7f126015e240 2024-04-24T23:58:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f125effd640 1 -- 172.21.15.113:0/2547052618 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1250032440 con 0x7f126015e240 2024-04-24T23:58:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f125effd640 1 -- 172.21.15.113:0/2547052618 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250032810 con 0x7f126015e240 2024-04-24T23:58:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 -- 172.21.15.113:0/2547052618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126015e240 msgr2=0x7f12601627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 --2- 172.21.15.113:0/2547052618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126015e240 0x7f12601627e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1250009870 tx=0x7f125002a210 comp rx=0 tx=0).stop 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 -- 172.21.15.113:0/2547052618 shutdown_connections 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 --2- 172.21.15.113:0/2547052618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 0x7f12600b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 --2- 172.21.15.113:0/2547052618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126015e240 0x7f12601627e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 -- 172.21.15.113:0/2547052618 >> 172.21.15.113:0/2547052618 conn(0x7f12600b1a50 msgr2=0x7f12600b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 -- 172.21.15.113:0/2547052618 shutdown_connections 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.379+0000 7f1266a37640 1 -- 172.21.15.113:0/2547052618 wait complete. 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 Processor -- start 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 -- start start 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 0x7f1260150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12600b5620 0x7f12601564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126015e240 0x7f126015fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12600727d0 con 0x7f12600b5620 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12600724f0 con 0x7f126015e240 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.381+0000 7f1266a37640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1260072ab0 con 0x7f126006e6e0 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12600b5620 0x7f12601564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12600b5620 0x7f12601564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48982/0 (socket says 172.21.15.113:48982) 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 -- 172.21.15.113:0/804063749 learned_addr learned my addr 172.21.15.113:0/804063749 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f1264fd4640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126015e240 0x7f126015fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 -- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 msgr2=0x7f1260150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 0x7f1260150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 -- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126015e240 msgr2=0x7f126015fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126015e240 0x7f126015fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 -- 172.21.15.113:0/804063749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1260145c90 con 0x7f12600b5620 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f1264fd4640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126015e240 0x7f126015fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125f7fe640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12600b5620 0x7f12601564a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f125400c400 tx=0x7f125400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125cff9640 1 -- 172.21.15.113:0/804063749 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254011790 con 0x7f12600b5620 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125cff9640 1 -- 172.21.15.113:0/804063749 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1254017070 con 0x7f12600b5620 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12601569e0 con 0x7f12600b5620 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f125cff9640 1 -- 172.21.15.113:0/804063749 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254011bb0 con 0x7f12600b5620 2024-04-24T23:58:19.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.382+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1260150280 con 0x7f12600b5620 2024-04-24T23:58:19.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.383+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12600691a0 con 0x7f12600b5620 2024-04-24T23:58:19.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.384+0000 7f125cff9640 1 -- 172.21.15.113:0/804063749 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f125401c460 con 0x7f12600b5620 2024-04-24T23:58:19.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.384+0000 7f125cff9640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f124c06be30 0x7f124c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.384+0000 7f125cff9640 1 -- 172.21.15.113:0/804063749 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f125408aca0 con 0x7f12600b5620 2024-04-24T23:58:19.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.384+0000 7f125ffff640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f124c06be30 0x7f124c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.385+0000 7f125ffff640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f124c06be30 0x7f124c06e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1250034000 tx=0x7f125002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.388+0000 7f125cff9640 1 -- 172.21.15.113:0/804063749 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1254059650 con 0x7f12600b5620 2024-04-24T23:58:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.563+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1260161b40 con 0x7f12600b5620 2024-04-24T23:58:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.564+0000 7f125cff9640 1 -- 172.21.15.113:0/804063749 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7f1254058f50 con 0x7f12600b5620 2024-04-24T23:58:19.564 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:19.565 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:18.584334+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f124c06be30 msgr2=0x7f124c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f124c06be30 0x7f124c06e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1250034000 tx=0x7f125002c000 comp rx=0 tx=0).stop 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12600b5620 msgr2=0x7f12601564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12600b5620 0x7f12601564a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f125400c400 tx=0x7f125400c8c0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 shutdown_connections 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f124c06be30 0x7f124c06e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126015e240 0x7f126015fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12600b5620 0x7f12601564a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 --2- 172.21.15.113:0/804063749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126006e6e0 0x7f1260150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 >> 172.21.15.113:0/804063749 conn(0x7f12600b1a50 msgr2=0x7f12600ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 shutdown_connections 2024-04-24T23:58:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.566+0000 7f1266a37640 1 -- 172.21.15.113:0/804063749 wait complete. 2024-04-24T23:58:19.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.674+0000 7fad6f088640 1 Processor -- start 2024-04-24T23:58:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.674+0000 7fad6f088640 1 -- start start 2024-04-24T23:58:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.675+0000 7fad6f088640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fad6806c9f0 0x7fad680713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.675+0000 7fad6f088640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad680cfbc0 con 0x7fad68161770 2024-04-24T23:58:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.675+0000 7fad6f088640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad680cfea0 con 0x7fad681623e0 2024-04-24T23:58:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.675+0000 7fad6f088640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fad6806d0f0 con 0x7fad6806c9f0 2024-04-24T23:58:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.675+0000 7fad6ce24640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fad68161770 0x7fad680cc510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40930/0 (socket says 172.21.15.113:40930) 2024-04-24T23:58:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.675+0000 7fad6ce24640 1 -- 172.21.15.113:0/2486521428 learned_addr learned my addr 172.21.15.113:0/2486521428 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.675+0000 7fad67fff640 1 --2- 172.21.15.113:0/2486521428 >> v2:172.21.15.178:3300/0 conn(0x7fad6806c9f0 0x7fad680713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.676+0000 7fad677fe640 1 -- 172.21.15.113:0/2486521428 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145573773 0 0) 0x7fad680cfbc0 con 0x7fad68161770 2024-04-24T23:58:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.676+0000 7fad677fe640 1 -- 172.21.15.113:0/2486521428 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad50003520 con 0x7fad68161770 2024-04-24T23:58:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.676+0000 7fad677fe640 1 -- 172.21.15.113:0/2486521428 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3084903553 0 0) 0x7fad50003520 con 0x7fad68161770 2024-04-24T23:58:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.676+0000 7fad67fff640 1 -- 172.21.15.113:0/2486521428 >> v1:172.21.15.178:6789/0 conn(0x7fad681623e0 legacy=0x7fad6806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.676+0000 7fad67fff640 1 -- 172.21.15.113:0/2486521428 >> v1:172.21.15.113:6789/0 conn(0x7fad68161770 legacy=0x7fad680cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.676+0000 7fad67fff640 1 -- 172.21.15.113:0/2486521428 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad680ccc20 con 0x7fad6806c9f0 2024-04-24T23:58:19.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad67fff640 1 --2- 172.21.15.113:0/2486521428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 0x7fad680713f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0097d0 tx=0x7fad5c02a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed73459aa02b57f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad677fe640 1 -- 172.21.15.113:0/2486521428 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c037070 con 0x7fad6806c9f0 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad677fe640 1 -- 172.21.15.113:0/2486521428 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5c02abf0 con 0x7fad6806c9f0 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad677fe640 1 -- 172.21.15.113:0/2486521428 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58001670 con 0x7fad6806c9f0 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad6f088640 1 -- 172.21.15.113:0/2486521428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 msgr2=0x7fad680713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad6f088640 1 --2- 172.21.15.113:0/2486521428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 0x7fad680713f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0097d0 tx=0x7fad5c02a160 comp rx=0 tx=0).stop 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad6f088640 1 -- 172.21.15.113:0/2486521428 shutdown_connections 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad6f088640 1 --2- 172.21.15.113:0/2486521428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 0x7fad680713f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad6f088640 1 -- 172.21.15.113:0/2486521428 >> 172.21.15.113:0/2486521428 conn(0x7fad680d1ec0 msgr2=0x7fad680d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad6f088640 1 -- 172.21.15.113:0/2486521428 shutdown_connections 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.677+0000 7fad6f088640 1 -- 172.21.15.113:0/2486521428 wait complete. 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.678+0000 7fad6f088640 1 Processor -- start 2024-04-24T23:58:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.678+0000 7fad6f088640 1 -- start start 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6f088640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 0x7fad681577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6f088640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad68161770 0x7fad681f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6f088640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad681623e0 0x7fad680b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6f088640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad6806cea0 con 0x7fad681623e0 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6f088640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad68161f50 con 0x7fad6806c9f0 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6f088640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad680d0180 con 0x7fad68161770 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6d625640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad681623e0 0x7fad680b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6d625640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad681623e0 0x7fad680b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48996/0 (socket says 172.21.15.113:48996) 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6d625640 1 -- 172.21.15.113:0/986890851 learned_addr learned my addr 172.21.15.113:0/986890851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.679+0000 7fad6ce24640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 0x7fad681577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad67fff640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad68161770 0x7fad681f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6d625640 1 -- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad68161770 msgr2=0x7fad681f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6d625640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad68161770 0x7fad681f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6d625640 1 -- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 msgr2=0x7fad681577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6d625640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 0x7fad681577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6d625640 1 -- 172.21.15.113:0/986890851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad681f54d0 con 0x7fad681623e0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6d625640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad681623e0 0x7fad680b6170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c6e0 tx=0x7fad6000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad657fa640 1 -- 172.21.15.113:0/986890851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60003b70 con 0x7fad681623e0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad657fa640 1 -- 172.21.15.113:0/986890851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad60003d30 con 0x7fad681623e0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad681f4f10 con 0x7fad681623e0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad657fa640 1 -- 172.21.15.113:0/986890851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60005310 con 0x7fad681623e0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.680+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681f5800 con 0x7fad681623e0 2024-04-24T23:58:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.681+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad68162830 con 0x7fad681623e0 2024-04-24T23:58:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.681+0000 7fad657fa640 1 -- 172.21.15.113:0/986890851 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad600054b0 con 0x7fad681623e0 2024-04-24T23:58:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.682+0000 7fad657fa640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.682+0000 7fad657fa640 1 -- 172.21.15.113:0/986890851 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fad6008aad0 con 0x7fad681623e0 2024-04-24T23:58:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.682+0000 7fad6ce24640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.682+0000 7fad6ce24640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad58005880 tx=0x7fad58000bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.686+0000 7fad657fa640 1 -- 172.21.15.113:0/986890851 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad68162830 con 0x7fad681623e0 2024-04-24T23:58:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.861+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad680691a0 con 0x7fad681623e0 2024-04-24T23:58:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.862+0000 7fad657fa640 1 -- 172.21.15.113:0/986890851 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7fad681f54d0 con 0x7fad681623e0 2024-04-24T23:58:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:19.862 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:18.584334+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 msgr2=0x7fad5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad58005880 tx=0x7fad58000bc0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad681623e0 msgr2=0x7fad680b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad681623e0 0x7fad680b6170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c6e0 tx=0x7fad6000cba0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 shutdown_connections 2024-04-24T23:58:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad681623e0 0x7fad680b6170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad68161770 0x7fad681f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 --2- 172.21.15.113:0/986890851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6806c9f0 0x7fad681577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 >> 172.21.15.113:0/986890851 conn(0x7fad680d1ec0 msgr2=0x7fad680cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 shutdown_connections 2024-04-24T23:58:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.864+0000 7fad6f088640 1 -- 172.21.15.113:0/986890851 wait complete. 2024-04-24T23:58:19.875 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:58:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.974+0000 7f91f411b640 1 Processor -- start 2024-04-24T23:58:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.974+0000 7f91f411b640 1 -- start start 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f411b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f91ec1606a0 0x7f91ec167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f411b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ec15d6b0 con 0x7f91ec0d06c0 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f411b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ec15c8f0 con 0x7f91ec13f050 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f411b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f91ec15c6a0 con 0x7f91ec1606a0 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f26b8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f91ec1606a0 0x7f91ec167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f16b6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f91ec0d06c0 0x7f91ec15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40946/0 (socket says 172.21.15.113:40946) 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f16b6640 1 -- 172.21.15.113:0/1694807909 learned_addr learned my addr 172.21.15.113:0/1694807909 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f26b8640 1 -- 172.21.15.113:0/1694807909 >> v1:172.21.15.178:6789/0 conn(0x7f91ec13f050 legacy=0x7f91ec0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f26b8640 1 -- 172.21.15.113:0/1694807909 >> v1:172.21.15.113:6789/0 conn(0x7f91ec0d06c0 legacy=0x7f91ec15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f26b8640 1 -- 172.21.15.113:0/1694807909 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec15cbd0 con 0x7f91ec1606a0 2024-04-24T23:58:19.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.975+0000 7f91f0eb5640 1 -- 172.21.15.113:0/1694807909 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340580372 0 0) 0x7f91ec15d6b0 con 0x7f91ec0d06c0 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f26b8640 1 --2- 172.21.15.113:0/1694807909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec1606a0 0x7f91ec167080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009d90 tx=0x7f91e402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8407011bbfd6bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f0eb5640 1 -- 172.21.15.113:0/1694807909 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4031600 con 0x7f91ec1606a0 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f0eb5640 1 -- 172.21.15.113:0/1694807909 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e40317a0 con 0x7f91ec1606a0 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f0eb5640 1 -- 172.21.15.113:0/1694807909 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4031b70 con 0x7f91ec1606a0 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f411b640 1 -- 172.21.15.113:0/1694807909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec1606a0 msgr2=0x7f91ec167080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f411b640 1 --2- 172.21.15.113:0/1694807909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec1606a0 0x7f91ec167080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009d90 tx=0x7f91e402a610 comp rx=0 tx=0).stop 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f411b640 1 -- 172.21.15.113:0/1694807909 shutdown_connections 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f411b640 1 --2- 172.21.15.113:0/1694807909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec1606a0 0x7f91ec167080 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.976+0000 7f91f411b640 1 -- 172.21.15.113:0/1694807909 >> 172.21.15.113:0/1694807909 conn(0x7f91ec06b0f0 msgr2=0x7f91ec06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.977+0000 7f91f411b640 1 -- 172.21.15.113:0/1694807909 shutdown_connections 2024-04-24T23:58:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.977+0000 7f91f411b640 1 -- 172.21.15.113:0/1694807909 wait complete. 2024-04-24T23:58:19.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 Processor -- start 2024-04-24T23:58:19.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 -- start start 2024-04-24T23:58:19.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec0d06c0 0x7f91ec131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91ec13f050 0x7f91ec155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ec1606a0 0x7f91ec15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec13f430 con 0x7f91ec0d06c0 2024-04-24T23:58:19.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec15d190 con 0x7f91ec1606a0 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f411b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec0cfa70 con 0x7f91ec13f050 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f1eb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec0d06c0 0x7f91ec131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f1eb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec0d06c0 0x7f91ec131800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49022/0 (socket says 172.21.15.113:49022) 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.978+0000 7f91f1eb7640 1 -- 172.21.15.113:0/3446719044 learned_addr learned my addr 172.21.15.113:0/3446719044 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f1eb7640 1 -- 172.21.15.113:0/3446719044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91ec13f050 msgr2=0x7f91ec155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f1eb7640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91ec13f050 0x7f91ec155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f1eb7640 1 -- 172.21.15.113:0/3446719044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ec1606a0 msgr2=0x7f91ec15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f1eb7640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ec1606a0 0x7f91ec15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f1eb7640 1 -- 172.21.15.113:0/3446719044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec154a00 con 0x7f91ec0d06c0 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f1eb7640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec0d06c0 0x7f91ec131800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c430 tx=0x7f91e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91e27fc640 1 -- 172.21.15.113:0/3446719044 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8011790 con 0x7f91ec0d06c0 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91e27fc640 1 -- 172.21.15.113:0/3446719044 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e8017070 con 0x7f91ec0d06c0 2024-04-24T23:58:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec154ce0 con 0x7f91ec0d06c0 2024-04-24T23:58:19.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91e27fc640 1 -- 172.21.15.113:0/3446719044 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8011bb0 con 0x7f91ec0d06c0 2024-04-24T23:58:19.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.979+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec1552a0 con 0x7f91ec0d06c0 2024-04-24T23:58:19.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.980+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec0691a0 con 0x7f91ec0d06c0 2024-04-24T23:58:19.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.980+0000 7f91e27fc640 1 -- 172.21.15.113:0/3446719044 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91e801c460 con 0x7f91ec0d06c0 2024-04-24T23:58:19.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.981+0000 7f91e27fc640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91d006be30 0x7f91d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:19.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.981+0000 7f91e27fc640 1 -- 172.21.15.113:0/3446719044 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f91e808ad60 con 0x7f91ec0d06c0 2024-04-24T23:58:19.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.981+0000 7f91f16b6640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91d006be30 0x7f91d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:19.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.981+0000 7f91f16b6640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91d006be30 0x7f91d006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91dc007fd0 tx=0x7f91dc007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:19.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:19.985+0000 7f91e27fc640 1 -- 172.21.15.113:0/3446719044 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91e8059710 con 0x7f91ec0d06c0 2024-04-24T23:58:20.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.193+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f91ec156770 con 0x7f91ec0d06c0 2024-04-24T23:58:20.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.787+0000 7f91e27fc640 1 -- 172.21.15.113:0/3446719044 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f91e8059010 con 0x7f91ec0d06c0 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91d006be30 msgr2=0x7f91d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91d006be30 0x7f91d006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91dc007fd0 tx=0x7f91dc007920 comp rx=0 tx=0).stop 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec0d06c0 msgr2=0x7f91ec131800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec0d06c0 0x7f91ec131800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c430 tx=0x7f91e800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 shutdown_connections 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91d006be30 0x7f91d006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ec1606a0 0x7f91ec15e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91ec13f050 0x7f91ec155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 --2- 172.21.15.113:0/3446719044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec0d06c0 0x7f91ec131800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 >> 172.21.15.113:0/3446719044 conn(0x7f91ec06b0f0 msgr2=0x7f91ec06cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:20.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 shutdown_connections 2024-04-24T23:58:20.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.790+0000 7f91f411b640 1 -- 172.21.15.113:0/3446719044 wait complete. 2024-04-24T23:58:20.802 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:58:20.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:58:20.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.907+0000 7f028a970640 1 Processor -- start 2024-04-24T23:58:20.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.908+0000 7f028a970640 1 -- start start 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.908+0000 7f028a970640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f028415e510 0x7f02840d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.908+0000 7f028a970640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02840d28c0 0x7f02840b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.908+0000 7f028a970640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02840d2e00 con 0x7f02840b61e0 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.908+0000 7f028a970640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f02840d30e0 con 0x7f02840d28c0 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.908+0000 7f028a970640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02840d33c0 con 0x7f028415e510 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f02837fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02840d28c0 0x7f02840b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f0288f0d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f02840b61e0 0x7f0284071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40958/0 (socket says 172.21.15.113:40958) 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f0288f0d640 1 -- 172.21.15.113:0/246029632 learned_addr learned my addr 172.21.15.113:0/246029632 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f02837fe640 1 -- 172.21.15.113:0/246029632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f028415e510 msgr2=0x7f02840d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f0283fff640 1 --2- 172.21.15.113:0/246029632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f028415e510 0x7f02840d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:20.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f0282ffd640 1 -- 172.21.15.113:0/246029632 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093708431 0 0) 0x7f02840d2e00 con 0x7f02840b61e0 2024-04-24T23:58:20.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f02837fe640 1 --2- 172.21.15.113:0/246029632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f028415e510 0x7f02840d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f02837fe640 1 -- 172.21.15.113:0/246029632 >> v1:172.21.15.113:6789/0 conn(0x7f02840b61e0 legacy=0x7f0284071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:20.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f02837fe640 1 -- 172.21.15.113:0/246029632 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02840d36a0 con 0x7f02840d28c0 2024-04-24T23:58:20.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.909+0000 7f0283fff640 1 --2- 172.21.15.113:0/246029632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f028415e510 0x7f02840d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:20.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f02837fe640 1 --2- 172.21.15.113:0/246029632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840d28c0 0x7f02840b5ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f02780097a0 tx=0x7f027802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9298216639137d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:20.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f0282ffd640 1 -- 172.21.15.113:0/246029632 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278037070 con 0x7f02840d28c0 2024-04-24T23:58:20.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f0282ffd640 1 -- 172.21.15.113:0/246029632 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027802ae50 con 0x7f02840d28c0 2024-04-24T23:58:20.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f0282ffd640 1 -- 172.21.15.113:0/246029632 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278032660 con 0x7f02840d28c0 2024-04-24T23:58:20.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 -- 172.21.15.113:0/246029632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840d28c0 msgr2=0x7f02840b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:20.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 --2- 172.21.15.113:0/246029632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840d28c0 0x7f02840b5ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f02780097a0 tx=0x7f027802a100 comp rx=0 tx=0).stop 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 -- 172.21.15.113:0/246029632 shutdown_connections 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 --2- 172.21.15.113:0/246029632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840d28c0 0x7f02840b5ca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 --2- 172.21.15.113:0/246029632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f028415e510 0x7f02840d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 -- 172.21.15.113:0/246029632 >> 172.21.15.113:0/246029632 conn(0x7f028406b720 msgr2=0x7f02840d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 -- 172.21.15.113:0/246029632 shutdown_connections 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.910+0000 7f028a970640 1 -- 172.21.15.113:0/246029632 wait complete. 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.911+0000 7f028a970640 1 Processor -- start 2024-04-24T23:58:20.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f028a970640 1 -- start start 2024-04-24T23:58:20.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f028a970640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840b61e0 0x7f0284153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:20.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f028a970640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02840d28c0 0x7f0284160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:20.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f028a970640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f028415e510 0x7f02840b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:20.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f028a970640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f02840b9dd0 con 0x7f02840b61e0 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f028a970640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02840d3ea0 con 0x7f028415e510 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f028a970640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02840b83d0 con 0x7f02840d28c0 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f0283fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840b61e0 0x7f0284153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f0283fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840b61e0 0x7f0284153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49046/0 (socket says 172.21.15.113:49046) 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.912+0000 7f0283fff640 1 -- 172.21.15.113:0/1719774131 learned_addr learned my addr 172.21.15.113:0/1719774131 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0283fff640 1 -- 172.21.15.113:0/1719774131 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02840d28c0 msgr2=0x7f0284160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f02837fe640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02840d28c0 0x7f0284160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0283fff640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02840d28c0 0x7f0284160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0283fff640 1 -- 172.21.15.113:0/1719774131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f028415e510 msgr2=0x7f02840b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0283fff640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f028415e510 0x7f02840b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0283fff640 1 -- 172.21.15.113:0/1719774131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02841f4b50 con 0x7f02840b61e0 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0283fff640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840b61e0 0x7f0284153c40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f027400c7f0 tx=0x7f027400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0280ff9640 1 -- 172.21.15.113:0/1719774131 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274015070 con 0x7f02840b61e0 2024-04-24T23:58:20.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0280ff9640 1 -- 172.21.15.113:0/1719774131 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0274007820 con 0x7f02840b61e0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0284153160 con 0x7f02840b61e0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f0280ff9640 1 -- 172.21.15.113:0/1719774131 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274007bd0 con 0x7f02840b61e0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.913+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0284153720 con 0x7f02840b61e0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.914+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02840691a0 con 0x7f02840b61e0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.914+0000 7f0280ff9640 1 -- 172.21.15.113:0/1719774131 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0274018070 con 0x7f02840b61e0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.915+0000 7f0280ff9640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f027006be30 0x7f027006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.915+0000 7f02837fe640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f027006be30 0x7f027006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.915+0000 7f0280ff9640 1 -- 172.21.15.113:0/1719774131 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f027408ada0 con 0x7f02840b61e0 2024-04-24T23:58:20.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.915+0000 7f02837fe640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f027006be30 0x7f027006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0278034000 tx=0x7f027802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:20.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:20.919+0000 7f0280ff9640 1 -- 172.21.15.113:0/1719774131 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0274059750 con 0x7f02840b61e0 2024-04-24T23:58:21.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:58:21.124 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:58:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.128+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f028415b5a0 con 0x7f02840b61e0 2024-04-24T23:58:21.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.791+0000 7f0280ff9640 1 -- 172.21.15.113:0/1719774131 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f0274059050 con 0x7f02840b61e0 2024-04-24T23:58:21.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f027006be30 msgr2=0x7f027006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f027006be30 0x7f027006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0278034000 tx=0x7f027802e000 comp rx=0 tx=0).stop 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840b61e0 msgr2=0x7f0284153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840b61e0 0x7f0284153c40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f027400c7f0 tx=0x7f027400ccb0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 shutdown_connections 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f027006be30 0x7f027006e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f028415e510 0x7f02840b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02840d28c0 0x7f0284160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 --2- 172.21.15.113:0/1719774131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02840b61e0 0x7f0284153c40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.793+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 >> 172.21.15.113:0/1719774131 conn(0x7f028406b720 msgr2=0x7f0284071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.794+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 shutdown_connections 2024-04-24T23:58:21.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.794+0000 7f028a970640 1 -- 172.21.15.113:0/1719774131 wait complete. 2024-04-24T23:58:21.804 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:58:21.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:21.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.900+0000 7f94070a5640 1 Processor -- start 2024-04-24T23:58:21.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.900+0000 7f94070a5640 1 -- start start 2024-04-24T23:58:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f94070a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f940015e320 0x7f94000d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f94070a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 0x7f94000d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f94070a5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94000d3de0 con 0x7f94000d2290 2024-04-24T23:58:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f94070a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f94000d14d0 con 0x7f940015e320 2024-04-24T23:58:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f94070a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94000d3410 con 0x7f94000d0b00 2024-04-24T23:58:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f9404e41640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f940015e320 0x7f94000d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f9404e41640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f940015e320 0x7f94000d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49056/0 (socket says 172.21.15.113:49056) 2024-04-24T23:58:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.901+0000 7f9404e41640 1 -- 172.21.15.113:0/4283220596 learned_addr learned my addr 172.21.15.113:0/4283220596 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f9404e41640 1 -- 172.21.15.113:0/4283220596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 msgr2=0x7f94000d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f9404e41640 1 --2- 172.21.15.113:0/4283220596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 0x7f94000d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f9404e41640 1 -- 172.21.15.113:0/4283220596 >> v1:172.21.15.178:6789/0 conn(0x7f94000d2290 legacy=0x7f94000b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f9404e41640 1 -- 172.21.15.113:0/4283220596 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94000d2800 con 0x7f940015e320 2024-04-24T23:58:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f9404e41640 1 --2- 172.21.15.113:0/4283220596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f940015e320 0x7f94000d38a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009870 tx=0x7f93f002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f1f5a30aa421ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f93ff7fe640 1 -- 172.21.15.113:0/4283220596 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0037070 con 0x7f940015e320 2024-04-24T23:58:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f93ff7fe640 1 -- 172.21.15.113:0/4283220596 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f0032440 con 0x7f940015e320 2024-04-24T23:58:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.902+0000 7f93ff7fe640 1 -- 172.21.15.113:0/4283220596 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0032810 con 0x7f940015e320 2024-04-24T23:58:21.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 -- 172.21.15.113:0/4283220596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f940015e320 msgr2=0x7f94000d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:21.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 --2- 172.21.15.113:0/4283220596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f940015e320 0x7f94000d38a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009870 tx=0x7f93f002a2a0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 -- 172.21.15.113:0/4283220596 shutdown_connections 2024-04-24T23:58:21.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 --2- 172.21.15.113:0/4283220596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 0x7f94000d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 --2- 172.21.15.113:0/4283220596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f940015e320 0x7f94000d38a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 -- 172.21.15.113:0/4283220596 >> 172.21.15.113:0/4283220596 conn(0x7f94000b5c20 msgr2=0x7f94000b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:21.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 -- 172.21.15.113:0/4283220596 shutdown_connections 2024-04-24T23:58:21.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.903+0000 7f94070a5640 1 -- 172.21.15.113:0/4283220596 wait complete. 2024-04-24T23:58:21.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.904+0000 7f94070a5640 1 Processor -- start 2024-04-24T23:58:21.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.904+0000 7f94070a5640 1 -- start start 2024-04-24T23:58:21.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f94070a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 0x7f9400153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:21.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f94070a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94000d2290 0x7f9400154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:21.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f94070a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f940015e320 0x7f94000ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:21.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f94070a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94000d30a0 con 0x7f94000d2290 2024-04-24T23:58:21.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f94070a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94000d2dc0 con 0x7f940015e320 2024-04-24T23:58:21.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f94070a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f940015e700 con 0x7f94000d0b00 2024-04-24T23:58:21.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94000d2290 0x7f9400154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:21.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94000d2290 0x7f9400154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49068/0 (socket says 172.21.15.113:49068) 2024-04-24T23:58:21.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 -- 172.21.15.113:0/2493923676 learned_addr learned my addr 172.21.15.113:0/2493923676 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:21.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f9404e41640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 0x7f9400153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:21.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f9405642640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f940015e320 0x7f94000ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:21.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 -- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 msgr2=0x7f9400153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:21.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 0x7f9400153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 -- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f940015e320 msgr2=0x7f94000ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:21.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f940015e320 0x7f94000ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:21.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.905+0000 7f93fffff640 1 -- 172.21.15.113:0/2493923676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94001568c0 con 0x7f94000d2290 2024-04-24T23:58:21.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.906+0000 7f93fffff640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94000d2290 0x7f9400154cb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c440 tx=0x7f93f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:21.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.906+0000 7f93fd7fa640 1 -- 172.21.15.113:0/2493923676 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4017070 con 0x7f94000d2290 2024-04-24T23:58:21.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.906+0000 7f93fd7fa640 1 -- 172.21.15.113:0/2493923676 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f40035f0 con 0x7f94000d2290 2024-04-24T23:58:21.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.906+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94001551f0 con 0x7f94000d2290 2024-04-24T23:58:21.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.906+0000 7f93fd7fa640 1 -- 172.21.15.113:0/2493923676 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40039e0 con 0x7f94000d2290 2024-04-24T23:58:21.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.906+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9400156e40 con 0x7f94000d2290 2024-04-24T23:58:21.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.907+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94000691a0 con 0x7f94000d2290 2024-04-24T23:58:21.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.907+0000 7f93fd7fa640 1 -- 172.21.15.113:0/2493923676 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93f401a070 con 0x7f94000d2290 2024-04-24T23:58:21.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.908+0000 7f93fd7fa640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93ec06be30 0x7f93ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:21.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.908+0000 7f93fd7fa640 1 -- 172.21.15.113:0/2493923676 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f93f408a1f0 con 0x7f94000d2290 2024-04-24T23:58:21.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.908+0000 7f9404e41640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93ec06be30 0x7f93ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:21.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.908+0000 7f9404e41640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93ec06be30 0x7f93ec06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f93f0034000 tx=0x7f93f002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:21.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:21.912+0000 7f93fd7fa640 1 -- 172.21.15.113:0/2493923676 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93f4058ba0 con 0x7f94000d2290 2024-04-24T23:58:22.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.126+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94000b4a20 con 0x7f94000d2290 2024-04-24T23:58:22.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.127+0000 7f93fd7fa640 1 -- 172.21.15.113:0/2493923676 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f93f40010c0 con 0x7f94000d2290 2024-04-24T23:58:22.127 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:22.128 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:22.128 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:22.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93ec06be30 msgr2=0x7f93ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93ec06be30 0x7f93ec06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f93f0034000 tx=0x7f93f002e000 comp rx=0 tx=0).stop 2024-04-24T23:58:22.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94000d2290 msgr2=0x7f9400154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94000d2290 0x7f9400154cb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c440 tx=0x7f93f400c900 comp rx=0 tx=0).stop 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 shutdown_connections 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93ec06be30 0x7f93ec06e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f940015e320 0x7f94000ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94000d2290 0x7f9400154cb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 --2- 172.21.15.113:0/2493923676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94000d0b00 0x7f9400153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 >> 172.21.15.113:0/2493923676 conn(0x7f94000b5c20 msgr2=0x7f94000b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 shutdown_connections 2024-04-24T23:58:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.129+0000 7f94070a5640 1 -- 172.21.15.113:0/2493923676 wait complete. 2024-04-24T23:58:22.140 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:22.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.236+0000 7fd9020fc640 1 Processor -- start 2024-04-24T23:58:22.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.236+0000 7fd9020fc640 1 -- start start 2024-04-24T23:58:22.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd9020fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8fc144910 0x7fd8fc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd9020fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd8fc0b5240 0x7fd8fc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd9020fc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8fc144cf0 con 0x7fd8fc0b56b0 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd9020fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd8fc0b4510 con 0x7fd8fc144910 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd9020fc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd8fc0b2f30 con 0x7fd8fc0b5240 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8fb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8fc144910 0x7fd8fc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8fb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8fc144910 0x7fd8fc0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49074/0 (socket says 172.21.15.113:49074) 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/2248340557 learned_addr learned my addr 172.21.15.113:0/2248340557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8faffd640 1 --2- 172.21.15.113:0/2248340557 >> v2:172.21.15.178:3300/0 conn(0x7fd8fc0b5240 0x7fd8fc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/2248340557 >> v2:172.21.15.178:3300/0 conn(0x7fd8fc0b5240 msgr2=0x7fd8fc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8fb7fe640 1 --2- 172.21.15.113:0/2248340557 >> v2:172.21.15.178:3300/0 conn(0x7fd8fc0b5240 0x7fd8fc06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/2248340557 >> v1:172.21.15.178:6789/0 conn(0x7fd8fc0b56b0 legacy=0x7fd8fc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.237+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/2248340557 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc0b3210 con 0x7fd8fc144910 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.238+0000 7fd8fb7fe640 1 --2- 172.21.15.113:0/2248340557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc144910 0x7fd8fc0b3fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009870 tx=0x7fd8ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99ceac23381152 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.238+0000 7fd8fa7fc640 1 -- 172.21.15.113:0/2248340557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec037070 con 0x7fd8fc144910 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.238+0000 7fd8fa7fc640 1 -- 172.21.15.113:0/2248340557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec032440 con 0x7fd8fc144910 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.238+0000 7fd8fa7fc640 1 -- 172.21.15.113:0/2248340557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec032810 con 0x7fd8fc144910 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.238+0000 7fd9020fc640 1 -- 172.21.15.113:0/2248340557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc144910 msgr2=0x7fd8fc0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.239+0000 7fd9020fc640 1 --2- 172.21.15.113:0/2248340557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc144910 0x7fd8fc0b3fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009870 tx=0x7fd8ec02a270 comp rx=0 tx=0).stop 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.239+0000 7fd9020fc640 1 -- 172.21.15.113:0/2248340557 shutdown_connections 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.239+0000 7fd9020fc640 1 --2- 172.21.15.113:0/2248340557 >> v2:172.21.15.178:3300/0 conn(0x7fd8fc0b5240 0x7fd8fc06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.239+0000 7fd9020fc640 1 --2- 172.21.15.113:0/2248340557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc144910 0x7fd8fc0b3fd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.239+0000 7fd9020fc640 1 -- 172.21.15.113:0/2248340557 >> 172.21.15.113:0/2248340557 conn(0x7fd8fc0b1a50 msgr2=0x7fd8fc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.239+0000 7fd9020fc640 1 -- 172.21.15.113:0/2248340557 shutdown_connections 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.239+0000 7fd9020fc640 1 -- 172.21.15.113:0/2248340557 wait complete. 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.240+0000 7fd9020fc640 1 Processor -- start 2024-04-24T23:58:22.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.240+0000 7fd9020fc640 1 -- start start 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0b5240 0x7fd8fc13f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8fc0b56b0 0x7fd8fc136900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8fc144910 0x7fd8fc13a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0b4da0 con 0x7fd8fc0b5240 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0b37d0 con 0x7fd8fc144910 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc0b4f20 con 0x7fd8fc0b56b0 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0b5240 0x7fd8fc13f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0b5240 0x7fd8fc13f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49082/0 (socket says 172.21.15.113:49082) 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/3145923583 learned_addr learned my addr 172.21.15.113:0/3145923583 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fbfff640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8fc144910 0x7fd8fc13a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8faffd640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8fc0b56b0 0x7fd8fc136900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8fc0b56b0 msgr2=0x7fd8fc136900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8fc0b56b0 0x7fd8fc136900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8fc144910 msgr2=0x7fd8fc13a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8fc144910 0x7fd8fc13a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 -- 172.21.15.113:0/3145923583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc13ae90 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8fb7fe640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0b5240 0x7fd8fc13f190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009870 tx=0x7fd8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8e3fff640 1 -- 172.21.15.113:0/3145923583 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec037040 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8e3fff640 1 -- 172.21.15.113:0/3145923583 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec004230 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc13b170 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd8e3fff640 1 -- 172.21.15.113:0/3145923583 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec003610 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.241+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc136e40 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.242+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc0691a0 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.243+0000 7fd8e3fff640 1 -- 172.21.15.113:0/3145923583 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8ec0037b0 con 0x7fd8fc0b5240 2024-04-24T23:58:22.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.243+0000 7fd8e3fff640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd8e806be30 0x7fd8e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.243+0000 7fd8e3fff640 1 -- 172.21.15.113:0/3145923583 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd8ec0abc20 con 0x7fd8fc0b5240 2024-04-24T23:58:22.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.243+0000 7fd8faffd640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd8e806be30 0x7fd8e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.243+0000 7fd8faffd640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd8e806be30 0x7fd8e806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc1f5530 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.247+0000 7fd8e3fff640 1 -- 172.21.15.113:0/3145923583 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd8ec07a5d0 con 0x7fd8fc0b5240 2024-04-24T23:58:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.421+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8fc0b07a0 con 0x7fd8fc0b5240 2024-04-24T23:58:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.422+0000 7fd8e3fff640 1 -- 172.21.15.113:0/3145923583 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fd8ec079ed0 con 0x7fd8fc0b5240 2024-04-24T23:58:22.422 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:22.422 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:22.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd8e806be30 msgr2=0x7fd8e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd8e806be30 0x7fd8e806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc1f5530 tx=0x7fd8f0008040 comp rx=0 tx=0).stop 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0b5240 msgr2=0x7fd8fc13f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0b5240 0x7fd8fc13f190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009870 tx=0x7fd8ec0046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 shutdown_connections 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd8e806be30 0x7fd8e806e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8fc144910 0x7fd8fc13a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8fc0b56b0 0x7fd8fc136900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 --2- 172.21.15.113:0/3145923583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0b5240 0x7fd8fc13f190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 >> 172.21.15.113:0/3145923583 conn(0x7fd8fc0b1a50 msgr2=0x7fd8fc071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:22.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 shutdown_connections 2024-04-24T23:58:22.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.424+0000 7fd9020fc640 1 -- 172.21.15.113:0/3145923583 wait complete. 2024-04-24T23:58:22.435 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:58:22.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:58:22.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.531+0000 7f5d608a7640 1 Processor -- start 2024-04-24T23:58:22.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.531+0000 7f5d608a7640 1 -- start start 2024-04-24T23:58:22.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d608a7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d608a7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d5c0b5240 0x7f5d5c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d608a7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d5c144cf0 con 0x7f5d5c0b5680 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d608a7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5c0b4510 con 0x7f5d5c0b5240 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d608a7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5c0b2f30 con 0x7f5d5c144910 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d53fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d5c0b5240 0x7f5d5c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d5ad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5d5c0b5680 0x7f5d5c0cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40974/0 (socket says 172.21.15.113:40974) 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d53fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d5c0b5240 0x7f5d5c06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49094/0 (socket says 172.21.15.113:49094) 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d5ad9d640 1 -- 172.21.15.113:0/3668989286 learned_addr learned my addr 172.21.15.113:0/3668989286 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d5a59c640 1 --2- 172.21.15.113:0/3668989286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d53fff640 1 -- 172.21.15.113:0/3668989286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 msgr2=0x7f5d5c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d53fff640 1 --2- 172.21.15.113:0/3668989286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d53fff640 1 -- 172.21.15.113:0/3668989286 >> v1:172.21.15.113:6789/0 conn(0x7f5d5c0b5680 legacy=0x7f5d5c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d53fff640 1 -- 172.21.15.113:0/3668989286 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c0b3210 con 0x7f5d5c0b5240 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.532+0000 7f5d59d9b640 1 -- 172.21.15.113:0/3668989286 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763511162 0 0) 0x7f5d5c144cf0 con 0x7f5d5c0b5680 2024-04-24T23:58:22.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d53fff640 1 --2- 172.21.15.113:0/3668989286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c06dfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d440097a0 tx=0x7f5d4402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a185dcc230abc8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d59d9b640 1 -- 172.21.15.113:0/3668989286 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44037070 con 0x7f5d5c0b5240 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d59d9b640 1 -- 172.21.15.113:0/3668989286 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d44032440 con 0x7f5d5c0b5240 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d59d9b640 1 -- 172.21.15.113:0/3668989286 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44032810 con 0x7f5d5c0b5240 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d608a7640 1 -- 172.21.15.113:0/3668989286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 msgr2=0x7f5d5c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d608a7640 1 --2- 172.21.15.113:0/3668989286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c06dfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d440097a0 tx=0x7f5d4402a180 comp rx=0 tx=0).stop 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d608a7640 1 -- 172.21.15.113:0/3668989286 shutdown_connections 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d608a7640 1 --2- 172.21.15.113:0/3668989286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c06dfd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d608a7640 1 --2- 172.21.15.113:0/3668989286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.533+0000 7f5d608a7640 1 -- 172.21.15.113:0/3668989286 >> 172.21.15.113:0/3668989286 conn(0x7f5d5c0b1a50 msgr2=0x7f5d5c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.534+0000 7f5d608a7640 1 -- 172.21.15.113:0/3668989286 shutdown_connections 2024-04-24T23:58:22.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.534+0000 7f5d608a7640 1 -- 172.21.15.113:0/3668989286 wait complete. 2024-04-24T23:58:22.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 Processor -- start 2024-04-24T23:58:22.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 -- start start 2024-04-24T23:58:22.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d5c0b5680 0x7f5d5c137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c0b4da0 con 0x7f5d5c0b5240 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c0b37d0 con 0x7f5d5c0b5680 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.535+0000 7f5d608a7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5c0b4f20 con 0x7f5d5c144910 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c1589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49108/0 (socket says 172.21.15.113:49108) 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 -- 172.21.15.113:0/1145279521 learned_addr learned my addr 172.21.15.113:0/1145279521 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d53fff640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d5c0b5680 0x7f5d5c137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5ad9d640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 -- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 msgr2=0x7f5d5c15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c15b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 -- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d5c0b5680 msgr2=0x7f5d5c137200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d5c0b5680 0x7f5d5c137200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 -- 172.21.15.113:0/1145279521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c15a2b0 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d53fff640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d5c0b5680 0x7f5d5c137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5ad9d640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d5a59c640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c1589a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00e9e0 tx=0x7f5d4c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d52ffd640 1 -- 172.21.15.113:0/1145279521 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c017070 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d52ffd640 1 -- 172.21.15.113:0/1145279521 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c00c9d0 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c15a550 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d52ffd640 1 -- 172.21.15.113:0/1145279521 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00cdc0 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.536+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c15ab10 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.537+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c0691a0 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.538+0000 7f5d52ffd640 1 -- 172.21.15.113:0/1145279521 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d4c01a070 con 0x7f5d5c0b5240 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.538+0000 7f5d52ffd640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d3c06be30 0x7f5d3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.538+0000 7f5d53fff640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d3c06be30 0x7f5d3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.538+0000 7f5d52ffd640 1 -- 172.21.15.113:0/1145279521 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d4c08ae90 con 0x7f5d5c0b5240 2024-04-24T23:58:22.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.538+0000 7f5d53fff640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d3c06be30 0x7f5d3c06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d44034000 tx=0x7f5d4402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.542+0000 7f5d52ffd640 1 -- 172.21.15.113:0/1145279521 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d4c059840 con 0x7f5d5c0b5240 2024-04-24T23:58:22.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.711+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d5c0cf870 con 0x7f5d3c06be30 2024-04-24T23:58:22.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.716+0000 7f5d52ffd640 1 -- 172.21.15.113:0/1145279521 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7f5d5c0cf870 con 0x7f5d3c06be30 2024-04-24T23:58:22.719 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:22.720 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d3c06be30 msgr2=0x7f5d3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d3c06be30 0x7f5d3c06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d44034000 tx=0x7f5d4402e000 comp rx=0 tx=0).stop 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 msgr2=0x7f5d5c1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c1589a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00e9e0 tx=0x7f5d4c00eea0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 shutdown_connections 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d3c06be30 0x7f5d3c06e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d5c0b5680 0x7f5d5c137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d5c144910 0x7f5d5c15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 --2- 172.21.15.113:0/1145279521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5c0b5240 0x7f5d5c1589a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 >> 172.21.15.113:0/1145279521 conn(0x7f5d5c0b1a50 msgr2=0x7f5d5c070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:22.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 shutdown_connections 2024-04-24T23:58:22.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.719+0000 7f5d608a7640 1 -- 172.21.15.113:0/1145279521 wait complete. 2024-04-24T23:58:22.730 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-24T23:58:21.112308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":235208,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739512,"statfs":{"total":773094113280,"available":772853260288,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000785"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T23:58:09.226976+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:58:09.226976+0000","last_peered":"2024-04-24T23:58:09.226976+0000","last_clean":"2024-04-24T23:58:09.226976+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:58:09.226976+0000","last_undegraded":"2024-04-24T23:58:09.226976+0000","last_fullsized":"2024-04-24T23:58:09.226976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:58:22.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:58:22.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.868+0000 7fce063d5640 1 Processor -- start 2024-04-24T23:58:22.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.868+0000 7fce063d5640 1 -- start start 2024-04-24T23:58:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fce063d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce000b3fd0 0x7fce0006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fce063d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0006e510 0x7fce000cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fce063d5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce00144cf0 con 0x7fce00144910 2024-04-24T23:58:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fce063d5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fce000b2f30 con 0x7fce000b3fd0 2024-04-24T23:58:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fce063d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce000b3210 con 0x7fce0006e510 2024-04-24T23:58:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdff7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce000b3fd0 0x7fce0006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdfffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fce00144910 0x7fce000b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40984/0 (socket says 172.21.15.113:40984) 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdfffff640 1 -- 172.21.15.113:0/3503956507 learned_addr learned my addr 172.21.15.113:0/3503956507 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fce04972640 1 --2- 172.21.15.113:0/3503956507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0006e510 0x7fce000cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdff7fe640 1 -- 172.21.15.113:0/3503956507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0006e510 msgr2=0x7fce000cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdff7fe640 1 --2- 172.21.15.113:0/3503956507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0006e510 0x7fce000cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdfeffd640 1 -- 172.21.15.113:0/3503956507 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485639551 0 0) 0x7fce00144cf0 con 0x7fce00144910 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdff7fe640 1 -- 172.21.15.113:0/3503956507 >> v1:172.21.15.113:6789/0 conn(0x7fce00144910 legacy=0x7fce000b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fcdff7fe640 1 -- 172.21.15.113:0/3503956507 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce000b34f0 con 0x7fce000b3fd0 2024-04-24T23:58:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.869+0000 7fce04972640 1 --2- 172.21.15.113:0/3503956507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0006e510 0x7fce000cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:22.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fcdff7fe640 1 --2- 172.21.15.113:0/3503956507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce000b3fd0 0x7fce0006dfd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40097d0 tx=0x7fcdf402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c589cf111ecee907 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fcdfeffd640 1 -- 172.21.15.113:0/3503956507 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4037070 con 0x7fce000b3fd0 2024-04-24T23:58:22.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fcdfeffd640 1 -- 172.21.15.113:0/3503956507 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf4032440 con 0x7fce000b3fd0 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fcdfeffd640 1 -- 172.21.15.113:0/3503956507 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4032810 con 0x7fce000b3fd0 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 -- 172.21.15.113:0/3503956507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce000b3fd0 msgr2=0x7fce0006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 --2- 172.21.15.113:0/3503956507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce000b3fd0 0x7fce0006dfd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40097d0 tx=0x7fcdf402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 -- 172.21.15.113:0/3503956507 shutdown_connections 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 --2- 172.21.15.113:0/3503956507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0006e510 0x7fce000cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 --2- 172.21.15.113:0/3503956507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce000b3fd0 0x7fce0006dfd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 -- 172.21.15.113:0/3503956507 >> 172.21.15.113:0/3503956507 conn(0x7fce000b1a50 msgr2=0x7fce000b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 -- 172.21.15.113:0/3503956507 shutdown_connections 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.870+0000 7fce063d5640 1 -- 172.21.15.113:0/3503956507 wait complete. 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 Processor -- start 2024-04-24T23:58:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 -- start start 2024-04-24T23:58:22.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce0006e510 0x7fce001371f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce000b3fd0 0x7fce0013e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce00144910 0x7fce000b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce000b4da0 con 0x7fce0006e510 2024-04-24T23:58:22.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce000b4440 con 0x7fce00144910 2024-04-24T23:58:22.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fce063d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce000b4f20 con 0x7fce000b3fd0 2024-04-24T23:58:22.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.872+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce0006e510 0x7fce001371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce0006e510 0x7fce001371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49124/0 (socket says 172.21.15.113:49124) 2024-04-24T23:58:22.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 -- 172.21.15.113:0/1551574938 learned_addr learned my addr 172.21.15.113:0/1551574938 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdff7fe640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce000b3fd0 0x7fce0013e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fce04972640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce00144910 0x7fce000b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 -- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce000b3fd0 msgr2=0x7fce0013e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce000b3fd0 0x7fce0013e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 -- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce00144910 msgr2=0x7fce000b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce00144910 0x7fce000b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 -- 172.21.15.113:0/1551574938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce00137730 con 0x7fce0006e510 2024-04-24T23:58:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfffff640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce0006e510 0x7fce001371f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000c500 tx=0x7fcdf000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfcff9640 1 -- 172.21.15.113:0/1551574938 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0018070 con 0x7fce0006e510 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfcff9640 1 -- 172.21.15.113:0/1551574938 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf0007580 con 0x7fce0006e510 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fcdfcff9640 1 -- 172.21.15.113:0/1551574938 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0004a20 con 0x7fce0006e510 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce00138f40 con 0x7fce0006e510 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.873+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce001394c0 con 0x7fce0006e510 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.874+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce000691a0 con 0x7fce0006e510 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.875+0000 7fcdfcff9640 1 -- 172.21.15.113:0/1551574938 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdf001b070 con 0x7fce0006e510 2024-04-24T23:58:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.875+0000 7fcdfcff9640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcdec06be30 0x7fcdec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.875+0000 7fcdfcff9640 1 -- 172.21.15.113:0/1551574938 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcdf008a0a0 con 0x7fce0006e510 2024-04-24T23:58:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.875+0000 7fcdff7fe640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcdec06be30 0x7fcdec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.875+0000 7fcdff7fe640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcdec06be30 0x7fcdec06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4034000 tx=0x7fcdf402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:22.879+0000 7fcdfcff9640 1 -- 172.21.15.113:0/1551574938 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce00137730 con 0x7fce0006e510 2024-04-24T23:58:23.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.049+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce00142d40 con 0x7fcdec06be30 2024-04-24T23:58:23.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.054+0000 7fcdfcff9640 1 -- 172.21.15.113:0/1551574938 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7fce00142d40 con 0x7fcdec06be30 2024-04-24T23:58:23.058 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:23.060 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:58:23.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.056+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcdec06be30 msgr2=0x7fcdec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.056+0000 7fce063d5640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcdec06be30 0x7fcdec06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4034000 tx=0x7fcdf402e000 comp rx=0 tx=0).stop 2024-04-24T23:58:23.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.056+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce0006e510 msgr2=0x7fce001371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.056+0000 7fce063d5640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce0006e510 0x7fce001371f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000c500 tx=0x7fcdf000c9c0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 shutdown_connections 2024-04-24T23:58:23.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcdec06be30 0x7fcdec06e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce00144910 0x7fce000b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce000b3fd0 0x7fce0013e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 --2- 172.21.15.113:0/1551574938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce0006e510 0x7fce001371f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 >> 172.21.15.113:0/1551574938 conn(0x7fce000b1a50 msgr2=0x7fce00143240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:23.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 shutdown_connections 2024-04-24T23:58:23.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.057+0000 7fce063d5640 1 -- 172.21.15.113:0/1551574938 wait complete. 2024-04-24T23:58:23.068 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-24T23:58:21.112308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":235208,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739512,"statfs":{"total":773094113280,"available":772853260288,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000785"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T23:58:09.226976+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:58:09.226976+0000","last_peered":"2024-04-24T23:58:09.226976+0000","last_clean":"2024-04-24T23:58:09.226976+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:58:09.226976+0000","last_undegraded":"2024-04-24T23:58:09.226976+0000","last_fullsized":"2024-04-24T23:58:09.226976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:18.227192+0000","last_change":"2024-04-24T23:58:18.227192+0000","last_active":"2024-04-24T23:58:18.227192+0000","last_peered":"2024-04-24T23:58:18.227192+0000","last_clean":"2024-04-24T23:58:18.227192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:18.227192+0000","last_undegraded":"2024-04-24T23:58:18.227192+0000","last_fullsized":"2024-04-24T23:58:18.227192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T23:58:17.223578+0000","last_change":"2024-04-24T23:58:17.223578+0000","last_active":"2024-04-24T23:58:17.223578+0000","last_peered":"2024-04-24T23:58:17.223578+0000","last_clean":"2024-04-24T23:58:17.223578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T23:58:17.223578+0000","last_undegraded":"2024-04-24T23:58:17.223578+0000","last_fullsized":"2024-04-24T23:58:17.223578+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:58:23.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:58:23.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.204+0000 7f1d55b22640 1 Processor -- start 2024-04-24T23:58:23.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.204+0000 7f1d55b22640 1 -- start start 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d55b22640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d5015a6b0 0x7f1d50070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d55b22640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 0x7f1d500b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d55b22640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d500b2d50 con 0x7f1d50072ea0 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d55b22640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1d50072a70 con 0x7f1d5015a6b0 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d55b22640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1d500b3010 con 0x7f1d500b3af0 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d5015a6b0 0x7f1d50070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1d50072ea0 0x7f1d50071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40996/0 (socket says 172.21.15.113:40996) 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4effd640 1 -- 172.21.15.113:0/1783948878 learned_addr learned my addr 172.21.15.113:0/1783948878 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4f7fe640 1 -- 172.21.15.113:0/1783948878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 msgr2=0x7f1d500b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4f7fe640 1 --2- 172.21.15.113:0/1783948878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 0x7f1d500b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4f7fe640 1 -- 172.21.15.113:0/1783948878 >> v1:172.21.15.113:6789/0 conn(0x7f1d50072ea0 legacy=0x7f1d50071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4e7fc640 1 -- 172.21.15.113:0/1783948878 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148114553 0 0) 0x7f1d500b2d50 con 0x7f1d50072ea0 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.205+0000 7f1d4f7fe640 1 -- 172.21.15.113:0/1783948878 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d500b32f0 con 0x7f1d5015a6b0 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.206+0000 7f1d4f7fe640 1 --2- 172.21.15.113:0/1783948878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d5015a6b0 0x7f1d50070d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d400097a0 tx=0x7f1d4002b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5194d6b43ee5c7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.206+0000 7f1d4e7fc640 1 -- 172.21.15.113:0/1783948878 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40038070 con 0x7f1d5015a6b0 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.206+0000 7f1d4e7fc640 1 -- 172.21.15.113:0/1783948878 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d40033440 con 0x7f1d5015a6b0 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.206+0000 7f1d4e7fc640 1 -- 172.21.15.113:0/1783948878 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400337f0 con 0x7f1d5015a6b0 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 -- 172.21.15.113:0/1783948878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d5015a6b0 msgr2=0x7f1d50070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 --2- 172.21.15.113:0/1783948878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d5015a6b0 0x7f1d50070d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d400097a0 tx=0x7f1d4002b290 comp rx=0 tx=0).stop 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 -- 172.21.15.113:0/1783948878 shutdown_connections 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 --2- 172.21.15.113:0/1783948878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 0x7f1d500b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 --2- 172.21.15.113:0/1783948878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d5015a6b0 0x7f1d50070d00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 -- 172.21.15.113:0/1783948878 >> 172.21.15.113:0/1783948878 conn(0x7f1d500b1a50 msgr2=0x7f1d500b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 -- 172.21.15.113:0/1783948878 shutdown_connections 2024-04-24T23:58:23.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.207+0000 7f1d55b22640 1 -- 172.21.15.113:0/1783948878 wait complete. 2024-04-24T23:58:23.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.208+0000 7f1d55b22640 1 Processor -- start 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.208+0000 7f1d55b22640 1 -- start start 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d50072ea0 0x7f1d50149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 0x7f1d500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d5015a6b0 0x7f1d500cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5015aa90 con 0x7f1d50072ea0 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50072720 con 0x7f1d5015a6b0 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5015ac10 con 0x7f1d500b3af0 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d50072ea0 0x7f1d50149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d50072ea0 0x7f1d50149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49146/0 (socket says 172.21.15.113:49146) 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 -- 172.21.15.113:0/3060358214 learned_addr learned my addr 172.21.15.113:0/3060358214 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 -- 172.21.15.113:0/3060358214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 msgr2=0x7f1d500ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 0x7f1d500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 -- 172.21.15.113:0/3060358214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d5015a6b0 msgr2=0x7f1d500cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d5015a6b0 0x7f1d500cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 -- 172.21.15.113:0/3060358214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d501f13b0 con 0x7f1d50072ea0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d4f7fe640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d50072ea0 0x7f1d50149740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1d400097a0 tx=0x7f1d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d37fff640 1 -- 172.21.15.113:0/3060358214 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40038040 con 0x7f1d50072ea0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d37fff640 1 -- 172.21.15.113:0/3060358214 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d40004250 con 0x7f1d50072ea0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5014af50 con 0x7f1d50072ea0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d37fff640 1 -- 172.21.15.113:0/3060358214 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40003610 con 0x7f1d50072ea0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.209+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5014b510 con 0x7f1d50072ea0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.210+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d50149ce0 con 0x7f1d50072ea0 2024-04-24T23:58:23.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.211+0000 7f1d37fff640 1 -- 172.21.15.113:0/3060358214 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d400037b0 con 0x7f1d50072ea0 2024-04-24T23:58:23.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.211+0000 7f1d37fff640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1d3c06be30 0x7f1d3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.211+0000 7f1d37fff640 1 -- 172.21.15.113:0/3060358214 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1d400ad1c0 con 0x7f1d50072ea0 2024-04-24T23:58:23.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.211+0000 7f1d4effd640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1d3c06be30 0x7f1d3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.212+0000 7f1d4effd640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1d3c06be30 0x7f1d3c06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1d44007fd0 tx=0x7f1d44007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.215+0000 7f1d37fff640 1 -- 172.21.15.113:0/3060358214 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1d4007b520 con 0x7f1d50072ea0 2024-04-24T23:58:23.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.386+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d500691a0 con 0x7f1d3c06be30 2024-04-24T23:58:23.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.392+0000 7f1d37fff640 1 -- 172.21.15.113:0/3060358214 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f1d500691a0 con 0x7f1d3c06be30 2024-04-24T23:58:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:23.396 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:58:23.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1d3c06be30 msgr2=0x7f1d3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1d3c06be30 0x7f1d3c06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1d44007fd0 tx=0x7f1d44007920 comp rx=0 tx=0).stop 2024-04-24T23:58:23.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d50072ea0 msgr2=0x7f1d50149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d50072ea0 0x7f1d50149740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1d400097a0 tx=0x7f1d400046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 shutdown_connections 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1d3c06be30 0x7f1d3c06e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d5015a6b0 0x7f1d500cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d500b3af0 0x7f1d500ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 --2- 172.21.15.113:0/3060358214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d50072ea0 0x7f1d50149740 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 >> 172.21.15.113:0/3060358214 conn(0x7f1d500b1a50 msgr2=0x7f1d500cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 shutdown_connections 2024-04-24T23:58:23.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.395+0000 7f1d55b22640 1 -- 172.21.15.113:0/3060358214 wait complete. 2024-04-24T23:58:23.406 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-24T23:58:23.112597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235608,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739112,"statfs":{"total":773094113280,"available":772852850688,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.885004"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678418+0000","last_change":"2024-04-24T23:58:19.270520+0000","last_active":"2024-04-24T23:58:20.678418+0000","last_peered":"2024-04-24T23:58:20.678418+0000","last_clean":"2024-04-24T23:58:20.678418+0000","last_became_active":"2024-04-24T23:58:19.270280+0000","last_became_peered":"2024-04-24T23:58:19.270280+0000","last_unstale":"2024-04-24T23:58:20.678418+0000","last_undegraded":"2024-04-24T23:58:20.678418+0000","last_fullsized":"2024-04-24T23:58:20.678418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254502+0000","last_change":"2024-04-24T23:58:19.252635+0000","last_active":"2024-04-24T23:58:20.254502+0000","last_peered":"2024-04-24T23:58:20.254502+0000","last_clean":"2024-04-24T23:58:20.254502+0000","last_became_active":"2024-04-24T23:58:19.252352+0000","last_became_peered":"2024-04-24T23:58:19.252352+0000","last_unstale":"2024-04-24T23:58:20.254502+0000","last_undegraded":"2024-04-24T23:58:20.254502+0000","last_fullsized":"2024-04-24T23:58:20.254502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254874+0000","last_change":"2024-04-24T23:58:19.257041+0000","last_active":"2024-04-24T23:58:20.254874+0000","last_peered":"2024-04-24T23:58:20.254874+0000","last_clean":"2024-04-24T23:58:20.254874+0000","last_became_active":"2024-04-24T23:58:19.256765+0000","last_became_peered":"2024-04-24T23:58:19.256765+0000","last_unstale":"2024-04-24T23:58:20.254874+0000","last_undegraded":"2024-04-24T23:58:20.254874+0000","last_fullsized":"2024-04-24T23:58:20.254874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.271981+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.271589+0000","last_became_peered":"2024-04-24T23:58:19.271589+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678540+0000","last_change":"2024-04-24T23:58:19.260799+0000","last_active":"2024-04-24T23:58:20.678540+0000","last_peered":"2024-04-24T23:58:20.678540+0000","last_clean":"2024-04-24T23:58:20.678540+0000","last_became_active":"2024-04-24T23:58:19.260437+0000","last_became_peered":"2024-04-24T23:58:19.260437+0000","last_unstale":"2024-04-24T23:58:20.678540+0000","last_undegraded":"2024-04-24T23:58:20.678540+0000","last_fullsized":"2024-04-24T23:58:20.678540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677754+0000","last_change":"2024-04-24T23:58:19.269955+0000","last_active":"2024-04-24T23:58:20.677754+0000","last_peered":"2024-04-24T23:58:20.677754+0000","last_clean":"2024-04-24T23:58:20.677754+0000","last_became_active":"2024-04-24T23:58:19.269675+0000","last_became_peered":"2024-04-24T23:58:19.269675+0000","last_unstale":"2024-04-24T23:58:20.677754+0000","last_undegraded":"2024-04-24T23:58:20.677754+0000","last_fullsized":"2024-04-24T23:58:20.677754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678346+0000","last_change":"2024-04-24T23:58:19.267752+0000","last_active":"2024-04-24T23:58:20.678346+0000","last_peered":"2024-04-24T23:58:20.678346+0000","last_clean":"2024-04-24T23:58:20.678346+0000","last_became_active":"2024-04-24T23:58:19.267463+0000","last_became_peered":"2024-04-24T23:58:19.267463+0000","last_unstale":"2024-04-24T23:58:20.678346+0000","last_undegraded":"2024-04-24T23:58:20.678346+0000","last_fullsized":"2024-04-24T23:58:20.678346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678319+0000","last_change":"2024-04-24T23:58:19.266736+0000","last_active":"2024-04-24T23:58:20.678319+0000","last_peered":"2024-04-24T23:58:20.678319+0000","last_clean":"2024-04-24T23:58:20.678319+0000","last_became_active":"2024-04-24T23:58:19.266438+0000","last_became_peered":"2024-04-24T23:58:19.266438+0000","last_unstale":"2024-04-24T23:58:20.678319+0000","last_undegraded":"2024-04-24T23:58:20.678319+0000","last_fullsized":"2024-04-24T23:58:20.678319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678538+0000","last_change":"2024-04-24T23:58:19.271086+0000","last_active":"2024-04-24T23:58:20.678538+0000","last_peered":"2024-04-24T23:58:20.678538+0000","last_clean":"2024-04-24T23:58:20.678538+0000","last_became_active":"2024-04-24T23:58:19.270722+0000","last_became_peered":"2024-04-24T23:58:19.270722+0000","last_unstale":"2024-04-24T23:58:20.678538+0000","last_undegraded":"2024-04-24T23:58:20.678538+0000","last_fullsized":"2024-04-24T23:58:20.678538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257102+0000","last_change":"2024-04-24T23:58:19.266845+0000","last_active":"2024-04-24T23:58:20.257102+0000","last_peered":"2024-04-24T23:58:20.257102+0000","last_clean":"2024-04-24T23:58:20.257102+0000","last_became_active":"2024-04-24T23:58:19.266573+0000","last_became_peered":"2024-04-24T23:58:19.266573+0000","last_unstale":"2024-04-24T23:58:20.257102+0000","last_undegraded":"2024-04-24T23:58:20.257102+0000","last_fullsized":"2024-04-24T23:58:20.257102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256863+0000","last_change":"2024-04-24T23:58:19.267588+0000","last_active":"2024-04-24T23:58:20.256863+0000","last_peered":"2024-04-24T23:58:20.256863+0000","last_clean":"2024-04-24T23:58:20.256863+0000","last_became_active":"2024-04-24T23:58:19.267247+0000","last_became_peered":"2024-04-24T23:58:19.267247+0000","last_unstale":"2024-04-24T23:58:20.256863+0000","last_undegraded":"2024-04-24T23:58:20.256863+0000","last_fullsized":"2024-04-24T23:58:20.256863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255357+0000","last_change":"2024-04-24T23:58:19.257310+0000","last_active":"2024-04-24T23:58:20.255357+0000","last_peered":"2024-04-24T23:58:20.255357+0000","last_clean":"2024-04-24T23:58:20.255357+0000","last_became_active":"2024-04-24T23:58:19.257023+0000","last_became_peered":"2024-04-24T23:58:19.257023+0000","last_unstale":"2024-04-24T23:58:20.255357+0000","last_undegraded":"2024-04-24T23:58:20.255357+0000","last_fullsized":"2024-04-24T23:58:20.255357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678413+0000","last_change":"2024-04-24T23:58:19.261091+0000","last_active":"2024-04-24T23:58:20.678413+0000","last_peered":"2024-04-24T23:58:20.678413+0000","last_clean":"2024-04-24T23:58:20.678413+0000","last_became_active":"2024-04-24T23:58:19.260837+0000","last_became_peered":"2024-04-24T23:58:19.260837+0000","last_unstale":"2024-04-24T23:58:20.678413+0000","last_undegraded":"2024-04-24T23:58:20.678413+0000","last_fullsized":"2024-04-24T23:58:20.678413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254811+0000","last_change":"2024-04-24T23:58:19.261328+0000","last_active":"2024-04-24T23:58:20.254811+0000","last_peered":"2024-04-24T23:58:20.254811+0000","last_clean":"2024-04-24T23:58:20.254811+0000","last_became_active":"2024-04-24T23:58:19.261045+0000","last_became_peered":"2024-04-24T23:58:19.261045+0000","last_unstale":"2024-04-24T23:58:20.254811+0000","last_undegraded":"2024-04-24T23:58:20.254811+0000","last_fullsized":"2024-04-24T23:58:20.254811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254583+0000","last_change":"2024-04-24T23:58:19.256356+0000","last_active":"2024-04-24T23:58:20.254583+0000","last_peered":"2024-04-24T23:58:20.254583+0000","last_clean":"2024-04-24T23:58:20.254583+0000","last_became_active":"2024-04-24T23:58:19.256099+0000","last_became_peered":"2024-04-24T23:58:19.256099+0000","last_unstale":"2024-04-24T23:58:20.254583+0000","last_undegraded":"2024-04-24T23:58:20.254583+0000","last_fullsized":"2024-04-24T23:58:20.254583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677420+0000","last_change":"2024-04-24T23:58:19.268419+0000","last_active":"2024-04-24T23:58:20.677420+0000","last_peered":"2024-04-24T23:58:20.677420+0000","last_clean":"2024-04-24T23:58:20.677420+0000","last_became_active":"2024-04-24T23:58:19.265742+0000","last_became_peered":"2024-04-24T23:58:19.265742+0000","last_unstale":"2024-04-24T23:58:20.677420+0000","last_undegraded":"2024-04-24T23:58:20.677420+0000","last_fullsized":"2024-04-24T23:58:20.677420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.273024+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.272551+0000","last_became_peered":"2024-04-24T23:58:19.272551+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.258361+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.258098+0000","last_became_peered":"2024-04-24T23:58:19.258098+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678669+0000","last_change":"2024-04-24T23:58:19.263085+0000","last_active":"2024-04-24T23:58:20.678669+0000","last_peered":"2024-04-24T23:58:20.678669+0000","last_clean":"2024-04-24T23:58:20.678669+0000","last_became_active":"2024-04-24T23:58:19.262807+0000","last_became_peered":"2024-04-24T23:58:19.262807+0000","last_unstale":"2024-04-24T23:58:20.678669+0000","last_undegraded":"2024-04-24T23:58:20.678669+0000","last_fullsized":"2024-04-24T23:58:20.678669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256867+0000","last_change":"2024-04-24T23:58:19.259501+0000","last_active":"2024-04-24T23:58:20.256867+0000","last_peered":"2024-04-24T23:58:20.256867+0000","last_clean":"2024-04-24T23:58:20.256867+0000","last_became_active":"2024-04-24T23:58:19.259233+0000","last_became_peered":"2024-04-24T23:58:19.259233+0000","last_unstale":"2024-04-24T23:58:20.256867+0000","last_undegraded":"2024-04-24T23:58:20.256867+0000","last_fullsized":"2024-04-24T23:58:20.256867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255041+0000","last_change":"2024-04-24T23:58:19.259960+0000","last_active":"2024-04-24T23:58:20.255041+0000","last_peered":"2024-04-24T23:58:20.255041+0000","last_clean":"2024-04-24T23:58:20.255041+0000","last_became_active":"2024-04-24T23:58:19.259548+0000","last_became_peered":"2024-04-24T23:58:19.259548+0000","last_unstale":"2024-04-24T23:58:20.255041+0000","last_undegraded":"2024-04-24T23:58:20.255041+0000","last_fullsized":"2024-04-24T23:58:20.255041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677851+0000","last_change":"2024-04-24T23:58:19.275499+0000","last_active":"2024-04-24T23:58:20.677851+0000","last_peered":"2024-04-24T23:58:20.677851+0000","last_clean":"2024-04-24T23:58:20.677851+0000","last_became_active":"2024-04-24T23:58:19.275213+0000","last_became_peered":"2024-04-24T23:58:19.275213+0000","last_unstale":"2024-04-24T23:58:20.677851+0000","last_undegraded":"2024-04-24T23:58:20.677851+0000","last_fullsized":"2024-04-24T23:58:20.677851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678362+0000","last_change":"2024-04-24T23:58:19.261963+0000","last_active":"2024-04-24T23:58:20.678362+0000","last_peered":"2024-04-24T23:58:20.678362+0000","last_clean":"2024-04-24T23:58:20.678362+0000","last_became_active":"2024-04-24T23:58:19.261710+0000","last_became_peered":"2024-04-24T23:58:19.261710+0000","last_unstale":"2024-04-24T23:58:20.678362+0000","last_undegraded":"2024-04-24T23:58:20.678362+0000","last_fullsized":"2024-04-24T23:58:20.678362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677517+0000","last_change":"2024-04-24T23:58:19.268171+0000","last_active":"2024-04-24T23:58:20.677517+0000","last_peered":"2024-04-24T23:58:20.677517+0000","last_clean":"2024-04-24T23:58:20.677517+0000","last_became_active":"2024-04-24T23:58:19.265379+0000","last_became_peered":"2024-04-24T23:58:19.265379+0000","last_unstale":"2024-04-24T23:58:20.677517+0000","last_undegraded":"2024-04-24T23:58:20.677517+0000","last_fullsized":"2024-04-24T23:58:20.677517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678064+0000","last_change":"2024-04-24T23:58:19.259615+0000","last_active":"2024-04-24T23:58:20.678064+0000","last_peered":"2024-04-24T23:58:20.678064+0000","last_clean":"2024-04-24T23:58:20.678064+0000","last_became_active":"2024-04-24T23:58:19.259355+0000","last_became_peered":"2024-04-24T23:58:19.259355+0000","last_unstale":"2024-04-24T23:58:20.678064+0000","last_undegraded":"2024-04-24T23:58:20.678064+0000","last_fullsized":"2024-04-24T23:58:20.678064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257106+0000","last_change":"2024-04-24T23:58:19.269905+0000","last_active":"2024-04-24T23:58:20.257106+0000","last_peered":"2024-04-24T23:58:20.257106+0000","last_clean":"2024-04-24T23:58:20.257106+0000","last_became_active":"2024-04-24T23:58:19.269639+0000","last_became_peered":"2024-04-24T23:58:19.269639+0000","last_unstale":"2024-04-24T23:58:20.257106+0000","last_undegraded":"2024-04-24T23:58:20.257106+0000","last_fullsized":"2024-04-24T23:58:20.257106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254304+0000","last_change":"2024-04-24T23:58:19.256747+0000","last_active":"2024-04-24T23:58:20.254304+0000","last_peered":"2024-04-24T23:58:20.254304+0000","last_clean":"2024-04-24T23:58:20.254304+0000","last_became_active":"2024-04-24T23:58:19.255723+0000","last_became_peered":"2024-04-24T23:58:19.255723+0000","last_unstale":"2024-04-24T23:58:20.254304+0000","last_undegraded":"2024-04-24T23:58:20.254304+0000","last_fullsized":"2024-04-24T23:58:20.254304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678051+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678051+0000","last_peered":"2024-04-24T23:58:20.678051+0000","last_clean":"2024-04-24T23:58:20.678051+0000","last_became_active":"2024-04-24T23:58:19.262977+0000","last_became_peered":"2024-04-24T23:58:19.262977+0000","last_unstale":"2024-04-24T23:58:20.678051+0000","last_undegraded":"2024-04-24T23:58:20.678051+0000","last_fullsized":"2024-04-24T23:58:20.678051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255177+0000","last_change":"2024-04-24T23:58:19.259745+0000","last_active":"2024-04-24T23:58:20.255177+0000","last_peered":"2024-04-24T23:58:20.255177+0000","last_clean":"2024-04-24T23:58:20.255177+0000","last_became_active":"2024-04-24T23:58:19.259201+0000","last_became_peered":"2024-04-24T23:58:19.259201+0000","last_unstale":"2024-04-24T23:58:20.255177+0000","last_undegraded":"2024-04-24T23:58:20.255177+0000","last_fullsized":"2024-04-24T23:58:20.255177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678071+0000","last_change":"2024-04-24T23:58:19.273016+0000","last_active":"2024-04-24T23:58:20.678071+0000","last_peered":"2024-04-24T23:58:20.678071+0000","last_clean":"2024-04-24T23:58:20.678071+0000","last_became_active":"2024-04-24T23:58:19.271540+0000","last_became_peered":"2024-04-24T23:58:19.271540+0000","last_unstale":"2024-04-24T23:58:20.678071+0000","last_undegraded":"2024-04-24T23:58:20.678071+0000","last_fullsized":"2024-04-24T23:58:20.678071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677646+0000","last_change":"2024-04-24T23:58:19.255458+0000","last_active":"2024-04-24T23:58:20.677646+0000","last_peered":"2024-04-24T23:58:20.677646+0000","last_clean":"2024-04-24T23:58:20.677646+0000","last_became_active":"2024-04-24T23:58:19.255070+0000","last_became_peered":"2024-04-24T23:58:19.255070+0000","last_unstale":"2024-04-24T23:58:20.677646+0000","last_undegraded":"2024-04-24T23:58:20.677646+0000","last_fullsized":"2024-04-24T23:58:20.677646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256632+0000","last_change":"2024-04-24T23:58:19.266657+0000","last_active":"2024-04-24T23:58:20.256632+0000","last_peered":"2024-04-24T23:58:20.256632+0000","last_clean":"2024-04-24T23:58:20.256632+0000","last_became_active":"2024-04-24T23:58:19.266354+0000","last_became_peered":"2024-04-24T23:58:19.266354+0000","last_unstale":"2024-04-24T23:58:20.256632+0000","last_undegraded":"2024-04-24T23:58:20.256632+0000","last_fullsized":"2024-04-24T23:58:20.256632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255049+0000","last_change":"2024-04-24T23:58:19.259253+0000","last_active":"2024-04-24T23:58:20.255049+0000","last_peered":"2024-04-24T23:58:20.255049+0000","last_clean":"2024-04-24T23:58:20.255049+0000","last_became_active":"2024-04-24T23:58:19.258553+0000","last_became_peered":"2024-04-24T23:58:19.258553+0000","last_unstale":"2024-04-24T23:58:20.255049+0000","last_undegraded":"2024-04-24T23:58:20.255049+0000","last_fullsized":"2024-04-24T23:58:20.255049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:53.284799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678523+0000","last_change":"2024-04-24T23:58:18.250614+0000","last_active":"2024-04-24T23:58:20.678523+0000","last_peered":"2024-04-24T23:58:20.678523+0000","last_clean":"2024-04-24T23:58:20.678523+0000","last_became_active":"2024-04-24T23:58:18.250306+0000","last_became_peered":"2024-04-24T23:58:18.250306+0000","last_unstale":"2024-04-24T23:58:20.678523+0000","last_undegraded":"2024-04-24T23:58:20.678523+0000","last_fullsized":"2024-04-24T23:58:20.678523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255184+0000","last_change":"2024-04-24T23:58:19.251648+0000","last_active":"2024-04-24T23:58:20.255184+0000","last_peered":"2024-04-24T23:58:20.255184+0000","last_clean":"2024-04-24T23:58:20.255184+0000","last_became_active":"2024-04-24T23:58:19.251271+0000","last_became_peered":"2024-04-24T23:58:19.251271+0000","last_unstale":"2024-04-24T23:58:20.255184+0000","last_undegraded":"2024-04-24T23:58:20.255184+0000","last_fullsized":"2024-04-24T23:58:20.255184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256708+0000","last_change":"2024-04-24T23:58:18.252764+0000","last_active":"2024-04-24T23:58:20.256708+0000","last_peered":"2024-04-24T23:58:20.256708+0000","last_clean":"2024-04-24T23:58:20.256708+0000","last_became_active":"2024-04-24T23:58:18.252491+0000","last_became_peered":"2024-04-24T23:58:18.252491+0000","last_unstale":"2024-04-24T23:58:20.256708+0000","last_undegraded":"2024-04-24T23:58:20.256708+0000","last_fullsized":"2024-04-24T23:58:20.256708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678357+0000","last_change":"2024-04-24T23:58:18.250003+0000","last_active":"2024-04-24T23:58:20.678357+0000","last_peered":"2024-04-24T23:58:20.678357+0000","last_clean":"2024-04-24T23:58:20.678357+0000","last_became_active":"2024-04-24T23:58:18.249727+0000","last_became_peered":"2024-04-24T23:58:18.249727+0000","last_unstale":"2024-04-24T23:58:20.678357+0000","last_undegraded":"2024-04-24T23:58:20.678357+0000","last_fullsized":"2024-04-24T23:58:20.678357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.318157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678399+0000","last_change":"2024-04-24T23:58:19.255033+0000","last_active":"2024-04-24T23:58:20.678399+0000","last_peered":"2024-04-24T23:58:20.678399+0000","last_clean":"2024-04-24T23:58:20.678399+0000","last_became_active":"2024-04-24T23:58:19.254746+0000","last_became_peered":"2024-04-24T23:58:19.254746+0000","last_unstale":"2024-04-24T23:58:20.678399+0000","last_undegraded":"2024-04-24T23:58:20.678399+0000","last_fullsized":"2024-04-24T23:58:20.678399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677538+0000","last_change":"2024-04-24T23:58:18.245516+0000","last_active":"2024-04-24T23:58:20.677538+0000","last_peered":"2024-04-24T23:58:20.677538+0000","last_clean":"2024-04-24T23:58:20.677538+0000","last_became_active":"2024-04-24T23:58:18.245212+0000","last_became_peered":"2024-04-24T23:58:18.245212+0000","last_unstale":"2024-04-24T23:58:20.677538+0000","last_undegraded":"2024-04-24T23:58:20.677538+0000","last_fullsized":"2024-04-24T23:58:20.677538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254479+0000","last_change":"2024-04-24T23:58:19.258354+0000","last_active":"2024-04-24T23:58:20.254479+0000","last_peered":"2024-04-24T23:58:20.254479+0000","last_clean":"2024-04-24T23:58:20.254479+0000","last_became_active":"2024-04-24T23:58:19.258067+0000","last_became_peered":"2024-04-24T23:58:19.258067+0000","last_unstale":"2024-04-24T23:58:20.254479+0000","last_undegraded":"2024-04-24T23:58:20.254479+0000","last_fullsized":"2024-04-24T23:58:20.254479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256985+0000","last_change":"2024-04-24T23:58:18.253806+0000","last_active":"2024-04-24T23:58:20.256985+0000","last_peered":"2024-04-24T23:58:20.256985+0000","last_clean":"2024-04-24T23:58:20.256985+0000","last_became_active":"2024-04-24T23:58:18.252657+0000","last_became_peered":"2024-04-24T23:58:18.252657+0000","last_unstale":"2024-04-24T23:58:20.256985+0000","last_undegraded":"2024-04-24T23:58:20.256985+0000","last_fullsized":"2024-04-24T23:58:20.256985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677952+0000","last_change":"2024-04-24T23:58:19.275012+0000","last_active":"2024-04-24T23:58:20.677952+0000","last_peered":"2024-04-24T23:58:20.677952+0000","last_clean":"2024-04-24T23:58:20.677952+0000","last_became_active":"2024-04-24T23:58:19.274740+0000","last_became_peered":"2024-04-24T23:58:19.274740+0000","last_unstale":"2024-04-24T23:58:20.677952+0000","last_undegraded":"2024-04-24T23:58:20.677952+0000","last_fullsized":"2024-04-24T23:58:20.677952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678001+0000","last_change":"2024-04-24T23:58:18.251763+0000","last_active":"2024-04-24T23:58:20.678001+0000","last_peered":"2024-04-24T23:58:20.678001+0000","last_clean":"2024-04-24T23:58:20.678001+0000","last_became_active":"2024-04-24T23:58:18.251508+0000","last_became_peered":"2024-04-24T23:58:18.251508+0000","last_unstale":"2024-04-24T23:58:20.678001+0000","last_undegraded":"2024-04-24T23:58:20.678001+0000","last_fullsized":"2024-04-24T23:58:20.678001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254979+0000","last_change":"2024-04-24T23:58:19.253275+0000","last_active":"2024-04-24T23:58:20.254979+0000","last_peered":"2024-04-24T23:58:20.254979+0000","last_clean":"2024-04-24T23:58:20.254979+0000","last_became_active":"2024-04-24T23:58:19.252783+0000","last_became_peered":"2024-04-24T23:58:19.252783+0000","last_unstale":"2024-04-24T23:58:20.254979+0000","last_undegraded":"2024-04-24T23:58:20.254979+0000","last_fullsized":"2024-04-24T23:58:20.254979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254408+0000","last_change":"2024-04-24T23:58:18.246446+0000","last_active":"2024-04-24T23:58:20.254408+0000","last_peered":"2024-04-24T23:58:20.254408+0000","last_clean":"2024-04-24T23:58:20.254408+0000","last_became_active":"2024-04-24T23:58:18.246036+0000","last_became_peered":"2024-04-24T23:58:18.246036+0000","last_unstale":"2024-04-24T23:58:20.254408+0000","last_undegraded":"2024-04-24T23:58:20.254408+0000","last_fullsized":"2024-04-24T23:58:20.254408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255261+0000","last_change":"2024-04-24T23:58:19.261655+0000","last_active":"2024-04-24T23:58:20.255261+0000","last_peered":"2024-04-24T23:58:20.255261+0000","last_clean":"2024-04-24T23:58:20.255261+0000","last_became_active":"2024-04-24T23:58:19.261331+0000","last_became_peered":"2024-04-24T23:58:19.261331+0000","last_unstale":"2024-04-24T23:58:20.255261+0000","last_undegraded":"2024-04-24T23:58:20.255261+0000","last_fullsized":"2024-04-24T23:58:20.255261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256970+0000","last_change":"2024-04-24T23:58:18.247264+0000","last_active":"2024-04-24T23:58:20.256970+0000","last_peered":"2024-04-24T23:58:20.256970+0000","last_clean":"2024-04-24T23:58:20.256970+0000","last_became_active":"2024-04-24T23:58:18.246959+0000","last_became_peered":"2024-04-24T23:58:18.246959+0000","last_unstale":"2024-04-24T23:58:20.256970+0000","last_undegraded":"2024-04-24T23:58:20.256970+0000","last_fullsized":"2024-04-24T23:58:20.256970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678015+0000","last_change":"2024-04-24T23:58:19.265255+0000","last_active":"2024-04-24T23:58:20.678015+0000","last_peered":"2024-04-24T23:58:20.678015+0000","last_clean":"2024-04-24T23:58:20.678015+0000","last_became_active":"2024-04-24T23:58:19.264997+0000","last_became_peered":"2024-04-24T23:58:19.264997+0000","last_unstale":"2024-04-24T23:58:20.678015+0000","last_undegraded":"2024-04-24T23:58:20.678015+0000","last_fullsized":"2024-04-24T23:58:20.678015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255275+0000","last_change":"2024-04-24T23:58:18.236929+0000","last_active":"2024-04-24T23:58:20.255275+0000","last_peered":"2024-04-24T23:58:20.255275+0000","last_clean":"2024-04-24T23:58:20.255275+0000","last_became_active":"2024-04-24T23:58:18.236660+0000","last_became_peered":"2024-04-24T23:58:18.236660+0000","last_unstale":"2024-04-24T23:58:20.255275+0000","last_undegraded":"2024-04-24T23:58:20.255275+0000","last_fullsized":"2024-04-24T23:58:20.255275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254944+0000","last_change":"2024-04-24T23:58:19.259532+0000","last_active":"2024-04-24T23:58:20.254944+0000","last_peered":"2024-04-24T23:58:20.254944+0000","last_clean":"2024-04-24T23:58:20.254944+0000","last_became_active":"2024-04-24T23:58:19.259298+0000","last_became_peered":"2024-04-24T23:58:19.259298+0000","last_unstale":"2024-04-24T23:58:20.254944+0000","last_undegraded":"2024-04-24T23:58:20.254944+0000","last_fullsized":"2024-04-24T23:58:20.254944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678194+0000","last_change":"2024-04-24T23:58:18.251116+0000","last_active":"2024-04-24T23:58:20.678194+0000","last_peered":"2024-04-24T23:58:20.678194+0000","last_clean":"2024-04-24T23:58:20.678194+0000","last_became_active":"2024-04-24T23:58:18.250814+0000","last_became_peered":"2024-04-24T23:58:18.250814+0000","last_unstale":"2024-04-24T23:58:20.678194+0000","last_undegraded":"2024-04-24T23:58:20.678194+0000","last_fullsized":"2024-04-24T23:58:20.678194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678564+0000","last_change":"2024-04-24T23:58:19.263724+0000","last_active":"2024-04-24T23:58:20.678564+0000","last_peered":"2024-04-24T23:58:20.678564+0000","last_clean":"2024-04-24T23:58:20.678564+0000","last_became_active":"2024-04-24T23:58:19.263448+0000","last_became_peered":"2024-04-24T23:58:19.263448+0000","last_unstale":"2024-04-24T23:58:20.678564+0000","last_undegraded":"2024-04-24T23:58:20.678564+0000","last_fullsized":"2024-04-24T23:58:20.678564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256648+0000","last_change":"2024-04-24T23:58:18.249529+0000","last_active":"2024-04-24T23:58:20.256648+0000","last_peered":"2024-04-24T23:58:20.256648+0000","last_clean":"2024-04-24T23:58:20.256648+0000","last_became_active":"2024-04-24T23:58:18.249135+0000","last_became_peered":"2024-04-24T23:58:18.249135+0000","last_unstale":"2024-04-24T23:58:20.256648+0000","last_undegraded":"2024-04-24T23:58:20.256648+0000","last_fullsized":"2024-04-24T23:58:20.256648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256714+0000","last_change":"2024-04-24T23:58:19.266083+0000","last_active":"2024-04-24T23:58:20.256714+0000","last_peered":"2024-04-24T23:58:20.256714+0000","last_clean":"2024-04-24T23:58:20.256714+0000","last_became_active":"2024-04-24T23:58:19.265757+0000","last_became_peered":"2024-04-24T23:58:19.265757+0000","last_unstale":"2024-04-24T23:58:20.256714+0000","last_undegraded":"2024-04-24T23:58:20.256714+0000","last_fullsized":"2024-04-24T23:58:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677645+0000","last_change":"2024-04-24T23:58:18.245808+0000","last_active":"2024-04-24T23:58:20.677645+0000","last_peered":"2024-04-24T23:58:20.677645+0000","last_clean":"2024-04-24T23:58:20.677645+0000","last_became_active":"2024-04-24T23:58:18.244769+0000","last_became_peered":"2024-04-24T23:58:18.244769+0000","last_unstale":"2024-04-24T23:58:20.677645+0000","last_undegraded":"2024-04-24T23:58:20.677645+0000","last_fullsized":"2024-04-24T23:58:20.677645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678393+0000","last_change":"2024-04-24T23:58:19.262773+0000","last_active":"2024-04-24T23:58:20.678393+0000","last_peered":"2024-04-24T23:58:20.678393+0000","last_clean":"2024-04-24T23:58:20.678393+0000","last_became_active":"2024-04-24T23:58:19.262384+0000","last_became_peered":"2024-04-24T23:58:19.262384+0000","last_unstale":"2024-04-24T23:58:20.678393+0000","last_undegraded":"2024-04-24T23:58:20.678393+0000","last_fullsized":"2024-04-24T23:58:20.678393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255516+0000","last_change":"2024-04-24T23:58:18.243528+0000","last_active":"2024-04-24T23:58:20.255516+0000","last_peered":"2024-04-24T23:58:20.255516+0000","last_clean":"2024-04-24T23:58:20.255516+0000","last_became_active":"2024-04-24T23:58:18.243262+0000","last_became_peered":"2024-04-24T23:58:18.243262+0000","last_unstale":"2024-04-24T23:58:20.255516+0000","last_undegraded":"2024-04-24T23:58:20.255516+0000","last_fullsized":"2024-04-24T23:58:20.255516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677877+0000","last_change":"2024-04-24T23:58:19.261251+0000","last_active":"2024-04-24T23:58:20.677877+0000","last_peered":"2024-04-24T23:58:20.677877+0000","last_clean":"2024-04-24T23:58:20.677877+0000","last_became_active":"2024-04-24T23:58:19.260977+0000","last_became_peered":"2024-04-24T23:58:19.260977+0000","last_unstale":"2024-04-24T23:58:20.677877+0000","last_undegraded":"2024-04-24T23:58:20.677877+0000","last_fullsized":"2024-04-24T23:58:20.677877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677909+0000","last_change":"2024-04-24T23:58:18.242128+0000","last_active":"2024-04-24T23:58:20.677909+0000","last_peered":"2024-04-24T23:58:20.677909+0000","last_clean":"2024-04-24T23:58:20.677909+0000","last_became_active":"2024-04-24T23:58:18.241804+0000","last_became_peered":"2024-04-24T23:58:18.241804+0000","last_unstale":"2024-04-24T23:58:20.677909+0000","last_undegraded":"2024-04-24T23:58:20.677909+0000","last_fullsized":"2024-04-24T23:58:20.677909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678616+0000","last_change":"2024-04-24T23:58:19.262407+0000","last_active":"2024-04-24T23:58:20.678616+0000","last_peered":"2024-04-24T23:58:20.678616+0000","last_clean":"2024-04-24T23:58:20.678616+0000","last_became_active":"2024-04-24T23:58:19.262123+0000","last_became_peered":"2024-04-24T23:58:19.262123+0000","last_unstale":"2024-04-24T23:58:20.678616+0000","last_undegraded":"2024-04-24T23:58:20.678616+0000","last_fullsized":"2024-04-24T23:58:20.678616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254597+0000","last_change":"2024-04-24T23:58:18.242601+0000","last_active":"2024-04-24T23:58:20.254597+0000","last_peered":"2024-04-24T23:58:20.254597+0000","last_clean":"2024-04-24T23:58:20.254597+0000","last_became_active":"2024-04-24T23:58:18.242314+0000","last_became_peered":"2024-04-24T23:58:18.242314+0000","last_unstale":"2024-04-24T23:58:20.254597+0000","last_undegraded":"2024-04-24T23:58:20.254597+0000","last_fullsized":"2024-04-24T23:58:20.254597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678172+0000","last_change":"2024-04-24T23:58:19.265899+0000","last_active":"2024-04-24T23:58:20.678172+0000","last_peered":"2024-04-24T23:58:20.678172+0000","last_clean":"2024-04-24T23:58:20.678172+0000","last_became_active":"2024-04-24T23:58:19.265596+0000","last_became_peered":"2024-04-24T23:58:19.265596+0000","last_unstale":"2024-04-24T23:58:20.678172+0000","last_undegraded":"2024-04-24T23:58:20.678172+0000","last_fullsized":"2024-04-24T23:58:20.678172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677741+0000","last_change":"2024-04-24T23:58:18.248287+0000","last_active":"2024-04-24T23:58:20.677741+0000","last_peered":"2024-04-24T23:58:20.677741+0000","last_clean":"2024-04-24T23:58:20.677741+0000","last_became_active":"2024-04-24T23:58:18.248034+0000","last_became_peered":"2024-04-24T23:58:18.248034+0000","last_unstale":"2024-04-24T23:58:20.677741+0000","last_undegraded":"2024-04-24T23:58:20.677741+0000","last_fullsized":"2024-04-24T23:58:20.677741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678510+0000","last_change":"2024-04-24T23:58:19.274141+0000","last_active":"2024-04-24T23:58:20.678510+0000","last_peered":"2024-04-24T23:58:20.678510+0000","last_clean":"2024-04-24T23:58:20.678510+0000","last_became_active":"2024-04-24T23:58:19.273864+0000","last_became_peered":"2024-04-24T23:58:19.273864+0000","last_unstale":"2024-04-24T23:58:20.678510+0000","last_undegraded":"2024-04-24T23:58:20.678510+0000","last_fullsized":"2024-04-24T23:58:20.678510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677425+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:58:20.677425+0000","last_peered":"2024-04-24T23:58:20.677425+0000","last_clean":"2024-04-24T23:58:20.677425+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:58:20.677425+0000","last_undegraded":"2024-04-24T23:58:20.677425+0000","last_fullsized":"2024-04-24T23:58:20.677425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677567+0000","last_change":"2024-04-24T23:58:18.247362+0000","last_active":"2024-04-24T23:58:20.677567+0000","last_peered":"2024-04-24T23:58:20.677567+0000","last_clean":"2024-04-24T23:58:20.677567+0000","last_became_active":"2024-04-24T23:58:18.247084+0000","last_became_peered":"2024-04-24T23:58:18.247084+0000","last_unstale":"2024-04-24T23:58:20.677567+0000","last_undegraded":"2024-04-24T23:58:20.677567+0000","last_fullsized":"2024-04-24T23:58:20.677567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678143+0000","last_change":"2024-04-24T23:58:19.272925+0000","last_active":"2024-04-24T23:58:20.678143+0000","last_peered":"2024-04-24T23:58:20.678143+0000","last_clean":"2024-04-24T23:58:20.678143+0000","last_became_active":"2024-04-24T23:58:19.272538+0000","last_became_peered":"2024-04-24T23:58:19.272538+0000","last_unstale":"2024-04-24T23:58:20.678143+0000","last_undegraded":"2024-04-24T23:58:20.678143+0000","last_fullsized":"2024-04-24T23:58:20.678143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677369+0000","last_change":"2024-04-24T23:58:18.251207+0000","last_active":"2024-04-24T23:58:20.677369+0000","last_peered":"2024-04-24T23:58:20.677369+0000","last_clean":"2024-04-24T23:58:20.677369+0000","last_became_active":"2024-04-24T23:58:18.250921+0000","last_became_peered":"2024-04-24T23:58:18.250921+0000","last_unstale":"2024-04-24T23:58:20.677369+0000","last_undegraded":"2024-04-24T23:58:20.677369+0000","last_fullsized":"2024-04-24T23:58:20.677369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254623+0000","last_change":"2024-04-24T23:58:19.248221+0000","last_active":"2024-04-24T23:58:20.254623+0000","last_peered":"2024-04-24T23:58:20.254623+0000","last_clean":"2024-04-24T23:58:20.254623+0000","last_became_active":"2024-04-24T23:58:19.247896+0000","last_became_peered":"2024-04-24T23:58:19.247896+0000","last_unstale":"2024-04-24T23:58:20.254623+0000","last_undegraded":"2024-04-24T23:58:20.254623+0000","last_fullsized":"2024-04-24T23:58:20.254623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:24.816499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677678+0000","last_change":"2024-04-24T23:58:18.245116+0000","last_active":"2024-04-24T23:58:20.677678+0000","last_peered":"2024-04-24T23:58:20.677678+0000","last_clean":"2024-04-24T23:58:20.677678+0000","last_became_active":"2024-04-24T23:58:18.244779+0000","last_became_peered":"2024-04-24T23:58:18.244779+0000","last_unstale":"2024-04-24T23:58:20.677678+0000","last_undegraded":"2024-04-24T23:58:20.677678+0000","last_fullsized":"2024-04-24T23:58:20.677678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254292+0000","last_change":"2024-04-24T23:58:19.258623+0000","last_active":"2024-04-24T23:58:20.254292+0000","last_peered":"2024-04-24T23:58:20.254292+0000","last_clean":"2024-04-24T23:58:20.254292+0000","last_became_active":"2024-04-24T23:58:19.258371+0000","last_became_peered":"2024-04-24T23:58:19.258371+0000","last_unstale":"2024-04-24T23:58:20.254292+0000","last_undegraded":"2024-04-24T23:58:20.254292+0000","last_fullsized":"2024-04-24T23:58:20.254292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678304+0000","last_change":"2024-04-24T23:58:18.250837+0000","last_active":"2024-04-24T23:58:20.678304+0000","last_peered":"2024-04-24T23:58:20.678304+0000","last_clean":"2024-04-24T23:58:20.678304+0000","last_became_active":"2024-04-24T23:58:18.250540+0000","last_became_peered":"2024-04-24T23:58:18.250540+0000","last_unstale":"2024-04-24T23:58:20.678304+0000","last_undegraded":"2024-04-24T23:58:20.678304+0000","last_fullsized":"2024-04-24T23:58:20.678304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:13.906429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255478+0000","last_change":"2024-04-24T23:58:19.254379+0000","last_active":"2024-04-24T23:58:20.255478+0000","last_peered":"2024-04-24T23:58:20.255478+0000","last_clean":"2024-04-24T23:58:20.255478+0000","last_became_active":"2024-04-24T23:58:19.252734+0000","last_became_peered":"2024-04-24T23:58:19.252734+0000","last_unstale":"2024-04-24T23:58:20.255478+0000","last_undegraded":"2024-04-24T23:58:20.255478+0000","last_fullsized":"2024-04-24T23:58:20.255478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:59.970682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254377+0000","last_change":"2024-04-24T23:58:18.247304+0000","last_active":"2024-04-24T23:58:20.254377+0000","last_peered":"2024-04-24T23:58:20.254377+0000","last_clean":"2024-04-24T23:58:20.254377+0000","last_became_active":"2024-04-24T23:58:18.247029+0000","last_became_peered":"2024-04-24T23:58:18.247029+0000","last_unstale":"2024-04-24T23:58:20.254377+0000","last_undegraded":"2024-04-24T23:58:20.254377+0000","last_fullsized":"2024-04-24T23:58:20.254377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678233+0000","last_change":"2024-04-24T23:58:19.268921+0000","last_active":"2024-04-24T23:58:20.678233+0000","last_peered":"2024-04-24T23:58:20.678233+0000","last_clean":"2024-04-24T23:58:20.678233+0000","last_became_active":"2024-04-24T23:58:19.265287+0000","last_became_peered":"2024-04-24T23:58:19.265287+0000","last_unstale":"2024-04-24T23:58:20.678233+0000","last_undegraded":"2024-04-24T23:58:20.678233+0000","last_fullsized":"2024-04-24T23:58:20.678233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678625+0000","last_change":"2024-04-24T23:58:18.249153+0000","last_active":"2024-04-24T23:58:20.678625+0000","last_peered":"2024-04-24T23:58:20.678625+0000","last_clean":"2024-04-24T23:58:20.678625+0000","last_became_active":"2024-04-24T23:58:18.248532+0000","last_became_peered":"2024-04-24T23:58:18.248532+0000","last_unstale":"2024-04-24T23:58:20.678625+0000","last_undegraded":"2024-04-24T23:58:20.678625+0000","last_fullsized":"2024-04-24T23:58:20.678625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254293+0000","last_change":"2024-04-24T23:58:19.257617+0000","last_active":"2024-04-24T23:58:20.254293+0000","last_peered":"2024-04-24T23:58:20.254293+0000","last_clean":"2024-04-24T23:58:20.254293+0000","last_became_active":"2024-04-24T23:58:19.257312+0000","last_became_peered":"2024-04-24T23:58:19.257312+0000","last_unstale":"2024-04-24T23:58:20.254293+0000","last_undegraded":"2024-04-24T23:58:20.254293+0000","last_fullsized":"2024-04-24T23:58:20.254293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256119+0000","last_change":"2024-04-24T23:58:18.249582+0000","last_active":"2024-04-24T23:58:20.256119+0000","last_peered":"2024-04-24T23:58:20.256119+0000","last_clean":"2024-04-24T23:58:20.256119+0000","last_became_active":"2024-04-24T23:58:18.249277+0000","last_became_peered":"2024-04-24T23:58:18.249277+0000","last_unstale":"2024-04-24T23:58:20.256119+0000","last_undegraded":"2024-04-24T23:58:20.256119+0000","last_fullsized":"2024-04-24T23:58:20.256119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678235+0000","last_change":"2024-04-24T23:58:19.265003+0000","last_active":"2024-04-24T23:58:20.678235+0000","last_peered":"2024-04-24T23:58:20.678235+0000","last_clean":"2024-04-24T23:58:20.678235+0000","last_became_active":"2024-04-24T23:58:19.264714+0000","last_became_peered":"2024-04-24T23:58:19.264714+0000","last_unstale":"2024-04-24T23:58:20.678235+0000","last_undegraded":"2024-04-24T23:58:20.678235+0000","last_fullsized":"2024-04-24T23:58:20.678235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257040+0000","last_change":"2024-04-24T23:58:18.250190+0000","last_active":"2024-04-24T23:58:20.257040+0000","last_peered":"2024-04-24T23:58:20.257040+0000","last_clean":"2024-04-24T23:58:20.257040+0000","last_became_active":"2024-04-24T23:58:18.249896+0000","last_became_peered":"2024-04-24T23:58:18.249896+0000","last_unstale":"2024-04-24T23:58:20.257040+0000","last_undegraded":"2024-04-24T23:58:20.257040+0000","last_fullsized":"2024-04-24T23:58:20.257040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255362+0000","last_change":"2024-04-24T23:58:19.261332+0000","last_active":"2024-04-24T23:58:20.255362+0000","last_peered":"2024-04-24T23:58:20.255362+0000","last_clean":"2024-04-24T23:58:20.255362+0000","last_became_active":"2024-04-24T23:58:19.261058+0000","last_became_peered":"2024-04-24T23:58:19.261058+0000","last_unstale":"2024-04-24T23:58:20.255362+0000","last_undegraded":"2024-04-24T23:58:20.255362+0000","last_fullsized":"2024-04-24T23:58:20.255362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677359+0000","last_change":"2024-04-24T23:58:18.251910+0000","last_active":"2024-04-24T23:58:20.677359+0000","last_peered":"2024-04-24T23:58:20.677359+0000","last_clean":"2024-04-24T23:58:20.677359+0000","last_became_active":"2024-04-24T23:58:18.251629+0000","last_became_peered":"2024-04-24T23:58:18.251629+0000","last_unstale":"2024-04-24T23:58:20.677359+0000","last_undegraded":"2024-04-24T23:58:20.677359+0000","last_fullsized":"2024-04-24T23:58:20.677359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255236+0000","last_change":"2024-04-24T23:58:19.256009+0000","last_active":"2024-04-24T23:58:20.255236+0000","last_peered":"2024-04-24T23:58:20.255236+0000","last_clean":"2024-04-24T23:58:20.255236+0000","last_became_active":"2024-04-24T23:58:19.255734+0000","last_became_peered":"2024-04-24T23:58:19.255734+0000","last_unstale":"2024-04-24T23:58:20.255236+0000","last_undegraded":"2024-04-24T23:58:20.255236+0000","last_fullsized":"2024-04-24T23:58:20.255236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:49.658803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254568+0000","last_change":"2024-04-24T23:58:18.250223+0000","last_active":"2024-04-24T23:58:20.254568+0000","last_peered":"2024-04-24T23:58:20.254568+0000","last_clean":"2024-04-24T23:58:20.254568+0000","last_became_active":"2024-04-24T23:58:18.249995+0000","last_became_peered":"2024-04-24T23:58:18.249995+0000","last_unstale":"2024-04-24T23:58:20.254568+0000","last_undegraded":"2024-04-24T23:58:20.254568+0000","last_fullsized":"2024-04-24T23:58:20.254568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.258258+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.258011+0000","last_became_peered":"2024-04-24T23:58:19.258011+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254356+0000","last_change":"2024-04-24T23:58:18.243148+0000","last_active":"2024-04-24T23:58:20.254356+0000","last_peered":"2024-04-24T23:58:20.254356+0000","last_clean":"2024-04-24T23:58:20.254356+0000","last_became_active":"2024-04-24T23:58:18.242863+0000","last_became_peered":"2024-04-24T23:58:18.242863+0000","last_unstale":"2024-04-24T23:58:20.254356+0000","last_undegraded":"2024-04-24T23:58:20.254356+0000","last_fullsized":"2024-04-24T23:58:20.254356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678415+0000","last_change":"2024-04-24T23:58:19.268382+0000","last_active":"2024-04-24T23:58:20.678415+0000","last_peered":"2024-04-24T23:58:20.678415+0000","last_clean":"2024-04-24T23:58:20.678415+0000","last_became_active":"2024-04-24T23:58:19.268132+0000","last_became_peered":"2024-04-24T23:58:19.268132+0000","last_unstale":"2024-04-24T23:58:20.678415+0000","last_undegraded":"2024-04-24T23:58:20.678415+0000","last_fullsized":"2024-04-24T23:58:20.678415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678646+0000","last_change":"2024-04-24T23:58:18.253346+0000","last_active":"2024-04-24T23:58:20.678646+0000","last_peered":"2024-04-24T23:58:20.678646+0000","last_clean":"2024-04-24T23:58:20.678646+0000","last_became_active":"2024-04-24T23:58:18.253094+0000","last_became_peered":"2024-04-24T23:58:18.253094+0000","last_unstale":"2024-04-24T23:58:20.678646+0000","last_undegraded":"2024-04-24T23:58:20.678646+0000","last_fullsized":"2024-04-24T23:58:20.678646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256888+0000","last_change":"2024-04-24T23:58:19.260209+0000","last_active":"2024-04-24T23:58:20.256888+0000","last_peered":"2024-04-24T23:58:20.256888+0000","last_clean":"2024-04-24T23:58:20.256888+0000","last_became_active":"2024-04-24T23:58:19.259941+0000","last_became_peered":"2024-04-24T23:58:19.259941+0000","last_unstale":"2024-04-24T23:58:20.256888+0000","last_undegraded":"2024-04-24T23:58:20.256888+0000","last_fullsized":"2024-04-24T23:58:20.256888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678038+0000","last_change":"2024-04-24T23:58:19.262656+0000","last_active":"2024-04-24T23:58:20.678038+0000","last_peered":"2024-04-24T23:58:20.678038+0000","last_clean":"2024-04-24T23:58:20.678038+0000","last_became_active":"2024-04-24T23:58:19.262043+0000","last_became_peered":"2024-04-24T23:58:19.262043+0000","last_unstale":"2024-04-24T23:58:20.678038+0000","last_undegraded":"2024-04-24T23:58:20.678038+0000","last_fullsized":"2024-04-24T23:58:20.678038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677720+0000","last_change":"2024-04-24T23:58:18.249958+0000","last_active":"2024-04-24T23:58:20.677720+0000","last_peered":"2024-04-24T23:58:20.677720+0000","last_clean":"2024-04-24T23:58:20.677720+0000","last_became_active":"2024-04-24T23:58:18.249660+0000","last_became_peered":"2024-04-24T23:58:18.249660+0000","last_unstale":"2024-04-24T23:58:20.677720+0000","last_undegraded":"2024-04-24T23:58:20.677720+0000","last_fullsized":"2024-04-24T23:58:20.677720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678297+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678297+0000","last_peered":"2024-04-24T23:58:20.678297+0000","last_clean":"2024-04-24T23:58:20.678297+0000","last_became_active":"2024-04-24T23:58:19.262976+0000","last_became_peered":"2024-04-24T23:58:19.262976+0000","last_unstale":"2024-04-24T23:58:20.678297+0000","last_undegraded":"2024-04-24T23:58:20.678297+0000","last_fullsized":"2024-04-24T23:58:20.678297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678466+0000","last_change":"2024-04-24T23:58:18.250117+0000","last_active":"2024-04-24T23:58:20.678466+0000","last_peered":"2024-04-24T23:58:20.678466+0000","last_clean":"2024-04-24T23:58:20.678466+0000","last_became_active":"2024-04-24T23:58:18.249797+0000","last_became_peered":"2024-04-24T23:58:18.249797+0000","last_unstale":"2024-04-24T23:58:20.678466+0000","last_undegraded":"2024-04-24T23:58:20.678466+0000","last_fullsized":"2024-04-24T23:58:20.678466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254736+0000","last_change":"2024-04-24T23:58:19.253506+0000","last_active":"2024-04-24T23:58:20.254736+0000","last_peered":"2024-04-24T23:58:20.254736+0000","last_clean":"2024-04-24T23:58:20.254736+0000","last_became_active":"2024-04-24T23:58:19.253032+0000","last_became_peered":"2024-04-24T23:58:19.253032+0000","last_unstale":"2024-04-24T23:58:20.254736+0000","last_undegraded":"2024-04-24T23:58:20.254736+0000","last_fullsized":"2024-04-24T23:58:20.254736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:14.910044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678152+0000","last_change":"2024-04-24T23:58:18.244986+0000","last_active":"2024-04-24T23:58:20.678152+0000","last_peered":"2024-04-24T23:58:20.678152+0000","last_clean":"2024-04-24T23:58:20.678152+0000","last_became_active":"2024-04-24T23:58:18.244674+0000","last_became_peered":"2024-04-24T23:58:18.244674+0000","last_unstale":"2024-04-24T23:58:20.678152+0000","last_undegraded":"2024-04-24T23:58:20.678152+0000","last_fullsized":"2024-04-24T23:58:20.678152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678003+0000","last_change":"2024-04-24T23:58:19.261722+0000","last_active":"2024-04-24T23:58:20.678003+0000","last_peered":"2024-04-24T23:58:20.678003+0000","last_clean":"2024-04-24T23:58:20.678003+0000","last_became_active":"2024-04-24T23:58:19.261474+0000","last_became_peered":"2024-04-24T23:58:19.261474+0000","last_unstale":"2024-04-24T23:58:20.678003+0000","last_undegraded":"2024-04-24T23:58:20.678003+0000","last_fullsized":"2024-04-24T23:58:20.678003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255072+0000","last_change":"2024-04-24T23:58:18.240520+0000","last_active":"2024-04-24T23:58:20.255072+0000","last_peered":"2024-04-24T23:58:20.255072+0000","last_clean":"2024-04-24T23:58:20.255072+0000","last_became_active":"2024-04-24T23:58:18.240253+0000","last_became_peered":"2024-04-24T23:58:18.240253+0000","last_unstale":"2024-04-24T23:58:20.255072+0000","last_undegraded":"2024-04-24T23:58:20.255072+0000","last_fullsized":"2024-04-24T23:58:20.255072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:17.091843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:58:23.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T23:58:23.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.544+0000 7f3dae6f5640 1 Processor -- start 2024-04-24T23:58:23.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.544+0000 7f3dae6f5640 1 -- start start 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.544+0000 7f3dae6f5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3da80b3fd0 0x7f3da806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.544+0000 7f3dae6f5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3da806e510 0x7f3da80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.544+0000 7f3dae6f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da8144cf0 con 0x7f3da8144910 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.544+0000 7f3dae6f5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3da806e980 con 0x7f3da80b3fd0 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.544+0000 7f3dae6f5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3da80b2f30 con 0x7f3da806e510 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3da80b3fd0 0x7f3da806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3da8144910 0x7f3da80b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41010/0 (socket says 172.21.15.113:41010) 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da7fff640 1 -- 172.21.15.113:0/1372294040 learned_addr learned my addr 172.21.15.113:0/1372294040 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da77fe640 1 -- 172.21.15.113:0/1372294040 >> v2:172.21.15.178:3300/0 conn(0x7f3da806e510 msgr2=0x7f3da80cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da77fe640 1 --2- 172.21.15.113:0/1372294040 >> v2:172.21.15.178:3300/0 conn(0x7f3da806e510 0x7f3da80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da77fe640 1 -- 172.21.15.113:0/1372294040 >> v1:172.21.15.113:6789/0 conn(0x7f3da8144910 legacy=0x7f3da80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da6ffd640 1 -- 172.21.15.113:0/1372294040 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731221922 0 0) 0x7f3da8144cf0 con 0x7f3da8144910 2024-04-24T23:58:23.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.545+0000 7f3da77fe640 1 -- 172.21.15.113:0/1372294040 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da80b3210 con 0x7f3da80b3fd0 2024-04-24T23:58:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3da77fe640 1 --2- 172.21.15.113:0/1372294040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da80b3fd0 0x7f3da806dfd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00ba40 tx=0x7f3d9c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3e55d43bf45fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3da6ffd640 1 -- 172.21.15.113:0/1372294040 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c032770 con 0x7f3da80b3fd0 2024-04-24T23:58:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3da6ffd640 1 -- 172.21.15.113:0/1372294040 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9c038070 con 0x7f3da80b3fd0 2024-04-24T23:58:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3da6ffd640 1 -- 172.21.15.113:0/1372294040 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c032b30 con 0x7f3da80b3fd0 2024-04-24T23:58:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 -- 172.21.15.113:0/1372294040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da80b3fd0 msgr2=0x7f3da806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/1372294040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da80b3fd0 0x7f3da806dfd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00ba40 tx=0x7f3d9c02c8b0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 -- 172.21.15.113:0/1372294040 shutdown_connections 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/1372294040 >> v2:172.21.15.178:3300/0 conn(0x7f3da806e510 0x7f3da80cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/1372294040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da80b3fd0 0x7f3da806dfd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 -- 172.21.15.113:0/1372294040 >> 172.21.15.113:0/1372294040 conn(0x7f3da80b1a50 msgr2=0x7f3da80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 -- 172.21.15.113:0/1372294040 shutdown_connections 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.546+0000 7f3dae6f5640 1 -- 172.21.15.113:0/1372294040 wait complete. 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.547+0000 7f3dae6f5640 1 Processor -- start 2024-04-24T23:58:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dae6f5640 1 -- start start 2024-04-24T23:58:23.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dae6f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da806e510 0x7f3da815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dae6f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da80b3fd0 0x7f3da80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dae6f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3da8144910 0x7f3da8140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dae6f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80b4440 con 0x7f3da806e510 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dae6f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80b37d0 con 0x7f3da80b3fd0 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dae6f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80b4da0 con 0x7f3da8144910 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da806e510 0x7f3da815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da806e510 0x7f3da815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49168/0 (socket says 172.21.15.113:49168) 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3da7fff640 1 -- 172.21.15.113:0/4094057288 learned_addr learned my addr 172.21.15.113:0/4094057288 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3da77fe640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da80b3fd0 0x7f3da80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.548+0000 7f3dacc92640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3da8144910 0x7f3da8140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da7fff640 1 -- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3da8144910 msgr2=0x7f3da8140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da7fff640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3da8144910 0x7f3da8140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da7fff640 1 -- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da80b3fd0 msgr2=0x7f3da80ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da7fff640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da80b3fd0 0x7f3da80ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da7fff640 1 -- 172.21.15.113:0/4094057288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da80ae200 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da7fff640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da806e510 0x7f3da815d810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c430 tx=0x7f3d9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da4ff9640 1 -- 172.21.15.113:0/4094057288 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980074e0 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da4ff9640 1 -- 172.21.15.113:0/4094057288 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d980047e0 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da80ae4e0 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3da4ff9640 1 -- 172.21.15.113:0/4094057288 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004bd0 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.549+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da80aeaa0 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.550+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da80691a0 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.550+0000 7f3da4ff9640 1 -- 172.21.15.113:0/4094057288 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d98011070 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.551+0000 7f3da4ff9640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9406be60 0x7f3d9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.551+0000 7f3da4ff9640 1 -- 172.21.15.113:0/4094057288 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d9808aad0 con 0x7f3da806e510 2024-04-24T23:58:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.551+0000 7f3da77fe640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9406be60 0x7f3d9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.551+0000 7f3da77fe640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9406be60 0x7f3d9406e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c037000 tx=0x7f3d9c032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.555+0000 7f3da4ff9640 1 -- 172.21.15.113:0/4094057288 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d98059480 con 0x7f3da806e510 2024-04-24T23:58:23.621 INFO:tasks.ceph.mds.c.smithi113.stderr:2024-04-24T23:58:23.621+0000 7f44c5f17640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:58:23.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.793+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3da8068a20 con 0x7f3da806e510 2024-04-24T23:58:23.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.794+0000 7f3da4ff9640 1 -- 172.21.15.113:0/4094057288 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1011 (secure 0 0 0) 0x7f3d98058d80 con 0x7f3da806e510 2024-04-24T23:58:23.794 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:23.794 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":241262592,"bytes_avail":772852850688,"bytes_total":773094113280,"write_bytes_sec":2436,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"c","status":"up:active","gid":4281}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-24T23:57:54.997614+0000","services":{}},"progress_events":{}} 2024-04-24T23:58:23.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9406be60 msgr2=0x7f3d9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9406be60 0x7f3d9406e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c037000 tx=0x7f3d9c032000 comp rx=0 tx=0).stop 2024-04-24T23:58:23.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da806e510 msgr2=0x7f3da815d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da806e510 0x7f3da815d810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c430 tx=0x7f3d9800c8f0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 shutdown_connections 2024-04-24T23:58:23.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9406be60 0x7f3d9406e320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da80b3fd0 0x7f3da80ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3da8144910 0x7f3da8140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 --2- 172.21.15.113:0/4094057288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da806e510 0x7f3da815d810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 >> 172.21.15.113:0/4094057288 conn(0x7f3da80b1a50 msgr2=0x7f3da815eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 shutdown_connections 2024-04-24T23:58:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.796+0000 7f3dae6f5640 1 -- 172.21.15.113:0/4094057288 wait complete. 2024-04-24T23:58:23.806 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T23:58:23.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:58:23.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.903+0000 7f096a279640 1 Processor -- start 2024-04-24T23:58:23.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.903+0000 7f096a279640 1 -- start start 2024-04-24T23:58:23.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f096a279640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f096415e320 0x7f09640d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f096a279640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d2290 0x7f09640b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f096a279640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09640d3de0 con 0x7f09640d0b00 2024-04-24T23:58:23.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f096a279640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f09640d3410 con 0x7f096415e320 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f096a279640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09640d2800 con 0x7f09640d2290 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f0963fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f096415e320 0x7f09640d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f0963fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f096415e320 0x7f09640d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49184/0 (socket says 172.21.15.113:49184) 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.904+0000 7f0963fff640 1 -- 172.21.15.113:0/2758102518 learned_addr learned my addr 172.21.15.113:0/2758102518 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f09637fe640 1 --2- 172.21.15.113:0/2758102518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d2290 0x7f09640b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0963fff640 1 -- 172.21.15.113:0/2758102518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d2290 msgr2=0x7f09640b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0963fff640 1 --2- 172.21.15.113:0/2758102518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d2290 0x7f09640b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0963fff640 1 -- 172.21.15.113:0/2758102518 >> v1:172.21.15.178:6789/0 conn(0x7f09640d0b00 legacy=0x7f09640b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0963fff640 1 -- 172.21.15.113:0/2758102518 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09640d2ae0 con 0x7f096415e320 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f09637fe640 1 --2- 172.21.15.113:0/2758102518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d2290 0x7f09640b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0963fff640 1 --2- 172.21.15.113:0/2758102518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f096415e320 0x7f09640d38a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f095402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ff0a3ec5f7f200 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0962ffd640 1 -- 172.21.15.113:0/2758102518 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954037070 con 0x7f096415e320 2024-04-24T23:58:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0962ffd640 1 -- 172.21.15.113:0/2758102518 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0954032440 con 0x7f096415e320 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.905+0000 7f0962ffd640 1 -- 172.21.15.113:0/2758102518 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954032810 con 0x7f096415e320 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 -- 172.21.15.113:0/2758102518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f096415e320 msgr2=0x7f09640d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 --2- 172.21.15.113:0/2758102518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f096415e320 0x7f09640d38a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f095402a2a0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 -- 172.21.15.113:0/2758102518 shutdown_connections 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 --2- 172.21.15.113:0/2758102518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d2290 0x7f09640b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 --2- 172.21.15.113:0/2758102518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f096415e320 0x7f09640d38a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 -- 172.21.15.113:0/2758102518 >> 172.21.15.113:0/2758102518 conn(0x7f09640b5c20 msgr2=0x7f09640b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 -- 172.21.15.113:0/2758102518 shutdown_connections 2024-04-24T23:58:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.906+0000 7f096a279640 1 -- 172.21.15.113:0/2758102518 wait complete. 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.907+0000 7f096a279640 1 Processor -- start 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.907+0000 7f096a279640 1 -- start start 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f096a279640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d0b00 0x7f0964154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f096a279640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09640d2290 0x7f096414a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f096a279640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f096415e320 0x7f09640cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f096a279640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09640b9fe0 con 0x7f09640d2290 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f096a279640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09640d30a0 con 0x7f096415e320 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f096a279640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f096415e700 con 0x7f09640d0b00 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09640d2290 0x7f096414a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09640d2290 0x7f096414a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49186/0 (socket says 172.21.15.113:49186) 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 -- 172.21.15.113:0/2634297796 learned_addr learned my addr 172.21.15.113:0/2634297796 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f0963fff640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d0b00 0x7f0964154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 -- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d0b00 msgr2=0x7f0964154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f0968816640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f096415e320 0x7f09640cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d0b00 0x7f0964154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 -- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f096415e320 msgr2=0x7f09640cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f096415e320 0x7f09640cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.908+0000 7f09637fe640 1 -- 172.21.15.113:0/2634297796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09641539d0 con 0x7f09640d2290 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.909+0000 7f09637fe640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09640d2290 0x7f096414a480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f095800c7f0 tx=0x7f095800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.909+0000 7f0960ff9640 1 -- 172.21.15.113:0/2634297796 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958015070 con 0x7f09640d2290 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.909+0000 7f0960ff9640 1 -- 172.21.15.113:0/2634297796 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0958007a30 con 0x7f09640d2290 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.909+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096414a9c0 con 0x7f09640d2290 2024-04-24T23:58:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.909+0000 7f0960ff9640 1 -- 172.21.15.113:0/2634297796 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958007e00 con 0x7f09640d2290 2024-04-24T23:58:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.909+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096415fd90 con 0x7f09640d2290 2024-04-24T23:58:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.910+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096415f8e0 con 0x7f09640d2290 2024-04-24T23:58:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.910+0000 7f0960ff9640 1 -- 172.21.15.113:0/2634297796 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0958018070 con 0x7f09640d2290 2024-04-24T23:58:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.911+0000 7f0960ff9640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.911+0000 7f0960ff9640 1 -- 172.21.15.113:0/2634297796 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f095808ade0 con 0x7f09640d2290 2024-04-24T23:58:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.911+0000 7f0963fff640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.911+0000 7f0963fff640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0954034000 tx=0x7f095402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:23.915+0000 7f0960ff9640 1 -- 172.21.15.113:0/2634297796 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0958059790 con 0x7f09640d2290 2024-04-24T23:58:24.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.083+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09640b4920 con 0x7f095006be30 2024-04-24T23:58:24.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.088+0000 7f0960ff9640 1 -- 172.21.15.113:0/2634297796 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f09640b4920 con 0x7f095006be30 2024-04-24T23:58:24.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:24.092 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:58:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 msgr2=0x7f095006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0954034000 tx=0x7f095402e000 comp rx=0 tx=0).stop 2024-04-24T23:58:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09640d2290 msgr2=0x7f096414a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09640d2290 0x7f096414a480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f095800c7f0 tx=0x7f095800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:58:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 shutdown_connections 2024-04-24T23:58:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:24.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f096415e320 0x7f09640cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:24.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09640d2290 0x7f096414a480 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:24.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 --2- 172.21.15.113:0/2634297796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09640d0b00 0x7f0964154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:24.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 >> 172.21.15.113:0/2634297796 conn(0x7f09640b5c20 msgr2=0x7f09640b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:24.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 shutdown_connections 2024-04-24T23:58:24.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:24.091+0000 7f096a279640 1 -- 172.21.15.113:0/2634297796 wait complete. 2024-04-24T23:58:24.102 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-24T23:58:23.112597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235608,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739112,"statfs":{"total":773094113280,"available":772852850688,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.885004"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678418+0000","last_change":"2024-04-24T23:58:19.270520+0000","last_active":"2024-04-24T23:58:20.678418+0000","last_peered":"2024-04-24T23:58:20.678418+0000","last_clean":"2024-04-24T23:58:20.678418+0000","last_became_active":"2024-04-24T23:58:19.270280+0000","last_became_peered":"2024-04-24T23:58:19.270280+0000","last_unstale":"2024-04-24T23:58:20.678418+0000","last_undegraded":"2024-04-24T23:58:20.678418+0000","last_fullsized":"2024-04-24T23:58:20.678418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254502+0000","last_change":"2024-04-24T23:58:19.252635+0000","last_active":"2024-04-24T23:58:20.254502+0000","last_peered":"2024-04-24T23:58:20.254502+0000","last_clean":"2024-04-24T23:58:20.254502+0000","last_became_active":"2024-04-24T23:58:19.252352+0000","last_became_peered":"2024-04-24T23:58:19.252352+0000","last_unstale":"2024-04-24T23:58:20.254502+0000","last_undegraded":"2024-04-24T23:58:20.254502+0000","last_fullsized":"2024-04-24T23:58:20.254502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254874+0000","last_change":"2024-04-24T23:58:19.257041+0000","last_active":"2024-04-24T23:58:20.254874+0000","last_peered":"2024-04-24T23:58:20.254874+0000","last_clean":"2024-04-24T23:58:20.254874+0000","last_became_active":"2024-04-24T23:58:19.256765+0000","last_became_peered":"2024-04-24T23:58:19.256765+0000","last_unstale":"2024-04-24T23:58:20.254874+0000","last_undegraded":"2024-04-24T23:58:20.254874+0000","last_fullsized":"2024-04-24T23:58:20.254874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.271981+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.271589+0000","last_became_peered":"2024-04-24T23:58:19.271589+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678540+0000","last_change":"2024-04-24T23:58:19.260799+0000","last_active":"2024-04-24T23:58:20.678540+0000","last_peered":"2024-04-24T23:58:20.678540+0000","last_clean":"2024-04-24T23:58:20.678540+0000","last_became_active":"2024-04-24T23:58:19.260437+0000","last_became_peered":"2024-04-24T23:58:19.260437+0000","last_unstale":"2024-04-24T23:58:20.678540+0000","last_undegraded":"2024-04-24T23:58:20.678540+0000","last_fullsized":"2024-04-24T23:58:20.678540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677754+0000","last_change":"2024-04-24T23:58:19.269955+0000","last_active":"2024-04-24T23:58:20.677754+0000","last_peered":"2024-04-24T23:58:20.677754+0000","last_clean":"2024-04-24T23:58:20.677754+0000","last_became_active":"2024-04-24T23:58:19.269675+0000","last_became_peered":"2024-04-24T23:58:19.269675+0000","last_unstale":"2024-04-24T23:58:20.677754+0000","last_undegraded":"2024-04-24T23:58:20.677754+0000","last_fullsized":"2024-04-24T23:58:20.677754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678346+0000","last_change":"2024-04-24T23:58:19.267752+0000","last_active":"2024-04-24T23:58:20.678346+0000","last_peered":"2024-04-24T23:58:20.678346+0000","last_clean":"2024-04-24T23:58:20.678346+0000","last_became_active":"2024-04-24T23:58:19.267463+0000","last_became_peered":"2024-04-24T23:58:19.267463+0000","last_unstale":"2024-04-24T23:58:20.678346+0000","last_undegraded":"2024-04-24T23:58:20.678346+0000","last_fullsized":"2024-04-24T23:58:20.678346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678319+0000","last_change":"2024-04-24T23:58:19.266736+0000","last_active":"2024-04-24T23:58:20.678319+0000","last_peered":"2024-04-24T23:58:20.678319+0000","last_clean":"2024-04-24T23:58:20.678319+0000","last_became_active":"2024-04-24T23:58:19.266438+0000","last_became_peered":"2024-04-24T23:58:19.266438+0000","last_unstale":"2024-04-24T23:58:20.678319+0000","last_undegraded":"2024-04-24T23:58:20.678319+0000","last_fullsized":"2024-04-24T23:58:20.678319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678538+0000","last_change":"2024-04-24T23:58:19.271086+0000","last_active":"2024-04-24T23:58:20.678538+0000","last_peered":"2024-04-24T23:58:20.678538+0000","last_clean":"2024-04-24T23:58:20.678538+0000","last_became_active":"2024-04-24T23:58:19.270722+0000","last_became_peered":"2024-04-24T23:58:19.270722+0000","last_unstale":"2024-04-24T23:58:20.678538+0000","last_undegraded":"2024-04-24T23:58:20.678538+0000","last_fullsized":"2024-04-24T23:58:20.678538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257102+0000","last_change":"2024-04-24T23:58:19.266845+0000","last_active":"2024-04-24T23:58:20.257102+0000","last_peered":"2024-04-24T23:58:20.257102+0000","last_clean":"2024-04-24T23:58:20.257102+0000","last_became_active":"2024-04-24T23:58:19.266573+0000","last_became_peered":"2024-04-24T23:58:19.266573+0000","last_unstale":"2024-04-24T23:58:20.257102+0000","last_undegraded":"2024-04-24T23:58:20.257102+0000","last_fullsized":"2024-04-24T23:58:20.257102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256863+0000","last_change":"2024-04-24T23:58:19.267588+0000","last_active":"2024-04-24T23:58:20.256863+0000","last_peered":"2024-04-24T23:58:20.256863+0000","last_clean":"2024-04-24T23:58:20.256863+0000","last_became_active":"2024-04-24T23:58:19.267247+0000","last_became_peered":"2024-04-24T23:58:19.267247+0000","last_unstale":"2024-04-24T23:58:20.256863+0000","last_undegraded":"2024-04-24T23:58:20.256863+0000","last_fullsized":"2024-04-24T23:58:20.256863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255357+0000","last_change":"2024-04-24T23:58:19.257310+0000","last_active":"2024-04-24T23:58:20.255357+0000","last_peered":"2024-04-24T23:58:20.255357+0000","last_clean":"2024-04-24T23:58:20.255357+0000","last_became_active":"2024-04-24T23:58:19.257023+0000","last_became_peered":"2024-04-24T23:58:19.257023+0000","last_unstale":"2024-04-24T23:58:20.255357+0000","last_undegraded":"2024-04-24T23:58:20.255357+0000","last_fullsized":"2024-04-24T23:58:20.255357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678413+0000","last_change":"2024-04-24T23:58:19.261091+0000","last_active":"2024-04-24T23:58:20.678413+0000","last_peered":"2024-04-24T23:58:20.678413+0000","last_clean":"2024-04-24T23:58:20.678413+0000","last_became_active":"2024-04-24T23:58:19.260837+0000","last_became_peered":"2024-04-24T23:58:19.260837+0000","last_unstale":"2024-04-24T23:58:20.678413+0000","last_undegraded":"2024-04-24T23:58:20.678413+0000","last_fullsized":"2024-04-24T23:58:20.678413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254811+0000","last_change":"2024-04-24T23:58:19.261328+0000","last_active":"2024-04-24T23:58:20.254811+0000","last_peered":"2024-04-24T23:58:20.254811+0000","last_clean":"2024-04-24T23:58:20.254811+0000","last_became_active":"2024-04-24T23:58:19.261045+0000","last_became_peered":"2024-04-24T23:58:19.261045+0000","last_unstale":"2024-04-24T23:58:20.254811+0000","last_undegraded":"2024-04-24T23:58:20.254811+0000","last_fullsized":"2024-04-24T23:58:20.254811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254583+0000","last_change":"2024-04-24T23:58:19.256356+0000","last_active":"2024-04-24T23:58:20.254583+0000","last_peered":"2024-04-24T23:58:20.254583+0000","last_clean":"2024-04-24T23:58:20.254583+0000","last_became_active":"2024-04-24T23:58:19.256099+0000","last_became_peered":"2024-04-24T23:58:19.256099+0000","last_unstale":"2024-04-24T23:58:20.254583+0000","last_undegraded":"2024-04-24T23:58:20.254583+0000","last_fullsized":"2024-04-24T23:58:20.254583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677420+0000","last_change":"2024-04-24T23:58:19.268419+0000","last_active":"2024-04-24T23:58:20.677420+0000","last_peered":"2024-04-24T23:58:20.677420+0000","last_clean":"2024-04-24T23:58:20.677420+0000","last_became_active":"2024-04-24T23:58:19.265742+0000","last_became_peered":"2024-04-24T23:58:19.265742+0000","last_unstale":"2024-04-24T23:58:20.677420+0000","last_undegraded":"2024-04-24T23:58:20.677420+0000","last_fullsized":"2024-04-24T23:58:20.677420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.273024+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.272551+0000","last_became_peered":"2024-04-24T23:58:19.272551+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.258361+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.258098+0000","last_became_peered":"2024-04-24T23:58:19.258098+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678669+0000","last_change":"2024-04-24T23:58:19.263085+0000","last_active":"2024-04-24T23:58:20.678669+0000","last_peered":"2024-04-24T23:58:20.678669+0000","last_clean":"2024-04-24T23:58:20.678669+0000","last_became_active":"2024-04-24T23:58:19.262807+0000","last_became_peered":"2024-04-24T23:58:19.262807+0000","last_unstale":"2024-04-24T23:58:20.678669+0000","last_undegraded":"2024-04-24T23:58:20.678669+0000","last_fullsized":"2024-04-24T23:58:20.678669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256867+0000","last_change":"2024-04-24T23:58:19.259501+0000","last_active":"2024-04-24T23:58:20.256867+0000","last_peered":"2024-04-24T23:58:20.256867+0000","last_clean":"2024-04-24T23:58:20.256867+0000","last_became_active":"2024-04-24T23:58:19.259233+0000","last_became_peered":"2024-04-24T23:58:19.259233+0000","last_unstale":"2024-04-24T23:58:20.256867+0000","last_undegraded":"2024-04-24T23:58:20.256867+0000","last_fullsized":"2024-04-24T23:58:20.256867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255041+0000","last_change":"2024-04-24T23:58:19.259960+0000","last_active":"2024-04-24T23:58:20.255041+0000","last_peered":"2024-04-24T23:58:20.255041+0000","last_clean":"2024-04-24T23:58:20.255041+0000","last_became_active":"2024-04-24T23:58:19.259548+0000","last_became_peered":"2024-04-24T23:58:19.259548+0000","last_unstale":"2024-04-24T23:58:20.255041+0000","last_undegraded":"2024-04-24T23:58:20.255041+0000","last_fullsized":"2024-04-24T23:58:20.255041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677851+0000","last_change":"2024-04-24T23:58:19.275499+0000","last_active":"2024-04-24T23:58:20.677851+0000","last_peered":"2024-04-24T23:58:20.677851+0000","last_clean":"2024-04-24T23:58:20.677851+0000","last_became_active":"2024-04-24T23:58:19.275213+0000","last_became_peered":"2024-04-24T23:58:19.275213+0000","last_unstale":"2024-04-24T23:58:20.677851+0000","last_undegraded":"2024-04-24T23:58:20.677851+0000","last_fullsized":"2024-04-24T23:58:20.677851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678362+0000","last_change":"2024-04-24T23:58:19.261963+0000","last_active":"2024-04-24T23:58:20.678362+0000","last_peered":"2024-04-24T23:58:20.678362+0000","last_clean":"2024-04-24T23:58:20.678362+0000","last_became_active":"2024-04-24T23:58:19.261710+0000","last_became_peered":"2024-04-24T23:58:19.261710+0000","last_unstale":"2024-04-24T23:58:20.678362+0000","last_undegraded":"2024-04-24T23:58:20.678362+0000","last_fullsized":"2024-04-24T23:58:20.678362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677517+0000","last_change":"2024-04-24T23:58:19.268171+0000","last_active":"2024-04-24T23:58:20.677517+0000","last_peered":"2024-04-24T23:58:20.677517+0000","last_clean":"2024-04-24T23:58:20.677517+0000","last_became_active":"2024-04-24T23:58:19.265379+0000","last_became_peered":"2024-04-24T23:58:19.265379+0000","last_unstale":"2024-04-24T23:58:20.677517+0000","last_undegraded":"2024-04-24T23:58:20.677517+0000","last_fullsized":"2024-04-24T23:58:20.677517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678064+0000","last_change":"2024-04-24T23:58:19.259615+0000","last_active":"2024-04-24T23:58:20.678064+0000","last_peered":"2024-04-24T23:58:20.678064+0000","last_clean":"2024-04-24T23:58:20.678064+0000","last_became_active":"2024-04-24T23:58:19.259355+0000","last_became_peered":"2024-04-24T23:58:19.259355+0000","last_unstale":"2024-04-24T23:58:20.678064+0000","last_undegraded":"2024-04-24T23:58:20.678064+0000","last_fullsized":"2024-04-24T23:58:20.678064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257106+0000","last_change":"2024-04-24T23:58:19.269905+0000","last_active":"2024-04-24T23:58:20.257106+0000","last_peered":"2024-04-24T23:58:20.257106+0000","last_clean":"2024-04-24T23:58:20.257106+0000","last_became_active":"2024-04-24T23:58:19.269639+0000","last_became_peered":"2024-04-24T23:58:19.269639+0000","last_unstale":"2024-04-24T23:58:20.257106+0000","last_undegraded":"2024-04-24T23:58:20.257106+0000","last_fullsized":"2024-04-24T23:58:20.257106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254304+0000","last_change":"2024-04-24T23:58:19.256747+0000","last_active":"2024-04-24T23:58:20.254304+0000","last_peered":"2024-04-24T23:58:20.254304+0000","last_clean":"2024-04-24T23:58:20.254304+0000","last_became_active":"2024-04-24T23:58:19.255723+0000","last_became_peered":"2024-04-24T23:58:19.255723+0000","last_unstale":"2024-04-24T23:58:20.254304+0000","last_undegraded":"2024-04-24T23:58:20.254304+0000","last_fullsized":"2024-04-24T23:58:20.254304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678051+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678051+0000","last_peered":"2024-04-24T23:58:20.678051+0000","last_clean":"2024-04-24T23:58:20.678051+0000","last_became_active":"2024-04-24T23:58:19.262977+0000","last_became_peered":"2024-04-24T23:58:19.262977+0000","last_unstale":"2024-04-24T23:58:20.678051+0000","last_undegraded":"2024-04-24T23:58:20.678051+0000","last_fullsized":"2024-04-24T23:58:20.678051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255177+0000","last_change":"2024-04-24T23:58:19.259745+0000","last_active":"2024-04-24T23:58:20.255177+0000","last_peered":"2024-04-24T23:58:20.255177+0000","last_clean":"2024-04-24T23:58:20.255177+0000","last_became_active":"2024-04-24T23:58:19.259201+0000","last_became_peered":"2024-04-24T23:58:19.259201+0000","last_unstale":"2024-04-24T23:58:20.255177+0000","last_undegraded":"2024-04-24T23:58:20.255177+0000","last_fullsized":"2024-04-24T23:58:20.255177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678071+0000","last_change":"2024-04-24T23:58:19.273016+0000","last_active":"2024-04-24T23:58:20.678071+0000","last_peered":"2024-04-24T23:58:20.678071+0000","last_clean":"2024-04-24T23:58:20.678071+0000","last_became_active":"2024-04-24T23:58:19.271540+0000","last_became_peered":"2024-04-24T23:58:19.271540+0000","last_unstale":"2024-04-24T23:58:20.678071+0000","last_undegraded":"2024-04-24T23:58:20.678071+0000","last_fullsized":"2024-04-24T23:58:20.678071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677646+0000","last_change":"2024-04-24T23:58:19.255458+0000","last_active":"2024-04-24T23:58:20.677646+0000","last_peered":"2024-04-24T23:58:20.677646+0000","last_clean":"2024-04-24T23:58:20.677646+0000","last_became_active":"2024-04-24T23:58:19.255070+0000","last_became_peered":"2024-04-24T23:58:19.255070+0000","last_unstale":"2024-04-24T23:58:20.677646+0000","last_undegraded":"2024-04-24T23:58:20.677646+0000","last_fullsized":"2024-04-24T23:58:20.677646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256632+0000","last_change":"2024-04-24T23:58:19.266657+0000","last_active":"2024-04-24T23:58:20.256632+0000","last_peered":"2024-04-24T23:58:20.256632+0000","last_clean":"2024-04-24T23:58:20.256632+0000","last_became_active":"2024-04-24T23:58:19.266354+0000","last_became_peered":"2024-04-24T23:58:19.266354+0000","last_unstale":"2024-04-24T23:58:20.256632+0000","last_undegraded":"2024-04-24T23:58:20.256632+0000","last_fullsized":"2024-04-24T23:58:20.256632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255049+0000","last_change":"2024-04-24T23:58:19.259253+0000","last_active":"2024-04-24T23:58:20.255049+0000","last_peered":"2024-04-24T23:58:20.255049+0000","last_clean":"2024-04-24T23:58:20.255049+0000","last_became_active":"2024-04-24T23:58:19.258553+0000","last_became_peered":"2024-04-24T23:58:19.258553+0000","last_unstale":"2024-04-24T23:58:20.255049+0000","last_undegraded":"2024-04-24T23:58:20.255049+0000","last_fullsized":"2024-04-24T23:58:20.255049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:53.284799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678523+0000","last_change":"2024-04-24T23:58:18.250614+0000","last_active":"2024-04-24T23:58:20.678523+0000","last_peered":"2024-04-24T23:58:20.678523+0000","last_clean":"2024-04-24T23:58:20.678523+0000","last_became_active":"2024-04-24T23:58:18.250306+0000","last_became_peered":"2024-04-24T23:58:18.250306+0000","last_unstale":"2024-04-24T23:58:20.678523+0000","last_undegraded":"2024-04-24T23:58:20.678523+0000","last_fullsized":"2024-04-24T23:58:20.678523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255184+0000","last_change":"2024-04-24T23:58:19.251648+0000","last_active":"2024-04-24T23:58:20.255184+0000","last_peered":"2024-04-24T23:58:20.255184+0000","last_clean":"2024-04-24T23:58:20.255184+0000","last_became_active":"2024-04-24T23:58:19.251271+0000","last_became_peered":"2024-04-24T23:58:19.251271+0000","last_unstale":"2024-04-24T23:58:20.255184+0000","last_undegraded":"2024-04-24T23:58:20.255184+0000","last_fullsized":"2024-04-24T23:58:20.255184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256708+0000","last_change":"2024-04-24T23:58:18.252764+0000","last_active":"2024-04-24T23:58:20.256708+0000","last_peered":"2024-04-24T23:58:20.256708+0000","last_clean":"2024-04-24T23:58:20.256708+0000","last_became_active":"2024-04-24T23:58:18.252491+0000","last_became_peered":"2024-04-24T23:58:18.252491+0000","last_unstale":"2024-04-24T23:58:20.256708+0000","last_undegraded":"2024-04-24T23:58:20.256708+0000","last_fullsized":"2024-04-24T23:58:20.256708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678357+0000","last_change":"2024-04-24T23:58:18.250003+0000","last_active":"2024-04-24T23:58:20.678357+0000","last_peered":"2024-04-24T23:58:20.678357+0000","last_clean":"2024-04-24T23:58:20.678357+0000","last_became_active":"2024-04-24T23:58:18.249727+0000","last_became_peered":"2024-04-24T23:58:18.249727+0000","last_unstale":"2024-04-24T23:58:20.678357+0000","last_undegraded":"2024-04-24T23:58:20.678357+0000","last_fullsized":"2024-04-24T23:58:20.678357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.318157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678399+0000","last_change":"2024-04-24T23:58:19.255033+0000","last_active":"2024-04-24T23:58:20.678399+0000","last_peered":"2024-04-24T23:58:20.678399+0000","last_clean":"2024-04-24T23:58:20.678399+0000","last_became_active":"2024-04-24T23:58:19.254746+0000","last_became_peered":"2024-04-24T23:58:19.254746+0000","last_unstale":"2024-04-24T23:58:20.678399+0000","last_undegraded":"2024-04-24T23:58:20.678399+0000","last_fullsized":"2024-04-24T23:58:20.678399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677538+0000","last_change":"2024-04-24T23:58:18.245516+0000","last_active":"2024-04-24T23:58:20.677538+0000","last_peered":"2024-04-24T23:58:20.677538+0000","last_clean":"2024-04-24T23:58:20.677538+0000","last_became_active":"2024-04-24T23:58:18.245212+0000","last_became_peered":"2024-04-24T23:58:18.245212+0000","last_unstale":"2024-04-24T23:58:20.677538+0000","last_undegraded":"2024-04-24T23:58:20.677538+0000","last_fullsized":"2024-04-24T23:58:20.677538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254479+0000","last_change":"2024-04-24T23:58:19.258354+0000","last_active":"2024-04-24T23:58:20.254479+0000","last_peered":"2024-04-24T23:58:20.254479+0000","last_clean":"2024-04-24T23:58:20.254479+0000","last_became_active":"2024-04-24T23:58:19.258067+0000","last_became_peered":"2024-04-24T23:58:19.258067+0000","last_unstale":"2024-04-24T23:58:20.254479+0000","last_undegraded":"2024-04-24T23:58:20.254479+0000","last_fullsized":"2024-04-24T23:58:20.254479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256985+0000","last_change":"2024-04-24T23:58:18.253806+0000","last_active":"2024-04-24T23:58:20.256985+0000","last_peered":"2024-04-24T23:58:20.256985+0000","last_clean":"2024-04-24T23:58:20.256985+0000","last_became_active":"2024-04-24T23:58:18.252657+0000","last_became_peered":"2024-04-24T23:58:18.252657+0000","last_unstale":"2024-04-24T23:58:20.256985+0000","last_undegraded":"2024-04-24T23:58:20.256985+0000","last_fullsized":"2024-04-24T23:58:20.256985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677952+0000","last_change":"2024-04-24T23:58:19.275012+0000","last_active":"2024-04-24T23:58:20.677952+0000","last_peered":"2024-04-24T23:58:20.677952+0000","last_clean":"2024-04-24T23:58:20.677952+0000","last_became_active":"2024-04-24T23:58:19.274740+0000","last_became_peered":"2024-04-24T23:58:19.274740+0000","last_unstale":"2024-04-24T23:58:20.677952+0000","last_undegraded":"2024-04-24T23:58:20.677952+0000","last_fullsized":"2024-04-24T23:58:20.677952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678001+0000","last_change":"2024-04-24T23:58:18.251763+0000","last_active":"2024-04-24T23:58:20.678001+0000","last_peered":"2024-04-24T23:58:20.678001+0000","last_clean":"2024-04-24T23:58:20.678001+0000","last_became_active":"2024-04-24T23:58:18.251508+0000","last_became_peered":"2024-04-24T23:58:18.251508+0000","last_unstale":"2024-04-24T23:58:20.678001+0000","last_undegraded":"2024-04-24T23:58:20.678001+0000","last_fullsized":"2024-04-24T23:58:20.678001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254979+0000","last_change":"2024-04-24T23:58:19.253275+0000","last_active":"2024-04-24T23:58:20.254979+0000","last_peered":"2024-04-24T23:58:20.254979+0000","last_clean":"2024-04-24T23:58:20.254979+0000","last_became_active":"2024-04-24T23:58:19.252783+0000","last_became_peered":"2024-04-24T23:58:19.252783+0000","last_unstale":"2024-04-24T23:58:20.254979+0000","last_undegraded":"2024-04-24T23:58:20.254979+0000","last_fullsized":"2024-04-24T23:58:20.254979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254408+0000","last_change":"2024-04-24T23:58:18.246446+0000","last_active":"2024-04-24T23:58:20.254408+0000","last_peered":"2024-04-24T23:58:20.254408+0000","last_clean":"2024-04-24T23:58:20.254408+0000","last_became_active":"2024-04-24T23:58:18.246036+0000","last_became_peered":"2024-04-24T23:58:18.246036+0000","last_unstale":"2024-04-24T23:58:20.254408+0000","last_undegraded":"2024-04-24T23:58:20.254408+0000","last_fullsized":"2024-04-24T23:58:20.254408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255261+0000","last_change":"2024-04-24T23:58:19.261655+0000","last_active":"2024-04-24T23:58:20.255261+0000","last_peered":"2024-04-24T23:58:20.255261+0000","last_clean":"2024-04-24T23:58:20.255261+0000","last_became_active":"2024-04-24T23:58:19.261331+0000","last_became_peered":"2024-04-24T23:58:19.261331+0000","last_unstale":"2024-04-24T23:58:20.255261+0000","last_undegraded":"2024-04-24T23:58:20.255261+0000","last_fullsized":"2024-04-24T23:58:20.255261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256970+0000","last_change":"2024-04-24T23:58:18.247264+0000","last_active":"2024-04-24T23:58:20.256970+0000","last_peered":"2024-04-24T23:58:20.256970+0000","last_clean":"2024-04-24T23:58:20.256970+0000","last_became_active":"2024-04-24T23:58:18.246959+0000","last_became_peered":"2024-04-24T23:58:18.246959+0000","last_unstale":"2024-04-24T23:58:20.256970+0000","last_undegraded":"2024-04-24T23:58:20.256970+0000","last_fullsized":"2024-04-24T23:58:20.256970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678015+0000","last_change":"2024-04-24T23:58:19.265255+0000","last_active":"2024-04-24T23:58:20.678015+0000","last_peered":"2024-04-24T23:58:20.678015+0000","last_clean":"2024-04-24T23:58:20.678015+0000","last_became_active":"2024-04-24T23:58:19.264997+0000","last_became_peered":"2024-04-24T23:58:19.264997+0000","last_unstale":"2024-04-24T23:58:20.678015+0000","last_undegraded":"2024-04-24T23:58:20.678015+0000","last_fullsized":"2024-04-24T23:58:20.678015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255275+0000","last_change":"2024-04-24T23:58:18.236929+0000","last_active":"2024-04-24T23:58:20.255275+0000","last_peered":"2024-04-24T23:58:20.255275+0000","last_clean":"2024-04-24T23:58:20.255275+0000","last_became_active":"2024-04-24T23:58:18.236660+0000","last_became_peered":"2024-04-24T23:58:18.236660+0000","last_unstale":"2024-04-24T23:58:20.255275+0000","last_undegraded":"2024-04-24T23:58:20.255275+0000","last_fullsized":"2024-04-24T23:58:20.255275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254944+0000","last_change":"2024-04-24T23:58:19.259532+0000","last_active":"2024-04-24T23:58:20.254944+0000","last_peered":"2024-04-24T23:58:20.254944+0000","last_clean":"2024-04-24T23:58:20.254944+0000","last_became_active":"2024-04-24T23:58:19.259298+0000","last_became_peered":"2024-04-24T23:58:19.259298+0000","last_unstale":"2024-04-24T23:58:20.254944+0000","last_undegraded":"2024-04-24T23:58:20.254944+0000","last_fullsized":"2024-04-24T23:58:20.254944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678194+0000","last_change":"2024-04-24T23:58:18.251116+0000","last_active":"2024-04-24T23:58:20.678194+0000","last_peered":"2024-04-24T23:58:20.678194+0000","last_clean":"2024-04-24T23:58:20.678194+0000","last_became_active":"2024-04-24T23:58:18.250814+0000","last_became_peered":"2024-04-24T23:58:18.250814+0000","last_unstale":"2024-04-24T23:58:20.678194+0000","last_undegraded":"2024-04-24T23:58:20.678194+0000","last_fullsized":"2024-04-24T23:58:20.678194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678564+0000","last_change":"2024-04-24T23:58:19.263724+0000","last_active":"2024-04-24T23:58:20.678564+0000","last_peered":"2024-04-24T23:58:20.678564+0000","last_clean":"2024-04-24T23:58:20.678564+0000","last_became_active":"2024-04-24T23:58:19.263448+0000","last_became_peered":"2024-04-24T23:58:19.263448+0000","last_unstale":"2024-04-24T23:58:20.678564+0000","last_undegraded":"2024-04-24T23:58:20.678564+0000","last_fullsized":"2024-04-24T23:58:20.678564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256648+0000","last_change":"2024-04-24T23:58:18.249529+0000","last_active":"2024-04-24T23:58:20.256648+0000","last_peered":"2024-04-24T23:58:20.256648+0000","last_clean":"2024-04-24T23:58:20.256648+0000","last_became_active":"2024-04-24T23:58:18.249135+0000","last_became_peered":"2024-04-24T23:58:18.249135+0000","last_unstale":"2024-04-24T23:58:20.256648+0000","last_undegraded":"2024-04-24T23:58:20.256648+0000","last_fullsized":"2024-04-24T23:58:20.256648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256714+0000","last_change":"2024-04-24T23:58:19.266083+0000","last_active":"2024-04-24T23:58:20.256714+0000","last_peered":"2024-04-24T23:58:20.256714+0000","last_clean":"2024-04-24T23:58:20.256714+0000","last_became_active":"2024-04-24T23:58:19.265757+0000","last_became_peered":"2024-04-24T23:58:19.265757+0000","last_unstale":"2024-04-24T23:58:20.256714+0000","last_undegraded":"2024-04-24T23:58:20.256714+0000","last_fullsized":"2024-04-24T23:58:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677645+0000","last_change":"2024-04-24T23:58:18.245808+0000","last_active":"2024-04-24T23:58:20.677645+0000","last_peered":"2024-04-24T23:58:20.677645+0000","last_clean":"2024-04-24T23:58:20.677645+0000","last_became_active":"2024-04-24T23:58:18.244769+0000","last_became_peered":"2024-04-24T23:58:18.244769+0000","last_unstale":"2024-04-24T23:58:20.677645+0000","last_undegraded":"2024-04-24T23:58:20.677645+0000","last_fullsized":"2024-04-24T23:58:20.677645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678393+0000","last_change":"2024-04-24T23:58:19.262773+0000","last_active":"2024-04-24T23:58:20.678393+0000","last_peered":"2024-04-24T23:58:20.678393+0000","last_clean":"2024-04-24T23:58:20.678393+0000","last_became_active":"2024-04-24T23:58:19.262384+0000","last_became_peered":"2024-04-24T23:58:19.262384+0000","last_unstale":"2024-04-24T23:58:20.678393+0000","last_undegraded":"2024-04-24T23:58:20.678393+0000","last_fullsized":"2024-04-24T23:58:20.678393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255516+0000","last_change":"2024-04-24T23:58:18.243528+0000","last_active":"2024-04-24T23:58:20.255516+0000","last_peered":"2024-04-24T23:58:20.255516+0000","last_clean":"2024-04-24T23:58:20.255516+0000","last_became_active":"2024-04-24T23:58:18.243262+0000","last_became_peered":"2024-04-24T23:58:18.243262+0000","last_unstale":"2024-04-24T23:58:20.255516+0000","last_undegraded":"2024-04-24T23:58:20.255516+0000","last_fullsized":"2024-04-24T23:58:20.255516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677877+0000","last_change":"2024-04-24T23:58:19.261251+0000","last_active":"2024-04-24T23:58:20.677877+0000","last_peered":"2024-04-24T23:58:20.677877+0000","last_clean":"2024-04-24T23:58:20.677877+0000","last_became_active":"2024-04-24T23:58:19.260977+0000","last_became_peered":"2024-04-24T23:58:19.260977+0000","last_unstale":"2024-04-24T23:58:20.677877+0000","last_undegraded":"2024-04-24T23:58:20.677877+0000","last_fullsized":"2024-04-24T23:58:20.677877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677909+0000","last_change":"2024-04-24T23:58:18.242128+0000","last_active":"2024-04-24T23:58:20.677909+0000","last_peered":"2024-04-24T23:58:20.677909+0000","last_clean":"2024-04-24T23:58:20.677909+0000","last_became_active":"2024-04-24T23:58:18.241804+0000","last_became_peered":"2024-04-24T23:58:18.241804+0000","last_unstale":"2024-04-24T23:58:20.677909+0000","last_undegraded":"2024-04-24T23:58:20.677909+0000","last_fullsized":"2024-04-24T23:58:20.677909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678616+0000","last_change":"2024-04-24T23:58:19.262407+0000","last_active":"2024-04-24T23:58:20.678616+0000","last_peered":"2024-04-24T23:58:20.678616+0000","last_clean":"2024-04-24T23:58:20.678616+0000","last_became_active":"2024-04-24T23:58:19.262123+0000","last_became_peered":"2024-04-24T23:58:19.262123+0000","last_unstale":"2024-04-24T23:58:20.678616+0000","last_undegraded":"2024-04-24T23:58:20.678616+0000","last_fullsized":"2024-04-24T23:58:20.678616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254597+0000","last_change":"2024-04-24T23:58:18.242601+0000","last_active":"2024-04-24T23:58:20.254597+0000","last_peered":"2024-04-24T23:58:20.254597+0000","last_clean":"2024-04-24T23:58:20.254597+0000","last_became_active":"2024-04-24T23:58:18.242314+0000","last_became_peered":"2024-04-24T23:58:18.242314+0000","last_unstale":"2024-04-24T23:58:20.254597+0000","last_undegraded":"2024-04-24T23:58:20.254597+0000","last_fullsized":"2024-04-24T23:58:20.254597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678172+0000","last_change":"2024-04-24T23:58:19.265899+0000","last_active":"2024-04-24T23:58:20.678172+0000","last_peered":"2024-04-24T23:58:20.678172+0000","last_clean":"2024-04-24T23:58:20.678172+0000","last_became_active":"2024-04-24T23:58:19.265596+0000","last_became_peered":"2024-04-24T23:58:19.265596+0000","last_unstale":"2024-04-24T23:58:20.678172+0000","last_undegraded":"2024-04-24T23:58:20.678172+0000","last_fullsized":"2024-04-24T23:58:20.678172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677741+0000","last_change":"2024-04-24T23:58:18.248287+0000","last_active":"2024-04-24T23:58:20.677741+0000","last_peered":"2024-04-24T23:58:20.677741+0000","last_clean":"2024-04-24T23:58:20.677741+0000","last_became_active":"2024-04-24T23:58:18.248034+0000","last_became_peered":"2024-04-24T23:58:18.248034+0000","last_unstale":"2024-04-24T23:58:20.677741+0000","last_undegraded":"2024-04-24T23:58:20.677741+0000","last_fullsized":"2024-04-24T23:58:20.677741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678510+0000","last_change":"2024-04-24T23:58:19.274141+0000","last_active":"2024-04-24T23:58:20.678510+0000","last_peered":"2024-04-24T23:58:20.678510+0000","last_clean":"2024-04-24T23:58:20.678510+0000","last_became_active":"2024-04-24T23:58:19.273864+0000","last_became_peered":"2024-04-24T23:58:19.273864+0000","last_unstale":"2024-04-24T23:58:20.678510+0000","last_undegraded":"2024-04-24T23:58:20.678510+0000","last_fullsized":"2024-04-24T23:58:20.678510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677425+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:58:20.677425+0000","last_peered":"2024-04-24T23:58:20.677425+0000","last_clean":"2024-04-24T23:58:20.677425+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:58:20.677425+0000","last_undegraded":"2024-04-24T23:58:20.677425+0000","last_fullsized":"2024-04-24T23:58:20.677425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677567+0000","last_change":"2024-04-24T23:58:18.247362+0000","last_active":"2024-04-24T23:58:20.677567+0000","last_peered":"2024-04-24T23:58:20.677567+0000","last_clean":"2024-04-24T23:58:20.677567+0000","last_became_active":"2024-04-24T23:58:18.247084+0000","last_became_peered":"2024-04-24T23:58:18.247084+0000","last_unstale":"2024-04-24T23:58:20.677567+0000","last_undegraded":"2024-04-24T23:58:20.677567+0000","last_fullsized":"2024-04-24T23:58:20.677567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678143+0000","last_change":"2024-04-24T23:58:19.272925+0000","last_active":"2024-04-24T23:58:20.678143+0000","last_peered":"2024-04-24T23:58:20.678143+0000","last_clean":"2024-04-24T23:58:20.678143+0000","last_became_active":"2024-04-24T23:58:19.272538+0000","last_became_peered":"2024-04-24T23:58:19.272538+0000","last_unstale":"2024-04-24T23:58:20.678143+0000","last_undegraded":"2024-04-24T23:58:20.678143+0000","last_fullsized":"2024-04-24T23:58:20.678143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677369+0000","last_change":"2024-04-24T23:58:18.251207+0000","last_active":"2024-04-24T23:58:20.677369+0000","last_peered":"2024-04-24T23:58:20.677369+0000","last_clean":"2024-04-24T23:58:20.677369+0000","last_became_active":"2024-04-24T23:58:18.250921+0000","last_became_peered":"2024-04-24T23:58:18.250921+0000","last_unstale":"2024-04-24T23:58:20.677369+0000","last_undegraded":"2024-04-24T23:58:20.677369+0000","last_fullsized":"2024-04-24T23:58:20.677369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254623+0000","last_change":"2024-04-24T23:58:19.248221+0000","last_active":"2024-04-24T23:58:20.254623+0000","last_peered":"2024-04-24T23:58:20.254623+0000","last_clean":"2024-04-24T23:58:20.254623+0000","last_became_active":"2024-04-24T23:58:19.247896+0000","last_became_peered":"2024-04-24T23:58:19.247896+0000","last_unstale":"2024-04-24T23:58:20.254623+0000","last_undegraded":"2024-04-24T23:58:20.254623+0000","last_fullsized":"2024-04-24T23:58:20.254623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:24.816499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677678+0000","last_change":"2024-04-24T23:58:18.245116+0000","last_active":"2024-04-24T23:58:20.677678+0000","last_peered":"2024-04-24T23:58:20.677678+0000","last_clean":"2024-04-24T23:58:20.677678+0000","last_became_active":"2024-04-24T23:58:18.244779+0000","last_became_peered":"2024-04-24T23:58:18.244779+0000","last_unstale":"2024-04-24T23:58:20.677678+0000","last_undegraded":"2024-04-24T23:58:20.677678+0000","last_fullsized":"2024-04-24T23:58:20.677678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254292+0000","last_change":"2024-04-24T23:58:19.258623+0000","last_active":"2024-04-24T23:58:20.254292+0000","last_peered":"2024-04-24T23:58:20.254292+0000","last_clean":"2024-04-24T23:58:20.254292+0000","last_became_active":"2024-04-24T23:58:19.258371+0000","last_became_peered":"2024-04-24T23:58:19.258371+0000","last_unstale":"2024-04-24T23:58:20.254292+0000","last_undegraded":"2024-04-24T23:58:20.254292+0000","last_fullsized":"2024-04-24T23:58:20.254292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678304+0000","last_change":"2024-04-24T23:58:18.250837+0000","last_active":"2024-04-24T23:58:20.678304+0000","last_peered":"2024-04-24T23:58:20.678304+0000","last_clean":"2024-04-24T23:58:20.678304+0000","last_became_active":"2024-04-24T23:58:18.250540+0000","last_became_peered":"2024-04-24T23:58:18.250540+0000","last_unstale":"2024-04-24T23:58:20.678304+0000","last_undegraded":"2024-04-24T23:58:20.678304+0000","last_fullsized":"2024-04-24T23:58:20.678304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:13.906429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255478+0000","last_change":"2024-04-24T23:58:19.254379+0000","last_active":"2024-04-24T23:58:20.255478+0000","last_peered":"2024-04-24T23:58:20.255478+0000","last_clean":"2024-04-24T23:58:20.255478+0000","last_became_active":"2024-04-24T23:58:19.252734+0000","last_became_peered":"2024-04-24T23:58:19.252734+0000","last_unstale":"2024-04-24T23:58:20.255478+0000","last_undegraded":"2024-04-24T23:58:20.255478+0000","last_fullsized":"2024-04-24T23:58:20.255478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:59.970682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254377+0000","last_change":"2024-04-24T23:58:18.247304+0000","last_active":"2024-04-24T23:58:20.254377+0000","last_peered":"2024-04-24T23:58:20.254377+0000","last_clean":"2024-04-24T23:58:20.254377+0000","last_became_active":"2024-04-24T23:58:18.247029+0000","last_became_peered":"2024-04-24T23:58:18.247029+0000","last_unstale":"2024-04-24T23:58:20.254377+0000","last_undegraded":"2024-04-24T23:58:20.254377+0000","last_fullsized":"2024-04-24T23:58:20.254377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678233+0000","last_change":"2024-04-24T23:58:19.268921+0000","last_active":"2024-04-24T23:58:20.678233+0000","last_peered":"2024-04-24T23:58:20.678233+0000","last_clean":"2024-04-24T23:58:20.678233+0000","last_became_active":"2024-04-24T23:58:19.265287+0000","last_became_peered":"2024-04-24T23:58:19.265287+0000","last_unstale":"2024-04-24T23:58:20.678233+0000","last_undegraded":"2024-04-24T23:58:20.678233+0000","last_fullsized":"2024-04-24T23:58:20.678233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678625+0000","last_change":"2024-04-24T23:58:18.249153+0000","last_active":"2024-04-24T23:58:20.678625+0000","last_peered":"2024-04-24T23:58:20.678625+0000","last_clean":"2024-04-24T23:58:20.678625+0000","last_became_active":"2024-04-24T23:58:18.248532+0000","last_became_peered":"2024-04-24T23:58:18.248532+0000","last_unstale":"2024-04-24T23:58:20.678625+0000","last_undegraded":"2024-04-24T23:58:20.678625+0000","last_fullsized":"2024-04-24T23:58:20.678625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254293+0000","last_change":"2024-04-24T23:58:19.257617+0000","last_active":"2024-04-24T23:58:20.254293+0000","last_peered":"2024-04-24T23:58:20.254293+0000","last_clean":"2024-04-24T23:58:20.254293+0000","last_became_active":"2024-04-24T23:58:19.257312+0000","last_became_peered":"2024-04-24T23:58:19.257312+0000","last_unstale":"2024-04-24T23:58:20.254293+0000","last_undegraded":"2024-04-24T23:58:20.254293+0000","last_fullsized":"2024-04-24T23:58:20.254293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256119+0000","last_change":"2024-04-24T23:58:18.249582+0000","last_active":"2024-04-24T23:58:20.256119+0000","last_peered":"2024-04-24T23:58:20.256119+0000","last_clean":"2024-04-24T23:58:20.256119+0000","last_became_active":"2024-04-24T23:58:18.249277+0000","last_became_peered":"2024-04-24T23:58:18.249277+0000","last_unstale":"2024-04-24T23:58:20.256119+0000","last_undegraded":"2024-04-24T23:58:20.256119+0000","last_fullsized":"2024-04-24T23:58:20.256119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678235+0000","last_change":"2024-04-24T23:58:19.265003+0000","last_active":"2024-04-24T23:58:20.678235+0000","last_peered":"2024-04-24T23:58:20.678235+0000","last_clean":"2024-04-24T23:58:20.678235+0000","last_became_active":"2024-04-24T23:58:19.264714+0000","last_became_peered":"2024-04-24T23:58:19.264714+0000","last_unstale":"2024-04-24T23:58:20.678235+0000","last_undegraded":"2024-04-24T23:58:20.678235+0000","last_fullsized":"2024-04-24T23:58:20.678235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257040+0000","last_change":"2024-04-24T23:58:18.250190+0000","last_active":"2024-04-24T23:58:20.257040+0000","last_peered":"2024-04-24T23:58:20.257040+0000","last_clean":"2024-04-24T23:58:20.257040+0000","last_became_active":"2024-04-24T23:58:18.249896+0000","last_became_peered":"2024-04-24T23:58:18.249896+0000","last_unstale":"2024-04-24T23:58:20.257040+0000","last_undegraded":"2024-04-24T23:58:20.257040+0000","last_fullsized":"2024-04-24T23:58:20.257040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255362+0000","last_change":"2024-04-24T23:58:19.261332+0000","last_active":"2024-04-24T23:58:20.255362+0000","last_peered":"2024-04-24T23:58:20.255362+0000","last_clean":"2024-04-24T23:58:20.255362+0000","last_became_active":"2024-04-24T23:58:19.261058+0000","last_became_peered":"2024-04-24T23:58:19.261058+0000","last_unstale":"2024-04-24T23:58:20.255362+0000","last_undegraded":"2024-04-24T23:58:20.255362+0000","last_fullsized":"2024-04-24T23:58:20.255362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677359+0000","last_change":"2024-04-24T23:58:18.251910+0000","last_active":"2024-04-24T23:58:20.677359+0000","last_peered":"2024-04-24T23:58:20.677359+0000","last_clean":"2024-04-24T23:58:20.677359+0000","last_became_active":"2024-04-24T23:58:18.251629+0000","last_became_peered":"2024-04-24T23:58:18.251629+0000","last_unstale":"2024-04-24T23:58:20.677359+0000","last_undegraded":"2024-04-24T23:58:20.677359+0000","last_fullsized":"2024-04-24T23:58:20.677359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255236+0000","last_change":"2024-04-24T23:58:19.256009+0000","last_active":"2024-04-24T23:58:20.255236+0000","last_peered":"2024-04-24T23:58:20.255236+0000","last_clean":"2024-04-24T23:58:20.255236+0000","last_became_active":"2024-04-24T23:58:19.255734+0000","last_became_peered":"2024-04-24T23:58:19.255734+0000","last_unstale":"2024-04-24T23:58:20.255236+0000","last_undegraded":"2024-04-24T23:58:20.255236+0000","last_fullsized":"2024-04-24T23:58:20.255236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:49.658803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254568+0000","last_change":"2024-04-24T23:58:18.250223+0000","last_active":"2024-04-24T23:58:20.254568+0000","last_peered":"2024-04-24T23:58:20.254568+0000","last_clean":"2024-04-24T23:58:20.254568+0000","last_became_active":"2024-04-24T23:58:18.249995+0000","last_became_peered":"2024-04-24T23:58:18.249995+0000","last_unstale":"2024-04-24T23:58:20.254568+0000","last_undegraded":"2024-04-24T23:58:20.254568+0000","last_fullsized":"2024-04-24T23:58:20.254568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.258258+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.258011+0000","last_became_peered":"2024-04-24T23:58:19.258011+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254356+0000","last_change":"2024-04-24T23:58:18.243148+0000","last_active":"2024-04-24T23:58:20.254356+0000","last_peered":"2024-04-24T23:58:20.254356+0000","last_clean":"2024-04-24T23:58:20.254356+0000","last_became_active":"2024-04-24T23:58:18.242863+0000","last_became_peered":"2024-04-24T23:58:18.242863+0000","last_unstale":"2024-04-24T23:58:20.254356+0000","last_undegraded":"2024-04-24T23:58:20.254356+0000","last_fullsized":"2024-04-24T23:58:20.254356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678415+0000","last_change":"2024-04-24T23:58:19.268382+0000","last_active":"2024-04-24T23:58:20.678415+0000","last_peered":"2024-04-24T23:58:20.678415+0000","last_clean":"2024-04-24T23:58:20.678415+0000","last_became_active":"2024-04-24T23:58:19.268132+0000","last_became_peered":"2024-04-24T23:58:19.268132+0000","last_unstale":"2024-04-24T23:58:20.678415+0000","last_undegraded":"2024-04-24T23:58:20.678415+0000","last_fullsized":"2024-04-24T23:58:20.678415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678646+0000","last_change":"2024-04-24T23:58:18.253346+0000","last_active":"2024-04-24T23:58:20.678646+0000","last_peered":"2024-04-24T23:58:20.678646+0000","last_clean":"2024-04-24T23:58:20.678646+0000","last_became_active":"2024-04-24T23:58:18.253094+0000","last_became_peered":"2024-04-24T23:58:18.253094+0000","last_unstale":"2024-04-24T23:58:20.678646+0000","last_undegraded":"2024-04-24T23:58:20.678646+0000","last_fullsized":"2024-04-24T23:58:20.678646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256888+0000","last_change":"2024-04-24T23:58:19.260209+0000","last_active":"2024-04-24T23:58:20.256888+0000","last_peered":"2024-04-24T23:58:20.256888+0000","last_clean":"2024-04-24T23:58:20.256888+0000","last_became_active":"2024-04-24T23:58:19.259941+0000","last_became_peered":"2024-04-24T23:58:19.259941+0000","last_unstale":"2024-04-24T23:58:20.256888+0000","last_undegraded":"2024-04-24T23:58:20.256888+0000","last_fullsized":"2024-04-24T23:58:20.256888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678038+0000","last_change":"2024-04-24T23:58:19.262656+0000","last_active":"2024-04-24T23:58:20.678038+0000","last_peered":"2024-04-24T23:58:20.678038+0000","last_clean":"2024-04-24T23:58:20.678038+0000","last_became_active":"2024-04-24T23:58:19.262043+0000","last_became_peered":"2024-04-24T23:58:19.262043+0000","last_unstale":"2024-04-24T23:58:20.678038+0000","last_undegraded":"2024-04-24T23:58:20.678038+0000","last_fullsized":"2024-04-24T23:58:20.678038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677720+0000","last_change":"2024-04-24T23:58:18.249958+0000","last_active":"2024-04-24T23:58:20.677720+0000","last_peered":"2024-04-24T23:58:20.677720+0000","last_clean":"2024-04-24T23:58:20.677720+0000","last_became_active":"2024-04-24T23:58:18.249660+0000","last_became_peered":"2024-04-24T23:58:18.249660+0000","last_unstale":"2024-04-24T23:58:20.677720+0000","last_undegraded":"2024-04-24T23:58:20.677720+0000","last_fullsized":"2024-04-24T23:58:20.677720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678297+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678297+0000","last_peered":"2024-04-24T23:58:20.678297+0000","last_clean":"2024-04-24T23:58:20.678297+0000","last_became_active":"2024-04-24T23:58:19.262976+0000","last_became_peered":"2024-04-24T23:58:19.262976+0000","last_unstale":"2024-04-24T23:58:20.678297+0000","last_undegraded":"2024-04-24T23:58:20.678297+0000","last_fullsized":"2024-04-24T23:58:20.678297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678466+0000","last_change":"2024-04-24T23:58:18.250117+0000","last_active":"2024-04-24T23:58:20.678466+0000","last_peered":"2024-04-24T23:58:20.678466+0000","last_clean":"2024-04-24T23:58:20.678466+0000","last_became_active":"2024-04-24T23:58:18.249797+0000","last_became_peered":"2024-04-24T23:58:18.249797+0000","last_unstale":"2024-04-24T23:58:20.678466+0000","last_undegraded":"2024-04-24T23:58:20.678466+0000","last_fullsized":"2024-04-24T23:58:20.678466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254736+0000","last_change":"2024-04-24T23:58:19.253506+0000","last_active":"2024-04-24T23:58:20.254736+0000","last_peered":"2024-04-24T23:58:20.254736+0000","last_clean":"2024-04-24T23:58:20.254736+0000","last_became_active":"2024-04-24T23:58:19.253032+0000","last_became_peered":"2024-04-24T23:58:19.253032+0000","last_unstale":"2024-04-24T23:58:20.254736+0000","last_undegraded":"2024-04-24T23:58:20.254736+0000","last_fullsized":"2024-04-24T23:58:20.254736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:14.910044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678152+0000","last_change":"2024-04-24T23:58:18.244986+0000","last_active":"2024-04-24T23:58:20.678152+0000","last_peered":"2024-04-24T23:58:20.678152+0000","last_clean":"2024-04-24T23:58:20.678152+0000","last_became_active":"2024-04-24T23:58:18.244674+0000","last_became_peered":"2024-04-24T23:58:18.244674+0000","last_unstale":"2024-04-24T23:58:20.678152+0000","last_undegraded":"2024-04-24T23:58:20.678152+0000","last_fullsized":"2024-04-24T23:58:20.678152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678003+0000","last_change":"2024-04-24T23:58:19.261722+0000","last_active":"2024-04-24T23:58:20.678003+0000","last_peered":"2024-04-24T23:58:20.678003+0000","last_clean":"2024-04-24T23:58:20.678003+0000","last_became_active":"2024-04-24T23:58:19.261474+0000","last_became_peered":"2024-04-24T23:58:19.261474+0000","last_unstale":"2024-04-24T23:58:20.678003+0000","last_undegraded":"2024-04-24T23:58:20.678003+0000","last_fullsized":"2024-04-24T23:58:20.678003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255072+0000","last_change":"2024-04-24T23:58:18.240520+0000","last_active":"2024-04-24T23:58:20.255072+0000","last_peered":"2024-04-24T23:58:20.255072+0000","last_clean":"2024-04-24T23:58:20.255072+0000","last_became_active":"2024-04-24T23:58:18.240253+0000","last_became_peered":"2024-04-24T23:58:18.240253+0000","last_unstale":"2024-04-24T23:58:20.255072+0000","last_undegraded":"2024-04-24T23:58:20.255072+0000","last_fullsized":"2024-04-24T23:58:20.255072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:17.091843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:58:27.106 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:58:27.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.214+0000 7f395c3e2640 1 Processor -- start 2024-04-24T23:58:27.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.214+0000 7f395c3e2640 1 -- start start 2024-04-24T23:58:27.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395c3e2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39540b3fd0 0x7f395406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395c3e2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3954144cf0 con 0x7f3954144910 2024-04-24T23:58:27.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395c3e2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39540b2f30 con 0x7f395406e510 2024-04-24T23:58:27.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395c3e2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f395406e980 con 0x7f39540b3fd0 2024-04-24T23:58:27.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395997d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39540b3fd0 0x7f395406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395a17e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3954144910 0x7f39540b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41014/0 (socket says 172.21.15.113:41014) 2024-04-24T23:58:27.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395a17e640 1 -- 172.21.15.113:0/2824465072 learned_addr learned my addr 172.21.15.113:0/2824465072 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:27.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395997d640 1 -- 172.21.15.113:0/2824465072 >> v1:172.21.15.178:6789/0 conn(0x7f395406e510 legacy=0x7f39540cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:27.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395997d640 1 -- 172.21.15.113:0/2824465072 >> v1:172.21.15.113:6789/0 conn(0x7f3954144910 legacy=0x7f39540b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.215+0000 7f395997d640 1 -- 172.21.15.113:0/2824465072 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39540b3210 con 0x7f39540b3fd0 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.216+0000 7f395997d640 1 --2- 172.21.15.113:0/2824465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39540b3fd0 0x7f395406dfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39440097d0 tx=0x7f394402c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80577cc7de51090 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.216+0000 7f395917c640 1 -- 172.21.15.113:0/2824465072 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944038070 con 0x7f39540b3fd0 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.216+0000 7f395917c640 1 -- 172.21.15.113:0/2824465072 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3944031710 con 0x7f39540b3fd0 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.216+0000 7f395917c640 1 -- 172.21.15.113:0/2824465072 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944031ae0 con 0x7f39540b3fd0 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.217+0000 7f395c3e2640 1 -- 172.21.15.113:0/2824465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39540b3fd0 msgr2=0x7f395406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.217+0000 7f395c3e2640 1 --2- 172.21.15.113:0/2824465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39540b3fd0 0x7f395406dfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39440097d0 tx=0x7f394402c420 comp rx=0 tx=0).stop 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.217+0000 7f395c3e2640 1 -- 172.21.15.113:0/2824465072 shutdown_connections 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.217+0000 7f395c3e2640 1 --2- 172.21.15.113:0/2824465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39540b3fd0 0x7f395406dfd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.217+0000 7f395c3e2640 1 -- 172.21.15.113:0/2824465072 >> 172.21.15.113:0/2824465072 conn(0x7f39540b1a50 msgr2=0x7f39540b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:27.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.217+0000 7f395c3e2640 1 -- 172.21.15.113:0/2824465072 shutdown_connections 2024-04-24T23:58:27.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.217+0000 7f395c3e2640 1 -- 172.21.15.113:0/2824465072 wait complete. 2024-04-24T23:58:27.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.218+0000 7f395c3e2640 1 Processor -- start 2024-04-24T23:58:27.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.218+0000 7f395c3e2640 1 -- start start 2024-04-24T23:58:27.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f395406e510 0x7f39540af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540b3fd0 0x7f3954142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954144910 0x7f39540abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f39540b4440 con 0x7f395406e510 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f39540b37d0 con 0x7f3954144910 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f39540b4da0 con 0x7f39540b3fd0 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f395406e510 0x7f39540af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f395406e510 0x7f39540af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49206/0 (socket says 172.21.15.113:49206) 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 -- 172.21.15.113:0/3200527505 learned_addr learned my addr 172.21.15.113:0/3200527505 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395997d640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540b3fd0 0x7f3954142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a97f640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954144910 0x7f39540abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 -- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540b3fd0 msgr2=0x7f3954142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540b3fd0 0x7f3954142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 -- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954144910 msgr2=0x7f39540abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954144910 0x7f39540abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 -- 172.21.15.113:0/3200527505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39540a9520 con 0x7f395406e510 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395a17e640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f395406e510 0x7f39540af000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f395000c3c0 tx=0x7f395000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f394affd640 1 -- 172.21.15.113:0/3200527505 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950017070 con 0x7f395406e510 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f394affd640 1 -- 172.21.15.113:0/3200527505 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3950005ce0 con 0x7f395406e510 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f394affd640 1 -- 172.21.15.113:0/3200527505 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500072f0 con 0x7f395406e510 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39541428d0 con 0x7f395406e510 2024-04-24T23:58:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.219+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3954142e90 con 0x7f395406e510 2024-04-24T23:58:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.220+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39540a96f0 con 0x7f395406e510 2024-04-24T23:58:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.221+0000 7f394affd640 1 -- 172.21.15.113:0/3200527505 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f395001a070 con 0x7f395406e510 2024-04-24T23:58:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.221+0000 7f394affd640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f393806be60 0x7f393806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.221+0000 7f395997d640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f393806be60 0x7f393806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.221+0000 7f394affd640 1 -- 172.21.15.113:0/3200527505 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f395008a160 con 0x7f395406e510 2024-04-24T23:58:27.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.222+0000 7f395997d640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f393806be60 0x7f393806e320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3944035000 tx=0x7f3944008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.226+0000 7f394affd640 1 -- 172.21.15.113:0/3200527505 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3950058430 con 0x7f395406e510 2024-04-24T23:58:27.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.400+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39540731a0 con 0x7f393806be60 2024-04-24T23:58:27.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.406+0000 7f394affd640 1 -- 172.21.15.113:0/3200527505 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f39540731a0 con 0x7f393806be60 2024-04-24T23:58:27.410 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f393806be60 msgr2=0x7f393806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f393806be60 0x7f393806e320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3944035000 tx=0x7f3944008030 comp rx=0 tx=0).stop 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f395406e510 msgr2=0x7f39540af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f395406e510 0x7f39540af000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f395000c3c0 tx=0x7f395000c880 comp rx=0 tx=0).stop 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 shutdown_connections 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f393806be60 0x7f393806e320 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954144910 0x7f39540abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540b3fd0 0x7f3954142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 --2- 172.21.15.113:0/3200527505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f395406e510 0x7f39540af000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 >> 172.21.15.113:0/3200527505 conn(0x7f39540b1a50 msgr2=0x7f3954071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 shutdown_connections 2024-04-24T23:58:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.409+0000 7f395c3e2640 1 -- 172.21.15.113:0/3200527505 wait complete. 2024-04-24T23:58:27.419 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-24T23:58:27.113389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235632,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739088,"statfs":{"total":773094113280,"available":772852826112,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001439"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678418+0000","last_change":"2024-04-24T23:58:19.270520+0000","last_active":"2024-04-24T23:58:20.678418+0000","last_peered":"2024-04-24T23:58:20.678418+0000","last_clean":"2024-04-24T23:58:20.678418+0000","last_became_active":"2024-04-24T23:58:19.270280+0000","last_became_peered":"2024-04-24T23:58:19.270280+0000","last_unstale":"2024-04-24T23:58:20.678418+0000","last_undegraded":"2024-04-24T23:58:20.678418+0000","last_fullsized":"2024-04-24T23:58:20.678418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254502+0000","last_change":"2024-04-24T23:58:19.252635+0000","last_active":"2024-04-24T23:58:20.254502+0000","last_peered":"2024-04-24T23:58:20.254502+0000","last_clean":"2024-04-24T23:58:20.254502+0000","last_became_active":"2024-04-24T23:58:19.252352+0000","last_became_peered":"2024-04-24T23:58:19.252352+0000","last_unstale":"2024-04-24T23:58:20.254502+0000","last_undegraded":"2024-04-24T23:58:20.254502+0000","last_fullsized":"2024-04-24T23:58:20.254502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254874+0000","last_change":"2024-04-24T23:58:19.257041+0000","last_active":"2024-04-24T23:58:20.254874+0000","last_peered":"2024-04-24T23:58:20.254874+0000","last_clean":"2024-04-24T23:58:20.254874+0000","last_became_active":"2024-04-24T23:58:19.256765+0000","last_became_peered":"2024-04-24T23:58:19.256765+0000","last_unstale":"2024-04-24T23:58:20.254874+0000","last_undegraded":"2024-04-24T23:58:20.254874+0000","last_fullsized":"2024-04-24T23:58:20.254874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.271981+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.271589+0000","last_became_peered":"2024-04-24T23:58:19.271589+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678540+0000","last_change":"2024-04-24T23:58:19.260799+0000","last_active":"2024-04-24T23:58:20.678540+0000","last_peered":"2024-04-24T23:58:20.678540+0000","last_clean":"2024-04-24T23:58:20.678540+0000","last_became_active":"2024-04-24T23:58:19.260437+0000","last_became_peered":"2024-04-24T23:58:19.260437+0000","last_unstale":"2024-04-24T23:58:20.678540+0000","last_undegraded":"2024-04-24T23:58:20.678540+0000","last_fullsized":"2024-04-24T23:58:20.678540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677754+0000","last_change":"2024-04-24T23:58:19.269955+0000","last_active":"2024-04-24T23:58:20.677754+0000","last_peered":"2024-04-24T23:58:20.677754+0000","last_clean":"2024-04-24T23:58:20.677754+0000","last_became_active":"2024-04-24T23:58:19.269675+0000","last_became_peered":"2024-04-24T23:58:19.269675+0000","last_unstale":"2024-04-24T23:58:20.677754+0000","last_undegraded":"2024-04-24T23:58:20.677754+0000","last_fullsized":"2024-04-24T23:58:20.677754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678346+0000","last_change":"2024-04-24T23:58:19.267752+0000","last_active":"2024-04-24T23:58:20.678346+0000","last_peered":"2024-04-24T23:58:20.678346+0000","last_clean":"2024-04-24T23:58:20.678346+0000","last_became_active":"2024-04-24T23:58:19.267463+0000","last_became_peered":"2024-04-24T23:58:19.267463+0000","last_unstale":"2024-04-24T23:58:20.678346+0000","last_undegraded":"2024-04-24T23:58:20.678346+0000","last_fullsized":"2024-04-24T23:58:20.678346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678319+0000","last_change":"2024-04-24T23:58:19.266736+0000","last_active":"2024-04-24T23:58:20.678319+0000","last_peered":"2024-04-24T23:58:20.678319+0000","last_clean":"2024-04-24T23:58:20.678319+0000","last_became_active":"2024-04-24T23:58:19.266438+0000","last_became_peered":"2024-04-24T23:58:19.266438+0000","last_unstale":"2024-04-24T23:58:20.678319+0000","last_undegraded":"2024-04-24T23:58:20.678319+0000","last_fullsized":"2024-04-24T23:58:20.678319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678538+0000","last_change":"2024-04-24T23:58:19.271086+0000","last_active":"2024-04-24T23:58:20.678538+0000","last_peered":"2024-04-24T23:58:20.678538+0000","last_clean":"2024-04-24T23:58:20.678538+0000","last_became_active":"2024-04-24T23:58:19.270722+0000","last_became_peered":"2024-04-24T23:58:19.270722+0000","last_unstale":"2024-04-24T23:58:20.678538+0000","last_undegraded":"2024-04-24T23:58:20.678538+0000","last_fullsized":"2024-04-24T23:58:20.678538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257102+0000","last_change":"2024-04-24T23:58:19.266845+0000","last_active":"2024-04-24T23:58:20.257102+0000","last_peered":"2024-04-24T23:58:20.257102+0000","last_clean":"2024-04-24T23:58:20.257102+0000","last_became_active":"2024-04-24T23:58:19.266573+0000","last_became_peered":"2024-04-24T23:58:19.266573+0000","last_unstale":"2024-04-24T23:58:20.257102+0000","last_undegraded":"2024-04-24T23:58:20.257102+0000","last_fullsized":"2024-04-24T23:58:20.257102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256863+0000","last_change":"2024-04-24T23:58:19.267588+0000","last_active":"2024-04-24T23:58:20.256863+0000","last_peered":"2024-04-24T23:58:20.256863+0000","last_clean":"2024-04-24T23:58:20.256863+0000","last_became_active":"2024-04-24T23:58:19.267247+0000","last_became_peered":"2024-04-24T23:58:19.267247+0000","last_unstale":"2024-04-24T23:58:20.256863+0000","last_undegraded":"2024-04-24T23:58:20.256863+0000","last_fullsized":"2024-04-24T23:58:20.256863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255357+0000","last_change":"2024-04-24T23:58:19.257310+0000","last_active":"2024-04-24T23:58:20.255357+0000","last_peered":"2024-04-24T23:58:20.255357+0000","last_clean":"2024-04-24T23:58:20.255357+0000","last_became_active":"2024-04-24T23:58:19.257023+0000","last_became_peered":"2024-04-24T23:58:19.257023+0000","last_unstale":"2024-04-24T23:58:20.255357+0000","last_undegraded":"2024-04-24T23:58:20.255357+0000","last_fullsized":"2024-04-24T23:58:20.255357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678413+0000","last_change":"2024-04-24T23:58:19.261091+0000","last_active":"2024-04-24T23:58:20.678413+0000","last_peered":"2024-04-24T23:58:20.678413+0000","last_clean":"2024-04-24T23:58:20.678413+0000","last_became_active":"2024-04-24T23:58:19.260837+0000","last_became_peered":"2024-04-24T23:58:19.260837+0000","last_unstale":"2024-04-24T23:58:20.678413+0000","last_undegraded":"2024-04-24T23:58:20.678413+0000","last_fullsized":"2024-04-24T23:58:20.678413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254811+0000","last_change":"2024-04-24T23:58:19.261328+0000","last_active":"2024-04-24T23:58:20.254811+0000","last_peered":"2024-04-24T23:58:20.254811+0000","last_clean":"2024-04-24T23:58:20.254811+0000","last_became_active":"2024-04-24T23:58:19.261045+0000","last_became_peered":"2024-04-24T23:58:19.261045+0000","last_unstale":"2024-04-24T23:58:20.254811+0000","last_undegraded":"2024-04-24T23:58:20.254811+0000","last_fullsized":"2024-04-24T23:58:20.254811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254583+0000","last_change":"2024-04-24T23:58:19.256356+0000","last_active":"2024-04-24T23:58:20.254583+0000","last_peered":"2024-04-24T23:58:20.254583+0000","last_clean":"2024-04-24T23:58:20.254583+0000","last_became_active":"2024-04-24T23:58:19.256099+0000","last_became_peered":"2024-04-24T23:58:19.256099+0000","last_unstale":"2024-04-24T23:58:20.254583+0000","last_undegraded":"2024-04-24T23:58:20.254583+0000","last_fullsized":"2024-04-24T23:58:20.254583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677420+0000","last_change":"2024-04-24T23:58:19.268419+0000","last_active":"2024-04-24T23:58:20.677420+0000","last_peered":"2024-04-24T23:58:20.677420+0000","last_clean":"2024-04-24T23:58:20.677420+0000","last_became_active":"2024-04-24T23:58:19.265742+0000","last_became_peered":"2024-04-24T23:58:19.265742+0000","last_unstale":"2024-04-24T23:58:20.677420+0000","last_undegraded":"2024-04-24T23:58:20.677420+0000","last_fullsized":"2024-04-24T23:58:20.677420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.273024+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.272551+0000","last_became_peered":"2024-04-24T23:58:19.272551+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.258361+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.258098+0000","last_became_peered":"2024-04-24T23:58:19.258098+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678669+0000","last_change":"2024-04-24T23:58:19.263085+0000","last_active":"2024-04-24T23:58:20.678669+0000","last_peered":"2024-04-24T23:58:20.678669+0000","last_clean":"2024-04-24T23:58:20.678669+0000","last_became_active":"2024-04-24T23:58:19.262807+0000","last_became_peered":"2024-04-24T23:58:19.262807+0000","last_unstale":"2024-04-24T23:58:20.678669+0000","last_undegraded":"2024-04-24T23:58:20.678669+0000","last_fullsized":"2024-04-24T23:58:20.678669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256867+0000","last_change":"2024-04-24T23:58:19.259501+0000","last_active":"2024-04-24T23:58:20.256867+0000","last_peered":"2024-04-24T23:58:20.256867+0000","last_clean":"2024-04-24T23:58:20.256867+0000","last_became_active":"2024-04-24T23:58:19.259233+0000","last_became_peered":"2024-04-24T23:58:19.259233+0000","last_unstale":"2024-04-24T23:58:20.256867+0000","last_undegraded":"2024-04-24T23:58:20.256867+0000","last_fullsized":"2024-04-24T23:58:20.256867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255041+0000","last_change":"2024-04-24T23:58:19.259960+0000","last_active":"2024-04-24T23:58:20.255041+0000","last_peered":"2024-04-24T23:58:20.255041+0000","last_clean":"2024-04-24T23:58:20.255041+0000","last_became_active":"2024-04-24T23:58:19.259548+0000","last_became_peered":"2024-04-24T23:58:19.259548+0000","last_unstale":"2024-04-24T23:58:20.255041+0000","last_undegraded":"2024-04-24T23:58:20.255041+0000","last_fullsized":"2024-04-24T23:58:20.255041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677851+0000","last_change":"2024-04-24T23:58:19.275499+0000","last_active":"2024-04-24T23:58:20.677851+0000","last_peered":"2024-04-24T23:58:20.677851+0000","last_clean":"2024-04-24T23:58:20.677851+0000","last_became_active":"2024-04-24T23:58:19.275213+0000","last_became_peered":"2024-04-24T23:58:19.275213+0000","last_unstale":"2024-04-24T23:58:20.677851+0000","last_undegraded":"2024-04-24T23:58:20.677851+0000","last_fullsized":"2024-04-24T23:58:20.677851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678362+0000","last_change":"2024-04-24T23:58:19.261963+0000","last_active":"2024-04-24T23:58:20.678362+0000","last_peered":"2024-04-24T23:58:20.678362+0000","last_clean":"2024-04-24T23:58:20.678362+0000","last_became_active":"2024-04-24T23:58:19.261710+0000","last_became_peered":"2024-04-24T23:58:19.261710+0000","last_unstale":"2024-04-24T23:58:20.678362+0000","last_undegraded":"2024-04-24T23:58:20.678362+0000","last_fullsized":"2024-04-24T23:58:20.678362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677517+0000","last_change":"2024-04-24T23:58:19.268171+0000","last_active":"2024-04-24T23:58:20.677517+0000","last_peered":"2024-04-24T23:58:20.677517+0000","last_clean":"2024-04-24T23:58:20.677517+0000","last_became_active":"2024-04-24T23:58:19.265379+0000","last_became_peered":"2024-04-24T23:58:19.265379+0000","last_unstale":"2024-04-24T23:58:20.677517+0000","last_undegraded":"2024-04-24T23:58:20.677517+0000","last_fullsized":"2024-04-24T23:58:20.677517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678064+0000","last_change":"2024-04-24T23:58:19.259615+0000","last_active":"2024-04-24T23:58:20.678064+0000","last_peered":"2024-04-24T23:58:20.678064+0000","last_clean":"2024-04-24T23:58:20.678064+0000","last_became_active":"2024-04-24T23:58:19.259355+0000","last_became_peered":"2024-04-24T23:58:19.259355+0000","last_unstale":"2024-04-24T23:58:20.678064+0000","last_undegraded":"2024-04-24T23:58:20.678064+0000","last_fullsized":"2024-04-24T23:58:20.678064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257106+0000","last_change":"2024-04-24T23:58:19.269905+0000","last_active":"2024-04-24T23:58:20.257106+0000","last_peered":"2024-04-24T23:58:20.257106+0000","last_clean":"2024-04-24T23:58:20.257106+0000","last_became_active":"2024-04-24T23:58:19.269639+0000","last_became_peered":"2024-04-24T23:58:19.269639+0000","last_unstale":"2024-04-24T23:58:20.257106+0000","last_undegraded":"2024-04-24T23:58:20.257106+0000","last_fullsized":"2024-04-24T23:58:20.257106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254304+0000","last_change":"2024-04-24T23:58:19.256747+0000","last_active":"2024-04-24T23:58:20.254304+0000","last_peered":"2024-04-24T23:58:20.254304+0000","last_clean":"2024-04-24T23:58:20.254304+0000","last_became_active":"2024-04-24T23:58:19.255723+0000","last_became_peered":"2024-04-24T23:58:19.255723+0000","last_unstale":"2024-04-24T23:58:20.254304+0000","last_undegraded":"2024-04-24T23:58:20.254304+0000","last_fullsized":"2024-04-24T23:58:20.254304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678051+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678051+0000","last_peered":"2024-04-24T23:58:20.678051+0000","last_clean":"2024-04-24T23:58:20.678051+0000","last_became_active":"2024-04-24T23:58:19.262977+0000","last_became_peered":"2024-04-24T23:58:19.262977+0000","last_unstale":"2024-04-24T23:58:20.678051+0000","last_undegraded":"2024-04-24T23:58:20.678051+0000","last_fullsized":"2024-04-24T23:58:20.678051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255177+0000","last_change":"2024-04-24T23:58:19.259745+0000","last_active":"2024-04-24T23:58:20.255177+0000","last_peered":"2024-04-24T23:58:20.255177+0000","last_clean":"2024-04-24T23:58:20.255177+0000","last_became_active":"2024-04-24T23:58:19.259201+0000","last_became_peered":"2024-04-24T23:58:19.259201+0000","last_unstale":"2024-04-24T23:58:20.255177+0000","last_undegraded":"2024-04-24T23:58:20.255177+0000","last_fullsized":"2024-04-24T23:58:20.255177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678071+0000","last_change":"2024-04-24T23:58:19.273016+0000","last_active":"2024-04-24T23:58:20.678071+0000","last_peered":"2024-04-24T23:58:20.678071+0000","last_clean":"2024-04-24T23:58:20.678071+0000","last_became_active":"2024-04-24T23:58:19.271540+0000","last_became_peered":"2024-04-24T23:58:19.271540+0000","last_unstale":"2024-04-24T23:58:20.678071+0000","last_undegraded":"2024-04-24T23:58:20.678071+0000","last_fullsized":"2024-04-24T23:58:20.678071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677646+0000","last_change":"2024-04-24T23:58:19.255458+0000","last_active":"2024-04-24T23:58:20.677646+0000","last_peered":"2024-04-24T23:58:20.677646+0000","last_clean":"2024-04-24T23:58:20.677646+0000","last_became_active":"2024-04-24T23:58:19.255070+0000","last_became_peered":"2024-04-24T23:58:19.255070+0000","last_unstale":"2024-04-24T23:58:20.677646+0000","last_undegraded":"2024-04-24T23:58:20.677646+0000","last_fullsized":"2024-04-24T23:58:20.677646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256632+0000","last_change":"2024-04-24T23:58:19.266657+0000","last_active":"2024-04-24T23:58:20.256632+0000","last_peered":"2024-04-24T23:58:20.256632+0000","last_clean":"2024-04-24T23:58:20.256632+0000","last_became_active":"2024-04-24T23:58:19.266354+0000","last_became_peered":"2024-04-24T23:58:19.266354+0000","last_unstale":"2024-04-24T23:58:20.256632+0000","last_undegraded":"2024-04-24T23:58:20.256632+0000","last_fullsized":"2024-04-24T23:58:20.256632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255049+0000","last_change":"2024-04-24T23:58:19.259253+0000","last_active":"2024-04-24T23:58:20.255049+0000","last_peered":"2024-04-24T23:58:20.255049+0000","last_clean":"2024-04-24T23:58:20.255049+0000","last_became_active":"2024-04-24T23:58:19.258553+0000","last_became_peered":"2024-04-24T23:58:19.258553+0000","last_unstale":"2024-04-24T23:58:20.255049+0000","last_undegraded":"2024-04-24T23:58:20.255049+0000","last_fullsized":"2024-04-24T23:58:20.255049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:53.284799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678523+0000","last_change":"2024-04-24T23:58:18.250614+0000","last_active":"2024-04-24T23:58:20.678523+0000","last_peered":"2024-04-24T23:58:20.678523+0000","last_clean":"2024-04-24T23:58:20.678523+0000","last_became_active":"2024-04-24T23:58:18.250306+0000","last_became_peered":"2024-04-24T23:58:18.250306+0000","last_unstale":"2024-04-24T23:58:20.678523+0000","last_undegraded":"2024-04-24T23:58:20.678523+0000","last_fullsized":"2024-04-24T23:58:20.678523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255184+0000","last_change":"2024-04-24T23:58:19.251648+0000","last_active":"2024-04-24T23:58:20.255184+0000","last_peered":"2024-04-24T23:58:20.255184+0000","last_clean":"2024-04-24T23:58:20.255184+0000","last_became_active":"2024-04-24T23:58:19.251271+0000","last_became_peered":"2024-04-24T23:58:19.251271+0000","last_unstale":"2024-04-24T23:58:20.255184+0000","last_undegraded":"2024-04-24T23:58:20.255184+0000","last_fullsized":"2024-04-24T23:58:20.255184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256708+0000","last_change":"2024-04-24T23:58:18.252764+0000","last_active":"2024-04-24T23:58:20.256708+0000","last_peered":"2024-04-24T23:58:20.256708+0000","last_clean":"2024-04-24T23:58:20.256708+0000","last_became_active":"2024-04-24T23:58:18.252491+0000","last_became_peered":"2024-04-24T23:58:18.252491+0000","last_unstale":"2024-04-24T23:58:20.256708+0000","last_undegraded":"2024-04-24T23:58:20.256708+0000","last_fullsized":"2024-04-24T23:58:20.256708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678357+0000","last_change":"2024-04-24T23:58:18.250003+0000","last_active":"2024-04-24T23:58:20.678357+0000","last_peered":"2024-04-24T23:58:20.678357+0000","last_clean":"2024-04-24T23:58:20.678357+0000","last_became_active":"2024-04-24T23:58:18.249727+0000","last_became_peered":"2024-04-24T23:58:18.249727+0000","last_unstale":"2024-04-24T23:58:20.678357+0000","last_undegraded":"2024-04-24T23:58:20.678357+0000","last_fullsized":"2024-04-24T23:58:20.678357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.318157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678399+0000","last_change":"2024-04-24T23:58:19.255033+0000","last_active":"2024-04-24T23:58:20.678399+0000","last_peered":"2024-04-24T23:58:20.678399+0000","last_clean":"2024-04-24T23:58:20.678399+0000","last_became_active":"2024-04-24T23:58:19.254746+0000","last_became_peered":"2024-04-24T23:58:19.254746+0000","last_unstale":"2024-04-24T23:58:20.678399+0000","last_undegraded":"2024-04-24T23:58:20.678399+0000","last_fullsized":"2024-04-24T23:58:20.678399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677538+0000","last_change":"2024-04-24T23:58:18.245516+0000","last_active":"2024-04-24T23:58:20.677538+0000","last_peered":"2024-04-24T23:58:20.677538+0000","last_clean":"2024-04-24T23:58:20.677538+0000","last_became_active":"2024-04-24T23:58:18.245212+0000","last_became_peered":"2024-04-24T23:58:18.245212+0000","last_unstale":"2024-04-24T23:58:20.677538+0000","last_undegraded":"2024-04-24T23:58:20.677538+0000","last_fullsized":"2024-04-24T23:58:20.677538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254479+0000","last_change":"2024-04-24T23:58:19.258354+0000","last_active":"2024-04-24T23:58:20.254479+0000","last_peered":"2024-04-24T23:58:20.254479+0000","last_clean":"2024-04-24T23:58:20.254479+0000","last_became_active":"2024-04-24T23:58:19.258067+0000","last_became_peered":"2024-04-24T23:58:19.258067+0000","last_unstale":"2024-04-24T23:58:20.254479+0000","last_undegraded":"2024-04-24T23:58:20.254479+0000","last_fullsized":"2024-04-24T23:58:20.254479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256985+0000","last_change":"2024-04-24T23:58:18.253806+0000","last_active":"2024-04-24T23:58:20.256985+0000","last_peered":"2024-04-24T23:58:20.256985+0000","last_clean":"2024-04-24T23:58:20.256985+0000","last_became_active":"2024-04-24T23:58:18.252657+0000","last_became_peered":"2024-04-24T23:58:18.252657+0000","last_unstale":"2024-04-24T23:58:20.256985+0000","last_undegraded":"2024-04-24T23:58:20.256985+0000","last_fullsized":"2024-04-24T23:58:20.256985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677952+0000","last_change":"2024-04-24T23:58:19.275012+0000","last_active":"2024-04-24T23:58:20.677952+0000","last_peered":"2024-04-24T23:58:20.677952+0000","last_clean":"2024-04-24T23:58:20.677952+0000","last_became_active":"2024-04-24T23:58:19.274740+0000","last_became_peered":"2024-04-24T23:58:19.274740+0000","last_unstale":"2024-04-24T23:58:20.677952+0000","last_undegraded":"2024-04-24T23:58:20.677952+0000","last_fullsized":"2024-04-24T23:58:20.677952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678001+0000","last_change":"2024-04-24T23:58:18.251763+0000","last_active":"2024-04-24T23:58:20.678001+0000","last_peered":"2024-04-24T23:58:20.678001+0000","last_clean":"2024-04-24T23:58:20.678001+0000","last_became_active":"2024-04-24T23:58:18.251508+0000","last_became_peered":"2024-04-24T23:58:18.251508+0000","last_unstale":"2024-04-24T23:58:20.678001+0000","last_undegraded":"2024-04-24T23:58:20.678001+0000","last_fullsized":"2024-04-24T23:58:20.678001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254979+0000","last_change":"2024-04-24T23:58:19.253275+0000","last_active":"2024-04-24T23:58:20.254979+0000","last_peered":"2024-04-24T23:58:20.254979+0000","last_clean":"2024-04-24T23:58:20.254979+0000","last_became_active":"2024-04-24T23:58:19.252783+0000","last_became_peered":"2024-04-24T23:58:19.252783+0000","last_unstale":"2024-04-24T23:58:20.254979+0000","last_undegraded":"2024-04-24T23:58:20.254979+0000","last_fullsized":"2024-04-24T23:58:20.254979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254408+0000","last_change":"2024-04-24T23:58:18.246446+0000","last_active":"2024-04-24T23:58:20.254408+0000","last_peered":"2024-04-24T23:58:20.254408+0000","last_clean":"2024-04-24T23:58:20.254408+0000","last_became_active":"2024-04-24T23:58:18.246036+0000","last_became_peered":"2024-04-24T23:58:18.246036+0000","last_unstale":"2024-04-24T23:58:20.254408+0000","last_undegraded":"2024-04-24T23:58:20.254408+0000","last_fullsized":"2024-04-24T23:58:20.254408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255261+0000","last_change":"2024-04-24T23:58:19.261655+0000","last_active":"2024-04-24T23:58:20.255261+0000","last_peered":"2024-04-24T23:58:20.255261+0000","last_clean":"2024-04-24T23:58:20.255261+0000","last_became_active":"2024-04-24T23:58:19.261331+0000","last_became_peered":"2024-04-24T23:58:19.261331+0000","last_unstale":"2024-04-24T23:58:20.255261+0000","last_undegraded":"2024-04-24T23:58:20.255261+0000","last_fullsized":"2024-04-24T23:58:20.255261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256970+0000","last_change":"2024-04-24T23:58:18.247264+0000","last_active":"2024-04-24T23:58:20.256970+0000","last_peered":"2024-04-24T23:58:20.256970+0000","last_clean":"2024-04-24T23:58:20.256970+0000","last_became_active":"2024-04-24T23:58:18.246959+0000","last_became_peered":"2024-04-24T23:58:18.246959+0000","last_unstale":"2024-04-24T23:58:20.256970+0000","last_undegraded":"2024-04-24T23:58:20.256970+0000","last_fullsized":"2024-04-24T23:58:20.256970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678015+0000","last_change":"2024-04-24T23:58:19.265255+0000","last_active":"2024-04-24T23:58:20.678015+0000","last_peered":"2024-04-24T23:58:20.678015+0000","last_clean":"2024-04-24T23:58:20.678015+0000","last_became_active":"2024-04-24T23:58:19.264997+0000","last_became_peered":"2024-04-24T23:58:19.264997+0000","last_unstale":"2024-04-24T23:58:20.678015+0000","last_undegraded":"2024-04-24T23:58:20.678015+0000","last_fullsized":"2024-04-24T23:58:20.678015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255275+0000","last_change":"2024-04-24T23:58:18.236929+0000","last_active":"2024-04-24T23:58:20.255275+0000","last_peered":"2024-04-24T23:58:20.255275+0000","last_clean":"2024-04-24T23:58:20.255275+0000","last_became_active":"2024-04-24T23:58:18.236660+0000","last_became_peered":"2024-04-24T23:58:18.236660+0000","last_unstale":"2024-04-24T23:58:20.255275+0000","last_undegraded":"2024-04-24T23:58:20.255275+0000","last_fullsized":"2024-04-24T23:58:20.255275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254944+0000","last_change":"2024-04-24T23:58:19.259532+0000","last_active":"2024-04-24T23:58:20.254944+0000","last_peered":"2024-04-24T23:58:20.254944+0000","last_clean":"2024-04-24T23:58:20.254944+0000","last_became_active":"2024-04-24T23:58:19.259298+0000","last_became_peered":"2024-04-24T23:58:19.259298+0000","last_unstale":"2024-04-24T23:58:20.254944+0000","last_undegraded":"2024-04-24T23:58:20.254944+0000","last_fullsized":"2024-04-24T23:58:20.254944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678194+0000","last_change":"2024-04-24T23:58:18.251116+0000","last_active":"2024-04-24T23:58:20.678194+0000","last_peered":"2024-04-24T23:58:20.678194+0000","last_clean":"2024-04-24T23:58:20.678194+0000","last_became_active":"2024-04-24T23:58:18.250814+0000","last_became_peered":"2024-04-24T23:58:18.250814+0000","last_unstale":"2024-04-24T23:58:20.678194+0000","last_undegraded":"2024-04-24T23:58:20.678194+0000","last_fullsized":"2024-04-24T23:58:20.678194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678564+0000","last_change":"2024-04-24T23:58:19.263724+0000","last_active":"2024-04-24T23:58:20.678564+0000","last_peered":"2024-04-24T23:58:20.678564+0000","last_clean":"2024-04-24T23:58:20.678564+0000","last_became_active":"2024-04-24T23:58:19.263448+0000","last_became_peered":"2024-04-24T23:58:19.263448+0000","last_unstale":"2024-04-24T23:58:20.678564+0000","last_undegraded":"2024-04-24T23:58:20.678564+0000","last_fullsized":"2024-04-24T23:58:20.678564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256648+0000","last_change":"2024-04-24T23:58:18.249529+0000","last_active":"2024-04-24T23:58:20.256648+0000","last_peered":"2024-04-24T23:58:20.256648+0000","last_clean":"2024-04-24T23:58:20.256648+0000","last_became_active":"2024-04-24T23:58:18.249135+0000","last_became_peered":"2024-04-24T23:58:18.249135+0000","last_unstale":"2024-04-24T23:58:20.256648+0000","last_undegraded":"2024-04-24T23:58:20.256648+0000","last_fullsized":"2024-04-24T23:58:20.256648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256714+0000","last_change":"2024-04-24T23:58:19.266083+0000","last_active":"2024-04-24T23:58:20.256714+0000","last_peered":"2024-04-24T23:58:20.256714+0000","last_clean":"2024-04-24T23:58:20.256714+0000","last_became_active":"2024-04-24T23:58:19.265757+0000","last_became_peered":"2024-04-24T23:58:19.265757+0000","last_unstale":"2024-04-24T23:58:20.256714+0000","last_undegraded":"2024-04-24T23:58:20.256714+0000","last_fullsized":"2024-04-24T23:58:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677645+0000","last_change":"2024-04-24T23:58:18.245808+0000","last_active":"2024-04-24T23:58:20.677645+0000","last_peered":"2024-04-24T23:58:20.677645+0000","last_clean":"2024-04-24T23:58:20.677645+0000","last_became_active":"2024-04-24T23:58:18.244769+0000","last_became_peered":"2024-04-24T23:58:18.244769+0000","last_unstale":"2024-04-24T23:58:20.677645+0000","last_undegraded":"2024-04-24T23:58:20.677645+0000","last_fullsized":"2024-04-24T23:58:20.677645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678393+0000","last_change":"2024-04-24T23:58:19.262773+0000","last_active":"2024-04-24T23:58:20.678393+0000","last_peered":"2024-04-24T23:58:20.678393+0000","last_clean":"2024-04-24T23:58:20.678393+0000","last_became_active":"2024-04-24T23:58:19.262384+0000","last_became_peered":"2024-04-24T23:58:19.262384+0000","last_unstale":"2024-04-24T23:58:20.678393+0000","last_undegraded":"2024-04-24T23:58:20.678393+0000","last_fullsized":"2024-04-24T23:58:20.678393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255516+0000","last_change":"2024-04-24T23:58:18.243528+0000","last_active":"2024-04-24T23:58:20.255516+0000","last_peered":"2024-04-24T23:58:20.255516+0000","last_clean":"2024-04-24T23:58:20.255516+0000","last_became_active":"2024-04-24T23:58:18.243262+0000","last_became_peered":"2024-04-24T23:58:18.243262+0000","last_unstale":"2024-04-24T23:58:20.255516+0000","last_undegraded":"2024-04-24T23:58:20.255516+0000","last_fullsized":"2024-04-24T23:58:20.255516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677877+0000","last_change":"2024-04-24T23:58:19.261251+0000","last_active":"2024-04-24T23:58:20.677877+0000","last_peered":"2024-04-24T23:58:20.677877+0000","last_clean":"2024-04-24T23:58:20.677877+0000","last_became_active":"2024-04-24T23:58:19.260977+0000","last_became_peered":"2024-04-24T23:58:19.260977+0000","last_unstale":"2024-04-24T23:58:20.677877+0000","last_undegraded":"2024-04-24T23:58:20.677877+0000","last_fullsized":"2024-04-24T23:58:20.677877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677909+0000","last_change":"2024-04-24T23:58:18.242128+0000","last_active":"2024-04-24T23:58:20.677909+0000","last_peered":"2024-04-24T23:58:20.677909+0000","last_clean":"2024-04-24T23:58:20.677909+0000","last_became_active":"2024-04-24T23:58:18.241804+0000","last_became_peered":"2024-04-24T23:58:18.241804+0000","last_unstale":"2024-04-24T23:58:20.677909+0000","last_undegraded":"2024-04-24T23:58:20.677909+0000","last_fullsized":"2024-04-24T23:58:20.677909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678616+0000","last_change":"2024-04-24T23:58:19.262407+0000","last_active":"2024-04-24T23:58:20.678616+0000","last_peered":"2024-04-24T23:58:20.678616+0000","last_clean":"2024-04-24T23:58:20.678616+0000","last_became_active":"2024-04-24T23:58:19.262123+0000","last_became_peered":"2024-04-24T23:58:19.262123+0000","last_unstale":"2024-04-24T23:58:20.678616+0000","last_undegraded":"2024-04-24T23:58:20.678616+0000","last_fullsized":"2024-04-24T23:58:20.678616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254597+0000","last_change":"2024-04-24T23:58:18.242601+0000","last_active":"2024-04-24T23:58:20.254597+0000","last_peered":"2024-04-24T23:58:20.254597+0000","last_clean":"2024-04-24T23:58:20.254597+0000","last_became_active":"2024-04-24T23:58:18.242314+0000","last_became_peered":"2024-04-24T23:58:18.242314+0000","last_unstale":"2024-04-24T23:58:20.254597+0000","last_undegraded":"2024-04-24T23:58:20.254597+0000","last_fullsized":"2024-04-24T23:58:20.254597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678172+0000","last_change":"2024-04-24T23:58:19.265899+0000","last_active":"2024-04-24T23:58:20.678172+0000","last_peered":"2024-04-24T23:58:20.678172+0000","last_clean":"2024-04-24T23:58:20.678172+0000","last_became_active":"2024-04-24T23:58:19.265596+0000","last_became_peered":"2024-04-24T23:58:19.265596+0000","last_unstale":"2024-04-24T23:58:20.678172+0000","last_undegraded":"2024-04-24T23:58:20.678172+0000","last_fullsized":"2024-04-24T23:58:20.678172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677741+0000","last_change":"2024-04-24T23:58:18.248287+0000","last_active":"2024-04-24T23:58:20.677741+0000","last_peered":"2024-04-24T23:58:20.677741+0000","last_clean":"2024-04-24T23:58:20.677741+0000","last_became_active":"2024-04-24T23:58:18.248034+0000","last_became_peered":"2024-04-24T23:58:18.248034+0000","last_unstale":"2024-04-24T23:58:20.677741+0000","last_undegraded":"2024-04-24T23:58:20.677741+0000","last_fullsized":"2024-04-24T23:58:20.677741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678510+0000","last_change":"2024-04-24T23:58:19.274141+0000","last_active":"2024-04-24T23:58:20.678510+0000","last_peered":"2024-04-24T23:58:20.678510+0000","last_clean":"2024-04-24T23:58:20.678510+0000","last_became_active":"2024-04-24T23:58:19.273864+0000","last_became_peered":"2024-04-24T23:58:19.273864+0000","last_unstale":"2024-04-24T23:58:20.678510+0000","last_undegraded":"2024-04-24T23:58:20.678510+0000","last_fullsized":"2024-04-24T23:58:20.678510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677425+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:58:20.677425+0000","last_peered":"2024-04-24T23:58:20.677425+0000","last_clean":"2024-04-24T23:58:20.677425+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:58:20.677425+0000","last_undegraded":"2024-04-24T23:58:20.677425+0000","last_fullsized":"2024-04-24T23:58:20.677425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677567+0000","last_change":"2024-04-24T23:58:18.247362+0000","last_active":"2024-04-24T23:58:20.677567+0000","last_peered":"2024-04-24T23:58:20.677567+0000","last_clean":"2024-04-24T23:58:20.677567+0000","last_became_active":"2024-04-24T23:58:18.247084+0000","last_became_peered":"2024-04-24T23:58:18.247084+0000","last_unstale":"2024-04-24T23:58:20.677567+0000","last_undegraded":"2024-04-24T23:58:20.677567+0000","last_fullsized":"2024-04-24T23:58:20.677567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678143+0000","last_change":"2024-04-24T23:58:19.272925+0000","last_active":"2024-04-24T23:58:20.678143+0000","last_peered":"2024-04-24T23:58:20.678143+0000","last_clean":"2024-04-24T23:58:20.678143+0000","last_became_active":"2024-04-24T23:58:19.272538+0000","last_became_peered":"2024-04-24T23:58:19.272538+0000","last_unstale":"2024-04-24T23:58:20.678143+0000","last_undegraded":"2024-04-24T23:58:20.678143+0000","last_fullsized":"2024-04-24T23:58:20.678143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677369+0000","last_change":"2024-04-24T23:58:18.251207+0000","last_active":"2024-04-24T23:58:20.677369+0000","last_peered":"2024-04-24T23:58:20.677369+0000","last_clean":"2024-04-24T23:58:20.677369+0000","last_became_active":"2024-04-24T23:58:18.250921+0000","last_became_peered":"2024-04-24T23:58:18.250921+0000","last_unstale":"2024-04-24T23:58:20.677369+0000","last_undegraded":"2024-04-24T23:58:20.677369+0000","last_fullsized":"2024-04-24T23:58:20.677369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254623+0000","last_change":"2024-04-24T23:58:19.248221+0000","last_active":"2024-04-24T23:58:20.254623+0000","last_peered":"2024-04-24T23:58:20.254623+0000","last_clean":"2024-04-24T23:58:20.254623+0000","last_became_active":"2024-04-24T23:58:19.247896+0000","last_became_peered":"2024-04-24T23:58:19.247896+0000","last_unstale":"2024-04-24T23:58:20.254623+0000","last_undegraded":"2024-04-24T23:58:20.254623+0000","last_fullsized":"2024-04-24T23:58:20.254623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:24.816499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677678+0000","last_change":"2024-04-24T23:58:18.245116+0000","last_active":"2024-04-24T23:58:20.677678+0000","last_peered":"2024-04-24T23:58:20.677678+0000","last_clean":"2024-04-24T23:58:20.677678+0000","last_became_active":"2024-04-24T23:58:18.244779+0000","last_became_peered":"2024-04-24T23:58:18.244779+0000","last_unstale":"2024-04-24T23:58:20.677678+0000","last_undegraded":"2024-04-24T23:58:20.677678+0000","last_fullsized":"2024-04-24T23:58:20.677678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254292+0000","last_change":"2024-04-24T23:58:19.258623+0000","last_active":"2024-04-24T23:58:20.254292+0000","last_peered":"2024-04-24T23:58:20.254292+0000","last_clean":"2024-04-24T23:58:20.254292+0000","last_became_active":"2024-04-24T23:58:19.258371+0000","last_became_peered":"2024-04-24T23:58:19.258371+0000","last_unstale":"2024-04-24T23:58:20.254292+0000","last_undegraded":"2024-04-24T23:58:20.254292+0000","last_fullsized":"2024-04-24T23:58:20.254292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678304+0000","last_change":"2024-04-24T23:58:18.250837+0000","last_active":"2024-04-24T23:58:20.678304+0000","last_peered":"2024-04-24T23:58:20.678304+0000","last_clean":"2024-04-24T23:58:20.678304+0000","last_became_active":"2024-04-24T23:58:18.250540+0000","last_became_peered":"2024-04-24T23:58:18.250540+0000","last_unstale":"2024-04-24T23:58:20.678304+0000","last_undegraded":"2024-04-24T23:58:20.678304+0000","last_fullsized":"2024-04-24T23:58:20.678304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:13.906429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255478+0000","last_change":"2024-04-24T23:58:19.254379+0000","last_active":"2024-04-24T23:58:20.255478+0000","last_peered":"2024-04-24T23:58:20.255478+0000","last_clean":"2024-04-24T23:58:20.255478+0000","last_became_active":"2024-04-24T23:58:19.252734+0000","last_became_peered":"2024-04-24T23:58:19.252734+0000","last_unstale":"2024-04-24T23:58:20.255478+0000","last_undegraded":"2024-04-24T23:58:20.255478+0000","last_fullsized":"2024-04-24T23:58:20.255478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:59.970682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254377+0000","last_change":"2024-04-24T23:58:18.247304+0000","last_active":"2024-04-24T23:58:20.254377+0000","last_peered":"2024-04-24T23:58:20.254377+0000","last_clean":"2024-04-24T23:58:20.254377+0000","last_became_active":"2024-04-24T23:58:18.247029+0000","last_became_peered":"2024-04-24T23:58:18.247029+0000","last_unstale":"2024-04-24T23:58:20.254377+0000","last_undegraded":"2024-04-24T23:58:20.254377+0000","last_fullsized":"2024-04-24T23:58:20.254377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678233+0000","last_change":"2024-04-24T23:58:19.268921+0000","last_active":"2024-04-24T23:58:20.678233+0000","last_peered":"2024-04-24T23:58:20.678233+0000","last_clean":"2024-04-24T23:58:20.678233+0000","last_became_active":"2024-04-24T23:58:19.265287+0000","last_became_peered":"2024-04-24T23:58:19.265287+0000","last_unstale":"2024-04-24T23:58:20.678233+0000","last_undegraded":"2024-04-24T23:58:20.678233+0000","last_fullsized":"2024-04-24T23:58:20.678233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678625+0000","last_change":"2024-04-24T23:58:18.249153+0000","last_active":"2024-04-24T23:58:20.678625+0000","last_peered":"2024-04-24T23:58:20.678625+0000","last_clean":"2024-04-24T23:58:20.678625+0000","last_became_active":"2024-04-24T23:58:18.248532+0000","last_became_peered":"2024-04-24T23:58:18.248532+0000","last_unstale":"2024-04-24T23:58:20.678625+0000","last_undegraded":"2024-04-24T23:58:20.678625+0000","last_fullsized":"2024-04-24T23:58:20.678625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254293+0000","last_change":"2024-04-24T23:58:19.257617+0000","last_active":"2024-04-24T23:58:20.254293+0000","last_peered":"2024-04-24T23:58:20.254293+0000","last_clean":"2024-04-24T23:58:20.254293+0000","last_became_active":"2024-04-24T23:58:19.257312+0000","last_became_peered":"2024-04-24T23:58:19.257312+0000","last_unstale":"2024-04-24T23:58:20.254293+0000","last_undegraded":"2024-04-24T23:58:20.254293+0000","last_fullsized":"2024-04-24T23:58:20.254293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256119+0000","last_change":"2024-04-24T23:58:18.249582+0000","last_active":"2024-04-24T23:58:20.256119+0000","last_peered":"2024-04-24T23:58:20.256119+0000","last_clean":"2024-04-24T23:58:20.256119+0000","last_became_active":"2024-04-24T23:58:18.249277+0000","last_became_peered":"2024-04-24T23:58:18.249277+0000","last_unstale":"2024-04-24T23:58:20.256119+0000","last_undegraded":"2024-04-24T23:58:20.256119+0000","last_fullsized":"2024-04-24T23:58:20.256119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678235+0000","last_change":"2024-04-24T23:58:19.265003+0000","last_active":"2024-04-24T23:58:20.678235+0000","last_peered":"2024-04-24T23:58:20.678235+0000","last_clean":"2024-04-24T23:58:20.678235+0000","last_became_active":"2024-04-24T23:58:19.264714+0000","last_became_peered":"2024-04-24T23:58:19.264714+0000","last_unstale":"2024-04-24T23:58:20.678235+0000","last_undegraded":"2024-04-24T23:58:20.678235+0000","last_fullsized":"2024-04-24T23:58:20.678235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257040+0000","last_change":"2024-04-24T23:58:18.250190+0000","last_active":"2024-04-24T23:58:20.257040+0000","last_peered":"2024-04-24T23:58:20.257040+0000","last_clean":"2024-04-24T23:58:20.257040+0000","last_became_active":"2024-04-24T23:58:18.249896+0000","last_became_peered":"2024-04-24T23:58:18.249896+0000","last_unstale":"2024-04-24T23:58:20.257040+0000","last_undegraded":"2024-04-24T23:58:20.257040+0000","last_fullsized":"2024-04-24T23:58:20.257040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255362+0000","last_change":"2024-04-24T23:58:19.261332+0000","last_active":"2024-04-24T23:58:20.255362+0000","last_peered":"2024-04-24T23:58:20.255362+0000","last_clean":"2024-04-24T23:58:20.255362+0000","last_became_active":"2024-04-24T23:58:19.261058+0000","last_became_peered":"2024-04-24T23:58:19.261058+0000","last_unstale":"2024-04-24T23:58:20.255362+0000","last_undegraded":"2024-04-24T23:58:20.255362+0000","last_fullsized":"2024-04-24T23:58:20.255362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677359+0000","last_change":"2024-04-24T23:58:18.251910+0000","last_active":"2024-04-24T23:58:20.677359+0000","last_peered":"2024-04-24T23:58:20.677359+0000","last_clean":"2024-04-24T23:58:20.677359+0000","last_became_active":"2024-04-24T23:58:18.251629+0000","last_became_peered":"2024-04-24T23:58:18.251629+0000","last_unstale":"2024-04-24T23:58:20.677359+0000","last_undegraded":"2024-04-24T23:58:20.677359+0000","last_fullsized":"2024-04-24T23:58:20.677359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255236+0000","last_change":"2024-04-24T23:58:19.256009+0000","last_active":"2024-04-24T23:58:20.255236+0000","last_peered":"2024-04-24T23:58:20.255236+0000","last_clean":"2024-04-24T23:58:20.255236+0000","last_became_active":"2024-04-24T23:58:19.255734+0000","last_became_peered":"2024-04-24T23:58:19.255734+0000","last_unstale":"2024-04-24T23:58:20.255236+0000","last_undegraded":"2024-04-24T23:58:20.255236+0000","last_fullsized":"2024-04-24T23:58:20.255236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:49.658803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254568+0000","last_change":"2024-04-24T23:58:18.250223+0000","last_active":"2024-04-24T23:58:20.254568+0000","last_peered":"2024-04-24T23:58:20.254568+0000","last_clean":"2024-04-24T23:58:20.254568+0000","last_became_active":"2024-04-24T23:58:18.249995+0000","last_became_peered":"2024-04-24T23:58:18.249995+0000","last_unstale":"2024-04-24T23:58:20.254568+0000","last_undegraded":"2024-04-24T23:58:20.254568+0000","last_fullsized":"2024-04-24T23:58:20.254568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.258258+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.258011+0000","last_became_peered":"2024-04-24T23:58:19.258011+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254356+0000","last_change":"2024-04-24T23:58:18.243148+0000","last_active":"2024-04-24T23:58:20.254356+0000","last_peered":"2024-04-24T23:58:20.254356+0000","last_clean":"2024-04-24T23:58:20.254356+0000","last_became_active":"2024-04-24T23:58:18.242863+0000","last_became_peered":"2024-04-24T23:58:18.242863+0000","last_unstale":"2024-04-24T23:58:20.254356+0000","last_undegraded":"2024-04-24T23:58:20.254356+0000","last_fullsized":"2024-04-24T23:58:20.254356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678415+0000","last_change":"2024-04-24T23:58:19.268382+0000","last_active":"2024-04-24T23:58:20.678415+0000","last_peered":"2024-04-24T23:58:20.678415+0000","last_clean":"2024-04-24T23:58:20.678415+0000","last_became_active":"2024-04-24T23:58:19.268132+0000","last_became_peered":"2024-04-24T23:58:19.268132+0000","last_unstale":"2024-04-24T23:58:20.678415+0000","last_undegraded":"2024-04-24T23:58:20.678415+0000","last_fullsized":"2024-04-24T23:58:20.678415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678646+0000","last_change":"2024-04-24T23:58:18.253346+0000","last_active":"2024-04-24T23:58:20.678646+0000","last_peered":"2024-04-24T23:58:20.678646+0000","last_clean":"2024-04-24T23:58:20.678646+0000","last_became_active":"2024-04-24T23:58:18.253094+0000","last_became_peered":"2024-04-24T23:58:18.253094+0000","last_unstale":"2024-04-24T23:58:20.678646+0000","last_undegraded":"2024-04-24T23:58:20.678646+0000","last_fullsized":"2024-04-24T23:58:20.678646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256888+0000","last_change":"2024-04-24T23:58:19.260209+0000","last_active":"2024-04-24T23:58:20.256888+0000","last_peered":"2024-04-24T23:58:20.256888+0000","last_clean":"2024-04-24T23:58:20.256888+0000","last_became_active":"2024-04-24T23:58:19.259941+0000","last_became_peered":"2024-04-24T23:58:19.259941+0000","last_unstale":"2024-04-24T23:58:20.256888+0000","last_undegraded":"2024-04-24T23:58:20.256888+0000","last_fullsized":"2024-04-24T23:58:20.256888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678038+0000","last_change":"2024-04-24T23:58:19.262656+0000","last_active":"2024-04-24T23:58:20.678038+0000","last_peered":"2024-04-24T23:58:20.678038+0000","last_clean":"2024-04-24T23:58:20.678038+0000","last_became_active":"2024-04-24T23:58:19.262043+0000","last_became_peered":"2024-04-24T23:58:19.262043+0000","last_unstale":"2024-04-24T23:58:20.678038+0000","last_undegraded":"2024-04-24T23:58:20.678038+0000","last_fullsized":"2024-04-24T23:58:20.678038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677720+0000","last_change":"2024-04-24T23:58:18.249958+0000","last_active":"2024-04-24T23:58:20.677720+0000","last_peered":"2024-04-24T23:58:20.677720+0000","last_clean":"2024-04-24T23:58:20.677720+0000","last_became_active":"2024-04-24T23:58:18.249660+0000","last_became_peered":"2024-04-24T23:58:18.249660+0000","last_unstale":"2024-04-24T23:58:20.677720+0000","last_undegraded":"2024-04-24T23:58:20.677720+0000","last_fullsized":"2024-04-24T23:58:20.677720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678297+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678297+0000","last_peered":"2024-04-24T23:58:20.678297+0000","last_clean":"2024-04-24T23:58:20.678297+0000","last_became_active":"2024-04-24T23:58:19.262976+0000","last_became_peered":"2024-04-24T23:58:19.262976+0000","last_unstale":"2024-04-24T23:58:20.678297+0000","last_undegraded":"2024-04-24T23:58:20.678297+0000","last_fullsized":"2024-04-24T23:58:20.678297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678466+0000","last_change":"2024-04-24T23:58:18.250117+0000","last_active":"2024-04-24T23:58:20.678466+0000","last_peered":"2024-04-24T23:58:20.678466+0000","last_clean":"2024-04-24T23:58:20.678466+0000","last_became_active":"2024-04-24T23:58:18.249797+0000","last_became_peered":"2024-04-24T23:58:18.249797+0000","last_unstale":"2024-04-24T23:58:20.678466+0000","last_undegraded":"2024-04-24T23:58:20.678466+0000","last_fullsized":"2024-04-24T23:58:20.678466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254736+0000","last_change":"2024-04-24T23:58:19.253506+0000","last_active":"2024-04-24T23:58:20.254736+0000","last_peered":"2024-04-24T23:58:20.254736+0000","last_clean":"2024-04-24T23:58:20.254736+0000","last_became_active":"2024-04-24T23:58:19.253032+0000","last_became_peered":"2024-04-24T23:58:19.253032+0000","last_unstale":"2024-04-24T23:58:20.254736+0000","last_undegraded":"2024-04-24T23:58:20.254736+0000","last_fullsized":"2024-04-24T23:58:20.254736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:14.910044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678152+0000","last_change":"2024-04-24T23:58:18.244986+0000","last_active":"2024-04-24T23:58:20.678152+0000","last_peered":"2024-04-24T23:58:20.678152+0000","last_clean":"2024-04-24T23:58:20.678152+0000","last_became_active":"2024-04-24T23:58:18.244674+0000","last_became_peered":"2024-04-24T23:58:18.244674+0000","last_unstale":"2024-04-24T23:58:20.678152+0000","last_undegraded":"2024-04-24T23:58:20.678152+0000","last_fullsized":"2024-04-24T23:58:20.678152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678003+0000","last_change":"2024-04-24T23:58:19.261722+0000","last_active":"2024-04-24T23:58:20.678003+0000","last_peered":"2024-04-24T23:58:20.678003+0000","last_clean":"2024-04-24T23:58:20.678003+0000","last_became_active":"2024-04-24T23:58:19.261474+0000","last_became_peered":"2024-04-24T23:58:19.261474+0000","last_unstale":"2024-04-24T23:58:20.678003+0000","last_undegraded":"2024-04-24T23:58:20.678003+0000","last_fullsized":"2024-04-24T23:58:20.678003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255072+0000","last_change":"2024-04-24T23:58:18.240520+0000","last_active":"2024-04-24T23:58:20.255072+0000","last_peered":"2024-04-24T23:58:20.255072+0000","last_clean":"2024-04-24T23:58:20.255072+0000","last_became_active":"2024-04-24T23:58:18.240253+0000","last_became_peered":"2024-04-24T23:58:18.240253+0000","last_unstale":"2024-04-24T23:58:20.255072+0000","last_undegraded":"2024-04-24T23:58:20.255072+0000","last_fullsized":"2024-04-24T23:58:20.255072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:17.091843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:58:27.424 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:58:27.424 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:58:27.424 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:58:27.424 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T23:58:27.425 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T23:58:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.555+0000 7fb83bad4640 1 Processor -- start 2024-04-24T23:58:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.555+0000 7fb83bad4640 1 -- start start 2024-04-24T23:58:27.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb83bad4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb83415e320 0x7fb8340d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb83bad4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb8340d2290 0x7fb8340b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb83bad4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8340d3de0 con 0x7fb8340d0b00 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb83bad4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb8340d3410 con 0x7fb83415e320 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb83bad4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb8340d2800 con 0x7fb8340d2290 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb839870640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb83415e320 0x7fb8340d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb83a071640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb8340d0b00 0x7fb8340b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55454/0 (socket says 172.21.15.113:55454) 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.556+0000 7fb83a071640 1 -- 172.21.15.113:0/986600426 learned_addr learned my addr 172.21.15.113:0/986600426 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.557+0000 7fb839870640 1 -- 172.21.15.113:0/986600426 >> v2:172.21.15.178:3300/0 conn(0x7fb8340d2290 msgr2=0x7fb8340b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.557+0000 7fb839870640 1 --2- 172.21.15.113:0/986600426 >> v2:172.21.15.178:3300/0 conn(0x7fb8340d2290 0x7fb8340b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.557+0000 7fb839870640 1 -- 172.21.15.113:0/986600426 >> v1:172.21.15.113:6789/0 conn(0x7fb8340d0b00 legacy=0x7fb8340b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.557+0000 7fb839870640 1 -- 172.21.15.113:0/986600426 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8340d2ae0 con 0x7fb83415e320 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.557+0000 7fb839870640 1 --2- 172.21.15.113:0/986600426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb83415e320 0x7fb8340d38a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb834072810 tx=0x7fb83002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad5081b24bb7c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.557+0000 7fb83886e640 1 -- 172.21.15.113:0/986600426 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830007740 con 0x7fb83415e320 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.557+0000 7fb83886e640 1 -- 172.21.15.113:0/986600426 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8300078e0 con 0x7fb83415e320 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83886e640 1 -- 172.21.15.113:0/986600426 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830007cb0 con 0x7fb83415e320 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 -- 172.21.15.113:0/986600426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb83415e320 msgr2=0x7fb8340d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 --2- 172.21.15.113:0/986600426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb83415e320 0x7fb8340d38a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb834072810 tx=0x7fb83002a710 comp rx=0 tx=0).stop 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 -- 172.21.15.113:0/986600426 shutdown_connections 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 --2- 172.21.15.113:0/986600426 >> v2:172.21.15.178:3300/0 conn(0x7fb8340d2290 0x7fb8340b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 --2- 172.21.15.113:0/986600426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb83415e320 0x7fb8340d38a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 -- 172.21.15.113:0/986600426 >> 172.21.15.113:0/986600426 conn(0x7fb8340b5c20 msgr2=0x7fb8340b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 -- 172.21.15.113:0/986600426 shutdown_connections 2024-04-24T23:58:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.558+0000 7fb83bad4640 1 -- 172.21.15.113:0/986600426 wait complete. 2024-04-24T23:58:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.559+0000 7fb83bad4640 1 Processor -- start 2024-04-24T23:58:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.559+0000 7fb83bad4640 1 -- start start 2024-04-24T23:58:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83bad4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8340d0b00 0x7fb834161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83bad4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8340d2290 0x7fb834158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83bad4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb83415e320 0x7fb8340b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83bad4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340b9fe0 con 0x7fb8340d0b00 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83bad4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340d30a0 con 0x7fb8340d2290 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83bad4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb83415e700 con 0x7fb83415e320 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8340d0b00 0x7fb834161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8340d0b00 0x7fb834161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50024/0 (socket says 172.21.15.113:50024) 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 -- 172.21.15.113:0/926439927 learned_addr learned my addr 172.21.15.113:0/926439927 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83906f640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8340d2290 0x7fb834158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83a071640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb83415e320 0x7fb8340b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 -- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb83415e320 msgr2=0x7fb8340b7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb83415e320 0x7fb8340b7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 -- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8340d2290 msgr2=0x7fb834158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8340d2290 0x7fb834158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 -- 172.21.15.113:0/926439927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb834160470 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb83a071640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb83415e320 0x7fb8340b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.560+0000 7fb839870640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8340d0b00 0x7fb834161510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb83002ad00 tx=0x7fb830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.561+0000 7fb82a7fc640 1 -- 172.21.15.113:0/926439927 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83003f070 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.561+0000 7fb82a7fc640 1 -- 172.21.15.113:0/926439927 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb830031d00 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.561+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb834160750 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.561+0000 7fb82a7fc640 1 -- 172.21.15.113:0/926439927 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830044660 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.561+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb834160d10 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.562+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8340691a0 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.562+0000 7fb82a7fc640 1 -- 172.21.15.113:0/926439927 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb83003b410 con 0x7fb8340d0b00 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.562+0000 7fb82a7fc640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81806be30 0x7fb81806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.562+0000 7fb82a7fc640 1 -- 172.21.15.113:0/926439927 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8300ac760 con 0x7fb8340d0b00 2024-04-24T23:58:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.562+0000 7fb83906f640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81806be30 0x7fb81806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.563+0000 7fb83906f640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81806be30 0x7fb81806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb83415db00 tx=0x7fb824008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.567+0000 7fb82a7fc640 1 -- 172.21.15.113:0/926439927 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb83007ab50 con 0x7fb8340d0b00 2024-04-24T23:58:27.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.783+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb834072320 con 0x7fb8340d0b00 2024-04-24T23:58:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.786+0000 7fb82a7fc640 1 -- 172.21.15.113:0/926439927 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172946 (secure 0 0 0) 0x7fb83007ee50 con 0x7fb8340d0b00 2024-04-24T23:58:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:27.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81806be30 msgr2=0x7fb81806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81806be30 0x7fb81806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb83415db00 tx=0x7fb824008040 comp rx=0 tx=0).stop 2024-04-24T23:58:27.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8340d0b00 msgr2=0x7fb834161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8340d0b00 0x7fb834161510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb83002ad00 tx=0x7fb830004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:27.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 shutdown_connections 2024-04-24T23:58:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81806be30 0x7fb81806e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8340d2290 0x7fb834158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb83415e320 0x7fb8340b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 --2- 172.21.15.113:0/926439927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8340d0b00 0x7fb834161510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 >> 172.21.15.113:0/926439927 conn(0x7fb8340b5c20 msgr2=0x7fb8340b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 shutdown_connections 2024-04-24T23:58:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.789+0000 7fb83bad4640 1 -- 172.21.15.113:0/926439927 wait complete. 2024-04-24T23:58:27.799 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3117017300},{"type":"v1","addr":"172.21.15.113:6833","nonce":3117017300}]},"active_addr":"172.21.15.113:6833/3117017300","active_change":"2024-04-24T23:57:59.101907+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1238366772}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3868816009}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1090692186}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1107108501}]}]} 2024-04-24T23:58:27.803 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T23:58:27.803 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T23:58:27.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.900+0000 7f7847d5a640 1 Processor -- start 2024-04-24T23:58:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.900+0000 7f7847d5a640 1 -- start start 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f7847d5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f784015a6b0 0x7f7840070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f7847d5a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78400b2d50 con 0x7f7840072ea0 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f7847d5a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7840072a70 con 0x7f78400b3af0 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f7847d5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f78400b3010 con 0x7f784015a6b0 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f78452f5640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7840072ea0 0x7f7840071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55456/0 (socket says 172.21.15.113:55456) 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f78452f5640 1 -- 172.21.15.113:0/3746483204 learned_addr learned my addr 172.21.15.113:0/3746483204 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595674753 0 0) 0x7f78400b2d50 con 0x7f7840072ea0 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f782c003520 con 0x7f7840072ea0 2024-04-24T23:58:27.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.901+0000 7f7845af6640 1 --2- 172.21.15.113:0/3746483204 >> v2:172.21.15.178:3300/0 conn(0x7f784015a6b0 0x7f7840070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1782066819 0 0) 0x7f782c003520 con 0x7f7840072ea0 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 >> v2:172.21.15.178:3300/0 conn(0x7f784015a6b0 msgr2=0x7f7840070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 --2- 172.21.15.113:0/3746483204 >> v2:172.21.15.178:3300/0 conn(0x7f784015a6b0 0x7f7840070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 >> v1:172.21.15.178:6789/0 conn(0x7f78400b3af0 legacy=0x7f78400b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78400b32f0 con 0x7f7840072ea0 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f7830001670 con 0x7f7840072ea0 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 >> v1:172.21.15.113:6789/0 conn(0x7f7840072ea0 legacy=0x7f7840071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0039b0 0x7f782c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f782c004800 0x7f782c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f782c0051c0 0x7f782c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f78400b3010 con 0x7f782c0051c0 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f782c00c110 con 0x7f782c0039b0 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f782c00c310 con 0x7f782c004800 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f78462f7640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f782c0051c0 0x7f782c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f7845af6640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0039b0 0x7f782c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f78452f5640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f782c004800 0x7f782c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f78462f7640 1 -- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f782c004800 msgr2=0x7f782c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f78462f7640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f782c004800 0x7f782c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f78462f7640 1 -- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0039b0 msgr2=0x7f782c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f78462f7640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0039b0 0x7f782c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.902+0000 7f78462f7640 1 -- 172.21.15.113:0/3746483204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c003520 con 0x7f782c0051c0 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f78462f7640 1 -- 172.21.15.113:0/3746483204 reap_dead start 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f78462f7640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f782c0051c0 0x7f782c00b9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7838001750 tx=0x7f782c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a52f01bccfb19d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838001370 con 0x7f782c0051c0 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7838005310 con 0x7f782c0051c0 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7844af4640 1 -- 172.21.15.113:0/3746483204 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783802abe0 con 0x7f782c0051c0 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7847d5a640 1 -- 172.21.15.113:0/3746483204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f782c0051c0 msgr2=0x7f782c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7847d5a640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f782c0051c0 0x7f782c00b9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7838001750 tx=0x7f782c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7847d5a640 1 -- 172.21.15.113:0/3746483204 shutdown_connections 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7847d5a640 1 --2- 172.21.15.113:0/3746483204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f782c0051c0 0x7f782c00b9c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7847d5a640 1 -- 172.21.15.113:0/3746483204 >> 172.21.15.113:0/3746483204 conn(0x7f78400b1a50 msgr2=0x7f78400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7847d5a640 1 -- 172.21.15.113:0/3746483204 shutdown_connections 2024-04-24T23:58:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.903+0000 7f7847d5a640 1 -- 172.21.15.113:0/3746483204 wait complete. 2024-04-24T23:58:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.904+0000 7f7847d5a640 1 Processor -- start 2024-04-24T23:58:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f7847d5a640 1 -- start start 2024-04-24T23:58:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f7847d5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0051c0 0x7f78401e1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f7847d5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784014e740 0x7f78401533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f7847d5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784014ebc0 0x7f7840155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f7847d5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f784015aa90 con 0x7f784014ebc0 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f7847d5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7840072720 con 0x7f782c0051c0 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f7847d5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f784015ac10 con 0x7f784014e740 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f78462f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784014ebc0 0x7f7840155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f78462f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784014ebc0 0x7f7840155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50040/0 (socket says 172.21.15.113:50040) 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.905+0000 7f78462f7640 1 -- 172.21.15.113:0/2983792238 learned_addr learned my addr 172.21.15.113:0/2983792238 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78452f5640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784014e740 0x7f78401533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78462f7640 1 -- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784014e740 msgr2=0x7f78401533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78462f7640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784014e740 0x7f78401533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f7845af6640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0051c0 0x7f78401e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78462f7640 1 -- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0051c0 msgr2=0x7f78401e1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78462f7640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0051c0 0x7f78401e1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78462f7640 1 -- 172.21.15.113:0/2983792238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7840154210 con 0x7f784014ebc0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78462f7640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784014ebc0 0x7f7840155a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7838000c00 tx=0x7f7838001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78367fc640 1 -- 172.21.15.113:0/2983792238 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838001370 con 0x7f784014ebc0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78367fc640 1 -- 172.21.15.113:0/2983792238 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f783802ae20 con 0x7f784014ebc0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f78367fc640 1 -- 172.21.15.113:0/2983792238 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838007a30 con 0x7f784014ebc0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78401e0550 con 0x7f784014ebc0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.906+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78401544f0 con 0x7f784014ebc0 2024-04-24T23:58:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.907+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78400691a0 con 0x7f784014ebc0 2024-04-24T23:58:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.907+0000 7f78367fc640 1 -- 172.21.15.113:0/2983792238 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7838046440 con 0x7f784014ebc0 2024-04-24T23:58:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.908+0000 7f78367fc640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f782406be30 0x7f782406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.908+0000 7f7845af6640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f782406be30 0x7f782406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.908+0000 7f78367fc640 1 -- 172.21.15.113:0/2983792238 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78380aa260 con 0x7f784014ebc0 2024-04-24T23:58:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.908+0000 7f7845af6640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f782406be30 0x7f782406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f783c009fb0 tx=0x7f783c009760 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:27.912+0000 7f78367fc640 1 -- 172.21.15.113:0/2983792238 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7838078ca0 con 0x7f784014ebc0 2024-04-24T23:58:28.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.086+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78400b17c0 con 0x7f784014ebc0 2024-04-24T23:58:28.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.087+0000 7f78367fc640 1 -- 172.21.15.113:0/2983792238 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f78380785a0 con 0x7f784014ebc0 2024-04-24T23:58:28.087 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:28.087 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f782406be30 msgr2=0x7f782406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f782406be30 0x7f782406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f783c009fb0 tx=0x7f783c009760 comp rx=0 tx=0).stop 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784014ebc0 msgr2=0x7f7840155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784014ebc0 0x7f7840155a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7838000c00 tx=0x7f7838001120 comp rx=0 tx=0).stop 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 shutdown_connections 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f782406be30 0x7f782406e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784014ebc0 0x7f7840155a30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784014e740 0x7f78401533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 --2- 172.21.15.113:0/2983792238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f782c0051c0 0x7f78401e1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 >> 172.21.15.113:0/2983792238 conn(0x7f78400b1a50 msgr2=0x7f78400cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 shutdown_connections 2024-04-24T23:58:28.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.089+0000 7f7847d5a640 1 -- 172.21.15.113:0/2983792238 wait complete. 2024-04-24T23:58:28.100 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T23:58:28.100 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:28.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.196+0000 7fb1d142a640 1 Processor -- start 2024-04-24T23:58:28.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.196+0000 7fb1d142a640 1 -- start start 2024-04-24T23:58:28.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1d142a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1cc15e320 0x7fb1cc0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1d142a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb1cc0d0b00 0x7fb1cc0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1d142a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1cc0d3de0 con 0x7fb1cc0d2290 2024-04-24T23:58:28.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1d142a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc0d14b0 con 0x7fb1cc15e320 2024-04-24T23:58:28.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1d142a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc0d3410 con 0x7fb1cc0d0b00 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1caffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1cc15e320 0x7fb1cc0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1ca7fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb1cc0d2290 0x7fb1cc0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55472/0 (socket says 172.21.15.113:55472) 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1ca7fc640 1 -- 172.21.15.113:0/1952888289 learned_addr learned my addr 172.21.15.113:0/1952888289 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1cb7fe640 1 --2- 172.21.15.113:0/1952888289 >> v2:172.21.15.178:3300/0 conn(0x7fb1cc0d0b00 0x7fb1cc0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1caffd640 1 -- 172.21.15.113:0/1952888289 >> v2:172.21.15.178:3300/0 conn(0x7fb1cc0d0b00 msgr2=0x7fb1cc0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1caffd640 1 --2- 172.21.15.113:0/1952888289 >> v2:172.21.15.178:3300/0 conn(0x7fb1cc0d0b00 0x7fb1cc0d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1caffd640 1 -- 172.21.15.113:0/1952888289 >> v1:172.21.15.113:6789/0 conn(0x7fb1cc0d2290 legacy=0x7fb1cc0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.197+0000 7fb1caffd640 1 -- 172.21.15.113:0/1952888289 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc0d2800 con 0x7fb1cc15e320 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.198+0000 7fb1caffd640 1 --2- 172.21.15.113:0/1952888289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc15e320 0x7fb1cc0d38a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0d2030 tx=0x7fb1c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1007d34c23634a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.198+0000 7fb1c9ffb640 1 -- 172.21.15.113:0/1952888289 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4037070 con 0x7fb1cc15e320 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.198+0000 7fb1c9ffb640 1 -- 172.21.15.113:0/1952888289 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c4032440 con 0x7fb1cc15e320 2024-04-24T23:58:28.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.198+0000 7fb1c9ffb640 1 -- 172.21.15.113:0/1952888289 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4032810 con 0x7fb1cc15e320 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 -- 172.21.15.113:0/1952888289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc15e320 msgr2=0x7fb1cc0d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 --2- 172.21.15.113:0/1952888289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc15e320 0x7fb1cc0d38a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0d2030 tx=0x7fb1c402a230 comp rx=0 tx=0).stop 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 -- 172.21.15.113:0/1952888289 shutdown_connections 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 --2- 172.21.15.113:0/1952888289 >> v2:172.21.15.178:3300/0 conn(0x7fb1cc0d0b00 0x7fb1cc0d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 --2- 172.21.15.113:0/1952888289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc15e320 0x7fb1cc0d38a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 -- 172.21.15.113:0/1952888289 >> 172.21.15.113:0/1952888289 conn(0x7fb1cc0b5c20 msgr2=0x7fb1cc0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 -- 172.21.15.113:0/1952888289 shutdown_connections 2024-04-24T23:58:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.199+0000 7fb1d142a640 1 -- 172.21.15.113:0/1952888289 wait complete. 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.200+0000 7fb1d142a640 1 Processor -- start 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.200+0000 7fb1d142a640 1 -- start start 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1cc0d0b00 0x7fb1cc0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc0d2290 0x7fb1cc14e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1cc15e320 0x7fb1cc1f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0d30a0 con 0x7fb1cc0d2290 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0d2dc0 con 0x7fb1cc15e320 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc15e700 con 0x7fb1cc0d0b00 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc0d2290 0x7fb1cc14e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc0d2290 0x7fb1cc14e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50058/0 (socket says 172.21.15.113:50058) 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 -- 172.21.15.113:0/3240753880 learned_addr learned my addr 172.21.15.113:0/3240753880 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 -- 172.21.15.113:0/3240753880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1cc0d0b00 msgr2=0x7fb1cc0ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1cc0d0b00 0x7fb1cc0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 -- 172.21.15.113:0/3240753880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1cc15e320 msgr2=0x7fb1cc1f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1cc15e320 0x7fb1cc1f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 -- 172.21.15.113:0/3240753880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc1f4280 con 0x7fb1cc0d2290 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1ca7fc640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc0d2290 0x7fb1cc14e6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000cb40 tx=0x7fb1c000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1b37fe640 1 -- 172.21.15.113:0/3240753880 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0017070 con 0x7fb1cc0d2290 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1b37fe640 1 -- 172.21.15.113:0/3240753880 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c000aca0 con 0x7fb1cc0d2290 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1cc1f4520 con 0x7fb1cc0d2290 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1b37fe640 1 -- 172.21.15.113:0/3240753880 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0012620 con 0x7fb1cc0d2290 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.201+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1cc1f4ae0 con 0x7fb1cc0d2290 2024-04-24T23:58:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.202+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1cc0691a0 con 0x7fb1cc0d2290 2024-04-24T23:58:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.203+0000 7fb1b37fe640 1 -- 172.21.15.113:0/3240753880 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1c001a070 con 0x7fb1cc0d2290 2024-04-24T23:58:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.203+0000 7fb1b37fe640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.203+0000 7fb1caffd640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.203+0000 7fb1b37fe640 1 -- 172.21.15.113:0/3240753880 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb1c008ac40 con 0x7fb1cc0d2290 2024-04-24T23:58:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.204+0000 7fb1caffd640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb1b806be30 0x7fb1b806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4034000 tx=0x7fb1c402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.207+0000 7fb1b37fe640 1 -- 172.21.15.113:0/3240753880 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb1c0059600 con 0x7fb1cc0d2290 2024-04-24T23:58:28.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.383+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1cc15d770 con 0x7fb1cc0d2290 2024-04-24T23:58:28.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.384+0000 7fb1b37fe640 1 -- 172.21.15.113:0/3240753880 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fb1c0058f00 con 0x7fb1cc0d2290 2024-04-24T23:58:28.384 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:28.384 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:28.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb1b806be30 msgr2=0x7fb1b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb1b806be30 0x7fb1b806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4034000 tx=0x7fb1c402c000 comp rx=0 tx=0).stop 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc0d2290 msgr2=0x7fb1cc14e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc0d2290 0x7fb1cc14e6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000cb40 tx=0x7fb1c000a060 comp rx=0 tx=0).stop 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 shutdown_connections 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb1b806be30 0x7fb1b806e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1cc15e320 0x7fb1cc1f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1cc0d2290 0x7fb1cc14e6e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 --2- 172.21.15.113:0/3240753880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1cc0d0b00 0x7fb1cc0ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 >> 172.21.15.113:0/3240753880 conn(0x7fb1cc0b5c20 msgr2=0x7fb1cc0b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 shutdown_connections 2024-04-24T23:58:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.386+0000 7fb1d142a640 1 -- 172.21.15.113:0/3240753880 wait complete. 2024-04-24T23:58:28.398 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T23:58:28.398 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T23:58:28.398 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T23:58:28.399 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T23:58:28.399 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T23:58:28.399 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T23:58:28.400 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T23:58:28.400 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T23:58:28.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.566+0000 7fcf8c2ba640 1 Processor -- start 2024-04-24T23:58:28.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.565+0000 7f6121c8c640 1 Processor -- start 2024-04-24T23:58:28.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.565+0000 7f6121c8c640 1 -- start start 2024-04-24T23:58:28.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.566+0000 7ff2538c4640 1 Processor -- start 2024-04-24T23:58:28.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7ff2538c4640 1 -- start start 2024-04-24T23:58:28.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.566+0000 7f6121c8c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f611c15b3e0 0x7f611c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f297dd90640 1 Processor -- start 2024-04-24T23:58:28.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f8c6089a640 1 Processor -- start 2024-04-24T23:58:28.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f8c6089a640 1 -- start start 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.566+0000 7f6121c8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c06e2e0 0x7f611c06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f6121c8c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f611c06e830 0x7f611c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f6121c8c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f611c0c7610 con 0x7f611c15b3e0 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f6121c8c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f611c0c78f0 con 0x7f611c06e830 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f6121c8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f611c0c7bd0 con 0x7f611c06e2e0 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f611c15b3e0 0x7f611c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f611c15b3e0 0x7f611c06dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50066/0 (socket says 172.21.15.113:50066) 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611b7fe640 1 -- 172.21.15.113:0/1503476362 learned_addr learned my addr 172.21.15.113:0/1503476362 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f9dcd86d640 1 Processor -- start 2024-04-24T23:58:28.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.566+0000 7fcf8c2ba640 1 -- start start 2024-04-24T23:58:28.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f9dcd86d640 1 -- start start 2024-04-24T23:58:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7fcf8c2ba640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf84162870 0x7fcf840cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7ff2538c4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c6089a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c540e1090 0x7f8c54038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c6089a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8c54050dd0 0x7f8c54039ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c6089a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c5403a400 0x7f8c540e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c6089a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5404f540 con 0x7f8c5403a400 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c6089a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5404f820 con 0x7f8c54050dd0 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c6089a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5404fb00 con 0x7f8c540e1090 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c5bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c5403a400 0x7f8c540e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c5bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c5403a400 0x7f8c540e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50082/0 (socket says 172.21.15.113:50082) 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f8c5bfff640 1 -- 172.21.15.113:0/834101592 learned_addr learned my addr 172.21.15.113:0/834101592 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f297dd90640 1 -- start start 2024-04-24T23:58:28.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f9dcd86d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dc80cfed0 0x7f9dc8162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f9dcd86d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc81481d0 con 0x7f9dc8071e10 2024-04-24T23:58:28.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f9dcd86d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc81484b0 con 0x7f9dc80cdb80 2024-04-24T23:58:28.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7ff2538c4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff24c0720f0 0x7ff24c0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7f9dcd86d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc8148790 con 0x7f9dc80cfed0 2024-04-24T23:58:28.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7fcf8c2ba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcf84071cc0 0x7fcf841637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7ff2538c4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff24c15e660 con 0x7ff24c0c5c40 2024-04-24T23:58:28.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7fcf8a056640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf84162870 0x7fcf840cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7ff251660640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff251660640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c06ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:37542/0 (socket says 172.21.15.113:37542) 2024-04-24T23:58:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff251660640 1 -- 172.21.15.113:0/1560987283 learned_addr learned my addr 172.21.15.113:0/1560987283 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff2538c4640 1 -- 172.21.15.113:0/1560987283 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff24c06f320 con 0x7ff24c0720f0 2024-04-24T23:58:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff2538c4640 1 -- 172.21.15.113:0/1560987283 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff24c0b48f0 con 0x7ff24c15e280 2024-04-24T23:58:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5affd640 1 --2- 172.21.15.113:0/834101592 >> v2:172.21.15.178:3300/0 conn(0x7f8c54050dd0 0x7f8c54039ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.568+0000 7fcf89855640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fcf840d0510 0x7fcf840715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:55944/0 (socket says 172.21.15.113:55944) 2024-04-24T23:58:28.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf8a056640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf84162870 0x7fcf840cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41514/0 (socket says 172.21.15.113:41514) 2024-04-24T23:58:28.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc67fc640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9dc80cdb80 0x7f9dc8071700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:55964/0 (socket says 172.21.15.113:55964) 2024-04-24T23:58:28.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc67fc640 1 -- 172.21.15.113:0/2996550287 learned_addr learned my addr 172.21.15.113:0/2996550287 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf89855640 1 -- 172.21.15.113:0/3429124945 learned_addr learned my addr 172.21.15.113:0/3429124945 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf8a857640 1 --2- 172.21.15.113:0/3429124945 >> v2:172.21.15.113:3300/0 conn(0x7fcf84071cc0 0x7fcf841637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc6ffd640 1 --2- 172.21.15.113:0/2996550287 >> v2:172.21.15.113:3300/0 conn(0x7f9dc80cfed0 0x7f9dc8162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611affd640 1 --2- 172.21.15.113:0/1503476362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c06e2e0 0x7f611c06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611bfff640 1 --2- 172.21.15.113:0/1503476362 >> v2:172.21.15.178:3300/0 conn(0x7f611c06e830 0x7f611c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611affd640 1 -- 172.21.15.113:0/1503476362 >> v2:172.21.15.178:3300/0 conn(0x7f611c06e830 msgr2=0x7f611c06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611affd640 1 --2- 172.21.15.113:0/1503476362 >> v2:172.21.15.178:3300/0 conn(0x7f611c06e830 0x7f611c06ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611affd640 1 -- 172.21.15.113:0/1503476362 >> v2:172.21.15.113:3300/0 conn(0x7f611c15b3e0 msgr2=0x7f611c06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611affd640 1 --2- 172.21.15.113:0/1503476362 >> v2:172.21.15.113:3300/0 conn(0x7f611c15b3e0 0x7f611c06dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611affd640 1 -- 172.21.15.113:0/1503476362 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c0cf890 con 0x7f611c06e2e0 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.567+0000 7f611b7fe640 1 --2- 172.21.15.113:0/1503476362 >> v2:172.21.15.113:3300/0 conn(0x7f611c15b3e0 0x7f611c06dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f611affd640 1 --2- 172.21.15.113:0/1503476362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c06e2e0 0x7f611c06b0f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f611c0c6180 tx=0x7f611402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af14608a095eb6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf84072320 con 0x7fcf840d0510 2024-04-24T23:58:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff243fff640 1 -- 172.21.15.113:0/1560987283 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 530139580 0 0) 0x7ff24c15e660 con 0x7ff24c0c5c40 2024-04-24T23:58:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff243fff640 1 -- 172.21.15.113:0/1560987283 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff238003520 con 0x7ff24c0c5c40 2024-04-24T23:58:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff250e5f640 1 --2- 172.21.15.113:0/1560987283 >> v2:172.21.15.113:3300/0 conn(0x7ff24c0720f0 0x7ff24c0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252142697 0 0) 0x7f9dc81484b0 con 0x7f9dc80cdb80 2024-04-24T23:58:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9db0003520 con 0x7f9dc80cdb80 2024-04-24T23:58:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f297dd90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f2978071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f297dd90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2978163140 con 0x7f29780cfed0 2024-04-24T23:58:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f297dd90640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29781481d0 con 0x7f2978071c40 2024-04-24T23:58:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f297dd90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f29780720b0 con 0x7f2978162ce0 2024-04-24T23:58:28.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f611a7fc640 1 -- 172.21.15.113:0/1503476362 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114031600 con 0x7f611c06e2e0 2024-04-24T23:58:28.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f611a7fc640 1 -- 172.21.15.113:0/1503476362 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61140317a0 con 0x7f611c06e2e0 2024-04-24T23:58:28.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5bfff640 1 -- 172.21.15.113:0/834101592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c540e1090 msgr2=0x7f8c54038630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5b7fe640 1 --2- 172.21.15.113:0/834101592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c540e1090 0x7f8c54038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5bfff640 1 --2- 172.21.15.113:0/834101592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c540e1090 0x7f8c54038630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5bfff640 1 -- 172.21.15.113:0/834101592 >> v2:172.21.15.178:3300/0 conn(0x7f8c54050dd0 msgr2=0x7f8c54039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5bfff640 1 --2- 172.21.15.113:0/834101592 >> v2:172.21.15.178:3300/0 conn(0x7f8c54050dd0 0x7f8c54039ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5bfff640 1 -- 172.21.15.113:0/834101592 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5404fde0 con 0x7f8c5403a400 2024-04-24T23:58:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f8c5affd640 1 --2- 172.21.15.113:0/834101592 >> v2:172.21.15.178:3300/0 conn(0x7f8c54050dd0 0x7f8c54039ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 832784817 0 0) 0x7f9db0003520 con 0x7f9dc80cdb80 2024-04-24T23:58:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 >> v2:172.21.15.113:3300/0 conn(0x7f9dc80cfed0 msgr2=0x7f9dc8162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc5ffb640 1 --2- 172.21.15.113:0/2996550287 >> v2:172.21.15.113:3300/0 conn(0x7f9dc80cfed0 0x7f9dc8162ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 >> v1:172.21.15.113:6789/0 conn(0x7f9dc8071e10 legacy=0x7f9dc80b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc8148a70 con 0x7f9dc80cdb80 2024-04-24T23:58:28.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f2978071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f2978071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:37570/0 (socket says 172.21.15.113:37570) 2024-04-24T23:58:28.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f2976ffd640 1 -- 172.21.15.113:0/1007526820 learned_addr learned my addr 172.21.15.113:0/1007526820 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f9dbc001670 con 0x7f9dc80cdb80 2024-04-24T23:58:28.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f63d804c640 1 Processor -- start 2024-04-24T23:58:28.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 >> v1:172.21.15.178:6789/0 conn(0x7f9dc80cdb80 legacy=0x7f9dc8071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00039b0 0x7f9db0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9db000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8148790 con 0x7f9db00039b0 2024-04-24T23:58:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9db000c110 con 0x7f9db0004800 2024-04-24T23:58:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9db000c310 con 0x7f9db00051c0 2024-04-24T23:58:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f63d804c640 1 -- start start 2024-04-24T23:58:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc67fc640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f2976ffd640 1 -- 172.21.15.113:0/1007526820 >> v1:172.21.15.178:6789/0 conn(0x7f2978071c40 legacy=0x7f29780b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc77fe640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9db000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f2976ffd640 1 -- 172.21.15.113:0/1007526820 >> v1:172.21.15.113:6789/0 conn(0x7f29780cfed0 legacy=0x7f29780cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff251660640 1 -- 172.21.15.113:0/1560987283 >> v2:172.21.15.113:3300/0 conn(0x7ff24c0720f0 msgr2=0x7ff24c0c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff251660640 1 --2- 172.21.15.113:0/1560987283 >> v2:172.21.15.113:3300/0 conn(0x7ff24c0720f0 0x7ff24c0c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff251660640 1 -- 172.21.15.113:0/1560987283 >> v1:172.21.15.178:6789/0 conn(0x7ff24c0c5c40 legacy=0x7ff24c0b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7ff251660640 1 -- 172.21.15.113:0/1560987283 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c0b4bd0 con 0x7ff24c15e280 2024-04-24T23:58:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7ff250e5f640 1 --2- 172.21.15.113:0/1560987283 >> v2:172.21.15.113:3300/0 conn(0x7ff24c0720f0 0x7ff24c0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f2976ffd640 1 -- 172.21.15.113:0/1007526820 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29781484b0 con 0x7f2978162ce0 2024-04-24T23:58:28.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f9dc77fe640 1 -- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db0004800 msgr2=0x7f9db0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f6121c8c640 1 -- 172.21.15.113:0/1503476362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c06e2e0 msgr2=0x7f611c06b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f611a7fc640 1 -- 172.21.15.113:0/1503476362 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114031b70 con 0x7f611c06e2e0 2024-04-24T23:58:28.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7f6121c8c640 1 --2- 172.21.15.113:0/1503476362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c06e2e0 0x7f611c06b0f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f611c0c6180 tx=0x7f611402a5d0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f9dc77fe640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f9dc77fe640 1 -- 172.21.15.113:0/2996550287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00039b0 msgr2=0x7f9db0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f9dc77fe640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00039b0 0x7f9db0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f9dc77fe640 1 -- 172.21.15.113:0/2996550287 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db0003520 con 0x7f9db00051c0 2024-04-24T23:58:28.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcf84072600 con 0x7fcf84071cc0 2024-04-24T23:58:28.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcf84162fe0 con 0x7fcf84162870 2024-04-24T23:58:28.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf8a857640 1 -- 172.21.15.113:0/3429124945 >> v2:172.21.15.178:3300/0 conn(0x7fcf84162870 msgr2=0x7fcf840cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.569+0000 7fcf8a857640 1 --2- 172.21.15.113:0/3429124945 >> v2:172.21.15.178:3300/0 conn(0x7fcf84162870 0x7fcf840cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7fcf8a857640 1 -- 172.21.15.113:0/3429124945 >> v1:172.21.15.178:6789/0 conn(0x7fcf840d0510 legacy=0x7fcf840715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7fcf8a857640 1 -- 172.21.15.113:0/3429124945 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf841632c0 con 0x7fcf84071cc0 2024-04-24T23:58:28.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7fcf8a857640 1 --2- 172.21.15.113:0/3429124945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf841637e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf8406e150 tx=0x7fcf7c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645ad0607ae34c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f0c85eae640 1 Processor -- start 2024-04-24T23:58:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f0c85eae640 1 -- start start 2024-04-24T23:58:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7fcf89054640 1 -- 172.21.15.113:0/3429124945 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c007740 con 0x7fcf84071cc0 2024-04-24T23:58:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.570+0000 7ff251660640 1 --2- 172.21.15.113:0/1560987283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c06ec60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff24c15e830 tx=0x7ff24402a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13c67a06c7d130f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7ff243fff640 1 -- 172.21.15.113:0/1560987283 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244036070 con 0x7ff24c15e280 2024-04-24T23:58:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f8c5bfff640 1 --2- 172.21.15.113:0/834101592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 0x7f8c540e4a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c540524d0 tx=0x7f8c4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89911a7304a9b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7fcf89054640 1 -- 172.21.15.113:0/3429124945 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c0078e0 con 0x7fcf84071cc0 2024-04-24T23:58:28.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7fcf89054640 1 -- 172.21.15.113:0/3429124945 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c007cf0 con 0x7fcf84071cc0 2024-04-24T23:58:28.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7ff243fff640 1 -- 172.21.15.113:0/1560987283 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff24402ae50 con 0x7ff24c15e280 2024-04-24T23:58:28.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7ff243fff640 1 -- 172.21.15.113:0/1560987283 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244031800 con 0x7ff24c15e280 2024-04-24T23:58:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f63d804c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63d015bff0 0x7f63d00d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f63d804c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f63d015b670 0x7f63d00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f63d804c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63d0071170 con 0x7f63d015bb20 2024-04-24T23:58:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f63d804c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f63d0071450 con 0x7f63d015bff0 2024-04-24T23:58:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f63d804c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f63d0071730 con 0x7f63d015b670 2024-04-24T23:58:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f8c5a7fc640 1 -- 172.21.15.113:0/834101592 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48037070 con 0x7f8c5403a400 2024-04-24T23:58:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f9dc77fe640 1 -- 172.21.15.113:0/2996550287 reap_dead start 2024-04-24T23:58:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f8c5a7fc640 1 -- 172.21.15.113:0/834101592 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c48032440 con 0x7f8c5403a400 2024-04-24T23:58:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f8c5a7fc640 1 -- 172.21.15.113:0/834101592 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48032810 con 0x7f8c5403a400 2024-04-24T23:58:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f6121c8c640 1 -- 172.21.15.113:0/1503476362 shutdown_connections 2024-04-24T23:58:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f6121c8c640 1 --2- 172.21.15.113:0/1503476362 >> v2:172.21.15.178:3300/0 conn(0x7f611c06e830 0x7f611c06ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f6121c8c640 1 --2- 172.21.15.113:0/1503476362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c06e2e0 0x7f611c06b0f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f6121c8c640 1 --2- 172.21.15.113:0/1503476362 >> v2:172.21.15.113:3300/0 conn(0x7f611c15b3e0 0x7f611c06dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f6121c8c640 1 -- 172.21.15.113:0/1503476362 >> 172.21.15.113:0/1503476362 conn(0x7f611c0cd9b0 msgr2=0x7f611c0cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f63d5de8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63d015bff0 0x7f63d00d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.571+0000 7f63d55e7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f63d015b670 0x7f63d00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f63d5de8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63d015bff0 0x7f63d00d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50108/0 (socket says 172.21.15.113:50108) 2024-04-24T23:58:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f63d55e7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f63d015b670 0x7f63d00cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41540/0 (socket says 172.21.15.113:41540) 2024-04-24T23:58:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f63d55e7640 1 -- 172.21.15.113:0/3532234399 learned_addr learned my addr 172.21.15.113:0/3532234399 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f9dc77fe640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9db000b9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9db80044b0 tx=0x7f9db0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df7e2e762c66a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8004080 con 0x7f9db00051c0 2024-04-24T23:58:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db802ad80 con 0x7f9db00051c0 2024-04-24T23:58:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f9dc5ffb640 1 -- 172.21.15.113:0/2996550287 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8004080 con 0x7f9db00051c0 2024-04-24T23:58:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f63d55e7640 1 -- 172.21.15.113:0/3532234399 >> v2:172.21.15.113:3300/0 conn(0x7f63d015bff0 msgr2=0x7f63d00d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 msgr2=0x7fcf841637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7ff2538c4640 1 -- 172.21.15.113:0/1560987283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 msgr2=0x7ff24c06ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f6121c8c640 1 -- 172.21.15.113:0/1503476362 shutdown_connections 2024-04-24T23:58:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7fcf8c2ba640 1 --2- 172.21.15.113:0/3429124945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf841637e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf8406e150 tx=0x7fcf7c02a610 comp rx=0 tx=0).stop 2024-04-24T23:58:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7ff2538c4640 1 --2- 172.21.15.113:0/1560987283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c06ec60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff24c15e830 tx=0x7ff24402a630 comp rx=0 tx=0).stop 2024-04-24T23:58:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f0c85eae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015e510 0x7f0c800d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7ff2538c4640 1 -- 172.21.15.113:0/1560987283 shutdown_connections 2024-04-24T23:58:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f2976ffd640 1 --2- 172.21.15.113:0/1007526820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f2978071700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f296c0098a0 tx=0x7f296c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87357c6b745b373 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7ff2538c4640 1 --2- 172.21.15.113:0/1560987283 >> v2:172.21.15.113:3300/0 conn(0x7ff24c0720f0 0x7ff24c0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7ff2538c4640 1 --2- 172.21.15.113:0/1560987283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c06ec60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7ff2538c4640 1 -- 172.21.15.113:0/1560987283 >> 172.21.15.113:0/1560987283 conn(0x7ff24c061f70 msgr2=0x7ff24c0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f8c6089a640 1 -- 172.21.15.113:0/834101592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 msgr2=0x7f8c540e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f8c6089a640 1 --2- 172.21.15.113:0/834101592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 0x7f8c540e4a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c540524d0 tx=0x7f8c4802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f0c7ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0c800b63b0 0x7f0c800b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55514/0 (socket says 172.21.15.113:55514) 2024-04-24T23:58:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f0c7effd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0c800d28c0 0x7f0c800b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:55998/0 (socket says 172.21.15.113:55998) 2024-04-24T23:58:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f8c6089a640 1 -- 172.21.15.113:0/834101592 shutdown_connections 2024-04-24T23:58:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f8c6089a640 1 --2- 172.21.15.113:0/834101592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 0x7f8c540e4a30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f8c6089a640 1 --2- 172.21.15.113:0/834101592 >> v2:172.21.15.178:3300/0 conn(0x7f8c54050dd0 0x7f8c54039ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f8c6089a640 1 --2- 172.21.15.113:0/834101592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c540e1090 0x7f8c54038630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f8c6089a640 1 -- 172.21.15.113:0/834101592 >> 172.21.15.113:0/834101592 conn(0x7f8c54009fb0 msgr2=0x7f8c5400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f63d55e7640 1 --2- 172.21.15.113:0/3532234399 >> v2:172.21.15.113:3300/0 conn(0x7f63d015bff0 0x7f63d00d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f63d55e7640 1 -- 172.21.15.113:0/3532234399 >> v1:172.21.15.178:6789/0 conn(0x7f63d015bb20 legacy=0x7f63d006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f0c7ffff640 1 -- 172.21.15.113:0/3909762166 learned_addr learned my addr 172.21.15.113:0/3909762166 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f63d4de6640 1 -- 172.21.15.113:0/3532234399 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2921075627 0 0) 0x7f63d0071170 con 0x7f63d015bb20 2024-04-24T23:58:28.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 shutdown_connections 2024-04-24T23:58:28.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7fcf8c2ba640 1 --2- 172.21.15.113:0/3429124945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf841637e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7fcf8c2ba640 1 --2- 172.21.15.113:0/3429124945 >> v2:172.21.15.178:3300/0 conn(0x7fcf84162870 0x7fcf840cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 >> 172.21.15.113:0/3429124945 conn(0x7fcf8406b0f0 msgr2=0x7fcf8406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f0c7f7fe640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015e510 0x7f0c800d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f63d55e7640 1 -- 172.21.15.113:0/3532234399 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d0071a10 con 0x7f63d015b670 2024-04-24T23:58:28.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f63d5de8640 1 --2- 172.21.15.113:0/3532234399 >> v2:172.21.15.113:3300/0 conn(0x7f63d015bff0 0x7f63d00d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:28.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c800d2e00 con 0x7f0c800b63b0 2024-04-24T23:58:28.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7ff2538c4640 1 -- 172.21.15.113:0/1560987283 shutdown_connections 2024-04-24T23:58:28.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 shutdown_connections 2024-04-24T23:58:28.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7ff2538c4640 1 -- 172.21.15.113:0/1560987283 wait complete. 2024-04-24T23:58:28.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c800d30e0 con 0x7f0c800d28c0 2024-04-24T23:58:28.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f29767fc640 1 -- 172.21.15.113:0/1007526820 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c037070 con 0x7f2978162ce0 2024-04-24T23:58:28.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f8c6089a640 1 -- 172.21.15.113:0/834101592 shutdown_connections 2024-04-24T23:58:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c800d33c0 con 0x7f0c8015e510 2024-04-24T23:58:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f9dcd86d640 1 -- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 msgr2=0x7f9db000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f9dcd86d640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9db000b9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9db80044b0 tx=0x7f9db0005830 comp rx=0 tx=0).stop 2024-04-24T23:58:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f29767fc640 1 -- 172.21.15.113:0/1007526820 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296c032440 con 0x7f2978162ce0 2024-04-24T23:58:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f29767fc640 1 -- 172.21.15.113:0/1007526820 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c032810 con 0x7f2978162ce0 2024-04-24T23:58:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f9dcd86d640 1 -- 172.21.15.113:0/2996550287 shutdown_connections 2024-04-24T23:58:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f9dcd86d640 1 --2- 172.21.15.113:0/2996550287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9db000b9c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f9dcd86d640 1 -- 172.21.15.113:0/2996550287 >> 172.21.15.113:0/2996550287 conn(0x7f9dc806b0f0 msgr2=0x7f9dc806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.572+0000 7f6121c8c640 1 -- 172.21.15.113:0/1503476362 wait complete. 2024-04-24T23:58:28.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f8c6089a640 1 -- 172.21.15.113:0/834101592 wait complete. 2024-04-24T23:58:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374834036 0 0) 0x7f0c800d2e00 con 0x7f0c800b63b0 2024-04-24T23:58:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c68003520 con 0x7f0c800b63b0 2024-04-24T23:58:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503774234 0 0) 0x7f0c800d30e0 con 0x7f0c800d28c0 2024-04-24T23:58:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c800d2e00 con 0x7f0c800d28c0 2024-04-24T23:58:28.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f9dcd86d640 1 -- 172.21.15.113:0/2996550287 shutdown_connections 2024-04-24T23:58:28.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 423514380 0 0) 0x7f0c68003520 con 0x7f0c800b63b0 2024-04-24T23:58:28.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015e510 msgr2=0x7f0c800d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015e510 0x7f0c800d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 >> v1:172.21.15.178:6789/0 conn(0x7f0c800d28c0 legacy=0x7f0c800b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c800d36a0 con 0x7f0c800b63b0 2024-04-24T23:58:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0c780019f0 con 0x7f0c800b63b0 2024-04-24T23:58:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 >> v1:172.21.15.113:6789/0 conn(0x7f0c800b63b0 legacy=0x7f0c800b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015e510 0x7f0c68003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c680040d0 0x7f0c68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c68004d40 0x7f0c680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800d33c0 con 0x7f0c680040d0 2024-04-24T23:58:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6800bb50 con 0x7f0c68004d40 2024-04-24T23:58:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6800bd20 con 0x7f0c8015e510 2024-04-24T23:58:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f297dd90640 1 -- 172.21.15.113:0/1007526820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 msgr2=0x7f2978071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f297dd90640 1 --2- 172.21.15.113:0/1007526820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f2978071700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f296c0098a0 tx=0x7f296c02a250 comp rx=0 tx=0).stop 2024-04-24T23:58:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f297dd90640 1 -- 172.21.15.113:0/1007526820 shutdown_connections 2024-04-24T23:58:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f297dd90640 1 --2- 172.21.15.113:0/1007526820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f2978071700 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f297dd90640 1 -- 172.21.15.113:0/1007526820 >> 172.21.15.113:0/1007526820 conn(0x7f297806b0f0 msgr2=0x7f297806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/3429124945 wait complete. 2024-04-24T23:58:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f0c7ffff640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c68004d40 0x7f0c680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f0c7effd640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c680040d0 0x7f0c68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f297dd90640 1 -- 172.21.15.113:0/1007526820 shutdown_connections 2024-04-24T23:58:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f297dd90640 1 -- 172.21.15.113:0/1007526820 wait complete. 2024-04-24T23:58:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7ff2538c4640 1 Processor -- start 2024-04-24T23:58:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7ff2538c4640 1 -- start start 2024-04-24T23:58:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f63d55e7640 1 --2- 172.21.15.113:0/3532234399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015b670 0x7f63d00cc200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cceb0 tx=0x7f63c002a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d393b015bc056d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f8c6089a640 1 Processor -- start 2024-04-24T23:58:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f8c6089a640 1 -- start start 2024-04-24T23:58:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f63d4de6640 1 -- 172.21.15.113:0/3532234399 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0037070 con 0x7f63d015b670 2024-04-24T23:58:28.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.573+0000 7f9dcd86d640 1 -- 172.21.15.113:0/2996550287 wait complete. 2024-04-24T23:58:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f63d4de6640 1 -- 172.21.15.113:0/3532234399 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c0032440 con 0x7f63d015b670 2024-04-24T23:58:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f63d4de6640 1 -- 172.21.15.113:0/3532234399 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0032810 con 0x7f63d015b670 2024-04-24T23:58:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f297dd90640 1 Processor -- start 2024-04-24T23:58:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f9dcd86d640 1 Processor -- start 2024-04-24T23:58:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f6121c8c640 1 Processor -- start 2024-04-24T23:58:28.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f0c7effd640 1 -- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015e510 msgr2=0x7f0c68003b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:58:28.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f0c7effd640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015e510 0x7f0c68003b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f0c7effd640 1 -- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c68004d40 msgr2=0x7f0c680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7fcf8c2ba640 1 Processor -- start 2024-04-24T23:58:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f0c7effd640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c68004d40 0x7f0c680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.574+0000 7f0c7effd640 1 -- 172.21.15.113:0/3909762166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c800d36a0 con 0x7f0c680040d0 2024-04-24T23:58:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f0c7effd640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c680040d0 0x7f0c68004800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0c700018e0 tx=0x7f0c68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cfe1e22cb0abad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.575+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70001500 con 0x7f0c680040d0 2024-04-24T23:58:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7002e030 con 0x7f0c680040d0 2024-04-24T23:58:28.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f0c7e7fc640 1 -- 172.21.15.113:0/3909762166 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70001500 con 0x7f0c680040d0 2024-04-24T23:58:28.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f6121c8c640 1 -- start start 2024-04-24T23:58:28.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f297dd90640 1 -- start start 2024-04-24T23:58:28.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7ff2538c4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c0720f0 0x7ff24c14eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f9dcd86d640 1 -- start start 2024-04-24T23:58:28.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c6089a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 0x7f8c540d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c6089a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c54050dd0 0x7f8c5404a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7ff2538c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff24c0c5c40 0x7ff24c141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c6089a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c540e1090 0x7f8c5404ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c6089a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54054610 con 0x7f8c5403a400 2024-04-24T23:58:28.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c6089a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c540503a0 con 0x7f8c540e1090 2024-04-24T23:58:28.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c6089a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5403a870 con 0x7f8c54050dd0 2024-04-24T23:58:28.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7ff251660640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c0720f0 0x7ff24c14eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7ff251660640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c0720f0 0x7ff24c14eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41562/0 (socket says 172.21.15.113:41562) 2024-04-24T23:58:28.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7fcf8c2ba640 1 -- start start 2024-04-24T23:58:28.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7ff251660640 1 -- 172.21.15.113:0/719600307 learned_addr learned my addr 172.21.15.113:0/719600307 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c5bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c540e1090 0x7f8c5404ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c5bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c540e1090 0x7f8c5404ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41572/0 (socket says 172.21.15.113:41572) 2024-04-24T23:58:28.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1262882818 learned_addr learned my addr 172.21.15.113:0/1262882818 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7f8c5affd640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c54050dd0 0x7f8c5404a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.576+0000 7ff2538c4640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c1f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c5b7fe640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 0x7f8c540d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c54050dd0 msgr2=0x7f8c5404a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c5bfff640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c54050dd0 0x7f8c5404a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 msgr2=0x7f8c540d77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c5bfff640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5403a400 0x7f8c540d77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c0b5ae0 con 0x7ff24c0c5c40 2024-04-24T23:58:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c680040d0 msgr2=0x7f0c68004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f0c85eae640 1 --2- 172.21.15.113:0/3909762166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c680040d0 0x7f0c68004800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0c700018e0 tx=0x7f0c68005830 comp rx=0 tx=0).stop 2024-04-24T23:58:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f0c7ffff640 1 -- 172.21.15.113:0/3909762166 reap_dead start 2024-04-24T23:58:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7ff251e61640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c1f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1262882818 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5404d6c0 con 0x7f8c540e1090 2024-04-24T23:58:28.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f297dd90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2978071c40 0x7f29781407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dcd86d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9dc8200500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dcd86d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc8200a40 0x7f9dc80b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dcd86d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc80b6110 0x7f9dc81f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dcd86d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc80d02b0 con 0x7f9dc8200a40 2024-04-24T23:58:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dcd86d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8147ea0 con 0x7f9dc80b6110 2024-04-24T23:58:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dcd86d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc80d0430 con 0x7f9db00051c0 2024-04-24T23:58:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 shutdown_connections 2024-04-24T23:58:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 >> 172.21.15.113:0/3909762166 conn(0x7f0c8006b720 msgr2=0x7f0c800d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7ff250e5f640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff24c0c5c40 0x7ff24c141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f297dd90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29780cfed0 0x7f29781655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c072550 con 0x7ff24c0720f0 2024-04-24T23:58:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f297dd90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f297813c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c5bfff640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c540e1090 0x7f8c5404ae40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c480090b0 tx=0x7f8c480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dc67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc8200a40 0x7f9dc80b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f297dd90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2978148d50 con 0x7f29780cfed0 2024-04-24T23:58:28.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f297dd90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2978148a70 con 0x7f2978071c40 2024-04-24T23:58:28.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f6121c8c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f611c06e2e0 0x7f611c0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dc67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc8200a40 0x7f9dc80b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50152/0 (socket says 172.21.15.113:50152) 2024-04-24T23:58:28.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dc67fc640 1 -- 172.21.15.113:0/3959874905 learned_addr learned my addr 172.21.15.113:0/3959874905 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dc6ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9dc8200500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f297dd90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2978147ea0 con 0x7f2978162ce0 2024-04-24T23:58:28.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff24c0b5c60 con 0x7ff24c15e280 2024-04-24T23:58:28.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f6121c8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f611c06e830 0x7f611c145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f29777fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2978071c40 0x7f29781407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f2977fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f297813c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48041070 con 0x7f8c540e1090 2024-04-24T23:58:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c4803c870 con 0x7f8c540e1090 2024-04-24T23:58:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c54048690 con 0x7f8c540e1090 2024-04-24T23:58:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f63d804c640 1 -- 172.21.15.113:0/3532234399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015b670 msgr2=0x7f63d00cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f63d804c640 1 --2- 172.21.15.113:0/3532234399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015b670 0x7f63d00cc200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cceb0 tx=0x7f63c002a150 comp rx=0 tx=0).stop 2024-04-24T23:58:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48037040 con 0x7f8c540e1090 2024-04-24T23:58:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29780cfed0 0x7f29781655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f63d804c640 1 -- 172.21.15.113:0/3532234399 shutdown_connections 2024-04-24T23:58:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f63d804c640 1 --2- 172.21.15.113:0/3532234399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015b670 0x7f63d00cc200 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cceb0 tx=0x7f63c002a150 comp rx=0 tx=0).stop 2024-04-24T23:58:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f63d804c640 1 --2- 172.21.15.113:0/3532234399 >> v2:172.21.15.113:3300/0 conn(0x7f63d015bff0 0x7f63d00d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f63d804c640 1 -- 172.21.15.113:0/3532234399 >> 172.21.15.113:0/3532234399 conn(0x7f63d006b0f0 msgr2=0x7f63d006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c54048c50 con 0x7f8c540e1090 2024-04-24T23:58:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29780cfed0 0x7f29781655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50168/0 (socket says 172.21.15.113:50168) 2024-04-24T23:58:28.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 shutdown_connections 2024-04-24T23:58:28.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f0c85eae640 1 -- 172.21.15.113:0/3909762166 wait complete. 2024-04-24T23:58:28.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f2976ffd640 1 -- 172.21.15.113:0/1434971424 learned_addr learned my addr 172.21.15.113:0/1434971424 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f9dc77fe640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc80b6110 0x7f9dc81f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7ff251660640 1 -- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 msgr2=0x7ff24c1f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7ff251660640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff24c15e280 0x7ff24c1f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7ff251660640 1 -- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff24c0c5c40 msgr2=0x7ff24c141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7ff251660640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff24c0c5c40 0x7ff24c141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7ff251660640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c14c200 con 0x7ff24c0720f0 2024-04-24T23:58:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7ff250e5f640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff24c0c5c40 0x7ff24c141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f63d804c640 1 -- 172.21.15.113:0/3532234399 shutdown_connections 2024-04-24T23:58:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7fcf8c2ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf84158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7fcf8c2ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840d0510 0x7fcf841f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7fcf8c2ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf84162870 0x7fcf841ed100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7fcf8c2ba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84163ea0 con 0x7fcf84071cc0 2024-04-24T23:58:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7fcf8a857640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf84162870 0x7fcf841ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f29777fe640 1 -- 172.21.15.113:0/1434971424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 msgr2=0x7f297813c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f29777fe640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2978162ce0 0x7f297813c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f29777fe640 1 -- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29780cfed0 msgr2=0x7f29781655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f29777fe640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29780cfed0 0x7f29781655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f29777fe640 1 -- 172.21.15.113:0/1434971424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29781fbfe0 con 0x7f2978071c40 2024-04-24T23:58:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f9dc6ffd640 1 -- 172.21.15.113:0/3959874905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc80b6110 msgr2=0x7f9dc81f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f9dc6ffd640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc80b6110 0x7f9dc81f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f9dc6ffd640 1 -- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc8200a40 msgr2=0x7f9dc80b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f9dc6ffd640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc8200a40 0x7f9dc80b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f9dc6ffd640 1 -- 172.21.15.113:0/3959874905 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc81f2ec0 con 0x7f9db00051c0 2024-04-24T23:58:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f9dc6ffd640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9dc8200500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000d5b0 tx=0x7f9dc000d9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f29777fe640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2978071c40 0x7f29781407f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f297000c6d0 tx=0x7f297000cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7ff251660640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c0720f0 0x7ff24c14eca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff244009850 tx=0x7ff244004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f611b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f611c06e2e0 0x7f611c0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.577+0000 7f611b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f611c06e2e0 0x7f611c0cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41604/0 (socket says 172.21.15.113:41604) 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f6121c8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c15b3e0 0x7f611c1459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f6121c8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f611c0d0090 con 0x7f611c06e830 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f6121c8c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f611c0cf580 con 0x7f611c06e2e0 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f6121c8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f611c0d0210 con 0x7f611c15b3e0 2024-04-24T23:58:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f611affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f611c06e830 0x7f611c145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f611affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f611c06e830 0x7f611c145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50176/0 (socket says 172.21.15.113:50176) 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f611affd640 1 -- 172.21.15.113:0/736913101 learned_addr learned my addr 172.21.15.113:0/736913101 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f611affd640 1 -- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c15b3e0 msgr2=0x7f611c1459c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f611affd640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f611c15b3e0 0x7f611c1459c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f611affd640 1 -- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f611c06e2e0 msgr2=0x7f611c0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f611affd640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f611c06e2e0 0x7f611c0cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f611affd640 1 -- 172.21.15.113:0/736913101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c0c8cc0 con 0x7f611c06e830 2024-04-24T23:58:28.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c54005390 con 0x7f8c540e1090 2024-04-24T23:58:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0012630 con 0x7f9db00051c0 2024-04-24T23:58:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc0019070 con 0x7f9db00051c0 2024-04-24T23:58:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f611b7fe640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f611c06e2e0 0x7f611c0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-24T23:58:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244036070 con 0x7ff24c0720f0 2024-04-24T23:58:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7fcf8a056640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf84158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7fcf8a056640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf84158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50192/0 (socket says 172.21.15.113:50192) 2024-04-24T23:58:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7fcf8a056640 1 -- 172.21.15.113:0/1292982839 learned_addr learned my addr 172.21.15.113:0/1292982839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7fcf89855640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840d0510 0x7fcf841f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84163d20 con 0x7fcf840d0510 2024-04-24T23:58:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcf84164020 con 0x7fcf84162870 2024-04-24T23:58:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00129f0 con 0x7f9db00051c0 2024-04-24T23:58:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc81f1e20 con 0x7f9db00051c0 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f0c85eae640 1 Processor -- start 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f0c85eae640 1 -- start start 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970017070 con 0x7f2978071c40 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2970003b70 con 0x7f2978071c40 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970017070 con 0x7f2978071c40 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29781fc2c0 con 0x7f2978071c40 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29781fc880 con 0x7f2978071c40 2024-04-24T23:58:28.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff244041070 con 0x7ff24c0720f0 2024-04-24T23:58:28.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24403c860 con 0x7ff24c0720f0 2024-04-24T23:58:28.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c160000 con 0x7ff24c0720f0 2024-04-24T23:58:28.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1605c0 con 0x7ff24c0720f0 2024-04-24T23:58:28.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.579+0000 7f611affd640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f611c06e830 0x7f611c145480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6114009e80 tx=0x7f6114004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.578+0000 7f63d804c640 1 -- 172.21.15.113:0/3532234399 wait complete. 2024-04-24T23:58:28.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f63d804c640 1 Processor -- start 2024-04-24T23:58:28.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f63d804c640 1 -- start start 2024-04-24T23:58:28.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7fcf8a857640 1 -- 172.21.15.113:0/1292982839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840d0510 msgr2=0x7fcf841f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7fcf8a857640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840d0510 0x7fcf841f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7fcf8a857640 1 -- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 msgr2=0x7fcf84158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7fcf8a857640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf84158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7fcf8a857640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf841f9760 con 0x7fcf84162870 2024-04-24T23:58:28.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7fcf8a056640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf84071cc0 0x7fcf84158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:28.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc81f23e0 con 0x7f9db00051c0 2024-04-24T23:58:28.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c48044070 con 0x7f8c540e1090 2024-04-24T23:58:28.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f8c47fff640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c4006bf30 0x7f8c4006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c480ab9d0 con 0x7f8c540e1090 2024-04-24T23:58:28.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f8c47fff640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7f8c400740d0 0x7f8c400764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.580+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 --> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c48008f00 con 0x7f8c400740d0 2024-04-24T23:58:28.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f8c5b7fe640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c4006bf30 0x7f8c4006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8c48076160 con 0x7f8c540e1090 2024-04-24T23:58:28.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114040070 con 0x7f611c06e830 2024-04-24T23:58:28.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611403c8f0 con 0x7f611c06e830 2024-04-24T23:58:28.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff24c068220 con 0x7ff24c0720f0 2024-04-24T23:58:28.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c85eae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c680040d0 0x7f0c801f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c85eae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015fbb0 0x7f0c80160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c85eae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80153620 0x7f0c80153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c85eae640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800b9dd0 con 0x7f0c80153620 2024-04-24T23:58:28.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c85eae640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800d3ea0 con 0x7f0c680040d0 2024-04-24T23:58:28.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c85eae640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c800d2380 con 0x7f0c8015fbb0 2024-04-24T23:58:28.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7fcf8a857640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf84162870 0x7fcf841ed100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c031000 tx=0x7fcf7c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c7f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c680040d0 0x7f0c801f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c7f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c680040d0 0x7f0c801f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41630/0 (socket says 172.21.15.113:41630) 2024-04-24T23:58:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c7f7fe640 1 -- 172.21.15.113:0/266201421 learned_addr learned my addr 172.21.15.113:0/266201421 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f0c7ffff640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80153620 0x7f0c80153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9dc8068220 con 0x7f9db00051c0 2024-04-24T23:58:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2978068220 con 0x7f2978071c40 2024-04-24T23:58:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff24403ca00 con 0x7ff24c0720f0 2024-04-24T23:58:28.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611c0c8fa0 con 0x7f611c06e830 2024-04-24T23:58:28.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.581+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611c0c9560 con 0x7f611c06e830 2024-04-24T23:58:28.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c7f7fe640 1 -- 172.21.15.113:0/266201421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015fbb0 msgr2=0x7f0c80160010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:58:28.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c7f7fe640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8015fbb0 0x7f0c80160010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c7f7fe640 1 -- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80153620 msgr2=0x7f0c80153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c7f7fe640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80153620 0x7f0c80153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c7f7fe640 1 -- 172.21.15.113:0/266201421 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c800c7f10 con 0x7f0c680040d0 2024-04-24T23:58:28.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c7f7fe640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c680040d0 0x7f0c801f39a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e820 tx=0x7f0c7400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74017070 con 0x7f0c680040d0 2024-04-24T23:58:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7400c8c0 con 0x7f0c680040d0 2024-04-24T23:58:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9dc0012b90 con 0x7f9db00051c0 2024-04-24T23:58:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400cc90 con 0x7f0c680040d0 2024-04-24T23:58:28.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c800c81f0 con 0x7f0c680040d0 2024-04-24T23:58:28.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c800c87b0 con 0x7f0c680040d0 2024-04-24T23:58:28.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f8c5affd640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7f8c400740d0 0x7f8c400764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f8c5b7fe640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c4006bf30 0x7f8c4006e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006fd0 tx=0x7f8c50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7ff241ffb640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff23006bf30 0x7ff23006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff2440abc90 con 0x7ff24c0720f0 2024-04-24T23:58:28.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611403cce0 con 0x7f611c06e830 2024-04-24T23:58:28.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d804c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63d015bb20 0x7f63d00c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d804c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015bff0 0x7f63d0157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d804c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d0157bb0 0x7f63d00ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f9daffff640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9db406bf30 0x7f9db406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7ff250e5f640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff23006bf30 0x7ff23006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d804c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f63d00d06f0 con 0x7f63d0157bb0 2024-04-24T23:58:28.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d804c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f63d00d0570 con 0x7f63d015bff0 2024-04-24T23:58:28.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7ff241ffb640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7ff230074150 0x7ff230076570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff230076c20 con 0x7ff230074150 2024-04-24T23:58:28.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d804c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f63d006d5b0 con 0x7f63d015bb20 2024-04-24T23:58:28.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9dc008b0d0 con 0x7f9db00051c0 2024-04-24T23:58:28.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff2440ac080 con 0x7ff24c0720f0 2024-04-24T23:58:28.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c040070 con 0x7fcf84162870 2024-04-24T23:58:28.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c031b40 con 0x7fcf84162870 2024-04-24T23:58:28.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf841585f0 con 0x7fcf84162870 2024-04-24T23:58:28.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf84158bb0 con 0x7fcf84162870 2024-04-24T23:58:28.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f9daffff640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f9db40741b0 0x7f9db40765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 --> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9db4076c80 con 0x7f9db40741b0 2024-04-24T23:58:28.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9dc0092050 con 0x7f9db00051c0 2024-04-24T23:58:28.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f9dc77fe640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f9db40741b0 0x7f9db40765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.582+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2970006b10 con 0x7f2978071c40 2024-04-24T23:58:28.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f295ffff640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f296406c230 0x7f296406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7ff251e61640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7ff230074150 0x7ff230076570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7ff250e5f640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff23006bf30 0x7ff23006e3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff24c15f720 tx=0x7ff248008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d5de8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63d015bb20 0x7f63d00c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d5de8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63d015bb20 0x7f63d00c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:37654/0 (socket says 172.21.15.113:37654) 2024-04-24T23:58:28.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.583+0000 7f63d65e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d0157bb0 0x7f63d00ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d5de8640 1 -- 172.21.15.113:0/2367285621 learned_addr learned my addr 172.21.15.113:0/2367285621 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d55e7640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015bff0 0x7f63d0157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c045460 con 0x7fcf84162870 2024-04-24T23:58:28.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f9dc67fc640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9db406bf30 0x7f9db406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6114039040 con 0x7f611c06e830 2024-04-24T23:58:28.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60fc001030 con 0x7f611c06e830 2024-04-24T23:58:28.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d5de8640 1 -- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015bff0 msgr2=0x7f63d0157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d5de8640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63d015bff0 0x7f63d0157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d5de8640 1 -- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d0157bb0 msgr2=0x7f63d00ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d5de8640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d0157bb0 0x7f63d00ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d5de8640 1 -- 172.21.15.113:0/2367285621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d00cbec0 con 0x7f63d015bb20 2024-04-24T23:58:28.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f63d65e9640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d0157bb0 0x7f63d00ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:28.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f9dc77fe640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f9db40741b0 0x7f9db40765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c7401a070 con 0x7f0c680040d0 2024-04-24T23:58:28.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf6c001030 con 0x7fcf84162870 2024-04-24T23:58:28.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f0c67fff640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c6c06be60 0x7f0c6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2970003970 con 0x7f2978071c40 2024-04-24T23:58:28.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f9dc67fc640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9db406bf30 0x7f9db406e3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9dc815e030 tx=0x7f9dbc0078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c60001030 con 0x7f0c680040d0 2024-04-24T23:58:28.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f295ffff640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f2964074430 0x7f2964076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 --> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2970004e50 con 0x7f2964074430 2024-04-24T23:58:28.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f8c5affd640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7f8c400740d0 0x7f8c400764f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f297008a5e0 con 0x7f2978071c40 2024-04-24T23:58:28.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f2977fff640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f2964074430 0x7f2964076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.584+0000 7f2976ffd640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f296406c230 0x7f296406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f2977fff640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f2964074430 0x7f2964076850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f6103fff640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f610806bf00 0x7f610806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f0c7effd640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c6c06be60 0x7f0c6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.585+0000 7f611b7fe640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f610806bf00 0x7f610806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f63d5de8640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63d015bb20 0x7f63d00c8ea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e990 tx=0x7f63cc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== osd.1 v2:172.21.15.113:6808/3114085930 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9db4076c80 con 0x7f9db40741b0 2024-04-24T23:58:28.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f2976ffd640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f296406c230 0x7f296406e6f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7ff251e61640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7ff230074150 0x7ff230076570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== osd.3 v2:172.21.15.113:6824/1975200730 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2970004e50 con 0x7f2964074430 2024-04-24T23:58:28.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf7c039040 con 0x7fcf84162870 2024-04-24T23:58:28.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc017070 con 0x7f63d015bb20 2024-04-24T23:58:28.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc00c9e0 con 0x7f63d015bb20 2024-04-24T23:58:28.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00cdb0 con 0x7f63d015bb20 2024-04-24T23:58:28.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f63d804c640 1 -- 172.21.15.113:0/2367285621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d00cb620 con 0x7f63d015bb20 2024-04-24T23:58:28.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f63d804c640 1 -- 172.21.15.113:0/2367285621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d00cbbe0 con 0x7f63d015bb20 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7fcf7affd640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf6806bf00 0x7fcf6806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf7c0ac490 con 0x7fcf84162870 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7fcf7affd640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7fcf68074120 0x7fcf68076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf68076bf0 con 0x7fcf68074120 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcf7c0b3050 con 0x7fcf84162870 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f611b7fe640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f610806bf00 0x7f610806e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f610c009a20 tx=0x7f610c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f61140ac0d0 con 0x7f611c06e830 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f6103fff640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f61080740a0 0x7f61080764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 --> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f611403c8f0 con 0x7f61080740a0 2024-04-24T23:58:28.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6114076600 con 0x7f611c06e830 2024-04-24T23:58:28.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.586+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== osd.7 v2:172.21.15.178:6824/3779386153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c48008f00 con 0x7f8c400740d0 2024-04-24T23:58:28.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f611bfff640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f61080740a0 0x7f61080764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7fcf89855640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7fcf68074120 0x7fcf68076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7fcf8a056640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf6806bf00 0x7fcf6806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f63d804c640 1 -- 172.21.15.113:0/2367285621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63d00691a0 con 0x7f63d015bb20 2024-04-24T23:58:28.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7fcf8a056640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf6806bf00 0x7fcf6806e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf74007e50 tx=0x7fcf74008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7fcf89855640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7fcf68074120 0x7fcf68076540 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== osd.4 v2:172.21.15.178:6808/146317928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff230076c20 con 0x7ff230074150 2024-04-24T23:58:28.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c7400ce30 con 0x7f0c680040d0 2024-04-24T23:58:28.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f0c7effd640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c6c06be60 0x7f0c6c06e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0c70032000 tx=0x7f0c70006000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f0c67fff640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7f0c6c074ce0 0x7f0c6c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 --> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c6c077850 con 0x7f0c6c074ce0 2024-04-24T23:58:28.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.587+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0c74055c40 con 0x7f0c680040d0 2024-04-24T23:58:28.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.588+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== osd.0 v2:172.21.15.113:6800/1851994484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf68076bf0 con 0x7fcf68074120 2024-04-24T23:58:28.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.588+0000 7f611bfff640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f61080740a0 0x7f61080764c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.588+0000 7f0c7ffff640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7f0c6c074ce0 0x7f0c6c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.589+0000 7f0c7ffff640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7f0c6c074ce0 0x7f0c6c077180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.589+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== osd.5 v2:172.21.15.178:6816/2661350487 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f611403c8f0 con 0x7f61080740a0 2024-04-24T23:58:28.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.590+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== osd.2 v2:172.21.15.113:6816/3097656860 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c6c077850 con 0x7f0c6c074ce0 2024-04-24T23:58:28.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.590+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63cc01a070 con 0x7f63d015bb20 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.591+0000 7f63c67fc640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63b406bf30 0x7f63b406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.591+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f63cc08ad10 con 0x7f63d015bb20 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.591+0000 7f63d55e7640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63b406bf30 0x7f63b406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.592+0000 7f63d55e7640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63b406bf30 0x7f63b406e3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f63c0034000 tx=0x7f63c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.592+0000 7f63c67fc640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f63b4074150 0x7f63b4076570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.592+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 --> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63cc017e50 con 0x7f63b4074150 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.592+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63cc08b100 con 0x7f63d015bb20 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.593+0000 7f63d65e9640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f63b4074150 0x7f63b4076570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.593+0000 7f63d65e9640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f63b4074150 0x7f63b4076570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.594+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== osd.6 v2:172.21.15.178:6800/1646944139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63cc017e50 con 0x7f63b4074150 2024-04-24T23:58:28.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.616+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 --> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c54048e20 con 0x7f8c400740d0 2024-04-24T23:58:28.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.616+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 --> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f29780ce5a0 con 0x7f2964074430 2024-04-24T23:58:28.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f295ffff640 1 -- 172.21.15.113:0/1434971424 <== osd.3 v2:172.21.15.113:6824/1975200730 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f29780ce5a0 con 0x7f2964074430 2024-04-24T23:58:28.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f8c47fff640 1 -- 172.21.15.113:0/1262882818 <== osd.7 v2:172.21.15.178:6824/3779386153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8c54048e20 con 0x7f8c400740d0 2024-04-24T23:58:28.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f2964074430 msgr2=0x7f2964076850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f297dd90640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f2964074430 0x7f2964076850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f296406c230 msgr2=0x7f296406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f297dd90640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f296406c230 0x7f296406e6f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c02e000 comp rx=0 tx=0).stop 2024-04-24T23:58:28.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2978071c40 msgr2=0x7f29781407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f297dd90640 1 --2- 172.21.15.113:0/1434971424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2978071c40 0x7f29781407f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f297000c6d0 tx=0x7f297000cb90 comp rx=0 tx=0).stop 2024-04-24T23:58:28.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.617+0000 7f2977fff640 1 -- 172.21.15.113:0/1434971424 reap_dead start 2024-04-24T23:58:28.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7f8c400740d0 msgr2=0x7f8c400764f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7f8c400740d0 0x7f8c400764f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 shutdown_connections 2024-04-24T23:58:28.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 >> 172.21.15.113:0/1434971424 conn(0x7f297806b0f0 msgr2=0x7f297806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c4006bf30 msgr2=0x7f8c4006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c4006bf30 0x7f8c4006e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006fd0 tx=0x7f8c50008040 comp rx=0 tx=0).stop 2024-04-24T23:58:28.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c540e1090 msgr2=0x7f8c5404ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 --2- 172.21.15.113:0/1262882818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c540e1090 0x7f8c5404ae40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c480090b0 tx=0x7f8c480046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 shutdown_connections 2024-04-24T23:58:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f297dd90640 1 -- 172.21.15.113:0/1434971424 wait complete. 2024-04-24T23:58:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1262882818 reap_dead start 2024-04-24T23:58:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 shutdown_connections 2024-04-24T23:58:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 >> 172.21.15.113:0/1262882818 conn(0x7f8c54009fb0 msgr2=0x7f8c540e4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.618+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 shutdown_connections 2024-04-24T23:58:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.619+0000 7f8c6089a640 1 -- 172.21.15.113:0/1262882818 wait complete. 2024-04-24T23:58:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.619+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 --> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9dc80b32f0 con 0x7f9db40741b0 2024-04-24T23:58:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.620+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 --> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff24c154910 con 0x7ff230074150 2024-04-24T23:58:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.620+0000 7ff241ffb640 1 -- 172.21.15.113:0/719600307 <== osd.4 v2:172.21.15.178:6808/146317928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff24c154910 con 0x7ff230074150 2024-04-24T23:58:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.620+0000 7f9daffff640 1 -- 172.21.15.113:0/3959874905 <== osd.1 v2:172.21.15.113:6808/3114085930 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9dc80b32f0 con 0x7f9db40741b0 2024-04-24T23:58:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7ff230074150 msgr2=0x7ff230076570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7ff230074150 0x7ff230076570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 --> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf6c002dd0 con 0x7fcf68074120 2024-04-24T23:58:28.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff23006bf30 msgr2=0x7ff23006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff23006bf30 0x7ff23006e3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff24c15f720 tx=0x7ff248008040 comp rx=0 tx=0).stop 2024-04-24T23:58:28.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c0720f0 msgr2=0x7ff24c14eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 --2- 172.21.15.113:0/719600307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c0720f0 0x7ff24c14eca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff244009850 tx=0x7ff244004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:28.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff251e61640 1 -- 172.21.15.113:0/719600307 reap_dead start 2024-04-24T23:58:28.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7fcf7affd640 1 -- 172.21.15.113:0/1292982839 <== osd.0 v2:172.21.15.113:6800/1851994484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcf6c002dd0 con 0x7fcf68074120 2024-04-24T23:58:28.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 shutdown_connections 2024-04-24T23:58:28.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 >> 172.21.15.113:0/719600307 conn(0x7ff24c061f70 msgr2=0x7ff24c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7fcf68074120 msgr2=0x7fcf68076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7fcf68074120 0x7fcf68076540 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf6806bf00 msgr2=0x7fcf6806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf6806bf00 0x7fcf6806e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf74007e50 tx=0x7fcf74008040 comp rx=0 tx=0).stop 2024-04-24T23:58:28.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f9db40741b0 msgr2=0x7f9db40765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7f9dcd86d640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f9db40741b0 0x7f9db40765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9db406bf30 msgr2=0x7f9db406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f9dcd86d640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9db406bf30 0x7f9db406e3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9dc815e030 tx=0x7f9dbc0078f0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 msgr2=0x7f9dc8200500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f9dcd86d640 1 --2- 172.21.15.113:0/3959874905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db00051c0 0x7f9dc8200500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000d5b0 tx=0x7f9dc000d9b0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f9dc77fe640 1 -- 172.21.15.113:0/3959874905 reap_dead start 2024-04-24T23:58:28.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 --> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c60002dd0 con 0x7f0c6c074ce0 2024-04-24T23:58:28.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.621+0000 7f0c67fff640 1 -- 172.21.15.113:0/266201421 <== osd.2 v2:172.21.15.113:6816/3097656860 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0c60002dd0 con 0x7f0c6c074ce0 2024-04-24T23:58:28.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7f0c6c074ce0 msgr2=0x7f0c6c077180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f0c85eae640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7f0c6c074ce0 0x7f0c6c077180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf84162870 msgr2=0x7fcf841ed100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 --2- 172.21.15.113:0/1292982839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf84162870 0x7fcf841ed100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c031000 tx=0x7fcf7c004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c6c06be60 msgr2=0x7f0c6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f0c85eae640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c6c06be60 0x7f0c6c06e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0c70032000 tx=0x7f0c70006000 comp rx=0 tx=0).stop 2024-04-24T23:58:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8a857640 1 -- 172.21.15.113:0/1292982839 reap_dead start 2024-04-24T23:58:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c680040d0 msgr2=0x7f0c801f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 shutdown_connections 2024-04-24T23:58:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 >> 172.21.15.113:0/1292982839 conn(0x7fcf8406b0f0 msgr2=0x7fcf840b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f0c85eae640 1 --2- 172.21.15.113:0/266201421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c680040d0 0x7f0c801f39a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e820 tx=0x7f0c7400ece0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 shutdown_connections 2024-04-24T23:58:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7ff2538c4640 1 -- 172.21.15.113:0/719600307 wait complete. 2024-04-24T23:58:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.622+0000 7f0c7ffff640 1 -- 172.21.15.113:0/266201421 reap_dead start 2024-04-24T23:58:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 shutdown_connections 2024-04-24T23:58:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 shutdown_connections 2024-04-24T23:58:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 >> 172.21.15.113:0/3959874905 conn(0x7f9dc806b0f0 msgr2=0x7f9dc81473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7fcf8c2ba640 1 -- 172.21.15.113:0/1292982839 wait complete. 2024-04-24T23:58:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 shutdown_connections 2024-04-24T23:58:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 >> 172.21.15.113:0/266201421 conn(0x7f0c8006b720 msgr2=0x7f0c80071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 --> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60fc002dd0 con 0x7f61080740a0 2024-04-24T23:58:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 shutdown_connections 2024-04-24T23:58:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 shutdown_connections 2024-04-24T23:58:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f9dcd86d640 1 -- 172.21.15.113:0/3959874905 wait complete. 2024-04-24T23:58:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f0c85eae640 1 -- 172.21.15.113:0/266201421 wait complete. 2024-04-24T23:58:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.623+0000 7f6103fff640 1 -- 172.21.15.113:0/736913101 <== osd.5 v2:172.21.15.178:6816/2661350487 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f60fc002dd0 con 0x7f61080740a0 2024-04-24T23:58:28.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f61080740a0 msgr2=0x7f61080764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f6121c8c640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f61080740a0 0x7f61080764c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f610806bf00 msgr2=0x7f610806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f6121c8c640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f610806bf00 0x7f610806e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f610c009a20 tx=0x7f610c008040 comp rx=0 tx=0).stop 2024-04-24T23:58:28.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f63d804c640 1 -- 172.21.15.113:0/2367285621 --> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63d0158040 con 0x7f63b4074150 2024-04-24T23:58:28.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f611c06e830 msgr2=0x7f611c145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f6121c8c640 1 --2- 172.21.15.113:0/736913101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f611c06e830 0x7f611c145480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6114009e80 tx=0x7f6114004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:28.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.624+0000 7f611bfff640 1 -- 172.21.15.113:0/736913101 reap_dead start 2024-04-24T23:58:28.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.625+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 shutdown_connections 2024-04-24T23:58:28.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.625+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 >> 172.21.15.113:0/736913101 conn(0x7f611c0cd9b0 msgr2=0x7f611c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.625+0000 7f63c67fc640 1 -- 172.21.15.113:0/2367285621 <== osd.6 v2:172.21.15.178:6800/1646944139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f63d0158040 con 0x7f63b4074150 2024-04-24T23:58:28.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.625+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 shutdown_connections 2024-04-24T23:58:28.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.625+0000 7f6121c8c640 1 -- 172.21.15.113:0/736913101 wait complete. 2024-04-24T23:58:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.626+0000 7f63b3fff640 1 -- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f63b4074150 msgr2=0x7f63b4076570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.626+0000 7f63b3fff640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f63b4074150 0x7f63b4076570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.626+0000 7f63b3fff640 1 -- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63b406bf30 msgr2=0x7f63b406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.626+0000 7f63b3fff640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63b406bf30 0x7f63b406e3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f63c0034000 tx=0x7f63c002e000 comp rx=0 tx=0).stop 2024-04-24T23:58:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.626+0000 7f63b3fff640 1 -- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63d015bb20 msgr2=0x7f63d00c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.626+0000 7f63b3fff640 1 --2- 172.21.15.113:0/2367285621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63d015bb20 0x7f63d00c8ea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e990 tx=0x7f63cc00ee50 comp rx=0 tx=0).stop 2024-04-24T23:58:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.627+0000 7f63d65e9640 1 -- 172.21.15.113:0/2367285621 reap_dead start 2024-04-24T23:58:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.627+0000 7f63b3fff640 1 -- 172.21.15.113:0/2367285621 shutdown_connections 2024-04-24T23:58:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.627+0000 7f63b3fff640 1 -- 172.21.15.113:0/2367285621 >> 172.21.15.113:0/2367285621 conn(0x7f63d006b0f0 msgr2=0x7f63d015d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.629+0000 7f63b3fff640 1 -- 172.21.15.113:0/2367285621 shutdown_connections 2024-04-24T23:58:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.629+0000 7f63b3fff640 1 -- 172.21.15.113:0/2367285621 wait complete. 2024-04-24T23:58:28.724 INFO:teuthology.orchestra.run.smithi113.stdout:55834574856 2024-04-24T23:58:28.725 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T23:58:28.725 INFO:teuthology.orchestra.run.smithi113.stdout:55834574855 2024-04-24T23:58:28.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T23:58:28.727 INFO:teuthology.orchestra.run.smithi113.stdout:55834574854 2024-04-24T23:58:28.727 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T23:58:28.728 INFO:teuthology.orchestra.run.smithi113.stdout:55834574855 2024-04-24T23:58:28.728 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T23:58:28.729 INFO:teuthology.orchestra.run.smithi113.stdout:55834574857 2024-04-24T23:58:28.729 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T23:58:28.730 INFO:teuthology.orchestra.run.smithi113.stdout:55834574855 2024-04-24T23:58:28.730 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T23:58:28.731 INFO:teuthology.orchestra.run.smithi113.stdout:55834574856 2024-04-24T23:58:28.731 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T23:58:28.732 INFO:teuthology.orchestra.run.smithi113.stdout:55834574856 2024-04-24T23:58:28.732 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T23:58:28.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.929+0000 7f385359e640 1 Processor -- start 2024-04-24T23:58:28.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.929+0000 7f385359e640 1 -- start start 2024-04-24T23:58:28.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.930+0000 7f5312f7e640 1 Processor -- start 2024-04-24T23:58:28.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.930+0000 7f5312f7e640 1 -- start start 2024-04-24T23:58:28.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.930+0000 7f350395a640 1 Processor -- start 2024-04-24T23:58:28.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7fcd5af5f640 1 Processor -- start 2024-04-24T23:58:28.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f385359e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 0x7f385415b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f385359e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f38540d0610 0x7f38540ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f385359e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38540ce830 con 0x7f38540ca6b0 2024-04-24T23:58:28.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f385359e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f38540ce310 con 0x7f38540d0610 2024-04-24T23:58:28.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f385359e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f38540cf160 con 0x7f385415a900 2024-04-24T23:58:28.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f3852d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f38540ca6b0 0x7f38540cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55528/0 (socket says 172.21.15.113:55528) 2024-04-24T23:58:28.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f3852d9d640 1 -- 172.21.15.113:0/3612111579 learned_addr learned my addr 172.21.15.113:0/3612111579 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f3851d9b640 1 --2- 172.21.15.113:0/3612111579 >> v2:172.21.15.178:3300/0 conn(0x7f38540d0610 0x7f38540ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.930+0000 7f350395a640 1 -- start start 2024-04-24T23:58:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7fcd5af5f640 1 -- start start 2024-04-24T23:58:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5312f7e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f53040e3360 0x7f53040061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5312f7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530403c0d0 0x7f530403aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5310d1a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f53040e3360 0x7f53040061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5310d1a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f53040e3360 0x7f53040061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41650/0 (socket says 172.21.15.113:41650) 2024-04-24T23:58:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5310d1a640 1 -- 172.21.15.113:0/4124411770 learned_addr learned my addr 172.21.15.113:0/4124411770 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f530bfff640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530403c0d0 0x7f530403aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5312f7e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5304058ca0 con 0x7f530403e9c0 2024-04-24T23:58:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5312f7e640 1 -- 172.21.15.113:0/4124411770 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5304057800 con 0x7f53040e3360 2024-04-24T23:58:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f5312f7e640 1 -- 172.21.15.113:0/4124411770 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5304006ae0 con 0x7f530403c0d0 2024-04-24T23:58:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726933120 0 0) 0x7f5304058ca0 con 0x7f530403e9c0 2024-04-24T23:58:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52fc003520 con 0x7f530403e9c0 2024-04-24T23:58:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.931+0000 7f350395a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f34fc162870 0x7f34fc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f385159a640 1 -- 172.21.15.113:0/3612111579 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794228319 0 0) 0x7f38540ce830 con 0x7f38540ca6b0 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f3851d9b640 1 -- 172.21.15.113:0/3612111579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 msgr2=0x7f385415b410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f3851d9b640 1 --2- 172.21.15.113:0/3612111579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 0x7f385415b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f3851d9b640 1 -- 172.21.15.113:0/3612111579 >> v1:172.21.15.113:6789/0 conn(0x7f38540ca6b0 legacy=0x7f38540cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f3851d9b640 1 -- 172.21.15.113:0/3612111579 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38540ceb10 con 0x7f38540d0610 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1575111887 0 0) 0x7f52fc003520 con 0x7f530403e9c0 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530403c0d0 msgr2=0x7f530403aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530403c0d0 0x7f530403aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 >> v2:172.21.15.178:3300/0 conn(0x7f53040e3360 msgr2=0x7f53040061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 --2- 172.21.15.113:0/4124411770 >> v2:172.21.15.178:3300/0 conn(0x7f53040e3360 0x7f53040061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5304006dc0 con 0x7f530403e9c0 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f530c062da0 con 0x7f530403e9c0 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 >> v1:172.21.15.178:6789/0 conn(0x7f530403e9c0 legacy=0x7f53040e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0039b0 0x7f52fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc004350 0x7f52fc004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc004df0 0x7f52fc005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5304057800 con 0x7f52fc004df0 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5304006ae0 con 0x7f52fc004350 2024-04-24T23:58:28.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52fc00bd90 con 0x7f52fc0039b0 2024-04-24T23:58:28.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f40d4130640 1 Processor -- start 2024-04-24T23:58:28.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f3500ef5640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f34fc0d0510 0x7f34fc0715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55542/0 (socket says 172.21.15.113:55542) 2024-04-24T23:58:28.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f531151b640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc004df0 0x7f52fc005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f5310d1a640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0039b0 0x7f52fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f35016f6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f34fc162870 0x7f34fc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f35016f6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f34fc162870 0x7f34fc0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41666/0 (socket says 172.21.15.113:41666) 2024-04-24T23:58:28.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f3500ef5640 1 -- 172.21.15.113:0/3192685597 learned_addr learned my addr 172.21.15.113:0/3192685597 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f3851d9b640 1 --2- 172.21.15.113:0/3612111579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f38540ca170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38480097a0 tx=0x7f384802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356e960c08c03d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f350395a640 1 --2- 172.21.15.113:0/3192685597 >> v2:172.21.15.113:3300/0 conn(0x7f34fc071cc0 0x7f34fc1637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7efc7c4a5640 1 Processor -- start 2024-04-24T23:58:28.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5af5f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 0x7fcd540d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5af5f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd5415e610 0x7fcd540cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5af5f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd540d28b0 con 0x7fcd540cf8a0 2024-04-24T23:58:28.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f237d09e640 1 Processor -- start 2024-04-24T23:58:28.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34fc072320 con 0x7f34fc0d0510 2024-04-24T23:58:28.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f237d09e640 1 -- start start 2024-04-24T23:58:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5af5f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcd540d2b90 con 0x7fcd5415e610 2024-04-24T23:58:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5af5f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540d2e70 con 0x7fcd540d07c0 2024-04-24T23:58:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5975c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd5415e610 0x7fcd540cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5975c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd5415e610 0x7fcd540cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41692/0 (socket says 172.21.15.113:41692) 2024-04-24T23:58:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd5975c640 1 -- 172.21.15.113:0/686479006 learned_addr learned my addr 172.21.15.113:0/686479006 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7fcd59f5d640 1 --2- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 0x7fcd540d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f530bfff640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc004350 0x7f52fc004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f5310d1a640 1 -- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc004350 msgr2=0x7f52fc004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f5310d1a640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc004350 0x7f52fc004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f5310d1a640 1 -- 172.21.15.113:0/4124411770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc004df0 msgr2=0x7f52fc005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f5310d1a640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc004df0 0x7f52fc005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7f5310d1a640 1 -- 172.21.15.113:0/4124411770 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5304006dc0 con 0x7f52fc0039b0 2024-04-24T23:58:28.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f3501ef7640 1 --2- 172.21.15.113:0/3192685597 >> v2:172.21.15.113:3300/0 conn(0x7f34fc071cc0 0x7f34fc1637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.932+0000 7f40d4130640 1 -- start start 2024-04-24T23:58:28.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7fcd5975c640 1 -- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 msgr2=0x7fcd540d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d4130640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40cc15a940 0x7f40cc06e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7fcd5975c640 1 --2- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 0x7fcd540d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7fcd5975c640 1 -- 172.21.15.113:0/686479006 >> v1:172.21.15.178:6789/0 conn(0x7fcd540cf8a0 legacy=0x7fcd540b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7fcd5975c640 1 -- 172.21.15.113:0/686479006 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd540d3150 con 0x7fcd5415e610 2024-04-24T23:58:28.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7fa107fff640 1 Processor -- start 2024-04-24T23:58:28.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f34f3fff640 1 -- 172.21.15.113:0/3192685597 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180887312 0 0) 0x7f34fc072320 con 0x7f34fc0d0510 2024-04-24T23:58:28.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d1ecc640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f40cc15a560 0x7f40cc06e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:56018/0 (socket says 172.21.15.113:56018) 2024-04-24T23:58:28.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f34fc072600 con 0x7f34fc071cc0 2024-04-24T23:58:28.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f34fc162fe0 con 0x7f34fc162870 2024-04-24T23:58:28.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f34f3fff640 1 -- 172.21.15.113:0/3192685597 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34e8003520 con 0x7f34fc0d0510 2024-04-24T23:58:28.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7fa107fff640 1 -- start start 2024-04-24T23:58:28.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f3501ef7640 1 -- 172.21.15.113:0/3192685597 >> v2:172.21.15.178:3300/0 conn(0x7f34fc162870 msgr2=0x7f34fc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d1ecc640 1 -- 172.21.15.113:0/4285839663 learned_addr learned my addr 172.21.15.113:0/4285839663 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d16cb640 1 --2- 172.21.15.113:0/4285839663 >> v2:172.21.15.113:3300/0 conn(0x7f40cc15a940 0x7f40cc06e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f34f3fff640 1 -- 172.21.15.113:0/3192685597 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3780632205 0 0) 0x7f34e8003520 con 0x7f34fc0d0510 2024-04-24T23:58:28.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f5310d1a640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0039b0 0x7f52fc003dd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5300009c10 tx=0x7f52fc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625f41f32e4f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d4130640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc143a20 0x7f40cc0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40cc06f020 con 0x7f40cc15a560 2024-04-24T23:58:28.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f40cc06e690 con 0x7f40cc15a940 2024-04-24T23:58:28.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc06f300 con 0x7f40cc143a20 2024-04-24T23:58:28.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300036070 con 0x7f52fc0039b0 2024-04-24T23:58:28.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fcd5975c640 1 --2- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5415e610 0x7fcd540cf360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009870 tx=0x7fcd4402a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f12b3ea0d7a6968 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.933+0000 7efc7c4a5640 1 -- start start 2024-04-24T23:58:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7c4a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efc74162930 0x7efc740cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7c4a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc740af8a0 0x7efc740c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7c4a5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc740733d0 con 0x7efc7406dac0 2024-04-24T23:58:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7c4a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efc740cd0f0 con 0x7efc740af8a0 2024-04-24T23:58:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7c4a5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efc740ccd80 con 0x7efc74162930 2024-04-24T23:58:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc79a40640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc740af8a0 0x7efc740c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fcd58f5b640 1 -- 172.21.15.113:0/686479006 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44037070 con 0x7fcd5415e610 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237d09e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2378162870 0x7f23780cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237d09e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2378071cc0 0x7f23781637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237d09e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2378072320 con 0x7f23780d0510 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237d09e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2378072600 con 0x7f2378071cc0 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237d09e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2378162fe0 con 0x7f2378162870 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23780d0510 0x7f23780715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55554/0 (socket says 172.21.15.113:55554) 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237659c640 1 -- 172.21.15.113:0/3891866174 learned_addr learned my addr 172.21.15.113:0/3891866174 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f385159a640 1 -- 172.21.15.113:0/3612111579 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848037070 con 0x7f38540d0610 2024-04-24T23:58:28.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f385159a640 1 -- 172.21.15.113:0/3612111579 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3848032440 con 0x7f38540d0610 2024-04-24T23:58:28.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f385159a640 1 -- 172.21.15.113:0/3612111579 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848032810 con 0x7f38540d0610 2024-04-24T23:58:28.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f2376d9d640 1 --2- 172.21.15.113:0/3891866174 >> v2:172.21.15.178:3300/0 conn(0x7f2378162870 0x7f23780cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fcd58f5b640 1 -- 172.21.15.113:0/686479006 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd44032440 con 0x7fcd5415e610 2024-04-24T23:58:28.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f237759e640 1 --2- 172.21.15.113:0/3891866174 >> v2:172.21.15.113:3300/0 conn(0x7f2378071cc0 0x7f23781637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7aa42640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7efc7406dac0 0x7efc740ad0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:56020/0 (socket says 172.21.15.113:56020) 2024-04-24T23:58:28.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7aa42640 1 -- 172.21.15.113:0/4150583324 learned_addr learned my addr 172.21.15.113:0/4150583324 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fa107fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1000ead00 0x7fa10004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fa107fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa10004ffd0 0x7fa100037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fcd58f5b640 1 -- 172.21.15.113:0/686479006 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44032810 con 0x7fcd5415e610 2024-04-24T23:58:28.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fa107fff640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa100053720 con 0x7fa100038110 2024-04-24T23:58:28.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fa107fff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa100054330 con 0x7fa1000ead00 2024-04-24T23:58:28.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fa107fff640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa100053a00 con 0x7fa10004ffd0 2024-04-24T23:58:28.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7efc7a241640 1 --2- 172.21.15.113:0/4150583324 >> v2:172.21.15.178:3300/0 conn(0x7efc74162930 0x7efc740cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5300031400 con 0x7f52fc0039b0 2024-04-24T23:58:28.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f530b7fe640 1 -- 172.21.15.113:0/4124411770 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000097e0 con 0x7f52fc0039b0 2024-04-24T23:58:28.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f3501ef7640 1 --2- 172.21.15.113:0/3192685597 >> v2:172.21.15.178:3300/0 conn(0x7f34fc162870 0x7f34fc0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7efc7923f640 1 -- 172.21.15.113:0/4150583324 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325576459 0 0) 0x7efc740733d0 con 0x7efc7406dac0 2024-04-24T23:58:28.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7efc7923f640 1 -- 172.21.15.113:0/4150583324 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc60003520 con 0x7efc7406dac0 2024-04-24T23:58:28.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7efc7a241640 1 -- 172.21.15.113:0/4150583324 >> v2:172.21.15.113:3300/0 conn(0x7efc740af8a0 msgr2=0x7efc740c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7efc7a241640 1 --2- 172.21.15.113:0/4150583324 >> v2:172.21.15.113:3300/0 conn(0x7efc740af8a0 0x7efc740c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7efc7a241640 1 -- 172.21.15.113:0/4150583324 >> v1:172.21.15.178:6789/0 conn(0x7efc7406dac0 legacy=0x7efc740ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 -- 172.21.15.113:0/3612111579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 msgr2=0x7f38540ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 --2- 172.21.15.113:0/3612111579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f38540ca170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38480097a0 tx=0x7f384802a250 comp rx=0 tx=0).stop 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f40d0eca640 1 -- 172.21.15.113:0/4285839663 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951160216 0 0) 0x7f40cc06f020 con 0x7f40cc15a560 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f40d26cd640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc143a20 0x7f40cc0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f40d16cb640 1 -- 172.21.15.113:0/4285839663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc143a20 msgr2=0x7f40cc0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f40d16cb640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc143a20 0x7f40cc0cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f40d16cb640 1 -- 172.21.15.113:0/4285839663 >> v1:172.21.15.178:6789/0 conn(0x7f40cc15a560 legacy=0x7f40cc06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f40d16cb640 1 -- 172.21.15.113:0/4285839663 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc06d950 con 0x7f40cc15a940 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f40d26cd640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc143a20 0x7f40cc0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:28.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f237759e640 1 -- 172.21.15.113:0/3891866174 >> v2:172.21.15.178:3300/0 conn(0x7f2378162870 msgr2=0x7f23780cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f237759e640 1 --2- 172.21.15.113:0/3891866174 >> v2:172.21.15.178:3300/0 conn(0x7f2378162870 0x7f23780cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f237759e640 1 -- 172.21.15.113:0/3891866174 >> v1:172.21.15.113:6789/0 conn(0x7f23780d0510 legacy=0x7f23780715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f237759e640 1 -- 172.21.15.113:0/3891866174 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23781632c0 con 0x7f2378071cc0 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f3501ef7640 1 -- 172.21.15.113:0/3192685597 >> v1:172.21.15.113:6789/0 conn(0x7f34fc0d0510 legacy=0x7f34fc0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.934+0000 7f3501ef7640 1 -- 172.21.15.113:0/3192685597 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc1632c0 con 0x7f34fc071cc0 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7f35016f6640 1 --2- 172.21.15.113:0/3192685597 >> v2:172.21.15.178:3300/0 conn(0x7f34fc162870 0x7f34fc0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f3501ef7640 1 --2- 172.21.15.113:0/3192685597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc071cc0 0x7f34fc1637e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06e150 tx=0x7f34f402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d589a72d2bfc48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f34f3fff640 1 -- 172.21.15.113:0/3192685597 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8001670 con 0x7f34fc071cc0 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f34f3fff640 1 -- 172.21.15.113:0/3192685597 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f4007540 con 0x7f34fc071cc0 2024-04-24T23:58:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f34f3fff640 1 -- 172.21.15.113:0/3192685597 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4007910 con 0x7f34fc071cc0 2024-04-24T23:58:28.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 -- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5415e610 msgr2=0x7fcd540cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5415e610 0x7fcd540cf360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009870 tx=0x7fcd4402a1d0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 -- 172.21.15.113:0/686479006 shutdown_connections 2024-04-24T23:58:28.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5415e610 0x7fcd540cf360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/686479006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 0x7fcd540d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 -- 172.21.15.113:0/686479006 >> 172.21.15.113:0/686479006 conn(0x7fcd540d3fd0 msgr2=0x7fcd540d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 -- 172.21.15.113:0/686479006 shutdown_connections 2024-04-24T23:58:28.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7efc7a241640 1 -- 172.21.15.113:0/4150583324 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc740cdac0 con 0x7efc74162930 2024-04-24T23:58:28.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7efc79a40640 1 --2- 172.21.15.113:0/4150583324 >> v2:172.21.15.113:3300/0 conn(0x7efc740af8a0 0x7efc740c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:28.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 -- 172.21.15.113:0/3612111579 shutdown_connections 2024-04-24T23:58:28.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 --2- 172.21.15.113:0/3612111579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f38540ca170 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 --2- 172.21.15.113:0/3612111579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 0x7f385415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 -- 172.21.15.113:0/3612111579 >> 172.21.15.113:0/3612111579 conn(0x7f38540cfb20 msgr2=0x7f38540cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fcd5af5f640 1 -- 172.21.15.113:0/686479006 wait complete. 2024-04-24T23:58:28.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.935+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa10004ffd0 0x7fa100037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa1077fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa100038110 0x7fa1000385a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:56036/0 (socket says 172.21.15.113:56036) 2024-04-24T23:58:28.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa1077fe640 1 -- 172.21.15.113:0/2661292013 learned_addr learned my addr 172.21.15.113:0/2661292013 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:28.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa106ffd640 1 --2- 172.21.15.113:0/2661292013 >> v2:172.21.15.113:3300/0 conn(0x7fa1000ead00 0x7fa10004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511143547 0 0) 0x7fa100053720 con 0x7fa100038110 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0f0003520 con 0x7fa100038110 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1822627700 0 0) 0x7fa0f0003520 con 0x7fa100038110 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa10004ffd0 msgr2=0x7fa100037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa10004ffd0 0x7fa100037bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 >> v2:172.21.15.113:3300/0 conn(0x7fa1000ead00 msgr2=0x7fa10004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 --2- 172.21.15.113:0/2661292013 >> v2:172.21.15.113:3300/0 conn(0x7fa1000ead00 0x7fa10004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f5312f7e640 1 -- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0039b0 msgr2=0x7f52fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f5312f7e640 1 --2- 172.21.15.113:0/4124411770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0039b0 0x7f52fc003dd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5300009c10 tx=0x7f52fc005830 comp rx=0 tx=0).stop 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f531151b640 1 -- 172.21.15.113:0/4124411770 reap_dead start 2024-04-24T23:58:28.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f5312f7e640 1 -- 172.21.15.113:0/4124411770 shutdown_connections 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f5312f7e640 1 -- 172.21.15.113:0/4124411770 >> 172.21.15.113:0/4124411770 conn(0x7f5304058fe0 msgr2=0x7f53040593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f5312f7e640 1 -- 172.21.15.113:0/4124411770 shutdown_connections 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7f5312f7e640 1 -- 172.21.15.113:0/4124411770 wait complete. 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100053ce0 con 0x7fa100038110 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.936+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fa0f8001650 con 0x7fa100038110 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 >> v1:172.21.15.178:6789/0 conn(0x7fa100038110 legacy=0x7fa1000385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa105ffb640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f00039d0 0x7fa0f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa105ffb640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f0004800 0x7fa0f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa105ffb640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f00051c0 0x7fa0f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa100054330 con 0x7fa0f00051c0 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa100053a00 con 0x7fa0f0004800 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f000c140 con 0x7fa0f00039d0 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1067fc640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa10004ffd0 0x7fa100037bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa106ffd640 1 --2- 172.21.15.113:0/2661292013 >> v2:172.21.15.113:3300/0 conn(0x7fa1000ead00 0x7fa10004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa106ffd640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f00039d0 0x7fa0f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1067fc640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f0004800 0x7fa0f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1077fe640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f00051c0 0x7fa0f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7efc7a241640 1 --2- 172.21.15.113:0/4150583324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740cd580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc740722a0 tx=0x7efc7002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a289ff0999684929 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f40d16cb640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a940 0x7f40cc06e150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40c80097a0 tx=0x7f40c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a153feb31d9feb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 -- 172.21.15.113:0/3612111579 shutdown_connections 2024-04-24T23:58:28.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f385359e640 1 -- 172.21.15.113:0/3612111579 wait complete. 2024-04-24T23:58:28.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f5312f7e640 1 Processor -- start 2024-04-24T23:58:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1067fc640 1 -- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f00039d0 msgr2=0x7fa0f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1067fc640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f00039d0 0x7fa0f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1067fc640 1 -- 172.21.15.113:0/2661292013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f00051c0 msgr2=0x7fa0f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1067fc640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f00051c0 0x7fa0f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa1067fc640 1 -- 172.21.15.113:0/2661292013 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100053ce0 con 0x7fa0f0004800 2024-04-24T23:58:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7fa106ffd640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f00039d0 0x7fa0f0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f40d0eca640 1 -- 172.21.15.113:0/4285839663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8037070 con 0x7f40cc15a940 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f40d0eca640 1 -- 172.21.15.113:0/4285839663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c8032440 con 0x7f40cc15a940 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7f40d0eca640 1 -- 172.21.15.113:0/4285839663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8032810 con 0x7f40cc15a940 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f237759e640 1 --2- 172.21.15.113:0/3891866174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2378071cc0 0x7f23781637e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f237806e150 tx=0x7f236c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dc9a29c8a96edd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f2375d9b640 1 -- 172.21.15.113:0/3891866174 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c037070 con 0x7f2378071cc0 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f2375d9b640 1 -- 172.21.15.113:0/3891866174 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c032440 con 0x7f2378071cc0 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc071cc0 msgr2=0x7f34fc1637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f350395a640 1 --2- 172.21.15.113:0/3192685597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc071cc0 0x7f34fc1637e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06e150 tx=0x7f34f402a570 comp rx=0 tx=0).stop 2024-04-24T23:58:28.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 shutdown_connections 2024-04-24T23:58:28.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f350395a640 1 --2- 172.21.15.113:0/3192685597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc071cc0 0x7f34fc1637e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f350395a640 1 --2- 172.21.15.113:0/3192685597 >> v2:172.21.15.178:3300/0 conn(0x7f34fc162870 0x7f34fc0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 >> 172.21.15.113:0/3192685597 conn(0x7f34fc06b0f0 msgr2=0x7f34fc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7fcd5af5f640 1 Processor -- start 2024-04-24T23:58:28.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7fcd5af5f640 1 -- start start 2024-04-24T23:58:28.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7efc7923f640 1 -- 172.21.15.113:0/4150583324 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70037070 con 0x7efc74162930 2024-04-24T23:58:28.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.937+0000 7efc7923f640 1 -- 172.21.15.113:0/4150583324 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc70032440 con 0x7efc74162930 2024-04-24T23:58:28.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7efc7c4a5640 1 -- 172.21.15.113:0/4150583324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 msgr2=0x7efc740cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7efc7c4a5640 1 --2- 172.21.15.113:0/4150583324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740cd580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc740722a0 tx=0x7efc7002a110 comp rx=0 tx=0).stop 2024-04-24T23:58:28.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f2375d9b640 1 -- 172.21.15.113:0/3891866174 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c02ae60 con 0x7f2378071cc0 2024-04-24T23:58:28.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a940 msgr2=0x7f40cc06e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f40d4130640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a940 0x7f40cc06e150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40c80097a0 tx=0x7f40c802a180 comp rx=0 tx=0).stop 2024-04-24T23:58:28.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7efc7c4a5640 1 -- 172.21.15.113:0/4150583324 shutdown_connections 2024-04-24T23:58:28.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7efc7c4a5640 1 --2- 172.21.15.113:0/4150583324 >> v2:172.21.15.113:3300/0 conn(0x7efc740af8a0 0x7efc740c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7efc7c4a5640 1 --2- 172.21.15.113:0/4150583324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740cd580 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7efc7c4a5640 1 -- 172.21.15.113:0/4150583324 >> 172.21.15.113:0/4150583324 conn(0x7efc74072710 msgr2=0x7efc74072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.938+0000 7f5312f7e640 1 -- start start 2024-04-24T23:58:28.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7efc7c4a5640 1 -- 172.21.15.113:0/4150583324 shutdown_connections 2024-04-24T23:58:28.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7fa1067fc640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f0004800 0x7fa0f0004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa108000b60 tx=0x7fa0f0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f803a909909289c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 shutdown_connections 2024-04-24T23:58:28.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f40d4130640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc143a20 0x7f40cc0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f40d4130640 1 --2- 172.21.15.113:0/4285839663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a940 0x7f40cc06e150 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 shutdown_connections 2024-04-24T23:58:28.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 >> 172.21.15.113:0/4285839663 conn(0x7f40cc061f70 msgr2=0x7f40cc0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f5312f7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc0039b0 0x7f530400da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f350395a640 1 -- 172.21.15.113:0/3192685597 wait complete. 2024-04-24T23:58:28.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f237d09e640 1 -- 172.21.15.113:0/3891866174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2378071cc0 msgr2=0x7f23781637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 shutdown_connections 2024-04-24T23:58:28.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f237d09e640 1 --2- 172.21.15.113:0/3891866174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2378071cc0 0x7f23781637e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f237806e150 tx=0x7f236c02a110 comp rx=0 tx=0).stop 2024-04-24T23:58:28.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f5310d1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc0039b0 0x7f530400da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080ae070 con 0x7fa0f0004800 2024-04-24T23:58:28.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1080aa6b0 con 0x7fa0f0004800 2024-04-24T23:58:28.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7fa105ffb640 1 -- 172.21.15.113:0/2661292013 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10806e040 con 0x7fa0f0004800 2024-04-24T23:58:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f5310d1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc0039b0 0x7f530400da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50270/0 (socket says 172.21.15.113:50270) 2024-04-24T23:58:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f5310d1a640 1 -- 172.21.15.113:0/624906362 learned_addr learned my addr 172.21.15.113:0/624906362 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f5312f7e640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530404f9b0 0x7f53040d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f5312f7e640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53040d4820 0x7f53040d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f5312f7e640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5304006720 con 0x7f52fc0039b0 2024-04-24T23:58:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f530bfff640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530404f9b0 0x7f53040d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f5312f7e640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5304057490 con 0x7f53040d4820 2024-04-24T23:58:28.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd5af5f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd540cf8a0 0x7fcd54149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f531151b640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53040d4820 0x7f53040d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f5312f7e640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53040068a0 con 0x7f530404f9b0 2024-04-24T23:58:28.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f40d4130640 1 -- 172.21.15.113:0/4285839663 wait complete. 2024-04-24T23:58:28.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd5af5f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 0x7fcd5415f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd59f5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd540cf8a0 0x7fcd54149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd5af5f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd5415e610 0x7fcd541e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f237d09e640 1 -- 172.21.15.113:0/3891866174 shutdown_connections 2024-04-24T23:58:28.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f237d09e640 1 --2- 172.21.15.113:0/3891866174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2378071cc0 0x7f23781637e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f237d09e640 1 --2- 172.21.15.113:0/3891866174 >> v2:172.21.15.178:3300/0 conn(0x7f2378162870 0x7f23780cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f237d09e640 1 -- 172.21.15.113:0/3891866174 >> 172.21.15.113:0/3891866174 conn(0x7f237806b0f0 msgr2=0x7f237806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fa107fff640 1 -- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f0004800 msgr2=0x7fa0f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fa107fff640 1 --2- 172.21.15.113:0/2661292013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f0004800 0x7fa0f0004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa108000b60 tx=0x7fa0f0005830 comp rx=0 tx=0).stop 2024-04-24T23:58:28.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd5af5f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540d2670 con 0x7fcd5415e610 2024-04-24T23:58:28.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fa1077fe640 1 -- 172.21.15.113:0/2661292013 reap_dead start 2024-04-24T23:58:28.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f237d09e640 1 -- 172.21.15.113:0/3891866174 shutdown_connections 2024-04-24T23:58:28.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd5af5f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540d24f0 con 0x7fcd540cf8a0 2024-04-24T23:58:28.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd5af5f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540b84f0 con 0x7fcd540d07c0 2024-04-24T23:58:28.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7fcd59f5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd540cf8a0 0x7fcd54149ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41734/0 (socket says 172.21.15.113:41734) 2024-04-24T23:58:28.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7efc7c4a5640 1 -- 172.21.15.113:0/4150583324 wait complete. 2024-04-24T23:58:28.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f5310d1a640 1 -- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530404f9b0 msgr2=0x7f53040d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f5310d1a640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530404f9b0 0x7f53040d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f5310d1a640 1 -- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53040d4820 msgr2=0x7f53040d4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f5310d1a640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53040d4820 0x7f53040d4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f5310d1a640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530400c9e0 con 0x7f52fc0039b0 2024-04-24T23:58:28.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd5415e610 0x7fcd541e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f385359e640 1 Processor -- start 2024-04-24T23:58:28.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.939+0000 7f385359e640 1 -- start start 2024-04-24T23:58:28.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.940+0000 7f237d09e640 1 -- 172.21.15.113:0/3891866174 wait complete. 2024-04-24T23:58:28.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f40d4130640 1 Processor -- start 2024-04-24T23:58:28.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd5415e610 0x7fcd541e6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50276/0 (socket says 172.21.15.113:50276) 2024-04-24T23:58:28.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 -- 172.21.15.113:0/1521080386 learned_addr learned my addr 172.21.15.113:0/1521080386 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38540ca6b0 0x7f385416b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f40d4130640 1 -- start start 2024-04-24T23:58:28.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f3854144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38540ca6b0 0x7f385416b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fa107fff640 1 -- 172.21.15.113:0/2661292013 shutdown_connections 2024-04-24T23:58:28.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fa107fff640 1 -- 172.21.15.113:0/2661292013 >> 172.21.15.113:0/2661292013 conn(0x7fa100009fb0 msgr2=0x7fa10000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:28.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38540ca6b0 0x7f385416b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50288/0 (socket says 172.21.15.113:50288) 2024-04-24T23:58:28.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 -- 172.21.15.113:0/2502823394 learned_addr learned my addr 172.21.15.113:0/2502823394 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fa107fff640 1 -- 172.21.15.113:0/2661292013 shutdown_connections 2024-04-24T23:58:28.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385359e640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 0x7f3854147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f5310d1a640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc0039b0 0x7f530400da80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f530002acb0 tx=0x7f5300004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f385406ce30 con 0x7f38540ca6b0 2024-04-24T23:58:28.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f385406ccb0 con 0x7f38540d0610 2024-04-24T23:58:28.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f3851d9b640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f3854144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f385406cfb0 con 0x7f385415a900 2024-04-24T23:58:28.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 -- 172.21.15.113:0/1521080386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 msgr2=0x7fcd5415f260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:58:28.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 0x7fcd5415f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7efc7c4a5640 1 Processor -- start 2024-04-24T23:58:28.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 -- 172.21.15.113:0/1521080386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd540cf8a0 msgr2=0x7fcd54149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f53097fa640 1 -- 172.21.15.113:0/624906362 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300041020 con 0x7f52fc0039b0 2024-04-24T23:58:28.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f53097fa640 1 -- 172.21.15.113:0/624906362 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5300037040 con 0x7f52fc0039b0 2024-04-24T23:58:28.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd540cf8a0 0x7fcd54149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fcd5a75e640 1 -- 172.21.15.113:0/1521080386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5414a760 con 0x7fcd5415e610 2024-04-24T23:58:28.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f53097fa640 1 -- 172.21.15.113:0/624906362 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300036070 con 0x7f52fc0039b0 2024-04-24T23:58:28.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7fcd59f5d640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd540cf8a0 0x7fcd54149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:28.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f5312f7e640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530400ccc0 con 0x7f52fc0039b0 2024-04-24T23:58:28.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7fcd5a75e640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd5415e610 0x7fcd541e6bc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c730 tx=0x7fcd4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f5312f7e640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530400d280 con 0x7f52fc0039b0 2024-04-24T23:58:28.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7fa107fff640 1 -- 172.21.15.113:0/2661292013 wait complete. 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f40d4130640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40cc143a20 0x7f40cc141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f40d4130640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a560 0x7f40cc0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f40d16cb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a560 0x7f40cc0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f40d4130640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc15a940 0x7f40cc132470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d4130640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0cf8b0 con 0x7f40cc15a560 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40cc143a20 0x7f40cc141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40cc143a20 0x7f40cc141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41756/0 (socket says 172.21.15.113:41756) 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d4130640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0cf730 con 0x7f40cc143a20 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 -- 172.21.15.113:0/269958205 learned_addr learned my addr 172.21.15.113:0/269958205 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d26cd640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc15a940 0x7f40cc132470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d4130640 1 -- 172.21.15.113:0/269958205 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc0cfa30 con 0x7f40cc15a940 2024-04-24T23:58:28.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f237d09e640 1 Processor -- start 2024-04-24T23:58:28.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f237d09e640 1 -- start start 2024-04-24T23:58:28.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f350395a640 1 Processor -- start 2024-04-24T23:58:28.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f350395a640 1 -- start start 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f350395a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34fc071cc0 0x7f34fc0b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f350395a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34fc0d0510 0x7f34fc1f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7f350395a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc162870 0x7f34fc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f350395a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc163ea0 con 0x7f34fc162870 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f350395a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc163d20 con 0x7f34fc071cc0 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f350395a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f34fc164020 con 0x7f34fc0d0510 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34fc071cc0 0x7f34fc0b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34fc071cc0 0x7f34fc0b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41760/0 (socket says 172.21.15.113:41760) 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 -- 172.21.15.113:0/1931424790 learned_addr learned my addr 172.21.15.113:0/1931424790 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f3500ef5640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34fc0d0510 0x7f34fc1f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7efc7c4a5640 1 -- start start 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.942+0000 7efc7c4a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406dac0 0x7efc740c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7c4a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc740af8a0 0x7efc74160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7c4a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc79a40640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc740af8a0 0x7efc74160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc79a40640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc740af8a0 0x7efc74160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:37756/0 (socket says 172.21.15.113:37756) 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc79a40640 1 -- 172.21.15.113:0/3353373082 learned_addr learned my addr 172.21.15.113:0/3353373082 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7c4a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efc740c3910 con 0x7efc7406dac0 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7aa42640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:28.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7c4a5640 1 -- 172.21.15.113:0/3353373082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc740c3790 con 0x7efc74162930 2024-04-24T23:58:28.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7c4a5640 1 -- 172.21.15.113:0/3353373082 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc740c3a90 con 0x7efc740af8a0 2024-04-24T23:58:28.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 -- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc15a940 msgr2=0x7f40cc132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc15a940 0x7f40cc132470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 -- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a560 msgr2=0x7f40cc0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a560 0x7f40cc0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f40d1ecc640 1 -- 172.21.15.113:0/269958205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc143540 con 0x7f40cc143a20 2024-04-24T23:58:28.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7fa107fff640 1 Processor -- start 2024-04-24T23:58:28.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7fa107fff640 1 -- start start 2024-04-24T23:58:28.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 -- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34fc0d0510 msgr2=0x7f34fc1f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34fc0d0510 0x7f34fc1f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 -- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc162870 msgr2=0x7f34fc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc162870 0x7f34fc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f35016f6640 1 -- 172.21.15.113:0/1931424790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc1f9ab0 con 0x7f34fc071cc0 2024-04-24T23:58:28.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237d09e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2378071cc0 0x7f23781eb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7aa42640 1 -- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc740af8a0 msgr2=0x7efc74160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7aa42640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc740af8a0 0x7efc74160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7aa42640 1 -- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406dac0 msgr2=0x7efc740c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:28.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7efc7aa42640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406dac0 0x7efc740c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:28.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7efc7aa42640 1 -- 172.21.15.113:0/3353373082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc740c4440 con 0x7efc74162930 2024-04-24T23:58:28.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f5312f7e640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5304005390 con 0x7f52fc0039b0 2024-04-24T23:58:28.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237d09e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780d0510 0x7f23781ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f35016f6640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34fc071cc0 0x7f34fc0b0ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00d7d0 tx=0x7f34ec00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7efc7aa42640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740c4c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00ca70 tx=0x7efc6c00cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:28.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237d09e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2378162870 0x7f23781ed140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:28.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237d09e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2378163ea0 con 0x7f23780d0510 2024-04-24T23:58:28.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237d09e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2378163d20 con 0x7f2378162870 2024-04-24T23:58:28.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237d09e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2378164020 con 0x7f2378071cc0 2024-04-24T23:58:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2378162870 0x7f23781ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2378162870 0x7f23781ed140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41772/0 (socket says 172.21.15.113:41772) 2024-04-24T23:58:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 -- 172.21.15.113:0/548759582 learned_addr learned my addr 172.21.15.113:0/548759582 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fcd4a7fc640 1 -- 172.21.15.113:0/1521080386 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c016020 con 0x7fcd5415e610 2024-04-24T23:58:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f2376d9d640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2378071cc0 0x7f23781eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7efc6affd640 1 -- 172.21.15.113:0/3353373082 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c017070 con 0x7efc74162930 2024-04-24T23:58:29.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f34f1ffb640 1 -- 172.21.15.113:0/1931424790 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec004030 con 0x7f34fc071cc0 2024-04-24T23:58:29.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fcd4a7fc640 1 -- 172.21.15.113:0/1521080386 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c006d50 con 0x7fcd5415e610 2024-04-24T23:58:29.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fcd4a7fc640 1 -- 172.21.15.113:0/1521080386 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c003bb0 con 0x7fcd5415e610 2024-04-24T23:58:29.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7efc6affd640 1 -- 172.21.15.113:0/3353373082 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6c00ab10 con 0x7efc74162930 2024-04-24T23:58:29.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f40d1ecc640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40cc143a20 0x7f40cc141b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c8c0 tx=0x7f40c400cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f34f1ffb640 1 -- 172.21.15.113:0/1931424790 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec0041d0 con 0x7f34fc071cc0 2024-04-24T23:58:29.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f350395a640 1 -- 172.21.15.113:0/1931424790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc0a96d0 con 0x7f34fc071cc0 2024-04-24T23:58:29.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5414aa40 con 0x7fcd5415e610 2024-04-24T23:58:29.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 -- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2378071cc0 msgr2=0x7f23781eb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2378071cc0 0x7f23781eb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 -- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780d0510 msgr2=0x7f23781ecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780d0510 0x7f23781ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 -- 172.21.15.113:0/548759582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23781ec3f0 con 0x7f2378162870 2024-04-24T23:58:29.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5414b000 con 0x7fcd5415e610 2024-04-24T23:58:29.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fa107fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0004800 0x7fa1000df550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fa107fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1000dfbd0 0x7fa100036590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f237759e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2378162870 0x7f23781ed140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f236c035040 tx=0x7f236c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fa107fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1000355f0 0x7fa100036ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fa107fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1000547c0 con 0x7fa0f0004800 2024-04-24T23:58:29.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fa107fff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa100038cb0 con 0x7fa1000355f0 2024-04-24T23:58:29.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7fa107fff640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa100054940 con 0x7fa1000dfbd0 2024-04-24T23:58:29.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f40c27fc640 1 -- 172.21.15.113:0/269958205 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4018070 con 0x7f40cc143a20 2024-04-24T23:58:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f40c27fc640 1 -- 172.21.15.113:0/269958205 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c400a640 con 0x7f40cc143a20 2024-04-24T23:58:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f40c27fc640 1 -- 172.21.15.113:0/269958205 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c400aa30 con 0x7f40cc143a20 2024-04-24T23:58:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f40d4130640 1 -- 172.21.15.113:0/269958205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc130a80 con 0x7f40cc143a20 2024-04-24T23:58:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f40d4130640 1 -- 172.21.15.113:0/269958205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc134350 con 0x7f40cc143a20 2024-04-24T23:58:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0004800 0x7fa1000df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7efc6affd640 1 -- 172.21.15.113:0/3353373082 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c012400 con 0x7efc74162930 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7efc7c4a5640 1 -- 172.21.15.113:0/3353373082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc740b2f90 con 0x7efc74162930 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0004800 0x7fa1000df550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50328/0 (socket says 172.21.15.113:50328) 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 -- 172.21.15.113:0/1393064706 learned_addr learned my addr 172.21.15.113:0/1393064706 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa1077fe640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1000355f0 0x7fa100036ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 -- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1000dfbd0 msgr2=0x7fa100036590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa1067fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1000dfbd0 0x7fa100036590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1000dfbd0 0x7fa100036590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 -- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1000355f0 msgr2=0x7fa100036ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1000355f0 0x7fa100036ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 -- 172.21.15.113:0/1393064706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100035bb0 con 0x7fa0f0004800 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa1067fc640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1000dfbd0 0x7fa100036590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7fa106ffd640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0004800 0x7fa1000df550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009fd0 tx=0x7fa0fc011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f235f7fe640 1 -- 172.21.15.113:0/548759582 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c037040 con 0x7f2378162870 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f235f7fe640 1 -- 172.21.15.113:0/548759582 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c042070 con 0x7f2378162870 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f235f7fe640 1 -- 172.21.15.113:0/548759582 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0308a0 con 0x7f2378162870 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23781580b0 con 0x7f2378162870 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2378158670 con 0x7f2378162870 2024-04-24T23:58:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7efc7c4a5640 1 -- 172.21.15.113:0/3353373082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc740b3550 con 0x7efc74162930 2024-04-24T23:58:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.946+0000 7f34f1ffb640 1 -- 172.21.15.113:0/1931424790 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec009620 con 0x7f34fc071cc0 2024-04-24T23:58:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f53097fa640 1 -- 172.21.15.113:0/624906362 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5300044020 con 0x7f52fc0039b0 2024-04-24T23:58:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 -- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 msgr2=0x7f3854147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 0x7f3854147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 -- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 msgr2=0x7f3854144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f3854144940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.946+0000 7f350395a640 1 -- 172.21.15.113:0/1931424790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc0a9c90 con 0x7f34fc071cc0 2024-04-24T23:58:29.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.941+0000 7f385259c640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38541575e0 con 0x7f38540ca6b0 2024-04-24T23:58:29.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.943+0000 7f3851d9b640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f3854144940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:29.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.944+0000 7f385259c640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38540ca6b0 0x7f385416b130 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f384400b3e0 tx=0x7f384400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7f34f1ffb640 1 -- 172.21.15.113:0/1931424790 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34ec009840 con 0x7f34fc071cc0 2024-04-24T23:58:29.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7f40d4130640 1 -- 172.21.15.113:0/269958205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc0691a0 con 0x7f40cc143a20 2024-04-24T23:58:29.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc74068220 con 0x7efc74162930 2024-04-24T23:58:29.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7f40c27fc640 1 -- 172.21.15.113:0/269958205 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40c401b070 con 0x7f40cc143a20 2024-04-24T23:58:29.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2358005290 con 0x7f2378162870 2024-04-24T23:58:29.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7f34f1ffb640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f34e006be80 0x7f34e006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7fa0ef7fe640 1 -- 172.21.15.113:0/1393064706 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc019070 con 0x7fa0f0004800 2024-04-24T23:58:29.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa100035e90 con 0x7fa0f0004800 2024-04-24T23:58:29.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f34f1ffb640 1 -- 172.21.15.113:0/1931424790 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f34ec08d030 con 0x7f34fc071cc0 2024-04-24T23:58:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7f40c27fc640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40b006bf00 0x7f40b006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f40c27fc640 1 -- 172.21.15.113:0/269958205 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f40c408a5c0 con 0x7f40cc143a20 2024-04-24T23:58:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f40d16cb640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40b006bf00 0x7f40b006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa100043680 con 0x7fa0f0004800 2024-04-24T23:58:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d4005290 con 0x7f34fc071cc0 2024-04-24T23:58:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f235f7fe640 1 -- 172.21.15.113:0/548759582 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f236c030a40 con 0x7f2378162870 2024-04-24T23:58:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7efc6affd640 1 -- 172.21.15.113:0/3353373082 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc6c01a070 con 0x7efc74162930 2024-04-24T23:58:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.945+0000 7f53097fa640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52f006bf30 0x7f52f006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.946+0000 7f53097fa640 1 -- 172.21.15.113:0/624906362 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53000ab1e0 con 0x7f52fc0039b0 2024-04-24T23:58:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.946+0000 7f530bfff640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52f006bf30 0x7f52f006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.947+0000 7f530bfff640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52f006bf30 0x7f52f006e3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53040d5470 tx=0x7f52f8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7efc6affd640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5806be30 0x7efc5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7fcd37fff640 1 -- 172.21.15.113:0/1521080386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd540d3940 con 0x7fcd5415e610 2024-04-24T23:58:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7efc7a241640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5806be30 0x7efc5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7efc6affd640 1 -- 172.21.15.113:0/3353373082 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efc6c08aa70 con 0x7efc74162930 2024-04-24T23:58:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f235f7fe640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406bf00 0x7f236406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7f235f7fe640 1 -- 172.21.15.113:0/548759582 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f236c0ab7a0 con 0x7f2378162870 2024-04-24T23:58:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f3842ffd640 1 -- 172.21.15.113:0/2502823394 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844015020 con 0x7f38540ca6b0 2024-04-24T23:58:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7f2376d9d640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406bf00 0x7f236406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38541578c0 con 0x7f38540ca6b0 2024-04-24T23:58:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.948+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38541476f0 con 0x7f38540ca6b0 2024-04-24T23:58:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7f2376d9d640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406bf00 0x7f236406e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2368009a30 tx=0x7f2368008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7f3500ef5640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f34e006be80 0x7f34e006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7f40d16cb640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40b006bf00 0x7f40b006e3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40c8034000 tx=0x7f40c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.949+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3854147200 con 0x7f38540ca6b0 2024-04-24T23:58:29.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.950+0000 7efc7a241640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5806be30 0x7efc5806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc70034000 tx=0x7efc7002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.950+0000 7f3500ef5640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f34e006be80 0x7f34e006e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f34fc1f9520 tx=0x7f34f8000bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.950+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa100005390 con 0x7fa0f0004800 2024-04-24T23:58:29.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.952+0000 7f40c27fc640 1 -- 172.21.15.113:0/269958205 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40cc0691a0 con 0x7f40cc143a20 2024-04-24T23:58:29.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.953+0000 7f53097fa640 1 -- 172.21.15.113:0/624906362 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f530004b030 con 0x7f52fc0039b0 2024-04-24T23:58:29.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.953+0000 7f3842ffd640 1 -- 172.21.15.113:0/2502823394 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3844011830 con 0x7f38540ca6b0 2024-04-24T23:58:29.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.953+0000 7f3842ffd640 1 -- 172.21.15.113:0/2502823394 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440091f0 con 0x7f38540ca6b0 2024-04-24T23:58:29.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.953+0000 7fa0ef7fe640 1 -- 172.21.15.113:0/1393064706 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00f9c0 con 0x7fa0f0004800 2024-04-24T23:58:29.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.953+0000 7fa0ef7fe640 1 -- 172.21.15.113:0/1393064706 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc023060 con 0x7fa0f0004800 2024-04-24T23:58:29.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.955+0000 7fcd4a7fc640 1 -- 172.21.15.113:0/1521080386 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd4c00a7f0 con 0x7fcd5415e610 2024-04-24T23:58:29.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.955+0000 7fcd4a7fc640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcd3806bf00 0x7fcd3806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.955+0000 7fcd4a7fc640 1 -- 172.21.15.113:0/1521080386 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcd4c08b8c0 con 0x7fcd5415e610 2024-04-24T23:58:29.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.955+0000 7f3842ffd640 1 -- 172.21.15.113:0/2502823394 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3844018020 con 0x7f38540ca6b0 2024-04-24T23:58:29.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.956+0000 7fcd59f5d640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcd3806bf00 0x7fcd3806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.956+0000 7f3842ffd640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f383406be30 0x7f383406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.956+0000 7fa0ef7fe640 1 -- 172.21.15.113:0/1393064706 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0fc01c070 con 0x7fa0f0004800 2024-04-24T23:58:29.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.956+0000 7f3851d9b640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f383406be30 0x7f383406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.956+0000 7fcd59f5d640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcd3806bf00 0x7fcd3806e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009ac0 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.957+0000 7fa0ef7fe640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa0f406be30 0x7fa0f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.957+0000 7efc6affd640 1 -- 172.21.15.113:0/3353373082 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc6c058d40 con 0x7efc74162930 2024-04-24T23:58:29.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.957+0000 7f3851d9b640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f383406be30 0x7f383406e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3848034000 tx=0x7f384802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.957+0000 7fa1067fc640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa0f406be30 0x7fa0f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.957+0000 7fa1067fc640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa0f406be30 0x7fa0f406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c110 tx=0x7fa1080b5040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.961+0000 7f235f7fe640 1 -- 172.21.15.113:0/548759582 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f236c07a130 con 0x7f2378162870 2024-04-24T23:58:29.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.962+0000 7f3842ffd640 1 -- 172.21.15.113:0/2502823394 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38440576d0 con 0x7f38540ca6b0 2024-04-24T23:58:29.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.963+0000 7fcd4a7fc640 1 -- 172.21.15.113:0/1521080386 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcd540d3940 con 0x7fcd5415e610 2024-04-24T23:58:29.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.963+0000 7fa0ef7fe640 1 -- 172.21.15.113:0/1393064706 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa0fc057c90 con 0x7fa0f0004800 2024-04-24T23:58:29.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.966+0000 7f34f1ffb640 1 -- 172.21.15.113:0/1931424790 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f34ec0591e0 con 0x7f34fc071cc0 2024-04-24T23:58:29.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.972+0000 7f3842ffd640 1 -- 172.21.15.113:0/2502823394 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f384401f020 con 0x7f38540ca6b0 2024-04-24T23:58:29.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:28.980+0000 7fa0ef7fe640 1 -- 172.21.15.113:0/1393064706 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0fc05a2f0 con 0x7fa0f0004800 2024-04-24T23:58:29.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.359+0000 7f5312f7e640 1 -- 172.21.15.113:0/624906362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f53040e45c0 con 0x7f52fc0039b0 2024-04-24T23:58:29.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.360+0000 7f53097fa640 1 -- 172.21.15.113:0/624906362 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5300079cc0 con 0x7f52fc0039b0 2024-04-24T23:58:29.360 INFO:teuthology.orchestra.run.smithi113.stdout:55834574855 2024-04-24T23:58:29.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.363+0000 7f52f6ffd640 1 -- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52f006bf30 msgr2=0x7f52f006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52f006bf30 0x7f52f006e3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53040d5470 tx=0x7f52f8009190 comp rx=0 tx=0).stop 2024-04-24T23:58:29.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 -- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc0039b0 msgr2=0x7f530400da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc0039b0 0x7f530400da80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f530002acb0 tx=0x7f5300004730 comp rx=0 tx=0).stop 2024-04-24T23:58:29.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 -- 172.21.15.113:0/624906362 shutdown_connections 2024-04-24T23:58:29.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52f006bf30 0x7f52f006e3f0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53040d5470 tx=0x7f52f8009190 comp rx=0 tx=0).stop 2024-04-24T23:58:29.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53040d4820 0x7f53040d4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f530404f9b0 0x7f53040d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.364+0000 7f52f6ffd640 1 --2- 172.21.15.113:0/624906362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52fc0039b0 0x7f530400da80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.365+0000 7f52f6ffd640 1 -- 172.21.15.113:0/624906362 >> 172.21.15.113:0/624906362 conn(0x7f5304058fe0 msgr2=0x7f5304057b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.365+0000 7f52f6ffd640 1 -- 172.21.15.113:0/624906362 shutdown_connections 2024-04-24T23:58:29.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.366+0000 7f52f6ffd640 1 -- 172.21.15.113:0/624906362 wait complete. 2024-04-24T23:58:29.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.367+0000 7f40d4130640 1 -- 172.21.15.113:0/269958205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f40cc068220 con 0x7f40cc143a20 2024-04-24T23:58:29.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.368+0000 7f40c27fc640 1 -- 172.21.15.113:0/269958205 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40c405de20 con 0x7f40cc143a20 2024-04-24T23:58:29.368 INFO:teuthology.orchestra.run.smithi113.stdout:55834574855 2024-04-24T23:58:29.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.371+0000 7f40affff640 1 -- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40b006bf00 msgr2=0x7f40b006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.371+0000 7f40affff640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40b006bf00 0x7f40b006e3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40c8034000 tx=0x7f40c802e000 comp rx=0 tx=0).stop 2024-04-24T23:58:29.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 -- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40cc143a20 msgr2=0x7f40cc141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40cc143a20 0x7f40cc141b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c8c0 tx=0x7f40c400cd80 comp rx=0 tx=0).stop 2024-04-24T23:58:29.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 -- 172.21.15.113:0/269958205 shutdown_connections 2024-04-24T23:58:29.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40b006bf00 0x7f40b006e3c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40cc15a940 0x7f40cc132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40cc15a560 0x7f40cc0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 --2- 172.21.15.113:0/269958205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40cc143a20 0x7f40cc141b00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 -- 172.21.15.113:0/269958205 >> 172.21.15.113:0/269958205 conn(0x7f40cc061f70 msgr2=0x7f40cc157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 -- 172.21.15.113:0/269958205 shutdown_connections 2024-04-24T23:58:29.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.372+0000 7f40affff640 1 -- 172.21.15.113:0/269958205 wait complete. 2024-04-24T23:58:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.375+0000 7fcd37fff640 1 -- 172.21.15.113:0/1521080386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcd54068220 con 0x7fcd5415e610 2024-04-24T23:58:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.376+0000 7fcd4a7fc640 1 -- 172.21.15.113:0/1521080386 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcd4c020020 con 0x7fcd5415e610 2024-04-24T23:58:29.376 INFO:teuthology.orchestra.run.smithi113.stdout:55834574856 2024-04-24T23:58:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.380+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcd3806bf00 msgr2=0x7fcd3806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.380+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcd3806bf00 0x7fcd3806e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009ac0 tx=0x7fcd50008040 comp rx=0 tx=0).stop 2024-04-24T23:58:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.380+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd5415e610 msgr2=0x7fcd541e6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.380+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd5415e610 0x7fcd541e6bc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c730 tx=0x7fcd4c00cbf0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.380+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 shutdown_connections 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.380+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcd3806bf00 0x7fcd3806e3c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.380+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd5415e610 0x7fcd541e6bc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.381+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd540d07c0 0x7fcd5415f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.381+0000 7fcd5af5f640 1 --2- 172.21.15.113:0/1521080386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd540cf8a0 0x7fcd54149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.381+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 >> 172.21.15.113:0/1521080386 conn(0x7fcd540d3fd0 msgr2=0x7fcd540b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.381+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 shutdown_connections 2024-04-24T23:58:29.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.381+0000 7fcd5af5f640 1 -- 172.21.15.113:0/1521080386 wait complete. 2024-04-24T23:58:29.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.382+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2358005580 con 0x7f2378162870 2024-04-24T23:58:29.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.382+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f34d4005850 con 0x7f34fc071cc0 2024-04-24T23:58:29.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.383+0000 7f34f1ffb640 1 -- 172.21.15.113:0/1931424790 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f34ec058ae0 con 0x7f34fc071cc0 2024-04-24T23:58:29.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.383+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efc740b6200 con 0x7efc74162930 2024-04-24T23:58:29.383 INFO:teuthology.orchestra.run.smithi113.stdout:55834574855 2024-04-24T23:58:29.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.383+0000 7efc6affd640 1 -- 172.21.15.113:0/3353373082 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc6c05d040 con 0x7efc74162930 2024-04-24T23:58:29.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-24T23:58:29.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.383+0000 7f235f7fe640 1 -- 172.21.15.113:0/548759582 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f236c079a30 con 0x7f2378162870 2024-04-24T23:58:29.385 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.386 INFO:teuthology.orchestra.run.smithi113.stdout:55834574857 2024-04-24T23:58:29.386 INFO:teuthology.orchestra.run.smithi113.stdout:55834574856 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406bf00 msgr2=0x7f236406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406bf00 0x7f236406e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2368009a30 tx=0x7f2368008040 comp rx=0 tx=0).stop 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2378162870 msgr2=0x7f23781ed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2378162870 0x7f23781ed140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f236c035040 tx=0x7f236c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f34e006be80 msgr2=0x7f34e006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f34e006be80 0x7f34e006e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f34fc1f9520 tx=0x7f34f8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34fc071cc0 msgr2=0x7f34fc0b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34fc071cc0 0x7f34fc0b0ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00d7d0 tx=0x7f34ec00dc90 comp rx=0 tx=0).stop 2024-04-24T23:58:29.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.386+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5806be30 msgr2=0x7efc5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.386+0000 7efc68ff9640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5806be30 0x7efc5806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc70034000 tx=0x7efc7002c000 comp rx=0 tx=0).stop 2024-04-24T23:58:29.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 msgr2=0x7efc740c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740c4c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00ca70 tx=0x7efc6c00cf30 comp rx=0 tx=0).stop 2024-04-24T23:58:29.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 shutdown_connections 2024-04-24T23:58:29.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 shutdown_connections 2024-04-24T23:58:29.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5806be30 0x7efc5806e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f34e006be80 0x7f34e006e340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc74162930 0x7efc740c4c10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34fc162870 0x7f34fc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc740af8a0 0x7efc74160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34fc0d0510 0x7f34fc1f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 --2- 172.21.15.113:0/1931424790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34fc071cc0 0x7f34fc0b0ec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 --2- 172.21.15.113:0/3353373082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406dac0 0x7efc740c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 >> 172.21.15.113:0/3353373082 conn(0x7efc74072710 msgr2=0x7efc740cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 >> 172.21.15.113:0/1931424790 conn(0x7f34fc06b0f0 msgr2=0x7f34fc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 shutdown_connections 2024-04-24T23:58:29.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406bf00 0x7f236406e3c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2378162870 0x7f23781ed140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780d0510 0x7f23781ecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 --2- 172.21.15.113:0/548759582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2378071cc0 0x7f23781eb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 >> 172.21.15.113:0/548759582 conn(0x7f237806b0f0 msgr2=0x7f23780b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 shutdown_connections 2024-04-24T23:58:29.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 shutdown_connections 2024-04-24T23:58:29.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f34e77fe640 1 -- 172.21.15.113:0/1931424790 wait complete. 2024-04-24T23:58:29.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7efc68ff9640 1 -- 172.21.15.113:0/3353373082 wait complete. 2024-04-24T23:58:29.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 shutdown_connections 2024-04-24T23:58:29.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.387+0000 7f237d09e640 1 -- 172.21.15.113:0/548759582 wait complete. 2024-04-24T23:58:29.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-24T23:58:29.394 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.397+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f38540691a0 con 0x7f38540ca6b0 2024-04-24T23:58:29.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.397+0000 7f3842ffd640 1 -- 172.21.15.113:0/2502823394 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f384400f2e0 con 0x7f38540ca6b0 2024-04-24T23:58:29.397 INFO:teuthology.orchestra.run.smithi113.stdout:55834574856 2024-04-24T23:58:29.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa100009000 con 0x7fa0f0004800 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f383406be30 msgr2=0x7f383406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f383406be30 0x7f383406e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3848034000 tx=0x7f384802e000 comp rx=0 tx=0).stop 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38540ca6b0 msgr2=0x7f385416b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38540ca6b0 0x7f385416b130 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f384400b3e0 tx=0x7f384400b8a0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 shutdown_connections 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f383406be30 0x7f383406e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38540d0610 0x7f3854144940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f385415a900 0x7f3854147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 --2- 172.21.15.113:0/2502823394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38540ca6b0 0x7f385416b130 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 >> 172.21.15.113:0/2502823394 conn(0x7f38540cfb20 msgr2=0x7f38540af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 shutdown_connections 2024-04-24T23:58:29.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.400+0000 7fa0ef7fe640 1 -- 172.21.15.113:0/1393064706 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0fc059bf0 con 0x7fa0f0004800 2024-04-24T23:58:29.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.401+0000 7f385359e640 1 -- 172.21.15.113:0/2502823394 wait complete. 2024-04-24T23:58:29.402 INFO:teuthology.orchestra.run.smithi113.stdout:55834574854 2024-04-24T23:58:29.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-24T23:58:29.404 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa0f406be30 msgr2=0x7fa0f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa0f406be30 0x7fa0f406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c110 tx=0x7fa1080b5040 comp rx=0 tx=0).stop 2024-04-24T23:58:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0004800 msgr2=0x7fa1000df550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0004800 0x7fa1000df550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009fd0 tx=0x7fa0fc011b30 comp rx=0 tx=0).stop 2024-04-24T23:58:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 shutdown_connections 2024-04-24T23:58:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa0f406be30 0x7fa0f406e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1000355f0 0x7fa100036ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1000dfbd0 0x7fa100036590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 --2- 172.21.15.113:0/1393064706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0004800 0x7fa1000df550 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009fd0 tx=0x7fa0fc011b30 comp rx=0 tx=0).stop 2024-04-24T23:58:29.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 >> 172.21.15.113:0/1393064706 conn(0x7fa100009fb0 msgr2=0x7fa100004930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 shutdown_connections 2024-04-24T23:58:29.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.404+0000 7fa107fff640 1 -- 172.21.15.113:0/1393064706 wait complete. 2024-04-24T23:58:29.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-04-24T23:58:29.406 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-24T23:58:29.407 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-24T23:58:29.408 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-04-24T23:58:29.415 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-24T23:58:29.416 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:29.416 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T23:58:29.416 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:58:29.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.552+0000 7f19d4a1c640 1 Processor -- start 2024-04-24T23:58:29.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.552+0000 7f19d4a1c640 1 -- start start 2024-04-24T23:58:29.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.552+0000 7f19d4a1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d00b3fd0 0x7f19d006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.552+0000 7f19d4a1c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f19d006e510 0x7f19d00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19d4a1c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d0144cf0 con 0x7f19d0144910 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19d4a1c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f19d00b2f30 con 0x7f19d006e510 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19d4a1c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f19d00b3210 con 0x7f19d00b3fd0 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19ced9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f19d006e510 0x7f19d00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19cdd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d00b3fd0 0x7f19d006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19cdd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d00b3fd0 0x7f19d006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:37788/0 (socket says 172.21.15.113:37788) 2024-04-24T23:58:29.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19cdd9b640 1 -- 172.21.15.113:0/3579743955 learned_addr learned my addr 172.21.15.113:0/3579743955 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19cdd9b640 1 -- 172.21.15.113:0/3579743955 >> v2:172.21.15.178:3300/0 conn(0x7f19d006e510 msgr2=0x7f19d00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19cdd9b640 1 --2- 172.21.15.113:0/3579743955 >> v2:172.21.15.178:3300/0 conn(0x7f19d006e510 0x7f19d00cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19cdd9b640 1 -- 172.21.15.113:0/3579743955 >> v1:172.21.15.178:6789/0 conn(0x7f19d0144910 legacy=0x7f19d00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19cdd9b640 1 -- 172.21.15.113:0/3579743955 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d00b34f0 con 0x7f19d00b3fd0 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.553+0000 7f19ced9d640 1 --2- 172.21.15.113:0/3579743955 >> v2:172.21.15.178:3300/0 conn(0x7f19d006e510 0x7f19d00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.554+0000 7f19cdd9b640 1 --2- 172.21.15.113:0/3579743955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d00b3fd0 0x7f19d006dfd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009870 tx=0x7f19c002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ee6062ce3a793d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.554+0000 7f19cd59a640 1 -- 172.21.15.113:0/3579743955 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0037070 con 0x7f19d00b3fd0 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.554+0000 7f19cd59a640 1 -- 172.21.15.113:0/3579743955 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c0032440 con 0x7f19d00b3fd0 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.554+0000 7f19cd59a640 1 -- 172.21.15.113:0/3579743955 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0032810 con 0x7f19d00b3fd0 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.554+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3579743955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d00b3fd0 msgr2=0x7f19d006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.555+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3579743955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d00b3fd0 0x7f19d006dfd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009870 tx=0x7f19c002a270 comp rx=0 tx=0).stop 2024-04-24T23:58:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.555+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3579743955 shutdown_connections 2024-04-24T23:58:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.555+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3579743955 >> v2:172.21.15.178:3300/0 conn(0x7f19d006e510 0x7f19d00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.555+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3579743955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d00b3fd0 0x7f19d006dfd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.555+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3579743955 >> 172.21.15.113:0/3579743955 conn(0x7f19d00b1a50 msgr2=0x7f19d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.555+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3579743955 shutdown_connections 2024-04-24T23:58:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.555+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3579743955 wait complete. 2024-04-24T23:58:29.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.556+0000 7f19d4a1c640 1 Processor -- start 2024-04-24T23:58:29.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.556+0000 7f19d4a1c640 1 -- start start 2024-04-24T23:58:29.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d006e510 0x7f19d01f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d00b3fd0 0x7f19d013fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d0144910 0x7f19d013ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f19d00b4da0 con 0x7f19d0144910 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19d00b4410 con 0x7f19d00b3fd0 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f19d00b4f20 con 0x7f19d006e510 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d0144910 0x7f19d013ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d0144910 0x7f19d013ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50334/0 (socket says 172.21.15.113:50334) 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 -- 172.21.15.113:0/3014664124 learned_addr learned my addr 172.21.15.113:0/3014664124 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ce59c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d006e510 0x7f19d01f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19cdd9b640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d00b3fd0 0x7f19d013fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 -- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d006e510 msgr2=0x7f19d01f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d006e510 0x7f19d01f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 -- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d00b3fd0 msgr2=0x7f19d013fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d00b3fd0 0x7f19d013fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 -- 172.21.15.113:0/3014664124 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d014c720 con 0x7f19d0144910 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19ced9d640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d0144910 0x7f19d013ffc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f19c800efc0 tx=0x7f19c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19beffd640 1 -- 172.21.15.113:0/3014664124 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8018030 con 0x7f19d0144910 2024-04-24T23:58:29.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19beffd640 1 -- 172.21.15.113:0/3014664124 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c800cd30 con 0x7f19d0144910 2024-04-24T23:58:29.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d014ca00 con 0x7f19d0144910 2024-04-24T23:58:29.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19beffd640 1 -- 172.21.15.113:0/3014664124 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8003bb0 con 0x7f19d0144910 2024-04-24T23:58:29.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.557+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d01f1bb0 con 0x7f19d0144910 2024-04-24T23:58:29.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.558+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d0071390 con 0x7f19d0144910 2024-04-24T23:58:29.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.559+0000 7f19beffd640 1 -- 172.21.15.113:0/3014664124 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19c801b070 con 0x7f19d0144910 2024-04-24T23:58:29.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.559+0000 7f19beffd640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f19b006be30 0x7f19b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.559+0000 7f19ce59c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f19b006be30 0x7f19b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.560+0000 7f19beffd640 1 -- 172.21.15.113:0/3014664124 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19c808b6e0 con 0x7f19d0144910 2024-04-24T23:58:29.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.560+0000 7f19ce59c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f19b006be30 0x7f19b006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f19c40016e0 tx=0x7f19c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.564+0000 7f19beffd640 1 -- 172.21.15.113:0/3014664124 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f19d0071390 con 0x7f19d0144910 2024-04-24T23:58:29.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.732+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19d00691a0 con 0x7f19b006be30 2024-04-24T23:58:29.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.738+0000 7f19beffd640 1 -- 172.21.15.113:0/3014664124 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f19d00691a0 con 0x7f19b006be30 2024-04-24T23:58:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:29.741 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f19b006be30 msgr2=0x7f19b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f19b006be30 0x7f19b006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f19c40016e0 tx=0x7f19c4009040 comp rx=0 tx=0).stop 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d0144910 msgr2=0x7f19d013ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d0144910 0x7f19d013ffc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f19c800efc0 tx=0x7f19c800c490 comp rx=0 tx=0).stop 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 shutdown_connections 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f19b006be30 0x7f19b006e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19d00b3fd0 0x7f19d013fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19d0144910 0x7f19d013ffc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 --2- 172.21.15.113:0/3014664124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19d006e510 0x7f19d01f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 >> 172.21.15.113:0/3014664124 conn(0x7f19d00b1a50 msgr2=0x7f19d015eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 shutdown_connections 2024-04-24T23:58:29.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.741+0000 7f19d4a1c640 1 -- 172.21.15.113:0/3014664124 wait complete. 2024-04-24T23:58:29.751 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-24T23:58:29.113873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235632,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739088,"statfs":{"total":773094113280,"available":772852826112,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.870942"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678418+0000","last_change":"2024-04-24T23:58:19.270520+0000","last_active":"2024-04-24T23:58:20.678418+0000","last_peered":"2024-04-24T23:58:20.678418+0000","last_clean":"2024-04-24T23:58:20.678418+0000","last_became_active":"2024-04-24T23:58:19.270280+0000","last_became_peered":"2024-04-24T23:58:19.270280+0000","last_unstale":"2024-04-24T23:58:20.678418+0000","last_undegraded":"2024-04-24T23:58:20.678418+0000","last_fullsized":"2024-04-24T23:58:20.678418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254502+0000","last_change":"2024-04-24T23:58:19.252635+0000","last_active":"2024-04-24T23:58:20.254502+0000","last_peered":"2024-04-24T23:58:20.254502+0000","last_clean":"2024-04-24T23:58:20.254502+0000","last_became_active":"2024-04-24T23:58:19.252352+0000","last_became_peered":"2024-04-24T23:58:19.252352+0000","last_unstale":"2024-04-24T23:58:20.254502+0000","last_undegraded":"2024-04-24T23:58:20.254502+0000","last_fullsized":"2024-04-24T23:58:20.254502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254874+0000","last_change":"2024-04-24T23:58:19.257041+0000","last_active":"2024-04-24T23:58:20.254874+0000","last_peered":"2024-04-24T23:58:20.254874+0000","last_clean":"2024-04-24T23:58:20.254874+0000","last_became_active":"2024-04-24T23:58:19.256765+0000","last_became_peered":"2024-04-24T23:58:19.256765+0000","last_unstale":"2024-04-24T23:58:20.254874+0000","last_undegraded":"2024-04-24T23:58:20.254874+0000","last_fullsized":"2024-04-24T23:58:20.254874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.271981+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.271589+0000","last_became_peered":"2024-04-24T23:58:19.271589+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678540+0000","last_change":"2024-04-24T23:58:19.260799+0000","last_active":"2024-04-24T23:58:20.678540+0000","last_peered":"2024-04-24T23:58:20.678540+0000","last_clean":"2024-04-24T23:58:20.678540+0000","last_became_active":"2024-04-24T23:58:19.260437+0000","last_became_peered":"2024-04-24T23:58:19.260437+0000","last_unstale":"2024-04-24T23:58:20.678540+0000","last_undegraded":"2024-04-24T23:58:20.678540+0000","last_fullsized":"2024-04-24T23:58:20.678540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677754+0000","last_change":"2024-04-24T23:58:19.269955+0000","last_active":"2024-04-24T23:58:20.677754+0000","last_peered":"2024-04-24T23:58:20.677754+0000","last_clean":"2024-04-24T23:58:20.677754+0000","last_became_active":"2024-04-24T23:58:19.269675+0000","last_became_peered":"2024-04-24T23:58:19.269675+0000","last_unstale":"2024-04-24T23:58:20.677754+0000","last_undegraded":"2024-04-24T23:58:20.677754+0000","last_fullsized":"2024-04-24T23:58:20.677754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678346+0000","last_change":"2024-04-24T23:58:19.267752+0000","last_active":"2024-04-24T23:58:20.678346+0000","last_peered":"2024-04-24T23:58:20.678346+0000","last_clean":"2024-04-24T23:58:20.678346+0000","last_became_active":"2024-04-24T23:58:19.267463+0000","last_became_peered":"2024-04-24T23:58:19.267463+0000","last_unstale":"2024-04-24T23:58:20.678346+0000","last_undegraded":"2024-04-24T23:58:20.678346+0000","last_fullsized":"2024-04-24T23:58:20.678346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678319+0000","last_change":"2024-04-24T23:58:19.266736+0000","last_active":"2024-04-24T23:58:20.678319+0000","last_peered":"2024-04-24T23:58:20.678319+0000","last_clean":"2024-04-24T23:58:20.678319+0000","last_became_active":"2024-04-24T23:58:19.266438+0000","last_became_peered":"2024-04-24T23:58:19.266438+0000","last_unstale":"2024-04-24T23:58:20.678319+0000","last_undegraded":"2024-04-24T23:58:20.678319+0000","last_fullsized":"2024-04-24T23:58:20.678319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678538+0000","last_change":"2024-04-24T23:58:19.271086+0000","last_active":"2024-04-24T23:58:20.678538+0000","last_peered":"2024-04-24T23:58:20.678538+0000","last_clean":"2024-04-24T23:58:20.678538+0000","last_became_active":"2024-04-24T23:58:19.270722+0000","last_became_peered":"2024-04-24T23:58:19.270722+0000","last_unstale":"2024-04-24T23:58:20.678538+0000","last_undegraded":"2024-04-24T23:58:20.678538+0000","last_fullsized":"2024-04-24T23:58:20.678538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257102+0000","last_change":"2024-04-24T23:58:19.266845+0000","last_active":"2024-04-24T23:58:20.257102+0000","last_peered":"2024-04-24T23:58:20.257102+0000","last_clean":"2024-04-24T23:58:20.257102+0000","last_became_active":"2024-04-24T23:58:19.266573+0000","last_became_peered":"2024-04-24T23:58:19.266573+0000","last_unstale":"2024-04-24T23:58:20.257102+0000","last_undegraded":"2024-04-24T23:58:20.257102+0000","last_fullsized":"2024-04-24T23:58:20.257102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256863+0000","last_change":"2024-04-24T23:58:19.267588+0000","last_active":"2024-04-24T23:58:20.256863+0000","last_peered":"2024-04-24T23:58:20.256863+0000","last_clean":"2024-04-24T23:58:20.256863+0000","last_became_active":"2024-04-24T23:58:19.267247+0000","last_became_peered":"2024-04-24T23:58:19.267247+0000","last_unstale":"2024-04-24T23:58:20.256863+0000","last_undegraded":"2024-04-24T23:58:20.256863+0000","last_fullsized":"2024-04-24T23:58:20.256863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255357+0000","last_change":"2024-04-24T23:58:19.257310+0000","last_active":"2024-04-24T23:58:20.255357+0000","last_peered":"2024-04-24T23:58:20.255357+0000","last_clean":"2024-04-24T23:58:20.255357+0000","last_became_active":"2024-04-24T23:58:19.257023+0000","last_became_peered":"2024-04-24T23:58:19.257023+0000","last_unstale":"2024-04-24T23:58:20.255357+0000","last_undegraded":"2024-04-24T23:58:20.255357+0000","last_fullsized":"2024-04-24T23:58:20.255357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678413+0000","last_change":"2024-04-24T23:58:19.261091+0000","last_active":"2024-04-24T23:58:20.678413+0000","last_peered":"2024-04-24T23:58:20.678413+0000","last_clean":"2024-04-24T23:58:20.678413+0000","last_became_active":"2024-04-24T23:58:19.260837+0000","last_became_peered":"2024-04-24T23:58:19.260837+0000","last_unstale":"2024-04-24T23:58:20.678413+0000","last_undegraded":"2024-04-24T23:58:20.678413+0000","last_fullsized":"2024-04-24T23:58:20.678413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254811+0000","last_change":"2024-04-24T23:58:19.261328+0000","last_active":"2024-04-24T23:58:20.254811+0000","last_peered":"2024-04-24T23:58:20.254811+0000","last_clean":"2024-04-24T23:58:20.254811+0000","last_became_active":"2024-04-24T23:58:19.261045+0000","last_became_peered":"2024-04-24T23:58:19.261045+0000","last_unstale":"2024-04-24T23:58:20.254811+0000","last_undegraded":"2024-04-24T23:58:20.254811+0000","last_fullsized":"2024-04-24T23:58:20.254811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254583+0000","last_change":"2024-04-24T23:58:19.256356+0000","last_active":"2024-04-24T23:58:20.254583+0000","last_peered":"2024-04-24T23:58:20.254583+0000","last_clean":"2024-04-24T23:58:20.254583+0000","last_became_active":"2024-04-24T23:58:19.256099+0000","last_became_peered":"2024-04-24T23:58:19.256099+0000","last_unstale":"2024-04-24T23:58:20.254583+0000","last_undegraded":"2024-04-24T23:58:20.254583+0000","last_fullsized":"2024-04-24T23:58:20.254583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677420+0000","last_change":"2024-04-24T23:58:19.268419+0000","last_active":"2024-04-24T23:58:20.677420+0000","last_peered":"2024-04-24T23:58:20.677420+0000","last_clean":"2024-04-24T23:58:20.677420+0000","last_became_active":"2024-04-24T23:58:19.265742+0000","last_became_peered":"2024-04-24T23:58:19.265742+0000","last_unstale":"2024-04-24T23:58:20.677420+0000","last_undegraded":"2024-04-24T23:58:20.677420+0000","last_fullsized":"2024-04-24T23:58:20.677420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.273024+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.272551+0000","last_became_peered":"2024-04-24T23:58:19.272551+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.258361+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.258098+0000","last_became_peered":"2024-04-24T23:58:19.258098+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678669+0000","last_change":"2024-04-24T23:58:19.263085+0000","last_active":"2024-04-24T23:58:20.678669+0000","last_peered":"2024-04-24T23:58:20.678669+0000","last_clean":"2024-04-24T23:58:20.678669+0000","last_became_active":"2024-04-24T23:58:19.262807+0000","last_became_peered":"2024-04-24T23:58:19.262807+0000","last_unstale":"2024-04-24T23:58:20.678669+0000","last_undegraded":"2024-04-24T23:58:20.678669+0000","last_fullsized":"2024-04-24T23:58:20.678669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256867+0000","last_change":"2024-04-24T23:58:19.259501+0000","last_active":"2024-04-24T23:58:20.256867+0000","last_peered":"2024-04-24T23:58:20.256867+0000","last_clean":"2024-04-24T23:58:20.256867+0000","last_became_active":"2024-04-24T23:58:19.259233+0000","last_became_peered":"2024-04-24T23:58:19.259233+0000","last_unstale":"2024-04-24T23:58:20.256867+0000","last_undegraded":"2024-04-24T23:58:20.256867+0000","last_fullsized":"2024-04-24T23:58:20.256867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255041+0000","last_change":"2024-04-24T23:58:19.259960+0000","last_active":"2024-04-24T23:58:20.255041+0000","last_peered":"2024-04-24T23:58:20.255041+0000","last_clean":"2024-04-24T23:58:20.255041+0000","last_became_active":"2024-04-24T23:58:19.259548+0000","last_became_peered":"2024-04-24T23:58:19.259548+0000","last_unstale":"2024-04-24T23:58:20.255041+0000","last_undegraded":"2024-04-24T23:58:20.255041+0000","last_fullsized":"2024-04-24T23:58:20.255041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677851+0000","last_change":"2024-04-24T23:58:19.275499+0000","last_active":"2024-04-24T23:58:20.677851+0000","last_peered":"2024-04-24T23:58:20.677851+0000","last_clean":"2024-04-24T23:58:20.677851+0000","last_became_active":"2024-04-24T23:58:19.275213+0000","last_became_peered":"2024-04-24T23:58:19.275213+0000","last_unstale":"2024-04-24T23:58:20.677851+0000","last_undegraded":"2024-04-24T23:58:20.677851+0000","last_fullsized":"2024-04-24T23:58:20.677851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678362+0000","last_change":"2024-04-24T23:58:19.261963+0000","last_active":"2024-04-24T23:58:20.678362+0000","last_peered":"2024-04-24T23:58:20.678362+0000","last_clean":"2024-04-24T23:58:20.678362+0000","last_became_active":"2024-04-24T23:58:19.261710+0000","last_became_peered":"2024-04-24T23:58:19.261710+0000","last_unstale":"2024-04-24T23:58:20.678362+0000","last_undegraded":"2024-04-24T23:58:20.678362+0000","last_fullsized":"2024-04-24T23:58:20.678362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677517+0000","last_change":"2024-04-24T23:58:19.268171+0000","last_active":"2024-04-24T23:58:20.677517+0000","last_peered":"2024-04-24T23:58:20.677517+0000","last_clean":"2024-04-24T23:58:20.677517+0000","last_became_active":"2024-04-24T23:58:19.265379+0000","last_became_peered":"2024-04-24T23:58:19.265379+0000","last_unstale":"2024-04-24T23:58:20.677517+0000","last_undegraded":"2024-04-24T23:58:20.677517+0000","last_fullsized":"2024-04-24T23:58:20.677517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678064+0000","last_change":"2024-04-24T23:58:19.259615+0000","last_active":"2024-04-24T23:58:20.678064+0000","last_peered":"2024-04-24T23:58:20.678064+0000","last_clean":"2024-04-24T23:58:20.678064+0000","last_became_active":"2024-04-24T23:58:19.259355+0000","last_became_peered":"2024-04-24T23:58:19.259355+0000","last_unstale":"2024-04-24T23:58:20.678064+0000","last_undegraded":"2024-04-24T23:58:20.678064+0000","last_fullsized":"2024-04-24T23:58:20.678064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257106+0000","last_change":"2024-04-24T23:58:19.269905+0000","last_active":"2024-04-24T23:58:20.257106+0000","last_peered":"2024-04-24T23:58:20.257106+0000","last_clean":"2024-04-24T23:58:20.257106+0000","last_became_active":"2024-04-24T23:58:19.269639+0000","last_became_peered":"2024-04-24T23:58:19.269639+0000","last_unstale":"2024-04-24T23:58:20.257106+0000","last_undegraded":"2024-04-24T23:58:20.257106+0000","last_fullsized":"2024-04-24T23:58:20.257106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254304+0000","last_change":"2024-04-24T23:58:19.256747+0000","last_active":"2024-04-24T23:58:20.254304+0000","last_peered":"2024-04-24T23:58:20.254304+0000","last_clean":"2024-04-24T23:58:20.254304+0000","last_became_active":"2024-04-24T23:58:19.255723+0000","last_became_peered":"2024-04-24T23:58:19.255723+0000","last_unstale":"2024-04-24T23:58:20.254304+0000","last_undegraded":"2024-04-24T23:58:20.254304+0000","last_fullsized":"2024-04-24T23:58:20.254304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678051+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678051+0000","last_peered":"2024-04-24T23:58:20.678051+0000","last_clean":"2024-04-24T23:58:20.678051+0000","last_became_active":"2024-04-24T23:58:19.262977+0000","last_became_peered":"2024-04-24T23:58:19.262977+0000","last_unstale":"2024-04-24T23:58:20.678051+0000","last_undegraded":"2024-04-24T23:58:20.678051+0000","last_fullsized":"2024-04-24T23:58:20.678051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255177+0000","last_change":"2024-04-24T23:58:19.259745+0000","last_active":"2024-04-24T23:58:20.255177+0000","last_peered":"2024-04-24T23:58:20.255177+0000","last_clean":"2024-04-24T23:58:20.255177+0000","last_became_active":"2024-04-24T23:58:19.259201+0000","last_became_peered":"2024-04-24T23:58:19.259201+0000","last_unstale":"2024-04-24T23:58:20.255177+0000","last_undegraded":"2024-04-24T23:58:20.255177+0000","last_fullsized":"2024-04-24T23:58:20.255177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678071+0000","last_change":"2024-04-24T23:58:19.273016+0000","last_active":"2024-04-24T23:58:20.678071+0000","last_peered":"2024-04-24T23:58:20.678071+0000","last_clean":"2024-04-24T23:58:20.678071+0000","last_became_active":"2024-04-24T23:58:19.271540+0000","last_became_peered":"2024-04-24T23:58:19.271540+0000","last_unstale":"2024-04-24T23:58:20.678071+0000","last_undegraded":"2024-04-24T23:58:20.678071+0000","last_fullsized":"2024-04-24T23:58:20.678071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677646+0000","last_change":"2024-04-24T23:58:19.255458+0000","last_active":"2024-04-24T23:58:20.677646+0000","last_peered":"2024-04-24T23:58:20.677646+0000","last_clean":"2024-04-24T23:58:20.677646+0000","last_became_active":"2024-04-24T23:58:19.255070+0000","last_became_peered":"2024-04-24T23:58:19.255070+0000","last_unstale":"2024-04-24T23:58:20.677646+0000","last_undegraded":"2024-04-24T23:58:20.677646+0000","last_fullsized":"2024-04-24T23:58:20.677646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256632+0000","last_change":"2024-04-24T23:58:19.266657+0000","last_active":"2024-04-24T23:58:20.256632+0000","last_peered":"2024-04-24T23:58:20.256632+0000","last_clean":"2024-04-24T23:58:20.256632+0000","last_became_active":"2024-04-24T23:58:19.266354+0000","last_became_peered":"2024-04-24T23:58:19.266354+0000","last_unstale":"2024-04-24T23:58:20.256632+0000","last_undegraded":"2024-04-24T23:58:20.256632+0000","last_fullsized":"2024-04-24T23:58:20.256632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255049+0000","last_change":"2024-04-24T23:58:19.259253+0000","last_active":"2024-04-24T23:58:20.255049+0000","last_peered":"2024-04-24T23:58:20.255049+0000","last_clean":"2024-04-24T23:58:20.255049+0000","last_became_active":"2024-04-24T23:58:19.258553+0000","last_became_peered":"2024-04-24T23:58:19.258553+0000","last_unstale":"2024-04-24T23:58:20.255049+0000","last_undegraded":"2024-04-24T23:58:20.255049+0000","last_fullsized":"2024-04-24T23:58:20.255049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:53.284799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678523+0000","last_change":"2024-04-24T23:58:18.250614+0000","last_active":"2024-04-24T23:58:20.678523+0000","last_peered":"2024-04-24T23:58:20.678523+0000","last_clean":"2024-04-24T23:58:20.678523+0000","last_became_active":"2024-04-24T23:58:18.250306+0000","last_became_peered":"2024-04-24T23:58:18.250306+0000","last_unstale":"2024-04-24T23:58:20.678523+0000","last_undegraded":"2024-04-24T23:58:20.678523+0000","last_fullsized":"2024-04-24T23:58:20.678523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255184+0000","last_change":"2024-04-24T23:58:19.251648+0000","last_active":"2024-04-24T23:58:20.255184+0000","last_peered":"2024-04-24T23:58:20.255184+0000","last_clean":"2024-04-24T23:58:20.255184+0000","last_became_active":"2024-04-24T23:58:19.251271+0000","last_became_peered":"2024-04-24T23:58:19.251271+0000","last_unstale":"2024-04-24T23:58:20.255184+0000","last_undegraded":"2024-04-24T23:58:20.255184+0000","last_fullsized":"2024-04-24T23:58:20.255184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256708+0000","last_change":"2024-04-24T23:58:18.252764+0000","last_active":"2024-04-24T23:58:20.256708+0000","last_peered":"2024-04-24T23:58:20.256708+0000","last_clean":"2024-04-24T23:58:20.256708+0000","last_became_active":"2024-04-24T23:58:18.252491+0000","last_became_peered":"2024-04-24T23:58:18.252491+0000","last_unstale":"2024-04-24T23:58:20.256708+0000","last_undegraded":"2024-04-24T23:58:20.256708+0000","last_fullsized":"2024-04-24T23:58:20.256708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678357+0000","last_change":"2024-04-24T23:58:18.250003+0000","last_active":"2024-04-24T23:58:20.678357+0000","last_peered":"2024-04-24T23:58:20.678357+0000","last_clean":"2024-04-24T23:58:20.678357+0000","last_became_active":"2024-04-24T23:58:18.249727+0000","last_became_peered":"2024-04-24T23:58:18.249727+0000","last_unstale":"2024-04-24T23:58:20.678357+0000","last_undegraded":"2024-04-24T23:58:20.678357+0000","last_fullsized":"2024-04-24T23:58:20.678357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.318157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678399+0000","last_change":"2024-04-24T23:58:19.255033+0000","last_active":"2024-04-24T23:58:20.678399+0000","last_peered":"2024-04-24T23:58:20.678399+0000","last_clean":"2024-04-24T23:58:20.678399+0000","last_became_active":"2024-04-24T23:58:19.254746+0000","last_became_peered":"2024-04-24T23:58:19.254746+0000","last_unstale":"2024-04-24T23:58:20.678399+0000","last_undegraded":"2024-04-24T23:58:20.678399+0000","last_fullsized":"2024-04-24T23:58:20.678399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677538+0000","last_change":"2024-04-24T23:58:18.245516+0000","last_active":"2024-04-24T23:58:20.677538+0000","last_peered":"2024-04-24T23:58:20.677538+0000","last_clean":"2024-04-24T23:58:20.677538+0000","last_became_active":"2024-04-24T23:58:18.245212+0000","last_became_peered":"2024-04-24T23:58:18.245212+0000","last_unstale":"2024-04-24T23:58:20.677538+0000","last_undegraded":"2024-04-24T23:58:20.677538+0000","last_fullsized":"2024-04-24T23:58:20.677538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254479+0000","last_change":"2024-04-24T23:58:19.258354+0000","last_active":"2024-04-24T23:58:20.254479+0000","last_peered":"2024-04-24T23:58:20.254479+0000","last_clean":"2024-04-24T23:58:20.254479+0000","last_became_active":"2024-04-24T23:58:19.258067+0000","last_became_peered":"2024-04-24T23:58:19.258067+0000","last_unstale":"2024-04-24T23:58:20.254479+0000","last_undegraded":"2024-04-24T23:58:20.254479+0000","last_fullsized":"2024-04-24T23:58:20.254479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256985+0000","last_change":"2024-04-24T23:58:18.253806+0000","last_active":"2024-04-24T23:58:20.256985+0000","last_peered":"2024-04-24T23:58:20.256985+0000","last_clean":"2024-04-24T23:58:20.256985+0000","last_became_active":"2024-04-24T23:58:18.252657+0000","last_became_peered":"2024-04-24T23:58:18.252657+0000","last_unstale":"2024-04-24T23:58:20.256985+0000","last_undegraded":"2024-04-24T23:58:20.256985+0000","last_fullsized":"2024-04-24T23:58:20.256985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677952+0000","last_change":"2024-04-24T23:58:19.275012+0000","last_active":"2024-04-24T23:58:20.677952+0000","last_peered":"2024-04-24T23:58:20.677952+0000","last_clean":"2024-04-24T23:58:20.677952+0000","last_became_active":"2024-04-24T23:58:19.274740+0000","last_became_peered":"2024-04-24T23:58:19.274740+0000","last_unstale":"2024-04-24T23:58:20.677952+0000","last_undegraded":"2024-04-24T23:58:20.677952+0000","last_fullsized":"2024-04-24T23:58:20.677952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678001+0000","last_change":"2024-04-24T23:58:18.251763+0000","last_active":"2024-04-24T23:58:20.678001+0000","last_peered":"2024-04-24T23:58:20.678001+0000","last_clean":"2024-04-24T23:58:20.678001+0000","last_became_active":"2024-04-24T23:58:18.251508+0000","last_became_peered":"2024-04-24T23:58:18.251508+0000","last_unstale":"2024-04-24T23:58:20.678001+0000","last_undegraded":"2024-04-24T23:58:20.678001+0000","last_fullsized":"2024-04-24T23:58:20.678001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254979+0000","last_change":"2024-04-24T23:58:19.253275+0000","last_active":"2024-04-24T23:58:20.254979+0000","last_peered":"2024-04-24T23:58:20.254979+0000","last_clean":"2024-04-24T23:58:20.254979+0000","last_became_active":"2024-04-24T23:58:19.252783+0000","last_became_peered":"2024-04-24T23:58:19.252783+0000","last_unstale":"2024-04-24T23:58:20.254979+0000","last_undegraded":"2024-04-24T23:58:20.254979+0000","last_fullsized":"2024-04-24T23:58:20.254979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254408+0000","last_change":"2024-04-24T23:58:18.246446+0000","last_active":"2024-04-24T23:58:20.254408+0000","last_peered":"2024-04-24T23:58:20.254408+0000","last_clean":"2024-04-24T23:58:20.254408+0000","last_became_active":"2024-04-24T23:58:18.246036+0000","last_became_peered":"2024-04-24T23:58:18.246036+0000","last_unstale":"2024-04-24T23:58:20.254408+0000","last_undegraded":"2024-04-24T23:58:20.254408+0000","last_fullsized":"2024-04-24T23:58:20.254408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255261+0000","last_change":"2024-04-24T23:58:19.261655+0000","last_active":"2024-04-24T23:58:20.255261+0000","last_peered":"2024-04-24T23:58:20.255261+0000","last_clean":"2024-04-24T23:58:20.255261+0000","last_became_active":"2024-04-24T23:58:19.261331+0000","last_became_peered":"2024-04-24T23:58:19.261331+0000","last_unstale":"2024-04-24T23:58:20.255261+0000","last_undegraded":"2024-04-24T23:58:20.255261+0000","last_fullsized":"2024-04-24T23:58:20.255261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256970+0000","last_change":"2024-04-24T23:58:18.247264+0000","last_active":"2024-04-24T23:58:20.256970+0000","last_peered":"2024-04-24T23:58:20.256970+0000","last_clean":"2024-04-24T23:58:20.256970+0000","last_became_active":"2024-04-24T23:58:18.246959+0000","last_became_peered":"2024-04-24T23:58:18.246959+0000","last_unstale":"2024-04-24T23:58:20.256970+0000","last_undegraded":"2024-04-24T23:58:20.256970+0000","last_fullsized":"2024-04-24T23:58:20.256970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678015+0000","last_change":"2024-04-24T23:58:19.265255+0000","last_active":"2024-04-24T23:58:20.678015+0000","last_peered":"2024-04-24T23:58:20.678015+0000","last_clean":"2024-04-24T23:58:20.678015+0000","last_became_active":"2024-04-24T23:58:19.264997+0000","last_became_peered":"2024-04-24T23:58:19.264997+0000","last_unstale":"2024-04-24T23:58:20.678015+0000","last_undegraded":"2024-04-24T23:58:20.678015+0000","last_fullsized":"2024-04-24T23:58:20.678015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255275+0000","last_change":"2024-04-24T23:58:18.236929+0000","last_active":"2024-04-24T23:58:20.255275+0000","last_peered":"2024-04-24T23:58:20.255275+0000","last_clean":"2024-04-24T23:58:20.255275+0000","last_became_active":"2024-04-24T23:58:18.236660+0000","last_became_peered":"2024-04-24T23:58:18.236660+0000","last_unstale":"2024-04-24T23:58:20.255275+0000","last_undegraded":"2024-04-24T23:58:20.255275+0000","last_fullsized":"2024-04-24T23:58:20.255275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254944+0000","last_change":"2024-04-24T23:58:19.259532+0000","last_active":"2024-04-24T23:58:20.254944+0000","last_peered":"2024-04-24T23:58:20.254944+0000","last_clean":"2024-04-24T23:58:20.254944+0000","last_became_active":"2024-04-24T23:58:19.259298+0000","last_became_peered":"2024-04-24T23:58:19.259298+0000","last_unstale":"2024-04-24T23:58:20.254944+0000","last_undegraded":"2024-04-24T23:58:20.254944+0000","last_fullsized":"2024-04-24T23:58:20.254944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678194+0000","last_change":"2024-04-24T23:58:18.251116+0000","last_active":"2024-04-24T23:58:20.678194+0000","last_peered":"2024-04-24T23:58:20.678194+0000","last_clean":"2024-04-24T23:58:20.678194+0000","last_became_active":"2024-04-24T23:58:18.250814+0000","last_became_peered":"2024-04-24T23:58:18.250814+0000","last_unstale":"2024-04-24T23:58:20.678194+0000","last_undegraded":"2024-04-24T23:58:20.678194+0000","last_fullsized":"2024-04-24T23:58:20.678194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678564+0000","last_change":"2024-04-24T23:58:19.263724+0000","last_active":"2024-04-24T23:58:20.678564+0000","last_peered":"2024-04-24T23:58:20.678564+0000","last_clean":"2024-04-24T23:58:20.678564+0000","last_became_active":"2024-04-24T23:58:19.263448+0000","last_became_peered":"2024-04-24T23:58:19.263448+0000","last_unstale":"2024-04-24T23:58:20.678564+0000","last_undegraded":"2024-04-24T23:58:20.678564+0000","last_fullsized":"2024-04-24T23:58:20.678564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256648+0000","last_change":"2024-04-24T23:58:18.249529+0000","last_active":"2024-04-24T23:58:20.256648+0000","last_peered":"2024-04-24T23:58:20.256648+0000","last_clean":"2024-04-24T23:58:20.256648+0000","last_became_active":"2024-04-24T23:58:18.249135+0000","last_became_peered":"2024-04-24T23:58:18.249135+0000","last_unstale":"2024-04-24T23:58:20.256648+0000","last_undegraded":"2024-04-24T23:58:20.256648+0000","last_fullsized":"2024-04-24T23:58:20.256648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256714+0000","last_change":"2024-04-24T23:58:19.266083+0000","last_active":"2024-04-24T23:58:20.256714+0000","last_peered":"2024-04-24T23:58:20.256714+0000","last_clean":"2024-04-24T23:58:20.256714+0000","last_became_active":"2024-04-24T23:58:19.265757+0000","last_became_peered":"2024-04-24T23:58:19.265757+0000","last_unstale":"2024-04-24T23:58:20.256714+0000","last_undegraded":"2024-04-24T23:58:20.256714+0000","last_fullsized":"2024-04-24T23:58:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677645+0000","last_change":"2024-04-24T23:58:18.245808+0000","last_active":"2024-04-24T23:58:20.677645+0000","last_peered":"2024-04-24T23:58:20.677645+0000","last_clean":"2024-04-24T23:58:20.677645+0000","last_became_active":"2024-04-24T23:58:18.244769+0000","last_became_peered":"2024-04-24T23:58:18.244769+0000","last_unstale":"2024-04-24T23:58:20.677645+0000","last_undegraded":"2024-04-24T23:58:20.677645+0000","last_fullsized":"2024-04-24T23:58:20.677645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678393+0000","last_change":"2024-04-24T23:58:19.262773+0000","last_active":"2024-04-24T23:58:20.678393+0000","last_peered":"2024-04-24T23:58:20.678393+0000","last_clean":"2024-04-24T23:58:20.678393+0000","last_became_active":"2024-04-24T23:58:19.262384+0000","last_became_peered":"2024-04-24T23:58:19.262384+0000","last_unstale":"2024-04-24T23:58:20.678393+0000","last_undegraded":"2024-04-24T23:58:20.678393+0000","last_fullsized":"2024-04-24T23:58:20.678393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255516+0000","last_change":"2024-04-24T23:58:18.243528+0000","last_active":"2024-04-24T23:58:20.255516+0000","last_peered":"2024-04-24T23:58:20.255516+0000","last_clean":"2024-04-24T23:58:20.255516+0000","last_became_active":"2024-04-24T23:58:18.243262+0000","last_became_peered":"2024-04-24T23:58:18.243262+0000","last_unstale":"2024-04-24T23:58:20.255516+0000","last_undegraded":"2024-04-24T23:58:20.255516+0000","last_fullsized":"2024-04-24T23:58:20.255516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677877+0000","last_change":"2024-04-24T23:58:19.261251+0000","last_active":"2024-04-24T23:58:20.677877+0000","last_peered":"2024-04-24T23:58:20.677877+0000","last_clean":"2024-04-24T23:58:20.677877+0000","last_became_active":"2024-04-24T23:58:19.260977+0000","last_became_peered":"2024-04-24T23:58:19.260977+0000","last_unstale":"2024-04-24T23:58:20.677877+0000","last_undegraded":"2024-04-24T23:58:20.677877+0000","last_fullsized":"2024-04-24T23:58:20.677877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677909+0000","last_change":"2024-04-24T23:58:18.242128+0000","last_active":"2024-04-24T23:58:20.677909+0000","last_peered":"2024-04-24T23:58:20.677909+0000","last_clean":"2024-04-24T23:58:20.677909+0000","last_became_active":"2024-04-24T23:58:18.241804+0000","last_became_peered":"2024-04-24T23:58:18.241804+0000","last_unstale":"2024-04-24T23:58:20.677909+0000","last_undegraded":"2024-04-24T23:58:20.677909+0000","last_fullsized":"2024-04-24T23:58:20.677909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678616+0000","last_change":"2024-04-24T23:58:19.262407+0000","last_active":"2024-04-24T23:58:20.678616+0000","last_peered":"2024-04-24T23:58:20.678616+0000","last_clean":"2024-04-24T23:58:20.678616+0000","last_became_active":"2024-04-24T23:58:19.262123+0000","last_became_peered":"2024-04-24T23:58:19.262123+0000","last_unstale":"2024-04-24T23:58:20.678616+0000","last_undegraded":"2024-04-24T23:58:20.678616+0000","last_fullsized":"2024-04-24T23:58:20.678616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254597+0000","last_change":"2024-04-24T23:58:18.242601+0000","last_active":"2024-04-24T23:58:20.254597+0000","last_peered":"2024-04-24T23:58:20.254597+0000","last_clean":"2024-04-24T23:58:20.254597+0000","last_became_active":"2024-04-24T23:58:18.242314+0000","last_became_peered":"2024-04-24T23:58:18.242314+0000","last_unstale":"2024-04-24T23:58:20.254597+0000","last_undegraded":"2024-04-24T23:58:20.254597+0000","last_fullsized":"2024-04-24T23:58:20.254597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678172+0000","last_change":"2024-04-24T23:58:19.265899+0000","last_active":"2024-04-24T23:58:20.678172+0000","last_peered":"2024-04-24T23:58:20.678172+0000","last_clean":"2024-04-24T23:58:20.678172+0000","last_became_active":"2024-04-24T23:58:19.265596+0000","last_became_peered":"2024-04-24T23:58:19.265596+0000","last_unstale":"2024-04-24T23:58:20.678172+0000","last_undegraded":"2024-04-24T23:58:20.678172+0000","last_fullsized":"2024-04-24T23:58:20.678172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677741+0000","last_change":"2024-04-24T23:58:18.248287+0000","last_active":"2024-04-24T23:58:20.677741+0000","last_peered":"2024-04-24T23:58:20.677741+0000","last_clean":"2024-04-24T23:58:20.677741+0000","last_became_active":"2024-04-24T23:58:18.248034+0000","last_became_peered":"2024-04-24T23:58:18.248034+0000","last_unstale":"2024-04-24T23:58:20.677741+0000","last_undegraded":"2024-04-24T23:58:20.677741+0000","last_fullsized":"2024-04-24T23:58:20.677741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678510+0000","last_change":"2024-04-24T23:58:19.274141+0000","last_active":"2024-04-24T23:58:20.678510+0000","last_peered":"2024-04-24T23:58:20.678510+0000","last_clean":"2024-04-24T23:58:20.678510+0000","last_became_active":"2024-04-24T23:58:19.273864+0000","last_became_peered":"2024-04-24T23:58:19.273864+0000","last_unstale":"2024-04-24T23:58:20.678510+0000","last_undegraded":"2024-04-24T23:58:20.678510+0000","last_fullsized":"2024-04-24T23:58:20.678510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677425+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:58:20.677425+0000","last_peered":"2024-04-24T23:58:20.677425+0000","last_clean":"2024-04-24T23:58:20.677425+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:58:20.677425+0000","last_undegraded":"2024-04-24T23:58:20.677425+0000","last_fullsized":"2024-04-24T23:58:20.677425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677567+0000","last_change":"2024-04-24T23:58:18.247362+0000","last_active":"2024-04-24T23:58:20.677567+0000","last_peered":"2024-04-24T23:58:20.677567+0000","last_clean":"2024-04-24T23:58:20.677567+0000","last_became_active":"2024-04-24T23:58:18.247084+0000","last_became_peered":"2024-04-24T23:58:18.247084+0000","last_unstale":"2024-04-24T23:58:20.677567+0000","last_undegraded":"2024-04-24T23:58:20.677567+0000","last_fullsized":"2024-04-24T23:58:20.677567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678143+0000","last_change":"2024-04-24T23:58:19.272925+0000","last_active":"2024-04-24T23:58:20.678143+0000","last_peered":"2024-04-24T23:58:20.678143+0000","last_clean":"2024-04-24T23:58:20.678143+0000","last_became_active":"2024-04-24T23:58:19.272538+0000","last_became_peered":"2024-04-24T23:58:19.272538+0000","last_unstale":"2024-04-24T23:58:20.678143+0000","last_undegraded":"2024-04-24T23:58:20.678143+0000","last_fullsized":"2024-04-24T23:58:20.678143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677369+0000","last_change":"2024-04-24T23:58:18.251207+0000","last_active":"2024-04-24T23:58:20.677369+0000","last_peered":"2024-04-24T23:58:20.677369+0000","last_clean":"2024-04-24T23:58:20.677369+0000","last_became_active":"2024-04-24T23:58:18.250921+0000","last_became_peered":"2024-04-24T23:58:18.250921+0000","last_unstale":"2024-04-24T23:58:20.677369+0000","last_undegraded":"2024-04-24T23:58:20.677369+0000","last_fullsized":"2024-04-24T23:58:20.677369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254623+0000","last_change":"2024-04-24T23:58:19.248221+0000","last_active":"2024-04-24T23:58:20.254623+0000","last_peered":"2024-04-24T23:58:20.254623+0000","last_clean":"2024-04-24T23:58:20.254623+0000","last_became_active":"2024-04-24T23:58:19.247896+0000","last_became_peered":"2024-04-24T23:58:19.247896+0000","last_unstale":"2024-04-24T23:58:20.254623+0000","last_undegraded":"2024-04-24T23:58:20.254623+0000","last_fullsized":"2024-04-24T23:58:20.254623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:24.816499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677678+0000","last_change":"2024-04-24T23:58:18.245116+0000","last_active":"2024-04-24T23:58:20.677678+0000","last_peered":"2024-04-24T23:58:20.677678+0000","last_clean":"2024-04-24T23:58:20.677678+0000","last_became_active":"2024-04-24T23:58:18.244779+0000","last_became_peered":"2024-04-24T23:58:18.244779+0000","last_unstale":"2024-04-24T23:58:20.677678+0000","last_undegraded":"2024-04-24T23:58:20.677678+0000","last_fullsized":"2024-04-24T23:58:20.677678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254292+0000","last_change":"2024-04-24T23:58:19.258623+0000","last_active":"2024-04-24T23:58:20.254292+0000","last_peered":"2024-04-24T23:58:20.254292+0000","last_clean":"2024-04-24T23:58:20.254292+0000","last_became_active":"2024-04-24T23:58:19.258371+0000","last_became_peered":"2024-04-24T23:58:19.258371+0000","last_unstale":"2024-04-24T23:58:20.254292+0000","last_undegraded":"2024-04-24T23:58:20.254292+0000","last_fullsized":"2024-04-24T23:58:20.254292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678304+0000","last_change":"2024-04-24T23:58:18.250837+0000","last_active":"2024-04-24T23:58:20.678304+0000","last_peered":"2024-04-24T23:58:20.678304+0000","last_clean":"2024-04-24T23:58:20.678304+0000","last_became_active":"2024-04-24T23:58:18.250540+0000","last_became_peered":"2024-04-24T23:58:18.250540+0000","last_unstale":"2024-04-24T23:58:20.678304+0000","last_undegraded":"2024-04-24T23:58:20.678304+0000","last_fullsized":"2024-04-24T23:58:20.678304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:13.906429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255478+0000","last_change":"2024-04-24T23:58:19.254379+0000","last_active":"2024-04-24T23:58:20.255478+0000","last_peered":"2024-04-24T23:58:20.255478+0000","last_clean":"2024-04-24T23:58:20.255478+0000","last_became_active":"2024-04-24T23:58:19.252734+0000","last_became_peered":"2024-04-24T23:58:19.252734+0000","last_unstale":"2024-04-24T23:58:20.255478+0000","last_undegraded":"2024-04-24T23:58:20.255478+0000","last_fullsized":"2024-04-24T23:58:20.255478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:59.970682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254377+0000","last_change":"2024-04-24T23:58:18.247304+0000","last_active":"2024-04-24T23:58:20.254377+0000","last_peered":"2024-04-24T23:58:20.254377+0000","last_clean":"2024-04-24T23:58:20.254377+0000","last_became_active":"2024-04-24T23:58:18.247029+0000","last_became_peered":"2024-04-24T23:58:18.247029+0000","last_unstale":"2024-04-24T23:58:20.254377+0000","last_undegraded":"2024-04-24T23:58:20.254377+0000","last_fullsized":"2024-04-24T23:58:20.254377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678233+0000","last_change":"2024-04-24T23:58:19.268921+0000","last_active":"2024-04-24T23:58:20.678233+0000","last_peered":"2024-04-24T23:58:20.678233+0000","last_clean":"2024-04-24T23:58:20.678233+0000","last_became_active":"2024-04-24T23:58:19.265287+0000","last_became_peered":"2024-04-24T23:58:19.265287+0000","last_unstale":"2024-04-24T23:58:20.678233+0000","last_undegraded":"2024-04-24T23:58:20.678233+0000","last_fullsized":"2024-04-24T23:58:20.678233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678625+0000","last_change":"2024-04-24T23:58:18.249153+0000","last_active":"2024-04-24T23:58:20.678625+0000","last_peered":"2024-04-24T23:58:20.678625+0000","last_clean":"2024-04-24T23:58:20.678625+0000","last_became_active":"2024-04-24T23:58:18.248532+0000","last_became_peered":"2024-04-24T23:58:18.248532+0000","last_unstale":"2024-04-24T23:58:20.678625+0000","last_undegraded":"2024-04-24T23:58:20.678625+0000","last_fullsized":"2024-04-24T23:58:20.678625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254293+0000","last_change":"2024-04-24T23:58:19.257617+0000","last_active":"2024-04-24T23:58:20.254293+0000","last_peered":"2024-04-24T23:58:20.254293+0000","last_clean":"2024-04-24T23:58:20.254293+0000","last_became_active":"2024-04-24T23:58:19.257312+0000","last_became_peered":"2024-04-24T23:58:19.257312+0000","last_unstale":"2024-04-24T23:58:20.254293+0000","last_undegraded":"2024-04-24T23:58:20.254293+0000","last_fullsized":"2024-04-24T23:58:20.254293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256119+0000","last_change":"2024-04-24T23:58:18.249582+0000","last_active":"2024-04-24T23:58:20.256119+0000","last_peered":"2024-04-24T23:58:20.256119+0000","last_clean":"2024-04-24T23:58:20.256119+0000","last_became_active":"2024-04-24T23:58:18.249277+0000","last_became_peered":"2024-04-24T23:58:18.249277+0000","last_unstale":"2024-04-24T23:58:20.256119+0000","last_undegraded":"2024-04-24T23:58:20.256119+0000","last_fullsized":"2024-04-24T23:58:20.256119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678235+0000","last_change":"2024-04-24T23:58:19.265003+0000","last_active":"2024-04-24T23:58:20.678235+0000","last_peered":"2024-04-24T23:58:20.678235+0000","last_clean":"2024-04-24T23:58:20.678235+0000","last_became_active":"2024-04-24T23:58:19.264714+0000","last_became_peered":"2024-04-24T23:58:19.264714+0000","last_unstale":"2024-04-24T23:58:20.678235+0000","last_undegraded":"2024-04-24T23:58:20.678235+0000","last_fullsized":"2024-04-24T23:58:20.678235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257040+0000","last_change":"2024-04-24T23:58:18.250190+0000","last_active":"2024-04-24T23:58:20.257040+0000","last_peered":"2024-04-24T23:58:20.257040+0000","last_clean":"2024-04-24T23:58:20.257040+0000","last_became_active":"2024-04-24T23:58:18.249896+0000","last_became_peered":"2024-04-24T23:58:18.249896+0000","last_unstale":"2024-04-24T23:58:20.257040+0000","last_undegraded":"2024-04-24T23:58:20.257040+0000","last_fullsized":"2024-04-24T23:58:20.257040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255362+0000","last_change":"2024-04-24T23:58:19.261332+0000","last_active":"2024-04-24T23:58:20.255362+0000","last_peered":"2024-04-24T23:58:20.255362+0000","last_clean":"2024-04-24T23:58:20.255362+0000","last_became_active":"2024-04-24T23:58:19.261058+0000","last_became_peered":"2024-04-24T23:58:19.261058+0000","last_unstale":"2024-04-24T23:58:20.255362+0000","last_undegraded":"2024-04-24T23:58:20.255362+0000","last_fullsized":"2024-04-24T23:58:20.255362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677359+0000","last_change":"2024-04-24T23:58:18.251910+0000","last_active":"2024-04-24T23:58:20.677359+0000","last_peered":"2024-04-24T23:58:20.677359+0000","last_clean":"2024-04-24T23:58:20.677359+0000","last_became_active":"2024-04-24T23:58:18.251629+0000","last_became_peered":"2024-04-24T23:58:18.251629+0000","last_unstale":"2024-04-24T23:58:20.677359+0000","last_undegraded":"2024-04-24T23:58:20.677359+0000","last_fullsized":"2024-04-24T23:58:20.677359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255236+0000","last_change":"2024-04-24T23:58:19.256009+0000","last_active":"2024-04-24T23:58:20.255236+0000","last_peered":"2024-04-24T23:58:20.255236+0000","last_clean":"2024-04-24T23:58:20.255236+0000","last_became_active":"2024-04-24T23:58:19.255734+0000","last_became_peered":"2024-04-24T23:58:19.255734+0000","last_unstale":"2024-04-24T23:58:20.255236+0000","last_undegraded":"2024-04-24T23:58:20.255236+0000","last_fullsized":"2024-04-24T23:58:20.255236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:49.658803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254568+0000","last_change":"2024-04-24T23:58:18.250223+0000","last_active":"2024-04-24T23:58:20.254568+0000","last_peered":"2024-04-24T23:58:20.254568+0000","last_clean":"2024-04-24T23:58:20.254568+0000","last_became_active":"2024-04-24T23:58:18.249995+0000","last_became_peered":"2024-04-24T23:58:18.249995+0000","last_unstale":"2024-04-24T23:58:20.254568+0000","last_undegraded":"2024-04-24T23:58:20.254568+0000","last_fullsized":"2024-04-24T23:58:20.254568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.258258+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.258011+0000","last_became_peered":"2024-04-24T23:58:19.258011+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254356+0000","last_change":"2024-04-24T23:58:18.243148+0000","last_active":"2024-04-24T23:58:20.254356+0000","last_peered":"2024-04-24T23:58:20.254356+0000","last_clean":"2024-04-24T23:58:20.254356+0000","last_became_active":"2024-04-24T23:58:18.242863+0000","last_became_peered":"2024-04-24T23:58:18.242863+0000","last_unstale":"2024-04-24T23:58:20.254356+0000","last_undegraded":"2024-04-24T23:58:20.254356+0000","last_fullsized":"2024-04-24T23:58:20.254356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678415+0000","last_change":"2024-04-24T23:58:19.268382+0000","last_active":"2024-04-24T23:58:20.678415+0000","last_peered":"2024-04-24T23:58:20.678415+0000","last_clean":"2024-04-24T23:58:20.678415+0000","last_became_active":"2024-04-24T23:58:19.268132+0000","last_became_peered":"2024-04-24T23:58:19.268132+0000","last_unstale":"2024-04-24T23:58:20.678415+0000","last_undegraded":"2024-04-24T23:58:20.678415+0000","last_fullsized":"2024-04-24T23:58:20.678415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678646+0000","last_change":"2024-04-24T23:58:18.253346+0000","last_active":"2024-04-24T23:58:20.678646+0000","last_peered":"2024-04-24T23:58:20.678646+0000","last_clean":"2024-04-24T23:58:20.678646+0000","last_became_active":"2024-04-24T23:58:18.253094+0000","last_became_peered":"2024-04-24T23:58:18.253094+0000","last_unstale":"2024-04-24T23:58:20.678646+0000","last_undegraded":"2024-04-24T23:58:20.678646+0000","last_fullsized":"2024-04-24T23:58:20.678646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256888+0000","last_change":"2024-04-24T23:58:19.260209+0000","last_active":"2024-04-24T23:58:20.256888+0000","last_peered":"2024-04-24T23:58:20.256888+0000","last_clean":"2024-04-24T23:58:20.256888+0000","last_became_active":"2024-04-24T23:58:19.259941+0000","last_became_peered":"2024-04-24T23:58:19.259941+0000","last_unstale":"2024-04-24T23:58:20.256888+0000","last_undegraded":"2024-04-24T23:58:20.256888+0000","last_fullsized":"2024-04-24T23:58:20.256888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678038+0000","last_change":"2024-04-24T23:58:19.262656+0000","last_active":"2024-04-24T23:58:20.678038+0000","last_peered":"2024-04-24T23:58:20.678038+0000","last_clean":"2024-04-24T23:58:20.678038+0000","last_became_active":"2024-04-24T23:58:19.262043+0000","last_became_peered":"2024-04-24T23:58:19.262043+0000","last_unstale":"2024-04-24T23:58:20.678038+0000","last_undegraded":"2024-04-24T23:58:20.678038+0000","last_fullsized":"2024-04-24T23:58:20.678038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677720+0000","last_change":"2024-04-24T23:58:18.249958+0000","last_active":"2024-04-24T23:58:20.677720+0000","last_peered":"2024-04-24T23:58:20.677720+0000","last_clean":"2024-04-24T23:58:20.677720+0000","last_became_active":"2024-04-24T23:58:18.249660+0000","last_became_peered":"2024-04-24T23:58:18.249660+0000","last_unstale":"2024-04-24T23:58:20.677720+0000","last_undegraded":"2024-04-24T23:58:20.677720+0000","last_fullsized":"2024-04-24T23:58:20.677720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678297+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678297+0000","last_peered":"2024-04-24T23:58:20.678297+0000","last_clean":"2024-04-24T23:58:20.678297+0000","last_became_active":"2024-04-24T23:58:19.262976+0000","last_became_peered":"2024-04-24T23:58:19.262976+0000","last_unstale":"2024-04-24T23:58:20.678297+0000","last_undegraded":"2024-04-24T23:58:20.678297+0000","last_fullsized":"2024-04-24T23:58:20.678297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678466+0000","last_change":"2024-04-24T23:58:18.250117+0000","last_active":"2024-04-24T23:58:20.678466+0000","last_peered":"2024-04-24T23:58:20.678466+0000","last_clean":"2024-04-24T23:58:20.678466+0000","last_became_active":"2024-04-24T23:58:18.249797+0000","last_became_peered":"2024-04-24T23:58:18.249797+0000","last_unstale":"2024-04-24T23:58:20.678466+0000","last_undegraded":"2024-04-24T23:58:20.678466+0000","last_fullsized":"2024-04-24T23:58:20.678466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254736+0000","last_change":"2024-04-24T23:58:19.253506+0000","last_active":"2024-04-24T23:58:20.254736+0000","last_peered":"2024-04-24T23:58:20.254736+0000","last_clean":"2024-04-24T23:58:20.254736+0000","last_became_active":"2024-04-24T23:58:19.253032+0000","last_became_peered":"2024-04-24T23:58:19.253032+0000","last_unstale":"2024-04-24T23:58:20.254736+0000","last_undegraded":"2024-04-24T23:58:20.254736+0000","last_fullsized":"2024-04-24T23:58:20.254736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:14.910044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678152+0000","last_change":"2024-04-24T23:58:18.244986+0000","last_active":"2024-04-24T23:58:20.678152+0000","last_peered":"2024-04-24T23:58:20.678152+0000","last_clean":"2024-04-24T23:58:20.678152+0000","last_became_active":"2024-04-24T23:58:18.244674+0000","last_became_peered":"2024-04-24T23:58:18.244674+0000","last_unstale":"2024-04-24T23:58:20.678152+0000","last_undegraded":"2024-04-24T23:58:20.678152+0000","last_fullsized":"2024-04-24T23:58:20.678152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678003+0000","last_change":"2024-04-24T23:58:19.261722+0000","last_active":"2024-04-24T23:58:20.678003+0000","last_peered":"2024-04-24T23:58:20.678003+0000","last_clean":"2024-04-24T23:58:20.678003+0000","last_became_active":"2024-04-24T23:58:19.261474+0000","last_became_peered":"2024-04-24T23:58:19.261474+0000","last_unstale":"2024-04-24T23:58:20.678003+0000","last_undegraded":"2024-04-24T23:58:20.678003+0000","last_fullsized":"2024-04-24T23:58:20.678003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255072+0000","last_change":"2024-04-24T23:58:18.240520+0000","last_active":"2024-04-24T23:58:20.255072+0000","last_peered":"2024-04-24T23:58:20.255072+0000","last_clean":"2024-04-24T23:58:20.255072+0000","last_became_active":"2024-04-24T23:58:18.240253+0000","last_became_peered":"2024-04-24T23:58:18.240253+0000","last_unstale":"2024-04-24T23:58:20.255072+0000","last_undegraded":"2024-04-24T23:58:20.255072+0000","last_fullsized":"2024-04-24T23:58:20.255072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:17.091843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:58:29.755 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:58:29.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.888+0000 7fc6f47a9640 1 Processor -- start 2024-04-24T23:58:29.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.888+0000 7fc6f47a9640 1 -- start start 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.889+0000 7fc6f47a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc6ec15e320 0x7fc6ec0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.889+0000 7fc6f47a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ec0d3de0 con 0x7fc6ec0d2290 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.889+0000 7fc6f47a9640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ec0d2800 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.889+0000 7fc6f47a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ec0d3410 con 0x7fc6ec15e320 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f2545640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc6ec15e320 0x7fc6ec0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f1d44640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc6ec0d2290 0x7fc6ec0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55558/0 (socket says 172.21.15.113:55558) 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f1d44640 1 -- 172.21.15.113:0/3537638458 learned_addr learned my addr 172.21.15.113:0/3537638458 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f2545640 1 -- 172.21.15.113:0/3537638458 >> v1:172.21.15.178:6789/0 conn(0x7fc6ec0d0b00 legacy=0x7fc6ec0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f2545640 1 -- 172.21.15.113:0/3537638458 >> v1:172.21.15.113:6789/0 conn(0x7fc6ec0d2290 legacy=0x7fc6ec0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f1543640 1 -- 172.21.15.113:0/3537638458 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094698737 0 0) 0x7fc6ec0d3de0 con 0x7fc6ec0d2290 2024-04-24T23:58:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f2545640 1 -- 172.21.15.113:0/3537638458 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec0d2ae0 con 0x7fc6ec15e320 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.890+0000 7fc6f2545640 1 --2- 172.21.15.113:0/3537638458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15e320 0x7fc6ec0d38a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80097a0 tx=0x7fc6e802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a32865ded647be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f1543640 1 -- 172.21.15.113:0/3537638458 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8037070 con 0x7fc6ec15e320 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f1543640 1 -- 172.21.15.113:0/3537638458 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e8032640 con 0x7fc6ec15e320 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f1543640 1 -- 172.21.15.113:0/3537638458 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8032a10 con 0x7fc6ec15e320 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f47a9640 1 -- 172.21.15.113:0/3537638458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15e320 msgr2=0x7fc6ec0d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/3537638458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15e320 0x7fc6ec0d38a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80097a0 tx=0x7fc6e802a2c0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f47a9640 1 -- 172.21.15.113:0/3537638458 shutdown_connections 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/3537638458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15e320 0x7fc6ec0d38a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f47a9640 1 -- 172.21.15.113:0/3537638458 >> 172.21.15.113:0/3537638458 conn(0x7fc6ec0b5c20 msgr2=0x7fc6ec0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:29.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f47a9640 1 -- 172.21.15.113:0/3537638458 shutdown_connections 2024-04-24T23:58:29.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.891+0000 7fc6f47a9640 1 -- 172.21.15.113:0/3537638458 wait complete. 2024-04-24T23:58:29.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.892+0000 7fc6f47a9640 1 Processor -- start 2024-04-24T23:58:29.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.892+0000 7fc6f47a9640 1 -- start start 2024-04-24T23:58:29.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f47a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec0d0b00 0x7fc6ec15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f47a9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6ec0d2290 0x7fc6ec0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f47a9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6ec15e320 0x7fc6ec15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f47a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec15e700 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f47a9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec0d30a0 con 0x7fc6ec0d2290 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f47a9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ec15e880 con 0x7fc6ec15e320 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec0d0b00 0x7fc6ec15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec0d0b00 0x7fc6ec15a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50348/0 (socket says 172.21.15.113:50348) 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 -- 172.21.15.113:0/1191793255 learned_addr learned my addr 172.21.15.113:0/1191793255 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f1d44640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6ec0d2290 0x7fc6ec0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2d46640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6ec15e320 0x7fc6ec15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 -- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6ec15e320 msgr2=0x7fc6ec15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6ec15e320 0x7fc6ec15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 -- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6ec0d2290 msgr2=0x7fc6ec0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6ec0d2290 0x7fc6ec0cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 -- 172.21.15.113:0/1191793255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec154ab0 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f1d44640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6ec0d2290 0x7fc6ec0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2d46640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6ec15e320 0x7fc6ec15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.893+0000 7fc6f2545640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec0d0b00 0x7fc6ec15a300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80098c0 tx=0x7fc6e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.894+0000 7fc6e2ffd640 1 -- 172.21.15.113:0/1191793255 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8041020 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.894+0000 7fc6e2ffd640 1 -- 172.21.15.113:0/1191793255 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e8030970 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.894+0000 7fc6e2ffd640 1 -- 172.21.15.113:0/1191793255 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8037040 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.894+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec154d50 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.894+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec0cce50 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.895+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec0d1000 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.895+0000 7fc6e2ffd640 1 -- 172.21.15.113:0/1191793255 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc6e8044020 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.895+0000 7fc6e2ffd640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc6d006be30 0x7fc6d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:29.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.896+0000 7fc6e2ffd640 1 -- 172.21.15.113:0/1191793255 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc6e80ab7e0 con 0x7fc6ec0d0b00 2024-04-24T23:58:29.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.896+0000 7fc6f1d44640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc6d006be30 0x7fc6d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:29.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.896+0000 7fc6f1d44640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc6d006be30 0x7fc6d006e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec15f1f0 tx=0x7fc6dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:29.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:29.900+0000 7fc6e2ffd640 1 -- 172.21.15.113:0/1191793255 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc6ec0d1000 con 0x7fc6ec0d0b00 2024-04-24T23:58:30.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.069+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6ec0691a0 con 0x7fc6d006be30 2024-04-24T23:58:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.075+0000 7fc6e2ffd640 1 -- 172.21.15.113:0/1191793255 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fc6ec0691a0 con 0x7fc6d006be30 2024-04-24T23:58:30.077 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:30.078 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:58:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc6d006be30 msgr2=0x7fc6d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc6d006be30 0x7fc6d006e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec15f1f0 tx=0x7fc6dc009040 comp rx=0 tx=0).stop 2024-04-24T23:58:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec0d0b00 msgr2=0x7fc6ec15a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec0d0b00 0x7fc6ec15a300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80098c0 tx=0x7fc6e80046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 shutdown_connections 2024-04-24T23:58:30.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc6d006be30 0x7fc6d006e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6ec0d2290 0x7fc6ec0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6ec15e320 0x7fc6ec15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 --2- 172.21.15.113:0/1191793255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec0d0b00 0x7fc6ec15a300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.077+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 >> 172.21.15.113:0/1191793255 conn(0x7fc6ec0b5c20 msgr2=0x7fc6ec0b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:30.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.078+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 shutdown_connections 2024-04-24T23:58:30.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.078+0000 7fc6f47a9640 1 -- 172.21.15.113:0/1191793255 wait complete. 2024-04-24T23:58:30.088 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-24T23:58:29.113873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235632,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739088,"statfs":{"total":773094113280,"available":772852826112,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.870942"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678418+0000","last_change":"2024-04-24T23:58:19.270520+0000","last_active":"2024-04-24T23:58:20.678418+0000","last_peered":"2024-04-24T23:58:20.678418+0000","last_clean":"2024-04-24T23:58:20.678418+0000","last_became_active":"2024-04-24T23:58:19.270280+0000","last_became_peered":"2024-04-24T23:58:19.270280+0000","last_unstale":"2024-04-24T23:58:20.678418+0000","last_undegraded":"2024-04-24T23:58:20.678418+0000","last_fullsized":"2024-04-24T23:58:20.678418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254502+0000","last_change":"2024-04-24T23:58:19.252635+0000","last_active":"2024-04-24T23:58:20.254502+0000","last_peered":"2024-04-24T23:58:20.254502+0000","last_clean":"2024-04-24T23:58:20.254502+0000","last_became_active":"2024-04-24T23:58:19.252352+0000","last_became_peered":"2024-04-24T23:58:19.252352+0000","last_unstale":"2024-04-24T23:58:20.254502+0000","last_undegraded":"2024-04-24T23:58:20.254502+0000","last_fullsized":"2024-04-24T23:58:20.254502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254874+0000","last_change":"2024-04-24T23:58:19.257041+0000","last_active":"2024-04-24T23:58:20.254874+0000","last_peered":"2024-04-24T23:58:20.254874+0000","last_clean":"2024-04-24T23:58:20.254874+0000","last_became_active":"2024-04-24T23:58:19.256765+0000","last_became_peered":"2024-04-24T23:58:19.256765+0000","last_unstale":"2024-04-24T23:58:20.254874+0000","last_undegraded":"2024-04-24T23:58:20.254874+0000","last_fullsized":"2024-04-24T23:58:20.254874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.271981+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.271589+0000","last_became_peered":"2024-04-24T23:58:19.271589+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678540+0000","last_change":"2024-04-24T23:58:19.260799+0000","last_active":"2024-04-24T23:58:20.678540+0000","last_peered":"2024-04-24T23:58:20.678540+0000","last_clean":"2024-04-24T23:58:20.678540+0000","last_became_active":"2024-04-24T23:58:19.260437+0000","last_became_peered":"2024-04-24T23:58:19.260437+0000","last_unstale":"2024-04-24T23:58:20.678540+0000","last_undegraded":"2024-04-24T23:58:20.678540+0000","last_fullsized":"2024-04-24T23:58:20.678540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677754+0000","last_change":"2024-04-24T23:58:19.269955+0000","last_active":"2024-04-24T23:58:20.677754+0000","last_peered":"2024-04-24T23:58:20.677754+0000","last_clean":"2024-04-24T23:58:20.677754+0000","last_became_active":"2024-04-24T23:58:19.269675+0000","last_became_peered":"2024-04-24T23:58:19.269675+0000","last_unstale":"2024-04-24T23:58:20.677754+0000","last_undegraded":"2024-04-24T23:58:20.677754+0000","last_fullsized":"2024-04-24T23:58:20.677754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678346+0000","last_change":"2024-04-24T23:58:19.267752+0000","last_active":"2024-04-24T23:58:20.678346+0000","last_peered":"2024-04-24T23:58:20.678346+0000","last_clean":"2024-04-24T23:58:20.678346+0000","last_became_active":"2024-04-24T23:58:19.267463+0000","last_became_peered":"2024-04-24T23:58:19.267463+0000","last_unstale":"2024-04-24T23:58:20.678346+0000","last_undegraded":"2024-04-24T23:58:20.678346+0000","last_fullsized":"2024-04-24T23:58:20.678346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678319+0000","last_change":"2024-04-24T23:58:19.266736+0000","last_active":"2024-04-24T23:58:20.678319+0000","last_peered":"2024-04-24T23:58:20.678319+0000","last_clean":"2024-04-24T23:58:20.678319+0000","last_became_active":"2024-04-24T23:58:19.266438+0000","last_became_peered":"2024-04-24T23:58:19.266438+0000","last_unstale":"2024-04-24T23:58:20.678319+0000","last_undegraded":"2024-04-24T23:58:20.678319+0000","last_fullsized":"2024-04-24T23:58:20.678319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678538+0000","last_change":"2024-04-24T23:58:19.271086+0000","last_active":"2024-04-24T23:58:20.678538+0000","last_peered":"2024-04-24T23:58:20.678538+0000","last_clean":"2024-04-24T23:58:20.678538+0000","last_became_active":"2024-04-24T23:58:19.270722+0000","last_became_peered":"2024-04-24T23:58:19.270722+0000","last_unstale":"2024-04-24T23:58:20.678538+0000","last_undegraded":"2024-04-24T23:58:20.678538+0000","last_fullsized":"2024-04-24T23:58:20.678538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257102+0000","last_change":"2024-04-24T23:58:19.266845+0000","last_active":"2024-04-24T23:58:20.257102+0000","last_peered":"2024-04-24T23:58:20.257102+0000","last_clean":"2024-04-24T23:58:20.257102+0000","last_became_active":"2024-04-24T23:58:19.266573+0000","last_became_peered":"2024-04-24T23:58:19.266573+0000","last_unstale":"2024-04-24T23:58:20.257102+0000","last_undegraded":"2024-04-24T23:58:20.257102+0000","last_fullsized":"2024-04-24T23:58:20.257102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256863+0000","last_change":"2024-04-24T23:58:19.267588+0000","last_active":"2024-04-24T23:58:20.256863+0000","last_peered":"2024-04-24T23:58:20.256863+0000","last_clean":"2024-04-24T23:58:20.256863+0000","last_became_active":"2024-04-24T23:58:19.267247+0000","last_became_peered":"2024-04-24T23:58:19.267247+0000","last_unstale":"2024-04-24T23:58:20.256863+0000","last_undegraded":"2024-04-24T23:58:20.256863+0000","last_fullsized":"2024-04-24T23:58:20.256863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255357+0000","last_change":"2024-04-24T23:58:19.257310+0000","last_active":"2024-04-24T23:58:20.255357+0000","last_peered":"2024-04-24T23:58:20.255357+0000","last_clean":"2024-04-24T23:58:20.255357+0000","last_became_active":"2024-04-24T23:58:19.257023+0000","last_became_peered":"2024-04-24T23:58:19.257023+0000","last_unstale":"2024-04-24T23:58:20.255357+0000","last_undegraded":"2024-04-24T23:58:20.255357+0000","last_fullsized":"2024-04-24T23:58:20.255357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678413+0000","last_change":"2024-04-24T23:58:19.261091+0000","last_active":"2024-04-24T23:58:20.678413+0000","last_peered":"2024-04-24T23:58:20.678413+0000","last_clean":"2024-04-24T23:58:20.678413+0000","last_became_active":"2024-04-24T23:58:19.260837+0000","last_became_peered":"2024-04-24T23:58:19.260837+0000","last_unstale":"2024-04-24T23:58:20.678413+0000","last_undegraded":"2024-04-24T23:58:20.678413+0000","last_fullsized":"2024-04-24T23:58:20.678413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:02.270735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254811+0000","last_change":"2024-04-24T23:58:19.261328+0000","last_active":"2024-04-24T23:58:20.254811+0000","last_peered":"2024-04-24T23:58:20.254811+0000","last_clean":"2024-04-24T23:58:20.254811+0000","last_became_active":"2024-04-24T23:58:19.261045+0000","last_became_peered":"2024-04-24T23:58:19.261045+0000","last_unstale":"2024-04-24T23:58:20.254811+0000","last_undegraded":"2024-04-24T23:58:20.254811+0000","last_fullsized":"2024-04-24T23:58:20.254811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254583+0000","last_change":"2024-04-24T23:58:19.256356+0000","last_active":"2024-04-24T23:58:20.254583+0000","last_peered":"2024-04-24T23:58:20.254583+0000","last_clean":"2024-04-24T23:58:20.254583+0000","last_became_active":"2024-04-24T23:58:19.256099+0000","last_became_peered":"2024-04-24T23:58:19.256099+0000","last_unstale":"2024-04-24T23:58:20.254583+0000","last_undegraded":"2024-04-24T23:58:20.254583+0000","last_fullsized":"2024-04-24T23:58:20.254583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677420+0000","last_change":"2024-04-24T23:58:19.268419+0000","last_active":"2024-04-24T23:58:20.677420+0000","last_peered":"2024-04-24T23:58:20.677420+0000","last_clean":"2024-04-24T23:58:20.677420+0000","last_became_active":"2024-04-24T23:58:19.265742+0000","last_became_peered":"2024-04-24T23:58:19.265742+0000","last_unstale":"2024-04-24T23:58:20.677420+0000","last_undegraded":"2024-04-24T23:58:20.677420+0000","last_fullsized":"2024-04-24T23:58:20.677420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.273024+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.272551+0000","last_became_peered":"2024-04-24T23:58:19.272551+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677871+0000","last_change":"2024-04-24T23:58:19.258361+0000","last_active":"2024-04-24T23:58:20.677871+0000","last_peered":"2024-04-24T23:58:20.677871+0000","last_clean":"2024-04-24T23:58:20.677871+0000","last_became_active":"2024-04-24T23:58:19.258098+0000","last_became_peered":"2024-04-24T23:58:19.258098+0000","last_unstale":"2024-04-24T23:58:20.677871+0000","last_undegraded":"2024-04-24T23:58:20.677871+0000","last_fullsized":"2024-04-24T23:58:20.677871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:32.596684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678669+0000","last_change":"2024-04-24T23:58:19.263085+0000","last_active":"2024-04-24T23:58:20.678669+0000","last_peered":"2024-04-24T23:58:20.678669+0000","last_clean":"2024-04-24T23:58:20.678669+0000","last_became_active":"2024-04-24T23:58:19.262807+0000","last_became_peered":"2024-04-24T23:58:19.262807+0000","last_unstale":"2024-04-24T23:58:20.678669+0000","last_undegraded":"2024-04-24T23:58:20.678669+0000","last_fullsized":"2024-04-24T23:58:20.678669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:16.415858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256867+0000","last_change":"2024-04-24T23:58:19.259501+0000","last_active":"2024-04-24T23:58:20.256867+0000","last_peered":"2024-04-24T23:58:20.256867+0000","last_clean":"2024-04-24T23:58:20.256867+0000","last_became_active":"2024-04-24T23:58:19.259233+0000","last_became_peered":"2024-04-24T23:58:19.259233+0000","last_unstale":"2024-04-24T23:58:20.256867+0000","last_undegraded":"2024-04-24T23:58:20.256867+0000","last_fullsized":"2024-04-24T23:58:20.256867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255041+0000","last_change":"2024-04-24T23:58:19.259960+0000","last_active":"2024-04-24T23:58:20.255041+0000","last_peered":"2024-04-24T23:58:20.255041+0000","last_clean":"2024-04-24T23:58:20.255041+0000","last_became_active":"2024-04-24T23:58:19.259548+0000","last_became_peered":"2024-04-24T23:58:19.259548+0000","last_unstale":"2024-04-24T23:58:20.255041+0000","last_undegraded":"2024-04-24T23:58:20.255041+0000","last_fullsized":"2024-04-24T23:58:20.255041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677851+0000","last_change":"2024-04-24T23:58:19.275499+0000","last_active":"2024-04-24T23:58:20.677851+0000","last_peered":"2024-04-24T23:58:20.677851+0000","last_clean":"2024-04-24T23:58:20.677851+0000","last_became_active":"2024-04-24T23:58:19.275213+0000","last_became_peered":"2024-04-24T23:58:19.275213+0000","last_unstale":"2024-04-24T23:58:20.677851+0000","last_undegraded":"2024-04-24T23:58:20.677851+0000","last_fullsized":"2024-04-24T23:58:20.677851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678362+0000","last_change":"2024-04-24T23:58:19.261963+0000","last_active":"2024-04-24T23:58:20.678362+0000","last_peered":"2024-04-24T23:58:20.678362+0000","last_clean":"2024-04-24T23:58:20.678362+0000","last_became_active":"2024-04-24T23:58:19.261710+0000","last_became_peered":"2024-04-24T23:58:19.261710+0000","last_unstale":"2024-04-24T23:58:20.678362+0000","last_undegraded":"2024-04-24T23:58:20.678362+0000","last_fullsized":"2024-04-24T23:58:20.678362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677517+0000","last_change":"2024-04-24T23:58:19.268171+0000","last_active":"2024-04-24T23:58:20.677517+0000","last_peered":"2024-04-24T23:58:20.677517+0000","last_clean":"2024-04-24T23:58:20.677517+0000","last_became_active":"2024-04-24T23:58:19.265379+0000","last_became_peered":"2024-04-24T23:58:19.265379+0000","last_unstale":"2024-04-24T23:58:20.677517+0000","last_undegraded":"2024-04-24T23:58:20.677517+0000","last_fullsized":"2024-04-24T23:58:20.677517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678064+0000","last_change":"2024-04-24T23:58:19.259615+0000","last_active":"2024-04-24T23:58:20.678064+0000","last_peered":"2024-04-24T23:58:20.678064+0000","last_clean":"2024-04-24T23:58:20.678064+0000","last_became_active":"2024-04-24T23:58:19.259355+0000","last_became_peered":"2024-04-24T23:58:19.259355+0000","last_unstale":"2024-04-24T23:58:20.678064+0000","last_undegraded":"2024-04-24T23:58:20.678064+0000","last_fullsized":"2024-04-24T23:58:20.678064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257106+0000","last_change":"2024-04-24T23:58:19.269905+0000","last_active":"2024-04-24T23:58:20.257106+0000","last_peered":"2024-04-24T23:58:20.257106+0000","last_clean":"2024-04-24T23:58:20.257106+0000","last_became_active":"2024-04-24T23:58:19.269639+0000","last_became_peered":"2024-04-24T23:58:19.269639+0000","last_unstale":"2024-04-24T23:58:20.257106+0000","last_undegraded":"2024-04-24T23:58:20.257106+0000","last_fullsized":"2024-04-24T23:58:20.257106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254304+0000","last_change":"2024-04-24T23:58:19.256747+0000","last_active":"2024-04-24T23:58:20.254304+0000","last_peered":"2024-04-24T23:58:20.254304+0000","last_clean":"2024-04-24T23:58:20.254304+0000","last_became_active":"2024-04-24T23:58:19.255723+0000","last_became_peered":"2024-04-24T23:58:19.255723+0000","last_unstale":"2024-04-24T23:58:20.254304+0000","last_undegraded":"2024-04-24T23:58:20.254304+0000","last_fullsized":"2024-04-24T23:58:20.254304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678051+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678051+0000","last_peered":"2024-04-24T23:58:20.678051+0000","last_clean":"2024-04-24T23:58:20.678051+0000","last_became_active":"2024-04-24T23:58:19.262977+0000","last_became_peered":"2024-04-24T23:58:19.262977+0000","last_unstale":"2024-04-24T23:58:20.678051+0000","last_undegraded":"2024-04-24T23:58:20.678051+0000","last_fullsized":"2024-04-24T23:58:20.678051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255177+0000","last_change":"2024-04-24T23:58:19.259745+0000","last_active":"2024-04-24T23:58:20.255177+0000","last_peered":"2024-04-24T23:58:20.255177+0000","last_clean":"2024-04-24T23:58:20.255177+0000","last_became_active":"2024-04-24T23:58:19.259201+0000","last_became_peered":"2024-04-24T23:58:19.259201+0000","last_unstale":"2024-04-24T23:58:20.255177+0000","last_undegraded":"2024-04-24T23:58:20.255177+0000","last_fullsized":"2024-04-24T23:58:20.255177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678071+0000","last_change":"2024-04-24T23:58:19.273016+0000","last_active":"2024-04-24T23:58:20.678071+0000","last_peered":"2024-04-24T23:58:20.678071+0000","last_clean":"2024-04-24T23:58:20.678071+0000","last_became_active":"2024-04-24T23:58:19.271540+0000","last_became_peered":"2024-04-24T23:58:19.271540+0000","last_unstale":"2024-04-24T23:58:20.678071+0000","last_undegraded":"2024-04-24T23:58:20.678071+0000","last_fullsized":"2024-04-24T23:58:20.678071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.787225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677646+0000","last_change":"2024-04-24T23:58:19.255458+0000","last_active":"2024-04-24T23:58:20.677646+0000","last_peered":"2024-04-24T23:58:20.677646+0000","last_clean":"2024-04-24T23:58:20.677646+0000","last_became_active":"2024-04-24T23:58:19.255070+0000","last_became_peered":"2024-04-24T23:58:19.255070+0000","last_unstale":"2024-04-24T23:58:20.677646+0000","last_undegraded":"2024-04-24T23:58:20.677646+0000","last_fullsized":"2024-04-24T23:58:20.677646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256632+0000","last_change":"2024-04-24T23:58:19.266657+0000","last_active":"2024-04-24T23:58:20.256632+0000","last_peered":"2024-04-24T23:58:20.256632+0000","last_clean":"2024-04-24T23:58:20.256632+0000","last_became_active":"2024-04-24T23:58:19.266354+0000","last_became_peered":"2024-04-24T23:58:19.266354+0000","last_unstale":"2024-04-24T23:58:20.256632+0000","last_undegraded":"2024-04-24T23:58:20.256632+0000","last_fullsized":"2024-04-24T23:58:20.256632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:45.844846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255049+0000","last_change":"2024-04-24T23:58:19.259253+0000","last_active":"2024-04-24T23:58:20.255049+0000","last_peered":"2024-04-24T23:58:20.255049+0000","last_clean":"2024-04-24T23:58:20.255049+0000","last_became_active":"2024-04-24T23:58:19.258553+0000","last_became_peered":"2024-04-24T23:58:19.258553+0000","last_unstale":"2024-04-24T23:58:20.255049+0000","last_undegraded":"2024-04-24T23:58:20.255049+0000","last_fullsized":"2024-04-24T23:58:20.255049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:53.284799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678523+0000","last_change":"2024-04-24T23:58:18.250614+0000","last_active":"2024-04-24T23:58:20.678523+0000","last_peered":"2024-04-24T23:58:20.678523+0000","last_clean":"2024-04-24T23:58:20.678523+0000","last_became_active":"2024-04-24T23:58:18.250306+0000","last_became_peered":"2024-04-24T23:58:18.250306+0000","last_unstale":"2024-04-24T23:58:20.678523+0000","last_undegraded":"2024-04-24T23:58:20.678523+0000","last_fullsized":"2024-04-24T23:58:20.678523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255184+0000","last_change":"2024-04-24T23:58:19.251648+0000","last_active":"2024-04-24T23:58:20.255184+0000","last_peered":"2024-04-24T23:58:20.255184+0000","last_clean":"2024-04-24T23:58:20.255184+0000","last_became_active":"2024-04-24T23:58:19.251271+0000","last_became_peered":"2024-04-24T23:58:19.251271+0000","last_unstale":"2024-04-24T23:58:20.255184+0000","last_undegraded":"2024-04-24T23:58:20.255184+0000","last_fullsized":"2024-04-24T23:58:20.255184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:04.091190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256708+0000","last_change":"2024-04-24T23:58:18.252764+0000","last_active":"2024-04-24T23:58:20.256708+0000","last_peered":"2024-04-24T23:58:20.256708+0000","last_clean":"2024-04-24T23:58:20.256708+0000","last_became_active":"2024-04-24T23:58:18.252491+0000","last_became_peered":"2024-04-24T23:58:18.252491+0000","last_unstale":"2024-04-24T23:58:20.256708+0000","last_undegraded":"2024-04-24T23:58:20.256708+0000","last_fullsized":"2024-04-24T23:58:20.256708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678357+0000","last_change":"2024-04-24T23:58:18.250003+0000","last_active":"2024-04-24T23:58:20.678357+0000","last_peered":"2024-04-24T23:58:20.678357+0000","last_clean":"2024-04-24T23:58:20.678357+0000","last_became_active":"2024-04-24T23:58:18.249727+0000","last_became_peered":"2024-04-24T23:58:18.249727+0000","last_unstale":"2024-04-24T23:58:20.678357+0000","last_undegraded":"2024-04-24T23:58:20.678357+0000","last_fullsized":"2024-04-24T23:58:20.678357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:17.318157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678399+0000","last_change":"2024-04-24T23:58:19.255033+0000","last_active":"2024-04-24T23:58:20.678399+0000","last_peered":"2024-04-24T23:58:20.678399+0000","last_clean":"2024-04-24T23:58:20.678399+0000","last_became_active":"2024-04-24T23:58:19.254746+0000","last_became_peered":"2024-04-24T23:58:19.254746+0000","last_unstale":"2024-04-24T23:58:20.678399+0000","last_undegraded":"2024-04-24T23:58:20.678399+0000","last_fullsized":"2024-04-24T23:58:20.678399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677538+0000","last_change":"2024-04-24T23:58:18.245516+0000","last_active":"2024-04-24T23:58:20.677538+0000","last_peered":"2024-04-24T23:58:20.677538+0000","last_clean":"2024-04-24T23:58:20.677538+0000","last_became_active":"2024-04-24T23:58:18.245212+0000","last_became_peered":"2024-04-24T23:58:18.245212+0000","last_unstale":"2024-04-24T23:58:20.677538+0000","last_undegraded":"2024-04-24T23:58:20.677538+0000","last_fullsized":"2024-04-24T23:58:20.677538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254479+0000","last_change":"2024-04-24T23:58:19.258354+0000","last_active":"2024-04-24T23:58:20.254479+0000","last_peered":"2024-04-24T23:58:20.254479+0000","last_clean":"2024-04-24T23:58:20.254479+0000","last_became_active":"2024-04-24T23:58:19.258067+0000","last_became_peered":"2024-04-24T23:58:19.258067+0000","last_unstale":"2024-04-24T23:58:20.254479+0000","last_undegraded":"2024-04-24T23:58:20.254479+0000","last_fullsized":"2024-04-24T23:58:20.254479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256985+0000","last_change":"2024-04-24T23:58:18.253806+0000","last_active":"2024-04-24T23:58:20.256985+0000","last_peered":"2024-04-24T23:58:20.256985+0000","last_clean":"2024-04-24T23:58:20.256985+0000","last_became_active":"2024-04-24T23:58:18.252657+0000","last_became_peered":"2024-04-24T23:58:18.252657+0000","last_unstale":"2024-04-24T23:58:20.256985+0000","last_undegraded":"2024-04-24T23:58:20.256985+0000","last_fullsized":"2024-04-24T23:58:20.256985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677952+0000","last_change":"2024-04-24T23:58:19.275012+0000","last_active":"2024-04-24T23:58:20.677952+0000","last_peered":"2024-04-24T23:58:20.677952+0000","last_clean":"2024-04-24T23:58:20.677952+0000","last_became_active":"2024-04-24T23:58:19.274740+0000","last_became_peered":"2024-04-24T23:58:19.274740+0000","last_unstale":"2024-04-24T23:58:20.677952+0000","last_undegraded":"2024-04-24T23:58:20.677952+0000","last_fullsized":"2024-04-24T23:58:20.677952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678001+0000","last_change":"2024-04-24T23:58:18.251763+0000","last_active":"2024-04-24T23:58:20.678001+0000","last_peered":"2024-04-24T23:58:20.678001+0000","last_clean":"2024-04-24T23:58:20.678001+0000","last_became_active":"2024-04-24T23:58:18.251508+0000","last_became_peered":"2024-04-24T23:58:18.251508+0000","last_unstale":"2024-04-24T23:58:20.678001+0000","last_undegraded":"2024-04-24T23:58:20.678001+0000","last_fullsized":"2024-04-24T23:58:20.678001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254979+0000","last_change":"2024-04-24T23:58:19.253275+0000","last_active":"2024-04-24T23:58:20.254979+0000","last_peered":"2024-04-24T23:58:20.254979+0000","last_clean":"2024-04-24T23:58:20.254979+0000","last_became_active":"2024-04-24T23:58:19.252783+0000","last_became_peered":"2024-04-24T23:58:19.252783+0000","last_unstale":"2024-04-24T23:58:20.254979+0000","last_undegraded":"2024-04-24T23:58:20.254979+0000","last_fullsized":"2024-04-24T23:58:20.254979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:38.514418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254408+0000","last_change":"2024-04-24T23:58:18.246446+0000","last_active":"2024-04-24T23:58:20.254408+0000","last_peered":"2024-04-24T23:58:20.254408+0000","last_clean":"2024-04-24T23:58:20.254408+0000","last_became_active":"2024-04-24T23:58:18.246036+0000","last_became_peered":"2024-04-24T23:58:18.246036+0000","last_unstale":"2024-04-24T23:58:20.254408+0000","last_undegraded":"2024-04-24T23:58:20.254408+0000","last_fullsized":"2024-04-24T23:58:20.254408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255261+0000","last_change":"2024-04-24T23:58:19.261655+0000","last_active":"2024-04-24T23:58:20.255261+0000","last_peered":"2024-04-24T23:58:20.255261+0000","last_clean":"2024-04-24T23:58:20.255261+0000","last_became_active":"2024-04-24T23:58:19.261331+0000","last_became_peered":"2024-04-24T23:58:19.261331+0000","last_unstale":"2024-04-24T23:58:20.255261+0000","last_undegraded":"2024-04-24T23:58:20.255261+0000","last_fullsized":"2024-04-24T23:58:20.255261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.991186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256970+0000","last_change":"2024-04-24T23:58:18.247264+0000","last_active":"2024-04-24T23:58:20.256970+0000","last_peered":"2024-04-24T23:58:20.256970+0000","last_clean":"2024-04-24T23:58:20.256970+0000","last_became_active":"2024-04-24T23:58:18.246959+0000","last_became_peered":"2024-04-24T23:58:18.246959+0000","last_unstale":"2024-04-24T23:58:20.256970+0000","last_undegraded":"2024-04-24T23:58:20.256970+0000","last_fullsized":"2024-04-24T23:58:20.256970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678015+0000","last_change":"2024-04-24T23:58:19.265255+0000","last_active":"2024-04-24T23:58:20.678015+0000","last_peered":"2024-04-24T23:58:20.678015+0000","last_clean":"2024-04-24T23:58:20.678015+0000","last_became_active":"2024-04-24T23:58:19.264997+0000","last_became_peered":"2024-04-24T23:58:19.264997+0000","last_unstale":"2024-04-24T23:58:20.678015+0000","last_undegraded":"2024-04-24T23:58:20.678015+0000","last_fullsized":"2024-04-24T23:58:20.678015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:08.241560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255275+0000","last_change":"2024-04-24T23:58:18.236929+0000","last_active":"2024-04-24T23:58:20.255275+0000","last_peered":"2024-04-24T23:58:20.255275+0000","last_clean":"2024-04-24T23:58:20.255275+0000","last_became_active":"2024-04-24T23:58:18.236660+0000","last_became_peered":"2024-04-24T23:58:18.236660+0000","last_unstale":"2024-04-24T23:58:20.255275+0000","last_undegraded":"2024-04-24T23:58:20.255275+0000","last_fullsized":"2024-04-24T23:58:20.255275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254944+0000","last_change":"2024-04-24T23:58:19.259532+0000","last_active":"2024-04-24T23:58:20.254944+0000","last_peered":"2024-04-24T23:58:20.254944+0000","last_clean":"2024-04-24T23:58:20.254944+0000","last_became_active":"2024-04-24T23:58:19.259298+0000","last_became_peered":"2024-04-24T23:58:19.259298+0000","last_unstale":"2024-04-24T23:58:20.254944+0000","last_undegraded":"2024-04-24T23:58:20.254944+0000","last_fullsized":"2024-04-24T23:58:20.254944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678194+0000","last_change":"2024-04-24T23:58:18.251116+0000","last_active":"2024-04-24T23:58:20.678194+0000","last_peered":"2024-04-24T23:58:20.678194+0000","last_clean":"2024-04-24T23:58:20.678194+0000","last_became_active":"2024-04-24T23:58:18.250814+0000","last_became_peered":"2024-04-24T23:58:18.250814+0000","last_unstale":"2024-04-24T23:58:20.678194+0000","last_undegraded":"2024-04-24T23:58:20.678194+0000","last_fullsized":"2024-04-24T23:58:20.678194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678564+0000","last_change":"2024-04-24T23:58:19.263724+0000","last_active":"2024-04-24T23:58:20.678564+0000","last_peered":"2024-04-24T23:58:20.678564+0000","last_clean":"2024-04-24T23:58:20.678564+0000","last_became_active":"2024-04-24T23:58:19.263448+0000","last_became_peered":"2024-04-24T23:58:19.263448+0000","last_unstale":"2024-04-24T23:58:20.678564+0000","last_undegraded":"2024-04-24T23:58:20.678564+0000","last_fullsized":"2024-04-24T23:58:20.678564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256648+0000","last_change":"2024-04-24T23:58:18.249529+0000","last_active":"2024-04-24T23:58:20.256648+0000","last_peered":"2024-04-24T23:58:20.256648+0000","last_clean":"2024-04-24T23:58:20.256648+0000","last_became_active":"2024-04-24T23:58:18.249135+0000","last_became_peered":"2024-04-24T23:58:18.249135+0000","last_unstale":"2024-04-24T23:58:20.256648+0000","last_undegraded":"2024-04-24T23:58:20.256648+0000","last_fullsized":"2024-04-24T23:58:20.256648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256714+0000","last_change":"2024-04-24T23:58:19.266083+0000","last_active":"2024-04-24T23:58:20.256714+0000","last_peered":"2024-04-24T23:58:20.256714+0000","last_clean":"2024-04-24T23:58:20.256714+0000","last_became_active":"2024-04-24T23:58:19.265757+0000","last_became_peered":"2024-04-24T23:58:19.265757+0000","last_unstale":"2024-04-24T23:58:20.256714+0000","last_undegraded":"2024-04-24T23:58:20.256714+0000","last_fullsized":"2024-04-24T23:58:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677645+0000","last_change":"2024-04-24T23:58:18.245808+0000","last_active":"2024-04-24T23:58:20.677645+0000","last_peered":"2024-04-24T23:58:20.677645+0000","last_clean":"2024-04-24T23:58:20.677645+0000","last_became_active":"2024-04-24T23:58:18.244769+0000","last_became_peered":"2024-04-24T23:58:18.244769+0000","last_unstale":"2024-04-24T23:58:20.677645+0000","last_undegraded":"2024-04-24T23:58:20.677645+0000","last_fullsized":"2024-04-24T23:58:20.677645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:43.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678393+0000","last_change":"2024-04-24T23:58:19.262773+0000","last_active":"2024-04-24T23:58:20.678393+0000","last_peered":"2024-04-24T23:58:20.678393+0000","last_clean":"2024-04-24T23:58:20.678393+0000","last_became_active":"2024-04-24T23:58:19.262384+0000","last_became_peered":"2024-04-24T23:58:19.262384+0000","last_unstale":"2024-04-24T23:58:20.678393+0000","last_undegraded":"2024-04-24T23:58:20.678393+0000","last_fullsized":"2024-04-24T23:58:20.678393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255516+0000","last_change":"2024-04-24T23:58:18.243528+0000","last_active":"2024-04-24T23:58:20.255516+0000","last_peered":"2024-04-24T23:58:20.255516+0000","last_clean":"2024-04-24T23:58:20.255516+0000","last_became_active":"2024-04-24T23:58:18.243262+0000","last_became_peered":"2024-04-24T23:58:18.243262+0000","last_unstale":"2024-04-24T23:58:20.255516+0000","last_undegraded":"2024-04-24T23:58:20.255516+0000","last_fullsized":"2024-04-24T23:58:20.255516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:15.332690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677877+0000","last_change":"2024-04-24T23:58:19.261251+0000","last_active":"2024-04-24T23:58:20.677877+0000","last_peered":"2024-04-24T23:58:20.677877+0000","last_clean":"2024-04-24T23:58:20.677877+0000","last_became_active":"2024-04-24T23:58:19.260977+0000","last_became_peered":"2024-04-24T23:58:19.260977+0000","last_unstale":"2024-04-24T23:58:20.677877+0000","last_undegraded":"2024-04-24T23:58:20.677877+0000","last_fullsized":"2024-04-24T23:58:20.677877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677909+0000","last_change":"2024-04-24T23:58:18.242128+0000","last_active":"2024-04-24T23:58:20.677909+0000","last_peered":"2024-04-24T23:58:20.677909+0000","last_clean":"2024-04-24T23:58:20.677909+0000","last_became_active":"2024-04-24T23:58:18.241804+0000","last_became_peered":"2024-04-24T23:58:18.241804+0000","last_unstale":"2024-04-24T23:58:20.677909+0000","last_undegraded":"2024-04-24T23:58:20.677909+0000","last_fullsized":"2024-04-24T23:58:20.677909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678616+0000","last_change":"2024-04-24T23:58:19.262407+0000","last_active":"2024-04-24T23:58:20.678616+0000","last_peered":"2024-04-24T23:58:20.678616+0000","last_clean":"2024-04-24T23:58:20.678616+0000","last_became_active":"2024-04-24T23:58:19.262123+0000","last_became_peered":"2024-04-24T23:58:19.262123+0000","last_unstale":"2024-04-24T23:58:20.678616+0000","last_undegraded":"2024-04-24T23:58:20.678616+0000","last_fullsized":"2024-04-24T23:58:20.678616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254597+0000","last_change":"2024-04-24T23:58:18.242601+0000","last_active":"2024-04-24T23:58:20.254597+0000","last_peered":"2024-04-24T23:58:20.254597+0000","last_clean":"2024-04-24T23:58:20.254597+0000","last_became_active":"2024-04-24T23:58:18.242314+0000","last_became_peered":"2024-04-24T23:58:18.242314+0000","last_unstale":"2024-04-24T23:58:20.254597+0000","last_undegraded":"2024-04-24T23:58:20.254597+0000","last_fullsized":"2024-04-24T23:58:20.254597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678172+0000","last_change":"2024-04-24T23:58:19.265899+0000","last_active":"2024-04-24T23:58:20.678172+0000","last_peered":"2024-04-24T23:58:20.678172+0000","last_clean":"2024-04-24T23:58:20.678172+0000","last_became_active":"2024-04-24T23:58:19.265596+0000","last_became_peered":"2024-04-24T23:58:19.265596+0000","last_unstale":"2024-04-24T23:58:20.678172+0000","last_undegraded":"2024-04-24T23:58:20.678172+0000","last_fullsized":"2024-04-24T23:58:20.678172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:48.469478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677741+0000","last_change":"2024-04-24T23:58:18.248287+0000","last_active":"2024-04-24T23:58:20.677741+0000","last_peered":"2024-04-24T23:58:20.677741+0000","last_clean":"2024-04-24T23:58:20.677741+0000","last_became_active":"2024-04-24T23:58:18.248034+0000","last_became_peered":"2024-04-24T23:58:18.248034+0000","last_unstale":"2024-04-24T23:58:20.677741+0000","last_undegraded":"2024-04-24T23:58:20.677741+0000","last_fullsized":"2024-04-24T23:58:20.677741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678510+0000","last_change":"2024-04-24T23:58:19.274141+0000","last_active":"2024-04-24T23:58:20.678510+0000","last_peered":"2024-04-24T23:58:20.678510+0000","last_clean":"2024-04-24T23:58:20.678510+0000","last_became_active":"2024-04-24T23:58:19.273864+0000","last_became_peered":"2024-04-24T23:58:19.273864+0000","last_unstale":"2024-04-24T23:58:20.678510+0000","last_undegraded":"2024-04-24T23:58:20.678510+0000","last_fullsized":"2024-04-24T23:58:20.678510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677425+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:58:20.677425+0000","last_peered":"2024-04-24T23:58:20.677425+0000","last_clean":"2024-04-24T23:58:20.677425+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:58:20.677425+0000","last_undegraded":"2024-04-24T23:58:20.677425+0000","last_fullsized":"2024-04-24T23:58:20.677425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677567+0000","last_change":"2024-04-24T23:58:18.247362+0000","last_active":"2024-04-24T23:58:20.677567+0000","last_peered":"2024-04-24T23:58:20.677567+0000","last_clean":"2024-04-24T23:58:20.677567+0000","last_became_active":"2024-04-24T23:58:18.247084+0000","last_became_peered":"2024-04-24T23:58:18.247084+0000","last_unstale":"2024-04-24T23:58:20.677567+0000","last_undegraded":"2024-04-24T23:58:20.677567+0000","last_fullsized":"2024-04-24T23:58:20.677567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678143+0000","last_change":"2024-04-24T23:58:19.272925+0000","last_active":"2024-04-24T23:58:20.678143+0000","last_peered":"2024-04-24T23:58:20.678143+0000","last_clean":"2024-04-24T23:58:20.678143+0000","last_became_active":"2024-04-24T23:58:19.272538+0000","last_became_peered":"2024-04-24T23:58:19.272538+0000","last_unstale":"2024-04-24T23:58:20.678143+0000","last_undegraded":"2024-04-24T23:58:20.678143+0000","last_fullsized":"2024-04-24T23:58:20.678143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677369+0000","last_change":"2024-04-24T23:58:18.251207+0000","last_active":"2024-04-24T23:58:20.677369+0000","last_peered":"2024-04-24T23:58:20.677369+0000","last_clean":"2024-04-24T23:58:20.677369+0000","last_became_active":"2024-04-24T23:58:18.250921+0000","last_became_peered":"2024-04-24T23:58:18.250921+0000","last_unstale":"2024-04-24T23:58:20.677369+0000","last_undegraded":"2024-04-24T23:58:20.677369+0000","last_fullsized":"2024-04-24T23:58:20.677369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254623+0000","last_change":"2024-04-24T23:58:19.248221+0000","last_active":"2024-04-24T23:58:20.254623+0000","last_peered":"2024-04-24T23:58:20.254623+0000","last_clean":"2024-04-24T23:58:20.254623+0000","last_became_active":"2024-04-24T23:58:19.247896+0000","last_became_peered":"2024-04-24T23:58:19.247896+0000","last_unstale":"2024-04-24T23:58:20.254623+0000","last_undegraded":"2024-04-24T23:58:20.254623+0000","last_fullsized":"2024-04-24T23:58:20.254623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:24.816499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677678+0000","last_change":"2024-04-24T23:58:18.245116+0000","last_active":"2024-04-24T23:58:20.677678+0000","last_peered":"2024-04-24T23:58:20.677678+0000","last_clean":"2024-04-24T23:58:20.677678+0000","last_became_active":"2024-04-24T23:58:18.244779+0000","last_became_peered":"2024-04-24T23:58:18.244779+0000","last_unstale":"2024-04-24T23:58:20.677678+0000","last_undegraded":"2024-04-24T23:58:20.677678+0000","last_fullsized":"2024-04-24T23:58:20.677678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254292+0000","last_change":"2024-04-24T23:58:19.258623+0000","last_active":"2024-04-24T23:58:20.254292+0000","last_peered":"2024-04-24T23:58:20.254292+0000","last_clean":"2024-04-24T23:58:20.254292+0000","last_became_active":"2024-04-24T23:58:19.258371+0000","last_became_peered":"2024-04-24T23:58:19.258371+0000","last_unstale":"2024-04-24T23:58:20.254292+0000","last_undegraded":"2024-04-24T23:58:20.254292+0000","last_fullsized":"2024-04-24T23:58:20.254292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:07.809099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678304+0000","last_change":"2024-04-24T23:58:18.250837+0000","last_active":"2024-04-24T23:58:20.678304+0000","last_peered":"2024-04-24T23:58:20.678304+0000","last_clean":"2024-04-24T23:58:20.678304+0000","last_became_active":"2024-04-24T23:58:18.250540+0000","last_became_peered":"2024-04-24T23:58:18.250540+0000","last_unstale":"2024-04-24T23:58:20.678304+0000","last_undegraded":"2024-04-24T23:58:20.678304+0000","last_fullsized":"2024-04-24T23:58:20.678304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:13.906429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255478+0000","last_change":"2024-04-24T23:58:19.254379+0000","last_active":"2024-04-24T23:58:20.255478+0000","last_peered":"2024-04-24T23:58:20.255478+0000","last_clean":"2024-04-24T23:58:20.255478+0000","last_became_active":"2024-04-24T23:58:19.252734+0000","last_became_peered":"2024-04-24T23:58:19.252734+0000","last_unstale":"2024-04-24T23:58:20.255478+0000","last_undegraded":"2024-04-24T23:58:20.255478+0000","last_fullsized":"2024-04-24T23:58:20.255478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:59.970682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254377+0000","last_change":"2024-04-24T23:58:18.247304+0000","last_active":"2024-04-24T23:58:20.254377+0000","last_peered":"2024-04-24T23:58:20.254377+0000","last_clean":"2024-04-24T23:58:20.254377+0000","last_became_active":"2024-04-24T23:58:18.247029+0000","last_became_peered":"2024-04-24T23:58:18.247029+0000","last_unstale":"2024-04-24T23:58:20.254377+0000","last_undegraded":"2024-04-24T23:58:20.254377+0000","last_fullsized":"2024-04-24T23:58:20.254377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678233+0000","last_change":"2024-04-24T23:58:19.268921+0000","last_active":"2024-04-24T23:58:20.678233+0000","last_peered":"2024-04-24T23:58:20.678233+0000","last_clean":"2024-04-24T23:58:20.678233+0000","last_became_active":"2024-04-24T23:58:19.265287+0000","last_became_peered":"2024-04-24T23:58:19.265287+0000","last_unstale":"2024-04-24T23:58:20.678233+0000","last_undegraded":"2024-04-24T23:58:20.678233+0000","last_fullsized":"2024-04-24T23:58:20.678233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:24.655470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678625+0000","last_change":"2024-04-24T23:58:18.249153+0000","last_active":"2024-04-24T23:58:20.678625+0000","last_peered":"2024-04-24T23:58:20.678625+0000","last_clean":"2024-04-24T23:58:20.678625+0000","last_became_active":"2024-04-24T23:58:18.248532+0000","last_became_peered":"2024-04-24T23:58:18.248532+0000","last_unstale":"2024-04-24T23:58:20.678625+0000","last_undegraded":"2024-04-24T23:58:20.678625+0000","last_fullsized":"2024-04-24T23:58:20.678625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:21.987572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254293+0000","last_change":"2024-04-24T23:58:19.257617+0000","last_active":"2024-04-24T23:58:20.254293+0000","last_peered":"2024-04-24T23:58:20.254293+0000","last_clean":"2024-04-24T23:58:20.254293+0000","last_became_active":"2024-04-24T23:58:19.257312+0000","last_became_peered":"2024-04-24T23:58:19.257312+0000","last_unstale":"2024-04-24T23:58:20.254293+0000","last_undegraded":"2024-04-24T23:58:20.254293+0000","last_fullsized":"2024-04-24T23:58:20.254293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256119+0000","last_change":"2024-04-24T23:58:18.249582+0000","last_active":"2024-04-24T23:58:20.256119+0000","last_peered":"2024-04-24T23:58:20.256119+0000","last_clean":"2024-04-24T23:58:20.256119+0000","last_became_active":"2024-04-24T23:58:18.249277+0000","last_became_peered":"2024-04-24T23:58:18.249277+0000","last_unstale":"2024-04-24T23:58:20.256119+0000","last_undegraded":"2024-04-24T23:58:20.256119+0000","last_fullsized":"2024-04-24T23:58:20.256119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678235+0000","last_change":"2024-04-24T23:58:19.265003+0000","last_active":"2024-04-24T23:58:20.678235+0000","last_peered":"2024-04-24T23:58:20.678235+0000","last_clean":"2024-04-24T23:58:20.678235+0000","last_became_active":"2024-04-24T23:58:19.264714+0000","last_became_peered":"2024-04-24T23:58:19.264714+0000","last_unstale":"2024-04-24T23:58:20.678235+0000","last_undegraded":"2024-04-24T23:58:20.678235+0000","last_fullsized":"2024-04-24T23:58:20.678235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:39.651256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.257040+0000","last_change":"2024-04-24T23:58:18.250190+0000","last_active":"2024-04-24T23:58:20.257040+0000","last_peered":"2024-04-24T23:58:20.257040+0000","last_clean":"2024-04-24T23:58:20.257040+0000","last_became_active":"2024-04-24T23:58:18.249896+0000","last_became_peered":"2024-04-24T23:58:18.249896+0000","last_unstale":"2024-04-24T23:58:20.257040+0000","last_undegraded":"2024-04-24T23:58:20.257040+0000","last_fullsized":"2024-04-24T23:58:20.257040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:30.918198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255362+0000","last_change":"2024-04-24T23:58:19.261332+0000","last_active":"2024-04-24T23:58:20.255362+0000","last_peered":"2024-04-24T23:58:20.255362+0000","last_clean":"2024-04-24T23:58:20.255362+0000","last_became_active":"2024-04-24T23:58:19.261058+0000","last_became_peered":"2024-04-24T23:58:19.261058+0000","last_unstale":"2024-04-24T23:58:20.255362+0000","last_undegraded":"2024-04-24T23:58:20.255362+0000","last_fullsized":"2024-04-24T23:58:20.255362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:34.104087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677359+0000","last_change":"2024-04-24T23:58:18.251910+0000","last_active":"2024-04-24T23:58:20.677359+0000","last_peered":"2024-04-24T23:58:20.677359+0000","last_clean":"2024-04-24T23:58:20.677359+0000","last_became_active":"2024-04-24T23:58:18.251629+0000","last_became_peered":"2024-04-24T23:58:18.251629+0000","last_unstale":"2024-04-24T23:58:20.677359+0000","last_undegraded":"2024-04-24T23:58:20.677359+0000","last_fullsized":"2024-04-24T23:58:20.677359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255236+0000","last_change":"2024-04-24T23:58:19.256009+0000","last_active":"2024-04-24T23:58:20.255236+0000","last_peered":"2024-04-24T23:58:20.255236+0000","last_clean":"2024-04-24T23:58:20.255236+0000","last_became_active":"2024-04-24T23:58:19.255734+0000","last_became_peered":"2024-04-24T23:58:19.255734+0000","last_unstale":"2024-04-24T23:58:20.255236+0000","last_undegraded":"2024-04-24T23:58:20.255236+0000","last_fullsized":"2024-04-24T23:58:20.255236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:49.658803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254568+0000","last_change":"2024-04-24T23:58:18.250223+0000","last_active":"2024-04-24T23:58:20.254568+0000","last_peered":"2024-04-24T23:58:20.254568+0000","last_clean":"2024-04-24T23:58:20.254568+0000","last_became_active":"2024-04-24T23:58:18.249995+0000","last_became_peered":"2024-04-24T23:58:18.249995+0000","last_unstale":"2024-04-24T23:58:20.254568+0000","last_undegraded":"2024-04-24T23:58:20.254568+0000","last_fullsized":"2024-04-24T23:58:20.254568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678448+0000","last_change":"2024-04-24T23:58:19.258258+0000","last_active":"2024-04-24T23:58:20.678448+0000","last_peered":"2024-04-24T23:58:20.678448+0000","last_clean":"2024-04-24T23:58:20.678448+0000","last_became_active":"2024-04-24T23:58:19.258011+0000","last_became_peered":"2024-04-24T23:58:19.258011+0000","last_unstale":"2024-04-24T23:58:20.678448+0000","last_undegraded":"2024-04-24T23:58:20.678448+0000","last_fullsized":"2024-04-24T23:58:20.678448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254356+0000","last_change":"2024-04-24T23:58:18.243148+0000","last_active":"2024-04-24T23:58:20.254356+0000","last_peered":"2024-04-24T23:58:20.254356+0000","last_clean":"2024-04-24T23:58:20.254356+0000","last_became_active":"2024-04-24T23:58:18.242863+0000","last_became_peered":"2024-04-24T23:58:18.242863+0000","last_unstale":"2024-04-24T23:58:20.254356+0000","last_undegraded":"2024-04-24T23:58:20.254356+0000","last_fullsized":"2024-04-24T23:58:20.254356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:33:41.734978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678415+0000","last_change":"2024-04-24T23:58:19.268382+0000","last_active":"2024-04-24T23:58:20.678415+0000","last_peered":"2024-04-24T23:58:20.678415+0000","last_clean":"2024-04-24T23:58:20.678415+0000","last_became_active":"2024-04-24T23:58:19.268132+0000","last_became_peered":"2024-04-24T23:58:19.268132+0000","last_unstale":"2024-04-24T23:58:20.678415+0000","last_undegraded":"2024-04-24T23:58:20.678415+0000","last_fullsized":"2024-04-24T23:58:20.678415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:24.998845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678646+0000","last_change":"2024-04-24T23:58:18.253346+0000","last_active":"2024-04-24T23:58:20.678646+0000","last_peered":"2024-04-24T23:58:20.678646+0000","last_clean":"2024-04-24T23:58:20.678646+0000","last_became_active":"2024-04-24T23:58:18.253094+0000","last_became_peered":"2024-04-24T23:58:18.253094+0000","last_unstale":"2024-04-24T23:58:20.678646+0000","last_undegraded":"2024-04-24T23:58:20.678646+0000","last_fullsized":"2024-04-24T23:58:20.678646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:31.593070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.256888+0000","last_change":"2024-04-24T23:58:19.260209+0000","last_active":"2024-04-24T23:58:20.256888+0000","last_peered":"2024-04-24T23:58:20.256888+0000","last_clean":"2024-04-24T23:58:20.256888+0000","last_became_active":"2024-04-24T23:58:19.259941+0000","last_became_peered":"2024-04-24T23:58:19.259941+0000","last_unstale":"2024-04-24T23:58:20.256888+0000","last_undegraded":"2024-04-24T23:58:20.256888+0000","last_fullsized":"2024-04-24T23:58:20.256888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:14.717658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678038+0000","last_change":"2024-04-24T23:58:19.262656+0000","last_active":"2024-04-24T23:58:20.678038+0000","last_peered":"2024-04-24T23:58:20.678038+0000","last_clean":"2024-04-24T23:58:20.678038+0000","last_became_active":"2024-04-24T23:58:19.262043+0000","last_became_peered":"2024-04-24T23:58:19.262043+0000","last_unstale":"2024-04-24T23:58:20.678038+0000","last_undegraded":"2024-04-24T23:58:20.678038+0000","last_fullsized":"2024-04-24T23:58:20.678038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:08.754197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.677720+0000","last_change":"2024-04-24T23:58:18.249958+0000","last_active":"2024-04-24T23:58:20.677720+0000","last_peered":"2024-04-24T23:58:20.677720+0000","last_clean":"2024-04-24T23:58:20.677720+0000","last_became_active":"2024-04-24T23:58:18.249660+0000","last_became_peered":"2024-04-24T23:58:18.249660+0000","last_unstale":"2024-04-24T23:58:20.677720+0000","last_undegraded":"2024-04-24T23:58:20.677720+0000","last_fullsized":"2024-04-24T23:58:20.677720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:52.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678297+0000","last_change":"2024-04-24T23:58:19.263280+0000","last_active":"2024-04-24T23:58:20.678297+0000","last_peered":"2024-04-24T23:58:20.678297+0000","last_clean":"2024-04-24T23:58:20.678297+0000","last_became_active":"2024-04-24T23:58:19.262976+0000","last_became_peered":"2024-04-24T23:58:19.262976+0000","last_unstale":"2024-04-24T23:58:20.678297+0000","last_undegraded":"2024-04-24T23:58:20.678297+0000","last_fullsized":"2024-04-24T23:58:20.678297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:18.306746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678466+0000","last_change":"2024-04-24T23:58:18.250117+0000","last_active":"2024-04-24T23:58:20.678466+0000","last_peered":"2024-04-24T23:58:20.678466+0000","last_clean":"2024-04-24T23:58:20.678466+0000","last_became_active":"2024-04-24T23:58:18.249797+0000","last_became_peered":"2024-04-24T23:58:18.249797+0000","last_unstale":"2024-04-24T23:58:20.678466+0000","last_undegraded":"2024-04-24T23:58:20.678466+0000","last_fullsized":"2024-04-24T23:58:20.678466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:48.655189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.254736+0000","last_change":"2024-04-24T23:58:19.253506+0000","last_active":"2024-04-24T23:58:20.254736+0000","last_peered":"2024-04-24T23:58:20.254736+0000","last_clean":"2024-04-24T23:58:20.254736+0000","last_became_active":"2024-04-24T23:58:19.253032+0000","last_became_peered":"2024-04-24T23:58:19.253032+0000","last_unstale":"2024-04-24T23:58:20.254736+0000","last_undegraded":"2024-04-24T23:58:20.254736+0000","last_fullsized":"2024-04-24T23:58:20.254736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:14.910044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678152+0000","last_change":"2024-04-24T23:58:18.244986+0000","last_active":"2024-04-24T23:58:20.678152+0000","last_peered":"2024-04-24T23:58:20.678152+0000","last_clean":"2024-04-24T23:58:20.678152+0000","last_became_active":"2024-04-24T23:58:18.244674+0000","last_became_peered":"2024-04-24T23:58:18.244674+0000","last_unstale":"2024-04-24T23:58:20.678152+0000","last_undegraded":"2024-04-24T23:58:20.678152+0000","last_fullsized":"2024-04-24T23:58:20.678152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:47.097835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.678003+0000","last_change":"2024-04-24T23:58:19.261722+0000","last_active":"2024-04-24T23:58:20.678003+0000","last_peered":"2024-04-24T23:58:20.678003+0000","last_clean":"2024-04-24T23:58:20.678003+0000","last_became_active":"2024-04-24T23:58:19.261474+0000","last_became_peered":"2024-04-24T23:58:19.261474+0000","last_unstale":"2024-04-24T23:58:20.678003+0000","last_undegraded":"2024-04-24T23:58:20.678003+0000","last_fullsized":"2024-04-24T23:58:20.678003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:18.227192+0000","last_clean_scrub_stamp":"2024-04-24T23:58:18.227192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:20.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T23:58:20.255072+0000","last_change":"2024-04-24T23:58:18.240520+0000","last_active":"2024-04-24T23:58:20.255072+0000","last_peered":"2024-04-24T23:58:20.255072+0000","last_clean":"2024-04-24T23:58:20.255072+0000","last_became_active":"2024-04-24T23:58:18.240253+0000","last_became_peered":"2024-04-24T23:58:18.240253+0000","last_unstale":"2024-04-24T23:58:20.255072+0000","last_undegraded":"2024-04-24T23:58:20.255072+0000","last_fullsized":"2024-04-24T23:58:20.255072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:17.223578+0000","last_clean_scrub_stamp":"2024-04-24T23:58:17.223578+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:17.091843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:58:30.092 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T23:58:30.092 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:58:30.092 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:58:30.092 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T23:58:30.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.224+0000 7f9c8ef7e640 1 Processor -- start 2024-04-24T23:58:30.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.224+0000 7f9c8ef7e640 1 -- start start 2024-04-24T23:58:30.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8ef7e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c8815a6b0 0x7f9c88070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8ef7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8ef7e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c880b2d50 con 0x7f9c88071780 2024-04-24T23:58:30.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8ef7e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9c88072a70 con 0x7f9c8815a6b0 2024-04-24T23:58:30.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8ef7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c880b3010 con 0x7f9c88072ea0 2024-04-24T23:58:30.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8cd1a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c8815a6b0 0x7f9c88070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8d51b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9c88071780 0x7f9c8815e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55560/0 (socket says 172.21.15.113:55560) 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8d51b640 1 -- 172.21.15.113:0/3105155156 learned_addr learned my addr 172.21.15.113:0/3105155156 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8cd1a640 1 -- 172.21.15.113:0/3105155156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 msgr2=0x7f9c88071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c87fff640 1 --2- 172.21.15.113:0/3105155156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c877fe640 1 -- 172.21.15.113:0/3105155156 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538690843 0 0) 0x7f9c880b2d50 con 0x7f9c88071780 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8cd1a640 1 --2- 172.21.15.113:0/3105155156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8cd1a640 1 -- 172.21.15.113:0/3105155156 >> v1:172.21.15.113:6789/0 conn(0x7f9c88071780 legacy=0x7f9c8815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c8cd1a640 1 -- 172.21.15.113:0/3105155156 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c880b32f0 con 0x7f9c8815a6b0 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.225+0000 7f9c87fff640 1 --2- 172.21.15.113:0/3105155156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.226+0000 7f9c8cd1a640 1 --2- 172.21.15.113:0/3105155156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8815a6b0 0x7f9c88070d00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800a940 tx=0x7f9c7802b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6263d8d888d52540 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.226+0000 7f9c877fe640 1 -- 172.21.15.113:0/3105155156 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800b040 con 0x7f9c8815a6b0 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.226+0000 7f9c877fe640 1 -- 172.21.15.113:0/3105155156 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c78030710 con 0x7f9c8815a6b0 2024-04-24T23:58:30.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.226+0000 7f9c877fe640 1 -- 172.21.15.113:0/3105155156 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78030ae0 con 0x7f9c8815a6b0 2024-04-24T23:58:30.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3105155156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8815a6b0 msgr2=0x7f9c88070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3105155156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8815a6b0 0x7f9c88070d00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800a940 tx=0x7f9c7802b550 comp rx=0 tx=0).stop 2024-04-24T23:58:30.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3105155156 shutdown_connections 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3105155156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3105155156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8815a6b0 0x7f9c88070d00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3105155156 >> 172.21.15.113:0/3105155156 conn(0x7f9c880b1a50 msgr2=0x7f9c880b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3105155156 shutdown_connections 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.227+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3105155156 wait complete. 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.228+0000 7f9c8ef7e640 1 Processor -- start 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.228+0000 7f9c8ef7e640 1 -- start start 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88071780 0x7f9c88153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8815a6b0 0x7f9c88151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8815aa90 con 0x7f9c88071780 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88072720 con 0x7f9c8815a6b0 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8815ac10 con 0x7f9c88072ea0 2024-04-24T23:58:30.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88071780 0x7f9c88153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88071780 0x7f9c88153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50368/0 (socket says 172.21.15.113:50368) 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 -- 172.21.15.113:0/3920685399 learned_addr learned my addr 172.21.15.113:0/3920685399 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8d51b640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8815a6b0 0x7f9c88151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c87fff640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 -- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 msgr2=0x7f9c88154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 -- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8815a6b0 msgr2=0x7f9c88151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8815a6b0 0x7f9c88151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 -- 172.21.15.113:0/3920685399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c881523e0 con 0x7f9c88071780 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8cd1a640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88071780 0x7f9c88153480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000c00 tx=0x7f9c78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c857fa640 1 -- 172.21.15.113:0/3920685399 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78032d40 con 0x7f9c88071780 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c857fa640 1 -- 172.21.15.113:0/3920685399 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c78040070 con 0x7f9c88071780 2024-04-24T23:58:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c881526c0 con 0x7f9c88071780 2024-04-24T23:58:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c857fa640 1 -- 172.21.15.113:0/3920685399 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7803b6a0 con 0x7f9c88071780 2024-04-24T23:58:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.229+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c88152c80 con 0x7f9c88071780 2024-04-24T23:58:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.231+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c880cc370 con 0x7f9c88071780 2024-04-24T23:58:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.231+0000 7f9c857fa640 1 -- 172.21.15.113:0/3920685399 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c7803b840 con 0x7f9c88071780 2024-04-24T23:58:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.231+0000 7f9c857fa640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7406be30 0x7f9c7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.231+0000 7f9c87fff640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7406be30 0x7f9c7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.231+0000 7f9c857fa640 1 -- 172.21.15.113:0/3920685399 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9c780ab8d0 con 0x7f9c88071780 2024-04-24T23:58:30.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.232+0000 7f9c87fff640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7406be30 0x7f9c7406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9c880732a0 tx=0x7f9c7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.236+0000 7f9c857fa640 1 -- 172.21.15.113:0/3920685399 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c880cc370 con 0x7f9c88071780 2024-04-24T23:58:30.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.447+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9c880691a0 con 0x7f9c88071780 2024-04-24T23:58:30.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.447+0000 7f9c857fa640 1 -- 172.21.15.113:0/3920685399 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9c7807a310 con 0x7f9c88071780 2024-04-24T23:58:30.447 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:30.448 INFO:teuthology.orchestra.run.smithi113.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7406be30 msgr2=0x7f9c7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7406be30 0x7f9c7406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9c880732a0 tx=0x7f9c7c008040 comp rx=0 tx=0).stop 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88071780 msgr2=0x7f9c88153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88071780 0x7f9c88153480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000c00 tx=0x7f9c78004730 comp rx=0 tx=0).stop 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 shutdown_connections 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7406be30 0x7f9c7406e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8815a6b0 0x7f9c88151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c88072ea0 0x7f9c88154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 --2- 172.21.15.113:0/3920685399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88071780 0x7f9c88153480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 >> 172.21.15.113:0/3920685399 conn(0x7f9c880b1a50 msgr2=0x7f9c880b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 shutdown_connections 2024-04-24T23:58:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.449+0000 7f9c8ef7e640 1 -- 172.21.15.113:0/3920685399 wait complete. 2024-04-24T23:58:30.460 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:58:30.460 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:30.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.556+0000 7fc0455a5640 1 Processor -- start 2024-04-24T23:58:30.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.556+0000 7fc0455a5640 1 -- start start 2024-04-24T23:58:30.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc0455a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc04015c130 0x7fc0400d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc0455a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400d09b0 0x7fc0400b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc0455a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0400b2080 0x7fc0400ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc0455a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc0400cfe10 con 0x7fc0400b2080 2024-04-24T23:58:30.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc0455a5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc0400ccdd0 con 0x7fc04015c130 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc0455a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc0400d0df0 con 0x7fc0400d09b0 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc03f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0400b2080 0x7fc0400ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc03f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0400b2080 0x7fc0400ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50374/0 (socket says 172.21.15.113:50374) 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc03f7fe640 1 -- 172.21.15.113:0/1502624777 learned_addr learned my addr 172.21.15.113:0/1502624777 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc03effd640 1 --2- 172.21.15.113:0/1502624777 >> v2:172.21.15.178:3300/0 conn(0x7fc04015c130 0x7fc0400d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.557+0000 7fc03e7fc640 1 --2- 172.21.15.113:0/1502624777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400d09b0 0x7fc0400b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03f7fe640 1 -- 172.21.15.113:0/1502624777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400d09b0 msgr2=0x7fc0400b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03f7fe640 1 --2- 172.21.15.113:0/1502624777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400d09b0 0x7fc0400b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03f7fe640 1 -- 172.21.15.113:0/1502624777 >> v2:172.21.15.178:3300/0 conn(0x7fc04015c130 msgr2=0x7fc0400d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03f7fe640 1 --2- 172.21.15.113:0/1502624777 >> v2:172.21.15.178:3300/0 conn(0x7fc04015c130 0x7fc0400d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03f7fe640 1 -- 172.21.15.113:0/1502624777 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0400ccac0 con 0x7fc0400b2080 2024-04-24T23:58:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03effd640 1 --2- 172.21.15.113:0/1502624777 >> v2:172.21.15.178:3300/0 conn(0x7fc04015c130 0x7fc0400d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03f7fe640 1 --2- 172.21.15.113:0/1502624777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400b2080 0x7fc0400ce8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc0400d2d70 tx=0x7fc03802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5581dffd2d2ad616 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03dffb640 1 -- 172.21.15.113:0/1502624777 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038037070 con 0x7fc0400b2080 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03dffb640 1 -- 172.21.15.113:0/1502624777 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038032440 con 0x7fc0400b2080 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.558+0000 7fc03dffb640 1 -- 172.21.15.113:0/1502624777 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038032810 con 0x7fc0400b2080 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 -- 172.21.15.113:0/1502624777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400b2080 msgr2=0x7fc0400ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 --2- 172.21.15.113:0/1502624777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400b2080 0x7fc0400ce8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc0400d2d70 tx=0x7fc03802a270 comp rx=0 tx=0).stop 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 -- 172.21.15.113:0/1502624777 shutdown_connections 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 --2- 172.21.15.113:0/1502624777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400b2080 0x7fc0400ce8a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 --2- 172.21.15.113:0/1502624777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400d09b0 0x7fc0400b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 --2- 172.21.15.113:0/1502624777 >> v2:172.21.15.178:3300/0 conn(0x7fc04015c130 0x7fc0400d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 -- 172.21.15.113:0/1502624777 >> 172.21.15.113:0/1502624777 conn(0x7fc04009aa20 msgr2=0x7fc04009ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 -- 172.21.15.113:0/1502624777 shutdown_connections 2024-04-24T23:58:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.559+0000 7fc0455a5640 1 -- 172.21.15.113:0/1502624777 wait complete. 2024-04-24T23:58:30.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.560+0000 7fc0455a5640 1 Processor -- start 2024-04-24T23:58:30.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.560+0000 7fc0455a5640 1 -- start start 2024-04-24T23:58:30.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc0455a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400b2080 0x7fc0400b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc0455a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400d09b0 0x7fc04006f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc0455a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04015c130 0x7fc0400af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc0455a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400b2a70 con 0x7fc0400d09b0 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc0455a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400b2510 con 0x7fc04015c130 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc0455a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc0400b2d50 con 0x7fc0400b2080 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400d09b0 0x7fc04006f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400d09b0 0x7fc04006f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50388/0 (socket says 172.21.15.113:50388) 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 -- 172.21.15.113:0/541411365 learned_addr learned my addr 172.21.15.113:0/541411365 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03f7fe640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04015c130 0x7fc0400af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03effd640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400b2080 0x7fc0400b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 -- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400b2080 msgr2=0x7fc0400b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400b2080 0x7fc0400b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 -- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04015c130 msgr2=0x7fc0400af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04015c130 0x7fc0400af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 -- 172.21.15.113:0/541411365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0401f1d50 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03f7fe640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04015c130 0x7fc0400af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.561+0000 7fc03e7fc640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400d09b0 0x7fc04006f7c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d800 tx=0x7fc03400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.562+0000 7fc0277fe640 1 -- 172.21.15.113:0/541411365 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03400b6c0 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.562+0000 7fc0277fe640 1 -- 172.21.15.113:0/541411365 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034019070 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.562+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0401f2030 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.562+0000 7fc0277fe640 1 -- 172.21.15.113:0/541411365 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03400ba80 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.562+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0401f1a40 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.563+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc040068220 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.563+0000 7fc0277fe640 1 -- 172.21.15.113:0/541411365 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc03400bc20 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.563+0000 7fc0277fe640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc02c06be30 0x7fc02c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.563+0000 7fc0277fe640 1 -- 172.21.15.113:0/541411365 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc03408b200 con 0x7fc0400d09b0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.563+0000 7fc03effd640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc02c06be30 0x7fc02c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.564+0000 7fc03effd640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc02c06be30 0x7fc02c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc030003aa0 tx=0x7fc030009870 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.568+0000 7fc0277fe640 1 -- 172.21.15.113:0/541411365 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc034059c40 con 0x7fc0400d09b0 2024-04-24T23:58:30.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.741+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0400cfbf0 con 0x7fc0400d09b0 2024-04-24T23:58:30.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.742+0000 7fc0277fe640 1 -- 172.21.15.113:0/541411365 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fc034059540 con 0x7fc0400d09b0 2024-04-24T23:58:30.743 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:30.743 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc02c06be30 msgr2=0x7fc02c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc02c06be30 0x7fc02c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc030003aa0 tx=0x7fc030009870 comp rx=0 tx=0).stop 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400d09b0 msgr2=0x7fc04006f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400d09b0 0x7fc04006f7c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d800 tx=0x7fc03400dcc0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 shutdown_connections 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc02c06be30 0x7fc02c06e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04015c130 0x7fc0400af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0400d09b0 0x7fc04006f7c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 --2- 172.21.15.113:0/541411365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0400b2080 0x7fc0400b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 >> 172.21.15.113:0/541411365 conn(0x7fc04009aa20 msgr2=0x7fc0400ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:30.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 shutdown_connections 2024-04-24T23:58:30.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.744+0000 7fc0455a5640 1 -- 172.21.15.113:0/541411365 wait complete. 2024-04-24T23:58:30.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:30.756 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:30.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.854+0000 7fb662a8c640 1 Processor -- start 2024-04-24T23:58:30.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.854+0000 7fb662a8c640 1 -- start start 2024-04-24T23:58:30.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb662a8c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb65c15e2d0 0x7fb65c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb662a8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b4a40 0x7fb65c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb662a8c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb65c15f4f0 con 0x7fb65c0b3080 2024-04-24T23:58:30.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb662a8c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb65c0cde00 con 0x7fb65c15e2d0 2024-04-24T23:58:30.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb662a8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb65c15f7d0 con 0x7fb65c0b4a40 2024-04-24T23:58:30.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb660828640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb65c15e2d0 0x7fb65c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb661029640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b4a40 0x7fb65c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb660828640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb65c15e2d0 0x7fb65c15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41852/0 (socket says 172.21.15.113:41852) 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb661029640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b4a40 0x7fb65c0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:37856/0 (socket says 172.21.15.113:37856) 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb660828640 1 -- 172.21.15.113:0/966884415 learned_addr learned my addr 172.21.15.113:0/966884415 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb65bfff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb65c0b3080 0x7fb65c0b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:56052/0 (socket says 172.21.15.113:56052) 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb661029640 1 -- 172.21.15.113:0/966884415 >> v2:172.21.15.178:3300/0 conn(0x7fb65c15e2d0 msgr2=0x7fb65c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb661029640 1 --2- 172.21.15.113:0/966884415 >> v2:172.21.15.178:3300/0 conn(0x7fb65c15e2d0 0x7fb65c15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb661029640 1 -- 172.21.15.113:0/966884415 >> v1:172.21.15.178:6789/0 conn(0x7fb65c0b3080 legacy=0x7fb65c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb661029640 1 -- 172.21.15.113:0/966884415 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c15fab0 con 0x7fb65c0b4a40 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.855+0000 7fb660828640 1 --2- 172.21.15.113:0/966884415 >> v2:172.21.15.178:3300/0 conn(0x7fb65c15e2d0 0x7fb65c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:30.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.856+0000 7fb661029640 1 --2- 172.21.15.113:0/966884415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b4a40 0x7fb65c0ce290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb654009d90 tx=0x7fb65402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd10f950967ff7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.856+0000 7fb65b7fe640 1 -- 172.21.15.113:0/966884415 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654007740 con 0x7fb65c0b4a40 2024-04-24T23:58:30.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.856+0000 7fb65b7fe640 1 -- 172.21.15.113:0/966884415 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6540078e0 con 0x7fb65c0b4a40 2024-04-24T23:58:30.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.856+0000 7fb65b7fe640 1 -- 172.21.15.113:0/966884415 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654007cb0 con 0x7fb65c0b4a40 2024-04-24T23:58:30.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 -- 172.21.15.113:0/966884415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b4a40 msgr2=0x7fb65c0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 --2- 172.21.15.113:0/966884415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b4a40 0x7fb65c0ce290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb654009d90 tx=0x7fb65402a660 comp rx=0 tx=0).stop 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 -- 172.21.15.113:0/966884415 shutdown_connections 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 --2- 172.21.15.113:0/966884415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b4a40 0x7fb65c0ce290 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 --2- 172.21.15.113:0/966884415 >> v2:172.21.15.178:3300/0 conn(0x7fb65c15e2d0 0x7fb65c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 -- 172.21.15.113:0/966884415 >> 172.21.15.113:0/966884415 conn(0x7fb65c06b0f0 msgr2=0x7fb65c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 -- 172.21.15.113:0/966884415 shutdown_connections 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.857+0000 7fb662a8c640 1 -- 172.21.15.113:0/966884415 wait complete. 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.858+0000 7fb662a8c640 1 Processor -- start 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.858+0000 7fb662a8c640 1 -- start start 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb662a8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b3080 0x7fb65c14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb662a8c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb65c0b4a40 0x7fb65c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb662a8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c15e2d0 0x7fb65c141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb662a8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c15f2b0 con 0x7fb65c15e2d0 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb662a8c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c15f130 con 0x7fb65c0b4a40 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb662a8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb65c15e6b0 con 0x7fb65c0b3080 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c15e2d0 0x7fb65c141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c15e2d0 0x7fb65c141b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50404/0 (socket says 172.21.15.113:50404) 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 -- 172.21.15.113:0/307959049 learned_addr learned my addr 172.21.15.113:0/307959049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb65bfff640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb65c0b4a40 0x7fb65c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb660828640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b3080 0x7fb65c14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 -- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b3080 msgr2=0x7fb65c14cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b3080 0x7fb65c14cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 -- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb65c0b4a40 msgr2=0x7fb65c155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:30.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb65c0b4a40 0x7fb65c155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 -- 172.21.15.113:0/307959049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c14cd50 con 0x7fb65c15e2d0 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.859+0000 7fb661029640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c15e2d0 0x7fb65c141b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb654009eb0 tx=0x7fb654004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.860+0000 7fb6597fa640 1 -- 172.21.15.113:0/307959049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654040070 con 0x7fb65c15e2d0 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.860+0000 7fb6597fa640 1 -- 172.21.15.113:0/307959049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb654031bc0 con 0x7fb65c15e2d0 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.860+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c1e5520 con 0x7fb65c15e2d0 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.860+0000 7fb6597fa640 1 -- 172.21.15.113:0/307959049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654009450 con 0x7fb65c15e2d0 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.860+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c1f1840 con 0x7fb65c15e2d0 2024-04-24T23:58:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.861+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65c0691a0 con 0x7fb65c15e2d0 2024-04-24T23:58:30.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.861+0000 7fb6597fa640 1 -- 172.21.15.113:0/307959049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb654039040 con 0x7fb65c15e2d0 2024-04-24T23:58:30.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.862+0000 7fb6597fa640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb64006be30 0x7fb64006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:30.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.862+0000 7fb6597fa640 1 -- 172.21.15.113:0/307959049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6540ac560 con 0x7fb65c15e2d0 2024-04-24T23:58:30.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.862+0000 7fb660828640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb64006be30 0x7fb64006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:30.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.862+0000 7fb660828640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb64006be30 0x7fb64006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb650006ca0 tx=0x7fb650009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:30.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:30.866+0000 7fb6597fa640 1 -- 172.21.15.113:0/307959049 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb65407a8c0 con 0x7fb65c15e2d0 2024-04-24T23:58:31.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.039+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb65c0684f0 con 0x7fb65c15e2d0 2024-04-24T23:58:31.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.039+0000 7fb6597fa640 1 -- 172.21.15.113:0/307959049 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb65407ebc0 con 0x7fb65c15e2d0 2024-04-24T23:58:31.039 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb64006be30 msgr2=0x7fb64006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb64006be30 0x7fb64006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb650006ca0 tx=0x7fb650009240 comp rx=0 tx=0).stop 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c15e2d0 msgr2=0x7fb65c141b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c15e2d0 0x7fb65c141b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb654009eb0 tx=0x7fb654004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 shutdown_connections 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb64006be30 0x7fb64006e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb65c0b4a40 0x7fb65c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c15e2d0 0x7fb65c141b50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 --2- 172.21.15.113:0/307959049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb65c0b3080 0x7fb65c14cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 >> 172.21.15.113:0/307959049 conn(0x7fb65c06b0f0 msgr2=0x7fb65c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 shutdown_connections 2024-04-24T23:58:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.041+0000 7fb662a8c640 1 -- 172.21.15.113:0/307959049 wait complete. 2024-04-24T23:58:31.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:31.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.152+0000 7efe25631640 1 Processor -- start 2024-04-24T23:58:31.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.152+0000 7efe25631640 1 -- start start 2024-04-24T23:58:31.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe25631640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efe200cf7a0 0x7efe200d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe25631640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe200d2a20 0x7efe200d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe25631640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe20162560 con 0x7efe200cf3c0 2024-04-24T23:58:31.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe25631640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efe20162840 con 0x7efe200d2a20 2024-04-24T23:58:31.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe25631640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efe20162b20 con 0x7efe200cf7a0 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe1f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe200d2a20 0x7efe200d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe1f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe200d2a20 0x7efe200d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50410/0 (socket says 172.21.15.113:50410) 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe1f7fe640 1 -- 172.21.15.113:0/4268192188 learned_addr learned my addr 172.21.15.113:0/4268192188 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe167fc640 1 --2- 172.21.15.113:0/4268192188 >> v2:172.21.15.178:3300/0 conn(0x7efe200cf7a0 0x7efe200d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe1f7fe640 1 -- 172.21.15.113:0/4268192188 >> v2:172.21.15.178:3300/0 conn(0x7efe200cf7a0 msgr2=0x7efe200d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe1f7fe640 1 --2- 172.21.15.113:0/4268192188 >> v2:172.21.15.178:3300/0 conn(0x7efe200cf7a0 0x7efe200d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe1f7fe640 1 -- 172.21.15.113:0/4268192188 >> v1:172.21.15.178:6789/0 conn(0x7efe200cf3c0 legacy=0x7efe200d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.153+0000 7efe1f7fe640 1 -- 172.21.15.113:0/4268192188 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe20162e00 con 0x7efe200d2a20 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.154+0000 7efe1f7fe640 1 --2- 172.21.15.113:0/4268192188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200d2a20 0x7efe200d1210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe20161f00 tx=0x7efe1802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec9d586ce9d87e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.154+0000 7efe1e7fc640 1 -- 172.21.15.113:0/4268192188 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18036070 con 0x7efe200d2a20 2024-04-24T23:58:31.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.154+0000 7efe1e7fc640 1 -- 172.21.15.113:0/4268192188 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe18031600 con 0x7efe200d2a20 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.154+0000 7efe1e7fc640 1 -- 172.21.15.113:0/4268192188 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe180319d0 con 0x7efe200d2a20 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 -- 172.21.15.113:0/4268192188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200d2a20 msgr2=0x7efe200d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 --2- 172.21.15.113:0/4268192188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200d2a20 0x7efe200d1210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe20161f00 tx=0x7efe1802a6c0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 -- 172.21.15.113:0/4268192188 shutdown_connections 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 --2- 172.21.15.113:0/4268192188 >> v2:172.21.15.178:3300/0 conn(0x7efe200cf7a0 0x7efe200d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 --2- 172.21.15.113:0/4268192188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200d2a20 0x7efe200d1210 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 -- 172.21.15.113:0/4268192188 >> 172.21.15.113:0/4268192188 conn(0x7efe200718f0 msgr2=0x7efe20071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 -- 172.21.15.113:0/4268192188 shutdown_connections 2024-04-24T23:58:31.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.155+0000 7efe25631640 1 -- 172.21.15.113:0/4268192188 wait complete. 2024-04-24T23:58:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.156+0000 7efe25631640 1 Processor -- start 2024-04-24T23:58:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.156+0000 7efe25631640 1 -- start start 2024-04-24T23:58:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe25631640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe200cf3c0 0x7efe20154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe25631640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200cf7a0 0x7efe20156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe25631640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe200d2a20 0x7efe20157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe25631640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe200d1750 con 0x7efe200cf7a0 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe25631640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe201621d0 con 0x7efe200cf3c0 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe25631640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe200d18d0 con 0x7efe200d2a20 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200cf7a0 0x7efe20156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200cf7a0 0x7efe20156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50416/0 (socket says 172.21.15.113:50416) 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 -- 172.21.15.113:0/198438860 learned_addr learned my addr 172.21.15.113:0/198438860 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe1effd640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe200cf3c0 0x7efe20154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe1f7fe640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe200d2a20 0x7efe20157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 -- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe200d2a20 msgr2=0x7efe20157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe200d2a20 0x7efe20157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 -- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe200cf3c0 msgr2=0x7efe20154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe200cf3c0 0x7efe20154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 -- 172.21.15.113:0/198438860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe20155360 con 0x7efe200cf7a0 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe167fc640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200cf7a0 0x7efe20156e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d800 tx=0x7efe0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe17fff640 1 -- 172.21.15.113:0/198438860 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0800b6c0 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe17fff640 1 -- 172.21.15.113:0/198438860 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe08019070 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe17fff640 1 -- 172.21.15.113:0/198438860 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0800ba80 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.157+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe200b74b0 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.158+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe200b7a70 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.158+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe200691a0 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.159+0000 7efe17fff640 1 -- 172.21.15.113:0/198438860 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe0800bc20 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.159+0000 7efe17fff640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0006be30 0x7efe0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.159+0000 7efe1effd640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0006be30 0x7efe0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.159+0000 7efe17fff640 1 -- 172.21.15.113:0/198438860 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efe0808b180 con 0x7efe200cf7a0 2024-04-24T23:58:31.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.160+0000 7efe1effd640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0006be30 0x7efe0006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe201602c0 tx=0x7efe10007e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.164+0000 7efe17fff640 1 -- 172.21.15.113:0/198438860 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe080594e0 con 0x7efe200cf7a0 2024-04-24T23:58:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.340+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe20158070 con 0x7efe200cf7a0 2024-04-24T23:58:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.340+0000 7efe17fff640 1 -- 172.21.15.113:0/198438860 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7efe0805d7e0 con 0x7efe200cf7a0 2024-04-24T23:58:31.340 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.342+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0006be30 msgr2=0x7efe0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.342+0000 7efe25631640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0006be30 0x7efe0006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe201602c0 tx=0x7efe10007e30 comp rx=0 tx=0).stop 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.342+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200cf7a0 msgr2=0x7efe20156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.342+0000 7efe25631640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200cf7a0 0x7efe20156e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d800 tx=0x7efe0800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 shutdown_connections 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0006be30 0x7efe0006e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe200d2a20 0x7efe20157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe200cf7a0 0x7efe20156e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 --2- 172.21.15.113:0/198438860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe200cf3c0 0x7efe20154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 >> 172.21.15.113:0/198438860 conn(0x7efe200718f0 msgr2=0x7efe200b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 shutdown_connections 2024-04-24T23:58:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.343+0000 7efe25631640 1 -- 172.21.15.113:0/198438860 wait complete. 2024-04-24T23:58:31.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:31.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.453+0000 7f78a2815640 1 Processor -- start 2024-04-24T23:58:31.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f78a2815640 1 -- start start 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f78a2815640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 0x7f789c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f78a2815640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f789c0b73c0 0x7f789c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f78a2815640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789c0d2e00 con 0x7f789c15e510 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f78a2815640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f789c0b6640 con 0x7f789c0b73c0 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f78a2815640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f789c0d30e0 con 0x7f789c0b5ca0 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f789bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f789c15e510 0x7f789c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55564/0 (socket says 172.21.15.113:55564) 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.454+0000 7f789bfff640 1 -- 172.21.15.113:0/1778126123 learned_addr learned my addr 172.21.15.113:0/1778126123 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:31.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452920282 0 0) 0x7f789c0d2e00 con 0x7f789c15e510 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7884003520 con 0x7f789c15e510 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f78a0db2640 1 --2- 172.21.15.113:0/1778126123 >> v2:172.21.15.178:3300/0 conn(0x7f789c0b73c0 0x7f789c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789b7fe640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 0x7f789c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4046674861 0 0) 0x7f7884003520 con 0x7f789c15e510 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 msgr2=0x7f789c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 0x7f789c0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 >> v2:172.21.15.178:3300/0 conn(0x7f789c0b73c0 msgr2=0x7f789c0b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 --2- 172.21.15.113:0/1778126123 >> v2:172.21.15.178:3300/0 conn(0x7f789c0b73c0 0x7f789c0b7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c0d33c0 con 0x7f789c15e510 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f788c001670 con 0x7f789c15e510 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 >> v1:172.21.15.113:6789/0 conn(0x7f789c15e510 legacy=0x7f789c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78840039b0 0x7f7884003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f7884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 0x7f78840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f789c0d30e0 con 0x7f7884004800 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f788400b9c0 con 0x7f78840039b0 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f788400bb40 con 0x7f789c0b5ca0 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789b7fe640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f7884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789bfff640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78840039b0 0x7f7884003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f78a0db2640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 0x7f78840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789b7fe640 1 -- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 msgr2=0x7f78840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789b7fe640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 0x7f78840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789b7fe640 1 -- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78840039b0 msgr2=0x7f7884003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789b7fe640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78840039b0 0x7f7884003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f789b7fe640 1 -- 172.21.15.113:0/1778126123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7884003520 con 0x7f7884004800 2024-04-24T23:58:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.455+0000 7f78a0db2640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c0b5ca0 0x7f78840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.456+0000 7f789b7fe640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f7884004c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7890009af0 tx=0x7f7884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d932c590c3c9be32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.456+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890036070 con 0x7f7884004800 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.456+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7890031400 con 0x7f7884004800 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.456+0000 7f789affd640 1 -- 172.21.15.113:0/1778126123 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890009690 con 0x7f7884004800 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.457+0000 7f78a2815640 1 -- 172.21.15.113:0/1778126123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 msgr2=0x7f7884004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.457+0000 7f78a2815640 1 --2- 172.21.15.113:0/1778126123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f7884004c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7890009af0 tx=0x7f7884005830 comp rx=0 tx=0).stop 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.457+0000 7f78a0db2640 1 -- 172.21.15.113:0/1778126123 reap_dead start 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.457+0000 7f78a2815640 1 -- 172.21.15.113:0/1778126123 shutdown_connections 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.457+0000 7f78a2815640 1 -- 172.21.15.113:0/1778126123 >> 172.21.15.113:0/1778126123 conn(0x7f789c06b720 msgr2=0x7f789c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.457+0000 7f78a2815640 1 -- 172.21.15.113:0/1778126123 shutdown_connections 2024-04-24T23:58:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.457+0000 7f78a2815640 1 -- 172.21.15.113:0/1778126123 wait complete. 2024-04-24T23:58:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.458+0000 7f78a2815640 1 Processor -- start 2024-04-24T23:58:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.458+0000 7f78a2815640 1 -- start start 2024-04-24T23:58:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f789c1f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f789c1f6790 0x7f789c1f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c15f600 0x7f789c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f789c0d3ea0 con 0x7f7884004800 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f789c0d3980 con 0x7f789c1f6790 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f789c0d2a90 con 0x7f789c15f600 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f789c1f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f789c1f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50436/0 (socket says 172.21.15.113:50436) 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 -- 172.21.15.113:0/2684249536 learned_addr learned my addr 172.21.15.113:0/2684249536 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789b7fe640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f789c1f6790 0x7f789c1f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 -- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c15f600 msgr2=0x7f789c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a0db2640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c15f600 0x7f789c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c15f600 0x7f789c15fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 -- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f789c1f6790 msgr2=0x7f789c1f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f789c1f6790 0x7f789c1f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 -- 172.21.15.113:0/2684249536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c160380 con 0x7f7884004800 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789b7fe640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f789c1f6790 0x7f789c1f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f789bfff640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f789c1f75c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f788c007340 tx=0x7f788c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f7898ff9640 1 -- 172.21.15.113:0/2684249536 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00f9c0 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f7898ff9640 1 -- 172.21.15.113:0/2684249536 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788c018070 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c14eaa0 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f7898ff9640 1 -- 172.21.15.113:0/2684249536 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00fda0 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.459+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c14f060 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.460+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f789c0691a0 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.461+0000 7f7898ff9640 1 -- 172.21.15.113:0/2684249536 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f788c01d460 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.461+0000 7f7898ff9640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f788806be30 0x7f788806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.461+0000 7f7898ff9640 1 -- 172.21.15.113:0/2684249536 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f788c08b050 con 0x7f7884004800 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.461+0000 7f789b7fe640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f788806be30 0x7f788806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.461+0000 7f789b7fe640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f788806be30 0x7f788806e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7890033000 tx=0x7f7890008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.465+0000 7f7898ff9640 1 -- 172.21.15.113:0/2684249536 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f788c059970 con 0x7f7884004800 2024-04-24T23:58:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.636+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f789c0d24b0 con 0x7f7884004800 2024-04-24T23:58:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.636+0000 7f7898ff9640 1 -- 172.21.15.113:0/2684249536 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f788c059270 con 0x7f7884004800 2024-04-24T23:58:31.636 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f788806be30 msgr2=0x7f788806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f788806be30 0x7f788806e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7890033000 tx=0x7f7890008030 comp rx=0 tx=0).stop 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 msgr2=0x7f789c1f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f789c1f75c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f788c007340 tx=0x7f788c007800 comp rx=0 tx=0).stop 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 shutdown_connections 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f788806be30 0x7f788806e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f789c1f6790 0x7f789c1f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789c15f600 0x7f789c15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 --2- 172.21.15.113:0/2684249536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7884004800 0x7f789c1f75c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 >> 172.21.15.113:0/2684249536 conn(0x7f789c06b720 msgr2=0x7f789c0b8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 shutdown_connections 2024-04-24T23:58:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.638+0000 7f78a2815640 1 -- 172.21.15.113:0/2684249536 wait complete. 2024-04-24T23:58:31.650 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:31.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.745+0000 7f96c37ee640 1 Processor -- start 2024-04-24T23:58:31.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.745+0000 7f96c37ee640 1 -- start start 2024-04-24T23:58:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c37ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96bc15e320 0x7f96bc0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c37ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c37ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96bc0d3de0 con 0x7f96bc0d0b00 2024-04-24T23:58:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c37ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f96bc0d3410 con 0x7f96bc15e320 2024-04-24T23:58:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c37ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f96bc0d2800 con 0x7f96bc0d2290 2024-04-24T23:58:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c1d8b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f96bc0d0b00 0x7f96bc0b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55572/0 (socket says 172.21.15.113:55572) 2024-04-24T23:58:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c1d8b640 1 -- 172.21.15.113:0/3138730630 learned_addr learned my addr 172.21.15.113:0/3138730630 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c0d89640 1 --2- 172.21.15.113:0/3138730630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.746+0000 7f96c158a640 1 --2- 172.21.15.113:0/3138730630 >> v2:172.21.15.178:3300/0 conn(0x7f96bc15e320 0x7f96bc0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96bbfff640 1 -- 172.21.15.113:0/3138730630 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1188819894 0 0) 0x7f96bc0d3de0 con 0x7f96bc0d0b00 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96bbfff640 1 -- 172.21.15.113:0/3138730630 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96a4003520 con 0x7f96bc0d0b00 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96c0d89640 1 -- 172.21.15.113:0/3138730630 >> v2:172.21.15.178:3300/0 conn(0x7f96bc15e320 msgr2=0x7f96bc0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96c0d89640 1 --2- 172.21.15.113:0/3138730630 >> v2:172.21.15.178:3300/0 conn(0x7f96bc15e320 0x7f96bc0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96c0d89640 1 -- 172.21.15.113:0/3138730630 >> v1:172.21.15.113:6789/0 conn(0x7f96bc0d0b00 legacy=0x7f96bc0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96c0d89640 1 -- 172.21.15.113:0/3138730630 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc0d2ae0 con 0x7f96bc0d2290 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96c158a640 1 --2- 172.21.15.113:0/3138730630 >> v2:172.21.15.178:3300/0 conn(0x7f96bc15e320 0x7f96bc0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:31.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.747+0000 7f96c0d89640 1 --2- 172.21.15.113:0/3138730630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc0b9aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78536c1cf871db0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96bbfff640 1 -- 172.21.15.113:0/3138730630 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0037070 con 0x7f96bc0d2290 2024-04-24T23:58:31.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96bbfff640 1 -- 172.21.15.113:0/3138730630 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b0032440 con 0x7f96bc0d2290 2024-04-24T23:58:31.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96bbfff640 1 -- 172.21.15.113:0/3138730630 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b00327f0 con 0x7f96bc0d2290 2024-04-24T23:58:31.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 -- 172.21.15.113:0/3138730630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 msgr2=0x7f96bc0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 --2- 172.21.15.113:0/3138730630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc0b9aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002a210 comp rx=0 tx=0).stop 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 -- 172.21.15.113:0/3138730630 shutdown_connections 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 --2- 172.21.15.113:0/3138730630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc0b9aa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 --2- 172.21.15.113:0/3138730630 >> v2:172.21.15.178:3300/0 conn(0x7f96bc15e320 0x7f96bc0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 -- 172.21.15.113:0/3138730630 >> 172.21.15.113:0/3138730630 conn(0x7f96bc0b5c20 msgr2=0x7f96bc0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 -- 172.21.15.113:0/3138730630 shutdown_connections 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.748+0000 7f96c37ee640 1 -- 172.21.15.113:0/3138730630 wait complete. 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.749+0000 7f96c37ee640 1 Processor -- start 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.749+0000 7f96c37ee640 1 -- start start 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c37ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96bc0d0b00 0x7f96bc152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c37ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c37ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96bc15e320 0x7f96bc15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c37ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc0b9fe0 con 0x7f96bc15e320 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c37ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc0d30a0 con 0x7f96bc0d0b00 2024-04-24T23:58:31.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c37ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f96bc15e700 con 0x7f96bc0d2290 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96bc15e320 0x7f96bc15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96bc15e320 0x7f96bc15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50440/0 (socket says 172.21.15.113:50440) 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 -- 172.21.15.113:0/1975098802 learned_addr learned my addr 172.21.15.113:0/1975098802 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 -- 172.21.15.113:0/1975098802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 msgr2=0x7f96bc159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 -- 172.21.15.113:0/1975098802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96bc0d0b00 msgr2=0x7f96bc152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96bc0d0b00 0x7f96bc152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.750+0000 7f96c1d8b640 1 -- 172.21.15.113:0/1975098802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc161000 con 0x7f96bc15e320 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.751+0000 7f96c1d8b640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96bc15e320 0x7f96bc15a260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00cb40 tx=0x7f96ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.751+0000 7f96b9ffb640 1 -- 172.21.15.113:0/1975098802 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac017070 con 0x7f96bc15e320 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.751+0000 7f96b9ffb640 1 -- 172.21.15.113:0/1975098802 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac00aca0 con 0x7f96bc15e320 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.751+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc157280 con 0x7f96bc15e320 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.751+0000 7f96b9ffb640 1 -- 172.21.15.113:0/1975098802 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac012620 con 0x7f96bc15e320 2024-04-24T23:58:31.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.751+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc1569e0 con 0x7f96bc15e320 2024-04-24T23:58:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.752+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96bc0691a0 con 0x7f96bc15e320 2024-04-24T23:58:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.752+0000 7f96b9ffb640 1 -- 172.21.15.113:0/1975098802 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96ac01a070 con 0x7f96bc15e320 2024-04-24T23:58:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.752+0000 7f96b9ffb640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f96a806be30 0x7f96a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.753+0000 7f96b9ffb640 1 -- 172.21.15.113:0/1975098802 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96ac08ac40 con 0x7f96bc15e320 2024-04-24T23:58:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.753+0000 7f96c158a640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f96a806be30 0x7f96a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.753+0000 7f96c158a640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f96a806be30 0x7f96a806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006e80 tx=0x7f96b4006dd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.757+0000 7f96b9ffb640 1 -- 172.21.15.113:0/1975098802 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96ac059680 con 0x7f96bc15e320 2024-04-24T23:58:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.966+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96bc0d2070 con 0x7f96bc15e320 2024-04-24T23:58:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.966+0000 7f96b9ffb640 1 -- 172.21.15.113:0/1975098802 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f96ac058f80 con 0x7f96bc15e320 2024-04-24T23:58:31.967 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:31.967 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:31.967 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:31.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f96a806be30 msgr2=0x7f96a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f96a806be30 0x7f96a806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006e80 tx=0x7f96b4006dd0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96bc15e320 msgr2=0x7f96bc15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96bc15e320 0x7f96bc15a260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00cb40 tx=0x7f96ac00a060 comp rx=0 tx=0).stop 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 shutdown_connections 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f96a806be30 0x7f96a806e2f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96bc15e320 0x7f96bc15a260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96bc0d2290 0x7f96bc159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 --2- 172.21.15.113:0/1975098802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96bc0d0b00 0x7f96bc152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 >> 172.21.15.113:0/1975098802 conn(0x7f96bc0b5c20 msgr2=0x7f96bc071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 shutdown_connections 2024-04-24T23:58:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:31.968+0000 7f96c37ee640 1 -- 172.21.15.113:0/1975098802 wait complete. 2024-04-24T23:58:31.979 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:32.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.075+0000 7f7f86aee640 1 Processor -- start 2024-04-24T23:58:32.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.075+0000 7f7f86aee640 1 -- start start 2024-04-24T23:58:32.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f86aee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f80144d60 0x7f7f800b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f86aee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f801453c0 0x7f7f8006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f86aee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f800cfad0 con 0x7f7f8006e510 2024-04-24T23:58:32.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f86aee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7f80145c40 con 0x7f7f801453c0 2024-04-24T23:58:32.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f86aee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8006e940 con 0x7f7f80144d60 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f7ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f801453c0 0x7f7f8006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f7ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f801453c0 0x7f7f8006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50444/0 (socket says 172.21.15.113:50444) 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f7ffff640 1 -- 172.21.15.113:0/1420769083 learned_addr learned my addr 172.21.15.113:0/1420769083 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f8488a640 1 --2- 172.21.15.113:0/1420769083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f80144d60 0x7f7f800b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f7ffff640 1 -- 172.21.15.113:0/1420769083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f80144d60 msgr2=0x7f7f800b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f7ffff640 1 --2- 172.21.15.113:0/1420769083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f80144d60 0x7f7f800b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.076+0000 7f7f7ffff640 1 -- 172.21.15.113:0/1420769083 >> v1:172.21.15.178:6789/0 conn(0x7f7f8006e510 legacy=0x7f7f8015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.077+0000 7f7f7ffff640 1 -- 172.21.15.113:0/1420769083 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f800cf230 con 0x7f7f801453c0 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.077+0000 7f7f7ffff640 1 --2- 172.21.15.113:0/1420769083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f8006dfd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009870 tx=0x7f7f7402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4accd4c17015d7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.077+0000 7f7f7f7fe640 1 -- 172.21.15.113:0/1420769083 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74037070 con 0x7f7f801453c0 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.077+0000 7f7f7f7fe640 1 -- 172.21.15.113:0/1420769083 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f74032440 con 0x7f7f801453c0 2024-04-24T23:58:32.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.077+0000 7f7f7f7fe640 1 -- 172.21.15.113:0/1420769083 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74032810 con 0x7f7f801453c0 2024-04-24T23:58:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 -- 172.21.15.113:0/1420769083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 msgr2=0x7f7f8006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 --2- 172.21.15.113:0/1420769083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f8006dfd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009870 tx=0x7f7f7402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 -- 172.21.15.113:0/1420769083 shutdown_connections 2024-04-24T23:58:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 --2- 172.21.15.113:0/1420769083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f8006dfd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 --2- 172.21.15.113:0/1420769083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f80144d60 0x7f7f800b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 -- 172.21.15.113:0/1420769083 >> 172.21.15.113:0/1420769083 conn(0x7f7f800b1a50 msgr2=0x7f7f800b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 -- 172.21.15.113:0/1420769083 shutdown_connections 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.078+0000 7f7f86aee640 1 -- 172.21.15.113:0/1420769083 wait complete. 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.079+0000 7f7f86aee640 1 Processor -- start 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.079+0000 7f7f86aee640 1 -- start start 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f86aee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f8006e510 0x7f7f801562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f86aee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f80144d60 0x7f7f80157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f86aee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f801584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f86aee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80070e10 con 0x7f7f801453c0 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f86aee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f800cf7f0 con 0x7f7f80144d60 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f86aee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f80145140 con 0x7f7f8006e510 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f801584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f801584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50454/0 (socket says 172.21.15.113:50454) 2024-04-24T23:58:32.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 -- 172.21.15.113:0/362717900 learned_addr learned my addr 172.21.15.113:0/362717900 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8488a640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f8006e510 0x7f7f801562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f7ffff640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f80144d60 0x7f7f80157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 -- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f8006e510 msgr2=0x7f7f801562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f8006e510 0x7f7f801562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 -- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f80144d60 msgr2=0x7f7f80157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f80144d60 0x7f7f80157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 -- 172.21.15.113:0/362717900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8015abf0 con 0x7f7f801453c0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.080+0000 7f7f8508b640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f801584a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800c730 tx=0x7f7f7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.081+0000 7f7f7d7fa640 1 -- 172.21.15.113:0/362717900 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78003b70 con 0x7f7f801453c0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.081+0000 7f7f7d7fa640 1 -- 172.21.15.113:0/362717900 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f78003d30 con 0x7f7f801453c0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.081+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8015aed0 con 0x7f7f801453c0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.081+0000 7f7f7d7fa640 1 -- 172.21.15.113:0/362717900 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78005310 con 0x7f7f801453c0 2024-04-24T23:58:32.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.081+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801f29c0 con 0x7f7f801453c0 2024-04-24T23:58:32.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.082+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f80075040 con 0x7f7f801453c0 2024-04-24T23:58:32.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.082+0000 7f7f7d7fa640 1 -- 172.21.15.113:0/362717900 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f780054b0 con 0x7f7f801453c0 2024-04-24T23:58:32.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.083+0000 7f7f7d7fa640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6c06be30 0x7f7f6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.083+0000 7f7f7d7fa640 1 -- 172.21.15.113:0/362717900 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f7808b530 con 0x7f7f801453c0 2024-04-24T23:58:32.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.083+0000 7f7f8488a640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6c06be30 0x7f7f6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.083+0000 7f7f8488a640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6c06be30 0x7f7f6c06e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f70006fd0 tx=0x7f7f70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.087+0000 7f7f7d7fa640 1 -- 172.21.15.113:0/362717900 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f80075040 con 0x7f7f801453c0 2024-04-24T23:58:32.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.263+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f800691a0 con 0x7f7f801453c0 2024-04-24T23:58:32.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.264+0000 7f7f7d7fa640 1 -- 172.21.15.113:0/362717900 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f7f78059a00 con 0x7f7f801453c0 2024-04-24T23:58:32.265 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:32.265 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6c06be30 msgr2=0x7f7f6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6c06be30 0x7f7f6c06e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f70006fd0 tx=0x7f7f70008040 comp rx=0 tx=0).stop 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 msgr2=0x7f7f801584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f801584a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800c730 tx=0x7f7f7800cbf0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 shutdown_connections 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6c06be30 0x7f7f6c06e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f80144d60 0x7f7f80157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f801453c0 0x7f7f801584a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 --2- 172.21.15.113:0/362717900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f8006e510 0x7f7f801562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.266+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 >> 172.21.15.113:0/362717900 conn(0x7f7f800b1a50 msgr2=0x7f7f800ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.267+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 shutdown_connections 2024-04-24T23:58:32.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.267+0000 7f7f86aee640 1 -- 172.21.15.113:0/362717900 wait complete. 2024-04-24T23:58:32.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:32.278 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.378+0000 7f1925d1d640 1 Processor -- start 2024-04-24T23:58:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.378+0000 7f1925d1d640 1 -- start start 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f1925d1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f192015b670 0x7f192015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f1925d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19200cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f1925d1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1920071170 con 0x7f19200cc740 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f1925d1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f192015c470 con 0x7f192015b670 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f1925d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1920071450 con 0x7f192015bff0 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f191f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f192015b670 0x7f192015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f191f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f192015b670 0x7f192015ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50462/0 (socket says 172.21.15.113:50462) 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f191f7fe640 1 -- 172.21.15.113:0/2667859604 learned_addr learned my addr 172.21.15.113:0/2667859604 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f191f7fe640 1 -- 172.21.15.113:0/2667859604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 msgr2=0x7f19200cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f191f7fe640 1 --2- 172.21.15.113:0/2667859604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19200cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f191f7fe640 1 -- 172.21.15.113:0/2667859604 >> v1:172.21.15.178:6789/0 conn(0x7f19200cc740 legacy=0x7f192006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.379+0000 7f191f7fe640 1 -- 172.21.15.113:0/2667859604 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1920071730 con 0x7f192015b670 2024-04-24T23:58:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f191f7fe640 1 --2- 172.21.15.113:0/2667859604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f192015ba50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f191000bb10 tx=0x7f191002c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc0a116381d8289 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f191e7fc640 1 -- 172.21.15.113:0/2667859604 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910031710 con 0x7f192015b670 2024-04-24T23:58:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f191e7fc640 1 -- 172.21.15.113:0/2667859604 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19100318b0 con 0x7f192015b670 2024-04-24T23:58:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f191e7fc640 1 -- 172.21.15.113:0/2667859604 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910031c80 con 0x7f192015b670 2024-04-24T23:58:32.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f1925d1d640 1 -- 172.21.15.113:0/2667859604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 msgr2=0x7f192015ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2667859604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f192015ba50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f191000bb10 tx=0x7f191002c7a0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f1925d1d640 1 -- 172.21.15.113:0/2667859604 shutdown_connections 2024-04-24T23:58:32.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2667859604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19200cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2667859604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f192015ba50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.380+0000 7f1925d1d640 1 -- 172.21.15.113:0/2667859604 >> 172.21.15.113:0/2667859604 conn(0x7f192006b0f0 msgr2=0x7f192006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.381+0000 7f1925d1d640 1 -- 172.21.15.113:0/2667859604 shutdown_connections 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.381+0000 7f1925d1d640 1 -- 172.21.15.113:0/2667859604 wait complete. 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 Processor -- start 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 -- start start 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19200cc740 0x7f1920147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f19201e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19201ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f192006d730 con 0x7f192015b670 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f192006d5b0 con 0x7f19200cc740 2024-04-24T23:58:32.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.382+0000 7f1925d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f19200d0080 con 0x7f192015bff0 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f19201e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f19201e1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50470/0 (socket says 172.21.15.113:50470) 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 -- 172.21.15.113:0/2400675895 learned_addr learned my addr 172.21.15.113:0/2400675895 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191f7fe640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19200cc740 0x7f1920147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191ffff640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19201ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 -- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 msgr2=0x7f19201ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19201ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 -- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19200cc740 msgr2=0x7f1920147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19200cc740 0x7f1920147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 -- 172.21.15.113:0/2400675895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19201e2e30 con 0x7f192015b670 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191f7fe640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19200cc740 0x7f1920147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:32.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191ffff640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19201ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:32.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f191effd640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f19201e1340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f191400b410 tx=0x7f191400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f1907fff640 1 -- 172.21.15.113:0/2400675895 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914015020 con 0x7f192015b670 2024-04-24T23:58:32.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f1907fff640 1 -- 172.21.15.113:0/2400675895 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1914011850 con 0x7f192015b670 2024-04-24T23:58:32.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19201e1d90 con 0x7f192015b670 2024-04-24T23:58:32.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f1907fff640 1 -- 172.21.15.113:0/2400675895 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914011c40 con 0x7f192015b670 2024-04-24T23:58:32.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.383+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19201e2350 con 0x7f192015b670 2024-04-24T23:58:32.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.384+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19200691a0 con 0x7f192015b670 2024-04-24T23:58:32.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.385+0000 7f1907fff640 1 -- 172.21.15.113:0/2400675895 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1914018020 con 0x7f192015b670 2024-04-24T23:58:32.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.385+0000 7f1907fff640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f190c06be60 0x7f190c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.385+0000 7f1907fff640 1 -- 172.21.15.113:0/2400675895 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f191408ac70 con 0x7f192015b670 2024-04-24T23:58:32.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.385+0000 7f191f7fe640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f190c06be60 0x7f190c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.386+0000 7f191f7fe640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f190c06be60 0x7f190c06e320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1910036000 tx=0x7f1910031000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.389+0000 7f1907fff640 1 -- 172.21.15.113:0/2400675895 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f191401f020 con 0x7f192015b670 2024-04-24T23:58:32.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.560+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f192006c100 con 0x7f192015b670 2024-04-24T23:58:32.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.561+0000 7f1907fff640 1 -- 172.21.15.113:0/2400675895 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f19140172c0 con 0x7f192015b670 2024-04-24T23:58:32.561 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f190c06be60 msgr2=0x7f190c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f190c06be60 0x7f190c06e320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1910036000 tx=0x7f1910031000 comp rx=0 tx=0).stop 2024-04-24T23:58:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 msgr2=0x7f19201e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f19201e1340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f191400b410 tx=0x7f191400b8d0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 shutdown_connections 2024-04-24T23:58:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f190c06be60 0x7f190c06e320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f192015bff0 0x7f19201ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192015b670 0x7f19201e1340 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 --2- 172.21.15.113:0/2400675895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19200cc740 0x7f1920147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 >> 172.21.15.113:0/2400675895 conn(0x7f192006b0f0 msgr2=0x7f19201574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 shutdown_connections 2024-04-24T23:58:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.563+0000 7f1925d1d640 1 -- 172.21.15.113:0/2400675895 wait complete. 2024-04-24T23:58:32.574 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:32.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.673+0000 7f58bd18b640 1 Processor -- start 2024-04-24T23:58:32.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.673+0000 7f58bd18b640 1 -- start start 2024-04-24T23:58:32.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58bd18b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f58b815c3c0 0x7f58b80d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58bd18b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58b80cd5b0 0x7f58b80b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58bd18b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b8072e80 con 0x7f58b8071790 2024-04-24T23:58:32.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58bd18b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f58b80d00c0 con 0x7f58b80cd5b0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58bd18b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f58b80cdb00 con 0x7f58b815c3c0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58b80cd5b0 0x7f58b80b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58b80cd5b0 0x7f58b80b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50482/0 (socket says 172.21.15.113:50482) 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b659c640 1 -- 172.21.15.113:0/1034461062 learned_addr learned my addr 172.21.15.113:0/1034461062 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b6d9d640 1 --2- 172.21.15.113:0/1034461062 >> v2:172.21.15.178:3300/0 conn(0x7f58b815c3c0 0x7f58b80d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b659c640 1 -- 172.21.15.113:0/1034461062 >> v2:172.21.15.178:3300/0 conn(0x7f58b815c3c0 msgr2=0x7f58b80d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b659c640 1 --2- 172.21.15.113:0/1034461062 >> v2:172.21.15.178:3300/0 conn(0x7f58b815c3c0 0x7f58b80d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b659c640 1 -- 172.21.15.113:0/1034461062 >> v1:172.21.15.178:6789/0 conn(0x7f58b8071790 legacy=0x7f58b80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.674+0000 7f58b659c640 1 -- 172.21.15.113:0/1034461062 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b8073160 con 0x7f58b80cd5b0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58b659c640 1 --2- 172.21.15.113:0/1034461062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b80cd5b0 0x7f58b80b0500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58b8071d40 tx=0x7f58ac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f141543a2b20e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58b5d9b640 1 -- 172.21.15.113:0/1034461062 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac037070 con 0x7f58b80cd5b0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58b5d9b640 1 -- 172.21.15.113:0/1034461062 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac02ae20 con 0x7f58b80cd5b0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58b5d9b640 1 -- 172.21.15.113:0/1034461062 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac032660 con 0x7f58b80cd5b0 2024-04-24T23:58:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 -- 172.21.15.113:0/1034461062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b80cd5b0 msgr2=0x7f58b80b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1034461062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b80cd5b0 0x7f58b80b0500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58b8071d40 tx=0x7f58ac02a110 comp rx=0 tx=0).stop 2024-04-24T23:58:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 -- 172.21.15.113:0/1034461062 shutdown_connections 2024-04-24T23:58:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1034461062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b80cd5b0 0x7f58b80b0500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1034461062 >> v2:172.21.15.178:3300/0 conn(0x7f58b815c3c0 0x7f58b80d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 -- 172.21.15.113:0/1034461062 >> 172.21.15.113:0/1034461062 conn(0x7f58b80b57e0 msgr2=0x7f58b8072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 -- 172.21.15.113:0/1034461062 shutdown_connections 2024-04-24T23:58:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.675+0000 7f58bd18b640 1 -- 172.21.15.113:0/1034461062 wait complete. 2024-04-24T23:58:32.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 Processor -- start 2024-04-24T23:58:32.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 -- start start 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b8071790 0x7f58b80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b80cd5b0 0x7f58b8150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b815c3c0 0x7f58b80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8068a10 con 0x7f58b815c3c0 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80d0380 con 0x7f58b80cd5b0 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.677+0000 7f58bd18b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58b8068b90 con 0x7f58b8071790 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b815c3c0 0x7f58b80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b815c3c0 0x7f58b80b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50494/0 (socket says 172.21.15.113:50494) 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 -- 172.21.15.113:0/1587135801 learned_addr learned my addr 172.21.15.113:0/1587135801 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b659c640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b80cd5b0 0x7f58b8150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b6d9d640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b8071790 0x7f58b80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 -- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b8071790 msgr2=0x7f58b80b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b8071790 0x7f58b80b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 -- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b80cd5b0 msgr2=0x7f58b8150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b80cd5b0 0x7f58b8150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 -- 172.21.15.113:0/1587135801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b80b3c60 con 0x7f58b815c3c0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b6d9d640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b8071790 0x7f58b80b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58b759e640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b815c3c0 0x7f58b80b31a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b9a0 tx=0x7f58b000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f589f7fe640 1 -- 172.21.15.113:0/1587135801 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0019070 con 0x7f58b815c3c0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f589f7fe640 1 -- 172.21.15.113:0/1587135801 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b000cba0 con 0x7f58b815c3c0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b80b3f40 con 0x7f58b815c3c0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f589f7fe640 1 -- 172.21.15.113:0/1587135801 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0014440 con 0x7f58b815c3c0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.678+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b80b4500 con 0x7f58b815c3c0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.679+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b80691a0 con 0x7f58b815c3c0 2024-04-24T23:58:32.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.680+0000 7f589f7fe640 1 -- 172.21.15.113:0/1587135801 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58b001c070 con 0x7f58b815c3c0 2024-04-24T23:58:32.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.680+0000 7f589f7fe640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a406be30 0x7f58a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.680+0000 7f58b6d9d640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a406be30 0x7f58a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.680+0000 7f589f7fe640 1 -- 172.21.15.113:0/1587135801 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58b008aac0 con 0x7f58b815c3c0 2024-04-24T23:58:32.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.681+0000 7f58b6d9d640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a406be30 0x7f58a406e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58a8006fd0 tx=0x7f58a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.684+0000 7f589f7fe640 1 -- 172.21.15.113:0/1587135801 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58b0059470 con 0x7f58b815c3c0 2024-04-24T23:58:32.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.856+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f58b80cdcb0 con 0x7f58b815c3c0 2024-04-24T23:58:32.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.856+0000 7f589f7fe640 1 -- 172.21.15.113:0/1587135801 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f58b0058d70 con 0x7f58b815c3c0 2024-04-24T23:58:32.856 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:32.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a406be30 msgr2=0x7f58a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a406be30 0x7f58a406e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58a8006fd0 tx=0x7f58a8008040 comp rx=0 tx=0).stop 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b815c3c0 msgr2=0x7f58b80b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b815c3c0 0x7f58b80b31a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b9a0 tx=0x7f58b000be60 comp rx=0 tx=0).stop 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 shutdown_connections 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a406be30 0x7f58a406e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b80cd5b0 0x7f58b8150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b815c3c0 0x7f58b80b31a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 --2- 172.21.15.113:0/1587135801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b8071790 0x7f58b80b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 >> 172.21.15.113:0/1587135801 conn(0x7f58b80b57e0 msgr2=0x7f58b80cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 shutdown_connections 2024-04-24T23:58:32.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.858+0000 7f58bd18b640 1 -- 172.21.15.113:0/1587135801 wait complete. 2024-04-24T23:58:32.869 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:32.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.968+0000 7f0babccd640 1 Processor -- start 2024-04-24T23:58:32.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.968+0000 7f0babccd640 1 -- start start 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0babccd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ba40b3080 0x7f0ba40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0babccd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba415f4f0 con 0x7f0ba415e2d0 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0babccd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba415f7d0 con 0x7f0ba415e6b0 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0babccd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba40cde00 con 0x7f0ba40b3080 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0ba9268640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ba40b3080 0x7f0ba40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0ba9a69640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0ba415e2d0 0x7f0ba415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55578/0 (socket says 172.21.15.113:55578) 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0ba9a69640 1 -- 172.21.15.113:0/3057997185 learned_addr learned my addr 172.21.15.113:0/3057997185 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:32.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0ba9268640 1 -- 172.21.15.113:0/3057997185 >> v1:172.21.15.178:6789/0 conn(0x7f0ba415e6b0 legacy=0x7f0ba40ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:32.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0ba9268640 1 -- 172.21.15.113:0/3057997185 >> v1:172.21.15.113:6789/0 conn(0x7f0ba415e2d0 legacy=0x7f0ba415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0ba9268640 1 -- 172.21.15.113:0/3057997185 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba415fab0 con 0x7f0ba40b3080 2024-04-24T23:58:32.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.969+0000 7f0ba8a67640 1 -- 172.21.15.113:0/3057997185 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1492346097 0 0) 0x7f0ba415f4f0 con 0x7f0ba415e2d0 2024-04-24T23:58:32.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0ba9268640 1 --2- 172.21.15.113:0/3057997185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba40b4330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b940097a0 tx=0x7f0b9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33261cce49c2b6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0ba8a67640 1 -- 172.21.15.113:0/3057997185 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94037070 con 0x7f0ba40b3080 2024-04-24T23:58:32.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0ba8a67640 1 -- 172.21.15.113:0/3057997185 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b94032440 con 0x7f0ba40b3080 2024-04-24T23:58:32.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0ba8a67640 1 -- 172.21.15.113:0/3057997185 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94032810 con 0x7f0ba40b3080 2024-04-24T23:58:32.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0babccd640 1 -- 172.21.15.113:0/3057997185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 msgr2=0x7f0ba40b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0babccd640 1 --2- 172.21.15.113:0/3057997185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba40b4330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b940097a0 tx=0x7f0b9402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0babccd640 1 -- 172.21.15.113:0/3057997185 shutdown_connections 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0babccd640 1 --2- 172.21.15.113:0/3057997185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba40b4330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0babccd640 1 -- 172.21.15.113:0/3057997185 >> 172.21.15.113:0/3057997185 conn(0x7f0ba406b0f0 msgr2=0x7f0ba406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0babccd640 1 -- 172.21.15.113:0/3057997185 shutdown_connections 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.970+0000 7f0babccd640 1 -- 172.21.15.113:0/3057997185 wait complete. 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.971+0000 7f0babccd640 1 Processor -- start 2024-04-24T23:58:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0babccd640 1 -- start start 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0babccd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba4147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0babccd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba415e2d0 0x7f0ba4158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0babccd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ba415e6b0 0x7f0ba4156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0babccd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40b49f0 con 0x7f0ba40b3080 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0babccd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40b4870 con 0x7f0ba415e2d0 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0babccd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba40b4b70 con 0x7f0ba415e6b0 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0ba9a69640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba4147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0ba9a69640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba4147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50518/0 (socket says 172.21.15.113:50518) 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.972+0000 7f0ba9a69640 1 -- 172.21.15.113:0/4196596170 learned_addr learned my addr 172.21.15.113:0/4196596170 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0ba9a69640 1 -- 172.21.15.113:0/4196596170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ba415e6b0 msgr2=0x7f0ba4156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0ba9a69640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ba415e6b0 0x7f0ba4156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0ba9a69640 1 -- 172.21.15.113:0/4196596170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba415e2d0 msgr2=0x7f0ba4158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0ba9a69640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba415e2d0 0x7f0ba4158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0ba9a69640 1 -- 172.21.15.113:0/4196596170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba415ca10 con 0x7f0ba40b3080 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0ba9a69640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba4147960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c4d0 tx=0x7f0ba000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0b9a7fc640 1 -- 172.21.15.113:0/4196596170 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0018070 con 0x7f0ba40b3080 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0b9a7fc640 1 -- 172.21.15.113:0/4196596170 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba0007500 con 0x7f0ba40b3080 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba415cd50 con 0x7f0ba40b3080 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0b9a7fc640 1 -- 172.21.15.113:0/4196596170 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0004a20 con 0x7f0ba40b3080 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.973+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba415d590 con 0x7f0ba40b3080 2024-04-24T23:58:32.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.974+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba40691a0 con 0x7f0ba40b3080 2024-04-24T23:58:32.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.974+0000 7f0b9a7fc640 1 -- 172.21.15.113:0/4196596170 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ba001b070 con 0x7f0ba40b3080 2024-04-24T23:58:32.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.975+0000 7f0b9a7fc640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b8806be30 0x7f0b8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:32.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.975+0000 7f0b9a7fc640 1 -- 172.21.15.113:0/4196596170 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ba008a1e0 con 0x7f0ba40b3080 2024-04-24T23:58:32.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.975+0000 7f0ba9268640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b8806be30 0x7f0b8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:32.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.975+0000 7f0ba9268640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b8806be30 0x7f0b8806e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b94034000 tx=0x7f0b9402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:32.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:32.979+0000 7f0b9a7fc640 1 -- 172.21.15.113:0/4196596170 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0ba415cd50 con 0x7f0ba40b3080 2024-04-24T23:58:33.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.152+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0ba4072050 con 0x7f0ba40b3080 2024-04-24T23:58:33.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.153+0000 7f0b9a7fc640 1 -- 172.21.15.113:0/4196596170 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0ba0058b90 con 0x7f0ba40b3080 2024-04-24T23:58:33.153 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b8806be30 msgr2=0x7f0b8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b8806be30 0x7f0b8806e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b94034000 tx=0x7f0b9402c000 comp rx=0 tx=0).stop 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 msgr2=0x7f0ba4147960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba4147960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c4d0 tx=0x7f0ba000c990 comp rx=0 tx=0).stop 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 shutdown_connections 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b8806be30 0x7f0b8806e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba415e2d0 0x7f0ba4158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ba415e6b0 0x7f0ba4156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 --2- 172.21.15.113:0/4196596170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ba40b3080 0x7f0ba4147960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 >> 172.21.15.113:0/4196596170 conn(0x7f0ba406b0f0 msgr2=0x7f0ba40ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:33.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 shutdown_connections 2024-04-24T23:58:33.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.155+0000 7f0babccd640 1 -- 172.21.15.113:0/4196596170 wait complete. 2024-04-24T23:58:33.165 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:33.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.261+0000 7f1fb7b4f640 1 Processor -- start 2024-04-24T23:58:33.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.261+0000 7f1fb7b4f640 1 -- start start 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb7b4f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1fb0144910 0x7f1fb00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb7b4f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb0144cf0 con 0x7f1fb00b5240 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb7b4f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb00b2f30 con 0x7f1fb00b56b0 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb7b4f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb00b4510 con 0x7f1fb0144910 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb50ea640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1fb00b5240 0x7f1fb006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55594/0 (socket says 172.21.15.113:55594) 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb50ea640 1 -- 172.21.15.113:0/1572182304 learned_addr learned my addr 172.21.15.113:0/1572182304 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb58eb640 1 --2- 172.21.15.113:0/1572182304 >> v2:172.21.15.178:3300/0 conn(0x7f1fb0144910 0x7f1fb00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118152469 0 0) 0x7f1fb0144cf0 con 0x7f1fb00b5240 2024-04-24T23:58:33.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.262+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f9c005730 con 0x7f1fb00b5240 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 426868245 0 0) 0x7f1f9c005730 con 0x7f1fb00b5240 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 >> v2:172.21.15.178:3300/0 conn(0x7f1fb0144910 msgr2=0x7f1fb00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 --2- 172.21.15.113:0/1572182304 >> v2:172.21.15.178:3300/0 conn(0x7f1fb0144910 0x7f1fb00b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 >> v1:172.21.15.178:6789/0 conn(0x7f1fb00b56b0 legacy=0x7f1fb00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb00b3210 con 0x7f1fb00b5240 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1fa0001670 con 0x7f1fb00b5240 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 >> v1:172.21.15.113:6789/0 conn(0x7f1fb00b5240 legacy=0x7f1fb006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f9c005bc0 0x7f1f9c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1f9c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f9c0073d0 0x7f1f9c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00b4510 con 0x7f1f9c006a10 2024-04-24T23:58:33.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c00e2a0 con 0x7f1f9c0073d0 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c00e4a0 con 0x7f1f9c005bc0 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb50ea640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1f9c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb60ec640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f9c0073d0 0x7f1f9c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb58eb640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f9c005bc0 0x7f1f9c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb50ea640 1 -- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f9c005bc0 msgr2=0x7f1f9c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb50ea640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f9c005bc0 0x7f1f9c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb50ea640 1 -- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f9c0073d0 msgr2=0x7f1f9c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb50ea640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f9c0073d0 0x7f1f9c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb50ea640 1 -- 172.21.15.113:0/1572182304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c005730 con 0x7f1f9c006a10 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb60ec640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f9c0073d0 0x7f1f9c00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.263+0000 7f1fb60ec640 1 -- 172.21.15.113:0/1572182304 reap_dead start 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb50ea640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1f9c006e90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c240 tx=0x7f1f9c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a81bc4f4cdee363 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0007d10 con 0x7f1f9c006a10 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa0005ad0 con 0x7f1f9c006a10 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb48e9640 1 -- 172.21.15.113:0/1572182304 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0007070 con 0x7f1f9c006a10 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1572182304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 msgr2=0x7f1f9c006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1f9c006e90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c240 tx=0x7f1f9c007a40 comp rx=0 tx=0).stop 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1572182304 shutdown_connections 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1572182304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1f9c006e90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.264+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1572182304 >> 172.21.15.113:0/1572182304 conn(0x7f1fb00b1a50 msgr2=0x7f1fb00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.265+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1572182304 shutdown_connections 2024-04-24T23:58:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.265+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1572182304 wait complete. 2024-04-24T23:58:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 Processor -- start 2024-04-24T23:58:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 -- start start 2024-04-24T23:58:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1fb00aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fb00aefb0 0x7f1fb013bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb013c250 0x7f1fb00ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00b4da0 con 0x7f1f9c006a10 2024-04-24T23:58:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00b37d0 con 0x7f1fb013c250 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb7b4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb00b4f20 con 0x7f1fb00aefb0 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb58eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1fb00aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb58eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1fb00aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50534/0 (socket says 172.21.15.113:50534) 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.266+0000 7f1fb58eb640 1 -- 172.21.15.113:0/1036605036 learned_addr learned my addr 172.21.15.113:0/1036605036 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb60ec640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb013c250 0x7f1fb00ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb50ea640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fb00aefb0 0x7f1fb013bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb58eb640 1 -- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fb00aefb0 msgr2=0x7f1fb013bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb58eb640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fb00aefb0 0x7f1fb013bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb58eb640 1 -- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb013c250 msgr2=0x7f1fb00ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb58eb640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb013c250 0x7f1fb00ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb58eb640 1 -- 172.21.15.113:0/1036605036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb013c6e0 con 0x7f1f9c006a10 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb58eb640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1fb00aea70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1fac007f30 tx=0x7f1fac011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fa67fc640 1 -- 172.21.15.113:0/1036605036 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac01a020 con 0x7f1f9c006a10 2024-04-24T23:58:33.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fa67fc640 1 -- 172.21.15.113:0/1036605036 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac016470 con 0x7f1f9c006a10 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb00ada30 con 0x7f1f9c006a10 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fa67fc640 1 -- 172.21.15.113:0/1036605036 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac016860 con 0x7f1f9c006a10 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.267+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb00adfb0 con 0x7f1f9c006a10 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.268+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb015e100 con 0x7f1f9c006a10 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.268+0000 7f1fa67fc640 1 -- 172.21.15.113:0/1036605036 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fac005d10 con 0x7f1f9c006a10 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.269+0000 7f1fa67fc640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.269+0000 7f1fa67fc640 1 -- 172.21.15.113:0/1036605036 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1fac08be30 con 0x7f1f9c006a10 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.269+0000 7f1fb50ea640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.269+0000 7f1fb50ea640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1f9406be30 0x7f1f9406e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0033000 tx=0x7f1fa002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.273+0000 7f1fa67fc640 1 -- 172.21.15.113:0/1036605036 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fb015e100 con 0x7f1f9c006a10 2024-04-24T23:58:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.483+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fb00731a0 con 0x7f1f9c006a10 2024-04-24T23:58:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.484+0000 7f1fa67fc640 1 -- 172.21.15.113:0/1036605036 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f1fac024030 con 0x7f1f9c006a10 2024-04-24T23:58:33.484 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:33.484 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1f9406be30 msgr2=0x7f1f9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1f9406be30 0x7f1f9406e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0033000 tx=0x7f1fa002e000 comp rx=0 tx=0).stop 2024-04-24T23:58:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 msgr2=0x7f1fb00aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1fb00aea70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1fac007f30 tx=0x7f1fac011660 comp rx=0 tx=0).stop 2024-04-24T23:58:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 shutdown_connections 2024-04-24T23:58:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb013c250 0x7f1fb00ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fb00aefb0 0x7f1fb013bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 --2- 172.21.15.113:0/1036605036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f9c006a10 0x7f1fb00aea70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 >> 172.21.15.113:0/1036605036 conn(0x7f1fb00b1a50 msgr2=0x7f1fb015f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 shutdown_connections 2024-04-24T23:58:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.486+0000 7f1fb7b4f640 1 -- 172.21.15.113:0/1036605036 wait complete. 2024-04-24T23:58:33.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:33.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.593+0000 7fbe35bee640 1 Processor -- start 2024-04-24T23:58:33.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.593+0000 7fbe35bee640 1 -- start start 2024-04-24T23:58:33.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe35bee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 0x7fbe3006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe35bee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbe300d00d0 0x7fbe3006b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe35bee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe300cedb0 con 0x7fbe3015c010 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe35bee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbe300cf030 con 0x7fbe300d00d0 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe35bee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbe300cf310 con 0x7fbe30074cf0 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbe3015c010 0x7fbe30160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55600/0 (socket says 172.21.15.113:55600) 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2f7fe640 1 -- 172.21.15.113:0/2979889520 learned_addr learned my addr 172.21.15.113:0/2979889520 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2ffff640 1 --2- 172.21.15.113:0/2979889520 >> v2:172.21.15.178:3300/0 conn(0x7fbe300d00d0 0x7fbe3006b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2effd640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 0x7fbe3006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323050152 0 0) 0x7fbe20001110 con 0x7fbe3015c010 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe18003520 con 0x7fbe3015c010 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 222906908 0 0) 0x7fbe18003520 con 0x7fbe3015c010 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 msgr2=0x7fbe3006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 0x7fbe3006b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 >> v2:172.21.15.178:3300/0 conn(0x7fbe300d00d0 msgr2=0x7fbe3006b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 --2- 172.21.15.113:0/2979889520 >> v2:172.21.15.178:3300/0 conn(0x7fbe300d00d0 0x7fbe3006b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe300cf5f0 con 0x7fbe3015c010 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fbe20001840 con 0x7fbe3015c010 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 >> v1:172.21.15.113:6789/0 conn(0x7fbe3015c010 legacy=0x7fbe30160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180039b0 0x7fbe18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe18004800 0x7fbe18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 0x7fbe180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe300cf030 con 0x7fbe18004800 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe300cf310 con 0x7fbe180039b0 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1800b9c0 con 0x7fbe30074cf0 2024-04-24T23:58:33.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.594+0000 7fbe2effd640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe18004800 0x7fbe18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2f7fe640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180039b0 0x7fbe18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2ffff640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 0x7fbe180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2effd640 1 -- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 msgr2=0x7fbe180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2effd640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 0x7fbe180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2effd640 1 -- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180039b0 msgr2=0x7fbe18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2effd640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180039b0 0x7fbe18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2effd640 1 -- 172.21.15.113:0/2979889520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe18003520 con 0x7fbe18004800 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2ffff640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe30074cf0 0x7fbe180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2f7fe640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180039b0 0x7fbe18003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2effd640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe18004800 0x7fbe18004c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009ac0 tx=0x7fbe18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4959e04631bff812 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24036070 con 0x7fbe18004800 2024-04-24T23:58:33.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe24031400 con 0x7fbe18004800 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.595+0000 7fbe2e7fc640 1 -- 172.21.15.113:0/2979889520 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24009690 con 0x7fbe18004800 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.596+0000 7fbe35bee640 1 -- 172.21.15.113:0/2979889520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe18004800 msgr2=0x7fbe18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.596+0000 7fbe35bee640 1 --2- 172.21.15.113:0/2979889520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe18004800 0x7fbe18004c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009ac0 tx=0x7fbe18005830 comp rx=0 tx=0).stop 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.596+0000 7fbe2ffff640 1 -- 172.21.15.113:0/2979889520 reap_dead start 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.596+0000 7fbe35bee640 1 -- 172.21.15.113:0/2979889520 shutdown_connections 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.596+0000 7fbe35bee640 1 -- 172.21.15.113:0/2979889520 >> 172.21.15.113:0/2979889520 conn(0x7fbe300b22c0 msgr2=0x7fbe300b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.596+0000 7fbe35bee640 1 -- 172.21.15.113:0/2979889520 shutdown_connections 2024-04-24T23:58:33.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.596+0000 7fbe35bee640 1 -- 172.21.15.113:0/2979889520 wait complete. 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.597+0000 7fbe35bee640 1 Processor -- start 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.597+0000 7fbe35bee640 1 -- start start 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe35bee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe18004800 0x7fbe3014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe35bee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe3014d5f0 0x7fbe3014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe35bee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe300b0380 0x7fbe300b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe35bee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3015c3f0 con 0x7fbe300b0380 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe35bee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe300cfbb0 con 0x7fbe18004800 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe35bee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3006b0f0 con 0x7fbe3014d5f0 2024-04-24T23:58:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe300b0380 0x7fbe300b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe300b0380 0x7fbe300b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50566/0 (socket says 172.21.15.113:50566) 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 -- 172.21.15.113:0/1809859751 learned_addr learned my addr 172.21.15.113:0/1809859751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 -- 172.21.15.113:0/1809859751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe3014d5f0 msgr2=0x7fbe3014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe3014d5f0 0x7fbe3014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 -- 172.21.15.113:0/1809859751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe18004800 msgr2=0x7fbe3014b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe18004800 0x7fbe3014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 -- 172.21.15.113:0/1809859751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3015c610 con 0x7fbe300b0380 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.598+0000 7fbe2ffff640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe300b0380 0x7fbe300b0830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe3006cf40 tx=0x7fbe28010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.599+0000 7fbe17fff640 1 -- 172.21.15.113:0/1809859751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28010da0 con 0x7fbe300b0380 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.599+0000 7fbe17fff640 1 -- 172.21.15.113:0/1809859751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe28003990 con 0x7fbe300b0380 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.599+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3014e160 con 0x7fbe300b0380 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.599+0000 7fbe17fff640 1 -- 172.21.15.113:0/1809859751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003d80 con 0x7fbe300b0380 2024-04-24T23:58:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.599+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3014e720 con 0x7fbe300b0380 2024-04-24T23:58:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.600+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe300691a0 con 0x7fbe300b0380 2024-04-24T23:58:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.600+0000 7fbe17fff640 1 -- 172.21.15.113:0/1809859751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe28005d10 con 0x7fbe300b0380 2024-04-24T23:58:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.600+0000 7fbe17fff640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe1c06be60 0x7fbe1c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.601+0000 7fbe2f7fe640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe1c06be60 0x7fbe1c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.601+0000 7fbe17fff640 1 -- 172.21.15.113:0/1809859751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe2808b5f0 con 0x7fbe300b0380 2024-04-24T23:58:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.601+0000 7fbe2f7fe640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe1c06be60 0x7fbe1c06e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe20001e20 tx=0x7fbe20001d90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.605+0000 7fbe17fff640 1 -- 172.21.15.113:0/1809859751 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe28025020 con 0x7fbe300b0380 2024-04-24T23:58:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.779+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe300cd870 con 0x7fbe300b0380 2024-04-24T23:58:33.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.780+0000 7fbe17fff640 1 -- 172.21.15.113:0/1809859751 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fbe28059fa0 con 0x7fbe300b0380 2024-04-24T23:58:33.780 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:33.780 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe1c06be60 msgr2=0x7fbe1c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe1c06be60 0x7fbe1c06e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe20001e20 tx=0x7fbe20001d90 comp rx=0 tx=0).stop 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe300b0380 msgr2=0x7fbe300b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe300b0380 0x7fbe300b0830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe3006cf40 tx=0x7fbe28010590 comp rx=0 tx=0).stop 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 shutdown_connections 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe1c06be60 0x7fbe1c06e320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe300b0380 0x7fbe300b0830 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe3014d5f0 0x7fbe3014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 --2- 172.21.15.113:0/1809859751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe18004800 0x7fbe3014b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 >> 172.21.15.113:0/1809859751 conn(0x7fbe300b22c0 msgr2=0x7fbe3015ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 shutdown_connections 2024-04-24T23:58:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.782+0000 7fbe35bee640 1 -- 172.21.15.113:0/1809859751 wait complete. 2024-04-24T23:58:33.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.889+0000 7f898d81c640 1 Processor -- start 2024-04-24T23:58:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.889+0000 7f898d81c640 1 -- start start 2024-04-24T23:58:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f898d81c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f89880d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f898d81c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f89880d2290 0x7f89880b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f898d81c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89880d3de0 con 0x7f89880d0b00 2024-04-24T23:58:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f898d81c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f89880d3410 con 0x7f89880d2290 2024-04-24T23:58:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f898d81c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f89880d2800 con 0x7f898815e320 2024-04-24T23:58:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f89867fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f89880d2290 0x7f89880b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f89867fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f89880d2290 0x7f89880b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50578/0 (socket says 172.21.15.113:50578) 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f89867fc640 1 -- 172.21.15.113:0/3357924754 learned_addr learned my addr 172.21.15.113:0/3357924754 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.890+0000 7f8986ffd640 1 --2- 172.21.15.113:0/3357924754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f89880d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f89867fc640 1 -- 172.21.15.113:0/3357924754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 msgr2=0x7f89880d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f89867fc640 1 --2- 172.21.15.113:0/3357924754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f89880d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f89867fc640 1 -- 172.21.15.113:0/3357924754 >> v1:172.21.15.178:6789/0 conn(0x7f89880d0b00 legacy=0x7f89880b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f89867fc640 1 -- 172.21.15.113:0/3357924754 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89880d2ae0 con 0x7f89880d2290 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f8986ffd640 1 --2- 172.21.15.113:0/3357924754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f89880d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f89867fc640 1 --2- 172.21.15.113:0/3357924754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89880b9aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a525d0c617eaba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f8985ffb640 1 -- 172.21.15.113:0/3357924754 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c037070 con 0x7f89880d2290 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f8985ffb640 1 -- 172.21.15.113:0/3357924754 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c032440 con 0x7f89880d2290 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.891+0000 7f8985ffb640 1 -- 172.21.15.113:0/3357924754 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c032810 con 0x7f89880d2290 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 -- 172.21.15.113:0/3357924754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 msgr2=0x7f89880b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 --2- 172.21.15.113:0/3357924754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89880b9aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c02a2a0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 -- 172.21.15.113:0/3357924754 shutdown_connections 2024-04-24T23:58:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 --2- 172.21.15.113:0/3357924754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89880b9aa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 --2- 172.21.15.113:0/3357924754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f89880d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 -- 172.21.15.113:0/3357924754 >> 172.21.15.113:0/3357924754 conn(0x7f89880b5c20 msgr2=0x7f89880b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 -- 172.21.15.113:0/3357924754 shutdown_connections 2024-04-24T23:58:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.892+0000 7f898d81c640 1 -- 172.21.15.113:0/3357924754 wait complete. 2024-04-24T23:58:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.893+0000 7f898d81c640 1 Processor -- start 2024-04-24T23:58:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.893+0000 7f898d81c640 1 -- start start 2024-04-24T23:58:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f898d81c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89880d0b00 0x7f89881464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f898d81c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89881469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f898d81c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f8988152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f898d81c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f89880b9fe0 con 0x7f89880d2290 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f898d81c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f89880d30a0 con 0x7f89880d0b00 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f898d81c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f898815e700 con 0x7f898815e320 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89881469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89881469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50580/0 (socket says 172.21.15.113:50580) 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 -- 172.21.15.113:0/3356865991 learned_addr learned my addr 172.21.15.113:0/3356865991 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89877fe640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f8988152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f8986ffd640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89880d0b00 0x7f89881464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 -- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 msgr2=0x7f8988152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f8988152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 -- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89880d0b00 msgr2=0x7f89881464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89880d0b00 0x7f89881464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 -- 172.21.15.113:0/3356865991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8988145ba0 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f8986ffd640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89880d0b00 0x7f89881464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.894+0000 7f89867fc640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89881469e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.895+0000 7f896ffff640 1 -- 172.21.15.113:0/3356865991 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c041020 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.895+0000 7f896ffff640 1 -- 172.21.15.113:0/3356865991 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c030970 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.895+0000 7f896ffff640 1 -- 172.21.15.113:0/3356865991 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c037040 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.895+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89880cd1d0 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.895+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89880cd790 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.896+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89880cff90 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.896+0000 7f896ffff640 1 -- 172.21.15.113:0/3356865991 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f897c044020 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.896+0000 7f896ffff640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f897406be30 0x7f897406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.896+0000 7f896ffff640 1 -- 172.21.15.113:0/3356865991 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f897c0ab700 con 0x7f89880d2290 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.896+0000 7f8986ffd640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f897406be30 0x7f897406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.897+0000 7f8986ffd640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f897406be30 0x7f897406e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89880ce1e0 tx=0x7f8978007640 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:33.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:33.901+0000 7f896ffff640 1 -- 172.21.15.113:0/3356865991 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f89880cff90 con 0x7f89880d2290 2024-04-24T23:58:34.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.109+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89880b4b20 con 0x7f89880d2290 2024-04-24T23:58:34.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.110+0000 7f896ffff640 1 -- 172.21.15.113:0/3356865991 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f897c04b030 con 0x7f89880d2290 2024-04-24T23:58:34.110 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:34.110 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:34.111 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f897406be30 msgr2=0x7f897406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f897406be30 0x7f897406e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89880ce1e0 tx=0x7f8978007640 comp rx=0 tx=0).stop 2024-04-24T23:58:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 msgr2=0x7f89881469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89881469e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 shutdown_connections 2024-04-24T23:58:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f897406be30 0x7f897406e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f898815e320 0x7f8988152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89880d2290 0x7f89881469e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 --2- 172.21.15.113:0/3356865991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89880d0b00 0x7f89881464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 >> 172.21.15.113:0/3356865991 conn(0x7f89880b5c20 msgr2=0x7f89880b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 shutdown_connections 2024-04-24T23:58:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.112+0000 7f898d81c640 1 -- 172.21.15.113:0/3356865991 wait complete. 2024-04-24T23:58:34.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:58:18.584673+0000', 'modified': '2024-04-24T23:58:21.129558+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6837/2734482903', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 2734482903}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 2734482903}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4281, 'qdb_cluster': [4281]} 2024-04-24T23:58:34.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:58:34.526 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:58:34.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.624+0000 7fea64316640 1 Processor -- start 2024-04-24T23:58:34.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.624+0000 7fea64316640 1 -- start start 2024-04-24T23:58:34.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.625+0000 7fea64316640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea5c0b6100 0x7fea5c0b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.625+0000 7fea64316640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c0d2e00 con 0x7fea5c15e510 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.625+0000 7fea64316640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c0d30e0 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.625+0000 7fea64316640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fea5c0d33c0 con 0x7fea5c0b6100 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.625+0000 7fea628b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea5c0b6100 0x7fea5c0b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.625+0000 7fea620b2640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fea5c15e510 0x7fea5c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55608/0 (socket says 172.21.15.113:55608) 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.625+0000 7fea620b2640 1 -- 172.21.15.113:0/3634591149 learned_addr learned my addr 172.21.15.113:0/3634591149 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.626+0000 7fea628b3640 1 -- 172.21.15.113:0/3634591149 >> v1:172.21.15.178:6789/0 conn(0x7fea5c0b5ca0 legacy=0x7fea5c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.626+0000 7fea628b3640 1 -- 172.21.15.113:0/3634591149 >> v1:172.21.15.113:6789/0 conn(0x7fea5c15e510 legacy=0x7fea5c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.626+0000 7fea610b0640 1 -- 172.21.15.113:0/3634591149 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033549589 0 0) 0x7fea5c0d2e00 con 0x7fea5c15e510 2024-04-24T23:58:34.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.626+0000 7fea628b3640 1 -- 172.21.15.113:0/3634591149 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c0d36a0 con 0x7fea5c0b6100 2024-04-24T23:58:34.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.626+0000 7fea628b3640 1 --2- 172.21.15.113:0/3634591149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b6100 0x7fea5c0b7ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea540097a0 tx=0x7fea5402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2745e99020badf35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.626+0000 7fea610b0640 1 -- 172.21.15.113:0/3634591149 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54037070 con 0x7fea5c0b6100 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.626+0000 7fea610b0640 1 -- 172.21.15.113:0/3634591149 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5402ae50 con 0x7fea5c0b6100 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea610b0640 1 -- 172.21.15.113:0/3634591149 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54032660 con 0x7fea5c0b6100 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea64316640 1 -- 172.21.15.113:0/3634591149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b6100 msgr2=0x7fea5c0b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea64316640 1 --2- 172.21.15.113:0/3634591149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b6100 0x7fea5c0b7ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea540097a0 tx=0x7fea5402a140 comp rx=0 tx=0).stop 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea64316640 1 -- 172.21.15.113:0/3634591149 shutdown_connections 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea64316640 1 --2- 172.21.15.113:0/3634591149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b6100 0x7fea5c0b7ad0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea64316640 1 -- 172.21.15.113:0/3634591149 >> 172.21.15.113:0/3634591149 conn(0x7fea5c06b720 msgr2=0x7fea5c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea64316640 1 -- 172.21.15.113:0/3634591149 shutdown_connections 2024-04-24T23:58:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.627+0000 7fea64316640 1 -- 172.21.15.113:0/3634591149 wait complete. 2024-04-24T23:58:34.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.628+0000 7fea64316640 1 Processor -- start 2024-04-24T23:58:34.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.628+0000 7fea64316640 1 -- start start 2024-04-24T23:58:34.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea64316640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b5ca0 0x7fea5c15ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea64316640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c0b6100 0x7fea5c1610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea64316640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c15e510 0x7fea5c1f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea64316640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0b6510 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea64316640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0d3ea0 con 0x7fea5c15e510 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea64316640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c0d2a90 con 0x7fea5c0b6100 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b5ca0 0x7fea5c15ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b5ca0 0x7fea5c15ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50600/0 (socket says 172.21.15.113:50600) 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 -- 172.21.15.113:0/954305151 learned_addr learned my addr 172.21.15.113:0/954305151 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 -- 172.21.15.113:0/954305151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c0b6100 msgr2=0x7fea5c1610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c0b6100 0x7fea5c1610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 -- 172.21.15.113:0/954305151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c15e510 msgr2=0x7fea5c1f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c15e510 0x7fea5c1f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 -- 172.21.15.113:0/954305151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c0cab50 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.629+0000 7fea620b2640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b5ca0 0x7fea5c15ff30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c890 tx=0x7fea5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.630+0000 7fea52ffd640 1 -- 172.21.15.113:0/954305151 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58018070 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.630+0000 7fea52ffd640 1 -- 172.21.15.113:0/954305151 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5800a640 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.630+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c0c9fd0 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.630+0000 7fea52ffd640 1 -- 172.21.15.113:0/954305151 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5800aa30 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.630+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c0ca870 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.631+0000 7fea52ffd640 1 -- 172.21.15.113:0/954305151 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea5801b070 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.631+0000 7fea52ffd640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea4006be30 0x7fea4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.632+0000 7fea52ffd640 1 -- 172.21.15.113:0/954305151 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea5808a5a0 con 0x7fea5c0b5ca0 2024-04-24T23:58:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.632+0000 7fea618b1640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea4006be30 0x7fea4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.632+0000 7fea618b1640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea4006be30 0x7fea4006e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea5c14de40 tx=0x7fea4c006ba0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.633+0000 7fea67019740 1 Processor -- start 2024-04-24T23:58:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.633+0000 7fea67019740 1 -- start start 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x5598396da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55983970ac80 0x55983970a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea67019740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970a540 0x5598396df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea67019740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559839642f00 con 0x55983970a540 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5598395eb750 con 0x55983970ac80 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55983963eec0 con 0x559839727b60 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970a540 0x5598396df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970a540 0x5598396df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50610/0 (socket says 172.21.15.113:50610) 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 -- 172.21.15.113:0/3384973830 learned_addr learned my addr 172.21.15.113:0/3384973830 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:34.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea618b1640 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55983970ac80 0x55983970a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea620b2640 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x5598396da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 -- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 msgr2=0x5598396da4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x5598396da4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 -- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55983970ac80 msgr2=0x55983970a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55983970ac80 0x55983970a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 -- 172.21.15.113:0/3384973830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598393572d0 con 0x55983970a540 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea620b2640 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x5598396da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea628b3640 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970a540 0x5598396df350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea5402a730 tx=0x7fea5402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf215eb1534547e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea3ffff640 1 -- 172.21.15.113:0/3384973830 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54041070 con 0x55983970a540 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea3ffff640 1 -- 172.21.15.113:0/3384973830 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5403c660 con 0x55983970a540 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.634+0000 7fea3ffff640 1 -- 172.21.15.113:0/3384973830 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54037040 con 0x55983970a540 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 -- 172.21.15.113:0/3384973830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970a540 msgr2=0x5598396df350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970a540 0x5598396df350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea5402a730 tx=0x7fea5402c070 comp rx=0 tx=0).stop 2024-04-24T23:58:34.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 -- 172.21.15.113:0/3384973830 shutdown_connections 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55983970ac80 0x55983970a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970a540 0x5598396df350 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 --2- 172.21.15.113:0/3384973830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x5598396da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 -- 172.21.15.113:0/3384973830 >> 172.21.15.113:0/3384973830 conn(0x5598396dc760 msgr2=0x559839725000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 -- 172.21.15.113:0/3384973830 shutdown_connections 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.635+0000 7fea67019740 1 -- 172.21.15.113:0/3384973830 wait complete. 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 -- start start 2024-04-24T23:58:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 Processor -- start 2024-04-24T23:58:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970ac80 0x559839701f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x55983971bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x5598396df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5598396de910 con 0x55983970ac80 2024-04-24T23:58:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5598395fecf0 con 0x5598396ded10 2024-04-24T23:58:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.636+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5598396ddd00 con 0x559839727b60 2024-04-24T23:58:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 10 client.0 _ll_register_callbacks cb 0x5598395fe6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970ac80 0x559839701f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970ac80 0x559839701f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50624/0 (socket says 172.21.15.113:50624) 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 -- 172.21.15.113:0/1986392977 learned_addr learned my addr 172.21.15.113:0/1986392977 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea628b3640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x5598396df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea618b1640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x55983971bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 -- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 msgr2=0x55983971bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x55983971bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 -- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 msgr2=0x5598396df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x5598396df190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 -- 172.21.15.113:0/1986392977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598394216b0 con 0x55983970ac80 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea618b1640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x55983971bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea620b2640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970ac80 0x559839701f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fea5800a430 tx=0x7fea58058990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23afaee33781d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1986392977 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58060fa0 con 0x55983970ac80 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1986392977 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea58061140 con 0x55983970ac80 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5403c660 con 0x55983970ac80 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1986392977 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fea58059b80 con 0x55983970ac80 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 10 client.4653 fetch_fsmap learned FSMap version 7 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x559839738f20 con 0x55983970ac80 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1986392977 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5403c660 con 0x55983970ac80 2024-04-24T23:58:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1986392977 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fea5806cce0 con 0x55983970ac80 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 10 client.4653 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 10 client.4653 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 20 client.4653 populate_metadata read hostname 'smithi113' 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.637+0000 7fea67019740 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x5598397096a0 0x559839709a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.638+0000 7fea67019740 4 client.4653 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.638+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 --> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559839650440 con 0x5598397096a0 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.638+0000 7fea618b1640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x5598397096a0 0x559839709a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.638+0000 7fea3d7fa640 10 client.4653 ms_handle_connect on v2:172.21.15.113:6835/2734482903 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.638+0000 7fea618b1640 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x5598397096a0 0x559839709a80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386c13c61d5cc668 server_cookie=1cd635e573806412 in_seq=0 out_seq=0 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1986392977 <== mds.0 v2:172.21.15.113:6835/2734482903 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559839650440 con 0x5598397096a0 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea3d7fa640 10 client.4653 handle_command_reply: tid=0 2024-04-24T23:58:34.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 client.4653 shutdown 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970ac80 msgr2=0x559839701f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970ac80 0x559839701f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fea5800a430 tx=0x7fea58058990 comp rx=0 tx=0).stop 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 shutdown_connections 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x5598397096a0 0x559839709a80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x5598396df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559839727b60 0x55983971bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 --2- 172.21.15.113:0/1986392977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55983970ac80 0x559839701f20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea3d7fa640 0 client.4653 ms_handle_reset on v2:172.21.15.113:6835/2734482903 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 >> 172.21.15.113:0/1986392977 conn(0x55983970a540 msgr2=0x5598396d9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 shutdown_connections 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 1 -- 172.21.15.113:0/1986392977 wait complete. 2024-04-24T23:58:34.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.639+0000 7fea67019740 20 client.4653 trim_cache size 0 max 16384 2024-04-24T23:58:34.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.657+0000 7fea67019740 1 Processor -- start 2024-04-24T23:58:34.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.657+0000 7fea67019740 1 -- start start 2024-04-24T23:58:34.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea67019740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598396ded10 0x5598396e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839734bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559839727b60 0x55983971b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea67019740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c2007b0 con 0x5598396ded10 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1ff890 con 0x559839727b60 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c1f4db0 con 0x55983970ac80 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598396ded10 0x5598396e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598396ded10 0x5598396e1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50632/0 (socket says 172.21.15.113:50632) 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 -- 172.21.15.113:0/3514208611 learned_addr learned my addr 172.21.15.113:0/3514208611 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea628b3640 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559839727b60 0x55983971b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea618b1640 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839734bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 -- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 msgr2=0x559839734bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839734bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 -- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559839727b60 msgr2=0x55983971b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559839727b60 0x55983971b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 -- 172.21.15.113:0/3514208611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598396e0fc0 con 0x5598396ded10 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea628b3640 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559839727b60 0x55983971b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea618b1640 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839734bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea620b2640 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598396ded10 0x5598396e1450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fea58054cb0 tx=0x7fea5805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08648e6be220a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea3f7fe640 1 -- 172.21.15.113:0/3514208611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58056b50 con 0x5598396ded10 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea3f7fe640 1 -- 172.21.15.113:0/3514208611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5808a250 con 0x5598396ded10 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.658+0000 7fea3f7fe640 1 -- 172.21.15.113:0/3514208611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5808a400 con 0x5598396ded10 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 -- 172.21.15.113:0/3514208611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598396ded10 msgr2=0x5598396e1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598396ded10 0x5598396e1450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fea58054cb0 tx=0x7fea5805e000 comp rx=0 tx=0).stop 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 -- 172.21.15.113:0/3514208611 shutdown_connections 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559839727b60 0x55983971b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839734bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 --2- 172.21.15.113:0/3514208611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598396ded10 0x5598396e1450 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 -- 172.21.15.113:0/3514208611 >> 172.21.15.113:0/3514208611 conn(0x55983970a540 msgr2=0x5598396e0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:34.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 -- 172.21.15.113:0/3514208611 shutdown_connections 2024-04-24T23:58:34.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 -- 172.21.15.113:0/3514208611 wait complete. 2024-04-24T23:58:34.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 -- start start 2024-04-24T23:58:34.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.659+0000 7fea67019740 1 Processor -- start 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x55983979f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea67019740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839728420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea67019740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559839727b60 0x559839728960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea67019740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559839705d90 con 0x559839727b60 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c153fc0 con 0x5598396ded10 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea67019740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559839735cc0 con 0x55983970ac80 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea67019740 10 client.0 _ll_register_callbacks cb 0x7fea58058f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea628b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559839727b60 0x559839728960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea628b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559839727b60 0x559839728960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50646/0 (socket says 172.21.15.113:50646) 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea628b3640 1 -- 172.21.15.113:0/1616612591 learned_addr learned my addr 172.21.15.113:0/1616612591 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.660+0000 7fea620b2640 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x55983979f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea618b1640 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839728420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea628b3640 1 -- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 msgr2=0x559839728420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea628b3640 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839728420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea628b3640 1 -- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 msgr2=0x55983979f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea628b3640 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x55983979f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea628b3640 1 -- 172.21.15.113:0/1616612591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559839721570 con 0x559839727b60 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea628b3640 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559839727b60 0x559839728960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea54040000 tx=0x7fea54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383df4e3f63a32e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1616612591 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54054060 con 0x559839727b60 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1616612591 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea54003820 con 0x559839727b60 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5808a250 con 0x559839727b60 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1616612591 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fea5405a050 con 0x559839727b60 2024-04-24T23:58:34.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 10 client.4659 fetch_fsmap learned FSMap version 7 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x559839727290 con 0x559839727b60 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1616612591 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5808a250 con 0x559839727b60 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1616612591 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fea5405d020 con 0x559839727b60 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 10 client.4659 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 10 client.4659 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 20 client.4659 populate_metadata read hostname 'smithi113' 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x559839708820 0x559839747c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 4 client.4659 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.661+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 --> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] -- command(tid 0: {"prefix": "status"}) -- 0x55983964a790 con 0x559839708820 2024-04-24T23:58:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea620b2640 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x559839708820 0x559839747c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea3d7fa640 10 client.4659 ms_handle_connect on v2:172.21.15.113:6835/2734482903 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea620b2640 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x559839708820 0x559839747c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9329e98afca205ad server_cookie=bafe418ea90ded56 in_seq=0 out_seq=0 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea3d7fa640 1 -- 172.21.15.113:0/1616612591 <== mds.0 v2:172.21.15.113:6835/2734482903 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55983964a790 con 0x559839708820 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea3d7fa640 10 client.4659 handle_command_reply: tid=0 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 client.4659 shutdown 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559839727b60 msgr2=0x559839728960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559839727b60 0x559839728960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea54040000 tx=0x7fea54004730 comp rx=0 tx=0).stop 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 shutdown_connections 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:6835/2734482903,v1:172.21.15.113:6837/2734482903] conn(0x559839708820 0x559839747c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559839727b60 0x559839728960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55983970ac80 0x559839728420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea3d7fa640 0 client.4659 ms_handle_reset on v2:172.21.15.113:6835/2734482903 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 --2- 172.21.15.113:0/1616612591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598396ded10 0x55983979f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 >> 172.21.15.113:0/1616612591 conn(0x55983970a540 msgr2=0x5598396d9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 shutdown_connections 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 1 -- 172.21.15.113:0/1616612591 wait complete. 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.662+0000 7fea67019740 20 client.4659 trim_cache size 0 max 16384 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4281, 2024-04-24T23:58:34.670 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 15.414485444, 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 7, 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 20, 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 20, 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 22.051296225000002 2024-04-24T23:58:34.671 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea4006be30 msgr2=0x7fea4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea4006be30 0x7fea4006e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea5c14de40 tx=0x7fea4c006ba0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b5ca0 msgr2=0x7fea5c15ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b5ca0 0x7fea5c15ff30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c890 tx=0x7fea5800cd50 comp rx=0 tx=0).stop 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 shutdown_connections 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea4006be30 0x7fea4006e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c15e510 0x7fea5c1f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c0b6100 0x7fea5c1610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 --2- 172.21.15.113:0/954305151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea5c0b5ca0 0x7fea5c15ff30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 >> 172.21.15.113:0/954305151 conn(0x7fea5c06b720 msgr2=0x7fea5c15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:34.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 shutdown_connections 2024-04-24T23:58:34.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:34.663+0000 7fea64316640 1 -- 172.21.15.113:0/954305151 wait complete. 2024-04-24T23:58:34.674 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T23:58:34.683 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T23:58:34.683 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T23:58:34.684 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T23:58:34.684 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T23:58:34.762 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T23:58:34.762 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-24T23:58:34.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:34.762 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-24T23:58:34.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:34.778 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-04-24T23:58:34.843 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-04-24T23:58:34.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:34.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:34.845 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-24T23:58:34.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:34.901 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-04-24T23:58:34.965 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-04-24T23:58:34.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:34.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:34.967 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-24T23:58:34.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:34.985 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-24T23:58:35.055 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2024-04-24T23:58:35.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:35.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:35.057 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-24T23:58:35.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:35.115 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-24T23:58:35.184 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2024-04-24T23:58:35.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:35.186 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-24T23:58:35.186 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:58:35.186 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:58:35.186 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:58:35.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.285+0000 7fa3b1ba0640 1 Processor -- start 2024-04-24T23:58:35.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.285+0000 7fa3b1ba0640 1 -- start start 2024-04-24T23:58:35.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.286+0000 7fa3b1ba0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa3ac15be70 0x7fa3ac0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.286+0000 7fa3b1ba0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac0cd330 con 0x7fa3ac0ccf00 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.286+0000 7fa3b1ba0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac0cdba0 con 0x7fa3ac15c250 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.286+0000 7fa3b1ba0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ac0c5ce0 con 0x7fa3ac15be70 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.286+0000 7fa3b139f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa3ac0ccf00 0x7fa3ac0c6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55622/0 (socket says 172.21.15.113:55622) 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.286+0000 7fa3b139f640 1 -- 172.21.15.113:0/1714866725 learned_addr learned my addr 172.21.15.113:0/1714866725 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753907935 0 0) 0x7fa3ac0cd330 con 0x7fa3ac0ccf00 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa394003520 con 0x7fa3ac0ccf00 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b0b9e640 1 --2- 172.21.15.113:0/1714866725 >> v2:172.21.15.178:3300/0 conn(0x7fa3ac15be70 0x7fa3ac0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1903772195 0 0) 0x7fa394003520 con 0x7fa3ac0ccf00 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 >> v2:172.21.15.178:3300/0 conn(0x7fa3ac15be70 msgr2=0x7fa3ac0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:35.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 --2- 172.21.15.113:0/1714866725 >> v2:172.21.15.178:3300/0 conn(0x7fa3ac15be70 0x7fa3ac0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 >> v1:172.21.15.178:6789/0 conn(0x7fa3ac15c250 legacy=0x7fa3ac0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac0cd5e0 con 0x7fa3ac0ccf00 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fa3a40019f0 con 0x7fa3ac0ccf00 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 >> v1:172.21.15.113:6789/0 conn(0x7fa3ac0ccf00 legacy=0x7fa3ac0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940039b0 0x7fa394003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa394004800 0x7fa394004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3940051c0 0x7fa39400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0c5ce0 con 0x7fa3940051c0 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa39400c060 con 0x7fa394004800 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa39400c260 con 0x7fa3940039b0 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b139f640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3940051c0 0x7fa39400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b0b9e640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940039b0 0x7fa394003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3abfff640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa394004800 0x7fa394004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b139f640 1 -- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940039b0 msgr2=0x7fa394003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:35.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b139f640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940039b0 0x7fa394003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b139f640 1 -- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa394004800 msgr2=0x7fa394004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:35.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b139f640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa394004800 0x7fa394004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.287+0000 7fa3b139f640 1 -- 172.21.15.113:0/1714866725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa394003520 con 0x7fa3940051c0 2024-04-24T23:58:35.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.288+0000 7fa3b139f640 1 -- 172.21.15.113:0/1714866725 reap_dead start 2024-04-24T23:58:35.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.288+0000 7fa3b139f640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3940051c0 0x7fa39400b9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400a860 tx=0x7fa394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fa8ec8d980dca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:35.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.289+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400a480 con 0x7fa3940051c0 2024-04-24T23:58:35.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.289+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a402f070 con 0x7fa3940051c0 2024-04-24T23:58:35.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.289+0000 7fa3ab7fe640 1 -- 172.21.15.113:0/1714866725 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a402bcc0 con 0x7fa3940051c0 2024-04-24T23:58:35.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.290+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/1714866725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3940051c0 msgr2=0x7fa39400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.290+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3940051c0 0x7fa39400b9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400a860 tx=0x7fa394005830 comp rx=0 tx=0).stop 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.290+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/1714866725 shutdown_connections 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.290+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/1714866725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3940051c0 0x7fa39400b9c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.290+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/1714866725 >> 172.21.15.113:0/1714866725 conn(0x7fa3ac062500 msgr2=0x7fa3ac09aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.290+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/1714866725 shutdown_connections 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.290+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/1714866725 wait complete. 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.291+0000 7fa3b1ba0640 1 Processor -- start 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.291+0000 7fa3b1ba0640 1 -- start start 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b1ba0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940051c0 0x7fa3ac0b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b1ba0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ac0b4bb0 0x7fa3ac150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b1ba0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3ac0cb530 0x7fa3ac151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b1ba0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0cc7f0 con 0x7fa3ac0cb530 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b1ba0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0c5970 con 0x7fa3ac0b4bb0 2024-04-24T23:58:35.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b1ba0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac0cc970 con 0x7fa3940051c0 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3ac0cb530 0x7fa3ac151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3ac0cb530 0x7fa3ac151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50672/0 (socket says 172.21.15.113:50672) 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 -- 172.21.15.113:0/3718442049 learned_addr learned my addr 172.21.15.113:0/3718442049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3abfff640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ac0b4bb0 0x7fa3ac150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b0b9e640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940051c0 0x7fa3ac0b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 -- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940051c0 msgr2=0x7fa3ac0b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940051c0 0x7fa3ac0b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 -- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ac0b4bb0 msgr2=0x7fa3ac150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ac0b4bb0 0x7fa3ac150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.292+0000 7fa3b139f640 1 -- 172.21.15.113:0/3718442049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac0b5090 con 0x7fa3ac0cb530 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.293+0000 7fa3b139f640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3ac0cb530 0x7fa3ac151340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400a980 tx=0x7fa3ac0cd7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.293+0000 7fa3a97fa640 1 -- 172.21.15.113:0/3718442049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4044070 con 0x7fa3ac0cb530 2024-04-24T23:58:35.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.293+0000 7fa3a97fa640 1 -- 172.21.15.113:0/3718442049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a402bdc0 con 0x7fa3ac0cb530 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.293+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac0caa50 con 0x7fa3ac0cb530 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.293+0000 7fa3a97fa640 1 -- 172.21.15.113:0/3718442049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400a480 con 0x7fa3ac0cb530 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.293+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac0b7a10 con 0x7fa3ac0cb530 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.294+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ac068220 con 0x7fa3ac0cb530 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.294+0000 7fa3a97fa640 1 -- 172.21.15.113:0/3718442049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3a4047070 con 0x7fa3ac0cb530 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.294+0000 7fa3a97fa640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39806be30 0x7fa39806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.294+0000 7fa3a97fa640 1 -- 172.21.15.113:0/3718442049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3a40aa7e0 con 0x7fa3ac0cb530 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.295+0000 7fa3b0b9e640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39806be30 0x7fa39806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.295+0000 7fa3b0b9e640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39806be30 0x7fa39806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009b40 tx=0x7fa39c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:35.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.299+0000 7fa3a97fa640 1 -- 172.21.15.113:0/3718442049 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3a4078c60 con 0x7fa3ac0cb530 2024-04-24T23:58:35.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.508+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa3ac15f690 con 0x7fa3ac0cb530 2024-04-24T23:58:35.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.509+0000 7fa3a97fa640 1 -- 172.21.15.113:0/3718442049 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fa3a407cf60 con 0x7fa3ac0cb530 2024-04-24T23:58:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:58:35.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39806be30 msgr2=0x7fa39806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:35.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39806be30 0x7fa39806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009b40 tx=0x7fa39c009190 comp rx=0 tx=0).stop 2024-04-24T23:58:35.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3ac0cb530 msgr2=0x7fa3ac151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:35.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3ac0cb530 0x7fa3ac151340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400a980 tx=0x7fa3ac0cd7b0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 shutdown_connections 2024-04-24T23:58:35.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39806be30 0x7fa39806e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ac0b4bb0 0x7fa3ac150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3ac0cb530 0x7fa3ac151340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 --2- 172.21.15.113:0/3718442049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3940051c0 0x7fa3ac0b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:35.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 >> 172.21.15.113:0/3718442049 conn(0x7fa3ac062500 msgr2=0x7fa3ac0c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:35.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 shutdown_connections 2024-04-24T23:58:35.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35.511+0000 7fa3b1ba0640 1 -- 172.21.15.113:0/3718442049 wait complete. 2024-04-24T23:58:35.521 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:58:35.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:58:35.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-24T23:58:35.521 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:58:35.522 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:58:35.522 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:58:35.522 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:58:35.522 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:58:35.522 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:58:35.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:35.522 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:35.537 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42765sec preferred_lft 42765sec 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-24T23:58:35.538 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:35.539 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:58:35.539 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:35.539 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:35.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:58:35.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2024-04-24T23:58:35.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2024-04-24T23:58:35.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:58:35.539 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:35.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:35.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:35.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:35.713 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:58:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-24T23:58:35.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:35.738 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-24T23:58:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-24T23:58:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:58:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:58:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:58:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:58:35.806 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:35.806 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:35.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:58:35.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:58:35.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:58:35.807 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:35.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:36.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:36.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:36.066 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-24T23:58:36.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:36.081 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-04-24T23:58:36.136 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:36.136 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:36.136 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:58:36.136 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:58:36.136 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:36.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:36.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:36.251 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:58:36.251 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:36.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:36.393 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:36.393 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:36.393 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-24T23:58:36.393 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:58:36.393 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:36.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:36.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:36.467 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:58:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:58:36.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:36.482 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:58:36.533 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:58:36.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:36.533 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:58:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:58:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:58:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:58:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:58:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:58:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:58:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:58:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:58:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:58:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-24T23:58:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-24T23:58:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-24T23:58:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:58:36.740 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:58:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:58:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:58:36.741 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:58:36.742 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:58:36.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:36.742 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:58:36.772 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:36.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:36.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:client4418 2024-04-24T23:58:36.871 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:36.872 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:36.887 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:36.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:36.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:client4418 2024-04-24T23:58:36.985 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:37.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:37.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:client4418 2024-04-24T23:58:37.044 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4eaaf81-6120-4e3f-a50c-97aa81453142.client4418/status') 2024-04-24T23:58:37.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.4418 (3)172.21.15.113:0/2667135521 2024-04-24T23:58:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2024-04-24T23:58:37.104 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2024-04-24T23:58:37.104 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2024-04-24T23:58:37.104 INFO:teuthology.orchestra.run.smithi113.stderr:71 bytes copied, 2.2435e-05 s, 3.2 MB/s 2024-04-24T23:58:37.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:37.111 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-24T23:58:37.111 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:58:37.111 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T23:58:37.112 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:58:37.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.209+0000 7f8b4647d640 1 Processor -- start 2024-04-24T23:58:37.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.209+0000 7f8b4647d640 1 -- start start 2024-04-24T23:58:37.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.209+0000 7f8b4647d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b40144d60 0x7f8b400b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:37.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b4647d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b401453c0 0x7f8b4006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b4647d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b4006e510 0x7f8b4015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b4647d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8b40145c40 con 0x7f8b4006e510 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b4647d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4006e940 con 0x7f8b401453c0 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b4647d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b400cfad0 con 0x7f8b40144d60 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b4006e510 0x7f8b4015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b4006e510 0x7f8b4015fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50680/0 (socket says 172.21.15.113:50680) 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3188832850 learned_addr learned my addr 172.21.15.113:0/3188832850 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b3f7fe640 1 --2- 172.21.15.113:0/3188832850 >> v2:172.21.15.178:3300/0 conn(0x7f8b401453c0 0x7f8b4006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b3ffff640 1 --2- 172.21.15.113:0/3188832850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b40144d60 0x7f8b400b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3188832850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b40144d60 msgr2=0x7f8b400b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 --2- 172.21.15.113:0/3188832850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b40144d60 0x7f8b400b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3188832850 >> v2:172.21.15.178:3300/0 conn(0x7f8b401453c0 msgr2=0x7f8b4006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 --2- 172.21.15.113:0/3188832850 >> v2:172.21.15.178:3300/0 conn(0x7f8b401453c0 0x7f8b4006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.210+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3188832850 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b400cf230 con 0x7f8b4006e510 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b44a1a640 1 --2- 172.21.15.113:0/3188832850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b4006e510 0x7f8b4015fc30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b4006f310 tx=0x7f8b3802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580d6ee6d3a3a4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b3effd640 1 -- 172.21.15.113:0/3188832850 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38036070 con 0x7f8b4006e510 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b3effd640 1 -- 172.21.15.113:0/3188832850 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b38031600 con 0x7f8b4006e510 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b3effd640 1 -- 172.21.15.113:0/3188832850 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380319d0 con 0x7f8b4006e510 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 -- 172.21.15.113:0/3188832850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b4006e510 msgr2=0x7f8b4015fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3188832850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b4006e510 0x7f8b4015fc30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b4006f310 tx=0x7f8b3802a630 comp rx=0 tx=0).stop 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 -- 172.21.15.113:0/3188832850 shutdown_connections 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3188832850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b4006e510 0x7f8b4015fc30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3188832850 >> v2:172.21.15.178:3300/0 conn(0x7f8b401453c0 0x7f8b4006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3188832850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b40144d60 0x7f8b400b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 -- 172.21.15.113:0/3188832850 >> 172.21.15.113:0/3188832850 conn(0x7f8b400b1a50 msgr2=0x7f8b400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 -- 172.21.15.113:0/3188832850 shutdown_connections 2024-04-24T23:58:37.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.211+0000 7f8b4647d640 1 -- 172.21.15.113:0/3188832850 wait complete. 2024-04-24T23:58:37.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 Processor -- start 2024-04-24T23:58:37.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 -- start start 2024-04-24T23:58:37.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b4006e510 0x7f8b401f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:37.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b40144d60 0x7f8b4015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b401453c0 0x7f8b4015ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b40070e10 con 0x7f8b401453c0 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b400cf7f0 con 0x7f8b40144d60 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.213+0000 7f8b4647d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b40145140 con 0x7f8b4006e510 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b401453c0 0x7f8b4015ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b401453c0 0x7f8b4015ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50692/0 (socket says 172.21.15.113:50692) 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3830123199 learned_addr learned my addr 172.21.15.113:0/3830123199 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b3ffff640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b4006e510 0x7f8b401f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b3f7fe640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b40144d60 0x7f8b4015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b4006e510 msgr2=0x7f8b401f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b4006e510 0x7f8b401f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b40144d60 msgr2=0x7f8b4015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:37.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b40144d60 0x7f8b4015af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 -- 172.21.15.113:0/3830123199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b401607d0 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b44a1a640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b401453c0 0x7f8b4015ffd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b38034040 tx=0x7f8b38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b3cff9640 1 -- 172.21.15.113:0/3830123199 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38036050 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b3cff9640 1 -- 172.21.15.113:0/3830123199 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b38042070 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b40160510 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b3cff9640 1 -- 172.21.15.113:0/3830123199 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38009510 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.214+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4015a470 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.215+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b400691a0 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.216+0000 7f8b3cff9640 1 -- 172.21.15.113:0/3830123199 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b38007bf0 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.216+0000 7f8b3cff9640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b2c06be30 0x7f8b2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.216+0000 7f8b3cff9640 1 -- 172.21.15.113:0/3830123199 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8b380abc10 con 0x7f8b401453c0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.216+0000 7f8b3ffff640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b2c06be30 0x7f8b2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:37.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.216+0000 7f8b3ffff640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b2c06be30 0x7f8b2c06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006fd0 tx=0x7f8b30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:37.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.220+0000 7f8b3cff9640 1 -- 172.21.15.113:0/3830123199 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8b3807a650 con 0x7f8b401453c0 2024-04-24T23:58:37.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.430+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b400b27e0 con 0x7f8b401453c0 2024-04-24T23:58:37.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.430+0000 7f8b3cff9640 1 -- 172.21.15.113:0/3830123199 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f8b38079f50 con 0x7f8b401453c0 2024-04-24T23:58:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:58:37.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b2c06be30 msgr2=0x7f8b2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b2c06be30 0x7f8b2c06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006fd0 tx=0x7f8b30008040 comp rx=0 tx=0).stop 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b401453c0 msgr2=0x7f8b4015ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b401453c0 0x7f8b4015ffd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b38034040 tx=0x7f8b38004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 shutdown_connections 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b2c06be30 0x7f8b2c06e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b40144d60 0x7f8b4015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b401453c0 0x7f8b4015ffd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 --2- 172.21.15.113:0/3830123199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b4006e510 0x7f8b401f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 >> 172.21.15.113:0/3830123199 conn(0x7f8b400b1a50 msgr2=0x7f8b40140fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 shutdown_connections 2024-04-24T23:58:37.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37.432+0000 7f8b4647d640 1 -- 172.21.15.113:0/3830123199 wait complete. 2024-04-24T23:58:37.443 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:58:37.443 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:58:37.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-24T23:58:37.444 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-24T23:58:37.444 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T23:58:37.444 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:58:37.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:58:37.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:58:37.444 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-24T23:58:37.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:37.444 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-24T23:58:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:37.459 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:37.459 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:37.459 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.459 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-24T23:58:37.460 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link tentative 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link tentative 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:37.461 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:58:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-24T23:58:37.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:37.523 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-24T23:58:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-24T23:58:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:58:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:58:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:58:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:58:37.579 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:37.579 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:37.579 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:58:37.579 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:58:37.579 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:58:37.579 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:37.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:37.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:37.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:37.701 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-24T23:58:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:58:37.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:37.717 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-04-24T23:58:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-24T23:58:37.772 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:37.772 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:37.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:58:37.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-24T23:58:37.772 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:37.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:37.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:37.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:37.886 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T23:58:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:37.909 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:37.909 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:e7:e0:ef:e4:cd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84e7:e0ff:feef:e4cd/64 scope link 2024-04-24T23:58:37.910 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:37.911 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T23:58:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 86:e7:e0:ef:e4:cd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::84e7:e0ff:feef:e4cd/64 scope link 2024-04-24T23:58:37.975 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:37.976 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-24T23:58:37.977 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:38.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:38.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:38.123 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:38.123 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:58:38.123 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.1 up 2024-04-24T23:58:38.123 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T23:58:38.123 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:58:38.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:38.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:38.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:38.196 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-24T23:58:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-24T23:58:38.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:38.211 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-24T23:58:38.267 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:58:38.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:38.268 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:58:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:58:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-24T23:58:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:58:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:58:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-24T23:58:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:58:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-24T23:58:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-24T23:58:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-24T23:58:38.372 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:58:38.372 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:58:38.372 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:58:38.372 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:58:38.374 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:58:38.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:38.374 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-24T23:58:38.438 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:38.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:38.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:client4555 2024-04-24T23:58:38.535 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:38.535 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:38.551 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:38.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:client4555 2024-04-24T23:58:38.647 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:38.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:client4555 2024-04-24T23:58:38.703 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4eaaf81-6120-4e3f-a50c-97aa81453142.client4555/status') 2024-04-24T23:58:38.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.4555 (3)172.21.15.113:0/2691837341 2024-04-24T23:58:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2024-04-24T23:58:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2024-04-24T23:58:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2024-04-24T23:58:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:71 bytes copied, 2.2009e-05 s, 3.2 MB/s 2024-04-24T23:58:38.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:38.767 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-24T23:58:38.767 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:58:38.767 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:58:38.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:58:38.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.865+0000 7f3cba23f640 1 Processor -- start 2024-04-24T23:58:38.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.865+0000 7f3cba23f640 1 -- start start 2024-04-24T23:58:38.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cba23f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 0x7f3cb4169160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cba23f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cb40b3c20 con 0x7f3cb415e240 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cba23f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cb415e620 con 0x7f3cb40b5620 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cba23f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb4072d90 con 0x7f3cb406e6e0 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3cb415e240 0x7f3cb41627e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58614/0 (socket says 172.21.15.113:58614) 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb37fe640 1 -- 172.21.15.113:0/2765231707 learned_addr learned my addr 172.21.15.113:0/2765231707 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb3fff640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 0x7f3cb4169160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1987423260 0 0) 0x7f3cb40b3c20 con 0x7f3cb415e240 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c9c003520 con 0x7f3cb415e240 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3073460272 0 0) 0x7f3c9c003520 con 0x7f3cb415e240 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 msgr2=0x7f3cb4169160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 0x7f3cb4169160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 >> v1:172.21.15.178:6789/0 conn(0x7f3cb40b5620 legacy=0x7f3cb406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb4072210 con 0x7f3cb415e240 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f3ca8001670 con 0x7f3cb415e240 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 >> v1:172.21.15.113:6789/0 conn(0x7f3cb415e240 legacy=0x7f3cb41627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c9c0039e0 0x7f3c9c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c004390 0x7f3c9c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 0x7f3c9c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4072d90 con 0x7f3c9c0039e0 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c00b9c0 con 0x7f3c9c004390 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.866+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c00bbc0 con 0x7f3cb406e6e0 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb37fe640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c9c0039e0 0x7f3c9c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb2ffd640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c004390 0x7f3c9c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb3fff640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 0x7f3c9c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb37fe640 1 -- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 msgr2=0x7f3c9c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb37fe640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb406e6e0 0x7f3c9c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb37fe640 1 -- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c004390 msgr2=0x7f3c9c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb37fe640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c004390 0x7f3c9c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb37fe640 1 -- 172.21.15.113:0/2765231707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c003520 con 0x7f3c9c0039e0 2024-04-24T23:58:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.867+0000 7f3cb37fe640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c9c0039e0 0x7f3c9c003dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800c290 tx=0x7f3c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b79fe6b5e8d89b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8007d50 con 0x7f3c9c0039e0 2024-04-24T23:58:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca8005af0 con 0x7f3c9c0039e0 2024-04-24T23:58:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cb27fc640 1 -- 172.21.15.113:0/2765231707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8007070 con 0x7f3c9c0039e0 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cba23f640 1 -- 172.21.15.113:0/2765231707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c9c0039e0 msgr2=0x7f3c9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cba23f640 1 --2- 172.21.15.113:0/2765231707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c9c0039e0 0x7f3c9c003dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800c290 tx=0x7f3c9c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cb3fff640 1 -- 172.21.15.113:0/2765231707 reap_dead start 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cba23f640 1 -- 172.21.15.113:0/2765231707 shutdown_connections 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cba23f640 1 -- 172.21.15.113:0/2765231707 >> 172.21.15.113:0/2765231707 conn(0x7f3cb40b1a50 msgr2=0x7f3cb40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cba23f640 1 -- 172.21.15.113:0/2765231707 shutdown_connections 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.868+0000 7f3cba23f640 1 -- 172.21.15.113:0/2765231707 wait complete. 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 Processor -- start 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 -- start start 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c0039e0 0x7f3cb415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb4160f40 0x7f3cb41613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb41462e0 0x7f3cb4146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4072ab0 con 0x7f3cb41462e0 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb40727d0 con 0x7f3c9c0039e0 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cba23f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb40ad890 con 0x7f3cb4160f40 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.870+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb41462e0 0x7f3cb4146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb41462e0 0x7f3cb4146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41224/0 (socket says 172.21.15.113:41224) 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 -- 172.21.15.113:0/1591969841 learned_addr learned my addr 172.21.15.113:0/1591969841 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb2ffd640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb4160f40 0x7f3cb41613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb37fe640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c0039e0 0x7f3cb415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 -- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb4160f40 msgr2=0x7f3cb41613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb4160f40 0x7f3cb41613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 -- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c0039e0 msgr2=0x7f3cb415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c0039e0 0x7f3cb415f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 -- 172.21.15.113:0/1591969841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb41588b0 con 0x7f3cb41462e0 2024-04-24T23:58:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cb3fff640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb41462e0 0x7f3cb4146760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40af260 tx=0x7f3cac0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3c9bfff640 1 -- 172.21.15.113:0/1591969841 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac003990 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3c9bfff640 1 -- 172.21.15.113:0/1591969841 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cac01b070 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb4158b90 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3c9bfff640 1 -- 172.21.15.113:0/1591969841 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac003b70 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.871+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb4159150 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.872+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb40691a0 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.873+0000 7f3c9bfff640 1 -- 172.21.15.113:0/1591969841 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cac005d10 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.873+0000 7f3c9bfff640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ca006be60 0x7f3ca006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.873+0000 7f3c9bfff640 1 -- 172.21.15.113:0/1591969841 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3cac08b320 con 0x7f3cb41462e0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.873+0000 7f3cb37fe640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ca006be60 0x7f3ca006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:38.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.874+0000 7f3cb37fe640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ca006be60 0x7f3ca006e320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800c290 tx=0x7f3ca802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:38.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:38.881+0000 7f3c9bfff640 1 -- 172.21.15.113:0/1591969841 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cac025020 con 0x7f3cb41462e0 2024-04-24T23:58:39.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.098+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3cb4161b40 con 0x7f3cb41462e0 2024-04-24T23:58:39.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.099+0000 7f3c9bfff640 1 -- 172.21.15.113:0/1591969841 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f3cac059d60 con 0x7f3cb41462e0 2024-04-24T23:58:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:58:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ca006be60 msgr2=0x7f3ca006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ca006be60 0x7f3ca006e320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800c290 tx=0x7f3ca802e000 comp rx=0 tx=0).stop 2024-04-24T23:58:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb41462e0 msgr2=0x7f3cb4146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb41462e0 0x7f3cb4146760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40af260 tx=0x7f3cac0115a0 comp rx=0 tx=0).stop 2024-04-24T23:58:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 shutdown_connections 2024-04-24T23:58:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ca006be60 0x7f3ca006e320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb41462e0 0x7f3cb4146760 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb4160f40 0x7f3cb41613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 --2- 172.21.15.113:0/1591969841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c9c0039e0 0x7f3cb415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 >> 172.21.15.113:0/1591969841 conn(0x7f3cb40b1a50 msgr2=0x7f3cb4070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 shutdown_connections 2024-04-24T23:58:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:39.101+0000 7f3cba23f640 1 -- 172.21.15.113:0/1591969841 wait complete. 2024-04-24T23:58:39.111 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:58:39.111 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:58:39.111 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2024-04-24T23:58:39.112 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:58:39.112 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:58:39.112 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:58:39.112 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:58:39.112 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:58:39.112 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:58:39.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:39.112 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:39.131 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f0 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f1 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 42753sec preferred_lft 42753sec 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::6a91:39dc:88af:d861/64 scope link noprefixroute 2024-04-24T23:58:39.132 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:39.132 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:58:39.133 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:39.133 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:39.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:58:39.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr flush dev ceph-brx 2024-04-24T23:58:39.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set ceph-brx up 2024-04-24T23:58:39.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:58:39.133 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:39.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:39.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:39.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:39.385 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:58:39.409 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-04-24T23:58:39.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:39.410 DEBUG:teuthology.orchestra.run.smithi178:> route 2024-04-24T23:58:39.480 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2024-04-24T23:58:39.480 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:58:39.480 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:58:39.480 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:58:39.480 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:58:39.480 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:39.480 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:39.480 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:58:39.481 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:58:39.481 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:58:39.481 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:39.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:39.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:39.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:39.776 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-24T23:58:39.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:39.791 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2024-04-24T23:58:39.847 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:39.847 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:39.847 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:58:39.847 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T23:58:39.847 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:39.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:39.968 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T23:58:39.968 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:40.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:40.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:40.094 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:40.095 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:40.095 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.0 up 2024-04-24T23:58:40.095 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:58:40.095 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:40.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:40.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:40.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:40.171 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:58:40.185 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:58:40.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:40.185 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:58:40.239 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:58:40.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:40.239 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:58:40.330 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:58:40.330 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-24T23:58:40.330 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-24T23:58:40.429 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:58:40.429 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-24T23:58:40.429 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-24T23:58:40.429 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:40.429 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-24T23:58:40.434 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-24T23:58:40.435 INFO:teuthology.orchestra.run.smithi178.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-24T23:58:40.435 INFO:teuthology.orchestra.run.smithi178.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:58:40.435 INFO:teuthology.orchestra.run.smithi178.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:58:40.435 INFO:teuthology.orchestra.run.smithi178.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:58:40.435 INFO:teuthology.orchestra.run.smithi178.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:58:40.437 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:58:40.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:40.437 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:58:40.461 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:40.557 INFO:teuthology.orchestra.run.smithi178.stdout:client4680 2024-04-24T23:58:40.557 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:40.557 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:40.572 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:40.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:40.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:40.668 INFO:teuthology.orchestra.run.smithi178.stdout:client4680 2024-04-24T23:58:40.668 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:40.725 INFO:teuthology.orchestra.run.smithi178.stdout:client4680 2024-04-24T23:58:40.725 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4eaaf81-6120-4e3f-a50c-97aa81453142.client4680/status') 2024-04-24T23:58:40.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:40.782 INFO:teuthology.orchestra.run.smithi178.stdout:instance: client.4680 (3)172.21.15.178:0/2611944684 2024-04-24T23:58:40.782 INFO:teuthology.orchestra.run.smithi178.stdout:blocklisted: false 2024-04-24T23:58:40.783 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records in 2024-04-24T23:58:40.783 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records out 2024-04-24T23:58:40.783 INFO:teuthology.orchestra.run.smithi178.stderr:71 bytes copied, 2.1736e-05 s, 3.3 MB/s 2024-04-24T23:58:40.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:40.789 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-24T23:58:40.790 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:58:40.790 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T23:58:40.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:58:40.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.899+0000 7f3c11c58640 1 Processor -- start 2024-04-24T23:58:40.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.899+0000 7f3c11c58640 1 -- start start 2024-04-24T23:58:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c11c58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c11c58640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3c0c0b5240 0x7f3c0c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c11c58640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c0c0b5680 0x7f3c0c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c11c58640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0c0b4510 con 0x7f3c0c0b5680 2024-04-24T23:58:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c11c58640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0c144cf0 con 0x7f3c0c0b5240 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c11c58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b2f30 con 0x7f3c0c144910 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c0c0b5680 0x7f3c0c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c0c0b5680 0x7f3c0c06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41238/0 (socket says 172.21.15.113:41238) 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 -- 172.21.15.113:0/4183541704 learned_addr learned my addr 172.21.15.113:0/4183541704 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0b7fe640 1 --2- 172.21.15.113:0/4183541704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0affd640 1 --2- 172.21.15.113:0/4183541704 >> v2:172.21.15.178:3300/0 conn(0x7f3c0c0b5240 0x7f3c0c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 -- 172.21.15.113:0/4183541704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 msgr2=0x7f3c0c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 --2- 172.21.15.113:0/4183541704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 -- 172.21.15.113:0/4183541704 >> v2:172.21.15.178:3300/0 conn(0x7f3c0c0b5240 msgr2=0x7f3c0c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 --2- 172.21.15.113:0/4183541704 >> v2:172.21.15.178:3300/0 conn(0x7f3c0c0b5240 0x7f3c0c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.900+0000 7f3c0bfff640 1 -- 172.21.15.113:0/4183541704 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c0b3210 con 0x7f3c0c0b5680 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.901+0000 7f3c0bfff640 1 --2- 172.21.15.113:0/4183541704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5680 0x7f3c0c06e510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c06f310 tx=0x7f3c0402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6165832319747d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.901+0000 7f3c0a7fc640 1 -- 172.21.15.113:0/4183541704 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04036070 con 0x7f3c0c0b5680 2024-04-24T23:58:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.901+0000 7f3c0a7fc640 1 -- 172.21.15.113:0/4183541704 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c04031600 con 0x7f3c0c0b5680 2024-04-24T23:58:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.901+0000 7f3c0a7fc640 1 -- 172.21.15.113:0/4183541704 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c040319d0 con 0x7f3c0c0b5680 2024-04-24T23:58:40.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 -- 172.21.15.113:0/4183541704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5680 msgr2=0x7f3c0c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:40.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 --2- 172.21.15.113:0/4183541704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5680 0x7f3c0c06e510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c06f310 tx=0x7f3c0402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 -- 172.21.15.113:0/4183541704 shutdown_connections 2024-04-24T23:58:40.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 --2- 172.21.15.113:0/4183541704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5680 0x7f3c0c06e510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 --2- 172.21.15.113:0/4183541704 >> v2:172.21.15.178:3300/0 conn(0x7f3c0c0b5240 0x7f3c0c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 --2- 172.21.15.113:0/4183541704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 -- 172.21.15.113:0/4183541704 >> 172.21.15.113:0/4183541704 conn(0x7f3c0c0b1a50 msgr2=0x7f3c0c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 -- 172.21.15.113:0/4183541704 shutdown_connections 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.902+0000 7f3c11c58640 1 -- 172.21.15.113:0/4183541704 wait complete. 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.903+0000 7f3c11c58640 1 Processor -- start 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.903+0000 7f3c11c58640 1 -- start start 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c11c58640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5240 0x7f3c0c14da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c11c58640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c0c0b5680 0x7f3c0c0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c11c58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c11c58640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b4da0 con 0x7f3c0c0b5240 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c11c58640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b37d0 con 0x7f3c0c0b5680 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c11c58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b4f20 con 0x7f3c0c144910 2024-04-24T23:58:40.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5240 0x7f3c0c14da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5240 0x7f3c0c14da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41248/0 (socket says 172.21.15.113:41248) 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 -- 172.21.15.113:0/1837429496 learned_addr learned my addr 172.21.15.113:0/1837429496 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0affd640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c0c0b5680 0x7f3c0c0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0bfff640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 -- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 msgr2=0x7f3c0c14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c14c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 -- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c0c0b5680 msgr2=0x7f3c0c0adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c0c0b5680 0x7f3c0c0adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.904+0000 7f3c0b7fe640 1 -- 172.21.15.113:0/1837429496 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c14a530 con 0x7f3c0c0b5240 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.905+0000 7f3c0b7fe640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5240 0x7f3c0c14da20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00d830 tx=0x7f3bfc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.905+0000 7f3bf3fff640 1 -- 172.21.15.113:0/1837429496 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00b6c0 con 0x7f3c0c0b5240 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.905+0000 7f3bf3fff640 1 -- 172.21.15.113:0/1837429496 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc019070 con 0x7f3c0c0b5240 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.905+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c14a7f0 con 0x7f3c0c0b5240 2024-04-24T23:58:40.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.905+0000 7f3bf3fff640 1 -- 172.21.15.113:0/1837429496 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00ba80 con 0x7f3c0c0b5240 2024-04-24T23:58:40.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.905+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c14b090 con 0x7f3c0c0b5240 2024-04-24T23:58:40.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.906+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c0691a0 con 0x7f3c0c0b5240 2024-04-24T23:58:40.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.906+0000 7f3bf3fff640 1 -- 172.21.15.113:0/1837429496 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bfc01e460 con 0x7f3c0c0b5240 2024-04-24T23:58:40.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.907+0000 7f3bf3fff640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3bf806be30 0x7f3bf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:40.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.907+0000 7f3bf3fff640 1 -- 172.21.15.113:0/1837429496 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3bfc08b530 con 0x7f3c0c0b5240 2024-04-24T23:58:40.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.907+0000 7f3c0affd640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3bf806be30 0x7f3bf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:40.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.907+0000 7f3c0affd640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3bf806be30 0x7f3bf806e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c00006fd0 tx=0x7f3c00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:40.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:40.911+0000 7f3bf3fff640 1 -- 172.21.15.113:0/1837429496 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3bfc059ee0 con 0x7f3c0c0b5240 2024-04-24T23:58:41.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.122+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3c0c15e090 con 0x7f3c0c0b5240 2024-04-24T23:58:41.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.123+0000 7f3bf3fff640 1 -- 172.21.15.113:0/1837429496 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f3bfc0597e0 con 0x7f3c0c0b5240 2024-04-24T23:58:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:58:41.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3bf806be30 msgr2=0x7f3bf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:41.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3bf806be30 0x7f3bf806e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c00006fd0 tx=0x7f3c00008040 comp rx=0 tx=0).stop 2024-04-24T23:58:41.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5240 msgr2=0x7f3c0c14da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5240 0x7f3c0c14da20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00d830 tx=0x7f3bfc00dcf0 comp rx=0 tx=0).stop 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 shutdown_connections 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3bf806be30 0x7f3bf806e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c0c0b5680 0x7f3c0c0adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c0c144910 0x7f3c0c14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 --2- 172.21.15.113:0/1837429496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c0c0b5240 0x7f3c0c14da20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 >> 172.21.15.113:0/1837429496 conn(0x7f3c0c0b1a50 msgr2=0x7f3c0c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 shutdown_connections 2024-04-24T23:58:41.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:41.125+0000 7f3c11c58640 1 -- 172.21.15.113:0/1837429496 wait complete. 2024-04-24T23:58:41.136 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:58:41.136 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:58:41.136 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2024-04-24T23:58:41.136 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-24T23:58:41.136 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T23:58:41.136 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:58:41.136 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:58:41.137 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:58:41.137 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-24T23:58:41.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.137 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2024-04-24T23:58:41.152 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:41.152 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f0 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f1 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:41.153 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 42751sec preferred_lft 42751sec 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::6a91:39dc:88af:d861/64 scope link noprefixroute 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 92:4e:e5:47:29:51 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::8814:37ff:fecf:2937/64 scope link 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:58:41.154 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 92:4e:e5:47:29:51 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:58:41.155 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::904e:e5ff:fe47:2951/64 scope link tentative 2024-04-24T23:58:41.155 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.155 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:58:41.215 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-04-24T23:58:41.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.217 DEBUG:teuthology.orchestra.run.smithi178:> route 2024-04-24T23:58:41.272 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2024-04-24T23:58:41.272 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:58:41.273 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:58:41.273 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:58:41.273 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:58:41.273 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:41.273 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:41.273 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:58:41.273 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:58:41.273 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:58:41.273 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:41.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:41.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:41.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.397 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-24T23:58:41.411 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T23:58:41.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.412 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2024-04-24T23:58:41.466 INFO:teuthology.orchestra.run.smithi178.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-24T23:58:41.466 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:41.467 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:41.467 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T23:58:41.467 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-24T23:58:41.467 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:41.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:41.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:41.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.580 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T23:58:41.603 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0e:09:c0:e3:69:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::c09:c0ff:fee3:69dd/64 scope link tentative 2024-04-24T23:58:41.604 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.605 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0e:09:c0:e3:69:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T23:58:41.669 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T23:58:41.670 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.670 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::c09:c0ff:fee3:69dd/64 scope link 2024-04-24T23:58:41.670 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:58:41.671 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-24T23:58:41.671 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:41.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:41.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:41.818 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:58:41.818 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-24T23:58:41.818 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.1 up 2024-04-24T23:58:41.818 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T23:58:41.818 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-24T23:58:41.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:41.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:41.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:41.892 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-24T23:58:41.907 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-24T23:58:41.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:58:41.907 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-24T23:58:41.961 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:58:41.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:41.961 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:58:42.042 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:58:42.042 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-24T23:58:42.042 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-24T23:58:42.050 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:58:42.050 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-24T23:58:42.050 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-24T23:58:42.050 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:42.050 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-24T23:58:42.055 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-24T23:58:42.056 INFO:teuthology.orchestra.run.smithi178.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-24T23:58:42.056 INFO:teuthology.orchestra.run.smithi178.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:58:42.056 INFO:teuthology.orchestra.run.smithi178.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:58:42.056 INFO:teuthology.orchestra.run.smithi178.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:58:42.056 INFO:teuthology.orchestra.run.smithi178.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:58:42.057 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:58:42.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:42.057 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-24T23:58:42.122 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:42.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:42.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:42.218 INFO:teuthology.orchestra.run.smithi178.stdout:client4439 2024-04-24T23:58:42.218 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:42.218 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:42.233 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:42.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:42.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:42.330 INFO:teuthology.orchestra.run.smithi178.stdout:client4439 2024-04-24T23:58:42.331 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:58:42.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:58:42.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:42.385 INFO:teuthology.orchestra.run.smithi178.stdout:client4439 2024-04-24T23:58:42.386 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4eaaf81-6120-4e3f-a50c-97aa81453142.client4439/status') 2024-04-24T23:58:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:58:42.442 INFO:teuthology.orchestra.run.smithi178.stdout:instance: client.4439 (3)192.168.0.2:0/199741192 2024-04-24T23:58:42.442 INFO:teuthology.orchestra.run.smithi178.stdout:blocklisted: false 2024-04-24T23:58:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records in 2024-04-24T23:58:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records out 2024-04-24T23:58:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:68 bytes copied, 2.1539e-05 s, 3.2 MB/s 2024-04-24T23:58:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-24T23:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:58:42.449 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-24T23:58:42.465 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.573+0000 7fd589c2a640 1 Processor -- start 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.573+0000 7fd589c2a640 1 -- start start 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.574+0000 7fd589c2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 0x7fd584070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.574+0000 7fd589c2a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5840b2d50 con 0x7fd5840b3af0 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.574+0000 7fd589c2a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd584072a70 con 0x7fd584072ea0 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.574+0000 7fd589c2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5840b3010 con 0x7fd58415a6b0 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.574+0000 7fd583fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd5840b3af0 0x7fd5840b3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58616/0 (socket says 172.21.15.113:58616) 2024-04-24T23:58:42.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.574+0000 7fd583fff640 1 -- 172.21.15.113:0/1411895829 learned_addr learned my addr 172.21.15.113:0/1411895829 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd5837fe640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 0x7fd584070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897055902 0 0) 0x7fd5840b2d50 con 0x7fd5840b3af0 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd570003520 con 0x7fd5840b3af0 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2109534409 0 0) 0x7fd570003520 con 0x7fd5840b3af0 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 msgr2=0x7fd584070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 0x7fd584070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 >> v1:172.21.15.178:6789/0 conn(0x7fd584072ea0 legacy=0x7fd584071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5840b32f0 con 0x7fd5840b3af0 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fd57c0019f0 con 0x7fd5840b3af0 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 >> v1:172.21.15.113:6789/0 conn(0x7fd5840b3af0 legacy=0x7fd5840b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5700039e0 0x7fd570003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 0x7fd570004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd5840b3010 con 0x7fd570004d40 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd57000b9c0 con 0x7fd5700039e0 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd57000bbc0 con 0x7fd58415a6b0 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd583fff640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd5837fe640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5700039e0 0x7fd570003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd57affd640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 0x7fd570004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd583fff640 1 -- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 msgr2=0x7fd570004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd583fff640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd58415a6b0 0x7fd570004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd583fff640 1 -- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5700039e0 msgr2=0x7fd570003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd583fff640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5700039e0 0x7fd570003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.575+0000 7fd583fff640 1 -- 172.21.15.113:0/1411895829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd570003520 con 0x7fd570004d40 2024-04-24T23:58:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.576+0000 7fd583fff640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00a860 tx=0x7fd570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe5765943e5bd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.576+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00a480 con 0x7fd570004d40 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.576+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57c02f070 con 0x7fd570004d40 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.576+0000 7fd582ffd640 1 -- 172.21.15.113:0/1411895829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c02bcb0 con 0x7fd570004d40 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.577+0000 7fd589c2a640 1 -- 172.21.15.113:0/1411895829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd570004d40 msgr2=0x7fd5700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.577+0000 7fd589c2a640 1 --2- 172.21.15.113:0/1411895829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd570004d40 0x7fd5700051c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00a860 tx=0x7fd570005830 comp rx=0 tx=0).stop 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.577+0000 7fd583fff640 1 -- 172.21.15.113:0/1411895829 reap_dead start 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.577+0000 7fd589c2a640 1 -- 172.21.15.113:0/1411895829 shutdown_connections 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.577+0000 7fd589c2a640 1 -- 172.21.15.113:0/1411895829 >> 172.21.15.113:0/1411895829 conn(0x7fd5840b1a50 msgr2=0x7fd5840b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.577+0000 7fd589c2a640 1 -- 172.21.15.113:0/1411895829 shutdown_connections 2024-04-24T23:58:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.577+0000 7fd589c2a640 1 -- 172.21.15.113:0/1411895829 wait complete. 2024-04-24T23:58:42.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.578+0000 7fd589c2a640 1 Processor -- start 2024-04-24T23:58:42.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.578+0000 7fd589c2a640 1 -- start start 2024-04-24T23:58:42.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd589c2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd584147de0 0x7fd58413bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd589c2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5841481c0 0x7fd584168700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd589c2a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd584168c40 0x7fd5841690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd589c2a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd58415aa90 con 0x7fd584168c40 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd589c2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd584072720 con 0x7fd5841481c0 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd589c2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd58415ac10 con 0x7fd584147de0 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd584168c40 0x7fd5841690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd584168c40 0x7fd5841690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41270/0 (socket says 172.21.15.113:41270) 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 -- 172.21.15.113:0/4245404758 learned_addr learned my addr 172.21.15.113:0/4245404758 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd57affd640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5841481c0 0x7fd584168700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 -- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd584147de0 msgr2=0x7fd58413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd5837fe640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd584147de0 0x7fd58413bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd584147de0 0x7fd58413bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 -- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5841481c0 msgr2=0x7fd584168700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5841481c0 0x7fd584168700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd583fff640 1 -- 172.21.15.113:0/4245404758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58414a130 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.579+0000 7fd57affd640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5841481c0 0x7fd584168700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.580+0000 7fd583fff640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd584168c40 0x7fd5841690d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd57c000f80 tx=0x7fd584072c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.580+0000 7fd580ff9640 1 -- 172.21.15.113:0/4245404758 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c044070 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.580+0000 7fd580ff9640 1 -- 172.21.15.113:0/4245404758 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57c0019e0 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.580+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58414a410 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.580+0000 7fd580ff9640 1 -- 172.21.15.113:0/4245404758 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00a480 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.580+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58414a9d0 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.581+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5840691a0 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.581+0000 7fd580ff9640 1 -- 172.21.15.113:0/4245404758 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd57c047070 con 0x7fd584168c40 2024-04-24T23:58:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.582+0000 7fd580ff9640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd56406be60 0x7fd56406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.582+0000 7fd580ff9640 1 -- 172.21.15.113:0/4245404758 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd57c0aa9c0 con 0x7fd584168c40 2024-04-24T23:58:42.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.582+0000 7fd5837fe640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd56406be60 0x7fd56406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.582+0000 7fd5837fe640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd56406be60 0x7fd56406e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd584142390 tx=0x7fd574008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:42.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.586+0000 7fd580ff9640 1 -- 172.21.15.113:0/4245404758 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd57c0ace20 con 0x7fd584168c40 2024-04-24T23:58:42.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.759+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5840691a0 con 0x7fd584168c40 2024-04-24T23:58:42.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.761+0000 7fd580ff9640 1 -- 172.21.15.113:0/4245404758 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fd57c07f030 con 0x7fd584168c40 2024-04-24T23:58:42.761 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd56406be60 msgr2=0x7fd56406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd56406be60 0x7fd56406e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd584142390 tx=0x7fd574008040 comp rx=0 tx=0).stop 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd584168c40 msgr2=0x7fd5841690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd584168c40 0x7fd5841690d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd57c000f80 tx=0x7fd584072c40 comp rx=0 tx=0).stop 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 shutdown_connections 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd56406be60 0x7fd56406e320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5841481c0 0x7fd584168700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd584168c40 0x7fd5841690d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 --2- 172.21.15.113:0/4245404758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd584147de0 0x7fd58413bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 >> 172.21.15.113:0/4245404758 conn(0x7fd5840b1a50 msgr2=0x7fd58406e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 shutdown_connections 2024-04-24T23:58:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.763+0000 7fd589c2a640 1 -- 172.21.15.113:0/4245404758 wait complete. 2024-04-24T23:58:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:42.774 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:42.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.874+0000 7f07e9e12640 1 Processor -- start 2024-04-24T23:58:42.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.874+0000 7f07e9e12640 1 -- start start 2024-04-24T23:58:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e9e12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e40b3080 0x7f07e40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e9e12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f07e415e6b0 0x7f07e40b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e9e12640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e415f4f0 con 0x7f07e415e2d0 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e9e12640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f07e40cde00 con 0x7f07e415e6b0 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e9e12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f07e415f7d0 con 0x7f07e40b3080 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f07e415e6b0 0x7f07e40b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f07e415e2d0 0x7f07e415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58628/0 (socket says 172.21.15.113:58628) 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e37fe640 1 -- 172.21.15.113:0/234116521 learned_addr learned my addr 172.21.15.113:0/234116521 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e2ffd640 1 --2- 172.21.15.113:0/234116521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e40b3080 0x7f07e40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e3fff640 1 -- 172.21.15.113:0/234116521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e40b3080 msgr2=0x7f07e40b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e3fff640 1 --2- 172.21.15.113:0/234116521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e40b3080 0x7f07e40b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e3fff640 1 -- 172.21.15.113:0/234116521 >> v1:172.21.15.113:6789/0 conn(0x7f07e415e2d0 legacy=0x7f07e415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e3fff640 1 -- 172.21.15.113:0/234116521 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e415fab0 con 0x7f07e415e6b0 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.875+0000 7f07e27fc640 1 -- 172.21.15.113:0/234116521 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1163552165 0 0) 0x7f07e415f4f0 con 0x7f07e415e2d0 2024-04-24T23:58:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e3fff640 1 --2- 172.21.15.113:0/234116521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e40b4870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009d90 tx=0x7f07dc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7f20f0e89daa48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e27fc640 1 -- 172.21.15.113:0/234116521 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc031600 con 0x7f07e415e6b0 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e27fc640 1 -- 172.21.15.113:0/234116521 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc0317a0 con 0x7f07e415e6b0 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e27fc640 1 -- 172.21.15.113:0/234116521 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc031b70 con 0x7f07e415e6b0 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e9e12640 1 -- 172.21.15.113:0/234116521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 msgr2=0x7f07e40b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e9e12640 1 --2- 172.21.15.113:0/234116521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e40b4870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009d90 tx=0x7f07dc02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e9e12640 1 -- 172.21.15.113:0/234116521 shutdown_connections 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e9e12640 1 --2- 172.21.15.113:0/234116521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e40b4870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e9e12640 1 --2- 172.21.15.113:0/234116521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e40b3080 0x7f07e40b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e9e12640 1 -- 172.21.15.113:0/234116521 >> 172.21.15.113:0/234116521 conn(0x7f07e406b0f0 msgr2=0x7f07e406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.876+0000 7f07e9e12640 1 -- 172.21.15.113:0/234116521 shutdown_connections 2024-04-24T23:58:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.877+0000 7f07e9e12640 1 -- 172.21.15.113:0/234116521 wait complete. 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 Processor -- start 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 -- start start 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e40b3080 0x7f07e41475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e415e2d0 0x7f07e41f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e4151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40ce410 con 0x7f07e415e6b0 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40ce290 con 0x7f07e40b3080 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.878+0000 7f07e9e12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f07e40ce590 con 0x7f07e415e2d0 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e4151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e4151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41282/0 (socket says 172.21.15.113:41282) 2024-04-24T23:58:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 -- 172.21.15.113:0/1653937404 learned_addr learned my addr 172.21.15.113:0/1653937404 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e2ffd640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e415e2d0 0x7f07e41f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e37fe640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e40b3080 0x7f07e41475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 -- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e415e2d0 msgr2=0x7f07e41f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e415e2d0 0x7f07e41f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 -- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e40b3080 msgr2=0x7f07e41475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e40b3080 0x7f07e41475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 -- 172.21.15.113:0/1653937404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e4151cd0 con 0x7f07e415e6b0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e3fff640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e4151790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009eb0 tx=0x7f07dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07cbfff640 1 -- 172.21.15.113:0/1653937404 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc03f070 con 0x7f07e415e6b0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07cbfff640 1 -- 172.21.15.113:0/1653937404 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc030b60 con 0x7f07e415e6b0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e4151f50 con 0x7f07e415e6b0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07cbfff640 1 -- 172.21.15.113:0/1653937404 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc009450 con 0x7f07e415e6b0 2024-04-24T23:58:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.879+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e4152510 con 0x7f07e415e6b0 2024-04-24T23:58:42.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.880+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e40691a0 con 0x7f07e415e6b0 2024-04-24T23:58:42.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.881+0000 7f07cbfff640 1 -- 172.21.15.113:0/1653937404 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07dc03b410 con 0x7f07e415e6b0 2024-04-24T23:58:42.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.881+0000 7f07cbfff640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f07d006be30 0x7f07d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:42.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.881+0000 7f07e37fe640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f07d006be30 0x7f07d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:42.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.881+0000 7f07cbfff640 1 -- 172.21.15.113:0/1653937404 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07dc0abf00 con 0x7f07e415e6b0 2024-04-24T23:58:42.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.881+0000 7f07e37fe640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f07d006be30 0x7f07d006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000f60 tx=0x7f07d4007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:42.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:42.885+0000 7f07cbfff640 1 -- 172.21.15.113:0/1653937404 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07e4151cd0 con 0x7f07e415e6b0 2024-04-24T23:58:43.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.057+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07e4072050 con 0x7f07e415e6b0 2024-04-24T23:58:43.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.057+0000 7f07cbfff640 1 -- 172.21.15.113:0/1653937404 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f07dc07a260 con 0x7f07e415e6b0 2024-04-24T23:58:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:43.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f07d006be30 msgr2=0x7f07d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f07d006be30 0x7f07d006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000f60 tx=0x7f07d4007d50 comp rx=0 tx=0).stop 2024-04-24T23:58:43.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 msgr2=0x7f07e4151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e4151790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009eb0 tx=0x7f07dc004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 shutdown_connections 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f07d006be30 0x7f07d006e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07e415e6b0 0x7f07e4151790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e415e2d0 0x7f07e41f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 --2- 172.21.15.113:0/1653937404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e40b3080 0x7f07e41475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 >> 172.21.15.113:0/1653937404 conn(0x7f07e406b0f0 msgr2=0x7f07e406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 shutdown_connections 2024-04-24T23:58:43.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.059+0000 7f07e9e12640 1 -- 172.21.15.113:0/1653937404 wait complete. 2024-04-24T23:58:43.070 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:43.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.170+0000 7fdf18e2a640 1 Processor -- start 2024-04-24T23:58:43.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.170+0000 7fdf18e2a640 1 -- start start 2024-04-24T23:58:43.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf18e2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf1415b670 0x7fdf1415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf18e2a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf1415bff0 0x7fdf140cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf18e2a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf14071170 con 0x7fdf140cc740 2024-04-24T23:58:43.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf18e2a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1415c470 con 0x7fdf1415bff0 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf18e2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdf14071450 con 0x7fdf1415b670 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf11d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf1415bff0 0x7fdf140cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf11d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf1415bff0 0x7fdf140cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41298/0 (socket says 172.21.15.113:41298) 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf11d9b640 1 -- 172.21.15.113:0/2600263729 learned_addr learned my addr 172.21.15.113:0/2600263729 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf1259c640 1 --2- 172.21.15.113:0/2600263729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf1415b670 0x7fdf1415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf11d9b640 1 -- 172.21.15.113:0/2600263729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf1415b670 msgr2=0x7fdf1415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf11d9b640 1 --2- 172.21.15.113:0/2600263729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf1415b670 0x7fdf1415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.171+0000 7fdf11d9b640 1 -- 172.21.15.113:0/2600263729 >> v1:172.21.15.178:6789/0 conn(0x7fdf140cc740 legacy=0x7fdf1406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.172+0000 7fdf11d9b640 1 -- 172.21.15.113:0/2600263729 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf14071730 con 0x7fdf1415bff0 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.172+0000 7fdf11d9b640 1 --2- 172.21.15.113:0/2600263729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf140cc200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800bb00 tx=0x7fdf0802c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56ed9c20b7cf58e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.172+0000 7fdf1159a640 1 -- 172.21.15.113:0/2600263729 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0803a070 con 0x7fdf1415bff0 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.172+0000 7fdf1159a640 1 -- 172.21.15.113:0/2600263729 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf08032480 con 0x7fdf1415bff0 2024-04-24T23:58:43.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.172+0000 7fdf1159a640 1 -- 172.21.15.113:0/2600263729 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08032850 con 0x7fdf1415bff0 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 -- 172.21.15.113:0/2600263729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 msgr2=0x7fdf140cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/2600263729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf140cc200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800bb00 tx=0x7fdf0802c570 comp rx=0 tx=0).stop 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 -- 172.21.15.113:0/2600263729 shutdown_connections 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/2600263729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf140cc200 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/2600263729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf1415b670 0x7fdf1415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 -- 172.21.15.113:0/2600263729 >> 172.21.15.113:0/2600263729 conn(0x7fdf1406b0f0 msgr2=0x7fdf1406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 -- 172.21.15.113:0/2600263729 shutdown_connections 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.173+0000 7fdf18e2a640 1 -- 172.21.15.113:0/2600263729 wait complete. 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.174+0000 7fdf18e2a640 1 Processor -- start 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.174+0000 7fdf18e2a640 1 -- start start 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf18e2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf140cc740 0x7fdf140cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf18e2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1415b670 0x7fdf141504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf18e2a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf141498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf18e2a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1406d730 con 0x7fdf1415bff0 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf18e2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1406d5b0 con 0x7fdf1415b670 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf18e2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdf140d0080 con 0x7fdf140cc740 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf141498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf141498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41308/0 (socket says 172.21.15.113:41308) 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 -- 172.21.15.113:0/3678692717 learned_addr learned my addr 172.21.15.113:0/3678692717 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf11d9b640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1415b670 0x7fdf141504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf1259c640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf140cc740 0x7fdf140cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 -- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf140cc740 msgr2=0x7fdf140cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf140cc740 0x7fdf140cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 -- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1415b670 msgr2=0x7fdf141504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1415b670 0x7fdf141504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.175+0000 7fdf12d9d640 1 -- 172.21.15.113:0/3678692717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf140cd8d0 con 0x7fdf1415bff0 2024-04-24T23:58:43.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.176+0000 7fdf12d9d640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf141498d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c410 tx=0x7fdf0c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.176+0000 7fdf02ffd640 1 -- 172.21.15.113:0/3678692717 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c011790 con 0x7fdf1415bff0 2024-04-24T23:58:43.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.176+0000 7fdf02ffd640 1 -- 172.21.15.113:0/3678692717 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0c017070 con 0x7fdf1415bff0 2024-04-24T23:58:43.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.176+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf140cdbb0 con 0x7fdf1415bff0 2024-04-24T23:58:43.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.176+0000 7fdf02ffd640 1 -- 172.21.15.113:0/3678692717 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c011b70 con 0x7fdf1415bff0 2024-04-24T23:58:43.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.176+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf140ce170 con 0x7fdf1415bff0 2024-04-24T23:58:43.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.177+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf140691a0 con 0x7fdf1415bff0 2024-04-24T23:58:43.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.177+0000 7fdf02ffd640 1 -- 172.21.15.113:0/3678692717 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf0c011d10 con 0x7fdf1415bff0 2024-04-24T23:58:43.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.178+0000 7fdf02ffd640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdef406be30 0x7fdef406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.178+0000 7fdf1259c640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdef406be30 0x7fdef406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.178+0000 7fdf02ffd640 1 -- 172.21.15.113:0/3678692717 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf0c08a9c0 con 0x7fdf1415bff0 2024-04-24T23:58:43.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.178+0000 7fdf1259c640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdef406be30 0x7fdef406e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf04008ef0 tx=0x7fdf04008440 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.182+0000 7fdf02ffd640 1 -- 172.21.15.113:0/3678692717 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf0c08ce20 con 0x7fdf1415bff0 2024-04-24T23:58:43.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.363+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdf140cc2a0 con 0x7fdf1415bff0 2024-04-24T23:58:43.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.363+0000 7fdf02ffd640 1 -- 172.21.15.113:0/3678692717 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fdf0c021020 con 0x7fdf1415bff0 2024-04-24T23:58:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:43.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdef406be30 msgr2=0x7fdef406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdef406be30 0x7fdef406e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf04008ef0 tx=0x7fdf04008440 comp rx=0 tx=0).stop 2024-04-24T23:58:43.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 msgr2=0x7fdf141498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf141498d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c410 tx=0x7fdf0c00c8d0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 shutdown_connections 2024-04-24T23:58:43.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdef406be30 0x7fdef406e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1415b670 0x7fdf141504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf1415bff0 0x7fdf141498d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 --2- 172.21.15.113:0/3678692717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf140cc740 0x7fdf140cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 >> 172.21.15.113:0/3678692717 conn(0x7fdf1406b0f0 msgr2=0x7fdf14158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 shutdown_connections 2024-04-24T23:58:43.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.365+0000 7fdf18e2a640 1 -- 172.21.15.113:0/3678692717 wait complete. 2024-04-24T23:58:43.378 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:43.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.477+0000 7f6226212640 1 Processor -- start 2024-04-24T23:58:43.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.477+0000 7f6226212640 1 -- start start 2024-04-24T23:58:43.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f6226212640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622015e2d0 0x7f622015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f6226212640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62200b4a40 0x7f62200ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f6226212640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f622015f4f0 con 0x7f62200b3080 2024-04-24T23:58:43.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f6226212640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f62200cde00 con 0x7f62200b4a40 2024-04-24T23:58:43.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f6226212640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f622015f7d0 con 0x7f622015e2d0 2024-04-24T23:58:43.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f621ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62200b4a40 0x7f62200ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f621ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62200b4a40 0x7f62200ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41324/0 (socket says 172.21.15.113:41324) 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f621ffff640 1 -- 172.21.15.113:0/243056479 learned_addr learned my addr 172.21.15.113:0/243056479 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.478+0000 7f621f7fe640 1 --2- 172.21.15.113:0/243056479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622015e2d0 0x7f622015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621ffff640 1 -- 172.21.15.113:0/243056479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622015e2d0 msgr2=0x7f622015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621ffff640 1 --2- 172.21.15.113:0/243056479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622015e2d0 0x7f622015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621ffff640 1 -- 172.21.15.113:0/243056479 >> v1:172.21.15.178:6789/0 conn(0x7f62200b3080 legacy=0x7f62200b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621ffff640 1 -- 172.21.15.113:0/243056479 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622015fab0 con 0x7f62200b4a40 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621ffff640 1 --2- 172.21.15.113:0/243056479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62200b4a40 0x7f62200ce290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6218009e50 tx=0x7f621802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474e87254bdc4f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621effd640 1 -- 172.21.15.113:0/243056479 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218036070 con 0x7f62200b4a40 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621effd640 1 -- 172.21.15.113:0/243056479 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218031600 con 0x7f62200b4a40 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.479+0000 7f621effd640 1 -- 172.21.15.113:0/243056479 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62180319d0 con 0x7f62200b4a40 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 -- 172.21.15.113:0/243056479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62200b4a40 msgr2=0x7f62200ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 --2- 172.21.15.113:0/243056479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62200b4a40 0x7f62200ce290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6218009e50 tx=0x7f621802a710 comp rx=0 tx=0).stop 2024-04-24T23:58:43.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 -- 172.21.15.113:0/243056479 shutdown_connections 2024-04-24T23:58:43.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 --2- 172.21.15.113:0/243056479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62200b4a40 0x7f62200ce290 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 --2- 172.21.15.113:0/243056479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f622015e2d0 0x7f622015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 -- 172.21.15.113:0/243056479 >> 172.21.15.113:0/243056479 conn(0x7f622006b0f0 msgr2=0x7f622006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 -- 172.21.15.113:0/243056479 shutdown_connections 2024-04-24T23:58:43.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.480+0000 7f6226212640 1 -- 172.21.15.113:0/243056479 wait complete. 2024-04-24T23:58:43.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.481+0000 7f6226212640 1 Processor -- start 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.481+0000 7f6226212640 1 -- start start 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6226212640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62200b3080 0x7f62201f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6226212640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62200b4a40 0x7f62201f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6226212640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622015e2d0 0x7f622015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6226212640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f622015f2b0 con 0x7f622015e2d0 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6226212640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f622015f130 con 0x7f62200b4a40 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6226212640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f622015e6b0 con 0x7f62200b3080 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622015e2d0 0x7f622015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622015e2d0 0x7f622015c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41340/0 (socket says 172.21.15.113:41340) 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 -- 172.21.15.113:0/1224778884 learned_addr learned my addr 172.21.15.113:0/1224778884 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621f7fe640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62200b3080 0x7f62201f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6217fff640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62200b4a40 0x7f62201f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 -- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62200b3080 msgr2=0x7f62201f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62200b3080 0x7f62201f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 -- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62200b4a40 msgr2=0x7f62201f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62200b4a40 0x7f62201f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621ffff640 1 -- 172.21.15.113:0/1224778884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622015d0f0 con 0x7f622015e2d0 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f621f7fe640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62200b3080 0x7f62201f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.482+0000 7f6217fff640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62200b4a40 0x7f62201f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.483+0000 7f621ffff640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622015e2d0 0x7f622015c530 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6218034040 tx=0x7f6218004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.483+0000 7f621cff9640 1 -- 172.21.15.113:0/1224778884 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218036050 con 0x7f622015e2d0 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.483+0000 7f621cff9640 1 -- 172.21.15.113:0/1224778884 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218042070 con 0x7f622015e2d0 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.483+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62200ad9c0 con 0x7f622015e2d0 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.483+0000 7f621cff9640 1 -- 172.21.15.113:0/1224778884 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218009510 con 0x7f622015e2d0 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.483+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62200adf40 con 0x7f622015e2d0 2024-04-24T23:58:43.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.484+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62200691a0 con 0x7f622015e2d0 2024-04-24T23:58:43.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.484+0000 7f621cff9640 1 -- 172.21.15.113:0/1224778884 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6218007c50 con 0x7f622015e2d0 2024-04-24T23:58:43.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.484+0000 7f621cff9640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f620006be30 0x7f620006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.485+0000 7f621cff9640 1 -- 172.21.15.113:0/1224778884 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62180abc70 con 0x7f622015e2d0 2024-04-24T23:58:43.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.485+0000 7f621f7fe640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f620006be30 0x7f620006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.485+0000 7f621f7fe640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f620006be30 0x7f620006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6210006fd0 tx=0x7f6210008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.489+0000 7f621cff9640 1 -- 172.21.15.113:0/1224778884 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f621807a6b0 con 0x7f622015e2d0 2024-04-24T23:58:43.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.661+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6220072050 con 0x7f622015e2d0 2024-04-24T23:58:43.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.661+0000 7f621cff9640 1 -- 172.21.15.113:0/1224778884 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f6218079fb0 con 0x7f622015e2d0 2024-04-24T23:58:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.663+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f620006be30 msgr2=0x7f620006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.663+0000 7f6226212640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f620006be30 0x7f620006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6210006fd0 tx=0x7f6210008040 comp rx=0 tx=0).stop 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.663+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622015e2d0 msgr2=0x7f622015c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.663+0000 7f6226212640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622015e2d0 0x7f622015c530 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6218034040 tx=0x7f6218004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 shutdown_connections 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f620006be30 0x7f620006e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62200b4a40 0x7f62201f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622015e2d0 0x7f622015c530 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 --2- 172.21.15.113:0/1224778884 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62200b3080 0x7f62201f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 >> 172.21.15.113:0/1224778884 conn(0x7f622006b0f0 msgr2=0x7f62200b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 shutdown_connections 2024-04-24T23:58:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.664+0000 7f6226212640 1 -- 172.21.15.113:0/1224778884 wait complete. 2024-04-24T23:58:43.675 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.771+0000 7f3739e9a640 1 Processor -- start 2024-04-24T23:58:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.771+0000 7f3739e9a640 1 -- start start 2024-04-24T23:58:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3739e9a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f373406e6e0 0x7f37340b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3739e9a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37341453c0 0x7f3734071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3739e9a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37340cfad0 con 0x7f3734144d60 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3739e9a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3734145c40 con 0x7f373406e6e0 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3739e9a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f37340cf230 con 0x7f37341453c0 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3732ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f373406e6e0 0x7f37340b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3732ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f373406e6e0 0x7f37340b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41348/0 (socket says 172.21.15.113:41348) 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f37337fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3734144d60 0x7f373406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58640/0 (socket says 172.21.15.113:58640) 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3732ffd640 1 -- 172.21.15.113:0/578378265 learned_addr learned my addr 172.21.15.113:0/578378265 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3732ffd640 1 -- 172.21.15.113:0/578378265 >> v2:172.21.15.178:3300/0 conn(0x7f37341453c0 msgr2=0x7f3734071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3732ffd640 1 --2- 172.21.15.113:0/578378265 >> v2:172.21.15.178:3300/0 conn(0x7f37341453c0 0x7f3734071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3732ffd640 1 -- 172.21.15.113:0/578378265 >> v1:172.21.15.113:6789/0 conn(0x7f3734144d60 legacy=0x7f373406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f3732ffd640 1 -- 172.21.15.113:0/578378265 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37340cf510 con 0x7f373406e6e0 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.772+0000 7f37327fc640 1 -- 172.21.15.113:0/578378265 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159790183 0 0) 0x7f37340cfad0 con 0x7f3734144d60 2024-04-24T23:58:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3732ffd640 1 --2- 172.21.15.113:0/578378265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f37340b5390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f37340733d0 tx=0x7f372802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25668972c7f9fc42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f37327fc640 1 -- 172.21.15.113:0/578378265 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728037070 con 0x7f373406e6e0 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f37327fc640 1 -- 172.21.15.113:0/578378265 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3728032440 con 0x7f373406e6e0 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f37327fc640 1 -- 172.21.15.113:0/578378265 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728032810 con 0x7f373406e6e0 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 -- 172.21.15.113:0/578378265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 msgr2=0x7f37340b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 --2- 172.21.15.113:0/578378265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f37340b5390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f37340733d0 tx=0x7f372802a150 comp rx=0 tx=0).stop 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 -- 172.21.15.113:0/578378265 shutdown_connections 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 --2- 172.21.15.113:0/578378265 >> v2:172.21.15.178:3300/0 conn(0x7f37341453c0 0x7f3734071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 --2- 172.21.15.113:0/578378265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f37340b5390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 -- 172.21.15.113:0/578378265 >> 172.21.15.113:0/578378265 conn(0x7f37340b1a50 msgr2=0x7f37340b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 -- 172.21.15.113:0/578378265 shutdown_connections 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.773+0000 7f3739e9a640 1 -- 172.21.15.113:0/578378265 wait complete. 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 Processor -- start 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 -- start start 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f373415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734144d60 0x7f373415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341453c0 0x7f373415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3734145140 con 0x7f373406e6e0 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3734070e10 con 0x7f3734144d60 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f3739e9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3734071870 con 0x7f37341453c0 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.775+0000 7f37337fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f373415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f373415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41350/0 (socket says 172.21.15.113:41350) 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 -- 172.21.15.113:0/1545922129 learned_addr learned my addr 172.21.15.113:0/1545922129 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f3732ffd640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734144d60 0x7f373415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f3733fff640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341453c0 0x7f373415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 -- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341453c0 msgr2=0x7f373415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341453c0 0x7f373415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 -- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734144d60 msgr2=0x7f373415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734144d60 0x7f373415ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 -- 172.21.15.113:0/1545922129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373415f8a0 con 0x7f373406e6e0 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f3733fff640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341453c0 0x7f373415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f3732ffd640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734144d60 0x7f373415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f37337fe640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f373415a880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f372400c500 tx=0x7f372400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f371bfff640 1 -- 172.21.15.113:0/1545922129 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724018070 con 0x7f373406e6e0 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f371bfff640 1 -- 172.21.15.113:0/1545922129 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3724007580 con 0x7f373406e6e0 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373415fb80 con 0x7f373406e6e0 2024-04-24T23:58:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f371bfff640 1 -- 172.21.15.113:0/1545922129 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724004a20 con 0x7f373406e6e0 2024-04-24T23:58:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.776+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3734160140 con 0x7f373406e6e0 2024-04-24T23:58:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.777+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3734160380 con 0x7f373406e6e0 2024-04-24T23:58:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.778+0000 7f371bfff640 1 -- 172.21.15.113:0/1545922129 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f372401b070 con 0x7f373406e6e0 2024-04-24T23:58:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.778+0000 7f371bfff640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f372006be30 0x7f372006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.778+0000 7f371bfff640 1 -- 172.21.15.113:0/1545922129 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f372408a140 con 0x7f373406e6e0 2024-04-24T23:58:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.778+0000 7f3732ffd640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f372006be30 0x7f372006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.778+0000 7f3732ffd640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f372006be30 0x7f372006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3728034000 tx=0x7f372802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:43.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.782+0000 7f371bfff640 1 -- 172.21.15.113:0/1545922129 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3734160380 con 0x7f373406e6e0 2024-04-24T23:58:43.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.959+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37340b6030 con 0x7f373406e6e0 2024-04-24T23:58:43.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.960+0000 7f371bfff640 1 -- 172.21.15.113:0/1545922129 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f3724058b80 con 0x7f373406e6e0 2024-04-24T23:58:43.960 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f372006be30 msgr2=0x7f372006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f372006be30 0x7f372006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3728034000 tx=0x7f372802e000 comp rx=0 tx=0).stop 2024-04-24T23:58:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 msgr2=0x7f373415a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f373415a880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f372400c500 tx=0x7f372400c9c0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 shutdown_connections 2024-04-24T23:58:43.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f372006be30 0x7f372006e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734144d60 0x7f373415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341453c0 0x7f373415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 --2- 172.21.15.113:0/1545922129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f373406e6e0 0x7f373415a880 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 >> 172.21.15.113:0/1545922129 conn(0x7f37340b1a50 msgr2=0x7f37340cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 shutdown_connections 2024-04-24T23:58:43.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:43.962+0000 7f3739e9a640 1 -- 172.21.15.113:0/1545922129 wait complete. 2024-04-24T23:58:43.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:43.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.073+0000 7f168acee640 1 Processor -- start 2024-04-24T23:58:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.073+0000 7f168acee640 1 -- start start 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.073+0000 7f168acee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16840b3080 0x7f16840b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.073+0000 7f168acee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.073+0000 7f168acee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168415f4f0 con 0x7f168415e2d0 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f168acee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f16840cde00 con 0x7f16840b3080 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f168acee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f168415f7d0 con 0x7f168415e6b0 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f1683fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16840b3080 0x7f16840b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f1688a8a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f168415e2d0 0x7f168415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58648/0 (socket says 172.21.15.113:58648) 2024-04-24T23:58:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f1688a8a640 1 -- 172.21.15.113:0/2468907294 learned_addr learned my addr 172.21.15.113:0/2468907294 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f168928b640 1 --2- 172.21.15.113:0/2468907294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f1683fff640 1 -- 172.21.15.113:0/2468907294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 msgr2=0x7f16840b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f1683fff640 1 --2- 172.21.15.113:0/2468907294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f1683fff640 1 -- 172.21.15.113:0/2468907294 >> v1:172.21.15.113:6789/0 conn(0x7f168415e2d0 legacy=0x7f168415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f1683fff640 1 -- 172.21.15.113:0/2468907294 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168415fab0 con 0x7f16840b3080 2024-04-24T23:58:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.074+0000 7f16837fe640 1 -- 172.21.15.113:0/2468907294 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1536133944 0 0) 0x7f168415f4f0 con 0x7f168415e2d0 2024-04-24T23:58:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f1683fff640 1 --2- 172.21.15.113:0/2468907294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16840b3080 0x7f16840b4330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f16700097a0 tx=0x7f167002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5dd7ff64a859d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f16837fe640 1 -- 172.21.15.113:0/2468907294 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670037070 con 0x7f16840b3080 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f16837fe640 1 -- 172.21.15.113:0/2468907294 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1670032440 con 0x7f16840b3080 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f16837fe640 1 -- 172.21.15.113:0/2468907294 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670032810 con 0x7f16840b3080 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 -- 172.21.15.113:0/2468907294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16840b3080 msgr2=0x7f16840b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 --2- 172.21.15.113:0/2468907294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16840b3080 0x7f16840b4330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f16700097a0 tx=0x7f167002a180 comp rx=0 tx=0).stop 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 -- 172.21.15.113:0/2468907294 shutdown_connections 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 --2- 172.21.15.113:0/2468907294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 --2- 172.21.15.113:0/2468907294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16840b3080 0x7f16840b4330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 -- 172.21.15.113:0/2468907294 >> 172.21.15.113:0/2468907294 conn(0x7f168406b0f0 msgr2=0x7f168406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 -- 172.21.15.113:0/2468907294 shutdown_connections 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.075+0000 7f168acee640 1 -- 172.21.15.113:0/2468907294 wait complete. 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.076+0000 7f168acee640 1 Processor -- start 2024-04-24T23:58:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f168acee640 1 -- start start 2024-04-24T23:58:44.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f168acee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16840b3080 0x7f168415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f168acee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168415e2d0 0x7f16840aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f168acee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f168acee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16840ce410 con 0x7f168415e2d0 2024-04-24T23:58:44.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f168acee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16840ce290 con 0x7f16840b3080 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f168acee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f16840ce590 con 0x7f168415e6b0 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.077+0000 7f1683fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168415e2d0 0x7f16840aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168415e2d0 0x7f16840aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41374/0 (socket says 172.21.15.113:41374) 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 -- 172.21.15.113:0/39637817 learned_addr learned my addr 172.21.15.113:0/39637817 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1688a8a640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16840b3080 0x7f168415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f168928b640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 -- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 msgr2=0x7f16840ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 -- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16840b3080 msgr2=0x7f168415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16840b3080 0x7f168415b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 -- 172.21.15.113:0/39637817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16840ab070 con 0x7f168415e2d0 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1688a8a640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16840b3080 0x7f168415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:44.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f168928b640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f1683fff640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168415e2d0 0x7f16840aa690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1670035040 tx=0x7f16700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f16817fa640 1 -- 172.21.15.113:0/39637817 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670037040 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f16817fa640 1 -- 172.21.15.113:0/39637817 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1670030770 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16840ab350 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f16817fa640 1 -- 172.21.15.113:0/39637817 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670030b80 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.078+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16840ab910 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.079+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16840691a0 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.080+0000 7f16817fa640 1 -- 172.21.15.113:0/39637817 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1670044020 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.080+0000 7f16817fa640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f166806be30 0x7f166806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.080+0000 7f1688a8a640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f166806be30 0x7f166806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.080+0000 7f16817fa640 1 -- 172.21.15.113:0/39637817 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f16700ab3f0 con 0x7f168415e2d0 2024-04-24T23:58:44.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.080+0000 7f1688a8a640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f166806be30 0x7f166806e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1678000f60 tx=0x7f1678007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.084+0000 7f16817fa640 1 -- 172.21.15.113:0/39637817 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1670032e20 con 0x7f168415e2d0 2024-04-24T23:58:44.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.260+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f168406c100 con 0x7f168415e2d0 2024-04-24T23:58:44.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.261+0000 7f16817fa640 1 -- 172.21.15.113:0/39637817 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1670079da0 con 0x7f168415e2d0 2024-04-24T23:58:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:44.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f166806be30 msgr2=0x7f166806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f166806be30 0x7f166806e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1678000f60 tx=0x7f1678007c10 comp rx=0 tx=0).stop 2024-04-24T23:58:44.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168415e2d0 msgr2=0x7f16840aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168415e2d0 0x7f16840aa690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1670035040 tx=0x7f16700046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 shutdown_connections 2024-04-24T23:58:44.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f166806be30 0x7f166806e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f168415e6b0 0x7f16840ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f168415e2d0 0x7f16840aa690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 --2- 172.21.15.113:0/39637817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16840b3080 0x7f168415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 >> 172.21.15.113:0/39637817 conn(0x7f168406b0f0 msgr2=0x7f168406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:44.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 shutdown_connections 2024-04-24T23:58:44.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.263+0000 7f168acee640 1 -- 172.21.15.113:0/39637817 wait complete. 2024-04-24T23:58:44.274 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:44.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.373+0000 7fa475d7d640 1 Processor -- start 2024-04-24T23:58:44.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.373+0000 7fa475d7d640 1 -- start start 2024-04-24T23:58:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa475d7d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa470074c00 0x7fa4700cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa475d7d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa47015cb40 con 0x7fa47006f180 2024-04-24T23:58:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa475d7d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa470160900 con 0x7fa47015c050 2024-04-24T23:58:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa475d7d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa47006d870 con 0x7fa470074c00 2024-04-24T23:58:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa46effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa470074c00 0x7fa4700cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa46ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa47006f180 0x7fa47006e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58658/0 (socket says 172.21.15.113:58658) 2024-04-24T23:58:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa46effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa470074c00 0x7fa4700cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41382/0 (socket says 172.21.15.113:41382) 2024-04-24T23:58:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa46ffff640 1 -- 172.21.15.113:0/695035515 learned_addr learned my addr 172.21.15.113:0/695035515 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa46effd640 1 -- 172.21.15.113:0/695035515 >> v1:172.21.15.178:6789/0 conn(0x7fa47015c050 legacy=0x7fa47015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa46effd640 1 -- 172.21.15.113:0/695035515 >> v1:172.21.15.113:6789/0 conn(0x7fa47006f180 legacy=0x7fa47006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.374+0000 7fa46effd640 1 -- 172.21.15.113:0/695035515 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47006db50 con 0x7fa470074c00 2024-04-24T23:58:44.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa46effd640 1 --2- 172.21.15.113:0/695035515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470074c00 0x7fa4700cf3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4640097a0 tx=0x7fa46402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35a112f8ed96ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa46e7fc640 1 -- 172.21.15.113:0/695035515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464037070 con 0x7fa470074c00 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa46e7fc640 1 -- 172.21.15.113:0/695035515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa464032440 con 0x7fa470074c00 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa46e7fc640 1 -- 172.21.15.113:0/695035515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464032810 con 0x7fa470074c00 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa475d7d640 1 -- 172.21.15.113:0/695035515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470074c00 msgr2=0x7fa4700cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa475d7d640 1 --2- 172.21.15.113:0/695035515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470074c00 0x7fa4700cf3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4640097a0 tx=0x7fa46402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa475d7d640 1 -- 172.21.15.113:0/695035515 shutdown_connections 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa475d7d640 1 --2- 172.21.15.113:0/695035515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa470074c00 0x7fa4700cf3f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa475d7d640 1 -- 172.21.15.113:0/695035515 >> 172.21.15.113:0/695035515 conn(0x7fa47006fb60 msgr2=0x7fa47006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa475d7d640 1 -- 172.21.15.113:0/695035515 shutdown_connections 2024-04-24T23:58:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.375+0000 7fa475d7d640 1 -- 172.21.15.113:0/695035515 wait complete. 2024-04-24T23:58:44.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.376+0000 7fa475d7d640 1 Processor -- start 2024-04-24T23:58:44.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa475d7d640 1 -- start start 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa475d7d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa47006f180 0x7fa4701515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa475d7d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470074c00 0x7fa4701500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa475d7d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa47015c050 0x7fa4701f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa475d7d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700c5880 con 0x7fa47006f180 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa475d7d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700c5700 con 0x7fa470074c00 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa475d7d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa4700cf930 con 0x7fa47015c050 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.377+0000 7fa46f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa47006f180 0x7fa4701515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa47006f180 0x7fa4701515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41386/0 (socket says 172.21.15.113:41386) 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 -- 172.21.15.113:0/1535308896 learned_addr learned my addr 172.21.15.113:0/1535308896 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 -- 172.21.15.113:0/1535308896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa47015c050 msgr2=0x7fa4701f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa47015c050 0x7fa4701f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46effd640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470074c00 0x7fa4701500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 -- 172.21.15.113:0/1535308896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470074c00 msgr2=0x7fa4701500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470074c00 0x7fa4701500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 -- 172.21.15.113:0/1535308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa470151b30 con 0x7fa47006f180 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa46f7fe640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa47006f180 0x7fa4701515f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa46000b3f0 tx=0x7fa46000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa457fff640 1 -- 172.21.15.113:0/1535308896 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460015020 con 0x7fa47006f180 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa457fff640 1 -- 172.21.15.113:0/1535308896 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa460011830 con 0x7fa47006f180 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa457fff640 1 -- 172.21.15.113:0/1535308896 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460011c20 con 0x7fa47006f180 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4701f2a10 con 0x7fa47006f180 2024-04-24T23:58:44.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.378+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa470150df0 con 0x7fa47006f180 2024-04-24T23:58:44.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.379+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4700691a0 con 0x7fa47006f180 2024-04-24T23:58:44.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.379+0000 7fa457fff640 1 -- 172.21.15.113:0/1535308896 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa460018020 con 0x7fa47006f180 2024-04-24T23:58:44.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.380+0000 7fa457fff640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa45c06be30 0x7fa45c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.380+0000 7fa46effd640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa45c06be30 0x7fa45c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.380+0000 7fa457fff640 1 -- 172.21.15.113:0/1535308896 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa46008aaa0 con 0x7fa47006f180 2024-04-24T23:58:44.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.380+0000 7fa46effd640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa45c06be30 0x7fa45c06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa464034000 tx=0x7fa46402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.384+0000 7fa457fff640 1 -- 172.21.15.113:0/1535308896 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa46001f030 con 0x7fa47006f180 2024-04-24T23:58:44.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.557+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4700b1b90 con 0x7fa47006f180 2024-04-24T23:58:44.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.558+0000 7fa457fff640 1 -- 172.21.15.113:0/1535308896 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa4600172c0 con 0x7fa47006f180 2024-04-24T23:58:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.559+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa45c06be30 msgr2=0x7fa45c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.559+0000 7fa475d7d640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa45c06be30 0x7fa45c06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa464034000 tx=0x7fa46402e000 comp rx=0 tx=0).stop 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.559+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa47006f180 msgr2=0x7fa4701515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.559+0000 7fa475d7d640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa47006f180 0x7fa4701515f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa46000b3f0 tx=0x7fa46000b8b0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 shutdown_connections 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa45c06be30 0x7fa45c06e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa470074c00 0x7fa4701500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa47015c050 0x7fa4701f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 --2- 172.21.15.113:0/1535308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa47006f180 0x7fa4701515f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 >> 172.21.15.113:0/1535308896 conn(0x7fa47006fb60 msgr2=0x7fa47015cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:44.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 shutdown_connections 2024-04-24T23:58:44.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.560+0000 7fa475d7d640 1 -- 172.21.15.113:0/1535308896 wait complete. 2024-04-24T23:58:44.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.670+0000 7f881359e640 1 Processor -- start 2024-04-24T23:58:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.670+0000 7f881359e640 1 -- start start 2024-04-24T23:58:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f881359e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f88140cd490 0x7f8814168de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f881359e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88140d0ae0 con 0x7f881415e260 2024-04-24T23:58:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f881359e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88140d0dc0 con 0x7f88140b3180 2024-04-24T23:58:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f881359e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f88140d10a0 con 0x7f88140cd490 2024-04-24T23:58:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f881259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f881415e260 0x7f88140d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58664/0 (socket says 172.21.15.113:58664) 2024-04-24T23:58:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f881259c640 1 -- 172.21.15.113:0/3603924448 learned_addr learned my addr 172.21.15.113:0/3603924448 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f8812d9d640 1 --2- 172.21.15.113:0/3603924448 >> v2:172.21.15.178:3300/0 conn(0x7f88140cd490 0x7f8814168de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f8811d9b640 1 -- 172.21.15.113:0/3603924448 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2348674186 0 0) 0x7f88140d0ae0 con 0x7f881415e260 2024-04-24T23:58:44.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f8811d9b640 1 -- 172.21.15.113:0/3603924448 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8800003520 con 0x7f881415e260 2024-04-24T23:58:44.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f8812d9d640 1 -- 172.21.15.113:0/3603924448 >> v1:172.21.15.178:6789/0 conn(0x7f88140b3180 legacy=0x7f88140d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f8812d9d640 1 -- 172.21.15.113:0/3603924448 >> v1:172.21.15.113:6789/0 conn(0x7f881415e260 legacy=0x7f88140d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.671+0000 7f8812d9d640 1 -- 172.21.15.113:0/3603924448 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88140d1380 con 0x7f88140cd490 2024-04-24T23:58:44.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.672+0000 7f8812d9d640 1 --2- 172.21.15.113:0/3603924448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88140cd490 0x7f8814168de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f880c009d90 tx=0x7f880c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602aa4671e75e5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.672+0000 7f8811d9b640 1 -- 172.21.15.113:0/3603924448 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c007740 con 0x7f88140cd490 2024-04-24T23:58:44.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.672+0000 7f8811d9b640 1 -- 172.21.15.113:0/3603924448 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c0078e0 con 0x7f88140cd490 2024-04-24T23:58:44.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.672+0000 7f8811d9b640 1 -- 172.21.15.113:0/3603924448 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c007c90 con 0x7f88140cd490 2024-04-24T23:58:44.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.673+0000 7f881359e640 1 -- 172.21.15.113:0/3603924448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88140cd490 msgr2=0x7f8814168de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.673+0000 7f881359e640 1 --2- 172.21.15.113:0/3603924448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88140cd490 0x7f8814168de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f880c009d90 tx=0x7f880c02a6f0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.673+0000 7f881359e640 1 -- 172.21.15.113:0/3603924448 shutdown_connections 2024-04-24T23:58:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.673+0000 7f881359e640 1 --2- 172.21.15.113:0/3603924448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88140cd490 0x7f8814168de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.673+0000 7f881359e640 1 -- 172.21.15.113:0/3603924448 >> 172.21.15.113:0/3603924448 conn(0x7f881409aa20 msgr2=0x7f881409ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.673+0000 7f881359e640 1 -- 172.21.15.113:0/3603924448 shutdown_connections 2024-04-24T23:58:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.673+0000 7f881359e640 1 -- 172.21.15.113:0/3603924448 wait complete. 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.674+0000 7f881359e640 1 Processor -- start 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.674+0000 7f881359e640 1 -- start start 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f881359e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88140b3180 0x7f88140c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f881359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88140cd490 0x7f88140b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f881359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f881415e260 0x7f88140b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f881359e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88140d0780 con 0x7f88140cd490 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f881359e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f881415e640 con 0x7f881415e260 2024-04-24T23:58:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f881359e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f88140d24a0 con 0x7f88140b3180 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88140cd490 0x7f88140b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88140cd490 0x7f88140b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41402/0 (socket says 172.21.15.113:41402) 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 -- 172.21.15.113:0/2451447189 learned_addr learned my addr 172.21.15.113:0/2451447189 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f8812d9d640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f881415e260 0x7f88140b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f881259c640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88140b3180 0x7f88140c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 -- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88140b3180 msgr2=0x7f88140c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88140b3180 0x7f88140c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 -- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f881415e260 msgr2=0x7f88140b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f881415e260 0x7f88140b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 -- 172.21.15.113:0/2451447189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88140c7960 con 0x7f88140cd490 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880bfff640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88140cd490 0x7f88140b0ff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c400 tx=0x7f87fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.675+0000 7f880affd640 1 -- 172.21.15.113:0/2451447189 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0052d0 con 0x7f88140cd490 2024-04-24T23:58:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.676+0000 7f880affd640 1 -- 172.21.15.113:0/2451447189 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87fc005490 con 0x7f88140cd490 2024-04-24T23:58:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.676+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88140c7c40 con 0x7f88140cd490 2024-04-24T23:58:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.676+0000 7f880affd640 1 -- 172.21.15.113:0/2451447189 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc007260 con 0x7f88140cd490 2024-04-24T23:58:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.676+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88140c8200 con 0x7f88140cd490 2024-04-24T23:58:44.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.677+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88140691a0 con 0x7f88140cd490 2024-04-24T23:58:44.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.677+0000 7f880affd640 1 -- 172.21.15.113:0/2451447189 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87fc007400 con 0x7f88140cd490 2024-04-24T23:58:44.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.677+0000 7f880affd640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87f406be30 0x7f87f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.677+0000 7f880affd640 1 -- 172.21.15.113:0/2451447189 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f87fc08a2f0 con 0x7f88140cd490 2024-04-24T23:58:44.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.678+0000 7f881259c640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87f406be30 0x7f87f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.678+0000 7f881259c640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87f406be30 0x7f87f406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f88040011e0 tx=0x7f8804007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.682+0000 7f880affd640 1 -- 172.21.15.113:0/2451447189 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f87fc058650 con 0x7f88140cd490 2024-04-24T23:58:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.855+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88140b2f60 con 0x7f88140cd490 2024-04-24T23:58:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.856+0000 7f880affd640 1 -- 172.21.15.113:0/2451447189 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f87fc05c950 con 0x7f88140cd490 2024-04-24T23:58:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87f406be30 msgr2=0x7f87f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87f406be30 0x7f87f406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f88040011e0 tx=0x7f8804007d30 comp rx=0 tx=0).stop 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88140cd490 msgr2=0x7f88140b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88140cd490 0x7f88140b0ff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c400 tx=0x7f87fc00c8c0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 shutdown_connections 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87f406be30 0x7f87f406e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f881415e260 0x7f88140b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88140cd490 0x7f88140b0ff0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 --2- 172.21.15.113:0/2451447189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88140b3180 0x7f88140c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 >> 172.21.15.113:0/2451447189 conn(0x7f881409aa20 msgr2=0x7f88140b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:44.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 shutdown_connections 2024-04-24T23:58:44.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.858+0000 7f881359e640 1 -- 172.21.15.113:0/2451447189 wait complete. 2024-04-24T23:58:44.869 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.965+0000 7f4023cda640 1 Processor -- start 2024-04-24T23:58:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.965+0000 7f4023cda640 1 -- start start 2024-04-24T23:58:44.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4023cda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 0x7f401c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4023cda640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401c144cf0 con 0x7f401c144910 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4023cda640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401c0b2f30 con 0x7f401c0b3fd0 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4023cda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f401c0b3210 con 0x7f401c06e6e0 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4021a76640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f401c144910 0x7f401c0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58672/0 (socket says 172.21.15.113:58672) 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4021a76640 1 -- 172.21.15.113:0/2499068366 learned_addr learned my addr 172.21.15.113:0/2499068366 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4022277640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 0x7f401c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009496542 0 0) 0x7f401c144cf0 con 0x7f401c144910 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4008003520 con 0x7f401c144910 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844280063 0 0) 0x7f401c0b2f30 con 0x7f401c0b3fd0 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f401c144cf0 con 0x7f401c0b3fd0 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 505418046 0 0) 0x7f4008003520 con 0x7f401c144910 2024-04-24T23:58:44.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 msgr2=0x7f401c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 0x7f401c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 >> v1:172.21.15.178:6789/0 conn(0x7f401c0b3fd0 legacy=0x7f401c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c0b34f0 con 0x7f401c144910 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f400c001670 con 0x7f401c144910 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4022277640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 0x7f401c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 >> v1:172.21.15.113:6789/0 conn(0x7f401c144910 legacy=0x7f401c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4008003b70 0x7f4008003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f4008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 0x7f40080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0b3210 con 0x7f4008004800 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f400800ba20 con 0x7f4008003b70 2024-04-24T23:58:44.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f400800bc20 con 0x7f401c06e6e0 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.966+0000 7f4021275640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f4008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4021a76640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4008003b70 0x7f4008003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4022277640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 0x7f40080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4021275640 1 -- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 msgr2=0x7f40080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4021275640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c06e6e0 0x7f40080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4021275640 1 -- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4008003b70 msgr2=0x7f4008003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4021275640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4008003b70 0x7f4008003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4021275640 1 -- 172.21.15.113:0/2499068366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4008003520 con 0x7f4008004800 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.967+0000 7f4021275640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f4008004c80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f40180018a0 tx=0x7f4008005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205d711aabe3e5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40180014f0 con 0x7f4008004800 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40180052d0 con 0x7f4008004800 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4020a74640 1 -- 172.21.15.113:0/2499068366 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401802ac40 con 0x7f4008004800 2024-04-24T23:58:44.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4023cda640 1 -- 172.21.15.113:0/2499068366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 msgr2=0x7f4008004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4023cda640 1 --2- 172.21.15.113:0/2499068366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f4008004c80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f40180018a0 tx=0x7f4008005830 comp rx=0 tx=0).stop 2024-04-24T23:58:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4022277640 1 -- 172.21.15.113:0/2499068366 reap_dead start 2024-04-24T23:58:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4023cda640 1 -- 172.21.15.113:0/2499068366 shutdown_connections 2024-04-24T23:58:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4023cda640 1 -- 172.21.15.113:0/2499068366 >> 172.21.15.113:0/2499068366 conn(0x7f401c0b1a50 msgr2=0x7f401c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4023cda640 1 -- 172.21.15.113:0/2499068366 shutdown_connections 2024-04-24T23:58:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.968+0000 7f4023cda640 1 -- 172.21.15.113:0/2499068366 wait complete. 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.969+0000 7f4023cda640 1 Processor -- start 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.969+0000 7f4023cda640 1 -- start start 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4023cda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f401c13f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4023cda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c140680 0x7f401c15db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4023cda640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401c0c6050 0x7f401c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4023cda640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0b4da0 con 0x7f4008004800 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4023cda640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0b4440 con 0x7f401c0c6050 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4023cda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f401c0b4f20 con 0x7f401c140680 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f401c13f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f401c13f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41430/0 (socket says 172.21.15.113:41430) 2024-04-24T23:58:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 -- 172.21.15.113:0/2840674187 learned_addr learned my addr 172.21.15.113:0/2840674187 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021275640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c140680 0x7f401c15db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4022277640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401c0c6050 0x7f401c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 -- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c140680 msgr2=0x7f401c15db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c140680 0x7f401c15db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 -- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401c0c6050 msgr2=0x7f401c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401c0c6050 0x7f401c0ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021a76640 1 -- 172.21.15.113:0/2840674187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c13f5e0 con 0x7f4008004800 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4021275640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c140680 0x7f401c15db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.970+0000 7f4022277640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401c0c6050 0x7f401c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.971+0000 7f4021a76640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f401c13f000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f400c007370 tx=0x7f400c007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.971+0000 7f40127fc640 1 -- 172.21.15.113:0/2840674187 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c01a070 con 0x7f4008004800 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.971+0000 7f40127fc640 1 -- 172.21.15.113:0/2840674187 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c00f8e0 con 0x7f4008004800 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.971+0000 7f40127fc640 1 -- 172.21.15.113:0/2840674187 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c00fcd0 con 0x7f4008004800 2024-04-24T23:58:44.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.971+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c13f8c0 con 0x7f4008004800 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.971+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c13fe80 con 0x7f4008004800 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.972+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c0691a0 con 0x7f4008004800 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.972+0000 7f40127fc640 1 -- 172.21.15.113:0/2840674187 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f400c01d070 con 0x7f4008004800 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.973+0000 7f40127fc640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.973+0000 7f4021275640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.973+0000 7f40127fc640 1 -- 172.21.15.113:0/2840674187 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f400c08a7d0 con 0x7f4008004800 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.973+0000 7f4021275640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4018032000 tx=0x7f4018009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:44.977+0000 7f40127fc640 1 -- 172.21.15.113:0/2840674187 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f400c059180 con 0x7f4008004800 2024-04-24T23:58:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.154+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f401c0731a0 con 0x7f4008004800 2024-04-24T23:58:45.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.155+0000 7f40127fc640 1 -- 172.21.15.113:0/2840674187 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f400c058a80 con 0x7f4008004800 2024-04-24T23:58:45.155 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:45.155 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 msgr2=0x7f400006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4018032000 tx=0x7f4018009030 comp rx=0 tx=0).stop 2024-04-24T23:58:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 msgr2=0x7f401c13f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f401c13f000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f400c007370 tx=0x7f400c007830 comp rx=0 tx=0).stop 2024-04-24T23:58:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 shutdown_connections 2024-04-24T23:58:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401c0c6050 0x7f401c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401c140680 0x7f401c15db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 --2- 172.21.15.113:0/2840674187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4008004800 0x7f401c13f000 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 >> 172.21.15.113:0/2840674187 conn(0x7f401c0b1a50 msgr2=0x7f401c072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:45.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 shutdown_connections 2024-04-24T23:58:45.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.157+0000 7f4023cda640 1 -- 172.21.15.113:0/2840674187 wait complete. 2024-04-24T23:58:45.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:45.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:45.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.268+0000 7f8d10c72640 1 Processor -- start 2024-04-24T23:58:45.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.268+0000 7f8d10c72640 1 -- start start 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d10c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c15e2d0 0x7f8d0c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d10c72640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d0c0b4a40 0x7f8d0c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d10c72640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d0c15f4f0 con 0x7f8d0c0b3080 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d10c72640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0c0cde00 con 0x7f8d0c0b4a40 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d10c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c15f7d0 con 0x7f8d0c15e2d0 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d0ad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d0c0b4a40 0x7f8d0c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d0ad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d0c0b4a40 0x7f8d0c0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41446/0 (socket says 172.21.15.113:41446) 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d0ad9d640 1 -- 172.21.15.113:0/460173719 learned_addr learned my addr 172.21.15.113:0/460173719 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.269+0000 7f8d0a59c640 1 --2- 172.21.15.113:0/460173719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c15e2d0 0x7f8d0c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0ad9d640 1 -- 172.21.15.113:0/460173719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c15e2d0 msgr2=0x7f8d0c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0ad9d640 1 --2- 172.21.15.113:0/460173719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c15e2d0 0x7f8d0c15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0ad9d640 1 -- 172.21.15.113:0/460173719 >> v1:172.21.15.178:6789/0 conn(0x7f8d0c0b3080 legacy=0x7f8d0c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0ad9d640 1 -- 172.21.15.113:0/460173719 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c15fab0 con 0x7f8d0c0b4a40 2024-04-24T23:58:45.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0ad9d640 1 --2- 172.21.15.113:0/460173719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b4a40 0x7f8d0c0ce290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009870 tx=0x7f8d0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edfa77f3f5ec5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0959a640 1 -- 172.21.15.113:0/460173719 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04037070 con 0x7f8d0c0b4a40 2024-04-24T23:58:45.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0959a640 1 -- 172.21.15.113:0/460173719 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d04032440 con 0x7f8d0c0b4a40 2024-04-24T23:58:45.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.270+0000 7f8d0959a640 1 -- 172.21.15.113:0/460173719 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04032810 con 0x7f8d0c0b4a40 2024-04-24T23:58:45.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 -- 172.21.15.113:0/460173719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b4a40 msgr2=0x7f8d0c0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 --2- 172.21.15.113:0/460173719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b4a40 0x7f8d0c0ce290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009870 tx=0x7f8d0402a270 comp rx=0 tx=0).stop 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 -- 172.21.15.113:0/460173719 shutdown_connections 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 --2- 172.21.15.113:0/460173719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b4a40 0x7f8d0c0ce290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 --2- 172.21.15.113:0/460173719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c15e2d0 0x7f8d0c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 -- 172.21.15.113:0/460173719 >> 172.21.15.113:0/460173719 conn(0x7f8d0c06b0f0 msgr2=0x7f8d0c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 -- 172.21.15.113:0/460173719 shutdown_connections 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.271+0000 7f8d10c72640 1 -- 172.21.15.113:0/460173719 wait complete. 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.272+0000 7f8d10c72640 1 Processor -- start 2024-04-24T23:58:45.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.272+0000 7f8d10c72640 1 -- start start 2024-04-24T23:58:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d10c72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b3080 0x7f8d0c14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d10c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c0b4a40 0x7f8d0c15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d10c72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d0c15e2d0 0x7f8d0c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d10c72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c15f2b0 con 0x7f8d0c0b3080 2024-04-24T23:58:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d10c72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c15f130 con 0x7f8d0c15e2d0 2024-04-24T23:58:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d10c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c15e6b0 con 0x7f8d0c0b4a40 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b3080 0x7f8d0c14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b3080 0x7f8d0c14ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41458/0 (socket says 172.21.15.113:41458) 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 -- 172.21.15.113:0/3649586280 learned_addr learned my addr 172.21.15.113:0/3649586280 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d09d9b640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c0b4a40 0x7f8d0c15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0ad9d640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d0c15e2d0 0x7f8d0c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 -- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c0b4a40 msgr2=0x7f8d0c15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c0b4a40 0x7f8d0c15ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 -- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d0c15e2d0 msgr2=0x7f8d0c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d0c15e2d0 0x7f8d0c15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.273+0000 7f8d0a59c640 1 -- 172.21.15.113:0/3649586280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c15ba10 con 0x7f8d0c0b3080 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.274+0000 7f8d0a59c640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b3080 0x7f8d0c14ee40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc007e40 tx=0x7f8cfc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.274+0000 7f8cfaffd640 1 -- 172.21.15.113:0/3649586280 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc014620 con 0x7f8d0c0b3080 2024-04-24T23:58:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.274+0000 7f8cfaffd640 1 -- 172.21.15.113:0/3649586280 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc019070 con 0x7f8d0c0b3080 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.274+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c15bcf0 con 0x7f8d0c0b3080 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.274+0000 7f8cfaffd640 1 -- 172.21.15.113:0/3649586280 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc014a00 con 0x7f8d0c0b3080 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.274+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c15c2b0 con 0x7f8d0c0b3080 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.275+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c0691a0 con 0x7f8d0c0b3080 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.275+0000 7f8cfaffd640 1 -- 172.21.15.113:0/3649586280 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cfc014ba0 con 0x7f8d0c0b3080 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.275+0000 7f8cfaffd640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cec06be30 0x7f8cec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.276+0000 7f8cfaffd640 1 -- 172.21.15.113:0/3649586280 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8cfc08b6c0 con 0x7f8d0c0b3080 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.276+0000 7f8d09d9b640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cec06be30 0x7f8cec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.276+0000 7f8d09d9b640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cec06be30 0x7f8cec06e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1e8960 tx=0x7f8d00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.280+0000 7f8cfaffd640 1 -- 172.21.15.113:0/3649586280 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8cfc05a070 con 0x7f8d0c0b3080 2024-04-24T23:58:45.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.454+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d0c072050 con 0x7f8d0c0b3080 2024-04-24T23:58:45.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.454+0000 7f8cfaffd640 1 -- 172.21.15.113:0/3649586280 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8cfc059970 con 0x7f8d0c0b3080 2024-04-24T23:58:45.455 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cec06be30 msgr2=0x7f8cec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cec06be30 0x7f8cec06e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1e8960 tx=0x7f8d00009040 comp rx=0 tx=0).stop 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b3080 msgr2=0x7f8d0c14ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b3080 0x7f8d0c14ee40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc007e40 tx=0x7f8cfc00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 shutdown_connections 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cec06be30 0x7f8cec06e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d0c15e2d0 0x7f8d0c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d0c0b4a40 0x7f8d0c15ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 --2- 172.21.15.113:0/3649586280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d0c0b3080 0x7f8d0c14ee40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 >> 172.21.15.113:0/3649586280 conn(0x7f8d0c06b0f0 msgr2=0x7f8d0c0afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 shutdown_connections 2024-04-24T23:58:45.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.456+0000 7f8d10c72640 1 -- 172.21.15.113:0/3649586280 wait complete. 2024-04-24T23:58:45.467 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.567+0000 7f4a12c4d640 1 Processor -- start 2024-04-24T23:58:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.567+0000 7f4a12c4d640 1 -- start start 2024-04-24T23:58:45.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.567+0000 7f4a12c4d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4a0c15e510 0x7f4a0c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.567+0000 7f4a12c4d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c0d2e00 con 0x7f4a0c0b63b0 2024-04-24T23:58:45.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.567+0000 7f4a12c4d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c0d30e0 con 0x7f4a0c0d28c0 2024-04-24T23:58:45.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.567+0000 7f4a12c4d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0c0d33c0 con 0x7f4a0c15e510 2024-04-24T23:58:45.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a111ea640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4a0c0b63b0 0x7f4a0c0b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58686/0 (socket says 172.21.15.113:58686) 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a111ea640 1 -- 172.21.15.113:0/231335316 learned_addr learned my addr 172.21.15.113:0/231335316 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a109e9640 1 --2- 172.21.15.113:0/231335316 >> v2:172.21.15.178:3300/0 conn(0x7f4a0c15e510 0x7f4a0c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1149032253 0 0) 0x7f4a0c0d2e00 con 0x7f4a0c0b63b0 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49fc003520 con 0x7f4a0c0b63b0 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 777917537 0 0) 0x7f49fc003520 con 0x7f4a0c0b63b0 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 >> v2:172.21.15.178:3300/0 conn(0x7f4a0c15e510 msgr2=0x7f4a0c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 --2- 172.21.15.113:0/231335316 >> v2:172.21.15.178:3300/0 conn(0x7f4a0c15e510 0x7f4a0c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 >> v1:172.21.15.178:6789/0 conn(0x7f4a0c0d28c0 legacy=0x7f4a0c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c0d36a0 con 0x7f4a0c0b63b0 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f4a040019f0 con 0x7f4a0c0b63b0 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 >> v1:172.21.15.113:6789/0 conn(0x7f4a0c0b63b0 legacy=0x7f4a0c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d33c0 con 0x7f49fc0039b0 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc00c160 con 0x7f49fc0051c0 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc00c360 con 0x7f49fc004800 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a109e9640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a111ea640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a0bfff640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.568+0000 7f4a109e9640 1 -- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49fc004800 msgr2=0x7f49fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a109e9640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a109e9640 1 -- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49fc0051c0 msgr2=0x7f49fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a109e9640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a111ea640 1 -- 172.21.15.113:0/231335316 reap_dead start 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a109e9640 1 -- 172.21.15.113:0/231335316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc003520 con 0x7f49fc0039b0 2024-04-24T23:58:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a109e9640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009ac0 tx=0x7f49fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7135ce67440115 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00036070 con 0x7f49fc0039b0 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a00031400 con 0x7f49fc0039b0 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.569+0000 7f4a0b7fe640 1 -- 172.21.15.113:0/231335316 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00009690 con 0x7f49fc0039b0 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.570+0000 7f4a12c4d640 1 -- 172.21.15.113:0/231335316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 msgr2=0x7f49fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.570+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009ac0 tx=0x7f49fc005830 comp rx=0 tx=0).stop 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.570+0000 7f4a12c4d640 1 -- 172.21.15.113:0/231335316 shutdown_connections 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.570+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/231335316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.570+0000 7f4a12c4d640 1 -- 172.21.15.113:0/231335316 >> 172.21.15.113:0/231335316 conn(0x7f4a0c06b720 msgr2=0x7f4a0c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.570+0000 7f4a12c4d640 1 -- 172.21.15.113:0/231335316 shutdown_connections 2024-04-24T23:58:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.570+0000 7f4a12c4d640 1 -- 172.21.15.113:0/231335316 wait complete. 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.571+0000 7f4a12c4d640 1 Processor -- start 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.571+0000 7f4a12c4d640 1 -- start start 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a12c4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f4a0c160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a12c4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a0c15e980 0x7f4a0c151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a12c4d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a0c0b8890 0x7f4a0c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a12c4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0b9dd0 con 0x7f49fc0039b0 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a12c4d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d3ea0 con 0x7f4a0c15e980 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a12c4d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d2380 con 0x7f4a0c0b8890 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f4a0c160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f4a0c160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41484/0 (socket says 172.21.15.113:41484) 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 -- 172.21.15.113:0/334270253 learned_addr learned my addr 172.21.15.113:0/334270253 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a111ea640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a0c0b8890 0x7f4a0c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a0bfff640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a0c15e980 0x7f4a0c151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 -- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a0c0b8890 msgr2=0x7f4a0c0b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a0c0b8890 0x7f4a0c0b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 -- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a0c15e980 msgr2=0x7f4a0c151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a0c15e980 0x7f4a0c151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 -- 172.21.15.113:0/334270253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c151550 con 0x7f49fc0039b0 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a111ea640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a0c0b8890 0x7f4a0c0b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.572+0000 7f4a109e9640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f4a0c160c30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a00034040 tx=0x7f4a00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.573+0000 7f4a097fa640 1 -- 172.21.15.113:0/334270253 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0003f020 con 0x7f49fc0039b0 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.573+0000 7f4a097fa640 1 -- 172.21.15.113:0/334270253 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a00037040 con 0x7f49fc0039b0 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.573+0000 7f4a097fa640 1 -- 172.21.15.113:0/334270253 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00036070 con 0x7f49fc0039b0 2024-04-24T23:58:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.573+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c15f0d0 con 0x7f49fc0039b0 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.573+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c15f690 con 0x7f49fc0039b0 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.574+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c0691a0 con 0x7f49fc0039b0 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.574+0000 7f4a097fa640 1 -- 172.21.15.113:0/334270253 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a00042020 con 0x7f49fc0039b0 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.574+0000 7f4a097fa640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49f006be30 0x7f49f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.574+0000 7f4a097fa640 1 -- 172.21.15.113:0/334270253 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a000aaea0 con 0x7f49fc0039b0 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.574+0000 7f4a0bfff640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49f006be30 0x7f49f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.575+0000 7f4a0bfff640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49f006be30 0x7f49f006e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000c00 tx=0x7f49f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.578+0000 7f4a097fa640 1 -- 172.21.15.113:0/334270253 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a00049030 con 0x7f49fc0039b0 2024-04-24T23:58:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.751+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4a0c0b4100 con 0x7f49fc0039b0 2024-04-24T23:58:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.751+0000 7f4a097fa640 1 -- 172.21.15.113:0/334270253 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f4a00079910 con 0x7f49fc0039b0 2024-04-24T23:58:45.751 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49f006be30 msgr2=0x7f49f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49f006be30 0x7f49f006e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000c00 tx=0x7f49f8009040 comp rx=0 tx=0).stop 2024-04-24T23:58:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 msgr2=0x7f4a0c160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f4a0c160c30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a00034040 tx=0x7f4a00004730 comp rx=0 tx=0).stop 2024-04-24T23:58:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 shutdown_connections 2024-04-24T23:58:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49f006be30 0x7f49f006e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a0c15e980 0x7f4a0c151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a0c0b8890 0x7f4a0c0b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 --2- 172.21.15.113:0/334270253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49fc0039b0 0x7f4a0c160c30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 >> 172.21.15.113:0/334270253 conn(0x7f4a0c06b720 msgr2=0x7f4a0c071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 shutdown_connections 2024-04-24T23:58:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.753+0000 7f4a12c4d640 1 -- 172.21.15.113:0/334270253 wait complete. 2024-04-24T23:58:45.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.863+0000 7f76a0ef5640 1 Processor -- start 2024-04-24T23:58:45.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.863+0000 7f76a0ef5640 1 -- start start 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.864+0000 7f76a0ef5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.864+0000 7f76a0ef5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f769c0cf3f0 0x7f769c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.864+0000 7f76a0ef5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769c06bdc0 con 0x7f769c06b800 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.864+0000 7f76a0ef5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f769c0b2280 con 0x7f769c0cf3f0 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.864+0000 7f76a0ef5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f769c0c5700 con 0x7f769c156600 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f769ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f769c0cf3f0 0x7f769c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f769a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f7699d9b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f769c06b800 0x7f769c06d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:38588/0 (socket says 172.21.15.113:38588) 2024-04-24T23:58:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f7699d9b640 1 -- 172.21.15.113:0/1299586414 learned_addr learned my addr 172.21.15.113:0/1299586414 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f769ad9d640 1 -- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 msgr2=0x7f769c15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f769ad9d640 1 --2- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f769ad9d640 1 -- 172.21.15.113:0/1299586414 >> v1:172.21.15.178:6789/0 conn(0x7f769c06b800 legacy=0x7f769c06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f769ad9d640 1 -- 172.21.15.113:0/1299586414 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c0c59e0 con 0x7f769c0cf3f0 2024-04-24T23:58:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.865+0000 7f769a59c640 1 --2- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:45.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.866+0000 7f769ad9d640 1 --2- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c0cf3f0 0x7f769c0cf880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097d0 tx=0x7f768c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f88a6b60ad20371 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.866+0000 7f769959a640 1 -- 172.21.15.113:0/1299586414 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c037070 con 0x7f769c0cf3f0 2024-04-24T23:58:45.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.866+0000 7f769959a640 1 -- 172.21.15.113:0/1299586414 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c032440 con 0x7f769c0cf3f0 2024-04-24T23:58:45.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.866+0000 7f769959a640 1 -- 172.21.15.113:0/1299586414 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c032810 con 0x7f769c0cf3f0 2024-04-24T23:58:45.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.866+0000 7f76a0ef5640 1 -- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c0cf3f0 msgr2=0x7f769c0cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.866+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c0cf3f0 0x7f769c0cf880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097d0 tx=0x7f768c02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.867+0000 7f76a0ef5640 1 -- 172.21.15.113:0/1299586414 shutdown_connections 2024-04-24T23:58:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.867+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c0cf3f0 0x7f769c0cf880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.867+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/1299586414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.867+0000 7f76a0ef5640 1 -- 172.21.15.113:0/1299586414 >> 172.21.15.113:0/1299586414 conn(0x7f769c06fb60 msgr2=0x7f769c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.867+0000 7f76a0ef5640 1 -- 172.21.15.113:0/1299586414 shutdown_connections 2024-04-24T23:58:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.867+0000 7f76a0ef5640 1 -- 172.21.15.113:0/1299586414 wait complete. 2024-04-24T23:58:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.868+0000 7f76a0ef5640 1 Processor -- start 2024-04-24T23:58:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.868+0000 7f76a0ef5640 1 -- start start 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c06b800 0x7f769c151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f769c0cf3f0 0x7f769c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c0c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f769c15fff0 con 0x7f769c0cf3f0 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f769c1569e0 con 0x7f769c06b800 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f769c160170 con 0x7f769c156600 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f769c0cf3f0 0x7f769c0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f769c0cf3f0 0x7f769c0c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41488/0 (socket says 172.21.15.113:41488) 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 -- 172.21.15.113:0/2824268532 learned_addr learned my addr 172.21.15.113:0/2824268532 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f769ad9d640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c0c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f769a59c640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c06b800 0x7f769c151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 -- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 msgr2=0x7f769c0c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c0c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 -- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c06b800 msgr2=0x7f769c151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c06b800 0x7f769c151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 -- 172.21.15.113:0/2824268532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c141f00 con 0x7f769c0cf3f0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f769ad9d640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c0c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f7699d9b640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f769c0cf3f0 0x7f769c0c9f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f769000cb10 tx=0x7f769000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f768affd640 1 -- 172.21.15.113:0/2824268532 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690017070 con 0x7f769c0cf3f0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f768affd640 1 -- 172.21.15.113:0/2824268532 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f769000ac70 con 0x7f769c0cf3f0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c1421e0 con 0x7f769c0cf3f0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f768affd640 1 -- 172.21.15.113:0/2824268532 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690012620 con 0x7f769c0cf3f0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.869+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c1427a0 con 0x7f769c0cf3f0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.870+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c142f70 con 0x7f769c0cf3f0 2024-04-24T23:58:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.871+0000 7f768affd640 1 -- 172.21.15.113:0/2824268532 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f769001a070 con 0x7f769c0cf3f0 2024-04-24T23:58:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.871+0000 7f768affd640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f767c06be60 0x7f767c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.871+0000 7f768affd640 1 -- 172.21.15.113:0/2824268532 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f769008aaa0 con 0x7f769c0cf3f0 2024-04-24T23:58:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.871+0000 7f769a59c640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f767c06be60 0x7f767c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.872+0000 7f769a59c640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f767c06be60 0x7f767c06e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f769c0c9d20 tx=0x7f7694009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:45.875+0000 7f768affd640 1 -- 172.21.15.113:0/2824268532 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7690058e90 con 0x7f769c0cf3f0 2024-04-24T23:58:46.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.050+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f769c074ea0 con 0x7f769c0cf3f0 2024-04-24T23:58:46.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.050+0000 7f768affd640 1 -- 172.21.15.113:0/2824268532 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f769005d190 con 0x7f769c0cf3f0 2024-04-24T23:58:46.050 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:46.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f767c06be60 msgr2=0x7f767c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f767c06be60 0x7f767c06e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f769c0c9d20 tx=0x7f7694009190 comp rx=0 tx=0).stop 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f769c0cf3f0 msgr2=0x7f769c0c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f769c0cf3f0 0x7f769c0c9f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f769000cb10 tx=0x7f769000a060 comp rx=0 tx=0).stop 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 shutdown_connections 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f767c06be60 0x7f767c06e320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f769c156600 0x7f769c0c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f769c0cf3f0 0x7f769c0c9f20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 --2- 172.21.15.113:0/2824268532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f769c06b800 0x7f769c151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 >> 172.21.15.113:0/2824268532 conn(0x7f769c06fb60 msgr2=0x7f769c0b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 shutdown_connections 2024-04-24T23:58:46.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.052+0000 7f76a0ef5640 1 -- 172.21.15.113:0/2824268532 wait complete. 2024-04-24T23:58:46.063 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:46.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.160+0000 7fbe4bb1d640 1 Processor -- start 2024-04-24T23:58:46.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.160+0000 7fbe4bb1d640 1 -- start start 2024-04-24T23:58:46.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe4bb1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe44162710 0x7fbe4406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe4bb1d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe44162bc0 con 0x7fbe44158910 2024-04-24T23:58:46.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe4bb1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe4415f1d0 con 0x7fbe4406b630 2024-04-24T23:58:46.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe4bb1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbe4406ba80 con 0x7fbe44162710 2024-04-24T23:58:46.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe490b8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe44162710 0x7fbe4406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe490b8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe44162710 0x7fbe4406b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41496/0 (socket says 172.21.15.113:41496) 2024-04-24T23:58:46.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe490b8640 1 -- 172.21.15.113:0/2463210 learned_addr learned my addr 172.21.15.113:0/2463210 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:46.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe498b9640 1 --1- 172.21.15.113:0/2463210 >> v1:172.21.15.113:6789/0 conn(0x7fbe44158910 0x7fbe4415e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58698/0 (socket says 172.21.15.113:58698) 2024-04-24T23:58:46.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe490b8640 1 -- 172.21.15.113:0/2463210 >> v1:172.21.15.178:6789/0 conn(0x7fbe4406b630 legacy=0x7fbe441438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:46.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe490b8640 1 -- 172.21.15.113:0/2463210 >> v1:172.21.15.113:6789/0 conn(0x7fbe44158910 legacy=0x7fbe4415e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe488b7640 1 -- 172.21.15.113:0/2463210 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18815489 0 0) 0x7fbe44162bc0 con 0x7fbe44158910 2024-04-24T23:58:46.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.161+0000 7fbe490b8640 1 -- 172.21.15.113:0/2463210 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4415f4b0 con 0x7fbe44162710 2024-04-24T23:58:46.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.162+0000 7fbe490b8640 1 --2- 172.21.15.113:0/2463210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe44162710 0x7fbe4406b0f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009870 tx=0x7fbe3402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c03e7cd18c08ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.162+0000 7fbe488b7640 1 -- 172.21.15.113:0/2463210 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34037070 con 0x7fbe44162710 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.162+0000 7fbe488b7640 1 -- 172.21.15.113:0/2463210 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe34032440 con 0x7fbe44162710 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.162+0000 7fbe488b7640 1 -- 172.21.15.113:0/2463210 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34032810 con 0x7fbe44162710 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.162+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/2463210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe44162710 msgr2=0x7fbe4406b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.162+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/2463210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe44162710 0x7fbe4406b0f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009870 tx=0x7fbe3402a2a0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.163+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/2463210 shutdown_connections 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.163+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/2463210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe44162710 0x7fbe4406b0f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.163+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/2463210 >> 172.21.15.113:0/2463210 conn(0x7fbe440b22c0 msgr2=0x7fbe440b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:46.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.163+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/2463210 shutdown_connections 2024-04-24T23:58:46.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.163+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/2463210 wait complete. 2024-04-24T23:58:46.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.164+0000 7fbe4bb1d640 1 Processor -- start 2024-04-24T23:58:46.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.164+0000 7fbe4bb1d640 1 -- start start 2024-04-24T23:58:46.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.164+0000 7fbe4bb1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4406b630 0x7fbe440cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.164+0000 7fbe4bb1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe44158910 0x7fbe4413ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.164+0000 7fbe4bb1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe44162710 0x7fbe441613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe4bb1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4415eeb0 con 0x7fbe4406b630 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe4bb1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441621f0 con 0x7fbe44162710 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe4bb1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbe440b3710 con 0x7fbe44158910 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4406b630 0x7fbe440cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4406b630 0x7fbe440cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41506/0 (socket says 172.21.15.113:41506) 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 -- 172.21.15.113:0/1686281758 learned_addr learned my addr 172.21.15.113:0/1686281758 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe4a0ba640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe44162710 0x7fbe441613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe490b8640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe44158910 0x7fbe4413ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 -- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe44158910 msgr2=0x7fbe4413ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe44158910 0x7fbe4413ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 -- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe44162710 msgr2=0x7fbe441613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe44162710 0x7fbe441613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 -- 172.21.15.113:0/1686281758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe440ce2d0 con 0x7fbe4406b630 2024-04-24T23:58:46.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe490b8640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe44158910 0x7fbe4413ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe4a0ba640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe44162710 0x7fbe441613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe498b9640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4406b630 0x7fbe440cd9a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c4d0 tx=0x7fbe4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe3a7fc640 1 -- 172.21.15.113:0/1686281758 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40018070 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe3a7fc640 1 -- 172.21.15.113:0/1686281758 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe40007500 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe440ce5b0 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe3a7fc640 1 -- 172.21.15.113:0/1686281758 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40004a20 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.165+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe441f4ee0 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.166+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe440691a0 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.167+0000 7fbe3a7fc640 1 -- 172.21.15.113:0/1686281758 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe4001b070 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.167+0000 7fbe3a7fc640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe2806be30 0x7fbe2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.167+0000 7fbe3a7fc640 1 -- 172.21.15.113:0/1686281758 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe4008a1e0 con 0x7fbe4406b630 2024-04-24T23:58:46.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.167+0000 7fbe490b8640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe2806be30 0x7fbe2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.168+0000 7fbe490b8640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe2806be30 0x7fbe2806e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbe34034000 tx=0x7fbe3402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.171+0000 7fbe3a7fc640 1 -- 172.21.15.113:0/1686281758 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe40058b90 con 0x7fbe4406b630 2024-04-24T23:58:46.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.346+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe44074c50 con 0x7fbe4406b630 2024-04-24T23:58:46.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.348+0000 7fbe3a7fc640 1 -- 172.21.15.113:0/1686281758 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fbe40058490 con 0x7fbe4406b630 2024-04-24T23:58:46.348 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:46.348 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.349+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe2806be30 msgr2=0x7fbe2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.349+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe2806be30 0x7fbe2806e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbe34034000 tx=0x7fbe3402e000 comp rx=0 tx=0).stop 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4406b630 msgr2=0x7fbe440cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4406b630 0x7fbe440cd9a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c4d0 tx=0x7fbe4000c990 comp rx=0 tx=0).stop 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 shutdown_connections 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbe2806be30 0x7fbe2806e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe44162710 0x7fbe441613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe44158910 0x7fbe4413ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 --2- 172.21.15.113:0/1686281758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe4406b630 0x7fbe440cd9a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 >> 172.21.15.113:0/1686281758 conn(0x7fbe440b22c0 msgr2=0x7fbe441421f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:46.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 shutdown_connections 2024-04-24T23:58:46.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.350+0000 7fbe4bb1d640 1 -- 172.21.15.113:0/1686281758 wait complete. 2024-04-24T23:58:46.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:46.361 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:46.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.461+0000 7ff19a831640 1 Processor -- start 2024-04-24T23:58:46.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.461+0000 7ff19a831640 1 -- start start 2024-04-24T23:58:46.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff19a831640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 0x7ff19406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff19a831640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff194061b80 0x7ff194062010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff19a831640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19415ea00 con 0x7ff194161fd0 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff19a831640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff1940cfc30 con 0x7ff194061b80 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff19a831640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff194162490 con 0x7ff1940cd4b0 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff1937fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff194161fd0 0x7ff1940cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58700/0 (socket says 172.21.15.113:58700) 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff1937fe640 1 -- 172.21.15.113:0/1757713625 learned_addr learned my addr 172.21.15.113:0/1757713625 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065362448 0 0) 0x7ff19415ea00 con 0x7ff194161fd0 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff193fff640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 0x7ff19406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff198dce640 1 --2- 172.21.15.113:0/1757713625 >> v2:172.21.15.178:3300/0 conn(0x7ff194061b80 0x7ff194062010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff17c003520 con 0x7ff194161fd0 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 200407346 0 0) 0x7ff17c003520 con 0x7ff194161fd0 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 msgr2=0x7ff19406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:46.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 0x7ff19406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 >> v2:172.21.15.178:3300/0 conn(0x7ff194061b80 msgr2=0x7ff194062010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 --2- 172.21.15.113:0/1757713625 >> v2:172.21.15.178:3300/0 conn(0x7ff194061b80 0x7ff194062010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff194162770 con 0x7ff194161fd0 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff1880017f0 con 0x7ff194161fd0 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 >> v1:172.21.15.113:6789/0 conn(0x7ff194161fd0 legacy=0x7ff1940cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff17c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 0x7ff17c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c004d40 0x7ff17c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1940cfc30 con 0x7ff17c0039b0 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff194162490 con 0x7ff17c004d40 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff17c00b9c0 con 0x7ff1940cd4b0 2024-04-24T23:58:46.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.462+0000 7ff193fff640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff17c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff1937fe640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 0x7ff17c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff198dce640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c004d40 0x7ff17c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff193fff640 1 -- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 msgr2=0x7ff17c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff193fff640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1940cd4b0 0x7ff17c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff193fff640 1 -- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c004d40 msgr2=0x7ff17c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff193fff640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c004d40 0x7ff17c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff193fff640 1 -- 172.21.15.113:0/1757713625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c003520 con 0x7ff17c0039b0 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff198dce640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c004d40 0x7ff17c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff193fff640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff17c003dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff184009ac0 tx=0x7ff17c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1b6429a22ff7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.463+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184036070 con 0x7ff17c0039b0 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff184031400 con 0x7ff17c0039b0 2024-04-24T23:58:46.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff192ffd640 1 -- 172.21.15.113:0/1757713625 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184009690 con 0x7ff17c0039b0 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff19a831640 1 -- 172.21.15.113:0/1757713625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 msgr2=0x7ff17c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff19a831640 1 --2- 172.21.15.113:0/1757713625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff17c003dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff184009ac0 tx=0x7ff17c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff198dce640 1 -- 172.21.15.113:0/1757713625 reap_dead start 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff19a831640 1 -- 172.21.15.113:0/1757713625 shutdown_connections 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff19a831640 1 -- 172.21.15.113:0/1757713625 >> 172.21.15.113:0/1757713625 conn(0x7ff1940d0e30 msgr2=0x7ff1940d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff19a831640 1 -- 172.21.15.113:0/1757713625 shutdown_connections 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.464+0000 7ff19a831640 1 -- 172.21.15.113:0/1757713625 wait complete. 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.465+0000 7ff19a831640 1 Processor -- start 2024-04-24T23:58:46.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.465+0000 7ff19a831640 1 -- start start 2024-04-24T23:58:46.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff19a831640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff1940c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff19a831640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1941f3990 0x7ff1940cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff19a831640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941526b0 0x7ff1940d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff19a831640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1940cf1d0 con 0x7ff17c0039b0 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff19a831640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff194162d30 con 0x7ff1941526b0 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff19a831640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff19406c210 con 0x7ff1941f3990 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff1940c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff1940c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41528/0 (socket says 172.21.15.113:41528) 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 -- 172.21.15.113:0/3379679441 learned_addr learned my addr 172.21.15.113:0/3379679441 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff1937fe640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1941f3990 0x7ff1940cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff198dce640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941526b0 0x7ff1940d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 -- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1941f3990 msgr2=0x7ff1940cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1941f3990 0x7ff1940cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 -- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941526b0 msgr2=0x7ff1940d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941526b0 0x7ff1940d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.466+0000 7ff193fff640 1 -- 172.21.15.113:0/3379679441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1941524c0 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.467+0000 7ff193fff640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff1940c7400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff184009ac0 tx=0x7ff184004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.467+0000 7ff190ff9640 1 -- 172.21.15.113:0/3379679441 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184040020 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.467+0000 7ff190ff9640 1 -- 172.21.15.113:0/3379679441 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff184003bc0 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.467+0000 7ff190ff9640 1 -- 172.21.15.113:0/3379679441 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184036040 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.467+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1940c6360 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.467+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1940c6920 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.468+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1940691a0 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.468+0000 7ff190ff9640 1 -- 172.21.15.113:0/3379679441 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff184043020 con 0x7ff17c0039b0 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.469+0000 7ff190ff9640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff18006be60 0x7ff18006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.469+0000 7ff1937fe640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff18006be60 0x7ff18006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.469+0000 7ff190ff9640 1 -- 172.21.15.113:0/3379679441 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff1840aaf00 con 0x7ff17c0039b0 2024-04-24T23:58:46.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.469+0000 7ff1937fe640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff18006be60 0x7ff18006e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1940b6450 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.473+0000 7ff190ff9640 1 -- 172.21.15.113:0/3379679441 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff18404a070 con 0x7ff17c0039b0 2024-04-24T23:58:46.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.642+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff19406ba10 con 0x7ff17c0039b0 2024-04-24T23:58:46.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.643+0000 7ff190ff9640 1 -- 172.21.15.113:0/3379679441 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff1840044e0 con 0x7ff17c0039b0 2024-04-24T23:58:46.643 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:46.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff18006be60 msgr2=0x7ff18006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff18006be60 0x7ff18006e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1940b6450 tx=0x7ff188009040 comp rx=0 tx=0).stop 2024-04-24T23:58:46.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 msgr2=0x7ff1940c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff1940c7400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff184009ac0 tx=0x7ff184004730 comp rx=0 tx=0).stop 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 shutdown_connections 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff18006be60 0x7ff18006e320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941526b0 0x7ff1940d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1941f3990 0x7ff1940cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 --2- 172.21.15.113:0/3379679441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff17c0039b0 0x7ff1940c7400 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 >> 172.21.15.113:0/3379679441 conn(0x7ff1940d0e30 msgr2=0x7ff1940d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 shutdown_connections 2024-04-24T23:58:46.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.645+0000 7ff19a831640 1 -- 172.21.15.113:0/3379679441 wait complete. 2024-04-24T23:58:46.658 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.758+0000 7fc8ffde9640 1 Processor -- start 2024-04-24T23:58:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.758+0000 7fc8ffde9640 1 -- start start 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8ffde9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc8f815e2d0 0x7fc8f815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8ffde9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f80b3080 0x7fc8f80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8ffde9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f815f4f0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8ffde9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f80cde00 con 0x7fc8f80b3080 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8ffde9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f815f7d0 con 0x7fc8f815e2d0 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fd384640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f80b3080 0x7fc8f80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fd384640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f80b3080 0x7fc8f80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41536/0 (socket says 172.21.15.113:41536) 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fd384640 1 -- 172.21.15.113:0/2796386917 learned_addr learned my addr 172.21.15.113:0/2796386917 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fe386640 1 --1- 172.21.15.113:0/2796386917 >> v1:172.21.15.113:6789/0 conn(0x7fc8f815e6b0 0x7fc8f8071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58712/0 (socket says 172.21.15.113:58712) 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fdb85640 1 --2- 172.21.15.113:0/2796386917 >> v2:172.21.15.178:3300/0 conn(0x7fc8f815e2d0 0x7fc8f815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fd384640 1 -- 172.21.15.113:0/2796386917 >> v2:172.21.15.178:3300/0 conn(0x7fc8f815e2d0 msgr2=0x7fc8f815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fd384640 1 --2- 172.21.15.113:0/2796386917 >> v2:172.21.15.178:3300/0 conn(0x7fc8f815e2d0 0x7fc8f815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fd384640 1 -- 172.21.15.113:0/2796386917 >> v1:172.21.15.113:6789/0 conn(0x7fc8f815e6b0 legacy=0x7fc8f8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fd384640 1 -- 172.21.15.113:0/2796386917 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f815fab0 con 0x7fc8f80b3080 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.759+0000 7fc8fcb83640 1 -- 172.21.15.113:0/2796386917 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405525589 0 0) 0x7fc8f815f4f0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8fd384640 1 --2- 172.21.15.113:0/2796386917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f80b3080 0x7fc8f80b4330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80b2d40 tx=0x7fc8e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ef570fa6bfd5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8fcb83640 1 -- 172.21.15.113:0/2796386917 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8037070 con 0x7fc8f80b3080 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8fcb83640 1 -- 172.21.15.113:0/2796386917 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e8032440 con 0x7fc8f80b3080 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8fcb83640 1 -- 172.21.15.113:0/2796386917 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8032810 con 0x7fc8f80b3080 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8ffde9640 1 -- 172.21.15.113:0/2796386917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f80b3080 msgr2=0x7fc8f80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/2796386917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f80b3080 0x7fc8f80b4330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80b2d40 tx=0x7fc8e802a270 comp rx=0 tx=0).stop 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8ffde9640 1 -- 172.21.15.113:0/2796386917 shutdown_connections 2024-04-24T23:58:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/2796386917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f80b3080 0x7fc8f80b4330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/2796386917 >> v2:172.21.15.178:3300/0 conn(0x7fc8f815e2d0 0x7fc8f815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.760+0000 7fc8ffde9640 1 -- 172.21.15.113:0/2796386917 >> 172.21.15.113:0/2796386917 conn(0x7fc8f806b0f0 msgr2=0x7fc8f806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.761+0000 7fc8ffde9640 1 -- 172.21.15.113:0/2796386917 shutdown_connections 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.761+0000 7fc8ffde9640 1 -- 172.21.15.113:0/2796386917 wait complete. 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 Processor -- start 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 -- start start 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f80b3080 0x7fc8f80ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f815e2d0 0x7fc8f80af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f815e6b0 0x7fc8f806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f815f2b0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f815f130 con 0x7fc8f815e2d0 2024-04-24T23:58:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8ffde9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f80cf2a0 con 0x7fc8f80b3080 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8fe386640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f815e6b0 0x7fc8f806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8fe386640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f815e6b0 0x7fc8f806cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41548/0 (socket says 172.21.15.113:41548) 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.762+0000 7fc8fe386640 1 -- 172.21.15.113:0/804348595 learned_addr learned my addr 172.21.15.113:0/804348595 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fd384640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f815e2d0 0x7fc8f80af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fdb85640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f80b3080 0x7fc8f80ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fe386640 1 -- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f80b3080 msgr2=0x7fc8f80ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fe386640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f80b3080 0x7fc8f80ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fe386640 1 -- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f815e2d0 msgr2=0x7fc8f80af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fe386640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f815e2d0 0x7fc8f80af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fe386640 1 -- 172.21.15.113:0/804348595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f80af1d0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fd384640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f815e2d0 0x7fc8f80af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8fe386640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f815e6b0 0x7fc8f806cea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c700 tx=0x7fc8f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8ee7fc640 1 -- 172.21.15.113:0/804348595 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0018070 con 0x7fc8f815e6b0 2024-04-24T23:58:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8ee7fc640 1 -- 172.21.15.113:0/804348595 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f000ce10 con 0x7fc8f815e6b0 2024-04-24T23:58:46.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f80ae930 con 0x7fc8f815e6b0 2024-04-24T23:58:46.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8ee7fc640 1 -- 172.21.15.113:0/804348595 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000a860 con 0x7fc8f815e6b0 2024-04-24T23:58:46.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.763+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f80aeef0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.764+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f80691a0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.765+0000 7fc8ee7fc640 1 -- 172.21.15.113:0/804348595 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8f001b070 con 0x7fc8f815e6b0 2024-04-24T23:58:46.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.765+0000 7fc8ee7fc640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc8dc06be30 0x7fc8dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:46.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.765+0000 7fc8ee7fc640 1 -- 172.21.15.113:0/804348595 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc8f008a3e0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.765+0000 7fc8fdb85640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc8dc06be30 0x7fc8dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:46.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.765+0000 7fc8fdb85640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc8dc06be30 0x7fc8dc06e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009ad0 tx=0x7fc8f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:46.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.769+0000 7fc8ee7fc640 1 -- 172.21.15.113:0/804348595 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc8f0058d90 con 0x7fc8f815e6b0 2024-04-24T23:58:46.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.940+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8f80cf7d0 con 0x7fc8f815e6b0 2024-04-24T23:58:46.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.941+0000 7fc8ee7fc640 1 -- 172.21.15.113:0/804348595 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc8f0058690 con 0x7fc8f815e6b0 2024-04-24T23:58:46.941 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc8dc06be30 msgr2=0x7fc8dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc8dc06be30 0x7fc8dc06e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009ad0 tx=0x7fc8f4008040 comp rx=0 tx=0).stop 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f815e6b0 msgr2=0x7fc8f806cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f815e6b0 0x7fc8f806cea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c700 tx=0x7fc8f000cbc0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 shutdown_connections 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc8dc06be30 0x7fc8dc06e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f815e2d0 0x7fc8f80af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8f815e6b0 0x7fc8f806cea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 --2- 172.21.15.113:0/804348595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f80b3080 0x7fc8f80ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 >> 172.21.15.113:0/804348595 conn(0x7fc8f806b0f0 msgr2=0x7fc8f8158990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 shutdown_connections 2024-04-24T23:58:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:46.943+0000 7fc8ffde9640 1 -- 172.21.15.113:0/804348595 wait complete. 2024-04-24T23:58:46.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:47.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.054+0000 7f2b82ad4640 1 Processor -- start 2024-04-24T23:58:47.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.054+0000 7f2b82ad4640 1 -- start start 2024-04-24T23:58:47.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.054+0000 7f2b82ad4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c15e510 0x7f2b7c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.054+0000 7f2b82ad4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2b7c0d28c0 0x7f2b7c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.054+0000 7f2b82ad4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c0d2e00 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.054+0000 7f2b82ad4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c0d30e0 con 0x7f2b7c0d28c0 2024-04-24T23:58:47.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.054+0000 7f2b82ad4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c0d33c0 con 0x7f2b7c15e510 2024-04-24T23:58:47.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b80870640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c15e510 0x7f2b7c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b7bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2b7c0d28c0 0x7f2b7c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b81071640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2b7c0b61e0 0x7f2b7c071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:38600/0 (socket says 172.21.15.113:38600) 2024-04-24T23:58:47.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b81071640 1 -- 172.21.15.113:0/2416300009 learned_addr learned my addr 172.21.15.113:0/2416300009 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:47.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b80870640 1 -- 172.21.15.113:0/2416300009 >> v2:172.21.15.178:3300/0 conn(0x7f2b7c0d28c0 msgr2=0x7f2b7c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:47.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b80870640 1 --2- 172.21.15.113:0/2416300009 >> v2:172.21.15.178:3300/0 conn(0x7f2b7c0d28c0 0x7f2b7c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b80870640 1 -- 172.21.15.113:0/2416300009 >> v1:172.21.15.178:6789/0 conn(0x7f2b7c0b61e0 legacy=0x7f2b7c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.055+0000 7f2b80870640 1 -- 172.21.15.113:0/2416300009 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0d36a0 con 0x7f2b7c15e510 2024-04-24T23:58:47.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b80870640 1 --2- 172.21.15.113:0/2416300009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c15e510 0x7f2b7c0d2380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2b700097a0 tx=0x7f2b7002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330fbd0050449b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b7b7fe640 1 -- 172.21.15.113:0/2416300009 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70037070 con 0x7f2b7c15e510 2024-04-24T23:58:47.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b7b7fe640 1 -- 172.21.15.113:0/2416300009 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7002ae50 con 0x7f2b7c15e510 2024-04-24T23:58:47.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b7b7fe640 1 -- 172.21.15.113:0/2416300009 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70032660 con 0x7f2b7c15e510 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b82ad4640 1 -- 172.21.15.113:0/2416300009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c15e510 msgr2=0x7f2b7c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/2416300009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c15e510 0x7f2b7c0d2380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2b700097a0 tx=0x7f2b7002a100 comp rx=0 tx=0).stop 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b82ad4640 1 -- 172.21.15.113:0/2416300009 shutdown_connections 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/2416300009 >> v2:172.21.15.178:3300/0 conn(0x7f2b7c0d28c0 0x7f2b7c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/2416300009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c15e510 0x7f2b7c0d2380 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b82ad4640 1 -- 172.21.15.113:0/2416300009 >> 172.21.15.113:0/2416300009 conn(0x7f2b7c06b720 msgr2=0x7f2b7c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.056+0000 7f2b82ad4640 1 -- 172.21.15.113:0/2416300009 shutdown_connections 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.057+0000 7f2b82ad4640 1 -- 172.21.15.113:0/2416300009 wait complete. 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 Processor -- start 2024-04-24T23:58:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 -- start start 2024-04-24T23:58:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b61e0 0x7f2b7c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c0d28c0 0x7f2b7c1509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7c15e510 0x7f2b7c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0b9dd0 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0d3ea0 con 0x7f2b7c15e510 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.058+0000 7f2b82ad4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c0b83d0 con 0x7f2b7c0d28c0 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b61e0 0x7f2b7c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b61e0 0x7f2b7c151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41554/0 (socket says 172.21.15.113:41554) 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 -- 172.21.15.113:0/615629346 learned_addr learned my addr 172.21.15.113:0/615629346 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b81071640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7c15e510 0x7f2b7c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b7bfff640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c0d28c0 0x7f2b7c1509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 -- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c0d28c0 msgr2=0x7f2b7c1509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c0d28c0 0x7f2b7c1509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 -- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7c15e510 msgr2=0x7f2b7c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7c15e510 0x7f2b7c0cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 -- 172.21.15.113:0/615629346 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0cd850 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b80870640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b61e0 0x7f2b7c151110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2b700098c0 tx=0x7f2b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b797fa640 1 -- 172.21.15.113:0/615629346 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70041030 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b797fa640 1 -- 172.21.15.113:0/615629346 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7002ac50 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c0c8040 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b797fa640 1 -- 172.21.15.113:0/615629346 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70037040 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.059+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c0c8600 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.060+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c0ce070 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.061+0000 7f2b797fa640 1 -- 172.21.15.113:0/615629346 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b70044070 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.061+0000 7f2b797fa640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b6006be30 0x7f2b6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.061+0000 7f2b7bfff640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b6006be30 0x7f2b6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.061+0000 7f2b797fa640 1 -- 172.21.15.113:0/615629346 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b700ab2e0 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.062+0000 7f2b7bfff640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b6006be30 0x7f2b6006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0c9d10 tx=0x7f2b68009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.066+0000 7f2b797fa640 1 -- 172.21.15.113:0/615629346 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b700797f0 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.238+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b7c0691a0 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.238+0000 7f2b797fa640 1 -- 172.21.15.113:0/615629346 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f2b7007daf0 con 0x7f2b7c0b61e0 2024-04-24T23:58:47.238 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b6006be30 msgr2=0x7f2b6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b6006be30 0x7f2b6006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0c9d10 tx=0x7f2b68009190 comp rx=0 tx=0).stop 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b61e0 msgr2=0x7f2b7c151110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b61e0 0x7f2b7c151110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2b700098c0 tx=0x7f2b700046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 shutdown_connections 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b6006be30 0x7f2b6006e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7c15e510 0x7f2b7c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b7c0d28c0 0x7f2b7c1509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 --2- 172.21.15.113:0/615629346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b7c0b61e0 0x7f2b7c151110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 >> 172.21.15.113:0/615629346 conn(0x7f2b7c06b720 msgr2=0x7f2b7c0b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 shutdown_connections 2024-04-24T23:58:47.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.240+0000 7f2b82ad4640 1 -- 172.21.15.113:0/615629346 wait complete. 2024-04-24T23:58:47.298 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-24T23:58:47.299 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-24T23:58:47.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:47.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.396+0000 7fdd30161640 1 Processor -- start 2024-04-24T23:58:47.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.396+0000 7fdd30161640 1 -- start start 2024-04-24T23:58:47.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.396+0000 7fdd30161640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 0x7fdd280b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd30161640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd28144cf0 con 0x7fdd2806e6e0 2024-04-24T23:58:47.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd30161640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd280b2f30 con 0x7fdd280b5240 2024-04-24T23:58:47.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd30161640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd280b4510 con 0x7fdd28144910 2024-04-24T23:58:47.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2e6fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdd2806e6e0 0x7fdd280cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58724/0 (socket says 172.21.15.113:58724) 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/3516417256 learned_addr learned my addr 172.21.15.113:0/3516417256 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2defd640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 0x7fdd280b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3933687386 0 0) 0x7fdd28144cf0 con 0x7fdd2806e6e0 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd10003520 con 0x7fdd2806e6e0 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3458242593 0 0) 0x7fdd10003520 con 0x7fdd2806e6e0 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 msgr2=0x7fdd280b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 0x7fdd280b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 >> v1:172.21.15.178:6789/0 conn(0x7fdd280b5240 legacy=0x7fdd2806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd280b3210 con 0x7fdd2806e6e0 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fdd18001670 con 0x7fdd2806e6e0 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 >> v1:172.21.15.113:6789/0 conn(0x7fdd2806e6e0 legacy=0x7fdd280cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd100039e0 0x7fdd10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 0x7fdd10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd10004d40 0x7fdd100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280b4510 con 0x7fdd10004d40 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1000b9c0 con 0x7fdd100039e0 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1000bbc0 con 0x7fdd28144910 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.397+0000 7fdd2e6fe640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd10004d40 0x7fdd100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2defd640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd100039e0 0x7fdd10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2d6fc640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 0x7fdd10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 msgr2=0x7fdd10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2e6fe640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28144910 0x7fdd10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd100039e0 msgr2=0x7fdd10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2e6fe640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd100039e0 0x7fdd10003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/3516417256 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd10003520 con 0x7fdd10004d40 2024-04-24T23:58:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.398+0000 7fdd2e6fe640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd10004d40 0x7fdd100051c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c240 tx=0x7fdd10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e96e1c74d282443 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18007d10 con 0x7fdd10004d40 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd180058d0 con 0x7fdd10004d40 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd2cefb640 1 -- 172.21.15.113:0/3516417256 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18005ce0 con 0x7fdd10004d40 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd30161640 1 -- 172.21.15.113:0/3516417256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd10004d40 msgr2=0x7fdd100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd30161640 1 --2- 172.21.15.113:0/3516417256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd10004d40 0x7fdd100051c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c240 tx=0x7fdd10005830 comp rx=0 tx=0).stop 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/3516417256 reap_dead start 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd30161640 1 -- 172.21.15.113:0/3516417256 shutdown_connections 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd30161640 1 -- 172.21.15.113:0/3516417256 >> 172.21.15.113:0/3516417256 conn(0x7fdd280b1a50 msgr2=0x7fdd280b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd30161640 1 -- 172.21.15.113:0/3516417256 shutdown_connections 2024-04-24T23:58:47.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.399+0000 7fdd30161640 1 -- 172.21.15.113:0/3516417256 wait complete. 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.400+0000 7fdd30161640 1 Processor -- start 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.400+0000 7fdd30161640 1 -- start start 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd30161640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28071080 0x7fdd2815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd30161640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28071460 0x7fdd2815c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd30161640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd281f22d0 0x7fdd281f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd30161640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280b4da0 con 0x7fdd281f22d0 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd30161640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280b37d0 con 0x7fdd28071460 2024-04-24T23:58:47.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd30161640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd280b4f20 con 0x7fdd28071080 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd281f22d0 0x7fdd281f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd281f22d0 0x7fdd281f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41572/0 (socket says 172.21.15.113:41572) 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/2487693821 learned_addr learned my addr 172.21.15.113:0/2487693821 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2d6fc640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28071460 0x7fdd2815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2defd640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28071080 0x7fdd2815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28071080 msgr2=0x7fdd2815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28071080 0x7fdd2815c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28071460 msgr2=0x7fdd2815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28071460 0x7fdd2815c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.401+0000 7fdd2e6fe640 1 -- 172.21.15.113:0/2487693821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd281f5660 con 0x7fdd281f22d0 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.402+0000 7fdd2e6fe640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd281f22d0 0x7fdd281f2750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000c00 tx=0x7fdd280b3100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.402+0000 7fdd267fc640 1 -- 172.21.15.113:0/2487693821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18042070 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.402+0000 7fdd267fc640 1 -- 172.21.15.113:0/2487693821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd18007070 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.402+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd281f5940 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.402+0000 7fdd267fc640 1 -- 172.21.15.113:0/2487693821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18038040 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.402+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd281f5f00 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.403+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd280691a0 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.403+0000 7fdd267fc640 1 -- 172.21.15.113:0/2487693821 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd18045070 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.404+0000 7fdd267fc640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd1406be60 0x7fdd1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.404+0000 7fdd267fc640 1 -- 172.21.15.113:0/2487693821 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdd180a9d40 con 0x7fdd281f22d0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.404+0000 7fdd2defd640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd1406be60 0x7fdd1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.404+0000 7fdd2defd640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd1406be60 0x7fdd1406e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd200099e0 tx=0x7fdd20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.408+0000 7fdd267fc640 1 -- 172.21.15.113:0/2487693821 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd18078010 con 0x7fdd281f22d0 2024-04-24T23:58:47.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.583+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd280731a0 con 0x7fdd281f22d0 2024-04-24T23:58:47.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.584+0000 7fdd267fc640 1 -- 172.21.15.113:0/2487693821 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fdd1807c310 con 0x7fdd281f22d0 2024-04-24T23:58:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:47.585 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd1406be60 msgr2=0x7fdd1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd1406be60 0x7fdd1406e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd200099e0 tx=0x7fdd20008040 comp rx=0 tx=0).stop 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd281f22d0 msgr2=0x7fdd281f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd281f22d0 0x7fdd281f2750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd18000c00 tx=0x7fdd280b3100 comp rx=0 tx=0).stop 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 shutdown_connections 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd1406be60 0x7fdd1406e320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd28071460 0x7fdd2815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd281f22d0 0x7fdd281f2750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 --2- 172.21.15.113:0/2487693821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd28071080 0x7fdd2815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.586+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 >> 172.21.15.113:0/2487693821 conn(0x7fdd280b1a50 msgr2=0x7fdd28143130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.587+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 shutdown_connections 2024-04-24T23:58:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.587+0000 7fdd30161640 1 -- 172.21.15.113:0/2487693821 wait complete. 2024-04-24T23:58:47.597 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:47.598 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.697+0000 7fd847c38640 1 Processor -- start 2024-04-24T23:58:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.697+0000 7fd847c38640 1 -- start start 2024-04-24T23:58:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd847c38640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd84015e2d0 0x7fd84015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd847c38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8400b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd847c38640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd84015f4f0 con 0x7fd84015e6b0 2024-04-24T23:58:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd847c38640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd8400cde00 con 0x7fd84015e2d0 2024-04-24T23:58:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd847c38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd84015f7d0 con 0x7fd8400b3080 2024-04-24T23:58:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8459d4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd84015e2d0 0x7fd84015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8451d3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8400b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8461d5640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd84015e6b0 0x7fd840071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:42406/0 (socket says 172.21.15.113:42406) 2024-04-24T23:58:47.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8461d5640 1 -- 172.21.15.113:0/2002673156 learned_addr learned my addr 172.21.15.113:0/2002673156 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:47.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8459d4640 1 -- 172.21.15.113:0/2002673156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 msgr2=0x7fd8400b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:47.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8459d4640 1 --2- 172.21.15.113:0/2002673156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8400b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8459d4640 1 -- 172.21.15.113:0/2002673156 >> v1:172.21.15.178:6789/0 conn(0x7fd84015e6b0 legacy=0x7fd840071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.698+0000 7fd8459d4640 1 -- 172.21.15.113:0/2002673156 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84015fab0 con 0x7fd84015e2d0 2024-04-24T23:58:47.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.699+0000 7fd8459d4640 1 --2- 172.21.15.113:0/2002673156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e2d0 0x7fd84015ebf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0097a0 tx=0x7fd83c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c7b0b3b1279430 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.699+0000 7fd8449d2640 1 -- 172.21.15.113:0/2002673156 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c037070 con 0x7fd84015e2d0 2024-04-24T23:58:47.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.699+0000 7fd8449d2640 1 -- 172.21.15.113:0/2002673156 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83c02ae50 con 0x7fd84015e2d0 2024-04-24T23:58:47.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.699+0000 7fd8449d2640 1 -- 172.21.15.113:0/2002673156 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c032660 con 0x7fd84015e2d0 2024-04-24T23:58:47.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 -- 172.21.15.113:0/2002673156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e2d0 msgr2=0x7fd84015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 --2- 172.21.15.113:0/2002673156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e2d0 0x7fd84015ebf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0097a0 tx=0x7fd83c02a100 comp rx=0 tx=0).stop 2024-04-24T23:58:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 -- 172.21.15.113:0/2002673156 shutdown_connections 2024-04-24T23:58:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 --2- 172.21.15.113:0/2002673156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8400b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 --2- 172.21.15.113:0/2002673156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e2d0 0x7fd84015ebf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 -- 172.21.15.113:0/2002673156 >> 172.21.15.113:0/2002673156 conn(0x7fd84006b0f0 msgr2=0x7fd84006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 -- 172.21.15.113:0/2002673156 shutdown_connections 2024-04-24T23:58:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.700+0000 7fd847c38640 1 -- 172.21.15.113:0/2002673156 wait complete. 2024-04-24T23:58:47.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.701+0000 7fd847c38640 1 Processor -- start 2024-04-24T23:58:47.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.701+0000 7fd847c38640 1 -- start start 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8401f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd84015e2d0 0x7fd84014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e6b0 0x7fd8400ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd84015f2b0 con 0x7fd84015e2d0 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd84015f130 con 0x7fd84015e6b0 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd8400cf2a0 con 0x7fd8400b3080 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd84015e2d0 0x7fd84014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd84015e2d0 0x7fd84014cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59160/0 (socket says 172.21.15.113:59160) 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 -- 172.21.15.113:0/2150676504 learned_addr learned my addr 172.21.15.113:0/2150676504 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8461d5640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e6b0 0x7fd8400ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8459d4640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8401f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 -- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 msgr2=0x7fd8401f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8401f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 -- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e6b0 msgr2=0x7fd8400ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e6b0 0x7fd8400ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 -- 172.21.15.113:0/2150676504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8400af2d0 con 0x7fd84015e2d0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8451d3640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd84015e2d0 0x7fd84014cc70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd83000e9b0 tx=0x7fd83000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8367fc640 1 -- 172.21.15.113:0/2150676504 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830017070 con 0x7fd84015e2d0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8367fc640 1 -- 172.21.15.113:0/2150676504 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83000c9a0 con 0x7fd84015e2d0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8401f8450 con 0x7fd84015e2d0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd8367fc640 1 -- 172.21.15.113:0/2150676504 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83000cd70 con 0x7fd84015e2d0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.702+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8401f8a10 con 0x7fd84015e2d0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.703+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8400691a0 con 0x7fd84015e2d0 2024-04-24T23:58:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.704+0000 7fd8367fc640 1 -- 172.21.15.113:0/2150676504 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd83001a070 con 0x7fd84015e2d0 2024-04-24T23:58:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.704+0000 7fd8367fc640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd82406be30 0x7fd82406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.704+0000 7fd8367fc640 1 -- 172.21.15.113:0/2150676504 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd83008ade0 con 0x7fd84015e2d0 2024-04-24T23:58:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.704+0000 7fd8459d4640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd82406be30 0x7fd82406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.705+0000 7fd8459d4640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd82406be30 0x7fd82406e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd83c034000 tx=0x7fd83c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.708+0000 7fd8367fc640 1 -- 172.21.15.113:0/2150676504 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd830059140 con 0x7fd84015e2d0 2024-04-24T23:58:47.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.880+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd840072050 con 0x7fd84015e2d0 2024-04-24T23:58:47.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.881+0000 7fd8367fc640 1 -- 172.21.15.113:0/2150676504 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd83005d440 con 0x7fd84015e2d0 2024-04-24T23:58:47.881 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd82406be30 msgr2=0x7fd82406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd82406be30 0x7fd82406e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd83c034000 tx=0x7fd83c02e000 comp rx=0 tx=0).stop 2024-04-24T23:58:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd84015e2d0 msgr2=0x7fd84014cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd84015e2d0 0x7fd84014cc70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd83000e9b0 tx=0x7fd83000ee70 comp rx=0 tx=0).stop 2024-04-24T23:58:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 shutdown_connections 2024-04-24T23:58:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd82406be30 0x7fd82406e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd84015e6b0 0x7fd8400ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd84015e2d0 0x7fd84014cc70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 --2- 172.21.15.113:0/2150676504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8400b3080 0x7fd8401f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 >> 172.21.15.113:0/2150676504 conn(0x7fd84006b0f0 msgr2=0x7fd8400afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 shutdown_connections 2024-04-24T23:58:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.883+0000 7fd847c38640 1 -- 172.21.15.113:0/2150676504 wait complete. 2024-04-24T23:58:47.894 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:47.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.994+0000 7f9549809640 1 Processor -- start 2024-04-24T23:58:47.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.994+0000 7f9549809640 1 -- start start 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.994+0000 7f9549809640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f954415ba50 0x7f954415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.994+0000 7f9549809640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9544071170 con 0x7f954406cea0 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.994+0000 7f9549809640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9544071450 con 0x7f954415b670 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.994+0000 7f9549809640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9544071730 con 0x7f954415ba50 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f953bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f954415ba50 0x7f954415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f95437fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f954406cea0 0x7f954406d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43106/0 (socket says 172.21.15.113:43106) 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f953bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f954415ba50 0x7f954415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59170/0 (socket says 172.21.15.113:59170) 2024-04-24T23:58:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f95437fe640 1 -- 172.21.15.113:0/3449796728 learned_addr learned my addr 172.21.15.113:0/3449796728 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f953bfff640 1 -- 172.21.15.113:0/3449796728 >> v1:172.21.15.178:6789/0 conn(0x7f954415b670 legacy=0x7f95440cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f953bfff640 1 -- 172.21.15.113:0/3449796728 >> v1:172.21.15.113:6789/0 conn(0x7f954406cea0 legacy=0x7f954406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f95427fc640 1 -- 172.21.15.113:0/3449796728 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975925555 0 0) 0x7f9544071170 con 0x7f954406cea0 2024-04-24T23:58:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.995+0000 7f953bfff640 1 -- 172.21.15.113:0/3449796728 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9544071a10 con 0x7f954415ba50 2024-04-24T23:58:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f953bfff640 1 --2- 172.21.15.113:0/3449796728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f954415bff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f952c00a940 tx=0x7f952c02b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffe5b8bc862804f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f95427fc640 1 -- 172.21.15.113:0/3449796728 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c00b040 con 0x7f954415ba50 2024-04-24T23:58:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f95427fc640 1 -- 172.21.15.113:0/3449796728 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c030710 con 0x7f954415ba50 2024-04-24T23:58:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f95427fc640 1 -- 172.21.15.113:0/3449796728 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c030ae0 con 0x7f954415ba50 2024-04-24T23:58:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f9549809640 1 -- 172.21.15.113:0/3449796728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 msgr2=0x7f954415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f9549809640 1 --2- 172.21.15.113:0/3449796728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f954415bff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f952c00a940 tx=0x7f952c02b4c0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f9549809640 1 -- 172.21.15.113:0/3449796728 shutdown_connections 2024-04-24T23:58:47.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f9549809640 1 --2- 172.21.15.113:0/3449796728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f954415bff0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:47.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f9549809640 1 -- 172.21.15.113:0/3449796728 >> 172.21.15.113:0/3449796728 conn(0x7f954406b0f0 msgr2=0x7f954406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:47.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f9549809640 1 -- 172.21.15.113:0/3449796728 shutdown_connections 2024-04-24T23:58:47.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.996+0000 7f9549809640 1 -- 172.21.15.113:0/3449796728 wait complete. 2024-04-24T23:58:47.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.997+0000 7f9549809640 1 Processor -- start 2024-04-24T23:58:47.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.997+0000 7f9549809640 1 -- start start 2024-04-24T23:58:48.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f9549809640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954406cea0 0x7f9544151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f9549809640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f954415b670 0x7f9544149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f9549809640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f95440b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f9549809640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f95440d0200 con 0x7f954415ba50 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f9549809640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95440d0080 con 0x7f954415b670 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f9549809640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f95440d0380 con 0x7f954406cea0 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f95437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f95440b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f95437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f95440b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59184/0 (socket says 172.21.15.113:59184) 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f95437fe640 1 -- 172.21.15.113:0/3986057922 learned_addr learned my addr 172.21.15.113:0/3986057922 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f953bfff640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f954415b670 0x7f9544149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.998+0000 7f9542ffd640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954406cea0 0x7f9544151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f95437fe640 1 -- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954406cea0 msgr2=0x7f9544151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f95437fe640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954406cea0 0x7f9544151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f95437fe640 1 -- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f954415b670 msgr2=0x7f9544149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f95437fe640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f954415b670 0x7f9544149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f95437fe640 1 -- 172.21.15.113:0/3986057922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9544156df0 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f953bfff640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f954415b670 0x7f9544149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f95437fe640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f95440b3ac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f953c00c7a0 tx=0x7f953c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f9540ff9640 1 -- 172.21.15.113:0/3986057922 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c018070 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f9540ff9640 1 -- 172.21.15.113:0/3986057922 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c00a5e0 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f9540ff9640 1 -- 172.21.15.113:0/3986057922 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c00aa50 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95441570d0 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:47.999+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9544157690 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.000+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95440691a0 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.000+0000 7f9540ff9640 1 -- 172.21.15.113:0/3986057922 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f953c00abf0 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.001+0000 7f9540ff9640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f952406be60 0x7f952406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.001+0000 7f9540ff9640 1 -- 172.21.15.113:0/3986057922 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f953c08a350 con 0x7f954415ba50 2024-04-24T23:58:48.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.001+0000 7f9542ffd640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f952406be60 0x7f952406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.001+0000 7f9542ffd640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f952406be60 0x7f952406e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9534000f60 tx=0x7f9534009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.005+0000 7f9540ff9640 1 -- 172.21.15.113:0/3986057922 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f953c058d00 con 0x7f954415ba50 2024-04-24T23:58:48.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.178+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95440cf5a0 con 0x7f954415ba50 2024-04-24T23:58:48.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.179+0000 7f9540ff9640 1 -- 172.21.15.113:0/3986057922 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f953c058600 con 0x7f954415ba50 2024-04-24T23:58:48.179 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f952406be60 msgr2=0x7f952406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f952406be60 0x7f952406e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9534000f60 tx=0x7f9534009040 comp rx=0 tx=0).stop 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 msgr2=0x7f95440b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f95440b3ac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f953c00c7a0 tx=0x7f953c00cc60 comp rx=0 tx=0).stop 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 shutdown_connections 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f952406be60 0x7f952406e320 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f954415b670 0x7f9544149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954415ba50 0x7f95440b3ac0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 --2- 172.21.15.113:0/3986057922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954406cea0 0x7f9544151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 >> 172.21.15.113:0/3986057922 conn(0x7f954406b0f0 msgr2=0x7f954406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 shutdown_connections 2024-04-24T23:58:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.181+0000 7f9549809640 1 -- 172.21.15.113:0/3986057922 wait complete. 2024-04-24T23:58:48.192 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:48.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.292+0000 7f0e4e038640 1 Processor -- start 2024-04-24T23:58:48.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.292+0000 7f0e4e038640 1 -- start start 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.292+0000 7f0e4e038640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 0x7f0e480b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.292+0000 7f0e4e038640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e480d2e00 con 0x7f0e480b5ca0 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.292+0000 7f0e4e038640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e480d30e0 con 0x7f0e4815e510 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.292+0000 7f0e4e038640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0e480d33c0 con 0x7f0e480b6100 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e46ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0e480b5ca0 0x7f0e480b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43120/0 (socket says 172.21.15.113:43120) 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e46ffd640 1 -- 172.21.15.113:0/707565553 learned_addr learned my addr 172.21.15.113:0/707565553 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3119102750 0 0) 0x7f0e480d2e00 con 0x7f0e480b5ca0 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e30003520 con 0x7f0e480b5ca0 2024-04-24T23:58:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e47fff640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 0x7f0e480b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1625836617 0 0) 0x7f0e30003520 con 0x7f0e480b5ca0 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 msgr2=0x7f0e480b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 0x7f0e480b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 >> v1:172.21.15.178:6789/0 conn(0x7f0e4815e510 legacy=0x7f0e480d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e480d36a0 con 0x7f0e480b5ca0 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0e3c001670 con 0x7f0e480b5ca0 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 >> v1:172.21.15.113:6789/0 conn(0x7f0e480b5ca0 legacy=0x7f0e480b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e300039e0 0x7f0e30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 0x7f0e30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e30004d40 0x7f0e300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480d33c0 con 0x7f0e30004d40 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3000baf0 con 0x7f0e300039e0 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3000bcf0 con 0x7f0e480b6100 2024-04-24T23:58:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e47fff640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e30004d40 0x7f0e300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e477fe640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e300039e0 0x7f0e30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.293+0000 7f0e46ffd640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 0x7f0e30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e47fff640 1 -- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 msgr2=0x7f0e30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e47fff640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e480b6100 0x7f0e30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e47fff640 1 -- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e300039e0 msgr2=0x7f0e30003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e47fff640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e300039e0 0x7f0e30003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e47fff640 1 -- 172.21.15.113:0/707565553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e30003520 con 0x7f0e30004d40 2024-04-24T23:58:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e47fff640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e30004d40 0x7f0e300051c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009fc0 tx=0x7f0e30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4776a5be60a33d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40009c10 con 0x7f0e30004d40 2024-04-24T23:58:48.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e40036070 con 0x7f0e30004d40 2024-04-24T23:58:48.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.294+0000 7f0e467fc640 1 -- 172.21.15.113:0/707565553 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40031880 con 0x7f0e30004d40 2024-04-24T23:58:48.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.295+0000 7f0e4e038640 1 -- 172.21.15.113:0/707565553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e30004d40 msgr2=0x7f0e300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.295+0000 7f0e4e038640 1 --2- 172.21.15.113:0/707565553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e30004d40 0x7f0e300051c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009fc0 tx=0x7f0e30005830 comp rx=0 tx=0).stop 2024-04-24T23:58:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.295+0000 7f0e47fff640 1 -- 172.21.15.113:0/707565553 reap_dead start 2024-04-24T23:58:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.295+0000 7f0e4e038640 1 -- 172.21.15.113:0/707565553 shutdown_connections 2024-04-24T23:58:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.295+0000 7f0e4e038640 1 -- 172.21.15.113:0/707565553 >> 172.21.15.113:0/707565553 conn(0x7f0e4806b720 msgr2=0x7f0e480d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.295+0000 7f0e4e038640 1 -- 172.21.15.113:0/707565553 shutdown_connections 2024-04-24T23:58:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.295+0000 7f0e4e038640 1 -- 172.21.15.113:0/707565553 wait complete. 2024-04-24T23:58:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.296+0000 7f0e4e038640 1 Processor -- start 2024-04-24T23:58:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.296+0000 7f0e4e038640 1 -- start start 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e4e038640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e30004d40 0x7f0e4814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e4e038640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e4814cab0 0x7f0e4815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e4e038640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e4815d700 0x7f0e480cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e4e038640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480d3ea0 con 0x7f0e4814cab0 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e4e038640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480b6550 con 0x7f0e4815d700 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e4e038640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0e480d2a90 con 0x7f0e30004d40 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e4814cab0 0x7f0e4815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e4814cab0 0x7f0e4815d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59214/0 (socket says 172.21.15.113:59214) 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 -- 172.21.15.113:0/723730598 learned_addr learned my addr 172.21.15.113:0/723730598 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e477fe640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e30004d40 0x7f0e4814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e47fff640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e4815d700 0x7f0e480cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 -- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e30004d40 msgr2=0x7f0e4814b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e30004d40 0x7f0e4814b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 -- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e4815d700 msgr2=0x7f0e480cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e4815d700 0x7f0e480cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 -- 172.21.15.113:0/723730598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e480cbfe0 con 0x7f0e4814cab0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.297+0000 7f0e46ffd640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e4814cab0 0x7f0e4815d1c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0072c0 tx=0x7f0e3c0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.298+0000 7f0e2ffff640 1 -- 172.21.15.113:0/723730598 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00f960 con 0x7f0e4814cab0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.298+0000 7f0e2ffff640 1 -- 172.21.15.113:0/723730598 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3c018070 con 0x7f0e4814cab0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.298+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4814bfd0 con 0x7f0e4814cab0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.298+0000 7f0e2ffff640 1 -- 172.21.15.113:0/723730598 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00fd40 con 0x7f0e4814cab0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.298+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4814bbe0 con 0x7f0e4814cab0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.299+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4815db10 con 0x7f0e4814cab0 2024-04-24T23:58:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.299+0000 7f0e2ffff640 1 -- 172.21.15.113:0/723730598 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e3c01d460 con 0x7f0e4814cab0 2024-04-24T23:58:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.299+0000 7f0e2ffff640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0e3406be30 0x7f0e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.300+0000 7f0e2ffff640 1 -- 172.21.15.113:0/723730598 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0e3c08af10 con 0x7f0e4814cab0 2024-04-24T23:58:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.300+0000 7f0e477fe640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0e3406be30 0x7f0e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.300+0000 7f0e477fe640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0e3406be30 0x7f0e3406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e380017e0 tx=0x7f0e38009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.304+0000 7f0e2ffff640 1 -- 172.21.15.113:0/723730598 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e4815db10 con 0x7f0e4814cab0 2024-04-24T23:58:48.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.478+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0e480691a0 con 0x7f0e4814cab0 2024-04-24T23:58:48.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.479+0000 7f0e2ffff640 1 -- 172.21.15.113:0/723730598 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0e3c0598c0 con 0x7f0e4814cab0 2024-04-24T23:58:48.479 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0e3406be30 msgr2=0x7f0e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0e3406be30 0x7f0e3406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e380017e0 tx=0x7f0e38009040 comp rx=0 tx=0).stop 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e4814cab0 msgr2=0x7f0e4815d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e4814cab0 0x7f0e4815d1c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0072c0 tx=0x7f0e3c0076c0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 shutdown_connections 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0e3406be30 0x7f0e3406e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e4815d700 0x7f0e480cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e4814cab0 0x7f0e4815d1c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 --2- 172.21.15.113:0/723730598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e30004d40 0x7f0e4814b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.480+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 >> 172.21.15.113:0/723730598 conn(0x7f0e4806b720 msgr2=0x7f0e480d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:48.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.481+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 shutdown_connections 2024-04-24T23:58:48.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.481+0000 7f0e4e038640 1 -- 172.21.15.113:0/723730598 wait complete. 2024-04-24T23:58:48.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-24T23:58:48.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.590+0000 7fe2f7dfb640 1 Processor -- start 2024-04-24T23:58:48.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.590+0000 7fe2f7dfb640 1 -- start start 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f7dfb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f7dfb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe2f00b3af0 0x7fe2f00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f7dfb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f00b2d50 con 0x7fe2f015a6b0 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f7dfb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f0072a70 con 0x7fe2f00b3af0 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f7dfb640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f00b3010 con 0x7fe2f0072ea0 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f6398640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe2f00b3af0 0x7fe2f00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f5b97640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe2f015a6b0 0x7fe2f0070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43130/0 (socket says 172.21.15.113:43130) 2024-04-24T23:58:48.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f6398640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe2f00b3af0 0x7fe2f00b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59216/0 (socket says 172.21.15.113:59216) 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f5b97640 1 -- 172.21.15.113:0/3751183916 learned_addr learned my addr 172.21.15.113:0/3751183916 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f5396640 1 --2- 172.21.15.113:0/3751183916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f6398640 1 -- 172.21.15.113:0/3751183916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 msgr2=0x7fe2f0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f6398640 1 --2- 172.21.15.113:0/3751183916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f6398640 1 -- 172.21.15.113:0/3751183916 >> v1:172.21.15.113:6789/0 conn(0x7fe2f015a6b0 legacy=0x7fe2f0070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f6398640 1 -- 172.21.15.113:0/3751183916 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f00b32f0 con 0x7fe2f00b3af0 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f4b95640 1 -- 172.21.15.113:0/3751183916 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2637015327 0 0) 0x7fe2f00b2d50 con 0x7fe2f015a6b0 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.591+0000 7fe2f5396640 1 --2- 172.21.15.113:0/3751183916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.592+0000 7fe2f6398640 1 --2- 172.21.15.113:0/3751183916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f00b3f80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80097a0 tx=0x7fe2e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b2de6301819da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.592+0000 7fe2f4b95640 1 -- 172.21.15.113:0/3751183916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8037070 con 0x7fe2f00b3af0 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.592+0000 7fe2f4b95640 1 -- 172.21.15.113:0/3751183916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e8032440 con 0x7fe2f00b3af0 2024-04-24T23:58:48.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.592+0000 7fe2f4b95640 1 -- 172.21.15.113:0/3751183916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8032810 con 0x7fe2f00b3af0 2024-04-24T23:58:48.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/3751183916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 msgr2=0x7fe2f00b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/3751183916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f00b3f80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80097a0 tx=0x7fe2e802a180 comp rx=0 tx=0).stop 2024-04-24T23:58:48.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/3751183916 shutdown_connections 2024-04-24T23:58:48.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/3751183916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f00b3f80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/3751183916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/3751183916 >> 172.21.15.113:0/3751183916 conn(0x7fe2f00b1a50 msgr2=0x7fe2f00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/3751183916 shutdown_connections 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.593+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/3751183916 wait complete. 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.594+0000 7fe2f7dfb640 1 Processor -- start 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.594+0000 7fe2f7dfb640 1 -- start start 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f0149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f015a6b0 0x7fe2f0149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f015aa90 con 0x7fe2f00b3af0 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f0072720 con 0x7fe2f015a6b0 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f015ac10 con 0x7fe2f0072ea0 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f0149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f0149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59228/0 (socket says 172.21.15.113:59228) 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 -- 172.21.15.113:0/274720335 learned_addr learned my addr 172.21.15.113:0/274720335 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f6398640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f015a6b0 0x7fe2f0149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5b97640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 -- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 msgr2=0x7fe2f0151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 -- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f015a6b0 msgr2=0x7fe2f0149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f015a6b0 0x7fe2f0149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 -- 172.21.15.113:0/274720335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f014d390 con 0x7fe2f00b3af0 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f6398640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f015a6b0 0x7fe2f0149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f5396640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f0149050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c7f0 tx=0x7fe2e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2e67fc640 1 -- 172.21.15.113:0/274720335 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0015070 con 0x7fe2f00b3af0 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2e67fc640 1 -- 172.21.15.113:0/274720335 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0007a30 con 0x7fe2f00b3af0 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f014d670 con 0x7fe2f00b3af0 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2e67fc640 1 -- 172.21.15.113:0/274720335 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0007e00 con 0x7fe2f00b3af0 2024-04-24T23:58:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.595+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f014dc30 con 0x7fe2f00b3af0 2024-04-24T23:58:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.596+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f00691a0 con 0x7fe2f00b3af0 2024-04-24T23:58:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.597+0000 7fe2e67fc640 1 -- 172.21.15.113:0/274720335 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2e0018070 con 0x7fe2f00b3af0 2024-04-24T23:58:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.597+0000 7fe2e67fc640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe2d406be30 0x7fe2d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.597+0000 7fe2e67fc640 1 -- 172.21.15.113:0/274720335 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe2e008ae10 con 0x7fe2f00b3af0 2024-04-24T23:58:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.597+0000 7fe2f5b97640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe2d406be30 0x7fe2d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.597+0000 7fe2f5b97640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe2d406be30 0x7fe2d406e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0012e0 tx=0x7fe2ec005eb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.601+0000 7fe2e67fc640 1 -- 172.21.15.113:0/274720335 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2e00597c0 con 0x7fe2f00b3af0 2024-04-24T23:58:48.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:48.844+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) -- 0x7fe2f00b6030 con 0x7fe2f00b3af0 2024-04-24T23:58:49.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.331+0000 7fe2e67fc640 1 -- 172.21.15.113:0/274720335 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v51) ==== 151+0+0 (secure 0 0 0) 0x7fe2e00590c0 con 0x7fe2f00b3af0 2024-04-24T23:58:49.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe2d406be30 msgr2=0x7fe2d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe2d406be30 0x7fe2d406e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0012e0 tx=0x7fe2ec005eb0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 msgr2=0x7fe2f0149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f0149050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c7f0 tx=0x7fe2e000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 shutdown_connections 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe2d406be30 0x7fe2d406e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f015a6b0 0x7fe2f0149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f00b3af0 0x7fe2f0149050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 --2- 172.21.15.113:0/274720335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f0072ea0 0x7fe2f0151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 >> 172.21.15.113:0/274720335 conn(0x7fe2f00b1a50 msgr2=0x7fe2f01572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:49.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 shutdown_connections 2024-04-24T23:58:49.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.334+0000 7fe2f7dfb640 1 -- 172.21.15.113:0/274720335 wait complete. 2024-04-24T23:58:49.345 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T23:58:49.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.444+0000 7f259b7b7640 1 Processor -- start 2024-04-24T23:58:49.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.444+0000 7f259b7b7640 1 -- start start 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f259b7b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f259415e2d0 0x7f259415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f259b7b7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25940b4a40 0x7f25940ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f259b7b7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259415f4f0 con 0x7f25940b3080 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f259b7b7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f25940cde00 con 0x7f25940b4a40 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f259b7b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f259415f7d0 con 0x7f259415e2d0 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599d54640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25940b4a40 0x7f25940ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599d54640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25940b4a40 0x7f25940ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59230/0 (socket says 172.21.15.113:59230) 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599d54640 1 -- 172.21.15.113:0/1468422839 learned_addr learned my addr 172.21.15.113:0/1468422839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599553640 1 --2- 172.21.15.113:0/1468422839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f259415e2d0 0x7f259415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599d54640 1 -- 172.21.15.113:0/1468422839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f259415e2d0 msgr2=0x7f259415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599d54640 1 --2- 172.21.15.113:0/1468422839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f259415e2d0 0x7f259415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599d54640 1 -- 172.21.15.113:0/1468422839 >> v1:172.21.15.178:6789/0 conn(0x7f25940b3080 legacy=0x7f25940b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.445+0000 7f2599d54640 1 -- 172.21.15.113:0/1468422839 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259415fab0 con 0x7f25940b4a40 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.446+0000 7f2599d54640 1 --2- 172.21.15.113:0/1468422839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25940b4a40 0x7f25940ce290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f258c009e50 tx=0x7f258c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778f84170015b688 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.446+0000 7f258bfff640 1 -- 172.21.15.113:0/1468422839 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c036070 con 0x7f25940b4a40 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.446+0000 7f258bfff640 1 -- 172.21.15.113:0/1468422839 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f258c031600 con 0x7f25940b4a40 2024-04-24T23:58:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.446+0000 7f258bfff640 1 -- 172.21.15.113:0/1468422839 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c0319d0 con 0x7f25940b4a40 2024-04-24T23:58:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.446+0000 7f259b7b7640 1 -- 172.21.15.113:0/1468422839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25940b4a40 msgr2=0x7f25940ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.446+0000 7f259b7b7640 1 --2- 172.21.15.113:0/1468422839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25940b4a40 0x7f25940ce290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f258c009e50 tx=0x7f258c02a710 comp rx=0 tx=0).stop 2024-04-24T23:58:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.447+0000 7f259b7b7640 1 -- 172.21.15.113:0/1468422839 shutdown_connections 2024-04-24T23:58:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.447+0000 7f259b7b7640 1 --2- 172.21.15.113:0/1468422839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25940b4a40 0x7f25940ce290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.447+0000 7f259b7b7640 1 --2- 172.21.15.113:0/1468422839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f259415e2d0 0x7f259415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.447+0000 7f259b7b7640 1 -- 172.21.15.113:0/1468422839 >> 172.21.15.113:0/1468422839 conn(0x7f259406b0f0 msgr2=0x7f259406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:49.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.447+0000 7f259b7b7640 1 -- 172.21.15.113:0/1468422839 shutdown_connections 2024-04-24T23:58:49.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.447+0000 7f259b7b7640 1 -- 172.21.15.113:0/1468422839 wait complete. 2024-04-24T23:58:49.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 Processor -- start 2024-04-24T23:58:49.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 -- start start 2024-04-24T23:58:49.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25940b3080 0x7f25941f0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25940b4a40 0x7f259415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f259415e2d0 0x7f259415c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f259415f2b0 con 0x7f259415e2d0 2024-04-24T23:58:49.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f259415f130 con 0x7f25940b4a40 2024-04-24T23:58:49.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.448+0000 7f259b7b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f259415e6b0 con 0x7f25940b3080 2024-04-24T23:58:49.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f259415e2d0 0x7f259415c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f259415e2d0 0x7f259415c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59244/0 (socket says 172.21.15.113:59244) 2024-04-24T23:58:49.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 -- 172.21.15.113:0/3490810012 learned_addr learned my addr 172.21.15.113:0/3490810012 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:49.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599553640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25940b3080 0x7f25941f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2598d52640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25940b4a40 0x7f259415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 -- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25940b3080 msgr2=0x7f25941f0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25940b3080 0x7f25941f0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 -- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25940b4a40 msgr2=0x7f259415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25940b4a40 0x7f259415bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 -- 172.21.15.113:0/3490810012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25941e61f0 con 0x7f259415e2d0 2024-04-24T23:58:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2599d54640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f259415e2d0 0x7f259415c410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f258c034040 tx=0x7f258c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2589ffb640 1 -- 172.21.15.113:0/3490810012 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c009510 con 0x7f259415e2d0 2024-04-24T23:58:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2589ffb640 1 -- 172.21.15.113:0/3490810012 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f258c007990 con 0x7f259415e2d0 2024-04-24T23:58:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25941e64d0 con 0x7f259415e2d0 2024-04-24T23:58:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f2589ffb640 1 -- 172.21.15.113:0/3490810012 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c007d80 con 0x7f259415e2d0 2024-04-24T23:58:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.449+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2594153df0 con 0x7f259415e2d0 2024-04-24T23:58:49.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.450+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25940691a0 con 0x7f259415e2d0 2024-04-24T23:58:49.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.451+0000 7f2589ffb640 1 -- 172.21.15.113:0/3490810012 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f258c039400 con 0x7f259415e2d0 2024-04-24T23:58:49.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.451+0000 7f2589ffb640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f257806be30 0x7f257806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.451+0000 7f2589ffb640 1 -- 172.21.15.113:0/3490810012 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f258c0adbf0 con 0x7f259415e2d0 2024-04-24T23:58:49.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.452+0000 7f2599553640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f257806be30 0x7f257806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.452+0000 7f2599553640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f257806be30 0x7f257806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2590006fd0 tx=0x7f2590008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:49.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.456+0000 7f2589ffb640 1 -- 172.21.15.113:0/3490810012 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f258c07c5a0 con 0x7f259415e2d0 2024-04-24T23:58:49.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.650+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2594072050 con 0x7f259415e2d0 2024-04-24T23:58:49.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.653+0000 7f2589ffb640 1 -- 172.21.15.113:0/3490810012 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f258c07bea0 con 0x7f259415e2d0 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f257806be30 msgr2=0x7f257806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f257806be30 0x7f257806e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2590006fd0 tx=0x7f2590008040 comp rx=0 tx=0).stop 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f259415e2d0 msgr2=0x7f259415c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f259415e2d0 0x7f259415c410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f258c034040 tx=0x7f258c004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 shutdown_connections 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f257806be30 0x7f257806e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25940b4a40 0x7f259415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f259415e2d0 0x7f259415c410 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 --2- 172.21.15.113:0/3490810012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25940b3080 0x7f25941f0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 >> 172.21.15.113:0/3490810012 conn(0x7f259406b0f0 msgr2=0x7f259415d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 shutdown_connections 2024-04-24T23:58:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:49.655+0000 7f259b7b7640 1 -- 172.21.15.113:0/3490810012 wait complete. 2024-04-24T23:58:49.666 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T23:58:49.666 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:58:49.702 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T23:58:49.742 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T23:58:49.743 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:49.743 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:49.758 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:49.758 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:49.814 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:49.814 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:49.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:50.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.005+0000 7f8e87a58640 1 Processor -- start 2024-04-24T23:58:50.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e87a58640 1 -- start start 2024-04-24T23:58:50.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e87a58640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8e800d2290 0x7f8e800b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e87a58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 0x7f8e800d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e87a58640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e800d2800 con 0x7f8e8015e320 2024-04-24T23:58:50.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e87a58640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8e800d14d0 con 0x7f8e800d2290 2024-04-24T23:58:50.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e87a58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e800d3410 con 0x7f8e800d0b00 2024-04-24T23:58:50.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e857f4640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8e8015e320 0x7f8e800d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43136/0 (socket says 172.21.15.113:43136) 2024-04-24T23:58:50.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.006+0000 7f8e857f4640 1 -- 172.21.15.113:0/1254707093 learned_addr learned my addr 172.21.15.113:0/1254707093 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e85ff5640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 0x7f8e800d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e84ff3640 1 --2- 172.21.15.113:0/1254707093 >> v2:172.21.15.178:3300/0 conn(0x7f8e800d2290 0x7f8e800b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2856299037 0 0) 0x7f8e800d2800 con 0x7f8e8015e320 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e6c003520 con 0x7f8e8015e320 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3428992574 0 0) 0x7f8e6c003520 con 0x7f8e8015e320 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 msgr2=0x7f8e800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 0x7f8e800d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 >> v2:172.21.15.178:3300/0 conn(0x7f8e800d2290 msgr2=0x7f8e800b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 --2- 172.21.15.113:0/1254707093 >> v2:172.21.15.178:3300/0 conn(0x7f8e800d2290 0x7f8e800b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e800d2ae0 con 0x7f8e8015e320 2024-04-24T23:58:50.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8e7c001690 con 0x7f8e8015e320 2024-04-24T23:58:50.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 >> v1:172.21.15.113:6789/0 conn(0x7f8e8015e320 legacy=0x7f8e800d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e6c004800 0x7f8e6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 0x7f8e6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800d14d0 con 0x7f8e6c0039b0 2024-04-24T23:58:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800d3410 con 0x7f8e6c004800 2024-04-24T23:58:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c00b9c0 con 0x7f8e800d0b00 2024-04-24T23:58:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e857f4640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e84ff3640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e6c004800 0x7f8e6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e85ff5640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 0x7f8e6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e857f4640 1 -- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 msgr2=0x7f8e6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e857f4640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e800d0b00 0x7f8e6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e857f4640 1 -- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e6c004800 msgr2=0x7f8e6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e857f4640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e6c004800 0x7f8e6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.007+0000 7f8e857f4640 1 -- 172.21.15.113:0/1254707093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c003520 con 0x7f8e6c0039b0 2024-04-24T23:58:50.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.008+0000 7f8e857f4640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e800b39a0 tx=0x7f8e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89da09f5ec221c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.008+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c02e070 con 0x7f8e6c0039b0 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.008+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c005ad0 con 0x7f8e6c0039b0 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.008+0000 7f8e77fff640 1 -- 172.21.15.113:0/1254707093 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c007d90 con 0x7f8e6c0039b0 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.009+0000 7f8e87a58640 1 -- 172.21.15.113:0/1254707093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 msgr2=0x7f8e6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.009+0000 7f8e87a58640 1 --2- 172.21.15.113:0/1254707093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e6c003dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e800b39a0 tx=0x7f8e6c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.009+0000 7f8e85ff5640 1 -- 172.21.15.113:0/1254707093 reap_dead start 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.009+0000 7f8e87a58640 1 -- 172.21.15.113:0/1254707093 shutdown_connections 2024-04-24T23:58:50.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.009+0000 7f8e87a58640 1 -- 172.21.15.113:0/1254707093 >> 172.21.15.113:0/1254707093 conn(0x7f8e800b5c20 msgr2=0x7f8e800b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.009+0000 7f8e87a58640 1 -- 172.21.15.113:0/1254707093 shutdown_connections 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.009+0000 7f8e87a58640 1 -- 172.21.15.113:0/1254707093 wait complete. 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.010+0000 7f8e87a58640 1 Processor -- start 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.010+0000 7f8e87a58640 1 -- start start 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e87a58640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e801f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e87a58640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e80161430 0x7f8e8015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e87a58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e80152a60 0x7f8e800cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e87a58640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800b9fe0 con 0x7f8e6c0039b0 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e87a58640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e800d30a0 con 0x7f8e80161430 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e87a58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8015e700 con 0x7f8e80152a60 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e801f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e801f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59268/0 (socket says 172.21.15.113:59268) 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 -- 172.21.15.113:0/1001179697 learned_addr learned my addr 172.21.15.113:0/1001179697 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 -- 172.21.15.113:0/1001179697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e80152a60 msgr2=0x7f8e800cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e80152a60 0x7f8e800cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 -- 172.21.15.113:0/1001179697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e80161430 msgr2=0x7f8e8015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e80161430 0x7f8e8015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 -- 172.21.15.113:0/1001179697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e80160390 con 0x7f8e6c0039b0 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e857f4640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e801f32f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02ae30 tx=0x7f8e7c001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e75ffb640 1 -- 172.21.15.113:0/1001179697 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c042020 con 0x7f8e6c0039b0 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e75ffb640 1 -- 172.21.15.113:0/1001179697 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0074a0 con 0x7f8e6c0039b0 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e80160670 con 0x7f8e6c0039b0 2024-04-24T23:58:50.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.011+0000 7f8e75ffb640 1 -- 172.21.15.113:0/1001179697 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c007d70 con 0x7f8e6c0039b0 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.012+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e80160f10 con 0x7f8e6c0039b0 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.013+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e800b7020 con 0x7f8e6c0039b0 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.013+0000 7f8e75ffb640 1 -- 172.21.15.113:0/1001179697 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e7c045020 con 0x7f8e6c0039b0 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.013+0000 7f8e75ffb640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6406be60 0x7f8e6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.013+0000 7f8e75ffb640 1 -- 172.21.15.113:0/1001179697 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8e7c0ab810 con 0x7f8e6c0039b0 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.013+0000 7f8e84ff3640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6406be60 0x7f8e6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.014+0000 7f8e84ff3640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6406be60 0x7f8e6406e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006b10 tx=0x7f8e70006aa0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.018+0000 7f8e75ffb640 1 -- 172.21.15.113:0/1001179697 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e800b7020 con 0x7f8e6c0039b0 2024-04-24T23:58:50.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.191+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e800691a0 con 0x7f8e6c0039b0 2024-04-24T23:58:50.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.192+0000 7f8e75ffb640 1 -- 172.21.15.113:0/1001179697 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f8e7c03b360 con 0x7f8e6c0039b0 2024-04-24T23:58:50.192 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:50.192 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6406be60 msgr2=0x7f8e6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6406be60 0x7f8e6406e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006b10 tx=0x7f8e70006aa0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 msgr2=0x7f8e801f32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e801f32f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02ae30 tx=0x7f8e7c001a60 comp rx=0 tx=0).stop 2024-04-24T23:58:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 shutdown_connections 2024-04-24T23:58:50.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6406be60 0x7f8e6406e320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e80161430 0x7f8e8015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e80152a60 0x7f8e800cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 --2- 172.21.15.113:0/1001179697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e6c0039b0 0x7f8e801f32f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 >> 172.21.15.113:0/1001179697 conn(0x7f8e800b5c20 msgr2=0x7f8e800b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:50.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 shutdown_connections 2024-04-24T23:58:50.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.194+0000 7f8e87a58640 1 -- 172.21.15.113:0/1001179697 wait complete. 2024-04-24T23:58:50.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:50.205 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.306+0000 7f812d14f640 1 Processor -- start 2024-04-24T23:58:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.306+0000 7f812d14f640 1 -- start start 2024-04-24T23:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.306+0000 7f812d14f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f812815e2d0 0x7f812815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.306+0000 7f812d14f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 0x7f81280b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812d14f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f812815f4f0 con 0x7f812815e6b0 2024-04-24T23:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812d14f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f81280cde00 con 0x7f812815e2d0 2024-04-24T23:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812d14f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f812815f7d0 con 0x7f81280b3080 2024-04-24T23:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812759e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f812815e6b0 0x7f8128071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43152/0 (socket says 172.21.15.113:43152) 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812759e640 1 -- 172.21.15.113:0/2431713775 learned_addr learned my addr 172.21.15.113:0/2431713775 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2821985460 0 0) 0x7f812815f4f0 con 0x7f812815e6b0 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8110003520 con 0x7f812815e6b0 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8126d9d640 1 --2- 172.21.15.113:0/2431713775 >> v2:172.21.15.178:3300/0 conn(0x7f812815e2d0 0x7f812815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812659c640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 0x7f81280b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1649562164 0 0) 0x7f8110003520 con 0x7f812815e6b0 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 msgr2=0x7f81280b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 0x7f81280b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 >> v2:172.21.15.178:3300/0 conn(0x7f812815e2d0 msgr2=0x7f812815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 --2- 172.21.15.113:0/2431713775 >> v2:172.21.15.178:3300/0 conn(0x7f812815e2d0 0x7f812815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812815fab0 con 0x7f812815e6b0 2024-04-24T23:58:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f81200019f0 con 0x7f812815e6b0 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 >> v1:172.21.15.113:6789/0 conn(0x7f812815e6b0 legacy=0x7f8128071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81100039b0 0x7f8110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 0x7f8110004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81280cde00 con 0x7f8110004d40 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f812815f7d0 con 0x7f81100039b0 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f811000b9c0 con 0x7f81280b3080 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812759e640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f8126d9d640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81100039b0 0x7f8110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.307+0000 7f812659c640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 0x7f8110004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.308+0000 7f812759e640 1 -- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 msgr2=0x7f8110004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.308+0000 7f812759e640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280b3080 0x7f8110004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.308+0000 7f812759e640 1 -- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81100039b0 msgr2=0x7f8110003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.308+0000 7f812759e640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81100039b0 0x7f8110003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.308+0000 7f812759e640 1 -- 172.21.15.113:0/2431713775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8110003520 con 0x7f8110004d40 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.308+0000 7f812759e640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81100051c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f812000a870 tx=0x7f8110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a2716a8f00fa3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.308+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812000a4c0 con 0x7f8110004d40 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812002f070 con 0x7f8110004d40 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f8125d9b640 1 -- 172.21.15.113:0/2431713775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812002bcc0 con 0x7f8110004d40 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f812d14f640 1 -- 172.21.15.113:0/2431713775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 msgr2=0x7f81100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f812d14f640 1 --2- 172.21.15.113:0/2431713775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81100051c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f812000a870 tx=0x7f8110005830 comp rx=0 tx=0).stop 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f812759e640 1 -- 172.21.15.113:0/2431713775 reap_dead start 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f812d14f640 1 -- 172.21.15.113:0/2431713775 shutdown_connections 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f812d14f640 1 -- 172.21.15.113:0/2431713775 >> 172.21.15.113:0/2431713775 conn(0x7f812806b0f0 msgr2=0x7f812806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f812d14f640 1 -- 172.21.15.113:0/2431713775 shutdown_connections 2024-04-24T23:58:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.309+0000 7f812d14f640 1 -- 172.21.15.113:0/2431713775 wait complete. 2024-04-24T23:58:50.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.310+0000 7f812d14f640 1 Processor -- start 2024-04-24T23:58:50.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812d14f640 1 -- start start 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812d14f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81281f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812d14f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81281f70a0 0x7f81280ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812d14f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280aecf0 0x7f81280b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812d14f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f812815f2b0 con 0x7f8110004d40 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812d14f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f812815f130 con 0x7f81281f70a0 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812d14f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81280cf2a0 con 0x7f81280aecf0 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f8126d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81281f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f8126d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81281f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59290/0 (socket says 172.21.15.113:59290) 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f8126d9d640 1 -- 172.21.15.113:0/3054911993 learned_addr learned my addr 172.21.15.113:0/3054911993 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812759e640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280aecf0 0x7f81280b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.311+0000 7f812659c640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81281f70a0 0x7f81280ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f8126d9d640 1 -- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280aecf0 msgr2=0x7f81280b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f8126d9d640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280aecf0 0x7f81280b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f8126d9d640 1 -- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81281f70a0 msgr2=0x7f81280ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f8126d9d640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81281f70a0 0x7f81280ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f8126d9d640 1 -- 172.21.15.113:0/3054911993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81281f7520 con 0x7f8110004d40 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f812659c640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81281f70a0 0x7f81280ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f8126d9d640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81281f41d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8118011990 tx=0x7f8118011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f810f7fe640 1 -- 172.21.15.113:0/3054911993 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800fa70 con 0x7f8110004d40 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f810f7fe640 1 -- 172.21.15.113:0/3054911993 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811800fc10 con 0x7f8110004d40 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f810f7fe640 1 -- 172.21.15.113:0/3054911993 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118014440 con 0x7f8110004d40 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8128151500 con 0x7f8110004d40 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.312+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8128151ac0 con 0x7f8110004d40 2024-04-24T23:58:50.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.313+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81280691a0 con 0x7f8110004d40 2024-04-24T23:58:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.313+0000 7f810f7fe640 1 -- 172.21.15.113:0/3054911993 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8118012040 con 0x7f8110004d40 2024-04-24T23:58:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.314+0000 7f810f7fe640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811406be30 0x7f811406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.314+0000 7f812659c640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811406be30 0x7f811406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.314+0000 7f810f7fe640 1 -- 172.21.15.113:0/3054911993 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f811808b430 con 0x7f8110004d40 2024-04-24T23:58:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.314+0000 7f812659c640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811406be30 0x7f811406e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f81280b0290 tx=0x7f811c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.318+0000 7f810f7fe640 1 -- 172.21.15.113:0/3054911993 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8118019030 con 0x7f8110004d40 2024-04-24T23:58:50.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.492+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8128072050 con 0x7f8110004d40 2024-04-24T23:58:50.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.492+0000 7f810f7fe640 1 -- 172.21.15.113:0/3054911993 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8118059e70 con 0x7f8110004d40 2024-04-24T23:58:50.493 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811406be30 msgr2=0x7f811406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811406be30 0x7f811406e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f81280b0290 tx=0x7f811c008040 comp rx=0 tx=0).stop 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 msgr2=0x7f81281f41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81281f41d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8118011990 tx=0x7f8118011e50 comp rx=0 tx=0).stop 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 shutdown_connections 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811406be30 0x7f811406e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81281f70a0 0x7f81280ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81280aecf0 0x7f81280b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 --2- 172.21.15.113:0/3054911993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8110004d40 0x7f81281f41d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 >> 172.21.15.113:0/3054911993 conn(0x7f812806b0f0 msgr2=0x7f81280b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:50.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 shutdown_connections 2024-04-24T23:58:50.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.494+0000 7f812d14f640 1 -- 172.21.15.113:0/3054911993 wait complete. 2024-04-24T23:58:50.505 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:50.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.604+0000 7fa8218bd640 1 Processor -- start 2024-04-24T23:58:50.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.604+0000 7fa8218bd640 1 -- start start 2024-04-24T23:58:50.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa8218bd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa81c15e840 0x7fa81c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa8218bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c0d23c0 0x7fa81c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa8218bd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81c06b420 con 0x7fa81c147640 2024-04-24T23:58:50.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa8218bd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa81c15ee60 con 0x7fa81c15e840 2024-04-24T23:58:50.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa8218bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c15f140 con 0x7fa81c0d23c0 2024-04-24T23:58:50.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa81a7fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa81c15e840 0x7fa81c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c0d23c0 0x7fa81c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa81affd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa81c147640 0x7fa81c06fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:42436/0 (socket says 172.21.15.113:42436) 2024-04-24T23:58:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.605+0000 7fa81affd640 1 -- 172.21.15.113:0/3504569518 learned_addr learned my addr 172.21.15.113:0/3504569518 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.606+0000 7fa81a7fc640 1 -- 172.21.15.113:0/3504569518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c0d23c0 msgr2=0x7fa81c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.606+0000 7fa81a7fc640 1 --2- 172.21.15.113:0/3504569518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c0d23c0 0x7fa81c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.606+0000 7fa81a7fc640 1 -- 172.21.15.113:0/3504569518 >> v1:172.21.15.178:6789/0 conn(0x7fa81c147640 legacy=0x7fa81c06fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.606+0000 7fa819ffb640 1 -- 172.21.15.113:0/3504569518 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126370170 0 0) 0x7fa81c06b420 con 0x7fa81c147640 2024-04-24T23:58:50.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.606+0000 7fa81a7fc640 1 -- 172.21.15.113:0/3504569518 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c15f420 con 0x7fa81c15e840 2024-04-24T23:58:50.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.606+0000 7fa81a7fc640 1 --2- 172.21.15.113:0/3504569518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c15e840 0x7fa81c0d1e80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa8100097a0 tx=0x7fa81002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec4d995a03f0893 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.606+0000 7fa819ffb640 1 -- 172.21.15.113:0/3504569518 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810037070 con 0x7fa81c15e840 2024-04-24T23:58:50.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa819ffb640 1 -- 172.21.15.113:0/3504569518 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa810032440 con 0x7fa81c15e840 2024-04-24T23:58:50.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa819ffb640 1 -- 172.21.15.113:0/3504569518 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810032810 con 0x7fa81c15e840 2024-04-24T23:58:50.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 -- 172.21.15.113:0/3504569518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c15e840 msgr2=0x7fa81c0d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 --2- 172.21.15.113:0/3504569518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c15e840 0x7fa81c0d1e80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa8100097a0 tx=0x7fa81002a180 comp rx=0 tx=0).stop 2024-04-24T23:58:50.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 -- 172.21.15.113:0/3504569518 shutdown_connections 2024-04-24T23:58:50.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 --2- 172.21.15.113:0/3504569518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c0d23c0 0x7fa81c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 --2- 172.21.15.113:0/3504569518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c15e840 0x7fa81c0d1e80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 -- 172.21.15.113:0/3504569518 >> 172.21.15.113:0/3504569518 conn(0x7fa81c0b7370 msgr2=0x7fa81c0b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:50.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 -- 172.21.15.113:0/3504569518 shutdown_connections 2024-04-24T23:58:50.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.607+0000 7fa8218bd640 1 -- 172.21.15.113:0/3504569518 wait complete. 2024-04-24T23:58:50.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.608+0000 7fa8218bd640 1 Processor -- start 2024-04-24T23:58:50.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.608+0000 7fa8218bd640 1 -- start start 2024-04-24T23:58:50.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa8218bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c0d23c0 0x7fa81c13acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa8218bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c147640 0x7fa81c13d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa8218bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81c15e840 0x7fa81c13d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa8218bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c15fbc0 con 0x7fa81c15e840 2024-04-24T23:58:50.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa8218bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c15fa40 con 0x7fa81c0d23c0 2024-04-24T23:58:50.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa8218bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c06be40 con 0x7fa81c147640 2024-04-24T23:58:50.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81c15e840 0x7fa81c13d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81c15e840 0x7fa81c13d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59292/0 (socket says 172.21.15.113:59292) 2024-04-24T23:58:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa81b7fe640 1 -- 172.21.15.113:0/1152052683 learned_addr learned my addr 172.21.15.113:0/1152052683 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa81a7fc640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c147640 0x7fa81c13d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.609+0000 7fa81affd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c0d23c0 0x7fa81c13acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa81b7fe640 1 -- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c147640 msgr2=0x7fa81c13d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa81b7fe640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c147640 0x7fa81c13d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa81b7fe640 1 -- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c0d23c0 msgr2=0x7fa81c13acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa81b7fe640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c0d23c0 0x7fa81c13acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa81b7fe640 1 -- 172.21.15.113:0/1152052683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c1e89b0 con 0x7fa81c15e840 2024-04-24T23:58:50.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa81affd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c0d23c0 0x7fa81c13acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:50.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa81b7fe640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81c15e840 0x7fa81c13d860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e930 tx=0x7fa81400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa803fff640 1 -- 172.21.15.113:0/1152052683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814017070 con 0x7fa81c15e840 2024-04-24T23:58:50.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa803fff640 1 -- 172.21.15.113:0/1152052683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81400c980 con 0x7fa81c15e840 2024-04-24T23:58:50.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa803fff640 1 -- 172.21.15.113:0/1152052683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81400cd70 con 0x7fa81c15e840 2024-04-24T23:58:50.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c1e8c90 con 0x7fa81c15e840 2024-04-24T23:58:50.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.610+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c0b31d0 con 0x7fa81c15e840 2024-04-24T23:58:50.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.611+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c0691a0 con 0x7fa81c15e840 2024-04-24T23:58:50.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.611+0000 7fa803fff640 1 -- 172.21.15.113:0/1152052683 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa81401a070 con 0x7fa81c15e840 2024-04-24T23:58:50.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.612+0000 7fa803fff640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80806be30 0x7fa80806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.612+0000 7fa803fff640 1 -- 172.21.15.113:0/1152052683 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa81408b160 con 0x7fa81c15e840 2024-04-24T23:58:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.612+0000 7fa81affd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80806be30 0x7fa80806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.612+0000 7fa81affd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80806be30 0x7fa80806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0012e0 tx=0x7fa80c007f90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.616+0000 7fa803fff640 1 -- 172.21.15.113:0/1152052683 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa814059580 con 0x7fa81c15e840 2024-04-24T23:58:50.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.789+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa81c0b7c20 con 0x7fa81c15e840 2024-04-24T23:58:50.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.789+0000 7fa803fff640 1 -- 172.21.15.113:0/1152052683 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa81405d880 con 0x7fa81c15e840 2024-04-24T23:58:50.789 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:50.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80806be30 msgr2=0x7fa80806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80806be30 0x7fa80806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0012e0 tx=0x7fa80c007f90 comp rx=0 tx=0).stop 2024-04-24T23:58:50.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81c15e840 msgr2=0x7fa81c13d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81c15e840 0x7fa81c13d860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e930 tx=0x7fa81400edf0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 shutdown_connections 2024-04-24T23:58:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80806be30 0x7fa80806e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81c15e840 0x7fa81c13d860 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa81c147640 0x7fa81c13d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 --2- 172.21.15.113:0/1152052683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa81c0d23c0 0x7fa81c13acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 >> 172.21.15.113:0/1152052683 conn(0x7fa81c0b7370 msgr2=0x7fa81c13e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 shutdown_connections 2024-04-24T23:58:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.791+0000 7fa8218bd640 1 -- 172.21.15.113:0/1152052683 wait complete. 2024-04-24T23:58:50.802 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.901+0000 7fc1aea15640 1 Processor -- start 2024-04-24T23:58:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.901+0000 7fc1aea15640 1 -- start start 2024-04-24T23:58:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.902+0000 7fc1aea15640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 0x7fc1a80d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.902+0000 7fc1aea15640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a806b420 con 0x7fc1a8147640 2024-04-24T23:58:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.902+0000 7fc1aea15640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a815ee60 con 0x7fc1a80d23c0 2024-04-24T23:58:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.902+0000 7fc1aea15640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a815f140 con 0x7fc1a815e840 2024-04-24T23:58:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.902+0000 7fc1a7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc1a8147640 0x7fc1a806fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43164/0 (socket says 172.21.15.113:43164) 2024-04-24T23:58:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.902+0000 7fc1a7fff640 1 -- 172.21.15.113:0/2193988684 learned_addr learned my addr 172.21.15.113:0/2193988684 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 0x7fc1a80d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 703726485 0 0) 0x7fc1a806b420 con 0x7fc1a8147640 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc190003520 con 0x7fc1a8147640 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849950344 0 0) 0x7fc1a815ee60 con 0x7fc1a80d23c0 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1a806b420 con 0x7fc1a80d23c0 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3254304380 0 0) 0x7fc190003520 con 0x7fc1a8147640 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 msgr2=0x7fc1a80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 0x7fc1a80d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 >> v1:172.21.15.178:6789/0 conn(0x7fc1a80d23c0 legacy=0x7fc1a806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a815f420 con 0x7fc1a8147640 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fc198001670 con 0x7fc1a8147640 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 >> v1:172.21.15.113:6789/0 conn(0x7fc1a8147640 legacy=0x7fc1a806fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 0x7fc190003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1900040d0 0x7fc190004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc190004d40 0x7fc1900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a815f140 con 0x7fc1900040d0 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc19000baa0 con 0x7fc190004d40 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc19000bca0 con 0x7fc1a815e840 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1900040d0 0x7fc190004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a7fff640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 0x7fc190003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1acfb2640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc190004d40 0x7fc1900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a77fe640 1 -- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 msgr2=0x7fc190003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1a815e840 0x7fc190003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a77fe640 1 -- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc190004d40 msgr2=0x7fc1900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc190004d40 0x7fc1900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.903+0000 7fc1a77fe640 1 -- 172.21.15.113:0/2193988684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc190003520 con 0x7fc1900040d0 2024-04-24T23:58:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.904+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1900040d0 0x7fc190004800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009aa0 tx=0x7fc190005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66911227e276a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.904+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c036070 con 0x7fc1900040d0 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.904+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c031400 con 0x7fc1900040d0 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.904+0000 7fc1a6ffd640 1 -- 172.21.15.113:0/2193988684 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c009670 con 0x7fc1900040d0 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.905+0000 7fc1aea15640 1 -- 172.21.15.113:0/2193988684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1900040d0 msgr2=0x7fc190004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.905+0000 7fc1aea15640 1 --2- 172.21.15.113:0/2193988684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1900040d0 0x7fc190004800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009aa0 tx=0x7fc190005830 comp rx=0 tx=0).stop 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.905+0000 7fc1acfb2640 1 -- 172.21.15.113:0/2193988684 reap_dead start 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.905+0000 7fc1aea15640 1 -- 172.21.15.113:0/2193988684 shutdown_connections 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.905+0000 7fc1aea15640 1 -- 172.21.15.113:0/2193988684 >> 172.21.15.113:0/2193988684 conn(0x7fc1a80b7370 msgr2=0x7fc1a80b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.905+0000 7fc1aea15640 1 -- 172.21.15.113:0/2193988684 shutdown_connections 2024-04-24T23:58:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.905+0000 7fc1aea15640 1 -- 172.21.15.113:0/2193988684 wait complete. 2024-04-24T23:58:50.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.906+0000 7fc1aea15640 1 Processor -- start 2024-04-24T23:58:50.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.906+0000 7fc1aea15640 1 -- start start 2024-04-24T23:58:50.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1aea15640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1900040d0 0x7fc1a813eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1aea15640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1a80cdf50 0x7fc1a80ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1aea15640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1a80d0120 0x7fc1a80d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1aea15640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a815fbc0 con 0x7fc1a80cdf50 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1aea15640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a815fa40 con 0x7fc1a80d0120 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1aea15640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a80d1e80 con 0x7fc1900040d0 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1a80cdf50 0x7fc1a80ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1a80cdf50 0x7fc1a80ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59314/0 (socket says 172.21.15.113:59314) 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 -- 172.21.15.113:0/2575393338 learned_addr learned my addr 172.21.15.113:0/2575393338 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1acfb2640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1a80d0120 0x7fc1a80d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a7fff640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1900040d0 0x7fc1a813eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 -- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1900040d0 msgr2=0x7fc1a813eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1900040d0 0x7fc1a813eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 -- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1a80d0120 msgr2=0x7fc1a80d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1a80d0120 0x7fc1a80d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 -- 172.21.15.113:0/2575393338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a80d1a60 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a7fff640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1900040d0 0x7fc1a813eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1acfb2640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1a80d0120 0x7fc1a80d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a77fe640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1a80cdf50 0x7fc1a80ce3d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc19c034040 tx=0x7fc19c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a4ff9640 1 -- 172.21.15.113:0/2575393338 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c041070 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a4ff9640 1 -- 172.21.15.113:0/2575393338 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c03d660 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a80d0ee0 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.907+0000 7fc1a4ff9640 1 -- 172.21.15.113:0/2575393338 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c036040 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.908+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a80d14a0 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.909+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a80691a0 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.909+0000 7fc1a4ff9640 1 -- 172.21.15.113:0/2575393338 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc19c044070 con 0x7fc1a80cdf50 2024-04-24T23:58:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.909+0000 7fc1a4ff9640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc19406be30 0x7fc19406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.910+0000 7fc1a4ff9640 1 -- 172.21.15.113:0/2575393338 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc19c0ab5f0 con 0x7fc1a80cdf50 2024-04-24T23:58:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.910+0000 7fc1a7fff640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc19406be30 0x7fc19406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.910+0000 7fc1a7fff640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc19406be30 0x7fc19406e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc198001050 tx=0x7fc198009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:50.914+0000 7fc1a4ff9640 1 -- 172.21.15.113:0/2575393338 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc19c0799e0 con 0x7fc1a80cdf50 2024-04-24T23:58:51.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.085+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1a80b2850 con 0x7fc1a80cdf50 2024-04-24T23:58:51.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.086+0000 7fc1a4ff9640 1 -- 172.21.15.113:0/2575393338 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc19c07dce0 con 0x7fc1a80cdf50 2024-04-24T23:58:51.086 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:51.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc19406be30 msgr2=0x7fc19406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc19406be30 0x7fc19406e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc198001050 tx=0x7fc198009040 comp rx=0 tx=0).stop 2024-04-24T23:58:51.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1a80cdf50 msgr2=0x7fc1a80ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1a80cdf50 0x7fc1a80ce3d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc19c034040 tx=0x7fc19c004730 comp rx=0 tx=0).stop 2024-04-24T23:58:51.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 shutdown_connections 2024-04-24T23:58:51.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc19406be30 0x7fc19406e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1a80d0120 0x7fc1a80d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1a80cdf50 0x7fc1a80ce3d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 --2- 172.21.15.113:0/2575393338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1900040d0 0x7fc1a813eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 >> 172.21.15.113:0/2575393338 conn(0x7fc1a80b7370 msgr2=0x7fc1a80b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:51.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 shutdown_connections 2024-04-24T23:58:51.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.088+0000 7fc1aea15640 1 -- 172.21.15.113:0/2575393338 wait complete. 2024-04-24T23:58:51.099 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:58:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.197+0000 7f9e77eda640 1 Processor -- start 2024-04-24T23:58:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.197+0000 7f9e77eda640 1 -- start start 2024-04-24T23:58:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e77eda640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9e7015e840 0x7f9e700d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e77eda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e700d23c0 0x7f9e7006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e77eda640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7006b420 con 0x7f9e70147640 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e77eda640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7015ee60 con 0x7f9e7015e840 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e77eda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7015f140 con 0x7f9e700d23c0 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e75475640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9e7015e840 0x7f9e700d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e75c76640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9e70147640 0x7f9e7006fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43172/0 (socket says 172.21.15.113:43172) 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.198+0000 7f9e75c76640 1 -- 172.21.15.113:0/1875386863 learned_addr learned my addr 172.21.15.113:0/1875386863 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e76477640 1 --2- 172.21.15.113:0/1875386863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e700d23c0 0x7f9e7006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e75475640 1 -- 172.21.15.113:0/1875386863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e700d23c0 msgr2=0x7f9e7006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e75475640 1 --2- 172.21.15.113:0/1875386863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e700d23c0 0x7f9e7006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e75475640 1 -- 172.21.15.113:0/1875386863 >> v1:172.21.15.113:6789/0 conn(0x7f9e70147640 legacy=0x7f9e7006fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e75475640 1 -- 172.21.15.113:0/1875386863 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7015f420 con 0x7f9e7015e840 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e74c74640 1 -- 172.21.15.113:0/1875386863 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029186382 0 0) 0x7f9e7006b420 con 0x7f9e70147640 2024-04-24T23:58:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e75475640 1 --2- 172.21.15.113:0/1875386863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e700d1e80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9e600097a0 tx=0x7f9e6002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d5d82bbd705f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:51.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e74c74640 1 -- 172.21.15.113:0/1875386863 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60037070 con 0x7f9e7015e840 2024-04-24T23:58:51.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e74c74640 1 -- 172.21.15.113:0/1875386863 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e60032440 con 0x7f9e7015e840 2024-04-24T23:58:51.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.199+0000 7f9e74c74640 1 -- 172.21.15.113:0/1875386863 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60032810 con 0x7f9e7015e840 2024-04-24T23:58:51.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 -- 172.21.15.113:0/1875386863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 msgr2=0x7f9e700d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 --2- 172.21.15.113:0/1875386863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e700d1e80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9e600097a0 tx=0x7f9e6002a180 comp rx=0 tx=0).stop 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 -- 172.21.15.113:0/1875386863 shutdown_connections 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 --2- 172.21.15.113:0/1875386863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e700d23c0 0x7f9e7006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 --2- 172.21.15.113:0/1875386863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e700d1e80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 -- 172.21.15.113:0/1875386863 >> 172.21.15.113:0/1875386863 conn(0x7f9e700b7370 msgr2=0x7f9e700b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 -- 172.21.15.113:0/1875386863 shutdown_connections 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.200+0000 7f9e77eda640 1 -- 172.21.15.113:0/1875386863 wait complete. 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.201+0000 7f9e77eda640 1 Processor -- start 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.201+0000 7f9e77eda640 1 -- start start 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e77eda640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e700d23c0 0x7f9e70144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e77eda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e70147640 0x7f9e70137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e77eda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e7015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e77eda640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7015fbc0 con 0x7f9e7015e840 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e77eda640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7015fa40 con 0x7f9e700d23c0 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e77eda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7006be40 con 0x7f9e70147640 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e7015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e7015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59336/0 (socket says 172.21.15.113:59336) 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 -- 172.21.15.113:0/3319556812 learned_addr learned my addr 172.21.15.113:0/3319556812 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e75475640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e70147640 0x7f9e70137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e75c76640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e700d23c0 0x7f9e70144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 -- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e70147640 msgr2=0x7f9e70137890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e70147640 0x7f9e70137890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 -- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e700d23c0 msgr2=0x7f9e70144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e700d23c0 0x7f9e70144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 -- 172.21.15.113:0/3319556812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e70140530 con 0x7f9e7015e840 2024-04-24T23:58:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.202+0000 7f9e76477640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e7015b380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e68007e40 tx=0x7f9e6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.203+0000 7f9e667fc640 1 -- 172.21.15.113:0/3319556812 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68014620 con 0x7f9e7015e840 2024-04-24T23:58:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.203+0000 7f9e667fc640 1 -- 172.21.15.113:0/3319556812 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e68019070 con 0x7f9e7015e840 2024-04-24T23:58:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.203+0000 7f9e667fc640 1 -- 172.21.15.113:0/3319556812 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68014a20 con 0x7f9e7015e840 2024-04-24T23:58:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.203+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e70140810 con 0x7f9e7015e840 2024-04-24T23:58:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.203+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e70140dd0 con 0x7f9e7015e840 2024-04-24T23:58:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.204+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7006de50 con 0x7f9e7015e840 2024-04-24T23:58:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.204+0000 7f9e667fc640 1 -- 172.21.15.113:0/3319556812 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e68014bc0 con 0x7f9e7015e840 2024-04-24T23:58:51.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.204+0000 7f9e667fc640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e5406be30 0x7f9e5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.204+0000 7f9e667fc640 1 -- 172.21.15.113:0/3319556812 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e6808b620 con 0x7f9e7015e840 2024-04-24T23:58:51.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.204+0000 7f9e75c76640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e5406be30 0x7f9e5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.205+0000 7f9e75c76640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e5406be30 0x7f9e5406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c000f60 tx=0x7f9e6c007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:51.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.208+0000 7f9e667fc640 1 -- 172.21.15.113:0/3319556812 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e7006de50 con 0x7f9e7015e840 2024-04-24T23:58:51.243 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:58:51.245 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:58:51.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.403+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9e700691a0 con 0x7f9e7015e840 2024-04-24T23:58:51.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.404+0000 7f9e667fc640 1 -- 172.21.15.113:0/3319556812 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f9e68059980 con 0x7f9e7015e840 2024-04-24T23:58:51.404 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e5406be30 msgr2=0x7f9e5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e5406be30 0x7f9e5406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c000f60 tx=0x7f9e6c007d50 comp rx=0 tx=0).stop 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 msgr2=0x7f9e7015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e7015b380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e68007e40 tx=0x7f9e6800e4b0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 shutdown_connections 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e5406be30 0x7f9e5406e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e7015e840 0x7f9e7015b380 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e70147640 0x7f9e70137890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 --2- 172.21.15.113:0/3319556812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e700d23c0 0x7f9e70144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 >> 172.21.15.113:0/3319556812 conn(0x7f9e700b7370 msgr2=0x7f9e701427a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:51.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 shutdown_connections 2024-04-24T23:58:51.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.406+0000 7f9e77eda640 1 -- 172.21.15.113:0/3319556812 wait complete. 2024-04-24T23:58:51.417 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:58:51.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:51.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-24T23:58:51.441 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:58:51.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:51.441 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:58:51.507 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:51.507 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:51.527 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:51.527 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:51.588 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:51.588 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:51.646 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:51.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.743+0000 7f2e27e9e640 1 Processor -- start 2024-04-24T23:58:51.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.743+0000 7f2e27e9e640 1 -- start start 2024-04-24T23:58:51.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e27e9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e20144910 0x7f2e200b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e27e9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e2006e6e0 0x7f2e200cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e27e9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e20144cf0 con 0x7f2e200b5240 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e27e9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2e200b4510 con 0x7f2e2006e6e0 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e27e9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e200b2f30 con 0x7f2e20144910 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e2643b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e2006e6e0 0x7f2e200cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e2643b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e2006e6e0 0x7f2e200cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59350/0 (socket says 172.21.15.113:59350) 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e25439640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2e200b5240 0x7f2e2006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43176/0 (socket says 172.21.15.113:43176) 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e2643b640 1 -- 172.21.15.113:0/1277348043 learned_addr learned my addr 172.21.15.113:0/1277348043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e25c3a640 1 --2- 172.21.15.113:0/1277348043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e20144910 0x7f2e200b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e2643b640 1 -- 172.21.15.113:0/1277348043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e20144910 msgr2=0x7f2e200b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e2643b640 1 --2- 172.21.15.113:0/1277348043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e20144910 0x7f2e200b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e2643b640 1 -- 172.21.15.113:0/1277348043 >> v1:172.21.15.113:6789/0 conn(0x7f2e200b5240 legacy=0x7f2e2006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e2643b640 1 -- 172.21.15.113:0/1277348043 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e200b3210 con 0x7f2e2006e6e0 2024-04-24T23:58:51.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.744+0000 7f2e24c38640 1 -- 172.21.15.113:0/1277348043 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3363557761 0 0) 0x7f2e20144cf0 con 0x7f2e200b5240 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.745+0000 7f2e2643b640 1 --2- 172.21.15.113:0/1277348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e2006e6e0 0x7f2e200cd500 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009870 tx=0x7f2e1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f71a4a4be41e68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.745+0000 7f2e24c38640 1 -- 172.21.15.113:0/1277348043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18037070 con 0x7f2e2006e6e0 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.745+0000 7f2e24c38640 1 -- 172.21.15.113:0/1277348043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e18032440 con 0x7f2e2006e6e0 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.745+0000 7f2e24c38640 1 -- 172.21.15.113:0/1277348043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18032810 con 0x7f2e2006e6e0 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 -- 172.21.15.113:0/1277348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e2006e6e0 msgr2=0x7f2e200cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/1277348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e2006e6e0 0x7f2e200cd500 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009870 tx=0x7f2e1802a270 comp rx=0 tx=0).stop 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 -- 172.21.15.113:0/1277348043 shutdown_connections 2024-04-24T23:58:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/1277348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e2006e6e0 0x7f2e200cd500 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/1277348043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e20144910 0x7f2e200b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 -- 172.21.15.113:0/1277348043 >> 172.21.15.113:0/1277348043 conn(0x7f2e200b1a50 msgr2=0x7f2e200b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:51.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 -- 172.21.15.113:0/1277348043 shutdown_connections 2024-04-24T23:58:51.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.746+0000 7f2e27e9e640 1 -- 172.21.15.113:0/1277348043 wait complete. 2024-04-24T23:58:51.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.747+0000 7f2e27e9e640 1 Processor -- start 2024-04-24T23:58:51.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.747+0000 7f2e27e9e640 1 -- start start 2024-04-24T23:58:51.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e2006e6e0 0x7f2e2013d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e200b5240 0x7f2e2006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e20144910 0x7f2e2006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200b4da0 con 0x7f2e200b5240 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200b37d0 con 0x7f2e20144910 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e200b4f20 con 0x7f2e2006e6e0 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e200b5240 0x7f2e2006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e200b5240 0x7f2e2006db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59354/0 (socket says 172.21.15.113:59354) 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 -- 172.21.15.113:0/2212245174 learned_addr learned my addr 172.21.15.113:0/2212245174 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25c3a640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e2006e6e0 0x7f2e2013d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e2643b640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e20144910 0x7f2e2006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 -- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e2006e6e0 msgr2=0x7f2e2013d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e2006e6e0 0x7f2e2013d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 -- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e20144910 msgr2=0x7f2e2006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e20144910 0x7f2e2006e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 -- 172.21.15.113:0/2212245174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e200aeeb0 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e2643b640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e20144910 0x7f2e2006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e25439640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e200b5240 0x7f2e2006db40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c4a0 tx=0x7f2e1000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e167fc640 1 -- 172.21.15.113:0/2212245174 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10018070 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e167fc640 1 -- 172.21.15.113:0/2212245174 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e100074e0 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e200af190 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e167fc640 1 -- 172.21.15.113:0/2212245174 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10004a20 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.748+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e200ae080 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.749+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e200691a0 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.750+0000 7f2e167fc640 1 -- 172.21.15.113:0/2212245174 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e1001b070 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.750+0000 7f2e167fc640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e0406be30 0x7f2e0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.750+0000 7f2e167fc640 1 -- 172.21.15.113:0/2212245174 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e1008a140 con 0x7f2e200b5240 2024-04-24T23:58:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.750+0000 7f2e25c3a640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e0406be30 0x7f2e0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:51.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.750+0000 7f2e25c3a640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e0406be30 0x7f2e0406e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c003b80 tx=0x7f2e1c0098d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:51.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.754+0000 7f2e167fc640 1 -- 172.21.15.113:0/2212245174 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e10058410 con 0x7f2e200b5240 2024-04-24T23:58:51.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.929+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e201439d0 con 0x7f2e200b5240 2024-04-24T23:58:51.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.931+0000 7f2e167fc640 1 -- 172.21.15.113:0/2212245174 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f2e1005c710 con 0x7f2e200b5240 2024-04-24T23:58:51.931 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:51.931 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.932+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e0406be30 msgr2=0x7f2e0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.932+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e0406be30 0x7f2e0406e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c003b80 tx=0x7f2e1c0098d0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.932+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e200b5240 msgr2=0x7f2e2006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.932+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e200b5240 0x7f2e2006db40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c4a0 tx=0x7f2e1000c960 comp rx=0 tx=0).stop 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 shutdown_connections 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e0406be30 0x7f2e0406e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e20144910 0x7f2e2006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e200b5240 0x7f2e2006db40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 --2- 172.21.15.113:0/2212245174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e2006e6e0 0x7f2e2013d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:51.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 >> 172.21.15.113:0/2212245174 conn(0x7f2e200b1a50 msgr2=0x7f2e200b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:51.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 shutdown_connections 2024-04-24T23:58:51.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:51.933+0000 7f2e27e9e640 1 -- 172.21.15.113:0/2212245174 wait complete. 2024-04-24T23:58:51.944 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:51.944 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:52.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.043+0000 7fd201989640 1 Processor -- start 2024-04-24T23:58:52.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.043+0000 7fd201989640 1 -- start start 2024-04-24T23:58:52.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd201989640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1fc15e2d0 0x7fd1fc15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd201989640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 0x7fd1fc0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd201989640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1fc15f4f0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd201989640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc0cde00 con 0x7fd1fc15e2d0 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd201989640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc15f7d0 con 0x7fd1fc0b3080 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1faffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1fc15e2d0 0x7fd1fc15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1fb7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd1fc15e6b0 0x7fd1fc071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43180/0 (socket says 172.21.15.113:43180) 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1faffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1fc15e2d0 0x7fd1fc15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59366/0 (socket says 172.21.15.113:59366) 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1fb7fe640 1 -- 172.21.15.113:0/2250750648 learned_addr learned my addr 172.21.15.113:0/2250750648 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1fa7fc640 1 --2- 172.21.15.113:0/2250750648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 0x7fd1fc0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1faffd640 1 -- 172.21.15.113:0/2250750648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 msgr2=0x7fd1fc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1faffd640 1 --2- 172.21.15.113:0/2250750648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 0x7fd1fc0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1faffd640 1 -- 172.21.15.113:0/2250750648 >> v1:172.21.15.113:6789/0 conn(0x7fd1fc15e6b0 legacy=0x7fd1fc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1f9ffb640 1 -- 172.21.15.113:0/2250750648 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565762468 0 0) 0x7fd1fc15f4f0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.044+0000 7fd1faffd640 1 -- 172.21.15.113:0/2250750648 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc15fab0 con 0x7fd1fc15e2d0 2024-04-24T23:58:52.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.045+0000 7fd1faffd640 1 --2- 172.21.15.113:0/2250750648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e2d0 0x7fd1fc15ebf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009d90 tx=0x7fd1ec02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a888260f62d7a90b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.045+0000 7fd1f9ffb640 1 -- 172.21.15.113:0/2250750648 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031600 con 0x7fd1fc15e2d0 2024-04-24T23:58:52.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.045+0000 7fd1f9ffb640 1 -- 172.21.15.113:0/2250750648 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec0317a0 con 0x7fd1fc15e2d0 2024-04-24T23:58:52.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.045+0000 7fd1f9ffb640 1 -- 172.21.15.113:0/2250750648 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031b70 con 0x7fd1fc15e2d0 2024-04-24T23:58:52.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 -- 172.21.15.113:0/2250750648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e2d0 msgr2=0x7fd1fc15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 --2- 172.21.15.113:0/2250750648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e2d0 0x7fd1fc15ebf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009d90 tx=0x7fd1ec02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 -- 172.21.15.113:0/2250750648 shutdown_connections 2024-04-24T23:58:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 --2- 172.21.15.113:0/2250750648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 0x7fd1fc0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 --2- 172.21.15.113:0/2250750648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e2d0 0x7fd1fc15ebf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 -- 172.21.15.113:0/2250750648 >> 172.21.15.113:0/2250750648 conn(0x7fd1fc06b0f0 msgr2=0x7fd1fc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 -- 172.21.15.113:0/2250750648 shutdown_connections 2024-04-24T23:58:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.046+0000 7fd201989640 1 -- 172.21.15.113:0/2250750648 wait complete. 2024-04-24T23:58:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.047+0000 7fd201989640 1 Processor -- start 2024-04-24T23:58:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.047+0000 7fd201989640 1 -- start start 2024-04-24T23:58:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd201989640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 0x7fd1fc15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd201989640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1fc15e2d0 0x7fd1fc1529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd201989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e6b0 0x7fd1fc0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd201989640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc15f2b0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd201989640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc15f130 con 0x7fd1fc15e2d0 2024-04-24T23:58:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd201989640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0cf2a0 con 0x7fd1fc0b3080 2024-04-24T23:58:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e6b0 0x7fd1fc0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e6b0 0x7fd1fc0ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59374/0 (socket says 172.21.15.113:59374) 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 -- 172.21.15.113:0/4106364933 learned_addr learned my addr 172.21.15.113:0/4106364933 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 -- 172.21.15.113:0/4106364933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 msgr2=0x7fd1fc15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 0x7fd1fc15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 -- 172.21.15.113:0/4106364933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1fc15e2d0 msgr2=0x7fd1fc1529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fa7fc640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1fc15e2d0 0x7fd1fc1529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1fc15e2d0 0x7fd1fc1529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.048+0000 7fd1fb7fe640 1 -- 172.21.15.113:0/4106364933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc153120 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.049+0000 7fd1fb7fe640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e6b0 0x7fd1fc0ce530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c410 tx=0x7fd1f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.049+0000 7fd1e3fff640 1 -- 172.21.15.113:0/4106364933 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f40074e0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.049+0000 7fd1e3fff640 1 -- 172.21.15.113:0/4106364933 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f40047e0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.049+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc153400 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.049+0000 7fd1e3fff640 1 -- 172.21.15.113:0/4106364933 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4004c10 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.049+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc1540a0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.050+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1fc0691a0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.050+0000 7fd1e3fff640 1 -- 172.21.15.113:0/4106364933 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1f4011070 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.051+0000 7fd1e3fff640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1e806be30 0x7fd1e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.051+0000 7fd1faffd640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1e806be30 0x7fd1e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.051+0000 7fd1e3fff640 1 -- 172.21.15.113:0/4106364933 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd1f408a900 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.051+0000 7fd1faffd640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1e806be30 0x7fd1e806e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec034000 tx=0x7fd1ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.055+0000 7fd1e3fff640 1 -- 172.21.15.113:0/4106364933 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1f401a2b0 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.224+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1fc072050 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.225+0000 7fd1e3fff640 1 -- 172.21.15.113:0/4106364933 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd1f4059340 con 0x7fd1fc15e6b0 2024-04-24T23:58:52.225 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1e806be30 msgr2=0x7fd1e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1e806be30 0x7fd1e806e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec034000 tx=0x7fd1ec02e000 comp rx=0 tx=0).stop 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e6b0 msgr2=0x7fd1fc0ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e6b0 0x7fd1fc0ce530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c410 tx=0x7fd1f400c8d0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 shutdown_connections 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1e806be30 0x7fd1e806e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1fc15e2d0 0x7fd1fc1529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc15e6b0 0x7fd1fc0ce530 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 --2- 172.21.15.113:0/4106364933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1fc0b3080 0x7fd1fc15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 >> 172.21.15.113:0/4106364933 conn(0x7fd1fc06b0f0 msgr2=0x7fd1fc15d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 shutdown_connections 2024-04-24T23:58:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.227+0000 7fd201989640 1 -- 172.21.15.113:0/4106364933 wait complete. 2024-04-24T23:58:52.238 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.336+0000 7f677e99e640 1 Processor -- start 2024-04-24T23:58:52.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.337+0000 7f677e99e640 1 -- start start 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.337+0000 7f677e99e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f677806e770 0x7f677806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.337+0000 7f677e99e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f677806d410 0x7f677806d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.337+0000 7f677e99e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677815fb90 con 0x7f677806b0f0 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.337+0000 7f677e99e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f677815fe70 con 0x7f677806d410 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.337+0000 7f677e99e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6778160150 con 0x7f677806e770 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677e19d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f677806d410 0x7f677806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677d99c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f677806b0f0 0x7f677806b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43192/0 (socket says 172.21.15.113:43192) 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677e19d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f677806d410 0x7f677806d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59380/0 (socket says 172.21.15.113:59380) 2024-04-24T23:58:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677d99c640 1 -- 172.21.15.113:0/2354766628 learned_addr learned my addr 172.21.15.113:0/2354766628 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677d19b640 1 --2- 172.21.15.113:0/2354766628 >> v2:172.21.15.178:3300/0 conn(0x7f677806e770 0x7f677806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677e19d640 1 -- 172.21.15.113:0/2354766628 >> v2:172.21.15.178:3300/0 conn(0x7f677806e770 msgr2=0x7f677806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677e19d640 1 --2- 172.21.15.113:0/2354766628 >> v2:172.21.15.178:3300/0 conn(0x7f677806e770 0x7f677806ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677e19d640 1 -- 172.21.15.113:0/2354766628 >> v1:172.21.15.113:6789/0 conn(0x7f677806b0f0 legacy=0x7f677806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.338+0000 7f677e19d640 1 -- 172.21.15.113:0/2354766628 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778160430 con 0x7f677806d410 2024-04-24T23:58:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.339+0000 7f677e19d640 1 --2- 172.21.15.113:0/2354766628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806d410 0x7f677806d840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f67780d14a0 tx=0x7f677002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1243de8004425a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677c99a640 1 -- 172.21.15.113:0/2354766628 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770007740 con 0x7f677806d410 2024-04-24T23:58:52.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677c99a640 1 -- 172.21.15.113:0/2354766628 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67700078e0 con 0x7f677806d410 2024-04-24T23:58:52.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677c99a640 1 -- 172.21.15.113:0/2354766628 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770007cb0 con 0x7f677806d410 2024-04-24T23:58:52.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 -- 172.21.15.113:0/2354766628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806d410 msgr2=0x7f677806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 --2- 172.21.15.113:0/2354766628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806d410 0x7f677806d840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f67780d14a0 tx=0x7f677002a680 comp rx=0 tx=0).stop 2024-04-24T23:58:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 -- 172.21.15.113:0/2354766628 shutdown_connections 2024-04-24T23:58:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 --2- 172.21.15.113:0/2354766628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806d410 0x7f677806d840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 --2- 172.21.15.113:0/2354766628 >> v2:172.21.15.178:3300/0 conn(0x7f677806e770 0x7f677806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 -- 172.21.15.113:0/2354766628 >> 172.21.15.113:0/2354766628 conn(0x7f67780b21c0 msgr2=0x7f67780b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 -- 172.21.15.113:0/2354766628 shutdown_connections 2024-04-24T23:58:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.341+0000 7f677e99e640 1 -- 172.21.15.113:0/2354766628 wait complete. 2024-04-24T23:58:52.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 Processor -- start 2024-04-24T23:58:52.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 -- start start 2024-04-24T23:58:52.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677806b0f0 0x7f677814df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f677806d410 0x7f67780a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806e770 0x7f67780cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6778153290 con 0x7f677806e770 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6778153110 con 0x7f677806d410 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.343+0000 7f677e99e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f67780ceaa0 con 0x7f677806b0f0 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806e770 0x7f67780cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806e770 0x7f67780cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59392/0 (socket says 172.21.15.113:59392) 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 -- 172.21.15.113:0/2861293062 learned_addr learned my addr 172.21.15.113:0/2861293062 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677d99c640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677806b0f0 0x7f677814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677d19b640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f677806d410 0x7f67780a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 -- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677806b0f0 msgr2=0x7f677814df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677806b0f0 0x7f677814df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 -- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f677806d410 msgr2=0x7f67780a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f677806d410 0x7f67780a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 -- 172.21.15.113:0/2861293062 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67780a9910 con 0x7f677806e770 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e19d640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806e770 0x7f67780cff20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f677002ac70 tx=0x7f6770004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f676e7fc640 1 -- 172.21.15.113:0/2861293062 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770040070 con 0x7f677806e770 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f676e7fc640 1 -- 172.21.15.113:0/2861293062 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6770031c40 con 0x7f677806e770 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f676e7fc640 1 -- 172.21.15.113:0/2861293062 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677003c420 con 0x7f677806e770 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67781f27d0 con 0x7f677806e770 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.344+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781f2d90 con 0x7f677806e770 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.345+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67781f2f90 con 0x7f677806e770 2024-04-24T23:58:52.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.346+0000 7f676e7fc640 1 -- 172.21.15.113:0/2861293062 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6770039040 con 0x7f677806e770 2024-04-24T23:58:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.346+0000 7f676e7fc640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675c06be30 0x7f675c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.346+0000 7f677d99c640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675c06be30 0x7f675c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.346+0000 7f676e7fc640 1 -- 172.21.15.113:0/2861293062 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f67700ac430 con 0x7f677806e770 2024-04-24T23:58:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.346+0000 7f677d99c640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675c06be30 0x7f675c06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6768007fd0 tx=0x7f6768007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.351+0000 7f676e7fc640 1 -- 172.21.15.113:0/2861293062 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f677007a700 con 0x7f677806e770 2024-04-24T23:58:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.523+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67780691a0 con 0x7f677806e770 2024-04-24T23:58:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.523+0000 7f676e7fc640 1 -- 172.21.15.113:0/2861293062 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f677007ea00 con 0x7f677806e770 2024-04-24T23:58:52.523 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675c06be30 msgr2=0x7f675c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675c06be30 0x7f675c06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6768007fd0 tx=0x7f6768007e80 comp rx=0 tx=0).stop 2024-04-24T23:58:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806e770 msgr2=0x7f67780cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806e770 0x7f67780cff20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f677002ac70 tx=0x7f6770004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 shutdown_connections 2024-04-24T23:58:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675c06be30 0x7f675c06e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f677806d410 0x7f67780a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677806e770 0x7f67780cff20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 --2- 172.21.15.113:0/2861293062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677806b0f0 0x7f677814df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 >> 172.21.15.113:0/2861293062 conn(0x7f67780b21c0 msgr2=0x7f67780b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 shutdown_connections 2024-04-24T23:58:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.525+0000 7f677e99e640 1 -- 172.21.15.113:0/2861293062 wait complete. 2024-04-24T23:58:52.536 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.636+0000 7f5546e58640 1 Processor -- start 2024-04-24T23:58:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.636+0000 7f5546e58640 1 -- start start 2024-04-24T23:58:52.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5546e58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5546e58640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f55400b63b0 0x7f55400b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5546e58640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55400d2e00 con 0x7f55400d28c0 2024-04-24T23:58:52.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5546e58640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f55400d30e0 con 0x7f55400b63b0 2024-04-24T23:58:52.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5546e58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f55400d33c0 con 0x7f554015e510 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f553ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f55400d28c0 0x7f55400b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43204/0 (socket says 172.21.15.113:43204) 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f553ffff640 1 -- 172.21.15.113:0/2364298736 learned_addr learned my addr 172.21.15.113:0/2364298736 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5544bf4640 1 --2- 172.21.15.113:0/2364298736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f55453f5640 1 --2- 172.21.15.113:0/2364298736 >> v2:172.21.15.178:3300/0 conn(0x7f55400b63b0 0x7f55400b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f553f7fe640 1 -- 172.21.15.113:0/2364298736 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719590984 0 0) 0x7f55400d2e00 con 0x7f55400d28c0 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f553f7fe640 1 -- 172.21.15.113:0/2364298736 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5528003520 con 0x7f55400d28c0 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5544bf4640 1 -- 172.21.15.113:0/2364298736 >> v2:172.21.15.178:3300/0 conn(0x7f55400b63b0 msgr2=0x7f55400b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5544bf4640 1 --2- 172.21.15.113:0/2364298736 >> v2:172.21.15.178:3300/0 conn(0x7f55400b63b0 0x7f55400b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5544bf4640 1 -- 172.21.15.113:0/2364298736 >> v1:172.21.15.113:6789/0 conn(0x7f55400d28c0 legacy=0x7f55400b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.637+0000 7f5544bf4640 1 -- 172.21.15.113:0/2364298736 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55400d36a0 con 0x7f554015e510 2024-04-24T23:58:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.638+0000 7f5544bf4640 1 --2- 172.21.15.113:0/2364298736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400d2380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5538009d90 tx=0x7f553802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2640a11821545db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.638+0000 7f553f7fe640 1 -- 172.21.15.113:0/2364298736 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538031600 con 0x7f554015e510 2024-04-24T23:58:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.638+0000 7f553f7fe640 1 -- 172.21.15.113:0/2364298736 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55380317a0 con 0x7f554015e510 2024-04-24T23:58:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.638+0000 7f553f7fe640 1 -- 172.21.15.113:0/2364298736 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538031b50 con 0x7f554015e510 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 -- 172.21.15.113:0/2364298736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 msgr2=0x7f55400d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 --2- 172.21.15.113:0/2364298736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400d2380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5538009d90 tx=0x7f553802a5d0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 -- 172.21.15.113:0/2364298736 shutdown_connections 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 --2- 172.21.15.113:0/2364298736 >> v2:172.21.15.178:3300/0 conn(0x7f55400b63b0 0x7f55400b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 --2- 172.21.15.113:0/2364298736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400d2380 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 -- 172.21.15.113:0/2364298736 >> 172.21.15.113:0/2364298736 conn(0x7f554006b720 msgr2=0x7f55400d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 -- 172.21.15.113:0/2364298736 shutdown_connections 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.639+0000 7f5546e58640 1 -- 172.21.15.113:0/2364298736 wait complete. 2024-04-24T23:58:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.640+0000 7f5546e58640 1 Processor -- start 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.640+0000 7f5546e58640 1 -- start start 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5546e58640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55400b63b0 0x7f55400c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5546e58640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55400d28c0 0x7f55400cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5546e58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5546e58640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f55400b9dd0 con 0x7f55400b63b0 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5546e58640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55400d3ea0 con 0x7f55400d28c0 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5546e58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f55400b7900 con 0x7f554015e510 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55400b63b0 0x7f55400c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55400b63b0 0x7f55400c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59402/0 (socket says 172.21.15.113:59402) 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 -- 172.21.15.113:0/4145221387 learned_addr learned my addr 172.21.15.113:0/4145221387 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f553ffff640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55400d28c0 0x7f55400cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f55453f5640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 -- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 msgr2=0x7f55400cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 -- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55400d28c0 msgr2=0x7f55400cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55400d28c0 0x7f55400cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f5544bf4640 1 -- 172.21.15.113:0/4145221387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55400cc8c0 con 0x7f55400b63b0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.641+0000 7f553ffff640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55400d28c0 0x7f55400cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.642+0000 7f5544bf4640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55400b63b0 0x7f55400c80a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5538009eb0 tx=0x7f5538004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.642+0000 7f553d7fa640 1 -- 172.21.15.113:0/4145221387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553803f070 con 0x7f55400b63b0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.642+0000 7f553d7fa640 1 -- 172.21.15.113:0/4145221387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5538030830 con 0x7f55400b63b0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.642+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55400cce40 con 0x7f55400b63b0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.642+0000 7f553d7fa640 1 -- 172.21.15.113:0/4145221387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538030c00 con 0x7f55400b63b0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.642+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55400cd400 con 0x7f55400b63b0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.643+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55400ccb60 con 0x7f55400b63b0 2024-04-24T23:58:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.643+0000 7f553d7fa640 1 -- 172.21.15.113:0/4145221387 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f553803b410 con 0x7f55400b63b0 2024-04-24T23:58:52.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.644+0000 7f553d7fa640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f552c06be30 0x7f552c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.644+0000 7f553d7fa640 1 -- 172.21.15.113:0/4145221387 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f55380ac400 con 0x7f55400b63b0 2024-04-24T23:58:52.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.644+0000 7f553ffff640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f552c06be30 0x7f552c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.644+0000 7f553ffff640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f552c06be30 0x7f552c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55400b8ac0 tx=0x7f5534009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.648+0000 7f553d7fa640 1 -- 172.21.15.113:0/4145221387 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f553807a6d0 con 0x7f55400b63b0 2024-04-24T23:58:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.821+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f55400b5570 con 0x7f55400b63b0 2024-04-24T23:58:52.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.822+0000 7f553d7fa640 1 -- 172.21.15.113:0/4145221387 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f553807e9d0 con 0x7f55400b63b0 2024-04-24T23:58:52.822 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.823+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f552c06be30 msgr2=0x7f552c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.823+0000 7f5546e58640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f552c06be30 0x7f552c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55400b8ac0 tx=0x7f5534009040 comp rx=0 tx=0).stop 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.823+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55400b63b0 msgr2=0x7f55400c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.823+0000 7f5546e58640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55400b63b0 0x7f55400c80a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5538009eb0 tx=0x7f5538004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 shutdown_connections 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f552c06be30 0x7f552c06e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55400d28c0 0x7f55400cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f554015e510 0x7f55400cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 --2- 172.21.15.113:0/4145221387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55400b63b0 0x7f55400c80a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 >> 172.21.15.113:0/4145221387 conn(0x7f554006b720 msgr2=0x7f5540071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:52.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 shutdown_connections 2024-04-24T23:58:52.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.824+0000 7f5546e58640 1 -- 172.21.15.113:0/4145221387 wait complete. 2024-04-24T23:58:52.835 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:58:52.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.932+0000 7fd07c6bd640 1 Processor -- start 2024-04-24T23:58:52.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.932+0000 7fd07c6bd640 1 -- start start 2024-04-24T23:58:52.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.933+0000 7fd07c6bd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd0740b5ca0 0x7fd0740b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.933+0000 7fd07c6bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0740d2e00 con 0x7fd0740b73c0 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.933+0000 7fd07c6bd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0740d30e0 con 0x7fd07415e510 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.933+0000 7fd07c6bd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd0740b6640 con 0x7fd0740b5ca0 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.933+0000 7fd07ac5a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd0740b73c0 0x7fd0740b77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43208/0 (socket says 172.21.15.113:43208) 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07ac5a640 1 -- 172.21.15.113:0/494879248 learned_addr learned my addr 172.21.15.113:0/494879248 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079c58640 1 --2- 172.21.15.113:0/494879248 >> v2:172.21.15.178:3300/0 conn(0x7fd0740b5ca0 0x7fd0740b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1215711126 0 0) 0x7fd0740d2e00 con 0x7fd0740b73c0 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd05c003520 con 0x7fd0740b73c0 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3339024654 0 0) 0x7fd05c003520 con 0x7fd0740b73c0 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 >> v2:172.21.15.178:3300/0 conn(0x7fd0740b5ca0 msgr2=0x7fd0740b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 --2- 172.21.15.113:0/494879248 >> v2:172.21.15.178:3300/0 conn(0x7fd0740b5ca0 0x7fd0740b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 >> v1:172.21.15.178:6789/0 conn(0x7fd07415e510 legacy=0x7fd0740d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0740d33c0 con 0x7fd0740b73c0 2024-04-24T23:58:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079c58640 1 --2- 172.21.15.113:0/494879248 >> v2:172.21.15.178:3300/0 conn(0x7fd0740b5ca0 0x7fd0740b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fd064001670 con 0x7fd0740b73c0 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 >> v1:172.21.15.113:6789/0 conn(0x7fd0740b73c0 legacy=0x7fd0740b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd05c0039b0 0x7fd05c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c0043b0 0x7fd05c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c005120 0x7fd05c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c00c030 con 0x7fd05c005120 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c00c1e0 con 0x7fd05c0043b0 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c00c3e0 con 0x7fd05c0039b0 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07ac5a640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c005120 0x7fd05c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd079c58640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c0043b0 0x7fd05c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07a459640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd05c0039b0 0x7fd05c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07ac5a640 1 -- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd05c0039b0 msgr2=0x7fd05c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07ac5a640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd05c0039b0 0x7fd05c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07ac5a640 1 -- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c0043b0 msgr2=0x7fd05c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07ac5a640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c0043b0 0x7fd05c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.934+0000 7fd07ac5a640 1 -- 172.21.15.113:0/494879248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c003520 con 0x7fd05c005120 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.935+0000 7fd07ac5a640 1 -- 172.21.15.113:0/494879248 reap_dead start 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.935+0000 7fd07ac5a640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c005120 0x7fd05c00b9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c210 tx=0x7fd05c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b04d1b846caea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.935+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064007ce0 con 0x7fd05c005120 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.935+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd064005af0 con 0x7fd05c005120 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.935+0000 7fd079457640 1 -- 172.21.15.113:0/494879248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064007070 con 0x7fd05c005120 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.936+0000 7fd07c6bd640 1 -- 172.21.15.113:0/494879248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c005120 msgr2=0x7fd05c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.936+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c005120 0x7fd05c00b9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c210 tx=0x7fd05c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.936+0000 7fd07c6bd640 1 -- 172.21.15.113:0/494879248 shutdown_connections 2024-04-24T23:58:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.936+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/494879248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c005120 0x7fd05c00b9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.936+0000 7fd07c6bd640 1 -- 172.21.15.113:0/494879248 >> 172.21.15.113:0/494879248 conn(0x7fd07406b720 msgr2=0x7fd0740d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.936+0000 7fd07c6bd640 1 -- 172.21.15.113:0/494879248 shutdown_connections 2024-04-24T23:58:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.936+0000 7fd07c6bd640 1 -- 172.21.15.113:0/494879248 wait complete. 2024-04-24T23:58:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.937+0000 7fd07c6bd640 1 Processor -- start 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.937+0000 7fd07c6bd640 1 -- start start 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07c6bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c005120 0x7fd0741f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07c6bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074160d90 0x7fd0741f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07c6bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0741f46c0 0x7fd0740b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07c6bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740d3ea0 con 0x7fd074160d90 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07c6bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740d3980 con 0x7fd05c005120 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07c6bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd0740b9dd0 con 0x7fd0741f46c0 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074160d90 0x7fd0741f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074160d90 0x7fd0741f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59428/0 (socket says 172.21.15.113:59428) 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 -- 172.21.15.113:0/260844348 learned_addr learned my addr 172.21.15.113:0/260844348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07a459640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c005120 0x7fd0741f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd07ac5a640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0741f46c0 0x7fd0740b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 -- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0741f46c0 msgr2=0x7fd0740b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0741f46c0 0x7fd0740b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 -- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c005120 msgr2=0x7fd0741f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c005120 0x7fd0741f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 -- 172.21.15.113:0/260844348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0741f3f30 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.938+0000 7fd079c58640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074160d90 0x7fd0741f7410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd06800f430 tx=0x7fd06800f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.939+0000 7fd072ffd640 1 -- 172.21.15.113:0/260844348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068003b90 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.939+0000 7fd072ffd640 1 -- 172.21.15.113:0/260844348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd068003d50 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.939+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0741f7c40 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.939+0000 7fd072ffd640 1 -- 172.21.15.113:0/260844348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068004270 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.939+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0741f7950 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.940+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0740b97b0 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.940+0000 7fd072ffd640 1 -- 172.21.15.113:0/260844348 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd068004410 con 0x7fd074160d90 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.940+0000 7fd072ffd640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd06006be60 0x7fd06006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.941+0000 7fd07a459640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd06006be60 0x7fd06006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.941+0000 7fd072ffd640 1 -- 172.21.15.113:0/260844348 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd06808bd80 con 0x7fd074160d90 2024-04-24T23:58:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.941+0000 7fd07a459640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd06006be60 0x7fd06006e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0018c0 tx=0x7fd06c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:52.945+0000 7fd072ffd640 1 -- 172.21.15.113:0/260844348 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd068019070 con 0x7fd074160d90 2024-04-24T23:58:53.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.131+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd0740691a0 con 0x7fd074160d90 2024-04-24T23:58:53.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.132+0000 7fd072ffd640 1 -- 172.21.15.113:0/260844348 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fd068010040 con 0x7fd074160d90 2024-04-24T23:58:53.132 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-24T23:58:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd06006be60 msgr2=0x7fd06006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd06006be60 0x7fd06006e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0018c0 tx=0x7fd06c009040 comp rx=0 tx=0).stop 2024-04-24T23:58:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074160d90 msgr2=0x7fd0741f7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074160d90 0x7fd0741f7410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd06800f430 tx=0x7fd06800f8f0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 shutdown_connections 2024-04-24T23:58:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd06006be60 0x7fd06006e320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0741f46c0 0x7fd0740b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074160d90 0x7fd0741f7410 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 --2- 172.21.15.113:0/260844348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd05c005120 0x7fd0741f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 >> 172.21.15.113:0/260844348 conn(0x7fd07406b720 msgr2=0x7fd07415b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 shutdown_connections 2024-04-24T23:58:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.134+0000 7fd07c6bd640 1 -- 172.21.15.113:0/260844348 wait complete. 2024-04-24T23:58:53.144 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-24T23:58:53.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:53.145 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-24T23:58:53.175 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:58:53.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:53.175 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T23:58:53.249 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:53.249 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:53.267 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:53.267 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:53.323 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:58:53.323 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:53.381 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:53.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.517+0000 7fc551a93640 1 Processor -- start 2024-04-24T23:58:53.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.517+0000 7fc551a93640 1 -- start start 2024-04-24T23:58:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc551a93640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc54c0b3fd0 0x7fc54c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc551a93640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc54c06e510 0x7fc54c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc551a93640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc54c144cf0 con 0x7fc54c144910 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc551a93640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc54c06e980 con 0x7fc54c06e510 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc551a93640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc54c0b2f30 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc54c06e510 0x7fc54c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc54c144910 0x7fc54c0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43222/0 (socket says 172.21.15.113:43222) 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54b7fe640 1 -- 172.21.15.113:0/1433498278 learned_addr learned my addr 172.21.15.113:0/1433498278 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc54c06e510 0x7fc54c0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59434/0 (socket says 172.21.15.113:59434) 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54affd640 1 --2- 172.21.15.113:0/1433498278 >> v2:172.21.15.178:3300/0 conn(0x7fc54c0b3fd0 0x7fc54c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54bfff640 1 -- 172.21.15.113:0/1433498278 >> v2:172.21.15.178:3300/0 conn(0x7fc54c0b3fd0 msgr2=0x7fc54c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54bfff640 1 --2- 172.21.15.113:0/1433498278 >> v2:172.21.15.178:3300/0 conn(0x7fc54c0b3fd0 0x7fc54c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54bfff640 1 -- 172.21.15.113:0/1433498278 >> v1:172.21.15.113:6789/0 conn(0x7fc54c144910 legacy=0x7fc54c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54bfff640 1 -- 172.21.15.113:0/1433498278 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0b3210 con 0x7fc54c06e510 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.518+0000 7fc54a7fc640 1 -- 172.21.15.113:0/1433498278 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180416181 0 0) 0x7fc54c144cf0 con 0x7fc54c144910 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc54bfff640 1 --2- 172.21.15.113:0/1433498278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c06e510 0x7fc54c0cd500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc544009d90 tx=0x7fc54402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5009372e59e817a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc54a7fc640 1 -- 172.21.15.113:0/1433498278 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544007740 con 0x7fc54c06e510 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc54a7fc640 1 -- 172.21.15.113:0/1433498278 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5440078e0 con 0x7fc54c06e510 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc54a7fc640 1 -- 172.21.15.113:0/1433498278 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544007cb0 con 0x7fc54c06e510 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc551a93640 1 -- 172.21.15.113:0/1433498278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c06e510 msgr2=0x7fc54c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc551a93640 1 --2- 172.21.15.113:0/1433498278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c06e510 0x7fc54c0cd500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc544009d90 tx=0x7fc54402a6a0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc551a93640 1 -- 172.21.15.113:0/1433498278 shutdown_connections 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc551a93640 1 --2- 172.21.15.113:0/1433498278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c06e510 0x7fc54c0cd500 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc551a93640 1 --2- 172.21.15.113:0/1433498278 >> v2:172.21.15.178:3300/0 conn(0x7fc54c0b3fd0 0x7fc54c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.519+0000 7fc551a93640 1 -- 172.21.15.113:0/1433498278 >> 172.21.15.113:0/1433498278 conn(0x7fc54c0b1a50 msgr2=0x7fc54c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:53.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.520+0000 7fc551a93640 1 -- 172.21.15.113:0/1433498278 shutdown_connections 2024-04-24T23:58:53.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.520+0000 7fc551a93640 1 -- 172.21.15.113:0/1433498278 wait complete. 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.521+0000 7fc551a93640 1 Processor -- start 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.521+0000 7fc551a93640 1 -- start start 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.521+0000 7fc551a93640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc54c06e510 0x7fc54c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.521+0000 7fc551a93640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c0b3fd0 0x7fc54c14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.521+0000 7fc551a93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc54c144910 0x7fc54c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc551a93640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0b4440 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc551a93640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0b37d0 con 0x7fc54c144910 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc551a93640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0b4da0 con 0x7fc54c06e510 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c0b3fd0 0x7fc54c14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c0b3fd0 0x7fc54c14ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59448/0 (socket says 172.21.15.113:59448) 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 -- 172.21.15.113:0/840409408 learned_addr learned my addr 172.21.15.113:0/840409408 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54bfff640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc54c144910 0x7fc54c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54b7fe640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc54c06e510 0x7fc54c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 -- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc54c06e510 msgr2=0x7fc54c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc54c06e510 0x7fc54c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 -- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc54c144910 msgr2=0x7fc54c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc54c144910 0x7fc54c070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 -- 172.21.15.113:0/840409408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c151560 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54b7fe640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc54c06e510 0x7fc54c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc54affd640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c0b3fd0 0x7fc54c14ea80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc54000d830 tx=0x7fc54000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc533fff640 1 -- 172.21.15.113:0/840409408 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000b6c0 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc533fff640 1 -- 172.21.15.113:0/840409408 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc540019070 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c14f4d0 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc533fff640 1 -- 172.21.15.113:0/840409408 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000ba80 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.522+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c14fa90 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.523+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c0691a0 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.524+0000 7fc533fff640 1 -- 172.21.15.113:0/840409408 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc54000bc20 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.524+0000 7fc533fff640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc53806be30 0x7fc53806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.524+0000 7fc533fff640 1 -- 172.21.15.113:0/840409408 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc54008b370 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.524+0000 7fc54b7fe640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc53806be30 0x7fc53806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.525+0000 7fc54b7fe640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc53806be30 0x7fc53806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc53c000f60 tx=0x7fc53c007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:53.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.528+0000 7fc533fff640 1 -- 172.21.15.113:0/840409408 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc540059c10 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.702+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc54c0756e0 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.703+0000 7fc533fff640 1 -- 172.21.15.113:0/840409408 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fc540059510 con 0x7fc54c0b3fd0 2024-04-24T23:58:53.703 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:53.703 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.704+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc53806be30 msgr2=0x7fc53806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.704+0000 7fc551a93640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc53806be30 0x7fc53806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc53c000f60 tx=0x7fc53c007ca0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.704+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c0b3fd0 msgr2=0x7fc54c14ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.704+0000 7fc551a93640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c0b3fd0 0x7fc54c14ea80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc54000d830 tx=0x7fc54000dcf0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 shutdown_connections 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc53806be30 0x7fc53806e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc54c144910 0x7fc54c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc54c0b3fd0 0x7fc54c14ea80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 --2- 172.21.15.113:0/840409408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc54c06e510 0x7fc54c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 >> 172.21.15.113:0/840409408 conn(0x7fc54c0b1a50 msgr2=0x7fc54c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 shutdown_connections 2024-04-24T23:58:53.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.705+0000 7fc551a93640 1 -- 172.21.15.113:0/840409408 wait complete. 2024-04-24T23:58:53.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:53.716 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.815+0000 7f4dc9d7e640 1 Processor -- start 2024-04-24T23:58:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.815+0000 7f4dc9d7e640 1 -- start start 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc9d7e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4dc40b3080 0x7f4dc40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc9d7e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dc415f4f0 con 0x7f4dc415e2d0 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc9d7e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dc415f7d0 con 0x7f4dc415e6b0 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc9d7e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4dc40cde00 con 0x7f4dc40b3080 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4dc415e2d0 0x7f4dc415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43228/0 (socket says 172.21.15.113:43228) 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc37fe640 1 -- 172.21.15.113:0/897382444 learned_addr learned my addr 172.21.15.113:0/897382444 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc2ffd640 1 --2- 172.21.15.113:0/897382444 >> v2:172.21.15.178:3300/0 conn(0x7f4dc40b3080 0x7f4dc40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057140164 0 0) 0x7f4dc415f4f0 con 0x7f4dc415e2d0 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.816+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dac003520 con 0x7f4dc415e2d0 2024-04-24T23:58:53.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1086897930 0 0) 0x7f4dac003520 con 0x7f4dc415e2d0 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 >> v2:172.21.15.178:3300/0 conn(0x7f4dc40b3080 msgr2=0x7f4dc40b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 --2- 172.21.15.113:0/897382444 >> v2:172.21.15.178:3300/0 conn(0x7f4dc40b3080 0x7f4dc40b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 >> v1:172.21.15.178:6789/0 conn(0x7f4dc415e6b0 legacy=0x7f4dc40ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc415fab0 con 0x7f4dc415e2d0 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f4db4001670 con 0x7f4dc415e2d0 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 >> v1:172.21.15.113:6789/0 conn(0x7f4dc415e2d0 legacy=0x7f4dc415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dac004800 0x7f4dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dac0051c0 0x7f4dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc40cde00 con 0x7f4dac0039b0 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac00c060 con 0x7f4dac0051c0 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4dac00e470 con 0x7f4dac004800 2024-04-24T23:58:53.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc37fe640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc2ffd640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dac004800 0x7f4dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc3fff640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dac0051c0 0x7f4dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc37fe640 1 -- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dac004800 msgr2=0x7f4dac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc37fe640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dac004800 0x7f4dac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc37fe640 1 -- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dac0051c0 msgr2=0x7f4dac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc37fe640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dac0051c0 0x7f4dac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc37fe640 1 -- 172.21.15.113:0/897382444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac003520 con 0x7f4dac0039b0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.817+0000 7f4dc3fff640 1 -- 172.21.15.113:0/897382444 reap_dead start 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc37fe640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dac003dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4db400c240 tx=0x7f4dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a96fc92331f72c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4007d10 con 0x7f4dac0039b0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db4005ad0 con 0x7f4dac0039b0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc27fc640 1 -- 172.21.15.113:0/897382444 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4007070 con 0x7f4dac0039b0 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/897382444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 msgr2=0x7f4dac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dac003dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4db400c240 tx=0x7f4dac005830 comp rx=0 tx=0).stop 2024-04-24T23:58:53.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/897382444 shutdown_connections 2024-04-24T23:58:53.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/897382444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dac003dd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/897382444 >> 172.21.15.113:0/897382444 conn(0x7f4dc406b0f0 msgr2=0x7f4dc406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:53.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/897382444 shutdown_connections 2024-04-24T23:58:53.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.818+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/897382444 wait complete. 2024-04-24T23:58:53.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.819+0000 7f4dc9d7e640 1 Processor -- start 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc9d7e640 1 -- start start 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc9d7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dc414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc9d7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dc414f7c0 0x7f4dc414d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc9d7e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dc40b18d0 0x7f4dc40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc9d7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc415ed70 con 0x7f4dac0039b0 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc9d7e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc415ebf0 con 0x7f4dc40b18d0 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc9d7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc415eef0 con 0x7f4dc414f7c0 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dc414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dc414e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59474/0 (socket says 172.21.15.113:59474) 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.820+0000 7f4dc37fe640 1 -- 172.21.15.113:0/327087223 learned_addr learned my addr 172.21.15.113:0/327087223 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc3fff640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dc40b18d0 0x7f4dc40b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc2ffd640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dc414f7c0 0x7f4dc414d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc37fe640 1 -- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dc414f7c0 msgr2=0x7f4dc414d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc37fe640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dc414f7c0 0x7f4dc414d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc37fe640 1 -- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dc40b18d0 msgr2=0x7f4dc40b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc37fe640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dc40b18d0 0x7f4dc40b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc37fe640 1 -- 172.21.15.113:0/327087223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc414fd30 con 0x7f4dac0039b0 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc2ffd640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dc414f7c0 0x7f4dc414d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc37fe640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dc414e1f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4db4034040 tx=0x7f4dc415f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dabfff640 1 -- 172.21.15.113:0/327087223 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4041070 con 0x7f4dac0039b0 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dabfff640 1 -- 172.21.15.113:0/327087223 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db4004e40 con 0x7f4dac0039b0 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dabfff640 1 -- 172.21.15.113:0/327087223 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4007d10 con 0x7f4dac0039b0 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc4150010 con 0x7f4dac0039b0 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.821+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc41505d0 con 0x7f4dac0039b0 2024-04-24T23:58:53.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.822+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc40b2290 con 0x7f4dac0039b0 2024-04-24T23:58:53.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.823+0000 7f4dabfff640 1 -- 172.21.15.113:0/327087223 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4db4044070 con 0x7f4dac0039b0 2024-04-24T23:58:53.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.823+0000 7f4dabfff640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4db006be30 0x7f4db006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:53.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.823+0000 7f4dabfff640 1 -- 172.21.15.113:0/327087223 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4db40ab1a0 con 0x7f4dac0039b0 2024-04-24T23:58:53.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.823+0000 7f4dc2ffd640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4db006be30 0x7f4db006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:53.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.823+0000 7f4dc2ffd640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4db006be30 0x7f4db006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dc41588e0 tx=0x7f4db8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:53.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.827+0000 7f4dabfff640 1 -- 172.21.15.113:0/327087223 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4db404b070 con 0x7f4dac0039b0 2024-04-24T23:58:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.999+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4dc40691a0 con 0x7f4dac0039b0 2024-04-24T23:58:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:53.999+0000 7f4dabfff640 1 -- 172.21.15.113:0/327087223 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4db4079b50 con 0x7f4dac0039b0 2024-04-24T23:58:53.999 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4db006be30 msgr2=0x7f4db006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4db006be30 0x7f4db006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dc41588e0 tx=0x7f4db8008040 comp rx=0 tx=0).stop 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 msgr2=0x7f4dc414e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dc414e1f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4db4034040 tx=0x7f4dc415f9a0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 shutdown_connections 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4db006be30 0x7f4db006e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dc40b18d0 0x7f4dc40b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4dc414f7c0 0x7f4dc414d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 --2- 172.21.15.113:0/327087223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dac0039b0 0x7f4dc414e1f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 >> 172.21.15.113:0/327087223 conn(0x7f4dc406b0f0 msgr2=0x7f4dc40b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:54.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 shutdown_connections 2024-04-24T23:58:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.001+0000 7f4dc9d7e640 1 -- 172.21.15.113:0/327087223 wait complete. 2024-04-24T23:58:54.012 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.112+0000 7f0a2825d640 1 Processor -- start 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.112+0000 7f0a2825d640 1 -- start start 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a2825d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 0x7f0a2015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a2825d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a2015f4f0 con 0x7f0a200b4a40 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a2825d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a2015f7d0 con 0x7f0a200b3080 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a2825d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a200cde00 con 0x7f0a2015e2d0 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a267fa640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0a200b4a40 0x7f0a200ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43230/0 (socket says 172.21.15.113:43230) 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a267fa640 1 -- 172.21.15.113:0/3010529202 learned_addr learned my addr 172.21.15.113:0/3010529202 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698306145 0 0) 0x7f0a2015f4f0 con 0x7f0a200b4a40 2024-04-24T23:58:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a25ff9640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 0x7f0a2015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.113+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a0c003520 con 0x7f0a200b4a40 2024-04-24T23:58:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3836862947 0 0) 0x7f0a0c003520 con 0x7f0a200b4a40 2024-04-24T23:58:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 msgr2=0x7f0a2015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 0x7f0a2015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 >> v1:172.21.15.178:6789/0 conn(0x7f0a200b3080 legacy=0x7f0a200b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2015fab0 con 0x7f0a200b4a40 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0a18001670 con 0x7f0a200b4a40 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a25ff9640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 0x7f0a2015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 >> v1:172.21.15.113:6789/0 conn(0x7f0a200b4a40 legacy=0x7f0a200ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a0c0039e0 0x7f0a0c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0c004800 0x7f0a0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 0x7f0a0c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a200cde00 con 0x7f0a0c004800 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c00bad0 con 0x7f0a0c0039e0 2024-04-24T23:58:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0c00bcd0 con 0x7f0a2015e2d0 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a257f8640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0c004800 0x7f0a0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a25ff9640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a0c0039e0 0x7f0a0c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a267fa640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 0x7f0a0c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a257f8640 1 -- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 msgr2=0x7f0a0c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a257f8640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a2015e2d0 0x7f0a0c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a257f8640 1 -- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a0c0039e0 msgr2=0x7f0a0c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a257f8640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a0c0039e0 0x7f0a0c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.114+0000 7f0a257f8640 1 -- 172.21.15.113:0/3010529202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c003520 con 0x7f0a0c004800 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.115+0000 7f0a257f8640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0c004800 0x7f0a0c004c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a10001720 tx=0x7f0a0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2844b1164a7a1ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.115+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10001370 con 0x7f0a0c004800 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.115+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a100052d0 con 0x7f0a0c004800 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.115+0000 7f0a24ff7640 1 -- 172.21.15.113:0/3010529202 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1002abc0 con 0x7f0a0c004800 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.115+0000 7f0a2825d640 1 -- 172.21.15.113:0/3010529202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0c004800 msgr2=0x7f0a0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.115+0000 7f0a2825d640 1 --2- 172.21.15.113:0/3010529202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0c004800 0x7f0a0c004c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a10001720 tx=0x7f0a0c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:54.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.115+0000 7f0a267fa640 1 -- 172.21.15.113:0/3010529202 reap_dead start 2024-04-24T23:58:54.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.116+0000 7f0a2825d640 1 -- 172.21.15.113:0/3010529202 shutdown_connections 2024-04-24T23:58:54.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.116+0000 7f0a2825d640 1 -- 172.21.15.113:0/3010529202 >> 172.21.15.113:0/3010529202 conn(0x7f0a2006b0f0 msgr2=0x7f0a2006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:54.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.116+0000 7f0a2825d640 1 -- 172.21.15.113:0/3010529202 shutdown_connections 2024-04-24T23:58:54.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.116+0000 7f0a2825d640 1 -- 172.21.15.113:0/3010529202 wait complete. 2024-04-24T23:58:54.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 Processor -- start 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 -- start start 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a0c004800 0x7f0a201e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a201e4990 0x7f0a200aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a200aebe0 0x7f0a20156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2015e830 con 0x7f0a200aebe0 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2015e6b0 con 0x7f0a201e4990 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.117+0000 7f0a2825d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2015e9b0 con 0x7f0a0c004800 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a200aebe0 0x7f0a20156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a200aebe0 0x7f0a20156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59494/0 (socket says 172.21.15.113:59494) 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 -- 172.21.15.113:0/2359468317 learned_addr learned my addr 172.21.15.113:0/2359468317 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a25ff9640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a0c004800 0x7f0a201e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a257f8640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a201e4990 0x7f0a200aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 -- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a0c004800 msgr2=0x7f0a201e5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a0c004800 0x7f0a201e5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 -- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a201e4990 msgr2=0x7f0a200aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a201e4990 0x7f0a200aa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 -- 172.21.15.113:0/2359468317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a201e4ee0 con 0x7f0a200aebe0 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a257f8640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a201e4990 0x7f0a200aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a267fa640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a200aebe0 0x7f0a20156200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a18000c00 tx=0x7f0a18007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a167fc640 1 -- 172.21.15.113:0/2359468317 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800f8b0 con 0x7f0a200aebe0 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a167fc640 1 -- 172.21.15.113:0/2359468317 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a18018070 con 0x7f0a200aebe0 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a167fc640 1 -- 172.21.15.113:0/2359468317 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800fcd0 con 0x7f0a200aebe0 2024-04-24T23:58:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a201e51c0 con 0x7f0a200aebe0 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.118+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a201e5780 con 0x7f0a200aebe0 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.119+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a200691a0 con 0x7f0a200aebe0 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.119+0000 7f0a167fc640 1 -- 172.21.15.113:0/2359468317 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a1801d460 con 0x7f0a200aebe0 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.120+0000 7f0a167fc640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a0406be30 0x7f0a0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.120+0000 7f0a167fc640 1 -- 172.21.15.113:0/2359468317 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a1808aba0 con 0x7f0a200aebe0 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.120+0000 7f0a25ff9640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a0406be30 0x7f0a0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.120+0000 7f0a25ff9640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a0406be30 0x7f0a0406e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0095c0 tx=0x7f0a1c009820 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.124+0000 7f0a167fc640 1 -- 172.21.15.113:0/2359468317 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a180595e0 con 0x7f0a200aebe0 2024-04-24T23:58:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.295+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a2015d740 con 0x7f0a200aebe0 2024-04-24T23:58:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.296+0000 7f0a167fc640 1 -- 172.21.15.113:0/2359468317 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f0a18058ee0 con 0x7f0a200aebe0 2024-04-24T23:58:54.296 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:54.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a0406be30 msgr2=0x7f0a0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a0406be30 0x7f0a0406e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0095c0 tx=0x7f0a1c009820 comp rx=0 tx=0).stop 2024-04-24T23:58:54.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a200aebe0 msgr2=0x7f0a20156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a200aebe0 0x7f0a20156200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a18000c00 tx=0x7f0a18007720 comp rx=0 tx=0).stop 2024-04-24T23:58:54.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 shutdown_connections 2024-04-24T23:58:54.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a0406be30 0x7f0a0406e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a201e4990 0x7f0a200aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a200aebe0 0x7f0a20156200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 --2- 172.21.15.113:0/2359468317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a0c004800 0x7f0a201e5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 >> 172.21.15.113:0/2359468317 conn(0x7f0a2006b0f0 msgr2=0x7f0a200b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 shutdown_connections 2024-04-24T23:58:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.298+0000 7f0a2825d640 1 -- 172.21.15.113:0/2359468317 wait complete. 2024-04-24T23:58:54.309 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:54.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.409+0000 7fe02a330640 1 Processor -- start 2024-04-24T23:58:54.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.409+0000 7fe02a330640 1 -- start start 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.409+0000 7fe02a330640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe02415e510 0x7fe0240d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.409+0000 7fe02a330640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0240d28c0 0x7fe0240b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.409+0000 7fe02a330640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0240d2e00 con 0x7fe0240b61e0 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.409+0000 7fe02a330640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe0240d30e0 con 0x7fe0240d28c0 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.409+0000 7fe02a330640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe0240d33c0 con 0x7fe02415e510 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0240d28c0 0x7fe0240b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0240d28c0 0x7fe0240b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59502/0 (socket says 172.21.15.113:59502) 2024-04-24T23:58:54.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 -- 172.21.15.113:0/1715114392 learned_addr learned my addr 172.21.15.113:0/1715114392 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:54.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe023fff640 1 --2- 172.21.15.113:0/1715114392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe02415e510 0x7fe0240d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 -- 172.21.15.113:0/1715114392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe02415e510 msgr2=0x7fe0240d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:54.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 --2- 172.21.15.113:0/1715114392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe02415e510 0x7fe0240d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 -- 172.21.15.113:0/1715114392 >> v1:172.21.15.178:6789/0 conn(0x7fe0240b61e0 legacy=0x7fe024071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:54.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 -- 172.21.15.113:0/1715114392 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0240d36a0 con 0x7fe0240d28c0 2024-04-24T23:58:54.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.410+0000 7fe0237fe640 1 --2- 172.21.15.113:0/1715114392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240d28c0 0x7fe0240b5ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe018009870 tx=0x7fe01802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0d0eabf44ba48a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe022ffd640 1 -- 172.21.15.113:0/1715114392 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018037070 con 0x7fe0240d28c0 2024-04-24T23:58:54.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe022ffd640 1 -- 172.21.15.113:0/1715114392 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe018032440 con 0x7fe0240d28c0 2024-04-24T23:58:54.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe022ffd640 1 -- 172.21.15.113:0/1715114392 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018032810 con 0x7fe0240d28c0 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 -- 172.21.15.113:0/1715114392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240d28c0 msgr2=0x7fe0240b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 --2- 172.21.15.113:0/1715114392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240d28c0 0x7fe0240b5ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe018009870 tx=0x7fe01802a180 comp rx=0 tx=0).stop 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 -- 172.21.15.113:0/1715114392 shutdown_connections 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 --2- 172.21.15.113:0/1715114392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240d28c0 0x7fe0240b5ca0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 --2- 172.21.15.113:0/1715114392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe02415e510 0x7fe0240d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 -- 172.21.15.113:0/1715114392 >> 172.21.15.113:0/1715114392 conn(0x7fe02406b720 msgr2=0x7fe0240d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 -- 172.21.15.113:0/1715114392 shutdown_connections 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.411+0000 7fe02a330640 1 -- 172.21.15.113:0/1715114392 wait complete. 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.412+0000 7fe02a330640 1 Processor -- start 2024-04-24T23:58:54.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe02a330640 1 -- start start 2024-04-24T23:58:54.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe02a330640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240b61e0 0x7fe02414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe02a330640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0240d28c0 0x7fe0240ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe02a330640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe02415e510 0x7fe024156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe02a330640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0240b9dd0 con 0x7fe0240b61e0 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe02a330640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0240d3ea0 con 0x7fe02415e510 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe02a330640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe0240b83d0 con 0x7fe0240d28c0 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe023fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240b61e0 0x7fe02414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe023fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240b61e0 0x7fe02414c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59514/0 (socket says 172.21.15.113:59514) 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.413+0000 7fe023fff640 1 -- 172.21.15.113:0/456222807 learned_addr learned my addr 172.21.15.113:0/456222807 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe023fff640 1 -- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0240d28c0 msgr2=0x7fe0240ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe0237fe640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0240d28c0 0x7fe0240ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe0288cd640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe02415e510 0x7fe024156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe023fff640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0240d28c0 0x7fe0240ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe023fff640 1 -- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe02415e510 msgr2=0x7fe024156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe023fff640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe02415e510 0x7fe024156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe023fff640 1 -- 172.21.15.113:0/456222807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0240ca9b0 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe0237fe640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0240d28c0 0x7fe0240ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe0288cd640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe02415e510 0x7fe024156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe023fff640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240b61e0 0x7fe02414c9d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe01400d5f0 tx=0x7fe01400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe020ff9640 1 -- 172.21.15.113:0/456222807 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140125f0 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe020ff9640 1 -- 172.21.15.113:0/456222807 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe014019070 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0240c93d0 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe020ff9640 1 -- 172.21.15.113:0/456222807 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140129d0 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.414+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0240c9c70 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.415+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0240691a0 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.415+0000 7fe020ff9640 1 -- 172.21.15.113:0/456222807 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe014012b70 con 0x7fe0240b61e0 2024-04-24T23:58:54.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.416+0000 7fe020ff9640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01006be30 0x7fe01006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.416+0000 7fe0237fe640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01006be30 0x7fe01006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.416+0000 7fe020ff9640 1 -- 172.21.15.113:0/456222807 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe01408b270 con 0x7fe0240b61e0 2024-04-24T23:58:54.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.416+0000 7fe0237fe640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01006be30 0x7fe01006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe018034000 tx=0x7fe01802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.420+0000 7fe020ff9640 1 -- 172.21.15.113:0/456222807 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe014059c20 con 0x7fe0240b61e0 2024-04-24T23:58:54.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.595+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe0240c9760 con 0x7fe0240b61e0 2024-04-24T23:58:54.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.595+0000 7fe020ff9640 1 -- 172.21.15.113:0/456222807 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe014059520 con 0x7fe0240b61e0 2024-04-24T23:58:54.595 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:54.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01006be30 msgr2=0x7fe01006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01006be30 0x7fe01006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe018034000 tx=0x7fe01802e000 comp rx=0 tx=0).stop 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240b61e0 msgr2=0x7fe02414c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240b61e0 0x7fe02414c9d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe01400d5f0 tx=0x7fe01400dab0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 shutdown_connections 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01006be30 0x7fe01006e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe02415e510 0x7fe024156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0240d28c0 0x7fe0240ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 --2- 172.21.15.113:0/456222807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0240b61e0 0x7fe02414c9d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 >> 172.21.15.113:0/456222807 conn(0x7fe02406b720 msgr2=0x7fe0240d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:54.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 shutdown_connections 2024-04-24T23:58:54.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.597+0000 7fe02a330640 1 -- 172.21.15.113:0/456222807 wait complete. 2024-04-24T23:58:54.608 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:58:54.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.705+0000 7f8d038bd640 1 Processor -- start 2024-04-24T23:58:54.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.705+0000 7f8d038bd640 1 -- start start 2024-04-24T23:58:54.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d038bd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8cfc0cd4b0 0x7f8cfc06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d038bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cfc15ea00 con 0x7f8cfc061b80 2024-04-24T23:58:54.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d038bd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cfc162490 con 0x7f8cfc161fd0 2024-04-24T23:58:54.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d038bd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8cfc0cfc30 con 0x7f8cfc0cd4b0 2024-04-24T23:58:54.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d01659640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8cfc0cd4b0 0x7f8cfc06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d01e5a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8cfc061b80 0x7f8cfc15eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43234/0 (socket says 172.21.15.113:43234) 2024-04-24T23:58:54.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d01e5a640 1 -- 172.21.15.113:0/1564688240 learned_addr learned my addr 172.21.15.113:0/1564688240 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:54.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d01659640 1 -- 172.21.15.113:0/1564688240 >> v1:172.21.15.178:6789/0 conn(0x7f8cfc161fd0 legacy=0x7f8cfc0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:54.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d01659640 1 -- 172.21.15.113:0/1564688240 >> v1:172.21.15.113:6789/0 conn(0x7f8cfc061b80 legacy=0x7f8cfc15eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.706+0000 7f8d01659640 1 -- 172.21.15.113:0/1564688240 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc162770 con 0x7f8cfc0cd4b0 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.707+0000 7f8d01659640 1 --2- 172.21.15.113:0/1564688240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc06c210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80097a0 tx=0x7f8cf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5770d05301381148 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.707+0000 7f8cf3fff640 1 -- 172.21.15.113:0/1564688240 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8037070 con 0x7f8cfc0cd4b0 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.707+0000 7f8cf3fff640 1 -- 172.21.15.113:0/1564688240 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf8032440 con 0x7f8cfc0cd4b0 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.707+0000 7f8cf3fff640 1 -- 172.21.15.113:0/1564688240 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8032810 con 0x7f8cfc0cd4b0 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.708+0000 7f8d038bd640 1 -- 172.21.15.113:0/1564688240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0cd4b0 msgr2=0x7f8cfc06c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.708+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1564688240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc06c210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80097a0 tx=0x7f8cf802a1e0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.708+0000 7f8d038bd640 1 -- 172.21.15.113:0/1564688240 shutdown_connections 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.708+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1564688240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc06c210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.708+0000 7f8d038bd640 1 -- 172.21.15.113:0/1564688240 >> 172.21.15.113:0/1564688240 conn(0x7f8cfc0d0e30 msgr2=0x7f8cfc0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:54.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.708+0000 7f8d038bd640 1 -- 172.21.15.113:0/1564688240 shutdown_connections 2024-04-24T23:58:54.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.708+0000 7f8d038bd640 1 -- 172.21.15.113:0/1564688240 wait complete. 2024-04-24T23:58:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 Processor -- start 2024-04-24T23:58:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 -- start start 2024-04-24T23:58:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc061b80 0x7f8cfc0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc15f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc161fd0 0x7f8cfc15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0cf1d0 con 0x7f8cfc161fd0 2024-04-24T23:58:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc162d30 con 0x7f8cfc0cd4b0 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.709+0000 7f8d038bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc15f5f0 con 0x7f8cfc061b80 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc161fd0 0x7f8cfc15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc161fd0 0x7f8cfc15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59534/0 (socket says 172.21.15.113:59534) 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 -- 172.21.15.113:0/1521356532 learned_addr learned my addr 172.21.15.113:0/1521356532 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01659640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc061b80 0x7f8cfc0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d00e58640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 -- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc061b80 msgr2=0x7f8cfc0b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc061b80 0x7f8cfc0b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 -- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc0cd4b0 msgr2=0x7f8cfc15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 -- 172.21.15.113:0/1521356532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc0b4010 con 0x7f8cfc161fd0 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d00e58640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d01e5a640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc161fd0 0x7f8cfc15fe60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400caa0 tx=0x7f8cf400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8cf1ffb640 1 -- 172.21.15.113:0/1521356532 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4017070 con 0x7f8cfc161fd0 2024-04-24T23:58:54.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8cf1ffb640 1 -- 172.21.15.113:0/1521356532 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf400ac00 con 0x7f8cfc161fd0 2024-04-24T23:58:54.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc0b4580 con 0x7f8cfc161fd0 2024-04-24T23:58:54.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8cf1ffb640 1 -- 172.21.15.113:0/1521356532 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4012620 con 0x7f8cfc161fd0 2024-04-24T23:58:54.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.710+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc1614d0 con 0x7f8cfc161fd0 2024-04-24T23:58:54.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.712+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cfc0691a0 con 0x7f8cfc161fd0 2024-04-24T23:58:54.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.712+0000 7f8cf1ffb640 1 -- 172.21.15.113:0/1521356532 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cf401a070 con 0x7f8cfc161fd0 2024-04-24T23:58:54.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.712+0000 7f8cf1ffb640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8ce006be60 0x7f8ce006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:54.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.712+0000 7f8cf1ffb640 1 -- 172.21.15.113:0/1521356532 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8cf408aa10 con 0x7f8cfc161fd0 2024-04-24T23:58:54.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.712+0000 7f8d01659640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8ce006be60 0x7f8ce006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:54.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.713+0000 7f8d01659640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8ce006be60 0x7f8ce006e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8034000 tx=0x7f8cf802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.717+0000 7f8cf1ffb640 1 -- 172.21.15.113:0/1521356532 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8cf4058e00 con 0x7f8cfc161fd0 2024-04-24T23:58:54.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.901+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8cfc15f0f0 con 0x7f8cfc161fd0 2024-04-24T23:58:54.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.901+0000 7f8cf1ffb640 1 -- 172.21.15.113:0/1521356532 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f8cf405d100 con 0x7f8cfc161fd0 2024-04-24T23:58:54.901 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-24T23:58:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8ce006be60 msgr2=0x7f8ce006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8ce006be60 0x7f8ce006e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8034000 tx=0x7f8cf802e000 comp rx=0 tx=0).stop 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc161fd0 msgr2=0x7f8cfc15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc161fd0 0x7f8cfc15fe60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400caa0 tx=0x7f8cf400cf60 comp rx=0 tx=0).stop 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 shutdown_connections 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8ce006be60 0x7f8ce006e320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc0cd4b0 0x7f8cfc15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc161fd0 0x7f8cfc15fe60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 --2- 172.21.15.113:0/1521356532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc061b80 0x7f8cfc0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 >> 172.21.15.113:0/1521356532 conn(0x7f8cfc0d0e30 msgr2=0x7f8cfc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 shutdown_connections 2024-04-24T23:58:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:54.903+0000 7f8d038bd640 1 -- 172.21.15.113:0/1521356532 wait complete. 2024-04-24T23:58:54.914 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-24T23:58:54.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:54.914 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-24T23:58:54.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T23:58:54.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:54.938 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T23:58:54.992 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:54.992 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:55.011 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:55.011 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:55.072 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-24T23:58:55.072 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:58:55.129 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:55.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.225+0000 7f77bb4ee640 1 Processor -- start 2024-04-24T23:58:55.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.226+0000 7f77bb4ee640 1 -- start start 2024-04-24T23:58:55.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.226+0000 7f77bb4ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 0x7f77b40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.226+0000 7f77bb4ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b40d2800 con 0x7f77b415e320 2024-04-24T23:58:55.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.226+0000 7f77bb4ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b40d2ae0 con 0x7f77b40d0b00 2024-04-24T23:58:55.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.226+0000 7f77bb4ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77b40d3410 con 0x7f77b40d2290 2024-04-24T23:58:55.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.226+0000 7f77b928a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f77b415e320 0x7f77b40d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43238/0 (socket says 172.21.15.113:43238) 2024-04-24T23:58:55.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.226+0000 7f77b928a640 1 -- 172.21.15.113:0/2751547134 learned_addr learned my addr 172.21.15.113:0/2751547134 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:55.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906955297 0 0) 0x7f77b40d2800 con 0x7f77b415e320 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a0003520 con 0x7f77b415e320 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b8a89640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 0x7f77b40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 714647156 0 0) 0x7f77a0003520 con 0x7f77b415e320 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 msgr2=0x7f77b40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 0x7f77b40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 >> v1:172.21.15.178:6789/0 conn(0x7f77b40d0b00 legacy=0x7f77b40b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b40d2dc0 con 0x7f77b415e320 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f77b0001670 con 0x7f77b415e320 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 >> v1:172.21.15.113:6789/0 conn(0x7f77b415e320 legacy=0x7f77b40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a00039e0 0x7f77a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 0x7f77a0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77b40d3410 con 0x7f77a0004800 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77a000bb80 con 0x7f77a00039e0 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77a000bd80 con 0x7f77b40d2290 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b8a89640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b928a640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a00039e0 0x7f77a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b9a8b640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 0x7f77a0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b8a89640 1 -- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 msgr2=0x7f77a0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b8a89640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 0x7f77a0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b8a89640 1 -- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a00039e0 msgr2=0x7f77a0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b8a89640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77a00039e0 0x7f77a0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b8a89640 1 -- 172.21.15.113:0/2751547134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a0003520 con 0x7f77a0004800 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.227+0000 7f77b9a8b640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40d2290 0x7f77a0005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.228+0000 7f77b8a89640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77a0004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ba30 tx=0x7f77a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a61fdcb2534972b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.228+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4030070 con 0x7f77a0004800 2024-04-24T23:58:55.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.228+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a402ade0 con 0x7f77a0004800 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.228+0000 7f77abfff640 1 -- 172.21.15.113:0/2751547134 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a400b600 con 0x7f77a0004800 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.229+0000 7f77bb4ee640 1 -- 172.21.15.113:0/2751547134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 msgr2=0x7f77a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.229+0000 7f77bb4ee640 1 --2- 172.21.15.113:0/2751547134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77a0004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ba30 tx=0x7f77a0005830 comp rx=0 tx=0).stop 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.229+0000 7f77b9a8b640 1 -- 172.21.15.113:0/2751547134 reap_dead start 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.229+0000 7f77bb4ee640 1 -- 172.21.15.113:0/2751547134 shutdown_connections 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.229+0000 7f77bb4ee640 1 -- 172.21.15.113:0/2751547134 >> 172.21.15.113:0/2751547134 conn(0x7f77b40b5c20 msgr2=0x7f77b40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.229+0000 7f77bb4ee640 1 -- 172.21.15.113:0/2751547134 shutdown_connections 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.229+0000 7f77bb4ee640 1 -- 172.21.15.113:0/2751547134 wait complete. 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.230+0000 7f77bb4ee640 1 Processor -- start 2024-04-24T23:58:55.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.230+0000 7f77bb4ee640 1 -- start start 2024-04-24T23:58:55.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77bb4ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77b41613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77bb4ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40cdd10 0x7f77b40cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77bb4ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77b40cce50 0x7f77b40cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77bb4ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77b415e880 con 0x7f77a0004800 2024-04-24T23:58:55.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77bb4ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77b415e700 con 0x7f77b40cce50 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77bb4ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77b40d0f90 con 0x7f77b40cdd10 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77b41613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77b41613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59560/0 (socket says 172.21.15.113:59560) 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 -- 172.21.15.113:0/450283567 learned_addr learned my addr 172.21.15.113:0/450283567 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 -- 172.21.15.113:0/450283567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40cdd10 msgr2=0x7f77b40cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40cdd10 0x7f77b40cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 -- 172.21.15.113:0/450283567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77b40cce50 msgr2=0x7f77b40cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77b40cce50 0x7f77b40cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 -- 172.21.15.113:0/450283567 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b4154080 con 0x7f77a0004800 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77b928a640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77b41613b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f77b0007340 tx=0x7f77b0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77a9ffb640 1 -- 172.21.15.113:0/450283567 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b000f940 con 0x7f77a0004800 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77a9ffb640 1 -- 172.21.15.113:0/450283567 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b0018070 con 0x7f77a0004800 2024-04-24T23:58:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b4154360 con 0x7f77a0004800 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.231+0000 7f77a9ffb640 1 -- 172.21.15.113:0/450283567 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b000fd40 con 0x7f77a0004800 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.232+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b4154920 con 0x7f77a0004800 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.232+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b40691a0 con 0x7f77a0004800 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.233+0000 7f77a9ffb640 1 -- 172.21.15.113:0/450283567 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77b001d460 con 0x7f77a0004800 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.233+0000 7f77a9ffb640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f779806be60 0x7f779806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.233+0000 7f77a9ffb640 1 -- 172.21.15.113:0/450283567 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f77b008b0e0 con 0x7f77a0004800 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.233+0000 7f77b8a89640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f779806be60 0x7f779806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.234+0000 7f77b8a89640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f779806be60 0x7f779806e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ba00 tx=0x7f77a402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.237+0000 7f77a9ffb640 1 -- 172.21.15.113:0/450283567 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77b0059a90 con 0x7f77a0004800 2024-04-24T23:58:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.414+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77b40d4890 con 0x7f77a0004800 2024-04-24T23:58:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.415+0000 7f77a9ffb640 1 -- 172.21.15.113:0/450283567 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f77b0059390 con 0x7f77a0004800 2024-04-24T23:58:55.415 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:55.415 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:55.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f779806be60 msgr2=0x7f779806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f779806be60 0x7f779806e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ba00 tx=0x7f77a402c000 comp rx=0 tx=0).stop 2024-04-24T23:58:55.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 msgr2=0x7f77b41613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77b41613b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f77b0007340 tx=0x7f77b0007800 comp rx=0 tx=0).stop 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 shutdown_connections 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f779806be60 0x7f779806e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77b40cce50 0x7f77b40cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77b40cdd10 0x7f77b40cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 --2- 172.21.15.113:0/450283567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a0004800 0x7f77b41613b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 >> 172.21.15.113:0/450283567 conn(0x7f77b40b5c20 msgr2=0x7f77b40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 shutdown_connections 2024-04-24T23:58:55.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.417+0000 7f77bb4ee640 1 -- 172.21.15.113:0/450283567 wait complete. 2024-04-24T23:58:55.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:55.428 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.528+0000 7f8c2a84c640 1 Processor -- start 2024-04-24T23:58:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.528+0000 7f8c2a84c640 1 -- start start 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.528+0000 7f8c2a84c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 0x7f8c240b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.528+0000 7f8c2a84c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c240d2e00 con 0x7f8c2415e510 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.528+0000 7f8c2a84c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c240d30e0 con 0x7f8c240b73c0 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.528+0000 7f8c2a84c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8c240b6640 con 0x7f8c240b5ca0 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c23fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8c2415e510 0x7f8c240d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43250/0 (socket says 172.21.15.113:43250) 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c23fff640 1 -- 172.21.15.113:0/3756951987 learned_addr learned my addr 172.21.15.113:0/3756951987 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 0x7f8c240b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559913286 0 0) 0x7f8c240d2e00 con 0x7f8c2415e510 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c0c003520 con 0x7f8c2415e510 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3243348050 0 0) 0x7f8c0c003520 con 0x7f8c2415e510 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 msgr2=0x7f8c240b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 0x7f8c240b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 >> v1:172.21.15.178:6789/0 conn(0x7f8c240b73c0 legacy=0x7f8c240b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c240d33c0 con 0x7f8c2415e510 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8c14001670 con 0x7f8c2415e510 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 >> v1:172.21.15.113:6789/0 conn(0x7f8c2415e510 legacy=0x7f8c240d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c0c0039e0 0x7f8c0c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c0c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 0x7f8c0c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c00ba70 con 0x7f8c0c004390 2024-04-24T23:58:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c00bbf0 con 0x7f8c0c0039e0 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c00bdf0 con 0x7f8c240b5ca0 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c0c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c23fff640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c0c0039e0 0x7f8c0c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.529+0000 7f8c28de9640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 0x7f8c0c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c237fe640 1 -- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 msgr2=0x7f8c0c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 0x7f8c0c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c237fe640 1 -- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c0c0039e0 msgr2=0x7f8c0c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c0c0039e0 0x7f8c0c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c237fe640 1 -- 172.21.15.113:0/3756951987 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c003520 con 0x7f8c0c004390 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c28de9640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240b5ca0 0x7f8c0c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c0c004860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009ac0 tx=0x7f8c0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa44d8e193073ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18036070 con 0x7f8c0c004390 2024-04-24T23:58:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c18031400 con 0x7f8c0c004390 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.530+0000 7f8c22ffd640 1 -- 172.21.15.113:0/3756951987 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18009690 con 0x7f8c0c004390 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.531+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3756951987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 msgr2=0x7f8c0c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.531+0000 7f8c2a84c640 1 --2- 172.21.15.113:0/3756951987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c0c004860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009ac0 tx=0x7f8c0c005830 comp rx=0 tx=0).stop 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.531+0000 7f8c28de9640 1 -- 172.21.15.113:0/3756951987 reap_dead start 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.531+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3756951987 shutdown_connections 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.531+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3756951987 >> 172.21.15.113:0/3756951987 conn(0x7f8c2406b720 msgr2=0x7f8c240d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.531+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3756951987 shutdown_connections 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.531+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3756951987 wait complete. 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.532+0000 7f8c2a84c640 1 Processor -- start 2024-04-24T23:58:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.532+0000 7f8c2a84c640 1 -- start start 2024-04-24T23:58:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c2a84c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c241f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c2a84c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240cd570 0x7f8c24161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c2a84c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c2415be90 0x7f8c2415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c2a84c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c240d3ea0 con 0x7f8c0c004390 2024-04-24T23:58:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c2a84c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c240d3980 con 0x7f8c2415be90 2024-04-24T23:58:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c2a84c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8c240d2a90 con 0x7f8c240cd570 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c241f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c241f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59574/0 (socket says 172.21.15.113:59574) 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 -- 172.21.15.113:0/3261426658 learned_addr learned my addr 172.21.15.113:0/3261426658 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c28de9640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c2415be90 0x7f8c2415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240cd570 0x7f8c24161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 -- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240cd570 msgr2=0x7f8c24161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240cd570 0x7f8c24161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 -- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c2415be90 msgr2=0x7f8c2415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c2415be90 0x7f8c2415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c23fff640 1 -- 172.21.15.113:0/3261426658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c240cdae0 con 0x7f8c0c004390 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c28de9640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c2415be90 0x7f8c2415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.533+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240cd570 0x7f8c24161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.534+0000 7f8c23fff640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c241f6ed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004f20 tx=0x7f8c140074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.534+0000 7f8c20ff9640 1 -- 172.21.15.113:0/3261426658 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400f7b0 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.534+0000 7f8c20ff9640 1 -- 172.21.15.113:0/3261426658 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c14018070 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.534+0000 7f8c20ff9640 1 -- 172.21.15.113:0/3261426658 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400fbb0 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.534+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c240cddc0 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.534+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c241616a0 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.535+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c2415c850 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.535+0000 7f8c20ff9640 1 -- 172.21.15.113:0/3261426658 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c1400fd50 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.536+0000 7f8c20ff9640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c1006be60 0x7f8c1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.536+0000 7f8c20ff9640 1 -- 172.21.15.113:0/3261426658 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c1408ad70 con 0x7f8c0c004390 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.536+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c1006be60 0x7f8c1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.536+0000 7f8c237fe640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c1006be60 0x7f8c1006e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c18033000 tx=0x7f8c18008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.540+0000 7f8c20ff9640 1 -- 172.21.15.113:0/3261426658 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8c140597b0 con 0x7f8c0c004390 2024-04-24T23:58:55.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.714+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c240691a0 con 0x7f8c0c004390 2024-04-24T23:58:55.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.714+0000 7f8c20ff9640 1 -- 172.21.15.113:0/3261426658 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8c140590b0 con 0x7f8c0c004390 2024-04-24T23:58:55.715 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:55.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c1006be60 msgr2=0x7f8c1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c1006be60 0x7f8c1006e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c18033000 tx=0x7f8c18008030 comp rx=0 tx=0).stop 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 msgr2=0x7f8c241f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c241f6ed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004f20 tx=0x7f8c140074f0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 shutdown_connections 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c1006be60 0x7f8c1006e320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c2415be90 0x7f8c2415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c240cd570 0x7f8c24161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 --2- 172.21.15.113:0/3261426658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0c004390 0x7f8c241f6ed0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 >> 172.21.15.113:0/3261426658 conn(0x7f8c2406b720 msgr2=0x7f8c240b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 shutdown_connections 2024-04-24T23:58:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.716+0000 7f8c2a84c640 1 -- 172.21.15.113:0/3261426658 wait complete. 2024-04-24T23:58:55.727 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:55.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.827+0000 7fa0bffce640 1 Processor -- start 2024-04-24T23:58:55.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.827+0000 7fa0bffce640 1 -- start start 2024-04-24T23:58:55.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bffce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0b80d06c0 0x7fa0b815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bffce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa0b81604d0 0x7fa0b80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bffce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b815d6b0 con 0x7fa0b813f050 2024-04-24T23:58:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bffce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b815c6a0 con 0x7fa0b80d06c0 2024-04-24T23:58:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bffce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b815c8f0 con 0x7fa0b81604d0 2024-04-24T23:58:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bdd6a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa0b813f050 0x7fa0b80cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43260/0 (socket says 172.21.15.113:43260) 2024-04-24T23:58:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bd569640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0b80d06c0 0x7fa0b815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bdd6a640 1 -- 172.21.15.113:0/1034949211 learned_addr learned my addr 172.21.15.113:0/1034949211 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bd569640 1 -- 172.21.15.113:0/1034949211 >> v2:172.21.15.178:3300/0 conn(0x7fa0b81604d0 msgr2=0x7fa0b80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0be56b640 1 --2- 172.21.15.113:0/1034949211 >> v2:172.21.15.178:3300/0 conn(0x7fa0b81604d0 0x7fa0b80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bd569640 1 --2- 172.21.15.113:0/1034949211 >> v2:172.21.15.178:3300/0 conn(0x7fa0b81604d0 0x7fa0b80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bd569640 1 -- 172.21.15.113:0/1034949211 >> v1:172.21.15.113:6789/0 conn(0x7fa0b813f050 legacy=0x7fa0b80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bcd68640 1 -- 172.21.15.113:0/1034949211 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804172117 0 0) 0x7fa0b815d6b0 con 0x7fa0b813f050 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.828+0000 7fa0bd569640 1 -- 172.21.15.113:0/1034949211 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b815cbd0 con 0x7fa0b80d06c0 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bd569640 1 --2- 172.21.15.113:0/1034949211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b80d06c0 0x7fa0b815ff90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8156d50 tx=0x7fa0a802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d4f50ee6f640ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bcd68640 1 -- 172.21.15.113:0/1034949211 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8037070 con 0x7fa0b80d06c0 2024-04-24T23:58:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bcd68640 1 -- 172.21.15.113:0/1034949211 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a8032440 con 0x7fa0b80d06c0 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bcd68640 1 -- 172.21.15.113:0/1034949211 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a802ae60 con 0x7fa0b80d06c0 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bffce640 1 -- 172.21.15.113:0/1034949211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b80d06c0 msgr2=0x7fa0b815ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1034949211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b80d06c0 0x7fa0b815ff90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8156d50 tx=0x7fa0a802a110 comp rx=0 tx=0).stop 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bffce640 1 -- 172.21.15.113:0/1034949211 shutdown_connections 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1034949211 >> v2:172.21.15.178:3300/0 conn(0x7fa0b81604d0 0x7fa0b80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1034949211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b80d06c0 0x7fa0b815ff90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.829+0000 7fa0bffce640 1 -- 172.21.15.113:0/1034949211 >> 172.21.15.113:0/1034949211 conn(0x7fa0b806b0f0 msgr2=0x7fa0b806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.830+0000 7fa0bffce640 1 -- 172.21.15.113:0/1034949211 shutdown_connections 2024-04-24T23:58:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.830+0000 7fa0bffce640 1 -- 172.21.15.113:0/1034949211 wait complete. 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 Processor -- start 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 -- start start 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0b80d06c0 0x7fa0b814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0b813f050 0x7fa0b81f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b81604d0 0x7fa0b813b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b813f430 con 0x7fa0b81604d0 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b815d190 con 0x7fa0b813f050 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0bffce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b8160960 con 0x7fa0b80d06c0 2024-04-24T23:58:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0be56b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b81604d0 0x7fa0b813b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0be56b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b81604d0 0x7fa0b813b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59596/0 (socket says 172.21.15.113:59596) 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.831+0000 7fa0be56b640 1 -- 172.21.15.113:0/1024558451 learned_addr learned my addr 172.21.15.113:0/1024558451 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0bdd6a640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0b80d06c0 0x7fa0b814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0bd569640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0b813f050 0x7fa0b81f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0be56b640 1 -- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0b80d06c0 msgr2=0x7fa0b814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0be56b640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0b80d06c0 0x7fa0b814fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0be56b640 1 -- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0b813f050 msgr2=0x7fa0b81f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0be56b640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0b813f050 0x7fa0b81f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0be56b640 1 -- 172.21.15.113:0/1024558451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b8159360 con 0x7fa0b81604d0 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0be56b640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b81604d0 0x7fa0b813b270 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000cc60 tx=0x7fa0b000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0ae7fc640 1 -- 172.21.15.113:0/1024558451 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003990 con 0x7fa0b81604d0 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0ae7fc640 1 -- 172.21.15.113:0/1024558451 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b0003b30 con 0x7fa0b81604d0 2024-04-24T23:58:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0ae7fc640 1 -- 172.21.15.113:0/1024558451 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0004020 con 0x7fa0b81604d0 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b8159640 con 0x7fa0b81604d0 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.832+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b8159c00 con 0x7fa0b81604d0 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.833+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b80cda80 con 0x7fa0b81604d0 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.833+0000 7fa0ae7fc640 1 -- 172.21.15.113:0/1024558451 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0b00041c0 con 0x7fa0b81604d0 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.834+0000 7fa0ae7fc640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa09c06be30 0x7fa09c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.834+0000 7fa0ae7fc640 1 -- 172.21.15.113:0/1024558451 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa0b008b360 con 0x7fa0b81604d0 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.834+0000 7fa0bdd6a640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa09c06be30 0x7fa09c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.834+0000 7fa0bdd6a640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa09c06be30 0x7fa09c06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4007fd0 tx=0x7fa0b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:55.838+0000 7fa0ae7fc640 1 -- 172.21.15.113:0/1024558451 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0b80cda80 con 0x7fa0b81604d0 2024-04-24T23:58:56.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.010+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa0b80691a0 con 0x7fa0b81604d0 2024-04-24T23:58:56.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.011+0000 7fa0ae7fc640 1 -- 172.21.15.113:0/1024558451 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa0b0014030 con 0x7fa0b81604d0 2024-04-24T23:58:56.011 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:56.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa09c06be30 msgr2=0x7fa09c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa09c06be30 0x7fa09c06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4007fd0 tx=0x7fa0b4009040 comp rx=0 tx=0).stop 2024-04-24T23:58:56.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b81604d0 msgr2=0x7fa0b813b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b81604d0 0x7fa0b813b270 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000cc60 tx=0x7fa0b000a5c0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 shutdown_connections 2024-04-24T23:58:56.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa09c06be30 0x7fa09c06e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0b813f050 0x7fa0b81f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b81604d0 0x7fa0b813b270 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 --2- 172.21.15.113:0/1024558451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0b80d06c0 0x7fa0b814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 >> 172.21.15.113:0/1024558451 conn(0x7fa0b806b0f0 msgr2=0x7fa0b815d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:56.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 shutdown_connections 2024-04-24T23:58:56.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.013+0000 7fa0bffce640 1 -- 172.21.15.113:0/1024558451 wait complete. 2024-04-24T23:58:56.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:56.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.123+0000 7fe8b875a640 1 Processor -- start 2024-04-24T23:58:56.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.123+0000 7fe8b875a640 1 -- start start 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b875a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe8b015c050 0x7fe8b015c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b875a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b875a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b0160900 con 0x7fe8b006f180 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b875a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b006d870 con 0x7fe8b015c050 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b875a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b006db50 con 0x7fe8b0074c00 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b64f6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe8b015c050 0x7fe8b015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b64f6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe8b015c050 0x7fe8b015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59604/0 (socket says 172.21.15.113:59604) 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b64f6640 1 -- 172.21.15.113:0/3651075648 learned_addr learned my addr 172.21.15.113:0/3651075648 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b5cf5640 1 --2- 172.21.15.113:0/3651075648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b64f6640 1 -- 172.21.15.113:0/3651075648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 msgr2=0x7fe8b00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b64f6640 1 --2- 172.21.15.113:0/3651075648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b00cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b64f6640 1 -- 172.21.15.113:0/3651075648 >> v1:172.21.15.178:6789/0 conn(0x7fe8b006f180 legacy=0x7fe8b006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.124+0000 7fe8b64f6640 1 -- 172.21.15.113:0/3651075648 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b0160be0 con 0x7fe8b015c050 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.125+0000 7fe8b64f6640 1 --2- 172.21.15.113:0/3651075648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b015c430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009870 tx=0x7fe8ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fe04c6c1e0a6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.125+0000 7fe8b54f4640 1 -- 172.21.15.113:0/3651075648 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac037070 con 0x7fe8b015c050 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.125+0000 7fe8b54f4640 1 -- 172.21.15.113:0/3651075648 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ac032440 con 0x7fe8b015c050 2024-04-24T23:58:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.125+0000 7fe8b54f4640 1 -- 172.21.15.113:0/3651075648 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac032810 con 0x7fe8b015c050 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.125+0000 7fe8b875a640 1 -- 172.21.15.113:0/3651075648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 msgr2=0x7fe8b015c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.125+0000 7fe8b875a640 1 --2- 172.21.15.113:0/3651075648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b015c430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009870 tx=0x7fe8ac02a270 comp rx=0 tx=0).stop 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.126+0000 7fe8b875a640 1 -- 172.21.15.113:0/3651075648 shutdown_connections 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.126+0000 7fe8b875a640 1 --2- 172.21.15.113:0/3651075648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b00cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.126+0000 7fe8b875a640 1 --2- 172.21.15.113:0/3651075648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b015c430 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.126+0000 7fe8b875a640 1 -- 172.21.15.113:0/3651075648 >> 172.21.15.113:0/3651075648 conn(0x7fe8b006fb60 msgr2=0x7fe8b006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.126+0000 7fe8b875a640 1 -- 172.21.15.113:0/3651075648 shutdown_connections 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.126+0000 7fe8b875a640 1 -- 172.21.15.113:0/3651075648 wait complete. 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 Processor -- start 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 -- start start 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b006f180 0x7fe8b01541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b015caf0 con 0x7fe8b015c050 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b015c970 con 0x7fe8b006f180 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.127+0000 7fe8b875a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b00cf930 con 0x7fe8b0074c00 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b0151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59616/0 (socket says 172.21.15.113:59616) 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 -- 172.21.15.113:0/2311213101 learned_addr learned my addr 172.21.15.113:0/2311213101 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b5cf5640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b64f6640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b006f180 0x7fe8b01541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 -- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 msgr2=0x7fe8b015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 -- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b006f180 msgr2=0x7fe8b01541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b006f180 0x7fe8b01541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 -- 172.21.15.113:0/2311213101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b01f5490 con 0x7fe8b015c050 2024-04-24T23:58:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b6cf7640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b0151f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c7a0 tx=0x7fe8a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8a6ffd640 1 -- 172.21.15.113:0/2311213101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8003b70 con 0x7fe8b015c050 2024-04-24T23:58:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8a6ffd640 1 -- 172.21.15.113:0/2311213101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a8003d30 con 0x7fe8b015c050 2024-04-24T23:58:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b0155370 con 0x7fe8b015c050 2024-04-24T23:58:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8a6ffd640 1 -- 172.21.15.113:0/2311213101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800aa90 con 0x7fe8b015c050 2024-04-24T23:58:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.128+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b0155090 con 0x7fe8b015c050 2024-04-24T23:58:56.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.129+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b00691a0 con 0x7fe8b015c050 2024-04-24T23:58:56.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.130+0000 7fe8a6ffd640 1 -- 172.21.15.113:0/2311213101 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8a800ac30 con 0x7fe8b015c050 2024-04-24T23:58:56.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.130+0000 7fe8a6ffd640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89406be30 0x7fe89406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.130+0000 7fe8a6ffd640 1 -- 172.21.15.113:0/2311213101 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8a808ac70 con 0x7fe8b015c050 2024-04-24T23:58:56.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.130+0000 7fe8b64f6640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89406be30 0x7fe89406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.130+0000 7fe8b64f6640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89406be30 0x7fe89406e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac034000 tx=0x7fe8ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.134+0000 7fe8a6ffd640 1 -- 172.21.15.113:0/2311213101 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe8a8058fd0 con 0x7fe8b015c050 2024-04-24T23:58:56.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.308+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8b01f5660 con 0x7fe8b015c050 2024-04-24T23:58:56.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.308+0000 7fe8a6ffd640 1 -- 172.21.15.113:0/2311213101 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe8a805d2d0 con 0x7fe8b015c050 2024-04-24T23:58:56.309 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.310+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89406be30 msgr2=0x7fe89406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.310+0000 7fe8b875a640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89406be30 0x7fe89406e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac034000 tx=0x7fe8ac02e000 comp rx=0 tx=0).stop 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.310+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 msgr2=0x7fe8b0151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.310+0000 7fe8b875a640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b0151f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c7a0 tx=0x7fe8a800cc60 comp rx=0 tx=0).stop 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 shutdown_connections 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89406be30 0x7fe89406e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b015c050 0x7fe8b0151f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b0074c00 0x7fe8b015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 --2- 172.21.15.113:0/2311213101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b006f180 0x7fe8b01541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 >> 172.21.15.113:0/2311213101 conn(0x7fe8b006fb60 msgr2=0x7fe8b00b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:56.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 shutdown_connections 2024-04-24T23:58:56.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.311+0000 7fe8b875a640 1 -- 172.21.15.113:0/2311213101 wait complete. 2024-04-24T23:58:56.321 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:58:56.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.419+0000 7f3f5d16a640 1 Processor -- start 2024-04-24T23:58:56.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.419+0000 7f3f5d16a640 1 -- start start 2024-04-24T23:58:56.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5d16a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5d16a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f580b63b0 0x7f3f580b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5d16a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f580d2e00 con 0x7f3f580d28c0 2024-04-24T23:58:56.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5d16a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3f580d30e0 con 0x7f3f580b63b0 2024-04-24T23:58:56.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5d16a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3f580d33c0 con 0x7f3f5815e510 2024-04-24T23:58:56.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f580b63b0 0x7f3f580b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3f580d28c0 0x7f3f580b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43274/0 (socket says 172.21.15.113:43274) 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f580b63b0 0x7f3f580b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59622/0 (socket says 172.21.15.113:59622) 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f5659c640 1 -- 172.21.15.113:0/2670891104 learned_addr learned my addr 172.21.15.113:0/2670891104 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.420+0000 7f3f56d9d640 1 --2- 172.21.15.113:0/2670891104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f5759e640 1 -- 172.21.15.113:0/2670891104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 msgr2=0x7f3f580d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f5759e640 1 --2- 172.21.15.113:0/2670891104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f5759e640 1 -- 172.21.15.113:0/2670891104 >> v1:172.21.15.113:6789/0 conn(0x7f3f580d28c0 legacy=0x7f3f580b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f5759e640 1 -- 172.21.15.113:0/2670891104 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f580d36a0 con 0x7f3f580b63b0 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f55d9b640 1 -- 172.21.15.113:0/2670891104 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653724321 0 0) 0x7f3f580d2e00 con 0x7f3f580d28c0 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f5759e640 1 --2- 172.21.15.113:0/2670891104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580b63b0 0x7f3f580b73c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f500097a0 tx=0x7f3f5002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b2e9e66b6a4d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f55d9b640 1 -- 172.21.15.113:0/2670891104 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50037070 con 0x7f3f580b63b0 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f55d9b640 1 -- 172.21.15.113:0/2670891104 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5002ae50 con 0x7f3f580b63b0 2024-04-24T23:58:56.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.421+0000 7f3f55d9b640 1 -- 172.21.15.113:0/2670891104 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50032660 con 0x7f3f580b63b0 2024-04-24T23:58:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 -- 172.21.15.113:0/2670891104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580b63b0 msgr2=0x7f3f580b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/2670891104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580b63b0 0x7f3f580b73c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f500097a0 tx=0x7f3f5002a100 comp rx=0 tx=0).stop 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 -- 172.21.15.113:0/2670891104 shutdown_connections 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/2670891104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580b63b0 0x7f3f580b73c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/2670891104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 -- 172.21.15.113:0/2670891104 >> 172.21.15.113:0/2670891104 conn(0x7f3f5806b720 msgr2=0x7f3f580d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 -- 172.21.15.113:0/2670891104 shutdown_connections 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.422+0000 7f3f5d16a640 1 -- 172.21.15.113:0/2670891104 wait complete. 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.423+0000 7f3f5d16a640 1 Processor -- start 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.423+0000 7f3f5d16a640 1 -- start start 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5d16a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f580b63b0 0x7f3f58154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5d16a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580d28c0 0x7f3f580ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5d16a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5d16a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f580b9dd0 con 0x7f3f580d28c0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5d16a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f580d3ea0 con 0x7f3f580b63b0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5d16a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3f580b7900 con 0x7f3f5815e510 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580d28c0 0x7f3f580ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580d28c0 0x7f3f580ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59626/0 (socket says 172.21.15.113:59626) 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 -- 172.21.15.113:0/1828986420 learned_addr learned my addr 172.21.15.113:0/1828986420 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5759e640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f56d9d640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f580b63b0 0x7f3f58154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 -- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 msgr2=0x7f3f580ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 -- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f580b63b0 msgr2=0x7f3f58154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f580b63b0 0x7f3f58154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.424+0000 7f3f5659c640 1 -- 172.21.15.113:0/1828986420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f581f40d0 con 0x7f3f580d28c0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.425+0000 7f3f5659c640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580d28c0 0x7f3f580ce7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c4d0 tx=0x7f3f4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.425+0000 7f3f3f7fe640 1 -- 172.21.15.113:0/1828986420 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c018070 con 0x7f3f580d28c0 2024-04-24T23:58:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.425+0000 7f3f3f7fe640 1 -- 172.21.15.113:0/1828986420 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4c007500 con 0x7f3f580d28c0 2024-04-24T23:58:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.425+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f581f43b0 con 0x7f3f580d28c0 2024-04-24T23:58:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.425+0000 7f3f3f7fe640 1 -- 172.21.15.113:0/1828986420 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c004a20 con 0x7f3f580d28c0 2024-04-24T23:58:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.425+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f581f3dc0 con 0x7f3f580d28c0 2024-04-24T23:58:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.426+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f580691a0 con 0x7f3f580d28c0 2024-04-24T23:58:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.426+0000 7f3f3f7fe640 1 -- 172.21.15.113:0/1828986420 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f4c01b070 con 0x7f3f580d28c0 2024-04-24T23:58:56.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.427+0000 7f3f3f7fe640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3f4406be30 0x7f3f4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.427+0000 7f3f3f7fe640 1 -- 172.21.15.113:0/1828986420 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f4c08a070 con 0x7f3f580d28c0 2024-04-24T23:58:56.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.427+0000 7f3f56d9d640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3f4406be30 0x7f3f4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.427+0000 7f3f56d9d640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3f4406be30 0x7f3f4406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f48003aa0 tx=0x7f3f48009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.431+0000 7f3f3f7fe640 1 -- 172.21.15.113:0/1828986420 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f4c058460 con 0x7f3f580d28c0 2024-04-24T23:58:56.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.615+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3f58155c00 con 0x7f3f580d28c0 2024-04-24T23:58:56.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.615+0000 7f3f3f7fe640 1 -- 172.21.15.113:0/1828986420 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f3f4c05c760 con 0x7f3f580d28c0 2024-04-24T23:58:56.616 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3f4406be30 msgr2=0x7f3f4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3f4406be30 0x7f3f4406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f48003aa0 tx=0x7f3f48009830 comp rx=0 tx=0).stop 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580d28c0 msgr2=0x7f3f580ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580d28c0 0x7f3f580ce7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c4d0 tx=0x7f3f4c00c990 comp rx=0 tx=0).stop 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 shutdown_connections 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3f4406be30 0x7f3f4406e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f5815e510 0x7f3f580ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580d28c0 0x7f3f580ce7c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 --2- 172.21.15.113:0/1828986420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f580b63b0 0x7f3f58154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.617+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 >> 172.21.15.113:0/1828986420 conn(0x7f3f5806b720 msgr2=0x7f3f5815d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.618+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 shutdown_connections 2024-04-24T23:58:56.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.618+0000 7f3f5d16a640 1 -- 172.21.15.113:0/1828986420 wait complete. 2024-04-24T23:58:56.628 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-24T23:58:56.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:56.629 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-24T23:58:56.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2024-04-24T23:58:56.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:58:56.659 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T23:58:56.717 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T23:58:56.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.814+0000 7eff8e4bd640 1 Processor -- start 2024-04-24T23:58:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.814+0000 7eff8e4bd640 1 -- start start 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff8e4bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff88072ea0 0x7eff88071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff8e4bd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7eff880b3af0 0x7eff880b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff8e4bd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff880b2d50 con 0x7eff8815a6b0 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff8e4bd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7eff88072a70 con 0x7eff880b3af0 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff8e4bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7eff880b3010 con 0x7eff88072ea0 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff8ca5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7eff880b3af0 0x7eff880b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff877fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff88072ea0 0x7eff88071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff877fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff88072ea0 0x7eff88071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:57210/0 (socket says 172.21.15.113:57210) 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff877fe640 1 -- 172.21.15.113:0/1986662250 learned_addr learned my addr 172.21.15.113:0/1986662250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.815+0000 7eff8ca5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7eff880b3af0 0x7eff880b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:52860/0 (socket says 172.21.15.113:52860) 2024-04-24T23:58:56.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.816+0000 7eff8ca5a640 1 -- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff88072ea0 msgr2=0x7eff88071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:56.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.816+0000 7eff8ca5a640 1 --2- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff88072ea0 0x7eff88071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.816+0000 7eff8ca5a640 1 -- 172.21.15.113:0/1986662250 >> v1:172.21.15.178:6789/0 conn(0x7eff8815a6b0 legacy=0x7eff88070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.816+0000 7eff8ca5a640 1 -- 172.21.15.113:0/1986662250 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff880b32f0 con 0x7eff880b3af0 2024-04-24T23:58:56.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.816+0000 7eff877fe640 1 --2- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff88072ea0 0x7eff88071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:56.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.816+0000 7eff8ca5a640 1 --2- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff880b3f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff80009d90 tx=0x7eff8002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d02ffbd6bbd4b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff86ffd640 1 -- 172.21.15.113:0/1986662250 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80007740 con 0x7eff880b3af0 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff86ffd640 1 -- 172.21.15.113:0/1986662250 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff800078e0 con 0x7eff880b3af0 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff86ffd640 1 -- 172.21.15.113:0/1986662250 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80007cb0 con 0x7eff880b3af0 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 -- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 msgr2=0x7eff880b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff880b3f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7eff80009d90 tx=0x7eff8002a660 comp rx=0 tx=0).stop 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 -- 172.21.15.113:0/1986662250 shutdown_connections 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff880b3f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/1986662250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff88072ea0 0x7eff88071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 -- 172.21.15.113:0/1986662250 >> 172.21.15.113:0/1986662250 conn(0x7eff880b1a50 msgr2=0x7eff880b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 -- 172.21.15.113:0/1986662250 shutdown_connections 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.817+0000 7eff8e4bd640 1 -- 172.21.15.113:0/1986662250 wait complete. 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.818+0000 7eff8e4bd640 1 Processor -- start 2024-04-24T23:58:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.818+0000 7eff8e4bd640 1 -- start start 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff8e4bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff88072ea0 0x7eff88145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff8e4bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff88071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff8e4bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff8815a6b0 0x7eff88151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff8e4bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff8815aa90 con 0x7eff88072ea0 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff8e4bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff88072720 con 0x7eff880b3af0 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff8e4bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7eff8815ac10 con 0x7eff8815a6b0 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff87fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff88072ea0 0x7eff88145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff87fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff88072ea0 0x7eff88145080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59636/0 (socket says 172.21.15.113:59636) 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff87fff640 1 -- 172.21.15.113:0/364814143 learned_addr learned my addr 172.21.15.113:0/364814143 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff8ca5a640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff8815a6b0 0x7eff88151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.819+0000 7eff877fe640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff88071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff87fff640 1 -- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff8815a6b0 msgr2=0x7eff88151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff87fff640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff8815a6b0 0x7eff88151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff87fff640 1 -- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 msgr2=0x7eff88071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff87fff640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff88071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff87fff640 1 -- 172.21.15.113:0/364814143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff88147110 con 0x7eff88072ea0 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff8ca5a640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff8815a6b0 0x7eff88151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff877fe640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff88071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff87fff640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff88072ea0 0x7eff88145080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c3c0 tx=0x7eff7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff84ff9640 1 -- 172.21.15.113:0/364814143 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff780052d0 con 0x7eff88072ea0 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff881455c0 con 0x7eff88072ea0 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff84ff9640 1 -- 172.21.15.113:0/364814143 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff78005490 con 0x7eff88072ea0 2024-04-24T23:58:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff84ff9640 1 -- 172.21.15.113:0/364814143 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78007260 con 0x7eff88072ea0 2024-04-24T23:58:56.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.820+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff88147690 con 0x7eff88072ea0 2024-04-24T23:58:56.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.821+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff880b1430 con 0x7eff88072ea0 2024-04-24T23:58:56.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.821+0000 7eff84ff9640 1 -- 172.21.15.113:0/364814143 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff78007400 con 0x7eff88072ea0 2024-04-24T23:58:56.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.822+0000 7eff84ff9640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff7406be30 0x7eff7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:56.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.822+0000 7eff877fe640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff7406be30 0x7eff7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:56.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.822+0000 7eff84ff9640 1 -- 172.21.15.113:0/364814143 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff7808a310 con 0x7eff88072ea0 2024-04-24T23:58:56.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.822+0000 7eff877fe640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff7406be30 0x7eff7406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009630 tx=0x7eff7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:56.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:56.826+0000 7eff84ff9640 1 -- 172.21.15.113:0/364814143 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7eff880b1430 con 0x7eff88072ea0 2024-04-24T23:58:57.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.033+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff88068a60 con 0x7eff88072ea0 2024-04-24T23:58:57.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.033+0000 7eff84ff9640 1 -- 172.21.15.113:0/364814143 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7eff88068a60 con 0x7eff88072ea0 2024-04-24T23:58:57.034 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:57.034 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:57.034 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff7406be30 msgr2=0x7eff7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff7406be30 0x7eff7406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009630 tx=0x7eff7c008040 comp rx=0 tx=0).stop 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff88072ea0 msgr2=0x7eff88145080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff88072ea0 0x7eff88145080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c3c0 tx=0x7eff7800c880 comp rx=0 tx=0).stop 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 shutdown_connections 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff7406be30 0x7eff7406e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff880b3af0 0x7eff88071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff8815a6b0 0x7eff88151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 --2- 172.21.15.113:0/364814143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff88072ea0 0x7eff88145080 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.035+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 >> 172.21.15.113:0/364814143 conn(0x7eff880b1a50 msgr2=0x7eff880b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.036+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 shutdown_connections 2024-04-24T23:58:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.036+0000 7eff8e4bd640 1 -- 172.21.15.113:0/364814143 wait complete. 2024-04-24T23:58:57.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.144+0000 7f7b0e89a640 1 Processor -- start 2024-04-24T23:58:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.144+0000 7f7b0e89a640 1 -- start start 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b0e89a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 0x7f7b081627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b0e89a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7b080b5620 0x7f7b0806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b0e89a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7b0806e510 0x7f7b08070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b0e89a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7b08072d90 con 0x7f7b080b5620 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b0e89a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0806e930 con 0x7f7b0806e510 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b0e89a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b08162d20 con 0x7f7b0815e240 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7b080b5620 0x7f7b0806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7b080b5620 0x7f7b0806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59642/0 (socket says 172.21.15.113:59642) 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 -- 172.21.15.113:0/1622914525 learned_addr learned my addr 172.21.15.113:0/1622914525 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 -- 172.21.15.113:0/1622914525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 msgr2=0x7f7b081627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 --2- 172.21.15.113:0/1622914525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 0x7f7b081627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 -- 172.21.15.113:0/1622914525 >> v2:172.21.15.178:3300/0 conn(0x7f7b0806e510 msgr2=0x7f7b08070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 --2- 172.21.15.113:0/1622914525 >> v2:172.21.15.178:3300/0 conn(0x7f7b0806e510 0x7f7b08070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.145+0000 7f7b077fe640 1 -- 172.21.15.113:0/1622914525 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b080b3c20 con 0x7f7b080b5620 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b077fe640 1 --2- 172.21.15.113:0/1622914525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b0806dfd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7b0806f980 tx=0x7f7af802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4877aed53110d980 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b06ffd640 1 -- 172.21.15.113:0/1622914525 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8037070 con 0x7f7b080b5620 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b06ffd640 1 -- 172.21.15.113:0/1622914525 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af8032440 con 0x7f7b080b5620 2024-04-24T23:58:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b06ffd640 1 -- 172.21.15.113:0/1622914525 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8032810 con 0x7f7b080b5620 2024-04-24T23:58:57.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 -- 172.21.15.113:0/1622914525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 msgr2=0x7f7b0806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/1622914525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b0806dfd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7b0806f980 tx=0x7f7af802a210 comp rx=0 tx=0).stop 2024-04-24T23:58:57.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 -- 172.21.15.113:0/1622914525 shutdown_connections 2024-04-24T23:58:57.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/1622914525 >> v2:172.21.15.178:3300/0 conn(0x7f7b0806e510 0x7f7b08070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/1622914525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b0806dfd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/1622914525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 0x7f7b081627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 -- 172.21.15.113:0/1622914525 >> 172.21.15.113:0/1622914525 conn(0x7f7b080b1a50 msgr2=0x7f7b080b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 -- 172.21.15.113:0/1622914525 shutdown_connections 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.146+0000 7f7b0e89a640 1 -- 172.21.15.113:0/1622914525 wait complete. 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 Processor -- start 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 -- start start 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b0806e510 0x7f7b0815f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b08161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 0x7f7b081e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b080724f0 con 0x7f7b080b5620 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b08072210 con 0x7f7b0806e510 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.148+0000 7f7b0e89a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b080727d0 con 0x7f7b0815e240 2024-04-24T23:58:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b08161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b08161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59658/0 (socket says 172.21.15.113:59658) 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 -- 172.21.15.113:0/2199006885 learned_addr learned my addr 172.21.15.113:0/2199006885 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b07fff640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b0806e510 0x7f7b0815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b0ce37640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 0x7f7b081e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 -- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 msgr2=0x7f7b081e9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 0x7f7b081e9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 -- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b0806e510 msgr2=0x7f7b0815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b0806e510 0x7f7b0815f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 -- 172.21.15.113:0/2199006885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0815f960 con 0x7f7b080b5620 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b077fe640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b08161240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7af8000c00 tx=0x7f7af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b04ff9640 1 -- 172.21.15.113:0/2199006885 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8041070 con 0x7f7b080b5620 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b04ff9640 1 -- 172.21.15.113:0/2199006885 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af803c870 con 0x7f7b080b5620 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0815fc40 con 0x7f7b080b5620 2024-04-24T23:58:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b04ff9640 1 -- 172.21.15.113:0/2199006885 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8037040 con 0x7f7b080b5620 2024-04-24T23:58:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.149+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b08160200 con 0x7f7b080b5620 2024-04-24T23:58:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.150+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b080691a0 con 0x7f7b080b5620 2024-04-24T23:58:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.151+0000 7f7b04ff9640 1 -- 172.21.15.113:0/2199006885 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7af8044070 con 0x7f7b080b5620 2024-04-24T23:58:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.151+0000 7f7b04ff9640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7af406be30 0x7f7af406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.151+0000 7f7b04ff9640 1 -- 172.21.15.113:0/2199006885 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7af80aba50 con 0x7f7b080b5620 2024-04-24T23:58:57.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.151+0000 7f7b07fff640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7af406be30 0x7f7af406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.151+0000 7f7b07fff640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7af406be30 0x7f7af406e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00b810 tx=0x7f7afc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.155+0000 7f7b04ff9640 1 -- 172.21.15.113:0/2199006885 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7af807a520 con 0x7f7b080b5620 2024-04-24T23:58:57.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.329+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b08161b40 con 0x7f7b080b5620 2024-04-24T23:58:57.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.330+0000 7f7b04ff9640 1 -- 172.21.15.113:0/2199006885 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f7af8079e20 con 0x7f7b080b5620 2024-04-24T23:58:57.330 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:57.331 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:57.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7af406be30 msgr2=0x7f7af406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7af406be30 0x7f7af406e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00b810 tx=0x7f7afc008040 comp rx=0 tx=0).stop 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 msgr2=0x7f7b08161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b08161240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7af8000c00 tx=0x7f7af80046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 shutdown_connections 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7af406be30 0x7f7af406e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b0815e240 0x7f7b081e9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b080b5620 0x7f7b08161240 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 --2- 172.21.15.113:0/2199006885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b0806e510 0x7f7b0815f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 >> 172.21.15.113:0/2199006885 conn(0x7f7b080b1a50 msgr2=0x7f7b0815b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 shutdown_connections 2024-04-24T23:58:57.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.332+0000 7f7b0e89a640 1 -- 172.21.15.113:0/2199006885 wait complete. 2024-04-24T23:58:57.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:58:17.001246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:58:17.549246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:58:57.344 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:58:57.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.444+0000 7f56b9190640 1 Processor -- start 2024-04-24T23:58:57.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.444+0000 7f56b9190640 1 -- start start 2024-04-24T23:58:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b9190640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f56b415e2d0 0x7f56b415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b9190640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56b40b3080 0x7f56b40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b9190640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b415e6b0 0x7f56b40b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b9190640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f56b40cde00 con 0x7f56b40b3080 2024-04-24T23:58:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b9190640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f56b415f4f0 con 0x7f56b415e2d0 2024-04-24T23:58:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b9190640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56b415f7d0 con 0x7f56b415e6b0 2024-04-24T23:58:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56b40b3080 0x7f56b40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56b40b3080 0x7f56b40b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59668/0 (socket says 172.21.15.113:59668) 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 -- 172.21.15.113:0/2581519154 learned_addr learned my addr 172.21.15.113:0/2581519154 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b2d9d640 1 --2- 172.21.15.113:0/2581519154 >> v2:172.21.15.178:3300/0 conn(0x7f56b415e2d0 0x7f56b415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b359e640 1 --2- 172.21.15.113:0/2581519154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b415e6b0 0x7f56b40b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 -- 172.21.15.113:0/2581519154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b415e6b0 msgr2=0x7f56b40b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 --2- 172.21.15.113:0/2581519154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b415e6b0 0x7f56b40b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 -- 172.21.15.113:0/2581519154 >> v2:172.21.15.178:3300/0 conn(0x7f56b415e2d0 msgr2=0x7f56b415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 --2- 172.21.15.113:0/2581519154 >> v2:172.21.15.178:3300/0 conn(0x7f56b415e2d0 0x7f56b415ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.445+0000 7f56b259c640 1 -- 172.21.15.113:0/2581519154 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b415fab0 con 0x7f56b40b3080 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.446+0000 7f56b259c640 1 --2- 172.21.15.113:0/2581519154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b40b3080 0x7f56b40b4330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f56b40b2d40 tx=0x7f569c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90c8f60ab03842b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.446+0000 7f56b1d9b640 1 -- 172.21.15.113:0/2581519154 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c037070 con 0x7f56b40b3080 2024-04-24T23:58:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.446+0000 7f56b1d9b640 1 -- 172.21.15.113:0/2581519154 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569c032440 con 0x7f56b40b3080 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.446+0000 7f56b1d9b640 1 -- 172.21.15.113:0/2581519154 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c032810 con 0x7f56b40b3080 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 -- 172.21.15.113:0/2581519154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b40b3080 msgr2=0x7f56b40b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 --2- 172.21.15.113:0/2581519154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b40b3080 0x7f56b40b4330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f56b40b2d40 tx=0x7f569c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 -- 172.21.15.113:0/2581519154 shutdown_connections 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 --2- 172.21.15.113:0/2581519154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b415e6b0 0x7f56b40b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 --2- 172.21.15.113:0/2581519154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b40b3080 0x7f56b40b4330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 --2- 172.21.15.113:0/2581519154 >> v2:172.21.15.178:3300/0 conn(0x7f56b415e2d0 0x7f56b415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 -- 172.21.15.113:0/2581519154 >> 172.21.15.113:0/2581519154 conn(0x7f56b406b0f0 msgr2=0x7f56b406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 -- 172.21.15.113:0/2581519154 shutdown_connections 2024-04-24T23:58:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.447+0000 7f56b9190640 1 -- 172.21.15.113:0/2581519154 wait complete. 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.448+0000 7f56b9190640 1 Processor -- start 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.448+0000 7f56b9190640 1 -- start start 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b9190640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b40b3080 0x7f56b4145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b9190640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56b415e2d0 0x7f56b41477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b9190640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b415e6b0 0x7f56b4152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b9190640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56b415f2b0 con 0x7f56b415e6b0 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b9190640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56b415f130 con 0x7f56b415e2d0 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b9190640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56b4071040 con 0x7f56b40b3080 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b415e6b0 0x7f56b4152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b415e6b0 0x7f56b4152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59682/0 (socket says 172.21.15.113:59682) 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 -- 172.21.15.113:0/1309907887 learned_addr learned my addr 172.21.15.113:0/1309907887 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b259c640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56b415e2d0 0x7f56b41477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b2d9d640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b40b3080 0x7f56b4145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 -- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b40b3080 msgr2=0x7f56b4145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b40b3080 0x7f56b4145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 -- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56b415e2d0 msgr2=0x7f56b41477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56b415e2d0 0x7f56b41477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 -- 172.21.15.113:0/1309907887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b4143af0 con 0x7f56b415e6b0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b259c640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56b415e2d0 0x7f56b41477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b2d9d640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b40b3080 0x7f56b4145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.449+0000 7f56b359e640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b415e6b0 0x7f56b4152680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56a400cc30 tx=0x7f56a400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.450+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1309907887 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a400ce40 con 0x7f56b415e6b0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.450+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1309907887 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a4003990 con 0x7f56b415e6b0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.450+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b4143dd0 con 0x7f56b415e6b0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.450+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1309907887 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003d40 con 0x7f56b415e6b0 2024-04-24T23:58:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.450+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b4147d30 con 0x7f56b415e6b0 2024-04-24T23:58:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.451+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b40691a0 con 0x7f56b415e6b0 2024-04-24T23:58:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.451+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1309907887 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56a4003ee0 con 0x7f56b415e6b0 2024-04-24T23:58:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.451+0000 7f56ab7fe640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f569406be60 0x7f569406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.452+0000 7f56b2d9d640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f569406be60 0x7f569406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.452+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1309907887 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56a408b230 con 0x7f56b415e6b0 2024-04-24T23:58:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.452+0000 7f56b2d9d640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f569406be60 0x7f569406e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56ac008890 tx=0x7f56ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.456+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1309907887 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56a4014030 con 0x7f56b415e6b0 2024-04-24T23:58:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.627+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56b40ce9a0 con 0x7f56b415e6b0 2024-04-24T23:58:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.627+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1309907887 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f56a4059c70 con 0x7f56b415e6b0 2024-04-24T23:58:57.627 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:58:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f569406be60 msgr2=0x7f569406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f569406be60 0x7f569406e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56ac008890 tx=0x7f56ac008040 comp rx=0 tx=0).stop 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b415e6b0 msgr2=0x7f56b4152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b415e6b0 0x7f56b4152680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56a400cc30 tx=0x7f56a400a5c0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 shutdown_connections 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f569406be60 0x7f569406e320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56b415e2d0 0x7f56b41477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b415e6b0 0x7f56b4152680 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 --2- 172.21.15.113:0/1309907887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56b40b3080 0x7f56b4145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 >> 172.21.15.113:0/1309907887 conn(0x7f56b406b0f0 msgr2=0x7f56b40b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 shutdown_connections 2024-04-24T23:58:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.629+0000 7f56b9190640 1 -- 172.21.15.113:0/1309907887 wait complete. 2024-04-24T23:58:57.640 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:58:57.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.739+0000 7f33734b3640 1 Processor -- start 2024-04-24T23:58:57.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.739+0000 7f33734b3640 1 -- start start 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f33734b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c13f050 0x7f336c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f33734b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f336c0d04f0 0x7f336c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f33734b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f336c1604d0 0x7f336c06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f33734b3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f336c15c6a0 con 0x7f336c0d04f0 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f33734b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f336c0d0970 con 0x7f336c1604d0 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f33734b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f336c15d6b0 con 0x7f336c13f050 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f336c0d04f0 0x7f336c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f336c0d04f0 0x7f336c15ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42494/0 (socket says 172.21.15.113:42494) 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 -- 172.21.15.113:0/1066535256 learned_addr learned my addr 172.21.15.113:0/1066535256 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3371a50640 1 --2- 172.21.15.113:0/1066535256 >> v2:172.21.15.178:3300/0 conn(0x7f336c1604d0 0x7f336c06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f337124f640 1 --2- 172.21.15.113:0/1066535256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c13f050 0x7f336c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 -- 172.21.15.113:0/1066535256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c13f050 msgr2=0x7f336c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 --2- 172.21.15.113:0/1066535256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c13f050 0x7f336c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 -- 172.21.15.113:0/1066535256 >> v2:172.21.15.178:3300/0 conn(0x7f336c1604d0 msgr2=0x7f336c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 --2- 172.21.15.113:0/1066535256 >> v2:172.21.15.178:3300/0 conn(0x7f336c1604d0 0x7f336c06cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3370a4e640 1 -- 172.21.15.113:0/1066535256 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c15c8f0 con 0x7f336c0d04f0 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f3371a50640 1 --2- 172.21.15.113:0/1066535256 >> v2:172.21.15.178:3300/0 conn(0x7f336c1604d0 0x7f336c06cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.740+0000 7f337124f640 1 --2- 172.21.15.113:0/1066535256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c13f050 0x7f336c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.741+0000 7f3370a4e640 1 --2- 172.21.15.113:0/1066535256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c0d04f0 0x7f336c15ff90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f336c06f460 tx=0x7f335c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e28e11a031a8fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.741+0000 7f3363fff640 1 -- 172.21.15.113:0/1066535256 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c037070 con 0x7f336c0d04f0 2024-04-24T23:58:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.741+0000 7f3363fff640 1 -- 172.21.15.113:0/1066535256 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c032440 con 0x7f336c0d04f0 2024-04-24T23:58:57.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.741+0000 7f3363fff640 1 -- 172.21.15.113:0/1066535256 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c032810 con 0x7f336c0d04f0 2024-04-24T23:58:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 -- 172.21.15.113:0/1066535256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c0d04f0 msgr2=0x7f336c15ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 --2- 172.21.15.113:0/1066535256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c0d04f0 0x7f336c15ff90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f336c06f460 tx=0x7f335c02a140 comp rx=0 tx=0).stop 2024-04-24T23:58:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 -- 172.21.15.113:0/1066535256 shutdown_connections 2024-04-24T23:58:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 --2- 172.21.15.113:0/1066535256 >> v2:172.21.15.178:3300/0 conn(0x7f336c1604d0 0x7f336c06cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 --2- 172.21.15.113:0/1066535256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c0d04f0 0x7f336c15ff90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 --2- 172.21.15.113:0/1066535256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c13f050 0x7f336c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 -- 172.21.15.113:0/1066535256 >> 172.21.15.113:0/1066535256 conn(0x7f336c06b0f0 msgr2=0x7f336c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 -- 172.21.15.113:0/1066535256 shutdown_connections 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.742+0000 7f33734b3640 1 -- 172.21.15.113:0/1066535256 wait complete. 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.743+0000 7f33734b3640 1 Processor -- start 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.743+0000 7f33734b3640 1 -- start start 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f33734b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c0d04f0 0x7f336c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f33734b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c13f050 0x7f336c1479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f33734b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f336c1604d0 0x7f336c1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f33734b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f336c15d190 con 0x7f336c13f050 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f33734b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f336c15ceb0 con 0x7f336c1604d0 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f33734b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f336c13f430 con 0x7f336c0d04f0 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c13f050 0x7f336c1479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c13f050 0x7f336c1479a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42508/0 (socket says 172.21.15.113:42508) 2024-04-24T23:58:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 -- 172.21.15.113:0/3072155118 learned_addr learned my addr 172.21.15.113:0/3072155118 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3371a50640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f336c1604d0 0x7f336c1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f337124f640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c0d04f0 0x7f336c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 -- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c0d04f0 msgr2=0x7f336c1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c0d04f0 0x7f336c1537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 -- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f336c1604d0 msgr2=0x7f336c1f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f336c1604d0 0x7f336c1f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 -- 172.21.15.113:0/3072155118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c153ee0 con 0x7f336c13f050 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f337124f640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c0d04f0 0x7f336c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3370a4e640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c13f050 0x7f336c1479a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f335c02a180 tx=0x7f335c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3361ffb640 1 -- 172.21.15.113:0/3072155118 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c037040 con 0x7f336c13f050 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3361ffb640 1 -- 172.21.15.113:0/3072155118 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c042070 con 0x7f336c13f050 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c155ff0 con 0x7f336c13f050 2024-04-24T23:58:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.744+0000 7f3361ffb640 1 -- 172.21.15.113:0/3072155118 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c030bb0 con 0x7f336c13f050 2024-04-24T23:58:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.745+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c1565b0 con 0x7f336c13f050 2024-04-24T23:58:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.746+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c0691a0 con 0x7f336c13f050 2024-04-24T23:58:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.746+0000 7f3361ffb640 1 -- 172.21.15.113:0/3072155118 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f335c047460 con 0x7f336c13f050 2024-04-24T23:58:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.746+0000 7f3361ffb640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f335006be30 0x7f335006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.746+0000 7f3361ffb640 1 -- 172.21.15.113:0/3072155118 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f335c0aba50 con 0x7f336c13f050 2024-04-24T23:58:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.746+0000 7f337124f640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f335006be30 0x7f335006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.747+0000 7f337124f640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f335006be30 0x7f335006e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f336800aa30 tx=0x7f3368008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:57.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.751+0000 7f3361ffb640 1 -- 172.21.15.113:0/3072155118 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f336c153ee0 con 0x7f336c13f050 2024-04-24T23:58:57.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.924+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f336c0d0e50 con 0x7f336c13f050 2024-04-24T23:58:57.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.924+0000 7f3361ffb640 1 -- 172.21.15.113:0/3072155118 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f335c07a400 con 0x7f336c13f050 2024-04-24T23:58:57.925 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f335006be30 msgr2=0x7f335006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f335006be30 0x7f335006e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f336800aa30 tx=0x7f3368008040 comp rx=0 tx=0).stop 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c13f050 msgr2=0x7f336c1479a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c13f050 0x7f336c1479a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f335c02a180 tx=0x7f335c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 shutdown_connections 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f335006be30 0x7f335006e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f336c1604d0 0x7f336c1f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f336c13f050 0x7f336c1479a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 --2- 172.21.15.113:0/3072155118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f336c0d04f0 0x7f336c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:57.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 >> 172.21.15.113:0/3072155118 conn(0x7f336c06b0f0 msgr2=0x7f336c13c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:57.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 shutdown_connections 2024-04-24T23:58:57.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:57.927+0000 7f33734b3640 1 -- 172.21.15.113:0/3072155118 wait complete. 2024-04-24T23:58:57.938 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:58:58.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.036+0000 7f7220577640 1 Processor -- start 2024-04-24T23:58:58.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.036+0000 7f7220577640 1 -- start start 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.037+0000 7f7220577640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f721815e2d0 0x7f721815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.037+0000 7f7220577640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72180b4a40 0x7f72180ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.037+0000 7f7220577640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721815f4f0 con 0x7f72180b3080 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.037+0000 7f7220577640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f72180cde00 con 0x7f72180b4a40 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.037+0000 7f7220577640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f721815f7d0 con 0x7f721815e2d0 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721eb14640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72180b4a40 0x7f72180ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721db12640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f72180b3080 0x7f72180b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42960/0 (socket says 172.21.15.113:42960) 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721db12640 1 -- 172.21.15.113:0/3205254198 learned_addr learned my addr 172.21.15.113:0/3205254198 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721eb14640 1 -- 172.21.15.113:0/3205254198 >> v2:172.21.15.178:3300/0 conn(0x7f721815e2d0 msgr2=0x7f721815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721e313640 1 --2- 172.21.15.113:0/3205254198 >> v2:172.21.15.178:3300/0 conn(0x7f721815e2d0 0x7f721815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721eb14640 1 --2- 172.21.15.113:0/3205254198 >> v2:172.21.15.178:3300/0 conn(0x7f721815e2d0 0x7f721815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721eb14640 1 -- 172.21.15.113:0/3205254198 >> v1:172.21.15.113:6789/0 conn(0x7f72180b3080 legacy=0x7f72180b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721d311640 1 -- 172.21.15.113:0/3205254198 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203740036 0 0) 0x7f721815f4f0 con 0x7f72180b3080 2024-04-24T23:58:58.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.038+0000 7f721eb14640 1 -- 172.21.15.113:0/3205254198 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721815fab0 con 0x7f72180b4a40 2024-04-24T23:58:58.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f721eb14640 1 --2- 172.21.15.113:0/3205254198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72180b4a40 0x7f72180ce290 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72180b2d40 tx=0x7f721002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff24c8d9937e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f721d311640 1 -- 172.21.15.113:0/3205254198 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210007740 con 0x7f72180b4a40 2024-04-24T23:58:58.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f721d311640 1 -- 172.21.15.113:0/3205254198 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72100078e0 con 0x7f72180b4a40 2024-04-24T23:58:58.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f721d311640 1 -- 172.21.15.113:0/3205254198 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210007cb0 con 0x7f72180b4a40 2024-04-24T23:58:58.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 -- 172.21.15.113:0/3205254198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72180b4a40 msgr2=0x7f72180ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 --2- 172.21.15.113:0/3205254198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72180b4a40 0x7f72180ce290 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72180b2d40 tx=0x7f721002a680 comp rx=0 tx=0).stop 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 -- 172.21.15.113:0/3205254198 shutdown_connections 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 --2- 172.21.15.113:0/3205254198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72180b4a40 0x7f72180ce290 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 --2- 172.21.15.113:0/3205254198 >> v2:172.21.15.178:3300/0 conn(0x7f721815e2d0 0x7f721815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 -- 172.21.15.113:0/3205254198 >> 172.21.15.113:0/3205254198 conn(0x7f721806b0f0 msgr2=0x7f721806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 -- 172.21.15.113:0/3205254198 shutdown_connections 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.039+0000 7f7220577640 1 -- 172.21.15.113:0/3205254198 wait complete. 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.040+0000 7f7220577640 1 Processor -- start 2024-04-24T23:58:58.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.040+0000 7f7220577640 1 -- start start 2024-04-24T23:58:58.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f7220577640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72180b3080 0x7f7218152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f7220577640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180b4a40 0x7f72180ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f7220577640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721815e2d0 0x7f7218149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f7220577640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f721815f2b0 con 0x7f721815e2d0 2024-04-24T23:58:58.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f7220577640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f721815f130 con 0x7f72180b3080 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f7220577640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f721815e6b0 con 0x7f72180b4a40 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f721eb14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721815e2d0 0x7f7218149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f721eb14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721815e2d0 0x7f7218149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42528/0 (socket says 172.21.15.113:42528) 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.041+0000 7f721eb14640 1 -- 172.21.15.113:0/3570606020 learned_addr learned my addr 172.21.15.113:0/3570606020 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721eb14640 1 -- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180b4a40 msgr2=0x7f72180ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721eb14640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180b4a40 0x7f72180ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721e313640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72180b3080 0x7f7218152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721eb14640 1 -- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72180b3080 msgr2=0x7f7218152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721eb14640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72180b3080 0x7f7218152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721eb14640 1 -- 172.21.15.113:0/3570606020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721814a320 con 0x7f721815e2d0 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721e313640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72180b3080 0x7f7218152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f721eb14640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721815e2d0 0x7f7218149de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f721002ac70 tx=0x7f7210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f720affd640 1 -- 172.21.15.113:0/3570606020 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210040070 con 0x7f721815e2d0 2024-04-24T23:58:58.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f720affd640 1 -- 172.21.15.113:0/3570606020 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7210031c90 con 0x7f721815e2d0 2024-04-24T23:58:58.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72180ae000 con 0x7f721815e2d0 2024-04-24T23:58:58.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f720affd640 1 -- 172.21.15.113:0/3570606020 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210045680 con 0x7f721815e2d0 2024-04-24T23:58:58.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.042+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72180ae5c0 con 0x7f721815e2d0 2024-04-24T23:58:58.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.043+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72180b5270 con 0x7f721815e2d0 2024-04-24T23:58:58.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.043+0000 7f720affd640 1 -- 172.21.15.113:0/3570606020 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7210039040 con 0x7f721815e2d0 2024-04-24T23:58:58.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.044+0000 7f720affd640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f71fc06be30 0x7f71fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.044+0000 7f721e313640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f71fc06be30 0x7f71fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.044+0000 7f720affd640 1 -- 172.21.15.113:0/3570606020 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f72100ac820 con 0x7f721815e2d0 2024-04-24T23:58:58.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.044+0000 7f721e313640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f71fc06be30 0x7f71fc06e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f720c00b820 tx=0x7f720c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.048+0000 7f720affd640 1 -- 172.21.15.113:0/3570606020 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72180b5270 con 0x7f721815e2d0 2024-04-24T23:58:58.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.221+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f72180691a0 con 0x7f721815e2d0 2024-04-24T23:58:58.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.221+0000 7f720affd640 1 -- 172.21.15.113:0/3570606020 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7210043360 con 0x7f721815e2d0 2024-04-24T23:58:58.221 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f71fc06be30 msgr2=0x7f71fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f71fc06be30 0x7f71fc06e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f720c00b820 tx=0x7f720c008040 comp rx=0 tx=0).stop 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721815e2d0 msgr2=0x7f7218149de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721815e2d0 0x7f7218149de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f721002ac70 tx=0x7f7210004a40 comp rx=0 tx=0).stop 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 shutdown_connections 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f71fc06be30 0x7f71fc06e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721815e2d0 0x7f7218149de0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180b4a40 0x7f72180ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 --2- 172.21.15.113:0/3570606020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72180b3080 0x7f7218152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.223+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 >> 172.21.15.113:0/3570606020 conn(0x7f721806b0f0 msgr2=0x7f72180d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.224+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 shutdown_connections 2024-04-24T23:58:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.224+0000 7f7220577640 1 -- 172.21.15.113:0/3570606020 wait complete. 2024-04-24T23:58:58.235 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.331+0000 7fe6e92b3640 1 Processor -- start 2024-04-24T23:58:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.331+0000 7fe6e92b3640 1 -- start start 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e92b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 0x7fe6e40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e92b3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e40d2800 con 0x7fe6e40d2290 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e92b3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e40d2ae0 con 0x7fe6e415e320 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e92b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e40d14d0 con 0x7fe6e40d0b00 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e27fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe6e40d2290 0x7fe6e40b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42964/0 (socket says 172.21.15.113:42964) 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e27fc640 1 -- 172.21.15.113:0/789095295 learned_addr learned my addr 172.21.15.113:0/789095295 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007843163 0 0) 0x7fe6e40d2800 con 0x7fe6e40d2290 2024-04-24T23:58:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.332+0000 7fe6e37fe640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 0x7fe6e40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6cc003520 con 0x7fe6e40d2290 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4125050607 0 0) 0x7fe6cc003520 con 0x7fe6e40d2290 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 msgr2=0x7fe6e40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 0x7fe6e40d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 >> v1:172.21.15.178:6789/0 conn(0x7fe6e415e320 legacy=0x7fe6e40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e40d2dc0 con 0x7fe6e40d2290 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe6d8001690 con 0x7fe6e40d2290 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 >> v1:172.21.15.113:6789/0 conn(0x7fe6e40d2290 legacy=0x7fe6e40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0039e0 0x7fe6cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 0x7fe6cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc004d40 0x7fe6cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40d14d0 con 0x7fe6cc004d40 2024-04-24T23:58:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc00bb00 con 0x7fe6cc0039e0 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6cc00bd00 con 0x7fe6e40d0b00 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e37fe640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc004d40 0x7fe6cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e2ffd640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0039e0 0x7fe6cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e27fc640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 0x7fe6cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e37fe640 1 -- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 msgr2=0x7fe6cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e37fe640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e40d0b00 0x7fe6cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e37fe640 1 -- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0039e0 msgr2=0x7fe6cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e37fe640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0039e0 0x7fe6cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.333+0000 7fe6e37fe640 1 -- 172.21.15.113:0/789095295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc003520 con 0x7fe6cc004d40 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.334+0000 7fe6e37fe640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc004d40 0x7fe6cc0051c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009a90 tx=0x7fe6cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a54728573361b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.334+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc036070 con 0x7fe6cc004d40 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.334+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc031400 con 0x7fe6cc004d40 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.334+0000 7fe6e1ffb640 1 -- 172.21.15.113:0/789095295 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc009690 con 0x7fe6cc004d40 2024-04-24T23:58:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.334+0000 7fe6e92b3640 1 -- 172.21.15.113:0/789095295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc004d40 msgr2=0x7fe6cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.334+0000 7fe6e92b3640 1 --2- 172.21.15.113:0/789095295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc004d40 0x7fe6cc0051c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009a90 tx=0x7fe6cc005830 comp rx=0 tx=0).stop 2024-04-24T23:58:58.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.334+0000 7fe6e37fe640 1 -- 172.21.15.113:0/789095295 reap_dead start 2024-04-24T23:58:58.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.335+0000 7fe6e92b3640 1 -- 172.21.15.113:0/789095295 shutdown_connections 2024-04-24T23:58:58.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.335+0000 7fe6e92b3640 1 -- 172.21.15.113:0/789095295 >> 172.21.15.113:0/789095295 conn(0x7fe6e40b5c20 msgr2=0x7fe6e40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:58.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.335+0000 7fe6e92b3640 1 -- 172.21.15.113:0/789095295 shutdown_connections 2024-04-24T23:58:58.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.335+0000 7fe6e92b3640 1 -- 172.21.15.113:0/789095295 wait complete. 2024-04-24T23:58:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.336+0000 7fe6e92b3640 1 Processor -- start 2024-04-24T23:58:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.336+0000 7fe6e92b3640 1 -- start start 2024-04-24T23:58:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e41f1590 0x7fe6e40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e40b4be0 0x7fe6e40b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e4155000 0x7fe6e4155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e415e700 con 0x7fe6e41f1590 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40d3410 con 0x7fe6e40b4be0 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e415e880 con 0x7fe6e4155000 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e41f1590 0x7fe6e40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e41f1590 0x7fe6e40cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42552/0 (socket says 172.21.15.113:42552) 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 -- 172.21.15.113:0/1762250633 learned_addr learned my addr 172.21.15.113:0/1762250633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e37fe640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e4155000 0x7fe6e4155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e27fc640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e40b4be0 0x7fe6e40b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 -- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e4155000 msgr2=0x7fe6e4155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e4155000 0x7fe6e4155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 -- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e40b4be0 msgr2=0x7fe6e40b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e40b4be0 0x7fe6e40b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 -- 172.21.15.113:0/1762250633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e41f1970 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e2ffd640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e41f1590 0x7fe6e40cc5e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400fb90 tx=0x7fe6d400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6cb7fe640 1 -- 172.21.15.113:0/1762250633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400fe90 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6cb7fe640 1 -- 172.21.15.113:0/1762250633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d4018070 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e41559c0 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6cb7fe640 1 -- 172.21.15.113:0/1762250633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4003970 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.337+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e4154520 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.338+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e40691a0 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.339+0000 7fe6cb7fe640 1 -- 172.21.15.113:0/1762250633 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6d4003b10 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.339+0000 7fe6cb7fe640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6d006c000 0x7fe6d006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.339+0000 7fe6e27fc640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6d006c000 0x7fe6d006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.339+0000 7fe6cb7fe640 1 -- 172.21.15.113:0/1762250633 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d408afc0 con 0x7fe6e41f1590 2024-04-24T23:58:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.339+0000 7fe6e27fc640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6d006c000 0x7fe6d006e4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40d0570 tx=0x7fe6d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.343+0000 7fe6cb7fe640 1 -- 172.21.15.113:0/1762250633 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6d4059a00 con 0x7fe6e41f1590 2024-04-24T23:58:58.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.553+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6e40b9310 con 0x7fe6e41f1590 2024-04-24T23:58:58.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.553+0000 7fe6cb7fe640 1 -- 172.21.15.113:0/1762250633 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7fe6d4059300 con 0x7fe6e41f1590 2024-04-24T23:58:58.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:58.554 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:58.554 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:58.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6d006c000 msgr2=0x7fe6d006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6d006c000 0x7fe6d006e4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40d0570 tx=0x7fe6d8009040 comp rx=0 tx=0).stop 2024-04-24T23:58:58.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e41f1590 msgr2=0x7fe6e40cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e41f1590 0x7fe6e40cc5e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400fb90 tx=0x7fe6d400d520 comp rx=0 tx=0).stop 2024-04-24T23:58:58.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 shutdown_connections 2024-04-24T23:58:58.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6d006c000 0x7fe6d006e4c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e40b4be0 0x7fe6e40b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e4155000 0x7fe6e4155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 --2- 172.21.15.113:0/1762250633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e41f1590 0x7fe6e40cc5e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 >> 172.21.15.113:0/1762250633 conn(0x7fe6e40b5c20 msgr2=0x7fe6e40b86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:58.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 shutdown_connections 2024-04-24T23:58:58.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.555+0000 7fe6e92b3640 1 -- 172.21.15.113:0/1762250633 wait complete. 2024-04-24T23:58:58.567 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:58.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.664+0000 7ffad1876640 1 Processor -- start 2024-04-24T23:58:58.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.664+0000 7ffad1876640 1 -- start start 2024-04-24T23:58:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffad1876640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc0b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffad1876640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffacc0b6810 0x7ffacc0b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffad1876640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffacc15e6a0 con 0x7ffacc0d24a0 2024-04-24T23:58:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffad1876640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ffacc0d2c80 con 0x7ffacc0b6810 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffad1876640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc0d2f60 con 0x7ffacc15e2c0 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc0b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacb7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffacc0b6810 0x7ffacc0b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc0b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:47620/0 (socket says 172.21.15.113:47620) 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacaffd640 1 -- 172.21.15.113:0/4263193321 learned_addr learned my addr 172.21.15.113:0/4263193321 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacaffd640 1 -- 172.21.15.113:0/4263193321 >> v2:172.21.15.178:3300/0 conn(0x7ffacc0b6810 msgr2=0x7ffacc0b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacaffd640 1 --2- 172.21.15.113:0/4263193321 >> v2:172.21.15.178:3300/0 conn(0x7ffacc0b6810 0x7ffacc0b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacaffd640 1 -- 172.21.15.113:0/4263193321 >> v1:172.21.15.178:6789/0 conn(0x7ffacc0d24a0 legacy=0x7ffacc0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffac9ffb640 1 -- 172.21.15.113:0/4263193321 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719858291 0 0) 0x7ffacc15e6a0 con 0x7ffacc0d24a0 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacaffd640 1 -- 172.21.15.113:0/4263193321 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc0d3240 con 0x7ffacc15e2c0 2024-04-24T23:58:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.665+0000 7ffacb7fe640 1 --2- 172.21.15.113:0/4263193321 >> v2:172.21.15.178:3300/0 conn(0x7ffacc0b6810 0x7ffacc0b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.666+0000 7ffacaffd640 1 --2- 172.21.15.113:0/4263193321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc0b9c90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009870 tx=0x7ffabc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3abaf9d6a7e5fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.666+0000 7ffac9ffb640 1 -- 172.21.15.113:0/4263193321 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc037070 con 0x7ffacc15e2c0 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.666+0000 7ffac9ffb640 1 -- 172.21.15.113:0/4263193321 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffabc032440 con 0x7ffacc15e2c0 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.666+0000 7ffac9ffb640 1 -- 172.21.15.113:0/4263193321 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc032810 con 0x7ffacc15e2c0 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 -- 172.21.15.113:0/4263193321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 msgr2=0x7ffacc0b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 --2- 172.21.15.113:0/4263193321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc0b9c90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009870 tx=0x7ffabc02a1c0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 -- 172.21.15.113:0/4263193321 shutdown_connections 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 --2- 172.21.15.113:0/4263193321 >> v2:172.21.15.178:3300/0 conn(0x7ffacc0b6810 0x7ffacc0b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 --2- 172.21.15.113:0/4263193321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc0b9c90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 -- 172.21.15.113:0/4263193321 >> 172.21.15.113:0/4263193321 conn(0x7ffacc06b510 msgr2=0x7ffacc06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 -- 172.21.15.113:0/4263193321 shutdown_connections 2024-04-24T23:58:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.667+0000 7ffad1876640 1 -- 172.21.15.113:0/4263193321 wait complete. 2024-04-24T23:58:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.668+0000 7ffad1876640 1 Processor -- start 2024-04-24T23:58:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.668+0000 7ffad1876640 1 -- start start 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffad1876640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc0b6810 0x7ffacc1e4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffad1876640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffacc0d24a0 0x7ffacc152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffad1876640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffad1876640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc0d2900 con 0x7ffacc0d24a0 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffad1876640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc0d3d20 con 0x7ffacc0b6810 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffad1876640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc0b7740 con 0x7ffacc15e2c0 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffacc0d24a0 0x7ffacc152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffacc0d24a0 0x7ffacc152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42556/0 (socket says 172.21.15.113:42556) 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 -- 172.21.15.113:0/1298543557 learned_addr learned my addr 172.21.15.113:0/1298543557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffacb7fe640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffacaffd640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc0b6810 0x7ffacc1e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 -- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 msgr2=0x7ffacc145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 -- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc0b6810 msgr2=0x7ffacc1e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc0b6810 0x7ffacc1e4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.669+0000 7ffaca7fc640 1 -- 172.21.15.113:0/1298543557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc151d00 con 0x7ffacc0d24a0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.670+0000 7ffaca7fc640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffacc0d24a0 0x7ffacc152da0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c550 tx=0x7ffac000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.670+0000 7ffab3fff640 1 -- 172.21.15.113:0/1298543557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0017070 con 0x7ffacc0d24a0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.670+0000 7ffab3fff640 1 -- 172.21.15.113:0/1298543557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac0005310 con 0x7ffacc0d24a0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.670+0000 7ffab3fff640 1 -- 172.21.15.113:0/1298543557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0003850 con 0x7ffacc0d24a0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.670+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc151fe0 con 0x7ffacc0d24a0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.670+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc1525a0 con 0x7ffacc0d24a0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.671+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffacc0691a0 con 0x7ffacc0d24a0 2024-04-24T23:58:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.671+0000 7ffab3fff640 1 -- 172.21.15.113:0/1298543557 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffac001a070 con 0x7ffacc0d24a0 2024-04-24T23:58:58.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.671+0000 7ffab3fff640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffab806be60 0x7ffab806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.672+0000 7ffab3fff640 1 -- 172.21.15.113:0/1298543557 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffac008a010 con 0x7ffacc0d24a0 2024-04-24T23:58:58.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.672+0000 7ffacaffd640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffab806be60 0x7ffab806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.672+0000 7ffacaffd640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffab806be60 0x7ffab806e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffabc034000 tx=0x7ffabc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.676+0000 7ffab3fff640 1 -- 172.21.15.113:0/1298543557 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffac0058400 con 0x7ffacc0d24a0 2024-04-24T23:58:58.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.853+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffacc0ca730 con 0x7ffacc0d24a0 2024-04-24T23:58:58.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.854+0000 7ffab3fff640 1 -- 172.21.15.113:0/1298543557 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7ffac005c700 con 0x7ffacc0d24a0 2024-04-24T23:58:58.854 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:58.854 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:20.248500+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:58.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffab806be60 msgr2=0x7ffab806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffab806be60 0x7ffab806e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffabc034000 tx=0x7ffabc02e000 comp rx=0 tx=0).stop 2024-04-24T23:58:58.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffacc0d24a0 msgr2=0x7ffacc152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffacc0d24a0 0x7ffacc152da0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c550 tx=0x7ffac000c950 comp rx=0 tx=0).stop 2024-04-24T23:58:58.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 shutdown_connections 2024-04-24T23:58:58.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffab806be60 0x7ffab806e320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffacc15e2c0 0x7ffacc145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffacc0d24a0 0x7ffacc152da0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 --2- 172.21.15.113:0/1298543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc0b6810 0x7ffacc1e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 >> 172.21.15.113:0/1298543557 conn(0x7ffacc06b510 msgr2=0x7ffacc0c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:58.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 shutdown_connections 2024-04-24T23:58:58.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.856+0000 7ffad1876640 1 -- 172.21.15.113:0/1298543557 wait complete. 2024-04-24T23:58:58.867 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:58:58.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.963+0000 7f4c700bd640 1 Processor -- start 2024-04-24T23:58:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.963+0000 7f4c700bd640 1 -- start start 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c700bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 0x7f4c6806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c700bd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4c6806e510 0x7f4c680cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c700bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c68144cf0 con 0x7f4c68144910 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c700bd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4c680b2f30 con 0x7f4c6806e510 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c700bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c680b3210 con 0x7f4c680b3fd0 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c6de59640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4c68144910 0x7f4c680b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42974/0 (socket says 172.21.15.113:42974) 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c6de59640 1 -- 172.21.15.113:0/639336590 learned_addr learned my addr 172.21.15.113:0/639336590 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 680073770 0 0) 0x7f4c68144cf0 con 0x7f4c68144910 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c54003520 con 0x7f4c68144910 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c6e65a640 1 --2- 172.21.15.113:0/639336590 >> v2:172.21.15.178:3300/0 conn(0x7f4c6806e510 0x7f4c680cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.964+0000 7f4c6d658640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 0x7f4c6806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266894992 0 0) 0x7f4c54003520 con 0x7f4c68144910 2024-04-24T23:58:58.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 msgr2=0x7f4c6806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 0x7f4c6806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 >> v2:172.21.15.178:3300/0 conn(0x7f4c6806e510 msgr2=0x7f4c680cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 --2- 172.21.15.113:0/639336590 >> v2:172.21.15.178:3300/0 conn(0x7f4c6806e510 0x7f4c680cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c680b34f0 con 0x7f4c68144910 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f4c640019f0 con 0x7f4c68144910 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 >> v1:172.21.15.113:6789/0 conn(0x7f4c68144910 legacy=0x7f4c680b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 0x7f4c540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54003fe0 0x7f4c54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4c680b2f30 con 0x7f4c54003fe0 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c680b3210 con 0x7f4c54004d40 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5400bac0 con 0x7f4c680b3fd0 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6d658640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54003fe0 0x7f4c54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6de59640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 0x7f4c540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6e65a640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6d658640 1 -- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 msgr2=0x7f4c540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6d658640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c680b3fd0 0x7f4c540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6d658640 1 -- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54004d40 msgr2=0x7f4c540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6d658640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6d658640 1 -- 172.21.15.113:0/639336590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c54003520 con 0x7f4c54003fe0 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.965+0000 7f4c6e65a640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c6d658640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54003fe0 0x7f4c54004800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ba30 tx=0x7f4c54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de41ddd3f3908c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58030070 con 0x7f4c54003fe0 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5802ade0 con 0x7f4c54003fe0 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c6ce57640 1 -- 172.21.15.113:0/639336590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5800b600 con 0x7f4c54003fe0 2024-04-24T23:58:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c700bd640 1 -- 172.21.15.113:0/639336590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54003fe0 msgr2=0x7f4c54004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c700bd640 1 --2- 172.21.15.113:0/639336590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54003fe0 0x7f4c54004800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ba30 tx=0x7f4c54005830 comp rx=0 tx=0).stop 2024-04-24T23:58:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c6e65a640 1 -- 172.21.15.113:0/639336590 reap_dead start 2024-04-24T23:58:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c700bd640 1 -- 172.21.15.113:0/639336590 shutdown_connections 2024-04-24T23:58:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c700bd640 1 -- 172.21.15.113:0/639336590 >> 172.21.15.113:0/639336590 conn(0x7f4c680b1a50 msgr2=0x7f4c680b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c700bd640 1 -- 172.21.15.113:0/639336590 shutdown_connections 2024-04-24T23:58:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.966+0000 7f4c700bd640 1 -- 172.21.15.113:0/639336590 wait complete. 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 Processor -- start 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 -- start start 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54003fe0 0x7f4c680ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c68134a20 0x7f4c6815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c6813a1b0 0x7f4c6813a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4c680b4da0 con 0x7f4c6813a1b0 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c680b4410 con 0x7f4c54003fe0 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c700bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c680b4f20 con 0x7f4c68134a20 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c6e65a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c6813a1b0 0x7f4c6813a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c6e65a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c6813a1b0 0x7f4c6813a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42584/0 (socket says 172.21.15.113:42584) 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.968+0000 7f4c6e65a640 1 -- 172.21.15.113:0/1169072763 learned_addr learned my addr 172.21.15.113:0/1169072763 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6de59640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54003fe0 0x7f4c680ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6d658640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c68134a20 0x7f4c6815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6e65a640 1 -- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c68134a20 msgr2=0x7f4c6815d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6e65a640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c68134a20 0x7f4c6815d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6e65a640 1 -- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54003fe0 msgr2=0x7f4c680ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6e65a640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54003fe0 0x7f4c680ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6e65a640 1 -- 172.21.15.113:0/1169072763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c68134ea0 con 0x7f4c6813a1b0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c6e65a640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c6813a1b0 0x7f4c6813a660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006fd0 tx=0x7f4c6000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c5e7fc640 1 -- 172.21.15.113:0/1169072763 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60018020 con 0x7f4c6813a1b0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c5e7fc640 1 -- 172.21.15.113:0/1169072763 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c60014660 con 0x7f4c6813a1b0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c680aedc0 con 0x7f4c6813a1b0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c5e7fc640 1 -- 172.21.15.113:0/1169072763 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60014a50 con 0x7f4c6813a1b0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.969+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c680cd710 con 0x7f4c6813a1b0 2024-04-24T23:58:58.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.970+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c680691a0 con 0x7f4c6813a1b0 2024-04-24T23:58:58.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.970+0000 7f4c5e7fc640 1 -- 172.21.15.113:0/1169072763 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c6001b020 con 0x7f4c6813a1b0 2024-04-24T23:58:58.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.971+0000 7f4c5e7fc640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4c4c06be30 0x7f4c4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:58.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.971+0000 7f4c5e7fc640 1 -- 172.21.15.113:0/1169072763 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4c6008afe0 con 0x7f4c6813a1b0 2024-04-24T23:58:58.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.971+0000 7f4c6de59640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4c4c06be30 0x7f4c4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:58.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.971+0000 7f4c6de59640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4c4c06be30 0x7f4c4c06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c680727b0 tx=0x7f4c64006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:58.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:58.975+0000 7f4c5e7fc640 1 -- 172.21.15.113:0/1169072763 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c60022020 con 0x7f4c6813a1b0 2024-04-24T23:58:59.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.184+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c680731a0 con 0x7f4c6813a1b0 2024-04-24T23:58:59.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.184+0000 7f4c5e7fc640 1 -- 172.21.15.113:0/1169072763 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f4c60059990 con 0x7f4c6813a1b0 2024-04-24T23:58:59.185 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:59.185 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:59.185 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4c4c06be30 msgr2=0x7f4c4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4c4c06be30 0x7f4c4c06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c680727b0 tx=0x7f4c64006040 comp rx=0 tx=0).stop 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c6813a1b0 msgr2=0x7f4c6813a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c6813a1b0 0x7f4c6813a660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006fd0 tx=0x7f4c6000f670 comp rx=0 tx=0).stop 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 shutdown_connections 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4c4c06be30 0x7f4c4c06e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c6813a1b0 0x7f4c6813a660 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c68134a20 0x7f4c6815d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 --2- 172.21.15.113:0/1169072763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c54003fe0 0x7f4c680ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.186+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 >> 172.21.15.113:0/1169072763 conn(0x7f4c680b1a50 msgr2=0x7f4c68071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:59.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.187+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 shutdown_connections 2024-04-24T23:58:59.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.187+0000 7f4c700bd640 1 -- 172.21.15.113:0/1169072763 wait complete. 2024-04-24T23:58:59.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:58:59.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.296+0000 7f2fa4845640 1 Processor -- start 2024-04-24T23:58:59.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.296+0000 7f2fa4845640 1 -- start start 2024-04-24T23:58:59.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2fa4845640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2fa015e320 0x7f2fa00d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2fa4845640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2fa00d2290 0x7f2fa00b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2fa4845640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00d0b00 0x7f2fa00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2fa4845640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa00d14d0 con 0x7f2fa015e320 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2fa4845640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa00d3410 con 0x7f2fa00d2290 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2fa4845640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa00d3de0 con 0x7f2fa00d0b00 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2f9e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2fa015e320 0x7f2fa00d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2f9e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2fa015e320 0x7f2fa00d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42600/0 (socket says 172.21.15.113:42600) 2024-04-24T23:58:59.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.297+0000 7f2f9e59c640 1 -- 172.21.15.113:0/565760663 learned_addr learned my addr 172.21.15.113:0/565760663 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:59.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9e59c640 1 -- 172.21.15.113:0/565760663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00d0b00 msgr2=0x7f2fa00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:59.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9e59c640 1 --2- 172.21.15.113:0/565760663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00d0b00 0x7f2fa00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9e59c640 1 -- 172.21.15.113:0/565760663 >> v2:172.21.15.178:3300/0 conn(0x7f2fa00d2290 msgr2=0x7f2fa00b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:59.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9e59c640 1 --2- 172.21.15.113:0/565760663 >> v2:172.21.15.178:3300/0 conn(0x7f2fa00d2290 0x7f2fa00b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9e59c640 1 -- 172.21.15.113:0/565760663 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa00d2800 con 0x7f2fa015e320 2024-04-24T23:58:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9e59c640 1 --2- 172.21.15.113:0/565760663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa015e320 0x7f2fa00d38a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0072810 tx=0x7f2f9002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5335fb46ad672db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9d59a640 1 -- 172.21.15.113:0/565760663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90036070 con 0x7f2fa015e320 2024-04-24T23:58:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9d59a640 1 -- 172.21.15.113:0/565760663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f90031600 con 0x7f2fa015e320 2024-04-24T23:58:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.298+0000 7f2f9d59a640 1 -- 172.21.15.113:0/565760663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900319d0 con 0x7f2fa015e320 2024-04-24T23:58:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 -- 172.21.15.113:0/565760663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa015e320 msgr2=0x7f2fa00d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 --2- 172.21.15.113:0/565760663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa015e320 0x7f2fa00d38a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0072810 tx=0x7f2f9002a730 comp rx=0 tx=0).stop 2024-04-24T23:58:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 -- 172.21.15.113:0/565760663 shutdown_connections 2024-04-24T23:58:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 --2- 172.21.15.113:0/565760663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00d0b00 0x7f2fa00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 --2- 172.21.15.113:0/565760663 >> v2:172.21.15.178:3300/0 conn(0x7f2fa00d2290 0x7f2fa00b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 --2- 172.21.15.113:0/565760663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa015e320 0x7f2fa00d38a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 -- 172.21.15.113:0/565760663 >> 172.21.15.113:0/565760663 conn(0x7f2fa00b5c20 msgr2=0x7f2fa00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:59.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 -- 172.21.15.113:0/565760663 shutdown_connections 2024-04-24T23:58:59.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.299+0000 7f2fa4845640 1 -- 172.21.15.113:0/565760663 wait complete. 2024-04-24T23:58:59.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.300+0000 7f2fa4845640 1 Processor -- start 2024-04-24T23:58:59.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.300+0000 7f2fa4845640 1 -- start start 2024-04-24T23:58:59.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2fa4845640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa00d0b00 0x7f2fa0154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2fa4845640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa00d2290 0x7f2fa01536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2fa4845640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa015e320 0x7f2fa00cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2fa4845640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00d30a0 con 0x7f2fa00d2290 2024-04-24T23:58:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2fa4845640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00d2dc0 con 0x7f2fa00d0b00 2024-04-24T23:58:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2fa4845640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa00b9fe0 con 0x7f2fa015e320 2024-04-24T23:58:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa00d2290 0x7f2fa01536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa00d2290 0x7f2fa01536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42612/0 (socket says 172.21.15.113:42612) 2024-04-24T23:58:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 -- 172.21.15.113:0/3259361958 learned_addr learned my addr 172.21.15.113:0/3259361958 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9e59c640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa00d0b00 0x7f2fa0154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9ed9d640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa015e320 0x7f2fa00cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 -- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa015e320 msgr2=0x7f2fa00cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa015e320 0x7f2fa00cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 -- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa00d0b00 msgr2=0x7f2fa0154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa00d0b00 0x7f2fa0154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.301+0000 7f2f9dd9b640 1 -- 172.21.15.113:0/3259361958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa015ffa0 con 0x7f2fa00d2290 2024-04-24T23:58:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2f9e59c640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa00d0b00 0x7f2fa0154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2f9ed9d640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa015e320 0x7f2fa00cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2f9dd9b640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa00d2290 0x7f2fa01536a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400b4a0 tx=0x7f2f9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2f8effd640 1 -- 172.21.15.113:0/3259361958 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94014070 con 0x7f2fa00d2290 2024-04-24T23:58:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2f8effd640 1 -- 172.21.15.113:0/3259361958 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f94010610 con 0x7f2fa00d2290 2024-04-24T23:58:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa0160280 con 0x7f2fa00d2290 2024-04-24T23:58:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2f8effd640 1 -- 172.21.15.113:0/3259361958 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94010a00 con 0x7f2fa00d2290 2024-04-24T23:58:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.302+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa0155320 con 0x7f2fa00d2290 2024-04-24T23:58:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.303+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa00cab10 con 0x7f2fa00d2290 2024-04-24T23:58:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.303+0000 7f2f8effd640 1 -- 172.21.15.113:0/3259361958 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f94017070 con 0x7f2fa00d2290 2024-04-24T23:58:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.304+0000 7f2f8effd640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2f8006be30 0x7f2f8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.304+0000 7f2f8effd640 1 -- 172.21.15.113:0/3259361958 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f9408a8f0 con 0x7f2fa00d2290 2024-04-24T23:58:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.304+0000 7f2f9e59c640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2f8006be30 0x7f2f8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.304+0000 7f2f9e59c640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2f8006be30 0x7f2f8006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f90033000 tx=0x7f2f90009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.308+0000 7f2f8effd640 1 -- 172.21.15.113:0/3259361958 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2fa00cab10 con 0x7f2fa00d2290 2024-04-24T23:58:59.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.522+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fa00691a0 con 0x7f2fa00d2290 2024-04-24T23:58:59.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.523+0000 7f2f8effd640 1 -- 172.21.15.113:0/3259361958 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f2f940592a0 con 0x7f2fa00d2290 2024-04-24T23:58:59.523 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:59.524 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3170369099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3170369099},{"type":"v1","addr":"172.21.15.178:6833","nonce":3170369099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3172378411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3172378411},{"type":"v1","addr":"172.21.15.178:6835","nonce":3172378411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1021269378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1021269378},{"type":"v1","addr":"172.21.15.113:6836","nonce":1021269378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:58:18.584673+0000","modified":"2024-04-24T23:58:21.129558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2734482903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2734482903},{"type":"v1","addr":"172.21.15.113:6837","nonce":2734482903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-24T23:58:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-24T23:58:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2f8006be30 msgr2=0x7f2f8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2f8006be30 0x7f2f8006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f90033000 tx=0x7f2f90009000 comp rx=0 tx=0).stop 2024-04-24T23:58:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa00d2290 msgr2=0x7f2fa01536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa00d2290 0x7f2fa01536a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400b4a0 tx=0x7f2f9400b960 comp rx=0 tx=0).stop 2024-04-24T23:58:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 shutdown_connections 2024-04-24T23:58:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2f8006be30 0x7f2f8006e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa015e320 0x7f2fa00cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fa00d2290 0x7f2fa01536a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 --2- 172.21.15.113:0/3259361958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa00d0b00 0x7f2fa0154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 >> 172.21.15.113:0/3259361958 conn(0x7f2fa00b5c20 msgr2=0x7f2fa0071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 shutdown_connections 2024-04-24T23:58:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.525+0000 7f2fa4845640 1 -- 172.21.15.113:0/3259361958 wait complete. 2024-04-24T23:58:59.536 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:58:59.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.632+0000 7fb4e9064640 1 Processor -- start 2024-04-24T23:58:59.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.632+0000 7fb4e9064640 1 -- start start 2024-04-24T23:58:59.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e9064640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb4e415a6b0 0x7fb4e4070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e9064640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e9064640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4e40b2d50 con 0x7fb4e4071780 2024-04-24T23:58:59.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e9064640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e4072a70 con 0x7fb4e415a6b0 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e9064640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e40b3010 con 0x7fb4e4072ea0 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e2d9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb4e415a6b0 0x7fb4e4070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e359e640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb4e4071780 0x7fb4e415e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:53162/0 (socket says 172.21.15.113:53162) 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.633+0000 7fb4e359e640 1 -- 172.21.15.113:0/350204367 learned_addr learned my addr 172.21.15.113:0/350204367 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.634+0000 7fb4dbfff640 1 -- 172.21.15.113:0/350204367 >> v2:172.21.15.178:3300/0 conn(0x7fb4e415a6b0 msgr2=0x7fb4e4070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.634+0000 7fb4dbfff640 1 --2- 172.21.15.113:0/350204367 >> v2:172.21.15.178:3300/0 conn(0x7fb4e415a6b0 0x7fb4e4070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.634+0000 7fb4dbfff640 1 -- 172.21.15.113:0/350204367 >> v1:172.21.15.178:6789/0 conn(0x7fb4e4071780 legacy=0x7fb4e415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.634+0000 7fb4dbfff640 1 -- 172.21.15.113:0/350204367 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e40b32f0 con 0x7fb4e4072ea0 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.634+0000 7fb4dbfff640 1 --2- 172.21.15.113:0/350204367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4071240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097a0 tx=0x7fb4cc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e65442dfb298fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e259c640 1 -- 172.21.15.113:0/350204367 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037070 con 0x7fb4e4072ea0 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e259c640 1 -- 172.21.15.113:0/350204367 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc032440 con 0x7fb4e4072ea0 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e259c640 1 -- 172.21.15.113:0/350204367 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc032810 con 0x7fb4e4072ea0 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 -- 172.21.15.113:0/350204367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 msgr2=0x7fb4e4071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 --2- 172.21.15.113:0/350204367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4071240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097a0 tx=0x7fb4cc02a180 comp rx=0 tx=0).stop 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 -- 172.21.15.113:0/350204367 shutdown_connections 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 --2- 172.21.15.113:0/350204367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4071240 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 --2- 172.21.15.113:0/350204367 >> v2:172.21.15.178:3300/0 conn(0x7fb4e415a6b0 0x7fb4e4070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 -- 172.21.15.113:0/350204367 >> 172.21.15.113:0/350204367 conn(0x7fb4e40b1a50 msgr2=0x7fb4e40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 -- 172.21.15.113:0/350204367 shutdown_connections 2024-04-24T23:58:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.635+0000 7fb4e9064640 1 -- 172.21.15.113:0/350204367 wait complete. 2024-04-24T23:58:59.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 Processor -- start 2024-04-24T23:58:59.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 -- start start 2024-04-24T23:58:59.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e4071780 0x7fb4e4141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e415a6b0 0x7fb4e413da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e415aa90 con 0x7fb4e4071780 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e4072720 con 0x7fb4e415a6b0 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e9064640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e415ac10 con 0x7fb4e4072ea0 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e4071780 0x7fb4e4141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e4071780 0x7fb4e4141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42626/0 (socket says 172.21.15.113:42626) 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.637+0000 7fb4e2d9d640 1 -- 172.21.15.113:0/2158413139 learned_addr learned my addr 172.21.15.113:0/2158413139 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4dbfff640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e2d9d640 1 -- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 msgr2=0x7fb4e4153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e2d9d640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e2d9d640 1 -- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e415a6b0 msgr2=0x7fb4e413da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e2d9d640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e415a6b0 0x7fb4e413da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e2d9d640 1 -- 172.21.15.113:0/2158413139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e413e770 con 0x7fb4e4071780 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4dbfff640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e2d9d640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e4071780 0x7fb4e4141bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e6c0 tx=0x7fb4d400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4db7fe640 1 -- 172.21.15.113:0/2158413139 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4016070 con 0x7fb4e4071780 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4db7fe640 1 -- 172.21.15.113:0/2158413139 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d400ca30 con 0x7fb4e4071780 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e413ea50 con 0x7fb4e4071780 2024-04-24T23:58:59.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4db7fe640 1 -- 172.21.15.113:0/2158413139 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400b040 con 0x7fb4e4071780 2024-04-24T23:58:59.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.638+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e413f010 con 0x7fb4e4071780 2024-04-24T23:58:59.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.639+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e413f440 con 0x7fb4e4071780 2024-04-24T23:58:59.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.640+0000 7fb4db7fe640 1 -- 172.21.15.113:0/2158413139 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4d4019070 con 0x7fb4e4071780 2024-04-24T23:58:59.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.640+0000 7fb4db7fe640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4c406be30 0x7fb4c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.640+0000 7fb4db7fe640 1 -- 172.21.15.113:0/2158413139 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fb4d408ad30 con 0x7fb4e4071780 2024-04-24T23:58:59.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.640+0000 7fb4dbfff640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4c406be30 0x7fb4c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.640+0000 7fb4dbfff640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4c406be30 0x7fb4c406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.644+0000 7fb4db7fe640 1 -- 172.21.15.113:0/2158413139 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb4d4059660 con 0x7fb4e4071780 2024-04-24T23:58:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.818+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4e40b2900 con 0x7fb4e4071780 2024-04-24T23:58:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.820+0000 7fb4db7fe640 1 -- 172.21.15.113:0/2158413139 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16845 (secure 0 0 0) 0x7fb4d4058f60 con 0x7fb4e4071780 2024-04-24T23:58:59.820 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:58:59.820 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:58:59.358714+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:58:17.001246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:58:17.549246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:58:59.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4c406be30 msgr2=0x7fb4c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4c406be30 0x7fb4c406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).stop 2024-04-24T23:58:59.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e4071780 msgr2=0x7fb4e4141bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e4071780 0x7fb4e4141bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e6c0 tx=0x7fb4d400eb80 comp rx=0 tx=0).stop 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 shutdown_connections 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4c406be30 0x7fb4c406e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e415a6b0 0x7fb4e413da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e4072ea0 0x7fb4e4153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.821+0000 7fb4e9064640 1 --2- 172.21.15.113:0/2158413139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e4071780 0x7fb4e4141bc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.822+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 >> 172.21.15.113:0/2158413139 conn(0x7fb4e40b1a50 msgr2=0x7fb4e40cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.822+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 shutdown_connections 2024-04-24T23:58:59.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.822+0000 7fb4e9064640 1 -- 172.21.15.113:0/2158413139 wait complete. 2024-04-24T23:58:59.833 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:58:59.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.930+0000 7fadc84e1640 1 Processor -- start 2024-04-24T23:58:59.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.930+0000 7fadc84e1640 1 -- start start 2024-04-24T23:58:59.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc84e1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fadc00b5ca0 0x7fadc00b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc84e1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadc00b73c0 0x7fadc00b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc84e1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc00d2e00 con 0x7fadc015e510 2024-04-24T23:58:59.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc84e1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fadc00b6640 con 0x7fadc00b73c0 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc84e1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fadc00d30e0 con 0x7fadc00b5ca0 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc6a7e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadc00b73c0 0x7fadc00b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc627d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fadc015e510 0x7fadc00d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42976/0 (socket says 172.21.15.113:42976) 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc627d640 1 -- 172.21.15.113:0/2048350850 learned_addr learned my addr 172.21.15.113:0/2048350850 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc6a7e640 1 -- 172.21.15.113:0/2048350850 >> v2:172.21.15.178:3300/0 conn(0x7fadc00b5ca0 msgr2=0x7fadc00b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc6a7e640 1 --2- 172.21.15.113:0/2048350850 >> v2:172.21.15.178:3300/0 conn(0x7fadc00b5ca0 0x7fadc00b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc6a7e640 1 -- 172.21.15.113:0/2048350850 >> v1:172.21.15.113:6789/0 conn(0x7fadc015e510 legacy=0x7fadc00d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc6a7e640 1 -- 172.21.15.113:0/2048350850 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc00d33c0 con 0x7fadc00b73c0 2024-04-24T23:58:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.931+0000 7fadc527b640 1 -- 172.21.15.113:0/2048350850 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642287242 0 0) 0x7fadc00d2e00 con 0x7fadc015e510 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.932+0000 7fadc6a7e640 1 --2- 172.21.15.113:0/2048350850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b73c0 0x7fadc00b77d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadc00d52f0 tx=0x7fadb802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56a314fa7f336e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.932+0000 7fadc527b640 1 -- 172.21.15.113:0/2048350850 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8007740 con 0x7fadc00b73c0 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.932+0000 7fadc527b640 1 -- 172.21.15.113:0/2048350850 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb80078e0 con 0x7fadc00b73c0 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.932+0000 7fadc527b640 1 -- 172.21.15.113:0/2048350850 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8007cb0 con 0x7fadc00b73c0 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.932+0000 7fadc84e1640 1 -- 172.21.15.113:0/2048350850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b73c0 msgr2=0x7fadc00b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.932+0000 7fadc84e1640 1 --2- 172.21.15.113:0/2048350850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b73c0 0x7fadc00b77d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadc00d52f0 tx=0x7fadb802a5f0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.933+0000 7fadc84e1640 1 -- 172.21.15.113:0/2048350850 shutdown_connections 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.933+0000 7fadc84e1640 1 --2- 172.21.15.113:0/2048350850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b73c0 0x7fadc00b77d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.933+0000 7fadc84e1640 1 --2- 172.21.15.113:0/2048350850 >> v2:172.21.15.178:3300/0 conn(0x7fadc00b5ca0 0x7fadc00b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.933+0000 7fadc84e1640 1 -- 172.21.15.113:0/2048350850 >> 172.21.15.113:0/2048350850 conn(0x7fadc006b720 msgr2=0x7fadc00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.933+0000 7fadc84e1640 1 -- 172.21.15.113:0/2048350850 shutdown_connections 2024-04-24T23:58:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.933+0000 7fadc84e1640 1 -- 172.21.15.113:0/2048350850 wait complete. 2024-04-24T23:58:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 Processor -- start 2024-04-24T23:58:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 -- start start 2024-04-24T23:58:59.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b5ca0 0x7fadc014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc00b73c0 0x7fadc0156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadc015e510 0x7fadc014d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadc00d3ea0 con 0x7fadc00b5ca0 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadc00d3980 con 0x7fadc00b73c0 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc84e1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadc00d2a90 con 0x7fadc015e510 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc627d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b5ca0 0x7fadc014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc627d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b5ca0 0x7fadc014c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42634/0 (socket says 172.21.15.113:42634) 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.934+0000 7fadc627d640 1 -- 172.21.15.113:0/3588577541 learned_addr learned my addr 172.21.15.113:0/3588577541 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc5a7c640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc00b73c0 0x7fadc0156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc6a7e640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadc015e510 0x7fadc014d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc627d640 1 -- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadc015e510 msgr2=0x7fadc014d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc627d640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadc015e510 0x7fadc014d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc627d640 1 -- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc00b73c0 msgr2=0x7fadc0156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc627d640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc00b73c0 0x7fadc0156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc627d640 1 -- 172.21.15.113:0/3588577541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc00c7fa0 con 0x7fadc00b5ca0 2024-04-24T23:58:59.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc6a7e640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadc015e510 0x7fadc014d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc627d640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b5ca0 0x7fadc014c5d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c4d0 tx=0x7fadbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadb6ffd640 1 -- 172.21.15.113:0/3588577541 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc018070 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadb6ffd640 1 -- 172.21.15.113:0/3588577541 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc007500 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc00c8280 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadb6ffd640 1 -- 172.21.15.113:0/3588577541 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc004a20 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.935+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc00c8840 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.936+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc00691a0 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.936+0000 7fadb6ffd640 1 -- 172.21.15.113:0/3588577541 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadbc01b070 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.937+0000 7fadb6ffd640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fada406be60 0x7fada406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.937+0000 7fadb6ffd640 1 -- 172.21.15.113:0/3588577541 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fadbc08a1a0 con 0x7fadc00b5ca0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.937+0000 7fadc5a7c640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fada406be60 0x7fada406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:59.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.937+0000 7fadc5a7c640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fada406be60 0x7fada406e320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fadc01590c0 tx=0x7fadb0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:59.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:58:59.941+0000 7fadb6ffd640 1 -- 172.21.15.113:0/3588577541 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fadbc058b60 con 0x7fadc00b5ca0 2024-04-24T23:59:00.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.149+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fadc0155bf0 con 0x7fadc00b5ca0 2024-04-24T23:59:00.150 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-24T23:59:00.150+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:59:00.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.151+0000 7fadb6ffd640 1 -- 172.21.15.113:0/3588577541 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7fadbc058460 con 0x7fadc00b5ca0 2024-04-24T23:59:00.151 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:59:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.154+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fada406be60 msgr2=0x7fada406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.154+0000 7fadc84e1640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fada406be60 0x7fada406e320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fadc01590c0 tx=0x7fadb0008040 comp rx=0 tx=0).stop 2024-04-24T23:59:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.154+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b5ca0 msgr2=0x7fadc014c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.154+0000 7fadc84e1640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b5ca0 0x7fadc014c5d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c4d0 tx=0x7fadbc00c990 comp rx=0 tx=0).stop 2024-04-24T23:59:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 shutdown_connections 2024-04-24T23:59:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fada406be60 0x7fada406e320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadc00b73c0 0x7fadc0156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadc015e510 0x7fadc014d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 --2- 172.21.15.113:0/3588577541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc00b5ca0 0x7fadc014c5d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 >> 172.21.15.113:0/3588577541 conn(0x7fadc006b720 msgr2=0x7fadc015d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:00.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 shutdown_connections 2024-04-24T23:59:00.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.155+0000 7fadc84e1640 1 -- 172.21.15.113:0/3588577541 wait complete. 2024-04-24T23:59:00.158 INFO:tasks.ceph.mds.c.smithi113.stderr: -7380> 2024-04-24T23:58:23.621+0000 7f44c5f17640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:59:00.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:59:00.189 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-24T23:59:00.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.267+0000 7f02dd7b0640 1 Processor -- start 2024-04-24T23:59:00.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.267+0000 7f02dd7b0640 1 -- start start 2024-04-24T23:59:00.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02dd7b0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f02d80cf7a0 0x7f02d80d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02dd7b0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02d80d2a20 0x7f02d80d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02dd7b0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d8162560 con 0x7f02d80cf3c0 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02dd7b0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f02d8162840 con 0x7f02d80d2a20 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02dd7b0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f02d8162b20 con 0x7f02d80cf7a0 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02d77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02d80d2a20 0x7f02d80d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02d77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02d80d2a20 0x7f02d80d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42658/0 (socket says 172.21.15.113:42658) 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02d77fe640 1 -- 172.21.15.113:0/164702367 learned_addr learned my addr 172.21.15.113:0/164702367 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02cffff640 1 --2- 172.21.15.113:0/164702367 >> v2:172.21.15.178:3300/0 conn(0x7f02d80cf7a0 0x7f02d80d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02d77fe640 1 -- 172.21.15.113:0/164702367 >> v2:172.21.15.178:3300/0 conn(0x7f02d80cf7a0 msgr2=0x7f02d80d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02d77fe640 1 --2- 172.21.15.113:0/164702367 >> v2:172.21.15.178:3300/0 conn(0x7f02d80cf7a0 0x7f02d80d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02d77fe640 1 -- 172.21.15.113:0/164702367 >> v1:172.21.15.178:6789/0 conn(0x7f02d80cf3c0 legacy=0x7f02d80d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02d77fe640 1 -- 172.21.15.113:0/164702367 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d8162e00 con 0x7f02d80d2a20 2024-04-24T23:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.268+0000 7f02cffff640 1 --2- 172.21.15.113:0/164702367 >> v2:172.21.15.178:3300/0 conn(0x7f02d80cf7a0 0x7f02d80d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.269+0000 7f02d77fe640 1 --2- 172.21.15.113:0/164702367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d1210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f02d8161f00 tx=0x7f02d002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfa89d3fef612ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.269+0000 7f02d67fc640 1 -- 172.21.15.113:0/164702367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0036070 con 0x7f02d80d2a20 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.269+0000 7f02d67fc640 1 -- 172.21.15.113:0/164702367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d0031600 con 0x7f02d80d2a20 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.269+0000 7f02d67fc640 1 -- 172.21.15.113:0/164702367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d00319d0 con 0x7f02d80d2a20 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 -- 172.21.15.113:0/164702367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 msgr2=0x7f02d80d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/164702367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d1210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f02d8161f00 tx=0x7f02d002a6c0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 -- 172.21.15.113:0/164702367 shutdown_connections 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/164702367 >> v2:172.21.15.178:3300/0 conn(0x7f02d80cf7a0 0x7f02d80d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/164702367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d1210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 -- 172.21.15.113:0/164702367 >> 172.21.15.113:0/164702367 conn(0x7f02d80718f0 msgr2=0x7f02d8071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 -- 172.21.15.113:0/164702367 shutdown_connections 2024-04-24T23:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.270+0000 7f02dd7b0640 1 -- 172.21.15.113:0/164702367 wait complete. 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.271+0000 7f02dd7b0640 1 Processor -- start 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.271+0000 7f02dd7b0640 1 -- start start 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02d80cf3c0 0x7f02d80ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02d80cf7a0 0x7f02d80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f02d80d1750 con 0x7f02d80d2a20 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02d81621d0 con 0x7f02d80cf3c0 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02d80d18d0 con 0x7f02d80cf7a0 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42666/0 (socket says 172.21.15.113:42666) 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 -- 172.21.15.113:0/3756463815 learned_addr learned my addr 172.21.15.113:0/3756463815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02cffff640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02d80cf7a0 0x7f02d80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d6ffd640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02d80cf3c0 0x7f02d80ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 -- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02d80cf7a0 msgr2=0x7f02d80cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02d80cf7a0 0x7f02d80cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 -- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02d80cf3c0 msgr2=0x7f02d80ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02d80cf3c0 0x7f02d80ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 -- 172.21.15.113:0/3756463815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d80ca9d0 con 0x7f02d80d2a20 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d6ffd640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02d80cf3c0 0x7f02d80ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02d77fe640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d2e00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009f10 tx=0x7f02d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02cf7fe640 1 -- 172.21.15.113:0/3756463815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0036050 con 0x7f02d80d2a20 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02cf7fe640 1 -- 172.21.15.113:0/3756463815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d0042070 con 0x7f02d80d2a20 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d80b7590 con 0x7f02d80d2a20 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02cf7fe640 1 -- 172.21.15.113:0/3756463815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0009510 con 0x7f02d80d2a20 2024-04-24T23:59:00.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.272+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d80b7b50 con 0x7f02d80d2a20 2024-04-24T23:59:00.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.273+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d80691a0 con 0x7f02d80d2a20 2024-04-24T23:59:00.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.274+0000 7f02cf7fe640 1 -- 172.21.15.113:0/3756463815 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02d0007bd0 con 0x7f02d80d2a20 2024-04-24T23:59:00.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.274+0000 7f02cf7fe640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f02b806be30 0x7f02b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:00.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.274+0000 7f02cf7fe640 1 -- 172.21.15.113:0/3756463815 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5911+0+0 (secure 0 0 0) 0x7f02d00abe50 con 0x7f02d80d2a20 2024-04-24T23:59:00.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.274+0000 7f02d6ffd640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f02b806be30 0x7f02b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:00.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.275+0000 7f02d6ffd640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f02b806be30 0x7f02b806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02c8003eb0 tx=0x7f02c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:00.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.278+0000 7f02cf7fe640 1 -- 172.21.15.113:0/3756463815 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02d007a730 con 0x7f02d80d2a20 2024-04-24T23:59:00.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:00.489+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f02d815c950 con 0x7f02d80d2a20 2024-04-24T23:59:01.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.152+0000 7f02cf7fe640 1 -- 172.21.15.113:0/3756463815 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f02d007a030 con 0x7f02d80d2a20 2024-04-24T23:59:01.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f02b806be30 msgr2=0x7f02b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f02b806be30 0x7f02b806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02c8003eb0 tx=0x7f02c8009040 comp rx=0 tx=0).stop 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 msgr2=0x7f02d80d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d2e00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009f10 tx=0x7f02d0004a40 comp rx=0 tx=0).stop 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 shutdown_connections 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f02b806be30 0x7f02b806e2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d80d2a20 0x7f02d80d2e00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02d80cf7a0 0x7f02d80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 --2- 172.21.15.113:0/3756463815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02d80cf3c0 0x7f02d80ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 >> 172.21.15.113:0/3756463815 conn(0x7f02d80718f0 msgr2=0x7f02d815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 shutdown_connections 2024-04-24T23:59:01.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.154+0000 7f02dd7b0640 1 -- 172.21.15.113:0/3756463815 wait complete. 2024-04-24T23:59:01.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:59:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.310+0000 7f419ca6b640 1 Processor -- start 2024-04-24T23:59:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.310+0000 7f419ca6b640 1 -- start start 2024-04-24T23:59:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419ca6b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4198145890 0x7f41980ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419ca6b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f419806e7e0 0x7f4198071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419ca6b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41980cf3a0 con 0x7f41980ced20 2024-04-24T23:59:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419ca6b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4198145c70 con 0x7f4198145890 2024-04-24T23:59:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419ca6b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f41980cf680 con 0x7f419806e7e0 2024-04-24T23:59:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4198145890 0x7f41980ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4198145890 0x7f41980ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42680/0 (socket says 172.21.15.113:42680) 2024-04-24T23:59:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419659c640 1 -- 172.21.15.113:0/2923590001 learned_addr learned my addr 172.21.15.113:0/2923590001 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419659c640 1 -- 172.21.15.113:0/2923590001 >> v2:172.21.15.178:3300/0 conn(0x7f419806e7e0 msgr2=0x7f4198071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419659c640 1 --2- 172.21.15.113:0/2923590001 >> v2:172.21.15.178:3300/0 conn(0x7f419806e7e0 0x7f4198071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419659c640 1 -- 172.21.15.113:0/2923590001 >> v1:172.21.15.178:6789/0 conn(0x7f41980ced20 legacy=0x7f419806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.311+0000 7f419659c640 1 -- 172.21.15.113:0/2923590001 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41980cf960 con 0x7f4198145890 2024-04-24T23:59:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419659c640 1 --2- 172.21.15.113:0/2923590001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f41980ce7e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f41980b60d0 tx=0x7f418802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b6926a01b659b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f4195d9b640 1 -- 172.21.15.113:0/2923590001 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188037070 con 0x7f4198145890 2024-04-24T23:59:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f4195d9b640 1 -- 172.21.15.113:0/2923590001 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4188032440 con 0x7f4198145890 2024-04-24T23:59:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f4195d9b640 1 -- 172.21.15.113:0/2923590001 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188032810 con 0x7f4198145890 2024-04-24T23:59:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 -- 172.21.15.113:0/2923590001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 msgr2=0x7f41980ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 --2- 172.21.15.113:0/2923590001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f41980ce7e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f41980b60d0 tx=0x7f418802a230 comp rx=0 tx=0).stop 2024-04-24T23:59:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 -- 172.21.15.113:0/2923590001 shutdown_connections 2024-04-24T23:59:01.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 --2- 172.21.15.113:0/2923590001 >> v2:172.21.15.178:3300/0 conn(0x7f419806e7e0 0x7f4198071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 --2- 172.21.15.113:0/2923590001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f41980ce7e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 -- 172.21.15.113:0/2923590001 >> 172.21.15.113:0/2923590001 conn(0x7f41980b1b30 msgr2=0x7f41980b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:01.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 -- 172.21.15.113:0/2923590001 shutdown_connections 2024-04-24T23:59:01.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.312+0000 7f419ca6b640 1 -- 172.21.15.113:0/2923590001 wait complete. 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.314+0000 7f419ca6b640 1 Processor -- start 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.314+0000 7f419ca6b640 1 -- start start 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f419806e7e0 0x7f41980ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41980ced20 0x7f4198138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f419815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4198071970 con 0x7f4198145890 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41980b53b0 con 0x7f41980ced20 2024-04-24T23:59:01.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4198071af0 con 0x7f419806e7e0 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f419815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f419815d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42692/0 (socket says 172.21.15.113:42692) 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 -- 172.21.15.113:0/3397528215 learned_addr learned my addr 172.21.15.113:0/3397528215 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419659c640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f419806e7e0 0x7f41980ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f418dd9b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41980ced20 0x7f4198138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 -- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f419806e7e0 msgr2=0x7f41980ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f419806e7e0 0x7f41980ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 -- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41980ced20 msgr2=0x7f4198138630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41980ced20 0x7f4198138630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 -- 172.21.15.113:0/3397528215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4198138b70 con 0x7f4198145890 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f418dd9b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41980ced20 0x7f4198138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f4196d9d640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f419815d580 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f419000b970 tx=0x7f419000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:01.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f418f7fe640 1 -- 172.21.15.113:0/3397528215 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190019070 con 0x7f4198145890 2024-04-24T23:59:01.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f418f7fe640 1 -- 172.21.15.113:0/3397528215 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419000cb60 con 0x7f4198145890 2024-04-24T23:59:01.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41980aed90 con 0x7f4198145890 2024-04-24T23:59:01.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f418f7fe640 1 -- 172.21.15.113:0/3397528215 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190014440 con 0x7f4198145890 2024-04-24T23:59:01.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.315+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41980b30b0 con 0x7f4198145890 2024-04-24T23:59:01.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.316+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41980af2b0 con 0x7f4198145890 2024-04-24T23:59:01.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.317+0000 7f418f7fe640 1 -- 172.21.15.113:0/3397528215 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f419001c070 con 0x7f4198145890 2024-04-24T23:59:01.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.317+0000 7f418f7fe640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f417806be30 0x7f417806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:01.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.317+0000 7f418f7fe640 1 -- 172.21.15.113:0/3397528215 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5911+0+0 (secure 0 0 0) 0x7f419008ac90 con 0x7f4198145890 2024-04-24T23:59:01.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.317+0000 7f419659c640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f417806be30 0x7f417806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:01.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.318+0000 7f419659c640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f417806be30 0x7f417806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4188034000 tx=0x7f418802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:01.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.321+0000 7f418f7fe640 1 -- 172.21.15.113:0/3397528215 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4190059570 con 0x7f4198145890 2024-04-24T23:59:01.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:01.494+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f41980691a0 con 0x7f4198145890 2024-04-24T23:59:02.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.371+0000 7f418f7fe640 1 -- 172.21.15.113:0/3397528215 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f4190058e70 con 0x7f4198145890 2024-04-24T23:59:02.372 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:59:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f417806be30 msgr2=0x7f417806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f417806be30 0x7f417806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4188034000 tx=0x7f418802e000 comp rx=0 tx=0).stop 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 msgr2=0x7f419815d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f419815d580 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f419000b970 tx=0x7f419000be30 comp rx=0 tx=0).stop 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 shutdown_connections 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f417806be30 0x7f417806e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41980ced20 0x7f4198138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4198145890 0x7f419815d580 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 --2- 172.21.15.113:0/3397528215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f419806e7e0 0x7f41980ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.376+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 >> 172.21.15.113:0/3397528215 conn(0x7f41980b1b30 msgr2=0x7f41980b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.377+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 shutdown_connections 2024-04-24T23:59:02.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.377+0000 7f419ca6b640 1 -- 172.21.15.113:0/3397528215 wait complete. 2024-04-24T23:59:02.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:59:02.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.496+0000 7f859b163640 1 Processor -- start 2024-04-24T23:59:02.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.496+0000 7f859b163640 1 -- start start 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.496+0000 7f859b163640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8594144a80 0x7f85940b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.496+0000 7f859b163640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f85940cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f859b163640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8594072f70 con 0x7f85940b3ff0 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f859b163640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f85940b2fd0 con 0x7f8594144a80 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f859b163640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85940b32b0 con 0x7f859406e7e0 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8598eff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8594144a80 0x7f85940b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8593fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f85940b3ff0 0x7f859406e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42984/0 (socket says 172.21.15.113:42984) 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8593fff640 1 -- 172.21.15.113:0/3632129480 learned_addr learned my addr 172.21.15.113:0/3632129480 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8599700640 1 --2- 172.21.15.113:0/3632129480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f85940cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:02.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8598eff640 1 -- 172.21.15.113:0/3632129480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 msgr2=0x7f85940cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8598eff640 1 --2- 172.21.15.113:0/3632129480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f85940cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8598eff640 1 -- 172.21.15.113:0/3632129480 >> v1:172.21.15.113:6789/0 conn(0x7f85940b3ff0 legacy=0x7f859406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8598eff640 1 -- 172.21.15.113:0/3632129480 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940b3590 con 0x7f8594144a80 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f85937fe640 1 -- 172.21.15.113:0/3632129480 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537796145 0 0) 0x7f8594072f70 con 0x7f85940b3ff0 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.497+0000 7f8599700640 1 --2- 172.21.15.113:0/3632129480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f85940cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f8598eff640 1 --2- 172.21.15.113:0/3632129480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8594144a80 0x7f85940b3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f85840097a0 tx=0x7f858402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d59e90a5e3f127a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f85937fe640 1 -- 172.21.15.113:0/3632129480 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584037070 con 0x7f8594144a80 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f85937fe640 1 -- 172.21.15.113:0/3632129480 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858402ae50 con 0x7f8594144a80 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f85937fe640 1 -- 172.21.15.113:0/3632129480 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584032660 con 0x7f8594144a80 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 -- 172.21.15.113:0/3632129480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8594144a80 msgr2=0x7f85940b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 --2- 172.21.15.113:0/3632129480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8594144a80 0x7f85940b3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f85840097a0 tx=0x7f858402a100 comp rx=0 tx=0).stop 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 -- 172.21.15.113:0/3632129480 shutdown_connections 2024-04-24T23:59:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 --2- 172.21.15.113:0/3632129480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f85940cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 --2- 172.21.15.113:0/3632129480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8594144a80 0x7f85940b3ab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 -- 172.21.15.113:0/3632129480 >> 172.21.15.113:0/3632129480 conn(0x7f85940b1b30 msgr2=0x7f85940b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:02.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 -- 172.21.15.113:0/3632129480 shutdown_connections 2024-04-24T23:59:02.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.498+0000 7f859b163640 1 -- 172.21.15.113:0/3632129480 wait complete. 2024-04-24T23:59:02.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.499+0000 7f859b163640 1 Processor -- start 2024-04-24T23:59:02.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f859b163640 1 -- start start 2024-04-24T23:59:02.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f859b163640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f859413a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f859b163640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85940b3ff0 0x7f859415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f859b163640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8594144a80 0x7f859415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f859b163640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cdf00 con 0x7f85940b3ff0 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f859b163640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85940cdd80 con 0x7f8594144a80 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f859b163640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8594072200 con 0x7f859406e7e0 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f8593fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85940b3ff0 0x7f859415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f8593fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85940b3ff0 0x7f859415b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42724/0 (socket says 172.21.15.113:42724) 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.500+0000 7f8593fff640 1 -- 172.21.15.113:0/1503756615 learned_addr learned my addr 172.21.15.113:0/1503756615 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8598eff640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f859413a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8593fff640 1 -- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 msgr2=0x7f859413a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8593fff640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f859413a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8599700640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8594144a80 0x7f859415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8593fff640 1 -- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8594144a80 msgr2=0x7f859415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8593fff640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8594144a80 0x7f859415b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8593fff640 1 -- 172.21.15.113:0/1503756615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85941e5760 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8598eff640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f859413a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8599700640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8594144a80 0x7f859415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f8593fff640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85940b3ff0 0x7f859415b4b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c500 tx=0x7f857c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f85917fa640 1 -- 172.21.15.113:0/1503756615 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c018070 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f85917fa640 1 -- 172.21.15.113:0/1503756615 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c007580 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85941e5a40 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f85917fa640 1 -- 172.21.15.113:0/1503756615 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c004a20 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.501+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85941e6000 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.502+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85940691a0 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.503+0000 7f85917fa640 1 -- 172.21.15.113:0/1503756615 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f857c01b070 con 0x7f85940b3ff0 2024-04-24T23:59:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.503+0000 7f85917fa640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f857406be30 0x7f857406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:02.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.503+0000 7f85917fa640 1 -- 172.21.15.113:0/1503756615 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5473+0+0 (secure 0 0 0) 0x7f857c08a060 con 0x7f85940b3ff0 2024-04-24T23:59:02.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.503+0000 7f8598eff640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f857406be30 0x7f857406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:02.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.504+0000 7f8598eff640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f857406be30 0x7f857406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8584034000 tx=0x7f858403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:02.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.507+0000 7f85917fa640 1 -- 172.21.15.113:0/1503756615 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f857c058b80 con 0x7f85940b3ff0 2024-04-24T23:59:02.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:02.680+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85940b5c80 con 0x7f85940b3ff0 2024-04-24T23:59:03.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.377+0000 7f85917fa640 1 -- 172.21.15.113:0/1503756615 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f857c058480 con 0x7f85940b3ff0 2024-04-24T23:59:03.378 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-24T23:59:03.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.381+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f857406be30 msgr2=0x7f857406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:03.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.381+0000 7f859b163640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f857406be30 0x7f857406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8584034000 tx=0x7f858403c040 comp rx=0 tx=0).stop 2024-04-24T23:59:03.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.381+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85940b3ff0 msgr2=0x7f859415b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.381+0000 7f859b163640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85940b3ff0 0x7f859415b4b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c500 tx=0x7f857c00c9c0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 shutdown_connections 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f857406be30 0x7f857406e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8594144a80 0x7f859415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85940b3ff0 0x7f859415b4b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 --2- 172.21.15.113:0/1503756615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f859406e7e0 0x7f859413a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 >> 172.21.15.113:0/1503756615 conn(0x7f85940b1b30 msgr2=0x7f85940b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 shutdown_connections 2024-04-24T23:59:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.382+0000 7f859b163640 1 -- 172.21.15.113:0/1503756615 wait complete. 2024-04-24T23:59:03.397 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T23:59:03.398 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T23:59:03.398 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T23:59:03.398 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:59:03.398 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T23:59:03.398 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T23:59:03.398 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T23:59:03.398 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:59:03.399 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T23:59:03.399 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T23:59:03.399 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T23:59:03.399 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:59:03.399 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T23:59:03.399 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T23:59:03.399 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T23:59:03.399 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:59:03.408 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T23:59:03.408 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T23:59:03.409 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T23:59:03.409 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T23:59:03.410 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T23:59:03.410 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T23:59:03.411 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T23:59:03.411 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T23:59:03.450 INFO:tasks.ceph.mds.a:Started 2024-04-24T23:59:03.451 DEBUG:teuthology.parallel:result is None 2024-04-24T23:59:03.451 INFO:tasks.ceph.mds.c:Started 2024-04-24T23:59:03.451 DEBUG:teuthology.parallel:result is None 2024-04-24T23:59:03.453 INFO:tasks.ceph.mds.b:Started 2024-04-24T23:59:03.454 DEBUG:teuthology.parallel:result is None 2024-04-24T23:59:03.454 INFO:tasks.ceph.mds.d:Started 2024-04-24T23:59:03.454 DEBUG:teuthology.parallel:result is None 2024-04-24T23:59:03.454 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T23:59:03.524 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-24T23:59:03.525 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-24T23:59:03.531 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-24T23:59:03.532 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-24T23:59:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.556+0000 7fce495b0640 1 Processor -- start 2024-04-24T23:59:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.556+0000 7fce495b0640 1 -- start start 2024-04-24T23:59:03.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce495b0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce4415e510 0x7fce440d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:03.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce495b0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce440d28c0 0x7fce440b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:03.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce495b0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce440d2e00 con 0x7fce440b61e0 2024-04-24T23:59:03.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce495b0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fce440d30e0 con 0x7fce440d28c0 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce495b0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce440d33c0 con 0x7fce4415e510 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce42ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce4415e510 0x7fce440d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce427fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce440d28c0 0x7fce440b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce42ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce4415e510 0x7fce440d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:47782/0 (socket says 172.21.15.113:47782) 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce437fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fce440b61e0 0x7fce44071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:53198/0 (socket says 172.21.15.113:53198) 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce42ffd640 1 -- 172.21.15.113:0/181146289 learned_addr learned my addr 172.21.15.113:0/181146289 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce42ffd640 1 -- 172.21.15.113:0/181146289 >> v2:172.21.15.178:3300/0 conn(0x7fce440d28c0 msgr2=0x7fce440b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce42ffd640 1 --2- 172.21.15.113:0/181146289 >> v2:172.21.15.178:3300/0 conn(0x7fce440d28c0 0x7fce440b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce42ffd640 1 -- 172.21.15.113:0/181146289 >> v1:172.21.15.178:6789/0 conn(0x7fce440b61e0 legacy=0x7fce44071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce41ffb640 1 -- 172.21.15.113:0/181146289 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551313037 0 0) 0x7fce440d2e00 con 0x7fce440b61e0 2024-04-24T23:59:03.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.557+0000 7fce42ffd640 1 -- 172.21.15.113:0/181146289 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce440d36a0 con 0x7fce4415e510 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.558+0000 7fce42ffd640 1 --2- 172.21.15.113:0/181146289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce4415e510 0x7fce440d2380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fce38009870 tx=0x7fce3802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28b2f10f554644f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.558+0000 7fce41ffb640 1 -- 172.21.15.113:0/181146289 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38037070 con 0x7fce4415e510 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.558+0000 7fce41ffb640 1 -- 172.21.15.113:0/181146289 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce38032440 con 0x7fce4415e510 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.558+0000 7fce41ffb640 1 -- 172.21.15.113:0/181146289 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38032810 con 0x7fce4415e510 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 -- 172.21.15.113:0/181146289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce4415e510 msgr2=0x7fce440d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 --2- 172.21.15.113:0/181146289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce4415e510 0x7fce440d2380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fce38009870 tx=0x7fce3802a1c0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 -- 172.21.15.113:0/181146289 shutdown_connections 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 --2- 172.21.15.113:0/181146289 >> v2:172.21.15.178:3300/0 conn(0x7fce440d28c0 0x7fce440b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 --2- 172.21.15.113:0/181146289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce4415e510 0x7fce440d2380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 -- 172.21.15.113:0/181146289 >> 172.21.15.113:0/181146289 conn(0x7fce4406b720 msgr2=0x7fce440d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 -- 172.21.15.113:0/181146289 shutdown_connections 2024-04-24T23:59:03.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.559+0000 7fce495b0640 1 -- 172.21.15.113:0/181146289 wait complete. 2024-04-24T23:59:03.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.560+0000 7fce495b0640 1 Processor -- start 2024-04-24T23:59:03.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.560+0000 7fce495b0640 1 -- start start 2024-04-24T23:59:03.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce440b61e0 0x7fce4414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce440d28c0 0x7fce4415f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce4415e510 0x7fce440c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce440b9dd0 con 0x7fce440d28c0 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce440d3ea0 con 0x7fce4415e510 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce440b83d0 con 0x7fce440b61e0 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce440d28c0 0x7fce4415f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce440d28c0 0x7fce4415f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42746/0 (socket says 172.21.15.113:42746) 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 -- 172.21.15.113:0/518424923 learned_addr learned my addr 172.21.15.113:0/518424923 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce42ffd640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce440b61e0 0x7fce4414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce437fe640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce4415e510 0x7fce440c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 -- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce440b61e0 msgr2=0x7fce4414d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce440b61e0 0x7fce4414d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 -- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce4415e510 msgr2=0x7fce440c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce4415e510 0x7fce440c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 -- 172.21.15.113:0/518424923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4414dc00 con 0x7fce440d28c0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce437fe640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce4415e510 0x7fce440c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce427fc640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce440d28c0 0x7fce4415f390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9b0 tx=0x7fce3400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce2b7fe640 1 -- 172.21.15.113:0/518424923 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34017070 con 0x7fce440d28c0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce2b7fe640 1 -- 172.21.15.113:0/518424923 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3400c9a0 con 0x7fce440d28c0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce2b7fe640 1 -- 172.21.15.113:0/518424923 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3400cd90 con 0x7fce440d28c0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4415fce0 con 0x7fce440d28c0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.561+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce440cbe70 con 0x7fce440d28c0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.562+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce440691a0 con 0x7fce440d28c0 2024-04-24T23:59:03.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.563+0000 7fce2b7fe640 1 -- 172.21.15.113:0/518424923 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce3401a070 con 0x7fce440d28c0 2024-04-24T23:59:03.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.563+0000 7fce2b7fe640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce3006be60 0x7fce3006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:03.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.563+0000 7fce2b7fe640 1 -- 172.21.15.113:0/518424923 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7fce3408aac0 con 0x7fce440d28c0 2024-04-24T23:59:03.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.563+0000 7fce42ffd640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce3006be60 0x7fce3006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:03.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.564+0000 7fce42ffd640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce3006be60 0x7fce3006e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:03.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.568+0000 7fce2b7fe640 1 -- 172.21.15.113:0/518424923 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce4414dc00 con 0x7fce440d28c0 2024-04-24T23:59:03.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:03.750+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fce440ba040 con 0x7fce440d28c0 2024-04-24T23:59:04.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.392+0000 7fce2b7fe640 1 -- 172.21.15.113:0/518424923 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7fce34059800 con 0x7fce440d28c0 2024-04-24T23:59:04.393 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-24T23:59:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce3006be60 msgr2=0x7fce3006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce3006be60 0x7fce3006e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802e000 comp rx=0 tx=0).stop 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce440d28c0 msgr2=0x7fce4415f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce440d28c0 0x7fce4415f390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9b0 tx=0x7fce3400ee70 comp rx=0 tx=0).stop 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 shutdown_connections 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce3006be60 0x7fce3006e320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce4415e510 0x7fce440c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce440d28c0 0x7fce4415f390 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 --2- 172.21.15.113:0/518424923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce440b61e0 0x7fce4414d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 >> 172.21.15.113:0/518424923 conn(0x7fce4406b720 msgr2=0x7fce440b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.394+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 shutdown_connections 2024-04-24T23:59:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.395+0000 7fce495b0640 1 -- 172.21.15.113:0/518424923 wait complete. 2024-04-24T23:59:04.409 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T23:59:04.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.507+0000 7feeb482a640 1 Processor -- start 2024-04-24T23:59:04.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.507+0000 7feeb482a640 1 -- start start 2024-04-24T23:59:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.507+0000 7feeb482a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feeb00d2290 0x7feeb00b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.507+0000 7feeb482a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.507+0000 7feeb482a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeb00d2800 con 0x7feeb015e320 2024-04-24T23:59:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.507+0000 7feeb482a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7feeb00d14d0 con 0x7feeb00d2290 2024-04-24T23:59:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.507+0000 7feeb482a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feeb00d3410 con 0x7feeb00d0b00 2024-04-24T23:59:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feeaed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feea7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feeb00d2290 0x7feeb00b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feeaed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb00d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:47802/0 (socket says 172.21.15.113:47802) 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feeae59c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7feeb015e320 0x7feeb00d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:53210/0 (socket says 172.21.15.113:53210) 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feea7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feeb00d2290 0x7feeb00b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:46036/0 (socket says 172.21.15.113:46036) 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feeaed9d640 1 -- 172.21.15.113:0/2033068190 learned_addr learned my addr 172.21.15.113:0/2033068190 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feea7fff640 1 -- 172.21.15.113:0/2033068190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 msgr2=0x7feeb00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feea7fff640 1 --2- 172.21.15.113:0/2033068190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feea7fff640 1 -- 172.21.15.113:0/2033068190 >> v1:172.21.15.178:6789/0 conn(0x7feeb015e320 legacy=0x7feeb00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.508+0000 7feea7fff640 1 -- 172.21.15.113:0/2033068190 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb00d2ae0 con 0x7feeb00d2290 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feea7fff640 1 --2- 172.21.15.113:0/2033068190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 0x7feeb00b9aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee980097a0 tx=0x7fee9802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54a190eb23dd8256 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeadd9b640 1 -- 172.21.15.113:0/2033068190 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98037070 con 0x7feeb00d2290 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeadd9b640 1 -- 172.21.15.113:0/2033068190 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee98032440 con 0x7feeb00d2290 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeadd9b640 1 -- 172.21.15.113:0/2033068190 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98032810 con 0x7feeb00d2290 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeb482a640 1 -- 172.21.15.113:0/2033068190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 msgr2=0x7feeb00b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeb482a640 1 --2- 172.21.15.113:0/2033068190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 0x7feeb00b9aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee980097a0 tx=0x7fee9802a210 comp rx=0 tx=0).stop 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeb482a640 1 -- 172.21.15.113:0/2033068190 shutdown_connections 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeb482a640 1 --2- 172.21.15.113:0/2033068190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeb482a640 1 --2- 172.21.15.113:0/2033068190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 0x7feeb00b9aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeb482a640 1 -- 172.21.15.113:0/2033068190 >> 172.21.15.113:0/2033068190 conn(0x7feeb00b5c20 msgr2=0x7feeb00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.509+0000 7feeb482a640 1 -- 172.21.15.113:0/2033068190 shutdown_connections 2024-04-24T23:59:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.510+0000 7feeb482a640 1 -- 172.21.15.113:0/2033068190 wait complete. 2024-04-24T23:59:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 Processor -- start 2024-04-24T23:59:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 -- start start 2024-04-24T23:59:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb01f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 0x7feeb00ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb015e320 0x7feeb00ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feeb00b9fe0 con 0x7feeb015e320 2024-04-24T23:59:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feeb00d30a0 con 0x7feeb00d2290 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feeb015e700 con 0x7feeb00d0b00 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb015e320 0x7feeb00ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb015e320 0x7feeb00ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42752/0 (socket says 172.21.15.113:42752) 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 -- 172.21.15.113:0/3087575737 learned_addr learned my addr 172.21.15.113:0/3087575737 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feea7fff640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 0x7feeb00ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeae59c640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb01f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 -- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 msgr2=0x7feeb01f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb01f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 -- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 msgr2=0x7feeb00ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 0x7feeb00ac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 -- 172.21.15.113:0/3087575737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb00ac0e0 con 0x7feeb015e320 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeaed9d640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb015e320 0x7feeb00ac950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feea8009d90 tx=0x7feea800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feea6ffd640 1 -- 172.21.15.113:0/3087575737 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea800cd20 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feea6ffd640 1 -- 172.21.15.113:0/3087575737 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea8003990 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb015eaf0 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feea6ffd640 1 -- 172.21.15.113:0/3087575737 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8003d60 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.512+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb015f0b0 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.513+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb00691a0 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.514+0000 7feea6ffd640 1 -- 172.21.15.113:0/3087575737 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feea8005d10 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.514+0000 7feea6ffd640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee9006be30 0x7fee9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.514+0000 7feeae59c640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee9006be30 0x7fee9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.514+0000 7feea6ffd640 1 -- 172.21.15.113:0/3087575737 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7feea808af50 con 0x7feeb015e320 2024-04-24T23:59:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.514+0000 7feeae59c640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee9006be30 0x7fee9006e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feea0000f60 tx=0x7feea0007e20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:04.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.519+0000 7feea6ffd640 1 -- 172.21.15.113:0/3087575737 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feea8022020 con 0x7feeb015e320 2024-04-24T23:59:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.756+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7feeb0074f50 con 0x7feeb015e320 2024-04-24T23:59:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.757+0000 7feea6ffd640 1 -- 172.21.15.113:0/3087575737 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7feea8010040 con 0x7feeb015e320 2024-04-24T23:59:04.757 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:04.757 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-24T23:59:04.757 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-24T23:59:04.757 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.757 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-24T23:59:04.757 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-24T23:59:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-24T23:59:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.761 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.762 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.763 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.764 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.764 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-24T23:59:04.764 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-24T23:59:04.764 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.764 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.764 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.765 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-24T23:59:04.766 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-24T23:59:04.767 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-24T23:59:04.768 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-24T23:59:04.769 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-24T23:59:04.770 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.770 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-24T23:59:04.770 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-24T23:59:04.770 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-24T23:59:04.770 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-24T23:59:04.770 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.bootstrap-mds", 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQCCnClm5QptOxAAqOuXypMIfi9Pp5G2ElnhhA==", 2024-04-24T23:59:04.771 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile bootstrap-mds" 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.bootstrap-mgr", 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQCCnClmexNtOxAAxQz1zj1OplE5wC5yhLMWzA==", 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.bootstrap-osd", 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQCCnClmFhptOxAARpPgu4RWcqAgXPxcuvX2PA==", 2024-04-24T23:59:04.772 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile bootstrap-osd" 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.bootstrap-rbd", 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQCCnClmcyBtOxAAXrFa268iyh3kGVCSFq0VCQ==", 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.773 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQCCnClmPydtOxAA9imdST8UmxvWWw/HyikySg==", 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.bootstrap-rgw", 2024-04-24T23:59:04.774 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQCCnClmHS5tOxAA9PKOG7HT58CpkiPS4Ba3nw==", 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-24T23:59:04.776 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-24T23:59:04.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.758+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee9006be30 msgr2=0x7fee9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.758+0000 7feeb482a640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee9006be30 0x7fee9006e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feea0000f60 tx=0x7feea0007e20 comp rx=0 tx=0).stop 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.758+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb015e320 msgr2=0x7feeb00ac950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.758+0000 7feeb482a640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb015e320 0x7feeb00ac950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feea8009d90 tx=0x7feea800c490 comp rx=0 tx=0).stop 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 shutdown_connections 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee9006be30 0x7fee9006e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb00d2290 0x7feeb00ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb015e320 0x7feeb00ac950 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 --2- 172.21.15.113:0/3087575737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb00d0b00 0x7feeb01f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 >> 172.21.15.113:0/3087575737 conn(0x7feeb00b5c20 msgr2=0x7feeb00d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 shutdown_connections 2024-04-24T23:59:04.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.759+0000 7feeb482a640 1 -- 172.21.15.113:0/3087575737 wait complete. 2024-04-24T23:59:04.780 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-24T23:59:04.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.878+0000 7fd223638640 1 Processor -- start 2024-04-24T23:59:04.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.878+0000 7fd223638640 1 -- start start 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd223638640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd21c0b5ca0 0x7fd21c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd223638640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd21c0b73c0 0x7fd21c0b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd223638640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c0d2e00 con 0x7fd21c15e510 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd223638640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd21c0b6640 con 0x7fd21c0b5ca0 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd223638640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd21c0d30e0 con 0x7fd21c0b73c0 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd220bd3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd21c0b5ca0 0x7fd21c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd220bd3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd21c0b5ca0 0x7fd21c0b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42764/0 (socket says 172.21.15.113:42764) 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.879+0000 7fd220bd3640 1 -- 172.21.15.113:0/2012915507 learned_addr learned my addr 172.21.15.113:0/2012915507 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd221bd5640 1 --2- 172.21.15.113:0/2012915507 >> v2:172.21.15.178:3300/0 conn(0x7fd21c0b73c0 0x7fd21c0b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd220bd3640 1 -- 172.21.15.113:0/2012915507 >> v2:172.21.15.178:3300/0 conn(0x7fd21c0b73c0 msgr2=0x7fd21c0b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd220bd3640 1 --2- 172.21.15.113:0/2012915507 >> v2:172.21.15.178:3300/0 conn(0x7fd21c0b73c0 0x7fd21c0b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd220bd3640 1 -- 172.21.15.113:0/2012915507 >> v1:172.21.15.178:6789/0 conn(0x7fd21c15e510 legacy=0x7fd21c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd220bd3640 1 -- 172.21.15.113:0/2012915507 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c0d33c0 con 0x7fd21c0b5ca0 2024-04-24T23:59:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd220bd3640 1 --2- 172.21.15.113:0/2012915507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b5ca0 0x7fd21c0b6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0d1d80 tx=0x7fd20c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd471493b616d030 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd213fff640 1 -- 172.21.15.113:0/2012915507 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037070 con 0x7fd21c0b5ca0 2024-04-24T23:59:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd213fff640 1 -- 172.21.15.113:0/2012915507 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c032440 con 0x7fd21c0b5ca0 2024-04-24T23:59:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.880+0000 7fd213fff640 1 -- 172.21.15.113:0/2012915507 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c032810 con 0x7fd21c0b5ca0 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 -- 172.21.15.113:0/2012915507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b5ca0 msgr2=0x7fd21c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 --2- 172.21.15.113:0/2012915507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b5ca0 0x7fd21c0b6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0d1d80 tx=0x7fd20c02a1e0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 -- 172.21.15.113:0/2012915507 shutdown_connections 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 --2- 172.21.15.113:0/2012915507 >> v2:172.21.15.178:3300/0 conn(0x7fd21c0b73c0 0x7fd21c0b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 --2- 172.21.15.113:0/2012915507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b5ca0 0x7fd21c0b6100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 -- 172.21.15.113:0/2012915507 >> 172.21.15.113:0/2012915507 conn(0x7fd21c06b720 msgr2=0x7fd21c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 -- 172.21.15.113:0/2012915507 shutdown_connections 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.881+0000 7fd223638640 1 -- 172.21.15.113:0/2012915507 wait complete. 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.882+0000 7fd223638640 1 Processor -- start 2024-04-24T23:59:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.882+0000 7fd223638640 1 -- start start 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd223638640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd21c0b5ca0 0x7fd21c1f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd223638640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b73c0 0x7fd21c158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd223638640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd21c15e510 0x7fd21c154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd223638640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0d3ea0 con 0x7fd21c0b73c0 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd223638640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0d3980 con 0x7fd21c15e510 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd223638640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c0d2a90 con 0x7fd21c0b5ca0 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b73c0 0x7fd21c158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b73c0 0x7fd21c158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42766/0 (socket says 172.21.15.113:42766) 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 -- 172.21.15.113:0/4094081196 learned_addr learned my addr 172.21.15.113:0/4094081196 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 -- 172.21.15.113:0/4094081196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd21c0b5ca0 msgr2=0x7fd21c1f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd21c0b5ca0 0x7fd21c1f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 -- 172.21.15.113:0/4094081196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd21c15e510 msgr2=0x7fd21c154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd21c15e510 0x7fd21c154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 -- 172.21.15.113:0/4094081196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c158e70 con 0x7fd21c0b73c0 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.883+0000 7fd220bd3640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b73c0 0x7fd21c158930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a220 tx=0x7fd20c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.884+0000 7fd211ffb640 1 -- 172.21.15.113:0/4094081196 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c041020 con 0x7fd21c0b73c0 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.884+0000 7fd211ffb640 1 -- 172.21.15.113:0/4094081196 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c0306c0 con 0x7fd21c0b73c0 2024-04-24T23:59:04.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.884+0000 7fd211ffb640 1 -- 172.21.15.113:0/4094081196 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037040 con 0x7fd21c0b73c0 2024-04-24T23:59:04.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.884+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c1f7ce0 con 0x7fd21c0b73c0 2024-04-24T23:59:04.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.884+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1f82a0 con 0x7fd21c0b73c0 2024-04-24T23:59:04.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.885+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c0691a0 con 0x7fd21c0b73c0 2024-04-24T23:59:04.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.885+0000 7fd211ffb640 1 -- 172.21.15.113:0/4094081196 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd20c044020 con 0x7fd21c0b73c0 2024-04-24T23:59:04.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.886+0000 7fd211ffb640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd20006be30 0x7fd20006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:04.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.886+0000 7fd211ffb640 1 -- 172.21.15.113:0/4094081196 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd20c0ab350 con 0x7fd21c0b73c0 2024-04-24T23:59:04.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.886+0000 7fd2213d4640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd20006be30 0x7fd20006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:04.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.886+0000 7fd2213d4640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd20006be30 0x7fd20006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd2180012e0 tx=0x7fd218007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:04.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:04.890+0000 7fd211ffb640 1 -- 172.21.15.113:0/4094081196 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd20c04b030 con 0x7fd21c0b73c0 2024-04-24T23:59:05.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.133+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd21c0b6420 con 0x7fd21c0b73c0 2024-04-24T23:59:05.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.134+0000 7fd211ffb640 1 -- 172.21.15.113:0/4094081196 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fd20c07a050 con 0x7fd21c0b73c0 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.136+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd20006be30 msgr2=0x7fd20006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.136+0000 7fd223638640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd20006be30 0x7fd20006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd2180012e0 tx=0x7fd218007d50 comp rx=0 tx=0).stop 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b73c0 msgr2=0x7fd21c158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b73c0 0x7fd21c158930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a220 tx=0x7fd20c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 shutdown_connections 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd20006be30 0x7fd20006e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd21c15e510 0x7fd21c154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd21c0b73c0 0x7fd21c158930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 --2- 172.21.15.113:0/4094081196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd21c0b5ca0 0x7fd21c1f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 >> 172.21.15.113:0/4094081196 conn(0x7fd21c06b720 msgr2=0x7fd21c15ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:05.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 shutdown_connections 2024-04-24T23:59:05.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.137+0000 7fd223638640 1 -- 172.21.15.113:0/4094081196 wait complete. 2024-04-24T23:59:05.147 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-24T23:59:05.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.245+0000 7f1a4693b640 1 Processor -- start 2024-04-24T23:59:05.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.245+0000 7f1a4693b640 1 -- start start 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.245+0000 7f1a4693b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a400b3080 0x7f1a400b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.245+0000 7f1a4693b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4015f4f0 con 0x7f1a4015e6b0 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a4693b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4015f7d0 con 0x7f1a4015e2d0 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a4693b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1a400cde00 con 0x7f1a400b3080 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a3f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a400b3080 0x7f1a400b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a44ed8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1a4015e6b0 0x7f1a400ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42998/0 (socket says 172.21.15.113:42998) 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a44ed8640 1 -- 172.21.15.113:0/778722415 learned_addr learned my addr 172.21.15.113:0/778722415 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:05.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a3effd640 1 -- 172.21.15.113:0/778722415 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049824148 0 0) 0x7f1a4015f4f0 con 0x7f1a4015e6b0 2024-04-24T23:59:05.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a3effd640 1 -- 172.21.15.113:0/778722415 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a28003520 con 0x7f1a4015e6b0 2024-04-24T23:59:05.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a3f7fe640 1 -- 172.21.15.113:0/778722415 >> v1:172.21.15.178:6789/0 conn(0x7f1a4015e2d0 legacy=0x7f1a4015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:05.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a3f7fe640 1 -- 172.21.15.113:0/778722415 >> v1:172.21.15.113:6789/0 conn(0x7f1a4015e6b0 legacy=0x7f1a400ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.246+0000 7f1a3f7fe640 1 -- 172.21.15.113:0/778722415 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4015fab0 con 0x7f1a400b3080 2024-04-24T23:59:05.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a3f7fe640 1 --2- 172.21.15.113:0/778722415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a400b3080 0x7f1a400b4330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1a340097a0 tx=0x7f1a3402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aec7b914b9789d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:05.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a3effd640 1 -- 172.21.15.113:0/778722415 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34037070 con 0x7f1a400b3080 2024-04-24T23:59:05.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a3effd640 1 -- 172.21.15.113:0/778722415 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a34032440 con 0x7f1a400b3080 2024-04-24T23:59:05.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a3effd640 1 -- 172.21.15.113:0/778722415 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34032810 con 0x7f1a400b3080 2024-04-24T23:59:05.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a4693b640 1 -- 172.21.15.113:0/778722415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a400b3080 msgr2=0x7f1a400b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a4693b640 1 --2- 172.21.15.113:0/778722415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a400b3080 0x7f1a400b4330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1a340097a0 tx=0x7f1a3402a1e0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a4693b640 1 -- 172.21.15.113:0/778722415 shutdown_connections 2024-04-24T23:59:05.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a4693b640 1 --2- 172.21.15.113:0/778722415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a400b3080 0x7f1a400b4330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a4693b640 1 -- 172.21.15.113:0/778722415 >> 172.21.15.113:0/778722415 conn(0x7f1a4006b0f0 msgr2=0x7f1a4006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:05.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a4693b640 1 -- 172.21.15.113:0/778722415 shutdown_connections 2024-04-24T23:59:05.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.247+0000 7f1a4693b640 1 -- 172.21.15.113:0/778722415 wait complete. 2024-04-24T23:59:05.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.248+0000 7f1a4693b640 1 Processor -- start 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a4693b640 1 -- start start 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a4693b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a400b3080 0x7f1a4014eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a4693b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a4015e2d0 0x7f1a40143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a4693b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4015e6b0 0x7f1a40153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a4693b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a400b49f0 con 0x7f1a4015e6b0 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a4693b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1a400b4870 con 0x7f1a4015e2d0 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a4693b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1a400b4b70 con 0x7f1a400b3080 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a44ed8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4015e6b0 0x7f1a40153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a44ed8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4015e6b0 0x7f1a40153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42782/0 (socket says 172.21.15.113:42782) 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.249+0000 7f1a44ed8640 1 -- 172.21.15.113:0/3764404500 learned_addr learned my addr 172.21.15.113:0/3764404500 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a44ed8640 1 -- 172.21.15.113:0/3764404500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a400b3080 msgr2=0x7f1a4014eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a44ed8640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a400b3080 0x7f1a4014eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a3f7fe640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a4015e2d0 0x7f1a40143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a44ed8640 1 -- 172.21.15.113:0/3764404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a4015e2d0 msgr2=0x7f1a40143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a44ed8640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a4015e2d0 0x7f1a40143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a44ed8640 1 -- 172.21.15.113:0/3764404500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a401535b0 con 0x7f1a4015e6b0 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a44ed8640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4015e6b0 0x7f1a40153070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800cb40 tx=0x7f1a3800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a3cff9640 1 -- 172.21.15.113:0/3764404500 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38017070 con 0x7f1a4015e6b0 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a3cff9640 1 -- 172.21.15.113:0/3764404500 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3800aca0 con 0x7f1a4015e6b0 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a40154930 con 0x7f1a4015e6b0 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a3cff9640 1 -- 172.21.15.113:0/3764404500 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38012620 con 0x7f1a4015e6b0 2024-04-24T23:59:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.250+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a400ae960 con 0x7f1a4015e6b0 2024-04-24T23:59:05.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.251+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a40154e50 con 0x7f1a4015e6b0 2024-04-24T23:59:05.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.252+0000 7f1a3cff9640 1 -- 172.21.15.113:0/3764404500 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a3801a070 con 0x7f1a4015e6b0 2024-04-24T23:59:05.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.252+0000 7f1a3cff9640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a2c06be30 0x7f1a2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.252+0000 7f1a3cff9640 1 -- 172.21.15.113:0/3764404500 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f1a3808a950 con 0x7f1a4015e6b0 2024-04-24T23:59:05.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.252+0000 7f1a3ffff640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a2c06be30 0x7f1a2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.252+0000 7f1a3ffff640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a2c06be30 0x7f1a2c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a30003cc0 tx=0x7f1a300079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:05.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.256+0000 7f1a3cff9640 1 -- 172.21.15.113:0/3764404500 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a38059650 con 0x7f1a4015e6b0 2024-04-24T23:59:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.496+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f1a400d0f70 con 0x7f1a4015e6b0 2024-04-24T23:59:05.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.500+0000 7f1a3cff9640 1 -- 172.21.15.113:0/3764404500 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f1a38058f50 con 0x7f1a4015e6b0 2024-04-24T23:59:05.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a2c06be30 msgr2=0x7f1a2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a2c06be30 0x7f1a2c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a30003cc0 tx=0x7f1a300079b0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4015e6b0 msgr2=0x7f1a40153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4015e6b0 0x7f1a40153070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800cb40 tx=0x7f1a3800a060 comp rx=0 tx=0).stop 2024-04-24T23:59:05.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 shutdown_connections 2024-04-24T23:59:05.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a2c06be30 0x7f1a2c06e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a4015e2d0 0x7f1a40143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4015e6b0 0x7f1a40153070 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 --2- 172.21.15.113:0/3764404500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a400b3080 0x7f1a4014eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 >> 172.21.15.113:0/3764404500 conn(0x7f1a4006b0f0 msgr2=0x7f1a400b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:05.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 shutdown_connections 2024-04-24T23:59:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.502+0000 7f1a4693b640 1 -- 172.21.15.113:0/3764404500 wait complete. 2024-04-24T23:59:05.554 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-24T23:59:05.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.651+0000 7fc4fbfb7640 1 Processor -- start 2024-04-24T23:59:05.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.652+0000 7fc4fbfb7640 1 -- start start 2024-04-24T23:59:05.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.652+0000 7fc4fbfb7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4f415b670 0x7fc4f40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.652+0000 7fc4fbfb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415bb20 0x7fc4f40cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.652+0000 7fc4fbfb7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f40d0740 con 0x7fc4f415bff0 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.652+0000 7fc4fbfb7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f4071170 con 0x7fc4f415b670 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.652+0000 7fc4fbfb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f4071450 con 0x7fc4f415bb20 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9552640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4f415b670 0x7fc4f40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9552640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4f415b670 0x7fc4f40cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42786/0 (socket says 172.21.15.113:42786) 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9d53640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc4f415bff0 0x7fc4f40d0030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43004/0 (socket says 172.21.15.113:43004) 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9552640 1 -- 172.21.15.113:0/3883522451 learned_addr learned my addr 172.21.15.113:0/3883522451 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4fa554640 1 --2- 172.21.15.113:0/3883522451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415bb20 0x7fc4f40cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9552640 1 -- 172.21.15.113:0/3883522451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415bb20 msgr2=0x7fc4f40cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9552640 1 --2- 172.21.15.113:0/3883522451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415bb20 0x7fc4f40cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9552640 1 -- 172.21.15.113:0/3883522451 >> v1:172.21.15.113:6789/0 conn(0x7fc4f415bff0 legacy=0x7fc4f40d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.653+0000 7fc4f9552640 1 -- 172.21.15.113:0/3883522451 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f4071730 con 0x7fc4f415b670 2024-04-24T23:59:05.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4f9552640 1 --2- 172.21.15.113:0/3883522451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415b670 0x7fc4f40cc200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400bb10 tx=0x7fc4e402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f39f365ae74a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:05.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4f8d51640 1 -- 172.21.15.113:0/3883522451 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4032700 con 0x7fc4f415b670 2024-04-24T23:59:05.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4f8d51640 1 -- 172.21.15.113:0/3883522451 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e40328a0 con 0x7fc4f415b670 2024-04-24T23:59:05.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4f8d51640 1 -- 172.21.15.113:0/3883522451 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4032c70 con 0x7fc4f415b670 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/3883522451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415b670 msgr2=0x7fc4f40cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/3883522451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415b670 0x7fc4f40cc200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400bb10 tx=0x7fc4e402c800 comp rx=0 tx=0).stop 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/3883522451 shutdown_connections 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/3883522451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415bb20 0x7fc4f40cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/3883522451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415b670 0x7fc4f40cc200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/3883522451 >> 172.21.15.113:0/3883522451 conn(0x7fc4f406b0f0 msgr2=0x7fc4f406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/3883522451 shutdown_connections 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.654+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/3883522451 wait complete. 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.655+0000 7fc4fbfb7640 1 Processor -- start 2024-04-24T23:59:05.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fbfb7640 1 -- start start 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fbfb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415b670 0x7fc4f414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fbfb7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f415bb20 0x7fc4f4159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fbfb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415bff0 0x7fc4f4159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fbfb7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f406cea0 con 0x7fc4f415bff0 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fbfb7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f415c3d0 con 0x7fc4f415bb20 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fbfb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f406d020 con 0x7fc4f415b670 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fa554640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415bff0 0x7fc4f4159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fa554640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415bff0 0x7fc4f4159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42790/0 (socket says 172.21.15.113:42790) 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.656+0000 7fc4fa554640 1 -- 172.21.15.113:0/545889184 learned_addr learned my addr 172.21.15.113:0/545889184 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4f9d53640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415b670 0x7fc4f414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4f9552640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f415bb20 0x7fc4f4159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fa554640 1 -- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415b670 msgr2=0x7fc4f414dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fa554640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415b670 0x7fc4f414dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fa554640 1 -- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f415bb20 msgr2=0x7fc4f4159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fa554640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f415bb20 0x7fc4f4159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fa554640 1 -- 172.21.15.113:0/545889184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f4156d30 con 0x7fc4f415bff0 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fa554640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415bff0 0x7fc4f4159750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007e40 tx=0x7fc4ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4ea7fc640 1 -- 172.21.15.113:0/545889184 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec014620 con 0x7fc4f415bff0 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4ea7fc640 1 -- 172.21.15.113:0/545889184 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec019070 con 0x7fc4f415bff0 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f4157010 con 0x7fc4f415bff0 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4ea7fc640 1 -- 172.21.15.113:0/545889184 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec014a20 con 0x7fc4f415bff0 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.657+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f414ef50 con 0x7fc4f415bff0 2024-04-24T23:59:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.658+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f40691a0 con 0x7fc4f415bff0 2024-04-24T23:59:05.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.658+0000 7fc4ea7fc640 1 -- 172.21.15.113:0/545889184 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4ec014bc0 con 0x7fc4f415bff0 2024-04-24T23:59:05.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.659+0000 7fc4ea7fc640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4d806be60 0x7fc4d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:05.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.659+0000 7fc4f9d53640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4d806be60 0x7fc4d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:05.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.659+0000 7fc4ea7fc640 1 -- 172.21.15.113:0/545889184 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fc4ec08b470 con 0x7fc4f415bff0 2024-04-24T23:59:05.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.659+0000 7fc4f9d53640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4d806be60 0x7fc4d806e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0000f60 tx=0x7fc4f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:05.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.663+0000 7fc4ea7fc640 1 -- 172.21.15.113:0/545889184 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4ec05a170 con 0x7fc4f415bff0 2024-04-24T23:59:05.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.905+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fc4f414ed20 con 0x7fc4f415bff0 2024-04-24T23:59:05.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.908+0000 7fc4ea7fc640 1 -- 172.21.15.113:0/545889184 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fc4ec059a70 con 0x7fc4f415bff0 2024-04-24T23:59:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4d806be60 msgr2=0x7fc4d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4d806be60 0x7fc4d806e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0000f60 tx=0x7fc4f0009040 comp rx=0 tx=0).stop 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415bff0 msgr2=0x7fc4f4159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415bff0 0x7fc4f4159750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007e40 tx=0x7fc4ec00e4b0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 shutdown_connections 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4d806be60 0x7fc4d806e320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f415bb20 0x7fc4f4159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f415bff0 0x7fc4f4159750 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 --2- 172.21.15.113:0/545889184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f415b670 0x7fc4f414dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 >> 172.21.15.113:0/545889184 conn(0x7fc4f406b0f0 msgr2=0x7fc4f40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 shutdown_connections 2024-04-24T23:59:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:05.911+0000 7fc4fbfb7640 1 -- 172.21.15.113:0/545889184 wait complete. 2024-04-24T23:59:05.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-24T23:59:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.020+0000 7f9205276640 1 Processor -- start 2024-04-24T23:59:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.020+0000 7f9205276640 1 -- start start 2024-04-24T23:59:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f9205276640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f92000cf3c0 0x7f92000d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f9205276640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f9205276640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9200162560 con 0x7f92000d2a20 2024-04-24T23:59:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f9205276640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9200162840 con 0x7f92000cf3c0 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f9205276640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9200162b20 con 0x7f92000cf7a0 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91feffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f92000cf3c0 0x7f92000d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:47834/0 (socket says 172.21.15.113:47834) 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91f7fff640 1 -- 172.21.15.113:0/3781645449 learned_addr learned my addr 172.21.15.113:0/3781645449 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91ff7fe640 1 --1- 172.21.15.113:0/3781645449 >> v1:172.21.15.178:6789/0 conn(0x7f92000d2a20 0x7f92000d1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:53244/0 (socket says 172.21.15.113:53244) 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91f7fff640 1 -- 172.21.15.113:0/3781645449 >> v2:172.21.15.178:3300/0 conn(0x7f92000cf3c0 msgr2=0x7f92000d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91f7fff640 1 --2- 172.21.15.113:0/3781645449 >> v2:172.21.15.178:3300/0 conn(0x7f92000cf3c0 0x7f92000d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91f7fff640 1 -- 172.21.15.113:0/3781645449 >> v1:172.21.15.178:6789/0 conn(0x7f92000d2a20 legacy=0x7f92000d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.021+0000 7f91f7fff640 1 -- 172.21.15.113:0/3781645449 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92000d40f0 con 0x7f92000cf7a0 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.022+0000 7f91f7fff640 1 --2- 172.21.15.113:0/3781645449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000d24e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009870 tx=0x7f91e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08b9a56d857925f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.022+0000 7f91fe7fc640 1 -- 172.21.15.113:0/3781645449 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8037070 con 0x7f92000cf7a0 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.022+0000 7f91fe7fc640 1 -- 172.21.15.113:0/3781645449 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e8032440 con 0x7f92000cf7a0 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.022+0000 7f91fe7fc640 1 -- 172.21.15.113:0/3781645449 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8032810 con 0x7f92000cf7a0 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 -- 172.21.15.113:0/3781645449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 msgr2=0x7f92000d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 --2- 172.21.15.113:0/3781645449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000d24e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009870 tx=0x7f91e802a270 comp rx=0 tx=0).stop 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 -- 172.21.15.113:0/3781645449 shutdown_connections 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 --2- 172.21.15.113:0/3781645449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000d24e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 --2- 172.21.15.113:0/3781645449 >> v2:172.21.15.178:3300/0 conn(0x7f92000cf3c0 0x7f92000d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 -- 172.21.15.113:0/3781645449 >> 172.21.15.113:0/3781645449 conn(0x7f92000718f0 msgr2=0x7f9200071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 -- 172.21.15.113:0/3781645449 shutdown_connections 2024-04-24T23:59:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.023+0000 7f9205276640 1 -- 172.21.15.113:0/3781645449 wait complete. 2024-04-24T23:59:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.024+0000 7f9205276640 1 Processor -- start 2024-04-24T23:59:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.024+0000 7f9205276640 1 -- start start 2024-04-24T23:59:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f9205276640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92000cf3c0 0x7f920014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f9205276640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f9205276640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92000d2a20 0x7f92000ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f9205276640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92000d1aa0 con 0x7f92000d2a20 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f9205276640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92000d1920 con 0x7f92000cf3c0 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f9205276640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9200161fc0 con 0x7f92000cf7a0 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92000d2a20 0x7f92000ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92000d2a20 0x7f92000ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42800/0 (socket says 172.21.15.113:42800) 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 -- 172.21.15.113:0/1925444172 learned_addr learned my addr 172.21.15.113:0/1925444172 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91feffd640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92000cf3c0 0x7f920014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91f7fff640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 -- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 msgr2=0x7f92000c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 -- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92000cf3c0 msgr2=0x7f920014b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92000cf3c0 0x7f920014b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 -- 172.21.15.113:0/1925444172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92000b8070 con 0x7f92000d2a20 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.025+0000 7f91ff7fe640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92000d2a20 0x7f92000ca320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c6c0 tx=0x7f91f800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.026+0000 7f91f77fe640 1 -- 172.21.15.113:0/1925444172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8018070 con 0x7f92000d2a20 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.026+0000 7f91f77fe640 1 -- 172.21.15.113:0/1925444172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8004d60 con 0x7f92000d2a20 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.026+0000 7f91f77fe640 1 -- 172.21.15.113:0/1925444172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f800a840 con 0x7f92000d2a20 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.026+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92000b8350 con 0x7f92000d2a20 2024-04-24T23:59:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.026+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92000b8910 con 0x7f92000d2a20 2024-04-24T23:59:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.027+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92000691a0 con 0x7f92000d2a20 2024-04-24T23:59:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.027+0000 7f91f77fe640 1 -- 172.21.15.113:0/1925444172 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91f800a9e0 con 0x7f92000d2a20 2024-04-24T23:59:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.027+0000 7f91f77fe640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.027+0000 7f91f77fe640 1 -- 172.21.15.113:0/1925444172 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f91f8089ff0 con 0x7f92000d2a20 2024-04-24T23:59:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.027+0000 7f91feffd640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.028+0000 7f91feffd640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009ad0 tx=0x7f91f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.032+0000 7f91f77fe640 1 -- 172.21.15.113:0/1925444172 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91f8058620 con 0x7f92000d2a20 2024-04-24T23:59:06.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.274+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f92000d3be0 con 0x7f92000d2a20 2024-04-24T23:59:06.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.277+0000 7f91f77fe640 1 -- 172.21.15.113:0/1925444172 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f91f805c920 con 0x7f92000d2a20 2024-04-24T23:59:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 msgr2=0x7f91e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009ad0 tx=0x7f91f0008040 comp rx=0 tx=0).stop 2024-04-24T23:59:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92000d2a20 msgr2=0x7f92000ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92000d2a20 0x7f92000ca320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c6c0 tx=0x7f91f800cb80 comp rx=0 tx=0).stop 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 shutdown_connections 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92000d2a20 0x7f92000ca320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92000cf7a0 0x7f92000c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 --2- 172.21.15.113:0/1925444172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92000cf3c0 0x7f920014b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 >> 172.21.15.113:0/1925444172 conn(0x7f92000718f0 msgr2=0x7f92000b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 shutdown_connections 2024-04-24T23:59:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.280+0000 7f9205276640 1 -- 172.21.15.113:0/1925444172 wait complete. 2024-04-24T23:59:06.291 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-24T23:59:06.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.387+0000 7fd78fa31640 1 Processor -- start 2024-04-24T23:59:06.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.388+0000 7fd78fa31640 1 -- start start 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.388+0000 7fd78fa31640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd78815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.388+0000 7fd78fa31640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7880b3080 0x7fd7880b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.388+0000 7fd78fa31640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78815f4f0 con 0x7fd78815e6b0 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.388+0000 7fd78fa31640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd7880cde00 con 0x7fd7880b3080 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.388+0000 7fd78fa31640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd78815f7d0 con 0x7fd78815e2d0 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78cfcc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7880b3080 0x7fd7880b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78dfce640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd78815e6b0 0x7fd788071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43010/0 (socket says 172.21.15.113:43010) 2024-04-24T23:59:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78dfce640 1 -- 172.21.15.113:0/2910809522 learned_addr learned my addr 172.21.15.113:0/2910809522 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78d7cd640 1 --2- 172.21.15.113:0/2910809522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd78815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78cfcc640 1 -- 172.21.15.113:0/2910809522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 msgr2=0x7fd78815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78cfcc640 1 --2- 172.21.15.113:0/2910809522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd78815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78cfcc640 1 -- 172.21.15.113:0/2910809522 >> v1:172.21.15.113:6789/0 conn(0x7fd78815e6b0 legacy=0x7fd788071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd78cfcc640 1 -- 172.21.15.113:0/2910809522 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78815fab0 con 0x7fd7880b3080 2024-04-24T23:59:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.389+0000 7fd77ffff640 1 -- 172.21.15.113:0/2910809522 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810985802 0 0) 0x7fd78815f4f0 con 0x7fd78815e6b0 2024-04-24T23:59:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78cfcc640 1 --2- 172.21.15.113:0/2910809522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7880b3080 0x7fd7880b4330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd7780097a0 tx=0x7fd77802c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5931cca24456e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd77ffff640 1 -- 172.21.15.113:0/2910809522 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778039070 con 0x7fd7880b3080 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd77ffff640 1 -- 172.21.15.113:0/2910809522 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd778034640 con 0x7fd7880b3080 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd77ffff640 1 -- 172.21.15.113:0/2910809522 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77802ce60 con 0x7fd7880b3080 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 -- 172.21.15.113:0/2910809522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7880b3080 msgr2=0x7fd7880b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2910809522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7880b3080 0x7fd7880b4330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd7780097a0 tx=0x7fd77802c310 comp rx=0 tx=0).stop 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 -- 172.21.15.113:0/2910809522 shutdown_connections 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2910809522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7880b3080 0x7fd7880b4330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2910809522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd78815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 -- 172.21.15.113:0/2910809522 >> 172.21.15.113:0/2910809522 conn(0x7fd78806b0f0 msgr2=0x7fd78806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 -- 172.21.15.113:0/2910809522 shutdown_connections 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.390+0000 7fd78fa31640 1 -- 172.21.15.113:0/2910809522 wait complete. 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.391+0000 7fd78fa31640 1 Processor -- start 2024-04-24T23:59:06.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.391+0000 7fd78fa31640 1 -- start start 2024-04-24T23:59:06.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78fa31640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7880b3080 0x7fd7880b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78fa31640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd788142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78fa31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd78815e6b0 0x7fd7881429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78fa31640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd78815f2b0 con 0x7fd78815e6b0 2024-04-24T23:59:06.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78fa31640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd78815f130 con 0x7fd7880b3080 2024-04-24T23:59:06.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78fa31640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd7880cf2a0 con 0x7fd78815e2d0 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78dfce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd78815e6b0 0x7fd7881429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78dfce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd78815e6b0 0x7fd7881429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42812/0 (socket says 172.21.15.113:42812) 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78dfce640 1 -- 172.21.15.113:0/2299257257 learned_addr learned my addr 172.21.15.113:0/2299257257 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78cfcc640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd788142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.392+0000 7fd78d7cd640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7880b3080 0x7fd7880b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78dfce640 1 -- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 msgr2=0x7fd788142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78dfce640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd788142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78dfce640 1 -- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7880b3080 msgr2=0x7fd7880b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78dfce640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7880b3080 0x7fd7880b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78dfce640 1 -- 172.21.15.113:0/2299257257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd788142f00 con 0x7fd78815e6b0 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78d7cd640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7880b3080 0x7fd7880b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78dfce640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd78815e6b0 0x7fd7881429c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c410 tx=0x7fd78000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd77dffb640 1 -- 172.21.15.113:0/2299257257 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800074e0 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd77dffb640 1 -- 172.21.15.113:0/2299257257 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7800047e0 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78814ecf0 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd77dffb640 1 -- 172.21.15.113:0/2299257257 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780004bd0 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.393+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78814f590 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.394+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7880691a0 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.394+0000 7fd77dffb640 1 -- 172.21.15.113:0/2299257257 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd780011070 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.395+0000 7fd77dffb640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd76c06be30 0x7fd76c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.395+0000 7fd77dffb640 1 -- 172.21.15.113:0/2299257257 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd78008a5e0 con 0x7fd78815e6b0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.395+0000 7fd78d7cd640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd76c06be30 0x7fd76c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.395+0000 7fd78d7cd640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd76c06be30 0x7fd76c06e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd784007330 tx=0x7fd78400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.399+0000 7fd77dffb640 1 -- 172.21.15.113:0/2299257257 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd780059370 con 0x7fd78815e6b0 2024-04-24T23:59:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.639+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd788072050 con 0x7fd78815e6b0 2024-04-24T23:59:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.641+0000 7fd77dffb640 1 -- 172.21.15.113:0/2299257257 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fd78005f070 con 0x7fd78815e6b0 2024-04-24T23:59:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd76c06be30 msgr2=0x7fd76c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd76c06be30 0x7fd76c06e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd784007330 tx=0x7fd78400b040 comp rx=0 tx=0).stop 2024-04-24T23:59:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd78815e6b0 msgr2=0x7fd7881429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd78815e6b0 0x7fd7881429c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c410 tx=0x7fd78000c8d0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 shutdown_connections 2024-04-24T23:59:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd76c06be30 0x7fd76c06e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd78815e6b0 0x7fd7881429c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd78815e2d0 0x7fd788142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 --2- 172.21.15.113:0/2299257257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7880b3080 0x7fd7880b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 >> 172.21.15.113:0/2299257257 conn(0x7fd78806b0f0 msgr2=0x7fd788159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 shutdown_connections 2024-04-24T23:59:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.643+0000 7fd78fa31640 1 -- 172.21.15.113:0/2299257257 wait complete. 2024-04-24T23:59:06.653 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-24T23:59:06.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.751+0000 7f318129a640 1 Processor -- start 2024-04-24T23:59:06.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.751+0000 7f318129a640 1 -- start start 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f318129a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f317c15e510 0x7f317c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f318129a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0b63b0 0x7f317c0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f318129a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c0d2e00 con 0x7f317c0d28c0 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f318129a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f317c0d30e0 con 0x7f317c15e510 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f318129a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f317c0d33c0 con 0x7f317c0b63b0 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f317affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f317c15e510 0x7f317c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f317a7fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f317c0d28c0 0x7f317c0b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43014/0 (socket says 172.21.15.113:43014) 2024-04-24T23:59:06.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f317a7fc640 1 -- 172.21.15.113:0/2979465072 learned_addr learned my addr 172.21.15.113:0/2979465072 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f317affd640 1 -- 172.21.15.113:0/2979465072 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0b63b0 msgr2=0x7f317c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f317affd640 1 --2- 172.21.15.113:0/2979465072 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0b63b0 0x7f317c0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f317affd640 1 -- 172.21.15.113:0/2979465072 >> v1:172.21.15.113:6789/0 conn(0x7f317c0d28c0 legacy=0x7f317c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f317affd640 1 -- 172.21.15.113:0/2979465072 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c0d36a0 con 0x7f317c15e510 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.752+0000 7f3179ffb640 1 -- 172.21.15.113:0/2979465072 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549927910 0 0) 0x7f317c0d2e00 con 0x7f317c0d28c0 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.753+0000 7f317affd640 1 --2- 172.21.15.113:0/2979465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c15e510 0x7f317c0d2380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869e1385cdb14ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.753+0000 7f3179ffb640 1 -- 172.21.15.113:0/2979465072 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c037070 con 0x7f317c15e510 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.753+0000 7f3179ffb640 1 -- 172.21.15.113:0/2979465072 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c032440 con 0x7f317c15e510 2024-04-24T23:59:06.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.753+0000 7f3179ffb640 1 -- 172.21.15.113:0/2979465072 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c02ae60 con 0x7f317c15e510 2024-04-24T23:59:06.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 -- 172.21.15.113:0/2979465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c15e510 msgr2=0x7f317c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 --2- 172.21.15.113:0/2979465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c15e510 0x7f317c0d2380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02a110 comp rx=0 tx=0).stop 2024-04-24T23:59:06.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 -- 172.21.15.113:0/2979465072 shutdown_connections 2024-04-24T23:59:06.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 --2- 172.21.15.113:0/2979465072 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0b63b0 0x7f317c0b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 --2- 172.21.15.113:0/2979465072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c15e510 0x7f317c0d2380 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 -- 172.21.15.113:0/2979465072 >> 172.21.15.113:0/2979465072 conn(0x7f317c06b720 msgr2=0x7f317c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:06.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 -- 172.21.15.113:0/2979465072 shutdown_connections 2024-04-24T23:59:06.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.754+0000 7f318129a640 1 -- 172.21.15.113:0/2979465072 wait complete. 2024-04-24T23:59:06.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.755+0000 7f318129a640 1 Processor -- start 2024-04-24T23:59:06.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.755+0000 7f318129a640 1 -- start start 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f318129a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c0b63b0 0x7f317c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f318129a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0d28c0 0x7f317c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f318129a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317c15e510 0x7f317c152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f318129a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f317c0b9dd0 con 0x7f317c0b63b0 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f318129a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f317c0d3ea0 con 0x7f317c15e510 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f318129a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f317c0b7900 con 0x7f317c0d28c0 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c0b63b0 0x7f317c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c0b63b0 0x7f317c15d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42824/0 (socket says 172.21.15.113:42824) 2024-04-24T23:59:06.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 -- 172.21.15.113:0/3597006311 learned_addr learned my addr 172.21.15.113:0/3597006311 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317a7fc640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0d28c0 0x7f317c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317b7fe640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317c15e510 0x7f317c152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 -- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0d28c0 msgr2=0x7f317c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0d28c0 0x7f317c15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 -- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317c15e510 msgr2=0x7f317c152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317c15e510 0x7f317c152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 -- 172.21.15.113:0/3597006311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c1f3f50 con 0x7f317c0b63b0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.756+0000 7f317affd640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c0b63b0 0x7f317c15d4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.757+0000 7f31637fe640 1 -- 172.21.15.113:0/3597006311 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c041070 con 0x7f317c0b63b0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.757+0000 7f31637fe640 1 -- 172.21.15.113:0/3597006311 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c03c870 con 0x7f317c0b63b0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.757+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c1f4230 con 0x7f317c0b63b0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.757+0000 7f31637fe640 1 -- 172.21.15.113:0/3597006311 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c037040 con 0x7f317c0b63b0 2024-04-24T23:59:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.757+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c1f4ad0 con 0x7f317c0b63b0 2024-04-24T23:59:06.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.758+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c1f4620 con 0x7f317c0b63b0 2024-04-24T23:59:06.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.758+0000 7f31637fe640 1 -- 172.21.15.113:0/3597006311 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f316c044070 con 0x7f317c0b63b0 2024-04-24T23:59:06.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.759+0000 7f31637fe640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f316806be30 0x7f316806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:06.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.759+0000 7f317a7fc640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f316806be30 0x7f316806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:06.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.759+0000 7f31637fe640 1 -- 172.21.15.113:0/3597006311 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f316c0ab610 con 0x7f317c0b63b0 2024-04-24T23:59:06.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.759+0000 7f317a7fc640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f316806be30 0x7f316806e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3170007fd0 tx=0x7f3170009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:06.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:06.763+0000 7f31637fe640 1 -- 172.21.15.113:0/3597006311 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f316c07a310 con 0x7f317c0b63b0 2024-04-24T23:59:07.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.001+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f317c0b5b10 con 0x7f317c0b63b0 2024-04-24T23:59:07.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.002+0000 7f31637fe640 1 -- 172.21.15.113:0/3597006311 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f316c079c10 con 0x7f317c0b63b0 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f316806be30 msgr2=0x7f316806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f316806be30 0x7f316806e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3170007fd0 tx=0x7f3170009040 comp rx=0 tx=0).stop 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c0b63b0 msgr2=0x7f317c15d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c0b63b0 0x7f317c15d4a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 shutdown_connections 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f316806be30 0x7f316806e2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317c15e510 0x7f317c152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f317c0d28c0 0x7f317c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 --2- 172.21.15.113:0/3597006311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317c0b63b0 0x7f317c15d4a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 >> 172.21.15.113:0/3597006311 conn(0x7f317c06b720 msgr2=0x7f317c0b91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 shutdown_connections 2024-04-24T23:59:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.004+0000 7f318129a640 1 -- 172.21.15.113:0/3597006311 wait complete. 2024-04-24T23:59:07.015 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:59:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.111+0000 7fe825664640 1 Processor -- start 2024-04-24T23:59:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.111+0000 7fe825664640 1 -- start start 2024-04-24T23:59:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe825664640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe820070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe825664640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe820072ea0 0x7fe820071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe825664640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe820071780 0x7fe82015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe825664640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe8200b2d50 con 0x7fe820072ea0 2024-04-24T23:59:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe825664640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe820072a70 con 0x7fe820071780 2024-04-24T23:59:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe825664640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8200b3010 con 0x7fe82015a6b0 2024-04-24T23:59:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe820072ea0 0x7fe820071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe820072ea0 0x7fe820071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42838/0 (socket says 172.21.15.113:42838) 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 -- 172.21.15.113:0/2845299101 learned_addr learned my addr 172.21.15.113:0/2845299101 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81f7fe640 1 --2- 172.21.15.113:0/2845299101 >> v2:172.21.15.178:3300/0 conn(0x7fe820071780 0x7fe82015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81effd640 1 --2- 172.21.15.113:0/2845299101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe820070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 -- 172.21.15.113:0/2845299101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 msgr2=0x7fe820070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 --2- 172.21.15.113:0/2845299101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe820070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 -- 172.21.15.113:0/2845299101 >> v2:172.21.15.178:3300/0 conn(0x7fe820071780 msgr2=0x7fe82015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 --2- 172.21.15.113:0/2845299101 >> v2:172.21.15.178:3300/0 conn(0x7fe820071780 0x7fe82015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.112+0000 7fe81e7fc640 1 -- 172.21.15.113:0/2845299101 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8200b32f0 con 0x7fe820072ea0 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.113+0000 7fe81e7fc640 1 --2- 172.21.15.113:0/2845299101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe820071240 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe820072c80 tx=0x7fe81402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22875e8340b9797 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.113+0000 7fe81dffb640 1 -- 172.21.15.113:0/2845299101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814037070 con 0x7fe820072ea0 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.113+0000 7fe81dffb640 1 -- 172.21.15.113:0/2845299101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe814032440 con 0x7fe820072ea0 2024-04-24T23:59:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.113+0000 7fe81dffb640 1 -- 172.21.15.113:0/2845299101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814032810 con 0x7fe820072ea0 2024-04-24T23:59:07.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 -- 172.21.15.113:0/2845299101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 msgr2=0x7fe820071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 --2- 172.21.15.113:0/2845299101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe820071240 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe820072c80 tx=0x7fe81402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 -- 172.21.15.113:0/2845299101 shutdown_connections 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 --2- 172.21.15.113:0/2845299101 >> v2:172.21.15.178:3300/0 conn(0x7fe820071780 0x7fe82015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 --2- 172.21.15.113:0/2845299101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe820071240 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 --2- 172.21.15.113:0/2845299101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe820070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 -- 172.21.15.113:0/2845299101 >> 172.21.15.113:0/2845299101 conn(0x7fe8200b1a50 msgr2=0x7fe8200b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 -- 172.21.15.113:0/2845299101 shutdown_connections 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.114+0000 7fe825664640 1 -- 172.21.15.113:0/2845299101 wait complete. 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.115+0000 7fe825664640 1 Processor -- start 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.115+0000 7fe825664640 1 -- start start 2024-04-24T23:59:07.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe825664640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe820071780 0x7fe820148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe825664640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe8200a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe825664640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe8200cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe825664640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe82015aa90 con 0x7fe820072ea0 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe825664640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe820072720 con 0x7fe820071780 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe825664640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe82015ac10 con 0x7fe82015a6b0 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe8200a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe8200a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42846/0 (socket says 172.21.15.113:42846) 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 -- 172.21.15.113:0/3571493229 learned_addr learned my addr 172.21.15.113:0/3571493229 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81effd640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe820071780 0x7fe820148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 -- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 msgr2=0x7fe8200cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81f7fe640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe8200cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe8200cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 -- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe820071780 msgr2=0x7fe820148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe820071780 0x7fe820148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 -- 172.21.15.113:0/3571493229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe820158c30 con 0x7fe820072ea0 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81effd640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe820071780 0x7fe820148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81f7fe640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe8200cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe81e7fc640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe8200a9d00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe81402a200 tx=0x7fe8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.116+0000 7fe8077fe640 1 -- 172.21.15.113:0/3571493229 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814041020 con 0x7fe820072ea0 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.117+0000 7fe8077fe640 1 -- 172.21.15.113:0/3571493229 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8140306c0 con 0x7fe820072ea0 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.117+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe820158f10 con 0x7fe820072ea0 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.117+0000 7fe8077fe640 1 -- 172.21.15.113:0/3571493229 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814037040 con 0x7fe820072ea0 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.117+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8201594d0 con 0x7fe820072ea0 2024-04-24T23:59:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.118+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8200691a0 con 0x7fe820072ea0 2024-04-24T23:59:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.118+0000 7fe8077fe640 1 -- 172.21.15.113:0/3571493229 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe814044020 con 0x7fe820072ea0 2024-04-24T23:59:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.118+0000 7fe8077fe640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe80c06be30 0x7fe80c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.118+0000 7fe81effd640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe80c06be30 0x7fe80c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.119+0000 7fe8077fe640 1 -- 172.21.15.113:0/3571493229 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fe8140ab3c0 con 0x7fe820072ea0 2024-04-24T23:59:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.119+0000 7fe81effd640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe80c06be30 0x7fe80c06e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe810009f50 tx=0x7fe81000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.123+0000 7fe8077fe640 1 -- 172.21.15.113:0/3571493229 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe81404b030 con 0x7fe820072ea0 2024-04-24T23:59:07.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.300+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8200ce7f0 con 0x7fe820072ea0 2024-04-24T23:59:07.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.301+0000 7fe8077fe640 1 -- 172.21.15.113:0/3571493229 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+12939 (secure 0 0 0) 0x7fe81407a030 con 0x7fe820072ea0 2024-04-24T23:59:07.301 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:07.301 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":28,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:59:04.406649+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe80c06be30 msgr2=0x7fe80c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe80c06be30 0x7fe80c06e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe810009f50 tx=0x7fe81000b040 comp rx=0 tx=0).stop 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 msgr2=0x7fe8200a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe8200a9d00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe81402a200 tx=0x7fe8140046c0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 shutdown_connections 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe80c06be30 0x7fe80c06e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82015a6b0 0x7fe8200cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe820072ea0 0x7fe8200a9d00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 --2- 172.21.15.113:0/3571493229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe820071780 0x7fe820148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 >> 172.21.15.113:0/3571493229 conn(0x7fe8200b1a50 msgr2=0x7fe820152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:07.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 shutdown_connections 2024-04-24T23:59:07.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.303+0000 7fe825664640 1 -- 172.21.15.113:0/3571493229 wait complete. 2024-04-24T23:59:07.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:59:07.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:59:07.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.415+0000 7fe12dcd2640 1 Processor -- start 2024-04-24T23:59:07.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.415+0000 7fe12dcd2640 1 -- start start 2024-04-24T23:59:07.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.416+0000 7fe12dcd2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1280b5ca0 0x7fe1280b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.416+0000 7fe12dcd2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1280d2e00 con 0x7fe1280b73c0 2024-04-24T23:59:07.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.416+0000 7fe12dcd2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1280d30e0 con 0x7fe12815e510 2024-04-24T23:59:07.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.416+0000 7fe12dcd2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe1280b6640 con 0x7fe1280b5ca0 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.416+0000 7fe11effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1280b5ca0 0x7fe1280b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.416+0000 7fe127fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe1280b73c0 0x7fe1280b77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43026/0 (socket says 172.21.15.113:43026) 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.416+0000 7fe127fff640 1 -- 172.21.15.113:0/912234986 learned_addr learned my addr 172.21.15.113:0/912234986 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe11effd640 1 -- 172.21.15.113:0/912234986 >> v1:172.21.15.178:6789/0 conn(0x7fe12815e510 legacy=0x7fe1280d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe11effd640 1 -- 172.21.15.113:0/912234986 >> v1:172.21.15.113:6789/0 conn(0x7fe1280b73c0 legacy=0x7fe1280b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe11effd640 1 -- 172.21.15.113:0/912234986 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1280d33c0 con 0x7fe1280b5ca0 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe126ffd640 1 -- 172.21.15.113:0/912234986 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4062287336 0 0) 0x7fe1280d2e00 con 0x7fe1280b73c0 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe11effd640 1 --2- 172.21.15.113:0/912234986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1280b5ca0 0x7fe1280b6100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1100097a0 tx=0x7fe11002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fd7e1010c94df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe126ffd640 1 -- 172.21.15.113:0/912234986 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110037070 con 0x7fe1280b5ca0 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe126ffd640 1 -- 172.21.15.113:0/912234986 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11002ae50 con 0x7fe1280b5ca0 2024-04-24T23:59:07.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.417+0000 7fe126ffd640 1 -- 172.21.15.113:0/912234986 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110032660 con 0x7fe1280b5ca0 2024-04-24T23:59:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.418+0000 7fe12dcd2640 1 -- 172.21.15.113:0/912234986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1280b5ca0 msgr2=0x7fe1280b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.418+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/912234986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1280b5ca0 0x7fe1280b6100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1100097a0 tx=0x7fe11002a140 comp rx=0 tx=0).stop 2024-04-24T23:59:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.418+0000 7fe12dcd2640 1 -- 172.21.15.113:0/912234986 shutdown_connections 2024-04-24T23:59:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.418+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/912234986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1280b5ca0 0x7fe1280b6100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.418+0000 7fe12dcd2640 1 -- 172.21.15.113:0/912234986 >> 172.21.15.113:0/912234986 conn(0x7fe12806b720 msgr2=0x7fe1280d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.418+0000 7fe12dcd2640 1 -- 172.21.15.113:0/912234986 shutdown_connections 2024-04-24T23:59:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.418+0000 7fe12dcd2640 1 -- 172.21.15.113:0/912234986 wait complete. 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.419+0000 7fe12dcd2640 1 Processor -- start 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.419+0000 7fe12dcd2640 1 -- start start 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe12dcd2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1280b5ca0 0x7fe128156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe12dcd2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1280b73c0 0x7fe128158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe12dcd2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe12815e510 0x7fe1280ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe12dcd2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d3ea0 con 0x7fe12815e510 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe12dcd2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280d3980 con 0x7fe1280b5ca0 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe12dcd2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe1280d2a90 con 0x7fe1280b73c0 2024-04-24T23:59:07.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe12815e510 0x7fe1280ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe12815e510 0x7fe1280ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42864/0 (socket says 172.21.15.113:42864) 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 -- 172.21.15.113:0/6229815 learned_addr learned my addr 172.21.15.113:0/6229815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 -- 172.21.15.113:0/6229815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1280b73c0 msgr2=0x7fe128158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1280b73c0 0x7fe128158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 -- 172.21.15.113:0/6229815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1280b5ca0 msgr2=0x7fe128156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1280b5ca0 0x7fe128156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.420+0000 7fe127fff640 1 -- 172.21.15.113:0/6229815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1281561b0 con 0x7fe12815e510 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.421+0000 7fe127fff640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe12815e510 0x7fe1280ca3a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe12000c7e0 tx=0x7fe12000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.421+0000 7fe124ff9640 1 -- 172.21.15.113:0/6229815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120018070 con 0x7fe12815e510 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.421+0000 7fe124ff9640 1 -- 172.21.15.113:0/6229815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12000a640 con 0x7fe12815e510 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.421+0000 7fe124ff9640 1 -- 172.21.15.113:0/6229815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12000aa30 con 0x7fe12815e510 2024-04-24T23:59:07.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.421+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe128152960 con 0x7fe12815e510 2024-04-24T23:59:07.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.421+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe128157130 con 0x7fe12815e510 2024-04-24T23:59:07.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.422+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1280691a0 con 0x7fe12815e510 2024-04-24T23:59:07.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.422+0000 7fe124ff9640 1 -- 172.21.15.113:0/6229815 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe12001b070 con 0x7fe12815e510 2024-04-24T23:59:07.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.423+0000 7fe124ff9640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe10806be30 0x7fe10806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.423+0000 7fe1277fe640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe10806be30 0x7fe10806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.423+0000 7fe124ff9640 1 -- 172.21.15.113:0/6229815 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fe12008a1b0 con 0x7fe12815e510 2024-04-24T23:59:07.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.423+0000 7fe1277fe640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe10806be30 0x7fe10806e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe118001560 tx=0x7fe118007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.427+0000 7fe124ff9640 1 -- 172.21.15.113:0/6229815 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe120058eb0 con 0x7fe12815e510 2024-04-24T23:59:07.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.600+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1280cfbc0 con 0x7fe12815e510 2024-04-24T23:59:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.600+0000 7fe124ff9640 1 -- 172.21.15.113:0/6229815 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fe1200587b0 con 0x7fe12815e510 2024-04-24T23:59:07.601 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe10806be30 msgr2=0x7fe10806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe10806be30 0x7fe10806e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe118001560 tx=0x7fe118007990 comp rx=0 tx=0).stop 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe12815e510 msgr2=0x7fe1280ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe12815e510 0x7fe1280ca3a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe12000c7e0 tx=0x7fe12000cca0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 shutdown_connections 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe10806be30 0x7fe10806e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe12815e510 0x7fe1280ca3a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1280b73c0 0x7fe128158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 --2- 172.21.15.113:0/6229815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1280b5ca0 0x7fe128156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 >> 172.21.15.113:0/6229815 conn(0x7fe12806b720 msgr2=0x7fe12815cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 shutdown_connections 2024-04-24T23:59:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.602+0000 7fe12dcd2640 1 -- 172.21.15.113:0/6229815 wait complete. 2024-04-24T23:59:07.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:59:07.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.710+0000 7f04c7529640 1 Processor -- start 2024-04-24T23:59:07.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.710+0000 7f04c7529640 1 -- start start 2024-04-24T23:59:07.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.711+0000 7f04c7529640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f04c0144d60 0x7f04c00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.711+0000 7f04c7529640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c0071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.711+0000 7f04c7529640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c00cfad0 con 0x7f04c01453c0 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.711+0000 7f04c7529640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f04c0145c40 con 0x7f04c0144d60 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.711+0000 7f04c7529640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04c00cf230 con 0x7f04c006e6e0 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c52c5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f04c0144d60 0x7f04c00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c5ac6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c4ac4640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f04c01453c0 0x7f04c006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33170/0 (socket says 172.21.15.113:33170) 2024-04-24T23:59:07.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c4ac4640 1 -- 172.21.15.113:0/3712674591 learned_addr learned my addr 172.21.15.113:0/3712674591 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c52c5640 1 -- 172.21.15.113:0/3712674591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 msgr2=0x7f04c0071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c52c5640 1 --2- 172.21.15.113:0/3712674591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c0071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c52c5640 1 -- 172.21.15.113:0/3712674591 >> v1:172.21.15.178:6789/0 conn(0x7f04c01453c0 legacy=0x7f04c006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.712+0000 7f04c52c5640 1 -- 172.21.15.113:0/3712674591 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c00cf510 con 0x7f04c0144d60 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.713+0000 7f04c52c5640 1 --2- 172.21.15.113:0/3712674591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 0x7f04c00b5390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0097a0 tx=0x7f04bc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7711da7b0df7daef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.713+0000 7f04b7fff640 1 -- 172.21.15.113:0/3712674591 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc037070 con 0x7f04c0144d60 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.713+0000 7f04b7fff640 1 -- 172.21.15.113:0/3712674591 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04bc02ae50 con 0x7f04c0144d60 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.713+0000 7f04b7fff640 1 -- 172.21.15.113:0/3712674591 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc032660 con 0x7f04c0144d60 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 -- 172.21.15.113:0/3712674591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 msgr2=0x7f04c00b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 --2- 172.21.15.113:0/3712674591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 0x7f04c00b5390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0097a0 tx=0x7f04bc02a100 comp rx=0 tx=0).stop 2024-04-24T23:59:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 -- 172.21.15.113:0/3712674591 shutdown_connections 2024-04-24T23:59:07.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 --2- 172.21.15.113:0/3712674591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c0071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 --2- 172.21.15.113:0/3712674591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 0x7f04c00b5390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 -- 172.21.15.113:0/3712674591 >> 172.21.15.113:0/3712674591 conn(0x7f04c00b1a50 msgr2=0x7f04c00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:07.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 -- 172.21.15.113:0/3712674591 shutdown_connections 2024-04-24T23:59:07.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.714+0000 7f04c7529640 1 -- 172.21.15.113:0/3712674591 wait complete. 2024-04-24T23:59:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.715+0000 7f04c7529640 1 Processor -- start 2024-04-24T23:59:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.715+0000 7f04c7529640 1 -- start start 2024-04-24T23:59:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c7529640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c013c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c7529640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 0x7f04c00a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c7529640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c01453c0 0x7f04c00a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c7529640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0145140 con 0x7f04c01453c0 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c7529640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0070e10 con 0x7f04c0144d60 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c7529640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04c0071870 con 0x7f04c006e6e0 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c01453c0 0x7f04c00a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c01453c0 0x7f04c00a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53352/0 (socket says 172.21.15.113:53352) 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 -- 172.21.15.113:0/3562940698 learned_addr learned my addr 172.21.15.113:0/3562940698 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c52c5640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c013c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c4ac4640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 0x7f04c00a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 -- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 msgr2=0x7f04c013c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c013c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 -- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 msgr2=0x7f04c00a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 0x7f04c00a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 -- 172.21.15.113:0/3562940698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c014ebc0 con 0x7f04c01453c0 2024-04-24T23:59:07.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c5ac6640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c01453c0 0x7f04c00a99c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009d20 tx=0x7f04b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04b5ffb640 1 -- 172.21.15.113:0/3562940698 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8018020 con 0x7f04c01453c0 2024-04-24T23:59:07.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04b5ffb640 1 -- 172.21.15.113:0/3562940698 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b8009b10 con 0x7f04c01453c0 2024-04-24T23:59:07.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c014eea0 con 0x7f04c01453c0 2024-04-24T23:59:07.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.716+0000 7f04b5ffb640 1 -- 172.21.15.113:0/3562940698 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b80146c0 con 0x7f04c01453c0 2024-04-24T23:59:07.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.717+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c014f460 con 0x7f04c01453c0 2024-04-24T23:59:07.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.718+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c00691a0 con 0x7f04c01453c0 2024-04-24T23:59:07.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.718+0000 7f04b5ffb640 1 -- 172.21.15.113:0/3562940698 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04b801b020 con 0x7f04c01453c0 2024-04-24T23:59:07.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.718+0000 7f04b5ffb640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a406be30 0x7f04a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:07.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.719+0000 7f04b5ffb640 1 -- 172.21.15.113:0/3562940698 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f04b808b570 con 0x7f04c01453c0 2024-04-24T23:59:07.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.719+0000 7f04c52c5640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a406be30 0x7f04a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:07.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.719+0000 7f04c52c5640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a406be30 0x7f04a406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04bc034000 tx=0x7f04bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:07.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.723+0000 7f04b5ffb640 1 -- 172.21.15.113:0/3562940698 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f04b8022030 con 0x7f04c01453c0 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.896+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f04c00b27e0 con 0x7f04c01453c0 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.897+0000 7f04b5ffb640 1 -- 172.21.15.113:0/3562940698 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f04b8010040 con 0x7f04c01453c0 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-24T23:59:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-24T23:59:07.898 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-24T23:59:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a406be30 msgr2=0x7f04a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a406be30 0x7f04a406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04bc034000 tx=0x7f04bc02e000 comp rx=0 tx=0).stop 2024-04-24T23:59:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c01453c0 msgr2=0x7f04c00a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c01453c0 0x7f04c00a99c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009d20 tx=0x7f04b800c490 comp rx=0 tx=0).stop 2024-04-24T23:59:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 shutdown_connections 2024-04-24T23:59:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a406be30 0x7f04a406e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c0144d60 0x7f04c00a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c01453c0 0x7f04c00a99c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 --2- 172.21.15.113:0/3562940698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c006e6e0 0x7f04c013c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 >> 172.21.15.113:0/3562940698 conn(0x7f04c00b1a50 msgr2=0x7f04c006d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 shutdown_connections 2024-04-24T23:59:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:07.899+0000 7f04c7529640 1 -- 172.21.15.113:0/3562940698 wait complete. 2024-04-24T23:59:07.909 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:59:07.910 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:59:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.011+0000 7f93e109a640 1 Processor -- start 2024-04-24T23:59:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.011+0000 7f93e109a640 1 -- start start 2024-04-24T23:59:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93e109a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 0x7f93dc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93e109a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f93dc06e610 0x7f93dc071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93e109a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93dc0cf3a0 con 0x7f93dc145890 2024-04-24T23:59:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93e109a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f93dc145c70 con 0x7f93dc06e610 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93e109a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93dc06ea60 con 0x7f93dc0ceef0 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93dad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f93dc145890 0x7f93dc0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40254/0 (socket says 172.21.15.113:40254) 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93dad9d640 1 -- 172.21.15.113:0/2295705387 learned_addr learned my addr 172.21.15.113:0/2295705387 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081422408 0 0) 0x7f93dc0cf3a0 con 0x7f93dc145890 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93c4003520 con 0x7f93dc145890 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93da59c640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 0x7f93dc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93db59e640 1 --2- 172.21.15.113:0/2295705387 >> v2:172.21.15.178:3300/0 conn(0x7f93dc06e610 0x7f93dc071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 291193086 0 0) 0x7f93c4003520 con 0x7f93dc145890 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 msgr2=0x7f93dc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 0x7f93dc06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 >> v2:172.21.15.178:3300/0 conn(0x7f93dc06e610 msgr2=0x7f93dc071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 --2- 172.21.15.113:0/2295705387 >> v2:172.21.15.178:3300/0 conn(0x7f93dc06e610 0x7f93dc071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:08.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc0cf680 con 0x7f93dc145890 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f93cc0019f0 con 0x7f93dc145890 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 >> v1:172.21.15.113:6789/0 conn(0x7f93dc145890 legacy=0x7f93dc0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c40039b0 0x7f93c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 0x7f93c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c4004d40 0x7f93c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc06ea60 con 0x7f93c4004d40 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93c400b9c0 con 0x7f93c40039b0 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.012+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93c400bb40 con 0x7f93dc0ceef0 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93db59e640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c4004d40 0x7f93c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93dad9d640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c40039b0 0x7f93c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93da59c640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 0x7f93c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93db59e640 1 -- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 msgr2=0x7f93c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93db59e640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc0ceef0 0x7f93c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:08.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93db59e640 1 -- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c40039b0 msgr2=0x7f93c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93db59e640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c40039b0 0x7f93c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93db59e640 1 -- 172.21.15.113:0/2295705387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c4003520 con 0x7f93c4004d40 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.013+0000 7f93db59e640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c4004d40 0x7f93c40051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009af0 tx=0x7f93c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6861877094816b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4036070 con 0x7f93c4004d40 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d4031400 con 0x7f93c4004d40 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93d9d9b640 1 -- 172.21.15.113:0/2295705387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4009690 con 0x7f93c4004d40 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93e109a640 1 -- 172.21.15.113:0/2295705387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c4004d40 msgr2=0x7f93c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93e109a640 1 --2- 172.21.15.113:0/2295705387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c4004d40 0x7f93c40051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009af0 tx=0x7f93c4005830 comp rx=0 tx=0).stop 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93db59e640 1 -- 172.21.15.113:0/2295705387 reap_dead start 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93e109a640 1 -- 172.21.15.113:0/2295705387 shutdown_connections 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93e109a640 1 -- 172.21.15.113:0/2295705387 >> 172.21.15.113:0/2295705387 conn(0x7f93dc0b1b30 msgr2=0x7f93dc0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93e109a640 1 -- 172.21.15.113:0/2295705387 shutdown_connections 2024-04-24T23:59:08.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.014+0000 7f93e109a640 1 -- 172.21.15.113:0/2295705387 wait complete. 2024-04-24T23:59:08.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 Processor -- start 2024-04-24T23:59:08.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 -- start start 2024-04-24T23:59:08.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc14d2a0 0x7f93dc140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93dc1512b0 0x7f93dc14d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc13d0d0 0x7f93dc15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc0b53b0 con 0x7f93dc13d0d0 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc070f10 con 0x7f93dc1512b0 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93e109a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93dc06e0d0 con 0x7f93dc14d2a0 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.016+0000 7f93db59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc13d0d0 0x7f93dc15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc13d0d0 0x7f93dc15cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53368/0 (socket says 172.21.15.113:53368) 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 -- 172.21.15.113:0/2261204136 learned_addr learned my addr 172.21.15.113:0/2261204136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93dad9d640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc14d2a0 0x7f93dc140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93da59c640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93dc1512b0 0x7f93dc14d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 -- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc14d2a0 msgr2=0x7f93dc140310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc14d2a0 0x7f93dc140310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 -- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93dc1512b0 msgr2=0x7f93dc14d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93dc1512b0 0x7f93dc14d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:08.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 -- 172.21.15.113:0/2261204136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc14dbc0 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93db59e640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc13d0d0 0x7f93dc15cef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009a10 tx=0x7f93d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93c37fe640 1 -- 172.21.15.113:0/2261204136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4036070 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93c37fe640 1 -- 172.21.15.113:0/2261204136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d4041070 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93dc13a450 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93c37fe640 1 -- 172.21.15.113:0/2261204136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4009690 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.017+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93dc13db80 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.018+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93dc0691a0 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.019+0000 7f93c37fe640 1 -- 172.21.15.113:0/2261204136 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93d403b640 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.019+0000 7f93c37fe640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93c806be30 0x7f93c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.019+0000 7f93c37fe640 1 -- 172.21.15.113:0/2261204136 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f93d40aaa80 con 0x7f93dc13d0d0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.019+0000 7f93dad9d640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93c806be30 0x7f93c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.019+0000 7f93dad9d640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93c806be30 0x7f93c806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000f80 tx=0x7f93cc006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:08.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.023+0000 7f93c37fe640 1 -- 172.21.15.113:0/2261204136 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93d4079840 con 0x7f93dc13d0d0 2024-04-24T23:59:08.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:08.197+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f93dc143960 con 0x7f93dc13d0d0 2024-04-24T23:59:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.005+0000 7f93c37fe640 1 -- 172.21.15.113:0/2261204136 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f93d4079140 con 0x7f93dc13d0d0 2024-04-24T23:59:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-24T23:59:09.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93c806be30 msgr2=0x7f93c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:09.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93c806be30 0x7f93c806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000f80 tx=0x7f93cc006040 comp rx=0 tx=0).stop 2024-04-24T23:59:09.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc13d0d0 msgr2=0x7f93dc15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:09.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc13d0d0 0x7f93dc15cef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009a10 tx=0x7f93d4004730 comp rx=0 tx=0).stop 2024-04-24T23:59:09.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 shutdown_connections 2024-04-24T23:59:09.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93c806be30 0x7f93c806e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93dc1512b0 0x7f93dc14d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc13d0d0 0x7f93dc15cef0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 --2- 172.21.15.113:0/2261204136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93dc14d2a0 0x7f93dc140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 >> 172.21.15.113:0/2261204136 conn(0x7f93dc0b1b30 msgr2=0x7f93dc15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 shutdown_connections 2024-04-24T23:59:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.008+0000 7f93e109a640 1 -- 172.21.15.113:0/2261204136 wait complete. 2024-04-24T23:59:09.023 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:59:09.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.129+0000 7fedb0e93640 1 Processor -- start 2024-04-24T23:59:09.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.129+0000 7fedb0e93640 1 -- start start 2024-04-24T23:59:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7fedb0e93640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 0x7fedac160760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7fedb0e93640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedac06b5f0 con 0x7fedac062290 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7fedb0e93640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedac06b890 con 0x7fedac15d150 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7fedb0e93640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fedac06bb70 con 0x7fedac06c650 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fedac062290 0x7fedac160050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40268/0 (socket says 172.21.15.113:40268) 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda3fff640 1 -- 172.21.15.113:0/3806595528 learned_addr learned my addr 172.21.15.113:0/3806595528 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310305505 0 0) 0x7fedac06b5f0 con 0x7fedac062290 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed98003520 con 0x7fedac062290 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7fedaad9d640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 0x7fedac160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2638975015 0 0) 0x7fed98003520 con 0x7fedac062290 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 msgr2=0x7fedac160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 0x7fedac160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 >> v1:172.21.15.178:6789/0 conn(0x7fedac15d150 legacy=0x7fedac061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac06be50 con 0x7fedac062290 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fed940016c0 con 0x7fedac062290 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.130+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 >> v1:172.21.15.113:6789/0 conn(0x7fedac062290 legacy=0x7fedac160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7feda9d9b640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed980039e0 0x7fed98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7feda9d9b640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 0x7fed98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7feda9d9b640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed98004d40 0x7fed980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fedac06bb70 con 0x7fed980039e0 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed9800ba70 con 0x7fed98004d40 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fed9800bc70 con 0x7fedac06c650 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7fedaa59c640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed980039e0 0x7fed98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7fedaad9d640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed98004d40 0x7fed980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7feda3fff640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 0x7fed98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7fedaa59c640 1 -- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 msgr2=0x7fed98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7fedaa59c640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac06c650 0x7fed98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7fedaa59c640 1 -- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed98004d40 msgr2=0x7fed980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7fedaa59c640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed98004d40 0x7fed980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.131+0000 7fedaa59c640 1 -- 172.21.15.113:0/3806595528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed98003520 con 0x7fed980039e0 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedaa59c640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed980039e0 0x7fed98003e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fed9c001720 tx=0x7fed98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc88b7b9758bd6b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c001370 con 0x7fed980039e0 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c0052d0 con 0x7fed980039e0 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7feda9d9b640 1 -- 172.21.15.113:0/3806595528 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c02abc0 con 0x7fed980039e0 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedb0e93640 1 -- 172.21.15.113:0/3806595528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed980039e0 msgr2=0x7fed98003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedb0e93640 1 --2- 172.21.15.113:0/3806595528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed980039e0 0x7fed98003e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fed9c001720 tx=0x7fed98005830 comp rx=0 tx=0).stop 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedaad9d640 1 -- 172.21.15.113:0/3806595528 reap_dead start 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedb0e93640 1 -- 172.21.15.113:0/3806595528 shutdown_connections 2024-04-24T23:59:09.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedb0e93640 1 -- 172.21.15.113:0/3806595528 >> 172.21.15.113:0/3806595528 conn(0x7fedac0d0e30 msgr2=0x7fedac0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:09.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedb0e93640 1 -- 172.21.15.113:0/3806595528 shutdown_connections 2024-04-24T23:59:09.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.132+0000 7fedb0e93640 1 -- 172.21.15.113:0/3806595528 wait complete. 2024-04-24T23:59:09.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 Processor -- start 2024-04-24T23:59:09.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 -- start start 2024-04-24T23:59:09.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed980039e0 0x7fedac14d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedac0af1d0 0x7fedac152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac1526c0 0x7fedac14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fedac15d6b0 con 0x7fedac0af1d0 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fedac15d530 con 0x7fed980039e0 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7fedb0e93640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fedac160760 con 0x7fedac1526c0 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7feda3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedac0af1d0 0x7fedac152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7feda3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedac0af1d0 0x7fedac152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53390/0 (socket says 172.21.15.113:53390) 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.134+0000 7feda3fff640 1 -- 172.21.15.113:0/2959643366 learned_addr learned my addr 172.21.15.113:0/2959643366 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7fedaa59c640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed980039e0 0x7fedac14d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7fedaad9d640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac1526c0 0x7fedac14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda3fff640 1 -- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac1526c0 msgr2=0x7fedac14fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda3fff640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac1526c0 0x7fedac14fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda3fff640 1 -- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed980039e0 msgr2=0x7fedac14d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda3fff640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed980039e0 0x7fedac14d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda3fff640 1 -- 172.21.15.113:0/2959643366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac1549b0 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7fedaad9d640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac1526c0 0x7fedac14fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda3fff640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedac0af1d0 0x7fedac152180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed94004fa0 tx=0x7fed940074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda2ffd640 1 -- 172.21.15.113:0/2959643366 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9401a070 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda2ffd640 1 -- 172.21.15.113:0/2959643366 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94007740 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedac15f290 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7feda2ffd640 1 -- 172.21.15.113:0/2959643366 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9400fbb0 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.135+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedac15f850 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.136+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedac0691a0 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.137+0000 7feda2ffd640 1 -- 172.21.15.113:0/2959643366 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed9401d070 con 0x7fedac0af1d0 2024-04-24T23:59:09.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.137+0000 7feda2ffd640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fed8c06be60 0x7fed8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:09.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.137+0000 7feda2ffd640 1 -- 172.21.15.113:0/2959643366 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7fed9408a4f0 con 0x7fedac0af1d0 2024-04-24T23:59:09.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.137+0000 7fedaa59c640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fed8c06be60 0x7fed8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:09.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.137+0000 7fedaa59c640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fed8c06be60 0x7fed8c06e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed9c032000 tx=0x7fed9c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:09.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.141+0000 7feda2ffd640 1 -- 172.21.15.113:0/2959643366 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed94058a10 con 0x7fedac0af1d0 2024-04-24T23:59:09.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:09.316+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fedac151f60 con 0x7fedac0af1d0 2024-04-24T23:59:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.015+0000 7feda2ffd640 1 -- 172.21.15.113:0/2959643366 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7fed9405cd10 con 0x7fedac0af1d0 2024-04-24T23:59:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-24T23:59:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fed8c06be60 msgr2=0x7fed8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fed8c06be60 0x7fed8c06e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed9c032000 tx=0x7fed9c02e000 comp rx=0 tx=0).stop 2024-04-24T23:59:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedac0af1d0 msgr2=0x7fedac152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedac0af1d0 0x7fedac152180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed94004fa0 tx=0x7fed940074f0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 shutdown_connections 2024-04-24T23:59:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fed8c06be60 0x7fed8c06e320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedac1526c0 0x7fedac14fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedac0af1d0 0x7fedac152180 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 --2- 172.21.15.113:0/2959643366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed980039e0 0x7fedac14d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.017+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 >> 172.21.15.113:0/2959643366 conn(0x7fedac0d0e30 msgr2=0x7fedac0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.018+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 shutdown_connections 2024-04-24T23:59:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.018+0000 7fedb0e93640 1 -- 172.21.15.113:0/2959643366 wait complete. 2024-04-24T23:59:10.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:59:10.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.142+0000 7f760b55a640 1 Processor -- start 2024-04-24T23:59:10.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.143+0000 7f760b55a640 1 -- start start 2024-04-24T23:59:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.143+0000 7f760b55a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f760413f050 0x7f76040cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.143+0000 7f760b55a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f760415d6b0 con 0x7f76040d04f0 2024-04-24T23:59:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.143+0000 7f760b55a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f760415c8f0 con 0x7f76041606a0 2024-04-24T23:59:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.143+0000 7f760b55a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f76040d09a0 con 0x7f760413f050 2024-04-24T23:59:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f7608af5640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f76040d04f0 0x7f760415ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40276/0 (socket says 172.21.15.113:40276) 2024-04-24T23:59:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f7608af5640 1 -- 172.21.15.113:0/2937287158 learned_addr learned my addr 172.21.15.113:0/2937287158 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386566424 0 0) 0x7f760415d6b0 con 0x7f76040d04f0 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75f0003520 con 0x7f76040d04f0 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f76092f6640 1 --2- 172.21.15.113:0/2937287158 >> v2:172.21.15.178:3300/0 conn(0x7f760413f050 0x7f76040cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3182132920 0 0) 0x7f75f0003520 con 0x7f76040d04f0 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 >> v2:172.21.15.178:3300/0 conn(0x7f760413f050 msgr2=0x7f76040cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 --2- 172.21.15.113:0/2937287158 >> v2:172.21.15.178:3300/0 conn(0x7f760413f050 0x7f76040cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 >> v1:172.21.15.178:6789/0 conn(0x7f76041606a0 legacy=0x7f76040cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760415cbd0 con 0x7f76040d04f0 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f75f8001670 con 0x7f76040d04f0 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 >> v1:172.21.15.113:6789/0 conn(0x7f76040d04f0 legacy=0x7f760415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75f00039b0 0x7f75f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f0004800 0x7f75f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f00051c0 0x7f75f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f76040d09a0 con 0x7f75f0004800 2024-04-24T23:59:10.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75f000c160 con 0x7f75f00051c0 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75f000c360 con 0x7f75f00039b0 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f7608af5640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f0004800 0x7f75f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f76092f6640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75f00039b0 0x7f75f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.144+0000 7f7609af7640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f00051c0 0x7f75f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7608af5640 1 -- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75f00039b0 msgr2=0x7f75f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7608af5640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75f00039b0 0x7f75f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7608af5640 1 -- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f00051c0 msgr2=0x7f75f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7608af5640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f00051c0 0x7f75f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7608af5640 1 -- 172.21.15.113:0/2937287158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f0003520 con 0x7f75f0004800 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7609af7640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f00051c0 0x7f75f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7609af7640 1 -- 172.21.15.113:0/2937287158 reap_dead start 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f76092f6640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75f00039b0 0x7f75f0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f7608af5640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f0004800 0x7f75f0004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f75f800c240 tx=0x7f75f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e71bb6a9fc85f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8007d10 con 0x7f75f0004800 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f8005ad0 con 0x7f75f0004800 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.145+0000 7f75f7fff640 1 -- 172.21.15.113:0/2937287158 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8007070 con 0x7f75f0004800 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.146+0000 7f760b55a640 1 -- 172.21.15.113:0/2937287158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f0004800 msgr2=0x7f75f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.146+0000 7f760b55a640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f0004800 0x7f75f0004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f75f800c240 tx=0x7f75f0005830 comp rx=0 tx=0).stop 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.146+0000 7f760b55a640 1 -- 172.21.15.113:0/2937287158 shutdown_connections 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.146+0000 7f760b55a640 1 --2- 172.21.15.113:0/2937287158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f0004800 0x7f75f0004c80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.146+0000 7f760b55a640 1 -- 172.21.15.113:0/2937287158 >> 172.21.15.113:0/2937287158 conn(0x7f760406b0f0 msgr2=0x7f760406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.146+0000 7f760b55a640 1 -- 172.21.15.113:0/2937287158 shutdown_connections 2024-04-24T23:59:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.146+0000 7f760b55a640 1 -- 172.21.15.113:0/2937287158 wait complete. 2024-04-24T23:59:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.147+0000 7f760b55a640 1 Processor -- start 2024-04-24T23:59:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.147+0000 7f760b55a640 1 -- start start 2024-04-24T23:59:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f760b55a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f0004800 0x7f76041537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f760b55a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76040af010 0x7f760413da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f760b55a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76041f4300 0x7f760415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f760b55a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f760415d190 con 0x7f76041f4300 2024-04-24T23:59:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f760b55a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f760415c6a0 con 0x7f75f0004800 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f760b55a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f760413f430 con 0x7f76040af010 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76041f4300 0x7f760415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76041f4300 0x7f760415b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53402/0 (socket says 172.21.15.113:53402) 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 -- 172.21.15.113:0/666231713 learned_addr learned my addr 172.21.15.113:0/666231713 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f76092f6640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f0004800 0x7f76041537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7608af5640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76040af010 0x7f760413da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 -- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76040af010 msgr2=0x7f760413da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76040af010 0x7f760413da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 -- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f0004800 msgr2=0x7f76041537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f0004800 0x7f76041537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 -- 172.21.15.113:0/666231713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7604071760 con 0x7f76041f4300 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f76092f6640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f0004800 0x7f76041537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7608af5640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76040af010 0x7f760413da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.148+0000 7f7609af7640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76041f4300 0x7f760415b1d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76040b5680 tx=0x7f75fc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.149+0000 7f75f5ffb640 1 -- 172.21.15.113:0/666231713 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc003970 con 0x7f76041f4300 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.149+0000 7f75f5ffb640 1 -- 172.21.15.113:0/666231713 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75fc018070 con 0x7f76041f4300 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.149+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7604071a40 con 0x7f76041f4300 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.149+0000 7f75f5ffb640 1 -- 172.21.15.113:0/666231713 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc003d50 con 0x7f76041f4300 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.149+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760413dfc0 con 0x7f76041f4300 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.150+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76040691a0 con 0x7f76041f4300 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.150+0000 7f75f5ffb640 1 -- 172.21.15.113:0/666231713 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75fc0050d0 con 0x7f76041f4300 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.150+0000 7f75f5ffb640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75e806be30 0x7f75e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.151+0000 7f76092f6640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75e806be30 0x7f75e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:10.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.151+0000 7f75f5ffb640 1 -- 172.21.15.113:0/666231713 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f75fc08aa50 con 0x7f76041f4300 2024-04-24T23:59:10.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.151+0000 7f76092f6640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75e806be30 0x7f75e806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7604050ce0 tx=0x7f7600008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:10.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.155+0000 7f75f5ffb640 1 -- 172.21.15.113:0/666231713 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75fc0595b0 con 0x7f76041f4300 2024-04-24T23:59:10.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:10.363+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f76040d0e50 con 0x7f76041f4300 2024-04-24T23:59:11.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.016+0000 7f75f5ffb640 1 -- 172.21.15.113:0/666231713 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f75fc01be40 con 0x7f76041f4300 2024-04-24T23:59:11.017 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 4 and data pool 5 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.019+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75e806be30 msgr2=0x7f75e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.019+0000 7f760b55a640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75e806be30 0x7f75e806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7604050ce0 tx=0x7f7600008040 comp rx=0 tx=0).stop 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.019+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76041f4300 msgr2=0x7f760415b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.019+0000 7f760b55a640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76041f4300 0x7f760415b1d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76040b5680 tx=0x7f75fc00eb90 comp rx=0 tx=0).stop 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.019+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 shutdown_connections 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.020+0000 7f760b55a640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75e806be30 0x7f75e806e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.020+0000 7f760b55a640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76041f4300 0x7f760415b1d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.020+0000 7f760b55a640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76040af010 0x7f760413da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.020+0000 7f760b55a640 1 --2- 172.21.15.113:0/666231713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75f0004800 0x7f76041537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.020+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 >> 172.21.15.113:0/666231713 conn(0x7f760406b0f0 msgr2=0x7f76040b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:11.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.020+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 shutdown_connections 2024-04-24T23:59:11.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.020+0000 7f760b55a640 1 -- 172.21.15.113:0/666231713 wait complete. 2024-04-24T23:59:11.044 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:59:11.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.143+0000 7f10df584640 1 Processor -- start 2024-04-24T23:59:11.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.144+0000 7f10df584640 1 -- start start 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.144+0000 7f10df584640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10d80d2290 0x7f10d80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.144+0000 7f10df584640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d80d2800 con 0x7f10d815e320 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.144+0000 7f10df584640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d80d2ae0 con 0x7f10d80d0b00 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.144+0000 7f10df584640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f10d80d3410 con 0x7f10d80d2290 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dcb1f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10d80d2290 0x7f10d80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dd320640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f10d815e320 0x7f10d80d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40284/0 (socket says 172.21.15.113:40284) 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dd320640 1 -- 172.21.15.113:0/2978519640 learned_addr learned my addr 172.21.15.113:0/2978519640 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dcb1f640 1 --2- 172.21.15.113:0/2978519640 >> v2:172.21.15.113:3300/0 conn(0x7f10d80d2290 0x7f10d80b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53408/0 (socket says 172.21.15.113:53408) 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dcb1f640 1 -- 172.21.15.113:0/2978519640 >> v1:172.21.15.178:6789/0 conn(0x7f10d80d0b00 legacy=0x7f10d80d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:11.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dcb1f640 1 -- 172.21.15.113:0/2978519640 >> v1:172.21.15.113:6789/0 conn(0x7f10d815e320 legacy=0x7f10d80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dcb1f640 1 -- 172.21.15.113:0/2978519640 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80d2dc0 con 0x7f10d80d2290 2024-04-24T23:59:11.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10cffff640 1 -- 172.21.15.113:0/2978519640 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1916430619 0 0) 0x7f10d80d2800 con 0x7f10d815e320 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.145+0000 7f10dcb1f640 1 --2- 172.21.15.113:0/2978519640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d80d2290 0x7f10d80b9aa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f10c80097a0 tx=0x7f10c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c689c37ae2dd8d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10cffff640 1 -- 172.21.15.113:0/2978519640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8037070 con 0x7f10d80d2290 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10cffff640 1 -- 172.21.15.113:0/2978519640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8032440 con 0x7f10d80d2290 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10cffff640 1 -- 172.21.15.113:0/2978519640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8032810 con 0x7f10d80d2290 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10df584640 1 -- 172.21.15.113:0/2978519640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d80d2290 msgr2=0x7f10d80b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10df584640 1 --2- 172.21.15.113:0/2978519640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d80d2290 0x7f10d80b9aa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f10c80097a0 tx=0x7f10c802a270 comp rx=0 tx=0).stop 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10df584640 1 -- 172.21.15.113:0/2978519640 shutdown_connections 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10df584640 1 --2- 172.21.15.113:0/2978519640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d80d2290 0x7f10d80b9aa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10df584640 1 -- 172.21.15.113:0/2978519640 >> 172.21.15.113:0/2978519640 conn(0x7f10d80b5c20 msgr2=0x7f10d80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:11.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10df584640 1 -- 172.21.15.113:0/2978519640 shutdown_connections 2024-04-24T23:59:11.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.146+0000 7f10df584640 1 -- 172.21.15.113:0/2978519640 wait complete. 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.147+0000 7f10df584640 1 Processor -- start 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.147+0000 7f10df584640 1 -- start start 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10df584640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d80d0b00 0x7f10d8153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10df584640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d80d2290 0x7f10d8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10df584640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d815e320 0x7f10d80b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10df584640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f10d815e700 con 0x7f10d815e320 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10df584640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80b9fe0 con 0x7f10d80d0b00 2024-04-24T23:59:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10df584640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10d815e880 con 0x7f10d80d2290 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10ddb21640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d815e320 0x7f10d80b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10ddb21640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d815e320 0x7f10d80b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53416/0 (socket says 172.21.15.113:53416) 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10ddb21640 1 -- 172.21.15.113:0/336616393 learned_addr learned my addr 172.21.15.113:0/336616393 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10dcb1f640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d80d2290 0x7f10d8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.148+0000 7f10dd320640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d80d0b00 0x7f10d8153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10ddb21640 1 -- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d80d2290 msgr2=0x7f10d8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10ddb21640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d80d2290 0x7f10d8157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10ddb21640 1 -- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d80d0b00 msgr2=0x7f10d8153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10ddb21640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d80d0b00 0x7f10d8153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10ddb21640 1 -- 172.21.15.113:0/336616393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d8158930 con 0x7f10d815e320 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10ddb21640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d815e320 0x7f10d80b7c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c440 tx=0x7f10d400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10cdffb640 1 -- 172.21.15.113:0/336616393 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4011790 con 0x7f10d815e320 2024-04-24T23:59:11.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10cdffb640 1 -- 172.21.15.113:0/336616393 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4017070 con 0x7f10d815e320 2024-04-24T23:59:11.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d8158c10 con 0x7f10d815e320 2024-04-24T23:59:11.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10cdffb640 1 -- 172.21.15.113:0/336616393 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4011b70 con 0x7f10d815e320 2024-04-24T23:59:11.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.149+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81591d0 con 0x7f10d815e320 2024-04-24T23:59:11.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.150+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d81605c0 con 0x7f10d815e320 2024-04-24T23:59:11.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.150+0000 7f10cdffb640 1 -- 172.21.15.113:0/336616393 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10d4011d10 con 0x7f10d815e320 2024-04-24T23:59:11.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.151+0000 7f10cdffb640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10bc06be60 0x7f10bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.151+0000 7f10cdffb640 1 -- 172.21.15.113:0/336616393 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f10d408aca0 con 0x7f10d815e320 2024-04-24T23:59:11.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.151+0000 7f10dd320640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10bc06be60 0x7f10bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.151+0000 7f10dd320640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10bc06be60 0x7f10bc06e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10d0007ed0 tx=0x7f10d0007e60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.155+0000 7f10cdffb640 1 -- 172.21.15.113:0/336616393 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f10d40595c0 con 0x7f10d815e320 2024-04-24T23:59:11.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.328+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10d80691a0 con 0x7f10d815e320 2024-04-24T23:59:11.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.329+0000 7f10cdffb640 1 -- 172.21.15.113:0/336616393 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16730 (secure 0 0 0) 0x7f10d4058ec0 con 0x7f10d815e320 2024-04-24T23:59:11.329 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:11.329 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:59:10.364963+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:59:08.215866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:59:09.335267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:59:11.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10bc06be60 msgr2=0x7f10bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10bc06be60 0x7f10bc06e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10d0007ed0 tx=0x7f10d0007e60 comp rx=0 tx=0).stop 2024-04-24T23:59:11.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d815e320 msgr2=0x7f10d80b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d815e320 0x7f10d80b7c90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c440 tx=0x7f10d400c900 comp rx=0 tx=0).stop 2024-04-24T23:59:11.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 shutdown_connections 2024-04-24T23:59:11.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10bc06be60 0x7f10bc06e320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d815e320 0x7f10d80b7c90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d80d2290 0x7f10d8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 --2- 172.21.15.113:0/336616393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d80d0b00 0x7f10d8153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 >> 172.21.15.113:0/336616393 conn(0x7f10d80b5c20 msgr2=0x7f10d80d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:11.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 shutdown_connections 2024-04-24T23:59:11.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.331+0000 7f10df584640 1 -- 172.21.15.113:0/336616393 wait complete. 2024-04-24T23:59:11.342 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:59:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.439+0000 7f3d056e7640 1 Processor -- start 2024-04-24T23:59:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.439+0000 7f3d056e7640 1 -- start start 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3d056e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d00158910 0x7f3d00162710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3d056e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d0006b800 0x7f3d001438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3d056e7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d0015f1d0 con 0x7f3d0015e7a0 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3d056e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3d00162c50 con 0x7f3d0006b800 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3d056e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0015f4b0 con 0x7f3d00158910 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cff7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d0006b800 0x7f3d001438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cff7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d0006b800 0x7f3d001438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53424/0 (socket says 172.21.15.113:53424) 2024-04-24T23:59:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cff7fe640 1 -- 172.21.15.113:0/4027163829 learned_addr learned my addr 172.21.15.113:0/4027163829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cfeffd640 1 --2- 172.21.15.113:0/4027163829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d00158910 0x7f3d00162710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cff7fe640 1 -- 172.21.15.113:0/4027163829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d00158910 msgr2=0x7f3d00162710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cff7fe640 1 --2- 172.21.15.113:0/4027163829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d00158910 0x7f3d00162710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cff7fe640 1 -- 172.21.15.113:0/4027163829 >> v1:172.21.15.178:6789/0 conn(0x7f3d0015e7a0 legacy=0x7f3d0006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.440+0000 7f3cff7fe640 1 -- 172.21.15.113:0/4027163829 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d00161f10 con 0x7f3d0006b800 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.441+0000 7f3cff7fe640 1 --2- 172.21.15.113:0/4027163829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0006b800 0x7f3d001438f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009870 tx=0x7f3cf802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2372a452693a1d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.441+0000 7f3cfdffb640 1 -- 172.21.15.113:0/4027163829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8037070 con 0x7f3d0006b800 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.441+0000 7f3cfdffb640 1 -- 172.21.15.113:0/4027163829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf8032440 con 0x7f3d0006b800 2024-04-24T23:59:11.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.441+0000 7f3cfdffb640 1 -- 172.21.15.113:0/4027163829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8032810 con 0x7f3d0006b800 2024-04-24T23:59:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 -- 172.21.15.113:0/4027163829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0006b800 msgr2=0x7f3d001438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 --2- 172.21.15.113:0/4027163829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0006b800 0x7f3d001438f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009870 tx=0x7f3cf802a210 comp rx=0 tx=0).stop 2024-04-24T23:59:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 -- 172.21.15.113:0/4027163829 shutdown_connections 2024-04-24T23:59:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 --2- 172.21.15.113:0/4027163829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0006b800 0x7f3d001438f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 --2- 172.21.15.113:0/4027163829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d00158910 0x7f3d00162710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 -- 172.21.15.113:0/4027163829 >> 172.21.15.113:0/4027163829 conn(0x7f3d000b22c0 msgr2=0x7f3d000b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 -- 172.21.15.113:0/4027163829 shutdown_connections 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.442+0000 7f3d056e7640 1 -- 172.21.15.113:0/4027163829 wait complete. 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.443+0000 7f3d056e7640 1 Processor -- start 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.443+0000 7f3d056e7640 1 -- start start 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d0006b800 0x7f3d000cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d00158910 0x7f3d000afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d0015e7a0 0x7f3d00157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d000b3890 con 0x7f3d00158910 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d000b3710 con 0x7f3d0006b800 2024-04-24T23:59:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0015ec20 con 0x7f3d0015e7a0 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d00158910 0x7f3d000afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d00158910 0x7f3d000afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53428/0 (socket says 172.21.15.113:53428) 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 -- 172.21.15.113:0/2693999870 learned_addr learned my addr 172.21.15.113:0/2693999870 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfeffd640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d0006b800 0x7f3d000cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cff7fe640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d0015e7a0 0x7f3d00157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 -- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d0015e7a0 msgr2=0x7f3d00157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d0015e7a0 0x7f3d00157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 -- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d0006b800 msgr2=0x7f3d000cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d0006b800 0x7f3d000cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 -- 172.21.15.113:0/2693999870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d000af650 con 0x7f3d00158910 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3cfe7fc640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d00158910 0x7f3d000afab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400b420 tx=0x7f3cf400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3ce77fe640 1 -- 172.21.15.113:0/2693999870 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4011830 con 0x7f3d00158910 2024-04-24T23:59:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3ce77fe640 1 -- 172.21.15.113:0/2693999870 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf4016070 con 0x7f3d00158910 2024-04-24T23:59:11.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d00157a70 con 0x7f3d00158910 2024-04-24T23:59:11.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3ce77fe640 1 -- 172.21.15.113:0/2693999870 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4011c10 con 0x7f3d00158910 2024-04-24T23:59:11.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.444+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d000cd8e0 con 0x7f3d00158910 2024-04-24T23:59:11.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.445+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d000691a0 con 0x7f3d00158910 2024-04-24T23:59:11.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.446+0000 7f3ce77fe640 1 -- 172.21.15.113:0/2693999870 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cf401b460 con 0x7f3d00158910 2024-04-24T23:59:11.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.446+0000 7f3ce77fe640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3cec06be60 0x7f3cec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.446+0000 7f3cfeffd640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3cec06be60 0x7f3cec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.447+0000 7f3ce77fe640 1 -- 172.21.15.113:0/2693999870 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f3cf408a820 con 0x7f3d00158910 2024-04-24T23:59:11.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.447+0000 7f3cfeffd640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3cec06be60 0x7f3cec06e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0003b80 tx=0x7f3cf00098d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.451+0000 7f3ce77fe640 1 -- 172.21.15.113:0/2693999870 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cf4020020 con 0x7f3d00158910 2024-04-24T23:59:11.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.628+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d000b4d80 con 0x7f3d00158910 2024-04-24T23:59:11.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.630+0000 7f3ce77fe640 1 -- 172.21.15.113:0/2693999870 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16730 (secure 0 0 0) 0x7f3cf4058af0 con 0x7f3d00158910 2024-04-24T23:59:11.630 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:11.630 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:59:10.364963+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:59:08.215866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:59:09.335267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3cec06be60 msgr2=0x7f3cec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3cec06be60 0x7f3cec06e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0003b80 tx=0x7f3cf00098d0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d00158910 msgr2=0x7f3d000afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d00158910 0x7f3d000afab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400b420 tx=0x7f3cf400b8e0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 shutdown_connections 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3cec06be60 0x7f3cec06e320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d0015e7a0 0x7f3d00157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d00158910 0x7f3d000afab0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 --2- 172.21.15.113:0/2693999870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d0006b800 0x7f3d000cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.631+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 >> 172.21.15.113:0/2693999870 conn(0x7f3d000b22c0 msgr2=0x7f3d0006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.632+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 shutdown_connections 2024-04-24T23:59:11.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.632+0000 7f3d056e7640 1 -- 172.21.15.113:0/2693999870 wait complete. 2024-04-24T23:59:11.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:59:11.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.742+0000 7f832522c640 1 Processor -- start 2024-04-24T23:59:11.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.742+0000 7f832522c640 1 -- start start 2024-04-24T23:59:11.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.742+0000 7f832522c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f832015e2d0 0x7f832015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.742+0000 7f832522c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83200b3080 0x7f83200b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.742+0000 7f832522c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f832015f4f0 con 0x7f832015e6b0 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.742+0000 7f832522c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f83200cde00 con 0x7f832015e2d0 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.742+0000 7f832522c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f832015f7d0 con 0x7f83200b3080 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f832015e2d0 0x7f832015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f832015e2d0 0x7f832015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53440/0 (socket says 172.21.15.113:53440) 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 -- 172.21.15.113:0/3830356633 learned_addr learned my addr 172.21.15.113:0/3830356633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831f59e640 1 --1- 172.21.15.113:0/3830356633 >> v1:172.21.15.113:6789/0 conn(0x7f832015e6b0 0x7f8320071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40290/0 (socket says 172.21.15.113:40290) 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831e59c640 1 --2- 172.21.15.113:0/3830356633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83200b3080 0x7f83200b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 -- 172.21.15.113:0/3830356633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83200b3080 msgr2=0x7f83200b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:11.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 --2- 172.21.15.113:0/3830356633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83200b3080 0x7f83200b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 -- 172.21.15.113:0/3830356633 >> v1:172.21.15.113:6789/0 conn(0x7f832015e6b0 legacy=0x7f8320071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831dd9b640 1 -- 172.21.15.113:0/3830356633 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397278484 0 0) 0x7f832015f4f0 con 0x7f832015e6b0 2024-04-24T23:59:11.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 -- 172.21.15.113:0/3830356633 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832015fab0 con 0x7f832015e2d0 2024-04-24T23:59:11.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.743+0000 7f831ed9d640 1 --2- 172.21.15.113:0/3830356633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f832015ebf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8314009870 tx=0x7f831402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25a145726752a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f831dd9b640 1 -- 172.21.15.113:0/3830356633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314037070 con 0x7f832015e2d0 2024-04-24T23:59:11.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f831dd9b640 1 -- 172.21.15.113:0/3830356633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8314032440 con 0x7f832015e2d0 2024-04-24T23:59:11.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f831dd9b640 1 -- 172.21.15.113:0/3830356633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314032810 con 0x7f832015e2d0 2024-04-24T23:59:11.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 -- 172.21.15.113:0/3830356633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 msgr2=0x7f832015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 --2- 172.21.15.113:0/3830356633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f832015ebf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8314009870 tx=0x7f831402a1c0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 -- 172.21.15.113:0/3830356633 shutdown_connections 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 --2- 172.21.15.113:0/3830356633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83200b3080 0x7f83200b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 --2- 172.21.15.113:0/3830356633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f832015ebf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 -- 172.21.15.113:0/3830356633 >> 172.21.15.113:0/3830356633 conn(0x7f832006b0f0 msgr2=0x7f832006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 -- 172.21.15.113:0/3830356633 shutdown_connections 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.744+0000 7f832522c640 1 -- 172.21.15.113:0/3830356633 wait complete. 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 Processor -- start 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 -- start start 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83200b3080 0x7f83200ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f83200ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832015e6b0 0x7f8320152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f832015f2b0 con 0x7f832015e2d0 2024-04-24T23:59:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f832015f130 con 0x7f83200b3080 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f832522c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83200cf2a0 con 0x7f832015e6b0 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f831e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f83200ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f831e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f83200ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53450/0 (socket says 172.21.15.113:53450) 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.746+0000 7f831e59c640 1 -- 172.21.15.113:0/3720695331 learned_addr learned my addr 172.21.15.113:0/3720695331 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f831e59c640 1 -- 172.21.15.113:0/3720695331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832015e6b0 msgr2=0x7f8320152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f831e59c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832015e6b0 0x7f8320152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f831e59c640 1 -- 172.21.15.113:0/3720695331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83200b3080 msgr2=0x7f83200ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f831e59c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83200b3080 0x7f83200ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f831e59c640 1 -- 172.21.15.113:0/3720695331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83200ac220 con 0x7f832015e2d0 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f831e59c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f83200ab8b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f831000c7f0 tx=0x7f831000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f83077fe640 1 -- 172.21.15.113:0/3720695331 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310015070 con 0x7f832015e2d0 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f83077fe640 1 -- 172.21.15.113:0/3720695331 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310007820 con 0x7f832015e2d0 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83200ac500 con 0x7f832015e2d0 2024-04-24T23:59:11.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f83077fe640 1 -- 172.21.15.113:0/3720695331 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310007bf0 con 0x7f832015e2d0 2024-04-24T23:59:11.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.747+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83200acac0 con 0x7f832015e2d0 2024-04-24T23:59:11.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.748+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83200691a0 con 0x7f832015e2d0 2024-04-24T23:59:11.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.748+0000 7f83077fe640 1 -- 172.21.15.113:0/3720695331 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8310018070 con 0x7f832015e2d0 2024-04-24T23:59:11.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.749+0000 7f83077fe640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f830c06be60 0x7f830c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:11.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.749+0000 7f83077fe640 1 -- 172.21.15.113:0/3720695331 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f831008b0b0 con 0x7f832015e2d0 2024-04-24T23:59:11.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.749+0000 7f831ed9d640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f830c06be60 0x7f830c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:11.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.749+0000 7f831ed9d640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f830c06be60 0x7f830c06e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8314034000 tx=0x7f831402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:11.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.753+0000 7f83077fe640 1 -- 172.21.15.113:0/3720695331 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8310059950 con 0x7f832015e2d0 2024-04-24T23:59:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:11.962+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f83200ab690 con 0x7f832015e2d0 2024-04-24T23:59:12.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.033+0000 7f83077fe640 1 -- 172.21.15.113:0/3720695331 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7f8310059250 con 0x7f832015e2d0 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f830c06be60 msgr2=0x7f830c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f830c06be60 0x7f830c06e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8314034000 tx=0x7f831402c000 comp rx=0 tx=0).stop 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 msgr2=0x7f83200ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f83200ab8b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f831000c7f0 tx=0x7f831000ccb0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 shutdown_connections 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f830c06be60 0x7f830c06e320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832015e6b0 0x7f8320152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f832015e2d0 0x7f83200ab8b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 --2- 172.21.15.113:0/3720695331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83200b3080 0x7f83200ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 >> 172.21.15.113:0/3720695331 conn(0x7f832006b0f0 msgr2=0x7f83200b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:12.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 shutdown_connections 2024-04-24T23:59:12.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.036+0000 7f832522c640 1 -- 172.21.15.113:0/3720695331 wait complete. 2024-04-24T23:59:12.047 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T23:59:12.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T23:59:12.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.148+0000 7fd9fdb42640 1 Processor -- start 2024-04-24T23:59:12.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9fdb42640 1 -- start start 2024-04-24T23:59:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9fdb42640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd9f80b5ca0 0x7fd9f80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9fdb42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd9f80b73c0 0x7fd9f80b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9fdb42640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f80d2e00 con 0x7fd9f815e510 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9fdb42640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f80b6640 con 0x7fd9f80b5ca0 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9fdb42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f80d30e0 con 0x7fd9f80b73c0 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9f77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd9f815e510 0x7fd9f80d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40300/0 (socket says 172.21.15.113:40300) 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9f77fe640 1 -- 172.21.15.113:0/1038630998 learned_addr learned my addr 172.21.15.113:0/1038630998 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.149+0000 7fd9effff640 1 --2- 172.21.15.113:0/1038630998 >> v2:172.21.15.113:3300/0 conn(0x7fd9f80b5ca0 0x7fd9f80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9f7fff640 1 --2- 172.21.15.113:0/1038630998 >> v2:172.21.15.178:3300/0 conn(0x7fd9f80b73c0 0x7fd9f80b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9f6ffd640 1 -- 172.21.15.113:0/1038630998 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3619529680 0 0) 0x7fd9f80d2e00 con 0x7fd9f815e510 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9f6ffd640 1 -- 172.21.15.113:0/1038630998 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9e4003520 con 0x7fd9f815e510 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9effff640 1 -- 172.21.15.113:0/1038630998 >> v2:172.21.15.178:3300/0 conn(0x7fd9f80b73c0 msgr2=0x7fd9f80b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9effff640 1 --2- 172.21.15.113:0/1038630998 >> v2:172.21.15.178:3300/0 conn(0x7fd9f80b73c0 0x7fd9f80b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9effff640 1 -- 172.21.15.113:0/1038630998 >> v1:172.21.15.113:6789/0 conn(0x7fd9f815e510 legacy=0x7fd9f80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:12.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9effff640 1 -- 172.21.15.113:0/1038630998 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f80d33c0 con 0x7fd9f80b5ca0 2024-04-24T23:59:12.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.150+0000 7fd9effff640 1 --2- 172.21.15.113:0/1038630998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b5ca0 0x7fd9f80b6100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80d1d80 tx=0x7fd9e002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11747c97a76dddd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:12.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9f6ffd640 1 -- 172.21.15.113:0/1038630998 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0037070 con 0x7fd9f80b5ca0 2024-04-24T23:59:12.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9f6ffd640 1 -- 172.21.15.113:0/1038630998 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e0032440 con 0x7fd9f80b5ca0 2024-04-24T23:59:12.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9f6ffd640 1 -- 172.21.15.113:0/1038630998 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e002ae60 con 0x7fd9f80b5ca0 2024-04-24T23:59:12.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 -- 172.21.15.113:0/1038630998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b5ca0 msgr2=0x7fd9f80b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/1038630998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b5ca0 0x7fd9f80b6100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80d1d80 tx=0x7fd9e002a110 comp rx=0 tx=0).stop 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 -- 172.21.15.113:0/1038630998 shutdown_connections 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/1038630998 >> v2:172.21.15.178:3300/0 conn(0x7fd9f80b73c0 0x7fd9f80b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/1038630998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b5ca0 0x7fd9f80b6100 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 -- 172.21.15.113:0/1038630998 >> 172.21.15.113:0/1038630998 conn(0x7fd9f806b720 msgr2=0x7fd9f80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 -- 172.21.15.113:0/1038630998 shutdown_connections 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.151+0000 7fd9fdb42640 1 -- 172.21.15.113:0/1038630998 wait complete. 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.152+0000 7fd9fdb42640 1 Processor -- start 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.152+0000 7fd9fdb42640 1 -- start start 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9fdb42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9f80b5ca0 0x7fd9f815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9fdb42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b73c0 0x7fd9f814b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9fdb42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f815e510 0x7fd9f81611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:12.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9fdb42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80d3ea0 con 0x7fd9f80b73c0 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9fdb42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80d3980 con 0x7fd9f815e510 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9fdb42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f80d2a90 con 0x7fd9f80b5ca0 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b73c0 0x7fd9f814b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b73c0 0x7fd9f814b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53472/0 (socket says 172.21.15.113:53472) 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 -- 172.21.15.113:0/641025665 learned_addr learned my addr 172.21.15.113:0/641025665 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9f7fff640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f815e510 0x7fd9f81611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9f77fe640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9f80b5ca0 0x7fd9f815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 -- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9f80b5ca0 msgr2=0x7fd9f815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9f80b5ca0 0x7fd9f815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 -- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f815e510 msgr2=0x7fd9f81611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:12.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f815e510 0x7fd9f81611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9effff640 1 -- 172.21.15.113:0/641025665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f8160100 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9f7fff640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f815e510 0x7fd9f81611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.153+0000 7fd9f77fe640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9f80b5ca0 0x7fd9f815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.154+0000 7fd9effff640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b73c0 0x7fd9f814b730 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0000c00 tx=0x7fd9e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.154+0000 7fd9f4ff9640 1 -- 172.21.15.113:0/641025665 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0037040 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.154+0000 7fd9f4ff9640 1 -- 172.21.15.113:0/641025665 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e0042070 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.154+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f81603e0 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.154+0000 7fd9f4ff9640 1 -- 172.21.15.113:0/641025665 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0030960 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.154+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81609a0 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.155+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f80691a0 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.155+0000 7fd9f4ff9640 1 -- 172.21.15.113:0/641025665 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9e0030b00 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.156+0000 7fd9f4ff9640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd9d806be30 0x7fd9d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.156+0000 7fd9f4ff9640 1 -- 172.21.15.113:0/641025665 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd9e00ab720 con 0x7fd9f80b73c0 2024-04-24T23:59:12.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.156+0000 7fd9f77fe640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd9d806be30 0x7fd9d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:12.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.156+0000 7fd9f77fe640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd9d806be30 0x7fd9d806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8005880 tx=0x7fd9e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:12.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.160+0000 7fd9f4ff9640 1 -- 172.21.15.113:0/641025665 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd9e007a280 con 0x7fd9f80b73c0 2024-04-24T23:59:12.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:12.370+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd9f815d6c0 con 0x7fd9f80b73c0 2024-04-24T23:59:13.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.038+0000 7fd9f4ff9640 1 -- 172.21.15.113:0/641025665 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fd9e0079b80 con 0x7fd9f80b73c0 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.040+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd9d806be30 msgr2=0x7fd9d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.040+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd9d806be30 0x7fd9d806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8005880 tx=0x7fd9e8009040 comp rx=0 tx=0).stop 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.040+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b73c0 msgr2=0x7fd9f814b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.040+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b73c0 0x7fd9f814b730 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0000c00 tx=0x7fd9e00046c0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 shutdown_connections 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd9d806be30 0x7fd9d806e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f815e510 0x7fd9f81611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f80b73c0 0x7fd9f814b730 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 --2- 172.21.15.113:0/641025665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9f80b5ca0 0x7fd9f815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 >> 172.21.15.113:0/641025665 conn(0x7fd9f806b720 msgr2=0x7fd9f8071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 shutdown_connections 2024-04-24T23:59:13.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.041+0000 7fd9fdb42640 1 -- 172.21.15.113:0/641025665 wait complete. 2024-04-24T23:59:13.054 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T23:59:13.054 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:13.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.155+0000 7f8a76fb7640 1 Processor -- start 2024-04-24T23:59:13.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.155+0000 7f8a76fb7640 1 -- start start 2024-04-24T23:59:13.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.156+0000 7f8a76fb7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8a70144d60 0x7f8a700b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.156+0000 7f8a76fb7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8a701453c0 0x7f8a7006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.156+0000 7f8a76fb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a7006e510 0x7f8a7015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.156+0000 7f8a76fb7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8a70145c40 con 0x7f8a701453c0 2024-04-24T23:59:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.156+0000 7f8a76fb7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8a700cfad0 con 0x7f8a70144d60 2024-04-24T23:59:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.156+0000 7f8a76fb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8a7006e920 con 0x7f8a7006e510 2024-04-24T23:59:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.156+0000 7f8a6ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8a701453c0 0x7f8a7006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8a701453c0 0x7f8a7006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53484/0 (socket says 172.21.15.113:53484) 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 -- 172.21.15.113:0/794047662 learned_addr learned my addr 172.21.15.113:0/794047662 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a74d53640 1 --2- 172.21.15.113:0/794047662 >> v2:172.21.15.178:3300/0 conn(0x7f8a70144d60 0x7f8a700b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a75554640 1 --2- 172.21.15.113:0/794047662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a7006e510 0x7f8a7015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 -- 172.21.15.113:0/794047662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a7006e510 msgr2=0x7f8a7015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 --2- 172.21.15.113:0/794047662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a7006e510 0x7f8a7015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 -- 172.21.15.113:0/794047662 >> v2:172.21.15.178:3300/0 conn(0x7f8a70144d60 msgr2=0x7f8a700b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 --2- 172.21.15.113:0/794047662 >> v2:172.21.15.178:3300/0 conn(0x7f8a70144d60 0x7f8a700b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 -- 172.21.15.113:0/794047662 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a700cf230 con 0x7f8a701453c0 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.157+0000 7f8a6ffff640 1 --2- 172.21.15.113:0/794047662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701453c0 0x7f8a7006dfd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8a7006f970 tx=0x7f8a6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b6b1ad0fc87e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a6f7fe640 1 -- 172.21.15.113:0/794047662 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64037070 con 0x7f8a701453c0 2024-04-24T23:59:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a6f7fe640 1 -- 172.21.15.113:0/794047662 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a64032440 con 0x7f8a701453c0 2024-04-24T23:59:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a6f7fe640 1 -- 172.21.15.113:0/794047662 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64032810 con 0x7f8a701453c0 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 -- 172.21.15.113:0/794047662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701453c0 msgr2=0x7f8a7006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/794047662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701453c0 0x7f8a7006dfd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8a7006f970 tx=0x7f8a6402a140 comp rx=0 tx=0).stop 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 -- 172.21.15.113:0/794047662 shutdown_connections 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/794047662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a7006e510 0x7f8a7015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/794047662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a701453c0 0x7f8a7006dfd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/794047662 >> v2:172.21.15.178:3300/0 conn(0x7f8a70144d60 0x7f8a700b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 -- 172.21.15.113:0/794047662 >> 172.21.15.113:0/794047662 conn(0x7f8a700b1a50 msgr2=0x7f8a700b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 -- 172.21.15.113:0/794047662 shutdown_connections 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.158+0000 7f8a76fb7640 1 -- 172.21.15.113:0/794047662 wait complete. 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.159+0000 7f8a76fb7640 1 Processor -- start 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.159+0000 7f8a76fb7640 1 -- start start 2024-04-24T23:59:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a76fb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7006e510 0x7f8a70157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a76fb7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a70144d60 0x7f8a70158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a76fb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a701453c0 0x7f8a7013ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a76fb7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70070e10 con 0x7f8a7006e510 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a76fb7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a700cf7f0 con 0x7f8a70144d60 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a76fb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8a70145140 con 0x7f8a701453c0 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a74d53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7006e510 0x7f8a70157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a74d53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7006e510 0x7f8a70157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53500/0 (socket says 172.21.15.113:53500) 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.160+0000 7f8a74d53640 1 -- 172.21.15.113:0/1555990769 learned_addr learned my addr 172.21.15.113:0/1555990769 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a75554640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a701453c0 0x7f8a7013ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a6ffff640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a70144d60 0x7f8a70158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a74d53640 1 -- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a701453c0 msgr2=0x7f8a7013ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a74d53640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a701453c0 0x7f8a7013ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a74d53640 1 -- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a70144d60 msgr2=0x7f8a70158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a74d53640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a70144d60 0x7f8a70158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a74d53640 1 -- 172.21.15.113:0/1555990769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a701e5b10 con 0x7f8a7006e510 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a74d53640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7006e510 0x7f8a70157ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d5f0 tx=0x7f8a6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a6d7fa640 1 -- 172.21.15.113:0/1555990769 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a600125f0 con 0x7f8a7006e510 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a6d7fa640 1 -- 172.21.15.113:0/1555990769 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a60019070 con 0x7f8a7006e510 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a701e5df0 con 0x7f8a7006e510 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a6d7fa640 1 -- 172.21.15.113:0/1555990769 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a600129d0 con 0x7f8a7006e510 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.161+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a701e63b0 con 0x7f8a7006e510 2024-04-24T23:59:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.162+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a700691a0 con 0x7f8a7006e510 2024-04-24T23:59:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.163+0000 7f8a6d7fa640 1 -- 172.21.15.113:0/1555990769 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a60012b70 con 0x7f8a7006e510 2024-04-24T23:59:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.163+0000 7f8a6d7fa640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a5c06be30 0x7f8a5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.163+0000 7f8a6ffff640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a5c06be30 0x7f8a5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.163+0000 7f8a6d7fa640 1 -- 172.21.15.113:0/1555990769 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8a6008b270 con 0x7f8a7006e510 2024-04-24T23:59:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.164+0000 7f8a6ffff640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a5c06be30 0x7f8a5c06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a64034000 tx=0x7f8a6402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.167+0000 7f8a6d7fa640 1 -- 172.21.15.113:0/1555990769 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a60059c20 con 0x7f8a7006e510 2024-04-24T23:59:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.377+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a701607b0 con 0x7f8a7006e510 2024-04-24T23:59:13.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.378+0000 7f8a6d7fa640 1 -- 172.21.15.113:0/1555990769 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5193 (secure 0 0 0) 0x7f8a60059520 con 0x7f8a7006e510 2024-04-24T23:59:13.378 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:13.378 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2613614055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2613614055},{"type":"v1","addr":"172.21.15.178:6834","nonce":2613614055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2494825991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2494825991},{"type":"v1","addr":"172.21.15.178:6835","nonce":2494825991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2843749120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2843749120},{"type":"v1","addr":"172.21.15.113:6836","nonce":2843749120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:59:10.365260+0000","modified":"2024-04-24T23:59:12.371753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3337939581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3337939581},{"type":"v1","addr":"172.21.15.113:6837","nonce":3337939581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-04-24T23:59:13.378 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 14 2024-04-24T23:59:13.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a5c06be30 msgr2=0x7f8a5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a5c06be30 0x7f8a5c06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a64034000 tx=0x7f8a6402e000 comp rx=0 tx=0).stop 2024-04-24T23:59:13.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7006e510 msgr2=0x7f8a70157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7006e510 0x7f8a70157ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d5f0 tx=0x7f8a6000dab0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 shutdown_connections 2024-04-24T23:59:13.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a5c06be30 0x7f8a5c06e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a70144d60 0x7f8a70158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a701453c0 0x7f8a7013ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 --2- 172.21.15.113:0/1555990769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7006e510 0x7f8a70157ad0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 >> 172.21.15.113:0/1555990769 conn(0x7f8a700b1a50 msgr2=0x7f8a7013f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 shutdown_connections 2024-04-24T23:59:13.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.380+0000 7f8a76fb7640 1 -- 172.21.15.113:0/1555990769 wait complete. 2024-04-24T23:59:13.391 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:59:13.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.487+0000 7f23edfee640 1 Processor -- start 2024-04-24T23:59:13.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.487+0000 7f23edfee640 1 -- start start 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.488+0000 7f23edfee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f23e815e320 0x7f23e80d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.488+0000 7f23edfee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.488+0000 7f23edfee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e80d3de0 con 0x7f23e80d0b00 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.488+0000 7f23edfee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f23e80d3410 con 0x7f23e815e320 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.488+0000 7f23edfee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f23e80d2800 con 0x7f23e80d2290 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e6ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e77fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f23e815e320 0x7f23e80d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e77fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f23e815e320 0x7f23e80d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55650/0 (socket says 172.21.15.113:55650) 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e7fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f23e80d0b00 0x7f23e80b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33238/0 (socket says 172.21.15.113:33238) 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e77fe640 1 -- 172.21.15.113:0/3075148960 learned_addr learned my addr 172.21.15.113:0/3075148960 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e6ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:54292/0 (socket says 172.21.15.113:54292) 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e6ffd640 1 -- 172.21.15.113:0/3075148960 >> v2:172.21.15.178:3300/0 conn(0x7f23e815e320 msgr2=0x7f23e80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e6ffd640 1 --2- 172.21.15.113:0/3075148960 >> v2:172.21.15.178:3300/0 conn(0x7f23e815e320 0x7f23e80d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e6ffd640 1 -- 172.21.15.113:0/3075148960 >> v1:172.21.15.178:6789/0 conn(0x7f23e80d0b00 legacy=0x7f23e80b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e6ffd640 1 -- 172.21.15.113:0/3075148960 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e80d2ae0 con 0x7f23e80d2290 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.489+0000 7f23e77fe640 1 --2- 172.21.15.113:0/3075148960 >> v2:172.21.15.178:3300/0 conn(0x7f23e815e320 0x7f23e80d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23e6ffd640 1 --2- 172.21.15.113:0/3075148960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b9aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0097a0 tx=0x7f23dc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbaa2b34536a98c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23e67fc640 1 -- 172.21.15.113:0/3075148960 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc037070 con 0x7f23e80d2290 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23e67fc640 1 -- 172.21.15.113:0/3075148960 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23dc032440 con 0x7f23e80d2290 2024-04-24T23:59:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23e67fc640 1 -- 172.21.15.113:0/3075148960 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc032810 con 0x7f23e80d2290 2024-04-24T23:59:13.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 -- 172.21.15.113:0/3075148960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 msgr2=0x7f23e80b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 --2- 172.21.15.113:0/3075148960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b9aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0097a0 tx=0x7f23dc02a210 comp rx=0 tx=0).stop 2024-04-24T23:59:13.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 -- 172.21.15.113:0/3075148960 shutdown_connections 2024-04-24T23:59:13.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 --2- 172.21.15.113:0/3075148960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b9aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 --2- 172.21.15.113:0/3075148960 >> v2:172.21.15.178:3300/0 conn(0x7f23e815e320 0x7f23e80d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 -- 172.21.15.113:0/3075148960 >> 172.21.15.113:0/3075148960 conn(0x7f23e80b5c20 msgr2=0x7f23e80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 -- 172.21.15.113:0/3075148960 shutdown_connections 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.490+0000 7f23edfee640 1 -- 172.21.15.113:0/3075148960 wait complete. 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 Processor -- start 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 -- start start 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80d0b00 0x7f23e80b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e815e320 0x7f23e815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23e80b9fe0 con 0x7f23e80d0b00 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23e80d30a0 con 0x7f23e815e320 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.492+0000 7f23edfee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f23e815e700 con 0x7f23e80d2290 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80d0b00 0x7f23e80b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80d0b00 0x7f23e80b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53508/0 (socket says 172.21.15.113:53508) 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 -- 172.21.15.113:0/3069364222 learned_addr learned my addr 172.21.15.113:0/3069364222 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e7fff640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e815e320 0x7f23e815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e6ffd640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 -- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 msgr2=0x7f23e80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 -- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e815e320 msgr2=0x7f23e815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e815e320 0x7f23e815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 -- 172.21.15.113:0/3069364222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81582a0 con 0x7f23e80d0b00 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23e77fe640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80d0b00 0x7f23e80b7dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23d800e9b0 tx=0x7f23d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23cffff640 1 -- 172.21.15.113:0/3069364222 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8017070 con 0x7f23e80d0b00 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23cffff640 1 -- 172.21.15.113:0/3069364222 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23d800c9a0 con 0x7f23e80d0b00 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e815eb80 con 0x7f23e80d0b00 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23cffff640 1 -- 172.21.15.113:0/3069364222 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d800cd50 con 0x7f23e80d0b00 2024-04-24T23:59:13.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.493+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e815f140 con 0x7f23e80d0b00 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.494+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e80691a0 con 0x7f23e80d0b00 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.495+0000 7f23cffff640 1 -- 172.21.15.113:0/3069364222 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23d801a070 con 0x7f23e80d0b00 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.495+0000 7f23cffff640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23d406be30 0x7f23d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.495+0000 7f23e6ffd640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23d406be30 0x7f23d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.495+0000 7f23cffff640 1 -- 172.21.15.113:0/3069364222 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f23d808abd0 con 0x7f23e80d0b00 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.496+0000 7f23e6ffd640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23d406be30 0x7f23d406e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23dc034000 tx=0x7f23dc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.499+0000 7f23cffff640 1 -- 172.21.15.113:0/3069364222 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23d8058fc0 con 0x7f23e80d0b00 2024-04-24T23:59:13.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.673+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23e8074f50 con 0x7f23e80d0b00 2024-04-24T23:59:13.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.675+0000 7f23cffff640 1 -- 172.21.15.113:0/3069364222 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16717 (secure 0 0 0) 0x7f23d805d2c0 con 0x7f23e80d0b00 2024-04-24T23:59:13.675 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:13.675 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":32,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-24T23:59:12.030360+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:59:08.215866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:59:09.335267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:59:13.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23d406be30 msgr2=0x7f23d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23d406be30 0x7f23d406e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23dc034000 tx=0x7f23dc02e000 comp rx=0 tx=0).stop 2024-04-24T23:59:13.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80d0b00 msgr2=0x7f23e80b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80d0b00 0x7f23e80b7dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23d800e9b0 tx=0x7f23d800ee70 comp rx=0 tx=0).stop 2024-04-24T23:59:13.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 shutdown_connections 2024-04-24T23:59:13.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23d406be30 0x7f23d406e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e815e320 0x7f23e815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23e80d2290 0x7f23e80b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 --2- 172.21.15.113:0/3069364222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80d0b00 0x7f23e80b7dc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 >> 172.21.15.113:0/3069364222 conn(0x7f23e80b5c20 msgr2=0x7f23e80b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 shutdown_connections 2024-04-24T23:59:13.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.677+0000 7f23edfee640 1 -- 172.21.15.113:0/3069364222 wait complete. 2024-04-24T23:59:13.688 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:59:13.688 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:59:13.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.785+0000 7fc84ce1b640 1 Processor -- start 2024-04-24T23:59:13.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.785+0000 7fc84ce1b640 1 -- start start 2024-04-24T23:59:13.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc84ce1b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8480b3fd0 0x7fc84806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc84ce1b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc84806e510 0x7fc8480cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc84ce1b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc848144cf0 con 0x7fc848144910 2024-04-24T23:59:13.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc84ce1b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc8480b2f30 con 0x7fc84806e510 2024-04-24T23:59:13.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc84ce1b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8480b3210 con 0x7fc8480b3fd0 2024-04-24T23:59:13.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc846d9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc84806e510 0x7fc8480cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc845d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8480b3fd0 0x7fc84806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc84659c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc848144910 0x7fc8480b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33246/0 (socket says 172.21.15.113:33246) 2024-04-24T23:59:13.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc84659c640 1 -- 172.21.15.113:0/1248710758 learned_addr learned my addr 172.21.15.113:0/1248710758 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:13.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc845d9b640 1 -- 172.21.15.113:0/1248710758 >> v2:172.21.15.178:3300/0 conn(0x7fc84806e510 msgr2=0x7fc8480cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:13.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc845d9b640 1 --2- 172.21.15.113:0/1248710758 >> v2:172.21.15.178:3300/0 conn(0x7fc84806e510 0x7fc8480cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc845d9b640 1 -- 172.21.15.113:0/1248710758 >> v1:172.21.15.178:6789/0 conn(0x7fc848144910 legacy=0x7fc8480b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.786+0000 7fc845d9b640 1 -- 172.21.15.113:0/1248710758 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8480b34f0 con 0x7fc8480b3fd0 2024-04-24T23:59:13.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.787+0000 7fc845d9b640 1 --2- 172.21.15.113:0/1248710758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8480b3fd0 0x7fc84806dfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097a0 tx=0x7fc83c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7a212790451121f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.787+0000 7fc84559a640 1 -- 172.21.15.113:0/1248710758 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c037070 con 0x7fc8480b3fd0 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.787+0000 7fc84559a640 1 -- 172.21.15.113:0/1248710758 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc83c032440 con 0x7fc8480b3fd0 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.787+0000 7fc84559a640 1 -- 172.21.15.113:0/1248710758 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c032810 con 0x7fc8480b3fd0 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 -- 172.21.15.113:0/1248710758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8480b3fd0 msgr2=0x7fc84806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/1248710758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8480b3fd0 0x7fc84806dfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097a0 tx=0x7fc83c02a180 comp rx=0 tx=0).stop 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 -- 172.21.15.113:0/1248710758 shutdown_connections 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/1248710758 >> v2:172.21.15.178:3300/0 conn(0x7fc84806e510 0x7fc8480cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/1248710758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8480b3fd0 0x7fc84806dfd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 -- 172.21.15.113:0/1248710758 >> 172.21.15.113:0/1248710758 conn(0x7fc8480b1a50 msgr2=0x7fc8480b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 -- 172.21.15.113:0/1248710758 shutdown_connections 2024-04-24T23:59:13.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.788+0000 7fc84ce1b640 1 -- 172.21.15.113:0/1248710758 wait complete. 2024-04-24T23:59:13.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.789+0000 7fc84ce1b640 1 Processor -- start 2024-04-24T23:59:13.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.789+0000 7fc84ce1b640 1 -- start start 2024-04-24T23:59:13.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc84ce1b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc84806e510 0x7fc848158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc84ce1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8480b3fd0 0x7fc8481f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc84ce1b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc848144910 0x7fc8481574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc84ce1b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8480b4da0 con 0x7fc8480b3fd0 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc84ce1b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8480b4410 con 0x7fc84806e510 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc84ce1b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8480b4f20 con 0x7fc848144910 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8480b3fd0 0x7fc8481f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8480b3fd0 0x7fc8481f48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53518/0 (socket says 172.21.15.113:53518) 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 -- 172.21.15.113:0/2299828168 learned_addr learned my addr 172.21.15.113:0/2299828168 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 -- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc848144910 msgr2=0x7fc8481574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc84659c640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc84806e510 0x7fc848158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc846d9d640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc848144910 0x7fc8481574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc848144910 0x7fc8481574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 -- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc84806e510 msgr2=0x7fc848158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc84806e510 0x7fc848158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.790+0000 7fc845d9b640 1 -- 172.21.15.113:0/2299828168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84813fd80 con 0x7fc8480b3fd0 2024-04-24T23:59:13.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.791+0000 7fc845d9b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8480b3fd0 0x7fc8481f48f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc83c035040 tx=0x7fc83c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.791+0000 7fc836ffd640 1 -- 172.21.15.113:0/2299828168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c03f070 con 0x7fc8480b3fd0 2024-04-24T23:59:13.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.791+0000 7fc836ffd640 1 -- 172.21.15.113:0/2299828168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc83c03d070 con 0x7fc8480b3fd0 2024-04-24T23:59:13.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.791+0000 7fc836ffd640 1 -- 172.21.15.113:0/2299828168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c037040 con 0x7fc8480b3fd0 2024-04-24T23:59:13.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.791+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc848140060 con 0x7fc8480b3fd0 2024-04-24T23:59:13.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.791+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84813fa50 con 0x7fc8480b3fd0 2024-04-24T23:59:13.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.792+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc848159f00 con 0x7fc8480b3fd0 2024-04-24T23:59:13.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.792+0000 7fc836ffd640 1 -- 172.21.15.113:0/2299828168 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc83c030b70 con 0x7fc8480b3fd0 2024-04-24T23:59:13.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.792+0000 7fc836ffd640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc82806be30 0x7fc82806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.793+0000 7fc836ffd640 1 -- 172.21.15.113:0/2299828168 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc83c0acae0 con 0x7fc8480b3fd0 2024-04-24T23:59:13.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.793+0000 7fc84659c640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc82806be30 0x7fc82806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.793+0000 7fc84659c640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc82806be30 0x7fc82806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc838000f60 tx=0x7fc838009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.797+0000 7fc836ffd640 1 -- 172.21.15.113:0/2299828168 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc83c07b520 con 0x7fc8480b3fd0 2024-04-24T23:59:13.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.966+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8480691a0 con 0x7fc82806be30 2024-04-24T23:59:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.972+0000 7fc836ffd640 1 -- 172.21.15.113:0/2299828168 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251878 (secure 0 0 0) 0x7fc8480691a0 con 0x7fc82806be30 2024-04-24T23:59:13.974 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:13.976 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:59:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.974+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc82806be30 msgr2=0x7fc82806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.974+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc82806be30 0x7fc82806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc838000f60 tx=0x7fc838009040 comp rx=0 tx=0).stop 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8480b3fd0 msgr2=0x7fc8481f48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8480b3fd0 0x7fc8481f48f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc83c035040 tx=0x7fc83c0046c0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 shutdown_connections 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc82806be30 0x7fc82806e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc848144910 0x7fc8481574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8480b3fd0 0x7fc8481f48f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 --2- 172.21.15.113:0/2299828168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc84806e510 0x7fc848158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 >> 172.21.15.113:0/2299828168 conn(0x7fc8480b1a50 msgr2=0x7fc8480cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 shutdown_connections 2024-04-24T23:59:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:13.975+0000 7fc84ce1b640 1 -- 172.21.15.113:0/2299828168 wait complete. 2024-04-24T23:59:13.985 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-04-24T23:59:13.121805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236620,"kb_used_data":3800,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738100,"statfs":{"total":773094113280,"available":772851814400,"internally_reserved":0,"allocated":3891200,"data_stored":2207645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.112370"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040722+0000","last_change":"2024-04-24T23:59:11.039494+0000","last_active":"2024-04-24T23:59:11.040722+0000","last_peered":"2024-04-24T23:59:11.040722+0000","last_clean":"2024-04-24T23:59:11.040722+0000","last_became_active":"2024-04-24T23:59:11.039116+0000","last_became_peered":"2024-04-24T23:59:11.039116+0000","last_unstale":"2024-04-24T23:59:11.040722+0000","last_undegraded":"2024-04-24T23:59:11.040722+0000","last_fullsized":"2024-04-24T23:59:11.040722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:14.713041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049679+0000","last_change":"2024-04-24T23:59:11.042430+0000","last_active":"2024-04-24T23:59:11.049679+0000","last_peered":"2024-04-24T23:59:11.049679+0000","last_clean":"2024-04-24T23:59:11.049679+0000","last_became_active":"2024-04-24T23:59:11.042192+0000","last_became_peered":"2024-04-24T23:59:11.042192+0000","last_unstale":"2024-04-24T23:59:11.049679+0000","last_undegraded":"2024-04-24T23:59:11.049679+0000","last_fullsized":"2024-04-24T23:59:11.049679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:58.979989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048419+0000","last_change":"2024-04-24T23:59:11.043364+0000","last_active":"2024-04-24T23:59:11.048419+0000","last_peered":"2024-04-24T23:59:11.048419+0000","last_clean":"2024-04-24T23:59:11.048419+0000","last_became_active":"2024-04-24T23:59:11.043056+0000","last_became_peered":"2024-04-24T23:59:11.043056+0000","last_unstale":"2024-04-24T23:59:11.048419+0000","last_undegraded":"2024-04-24T23:59:11.048419+0000","last_fullsized":"2024-04-24T23:59:11.048419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:58.900435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049016+0000","last_change":"2024-04-24T23:59:11.046150+0000","last_active":"2024-04-24T23:59:11.049016+0000","last_peered":"2024-04-24T23:59:11.049016+0000","last_clean":"2024-04-24T23:59:11.049016+0000","last_became_active":"2024-04-24T23:59:11.045715+0000","last_became_peered":"2024-04-24T23:59:11.045715+0000","last_unstale":"2024-04-24T23:59:11.049016+0000","last_undegraded":"2024-04-24T23:59:11.049016+0000","last_fullsized":"2024-04-24T23:59:11.049016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:48.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043266+0000","last_change":"2024-04-24T23:59:11.037172+0000","last_active":"2024-04-24T23:59:11.043266+0000","last_peered":"2024-04-24T23:59:11.043266+0000","last_clean":"2024-04-24T23:59:11.043266+0000","last_became_active":"2024-04-24T23:59:11.036894+0000","last_became_peered":"2024-04-24T23:59:11.036894+0000","last_unstale":"2024-04-24T23:59:11.043266+0000","last_undegraded":"2024-04-24T23:59:11.043266+0000","last_fullsized":"2024-04-24T23:59:11.043266+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:16.435100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048816+0000","last_change":"2024-04-24T23:59:11.040247+0000","last_active":"2024-04-24T23:59:11.048816+0000","last_peered":"2024-04-24T23:59:11.048816+0000","last_clean":"2024-04-24T23:59:11.048816+0000","last_became_active":"2024-04-24T23:59:11.039965+0000","last_became_peered":"2024-04-24T23:59:11.039965+0000","last_unstale":"2024-04-24T23:59:11.048816+0000","last_undegraded":"2024-04-24T23:59:11.048816+0000","last_fullsized":"2024-04-24T23:59:11.048816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:03.928528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039974+0000","last_change":"2024-04-24T23:59:11.031683+0000","last_active":"2024-04-24T23:59:11.039974+0000","last_peered":"2024-04-24T23:59:11.039974+0000","last_clean":"2024-04-24T23:59:11.039974+0000","last_became_active":"2024-04-24T23:59:11.031380+0000","last_became_peered":"2024-04-24T23:59:11.031380+0000","last_unstale":"2024-04-24T23:59:11.039974+0000","last_undegraded":"2024-04-24T23:59:11.039974+0000","last_fullsized":"2024-04-24T23:59:11.039974+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:06.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040004+0000","last_change":"2024-04-24T23:59:11.031954+0000","last_active":"2024-04-24T23:59:11.040004+0000","last_peered":"2024-04-24T23:59:11.040004+0000","last_clean":"2024-04-24T23:59:11.040004+0000","last_became_active":"2024-04-24T23:59:11.031656+0000","last_became_peered":"2024-04-24T23:59:11.031656+0000","last_unstale":"2024-04-24T23:59:11.040004+0000","last_undegraded":"2024-04-24T23:59:11.040004+0000","last_fullsized":"2024-04-24T23:59:11.040004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:48.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041759+0000","last_change":"2024-04-24T23:59:11.034961+0000","last_active":"2024-04-24T23:59:11.041759+0000","last_peered":"2024-04-24T23:59:11.041759+0000","last_clean":"2024-04-24T23:59:11.041759+0000","last_became_active":"2024-04-24T23:59:11.034633+0000","last_became_peered":"2024-04-24T23:59:11.034633+0000","last_unstale":"2024-04-24T23:59:11.041759+0000","last_undegraded":"2024-04-24T23:59:11.041759+0000","last_fullsized":"2024-04-24T23:59:11.041759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:00.478646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046579+0000","last_change":"2024-04-24T23:59:11.038976+0000","last_active":"2024-04-24T23:59:11.046579+0000","last_peered":"2024-04-24T23:59:11.046579+0000","last_clean":"2024-04-24T23:59:11.046579+0000","last_became_active":"2024-04-24T23:59:11.038674+0000","last_became_peered":"2024-04-24T23:59:11.038674+0000","last_unstale":"2024-04-24T23:59:11.046579+0000","last_undegraded":"2024-04-24T23:59:11.046579+0000","last_fullsized":"2024-04-24T23:59:11.046579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:06.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043445+0000","last_change":"2024-04-24T23:59:11.041937+0000","last_active":"2024-04-24T23:59:11.043445+0000","last_peered":"2024-04-24T23:59:11.043445+0000","last_clean":"2024-04-24T23:59:11.043445+0000","last_became_active":"2024-04-24T23:59:11.041670+0000","last_became_peered":"2024-04-24T23:59:11.041670+0000","last_unstale":"2024-04-24T23:59:11.043445+0000","last_undegraded":"2024-04-24T23:59:11.043445+0000","last_fullsized":"2024-04-24T23:59:11.043445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039788+0000","last_change":"2024-04-24T23:59:11.037582+0000","last_active":"2024-04-24T23:59:11.039788+0000","last_peered":"2024-04-24T23:59:11.039788+0000","last_clean":"2024-04-24T23:59:11.039788+0000","last_became_active":"2024-04-24T23:59:11.037283+0000","last_became_peered":"2024-04-24T23:59:11.037283+0000","last_unstale":"2024-04-24T23:59:11.039788+0000","last_undegraded":"2024-04-24T23:59:11.039788+0000","last_fullsized":"2024-04-24T23:59:11.039788+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:48.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049631+0000","last_change":"2024-04-24T23:59:11.045621+0000","last_active":"2024-04-24T23:59:11.049631+0000","last_peered":"2024-04-24T23:59:11.049631+0000","last_clean":"2024-04-24T23:59:11.049631+0000","last_became_active":"2024-04-24T23:59:11.045370+0000","last_became_peered":"2024-04-24T23:59:11.045370+0000","last_unstale":"2024-04-24T23:59:11.049631+0000","last_undegraded":"2024-04-24T23:59:11.049631+0000","last_fullsized":"2024-04-24T23:59:11.049631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:38.069745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049799+0000","last_change":"2024-04-24T23:59:11.045863+0000","last_active":"2024-04-24T23:59:11.049799+0000","last_peered":"2024-04-24T23:59:11.049799+0000","last_clean":"2024-04-24T23:59:11.049799+0000","last_became_active":"2024-04-24T23:59:11.045093+0000","last_became_peered":"2024-04-24T23:59:11.045093+0000","last_unstale":"2024-04-24T23:59:11.049799+0000","last_undegraded":"2024-04-24T23:59:11.049799+0000","last_fullsized":"2024-04-24T23:59:11.049799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:34.170794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046916+0000","last_change":"2024-04-24T23:59:11.041196+0000","last_active":"2024-04-24T23:59:11.046916+0000","last_peered":"2024-04-24T23:59:11.046916+0000","last_clean":"2024-04-24T23:59:11.046916+0000","last_became_active":"2024-04-24T23:59:11.040877+0000","last_became_peered":"2024-04-24T23:59:11.040877+0000","last_unstale":"2024-04-24T23:59:11.046916+0000","last_undegraded":"2024-04-24T23:59:11.046916+0000","last_fullsized":"2024-04-24T23:59:11.046916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:15.733360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040524+0000","last_change":"2024-04-24T23:59:11.037084+0000","last_active":"2024-04-24T23:59:11.040524+0000","last_peered":"2024-04-24T23:59:11.040524+0000","last_clean":"2024-04-24T23:59:11.040524+0000","last_became_active":"2024-04-24T23:59:11.036720+0000","last_became_peered":"2024-04-24T23:59:11.036720+0000","last_unstale":"2024-04-24T23:59:11.040524+0000","last_undegraded":"2024-04-24T23:59:11.040524+0000","last_fullsized":"2024-04-24T23:59:11.040524+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:13.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040033+0000","last_change":"2024-04-24T23:59:11.032047+0000","last_active":"2024-04-24T23:59:11.040033+0000","last_peered":"2024-04-24T23:59:11.040033+0000","last_clean":"2024-04-24T23:59:11.040033+0000","last_became_active":"2024-04-24T23:59:11.031793+0000","last_became_peered":"2024-04-24T23:59:11.031793+0000","last_unstale":"2024-04-24T23:59:11.040033+0000","last_undegraded":"2024-04-24T23:59:11.040033+0000","last_fullsized":"2024-04-24T23:59:11.040033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:15.733360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049858+0000","last_change":"2024-04-24T23:59:11.047504+0000","last_active":"2024-04-24T23:59:11.049858+0000","last_peered":"2024-04-24T23:59:11.049858+0000","last_clean":"2024-04-24T23:59:11.049858+0000","last_became_active":"2024-04-24T23:59:11.047229+0000","last_became_peered":"2024-04-24T23:59:11.047229+0000","last_unstale":"2024-04-24T23:59:11.049858+0000","last_undegraded":"2024-04-24T23:59:11.049858+0000","last_fullsized":"2024-04-24T23:59:11.049858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:45.377965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049865+0000","last_change":"2024-04-24T23:59:11.047852+0000","last_active":"2024-04-24T23:59:11.049865+0000","last_peered":"2024-04-24T23:59:11.049865+0000","last_clean":"2024-04-24T23:59:11.049865+0000","last_became_active":"2024-04-24T23:59:11.047496+0000","last_became_peered":"2024-04-24T23:59:11.047496+0000","last_unstale":"2024-04-24T23:59:11.049865+0000","last_undegraded":"2024-04-24T23:59:11.049865+0000","last_fullsized":"2024-04-24T23:59:11.049865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:57.168655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043313+0000","last_change":"2024-04-24T23:59:11.039729+0000","last_active":"2024-04-24T23:59:11.043313+0000","last_peered":"2024-04-24T23:59:11.043313+0000","last_clean":"2024-04-24T23:59:11.043313+0000","last_became_active":"2024-04-24T23:59:11.039477+0000","last_became_peered":"2024-04-24T23:59:11.039477+0000","last_unstale":"2024-04-24T23:59:11.043313+0000","last_undegraded":"2024-04-24T23:59:11.043313+0000","last_fullsized":"2024-04-24T23:59:11.043313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:08.195488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.045338+0000","last_change":"2024-04-24T23:59:11.028956+0000","last_active":"2024-04-24T23:59:11.045338+0000","last_peered":"2024-04-24T23:59:11.045338+0000","last_clean":"2024-04-24T23:59:11.045338+0000","last_became_active":"2024-04-24T23:59:11.028625+0000","last_became_peered":"2024-04-24T23:59:11.028625+0000","last_unstale":"2024-04-24T23:59:11.045338+0000","last_undegraded":"2024-04-24T23:59:11.045338+0000","last_fullsized":"2024-04-24T23:59:11.045338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:34.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041853+0000","last_change":"2024-04-24T23:59:11.035652+0000","last_active":"2024-04-24T23:59:11.041853+0000","last_peered":"2024-04-24T23:59:11.041853+0000","last_clean":"2024-04-24T23:59:11.041853+0000","last_became_active":"2024-04-24T23:59:11.035343+0000","last_became_peered":"2024-04-24T23:59:11.035343+0000","last_unstale":"2024-04-24T23:59:11.041853+0000","last_undegraded":"2024-04-24T23:59:11.041853+0000","last_fullsized":"2024-04-24T23:59:11.041853+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:04.294928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.033727+0000","last_change":"2024-04-24T23:59:11.030693+0000","last_active":"2024-04-24T23:59:11.033727+0000","last_peered":"2024-04-24T23:59:11.033727+0000","last_clean":"2024-04-24T23:59:11.033727+0000","last_became_active":"2024-04-24T23:59:11.030444+0000","last_became_peered":"2024-04-24T23:59:11.030444+0000","last_unstale":"2024-04-24T23:59:11.033727+0000","last_undegraded":"2024-04-24T23:59:11.033727+0000","last_fullsized":"2024-04-24T23:59:11.033727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:06.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048473+0000","last_change":"2024-04-24T23:59:11.045108+0000","last_active":"2024-04-24T23:59:11.048473+0000","last_peered":"2024-04-24T23:59:11.048473+0000","last_clean":"2024-04-24T23:59:11.048473+0000","last_became_active":"2024-04-24T23:59:11.044658+0000","last_became_peered":"2024-04-24T23:59:11.044658+0000","last_unstale":"2024-04-24T23:59:11.048473+0000","last_undegraded":"2024-04-24T23:59:11.048473+0000","last_fullsized":"2024-04-24T23:59:11.048473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:58.979989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041078+0000","last_change":"2024-04-24T23:59:11.029775+0000","last_active":"2024-04-24T23:59:11.041078+0000","last_peered":"2024-04-24T23:59:11.041078+0000","last_clean":"2024-04-24T23:59:11.041078+0000","last_became_active":"2024-04-24T23:59:11.029511+0000","last_became_peered":"2024-04-24T23:59:11.029511+0000","last_unstale":"2024-04-24T23:59:11.041078+0000","last_undegraded":"2024-04-24T23:59:11.041078+0000","last_fullsized":"2024-04-24T23:59:11.041078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042600+0000","last_change":"2024-04-24T23:59:11.035799+0000","last_active":"2024-04-24T23:59:11.042600+0000","last_peered":"2024-04-24T23:59:11.042600+0000","last_clean":"2024-04-24T23:59:11.042600+0000","last_became_active":"2024-04-24T23:59:11.035487+0000","last_became_peered":"2024-04-24T23:59:11.035487+0000","last_unstale":"2024-04-24T23:59:11.042600+0000","last_undegraded":"2024-04-24T23:59:11.042600+0000","last_fullsized":"2024-04-24T23:59:11.042600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:58.900435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050948+0000","last_change":"2024-04-24T23:59:11.046416+0000","last_active":"2024-04-24T23:59:11.050948+0000","last_peered":"2024-04-24T23:59:11.050948+0000","last_clean":"2024-04-24T23:59:11.050948+0000","last_became_active":"2024-04-24T23:59:11.046138+0000","last_became_peered":"2024-04-24T23:59:11.046138+0000","last_unstale":"2024-04-24T23:59:11.050948+0000","last_undegraded":"2024-04-24T23:59:11.050948+0000","last_fullsized":"2024-04-24T23:59:11.050948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:13.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047173+0000","last_change":"2024-04-24T23:59:11.042605+0000","last_active":"2024-04-24T23:59:11.047173+0000","last_peered":"2024-04-24T23:59:11.047173+0000","last_clean":"2024-04-24T23:59:11.047173+0000","last_became_active":"2024-04-24T23:59:11.042168+0000","last_became_peered":"2024-04-24T23:59:11.042168+0000","last_unstale":"2024-04-24T23:59:11.047173+0000","last_undegraded":"2024-04-24T23:59:11.047173+0000","last_fullsized":"2024-04-24T23:59:11.047173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:13.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.045356+0000","last_change":"2024-04-24T23:59:11.035140+0000","last_active":"2024-04-24T23:59:11.045356+0000","last_peered":"2024-04-24T23:59:11.045356+0000","last_clean":"2024-04-24T23:59:11.045356+0000","last_became_active":"2024-04-24T23:59:11.034830+0000","last_became_peered":"2024-04-24T23:59:11.034830+0000","last_unstale":"2024-04-24T23:59:11.045356+0000","last_undegraded":"2024-04-24T23:59:11.045356+0000","last_fullsized":"2024-04-24T23:59:11.045356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:35.978791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039615+0000","last_change":"2024-04-24T23:59:11.030772+0000","last_active":"2024-04-24T23:59:11.039615+0000","last_peered":"2024-04-24T23:59:11.039615+0000","last_clean":"2024-04-24T23:59:11.039615+0000","last_became_active":"2024-04-24T23:59:11.030494+0000","last_became_peered":"2024-04-24T23:59:11.030494+0000","last_unstale":"2024-04-24T23:59:11.039615+0000","last_undegraded":"2024-04-24T23:59:11.039615+0000","last_fullsized":"2024-04-24T23:59:11.039615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039445+0000","last_change":"2024-04-24T23:59:11.037548+0000","last_active":"2024-04-24T23:59:11.039445+0000","last_peered":"2024-04-24T23:59:11.039445+0000","last_clean":"2024-04-24T23:59:11.039445+0000","last_became_active":"2024-04-24T23:59:11.036716+0000","last_became_peered":"2024-04-24T23:59:11.036716+0000","last_unstale":"2024-04-24T23:59:11.039445+0000","last_undegraded":"2024-04-24T23:59:11.039445+0000","last_fullsized":"2024-04-24T23:59:11.039445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:34.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047933+0000","last_change":"2024-04-24T23:59:11.044996+0000","last_active":"2024-04-24T23:59:11.047933+0000","last_peered":"2024-04-24T23:59:11.047933+0000","last_clean":"2024-04-24T23:59:11.047933+0000","last_became_active":"2024-04-24T23:59:11.044721+0000","last_became_peered":"2024-04-24T23:59:11.044721+0000","last_unstale":"2024-04-24T23:59:11.047933+0000","last_undegraded":"2024-04-24T23:59:11.047933+0000","last_fullsized":"2024-04-24T23:59:11.047933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:03.928528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.032258+0000","last_change":"2024-04-24T23:59:11.029934+0000","last_active":"2024-04-24T23:59:11.032258+0000","last_peered":"2024-04-24T23:59:11.032258+0000","last_clean":"2024-04-24T23:59:11.032258+0000","last_became_active":"2024-04-24T23:59:11.029682+0000","last_became_peered":"2024-04-24T23:59:11.029682+0000","last_unstale":"2024-04-24T23:59:11.032258+0000","last_undegraded":"2024-04-24T23:59:11.032258+0000","last_fullsized":"2024-04-24T23:59:11.032258+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047588+0000","last_change":"2024-04-24T23:59:10.031868+0000","last_active":"2024-04-24T23:59:11.047588+0000","last_peered":"2024-04-24T23:59:11.047588+0000","last_clean":"2024-04-24T23:59:11.047588+0000","last_became_active":"2024-04-24T23:59:10.031602+0000","last_became_peered":"2024-04-24T23:59:10.031602+0000","last_unstale":"2024-04-24T23:59:11.047588+0000","last_undegraded":"2024-04-24T23:59:11.047588+0000","last_fullsized":"2024-04-24T23:59:11.047588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:26.494851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.044617+0000","last_change":"2024-04-24T23:59:11.043135+0000","last_active":"2024-04-24T23:59:11.044617+0000","last_peered":"2024-04-24T23:59:11.044617+0000","last_clean":"2024-04-24T23:59:11.044617+0000","last_became_active":"2024-04-24T23:59:11.042870+0000","last_became_peered":"2024-04-24T23:59:11.042870+0000","last_unstale":"2024-04-24T23:59:11.044617+0000","last_undegraded":"2024-04-24T23:59:11.044617+0000","last_fullsized":"2024-04-24T23:59:11.044617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:00.478646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041742+0000","last_change":"2024-04-24T23:59:10.035978+0000","last_active":"2024-04-24T23:59:11.041742+0000","last_peered":"2024-04-24T23:59:11.041742+0000","last_clean":"2024-04-24T23:59:11.041742+0000","last_became_active":"2024-04-24T23:59:10.035623+0000","last_became_peered":"2024-04-24T23:59:10.035623+0000","last_unstale":"2024-04-24T23:59:11.041742+0000","last_undegraded":"2024-04-24T23:59:11.041742+0000","last_fullsized":"2024-04-24T23:59:11.041742+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:33.084138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041967+0000","last_change":"2024-04-24T23:59:11.036208+0000","last_active":"2024-04-24T23:59:11.041967+0000","last_peered":"2024-04-24T23:59:11.041967+0000","last_clean":"2024-04-24T23:59:11.041967+0000","last_became_active":"2024-04-24T23:59:11.035884+0000","last_became_peered":"2024-04-24T23:59:11.035884+0000","last_unstale":"2024-04-24T23:59:11.041967+0000","last_undegraded":"2024-04-24T23:59:11.041967+0000","last_fullsized":"2024-04-24T23:59:11.041967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049800+0000","last_change":"2024-04-24T23:59:10.039192+0000","last_active":"2024-04-24T23:59:11.049800+0000","last_peered":"2024-04-24T23:59:11.049800+0000","last_clean":"2024-04-24T23:59:11.049800+0000","last_became_active":"2024-04-24T23:59:10.038852+0000","last_became_peered":"2024-04-24T23:59:10.038852+0000","last_unstale":"2024-04-24T23:59:11.049800+0000","last_undegraded":"2024-04-24T23:59:11.049800+0000","last_fullsized":"2024-04-24T23:59:11.049800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:14.731367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050975+0000","last_change":"2024-04-24T23:59:11.047033+0000","last_active":"2024-04-24T23:59:11.050975+0000","last_peered":"2024-04-24T23:59:11.050975+0000","last_clean":"2024-04-24T23:59:11.050975+0000","last_became_active":"2024-04-24T23:59:11.046714+0000","last_became_peered":"2024-04-24T23:59:11.046714+0000","last_unstale":"2024-04-24T23:59:11.050975+0000","last_undegraded":"2024-04-24T23:59:11.050975+0000","last_fullsized":"2024-04-24T23:59:11.050975+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046045+0000","last_change":"2024-04-24T23:59:10.038217+0000","last_active":"2024-04-24T23:59:11.046045+0000","last_peered":"2024-04-24T23:59:11.046045+0000","last_clean":"2024-04-24T23:59:11.046045+0000","last_became_active":"2024-04-24T23:59:10.037635+0000","last_became_peered":"2024-04-24T23:59:10.037635+0000","last_unstale":"2024-04-24T23:59:11.046045+0000","last_undegraded":"2024-04-24T23:59:11.046045+0000","last_fullsized":"2024-04-24T23:59:11.046045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:59.531835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047829+0000","last_change":"2024-04-24T23:59:11.044307+0000","last_active":"2024-04-24T23:59:11.047829+0000","last_peered":"2024-04-24T23:59:11.047829+0000","last_clean":"2024-04-24T23:59:11.047829+0000","last_became_active":"2024-04-24T23:59:11.043805+0000","last_became_peered":"2024-04-24T23:59:11.043805+0000","last_unstale":"2024-04-24T23:59:11.047829+0000","last_undegraded":"2024-04-24T23:59:11.047829+0000","last_fullsized":"2024-04-24T23:59:11.047829+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:58.900435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049963+0000","last_change":"2024-04-24T23:59:10.029038+0000","last_active":"2024-04-24T23:59:11.049963+0000","last_peered":"2024-04-24T23:59:11.049963+0000","last_clean":"2024-04-24T23:59:11.049963+0000","last_became_active":"2024-04-24T23:59:10.028736+0000","last_became_peered":"2024-04-24T23:59:10.028736+0000","last_unstale":"2024-04-24T23:59:11.049963+0000","last_undegraded":"2024-04-24T23:59:11.049963+0000","last_fullsized":"2024-04-24T23:59:11.049963+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:00.490325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048436+0000","last_change":"2024-04-24T23:59:11.043782+0000","last_active":"2024-04-24T23:59:11.048436+0000","last_peered":"2024-04-24T23:59:11.048436+0000","last_clean":"2024-04-24T23:59:11.048436+0000","last_became_active":"2024-04-24T23:59:11.043494+0000","last_became_peered":"2024-04-24T23:59:11.043494+0000","last_unstale":"2024-04-24T23:59:11.048436+0000","last_undegraded":"2024-04-24T23:59:11.048436+0000","last_fullsized":"2024-04-24T23:59:11.048436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049885+0000","last_change":"2024-04-24T23:59:10.031300+0000","last_active":"2024-04-24T23:59:11.049885+0000","last_peered":"2024-04-24T23:59:11.049885+0000","last_clean":"2024-04-24T23:59:11.049885+0000","last_became_active":"2024-04-24T23:59:10.031043+0000","last_became_peered":"2024-04-24T23:59:10.031043+0000","last_unstale":"2024-04-24T23:59:11.049885+0000","last_undegraded":"2024-04-24T23:59:11.049885+0000","last_fullsized":"2024-04-24T23:59:11.049885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.028196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041841+0000","last_change":"2024-04-24T23:59:11.033273+0000","last_active":"2024-04-24T23:59:11.041841+0000","last_peered":"2024-04-24T23:59:11.041841+0000","last_clean":"2024-04-24T23:59:11.041841+0000","last_became_active":"2024-04-24T23:59:11.032976+0000","last_became_peered":"2024-04-24T23:59:11.032976+0000","last_unstale":"2024-04-24T23:59:11.041841+0000","last_undegraded":"2024-04-24T23:59:11.041841+0000","last_fullsized":"2024-04-24T23:59:11.041841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:14.713041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.030707+0000","last_change":"2024-04-24T23:59:10.033511+0000","last_active":"2024-04-24T23:59:11.030707+0000","last_peered":"2024-04-24T23:59:11.030707+0000","last_clean":"2024-04-24T23:59:11.030707+0000","last_became_active":"2024-04-24T23:59:10.033224+0000","last_became_peered":"2024-04-24T23:59:10.033224+0000","last_unstale":"2024-04-24T23:59:11.030707+0000","last_undegraded":"2024-04-24T23:59:11.030707+0000","last_fullsized":"2024-04-24T23:59:11.030707+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041395+0000","last_change":"2024-04-24T23:59:11.033930+0000","last_active":"2024-04-24T23:59:11.041395+0000","last_peered":"2024-04-24T23:59:11.041395+0000","last_clean":"2024-04-24T23:59:11.041395+0000","last_became_active":"2024-04-24T23:59:11.033665+0000","last_became_peered":"2024-04-24T23:59:11.033665+0000","last_unstale":"2024-04-24T23:59:11.041395+0000","last_undegraded":"2024-04-24T23:59:11.041395+0000","last_fullsized":"2024-04-24T23:59:11.041395+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.022107+0000","last_change":"2024-04-24T23:59:10.028386+0000","last_active":"2024-04-24T23:59:11.022107+0000","last_peered":"2024-04-24T23:59:11.022107+0000","last_clean":"2024-04-24T23:59:11.022107+0000","last_became_active":"2024-04-24T23:59:10.027914+0000","last_became_peered":"2024-04-24T23:59:10.027914+0000","last_unstale":"2024-04-24T23:59:11.022107+0000","last_undegraded":"2024-04-24T23:59:11.022107+0000","last_fullsized":"2024-04-24T23:59:11.022107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:35.284706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048523+0000","last_change":"2024-04-24T23:59:11.044428+0000","last_active":"2024-04-24T23:59:11.048523+0000","last_peered":"2024-04-24T23:59:11.048523+0000","last_clean":"2024-04-24T23:59:11.048523+0000","last_became_active":"2024-04-24T23:59:11.043622+0000","last_became_peered":"2024-04-24T23:59:11.043622+0000","last_unstale":"2024-04-24T23:59:11.048523+0000","last_undegraded":"2024-04-24T23:59:11.048523+0000","last_fullsized":"2024-04-24T23:59:11.048523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:38.949687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.034432+0000","last_change":"2024-04-24T23:59:10.034873+0000","last_active":"2024-04-24T23:59:11.034432+0000","last_peered":"2024-04-24T23:59:11.034432+0000","last_clean":"2024-04-24T23:59:11.034432+0000","last_became_active":"2024-04-24T23:59:10.034543+0000","last_became_peered":"2024-04-24T23:59:10.034543+0000","last_unstale":"2024-04-24T23:59:11.034432+0000","last_undegraded":"2024-04-24T23:59:11.034432+0000","last_fullsized":"2024-04-24T23:59:11.034432+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:12.399529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050988+0000","last_change":"2024-04-24T23:59:11.048598+0000","last_active":"2024-04-24T23:59:11.050988+0000","last_peered":"2024-04-24T23:59:11.050988+0000","last_clean":"2024-04-24T23:59:11.050988+0000","last_became_active":"2024-04-24T23:59:11.048252+0000","last_became_peered":"2024-04-24T23:59:11.048252+0000","last_unstale":"2024-04-24T23:59:11.050988+0000","last_undegraded":"2024-04-24T23:59:11.050988+0000","last_fullsized":"2024-04-24T23:59:11.050988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:18.404868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021522+0000","last_change":"2024-04-24T23:59:10.028605+0000","last_active":"2024-04-24T23:59:11.021522+0000","last_peered":"2024-04-24T23:59:11.021522+0000","last_clean":"2024-04-24T23:59:11.021522+0000","last_became_active":"2024-04-24T23:59:10.028297+0000","last_became_peered":"2024-04-24T23:59:10.028297+0000","last_unstale":"2024-04-24T23:59:11.021522+0000","last_undegraded":"2024-04-24T23:59:11.021522+0000","last_fullsized":"2024-04-24T23:59:11.021522+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:59.476653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049706+0000","last_change":"2024-04-24T23:59:11.042293+0000","last_active":"2024-04-24T23:59:11.049706+0000","last_peered":"2024-04-24T23:59:11.049706+0000","last_clean":"2024-04-24T23:59:11.049706+0000","last_became_active":"2024-04-24T23:59:11.042028+0000","last_became_peered":"2024-04-24T23:59:11.042028+0000","last_unstale":"2024-04-24T23:59:11.049706+0000","last_undegraded":"2024-04-24T23:59:11.049706+0000","last_fullsized":"2024-04-24T23:59:11.049706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:38.949687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049231+0000","last_change":"2024-04-24T23:59:10.036821+0000","last_active":"2024-04-24T23:59:11.049231+0000","last_peered":"2024-04-24T23:59:11.049231+0000","last_clean":"2024-04-24T23:59:11.049231+0000","last_became_active":"2024-04-24T23:59:10.036523+0000","last_became_peered":"2024-04-24T23:59:10.036523+0000","last_unstale":"2024-04-24T23:59:11.049231+0000","last_undegraded":"2024-04-24T23:59:11.049231+0000","last_fullsized":"2024-04-24T23:59:11.049231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:05.627877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049301+0000","last_change":"2024-04-24T23:59:11.048171+0000","last_active":"2024-04-24T23:59:11.049301+0000","last_peered":"2024-04-24T23:59:11.049301+0000","last_clean":"2024-04-24T23:59:11.049301+0000","last_became_active":"2024-04-24T23:59:11.047882+0000","last_became_peered":"2024-04-24T23:59:11.047882+0000","last_unstale":"2024-04-24T23:59:11.049301+0000","last_undegraded":"2024-04-24T23:59:11.049301+0000","last_fullsized":"2024-04-24T23:59:11.049301+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:34.170794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.052026+0000","last_change":"2024-04-24T23:59:10.039855+0000","last_active":"2024-04-24T23:59:11.052026+0000","last_peered":"2024-04-24T23:59:11.052026+0000","last_clean":"2024-04-24T23:59:11.052026+0000","last_became_active":"2024-04-24T23:59:10.039645+0000","last_became_peered":"2024-04-24T23:59:10.039645+0000","last_unstale":"2024-04-24T23:59:11.052026+0000","last_undegraded":"2024-04-24T23:59:11.052026+0000","last_fullsized":"2024-04-24T23:59:11.052026+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:47.344139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041155+0000","last_change":"2024-04-24T23:59:11.034592+0000","last_active":"2024-04-24T23:59:11.041155+0000","last_peered":"2024-04-24T23:59:11.041155+0000","last_clean":"2024-04-24T23:59:11.041155+0000","last_became_active":"2024-04-24T23:59:11.034331+0000","last_became_peered":"2024-04-24T23:59:11.034331+0000","last_unstale":"2024-04-24T23:59:11.041155+0000","last_undegraded":"2024-04-24T23:59:11.041155+0000","last_fullsized":"2024-04-24T23:59:11.041155+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047357+0000","last_change":"2024-04-24T23:59:10.029324+0000","last_active":"2024-04-24T23:59:11.047357+0000","last_peered":"2024-04-24T23:59:11.047357+0000","last_clean":"2024-04-24T23:59:11.047357+0000","last_became_active":"2024-04-24T23:59:10.027966+0000","last_became_peered":"2024-04-24T23:59:10.027966+0000","last_unstale":"2024-04-24T23:59:11.047357+0000","last_undegraded":"2024-04-24T23:59:11.047357+0000","last_fullsized":"2024-04-24T23:59:11.047357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:31.859019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043539+0000","last_change":"2024-04-24T23:59:11.035606+0000","last_active":"2024-04-24T23:59:11.043539+0000","last_peered":"2024-04-24T23:59:11.043539+0000","last_clean":"2024-04-24T23:59:11.043539+0000","last_became_active":"2024-04-24T23:59:11.035347+0000","last_became_peered":"2024-04-24T23:59:11.035347+0000","last_unstale":"2024-04-24T23:59:11.043539+0000","last_undegraded":"2024-04-24T23:59:11.043539+0000","last_fullsized":"2024-04-24T23:59:11.043539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.023593+0000","last_change":"2024-04-24T23:59:10.025754+0000","last_active":"2024-04-24T23:59:11.023593+0000","last_peered":"2024-04-24T23:59:11.023593+0000","last_clean":"2024-04-24T23:59:11.023593+0000","last_became_active":"2024-04-24T23:59:10.025494+0000","last_became_peered":"2024-04-24T23:59:10.025494+0000","last_unstale":"2024-04-24T23:59:11.023593+0000","last_undegraded":"2024-04-24T23:59:11.023593+0000","last_fullsized":"2024-04-24T23:59:11.023593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:33.084138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047955+0000","last_change":"2024-04-24T23:59:11.045675+0000","last_active":"2024-04-24T23:59:11.047955+0000","last_peered":"2024-04-24T23:59:11.047955+0000","last_clean":"2024-04-24T23:59:11.047955+0000","last_became_active":"2024-04-24T23:59:11.045387+0000","last_became_peered":"2024-04-24T23:59:11.045387+0000","last_unstale":"2024-04-24T23:59:11.047955+0000","last_undegraded":"2024-04-24T23:59:11.047955+0000","last_fullsized":"2024-04-24T23:59:11.047955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:38.949687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.030840+0000","last_change":"2024-04-24T23:59:10.033656+0000","last_active":"2024-04-24T23:59:11.030840+0000","last_peered":"2024-04-24T23:59:11.030840+0000","last_clean":"2024-04-24T23:59:11.030840+0000","last_became_active":"2024-04-24T23:59:10.033394+0000","last_became_peered":"2024-04-24T23:59:10.033394+0000","last_unstale":"2024-04-24T23:59:11.030840+0000","last_undegraded":"2024-04-24T23:59:11.030840+0000","last_fullsized":"2024-04-24T23:59:11.030840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:05.627877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.044590+0000","last_change":"2024-04-24T23:59:11.043207+0000","last_active":"2024-04-24T23:59:11.044590+0000","last_peered":"2024-04-24T23:59:11.044590+0000","last_clean":"2024-04-24T23:59:11.044590+0000","last_became_active":"2024-04-24T23:59:11.042921+0000","last_became_peered":"2024-04-24T23:59:11.042921+0000","last_unstale":"2024-04-24T23:59:11.044590+0000","last_undegraded":"2024-04-24T23:59:11.044590+0000","last_fullsized":"2024-04-24T23:59:11.044590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:35.978791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050727+0000","last_change":"2024-04-24T23:59:10.039079+0000","last_active":"2024-04-24T23:59:11.050727+0000","last_peered":"2024-04-24T23:59:11.050727+0000","last_clean":"2024-04-24T23:59:11.050727+0000","last_became_active":"2024-04-24T23:59:10.038790+0000","last_became_peered":"2024-04-24T23:59:10.038790+0000","last_unstale":"2024-04-24T23:59:11.050727+0000","last_undegraded":"2024-04-24T23:59:11.050727+0000","last_fullsized":"2024-04-24T23:59:11.050727+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:19.763859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042149+0000","last_change":"2024-04-24T23:59:11.039583+0000","last_active":"2024-04-24T23:59:11.042149+0000","last_peered":"2024-04-24T23:59:11.042149+0000","last_clean":"2024-04-24T23:59:11.042149+0000","last_became_active":"2024-04-24T23:59:11.039313+0000","last_became_peered":"2024-04-24T23:59:11.039313+0000","last_unstale":"2024-04-24T23:59:11.042149+0000","last_undegraded":"2024-04-24T23:59:11.042149+0000","last_fullsized":"2024-04-24T23:59:11.042149+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050236+0000","last_change":"2024-04-24T23:59:10.026367+0000","last_active":"2024-04-24T23:59:11.050236+0000","last_peered":"2024-04-24T23:59:11.050236+0000","last_clean":"2024-04-24T23:59:11.050236+0000","last_became_active":"2024-04-24T23:59:10.026092+0000","last_became_peered":"2024-04-24T23:59:10.026092+0000","last_unstale":"2024-04-24T23:59:11.050236+0000","last_undegraded":"2024-04-24T23:59:11.050236+0000","last_fullsized":"2024-04-24T23:59:11.050236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:59.476653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048410+0000","last_change":"2024-04-24T23:59:11.044314+0000","last_active":"2024-04-24T23:59:11.048410+0000","last_peered":"2024-04-24T23:59:11.048410+0000","last_clean":"2024-04-24T23:59:11.048410+0000","last_became_active":"2024-04-24T23:59:11.043995+0000","last_became_peered":"2024-04-24T23:59:11.043995+0000","last_unstale":"2024-04-24T23:59:11.048410+0000","last_undegraded":"2024-04-24T23:59:11.048410+0000","last_fullsized":"2024-04-24T23:59:11.048410+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:03.928528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046184+0000","last_change":"2024-04-24T23:59:10.027958+0000","last_active":"2024-04-24T23:59:11.046184+0000","last_peered":"2024-04-24T23:59:11.046184+0000","last_clean":"2024-04-24T23:59:11.046184+0000","last_became_active":"2024-04-24T23:59:10.027542+0000","last_became_peered":"2024-04-24T23:59:10.027542+0000","last_unstale":"2024-04-24T23:59:11.046184+0000","last_undegraded":"2024-04-24T23:59:11.046184+0000","last_fullsized":"2024-04-24T23:59:11.046184+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:55.726352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049040+0000","last_change":"2024-04-24T23:59:11.046295+0000","last_active":"2024-04-24T23:59:11.049040+0000","last_peered":"2024-04-24T23:59:11.049040+0000","last_clean":"2024-04-24T23:59:11.049040+0000","last_became_active":"2024-04-24T23:59:11.045413+0000","last_became_peered":"2024-04-24T23:59:11.045413+0000","last_unstale":"2024-04-24T23:59:11.049040+0000","last_undegraded":"2024-04-24T23:59:11.049040+0000","last_fullsized":"2024-04-24T23:59:11.049040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:15.733360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048957+0000","last_change":"2024-04-24T23:59:10.032417+0000","last_active":"2024-04-24T23:59:11.048957+0000","last_peered":"2024-04-24T23:59:11.048957+0000","last_clean":"2024-04-24T23:59:11.048957+0000","last_became_active":"2024-04-24T23:59:10.031940+0000","last_became_peered":"2024-04-24T23:59:10.031940+0000","last_unstale":"2024-04-24T23:59:11.048957+0000","last_undegraded":"2024-04-24T23:59:11.048957+0000","last_fullsized":"2024-04-24T23:59:11.048957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:12.125339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039184+0000","last_change":"2024-04-24T23:59:11.032371+0000","last_active":"2024-04-24T23:59:11.039184+0000","last_peered":"2024-04-24T23:59:11.039184+0000","last_clean":"2024-04-24T23:59:11.039184+0000","last_became_active":"2024-04-24T23:59:11.032126+0000","last_became_peered":"2024-04-24T23:59:11.032126+0000","last_unstale":"2024-04-24T23:59:11.039184+0000","last_undegraded":"2024-04-24T23:59:11.039184+0000","last_fullsized":"2024-04-24T23:59:11.039184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.566855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.022296+0000","last_change":"2024-04-24T23:59:10.032454+0000","last_active":"2024-04-24T23:59:11.022296+0000","last_peered":"2024-04-24T23:59:11.022296+0000","last_clean":"2024-04-24T23:59:11.022296+0000","last_became_active":"2024-04-24T23:59:10.032050+0000","last_became_peered":"2024-04-24T23:59:10.032050+0000","last_unstale":"2024-04-24T23:59:11.022296+0000","last_undegraded":"2024-04-24T23:59:11.022296+0000","last_fullsized":"2024-04-24T23:59:11.022296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:26.494851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049087+0000","last_change":"2024-04-24T23:59:11.043989+0000","last_active":"2024-04-24T23:59:11.049087+0000","last_peered":"2024-04-24T23:59:11.049087+0000","last_clean":"2024-04-24T23:59:11.049087+0000","last_became_active":"2024-04-24T23:59:11.043712+0000","last_became_peered":"2024-04-24T23:59:11.043712+0000","last_unstale":"2024-04-24T23:59:11.049087+0000","last_undegraded":"2024-04-24T23:59:11.049087+0000","last_fullsized":"2024-04-24T23:59:11.049087+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:43.033725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042942+0000","last_change":"2024-04-24T23:59:10.025961+0000","last_active":"2024-04-24T23:59:11.042942+0000","last_peered":"2024-04-24T23:59:11.042942+0000","last_clean":"2024-04-24T23:59:11.042942+0000","last_became_active":"2024-04-24T23:59:10.025663+0000","last_became_peered":"2024-04-24T23:59:10.025663+0000","last_unstale":"2024-04-24T23:59:11.042942+0000","last_undegraded":"2024-04-24T23:59:11.042942+0000","last_fullsized":"2024-04-24T23:59:11.042942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:34.976798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042984+0000","last_change":"2024-04-24T23:59:11.036579+0000","last_active":"2024-04-24T23:59:11.042984+0000","last_peered":"2024-04-24T23:59:11.042984+0000","last_clean":"2024-04-24T23:59:11.042984+0000","last_became_active":"2024-04-24T23:59:11.036268+0000","last_became_peered":"2024-04-24T23:59:11.036268+0000","last_unstale":"2024-04-24T23:59:11.042984+0000","last_undegraded":"2024-04-24T23:59:11.042984+0000","last_fullsized":"2024-04-24T23:59:11.042984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:10.086376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.026498+0000","last_change":"2024-04-24T23:59:10.035596+0000","last_active":"2024-04-24T23:59:11.026498+0000","last_peered":"2024-04-24T23:59:11.026498+0000","last_clean":"2024-04-24T23:59:11.026498+0000","last_became_active":"2024-04-24T23:59:10.035272+0000","last_became_peered":"2024-04-24T23:59:10.035272+0000","last_unstale":"2024-04-24T23:59:11.026498+0000","last_undegraded":"2024-04-24T23:59:11.026498+0000","last_fullsized":"2024-04-24T23:59:11.026498+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.028196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043245+0000","last_change":"2024-04-24T23:59:11.037157+0000","last_active":"2024-04-24T23:59:11.043245+0000","last_peered":"2024-04-24T23:59:11.043245+0000","last_clean":"2024-04-24T23:59:11.043245+0000","last_became_active":"2024-04-24T23:59:11.036373+0000","last_became_peered":"2024-04-24T23:59:11.036373+0000","last_unstale":"2024-04-24T23:59:11.043245+0000","last_undegraded":"2024-04-24T23:59:11.043245+0000","last_fullsized":"2024-04-24T23:59:11.043245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:40.249109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021438+0000","last_change":"2024-04-24T23:59:10.035561+0000","last_active":"2024-04-24T23:59:11.021438+0000","last_peered":"2024-04-24T23:59:11.021438+0000","last_clean":"2024-04-24T23:59:11.021438+0000","last_became_active":"2024-04-24T23:59:10.035270+0000","last_became_peered":"2024-04-24T23:59:10.035270+0000","last_unstale":"2024-04-24T23:59:11.021438+0000","last_undegraded":"2024-04-24T23:59:11.021438+0000","last_fullsized":"2024-04-24T23:59:11.021438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:37.067752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039954+0000","last_change":"2024-04-24T23:59:11.031210+0000","last_active":"2024-04-24T23:59:11.039954+0000","last_peered":"2024-04-24T23:59:11.039954+0000","last_clean":"2024-04-24T23:59:11.039954+0000","last_became_active":"2024-04-24T23:59:11.030701+0000","last_became_peered":"2024-04-24T23:59:11.030701+0000","last_unstale":"2024-04-24T23:59:11.039954+0000","last_undegraded":"2024-04-24T23:59:11.039954+0000","last_fullsized":"2024-04-24T23:59:11.039954+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.032587+0000","last_change":"2024-04-24T23:59:10.036837+0000","last_active":"2024-04-24T23:59:11.032587+0000","last_peered":"2024-04-24T23:59:11.032587+0000","last_clean":"2024-04-24T23:59:11.032587+0000","last_became_active":"2024-04-24T23:59:10.036539+0000","last_became_peered":"2024-04-24T23:59:10.036539+0000","last_unstale":"2024-04-24T23:59:11.032587+0000","last_undegraded":"2024-04-24T23:59:11.032587+0000","last_fullsized":"2024-04-24T23:59:11.032587+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043264+0000","last_change":"2024-04-24T23:59:11.040393+0000","last_active":"2024-04-24T23:59:11.043264+0000","last_peered":"2024-04-24T23:59:11.043264+0000","last_clean":"2024-04-24T23:59:11.043264+0000","last_became_active":"2024-04-24T23:59:11.040119+0000","last_became_peered":"2024-04-24T23:59:11.040119+0000","last_unstale":"2024-04-24T23:59:11.043264+0000","last_undegraded":"2024-04-24T23:59:11.043264+0000","last_fullsized":"2024-04-24T23:59:11.043264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:40.343708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049207+0000","last_change":"2024-04-24T23:59:10.032308+0000","last_active":"2024-04-24T23:59:11.049207+0000","last_peered":"2024-04-24T23:59:11.049207+0000","last_clean":"2024-04-24T23:59:11.049207+0000","last_became_active":"2024-04-24T23:59:10.031672+0000","last_became_peered":"2024-04-24T23:59:10.031672+0000","last_unstale":"2024-04-24T23:59:11.049207+0000","last_undegraded":"2024-04-24T23:59:11.049207+0000","last_fullsized":"2024-04-24T23:59:11.049207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:34.976798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049248+0000","last_change":"2024-04-24T23:59:11.048263+0000","last_active":"2024-04-24T23:59:11.049248+0000","last_peered":"2024-04-24T23:59:11.049248+0000","last_clean":"2024-04-24T23:59:11.049248+0000","last_became_active":"2024-04-24T23:59:11.047935+0000","last_became_peered":"2024-04-24T23:59:11.047935+0000","last_unstale":"2024-04-24T23:59:11.049248+0000","last_undegraded":"2024-04-24T23:59:11.049248+0000","last_fullsized":"2024-04-24T23:59:11.049248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:45.377965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021400+0000","last_change":"2024-04-24T23:59:10.026005+0000","last_active":"2024-04-24T23:59:11.021400+0000","last_peered":"2024-04-24T23:59:11.021400+0000","last_clean":"2024-04-24T23:59:11.021400+0000","last_became_active":"2024-04-24T23:59:10.025634+0000","last_became_peered":"2024-04-24T23:59:10.025634+0000","last_unstale":"2024-04-24T23:59:11.021400+0000","last_undegraded":"2024-04-24T23:59:11.021400+0000","last_fullsized":"2024-04-24T23:59:11.021400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:13.711048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.031635+0000","last_change":"2024-04-24T23:59:11.025721+0000","last_active":"2024-04-24T23:59:11.031635+0000","last_peered":"2024-04-24T23:59:11.031635+0000","last_clean":"2024-04-24T23:59:11.031635+0000","last_became_active":"2024-04-24T23:59:11.025412+0000","last_became_peered":"2024-04-24T23:59:11.025412+0000","last_unstale":"2024-04-24T23:59:11.031635+0000","last_undegraded":"2024-04-24T23:59:11.031635+0000","last_fullsized":"2024-04-24T23:59:11.031635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.024525+0000","last_change":"2024-04-24T23:59:10.034763+0000","last_active":"2024-04-24T23:59:11.024525+0000","last_peered":"2024-04-24T23:59:11.024525+0000","last_clean":"2024-04-24T23:59:11.024525+0000","last_became_active":"2024-04-24T23:59:10.034489+0000","last_became_peered":"2024-04-24T23:59:10.034489+0000","last_unstale":"2024-04-24T23:59:11.024525+0000","last_undegraded":"2024-04-24T23:59:11.024525+0000","last_fullsized":"2024-04-24T23:59:11.024525+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:03.292935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039374+0000","last_change":"2024-04-24T23:59:11.035773+0000","last_active":"2024-04-24T23:59:11.039374+0000","last_peered":"2024-04-24T23:59:11.039374+0000","last_clean":"2024-04-24T23:59:11.039374+0000","last_became_active":"2024-04-24T23:59:11.035492+0000","last_became_peered":"2024-04-24T23:59:11.035492+0000","last_unstale":"2024-04-24T23:59:11.039374+0000","last_undegraded":"2024-04-24T23:59:11.039374+0000","last_fullsized":"2024-04-24T23:59:11.039374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:32.861012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.022066+0000","last_change":"2024-04-24T23:59:10.029707+0000","last_active":"2024-04-24T23:59:11.022066+0000","last_peered":"2024-04-24T23:59:11.022066+0000","last_clean":"2024-04-24T23:59:11.022066+0000","last_became_active":"2024-04-24T23:59:10.029460+0000","last_became_peered":"2024-04-24T23:59:10.029460+0000","last_unstale":"2024-04-24T23:59:11.022066+0000","last_undegraded":"2024-04-24T23:59:11.022066+0000","last_fullsized":"2024-04-24T23:59:11.022066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:03.292935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.033218+0000","last_change":"2024-04-24T23:59:11.030686+0000","last_active":"2024-04-24T23:59:11.033218+0000","last_peered":"2024-04-24T23:59:11.033218+0000","last_clean":"2024-04-24T23:59:11.033218+0000","last_became_active":"2024-04-24T23:59:11.030403+0000","last_became_peered":"2024-04-24T23:59:11.030403+0000","last_unstale":"2024-04-24T23:59:11.033218+0000","last_undegraded":"2024-04-24T23:59:11.033218+0000","last_fullsized":"2024-04-24T23:59:11.033218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.023736+0000","last_change":"2024-04-24T23:59:10.030771+0000","last_active":"2024-04-24T23:59:11.023736+0000","last_peered":"2024-04-24T23:59:11.023736+0000","last_clean":"2024-04-24T23:59:11.023736+0000","last_became_active":"2024-04-24T23:59:10.030513+0000","last_became_peered":"2024-04-24T23:59:10.030513+0000","last_unstale":"2024-04-24T23:59:11.023736+0000","last_undegraded":"2024-04-24T23:59:11.023736+0000","last_fullsized":"2024-04-24T23:59:11.023736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:13.711048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049873+0000","last_change":"2024-04-24T23:59:11.047902+0000","last_active":"2024-04-24T23:59:11.049873+0000","last_peered":"2024-04-24T23:59:11.049873+0000","last_clean":"2024-04-24T23:59:11.049873+0000","last_became_active":"2024-04-24T23:59:11.047429+0000","last_became_peered":"2024-04-24T23:59:11.047429+0000","last_unstale":"2024-04-24T23:59:11.049873+0000","last_undegraded":"2024-04-24T23:59:11.049873+0000","last_fullsized":"2024-04-24T23:59:11.049873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:04.507679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021608+0000","last_change":"2024-04-24T23:59:10.030560+0000","last_active":"2024-04-24T23:59:11.021608+0000","last_peered":"2024-04-24T23:59:11.021608+0000","last_clean":"2024-04-24T23:59:11.021608+0000","last_became_active":"2024-04-24T23:59:10.030266+0000","last_became_peered":"2024-04-24T23:59:10.030266+0000","last_unstale":"2024-04-24T23:59:11.021608+0000","last_undegraded":"2024-04-24T23:59:11.021608+0000","last_fullsized":"2024-04-24T23:59:11.021608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:03.853095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041815+0000","last_change":"2024-04-24T23:59:11.036733+0000","last_active":"2024-04-24T23:59:11.041815+0000","last_peered":"2024-04-24T23:59:11.041815+0000","last_clean":"2024-04-24T23:59:11.041815+0000","last_became_active":"2024-04-24T23:59:11.036342+0000","last_became_peered":"2024-04-24T23:59:11.036342+0000","last_unstale":"2024-04-24T23:59:11.041815+0000","last_undegraded":"2024-04-24T23:59:11.041815+0000","last_fullsized":"2024-04-24T23:59:11.041815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.030000+0000","last_change":"2024-04-24T23:59:10.034120+0000","last_active":"2024-04-24T23:59:11.030000+0000","last_peered":"2024-04-24T23:59:11.030000+0000","last_clean":"2024-04-24T23:59:11.030000+0000","last_became_active":"2024-04-24T23:59:10.033836+0000","last_became_peered":"2024-04-24T23:59:10.033836+0000","last_unstale":"2024-04-24T23:59:11.030000+0000","last_undegraded":"2024-04-24T23:59:11.030000+0000","last_fullsized":"2024-04-24T23:59:11.030000+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:31.467243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039948+0000","last_change":"2024-04-24T23:59:11.031033+0000","last_active":"2024-04-24T23:59:11.039948+0000","last_peered":"2024-04-24T23:59:11.039948+0000","last_clean":"2024-04-24T23:59:11.039948+0000","last_became_active":"2024-04-24T23:59:11.030686+0000","last_became_peered":"2024-04-24T23:59:11.030686+0000","last_unstale":"2024-04-24T23:59:11.039948+0000","last_undegraded":"2024-04-24T23:59:11.039948+0000","last_fullsized":"2024-04-24T23:59:11.039948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.026020+0000","last_change":"2024-04-24T23:59:10.034872+0000","last_active":"2024-04-24T23:59:11.026020+0000","last_peered":"2024-04-24T23:59:11.026020+0000","last_clean":"2024-04-24T23:59:11.026020+0000","last_became_active":"2024-04-24T23:59:10.034565+0000","last_became_peered":"2024-04-24T23:59:10.034565+0000","last_unstale":"2024-04-24T23:59:11.026020+0000","last_undegraded":"2024-04-24T23:59:11.026020+0000","last_fullsized":"2024-04-24T23:59:11.026020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:00.490325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.027904+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:59:11.027904+0000","last_peered":"2024-04-24T23:59:11.027904+0000","last_clean":"2024-04-24T23:59:11.027904+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:59:11.027904+0000","last_undegraded":"2024-04-24T23:59:11.027904+0000","last_fullsized":"2024-04-24T23:59:11.027904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":929792,"data_stored":717876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":134353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:59:13.990 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:59:14.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.122+0000 7fd4f8d42640 1 Processor -- start 2024-04-24T23:59:14.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.122+0000 7fd4f8d42640 1 -- start start 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f8d42640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd4f4144910 0x7fd4f40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f8d42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f40b5240 0x7fd4f406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f8d42640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f4144cf0 con 0x7fd4f40b5680 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f8d42640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f40b4510 con 0x7fd4f4144910 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f8d42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f40b2f30 con 0x7fd4f40b5240 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd4f4144910 0x7fd4f40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f2d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd4f40b5680 0x7fd4f40cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40306/0 (socket says 172.21.15.113:40306) 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f2d9d640 1 -- 172.21.15.113:0/4207188462 learned_addr learned my addr 172.21.15.113:0/4207188462 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f259c640 1 -- 172.21.15.113:0/4207188462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f40b5240 msgr2=0x7fd4f406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f1d9b640 1 --2- 172.21.15.113:0/4207188462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f40b5240 0x7fd4f406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f159a640 1 -- 172.21.15.113:0/4207188462 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125898485 0 0) 0x7fd4f4144cf0 con 0x7fd4f40b5680 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f259c640 1 --2- 172.21.15.113:0/4207188462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f40b5240 0x7fd4f406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f259c640 1 -- 172.21.15.113:0/4207188462 >> v1:172.21.15.113:6789/0 conn(0x7fd4f40b5680 legacy=0x7fd4f40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.123+0000 7fd4f259c640 1 -- 172.21.15.113:0/4207188462 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f40b3210 con 0x7fd4f4144910 2024-04-24T23:59:14.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.124+0000 7fd4f259c640 1 --2- 172.21.15.113:0/4207188462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f4144910 0x7fd4f40b3fd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d90 tx=0x7fd4e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873ebe4eb3878476 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.124+0000 7fd4f159a640 1 -- 172.21.15.113:0/4207188462 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4007740 con 0x7fd4f4144910 2024-04-24T23:59:14.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.124+0000 7fd4f159a640 1 -- 172.21.15.113:0/4207188462 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e40078e0 con 0x7fd4f4144910 2024-04-24T23:59:14.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.124+0000 7fd4f159a640 1 -- 172.21.15.113:0/4207188462 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4007cb0 con 0x7fd4f4144910 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 -- 172.21.15.113:0/4207188462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f4144910 msgr2=0x7fd4f40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/4207188462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f4144910 0x7fd4f40b3fd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d90 tx=0x7fd4e402a660 comp rx=0 tx=0).stop 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 -- 172.21.15.113:0/4207188462 shutdown_connections 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/4207188462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f40b5240 0x7fd4f406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/4207188462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f4144910 0x7fd4f40b3fd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 -- 172.21.15.113:0/4207188462 >> 172.21.15.113:0/4207188462 conn(0x7fd4f40b1a50 msgr2=0x7fd4f40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 -- 172.21.15.113:0/4207188462 shutdown_connections 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.125+0000 7fd4f8d42640 1 -- 172.21.15.113:0/4207188462 wait complete. 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.126+0000 7fd4f8d42640 1 Processor -- start 2024-04-24T23:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.126+0000 7fd4f8d42640 1 -- start start 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f8d42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f40b5240 0x7fd4f41502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f8d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f40b5680 0x7fd4f413f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f8d42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f4144910 0x7fd4f40abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f8d42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f40b4da0 con 0x7fd4f40b5680 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f8d42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f40b37d0 con 0x7fd4f40b5240 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f8d42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f40b4f20 con 0x7fd4f4144910 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f40b5680 0x7fd4f413f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f40b5680 0x7fd4f413f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53528/0 (socket says 172.21.15.113:53528) 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 -- 172.21.15.113:0/669270743 learned_addr learned my addr 172.21.15.113:0/669270743 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 -- 172.21.15.113:0/669270743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f4144910 msgr2=0x7fd4f40abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f4144910 0x7fd4f40abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 -- 172.21.15.113:0/669270743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f40b5240 msgr2=0x7fd4f41502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f259c640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f40b5240 0x7fd4f41502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f40b5240 0x7fd4f41502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.127+0000 7fd4f1d9b640 1 -- 172.21.15.113:0/669270743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f40ac670 con 0x7fd4f40b5680 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.128+0000 7fd4f1d9b640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f40b5680 0x7fd4f413f390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c7f0 tx=0x7fd4e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.128+0000 7fd4e2ffd640 1 -- 172.21.15.113:0/669270743 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8015070 con 0x7fd4f40b5680 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.128+0000 7fd4e2ffd640 1 -- 172.21.15.113:0/669270743 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e8004220 con 0x7fd4f40b5680 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.128+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f40ac950 con 0x7fd4f40b5680 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.128+0000 7fd4e2ffd640 1 -- 172.21.15.113:0/669270743 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8007620 con 0x7fd4f40b5680 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.128+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f40acf10 con 0x7fd4f40b5680 2024-04-24T23:59:14.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.129+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4144170 con 0x7fd4f40b5680 2024-04-24T23:59:14.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.129+0000 7fd4e2ffd640 1 -- 172.21.15.113:0/669270743 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4e80077c0 con 0x7fd4f40b5680 2024-04-24T23:59:14.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.130+0000 7fd4e2ffd640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4d406be30 0x7fd4d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.130+0000 7fd4f259c640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4d406be30 0x7fd4d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.130+0000 7fd4e2ffd640 1 -- 172.21.15.113:0/669270743 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd4e808acc0 con 0x7fd4f40b5680 2024-04-24T23:59:14.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.130+0000 7fd4f259c640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4d406be30 0x7fd4d406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4034000 tx=0x7fd4e402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.134+0000 7fd4e2ffd640 1 -- 172.21.15.113:0/669270743 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd4f4144170 con 0x7fd4f40b5680 2024-04-24T23:59:14.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.301+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4f40691a0 con 0x7fd4d406be30 2024-04-24T23:59:14.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.307+0000 7fd4e2ffd640 1 -- 172.21.15.113:0/669270743 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251878 (secure 0 0 0) 0x7fd4f40691a0 con 0x7fd4d406be30 2024-04-24T23:59:14.310 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:14.311 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-24T23:59:14.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4d406be30 msgr2=0x7fd4d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4d406be30 0x7fd4d406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4034000 tx=0x7fd4e402f000 comp rx=0 tx=0).stop 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f40b5680 msgr2=0x7fd4f413f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f40b5680 0x7fd4f413f390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c7f0 tx=0x7fd4e800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 shutdown_connections 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4d406be30 0x7fd4d406e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4f4144910 0x7fd4f40abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4f40b5680 0x7fd4f413f390 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 --2- 172.21.15.113:0/669270743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f40b5240 0x7fd4f41502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 >> 172.21.15.113:0/669270743 conn(0x7fd4f40b1a50 msgr2=0x7fd4f40b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 shutdown_connections 2024-04-24T23:59:14.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.310+0000 7fd4f8d42640 1 -- 172.21.15.113:0/669270743 wait complete. 2024-04-24T23:59:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-04-24T23:59:13.121805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236620,"kb_used_data":3800,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738100,"statfs":{"total":773094113280,"available":772851814400,"internally_reserved":0,"allocated":3891200,"data_stored":2207645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.112370"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040722+0000","last_change":"2024-04-24T23:59:11.039494+0000","last_active":"2024-04-24T23:59:11.040722+0000","last_peered":"2024-04-24T23:59:11.040722+0000","last_clean":"2024-04-24T23:59:11.040722+0000","last_became_active":"2024-04-24T23:59:11.039116+0000","last_became_peered":"2024-04-24T23:59:11.039116+0000","last_unstale":"2024-04-24T23:59:11.040722+0000","last_undegraded":"2024-04-24T23:59:11.040722+0000","last_fullsized":"2024-04-24T23:59:11.040722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:14.713041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049679+0000","last_change":"2024-04-24T23:59:11.042430+0000","last_active":"2024-04-24T23:59:11.049679+0000","last_peered":"2024-04-24T23:59:11.049679+0000","last_clean":"2024-04-24T23:59:11.049679+0000","last_became_active":"2024-04-24T23:59:11.042192+0000","last_became_peered":"2024-04-24T23:59:11.042192+0000","last_unstale":"2024-04-24T23:59:11.049679+0000","last_undegraded":"2024-04-24T23:59:11.049679+0000","last_fullsized":"2024-04-24T23:59:11.049679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:58.979989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048419+0000","last_change":"2024-04-24T23:59:11.043364+0000","last_active":"2024-04-24T23:59:11.048419+0000","last_peered":"2024-04-24T23:59:11.048419+0000","last_clean":"2024-04-24T23:59:11.048419+0000","last_became_active":"2024-04-24T23:59:11.043056+0000","last_became_peered":"2024-04-24T23:59:11.043056+0000","last_unstale":"2024-04-24T23:59:11.048419+0000","last_undegraded":"2024-04-24T23:59:11.048419+0000","last_fullsized":"2024-04-24T23:59:11.048419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:58.900435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049016+0000","last_change":"2024-04-24T23:59:11.046150+0000","last_active":"2024-04-24T23:59:11.049016+0000","last_peered":"2024-04-24T23:59:11.049016+0000","last_clean":"2024-04-24T23:59:11.049016+0000","last_became_active":"2024-04-24T23:59:11.045715+0000","last_became_peered":"2024-04-24T23:59:11.045715+0000","last_unstale":"2024-04-24T23:59:11.049016+0000","last_undegraded":"2024-04-24T23:59:11.049016+0000","last_fullsized":"2024-04-24T23:59:11.049016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:48.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043266+0000","last_change":"2024-04-24T23:59:11.037172+0000","last_active":"2024-04-24T23:59:11.043266+0000","last_peered":"2024-04-24T23:59:11.043266+0000","last_clean":"2024-04-24T23:59:11.043266+0000","last_became_active":"2024-04-24T23:59:11.036894+0000","last_became_peered":"2024-04-24T23:59:11.036894+0000","last_unstale":"2024-04-24T23:59:11.043266+0000","last_undegraded":"2024-04-24T23:59:11.043266+0000","last_fullsized":"2024-04-24T23:59:11.043266+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:40:16.435100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048816+0000","last_change":"2024-04-24T23:59:11.040247+0000","last_active":"2024-04-24T23:59:11.048816+0000","last_peered":"2024-04-24T23:59:11.048816+0000","last_clean":"2024-04-24T23:59:11.048816+0000","last_became_active":"2024-04-24T23:59:11.039965+0000","last_became_peered":"2024-04-24T23:59:11.039965+0000","last_unstale":"2024-04-24T23:59:11.048816+0000","last_undegraded":"2024-04-24T23:59:11.048816+0000","last_fullsized":"2024-04-24T23:59:11.048816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:03.928528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039974+0000","last_change":"2024-04-24T23:59:11.031683+0000","last_active":"2024-04-24T23:59:11.039974+0000","last_peered":"2024-04-24T23:59:11.039974+0000","last_clean":"2024-04-24T23:59:11.039974+0000","last_became_active":"2024-04-24T23:59:11.031380+0000","last_became_peered":"2024-04-24T23:59:11.031380+0000","last_unstale":"2024-04-24T23:59:11.039974+0000","last_undegraded":"2024-04-24T23:59:11.039974+0000","last_fullsized":"2024-04-24T23:59:11.039974+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:06.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040004+0000","last_change":"2024-04-24T23:59:11.031954+0000","last_active":"2024-04-24T23:59:11.040004+0000","last_peered":"2024-04-24T23:59:11.040004+0000","last_clean":"2024-04-24T23:59:11.040004+0000","last_became_active":"2024-04-24T23:59:11.031656+0000","last_became_peered":"2024-04-24T23:59:11.031656+0000","last_unstale":"2024-04-24T23:59:11.040004+0000","last_undegraded":"2024-04-24T23:59:11.040004+0000","last_fullsized":"2024-04-24T23:59:11.040004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:48.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041759+0000","last_change":"2024-04-24T23:59:11.034961+0000","last_active":"2024-04-24T23:59:11.041759+0000","last_peered":"2024-04-24T23:59:11.041759+0000","last_clean":"2024-04-24T23:59:11.041759+0000","last_became_active":"2024-04-24T23:59:11.034633+0000","last_became_peered":"2024-04-24T23:59:11.034633+0000","last_unstale":"2024-04-24T23:59:11.041759+0000","last_undegraded":"2024-04-24T23:59:11.041759+0000","last_fullsized":"2024-04-24T23:59:11.041759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:00.478646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046579+0000","last_change":"2024-04-24T23:59:11.038976+0000","last_active":"2024-04-24T23:59:11.046579+0000","last_peered":"2024-04-24T23:59:11.046579+0000","last_clean":"2024-04-24T23:59:11.046579+0000","last_became_active":"2024-04-24T23:59:11.038674+0000","last_became_peered":"2024-04-24T23:59:11.038674+0000","last_unstale":"2024-04-24T23:59:11.046579+0000","last_undegraded":"2024-04-24T23:59:11.046579+0000","last_fullsized":"2024-04-24T23:59:11.046579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:06.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043445+0000","last_change":"2024-04-24T23:59:11.041937+0000","last_active":"2024-04-24T23:59:11.043445+0000","last_peered":"2024-04-24T23:59:11.043445+0000","last_clean":"2024-04-24T23:59:11.043445+0000","last_became_active":"2024-04-24T23:59:11.041670+0000","last_became_peered":"2024-04-24T23:59:11.041670+0000","last_unstale":"2024-04-24T23:59:11.043445+0000","last_undegraded":"2024-04-24T23:59:11.043445+0000","last_fullsized":"2024-04-24T23:59:11.043445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039788+0000","last_change":"2024-04-24T23:59:11.037582+0000","last_active":"2024-04-24T23:59:11.039788+0000","last_peered":"2024-04-24T23:59:11.039788+0000","last_clean":"2024-04-24T23:59:11.039788+0000","last_became_active":"2024-04-24T23:59:11.037283+0000","last_became_peered":"2024-04-24T23:59:11.037283+0000","last_unstale":"2024-04-24T23:59:11.039788+0000","last_undegraded":"2024-04-24T23:59:11.039788+0000","last_fullsized":"2024-04-24T23:59:11.039788+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:48.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049631+0000","last_change":"2024-04-24T23:59:11.045621+0000","last_active":"2024-04-24T23:59:11.049631+0000","last_peered":"2024-04-24T23:59:11.049631+0000","last_clean":"2024-04-24T23:59:11.049631+0000","last_became_active":"2024-04-24T23:59:11.045370+0000","last_became_peered":"2024-04-24T23:59:11.045370+0000","last_unstale":"2024-04-24T23:59:11.049631+0000","last_undegraded":"2024-04-24T23:59:11.049631+0000","last_fullsized":"2024-04-24T23:59:11.049631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:38.069745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049799+0000","last_change":"2024-04-24T23:59:11.045863+0000","last_active":"2024-04-24T23:59:11.049799+0000","last_peered":"2024-04-24T23:59:11.049799+0000","last_clean":"2024-04-24T23:59:11.049799+0000","last_became_active":"2024-04-24T23:59:11.045093+0000","last_became_peered":"2024-04-24T23:59:11.045093+0000","last_unstale":"2024-04-24T23:59:11.049799+0000","last_undegraded":"2024-04-24T23:59:11.049799+0000","last_fullsized":"2024-04-24T23:59:11.049799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:34.170794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046916+0000","last_change":"2024-04-24T23:59:11.041196+0000","last_active":"2024-04-24T23:59:11.046916+0000","last_peered":"2024-04-24T23:59:11.046916+0000","last_clean":"2024-04-24T23:59:11.046916+0000","last_became_active":"2024-04-24T23:59:11.040877+0000","last_became_peered":"2024-04-24T23:59:11.040877+0000","last_unstale":"2024-04-24T23:59:11.046916+0000","last_undegraded":"2024-04-24T23:59:11.046916+0000","last_fullsized":"2024-04-24T23:59:11.046916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:15.733360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040524+0000","last_change":"2024-04-24T23:59:11.037084+0000","last_active":"2024-04-24T23:59:11.040524+0000","last_peered":"2024-04-24T23:59:11.040524+0000","last_clean":"2024-04-24T23:59:11.040524+0000","last_became_active":"2024-04-24T23:59:11.036720+0000","last_became_peered":"2024-04-24T23:59:11.036720+0000","last_unstale":"2024-04-24T23:59:11.040524+0000","last_undegraded":"2024-04-24T23:59:11.040524+0000","last_fullsized":"2024-04-24T23:59:11.040524+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:13.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.040033+0000","last_change":"2024-04-24T23:59:11.032047+0000","last_active":"2024-04-24T23:59:11.040033+0000","last_peered":"2024-04-24T23:59:11.040033+0000","last_clean":"2024-04-24T23:59:11.040033+0000","last_became_active":"2024-04-24T23:59:11.031793+0000","last_became_peered":"2024-04-24T23:59:11.031793+0000","last_unstale":"2024-04-24T23:59:11.040033+0000","last_undegraded":"2024-04-24T23:59:11.040033+0000","last_fullsized":"2024-04-24T23:59:11.040033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:15.733360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049858+0000","last_change":"2024-04-24T23:59:11.047504+0000","last_active":"2024-04-24T23:59:11.049858+0000","last_peered":"2024-04-24T23:59:11.049858+0000","last_clean":"2024-04-24T23:59:11.049858+0000","last_became_active":"2024-04-24T23:59:11.047229+0000","last_became_peered":"2024-04-24T23:59:11.047229+0000","last_unstale":"2024-04-24T23:59:11.049858+0000","last_undegraded":"2024-04-24T23:59:11.049858+0000","last_fullsized":"2024-04-24T23:59:11.049858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:45.377965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049865+0000","last_change":"2024-04-24T23:59:11.047852+0000","last_active":"2024-04-24T23:59:11.049865+0000","last_peered":"2024-04-24T23:59:11.049865+0000","last_clean":"2024-04-24T23:59:11.049865+0000","last_became_active":"2024-04-24T23:59:11.047496+0000","last_became_peered":"2024-04-24T23:59:11.047496+0000","last_unstale":"2024-04-24T23:59:11.049865+0000","last_undegraded":"2024-04-24T23:59:11.049865+0000","last_fullsized":"2024-04-24T23:59:11.049865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:57.168655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043313+0000","last_change":"2024-04-24T23:59:11.039729+0000","last_active":"2024-04-24T23:59:11.043313+0000","last_peered":"2024-04-24T23:59:11.043313+0000","last_clean":"2024-04-24T23:59:11.043313+0000","last_became_active":"2024-04-24T23:59:11.039477+0000","last_became_peered":"2024-04-24T23:59:11.039477+0000","last_unstale":"2024-04-24T23:59:11.043313+0000","last_undegraded":"2024-04-24T23:59:11.043313+0000","last_fullsized":"2024-04-24T23:59:11.043313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:08.195488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.045338+0000","last_change":"2024-04-24T23:59:11.028956+0000","last_active":"2024-04-24T23:59:11.045338+0000","last_peered":"2024-04-24T23:59:11.045338+0000","last_clean":"2024-04-24T23:59:11.045338+0000","last_became_active":"2024-04-24T23:59:11.028625+0000","last_became_peered":"2024-04-24T23:59:11.028625+0000","last_unstale":"2024-04-24T23:59:11.045338+0000","last_undegraded":"2024-04-24T23:59:11.045338+0000","last_fullsized":"2024-04-24T23:59:11.045338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:34.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041853+0000","last_change":"2024-04-24T23:59:11.035652+0000","last_active":"2024-04-24T23:59:11.041853+0000","last_peered":"2024-04-24T23:59:11.041853+0000","last_clean":"2024-04-24T23:59:11.041853+0000","last_became_active":"2024-04-24T23:59:11.035343+0000","last_became_peered":"2024-04-24T23:59:11.035343+0000","last_unstale":"2024-04-24T23:59:11.041853+0000","last_undegraded":"2024-04-24T23:59:11.041853+0000","last_fullsized":"2024-04-24T23:59:11.041853+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:04.294928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.033727+0000","last_change":"2024-04-24T23:59:11.030693+0000","last_active":"2024-04-24T23:59:11.033727+0000","last_peered":"2024-04-24T23:59:11.033727+0000","last_clean":"2024-04-24T23:59:11.033727+0000","last_became_active":"2024-04-24T23:59:11.030444+0000","last_became_peered":"2024-04-24T23:59:11.030444+0000","last_unstale":"2024-04-24T23:59:11.033727+0000","last_undegraded":"2024-04-24T23:59:11.033727+0000","last_fullsized":"2024-04-24T23:59:11.033727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:06.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048473+0000","last_change":"2024-04-24T23:59:11.045108+0000","last_active":"2024-04-24T23:59:11.048473+0000","last_peered":"2024-04-24T23:59:11.048473+0000","last_clean":"2024-04-24T23:59:11.048473+0000","last_became_active":"2024-04-24T23:59:11.044658+0000","last_became_peered":"2024-04-24T23:59:11.044658+0000","last_unstale":"2024-04-24T23:59:11.048473+0000","last_undegraded":"2024-04-24T23:59:11.048473+0000","last_fullsized":"2024-04-24T23:59:11.048473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:58.979989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041078+0000","last_change":"2024-04-24T23:59:11.029775+0000","last_active":"2024-04-24T23:59:11.041078+0000","last_peered":"2024-04-24T23:59:11.041078+0000","last_clean":"2024-04-24T23:59:11.041078+0000","last_became_active":"2024-04-24T23:59:11.029511+0000","last_became_peered":"2024-04-24T23:59:11.029511+0000","last_unstale":"2024-04-24T23:59:11.041078+0000","last_undegraded":"2024-04-24T23:59:11.041078+0000","last_fullsized":"2024-04-24T23:59:11.041078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042600+0000","last_change":"2024-04-24T23:59:11.035799+0000","last_active":"2024-04-24T23:59:11.042600+0000","last_peered":"2024-04-24T23:59:11.042600+0000","last_clean":"2024-04-24T23:59:11.042600+0000","last_became_active":"2024-04-24T23:59:11.035487+0000","last_became_peered":"2024-04-24T23:59:11.035487+0000","last_unstale":"2024-04-24T23:59:11.042600+0000","last_undegraded":"2024-04-24T23:59:11.042600+0000","last_fullsized":"2024-04-24T23:59:11.042600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:58.900435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050948+0000","last_change":"2024-04-24T23:59:11.046416+0000","last_active":"2024-04-24T23:59:11.050948+0000","last_peered":"2024-04-24T23:59:11.050948+0000","last_clean":"2024-04-24T23:59:11.050948+0000","last_became_active":"2024-04-24T23:59:11.046138+0000","last_became_peered":"2024-04-24T23:59:11.046138+0000","last_unstale":"2024-04-24T23:59:11.050948+0000","last_undegraded":"2024-04-24T23:59:11.050948+0000","last_fullsized":"2024-04-24T23:59:11.050948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:13.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047173+0000","last_change":"2024-04-24T23:59:11.042605+0000","last_active":"2024-04-24T23:59:11.047173+0000","last_peered":"2024-04-24T23:59:11.047173+0000","last_clean":"2024-04-24T23:59:11.047173+0000","last_became_active":"2024-04-24T23:59:11.042168+0000","last_became_peered":"2024-04-24T23:59:11.042168+0000","last_unstale":"2024-04-24T23:59:11.047173+0000","last_undegraded":"2024-04-24T23:59:11.047173+0000","last_fullsized":"2024-04-24T23:59:11.047173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:13.401522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.045356+0000","last_change":"2024-04-24T23:59:11.035140+0000","last_active":"2024-04-24T23:59:11.045356+0000","last_peered":"2024-04-24T23:59:11.045356+0000","last_clean":"2024-04-24T23:59:11.045356+0000","last_became_active":"2024-04-24T23:59:11.034830+0000","last_became_peered":"2024-04-24T23:59:11.034830+0000","last_unstale":"2024-04-24T23:59:11.045356+0000","last_undegraded":"2024-04-24T23:59:11.045356+0000","last_fullsized":"2024-04-24T23:59:11.045356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:35.978791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039615+0000","last_change":"2024-04-24T23:59:11.030772+0000","last_active":"2024-04-24T23:59:11.039615+0000","last_peered":"2024-04-24T23:59:11.039615+0000","last_clean":"2024-04-24T23:59:11.039615+0000","last_became_active":"2024-04-24T23:59:11.030494+0000","last_became_peered":"2024-04-24T23:59:11.030494+0000","last_unstale":"2024-04-24T23:59:11.039615+0000","last_undegraded":"2024-04-24T23:59:11.039615+0000","last_fullsized":"2024-04-24T23:59:11.039615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039445+0000","last_change":"2024-04-24T23:59:11.037548+0000","last_active":"2024-04-24T23:59:11.039445+0000","last_peered":"2024-04-24T23:59:11.039445+0000","last_clean":"2024-04-24T23:59:11.039445+0000","last_became_active":"2024-04-24T23:59:11.036716+0000","last_became_peered":"2024-04-24T23:59:11.036716+0000","last_unstale":"2024-04-24T23:59:11.039445+0000","last_undegraded":"2024-04-24T23:59:11.039445+0000","last_fullsized":"2024-04-24T23:59:11.039445+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:34.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047933+0000","last_change":"2024-04-24T23:59:11.044996+0000","last_active":"2024-04-24T23:59:11.047933+0000","last_peered":"2024-04-24T23:59:11.047933+0000","last_clean":"2024-04-24T23:59:11.047933+0000","last_became_active":"2024-04-24T23:59:11.044721+0000","last_became_peered":"2024-04-24T23:59:11.044721+0000","last_unstale":"2024-04-24T23:59:11.047933+0000","last_undegraded":"2024-04-24T23:59:11.047933+0000","last_fullsized":"2024-04-24T23:59:11.047933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:03.928528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.032258+0000","last_change":"2024-04-24T23:59:11.029934+0000","last_active":"2024-04-24T23:59:11.032258+0000","last_peered":"2024-04-24T23:59:11.032258+0000","last_clean":"2024-04-24T23:59:11.032258+0000","last_became_active":"2024-04-24T23:59:11.029682+0000","last_became_peered":"2024-04-24T23:59:11.029682+0000","last_unstale":"2024-04-24T23:59:11.032258+0000","last_undegraded":"2024-04-24T23:59:11.032258+0000","last_fullsized":"2024-04-24T23:59:11.032258+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047588+0000","last_change":"2024-04-24T23:59:10.031868+0000","last_active":"2024-04-24T23:59:11.047588+0000","last_peered":"2024-04-24T23:59:11.047588+0000","last_clean":"2024-04-24T23:59:11.047588+0000","last_became_active":"2024-04-24T23:59:10.031602+0000","last_became_peered":"2024-04-24T23:59:10.031602+0000","last_unstale":"2024-04-24T23:59:11.047588+0000","last_undegraded":"2024-04-24T23:59:11.047588+0000","last_fullsized":"2024-04-24T23:59:11.047588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:26.494851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.044617+0000","last_change":"2024-04-24T23:59:11.043135+0000","last_active":"2024-04-24T23:59:11.044617+0000","last_peered":"2024-04-24T23:59:11.044617+0000","last_clean":"2024-04-24T23:59:11.044617+0000","last_became_active":"2024-04-24T23:59:11.042870+0000","last_became_peered":"2024-04-24T23:59:11.042870+0000","last_unstale":"2024-04-24T23:59:11.044617+0000","last_undegraded":"2024-04-24T23:59:11.044617+0000","last_fullsized":"2024-04-24T23:59:11.044617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:00.478646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041742+0000","last_change":"2024-04-24T23:59:10.035978+0000","last_active":"2024-04-24T23:59:11.041742+0000","last_peered":"2024-04-24T23:59:11.041742+0000","last_clean":"2024-04-24T23:59:11.041742+0000","last_became_active":"2024-04-24T23:59:10.035623+0000","last_became_peered":"2024-04-24T23:59:10.035623+0000","last_unstale":"2024-04-24T23:59:11.041742+0000","last_undegraded":"2024-04-24T23:59:11.041742+0000","last_fullsized":"2024-04-24T23:59:11.041742+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:33.084138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041967+0000","last_change":"2024-04-24T23:59:11.036208+0000","last_active":"2024-04-24T23:59:11.041967+0000","last_peered":"2024-04-24T23:59:11.041967+0000","last_clean":"2024-04-24T23:59:11.041967+0000","last_became_active":"2024-04-24T23:59:11.035884+0000","last_became_peered":"2024-04-24T23:59:11.035884+0000","last_unstale":"2024-04-24T23:59:11.041967+0000","last_undegraded":"2024-04-24T23:59:11.041967+0000","last_fullsized":"2024-04-24T23:59:11.041967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049800+0000","last_change":"2024-04-24T23:59:10.039192+0000","last_active":"2024-04-24T23:59:11.049800+0000","last_peered":"2024-04-24T23:59:11.049800+0000","last_clean":"2024-04-24T23:59:11.049800+0000","last_became_active":"2024-04-24T23:59:10.038852+0000","last_became_peered":"2024-04-24T23:59:10.038852+0000","last_unstale":"2024-04-24T23:59:11.049800+0000","last_undegraded":"2024-04-24T23:59:11.049800+0000","last_fullsized":"2024-04-24T23:59:11.049800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:14.731367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050975+0000","last_change":"2024-04-24T23:59:11.047033+0000","last_active":"2024-04-24T23:59:11.050975+0000","last_peered":"2024-04-24T23:59:11.050975+0000","last_clean":"2024-04-24T23:59:11.050975+0000","last_became_active":"2024-04-24T23:59:11.046714+0000","last_became_peered":"2024-04-24T23:59:11.046714+0000","last_unstale":"2024-04-24T23:59:11.050975+0000","last_undegraded":"2024-04-24T23:59:11.050975+0000","last_fullsized":"2024-04-24T23:59:11.050975+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046045+0000","last_change":"2024-04-24T23:59:10.038217+0000","last_active":"2024-04-24T23:59:11.046045+0000","last_peered":"2024-04-24T23:59:11.046045+0000","last_clean":"2024-04-24T23:59:11.046045+0000","last_became_active":"2024-04-24T23:59:10.037635+0000","last_became_peered":"2024-04-24T23:59:10.037635+0000","last_unstale":"2024-04-24T23:59:11.046045+0000","last_undegraded":"2024-04-24T23:59:11.046045+0000","last_fullsized":"2024-04-24T23:59:11.046045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:59.531835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047829+0000","last_change":"2024-04-24T23:59:11.044307+0000","last_active":"2024-04-24T23:59:11.047829+0000","last_peered":"2024-04-24T23:59:11.047829+0000","last_clean":"2024-04-24T23:59:11.047829+0000","last_became_active":"2024-04-24T23:59:11.043805+0000","last_became_peered":"2024-04-24T23:59:11.043805+0000","last_unstale":"2024-04-24T23:59:11.047829+0000","last_undegraded":"2024-04-24T23:59:11.047829+0000","last_fullsized":"2024-04-24T23:59:11.047829+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:58.900435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049963+0000","last_change":"2024-04-24T23:59:10.029038+0000","last_active":"2024-04-24T23:59:11.049963+0000","last_peered":"2024-04-24T23:59:11.049963+0000","last_clean":"2024-04-24T23:59:11.049963+0000","last_became_active":"2024-04-24T23:59:10.028736+0000","last_became_peered":"2024-04-24T23:59:10.028736+0000","last_unstale":"2024-04-24T23:59:11.049963+0000","last_undegraded":"2024-04-24T23:59:11.049963+0000","last_fullsized":"2024-04-24T23:59:11.049963+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:00.490325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048436+0000","last_change":"2024-04-24T23:59:11.043782+0000","last_active":"2024-04-24T23:59:11.048436+0000","last_peered":"2024-04-24T23:59:11.048436+0000","last_clean":"2024-04-24T23:59:11.048436+0000","last_became_active":"2024-04-24T23:59:11.043494+0000","last_became_peered":"2024-04-24T23:59:11.043494+0000","last_unstale":"2024-04-24T23:59:11.048436+0000","last_undegraded":"2024-04-24T23:59:11.048436+0000","last_fullsized":"2024-04-24T23:59:11.048436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049885+0000","last_change":"2024-04-24T23:59:10.031300+0000","last_active":"2024-04-24T23:59:11.049885+0000","last_peered":"2024-04-24T23:59:11.049885+0000","last_clean":"2024-04-24T23:59:11.049885+0000","last_became_active":"2024-04-24T23:59:10.031043+0000","last_became_peered":"2024-04-24T23:59:10.031043+0000","last_unstale":"2024-04-24T23:59:11.049885+0000","last_undegraded":"2024-04-24T23:59:11.049885+0000","last_fullsized":"2024-04-24T23:59:11.049885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.028196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041841+0000","last_change":"2024-04-24T23:59:11.033273+0000","last_active":"2024-04-24T23:59:11.041841+0000","last_peered":"2024-04-24T23:59:11.041841+0000","last_clean":"2024-04-24T23:59:11.041841+0000","last_became_active":"2024-04-24T23:59:11.032976+0000","last_became_peered":"2024-04-24T23:59:11.032976+0000","last_unstale":"2024-04-24T23:59:11.041841+0000","last_undegraded":"2024-04-24T23:59:11.041841+0000","last_fullsized":"2024-04-24T23:59:11.041841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:14.713041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.030707+0000","last_change":"2024-04-24T23:59:10.033511+0000","last_active":"2024-04-24T23:59:11.030707+0000","last_peered":"2024-04-24T23:59:11.030707+0000","last_clean":"2024-04-24T23:59:11.030707+0000","last_became_active":"2024-04-24T23:59:10.033224+0000","last_became_peered":"2024-04-24T23:59:10.033224+0000","last_unstale":"2024-04-24T23:59:11.030707+0000","last_undegraded":"2024-04-24T23:59:11.030707+0000","last_fullsized":"2024-04-24T23:59:11.030707+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041395+0000","last_change":"2024-04-24T23:59:11.033930+0000","last_active":"2024-04-24T23:59:11.041395+0000","last_peered":"2024-04-24T23:59:11.041395+0000","last_clean":"2024-04-24T23:59:11.041395+0000","last_became_active":"2024-04-24T23:59:11.033665+0000","last_became_peered":"2024-04-24T23:59:11.033665+0000","last_unstale":"2024-04-24T23:59:11.041395+0000","last_undegraded":"2024-04-24T23:59:11.041395+0000","last_fullsized":"2024-04-24T23:59:11.041395+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.022107+0000","last_change":"2024-04-24T23:59:10.028386+0000","last_active":"2024-04-24T23:59:11.022107+0000","last_peered":"2024-04-24T23:59:11.022107+0000","last_clean":"2024-04-24T23:59:11.022107+0000","last_became_active":"2024-04-24T23:59:10.027914+0000","last_became_peered":"2024-04-24T23:59:10.027914+0000","last_unstale":"2024-04-24T23:59:11.022107+0000","last_undegraded":"2024-04-24T23:59:11.022107+0000","last_fullsized":"2024-04-24T23:59:11.022107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:35.284706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048523+0000","last_change":"2024-04-24T23:59:11.044428+0000","last_active":"2024-04-24T23:59:11.048523+0000","last_peered":"2024-04-24T23:59:11.048523+0000","last_clean":"2024-04-24T23:59:11.048523+0000","last_became_active":"2024-04-24T23:59:11.043622+0000","last_became_peered":"2024-04-24T23:59:11.043622+0000","last_unstale":"2024-04-24T23:59:11.048523+0000","last_undegraded":"2024-04-24T23:59:11.048523+0000","last_fullsized":"2024-04-24T23:59:11.048523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:38.949687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.034432+0000","last_change":"2024-04-24T23:59:10.034873+0000","last_active":"2024-04-24T23:59:11.034432+0000","last_peered":"2024-04-24T23:59:11.034432+0000","last_clean":"2024-04-24T23:59:11.034432+0000","last_became_active":"2024-04-24T23:59:10.034543+0000","last_became_peered":"2024-04-24T23:59:10.034543+0000","last_unstale":"2024-04-24T23:59:11.034432+0000","last_undegraded":"2024-04-24T23:59:11.034432+0000","last_fullsized":"2024-04-24T23:59:11.034432+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:12.399529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050988+0000","last_change":"2024-04-24T23:59:11.048598+0000","last_active":"2024-04-24T23:59:11.050988+0000","last_peered":"2024-04-24T23:59:11.050988+0000","last_clean":"2024-04-24T23:59:11.050988+0000","last_became_active":"2024-04-24T23:59:11.048252+0000","last_became_peered":"2024-04-24T23:59:11.048252+0000","last_unstale":"2024-04-24T23:59:11.050988+0000","last_undegraded":"2024-04-24T23:59:11.050988+0000","last_fullsized":"2024-04-24T23:59:11.050988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:18.404868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021522+0000","last_change":"2024-04-24T23:59:10.028605+0000","last_active":"2024-04-24T23:59:11.021522+0000","last_peered":"2024-04-24T23:59:11.021522+0000","last_clean":"2024-04-24T23:59:11.021522+0000","last_became_active":"2024-04-24T23:59:10.028297+0000","last_became_peered":"2024-04-24T23:59:10.028297+0000","last_unstale":"2024-04-24T23:59:11.021522+0000","last_undegraded":"2024-04-24T23:59:11.021522+0000","last_fullsized":"2024-04-24T23:59:11.021522+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:59.476653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049706+0000","last_change":"2024-04-24T23:59:11.042293+0000","last_active":"2024-04-24T23:59:11.049706+0000","last_peered":"2024-04-24T23:59:11.049706+0000","last_clean":"2024-04-24T23:59:11.049706+0000","last_became_active":"2024-04-24T23:59:11.042028+0000","last_became_peered":"2024-04-24T23:59:11.042028+0000","last_unstale":"2024-04-24T23:59:11.049706+0000","last_undegraded":"2024-04-24T23:59:11.049706+0000","last_fullsized":"2024-04-24T23:59:11.049706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:38.949687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049231+0000","last_change":"2024-04-24T23:59:10.036821+0000","last_active":"2024-04-24T23:59:11.049231+0000","last_peered":"2024-04-24T23:59:11.049231+0000","last_clean":"2024-04-24T23:59:11.049231+0000","last_became_active":"2024-04-24T23:59:10.036523+0000","last_became_peered":"2024-04-24T23:59:10.036523+0000","last_unstale":"2024-04-24T23:59:11.049231+0000","last_undegraded":"2024-04-24T23:59:11.049231+0000","last_fullsized":"2024-04-24T23:59:11.049231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:05.627877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049301+0000","last_change":"2024-04-24T23:59:11.048171+0000","last_active":"2024-04-24T23:59:11.049301+0000","last_peered":"2024-04-24T23:59:11.049301+0000","last_clean":"2024-04-24T23:59:11.049301+0000","last_became_active":"2024-04-24T23:59:11.047882+0000","last_became_peered":"2024-04-24T23:59:11.047882+0000","last_unstale":"2024-04-24T23:59:11.049301+0000","last_undegraded":"2024-04-24T23:59:11.049301+0000","last_fullsized":"2024-04-24T23:59:11.049301+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:34.170794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.052026+0000","last_change":"2024-04-24T23:59:10.039855+0000","last_active":"2024-04-24T23:59:11.052026+0000","last_peered":"2024-04-24T23:59:11.052026+0000","last_clean":"2024-04-24T23:59:11.052026+0000","last_became_active":"2024-04-24T23:59:10.039645+0000","last_became_peered":"2024-04-24T23:59:10.039645+0000","last_unstale":"2024-04-24T23:59:11.052026+0000","last_undegraded":"2024-04-24T23:59:11.052026+0000","last_fullsized":"2024-04-24T23:59:11.052026+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:47.344139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041155+0000","last_change":"2024-04-24T23:59:11.034592+0000","last_active":"2024-04-24T23:59:11.041155+0000","last_peered":"2024-04-24T23:59:11.041155+0000","last_clean":"2024-04-24T23:59:11.041155+0000","last_became_active":"2024-04-24T23:59:11.034331+0000","last_became_peered":"2024-04-24T23:59:11.034331+0000","last_unstale":"2024-04-24T23:59:11.041155+0000","last_undegraded":"2024-04-24T23:59:11.041155+0000","last_fullsized":"2024-04-24T23:59:11.041155+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047357+0000","last_change":"2024-04-24T23:59:10.029324+0000","last_active":"2024-04-24T23:59:11.047357+0000","last_peered":"2024-04-24T23:59:11.047357+0000","last_clean":"2024-04-24T23:59:11.047357+0000","last_became_active":"2024-04-24T23:59:10.027966+0000","last_became_peered":"2024-04-24T23:59:10.027966+0000","last_unstale":"2024-04-24T23:59:11.047357+0000","last_undegraded":"2024-04-24T23:59:11.047357+0000","last_fullsized":"2024-04-24T23:59:11.047357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:31.859019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043539+0000","last_change":"2024-04-24T23:59:11.035606+0000","last_active":"2024-04-24T23:59:11.043539+0000","last_peered":"2024-04-24T23:59:11.043539+0000","last_clean":"2024-04-24T23:59:11.043539+0000","last_became_active":"2024-04-24T23:59:11.035347+0000","last_became_peered":"2024-04-24T23:59:11.035347+0000","last_unstale":"2024-04-24T23:59:11.043539+0000","last_undegraded":"2024-04-24T23:59:11.043539+0000","last_fullsized":"2024-04-24T23:59:11.043539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:32.389674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.023593+0000","last_change":"2024-04-24T23:59:10.025754+0000","last_active":"2024-04-24T23:59:11.023593+0000","last_peered":"2024-04-24T23:59:11.023593+0000","last_clean":"2024-04-24T23:59:11.023593+0000","last_became_active":"2024-04-24T23:59:10.025494+0000","last_became_peered":"2024-04-24T23:59:10.025494+0000","last_unstale":"2024-04-24T23:59:11.023593+0000","last_undegraded":"2024-04-24T23:59:11.023593+0000","last_fullsized":"2024-04-24T23:59:11.023593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:33.084138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.047955+0000","last_change":"2024-04-24T23:59:11.045675+0000","last_active":"2024-04-24T23:59:11.047955+0000","last_peered":"2024-04-24T23:59:11.047955+0000","last_clean":"2024-04-24T23:59:11.047955+0000","last_became_active":"2024-04-24T23:59:11.045387+0000","last_became_peered":"2024-04-24T23:59:11.045387+0000","last_unstale":"2024-04-24T23:59:11.047955+0000","last_undegraded":"2024-04-24T23:59:11.047955+0000","last_fullsized":"2024-04-24T23:59:11.047955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:38.949687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.030840+0000","last_change":"2024-04-24T23:59:10.033656+0000","last_active":"2024-04-24T23:59:11.030840+0000","last_peered":"2024-04-24T23:59:11.030840+0000","last_clean":"2024-04-24T23:59:11.030840+0000","last_became_active":"2024-04-24T23:59:10.033394+0000","last_became_peered":"2024-04-24T23:59:10.033394+0000","last_unstale":"2024-04-24T23:59:11.030840+0000","last_undegraded":"2024-04-24T23:59:11.030840+0000","last_fullsized":"2024-04-24T23:59:11.030840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:05.627877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.044590+0000","last_change":"2024-04-24T23:59:11.043207+0000","last_active":"2024-04-24T23:59:11.044590+0000","last_peered":"2024-04-24T23:59:11.044590+0000","last_clean":"2024-04-24T23:59:11.044590+0000","last_became_active":"2024-04-24T23:59:11.042921+0000","last_became_peered":"2024-04-24T23:59:11.042921+0000","last_unstale":"2024-04-24T23:59:11.044590+0000","last_undegraded":"2024-04-24T23:59:11.044590+0000","last_fullsized":"2024-04-24T23:59:11.044590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:35.978791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050727+0000","last_change":"2024-04-24T23:59:10.039079+0000","last_active":"2024-04-24T23:59:11.050727+0000","last_peered":"2024-04-24T23:59:11.050727+0000","last_clean":"2024-04-24T23:59:11.050727+0000","last_became_active":"2024-04-24T23:59:10.038790+0000","last_became_peered":"2024-04-24T23:59:10.038790+0000","last_unstale":"2024-04-24T23:59:11.050727+0000","last_undegraded":"2024-04-24T23:59:11.050727+0000","last_fullsized":"2024-04-24T23:59:11.050727+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:19.763859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042149+0000","last_change":"2024-04-24T23:59:11.039583+0000","last_active":"2024-04-24T23:59:11.042149+0000","last_peered":"2024-04-24T23:59:11.042149+0000","last_clean":"2024-04-24T23:59:11.042149+0000","last_became_active":"2024-04-24T23:59:11.039313+0000","last_became_peered":"2024-04-24T23:59:11.039313+0000","last_unstale":"2024-04-24T23:59:11.042149+0000","last_undegraded":"2024-04-24T23:59:11.042149+0000","last_fullsized":"2024-04-24T23:59:11.042149+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.050236+0000","last_change":"2024-04-24T23:59:10.026367+0000","last_active":"2024-04-24T23:59:11.050236+0000","last_peered":"2024-04-24T23:59:11.050236+0000","last_clean":"2024-04-24T23:59:11.050236+0000","last_became_active":"2024-04-24T23:59:10.026092+0000","last_became_peered":"2024-04-24T23:59:10.026092+0000","last_unstale":"2024-04-24T23:59:11.050236+0000","last_undegraded":"2024-04-24T23:59:11.050236+0000","last_fullsized":"2024-04-24T23:59:11.050236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:59.476653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048410+0000","last_change":"2024-04-24T23:59:11.044314+0000","last_active":"2024-04-24T23:59:11.048410+0000","last_peered":"2024-04-24T23:59:11.048410+0000","last_clean":"2024-04-24T23:59:11.048410+0000","last_became_active":"2024-04-24T23:59:11.043995+0000","last_became_peered":"2024-04-24T23:59:11.043995+0000","last_unstale":"2024-04-24T23:59:11.048410+0000","last_undegraded":"2024-04-24T23:59:11.048410+0000","last_fullsized":"2024-04-24T23:59:11.048410+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:03.928528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.046184+0000","last_change":"2024-04-24T23:59:10.027958+0000","last_active":"2024-04-24T23:59:11.046184+0000","last_peered":"2024-04-24T23:59:11.046184+0000","last_clean":"2024-04-24T23:59:11.046184+0000","last_became_active":"2024-04-24T23:59:10.027542+0000","last_became_peered":"2024-04-24T23:59:10.027542+0000","last_unstale":"2024-04-24T23:59:11.046184+0000","last_undegraded":"2024-04-24T23:59:11.046184+0000","last_fullsized":"2024-04-24T23:59:11.046184+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:55.726352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049040+0000","last_change":"2024-04-24T23:59:11.046295+0000","last_active":"2024-04-24T23:59:11.049040+0000","last_peered":"2024-04-24T23:59:11.049040+0000","last_clean":"2024-04-24T23:59:11.049040+0000","last_became_active":"2024-04-24T23:59:11.045413+0000","last_became_peered":"2024-04-24T23:59:11.045413+0000","last_unstale":"2024-04-24T23:59:11.049040+0000","last_undegraded":"2024-04-24T23:59:11.049040+0000","last_fullsized":"2024-04-24T23:59:11.049040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:15.733360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.048957+0000","last_change":"2024-04-24T23:59:10.032417+0000","last_active":"2024-04-24T23:59:11.048957+0000","last_peered":"2024-04-24T23:59:11.048957+0000","last_clean":"2024-04-24T23:59:11.048957+0000","last_became_active":"2024-04-24T23:59:10.031940+0000","last_became_peered":"2024-04-24T23:59:10.031940+0000","last_unstale":"2024-04-24T23:59:11.048957+0000","last_undegraded":"2024-04-24T23:59:11.048957+0000","last_fullsized":"2024-04-24T23:59:11.048957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:12.125339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039184+0000","last_change":"2024-04-24T23:59:11.032371+0000","last_active":"2024-04-24T23:59:11.039184+0000","last_peered":"2024-04-24T23:59:11.039184+0000","last_clean":"2024-04-24T23:59:11.039184+0000","last_became_active":"2024-04-24T23:59:11.032126+0000","last_became_peered":"2024-04-24T23:59:11.032126+0000","last_unstale":"2024-04-24T23:59:11.039184+0000","last_undegraded":"2024-04-24T23:59:11.039184+0000","last_fullsized":"2024-04-24T23:59:11.039184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.566855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.022296+0000","last_change":"2024-04-24T23:59:10.032454+0000","last_active":"2024-04-24T23:59:11.022296+0000","last_peered":"2024-04-24T23:59:11.022296+0000","last_clean":"2024-04-24T23:59:11.022296+0000","last_became_active":"2024-04-24T23:59:10.032050+0000","last_became_peered":"2024-04-24T23:59:10.032050+0000","last_unstale":"2024-04-24T23:59:11.022296+0000","last_undegraded":"2024-04-24T23:59:11.022296+0000","last_fullsized":"2024-04-24T23:59:11.022296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:26.494851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049087+0000","last_change":"2024-04-24T23:59:11.043989+0000","last_active":"2024-04-24T23:59:11.049087+0000","last_peered":"2024-04-24T23:59:11.049087+0000","last_clean":"2024-04-24T23:59:11.049087+0000","last_became_active":"2024-04-24T23:59:11.043712+0000","last_became_peered":"2024-04-24T23:59:11.043712+0000","last_unstale":"2024-04-24T23:59:11.049087+0000","last_undegraded":"2024-04-24T23:59:11.049087+0000","last_fullsized":"2024-04-24T23:59:11.049087+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:43.033725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042942+0000","last_change":"2024-04-24T23:59:10.025961+0000","last_active":"2024-04-24T23:59:11.042942+0000","last_peered":"2024-04-24T23:59:11.042942+0000","last_clean":"2024-04-24T23:59:11.042942+0000","last_became_active":"2024-04-24T23:59:10.025663+0000","last_became_peered":"2024-04-24T23:59:10.025663+0000","last_unstale":"2024-04-24T23:59:11.042942+0000","last_undegraded":"2024-04-24T23:59:11.042942+0000","last_fullsized":"2024-04-24T23:59:11.042942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:34.976798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.042984+0000","last_change":"2024-04-24T23:59:11.036579+0000","last_active":"2024-04-24T23:59:11.042984+0000","last_peered":"2024-04-24T23:59:11.042984+0000","last_clean":"2024-04-24T23:59:11.042984+0000","last_became_active":"2024-04-24T23:59:11.036268+0000","last_became_peered":"2024-04-24T23:59:11.036268+0000","last_unstale":"2024-04-24T23:59:11.042984+0000","last_undegraded":"2024-04-24T23:59:11.042984+0000","last_fullsized":"2024-04-24T23:59:11.042984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:10.086376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.026498+0000","last_change":"2024-04-24T23:59:10.035596+0000","last_active":"2024-04-24T23:59:11.026498+0000","last_peered":"2024-04-24T23:59:11.026498+0000","last_clean":"2024-04-24T23:59:11.026498+0000","last_became_active":"2024-04-24T23:59:10.035272+0000","last_became_peered":"2024-04-24T23:59:10.035272+0000","last_unstale":"2024-04-24T23:59:11.026498+0000","last_undegraded":"2024-04-24T23:59:11.026498+0000","last_fullsized":"2024-04-24T23:59:11.026498+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:17.028196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043245+0000","last_change":"2024-04-24T23:59:11.037157+0000","last_active":"2024-04-24T23:59:11.043245+0000","last_peered":"2024-04-24T23:59:11.043245+0000","last_clean":"2024-04-24T23:59:11.043245+0000","last_became_active":"2024-04-24T23:59:11.036373+0000","last_became_peered":"2024-04-24T23:59:11.036373+0000","last_unstale":"2024-04-24T23:59:11.043245+0000","last_undegraded":"2024-04-24T23:59:11.043245+0000","last_fullsized":"2024-04-24T23:59:11.043245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:40.249109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021438+0000","last_change":"2024-04-24T23:59:10.035561+0000","last_active":"2024-04-24T23:59:11.021438+0000","last_peered":"2024-04-24T23:59:11.021438+0000","last_clean":"2024-04-24T23:59:11.021438+0000","last_became_active":"2024-04-24T23:59:10.035270+0000","last_became_peered":"2024-04-24T23:59:10.035270+0000","last_unstale":"2024-04-24T23:59:11.021438+0000","last_undegraded":"2024-04-24T23:59:11.021438+0000","last_fullsized":"2024-04-24T23:59:11.021438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:37.067752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039954+0000","last_change":"2024-04-24T23:59:11.031210+0000","last_active":"2024-04-24T23:59:11.039954+0000","last_peered":"2024-04-24T23:59:11.039954+0000","last_clean":"2024-04-24T23:59:11.039954+0000","last_became_active":"2024-04-24T23:59:11.030701+0000","last_became_peered":"2024-04-24T23:59:11.030701+0000","last_unstale":"2024-04-24T23:59:11.039954+0000","last_undegraded":"2024-04-24T23:59:11.039954+0000","last_fullsized":"2024-04-24T23:59:11.039954+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.032587+0000","last_change":"2024-04-24T23:59:10.036837+0000","last_active":"2024-04-24T23:59:11.032587+0000","last_peered":"2024-04-24T23:59:11.032587+0000","last_clean":"2024-04-24T23:59:11.032587+0000","last_became_active":"2024-04-24T23:59:10.036539+0000","last_became_peered":"2024-04-24T23:59:10.036539+0000","last_unstale":"2024-04-24T23:59:11.032587+0000","last_undegraded":"2024-04-24T23:59:11.032587+0000","last_fullsized":"2024-04-24T23:59:11.032587+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:24.716999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.043264+0000","last_change":"2024-04-24T23:59:11.040393+0000","last_active":"2024-04-24T23:59:11.043264+0000","last_peered":"2024-04-24T23:59:11.043264+0000","last_clean":"2024-04-24T23:59:11.043264+0000","last_became_active":"2024-04-24T23:59:11.040119+0000","last_became_peered":"2024-04-24T23:59:11.040119+0000","last_unstale":"2024-04-24T23:59:11.043264+0000","last_undegraded":"2024-04-24T23:59:11.043264+0000","last_fullsized":"2024-04-24T23:59:11.043264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:40.343708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049207+0000","last_change":"2024-04-24T23:59:10.032308+0000","last_active":"2024-04-24T23:59:11.049207+0000","last_peered":"2024-04-24T23:59:11.049207+0000","last_clean":"2024-04-24T23:59:11.049207+0000","last_became_active":"2024-04-24T23:59:10.031672+0000","last_became_peered":"2024-04-24T23:59:10.031672+0000","last_unstale":"2024-04-24T23:59:11.049207+0000","last_undegraded":"2024-04-24T23:59:11.049207+0000","last_fullsized":"2024-04-24T23:59:11.049207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:34.976798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049248+0000","last_change":"2024-04-24T23:59:11.048263+0000","last_active":"2024-04-24T23:59:11.049248+0000","last_peered":"2024-04-24T23:59:11.049248+0000","last_clean":"2024-04-24T23:59:11.049248+0000","last_became_active":"2024-04-24T23:59:11.047935+0000","last_became_peered":"2024-04-24T23:59:11.047935+0000","last_unstale":"2024-04-24T23:59:11.049248+0000","last_undegraded":"2024-04-24T23:59:11.049248+0000","last_fullsized":"2024-04-24T23:59:11.049248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:45.377965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021400+0000","last_change":"2024-04-24T23:59:10.026005+0000","last_active":"2024-04-24T23:59:11.021400+0000","last_peered":"2024-04-24T23:59:11.021400+0000","last_clean":"2024-04-24T23:59:11.021400+0000","last_became_active":"2024-04-24T23:59:10.025634+0000","last_became_peered":"2024-04-24T23:59:10.025634+0000","last_unstale":"2024-04-24T23:59:11.021400+0000","last_undegraded":"2024-04-24T23:59:11.021400+0000","last_fullsized":"2024-04-24T23:59:11.021400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:13.711048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.031635+0000","last_change":"2024-04-24T23:59:11.025721+0000","last_active":"2024-04-24T23:59:11.031635+0000","last_peered":"2024-04-24T23:59:11.031635+0000","last_clean":"2024-04-24T23:59:11.031635+0000","last_became_active":"2024-04-24T23:59:11.025412+0000","last_became_peered":"2024-04-24T23:59:11.025412+0000","last_unstale":"2024-04-24T23:59:11.031635+0000","last_undegraded":"2024-04-24T23:59:11.031635+0000","last_fullsized":"2024-04-24T23:59:11.031635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:20.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.024525+0000","last_change":"2024-04-24T23:59:10.034763+0000","last_active":"2024-04-24T23:59:11.024525+0000","last_peered":"2024-04-24T23:59:11.024525+0000","last_clean":"2024-04-24T23:59:11.024525+0000","last_became_active":"2024-04-24T23:59:10.034489+0000","last_became_peered":"2024-04-24T23:59:10.034489+0000","last_unstale":"2024-04-24T23:59:11.024525+0000","last_undegraded":"2024-04-24T23:59:11.024525+0000","last_fullsized":"2024-04-24T23:59:11.024525+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:03.292935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039374+0000","last_change":"2024-04-24T23:59:11.035773+0000","last_active":"2024-04-24T23:59:11.039374+0000","last_peered":"2024-04-24T23:59:11.039374+0000","last_clean":"2024-04-24T23:59:11.039374+0000","last_became_active":"2024-04-24T23:59:11.035492+0000","last_became_peered":"2024-04-24T23:59:11.035492+0000","last_unstale":"2024-04-24T23:59:11.039374+0000","last_undegraded":"2024-04-24T23:59:11.039374+0000","last_fullsized":"2024-04-24T23:59:11.039374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:32.861012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.022066+0000","last_change":"2024-04-24T23:59:10.029707+0000","last_active":"2024-04-24T23:59:11.022066+0000","last_peered":"2024-04-24T23:59:11.022066+0000","last_clean":"2024-04-24T23:59:11.022066+0000","last_became_active":"2024-04-24T23:59:10.029460+0000","last_became_peered":"2024-04-24T23:59:10.029460+0000","last_unstale":"2024-04-24T23:59:11.022066+0000","last_undegraded":"2024-04-24T23:59:11.022066+0000","last_fullsized":"2024-04-24T23:59:11.022066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:03.292935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.033218+0000","last_change":"2024-04-24T23:59:11.030686+0000","last_active":"2024-04-24T23:59:11.033218+0000","last_peered":"2024-04-24T23:59:11.033218+0000","last_clean":"2024-04-24T23:59:11.033218+0000","last_became_active":"2024-04-24T23:59:11.030403+0000","last_became_peered":"2024-04-24T23:59:11.030403+0000","last_unstale":"2024-04-24T23:59:11.033218+0000","last_undegraded":"2024-04-24T23:59:11.033218+0000","last_fullsized":"2024-04-24T23:59:11.033218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:25.718992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.023736+0000","last_change":"2024-04-24T23:59:10.030771+0000","last_active":"2024-04-24T23:59:11.023736+0000","last_peered":"2024-04-24T23:59:11.023736+0000","last_clean":"2024-04-24T23:59:11.023736+0000","last_became_active":"2024-04-24T23:59:10.030513+0000","last_became_peered":"2024-04-24T23:59:10.030513+0000","last_unstale":"2024-04-24T23:59:11.023736+0000","last_undegraded":"2024-04-24T23:59:11.023736+0000","last_fullsized":"2024-04-24T23:59:11.023736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:13.711048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.049873+0000","last_change":"2024-04-24T23:59:11.047902+0000","last_active":"2024-04-24T23:59:11.049873+0000","last_peered":"2024-04-24T23:59:11.049873+0000","last_clean":"2024-04-24T23:59:11.049873+0000","last_became_active":"2024-04-24T23:59:11.047429+0000","last_became_peered":"2024-04-24T23:59:11.047429+0000","last_unstale":"2024-04-24T23:59:11.049873+0000","last_undegraded":"2024-04-24T23:59:11.049873+0000","last_fullsized":"2024-04-24T23:59:11.049873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:04.507679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.021608+0000","last_change":"2024-04-24T23:59:10.030560+0000","last_active":"2024-04-24T23:59:11.021608+0000","last_peered":"2024-04-24T23:59:11.021608+0000","last_clean":"2024-04-24T23:59:11.021608+0000","last_became_active":"2024-04-24T23:59:10.030266+0000","last_became_peered":"2024-04-24T23:59:10.030266+0000","last_unstale":"2024-04-24T23:59:11.021608+0000","last_undegraded":"2024-04-24T23:59:11.021608+0000","last_fullsized":"2024-04-24T23:59:11.021608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:03.853095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.041815+0000","last_change":"2024-04-24T23:59:11.036733+0000","last_active":"2024-04-24T23:59:11.041815+0000","last_peered":"2024-04-24T23:59:11.041815+0000","last_clean":"2024-04-24T23:59:11.041815+0000","last_became_active":"2024-04-24T23:59:11.036342+0000","last_became_peered":"2024-04-24T23:59:11.036342+0000","last_unstale":"2024-04-24T23:59:11.041815+0000","last_undegraded":"2024-04-24T23:59:11.041815+0000","last_fullsized":"2024-04-24T23:59:11.041815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.030000+0000","last_change":"2024-04-24T23:59:10.034120+0000","last_active":"2024-04-24T23:59:11.030000+0000","last_peered":"2024-04-24T23:59:11.030000+0000","last_clean":"2024-04-24T23:59:11.030000+0000","last_became_active":"2024-04-24T23:59:10.033836+0000","last_became_peered":"2024-04-24T23:59:10.033836+0000","last_unstale":"2024-04-24T23:59:11.030000+0000","last_undegraded":"2024-04-24T23:59:11.030000+0000","last_fullsized":"2024-04-24T23:59:11.030000+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:31.467243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.039948+0000","last_change":"2024-04-24T23:59:11.031033+0000","last_active":"2024-04-24T23:59:11.039948+0000","last_peered":"2024-04-24T23:59:11.039948+0000","last_clean":"2024-04-24T23:59:11.039948+0000","last_became_active":"2024-04-24T23:59:11.030686+0000","last_became_peered":"2024-04-24T23:59:11.030686+0000","last_unstale":"2024-04-24T23:59:11.039948+0000","last_undegraded":"2024-04-24T23:59:11.039948+0000","last_fullsized":"2024-04-24T23:59:11.039948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:10.006822+0000","last_clean_scrub_stamp":"2024-04-24T23:59:10.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:32.469236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.026020+0000","last_change":"2024-04-24T23:59:10.034872+0000","last_active":"2024-04-24T23:59:11.026020+0000","last_peered":"2024-04-24T23:59:11.026020+0000","last_clean":"2024-04-24T23:59:11.026020+0000","last_became_active":"2024-04-24T23:59:10.034565+0000","last_became_peered":"2024-04-24T23:59:10.034565+0000","last_unstale":"2024-04-24T23:59:11.026020+0000","last_undegraded":"2024-04-24T23:59:11.026020+0000","last_fullsized":"2024-04-24T23:59:11.026020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:59:09.004829+0000","last_clean_scrub_stamp":"2024-04-24T23:59:09.004829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:00.490325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-24T23:59:11.027904+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-24T23:59:11.027904+0000","last_peered":"2024-04-24T23:59:11.027904+0000","last_clean":"2024-04-24T23:59:11.027904+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-24T23:59:11.027904+0000","last_undegraded":"2024-04-24T23:59:11.027904+0000","last_fullsized":"2024-04-24T23:59:11.027904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":929792,"data_stored":717876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":134353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:59:14.325 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:59:14.325 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T23:59:14.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.464+0000 7f8180b4f640 1 Processor -- start 2024-04-24T23:59:14.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.464+0000 7f8180b4f640 1 -- start start 2024-04-24T23:59:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.465+0000 7f8180b4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.465+0000 7f8180b4f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f817c0b63b0 0x7f817c0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.465+0000 7f8180b4f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f817c0d2e00 con 0x7f817c0d28c0 2024-04-24T23:59:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.465+0000 7f8180b4f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f817c0d30e0 con 0x7f817c0b63b0 2024-04-24T23:59:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.465+0000 7f8180b4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f817c0d33c0 con 0x7f817c15e510 2024-04-24T23:59:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f817c0b63b0 0x7f817c0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f817c0b63b0 0x7f817c0b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55690/0 (socket says 172.21.15.113:55690) 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:54342/0 (socket says 172.21.15.113:54342) 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817ad9d640 1 -- 172.21.15.113:0/4068807029 learned_addr learned my addr 172.21.15.113:0/4068807029 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f8179d9b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f817c0d28c0 0x7f817c0b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33250/0 (socket says 172.21.15.113:33250) 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817a59c640 1 -- 172.21.15.113:0/4068807029 >> v2:172.21.15.178:3300/0 conn(0x7f817c0b63b0 msgr2=0x7f817c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817a59c640 1 --2- 172.21.15.113:0/4068807029 >> v2:172.21.15.178:3300/0 conn(0x7f817c0b63b0 0x7f817c0b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817a59c640 1 -- 172.21.15.113:0/4068807029 >> v1:172.21.15.178:6789/0 conn(0x7f817c0d28c0 legacy=0x7f817c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817a59c640 1 -- 172.21.15.113:0/4068807029 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c0d36a0 con 0x7f817c15e510 2024-04-24T23:59:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.466+0000 7f817ad9d640 1 --2- 172.21.15.113:0/4068807029 >> v2:172.21.15.178:3300/0 conn(0x7f817c0b63b0 0x7f817c0b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:14.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f817a59c640 1 --2- 172.21.15.113:0/4068807029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0d2380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f816c009d90 tx=0x7f816c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70a247a799bf371 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f817959a640 1 -- 172.21.15.113:0/4068807029 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c031600 con 0x7f817c15e510 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f817959a640 1 -- 172.21.15.113:0/4068807029 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816c0317a0 con 0x7f817c15e510 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f817959a640 1 -- 172.21.15.113:0/4068807029 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c031b70 con 0x7f817c15e510 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f8180b4f640 1 -- 172.21.15.113:0/4068807029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 msgr2=0x7f817c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f8180b4f640 1 --2- 172.21.15.113:0/4068807029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0d2380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f816c009d90 tx=0x7f816c02a5d0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f8180b4f640 1 -- 172.21.15.113:0/4068807029 shutdown_connections 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f8180b4f640 1 --2- 172.21.15.113:0/4068807029 >> v2:172.21.15.178:3300/0 conn(0x7f817c0b63b0 0x7f817c0b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f8180b4f640 1 --2- 172.21.15.113:0/4068807029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0d2380 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.467+0000 7f8180b4f640 1 -- 172.21.15.113:0/4068807029 >> 172.21.15.113:0/4068807029 conn(0x7f817c06b720 msgr2=0x7f817c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.468+0000 7f8180b4f640 1 -- 172.21.15.113:0/4068807029 shutdown_connections 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.468+0000 7f8180b4f640 1 -- 172.21.15.113:0/4068807029 wait complete. 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 Processor -- start 2024-04-24T23:59:14.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 -- start start 2024-04-24T23:59:14.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f817c0b63b0 0x7f817c1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f817c0d28c0 0x7f817c0b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f817c0b9dd0 con 0x7f817c0d28c0 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f817c0d3ea0 con 0x7f817c0b63b0 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.469+0000 7f8180b4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f817c0b7900 con 0x7f817c15e510 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f817c0d28c0 0x7f817c0b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f817c0d28c0 0x7f817c0b7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53532/0 (socket says 172.21.15.113:53532) 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 -- 172.21.15.113:0/63773907 learned_addr learned my addr 172.21.15.113:0/63773907 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f817a59c640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f817c0b63b0 0x7f817c1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f817ad9d640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 -- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 msgr2=0x7f817c0b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 -- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f817c0b63b0 msgr2=0x7f817c1603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f817c0b63b0 0x7f817c1603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 -- 172.21.15.113:0/63773907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c160ce0 con 0x7f817c0d28c0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8179d9b640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f817c0d28c0 0x7f817c0b7fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f817000c430 tx=0x7f817000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f816affd640 1 -- 172.21.15.113:0/63773907 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81700074e0 con 0x7f817c0d28c0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f816affd640 1 -- 172.21.15.113:0/63773907 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8170007680 con 0x7f817c0d28c0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f816affd640 1 -- 172.21.15.113:0/63773907 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170004a00 con 0x7f817c0d28c0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c0c7e80 con 0x7f817c0d28c0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.470+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c0c8440 con 0x7f817c0d28c0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.471+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817c0691a0 con 0x7f817c0d28c0 2024-04-24T23:59:14.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.472+0000 7f816affd640 1 -- 172.21.15.113:0/63773907 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f817001a020 con 0x7f817c0d28c0 2024-04-24T23:59:14.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.472+0000 7f816affd640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f815c06be30 0x7f815c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.472+0000 7f816affd640 1 -- 172.21.15.113:0/63773907 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f817008a5e0 con 0x7f817c0d28c0 2024-04-24T23:59:14.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.472+0000 7f817a59c640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f815c06be30 0x7f815c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.473+0000 7f817a59c640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f815c06be30 0x7f815c06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f816c034000 tx=0x7f816c033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.476+0000 7f816affd640 1 -- 172.21.15.113:0/63773907 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8170059020 con 0x7f817c0d28c0 2024-04-24T23:59:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.715+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f817c0d0c20 con 0x7f817c0d28c0 2024-04-24T23:59:14.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.717+0000 7f816affd640 1 -- 172.21.15.113:0/63773907 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f8170058920 con 0x7f817c0d28c0 2024-04-24T23:59:14.717 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-24T23:59:14.717 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-24T23:59:14.717 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-24T23:59:14.717 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-24T23:59:14.717 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T23:59:14.717 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-24T23:59:14.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f815c06be30 msgr2=0x7f815c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f815c06be30 0x7f815c06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f816c034000 tx=0x7f816c033040 comp rx=0 tx=0).stop 2024-04-24T23:59:14.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f817c0d28c0 msgr2=0x7f817c0b7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f817c0d28c0 0x7f817c0b7fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f817000c430 tx=0x7f817000c8f0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 shutdown_connections 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f815c06be30 0x7f815c06e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f817c15e510 0x7f817c0b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f817c0d28c0 0x7f817c0b7fc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 --2- 172.21.15.113:0/63773907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f817c0b63b0 0x7f817c1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 >> 172.21.15.113:0/63773907 conn(0x7f817c06b720 msgr2=0x7f817c15bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 shutdown_connections 2024-04-24T23:59:14.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.719+0000 7f8180b4f640 1 -- 172.21.15.113:0/63773907 wait complete. 2024-04-24T23:59:14.730 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:14.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.827+0000 7f60f7342640 1 Processor -- start 2024-04-24T23:59:14.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.827+0000 7f60f7342640 1 -- start start 2024-04-24T23:59:14.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f7342640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f006e6e0 0x7f60f00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f7342640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f0144cf0 con 0x7f60f0144910 2024-04-24T23:59:14.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f7342640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f00b2f30 con 0x7f60f00b3fd0 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f7342640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60f00b3210 con 0x7f60f006e6e0 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f50de640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f60f0144910 0x7f60f00b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40316/0 (socket says 172.21.15.113:40316) 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f50de640 1 -- 172.21.15.113:0/401545955 learned_addr learned my addr 172.21.15.113:0/401545955 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f58df640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f006e6e0 0x7f60f00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359641701 0 0) 0x7f60f0144cf0 con 0x7f60f0144910 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60dc003520 con 0x7f60f0144910 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2760553709 0 0) 0x7f60dc003520 con 0x7f60f0144910 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f006e6e0 msgr2=0x7f60f00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f006e6e0 0x7f60f00cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 >> v1:172.21.15.178:6789/0 conn(0x7f60f00b3fd0 legacy=0x7f60f006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f00b34f0 con 0x7f60f0144910 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f60ec001670 con 0x7f60f0144910 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 >> v1:172.21.15.113:6789/0 conn(0x7f60f0144910 legacy=0x7f60f00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0039e0 0x7f60dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60dc0043d0 0x7f60dc0048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0050a0 0x7f60dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60f00b3210 con 0x7f60dc0050a0 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc00bfb0 con 0x7f60dc0043d0 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc00c1b0 con 0x7f60dc0039e0 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.828+0000 7f60f58df640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f006e6e0 0x7f60f00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f58df640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0050a0 0x7f60dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f48dd640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60dc0043d0 0x7f60dc0048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f50de640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0039e0 0x7f60dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f58df640 1 -- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0039e0 msgr2=0x7f60dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f58df640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0039e0 0x7f60dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f58df640 1 -- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60dc0043d0 msgr2=0x7f60dc0048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f58df640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60dc0043d0 0x7f60dc0048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.829+0000 7f60f58df640 1 -- 172.21.15.113:0/401545955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc003520 con 0x7f60dc0050a0 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f58df640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0050a0 0x7f60dc00b9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f60f00730f0 tx=0x7f60dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798252754fdfbb59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8036070 con 0x7f60dc0050a0 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e8003c50 con 0x7f60dc0050a0 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60e7fff640 1 -- 172.21.15.113:0/401545955 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8009d40 con 0x7f60dc0050a0 2024-04-24T23:59:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f7342640 1 -- 172.21.15.113:0/401545955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0050a0 msgr2=0x7f60dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f7342640 1 --2- 172.21.15.113:0/401545955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0050a0 0x7f60dc00b9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f60f00730f0 tx=0x7f60dc005830 comp rx=0 tx=0).stop 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f58df640 1 -- 172.21.15.113:0/401545955 reap_dead start 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f7342640 1 -- 172.21.15.113:0/401545955 shutdown_connections 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f7342640 1 -- 172.21.15.113:0/401545955 >> 172.21.15.113:0/401545955 conn(0x7f60f00b1a50 msgr2=0x7f60f00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f7342640 1 -- 172.21.15.113:0/401545955 shutdown_connections 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.830+0000 7f60f7342640 1 -- 172.21.15.113:0/401545955 wait complete. 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.831+0000 7f60f7342640 1 Processor -- start 2024-04-24T23:59:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.831+0000 7f60f7342640 1 -- start start 2024-04-24T23:59:14.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f7342640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0050a0 0x7f60f0135840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f7342640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60f0154b90 0x7f60f016a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f7342640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f0134950 0x7f60f0134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f7342640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60f00b4da0 con 0x7f60f0154b90 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f7342640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60f00b4440 con 0x7f60f0134950 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f7342640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60f00b4f20 con 0x7f60dc0050a0 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f48dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60f0154b90 0x7f60f016a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f48dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60f0154b90 0x7f60f016a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53544/0 (socket says 172.21.15.113:53544) 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.832+0000 7f60f48dd640 1 -- 172.21.15.113:0/1891972023 learned_addr learned my addr 172.21.15.113:0/1891972023 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f50de640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0050a0 0x7f60f0135840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f58df640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f0134950 0x7f60f0134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f48dd640 1 -- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0050a0 msgr2=0x7f60f0135840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f48dd640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0050a0 0x7f60f0135840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f48dd640 1 -- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f0134950 msgr2=0x7f60f0134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f48dd640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f0134950 0x7f60f0134dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f48dd640 1 -- 172.21.15.113:0/1891972023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f00ae940 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f50de640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0050a0 0x7f60f0135840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f48dd640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60f0154b90 0x7f60f016a8c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f60e000f850 tx=0x7f60e000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60e5ffb640 1 -- 172.21.15.113:0/1891972023 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e000fe40 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60e5ffb640 1 -- 172.21.15.113:0/1891972023 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e0018070 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f00aebb0 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60e5ffb640 1 -- 172.21.15.113:0/1891972023 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e000d730 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.833+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f00af170 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.834+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f00691a0 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.834+0000 7f60e5ffb640 1 -- 172.21.15.113:0/1891972023 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60e000d8d0 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.835+0000 7f60e5ffb640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60d406be30 0x7f60d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.835+0000 7f60e5ffb640 1 -- 172.21.15.113:0/1891972023 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f60e008acc0 con 0x7f60f0154b90 2024-04-24T23:59:14.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.835+0000 7f60f50de640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60d406be30 0x7f60d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:14.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.835+0000 7f60f50de640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60d406be30 0x7f60d406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f60ec007d90 tx=0x7f60ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:14.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:14.839+0000 7f60e5ffb640 1 -- 172.21.15.113:0/1891972023 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60e0059670 con 0x7f60f0154b90 2024-04-24T23:59:15.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.049+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60f00731a0 con 0x7f60f0154b90 2024-04-24T23:59:15.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.050+0000 7f60e5ffb640 1 -- 172.21.15.113:0/1891972023 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5193 (secure 0 0 0) 0x7f60e0058f70 con 0x7f60f0154b90 2024-04-24T23:59:15.050 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T23:59:15.050 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2613614055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2613614055},{"type":"v1","addr":"172.21.15.178:6834","nonce":2613614055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2494825991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2494825991},{"type":"v1","addr":"172.21.15.178:6835","nonce":2494825991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2843749120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2843749120},{"type":"v1","addr":"172.21.15.113:6836","nonce":2843749120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:59:10.365260+0000","modified":"2024-04-24T23:59:12.371753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3337939581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3337939581},{"type":"v1","addr":"172.21.15.113:6837","nonce":3337939581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-04-24T23:59:15.050 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 14 2024-04-24T23:59:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60d406be30 msgr2=0x7f60d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60d406be30 0x7f60d406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f60ec007d90 tx=0x7f60ec009040 comp rx=0 tx=0).stop 2024-04-24T23:59:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60f0154b90 msgr2=0x7f60f016a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60f0154b90 0x7f60f016a8c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f60e000f850 tx=0x7f60e000fd10 comp rx=0 tx=0).stop 2024-04-24T23:59:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 shutdown_connections 2024-04-24T23:59:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60d406be30 0x7f60d406e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f0134950 0x7f60f0134dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60f0154b90 0x7f60f016a8c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 --2- 172.21.15.113:0/1891972023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60dc0050a0 0x7f60f0135840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 >> 172.21.15.113:0/1891972023 conn(0x7f60f00b1a50 msgr2=0x7f60f0072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 shutdown_connections 2024-04-24T23:59:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.052+0000 7f60f7342640 1 -- 172.21.15.113:0/1891972023 wait complete. 2024-04-24T23:59:15.394 INFO:tasks.ceph.mds.c.smithi113.stderr:2024-04-24T23:59:15.394+0000 7f2c91200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:59:15.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:59:10.365260+0000', 'modified': '2024-04-24T23:59:12.371753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5082}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5082': {'gid': 5082, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6837/3337939581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 3337939581}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3337939581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5082, 'qdb_cluster': [5082]} 2024-04-24T23:59:15.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:59:15.467 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T23:59:15.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.565+0000 7fdaba25d640 1 Processor -- start 2024-04-24T23:59:15.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.565+0000 7fdaba25d640 1 -- start start 2024-04-24T23:59:15.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdaba25d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdab415ba50 0x7fdab415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdaba25d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab4071170 con 0x7fdab406cea0 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdaba25d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab4071450 con 0x7fdab415b670 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdaba25d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdab4071730 con 0x7fdab415ba50 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdab2ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdab415ba50 0x7fdab415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdab3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdab406cea0 0x7fdab406d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40324/0 (socket says 172.21.15.113:40324) 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdab3fff640 1 -- 172.21.15.113:0/3942904625 learned_addr learned my addr 172.21.15.113:0/3942904625 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdab2ffd640 1 -- 172.21.15.113:0/3942904625 >> v1:172.21.15.178:6789/0 conn(0x7fdab415b670 legacy=0x7fdab40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdab2ffd640 1 -- 172.21.15.113:0/3942904625 >> v1:172.21.15.113:6789/0 conn(0x7fdab406cea0 legacy=0x7fdab406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdab2ffd640 1 -- 172.21.15.113:0/3942904625 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab4071a10 con 0x7fdab415ba50 2024-04-24T23:59:15.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.566+0000 7fdab27fc640 1 -- 172.21.15.113:0/3942904625 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908733014 0 0) 0x7fdab4071170 con 0x7fdab406cea0 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.567+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3942904625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab415ba50 0x7fdab415bff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800a940 tx=0x7fdaa802b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6766c4e97bea5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.567+0000 7fdab27fc640 1 -- 172.21.15.113:0/3942904625 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800b040 con 0x7fdab415ba50 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.567+0000 7fdab27fc640 1 -- 172.21.15.113:0/3942904625 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa8030710 con 0x7fdab415ba50 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.567+0000 7fdab27fc640 1 -- 172.21.15.113:0/3942904625 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8030ae0 con 0x7fdab415ba50 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.568+0000 7fdaba25d640 1 -- 172.21.15.113:0/3942904625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab415ba50 msgr2=0x7fdab415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.568+0000 7fdaba25d640 1 --2- 172.21.15.113:0/3942904625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab415ba50 0x7fdab415bff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800a940 tx=0x7fdaa802b480 comp rx=0 tx=0).stop 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.568+0000 7fdaba25d640 1 -- 172.21.15.113:0/3942904625 shutdown_connections 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.568+0000 7fdaba25d640 1 --2- 172.21.15.113:0/3942904625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab415ba50 0x7fdab415bff0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.568+0000 7fdaba25d640 1 -- 172.21.15.113:0/3942904625 >> 172.21.15.113:0/3942904625 conn(0x7fdab406b0f0 msgr2=0x7fdab406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.568+0000 7fdaba25d640 1 -- 172.21.15.113:0/3942904625 shutdown_connections 2024-04-24T23:59:15.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.568+0000 7fdaba25d640 1 -- 172.21.15.113:0/3942904625 wait complete. 2024-04-24T23:59:15.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.569+0000 7fdaba25d640 1 Processor -- start 2024-04-24T23:59:15.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.569+0000 7fdaba25d640 1 -- start start 2024-04-24T23:59:15.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdaba25d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab406cea0 0x7fdab4149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdaba25d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab415b670 0x7fdab40cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdaba25d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab415ba50 0x7fdab414a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdaba25d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40d0200 con 0x7fdab406cea0 2024-04-24T23:59:15.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdaba25d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40d0080 con 0x7fdab415b670 2024-04-24T23:59:15.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdaba25d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdab40d0380 con 0x7fdab415ba50 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab406cea0 0x7fdab4149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab406cea0 0x7fdab4149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53562/0 (socket says 172.21.15.113:53562) 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 -- 172.21.15.113:0/575900158 learned_addr learned my addr 172.21.15.113:0/575900158 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab3fff640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab415ba50 0x7fdab414a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab415b670 0x7fdab40cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 -- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab415ba50 msgr2=0x7fdab414a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab415ba50 0x7fdab414a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 -- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab415b670 msgr2=0x7fdab40cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab415b670 0x7fdab40cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 -- 172.21.15.113:0/575900158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab414a180 con 0x7fdab406cea0 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab3fff640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab415ba50 0x7fdab414a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab415b670 0x7fdab40cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:15.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdab37fe640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab406cea0 0x7fdab4149c40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c460 tx=0x7fdaa400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fda9bfff640 1 -- 172.21.15.113:0/575900158 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4016020 con 0x7fdab406cea0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fda9bfff640 1 -- 172.21.15.113:0/575900158 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa4012870 con 0x7fdab406cea0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab414d4d0 con 0x7fdab406cea0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.570+0000 7fda9bfff640 1 -- 172.21.15.113:0/575900158 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4012c60 con 0x7fdab406cea0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.571+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab4147940 con 0x7fdab406cea0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.572+0000 7fda9bfff640 1 -- 172.21.15.113:0/575900158 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdaa4019020 con 0x7fdab406cea0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.572+0000 7fda9bfff640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.572+0000 7fda9bfff640 1 -- 172.21.15.113:0/575900158 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdaa408ab50 con 0x7fdab406cea0 2024-04-24T23:59:15.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.572+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.573+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8035000 tx=0x7fdaa800c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.573+0000 7fdabcf6e740 1 Processor -- start 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.573+0000 7fdabcf6e740 1 -- start start 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183d540 0x563a11812350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563a11775f00 con 0x563a1185ab60 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a1171e750 con 0x563a1183dc80 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563a11771ec0 con 0x563a1183d540 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53564/0 (socket says 172.21.15.113:53564) 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.574+0000 7fdab37fe640 1 -- 172.21.15.113:0/1600825935 learned_addr learned my addr 172.21.15.113:0/1600825935 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:15.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdab37fe640 1 -- 172.21.15.113:0/1600825935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183d540 msgr2=0x563a11812350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdab37fe640 1 --2- 172.21.15.113:0/1600825935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183d540 0x563a11812350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdab37fe640 1 -- 172.21.15.113:0/1600825935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 msgr2=0x563a1183d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdab37fe640 1 --2- 172.21.15.113:0/1600825935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdab37fe640 1 -- 172.21.15.113:0/1600825935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a1148a2d0 con 0x563a1185ab60 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdab37fe640 1 --2- 172.21.15.113:0/1600825935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180d4f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4089130 tx=0x7fdaa4059640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b9042f68a0d466 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fda997fa640 1 -- 172.21.15.113:0/1600825935 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4074020 con 0x563a1185ab60 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fda997fa640 1 -- 172.21.15.113:0/1600825935 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa4089190 con 0x563a1185ab60 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fda997fa640 1 -- 172.21.15.113:0/1600825935 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4057110 con 0x563a1185ab60 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdabcf6e740 1 -- 172.21.15.113:0/1600825935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 msgr2=0x563a1180d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/1600825935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180d4f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4089130 tx=0x7fdaa4059640 comp rx=0 tx=0).stop 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdabcf6e740 1 -- 172.21.15.113:0/1600825935 shutdown_connections 2024-04-24T23:59:15.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/1600825935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/1600825935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183d540 0x563a11812350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/1600825935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180d4f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.575+0000 7fdabcf6e740 1 -- 172.21.15.113:0/1600825935 >> 172.21.15.113:0/1600825935 conn(0x563a1180f760 msgr2=0x563a11858000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.576+0000 7fdabcf6e740 1 -- 172.21.15.113:0/1600825935 shutdown_connections 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.576+0000 7fdabcf6e740 1 -- 172.21.15.113:0/1600825935 wait complete. 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.576+0000 7fdabcf6e740 1 -- start start 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.576+0000 7fdabcf6e740 1 Processor -- start 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1185ab60 0x563a1182a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1186b7c0 0x563a1184f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563a11811910 con 0x563a1186b7c0 2024-04-24T23:59:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a11731cf0 con 0x563a1183dc80 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563a11810d00 con 0x563a1185ab60 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdabcf6e740 10 client.0 _ll_register_callbacks cb 0x563a117316e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1186b7c0 0x563a1184f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1186b7c0 0x563a1184f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53578/0 (socket says 172.21.15.113:53578) 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 -- 172.21.15.113:0/176484017 learned_addr learned my addr 172.21.15.113:0/176484017 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 -- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1185ab60 msgr2=0x563a1182a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1185ab60 0x563a1182a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab37fe640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1185ab60 0x563a1182a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 -- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 msgr2=0x563a1183f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 -- 172.21.15.113:0/176484017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a115546b0 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1185ab60 0x563a1182a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab37fe640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fdab3fff640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1186b7c0 0x563a1184f0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00d4b0 tx=0x7fdaac00d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665cca18123e642e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.577+0000 7fda927fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fda927fc640 1 -- 172.21.15.113:0/176484017 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0070d0 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fda927fc640 1 -- 172.21.15.113:0/176484017 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdaac013a80 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa4089190 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fda927fc640 1 -- 172.21.15.113:0/176484017 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fdaac013c80 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 10 client.5259 fetch_fsmap learned FSMap version 14 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x563a1186c480 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fda927fc640 1 -- 172.21.15.113:0/176484017 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa4089190 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fda927fc640 1 -- 172.21.15.113:0/176484017 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7fdaac013420 con 0x563a1186b7c0 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 10 client.5259 fetch_fsmap finished waiting for FSMap version 14 2024-04-24T23:59:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 10 client.5259 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 20 client.5259 populate_metadata read hostname 'smithi113' 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1183e600 0x563a1183e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 4 client.5259 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a11783440 con 0x563a1183e600 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdab37fe640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1183e600 0x563a1183e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fdab37fe640 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1183e600 0x563a1183e9e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6039fe767dfb9b0e server_cookie=c4892cbb818e3ac9 in_seq=0 out_seq=0 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.578+0000 7fda927fc640 10 client.5259 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fda927fc640 1 -- 172.21.15.113:0/176484017 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a11783440 con 0x563a1183e600 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fda927fc640 10 client.5259 handle_command_reply: tid=0 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 client.5259 shutdown 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1186b7c0 msgr2=0x563a1184f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1186b7c0 0x563a1184f0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00d4b0 tx=0x7fdaac00d970 comp rx=0 tx=0).stop 2024-04-24T23:59:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 shutdown_connections 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1183e600 0x563a1183e9e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1186b7c0 0x563a1184f0e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1185ab60 0x563a1182a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/176484017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1183dc80 0x563a1183f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fda927fc640 0 client.5259 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 >> 172.21.15.113:0/176484017 conn(0x563a1183d540 msgr2=0x563a1185a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 shutdown_connections 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 1 -- 172.21.15.113:0/176484017 wait complete. 2024-04-24T23:59:15.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.579+0000 7fdabcf6e740 20 client.5259 trim_cache size 0 max 16384 2024-04-24T23:59:15.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.597+0000 7fdabcf6e740 1 Processor -- start 2024-04-24T23:59:15.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.598+0000 7fdabcf6e740 1 -- start start 2024-04-24T23:59:15.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.598+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183dc80 0x563a1180cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.598+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.598+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1186b7c0 0x563a1183fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.598+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdab40cd6d0 con 0x563a1185ab60 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.598+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4159370 con 0x563a1186b7c0 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.598+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdab40cd080 con 0x563a1183dc80 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53586/0 (socket says 172.21.15.113:53586) 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 -- 172.21.15.113:0/3805240964 learned_addr learned my addr 172.21.15.113:0/3805240964 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab37fe640 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183dc80 0x563a1180cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab3fff640 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1186b7c0 0x563a1183fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 -- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183dc80 msgr2=0x563a1180cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183dc80 0x563a1180cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 -- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1186b7c0 msgr2=0x563a1183fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1186b7c0 0x563a1183fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 -- 172.21.15.113:0/3805240964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a1187c3f0 con 0x563a1185ab60 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab3fff640 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1186b7c0 0x563a1183fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180c330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdaa803e040 tx=0x7fdaa802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476b60701764803b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fda98ff9640 1 -- 172.21.15.113:0/3805240964 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa804b030 con 0x563a1185ab60 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fda98ff9640 1 -- 172.21.15.113:0/3805240964 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa8030cc0 con 0x563a1185ab60 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.599+0000 7fda98ff9640 1 -- 172.21.15.113:0/3805240964 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8040040 con 0x563a1185ab60 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3805240964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 msgr2=0x563a1180c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180c330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdaa803e040 tx=0x7fdaa802d000 comp rx=0 tx=0).stop 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3805240964 shutdown_connections 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1186b7c0 0x563a1183fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1185ab60 0x563a1180c330 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3805240964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1183dc80 0x563a1180cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3805240964 >> 172.21.15.113:0/3805240964 conn(0x563a1183d540 msgr2=0x563a118511e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3805240964 shutdown_connections 2024-04-24T23:59:15.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3805240964 wait complete. 2024-04-24T23:59:15.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 -- start start 2024-04-24T23:59:15.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.600+0000 7fdabcf6e740 1 Processor -- start 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1183dc80 0x563a1183e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1185ab60 0x563a1183a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdabcf6e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1186b7c0 0x563a118b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563a11869050 con 0x563a1183dc80 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a1183fcd0 con 0x563a1185ab60 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdabcf6e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563a11813100 con 0x563a1186b7c0 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdabcf6e740 10 client.0 _ll_register_callbacks cb 0x7fdaac002000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:59:15.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1183dc80 0x563a1183e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1183dc80 0x563a1183e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53600/0 (socket says 172.21.15.113:53600) 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdab37fe640 1 -- 172.21.15.113:0/3314154993 learned_addr learned my addr 172.21.15.113:0/3314154993 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1185ab60 0x563a1183a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.601+0000 7fdab3fff640 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1186b7c0 0x563a118b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdab37fe640 1 -- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1186b7c0 msgr2=0x563a118b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdab37fe640 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1186b7c0 0x563a118b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdab37fe640 1 -- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1185ab60 msgr2=0x563a1183a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdab37fe640 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1185ab60 0x563a1183a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdab37fe640 1 -- 172.21.15.113:0/3314154993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a1183f2e0 con 0x563a1183dc80 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdab37fe640 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1183dc80 0x563a1183e9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40588e0 tx=0x7fdaa4003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5821b3b6ba3d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fda927fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fda927fc640 1 -- 172.21.15.113:0/3314154993 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4087040 con 0x563a1183dc80 2024-04-24T23:59:15.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fda927fc640 1 -- 172.21.15.113:0/3314154993 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdaa4055480 con 0x563a1183dc80 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa8030cc0 con 0x563a1183dc80 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fda927fc640 1 -- 172.21.15.113:0/3314154993 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fdaa4067050 con 0x563a1183dc80 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 10 client.5265 fetch_fsmap learned FSMap version 14 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x563a1183a6b0 con 0x563a1183dc80 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fda927fc640 1 -- 172.21.15.113:0/3314154993 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa8030cc0 con 0x563a1183dc80 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fda927fc640 1 -- 172.21.15.113:0/3314154993 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7fdaa406a020 con 0x563a1183dc80 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 10 client.5265 fetch_fsmap finished waiting for FSMap version 14 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 10 client.5265 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 20 client.5265 populate_metadata read hostname 'smithi113' 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1180d0d0 0x563a1187ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 4 client.5265 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "status"}) -- 0x563a1177d790 con 0x563a1180d0d0 2024-04-24T23:59:15.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.602+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1180d0d0 0x563a1187ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fda927fc640 10 client.5265 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1180d0d0 0x563a1187ac70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56465cea71e43a61 server_cookie=8474ac3cfddf540b in_seq=0 out_seq=0 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fda927fc640 1 -- 172.21.15.113:0/3314154993 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563a1177d790 con 0x563a1180d0d0 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fda927fc640 10 client.5265 handle_command_reply: tid=0 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 client.5265 shutdown 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1183dc80 msgr2=0x563a1183e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1183dc80 0x563a1183e9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40588e0 tx=0x7fdaa4003920 comp rx=0 tx=0).stop 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 shutdown_connections 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x563a1180d0d0 0x563a1187ac70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a1185ab60 0x563a1183a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563a1186b7c0 0x563a118b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fda927fc640 0 client.5265 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-24T23:59:15.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 --2- 172.21.15.113:0/3314154993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a1183dc80 0x563a1183e9e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 >> 172.21.15.113:0/3314154993 conn(0x563a1183d540 msgr2=0x563a11864760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 shutdown_connections 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 1 -- 172.21.15.113:0/3314154993 wait complete. 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.603+0000 7fdabcf6e740 20 client.5265 trim_cache size 0 max 16384 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 5082, 2024-04-24T23:59:15.610 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.5859046049999996, 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 14, 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 32, 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 32, 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 12.076418405 2024-04-24T23:59:15.611 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 msgr2=0x7fdaa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8035000 tx=0x7fdaa800c040 comp rx=0 tx=0).stop 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab406cea0 msgr2=0x7fdab4149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab406cea0 0x7fdab4149c40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c460 tx=0x7fdaa400c920 comp rx=0 tx=0).stop 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 shutdown_connections 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8035000 tx=0x7fdaa800c040 comp rx=0 tx=0).stop 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab415b670 0x7fdab40cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab415ba50 0x7fdab414a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 --2- 172.21.15.113:0/575900158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab406cea0 0x7fdab4149c40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 >> 172.21.15.113:0/575900158 conn(0x7fdab406b0f0 msgr2=0x7fdab406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 shutdown_connections 2024-04-24T23:59:15.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.604+0000 7fdaba25d640 1 -- 172.21.15.113:0/575900158 wait complete. 2024-04-24T23:59:15.614 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T23:59:15.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.710+0000 7f0715342640 1 Processor -- start 2024-04-24T23:59:15.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.710+0000 7f0715342640 1 -- start start 2024-04-24T23:59:15.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f0715342640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f071006e6e0 0x7f07100b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f0715342640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07100cfad0 con 0x7f0710144d60 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f0715342640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07100cf230 con 0x7f07101453c0 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f0715342640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0710145c40 con 0x7f071006e6e0 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f070effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0710144d60 0x7f071006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40340/0 (socket says 172.21.15.113:40340) 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f070effd640 1 -- 172.21.15.113:0/595360095 learned_addr learned my addr 172.21.15.113:0/595360095 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f070e7fc640 1 --2- 172.21.15.113:0/595360095 >> v2:172.21.15.178:3300/0 conn(0x7f071006e6e0 0x7f07100b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f070dffb640 1 -- 172.21.15.113:0/595360095 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2458556999 0 0) 0x7f07100cfad0 con 0x7f0710144d60 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.711+0000 7f070dffb640 1 -- 172.21.15.113:0/595360095 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06f8003520 con 0x7f0710144d60 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.712+0000 7f070e7fc640 1 -- 172.21.15.113:0/595360095 >> v1:172.21.15.178:6789/0 conn(0x7f07101453c0 legacy=0x7f0710071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.712+0000 7f070e7fc640 1 -- 172.21.15.113:0/595360095 >> v1:172.21.15.113:6789/0 conn(0x7f0710144d60 legacy=0x7f071006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.712+0000 7f070dffb640 1 -- 172.21.15.113:0/595360095 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 43593688 0 0) 0x7f06f8003520 con 0x7f0710144d60 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.712+0000 7f070e7fc640 1 -- 172.21.15.113:0/595360095 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07100cf510 con 0x7f071006e6e0 2024-04-24T23:59:15.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.712+0000 7f070e7fc640 1 --2- 172.21.15.113:0/595360095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071006e6e0 0x7f07100b5390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f07040097a0 tx=0x7f070402a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6093b98382cb970 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f070dffb640 1 -- 172.21.15.113:0/595360095 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704037070 con 0x7f071006e6e0 2024-04-24T23:59:15.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f070dffb640 1 -- 172.21.15.113:0/595360095 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070402ab30 con 0x7f071006e6e0 2024-04-24T23:59:15.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f070dffb640 1 -- 172.21.15.113:0/595360095 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000019f0 con 0x7f071006e6e0 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f0715342640 1 -- 172.21.15.113:0/595360095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071006e6e0 msgr2=0x7f07100b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f0715342640 1 --2- 172.21.15.113:0/595360095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071006e6e0 0x7f07100b5390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f07040097a0 tx=0x7f070402a0a0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f0715342640 1 -- 172.21.15.113:0/595360095 shutdown_connections 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f0715342640 1 --2- 172.21.15.113:0/595360095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071006e6e0 0x7f07100b5390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f0715342640 1 -- 172.21.15.113:0/595360095 >> 172.21.15.113:0/595360095 conn(0x7f07100b1a50 msgr2=0x7f07100b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f0715342640 1 -- 172.21.15.113:0/595360095 shutdown_connections 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.713+0000 7f0715342640 1 -- 172.21.15.113:0/595360095 wait complete. 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.714+0000 7f0715342640 1 Processor -- start 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.714+0000 7f0715342640 1 -- start start 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f0715342640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071006e6e0 0x7f07101324e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f0715342640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0710144d60 0x7f0710143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f0715342640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07101453c0 0x7f071006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f0715342640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0710145140 con 0x7f071006e6e0 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f0715342640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0710070e10 con 0x7f0710144d60 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f0715342640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0710071a40 con 0x7f07101453c0 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f070effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071006e6e0 0x7f07101324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f070effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071006e6e0 0x7f07101324e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53616/0 (socket says 172.21.15.113:53616) 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.715+0000 7f070effd640 1 -- 172.21.15.113:0/1393577883 learned_addr learned my addr 172.21.15.113:0/1393577883 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070f7fe640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07101453c0 0x7f071006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070e7fc640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0710144d60 0x7f0710143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070effd640 1 -- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07101453c0 msgr2=0x7f071006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070effd640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07101453c0 0x7f071006d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070effd640 1 -- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0710144d60 msgr2=0x7f0710143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070effd640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0710144d60 0x7f0710143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070effd640 1 -- 172.21.15.113:0/1393577883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0710132a20 con 0x7f071006e6e0 2024-04-24T23:59:15.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f070effd640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071006e6e0 0x7f07101324e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0700001890 tx=0x7f070000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f06f77fe640 1 -- 172.21.15.113:0/1393577883 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700017070 con 0x7f071006e6e0 2024-04-24T23:59:15.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f06f77fe640 1 -- 172.21.15.113:0/1393577883 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0700012400 con 0x7f071006e6e0 2024-04-24T23:59:15.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0710143f70 con 0x7f071006e6e0 2024-04-24T23:59:15.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f06f77fe640 1 -- 172.21.15.113:0/1393577883 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700012830 con 0x7f071006e6e0 2024-04-24T23:59:15.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.716+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0710156790 con 0x7f071006e6e0 2024-04-24T23:59:15.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.717+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07100691a0 con 0x7f071006e6e0 2024-04-24T23:59:15.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.717+0000 7f06f77fe640 1 -- 172.21.15.113:0/1393577883 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f070001a070 con 0x7f071006e6e0 2024-04-24T23:59:15.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.718+0000 7f06f77fe640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06fc06be30 0x7f06fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:15.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.718+0000 7f06f77fe640 1 -- 172.21.15.113:0/1393577883 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f070008add0 con 0x7f071006e6e0 2024-04-24T23:59:15.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.718+0000 7f070e7fc640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06fc06be30 0x7f06fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:15.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.718+0000 7f070e7fc640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06fc06be30 0x7f06fc06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0704034000 tx=0x7f070402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:15.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.722+0000 7f06f77fe640 1 -- 172.21.15.113:0/1393577883 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0700059810 con 0x7f071006e6e0 2024-04-24T23:59:15.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.932+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0710140aa0 con 0x7f071006e6e0 2024-04-24T23:59:15.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.932+0000 7f06f77fe640 1 -- 172.21.15.113:0/1393577883 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f070008c070 con 0x7f071006e6e0 2024-04-24T23:59:15.932 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T23:59:15.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06fc06be30 msgr2=0x7f06fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06fc06be30 0x7f06fc06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0704034000 tx=0x7f070402e000 comp rx=0 tx=0).stop 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071006e6e0 msgr2=0x7f07101324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071006e6e0 0x7f07101324e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0700001890 tx=0x7f070000a430 comp rx=0 tx=0).stop 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 shutdown_connections 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06fc06be30 0x7f06fc06e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0710144d60 0x7f0710143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07101453c0 0x7f071006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 --2- 172.21.15.113:0/1393577883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071006e6e0 0x7f07101324e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 >> 172.21.15.113:0/1393577883 conn(0x7f07100b1a50 msgr2=0x7f071006cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 shutdown_connections 2024-04-24T23:59:15.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:15.934+0000 7f0715342640 1 -- 172.21.15.113:0/1393577883 wait complete. 2024-04-24T23:59:15.946 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:59:15.946 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:59:15.946 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-24T23:59:15.946 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:59:15.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:59:15.946 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:59:15.947 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-24T23:59:15.947 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:59:15.947 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:59:15.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:15.947 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-24T23:59:15.964 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:59:15.964 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:59:15.964 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:59:15.964 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:59:15.965 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-24T23:59:15.965 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:59:15.965 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:59:15.965 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:59:15.965 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-24T23:59:15.965 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:59:15.966 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:59:15.966 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-24T23:59:15.966 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:59:15.966 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:59:15.966 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:59:15.967 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:59:15.967 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:59:15.967 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42724sec preferred_lft 42724sec 2024-04-24T23:59:15.967 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-24T23:59:15.967 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:59:15.967 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T23:59:15.968 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-24T23:59:15.968 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T23:59:15.968 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:59:15.968 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-24T23:59:15.968 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:59:15.968 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:59:15.969 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:59:15.969 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-24T23:59:15.969 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:59:15.969 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T23:59:15.969 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T23:59:15.969 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-24T23:59:15.969 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:59:15.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:15.970 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:59:16.026 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-24T23:59:16.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:16.027 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-24T23:59:16.083 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-24T23:59:16.083 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:59:16.083 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:59:16.083 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:59:16.084 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:59:16.084 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:59:16.084 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-24T23:59:16.084 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:59:16.084 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:59:16.084 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:59:16.084 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-24T23:59:16.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:59:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:59:16.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:16.207 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-24T23:59:16.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T23:59:16.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:59:16.222 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:59:16.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:16.222 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-24T23:59:16.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:59:16.286 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:59:16.340 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:59:16.341 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:59:16.341 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:59:16.395 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-24T23:59:16.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:16.395 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:59:16.475 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:59:16.475 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:59:16.475 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:59:16.483 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:59:16.483 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:59:16.483 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:59:16.483 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:59:16.483 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-24T23:59:16.488 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-24T23:59:16.489 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:59:16.489 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:59:16.489 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:59:16.489 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:59:16.489 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:59:16.490 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:59:16.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:16.491 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:59:16.555 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:59:16.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:59:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:59:16.654 INFO:teuthology.orchestra.run.smithi113.stdout:client5274 2024-04-24T23:59:16.655 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:59:16.655 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:59:16.670 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-24T23:59:16.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.810+0000 7fc895f29640 1 Processor -- start 2024-04-24T23:59:16.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc895f29640 1 -- start start 2024-04-24T23:59:16.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc895f29640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc89015e510 0x7fc890161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc895f29640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc8900b63b0 0x7fc8900d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc895f29640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8901626f0 con 0x7fc8900b9bb0 2024-04-24T23:59:16.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc895f29640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc8901629d0 con 0x7fc89015e510 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc895f29640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc890162cb0 con 0x7fc8900b63b0 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc88f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc89015e510 0x7fc890161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc88f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc89015e510 0x7fc890161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53624/0 (socket says 172.21.15.113:53624) 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.811+0000 7fc88f7fe640 1 -- 172.21.15.113:0/1235302499 learned_addr learned my addr 172.21.15.113:0/1235302499 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.812+0000 7fc88ffff640 1 --2- 172.21.15.113:0/1235302499 >> v2:172.21.15.178:3300/0 conn(0x7fc8900b63b0 0x7fc8900d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.812+0000 7fc88f7fe640 1 -- 172.21.15.113:0/1235302499 >> v2:172.21.15.178:3300/0 conn(0x7fc8900b63b0 msgr2=0x7fc8900d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.812+0000 7fc88f7fe640 1 --2- 172.21.15.113:0/1235302499 >> v2:172.21.15.178:3300/0 conn(0x7fc8900b63b0 0x7fc8900d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.812+0000 7fc88f7fe640 1 -- 172.21.15.113:0/1235302499 >> v1:172.21.15.178:6789/0 conn(0x7fc8900b9bb0 legacy=0x7fc8900b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.812+0000 7fc88f7fe640 1 -- 172.21.15.113:0/1235302499 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8900d2660 con 0x7fc89015e510 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.812+0000 7fc88f7fe640 1 --2- 172.21.15.113:0/1235302499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc89015e510 0x7fc890161ea0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc890162f90 tx=0x7fc88002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf7f72879289024 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.812+0000 7fc88e7fc640 1 -- 172.21.15.113:0/1235302499 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880037070 con 0x7fc89015e510 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc88e7fc640 1 -- 172.21.15.113:0/1235302499 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc880032440 con 0x7fc89015e510 2024-04-24T23:59:16.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc88e7fc640 1 -- 172.21.15.113:0/1235302499 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880032810 con 0x7fc89015e510 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 -- 172.21.15.113:0/1235302499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc89015e510 msgr2=0x7fc890161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 --2- 172.21.15.113:0/1235302499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc89015e510 0x7fc890161ea0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc890162f90 tx=0x7fc88002a1e0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 -- 172.21.15.113:0/1235302499 shutdown_connections 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 --2- 172.21.15.113:0/1235302499 >> v2:172.21.15.178:3300/0 conn(0x7fc8900b63b0 0x7fc8900d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 --2- 172.21.15.113:0/1235302499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc89015e510 0x7fc890161ea0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 -- 172.21.15.113:0/1235302499 >> 172.21.15.113:0/1235302499 conn(0x7fc89006b720 msgr2=0x7fc8900d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 -- 172.21.15.113:0/1235302499 shutdown_connections 2024-04-24T23:59:16.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.813+0000 7fc895f29640 1 -- 172.21.15.113:0/1235302499 wait complete. 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.814+0000 7fc895f29640 1 Processor -- start 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.814+0000 7fc895f29640 1 -- start start 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc895f29640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8900b63b0 0x7fc8900b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc895f29640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8900b9bb0 0x7fc8900c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc895f29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc89015e510 0x7fc8901503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc895f29640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8900b73c0 con 0x7fc8900b9bb0 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc895f29640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901623e0 con 0x7fc8900b63b0 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc895f29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8900b7540 con 0x7fc89015e510 2024-04-24T23:59:16.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc88effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8900b9bb0 0x7fc8900c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc88effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8900b9bb0 0x7fc8900c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53638/0 (socket says 172.21.15.113:53638) 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc88effd640 1 -- 172.21.15.113:0/20657482 learned_addr learned my addr 172.21.15.113:0/20657482 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc88f7fe640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8900b63b0 0x7fc8900b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.815+0000 7fc88ffff640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc89015e510 0x7fc8901503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc88effd640 1 -- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc89015e510 msgr2=0x7fc8901503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc88effd640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc89015e510 0x7fc8901503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc88effd640 1 -- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8900b63b0 msgr2=0x7fc8900b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc88effd640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8900b63b0 0x7fc8900b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc88effd640 1 -- 172.21.15.113:0/20657482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8901508e0 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc88effd640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8900b9bb0 0x7fc8900c8030 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c470 tx=0x7fc884001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc877fff640 1 -- 172.21.15.113:0/20657482 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8840052d0 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc877fff640 1 -- 172.21.15.113:0/20657482 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc884005470 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc877fff640 1 -- 172.21.15.113:0/20657482 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884007260 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8900cdc20 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.816+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8900c9350 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.817+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8900cf880 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.817+0000 7fc877fff640 1 -- 172.21.15.113:0/20657482 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc884007400 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.818+0000 7fc877fff640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87c06be60 0x7fc87c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.818+0000 7fc877fff640 1 -- 172.21.15.113:0/20657482 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc88408a1f0 con 0x7fc8900b9bb0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.818+0000 7fc88f7fe640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87c06be60 0x7fc87c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.818+0000 7fc88f7fe640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87c06be60 0x7fc87c06e320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc880034000 tx=0x7fc88002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:16.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:16.822+0000 7fc877fff640 1 -- 172.21.15.113:0/20657482 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc8900cf880 con 0x7fc8900b9bb0 2024-04-24T23:59:17.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.030+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc8900691a0 con 0x7fc8900b9bb0 2024-04-24T23:59:17.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.051+0000 7fc877fff640 1 -- 172.21.15.113:0/20657482 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v15) ==== 167+0+0 (secure 0 0 0) 0x7fc8901508e0 con 0x7fc8900b9bb0 2024-04-24T23:59:17.051 INFO:teuthology.orchestra.run.smithi113.stderr:enabled new snapshots 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.053+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87c06be60 msgr2=0x7fc87c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.053+0000 7fc895f29640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87c06be60 0x7fc87c06e320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc880034000 tx=0x7fc88002e000 comp rx=0 tx=0).stop 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.053+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8900b9bb0 msgr2=0x7fc8900c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.053+0000 7fc895f29640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8900b9bb0 0x7fc8900c8030 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c470 tx=0x7fc884001e70 comp rx=0 tx=0).stop 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 shutdown_connections 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87c06be60 0x7fc87c06e320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc89015e510 0x7fc8901503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8900b9bb0 0x7fc8900c8030 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 --2- 172.21.15.113:0/20657482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8900b63b0 0x7fc8900b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 >> 172.21.15.113:0/20657482 conn(0x7fc89006b720 msgr2=0x7fc8900d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 shutdown_connections 2024-04-24T23:59:17.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:17.054+0000 7fc895f29640 1 -- 172.21.15.113:0/20657482 wait complete. 2024-04-24T23:59:17.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:17.064 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-24T23:59:17.085 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T23:59:17.085 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> import uuid 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> n = 2000 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> create_hard_links = False 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> if 3 > 0: 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> create_hard_links = True 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-24T23:59:17.141 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> dpath = os.path.dirname(path) 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> fnameprefix = os.path.basename(path) 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> os.makedirs(dpath, exist_ok=True) 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> try: 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> for i in range(n): 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> with open(fpath, '"'"'w'"'"') as f: 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> f.write(f"{i}") 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> f.flush() 2024-04-24T23:59:17.142 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(f.fileno()) 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> os.unlink(fpath) 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(dirfd) 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> if create_hard_links: 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> for j in range(3): 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(dirfd) 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> finally: 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> os.close(dirfd) 2024-04-24T23:59:17.143 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-24T23:59:21.295 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:59:21.298 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:59:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T23:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:59:35.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:35.367 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-24T23:59:35.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:59:35.465 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-24T23:59:51.390 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T23:59:51.393 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:00:02.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:02.737 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T00:00:02.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T00:00:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.918+0000 7fdde0d5a640 1 Processor -- start 2024-04-25T00:00:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.918+0000 7fdde0d5a640 1 -- start start 2024-04-25T00:00:02.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fdde0d5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 0x7fdddc15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fdde0d5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdddc0b4a40 0x7fdddc0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fdde0d5a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdddc15f4f0 con 0x7fdddc0b3080 2024-04-25T00:00:02.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fdde0d5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdddc0cde00 con 0x7fdddc0b4a40 2024-04-25T00:00:02.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fdde0d5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc15f7d0 con 0x7fdddc15e2d0 2024-04-25T00:00:02.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd9d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdddc0b3080 0x7fdddc0b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54782/0 (socket says 172.21.15.113:54782) 2024-04-25T00:00:02.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd9d9b640 1 -- 172.21.15.113:0/1275589591 learned_addr learned my addr 172.21.15.113:0/1275589591 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:02.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031146759 0 0) 0x7fdddc15f4f0 con 0x7fdddc0b3080 2024-04-25T00:00:02.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc4003520 con 0x7fdddc0b3080 2024-04-25T00:00:02.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddda59c640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 0x7fdddc15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fdddad9d640 1 --2- 172.21.15.113:0/1275589591 >> v2:172.21.15.178:3300/0 conn(0x7fdddc0b4a40 0x7fdddc0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4166521013 0 0) 0x7fddc4003520 con 0x7fdddc0b3080 2024-04-25T00:00:02.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 msgr2=0x7fdddc15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 0x7fdddc15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 >> v2:172.21.15.178:3300/0 conn(0x7fdddc0b4a40 msgr2=0x7fdddc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 --2- 172.21.15.113:0/1275589591 >> v2:172.21.15.178:3300/0 conn(0x7fdddc0b4a40 0x7fdddc0ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc15fab0 con 0x7fdddc0b3080 2024-04-25T00:00:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fddd0001670 con 0x7fdddc0b3080 2024-04-25T00:00:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 >> v1:172.21.15.113:6789/0 conn(0x7fdddc0b3080 legacy=0x7fdddc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 0x7fddc40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc4003f30 0x7fddc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.919+0000 7fddd959a640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4004d40 0x7fddc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0cde00 con 0x7fddc4003f30 2024-04-25T00:00:02.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc15f7d0 con 0x7fddc4004d40 2024-04-25T00:00:02.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fddc4004560 con 0x7fdddc15e2d0 2024-04-25T00:00:02.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc4003f30 0x7fddc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddda59c640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 0x7fddc40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fdddad9d640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4004d40 0x7fddc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd9d9b640 1 -- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 msgr2=0x7fddc40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc15e2d0 0x7fddc40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd9d9b640 1 -- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4004d40 msgr2=0x7fddc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4004d40 0x7fddc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.920+0000 7fddd9d9b640 1 -- 172.21.15.113:0/1275589591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc4003520 con 0x7fddc4003f30 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.921+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc4003f30 0x7fddc4004800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b34c0 tx=0x7fddc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3084ef410cdc20db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.921+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0007d50 con 0x7fddc4003f30 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.921+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd00058d0 con 0x7fddc4003f30 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.921+0000 7fddd959a640 1 -- 172.21.15.113:0/1275589591 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0005cc0 con 0x7fddc4003f30 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.921+0000 7fdde0d5a640 1 -- 172.21.15.113:0/1275589591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc4003f30 msgr2=0x7fddc4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.921+0000 7fdde0d5a640 1 --2- 172.21.15.113:0/1275589591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc4003f30 0x7fddc4004800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b34c0 tx=0x7fddc4005830 comp rx=0 tx=0).stop 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.921+0000 7fdddad9d640 1 -- 172.21.15.113:0/1275589591 reap_dead start 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.922+0000 7fdde0d5a640 1 -- 172.21.15.113:0/1275589591 shutdown_connections 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.922+0000 7fdde0d5a640 1 -- 172.21.15.113:0/1275589591 >> 172.21.15.113:0/1275589591 conn(0x7fdddc06b0f0 msgr2=0x7fdddc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.922+0000 7fdde0d5a640 1 -- 172.21.15.113:0/1275589591 shutdown_connections 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.922+0000 7fdde0d5a640 1 -- 172.21.15.113:0/1275589591 wait complete. 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.923+0000 7fdde0d5a640 1 Processor -- start 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.923+0000 7fdde0d5a640 1 -- start start 2024-04-25T00:00:02.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdde0d5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4003f30 0x7fdddc153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdde0d5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc0a9480 0x7fdddc157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdde0d5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc154600 0x7fdddc157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdde0d5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc15e830 con 0x7fdddc154600 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdde0d5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc15e6b0 con 0x7fddc4003f30 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdde0d5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc15e9b0 con 0x7fdddc0a9480 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc154600 0x7fdddc157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc154600 0x7fdddc157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36886/0 (socket says 172.21.15.113:36886) 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 -- 172.21.15.113:0/778578195 learned_addr learned my addr 172.21.15.113:0/778578195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc0a9480 0x7fdddc157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fddda59c640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4003f30 0x7fdddc153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 -- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc0a9480 msgr2=0x7fdddc157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc0a9480 0x7fdddc157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 -- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4003f30 msgr2=0x7fdddc153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4003f30 0x7fdddc153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.924+0000 7fdddad9d640 1 -- 172.21.15.113:0/778578195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc0a9940 con 0x7fdddc154600 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.925+0000 7fdddad9d640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc154600 0x7fdddc157d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdddc15f700 tx=0x7fddd400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.925+0000 7fddcaffd640 1 -- 172.21.15.113:0/778578195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd400fcb0 con 0x7fdddc154600 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.925+0000 7fddcaffd640 1 -- 172.21.15.113:0/778578195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd401d070 con 0x7fdddc154600 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.925+0000 7fddcaffd640 1 -- 172.21.15.113:0/778578195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4015670 con 0x7fdddc154600 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.925+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc15afc0 con 0x7fdddc154600 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.925+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc15b580 con 0x7fdddc154600 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.926+0000 7fddcaffd640 1 -- 172.21.15.113:0/778578195 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddd4015810 con 0x7fdddc154600 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.927+0000 7fddcaffd640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.927+0000 7fddcaffd640 1 -- 172.21.15.113:0/778578195 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fddd408b650 con 0x7fdddc154600 2024-04-25T00:00:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.927+0000 7fddda59c640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.927+0000 7fddda59c640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fddbc06be60 0x7fddbc06e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009ae0 tx=0x7fddcc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.929+0000 7fdde3a64740 1 Processor -- start 2024-04-25T00:00:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.929+0000 7fdde3a64740 1 -- start start 2024-04-25T00:00:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c14510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c44020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c44560 0x55a1a1c19370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdde3a64740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a1a1b7d330 con 0x55a1a1c44560 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a1a1b25ab0 con 0x55a1a1c61b80 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a1a1b792f0 con 0x55a1a1c44ca0 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c44560 0x55a1a1c19370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c44560 0x55a1a1c19370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36902/0 (socket says 172.21.15.113:36902) 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.930+0000 7fdddad9d640 1 -- 172.21.15.113:0/4195141540 learned_addr learned my addr 172.21.15.113:0/4195141540 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fddda59c640 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c14510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c44020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fdddad9d640 1 -- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 msgr2=0x55a1a1c44020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fdddad9d640 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c44020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fdddad9d640 1 -- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 msgr2=0x55a1a1c14510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fdddad9d640 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c14510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fdddad9d640 1 -- 172.21.15.113:0/4195141540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a195b7d0 con 0x55a1a1c44560 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fdddad9d640 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c44560 0x55a1a1c19370 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fddd4061f10 tx=0x7fddd4059fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd29536573a26ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fddbbfff640 1 -- 172.21.15.113:0/4195141540 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4062440 con 0x55a1a1c44560 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.931+0000 7fddbbfff640 1 -- 172.21.15.113:0/4195141540 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd400fe50 con 0x55a1a1c44560 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 -- 172.21.15.113:0/4195141540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c44560 msgr2=0x55a1a1c19370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c44560 0x55a1a1c19370 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fddd4061f10 tx=0x7fddd4059fe0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fddbbfff640 1 -- 172.21.15.113:0/4195141540 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4057b60 con 0x55a1a1c44560 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 -- 172.21.15.113:0/4195141540 shutdown_connections 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c44560 0x55a1a1c19370 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c44020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 --2- 172.21.15.113:0/4195141540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c14510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 -- 172.21.15.113:0/4195141540 >> 172.21.15.113:0/4195141540 conn(0x55a1a1c16780 msgr2=0x55a1a1c5f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 -- 172.21.15.113:0/4195141540 shutdown_connections 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.932+0000 7fdde3a64740 1 -- 172.21.15.113:0/4195141540 wait complete. 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.933+0000 7fdde3a64740 1 -- start start 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.933+0000 7fdde3a64740 1 Processor -- start 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c6f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c6fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdde3a64740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a1a1c18930 con 0x55a1a1c45b70 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a1a1b38f40 con 0x55a1a1c61b80 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a1a1c17d20 con 0x55a1a1c44ca0 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdde3a64740 10 client.0 _ll_register_callbacks cb 0x55a1a1890be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c6fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c6fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36908/0 (socket says 172.21.15.113:36908) 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 -- 172.21.15.113:0/3546328819 learned_addr learned my addr 172.21.15.113:0/3546328819 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fddda59c640 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c6f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 -- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 msgr2=0x55a1a1c47d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c47d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 -- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 msgr2=0x55a1a1c6f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c6f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.934+0000 7fdddad9d640 1 -- 172.21.15.113:0/3546328819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a18510f0 con 0x55a1a1c45b70 2024-04-25T00:00:02.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdddad9d640 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c6fba0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fddd4061f10 tx=0x7fddd408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37ae223434154a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fddb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:02.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fddb97fa640 1 -- 172.21.15.113:0/3546328819 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4062440 con 0x55a1a1c45b70 2024-04-25T00:00:02.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fddb97fa640 1 -- 172.21.15.113:0/3546328819 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fddd4057b60 con 0x55a1a1c45b70 2024-04-25T00:00:02.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd400fe50 con 0x55a1a1c45b70 2024-04-25T00:00:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fddb97fa640 1 -- 172.21.15.113:0/3546328819 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fddd4074020 con 0x55a1a1c45b70 2024-04-25T00:00:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 10 client.5292 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55a1a19bbf60 con 0x55a1a1c45b70 2024-04-25T00:00:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fddb97fa640 1 -- 172.21.15.113:0/3546328819 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd400fe50 con 0x55a1a1c45b70 2024-04-25T00:00:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fddb97fa640 1 -- 172.21.15.113:0/3546328819 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7fddd4055ed0 con 0x55a1a1c45b70 2024-04-25T00:00:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 10 client.5292 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:02.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 10 client.5292 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:02.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 20 client.5292 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:02.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c60670 0x55a1a1c7ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 4 client.5292 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:00:02.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1a1b8a870 con 0x55a1a1c60670 2024-04-25T00:00:02.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.935+0000 7fddda59c640 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c60670 0x55a1a1c7ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.936+0000 7fddda59c640 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c60670 0x55a1a1c7ce50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eafd2c377aeb0cb server_cookie=533af3d7b9d36069 in_seq=0 out_seq=0 2024-04-25T00:00:02.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.936+0000 7fddb97fa640 10 client.5292 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:02.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fddb97fa640 1 -- 172.21.15.113:0/3546328819 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1a1b8a870 con 0x55a1a1c60670 2024-04-25T00:00:02.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fddb97fa640 10 client.5292 handle_command_reply: tid=0 2024-04-25T00:00:02.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 client.5292 shutdown 2024-04-25T00:00:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 msgr2=0x55a1a1c6fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c6fba0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fddd4061f10 tx=0x7fddd408b040 comp rx=0 tx=0).stop 2024-04-25T00:00:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 shutdown_connections 2024-04-25T00:00:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c60670 0x55a1a1c7ce50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c61b80 0x55a1a1c6f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c6fba0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fddb97fa640 0 client.5292 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 --2- 172.21.15.113:0/3546328819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c47d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 >> 172.21.15.113:0/3546328819 conn(0x55a1a1c44560 msgr2=0x55a1a1c145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 shutdown_connections 2024-04-25T00:00:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.937+0000 7fdde3a64740 1 -- 172.21.15.113:0/3546328819 wait complete. 2024-04-25T00:00:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.938+0000 7fdde3a64740 20 client.5292 trim_cache size 0 max 16384 2024-04-25T00:00:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.954+0000 7fdde3a64740 1 Processor -- start 2024-04-25T00:00:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.954+0000 7fdde3a64740 1 -- start start 2024-04-25T00:00:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c15250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c15790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c60670 0x55a1a1c60fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fdde3a64740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1fca80 con 0x55a1a1c45b70 2024-04-25T00:00:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1fbae0 con 0x55a1a1c60670 2024-04-25T00:00:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc15cca0 con 0x55a1a1c44ca0 2024-04-25T00:00:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c15790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c15790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36924/0 (socket says 172.21.15.113:36924) 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 -- 172.21.15.113:0/487750057 learned_addr learned my addr 172.21.15.113:0/487750057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddda59c640 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c15250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 -- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 msgr2=0x55a1a1c15250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c15250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fdddad9d640 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c60670 0x55a1a1c60fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 -- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c60670 msgr2=0x55a1a1c60fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c60670 0x55a1a1c60fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 -- 172.21.15.113:0/487750057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a182a1b0 con 0x55a1a1c45b70 2024-04-25T00:00:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c15790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddd0007de0 tx=0x7fddd0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5628bedfc1783f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddbb7fe640 1 -- 172.21.15.113:0/487750057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0001430 con 0x55a1a1c45b70 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddbb7fe640 1 -- 172.21.15.113:0/487750057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd0039040 con 0x55a1a1c45b70 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.955+0000 7fddbb7fe640 1 -- 172.21.15.113:0/487750057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd00015f0 con 0x55a1a1c45b70 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 -- 172.21.15.113:0/487750057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 msgr2=0x55a1a1c15790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c15790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddd0007de0 tx=0x7fddd0007000 comp rx=0 tx=0).stop 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 -- 172.21.15.113:0/487750057 shutdown_connections 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c60670 0x55a1a1c60fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c45b70 0x55a1a1c15790 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 --2- 172.21.15.113:0/487750057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c44ca0 0x55a1a1c15250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 -- 172.21.15.113:0/487750057 >> 172.21.15.113:0/487750057 conn(0x55a1a1c44560 msgr2=0x55a1a1c0f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 -- 172.21.15.113:0/487750057 shutdown_connections 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.956+0000 7fdde3a64740 1 -- 172.21.15.113:0/487750057 wait complete. 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.957+0000 7fdde3a64740 1 -- start start 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.957+0000 7fdde3a64740 1 Processor -- start 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c44ca0 0x55a1a1cd7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c45b70 0x55a1a1c5b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c60670 0x55a1a1c5be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc1fb490 con 0x55a1a1c60670 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a1a1c45760 con 0x55a1a1c44ca0 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a1a1cdc140 con 0x55a1a1c45b70 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 10 client.0 _ll_register_callbacks cb 0x7fddd4070070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c60670 0x55a1a1c5be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c60670 0x55a1a1c5be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36934/0 (socket says 172.21.15.113:36934) 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 -- 172.21.15.113:0/364391302 learned_addr learned my addr 172.21.15.113:0/364391302 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c45b70 0x55a1a1c5b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fddda59c640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c44ca0 0x55a1a1cd7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 -- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c45b70 msgr2=0x55a1a1c5b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c45b70 0x55a1a1c5b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 -- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c44ca0 msgr2=0x55a1a1cd7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c44ca0 0x55a1a1cd7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 -- 172.21.15.113:0/364391302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a1c1b8a0 con 0x55a1a1c60670 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fddd9d9b640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c45b70 0x55a1a1c5b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdddad9d640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c60670 0x55a1a1c5be10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fddd405a110 tx=0x7fddd4055c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadd85669f4fb5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fddb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fddb97fa640 1 -- 172.21.15.113:0/364391302 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd405ae60 con 0x55a1a1c60670 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fddb97fa640 1 -- 172.21.15.113:0/364391302 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fddd4057b60 con 0x55a1a1c60670 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.958+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd0039040 con 0x55a1a1c60670 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fddb97fa640 1 -- 172.21.15.113:0/364391302 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fddd407e070 con 0x55a1a1c60670 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 10 client.5298 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55a1a1cdbe80 con 0x55a1a1c60670 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fddb97fa640 1 -- 172.21.15.113:0/364391302 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd0039040 con 0x55a1a1c60670 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fddb97fa640 1 -- 172.21.15.113:0/364391302 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7fddd4015b00 con 0x55a1a1c60670 2024-04-25T00:00:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 10 client.5298 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 10 client.5298 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 20 client.5298 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c61b80 0x55a1a1c15250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 4 client.5298 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55a1a1c56ef0 con 0x55a1a1c61b80 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fddda59c640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c61b80 0x55a1a1c15250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fddb97fa640 10 client.5298 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.959+0000 7fddda59c640 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c61b80 0x55a1a1c15250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d364bb27aa9b4b server_cookie=a952501f10cdc0d5 in_seq=0 out_seq=0 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fddb97fa640 1 -- 172.21.15.113:0/364391302 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17986 (crc 0 0 0) 0x55a1a1c56ef0 con 0x55a1a1c61b80 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fddb97fa640 10 client.5298 handle_command_reply: tid=0 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 client.5298 shutdown 2024-04-25T00:00:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c60670 msgr2=0x55a1a1c5be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c60670 0x55a1a1c5be10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fddd405a110 tx=0x7fddd4055c30 comp rx=0 tx=0).stop 2024-04-25T00:00:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 shutdown_connections 2024-04-25T00:00:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x55a1a1c61b80 0x55a1a1c15250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a1a1c60670 0x55a1a1c5be10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a1a1c45b70 0x55a1a1c5b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 --2- 172.21.15.113:0/364391302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1a1c44ca0 0x55a1a1cd7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fddb97fa640 0 client.5298 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 >> 172.21.15.113:0/364391302 conn(0x55a1a1c44560 msgr2=0x55a1a1c46eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 shutdown_connections 2024-04-25T00:00:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 1 -- 172.21.15.113:0/364391302 wait complete. 2024-04-25T00:00:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.960+0000 7fdde3a64740 20 client.5298 trim_cache size 0 max 16384 2024-04-25T00:00:02.972 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:02.972 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 787, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 781, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 299216, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 12947646, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 8, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.113511981, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.061982390, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.022773102, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.025659018, 2024-04-25T00:00:02.973 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 781, 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.166634335, 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000213360 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2, 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000299, 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000149 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 116048, 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 17488 2024-04-25T00:00:02.974 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 28061, 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 50521, 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 8876512, 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 44520809, 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 8, 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:00:02.975 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 2.756540020, 2024-04-25T00:00:02.976 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 2.146634321, 2024-04-25T00:00:02.976 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 1.175663939, 2024-04-25T00:00:02.976 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.038781893, 2024-04-25T00:00:02.976 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:00:02.976 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 50520, 2024-04-25T00:00:02.976 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.101660321, 2024-04-25T00:00:02.977 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000021806 2024-04-25T00:00:02.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.977 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:00:02.977 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 49052, 2024-04-25T00:00:02.977 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.007439847, 2024-04-25T00:00:02.977 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000151 2024-04-25T00:00:02.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.978 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:00:02.978 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 105584 2024-04-25T00:00:02.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.978 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:00:02.978 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2645, 2024-04-25T00:00:02.979 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 2645, 2024-04-25T00:00:02.979 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 839179, 2024-04-25T00:00:02.979 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 55113130, 2024-04-25T00:00:02.979 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-04-25T00:00:02.979 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:00:02.980 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.472054931, 2024-04-25T00:00:02.980 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.192263751, 2024-04-25T00:00:02.980 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.050856829, 2024-04-25T00:00:02.980 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.215027291, 2024-04-25T00:00:02.980 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:00:02.980 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2644, 2024-04-25T00:00:02.981 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.574616221, 2024-04-25T00:00:02.981 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000217328 2024-04-25T00:00:02.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.981 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:00:02.981 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:02.981 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-25T00:00:02.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1190, 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.035273000, 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.015155691 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.983 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-25T00:00:02.984 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:00:02.984 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:00:02.984 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1793, 2024-04-25T00:00:02.984 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.029658025, 2024-04-25T00:00:02.984 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000574265 2024-04-25T00:00:02.984 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:02.985 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.985 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-25T00:00:02.985 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 25194, 2024-04-25T00:00:02.985 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 25194, 2024-04-25T00:00:02.985 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-25T00:00:02.985 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 25194, 2024-04-25T00:00:02.985 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 435.593600869, 2024-04-25T00:00:02.986 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.017289576 2024-04-25T00:00:02.986 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.986 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-25T00:00:02.986 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-04-25T00:00:02.986 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-04-25T00:00:02.986 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-04-25T00:00:02.987 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 12, 2024-04-25T00:00:02.987 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-04-25T00:00:02.987 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-04-25T00:00:02.987 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 19861, 2024-04-25T00:00:02.987 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 0, 2024-04-25T00:00:02.987 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 19861, 2024-04-25T00:00:02.988 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-04-25T00:00:02.988 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 19860, 2024-04-25T00:00:02.988 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-04-25T00:00:02.988 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 32, 2024-04-25T00:00:02.988 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 32, 2024-04-25T00:00:02.988 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 2, 2024-04-25T00:00:02.989 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 45461, 2024-04-25T00:00:02.989 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 45027, 2024-04-25T00:00:02.989 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-25T00:00:02.989 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-04-25T00:00:02.989 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:00:02.989 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-25T00:00:02.989 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 83, 2024-04-25T00:00:02.990 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-25T00:00:02.990 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-25T00:00:02.990 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-04-25T00:00:02.990 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-04-25T00:00:02.990 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-04-25T00:00:02.990 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-04-25T00:00:02.990 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:00:02.991 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 1039, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 4034, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:00:02.992 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 3897, 2024-04-25T00:00:02.993 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 3364, 2024-04-25T00:00:02.993 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 52, 2024-04-25T00:00:02.993 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 13971 2024-04-25T00:00:02.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.993 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-25T00:00:02.993 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-25T00:00:02.993 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-25T00:00:02.994 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-25T00:00:02.994 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-04-25T00:00:02.994 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-04-25T00:00:02.994 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 2000, 2024-04-25T00:00:02.994 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 315, 2024-04-25T00:00:02.994 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:00:02.994 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 4087, 2024-04-25T00:00:02.995 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 1655, 2024-04-25T00:00:02.995 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 2166, 2024-04-25T00:00:02.995 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-25T00:00:02.995 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-25T00:00:02.995 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:00:02.995 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.996 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 24893, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 24893, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 34, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 34, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 4, 2024-04-25T00:00:02.997 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 96398149, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 0, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2508, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 124.014904573, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.049447729 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 0 2024-04-25T00:00:02.998 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 2014, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 2014, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 0, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 13, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 13, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 0, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 19861, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 19862, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 1, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 32, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 7332, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 7300, 2024-04-25T00:00:02.999 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 114768, 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 231788 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 27961, 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 25194, 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 3, 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-25T00:00:03.000 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2000, 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.697430093, 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000848715 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 30, 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.004923994, 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000164133 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-25T00:00:03.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6000, 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 7.225157615, 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001204192 2024-04-25T00:00:03.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6984, 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 5.508196221, 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000788687 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.003 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-25T00:00:03.004 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.005 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.005 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.005 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-25T00:00:03.005 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.005 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.005 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.006 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-25T00:00:03.006 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:03.006 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000037645, 2024-04-25T00:00:03.006 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000037645 2024-04-25T00:00:03.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.007 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-25T00:00:03.007 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.007 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.007 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.007 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-25T00:00:03.008 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:03.008 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.002698061, 2024-04-25T00:00:03.008 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002698061 2024-04-25T00:00:03.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.008 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-25T00:00:03.008 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.009 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.009 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.009 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-25T00:00:03.009 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:03.009 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.075930595, 2024-04-25T00:00:03.010 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.075930595 2024-04-25T00:00:03.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.010 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-25T00:00:03.010 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:03.010 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000084231, 2024-04-25T00:00:03.010 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000084231 2024-04-25T00:00:03.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.011 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-25T00:00:03.011 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8, 2024-04-25T00:00:03.011 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.582495310, 2024-04-25T00:00:03.011 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.197811913 2024-04-25T00:00:03.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2166, 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 401.207425239, 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.185229651 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.013 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.060742662, 2024-04-25T00:00:03.014 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.060742662 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-25T00:00:03.015 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:03.016 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000277912, 2024-04-25T00:00:03.016 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000277912 2024-04-25T00:00:03.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.016 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-25T00:00:03.016 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.016 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.016 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-25T00:00:03.017 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.018 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.018 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.018 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-25T00:00:03.018 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.018 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8000, 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.228201291, 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002278525 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.020 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 1, 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 0, 2024-04-25T00:00:03.021 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 18339, 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 18339, 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 42, 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-25T00:00:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:00:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:00:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-25T00:00:03.025 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:00:03.026 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 17115150, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 6315, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 440, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 5, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 5312, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 75, 2024-04-25T00:00:03.027 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 20363319, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 856789, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-25T00:00:03.028 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 4388, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 92654235, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 37, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 4388, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4388, 2024-04-25T00:00:03.029 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 16.678641706, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.003800966 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4388, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 4478 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 4388, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 47, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 4341, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-25T00:00:03.030 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 11, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 47, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 2555, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 30, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 1655, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 33, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-25T00:00:03.031 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 22, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-25T00:00:03.032 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-25T00:00:03.033 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 32, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 1, 2024-04-25T00:00:03.034 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 8, 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 8, 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 98, 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 0, 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 27 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.035 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 1, 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-25T00:00:03.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 32, 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 1655, 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 1655, 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-25T00:00:03.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 31488, 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 7608621, 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:03.038 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 31488, 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 31488, 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 7608621, 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.039 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:00:03.040 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 4388, 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 92834630, 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 4388, 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 92834630, 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.041 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 4388, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 4388, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 4388, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 4388, 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:03.043 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:00:03.044 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 1655, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 180395, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 1655, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 47, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 180395, 2024-04-25T00:00:03.045 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x56499da12500": { 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 84, 2024-04-25T00:00:03.046 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:00:03.047 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:03.047 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 24893, 2024-04-25T00:00:03.047 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 92203845, 2024-04-25T00:00:03.047 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 24893, 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2508, 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 92203761, 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:03.049 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:03.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.049 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fddbc06be60 msgr2=0x7fddbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fddbc06be60 0x7fddbc06e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009ae0 tx=0x7fddcc009190 comp rx=0 tx=0).stop 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc154600 msgr2=0x7fdddc157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc154600 0x7fdddc157d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdddc15f700 tx=0x7fddd400f1b0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 shutdown_connections 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc154600 0x7fdddc157d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc0a9480 0x7fdddc157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 --2- 172.21.15.113:0/778578195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc4003f30 0x7fdddc153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 >> 172.21.15.113:0/778578195 conn(0x7fdddc06b0f0 msgr2=0x7fdddc0729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 shutdown_connections 2024-04-25T00:00:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:02.961+0000 7fdde0d5a640 1 -- 172.21.15.113:0/778578195 wait complete. 2024-04-25T00:00:03.052 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T00:00:03.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:03.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T00:00:03.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.155+0000 7f2d5a3ed640 1 Processor -- start 2024-04-25T00:00:03.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.155+0000 7f2d5a3ed640 1 -- start start 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d5a3ed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d540b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d5a3ed640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d540b3ff0 0x7f2d5406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d5a3ed640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d54072f70 con 0x7f2d5406e610 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d5a3ed640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2d540b2fd0 con 0x7f2d540b3ff0 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d5a3ed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d540b32b0 con 0x7f2d54144a80 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d537fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d540b3ff0 0x7f2d5406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d5898a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2d5406e610 0x7f2d540cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54788/0 (socket says 172.21.15.113:54788) 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d5898a640 1 -- 172.21.15.113:0/590271866 learned_addr learned my addr 172.21.15.113:0/590271866 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d53fff640 1 --2- 172.21.15.113:0/590271866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d540b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d537fe640 1 -- 172.21.15.113:0/590271866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 msgr2=0x7f2d540b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.156+0000 7f2d537fe640 1 --2- 172.21.15.113:0/590271866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d540b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.157+0000 7f2d537fe640 1 -- 172.21.15.113:0/590271866 >> v1:172.21.15.113:6789/0 conn(0x7f2d5406e610 legacy=0x7f2d540cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.157+0000 7f2d52ffd640 1 -- 172.21.15.113:0/590271866 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2893256800 0 0) 0x7f2d54072f70 con 0x7f2d5406e610 2024-04-25T00:00:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.157+0000 7f2d537fe640 1 -- 172.21.15.113:0/590271866 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d540b3590 con 0x7f2d540b3ff0 2024-04-25T00:00:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.157+0000 7f2d537fe640 1 --2- 172.21.15.113:0/590271866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d540b3ff0 0x7f2d5406e0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2d480097a0 tx=0x7f2d4802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf27feccb90b6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.157+0000 7f2d52ffd640 1 -- 172.21.15.113:0/590271866 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48037070 con 0x7f2d540b3ff0 2024-04-25T00:00:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.157+0000 7f2d52ffd640 1 -- 172.21.15.113:0/590271866 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4802ae50 con 0x7f2d540b3ff0 2024-04-25T00:00:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.157+0000 7f2d52ffd640 1 -- 172.21.15.113:0/590271866 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48032660 con 0x7f2d540b3ff0 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/590271866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d540b3ff0 msgr2=0x7f2d5406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/590271866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d540b3ff0 0x7f2d5406e0d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2d480097a0 tx=0x7f2d4802a100 comp rx=0 tx=0).stop 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/590271866 shutdown_connections 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/590271866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d540b3ff0 0x7f2d5406e0d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/590271866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d540b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/590271866 >> 172.21.15.113:0/590271866 conn(0x7f2d540b1b30 msgr2=0x7f2d540b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/590271866 shutdown_connections 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.158+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/590271866 wait complete. 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.159+0000 7f2d5a3ed640 1 Processor -- start 2024-04-25T00:00:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.159+0000 7f2d5a3ed640 1 -- start start 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d5a3ed640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5406e610 0x7f2d541f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d5a3ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d540b3ff0 0x7f2d5415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d5a3ed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d54142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d5a3ed640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54072200 con 0x7f2d5406e610 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d5a3ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d540cdf90 con 0x7f2d540b3ff0 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d5a3ed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d54072380 con 0x7f2d54144a80 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5406e610 0x7f2d541f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5406e610 0x7f2d541f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36954/0 (socket says 172.21.15.113:36954) 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 -- 172.21.15.113:0/1436074489 learned_addr learned my addr 172.21.15.113:0/1436074489 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d5898a640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d54142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d537fe640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d540b3ff0 0x7f2d5415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 -- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 msgr2=0x7f2d54142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d54142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 -- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d540b3ff0 msgr2=0x7f2d5415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d540b3ff0 0x7f2d5415a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.160+0000 7f2d53fff640 1 -- 172.21.15.113:0/1436074489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d54143480 con 0x7f2d5406e610 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.161+0000 7f2d53fff640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5406e610 0x7f2d541f0e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d5f0 tx=0x7f2d4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.161+0000 7f2d50ff9640 1 -- 172.21.15.113:0/1436074489 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d440125f0 con 0x7f2d5406e610 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.161+0000 7f2d50ff9640 1 -- 172.21.15.113:0/1436074489 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d44019070 con 0x7f2d5406e610 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.161+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5415bd20 con 0x7f2d5406e610 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.161+0000 7f2d50ff9640 1 -- 172.21.15.113:0/1436074489 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d440129d0 con 0x7f2d5406e610 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.161+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5415c2e0 con 0x7f2d5406e610 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.162+0000 7f2d50ff9640 1 -- 172.21.15.113:0/1436074489 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d44012b70 con 0x7f2d5406e610 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.163+0000 7f2d50ff9640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d4006be30 0x7f2d4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.163+0000 7f2d537fe640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d4006be30 0x7f2d4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.163+0000 7f2d50ff9640 1 -- 172.21.15.113:0/1436074489 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2d4408b1e0 con 0x7f2d5406e610 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.163+0000 7f2d537fe640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d4006be30 0x7f2d4006e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d48034000 tx=0x7f2d4802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.164+0000 7f2d5d1cf740 1 Processor -- start 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.164+0000 7f2d5d1cf740 1 -- start start 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x563129903d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631298e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563129842120 con 0x5631298e4210 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563129847370 con 0x5631299039b0 2024-04-25T00:00:03.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5631297efa00 con 0x5631298e57a0 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36966/0 (socket says 172.21.15.113:36966) 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 -- 172.21.15.113:0/3248022126 learned_addr learned my addr 172.21.15.113:0/3248022126 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d537fe640 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631298e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d53fff640 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x563129903d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 -- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 msgr2=0x5631298e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631298e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 -- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 msgr2=0x563129903d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x563129903d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 -- 172.21.15.113:0/3248022126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631297cf8d0 con 0x5631298e4210 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d537fe640 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631298e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d53fff640 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x563129903d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d5898a640 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990b8d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0072d0 tx=0x7f2d4c007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294e71cf14ea8081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d39ffb640 1 -- 172.21.15.113:0/3248022126 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c019070 con 0x5631298e4210 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d39ffb640 1 -- 172.21.15.113:0/3248022126 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c0079e0 con 0x5631298e4210 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.165+0000 7f2d39ffb640 1 -- 172.21.15.113:0/3248022126 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c002060 con 0x5631298e4210 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3248022126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 msgr2=0x56312990b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990b8d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0072d0 tx=0x7f2d4c007790 comp rx=0 tx=0).stop 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3248022126 shutdown_connections 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990b8d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631298e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3248022126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x563129903d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3248022126 >> 172.21.15.113:0/3248022126 conn(0x5631298ab400 msgr2=0x563129929a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3248022126 shutdown_connections 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3248022126 wait complete. 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.166+0000 7f2d5d1cf740 1 -- start start 2024-04-25T00:00:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 1 Processor -- start 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x563129900890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631299240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631299039b0 0x5631299245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563129802fa0 con 0x5631299039b0 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563129843330 con 0x5631298e4210 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56312990fa30 con 0x5631298e57a0 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5d1cf740 10 client.0 _ll_register_callbacks cb 0x563129802990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5898a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631299039b0 0x5631299245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5898a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631299039b0 0x5631299245f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36980/0 (socket says 172.21.15.113:36980) 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.167+0000 7f2d5898a640 1 -- 172.21.15.113:0/3605081695 learned_addr learned my addr 172.21.15.113:0/3605081695 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d53fff640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x563129900890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d537fe640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631299240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5898a640 1 -- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 msgr2=0x5631299240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5898a640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631299240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5898a640 1 -- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 msgr2=0x563129900890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5898a640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x563129900890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5898a640 1 -- 172.21.15.113:0/3605081695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563129627940 con 0x5631299039b0 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d53fff640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x563129900890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5898a640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631299039b0 0x5631299245f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0077d0 tx=0x7f2d4c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61f69a6d8281957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d32ffd640 1 -- 172.21.15.113:0/3605081695 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c023060 con 0x5631299039b0 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d32ffd640 1 -- 172.21.15.113:0/3605081695 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2d4c004b60 con 0x5631299039b0 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d4c0079e0 con 0x5631299039b0 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d32ffd640 1 -- 172.21.15.113:0/3605081695 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f2d4c029050 con 0x5631299039b0 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 10 client.5313 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5631296880d0 con 0x5631299039b0 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d32ffd640 1 -- 172.21.15.113:0/3605081695 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c0079e0 con 0x5631299039b0 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d32ffd640 1 -- 172.21.15.113:0/3605081695 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f2d4c01e410 con 0x5631299039b0 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 10 client.5313 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 10 client.5313 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 20 client.5313 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x56312992a900 0x56312992ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 4 client.5313 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.168+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631298548b0 con 0x56312992a900 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.169+0000 7f2d53fff640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x56312992a900 0x56312992ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.169+0000 7f2d32ffd640 10 client.5313 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.169+0000 7f2d53fff640 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x56312992a900 0x56312992ad50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6cf1619d02f80d1 server_cookie=7d932616990890b4 in_seq=0 out_seq=0 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.169+0000 7f2d32ffd640 1 -- 172.21.15.113:0/3605081695 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631298548b0 con 0x56312992a900 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.169+0000 7f2d32ffd640 10 client.5313 handle_command_reply: tid=0 2024-04-25T00:00:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.169+0000 7f2d5d1cf740 1 client.5313 shutdown 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631299039b0 msgr2=0x5631299245f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631299039b0 0x5631299245f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0077d0 tx=0x7f2d4c001250 comp rx=0 tx=0).stop 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 shutdown_connections 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x56312992a900 0x56312992ad50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631299039b0 0x5631299245f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x5631299240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/3605081695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x563129900890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d32ffd640 0 client.5313 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 >> 172.21.15.113:0/3605081695 conn(0x5631298ab400 msgr2=0x563129903200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 shutdown_connections 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/3605081695 wait complete. 2024-04-25T00:00:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.170+0000 7f2d5d1cf740 20 client.5313 trim_cache size 0 max 16384 2024-04-25T00:00:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.187+0000 7f2d5d1cf740 1 Processor -- start 2024-04-25T00:00:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.187+0000 7f2d5d1cf740 1 -- start start 2024-04-25T00:00:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x56312990b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x56312990bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541fc230 con 0x5631298e4210 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541f3f70 con 0x5631299039b0 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d541f32d0 con 0x5631298e57a0 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36994/0 (socket says 172.21.15.113:36994) 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 -- 172.21.15.113:0/1645089791 learned_addr learned my addr 172.21.15.113:0/1645089791 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5898a640 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x56312990bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d537fe640 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x56312990b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 -- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 msgr2=0x56312990b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x56312990b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 -- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 msgr2=0x56312990bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x56312990bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 -- 172.21.15.113:0/1645089791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312951b230 con 0x5631298e4210 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d5898a640 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x56312990bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d537fe640 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x56312990b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.188+0000 7f2d53fff640 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990ff40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d44055980 tx=0x7f2d4408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff9535df3bcee46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d397fa640 1 -- 172.21.15.113:0/1645089791 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44071020 con 0x5631298e4210 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d397fa640 1 -- 172.21.15.113:0/1645089791 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d44062100 con 0x5631298e4210 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d397fa640 1 -- 172.21.15.113:0/1645089791 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d440575c0 con 0x5631298e4210 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/1645089791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 msgr2=0x56312990ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990ff40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d44055980 tx=0x7f2d4408b040 comp rx=0 tx=0).stop 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/1645089791 shutdown_connections 2024-04-25T00:00:03.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631299039b0 0x56312990bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631298e57a0 0x56312990b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/1645089791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e4210 0x56312990ff40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/1645089791 >> 172.21.15.113:0/1645089791 conn(0x5631298ab400 msgr2=0x5631299240b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/1645089791 shutdown_connections 2024-04-25T00:00:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.189+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/1645089791 wait complete. 2024-04-25T00:00:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.190+0000 7f2d5d1cf740 1 -- start start 2024-04-25T00:00:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.190+0000 7f2d5d1cf740 1 Processor -- start 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x5631299a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e57a0 0x56312992cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631299039b0 0x56312992b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541f3920 con 0x5631298e57a0 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631298e5310 con 0x5631298e4210 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56312992d840 con 0x5631299039b0 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 10 client.0 _ll_register_callbacks cb 0x7f2d4c007e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e57a0 0x56312992cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e57a0 0x56312992cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37004/0 (socket says 172.21.15.113:37004) 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 -- 172.21.15.113:0/2794947620 learned_addr learned my addr 172.21.15.113:0/2794947620 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d53fff640 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x5631299a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5898a640 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631299039b0 0x56312992b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 -- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631299039b0 msgr2=0x56312992b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631299039b0 0x56312992b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 -- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 msgr2=0x5631299a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x5631299a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 -- 172.21.15.113:0/2794947620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631298ffd70 con 0x5631298e57a0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d537fe640 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e57a0 0x56312992cd50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009560 tx=0x7f2d48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a24c8fa619f0e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d32ffd640 1 -- 172.21.15.113:0/2794947620 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4804e070 con 0x5631298e57a0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d32ffd640 1 -- 172.21.15.113:0/2794947620 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2d4803c850 con 0x5631298e57a0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d44062100 con 0x5631298e57a0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d32ffd640 1 -- 172.21.15.113:0/2794947620 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f2d48037040 con 0x5631298e57a0 2024-04-25T00:00:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.191+0000 7f2d5d1cf740 10 client.5319 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x56312992d290 con 0x5631298e57a0 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d32ffd640 1 -- 172.21.15.113:0/2794947620 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d44062100 con 0x5631298e57a0 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d32ffd640 1 -- 172.21.15.113:0/2794947620 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f2d4803ca50 con 0x5631298e57a0 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d5d1cf740 10 client.5319 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d5d1cf740 10 client.5319 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d5d1cf740 20 client.5319 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5631298e4b90 0x5631298e2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d5d1cf740 4 client.5319 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) -- 0x56312984ec00 con 0x5631298e4b90 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d53fff640 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5631298e4b90 0x5631298e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d32ffd640 10 client.5319 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.192+0000 7f2d53fff640 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5631298e4b90 0x5631298e2660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e690d9adc546734a server_cookie=7a717c20f7548a61 in_seq=0 out_seq=0 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d32ffd640 1 -- 172.21.15.113:0/2794947620 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56312984ec00 con 0x5631298e4b90 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d32ffd640 10 client.5319 handle_command_reply: tid=0 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 client.5319 shutdown 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e57a0 msgr2=0x56312992cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e57a0 0x56312992cd50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009560 tx=0x7f2d48004730 comp rx=0 tx=0).stop 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 shutdown_connections 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5631298e4b90 0x5631298e2660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631299039b0 0x56312992b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631298e57a0 0x56312992cd50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 --2- 172.21.15.113:0/2794947620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631298e4210 0x5631299a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 >> 172.21.15.113:0/2794947620 conn(0x5631298ab400 msgr2=0x563129902d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d32ffd640 0 client.5319 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 shutdown_connections 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 1 -- 172.21.15.113:0/2794947620 wait complete. 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.196+0000 7f2d5d1cf740 20 client.5319 trim_cache size 0 max 16384 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "0e47a352-c82b-47ca-93cb-9f2424f27fd5", 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:00:03.200 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:03.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d4006be30 msgr2=0x7f2d4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d4006be30 0x7f2d4006e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d48034000 tx=0x7f2d4802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:03.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5406e610 msgr2=0x7f2d541f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5406e610 0x7f2d541f0e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d5f0 tx=0x7f2d4400dab0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 shutdown_connections 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d4006be30 0x7f2d4006e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d540b3ff0 0x7f2d5415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54144a80 0x7f2d54142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 --2- 172.21.15.113:0/1436074489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5406e610 0x7f2d541f0e50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 >> 172.21.15.113:0/1436074489 conn(0x7f2d540b1b30 msgr2=0x7f2d5413fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 shutdown_connections 2024-04-25T00:00:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.197+0000 7f2d5a3ed640 1 -- 172.21.15.113:0/1436074489 wait complete. 2024-04-25T00:00:03.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:03.211 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T00:00:03.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.331+0000 7f878e77e640 1 Processor -- start 2024-04-25T00:00:03.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.331+0000 7f878e77e640 1 -- start start 2024-04-25T00:00:03.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f878e77e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f878815e2d0 0x7f878815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f878e77e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f87880b3080 0x7f87880b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f878e77e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878815f4f0 con 0x7f878815e6b0 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f878e77e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f87880cde00 con 0x7f878815e2d0 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f878e77e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f878815f7d0 con 0x7f87880b3080 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f8787fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f878815e2d0 0x7f878815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f87877fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f87880b3080 0x7f87880b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f8787fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f878815e2d0 0x7f878815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37016/0 (socket says 172.21.15.113:37016) 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f8787fff640 1 -- 172.21.15.113:0/2710285271 learned_addr learned my addr 172.21.15.113:0/2710285271 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f8787fff640 1 -- 172.21.15.113:0/2710285271 >> v2:172.21.15.178:3300/0 conn(0x7f87880b3080 msgr2=0x7f87880b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f8787fff640 1 --2- 172.21.15.113:0/2710285271 >> v2:172.21.15.178:3300/0 conn(0x7f87880b3080 0x7f87880b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f8787fff640 1 -- 172.21.15.113:0/2710285271 >> v1:172.21.15.178:6789/0 conn(0x7f878815e6b0 legacy=0x7f8788071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.332+0000 7f8787fff640 1 -- 172.21.15.113:0/2710285271 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878815fab0 con 0x7f878815e2d0 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.333+0000 7f8787fff640 1 --2- 172.21.15.113:0/2710285271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e2d0 0x7f878815ebf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8788070ff0 tx=0x7f877802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6d5f188f485b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.334+0000 7f8786ffd640 1 -- 172.21.15.113:0/2710285271 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778037070 con 0x7f878815e2d0 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.334+0000 7f8786ffd640 1 -- 172.21.15.113:0/2710285271 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8778032440 con 0x7f878815e2d0 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.334+0000 7f8786ffd640 1 -- 172.21.15.113:0/2710285271 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778032810 con 0x7f878815e2d0 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 -- 172.21.15.113:0/2710285271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e2d0 msgr2=0x7f878815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 --2- 172.21.15.113:0/2710285271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e2d0 0x7f878815ebf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8788070ff0 tx=0x7f877802a270 comp rx=0 tx=0).stop 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 -- 172.21.15.113:0/2710285271 shutdown_connections 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 --2- 172.21.15.113:0/2710285271 >> v2:172.21.15.178:3300/0 conn(0x7f87880b3080 0x7f87880b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 --2- 172.21.15.113:0/2710285271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e2d0 0x7f878815ebf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 -- 172.21.15.113:0/2710285271 >> 172.21.15.113:0/2710285271 conn(0x7f878806b0f0 msgr2=0x7f878806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 -- 172.21.15.113:0/2710285271 shutdown_connections 2024-04-25T00:00:03.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.335+0000 7f878e77e640 1 -- 172.21.15.113:0/2710285271 wait complete. 2024-04-25T00:00:03.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.337+0000 7f878e77e640 1 Processor -- start 2024-04-25T00:00:03.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.337+0000 7f878e77e640 1 -- start start 2024-04-25T00:00:03.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878e77e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87880b3080 0x7f87880b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878e77e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878815e2d0 0x7f87880b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878e77e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e6b0 0x7f87881f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878e77e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f878815f2b0 con 0x7f878815e6b0 2024-04-25T00:00:03.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878e77e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f878815f130 con 0x7f878815e2d0 2024-04-25T00:00:03.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878e77e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87880cf2a0 con 0x7f87880b3080 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e6b0 0x7f87881f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e6b0 0x7f87881f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37018/0 (socket says 172.21.15.113:37018) 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 -- 172.21.15.113:0/500275588 learned_addr learned my addr 172.21.15.113:0/500275588 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f8787fff640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87880b3080 0x7f87880b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f87877fe640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878815e2d0 0x7f87880b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 -- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87880b3080 msgr2=0x7f87880b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87880b3080 0x7f87880b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 -- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878815e2d0 msgr2=0x7f87880b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878815e2d0 0x7f87880b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.338+0000 7f878cd1b640 1 -- 172.21.15.113:0/500275588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87880b0820 con 0x7f878815e6b0 2024-04-25T00:00:03.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.339+0000 7f878cd1b640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e6b0 0x7f87881f4660 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f878000c6b0 tx=0x7f878000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.339+0000 7f8784ff9640 1 -- 172.21.15.113:0/500275588 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780003b70 con 0x7f878815e6b0 2024-04-25T00:00:03.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.339+0000 7f8784ff9640 1 -- 172.21.15.113:0/500275588 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8780003d30 con 0x7f878815e6b0 2024-04-25T00:00:03.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.339+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87880b0b00 con 0x7f878815e6b0 2024-04-25T00:00:03.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.339+0000 7f8784ff9640 1 -- 172.21.15.113:0/500275588 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780005310 con 0x7f878815e6b0 2024-04-25T00:00:03.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.339+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87880b10c0 con 0x7f878815e6b0 2024-04-25T00:00:03.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.341+0000 7f8784ff9640 1 -- 172.21.15.113:0/500275588 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87800054b0 con 0x7f878815e6b0 2024-04-25T00:00:03.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.341+0000 7f8784ff9640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f877406be60 0x7f877406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.342+0000 7f8784ff9640 1 -- 172.21.15.113:0/500275588 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f878008ab40 con 0x7f878815e6b0 2024-04-25T00:00:03.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.342+0000 7f8787fff640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f877406be60 0x7f877406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.343+0000 7f8787fff640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f877406be60 0x7f877406e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8778034000 tx=0x7f877802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.343+0000 7f879156e740 1 Processor -- start 2024-04-25T00:00:03.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.343+0000 7f879156e740 1 -- start start 2024-04-25T00:00:03.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415049140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f879156e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x564415066b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441501ea40 0x564415064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f879156e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564414f826b0 con 0x564415066600 2024-04-25T00:00:03.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f879156e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564414f2a880 con 0x564415048d60 2024-04-25T00:00:03.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f879156e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564414f7e670 con 0x56441501ea40 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f87877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x564415066b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f87877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x564415066b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37032/0 (socket says 172.21.15.113:37032) 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f87877fe640 1 -- 172.21.15.113:0/3800967388 learned_addr learned my addr 172.21.15.113:0/3800967388 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f878cd1b640 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441501ea40 0x564415064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f8787fff640 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415049140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f87877fe640 1 -- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441501ea40 msgr2=0x564415064cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f87877fe640 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441501ea40 0x564415064cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f87877fe640 1 -- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 msgr2=0x564415049140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.344+0000 7f87877fe640 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415049140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.345+0000 7f87877fe640 1 -- 172.21.15.113:0/3800967388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564414d607d0 con 0x564415066600 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.345+0000 7f878cd1b640 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441501ea40 0x564415064cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.345+0000 7f8787fff640 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415049140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.345+0000 7f87877fe640 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x564415066b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f877c009bd0 tx=0x7f877c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfe0a10c096afd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.345+0000 7f876dffb640 1 -- 172.21.15.113:0/3800967388 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c019070 con 0x564415066600 2024-04-25T00:00:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.345+0000 7f876dffb640 1 -- 172.21.15.113:0/3800967388 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c004030 con 0x564415066600 2024-04-25T00:00:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.345+0000 7f876dffb640 1 -- 172.21.15.113:0/3800967388 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c0043e0 con 0x564415066600 2024-04-25T00:00:03.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 -- 172.21.15.113:0/3800967388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 msgr2=0x564415066b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x564415066b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f877c009bd0 tx=0x7f877c011690 comp rx=0 tx=0).stop 2024-04-25T00:00:03.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 -- 172.21.15.113:0/3800967388 shutdown_connections 2024-04-25T00:00:03.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441501ea40 0x564415064cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x564415066b50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 --2- 172.21.15.113:0/3800967388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415049140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 -- 172.21.15.113:0/3800967388 >> 172.21.15.113:0/3800967388 conn(0x56441501b600 msgr2=0x564415063b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 -- 172.21.15.113:0/3800967388 shutdown_connections 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.346+0000 7f879156e740 1 -- 172.21.15.113:0/3800967388 wait complete. 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.347+0000 7f879156e740 1 -- start start 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.347+0000 7f879156e740 1 Processor -- start 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415035550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f879156e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441504a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f8787fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415035550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441504aa20 0x564415018260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f879156e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56441501f7e0 con 0x564415066600 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f8787fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415035550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:56328/0 (socket says 172.21.15.113:56328) 2024-04-25T00:00:03.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564414f3df50 con 0x564415048d60 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f8787fff640 1 -- 172.21.15.113:0/3903505295 learned_addr learned my addr 172.21.15.113:0/3903505295 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564415019530 con 0x56441504aa20 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f879156e740 10 client.0 _ll_register_callbacks cb 0x564414f79000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f878cd1b640 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441504aa20 0x564415018260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f8787fff640 1 -- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441504aa20 msgr2=0x564415018260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f8787fff640 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441504aa20 0x564415018260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f8787fff640 1 -- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 msgr2=0x56441504a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.349+0000 7f87877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441504a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f8787fff640 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441504a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f8787fff640 1 -- 172.21.15.113:0/3903505295 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564414c963f0 con 0x564415048d60 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f87877fe640 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441504a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f8787fff640 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415035550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877802a860 tx=0x7f8778004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0235cc9c2b79661 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f8766ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f8766ffd640 1 -- 172.21.15.113:0/3903505295 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877804e070 con 0x564415048d60 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f8766ffd640 1 -- 172.21.15.113:0/3903505295 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f877803c850 con 0x564415048d60 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.350+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c004030 con 0x564415048d60 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f8766ffd640 1 -- 172.21.15.113:0/3903505295 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f8778059050 con 0x564415048d60 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 10 client.4975 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x564414c560f0 con 0x564415048d60 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f8766ffd640 1 -- 172.21.15.113:0/3903505295 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c004030 con 0x564415048d60 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f8766ffd640 1 -- 172.21.15.113:0/3903505295 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f877805c020 con 0x564415048d60 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 10 client.4975 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 10 client.4975 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 20 client.4975 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x564415037470 0x5644150378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 4 client.4975 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:00:03.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f87877fe640 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x564415037470 0x5644150378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.351+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564414f8fbf0 con 0x564415037470 2024-04-25T00:00:03.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.352+0000 7f8766ffd640 10 client.4975 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.352+0000 7f87877fe640 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x564415037470 0x5644150378b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf817a60bd1026bb server_cookie=337f64c91366b539 in_seq=0 out_seq=0 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f8766ffd640 1 -- 172.21.15.113:0/3903505295 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564414f8fbf0 con 0x564415037470 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f8766ffd640 10 client.4975 handle_command_reply: tid=0 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 client.4975 shutdown 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 msgr2=0x564415035550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415035550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877802a860 tx=0x7f8778004730 comp rx=0 tx=0).stop 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 shutdown_connections 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x564415037470 0x5644150378b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56441504aa20 0x564415018260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441504a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 --2- 172.21.15.113:0/3903505295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564415048d60 0x564415035550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 >> 172.21.15.113:0/3903505295 conn(0x56441501ea40 msgr2=0x564415047de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f8766ffd640 0 client.4975 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 shutdown_connections 2024-04-25T00:00:03.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.353+0000 7f879156e740 1 -- 172.21.15.113:0/3903505295 wait complete. 2024-04-25T00:00:03.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.354+0000 7f879156e740 20 client.4975 trim_cache size 0 max 16384 2024-04-25T00:00:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.384+0000 7f879156e740 1 Processor -- start 2024-04-25T00:00:03.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.384+0000 7f879156e740 1 -- start start 2024-04-25T00:00:03.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f879156e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415048d60 0x5644150652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x5644150647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415066600 0x5644150357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f8787fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415048d60 0x5644150652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f8787fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415048d60 0x5644150652a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37050/0 (socket says 172.21.15.113:37050) 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f8787fff640 1 -- 172.21.15.113:0/535616982 learned_addr learned my addr 172.21.15.113:0/535616982 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87880ac050 con 0x564415048d60 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87880ab090 con 0x56441504aa20 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87881518a0 con 0x564415066600 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f87877fe640 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x5644150647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.385+0000 7f878cd1b640 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415066600 0x5644150357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f8787fff640 1 -- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415066600 msgr2=0x5644150357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f8787fff640 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415066600 0x5644150357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f8787fff640 1 -- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 msgr2=0x5644150647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f8787fff640 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x5644150647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f8787fff640 1 -- 172.21.15.113:0/535616982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644150658f0 con 0x564415048d60 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f87877fe640 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x5644150647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f878cd1b640 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415066600 0x5644150357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f8787fff640 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415048d60 0x5644150652a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8778035040 tx=0x7f8778047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f715a96801f6ffa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.386+0000 7f876d7fa640 1 -- 172.21.15.113:0/535616982 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877804e030 con 0x564415048d60 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f876d7fa640 1 -- 172.21.15.113:0/535616982 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8778004040 con 0x564415048d60 2024-04-25T00:00:03.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f876d7fa640 1 -- 172.21.15.113:0/535616982 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778037040 con 0x564415048d60 2024-04-25T00:00:03.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415048d60 msgr2=0x5644150652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415048d60 0x5644150652a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8778035040 tx=0x7f8778047000 comp rx=0 tx=0).stop 2024-04-25T00:00:03.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 shutdown_connections 2024-04-25T00:00:03.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x5644150647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415066600 0x5644150357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 --2- 172.21.15.113:0/535616982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415048d60 0x5644150652a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 >> 172.21.15.113:0/535616982 conn(0x56441501ea40 msgr2=0x564415065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.387+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 shutdown_connections 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.388+0000 7f879156e740 1 -- 172.21.15.113:0/535616982 wait complete. 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.388+0000 7f879156e740 1 -- start start 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.389+0000 7f879156e740 1 Processor -- start 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415048d60 0x56441507b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f879156e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x56441501ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f879156e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441501f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f8787fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415048d60 0x56441507b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f8787fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415048d60 0x56441507b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:56592/0 (socket says 172.21.15.113:56592) 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f87877fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x56441501ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f87877fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x56441501ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:56344/0 (socket says 172.21.15.113:56344) 2024-04-25T00:00:03.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f8787fff640 1 -- 172.21.15.113:0/1505789964 learned_addr learned my addr 172.21.15.113:0/1505789964 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564415018b30 con 0x564415066600 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87880aaa40 con 0x56441504aa20 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564414f08490 con 0x564415048d60 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f879156e740 10 client.0 _ll_register_callbacks cb 0x7f87780096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.390+0000 7f878cd1b640 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441501f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f87877fe640 1 -- 172.21.15.113:0/1505789964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415048d60 msgr2=0x56441507b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f87877fe640 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415048d60 0x56441507b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f87877fe640 1 -- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 msgr2=0x56441501f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f87877fe640 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441501f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f87877fe640 1 -- 172.21.15.113:0/1505789964 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56441507ed50 con 0x56441504aa20 2024-04-25T00:00:03.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f878cd1b640 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441501f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f87877fe640 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x56441501ee20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f877c01b000 tx=0x7f877c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5327dd0d6fa56c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f8766ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-04-25T00:00:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f8766ffd640 1 -- 172.21.15.113:0/1505789964 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c019040 con 0x56441504aa20 2024-04-25T00:00:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8778004040 con 0x56441504aa20 2024-04-25T00:00:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.391+0000 7f8766ffd640 1 -- 172.21.15.113:0/1505789964 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f877c004440 con 0x56441504aa20 2024-04-25T00:00:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f8766ffd640 1 -- 172.21.15.113:0/1505789964 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f877c003610 con 0x56441504aa20 2024-04-25T00:00:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 10 client.4981 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:03.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5644150e9b10 con 0x56441504aa20 2024-04-25T00:00:03.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f8766ffd640 1 -- 172.21.15.113:0/1505789964 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8778004040 con 0x56441504aa20 2024-04-25T00:00:03.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f8766ffd640 1 -- 172.21.15.113:0/1505789964 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f877c0037b0 con 0x56441504aa20 2024-04-25T00:00:03.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 10 client.4981 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:03.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 10 client.4981 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:03.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 20 client.4981 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:03.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5644150dd0f0 0x5644150e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 4 client.4981 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:00:03.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564414f89f40 con 0x5644150dd0f0 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.392+0000 7f8787fff640 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5644150dd0f0 0x5644150e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.393+0000 7f8766ffd640 10 client.4981 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.393+0000 7f8787fff640 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5644150dd0f0 0x5644150e6870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2321ffa89154a8 server_cookie=5d742b8575db8d4 in_seq=0 out_seq=0 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.393+0000 7f8766ffd640 1 -- 172.21.15.113:0/1505789964 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+274 (crc 0 0 0) 0x564414f89f40 con 0x5644150dd0f0 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.393+0000 7f8766ffd640 10 client.4981 handle_command_reply: tid=0 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 client.4981 shutdown 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 msgr2=0x56441501ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x56441501ee20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f877c01b000 tx=0x7f877c004730 comp rx=0 tx=0).stop 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 shutdown_connections 2024-04-25T00:00:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5644150dd0f0 0x5644150e6870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f8766ffd640 0 client.4981 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56441504aa20 0x56441501ee20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564415066600 0x56441501f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 --2- 172.21.15.113:0/1505789964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564415048d60 0x56441507b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 >> 172.21.15.113:0/1505789964 conn(0x56441501ea40 msgr2=0x5644150640a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "scrub active (722 inodes in the stack)", 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "0e47a352-c82b-47ca-93cb-9f2424f27fd5": { 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "0e47a352-c82b-47ca-93cb-9f2424f27fd5", 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:03.401 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:03.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 shutdown_connections 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 1 -- 172.21.15.113:0/1505789964 wait complete. 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.394+0000 7f879156e740 20 client.4981 trim_cache size 0 max 16384 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f877406be60 msgr2=0x7f877406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f877406be60 0x7f877406e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8778034000 tx=0x7f877802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e6b0 msgr2=0x7f87881f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e6b0 0x7f87881f4660 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f878000c6b0 tx=0x7f878000cb70 comp rx=0 tx=0).stop 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 shutdown_connections 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f877406be60 0x7f877406e320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878815e2d0 0x7f87880b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f878815e6b0 0x7f87881f4660 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 --2- 172.21.15.113:0/500275588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87880b3080 0x7f87880b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 >> 172.21.15.113:0/500275588 conn(0x7f878806b0f0 msgr2=0x7f87880d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.395+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 shutdown_connections 2024-04-25T00:00:03.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:03.396+0000 7f878e77e640 1 -- 172.21.15.113:0/500275588 wait complete. 2024-04-25T00:00:03.411 INFO:tasks.cephfs.filesystem:scrub status for tag:0e47a352-c82b-47ca-93cb-9f2424f27fd5 - {'path': '/', 'tag': '0e47a352-c82b-47ca-93cb-9f2424f27fd5', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T00:00:21.833 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:00:21.837 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:00:33.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:33.414 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T00:00:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.530+0000 7ffa27d32640 1 Processor -- start 2024-04-25T00:00:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.530+0000 7ffa27d32640 1 -- start start 2024-04-25T00:00:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa27d32640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa200d06c0 0x7ffa2015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa27d32640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa201604d0 0x7ffa200cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa27d32640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa2015d6b0 con 0x7ffa2013f050 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa27d32640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2015c6a0 con 0x7ffa201604d0 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa27d32640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2015c8f0 con 0x7ffa200d06c0 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa262cf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa201604d0 0x7ffa200cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa262cf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa201604d0 0x7ffa200cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58552/0 (socket says 172.21.15.113:58552) 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa262cf640 1 -- 172.21.15.113:0/244017250 learned_addr learned my addr 172.21.15.113:0/244017250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa252cd640 1 --2- 172.21.15.113:0/244017250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa200d06c0 0x7ffa2015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa262cf640 1 -- 172.21.15.113:0/244017250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa200d06c0 msgr2=0x7ffa2015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa262cf640 1 --2- 172.21.15.113:0/244017250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa200d06c0 0x7ffa2015ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa262cf640 1 -- 172.21.15.113:0/244017250 >> v1:172.21.15.178:6789/0 conn(0x7ffa2013f050 legacy=0x7ffa200cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa262cf640 1 -- 172.21.15.113:0/244017250 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2015cbd0 con 0x7ffa201604d0 2024-04-25T00:00:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.531+0000 7ffa252cd640 1 --2- 172.21.15.113:0/244017250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa200d06c0 0x7ffa2015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.532+0000 7ffa262cf640 1 --2- 172.21.15.113:0/244017250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa201604d0 0x7ffa200cea60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009e50 tx=0x7ffa1802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9deeb808cf6449d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.532+0000 7ffa24acc640 1 -- 172.21.15.113:0/244017250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18036070 con 0x7ffa201604d0 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.532+0000 7ffa24acc640 1 -- 172.21.15.113:0/244017250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa18031600 con 0x7ffa201604d0 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.532+0000 7ffa24acc640 1 -- 172.21.15.113:0/244017250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180319d0 con 0x7ffa201604d0 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 -- 172.21.15.113:0/244017250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa201604d0 msgr2=0x7ffa200cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 --2- 172.21.15.113:0/244017250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa201604d0 0x7ffa200cea60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009e50 tx=0x7ffa1802a630 comp rx=0 tx=0).stop 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 -- 172.21.15.113:0/244017250 shutdown_connections 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 --2- 172.21.15.113:0/244017250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa200d06c0 0x7ffa2015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 --2- 172.21.15.113:0/244017250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa201604d0 0x7ffa200cea60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 -- 172.21.15.113:0/244017250 >> 172.21.15.113:0/244017250 conn(0x7ffa2006b0f0 msgr2=0x7ffa2006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 -- 172.21.15.113:0/244017250 shutdown_connections 2024-04-25T00:00:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.533+0000 7ffa27d32640 1 -- 172.21.15.113:0/244017250 wait complete. 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.534+0000 7ffa27d32640 1 Processor -- start 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.534+0000 7ffa27d32640 1 -- start start 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa27d32640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa200d06c0 0x7ffa201f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa27d32640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2013f050 0x7ffa2014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa27d32640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa201604d0 0x7ffa2012d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa27d32640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2013f430 con 0x7ffa200d06c0 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa27d32640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2015d190 con 0x7ffa201604d0 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa27d32640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa201608d0 con 0x7ffa2013f050 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa200d06c0 0x7ffa201f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa200d06c0 0x7ffa201f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58562/0 (socket says 172.21.15.113:58562) 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 -- 172.21.15.113:0/870843607 learned_addr learned my addr 172.21.15.113:0/870843607 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa262cf640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa201604d0 0x7ffa2012d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa252cd640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2013f050 0x7ffa2014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 -- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2013f050 msgr2=0x7ffa2014be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2013f050 0x7ffa2014be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 -- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa201604d0 msgr2=0x7ffa2012d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa201604d0 0x7ffa2012d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 -- 172.21.15.113:0/870843607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2014d840 con 0x7ffa200d06c0 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa252cd640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2013f050 0x7ffa2014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.535+0000 7ffa25ace640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa200d06c0 0x7ffa201f2650 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c460 tx=0x7ffa1c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.536+0000 7ffa167fc640 1 -- 172.21.15.113:0/870843607 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c016020 con 0x7ffa200d06c0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.536+0000 7ffa167fc640 1 -- 172.21.15.113:0/870843607 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c012870 con 0x7ffa200d06c0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.536+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2014c3b0 con 0x7ffa200d06c0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.536+0000 7ffa167fc640 1 -- 172.21.15.113:0/870843607 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c012c60 con 0x7ffa200d06c0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.536+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2014e0a0 con 0x7ffa200d06c0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.537+0000 7ffa167fc640 1 -- 172.21.15.113:0/870843607 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa1c019020 con 0x7ffa200d06c0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.538+0000 7ffa167fc640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffa0406be30 0x7ffa0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.538+0000 7ffa252cd640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffa0406be30 0x7ffa0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.538+0000 7ffa167fc640 1 -- 172.21.15.113:0/870843607 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7ffa1c08ac50 con 0x7ffa200d06c0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.538+0000 7ffa252cd640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffa0406be30 0x7ffa0406e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006fd0 tx=0x7ffa10008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.539+0000 7ffa2aa3d740 1 Processor -- start 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.539+0000 7ffa2aa3d740 1 -- start start 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e49d60 0x5558a8e4a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e67600 0x5558a8e67b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e1fa40 0x5558a8e65cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5558a8d836b0 con 0x5558a8e49d60 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558a8d2b880 con 0x5558a8e1fa40 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5558a8d7f670 con 0x5558a8e67600 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e49d60 0x5558a8e4a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e49d60 0x5558a8e4a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58564/0 (socket says 172.21.15.113:58564) 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 -- 172.21.15.113:0/265949727 learned_addr learned my addr 172.21.15.113:0/265949727 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa252cd640 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e67600 0x5558a8e67b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa262cf640 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e1fa40 0x5558a8e65cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 -- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e67600 msgr2=0x5558a8e67b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e67600 0x5558a8e67b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 -- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e1fa40 msgr2=0x5558a8e65cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e1fa40 0x5558a8e65cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 -- 172.21.15.113:0/265949727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a8b617d0 con 0x5558a8e49d60 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa262cf640 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e1fa40 0x5558a8e65cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa25ace640 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e49d60 0x5558a8e4a140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0039d0 tx=0x7ffa1c05d1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cab85532ed1030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa037fe640 1 -- 172.21.15.113:0/265949727 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c086020 con 0x5558a8e49d60 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa037fe640 1 -- 172.21.15.113:0/265949727 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c0891d0 con 0x5558a8e49d60 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.540+0000 7ffa037fe640 1 -- 172.21.15.113:0/265949727 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c05b040 con 0x5558a8e49d60 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/265949727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e49d60 msgr2=0x5558a8e4a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e49d60 0x5558a8e4a140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0039d0 tx=0x7ffa1c05d1f0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/265949727 shutdown_connections 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e1fa40 0x5558a8e65cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e67600 0x5558a8e67b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/265949727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e49d60 0x5558a8e4a140 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/265949727 >> 172.21.15.113:0/265949727 conn(0x5558a8e1c600 msgr2=0x5558a8e64b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/265949727 shutdown_connections 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.541+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/265949727 wait complete. 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.542+0000 7ffa2aa3d740 1 -- start start 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.542+0000 7ffa2aa3d740 1 Processor -- start 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e49d60 0x5558a8e4cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e67600 0x5558a8e47d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e45f10 0x5558a8e45380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5558a8e207e0 con 0x5558a8e67600 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558a8d3ef50 con 0x5558a8e49d60 2024-04-25T00:00:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5558a8e1a530 con 0x5558a8e45f10 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 10 client.0 _ll_register_callbacks cb 0x5558a8d7a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e67600 0x5558a8e47d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e67600 0x5558a8e47d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58574/0 (socket says 172.21.15.113:58574) 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 -- 172.21.15.113:0/3045854452 learned_addr learned my addr 172.21.15.113:0/3045854452 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa25ace640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e49d60 0x5558a8e4cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa262cf640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e45f10 0x5558a8e45380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 -- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e45f10 msgr2=0x5558a8e45380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e45f10 0x5558a8e45380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 -- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e49d60 msgr2=0x5558a8e4cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e49d60 0x5558a8e4cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 -- 172.21.15.113:0/3045854452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a8a973f0 con 0x5558a8e67600 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa262cf640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e45f10 0x5558a8e45380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa252cd640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e67600 0x5558a8e47d10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ffa2013c080 tx=0x7ffa100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4b5677d9072ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa00ff9640 1 -- 172.21.15.113:0/3045854452 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10023070 con 0x5558a8e67600 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa00ff9640 1 -- 172.21.15.113:0/3045854452 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7ffa1001f800 con 0x5558a8e67600 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.543+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1c0891d0 con 0x5558a8e67600 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa00ff9640 1 -- 172.21.15.113:0/3045854452 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7ffa1002e050 con 0x5558a8e67600 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 10 client.5346 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5558a8a570f0 con 0x5558a8e67600 2024-04-25T00:00:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa00ff9640 1 -- 172.21.15.113:0/3045854452 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c0891d0 con 0x5558a8e67600 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa00ff9640 1 -- 172.21.15.113:0/3045854452 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7ffa10031020 con 0x5558a8e67600 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 10 client.5346 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 10 client.5346 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 20 client.5346 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e75540 0x5558a8e75940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 4 client.5346 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a8d90bf0 con 0x5558a8e75540 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa25ace640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e75540 0x5558a8e75940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa00ff9640 10 client.5346 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.544+0000 7ffa25ace640 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e75540 0x5558a8e75940 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd687361d3665ad server_cookie=7a98d80fa6c06aa0 in_seq=0 out_seq=0 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa00ff9640 1 -- 172.21.15.113:0/3045854452 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558a8d90bf0 con 0x5558a8e75540 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa00ff9640 10 client.5346 handle_command_reply: tid=0 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 client.5346 shutdown 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e67600 msgr2=0x5558a8e47d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e67600 0x5558a8e47d10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ffa2013c080 tx=0x7ffa100046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 shutdown_connections 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e75540 0x5558a8e75940 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e45f10 0x5558a8e45380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa00ff9640 0 client.5346 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e67600 0x5558a8e47d10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/3045854452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e49d60 0x5558a8e4cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 >> 172.21.15.113:0/3045854452 conn(0x5558a8e1fa40 msgr2=0x5558a8e68980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 shutdown_connections 2024-04-25T00:00:33.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/3045854452 wait complete. 2024-04-25T00:00:33.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.545+0000 7ffa2aa3d740 20 client.5346 trim_cache size 0 max 16384 2024-04-25T00:00:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.562+0000 7ffa2aa3d740 1 Processor -- start 2024-04-25T00:00:33.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa2aa3d740 1 -- start start 2024-04-25T00:00:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e5e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e1fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e20360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201fe690 con 0x5558a8e45f10 2024-04-25T00:00:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201fd6a0 con 0x5558a8e67600 2024-04-25T00:00:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa201f5f80 con 0x5558a8e49d60 2024-04-25T00:00:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e5e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e5e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58580/0 (socket says 172.21.15.113:58580) 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.563+0000 7ffa25ace640 1 -- 172.21.15.113:0/758694951 learned_addr learned my addr 172.21.15.113:0/758694951 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa25ace640 1 -- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 msgr2=0x5558a8e1fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa25ace640 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e1fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa25ace640 1 -- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 msgr2=0x5558a8e20360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa262cf640 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e20360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa25ace640 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e20360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa25ace640 1 -- 172.21.15.113:0/758694951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a8e4d630 con 0x5558a8e45f10 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa262cf640 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e20360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa25ace640 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e5e1b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0552a0 tx=0x7ffa1c070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f5fc6fcf4a5aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa02ffd640 1 -- 172.21.15.113:0/758694951 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c086040 con 0x5558a8e45f10 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa02ffd640 1 -- 172.21.15.113:0/758694951 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c066070 con 0x5558a8e45f10 2024-04-25T00:00:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa02ffd640 1 -- 172.21.15.113:0/758694951 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c05b040 con 0x5558a8e45f10 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/758694951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 msgr2=0x5558a8e5e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.564+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e5e1b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0552a0 tx=0x7ffa1c070000 comp rx=0 tx=0).stop 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/758694951 shutdown_connections 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e20360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e1fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/758694951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e5e1b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/758694951 >> 172.21.15.113:0/758694951 conn(0x5558a8e1fa40 msgr2=0x5558a8e45150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/758694951 shutdown_connections 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/758694951 wait complete. 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 -- start start 2024-04-25T00:00:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.565+0000 7ffa2aa3d740 1 Processor -- start 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e1fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e20360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa2aa3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201f1740 con 0x5558a8e45f10 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558a8e4d0f0 con 0x5558a8e67600 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa2aa3d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5558a8e5a8f0 con 0x5558a8e49d60 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa2aa3d740 10 client.0 _ll_register_callbacks cb 0x7ffa1000d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e1fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa25ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e1fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58582/0 (socket says 172.21.15.113:58582) 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa25ace640 1 -- 172.21.15.113:0/306004890 learned_addr learned my addr 172.21.15.113:0/306004890 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa252cd640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e20360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa262cf640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa25ace640 1 -- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 msgr2=0x5558a8e20360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa25ace640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e20360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.566+0000 7ffa25ace640 1 -- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 msgr2=0x5558a8e5bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa25ace640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e5bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa25ace640 1 -- 172.21.15.113:0/306004890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a8e5d190 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa252cd640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e20360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa262cf640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa25ace640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e1fe20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0890c0 tx=0x7ffa1c003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f615da3c51da49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa00ff9640 1 -- 172.21.15.113:0/306004890 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c086040 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa00ff9640 1 -- 172.21.15.113:0/306004890 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7ffa1c057140 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1c066070 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa00ff9640 1 -- 172.21.15.113:0/306004890 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7ffa1c04b050 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 10 client.5352 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5558a8ec7c50 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa00ff9640 1 -- 172.21.15.113:0/306004890 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c066070 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa00ff9640 1 -- 172.21.15.113:0/306004890 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7ffa1c04e020 con 0x5558a8e45f10 2024-04-25T00:00:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 10 client.5352 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 10 client.5352 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 20 client.5352 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e46b10 0x5558a8e5c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 4 client.5352 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5558a8d8af40 con 0x5558a8e46b10 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.567+0000 7ffa252cd640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e46b10 0x5558a8e5c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa00ff9640 10 client.5352 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa252cd640 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e46b10 0x5558a8e5c170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2547ec9914dd58c4 server_cookie=9e97a3ca70a86641 in_seq=0 out_seq=0 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa00ff9640 1 -- 172.21.15.113:0/306004890 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5558a8d8af40 con 0x5558a8e46b10 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa00ff9640 10 client.5352 handle_command_reply: tid=0 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 client.5352 shutdown 2024-04-25T00:00:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 msgr2=0x5558a8e1fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e1fe20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0890c0 tx=0x7ffa1c003920 comp rx=0 tx=0).stop 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 shutdown_connections 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5558a8e46b10 0x5558a8e5c170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558a8e67600 0x5558a8e5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa00ff9640 0 client.5352 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5558a8e49d60 0x5558a8e20360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 --2- 172.21.15.113:0/306004890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5558a8e45f10 0x5558a8e1fe20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 >> 172.21.15.113:0/306004890 conn(0x5558a8e1fa40 msgr2=0x5558a8e485e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 shutdown_connections 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 1 -- 172.21.15.113:0/306004890 wait complete. 2024-04-25T00:00:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.568+0000 7ffa2aa3d740 20 client.5352 trim_cache size 0 max 16384 2024-04-25T00:00:33.577 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:33.577 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:00:33.577 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:00:33.577 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:33.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffa0406be30 msgr2=0x7ffa0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffa0406be30 0x7ffa0406e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006fd0 tx=0x7ffa10008040 comp rx=0 tx=0).stop 2024-04-25T00:00:33.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa200d06c0 msgr2=0x7ffa201f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa200d06c0 0x7ffa201f2650 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c460 tx=0x7ffa1c00c920 comp rx=0 tx=0).stop 2024-04-25T00:00:33.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 shutdown_connections 2024-04-25T00:00:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffa0406be30 0x7ffa0406e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa201604d0 0x7ffa2012d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2013f050 0x7ffa2014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 --2- 172.21.15.113:0/870843607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa200d06c0 0x7ffa201f2650 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 >> 172.21.15.113:0/870843607 conn(0x7ffa2006b0f0 msgr2=0x7ffa2013cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 shutdown_connections 2024-04-25T00:00:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.569+0000 7ffa27d32640 1 -- 172.21.15.113:0/870843607 wait complete. 2024-04-25T00:00:33.580 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:00:33.581 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T00:00:33.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.680+0000 7f0ccd7f4640 1 Processor -- start 2024-04-25T00:00:33.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.680+0000 7f0ccd7f4640 1 -- start start 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0ccd7f4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc815e510 0x7f0cc80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0ccd7f4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cc80d28c0 0x7f0cc80b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0ccd7f4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cc80d2e00 con 0x7f0cc80b61e0 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0ccd7f4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80d30e0 con 0x7f0cc80d28c0 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0ccd7f4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc80d33c0 con 0x7f0cc815e510 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cbffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cc80d28c0 0x7f0cc80b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cbffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cc80d28c0 0x7f0cc80b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58596/0 (socket says 172.21.15.113:58596) 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cbffff640 1 -- 172.21.15.113:0/813627870 learned_addr learned my addr 172.21.15.113:0/813627870 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cc77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0cc80b61e0 0x7f0cc8071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58890/0 (socket says 172.21.15.113:58890) 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cbffff640 1 -- 172.21.15.113:0/813627870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc815e510 msgr2=0x7f0cc80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cbffff640 1 --2- 172.21.15.113:0/813627870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc815e510 0x7f0cc80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cbffff640 1 -- 172.21.15.113:0/813627870 >> v1:172.21.15.113:6789/0 conn(0x7f0cc80b61e0 legacy=0x7f0cc8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cc67fc640 1 -- 172.21.15.113:0/813627870 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730141336 0 0) 0x7f0cc80d2e00 con 0x7f0cc80b61e0 2024-04-25T00:00:33.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.681+0000 7f0cbffff640 1 -- 172.21.15.113:0/813627870 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc80d36a0 con 0x7f0cc80d28c0 2024-04-25T00:00:33.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0cbffff640 1 --2- 172.21.15.113:0/813627870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc80d28c0 0x7f0cc80b5ca0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009870 tx=0x7f0cb002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c2cb413a52b156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0cc67fc640 1 -- 172.21.15.113:0/813627870 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0037070 con 0x7f0cc80d28c0 2024-04-25T00:00:33.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0cc67fc640 1 -- 172.21.15.113:0/813627870 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb0032440 con 0x7f0cc80d28c0 2024-04-25T00:00:33.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0cc67fc640 1 -- 172.21.15.113:0/813627870 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0032810 con 0x7f0cc80d28c0 2024-04-25T00:00:33.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/813627870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc80d28c0 msgr2=0x7f0cc80b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/813627870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc80d28c0 0x7f0cc80b5ca0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009870 tx=0x7f0cb002a1c0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/813627870 shutdown_connections 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/813627870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc80d28c0 0x7f0cc80b5ca0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/813627870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc815e510 0x7f0cc80d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.682+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/813627870 >> 172.21.15.113:0/813627870 conn(0x7f0cc806b720 msgr2=0x7f0cc80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.683+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/813627870 shutdown_connections 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.683+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/813627870 wait complete. 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 Processor -- start 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 -- start start 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cc80b61e0 0x7f0cc80b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc80d28c0 0x7f0cc80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc815e510 0x7f0cc81f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80b9dd0 con 0x7f0cc815e510 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80d3ea0 con 0x7f0cc80b61e0 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0ccd7f4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc80b83d0 con 0x7f0cc80d28c0 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc815e510 0x7f0cc81f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc815e510 0x7f0cc81f3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58602/0 (socket says 172.21.15.113:58602) 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.684+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1270006043 learned_addr learned my addr 172.21.15.113:0/1270006043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cbffff640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc80d28c0 0x7f0cc80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cc80b61e0 0x7f0cc80b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc80d28c0 msgr2=0x7f0cc80b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc80d28c0 0x7f0cc80b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cc80b61e0 msgr2=0x7f0cc80b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cc80b61e0 0x7f0cc80b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1270006043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc81f7590 con 0x7f0cc815e510 2024-04-25T00:00:33.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cc80b61e0 0x7f0cc80b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:33.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc815e510 0x7f0cc81f3ec0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c7a0 tx=0x7f0cc000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cbf7fe640 1 -- 172.21.15.113:0/1270006043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0018070 con 0x7f0cc815e510 2024-04-25T00:00:33.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cbf7fe640 1 -- 172.21.15.113:0/1270006043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc000a640 con 0x7f0cc815e510 2024-04-25T00:00:33.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc81f7870 con 0x7f0cc815e510 2024-04-25T00:00:33.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0cbf7fe640 1 -- 172.21.15.113:0/1270006043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc000aa70 con 0x7f0cc815e510 2024-04-25T00:00:33.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.685+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc81f7e30 con 0x7f0cc815e510 2024-04-25T00:00:33.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.686+0000 7f0cbf7fe640 1 -- 172.21.15.113:0/1270006043 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cc001b070 con 0x7f0cc815e510 2024-04-25T00:00:33.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.687+0000 7f0cbf7fe640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ca806be30 0x7f0ca806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.687+0000 7f0cbf7fe640 1 -- 172.21.15.113:0/1270006043 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f0cc008a4c0 con 0x7f0cc815e510 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.687+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ca806be30 0x7f0ca806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.687+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ca806be30 0x7f0ca806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d0d80 tx=0x7f0cb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.688+0000 7f0cd05d7740 1 Processor -- start 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.688+0000 7f0cd05d7740 1 -- start start 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acffbb80 0x5645acfae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acfdeca0 0x5645acfde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfde560 0x5645acfb3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5645acf17330 con 0x5645acfde560 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5645acebfab0 con 0x5645acffbb80 2024-04-25T00:00:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5645acf132f0 con 0x5645acfdeca0 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfde560 0x5645acfb3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfde560 0x5645acfb3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58604/0 (socket says 172.21.15.113:58604) 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1322295821 learned_addr learned my addr 172.21.15.113:0/1322295821 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acffbb80 0x5645acfae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cbffff640 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acfdeca0 0x5645acfde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acfdeca0 msgr2=0x5645acfde020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acfdeca0 0x5645acfde020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acffbb80 msgr2=0x5645acfae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acffbb80 0x5645acfae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 -- 172.21.15.113:0/1322295821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645accf57d0 con 0x5645acfde560 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfde560 0x5645acfb3370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00888a0 tx=0x7f0cc00581e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7f94cb3c05832f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cbcff9640 1 -- 172.21.15.113:0/1322295821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0056a60 con 0x5645acfde560 2024-04-25T00:00:33.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cbcff9640 1 -- 172.21.15.113:0/1322295821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc0054dd0 con 0x5645acfde560 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.689+0000 7f0cbcff9640 1 -- 172.21.15.113:0/1322295821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0058bc0 con 0x5645acfde560 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1322295821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfde560 msgr2=0x5645acfb3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfde560 0x5645acfb3370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00888a0 tx=0x7f0cc00581e0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1322295821 shutdown_connections 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfde560 0x5645acfb3370 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acfdeca0 0x5645acfde020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1322295821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acffbb80 0x5645acfae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1322295821 >> 172.21.15.113:0/1322295821 conn(0x5645acfb0780 msgr2=0x5645acff9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1322295821 shutdown_connections 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.690+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1322295821 wait complete. 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 -- start start 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 Processor -- start 2024-04-25T00:00:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acfe1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acffbb80 0x5645ad00c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645ad00cbf0 0x5645acfcafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5645acfb2930 con 0x5645acffbb80 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5645aced2f40 con 0x5645acfdeca0 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5645acfb1d20 con 0x5645ad00cbf0 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cd05d7740 10 client.0 _ll_register_callbacks cb 0x5645acc2abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.691+0000 7f0cbffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acffbb80 0x5645ad00c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acffbb80 0x5645ad00c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58606/0 (socket says 172.21.15.113:58606) 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 -- 172.21.15.113:0/3286276502 learned_addr learned my addr 172.21.15.113:0/3286276502 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645ad00cbf0 0x5645acfcafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acfe1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 -- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645ad00cbf0 msgr2=0x5645acfcafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645ad00cbf0 0x5645acfcafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 -- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 msgr2=0x5645acfe1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acfe1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 -- 172.21.15.113:0/3286276502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645acbeb0f0 con 0x5645acffbb80 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cbffff640 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acffbb80 0x5645ad00c6b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00090b0 tx=0x7f0cb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7599fbf1cd465f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0ca5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/3286276502 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0037040 con 0x5645acffbb80 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/3286276502 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f0cb0003a20 con 0x5645acffbb80 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc0054dd0 con 0x5645acffbb80 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/3286276502 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f0cb0003c20 con 0x5645acffbb80 2024-04-25T00:00:33.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cd05d7740 10 client.5367 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5645acd55f60 con 0x5645acffbb80 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/3286276502 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc0054dd0 con 0x5645acffbb80 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/3286276502 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f0cb0032440 con 0x5645acffbb80 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cd05d7740 10 client.5367 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cd05d7740 10 client.5367 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.692+0000 7f0cd05d7740 20 client.5367 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645acfcb9c0 0x5645acff99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0cd05d7740 4 client.5367 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645acf24870 con 0x5645acfcb9c0 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645acfcb9c0 0x5645acff99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645acfcb9c0 0x5645acff99c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6dd8807d8a88524 server_cookie=20af8f6d4924e155 in_seq=0 out_seq=0 2024-04-25T00:00:33.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0ca5ffb640 10 client.5367 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/3286276502 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645acf24870 con 0x5645acfcb9c0 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0ca5ffb640 10 client.5367 handle_command_reply: tid=0 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.693+0000 7f0cd05d7740 1 client.5367 shutdown 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acffbb80 msgr2=0x5645ad00c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acffbb80 0x5645ad00c6b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00090b0 tx=0x7f0cb0004730 comp rx=0 tx=0).stop 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 shutdown_connections 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645acfcb9c0 0x5645acff99c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645ad00cbf0 0x5645acfcafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0ca5ffb640 0 client.5367 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acffbb80 0x5645ad00c6b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3286276502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acfe1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 >> 172.21.15.113:0/3286276502 conn(0x5645acfde560 msgr2=0x5645acfadcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 shutdown_connections 2024-04-25T00:00:33.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3286276502 wait complete. 2024-04-25T00:00:33.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.694+0000 7f0cd05d7740 20 client.5367 trim_cache size 0 max 16384 2024-04-25T00:00:33.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 Processor -- start 2024-04-25T00:00:33.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 -- start start 2024-04-25T00:00:33.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acffc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645acffc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645ad00cbf0 0x5645ad01bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8200890 con 0x5645ad00cbf0 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ff970 con 0x5645acfdeca0 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc80c8f40 con 0x5645acffbb80 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645ad00cbf0 0x5645ad01bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645ad00cbf0 0x5645ad01bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58618/0 (socket says 172.21.15.113:58618) 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.710+0000 7f0cc77fe640 1 -- 172.21.15.113:0/3825950260 learned_addr learned my addr 172.21.15.113:0/3825950260 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acffc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cbffff640 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645acffc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc77fe640 1 -- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 msgr2=0x5645acffc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645acffc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc77fe640 1 -- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 msgr2=0x5645acffc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acffc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc77fe640 1 -- 172.21.15.113:0/3825950260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645acff1420 con 0x5645ad00cbf0 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acffc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645ad00cbf0 0x5645ad01bb40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00586a0 tx=0x7f0cc005cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342d2aca31f4b6db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0ca7fff640 1 -- 172.21.15.113:0/3825950260 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0054dd0 con 0x5645ad00cbf0 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0ca7fff640 1 -- 172.21.15.113:0/3825950260 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc00589c0 con 0x5645ad00cbf0 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0ca7fff640 1 -- 172.21.15.113:0/3825950260 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0060e00 con 0x5645ad00cbf0 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3825950260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645ad00cbf0 msgr2=0x5645ad01bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645ad00cbf0 0x5645ad01bb40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00586a0 tx=0x7f0cc005cbe0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3825950260 shutdown_connections 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645ad00cbf0 0x5645ad01bb40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645acffc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/3825950260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645acfdeca0 0x5645acffc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.711+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3825950260 >> 172.21.15.113:0/3825950260 conn(0x5645acfde560 msgr2=0x5645acfb4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.712+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3825950260 shutdown_connections 2024-04-25T00:00:33.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.712+0000 7f0cd05d7740 1 -- 172.21.15.113:0/3825950260 wait complete. 2024-04-25T00:00:33.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.712+0000 7f0cd05d7740 1 -- start start 2024-04-25T00:00:33.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.712+0000 7f0cd05d7740 1 Processor -- start 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfdeca0 0x5645acfb45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645ad007f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645ad00cbf0 0x5645acff56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5645acfaa340 con 0x5645acfdeca0 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80c9590 con 0x5645ad00cbf0 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5645acfad070 con 0x5645acffbb80 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 10 client.0 _ll_register_callbacks cb 0x7f0cb00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfdeca0 0x5645acfb45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfdeca0 0x5645acfb45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58622/0 (socket says 172.21.15.113:58622) 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 -- 172.21.15.113:0/1380091499 learned_addr learned my addr 172.21.15.113:0/1380091499 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645ad00cbf0 0x5645acff56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cbffff640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645ad007f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 -- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 msgr2=0x5645ad007f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645ad007f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 -- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645ad00cbf0 msgr2=0x5645acff56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645ad00cbf0 0x5645acff56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 -- 172.21.15.113:0/1380091499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645acbc41b0 con 0x5645acfdeca0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc77fe640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645ad00cbf0 0x5645acff56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cc6ffd640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfdeca0 0x5645acfb45b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x5645acfcbf10 tx=0x7f0cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc30acbdeea51c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0ca5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/1380091499 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8032060 con 0x5645acfdeca0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/1380091499 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f0cb800f040 con 0x5645acfdeca0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.713+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc00589c0 con 0x5645acfdeca0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/1380091499 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f0cb8038050 con 0x5645acfdeca0 2024-04-25T00:00:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 10 client.5373 fetch_fsmap learned FSMap version 15 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x5645ad0084d0 con 0x5645acfdeca0 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/1380091499 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc00589c0 con 0x5645acfdeca0 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/1380091499 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f0cb803b020 con 0x5645acfdeca0 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 10 client.5373 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 10 client.5373 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 20 client.5373 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645ad073e10 0x5645ad00a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 4 client.5373 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T00:00:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 --> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] -- command(tid 0: {"prefix": "perf dump"}) -- 0x5645acf1ebc0 con 0x5645ad073e10 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cbffff640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645ad073e10 0x5645ad00a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0ca5ffb640 10 client.5373 ms_handle_connect on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.714+0000 7f0cbffff640 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645ad073e10 0x5645ad00a7e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=414f9b781d92a247 server_cookie=49830ef0d088b505 in_seq=0 out_seq=0 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0ca5ffb640 1 -- 172.21.15.113:0/1380091499 <== mds.0 v2:172.21.15.113:6835/3337939581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+18002 (crc 0 0 0) 0x5645acf1ebc0 con 0x5645ad073e10 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0ca5ffb640 10 client.5373 handle_command_reply: tid=0 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 client.5373 shutdown 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfdeca0 msgr2=0x5645acfb45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfdeca0 0x5645acfb45b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x5645acfcbf10 tx=0x7f0cb80046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 shutdown_connections 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:6835/3337939581,v1:172.21.15.113:6837/3337939581] conn(0x5645ad073e10 0x5645ad00a7e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645ad00cbf0 0x5645acff56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0ca5ffb640 0 client.5373 ms_handle_reset on v2:172.21.15.113:6835/3337939581 2024-04-25T00:00:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5645acffbb80 0x5645ad007f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 --2- 172.21.15.113:0/1380091499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645acfdeca0 0x5645acfb45b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 >> 172.21.15.113:0/1380091499 conn(0x5645acfde560 msgr2=0x5645acffc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 shutdown_connections 2024-04-25T00:00:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 1 -- 172.21.15.113:0/1380091499 wait complete. 2024-04-25T00:00:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.715+0000 7f0cd05d7740 20 client.5373 trim_cache size 0 max 16384 2024-04-25T00:00:33.724 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:33.724 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 1682, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 1676, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 526466, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 13354579, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 16, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 6, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.169562248, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.080499954, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.037281924, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.042647106, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1676, 2024-04-25T00:00:33.725 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.210794033, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000125772 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 10, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000001177, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000117 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 120192, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 24464 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 29127, 2024-04-25T00:00:33.726 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 51561, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 9152184, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 44942264, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 8, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 2.831033689, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 2.177468518, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 1.191403896, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.060452327, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 51560, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.147239146, 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000022250 2024-04-25T00:00:33.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 49117, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.007453800, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000151 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 116432 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 4778, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 4778, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1584403, 2024-04-25T00:00:33.728 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 57157691, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.591606622, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.237460648, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.076988831, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.255285116, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4777, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.678834785, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000142104 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:00:33.729 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:00:33.730 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2693, 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.391230368, 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.006829272 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4129, 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.313993883, 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000318235 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.731 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 25194, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 25194, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 25194, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 435.593600869, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.017289576 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 12, 2024-04-25T00:00:33.732 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 17861, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 0, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 17861, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 17860, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-04-25T00:00:33.733 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 2, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 2, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 2, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 45462, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 45028, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 83, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-04-25T00:00:33.734 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 13, 2024-04-25T00:00:33.735 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 2000, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 1039, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 4034, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 3897, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 3364, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 53, 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 13971 2024-04-25T00:00:33.736 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 4087, 2024-04-25T00:00:33.737 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 3655, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 2166, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-25T00:00:33.738 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 26900, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 26900, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 36, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-04-25T00:00:33.739 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 36, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 4, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 97609987, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 0, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2513, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 124.057359012, 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.049366239 2024-04-25T00:00:33.740 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 0 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 14, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 2016, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 2002, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 13, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 13, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 0, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 17861, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 19862, 2024-04-25T00:00:33.741 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 2001, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 2, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 7332, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 7330, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 102328, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 231788 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 27961, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 25194, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 5, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-25T00:00:33.742 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2000, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.697430093, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000848715 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 30, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.004923994, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000164133 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.743 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.744 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.744 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-25T00:00:33.744 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6000, 2024-04-25T00:00:33.744 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 7.225157615, 2024-04-25T00:00:33.744 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001204192 2024-04-25T00:00:33.744 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.744 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6984, 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 5.508196221, 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000788687 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.745 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.746 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000037645, 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000037645 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-25T00:00:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.002698061, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002698061 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.075930595, 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.075930595 2024-04-25T00:00:33.748 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000084231, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000084231 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.582495310, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.197811913 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2166, 2024-04-25T00:00:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 401.207425239, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.185229651 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.750 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.060742662, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.060742662 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.751 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000277912, 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000277912 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.752 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-25T00:00:33.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8000, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.228201291, 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002278525 2024-04-25T00:00:33.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 1, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 0, 2024-04-25T00:00:33.755 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 18339, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 18339, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 77, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:00:33.756 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-25T00:00:33.757 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-25T00:00:33.758 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 250233, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 92, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 440, 2024-04-25T00:00:33.759 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 5, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 5416, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 77, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-25T00:00:33.760 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 13449557, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 661114, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 8426, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 94080729, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 37, 2024-04-25T00:00:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 8426, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8426, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 29.542365377, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.003506096 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8426, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 8519 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 8426, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 2069, 2024-04-25T00:00:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 6357, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 0, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 11, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 54, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 2565, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 32, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 1655, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 2034, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-25T00:00:33.763 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 2015, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 22, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-25T00:00:33.764 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-25T00:00:33.765 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 34, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 3, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 8, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 8, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 102, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 0, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 29 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.766 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 1, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 32, 2024-04-25T00:00:33.767 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 3655, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 3655, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 35582, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 8488178, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 35582, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:00:33.768 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 35582, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 8488178, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:33.769 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 8426, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 94463124, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 8426, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 94463124, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 8426, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 8426, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 8426, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 8426, 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:33.771 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 3655, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 382395, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 3655, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:00:33.772 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 52, 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 382395, 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x56499da12500": { 2024-04-25T00:00:33.773 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 26900, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 93415683, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 26900, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2513, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 93415683, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:00:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:00:33.775 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.775 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:33.775 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ca806be30 msgr2=0x7f0ca806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ca806be30 0x7f0ca806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d0d80 tx=0x7f0cb8008040 comp rx=0 tx=0).stop 2024-04-25T00:00:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc815e510 msgr2=0x7f0cc81f3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc815e510 0x7f0cc81f3ec0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c7a0 tx=0x7f0cc000cc60 comp rx=0 tx=0).stop 2024-04-25T00:00:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 shutdown_connections 2024-04-25T00:00:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ca806be30 0x7f0ca806e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc815e510 0x7f0cc81f3ec0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cc80d28c0 0x7f0cc80b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 --2- 172.21.15.113:0/1270006043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cc80b61e0 0x7f0cc80b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 >> 172.21.15.113:0/1270006043 conn(0x7f0cc806b720 msgr2=0x7f0cc80b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 shutdown_connections 2024-04-25T00:00:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:33.716+0000 7f0ccd7f4640 1 -- 172.21.15.113:0/1270006043 wait complete. 2024-04-25T00:00:33.779 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:00:33.779 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:00:33.813 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:00:33.857 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:00:33.857 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:00:33.857 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:00:33.857 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:33.874 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:00:33.874 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:33.929 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.064+0000 7fbc2ddee640 1 Processor -- start 2024-04-25T00:00:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.064+0000 7fbc2ddee640 1 -- start start 2024-04-25T00:00:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc2ddee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbc28144910 0x7fbc280b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc2ddee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc280b56b0 0x7fbc280cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc2ddee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc28144cf0 con 0x7fbc280b5240 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc2ddee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbc280b4510 con 0x7fbc280b56b0 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc2ddee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbc280b2f30 con 0x7fbc28144910 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc27fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc280b56b0 0x7fbc280cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc27fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc280b56b0 0x7fbc280cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58628/0 (socket says 172.21.15.113:58628) 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc27fff640 1 -- 172.21.15.113:0/1853581930 learned_addr learned my addr 172.21.15.113:0/1853581930 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc27fff640 1 -- 172.21.15.113:0/1853581930 >> v2:172.21.15.178:3300/0 conn(0x7fbc28144910 msgr2=0x7fbc280b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc277fe640 1 --2- 172.21.15.113:0/1853581930 >> v2:172.21.15.178:3300/0 conn(0x7fbc28144910 0x7fbc280b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc27fff640 1 --2- 172.21.15.113:0/1853581930 >> v2:172.21.15.178:3300/0 conn(0x7fbc28144910 0x7fbc280b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc27fff640 1 -- 172.21.15.113:0/1853581930 >> v1:172.21.15.178:6789/0 conn(0x7fbc280b5240 legacy=0x7fbc2806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc27fff640 1 -- 172.21.15.113:0/1853581930 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc280b3210 con 0x7fbc280b56b0 2024-04-25T00:00:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.065+0000 7fbc277fe640 1 --2- 172.21.15.113:0/1853581930 >> v2:172.21.15.178:3300/0 conn(0x7fbc28144910 0x7fbc280b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.066+0000 7fbc27fff640 1 --2- 172.21.15.113:0/1853581930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b56b0 0x7fbc280cd500 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009e50 tx=0x7fbc2002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2a31d745b67e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.066+0000 7fbc267fc640 1 -- 172.21.15.113:0/1853581930 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20036070 con 0x7fbc280b56b0 2024-04-25T00:00:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.066+0000 7fbc267fc640 1 -- 172.21.15.113:0/1853581930 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc20031600 con 0x7fbc280b56b0 2024-04-25T00:00:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.066+0000 7fbc267fc640 1 -- 172.21.15.113:0/1853581930 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200319d0 con 0x7fbc280b56b0 2024-04-25T00:00:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1853581930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b56b0 msgr2=0x7fbc280cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1853581930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b56b0 0x7fbc280cd500 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009e50 tx=0x7fbc2002a710 comp rx=0 tx=0).stop 2024-04-25T00:00:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1853581930 shutdown_connections 2024-04-25T00:00:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1853581930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b56b0 0x7fbc280cd500 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1853581930 >> v2:172.21.15.178:3300/0 conn(0x7fbc28144910 0x7fbc280b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1853581930 >> 172.21.15.113:0/1853581930 conn(0x7fbc280b1a50 msgr2=0x7fbc280b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1853581930 shutdown_connections 2024-04-25T00:00:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.067+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1853581930 wait complete. 2024-04-25T00:00:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.068+0000 7fbc2ddee640 1 Processor -- start 2024-04-25T00:00:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.068+0000 7fbc2ddee640 1 -- start start 2024-04-25T00:00:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc2ddee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b5240 0x7fbc281e4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc2ddee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc280b56b0 0x7fbc2815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc2ddee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc28144910 0x7fbc280aa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc2ddee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280b4da0 con 0x7fbc280b5240 2024-04-25T00:00:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc2ddee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280b37d0 con 0x7fbc28144910 2024-04-25T00:00:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc2ddee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc280b4f20 con 0x7fbc280b56b0 2024-04-25T00:00:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b5240 0x7fbc281e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b5240 0x7fbc281e4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58644/0 (socket says 172.21.15.113:58644) 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 -- 172.21.15.113:0/1978543008 learned_addr learned my addr 172.21.15.113:0/1978543008 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc27fff640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc28144910 0x7fbc280aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc26ffd640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc280b56b0 0x7fbc2815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 -- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc280b56b0 msgr2=0x7fbc2815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc280b56b0 0x7fbc2815d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 -- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc28144910 msgr2=0x7fbc280aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc28144910 0x7fbc280aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 -- 172.21.15.113:0/1978543008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc281e5230 con 0x7fbc280b5240 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc27fff640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc28144910 0x7fbc280aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc26ffd640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc280b56b0 0x7fbc2815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc277fe640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b5240 0x7fbc281e4970 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800c820 tx=0x7fbc1800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.069+0000 7fbc0ffff640 1 -- 172.21.15.113:0/1978543008 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18015070 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.070+0000 7fbc0ffff640 1 -- 172.21.15.113:0/1978543008 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc18007a30 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.070+0000 7fbc0ffff640 1 -- 172.21.15.113:0/1978543008 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18007de0 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.070+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc281e5510 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.070+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc281e5ad0 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.071+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc280691a0 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.071+0000 7fbc0ffff640 1 -- 172.21.15.113:0/1978543008 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc18018070 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.071+0000 7fbc0ffff640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbc1406be30 0x7fbc1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.071+0000 7fbc0ffff640 1 -- 172.21.15.113:0/1978543008 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7fbc1808ad10 con 0x7fbc280b5240 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.072+0000 7fbc26ffd640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbc1406be30 0x7fbc1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.072+0000 7fbc26ffd640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbc1406be30 0x7fbc1406e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc2806ea90 tx=0x7fbc1c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.076+0000 7fbc0ffff640 1 -- 172.21.15.113:0/1978543008 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbc18059730 con 0x7fbc280b5240 2024-04-25T00:00:34.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.250+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc281e5e70 con 0x7fbc280b5240 2024-04-25T00:00:34.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.251+0000 7fbc0ffff640 1 -- 172.21.15.113:0/1978543008 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16782 (secure 0 0 0) 0x7fbc18059030 con 0x7fbc280b5240 2024-04-25T00:00:34.253 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:34.253 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:09.800045+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:59:08.215866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:59:09.335267+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":33,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:34.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbc1406be30 msgr2=0x7fbc1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbc1406be30 0x7fbc1406e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc2806ea90 tx=0x7fbc1c009040 comp rx=0 tx=0).stop 2024-04-25T00:00:34.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b5240 msgr2=0x7fbc281e4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b5240 0x7fbc281e4970 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800c820 tx=0x7fbc1800cce0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 shutdown_connections 2024-04-25T00:00:34.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbc1406be30 0x7fbc1406e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc28144910 0x7fbc280aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc280b56b0 0x7fbc2815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 --2- 172.21.15.113:0/1978543008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc280b5240 0x7fbc281e4970 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 >> 172.21.15.113:0/1978543008 conn(0x7fbc280b1a50 msgr2=0x7fbc280b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 shutdown_connections 2024-04-25T00:00:34.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.253+0000 7fbc2ddee640 1 -- 172.21.15.113:0/1978543008 wait complete. 2024-04-25T00:00:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:59:08.215866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:59:09.335267+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 33, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:00:34.265 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:00:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.364+0000 7fe4c5476640 1 Processor -- start 2024-04-25T00:00:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.364+0000 7fe4c5476640 1 -- start start 2024-04-25T00:00:34.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4c5476640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe4c015e510 0x7fe4c00d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4c5476640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4c00d28c0 0x7fe4c00b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4c5476640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c00d2e00 con 0x7fe4c00b61e0 2024-04-25T00:00:34.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4c5476640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c00b6610 con 0x7fe4c00d28c0 2024-04-25T00:00:34.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4c5476640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c00d30e0 con 0x7fe4c015e510 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4be7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4c00d28c0 0x7fe4c00b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4be7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4c00d28c0 0x7fe4c00b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58660/0 (socket says 172.21.15.113:58660) 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4be7fc640 1 -- 172.21.15.113:0/1906870663 learned_addr learned my addr 172.21.15.113:0/1906870663 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4beffd640 1 --2- 172.21.15.113:0/1906870663 >> v2:172.21.15.178:3300/0 conn(0x7fe4c015e510 0x7fe4c00d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4be7fc640 1 -- 172.21.15.113:0/1906870663 >> v2:172.21.15.178:3300/0 conn(0x7fe4c015e510 msgr2=0x7fe4c00d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4be7fc640 1 --2- 172.21.15.113:0/1906870663 >> v2:172.21.15.178:3300/0 conn(0x7fe4c015e510 0x7fe4c00d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4be7fc640 1 -- 172.21.15.113:0/1906870663 >> v1:172.21.15.178:6789/0 conn(0x7fe4c00b61e0 legacy=0x7fe4c0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.365+0000 7fe4be7fc640 1 -- 172.21.15.113:0/1906870663 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c00d33c0 con 0x7fe4c00d28c0 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.366+0000 7fe4be7fc640 1 --2- 172.21.15.113:0/1906870663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c00d28c0 0x7fe4c00b5ca0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00d52a0 tx=0x7fe4b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dd5083ba2d2e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.366+0000 7fe4bdffb640 1 -- 172.21.15.113:0/1906870663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4037070 con 0x7fe4c00d28c0 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.366+0000 7fe4bdffb640 1 -- 172.21.15.113:0/1906870663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b4032440 con 0x7fe4c00d28c0 2024-04-25T00:00:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.366+0000 7fe4bdffb640 1 -- 172.21.15.113:0/1906870663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4032810 con 0x7fe4c00d28c0 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 -- 172.21.15.113:0/1906870663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c00d28c0 msgr2=0x7fe4c00b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 --2- 172.21.15.113:0/1906870663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c00d28c0 0x7fe4c00b5ca0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00d52a0 tx=0x7fe4b402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 -- 172.21.15.113:0/1906870663 shutdown_connections 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 --2- 172.21.15.113:0/1906870663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c00d28c0 0x7fe4c00b5ca0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 --2- 172.21.15.113:0/1906870663 >> v2:172.21.15.178:3300/0 conn(0x7fe4c015e510 0x7fe4c00d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 -- 172.21.15.113:0/1906870663 >> 172.21.15.113:0/1906870663 conn(0x7fe4c006b720 msgr2=0x7fe4c00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 -- 172.21.15.113:0/1906870663 shutdown_connections 2024-04-25T00:00:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.367+0000 7fe4c5476640 1 -- 172.21.15.113:0/1906870663 wait complete. 2024-04-25T00:00:34.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.368+0000 7fe4c5476640 1 Processor -- start 2024-04-25T00:00:34.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.368+0000 7fe4c5476640 1 -- start start 2024-04-25T00:00:34.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4c5476640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c00b61e0 0x7fe4c014d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4c5476640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c00d28c0 0x7fe4c01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4c5476640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c015e510 0x7fe4c00c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4c5476640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c00d3ea0 con 0x7fe4c015e510 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4c5476640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c00d3980 con 0x7fe4c00b61e0 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4c5476640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c00b9dd0 con 0x7fe4c00d28c0 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c015e510 0x7fe4c00c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c015e510 0x7fe4c00c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58676/0 (socket says 172.21.15.113:58676) 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 -- 172.21.15.113:0/786605641 learned_addr learned my addr 172.21.15.113:0/786605641 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4beffd640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c00b61e0 0x7fe4c014d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4be7fc640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c00d28c0 0x7fe4c01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 -- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c00d28c0 msgr2=0x7fe4c01608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c00d28c0 0x7fe4c01608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 -- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c00b61e0 msgr2=0x7fe4c014d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c00b61e0 0x7fe4c014d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 -- 172.21.15.113:0/786605641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c015f5a0 con 0x7fe4c015e510 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4bf7fe640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c015e510 0x7fe4c00c7d50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c730 tx=0x7fe4b800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.369+0000 7fe4a77fe640 1 -- 172.21.15.113:0/786605641 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8016020 con 0x7fe4c015e510 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.370+0000 7fe4a77fe640 1 -- 172.21.15.113:0/786605641 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8006d50 con 0x7fe4c015e510 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.370+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c015f880 con 0x7fe4c015e510 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.370+0000 7fe4a77fe640 1 -- 172.21.15.113:0/786605641 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8003bb0 con 0x7fe4c015e510 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.370+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c015fe40 con 0x7fe4c015e510 2024-04-25T00:00:34.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.371+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c00691a0 con 0x7fe4c015e510 2024-04-25T00:00:34.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.371+0000 7fe4a77fe640 1 -- 172.21.15.113:0/786605641 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4b80050d0 con 0x7fe4c015e510 2024-04-25T00:00:34.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.371+0000 7fe4a77fe640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.372+0000 7fe4a77fe640 1 -- 172.21.15.113:0/786605641 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7fe4b808a8f0 con 0x7fe4c015e510 2024-04-25T00:00:34.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.372+0000 7fe4beffd640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.372+0000 7fe4beffd640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009aa0 tx=0x7fe4b0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.376+0000 7fe4a77fe640 1 -- 172.21.15.113:0/786605641 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe4b8020020 con 0x7fe4c015e510 2024-04-25T00:00:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.546+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4c014d960 con 0x7fe4c015e510 2024-04-25T00:00:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.547+0000 7fe4a77fe640 1 -- 172.21.15.113:0/786605641 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fe4b8058cc0 con 0x7fe4c015e510 2024-04-25T00:00:34.548 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 msgr2=0x7fe4ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009aa0 tx=0x7fe4b0009190 comp rx=0 tx=0).stop 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c015e510 msgr2=0x7fe4c00c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c015e510 0x7fe4c00c7d50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c730 tx=0x7fe4b800cb30 comp rx=0 tx=0).stop 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 shutdown_connections 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c015e510 0x7fe4c00c7d50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c00d28c0 0x7fe4c01608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 --2- 172.21.15.113:0/786605641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c00b61e0 0x7fe4c014d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 >> 172.21.15.113:0/786605641 conn(0x7fe4c006b720 msgr2=0x7fe4c015d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 shutdown_connections 2024-04-25T00:00:34.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.549+0000 7fe4c5476640 1 -- 172.21.15.113:0/786605641 wait complete. 2024-04-25T00:00:34.561 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:00:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.659+0000 7f5608f6a640 1 Processor -- start 2024-04-25T00:00:34.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.659+0000 7f5608f6a640 1 -- start start 2024-04-25T00:00:34.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f5608f6a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f560413f050 0x7f56040cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f5608f6a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560415d6b0 con 0x7f56040d04f0 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f5608f6a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560415c8f0 con 0x7f56041606a0 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f5608f6a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f56040d09a0 con 0x7f560413f050 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f560259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f560413f050 0x7f56040cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f560259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f560413f050 0x7f56040cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58680/0 (socket says 172.21.15.113:58680) 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f55fbfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f56040d04f0 0x7f560415ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58902/0 (socket says 172.21.15.113:58902) 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f560259c640 1 -- 172.21.15.113:0/2537837018 learned_addr learned my addr 172.21.15.113:0/2537837018 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f560259c640 1 -- 172.21.15.113:0/2537837018 >> v1:172.21.15.178:6789/0 conn(0x7f56041606a0 legacy=0x7f56040cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f560259c640 1 -- 172.21.15.113:0/2537837018 >> v1:172.21.15.113:6789/0 conn(0x7f56040d04f0 legacy=0x7f560415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f560259c640 1 -- 172.21.15.113:0/2537837018 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560415cbd0 con 0x7f560413f050 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.660+0000 7f5601d9b640 1 -- 172.21.15.113:0/2537837018 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2006767378 0 0) 0x7f560415d6b0 con 0x7f56040d04f0 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f560259c640 1 --2- 172.21.15.113:0/2537837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560413f050 0x7f56040cffb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009870 tx=0x7f55f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b9aa8209ec89fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5601d9b640 1 -- 172.21.15.113:0/2537837018 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4037070 con 0x7f560413f050 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5601d9b640 1 -- 172.21.15.113:0/2537837018 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f4032440 con 0x7f560413f050 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5601d9b640 1 -- 172.21.15.113:0/2537837018 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4032810 con 0x7f560413f050 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5608f6a640 1 -- 172.21.15.113:0/2537837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560413f050 msgr2=0x7f56040cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2537837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560413f050 0x7f56040cffb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009870 tx=0x7f55f402a270 comp rx=0 tx=0).stop 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5608f6a640 1 -- 172.21.15.113:0/2537837018 shutdown_connections 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2537837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560413f050 0x7f56040cffb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5608f6a640 1 -- 172.21.15.113:0/2537837018 >> 172.21.15.113:0/2537837018 conn(0x7f560406b0f0 msgr2=0x7f560406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5608f6a640 1 -- 172.21.15.113:0/2537837018 shutdown_connections 2024-04-25T00:00:34.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.661+0000 7f5608f6a640 1 -- 172.21.15.113:0/2537837018 wait complete. 2024-04-25T00:00:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 Processor -- start 2024-04-25T00:00:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 -- start start 2024-04-25T00:00:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56040d04f0 0x7f56041e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f560413f050 0x7f56040b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56041606a0 0x7f560413b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f560415d190 con 0x7f56040d04f0 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f560415c6a0 con 0x7f56041606a0 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.663+0000 7f5608f6a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f560413f430 con 0x7f560413f050 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56040d04f0 0x7f56041e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56040d04f0 0x7f56041e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58690/0 (socket says 172.21.15.113:58690) 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 -- 172.21.15.113:0/2380097195 learned_addr learned my addr 172.21.15.113:0/2380097195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f5602d9d640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56041606a0 0x7f560413b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f55fbfff640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f560413f050 0x7f56040b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 -- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f560413f050 msgr2=0x7f56040b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f560413f050 0x7f56040b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 -- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56041606a0 msgr2=0x7f560413b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56041606a0 0x7f560413b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 -- 172.21.15.113:0/2380097195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560413b900 con 0x7f56040d04f0 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f5602d9d640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56041606a0 0x7f560413b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f55fbfff640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f560413f050 0x7f56040b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f560259c640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56040d04f0 0x7f56041e74f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009870 tx=0x7f55f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f55faffd640 1 -- 172.21.15.113:0/2380097195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4041020 con 0x7f56040d04f0 2024-04-25T00:00:34.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f55faffd640 1 -- 172.21.15.113:0/2380097195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f4030970 con 0x7f56040d04f0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f55faffd640 1 -- 172.21.15.113:0/2380097195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4037040 con 0x7f56040d04f0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56041e6450 con 0x7f56040d04f0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.664+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041e6a10 con 0x7f56040d04f0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.665+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56040691a0 con 0x7f56040d04f0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.666+0000 7f55faffd640 1 -- 172.21.15.113:0/2380097195 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55f4044020 con 0x7f56040d04f0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.666+0000 7f55faffd640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f55e406be30 0x7f55e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.666+0000 7f55faffd640 1 -- 172.21.15.113:0/2380097195 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f55f40ab810 con 0x7f56040d04f0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.666+0000 7f55fbfff640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f55e406be30 0x7f55e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.666+0000 7f55fbfff640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f55e406be30 0x7f55e406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f56040b1960 tx=0x7f55ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.670+0000 7f55faffd640 1 -- 172.21.15.113:0/2380097195 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f55f404b030 con 0x7f56040d04f0 2024-04-25T00:00:34.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.843+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f56040d0e50 con 0x7f56040d04f0 2024-04-25T00:00:34.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.843+0000 7f55faffd640 1 -- 172.21.15.113:0/2380097195 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f55f407a210 con 0x7f56040d04f0 2024-04-25T00:00:34.844 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f55e406be30 msgr2=0x7f55e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f55e406be30 0x7f55e406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f56040b1960 tx=0x7f55ec009040 comp rx=0 tx=0).stop 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56040d04f0 msgr2=0x7f56041e74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56040d04f0 0x7f56041e74f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009870 tx=0x7f55f40046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 shutdown_connections 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f55e406be30 0x7f55e406e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56041606a0 0x7f560413b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f560413f050 0x7f56040b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 --2- 172.21.15.113:0/2380097195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56040d04f0 0x7f56041e74f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 >> 172.21.15.113:0/2380097195 conn(0x7f560406b0f0 msgr2=0x7f56041523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 shutdown_connections 2024-04-25T00:00:34.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.845+0000 7f5608f6a640 1 -- 172.21.15.113:0/2380097195 wait complete. 2024-04-25T00:00:34.857 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:00:34.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.955+0000 7f371e514640 1 Processor -- start 2024-04-25T00:00:34.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.955+0000 7f371e514640 1 -- start start 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f371e514640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f371815b670 0x7f371815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f371e514640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371806cea0 0x7f371806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f371e514640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3718071170 con 0x7f371815bff0 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f371e514640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f371815c4a0 con 0x7f371815b670 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f371e514640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f371806d810 con 0x7f371806cea0 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f3717fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f371815b670 0x7f371815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f3717fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f371815b670 0x7f371815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58696/0 (socket says 172.21.15.113:58696) 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.956+0000 7f3717fff640 1 -- 172.21.15.113:0/2272696202 learned_addr learned my addr 172.21.15.113:0/2272696202 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3717fff640 1 -- 172.21.15.113:0/2272696202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371806cea0 msgr2=0x7f371806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:34.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f371cab1640 1 --2- 172.21.15.113:0/2272696202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371806cea0 0x7f371806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3717fff640 1 --2- 172.21.15.113:0/2272696202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371806cea0 0x7f371806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3717fff640 1 -- 172.21.15.113:0/2272696202 >> v1:172.21.15.178:6789/0 conn(0x7f371815bff0 legacy=0x7f37180cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:34.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3717fff640 1 -- 172.21.15.113:0/2272696202 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3718071450 con 0x7f371815b670 2024-04-25T00:00:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3717fff640 1 --2- 172.21.15.113:0/2272696202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815b670 0x7f371815ba50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f371806f1a0 tx=0x7f370802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867a72c60822d4b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3716ffd640 1 -- 172.21.15.113:0/2272696202 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708037070 con 0x7f371815b670 2024-04-25T00:00:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3716ffd640 1 -- 172.21.15.113:0/2272696202 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370802ae10 con 0x7f371815b670 2024-04-25T00:00:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.957+0000 7f3716ffd640 1 -- 172.21.15.113:0/2272696202 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708032660 con 0x7f371815b670 2024-04-25T00:00:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 -- 172.21.15.113:0/2272696202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815b670 msgr2=0x7f371815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 --2- 172.21.15.113:0/2272696202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815b670 0x7f371815ba50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f371806f1a0 tx=0x7f370802a100 comp rx=0 tx=0).stop 2024-04-25T00:00:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 -- 172.21.15.113:0/2272696202 shutdown_connections 2024-04-25T00:00:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 --2- 172.21.15.113:0/2272696202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371806cea0 0x7f371806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 --2- 172.21.15.113:0/2272696202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815b670 0x7f371815ba50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 -- 172.21.15.113:0/2272696202 >> 172.21.15.113:0/2272696202 conn(0x7f371806b0f0 msgr2=0x7f371806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 -- 172.21.15.113:0/2272696202 shutdown_connections 2024-04-25T00:00:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.958+0000 7f371e514640 1 -- 172.21.15.113:0/2272696202 wait complete. 2024-04-25T00:00:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.959+0000 7f371e514640 1 Processor -- start 2024-04-25T00:00:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.959+0000 7f371e514640 1 -- start start 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371e514640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f371806cea0 0x7f3718147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371e514640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371815b670 0x7f37181540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371e514640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815bff0 0x7f37181504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371e514640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37180cc910 con 0x7f371815bff0 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371e514640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3718071a10 con 0x7f371806cea0 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371e514640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37180cca90 con 0x7f371815b670 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815bff0 0x7f37181504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815bff0 0x7f37181504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58700/0 (socket says 172.21.15.113:58700) 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 -- 172.21.15.113:0/3595748914 learned_addr learned my addr 172.21.15.113:0/3595748914 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f37177fe640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371815b670 0x7f37181540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f3717fff640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f371806cea0 0x7f3718147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 -- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371815b670 msgr2=0x7f37181540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371815b670 0x7f37181540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 -- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f371806cea0 msgr2=0x7f3718147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f371806cea0 0x7f3718147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.960+0000 7f371cab1640 1 -- 172.21.15.113:0/3595748914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37181548f0 con 0x7f371815bff0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.961+0000 7f371cab1640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815bff0 0x7f37181504e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f371000cc30 tx=0x7f371000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.961+0000 7f3714ff9640 1 -- 172.21.15.113:0/3595748914 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371000ce40 con 0x7f371815bff0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.961+0000 7f3714ff9640 1 -- 172.21.15.113:0/3595748914 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3710003990 con 0x7f371815bff0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.961+0000 7f3714ff9640 1 -- 172.21.15.113:0/3595748914 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710003d40 con 0x7f371815bff0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.961+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37181578b0 con 0x7f371815bff0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.961+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3718157e70 con 0x7f371815bff0 2024-04-25T00:00:34.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.962+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37180691a0 con 0x7f371815bff0 2024-04-25T00:00:34.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.962+0000 7f3714ff9640 1 -- 172.21.15.113:0/3595748914 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3710003ee0 con 0x7f371815bff0 2024-04-25T00:00:34.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.963+0000 7f3714ff9640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f370406be30 0x7f370406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.963+0000 7f3714ff9640 1 -- 172.21.15.113:0/3595748914 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f371008d190 con 0x7f371815bff0 2024-04-25T00:00:34.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.963+0000 7f3717fff640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f370406be30 0x7f370406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.963+0000 7f3717fff640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f370406be30 0x7f370406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3708034000 tx=0x7f370802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:34.967+0000 7f3714ff9640 1 -- 172.21.15.113:0/3595748914 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f371005bbb0 con 0x7f371815bff0 2024-04-25T00:00:35.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.138+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f37181545e0 con 0x7f371815bff0 2024-04-25T00:00:35.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.138+0000 7f3714ff9640 1 -- 172.21.15.113:0/3595748914 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f371005b4b0 con 0x7f371815bff0 2024-04-25T00:00:35.140 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.140+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f370406be30 msgr2=0x7f370406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.140+0000 7f371e514640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f370406be30 0x7f370406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3708034000 tx=0x7f370802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.140+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815bff0 msgr2=0x7f37181504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.140+0000 7f371e514640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815bff0 0x7f37181504e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f371000cc30 tx=0x7f371000a5c0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 shutdown_connections 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f370406be30 0x7f370406e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371815bff0 0x7f37181504e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f371815b670 0x7f37181540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 --2- 172.21.15.113:0/3595748914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f371806cea0 0x7f3718147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 >> 172.21.15.113:0/3595748914 conn(0x7f371806b0f0 msgr2=0x7f37180cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 shutdown_connections 2024-04-25T00:00:35.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.141+0000 7f371e514640 1 -- 172.21.15.113:0/3595748914 wait complete. 2024-04-25T00:00:35.152 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:00:35.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.249+0000 7f84e6bee640 1 Processor -- start 2024-04-25T00:00:35.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.249+0000 7f84e6bee640 1 -- start start 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.249+0000 7f84e6bee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e015e2d0 0x7f84e015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.249+0000 7f84e6bee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84e00b4a40 0x7f84e00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.249+0000 7f84e6bee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e015f4f0 con 0x7f84e00b3080 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.249+0000 7f84e6bee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f84e00cde00 con 0x7f84e00b4a40 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.249+0000 7f84e6bee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84e015f7d0 con 0x7f84e015e2d0 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84e518b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84e00b4a40 0x7f84e00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84e518b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84e00b4a40 0x7f84e00ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58716/0 (socket says 172.21.15.113:58716) 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84dffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f84e00b3080 0x7f84e00b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58918/0 (socket says 172.21.15.113:58918) 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84e518b640 1 -- 172.21.15.113:0/4137809343 learned_addr learned my addr 172.21.15.113:0/4137809343 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84e518b640 1 -- 172.21.15.113:0/4137809343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e015e2d0 msgr2=0x7f84e015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:35.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84e518b640 1 --2- 172.21.15.113:0/4137809343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e015e2d0 0x7f84e015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84e518b640 1 -- 172.21.15.113:0/4137809343 >> v1:172.21.15.113:6789/0 conn(0x7f84e00b3080 legacy=0x7f84e00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84e518b640 1 -- 172.21.15.113:0/4137809343 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e015fab0 con 0x7f84e00b4a40 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.250+0000 7f84df7fe640 1 -- 172.21.15.113:0/4137809343 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307544405 0 0) 0x7f84e015f4f0 con 0x7f84e00b3080 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e518b640 1 --2- 172.21.15.113:0/4137809343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e00b4a40 0x7f84e00ce290 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009e50 tx=0x7f84d802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d57e0ecd392cff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84df7fe640 1 -- 172.21.15.113:0/4137809343 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8036070 con 0x7f84e00b4a40 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84df7fe640 1 -- 172.21.15.113:0/4137809343 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8031600 con 0x7f84e00b4a40 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84df7fe640 1 -- 172.21.15.113:0/4137809343 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d80319d0 con 0x7f84e00b4a40 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 -- 172.21.15.113:0/4137809343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e00b4a40 msgr2=0x7f84e00ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 --2- 172.21.15.113:0/4137809343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e00b4a40 0x7f84e00ce290 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009e50 tx=0x7f84d802a710 comp rx=0 tx=0).stop 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 -- 172.21.15.113:0/4137809343 shutdown_connections 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 --2- 172.21.15.113:0/4137809343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e00b4a40 0x7f84e00ce290 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 --2- 172.21.15.113:0/4137809343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e015e2d0 0x7f84e015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 -- 172.21.15.113:0/4137809343 >> 172.21.15.113:0/4137809343 conn(0x7f84e006b0f0 msgr2=0x7f84e006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 -- 172.21.15.113:0/4137809343 shutdown_connections 2024-04-25T00:00:35.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.251+0000 7f84e6bee640 1 -- 172.21.15.113:0/4137809343 wait complete. 2024-04-25T00:00:35.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.252+0000 7f84e6bee640 1 Processor -- start 2024-04-25T00:00:35.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.252+0000 7f84e6bee640 1 -- start start 2024-04-25T00:00:35.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e6bee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e00b3080 0x7f84e0140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e6bee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e00b4a40 0x7f84e0147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e6bee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e015e2d0 0x7f84e0147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e6bee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f84e015f2b0 con 0x7f84e015e2d0 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e6bee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84e015f130 con 0x7f84e00b4a40 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e6bee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84e015e6b0 con 0x7f84e00b3080 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e518b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e015e2d0 0x7f84e0147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e518b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e015e2d0 0x7f84e0147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58724/0 (socket says 172.21.15.113:58724) 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.253+0000 7f84e518b640 1 -- 172.21.15.113:0/3719162278 learned_addr learned my addr 172.21.15.113:0/3719162278 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e498a640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e00b3080 0x7f84e0140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84dffff640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e00b4a40 0x7f84e0147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e518b640 1 -- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e00b3080 msgr2=0x7f84e0140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e518b640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e00b3080 0x7f84e0140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e518b640 1 -- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e00b4a40 msgr2=0x7f84e0147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e518b640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e00b4a40 0x7f84e0147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e518b640 1 -- 172.21.15.113:0/3719162278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e0153fc0 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e498a640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e00b3080 0x7f84e0140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e518b640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e015e2d0 0x7f84e0147e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009e50 tx=0x7f84d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84dd7fa640 1 -- 172.21.15.113:0/3719162278 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8041020 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84dd7fa640 1 -- 172.21.15.113:0/3719162278 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8007780 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e0144b20 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84dd7fa640 1 -- 172.21.15.113:0/3719162278 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8036040 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.254+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01450e0 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.255+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e00691a0 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.256+0000 7f84dd7fa640 1 -- 172.21.15.113:0/3719162278 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84d8044020 con 0x7f84e015e2d0 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.256+0000 7f84dd7fa640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84c406be30 0x7f84c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.256+0000 7f84dd7fa640 1 -- 172.21.15.113:0/3719162278 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f84d80abbe0 con 0x7f84e015e2d0 2024-04-25T00:00:35.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.256+0000 7f84e498a640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84c406be30 0x7f84c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.256+0000 7f84e498a640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84c406be30 0x7f84c406e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f84e0148980 tx=0x7f84d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.260+0000 7f84dd7fa640 1 -- 172.21.15.113:0/3719162278 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f84d804b030 con 0x7f84e015e2d0 2024-04-25T00:00:35.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.449+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84e0154690 con 0x7f84e015e2d0 2024-04-25T00:00:35.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.450+0000 7f84dd7fa640 1 -- 172.21.15.113:0/3719162278 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f84e0153fc0 con 0x7f84e015e2d0 2024-04-25T00:00:35.451 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84c406be30 msgr2=0x7f84c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84c406be30 0x7f84c406e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f84e0148980 tx=0x7f84d4008040 comp rx=0 tx=0).stop 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e015e2d0 msgr2=0x7f84e0147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e015e2d0 0x7f84e0147e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009e50 tx=0x7f84d8004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 shutdown_connections 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84c406be30 0x7f84c406e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e00b4a40 0x7f84e0147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e015e2d0 0x7f84e0147e90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 --2- 172.21.15.113:0/3719162278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84e00b3080 0x7f84e0140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 >> 172.21.15.113:0/3719162278 conn(0x7f84e006b0f0 msgr2=0x7f84e00b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 shutdown_connections 2024-04-25T00:00:35.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.452+0000 7f84e6bee640 1 -- 172.21.15.113:0/3719162278 wait complete. 2024-04-25T00:00:35.464 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:00:35.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:35.464 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:00:35.508 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:00:35.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:35.508 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:00:35.563 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:00:35.563 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:00:35.563 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:35.580 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:00:35.580 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:00:35.580 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:00:35.580 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:35.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:00:35.618 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:00:35.618 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:00:35.618 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:35.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:00:35.635 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:35.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.753+0000 7f7395272640 1 Processor -- start 2024-04-25T00:00:35.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.753+0000 7f7395272640 1 -- start start 2024-04-25T00:00:35.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f7395272640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 0x7f73900b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f7395272640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73901453c0 0x7f7390071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f7395272640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73900cfad0 con 0x7f7390144d60 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f7395272640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7390145c40 con 0x7f73901453c0 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f7395272640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73900cf230 con 0x7f739006e6e0 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f738f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73901453c0 0x7f7390071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f738f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73901453c0 0x7f7390071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58726/0 (socket says 172.21.15.113:58726) 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f738f7fe640 1 -- 172.21.15.113:0/1333114013 learned_addr learned my addr 172.21.15.113:0/1333114013 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f738f7fe640 1 -- 172.21.15.113:0/1333114013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 msgr2=0x7f73900b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f738f7fe640 1 --2- 172.21.15.113:0/1333114013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 0x7f73900b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f738f7fe640 1 -- 172.21.15.113:0/1333114013 >> v1:172.21.15.178:6789/0 conn(0x7f7390144d60 legacy=0x7f739006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:35.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.754+0000 7f738f7fe640 1 -- 172.21.15.113:0/1333114013 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73900cf510 con 0x7f73901453c0 2024-04-25T00:00:35.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.755+0000 7f738f7fe640 1 --2- 172.21.15.113:0/1333114013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73901453c0 0x7f7390071330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7388009870 tx=0x7f738802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76eb573acde9f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.755+0000 7f738dffb640 1 -- 172.21.15.113:0/1333114013 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388037070 con 0x7f73901453c0 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.755+0000 7f738dffb640 1 -- 172.21.15.113:0/1333114013 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7388032440 con 0x7f73901453c0 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.755+0000 7f738dffb640 1 -- 172.21.15.113:0/1333114013 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388032810 con 0x7f73901453c0 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 -- 172.21.15.113:0/1333114013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73901453c0 msgr2=0x7f7390071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 --2- 172.21.15.113:0/1333114013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73901453c0 0x7f7390071330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7388009870 tx=0x7f738802a180 comp rx=0 tx=0).stop 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 -- 172.21.15.113:0/1333114013 shutdown_connections 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 --2- 172.21.15.113:0/1333114013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 0x7f73900b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 --2- 172.21.15.113:0/1333114013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73901453c0 0x7f7390071330 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 -- 172.21.15.113:0/1333114013 >> 172.21.15.113:0/1333114013 conn(0x7f73900b1a50 msgr2=0x7f73900b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 -- 172.21.15.113:0/1333114013 shutdown_connections 2024-04-25T00:00:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.756+0000 7f7395272640 1 -- 172.21.15.113:0/1333114013 wait complete. 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.757+0000 7f7395272640 1 Processor -- start 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.757+0000 7f7395272640 1 -- start start 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 0x7f73901e7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7390144d60 0x7f739014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73901453c0 0x7f7390135a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7390145140 con 0x7f7390144d60 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7390070e10 con 0x7f73901453c0 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7390071870 con 0x7f739006e6e0 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7390144d60 0x7f739014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7390144d60 0x7f739014bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58728/0 (socket says 172.21.15.113:58728) 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 -- 172.21.15.113:0/104717209 learned_addr learned my addr 172.21.15.113:0/104717209 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738effd640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 0x7f73901e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738f7fe640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73901453c0 0x7f7390135a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 -- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 msgr2=0x7f73901e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 0x7f73901e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 -- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73901453c0 msgr2=0x7f7390135a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73901453c0 0x7f7390135a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 -- 172.21.15.113:0/104717209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739014c220 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f738e7fc640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7390144d60 0x7f739014bce0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f738400b4d0 tx=0x7f738400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f73777fe640 1 -- 172.21.15.113:0/104717209 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384014070 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7390135fb0 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f73777fe640 1 -- 172.21.15.113:0/104717209 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7384010870 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f73777fe640 1 -- 172.21.15.113:0/104717209 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384010c60 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.758+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7390139820 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.759+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7390068a60 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.760+0000 7f73777fe640 1 -- 172.21.15.113:0/104717209 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7384017070 con 0x7f7390144d60 2024-04-25T00:00:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.760+0000 7f73777fe640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f737c06be30 0x7f737c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.760+0000 7f73777fe640 1 -- 172.21.15.113:0/104717209 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f738408aca0 con 0x7f7390144d60 2024-04-25T00:00:35.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.760+0000 7f738effd640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f737c06be30 0x7f737c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.761+0000 7f738effd640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f737c06be30 0x7f737c06e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f739013f0b0 tx=0x7f7380009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.765+0000 7f73777fe640 1 -- 172.21.15.113:0/104717209 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7390068a60 con 0x7f7390144d60 2024-04-25T00:00:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.972+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73900b27e0 con 0x7f7390144d60 2024-04-25T00:00:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.972+0000 7f73777fe640 1 -- 172.21.15.113:0/104717209 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5193 (secure 0 0 0) 0x7f739014c220 con 0x7f7390144d60 2024-04-25T00:00:35.974 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2613614055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2613614055},{"type":"v1","addr":"172.21.15.178:6834","nonce":2613614055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2494825991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2494825991},{"type":"v1","addr":"172.21.15.178:6835","nonce":2494825991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2843749120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2843749120},{"type":"v1","addr":"172.21.15.113:6836","nonce":2843749120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-24T23:59:10.365260+0000","modified":"2024-04-24T23:59:17.031612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3337939581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3337939581},{"type":"v1","addr":"172.21.15.113:6837","nonce":3337939581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-04-25T00:00:35.974 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 15 2024-04-25T00:00:35.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f737c06be30 msgr2=0x7f737c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f737c06be30 0x7f737c06e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f739013f0b0 tx=0x7f7380009040 comp rx=0 tx=0).stop 2024-04-25T00:00:35.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7390144d60 msgr2=0x7f739014bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7390144d60 0x7f739014bce0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f738400b4d0 tx=0x7f738400b990 comp rx=0 tx=0).stop 2024-04-25T00:00:35.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 shutdown_connections 2024-04-25T00:00:35.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f737c06be30 0x7f737c06e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73901453c0 0x7f7390135a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7390144d60 0x7f739014bce0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 --2- 172.21.15.113:0/104717209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f739006e6e0 0x7f73901e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.974+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 >> 172.21.15.113:0/104717209 conn(0x7f73900b1a50 msgr2=0x7f7390071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.975+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 shutdown_connections 2024-04-25T00:00:35.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:35.975+0000 7f7395272640 1 -- 172.21.15.113:0/104717209 wait complete. 2024-04-25T00:00:35.986 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:36.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.082+0000 7fe04b1b7640 1 Processor -- start 2024-04-25T00:00:36.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.082+0000 7fe04b1b7640 1 -- start start 2024-04-25T00:00:36.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.082+0000 7fe04b1b7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe044144910 0x7fe0440b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.082+0000 7fe04b1b7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe044144cf0 con 0x7fe0440b56b0 2024-04-25T00:00:36.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.082+0000 7fe04b1b7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0440b2f30 con 0x7fe0440b5240 2024-04-25T00:00:36.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.082+0000 7fe04b1b7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe0440b4510 con 0x7fe044144910 2024-04-25T00:00:36.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe049754640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe0440b56b0 0x7fe0440cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58928/0 (socket says 172.21.15.113:58928) 2024-04-25T00:00:36.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe049754640 1 -- 172.21.15.113:0/1161397564 learned_addr learned my addr 172.21.15.113:0/1161397564 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:36.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093529521 0 0) 0x7fe044144cf0 con 0x7fe0440b56b0 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe034003520 con 0x7fe0440b56b0 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe048f53640 1 --2- 172.21.15.113:0/1161397564 >> v2:172.21.15.178:3300/0 conn(0x7fe044144910 0x7fe0440b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4291097335 0 0) 0x7fe034003520 con 0x7fe0440b56b0 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 >> v2:172.21.15.178:3300/0 conn(0x7fe044144910 msgr2=0x7fe0440b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 --2- 172.21.15.113:0/1161397564 >> v2:172.21.15.178:3300/0 conn(0x7fe044144910 0x7fe0440b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 >> v1:172.21.15.178:6789/0 conn(0x7fe0440b5240 legacy=0x7fe04406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0440b3210 con 0x7fe0440b56b0 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe03c0019f0 con 0x7fe0440b56b0 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 >> v1:172.21.15.113:6789/0 conn(0x7fe0440b56b0 legacy=0x7fe0440cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe034003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe034004800 0x7fe034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0340051c0 0x7fe03400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440b4510 con 0x7fe0340039b0 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe03400c090 con 0x7fe0340051c0 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe03400c290 con 0x7fe034004800 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe048f53640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe034003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.083+0000 7fe049754640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0340051c0 0x7fe03400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe043fff640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe034004800 0x7fe034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe048f53640 1 -- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe034004800 msgr2=0x7fe034004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe048f53640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe034004800 0x7fe034004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe048f53640 1 -- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0340051c0 msgr2=0x7fe03400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe048f53640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0340051c0 0x7fe03400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe048f53640 1 -- 172.21.15.113:0/1161397564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe034003520 con 0x7fe0340039b0 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe049754640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0340051c0 0x7fe03400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe049754640 1 -- 172.21.15.113:0/1161397564 reap_dead start 2024-04-25T00:00:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe048f53640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe034003dd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe038009650 tx=0x7fe034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc87a61669365494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038038070 con 0x7fe0340039b0 2024-04-25T00:00:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe038033400 con 0x7fe0340039b0 2024-04-25T00:00:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.084+0000 7fe0437fe640 1 -- 172.21.15.113:0/1161397564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038009690 con 0x7fe0340039b0 2024-04-25T00:00:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.085+0000 7fe04b1b7640 1 -- 172.21.15.113:0/1161397564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 msgr2=0x7fe034003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.085+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe034003dd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe038009650 tx=0x7fe034005830 comp rx=0 tx=0).stop 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.085+0000 7fe04b1b7640 1 -- 172.21.15.113:0/1161397564 shutdown_connections 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.085+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/1161397564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe034003dd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.085+0000 7fe04b1b7640 1 -- 172.21.15.113:0/1161397564 >> 172.21.15.113:0/1161397564 conn(0x7fe0440b1a50 msgr2=0x7fe0440b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.085+0000 7fe04b1b7640 1 -- 172.21.15.113:0/1161397564 shutdown_connections 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.085+0000 7fe04b1b7640 1 -- 172.21.15.113:0/1161397564 wait complete. 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.086+0000 7fe04b1b7640 1 Processor -- start 2024-04-25T00:00:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.086+0000 7fe04b1b7640 1 -- start start 2024-04-25T00:00:36.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe04b1b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe0441f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe04b1b7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0440ab140 0x7fe0440ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe04b1b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe044142830 0x7fe044142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe04b1b7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440b4da0 con 0x7fe0340039b0 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe04b1b7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440b37d0 con 0x7fe0440ab140 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe04b1b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe0440b4f20 con 0x7fe044142830 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe0441f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe0441f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58742/0 (socket says 172.21.15.113:58742) 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 -- 172.21.15.113:0/834407760 learned_addr learned my addr 172.21.15.113:0/834407760 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe043fff640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0440ab140 0x7fe0440ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe049754640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe044142830 0x7fe044142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 -- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe044142830 msgr2=0x7fe044142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe044142830 0x7fe044142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 -- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0440ab140 msgr2=0x7fe0440ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0440ab140 0x7fe0440ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe048f53640 1 -- 172.21.15.113:0/834407760 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe044141240 con 0x7fe0340039b0 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.087+0000 7fe043fff640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0440ab140 0x7fe0440ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.088+0000 7fe048f53640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe0441f2f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe038009bb0 tx=0x7fe038004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.088+0000 7fe0417fa640 1 -- 172.21.15.113:0/834407760 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038038050 con 0x7fe0340039b0 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.088+0000 7fe0417fa640 1 -- 172.21.15.113:0/834407760 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe038044070 con 0x7fe0340039b0 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.088+0000 7fe0417fa640 1 -- 172.21.15.113:0/834407760 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038003df0 con 0x7fe0340039b0 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.088+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0440abbe0 con 0x7fe0340039b0 2024-04-25T00:00:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.088+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0440ac1a0 con 0x7fe0340039b0 2024-04-25T00:00:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.089+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0441431f0 con 0x7fe0340039b0 2024-04-25T00:00:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.089+0000 7fe0417fa640 1 -- 172.21.15.113:0/834407760 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe03803f450 con 0x7fe0340039b0 2024-04-25T00:00:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.090+0000 7fe0417fa640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe02806be60 0x7fe02806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.090+0000 7fe0417fa640 1 -- 172.21.15.113:0/834407760 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7fe0380ad250 con 0x7fe0340039b0 2024-04-25T00:00:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.090+0000 7fe043fff640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe02806be60 0x7fe02806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.090+0000 7fe043fff640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe02806be60 0x7fe02806e320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe044143870 tx=0x7fe030009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.094+0000 7fe0417fa640 1 -- 172.21.15.113:0/834407760 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe03804d020 con 0x7fe0340039b0 2024-04-25T00:00:36.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.268+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0440691a0 con 0x7fe0340039b0 2024-04-25T00:00:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.269+0000 7fe0417fa640 1 -- 172.21.15.113:0/834407760 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16782 (secure 0 0 0) 0x7fe044141240 con 0x7fe0340039b0 2024-04-25T00:00:36.270 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:09.800045+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:59:08.215866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:59:09.335267+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":33,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe02806be60 msgr2=0x7fe02806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe02806be60 0x7fe02806e320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe044143870 tx=0x7fe030009040 comp rx=0 tx=0).stop 2024-04-25T00:00:36.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 msgr2=0x7fe0441f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe0441f2f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe038009bb0 tx=0x7fe038004730 comp rx=0 tx=0).stop 2024-04-25T00:00:36.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 shutdown_connections 2024-04-25T00:00:36.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe02806be60 0x7fe02806e320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0440ab140 0x7fe0440ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe044142830 0x7fe044142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 --2- 172.21.15.113:0/834407760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0340039b0 0x7fe0441f2f20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 >> 172.21.15.113:0/834407760 conn(0x7fe0440b1a50 msgr2=0x7fe044071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:36.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 shutdown_connections 2024-04-25T00:00:36.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.271+0000 7fe04b1b7640 1 -- 172.21.15.113:0/834407760 wait complete. 2024-04-25T00:00:36.283 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:59:08.215866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:59:09.335267+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 33, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:00:36.283 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:00:36.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.382+0000 7f872e053640 1 Processor -- start 2024-04-25T00:00:36.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.382+0000 7f872e053640 1 -- start start 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.382+0000 7f872e053640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f872815e510 0x7f87280d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.382+0000 7f872e053640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87280d28c0 0x7f87280b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.382+0000 7f872e053640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87280d2e00 con 0x7f87280b61e0 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.382+0000 7f872e053640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f87280b6610 con 0x7f87280d28c0 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.382+0000 7f872e053640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f87280d30e0 con 0x7f872815e510 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f8726ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87280d28c0 0x7f87280b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f8726ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87280d28c0 0x7f87280b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58748/0 (socket says 172.21.15.113:58748) 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f8726ffd640 1 -- 172.21.15.113:0/4231298847 learned_addr learned my addr 172.21.15.113:0/4231298847 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:36.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f87277fe640 1 --2- 172.21.15.113:0/4231298847 >> v2:172.21.15.178:3300/0 conn(0x7f872815e510 0x7f87280d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f8726ffd640 1 -- 172.21.15.113:0/4231298847 >> v2:172.21.15.178:3300/0 conn(0x7f872815e510 msgr2=0x7f87280d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f8726ffd640 1 --2- 172.21.15.113:0/4231298847 >> v2:172.21.15.178:3300/0 conn(0x7f872815e510 0x7f87280d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f8726ffd640 1 -- 172.21.15.113:0/4231298847 >> v1:172.21.15.178:6789/0 conn(0x7f87280b61e0 legacy=0x7f8728071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f8726ffd640 1 -- 172.21.15.113:0/4231298847 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87280d33c0 con 0x7f87280d28c0 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.383+0000 7f87277fe640 1 --2- 172.21.15.113:0/4231298847 >> v2:172.21.15.178:3300/0 conn(0x7f872815e510 0x7f87280d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f8726ffd640 1 --2- 172.21.15.113:0/4231298847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280b5ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f87280d52a0 tx=0x7f871c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d87365699eb349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f87267fc640 1 -- 172.21.15.113:0/4231298847 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c037070 con 0x7f87280d28c0 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f87267fc640 1 -- 172.21.15.113:0/4231298847 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c032440 con 0x7f87280d28c0 2024-04-25T00:00:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f87267fc640 1 -- 172.21.15.113:0/4231298847 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c032810 con 0x7f87280d28c0 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 -- 172.21.15.113:0/4231298847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 msgr2=0x7f87280b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 --2- 172.21.15.113:0/4231298847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280b5ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f87280d52a0 tx=0x7f871c02a1e0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 -- 172.21.15.113:0/4231298847 shutdown_connections 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 --2- 172.21.15.113:0/4231298847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280b5ca0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 --2- 172.21.15.113:0/4231298847 >> v2:172.21.15.178:3300/0 conn(0x7f872815e510 0x7f87280d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 -- 172.21.15.113:0/4231298847 >> 172.21.15.113:0/4231298847 conn(0x7f872806b720 msgr2=0x7f87280d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 -- 172.21.15.113:0/4231298847 shutdown_connections 2024-04-25T00:00:36.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.384+0000 7f872e053640 1 -- 172.21.15.113:0/4231298847 wait complete. 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.385+0000 7f872e053640 1 Processor -- start 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f872e053640 1 -- start start 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f872e053640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87280b61e0 0x7f8728152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f872e053640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f872e053640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f872815e510 0x7f87280c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f872e053640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87280d3ea0 con 0x7f87280d28c0 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f872e053640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87280d3980 con 0x7f872815e510 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f872e053640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87280b9dd0 con 0x7f87280b61e0 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f8726ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f8726ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58754/0 (socket says 172.21.15.113:58754) 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.386+0000 7f8726ffd640 1 -- 172.21.15.113:0/1124516367 learned_addr learned my addr 172.21.15.113:0/1124516367 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f87277fe640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87280b61e0 0x7f8728152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8727fff640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f872815e510 0x7f87280c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8726ffd640 1 -- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87280b61e0 msgr2=0x7f8728152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8726ffd640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87280b61e0 0x7f8728152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8726ffd640 1 -- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f872815e510 msgr2=0x7f87280c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8726ffd640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f872815e510 0x7f87280c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8726ffd640 1 -- 172.21.15.113:0/1124516367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87280cb390 con 0x7f87280d28c0 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f87277fe640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87280b61e0 0x7f8728152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8727fff640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f872815e510 0x7f87280c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f8726ffd640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280c7d50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f871c02a220 tx=0x7f871c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f870ffff640 1 -- 172.21.15.113:0/1124516367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c041020 con 0x7f87280d28c0 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f870ffff640 1 -- 172.21.15.113:0/1124516367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c0306c0 con 0x7f87280d28c0 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87280cb670 con 0x7f87280d28c0 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f870ffff640 1 -- 172.21.15.113:0/1124516367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c037040 con 0x7f87280d28c0 2024-04-25T00:00:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.387+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87280cbc30 con 0x7f87280d28c0 2024-04-25T00:00:36.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.388+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87280691a0 con 0x7f87280d28c0 2024-04-25T00:00:36.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.389+0000 7f870ffff640 1 -- 172.21.15.113:0/1124516367 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f871c044020 con 0x7f87280d28c0 2024-04-25T00:00:36.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.389+0000 7f870ffff640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f871406be60 0x7f871406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.389+0000 7f870ffff640 1 -- 172.21.15.113:0/1124516367 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f871c0ab6a0 con 0x7f87280d28c0 2024-04-25T00:00:36.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.389+0000 7f87277fe640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f871406be60 0x7f871406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.390+0000 7f87277fe640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f871406be60 0x7f871406e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87281f7560 tx=0x7f8718008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.393+0000 7f870ffff640 1 -- 172.21.15.113:0/1124516367 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f871c04b030 con 0x7f87280d28c0 2024-04-25T00:00:36.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.566+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87280b5a30 con 0x7f87280d28c0 2024-04-25T00:00:36.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.566+0000 7f870ffff640 1 -- 172.21.15.113:0/1124516367 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f871c07a040 con 0x7f87280d28c0 2024-04-25T00:00:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:00:36.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f871406be60 msgr2=0x7f871406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f871406be60 0x7f871406e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87281f7560 tx=0x7f8718008040 comp rx=0 tx=0).stop 2024-04-25T00:00:36.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 msgr2=0x7f87280c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280c7d50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f871c02a220 tx=0x7f871c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 shutdown_connections 2024-04-25T00:00:36.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f871406be60 0x7f871406e320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f872815e510 0x7f87280c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87280d28c0 0x7f87280c7d50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 --2- 172.21.15.113:0/1124516367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87280b61e0 0x7f8728152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 >> 172.21.15.113:0/1124516367 conn(0x7f872806b720 msgr2=0x7f872815d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:36.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 shutdown_connections 2024-04-25T00:00:36.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.568+0000 7f872e053640 1 -- 172.21.15.113:0/1124516367 wait complete. 2024-04-25T00:00:36.580 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:00:36.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.679+0000 7fc5886d4640 1 Processor -- start 2024-04-25T00:00:36.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.679+0000 7fc5886d4640 1 -- start start 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc5886d4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc58015e2d0 0x7fc58015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc5886d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b4a40 0x7fc5800ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc5886d4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc58015f4f0 con 0x7fc5800b3080 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc5886d4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc5800cde00 con 0x7fc58015e2d0 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc5886d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc58015f7d0 con 0x7fc5800b4a40 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc586c71640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b4a40 0x7fc5800ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc586470640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc58015e2d0 0x7fc58015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc585c6f640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc5800b3080 0x7fc5800b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:59996/0 (socket says 172.21.15.113:59996) 2024-04-25T00:00:36.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc585c6f640 1 -- 172.21.15.113:0/3375376961 learned_addr learned my addr 172.21.15.113:0/3375376961 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:36.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc586470640 1 -- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b4a40 msgr2=0x7fc5800ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc586470640 1 --2- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b4a40 0x7fc5800ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc586470640 1 -- 172.21.15.113:0/3375376961 >> v1:172.21.15.178:6789/0 conn(0x7fc5800b3080 legacy=0x7fc5800b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc586470640 1 -- 172.21.15.113:0/3375376961 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58015fab0 con 0x7fc58015e2d0 2024-04-25T00:00:36.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc58546e640 1 -- 172.21.15.113:0/3375376961 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2315848584 0 0) 0x7fc58015f4f0 con 0x7fc5800b3080 2024-04-25T00:00:36.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.680+0000 7fc586c71640 1 --2- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b4a40 0x7fc5800ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.681+0000 7fc586470640 1 --2- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 0x7fc58015ebf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0097a0 tx=0x7fc57c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b397af7f08a97d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.681+0000 7fc58546e640 1 -- 172.21.15.113:0/3375376961 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c037070 con 0x7fc58015e2d0 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.681+0000 7fc58546e640 1 -- 172.21.15.113:0/3375376961 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c032440 con 0x7fc58015e2d0 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.681+0000 7fc58546e640 1 -- 172.21.15.113:0/3375376961 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c032810 con 0x7fc58015e2d0 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 -- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 msgr2=0x7fc58015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 --2- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 0x7fc58015ebf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0097a0 tx=0x7fc57c02a180 comp rx=0 tx=0).stop 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 -- 172.21.15.113:0/3375376961 shutdown_connections 2024-04-25T00:00:36.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 --2- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b4a40 0x7fc5800ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 --2- 172.21.15.113:0/3375376961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 0x7fc58015ebf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 -- 172.21.15.113:0/3375376961 >> 172.21.15.113:0/3375376961 conn(0x7fc58006b0f0 msgr2=0x7fc58006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:36.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 -- 172.21.15.113:0/3375376961 shutdown_connections 2024-04-25T00:00:36.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.682+0000 7fc5886d4640 1 -- 172.21.15.113:0/3375376961 wait complete. 2024-04-25T00:00:36.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.683+0000 7fc5886d4640 1 Processor -- start 2024-04-25T00:00:36.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.683+0000 7fc5886d4640 1 -- start start 2024-04-25T00:00:36.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b3080 0x7fc58014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800b4a40 0x7fc5800b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 0x7fc5800b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc58015f2b0 con 0x7fc5800b4a40 2024-04-25T00:00:36.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc58015f130 con 0x7fc58015e2d0 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc58015e6b0 con 0x7fc5800b3080 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800b4a40 0x7fc5800b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc586470640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b3080 0x7fc58014ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800b4a40 0x7fc5800b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58764/0 (socket says 172.21.15.113:58764) 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 -- 172.21.15.113:0/1840092523 learned_addr learned my addr 172.21.15.113:0/1840092523 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc586c71640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 0x7fc5800b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 -- 172.21.15.113:0/1840092523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b3080 msgr2=0x7fc58014ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b3080 0x7fc58014ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 -- 172.21.15.113:0/1840092523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 msgr2=0x7fc5800b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 0x7fc5800b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 -- 172.21.15.113:0/1840092523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc580151dc0 con 0x7fc5800b4a40 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc585c6f640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800b4a40 0x7fc5800b1600 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c4d0 tx=0x7fc57000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc576ffd640 1 -- 172.21.15.113:0/1840092523 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570018070 con 0x7fc5800b4a40 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc576ffd640 1 -- 172.21.15.113:0/1840092523 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc570007500 con 0x7fc5800b4a40 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5801520a0 con 0x7fc5800b4a40 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc576ffd640 1 -- 172.21.15.113:0/1840092523 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570004a20 con 0x7fc5800b4a40 2024-04-25T00:00:36.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.684+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc580152660 con 0x7fc5800b4a40 2024-04-25T00:00:36.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.685+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5800691a0 con 0x7fc5800b4a40 2024-04-25T00:00:36.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.686+0000 7fc576ffd640 1 -- 172.21.15.113:0/1840092523 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc57001b070 con 0x7fc5800b4a40 2024-04-25T00:00:36.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.686+0000 7fc576ffd640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc56406be60 0x7fc56406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.686+0000 7fc576ffd640 1 -- 172.21.15.113:0/1840092523 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7fc57008a090 con 0x7fc5800b4a40 2024-04-25T00:00:36.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.686+0000 7fc586470640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc56406be60 0x7fc56406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.687+0000 7fc586470640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc56406be60 0x7fc56406e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc57c034000 tx=0x7fc57c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.690+0000 7fc576ffd640 1 -- 172.21.15.113:0/1840092523 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc570058460 con 0x7fc5800b4a40 2024-04-25T00:00:36.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.863+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc580072050 con 0x7fc5800b4a40 2024-04-25T00:00:36.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.863+0000 7fc576ffd640 1 -- 172.21.15.113:0/1840092523 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7fc57005c760 con 0x7fc5800b4a40 2024-04-25T00:00:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:00:36.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc56406be60 msgr2=0x7fc56406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc56406be60 0x7fc56406e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc57c034000 tx=0x7fc57c02e000 comp rx=0 tx=0).stop 2024-04-25T00:00:36.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800b4a40 msgr2=0x7fc5800b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800b4a40 0x7fc5800b1600 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c4d0 tx=0x7fc57000c990 comp rx=0 tx=0).stop 2024-04-25T00:00:36.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 shutdown_connections 2024-04-25T00:00:36.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc56406be60 0x7fc56406e320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc58015e2d0 0x7fc5800b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800b4a40 0x7fc5800b1600 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 --2- 172.21.15.113:0/1840092523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5800b3080 0x7fc58014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.865+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 >> 172.21.15.113:0/1840092523 conn(0x7fc58006b0f0 msgr2=0x7fc5800b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:36.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.866+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 shutdown_connections 2024-04-25T00:00:36.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.866+0000 7fc5886d4640 1 -- 172.21.15.113:0/1840092523 wait complete. 2024-04-25T00:00:36.878 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:00:36.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.976+0000 7f80ddfab640 1 Processor -- start 2024-04-25T00:00:36.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.976+0000 7f80ddfab640 1 -- start start 2024-04-25T00:00:36.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80ddfab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80d8165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80ddfab640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d815f4f0 con 0x7f80d80b3080 2024-04-25T00:00:36.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80ddfab640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d815f7d0 con 0x7f80d815e2d0 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80ddfab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f80d80cde00 con 0x7f80d80b4a40 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80cffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f80d80b3080 0x7f80d80b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58936/0 (socket says 172.21.15.113:58936) 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80cffff640 1 -- 172.21.15.113:0/4013037885 learned_addr learned my addr 172.21.15.113:0/4013037885 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d7fff640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80d8165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316477363 0 0) 0x7f80d815f4f0 con 0x7f80d80b3080 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80c4003520 con 0x7f80d80b3080 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341451318 0 0) 0x7f80d815f7d0 con 0x7f80d815e2d0 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80d815f4f0 con 0x7f80d815e2d0 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1487616151 0 0) 0x7f80c4003520 con 0x7f80d80b3080 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 msgr2=0x7f80d8165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80d8165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 >> v1:172.21.15.178:6789/0 conn(0x7f80d815e2d0 legacy=0x7f80d815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d815fab0 con 0x7f80d80b3080 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f80c0001670 con 0x7f80d80b3080 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d7fff640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80d8165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 >> v1:172.21.15.113:6789/0 conn(0x7f80d80b3080 legacy=0x7f80d80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80c4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c40040b0 0x7f80c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c4004d40 0x7f80c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f80d80cde00 con 0x7f80c40040b0 2024-04-25T00:00:36.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80c400ba70 con 0x7f80c4004d40 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f80c400bc70 con 0x7f80d80b4a40 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.977+0000 7f80cffff640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c40040b0 0x7f80c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80d77fe640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80c4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80d7fff640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c4004d40 0x7f80c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80cffff640 1 -- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 msgr2=0x7f80c4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80cffff640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80c4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80cffff640 1 -- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c4004d40 msgr2=0x7f80c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80cffff640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c4004d40 0x7f80c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80cffff640 1 -- 172.21.15.113:0/4013037885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c4003520 con 0x7f80c40040b0 2024-04-25T00:00:36.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80d77fe640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d80b4a40 0x7f80c4003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80cffff640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c40040b0 0x7f80c4004800 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c190 tx=0x7f80c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c87a5f2a93b136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0007cc0 con 0x7f80c40040b0 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.978+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c0005af0 con 0x7f80c40040b0 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80d6ffd640 1 -- 172.21.15.113:0/4013037885 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0007070 con 0x7f80c40040b0 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80ddfab640 1 -- 172.21.15.113:0/4013037885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c40040b0 msgr2=0x7f80c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80ddfab640 1 --2- 172.21.15.113:0/4013037885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c40040b0 0x7f80c4004800 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c190 tx=0x7f80c4005830 comp rx=0 tx=0).stop 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80d7fff640 1 -- 172.21.15.113:0/4013037885 reap_dead start 2024-04-25T00:00:36.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80ddfab640 1 -- 172.21.15.113:0/4013037885 shutdown_connections 2024-04-25T00:00:36.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80ddfab640 1 -- 172.21.15.113:0/4013037885 >> 172.21.15.113:0/4013037885 conn(0x7f80d806b0f0 msgr2=0x7f80d806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:36.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80ddfab640 1 -- 172.21.15.113:0/4013037885 shutdown_connections 2024-04-25T00:00:36.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.979+0000 7f80ddfab640 1 -- 172.21.15.113:0/4013037885 wait complete. 2024-04-25T00:00:36.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.980+0000 7f80ddfab640 1 Processor -- start 2024-04-25T00:00:36.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.980+0000 7f80ddfab640 1 -- start start 2024-04-25T00:00:36.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80ddfab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c40040b0 0x7f80d8153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80ddfab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80d8144d10 0x7f80d8151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80ddfab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d8152430 0x7f80d815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80ddfab640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f80d80cf420 con 0x7f80d8144d10 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80ddfab640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80d80cf2a0 con 0x7f80c40040b0 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80ddfab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f80d80cf5a0 con 0x7f80d8152430 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80d8144d10 0x7f80d8151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80d8144d10 0x7f80d8151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58778/0 (socket says 172.21.15.113:58778) 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 -- 172.21.15.113:0/1366932037 learned_addr learned my addr 172.21.15.113:0/1366932037 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 -- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d8152430 msgr2=0x7f80d815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80d7fff640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d8152430 0x7f80d815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80d77fe640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c40040b0 0x7f80d8153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d8152430 0x7f80d815aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 -- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c40040b0 msgr2=0x7f80d8153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:36.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c40040b0 0x7f80d8153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:36.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80cffff640 1 -- 172.21.15.113:0/1366932037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d8145190 con 0x7f80d8144d10 2024-04-25T00:00:36.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.981+0000 7f80d77fe640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c40040b0 0x7f80d8153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:36.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.982+0000 7f80cffff640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80d8144d10 0x7f80d8151ef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c1f0 tx=0x7f80d815f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.982+0000 7f80d4ff9640 1 -- 172.21.15.113:0/1366932037 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0041020 con 0x7f80d8144d10 2024-04-25T00:00:36.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.982+0000 7f80d4ff9640 1 -- 172.21.15.113:0/1366932037 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c0001470 con 0x7f80d8144d10 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.982+0000 7f80d4ff9640 1 -- 172.21.15.113:0/1366932037 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0007270 con 0x7f80d8144d10 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.982+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d814fd00 con 0x7f80d8144d10 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.982+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d81502c0 con 0x7f80d8144d10 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.983+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d80691a0 con 0x7f80d8144d10 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.983+0000 7f80d4ff9640 1 -- 172.21.15.113:0/1366932037 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80c0044020 con 0x7f80d8144d10 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.983+0000 7f80d4ff9640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80b806be60 0x7f80b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.983+0000 7f80d4ff9640 1 -- 172.21.15.113:0/1366932037 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f80c00ab130 con 0x7f80d8144d10 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.984+0000 7f80d77fe640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80b806be60 0x7f80b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.984+0000 7f80d77fe640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80b806be60 0x7f80b806e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80c8001770 tx=0x7f80c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:36.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:36.988+0000 7f80d4ff9640 1 -- 172.21.15.113:0/1366932037 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80c0079ad0 con 0x7f80d8144d10 2024-04-25T00:00:37.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.166+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f80d8152810 con 0x7f80d8144d10 2024-04-25T00:00:37.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.167+0000 7f80d4ff9640 1 -- 172.21.15.113:0/1366932037 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f80c00793d0 con 0x7f80d8144d10 2024-04-25T00:00:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:00:37.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80b806be60 msgr2=0x7f80b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80b806be60 0x7f80b806e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80c8001770 tx=0x7f80c8009040 comp rx=0 tx=0).stop 2024-04-25T00:00:37.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80d8144d10 msgr2=0x7f80d8151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80d8144d10 0x7f80d8151ef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c1f0 tx=0x7f80d815f9a0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 shutdown_connections 2024-04-25T00:00:37.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80b806be60 0x7f80b806e320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80d8152430 0x7f80d815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80d8144d10 0x7f80d8151ef0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 --2- 172.21.15.113:0/1366932037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c40040b0 0x7f80d8153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 >> 172.21.15.113:0/1366932037 conn(0x7f80d806b0f0 msgr2=0x7f80d80b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 shutdown_connections 2024-04-25T00:00:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.169+0000 7f80ddfab640 1 -- 172.21.15.113:0/1366932037 wait complete. 2024-04-25T00:00:37.181 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:37.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.275+0000 7ff842963640 1 Processor -- start 2024-04-25T00:00:37.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.275+0000 7ff842963640 1 -- start start 2024-04-25T00:00:37.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.276+0000 7ff842963640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff83c15e320 0x7ff83c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.276+0000 7ff842963640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 0x7ff83c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.276+0000 7ff842963640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83c0d3de0 con 0x7ff83c0d2290 2024-04-25T00:00:37.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.276+0000 7ff842963640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff83c0d14d0 con 0x7ff83c15e320 2024-04-25T00:00:37.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.276+0000 7ff842963640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c0d3410 con 0x7ff83c0d0b00 2024-04-25T00:00:37.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.276+0000 7ff83b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff83c0d2290 0x7ff83c0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58940/0 (socket says 172.21.15.113:58940) 2024-04-25T00:00:37.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.276+0000 7ff83b7fe640 1 -- 172.21.15.113:0/1027544593 learned_addr learned my addr 172.21.15.113:0/1027544593 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058429640 0 0) 0x7ff83c0d3de0 con 0x7ff83c0d2290 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff824003520 con 0x7ff83c0d2290 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83bfff640 1 --2- 172.21.15.113:0/1027544593 >> v2:172.21.15.178:3300/0 conn(0x7ff83c15e320 0x7ff83c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff840f00640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 0x7ff83c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3722271213 0 0) 0x7ff824003520 con 0x7ff83c0d2290 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 msgr2=0x7ff83c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 0x7ff83c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 >> v2:172.21.15.178:3300/0 conn(0x7ff83c15e320 msgr2=0x7ff83c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 --2- 172.21.15.113:0/1027544593 >> v2:172.21.15.178:3300/0 conn(0x7ff83c15e320 0x7ff83c0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c0d2800 con 0x7ff83c0d2290 2024-04-25T00:00:37.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff830001670 con 0x7ff83c0d2290 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 >> v1:172.21.15.113:6789/0 conn(0x7ff83c0d2290 legacy=0x7ff83c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8240039b0 0x7ff824003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004800 0x7ff824004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 0x7ff8240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0d14d0 con 0x7ff824004800 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0d3410 con 0x7ff8240039b0 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff82400b9c0 con 0x7ff83c0d0b00 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83b7fe640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004800 0x7ff824004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83bfff640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8240039b0 0x7ff824003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff840f00640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 0x7ff8240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83b7fe640 1 -- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 msgr2=0x7ff8240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83b7fe640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0d0b00 0x7ff8240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83b7fe640 1 -- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8240039b0 msgr2=0x7ff824003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83b7fe640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8240039b0 0x7ff824003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.277+0000 7ff83b7fe640 1 -- 172.21.15.113:0/1027544593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff824003520 con 0x7ff824004800 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.278+0000 7ff83b7fe640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004800 0x7ff824004c80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff830001050 tx=0x7ff824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f751740c2bfc02e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.278+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83002e070 con 0x7ff824004800 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.278+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff830005af0 con 0x7ff824004800 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.278+0000 7ff83affd640 1 -- 172.21.15.113:0/1027544593 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830007d70 con 0x7ff824004800 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.279+0000 7ff842963640 1 -- 172.21.15.113:0/1027544593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004800 msgr2=0x7ff824004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.279+0000 7ff842963640 1 --2- 172.21.15.113:0/1027544593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004800 0x7ff824004c80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff830001050 tx=0x7ff824005830 comp rx=0 tx=0).stop 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.279+0000 7ff840f00640 1 -- 172.21.15.113:0/1027544593 reap_dead start 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.279+0000 7ff842963640 1 -- 172.21.15.113:0/1027544593 shutdown_connections 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.279+0000 7ff842963640 1 -- 172.21.15.113:0/1027544593 >> 172.21.15.113:0/1027544593 conn(0x7ff83c0b5c20 msgr2=0x7ff83c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.279+0000 7ff842963640 1 -- 172.21.15.113:0/1027544593 shutdown_connections 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.279+0000 7ff842963640 1 -- 172.21.15.113:0/1027544593 wait complete. 2024-04-25T00:00:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.280+0000 7ff842963640 1 Processor -- start 2024-04-25T00:00:37.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.280+0000 7ff842963640 1 -- start start 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff842963640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff824004800 0x7ff83c161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff842963640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0b7c40 0x7ff83c0b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff842963640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff83c158580 0x7ff83c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff842963640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0d30a0 con 0x7ff83c158580 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff842963640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0d2dc0 con 0x7ff824004800 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff842963640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c15e700 con 0x7ff83c0b7c40 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff83c158580 0x7ff83c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff83c158580 0x7ff83c158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58800/0 (socket says 172.21.15.113:58800) 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 -- 172.21.15.113:0/4152843334 learned_addr learned my addr 172.21.15.113:0/4152843334 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff83b7fe640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0b7c40 0x7ff83c0b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff83bfff640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff824004800 0x7ff83c161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 -- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0b7c40 msgr2=0x7ff83c0b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0b7c40 0x7ff83c0b80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 -- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff824004800 msgr2=0x7ff83c161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff824004800 0x7ff83c161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 -- 172.21.15.113:0/4152843334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c155a20 con 0x7ff83c158580 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff83bfff640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff824004800 0x7ff83c161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff840f00640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff83c158580 0x7ff83c158a00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0b97f0 tx=0x7ff83400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff838ff9640 1 -- 172.21.15.113:0/4152843334 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83400fe10 con 0x7ff83c158580 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.281+0000 7ff838ff9640 1 -- 172.21.15.113:0/4152843334 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff834015430 con 0x7ff83c158580 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.282+0000 7ff838ff9640 1 -- 172.21.15.113:0/4152843334 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834015820 con 0x7ff83c158580 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.282+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c155ce0 con 0x7ff83c158580 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.282+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c1562a0 con 0x7ff83c158580 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.282+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c0682b0 con 0x7ff83c158580 2024-04-25T00:00:37.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.283+0000 7ff838ff9640 1 -- 172.21.15.113:0/4152843334 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8340159c0 con 0x7ff83c158580 2024-04-25T00:00:37.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.283+0000 7ff838ff9640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff82806be60 0x7ff82806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.283+0000 7ff838ff9640 1 -- 172.21.15.113:0/4152843334 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7ff83408bbd0 con 0x7ff83c158580 2024-04-25T00:00:37.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.283+0000 7ff83bfff640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff82806be60 0x7ff82806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.284+0000 7ff83bfff640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff82806be60 0x7ff82806e320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009bc0 tx=0x7ff82c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.287+0000 7ff838ff9640 1 -- 172.21.15.113:0/4152843334 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff83c0682b0 con 0x7ff83c158580 2024-04-25T00:00:37.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.497+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff83c0691a0 con 0x7ff83c158580 2024-04-25T00:00:37.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.497+0000 7ff838ff9640 1 -- 172.21.15.113:0/4152843334 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5193 (secure 0 0 0) 0x7ff834019030 con 0x7ff83c158580 2024-04-25T00:00:37.499 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2613614055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2613614055},{"type":"v1","addr":"172.21.15.178:6834","nonce":2613614055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2494825991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2494825991},{"type":"v1","addr":"172.21.15.178:6835","nonce":2494825991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2843749120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2843749120},{"type":"v1","addr":"172.21.15.113:6836","nonce":2843749120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-24T23:59:10.365260+0000","modified":"2024-04-24T23:59:17.031612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3337939581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3337939581},{"type":"v1","addr":"172.21.15.113:6837","nonce":3337939581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-04-25T00:00:37.499 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 15 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff82806be60 msgr2=0x7ff82806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff82806be60 0x7ff82806e320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009bc0 tx=0x7ff82c008040 comp rx=0 tx=0).stop 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff83c158580 msgr2=0x7ff83c158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff83c158580 0x7ff83c158a00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0b97f0 tx=0x7ff83400f580 comp rx=0 tx=0).stop 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 shutdown_connections 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff82806be60 0x7ff82806e320 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff83c158580 0x7ff83c158a00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff83c0b7c40 0x7ff83c0b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 --2- 172.21.15.113:0/4152843334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff824004800 0x7ff83c161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.499+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 >> 172.21.15.113:0/4152843334 conn(0x7ff83c0b5c20 msgr2=0x7ff83c0cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.500+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 shutdown_connections 2024-04-25T00:00:37.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.500+0000 7ff842963640 1 -- 172.21.15.113:0/4152843334 wait complete. 2024-04-25T00:00:37.511 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:37.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.606+0000 7f324875a640 1 Processor -- start 2024-04-25T00:00:37.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.606+0000 7f324875a640 1 -- start start 2024-04-25T00:00:37.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f324875a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 0x7f32400b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f324875a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32400cfad0 con 0x7f32401453c0 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f324875a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32400cf230 con 0x7f3240144d60 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f324875a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3240145c40 con 0x7f324006e6e0 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f3246cf7640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f32401453c0 0x7f3240071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52992/0 (socket says 172.21.15.113:52992) 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f3246cf7640 1 -- 172.21.15.113:0/3314974949 learned_addr learned my addr 172.21.15.113:0/3314974949 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615728420 0 0) 0x7f32400cfad0 con 0x7f32401453c0 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f322c003520 con 0x7f32401453c0 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f3245cf5640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 0x7f32400b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 497064884 0 0) 0x7f322c003520 con 0x7f32401453c0 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 msgr2=0x7f32400b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 0x7f32400b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 >> v1:172.21.15.178:6789/0 conn(0x7f3240144d60 legacy=0x7f324006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32400cf510 con 0x7f32401453c0 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f32380019f0 con 0x7f32401453c0 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 >> v1:172.21.15.113:6789/0 conn(0x7f32401453c0 legacy=0x7f3240071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f322c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c004800 0x7f322c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 0x7f322c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3240145c40 con 0x7f322c0039e0 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f322c005520 con 0x7f322c004800 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f322c00bf00 con 0x7f324006e6e0 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.607+0000 7f32464f6640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f322c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f3245cf5640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c004800 0x7f322c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f3246cf7640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 0x7f322c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f32464f6640 1 -- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 msgr2=0x7f322c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f32464f6640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 0x7f322c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f32464f6640 1 -- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c004800 msgr2=0x7f322c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f32464f6640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c004800 0x7f322c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f32464f6640 1 -- 172.21.15.113:0/3314974949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c003520 con 0x7f322c0039e0 2024-04-25T00:00:37.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f3246cf7640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f324006e6e0 0x7f322c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:37.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.608+0000 7f32464f6640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f322c003e00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f323c001750 tx=0x7f322c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6fb2dcc0d2e4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c001370 con 0x7f322c0039e0 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c0052d0 con 0x7f322c0039e0 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f32454f4640 1 -- 172.21.15.113:0/3314974949 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c02abe0 con 0x7f322c0039e0 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f324875a640 1 -- 172.21.15.113:0/3314974949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 msgr2=0x7f322c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f324875a640 1 --2- 172.21.15.113:0/3314974949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f322c003e00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f323c001750 tx=0x7f322c005830 comp rx=0 tx=0).stop 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f3246cf7640 1 -- 172.21.15.113:0/3314974949 reap_dead start 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f324875a640 1 -- 172.21.15.113:0/3314974949 shutdown_connections 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f324875a640 1 -- 172.21.15.113:0/3314974949 >> 172.21.15.113:0/3314974949 conn(0x7f32400b1a50 msgr2=0x7f32400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f324875a640 1 -- 172.21.15.113:0/3314974949 shutdown_connections 2024-04-25T00:00:37.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.609+0000 7f324875a640 1 -- 172.21.15.113:0/3314974949 wait complete. 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.610+0000 7f324875a640 1 Processor -- start 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.610+0000 7f324875a640 1 -- start start 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f324875a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f32401365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f324875a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f324013be60 0x7f3240137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f324875a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3240138500 0x7f32401f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f324875a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3240145140 con 0x7f322c0039e0 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f324875a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3240070e10 con 0x7f324013be60 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f324875a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f32400b5390 con 0x7f3240138500 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f32401365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f32401365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52436/0 (socket says 172.21.15.113:52436) 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 -- 172.21.15.113:0/152417318 learned_addr learned my addr 172.21.15.113:0/152417318 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f3245cf5640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f324013be60 0x7f3240137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f3246cf7640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3240138500 0x7f32401f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 -- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3240138500 msgr2=0x7f32401f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3240138500 0x7f32401f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 -- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f324013be60 msgr2=0x7f3240137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f324013be60 0x7f3240137fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 -- 172.21.15.113:0/152417318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3240136f20 con 0x7f322c0039e0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f3245cf5640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f324013be60 0x7f3240137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f3246cf7640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3240138500 0x7f32401f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.611+0000 7f32464f6640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f32401365b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f323c008fd0 tx=0x7f323c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.612+0000 7f3236ffd640 1 -- 172.21.15.113:0/152417318 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c037040 con 0x7f322c0039e0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.612+0000 7f3236ffd640 1 -- 172.21.15.113:0/152417318 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c005480 con 0x7f322c0039e0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.612+0000 7f3236ffd640 1 -- 172.21.15.113:0/152417318 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c007a30 con 0x7f322c0039e0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.612+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3240137200 con 0x7f322c0039e0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.612+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401377c0 con 0x7f322c0039e0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.612+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32400691a0 con 0x7f322c0039e0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.614+0000 7f3236ffd640 1 -- 172.21.15.113:0/152417318 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f323c044020 con 0x7f322c0039e0 2024-04-25T00:00:37.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.614+0000 7f3236ffd640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f322406be30 0x7f322406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.614+0000 7f3236ffd640 1 -- 172.21.15.113:0/152417318 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f323c0aa920 con 0x7f322c0039e0 2024-04-25T00:00:37.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.614+0000 7f3245cf5640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f322406be30 0x7f322406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.615+0000 7f3245cf5640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f322406be30 0x7f322406e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f324013f820 tx=0x7f3230008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.617+0000 7f3236ffd640 1 -- 172.21.15.113:0/152417318 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f323c04b020 con 0x7f322c0039e0 2024-04-25T00:00:37.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.789+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32400b27e0 con 0x7f322c0039e0 2024-04-25T00:00:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.790+0000 7f3236ffd640 1 -- 172.21.15.113:0/152417318 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16782 (secure 0 0 0) 0x7f323c0792b0 con 0x7f322c0039e0 2024-04-25T00:00:37.791 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:09.800045+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:59:08.215866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:59:09.335267+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":33,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f322406be30 msgr2=0x7f322406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f322406be30 0x7f322406e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f324013f820 tx=0x7f3230008040 comp rx=0 tx=0).stop 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 msgr2=0x7f32401365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f32401365b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f323c008fd0 tx=0x7f323c001120 comp rx=0 tx=0).stop 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 shutdown_connections 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f322406be30 0x7f322406e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f324013be60 0x7f3240137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3240138500 0x7f32401f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 --2- 172.21.15.113:0/152417318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0039e0 0x7f32401365b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 >> 172.21.15.113:0/152417318 conn(0x7f32400b1a50 msgr2=0x7f3240143f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 shutdown_connections 2024-04-25T00:00:37.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.792+0000 7f324875a640 1 -- 172.21.15.113:0/152417318 wait complete. 2024-04-25T00:00:37.803 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:00:37.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.900+0000 7f228a038640 1 Processor -- start 2024-04-25T00:00:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.900+0000 7f228a038640 1 -- start start 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f228a038640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f228406e6e0 0x7f22840cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f228a038640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2284144cf0 con 0x7f2284144910 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f228a038640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22840b2f30 con 0x7f22840b3fd0 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f228a038640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f22840b3210 con 0x7f228406e6e0 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f22837fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2284144910 0x7f22840b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53006/0 (socket says 172.21.15.113:53006) 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f22837fe640 1 -- 172.21.15.113:0/481568626 learned_addr learned my addr 172.21.15.113:0/481568626 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f2283fff640 1 --2- 172.21.15.113:0/481568626 >> v2:172.21.15.113:3300/0 conn(0x7f228406e6e0 0x7f22840cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f22827fc640 1 -- 172.21.15.113:0/481568626 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556962075 0 0) 0x7f2284144cf0 con 0x7f2284144910 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f22827fc640 1 -- 172.21.15.113:0/481568626 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f226c003520 con 0x7f2284144910 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f2283fff640 1 -- 172.21.15.113:0/481568626 >> v1:172.21.15.178:6789/0 conn(0x7f22840b3fd0 legacy=0x7f228406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f2283fff640 1 -- 172.21.15.113:0/481568626 >> v1:172.21.15.113:6789/0 conn(0x7f2284144910 legacy=0x7f22840b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.901+0000 7f2283fff640 1 -- 172.21.15.113:0/481568626 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22840b34f0 con 0x7f228406e6e0 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.902+0000 7f2283fff640 1 --2- 172.21.15.113:0/481568626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f22840cd500 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f227c009dc0 tx=0x7f227c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659d2421c8eb46a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.902+0000 7f22827fc640 1 -- 172.21.15.113:0/481568626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c007740 con 0x7f228406e6e0 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.902+0000 7f22827fc640 1 -- 172.21.15.113:0/481568626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c0078e0 con 0x7f228406e6e0 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.902+0000 7f22827fc640 1 -- 172.21.15.113:0/481568626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c007c90 con 0x7f228406e6e0 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.903+0000 7f228a038640 1 -- 172.21.15.113:0/481568626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 msgr2=0x7f22840cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.903+0000 7f228a038640 1 --2- 172.21.15.113:0/481568626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f22840cd500 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f227c009dc0 tx=0x7f227c02a6d0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.903+0000 7f228a038640 1 -- 172.21.15.113:0/481568626 shutdown_connections 2024-04-25T00:00:37.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.903+0000 7f228a038640 1 --2- 172.21.15.113:0/481568626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f22840cd500 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.903+0000 7f228a038640 1 -- 172.21.15.113:0/481568626 >> 172.21.15.113:0/481568626 conn(0x7f22840b1a50 msgr2=0x7f22840b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.903+0000 7f228a038640 1 -- 172.21.15.113:0/481568626 shutdown_connections 2024-04-25T00:00:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.903+0000 7f228a038640 1 -- 172.21.15.113:0/481568626 wait complete. 2024-04-25T00:00:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.904+0000 7f228a038640 1 Processor -- start 2024-04-25T00:00:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.904+0000 7f228a038640 1 -- start start 2024-04-25T00:00:37.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f228a038640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f228413b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f228a038640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22840b3fd0 0x7f228413bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f228a038640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2284144910 0x7f22841546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f228a038640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22840b4da0 con 0x7f228406e6e0 2024-04-25T00:00:37.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f228a038640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22840b4440 con 0x7f22840b3fd0 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f228a038640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f22840b4f20 con 0x7f2284144910 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f228413b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f228413b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52450/0 (socket says 172.21.15.113:52450) 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 -- 172.21.15.113:0/15679984 learned_addr learned my addr 172.21.15.113:0/15679984 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f2282ffd640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22840b3fd0 0x7f228413bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f2283fff640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2284144910 0x7f22841546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 -- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2284144910 msgr2=0x7f22841546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2284144910 0x7f22841546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 -- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22840b3fd0 msgr2=0x7f228413bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22840b3fd0 0x7f228413bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 -- 172.21.15.113:0/15679984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22841564b0 con 0x7f228406e6e0 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f2282ffd640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22840b3fd0 0x7f228413bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:37.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f22837fe640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f228413b710 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f227400c390 tx=0x7f227400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.905+0000 7f226bfff640 1 -- 172.21.15.113:0/15679984 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274017070 con 0x7f228406e6e0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.906+0000 7f226bfff640 1 -- 172.21.15.113:0/15679984 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2274005ce0 con 0x7f228406e6e0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.906+0000 7f226bfff640 1 -- 172.21.15.113:0/15679984 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740072f0 con 0x7f228406e6e0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.906+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2284156790 con 0x7f228406e6e0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.906+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228413ad00 con 0x7f228406e6e0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.906+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22840691a0 con 0x7f228406e6e0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.907+0000 7f226bfff640 1 -- 172.21.15.113:0/15679984 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f227401a070 con 0x7f228406e6e0 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.907+0000 7f226bfff640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.907+0000 7f2282ffd640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.907+0000 7f226bfff640 1 -- 172.21.15.113:0/15679984 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f227408a230 con 0x7f228406e6e0 2024-04-25T00:00:37.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.908+0000 7f2282ffd640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f227006be30 0x7f227006e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22841556c0 tx=0x7f2278009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:37.912+0000 7f226bfff640 1 -- 172.21.15.113:0/15679984 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2274058c50 con 0x7f228406e6e0 2024-04-25T00:00:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.123+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22840731a0 con 0x7f228406e6e0 2024-04-25T00:00:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.123+0000 7f226bfff640 1 -- 172.21.15.113:0/15679984 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5193 (secure 0 0 0) 0x7f2274058550 con 0x7f228406e6e0 2024-04-25T00:00:38.124 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2613614055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2613614055},{"type":"v1","addr":"172.21.15.178:6834","nonce":2613614055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2494825991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2494825991},{"type":"v1","addr":"172.21.15.178:6835","nonce":2494825991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2843749120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2843749120},{"type":"v1","addr":"172.21.15.113:6836","nonce":2843749120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-24T23:59:10.365260+0000","modified":"2024-04-24T23:59:17.031612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3337939581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3337939581},{"type":"v1","addr":"172.21.15.113:6837","nonce":3337939581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-04-25T00:00:38.124 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 15 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f227006be30 msgr2=0x7f227006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f227006be30 0x7f227006e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22841556c0 tx=0x7f2278009040 comp rx=0 tx=0).stop 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 msgr2=0x7f228413b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f228413b710 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f227400c390 tx=0x7f227400c850 comp rx=0 tx=0).stop 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 shutdown_connections 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22840b3fd0 0x7f228413bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2284144910 0x7f22841546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 --2- 172.21.15.113:0/15679984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f228406e6e0 0x7f228413b710 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 >> 172.21.15.113:0/15679984 conn(0x7f22840b1a50 msgr2=0x7f2284071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 shutdown_connections 2024-04-25T00:00:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.125+0000 7f228a038640 1 -- 172.21.15.113:0/15679984 wait complete. 2024-04-25T00:00:38.136 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:38.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.233+0000 7f8ebd4b3640 1 Processor -- start 2024-04-25T00:00:38.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.233+0000 7f8ebd4b3640 1 -- start start 2024-04-25T00:00:38.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.233+0000 7f8ebd4b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8eb8158910 0x7f8eb8162710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.233+0000 7f8ebd4b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8eb806b800 0x7f8eb81438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.233+0000 7f8ebd4b3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb815f1d0 con 0x7f8eb815e7a0 2024-04-25T00:00:38.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.233+0000 7f8ebd4b3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb8162c50 con 0x7f8eb806b800 2024-04-25T00:00:38.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.233+0000 7f8ebd4b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb815f4b0 con 0x7f8eb8158910 2024-04-25T00:00:38.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8eb806b800 0x7f8eb81438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8eb806b800 0x7f8eb81438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52466/0 (socket says 172.21.15.113:52466) 2024-04-25T00:00:38.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb77fe640 1 -- 172.21.15.113:0/131814417 learned_addr learned my addr 172.21.15.113:0/131814417 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:38.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb6ffd640 1 --2- 172.21.15.113:0/131814417 >> v2:172.21.15.178:3300/0 conn(0x7f8eb8158910 0x7f8eb8162710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb77fe640 1 -- 172.21.15.113:0/131814417 >> v2:172.21.15.178:3300/0 conn(0x7f8eb8158910 msgr2=0x7f8eb8162710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb77fe640 1 --2- 172.21.15.113:0/131814417 >> v2:172.21.15.178:3300/0 conn(0x7f8eb8158910 0x7f8eb8162710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb77fe640 1 -- 172.21.15.113:0/131814417 >> v1:172.21.15.178:6789/0 conn(0x7f8eb815e7a0 legacy=0x7f8eb806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.234+0000 7f8eb77fe640 1 -- 172.21.15.113:0/131814417 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb8161f10 con 0x7f8eb806b800 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8eb77fe640 1 --2- 172.21.15.113:0/131814417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb81438f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80b0100 tx=0x7f8eb002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c84e0ec1dd948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8eb5ffb640 1 -- 172.21.15.113:0/131814417 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0036070 con 0x7f8eb806b800 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8eb5ffb640 1 -- 172.21.15.113:0/131814417 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb0031600 con 0x7f8eb806b800 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8eb5ffb640 1 -- 172.21.15.113:0/131814417 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00319d0 con 0x7f8eb806b800 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/131814417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 msgr2=0x7f8eb81438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/131814417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb81438f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80b0100 tx=0x7f8eb002a6f0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/131814417 shutdown_connections 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/131814417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb81438f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/131814417 >> v2:172.21.15.178:3300/0 conn(0x7f8eb8158910 0x7f8eb8162710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/131814417 >> 172.21.15.113:0/131814417 conn(0x7f8eb80b22c0 msgr2=0x7f8eb80b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:38.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/131814417 shutdown_connections 2024-04-25T00:00:38.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.235+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/131814417 wait complete. 2024-04-25T00:00:38.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.236+0000 7f8ebd4b3640 1 Processor -- start 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8ebd4b3640 1 -- start start 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8ebd4b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb813cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8ebd4b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb8158910 0x7f8eb8160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8ebd4b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8eb815e7a0 0x7f8eb8161220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8ebd4b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb80b3890 con 0x7f8eb806b800 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8ebd4b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb80b3710 con 0x7f8eb8158910 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8ebd4b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb815ec50 con 0x7f8eb815e7a0 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb813cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb813cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52478/0 (socket says 172.21.15.113:52478) 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.237+0000 7f8eb6ffd640 1 -- 172.21.15.113:0/3159339813 learned_addr learned my addr 172.21.15.113:0/3159339813 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eb6ffd640 1 -- 172.21.15.113:0/3159339813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8eb815e7a0 msgr2=0x7f8eb8161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eb6ffd640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8eb815e7a0 0x7f8eb8161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eb6ffd640 1 -- 172.21.15.113:0/3159339813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb8158910 msgr2=0x7f8eb8160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eb67fc640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb8158910 0x7f8eb8160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eb6ffd640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb8158910 0x7f8eb8160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eb6ffd640 1 -- 172.21.15.113:0/3159339813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb813d040 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eb6ffd640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb813cb00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c7f0 tx=0x7f8ea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eaf7fe640 1 -- 172.21.15.113:0/3159339813 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8015070 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eaf7fe640 1 -- 172.21.15.113:0/3159339813 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea8004240 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb81e7d50 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8eaf7fe640 1 -- 172.21.15.113:0/3159339813 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8007620 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.238+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81e82d0 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.239+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb80691a0 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.239+0000 7f8eaf7fe640 1 -- 172.21.15.113:0/3159339813 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ea80077c0 con 0x7f8eb806b800 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.240+0000 7f8eaf7fe640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e9806be30 0x7f8e9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.240+0000 7f8eaf7fe640 1 -- 172.21.15.113:0/3159339813 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f8ea808b090 con 0x7f8eb806b800 2024-04-25T00:00:38.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.240+0000 7f8eb67fc640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e9806be30 0x7f8e9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.240+0000 7f8eb67fc640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e9806be30 0x7f8e9806e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8eb815e770 tx=0x7f8ea0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.244+0000 7f8eaf7fe640 1 -- 172.21.15.113:0/3159339813 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8eb813d040 con 0x7f8eb806b800 2024-04-25T00:00:38.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.453+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8eb806c5e0 con 0x7f8eb806b800 2024-04-25T00:00:38.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.453+0000 7f8eaf7fe640 1 -- 172.21.15.113:0/3159339813 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5193 (secure 0 0 0) 0x7f8ea8059a20 con 0x7f8eb806b800 2024-04-25T00:00:38.454 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2613614055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2613614055},{"type":"v1","addr":"172.21.15.178:6834","nonce":2613614055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2494825991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2494825991},{"type":"v1","addr":"172.21.15.178:6835","nonce":2494825991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2843749120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2843749120},{"type":"v1","addr":"172.21.15.113:6836","nonce":2843749120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-24T23:59:10.365260+0000","modified":"2024-04-24T23:59:17.031612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3337939581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3337939581},{"type":"v1","addr":"172.21.15.113:6837","nonce":3337939581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-04-25T00:00:38.454 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 15 2024-04-25T00:00:38.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e9806be30 msgr2=0x7f8e9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e9806be30 0x7f8e9806e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8eb815e770 tx=0x7f8ea0008040 comp rx=0 tx=0).stop 2024-04-25T00:00:38.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 msgr2=0x7f8eb813cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb813cb00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c7f0 tx=0x7f8ea800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 shutdown_connections 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e9806be30 0x7f8e9806e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb8158910 0x7f8eb8160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8eb815e7a0 0x7f8eb8161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 --2- 172.21.15.113:0/3159339813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806b800 0x7f8eb813cb00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 >> 172.21.15.113:0/3159339813 conn(0x7f8eb80b22c0 msgr2=0x7f8eb80b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 shutdown_connections 2024-04-25T00:00:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.456+0000 7f8ebd4b3640 1 -- 172.21.15.113:0/3159339813 wait complete. 2024-04-25T00:00:38.467 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:38.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.563+0000 7f5cd59ee640 1 Processor -- start 2024-04-25T00:00:38.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.563+0000 7f5cd59ee640 1 -- start start 2024-04-25T00:00:38.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5cd59ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 0x7f5cd00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5cd59ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd0144cf0 con 0x7f5cd0144910 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5cd59ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd00b2f30 con 0x7f5cd00b3fd0 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5cd59ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd00b3210 con 0x7f5cd006e6e0 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5cceffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5cd0144910 0x7f5cd00b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53008/0 (socket says 172.21.15.113:53008) 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5cceffd640 1 -- 172.21.15.113:0/2001945818 learned_addr learned my addr 172.21.15.113:0/2001945818 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3362948277 0 0) 0x7f5cd0144cf0 con 0x7f5cd0144910 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cb8003520 con 0x7f5cd0144910 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.564+0000 7f5ccf7fe640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 0x7f5cd00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2456456482 0 0) 0x7f5cb8003520 con 0x7f5cd0144910 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 msgr2=0x7f5cd00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 0x7f5cd00cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 >> v1:172.21.15.178:6789/0 conn(0x7f5cd00b3fd0 legacy=0x7f5cd006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd00b34f0 con 0x7f5cd0144910 2024-04-25T00:00:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f5cc00019f0 con 0x7f5cd0144910 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccf7fe640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 0x7f5cd00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 >> v1:172.21.15.113:6789/0 conn(0x7f5cd0144910 legacy=0x7f5cd00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb80039e0 0x7f5cb8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 0x7f5cb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cb8004d40 0x7f5cb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd00b3210 con 0x7f5cb80039e0 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb80099e0 con 0x7f5cb8004d40 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb8009be0 con 0x7f5cd006e6e0 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb80039e0 0x7f5cb8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5cce7fc640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 0x7f5cb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5ccf7fe640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cb8004d40 0x7f5cb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5cceffd640 1 -- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 msgr2=0x7f5cb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cd006e6e0 0x7f5cb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5cceffd640 1 -- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cb8004d40 msgr2=0x7f5cb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cb8004d40 0x7f5cb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.565+0000 7f5cceffd640 1 -- 172.21.15.113:0/2001945818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb8003520 con 0x7f5cb80039e0 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb80039e0 0x7f5cb8003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000a830 tx=0x7f5cb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ac005649e77308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000a480 con 0x7f5cb80039e0 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc002f070 con 0x7f5cb80039e0 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5ccdffb640 1 -- 172.21.15.113:0/2001945818 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc002bc00 con 0x7f5cb80039e0 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2001945818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb80039e0 msgr2=0x7f5cb8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5cd59ee640 1 --2- 172.21.15.113:0/2001945818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb80039e0 0x7f5cb8003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000a830 tx=0x7f5cb8005830 comp rx=0 tx=0).stop 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5ccf7fe640 1 -- 172.21.15.113:0/2001945818 reap_dead start 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2001945818 shutdown_connections 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2001945818 >> 172.21.15.113:0/2001945818 conn(0x7f5cd00b1a50 msgr2=0x7f5cd00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2001945818 shutdown_connections 2024-04-25T00:00:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.566+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2001945818 wait complete. 2024-04-25T00:00:38.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 Processor -- start 2024-04-25T00:00:38.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 -- start start 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cb80039e0 0x7f5cd01f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cd013d830 0x7f5cd0152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd013dcb0 0x7f5cd0153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd00b4da0 con 0x7f5cd013d830 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd00b4440 con 0x7f5cd013dcb0 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.568+0000 7f5cd59ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd00b4f20 con 0x7f5cb80039e0 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cd013d830 0x7f5cd0152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cd013d830 0x7f5cd0152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52482/0 (socket says 172.21.15.113:52482) 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 -- 172.21.15.113:0/2805480378 learned_addr learned my addr 172.21.15.113:0/2805480378 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5ccf7fe640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd013dcb0 0x7f5cd0153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cb80039e0 0x7f5cd01f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 -- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cb80039e0 msgr2=0x7f5cd01f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cb80039e0 0x7f5cd01f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 -- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd013dcb0 msgr2=0x7f5cd0153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd013dcb0 0x7f5cd0153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 -- 172.21.15.113:0/2805480378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd013e240 con 0x7f5cd013d830 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5ccf7fe640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd013dcb0 0x7f5cd0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cb80039e0 0x7f5cd01f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cce7fc640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cd013d830 0x7f5cd0152fc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400f820 tx=0x7f5cc400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cb7fff640 1 -- 172.21.15.113:0/2805480378 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40050d0 con 0x7f5cd013d830 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cb7fff640 1 -- 172.21.15.113:0/2805480378 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc4018070 con 0x7f5cd013d830 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cb7fff640 1 -- 172.21.15.113:0/2805480378 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4005490 con 0x7f5cd013d830 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd013e520 con 0x7f5cd013d830 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.569+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd013eae0 con 0x7f5cd013d830 2024-04-25T00:00:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.570+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd00691a0 con 0x7f5cd013d830 2024-04-25T00:00:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.571+0000 7f5cb7fff640 1 -- 172.21.15.113:0/2805480378 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cc400d6d0 con 0x7f5cd013d830 2024-04-25T00:00:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.571+0000 7f5cb7fff640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5cbc06be60 0x7f5cbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.571+0000 7f5cb7fff640 1 -- 172.21.15.113:0/2805480378 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7f5cc408aca0 con 0x7f5cd013d830 2024-04-25T00:00:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.571+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5cbc06be60 0x7f5cbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.571+0000 7f5cceffd640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5cbc06be60 0x7f5cbc06e320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0033000 tx=0x7f5cc0009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.575+0000 7f5cb7fff640 1 -- 172.21.15.113:0/2805480378 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5cc4059630 con 0x7f5cd013d830 2024-04-25T00:00:38.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.748+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cd00ce630 con 0x7f5cd013d830 2024-04-25T00:00:38.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.749+0000 7f5cb7fff640 1 -- 172.21.15.113:0/2805480378 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16782 (secure 0 0 0) 0x7f5cc4058f30 con 0x7f5cd013d830 2024-04-25T00:00:38.749 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:09.800045+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:59:08.215866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T23:59:09.335267+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":33,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:38.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5cbc06be60 msgr2=0x7f5cbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5cbc06be60 0x7f5cbc06e320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0033000 tx=0x7f5cc0009030 comp rx=0 tx=0).stop 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cd013d830 msgr2=0x7f5cd0152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cd013d830 0x7f5cd0152fc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400f820 tx=0x7f5cc400fce0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 shutdown_connections 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5cbc06be60 0x7f5cbc06e320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cd013dcb0 0x7f5cd0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cd013d830 0x7f5cd0152fc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 --2- 172.21.15.113:0/2805480378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cb80039e0 0x7f5cd01f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 >> 172.21.15.113:0/2805480378 conn(0x7f5cd00b1a50 msgr2=0x7f5cd00cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 shutdown_connections 2024-04-25T00:00:38.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.751+0000 7f5cd59ee640 1 -- 172.21.15.113:0/2805480378 wait complete. 2024-04-25T00:00:38.762 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:00:38.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.860+0000 7fa3c07f4640 1 Processor -- start 2024-04-25T00:00:38.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.860+0000 7fa3c07f4640 1 -- start start 2024-04-25T00:00:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.861+0000 7fa3c07f4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa3b815e510 0x7fa3b80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.861+0000 7fa3c07f4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3b80b63b0 0x7fa3b80b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.861+0000 7fa3c07f4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b80d2e00 con 0x7fa3b80d28c0 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.861+0000 7fa3c07f4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b80d30e0 con 0x7fa3b80b63b0 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.861+0000 7fa3c07f4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b80d33c0 con 0x7fa3b815e510 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bed91640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3b80b63b0 0x7fa3b80b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bdd8f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa3b80d28c0 0x7fa3b80b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53022/0 (socket says 172.21.15.113:53022) 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bed91640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3b80b63b0 0x7fa3b80b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52484/0 (socket says 172.21.15.113:52484) 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bdd8f640 1 -- 172.21.15.113:0/1766562085 learned_addr learned my addr 172.21.15.113:0/1766562085 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3be590640 1 --2- 172.21.15.113:0/1766562085 >> v2:172.21.15.178:3300/0 conn(0x7fa3b815e510 0x7fa3b80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bed91640 1 -- 172.21.15.113:0/1766562085 >> v2:172.21.15.178:3300/0 conn(0x7fa3b815e510 msgr2=0x7fa3b80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bed91640 1 --2- 172.21.15.113:0/1766562085 >> v2:172.21.15.178:3300/0 conn(0x7fa3b815e510 0x7fa3b80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bed91640 1 -- 172.21.15.113:0/1766562085 >> v1:172.21.15.113:6789/0 conn(0x7fa3b80d28c0 legacy=0x7fa3b80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bed91640 1 -- 172.21.15.113:0/1766562085 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b80d36a0 con 0x7fa3b80b63b0 2024-04-25T00:00:38.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.862+0000 7fa3bd58e640 1 -- 172.21.15.113:0/1766562085 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4074237564 0 0) 0x7fa3b80d2e00 con 0x7fa3b80d28c0 2024-04-25T00:00:38.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3bed91640 1 --2- 172.21.15.113:0/1766562085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b80b63b0 0x7fa3b80b73c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80d52a0 tx=0x7fa3b002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eb3d43365ba556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3bd58e640 1 -- 172.21.15.113:0/1766562085 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0007740 con 0x7fa3b80b63b0 2024-04-25T00:00:38.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3bd58e640 1 -- 172.21.15.113:0/1766562085 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b00078e0 con 0x7fa3b80b63b0 2024-04-25T00:00:38.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3bd58e640 1 -- 172.21.15.113:0/1766562085 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0007cb0 con 0x7fa3b80b63b0 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 -- 172.21.15.113:0/1766562085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b80b63b0 msgr2=0x7fa3b80b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/1766562085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b80b63b0 0x7fa3b80b73c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80d52a0 tx=0x7fa3b002a5f0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 -- 172.21.15.113:0/1766562085 shutdown_connections 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/1766562085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b80b63b0 0x7fa3b80b73c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/1766562085 >> v2:172.21.15.178:3300/0 conn(0x7fa3b815e510 0x7fa3b80d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 -- 172.21.15.113:0/1766562085 >> 172.21.15.113:0/1766562085 conn(0x7fa3b806b720 msgr2=0x7fa3b80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 -- 172.21.15.113:0/1766562085 shutdown_connections 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.863+0000 7fa3c07f4640 1 -- 172.21.15.113:0/1766562085 wait complete. 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.864+0000 7fa3c07f4640 1 Processor -- start 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.864+0000 7fa3c07f4640 1 -- start start 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3c07f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b80b63b0 0x7fa3b80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3c07f4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3b80d28c0 0x7fa3b80cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3c07f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b815e510 0x7fa3b81422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3c07f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b80b9dd0 con 0x7fa3b815e510 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3c07f4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b80d3ea0 con 0x7fa3b80b63b0 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3c07f4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b80b7900 con 0x7fa3b80d28c0 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b815e510 0x7fa3b81422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b815e510 0x7fa3b81422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52494/0 (socket says 172.21.15.113:52494) 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 -- 172.21.15.113:0/315977970 learned_addr learned my addr 172.21.15.113:0/315977970 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3be590640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b80b63b0 0x7fa3b80ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bdd8f640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3b80d28c0 0x7fa3b80cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 -- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3b80d28c0 msgr2=0x7fa3b80cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3b80d28c0 0x7fa3b80cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 -- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b80b63b0 msgr2=0x7fa3b80ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b80b63b0 0x7fa3b80ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.865+0000 7fa3bed91640 1 -- 172.21.15.113:0/315977970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b8160950 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.866+0000 7fa3bdd8f640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3b80d28c0 0x7fa3b80cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.866+0000 7fa3bed91640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b815e510 0x7fa3b81422c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa3b002abe0 tx=0x7fa3b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.866+0000 7fa3aeffd640 1 -- 172.21.15.113:0/315977970 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0040070 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.866+0000 7fa3aeffd640 1 -- 172.21.15.113:0/315977970 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b0031bc0 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.866+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b8160c30 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.866+0000 7fa3aeffd640 1 -- 172.21.15.113:0/315977970 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0045460 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.866+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b81611f0 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.867+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b80691a0 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.867+0000 7fa3aeffd640 1 -- 172.21.15.113:0/315977970 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3b0039040 con 0x7fa3b815e510 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.868+0000 7fa3aeffd640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39c06be30 0x7fa39c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.868+0000 7fa3aeffd640 1 -- 172.21.15.113:0/315977970 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5737+0+0 (secure 0 0 0) 0x7fa3b00ac6b0 con 0x7fa3b815e510 2024-04-25T00:00:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.868+0000 7fa3be590640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39c06be30 0x7fa39c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.868+0000 7fa3be590640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39c06be30 0x7fa39c06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80079a0 tx=0x7fa3a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:38.872+0000 7fa3aeffd640 1 -- 172.21.15.113:0/315977970 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3b007b040 con 0x7fa3b815e510 2024-04-25T00:00:39.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.087+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa3b80cab40 con 0x7fa3b815e510 2024-04-25T00:00:39.088 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:00:39.088+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:00:39.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.089+0000 7fa3aeffd640 1 -- 172.21.15.113:0/315977970 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) ==== 161+0+0 (secure 0 0 0) 0x7fa3b007a940 con 0x7fa3b815e510 2024-04-25T00:00:39.089 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:00:39.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39c06be30 msgr2=0x7fa39c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39c06be30 0x7fa39c06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80079a0 tx=0x7fa3a8008040 comp rx=0 tx=0).stop 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b815e510 msgr2=0x7fa3b81422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b815e510 0x7fa3b81422c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa3b002abe0 tx=0x7fa3b0004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 shutdown_connections 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa39c06be30 0x7fa39c06e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3b815e510 0x7fa3b81422c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3b80d28c0 0x7fa3b80cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 --2- 172.21.15.113:0/315977970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b80b63b0 0x7fa3b80ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 >> 172.21.15.113:0/315977970 conn(0x7fa3b806b720 msgr2=0x7fa3b80b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 shutdown_connections 2024-04-25T00:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.093+0000 7fa3c07f4640 1 -- 172.21.15.113:0/315977970 wait complete. 2024-04-25T00:00:39.105 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:00:39.132 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:00:39.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.205+0000 7fa1bd2bd640 1 Processor -- start 2024-04-25T00:00:39.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.205+0000 7fa1bd2bd640 1 -- start start 2024-04-25T00:00:39.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1bd2bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b80d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:39.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1bd2bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b8162560 con 0x7fa1b8161fc0 2024-04-25T00:00:39.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1bd2bd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b8162840 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1bd2bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b8162b20 con 0x7fa1b80cf3c0 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1b77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa1b8161fc0 0x7fa1b80d1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53040/0 (socket says 172.21.15.113:53040) 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1b77fe640 1 -- 172.21.15.113:0/6069981 learned_addr learned my addr 172.21.15.113:0/6069981 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1b6ffd640 1 --2- 172.21.15.113:0/6069981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b80d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1b5ffb640 1 -- 172.21.15.113:0/6069981 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170414262 0 0) 0x7fa1b8162560 con 0x7fa1b8161fc0 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1b5ffb640 1 -- 172.21.15.113:0/6069981 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1a0003520 con 0x7fa1b8161fc0 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.206+0000 7fa1b5ffb640 1 -- 172.21.15.113:0/6069981 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283082408 0 0) 0x7fa1b8162840 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.207+0000 7fa1b5ffb640 1 -- 172.21.15.113:0/6069981 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1b8162560 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.207+0000 7fa1b6ffd640 1 -- 172.21.15.113:0/6069981 >> v1:172.21.15.178:6789/0 conn(0x7fa1b80cf7a0 legacy=0x7fa1b80d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.207+0000 7fa1b6ffd640 1 -- 172.21.15.113:0/6069981 >> v1:172.21.15.113:6789/0 conn(0x7fa1b8161fc0 legacy=0x7fa1b80d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.207+0000 7fa1b6ffd640 1 -- 172.21.15.113:0/6069981 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b80d40f0 con 0x7fa1b80cf3c0 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.207+0000 7fa1b6ffd640 1 --2- 172.21.15.113:0/6069981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b80d3bb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80097d0 tx=0x7fa1a802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25647b53ba280a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.207+0000 7fa1b5ffb640 1 -- 172.21.15.113:0/6069981 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8037070 con 0x7fa1b80cf3c0 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1b5ffb640 1 -- 172.21.15.113:0/6069981 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a8032440 con 0x7fa1b80cf3c0 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1b5ffb640 1 -- 172.21.15.113:0/6069981 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80327f0 con 0x7fa1b80cf3c0 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/6069981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 msgr2=0x7fa1b80d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/6069981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b80d3bb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80097d0 tx=0x7fa1a802a210 comp rx=0 tx=0).stop 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/6069981 shutdown_connections 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/6069981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b80d3bb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/6069981 >> 172.21.15.113:0/6069981 conn(0x7fa1b80718f0 msgr2=0x7fa1b8071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/6069981 shutdown_connections 2024-04-25T00:00:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.208+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/6069981 wait complete. 2024-04-25T00:00:39.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.209+0000 7fa1bd2bd640 1 Processor -- start 2024-04-25T00:00:39.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.209+0000 7fa1bd2bd640 1 -- start start 2024-04-25T00:00:39.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1bd2bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b81f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:39.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1bd2bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b80cf7a0 0x7fa1b80b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:39.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1bd2bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1b8161fc0 0x7fa1b80cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1bd2bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b80d1920 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1bd2bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b80d2bf0 con 0x7fa1b8161fc0 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1bd2bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b80d1aa0 con 0x7fa1b80cf3c0 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b80cf7a0 0x7fa1b80b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b80cf7a0 0x7fa1b80b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52512/0 (socket says 172.21.15.113:52512) 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b67fc640 1 -- 172.21.15.113:0/2943659392 learned_addr learned my addr 172.21.15.113:0/2943659392 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b77fe640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1b8161fc0 0x7fa1b80cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b6ffd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b81f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b67fc640 1 -- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 msgr2=0x7fa1b81f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b67fc640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b81f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.210+0000 7fa1b67fc640 1 -- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1b8161fc0 msgr2=0x7fa1b80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa1b67fc640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1b8161fc0 0x7fa1b80cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa1b67fc640 1 -- 172.21.15.113:0/2943659392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b80b7860 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa1b67fc640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b80cf7a0 0x7fa1b80b50d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c430 tx=0x7fa1ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa19f7fe640 1 -- 172.21.15.113:0/2943659392 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac017070 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa19f7fe640 1 -- 172.21.15.113:0/2943659392 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ac0052f0 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b80b7b40 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa19f7fe640 1 -- 172.21.15.113:0/2943659392 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac003830 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.211+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b80b74c0 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.212+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b80d43b0 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.212+0000 7fa19f7fe640 1 -- 172.21.15.113:0/2943659392 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1ac01a070 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.213+0000 7fa19f7fe640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1a406be30 0x7fa1a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.213+0000 7fa19f7fe640 1 -- 172.21.15.113:0/2943659392 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fa1ac08a220 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.213+0000 7fa1b6ffd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1a406be30 0x7fa1a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.213+0000 7fa1b6ffd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1a406be30 0x7fa1a406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8034000 tx=0x7fa1a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:39.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.217+0000 7fa19f7fe640 1 -- 172.21.15.113:0/2943659392 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1ac058c00 con 0x7fa1b80cf7a0 2024-04-25T00:00:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:39.430+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa1b80691a0 con 0x7fa1b80cf7a0 2024-04-25T00:00:40.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.096+0000 7fa19f7fe640 1 -- 172.21.15.113:0/2943659392 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7fa1ac058500 con 0x7fa1b80cf7a0 2024-04-25T00:00:40.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1a406be30 msgr2=0x7fa1a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:40.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1a406be30 0x7fa1a406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8034000 tx=0x7fa1a802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:40.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b80cf7a0 msgr2=0x7fa1b80b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:40.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b80cf7a0 0x7fa1b80b50d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c430 tx=0x7fa1ac00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 shutdown_connections 2024-04-25T00:00:40.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1a406be30 0x7fa1a406e2f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1b8161fc0 0x7fa1b80cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b80cf7a0 0x7fa1b80b50d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 --2- 172.21.15.113:0/2943659392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1b80cf3c0 0x7fa1b81f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 >> 172.21.15.113:0/2943659392 conn(0x7fa1b80718f0 msgr2=0x7fa1b80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:40.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 shutdown_connections 2024-04-25T00:00:40.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.100+0000 7fa1bd2bd640 1 -- 172.21.15.113:0/2943659392 wait complete. 2024-04-25T00:00:40.115 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:00:40.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.262+0000 7f5a4df89640 1 Processor -- start 2024-04-25T00:00:40.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.262+0000 7f5a4df89640 1 -- start start 2024-04-25T00:00:40.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.263+0000 7f5a4df89640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5a4815e480 0x7f5a48061b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:40.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.263+0000 7f5a4df89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a480627d0 0x7f5a480620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:40.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.263+0000 7f5a4df89640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4815e860 con 0x7f5a480ce450 2024-04-25T00:00:40.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.263+0000 7f5a4df89640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4806c100 con 0x7f5a4815e480 2024-04-25T00:00:40.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.263+0000 7f5a4df89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5a480d0940 con 0x7f5a480627d0 2024-04-25T00:00:40.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a477fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5a4815e480 0x7f5a48061b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:40.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a47fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a480627d0 0x7f5a480620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:40.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a477fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5a4815e480 0x7f5a48061b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:45258/0 (socket says 172.21.15.113:45258) 2024-04-25T00:00:40.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a477fe640 1 -- 172.21.15.113:0/719670505 learned_addr learned my addr 172.21.15.113:0/719670505 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:40.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a46ffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5a480ce450 0x7f5a480cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:34782/0 (socket says 172.21.15.113:34782) 2024-04-25T00:00:40.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a477fe640 1 -- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a480627d0 msgr2=0x7f5a480620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:40.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a477fe640 1 --2- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a480627d0 0x7f5a480620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a477fe640 1 -- 172.21.15.113:0/719670505 >> v1:172.21.15.178:6789/0 conn(0x7f5a480ce450 legacy=0x7f5a480cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:40.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a477fe640 1 -- 172.21.15.113:0/719670505 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4806eaa0 con 0x7f5a4815e480 2024-04-25T00:00:40.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.264+0000 7f5a47fff640 1 --2- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a480627d0 0x7f5a480620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a477fe640 1 --2- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a4815e480 0x7f5a48061b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009870 tx=0x7f5a3802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb5711343527bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a467fc640 1 -- 172.21.15.113:0/719670505 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38037070 con 0x7f5a4815e480 2024-04-25T00:00:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a467fc640 1 -- 172.21.15.113:0/719670505 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a38032440 con 0x7f5a4815e480 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a467fc640 1 -- 172.21.15.113:0/719670505 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38032810 con 0x7f5a4815e480 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a4df89640 1 -- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a4815e480 msgr2=0x7f5a48061b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a4df89640 1 --2- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a4815e480 0x7f5a48061b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009870 tx=0x7f5a3802a270 comp rx=0 tx=0).stop 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a4df89640 1 -- 172.21.15.113:0/719670505 shutdown_connections 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a4df89640 1 --2- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a480627d0 0x7f5a480620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a4df89640 1 --2- 172.21.15.113:0/719670505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a4815e480 0x7f5a48061b80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.265+0000 7f5a4df89640 1 -- 172.21.15.113:0/719670505 >> 172.21.15.113:0/719670505 conn(0x7f5a480cd930 msgr2=0x7f5a480cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.266+0000 7f5a4df89640 1 -- 172.21.15.113:0/719670505 shutdown_connections 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.266+0000 7f5a4df89640 1 -- 172.21.15.113:0/719670505 wait complete. 2024-04-25T00:00:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.267+0000 7f5a4df89640 1 Processor -- start 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.267+0000 7f5a4df89640 1 -- start start 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.267+0000 7f5a4df89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a480627d0 0x7f5a480a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.267+0000 7f5a4df89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a480ce450 0x7f5a480b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a4df89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a4815e480 0x7f5a480b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a4df89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a480b0620 con 0x7f5a480627d0 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a4df89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4806f060 con 0x7f5a480ce450 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a4df89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4806f580 con 0x7f5a4815e480 2024-04-25T00:00:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a480627d0 0x7f5a480a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a480627d0 0x7f5a480a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52522/0 (socket says 172.21.15.113:52522) 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 -- 172.21.15.113:0/511453852 learned_addr learned my addr 172.21.15.113:0/511453852 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a46ffd640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a480ce450 0x7f5a480b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a47fff640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a4815e480 0x7f5a480b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 -- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a4815e480 msgr2=0x7f5a480b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a4815e480 0x7f5a480b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 -- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a480ce450 msgr2=0x7f5a480b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a480ce450 0x7f5a480b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 -- 172.21.15.113:0/511453852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a480c9800 con 0x7f5a480627d0 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a46ffd640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a480ce450 0x7f5a480b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a47fff640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a4815e480 0x7f5a480b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a477fe640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a480627d0 0x7f5a480a0650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a2ffff640 1 -- 172.21.15.113:0/511453852 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38041020 con 0x7f5a480627d0 2024-04-25T00:00:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a2ffff640 1 -- 172.21.15.113:0/511453852 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3803d620 con 0x7f5a480627d0 2024-04-25T00:00:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a2ffff640 1 -- 172.21.15.113:0/511453852 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38037040 con 0x7f5a480627d0 2024-04-25T00:00:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a480c9ae0 con 0x7f5a480627d0 2024-04-25T00:00:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.268+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a480a0b90 con 0x7f5a480627d0 2024-04-25T00:00:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.269+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a480691a0 con 0x7f5a480627d0 2024-04-25T00:00:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.270+0000 7f5a2ffff640 1 -- 172.21.15.113:0/511453852 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a38044020 con 0x7f5a480627d0 2024-04-25T00:00:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.270+0000 7f5a2ffff640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a3406be30 0x7f5a3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.270+0000 7f5a2ffff640 1 -- 172.21.15.113:0/511453852 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f5a380ab660 con 0x7f5a480627d0 2024-04-25T00:00:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.270+0000 7f5a46ffd640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a3406be30 0x7f5a3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.271+0000 7f5a46ffd640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a3406be30 0x7f5a3406e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a480d02e0 tx=0x7f5a3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:40.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.275+0000 7f5a2ffff640 1 -- 172.21.15.113:0/511453852 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a3804b030 con 0x7f5a480627d0 2024-04-25T00:00:40.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:40.449+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5a480d0030 con 0x7f5a480627d0 2024-04-25T00:00:41.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.100+0000 7f5a2ffff640 1 -- 172.21.15.113:0/511453852 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f5a38079f30 con 0x7f5a480627d0 2024-04-25T00:00:41.100 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:00:41.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a3406be30 msgr2=0x7f5a3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:41.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a3406be30 0x7f5a3406e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a480d02e0 tx=0x7f5a3c009040 comp rx=0 tx=0).stop 2024-04-25T00:00:41.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a480627d0 msgr2=0x7f5a480a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:41.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a480627d0 0x7f5a480a0650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a380046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 shutdown_connections 2024-04-25T00:00:41.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a3406be30 0x7f5a3406e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a480ce450 0x7f5a480b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a4815e480 0x7f5a480b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 --2- 172.21.15.113:0/511453852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a480627d0 0x7f5a480a0650 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.103+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 >> 172.21.15.113:0/511453852 conn(0x7f5a480cd930 msgr2=0x7f5a480b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:41.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.105+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 shutdown_connections 2024-04-25T00:00:41.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.105+0000 7f5a4df89640 1 -- 172.21.15.113:0/511453852 wait complete. 2024-04-25T00:00:41.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:00:41.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.226+0000 7f0c19512640 1 Processor -- start 2024-04-25T00:00:41.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.226+0000 7f0c19512640 1 -- start start 2024-04-25T00:00:41.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.227+0000 7f0c19512640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0c140b9bb0 0x7f0c140b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:41.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.227+0000 7f0c19512640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c140b61e0 0x7f0c140d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:41.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.227+0000 7f0c19512640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c141626f0 con 0x7f0c1415e510 2024-04-25T00:00:41.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.227+0000 7f0c19512640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0c141629d0 con 0x7f0c140b9bb0 2024-04-25T00:00:41.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.227+0000 7f0c19512640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c14162cb0 con 0x7f0c140b61e0 2024-04-25T00:00:41.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c127fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0c140b9bb0 0x7f0c140b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:41.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c140b61e0 0x7f0c140d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:41.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c12ffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0c1415e510 0x7f0c14161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:34794/0 (socket says 172.21.15.113:34794) 2024-04-25T00:00:41.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c12ffd640 1 -- 172.21.15.113:0/3201202799 learned_addr learned my addr 172.21.15.113:0/3201202799 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:41.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c127fc640 1 -- 172.21.15.113:0/3201202799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c140b61e0 msgr2=0x7f0c140d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:41.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c127fc640 1 --2- 172.21.15.113:0/3201202799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c140b61e0 0x7f0c140d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c127fc640 1 -- 172.21.15.113:0/3201202799 >> v1:172.21.15.178:6789/0 conn(0x7f0c1415e510 legacy=0x7f0c14161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:41.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c127fc640 1 -- 172.21.15.113:0/3201202799 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c140d2660 con 0x7f0c140b9bb0 2024-04-25T00:00:41.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.228+0000 7f0c11ffb640 1 -- 172.21.15.113:0/3201202799 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914670999 0 0) 0x7f0c141626f0 con 0x7f0c1415e510 2024-04-25T00:00:41.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.229+0000 7f0c127fc640 1 --2- 172.21.15.113:0/3201202799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b9bb0 0x7f0c140b5ca0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097a0 tx=0x7f0c0802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d0cfcdf0b63d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:41.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.229+0000 7f0c11ffb640 1 -- 172.21.15.113:0/3201202799 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08037070 con 0x7f0c140b9bb0 2024-04-25T00:00:41.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.229+0000 7f0c11ffb640 1 -- 172.21.15.113:0/3201202799 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0802ae50 con 0x7f0c140b9bb0 2024-04-25T00:00:41.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.229+0000 7f0c11ffb640 1 -- 172.21.15.113:0/3201202799 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08032660 con 0x7f0c140b9bb0 2024-04-25T00:00:41.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 -- 172.21.15.113:0/3201202799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b9bb0 msgr2=0x7f0c140b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:41.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 --2- 172.21.15.113:0/3201202799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b9bb0 0x7f0c140b5ca0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097a0 tx=0x7f0c0802a100 comp rx=0 tx=0).stop 2024-04-25T00:00:41.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 -- 172.21.15.113:0/3201202799 shutdown_connections 2024-04-25T00:00:41.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 --2- 172.21.15.113:0/3201202799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c140b61e0 0x7f0c140d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 --2- 172.21.15.113:0/3201202799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b9bb0 0x7f0c140b5ca0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 -- 172.21.15.113:0/3201202799 >> 172.21.15.113:0/3201202799 conn(0x7f0c1406b720 msgr2=0x7f0c140d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 -- 172.21.15.113:0/3201202799 shutdown_connections 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.230+0000 7f0c19512640 1 -- 172.21.15.113:0/3201202799 wait complete. 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.231+0000 7f0c19512640 1 Processor -- start 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.231+0000 7f0c19512640 1 -- start start 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b61e0 0x7f0c1414be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140b9bb0 0x7f0c1414c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:41.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c1415e510 0x7f0c14155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140b7540 con 0x7f0c140b9bb0 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140b73c0 con 0x7f0c140b61e0 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140d33a0 con 0x7f0c1415e510 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140b9bb0 0x7f0c1414c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140b9bb0 0x7f0c1414c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52528/0 (socket says 172.21.15.113:52528) 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 -- 172.21.15.113:0/1189274634 learned_addr learned my addr 172.21.15.113:0/1189274634 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c12ffd640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b61e0 0x7f0c1414be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c137fe640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c1415e510 0x7f0c14155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 -- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c1415e510 msgr2=0x7f0c14155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c1415e510 0x7f0c14155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 -- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b61e0 msgr2=0x7f0c1414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b61e0 0x7f0c1414be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 -- 172.21.15.113:0/1189274634 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1415fdb0 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c127fc640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140b9bb0 0x7f0c1414c5a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097a0 tx=0x7f0c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0bfb7fe640 1 -- 172.21.15.113:0/1189274634 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08037040 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0bfb7fe640 1 -- 172.21.15.113:0/1189274634 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c08004030 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c14160090 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0bfb7fe640 1 -- 172.21.15.113:0/1189274634 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08004420 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.232+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c14160650 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.233+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c140691a0 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.234+0000 7f0bfb7fe640 1 -- 172.21.15.113:0/1189274634 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c08003610 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.234+0000 7f0bfb7fe640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c0006be30 0x7f0c0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.234+0000 7f0bfb7fe640 1 -- 172.21.15.113:0/1189274634 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7f0c080ab940 con 0x7f0c140b9bb0 2024-04-25T00:00:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.234+0000 7f0c12ffd640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c0006be30 0x7f0c0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.235+0000 7f0c12ffd640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c0006be30 0x7f0c0006e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c04007fd0 tx=0x7f0c04007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.238+0000 7f0bfb7fe640 1 -- 172.21.15.113:0/1189274634 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c08079f20 con 0x7f0c140b9bb0 2024-04-25T00:00:41.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:41.422+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0c140d1f30 con 0x7f0c140b9bb0 2024-04-25T00:00:42.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.106+0000 7f0bfb7fe640 1 -- 172.21.15.113:0/1189274634 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7f0c0807e220 con 0x7f0c140b9bb0 2024-04-25T00:00:42.106 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:00:42.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.109+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c0006be30 msgr2=0x7f0c0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.109+0000 7f0c19512640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c0006be30 0x7f0c0006e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c04007fd0 tx=0x7f0c04007de0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.109+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140b9bb0 msgr2=0x7f0c1414c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.109+0000 7f0c19512640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140b9bb0 0x7f0c1414c5a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097a0 tx=0x7f0c080046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.110+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 shutdown_connections 2024-04-25T00:00:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.110+0000 7f0c19512640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0c0006be30 0x7f0c0006e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.110+0000 7f0c19512640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c1415e510 0x7f0c14155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.110+0000 7f0c19512640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140b9bb0 0x7f0c1414c5a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.110+0000 7f0c19512640 1 --2- 172.21.15.113:0/1189274634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c140b61e0 0x7f0c1414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.110+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 >> 172.21.15.113:0/1189274634 conn(0x7f0c1406b720 msgr2=0x7f0c1415d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:42.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.112+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 shutdown_connections 2024-04-25T00:00:42.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.112+0000 7f0c19512640 1 -- 172.21.15.113:0/1189274634 wait complete. 2024-04-25T00:00:42.129 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:00:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.229+0000 7f1793c72640 1 Processor -- start 2024-04-25T00:00:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f1793c72640 1 -- start start 2024-04-25T00:00:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f1793c72640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f178c0b5ca0 0x7f178c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f1793c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0b73c0 0x7f178c0b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f1793c72640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f178c0d2e00 con 0x7f178c15e510 2024-04-25T00:00:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f1793c72640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f178c0b6640 con 0x7f178c0b5ca0 2024-04-25T00:00:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f1793c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f178c0d30e0 con 0x7f178c0b73c0 2024-04-25T00:00:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f179120d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f178c0b5ca0 0x7f178c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.230+0000 7f1791a0e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f178c15e510 0x7f178c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53042/0 (socket says 172.21.15.113:53042) 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f1791a0e640 1 -- 172.21.15.113:0/3999118168 learned_addr learned my addr 172.21.15.113:0/3999118168 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f1790a0c640 1 -- 172.21.15.113:0/3999118168 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2460320700 0 0) 0x7f178c0d2e00 con 0x7f178c15e510 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f1790a0c640 1 -- 172.21.15.113:0/3999118168 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1778003520 con 0x7f178c15e510 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f179120d640 1 -- 172.21.15.113:0/3999118168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0b73c0 msgr2=0x7f178c0b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f179120d640 1 --2- 172.21.15.113:0/3999118168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0b73c0 0x7f178c0b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f179120d640 1 -- 172.21.15.113:0/3999118168 >> v1:172.21.15.113:6789/0 conn(0x7f178c15e510 legacy=0x7f178c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f179120d640 1 -- 172.21.15.113:0/3999118168 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c0d33c0 con 0x7f178c0b5ca0 2024-04-25T00:00:42.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.231+0000 7f179120d640 1 --2- 172.21.15.113:0/3999118168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0b6100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f177c0097a0 tx=0x7f177c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ec192e1123831d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1790a0c640 1 -- 172.21.15.113:0/3999118168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c037070 con 0x7f178c0b5ca0 2024-04-25T00:00:42.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1790a0c640 1 -- 172.21.15.113:0/3999118168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177c02ae50 con 0x7f178c0b5ca0 2024-04-25T00:00:42.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1790a0c640 1 -- 172.21.15.113:0/3999118168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c032660 con 0x7f178c0b5ca0 2024-04-25T00:00:42.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 -- 172.21.15.113:0/3999118168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 msgr2=0x7f178c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 --2- 172.21.15.113:0/3999118168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0b6100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f177c0097a0 tx=0x7f177c02a100 comp rx=0 tx=0).stop 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 -- 172.21.15.113:0/3999118168 shutdown_connections 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 --2- 172.21.15.113:0/3999118168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0b73c0 0x7f178c0b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 --2- 172.21.15.113:0/3999118168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0b6100 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 -- 172.21.15.113:0/3999118168 >> 172.21.15.113:0/3999118168 conn(0x7f178c06b720 msgr2=0x7f178c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 -- 172.21.15.113:0/3999118168 shutdown_connections 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.232+0000 7f1793c72640 1 -- 172.21.15.113:0/3999118168 wait complete. 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.233+0000 7f1793c72640 1 Processor -- start 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.233+0000 7f1793c72640 1 -- start start 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1793c72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1793c72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0b73c0 0x7f178c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1793c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c15e510 0x7f178c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1793c72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f178c0d3ea0 con 0x7f178c0b5ca0 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1793c72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f178c0d3980 con 0x7f178c0b73c0 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1793c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f178c0d2a90 con 0x7f178c15e510 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1791a0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1791a0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52544/0 (socket says 172.21.15.113:52544) 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1791a0e640 1 -- 172.21.15.113:0/1729217682 learned_addr learned my addr 172.21.15.113:0/1729217682 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.234+0000 7f1791a0e640 1 -- 172.21.15.113:0/1729217682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c15e510 msgr2=0x7f178c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f1791a0e640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c15e510 0x7f178c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f1791a0e640 1 -- 172.21.15.113:0/1729217682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0b73c0 msgr2=0x7f178c156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f1791a0e640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0b73c0 0x7f178c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f1791a0e640 1 -- 172.21.15.113:0/1729217682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c157090 con 0x7f178c0b5ca0 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f1791a0e640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0c8870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f178400c430 tx=0x7f178400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f17827fc640 1 -- 172.21.15.113:0/1729217682 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784017070 con 0x7f178c0b5ca0 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f17827fc640 1 -- 172.21.15.113:0/1729217682 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1784005ce0 con 0x7f178c0b5ca0 2024-04-25T00:00:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c0cb830 con 0x7f178c0b5ca0 2024-04-25T00:00:42.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f17827fc640 1 -- 172.21.15.113:0/1729217682 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17840072f0 con 0x7f178c0b5ca0 2024-04-25T00:00:42.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.235+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c0c9250 con 0x7f178c0b5ca0 2024-04-25T00:00:42.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.236+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178c0691a0 con 0x7f178c0b5ca0 2024-04-25T00:00:42.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.236+0000 7f17827fc640 1 -- 172.21.15.113:0/1729217682 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f178401a070 con 0x7f178c0b5ca0 2024-04-25T00:00:42.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.237+0000 7f17827fc640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f177006be30 0x7f177006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.237+0000 7f17827fc640 1 -- 172.21.15.113:0/1729217682 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f1784089fe0 con 0x7f178c0b5ca0 2024-04-25T00:00:42.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.237+0000 7f179120d640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f177006be30 0x7f177006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.237+0000 7f179120d640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f177006be30 0x7f177006e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f177c034000 tx=0x7f177c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.241+0000 7f17827fc640 1 -- 172.21.15.113:0/1729217682 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f178c157090 con 0x7f178c0b5ca0 2024-04-25T00:00:42.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.414+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f178c155a20 con 0x7f178c0b5ca0 2024-04-25T00:00:42.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.417+0000 7f17827fc640 1 -- 172.21.15.113:0/1729217682 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f1784058c90 con 0x7f178c0b5ca0 2024-04-25T00:00:42.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f177006be30 msgr2=0x7f177006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f177006be30 0x7f177006e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f177c034000 tx=0x7f177c02c000 comp rx=0 tx=0).stop 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 msgr2=0x7f178c0c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0c8870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f178400c430 tx=0x7f178400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 shutdown_connections 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f177006be30 0x7f177006e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0b73c0 0x7f178c156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c15e510 0x7f178c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 --2- 172.21.15.113:0/1729217682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f178c0b5ca0 0x7f178c0c8870 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 >> 172.21.15.113:0/1729217682 conn(0x7f178c06b720 msgr2=0x7f178c0cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 shutdown_connections 2024-04-25T00:00:42.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.419+0000 7f1793c72640 1 -- 172.21.15.113:0/1729217682 wait complete. 2024-04-25T00:00:42.430 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T00:00:42.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.526+0000 7f6990b5a640 1 Processor -- start 2024-04-25T00:00:42.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.526+0000 7f6990b5a640 1 -- start start 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f6990b5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f698c15a6b0 0x7f698c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f6990b5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 0x7f698c071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f6990b5a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698c0b2d50 con 0x7f698c071780 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f6990b5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f698c072a70 con 0x7f698c15a6b0 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f6990b5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f698c0b3010 con 0x7f698c072ea0 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f698ad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f698c071780 0x7f698c15e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53048/0 (socket says 172.21.15.113:53048) 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f698ad9d640 1 -- 172.21.15.113:0/2446308381 learned_addr learned my addr 172.21.15.113:0/2446308381 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059530033 0 0) 0x7f698c0b2d50 con 0x7f698c071780 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6974003520 con 0x7f698c071780 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f698a59c640 1 --2- 172.21.15.113:0/2446308381 >> v2:172.21.15.178:3300/0 conn(0x7f698c15a6b0 0x7f698c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.527+0000 7f6989d9b640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 0x7f698c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065263017 0 0) 0x7f6974003520 con 0x7f698c071780 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 msgr2=0x7f698c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 0x7f698c071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 >> v2:172.21.15.178:3300/0 conn(0x7f698c15a6b0 msgr2=0x7f698c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 --2- 172.21.15.113:0/2446308381 >> v2:172.21.15.178:3300/0 conn(0x7f698c15a6b0 0x7f698c070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c0b32f0 con 0x7f698c071780 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6984001670 con 0x7f698c071780 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 >> v1:172.21.15.113:6789/0 conn(0x7f698c071780 legacy=0x7f698c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69740039b0 0x7f6974003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 0x7f6974004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6974004d40 0x7f69740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f698c072a70 con 0x7f6974004d40 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f698c0b3010 con 0x7f69740039b0 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f697400b9c0 con 0x7f698c072ea0 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698ad9d640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6974004d40 0x7f69740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698a59c640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69740039b0 0x7f6974003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f6989d9b640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 0x7f6974004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698ad9d640 1 -- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 msgr2=0x7f6974004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698ad9d640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c072ea0 0x7f6974004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698ad9d640 1 -- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69740039b0 msgr2=0x7f6974003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698ad9d640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69740039b0 0x7f6974003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.528+0000 7f698ad9d640 1 -- 172.21.15.113:0/2446308381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6974003520 con 0x7f6974004d40 2024-04-25T00:00:42.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.529+0000 7f698ad9d640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6974004d40 0x7f69740051c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6984007fd0 tx=0x7f6974005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664f8ce2c8185229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.529+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984007d50 con 0x7f6974004d40 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.529+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6984005ad0 con 0x7f6974004d40 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.529+0000 7f698959a640 1 -- 172.21.15.113:0/2446308381 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984007070 con 0x7f6974004d40 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.529+0000 7f6990b5a640 1 -- 172.21.15.113:0/2446308381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6974004d40 msgr2=0x7f69740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.529+0000 7f6990b5a640 1 --2- 172.21.15.113:0/2446308381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6974004d40 0x7f69740051c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6984007fd0 tx=0x7f6974005830 comp rx=0 tx=0).stop 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.529+0000 7f698ad9d640 1 -- 172.21.15.113:0/2446308381 reap_dead start 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.530+0000 7f6990b5a640 1 -- 172.21.15.113:0/2446308381 shutdown_connections 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.530+0000 7f6990b5a640 1 -- 172.21.15.113:0/2446308381 >> 172.21.15.113:0/2446308381 conn(0x7f698c0b1a50 msgr2=0x7f698c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.530+0000 7f6990b5a640 1 -- 172.21.15.113:0/2446308381 shutdown_connections 2024-04-25T00:00:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.530+0000 7f6990b5a640 1 -- 172.21.15.113:0/2446308381 wait complete. 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.531+0000 7f6990b5a640 1 Processor -- start 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.531+0000 7f6990b5a640 1 -- start start 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6974004d40 0x7f698c14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698c1436f0 0x7f698c1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c1442e0 0x7f698c140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f698c15aa90 con 0x7f698c1436f0 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f698c072720 con 0x7f6974004d40 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f698c15ac10 con 0x7f698c1442e0 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698c1436f0 0x7f698c1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698c1436f0 0x7f698c1405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52570/0 (socket says 172.21.15.113:52570) 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 -- 172.21.15.113:0/4037877365 learned_addr learned my addr 172.21.15.113:0/4037877365 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f698a59c640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6974004d40 0x7f698c14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f698ad9d640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c1442e0 0x7f698c140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 -- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c1442e0 msgr2=0x7f698c140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c1442e0 0x7f698c140b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 -- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6974004d40 msgr2=0x7f698c14a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6974004d40 0x7f698c14a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 -- 172.21.15.113:0/4037877365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c147900 con 0x7f698c1436f0 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6989d9b640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698c1436f0 0x7f698c1405f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f69800115c0 tx=0x7f6980011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f697affd640 1 -- 172.21.15.113:0/4037877365 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004220 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f697affd640 1 -- 172.21.15.113:0/4037877365 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69800043c0 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c147ba0 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f697affd640 1 -- 172.21.15.113:0/4037877365 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980015870 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.532+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c148830 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.533+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c0691a0 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.534+0000 7f697affd640 1 -- 172.21.15.113:0/4037877365 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6980015a10 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.534+0000 7f697affd640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be30 0x7f696c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.534+0000 7f697affd640 1 -- 172.21.15.113:0/4037877365 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f698008afc0 con 0x7f698c1436f0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.534+0000 7f698a59c640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be30 0x7f696c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.535+0000 7f698a59c640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be30 0x7f696c06e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f697c014450 tx=0x7f697c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.538+0000 7f697affd640 1 -- 172.21.15.113:0/4037877365 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6980019030 con 0x7f698c1436f0 2024-04-25T00:00:42.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:42.792+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) -- 0x7f698c0b2900 con 0x7f698c1436f0 2024-04-25T00:00:43.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.107+0000 7f697affd640 1 -- 172.21.15.113:0/4037877365 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v126) ==== 148+0+0 (secure 0 0 0) 0x7f69800596b0 con 0x7f698c1436f0 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be30 msgr2=0x7f696c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be30 0x7f696c06e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f697c014450 tx=0x7f697c00b040 comp rx=0 tx=0).stop 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698c1436f0 msgr2=0x7f698c1405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698c1436f0 0x7f698c1405f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f69800115c0 tx=0x7f6980011a80 comp rx=0 tx=0).stop 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 shutdown_connections 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be30 0x7f696c06e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698c1442e0 0x7f698c140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698c1436f0 0x7f698c1405f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 --2- 172.21.15.113:0/4037877365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6974004d40 0x7f698c14a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 >> 172.21.15.113:0/4037877365 conn(0x7f698c0b1a50 msgr2=0x7f698c0b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 shutdown_connections 2024-04-25T00:00:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.109+0000 7f6990b5a640 1 -- 172.21.15.113:0/4037877365 wait complete. 2024-04-25T00:00:43.121 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T00:00:43.122 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T00:00:43.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:43.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.219+0000 7f872044d640 1 Processor -- start 2024-04-25T00:00:43.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.220+0000 7f872044d640 1 -- start start 2024-04-25T00:00:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.220+0000 7f872044d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8718144910 0x7f87180b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.220+0000 7f872044d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87180b5240 0x7f871806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.220+0000 7f872044d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8718144cf0 con 0x7f87180b56b0 2024-04-25T00:00:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.220+0000 7f872044d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f87180b4510 con 0x7f87180b5240 2024-04-25T00:00:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.220+0000 7f872044d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f87180b2f30 con 0x7f8718144910 2024-04-25T00:00:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871d9e8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87180b5240 0x7f871806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871d9e8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87180b5240 0x7f871806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52582/0 (socket says 172.21.15.113:52582) 2024-04-25T00:00:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871d9e8640 1 -- 172.21.15.113:0/535640612 learned_addr learned my addr 172.21.15.113:0/535640612 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871e1e9640 1 --2- 172.21.15.113:0/535640612 >> v2:172.21.15.178:3300/0 conn(0x7f8718144910 0x7f87180b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871d9e8640 1 -- 172.21.15.113:0/535640612 >> v2:172.21.15.178:3300/0 conn(0x7f8718144910 msgr2=0x7f87180b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871d9e8640 1 --2- 172.21.15.113:0/535640612 >> v2:172.21.15.178:3300/0 conn(0x7f8718144910 0x7f87180b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871d9e8640 1 -- 172.21.15.113:0/535640612 >> v1:172.21.15.178:6789/0 conn(0x7f87180b56b0 legacy=0x7f87180cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.221+0000 7f871d9e8640 1 -- 172.21.15.113:0/535640612 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87180b3210 con 0x7f87180b5240 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f871d9e8640 1 --2- 172.21.15.113:0/535640612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87180b5240 0x7f871806dfd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8708009870 tx=0x7f870802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ae5f89d74b8fae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f871d1e7640 1 -- 172.21.15.113:0/535640612 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708037070 con 0x7f87180b5240 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f871d1e7640 1 -- 172.21.15.113:0/535640612 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8708032440 con 0x7f87180b5240 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f871d1e7640 1 -- 172.21.15.113:0/535640612 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708032810 con 0x7f87180b5240 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f872044d640 1 -- 172.21.15.113:0/535640612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87180b5240 msgr2=0x7f871806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f872044d640 1 --2- 172.21.15.113:0/535640612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87180b5240 0x7f871806dfd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8708009870 tx=0x7f870802a2a0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f872044d640 1 -- 172.21.15.113:0/535640612 shutdown_connections 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f872044d640 1 --2- 172.21.15.113:0/535640612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87180b5240 0x7f871806dfd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f872044d640 1 --2- 172.21.15.113:0/535640612 >> v2:172.21.15.178:3300/0 conn(0x7f8718144910 0x7f87180b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f872044d640 1 -- 172.21.15.113:0/535640612 >> 172.21.15.113:0/535640612 conn(0x7f87180b1a50 msgr2=0x7f87180b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.222+0000 7f872044d640 1 -- 172.21.15.113:0/535640612 shutdown_connections 2024-04-25T00:00:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.223+0000 7f872044d640 1 -- 172.21.15.113:0/535640612 wait complete. 2024-04-25T00:00:43.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 Processor -- start 2024-04-25T00:00:43.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 -- start start 2024-04-25T00:00:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87180b5240 0x7f871815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87180b56b0 0x7f87181f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8718144910 0x7f87180b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87180b4da0 con 0x7f8718144910 2024-04-25T00:00:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87180b37d0 con 0x7f87180b5240 2024-04-25T00:00:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f872044d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87180b4f20 con 0x7f87180b56b0 2024-04-25T00:00:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f871e9ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8718144910 0x7f87180b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f871e9ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8718144910 0x7f87180b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52598/0 (socket says 172.21.15.113:52598) 2024-04-25T00:00:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.224+0000 7f871e9ea640 1 -- 172.21.15.113:0/1502796479 learned_addr learned my addr 172.21.15.113:0/1502796479 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e1e9640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87180b5240 0x7f871815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871d9e8640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87180b56b0 0x7f87181f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e9ea640 1 -- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87180b56b0 msgr2=0x7f87181f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e9ea640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87180b56b0 0x7f87181f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e9ea640 1 -- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87180b5240 msgr2=0x7f871815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e9ea640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87180b5240 0x7f871815a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e9ea640 1 -- 172.21.15.113:0/1502796479 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871815b000 con 0x7f8718144910 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e1e9640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87180b5240 0x7f871815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f871e9ea640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8718144910 0x7f87180b07e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f871400c3f0 tx=0x7f871400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f870effd640 1 -- 172.21.15.113:0/1502796479 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714017070 con 0x7f8718144910 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f870effd640 1 -- 172.21.15.113:0/1502796479 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87140052d0 con 0x7f8718144910 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871815b2e0 con 0x7f8718144910 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f870effd640 1 -- 172.21.15.113:0/1502796479 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714003830 con 0x7f8718144910 2024-04-25T00:00:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.225+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871815ac60 con 0x7f8718144910 2024-04-25T00:00:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.226+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87180691a0 con 0x7f8718144910 2024-04-25T00:00:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.226+0000 7f870effd640 1 -- 172.21.15.113:0/1502796479 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f871401a070 con 0x7f8718144910 2024-04-25T00:00:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.227+0000 7f870effd640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86fc06be60 0x7f86fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.227+0000 7f871e1e9640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86fc06be60 0x7f86fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.227+0000 7f870effd640 1 -- 172.21.15.113:0/1502796479 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f8714089df0 con 0x7f8718144910 2024-04-25T00:00:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.227+0000 7f871e1e9640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86fc06be60 0x7f86fc06e320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8710009fd0 tx=0x7f871000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.231+0000 7f870effd640 1 -- 172.21.15.113:0/1502796479 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8714058aa0 con 0x7f8718144910 2024-04-25T00:00:43.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.408+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87180b4970 con 0x7f8718144910 2024-04-25T00:00:43.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.409+0000 7f870effd640 1 -- 172.21.15.113:0/1502796479 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13070 (secure 0 0 0) 0x7f87140583a0 con 0x7f8718144910 2024-04-25T00:00:43.409 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:42.101304+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6837/3337939581":"2024-04-26T00:00:39.089148+0000","172.21.15.113:6835/3337939581":"2024-04-26T00:00:39.089148+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:43.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86fc06be60 msgr2=0x7f86fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86fc06be60 0x7f86fc06e320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8710009fd0 tx=0x7f871000b040 comp rx=0 tx=0).stop 2024-04-25T00:00:43.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8718144910 msgr2=0x7f87180b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8718144910 0x7f87180b07e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f871400c3f0 tx=0x7f871400c8b0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 shutdown_connections 2024-04-25T00:00:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86fc06be60 0x7f86fc06e320 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8718144910 0x7f87180b07e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87180b56b0 0x7f87181f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 --2- 172.21.15.113:0/1502796479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87180b5240 0x7f871815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 >> 172.21.15.113:0/1502796479 conn(0x7f87180b1a50 msgr2=0x7f87180cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 shutdown_connections 2024-04-25T00:00:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.411+0000 7f872044d640 1 -- 172.21.15.113:0/1502796479 wait complete. 2024-04-25T00:00:43.422 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:00:43.423 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:00:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.522+0000 7f3174b1d640 1 Processor -- start 2024-04-25T00:00:43.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.522+0000 7f3174b1d640 1 -- start start 2024-04-25T00:00:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f3174b1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f317015e510 0x7f31700d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f3174b1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700d28c0 0x7f31700b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f3174b1d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31700d2e00 con 0x7f31700b61e0 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f3174b1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f31700d30e0 con 0x7f317015e510 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f3174b1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31700d33c0 con 0x7f31700d28c0 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f316e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f317015e510 0x7f31700d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f316ed9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f31700b61e0 0x7f3170071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53056/0 (socket says 172.21.15.113:53056) 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f316e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f317015e510 0x7f31700d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52604/0 (socket says 172.21.15.113:52604) 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.523+0000 7f316ed9d640 1 -- 172.21.15.113:0/3729915834 learned_addr learned my addr 172.21.15.113:0/3729915834 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.524+0000 7f316dd9b640 1 --2- 172.21.15.113:0/3729915834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700d28c0 0x7f31700b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.524+0000 7f316e59c640 1 -- 172.21.15.113:0/3729915834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700d28c0 msgr2=0x7f31700b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.524+0000 7f316e59c640 1 --2- 172.21.15.113:0/3729915834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700d28c0 0x7f31700b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.524+0000 7f316e59c640 1 -- 172.21.15.113:0/3729915834 >> v1:172.21.15.113:6789/0 conn(0x7f31700b61e0 legacy=0x7f3170071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.524+0000 7f316e59c640 1 -- 172.21.15.113:0/3729915834 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31700d36a0 con 0x7f317015e510 2024-04-25T00:00:43.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.524+0000 7f316e59c640 1 --2- 172.21.15.113:0/3729915834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317015e510 0x7f31700d2380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdbf7bd275e6431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.524+0000 7f316d59a640 1 -- 172.21.15.113:0/3729915834 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160031600 con 0x7f317015e510 2024-04-25T00:00:43.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f316d59a640 1 -- 172.21.15.113:0/3729915834 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31600317a0 con 0x7f317015e510 2024-04-25T00:00:43.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f316d59a640 1 -- 172.21.15.113:0/3729915834 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160031b70 con 0x7f317015e510 2024-04-25T00:00:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 -- 172.21.15.113:0/3729915834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317015e510 msgr2=0x7f31700d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 --2- 172.21.15.113:0/3729915834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317015e510 0x7f31700d2380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316002a5d0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 -- 172.21.15.113:0/3729915834 shutdown_connections 2024-04-25T00:00:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 --2- 172.21.15.113:0/3729915834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700d28c0 0x7f31700b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 --2- 172.21.15.113:0/3729915834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f317015e510 0x7f31700d2380 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 -- 172.21.15.113:0/3729915834 >> 172.21.15.113:0/3729915834 conn(0x7f317006b720 msgr2=0x7f31700d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 -- 172.21.15.113:0/3729915834 shutdown_connections 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.525+0000 7f3174b1d640 1 -- 172.21.15.113:0/3729915834 wait complete. 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.526+0000 7f3174b1d640 1 Processor -- start 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f3174b1d640 1 -- start start 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f3174b1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700b61e0 0x7f31700c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f3174b1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31700d28c0 0x7f31701e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f3174b1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317015e510 0x7f31700cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f3174b1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f31700b9dd0 con 0x7f31700d28c0 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f3174b1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31700d3ea0 con 0x7f317015e510 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f3174b1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31700b83d0 con 0x7f31700b61e0 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31700d28c0 0x7f31701e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31700d28c0 0x7f31701e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52614/0 (socket says 172.21.15.113:52614) 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f316dd9b640 1 -- 172.21.15.113:0/2621409275 learned_addr learned my addr 172.21.15.113:0/2621409275 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f316ed9d640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317015e510 0x7f31700cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.527+0000 7f316e59c640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700b61e0 0x7f31700c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f316dd9b640 1 -- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700b61e0 msgr2=0x7f31700c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f316dd9b640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700b61e0 0x7f31700c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f316dd9b640 1 -- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317015e510 msgr2=0x7f31700cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f316dd9b640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317015e510 0x7f31700cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f316dd9b640 1 -- 172.21.15.113:0/2621409275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31701e7680 con 0x7f31700d28c0 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f316dd9b640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31700d28c0 0x7f31701e6b60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f316400c7f0 tx=0x7f316400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f315effd640 1 -- 172.21.15.113:0/2621409275 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164015070 con 0x7f31700d28c0 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f315effd640 1 -- 172.21.15.113:0/2621409275 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3164004240 con 0x7f31700d28c0 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31701e7960 con 0x7f31700d28c0 2024-04-25T00:00:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f315effd640 1 -- 172.21.15.113:0/2621409275 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164007620 con 0x7f31700d28c0 2024-04-25T00:00:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.528+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31700cd640 con 0x7f31700d28c0 2024-04-25T00:00:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.529+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31700cd840 con 0x7f31700d28c0 2024-04-25T00:00:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.529+0000 7f315effd640 1 -- 172.21.15.113:0/2621409275 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31640079d0 con 0x7f31700d28c0 2024-04-25T00:00:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.530+0000 7f315effd640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315006be30 0x7f315006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.530+0000 7f316e59c640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315006be30 0x7f315006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.530+0000 7f315effd640 1 -- 172.21.15.113:0/2621409275 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f316408ae70 con 0x7f31700d28c0 2024-04-25T00:00:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.530+0000 7f316e59c640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315006be30 0x7f315006e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3160034000 tx=0x7f316002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.534+0000 7f315effd640 1 -- 172.21.15.113:0/2621409275 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31700cd840 con 0x7f31700d28c0 2024-04-25T00:00:43.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.709+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31700691a0 con 0x7f31700d28c0 2024-04-25T00:00:43.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.710+0000 7f315effd640 1 -- 172.21.15.113:0/2621409275 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f3164059560 con 0x7f31700d28c0 2024-04-25T00:00:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:00:43.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315006be30 msgr2=0x7f315006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315006be30 0x7f315006e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3160034000 tx=0x7f316002f000 comp rx=0 tx=0).stop 2024-04-25T00:00:43.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31700d28c0 msgr2=0x7f31701e6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31700d28c0 0x7f31701e6b60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f316400c7f0 tx=0x7f316400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 shutdown_connections 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315006be30 0x7f315006e2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f317015e510 0x7f31700cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31700d28c0 0x7f31701e6b60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 --2- 172.21.15.113:0/2621409275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31700b61e0 0x7f31700c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 >> 172.21.15.113:0/2621409275 conn(0x7f317006b720 msgr2=0x7f31700d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 shutdown_connections 2024-04-25T00:00:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.712+0000 7f3174b1d640 1 -- 172.21.15.113:0/2621409275 wait complete. 2024-04-25T00:00:43.723 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T00:00:43.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.819+0000 7f9c2465d640 1 Processor -- start 2024-04-25T00:00:43.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.819+0000 7f9c2465d640 1 -- start start 2024-04-25T00:00:43.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c2465d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c1c06e6e0 0x7f9c1c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c2465d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c1c144cf0 con 0x7f9c1c144910 2024-04-25T00:00:43.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c2465d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c1c0b2f30 con 0x7f9c1c0b3fd0 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c2465d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9c1c0b3210 con 0x7f9c1c06e6e0 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c22bfa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c1c06e6e0 0x7f9c1c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c22bfa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c1c06e6e0 0x7f9c1c0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52624/0 (socket says 172.21.15.113:52624) 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c22bfa640 1 -- 172.21.15.113:0/1278342006 learned_addr learned my addr 172.21.15.113:0/1278342006 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c223f9640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9c1c144910 0x7f9c1c0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53066/0 (socket says 172.21.15.113:53066) 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c22bfa640 1 -- 172.21.15.113:0/1278342006 >> v1:172.21.15.178:6789/0 conn(0x7f9c1c0b3fd0 legacy=0x7f9c1c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c22bfa640 1 -- 172.21.15.113:0/1278342006 >> v1:172.21.15.113:6789/0 conn(0x7f9c1c144910 legacy=0x7f9c1c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c22bfa640 1 -- 172.21.15.113:0/1278342006 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c0b34f0 con 0x7f9c1c06e6e0 2024-04-25T00:00:43.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.820+0000 7f9c213f7640 1 -- 172.21.15.113:0/1278342006 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141198766 0 0) 0x7f9c1c144cf0 con 0x7f9c1c144910 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.821+0000 7f9c22bfa640 1 --2- 172.21.15.113:0/1278342006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c06e6e0 0x7f9c1c0cd500 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009e80 tx=0x7f9c1402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafa763036e4cf19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.821+0000 7f9c213f7640 1 -- 172.21.15.113:0/1278342006 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14036070 con 0x7f9c1c06e6e0 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.821+0000 7f9c213f7640 1 -- 172.21.15.113:0/1278342006 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c14031600 con 0x7f9c1c06e6e0 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.821+0000 7f9c213f7640 1 -- 172.21.15.113:0/1278342006 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c140319d0 con 0x7f9c1c06e6e0 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.822+0000 7f9c2465d640 1 -- 172.21.15.113:0/1278342006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c06e6e0 msgr2=0x7f9c1c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.822+0000 7f9c2465d640 1 --2- 172.21.15.113:0/1278342006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c06e6e0 0x7f9c1c0cd500 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009e80 tx=0x7f9c1402a7a0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.822+0000 7f9c2465d640 1 -- 172.21.15.113:0/1278342006 shutdown_connections 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.822+0000 7f9c2465d640 1 --2- 172.21.15.113:0/1278342006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c06e6e0 0x7f9c1c0cd500 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.822+0000 7f9c2465d640 1 -- 172.21.15.113:0/1278342006 >> 172.21.15.113:0/1278342006 conn(0x7f9c1c0b1a50 msgr2=0x7f9c1c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.822+0000 7f9c2465d640 1 -- 172.21.15.113:0/1278342006 shutdown_connections 2024-04-25T00:00:43.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.822+0000 7f9c2465d640 1 -- 172.21.15.113:0/1278342006 wait complete. 2024-04-25T00:00:43.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.823+0000 7f9c2465d640 1 Processor -- start 2024-04-25T00:00:43.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.823+0000 7f9c2465d640 1 -- start start 2024-04-25T00:00:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.823+0000 7f9c2465d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c06e6e0 0x7f9c1c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c2465d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0b3fd0 0x7f9c1c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c2465d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c144910 0x7f9c1c135060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c2465d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c0b4da0 con 0x7f9c1c144910 2024-04-25T00:00:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c2465d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c0b4440 con 0x7f9c1c0b3fd0 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c2465d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1c0b4f20 con 0x7f9c1c06e6e0 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c144910 0x7f9c1c135060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c144910 0x7f9c1c135060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52636/0 (socket says 172.21.15.113:52636) 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 -- 172.21.15.113:0/112225637 learned_addr learned my addr 172.21.15.113:0/112225637 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c21bf8640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0b3fd0 0x7f9c1c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c223f9640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c06e6e0 0x7f9c1c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 -- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c06e6e0 msgr2=0x7f9c1c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c06e6e0 0x7f9c1c0cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 -- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0b3fd0 msgr2=0x7f9c1c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0b3fd0 0x7f9c1c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 -- 172.21.15.113:0/112225637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c12f820 con 0x7f9c1c144910 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c21bf8640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0b3fd0 0x7f9c1c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c22bfa640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c144910 0x7f9c1c135060 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009fa0 tx=0x7f9c14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c12ffd640 1 -- 172.21.15.113:0/112225637 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14036040 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c12ffd640 1 -- 172.21.15.113:0/112225637 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c14003990 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c12fb00 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c12ffd640 1 -- 172.21.15.113:0/112225637 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14009510 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.824+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c1300c0 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.825+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c0691a0 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.826+0000 7f9c12ffd640 1 -- 172.21.15.113:0/112225637 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c140077a0 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.826+0000 7f9c12ffd640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c0006be60 0x7f9c0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.826+0000 7f9c12ffd640 1 -- 172.21.15.113:0/112225637 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f9c140abc70 con 0x7f9c1c144910 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.826+0000 7f9c223f9640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c0006be60 0x7f9c0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.827+0000 7f9c223f9640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c0006be60 0x7f9c0006e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c18000f60 tx=0x7f9c18009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:43.830+0000 7f9c12ffd640 1 -- 172.21.15.113:0/112225637 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c1407aa70 con 0x7f9c1c144910 2024-04-25T00:00:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.069+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) -- 0x7f9c1c0731a0 con 0x7f9c1c144910 2024-04-25T00:00:44.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.114+0000 7f9c12ffd640 1 -- 172.21.15.113:0/112225637 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v127) ==== 146+0+0 (secure 0 0 0) 0x7f9c1407a370 con 0x7f9c1c144910 2024-04-25T00:00:44.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c0006be60 msgr2=0x7f9c0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c0006be60 0x7f9c0006e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c18000f60 tx=0x7f9c18009040 comp rx=0 tx=0).stop 2024-04-25T00:00:44.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c144910 msgr2=0x7f9c1c135060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c144910 0x7f9c1c135060 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009fa0 tx=0x7f9c14004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 shutdown_connections 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c0006be60 0x7f9c0006e320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0b3fd0 0x7f9c1c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1c144910 0x7f9c1c135060 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 --2- 172.21.15.113:0/112225637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c06e6e0 0x7f9c1c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 >> 172.21.15.113:0/112225637 conn(0x7f9c1c0b1a50 msgr2=0x7f9c1c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:44.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 shutdown_connections 2024-04-25T00:00:44.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.116+0000 7f9c2465d640 1 -- 172.21.15.113:0/112225637 wait complete. 2024-04-25T00:00:44.127 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:00:44.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.226+0000 7f26c7f90640 1 Processor -- start 2024-04-25T00:00:44.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.227+0000 7f26c7f90640 1 -- start start 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.227+0000 7f26c7f90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26c015e2d0 0x7f26c015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.227+0000 7f26c7f90640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f26c00b3080 0x7f26c00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.227+0000 7f26c7f90640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c015f4f0 con 0x7f26c015e6b0 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.227+0000 7f26c7f90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f26c00cde00 con 0x7f26c015e2d0 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.227+0000 7f26c7f90640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f26c015f7d0 con 0x7f26c00b3080 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c5d2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26c015e2d0 0x7f26c015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c5d2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26c015e2d0 0x7f26c015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52638/0 (socket says 172.21.15.113:52638) 2024-04-25T00:00:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c5d2c640 1 -- 172.21.15.113:0/3211110063 learned_addr learned my addr 172.21.15.113:0/3211110063 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:44.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c552b640 1 --2- 172.21.15.113:0/3211110063 >> v2:172.21.15.178:3300/0 conn(0x7f26c00b3080 0x7f26c00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c5d2c640 1 -- 172.21.15.113:0/3211110063 >> v2:172.21.15.178:3300/0 conn(0x7f26c00b3080 msgr2=0x7f26c00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:44.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c5d2c640 1 --2- 172.21.15.113:0/3211110063 >> v2:172.21.15.178:3300/0 conn(0x7f26c00b3080 0x7f26c00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c5d2c640 1 -- 172.21.15.113:0/3211110063 >> v1:172.21.15.178:6789/0 conn(0x7f26c015e6b0 legacy=0x7f26c0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:44.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.228+0000 7f26c5d2c640 1 -- 172.21.15.113:0/3211110063 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c015fab0 con 0x7f26c015e2d0 2024-04-25T00:00:44.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c5d2c640 1 --2- 172.21.15.113:0/3211110063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c015ebf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26c0070ff0 tx=0x7f26b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2d0302378de846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:44.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c4d2a640 1 -- 172.21.15.113:0/3211110063 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0037070 con 0x7f26c015e2d0 2024-04-25T00:00:44.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c4d2a640 1 -- 172.21.15.113:0/3211110063 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b0032440 con 0x7f26c015e2d0 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c4d2a640 1 -- 172.21.15.113:0/3211110063 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0032810 con 0x7f26c015e2d0 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 -- 172.21.15.113:0/3211110063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 msgr2=0x7f26c015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 --2- 172.21.15.113:0/3211110063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c015ebf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26c0070ff0 tx=0x7f26b002a230 comp rx=0 tx=0).stop 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 -- 172.21.15.113:0/3211110063 shutdown_connections 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 --2- 172.21.15.113:0/3211110063 >> v2:172.21.15.178:3300/0 conn(0x7f26c00b3080 0x7f26c00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 --2- 172.21.15.113:0/3211110063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c015ebf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 -- 172.21.15.113:0/3211110063 >> 172.21.15.113:0/3211110063 conn(0x7f26c006b0f0 msgr2=0x7f26c006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 -- 172.21.15.113:0/3211110063 shutdown_connections 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.229+0000 7f26c7f90640 1 -- 172.21.15.113:0/3211110063 wait complete. 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.230+0000 7f26c7f90640 1 Processor -- start 2024-04-25T00:00:44.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.230+0000 7f26c7f90640 1 -- start start 2024-04-25T00:00:44.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c7f90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c00b3080 0x7f26c014e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c7f90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c7f90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c015e6b0 0x7f26c006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c7f90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f26c015f2b0 con 0x7f26c015e2d0 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c7f90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26c015f130 con 0x7f26c015e6b0 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c7f90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f26c00cf2a0 con 0x7f26c00b3080 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c00cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52646/0 (socket says 172.21.15.113:52646) 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 -- 172.21.15.113:0/2626794508 learned_addr learned my addr 172.21.15.113:0/2626794508 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 -- 172.21.15.113:0/2626794508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c00b3080 msgr2=0x7f26c014e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c00b3080 0x7f26c014e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 -- 172.21.15.113:0/2626794508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c015e6b0 msgr2=0x7f26c006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c015e6b0 0x7f26c006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.231+0000 7f26c552b640 1 -- 172.21.15.113:0/2626794508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c0155af0 con 0x7f26c015e2d0 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.232+0000 7f26c552b640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c00cc200 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00d800 tx=0x7f26bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.232+0000 7f26b67fc640 1 -- 172.21.15.113:0/2626794508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00b6c0 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.232+0000 7f26b67fc640 1 -- 172.21.15.113:0/2626794508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26bc019070 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.232+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c0155dd0 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.232+0000 7f26b67fc640 1 -- 172.21.15.113:0/2626794508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00ba80 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.232+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c0156390 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.233+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c0150fc0 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.233+0000 7f26b67fc640 1 -- 172.21.15.113:0/2626794508 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26bc00bc20 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.233+0000 7f26b67fc640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f26a406be30 0x7f26a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.234+0000 7f26b67fc640 1 -- 172.21.15.113:0/2626794508 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f26bc08b010 con 0x7f26c015e2d0 2024-04-25T00:00:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.234+0000 7f26c5d2c640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f26a406be30 0x7f26a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.234+0000 7f26c5d2c640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f26a406be30 0x7f26a406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26b0034000 tx=0x7f26b002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:44.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.238+0000 7f26b67fc640 1 -- 172.21.15.113:0/2626794508 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f26bc059cc0 con 0x7f26c015e2d0 2024-04-25T00:00:44.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.416+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f26c0072050 con 0x7f26c015e2d0 2024-04-25T00:00:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.422+0000 7f26b67fc640 1 -- 172.21.15.113:0/2626794508 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f26bc0595c0 con 0x7f26c015e2d0 2024-04-25T00:00:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f26a406be30 msgr2=0x7f26a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f26a406be30 0x7f26a406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26b0034000 tx=0x7f26b002c000 comp rx=0 tx=0).stop 2024-04-25T00:00:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 msgr2=0x7f26c00cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c00cc200 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00d800 tx=0x7f26bc00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 shutdown_connections 2024-04-25T00:00:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f26a406be30 0x7f26a406e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c015e6b0 0x7f26c006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c015e2d0 0x7f26c00cc200 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 --2- 172.21.15.113:0/2626794508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c00b3080 0x7f26c014e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 >> 172.21.15.113:0/2626794508 conn(0x7f26c006b0f0 msgr2=0x7f26c00d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 shutdown_connections 2024-04-25T00:00:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.424+0000 7f26c7f90640 1 -- 172.21.15.113:0/2626794508 wait complete. 2024-04-25T00:00:44.435 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:00:44.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:00:44.468 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:00:44.508 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:00:44.508 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:00:44.509 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:44.523 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:00:44.523 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:00:44.523 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:44.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:00:44.568 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:00:44.568 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:44.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:00:44.582 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:00:44.583 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:44.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:00:44.625 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:00:44.626 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:44.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.721+0000 7fa300c8c640 1 Processor -- start 2024-04-25T00:00:44.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.721+0000 7fa300c8c640 1 -- start start 2024-04-25T00:00:44.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.722+0000 7fa300c8c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa2fc072ea0 0x7fa2fc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.722+0000 7fa300c8c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2fc0b2d50 con 0x7fa2fc0b3af0 2024-04-25T00:00:44.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.722+0000 7fa300c8c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2fc072a70 con 0x7fa2fc15a6b0 2024-04-25T00:00:44.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.722+0000 7fa300c8c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa2fc0b3010 con 0x7fa2fc072ea0 2024-04-25T00:00:44.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.722+0000 7fa2fad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa2fc0b3af0 0x7fa2fc0b3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53082/0 (socket says 172.21.15.113:53082) 2024-04-25T00:00:44.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.722+0000 7fa2fad9d640 1 -- 172.21.15.113:0/1348995296 learned_addr learned my addr 172.21.15.113:0/1348995296 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:44.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463820614 0 0) 0x7fa2fc0b2d50 con 0x7fa2fc0b3af0 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2e4003520 con 0x7fa2fc0b3af0 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/1348995296 >> v2:172.21.15.178:3300/0 conn(0x7fa2fc072ea0 0x7fa2fc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3180385843 0 0) 0x7fa2e4003520 con 0x7fa2fc0b3af0 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 >> v2:172.21.15.178:3300/0 conn(0x7fa2fc072ea0 msgr2=0x7fa2fc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 --2- 172.21.15.113:0/1348995296 >> v2:172.21.15.178:3300/0 conn(0x7fa2fc072ea0 0x7fa2fc071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 >> v1:172.21.15.178:6789/0 conn(0x7fa2fc15a6b0 legacy=0x7fa2fc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc0b32f0 con 0x7fa2fc0b3af0 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fa2f4001670 con 0x7fa2fc0b3af0 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 >> v1:172.21.15.113:6789/0 conn(0x7fa2fc0b3af0 legacy=0x7fa2fc0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e40039b0 0x7fa2e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e40051c0 0x7fa2e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc0b3010 con 0x7fa2e4004800 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e400c110 con 0x7fa2e40051c0 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e400c310 con 0x7fa2e40039b0 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2fad9d640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e40051c0 0x7fa2e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2fa59c640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e40039b0 0x7fa2e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f9d9b640 1 -- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e40039b0 msgr2=0x7fa2e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e40039b0 0x7fa2e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f9d9b640 1 -- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e40051c0 msgr2=0x7fa2e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e40051c0 0x7fa2e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2f9d9b640 1 -- 172.21.15.113:0/1348995296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e4003520 con 0x7fa2e4004800 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2fad9d640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e40051c0 0x7fa2e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:44.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.723+0000 7fa2fad9d640 1 -- 172.21.15.113:0/1348995296 reap_dead start 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.724+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000ba30 tx=0x7fa2e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7612844f9e8b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.724+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0030070 con 0x7fa2e4004800 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.724+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f002ade0 con 0x7fa2e4004800 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.724+0000 7fa2f959a640 1 -- 172.21.15.113:0/1348995296 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000b600 con 0x7fa2e4004800 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.724+0000 7fa300c8c640 1 -- 172.21.15.113:0/1348995296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e4004800 msgr2=0x7fa2e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.724+0000 7fa300c8c640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000ba30 tx=0x7fa2e4005830 comp rx=0 tx=0).stop 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.725+0000 7fa300c8c640 1 -- 172.21.15.113:0/1348995296 shutdown_connections 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.725+0000 7fa300c8c640 1 --2- 172.21.15.113:0/1348995296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.725+0000 7fa300c8c640 1 -- 172.21.15.113:0/1348995296 >> 172.21.15.113:0/1348995296 conn(0x7fa2fc0b1a50 msgr2=0x7fa2fc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.725+0000 7fa300c8c640 1 -- 172.21.15.113:0/1348995296 shutdown_connections 2024-04-25T00:00:44.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.725+0000 7fa300c8c640 1 -- 172.21.15.113:0/1348995296 wait complete. 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.726+0000 7fa300c8c640 1 Processor -- start 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.726+0000 7fa300c8c640 1 -- start start 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e4004800 0x7fa2fc167570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc146590 0x7fa2fc147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2fc0ce710 0x7fa2fc15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc15aa90 con 0x7fa2fc146590 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc072720 con 0x7fa2e4004800 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa2fc15ac10 con 0x7fa2fc0ce710 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc146590 0x7fa2fc147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc146590 0x7fa2fc147460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52674/0 (socket says 172.21.15.113:52674) 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 -- 172.21.15.113:0/3736690881 learned_addr learned my addr 172.21.15.113:0/3736690881 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:44.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2fa59c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e4004800 0x7fa2fc167570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 -- 172.21.15.113:0/3736690881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2fc0ce710 msgr2=0x7fa2fc15cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2fc0ce710 0x7fa2fc15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 -- 172.21.15.113:0/3736690881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e4004800 msgr2=0x7fa2fc167570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e4004800 0x7fa2fc167570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 -- 172.21.15.113:0/3736690881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc1401a0 con 0x7fa2fc146590 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2f9d9b640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc146590 0x7fa2fc147460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa2f002a8d0 tx=0x7fa2f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2eaffd640 1 -- 172.21.15.113:0/3736690881 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0043030 con 0x7fa2fc146590 2024-04-25T00:00:44.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2eaffd640 1 -- 172.21.15.113:0/3736690881 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f0035e90 con 0x7fa2fc146590 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa2eaffd640 1 -- 172.21.15.113:0/3736690881 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000b600 con 0x7fa2fc146590 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc140480 con 0x7fa2fc146590 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.727+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc1479a0 con 0x7fa2fc146590 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.728+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2fc0691a0 con 0x7fa2fc146590 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.729+0000 7fa2eaffd640 1 -- 172.21.15.113:0/3736690881 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2f0046070 con 0x7fa2fc146590 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.729+0000 7fa2eaffd640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2dc06be60 0x7fa2dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.729+0000 7fa2eaffd640 1 -- 172.21.15.113:0/3736690881 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fa2f00ab250 con 0x7fa2fc146590 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.729+0000 7fa2fa59c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2dc06be60 0x7fa2dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:44.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.730+0000 7fa2fa59c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2dc06be60 0x7fa2dc06e320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec007fd0 tx=0x7fa2ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:44.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.733+0000 7fa2eaffd640 1 -- 172.21.15.113:0/3736690881 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa2f004d070 con 0x7fa2fc146590 2024-04-25T00:00:44.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.944+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2fc0b2900 con 0x7fa2fc146590 2024-04-25T00:00:44.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.945+0000 7fa2eaffd640 1 -- 172.21.15.113:0/3736690881 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+3881 (secure 0 0 0) 0x7fa2f0079f00 con 0x7fa2fc146590 2024-04-25T00:00:44.945 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2613614055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2613614055},{"type":"v1","addr":"172.21.15.178:6834","nonce":2613614055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2494825991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2494825991},{"type":"v1","addr":"172.21.15.178:6835","nonce":2494825991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2843749120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2843749120},{"type":"v1","addr":"172.21.15.113:6836","nonce":2843749120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3942092559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3942092559},{"type":"v1","addr":"172.21.15.113:6837","nonce":3942092559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[]} 2024-04-25T00:00:44.945 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 17 2024-04-25T00:00:44.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2dc06be60 msgr2=0x7fa2dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2dc06be60 0x7fa2dc06e320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec007fd0 tx=0x7fa2ec009040 comp rx=0 tx=0).stop 2024-04-25T00:00:44.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc146590 msgr2=0x7fa2fc147460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:44.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc146590 0x7fa2fc147460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa2f002a8d0 tx=0x7fa2f00046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 shutdown_connections 2024-04-25T00:00:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2dc06be60 0x7fa2dc06e320 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2fc0ce710 0x7fa2fc15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc146590 0x7fa2fc147460 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 --2- 172.21.15.113:0/3736690881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e4004800 0x7fa2fc167570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 >> 172.21.15.113:0/3736690881 conn(0x7fa2fc0b1a50 msgr2=0x7fa2fc0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 shutdown_connections 2024-04-25T00:00:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:44.947+0000 7fa300c8c640 1 -- 172.21.15.113:0/3736690881 wait complete. 2024-04-25T00:00:44.958 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:00:44.959 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:00:44.959 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:00:44.959 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:00:44.959 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:00:44.959 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:00:44.959 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:00:44.959 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:00:44.960 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:00:44.960 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:00:44.960 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:00:44.960 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:00:44.960 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:00:44.960 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:00:44.960 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:00:44.960 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:00:44.971 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:00:44.971 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:00:44.972 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:00:44.972 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:00:44.973 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:00:44.973 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:00:44.974 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:00:44.974 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:00:45.013 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:00:45.014 DEBUG:teuthology.parallel:result is None 2024-04-25T00:00:45.015 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:00:45.015 DEBUG:teuthology.parallel:result is None 2024-04-25T00:00:45.016 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:00:45.016 DEBUG:teuthology.parallel:result is None 2024-04-25T00:00:45.017 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:00:45.017 DEBUG:teuthology.parallel:result is None 2024-04-25T00:00:45.017 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:00:45.085 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:00:45.085 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:00:45.098 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:00:45.098 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:00:45.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.120+0000 7ff10c7e7640 1 Processor -- start 2024-04-25T00:00:45.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.120+0000 7ff10c7e7640 1 -- start start 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.121+0000 7ff10c7e7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1040b60a0 0x7ff1040cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.121+0000 7ff10c7e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff10406ddb0 con 0x7ff1041566f0 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.121+0000 7ff10c7e7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1041606e0 con 0x7ff10415fcd0 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.121+0000 7ff10c7e7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff104160ca0 con 0x7ff1040b60a0 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.121+0000 7ff10a583640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff1041566f0 0x7ff10406e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53094/0 (socket says 172.21.15.113:53094) 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.121+0000 7ff10a583640 1 -- 172.21.15.113:0/3380425606 learned_addr learned my addr 172.21.15.113:0/3380425606 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.121+0000 7ff10ad84640 1 --2- 172.21.15.113:0/3380425606 >> v2:172.21.15.178:3300/0 conn(0x7ff1040b60a0 0x7ff1040cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:45.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.122+0000 7ff10ad84640 1 -- 172.21.15.113:0/3380425606 >> v1:172.21.15.178:6789/0 conn(0x7ff10415fcd0 legacy=0x7ff1040b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:45.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.122+0000 7ff10ad84640 1 -- 172.21.15.113:0/3380425606 >> v1:172.21.15.113:6789/0 conn(0x7ff1041566f0 legacy=0x7ff10406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:45.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.122+0000 7ff10ad84640 1 -- 172.21.15.113:0/3380425606 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1041609c0 con 0x7ff1040b60a0 2024-04-25T00:00:45.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.122+0000 7ff109581640 1 -- 172.21.15.113:0/3380425606 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184866668 0 0) 0x7ff1041606e0 con 0x7ff10415fcd0 2024-04-25T00:00:45.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.123+0000 7ff10ad84640 1 --2- 172.21.15.113:0/3380425606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 0x7ff1040cd9a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009d90 tx=0x7ff0fc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d50792e076c999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:45.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.123+0000 7ff109581640 1 -- 172.21.15.113:0/3380425606 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc007740 con 0x7ff1040b60a0 2024-04-25T00:00:45.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.123+0000 7ff109581640 1 -- 172.21.15.113:0/3380425606 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0fc0078e0 con 0x7ff1040b60a0 2024-04-25T00:00:45.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.123+0000 7ff109581640 1 -- 172.21.15.113:0/3380425606 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc007cb0 con 0x7ff1040b60a0 2024-04-25T00:00:45.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.123+0000 7ff10c7e7640 1 -- 172.21.15.113:0/3380425606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 msgr2=0x7ff1040cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:45.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.123+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/3380425606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 0x7ff1040cd9a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009d90 tx=0x7ff0fc02a6f0 comp rx=0 tx=0).stop 2024-04-25T00:00:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.124+0000 7ff10c7e7640 1 -- 172.21.15.113:0/3380425606 shutdown_connections 2024-04-25T00:00:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.124+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/3380425606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 0x7ff1040cd9a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.124+0000 7ff10c7e7640 1 -- 172.21.15.113:0/3380425606 >> 172.21.15.113:0/3380425606 conn(0x7ff10406b0f0 msgr2=0x7ff10406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.124+0000 7ff10c7e7640 1 -- 172.21.15.113:0/3380425606 shutdown_connections 2024-04-25T00:00:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.124+0000 7ff10c7e7640 1 -- 172.21.15.113:0/3380425606 wait complete. 2024-04-25T00:00:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.125+0000 7ff10c7e7640 1 Processor -- start 2024-04-25T00:00:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.125+0000 7ff10c7e7640 1 -- start start 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10c7e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 0x7ff104153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10c7e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1041566f0 0x7ff10415c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10c7e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10415fcd0 0x7ff10415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10c7e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040d0240 con 0x7ff1041566f0 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10c7e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff104156ad0 con 0x7ff1040b60a0 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10c7e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff1040d03c0 con 0x7ff10415fcd0 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff109d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1041566f0 0x7ff10415c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff109d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1041566f0 0x7ff10415c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52710/0 (socket says 172.21.15.113:52710) 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff109d82640 1 -- 172.21.15.113:0/4056023649 learned_addr learned my addr 172.21.15.113:0/4056023649 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10a583640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 0x7ff104153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.126+0000 7ff10ad84640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10415fcd0 0x7ff10415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff109d82640 1 -- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10415fcd0 msgr2=0x7ff10415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff109d82640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10415fcd0 0x7ff10415cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff109d82640 1 -- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 msgr2=0x7ff104153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff109d82640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 0x7ff104153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff109d82640 1 -- 172.21.15.113:0/4056023649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1041555d0 con 0x7ff1041566f0 2024-04-25T00:00:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff109d82640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1041566f0 0x7ff10415c750 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c410 tx=0x7ff0f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff0faffd640 1 -- 172.21.15.113:0/4056023649 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4017070 con 0x7ff1041566f0 2024-04-25T00:00:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff0faffd640 1 -- 172.21.15.113:0/4056023649 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f40035f0 con 0x7ff1041566f0 2024-04-25T00:00:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff104154530 con 0x7ff1041566f0 2024-04-25T00:00:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff0faffd640 1 -- 172.21.15.113:0/4056023649 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40039e0 con 0x7ff1041566f0 2024-04-25T00:00:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.127+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff104154af0 con 0x7ff1041566f0 2024-04-25T00:00:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.128+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10406c080 con 0x7ff1041566f0 2024-04-25T00:00:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.129+0000 7ff0faffd640 1 -- 172.21.15.113:0/4056023649 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0f401a070 con 0x7ff1041566f0 2024-04-25T00:00:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.130+0000 7ff0faffd640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0e806be60 0x7ff0e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.130+0000 7ff10a583640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0e806be60 0x7ff0e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.130+0000 7ff10a583640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0e806be60 0x7ff0e806e320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff100000f60 tx=0x7ff100007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.131+0000 7ff0faffd640 1 -- 172.21.15.113:0/4056023649 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7ff0f40565a0 con 0x7ff1041566f0 2024-04-25T00:00:45.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.136+0000 7ff0faffd640 1 -- 172.21.15.113:0/4056023649 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0f4058500 con 0x7ff1041566f0 2024-04-25T00:00:45.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:45.323+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff104068220 con 0x7ff1041566f0 2024-04-25T00:00:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.140+0000 7ff0faffd640 1 -- 172.21.15.113:0/4056023649 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) ==== 97+0+0 (secure 0 0 0) 0x7ff0f405c800 con 0x7ff1041566f0 2024-04-25T00:00:46.140 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0e806be60 msgr2=0x7ff0e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0e806be60 0x7ff0e806e320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff100000f60 tx=0x7ff100007e80 comp rx=0 tx=0).stop 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1041566f0 msgr2=0x7ff10415c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1041566f0 0x7ff10415c750 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c410 tx=0x7ff0f400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 shutdown_connections 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0e806be60 0x7ff0e806e320 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10415fcd0 0x7ff10415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1041566f0 0x7ff10415c750 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 --2- 172.21.15.113:0/4056023649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040b60a0 0x7ff104153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 >> 172.21.15.113:0/4056023649 conn(0x7ff10406b0f0 msgr2=0x7ff1040b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 shutdown_connections 2024-04-25T00:00:46.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.144+0000 7ff10c7e7640 1 -- 172.21.15.113:0/4056023649 wait complete. 2024-04-25T00:00:46.158 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:00:46.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.259+0000 7f37f9509640 1 Processor -- start 2024-04-25T00:00:46.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.259+0000 7f37f9509640 1 -- start start 2024-04-25T00:00:46.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f9509640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37f4144910 0x7f37f40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f9509640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f4144cf0 con 0x7f37f40b56b0 2024-04-25T00:00:46.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f9509640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f40b2f30 con 0x7f37f40b5240 2024-04-25T00:00:46.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f9509640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f37f40b4510 con 0x7f37f4144910 2024-04-25T00:00:46.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f37f40b56b0 0x7f37f40cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53100/0 (socket says 172.21.15.113:53100) 2024-04-25T00:00:46.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f37fe640 1 -- 172.21.15.113:0/3131689762 learned_addr learned my addr 172.21.15.113:0/3131689762 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:46.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f2ffd640 1 --2- 172.21.15.113:0/3131689762 >> v2:172.21.15.178:3300/0 conn(0x7f37f4144910 0x7f37f40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924007044 0 0) 0x7f37f4144cf0 con 0x7f37f40b56b0 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.260+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37e0005730 con 0x7f37f40b56b0 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3257405999 0 0) 0x7f37e0005730 con 0x7f37f40b56b0 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 >> v2:172.21.15.178:3300/0 conn(0x7f37f4144910 msgr2=0x7f37f40b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 --2- 172.21.15.113:0/3131689762 >> v2:172.21.15.178:3300/0 conn(0x7f37f4144910 0x7f37f40b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 >> v1:172.21.15.178:6789/0 conn(0x7f37f40b5240 legacy=0x7f37f406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f40b3210 con 0x7f37f40b56b0 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f37ec0019f0 con 0x7f37f40b56b0 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 >> v1:172.21.15.113:6789/0 conn(0x7f37f40b56b0 legacy=0x7f37f40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37e0005bc0 0x7f37e0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e0006a10 0x7f37e0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00073d0 0x7f37e000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40b4510 con 0x7f37e00073d0 2024-04-25T00:00:46.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37e000e2a0 con 0x7f37e0005bc0 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37e000e4a0 con 0x7f37e0006a10 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f37fe640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00073d0 0x7f37e000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f2ffd640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37e0005bc0 0x7f37e0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37ebfff640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e0006a10 0x7f37e0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f37fe640 1 -- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e0006a10 msgr2=0x7f37e0006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f37fe640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e0006a10 0x7f37e0006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f37fe640 1 -- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37e0005bc0 msgr2=0x7f37e0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f37fe640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37e0005bc0 0x7f37e0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37f37fe640 1 -- 172.21.15.113:0/3131689762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e0005730 con 0x7f37e00073d0 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.261+0000 7f37ebfff640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e0006a10 0x7f37e0006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.262+0000 7f37f37fe640 1 -- 172.21.15.113:0/3131689762 reap_dead start 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.262+0000 7f37f37fe640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00073d0 0x7f37e000dbd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f37f40752f0 tx=0x7f37e0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a97ed286e6b919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.262+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec00a480 con 0x7f37e00073d0 2024-04-25T00:00:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.262+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ec02f070 con 0x7f37e00073d0 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.262+0000 7f37f27fc640 1 -- 172.21.15.113:0/3131689762 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec02bc30 con 0x7f37e00073d0 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.262+0000 7f37f9509640 1 -- 172.21.15.113:0/3131689762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00073d0 msgr2=0x7f37e000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.262+0000 7f37f9509640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00073d0 0x7f37e000dbd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f37f40752f0 tx=0x7f37e0007a40 comp rx=0 tx=0).stop 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.263+0000 7f37f9509640 1 -- 172.21.15.113:0/3131689762 shutdown_connections 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.263+0000 7f37f9509640 1 --2- 172.21.15.113:0/3131689762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00073d0 0x7f37e000dbd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.263+0000 7f37f9509640 1 -- 172.21.15.113:0/3131689762 >> 172.21.15.113:0/3131689762 conn(0x7f37f40b1a50 msgr2=0x7f37f40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.263+0000 7f37f9509640 1 -- 172.21.15.113:0/3131689762 shutdown_connections 2024-04-25T00:00:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.263+0000 7f37f9509640 1 -- 172.21.15.113:0/3131689762 wait complete. 2024-04-25T00:00:46.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 Processor -- start 2024-04-25T00:00:46.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 -- start start 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e00073d0 0x7f37f413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37f40cd4c0 0x7f37f413c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37f40cead0 0x7f37f413e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40b4da0 con 0x7f37f40cd4c0 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40b37d0 con 0x7f37f40cead0 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37f9509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37f40b4f20 con 0x7f37e00073d0 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.264+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37f40cd4c0 0x7f37f413c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37f40cd4c0 0x7f37f413c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52730/0 (socket says 172.21.15.113:52730) 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 -- 172.21.15.113:0/2833042864 learned_addr learned my addr 172.21.15.113:0/2833042864 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 -- 172.21.15.113:0/2833042864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e00073d0 msgr2=0x7f37f413bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e00073d0 0x7f37f413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 -- 172.21.15.113:0/2833042864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37f40cead0 msgr2=0x7f37f413e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37f40cead0 0x7f37f413e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 -- 172.21.15.113:0/2833042864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f40cda30 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37ebfff640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37f40cd4c0 0x7f37f413c090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00e500 tx=0x7f37dc00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37eb7fe640 1 -- 172.21.15.113:0/2833042864 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0052e0 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37eb7fe640 1 -- 172.21.15.113:0/2833042864 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc018070 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f40cdd10 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37eb7fe640 1 -- 172.21.15.113:0/2833042864 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc003810 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.265+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f40ce2d0 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.266+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f4137f60 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.267+0000 7f37eb7fe640 1 -- 172.21.15.113:0/2833042864 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37dc0039b0 con 0x7f37f40cd4c0 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.267+0000 7f37eb7fe640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37d406be30 0x7f37d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.267+0000 7f37f2ffd640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37d406be30 0x7f37d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.267+0000 7f37eb7fe640 1 -- 172.21.15.113:0/2833042864 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f37dc08a7f0 con 0x7f37f40cd4c0 2024-04-25T00:00:46.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.268+0000 7f37f2ffd640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37d406be30 0x7f37d406e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009a40 tx=0x7f37e4009320 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.271+0000 7f37eb7fe640 1 -- 172.21.15.113:0/2833042864 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37dc059500 con 0x7f37f40cd4c0 2024-04-25T00:00:46.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.508+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f37f40731a0 con 0x7f37f40cd4c0 2024-04-25T00:00:46.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.508+0000 7f37eb7fe640 1 -- 172.21.15.113:0/2833042864 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f37dc058e00 con 0x7f37f40cd4c0 2024-04-25T00:00:46.508 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:46.508 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:46.508 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:00:46.509 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:00:46.510 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.511 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:00:46.512 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:00:46.514 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.516 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.517 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.518 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.520 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:46.521 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:00:46.522 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37d406be30 msgr2=0x7f37d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37d406be30 0x7f37d406e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009a40 tx=0x7f37e4009320 comp rx=0 tx=0).stop 2024-04-25T00:00:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37f40cd4c0 msgr2=0x7f37f413c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37f40cd4c0 0x7f37f413c090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00e500 tx=0x7f37dc00e9c0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 shutdown_connections 2024-04-25T00:00:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37d406be30 0x7f37d406e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37f40cead0 0x7f37f413e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37f40cd4c0 0x7f37f413c090 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 --2- 172.21.15.113:0/2833042864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37e00073d0 0x7f37f413bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 >> 172.21.15.113:0/2833042864 conn(0x7f37f40b1a50 msgr2=0x7f37f40b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 shutdown_connections 2024-04-25T00:00:46.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.510+0000 7f37f9509640 1 -- 172.21.15.113:0/2833042864 wait complete. 2024-04-25T00:00:46.524 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:46.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.620+0000 7f302bfc8640 1 Processor -- start 2024-04-25T00:00:46.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.620+0000 7f302bfc8640 1 -- start start 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302bfc8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3024144910 0x7f30240b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302bfc8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f30240b56b0 0x7f30240cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302bfc8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3024144cf0 con 0x7f30240b5240 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302bfc8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f30240b4510 con 0x7f30240b56b0 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302bfc8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f30240b2f30 con 0x7f3024144910 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302a565640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f30240b56b0 0x7f30240cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302a565640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f30240b56b0 0x7f30240cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52732/0 (socket says 172.21.15.113:52732) 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302a565640 1 -- 172.21.15.113:0/212553337 learned_addr learned my addr 172.21.15.113:0/212553337 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302a565640 1 -- 172.21.15.113:0/212553337 >> v2:172.21.15.178:3300/0 conn(0x7f3024144910 msgr2=0x7f30240b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302a565640 1 --2- 172.21.15.113:0/212553337 >> v2:172.21.15.178:3300/0 conn(0x7f3024144910 0x7f30240b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302a565640 1 -- 172.21.15.113:0/212553337 >> v1:172.21.15.178:6789/0 conn(0x7f30240b5240 legacy=0x7f302406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.621+0000 7f302a565640 1 -- 172.21.15.113:0/212553337 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240b3210 con 0x7f30240b56b0 2024-04-25T00:00:46.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.622+0000 7f302a565640 1 --2- 172.21.15.113:0/212553337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b56b0 0x7f30240cd500 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f301c00bb10 tx=0x7f301c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7232c85d0100fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.622+0000 7f3028d62640 1 -- 172.21.15.113:0/212553337 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c02f070 con 0x7f30240b56b0 2024-04-25T00:00:46.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.622+0000 7f3028d62640 1 -- 172.21.15.113:0/212553337 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c038070 con 0x7f30240b56b0 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.622+0000 7f3028d62640 1 -- 172.21.15.113:0/212553337 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c032a00 con 0x7f30240b56b0 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 -- 172.21.15.113:0/212553337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b56b0 msgr2=0x7f30240cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 --2- 172.21.15.113:0/212553337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b56b0 0x7f30240cd500 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f301c00bb10 tx=0x7f301c02c920 comp rx=0 tx=0).stop 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 -- 172.21.15.113:0/212553337 shutdown_connections 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 --2- 172.21.15.113:0/212553337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b56b0 0x7f30240cd500 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 --2- 172.21.15.113:0/212553337 >> v2:172.21.15.178:3300/0 conn(0x7f3024144910 0x7f30240b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 -- 172.21.15.113:0/212553337 >> 172.21.15.113:0/212553337 conn(0x7f30240b1a50 msgr2=0x7f30240b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 -- 172.21.15.113:0/212553337 shutdown_connections 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.623+0000 7f302bfc8640 1 -- 172.21.15.113:0/212553337 wait complete. 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.624+0000 7f302bfc8640 1 Processor -- start 2024-04-25T00:00:46.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.624+0000 7f302bfc8640 1 -- start start 2024-04-25T00:00:46.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f302bfc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b5240 0x7f30241391a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f302bfc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30240b56b0 0x7f302415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f302bfc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3024144910 0x7f30240ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f302bfc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b4da0 con 0x7f30240b5240 2024-04-25T00:00:46.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f302bfc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b37d0 con 0x7f3024144910 2024-04-25T00:00:46.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f302bfc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f30240b4f20 con 0x7f30240b56b0 2024-04-25T00:00:46.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b5240 0x7f30241391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b5240 0x7f30241391a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52738/0 (socket says 172.21.15.113:52738) 2024-04-25T00:00:46.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 -- 172.21.15.113:0/4100593970 learned_addr learned my addr 172.21.15.113:0/4100593970 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:46.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f302a565640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3024144910 0x7f30240ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 -- 172.21.15.113:0/4100593970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30240b56b0 msgr2=0x7f302415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30240b56b0 0x7f302415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 -- 172.21.15.113:0/4100593970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3024144910 msgr2=0x7f30240ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3024144910 0x7f30240ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 -- 172.21.15.113:0/4100593970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302413c4b0 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f3029d64640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b5240 0x7f30241391a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f302000ba50 tx=0x7f302000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.625+0000 7f301a7fc640 1 -- 172.21.15.113:0/4100593970 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020019070 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.626+0000 7f301a7fc640 1 -- 172.21.15.113:0/4100593970 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302000cc00 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.626+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302413c790 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.626+0000 7f301a7fc640 1 -- 172.21.15.113:0/4100593970 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020014440 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.626+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302413cd50 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.626+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30240cce00 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.627+0000 7f301a7fc640 1 -- 172.21.15.113:0/4100593970 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f302001c070 con 0x7f30240b5240 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.627+0000 7f301a7fc640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f300806be60 0x7f300806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.627+0000 7f301a7fc640 1 -- 172.21.15.113:0/4100593970 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f302008a8b0 con 0x7f30240b5240 2024-04-25T00:00:46.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.627+0000 7f3029563640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f300806be60 0x7f300806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.628+0000 7f3029563640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f300806be60 0x7f300806e320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f302413b790 tx=0x7f3014007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.631+0000 7f301a7fc640 1 -- 172.21.15.113:0/4100593970 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30240cce00 con 0x7f30240b5240 2024-04-25T00:00:46.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.805+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30240691a0 con 0x7f30240b5240 2024-04-25T00:00:46.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.806+0000 7f301a7fc640 1 -- 172.21.15.113:0/4100593970 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+12939 (secure 0 0 0) 0x7f30200595b0 con 0x7f30240b5240 2024-04-25T00:00:46.806 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:46.806 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":42,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:46.137448+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:46.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f300806be60 msgr2=0x7f300806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f300806be60 0x7f300806e320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f302413b790 tx=0x7f3014007990 comp rx=0 tx=0).stop 2024-04-25T00:00:46.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b5240 msgr2=0x7f30241391a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b5240 0x7f30241391a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f302000ba50 tx=0x7f302000bf10 comp rx=0 tx=0).stop 2024-04-25T00:00:46.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 shutdown_connections 2024-04-25T00:00:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f300806be60 0x7f300806e320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3024144910 0x7f30240ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30240b56b0 0x7f302415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 --2- 172.21.15.113:0/4100593970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30240b5240 0x7f30241391a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 >> 172.21.15.113:0/4100593970 conn(0x7f30240b1a50 msgr2=0x7f3024142810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 shutdown_connections 2024-04-25T00:00:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.808+0000 7f302bfc8640 1 -- 172.21.15.113:0/4100593970 wait complete. 2024-04-25T00:00:46.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:00:46.819 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:00:46.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.919+0000 7f96887c8640 1 Processor -- start 2024-04-25T00:00:46.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.919+0000 7f96887c8640 1 -- start start 2024-04-25T00:00:46.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f96887c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 0x7f968015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f96887c8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968015f4f0 con 0x7f96800b4a40 2024-04-25T00:00:46.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f96887c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968015f7d0 con 0x7f96800b3080 2024-04-25T00:00:46.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f96887c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f96800cde00 con 0x7f968015e2d0 2024-04-25T00:00:46.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9686d65640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f96800b4a40 0x7f96800ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53106/0 (socket says 172.21.15.113:53106) 2024-04-25T00:00:46.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9686d65640 1 -- 172.21.15.113:0/1944950756 learned_addr learned my addr 172.21.15.113:0/1944950756 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:46.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9686564640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 0x7f968015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71227088 0 0) 0x7f968015f4f0 con 0x7f96800b4a40 2024-04-25T00:00:46.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f966c003520 con 0x7f96800b4a40 2024-04-25T00:00:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 743402689 0 0) 0x7f968015f7d0 con 0x7f96800b3080 2024-04-25T00:00:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f968015f4f0 con 0x7f96800b3080 2024-04-25T00:00:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1644997387 0 0) 0x7f966c003520 con 0x7f96800b4a40 2024-04-25T00:00:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 msgr2=0x7f968015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 0x7f968015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 >> v1:172.21.15.178:6789/0 conn(0x7f96800b3080 legacy=0x7f96800b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968015fab0 con 0x7f96800b4a40 2024-04-25T00:00:46.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f967c0019f0 con 0x7f96800b4a40 2024-04-25T00:00:46.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 >> v1:172.21.15.113:6789/0 conn(0x7f96800b4a40 legacy=0x7f96800ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 0x7f966c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f966c004140 0x7f966c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004d40 0x7f966c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96800cde00 con 0x7f966c004140 2024-04-25T00:00:46.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f966c00bb00 con 0x7f966c004d40 2024-04-25T00:00:46.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f966c00df10 con 0x7f968015e2d0 2024-04-25T00:00:46.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.920+0000 7f9685d63640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f966c004140 0x7f966c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9686564640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 0x7f966c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9686d65640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004d40 0x7f966c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9685d63640 1 -- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 msgr2=0x7f966c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9685d63640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f968015e2d0 0x7f966c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9685d63640 1 -- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004d40 msgr2=0x7f966c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:46.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9685d63640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004d40 0x7f966c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9685d63640 1 -- 172.21.15.113:0/1944950756 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c003520 con 0x7f966c004140 2024-04-25T00:00:46.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9685d63640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f966c004140 0x7f966c004800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f96700018a0 tx=0x7f966c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0c4d3e27181da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.921+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700014f0 con 0x7f966c004140 2024-04-25T00:00:46.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96700052d0 con 0x7f966c004140 2024-04-25T00:00:46.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f9685562640 1 -- 172.21.15.113:0/1944950756 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967002ab90 con 0x7f966c004140 2024-04-25T00:00:46.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f96887c8640 1 -- 172.21.15.113:0/1944950756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f966c004140 msgr2=0x7f966c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f96887c8640 1 --2- 172.21.15.113:0/1944950756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f966c004140 0x7f966c004800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f96700018a0 tx=0x7f966c005830 comp rx=0 tx=0).stop 2024-04-25T00:00:46.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f9686d65640 1 -- 172.21.15.113:0/1944950756 reap_dead start 2024-04-25T00:00:46.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f96887c8640 1 -- 172.21.15.113:0/1944950756 shutdown_connections 2024-04-25T00:00:46.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f96887c8640 1 -- 172.21.15.113:0/1944950756 >> 172.21.15.113:0/1944950756 conn(0x7f968006b0f0 msgr2=0x7f968006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f96887c8640 1 -- 172.21.15.113:0/1944950756 shutdown_connections 2024-04-25T00:00:46.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.922+0000 7f96887c8640 1 -- 172.21.15.113:0/1944950756 wait complete. 2024-04-25T00:00:46.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.923+0000 7f96887c8640 1 Processor -- start 2024-04-25T00:00:46.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.923+0000 7f96887c8640 1 -- start start 2024-04-25T00:00:46.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f96887c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004140 0x7f96801518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f96887c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9680142980 0x7f96801569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f96887c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9680156f20 0x7f96801573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f96887c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f968015e830 con 0x7f9680156f20 2024-04-25T00:00:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f96887c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f968015e6b0 con 0x7f966c004140 2024-04-25T00:00:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f96887c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f968015e9b0 con 0x7f9680142980 2024-04-25T00:00:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9680156f20 0x7f96801573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9680156f20 0x7f96801573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52748/0 (socket says 172.21.15.113:52748) 2024-04-25T00:00:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 -- 172.21.15.113:0/278405824 learned_addr learned my addr 172.21.15.113:0/278405824 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 -- 172.21.15.113:0/278405824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9680142980 msgr2=0x7f96801569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9680142980 0x7f96801569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 -- 172.21.15.113:0/278405824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004140 msgr2=0x7f96801518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004140 0x7f96801518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 -- 172.21.15.113:0/278405824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9680152ab0 con 0x7f9680156f20 2024-04-25T00:00:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.924+0000 7f9686d65640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9680156f20 0x7f96801573a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9680050ce0 tx=0x7f967c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.925+0000 7f9676ffd640 1 -- 172.21.15.113:0/278405824 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00aaf0 con 0x7f9680156f20 2024-04-25T00:00:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.925+0000 7f9676ffd640 1 -- 172.21.15.113:0/278405824 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967c018070 con 0x7f9680156f20 2024-04-25T00:00:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.925+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968014fa20 con 0x7f9680156f20 2024-04-25T00:00:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.925+0000 7f9676ffd640 1 -- 172.21.15.113:0/278405824 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0133f0 con 0x7f9680156f20 2024-04-25T00:00:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.925+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968014ffe0 con 0x7f9680156f20 2024-04-25T00:00:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.926+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96801501e0 con 0x7f9680156f20 2024-04-25T00:00:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.926+0000 7f9676ffd640 1 -- 172.21.15.113:0/278405824 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f967c013590 con 0x7f9680156f20 2024-04-25T00:00:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.926+0000 7f9676ffd640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f966406be60 0x7f966406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.927+0000 7f9676ffd640 1 -- 172.21.15.113:0/278405824 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f967c08afc0 con 0x7f9680156f20 2024-04-25T00:00:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.927+0000 7f9686564640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f966406be60 0x7f966406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.927+0000 7f9686564640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f966406be60 0x7f966406e320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f96780099e0 tx=0x7f9678006c30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:46.931+0000 7f9676ffd640 1 -- 172.21.15.113:0/278405824 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96801501e0 con 0x7f9680156f20 2024-04-25T00:00:47.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.103+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9680072050 con 0x7f9680156f20 2024-04-25T00:00:47.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.104+0000 7f9676ffd640 1 -- 172.21.15.113:0/278405824 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f967c059d50 con 0x7f9680156f20 2024-04-25T00:00:47.104 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:00:47.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f966406be60 msgr2=0x7f966406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f966406be60 0x7f966406e320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f96780099e0 tx=0x7f9678006c30 comp rx=0 tx=0).stop 2024-04-25T00:00:47.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9680156f20 msgr2=0x7f96801573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9680156f20 0x7f96801573a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9680050ce0 tx=0x7f967c007bd0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 shutdown_connections 2024-04-25T00:00:47.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f966406be60 0x7f966406e320 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9680156f20 0x7f96801573a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9680142980 0x7f96801569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.105+0000 7f96887c8640 1 --2- 172.21.15.113:0/278405824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f966c004140 0x7f96801518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.106+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 >> 172.21.15.113:0/278405824 conn(0x7f968006b0f0 msgr2=0x7f968006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.106+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 shutdown_connections 2024-04-25T00:00:47.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.106+0000 7f96887c8640 1 -- 172.21.15.113:0/278405824 wait complete. 2024-04-25T00:00:47.117 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:00:47.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.214+0000 7f40fbe6b640 1 Processor -- start 2024-04-25T00:00:47.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.214+0000 7f40fbe6b640 1 -- start start 2024-04-25T00:00:47.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40fbe6b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40f415e320 0x7f40f40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40fbe6b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f40d2290 0x7f40f40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40fbe6b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f40d3de0 con 0x7f40f40d0b00 2024-04-25T00:00:47.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40fbe6b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f40f40d3410 con 0x7f40f415e320 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40fbe6b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f40f40d2800 con 0x7f40f40d2290 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40f9c07640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40f415e320 0x7f40f40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40fa408640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f40f40d0b00 0x7f40f40b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53114/0 (socket says 172.21.15.113:53114) 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40fa408640 1 -- 172.21.15.113:0/793266692 learned_addr learned my addr 172.21.15.113:0/793266692 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.215+0000 7f40f9406640 1 --2- 172.21.15.113:0/793266692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f40d2290 0x7f40f40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f9c07640 1 -- 172.21.15.113:0/793266692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f40d2290 msgr2=0x7f40f40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f9c07640 1 --2- 172.21.15.113:0/793266692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f40d2290 0x7f40f40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f9c07640 1 -- 172.21.15.113:0/793266692 >> v1:172.21.15.113:6789/0 conn(0x7f40f40d0b00 legacy=0x7f40f40b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f8c05640 1 -- 172.21.15.113:0/793266692 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819699586 0 0) 0x7f40f40d3de0 con 0x7f40f40d0b00 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f9c07640 1 -- 172.21.15.113:0/793266692 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f40d2ae0 con 0x7f40f415e320 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f9c07640 1 --2- 172.21.15.113:0/793266692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415e320 0x7f40f40d38a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f40f00097a0 tx=0x7f40f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d19183020a2e38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f8c05640 1 -- 172.21.15.113:0/793266692 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0037070 con 0x7f40f415e320 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f8c05640 1 -- 172.21.15.113:0/793266692 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f0032440 con 0x7f40f415e320 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.216+0000 7f40f8c05640 1 -- 172.21.15.113:0/793266692 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0032810 con 0x7f40f415e320 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 -- 172.21.15.113:0/793266692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415e320 msgr2=0x7f40f40d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/793266692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415e320 0x7f40f40d38a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f40f00097a0 tx=0x7f40f002a210 comp rx=0 tx=0).stop 2024-04-25T00:00:47.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 -- 172.21.15.113:0/793266692 shutdown_connections 2024-04-25T00:00:47.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/793266692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f40d2290 0x7f40f40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/793266692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415e320 0x7f40f40d38a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 -- 172.21.15.113:0/793266692 >> 172.21.15.113:0/793266692 conn(0x7f40f40b5c20 msgr2=0x7f40f40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 -- 172.21.15.113:0/793266692 shutdown_connections 2024-04-25T00:00:47.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.217+0000 7f40fbe6b640 1 -- 172.21.15.113:0/793266692 wait complete. 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.218+0000 7f40fbe6b640 1 Processor -- start 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.218+0000 7f40fbe6b640 1 -- start start 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40fbe6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f40d0b00 0x7f40f414e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40fbe6b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f40d2290 0x7f40f40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40fbe6b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f415e320 0x7f40f40b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40fbe6b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f40f40b9fe0 con 0x7f40f40d0b00 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40fbe6b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40f40d30a0 con 0x7f40f40d2290 2024-04-25T00:00:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40fbe6b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f40f415e700 con 0x7f40f415e320 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f40d0b00 0x7f40f414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f40d0b00 0x7f40f414e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52760/0 (socket says 172.21.15.113:52760) 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 -- 172.21.15.113:0/3521106249 learned_addr learned my addr 172.21.15.113:0/3521106249 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 -- 172.21.15.113:0/3521106249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f415e320 msgr2=0x7f40f40b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f415e320 0x7f40f40b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 -- 172.21.15.113:0/3521106249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f40d2290 msgr2=0x7f40f40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f40d2290 0x7f40f40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 -- 172.21.15.113:0/3521106249 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f40cdec0 con 0x7f40f40d0b00 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.219+0000 7f40f9c07640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f40d0b00 0x7f40f414e800 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f40f00098c0 tx=0x7f40f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.220+0000 7f40ea7fc640 1 -- 172.21.15.113:0/3521106249 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0037040 con 0x7f40f40d0b00 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.220+0000 7f40ea7fc640 1 -- 172.21.15.113:0/3521106249 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f0030770 con 0x7f40f40d0b00 2024-04-25T00:00:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.220+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f41544f0 con 0x7f40f40d0b00 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.220+0000 7f40ea7fc640 1 -- 172.21.15.113:0/3521106249 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0030b40 con 0x7f40f40d0b00 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.220+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f4154fd0 con 0x7f40f40d0b00 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.221+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f40691a0 con 0x7f40f40d0b00 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.221+0000 7f40ea7fc640 1 -- 172.21.15.113:0/3521106249 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40f0044020 con 0x7f40f40d0b00 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.221+0000 7f40ea7fc640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40d806be60 0x7f40d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.222+0000 7f40ea7fc640 1 -- 172.21.15.113:0/3521106249 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f40f00aafd0 con 0x7f40f40d0b00 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.222+0000 7f40f9406640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40d806be60 0x7f40d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.222+0000 7f40f9406640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40d806be60 0x7f40d806e320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f40f40cd1a0 tx=0x7f40e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.226+0000 7f40ea7fc640 1 -- 172.21.15.113:0/3521106249 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40f0032e20 con 0x7f40f40d0b00 2024-04-25T00:00:47.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.404+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f40f40b3260 con 0x7f40f40d0b00 2024-04-25T00:00:47.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.404+0000 7f40ea7fc640 1 -- 172.21.15.113:0/3521106249 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7f40f0079cd0 con 0x7f40f40d0b00 2024-04-25T00:00:47.405 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:47.405 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:00:47.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:00:47.405 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:00:47.405 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:00:47.405 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:00:47.405 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:00:47.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40d806be60 msgr2=0x7f40d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40d806be60 0x7f40d806e320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f40f40cd1a0 tx=0x7f40e4008040 comp rx=0 tx=0).stop 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f40d0b00 msgr2=0x7f40f414e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f40d0b00 0x7f40f414e800 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f40f00098c0 tx=0x7f40f00046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 shutdown_connections 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f40d806be60 0x7f40d806e320 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40f40d2290 0x7f40f40cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40f415e320 0x7f40f40b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 --2- 172.21.15.113:0/3521106249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f40d0b00 0x7f40f414e800 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 >> 172.21.15.113:0/3521106249 conn(0x7f40f40b5c20 msgr2=0x7f40f40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 shutdown_connections 2024-04-25T00:00:47.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.406+0000 7f40fbe6b640 1 -- 172.21.15.113:0/3521106249 wait complete. 2024-04-25T00:00:47.417 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:00:47.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:00:47.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.517+0000 7f1748ddc640 1 Processor -- start 2024-04-25T00:00:47.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.517+0000 7f1748ddc640 1 -- start start 2024-04-25T00:00:47.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.518+0000 7f1748ddc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f174415e510 0x7f1744161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.518+0000 7f1748ddc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17440b9bb0 0x7f17440b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.518+0000 7f1748ddc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17441626f0 con 0x7f17440b61e0 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.518+0000 7f1748ddc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f17441629d0 con 0x7f17440b9bb0 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.518+0000 7f1748ddc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1744162cb0 con 0x7f174415e510 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f1741d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17440b9bb0 0x7f17440b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f1742d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f17440b61e0 0x7f1744071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43780/0 (socket says 172.21.15.113:43780) 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f1742d9d640 1 -- 172.21.15.113:0/255735916 learned_addr learned my addr 172.21.15.113:0/255735916 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f174259c640 1 --2- 172.21.15.113:0/255735916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f174415e510 0x7f1744161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f1741d9b640 1 -- 172.21.15.113:0/255735916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f174415e510 msgr2=0x7f1744161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f1741d9b640 1 --2- 172.21.15.113:0/255735916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f174415e510 0x7f1744161ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f1741d9b640 1 -- 172.21.15.113:0/255735916 >> v1:172.21.15.113:6789/0 conn(0x7f17440b61e0 legacy=0x7f1744071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f1741d9b640 1 -- 172.21.15.113:0/255735916 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17440d2660 con 0x7f17440b9bb0 2024-04-25T00:00:47.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.519+0000 7f174159a640 1 -- 172.21.15.113:0/255735916 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058077275 0 0) 0x7f17441626f0 con 0x7f17440b61e0 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1741d9b640 1 --2- 172.21.15.113:0/255735916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b9bb0 0x7f17440b5ca0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f17380097a0 tx=0x7f173802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbdc1efe7199a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f174159a640 1 -- 172.21.15.113:0/255735916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738037070 con 0x7f17440b9bb0 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f174159a640 1 -- 172.21.15.113:0/255735916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173802ae50 con 0x7f17440b9bb0 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f174159a640 1 -- 172.21.15.113:0/255735916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738032660 con 0x7f17440b9bb0 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 -- 172.21.15.113:0/255735916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b9bb0 msgr2=0x7f17440b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 --2- 172.21.15.113:0/255735916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b9bb0 0x7f17440b5ca0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f17380097a0 tx=0x7f173802a100 comp rx=0 tx=0).stop 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 -- 172.21.15.113:0/255735916 shutdown_connections 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 --2- 172.21.15.113:0/255735916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b9bb0 0x7f17440b5ca0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 --2- 172.21.15.113:0/255735916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f174415e510 0x7f1744161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 -- 172.21.15.113:0/255735916 >> 172.21.15.113:0/255735916 conn(0x7f174406b720 msgr2=0x7f17440d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 -- 172.21.15.113:0/255735916 shutdown_connections 2024-04-25T00:00:47.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.520+0000 7f1748ddc640 1 -- 172.21.15.113:0/255735916 wait complete. 2024-04-25T00:00:47.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 Processor -- start 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 -- start start 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b61e0 0x7f174414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17440b9bb0 0x7f17440b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174415e510 0x7f17441595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f17440d3e70 con 0x7f17440b61e0 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17441623e0 con 0x7f174415e510 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f1748ddc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17440b73c0 con 0x7f17440b9bb0 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f174259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b61e0 0x7f174414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f174259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b61e0 0x7f174414c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54354/0 (socket says 172.21.15.113:54354) 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.522+0000 7f174259c640 1 -- 172.21.15.113:0/2546167712 learned_addr learned my addr 172.21.15.113:0/2546167712 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1741d9b640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17440b9bb0 0x7f17440b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1742d9d640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174415e510 0x7f17441595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f174259c640 1 -- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17440b9bb0 msgr2=0x7f17440b8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f174259c640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17440b9bb0 0x7f17440b8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f174259c640 1 -- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174415e510 msgr2=0x7f17441595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f174259c640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174415e510 0x7f17441595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f174259c640 1 -- 172.21.15.113:0/2546167712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174414ce30 con 0x7f17440b61e0 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1741d9b640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17440b9bb0 0x7f17440b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f174259c640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b61e0 0x7f174414c8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1734007e40 tx=0x7f173400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1732ffd640 1 -- 172.21.15.113:0/2546167712 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734014620 con 0x7f17440b61e0 2024-04-25T00:00:47.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1732ffd640 1 -- 172.21.15.113:0/2546167712 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1734019070 con 0x7f17440b61e0 2024-04-25T00:00:47.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17440b9590 con 0x7f17440b61e0 2024-04-25T00:00:47.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1732ffd640 1 -- 172.21.15.113:0/2546167712 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734014a00 con 0x7f17440b61e0 2024-04-25T00:00:47.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.523+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17441f7cd0 con 0x7f17440b61e0 2024-04-25T00:00:47.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.524+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17440691a0 con 0x7f17440b61e0 2024-04-25T00:00:47.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.524+0000 7f1732ffd640 1 -- 172.21.15.113:0/2546167712 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1734014ba0 con 0x7f17440b61e0 2024-04-25T00:00:47.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.525+0000 7f1732ffd640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f172406be30 0x7f172406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.525+0000 7f1732ffd640 1 -- 172.21.15.113:0/2546167712 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f173408b2e0 con 0x7f17440b61e0 2024-04-25T00:00:47.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.525+0000 7f1741d9b640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f172406be30 0x7f172406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.525+0000 7f1741d9b640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f172406be30 0x7f172406e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1738034000 tx=0x7f173802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.529+0000 7f1732ffd640 1 -- 172.21.15.113:0/2546167712 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f173405a070 con 0x7f17440b61e0 2024-04-25T00:00:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:47.705+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f17440d1f30 con 0x7f17440b61e0 2024-04-25T00:00:48.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.149+0000 7f1732ffd640 1 -- 172.21.15.113:0/2546167712 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f1734059970 con 0x7f17440b61e0 2024-04-25T00:00:48.150 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:00:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f172406be30 msgr2=0x7f172406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f172406be30 0x7f172406e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1738034000 tx=0x7f173802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b61e0 msgr2=0x7f174414c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b61e0 0x7f174414c8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1734007e40 tx=0x7f173400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 shutdown_connections 2024-04-25T00:00:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f172406be30 0x7f172406e2f0 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1738034000 tx=0x7f173802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174415e510 0x7f17441595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17440b9bb0 0x7f17440b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 --2- 172.21.15.113:0/2546167712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17440b61e0 0x7f174414c8f0 secure :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1734007e40 tx=0x7f173400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.153+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 >> 172.21.15.113:0/2546167712 conn(0x7f174406b720 msgr2=0x7f17440b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.154+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 shutdown_connections 2024-04-25T00:00:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.154+0000 7f1748ddc640 1 -- 172.21.15.113:0/2546167712 wait complete. 2024-04-25T00:00:48.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:00:48.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.273+0000 7f100adee640 1 Processor -- start 2024-04-25T00:00:48.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.273+0000 7f100adee640 1 -- start start 2024-04-25T00:00:48.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.273+0000 7f100adee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f100415e510 0x7f10040d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.273+0000 7f100adee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10040d2e00 con 0x7f10040b63b0 2024-04-25T00:00:48.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.273+0000 7f100adee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10040d30e0 con 0x7f10040d28c0 2024-04-25T00:00:48.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.273+0000 7f100adee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f10040d33c0 con 0x7f100415e510 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f100938b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f10040b63b0 0x7f10040b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43792/0 (socket says 172.21.15.113:43792) 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f100938b640 1 -- 172.21.15.113:0/1640691498 learned_addr learned my addr 172.21.15.113:0/1640691498 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f1008b8a640 1 --2- 172.21.15.113:0/1640691498 >> v2:172.21.15.178:3300/0 conn(0x7f100415e510 0x7f10040d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878665173 0 0) 0x7f10040d2e00 con 0x7f10040b63b0 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ff4003520 con 0x7f10040b63b0 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3427570941 0 0) 0x7f10040d30e0 con 0x7f10040d28c0 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10040d2e00 con 0x7f10040d28c0 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f1008b8a640 1 -- 172.21.15.113:0/1640691498 >> v1:172.21.15.178:6789/0 conn(0x7f10040d28c0 legacy=0x7f10040b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1132985857 0 0) 0x7f0ff4003520 con 0x7f10040b63b0 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f1008b8a640 1 -- 172.21.15.113:0/1640691498 >> v1:172.21.15.113:6789/0 conn(0x7f10040b63b0 legacy=0x7f10040b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.274+0000 7f1008b8a640 1 -- 172.21.15.113:0/1640691498 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10040d36a0 con 0x7f100415e510 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f1008b8a640 1 --2- 172.21.15.113:0/1640691498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f100415e510 0x7f10040d2380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80097a0 tx=0x7f0ff802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2e9bc1132d6b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8037070 con 0x7f100415e510 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff802abc0 con 0x7f100415e510 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f10037fe640 1 -- 172.21.15.113:0/1640691498 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0019f0 con 0x7f100415e510 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f100adee640 1 -- 172.21.15.113:0/1640691498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f100415e510 msgr2=0x7f10040d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f100adee640 1 --2- 172.21.15.113:0/1640691498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f100415e510 0x7f10040d2380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80097a0 tx=0x7f0ff802a130 comp rx=0 tx=0).stop 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f100adee640 1 -- 172.21.15.113:0/1640691498 shutdown_connections 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f100adee640 1 --2- 172.21.15.113:0/1640691498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f100415e510 0x7f10040d2380 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.275+0000 7f100adee640 1 -- 172.21.15.113:0/1640691498 >> 172.21.15.113:0/1640691498 conn(0x7f100406b720 msgr2=0x7f10040d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:48.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.276+0000 7f100adee640 1 -- 172.21.15.113:0/1640691498 shutdown_connections 2024-04-25T00:00:48.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.276+0000 7f100adee640 1 -- 172.21.15.113:0/1640691498 wait complete. 2024-04-25T00:00:48.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 Processor -- start 2024-04-25T00:00:48.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 -- start start 2024-04-25T00:00:48.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10040b63b0 0x7f100415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10040d28c0 0x7f1004155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100415e510 0x7f1004152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f10040b9dd0 con 0x7f100415e510 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10040d3ea0 con 0x7f10040d28c0 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.277+0000 7f100adee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10040b7ad0 con 0x7f10040b63b0 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100415e510 0x7f1004152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100415e510 0x7f1004152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54368/0 (socket says 172.21.15.113:54368) 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 -- 172.21.15.113:0/1371147139 learned_addr learned my addr 172.21.15.113:0/1371147139 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f1003fff640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10040d28c0 0x7f1004155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f1008b8a640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10040b63b0 0x7f100415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 -- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10040b63b0 msgr2=0x7f100415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10040b63b0 0x7f100415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 -- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10040d28c0 msgr2=0x7f1004155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10040d28c0 0x7f1004155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 -- 172.21.15.113:0/1371147139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10041577f0 con 0x7f100415e510 2024-04-25T00:00:48.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100938b640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100415e510 0x7f1004152290 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005c00 tx=0x7f0ffc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f10017fa640 1 -- 172.21.15.113:0/1371147139 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc017070 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f10017fa640 1 -- 172.21.15.113:0/1371147139 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc00ce50 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10040c8c50 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f10017fa640 1 -- 172.21.15.113:0/1371147139 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc012640 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.278+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10040c9210 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.279+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10040c9410 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.280+0000 7f10017fa640 1 -- 172.21.15.113:0/1371147139 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ffc01a070 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.280+0000 7f10017fa640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fe806be30 0x7f0fe806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.280+0000 7f10017fa640 1 -- 172.21.15.113:0/1371147139 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7f0ffc08ab80 con 0x7f100415e510 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.280+0000 7f1008b8a640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fe806be30 0x7f0fe806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:48.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.281+0000 7f1008b8a640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fe806be30 0x7f0fe806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:48.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.285+0000 7f10017fa640 1 -- 172.21.15.113:0/1371147139 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f10040c9410 con 0x7f100415e510 2024-04-25T00:00:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:48.458+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f10040691a0 con 0x7f100415e510 2024-04-25T00:00:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.154+0000 7f10017fa640 1 -- 172.21.15.113:0/1371147139 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7f0ffc0597a0 con 0x7f100415e510 2024-04-25T00:00:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fe806be30 msgr2=0x7f0fe806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fe806be30 0x7f0fe806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100415e510 msgr2=0x7f1004152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100415e510 0x7f1004152290 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005c00 tx=0x7f0ffc00a430 comp rx=0 tx=0).stop 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 shutdown_connections 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fe806be30 0x7f0fe806e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10040d28c0 0x7f1004155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100415e510 0x7f1004152290 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 --2- 172.21.15.113:0/1371147139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10040b63b0 0x7f100415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 >> 172.21.15.113:0/1371147139 conn(0x7f100406b720 msgr2=0x7f100415c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.159+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 shutdown_connections 2024-04-25T00:00:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.160+0000 7f100adee640 1 -- 172.21.15.113:0/1371147139 wait complete. 2024-04-25T00:00:49.184 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:00:49.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd265fa9640 1 Processor -- start 2024-04-25T00:00:49.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd265fa9640 1 -- start start 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd265fa9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd26015e510 0x7fd2600d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd265fa9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2600d2e00 con 0x7fd2600d28c0 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd265fa9640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2600d30e0 con 0x7fd2600b63b0 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd265fa9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd2600d33c0 con 0x7fd26015e510 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd25f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd26015e510 0x7fd2600d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd25effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd2600d28c0 0x7fd2600b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43798/0 (socket says 172.21.15.113:43798) 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd25f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd26015e510 0x7fd2600d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54378/0 (socket says 172.21.15.113:54378) 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.286+0000 7fd25effd640 1 -- 172.21.15.113:0/1443267083 learned_addr learned my addr 172.21.15.113:0/1443267083 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.287+0000 7fd25f7fe640 1 -- 172.21.15.113:0/1443267083 >> v1:172.21.15.178:6789/0 conn(0x7fd2600b63b0 legacy=0x7fd2600b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:49.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.287+0000 7fd25f7fe640 1 -- 172.21.15.113:0/1443267083 >> v1:172.21.15.113:6789/0 conn(0x7fd2600d28c0 legacy=0x7fd2600b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.287+0000 7fd25e7fc640 1 -- 172.21.15.113:0/1443267083 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344578648 0 0) 0x7fd2600d2e00 con 0x7fd2600d28c0 2024-04-25T00:00:49.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.287+0000 7fd25f7fe640 1 -- 172.21.15.113:0/1443267083 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2600d36a0 con 0x7fd26015e510 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.287+0000 7fd25f7fe640 1 --2- 172.21.15.113:0/1443267083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd2600d2380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd2500097a0 tx=0x7fd25002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d1f8cdcb0517ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd25e7fc640 1 -- 172.21.15.113:0/1443267083 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250037070 con 0x7fd26015e510 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd25e7fc640 1 -- 172.21.15.113:0/1443267083 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25002ae50 con 0x7fd26015e510 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd25e7fc640 1 -- 172.21.15.113:0/1443267083 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250032660 con 0x7fd26015e510 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd265fa9640 1 -- 172.21.15.113:0/1443267083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 msgr2=0x7fd2600d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1443267083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd2600d2380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd2500097a0 tx=0x7fd25002a140 comp rx=0 tx=0).stop 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd265fa9640 1 -- 172.21.15.113:0/1443267083 shutdown_connections 2024-04-25T00:00:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1443267083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd2600d2380 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd265fa9640 1 -- 172.21.15.113:0/1443267083 >> 172.21.15.113:0/1443267083 conn(0x7fd26006b720 msgr2=0x7fd2600d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:49.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd265fa9640 1 -- 172.21.15.113:0/1443267083 shutdown_connections 2024-04-25T00:00:49.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.288+0000 7fd265fa9640 1 -- 172.21.15.113:0/1443267083 wait complete. 2024-04-25T00:00:49.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.289+0000 7fd265fa9640 1 Processor -- start 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.289+0000 7fd265fa9640 1 -- start start 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd265fa9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2600b63b0 0x7fd2600ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd265fa9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2600d28c0 0x7fd26014a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd265fa9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd260160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd265fa9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd2600b9dd0 con 0x7fd26015e510 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd265fa9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd2600d3ea0 con 0x7fd2600d28c0 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd265fa9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2600b7ad0 con 0x7fd2600b63b0 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd260160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd260160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54384/0 (socket says 172.21.15.113:54384) 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd25ffff640 1 -- 172.21.15.113:0/1996345747 learned_addr learned my addr 172.21.15.113:0/1996345747 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd25effd640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2600d28c0 0x7fd26014a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd25f7fe640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2600b63b0 0x7fd2600ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd25ffff640 1 -- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2600b63b0 msgr2=0x7fd2600ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.290+0000 7fd25ffff640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2600b63b0 0x7fd2600ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd25ffff640 1 -- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2600d28c0 msgr2=0x7fd26014a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd25ffff640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2600d28c0 0x7fd26014a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd25ffff640 1 -- 172.21.15.113:0/1996345747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2600c97e0 con 0x7fd26015e510 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd25ffff640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd260160990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c7b0 tx=0x7fd25800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd247fff640 1 -- 172.21.15.113:0/1996345747 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258017070 con 0x7fd26015e510 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd247fff640 1 -- 172.21.15.113:0/1996345747 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd258003da0 con 0x7fd26015e510 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd247fff640 1 -- 172.21.15.113:0/1996345747 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25800ce60 con 0x7fd26015e510 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2600c9ac0 con 0x7fd26015e510 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.291+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2600ca080 con 0x7fd26015e510 2024-04-25T00:00:49.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.292+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2600691a0 con 0x7fd26015e510 2024-04-25T00:00:49.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.292+0000 7fd247fff640 1 -- 172.21.15.113:0/1996345747 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd25801a070 con 0x7fd26015e510 2024-04-25T00:00:49.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.293+0000 7fd247fff640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd24c06be30 0x7fd24c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.293+0000 7fd247fff640 1 -- 172.21.15.113:0/1996345747 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7fd25808a4a0 con 0x7fd26015e510 2024-04-25T00:00:49.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.293+0000 7fd25f7fe640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd24c06be30 0x7fd24c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.293+0000 7fd25f7fe640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd24c06be30 0x7fd24c06e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd250034000 tx=0x7fd25002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.297+0000 7fd247fff640 1 -- 172.21.15.113:0/1996345747 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd258058eb0 con 0x7fd26015e510 2024-04-25T00:00:49.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:49.510+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd26014aad0 con 0x7fd26015e510 2024-04-25T00:00:50.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.156+0000 7fd247fff640 1 -- 172.21.15.113:0/1996345747 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) ==== 172+0+0 (secure 0 0 0) 0x7fd25805f080 con 0x7fd26015e510 2024-04-25T00:00:50.156 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T00:00:50.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.162+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd24c06be30 msgr2=0x7fd24c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.162+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd24c06be30 0x7fd24c06e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd250034000 tx=0x7fd25002e000 comp rx=0 tx=0).stop 2024-04-25T00:00:50.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.162+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 msgr2=0x7fd260160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.162+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd260160990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c7b0 tx=0x7fd25800cc70 comp rx=0 tx=0).stop 2024-04-25T00:00:50.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.163+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 shutdown_connections 2024-04-25T00:00:50.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.163+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd24c06be30 0x7fd24c06e2f0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd250034000 tx=0x7fd25002e000 comp rx=0 tx=0).stop 2024-04-25T00:00:50.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.163+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2600d28c0 0x7fd26014a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.163+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd26015e510 0x7fd260160990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.163+0000 7fd265fa9640 1 --2- 172.21.15.113:0/1996345747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2600b63b0 0x7fd2600ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.163+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 >> 172.21.15.113:0/1996345747 conn(0x7fd26006b720 msgr2=0x7fd2601558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.173+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 shutdown_connections 2024-04-25T00:00:50.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.174+0000 7fd265fa9640 1 -- 172.21.15.113:0/1996345747 wait complete. 2024-04-25T00:00:50.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:50.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.295+0000 7fe6d359e640 1 Processor -- start 2024-04-25T00:00:50.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.295+0000 7fe6d359e640 1 -- start start 2024-04-25T00:00:50.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d359e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6cc0cac50 0x7fe6cc0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d359e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6cc0ca810 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d359e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6cc0cb830 con 0x7fe6cc162950 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d359e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe6cc0cb5f0 con 0x7fe6cc0cac50 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe6cc06e6f0 0x7fe6cc06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43810/0 (socket says 172.21.15.113:43810) 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d259c640 1 -- 172.21.15.113:0/2242268546 learned_addr learned my addr 172.21.15.113:0/2242268546 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d1d9b640 1 --2- 172.21.15.113:0/2242268546 >> v2:172.21.15.113:3300/0 conn(0x7fe6cc0cac50 0x7fe6cc0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d159a640 1 -- 172.21.15.113:0/2242268546 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1587593812 0 0) 0x7fe6cc0ca810 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d159a640 1 -- 172.21.15.113:0/2242268546 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6b8003520 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d1d9b640 1 -- 172.21.15.113:0/2242268546 >> v1:172.21.15.178:6789/0 conn(0x7fe6cc162950 legacy=0x7fe6cc164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d1d9b640 1 -- 172.21.15.113:0/2242268546 >> v1:172.21.15.113:6789/0 conn(0x7fe6cc06e6f0 legacy=0x7fe6cc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.296+0000 7fe6d1d9b640 1 -- 172.21.15.113:0/2242268546 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc072140 con 0x7fe6cc0cac50 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.297+0000 7fe6d1d9b640 1 --2- 172.21.15.113:0/2242268546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc0cb0b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097a0 tx=0x7fe6c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533712183c2f208a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.297+0000 7fe6d159a640 1 -- 172.21.15.113:0/2242268546 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4037070 con 0x7fe6cc0cac50 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.297+0000 7fe6d159a640 1 -- 172.21.15.113:0/2242268546 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c4032440 con 0x7fe6cc0cac50 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.297+0000 7fe6d159a640 1 -- 172.21.15.113:0/2242268546 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4032810 con 0x7fe6cc0cac50 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.297+0000 7fe6d359e640 1 -- 172.21.15.113:0/2242268546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0cac50 msgr2=0x7fe6cc0cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.297+0000 7fe6d359e640 1 --2- 172.21.15.113:0/2242268546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc0cb0b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40097a0 tx=0x7fe6c402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.298+0000 7fe6d359e640 1 -- 172.21.15.113:0/2242268546 shutdown_connections 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.298+0000 7fe6d359e640 1 --2- 172.21.15.113:0/2242268546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc0cb0b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.298+0000 7fe6d359e640 1 -- 172.21.15.113:0/2242268546 >> 172.21.15.113:0/2242268546 conn(0x7fe6cc0cfaf0 msgr2=0x7fe6cc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.298+0000 7fe6d359e640 1 -- 172.21.15.113:0/2242268546 shutdown_connections 2024-04-25T00:00:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.298+0000 7fe6d359e640 1 -- 172.21.15.113:0/2242268546 wait complete. 2024-04-25T00:00:50.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.299+0000 7fe6d359e640 1 Processor -- start 2024-04-25T00:00:50.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.299+0000 7fe6d359e640 1 -- start start 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc06e6f0 0x7fe6cc1f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6cc162950 0x7fe6cc0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc072700 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc072a70 con 0x7fe6cc0cac50 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6cc15e530 con 0x7fe6cc162950 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc06e6f0 0x7fe6cc1f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc06e6f0 0x7fe6cc1f05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54404/0 (socket says 172.21.15.113:54404) 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 -- 172.21.15.113:0/3943768120 learned_addr learned my addr 172.21.15.113:0/3943768120 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:50.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d1d9b640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 -- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6cc162950 msgr2=0x7fe6cc0c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6cc162950 0x7fe6cc0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 -- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0cac50 msgr2=0x7fe6cc1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 -- 172.21.15.113:0/3943768120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc0c4580 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d1d9b640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d259c640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc06e6f0 0x7fe6cc1f05f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000c500 tx=0x7fe6c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6beffd640 1 -- 172.21.15.113:0/3943768120 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0017070 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6beffd640 1 -- 172.21.15.113:0/3943768120 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c0005ce0 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc1f0c20 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6beffd640 1 -- 172.21.15.113:0/3943768120 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0007330 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.300+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc0c3560 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.301+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc0691a0 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.302+0000 7fe6beffd640 1 -- 172.21.15.113:0/3943768120 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6c001a070 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.302+0000 7fe6beffd640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6b006be30 0x7fe6b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.302+0000 7fe6d1d9b640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6b006be30 0x7fe6b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.302+0000 7fe6beffd640 1 -- 172.21.15.113:0/3943768120 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe6c008a240 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.303+0000 7fe6d1d9b640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6b006be30 0x7fe6b006e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4034000 tx=0x7fe6c402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.307+0000 7fe6beffd640 1 -- 172.21.15.113:0/3943768120 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6c0058d40 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.481+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6cc0ca140 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.482+0000 7fe6beffd640 1 -- 172.21.15.113:0/3943768120 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16674 (secure 0 0 0) 0x7fe6c0058640 con 0x7fe6cc06e6f0 2024-04-25T00:00:50.482 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:50.482 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":45,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:49.512132+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:47.724388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:48.476831+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:50.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6b006be30 msgr2=0x7fe6b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6b006be30 0x7fe6b006e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4034000 tx=0x7fe6c402e000 comp rx=0 tx=0).stop 2024-04-25T00:00:50.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc06e6f0 msgr2=0x7fe6cc1f05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc06e6f0 0x7fe6cc1f05f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000c500 tx=0x7fe6c000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 shutdown_connections 2024-04-25T00:00:50.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe6b006be30 0x7fe6b006e2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6cc0cac50 0x7fe6cc1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6cc162950 0x7fe6cc0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 --2- 172.21.15.113:0/3943768120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc06e6f0 0x7fe6cc1f05f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 >> 172.21.15.113:0/3943768120 conn(0x7fe6cc0cfaf0 msgr2=0x7fe6cc0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 shutdown_connections 2024-04-25T00:00:50.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.484+0000 7fe6d359e640 1 -- 172.21.15.113:0/3943768120 wait complete. 2024-04-25T00:00:50.495 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:50.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.591+0000 7f235d0f5640 1 Processor -- start 2024-04-25T00:00:50.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.591+0000 7f235d0f5640 1 -- start start 2024-04-25T00:00:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f235d0f5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f23580b3fd0 0x7f235806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f235d0f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f235806e510 0x7f23580cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f235d0f5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2358144cf0 con 0x7f2358144910 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f235d0f5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f23580b2f30 con 0x7f23580b3fd0 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f235d0f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f23580b3210 con 0x7f235806e510 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f2356d9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2358144910 0x7f23580b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:32878/0 (socket says 172.21.15.113:32878) 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f2356d9d640 1 -- 172.21.15.113:0/2207095448 learned_addr learned my addr 172.21.15.113:0/2207095448 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f235759e640 1 --2- 172.21.15.113:0/2207095448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f235806e510 0x7f23580cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.592+0000 7f235659c640 1 --2- 172.21.15.113:0/2207095448 >> v2:172.21.15.178:3300/0 conn(0x7f23580b3fd0 0x7f235806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.593+0000 7f2355d9b640 1 -- 172.21.15.113:0/2207095448 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2801669142 0 0) 0x7f2358144cf0 con 0x7f2358144910 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.593+0000 7f2355d9b640 1 -- 172.21.15.113:0/2207095448 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2344003520 con 0x7f2358144910 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.593+0000 7f235759e640 1 -- 172.21.15.113:0/2207095448 >> v2:172.21.15.178:3300/0 conn(0x7f23580b3fd0 msgr2=0x7f235806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.593+0000 7f235759e640 1 --2- 172.21.15.113:0/2207095448 >> v2:172.21.15.178:3300/0 conn(0x7f23580b3fd0 0x7f235806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.593+0000 7f235759e640 1 -- 172.21.15.113:0/2207095448 >> v1:172.21.15.178:6789/0 conn(0x7f2358144910 legacy=0x7f23580b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.593+0000 7f235759e640 1 -- 172.21.15.113:0/2207095448 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23580b34f0 con 0x7f235806e510 2024-04-25T00:00:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.593+0000 7f235659c640 1 --2- 172.21.15.113:0/2207095448 >> v2:172.21.15.178:3300/0 conn(0x7f23580b3fd0 0x7f235806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235759e640 1 --2- 172.21.15.113:0/2207095448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f235806e510 0x7f23580cd500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23480097d0 tx=0x7f234802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24bbdb373f7b2be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f2355d9b640 1 -- 172.21.15.113:0/2207095448 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348037070 con 0x7f235806e510 2024-04-25T00:00:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f2355d9b640 1 -- 172.21.15.113:0/2207095448 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2348032440 con 0x7f235806e510 2024-04-25T00:00:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f2355d9b640 1 -- 172.21.15.113:0/2207095448 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23480327f0 con 0x7f235806e510 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 -- 172.21.15.113:0/2207095448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f235806e510 msgr2=0x7f23580cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 --2- 172.21.15.113:0/2207095448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f235806e510 0x7f23580cd500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23480097d0 tx=0x7f234802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 -- 172.21.15.113:0/2207095448 shutdown_connections 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 --2- 172.21.15.113:0/2207095448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f235806e510 0x7f23580cd500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 --2- 172.21.15.113:0/2207095448 >> v2:172.21.15.178:3300/0 conn(0x7f23580b3fd0 0x7f235806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 -- 172.21.15.113:0/2207095448 >> 172.21.15.113:0/2207095448 conn(0x7f23580b1a50 msgr2=0x7f23580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 -- 172.21.15.113:0/2207095448 shutdown_connections 2024-04-25T00:00:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.594+0000 7f235d0f5640 1 -- 172.21.15.113:0/2207095448 wait complete. 2024-04-25T00:00:50.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.596+0000 7f235d0f5640 1 Processor -- start 2024-04-25T00:00:50.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.596+0000 7f235d0f5640 1 -- start start 2024-04-25T00:00:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.596+0000 7f235d0f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f235806e510 0x7f23580abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.596+0000 7f235d0f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580b3fd0 0x7f23580ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235d0f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2358144910 0x7f23580adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235d0f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23580b4da0 con 0x7f2358144910 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235d0f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23580b4440 con 0x7f235806e510 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235d0f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f23580b4f20 con 0x7f23580b3fd0 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2358144910 0x7f23580adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2358144910 0x7f23580adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54406/0 (socket says 172.21.15.113:54406) 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 -- 172.21.15.113:0/3270095103 learned_addr learned my addr 172.21.15.113:0/3270095103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f2356d9d640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f235806e510 0x7f23580abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235659c640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580b3fd0 0x7f23580ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 -- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580b3fd0 msgr2=0x7f23580ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580b3fd0 0x7f23580ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 -- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f235806e510 msgr2=0x7f23580abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f235806e510 0x7f23580abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 -- 172.21.15.113:0/3270095103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235815dd40 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235759e640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2358144910 0x7f23580adf90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f23480098f0 tx=0x7f23480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f234f7fe640 1 -- 172.21.15.113:0/3270095103 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348037040 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f234f7fe640 1 -- 172.21.15.113:0/3270095103 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2348042070 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23581e6620 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f234f7fe640 1 -- 172.21.15.113:0/3270095103 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348006e10 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.597+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581e6ec0 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.598+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23580b6070 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.599+0000 7f234f7fe640 1 -- 172.21.15.113:0/3270095103 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2348032830 con 0x7f2358144910 2024-04-25T00:00:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.599+0000 7f234f7fe640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f233806be30 0x7f233806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.599+0000 7f2356d9d640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f233806be30 0x7f233806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.599+0000 7f234f7fe640 1 -- 172.21.15.113:0/3270095103 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f23480ab4f0 con 0x7f2358144910 2024-04-25T00:00:50.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.600+0000 7f2356d9d640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f233806be30 0x7f233806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2350005c00 tx=0x7f2350009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.604+0000 7f234f7fe640 1 -- 172.21.15.113:0/3270095103 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23580b6070 con 0x7f2358144910 2024-04-25T00:00:50.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.776+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23580691a0 con 0x7f2358144910 2024-04-25T00:00:50.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.777+0000 7f234f7fe640 1 -- 172.21.15.113:0/3270095103 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16674 (secure 0 0 0) 0x7f2348079a00 con 0x7f2358144910 2024-04-25T00:00:50.777 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:50.778 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":45,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:49.512132+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:47.724388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:48.476831+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:50.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f233806be30 msgr2=0x7f233806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f233806be30 0x7f233806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2350005c00 tx=0x7f2350009040 comp rx=0 tx=0).stop 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2358144910 msgr2=0x7f23580adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2358144910 0x7f23580adf90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f23480098f0 tx=0x7f23480046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 shutdown_connections 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f233806be30 0x7f233806e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2358144910 0x7f23580adf90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580b3fd0 0x7f23580ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 --2- 172.21.15.113:0/3270095103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f235806e510 0x7f23580abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 >> 172.21.15.113:0/3270095103 conn(0x7f23580b1a50 msgr2=0x7f235815f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 shutdown_connections 2024-04-25T00:00:50.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.779+0000 7f235d0f5640 1 -- 172.21.15.113:0/3270095103 wait complete. 2024-04-25T00:00:50.791 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:00:50.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.889+0000 7f6dcfbb7640 1 Processor -- start 2024-04-25T00:00:50.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.890+0000 7f6dcfbb7640 1 -- start start 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.890+0000 7f6dcfbb7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.890+0000 7f6dcfbb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b3080 0x7f6dc80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.890+0000 7f6dcfbb7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc815f4f0 con 0x7f6dc815e6b0 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.890+0000 7f6dcfbb7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc80cde00 con 0x7f6dc815e2d0 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.890+0000 7f6dcfbb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc815f7d0 con 0x7f6dc80b3080 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd152640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b3080 0x7f6dc80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd953640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd152640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b3080 0x7f6dc80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:44840/0 (socket says 172.21.15.113:44840) 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd152640 1 -- 172.21.15.113:0/2230807807 learned_addr learned my addr 172.21.15.113:0/2230807807 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd152640 1 -- 172.21.15.113:0/2230807807 >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 msgr2=0x7f6dc815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd152640 1 --2- 172.21.15.113:0/2230807807 >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd152640 1 -- 172.21.15.113:0/2230807807 >> v1:172.21.15.178:6789/0 conn(0x7f6dc815e6b0 legacy=0x7f6dc8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.891+0000 7f6dcd152640 1 -- 172.21.15.113:0/2230807807 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc815fab0 con 0x7f6dc80b3080 2024-04-25T00:00:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcd152640 1 --2- 172.21.15.113:0/2230807807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b3080 0x7f6dc80b4330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f49e36e3abf6651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcc951640 1 -- 172.21.15.113:0/2230807807 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4037070 con 0x7f6dc80b3080 2024-04-25T00:00:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcc951640 1 -- 172.21.15.113:0/2230807807 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4032440 con 0x7f6dc80b3080 2024-04-25T00:00:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcc951640 1 -- 172.21.15.113:0/2230807807 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4032810 con 0x7f6dc80b3080 2024-04-25T00:00:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2230807807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b3080 msgr2=0x7f6dc80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2230807807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b3080 0x7f6dc80b4330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2230807807 shutdown_connections 2024-04-25T00:00:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2230807807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b3080 0x7f6dc80b4330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2230807807 >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.892+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2230807807 >> 172.21.15.113:0/2230807807 conn(0x7f6dc806b0f0 msgr2=0x7f6dc806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.893+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2230807807 shutdown_connections 2024-04-25T00:00:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.893+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2230807807 wait complete. 2024-04-25T00:00:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 Processor -- start 2024-04-25T00:00:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 -- start start 2024-04-25T00:00:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80b3080 0x7f6dc806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc815e2d0 0x7f6dc81f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc815e6b0 0x7f6dc814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc815f2b0 con 0x7f6dc815e6b0 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc815f130 con 0x7f6dc80b3080 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dcfbb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc80cf2a0 con 0x7f6dc815e2d0 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dce154640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc815e6b0 0x7f6dc814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dce154640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc815e6b0 0x7f6dc814fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54414/0 (socket says 172.21.15.113:54414) 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.894+0000 7f6dce154640 1 -- 172.21.15.113:0/2592822975 learned_addr learned my addr 172.21.15.113:0/2592822975 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dcd152640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc815e2d0 0x7f6dc81f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dcd953640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80b3080 0x7f6dc806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dce154640 1 -- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc815e2d0 msgr2=0x7f6dc81f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dce154640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc815e2d0 0x7f6dc81f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dce154640 1 -- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80b3080 msgr2=0x7f6dc806d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:50.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dce154640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80b3080 0x7f6dc806d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dce154640 1 -- 172.21.15.113:0/2592822975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc81f49f0 con 0x7f6dc815e6b0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dce154640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc815e6b0 0x7f6dc814fc50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000c6b0 tx=0x7f6dc000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/2592822975 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0003b70 con 0x7f6dc815e6b0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/2592822975 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0003d30 con 0x7f6dc815e6b0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc81f4cd0 con 0x7f6dc815e6b0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/2592822975 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0005310 con 0x7f6dc815e6b0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.895+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc8151b40 con 0x7f6dc815e6b0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.896+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc81f4ea0 con 0x7f6dc815e6b0 2024-04-25T00:00:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.896+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/2592822975 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dc00054b0 con 0x7f6dc815e6b0 2024-04-25T00:00:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.897+0000 7f6dbe7fc640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.897+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/2592822975 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f6dc008ac50 con 0x7f6dc815e6b0 2024-04-25T00:00:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.897+0000 7f6dcd953640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.897+0000 7f6dcd953640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8159c80 tx=0x7f6db8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:50.901+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/2592822975 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6dc0058ed0 con 0x7f6dc815e6b0 2024-04-25T00:00:51.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.109+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6dc80691a0 con 0x7f6dc815e6b0 2024-04-25T00:00:51.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.176+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/2592822975 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) ==== 118+0+0 (secure 0 0 0) 0x7f6dc005d1d0 con 0x7f6dc815e6b0 2024-04-25T00:00:51.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 msgr2=0x7f6dac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8159c80 tx=0x7f6db8009190 comp rx=0 tx=0).stop 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc815e6b0 msgr2=0x7f6dc814fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc815e6b0 0x7f6dc814fc50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000c6b0 tx=0x7f6dc000cb70 comp rx=0 tx=0).stop 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 shutdown_connections 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc815e6b0 0x7f6dc814fc50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc815e2d0 0x7f6dc81f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 --2- 172.21.15.113:0/2592822975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80b3080 0x7f6dc806d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 >> 172.21.15.113:0/2592822975 conn(0x7f6dc806b0f0 msgr2=0x7f6dc80b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 shutdown_connections 2024-04-25T00:00:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.178+0000 7f6dcfbb7640 1 -- 172.21.15.113:0/2592822975 wait complete. 2024-04-25T00:00:51.189 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:00:51.189 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:00:51.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.288+0000 7f691f1c8640 1 Processor -- start 2024-04-25T00:00:51.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.288+0000 7f691f1c8640 1 -- start start 2024-04-25T00:00:51.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.289+0000 7f691f1c8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f69180b6100 0x7f69180b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.289+0000 7f691f1c8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69180d2e00 con 0x7f691815e510 2024-04-25T00:00:51.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.289+0000 7f691f1c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69180d30e0 con 0x7f69180b5ca0 2024-04-25T00:00:51.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.289+0000 7f691f1c8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f69180d33c0 con 0x7f69180b6100 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.289+0000 7f691cf64640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f691815e510 0x7f69180d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43812/0 (socket says 172.21.15.113:43812) 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.289+0000 7f691cf64640 1 -- 172.21.15.113:0/1954246472 learned_addr learned my addr 172.21.15.113:0/1954246472 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691d765640 1 --2- 172.21.15.113:0/1954246472 >> v2:172.21.15.178:3300/0 conn(0x7f69180b6100 0x7f69180b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2818490136 0 0) 0x7f69180d2e00 con 0x7f691815e510 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6900003520 con 0x7f691815e510 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 775336274 0 0) 0x7f6900003520 con 0x7f691815e510 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 >> v2:172.21.15.178:3300/0 conn(0x7f69180b6100 msgr2=0x7f69180b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 --2- 172.21.15.113:0/1954246472 >> v2:172.21.15.178:3300/0 conn(0x7f69180b6100 0x7f69180b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 >> v1:172.21.15.178:6789/0 conn(0x7f69180b5ca0 legacy=0x7f69180b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69180d36a0 con 0x7f691815e510 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6908001670 con 0x7f691815e510 2024-04-25T00:00:51.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 >> v1:172.21.15.113:6789/0 conn(0x7f691815e510 legacy=0x7f69180d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6900004800 0x7f6900004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000051c0 0x7f690000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f69180d33c0 con 0x7f69000039b0 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f690000c160 con 0x7f69000051c0 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f690000c360 con 0x7f6900004800 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691cf64640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691d765640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000051c0 0x7f690000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f6917fff640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6900004800 0x7f6900004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691cf64640 1 -- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6900004800 msgr2=0x7f6900004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691cf64640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6900004800 0x7f6900004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691cf64640 1 -- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000051c0 msgr2=0x7f690000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:51.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691cf64640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000051c0 0x7f690000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691cf64640 1 -- 172.21.15.113:0/1954246472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6900003520 con 0x7f69000039b0 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691d765640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000051c0 0x7f690000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.290+0000 7f691d765640 1 -- 172.21.15.113:0/1954246472 reap_dead start 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.291+0000 7f691cf64640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f690800c240 tx=0x7f6900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8948d14bf260621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.291+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908007d10 con 0x7f69000039b0 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.291+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69080058d0 con 0x7f69000039b0 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.291+0000 7f69177fe640 1 -- 172.21.15.113:0/1954246472 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908005d40 con 0x7f69000039b0 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.292+0000 7f691f1c8640 1 -- 172.21.15.113:0/1954246472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69000039b0 msgr2=0x7f6900003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.292+0000 7f691f1c8640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f690800c240 tx=0x7f6900005830 comp rx=0 tx=0).stop 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.292+0000 7f691f1c8640 1 -- 172.21.15.113:0/1954246472 shutdown_connections 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.292+0000 7f691f1c8640 1 --2- 172.21.15.113:0/1954246472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69000039b0 0x7f6900003dd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.292+0000 7f691f1c8640 1 -- 172.21.15.113:0/1954246472 >> 172.21.15.113:0/1954246472 conn(0x7f691806b720 msgr2=0x7f69180d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:51.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.292+0000 7f691f1c8640 1 -- 172.21.15.113:0/1954246472 shutdown_connections 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.292+0000 7f691f1c8640 1 -- 172.21.15.113:0/1954246472 wait complete. 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.293+0000 7f691f1c8640 1 Processor -- start 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.293+0000 7f691f1c8640 1 -- start start 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691f1c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000039b0 0x7f69181542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691f1c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181f7780 0x7f69181f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691f1c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6918149050 0x7f69181494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691f1c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f69180b6510 con 0x7f69181f7780 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691f1c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69180d3ea0 con 0x7f69000039b0 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691f1c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f69180b9dd0 con 0x7f6918149050 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181f7780 0x7f69181f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181f7780 0x7f69181f7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54432/0 (socket says 172.21.15.113:54432) 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 -- 172.21.15.113:0/3940266833 learned_addr learned my addr 172.21.15.113:0/3940266833 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691cf64640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000039b0 0x7f69181542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f691d765640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6918149050 0x7f69181494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 -- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6918149050 msgr2=0x7f69181494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6918149050 0x7f69181494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 -- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000039b0 msgr2=0x7f69181542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000039b0 0x7f69181542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 -- 172.21.15.113:0/3940266833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69181f8240 con 0x7f69181f7780 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f6917fff640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181f7780 0x7f69181f7c00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f690c00e4d0 tx=0x7f690c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.294+0000 7f69157fa640 1 -- 172.21.15.113:0/3940266833 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c0053a0 con 0x7f69181f7780 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.295+0000 7f69157fa640 1 -- 172.21.15.113:0/3940266833 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c018070 con 0x7f69181f7780 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.295+0000 7f69157fa640 1 -- 172.21.15.113:0/3940266833 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c003810 con 0x7f69181f7780 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.295+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69181f8520 con 0x7f69181f7780 2024-04-25T00:00:51.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.295+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69180cb7e0 con 0x7f69181f7780 2024-04-25T00:00:51.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.296+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69180691a0 con 0x7f69181f7780 2024-04-25T00:00:51.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.296+0000 7f69157fa640 1 -- 172.21.15.113:0/3940266833 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f690c0039b0 con 0x7f69181f7780 2024-04-25T00:00:51.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.297+0000 7f69157fa640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f690406c030 0x7f690406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.297+0000 7f69157fa640 1 -- 172.21.15.113:0/3940266833 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f690c0035f0 con 0x7f69181f7780 2024-04-25T00:00:51.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.297+0000 7f691cf64640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f690406c030 0x7f690406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.298+0000 7f691cf64640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f690406c030 0x7f690406e4f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6908033000 tx=0x7f690802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.302+0000 7f69157fa640 1 -- 172.21.15.113:0/3940266833 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f690c059320 con 0x7f69181f7780 2024-04-25T00:00:51.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:51.511+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f69180cb2d0 con 0x7f69181f7780 2024-04-25T00:00:51.949 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:00:51.952 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:00:52.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.177+0000 7f69157fa640 1 -- 172.21.15.113:0/3940266833 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) ==== 122+0+0 (secure 0 0 0) 0x7f690c058c20 con 0x7f69181f7780 2024-04-25T00:00:52.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f690406c030 msgr2=0x7f690406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f690406c030 0x7f690406e4f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6908033000 tx=0x7f690802e000 comp rx=0 tx=0).stop 2024-04-25T00:00:52.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181f7780 msgr2=0x7f69181f7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181f7780 0x7f69181f7c00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f690c00e4d0 tx=0x7f690c00e990 comp rx=0 tx=0).stop 2024-04-25T00:00:52.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 shutdown_connections 2024-04-25T00:00:52.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f690406c030 0x7f690406e4f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6918149050 0x7f69181494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181f7780 0x7f69181f7c00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 --2- 172.21.15.113:0/3940266833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69000039b0 0x7f69181542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 >> 172.21.15.113:0/3940266833 conn(0x7f691806b720 msgr2=0x7f691815c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 shutdown_connections 2024-04-25T00:00:52.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.180+0000 7f691f1c8640 1 -- 172.21.15.113:0/3940266833 wait complete. 2024-04-25T00:00:52.192 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:00:52.193 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:52.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.289+0000 7f1caa0a3640 1 Processor -- start 2024-04-25T00:00:52.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.289+0000 7f1caa0a3640 1 -- start start 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1caa0a3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ca4072ea0 0x7f1ca4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1caa0a3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca40b3af0 0x7f1ca40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1caa0a3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca40b2d50 con 0x7f1ca415a6b0 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1caa0a3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca4072a70 con 0x7f1ca4072ea0 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1caa0a3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca40b3010 con 0x7f1ca40b3af0 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1c9bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ca4072ea0 0x7f1ca4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1c9bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ca4072ea0 0x7f1ca4071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54436/0 (socket says 172.21.15.113:54436) 2024-04-25T00:00:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1c9bfff640 1 -- 172.21.15.113:0/3626213428 learned_addr learned my addr 172.21.15.113:0/3626213428 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1ca3fff640 1 --2- 172.21.15.113:0/3626213428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca40b3af0 0x7f1ca40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1c9bfff640 1 -- 172.21.15.113:0/3626213428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca40b3af0 msgr2=0x7f1ca40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1c9bfff640 1 --2- 172.21.15.113:0/3626213428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca40b3af0 0x7f1ca40b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1c9bfff640 1 -- 172.21.15.113:0/3626213428 >> v1:172.21.15.178:6789/0 conn(0x7f1ca415a6b0 legacy=0x7f1ca4070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1c9bfff640 1 -- 172.21.15.113:0/3626213428 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca40b32f0 con 0x7f1ca4072ea0 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.290+0000 7f1ca3fff640 1 --2- 172.21.15.113:0/3626213428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca40b3af0 0x7f1ca40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.291+0000 7f1c9bfff640 1 --2- 172.21.15.113:0/3626213428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca4072ea0 0x7f1ca4071410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00bb10 tx=0x7f1c8c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a7861fbe0f8a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.291+0000 7f1ca2ffd640 1 -- 172.21.15.113:0/3626213428 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c036020 con 0x7f1ca4072ea0 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.291+0000 7f1ca2ffd640 1 -- 172.21.15.113:0/3626213428 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8c032820 con 0x7f1ca4072ea0 2024-04-25T00:00:52.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.291+0000 7f1ca2ffd640 1 -- 172.21.15.113:0/3626213428 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c032bf0 con 0x7f1ca4072ea0 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.291+0000 7f1caa0a3640 1 -- 172.21.15.113:0/3626213428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca4072ea0 msgr2=0x7f1ca4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.291+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/3626213428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca4072ea0 0x7f1ca4071410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00bb10 tx=0x7f1c8c02c920 comp rx=0 tx=0).stop 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.292+0000 7f1caa0a3640 1 -- 172.21.15.113:0/3626213428 shutdown_connections 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.292+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/3626213428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca40b3af0 0x7f1ca40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.292+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/3626213428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca4072ea0 0x7f1ca4071410 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.292+0000 7f1caa0a3640 1 -- 172.21.15.113:0/3626213428 >> 172.21.15.113:0/3626213428 conn(0x7f1ca40b1a50 msgr2=0x7f1ca40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.292+0000 7f1caa0a3640 1 -- 172.21.15.113:0/3626213428 shutdown_connections 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.292+0000 7f1caa0a3640 1 -- 172.21.15.113:0/3626213428 wait complete. 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 Processor -- start 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 -- start start 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca4072ea0 0x7f1ca414e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca40b3af0 0x7f1ca4148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca415a6b0 0x7f1ca41e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca415aa90 con 0x7f1ca415a6b0 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4072720 con 0x7f1ca40b3af0 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.293+0000 7f1caa0a3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca415ac10 con 0x7f1ca4072ea0 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca415a6b0 0x7f1ca41e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca415a6b0 0x7f1ca41e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54450/0 (socket says 172.21.15.113:54450) 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 -- 172.21.15.113:0/2840387862 learned_addr learned my addr 172.21.15.113:0/2840387862 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1c9bfff640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca40b3af0 0x7f1ca4148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca37fe640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca4072ea0 0x7f1ca414e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 -- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca4072ea0 msgr2=0x7f1ca414e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca4072ea0 0x7f1ca414e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 -- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca40b3af0 msgr2=0x7f1ca4148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca40b3af0 0x7f1ca4148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 -- 172.21.15.113:0/2840387862 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca41488b0 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1c9bfff640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca40b3af0 0x7f1ca4148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca3fff640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca415a6b0 0x7f1ca41e42e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00ce40 tx=0x7f1c9c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca0ff9640 1 -- 172.21.15.113:0/2840387862 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c01a070 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca0ff9640 1 -- 172.21.15.113:0/2840387862 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c00d840 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca4149900 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1ca0ff9640 1 -- 172.21.15.113:0/2840387862 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00dc30 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.294+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca41e1db0 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.295+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca40691a0 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.296+0000 7f1ca0ff9640 1 -- 172.21.15.113:0/2840387862 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c9c01d070 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.296+0000 7f1ca0ff9640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c8406be30 0x7f1c8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.296+0000 7f1ca37fe640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c8406be30 0x7f1c8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.296+0000 7f1ca0ff9640 1 -- 172.21.15.113:0/2840387862 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1c9c08b5e0 con 0x7f1ca415a6b0 2024-04-25T00:00:52.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.296+0000 7f1ca37fe640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c8406be30 0x7f1c8406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c94007fd0 tx=0x7f1c94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.300+0000 7f1ca0ff9640 1 -- 172.21.15.113:0/2840387862 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ca41488b0 con 0x7f1ca415a6b0 2024-04-25T00:00:52.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.509+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ca40b4b40 con 0x7f1ca415a6b0 2024-04-25T00:00:52.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.510+0000 7f1ca0ff9640 1 -- 172.21.15.113:0/2840387862 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7f1c9c05a020 con 0x7f1ca415a6b0 2024-04-25T00:00:52.510 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:52.510 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:00:52.511 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:00:52.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c8406be30 msgr2=0x7f1c8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c8406be30 0x7f1c8406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c94007fd0 tx=0x7f1c94009040 comp rx=0 tx=0).stop 2024-04-25T00:00:52.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca415a6b0 msgr2=0x7f1ca41e42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca415a6b0 0x7f1ca41e42e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00ce40 tx=0x7f1c9c00a710 comp rx=0 tx=0).stop 2024-04-25T00:00:52.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 shutdown_connections 2024-04-25T00:00:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c8406be30 0x7f1c8406e2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca40b3af0 0x7f1ca4148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ca415a6b0 0x7f1ca41e42e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 --2- 172.21.15.113:0/2840387862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca4072ea0 0x7f1ca414e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 >> 172.21.15.113:0/2840387862 conn(0x7f1ca40b1a50 msgr2=0x7f1ca40b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 shutdown_connections 2024-04-25T00:00:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.512+0000 7f1caa0a3640 1 -- 172.21.15.113:0/2840387862 wait complete. 2024-04-25T00:00:52.523 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.619+0000 7f0fef14f640 1 Processor -- start 2024-04-25T00:00:52.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.619+0000 7f0fef14f640 1 -- start start 2024-04-25T00:00:52.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fef14f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 0x7f0fe80d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fef14f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe80d3de0 con 0x7f0fe80d0b00 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fef14f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe80d2800 con 0x7f0fe80d2290 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fef14f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe80d3410 con 0x7f0fe815e320 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fed6ec640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0fe80d0b00 0x7f0fe80b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43828/0 (socket says 172.21.15.113:43828) 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fed6ec640 1 -- 172.21.15.113:0/268594838 learned_addr learned my addr 172.21.15.113:0/268594838 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4162937226 0 0) 0x7f0fe80d3de0 con 0x7f0fe80d0b00 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fd4003520 con 0x7f0fe80d0b00 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0feceeb640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 0x7f0fe80d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3776946397 0 0) 0x7f0fd4003520 con 0x7f0fe80d0b00 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.620+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 msgr2=0x7f0fe80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 0x7f0fe80d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 >> v1:172.21.15.178:6789/0 conn(0x7f0fe80d2290 legacy=0x7f0fe80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe80d2ae0 con 0x7f0fe80d0b00 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0fd8001670 con 0x7f0fe80d0b00 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 >> v1:172.21.15.113:6789/0 conn(0x7f0fe80d0b00 legacy=0x7f0fe80b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd40039e0 0x7f0fd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 0x7f0fd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd4004d40 0x7f0fd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80d3410 con 0x7f0fd40039e0 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0feceeb640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd40039e0 0x7f0fd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd400bb80 con 0x7f0fd4004d40 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd400bd80 con 0x7f0fe815e320 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fed6ec640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd4004d40 0x7f0fd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0fe7fff640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 0x7f0fd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0feceeb640 1 -- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 msgr2=0x7f0fd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0feceeb640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe815e320 0x7f0fd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0feceeb640 1 -- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd4004d40 msgr2=0x7f0fd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0feceeb640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd4004d40 0x7f0fd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.621+0000 7f0feceeb640 1 -- 172.21.15.113:0/268594838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd4003520 con 0x7f0fd40039e0 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0feceeb640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd40039e0 0x7f0fd4003de0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007360 tx=0x7f0fd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83f36b23b6bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0034020 con 0x7f0fd40039e0 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe0030670 con 0x7f0fd40039e0 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fe77fe640 1 -- 172.21.15.113:0/268594838 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0009be0 con 0x7f0fd40039e0 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fef14f640 1 -- 172.21.15.113:0/268594838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd40039e0 msgr2=0x7f0fd4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fef14f640 1 --2- 172.21.15.113:0/268594838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fd40039e0 0x7f0fd4003de0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007360 tx=0x7f0fd4005830 comp rx=0 tx=0).stop 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fed6ec640 1 -- 172.21.15.113:0/268594838 reap_dead start 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fef14f640 1 -- 172.21.15.113:0/268594838 shutdown_connections 2024-04-25T00:00:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fef14f640 1 -- 172.21.15.113:0/268594838 >> 172.21.15.113:0/268594838 conn(0x7f0fe80b5c20 msgr2=0x7f0fe80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fef14f640 1 -- 172.21.15.113:0/268594838 shutdown_connections 2024-04-25T00:00:52.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.622+0000 7f0fef14f640 1 -- 172.21.15.113:0/268594838 wait complete. 2024-04-25T00:00:52.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.624+0000 7f0fef14f640 1 Processor -- start 2024-04-25T00:00:52.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.624+0000 7f0fef14f640 1 -- start start 2024-04-25T00:00:52.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd40039e0 0x7f0fe81f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fe8158610 0x7f0fe815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe80b8ca0 0x7f0fe80b9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe815e700 con 0x7f0fe8158610 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80d30a0 con 0x7f0fd40039e0 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe815e880 con 0x7f0fe80b8ca0 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fe8158610 0x7f0fe815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fe8158610 0x7f0fe815ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54478/0 (socket says 172.21.15.113:54478) 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 -- 172.21.15.113:0/2805611924 learned_addr learned my addr 172.21.15.113:0/2805611924 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0feceeb640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd40039e0 0x7f0fe81f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fed6ec640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe80b8ca0 0x7f0fe80b9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 -- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe80b8ca0 msgr2=0x7f0fe80b9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe80b8ca0 0x7f0fe80b9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 -- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd40039e0 msgr2=0x7f0fe81f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd40039e0 0x7f0fe81f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 -- 172.21.15.113:0/2805611924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe81f4fa0 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe7fff640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fe8158610 0x7f0fe815ab60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e4e0 tx=0x7f0fd000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe57fa640 1 -- 172.21.15.113:0/2805611924 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00135f0 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe57fa640 1 -- 172.21.15.113:0/2805611924 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd001a070 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fe57fa640 1 -- 172.21.15.113:0/2805611924 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00139f0 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe81f46a0 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.625+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe81f4c60 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.626+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe80691a0 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.627+0000 7f0fe57fa640 1 -- 172.21.15.113:0/2805611924 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fd0013b90 con 0x7f0fe8158610 2024-04-25T00:00:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.627+0000 7f0fe57fa640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fc806be30 0x7f0fc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.627+0000 7f0fe57fa640 1 -- 172.21.15.113:0/2805611924 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f0fd008af30 con 0x7f0fe8158610 2024-04-25T00:00:52.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.627+0000 7f0feceeb640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fc806be30 0x7f0fc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.628+0000 7f0feceeb640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fc806be30 0x7f0fc806e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0035000 tx=0x7f0fe000b030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.631+0000 7f0fe57fa640 1 -- 172.21.15.113:0/2805611924 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0fd0024020 con 0x7f0fe8158610 2024-04-25T00:00:52.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.808+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fe80b49c0 con 0x7f0fe8158610 2024-04-25T00:00:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.809+0000 7f0fe57fa640 1 -- 172.21.15.113:0/2805611924 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16661 (secure 0 0 0) 0x7f0fd00598e0 con 0x7f0fe8158610 2024-04-25T00:00:52.809 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:52.809 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":46,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:00:51.157092+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:47.724388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:48.476831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fc806be30 msgr2=0x7f0fc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fc806be30 0x7f0fc806e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0035000 tx=0x7f0fe000b030 comp rx=0 tx=0).stop 2024-04-25T00:00:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fe8158610 msgr2=0x7f0fe815ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fe8158610 0x7f0fe815ab60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e4e0 tx=0x7f0fd000e9a0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 shutdown_connections 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0fc806be30 0x7f0fc806e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fe80b8ca0 0x7f0fe80b9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fe8158610 0x7f0fe815ab60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 --2- 172.21.15.113:0/2805611924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd40039e0 0x7f0fe81f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 >> 172.21.15.113:0/2805611924 conn(0x7f0fe80b5c20 msgr2=0x7f0fe80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 shutdown_connections 2024-04-25T00:00:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.811+0000 7f0fef14f640 1 -- 172.21.15.113:0/2805611924 wait complete. 2024-04-25T00:00:52.822 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:00:52.823 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:52.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.919+0000 7fad78f70640 1 Processor -- start 2024-04-25T00:00:52.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.919+0000 7fad78f70640 1 -- start start 2024-04-25T00:00:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad78f70640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad74144910 0x7fad740b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad78f70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5240 0x7fad7406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad78f70640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fad740b5680 0x7fad7406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad78f70640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fad740b4510 con 0x7fad74144910 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad78f70640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fad74144cf0 con 0x7fad740b5680 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad78f70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad740b2f30 con 0x7fad740b5240 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad74144910 0x7fad740b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad74144910 0x7fad740b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54490/0 (socket says 172.21.15.113:54490) 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 -- 172.21.15.113:0/3283239419 learned_addr learned my addr 172.21.15.113:0/3283239419 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad72d9d640 1 --2- 172.21.15.113:0/3283239419 >> v2:172.21.15.178:3300/0 conn(0x7fad740b5680 0x7fad7406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad71d9b640 1 --2- 172.21.15.113:0/3283239419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5240 0x7fad7406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 -- 172.21.15.113:0/3283239419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5240 msgr2=0x7fad7406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 --2- 172.21.15.113:0/3283239419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5240 0x7fad7406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 -- 172.21.15.113:0/3283239419 >> v2:172.21.15.178:3300/0 conn(0x7fad740b5680 msgr2=0x7fad7406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 --2- 172.21.15.113:0/3283239419 >> v2:172.21.15.178:3300/0 conn(0x7fad740b5680 0x7fad7406e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad7259c640 1 -- 172.21.15.113:0/3283239419 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740b3210 con 0x7fad74144910 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.920+0000 7fad72d9d640 1 --2- 172.21.15.113:0/3283239419 >> v2:172.21.15.178:3300/0 conn(0x7fad740b5680 0x7fad7406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.921+0000 7fad7259c640 1 --2- 172.21.15.113:0/3283239419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad74144910 0x7fad740b3fd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fad74062820 tx=0x7fad6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea53d42021cb73bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.921+0000 7fad7159a640 1 -- 172.21.15.113:0/3283239419 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64036070 con 0x7fad74144910 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.921+0000 7fad7159a640 1 -- 172.21.15.113:0/3283239419 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad64031600 con 0x7fad74144910 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.921+0000 7fad7159a640 1 -- 172.21.15.113:0/3283239419 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640319d0 con 0x7fad74144910 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 -- 172.21.15.113:0/3283239419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad74144910 msgr2=0x7fad740b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 --2- 172.21.15.113:0/3283239419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad74144910 0x7fad740b3fd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fad74062820 tx=0x7fad6402a6a0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 -- 172.21.15.113:0/3283239419 shutdown_connections 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 --2- 172.21.15.113:0/3283239419 >> v2:172.21.15.178:3300/0 conn(0x7fad740b5680 0x7fad7406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 --2- 172.21.15.113:0/3283239419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5240 0x7fad7406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 --2- 172.21.15.113:0/3283239419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad74144910 0x7fad740b3fd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 -- 172.21.15.113:0/3283239419 >> 172.21.15.113:0/3283239419 conn(0x7fad740b1a50 msgr2=0x7fad740b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 -- 172.21.15.113:0/3283239419 shutdown_connections 2024-04-25T00:00:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.922+0000 7fad78f70640 1 -- 172.21.15.113:0/3283239419 wait complete. 2024-04-25T00:00:52.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.923+0000 7fad78f70640 1 Processor -- start 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.923+0000 7fad78f70640 1 -- start start 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad78f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad740b5240 0x7fad74141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad78f70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5680 0x7fad740ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad78f70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad74144910 0x7fad741f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad78f70640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b4da0 con 0x7fad740b5240 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad78f70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b37d0 con 0x7fad74144910 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad78f70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad740b4f20 con 0x7fad740b5680 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad740b5240 0x7fad74141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad740b5240 0x7fad74141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54504/0 (socket says 172.21.15.113:54504) 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 -- 172.21.15.113:0/1557505800 learned_addr learned my addr 172.21.15.113:0/1557505800 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad72d9d640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad74144910 0x7fad741f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad71d9b640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5680 0x7fad740ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 -- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5680 msgr2=0x7fad740ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5680 0x7fad740ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 -- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad74144910 msgr2=0x7fad741f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad74144910 0x7fad741f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 -- 172.21.15.113:0/1557505800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740ad1c0 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.924+0000 7fad7259c640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad740b5240 0x7fad74141e50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad64034040 tx=0x7fad64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.925+0000 7fad62ffd640 1 -- 172.21.15.113:0/1557505800 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64036050 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.925+0000 7fad62ffd640 1 -- 172.21.15.113:0/1557505800 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad64042070 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.925+0000 7fad62ffd640 1 -- 172.21.15.113:0/1557505800 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64009510 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.925+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad740ac5e0 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.925+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad740c4cc0 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.926+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad740691a0 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.926+0000 7fad62ffd640 1 -- 172.21.15.113:0/1557505800 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad64007c30 con 0x7fad740b5240 2024-04-25T00:00:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.926+0000 7fad62ffd640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.926+0000 7fad62ffd640 1 -- 172.21.15.113:0/1557505800 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fad640abda0 con 0x7fad740b5240 2024-04-25T00:00:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.926+0000 7fad71d9b640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.927+0000 7fad71d9b640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fad68006fd0 tx=0x7fad68008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:52.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:52.931+0000 7fad62ffd640 1 -- 172.21.15.113:0/1557505800 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad6407a750 con 0x7fad740b5240 2024-04-25T00:00:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.101+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad741f37b0 con 0x7fad5406be30 2024-04-25T00:00:53.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.107+0000 7fad62ffd640 1 -- 172.21.15.113:0/1557505800 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251892 (secure 0 0 0) 0x7fad741f37b0 con 0x7fad5406be30 2024-04-25T00:00:53.110 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:53.111 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:00:53.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 msgr2=0x7fad5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fad68006fd0 tx=0x7fad68008040 comp rx=0 tx=0).stop 2024-04-25T00:00:53.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad740b5240 msgr2=0x7fad74141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad740b5240 0x7fad74141e50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad64034040 tx=0x7fad64004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:53.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 shutdown_connections 2024-04-25T00:00:53.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad5406be30 0x7fad5406e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad74144910 0x7fad741f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad740b5680 0x7fad740ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 --2- 172.21.15.113:0/1557505800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad740b5240 0x7fad74141e50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 >> 172.21.15.113:0/1557505800 conn(0x7fad740b1a50 msgr2=0x7fad740cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 shutdown_connections 2024-04-25T00:00:53.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.110+0000 7fad78f70640 1 -- 172.21.15.113:0/1557505800 wait complete. 2024-04-25T00:00:53.121 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-04-25T00:00:51.158724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300592,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754674128,"statfs":{"total":773094113280,"available":772786307072,"internally_reserved":0,"allocated":5480448,"data_stored":2906668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018681"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179078+0000","last_change":"2024-04-25T00:00:50.176895+0000","last_active":"2024-04-25T00:00:50.179078+0000","last_peered":"2024-04-25T00:00:50.179078+0000","last_clean":"2024-04-25T00:00:50.179078+0000","last_became_active":"2024-04-25T00:00:50.176568+0000","last_became_peered":"2024-04-25T00:00:50.176568+0000","last_unstale":"2024-04-25T00:00:50.179078+0000","last_undegraded":"2024-04-25T00:00:50.179078+0000","last_fullsized":"2024-04-25T00:00:50.179078+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:05.593209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179607+0000","last_change":"2024-04-25T00:00:50.178757+0000","last_active":"2024-04-25T00:00:50.179607+0000","last_peered":"2024-04-25T00:00:50.179607+0000","last_clean":"2024-04-25T00:00:50.179607+0000","last_became_active":"2024-04-25T00:00:50.178492+0000","last_became_peered":"2024-04-25T00:00:50.178492+0000","last_unstale":"2024-04-25T00:00:50.179607+0000","last_undegraded":"2024-04-25T00:00:50.179607+0000","last_fullsized":"2024-04-25T00:00:50.179607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:50.761828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174189+0000","last_change":"2024-04-25T00:00:50.170062+0000","last_active":"2024-04-25T00:00:50.174189+0000","last_peered":"2024-04-25T00:00:50.174189+0000","last_clean":"2024-04-25T00:00:50.174189+0000","last_became_active":"2024-04-25T00:00:50.169751+0000","last_became_peered":"2024-04-25T00:00:50.169751+0000","last_unstale":"2024-04-25T00:00:50.174189+0000","last_undegraded":"2024-04-25T00:00:50.174189+0000","last_fullsized":"2024-04-25T00:00:50.174189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:33.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182777+0000","last_change":"2024-04-25T00:00:50.178974+0000","last_active":"2024-04-25T00:00:50.182777+0000","last_peered":"2024-04-25T00:00:50.182777+0000","last_clean":"2024-04-25T00:00:50.182777+0000","last_became_active":"2024-04-25T00:00:50.178114+0000","last_became_peered":"2024-04-25T00:00:50.178114+0000","last_unstale":"2024-04-25T00:00:50.182777+0000","last_undegraded":"2024-04-25T00:00:50.182777+0000","last_fullsized":"2024-04-25T00:00:50.182777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:41.076029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179147+0000","last_change":"2024-04-25T00:00:50.177282+0000","last_active":"2024-04-25T00:00:50.179147+0000","last_peered":"2024-04-25T00:00:50.179147+0000","last_clean":"2024-04-25T00:00:50.179147+0000","last_became_active":"2024-04-25T00:00:50.177002+0000","last_became_peered":"2024-04-25T00:00:50.177002+0000","last_unstale":"2024-04-25T00:00:50.179147+0000","last_undegraded":"2024-04-25T00:00:50.179147+0000","last_fullsized":"2024-04-25T00:00:50.179147+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:07.477832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178666+0000","last_change":"2024-04-25T00:00:50.176031+0000","last_active":"2024-04-25T00:00:50.178666+0000","last_peered":"2024-04-25T00:00:50.178666+0000","last_clean":"2024-04-25T00:00:50.178666+0000","last_became_active":"2024-04-25T00:00:50.175530+0000","last_became_peered":"2024-04-25T00:00:50.175530+0000","last_unstale":"2024-04-25T00:00:50.178666+0000","last_undegraded":"2024-04-25T00:00:50.178666+0000","last_fullsized":"2024-04-25T00:00:50.178666+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:00.602047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.176249+0000","last_change":"2024-04-25T00:00:50.175027+0000","last_active":"2024-04-25T00:00:50.176249+0000","last_peered":"2024-04-25T00:00:50.176249+0000","last_clean":"2024-04-25T00:00:50.176249+0000","last_became_active":"2024-04-25T00:00:50.174755+0000","last_became_peered":"2024-04-25T00:00:50.174755+0000","last_unstale":"2024-04-25T00:00:50.176249+0000","last_undegraded":"2024-04-25T00:00:50.176249+0000","last_fullsized":"2024-04-25T00:00:50.176249+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:22.909301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184884+0000","last_change":"2024-04-25T00:00:50.180116+0000","last_active":"2024-04-25T00:00:50.184884+0000","last_peered":"2024-04-25T00:00:50.184884+0000","last_clean":"2024-04-25T00:00:50.184884+0000","last_became_active":"2024-04-25T00:00:50.177826+0000","last_became_peered":"2024-04-25T00:00:50.177826+0000","last_unstale":"2024-04-25T00:00:50.184884+0000","last_undegraded":"2024-04-25T00:00:50.184884+0000","last_fullsized":"2024-04-25T00:00:50.184884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:07.584903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182684+0000","last_change":"2024-04-25T00:00:50.179300+0000","last_active":"2024-04-25T00:00:50.182684+0000","last_peered":"2024-04-25T00:00:50.182684+0000","last_clean":"2024-04-25T00:00:50.182684+0000","last_became_active":"2024-04-25T00:00:50.179034+0000","last_became_peered":"2024-04-25T00:00:50.179034+0000","last_unstale":"2024-04-25T00:00:50.182684+0000","last_undegraded":"2024-04-25T00:00:50.182684+0000","last_fullsized":"2024-04-25T00:00:50.182684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:07.670348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189760+0000","last_change":"2024-04-25T00:00:50.184673+0000","last_active":"2024-04-25T00:00:50.189760+0000","last_peered":"2024-04-25T00:00:50.189760+0000","last_clean":"2024-04-25T00:00:50.189760+0000","last_became_active":"2024-04-25T00:00:50.182827+0000","last_became_peered":"2024-04-25T00:00:50.182827+0000","last_unstale":"2024-04-25T00:00:50.189760+0000","last_undegraded":"2024-04-25T00:00:50.189760+0000","last_fullsized":"2024-04-25T00:00:50.189760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:33.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182718+0000","last_change":"2024-04-25T00:00:50.179352+0000","last_active":"2024-04-25T00:00:50.182718+0000","last_peered":"2024-04-25T00:00:50.182718+0000","last_clean":"2024-04-25T00:00:50.182718+0000","last_became_active":"2024-04-25T00:00:50.179080+0000","last_became_peered":"2024-04-25T00:00:50.179080+0000","last_unstale":"2024-04-25T00:00:50.182718+0000","last_undegraded":"2024-04-25T00:00:50.182718+0000","last_fullsized":"2024-04-25T00:00:50.182718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:42.118208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185343+0000","last_change":"2024-04-25T00:00:50.183962+0000","last_active":"2024-04-25T00:00:50.185343+0000","last_peered":"2024-04-25T00:00:50.185343+0000","last_clean":"2024-04-25T00:00:50.185343+0000","last_became_active":"2024-04-25T00:00:50.183669+0000","last_became_peered":"2024-04-25T00:00:50.183669+0000","last_unstale":"2024-04-25T00:00:50.185343+0000","last_undegraded":"2024-04-25T00:00:50.185343+0000","last_fullsized":"2024-04-25T00:00:50.185343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:22.909301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182424+0000","last_change":"2024-04-25T00:00:50.174814+0000","last_active":"2024-04-25T00:00:50.182424+0000","last_peered":"2024-04-25T00:00:50.182424+0000","last_clean":"2024-04-25T00:00:50.182424+0000","last_became_active":"2024-04-25T00:00:50.174563+0000","last_became_peered":"2024-04-25T00:00:50.174563+0000","last_unstale":"2024-04-25T00:00:50.182424+0000","last_undegraded":"2024-04-25T00:00:50.182424+0000","last_fullsized":"2024-04-25T00:00:50.182424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:11.103329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184856+0000","last_change":"2024-04-25T00:00:50.179180+0000","last_active":"2024-04-25T00:00:50.184856+0000","last_peered":"2024-04-25T00:00:50.184856+0000","last_clean":"2024-04-25T00:00:50.184856+0000","last_became_active":"2024-04-25T00:00:50.178849+0000","last_became_peered":"2024-04-25T00:00:50.178849+0000","last_unstale":"2024-04-25T00:00:50.184856+0000","last_undegraded":"2024-04-25T00:00:50.184856+0000","last_fullsized":"2024-04-25T00:00:50.184856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:32.658955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.173308+0000","last_change":"2024-04-25T00:00:50.169414+0000","last_active":"2024-04-25T00:00:50.173308+0000","last_peered":"2024-04-25T00:00:50.173308+0000","last_clean":"2024-04-25T00:00:50.173308+0000","last_became_active":"2024-04-25T00:00:50.169138+0000","last_became_peered":"2024-04-25T00:00:50.169138+0000","last_unstale":"2024-04-25T00:00:50.173308+0000","last_undegraded":"2024-04-25T00:00:50.173308+0000","last_fullsized":"2024-04-25T00:00:50.173308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:40.318647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174775+0000","last_change":"2024-04-25T00:00:50.171041+0000","last_active":"2024-04-25T00:00:50.174775+0000","last_peered":"2024-04-25T00:00:50.174775+0000","last_clean":"2024-04-25T00:00:50.174775+0000","last_became_active":"2024-04-25T00:00:50.170766+0000","last_became_peered":"2024-04-25T00:00:50.170766+0000","last_unstale":"2024-04-25T00:00:50.174775+0000","last_undegraded":"2024-04-25T00:00:50.174775+0000","last_fullsized":"2024-04-25T00:00:50.174775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:22.130163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171079+0000","last_change":"2024-04-25T00:00:50.167295+0000","last_active":"2024-04-25T00:00:50.171079+0000","last_peered":"2024-04-25T00:00:50.171079+0000","last_clean":"2024-04-25T00:00:50.171079+0000","last_became_active":"2024-04-25T00:00:50.166974+0000","last_became_peered":"2024-04-25T00:00:50.166974+0000","last_unstale":"2024-04-25T00:00:50.171079+0000","last_undegraded":"2024-04-25T00:00:50.171079+0000","last_fullsized":"2024-04-25T00:00:50.171079+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:42.049285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174775+0000","last_change":"2024-04-25T00:00:50.170868+0000","last_active":"2024-04-25T00:00:50.174775+0000","last_peered":"2024-04-25T00:00:50.174775+0000","last_clean":"2024-04-25T00:00:50.174775+0000","last_became_active":"2024-04-25T00:00:50.169071+0000","last_became_peered":"2024-04-25T00:00:50.169071+0000","last_unstale":"2024-04-25T00:00:50.174775+0000","last_undegraded":"2024-04-25T00:00:50.174775+0000","last_fullsized":"2024-04-25T00:00:50.174775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:52.739391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182238+0000","last_change":"2024-04-25T00:00:50.181255+0000","last_active":"2024-04-25T00:00:50.182238+0000","last_peered":"2024-04-25T00:00:50.182238+0000","last_clean":"2024-04-25T00:00:50.182238+0000","last_became_active":"2024-04-25T00:00:50.180923+0000","last_became_peered":"2024-04-25T00:00:50.180923+0000","last_unstale":"2024-04-25T00:00:50.182238+0000","last_undegraded":"2024-04-25T00:00:50.182238+0000","last_fullsized":"2024-04-25T00:00:50.182238+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:13.489857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182321+0000","last_change":"2024-04-25T00:00:50.172671+0000","last_active":"2024-04-25T00:00:50.182321+0000","last_peered":"2024-04-25T00:00:50.182321+0000","last_clean":"2024-04-25T00:00:50.182321+0000","last_became_active":"2024-04-25T00:00:50.172421+0000","last_became_peered":"2024-04-25T00:00:50.172421+0000","last_unstale":"2024-04-25T00:00:50.182321+0000","last_undegraded":"2024-04-25T00:00:50.182321+0000","last_fullsized":"2024-04-25T00:00:50.182321+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:07.165116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.175124+0000","last_change":"2024-04-25T00:00:50.168942+0000","last_active":"2024-04-25T00:00:50.175124+0000","last_peered":"2024-04-25T00:00:50.175124+0000","last_clean":"2024-04-25T00:00:50.175124+0000","last_became_active":"2024-04-25T00:00:50.168670+0000","last_became_peered":"2024-04-25T00:00:50.168670+0000","last_unstale":"2024-04-25T00:00:50.175124+0000","last_undegraded":"2024-04-25T00:00:50.175124+0000","last_fullsized":"2024-04-25T00:00:50.175124+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.112405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182630+0000","last_change":"2024-04-25T00:00:50.178640+0000","last_active":"2024-04-25T00:00:50.182630+0000","last_peered":"2024-04-25T00:00:50.182630+0000","last_clean":"2024-04-25T00:00:50.182630+0000","last_became_active":"2024-04-25T00:00:50.178337+0000","last_became_peered":"2024-04-25T00:00:50.178337+0000","last_unstale":"2024-04-25T00:00:50.182630+0000","last_undegraded":"2024-04-25T00:00:50.182630+0000","last_fullsized":"2024-04-25T00:00:50.182630+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:17.423642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177064+0000","last_change":"2024-04-25T00:00:50.172769+0000","last_active":"2024-04-25T00:00:50.177064+0000","last_peered":"2024-04-25T00:00:50.177064+0000","last_clean":"2024-04-25T00:00:50.177064+0000","last_became_active":"2024-04-25T00:00:50.172145+0000","last_became_peered":"2024-04-25T00:00:50.172145+0000","last_unstale":"2024-04-25T00:00:50.177064+0000","last_undegraded":"2024-04-25T00:00:50.177064+0000","last_fullsized":"2024-04-25T00:00:50.177064+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:07.165116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182379+0000","last_change":"2024-04-25T00:00:50.178526+0000","last_active":"2024-04-25T00:00:50.182379+0000","last_peered":"2024-04-25T00:00:50.182379+0000","last_clean":"2024-04-25T00:00:50.182379+0000","last_became_active":"2024-04-25T00:00:50.178262+0000","last_became_peered":"2024-04-25T00:00:50.178262+0000","last_unstale":"2024-04-25T00:00:50.182379+0000","last_undegraded":"2024-04-25T00:00:50.182379+0000","last_fullsized":"2024-04-25T00:00:50.182379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:58.265142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177694+0000","last_change":"2024-04-25T00:00:50.174190+0000","last_active":"2024-04-25T00:00:50.177694+0000","last_peered":"2024-04-25T00:00:50.177694+0000","last_clean":"2024-04-25T00:00:50.177694+0000","last_became_active":"2024-04-25T00:00:50.173471+0000","last_became_peered":"2024-04-25T00:00:50.173471+0000","last_unstale":"2024-04-25T00:00:50.177694+0000","last_undegraded":"2024-04-25T00:00:50.177694+0000","last_fullsized":"2024-04-25T00:00:50.177694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:17.944703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182211+0000","last_change":"2024-04-25T00:00:50.175787+0000","last_active":"2024-04-25T00:00:50.182211+0000","last_peered":"2024-04-25T00:00:50.182211+0000","last_clean":"2024-04-25T00:00:50.182211+0000","last_became_active":"2024-04-25T00:00:50.175544+0000","last_became_peered":"2024-04-25T00:00:50.175544+0000","last_unstale":"2024-04-25T00:00:50.182211+0000","last_undegraded":"2024-04-25T00:00:50.182211+0000","last_fullsized":"2024-04-25T00:00:50.182211+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:17.944703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.169671+0000","last_change":"2024-04-25T00:00:50.167288+0000","last_active":"2024-04-25T00:00:50.169671+0000","last_peered":"2024-04-25T00:00:50.169671+0000","last_clean":"2024-04-25T00:00:50.169671+0000","last_became_active":"2024-04-25T00:00:50.166905+0000","last_became_peered":"2024-04-25T00:00:50.166905+0000","last_unstale":"2024-04-25T00:00:50.169671+0000","last_undegraded":"2024-04-25T00:00:50.169671+0000","last_fullsized":"2024-04-25T00:00:50.169671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:07.584903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185363+0000","last_change":"2024-04-25T00:00:50.184000+0000","last_active":"2024-04-25T00:00:50.185363+0000","last_peered":"2024-04-25T00:00:50.185363+0000","last_clean":"2024-04-25T00:00:50.185363+0000","last_became_active":"2024-04-25T00:00:50.183756+0000","last_became_peered":"2024-04-25T00:00:50.183756+0000","last_unstale":"2024-04-25T00:00:50.185363+0000","last_undegraded":"2024-04-25T00:00:50.185363+0000","last_fullsized":"2024-04-25T00:00:50.185363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:20.121342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177020+0000","last_change":"2024-04-25T00:00:50.172382+0000","last_active":"2024-04-25T00:00:50.177020+0000","last_peered":"2024-04-25T00:00:50.177020+0000","last_clean":"2024-04-25T00:00:50.177020+0000","last_became_active":"2024-04-25T00:00:50.172117+0000","last_became_peered":"2024-04-25T00:00:50.172117+0000","last_unstale":"2024-04-25T00:00:50.177020+0000","last_undegraded":"2024-04-25T00:00:50.177020+0000","last_fullsized":"2024-04-25T00:00:50.177020+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:53.780711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.176732+0000","last_change":"2024-04-25T00:00:50.175771+0000","last_active":"2024-04-25T00:00:50.176732+0000","last_peered":"2024-04-25T00:00:50.176732+0000","last_clean":"2024-04-25T00:00:50.176732+0000","last_became_active":"2024-04-25T00:00:50.175510+0000","last_became_peered":"2024-04-25T00:00:50.175510+0000","last_unstale":"2024-04-25T00:00:50.176732+0000","last_undegraded":"2024-04-25T00:00:50.176732+0000","last_fullsized":"2024-04-25T00:00:50.176732+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:10.440124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.173284+0000","last_change":"2024-04-25T00:00:50.168560+0000","last_active":"2024-04-25T00:00:50.173284+0000","last_peered":"2024-04-25T00:00:50.173284+0000","last_clean":"2024-04-25T00:00:50.173284+0000","last_became_active":"2024-04-25T00:00:50.168317+0000","last_became_peered":"2024-04-25T00:00:50.168317+0000","last_unstale":"2024-04-25T00:00:50.173284+0000","last_undegraded":"2024-04-25T00:00:50.173284+0000","last_fullsized":"2024-04-25T00:00:50.173284+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:31.172329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182248+0000","last_change":"2024-04-25T00:00:50.171221+0000","last_active":"2024-04-25T00:00:50.182248+0000","last_peered":"2024-04-25T00:00:50.182248+0000","last_clean":"2024-04-25T00:00:50.182248+0000","last_became_active":"2024-04-25T00:00:50.170906+0000","last_became_peered":"2024-04-25T00:00:50.170906+0000","last_unstale":"2024-04-25T00:00:50.182248+0000","last_undegraded":"2024-04-25T00:00:50.182248+0000","last_fullsized":"2024-04-25T00:00:50.182248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:53.780711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.186614+0000","last_change":"2024-04-25T00:00:50.181146+0000","last_active":"2024-04-25T00:00:50.186614+0000","last_peered":"2024-04-25T00:00:50.186614+0000","last_clean":"2024-04-25T00:00:50.186614+0000","last_became_active":"2024-04-25T00:00:50.180456+0000","last_became_peered":"2024-04-25T00:00:50.180456+0000","last_unstale":"2024-04-25T00:00:50.186614+0000","last_undegraded":"2024-04-25T00:00:50.186614+0000","last_fullsized":"2024-04-25T00:00:50.186614+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.081284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189962+0000","last_change":"2024-04-25T00:00:49.167909+0000","last_active":"2024-04-25T00:00:50.189962+0000","last_peered":"2024-04-25T00:00:50.189962+0000","last_clean":"2024-04-25T00:00:50.189962+0000","last_became_active":"2024-04-25T00:00:49.167641+0000","last_became_peered":"2024-04-25T00:00:49.167641+0000","last_unstale":"2024-04-25T00:00:50.189962+0000","last_undegraded":"2024-04-25T00:00:50.189962+0000","last_fullsized":"2024-04-25T00:00:50.189962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:21.903615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.181680+0000","last_change":"2024-04-25T00:00:50.179521+0000","last_active":"2024-04-25T00:00:50.181680+0000","last_peered":"2024-04-25T00:00:50.181680+0000","last_clean":"2024-04-25T00:00:50.181680+0000","last_became_active":"2024-04-25T00:00:50.179249+0000","last_became_peered":"2024-04-25T00:00:50.179249+0000","last_unstale":"2024-04-25T00:00:50.181680+0000","last_undegraded":"2024-04-25T00:00:50.181680+0000","last_fullsized":"2024-04-25T00:00:50.181680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:12.445539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182768+0000","last_change":"2024-04-25T00:00:49.175235+0000","last_active":"2024-04-25T00:00:50.182768+0000","last_peered":"2024-04-25T00:00:50.182768+0000","last_clean":"2024-04-25T00:00:50.182768+0000","last_became_active":"2024-04-25T00:00:49.174947+0000","last_became_peered":"2024-04-25T00:00:49.174947+0000","last_unstale":"2024-04-25T00:00:50.182768+0000","last_undegraded":"2024-04-25T00:00:50.182768+0000","last_fullsized":"2024-04-25T00:00:50.182768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:41.112523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.172464+0000","last_change":"2024-04-25T00:00:50.171268+0000","last_active":"2024-04-25T00:00:50.172464+0000","last_peered":"2024-04-25T00:00:50.172464+0000","last_clean":"2024-04-25T00:00:50.172464+0000","last_became_active":"2024-04-25T00:00:50.170979+0000","last_became_peered":"2024-04-25T00:00:50.170979+0000","last_unstale":"2024-04-25T00:00:50.172464+0000","last_undegraded":"2024-04-25T00:00:50.172464+0000","last_fullsized":"2024-04-25T00:00:50.172464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:58.570596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184112+0000","last_change":"2024-04-25T00:00:49.172463+0000","last_active":"2024-04-25T00:00:50.184112+0000","last_peered":"2024-04-25T00:00:50.184112+0000","last_clean":"2024-04-25T00:00:50.184112+0000","last_became_active":"2024-04-25T00:00:49.171973+0000","last_became_peered":"2024-04-25T00:00:49.171973+0000","last_unstale":"2024-04-25T00:00:50.184112+0000","last_undegraded":"2024-04-25T00:00:50.184112+0000","last_fullsized":"2024-04-25T00:00:50.184112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:32.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.186016+0000","last_change":"2024-04-25T00:00:50.181954+0000","last_active":"2024-04-25T00:00:50.186016+0000","last_peered":"2024-04-25T00:00:50.186016+0000","last_clean":"2024-04-25T00:00:50.186016+0000","last_became_active":"2024-04-25T00:00:50.181378+0000","last_became_peered":"2024-04-25T00:00:50.181378+0000","last_unstale":"2024-04-25T00:00:50.186016+0000","last_undegraded":"2024-04-25T00:00:50.186016+0000","last_fullsized":"2024-04-25T00:00:50.186016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:52.739391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185501+0000","last_change":"2024-04-25T00:00:49.174628+0000","last_active":"2024-04-25T00:00:50.185501+0000","last_peered":"2024-04-25T00:00:50.185501+0000","last_clean":"2024-04-25T00:00:50.185501+0000","last_became_active":"2024-04-25T00:00:49.174304+0000","last_became_peered":"2024-04-25T00:00:49.174304+0000","last_unstale":"2024-04-25T00:00:50.185501+0000","last_undegraded":"2024-04-25T00:00:50.185501+0000","last_fullsized":"2024-04-25T00:00:50.185501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:49.756143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.175089+0000","last_change":"2024-04-25T00:00:50.173978+0000","last_active":"2024-04-25T00:00:50.175089+0000","last_peered":"2024-04-25T00:00:50.175089+0000","last_clean":"2024-04-25T00:00:50.175089+0000","last_became_active":"2024-04-25T00:00:50.173585+0000","last_became_peered":"2024-04-25T00:00:50.173585+0000","last_unstale":"2024-04-25T00:00:50.175089+0000","last_undegraded":"2024-04-25T00:00:50.175089+0000","last_fullsized":"2024-04-25T00:00:50.175089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:21.820344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189932+0000","last_change":"2024-04-25T00:00:49.172459+0000","last_active":"2024-04-25T00:00:50.189932+0000","last_peered":"2024-04-25T00:00:50.189932+0000","last_clean":"2024-04-25T00:00:50.189932+0000","last_became_active":"2024-04-25T00:00:49.172030+0000","last_became_peered":"2024-04-25T00:00:49.172030+0000","last_unstale":"2024-04-25T00:00:50.189932+0000","last_undegraded":"2024-04-25T00:00:50.189932+0000","last_fullsized":"2024-04-25T00:00:50.189932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:46.658861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189900+0000","last_change":"2024-04-25T00:00:50.186151+0000","last_active":"2024-04-25T00:00:50.189900+0000","last_peered":"2024-04-25T00:00:50.189900+0000","last_clean":"2024-04-25T00:00:50.189900+0000","last_became_active":"2024-04-25T00:00:50.185447+0000","last_became_peered":"2024-04-25T00:00:50.185447+0000","last_unstale":"2024-04-25T00:00:50.189900+0000","last_undegraded":"2024-04-25T00:00:50.189900+0000","last_fullsized":"2024-04-25T00:00:50.189900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:52.739391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.180377+0000","last_change":"2024-04-25T00:00:49.174057+0000","last_active":"2024-04-25T00:00:50.180377+0000","last_peered":"2024-04-25T00:00:50.180377+0000","last_clean":"2024-04-25T00:00:50.180377+0000","last_became_active":"2024-04-25T00:00:49.173752+0000","last_became_peered":"2024-04-25T00:00:49.173752+0000","last_unstale":"2024-04-25T00:00:50.180377+0000","last_undegraded":"2024-04-25T00:00:50.180377+0000","last_fullsized":"2024-04-25T00:00:50.180377+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:06.664663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187712+0000","last_change":"2024-04-25T00:00:50.182807+0000","last_active":"2024-04-25T00:00:50.187712+0000","last_peered":"2024-04-25T00:00:50.187712+0000","last_clean":"2024-04-25T00:00:50.187712+0000","last_became_active":"2024-04-25T00:00:50.179689+0000","last_became_peered":"2024-04-25T00:00:50.179689+0000","last_unstale":"2024-04-25T00:00:50.187712+0000","last_undegraded":"2024-04-25T00:00:50.187712+0000","last_fullsized":"2024-04-25T00:00:50.187712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.434593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.163397+0000","last_change":"2024-04-25T00:00:49.170345+0000","last_active":"2024-04-25T00:00:50.163397+0000","last_peered":"2024-04-25T00:00:50.163397+0000","last_clean":"2024-04-25T00:00:50.163397+0000","last_became_active":"2024-04-25T00:00:49.169838+0000","last_became_peered":"2024-04-25T00:00:49.169838+0000","last_unstale":"2024-04-25T00:00:50.163397+0000","last_undegraded":"2024-04-25T00:00:50.163397+0000","last_fullsized":"2024-04-25T00:00:50.163397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:14.384270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177087+0000","last_change":"2024-04-25T00:00:50.172999+0000","last_active":"2024-04-25T00:00:50.177087+0000","last_peered":"2024-04-25T00:00:50.177087+0000","last_clean":"2024-04-25T00:00:50.177087+0000","last_became_active":"2024-04-25T00:00:50.172508+0000","last_became_peered":"2024-04-25T00:00:50.172508+0000","last_unstale":"2024-04-25T00:00:50.177087+0000","last_undegraded":"2024-04-25T00:00:50.177087+0000","last_fullsized":"2024-04-25T00:00:50.177087+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:11.103329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.165954+0000","last_change":"2024-04-25T00:00:49.169507+0000","last_active":"2024-04-25T00:00:50.165954+0000","last_peered":"2024-04-25T00:00:50.165954+0000","last_clean":"2024-04-25T00:00:50.165954+0000","last_became_active":"2024-04-25T00:00:49.169219+0000","last_became_peered":"2024-04-25T00:00:49.169219+0000","last_unstale":"2024-04-25T00:00:50.165954+0000","last_undegraded":"2024-04-25T00:00:50.165954+0000","last_fullsized":"2024-04-25T00:00:50.165954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:41.530593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171023+0000","last_change":"2024-04-25T00:00:50.169339+0000","last_active":"2024-04-25T00:00:50.171023+0000","last_peered":"2024-04-25T00:00:50.171023+0000","last_clean":"2024-04-25T00:00:50.171023+0000","last_became_active":"2024-04-25T00:00:50.168705+0000","last_became_peered":"2024-04-25T00:00:50.168705+0000","last_unstale":"2024-04-25T00:00:50.171023+0000","last_undegraded":"2024-04-25T00:00:50.171023+0000","last_fullsized":"2024-04-25T00:00:50.171023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.434593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.160179+0000","last_change":"2024-04-25T00:00:49.171738+0000","last_active":"2024-04-25T00:00:50.160179+0000","last_peered":"2024-04-25T00:00:50.160179+0000","last_clean":"2024-04-25T00:00:50.160179+0000","last_became_active":"2024-04-25T00:00:49.171454+0000","last_became_peered":"2024-04-25T00:00:49.171454+0000","last_unstale":"2024-04-25T00:00:50.160179+0000","last_undegraded":"2024-04-25T00:00:50.160179+0000","last_fullsized":"2024-04-25T00:00:50.160179+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:16.417956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178121+0000","last_change":"2024-04-25T00:00:50.176316+0000","last_active":"2024-04-25T00:00:50.178121+0000","last_peered":"2024-04-25T00:00:50.178121+0000","last_clean":"2024-04-25T00:00:50.178121+0000","last_became_active":"2024-04-25T00:00:50.175979+0000","last_became_peered":"2024-04-25T00:00:50.175979+0000","last_unstale":"2024-04-25T00:00:50.178121+0000","last_undegraded":"2024-04-25T00:00:50.178121+0000","last_fullsized":"2024-04-25T00:00:50.178121+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:47.664547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.167575+0000","last_change":"2024-04-25T00:00:49.167831+0000","last_active":"2024-04-25T00:00:50.167575+0000","last_peered":"2024-04-25T00:00:50.167575+0000","last_clean":"2024-04-25T00:00:50.167575+0000","last_became_active":"2024-04-25T00:00:49.167561+0000","last_became_peered":"2024-04-25T00:00:49.167561+0000","last_unstale":"2024-04-25T00:00:50.167575+0000","last_undegraded":"2024-04-25T00:00:50.167575+0000","last_fullsized":"2024-04-25T00:00:50.167575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:00.788977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187445+0000","last_change":"2024-04-25T00:00:50.184339+0000","last_active":"2024-04-25T00:00:50.187445+0000","last_peered":"2024-04-25T00:00:50.187445+0000","last_clean":"2024-04-25T00:00:50.187445+0000","last_became_active":"2024-04-25T00:00:50.183559+0000","last_became_peered":"2024-04-25T00:00:50.183559+0000","last_unstale":"2024-04-25T00:00:50.187445+0000","last_undegraded":"2024-04-25T00:00:50.187445+0000","last_fullsized":"2024-04-25T00:00:50.187445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:43.532179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185714+0000","last_change":"2024-04-25T00:00:49.171657+0000","last_active":"2024-04-25T00:00:50.185714+0000","last_peered":"2024-04-25T00:00:50.185714+0000","last_clean":"2024-04-25T00:00:50.185714+0000","last_became_active":"2024-04-25T00:00:49.171324+0000","last_became_peered":"2024-04-25T00:00:49.171324+0000","last_unstale":"2024-04-25T00:00:50.185714+0000","last_undegraded":"2024-04-25T00:00:50.185714+0000","last_fullsized":"2024-04-25T00:00:50.185714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:47.973032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177949+0000","last_change":"2024-04-25T00:00:50.175787+0000","last_active":"2024-04-25T00:00:50.177949+0000","last_peered":"2024-04-25T00:00:50.177949+0000","last_clean":"2024-04-25T00:00:50.177949+0000","last_became_active":"2024-04-25T00:00:50.175463+0000","last_became_peered":"2024-04-25T00:00:50.175463+0000","last_unstale":"2024-04-25T00:00:50.177949+0000","last_undegraded":"2024-04-25T00:00:50.177949+0000","last_fullsized":"2024-04-25T00:00:50.177949+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:20.121342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.191497+0000","last_change":"2024-04-25T00:00:49.175971+0000","last_active":"2024-04-25T00:00:50.191497+0000","last_peered":"2024-04-25T00:00:50.191497+0000","last_clean":"2024-04-25T00:00:50.191497+0000","last_became_active":"2024-04-25T00:00:49.175652+0000","last_became_peered":"2024-04-25T00:00:49.175652+0000","last_unstale":"2024-04-25T00:00:50.191497+0000","last_undegraded":"2024-04-25T00:00:50.191497+0000","last_fullsized":"2024-04-25T00:00:50.191497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:37.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182348+0000","last_change":"2024-04-25T00:00:50.180393+0000","last_active":"2024-04-25T00:00:50.182348+0000","last_peered":"2024-04-25T00:00:50.182348+0000","last_clean":"2024-04-25T00:00:50.182348+0000","last_became_active":"2024-04-25T00:00:50.180143+0000","last_became_peered":"2024-04-25T00:00:50.180143+0000","last_unstale":"2024-04-25T00:00:50.182348+0000","last_undegraded":"2024-04-25T00:00:50.182348+0000","last_fullsized":"2024-04-25T00:00:50.182348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:50.934303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.190309+0000","last_change":"2024-04-25T00:00:49.170274+0000","last_active":"2024-04-25T00:00:50.190309+0000","last_peered":"2024-04-25T00:00:50.190309+0000","last_clean":"2024-04-25T00:00:50.190309+0000","last_became_active":"2024-04-25T00:00:49.169953+0000","last_became_peered":"2024-04-25T00:00:49.169953+0000","last_unstale":"2024-04-25T00:00:50.190309+0000","last_undegraded":"2024-04-25T00:00:50.190309+0000","last_fullsized":"2024-04-25T00:00:50.190309+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:19.115656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189878+0000","last_change":"2024-04-25T00:00:50.185927+0000","last_active":"2024-04-25T00:00:50.189878+0000","last_peered":"2024-04-25T00:00:50.189878+0000","last_clean":"2024-04-25T00:00:50.189878+0000","last_became_active":"2024-04-25T00:00:50.185099+0000","last_became_peered":"2024-04-25T00:00:50.185099+0000","last_unstale":"2024-04-25T00:00:50.189878+0000","last_undegraded":"2024-04-25T00:00:50.189878+0000","last_fullsized":"2024-04-25T00:00:50.189878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.081284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.168146+0000","last_change":"2024-04-25T00:00:49.176979+0000","last_active":"2024-04-25T00:00:50.168146+0000","last_peered":"2024-04-25T00:00:50.168146+0000","last_clean":"2024-04-25T00:00:50.168146+0000","last_became_active":"2024-04-25T00:00:49.176548+0000","last_became_peered":"2024-04-25T00:00:49.176548+0000","last_unstale":"2024-04-25T00:00:50.168146+0000","last_undegraded":"2024-04-25T00:00:50.168146+0000","last_fullsized":"2024-04-25T00:00:50.168146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:48.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.180170+0000","last_change":"2024-04-25T00:00:50.179399+0000","last_active":"2024-04-25T00:00:50.180170+0000","last_peered":"2024-04-25T00:00:50.180170+0000","last_clean":"2024-04-25T00:00:50.180170+0000","last_became_active":"2024-04-25T00:00:50.179135+0000","last_became_peered":"2024-04-25T00:00:50.179135+0000","last_unstale":"2024-04-25T00:00:50.180170+0000","last_undegraded":"2024-04-25T00:00:50.180170+0000","last_fullsized":"2024-04-25T00:00:50.180170+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:17.037457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.164044+0000","last_change":"2024-04-25T00:00:49.167252+0000","last_active":"2024-04-25T00:00:50.164044+0000","last_peered":"2024-04-25T00:00:50.164044+0000","last_clean":"2024-04-25T00:00:50.164044+0000","last_became_active":"2024-04-25T00:00:49.166310+0000","last_became_peered":"2024-04-25T00:00:49.166310+0000","last_unstale":"2024-04-25T00:00:50.164044+0000","last_undegraded":"2024-04-25T00:00:50.164044+0000","last_fullsized":"2024-04-25T00:00:50.164044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:49.756143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182461+0000","last_change":"2024-04-25T00:00:50.181502+0000","last_active":"2024-04-25T00:00:50.182461+0000","last_peered":"2024-04-25T00:00:50.182461+0000","last_clean":"2024-04-25T00:00:50.182461+0000","last_became_active":"2024-04-25T00:00:50.181212+0000","last_became_peered":"2024-04-25T00:00:50.181212+0000","last_unstale":"2024-04-25T00:00:50.182461+0000","last_undegraded":"2024-04-25T00:00:50.182461+0000","last_fullsized":"2024-04-25T00:00:50.182461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:48.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.188613+0000","last_change":"2024-04-25T00:00:49.170215+0000","last_active":"2024-04-25T00:00:50.188613+0000","last_peered":"2024-04-25T00:00:50.188613+0000","last_clean":"2024-04-25T00:00:50.188613+0000","last_became_active":"2024-04-25T00:00:49.169944+0000","last_became_peered":"2024-04-25T00:00:49.169944+0000","last_unstale":"2024-04-25T00:00:50.188613+0000","last_undegraded":"2024-04-25T00:00:50.188613+0000","last_fullsized":"2024-04-25T00:00:50.188613+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:57.259456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178082+0000","last_change":"2024-04-25T00:00:50.169260+0000","last_active":"2024-04-25T00:00:50.178082+0000","last_peered":"2024-04-25T00:00:50.178082+0000","last_clean":"2024-04-25T00:00:50.178082+0000","last_became_active":"2024-04-25T00:00:50.168977+0000","last_became_peered":"2024-04-25T00:00:50.168977+0000","last_unstale":"2024-04-25T00:00:50.178082+0000","last_undegraded":"2024-04-25T00:00:50.178082+0000","last_fullsized":"2024-04-25T00:00:50.178082+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:22.130163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.183529+0000","last_change":"2024-04-25T00:00:49.168844+0000","last_active":"2024-04-25T00:00:50.183529+0000","last_peered":"2024-04-25T00:00:50.183529+0000","last_clean":"2024-04-25T00:00:50.183529+0000","last_became_active":"2024-04-25T00:00:49.168530+0000","last_became_peered":"2024-04-25T00:00:49.168530+0000","last_unstale":"2024-04-25T00:00:50.183529+0000","last_undegraded":"2024-04-25T00:00:50.183529+0000","last_fullsized":"2024-04-25T00:00:50.183529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:58.853340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182665+0000","last_change":"2024-04-25T00:00:50.178521+0000","last_active":"2024-04-25T00:00:50.182665+0000","last_peered":"2024-04-25T00:00:50.182665+0000","last_clean":"2024-04-25T00:00:50.182665+0000","last_became_active":"2024-04-25T00:00:50.178209+0000","last_became_peered":"2024-04-25T00:00:50.178209+0000","last_unstale":"2024-04-25T00:00:50.182665+0000","last_undegraded":"2024-04-25T00:00:50.182665+0000","last_fullsized":"2024-04-25T00:00:50.182665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:31.835102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187065+0000","last_change":"2024-04-25T00:00:49.171183+0000","last_active":"2024-04-25T00:00:50.187065+0000","last_peered":"2024-04-25T00:00:50.187065+0000","last_clean":"2024-04-25T00:00:50.187065+0000","last_became_active":"2024-04-25T00:00:49.170878+0000","last_became_peered":"2024-04-25T00:00:49.170878+0000","last_unstale":"2024-04-25T00:00:50.187065+0000","last_undegraded":"2024-04-25T00:00:50.187065+0000","last_fullsized":"2024-04-25T00:00:50.187065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:40.070343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174471+0000","last_change":"2024-04-25T00:00:50.170486+0000","last_active":"2024-04-25T00:00:50.174471+0000","last_peered":"2024-04-25T00:00:50.174471+0000","last_clean":"2024-04-25T00:00:50.174471+0000","last_became_active":"2024-04-25T00:00:50.169865+0000","last_became_peered":"2024-04-25T00:00:50.169865+0000","last_unstale":"2024-04-25T00:00:50.174471+0000","last_undegraded":"2024-04-25T00:00:50.174471+0000","last_fullsized":"2024-04-25T00:00:50.174471+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:38.222251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184509+0000","last_change":"2024-04-25T00:00:49.173921+0000","last_active":"2024-04-25T00:00:50.184509+0000","last_peered":"2024-04-25T00:00:50.184509+0000","last_clean":"2024-04-25T00:00:50.184509+0000","last_became_active":"2024-04-25T00:00:49.173654+0000","last_became_peered":"2024-04-25T00:00:49.173654+0000","last_unstale":"2024-04-25T00:00:50.184509+0000","last_undegraded":"2024-04-25T00:00:50.184509+0000","last_fullsized":"2024-04-25T00:00:50.184509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:43.177800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179027+0000","last_change":"2024-04-25T00:00:50.177561+0000","last_active":"2024-04-25T00:00:50.179027+0000","last_peered":"2024-04-25T00:00:50.179027+0000","last_clean":"2024-04-25T00:00:50.179027+0000","last_became_active":"2024-04-25T00:00:50.177238+0000","last_became_peered":"2024-04-25T00:00:50.177238+0000","last_unstale":"2024-04-25T00:00:50.179027+0000","last_undegraded":"2024-04-25T00:00:50.179027+0000","last_fullsized":"2024-04-25T00:00:50.179027+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:42.536278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.163421+0000","last_change":"2024-04-25T00:00:49.171429+0000","last_active":"2024-04-25T00:00:50.163421+0000","last_peered":"2024-04-25T00:00:50.163421+0000","last_clean":"2024-04-25T00:00:50.163421+0000","last_became_active":"2024-04-25T00:00:49.171154+0000","last_became_peered":"2024-04-25T00:00:49.171154+0000","last_unstale":"2024-04-25T00:00:50.163421+0000","last_undegraded":"2024-04-25T00:00:50.163421+0000","last_fullsized":"2024-04-25T00:00:50.163421+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:06.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.176380+0000","last_change":"2024-04-25T00:00:50.170640+0000","last_active":"2024-04-25T00:00:50.176380+0000","last_peered":"2024-04-25T00:00:50.176380+0000","last_clean":"2024-04-25T00:00:50.176380+0000","last_became_active":"2024-04-25T00:00:50.170391+0000","last_became_peered":"2024-04-25T00:00:50.170391+0000","last_unstale":"2024-04-25T00:00:50.176380+0000","last_undegraded":"2024-04-25T00:00:50.176380+0000","last_fullsized":"2024-04-25T00:00:50.176380+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:24.139365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.190046+0000","last_change":"2024-04-25T00:00:49.177013+0000","last_active":"2024-04-25T00:00:50.190046+0000","last_peered":"2024-04-25T00:00:50.190046+0000","last_clean":"2024-04-25T00:00:50.190046+0000","last_became_active":"2024-04-25T00:00:49.176636+0000","last_became_peered":"2024-04-25T00:00:49.176636+0000","last_unstale":"2024-04-25T00:00:50.190046+0000","last_undegraded":"2024-04-25T00:00:50.190046+0000","last_fullsized":"2024-04-25T00:00:50.190046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:04.587524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.172793+0000","last_change":"2024-04-25T00:00:50.172328+0000","last_active":"2024-04-25T00:00:50.172793+0000","last_peered":"2024-04-25T00:00:50.172793+0000","last_clean":"2024-04-25T00:00:50.172793+0000","last_became_active":"2024-04-25T00:00:50.172058+0000","last_became_peered":"2024-04-25T00:00:50.172058+0000","last_unstale":"2024-04-25T00:00:50.172793+0000","last_undegraded":"2024-04-25T00:00:50.172793+0000","last_fullsized":"2024-04-25T00:00:50.172793+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:42.049285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.163934+0000","last_change":"2024-04-25T00:00:49.174029+0000","last_active":"2024-04-25T00:00:50.163934+0000","last_peered":"2024-04-25T00:00:50.163934+0000","last_clean":"2024-04-25T00:00:50.163934+0000","last_became_active":"2024-04-25T00:00:49.173181+0000","last_became_peered":"2024-04-25T00:00:49.173181+0000","last_unstale":"2024-04-25T00:00:50.163934+0000","last_undegraded":"2024-04-25T00:00:50.163934+0000","last_fullsized":"2024-04-25T00:00:50.163934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:54.587947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182273+0000","last_change":"2024-04-25T00:00:50.178288+0000","last_active":"2024-04-25T00:00:50.182273+0000","last_peered":"2024-04-25T00:00:50.182273+0000","last_clean":"2024-04-25T00:00:50.182273+0000","last_became_active":"2024-04-25T00:00:50.177948+0000","last_became_peered":"2024-04-25T00:00:50.177948+0000","last_unstale":"2024-04-25T00:00:50.182273+0000","last_undegraded":"2024-04-25T00:00:50.182273+0000","last_fullsized":"2024-04-25T00:00:50.182273+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:42.536278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.166151+0000","last_change":"2024-04-25T00:00:49.169645+0000","last_active":"2024-04-25T00:00:50.166151+0000","last_peered":"2024-04-25T00:00:50.166151+0000","last_clean":"2024-04-25T00:00:50.166151+0000","last_became_active":"2024-04-25T00:00:49.168693+0000","last_became_peered":"2024-04-25T00:00:49.168693+0000","last_unstale":"2024-04-25T00:00:50.166151+0000","last_undegraded":"2024-04-25T00:00:50.166151+0000","last_fullsized":"2024-04-25T00:00:50.166151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:11.439854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178709+0000","last_change":"2024-04-25T00:00:50.176283+0000","last_active":"2024-04-25T00:00:50.178709+0000","last_peered":"2024-04-25T00:00:50.178709+0000","last_clean":"2024-04-25T00:00:50.178709+0000","last_became_active":"2024-04-25T00:00:50.175992+0000","last_became_peered":"2024-04-25T00:00:50.175992+0000","last_unstale":"2024-04-25T00:00:50.178709+0000","last_undegraded":"2024-04-25T00:00:50.178709+0000","last_fullsized":"2024-04-25T00:00:50.178709+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:46.053871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.169232+0000","last_change":"2024-04-25T00:00:49.176890+0000","last_active":"2024-04-25T00:00:50.169232+0000","last_peered":"2024-04-25T00:00:50.169232+0000","last_clean":"2024-04-25T00:00:50.169232+0000","last_became_active":"2024-04-25T00:00:49.176601+0000","last_became_peered":"2024-04-25T00:00:49.176601+0000","last_unstale":"2024-04-25T00:00:50.169232+0000","last_undegraded":"2024-04-25T00:00:50.169232+0000","last_fullsized":"2024-04-25T00:00:50.169232+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:06.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182289+0000","last_change":"2024-04-25T00:00:50.181444+0000","last_active":"2024-04-25T00:00:50.182289+0000","last_peered":"2024-04-25T00:00:50.182289+0000","last_clean":"2024-04-25T00:00:50.182289+0000","last_became_active":"2024-04-25T00:00:50.181190+0000","last_became_peered":"2024-04-25T00:00:50.181190+0000","last_unstale":"2024-04-25T00:00:50.182289+0000","last_undegraded":"2024-04-25T00:00:50.182289+0000","last_fullsized":"2024-04-25T00:00:50.182289+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:01.794662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.165391+0000","last_change":"2024-04-25T00:00:49.175495+0000","last_active":"2024-04-25T00:00:50.165391+0000","last_peered":"2024-04-25T00:00:50.165391+0000","last_clean":"2024-04-25T00:00:50.165391+0000","last_became_active":"2024-04-25T00:00:49.175220+0000","last_became_peered":"2024-04-25T00:00:49.175220+0000","last_unstale":"2024-04-25T00:00:50.165391+0000","last_undegraded":"2024-04-25T00:00:50.165391+0000","last_fullsized":"2024-04-25T00:00:50.165391+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:47.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174109+0000","last_change":"2024-04-25T00:00:50.166958+0000","last_active":"2024-04-25T00:00:50.174109+0000","last_peered":"2024-04-25T00:00:50.174109+0000","last_clean":"2024-04-25T00:00:50.174109+0000","last_became_active":"2024-04-25T00:00:50.166655+0000","last_became_peered":"2024-04-25T00:00:50.166655+0000","last_unstale":"2024-04-25T00:00:50.174109+0000","last_undegraded":"2024-04-25T00:00:50.174109+0000","last_fullsized":"2024-04-25T00:00:50.174109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:13.489857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.168792+0000","last_change":"2024-04-25T00:00:49.173103+0000","last_active":"2024-04-25T00:00:50.168792+0000","last_peered":"2024-04-25T00:00:50.168792+0000","last_clean":"2024-04-25T00:00:50.168792+0000","last_became_active":"2024-04-25T00:00:49.172838+0000","last_became_peered":"2024-04-25T00:00:49.172838+0000","last_unstale":"2024-04-25T00:00:50.168792+0000","last_undegraded":"2024-04-25T00:00:50.168792+0000","last_fullsized":"2024-04-25T00:00:50.168792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:57.564911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185303+0000","last_change":"2024-04-25T00:00:50.183725+0000","last_active":"2024-04-25T00:00:50.185303+0000","last_peered":"2024-04-25T00:00:50.185303+0000","last_clean":"2024-04-25T00:00:50.185303+0000","last_became_active":"2024-04-25T00:00:50.183451+0000","last_became_peered":"2024-04-25T00:00:50.183451+0000","last_unstale":"2024-04-25T00:00:50.185303+0000","last_undegraded":"2024-04-25T00:00:50.185303+0000","last_fullsized":"2024-04-25T00:00:50.185303+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:31.172329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171344+0000","last_change":"2024-04-25T00:00:49.171112+0000","last_active":"2024-04-25T00:00:50.171344+0000","last_peered":"2024-04-25T00:00:50.171344+0000","last_clean":"2024-04-25T00:00:50.171344+0000","last_became_active":"2024-04-25T00:00:49.170787+0000","last_became_peered":"2024-04-25T00:00:49.170787+0000","last_unstale":"2024-04-25T00:00:50.171344+0000","last_undegraded":"2024-04-25T00:00:50.171344+0000","last_fullsized":"2024-04-25T00:00:50.171344+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:08.428908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178646+0000","last_change":"2024-04-25T00:00:50.176148+0000","last_active":"2024-04-25T00:00:50.178646+0000","last_peered":"2024-04-25T00:00:50.178646+0000","last_clean":"2024-04-25T00:00:50.178646+0000","last_became_active":"2024-04-25T00:00:50.175876+0000","last_became_peered":"2024-04-25T00:00:50.175876+0000","last_unstale":"2024-04-25T00:00:50.178646+0000","last_undegraded":"2024-04-25T00:00:50.178646+0000","last_fullsized":"2024-04-25T00:00:50.178646+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:42.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.168220+0000","last_change":"2024-04-25T00:00:49.173749+0000","last_active":"2024-04-25T00:00:50.168220+0000","last_peered":"2024-04-25T00:00:50.168220+0000","last_clean":"2024-04-25T00:00:50.168220+0000","last_became_active":"2024-04-25T00:00:49.173485+0000","last_became_peered":"2024-04-25T00:00:49.173485+0000","last_unstale":"2024-04-25T00:00:50.168220+0000","last_undegraded":"2024-04-25T00:00:50.168220+0000","last_fullsized":"2024-04-25T00:00:50.168220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:43.177800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187470+0000","last_change":"2024-04-25T00:00:50.183923+0000","last_active":"2024-04-25T00:00:50.187470+0000","last_peered":"2024-04-25T00:00:50.187470+0000","last_clean":"2024-04-25T00:00:50.187470+0000","last_became_active":"2024-04-25T00:00:50.183580+0000","last_became_peered":"2024-04-25T00:00:50.183580+0000","last_unstale":"2024-04-25T00:00:50.187470+0000","last_undegraded":"2024-04-25T00:00:50.187470+0000","last_fullsized":"2024-04-25T00:00:50.187470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:26.866350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.160415+0000","last_change":"2024-04-25T00:00:49.174628+0000","last_active":"2024-04-25T00:00:50.160415+0000","last_peered":"2024-04-25T00:00:50.160415+0000","last_clean":"2024-04-25T00:00:50.160415+0000","last_became_active":"2024-04-25T00:00:49.174339+0000","last_became_peered":"2024-04-25T00:00:49.174339+0000","last_unstale":"2024-04-25T00:00:50.160415+0000","last_undegraded":"2024-04-25T00:00:50.160415+0000","last_fullsized":"2024-04-25T00:00:50.160415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:31.653269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.170807+0000","last_change":"2024-04-25T00:00:50.168796+0000","last_active":"2024-04-25T00:00:50.170807+0000","last_peered":"2024-04-25T00:00:50.170807+0000","last_clean":"2024-04-25T00:00:50.170807+0000","last_became_active":"2024-04-25T00:00:50.168530+0000","last_became_peered":"2024-04-25T00:00:50.168530+0000","last_unstale":"2024-04-25T00:00:50.170807+0000","last_undegraded":"2024-04-25T00:00:50.170807+0000","last_fullsized":"2024-04-25T00:00:50.170807+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:48.978718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171073+0000","last_change":"2024-04-25T00:00:49.169281+0000","last_active":"2024-04-25T00:00:50.171073+0000","last_peered":"2024-04-25T00:00:50.171073+0000","last_clean":"2024-04-25T00:00:50.171073+0000","last_became_active":"2024-04-25T00:00:49.169029+0000","last_became_peered":"2024-04-25T00:00:49.169029+0000","last_unstale":"2024-04-25T00:00:50.171073+0000","last_undegraded":"2024-04-25T00:00:50.171073+0000","last_fullsized":"2024-04-25T00:00:50.171073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:06.579218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174441+0000","last_change":"2024-04-25T00:00:50.170324+0000","last_active":"2024-04-25T00:00:50.174441+0000","last_peered":"2024-04-25T00:00:50.174441+0000","last_clean":"2024-04-25T00:00:50.174441+0000","last_became_active":"2024-04-25T00:00:50.170048+0000","last_became_peered":"2024-04-25T00:00:50.170048+0000","last_unstale":"2024-04-25T00:00:50.174441+0000","last_undegraded":"2024-04-25T00:00:50.174441+0000","last_fullsized":"2024-04-25T00:00:50.174441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.081284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.166261+0000","last_change":"2024-04-25T00:00:49.168252+0000","last_active":"2024-04-25T00:00:50.166261+0000","last_peered":"2024-04-25T00:00:50.166261+0000","last_clean":"2024-04-25T00:00:50.166261+0000","last_became_active":"2024-04-25T00:00:49.167986+0000","last_became_peered":"2024-04-25T00:00:49.167986+0000","last_unstale":"2024-04-25T00:00:50.166261+0000","last_undegraded":"2024-04-25T00:00:50.166261+0000","last_fullsized":"2024-04-25T00:00:50.166261+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:49.928618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187940+0000","last_change":"2024-04-25T00:00:50.183641+0000","last_active":"2024-04-25T00:00:50.187940+0000","last_peered":"2024-04-25T00:00:50.187940+0000","last_clean":"2024-04-25T00:00:50.187940+0000","last_became_active":"2024-04-25T00:00:50.183181+0000","last_became_peered":"2024-04-25T00:00:50.183181+0000","last_unstale":"2024-04-25T00:00:50.187940+0000","last_undegraded":"2024-04-25T00:00:50.187940+0000","last_fullsized":"2024-04-25T00:00:50.187940+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:42.049285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.180331+0000","last_change":"2024-04-25T00:00:49.173098+0000","last_active":"2024-04-25T00:00:50.180331+0000","last_peered":"2024-04-25T00:00:50.180331+0000","last_clean":"2024-04-25T00:00:50.180331+0000","last_became_active":"2024-04-25T00:00:49.172819+0000","last_became_peered":"2024-04-25T00:00:49.172819+0000","last_unstale":"2024-04-25T00:00:50.180331+0000","last_undegraded":"2024-04-25T00:00:50.180331+0000","last_fullsized":"2024-04-25T00:00:50.180331+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:48.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.169284+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:00:50.169284+0000","last_peered":"2024-04-25T00:00:50.169284+0000","last_clean":"2024-04-25T00:00:50.169284+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:00:50.169284+0000","last_undegraded":"2024-04-25T00:00:50.169284+0000","last_fullsized":"2024-04-25T00:00:50.169284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40604,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331236,"statfs":{"total":96636764160,"available":96595185664,"internally_reserved":0,"allocated":1126400,"data_stored":805597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":536576,"data_stored":215811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40024,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331816,"statfs":{"total":96636764160,"available":96595779584,"internally_reserved":0,"allocated":532480,"data_stored":215209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44192,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327648,"statfs":{"total":96636764160,"available":96591511552,"internally_reserved":0,"allocated":540672,"data_stored":216751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":540672,"data_stored":215777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":216773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":532480,"data_stored":215209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:53.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.258+0000 7f789557e640 1 Processor -- start 2024-04-25T00:00:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.258+0000 7f789557e640 1 -- start start 2024-04-25T00:00:53.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f789557e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f78900d2290 0x7f78900b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f789557e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78900d0b00 0x7f78900d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f789557e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78900d1480 con 0x7f789015e320 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f789557e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f78900d3410 con 0x7f78900d0b00 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f789557e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f78900d2800 con 0x7f78900d2290 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f788f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78900d0b00 0x7f78900d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f788effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f789015e320 0x7f78900d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43836/0 (socket says 172.21.15.113:43836) 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f788effd640 1 -- 172.21.15.113:0/754380141 learned_addr learned my addr 172.21.15.113:0/754380141 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.259+0000 7f788f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78900d0b00 0x7f78900d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54518/0 (socket says 172.21.15.113:54518) 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788e7fc640 1 --2- 172.21.15.113:0/754380141 >> v2:172.21.15.178:3300/0 conn(0x7f78900d2290 0x7f78900b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788f7fe640 1 -- 172.21.15.113:0/754380141 >> v2:172.21.15.178:3300/0 conn(0x7f78900d2290 msgr2=0x7f78900b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788f7fe640 1 --2- 172.21.15.113:0/754380141 >> v2:172.21.15.178:3300/0 conn(0x7f78900d2290 0x7f78900b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788f7fe640 1 -- 172.21.15.113:0/754380141 >> v1:172.21.15.113:6789/0 conn(0x7f789015e320 legacy=0x7f78900d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788f7fe640 1 -- 172.21.15.113:0/754380141 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78900d2ae0 con 0x7f78900d0b00 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788dffb640 1 -- 172.21.15.113:0/754380141 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301429180 0 0) 0x7f78900d1480 con 0x7f789015e320 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788e7fc640 1 --2- 172.21.15.113:0/754380141 >> v2:172.21.15.178:3300/0 conn(0x7f78900d2290 0x7f78900b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788f7fe640 1 --2- 172.21.15.113:0/754380141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d0b00 0x7f78900d0f40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7890072810 tx=0x7f788802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2680fbcb23939b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788dffb640 1 -- 172.21.15.113:0/754380141 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888037070 con 0x7f78900d0b00 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788dffb640 1 -- 172.21.15.113:0/754380141 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7888032440 con 0x7f78900d0b00 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.260+0000 7f788dffb640 1 -- 172.21.15.113:0/754380141 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888032810 con 0x7f78900d0b00 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 -- 172.21.15.113:0/754380141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d0b00 msgr2=0x7f78900d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 --2- 172.21.15.113:0/754380141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d0b00 0x7f78900d0f40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7890072810 tx=0x7f788802a230 comp rx=0 tx=0).stop 2024-04-25T00:00:53.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 -- 172.21.15.113:0/754380141 shutdown_connections 2024-04-25T00:00:53.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 --2- 172.21.15.113:0/754380141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d0b00 0x7f78900d0f40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 --2- 172.21.15.113:0/754380141 >> v2:172.21.15.178:3300/0 conn(0x7f78900d2290 0x7f78900b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 -- 172.21.15.113:0/754380141 >> 172.21.15.113:0/754380141 conn(0x7f78900b5c20 msgr2=0x7f78900b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 -- 172.21.15.113:0/754380141 shutdown_connections 2024-04-25T00:00:53.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.261+0000 7f789557e640 1 -- 172.21.15.113:0/754380141 wait complete. 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.262+0000 7f789557e640 1 Processor -- start 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.262+0000 7f789557e640 1 -- start start 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f789557e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78900d0b00 0x7f789015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f789557e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d2290 0x7f78901526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f789557e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789015e320 0x7f78901f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f789557e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f78900b9fe0 con 0x7f78900d2290 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f789557e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78900d30a0 con 0x7f78900d0b00 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f789557e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f789015e700 con 0x7f789015e320 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d2290 0x7f78901526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d2290 0x7f78901526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54528/0 (socket says 172.21.15.113:54528) 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 -- 172.21.15.113:0/4121805143 learned_addr learned my addr 172.21.15.113:0/4121805143 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788effd640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78900d0b00 0x7f789015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788f7fe640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789015e320 0x7f78901f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 -- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789015e320 msgr2=0x7f78901f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789015e320 0x7f78901f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 -- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78900d0b00 msgr2=0x7f789015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78900d0b00 0x7f789015f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788e7fc640 1 -- 172.21.15.113:0/4121805143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789015fa60 con 0x7f78900d2290 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.263+0000 7f788f7fe640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789015e320 0x7f78901f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.264+0000 7f788e7fc640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d2290 0x7f78901526a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f788400c7f0 tx=0x7f788400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.264+0000 7f78777fe640 1 -- 172.21.15.113:0/4121805143 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884015070 con 0x7f78900d2290 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.264+0000 7f78777fe640 1 -- 172.21.15.113:0/4121805143 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7884007a30 con 0x7f78900d2290 2024-04-25T00:00:53.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.264+0000 7f78777fe640 1 -- 172.21.15.113:0/4121805143 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884007e00 con 0x7f78900d2290 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.264+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789015fd40 con 0x7f78900d2290 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.264+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7890160300 con 0x7f78900d2290 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.265+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78900691a0 con 0x7f78900d2290 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.265+0000 7f78777fe640 1 -- 172.21.15.113:0/4121805143 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7884018070 con 0x7f78900d2290 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.266+0000 7f78777fe640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f787c06be30 0x7f787c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.266+0000 7f78777fe640 1 -- 172.21.15.113:0/4121805143 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f788408ad10 con 0x7f78900d2290 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.266+0000 7f788effd640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f787c06be30 0x7f787c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.266+0000 7f788effd640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f787c06be30 0x7f787c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78800012e0 tx=0x7f7880001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.270+0000 7f78777fe640 1 -- 172.21.15.113:0/4121805143 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7884059100 con 0x7f78900d2290 2024-04-25T00:00:53.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.440+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78900d3c20 con 0x7f787c06be30 2024-04-25T00:00:53.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.446+0000 7f78777fe640 1 -- 172.21.15.113:0/4121805143 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251892 (secure 0 0 0) 0x7f78900d3c20 con 0x7f787c06be30 2024-04-25T00:00:53.449 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:53.450 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:00:53.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f787c06be30 msgr2=0x7f787c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f787c06be30 0x7f787c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78800012e0 tx=0x7f7880001700 comp rx=0 tx=0).stop 2024-04-25T00:00:53.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d2290 msgr2=0x7f78901526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d2290 0x7f78901526a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f788400c7f0 tx=0x7f788400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 shutdown_connections 2024-04-25T00:00:53.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f787c06be30 0x7f787c06e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f789015e320 0x7f78901f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78900d2290 0x7f78901526a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 --2- 172.21.15.113:0/4121805143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78900d0b00 0x7f789015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 >> 172.21.15.113:0/4121805143 conn(0x7f78900b5c20 msgr2=0x7f78900b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 shutdown_connections 2024-04-25T00:00:53.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.449+0000 7f789557e640 1 -- 172.21.15.113:0/4121805143 wait complete. 2024-04-25T00:00:53.462 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-25T00:00:53.141171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300592,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754674128,"statfs":{"total":773094113280,"available":772786307072,"internally_reserved":0,"allocated":5480448,"data_stored":2906668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989350"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179078+0000","last_change":"2024-04-25T00:00:50.176895+0000","last_active":"2024-04-25T00:00:50.179078+0000","last_peered":"2024-04-25T00:00:50.179078+0000","last_clean":"2024-04-25T00:00:50.179078+0000","last_became_active":"2024-04-25T00:00:50.176568+0000","last_became_peered":"2024-04-25T00:00:50.176568+0000","last_unstale":"2024-04-25T00:00:50.179078+0000","last_undegraded":"2024-04-25T00:00:50.179078+0000","last_fullsized":"2024-04-25T00:00:50.179078+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:05.593209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179607+0000","last_change":"2024-04-25T00:00:50.178757+0000","last_active":"2024-04-25T00:00:50.179607+0000","last_peered":"2024-04-25T00:00:50.179607+0000","last_clean":"2024-04-25T00:00:50.179607+0000","last_became_active":"2024-04-25T00:00:50.178492+0000","last_became_peered":"2024-04-25T00:00:50.178492+0000","last_unstale":"2024-04-25T00:00:50.179607+0000","last_undegraded":"2024-04-25T00:00:50.179607+0000","last_fullsized":"2024-04-25T00:00:50.179607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:50.761828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174189+0000","last_change":"2024-04-25T00:00:50.170062+0000","last_active":"2024-04-25T00:00:50.174189+0000","last_peered":"2024-04-25T00:00:50.174189+0000","last_clean":"2024-04-25T00:00:50.174189+0000","last_became_active":"2024-04-25T00:00:50.169751+0000","last_became_peered":"2024-04-25T00:00:50.169751+0000","last_unstale":"2024-04-25T00:00:50.174189+0000","last_undegraded":"2024-04-25T00:00:50.174189+0000","last_fullsized":"2024-04-25T00:00:50.174189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:33.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182777+0000","last_change":"2024-04-25T00:00:50.178974+0000","last_active":"2024-04-25T00:00:50.182777+0000","last_peered":"2024-04-25T00:00:50.182777+0000","last_clean":"2024-04-25T00:00:50.182777+0000","last_became_active":"2024-04-25T00:00:50.178114+0000","last_became_peered":"2024-04-25T00:00:50.178114+0000","last_unstale":"2024-04-25T00:00:50.182777+0000","last_undegraded":"2024-04-25T00:00:50.182777+0000","last_fullsized":"2024-04-25T00:00:50.182777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:41.076029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179147+0000","last_change":"2024-04-25T00:00:50.177282+0000","last_active":"2024-04-25T00:00:50.179147+0000","last_peered":"2024-04-25T00:00:50.179147+0000","last_clean":"2024-04-25T00:00:50.179147+0000","last_became_active":"2024-04-25T00:00:50.177002+0000","last_became_peered":"2024-04-25T00:00:50.177002+0000","last_unstale":"2024-04-25T00:00:50.179147+0000","last_undegraded":"2024-04-25T00:00:50.179147+0000","last_fullsized":"2024-04-25T00:00:50.179147+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:07.477832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178666+0000","last_change":"2024-04-25T00:00:50.176031+0000","last_active":"2024-04-25T00:00:50.178666+0000","last_peered":"2024-04-25T00:00:50.178666+0000","last_clean":"2024-04-25T00:00:50.178666+0000","last_became_active":"2024-04-25T00:00:50.175530+0000","last_became_peered":"2024-04-25T00:00:50.175530+0000","last_unstale":"2024-04-25T00:00:50.178666+0000","last_undegraded":"2024-04-25T00:00:50.178666+0000","last_fullsized":"2024-04-25T00:00:50.178666+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:00.602047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.176249+0000","last_change":"2024-04-25T00:00:50.175027+0000","last_active":"2024-04-25T00:00:50.176249+0000","last_peered":"2024-04-25T00:00:50.176249+0000","last_clean":"2024-04-25T00:00:50.176249+0000","last_became_active":"2024-04-25T00:00:50.174755+0000","last_became_peered":"2024-04-25T00:00:50.174755+0000","last_unstale":"2024-04-25T00:00:50.176249+0000","last_undegraded":"2024-04-25T00:00:50.176249+0000","last_fullsized":"2024-04-25T00:00:50.176249+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:22.909301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184884+0000","last_change":"2024-04-25T00:00:50.180116+0000","last_active":"2024-04-25T00:00:50.184884+0000","last_peered":"2024-04-25T00:00:50.184884+0000","last_clean":"2024-04-25T00:00:50.184884+0000","last_became_active":"2024-04-25T00:00:50.177826+0000","last_became_peered":"2024-04-25T00:00:50.177826+0000","last_unstale":"2024-04-25T00:00:50.184884+0000","last_undegraded":"2024-04-25T00:00:50.184884+0000","last_fullsized":"2024-04-25T00:00:50.184884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:07.584903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182684+0000","last_change":"2024-04-25T00:00:50.179300+0000","last_active":"2024-04-25T00:00:50.182684+0000","last_peered":"2024-04-25T00:00:50.182684+0000","last_clean":"2024-04-25T00:00:50.182684+0000","last_became_active":"2024-04-25T00:00:50.179034+0000","last_became_peered":"2024-04-25T00:00:50.179034+0000","last_unstale":"2024-04-25T00:00:50.182684+0000","last_undegraded":"2024-04-25T00:00:50.182684+0000","last_fullsized":"2024-04-25T00:00:50.182684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:07.670348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189760+0000","last_change":"2024-04-25T00:00:50.184673+0000","last_active":"2024-04-25T00:00:50.189760+0000","last_peered":"2024-04-25T00:00:50.189760+0000","last_clean":"2024-04-25T00:00:50.189760+0000","last_became_active":"2024-04-25T00:00:50.182827+0000","last_became_peered":"2024-04-25T00:00:50.182827+0000","last_unstale":"2024-04-25T00:00:50.189760+0000","last_undegraded":"2024-04-25T00:00:50.189760+0000","last_fullsized":"2024-04-25T00:00:50.189760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:33.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182718+0000","last_change":"2024-04-25T00:00:50.179352+0000","last_active":"2024-04-25T00:00:50.182718+0000","last_peered":"2024-04-25T00:00:50.182718+0000","last_clean":"2024-04-25T00:00:50.182718+0000","last_became_active":"2024-04-25T00:00:50.179080+0000","last_became_peered":"2024-04-25T00:00:50.179080+0000","last_unstale":"2024-04-25T00:00:50.182718+0000","last_undegraded":"2024-04-25T00:00:50.182718+0000","last_fullsized":"2024-04-25T00:00:50.182718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:42.118208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185343+0000","last_change":"2024-04-25T00:00:50.183962+0000","last_active":"2024-04-25T00:00:50.185343+0000","last_peered":"2024-04-25T00:00:50.185343+0000","last_clean":"2024-04-25T00:00:50.185343+0000","last_became_active":"2024-04-25T00:00:50.183669+0000","last_became_peered":"2024-04-25T00:00:50.183669+0000","last_unstale":"2024-04-25T00:00:50.185343+0000","last_undegraded":"2024-04-25T00:00:50.185343+0000","last_fullsized":"2024-04-25T00:00:50.185343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:22.909301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182424+0000","last_change":"2024-04-25T00:00:50.174814+0000","last_active":"2024-04-25T00:00:50.182424+0000","last_peered":"2024-04-25T00:00:50.182424+0000","last_clean":"2024-04-25T00:00:50.182424+0000","last_became_active":"2024-04-25T00:00:50.174563+0000","last_became_peered":"2024-04-25T00:00:50.174563+0000","last_unstale":"2024-04-25T00:00:50.182424+0000","last_undegraded":"2024-04-25T00:00:50.182424+0000","last_fullsized":"2024-04-25T00:00:50.182424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:11.103329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184856+0000","last_change":"2024-04-25T00:00:50.179180+0000","last_active":"2024-04-25T00:00:50.184856+0000","last_peered":"2024-04-25T00:00:50.184856+0000","last_clean":"2024-04-25T00:00:50.184856+0000","last_became_active":"2024-04-25T00:00:50.178849+0000","last_became_peered":"2024-04-25T00:00:50.178849+0000","last_unstale":"2024-04-25T00:00:50.184856+0000","last_undegraded":"2024-04-25T00:00:50.184856+0000","last_fullsized":"2024-04-25T00:00:50.184856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:32.658955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.173308+0000","last_change":"2024-04-25T00:00:50.169414+0000","last_active":"2024-04-25T00:00:50.173308+0000","last_peered":"2024-04-25T00:00:50.173308+0000","last_clean":"2024-04-25T00:00:50.173308+0000","last_became_active":"2024-04-25T00:00:50.169138+0000","last_became_peered":"2024-04-25T00:00:50.169138+0000","last_unstale":"2024-04-25T00:00:50.173308+0000","last_undegraded":"2024-04-25T00:00:50.173308+0000","last_fullsized":"2024-04-25T00:00:50.173308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:40.318647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174775+0000","last_change":"2024-04-25T00:00:50.171041+0000","last_active":"2024-04-25T00:00:50.174775+0000","last_peered":"2024-04-25T00:00:50.174775+0000","last_clean":"2024-04-25T00:00:50.174775+0000","last_became_active":"2024-04-25T00:00:50.170766+0000","last_became_peered":"2024-04-25T00:00:50.170766+0000","last_unstale":"2024-04-25T00:00:50.174775+0000","last_undegraded":"2024-04-25T00:00:50.174775+0000","last_fullsized":"2024-04-25T00:00:50.174775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:22.130163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171079+0000","last_change":"2024-04-25T00:00:50.167295+0000","last_active":"2024-04-25T00:00:50.171079+0000","last_peered":"2024-04-25T00:00:50.171079+0000","last_clean":"2024-04-25T00:00:50.171079+0000","last_became_active":"2024-04-25T00:00:50.166974+0000","last_became_peered":"2024-04-25T00:00:50.166974+0000","last_unstale":"2024-04-25T00:00:50.171079+0000","last_undegraded":"2024-04-25T00:00:50.171079+0000","last_fullsized":"2024-04-25T00:00:50.171079+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:42.049285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174775+0000","last_change":"2024-04-25T00:00:50.170868+0000","last_active":"2024-04-25T00:00:50.174775+0000","last_peered":"2024-04-25T00:00:50.174775+0000","last_clean":"2024-04-25T00:00:50.174775+0000","last_became_active":"2024-04-25T00:00:50.169071+0000","last_became_peered":"2024-04-25T00:00:50.169071+0000","last_unstale":"2024-04-25T00:00:50.174775+0000","last_undegraded":"2024-04-25T00:00:50.174775+0000","last_fullsized":"2024-04-25T00:00:50.174775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:52.739391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182238+0000","last_change":"2024-04-25T00:00:50.181255+0000","last_active":"2024-04-25T00:00:50.182238+0000","last_peered":"2024-04-25T00:00:50.182238+0000","last_clean":"2024-04-25T00:00:50.182238+0000","last_became_active":"2024-04-25T00:00:50.180923+0000","last_became_peered":"2024-04-25T00:00:50.180923+0000","last_unstale":"2024-04-25T00:00:50.182238+0000","last_undegraded":"2024-04-25T00:00:50.182238+0000","last_fullsized":"2024-04-25T00:00:50.182238+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:13.489857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182321+0000","last_change":"2024-04-25T00:00:50.172671+0000","last_active":"2024-04-25T00:00:50.182321+0000","last_peered":"2024-04-25T00:00:50.182321+0000","last_clean":"2024-04-25T00:00:50.182321+0000","last_became_active":"2024-04-25T00:00:50.172421+0000","last_became_peered":"2024-04-25T00:00:50.172421+0000","last_unstale":"2024-04-25T00:00:50.182321+0000","last_undegraded":"2024-04-25T00:00:50.182321+0000","last_fullsized":"2024-04-25T00:00:50.182321+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:07.165116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.175124+0000","last_change":"2024-04-25T00:00:50.168942+0000","last_active":"2024-04-25T00:00:50.175124+0000","last_peered":"2024-04-25T00:00:50.175124+0000","last_clean":"2024-04-25T00:00:50.175124+0000","last_became_active":"2024-04-25T00:00:50.168670+0000","last_became_peered":"2024-04-25T00:00:50.168670+0000","last_unstale":"2024-04-25T00:00:50.175124+0000","last_undegraded":"2024-04-25T00:00:50.175124+0000","last_fullsized":"2024-04-25T00:00:50.175124+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:24.112405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182630+0000","last_change":"2024-04-25T00:00:50.178640+0000","last_active":"2024-04-25T00:00:50.182630+0000","last_peered":"2024-04-25T00:00:50.182630+0000","last_clean":"2024-04-25T00:00:50.182630+0000","last_became_active":"2024-04-25T00:00:50.178337+0000","last_became_peered":"2024-04-25T00:00:50.178337+0000","last_unstale":"2024-04-25T00:00:50.182630+0000","last_undegraded":"2024-04-25T00:00:50.182630+0000","last_fullsized":"2024-04-25T00:00:50.182630+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:17.423642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177064+0000","last_change":"2024-04-25T00:00:50.172769+0000","last_active":"2024-04-25T00:00:50.177064+0000","last_peered":"2024-04-25T00:00:50.177064+0000","last_clean":"2024-04-25T00:00:50.177064+0000","last_became_active":"2024-04-25T00:00:50.172145+0000","last_became_peered":"2024-04-25T00:00:50.172145+0000","last_unstale":"2024-04-25T00:00:50.177064+0000","last_undegraded":"2024-04-25T00:00:50.177064+0000","last_fullsized":"2024-04-25T00:00:50.177064+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:07.165116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182379+0000","last_change":"2024-04-25T00:00:50.178526+0000","last_active":"2024-04-25T00:00:50.182379+0000","last_peered":"2024-04-25T00:00:50.182379+0000","last_clean":"2024-04-25T00:00:50.182379+0000","last_became_active":"2024-04-25T00:00:50.178262+0000","last_became_peered":"2024-04-25T00:00:50.178262+0000","last_unstale":"2024-04-25T00:00:50.182379+0000","last_undegraded":"2024-04-25T00:00:50.182379+0000","last_fullsized":"2024-04-25T00:00:50.182379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:58.265142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177694+0000","last_change":"2024-04-25T00:00:50.174190+0000","last_active":"2024-04-25T00:00:50.177694+0000","last_peered":"2024-04-25T00:00:50.177694+0000","last_clean":"2024-04-25T00:00:50.177694+0000","last_became_active":"2024-04-25T00:00:50.173471+0000","last_became_peered":"2024-04-25T00:00:50.173471+0000","last_unstale":"2024-04-25T00:00:50.177694+0000","last_undegraded":"2024-04-25T00:00:50.177694+0000","last_fullsized":"2024-04-25T00:00:50.177694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:17.944703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182211+0000","last_change":"2024-04-25T00:00:50.175787+0000","last_active":"2024-04-25T00:00:50.182211+0000","last_peered":"2024-04-25T00:00:50.182211+0000","last_clean":"2024-04-25T00:00:50.182211+0000","last_became_active":"2024-04-25T00:00:50.175544+0000","last_became_peered":"2024-04-25T00:00:50.175544+0000","last_unstale":"2024-04-25T00:00:50.182211+0000","last_undegraded":"2024-04-25T00:00:50.182211+0000","last_fullsized":"2024-04-25T00:00:50.182211+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:56:17.944703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.169671+0000","last_change":"2024-04-25T00:00:50.167288+0000","last_active":"2024-04-25T00:00:50.169671+0000","last_peered":"2024-04-25T00:00:50.169671+0000","last_clean":"2024-04-25T00:00:50.169671+0000","last_became_active":"2024-04-25T00:00:50.166905+0000","last_became_peered":"2024-04-25T00:00:50.166905+0000","last_unstale":"2024-04-25T00:00:50.169671+0000","last_undegraded":"2024-04-25T00:00:50.169671+0000","last_fullsized":"2024-04-25T00:00:50.169671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:07.584903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185363+0000","last_change":"2024-04-25T00:00:50.184000+0000","last_active":"2024-04-25T00:00:50.185363+0000","last_peered":"2024-04-25T00:00:50.185363+0000","last_clean":"2024-04-25T00:00:50.185363+0000","last_became_active":"2024-04-25T00:00:50.183756+0000","last_became_peered":"2024-04-25T00:00:50.183756+0000","last_unstale":"2024-04-25T00:00:50.185363+0000","last_undegraded":"2024-04-25T00:00:50.185363+0000","last_fullsized":"2024-04-25T00:00:50.185363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:20.121342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177020+0000","last_change":"2024-04-25T00:00:50.172382+0000","last_active":"2024-04-25T00:00:50.177020+0000","last_peered":"2024-04-25T00:00:50.177020+0000","last_clean":"2024-04-25T00:00:50.177020+0000","last_became_active":"2024-04-25T00:00:50.172117+0000","last_became_peered":"2024-04-25T00:00:50.172117+0000","last_unstale":"2024-04-25T00:00:50.177020+0000","last_undegraded":"2024-04-25T00:00:50.177020+0000","last_fullsized":"2024-04-25T00:00:50.177020+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:53.780711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.176732+0000","last_change":"2024-04-25T00:00:50.175771+0000","last_active":"2024-04-25T00:00:50.176732+0000","last_peered":"2024-04-25T00:00:50.176732+0000","last_clean":"2024-04-25T00:00:50.176732+0000","last_became_active":"2024-04-25T00:00:50.175510+0000","last_became_peered":"2024-04-25T00:00:50.175510+0000","last_unstale":"2024-04-25T00:00:50.176732+0000","last_undegraded":"2024-04-25T00:00:50.176732+0000","last_fullsized":"2024-04-25T00:00:50.176732+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:10.440124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.173284+0000","last_change":"2024-04-25T00:00:50.168560+0000","last_active":"2024-04-25T00:00:50.173284+0000","last_peered":"2024-04-25T00:00:50.173284+0000","last_clean":"2024-04-25T00:00:50.173284+0000","last_became_active":"2024-04-25T00:00:50.168317+0000","last_became_peered":"2024-04-25T00:00:50.168317+0000","last_unstale":"2024-04-25T00:00:50.173284+0000","last_undegraded":"2024-04-25T00:00:50.173284+0000","last_fullsized":"2024-04-25T00:00:50.173284+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:31.172329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182248+0000","last_change":"2024-04-25T00:00:50.171221+0000","last_active":"2024-04-25T00:00:50.182248+0000","last_peered":"2024-04-25T00:00:50.182248+0000","last_clean":"2024-04-25T00:00:50.182248+0000","last_became_active":"2024-04-25T00:00:50.170906+0000","last_became_peered":"2024-04-25T00:00:50.170906+0000","last_unstale":"2024-04-25T00:00:50.182248+0000","last_undegraded":"2024-04-25T00:00:50.182248+0000","last_fullsized":"2024-04-25T00:00:50.182248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:59:53.780711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.186614+0000","last_change":"2024-04-25T00:00:50.181146+0000","last_active":"2024-04-25T00:00:50.186614+0000","last_peered":"2024-04-25T00:00:50.186614+0000","last_clean":"2024-04-25T00:00:50.186614+0000","last_became_active":"2024-04-25T00:00:50.180456+0000","last_became_peered":"2024-04-25T00:00:50.180456+0000","last_unstale":"2024-04-25T00:00:50.186614+0000","last_undegraded":"2024-04-25T00:00:50.186614+0000","last_fullsized":"2024-04-25T00:00:50.186614+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.081284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189962+0000","last_change":"2024-04-25T00:00:49.167909+0000","last_active":"2024-04-25T00:00:50.189962+0000","last_peered":"2024-04-25T00:00:50.189962+0000","last_clean":"2024-04-25T00:00:50.189962+0000","last_became_active":"2024-04-25T00:00:49.167641+0000","last_became_peered":"2024-04-25T00:00:49.167641+0000","last_unstale":"2024-04-25T00:00:50.189962+0000","last_undegraded":"2024-04-25T00:00:50.189962+0000","last_fullsized":"2024-04-25T00:00:50.189962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:21.903615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.181680+0000","last_change":"2024-04-25T00:00:50.179521+0000","last_active":"2024-04-25T00:00:50.181680+0000","last_peered":"2024-04-25T00:00:50.181680+0000","last_clean":"2024-04-25T00:00:50.181680+0000","last_became_active":"2024-04-25T00:00:50.179249+0000","last_became_peered":"2024-04-25T00:00:50.179249+0000","last_unstale":"2024-04-25T00:00:50.181680+0000","last_undegraded":"2024-04-25T00:00:50.181680+0000","last_fullsized":"2024-04-25T00:00:50.181680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:12.445539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182768+0000","last_change":"2024-04-25T00:00:49.175235+0000","last_active":"2024-04-25T00:00:50.182768+0000","last_peered":"2024-04-25T00:00:50.182768+0000","last_clean":"2024-04-25T00:00:50.182768+0000","last_became_active":"2024-04-25T00:00:49.174947+0000","last_became_peered":"2024-04-25T00:00:49.174947+0000","last_unstale":"2024-04-25T00:00:50.182768+0000","last_undegraded":"2024-04-25T00:00:50.182768+0000","last_fullsized":"2024-04-25T00:00:50.182768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:41.112523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.172464+0000","last_change":"2024-04-25T00:00:50.171268+0000","last_active":"2024-04-25T00:00:50.172464+0000","last_peered":"2024-04-25T00:00:50.172464+0000","last_clean":"2024-04-25T00:00:50.172464+0000","last_became_active":"2024-04-25T00:00:50.170979+0000","last_became_peered":"2024-04-25T00:00:50.170979+0000","last_unstale":"2024-04-25T00:00:50.172464+0000","last_undegraded":"2024-04-25T00:00:50.172464+0000","last_fullsized":"2024-04-25T00:00:50.172464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:58.570596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184112+0000","last_change":"2024-04-25T00:00:49.172463+0000","last_active":"2024-04-25T00:00:50.184112+0000","last_peered":"2024-04-25T00:00:50.184112+0000","last_clean":"2024-04-25T00:00:50.184112+0000","last_became_active":"2024-04-25T00:00:49.171973+0000","last_became_peered":"2024-04-25T00:00:49.171973+0000","last_unstale":"2024-04-25T00:00:50.184112+0000","last_undegraded":"2024-04-25T00:00:50.184112+0000","last_fullsized":"2024-04-25T00:00:50.184112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:32.465721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.186016+0000","last_change":"2024-04-25T00:00:50.181954+0000","last_active":"2024-04-25T00:00:50.186016+0000","last_peered":"2024-04-25T00:00:50.186016+0000","last_clean":"2024-04-25T00:00:50.186016+0000","last_became_active":"2024-04-25T00:00:50.181378+0000","last_became_peered":"2024-04-25T00:00:50.181378+0000","last_unstale":"2024-04-25T00:00:50.186016+0000","last_undegraded":"2024-04-25T00:00:50.186016+0000","last_fullsized":"2024-04-25T00:00:50.186016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:52.739391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185501+0000","last_change":"2024-04-25T00:00:49.174628+0000","last_active":"2024-04-25T00:00:50.185501+0000","last_peered":"2024-04-25T00:00:50.185501+0000","last_clean":"2024-04-25T00:00:50.185501+0000","last_became_active":"2024-04-25T00:00:49.174304+0000","last_became_peered":"2024-04-25T00:00:49.174304+0000","last_unstale":"2024-04-25T00:00:50.185501+0000","last_undegraded":"2024-04-25T00:00:50.185501+0000","last_fullsized":"2024-04-25T00:00:50.185501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:49.756143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.175089+0000","last_change":"2024-04-25T00:00:50.173978+0000","last_active":"2024-04-25T00:00:50.175089+0000","last_peered":"2024-04-25T00:00:50.175089+0000","last_clean":"2024-04-25T00:00:50.175089+0000","last_became_active":"2024-04-25T00:00:50.173585+0000","last_became_peered":"2024-04-25T00:00:50.173585+0000","last_unstale":"2024-04-25T00:00:50.175089+0000","last_undegraded":"2024-04-25T00:00:50.175089+0000","last_fullsized":"2024-04-25T00:00:50.175089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:21.820344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189932+0000","last_change":"2024-04-25T00:00:49.172459+0000","last_active":"2024-04-25T00:00:50.189932+0000","last_peered":"2024-04-25T00:00:50.189932+0000","last_clean":"2024-04-25T00:00:50.189932+0000","last_became_active":"2024-04-25T00:00:49.172030+0000","last_became_peered":"2024-04-25T00:00:49.172030+0000","last_unstale":"2024-04-25T00:00:50.189932+0000","last_undegraded":"2024-04-25T00:00:50.189932+0000","last_fullsized":"2024-04-25T00:00:50.189932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:46.658861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189900+0000","last_change":"2024-04-25T00:00:50.186151+0000","last_active":"2024-04-25T00:00:50.189900+0000","last_peered":"2024-04-25T00:00:50.189900+0000","last_clean":"2024-04-25T00:00:50.189900+0000","last_became_active":"2024-04-25T00:00:50.185447+0000","last_became_peered":"2024-04-25T00:00:50.185447+0000","last_unstale":"2024-04-25T00:00:50.189900+0000","last_undegraded":"2024-04-25T00:00:50.189900+0000","last_fullsized":"2024-04-25T00:00:50.189900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:52.739391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.180377+0000","last_change":"2024-04-25T00:00:49.174057+0000","last_active":"2024-04-25T00:00:50.180377+0000","last_peered":"2024-04-25T00:00:50.180377+0000","last_clean":"2024-04-25T00:00:50.180377+0000","last_became_active":"2024-04-25T00:00:49.173752+0000","last_became_peered":"2024-04-25T00:00:49.173752+0000","last_unstale":"2024-04-25T00:00:50.180377+0000","last_undegraded":"2024-04-25T00:00:50.180377+0000","last_fullsized":"2024-04-25T00:00:50.180377+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:06.664663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187712+0000","last_change":"2024-04-25T00:00:50.182807+0000","last_active":"2024-04-25T00:00:50.187712+0000","last_peered":"2024-04-25T00:00:50.187712+0000","last_clean":"2024-04-25T00:00:50.187712+0000","last_became_active":"2024-04-25T00:00:50.179689+0000","last_became_peered":"2024-04-25T00:00:50.179689+0000","last_unstale":"2024-04-25T00:00:50.187712+0000","last_undegraded":"2024-04-25T00:00:50.187712+0000","last_fullsized":"2024-04-25T00:00:50.187712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.434593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.163397+0000","last_change":"2024-04-25T00:00:49.170345+0000","last_active":"2024-04-25T00:00:50.163397+0000","last_peered":"2024-04-25T00:00:50.163397+0000","last_clean":"2024-04-25T00:00:50.163397+0000","last_became_active":"2024-04-25T00:00:49.169838+0000","last_became_peered":"2024-04-25T00:00:49.169838+0000","last_unstale":"2024-04-25T00:00:50.163397+0000","last_undegraded":"2024-04-25T00:00:50.163397+0000","last_fullsized":"2024-04-25T00:00:50.163397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:14.384270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177087+0000","last_change":"2024-04-25T00:00:50.172999+0000","last_active":"2024-04-25T00:00:50.177087+0000","last_peered":"2024-04-25T00:00:50.177087+0000","last_clean":"2024-04-25T00:00:50.177087+0000","last_became_active":"2024-04-25T00:00:50.172508+0000","last_became_peered":"2024-04-25T00:00:50.172508+0000","last_unstale":"2024-04-25T00:00:50.177087+0000","last_undegraded":"2024-04-25T00:00:50.177087+0000","last_fullsized":"2024-04-25T00:00:50.177087+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:11.103329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.165954+0000","last_change":"2024-04-25T00:00:49.169507+0000","last_active":"2024-04-25T00:00:50.165954+0000","last_peered":"2024-04-25T00:00:50.165954+0000","last_clean":"2024-04-25T00:00:50.165954+0000","last_became_active":"2024-04-25T00:00:49.169219+0000","last_became_peered":"2024-04-25T00:00:49.169219+0000","last_unstale":"2024-04-25T00:00:50.165954+0000","last_undegraded":"2024-04-25T00:00:50.165954+0000","last_fullsized":"2024-04-25T00:00:50.165954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:41.530593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171023+0000","last_change":"2024-04-25T00:00:50.169339+0000","last_active":"2024-04-25T00:00:50.171023+0000","last_peered":"2024-04-25T00:00:50.171023+0000","last_clean":"2024-04-25T00:00:50.171023+0000","last_became_active":"2024-04-25T00:00:50.168705+0000","last_became_peered":"2024-04-25T00:00:50.168705+0000","last_unstale":"2024-04-25T00:00:50.171023+0000","last_undegraded":"2024-04-25T00:00:50.171023+0000","last_fullsized":"2024-04-25T00:00:50.171023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.434593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.160179+0000","last_change":"2024-04-25T00:00:49.171738+0000","last_active":"2024-04-25T00:00:50.160179+0000","last_peered":"2024-04-25T00:00:50.160179+0000","last_clean":"2024-04-25T00:00:50.160179+0000","last_became_active":"2024-04-25T00:00:49.171454+0000","last_became_peered":"2024-04-25T00:00:49.171454+0000","last_unstale":"2024-04-25T00:00:50.160179+0000","last_undegraded":"2024-04-25T00:00:50.160179+0000","last_fullsized":"2024-04-25T00:00:50.160179+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:16.417956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178121+0000","last_change":"2024-04-25T00:00:50.176316+0000","last_active":"2024-04-25T00:00:50.178121+0000","last_peered":"2024-04-25T00:00:50.178121+0000","last_clean":"2024-04-25T00:00:50.178121+0000","last_became_active":"2024-04-25T00:00:50.175979+0000","last_became_peered":"2024-04-25T00:00:50.175979+0000","last_unstale":"2024-04-25T00:00:50.178121+0000","last_undegraded":"2024-04-25T00:00:50.178121+0000","last_fullsized":"2024-04-25T00:00:50.178121+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:47.664547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.167575+0000","last_change":"2024-04-25T00:00:49.167831+0000","last_active":"2024-04-25T00:00:50.167575+0000","last_peered":"2024-04-25T00:00:50.167575+0000","last_clean":"2024-04-25T00:00:50.167575+0000","last_became_active":"2024-04-25T00:00:49.167561+0000","last_became_peered":"2024-04-25T00:00:49.167561+0000","last_unstale":"2024-04-25T00:00:50.167575+0000","last_undegraded":"2024-04-25T00:00:50.167575+0000","last_fullsized":"2024-04-25T00:00:50.167575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:00.788977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187445+0000","last_change":"2024-04-25T00:00:50.184339+0000","last_active":"2024-04-25T00:00:50.187445+0000","last_peered":"2024-04-25T00:00:50.187445+0000","last_clean":"2024-04-25T00:00:50.187445+0000","last_became_active":"2024-04-25T00:00:50.183559+0000","last_became_peered":"2024-04-25T00:00:50.183559+0000","last_unstale":"2024-04-25T00:00:50.187445+0000","last_undegraded":"2024-04-25T00:00:50.187445+0000","last_fullsized":"2024-04-25T00:00:50.187445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:43.532179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185714+0000","last_change":"2024-04-25T00:00:49.171657+0000","last_active":"2024-04-25T00:00:50.185714+0000","last_peered":"2024-04-25T00:00:50.185714+0000","last_clean":"2024-04-25T00:00:50.185714+0000","last_became_active":"2024-04-25T00:00:49.171324+0000","last_became_peered":"2024-04-25T00:00:49.171324+0000","last_unstale":"2024-04-25T00:00:50.185714+0000","last_undegraded":"2024-04-25T00:00:50.185714+0000","last_fullsized":"2024-04-25T00:00:50.185714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:47.973032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.177949+0000","last_change":"2024-04-25T00:00:50.175787+0000","last_active":"2024-04-25T00:00:50.177949+0000","last_peered":"2024-04-25T00:00:50.177949+0000","last_clean":"2024-04-25T00:00:50.177949+0000","last_became_active":"2024-04-25T00:00:50.175463+0000","last_became_peered":"2024-04-25T00:00:50.175463+0000","last_unstale":"2024-04-25T00:00:50.177949+0000","last_undegraded":"2024-04-25T00:00:50.177949+0000","last_fullsized":"2024-04-25T00:00:50.177949+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:20.121342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.191497+0000","last_change":"2024-04-25T00:00:49.175971+0000","last_active":"2024-04-25T00:00:50.191497+0000","last_peered":"2024-04-25T00:00:50.191497+0000","last_clean":"2024-04-25T00:00:50.191497+0000","last_became_active":"2024-04-25T00:00:49.175652+0000","last_became_peered":"2024-04-25T00:00:49.175652+0000","last_unstale":"2024-04-25T00:00:50.191497+0000","last_undegraded":"2024-04-25T00:00:50.191497+0000","last_fullsized":"2024-04-25T00:00:50.191497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:37.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182348+0000","last_change":"2024-04-25T00:00:50.180393+0000","last_active":"2024-04-25T00:00:50.182348+0000","last_peered":"2024-04-25T00:00:50.182348+0000","last_clean":"2024-04-25T00:00:50.182348+0000","last_became_active":"2024-04-25T00:00:50.180143+0000","last_became_peered":"2024-04-25T00:00:50.180143+0000","last_unstale":"2024-04-25T00:00:50.182348+0000","last_undegraded":"2024-04-25T00:00:50.182348+0000","last_fullsized":"2024-04-25T00:00:50.182348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:50.934303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.190309+0000","last_change":"2024-04-25T00:00:49.170274+0000","last_active":"2024-04-25T00:00:50.190309+0000","last_peered":"2024-04-25T00:00:50.190309+0000","last_clean":"2024-04-25T00:00:50.190309+0000","last_became_active":"2024-04-25T00:00:49.169953+0000","last_became_peered":"2024-04-25T00:00:49.169953+0000","last_unstale":"2024-04-25T00:00:50.190309+0000","last_undegraded":"2024-04-25T00:00:50.190309+0000","last_fullsized":"2024-04-25T00:00:50.190309+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:19.115656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.189878+0000","last_change":"2024-04-25T00:00:50.185927+0000","last_active":"2024-04-25T00:00:50.189878+0000","last_peered":"2024-04-25T00:00:50.189878+0000","last_clean":"2024-04-25T00:00:50.189878+0000","last_became_active":"2024-04-25T00:00:50.185099+0000","last_became_peered":"2024-04-25T00:00:50.185099+0000","last_unstale":"2024-04-25T00:00:50.189878+0000","last_undegraded":"2024-04-25T00:00:50.189878+0000","last_fullsized":"2024-04-25T00:00:50.189878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.081284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.168146+0000","last_change":"2024-04-25T00:00:49.176979+0000","last_active":"2024-04-25T00:00:50.168146+0000","last_peered":"2024-04-25T00:00:50.168146+0000","last_clean":"2024-04-25T00:00:50.168146+0000","last_became_active":"2024-04-25T00:00:49.176548+0000","last_became_peered":"2024-04-25T00:00:49.176548+0000","last_unstale":"2024-04-25T00:00:50.168146+0000","last_undegraded":"2024-04-25T00:00:50.168146+0000","last_fullsized":"2024-04-25T00:00:50.168146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:48.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.180170+0000","last_change":"2024-04-25T00:00:50.179399+0000","last_active":"2024-04-25T00:00:50.180170+0000","last_peered":"2024-04-25T00:00:50.180170+0000","last_clean":"2024-04-25T00:00:50.180170+0000","last_became_active":"2024-04-25T00:00:50.179135+0000","last_became_peered":"2024-04-25T00:00:50.179135+0000","last_unstale":"2024-04-25T00:00:50.180170+0000","last_undegraded":"2024-04-25T00:00:50.180170+0000","last_fullsized":"2024-04-25T00:00:50.180170+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:17.037457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.164044+0000","last_change":"2024-04-25T00:00:49.167252+0000","last_active":"2024-04-25T00:00:50.164044+0000","last_peered":"2024-04-25T00:00:50.164044+0000","last_clean":"2024-04-25T00:00:50.164044+0000","last_became_active":"2024-04-25T00:00:49.166310+0000","last_became_peered":"2024-04-25T00:00:49.166310+0000","last_unstale":"2024-04-25T00:00:50.164044+0000","last_undegraded":"2024-04-25T00:00:50.164044+0000","last_fullsized":"2024-04-25T00:00:50.164044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:49.756143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182461+0000","last_change":"2024-04-25T00:00:50.181502+0000","last_active":"2024-04-25T00:00:50.182461+0000","last_peered":"2024-04-25T00:00:50.182461+0000","last_clean":"2024-04-25T00:00:50.182461+0000","last_became_active":"2024-04-25T00:00:50.181212+0000","last_became_peered":"2024-04-25T00:00:50.181212+0000","last_unstale":"2024-04-25T00:00:50.182461+0000","last_undegraded":"2024-04-25T00:00:50.182461+0000","last_fullsized":"2024-04-25T00:00:50.182461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:48.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.188613+0000","last_change":"2024-04-25T00:00:49.170215+0000","last_active":"2024-04-25T00:00:50.188613+0000","last_peered":"2024-04-25T00:00:50.188613+0000","last_clean":"2024-04-25T00:00:50.188613+0000","last_became_active":"2024-04-25T00:00:49.169944+0000","last_became_peered":"2024-04-25T00:00:49.169944+0000","last_unstale":"2024-04-25T00:00:50.188613+0000","last_undegraded":"2024-04-25T00:00:50.188613+0000","last_fullsized":"2024-04-25T00:00:50.188613+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:57.259456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178082+0000","last_change":"2024-04-25T00:00:50.169260+0000","last_active":"2024-04-25T00:00:50.178082+0000","last_peered":"2024-04-25T00:00:50.178082+0000","last_clean":"2024-04-25T00:00:50.178082+0000","last_became_active":"2024-04-25T00:00:50.168977+0000","last_became_peered":"2024-04-25T00:00:50.168977+0000","last_unstale":"2024-04-25T00:00:50.178082+0000","last_undegraded":"2024-04-25T00:00:50.178082+0000","last_fullsized":"2024-04-25T00:00:50.178082+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:22.130163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.183529+0000","last_change":"2024-04-25T00:00:49.168844+0000","last_active":"2024-04-25T00:00:50.183529+0000","last_peered":"2024-04-25T00:00:50.183529+0000","last_clean":"2024-04-25T00:00:50.183529+0000","last_became_active":"2024-04-25T00:00:49.168530+0000","last_became_peered":"2024-04-25T00:00:49.168530+0000","last_unstale":"2024-04-25T00:00:50.183529+0000","last_undegraded":"2024-04-25T00:00:50.183529+0000","last_fullsized":"2024-04-25T00:00:50.183529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:58.853340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182665+0000","last_change":"2024-04-25T00:00:50.178521+0000","last_active":"2024-04-25T00:00:50.182665+0000","last_peered":"2024-04-25T00:00:50.182665+0000","last_clean":"2024-04-25T00:00:50.182665+0000","last_became_active":"2024-04-25T00:00:50.178209+0000","last_became_peered":"2024-04-25T00:00:50.178209+0000","last_unstale":"2024-04-25T00:00:50.182665+0000","last_undegraded":"2024-04-25T00:00:50.182665+0000","last_fullsized":"2024-04-25T00:00:50.182665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:31.835102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187065+0000","last_change":"2024-04-25T00:00:49.171183+0000","last_active":"2024-04-25T00:00:50.187065+0000","last_peered":"2024-04-25T00:00:50.187065+0000","last_clean":"2024-04-25T00:00:50.187065+0000","last_became_active":"2024-04-25T00:00:49.170878+0000","last_became_peered":"2024-04-25T00:00:49.170878+0000","last_unstale":"2024-04-25T00:00:50.187065+0000","last_undegraded":"2024-04-25T00:00:50.187065+0000","last_fullsized":"2024-04-25T00:00:50.187065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:40.070343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174471+0000","last_change":"2024-04-25T00:00:50.170486+0000","last_active":"2024-04-25T00:00:50.174471+0000","last_peered":"2024-04-25T00:00:50.174471+0000","last_clean":"2024-04-25T00:00:50.174471+0000","last_became_active":"2024-04-25T00:00:50.169865+0000","last_became_peered":"2024-04-25T00:00:50.169865+0000","last_unstale":"2024-04-25T00:00:50.174471+0000","last_undegraded":"2024-04-25T00:00:50.174471+0000","last_fullsized":"2024-04-25T00:00:50.174471+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:38.222251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.184509+0000","last_change":"2024-04-25T00:00:49.173921+0000","last_active":"2024-04-25T00:00:50.184509+0000","last_peered":"2024-04-25T00:00:50.184509+0000","last_clean":"2024-04-25T00:00:50.184509+0000","last_became_active":"2024-04-25T00:00:49.173654+0000","last_became_peered":"2024-04-25T00:00:49.173654+0000","last_unstale":"2024-04-25T00:00:50.184509+0000","last_undegraded":"2024-04-25T00:00:50.184509+0000","last_fullsized":"2024-04-25T00:00:50.184509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:43.177800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.179027+0000","last_change":"2024-04-25T00:00:50.177561+0000","last_active":"2024-04-25T00:00:50.179027+0000","last_peered":"2024-04-25T00:00:50.179027+0000","last_clean":"2024-04-25T00:00:50.179027+0000","last_became_active":"2024-04-25T00:00:50.177238+0000","last_became_peered":"2024-04-25T00:00:50.177238+0000","last_unstale":"2024-04-25T00:00:50.179027+0000","last_undegraded":"2024-04-25T00:00:50.179027+0000","last_fullsized":"2024-04-25T00:00:50.179027+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:42.536278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.163421+0000","last_change":"2024-04-25T00:00:49.171429+0000","last_active":"2024-04-25T00:00:50.163421+0000","last_peered":"2024-04-25T00:00:50.163421+0000","last_clean":"2024-04-25T00:00:50.163421+0000","last_became_active":"2024-04-25T00:00:49.171154+0000","last_became_peered":"2024-04-25T00:00:49.171154+0000","last_unstale":"2024-04-25T00:00:50.163421+0000","last_undegraded":"2024-04-25T00:00:50.163421+0000","last_fullsized":"2024-04-25T00:00:50.163421+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:06.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.176380+0000","last_change":"2024-04-25T00:00:50.170640+0000","last_active":"2024-04-25T00:00:50.176380+0000","last_peered":"2024-04-25T00:00:50.176380+0000","last_clean":"2024-04-25T00:00:50.176380+0000","last_became_active":"2024-04-25T00:00:50.170391+0000","last_became_peered":"2024-04-25T00:00:50.170391+0000","last_unstale":"2024-04-25T00:00:50.176380+0000","last_undegraded":"2024-04-25T00:00:50.176380+0000","last_fullsized":"2024-04-25T00:00:50.176380+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:24.139365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.190046+0000","last_change":"2024-04-25T00:00:49.177013+0000","last_active":"2024-04-25T00:00:50.190046+0000","last_peered":"2024-04-25T00:00:50.190046+0000","last_clean":"2024-04-25T00:00:50.190046+0000","last_became_active":"2024-04-25T00:00:49.176636+0000","last_became_peered":"2024-04-25T00:00:49.176636+0000","last_unstale":"2024-04-25T00:00:50.190046+0000","last_undegraded":"2024-04-25T00:00:50.190046+0000","last_fullsized":"2024-04-25T00:00:50.190046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:04.587524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.172793+0000","last_change":"2024-04-25T00:00:50.172328+0000","last_active":"2024-04-25T00:00:50.172793+0000","last_peered":"2024-04-25T00:00:50.172793+0000","last_clean":"2024-04-25T00:00:50.172793+0000","last_became_active":"2024-04-25T00:00:50.172058+0000","last_became_peered":"2024-04-25T00:00:50.172058+0000","last_unstale":"2024-04-25T00:00:50.172793+0000","last_undegraded":"2024-04-25T00:00:50.172793+0000","last_fullsized":"2024-04-25T00:00:50.172793+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:42.049285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.163934+0000","last_change":"2024-04-25T00:00:49.174029+0000","last_active":"2024-04-25T00:00:50.163934+0000","last_peered":"2024-04-25T00:00:50.163934+0000","last_clean":"2024-04-25T00:00:50.163934+0000","last_became_active":"2024-04-25T00:00:49.173181+0000","last_became_peered":"2024-04-25T00:00:49.173181+0000","last_unstale":"2024-04-25T00:00:50.163934+0000","last_undegraded":"2024-04-25T00:00:50.163934+0000","last_fullsized":"2024-04-25T00:00:50.163934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:54.587947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182273+0000","last_change":"2024-04-25T00:00:50.178288+0000","last_active":"2024-04-25T00:00:50.182273+0000","last_peered":"2024-04-25T00:00:50.182273+0000","last_clean":"2024-04-25T00:00:50.182273+0000","last_became_active":"2024-04-25T00:00:50.177948+0000","last_became_peered":"2024-04-25T00:00:50.177948+0000","last_unstale":"2024-04-25T00:00:50.182273+0000","last_undegraded":"2024-04-25T00:00:50.182273+0000","last_fullsized":"2024-04-25T00:00:50.182273+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:42.536278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.166151+0000","last_change":"2024-04-25T00:00:49.169645+0000","last_active":"2024-04-25T00:00:50.166151+0000","last_peered":"2024-04-25T00:00:50.166151+0000","last_clean":"2024-04-25T00:00:50.166151+0000","last_became_active":"2024-04-25T00:00:49.168693+0000","last_became_peered":"2024-04-25T00:00:49.168693+0000","last_unstale":"2024-04-25T00:00:50.166151+0000","last_undegraded":"2024-04-25T00:00:50.166151+0000","last_fullsized":"2024-04-25T00:00:50.166151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:11.439854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178709+0000","last_change":"2024-04-25T00:00:50.176283+0000","last_active":"2024-04-25T00:00:50.178709+0000","last_peered":"2024-04-25T00:00:50.178709+0000","last_clean":"2024-04-25T00:00:50.178709+0000","last_became_active":"2024-04-25T00:00:50.175992+0000","last_became_peered":"2024-04-25T00:00:50.175992+0000","last_unstale":"2024-04-25T00:00:50.178709+0000","last_undegraded":"2024-04-25T00:00:50.178709+0000","last_fullsized":"2024-04-25T00:00:50.178709+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:46.053871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.169232+0000","last_change":"2024-04-25T00:00:49.176890+0000","last_active":"2024-04-25T00:00:50.169232+0000","last_peered":"2024-04-25T00:00:50.169232+0000","last_clean":"2024-04-25T00:00:50.169232+0000","last_became_active":"2024-04-25T00:00:49.176601+0000","last_became_peered":"2024-04-25T00:00:49.176601+0000","last_unstale":"2024-04-25T00:00:50.169232+0000","last_undegraded":"2024-04-25T00:00:50.169232+0000","last_fullsized":"2024-04-25T00:00:50.169232+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:06.472147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.182289+0000","last_change":"2024-04-25T00:00:50.181444+0000","last_active":"2024-04-25T00:00:50.182289+0000","last_peered":"2024-04-25T00:00:50.182289+0000","last_clean":"2024-04-25T00:00:50.182289+0000","last_became_active":"2024-04-25T00:00:50.181190+0000","last_became_peered":"2024-04-25T00:00:50.181190+0000","last_unstale":"2024-04-25T00:00:50.182289+0000","last_undegraded":"2024-04-25T00:00:50.182289+0000","last_fullsized":"2024-04-25T00:00:50.182289+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:59:01.794662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.165391+0000","last_change":"2024-04-25T00:00:49.175495+0000","last_active":"2024-04-25T00:00:50.165391+0000","last_peered":"2024-04-25T00:00:50.165391+0000","last_clean":"2024-04-25T00:00:50.165391+0000","last_became_active":"2024-04-25T00:00:49.175220+0000","last_became_peered":"2024-04-25T00:00:49.175220+0000","last_unstale":"2024-04-25T00:00:50.165391+0000","last_undegraded":"2024-04-25T00:00:50.165391+0000","last_fullsized":"2024-04-25T00:00:50.165391+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:47.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174109+0000","last_change":"2024-04-25T00:00:50.166958+0000","last_active":"2024-04-25T00:00:50.174109+0000","last_peered":"2024-04-25T00:00:50.174109+0000","last_clean":"2024-04-25T00:00:50.174109+0000","last_became_active":"2024-04-25T00:00:50.166655+0000","last_became_peered":"2024-04-25T00:00:50.166655+0000","last_unstale":"2024-04-25T00:00:50.174109+0000","last_undegraded":"2024-04-25T00:00:50.174109+0000","last_fullsized":"2024-04-25T00:00:50.174109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:13.489857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.168792+0000","last_change":"2024-04-25T00:00:49.173103+0000","last_active":"2024-04-25T00:00:50.168792+0000","last_peered":"2024-04-25T00:00:50.168792+0000","last_clean":"2024-04-25T00:00:50.168792+0000","last_became_active":"2024-04-25T00:00:49.172838+0000","last_became_peered":"2024-04-25T00:00:49.172838+0000","last_unstale":"2024-04-25T00:00:50.168792+0000","last_undegraded":"2024-04-25T00:00:50.168792+0000","last_fullsized":"2024-04-25T00:00:50.168792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:57.564911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.185303+0000","last_change":"2024-04-25T00:00:50.183725+0000","last_active":"2024-04-25T00:00:50.185303+0000","last_peered":"2024-04-25T00:00:50.185303+0000","last_clean":"2024-04-25T00:00:50.185303+0000","last_became_active":"2024-04-25T00:00:50.183451+0000","last_became_peered":"2024-04-25T00:00:50.183451+0000","last_unstale":"2024-04-25T00:00:50.185303+0000","last_undegraded":"2024-04-25T00:00:50.185303+0000","last_fullsized":"2024-04-25T00:00:50.185303+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:31.172329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171344+0000","last_change":"2024-04-25T00:00:49.171112+0000","last_active":"2024-04-25T00:00:50.171344+0000","last_peered":"2024-04-25T00:00:50.171344+0000","last_clean":"2024-04-25T00:00:50.171344+0000","last_became_active":"2024-04-25T00:00:49.170787+0000","last_became_peered":"2024-04-25T00:00:49.170787+0000","last_unstale":"2024-04-25T00:00:50.171344+0000","last_undegraded":"2024-04-25T00:00:50.171344+0000","last_fullsized":"2024-04-25T00:00:50.171344+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:08.428908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.178646+0000","last_change":"2024-04-25T00:00:50.176148+0000","last_active":"2024-04-25T00:00:50.178646+0000","last_peered":"2024-04-25T00:00:50.178646+0000","last_clean":"2024-04-25T00:00:50.178646+0000","last_became_active":"2024-04-25T00:00:50.175876+0000","last_became_peered":"2024-04-25T00:00:50.175876+0000","last_unstale":"2024-04-25T00:00:50.178646+0000","last_undegraded":"2024-04-25T00:00:50.178646+0000","last_fullsized":"2024-04-25T00:00:50.178646+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:42.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.168220+0000","last_change":"2024-04-25T00:00:49.173749+0000","last_active":"2024-04-25T00:00:50.168220+0000","last_peered":"2024-04-25T00:00:50.168220+0000","last_clean":"2024-04-25T00:00:50.168220+0000","last_became_active":"2024-04-25T00:00:49.173485+0000","last_became_peered":"2024-04-25T00:00:49.173485+0000","last_unstale":"2024-04-25T00:00:50.168220+0000","last_undegraded":"2024-04-25T00:00:50.168220+0000","last_fullsized":"2024-04-25T00:00:50.168220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:43.177800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187470+0000","last_change":"2024-04-25T00:00:50.183923+0000","last_active":"2024-04-25T00:00:50.187470+0000","last_peered":"2024-04-25T00:00:50.187470+0000","last_clean":"2024-04-25T00:00:50.187470+0000","last_became_active":"2024-04-25T00:00:50.183580+0000","last_became_peered":"2024-04-25T00:00:50.183580+0000","last_unstale":"2024-04-25T00:00:50.187470+0000","last_undegraded":"2024-04-25T00:00:50.187470+0000","last_fullsized":"2024-04-25T00:00:50.187470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:26.866350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.160415+0000","last_change":"2024-04-25T00:00:49.174628+0000","last_active":"2024-04-25T00:00:50.160415+0000","last_peered":"2024-04-25T00:00:50.160415+0000","last_clean":"2024-04-25T00:00:50.160415+0000","last_became_active":"2024-04-25T00:00:49.174339+0000","last_became_peered":"2024-04-25T00:00:49.174339+0000","last_unstale":"2024-04-25T00:00:50.160415+0000","last_undegraded":"2024-04-25T00:00:50.160415+0000","last_fullsized":"2024-04-25T00:00:50.160415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:31.653269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.170807+0000","last_change":"2024-04-25T00:00:50.168796+0000","last_active":"2024-04-25T00:00:50.170807+0000","last_peered":"2024-04-25T00:00:50.170807+0000","last_clean":"2024-04-25T00:00:50.170807+0000","last_became_active":"2024-04-25T00:00:50.168530+0000","last_became_peered":"2024-04-25T00:00:50.168530+0000","last_unstale":"2024-04-25T00:00:50.170807+0000","last_undegraded":"2024-04-25T00:00:50.170807+0000","last_fullsized":"2024-04-25T00:00:50.170807+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:48.978718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.171073+0000","last_change":"2024-04-25T00:00:49.169281+0000","last_active":"2024-04-25T00:00:50.171073+0000","last_peered":"2024-04-25T00:00:50.171073+0000","last_clean":"2024-04-25T00:00:50.171073+0000","last_became_active":"2024-04-25T00:00:49.169029+0000","last_became_peered":"2024-04-25T00:00:49.169029+0000","last_unstale":"2024-04-25T00:00:50.171073+0000","last_undegraded":"2024-04-25T00:00:50.171073+0000","last_fullsized":"2024-04-25T00:00:50.171073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:06.579218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.174441+0000","last_change":"2024-04-25T00:00:50.170324+0000","last_active":"2024-04-25T00:00:50.174441+0000","last_peered":"2024-04-25T00:00:50.174441+0000","last_clean":"2024-04-25T00:00:50.174441+0000","last_became_active":"2024-04-25T00:00:50.170048+0000","last_became_peered":"2024-04-25T00:00:50.170048+0000","last_unstale":"2024-04-25T00:00:50.174441+0000","last_undegraded":"2024-04-25T00:00:50.174441+0000","last_fullsized":"2024-04-25T00:00:50.174441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.081284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.166261+0000","last_change":"2024-04-25T00:00:49.168252+0000","last_active":"2024-04-25T00:00:50.166261+0000","last_peered":"2024-04-25T00:00:50.166261+0000","last_clean":"2024-04-25T00:00:50.166261+0000","last_became_active":"2024-04-25T00:00:49.167986+0000","last_became_peered":"2024-04-25T00:00:49.167986+0000","last_unstale":"2024-04-25T00:00:50.166261+0000","last_undegraded":"2024-04-25T00:00:50.166261+0000","last_fullsized":"2024-04-25T00:00:50.166261+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:49.928618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.187940+0000","last_change":"2024-04-25T00:00:50.183641+0000","last_active":"2024-04-25T00:00:50.187940+0000","last_peered":"2024-04-25T00:00:50.187940+0000","last_clean":"2024-04-25T00:00:50.187940+0000","last_became_active":"2024-04-25T00:00:50.183181+0000","last_became_peered":"2024-04-25T00:00:50.183181+0000","last_unstale":"2024-04-25T00:00:50.187940+0000","last_undegraded":"2024-04-25T00:00:50.187940+0000","last_fullsized":"2024-04-25T00:00:50.187940+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:49.151173+0000","last_clean_scrub_stamp":"2024-04-25T00:00:49.151173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:42.049285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.180331+0000","last_change":"2024-04-25T00:00:49.173098+0000","last_active":"2024-04-25T00:00:50.180331+0000","last_peered":"2024-04-25T00:00:50.180331+0000","last_clean":"2024-04-25T00:00:50.180331+0000","last_became_active":"2024-04-25T00:00:49.172819+0000","last_became_peered":"2024-04-25T00:00:49.172819+0000","last_unstale":"2024-04-25T00:00:50.180331+0000","last_undegraded":"2024-04-25T00:00:50.180331+0000","last_fullsized":"2024-04-25T00:00:50.180331+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:48.145487+0000","last_clean_scrub_stamp":"2024-04-25T00:00:48.145487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:48.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T00:00:50.169284+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:00:50.169284+0000","last_peered":"2024-04-25T00:00:50.169284+0000","last_clean":"2024-04-25T00:00:50.169284+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:00:50.169284+0000","last_undegraded":"2024-04-25T00:00:50.169284+0000","last_fullsized":"2024-04-25T00:00:50.169284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40604,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331236,"statfs":{"total":96636764160,"available":96595185664,"internally_reserved":0,"allocated":1126400,"data_stored":805597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":536576,"data_stored":215811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40024,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331816,"statfs":{"total":96636764160,"available":96595779584,"internally_reserved":0,"allocated":532480,"data_stored":215209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44192,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327648,"statfs":{"total":96636764160,"available":96591511552,"internally_reserved":0,"allocated":540672,"data_stored":216751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":540672,"data_stored":215777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":216773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":532480,"data_stored":215209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:53.466 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:00:53.466 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:00:53.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.605+0000 7fba3576a640 1 Processor -- start 2024-04-25T00:00:53.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.605+0000 7fba3576a640 1 -- start start 2024-04-25T00:00:53.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba3576a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba300cf3c0 0x7fba300d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba3576a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba30161fc0 0x7fba300d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba3576a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba30162560 con 0x7fba300cf7a0 2024-04-25T00:00:53.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba3576a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fba30162840 con 0x7fba300cf3c0 2024-04-25T00:00:53.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba3576a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fba30162b20 con 0x7fba30161fc0 2024-04-25T00:00:53.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba300cf3c0 0x7fba300d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba300cf3c0 0x7fba300d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54534/0 (socket says 172.21.15.113:54534) 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2effd640 1 -- 172.21.15.113:0/1376097389 learned_addr learned my addr 172.21.15.113:0/1376097389 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2f7fe640 1 --2- 172.21.15.113:0/1376097389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba30161fc0 0x7fba300d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2effd640 1 -- 172.21.15.113:0/1376097389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba30161fc0 msgr2=0x7fba300d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2effd640 1 --2- 172.21.15.113:0/1376097389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba30161fc0 0x7fba300d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2effd640 1 -- 172.21.15.113:0/1376097389 >> v1:172.21.15.178:6789/0 conn(0x7fba300cf7a0 legacy=0x7fba300d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.606+0000 7fba2effd640 1 -- 172.21.15.113:0/1376097389 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba300d40f0 con 0x7fba300cf3c0 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.607+0000 7fba2effd640 1 --2- 172.21.15.113:0/1376097389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba300d3bb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fba20009e50 tx=0x7fba2002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73814cc28dd71be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.607+0000 7fba2dffb640 1 -- 172.21.15.113:0/1376097389 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20036070 con 0x7fba300cf3c0 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.607+0000 7fba2dffb640 1 -- 172.21.15.113:0/1376097389 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba20031600 con 0x7fba300cf3c0 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.607+0000 7fba2dffb640 1 -- 172.21.15.113:0/1376097389 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba200319d0 con 0x7fba300cf3c0 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 -- 172.21.15.113:0/1376097389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 msgr2=0x7fba300d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 --2- 172.21.15.113:0/1376097389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba300d3bb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fba20009e50 tx=0x7fba2002a710 comp rx=0 tx=0).stop 2024-04-25T00:00:53.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 -- 172.21.15.113:0/1376097389 shutdown_connections 2024-04-25T00:00:53.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 --2- 172.21.15.113:0/1376097389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba30161fc0 0x7fba300d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 --2- 172.21.15.113:0/1376097389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba300d3bb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 -- 172.21.15.113:0/1376097389 >> 172.21.15.113:0/1376097389 conn(0x7fba300718f0 msgr2=0x7fba30071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 -- 172.21.15.113:0/1376097389 shutdown_connections 2024-04-25T00:00:53.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.608+0000 7fba3576a640 1 -- 172.21.15.113:0/1376097389 wait complete. 2024-04-25T00:00:53.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.609+0000 7fba3576a640 1 Processor -- start 2024-04-25T00:00:53.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.609+0000 7fba3576a640 1 -- start start 2024-04-25T00:00:53.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba3576a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba3014b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba3576a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba300cf7a0 0x7fba301f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba3576a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30161fc0 0x7fba301f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba3576a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba300d1750 con 0x7fba300cf3c0 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba3576a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba300d2bf0 con 0x7fba30161fc0 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba3576a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fba300d18d0 con 0x7fba300cf7a0 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba3014b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba3014b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54550/0 (socket says 172.21.15.113:54550) 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 -- 172.21.15.113:0/3024976524 learned_addr learned my addr 172.21.15.113:0/3024976524 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2f7fe640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30161fc0 0x7fba301f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2e7fc640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba300cf7a0 0x7fba301f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 -- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba300cf7a0 msgr2=0x7fba301f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba300cf7a0 0x7fba301f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 -- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30161fc0 msgr2=0x7fba301f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30161fc0 0x7fba301f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 -- 172.21.15.113:0/3024976524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba301502b0 con 0x7fba300cf3c0 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2f7fe640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30161fc0 0x7fba301f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2e7fc640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba300cf7a0 0x7fba301f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba2effd640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba3014b780 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fba20034040 tx=0x7fba20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba177fe640 1 -- 172.21.15.113:0/3024976524 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2003f020 con 0x7fba300cf3c0 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba177fe640 1 -- 172.21.15.113:0/3024976524 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba20037040 con 0x7fba300cf3c0 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.610+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba30151f60 con 0x7fba300cf3c0 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.611+0000 7fba177fe640 1 -- 172.21.15.113:0/3024976524 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20036070 con 0x7fba300cf3c0 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.611+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba30152520 con 0x7fba300cf3c0 2024-04-25T00:00:53.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.612+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba300691a0 con 0x7fba300cf3c0 2024-04-25T00:00:53.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.612+0000 7fba177fe640 1 -- 172.21.15.113:0/3024976524 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba20042020 con 0x7fba300cf3c0 2024-04-25T00:00:53.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.612+0000 7fba177fe640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fba1c06be30 0x7fba1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.612+0000 7fba177fe640 1 -- 172.21.15.113:0/3024976524 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fba200ab300 con 0x7fba300cf3c0 2024-04-25T00:00:53.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.612+0000 7fba2e7fc640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fba1c06be30 0x7fba1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.613+0000 7fba2e7fc640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fba1c06be30 0x7fba1c06e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba3014ea40 tx=0x7fba24009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.616+0000 7fba177fe640 1 -- 172.21.15.113:0/3024976524 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba20049030 con 0x7fba300cf3c0 2024-04-25T00:00:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.859+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fba3015ac20 con 0x7fba300cf3c0 2024-04-25T00:00:53.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.861+0000 7fba177fe640 1 -- 172.21.15.113:0/3024976524 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fba20079d40 con 0x7fba300cf3c0 2024-04-25T00:00:53.862 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:00:53.862 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:00:53.862 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:00:53.862 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:00:53.862 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:00:53.863 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fba1c06be30 msgr2=0x7fba1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fba1c06be30 0x7fba1c06e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba3014ea40 tx=0x7fba24009040 comp rx=0 tx=0).stop 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 msgr2=0x7fba3014b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba3014b780 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fba20034040 tx=0x7fba20004a40 comp rx=0 tx=0).stop 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 shutdown_connections 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fba1c06be30 0x7fba1c06e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30161fc0 0x7fba301f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba300cf7a0 0x7fba301f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 --2- 172.21.15.113:0/3024976524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba300cf3c0 0x7fba3014b780 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 >> 172.21.15.113:0/3024976524 conn(0x7fba300718f0 msgr2=0x7fba30071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 shutdown_connections 2024-04-25T00:00:53.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.863+0000 7fba3576a640 1 -- 172.21.15.113:0/3024976524 wait complete. 2024-04-25T00:00:53.875 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:53.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.970+0000 7fd00b1ab640 1 Processor -- start 2024-04-25T00:00:53.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.970+0000 7fd00b1ab640 1 -- start start 2024-04-25T00:00:53.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd00b1ab640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd00415e320 0x7fd0040d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd00b1ab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0040d2290 0x7fd0040b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd00b1ab640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0040d3de0 con 0x7fd0040d0b00 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd00b1ab640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd0040d3410 con 0x7fd00415e320 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd00b1ab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd0040d2800 con 0x7fd0040d2290 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd008f47640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd00415e320 0x7fd0040d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd008f47640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd00415e320 0x7fd0040d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54558/0 (socket says 172.21.15.113:54558) 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.971+0000 7fd008f47640 1 -- 172.21.15.113:0/2011806009 learned_addr learned my addr 172.21.15.113:0/2011806009 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd008f47640 1 -- 172.21.15.113:0/2011806009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0040d2290 msgr2=0x7fd0040b9aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd008f47640 1 --2- 172.21.15.113:0/2011806009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0040d2290 0x7fd0040b9aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd008f47640 1 -- 172.21.15.113:0/2011806009 >> v1:172.21.15.178:6789/0 conn(0x7fd0040d0b00 legacy=0x7fd0040b7240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd008f47640 1 -- 172.21.15.113:0/2011806009 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0040d2ae0 con 0x7fd00415e320 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd008f47640 1 --2- 172.21.15.113:0/2011806009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00415e320 0x7fd0040d38a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009870 tx=0x7fcff402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ffd97e1243c572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd0037fe640 1 -- 172.21.15.113:0/2011806009 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4037070 con 0x7fd00415e320 2024-04-25T00:00:53.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd0037fe640 1 -- 172.21.15.113:0/2011806009 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff4032440 con 0x7fd00415e320 2024-04-25T00:00:53.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.972+0000 7fd0037fe640 1 -- 172.21.15.113:0/2011806009 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4032810 con 0x7fd00415e320 2024-04-25T00:00:53.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 -- 172.21.15.113:0/2011806009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00415e320 msgr2=0x7fd0040d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:53.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/2011806009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00415e320 0x7fd0040d38a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009870 tx=0x7fcff402a2c0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 -- 172.21.15.113:0/2011806009 shutdown_connections 2024-04-25T00:00:53.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/2011806009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0040d2290 0x7fd0040b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/2011806009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00415e320 0x7fd0040d38a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 -- 172.21.15.113:0/2011806009 >> 172.21.15.113:0/2011806009 conn(0x7fd0040b5c20 msgr2=0x7fd0040b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 -- 172.21.15.113:0/2011806009 shutdown_connections 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.973+0000 7fd00b1ab640 1 -- 172.21.15.113:0/2011806009 wait complete. 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.974+0000 7fd00b1ab640 1 Processor -- start 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.974+0000 7fd00b1ab640 1 -- start start 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd00b1ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0040d0b00 0x7fd00415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd00b1ab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0040d2290 0x7fd0040acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd00b1ab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd00415e320 0x7fd0041f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd00b1ab640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0040b9fe0 con 0x7fd0040d0b00 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd00b1ab640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0040d30a0 con 0x7fd0040d2290 2024-04-25T00:00:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd00b1ab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd00415e700 con 0x7fd00415e320 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0040d0b00 0x7fd00415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0040d0b00 0x7fd00415c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54560/0 (socket says 172.21.15.113:54560) 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 -- 172.21.15.113:0/194813239 learned_addr learned my addr 172.21.15.113:0/194813239 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 -- 172.21.15.113:0/194813239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd00415e320 msgr2=0x7fd0041f3fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd00415e320 0x7fd0041f3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 -- 172.21.15.113:0/194813239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0040d2290 msgr2=0x7fd0040acc50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0040d2290 0x7fd0040acc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.975+0000 7fd008f47640 1 -- 172.21.15.113:0/194813239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00415ec80 con 0x7fd0040d0b00 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.976+0000 7fd008f47640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0040d0b00 0x7fd00415c610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009870 tx=0x7fcff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.976+0000 7fd0017fa640 1 -- 172.21.15.113:0/194813239 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4041070 con 0x7fd0040d0b00 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.976+0000 7fd0017fa640 1 -- 172.21.15.113:0/194813239 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff403c870 con 0x7fd0040d0b00 2024-04-25T00:00:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.976+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00415ef60 con 0x7fd0040d0b00 2024-04-25T00:00:53.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.976+0000 7fd0017fa640 1 -- 172.21.15.113:0/194813239 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4037040 con 0x7fd0040d0b00 2024-04-25T00:00:53.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.976+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00415f520 con 0x7fd0040d0b00 2024-04-25T00:00:53.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.977+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0040691a0 con 0x7fd0040d0b00 2024-04-25T00:00:53.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.977+0000 7fd0017fa640 1 -- 172.21.15.113:0/194813239 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcff4044070 con 0x7fd0040d0b00 2024-04-25T00:00:53.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.978+0000 7fd0017fa640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcff006be60 0x7fcff006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:53.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.978+0000 7fd0017fa640 1 -- 172.21.15.113:0/194813239 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fcff40ab8b0 con 0x7fd0040d0b00 2024-04-25T00:00:53.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.978+0000 7fd003fff640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcff006be60 0x7fcff006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:53.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.978+0000 7fd003fff640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcff006be60 0x7fcff006e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd004161b00 tx=0x7fcff8006bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:53.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:53.982+0000 7fd0017fa640 1 -- 172.21.15.113:0/194813239 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcff407a2f0 con 0x7fd0040d0b00 2024-04-25T00:00:54.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.191+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd00415db40 con 0x7fd0040d0b00 2024-04-25T00:00:54.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.192+0000 7fd0017fa640 1 -- 172.21.15.113:0/194813239 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7fcff4079bf0 con 0x7fd0040d0b00 2024-04-25T00:00:54.192 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:54.192 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:00:54.193 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:00:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcff006be60 msgr2=0x7fcff006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcff006be60 0x7fcff006e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd004161b00 tx=0x7fcff8006bd0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0040d0b00 msgr2=0x7fd00415c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0040d0b00 0x7fd00415c610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009870 tx=0x7fcff40046c0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 shutdown_connections 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcff006be60 0x7fcff006e320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0040d2290 0x7fd0040acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd00415e320 0x7fd0041f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 --2- 172.21.15.113:0/194813239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0040d0b00 0x7fd00415c610 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 >> 172.21.15.113:0/194813239 conn(0x7fd0040b5c20 msgr2=0x7fd0040b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 shutdown_connections 2024-04-25T00:00:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.194+0000 7fd00b1ab640 1 -- 172.21.15.113:0/194813239 wait complete. 2024-04-25T00:00:54.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:00:49.512476+0000', 'modified': '2024-04-25T00:00:51.512287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5556': {'gid': 5556, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6836/1715251400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 1715251400}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 1715251400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5556, 'qdb_cluster': [5556]} 2024-04-25T00:00:54.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:00:54.608 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:00:54.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.707+0000 7f8f4aa3f640 1 Processor -- start 2024-04-25T00:00:54.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.707+0000 7f8f4aa3f640 1 -- start start 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f4aa3f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8f4415ba50 0x7f8f4415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f4aa3f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f44071170 con 0x7f8f4415b670 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f4aa3f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f44071450 con 0x7f8f4406cea0 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f4aa3f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8f44071730 con 0x7f8f4415ba50 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f43fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8f4415b670 0x7f8f440cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43840/0 (socket says 172.21.15.113:43840) 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f43fff640 1 -- 172.21.15.113:0/3819909049 learned_addr learned my addr 172.21.15.113:0/3819909049 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f437fe640 1 --2- 172.21.15.113:0/3819909049 >> v2:172.21.15.178:3300/0 conn(0x7f8f4415ba50 0x7f8f4415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005307001 0 0) 0x7f8f44071170 con 0x7f8f4415b670 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.708+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f2c003520 con 0x7f8f4415b670 2024-04-25T00:00:54.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 989470541 0 0) 0x7f8f2c003520 con 0x7f8f4415b670 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 >> v2:172.21.15.178:3300/0 conn(0x7f8f4415ba50 msgr2=0x7f8f4415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 --2- 172.21.15.113:0/3819909049 >> v2:172.21.15.178:3300/0 conn(0x7f8f4415ba50 0x7f8f4415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 >> v1:172.21.15.178:6789/0 conn(0x7f8f4406cea0 legacy=0x7f8f4406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f44071a10 con 0x7f8f4415b670 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8f34001670 con 0x7f8f4415b670 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 >> v1:172.21.15.113:6789/0 conn(0x7f8f4415b670 legacy=0x7f8f440cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2c0039b0 0x7f8f2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2c004800 0x7f8f2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f44071730 con 0x7f8f2c0052e0 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c00c1b0 con 0x7f8f2c004800 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2c00c3b0 con 0x7f8f2c0039b0 2024-04-25T00:00:54.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f43fff640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2c0039b0 0x7f8f2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f437fe640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2c004800 0x7f8f2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f48fdc640 1 -- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2c0039b0 msgr2=0x7f8f2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2c0039b0 0x7f8f2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f48fdc640 1 -- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2c004800 msgr2=0x7f8f2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2c004800 0x7f8f2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.709+0000 7f8f48fdc640 1 -- 172.21.15.113:0/3819909049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c003520 con 0x7f8f2c0052e0 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f48fdc640 1 -- 172.21.15.113:0/3819909049 reap_dead start 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f2c00b9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c001e20 tx=0x7f8f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52db5ae433196fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c001a70 con 0x7f8f2c0052e0 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f3c02ae40 con 0x7f8f2c0052e0 2024-04-25T00:00:54.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f42ffd640 1 -- 172.21.15.113:0/3819909049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c003bb0 con 0x7f8f2c0052e0 2024-04-25T00:00:54.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/3819909049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 msgr2=0x7f8f2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f2c00b9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c001e20 tx=0x7f8f2c005830 comp rx=0 tx=0).stop 2024-04-25T00:00:54.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/3819909049 shutdown_connections 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.710+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/3819909049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f2c00b9c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.711+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/3819909049 >> 172.21.15.113:0/3819909049 conn(0x7f8f4406b0f0 msgr2=0x7f8f4406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.711+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/3819909049 shutdown_connections 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.711+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/3819909049 wait complete. 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.712+0000 7f8f4aa3f640 1 Processor -- start 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.712+0000 7f8f4aa3f640 1 -- start start 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f441f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f441e2980 0x7f8f441e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f441f57c0 0x7f8f4413d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4415c380 con 0x7f8f2c0052e0 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4415c200 con 0x7f8f441f57c0 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4406d2a0 con 0x7f8f441e2980 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f441f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f441f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54572/0 (socket says 172.21.15.113:54572) 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 -- 172.21.15.113:0/194043294 learned_addr learned my addr 172.21.15.113:0/194043294 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f437fe640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f441e2980 0x7f8f441e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f441f57c0 0x7f8f4413d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 -- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f441e2980 msgr2=0x7f8f441e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f441e2980 0x7f8f441e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 -- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f441f57c0 msgr2=0x7f8f4413d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f441f57c0 0x7f8f4413d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 -- 172.21.15.113:0/194043294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f441e5a80 con 0x7f8f2c0052e0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f43fff640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f441f60f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8f340072a0 tx=0x7f8f340076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f40ff9640 1 -- 172.21.15.113:0/194043294 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400c040 con 0x7f8f2c0052e0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f40ff9640 1 -- 172.21.15.113:0/194043294 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f34017070 con 0x7f8f2c0052e0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f441e5d20 con 0x7f8f2c0052e0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f40ff9640 1 -- 172.21.15.113:0/194043294 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34006570 con 0x7f8f2c0052e0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.713+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f441e62e0 con 0x7f8f2c0052e0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.715+0000 7f8f40ff9640 1 -- 172.21.15.113:0/194043294 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f340047e0 con 0x7f8f2c0052e0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.715+0000 7f8f40ff9640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f3006be30 0x7f8f3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.715+0000 7f8f437fe640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f3006be30 0x7f8f3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.715+0000 7f8f40ff9640 1 -- 172.21.15.113:0/194043294 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8f3408acf0 con 0x7f8f2c0052e0 2024-04-25T00:00:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.716+0000 7f8f437fe640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f3006be30 0x7f8f3006e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f44150da0 tx=0x7f8f38009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.716+0000 7f8f4d82f740 1 Processor -- start 2024-04-25T00:00:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.716+0000 7f8f4d82f740 1 -- start start 2024-04-25T00:00:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbb9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9540 0x55f2edb8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f2edaf1f00 con 0x55f2edbb9c80 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2eda9a750 con 0x55f2edbb9540 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f2edaedec0 con 0x55f2edbd6b60 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbb9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbb9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54574/0 (socket says 172.21.15.113:54574) 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 -- 172.21.15.113:0/985938821 learned_addr learned my addr 172.21.15.113:0/985938821 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f43fff640 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 -- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 msgr2=0x55f2edb894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9540 0x55f2edb8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb894f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 -- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9540 msgr2=0x55f2edb8e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9540 0x55f2edb8e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 -- 172.21.15.113:0/985938821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ed8062d0 con 0x55f2edbb9c80 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9540 0x55f2edb8e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.717+0000 7f8f437fe640 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbb9000 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8f38014f50 tx=0x7f8f38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ba8355072510d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f29ffb640 1 -- 172.21.15.113:0/985938821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3801b070 con 0x55f2edbb9c80 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f29ffb640 1 -- 172.21.15.113:0/985938821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f38015e30 con 0x55f2edbb9c80 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f29ffb640 1 -- 172.21.15.113:0/985938821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3802a630 con 0x55f2edbb9c80 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 -- 172.21.15.113:0/985938821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 msgr2=0x55f2edbb9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbb9000 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8f38014f50 tx=0x7f8f38017000 comp rx=0 tx=0).stop 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 -- 172.21.15.113:0/985938821 shutdown_connections 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9540 0x55f2edb8e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbb9000 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/985938821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 -- 172.21.15.113:0/985938821 >> 172.21.15.113:0/985938821 conn(0x55f2edb8b760 msgr2=0x55f2edbd4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 -- 172.21.15.113:0/985938821 shutdown_connections 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.718+0000 7f8f4d82f740 1 -- 172.21.15.113:0/985938821 wait complete. 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.719+0000 7f8f4d82f740 1 -- start start 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.719+0000 7f8f4d82f740 1 Processor -- start 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb8ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb8f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f2edb8d910 con 0x55f2edbe7510 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2edaadcf0 con 0x55f2edbb9c80 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f2edb8cd00 con 0x55f2edbd6b60 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 10 client.0 _ll_register_callbacks cb 0x55f2edaad6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb8f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb8f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54582/0 (socket says 172.21.15.113:54582) 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 -- 172.21.15.113:0/694237454 learned_addr learned my addr 172.21.15.113:0/694237454 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f43fff640 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f437fe640 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb8ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 -- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 msgr2=0x55f2edb8ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb8ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 -- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 msgr2=0x55f2edbbc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 -- 172.21.15.113:0/694237454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ed8d06b0 con 0x55f2edbe7510 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb8f2a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c001f40 tx=0x7f8f3c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45495a28905ae91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f22ffd640 1 -- 172.21.15.113:0/694237454 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c042060 con 0x55f2edbe7510 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f22ffd640 1 -- 172.21.15.113:0/694237454 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8f3c006d50 con 0x55f2edbe7510 2024-04-25T00:00:54.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.720+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f38015e30 con 0x55f2edbe7510 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f22ffd640 1 -- 172.21.15.113:0/694237454 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f3c048050 con 0x55f2edbe7510 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 10 client.5697 fetch_fsmap learned FSMap version 21 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f2edbcacc0 con 0x55f2edbe7510 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f22ffd640 1 -- 172.21.15.113:0/694237454 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f38015e30 con 0x55f2edbe7510 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f22ffd640 1 -- 172.21.15.113:0/694237454 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f8f3c04b020 con 0x55f2edbe7510 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 10 client.5697 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 10 client.5697 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 20 client.5697 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbba550 0x55f2edbba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 4 client.5697 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2edaff440 con 0x55f2edbba550 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f43fff640 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbba550 0x55f2edbba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f22ffd640 10 client.5697 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.721+0000 7f8f43fff640 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbba550 0x55f2edbba930 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93bf0b6178477fc server_cookie=e5621b318249ee3c in_seq=0 out_seq=0 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f22ffd640 1 -- 172.21.15.113:0/694237454 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2edaff440 con 0x55f2edbba550 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f22ffd640 10 client.5697 handle_command_reply: tid=0 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 client.5697 shutdown 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 msgr2=0x55f2edb8f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb8f2a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c001f40 tx=0x7f8f3c001510 comp rx=0 tx=0).stop 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 shutdown_connections 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbba550 0x55f2edbba930 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb8f2a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edb8ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f22ffd640 0 client.5697 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/694237454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 >> 172.21.15.113:0/694237454 conn(0x55f2edbb9540 msgr2=0x55f2edbd5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 shutdown_connections 2024-04-25T00:00:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 1 -- 172.21.15.113:0/694237454 wait complete. 2024-04-25T00:00:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.722+0000 7f8f4d82f740 20 client.5697 trim_cache size 0 max 16384 2024-04-25T00:00:54.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.740+0000 7f8f4d82f740 1 Processor -- start 2024-04-25T00:00:54.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.740+0000 7f8f4d82f740 1 -- start start 2024-04-25T00:00:54.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbd6b60 0x55f2edb8efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbe7510 0x55f2edb88250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f441fdc40 con 0x55f2edbb9c80 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4414fe50 con 0x55f2edbd6b60 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8f441e7990 con 0x55f2edbe7510 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54594/0 (socket says 172.21.15.113:54594) 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 -- 172.21.15.113:0/36285153 learned_addr learned my addr 172.21.15.113:0/36285153 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 -- 172.21.15.113:0/36285153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbe7510 msgr2=0x55f2edb88250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 --2- 172.21.15.113:0/36285153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbe7510 0x55f2edb88250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 -- 172.21.15.113:0/36285153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbd6b60 msgr2=0x55f2edb8efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 --2- 172.21.15.113:0/36285153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbd6b60 0x55f2edb8efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 -- 172.21.15.113:0/36285153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2edbba330 con 0x55f2edbb9c80 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f43fff640 1 --2- 172.21.15.113:0/36285153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc480 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8f34055530 tx=0x7f8f34014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fe8eb2729e679d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f297fa640 1 -- 172.21.15.113:0/36285153 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3405d480 con 0x55f2edbb9c80 2024-04-25T00:00:54.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f297fa640 1 -- 172.21.15.113:0/36285153 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f34061cb0 con 0x55f2edbb9c80 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.741+0000 7f8f297fa640 1 -- 172.21.15.113:0/36285153 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f340573d0 con 0x55f2edbb9c80 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 -- 172.21.15.113:0/36285153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 msgr2=0x55f2edbbc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/36285153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc480 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8f34055530 tx=0x7f8f34014040 comp rx=0 tx=0).stop 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 -- 172.21.15.113:0/36285153 shutdown_connections 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/36285153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbd6b60 0x55f2edb8efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/36285153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbe7510 0x55f2edb88250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/36285153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbb9c80 0x55f2edbbc480 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 -- 172.21.15.113:0/36285153 >> 172.21.15.113:0/36285153 conn(0x55f2edbb9540 msgr2=0x55f2edbb4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 -- 172.21.15.113:0/36285153 shutdown_connections 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.742+0000 7f8f4d82f740 1 -- 172.21.15.113:0/36285153 wait complete. 2024-04-25T00:00:54.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.743+0000 7f8f4d82f740 1 -- start start 2024-04-25T00:00:54.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.743+0000 7f8f4d82f740 1 Processor -- start 2024-04-25T00:00:54.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.743+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edc4ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.743+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edbd3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.743+0000 7f8f4d82f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb88250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f2edbe4500 con 0x55f2edbe7510 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2edbe4740 con 0x55f2edbb9c80 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f4d82f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f2edbcc7e0 con 0x55f2edbd6b60 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f4d82f740 10 client.0 _ll_register_callbacks cb 0x7f8f38009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb88250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb88250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54606/0 (socket says 172.21.15.113:54606) 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 -- 172.21.15.113:0/311356292 learned_addr learned my addr 172.21.15.113:0/311356292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f43fff640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edc4ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f437fe640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edbd3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 -- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 msgr2=0x55f2edbd3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edbd3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 -- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 msgr2=0x55f2edc4ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edc4ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 -- 172.21.15.113:0/311356292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2edbe34b0 con 0x55f2edbe7510 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f437fe640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edbd3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f48fdc640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb88250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c001220 tx=0x7f8f3c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8046716d2069dfa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f22ffd640 1 -- 172.21.15.113:0/311356292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c032850 con 0x55f2edbe7510 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f22ffd640 1 -- 172.21.15.113:0/311356292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8f3c0329f0 con 0x55f2edbe7510 2024-04-25T00:00:54.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f34061cb0 con 0x55f2edbe7510 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f22ffd640 1 -- 172.21.15.113:0/311356292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f3c02a980 con 0x55f2edbe7510 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f4d82f740 10 client.5703 fetch_fsmap learned FSMap version 21 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f2edbb5e90 con 0x55f2edbe7510 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f22ffd640 1 -- 172.21.15.113:0/311356292 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f34061cb0 con 0x55f2edbe7510 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.744+0000 7f8f22ffd640 1 -- 172.21.15.113:0/311356292 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f8f3c045070 con 0x55f2edbe7510 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 10 client.5703 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 10 client.5703 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 20 client.5703 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbd5130 0x55f2edb88790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 4 client.5703 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "status"}) -- 0x55f2edaf9790 con 0x55f2edbd5130 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f43fff640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbd5130 0x55f2edb88790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f22ffd640 10 client.5703 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f43fff640 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbd5130 0x55f2edb88790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9561d2523ec976 server_cookie=690259ca8ea247fe in_seq=0 out_seq=0 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f22ffd640 1 -- 172.21.15.113:0/311356292 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f2edaf9790 con 0x55f2edbd5130 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f22ffd640 10 client.5703 handle_command_reply: tid=0 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 client.5703 shutdown 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 msgr2=0x55f2edb88250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb88250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c001220 tx=0x7f8f3c0014a0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 shutdown_connections 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f2edbd5130 0x55f2edb88790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f2edbe7510 0x55f2edb88250 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f2edbd6b60 0x55f2edbd3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f22ffd640 0 client.5703 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 --2- 172.21.15.113:0/311356292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2edbb9c80 0x55f2edc4ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 >> 172.21.15.113:0/311356292 conn(0x55f2edbb9540 msgr2=0x55f2edbd6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 shutdown_connections 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.745+0000 7f8f4d82f740 1 -- 172.21.15.113:0/311356292 wait complete. 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4d82f740 20 client.5703 trim_cache size 0 max 16384 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 5556, 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:00:54.753 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.5888673799999999, 2024-04-25T00:00:54.754 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 21, 2024-04-25T00:00:54.754 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 46, 2024-04-25T00:00:54.754 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 46, 2024-04-25T00:00:54.754 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 9.6587861119999996 2024-04-25T00:00:54.754 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f3006be30 msgr2=0x7f8f3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f3006be30 0x7f8f3006e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f44150da0 tx=0x7f8f38009190 comp rx=0 tx=0).stop 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 msgr2=0x7f8f441f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f441f60f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8f340072a0 tx=0x7f8f340076a0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 shutdown_connections 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f3006be30 0x7f8f3006e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f441f57c0 0x7f8f4413d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f441e2980 0x7f8f441e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 --2- 172.21.15.113:0/194043294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f2c0052e0 0x7f8f441f60f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 >> 172.21.15.113:0/194043294 conn(0x7f8f4406b0f0 msgr2=0x7f8f440cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 shutdown_connections 2024-04-25T00:00:54.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.746+0000 7f8f4aa3f640 1 -- 172.21.15.113:0/194043294 wait complete. 2024-04-25T00:00:54.757 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:00:54.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.853+0000 7fcb26c4c640 1 Processor -- start 2024-04-25T00:00:54.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.853+0000 7fcb26c4c640 1 -- start start 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb26c4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb200b3fd0 0x7fcb2006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb26c4c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb2006e510 0x7fcb200cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb26c4c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb20144cf0 con 0x7fcb20144910 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb26c4c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcb200b2f30 con 0x7fcb200b3fd0 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb26c4c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcb200b3210 con 0x7fcb2006e510 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb200b3fd0 0x7fcb2006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb200b3fd0 0x7fcb2006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54622/0 (socket says 172.21.15.113:54622) 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1ffff640 1 -- 172.21.15.113:0/989093930 learned_addr learned my addr 172.21.15.113:0/989093930 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb249e8640 1 --1- 172.21.15.113:0/989093930 >> v1:172.21.15.113:6789/0 conn(0x7fcb20144910 0x7fcb200b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43850/0 (socket says 172.21.15.113:43850) 2024-04-25T00:00:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb251e9640 1 --2- 172.21.15.113:0/989093930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb2006e510 0x7fcb200cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1ffff640 1 -- 172.21.15.113:0/989093930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb2006e510 msgr2=0x7fcb200cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1ffff640 1 --2- 172.21.15.113:0/989093930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb2006e510 0x7fcb200cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1ffff640 1 -- 172.21.15.113:0/989093930 >> v1:172.21.15.113:6789/0 conn(0x7fcb20144910 legacy=0x7fcb200b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1ffff640 1 -- 172.21.15.113:0/989093930 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb200b34f0 con 0x7fcb200b3fd0 2024-04-25T00:00:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.854+0000 7fcb1f7fe640 1 -- 172.21.15.113:0/989093930 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447117992 0 0) 0x7fcb20144cf0 con 0x7fcb20144910 2024-04-25T00:00:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.855+0000 7fcb1ffff640 1 --2- 172.21.15.113:0/989093930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb200b3fd0 0x7fcb2006dfd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0098a0 tx=0x7fcb0c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595533386bdaec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.855+0000 7fcb1f7fe640 1 -- 172.21.15.113:0/989093930 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c037070 con 0x7fcb200b3fd0 2024-04-25T00:00:54.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.855+0000 7fcb1f7fe640 1 -- 172.21.15.113:0/989093930 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0c032440 con 0x7fcb200b3fd0 2024-04-25T00:00:54.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.855+0000 7fcb1f7fe640 1 -- 172.21.15.113:0/989093930 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c032810 con 0x7fcb200b3fd0 2024-04-25T00:00:54.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.855+0000 7fcb26c4c640 1 -- 172.21.15.113:0/989093930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb200b3fd0 msgr2=0x7fcb2006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.855+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/989093930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb200b3fd0 0x7fcb2006dfd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0098a0 tx=0x7fcb0c02a290 comp rx=0 tx=0).stop 2024-04-25T00:00:54.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.856+0000 7fcb26c4c640 1 -- 172.21.15.113:0/989093930 shutdown_connections 2024-04-25T00:00:54.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.856+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/989093930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb2006e510 0x7fcb200cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.856+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/989093930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb200b3fd0 0x7fcb2006dfd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.856+0000 7fcb26c4c640 1 -- 172.21.15.113:0/989093930 >> 172.21.15.113:0/989093930 conn(0x7fcb200b1a50 msgr2=0x7fcb200b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.856+0000 7fcb26c4c640 1 -- 172.21.15.113:0/989093930 shutdown_connections 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.856+0000 7fcb26c4c640 1 -- 172.21.15.113:0/989093930 wait complete. 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.857+0000 7fcb26c4c640 1 Processor -- start 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.857+0000 7fcb26c4c640 1 -- start start 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.857+0000 7fcb26c4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2006e510 0x7fcb2013c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.857+0000 7fcb26c4c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb200b3fd0 0x7fcb2015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb26c4c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb20144910 0x7fcb201e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb26c4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcb200b4da0 con 0x7fcb2006e510 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb26c4c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb200b4440 con 0x7fcb20144910 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb26c4c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcb200b4f20 con 0x7fcb200b3fd0 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2006e510 0x7fcb2013c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2006e510 0x7fcb2013c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54628/0 (socket says 172.21.15.113:54628) 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 -- 172.21.15.113:0/1843300633 learned_addr learned my addr 172.21.15.113:0/1843300633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb1ffff640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb200b3fd0 0x7fcb2015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb251e9640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb20144910 0x7fcb201e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 -- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb200b3fd0 msgr2=0x7fcb2015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb200b3fd0 0x7fcb2015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 -- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb20144910 msgr2=0x7fcb201e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb20144910 0x7fcb201e6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 -- 172.21.15.113:0/1843300633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb201f5d00 con 0x7fcb2006e510 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb249e8640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2006e510 0x7fcb2013c3d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c4d0 tx=0x7fcb1400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb1d7fa640 1 -- 172.21.15.113:0/1843300633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018070 con 0x7fcb2006e510 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb1d7fa640 1 -- 172.21.15.113:0/1843300633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb14007500 con 0x7fcb2006e510 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb1d7fa640 1 -- 172.21.15.113:0/1843300633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14004a20 con 0x7fcb2006e510 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb201f5fe0 con 0x7fcb2006e510 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.858+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb201f65a0 con 0x7fcb2006e510 2024-04-25T00:00:54.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.859+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb200691a0 con 0x7fcb2006e510 2024-04-25T00:00:54.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.860+0000 7fcb1d7fa640 1 -- 172.21.15.113:0/1843300633 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb1401b070 con 0x7fcb2006e510 2024-04-25T00:00:54.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.860+0000 7fcb1d7fa640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb0406be30 0x7fcb0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.860+0000 7fcb1d7fa640 1 -- 172.21.15.113:0/1843300633 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fcb1408a130 con 0x7fcb2006e510 2024-04-25T00:00:54.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.860+0000 7fcb1ffff640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb0406be30 0x7fcb0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.860+0000 7fcb1ffff640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb0406be30 0x7fcb0406e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c034000 tx=0x7fcb0c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:54.864+0000 7fcb1d7fa640 1 -- 172.21.15.113:0/1843300633 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb14058490 con 0x7fcb2006e510 2024-04-25T00:00:55.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.072+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcb20072830 con 0x7fcb2006e510 2024-04-25T00:00:55.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.072+0000 7fcb1d7fa640 1 -- 172.21.15.113:0/1843300633 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7fcb1405c790 con 0x7fcb2006e510 2024-04-25T00:00:55.073 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:00:55.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.074+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb0406be30 msgr2=0x7fcb0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.074+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb0406be30 0x7fcb0406e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c034000 tx=0x7fcb0c02e000 comp rx=0 tx=0).stop 2024-04-25T00:00:55.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.074+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2006e510 msgr2=0x7fcb2013c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:55.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.074+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2006e510 0x7fcb2013c3d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c4d0 tx=0x7fcb1400c990 comp rx=0 tx=0).stop 2024-04-25T00:00:55.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 shutdown_connections 2024-04-25T00:00:55.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb0406be30 0x7fcb0406e2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb20144910 0x7fcb201e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb200b3fd0 0x7fcb2015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 --2- 172.21.15.113:0/1843300633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2006e510 0x7fcb2013c3d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:55.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 >> 172.21.15.113:0/1843300633 conn(0x7fcb200b1a50 msgr2=0x7fcb200cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:55.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 shutdown_connections 2024-04-25T00:00:55.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55.075+0000 7fcb26c4c640 1 -- 172.21.15.113:0/1843300633 wait complete. 2024-04-25T00:00:55.086 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:00:55.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:00:55.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:00:55.086 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:00:55.087 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:00:55.087 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:00:55.087 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:00:55.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:00:55.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:00:55.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.088 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:00:55.107 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42625sec preferred_lft 42625sec 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:00:55.108 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:00:55.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.110 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:00:55.169 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:00:55.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.171 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:00:55.226 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:00:55.226 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:00:55.226 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:00:55.227 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:00:55.227 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:00:55.227 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:00:55.227 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:00:55.227 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:00:55.227 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:00:55.227 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:00:55.227 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:00:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:00:55.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:00:55.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.350 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:00:55.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:00:55.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:00:55.365 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:00:55.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.365 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:00:55.427 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:00:55.427+0000 7fb0a2c8f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:00:55.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:00:55.428 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:00:55.482 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:00:55.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:00:55.483 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:00:55.537 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:00:55.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.537 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:00:55.617 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:00:55.617 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:00:55.618 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:00:55.626 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:00:55.626 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:00:55.626 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:00:55.626 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:00:55.626 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:00:55.640 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:00:55.640 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:00:55.640 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:00:55.640 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:00:55.640 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:00:55.641 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:00:55.641 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:00:55.641 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:00:55.641 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:00:55.641 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:00:55.641 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:00:55.641 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:00:55.642 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:00:55.643 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:00:55.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:00:55.708 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:00:55.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:00:55.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:00:55.807 INFO:teuthology.orchestra.run.smithi113.stdout:client5715 2024-04-25T00:00:55.807 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:00:55.807 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:00:55.822 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:00:55.822 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T00:00:55.822 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T00:00:55.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:55.823 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T00:00:55.883 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:00:55.883 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:00:55.883 DEBUG:teuthology.orchestra.run.smithi113:> import stat 2024-04-25T00:00:55.883 DEBUG:teuthology.orchestra.run.smithi113:> import json 2024-04-25T00:00:55.883 DEBUG:teuthology.orchestra.run.smithi113:> import sys 2024-04-25T00:00:55.883 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:00:55.883 DEBUG:teuthology.orchestra.run.smithi113:> try: 2024-04-25T00:00:55.884 DEBUG:teuthology.orchestra.run.smithi113:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T00:00:55.884 DEBUG:teuthology.orchestra.run.smithi113:> except OSError as e: 2024-04-25T00:00:55.884 DEBUG:teuthology.orchestra.run.smithi113:> sys.exit(e.errno) 2024-04-25T00:00:55.884 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:00:55.884 DEBUG:teuthology.orchestra.run.smithi113:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T00:00:55.884 DEBUG:teuthology.orchestra.run.smithi113:> print(json.dumps( 2024-04-25T00:00:55.884 DEBUG:teuthology.orchestra.run.smithi113:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T00:00:55.885 DEBUG:teuthology.orchestra.run.smithi113:> indent=2)) 2024-04-25T00:00:55.885 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:00:55.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:55 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T00:00:56.000 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T00:00:56.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:00:56.001 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T00:00:56.057 INFO:teuthology.orchestra.run.smithi113.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T00:00:56.225 INFO:teuthology.orchestra.run.smithi113.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T00:00:59.160 INFO:teuthology.orchestra.run.smithi113.stderr:Updating files: 46% (651/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 93% (1306/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T00:00:59.165 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T00:00:59.165 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T00:00:59.166 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:59.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.303+0000 7fb919590640 1 Processor -- start 2024-04-25T00:00:59.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.303+0000 7fb919590640 1 -- start start 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb919590640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb914144910 0x7fb9140b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb919590640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9140b5240 0x7fb91406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb919590640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb914144cf0 con 0x7fb9140b56b0 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb919590640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb9140b4510 con 0x7fb914144910 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb919590640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb9140b2f30 con 0x7fb9140b5240 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb912ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb914144910 0x7fb9140b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb9137fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb9140b56b0 0x7fb9140cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33602/0 (socket says 172.21.15.113:33602) 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.304+0000 7fb9137fe640 1 -- 172.21.15.113:0/1773455302 learned_addr learned my addr 172.21.15.113:0/1773455302 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.305+0000 7fb9127fc640 1 --2- 172.21.15.113:0/1773455302 >> v2:172.21.15.178:3300/0 conn(0x7fb9140b5240 0x7fb91406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.305+0000 7fb912ffd640 1 -- 172.21.15.113:0/1773455302 >> v2:172.21.15.178:3300/0 conn(0x7fb9140b5240 msgr2=0x7fb91406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.305+0000 7fb912ffd640 1 --2- 172.21.15.113:0/1773455302 >> v2:172.21.15.178:3300/0 conn(0x7fb9140b5240 0x7fb91406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.305+0000 7fb912ffd640 1 -- 172.21.15.113:0/1773455302 >> v1:172.21.15.113:6789/0 conn(0x7fb9140b56b0 legacy=0x7fb9140cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.305+0000 7fb911ffb640 1 -- 172.21.15.113:0/1773455302 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053887492 0 0) 0x7fb914144cf0 con 0x7fb9140b56b0 2024-04-25T00:00:59.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.305+0000 7fb912ffd640 1 -- 172.21.15.113:0/1773455302 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9140b3210 con 0x7fb914144910 2024-04-25T00:00:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.305+0000 7fb912ffd640 1 --2- 172.21.15.113:0/1773455302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb914144910 0x7fb9140b3fd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb904009d90 tx=0x7fb90402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef43c8e05966b9f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb911ffb640 1 -- 172.21.15.113:0/1773455302 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904007740 con 0x7fb914144910 2024-04-25T00:00:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb911ffb640 1 -- 172.21.15.113:0/1773455302 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9040078e0 con 0x7fb914144910 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb911ffb640 1 -- 172.21.15.113:0/1773455302 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904007cb0 con 0x7fb914144910 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 -- 172.21.15.113:0/1773455302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb914144910 msgr2=0x7fb9140b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 --2- 172.21.15.113:0/1773455302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb914144910 0x7fb9140b3fd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb904009d90 tx=0x7fb90402a6a0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 -- 172.21.15.113:0/1773455302 shutdown_connections 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 --2- 172.21.15.113:0/1773455302 >> v2:172.21.15.178:3300/0 conn(0x7fb9140b5240 0x7fb91406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 --2- 172.21.15.113:0/1773455302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb914144910 0x7fb9140b3fd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 -- 172.21.15.113:0/1773455302 >> 172.21.15.113:0/1773455302 conn(0x7fb9140b1a50 msgr2=0x7fb9140b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 -- 172.21.15.113:0/1773455302 shutdown_connections 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.306+0000 7fb919590640 1 -- 172.21.15.113:0/1773455302 wait complete. 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.307+0000 7fb919590640 1 Processor -- start 2024-04-25T00:00:59.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb919590640 1 -- start start 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb919590640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9140b5240 0x7fb9141f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb919590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140b56b0 0x7fb9140ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb919590640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb914144910 0x7fb9140c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb919590640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9140b4da0 con 0x7fb9140b56b0 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb919590640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9140b37d0 con 0x7fb9140b5240 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb919590640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9140b4f20 con 0x7fb914144910 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb9127fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140b56b0 0x7fb9140ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb9127fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140b56b0 0x7fb9140ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38478/0 (socket says 172.21.15.113:38478) 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.308+0000 7fb9127fc640 1 -- 172.21.15.113:0/3338352264 learned_addr learned my addr 172.21.15.113:0/3338352264 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb912ffd640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9140b5240 0x7fb9141f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb9127fc640 1 -- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb914144910 msgr2=0x7fb9140c60d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb9127fc640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb914144910 0x7fb9140c60d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb9127fc640 1 -- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9140b5240 msgr2=0x7fb9141f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb9127fc640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9140b5240 0x7fb9141f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb9127fc640 1 -- 172.21.15.113:0/3338352264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9140c5030 con 0x7fb9140b56b0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb912ffd640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9140b5240 0x7fb9141f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb9127fc640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140b56b0 0x7fb9140ade50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d800 tx=0x7fb90800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb8fb7fe640 1 -- 172.21.15.113:0/3338352264 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90800b6c0 con 0x7fb9140b56b0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb8fb7fe640 1 -- 172.21.15.113:0/3338352264 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb908019070 con 0x7fb9140b56b0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9140c5310 con 0x7fb9140b56b0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb8fb7fe640 1 -- 172.21.15.113:0/3338352264 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90800ba80 con 0x7fb9140b56b0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.309+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9140c58d0 con 0x7fb9140b56b0 2024-04-25T00:00:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.310+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9140691a0 con 0x7fb9140b56b0 2024-04-25T00:00:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.310+0000 7fb8fb7fe640 1 -- 172.21.15.113:0/3338352264 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb90800bc20 con 0x7fb9140b56b0 2024-04-25T00:00:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.311+0000 7fb8fb7fe640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb90006be30 0x7fb90006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.311+0000 7fb8fb7fe640 1 -- 172.21.15.113:0/3338352264 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb90808b270 con 0x7fb9140b56b0 2024-04-25T00:00:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.311+0000 7fb912ffd640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb90006be30 0x7fb90006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.311+0000 7fb912ffd640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb90006be30 0x7fb90006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb904034000 tx=0x7fb904033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.315+0000 7fb8fb7fe640 1 -- 172.21.15.113:0/3338352264 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb908059cb0 con 0x7fb9140b56b0 2024-04-25T00:00:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.527+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb91406c100 con 0x7fb9140b56b0 2024-04-25T00:00:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.528+0000 7fb8fb7fe640 1 -- 172.21.15.113:0/3338352264 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7fb9080595b0 con 0x7fb9140b56b0 2024-04-25T00:00:59.528 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:00:59.528 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:00:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.529+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb90006be30 msgr2=0x7fb90006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.529+0000 7fb919590640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb90006be30 0x7fb90006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb904034000 tx=0x7fb904033040 comp rx=0 tx=0).stop 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.529+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140b56b0 msgr2=0x7fb9140ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.529+0000 7fb919590640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140b56b0 0x7fb9140ade50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d800 tx=0x7fb90800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 shutdown_connections 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb90006be30 0x7fb90006e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb914144910 0x7fb9140c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140b56b0 0x7fb9140ade50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 --2- 172.21.15.113:0/3338352264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9140b5240 0x7fb9141f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 >> 172.21.15.113:0/3338352264 conn(0x7fb9140b1a50 msgr2=0x7fb9140b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 shutdown_connections 2024-04-25T00:00:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.530+0000 7fb919590640 1 -- 172.21.15.113:0/3338352264 wait complete. 2024-04-25T00:00:59.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T00:00:59.644 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:59.644 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": -2 2024-04-25T00:00:59.644 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:59.652 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T00:00:59.653 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T00:00:59.653 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T00:00:59.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.751+0000 7fc38257e640 1 Processor -- start 2024-04-25T00:00:59.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.751+0000 7fc38257e640 1 -- start start 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc38257e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc38257e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc37c0d04f0 0x7fc37c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc38257e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37c0d0970 con 0x7fc37c1604d0 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc38257e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c15c6a0 con 0x7fc37c0d04f0 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc38257e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c15d6b0 con 0x7fc37c13f050 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc37b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc37c0d04f0 0x7fc37c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc37b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc37c0d04f0 0x7fc37c15ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38486/0 (socket says 172.21.15.113:38486) 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc37b7fe640 1 -- 172.21.15.113:0/1454507077 learned_addr learned my addr 172.21.15.113:0/1454507077 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.752+0000 7fc37bfff640 1 --2- 172.21.15.113:0/1454507077 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37b7fe640 1 -- 172.21.15.113:0/1454507077 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 msgr2=0x7fc37c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/1454507077 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37b7fe640 1 -- 172.21.15.113:0/1454507077 >> v1:172.21.15.178:6789/0 conn(0x7fc37c1604d0 legacy=0x7fc37c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37b7fe640 1 -- 172.21.15.113:0/1454507077 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c15c8f0 con 0x7fc37c0d04f0 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37bfff640 1 --2- 172.21.15.113:0/1454507077 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/1454507077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c0d04f0 0x7fc37c15ff90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc37002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13069115dc248787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37affd640 1 -- 172.21.15.113:0/1454507077 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037070 con 0x7fc37c0d04f0 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37affd640 1 -- 172.21.15.113:0/1454507077 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370032440 con 0x7fc37c0d04f0 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.753+0000 7fc37affd640 1 -- 172.21.15.113:0/1454507077 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370032810 con 0x7fc37c0d04f0 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 -- 172.21.15.113:0/1454507077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c0d04f0 msgr2=0x7fc37c15ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 --2- 172.21.15.113:0/1454507077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c0d04f0 0x7fc37c15ff90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc37002a180 comp rx=0 tx=0).stop 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 -- 172.21.15.113:0/1454507077 shutdown_connections 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 --2- 172.21.15.113:0/1454507077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c0d04f0 0x7fc37c15ff90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 --2- 172.21.15.113:0/1454507077 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 -- 172.21.15.113:0/1454507077 >> 172.21.15.113:0/1454507077 conn(0x7fc37c06b0f0 msgr2=0x7fc37c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 -- 172.21.15.113:0/1454507077 shutdown_connections 2024-04-25T00:00:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.754+0000 7fc38257e640 1 -- 172.21.15.113:0/1454507077 wait complete. 2024-04-25T00:00:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.755+0000 7fc38257e640 1 Processor -- start 2024-04-25T00:00:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.755+0000 7fc38257e640 1 -- start start 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc38257e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc37c0d04f0 0x7fc37c13bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc38257e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc38257e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c1604d0 0x7fc37c15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc38257e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c15d190 con 0x7fc37c1604d0 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc38257e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c15ceb0 con 0x7fc37c0d04f0 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc38257e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c13f430 con 0x7fc37c13f050 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c1604d0 0x7fc37c15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c1604d0 0x7fc37c15b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38494/0 (socket says 172.21.15.113:38494) 2024-04-25T00:00:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 -- 172.21.15.113:0/2614477957 learned_addr learned my addr 172.21.15.113:0/2614477957 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 -- 172.21.15.113:0/2614477957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 msgr2=0x7fc37c0728e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0728e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 -- 172.21.15.113:0/2614477957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc37c0d04f0 msgr2=0x7fc37c13bb60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc37c0d04f0 0x7fc37c13bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.756+0000 7fc380b1b640 1 -- 172.21.15.113:0/2614477957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c071840 con 0x7fc37c1604d0 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.757+0000 7fc380b1b640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c1604d0 0x7fc37c15b1d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc37400b970 tx=0x7fc37400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.757+0000 7fc378ff9640 1 -- 172.21.15.113:0/2614477957 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374019070 con 0x7fc37c1604d0 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.757+0000 7fc378ff9640 1 -- 172.21.15.113:0/2614477957 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37400cb40 con 0x7fc37c1604d0 2024-04-25T00:00:59.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.757+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c071b20 con 0x7fc37c1604d0 2024-04-25T00:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.757+0000 7fc378ff9640 1 -- 172.21.15.113:0/2614477957 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374014440 con 0x7fc37c1604d0 2024-04-25T00:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.757+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c0720e0 con 0x7fc37c1604d0 2024-04-25T00:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.758+0000 7fc378ff9640 1 -- 172.21.15.113:0/2614477957 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc37401c070 con 0x7fc37c1604d0 2024-04-25T00:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.758+0000 7fc378ff9640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc36806be60 0x7fc36806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.759+0000 7fc378ff9640 1 -- 172.21.15.113:0/2614477957 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc37408aad0 con 0x7fc37c1604d0 2024-04-25T00:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.759+0000 7fc37bfff640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc36806be60 0x7fc36806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.759+0000 7fc37bfff640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc36806be60 0x7fc36806e320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc37c15bd70 tx=0x7fc36c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.759+0000 7fc38536f740 1 Processor -- start 2024-04-25T00:00:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc38536f740 1 -- start start 2024-04-25T00:00:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d819d0 0x55f630d34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc38536f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630d391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc38536f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f630c9c570 con 0x55f630d64af0 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f630c44ba0 con 0x55f630d643b0 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f630c98530 con 0x55f630d819d0 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38498/0 (socket says 172.21.15.113:38498) 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.760+0000 7fc37b7fe640 1 -- 172.21.15.113:0/100503248 learned_addr learned my addr 172.21.15.113:0/100503248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc37b7fe640 1 -- 172.21.15.113:0/100503248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d819d0 msgr2=0x55f630d34310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/100503248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d819d0 0x55f630d34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc37b7fe640 1 -- 172.21.15.113:0/100503248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 msgr2=0x55f630d391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/100503248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630d391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc37b7fe640 1 -- 172.21.15.113:0/100503248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f630a7c9f0 con 0x55f630d64af0 2024-04-25T00:00:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/100503248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63e70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc37002a1c0 tx=0x7fc37002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955ff71fc9353404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc361ffb640 1 -- 172.21.15.113:0/100503248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037040 con 0x55f630d64af0 2024-04-25T00:00:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc361ffb640 1 -- 172.21.15.113:0/100503248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370042070 con 0x55f630d64af0 2024-04-25T00:00:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc361ffb640 1 -- 172.21.15.113:0/100503248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370032940 con 0x55f630d64af0 2024-04-25T00:00:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 -- 172.21.15.113:0/100503248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 msgr2=0x55f630d63e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 --2- 172.21.15.113:0/100503248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63e70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc37002a1c0 tx=0x7fc37002c000 comp rx=0 tx=0).stop 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 -- 172.21.15.113:0/100503248 shutdown_connections 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 --2- 172.21.15.113:0/100503248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630d391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 --2- 172.21.15.113:0/100503248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63e70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 --2- 172.21.15.113:0/100503248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d819d0 0x55f630d34310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 -- 172.21.15.113:0/100503248 >> 172.21.15.113:0/100503248 conn(0x55f630d365d0 msgr2=0x55f630d7ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.761+0000 7fc38536f740 1 -- 172.21.15.113:0/100503248 shutdown_connections 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.762+0000 7fc38536f740 1 -- 172.21.15.113:0/100503248 wait complete. 2024-04-25T00:00:59.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.762+0000 7fc38536f740 1 -- start start 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.762+0000 7fc38536f740 1 Processor -- start 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d64af0 0x55f630d67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc38536f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d819d0 0x55f630d65c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d80d90 0x55f630d4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc38536f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f630d38780 con 0x55f630d819d0 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f630c58130 con 0x55f630d64af0 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f630d37b70 con 0x55f630d80d90 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc38536f740 10 client.0 _ll_register_callbacks cb 0x55f630c57b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d819d0 0x55f630d65c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d819d0 0x55f630d65c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38510/0 (socket says 172.21.15.113:38510) 2024-04-25T00:00:59.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 -- 172.21.15.113:0/3335161080 learned_addr learned my addr 172.21.15.113:0/3335161080 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 -- 172.21.15.113:0/3335161080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d80d90 msgr2=0x55f630d4f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d80d90 0x55f630d4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 -- 172.21.15.113:0/3335161080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d64af0 msgr2=0x55f630d67cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d64af0 0x55f630d67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 -- 172.21.15.113:0/3335161080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6309702e0 con 0x55f630d819d0 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d819d0 0x55f630d65c60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc370009990 tx=0x7fc370004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c968366a8c1cae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.763+0000 7fc35affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc35affd640 1 -- 172.21.15.113:0/3335161080 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037040 con 0x55f630d819d0 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc35affd640 1 -- 172.21.15.113:0/3335161080 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc370003a20 con 0x55f630d819d0 2024-04-25T00:00:59.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc370042070 con 0x55f630d819d0 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc35affd640 1 -- 172.21.15.113:0/3335161080 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc370056050 con 0x55f630d819d0 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 10 client.5736 fetch_fsmap learned FSMap version 21 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f630add180 con 0x55f630d819d0 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc35affd640 1 -- 172.21.15.113:0/3335161080 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370042070 con 0x55f630d819d0 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc35affd640 1 -- 172.21.15.113:0/3335161080 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fc370059020 con 0x55f630d819d0 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 10 client.5736 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 10 client.5736 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 20 client.5736 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d4e680 0x55f630d61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 4 client.5736 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f630ca9ab0 con 0x55f630d4e680 2024-04-25T00:00:59.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc37bfff640 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d4e680 0x55f630d61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc35affd640 10 client.5736 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.764+0000 7fc37bfff640 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d4e680 0x55f630d61e50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655544cfecde0450 server_cookie=969064b6e94927e2 in_seq=0 out_seq=0 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc35affd640 1 -- 172.21.15.113:0/3335161080 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f630ca9ab0 con 0x55f630d4e680 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc35affd640 10 client.5736 handle_command_reply: tid=0 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 client.5736 shutdown 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d819d0 msgr2=0x55f630d65c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d819d0 0x55f630d65c60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc370009990 tx=0x7fc370004730 comp rx=0 tx=0).stop 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 shutdown_connections 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d4e680 0x55f630d61e50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d80d90 0x55f630d4f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d819d0 0x55f630d65c60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc35affd640 0 client.5736 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:59.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 --2- 172.21.15.113:0/3335161080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d64af0 0x55f630d67cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 >> 172.21.15.113:0/3335161080 conn(0x55f630d643b0 msgr2=0x55f630d82290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 shutdown_connections 2024-04-25T00:00:59.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 1 -- 172.21.15.113:0/3335161080 wait complete. 2024-04-25T00:00:59.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.765+0000 7fc38536f740 20 client.5736 trim_cache size 0 max 16384 2024-04-25T00:00:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 Processor -- start 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 -- start start 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d643b0 0x55f630d7f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d80d90 0x55f630d37fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c200920 con 0x55f630d64af0 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1ffa90 con 0x55f630d80d90 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.783+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c14ade0 con 0x55f630d643b0 2024-04-25T00:00:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38518/0 (socket says 172.21.15.113:38518) 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 -- 172.21.15.113:0/196827950 learned_addr learned my addr 172.21.15.113:0/196827950 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc380b1b640 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d80d90 0x55f630d37fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37bfff640 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d643b0 0x55f630d7f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 -- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d643b0 msgr2=0x55f630d7f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d643b0 0x55f630d7f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 -- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d80d90 msgr2=0x55f630d37fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d80d90 0x55f630d37fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 -- 172.21.15.113:0/196827950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6309493a0 con 0x55f630d64af0 2024-04-25T00:00:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc370009960 tx=0x7fc370045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc362e72c18211c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc3617fa640 1 -- 172.21.15.113:0/196827950 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037040 con 0x55f630d64af0 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc3617fa640 1 -- 172.21.15.113:0/196827950 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370042070 con 0x55f630d64af0 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.784+0000 7fc3617fa640 1 -- 172.21.15.113:0/196827950 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370032700 con 0x55f630d64af0 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 -- 172.21.15.113:0/196827950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 msgr2=0x55f630d63180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc370009960 tx=0x7fc370045000 comp rx=0 tx=0).stop 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 -- 172.21.15.113:0/196827950 shutdown_connections 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d80d90 0x55f630d37fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d64af0 0x55f630d63180 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 --2- 172.21.15.113:0/196827950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d643b0 0x55f630d7f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 -- 172.21.15.113:0/196827950 >> 172.21.15.113:0/196827950 conn(0x55f630d3b5c0 msgr2=0x55f630d2f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 -- 172.21.15.113:0/196827950 shutdown_connections 2024-04-25T00:00:59.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 -- 172.21.15.113:0/196827950 wait complete. 2024-04-25T00:00:59.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 -- start start 2024-04-25T00:00:59.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.785+0000 7fc38536f740 1 Processor -- start 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630ddec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc38536f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d64af0 0x55f630d7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc38536f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d80d90 0x55f630d7e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc38536f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f630d665a0 con 0x55f630d80d90 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c14b430 con 0x55f630d643b0 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc38536f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f630d3b250 con 0x55f630d64af0 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc38536f740 10 client.0 _ll_register_callbacks cb 0x7fc3700096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc380b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d80d90 0x55f630d7e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc380b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d80d90 0x55f630d7e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38528/0 (socket says 172.21.15.113:38528) 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc380b1b640 1 -- 172.21.15.113:0/140718703 learned_addr learned my addr 172.21.15.113:0/140718703 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.786+0000 7fc37b7fe640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d64af0 0x55f630d7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc37bfff640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630ddec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc380b1b640 1 -- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d64af0 msgr2=0x55f630d7ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc380b1b640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d64af0 0x55f630d7ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc380b1b640 1 -- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 msgr2=0x55f630ddec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc380b1b640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630ddec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc380b1b640 1 -- 172.21.15.113:0/140718703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f630d8ec70 con 0x55f630d80d90 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc37bfff640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630ddec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc380b1b640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d80d90 0x55f630d7e310 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc374089020 tx=0x7fc37405d130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e805f5811e5dba6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc35affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:00:59.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc35affd640 1 -- 172.21.15.113:0/140718703 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374087060 con 0x55f630d80d90 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc35affd640 1 -- 172.21.15.113:0/140718703 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc3740616d0 con 0x55f630d80d90 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc370042070 con 0x55f630d80d90 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc35affd640 1 -- 172.21.15.113:0/140718703 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc374075050 con 0x55f630d80d90 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 10 client.5742 fetch_fsmap learned FSMap version 21 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f630d8e010 con 0x55f630d80d90 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc35affd640 1 -- 172.21.15.113:0/140718703 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370042070 con 0x55f630d80d90 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc35affd640 1 -- 172.21.15.113:0/140718703 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fc3740581d0 con 0x55f630d80d90 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 10 client.5742 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 10 client.5742 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 20 client.5742 populate_metadata read hostname 'smithi113' 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d819d0 0x55f630ddf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 4 client.5742 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T00:00:59.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) -- 0x55f630d677a0 con 0x55f630d819d0 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.787+0000 7fc37bfff640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d819d0 0x55f630ddf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc35affd640 10 client.5742 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc37bfff640 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d819d0 0x55f630ddf5f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c694d8a995dff8 server_cookie=8728896050062bee in_seq=0 out_seq=0 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc35affd640 1 -- 172.21.15.113:0/140718703 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+26 (crc 0 0 0) 0x55f630d677a0 con 0x55f630d819d0 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc35affd640 10 client.5742 handle_command_reply: tid=0 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 client.5742 shutdown 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d80d90 msgr2=0x55f630d7e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d80d90 0x55f630d7e310 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc374089020 tx=0x7fc37405d130 comp rx=0 tx=0).stop 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 shutdown_connections 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f630d819d0 0x55f630ddf5f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f630d80d90 0x55f630d7e310 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f630d64af0 0x55f630d7ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc35affd640 0 client.5742 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 --2- 172.21.15.113:0/140718703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f630d643b0 0x55f630ddec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 >> 172.21.15.113:0/140718703 conn(0x55f630d3b5c0 msgr2=0x55f630d63180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 shutdown_connections 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 1 -- 172.21.15.113:0/140718703 wait complete. 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.788+0000 7fc38536f740 20 client.5742 trim_cache size 0 max 16384 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": -2 2024-04-25T00:00:59.797 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:00:59.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc36806be60 msgr2=0x7fc36806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc36806be60 0x7fc36806e320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc37c15bd70 tx=0x7fc36c008040 comp rx=0 tx=0).stop 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c1604d0 msgr2=0x7fc37c15b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c1604d0 0x7fc37c15b1d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc37400b970 tx=0x7fc37400be30 comp rx=0 tx=0).stop 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 shutdown_connections 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc36806be60 0x7fc36806e320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37c1604d0 0x7fc37c15b1d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc37c13f050 0x7fc37c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 --2- 172.21.15.113:0/2614477957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc37c0d04f0 0x7fc37c13bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 >> 172.21.15.113:0/2614477957 conn(0x7fc37c06b0f0 msgr2=0x7fc37c13c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 shutdown_connections 2024-04-25T00:00:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.789+0000 7fc38257e640 1 -- 172.21.15.113:0/2614477957 wait complete. 2024-04-25T00:00:59.800 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T00:00:59.800 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T00:00:59.800 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T00:00:59.801 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:59.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.897+0000 7fb226664640 1 Processor -- start 2024-04-25T00:00:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.897+0000 7fb226664640 1 -- start start 2024-04-25T00:00:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb226664640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb22015a6b0 0x7fb220070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb226664640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb220072ea0 0x7fb220071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb226664640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200b2d50 con 0x7fb220071780 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb226664640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb220072a70 con 0x7fb220072ea0 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb226664640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb2200b3010 con 0x7fb22015a6b0 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb21f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb220072ea0 0x7fb220071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb21f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb220072ea0 0x7fb220071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38534/0 (socket says 172.21.15.113:38534) 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.898+0000 7fb21f7fe640 1 -- 172.21.15.113:0/1897615822 learned_addr learned my addr 172.21.15.113:0/1897615822 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21f7fe640 1 -- 172.21.15.113:0/1897615822 >> v2:172.21.15.178:3300/0 conn(0x7fb22015a6b0 msgr2=0x7fb220070d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:00:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21f7fe640 1 --2- 172.21.15.113:0/1897615822 >> v2:172.21.15.178:3300/0 conn(0x7fb22015a6b0 0x7fb220070d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21f7fe640 1 -- 172.21.15.113:0/1897615822 >> v1:172.21.15.113:6789/0 conn(0x7fb220071780 legacy=0x7fb22015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21f7fe640 1 -- 172.21.15.113:0/1897615822 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200b32f0 con 0x7fb220072ea0 2024-04-25T00:00:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21effd640 1 -- 172.21.15.113:0/1897615822 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511742656 0 0) 0x7fb2200b2d50 con 0x7fb220071780 2024-04-25T00:00:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21f7fe640 1 --2- 172.21.15.113:0/1897615822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220072ea0 0x7fb220071240 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb220062850 tx=0x7fb21402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a145267d041b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21effd640 1 -- 172.21.15.113:0/1897615822 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214037070 con 0x7fb220072ea0 2024-04-25T00:00:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21effd640 1 -- 172.21.15.113:0/1897615822 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb214032440 con 0x7fb220072ea0 2024-04-25T00:00:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.899+0000 7fb21effd640 1 -- 172.21.15.113:0/1897615822 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140327f0 con 0x7fb220072ea0 2024-04-25T00:00:59.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 -- 172.21.15.113:0/1897615822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220072ea0 msgr2=0x7fb220071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 --2- 172.21.15.113:0/1897615822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220072ea0 0x7fb220071240 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb220062850 tx=0x7fb21402a290 comp rx=0 tx=0).stop 2024-04-25T00:00:59.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 -- 172.21.15.113:0/1897615822 shutdown_connections 2024-04-25T00:00:59.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 --2- 172.21.15.113:0/1897615822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220072ea0 0x7fb220071240 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 --2- 172.21.15.113:0/1897615822 >> v2:172.21.15.178:3300/0 conn(0x7fb22015a6b0 0x7fb220070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 -- 172.21.15.113:0/1897615822 >> 172.21.15.113:0/1897615822 conn(0x7fb2200b1a50 msgr2=0x7fb2200b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 -- 172.21.15.113:0/1897615822 shutdown_connections 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.900+0000 7fb226664640 1 -- 172.21.15.113:0/1897615822 wait complete. 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.901+0000 7fb226664640 1 Processor -- start 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.901+0000 7fb226664640 1 -- start start 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb226664640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220071780 0x7fb22014eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb226664640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb220072ea0 0x7fb22014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb226664640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22015a6b0 0x7fb2200aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb226664640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb22015aa90 con 0x7fb220071780 2024-04-25T00:00:59.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb226664640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb220072720 con 0x7fb22015a6b0 2024-04-25T00:00:59.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb226664640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb22015ac10 con 0x7fb220072ea0 2024-04-25T00:00:59.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220071780 0x7fb22014eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220071780 0x7fb22014eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38536/0 (socket says 172.21.15.113:38536) 2024-04-25T00:00:59.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 -- 172.21.15.113:0/2257157559 learned_addr learned my addr 172.21.15.113:0/2257157559 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21f7fe640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb220072ea0 0x7fb22014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb224c01640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22015a6b0 0x7fb2200aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 -- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb220072ea0 msgr2=0x7fb22014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb220072ea0 0x7fb22014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 -- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22015a6b0 msgr2=0x7fb2200aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22015a6b0 0x7fb2200aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21ffff640 1 -- 172.21.15.113:0/2257157559 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200a98c0 con 0x7fb220071780 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb21f7fe640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb220072ea0 0x7fb22014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.902+0000 7fb224c01640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22015a6b0 0x7fb2200aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.903+0000 7fb21ffff640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220071780 0x7fb22014eed0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb21000b410 tx=0x7fb21000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.903+0000 7fb21cff9640 1 -- 172.21.15.113:0/2257157559 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210015020 con 0x7fb220071780 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.903+0000 7fb21cff9640 1 -- 172.21.15.113:0/2257157559 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210011830 con 0x7fb220071780 2024-04-25T00:00:59.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.903+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200a9ba0 con 0x7fb220071780 2024-04-25T00:00:59.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.903+0000 7fb21cff9640 1 -- 172.21.15.113:0/2257157559 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210011c20 con 0x7fb220071780 2024-04-25T00:00:59.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.903+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2200aa160 con 0x7fb220071780 2024-04-25T00:00:59.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.904+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2200691a0 con 0x7fb220071780 2024-04-25T00:00:59.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.904+0000 7fb21cff9640 1 -- 172.21.15.113:0/2257157559 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb210018020 con 0x7fb220071780 2024-04-25T00:00:59.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.904+0000 7fb21cff9640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb20c06be30 0x7fb20c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:59.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.905+0000 7fb21f7fe640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb20c06be30 0x7fb20c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:59.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.905+0000 7fb21cff9640 1 -- 172.21.15.113:0/2257157559 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb21008ad90 con 0x7fb220071780 2024-04-25T00:00:59.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.905+0000 7fb21f7fe640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb20c06be30 0x7fb20c06e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb214034000 tx=0x7fb21402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:59.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:00:59.909+0000 7fb21cff9640 1 -- 172.21.15.113:0/2257157559 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb21001f020 con 0x7fb220071780 2024-04-25T00:01:00.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.134+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2200cdad0 con 0x7fb220071780 2024-04-25T00:01:00.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.135+0000 7fb21cff9640 1 -- 172.21.15.113:0/2257157559 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7fb2100172c0 con 0x7fb220071780 2024-04-25T00:01:00.135 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:00.135 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:00.135 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:01:00.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb20c06be30 msgr2=0x7fb20c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb20c06be30 0x7fb20c06e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb214034000 tx=0x7fb21402e000 comp rx=0 tx=0).stop 2024-04-25T00:01:00.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220071780 msgr2=0x7fb22014eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220071780 0x7fb22014eed0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb21000b410 tx=0x7fb21000b8d0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 shutdown_connections 2024-04-25T00:01:00.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb20c06be30 0x7fb20c06e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22015a6b0 0x7fb2200aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb220072ea0 0x7fb22014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 --2- 172.21.15.113:0/2257157559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb220071780 0x7fb22014eed0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 >> 172.21.15.113:0/2257157559 conn(0x7fb2200b1a50 msgr2=0x7fb220159770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 shutdown_connections 2024-04-25T00:01:00.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.137+0000 7fb226664640 1 -- 172.21.15.113:0/2257157559 wait complete. 2024-04-25T00:01:00.148 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T00:01:00.258 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:00.258 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:00.258 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:00.267 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:00.267 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T00:01:00.267 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T00:01:00.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.367+0000 7efd84a12640 1 Processor -- start 2024-04-25T00:01:00.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.367+0000 7efd84a12640 1 -- start start 2024-04-25T00:01:00.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.367+0000 7efd84a12640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd8015e510 0x7efd800d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.367+0000 7efd84a12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd800b63b0 0x7efd800b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.367+0000 7efd84a12640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd800d2e00 con 0x7efd800d28c0 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.367+0000 7efd84a12640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efd800d30e0 con 0x7efd8015e510 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.367+0000 7efd84a12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd800d33c0 con 0x7efd800b63b0 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efd800d28c0 0x7efd800b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33626/0 (socket says 172.21.15.113:33626) 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 -- 172.21.15.113:0/2126150128 learned_addr learned my addr 172.21.15.113:0/2126150128 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985395095 0 0) 0x7efd800d2e00 con 0x7efd800d28c0 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd6c003520 con 0x7efd800d28c0 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2936893325 0 0) 0x7efd6c003520 con 0x7efd800d28c0 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd800b63b0 msgr2=0x7efd800b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd800b63b0 0x7efd800b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 >> v2:172.21.15.178:3300/0 conn(0x7efd8015e510 msgr2=0x7efd800d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 --2- 172.21.15.113:0/2126150128 >> v2:172.21.15.178:3300/0 conn(0x7efd8015e510 0x7efd800d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd800d36a0 con 0x7efd800d28c0 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7efd68001670 con 0x7efd800d28c0 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 >> v1:172.21.15.113:6789/0 conn(0x7efd800d28c0 legacy=0x7efd800b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0039b0 0x7efd6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c004800 0x7efd6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd800b63b0 0x7efd6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd800d30e0 con 0x7efd6c004800 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd800d33c0 con 0x7efd6c0039b0 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd6c005600 con 0x7efd800b63b0 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c004800 0x7efd6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd800b63b0 0x7efd6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7e59c640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0039b0 0x7efd6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 -- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd800b63b0 msgr2=0x7efd6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd800b63b0 0x7efd6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 -- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0039b0 msgr2=0x7efd6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0039b0 0x7efd6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd77fff640 1 -- 172.21.15.113:0/2126150128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c003520 con 0x7efd6c004800 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.368+0000 7efd7e59c640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0039b0 0x7efd6c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.369+0000 7efd77fff640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c004800 0x7efd6c004c80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c270 tx=0x7efd6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6b2c53c4ab30a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.369+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68007d60 con 0x7efd6c004800 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.369+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6802e070 con 0x7efd6c004800 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.369+0000 7efd7dd9b640 1 -- 172.21.15.113:0/2126150128 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68005d10 con 0x7efd6c004800 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.370+0000 7efd84a12640 1 -- 172.21.15.113:0/2126150128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c004800 msgr2=0x7efd6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.370+0000 7efd84a12640 1 --2- 172.21.15.113:0/2126150128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c004800 0x7efd6c004c80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c270 tx=0x7efd6c005830 comp rx=0 tx=0).stop 2024-04-25T00:01:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.370+0000 7efd7ed9d640 1 -- 172.21.15.113:0/2126150128 reap_dead start 2024-04-25T00:01:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.370+0000 7efd84a12640 1 -- 172.21.15.113:0/2126150128 shutdown_connections 2024-04-25T00:01:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.370+0000 7efd84a12640 1 -- 172.21.15.113:0/2126150128 >> 172.21.15.113:0/2126150128 conn(0x7efd8006b720 msgr2=0x7efd800d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.370+0000 7efd84a12640 1 -- 172.21.15.113:0/2126150128 shutdown_connections 2024-04-25T00:01:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.370+0000 7efd84a12640 1 -- 172.21.15.113:0/2126150128 wait complete. 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.371+0000 7efd84a12640 1 Processor -- start 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.371+0000 7efd84a12640 1 -- start start 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd84a12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c004800 0x7efd80145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd84a12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd80160d10 0x7efd800ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd84a12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd801433f0 0x7efd80143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd84a12640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd800b9dd0 con 0x7efd80160d10 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd84a12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd800d3ea0 con 0x7efd6c004800 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd84a12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd800b73c0 con 0x7efd801433f0 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd80160d10 0x7efd800ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd80160d10 0x7efd800ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38546/0 (socket says 172.21.15.113:38546) 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 -- 172.21.15.113:0/2032374049 learned_addr learned my addr 172.21.15.113:0/2032374049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 -- 172.21.15.113:0/2032374049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd801433f0 msgr2=0x7efd80143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd801433f0 0x7efd80143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 -- 172.21.15.113:0/2032374049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c004800 msgr2=0x7efd80145e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c004800 0x7efd80145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 -- 172.21.15.113:0/2032374049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8015f9d0 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd77fff640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd80160d10 0x7efd800ca4c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd68034040 tx=0x7efd800d2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd76ffd640 1 -- 172.21.15.113:0/2032374049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68007280 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.372+0000 7efd76ffd640 1 -- 172.21.15.113:0/2032374049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68007420 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.373+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd801463b0 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.373+0000 7efd76ffd640 1 -- 172.21.15.113:0/2032374049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680077f0 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.373+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8015fc70 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.374+0000 7efd76ffd640 1 -- 172.21.15.113:0/2032374049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd68045030 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.374+0000 7efd76ffd640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd6006be30 0x7efd6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.374+0000 7efd76ffd640 1 -- 172.21.15.113:0/2032374049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7efd680ab310 con 0x7efd80160d10 2024-04-25T00:01:00.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.374+0000 7efd7e59c640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd6006be30 0x7efd6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.375+0000 7efd7e59c640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd6006be30 0x7efd6006e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd7000a8b0 tx=0x7efd70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.375+0000 7efd877f9740 1 Processor -- start 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd877f9740 1 -- start start 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a07897220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd877f9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0786cb20 0x564a078b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd877f9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564a077d04b0 con 0x564a0786cb20 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564a07778b30 con 0x564a078b46e0 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564a077cc470 con 0x564a07896e40 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.376+0000 7efd7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0786cb20 0x564a078b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0786cb20 0x564a078b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38558/0 (socket says 172.21.15.113:38558) 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 -- 172.21.15.113:0/816364417 learned_addr learned my addr 172.21.15.113:0/816364417 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7e59c640 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a07897220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd77fff640 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 -- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 msgr2=0x564a07897220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a07897220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 -- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 msgr2=0x564a078b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 -- 172.21.15.113:0/816364417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a075b0b00 con 0x564a0786cb20 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0786cb20 0x564a078b2da0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efd800d1bb0 tx=0x7efd78009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac476a425818ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd5ffff640 1 -- 172.21.15.113:0/816364417 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78008040 con 0x564a0786cb20 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd5ffff640 1 -- 172.21.15.113:0/816364417 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd78006e50 con 0x564a0786cb20 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.377+0000 7efd5ffff640 1 -- 172.21.15.113:0/816364417 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78003bb0 con 0x564a0786cb20 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 -- 172.21.15.113:0/816364417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0786cb20 msgr2=0x564a078b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0786cb20 0x564a078b2da0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efd800d1bb0 tx=0x7efd78009f60 comp rx=0 tx=0).stop 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 -- 172.21.15.113:0/816364417 shutdown_connections 2024-04-25T00:01:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0786cb20 0x564a078b2da0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 --2- 172.21.15.113:0/816364417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a07897220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 -- 172.21.15.113:0/816364417 >> 172.21.15.113:0/816364417 conn(0x564a07869450 msgr2=0x564a078b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 -- 172.21.15.113:0/816364417 shutdown_connections 2024-04-25T00:01:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 -- 172.21.15.113:0/816364417 wait complete. 2024-04-25T00:01:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 -- start start 2024-04-25T00:01:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.378+0000 7efd877f9740 1 Processor -- start 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a078a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd877f9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0789a690 0x564a078996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd877f9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564a0786d8c0 con 0x564a0789a690 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564a0778c0d0 con 0x564a078b46e0 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564a07867380 con 0x564a07896e40 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd877f9740 10 client.0 _ll_register_callbacks cb 0x564a074e3ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:00.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0789a690 0x564a078996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0789a690 0x564a078996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38562/0 (socket says 172.21.15.113:38562) 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd7ed9d640 1 -- 172.21.15.113:0/4230720364 learned_addr learned my addr 172.21.15.113:0/4230720364 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.379+0000 7efd7e59c640 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a078a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd77fff640 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd7ed9d640 1 -- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 msgr2=0x564a078a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a078a8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd7ed9d640 1 -- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 msgr2=0x564a078b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd7ed9d640 1 -- 172.21.15.113:0/4230720364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a074a43f0 con 0x564a0789a690 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0789a690 0x564a078996e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd78004000 tx=0x7efd78004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305957bd31b9c886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd5d7fa640 1 -- 172.21.15.113:0/4230720364 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78028060 con 0x564a0789a690 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd5d7fa640 1 -- 172.21.15.113:0/4230720364 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7efd78004240 con 0x564a0789a690 2024-04-25T00:01:00.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd78006e50 con 0x564a0789a690 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd5d7fa640 1 -- 172.21.15.113:0/4230720364 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd7802e050 con 0x564a0789a690 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 10 client.5763 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x564a07611290 con 0x564a0789a690 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd5d7fa640 1 -- 172.21.15.113:0/4230720364 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd78006e50 con 0x564a0789a690 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd5d7fa640 1 -- 172.21.15.113:0/4230720364 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7efd78031020 con 0x564a0789a690 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 10 client.5763 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 10 client.5763 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 20 client.5763 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a07899c20 0x564a07892f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 4 client.5763 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.380+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a077dd9f0 con 0x564a07899c20 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.381+0000 7efd7e59c640 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a07899c20 0x564a07892f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.381+0000 7efd5d7fa640 10 client.5763 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.381+0000 7efd7e59c640 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a07899c20 0x564a07892f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c625166643a027 server_cookie=79ec28d7972bda19 in_seq=0 out_seq=0 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.381+0000 7efd5d7fa640 1 -- 172.21.15.113:0/4230720364 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a077dd9f0 con 0x564a07899c20 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.381+0000 7efd5d7fa640 10 client.5763 handle_command_reply: tid=0 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.381+0000 7efd877f9740 1 client.5763 shutdown 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0789a690 msgr2=0x564a078996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0789a690 0x564a078996e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd78004000 tx=0x7efd78004b50 comp rx=0 tx=0).stop 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 shutdown_connections 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a07899c20 0x564a07892f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a078b46e0 0x564a078b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a0789a690 0x564a078996e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 --2- 172.21.15.113:0/4230720364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a07896e40 0x564a078a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd5d7fa640 0 client.5763 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 >> 172.21.15.113:0/4230720364 conn(0x564a0786cb20 msgr2=0x564a078b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 shutdown_connections 2024-04-25T00:01:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 1 -- 172.21.15.113:0/4230720364 wait complete. 2024-04-25T00:01:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.382+0000 7efd877f9740 20 client.5763 trim_cache size 0 max 16384 2024-04-25T00:01:00.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.399+0000 7efd877f9740 1 Processor -- start 2024-04-25T00:01:00.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.399+0000 7efd877f9740 1 -- start start 2024-04-25T00:01:00.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd877f9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a07896e40 0x564a078c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0789a690 0x564a078b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a078b46e0 0x564a07891ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd877f9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd80201130 con 0x564a07896e40 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd80200150 con 0x564a0789a690 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd8015c780 con 0x564a078b46e0 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a07896e40 0x564a078c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a07896e40 0x564a078c0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38576/0 (socket says 172.21.15.113:38576) 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 -- 172.21.15.113:0/1578157742 learned_addr learned my addr 172.21.15.113:0/1578157742 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a078b46e0 0x564a07891ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd77fff640 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0789a690 0x564a078b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 -- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a078b46e0 msgr2=0x564a07891ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a078b46e0 0x564a07891ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 -- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0789a690 msgr2=0x564a078b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0789a690 0x564a078b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 -- 172.21.15.113:0/1578157742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0747d4b0 con 0x564a07896e40 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd7e59c640 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a07896e40 0x564a078c0d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c520 tx=0x7efd70012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d947000e23932df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd5f7fe640 1 -- 172.21.15.113:0/1578157742 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70030070 con 0x564a07896e40 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd5f7fe640 1 -- 172.21.15.113:0/1578157742 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd700225d0 con 0x564a07896e40 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.400+0000 7efd5f7fe640 1 -- 172.21.15.113:0/1578157742 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700227a0 con 0x564a07896e40 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 -- 172.21.15.113:0/1578157742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a07896e40 msgr2=0x564a078c0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a07896e40 0x564a078c0d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c520 tx=0x7efd70012000 comp rx=0 tx=0).stop 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 -- 172.21.15.113:0/1578157742 shutdown_connections 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0789a690 0x564a078b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a078b46e0 0x564a07891ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 --2- 172.21.15.113:0/1578157742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a07896e40 0x564a078c0d80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 -- 172.21.15.113:0/1578157742 >> 172.21.15.113:0/1578157742 conn(0x564a0786cb20 msgr2=0x564a07895c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 -- 172.21.15.113:0/1578157742 shutdown_connections 2024-04-25T00:01:00.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.401+0000 7efd877f9740 1 -- 172.21.15.113:0/1578157742 wait complete. 2024-04-25T00:01:00.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.402+0000 7efd877f9740 1 -- start start 2024-04-25T00:01:00.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.402+0000 7efd877f9740 1 Processor -- start 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.402+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a07896e40 0x564a07898510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.402+0000 7efd877f9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a0789a690 0x564a07866500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.402+0000 7efd877f9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a078b46e0 0x564a07866a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd877f9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd801ffb00 con 0x564a078b46e0 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564a078b4450 con 0x564a07896e40 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd877f9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564a0792c040 con 0x564a0789a690 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd877f9740 10 client.0 _ll_register_callbacks cb 0x7efd78009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a078b46e0 0x564a07866a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a078b46e0 0x564a07866a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38588/0 (socket says 172.21.15.113:38588) 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 -- 172.21.15.113:0/1304135633 learned_addr learned my addr 172.21.15.113:0/1304135633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd77fff640 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a0789a690 0x564a07866500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7e59c640 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a07896e40 0x564a07898510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 -- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a0789a690 msgr2=0x564a07866500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a0789a690 0x564a07866500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 -- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a07896e40 msgr2=0x564a07898510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a07896e40 0x564a07898510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 -- 172.21.15.113:0/1304135633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a07895e50 con 0x564a078b46e0 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd7ed9d640 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a078b46e0 0x564a07866a40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efd78004650 tx=0x7efd780095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8e7a739fd981ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd5d7fa640 1 -- 172.21.15.113:0/1304135633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78008040 con 0x564a078b46e0 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd5d7fa640 1 -- 172.21.15.113:0/1304135633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7efd780047d0 con 0x564a078b46e0 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd700225d0 con 0x564a078b46e0 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd5d7fa640 1 -- 172.21.15.113:0/1304135633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd7803f050 con 0x564a078b46e0 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd877f9740 10 client.5769 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x564a078cc340 con 0x564a078b46e0 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.403+0000 7efd5d7fa640 1 -- 172.21.15.113:0/1304135633 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd700225d0 con 0x564a078b46e0 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd5d7fa640 1 -- 172.21.15.113:0/1304135633 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7efd78042020 con 0x564a078b46e0 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd877f9740 10 client.5769 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd877f9740 10 client.5769 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd877f9740 20 client.5769 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd877f9740 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a0792af50 0x564a0792b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd877f9740 4 client.5769 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T00:01:00.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) -- 0x564a077d7d40 con 0x564a0792af50 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd7e59c640 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a0792af50 0x564a0792b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd5d7fa640 10 client.5769 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd7e59c640 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a0792af50 0x564a0792b360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96943d05623c516 server_cookie=94972bafcf1014d6 in_seq=0 out_seq=0 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd5d7fa640 1 -- 172.21.15.113:0/1304135633 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564a077d7d40 con 0x564a0792af50 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd5d7fa640 10 client.5769 handle_command_reply: tid=0 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.404+0000 7efd877f9740 1 client.5769 shutdown 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a078b46e0 msgr2=0x564a07866a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a078b46e0 0x564a07866a40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efd78004650 tx=0x7efd780095d0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 shutdown_connections 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x564a0792af50 0x564a0792b360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564a078b46e0 0x564a07866a40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564a0789a690 0x564a07866500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 --2- 172.21.15.113:0/1304135633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a07896e40 0x564a07898510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd5d7fa640 0 client.5769 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 >> 172.21.15.113:0/1304135633 conn(0x564a0786cb20 msgr2=0x564a078b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 shutdown_connections 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 1 -- 172.21.15.113:0/1304135633 wait complete. 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd877f9740 20 client.5769 trim_cache size 0 max 16384 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "5c0e312a-0efe-49ff-beb1-a1740478ffbf", 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:00.412 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:00.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd6006be30 msgr2=0x7efd6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd6006be30 0x7efd6006e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd7000a8b0 tx=0x7efd70008040 comp rx=0 tx=0).stop 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd80160d10 msgr2=0x7efd800ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd80160d10 0x7efd800ca4c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd68034040 tx=0x7efd800d2fd0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 shutdown_connections 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd6006be30 0x7efd6006e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd801433f0 0x7efd80143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd80160d10 0x7efd800ca4c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 --2- 172.21.15.113:0/2032374049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c004800 0x7efd80145e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 >> 172.21.15.113:0/2032374049 conn(0x7efd8006b720 msgr2=0x7efd800b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 shutdown_connections 2024-04-25T00:01:00.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.405+0000 7efd84a12640 1 -- 172.21.15.113:0/2032374049 wait complete. 2024-04-25T00:01:00.416 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '5c0e312a-0efe-49ff-beb1-a1740478ffbf', 'mode': 'asynchronous'}' 2024-04-25T00:01:00.416 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T00:01:00.416 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T00:01:00.416 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:00.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.513+0000 7fe8b6363640 1 Processor -- start 2024-04-25T00:01:00.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.513+0000 7fe8b6363640 1 -- start start 2024-04-25T00:01:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8b6363640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe8b00b4b00 0x7fe8b01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8b6363640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b0071b20 con 0x7fe8b015a6b0 2024-04-25T00:01:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8b6363640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b00b2d50 con 0x7fe8b0072ea0 2024-04-25T00:01:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8b6363640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b0072a70 con 0x7fe8b00b4b00 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8affff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe8b015a6b0 0x7fe8b0070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33636/0 (socket says 172.21.15.113:33636) 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8affff640 1 -- 172.21.15.113:0/64701781 learned_addr learned my addr 172.21.15.113:0/64701781 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8b4900640 1 --2- 172.21.15.113:0/64701781 >> v2:172.21.15.178:3300/0 conn(0x7fe8b00b4b00 0x7fe8b01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145526255 0 0) 0x7fe8b0071b20 con 0x7fe8b015a6b0 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe898003520 con 0x7fe8b015a6b0 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3179549246 0 0) 0x7fe898003520 con 0x7fe8b015a6b0 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 >> v2:172.21.15.178:3300/0 conn(0x7fe8b00b4b00 msgr2=0x7fe8b01656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 --2- 172.21.15.113:0/64701781 >> v2:172.21.15.178:3300/0 conn(0x7fe8b00b4b00 0x7fe8b01656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 >> v1:172.21.15.178:6789/0 conn(0x7fe8b0072ea0 legacy=0x7fe8b0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b00b3010 con 0x7fe8b015a6b0 2024-04-25T00:01:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe8a0001670 con 0x7fe8b015a6b0 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 >> v1:172.21.15.113:6789/0 conn(0x7fe8b015a6b0 legacy=0x7fe8b0070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe898003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe898004800 0x7fe898004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8980051c0 0x7fe89800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0072a70 con 0x7fe8980039b0 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe89800c060 con 0x7fe8980051c0 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.514+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe89800c260 con 0x7fe898004800 2024-04-25T00:01:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8affff640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe898003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8af7fe640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe898004800 0x7fe898004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8affff640 1 -- 172.21.15.113:0/64701781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe898004800 msgr2=0x7fe898004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8affff640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe898004800 0x7fe898004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8affff640 1 -- 172.21.15.113:0/64701781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8980051c0 msgr2=0x7fe89800b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8affff640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8980051c0 0x7fe89800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8b4900640 1 -- 172.21.15.113:0/64701781 reap_dead start 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.515+0000 7fe8affff640 1 -- 172.21.15.113:0/64701781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe898003520 con 0x7fe8980039b0 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8affff640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe898003dd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c240 tx=0x7fe898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e913770e55aa3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0007d10 con 0x7fe8980039b0 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a0005ad0 con 0x7fe8980039b0 2024-04-25T00:01:00.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8aeffd640 1 -- 172.21.15.113:0/64701781 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0007070 con 0x7fe8980039b0 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8b6363640 1 -- 172.21.15.113:0/64701781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 msgr2=0x7fe898003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8b6363640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe898003dd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c240 tx=0x7fe898005830 comp rx=0 tx=0).stop 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8b6363640 1 -- 172.21.15.113:0/64701781 shutdown_connections 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8b6363640 1 --2- 172.21.15.113:0/64701781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe898003dd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8b6363640 1 -- 172.21.15.113:0/64701781 >> 172.21.15.113:0/64701781 conn(0x7fe8b00b1a50 msgr2=0x7fe8b00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8b6363640 1 -- 172.21.15.113:0/64701781 shutdown_connections 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.516+0000 7fe8b6363640 1 -- 172.21.15.113:0/64701781 wait complete. 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.517+0000 7fe8b6363640 1 Processor -- start 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b6363640 1 -- start start 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b6363640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe8b01508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b6363640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b00ccdf0 0x7fe8b0145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b6363640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b00ce3e0 0x7fe8b00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b6363640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0072720 con 0x7fe8980039b0 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b6363640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b00b35d0 con 0x7fe8b00ce3e0 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b6363640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b015aa90 con 0x7fe8b00ccdf0 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8affff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe8b01508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8affff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe8b01508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38604/0 (socket says 172.21.15.113:38604) 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8affff640 1 -- 172.21.15.113:0/3367011340 learned_addr learned my addr 172.21.15.113:0/3367011340 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8b4900640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b00ce3e0 0x7fe8b00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.518+0000 7fe8af7fe640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b00ccdf0 0x7fe8b0145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8affff640 1 -- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b00ccdf0 msgr2=0x7fe8b0145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8affff640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b00ccdf0 0x7fe8b0145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8affff640 1 -- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b00ce3e0 msgr2=0x7fe8b00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8affff640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b00ce3e0 0x7fe8b00ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8affff640 1 -- 172.21.15.113:0/3367011340 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b00cd340 con 0x7fe8980039b0 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8b4900640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b00ce3e0 0x7fe8b00ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8affff640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe8b01508b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0001970 tx=0x7fe8b00b31e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8acff9640 1 -- 172.21.15.113:0/3367011340 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0041070 con 0x7fe8980039b0 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8acff9640 1 -- 172.21.15.113:0/3367011340 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a0000ed0 con 0x7fe8980039b0 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b00cd620 con 0x7fe8980039b0 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8acff9640 1 -- 172.21.15.113:0/3367011340 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0007d10 con 0x7fe8980039b0 2024-04-25T00:01:00.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.519+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b00cdbe0 con 0x7fe8980039b0 2024-04-25T00:01:00.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.520+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b00691a0 con 0x7fe8980039b0 2024-04-25T00:01:00.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.520+0000 7fe8acff9640 1 -- 172.21.15.113:0/3367011340 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8a0044070 con 0x7fe8980039b0 2024-04-25T00:01:00.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.521+0000 7fe8acff9640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89c06be30 0x7fe89c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.521+0000 7fe8acff9640 1 -- 172.21.15.113:0/3367011340 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe8a00ab530 con 0x7fe8980039b0 2024-04-25T00:01:00.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.521+0000 7fe8af7fe640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89c06be30 0x7fe89c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.521+0000 7fe8af7fe640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89c06be30 0x7fe89c06e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000c00 tx=0x7fe8a4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.525+0000 7fe8acff9640 1 -- 172.21.15.113:0/3367011340 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe8a0079f30 con 0x7fe8980039b0 2024-04-25T00:01:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.735+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8b00b03c0 con 0x7fe8980039b0 2024-04-25T00:01:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.735+0000 7fe8acff9640 1 -- 172.21.15.113:0/3367011340 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7fe8a0079830 con 0x7fe8980039b0 2024-04-25T00:01:00.736 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:00.736 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:01:00.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89c06be30 msgr2=0x7fe89c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89c06be30 0x7fe89c06e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000c00 tx=0x7fe8a4009040 comp rx=0 tx=0).stop 2024-04-25T00:01:00.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 msgr2=0x7fe8b01508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe8b01508b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0001970 tx=0x7fe8b00b31e0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 shutdown_connections 2024-04-25T00:01:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe89c06be30 0x7fe89c06e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8b00ce3e0 0x7fe8b00ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8b00ccdf0 0x7fe8b0145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 --2- 172.21.15.113:0/3367011340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8980039b0 0x7fe8b01508b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 >> 172.21.15.113:0/3367011340 conn(0x7fe8b00b1a50 msgr2=0x7fe8b00b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 shutdown_connections 2024-04-25T00:01:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.738+0000 7fe8b6363640 1 -- 172.21.15.113:0/3367011340 wait complete. 2024-04-25T00:01:00.749 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T00:01:00.856 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:00.857 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:00.857 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:00.866 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:00.866 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T00:01:00.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T00:01:00.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.967+0000 7fd5dacb3640 1 Processor -- start 2024-04-25T00:01:00.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.967+0000 7fd5dacb3640 1 -- start start 2024-04-25T00:01:00.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d40719d0 0x7fd5d415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5dacb3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5d406cea0 0x7fd5d40cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5dacb3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d4071580 con 0x7fd5d4071db0 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5dacb3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d4141620 con 0x7fd5d406cea0 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d4141fc0 con 0x7fd5d40719d0 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5d9250640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5d406cea0 0x7fd5d40cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5cbfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd5d4071db0 0x7fd5d41418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33646/0 (socket says 172.21.15.113:33646) 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5cbfff640 1 -- 172.21.15.113:0/3321209717 learned_addr learned my addr 172.21.15.113:0/3321209717 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5d9250640 1 -- 172.21.15.113:0/3321209717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d40719d0 msgr2=0x7fd5d415f2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5d9250640 1 --2- 172.21.15.113:0/3321209717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d40719d0 0x7fd5d415f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5d9250640 1 -- 172.21.15.113:0/3321209717 >> v1:172.21.15.113:6789/0 conn(0x7fd5d4071db0 legacy=0x7fd5d41418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5cb7fe640 1 -- 172.21.15.113:0/3321209717 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485116913 0 0) 0x7fd5c4001110 con 0x7fd5d4071db0 2024-04-25T00:01:00.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.968+0000 7fd5d9250640 1 -- 172.21.15.113:0/3321209717 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d406d3f0 con 0x7fd5d406cea0 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.969+0000 7fd5d9250640 1 --2- 172.21.15.113:0/3321209717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d40cf020 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009d90 tx=0x7fd5cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30b0ffe4a655472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.969+0000 7fd5cb7fe640 1 -- 172.21.15.113:0/3321209717 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc007740 con 0x7fd5d406cea0 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.969+0000 7fd5cb7fe640 1 -- 172.21.15.113:0/3321209717 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc0078e0 con 0x7fd5d406cea0 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.969+0000 7fd5cb7fe640 1 -- 172.21.15.113:0/3321209717 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc007cb0 con 0x7fd5d406cea0 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 -- 172.21.15.113:0/3321209717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 msgr2=0x7fd5d40cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/3321209717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d40cf020 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009d90 tx=0x7fd5cc02a710 comp rx=0 tx=0).stop 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 -- 172.21.15.113:0/3321209717 shutdown_connections 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/3321209717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d40cf020 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/3321209717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d40719d0 0x7fd5d415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 -- 172.21.15.113:0/3321209717 >> 172.21.15.113:0/3321209717 conn(0x7fd5d406b0f0 msgr2=0x7fd5d406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 -- 172.21.15.113:0/3321209717 shutdown_connections 2024-04-25T00:01:00.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.970+0000 7fd5dacb3640 1 -- 172.21.15.113:0/3321209717 wait complete. 2024-04-25T00:01:00.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.971+0000 7fd5dacb3640 1 Processor -- start 2024-04-25T00:01:00.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.971+0000 7fd5dacb3640 1 -- start start 2024-04-25T00:01:00.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d40719d0 0x7fd5d4155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d4071db0 0x7fd5d40b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41439e0 con 0x7fd5d406cea0 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d406d9b0 con 0x7fd5d40719d0 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d406dc90 con 0x7fd5d4071db0 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d4158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38622/0 (socket says 172.21.15.113:38622) 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/432449452 learned_addr learned my addr 172.21.15.113:0/432449452 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d40719d0 0x7fd5d4155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/432449452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d4071db0 msgr2=0x7fd5d40b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d4071db0 0x7fd5d40b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/432449452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d40719d0 msgr2=0x7fd5d4155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d40719d0 0x7fd5d4155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/432449452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d4158c90 con 0x7fd5d406cea0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d4158750 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b450 tx=0x7fd5d000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5c97fa640 1 -- 172.21.15.113:0/432449452 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0015020 con 0x7fd5d406cea0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5c97fa640 1 -- 172.21.15.113:0/432449452 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d0011850 con 0x7fd5d406cea0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d40b2310 con 0x7fd5d406cea0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5c97fa640 1 -- 172.21.15.113:0/432449452 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0011c40 con 0x7fd5d406cea0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.972+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d40b28d0 con 0x7fd5d406cea0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.974+0000 7fd5c97fa640 1 -- 172.21.15.113:0/432449452 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5d0018020 con 0x7fd5d406cea0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.974+0000 7fd5c97fa640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5c006be30 0x7fd5c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.975+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5c006be30 0x7fd5c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.975+0000 7fd5c97fa640 1 -- 172.21.15.113:0/432449452 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd5d008aee0 con 0x7fd5d406cea0 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.975+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5c006be30 0x7fd5c006e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5d41318d0 tx=0x7fd5c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.975+0000 7fd5dd9c2740 1 Processor -- start 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.975+0000 7fd5dd9c2740 1 -- start start 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104d9f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f4560 0x56104da227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da22ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56104d95b730 con 0x56104d9f7d80 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56104d903de0 con 0x56104d9f4560 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56104d9576f0 con 0x56104da1ffc0 2024-04-25T00:01:00.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104d9f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104d9f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38624/0 (socket says 172.21.15.113:38624) 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/170067566 learned_addr learned my addr 172.21.15.113:0/170067566 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d9250640 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da22ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 msgr2=0x56104da22ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da22ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f4560 msgr2=0x56104da227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f4560 0x56104da227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/170067566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104d73bf00 con 0x56104d9f7d80 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.976+0000 7fd5d9250640 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da22ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104d9f8160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0061760 tx=0x7fd5d00591f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24ba0c73d25543f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5ba7fc640 1 -- 172.21.15.113:0/170067566 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00710d0 con 0x56104d9f7d80 2024-04-25T00:01:00.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5ba7fc640 1 -- 172.21.15.113:0/170067566 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d0057150 con 0x56104d9f7d80 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5ba7fc640 1 -- 172.21.15.113:0/170067566 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0055720 con 0x56104d9f7d80 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/170067566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 msgr2=0x56104d9f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104d9f8160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0061760 tx=0x7fd5d00591f0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/170067566 shutdown_connections 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f4560 0x56104da227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da22ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/170067566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104d9f8160 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/170067566 >> 172.21.15.113:0/170067566 conn(0x56104d9bf3c0 msgr2=0x56104da3d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/170067566 shutdown_connections 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.977+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/170067566 wait complete. 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.978+0000 7fd5dd9c2740 1 -- start start 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.978+0000 7fd5dd9c2740 1 Processor -- start 2024-04-25T00:01:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 0x56104da14390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56104d9f89a0 con 0x56104d9f7d80 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56104d917360 con 0x56104da1ffc0 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56104d9f8cd0 con 0x56104d9f4560 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5dd9c2740 10 client.0 _ll_register_callbacks cb 0x56104d9565b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38636/0 (socket says 172.21.15.113:38636) 2024-04-25T00:01:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 -- 172.21.15.113:0/1006575294 learned_addr learned my addr 172.21.15.113:0/1006575294 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 -- 172.21.15.113:0/1006575294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 msgr2=0x56104da14390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 0x56104da14390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5d9250640 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 -- 172.21.15.113:0/1006575294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 msgr2=0x56104da514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.979+0000 7fd5cbfff640 1 -- 172.21.15.113:0/1006575294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104d62f7f0 con 0x56104d9f7d80 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da148d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4005a80 tx=0x7fd5c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aee28c487b6b05a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5b37fe640 1 -- 172.21.15.113:0/1006575294 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400de60 con 0x56104d9f7d80 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5b37fe640 1 -- 172.21.15.113:0/1006575294 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd5c4022b10 con 0x56104d9f7d80 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d0057150 con 0x56104d9f7d80 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5b37fe640 1 -- 172.21.15.113:0/1006575294 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd5c4022d10 con 0x56104d9f7d80 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 10 client.5790 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56104d79c690 con 0x56104d9f7d80 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5b37fe640 1 -- 172.21.15.113:0/1006575294 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d0057150 con 0x56104d9f7d80 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5b37fe640 1 -- 172.21.15.113:0/1006575294 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fd5c4029050 con 0x56104d9f7d80 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 10 client.5790 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 10 client.5790 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 20 client.5790 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:00.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104da22720 0x56104da22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 4 client.5790 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104d968c70 con 0x56104da22720 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104da22720 0x56104da22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5b37fe640 10 client.5790 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.980+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104da22720 0x56104da22b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4a0dce067bcba3 server_cookie=1f8707c3b601eca6 in_seq=0 out_seq=0 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5b37fe640 1 -- 172.21.15.113:0/1006575294 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56104d968c70 con 0x56104da22720 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5b37fe640 10 client.5790 handle_command_reply: tid=0 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 client.5790 shutdown 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 msgr2=0x56104da148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da148d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4005a80 tx=0x7fd5c4005e80 comp rx=0 tx=0).stop 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 shutdown_connections 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104da22720 0x56104da22b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da148d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/1006575294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 0x56104da14390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5b37fe640 0 client.5790 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 >> 172.21.15.113:0/1006575294 conn(0x56104d9bf3c0 msgr2=0x56104d9f6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 shutdown_connections 2024-04-25T00:01:00.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/1006575294 wait complete. 2024-04-25T00:01:00.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.981+0000 7fd5dd9c2740 20 client.5790 trim_cache size 0 max 16384 2024-04-25T00:01:01.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 Processor -- start 2024-04-25T00:01:01.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 -- start start 2024-04-25T00:01:01.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 0x56104da26000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da20570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4200aa0 con 0x56104d9f7d80 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41ffad0 con 0x56104da1ffc0 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d4134440 con 0x56104d9f4560 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5cbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da20570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5cbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da20570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38646/0 (socket says 172.21.15.113:38646) 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:00.999+0000 7fd5cbfff640 1 -- 172.21.15.113:0/3346646444 learned_addr learned my addr 172.21.15.113:0/3346646444 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5d9250640 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da22cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 0x56104da26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5cbfff640 1 -- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 msgr2=0x56104da26000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 0x56104da26000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5cbfff640 1 -- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 msgr2=0x56104da22cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da22cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5cbfff640 1 -- 172.21.15.113:0/3346646444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104d6088b0 con 0x56104d9f7d80 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5d9250640 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da20570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40077a0 tx=0x7fd5c401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5b5cbfc6e3ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5b9ffb640 1 -- 172.21.15.113:0/3346646444 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4030030 con 0x56104d9f7d80 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5b9ffb640 1 -- 172.21.15.113:0/3346646444 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c402b7e0 con 0x56104d9f7d80 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.000+0000 7fd5b9ffb640 1 -- 172.21.15.113:0/3346646444 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400dea0 con 0x56104d9f7d80 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3346646444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 msgr2=0x56104da20570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da20570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40077a0 tx=0x7fd5c401c000 comp rx=0 tx=0).stop 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3346646444 shutdown_connections 2024-04-25T00:01:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104da1ffc0 0x56104da22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f7d80 0x56104da20570 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3346646444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104d9f4560 0x56104da26000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3346646444 >> 172.21.15.113:0/3346646444 conn(0x56104d9bf3c0 msgr2=0x56104da23a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3346646444 shutdown_connections 2024-04-25T00:01:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3346646444 wait complete. 2024-04-25T00:01:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 -- start start 2024-04-25T00:01:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.001+0000 7fd5dd9c2740 1 Processor -- start 2024-04-25T00:01:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f4560 0x56104da18320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f7d80 0x56104da22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5dd9c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da23230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4157f80 con 0x56104d9f4560 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56104d9f2810 con 0x56104d9f7d80 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5dd9c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56104da1eb70 con 0x56104da1ffc0 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5dd9c2740 10 client.0 _ll_register_callbacks cb 0x7fd5c4001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f4560 0x56104da18320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f4560 0x56104da18320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38654/0 (socket says 172.21.15.113:38654) 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/3379347548 learned_addr learned my addr 172.21.15.113:0/3379347548 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/3379347548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 msgr2=0x56104da23230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da23230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/3379347548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f7d80 msgr2=0x56104da22cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f7d80 0x56104da22cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.002+0000 7fd5d8a4f640 1 -- 172.21.15.113:0/3379347548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104da1ee50 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5d8a4f640 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f4560 0x56104da18320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd5d005d620 tx=0x7fd5d0089200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbdb9ed0efd5bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5b37fe640 1 -- 172.21.15.113:0/3379347548 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0086070 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5b37fe640 1 -- 172.21.15.113:0/3379347548 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd5d00573b0 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c402b7e0 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5b37fe640 1 -- 172.21.15.113:0/3379347548 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd5d0071030 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 10 client.5796 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56104dab72d0 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5b37fe640 1 -- 172.21.15.113:0/3379347548 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c402b7e0 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5b37fe640 1 -- 172.21.15.113:0/3379347548 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fd5d005d040 con 0x56104d9f4560 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 10 client.5796 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:01.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 10 client.5796 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 20 client.5796 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104daae3f0 0x56104da24d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 4 client.5796 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) -- 0x56104d962fc0 con 0x56104daae3f0 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104daae3f0 0x56104da24d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5b37fe640 10 client.5796 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.003+0000 7fd5cbfff640 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104daae3f0 0x56104da24d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a095b2bd05c3b14 server_cookie=47f79f0183f30b5a in_seq=0 out_seq=0 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5b37fe640 1 -- 172.21.15.113:0/3379347548 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56104d962fc0 con 0x56104daae3f0 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5b37fe640 10 client.5796 handle_command_reply: tid=0 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 client.5796 shutdown 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f4560 msgr2=0x56104da18320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f4560 0x56104da18320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd5d005d620 tx=0x7fd5d0089200 comp rx=0 tx=0).stop 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 shutdown_connections 2024-04-25T00:01:01.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56104daae3f0 0x56104da24d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56104d9f7d80 0x56104da22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56104da1ffc0 0x56104da23230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5b37fe640 0 client.5796 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 --2- 172.21.15.113:0/3379347548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56104d9f4560 0x56104da18320 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 >> 172.21.15.113:0/3379347548 conn(0x56104d9bf3c0 msgr2=0x56104da14640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 shutdown_connections 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.004+0000 7fd5dd9c2740 1 -- 172.21.15.113:0/3379347548 wait complete. 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dd9c2740 20 client.5796 trim_cache size 0 max 16384 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:01.011 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:01.012 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "c753840d-5202-4ec0-b5d8-014dcabcccd1", 2024-04-25T00:01:01.012 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:01.012 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5c006be30 msgr2=0x7fd5c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5c006be30 0x7fd5c006e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5d41318d0 tx=0x7fd5c4009040 comp rx=0 tx=0).stop 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 msgr2=0x7fd5d4158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d4158750 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b450 tx=0x7fd5d000b910 comp rx=0 tx=0).stop 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 shutdown_connections 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5c006be30 0x7fd5c006e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d40719d0 0x7fd5d4155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5d4071db0 0x7fd5d40b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 --2- 172.21.15.113:0/432449452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d406cea0 0x7fd5d4158750 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 >> 172.21.15.113:0/432449452 conn(0x7fd5d406b0f0 msgr2=0x7fd5d415cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 shutdown_connections 2024-04-25T00:01:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.005+0000 7fd5dacb3640 1 -- 172.21.15.113:0/432449452 wait complete. 2024-04-25T00:01:01.015 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'c753840d-5202-4ec0-b5d8-014dcabcccd1', 'mode': 'asynchronous'}' 2024-04-25T00:01:01.016 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T00:01:01.016 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T00:01:01.016 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.112+0000 7f3b321c8640 1 Processor -- start 2024-04-25T00:01:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.112+0000 7f3b321c8640 1 -- start start 2024-04-25T00:01:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b321c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c062290 0x7f3b2c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b321c8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b2c09eaf0 0x7f3b2c06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b321c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c0cb210 con 0x7f3b2c15c280 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b321c8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2c06bf60 con 0x7f3b2c09eaf0 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b321c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c06b590 con 0x7f3b2c062290 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b2c09eaf0 0x7f3b2c06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b2c09eaf0 0x7f3b2c06ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38656/0 (socket says 172.21.15.113:38656) 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2bfff640 1 -- 172.21.15.113:0/1788086448 learned_addr learned my addr 172.21.15.113:0/1788086448 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2affd640 1 --2- 172.21.15.113:0/1788086448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c062290 0x7f3b2c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2bfff640 1 -- 172.21.15.113:0/1788086448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c062290 msgr2=0x7f3b2c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2bfff640 1 --2- 172.21.15.113:0/1788086448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c062290 0x7f3b2c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2bfff640 1 -- 172.21.15.113:0/1788086448 >> v1:172.21.15.178:6789/0 conn(0x7f3b2c15c280 legacy=0x7f3b2c061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.113+0000 7f3b2bfff640 1 -- 172.21.15.113:0/1788086448 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0cbb30 con 0x7f3b2c09eaf0 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.114+0000 7f3b2bfff640 1 --2- 172.21.15.113:0/1788086448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c06ba20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009e50 tx=0x7f3b2402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c55dd43811e451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.114+0000 7f3b2a7fc640 1 -- 172.21.15.113:0/1788086448 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24036070 con 0x7f3b2c09eaf0 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.114+0000 7f3b2a7fc640 1 -- 172.21.15.113:0/1788086448 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24031600 con 0x7f3b2c09eaf0 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.114+0000 7f3b2a7fc640 1 -- 172.21.15.113:0/1788086448 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240319d0 con 0x7f3b2c09eaf0 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 -- 172.21.15.113:0/1788086448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c09eaf0 msgr2=0x7f3b2c06ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 --2- 172.21.15.113:0/1788086448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c06ba20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009e50 tx=0x7f3b2402a680 comp rx=0 tx=0).stop 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 -- 172.21.15.113:0/1788086448 shutdown_connections 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 --2- 172.21.15.113:0/1788086448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c062290 0x7f3b2c0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 --2- 172.21.15.113:0/1788086448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c06ba20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 -- 172.21.15.113:0/1788086448 >> 172.21.15.113:0/1788086448 conn(0x7f3b2c0cd860 msgr2=0x7f3b2c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 -- 172.21.15.113:0/1788086448 shutdown_connections 2024-04-25T00:01:01.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.115+0000 7f3b321c8640 1 -- 172.21.15.113:0/1788086448 wait complete. 2024-04-25T00:01:01.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.116+0000 7f3b321c8640 1 Processor -- start 2024-04-25T00:01:01.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.116+0000 7f3b321c8640 1 -- start start 2024-04-25T00:01:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c062290 0x7f3b2c1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c15c280 0x7f3b2c146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c0cc0f0 con 0x7f3b2c062290 2024-04-25T00:01:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c06b220 con 0x7f3b2c09eaf0 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c15fc90 con 0x7f3b2c15c280 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c062290 0x7f3b2c1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c062290 0x7f3b2c1482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38670/0 (socket says 172.21.15.113:38670) 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 -- 172.21.15.113:0/734340594 learned_addr learned my addr 172.21.15.113:0/734340594 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2affd640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2bfff640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c15c280 0x7f3b2c146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 -- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c15c280 msgr2=0x7f3b2c146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c15c280 0x7f3b2c146870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 -- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b2c09eaf0 msgr2=0x7f3b2c145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 -- 172.21.15.113:0/734340594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c147250 con 0x7f3b2c062290 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2affd640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b2b7fe640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c062290 0x7f3b2c1482f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c3f0 tx=0x7f3b1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b13fff640 1 -- 172.21.15.113:0/734340594 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0052d0 con 0x7f3b2c062290 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b13fff640 1 -- 172.21.15.113:0/734340594 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c005490 con 0x7f3b2c062290 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c147530 con 0x7f3b2c062290 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b13fff640 1 -- 172.21.15.113:0/734340594 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c007260 con 0x7f3b2c062290 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.117+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c147af0 con 0x7f3b2c062290 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.118+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c0691a0 con 0x7f3b2c062290 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.119+0000 7f3b13fff640 1 -- 172.21.15.113:0/734340594 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b1c007400 con 0x7f3b2c062290 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.119+0000 7f3b13fff640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b1806be30 0x7f3b1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.119+0000 7f3b2affd640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b1806be30 0x7f3b1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.119+0000 7f3b13fff640 1 -- 172.21.15.113:0/734340594 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f3b1c08a470 con 0x7f3b2c062290 2024-04-25T00:01:01.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.120+0000 7f3b2affd640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b1806be30 0x7f3b1806e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0aebd0 tx=0x7f3b20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.124+0000 7f3b13fff640 1 -- 172.21.15.113:0/734340594 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b1c058cd0 con 0x7f3b2c062290 2024-04-25T00:01:01.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.341+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b2c06dfa0 con 0x7f3b2c062290 2024-04-25T00:01:01.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.342+0000 7f3b13fff640 1 -- 172.21.15.113:0/734340594 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7f3b1c0585d0 con 0x7f3b2c062290 2024-04-25T00:01:01.342 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:01.342 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:01.342 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b1806be30 msgr2=0x7f3b1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b1806be30 0x7f3b1806e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0aebd0 tx=0x7f3b20008040 comp rx=0 tx=0).stop 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c062290 msgr2=0x7f3b2c1482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c062290 0x7f3b2c1482f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c3f0 tx=0x7f3b1c00c8b0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 shutdown_connections 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b1806be30 0x7f3b1806e2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b2c09eaf0 0x7f3b2c145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b2c15c280 0x7f3b2c146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 --2- 172.21.15.113:0/734340594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b2c062290 0x7f3b2c1482f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 >> 172.21.15.113:0/734340594 conn(0x7f3b2c0cd860 msgr2=0x7f3b2c0b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 shutdown_connections 2024-04-25T00:01:01.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.344+0000 7f3b321c8640 1 -- 172.21.15.113:0/734340594 wait complete. 2024-04-25T00:01:01.355 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T00:01:01.462 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:01.462 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:01.462 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:01.472 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:01.472 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T00:01:01.472 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T00:01:01.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.570+0000 7f2e1f3a2640 1 Processor -- start 2024-04-25T00:01:01.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.570+0000 7f2e1f3a2640 1 -- start start 2024-04-25T00:01:01.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e1f3a2640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e180b4a40 0x7f2e18165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e1f3a2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e1815f4f0 con 0x7f2e1815e2d0 2024-04-25T00:01:01.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e1f3a2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e1815f7d0 con 0x7f2e180b3080 2024-04-25T00:01:01.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e1f3a2640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2e180cde00 con 0x7f2e180b4a40 2024-04-25T00:01:01.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e1d13e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2e1815e2d0 0x7f2e1815ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33650/0 (socket says 172.21.15.113:33650) 2024-04-25T00:01:01.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e1d13e640 1 -- 172.21.15.113:0/2141818991 learned_addr learned my addr 172.21.15.113:0/2141818991 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4051136979 0 0) 0x7f2e1815f4f0 con 0x7f2e1815e2d0 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/2141818991 >> v2:172.21.15.178:3300/0 conn(0x7f2e180b4a40 0x7f2e18165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e04003520 con 0x7f2e1815e2d0 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1309776630 0 0) 0x7f2e04003520 con 0x7f2e1815e2d0 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 >> v2:172.21.15.178:3300/0 conn(0x7f2e180b4a40 msgr2=0x7f2e18165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 --2- 172.21.15.113:0/2141818991 >> v2:172.21.15.178:3300/0 conn(0x7f2e180b4a40 0x7f2e18165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 >> v1:172.21.15.178:6789/0 conn(0x7f2e180b3080 legacy=0x7f2e180b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1815fab0 con 0x7f2e1815e2d0 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f2e14001670 con 0x7f2e1815e2d0 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.571+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 >> v1:172.21.15.113:6789/0 conn(0x7f2e1815e2d0 legacy=0x7f2e1815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e0ffff640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e040039b0 0x7f2e04003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e0ffff640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e0ffff640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e040051c0 0x7f2e0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180cde00 con 0x7f2e04004800 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0400c090 con 0x7f2e040039b0 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0400c290 con 0x7f2e040051c0 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e040039b0 0x7f2e04003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e040051c0 0x7f2e0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1c93d640 1 -- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e040051c0 msgr2=0x7f2e0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e040051c0 0x7f2e0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1c93d640 1 -- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e040039b0 msgr2=0x7f2e04003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e040039b0 0x7f2e04003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1c93d640 1 -- 172.21.15.113:0/2141818991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e04003520 con 0x7f2e04004800 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.572+0000 7f2e1d93f640 1 -- 172.21.15.113:0/2141818991 reap_dead start 2024-04-25T00:01:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e04004c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2e08001750 tx=0x7f2e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d01df2177de431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08001370 con 0x7f2e04004800 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e080052d0 con 0x7f2e04004800 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e0ffff640 1 -- 172.21.15.113:0/2141818991 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0802abc0 con 0x7f2e04004800 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/2141818991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 msgr2=0x7f2e04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e04004c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2e08001750 tx=0x7f2e04005830 comp rx=0 tx=0).stop 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/2141818991 shutdown_connections 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/2141818991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e04004c80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/2141818991 >> 172.21.15.113:0/2141818991 conn(0x7f2e1806b0f0 msgr2=0x7f2e1806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/2141818991 shutdown_connections 2024-04-25T00:01:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.573+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/2141818991 wait complete. 2024-04-25T00:01:01.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 Processor -- start 2024-04-25T00:01:01.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 -- start start 2024-04-25T00:01:01.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e1814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e1814db00 0x7f2e1814b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e18142a40 0x7f2e180b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1815e830 con 0x7f2e04004800 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1815e6b0 con 0x7f2e18142a40 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1f3a2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1815e9b0 con 0x7f2e1814db00 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1d13e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e1814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1d13e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e1814eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38688/0 (socket says 172.21.15.113:38688) 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.575+0000 7f2e1d13e640 1 -- 172.21.15.113:0/3100786310 learned_addr learned my addr 172.21.15.113:0/3100786310 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e1814db00 0x7f2e1814b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e18142a40 0x7f2e180b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d13e640 1 -- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e1814db00 msgr2=0x7f2e1814b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e1814db00 0x7f2e1814b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d13e640 1 -- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e18142a40 msgr2=0x7f2e180b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e18142a40 0x7f2e180b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d13e640 1 -- 172.21.15.113:0/3100786310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1814a250 con 0x7f2e04004800 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e18142a40 0x7f2e180b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:01.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e1814eb10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2e140072f0 tx=0x7f2e140077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e0dffb640 1 -- 172.21.15.113:0/3100786310 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1401a070 con 0x7f2e04004800 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e0dffb640 1 -- 172.21.15.113:0/3100786310 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1400f960 con 0x7f2e04004800 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1814a530 con 0x7f2e04004800 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e0dffb640 1 -- 172.21.15.113:0/3100786310 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400fd10 con 0x7f2e04004800 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.576+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1814aaf0 con 0x7f2e04004800 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.577+0000 7f2e0dffb640 1 -- 172.21.15.113:0/3100786310 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e1401d070 con 0x7f2e04004800 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.578+0000 7f2e0dffb640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.578+0000 7f2e0dffb640 1 -- 172.21.15.113:0/3100786310 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f2e1408a800 con 0x7f2e04004800 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.578+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.578+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e08001750 tx=0x7f2e0802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.579+0000 7f2e22192740 1 Processor -- start 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.579+0000 7f2e22192740 1 -- start start 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8b6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e22192740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8b0ec0 0x56378c8b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e22192740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56378c819220 con 0x56378c8b0ec0 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56378c7c18b0 con 0x56378c8b5170 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56378c8151e0 con 0x56378c8b6030 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8b0ec0 0x56378c8b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8b0ec0 0x56378c8b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38692/0 (socket says 172.21.15.113:38692) 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 -- 172.21.15.113:0/360989701 learned_addr learned my addr 172.21.15.113:0/360989701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 -- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 msgr2=0x56378c8b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 -- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 msgr2=0x56378c8b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 -- 172.21.15.113:0/360989701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378c5f9690 con 0x56378c8b0ec0 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8b0ec0 0x56378c8b4c30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e08001840 tx=0x7f2e0802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd72c4a4fdd01636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e02ffd640 1 -- 172.21.15.113:0/360989701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08037040 con 0x56378c8b0ec0 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e02ffd640 1 -- 172.21.15.113:0/360989701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e08005410 con 0x56378c8b0ec0 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.580+0000 7f2e02ffd640 1 -- 172.21.15.113:0/360989701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080067d0 con 0x56378c8b0ec0 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 -- 172.21.15.113:0/360989701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8b0ec0 msgr2=0x56378c8b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8b0ec0 0x56378c8b4c30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e08001840 tx=0x7f2e0802c000 comp rx=0 tx=0).stop 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 -- 172.21.15.113:0/360989701 shutdown_connections 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8b0ec0 0x56378c8b4c30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 --2- 172.21.15.113:0/360989701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 -- 172.21.15.113:0/360989701 >> 172.21.15.113:0/360989701 conn(0x56378c8b3130 msgr2=0x56378c8fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 -- 172.21.15.113:0/360989701 shutdown_connections 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 -- 172.21.15.113:0/360989701 wait complete. 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.581+0000 7f2e22192740 1 -- start start 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 1 Processor -- start 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8d6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8d6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56378c8e1bc0 con 0x56378c8fdc90 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56378c7d4e40 con 0x56378c8b5170 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56378c8e1040 con 0x56378c8b6030 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e22192740 10 client.0 _ll_register_callbacks cb 0x56378c7d9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.582+0000 7f2e1d93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38696/0 (socket says 172.21.15.113:38696) 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 -- 172.21.15.113:0/3291546561 learned_addr learned my addr 172.21.15.113:0/3291546561 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8d6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8d6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 -- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 msgr2=0x56378c8d6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8d6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 -- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 msgr2=0x56378c8d6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8d6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 -- 172.21.15.113:0/3291546561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378c659e20 con 0x56378c8fdc90 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8cfa40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2e180cddb0 tx=0x7f2e10004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8bf1b6072556c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2df7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2df7fff640 1 -- 172.21.15.113:0/3291546561 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10007c40 con 0x56378c8fdc90 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2df7fff640 1 -- 172.21.15.113:0/3291546561 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f2e10007de0 con 0x56378c8fdc90 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e08005410 con 0x56378c8fdc90 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2df7fff640 1 -- 172.21.15.113:0/3291546561 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2e10026050 con 0x56378c8fdc90 2024-04-25T00:01:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 10 client.5814 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56378c4ecf80 con 0x56378c8fdc90 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2df7fff640 1 -- 172.21.15.113:0/3291546561 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e08005410 con 0x56378c8fdc90 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2df7fff640 1 -- 172.21.15.113:0/3291546561 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f2e10029020 con 0x56378c8fdc90 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 10 client.5814 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 10 client.5814 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 20 client.5814 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d0020 0x56378c8d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 4 client.5814 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.583+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56378c7d6cc0 con 0x56378c8d0020 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.584+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d0020 0x56378c8d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.584+0000 7f2df7fff640 10 client.5814 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.584+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d0020 0x56378c8d0420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b323d7cc9fcccb8e server_cookie=c5c291cd3add6799 in_seq=0 out_seq=0 2024-04-25T00:01:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.584+0000 7f2df7fff640 1 -- 172.21.15.113:0/3291546561 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56378c7d6cc0 con 0x56378c8d0020 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.584+0000 7f2df7fff640 10 client.5814 handle_command_reply: tid=0 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.584+0000 7f2e22192740 1 client.5814 shutdown 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 msgr2=0x56378c8cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8cfa40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2e180cddb0 tx=0x7f2e10004700 comp rx=0 tx=0).stop 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 shutdown_connections 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d0020 0x56378c8d0420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8cfa40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b6030 0x56378c8d6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2df7fff640 0 client.5814 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 --2- 172.21.15.113:0/3291546561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b5170 0x56378c8d6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 >> 172.21.15.113:0/3291546561 conn(0x56378c8b0ec0 msgr2=0x56378c8b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 shutdown_connections 2024-04-25T00:01:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 1 -- 172.21.15.113:0/3291546561 wait complete. 2024-04-25T00:01:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.585+0000 7f2e22192740 20 client.5814 trim_cache size 0 max 16384 2024-04-25T00:01:01.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 Processor -- start 2024-04-25T00:01:01.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 -- start start 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c8d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c8b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181fee80 con 0x56378c8fdc90 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181fdeb0 con 0x56378c8b6030 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.603+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e180b1da0 con 0x56378c8b5170 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8fe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38710/0 (socket says 172.21.15.113:38710) 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 -- 172.21.15.113:0/1332014143 learned_addr learned my addr 172.21.15.113:0/1332014143 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c8d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c8b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 -- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 msgr2=0x56378c8d63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c8d63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 -- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 msgr2=0x56378c8b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c8b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 -- 172.21.15.113:0/1332014143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378c9204d0 con 0x56378c8fdc90 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8fe360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2e100095d0 tx=0x7f2e10018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a96155f7c547637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e027fc640 1 -- 172.21.15.113:0/1332014143 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10007b00 con 0x56378c8fdc90 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e027fc640 1 -- 172.21.15.113:0/1332014143 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1002e020 con 0x56378c8fdc90 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.604+0000 7f2e027fc640 1 -- 172.21.15.113:0/1332014143 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10017810 con 0x56378c8fdc90 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 -- 172.21.15.113:0/1332014143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 msgr2=0x56378c8fe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8fe360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2e100095d0 tx=0x7f2e10018000 comp rx=0 tx=0).stop 2024-04-25T00:01:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 -- 172.21.15.113:0/1332014143 shutdown_connections 2024-04-25T00:01:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c8b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8fe360 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 --2- 172.21.15.113:0/1332014143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c8d63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 -- 172.21.15.113:0/1332014143 >> 172.21.15.113:0/1332014143 conn(0x56378c8b0ec0 msgr2=0x56378c916370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 -- 172.21.15.113:0/1332014143 shutdown_connections 2024-04-25T00:01:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 -- 172.21.15.113:0/1332014143 wait complete. 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 -- start start 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.605+0000 7f2e22192740 1 Processor -- start 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c979e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e22192740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c97d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e22192740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e22192740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56378c8fc010 con 0x56378c8fdc90 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181fd860 con 0x56378c8b6030 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e22192740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56378c8b4fb0 con 0x56378c8b5170 2024-04-25T00:01:01.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e22192740 10 client.0 _ll_register_callbacks cb 0x7f2e10009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e1d93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e1d93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38716/0 (socket says 172.21.15.113:38716) 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e1d93f640 1 -- 172.21.15.113:0/2509419608 learned_addr learned my addr 172.21.15.113:0/2509419608 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e1c93d640 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c97d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.606+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c979e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e1d93f640 1 -- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 msgr2=0x56378c979e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c979e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e1d93f640 1 -- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 msgr2=0x56378c97d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c97d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e1d93f640 1 -- 172.21.15.113:0/2509419608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378c8d3770 con 0x56378c8fdc90 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e1d93f640 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8d0d30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2e10030040 tx=0x7f2e10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42a2625a5b803ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2df7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:01.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2df7fff640 1 -- 172.21.15.113:0/2509419608 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10017ce0 con 0x56378c8fdc90 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2df7fff640 1 -- 172.21.15.113:0/2509419608 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f2e10033460 con 0x56378c8fdc90 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1002e020 con 0x56378c8fdc90 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2df7fff640 1 -- 172.21.15.113:0/2509419608 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2e10040050 con 0x56378c8fdc90 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 10 client.5820 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56378c9200e0 con 0x56378c8fdc90 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2df7fff640 1 -- 172.21.15.113:0/2509419608 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1002e020 con 0x56378c8fdc90 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2df7fff640 1 -- 172.21.15.113:0/2509419608 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f2e10043020 con 0x56378c8fdc90 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 10 client.5820 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 10 client.5820 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 20 client.5820 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d1270 0x56378c91edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 4 client.5820 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.607+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) -- 0x56378c826760 con 0x56378c8d1270 2024-04-25T00:01:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d1270 0x56378c91edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2df7fff640 10 client.5820 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e1d13e640 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d1270 0x56378c91edb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde496f15d550ce0 server_cookie=ae7267947815ae8e in_seq=0 out_seq=0 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2df7fff640 1 -- 172.21.15.113:0/2509419608 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56378c826760 con 0x56378c8d1270 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2df7fff640 10 client.5820 handle_command_reply: tid=0 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 client.5820 shutdown 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 msgr2=0x56378c8d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8d0d30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2e10030040 tx=0x7f2e10004ab0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 shutdown_connections 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56378c8d1270 0x56378c91edb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c8b6030 0x56378c97d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56378c8fdc90 0x56378c8d0d30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2df7fff640 0 client.5820 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 --2- 172.21.15.113:0/2509419608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56378c8b5170 0x56378c979e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.608+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 >> 172.21.15.113:0/2509419608 conn(0x56378c8b0ec0 msgr2=0x56378c8d6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 shutdown_connections 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e22192740 1 -- 172.21.15.113:0/2509419608 wait complete. 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e22192740 20 client.5820 trim_cache size 0 max 16384 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "78037e16-5de2-463a-8abe-1b90b8373868", 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:01.615 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:01.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dfc06be30 msgr2=0x7f2dfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e08001750 tx=0x7f2e0802e000 comp rx=0 tx=0).stop 2024-04-25T00:01:01.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 msgr2=0x7f2e1814eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e1814eb10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2e140072f0 tx=0x7f2e140077b0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 shutdown_connections 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e18142a40 0x7f2e180b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e1814db00 0x7f2e1814b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 --2- 172.21.15.113:0/3100786310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e04004800 0x7f2e1814eb10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.609+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 >> 172.21.15.113:0/3100786310 conn(0x7f2e1806b0f0 msgr2=0x7f2e180729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.610+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 shutdown_connections 2024-04-25T00:01:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.610+0000 7f2e1f3a2640 1 -- 172.21.15.113:0/3100786310 wait complete. 2024-04-25T00:01:01.620 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '78037e16-5de2-463a-8abe-1b90b8373868', 'mode': 'asynchronous'}' 2024-04-25T00:01:01.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:01.621 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T00:01:01.636 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T00:01:01.636 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:01.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.773+0000 7f29d784c640 1 Processor -- start 2024-04-25T00:01:01.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.773+0000 7f29d784c640 1 -- start start 2024-04-25T00:01:01.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d784c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29d006e6e0 0x7f29d00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d784c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d0144cf0 con 0x7f29d0144910 2024-04-25T00:01:01.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d784c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d00b2f30 con 0x7f29d00b3fd0 2024-04-25T00:01:01.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d784c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f29d00b3210 con 0x7f29d006e6e0 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d55e8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f29d0144910 0x7f29d00b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33666/0 (socket says 172.21.15.113:33666) 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d55e8640 1 -- 172.21.15.113:0/2686145056 learned_addr learned my addr 172.21.15.113:0/2686145056 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d5de9640 1 --2- 172.21.15.113:0/2686145056 >> v2:172.21.15.113:3300/0 conn(0x7f29d006e6e0 0x7f29d00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29c7fff640 1 -- 172.21.15.113:0/2686145056 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256060124 0 0) 0x7f29d0144cf0 con 0x7f29d0144910 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29c7fff640 1 -- 172.21.15.113:0/2686145056 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29bc003520 con 0x7f29d0144910 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d5de9640 1 -- 172.21.15.113:0/2686145056 >> v1:172.21.15.178:6789/0 conn(0x7f29d00b3fd0 legacy=0x7f29d006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d5de9640 1 -- 172.21.15.113:0/2686145056 >> v1:172.21.15.113:6789/0 conn(0x7f29d0144910 legacy=0x7f29d00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.774+0000 7f29d5de9640 1 -- 172.21.15.113:0/2686145056 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d00b34f0 con 0x7f29d006e6e0 2024-04-25T00:01:01.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.775+0000 7f29d5de9640 1 --2- 172.21.15.113:0/2686145056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d006e6e0 0x7f29d00cd500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009dc0 tx=0x7f29c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a624f5d0f093d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.775+0000 7f29c7fff640 1 -- 172.21.15.113:0/2686145056 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8007740 con 0x7f29d006e6e0 2024-04-25T00:01:01.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.775+0000 7f29c7fff640 1 -- 172.21.15.113:0/2686145056 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c80078e0 con 0x7f29d006e6e0 2024-04-25T00:01:01.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.775+0000 7f29c7fff640 1 -- 172.21.15.113:0/2686145056 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8007cb0 con 0x7f29d006e6e0 2024-04-25T00:01:01.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.776+0000 7f29d784c640 1 -- 172.21.15.113:0/2686145056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d006e6e0 msgr2=0x7f29d00cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.776+0000 7f29d784c640 1 --2- 172.21.15.113:0/2686145056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d006e6e0 0x7f29d00cd500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009dc0 tx=0x7f29c802a730 comp rx=0 tx=0).stop 2024-04-25T00:01:01.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.776+0000 7f29d784c640 1 -- 172.21.15.113:0/2686145056 shutdown_connections 2024-04-25T00:01:01.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.776+0000 7f29d784c640 1 --2- 172.21.15.113:0/2686145056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d006e6e0 0x7f29d00cd500 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.776+0000 7f29d784c640 1 -- 172.21.15.113:0/2686145056 >> 172.21.15.113:0/2686145056 conn(0x7f29d00b1a50 msgr2=0x7f29d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.776+0000 7f29d784c640 1 -- 172.21.15.113:0/2686145056 shutdown_connections 2024-04-25T00:01:01.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.776+0000 7f29d784c640 1 -- 172.21.15.113:0/2686145056 wait complete. 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.777+0000 7f29d784c640 1 Processor -- start 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.777+0000 7f29d784c640 1 -- start start 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d784c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29d006e6e0 0x7f29d015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d784c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d00b3fd0 0x7f29d012fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d784c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0144910 0x7f29d0136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d784c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f29d00b4da0 con 0x7f29d00b3fd0 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d784c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29d00b4440 con 0x7f29d0144910 2024-04-25T00:01:01.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d784c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f29d00b4f20 con 0x7f29d006e6e0 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d00b3fd0 0x7f29d012fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d00b3fd0 0x7f29d012fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38736/0 (socket says 172.21.15.113:38736) 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 -- 172.21.15.113:0/2506963268 learned_addr learned my addr 172.21.15.113:0/2506963268 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d5de9640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0144910 0x7f29d0136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d55e8640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29d006e6e0 0x7f29d015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 -- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29d006e6e0 msgr2=0x7f29d015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29d006e6e0 0x7f29d015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 -- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0144910 msgr2=0x7f29d0136fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0144910 0x7f29d0136fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 -- 172.21.15.113:0/2506963268 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d00b08b0 con 0x7f29d00b3fd0 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d55e8640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29d006e6e0 0x7f29d015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d4de7640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d00b3fd0 0x7f29d012fec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c430 tx=0x7f29c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29c5ffb640 1 -- 172.21.15.113:0/2506963268 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c00052d0 con 0x7f29d00b3fd0 2024-04-25T00:01:01.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29c5ffb640 1 -- 172.21.15.113:0/2506963268 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c0005490 con 0x7f29d00b3fd0 2024-04-25T00:01:01.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d00b05d0 con 0x7f29d00b3fd0 2024-04-25T00:01:01.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.778+0000 7f29c5ffb640 1 -- 172.21.15.113:0/2506963268 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0007260 con 0x7f29d00b3fd0 2024-04-25T00:01:01.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.779+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d012f6c0 con 0x7f29d00b3fd0 2024-04-25T00:01:01.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.780+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d00691a0 con 0x7f29d00b3fd0 2024-04-25T00:01:01.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.780+0000 7f29c5ffb640 1 -- 172.21.15.113:0/2506963268 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29c0007400 con 0x7f29d00b3fd0 2024-04-25T00:01:01.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.780+0000 7f29c5ffb640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29b406be30 0x7f29b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.780+0000 7f29c5ffb640 1 -- 172.21.15.113:0/2506963268 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f29c008a3c0 con 0x7f29d00b3fd0 2024-04-25T00:01:01.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.780+0000 7f29d55e8640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29b406be30 0x7f29b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.781+0000 7f29d55e8640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29b406be30 0x7f29b406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0011e0 tx=0x7f29cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.784+0000 7f29c5ffb640 1 -- 172.21.15.113:0/2506963268 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f29c0058d30 con 0x7f29d00b3fd0 2024-04-25T00:01:01.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.994+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29d015f420 con 0x7f29d00b3fd0 2024-04-25T00:01:01.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.994+0000 7f29c5ffb640 1 -- 172.21.15.113:0/2506963268 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7f29c0058630 con 0x7f29d00b3fd0 2024-04-25T00:01:01.995 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:01.995 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:01.995 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29b406be30 msgr2=0x7f29b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29b406be30 0x7f29b406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0011e0 tx=0x7f29cc009040 comp rx=0 tx=0).stop 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d00b3fd0 msgr2=0x7f29d012fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d00b3fd0 0x7f29d012fec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c430 tx=0x7f29c000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 shutdown_connections 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29b406be30 0x7f29b406e2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0144910 0x7f29d0136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d00b3fd0 0x7f29d012fec0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 --2- 172.21.15.113:0/2506963268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29d006e6e0 0x7f29d015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 >> 172.21.15.113:0/2506963268 conn(0x7f29d00b1a50 msgr2=0x7f29d0138290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 shutdown_connections 2024-04-25T00:01:01.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:01.996+0000 7f29d784c640 1 -- 172.21.15.113:0/2506963268 wait complete. 2024-04-25T00:01:02.007 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T00:01:02.115 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:02.115 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:02.115 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:02.124 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:02.124 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:02.124 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:02.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:02.140 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:01:02.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:01:02.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:01:02.205 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879507 s, 119 MB/s 2024-04-25T00:01:02.205 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T00:01:02.205 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:02.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.342+0000 7fa81d912640 1 Processor -- start 2024-04-25T00:01:02.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.342+0000 7fa81d912640 1 -- start start 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.343+0000 7fa81d912640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8181453c0 0x7fa8180b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.343+0000 7fa81d912640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8180cfad0 con 0x7fa81806e6e0 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.343+0000 7fa81d912640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8180cf230 con 0x7fa818144d60 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.343+0000 7fa81d912640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa818145c40 con 0x7fa8181453c0 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.343+0000 7fa8167fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa81806e6e0 0x7fa818071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33670/0 (socket says 172.21.15.113:33670) 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.343+0000 7fa8167fc640 1 -- 172.21.15.113:0/797363366 learned_addr learned my addr 172.21.15.113:0/797363366 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.343+0000 7fa8177fe640 1 --2- 172.21.15.113:0/797363366 >> v2:172.21.15.113:3300/0 conn(0x7fa8181453c0 0x7fa8180b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa815ffb640 1 -- 172.21.15.113:0/797363366 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522323463 0 0) 0x7fa8180cfad0 con 0x7fa81806e6e0 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa815ffb640 1 -- 172.21.15.113:0/797363366 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa800003520 con 0x7fa81806e6e0 2024-04-25T00:01:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa8177fe640 1 -- 172.21.15.113:0/797363366 >> v1:172.21.15.178:6789/0 conn(0x7fa818144d60 legacy=0x7fa81806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa8177fe640 1 -- 172.21.15.113:0/797363366 >> v1:172.21.15.113:6789/0 conn(0x7fa81806e6e0 legacy=0x7fa818071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa8177fe640 1 -- 172.21.15.113:0/797363366 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8180cf510 con 0x7fa8181453c0 2024-04-25T00:01:02.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa8177fe640 1 --2- 172.21.15.113:0/797363366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8181453c0 0x7fa8180b5390 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa8100097a0 tx=0x7fa81002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2fbfdbbefe2ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa815ffb640 1 -- 172.21.15.113:0/797363366 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810037070 con 0x7fa8181453c0 2024-04-25T00:01:02.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa815ffb640 1 -- 172.21.15.113:0/797363366 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa810032440 con 0x7fa8181453c0 2024-04-25T00:01:02.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.344+0000 7fa815ffb640 1 -- 172.21.15.113:0/797363366 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81002ae60 con 0x7fa8181453c0 2024-04-25T00:01:02.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.345+0000 7fa81d912640 1 -- 172.21.15.113:0/797363366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8181453c0 msgr2=0x7fa8180b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.345+0000 7fa81d912640 1 --2- 172.21.15.113:0/797363366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8181453c0 0x7fa8180b5390 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa8100097a0 tx=0x7fa81002a110 comp rx=0 tx=0).stop 2024-04-25T00:01:02.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.345+0000 7fa81d912640 1 -- 172.21.15.113:0/797363366 shutdown_connections 2024-04-25T00:01:02.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.345+0000 7fa81d912640 1 --2- 172.21.15.113:0/797363366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8181453c0 0x7fa8180b5390 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.345+0000 7fa81d912640 1 -- 172.21.15.113:0/797363366 >> 172.21.15.113:0/797363366 conn(0x7fa8180b1a50 msgr2=0x7fa8180b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.345+0000 7fa81d912640 1 -- 172.21.15.113:0/797363366 shutdown_connections 2024-04-25T00:01:02.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.345+0000 7fa81d912640 1 -- 172.21.15.113:0/797363366 wait complete. 2024-04-25T00:01:02.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.346+0000 7fa81d912640 1 Processor -- start 2024-04-25T00:01:02.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.346+0000 7fa81d912640 1 -- start start 2024-04-25T00:01:02.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa81d912640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81806e6e0 0x7fa81815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa81d912640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa818144d60 0x7fa8181f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa81d912640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8181453c0 0x7fa81806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa81d912640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa818145140 con 0x7fa81806e6e0 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa81d912640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa818070e10 con 0x7fa8181453c0 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa81d912640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa818071a40 con 0x7fa818144d60 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa816ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81806e6e0 0x7fa81815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa816ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81806e6e0 0x7fa81815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38740/0 (socket says 172.21.15.113:38740) 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa816ffd640 1 -- 172.21.15.113:0/2732669299 learned_addr learned my addr 172.21.15.113:0/2732669299 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa8177fe640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8181453c0 0x7fa81806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.347+0000 7fa8167fc640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa818144d60 0x7fa8181f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa816ffd640 1 -- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa818144d60 msgr2=0x7fa8181f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa816ffd640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa818144d60 0x7fa8181f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa816ffd640 1 -- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8181453c0 msgr2=0x7fa81806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa816ffd640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8181453c0 0x7fa81806d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa816ffd640 1 -- 172.21.15.113:0/2732669299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81815d690 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa816ffd640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81806e6e0 0x7fa81815b800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c7c0 tx=0x7fa80800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa7fffff640 1 -- 172.21.15.113:0/2732669299 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808016020 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa7fffff640 1 -- 172.21.15.113:0/2732669299 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa808006d50 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81815e390 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa7fffff640 1 -- 172.21.15.113:0/2732669299 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808003bb0 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.348+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81815e950 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.349+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8180691a0 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.349+0000 7fa7fffff640 1 -- 172.21.15.113:0/2732669299 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa80800a850 con 0x7fa81806e6e0 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.350+0000 7fa7fffff640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80406be30 0x7fa80406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.350+0000 7fa7fffff640 1 -- 172.21.15.113:0/2732669299 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa80808ab60 con 0x7fa81806e6e0 2024-04-25T00:01:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.350+0000 7fa8167fc640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80406be30 0x7fa80406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.350+0000 7fa8167fc640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80406be30 0x7fa80406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa818140f20 tx=0x7fa80c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.354+0000 7fa7fffff640 1 -- 172.21.15.113:0/2732669299 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa808020020 con 0x7fa81806e6e0 2024-04-25T00:01:02.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.562+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8180b2290 con 0x7fa81806e6e0 2024-04-25T00:01:02.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.562+0000 7fa7fffff640 1 -- 172.21.15.113:0/2732669299 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5190 (secure 0 0 0) 0x7fa808019310 con 0x7fa81806e6e0 2024-04-25T00:01:02.563 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:02.563 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:00:51.512287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:02.563 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-25T00:01:02.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80406be30 msgr2=0x7fa80406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80406be30 0x7fa80406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa818140f20 tx=0x7fa80c009040 comp rx=0 tx=0).stop 2024-04-25T00:01:02.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81806e6e0 msgr2=0x7fa81815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81806e6e0 0x7fa81815b800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c7c0 tx=0x7fa80800cc80 comp rx=0 tx=0).stop 2024-04-25T00:01:02.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 shutdown_connections 2024-04-25T00:01:02.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa80406be30 0x7fa80406e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8181453c0 0x7fa81806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa818144d60 0x7fa8181f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 --2- 172.21.15.113:0/2732669299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa81806e6e0 0x7fa81815b800 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.564+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 >> 172.21.15.113:0/2732669299 conn(0x7fa8180b1a50 msgr2=0x7fa8180ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.565+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 shutdown_connections 2024-04-25T00:01:02.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.565+0000 7fa81d912640 1 -- 172.21.15.113:0/2732669299 wait complete. 2024-04-25T00:01:02.575 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T00:01:02.683 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:02.683 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:02.684 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:02.692 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:02.693 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:01:02.693 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:01:02.693 DEBUG:teuthology.orchestra.run.smithi113:> import stat 2024-04-25T00:01:02.693 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:01:02.693 DEBUG:teuthology.orchestra.run.smithi113:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T00:01:02.693 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:01:02.757 INFO:teuthology.orchestra.run.smithi113.stdout:1099511630412 2024-04-25T00:01:02.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:02.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T00:01:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.905+0000 7f98dcfce640 1 Processor -- start 2024-04-25T00:01:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.905+0000 7f98dcfce640 1 -- start start 2024-04-25T00:01:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98dcfce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f98d815e2d0 0x7f98d815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98dcfce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f98d80b3080 0x7f98d80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98dcfce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d815f4f0 con 0x7f98d815e6b0 2024-04-25T00:01:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98dcfce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f98d80cde00 con 0x7f98d80b3080 2024-04-25T00:01:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98dcfce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f98d815f7d0 con 0x7f98d815e2d0 2024-04-25T00:01:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d5d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f98d80b3080 0x7f98d80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d6d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f98d815e6b0 0x7f98d8071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33680/0 (socket says 172.21.15.113:33680) 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d6d9d640 1 -- 172.21.15.113:0/805784587 learned_addr learned my addr 172.21.15.113:0/805784587 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d659c640 1 --2- 172.21.15.113:0/805784587 >> v2:172.21.15.178:3300/0 conn(0x7f98d815e2d0 0x7f98d815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d5d9b640 1 -- 172.21.15.113:0/805784587 >> v2:172.21.15.178:3300/0 conn(0x7f98d815e2d0 msgr2=0x7f98d815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/805784587 >> v2:172.21.15.178:3300/0 conn(0x7f98d815e2d0 0x7f98d815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d5d9b640 1 -- 172.21.15.113:0/805784587 >> v1:172.21.15.113:6789/0 conn(0x7f98d815e6b0 legacy=0x7f98d8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d559a640 1 -- 172.21.15.113:0/805784587 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 495281111 0 0) 0x7f98d815f4f0 con 0x7f98d815e6b0 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.906+0000 7f98d5d9b640 1 -- 172.21.15.113:0/805784587 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d815fab0 con 0x7f98d80b3080 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.907+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/805784587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b4330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f98d80b2d40 tx=0x7f98cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb87d0ec152eebf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.907+0000 7f98d559a640 1 -- 172.21.15.113:0/805784587 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc037070 con 0x7f98d80b3080 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.907+0000 7f98d559a640 1 -- 172.21.15.113:0/805784587 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc032440 con 0x7f98d80b3080 2024-04-25T00:01:02.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.907+0000 7f98d559a640 1 -- 172.21.15.113:0/805784587 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc032810 con 0x7f98d80b3080 2024-04-25T00:01:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 -- 172.21.15.113:0/805784587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 msgr2=0x7f98d80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 --2- 172.21.15.113:0/805784587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b4330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f98d80b2d40 tx=0x7f98cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 -- 172.21.15.113:0/805784587 shutdown_connections 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 --2- 172.21.15.113:0/805784587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b4330 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 --2- 172.21.15.113:0/805784587 >> v2:172.21.15.178:3300/0 conn(0x7f98d815e2d0 0x7f98d815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 -- 172.21.15.113:0/805784587 >> 172.21.15.113:0/805784587 conn(0x7f98d806b0f0 msgr2=0x7f98d806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 -- 172.21.15.113:0/805784587 shutdown_connections 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.908+0000 7f98dcfce640 1 -- 172.21.15.113:0/805784587 wait complete. 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.909+0000 7f98dcfce640 1 Processor -- start 2024-04-25T00:01:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.909+0000 7f98dcfce640 1 -- start start 2024-04-25T00:01:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98dcfce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98dcfce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98d815e2d0 0x7f98d80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98dcfce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d815e6b0 0x7f98d8151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98dcfce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98d815f2b0 con 0x7f98d80b3080 2024-04-25T00:01:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98dcfce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98d815f130 con 0x7f98d815e6b0 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98dcfce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80cf2a0 con 0x7f98d815e2d0 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38754/0 (socket says 172.21.15.113:38754) 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 -- 172.21.15.113:0/1084170252 learned_addr learned my addr 172.21.15.113:0/1084170252 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98d815e2d0 0x7f98d80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d6d9d640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d815e6b0 0x7f98d8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 -- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98d815e2d0 msgr2=0x7f98d80ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98d815e2d0 0x7f98d80ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 -- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d815e6b0 msgr2=0x7f98d8151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d815e6b0 0x7f98d8151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d659c640 1 -- 172.21.15.113:0/1084170252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d80afee0 con 0x7f98d80b3080 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.910+0000 7f98d6d9d640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d815e6b0 0x7f98d8151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.911+0000 7f98d659c640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b2840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009db0 tx=0x7f98c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.911+0000 7f98c6ffd640 1 -- 172.21.15.113:0/1084170252 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8018020 con 0x7f98d80b3080 2024-04-25T00:01:02.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.911+0000 7f98c6ffd640 1 -- 172.21.15.113:0/1084170252 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c8014480 con 0x7f98d80b3080 2024-04-25T00:01:02.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.911+0000 7f98c6ffd640 1 -- 172.21.15.113:0/1084170252 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8014850 con 0x7f98d80b3080 2024-04-25T00:01:02.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.911+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d80b01c0 con 0x7f98d80b3080 2024-04-25T00:01:02.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.911+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d80b0780 con 0x7f98d80b3080 2024-04-25T00:01:02.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.912+0000 7f98c6ffd640 1 -- 172.21.15.113:0/1084170252 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98c8005d10 con 0x7f98d80b3080 2024-04-25T00:01:02.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.912+0000 7f98c6ffd640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f98b806be30 0x7f98b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.913+0000 7f98c6ffd640 1 -- 172.21.15.113:0/1084170252 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f98c808b9c0 con 0x7f98d80b3080 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.913+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f98b806be30 0x7f98b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.913+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f98b806be30 0x7f98b806e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98cc034000 tx=0x7f98cc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.913+0000 7f98dfdb9740 1 Processor -- start 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98dfdb9740 1 -- start start 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c46950 0x560c11bf92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 0x560c11c28df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c29330 0x560c11bfe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560c11b62670 con 0x560c11c29330 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560c11b0a970 con 0x560c11c46950 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560c11b5e630 con 0x560c11c29a70 2024-04-25T00:01:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c29330 0x560c11bfe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c29330 0x560c11bfe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38758/0 (socket says 172.21.15.113:38758) 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.914+0000 7f98d6d9d640 1 -- 172.21.15.113:0/2852725381 learned_addr learned my addr 172.21.15.113:0/2852725381 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98d6d9d640 1 -- 172.21.15.113:0/2852725381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 msgr2=0x560c11c28df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98d6d9d640 1 --2- 172.21.15.113:0/2852725381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 0x560c11c28df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98d6d9d640 1 -- 172.21.15.113:0/2852725381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c46950 msgr2=0x560c11bf92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98d6d9d640 1 --2- 172.21.15.113:0/2852725381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c46950 0x560c11bf92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98d6d9d640 1 -- 172.21.15.113:0/2852725381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c11b2e400 con 0x560c11c29330 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98d6d9d640 1 --2- 172.21.15.113:0/2852725381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c29330 0x560c11bfe140 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f98d0001790 tx=0x7f98d0001c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7872d6c9969a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98b7fff640 1 -- 172.21.15.113:0/2852725381 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00074e0 con 0x560c11c29330 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98b7fff640 1 -- 172.21.15.113:0/2852725381 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d00047e0 con 0x560c11c29330 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.915+0000 7f98b7fff640 1 -- 172.21.15.113:0/2852725381 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0004b90 con 0x560c11c29330 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 -- 172.21.15.113:0/2852725381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c29330 msgr2=0x560c11bfe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/2852725381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c29330 0x560c11bfe140 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f98d0001790 tx=0x7f98d0001c50 comp rx=0 tx=0).stop 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 -- 172.21.15.113:0/2852725381 shutdown_connections 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/2852725381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c29330 0x560c11bfe140 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/2852725381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 0x560c11c28df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/2852725381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c46950 0x560c11bf92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 -- 172.21.15.113:0/2852725381 >> 172.21.15.113:0/2852725381 conn(0x560c11bfb550 msgr2=0x560c11c43df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 -- 172.21.15.113:0/2852725381 shutdown_connections 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 -- 172.21.15.113:0/2852725381 wait complete. 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 -- start start 2024-04-25T00:01:02.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.916+0000 7f98dfdb9740 1 Processor -- start 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 0x560c11c2b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c46950 0x560c11bfed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11bff260 0x560c11c16e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560c11bfd700 con 0x560c11c46950 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560c11b1e050 con 0x560c11c29a70 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560c11bfcaf0 con 0x560c11bff260 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98dfdb9740 10 client.0 _ll_register_callbacks cb 0x560c11875be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98d5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c46950 0x560c11bfed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98d5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c46950 0x560c11bfed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38766/0 (socket says 172.21.15.113:38766) 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98d5d9b640 1 -- 172.21.15.113:0/286137764 learned_addr learned my addr 172.21.15.113:0/286137764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98d659c640 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 0x560c11c2b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.917+0000 7f98d6d9d640 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11bff260 0x560c11c16e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98d5d9b640 1 -- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11bff260 msgr2=0x560c11c16e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11bff260 0x560c11c16e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98d5d9b640 1 -- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 msgr2=0x560c11c2b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 0x560c11c2b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98d5d9b640 1 -- 172.21.15.113:0/286137764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c11aefdc0 con 0x560c11c46950 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c46950 0x560c11bfed20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009890 tx=0x7f98cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d44364efd83d829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98b57fa640 1 -- 172.21.15.113:0/286137764 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc04f060 con 0x560c11c46950 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98b57fa640 1 -- 172.21.15.113:0/286137764 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f98cc032440 con 0x560c11c46950 2024-04-25T00:01:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d00047e0 con 0x560c11c46950 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98b57fa640 1 -- 172.21.15.113:0/286137764 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f98cc055050 con 0x560c11c46950 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 10 client.5853 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x560c118763f0 con 0x560c11c46950 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98b57fa640 1 -- 172.21.15.113:0/286137764 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d00047e0 con 0x560c11c46950 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98b57fa640 1 -- 172.21.15.113:0/286137764 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f98cc058020 con 0x560c11c46950 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 10 client.5853 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 10 client.5853 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 20 client.5853 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c28cb0 0x560c11c26db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 4 client.5853 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c11b6fbb0 con 0x560c11c28cb0 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.918+0000 7f98d659c640 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c28cb0 0x560c11c26db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98b57fa640 10 client.5853 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98d659c640 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c28cb0 0x560c11c26db0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86119c59c41b536f server_cookie=be67e713b54ad848 in_seq=0 out_seq=0 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98b57fa640 1 -- 172.21.15.113:0/286137764 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c11b6fbb0 con 0x560c11c28cb0 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98b57fa640 10 client.5853 handle_command_reply: tid=0 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 client.5853 shutdown 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c46950 msgr2=0x560c11bfed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c46950 0x560c11bfed20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009890 tx=0x7f98cc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 shutdown_connections 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c28cb0 0x560c11c26db0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11bff260 0x560c11c16e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11c46950 0x560c11bfed20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/286137764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 0x560c11c2b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98b57fa640 0 client.5853 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.919+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 >> 172.21.15.113:0/286137764 conn(0x560c11c29330 msgr2=0x560c11c43df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.920+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 shutdown_connections 2024-04-25T00:01:02.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.920+0000 7f98dfdb9740 1 -- 172.21.15.113:0/286137764 wait complete. 2024-04-25T00:01:02.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.920+0000 7f98dfdb9740 20 client.5853 trim_cache size 0 max 16384 2024-04-25T00:01:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.933+0000 7f98dfdb9740 1 Processor -- start 2024-04-25T00:01:02.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.933+0000 7f98dfdb9740 1 -- start start 2024-04-25T00:01:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29330 0x560c11c41200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11c23f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 0x560c11c24490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98d82006b0 con 0x560c11bff260 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81ff6f0 con 0x560c11c29330 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f98d81fea50 con 0x560c11c29a70 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11c23f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11c23f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38776/0 (socket says 172.21.15.113:38776) 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 -- 172.21.15.113:0/1725144829 learned_addr learned my addr 172.21.15.113:0/1725144829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 -- 172.21.15.113:0/1725144829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 msgr2=0x560c11c24490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 0x560c11c24490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 -- 172.21.15.113:0/1725144829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29330 msgr2=0x560c11c41200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d659c640 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29330 0x560c11c41200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29330 0x560c11c41200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 -- 172.21.15.113:0/1725144829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c119407d0 con 0x560c11bff260 2024-04-25T00:01:02.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11c23f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f98cc02a790 tx=0x7f98cc048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b619c819d4e83ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98b77fe640 1 -- 172.21.15.113:0/1725144829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc05e070 con 0x560c11bff260 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98b77fe640 1 -- 172.21.15.113:0/1725144829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc038040 con 0x560c11bff260 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.934+0000 7f98b77fe640 1 -- 172.21.15.113:0/1725144829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc037050 con 0x560c11bff260 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 -- 172.21.15.113:0/1725144829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 msgr2=0x560c11c23f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11c23f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f98cc02a790 tx=0x7f98cc048000 comp rx=0 tx=0).stop 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 -- 172.21.15.113:0/1725144829 shutdown_connections 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29a70 0x560c11c24490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11c23f50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/1725144829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29330 0x560c11c41200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 -- 172.21.15.113:0/1725144829 >> 172.21.15.113:0/1725144829 conn(0x560c11c2cdc0 msgr2=0x560c11c167c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 -- 172.21.15.113:0/1725144829 shutdown_connections 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 -- 172.21.15.113:0/1725144829 wait complete. 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 -- start start 2024-04-25T00:01:02.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.935+0000 7f98dfdb9740 1 Processor -- start 2024-04-25T00:01:02.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11cbdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29330 0x560c11c439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98dfdb9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 0x560c11c43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560c11c17000 con 0x560c11bff260 2024-04-25T00:01:02.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81ff0a0 con 0x560c11c29a70 2024-04-25T00:01:02.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98dfdb9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560c11c5da70 con 0x560c11c29330 2024-04-25T00:01:02.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98dfdb9740 10 client.0 _ll_register_callbacks cb 0x7f98cc035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11cbdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11cbdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38790/0 (socket says 172.21.15.113:38790) 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.936+0000 7f98d659c640 1 -- 172.21.15.113:0/761383242 learned_addr learned my addr 172.21.15.113:0/761383242 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98d659c640 1 -- 172.21.15.113:0/761383242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29330 msgr2=0x560c11c439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98d659c640 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29330 0x560c11c439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98d659c640 1 -- 172.21.15.113:0/761383242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 msgr2=0x560c11c43ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98d659c640 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 0x560c11c43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98d659c640 1 -- 172.21.15.113:0/761383242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c11cbf530 con 0x560c11bff260 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98d659c640 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11cbdf30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f98c8056090 tx=0x7f98c805a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707262d409a4c8de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98b57fa640 1 -- 172.21.15.113:0/761383242 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8060040 con 0x560c11bff260 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98b57fa640 1 -- 172.21.15.113:0/761383242 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f98c8011b40 con 0x560c11bff260 2024-04-25T00:01:02.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98cc038040 con 0x560c11bff260 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98b57fa640 1 -- 172.21.15.113:0/761383242 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f98c8011d40 con 0x560c11bff260 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 10 client.5859 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x560c11c47460 con 0x560c11bff260 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98b57fa640 1 -- 172.21.15.113:0/761383242 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc038040 con 0x560c11bff260 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98b57fa640 1 -- 172.21.15.113:0/761383242 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f98c8059080 con 0x560c11bff260 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 10 client.5859 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 10 client.5859 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 20 client.5859 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c46950 0x560c11c45a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 4 client.5859 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c11bfa390 con 0x560c11c46950 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.937+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c46950 0x560c11c45a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98b57fa640 10 client.5859 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98d5d9b640 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c46950 0x560c11c45a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59691a2820c5ab85 server_cookie=775e2a5230e23609 in_seq=0 out_seq=0 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98b57fa640 1 -- 172.21.15.113:0/761383242 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c11bfa390 con 0x560c11c46950 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98b57fa640 10 client.5859 handle_command_reply: tid=0 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 client.5859 shutdown 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 msgr2=0x560c11cbdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11cbdf30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f98c8056090 tx=0x7f98c805a490 comp rx=0 tx=0).stop 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 shutdown_connections 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x560c11c46950 0x560c11c45a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c11c29a70 0x560c11c43ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c11c29330 0x560c11c439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 --2- 172.21.15.113:0/761383242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c11bff260 0x560c11cbdf30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98b57fa640 0 client.5859 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 >> 172.21.15.113:0/761383242 conn(0x560c11c2cdc0 msgr2=0x560c11c167c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 shutdown_connections 2024-04-25T00:01:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 1 -- 172.21.15.113:0/761383242 wait complete. 2024-04-25T00:01:02.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.938+0000 7f98dfdb9740 20 client.5859 trim_cache size 0 max 16384 2024-04-25T00:01:02.945 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f98b806be30 msgr2=0x7f98b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f98b806be30 0x7f98b806e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98cc034000 tx=0x7f98cc02e000 comp rx=0 tx=0).stop 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 msgr2=0x7f98d80b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b2840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009db0 tx=0x7f98c800c490 comp rx=0 tx=0).stop 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 shutdown_connections 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f98b806be30 0x7f98b806e2f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d815e6b0 0x7f98d8151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98d815e2d0 0x7f98d80ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 --2- 172.21.15.113:0/1084170252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98d80b3080 0x7f98d80b2840 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 >> 172.21.15.113:0/1084170252 conn(0x7f98d806b0f0 msgr2=0x7f98d80b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 shutdown_connections 2024-04-25T00:01:02.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.939+0000 7f98dcfce640 1 -- 172.21.15.113:0/1084170252 wait complete. 2024-04-25T00:01:02.949 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4c.00000000 parent 2024-04-25T00:01:02.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.990+0000 7f7cabb62f40 1 Processor -- start 2024-04-25T00:01:02.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.990+0000 7f7cabb62f40 1 -- start start 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7cabb62f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca85e8880 0x55fca8611560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7cabb62f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 0x55fca86b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7cabb62f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca86121a0 0x55fca86b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7cabb62f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fca858f640 con 0x55fca85e8880 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7cabb62f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fca8593680 con 0x55fca86121a0 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7cabb62f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fca8630920 con 0x55fca85e8c60 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca85e8880 0x55fca8611560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca85e8880 0x55fca8611560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38816/0 (socket says 172.21.15.113:38816) 2024-04-25T00:01:02.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 -- 172.21.15.113:0/2799061154 learned_addr learned my addr 172.21.15.113:0/2799061154 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9e64640 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca86121a0 0x55fca86b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca3fff640 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 0x55fca86b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 -- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 msgr2=0x55fca86b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 0x55fca86b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 -- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca86121a0 msgr2=0x55fca86b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca86121a0 0x55fca86b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 -- 172.21.15.113:0/2799061154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca86328f0 con 0x55fca85e8880 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9e64640 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca86121a0 0x55fca86b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.991+0000 7f7ca9663640 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca85e8880 0x55fca8611560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00a5c0 tx=0x7f7c9c00aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22c35ae80438a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7ca37fe640 1 -- 172.21.15.113:0/2799061154 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0089f0 con 0x55fca85e8880 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7ca37fe640 1 -- 172.21.15.113:0/2799061154 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c9c00f070 con 0x55fca85e8880 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7ca37fe640 1 -- 172.21.15.113:0/2799061154 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c008df0 con 0x55fca85e8880 2024-04-25T00:01:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 -- 172.21.15.113:0/2799061154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca85e8880 msgr2=0x55fca8611560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca85e8880 0x55fca8611560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00a5c0 tx=0x7f7c9c00aa80 comp rx=0 tx=0).stop 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 -- 172.21.15.113:0/2799061154 shutdown_connections 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca86121a0 0x55fca86b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 0x55fca86b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/2799061154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca85e8880 0x55fca8611560 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 -- 172.21.15.113:0/2799061154 >> 172.21.15.113:0/2799061154 conn(0x55fca8613a50 msgr2=0x55fca8617ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 -- 172.21.15.113:0/2799061154 shutdown_connections 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.992+0000 7f7cabb62f40 1 -- 172.21.15.113:0/2799061154 wait complete. 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 Processor -- start 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 -- start start 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca85e8880 0x55fca8749cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 0x55fca8616f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca86121a0 0x55fca8617470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fca8618600 con 0x55fca86121a0 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fca86326a0 con 0x55fca85e8880 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.993+0000 7f7cabb62f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fca8618850 con 0x55fca85e8c60 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca86121a0 0x55fca8617470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca86121a0 0x55fca8617470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38818/0 (socket says 172.21.15.113:38818) 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 -- 172.21.15.113:0/3292730252 learned_addr learned my addr 172.21.15.113:0/3292730252 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca3fff640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 0x55fca8616f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9663640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca85e8880 0x55fca8749cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 -- 172.21.15.113:0/3292730252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 msgr2=0x55fca8616f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fca85e8c60 0x55fca8616f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 -- 172.21.15.113:0/3292730252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca85e8880 msgr2=0x55fca8749cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fca85e8880 0x55fca8749cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 -- 172.21.15.113:0/3292730252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca87496c0 con 0x55fca86121a0 2024-04-25T00:01:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca9e64640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca86121a0 0x55fca8617470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c5f0 tx=0x7f7ca400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca27fc640 1 -- 172.21.15.113:0/3292730252 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004520 con 0x55fca86121a0 2024-04-25T00:01:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca27fc640 1 -- 172.21.15.113:0/3292730252 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca40046c0 con 0x55fca86121a0 2024-04-25T00:01:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7ca27fc640 1 -- 172.21.15.113:0/3292730252 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004a90 con 0x55fca86121a0 2024-04-25T00:01:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fca860d150 con 0x55fca86121a0 2024-04-25T00:01:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.994+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca860ce30 con 0x55fca86121a0 2024-04-25T00:01:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.995+0000 7f7ca27fc640 1 -- 172.21.15.113:0/3292730252 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ca400b030 con 0x55fca86121a0 2024-04-25T00:01:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.996+0000 7f7ca27fc640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8c06a840 0x7f7c8c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.996+0000 7f7ca27fc640 1 -- 172.21.15.113:0/3292730252 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f7ca408baa0 con 0x55fca86121a0 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.996+0000 7f7ca9663640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8c06a840 0x7f7c8c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.996+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x55fca85ea7e0 0x55fca860ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.996+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 --> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] -- osd_op(unknown.0.0:1 7.3a 7:5fa3b718:::10000000a4c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) -- 0x55fca874a230 con 0x55fca85ea7e0 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.996+0000 7f7ca3fff640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x55fca85ea7e0 0x55fca860ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.996+0000 7f7ca9663640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8c06a840 0x7f7c8c06cd00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00aac0 tx=0x7f7c9c002000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.997+0000 7f7ca3fff640 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x55fca85ea7e0 0x55fca860ed80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7ca3fff640 1 -- 172.21.15.113:0/3292730252 <== osd.3 v2:172.21.15.113:6824/1975200730 1 ==== osd_op_reply(1 10000000a4c.00000000 [rmxattr] v48'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7c94054880 con 0x55fca85ea7e0 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x55fca85ea7e0 msgr2=0x55fca860ed80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x55fca85ea7e0 0x55fca860ed80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8c06a840 msgr2=0x7f7c8c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8c06a840 0x7f7c8c06cd00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00aac0 tx=0x7f7c9c002000 comp rx=0 tx=0).stop 2024-04-25T00:01:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca86121a0 msgr2=0x55fca8617470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 --2- 172.21.15.113:0/3292730252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fca86121a0 0x55fca8617470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c5f0 tx=0x7f7ca400cab0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7ca9e64640 1 -- 172.21.15.113:0/3292730252 reap_dead start 2024-04-25T00:01:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 shutdown_connections 2024-04-25T00:01:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 >> 172.21.15.113:0/3292730252 conn(0x55fca8613a50 msgr2=0x55fca8611440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 shutdown_connections 2024-04-25T00:01:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:02.998+0000 7f7cabb62f40 1 -- 172.21.15.113:0/3292730252 wait complete. 2024-04-25T00:01:03.003 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T00:01:03.003 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T00:01:03.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.141+0000 7f08f7ddc640 1 Processor -- start 2024-04-25T00:01:03.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.141+0000 7f08f7ddc640 1 -- start start 2024-04-25T00:01:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f7ddc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f015b670 0x7f08f015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f7ddc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08f015bff0 0x7f08f00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f7ddc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f0071170 con 0x7f08f00cc740 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f7ddc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f08f015c470 con 0x7f08f015bff0 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f7ddc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08f0071450 con 0x7f08f015b670 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f5377640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08f015bff0 0x7f08f00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f5377640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08f015bff0 0x7f08f00cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38828/0 (socket says 172.21.15.113:38828) 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f5377640 1 -- 172.21.15.113:0/896071885 learned_addr learned my addr 172.21.15.113:0/896071885 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f5377640 1 -- 172.21.15.113:0/896071885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f015b670 msgr2=0x7f08f015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f5377640 1 --2- 172.21.15.113:0/896071885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f015b670 0x7f08f015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f5377640 1 -- 172.21.15.113:0/896071885 >> v1:172.21.15.178:6789/0 conn(0x7f08f00cc740 legacy=0x7f08f006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.142+0000 7f08f5377640 1 -- 172.21.15.113:0/896071885 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f0071730 con 0x7f08f015bff0 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.143+0000 7f08f5377640 1 --2- 172.21.15.113:0/896071885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015bff0 0x7f08f00cc200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009870 tx=0x7f08e002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d271c47f0fdc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.143+0000 7f08f4b76640 1 -- 172.21.15.113:0/896071885 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0037070 con 0x7f08f015bff0 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.143+0000 7f08f4b76640 1 -- 172.21.15.113:0/896071885 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e002ae10 con 0x7f08f015bff0 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.143+0000 7f08f4b76640 1 -- 172.21.15.113:0/896071885 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0032660 con 0x7f08f015bff0 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 -- 172.21.15.113:0/896071885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015bff0 msgr2=0x7f08f00cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/896071885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015bff0 0x7f08f00cc200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009870 tx=0x7f08e002a100 comp rx=0 tx=0).stop 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 -- 172.21.15.113:0/896071885 shutdown_connections 2024-04-25T00:01:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/896071885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015bff0 0x7f08f00cc200 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/896071885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f015b670 0x7f08f015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 -- 172.21.15.113:0/896071885 >> 172.21.15.113:0/896071885 conn(0x7f08f006b0f0 msgr2=0x7f08f006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 -- 172.21.15.113:0/896071885 shutdown_connections 2024-04-25T00:01:03.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.144+0000 7f08f7ddc640 1 -- 172.21.15.113:0/896071885 wait complete. 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.145+0000 7f08f7ddc640 1 Processor -- start 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.145+0000 7f08f7ddc640 1 -- start start 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f7ddc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00cc740 0x7f08f014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f7ddc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015b670 0x7f08f0142a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f7ddc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f015bff0 0x7f08f01499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f7ddc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f08f006d730 con 0x7f08f015b670 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f7ddc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08f006d5b0 con 0x7f08f015bff0 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f7ddc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08f00d0080 con 0x7f08f00cc740 2024-04-25T00:01:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015b670 0x7f08f0142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015b670 0x7f08f0142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38842/0 (socket says 172.21.15.113:38842) 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 -- 172.21.15.113:0/1855694573 learned_addr learned my addr 172.21.15.113:0/1855694573 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f6379640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f015bff0 0x7f08f01499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 -- 172.21.15.113:0/1855694573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00cc740 msgr2=0x7f08f014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00cc740 0x7f08f014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 -- 172.21.15.113:0/1855694573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f015bff0 msgr2=0x7f08f01499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f015bff0 0x7f08f01499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 -- 172.21.15.113:0/1855694573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f014d4e0 con 0x7f08f015b670 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.146+0000 7f08f5377640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015b670 0x7f08f0142a40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009870 tx=0x7f08e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.147+0000 7f08e67fc640 1 -- 172.21.15.113:0/1855694573 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e00433f0 con 0x7f08f015b670 2024-04-25T00:01:03.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.147+0000 7f08e67fc640 1 -- 172.21.15.113:0/1855694573 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e0043590 con 0x7f08f015b670 2024-04-25T00:01:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.147+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f014e020 con 0x7f08f015b670 2024-04-25T00:01:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.147+0000 7f08e67fc640 1 -- 172.21.15.113:0/1855694573 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0043980 con 0x7f08f015b670 2024-04-25T00:01:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.147+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f014da60 con 0x7f08f015b670 2024-04-25T00:01:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.148+0000 7f08e67fc640 1 -- 172.21.15.113:0/1855694573 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08e004a060 con 0x7f08f015b670 2024-04-25T00:01:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.149+0000 7f08e67fc640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08d406be30 0x7f08d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.149+0000 7f08e67fc640 1 -- 172.21.15.113:0/1855694573 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f08e00acc30 con 0x7f08f015b670 2024-04-25T00:01:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.149+0000 7f08f5b78640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08d406be30 0x7f08d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.149+0000 7f08f5b78640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08d406be30 0x7f08d406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009710 tx=0x7f08ec006c90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.149+0000 7f08fabc8740 1 Processor -- start 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08fabc8740 1 -- start start 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efc19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efdf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef972b0 0x56068efdd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08fabc8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56068eef9660 con 0x56068efc15d0 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56068eea1800 con 0x56068efdee70 2024-04-25T00:01:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56068eef5620 con 0x56068ef972b0 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08f5b78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efc19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08f5b78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efc19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38848/0 (socket says 172.21.15.113:38848) 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.150+0000 7f08f5b78640 1 -- 172.21.15.113:0/1202623465 learned_addr learned my addr 172.21.15.113:0/1202623465 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f5377640 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efdf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f6379640 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef972b0 0x56068efdd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f5b78640 1 -- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef972b0 msgr2=0x56068efdd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f5b78640 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef972b0 0x56068efdd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f5b78640 1 -- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 msgr2=0x56068efdf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f5b78640 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efdf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f5b78640 1 -- 172.21.15.113:0/1202623465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068ecd9c00 con 0x56068efc15d0 2024-04-25T00:01:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f6379640 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef972b0 0x56068efdd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08f5b78640 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efc19b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08ec015890 tx=0x7f08ec00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefe7b1a6e91fcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08d37fe640 1 -- 172.21.15.113:0/1202623465 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec023070 con 0x56068efc15d0 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08d37fe640 1 -- 172.21.15.113:0/1202623465 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec0152e0 con 0x56068efc15d0 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.151+0000 7f08d37fe640 1 -- 172.21.15.113:0/1202623465 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00bb80 con 0x56068efc15d0 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 -- 172.21.15.113:0/1202623465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 msgr2=0x56068efc19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efc19b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08ec015890 tx=0x7f08ec00e000 comp rx=0 tx=0).stop 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 -- 172.21.15.113:0/1202623465 shutdown_connections 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efdf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef972b0 0x56068efdd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1202623465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efc19b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 -- 172.21.15.113:0/1202623465 >> 172.21.15.113:0/1202623465 conn(0x56068ef93be0 msgr2=0x56068efdc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 -- 172.21.15.113:0/1202623465 shutdown_connections 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 -- 172.21.15.113:0/1202623465 wait complete. 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 -- start start 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.152+0000 7f08fabc8740 1 Processor -- start 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efb2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efc3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 0x56068ef90ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08fabc8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56068ef98050 con 0x56068efc15d0 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56068eeb4f10 con 0x56068efdee70 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56068ef91b10 con 0x56068ef90860 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08fabc8740 10 client.0 _ll_register_callbacks cb 0x56068ec0cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efb2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efb2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38862/0 (socket says 172.21.15.113:38862) 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 -- 172.21.15.113:0/3090172545 learned_addr learned my addr 172.21.15.113:0/3090172545 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f6379640 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 0x56068ef90ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5377640 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efc3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 -- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 msgr2=0x56068ef90ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 0x56068ef90ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 -- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 msgr2=0x56068efc3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efc3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.153+0000 7f08f5b78640 1 -- 172.21.15.113:0/3090172545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068ebcd4f0 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08f5b78640 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efb2300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0090d0 tx=0x7f08ec004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ec3b46c4a37bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08d0ff9640 1 -- 172.21.15.113:0/3090172545 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec023070 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08d0ff9640 1 -- 172.21.15.113:0/3090172545 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f08ec013630 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec0152e0 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08d0ff9640 1 -- 172.21.15.113:0/3090172545 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f08ec033050 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 10 client.5883 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56068ed3a390 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08d0ff9640 1 -- 172.21.15.113:0/3090172545 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec0152e0 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08d0ff9640 1 -- 172.21.15.113:0/3090172545 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f08ec036020 con 0x56068efc15d0 2024-04-25T00:01:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 10 client.5883 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 10 client.5883 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 20 client.5883 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efc2760 0x56068efc2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 4 client.5883 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56068ef06ba0 con 0x56068efc2760 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.154+0000 7f08f5377640 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efc2760 0x56068efc2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08d0ff9640 10 client.5883 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08f5377640 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efc2760 0x56068efc2bc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7be7256d1a67d4 server_cookie=9bf1a5d0bbb64ceb in_seq=0 out_seq=0 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08d0ff9640 1 -- 172.21.15.113:0/3090172545 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56068ef06ba0 con 0x56068efc2760 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08d0ff9640 10 client.5883 handle_command_reply: tid=0 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 client.5883 shutdown 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 msgr2=0x56068efb2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efb2300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0090d0 tx=0x7f08ec004840 comp rx=0 tx=0).stop 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 shutdown_connections 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efc2760 0x56068efc2bc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efc3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08d0ff9640 0 client.5883 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 0x56068ef90ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 --2- 172.21.15.113:0/3090172545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efb2300 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.155+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 >> 172.21.15.113:0/3090172545 conn(0x56068ef972b0 msgr2=0x56068efdbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.156+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 shutdown_connections 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.156+0000 7f08fabc8740 1 -- 172.21.15.113:0/3090172545 wait complete. 2024-04-25T00:01:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.156+0000 7f08fabc8740 20 client.5883 trim_cache size 0 max 16384 2024-04-25T00:01:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.173+0000 7f08fabc8740 1 Processor -- start 2024-04-25T00:01:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.173+0000 7f08fabc8740 1 -- start start 2024-04-25T00:01:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068ef90860 0x56068ef94780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068efc15d0 0x56068efe0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efd5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08fabc8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56068ef917f0 con 0x56068ef90860 2024-04-25T00:01:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01fc6e0 con 0x56068efdee70 2024-04-25T00:01:03.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08f01fb6e0 con 0x56068efc15d0 2024-04-25T00:01:03.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08f5b78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068ef90860 0x56068ef94780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08f5b78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068ef90860 0x56068ef94780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38878/0 (socket says 172.21.15.113:38878) 2024-04-25T00:01:03.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.174+0000 7f08f5b78640 1 -- 172.21.15.113:0/4180973707 learned_addr learned my addr 172.21.15.113:0/4180973707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08f5b78640 1 -- 172.21.15.113:0/4180973707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068efc15d0 msgr2=0x56068efe0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08f5b78640 1 --2- 172.21.15.113:0/4180973707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068efc15d0 0x56068efe0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08f5b78640 1 -- 172.21.15.113:0/4180973707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 msgr2=0x56068efd5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08f5b78640 1 --2- 172.21.15.113:0/4180973707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efd5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08f5b78640 1 -- 172.21.15.113:0/4180973707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068eba6670 con 0x56068ef90860 2024-04-25T00:01:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08f5b78640 1 --2- 172.21.15.113:0/4180973707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068ef90860 0x56068ef94780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006b10 tx=0x7f08ec023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774d17d1ef7b1b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08d2ffd640 1 -- 172.21.15.113:0/4180973707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec03c070 con 0x56068ef90860 2024-04-25T00:01:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08d2ffd640 1 -- 172.21.15.113:0/4180973707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec013630 con 0x56068ef90860 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08d2ffd640 1 -- 172.21.15.113:0/4180973707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec013b00 con 0x56068ef90860 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08fabc8740 1 -- 172.21.15.113:0/4180973707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068ef90860 msgr2=0x56068ef94780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08fabc8740 1 --2- 172.21.15.113:0/4180973707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068ef90860 0x56068ef94780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006b10 tx=0x7f08ec023000 comp rx=0 tx=0).stop 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08fabc8740 1 -- 172.21.15.113:0/4180973707 shutdown_connections 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08fabc8740 1 --2- 172.21.15.113:0/4180973707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068efd5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08fabc8740 1 --2- 172.21.15.113:0/4180973707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068efc15d0 0x56068efe0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08fabc8740 1 --2- 172.21.15.113:0/4180973707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068ef90860 0x56068ef94780 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.175+0000 7f08fabc8740 1 -- 172.21.15.113:0/4180973707 >> 172.21.15.113:0/4180973707 conn(0x56068ef972b0 msgr2=0x56068ef90ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.176+0000 7f08fabc8740 1 -- 172.21.15.113:0/4180973707 shutdown_connections 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.176+0000 7f08fabc8740 1 -- 172.21.15.113:0/4180973707 wait complete. 2024-04-25T00:01:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.176+0000 7f08fabc8740 1 -- start start 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.176+0000 7f08fabc8740 1 Processor -- start 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 0x56068efef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efdbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08fabc8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068f04bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08fabc8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0167150 con 0x56068efc15d0 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08f014caa0 con 0x56068efdee70 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08fabc8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56068efaff20 con 0x56068ef90860 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08fabc8740 10 client.0 _ll_register_callbacks cb 0x7f08ec021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efdbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efdbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38880/0 (socket says 172.21.15.113:38880) 2024-04-25T00:01:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 -- 172.21.15.113:0/1883369564 learned_addr learned my addr 172.21.15.113:0/1883369564 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 -- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 msgr2=0x56068efef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f6379640 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068f04bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 0x56068efef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 -- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 msgr2=0x56068f04bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068f04bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f5377640 1 -- 172.21.15.113:0/1883369564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068efdab10 con 0x56068efc15d0 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.177+0000 7f08f6379640 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068f04bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08f5377640 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efdbe50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f08e00ab1b0 tx=0x7f08e007ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dbc2f4b04b29ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08d0ff9640 1 -- 172.21.15.113:0/1883369564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a8070 con 0x56068efc15d0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08d0ff9640 1 -- 172.21.15.113:0/1883369564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f08e00775a0 con 0x56068efc15d0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec013630 con 0x56068efc15d0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08d0ff9640 1 -- 172.21.15.113:0/1883369564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f08e0083b10 con 0x56068efc15d0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 10 client.5889 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56068f056020 con 0x56068efc15d0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08d0ff9640 1 -- 172.21.15.113:0/1883369564 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec013630 con 0x56068efc15d0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08d0ff9640 1 -- 172.21.15.113:0/1883369564 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f08e004a350 con 0x56068efc15d0 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 10 client.5889 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 10 client.5889 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 20 client.5889 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efdc3e0 0x56068efdc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 4 client.5889 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) -- 0x56068ef00ef0 con 0x56068efdc3e0 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08f5b78640 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efdc3e0 0x56068efdc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08d0ff9640 10 client.5889 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.178+0000 7f08f5b78640 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efdc3e0 0x56068efdc7f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f687c50cbd226c1 server_cookie=3bfcd98f3a6c12d0 in_seq=0 out_seq=0 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08d0ff9640 1 -- 172.21.15.113:0/1883369564 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56068ef00ef0 con 0x56068efdc3e0 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08d0ff9640 10 client.5889 handle_command_reply: tid=0 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 client.5889 shutdown 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 msgr2=0x56068efdbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efdbe50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f08e00ab1b0 tx=0x7f08e007ab90 comp rx=0 tx=0).stop 2024-04-25T00:01:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 shutdown_connections 2024-04-25T00:01:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56068efdc3e0 0x56068efdc7f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56068efdee70 0x56068f04bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56068efc15d0 0x56068efdbe50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 --2- 172.21.15.113:0/1883369564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56068ef90860 0x56068efef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 >> 172.21.15.113:0/1883369564 conn(0x56068ef972b0 msgr2=0x56068ef98780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08d0ff9640 0 client.5889 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.189 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:01:03.179+0000 7fb09ec87640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4c [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v442 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5715=0-4194304@1} caps={5715=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5715 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55916b2f1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000a4c:[<0x10000000001/new_file_0 v442>,<0x10000000000/ceph-qa-suite v40>,<0x1/scrub_test_path v13>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T00:01:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 shutdown_connections 2024-04-25T00:01:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 1 -- 172.21.15.113:0/1883369564 wait complete. 2024-04-25T00:01:03.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.179+0000 7f08fabc8740 20 client.5889 trim_cache size 0 max 16384 2024-04-25T00:01:03.191 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:03.191 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:03.191 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "4121607b-d340-4f03-8757-9b38cd2fa7ad", 2024-04-25T00:01:03.191 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:03.191 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:03.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08d406be30 msgr2=0x7f08d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08d406be30 0x7f08d406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009710 tx=0x7f08ec006c90 comp rx=0 tx=0).stop 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015b670 msgr2=0x7f08f0142a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015b670 0x7f08f0142a40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009870 tx=0x7f08e00046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 shutdown_connections 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08d406be30 0x7f08d406e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f015bff0 0x7f08f01499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f015b670 0x7f08f0142a40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 --2- 172.21.15.113:0/1855694573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00cc740 0x7f08f014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 >> 172.21.15.113:0/1855694573 conn(0x7f08f006b0f0 msgr2=0x7f08f00cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 shutdown_connections 2024-04-25T00:01:03.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.180+0000 7f08f7ddc640 1 -- 172.21.15.113:0/1855694573 wait complete. 2024-04-25T00:01:03.194 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '4121607b-d340-4f03-8757-9b38cd2fa7ad', 'mode': 'asynchronous'}' 2024-04-25T00:01:03.194 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T00:01:03.194 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T00:01:03.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.293+0000 7f9a2b35c640 1 Processor -- start 2024-04-25T00:01:03.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a2b35c640 1 -- start start 2024-04-25T00:01:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a2b35c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415b670 0x7f9a2415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a2b35c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a2406cea0 0x7f9a2406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a2b35c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a24071170 con 0x7f9a2415bff0 2024-04-25T00:01:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a2b35c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9a2406d830 con 0x7f9a2406cea0 2024-04-25T00:01:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a2b35c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2415c470 con 0x7f9a2415b670 2024-04-25T00:01:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a298f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a2406cea0 0x7f9a2406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a298f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a2406cea0 0x7f9a2406d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38882/0 (socket says 172.21.15.113:38882) 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.294+0000 7f9a298f9640 1 -- 172.21.15.113:0/2825008818 learned_addr learned my addr 172.21.15.113:0/2825008818 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.295+0000 7f9a298f9640 1 -- 172.21.15.113:0/2825008818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415b670 msgr2=0x7f9a2415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.295+0000 7f9a298f9640 1 --2- 172.21.15.113:0/2825008818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415b670 0x7f9a2415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.295+0000 7f9a298f9640 1 -- 172.21.15.113:0/2825008818 >> v1:172.21.15.178:6789/0 conn(0x7f9a2415bff0 legacy=0x7f9a240cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.295+0000 7f9a298f9640 1 -- 172.21.15.113:0/2825008818 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a24071450 con 0x7f9a2406cea0 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.295+0000 7f9a298f9640 1 --2- 172.21.15.113:0/2825008818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2406cea0 0x7f9a2406d2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009e80 tx=0x7f9a1c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1846e1614e285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.295+0000 7f9a1bfff640 1 -- 172.21.15.113:0/2825008818 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c036070 con 0x7f9a2406cea0 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a1bfff640 1 -- 172.21.15.113:0/2825008818 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c031600 con 0x7f9a2406cea0 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a1bfff640 1 -- 172.21.15.113:0/2825008818 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0319d0 con 0x7f9a2406cea0 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 -- 172.21.15.113:0/2825008818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2406cea0 msgr2=0x7f9a2406d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/2825008818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2406cea0 0x7f9a2406d2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009e80 tx=0x7f9a1c02a5d0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 -- 172.21.15.113:0/2825008818 shutdown_connections 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/2825008818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2406cea0 0x7f9a2406d2f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/2825008818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415b670 0x7f9a2415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 -- 172.21.15.113:0/2825008818 >> 172.21.15.113:0/2825008818 conn(0x7f9a2406b0f0 msgr2=0x7f9a2406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 -- 172.21.15.113:0/2825008818 shutdown_connections 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.296+0000 7f9a2b35c640 1 -- 172.21.15.113:0/2825008818 wait complete. 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.297+0000 7f9a2b35c640 1 Processor -- start 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.297+0000 7f9a2b35c640 1 -- start start 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a2b35c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2406cea0 0x7f9a241f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a2b35c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2415b670 0x7f9a240c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a2b35c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415bff0 0x7f9a2414cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a2b35c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9a240cc910 con 0x7f9a2415b670 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a2b35c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24071a10 con 0x7f9a2406cea0 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a2b35c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9a240cca90 con 0x7f9a2415bff0 2024-04-25T00:01:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2415b670 0x7f9a240c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2415b670 0x7f9a240c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38890/0 (socket says 172.21.15.113:38890) 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 -- 172.21.15.113:0/1326048266 learned_addr learned my addr 172.21.15.113:0/1326048266 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a298f9640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415bff0 0x7f9a2414cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2406cea0 0x7f9a241f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 -- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415bff0 msgr2=0x7f9a2414cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415bff0 0x7f9a2414cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 -- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2406cea0 msgr2=0x7f9a241f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2406cea0 0x7f9a241f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.298+0000 7f9a288f7640 1 -- 172.21.15.113:0/1326048266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a241f2620 con 0x7f9a2415b670 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.299+0000 7f9a288f7640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2415b670 0x7f9a240c84e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c440 tx=0x7f9a1400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.299+0000 7f9a19ffb640 1 -- 172.21.15.113:0/1326048266 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14017070 con 0x7f9a2415b670 2024-04-25T00:01:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.299+0000 7f9a19ffb640 1 -- 172.21.15.113:0/1326048266 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a140035f0 con 0x7f9a2415b670 2024-04-25T00:01:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.299+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a240c9450 con 0x7f9a2415b670 2024-04-25T00:01:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.299+0000 7f9a19ffb640 1 -- 172.21.15.113:0/1326048266 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140039e0 con 0x7f9a2415b670 2024-04-25T00:01:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.299+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a240c9a10 con 0x7f9a2415b670 2024-04-25T00:01:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.300+0000 7f9a19ffb640 1 -- 172.21.15.113:0/1326048266 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a1401a070 con 0x7f9a2415b670 2024-04-25T00:01:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.301+0000 7f9a19ffb640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a0806be30 0x7f9a0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.301+0000 7f9a19ffb640 1 -- 172.21.15.113:0/1326048266 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f9a1408a0e0 con 0x7f9a2415b670 2024-04-25T00:01:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.301+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a0806be30 0x7f9a0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.301+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a0806be30 0x7f9a0806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a200098e0 tx=0x7f9a20009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.302+0000 7f9a2e060740 1 Processor -- start 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.302+0000 7f9a2e060740 1 -- start start 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f282a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f28277a40 0x559f282bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a2e060740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559f281db6b0 con 0x559f282bf600 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f28183880 con 0x559f28277a40 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559f281d7670 con 0x559f282a1d60 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282bfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38904/0 (socket says 172.21.15.113:38904) 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 -- 172.21.15.113:0/3169432975 learned_addr learned my addr 172.21.15.113:0/3169432975 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 -- 172.21.15.113:0/3169432975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 msgr2=0x559f282a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 --2- 172.21.15.113:0/3169432975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f282a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 -- 172.21.15.113:0/3169432975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f28277a40 msgr2=0x559f282bdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 --2- 172.21.15.113:0/3169432975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f28277a40 0x559f282bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 -- 172.21.15.113:0/3169432975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f27fb97d0 con 0x559f282bf600 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a288f7640 1 --2- 172.21.15.113:0/3169432975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282bfb50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9a140055e0 tx=0x7f9a14057f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58025623a2c2b348 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a0effd640 1 -- 172.21.15.113:0/3169432975 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14054b00 con 0x559f282bf600 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a0effd640 1 -- 172.21.15.113:0/3169432975 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a14060b30 con 0x559f282bf600 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.303+0000 7f9a0effd640 1 -- 172.21.15.113:0/3169432975 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14054ca0 con 0x559f282bf600 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 -- 172.21.15.113:0/3169432975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 msgr2=0x559f282bfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 --2- 172.21.15.113:0/3169432975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282bfb50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9a140055e0 tx=0x7f9a14057f20 comp rx=0 tx=0).stop 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 -- 172.21.15.113:0/3169432975 shutdown_connections 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 --2- 172.21.15.113:0/3169432975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f28277a40 0x559f282bdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 --2- 172.21.15.113:0/3169432975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282bfb50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 --2- 172.21.15.113:0/3169432975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f282a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 -- 172.21.15.113:0/3169432975 >> 172.21.15.113:0/3169432975 conn(0x559f28274600 msgr2=0x559f282bcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 -- 172.21.15.113:0/3169432975 shutdown_connections 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 -- 172.21.15.113:0/3169432975 wait complete. 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 -- start start 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.304+0000 7f9a2e060740 1 Processor -- start 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f282a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282bf600 0x559f282d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f2829e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a2e060740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559f282787e0 con 0x559f2829e040 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f28196f50 con 0x559f282bf600 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559f28272530 con 0x559f282a1d60 2024-04-25T00:01:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a2e060740 10 client.0 _ll_register_callbacks cb 0x559f281d2000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a298f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f2829e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a298f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f2829e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38916/0 (socket says 172.21.15.113:38916) 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a298f9640 1 -- 172.21.15.113:0/1491179294 learned_addr learned my addr 172.21.15.113:0/1491179294 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a288f7640 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282bf600 0x559f282d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.305+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f282a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a298f9640 1 -- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 msgr2=0x559f282a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a298f9640 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f282a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a298f9640 1 -- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282bf600 msgr2=0x559f282d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a298f9640 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282bf600 0x559f282d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a298f9640 1 -- 172.21.15.113:0/1491179294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f27eef3f0 con 0x559f2829e040 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a298f9640 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f2829e4a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009fa0 tx=0x7f9a1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda4a1cdb0665154 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a03fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a03fff640 1 -- 172.21.15.113:0/1491179294 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c041070 con 0x559f2829e040 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a03fff640 1 -- 172.21.15.113:0/1491179294 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f9a1c003b90 con 0x559f2829e040 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a14060b30 con 0x559f2829e040 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a03fff640 1 -- 172.21.15.113:0/1491179294 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9a1c04c050 con 0x559f2829e040 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 10 client.5901 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x559f27eaf0f0 con 0x559f2829e040 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a03fff640 1 -- 172.21.15.113:0/1491179294 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a14060b30 con 0x559f2829e040 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a03fff640 1 -- 172.21.15.113:0/1491179294 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f9a1c04f020 con 0x559f2829e040 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 10 client.5901 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 10 client.5901 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 20 client.5901 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282a30e0 0x559f282cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 4 client.5901 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f281e8bf0 con 0x559f282a30e0 2024-04-25T00:01:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.306+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282a30e0 0x559f282cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a03fff640 10 client.5901 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282a30e0 0x559f282cd0f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ad00a23755ac4f server_cookie=df20b922be35e27b in_seq=0 out_seq=0 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a03fff640 1 -- 172.21.15.113:0/1491179294 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f281e8bf0 con 0x559f282a30e0 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a03fff640 10 client.5901 handle_command_reply: tid=0 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 client.5901 shutdown 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 msgr2=0x559f2829e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f2829e4a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009fa0 tx=0x7f9a1c004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 shutdown_connections 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282a30e0 0x559f282cd0f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282bf600 0x559f282d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f2829e4a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a03fff640 0 client.5901 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1491179294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f282a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.307+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 >> 172.21.15.113:0/1491179294 conn(0x559f28277a40 msgr2=0x559f282c00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.308+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 shutdown_connections 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.308+0000 7f9a2e060740 1 -- 172.21.15.113:0/1491179294 wait complete. 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.308+0000 7f9a2e060740 20 client.5901 trim_cache size 0 max 16384 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.325+0000 7f9a2e060740 1 Processor -- start 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.325+0000 7f9a2e060740 1 -- start start 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f282d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282a1d60 0x559f282a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282bf600 0x559f28277e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a2e060740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241fe880 con 0x559f2829e040 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241fd9c0 con 0x559f282a1d60 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9a241f58b0 con 0x559f282bf600 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f282d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f282d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38932/0 (socket says 172.21.15.113:38932) 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 -- 172.21.15.113:0/2443464577 learned_addr learned my addr 172.21.15.113:0/2443464577 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 -- 172.21.15.113:0/2443464577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282bf600 msgr2=0x559f28277e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 --2- 172.21.15.113:0/2443464577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282bf600 0x559f28277e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 -- 172.21.15.113:0/2443464577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282a1d60 msgr2=0x559f282a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 --2- 172.21.15.113:0/2443464577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282a1d60 0x559f282a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 -- 172.21.15.113:0/2443464577 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f28278f90 con 0x559f2829e040 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a290f8640 1 --2- 172.21.15.113:0/2443464577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f282d0990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000ed90 tx=0x7f9a20011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd43334958d07b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a0e7fc640 1 -- 172.21.15.113:0/2443464577 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2000edc0 con 0x559f2829e040 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a0e7fc640 1 -- 172.21.15.113:0/2443464577 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2002b070 con 0x559f2829e040 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.326+0000 7f9a0e7fc640 1 -- 172.21.15.113:0/2443464577 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a20025380 con 0x559f2829e040 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 -- 172.21.15.113:0/2443464577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 msgr2=0x559f282d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 --2- 172.21.15.113:0/2443464577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f282d0990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000ed90 tx=0x7f9a20011000 comp rx=0 tx=0).stop 2024-04-25T00:01:03.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 -- 172.21.15.113:0/2443464577 shutdown_connections 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 --2- 172.21.15.113:0/2443464577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f282a1d60 0x559f282a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 --2- 172.21.15.113:0/2443464577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282bf600 0x559f28277e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 --2- 172.21.15.113:0/2443464577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f2829e040 0x559f282d0990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 -- 172.21.15.113:0/2443464577 >> 172.21.15.113:0/2443464577 conn(0x559f28277a40 msgr2=0x559f2829d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 -- 172.21.15.113:0/2443464577 shutdown_connections 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 -- 172.21.15.113:0/2443464577 wait complete. 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 -- start start 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.327+0000 7f9a2e060740 1 Processor -- start 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f2829e040 0x559f282d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f28336cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a2e060740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a2e060740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559f282bf2a0 con 0x559f282bf600 2024-04-25T00:01:03.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241f5f00 con 0x559f2829e040 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a2e060740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559f282a2a20 con 0x559f282a1d60 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a2e060740 10 client.0 _ll_register_callbacks cb 0x7f9a1c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a298f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a298f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38946/0 (socket says 172.21.15.113:38946) 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a298f9640 1 -- 172.21.15.113:0/1427168313 learned_addr learned my addr 172.21.15.113:0/1427168313 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.328+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f2829e040 0x559f282d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a288f7640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f28336cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a298f9640 1 -- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 msgr2=0x559f28336cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a298f9640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f28336cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a298f9640 1 -- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f2829e040 msgr2=0x559f282d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a298f9640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f2829e040 0x559f282d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a298f9640 1 -- 172.21.15.113:0/1427168313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f28276110 con 0x559f282bf600 2024-04-25T00:01:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f2829e040 0x559f282d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a298f9640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282b43c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0094c0 tx=0x7f9a1c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d861b4090b62140b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a03fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a03fff640 1 -- 172.21.15.113:0/1427168313 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c054070 con 0x559f282bf600 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a03fff640 1 -- 172.21.15.113:0/1427168313 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f9a1c003fa0 con 0x559f282bf600 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a2002b070 con 0x559f282bf600 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a03fff640 1 -- 172.21.15.113:0/1427168313 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9a1c05f050 con 0x559f282bf600 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 10 client.5907 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x559f282b4990 con 0x559f282bf600 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a03fff640 1 -- 172.21.15.113:0/1427168313 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2002b070 con 0x559f282bf600 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a03fff640 1 -- 172.21.15.113:0/1427168313 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f9a1c062020 con 0x559f282bf600 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 10 client.5907 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 10 client.5907 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 20 client.5907 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282cee20 0x559f28277e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 4 client.5907 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f281e2f40 con 0x559f282cee20 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.329+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282cee20 0x559f28277e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a03fff640 10 client.5907 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a290f8640 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282cee20 0x559f28277e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa6dc3228ebfdd1 server_cookie=a3548bd8fa032c74 in_seq=0 out_seq=0 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a03fff640 1 -- 172.21.15.113:0/1427168313 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+903 (crc 0 0 0) 0x559f281e2f40 con 0x559f282cee20 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a03fff640 10 client.5907 handle_command_reply: tid=0 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 client.5907 shutdown 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 msgr2=0x559f282b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282b43c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0094c0 tx=0x7f9a1c009810 comp rx=0 tx=0).stop 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 shutdown_connections 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x559f282cee20 0x559f28277e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559f282bf600 0x559f282b43c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559f282a1d60 0x559f28336cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 --2- 172.21.15.113:0/1427168313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f2829e040 0x559f282d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a03fff640 0 client.5907 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 >> 172.21.15.113:0/1427168313 conn(0x559f28277a40 msgr2=0x559f282a0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 shutdown_connections 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 1 -- 172.21.15.113:0/1427168313 wait complete. 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.330+0000 7f9a2e060740 20 client.5907 trim_cache size 0 max 16384 2024-04-25T00:01:03.430 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "4121607b-d340-4f03-8757-9b38cd2fa7ad": { 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "4121607b-d340-4f03-8757-9b38cd2fa7ad", 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "5c0e312a-0efe-49ff-beb1-a1740478ffbf": { 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "5c0e312a-0efe-49ff-beb1-a1740478ffbf", 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "78037e16-5de2-463a-8abe-1b90b8373868": { 2024-04-25T00:01:03.431 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "78037e16-5de2-463a-8abe-1b90b8373868", 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: "c753840d-5202-4ec0-b5d8-014dcabcccd1": { 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "c753840d-5202-4ec0-b5d8-014dcabcccd1", 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:03.432 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a0806be30 msgr2=0x7f9a0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a0806be30 0x7f9a0806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a200098e0 tx=0x7f9a20009060 comp rx=0 tx=0).stop 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2415b670 msgr2=0x7f9a240c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2415b670 0x7f9a240c84e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c440 tx=0x7f9a1400c900 comp rx=0 tx=0).stop 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 shutdown_connections 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a0806be30 0x7f9a0806e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a2415bff0 0x7f9a2414cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2415b670 0x7f9a240c84e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 --2- 172.21.15.113:0/1326048266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2406cea0 0x7f9a241f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 >> 172.21.15.113:0/1326048266 conn(0x7f9a2406b0f0 msgr2=0x7f9a2406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 shutdown_connections 2024-04-25T00:01:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:03.331+0000 7f9a2b35c640 1 -- 172.21.15.113:0/1326048266 wait complete. 2024-04-25T00:01:06.225 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:01:06.225+0000 7f15dd321640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T00:01:08.435 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T00:01:08.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T00:01:08.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.547+0000 7fad22977640 1 Processor -- start 2024-04-25T00:01:08.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.547+0000 7fad22977640 1 -- start start 2024-04-25T00:01:08.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad22977640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad1c13f050 0x7fad1c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad22977640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad22977640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad1c15d6b0 con 0x7fad1c0d04f0 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad22977640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fad1c0d09a0 con 0x7fad1c13f050 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad22977640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad1c15c6a0 con 0x7fad1c1606a0 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad1c13f050 0x7fad1c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fad1c0d04f0 0x7fad1c15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52314/0 (socket says 172.21.15.113:52314) 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1b7fe640 1 -- 172.21.15.113:0/4176133745 learned_addr learned my addr 172.21.15.113:0/4176133745 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad20f14640 1 --2- 172.21.15.113:0/4176133745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1bfff640 1 -- 172.21.15.113:0/4176133745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 msgr2=0x7fad1c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1bfff640 1 --2- 172.21.15.113:0/4176133745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1bfff640 1 -- 172.21.15.113:0/4176133745 >> v1:172.21.15.113:6789/0 conn(0x7fad1c0d04f0 legacy=0x7fad1c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1bfff640 1 -- 172.21.15.113:0/4176133745 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c15c8f0 con 0x7fad1c13f050 2024-04-25T00:01:08.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.548+0000 7fad1affd640 1 -- 172.21.15.113:0/4176133745 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 831278539 0 0) 0x7fad1c15d6b0 con 0x7fad1c0d04f0 2024-04-25T00:01:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.549+0000 7fad1bfff640 1 --2- 172.21.15.113:0/4176133745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c13f050 0x7fad1c0cffb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0097a0 tx=0x7fad0c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2093fb342452ddbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.549+0000 7fad1affd640 1 -- 172.21.15.113:0/4176133745 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c037070 con 0x7fad1c13f050 2024-04-25T00:01:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.549+0000 7fad1affd640 1 -- 172.21.15.113:0/4176133745 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c02ae50 con 0x7fad1c13f050 2024-04-25T00:01:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.549+0000 7fad1affd640 1 -- 172.21.15.113:0/4176133745 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c032660 con 0x7fad1c13f050 2024-04-25T00:01:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 -- 172.21.15.113:0/4176133745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c13f050 msgr2=0x7fad1c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 --2- 172.21.15.113:0/4176133745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c13f050 0x7fad1c0cffb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0097a0 tx=0x7fad0c02a100 comp rx=0 tx=0).stop 2024-04-25T00:01:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 -- 172.21.15.113:0/4176133745 shutdown_connections 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 --2- 172.21.15.113:0/4176133745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 --2- 172.21.15.113:0/4176133745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c13f050 0x7fad1c0cffb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 -- 172.21.15.113:0/4176133745 >> 172.21.15.113:0/4176133745 conn(0x7fad1c06b0f0 msgr2=0x7fad1c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 -- 172.21.15.113:0/4176133745 shutdown_connections 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.550+0000 7fad22977640 1 -- 172.21.15.113:0/4176133745 wait complete. 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.551+0000 7fad22977640 1 Processor -- start 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.551+0000 7fad22977640 1 -- start start 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad22977640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c0d04f0 0x7fad1c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad22977640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad1c13f050 0x7fad1c13aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad22977640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad22977640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c15d190 con 0x7fad1c0d04f0 2024-04-25T00:01:08.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad22977640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c15ceb0 con 0x7fad1c13f050 2024-04-25T00:01:08.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad22977640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad1c13f430 con 0x7fad1c1606a0 2024-04-25T00:01:08.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c0d04f0 0x7fad1c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c0d04f0 0x7fad1c13e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32906/0 (socket says 172.21.15.113:32906) 2024-04-25T00:01:08.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 -- 172.21.15.113:0/1355144822 learned_addr learned my addr 172.21.15.113:0/1355144822 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad20f14640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad1c13f050 0x7fad1c13aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 -- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 msgr2=0x7fad1c159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 -- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad1c13f050 msgr2=0x7fad1c13aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad1c13f050 0x7fad1c13aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 -- 172.21.15.113:0/1355144822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c15b680 con 0x7fad1c0d04f0 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.552+0000 7fad1bfff640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c0d04f0 0x7fad1c13e170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0098c0 tx=0x7fad0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.553+0000 7fad18ff9640 1 -- 172.21.15.113:0/1355144822 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c041030 con 0x7fad1c0d04f0 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.553+0000 7fad18ff9640 1 -- 172.21.15.113:0/1355144822 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c02ac50 con 0x7fad1c0d04f0 2024-04-25T00:01:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.553+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1c15b960 con 0x7fad1c0d04f0 2024-04-25T00:01:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.553+0000 7fad18ff9640 1 -- 172.21.15.113:0/1355144822 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c037040 con 0x7fad1c0d04f0 2024-04-25T00:01:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.553+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1c13af90 con 0x7fad1c0d04f0 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.554+0000 7fad18ff9640 1 -- 172.21.15.113:0/1355144822 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad0c044070 con 0x7fad1c0d04f0 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.554+0000 7fad18ff9640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facfc06be30 0x7facfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.555+0000 7fad18ff9640 1 -- 172.21.15.113:0/1355144822 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fad0c0ab440 con 0x7fad1c0d04f0 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.555+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facfc06be30 0x7facfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.555+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facfc06be30 0x7facfc06e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fad1c157200 tx=0x7fad04007b70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 Processor -- start 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 -- start start 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e54600 0x55ce88e54b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e0ca40 0x55ce88e52cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ce88d706b0 con 0x55ce88e54600 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce88d18880 con 0x55ce88e36d60 2024-04-25T00:01:08.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ce88d6c670 con 0x55ce88e0ca40 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e54600 0x55ce88e54b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e54600 0x55ce88e54b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32916/0 (socket says 172.21.15.113:32916) 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.556+0000 7fad1b7fe640 1 -- 172.21.15.113:0/2581616069 learned_addr learned my addr 172.21.15.113:0/2581616069 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad20f14640 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e0ca40 0x55ce88e52cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad1bfff640 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad1b7fe640 1 -- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e0ca40 msgr2=0x55ce88e52cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e0ca40 0x55ce88e52cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad1b7fe640 1 -- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 msgr2=0x55ce88e37140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad1b7fe640 1 -- 172.21.15.113:0/2581616069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce88b4e7d0 con 0x55ce88e54600 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad20f14640 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e0ca40 0x55ce88e52cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:08.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e54600 0x55ce88e54b50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad1c156a20 tx=0x7fad04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd3bb7dc427428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad11ffb640 1 -- 172.21.15.113:0/2581616069 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04015110 con 0x55ce88e54600 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad11ffb640 1 -- 172.21.15.113:0/2581616069 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad04015990 con 0x55ce88e54600 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.557+0000 7fad11ffb640 1 -- 172.21.15.113:0/2581616069 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04015d40 con 0x55ce88e54600 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 -- 172.21.15.113:0/2581616069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e54600 msgr2=0x55ce88e54b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e54600 0x55ce88e54b50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad1c156a20 tx=0x7fad04012000 comp rx=0 tx=0).stop 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 -- 172.21.15.113:0/2581616069 shutdown_connections 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e0ca40 0x55ce88e52cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e54600 0x55ce88e54b50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 --2- 172.21.15.113:0/2581616069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 -- 172.21.15.113:0/2581616069 >> 172.21.15.113:0/2581616069 conn(0x55ce88e09600 msgr2=0x55ce88e51b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 -- 172.21.15.113:0/2581616069 shutdown_connections 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 -- 172.21.15.113:0/2581616069 wait complete. 2024-04-25T00:01:08.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 -- start start 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.558+0000 7fad25766740 1 Processor -- start 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e62e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e4ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad25766740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e50530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad25766740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ce88e0d7e0 con 0x55ce88e500b0 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce88d2bf50 con 0x55ce88e36d60 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ce88e07530 con 0x55ce88e54600 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad25766740 10 client.0 _ll_register_callbacks cb 0x55ce88d67000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad20f14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e50530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad20f14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e50530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32930/0 (socket says 172.21.15.113:32930) 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.559+0000 7fad20f14640 1 -- 172.21.15.113:0/1885181709 learned_addr learned my addr 172.21.15.113:0/1885181709 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad1bfff640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e62e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e4ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad20f14640 1 -- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 msgr2=0x55ce88e4ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad20f14640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e4ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad20f14640 1 -- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 msgr2=0x55ce88e62e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad20f14640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e62e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad20f14640 1 -- 172.21.15.113:0/1885181709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce88a843f0 con 0x55ce88e500b0 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e4ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad1bfff640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e62e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad20f14640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e50530 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad14007a00 tx=0x7fad14007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e8e8b8626a0a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7facfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7facfaffd640 1 -- 172.21.15.113:0/1885181709 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1401b020 con 0x55ce88e500b0 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7facfaffd640 1 -- 172.21.15.113:0/1885181709 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fad14011a80 con 0x55ce88e500b0 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad04015990 con 0x55ce88e500b0 2024-04-25T00:01:08.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7facfaffd640 1 -- 172.21.15.113:0/1885181709 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fad14025050 con 0x55ce88e500b0 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 10 client.5922 fetch_fsmap learned FSMap version 22 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55ce88a440f0 con 0x55ce88e500b0 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7facfaffd640 1 -- 172.21.15.113:0/1885181709 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad04015990 con 0x55ce88e500b0 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7facfaffd640 1 -- 172.21.15.113:0/1885181709 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fad14028020 con 0x55ce88e500b0 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 10 client.5922 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 10 client.5922 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 20 client.5922 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88e381e0 0x55ce88e38650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 4 client.5922 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:08.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce88d7dbf0 con 0x55ce88e381e0 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.560+0000 7fad1bfff640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88e381e0 0x55ce88e38650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.561+0000 7facfaffd640 10 client.5922 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.561+0000 7fad1bfff640 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88e381e0 0x55ce88e38650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de780f73beb6560 server_cookie=313b5c0d1b43f88e in_seq=0 out_seq=0 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.561+0000 7facfaffd640 1 -- 172.21.15.113:0/1885181709 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce88d7dbf0 con 0x55ce88e381e0 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.561+0000 7facfaffd640 10 client.5922 handle_command_reply: tid=0 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.561+0000 7fad25766740 1 client.5922 shutdown 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 msgr2=0x55ce88e50530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e50530 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad14007a00 tx=0x7fad14007ec0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 shutdown_connections 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88e381e0 0x55ce88e38650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e50530 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e4ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 --2- 172.21.15.113:0/1885181709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e62e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 >> 172.21.15.113:0/1885181709 conn(0x55ce88e0ca40 msgr2=0x55ce88e51380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7facfaffd640 0 client.5922 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 shutdown_connections 2024-04-25T00:01:08.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 1 -- 172.21.15.113:0/1885181709 wait complete. 2024-04-25T00:01:08.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.562+0000 7fad25766740 20 client.5922 trim_cache size 0 max 16384 2024-04-25T00:01:08.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.579+0000 7fad25766740 1 Processor -- start 2024-04-25T00:01:08.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.579+0000 7fad25766740 1 -- start start 2024-04-25T00:01:08.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad25766740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e500b0 0x55ce88e38e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e0bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad25766740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1fe650 con 0x55ce88e36d60 2024-04-25T00:01:08.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1fd6b0 con 0x55ce88e500b0 2024-04-25T00:01:08.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad1c1fca90 con 0x55ce88e54600 2024-04-25T00:01:08.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32944/0 (socket says 172.21.15.113:32944) 2024-04-25T00:01:08.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 -- 172.21.15.113:0/2820654977 learned_addr learned my addr 172.21.15.113:0/2820654977 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e500b0 0x55ce88e38e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 -- 172.21.15.113:0/2820654977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 msgr2=0x55ce88e0bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e0bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 -- 172.21.15.113:0/2820654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e500b0 msgr2=0x55ce88e38e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e500b0 0x55ce88e38e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 -- 172.21.15.113:0/2820654977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce88e4c9a0 con 0x55ce88e36d60 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad1bfff640 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fad0c079640 tx=0x7fad0c0a9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a067074f1690f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad117fa640 1 -- 172.21.15.113:0/2820654977 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c079db0 con 0x55ce88e36d60 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad117fa640 1 -- 172.21.15.113:0/2820654977 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c0796b0 con 0x55ce88e36d60 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.580+0000 7fad117fa640 1 -- 172.21.15.113:0/2820654977 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c07da60 con 0x55ce88e36d60 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 -- 172.21.15.113:0/2820654977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e36d60 msgr2=0x55ce88e37140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fad0c079640 tx=0x7fad0c0a9800 comp rx=0 tx=0).stop 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 -- 172.21.15.113:0/2820654977 shutdown_connections 2024-04-25T00:01:08.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e500b0 0x55ce88e38e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e0bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 --2- 172.21.15.113:0/2820654977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e36d60 0x55ce88e37140 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 -- 172.21.15.113:0/2820654977 >> 172.21.15.113:0/2820654977 conn(0x55ce88e0ca40 msgr2=0x55ce88e4a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 -- 172.21.15.113:0/2820654977 shutdown_connections 2024-04-25T00:01:08.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.581+0000 7fad25766740 1 -- 172.21.15.113:0/2820654977 wait complete. 2024-04-25T00:01:08.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 -- start start 2024-04-25T00:01:08.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 Processor -- start 2024-04-25T00:01:08.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e02c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e38b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ce88e76770 con 0x55ce88e500b0 2024-04-25T00:01:08.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1fd060 con 0x55ce88e36d60 2024-04-25T00:01:08.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.582+0000 7fad25766740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ce88e61f40 con 0x55ce88e54600 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad25766740 10 client.0 _ll_register_callbacks cb 0x7fad14009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32952/0 (socket says 172.21.15.113:32952) 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 -- 172.21.15.113:0/3596364569 learned_addr learned my addr 172.21.15.113:0/3596364569 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1bfff640 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e02c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad20f14640 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e38b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 -- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 msgr2=0x55ce88e38b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e38b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 -- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 msgr2=0x55ce88e02c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e02c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 -- 172.21.15.113:0/3596364569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce88e6c640 con 0x55ce88e500b0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad1b7fe640 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e385c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fad04000f60 tx=0x7fad04000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb349e9fea93e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7facfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7facfaffd640 1 -- 172.21.15.113:0/3596364569 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04015ba0 con 0x55ce88e500b0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7facfaffd640 1 -- 172.21.15.113:0/3596364569 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fad04015d40 con 0x55ce88e500b0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0c0796b0 con 0x55ce88e500b0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7facfaffd640 1 -- 172.21.15.113:0/3596364569 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fad04035050 con 0x55ce88e500b0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad25766740 10 client.5928 fetch_fsmap learned FSMap version 22 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.583+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55ce88e0ac60 con 0x55ce88e500b0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7facfaffd640 1 -- 172.21.15.113:0/3596364569 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c0796b0 con 0x55ce88e500b0 2024-04-25T00:01:08.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7facfaffd640 1 -- 172.21.15.113:0/3596364569 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7fad04038020 con 0x55ce88e500b0 2024-04-25T00:01:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 10 client.5928 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:01:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 10 client.5928 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 20 client.5928 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88ec54e0 0x55ce88e5ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 4 client.5928 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:01:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce88d77f40 con 0x55ce88ec54e0 2024-04-25T00:01:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad1bfff640 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88ec54e0 0x55ce88e5ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7facfaffd640 10 client.5928 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad1bfff640 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88ec54e0 0x55ce88e5ea60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4c400e46d20ec58 server_cookie=905f9a3d27922c44 in_seq=0 out_seq=0 2024-04-25T00:01:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7facfaffd640 1 -- 172.21.15.113:0/3596364569 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce88d77f40 con 0x55ce88ec54e0 2024-04-25T00:01:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7facfaffd640 10 client.5928 handle_command_reply: tid=0 2024-04-25T00:01:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 client.5928 shutdown 2024-04-25T00:01:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 msgr2=0x55ce88e385c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e385c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fad04000f60 tx=0x7fad04000ed0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 shutdown_connections 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55ce88ec54e0 0x55ce88e5ea60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.584+0000 7fad25766740 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ce88e54600 0x55ce88e38b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad25766740 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce88e500b0 0x55ce88e385c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7facfaffd640 0 client.5928 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad25766740 1 --2- 172.21.15.113:0/3596364569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce88e36d60 0x55ce88e02c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 >> 172.21.15.113:0/3596364569 conn(0x55ce88e0ca40 msgr2=0x55ce88e65d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 shutdown_connections 2024-04-25T00:01:08.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad25766740 1 -- 172.21.15.113:0/3596364569 wait complete. 2024-04-25T00:01:08.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad25766740 20 client.5928 trim_cache size 0 max 16384 2024-04-25T00:01:08.595 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:01:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:01:08.596 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:08.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facfc06be30 msgr2=0x7facfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facfc06be30 0x7facfc06e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fad1c157200 tx=0x7fad04007b70 comp rx=0 tx=0).stop 2024-04-25T00:01:08.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c0d04f0 msgr2=0x7fad1c13e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c0d04f0 0x7fad1c13e170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0098c0 tx=0x7fad0c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 shutdown_connections 2024-04-25T00:01:08.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facfc06be30 0x7facfc06e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad1c13f050 0x7fad1c13aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad1c1606a0 0x7fad1c159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 --2- 172.21.15.113:0/1355144822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad1c0d04f0 0x7fad1c13e170 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 >> 172.21.15.113:0/1355144822 conn(0x7fad1c06b0f0 msgr2=0x7fad1c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 shutdown_connections 2024-04-25T00:01:08.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.585+0000 7fad22977640 1 -- 172.21.15.113:0/1355144822 wait complete. 2024-04-25T00:01:08.599 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:01:08.599 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T00:01:08.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.698+0000 7f91cb7c8640 1 Processor -- start 2024-04-25T00:01:08.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.698+0000 7f91cb7c8640 1 -- start start 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.699+0000 7f91cb7c8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f91c40b3080 0x7f91c40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.699+0000 7f91cb7c8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c415f4f0 con 0x7f91c415e6b0 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.699+0000 7f91cb7c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c415f7d0 con 0x7f91c415e2d0 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.699+0000 7f91cb7c8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f91c40cde00 con 0x7f91c40b3080 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.699+0000 7f91c9d65640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f91c415e6b0 0x7f91c40ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52328/0 (socket says 172.21.15.113:52328) 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.699+0000 7f91c9d65640 1 -- 172.21.15.113:0/606155417 learned_addr learned my addr 172.21.15.113:0/606155417 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.699+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336376369 0 0) 0x7f91c415f4f0 con 0x7f91c415e6b0 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b0003520 con 0x7f91c415e6b0 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1537103893 0 0) 0x7f91b0003520 con 0x7f91c415e6b0 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 >> v2:172.21.15.178:3300/0 conn(0x7f91c40b3080 msgr2=0x7f91c40b4330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:08.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 --2- 172.21.15.113:0/606155417 >> v2:172.21.15.178:3300/0 conn(0x7f91c40b3080 0x7f91c40b4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 >> v1:172.21.15.178:6789/0 conn(0x7f91c415e2d0 legacy=0x7f91c415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c415fab0 con 0x7f91c415e6b0 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f91bc001670 con 0x7f91c415e6b0 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 >> v1:172.21.15.113:6789/0 conn(0x7f91c415e6b0 legacy=0x7f91c40ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b00039b0 0x7f91b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b0004800 0x7f91b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0005210 0x7f91b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91c40cde00 con 0x7f91b0005210 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91b000c160 con 0x7f91b00039b0 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91b000c360 con 0x7f91b0004800 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91c9d65640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0005210 0x7f91b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91c9d65640 1 -- 172.21.15.113:0/606155417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b0004800 msgr2=0x7f91b0004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:08.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91c9d65640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b0004800 0x7f91b0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91c9d65640 1 -- 172.21.15.113:0/606155417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b00039b0 msgr2=0x7f91b0003d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91c9d65640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b00039b0 0x7f91b0003d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.700+0000 7f91c9d65640 1 -- 172.21.15.113:0/606155417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b0003520 con 0x7f91b0005210 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.701+0000 7f91c9d65640 1 -- 172.21.15.113:0/606155417 reap_dead start 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.701+0000 7f91c9d65640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0005210 0x7f91b000b9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c210 tx=0x7f91b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12834bf109d09b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.701+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc007d50 con 0x7f91b0005210 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.701+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc005af0 con 0x7f91b0005210 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.701+0000 7f91bbfff640 1 -- 172.21.15.113:0/606155417 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc007070 con 0x7f91b0005210 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.701+0000 7f91cb7c8640 1 -- 172.21.15.113:0/606155417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0005210 msgr2=0x7f91b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.701+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0005210 0x7f91b000b9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c210 tx=0x7f91b0005830 comp rx=0 tx=0).stop 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.702+0000 7f91cb7c8640 1 -- 172.21.15.113:0/606155417 shutdown_connections 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.702+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/606155417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0005210 0x7f91b000b9c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.702+0000 7f91cb7c8640 1 -- 172.21.15.113:0/606155417 >> 172.21.15.113:0/606155417 conn(0x7f91c406b0f0 msgr2=0x7f91c406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.702+0000 7f91cb7c8640 1 -- 172.21.15.113:0/606155417 shutdown_connections 2024-04-25T00:01:08.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.702+0000 7f91cb7c8640 1 -- 172.21.15.113:0/606155417 wait complete. 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.703+0000 7f91cb7c8640 1 Processor -- start 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.703+0000 7f91cb7c8640 1 -- start start 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.703+0000 7f91cb7c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b0005210 0x7f91c4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.703+0000 7f91cb7c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91c4155cd0 0x7f91c414e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.703+0000 7f91cb7c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91c4157ad0 0x7f91c4157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91cb7c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91c415ed70 con 0x7f91c4157ad0 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91cb7c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91c415ebf0 con 0x7f91b0005210 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91cb7c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91c415eef0 con 0x7f91c4155cd0 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91c4157ad0 0x7f91c4157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91c4157ad0 0x7f91c4157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32982/0 (socket says 172.21.15.113:32982) 2024-04-25T00:01:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 -- 172.21.15.113:0/3401775625 learned_addr learned my addr 172.21.15.113:0/3401775625 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9564640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b0005210 0x7f91c4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c8d63640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91c4155cd0 0x7f91c414e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 -- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91c4155cd0 msgr2=0x7f91c414e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91c4155cd0 0x7f91c414e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 -- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b0005210 msgr2=0x7f91c4150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b0005210 0x7f91c4150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 -- 172.21.15.113:0/3401775625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c415aaa0 con 0x7f91c4157ad0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91c9d65640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91c4157ad0 0x7f91c4157f50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c210 tx=0x7f91c415f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91b9ffb640 1 -- 172.21.15.113:0/3401775625 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc041020 con 0x7f91c4157ad0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91b9ffb640 1 -- 172.21.15.113:0/3401775625 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc007270 con 0x7f91c4157ad0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91b9ffb640 1 -- 172.21.15.113:0/3401775625 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc007d50 con 0x7f91c4157ad0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c4150fc0 con 0x7f91c4157ad0 2024-04-25T00:01:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.704+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c4151580 con 0x7f91c4157ad0 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.706+0000 7f91b9ffb640 1 -- 172.21.15.113:0/3401775625 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91bc03c460 con 0x7f91c4157ad0 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.706+0000 7f91b9ffb640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91a806be60 0x7f91a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.707+0000 7f91b9ffb640 1 -- 172.21.15.113:0/3401775625 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f91bc0aab80 con 0x7f91c4157ad0 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.707+0000 7f91c9564640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91a806be60 0x7f91a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.707+0000 7f91c9564640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91a806be60 0x7f91a806e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91c00019b0 tx=0x7f91c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.707+0000 7f91ce5b1740 1 Processor -- start 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.707+0000 7f91ce5b1740 1 -- start start 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bb7c40 0x5560d7bbba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7b8f950 0x5560d7bb8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bbadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5560d7af4670 con 0x5560d7b8f950 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5560d7a9c970 con 0x5560d7bb7c40 2024-04-25T00:01:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5560d7af0630 con 0x5560d7bb8560 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7b8f950 0x5560d7bb8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7b8f950 0x5560d7bb8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32992/0 (socket says 172.21.15.113:32992) 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 -- 172.21.15.113:0/1252943378 learned_addr learned my addr 172.21.15.113:0/1252943378 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c9d65640 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bbadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c9564640 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bb7c40 0x5560d7bbba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 -- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 msgr2=0x5560d7bbadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bbadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 -- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bb7c40 msgr2=0x5560d7bbba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bb7c40 0x5560d7bbba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 -- 172.21.15.113:0/1252943378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d7ac0400 con 0x5560d7b8f950 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.708+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7b8f950 0x5560d7bb8020 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f91b40099f0 tx=0x7f91b4009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf195dbbd85a768 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91a6ffd640 1 -- 172.21.15.113:0/1252943378 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4017070 con 0x5560d7b8f950 2024-04-25T00:01:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91a6ffd640 1 -- 172.21.15.113:0/1252943378 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b400ea80 con 0x5560d7b8f950 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91a6ffd640 1 -- 172.21.15.113:0/1252943378 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b400ee30 con 0x5560d7b8f950 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1252943378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7b8f950 msgr2=0x5560d7bb8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7b8f950 0x5560d7bb8020 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f91b40099f0 tx=0x7f91b4009eb0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1252943378 shutdown_connections 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bbadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7b8f950 0x5560d7bb8020 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1252943378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bb7c40 0x5560d7bbba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1252943378 >> 172.21.15.113:0/1252943378 conn(0x5560d7b8d550 msgr2=0x5560d7bd5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1252943378 shutdown_connections 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.709+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1252943378 wait complete. 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.710+0000 7f91ce5b1740 1 -- start start 2024-04-25T00:01:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.710+0000 7f91ce5b1740 1 Processor -- start 2024-04-25T00:01:08.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bd4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7baf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7bb0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5560d7b8b8a0 con 0x5560d7bb8560 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5560d7ab0050 con 0x5560d7bea590 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5560d7b8b2e0 con 0x5560d7bb7c40 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 10 client.0 _ll_register_callbacks cb 0x5560d7807be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7baf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7baf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33002/0 (socket says 172.21.15.113:33002) 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 -- 172.21.15.113:0/1143813299 learned_addr learned my addr 172.21.15.113:0/1143813299 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c9564640 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bd4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 -- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 msgr2=0x5560d7bd4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bd4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 -- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 msgr2=0x5560d7bb0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7bb0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 -- 172.21.15.113:0/1143813299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d7a81dc0 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c9564640 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7baf500 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f91b4008000 tx=0x7f91b40046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0a4665bd00a507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f919ffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f919ffff640 1 -- 172.21.15.113:0/1143813299 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4017040 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f919ffff640 1 -- 172.21.15.113:0/1143813299 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f91b4004020 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.711+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b400ea80 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f919ffff640 1 -- 172.21.15.113:0/1143813299 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f91b402e050 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 10 client.5940 fetch_fsmap learned FSMap version 22 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5560d78083f0 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f919ffff640 1 -- 172.21.15.113:0/1143813299 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b400ea80 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f919ffff640 1 -- 172.21.15.113:0/1143813299 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f91b4031020 con 0x5560d7bb8560 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 10 client.5940 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:01:08.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 10 client.5940 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 20 client.5940 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7bd6160 0x5560d7bedb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 4 client.5940 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d7b01bb0 con 0x5560d7bd6160 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91c9564640 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7bd6160 0x5560d7bedb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f919ffff640 10 client.5940 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.712+0000 7f91c9564640 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7bd6160 0x5560d7bedb00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d7dca35c4d830a4 server_cookie=bddf75fbbbfadecc in_seq=0 out_seq=0 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f919ffff640 1 -- 172.21.15.113:0/1143813299 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560d7b01bb0 con 0x5560d7bd6160 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f919ffff640 10 client.5940 handle_command_reply: tid=0 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 client.5940 shutdown 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 msgr2=0x5560d7baf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7baf500 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f91b4008000 tx=0x7f91b40046e0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 shutdown_connections 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7bd6160 0x5560d7bedb00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7bb0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7baf500 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1143813299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f919ffff640 0 client.5940 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 >> 172.21.15.113:0/1143813299 conn(0x5560d7b8f950 msgr2=0x5560d7b8fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 shutdown_connections 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1143813299 wait complete. 2024-04-25T00:01:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.713+0000 7f91ce5b1740 20 client.5940 trim_cache size 0 max 16384 2024-04-25T00:01:08.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.726+0000 7f91ce5b1740 1 Processor -- start 2024-04-25T00:01:08.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.726+0000 7f91ce5b1740 1 -- start start 2024-04-25T00:01:08.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bb7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7b90860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7b90da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41fc9e0 con 0x5560d7bb8560 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41549f0 con 0x5560d7bea590 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91c415d4a0 con 0x5560d7bb7c40 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7b90860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7b90860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33012/0 (socket says 172.21.15.113:33012) 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 -- 172.21.15.113:0/1119721549 learned_addr learned my addr 172.21.15.113:0/1119721549 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 -- 172.21.15.113:0/1119721549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 msgr2=0x5560d7bb7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1119721549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bb7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 -- 172.21.15.113:0/1119721549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 msgr2=0x5560d7b90da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1119721549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7b90da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 -- 172.21.15.113:0/1119721549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d78d27d0 con 0x5560d7bb8560 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.727+0000 7f91c8d63640 1 --2- 172.21.15.113:0/1119721549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7b90860 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f91b4010010 tx=0x7f91b401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91c2b41cc49c016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91a67fc640 1 -- 172.21.15.113:0/1119721549 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4037070 con 0x5560d7bb8560 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91a67fc640 1 -- 172.21.15.113:0/1119721549 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b400ea80 con 0x5560d7bb8560 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91a67fc640 1 -- 172.21.15.113:0/1119721549 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4017040 con 0x5560d7bb8560 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1119721549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 msgr2=0x5560d7b90860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1119721549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7b90860 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f91b4010010 tx=0x7f91b401e040 comp rx=0 tx=0).stop 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1119721549 shutdown_connections 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1119721549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7b90da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1119721549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb8560 0x5560d7b90860 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/1119721549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb7c40 0x5560d7bb7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1119721549 >> 172.21.15.113:0/1119721549 conn(0x5560d7b8f950 msgr2=0x5560d7b8e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1119721549 shutdown_connections 2024-04-25T00:01:08.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.728+0000 7f91ce5b1740 1 -- 172.21.15.113:0/1119721549 wait complete. 2024-04-25T00:01:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.729+0000 7f91ce5b1740 1 -- start start 2024-04-25T00:01:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.729+0000 7f91ce5b1740 1 Processor -- start 2024-04-25T00:01:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb7c40 0x5560d7b90e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bfd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7c53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41543a0 con 0x5560d7bb7c40 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5560d7bd35f0 con 0x5560d7bea590 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5560d7c53f70 con 0x5560d7bb8560 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 10 client.0 _ll_register_callbacks cb 0x7f91b40097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb7c40 0x5560d7b90e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb7c40 0x5560d7b90e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33020/0 (socket says 172.21.15.113:33020) 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 -- 172.21.15.113:0/622031626 learned_addr learned my addr 172.21.15.113:0/622031626 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c8d63640 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bfd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9d65640 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7c53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 -- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 msgr2=0x5560d7bfd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bfd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 -- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 msgr2=0x5560d7c53230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7c53230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 -- 172.21.15.113:0/622031626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d7bd3380 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91c9564640 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb7c40 0x5560d7b90e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5560d7bd5490 tx=0x7f91c0014b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cb9e7b6cf816cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f919ffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f919ffff640 1 -- 172.21.15.113:0/622031626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c00237c0 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f919ffff640 1 -- 172.21.15.113:0/622031626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f91c0023960 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b400ea80 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f919ffff640 1 -- 172.21.15.113:0/622031626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f91c0038050 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 10 client.5946 fetch_fsmap learned FSMap version 22 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.730+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5560d7bfdca0 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f919ffff640 1 -- 172.21.15.113:0/622031626 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b400ea80 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f919ffff640 1 -- 172.21.15.113:0/622031626 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f91c003b020 con 0x5560d7bb7c40 2024-04-25T00:01:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 10 client.5946 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 10 client.5946 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 20 client.5946 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7c53770 0x5560d7be8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 4 client.5946 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560d7afbf00 con 0x5560d7c53770 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91c8d63640 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7c53770 0x5560d7be8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f919ffff640 10 client.5946 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91c8d63640 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7c53770 0x5560d7be8b40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c469aafad90cf5de server_cookie=bf2b145c4de20985 in_seq=0 out_seq=0 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f919ffff640 1 -- 172.21.15.113:0/622031626 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+168 (crc 0 0 0) 0x5560d7afbf00 con 0x5560d7c53770 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f919ffff640 10 client.5946 handle_command_reply: tid=0 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 client.5946 shutdown 2024-04-25T00:01:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb7c40 msgr2=0x5560d7b90e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb7c40 0x5560d7b90e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5560d7bd5490 tx=0x7f91c0014b20 comp rx=0 tx=0).stop 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 shutdown_connections 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560d7c53770 0x5560d7be8b40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560d7bea590 0x5560d7c53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560d7bb8560 0x5560d7bfd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 --2- 172.21.15.113:0/622031626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560d7bb7c40 0x5560d7b90e00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f919ffff640 0 client.5946 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.731+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 >> 172.21.15.113:0/622031626 conn(0x5560d7b8f950 msgr2=0x5560d7b86a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 shutdown_connections 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91ce5b1740 1 -- 172.21.15.113:0/622031626 wait complete. 2024-04-25T00:01:08.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91ce5b1740 20 client.5946 trim_cache size 0 max 16384 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "damage_type": "backtrace", 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 1690121373, 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 1099511630412, 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:08.739 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:01:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91a806be60 msgr2=0x7f91a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91a806be60 0x7f91a806e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91c00019b0 tx=0x7f91c0009040 comp rx=0 tx=0).stop 2024-04-25T00:01:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91c4157ad0 msgr2=0x7f91c4157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91c4157ad0 0x7f91c4157f50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c210 tx=0x7f91c415f9a0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 shutdown_connections 2024-04-25T00:01:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91a806be60 0x7f91a806e320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91c4157ad0 0x7f91c4157f50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91c4155cd0 0x7f91c414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 --2- 172.21.15.113:0/3401775625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b0005210 0x7f91c4150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 >> 172.21.15.113:0/3401775625 conn(0x7f91c406b0f0 msgr2=0x7f91c40af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 shutdown_connections 2024-04-25T00:01:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.732+0000 7f91cb7c8640 1 -- 172.21.15.113:0/3401775625 wait complete. 2024-04-25T00:01:08.743 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:08.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.839+0000 7f477f916640 1 Processor -- start 2024-04-25T00:01:08.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477f916640 1 -- start start 2024-04-25T00:01:08.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477f916640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477815e320 0x7f47780d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477f916640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f47780d2290 0x7f47780b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477f916640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f47780d0b00 0x7f47780d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477f916640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f47780d14d0 con 0x7f47780d0b00 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477f916640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f47780d3410 con 0x7f47780d2290 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477f916640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47780d3de0 con 0x7f477815e320 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477deb3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f47780d0b00 0x7f47780d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.840+0000 7f477deb3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f47780d0b00 0x7f47780d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33024/0 (socket says 172.21.15.113:33024) 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477deb3640 1 -- 172.21.15.113:0/2962452394 learned_addr learned my addr 172.21.15.113:0/2962452394 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477ceb1640 1 --2- 172.21.15.113:0/2962452394 >> v2:172.21.15.178:3300/0 conn(0x7f47780d2290 0x7f47780b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477d6b2640 1 --2- 172.21.15.113:0/2962452394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477815e320 0x7f47780d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477deb3640 1 -- 172.21.15.113:0/2962452394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477815e320 msgr2=0x7f47780d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477deb3640 1 --2- 172.21.15.113:0/2962452394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477815e320 0x7f47780d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477deb3640 1 -- 172.21.15.113:0/2962452394 >> v2:172.21.15.178:3300/0 conn(0x7f47780d2290 msgr2=0x7f47780b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477deb3640 1 --2- 172.21.15.113:0/2962452394 >> v2:172.21.15.178:3300/0 conn(0x7f47780d2290 0x7f47780b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477deb3640 1 -- 172.21.15.113:0/2962452394 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47780d2800 con 0x7f47780d0b00 2024-04-25T00:01:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.841+0000 7f477deb3640 1 --2- 172.21.15.113:0/2962452394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d0b00 0x7f47780d0f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4778072720 tx=0x7f477002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23508dd86e1a2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f476ffff640 1 -- 172.21.15.113:0/2962452394 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770037070 con 0x7f47780d0b00 2024-04-25T00:01:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f476ffff640 1 -- 172.21.15.113:0/2962452394 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4770032440 con 0x7f47780d0b00 2024-04-25T00:01:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f476ffff640 1 -- 172.21.15.113:0/2962452394 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770032810 con 0x7f47780d0b00 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 -- 172.21.15.113:0/2962452394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d0b00 msgr2=0x7f47780d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 --2- 172.21.15.113:0/2962452394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d0b00 0x7f47780d0f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4778072720 tx=0x7f477002a270 comp rx=0 tx=0).stop 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 -- 172.21.15.113:0/2962452394 shutdown_connections 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 --2- 172.21.15.113:0/2962452394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d0b00 0x7f47780d0f90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 --2- 172.21.15.113:0/2962452394 >> v2:172.21.15.178:3300/0 conn(0x7f47780d2290 0x7f47780b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 --2- 172.21.15.113:0/2962452394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477815e320 0x7f47780d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 -- 172.21.15.113:0/2962452394 >> 172.21.15.113:0/2962452394 conn(0x7f47780b5c20 msgr2=0x7f47780b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 -- 172.21.15.113:0/2962452394 shutdown_connections 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.842+0000 7f477f916640 1 -- 172.21.15.113:0/2962452394 wait complete. 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.843+0000 7f477f916640 1 Processor -- start 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.843+0000 7f477f916640 1 -- start start 2024-04-25T00:01:08.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477f916640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47780d0b00 0x7f4778160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477f916640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d2290 0x7f4778160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477f916640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477815e320 0x7f47781f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477f916640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f47780d30a0 con 0x7f47780d2290 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477f916640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47780d2dc0 con 0x7f477815e320 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477f916640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47780b9fe0 con 0x7f47780d0b00 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d2290 0x7f4778160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d2290 0x7f4778160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33032/0 (socket says 172.21.15.113:33032) 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 -- 172.21.15.113:0/310540909 learned_addr learned my addr 172.21.15.113:0/310540909 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477d6b2640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47780d0b00 0x7f4778160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477deb3640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477815e320 0x7f47781f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 -- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47780d0b00 msgr2=0x7f4778160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47780d0b00 0x7f4778160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 -- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477815e320 msgr2=0x7f47781f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477815e320 0x7f47781f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.844+0000 7f477ceb1640 1 -- 172.21.15.113:0/310540909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4778158c20 con 0x7f47780d2290 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.845+0000 7f477ceb1640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d2290 0x7f4778160ea0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f476800d800 tx=0x7f476800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.845+0000 7f476dffb640 1 -- 172.21.15.113:0/310540909 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768018020 con 0x7f47780d2290 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.845+0000 7f476dffb640 1 -- 172.21.15.113:0/310540909 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47680096a0 con 0x7f47780d2290 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.845+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47780cbd90 con 0x7f47780d2290 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.845+0000 7f476dffb640 1 -- 172.21.15.113:0/310540909 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768009a70 con 0x7f47780d2290 2024-04-25T00:01:08.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.845+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47780cc350 con 0x7f47780d2290 2024-04-25T00:01:08.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.846+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47780691a0 con 0x7f47780d2290 2024-04-25T00:01:08.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.846+0000 7f476dffb640 1 -- 172.21.15.113:0/310540909 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f476801b020 con 0x7f47780d2290 2024-04-25T00:01:08.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.847+0000 7f476dffb640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f475c06be30 0x7f475c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.847+0000 7f477d6b2640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f475c06be30 0x7f475c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.847+0000 7f476dffb640 1 -- 172.21.15.113:0/310540909 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f476808afc0 con 0x7f47780d2290 2024-04-25T00:01:08.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.847+0000 7f477d6b2640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f475c06be30 0x7f475c06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4774003aa0 tx=0x7f4774009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:08.851+0000 7f476dffb640 1 -- 172.21.15.113:0/310540909 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4768022030 con 0x7f47780d2290 2024-04-25T00:01:09.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.060+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47780cdff0 con 0x7f47780d2290 2024-04-25T00:01:09.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.060+0000 7f476dffb640 1 -- 172.21.15.113:0/310540909 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5190 (secure 0 0 0) 0x7f47680599c0 con 0x7f47780d2290 2024-04-25T00:01:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:09.061 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:06.192091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:09.061 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f475c06be30 msgr2=0x7f475c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f475c06be30 0x7f475c06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4774003aa0 tx=0x7f4774009830 comp rx=0 tx=0).stop 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d2290 msgr2=0x7f4778160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d2290 0x7f4778160ea0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f476800d800 tx=0x7f476800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 shutdown_connections 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f475c06be30 0x7f475c06e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477815e320 0x7f47781f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47780d2290 0x7f4778160ea0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 --2- 172.21.15.113:0/310540909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47780d0b00 0x7f4778160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.062+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 >> 172.21.15.113:0/310540909 conn(0x7f47780b5c20 msgr2=0x7f4778071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.063+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 shutdown_connections 2024-04-25T00:01:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.063+0000 7f477f916640 1 -- 172.21.15.113:0/310540909 wait complete. 2024-04-25T00:01:09.074 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 1690121373 2024-04-25T00:01:09.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.175+0000 7ff30f763640 1 Processor -- start 2024-04-25T00:01:09.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.175+0000 7ff30f763640 1 -- start start 2024-04-25T00:01:09.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30f763640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff30813f050 0x7ff3080cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30f763640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3080d04f0 0x7ff30815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30f763640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3081604d0 0x7ff30806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30f763640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff3080d09a0 con 0x7ff3080d04f0 2024-04-25T00:01:09.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30f763640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff30815c6a0 con 0x7ff30813f050 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30f763640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff30815d6b0 con 0x7ff3081604d0 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3080d04f0 0x7ff30815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3080d04f0 0x7ff30815ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33044/0 (socket says 172.21.15.113:33044) 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 -- 172.21.15.113:0/1092511141 learned_addr learned my addr 172.21.15.113:0/1092511141 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/1092511141 >> v2:172.21.15.178:3300/0 conn(0x7ff30813f050 0x7ff3080cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30dd00640 1 --2- 172.21.15.113:0/1092511141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3081604d0 0x7ff30806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 -- 172.21.15.113:0/1092511141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3081604d0 msgr2=0x7ff30806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/1092511141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3081604d0 0x7ff30806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 -- 172.21.15.113:0/1092511141 >> v2:172.21.15.178:3300/0 conn(0x7ff30813f050 msgr2=0x7ff3080cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/1092511141 >> v2:172.21.15.178:3300/0 conn(0x7ff30813f050 0x7ff3080cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30ccfe640 1 -- 172.21.15.113:0/1092511141 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30815c8f0 con 0x7ff3080d04f0 2024-04-25T00:01:09.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.176+0000 7ff30dd00640 1 --2- 172.21.15.113:0/1092511141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3081604d0 0x7ff30806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/1092511141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3080d04f0 0x7ff30815ff90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff30806f550 tx=0x7ff2f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5508eab324aece90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff2fffff640 1 -- 172.21.15.113:0/1092511141 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8037070 con 0x7ff3080d04f0 2024-04-25T00:01:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff2fffff640 1 -- 172.21.15.113:0/1092511141 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f8032440 con 0x7ff3080d04f0 2024-04-25T00:01:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff2fffff640 1 -- 172.21.15.113:0/1092511141 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8032810 con 0x7ff3080d04f0 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 -- 172.21.15.113:0/1092511141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3080d04f0 msgr2=0x7ff30815ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 --2- 172.21.15.113:0/1092511141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3080d04f0 0x7ff30815ff90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff30806f550 tx=0x7ff2f802a140 comp rx=0 tx=0).stop 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 -- 172.21.15.113:0/1092511141 shutdown_connections 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 --2- 172.21.15.113:0/1092511141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3081604d0 0x7ff30806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 --2- 172.21.15.113:0/1092511141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3080d04f0 0x7ff30815ff90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 --2- 172.21.15.113:0/1092511141 >> v2:172.21.15.178:3300/0 conn(0x7ff30813f050 0x7ff3080cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 -- 172.21.15.113:0/1092511141 >> 172.21.15.113:0/1092511141 conn(0x7ff30806b0f0 msgr2=0x7ff30806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 -- 172.21.15.113:0/1092511141 shutdown_connections 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.177+0000 7ff30f763640 1 -- 172.21.15.113:0/1092511141 wait complete. 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.178+0000 7ff30f763640 1 Processor -- start 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30f763640 1 -- start start 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30f763640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3080d04f0 0x7ff3080aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30f763640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff30813f050 0x7ff30813ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30f763640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3081604d0 0x7ff3081540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30f763640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff30815d190 con 0x7ff3081604d0 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30f763640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff30815ceb0 con 0x7ff30813f050 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30f763640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff30813f430 con 0x7ff3080d04f0 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30dd00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3081604d0 0x7ff3081540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30dd00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3081604d0 0x7ff3081540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33054/0 (socket says 172.21.15.113:33054) 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.179+0000 7ff30dd00640 1 -- 172.21.15.113:0/2926780865 learned_addr learned my addr 172.21.15.113:0/2926780865 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff30813f050 0x7ff30813ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3080d04f0 0x7ff3080aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30dd00640 1 -- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3080d04f0 msgr2=0x7ff3080aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30dd00640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3080d04f0 0x7ff3080aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30dd00640 1 -- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff30813f050 msgr2=0x7ff30813ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30dd00640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff30813f050 0x7ff30813ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30dd00640 1 -- 172.21.15.113:0/2926780865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3080ade50 con 0x7ff3081604d0 2024-04-25T00:01:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30dd00640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3081604d0 0x7ff3081540d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff30000cd50 tx=0x7ff30000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff2fdffb640 1 -- 172.21.15.113:0/2926780865 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30000f680 con 0x7ff3081604d0 2024-04-25T00:01:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff2fdffb640 1 -- 172.21.15.113:0/2926780865 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30000f840 con 0x7ff3081604d0 2024-04-25T00:01:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3080ae130 con 0x7ff3081604d0 2024-04-25T00:01:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff2fdffb640 1 -- 172.21.15.113:0/2926780865 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30000fc30 con 0x7ff3081604d0 2024-04-25T00:01:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.180+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3080ae6f0 con 0x7ff3081604d0 2024-04-25T00:01:09.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.181+0000 7ff2fdffb640 1 -- 172.21.15.113:0/2926780865 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3000213f0 con 0x7ff3081604d0 2024-04-25T00:01:09.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.182+0000 7ff2fdffb640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2ec06be30 0x7ff2ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.182+0000 7ff2fdffb640 1 -- 172.21.15.113:0/2926780865 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7ff30008bc40 con 0x7ff3081604d0 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.182+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2ec06be30 0x7ff2ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.182+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2ec06be30 0x7ff2ec06e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff304006fd0 tx=0x7ff304008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.183+0000 7ff312474740 1 Processor -- start 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.183+0000 7ff312474740 1 -- start start 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.183+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771b050 0x55bda76cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.183+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.183+0000 7ff312474740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fda30 0x55bda76d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.183+0000 7ff312474740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bda7636230 con 0x55bda76fda30 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.183+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bda75de890 con 0x55bda76fe170 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bda76321f0 con 0x55bda771b050 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fda30 0x55bda76d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fda30 0x55bda76d2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33060/0 (socket says 172.21.15.113:33060) 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 -- 172.21.15.113:0/3541640333 learned_addr learned my addr 172.21.15.113:0/3541640333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771b050 0x55bda76cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 -- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771b050 msgr2=0x55bda76cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771b050 0x55bda76cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 -- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 msgr2=0x55bda76fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 -- 172.21.15.113:0/3541640333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda75f6250 con 0x55bda76fda30 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771b050 0x55bda76cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff30dd00640 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fda30 0x55bda76d2840 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff30008a0a0 tx=0x7ff300059b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669d50e7a9e18aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff2f2ffd640 1 -- 172.21.15.113:0/3541640333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300087020 con 0x55bda76fda30 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff2f2ffd640 1 -- 172.21.15.113:0/3541640333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30000fdd0 con 0x55bda76fda30 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.184+0000 7ff2f2ffd640 1 -- 172.21.15.113:0/3541640333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000581b0 con 0x55bda76fda30 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 -- 172.21.15.113:0/3541640333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fda30 msgr2=0x55bda76d2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fda30 0x55bda76d2840 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff30008a0a0 tx=0x7ff300059b20 comp rx=0 tx=0).stop 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 -- 172.21.15.113:0/3541640333 shutdown_connections 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fda30 0x55bda76d2840 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 --2- 172.21.15.113:0/3541640333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771b050 0x55bda76cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 -- 172.21.15.113:0/3541640333 >> 172.21.15.113:0/3541640333 conn(0x55bda76cfc50 msgr2=0x55bda77184f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 -- 172.21.15.113:0/3541640333 shutdown_connections 2024-04-25T00:01:09.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 -- 172.21.15.113:0/3541640333 wait complete. 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 -- start start 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.185+0000 7ff312474740 1 Processor -- start 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76f9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff312474740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771b050 0x55bda76fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda76fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff312474740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bda76d1e00 con 0x55bda771b050 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bda75f1e20 con 0x55bda76fe170 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bda76d11f0 con 0x55bda771a500 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff312474740 10 client.0 _ll_register_callbacks cb 0x55bda75f1810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff30ccfe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771b050 0x55bda76fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff30ccfe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771b050 0x55bda76fc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33064/0 (socket says 172.21.15.113:33064) 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.186+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2704724404 learned_addr learned my addr 172.21.15.113:0/2704724404 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30dd00640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda76fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 msgr2=0x55bda76fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda76fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 msgr2=0x55bda76f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2704724404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda7416710 con 0x55bda771b050 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30dd00640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda76fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771b050 0x55bda76fc960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80090b0 tx=0x7ff2f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d14037ca59f6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2704724404 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8037040 con 0x55bda771b050 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2704724404 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7ff2f8046630 con 0x55bda771b050 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30000fdd0 con 0x55bda771b050 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2704724404 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff2f804e050 con 0x55bda771b050 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 10 client.5964 fetch_fsmap learned FSMap version 22 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55bda7476ea0 con 0x55bda771b050 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2704724404 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30000fdd0 con 0x55bda771b050 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2704724404 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7ff2f8051020 con 0x55bda771b050 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 10 client.5964 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 10 client.5964 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 20 client.5964 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda772be20 0x55bda77187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 4 client.5964 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.187+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bda7643770 con 0x55bda772be20 2024-04-25T00:01:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda772be20 0x55bda77187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda772be20 0x55bda77187f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce6d17b37ee95c0 server_cookie=8ea33ccec34f9d0b in_seq=0 out_seq=0 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff2e7fff640 10 client.5964 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2704724404 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bda7643770 con 0x55bda772be20 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff2e7fff640 10 client.5964 handle_command_reply: tid=0 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff312474740 1 client.5964 shutdown 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771b050 msgr2=0x55bda76fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff312474740 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771b050 0x55bda76fc960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80090b0 tx=0x7ff2f8004730 comp rx=0 tx=0).stop 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 shutdown_connections 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff312474740 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda772be20 0x55bda77187f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.188+0000 7ff312474740 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda76fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.189+0000 7ff2e7fff640 0 client.5964 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:09.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.189+0000 7ff312474740 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771b050 0x55bda76fc960 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.189+0000 7ff312474740 1 --2- 172.21.15.113:0/2704724404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda76fe170 0x55bda76f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.189+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 >> 172.21.15.113:0/2704724404 conn(0x55bda76fda30 msgr2=0x55bda76d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.189+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 shutdown_connections 2024-04-25T00:01:09.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.189+0000 7ff312474740 1 -- 172.21.15.113:0/2704724404 wait complete. 2024-04-25T00:01:09.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.189+0000 7ff312474740 20 client.5964 trim_cache size 0 max 16384 2024-04-25T00:01:09.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.202+0000 7ff312474740 1 Processor -- start 2024-04-25T00:01:09.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.202+0000 7ff312474740 1 -- start start 2024-04-25T00:01:09.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff312474740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fe170 0x55bda7717430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda7717970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 0x55bda7718770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff312474740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081fd6f0 con 0x55bda76fe170 2024-04-25T00:01:09.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3080ab330 con 0x55bda771b050 2024-04-25T00:01:09.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff3080ab980 con 0x55bda771a500 2024-04-25T00:01:09.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fe170 0x55bda7717430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fe170 0x55bda7717430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33072/0 (socket says 172.21.15.113:33072) 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 -- 172.21.15.113:0/953379584 learned_addr learned my addr 172.21.15.113:0/953379584 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda7717970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 -- 172.21.15.113:0/953379584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 msgr2=0x55bda7717970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda7717970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 -- 172.21.15.113:0/953379584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 msgr2=0x55bda7718770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 0x55bda7718770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 -- 172.21.15.113:0/953379584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda76d0d90 con 0x55bda76fe170 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fe170 0x55bda7717430 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff30400ebd0 tx=0x7ff304014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7476046f8e64166f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff2f27fc640 1 -- 172.21.15.113:0/953379584 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30402e070 con 0x55bda76fe170 2024-04-25T00:01:09.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff2f27fc640 1 -- 172.21.15.113:0/953379584 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3040281d0 con 0x55bda76fe170 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.203+0000 7ff2f27fc640 1 -- 172.21.15.113:0/953379584 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400f040 con 0x55bda76fe170 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 -- 172.21.15.113:0/953379584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fe170 msgr2=0x55bda7717430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fe170 0x55bda7717430 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff30400ebd0 tx=0x7ff304014000 comp rx=0 tx=0).stop 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 -- 172.21.15.113:0/953379584 shutdown_connections 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 0x55bda7718770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda771a500 0x55bda7717970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 --2- 172.21.15.113:0/953379584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda76fe170 0x55bda7717430 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 -- 172.21.15.113:0/953379584 >> 172.21.15.113:0/953379584 conn(0x55bda76fda30 msgr2=0x55bda773c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 -- 172.21.15.113:0/953379584 shutdown_connections 2024-04-25T00:01:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.204+0000 7ff312474740 1 -- 172.21.15.113:0/953379584 wait complete. 2024-04-25T00:01:09.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.205+0000 7ff312474740 1 -- start start 2024-04-25T00:01:09.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.205+0000 7ff312474740 1 Processor -- start 2024-04-25T00:01:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda76fe170 0x55bda772eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771a500 0x55bda7718c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 0x55bda7700470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bda772bca0 con 0x55bda771a500 2024-04-25T00:01:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bda7728f80 con 0x55bda771b050 2024-04-25T00:01:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bda772c290 con 0x55bda76fe170 2024-04-25T00:01:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 10 client.0 _ll_register_callbacks cb 0x7ff2f8032820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771a500 0x55bda7718c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771a500 0x55bda7718c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33084/0 (socket says 172.21.15.113:33084) 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2595995947 learned_addr learned my addr 172.21.15.113:0/2595995947 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda76fe170 0x55bda772eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30dd00640 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 0x55bda7700470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda76fe170 msgr2=0x55bda772eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda76fe170 0x55bda772eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 msgr2=0x55bda7700470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 0x55bda7700470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 -- 172.21.15.113:0/2595995947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda730a000 con 0x55bda771a500 2024-04-25T00:01:09.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff30ccfe640 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771a500 0x55bda7718c30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009210 tx=0x7ff2f8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427fd4e3a2e53f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2595995947 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8057060 con 0x55bda771a500 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2595995947 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7ff2f8004450 con 0x55bda771a500 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3040281d0 con 0x55bda771a500 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2595995947 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff2f8037040 con 0x55bda771a500 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 10 client.5970 fetch_fsmap learned FSMap version 22 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.206+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x55bda7732760 con 0x55bda771a500 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2595995947 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3040281d0 con 0x55bda771a500 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2595995947 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7ff2f8051070 con 0x55bda771a500 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 10 client.5970 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:01:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 10 client.5970 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 20 client.5970 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda77009b0 0x55bda770de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 4 client.5970 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1690121373}] 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1690121373}) -- 0x55bda763dac0 con 0x55bda77009b0 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda77009b0 0x55bda770de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff2e7fff640 10 client.5970 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff30d4ff640 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda77009b0 0x55bda770de80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8289a9ae2dcbb7c server_cookie=7b20c19ada8c0e59 in_seq=0 out_seq=0 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff2e7fff640 1 -- 172.21.15.113:0/2595995947 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55bda763dac0 con 0x55bda77009b0 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff2e7fff640 10 client.5970 handle_command_reply: tid=0 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 client.5970 shutdown 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771a500 msgr2=0x55bda7718c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771a500 0x55bda7718c30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009210 tx=0x7ff2f8009240 comp rx=0 tx=0).stop 2024-04-25T00:01:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 shutdown_connections 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55bda77009b0 0x55bda770de80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bda771b050 0x55bda7700470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff2e7fff640 0 client.5970 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bda771a500 0x55bda7718c30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 --2- 172.21.15.113:0/2595995947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bda76fe170 0x55bda772eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.207+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 >> 172.21.15.113:0/2595995947 conn(0x55bda76fda30 msgr2=0x55bda7714680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 shutdown_connections 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff312474740 1 -- 172.21.15.113:0/2595995947 wait complete. 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff312474740 20 client.5970 trim_cache size 0 max 16384 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2ec06be30 msgr2=0x7ff2ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2ec06be30 0x7ff2ec06e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff304006fd0 tx=0x7ff304008040 comp rx=0 tx=0).stop 2024-04-25T00:01:09.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3081604d0 msgr2=0x7ff3081540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3081604d0 0x7ff3081540d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff30000cd50 tx=0x7ff30000a650 comp rx=0 tx=0).stop 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 shutdown_connections 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2ec06be30 0x7ff2ec06e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff30813f050 0x7ff30813ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3081604d0 0x7ff3081540d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 --2- 172.21.15.113:0/2926780865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3080d04f0 0x7ff3080aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 >> 172.21.15.113:0/2926780865 conn(0x7ff30806b0f0 msgr2=0x7ff3080cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 shutdown_connections 2024-04-25T00:01:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.208+0000 7ff30f763640 1 -- 172.21.15.113:0/2926780865 wait complete. 2024-04-25T00:01:09.220 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T00:01:09.220 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:09.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.317+0000 7f39225ce640 1 Processor -- start 2024-04-25T00:01:09.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.317+0000 7f39225ce640 1 -- start start 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f39225ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f391c06e6e0 0x7f391c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f39225ce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f391c1453c0 0x7f391c071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f39225ce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391c0cfad0 con 0x7f391c144d60 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f39225ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f391c145c40 con 0x7f391c06e6e0 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f39225ce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f391c0cf230 con 0x7f391c1453c0 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f391b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f391c06e6e0 0x7f391c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f391b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f391c06e6e0 0x7f391c0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33100/0 (socket says 172.21.15.113:33100) 2024-04-25T00:01:09.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.318+0000 7f391b7fe640 1 -- 172.21.15.113:0/2482369347 learned_addr learned my addr 172.21.15.113:0/2482369347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.319+0000 7f3920b6b640 1 --2- 172.21.15.113:0/2482369347 >> v2:172.21.15.178:3300/0 conn(0x7f391c1453c0 0x7f391c071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.319+0000 7f391b7fe640 1 -- 172.21.15.113:0/2482369347 >> v2:172.21.15.178:3300/0 conn(0x7f391c1453c0 msgr2=0x7f391c071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.319+0000 7f391b7fe640 1 --2- 172.21.15.113:0/2482369347 >> v2:172.21.15.178:3300/0 conn(0x7f391c1453c0 0x7f391c071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.319+0000 7f391b7fe640 1 -- 172.21.15.113:0/2482369347 >> v1:172.21.15.178:6789/0 conn(0x7f391c144d60 legacy=0x7f391c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.319+0000 7f391b7fe640 1 -- 172.21.15.113:0/2482369347 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c0cf510 con 0x7f391c06e6e0 2024-04-25T00:01:09.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.319+0000 7f3920b6b640 1 --2- 172.21.15.113:0/2482369347 >> v2:172.21.15.178:3300/0 conn(0x7f391c1453c0 0x7f391c071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:09.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.319+0000 7f391b7fe640 1 --2- 172.21.15.113:0/2482369347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b5390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f391c0733d0 tx=0x7f391002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8297b3a56100d0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f391affd640 1 -- 172.21.15.113:0/2482369347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910037070 con 0x7f391c06e6e0 2024-04-25T00:01:09.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f391affd640 1 -- 172.21.15.113:0/2482369347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910032440 con 0x7f391c06e6e0 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f391affd640 1 -- 172.21.15.113:0/2482369347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910032810 con 0x7f391c06e6e0 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 -- 172.21.15.113:0/2482369347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 msgr2=0x7f391c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 --2- 172.21.15.113:0/2482369347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b5390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f391c0733d0 tx=0x7f391002a150 comp rx=0 tx=0).stop 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 -- 172.21.15.113:0/2482369347 shutdown_connections 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 --2- 172.21.15.113:0/2482369347 >> v2:172.21.15.178:3300/0 conn(0x7f391c1453c0 0x7f391c071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 --2- 172.21.15.113:0/2482369347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b5390 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 -- 172.21.15.113:0/2482369347 >> 172.21.15.113:0/2482369347 conn(0x7f391c0b1a50 msgr2=0x7f391c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 -- 172.21.15.113:0/2482369347 shutdown_connections 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.320+0000 7f39225ce640 1 -- 172.21.15.113:0/2482369347 wait complete. 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.321+0000 7f39225ce640 1 Processor -- start 2024-04-25T00:01:09.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.321+0000 7f39225ce640 1 -- start start 2024-04-25T00:01:09.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f39225ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f39225ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f391c144d60 0x7f391c13bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f39225ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f391c1453c0 0x7f391c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f39225ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f391c145140 con 0x7f391c06e6e0 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f39225ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f391c070e10 con 0x7f391c144d60 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f39225ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f391c071870 con 0x7f391c1453c0 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f391bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f391bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33112/0 (socket says 172.21.15.113:33112) 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.322+0000 7f391bfff640 1 -- 172.21.15.113:0/1531966108 learned_addr learned my addr 172.21.15.113:0/1531966108 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f391b7fe640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f391c144d60 0x7f391c13bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f391bfff640 1 -- 172.21.15.113:0/1531966108 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f391c1453c0 msgr2=0x7f391c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f391bfff640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f391c1453c0 0x7f391c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f391bfff640 1 -- 172.21.15.113:0/1531966108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f391c144d60 msgr2=0x7f391c13bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f391bfff640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f391c144d60 0x7f391c13bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f391bfff640 1 -- 172.21.15.113:0/1531966108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c0b39a0 con 0x7f391c06e6e0 2024-04-25T00:01:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f391bfff640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b4a40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f390c00c400 tx=0x7f390c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f3918ff9640 1 -- 172.21.15.113:0/1531966108 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c0052d0 con 0x7f391c06e6e0 2024-04-25T00:01:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f3918ff9640 1 -- 172.21.15.113:0/1531966108 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390c005490 con 0x7f391c06e6e0 2024-04-25T00:01:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c0b3c80 con 0x7f391c06e6e0 2024-04-25T00:01:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f3918ff9640 1 -- 172.21.15.113:0/1531966108 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c007260 con 0x7f391c06e6e0 2024-04-25T00:01:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.323+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c0b4240 con 0x7f391c06e6e0 2024-04-25T00:01:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.324+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391c0691a0 con 0x7f391c06e6e0 2024-04-25T00:01:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.324+0000 7f3918ff9640 1 -- 172.21.15.113:0/1531966108 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f390c007400 con 0x7f391c06e6e0 2024-04-25T00:01:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.325+0000 7f3918ff9640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f390806be30 0x7f390806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.325+0000 7f391b7fe640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f390806be30 0x7f390806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.325+0000 7f3918ff9640 1 -- 172.21.15.113:0/1531966108 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f390c08a330 con 0x7f391c06e6e0 2024-04-25T00:01:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.325+0000 7f391b7fe640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f390806be30 0x7f390806e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3910034000 tx=0x7f391002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.329+0000 7f3918ff9640 1 -- 172.21.15.113:0/1531966108 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f390c058d30 con 0x7f391c06e6e0 2024-04-25T00:01:09.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.539+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f391c13c1a0 con 0x7f391c06e6e0 2024-04-25T00:01:09.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.540+0000 7f3918ff9640 1 -- 172.21.15.113:0/1531966108 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5190 (secure 0 0 0) 0x7f390c058630 con 0x7f391c06e6e0 2024-04-25T00:01:09.540 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:09.540 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:06.192091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:09.541 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-25T00:01:09.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f390806be30 msgr2=0x7f390806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f390806be30 0x7f390806e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3910034000 tx=0x7f391002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:09.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 msgr2=0x7f391c0b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b4a40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f390c00c400 tx=0x7f390c00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 shutdown_connections 2024-04-25T00:01:09.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f390806be30 0x7f390806e2f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f391c144d60 0x7f391c13bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f391c1453c0 0x7f391c0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 --2- 172.21.15.113:0/1531966108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f391c06e6e0 0x7f391c0b4a40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 >> 172.21.15.113:0/1531966108 conn(0x7f391c0b1a50 msgr2=0x7f391c1412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 shutdown_connections 2024-04-25T00:01:09.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.542+0000 7f39225ce640 1 -- 172.21.15.113:0/1531966108 wait complete. 2024-04-25T00:01:09.553 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T00:01:09.658 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:09.658 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:09.658 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:09.667 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:09.668 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:09.668 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T00:01:09.668 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T00:01:09.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:09.668 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T00:01:09.683 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:01:09.683 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:01:09.683 DEBUG:teuthology.orchestra.run.smithi113:> import stat 2024-04-25T00:01:09.683 DEBUG:teuthology.orchestra.run.smithi113:> import json 2024-04-25T00:01:09.683 DEBUG:teuthology.orchestra.run.smithi113:> import sys 2024-04-25T00:01:09.683 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> try: 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> except OSError as e: 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> sys.exit(e.errno) 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> print(json.dumps( 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> indent=2)) 2024-04-25T00:01:09.684 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout: "st_mode": 16877, 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout: "st_ino": 1099511627777, 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout: "st_dev": 43, 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout: "st_nlink": 16, 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout: "st_uid": 1000, 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout: "st_gid": 1316, 2024-04-25T00:01:09.789 INFO:teuthology.orchestra.run.smithi113.stdout: "st_size": 18, 2024-04-25T00:01:09.790 INFO:teuthology.orchestra.run.smithi113.stdout: "st_atime": 1714003256.0565155, 2024-04-25T00:01:09.790 INFO:teuthology.orchestra.run.smithi113.stdout: "st_mtime": 1714003262.1943831, 2024-04-25T00:01:09.790 INFO:teuthology.orchestra.run.smithi113.stdout: "st_ctime": 1714003262.1943831 2024-04-25T00:01:09.790 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:09.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:09.798 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T00:01:09.798 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T00:01:09.798 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:09.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.934+0000 7f2a3cbee640 1 Processor -- start 2024-04-25T00:01:09.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.934+0000 7f2a3cbee640 1 -- start start 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.935+0000 7f2a3cbee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 0x7f2a380b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.935+0000 7f2a3cbee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2a381453c0 0x7f2a3806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.935+0000 7f2a3cbee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a3806e510 0x7f2a3815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.935+0000 7f2a3cbee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2a38145c40 con 0x7f2a3806e510 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.935+0000 7f2a3cbee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2a3806e940 con 0x7f2a381453c0 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.935+0000 7f2a3cbee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2a380cfad0 con 0x7f2a38144d60 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a3806e510 0x7f2a3815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a3806e510 0x7f2a3815fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33118/0 (socket says 172.21.15.113:33118) 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 -- 172.21.15.113:0/141503627 learned_addr learned my addr 172.21.15.113:0/141503627 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a35d9b640 1 --2- 172.21.15.113:0/141503627 >> v2:172.21.15.178:3300/0 conn(0x7f2a381453c0 0x7f2a3806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a3659c640 1 --2- 172.21.15.113:0/141503627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 0x7f2a380b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 -- 172.21.15.113:0/141503627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 msgr2=0x7f2a380b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 --2- 172.21.15.113:0/141503627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 0x7f2a380b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 -- 172.21.15.113:0/141503627 >> v2:172.21.15.178:3300/0 conn(0x7f2a381453c0 msgr2=0x7f2a3806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:09.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 --2- 172.21.15.113:0/141503627 >> v2:172.21.15.178:3300/0 conn(0x7f2a381453c0 0x7f2a3806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.936+0000 7f2a36d9d640 1 -- 172.21.15.113:0/141503627 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a380cf230 con 0x7f2a3806e510 2024-04-25T00:01:09.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a36d9d640 1 --2- 172.21.15.113:0/141503627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a3815fc30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2a3806f310 tx=0x7f2a2802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40910be8e340e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3559a640 1 -- 172.21.15.113:0/141503627 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28037070 con 0x7f2a3806e510 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3559a640 1 -- 172.21.15.113:0/141503627 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a28032440 con 0x7f2a3806e510 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3559a640 1 -- 172.21.15.113:0/141503627 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28032810 con 0x7f2a3806e510 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 -- 172.21.15.113:0/141503627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 msgr2=0x7f2a3815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/141503627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a3815fc30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2a3806f310 tx=0x7f2a2802a180 comp rx=0 tx=0).stop 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 -- 172.21.15.113:0/141503627 shutdown_connections 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/141503627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a3815fc30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/141503627 >> v2:172.21.15.178:3300/0 conn(0x7f2a381453c0 0x7f2a3806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/141503627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 0x7f2a380b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 -- 172.21.15.113:0/141503627 >> 172.21.15.113:0/141503627 conn(0x7f2a380b1a50 msgr2=0x7f2a380b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 -- 172.21.15.113:0/141503627 shutdown_connections 2024-04-25T00:01:09.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.937+0000 7f2a3cbee640 1 -- 172.21.15.113:0/141503627 wait complete. 2024-04-25T00:01:09.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 Processor -- start 2024-04-25T00:01:09.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 -- start start 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a38143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 0x7f2a38157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a381453c0 0x7f2a381f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2a38070e10 con 0x7f2a3806e510 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2a380cf7f0 con 0x7f2a381453c0 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3cbee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2a38145140 con 0x7f2a38144d60 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.939+0000 7f2a3659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a38143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a38143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33130/0 (socket says 172.21.15.113:33130) 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 -- 172.21.15.113:0/3969047176 learned_addr learned my addr 172.21.15.113:0/3969047176 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a36d9d640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a381453c0 0x7f2a381f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 -- 172.21.15.113:0/3969047176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 msgr2=0x7f2a38157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 0x7f2a38157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 -- 172.21.15.113:0/3969047176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a381453c0 msgr2=0x7f2a381f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a381453c0 0x7f2a381f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 -- 172.21.15.113:0/3969047176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a38143880 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3659c640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a38143cb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2a30007e90 tx=0x7f2a3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a2effd640 1 -- 172.21.15.113:0/3969047176 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30014420 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a2effd640 1 -- 172.21.15.113:0/3969047176 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30019070 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a38142850 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a2effd640 1 -- 172.21.15.113:0/3969047176 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300147e0 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.940+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a38142e10 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.941+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a380691a0 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.941+0000 7f2a2effd640 1 -- 172.21.15.113:0/3969047176 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a30014980 con 0x7f2a3806e510 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.942+0000 7f2a2effd640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2a1806be60 0x7f2a1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:09.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.942+0000 7f2a2effd640 1 -- 172.21.15.113:0/3969047176 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f2a3008b680 con 0x7f2a3806e510 2024-04-25T00:01:09.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.942+0000 7f2a35d9b640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2a1806be60 0x7f2a1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:09.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.942+0000 7f2a35d9b640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2a1806be60 0x7f2a1806e320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a381434c0 tx=0x7f2a20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:09.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:09.946+0000 7f2a2effd640 1 -- 172.21.15.113:0/3969047176 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a30059ff0 con 0x7f2a3806e510 2024-04-25T00:01:10.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.155+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a380b27e0 con 0x7f2a3806e510 2024-04-25T00:01:10.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.155+0000 7f2a2effd640 1 -- 172.21.15.113:0/3969047176 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5190 (secure 0 0 0) 0x7f2a300598f0 con 0x7f2a3806e510 2024-04-25T00:01:10.156 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:10.156 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:06.192091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:10.156 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-25T00:01:10.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2a1806be60 msgr2=0x7f2a1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2a1806be60 0x7f2a1806e320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a381434c0 tx=0x7f2a20008040 comp rx=0 tx=0).stop 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 msgr2=0x7f2a38143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a38143cb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2a30007e90 tx=0x7f2a3000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 shutdown_connections 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2a1806be60 0x7f2a1806e320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a381453c0 0x7f2a381f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a38144d60 0x7f2a38157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 --2- 172.21.15.113:0/3969047176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3806e510 0x7f2a38143cb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 >> 172.21.15.113:0/3969047176 conn(0x7f2a380b1a50 msgr2=0x7f2a3813f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 shutdown_connections 2024-04-25T00:01:10.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.157+0000 7f2a3cbee640 1 -- 172.21.15.113:0/3969047176 wait complete. 2024-04-25T00:01:10.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T00:01:10.273 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:10.273 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": -2 2024-04-25T00:01:10.273 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:10.282 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T00:01:10.282 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T00:01:10.282 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T00:01:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.382+0000 7fe9be9ce640 1 Processor -- start 2024-04-25T00:01:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.382+0000 7fe9be9ce640 1 -- start start 2024-04-25T00:01:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9be9ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9b80b3080 0x7fe9b80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9be9ce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe9b815e6b0 0x7fe9b80b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9be9ce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b815f4f0 con 0x7fe9b815e2d0 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9be9ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b80cde00 con 0x7fe9b80b3080 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9be9ce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b815f7d0 con 0x7fe9b815e6b0 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9b77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9b80b3080 0x7fe9b80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9b77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9b80b3080 0x7fe9b80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33146/0 (socket says 172.21.15.113:33146) 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9b77fe640 1 -- 172.21.15.113:0/179203469 learned_addr learned my addr 172.21.15.113:0/179203469 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9b77fe640 1 -- 172.21.15.113:0/179203469 >> v2:172.21.15.178:3300/0 conn(0x7fe9b815e6b0 msgr2=0x7fe9b80b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/179203469 >> v2:172.21.15.178:3300/0 conn(0x7fe9b815e6b0 0x7fe9b80b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9b77fe640 1 -- 172.21.15.113:0/179203469 >> v1:172.21.15.178:6789/0 conn(0x7fe9b815e2d0 legacy=0x7fe9b815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.383+0000 7fe9b77fe640 1 -- 172.21.15.113:0/179203469 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b815fab0 con 0x7fe9b80b3080 2024-04-25T00:01:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/179203469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b80b3080 0x7fe9b80b4330 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80d0f20 tx=0x7fe9ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930cc0e0886504c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9b6ffd640 1 -- 172.21.15.113:0/179203469 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac037070 con 0x7fe9b80b3080 2024-04-25T00:01:10.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9b6ffd640 1 -- 172.21.15.113:0/179203469 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac032440 con 0x7fe9b80b3080 2024-04-25T00:01:10.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9b6ffd640 1 -- 172.21.15.113:0/179203469 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac032810 con 0x7fe9b80b3080 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 -- 172.21.15.113:0/179203469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b80b3080 msgr2=0x7fe9b80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/179203469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b80b3080 0x7fe9b80b4330 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80d0f20 tx=0x7fe9ac02a230 comp rx=0 tx=0).stop 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 -- 172.21.15.113:0/179203469 shutdown_connections 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/179203469 >> v2:172.21.15.178:3300/0 conn(0x7fe9b815e6b0 0x7fe9b80b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/179203469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b80b3080 0x7fe9b80b4330 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 -- 172.21.15.113:0/179203469 >> 172.21.15.113:0/179203469 conn(0x7fe9b806b0f0 msgr2=0x7fe9b806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 -- 172.21.15.113:0/179203469 shutdown_connections 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.384+0000 7fe9be9ce640 1 -- 172.21.15.113:0/179203469 wait complete. 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.385+0000 7fe9be9ce640 1 Processor -- start 2024-04-25T00:01:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9be9ce640 1 -- start start 2024-04-25T00:01:10.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9be9ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b80b3080 0x7fe9b81583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9be9ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b815e2d0 0x7fe9b8155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9be9ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9b815e6b0 0x7fe9b8155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9be9ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80ce410 con 0x7fe9b815e2d0 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9be9ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80ce290 con 0x7fe9b80b3080 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9be9ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b80ce590 con 0x7fe9b815e6b0 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b815e2d0 0x7fe9b8155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b815e2d0 0x7fe9b8155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33156/0 (socket says 172.21.15.113:33156) 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.386+0000 7fe9b77fe640 1 -- 172.21.15.113:0/2907130524 learned_addr learned my addr 172.21.15.113:0/2907130524 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b80b3080 0x7fe9b81583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9b815e6b0 0x7fe9b8155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b77fe640 1 -- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9b815e6b0 msgr2=0x7fe9b8155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9b815e6b0 0x7fe9b8155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b77fe640 1 -- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b80b3080 msgr2=0x7fe9b81583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b80b3080 0x7fe9b81583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b77fe640 1 -- 172.21.15.113:0/2907130524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b8157b10 con 0x7fe9b815e2d0 2024-04-25T00:01:10.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b815e2d0 0x7fe9b8155860 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac000c00 tx=0x7fe9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b4ff9640 1 -- 172.21.15.113:0/2907130524 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac041070 con 0x7fe9b815e2d0 2024-04-25T00:01:10.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b4ff9640 1 -- 172.21.15.113:0/2907130524 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac03c870 con 0x7fe9b815e2d0 2024-04-25T00:01:10.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9b4ff9640 1 -- 172.21.15.113:0/2907130524 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac037040 con 0x7fe9b815e2d0 2024-04-25T00:01:10.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b8157df0 con 0x7fe9b815e2d0 2024-04-25T00:01:10.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.387+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b815a7e0 con 0x7fe9b815e2d0 2024-04-25T00:01:10.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.389+0000 7fe9b4ff9640 1 -- 172.21.15.113:0/2907130524 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9ac044070 con 0x7fe9b815e2d0 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.389+0000 7fe9b4ff9640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9a406be30 0x7fe9a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.389+0000 7fe9b4ff9640 1 -- 172.21.15.113:0/2907130524 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe9ac0ab970 con 0x7fe9b815e2d0 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.389+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9a406be30 0x7fe9a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.390+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9a406be30 0x7fe9a406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80017f0 tx=0x7fe9a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.390+0000 7fe9c17b8740 1 Processor -- start 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.390+0000 7fe9c17b8740 1 -- start start 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719789d0 0x56217192b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x56217195ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195b3b0 0x5621719301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562171893570 con 0x56217195baf0 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56217183bba0 con 0x5621719789d0 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56217188f530 con 0x56217195b3b0 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x56217195ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x56217195ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33168/0 (socket says 172.21.15.113:33168) 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 -- 172.21.15.113:0/812536906 learned_addr learned my addr 172.21.15.113:0/812536906 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719789d0 0x56217192b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195b3b0 0x5621719301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 -- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195b3b0 msgr2=0x5621719301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195b3b0 0x5621719301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 -- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719789d0 msgr2=0x56217192b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719789d0 0x56217192b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 -- 172.21.15.113:0/812536906 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621716739f0 con 0x56217195baf0 2024-04-25T00:01:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x56217195ae70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac076180 tx=0x7fe9ac079de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81049c26a76934e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe99dffb640 1 -- 172.21.15.113:0/812536906 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac04b040 con 0x56217195baf0 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe99dffb640 1 -- 172.21.15.113:0/812536906 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac078020 con 0x56217195baf0 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.391+0000 7fe99dffb640 1 -- 172.21.15.113:0/812536906 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac03cc40 con 0x56217195baf0 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 -- 172.21.15.113:0/812536906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 msgr2=0x56217195ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x56217195ae70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac076180 tx=0x7fe9ac079de0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 -- 172.21.15.113:0/812536906 shutdown_connections 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195b3b0 0x5621719301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x56217195ae70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/812536906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719789d0 0x56217192b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 -- 172.21.15.113:0/812536906 >> 172.21.15.113:0/812536906 conn(0x56217192d5d0 msgr2=0x562171975e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 -- 172.21.15.113:0/812536906 shutdown_connections 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.392+0000 7fe9c17b8740 1 -- 172.21.15.113:0/812536906 wait complete. 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 -- start start 2024-04-25T00:01:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 Processor -- start 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195baf0 0x562171972780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x562171977800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171974a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562171893570 con 0x5621719789d0 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56217184f130 con 0x5621719566c0 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56217192f780 con 0x56217195baf0 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.393+0000 7fe9c17b8740 10 client.0 _ll_register_callbacks cb 0x56217184eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x562171977800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x562171977800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33176/0 (socket says 172.21.15.113:33176) 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 -- 172.21.15.113:0/257383423 learned_addr learned my addr 172.21.15.113:0/257383423 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171974a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195baf0 0x562171972780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 -- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195baf0 msgr2=0x562171972780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195baf0 0x562171972780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 -- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 msgr2=0x562171974a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171974a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 -- 172.21.15.113:0/257383423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621715672e0 con 0x5621719789d0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x562171977800 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0a9f40 tx=0x7fe9ac0091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1003c3768e4e35b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe996ffd640 1 -- 172.21.15.113:0/257383423 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac096060 con 0x5621719789d0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe996ffd640 1 -- 172.21.15.113:0/257383423 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe9ac0825c0 con 0x5621719789d0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9ac078020 con 0x5621719789d0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe996ffd640 1 -- 172.21.15.113:0/257383423 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe9ac04b040 con 0x5621719789d0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9c17b8740 10 client.5994 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5621716d4180 con 0x5621719789d0 2024-04-25T00:01:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe996ffd640 1 -- 172.21.15.113:0/257383423 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac078020 con 0x5621719789d0 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe996ffd640 1 -- 172.21.15.113:0/257383423 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fe9ac079170 con 0x5621719789d0 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9c17b8740 10 client.5994 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9c17b8740 10 client.5994 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9c17b8740 20 client.5994 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.394+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56217195d030 0x562171975e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe9c17b8740 4 client.5994 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621718a0ab0 con 0x56217195d030 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56217195d030 0x562171975e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56217195d030 0x562171975e70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c982c5ac2af786e8 server_cookie=7579b0ce3aa887f0 in_seq=0 out_seq=0 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe996ffd640 10 client.5994 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe996ffd640 1 -- 172.21.15.113:0/257383423 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621718a0ab0 con 0x56217195d030 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe996ffd640 10 client.5994 handle_command_reply: tid=0 2024-04-25T00:01:10.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.395+0000 7fe9c17b8740 1 client.5994 shutdown 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 msgr2=0x562171977800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x562171977800 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0a9f40 tx=0x7fe9ac0091c0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 shutdown_connections 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x56217195d030 0x562171975e70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171974a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x562171977800 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/257383423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56217195baf0 0x562171972780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe996ffd640 0 client.5994 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 >> 172.21.15.113:0/257383423 conn(0x56217195b3b0 msgr2=0x56217192ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 shutdown_connections 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 1 -- 172.21.15.113:0/257383423 wait complete. 2024-04-25T00:01:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.396+0000 7fe9c17b8740 20 client.5994 trim_cache size 0 max 16384 2024-04-25T00:01:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.413+0000 7fe9c17b8740 1 Processor -- start 2024-04-25T00:01:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.413+0000 7fe9c17b8740 1 -- start start 2024-04-25T00:01:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719566c0 0x562171999a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56217195baf0 0x56217195d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x5621719e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b82008e0 con 0x5621719789d0 2024-04-25T00:01:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b81ff920 con 0x56217195baf0 2024-04-25T00:01:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b81fed50 con 0x5621719566c0 2024-04-25T00:01:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x5621719e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x5621719e9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33182/0 (socket says 172.21.15.113:33182) 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 -- 172.21.15.113:0/1042708357 learned_addr learned my addr 172.21.15.113:0/1042708357 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719566c0 0x562171999a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56217195baf0 0x56217195d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 -- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719566c0 msgr2=0x562171999a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719566c0 0x562171999a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 -- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56217195baf0 msgr2=0x56217195d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56217195baf0 0x56217195d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.414+0000 7fe9bcf6b640 1 -- 172.21.15.113:0/1042708357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621715403a0 con 0x5621719789d0 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x5621719e9cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009f20 tx=0x7fe9b000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8e61737b07ae80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe99d7fa640 1 -- 172.21.15.113:0/1042708357 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0016890 con 0x5621719789d0 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe99d7fa640 1 -- 172.21.15.113:0/1042708357 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b001c070 con 0x5621719789d0 2024-04-25T00:01:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe99d7fa640 1 -- 172.21.15.113:0/1042708357 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0016c30 con 0x5621719789d0 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1042708357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 msgr2=0x5621719e9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x5621719e9cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009f20 tx=0x7fe9b000e040 comp rx=0 tx=0).stop 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1042708357 shutdown_connections 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56217195baf0 0x56217195d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5621719789d0 0x5621719e9cd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1042708357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719566c0 0x562171999a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1042708357 >> 172.21.15.113:0/1042708357 conn(0x56217195b3b0 msgr2=0x56217192e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1042708357 shutdown_connections 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.415+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1042708357 wait complete. 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.416+0000 7fe9c17b8740 1 -- start start 2024-04-25T00:01:10.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.416+0000 7fe9c17b8740 1 Processor -- start 2024-04-25T00:01:10.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171954d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x562171955270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719789d0 0x5621719989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562171926ed0 con 0x56217195baf0 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b81ff2d0 con 0x5621719566c0 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562171977be0 con 0x5621719789d0 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 10 client.0 _ll_register_callbacks cb 0x7fe9ac0798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x562171955270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x562171955270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33198/0 (socket says 172.21.15.113:33198) 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 -- 172.21.15.113:0/1674227137 learned_addr learned my addr 172.21.15.113:0/1674227137 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171954d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719789d0 0x5621719989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 -- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719789d0 msgr2=0x5621719989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719789d0 0x5621719989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 -- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 msgr2=0x562171954d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171954d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 -- 172.21.15.113:0/1674227137 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621719e6cb0 con 0x56217195baf0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9bcf6b640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719789d0 0x5621719989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9b77fe640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x562171955270 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac077fd0 tx=0x7fe9ac07b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2489f13326d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe996ffd640 1 -- 172.21.15.113:0/1674227137 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0a3390 con 0x56217195baf0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe996ffd640 1 -- 172.21.15.113:0/1674227137 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe9ac0a3530 con 0x56217195baf0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.417+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b001c070 con 0x56217195baf0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe996ffd640 1 -- 172.21.15.113:0/1674227137 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe9ac080220 con 0x56217195baf0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 10 client.6000 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5621719e6cb0 con 0x56217195baf0 2024-04-25T00:01:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe996ffd640 1 -- 172.21.15.113:0/1674227137 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b001c070 con 0x56217195baf0 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe996ffd640 1 -- 172.21.15.113:0/1674227137 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fe9ac0803c0 con 0x56217195baf0 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 10 client.6000 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 10 client.6000 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 20 client.6000 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5621719586a0 0x562171998f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 4 client.6000 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) -- 0x56217189ae00 con 0x5621719586a0 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5621719586a0 0x562171998f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe996ffd640 10 client.6000 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.418+0000 7fe9b7fff640 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5621719586a0 0x562171998f20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c8921264bb70e2 server_cookie=751e24604755a5c6 in_seq=0 out_seq=0 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe996ffd640 1 -- 172.21.15.113:0/1674227137 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+26 (crc 0 0 0) 0x56217189ae00 con 0x5621719586a0 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe996ffd640 10 client.6000 handle_command_reply: tid=0 2024-04-25T00:01:10.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 client.6000 shutdown 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 msgr2=0x562171955270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x562171955270 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac077fd0 tx=0x7fe9ac07b000 comp rx=0 tx=0).stop 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 shutdown_connections 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5621719586a0 0x562171998f20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5621719789d0 0x5621719989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe996ffd640 0 client.6000 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56217195baf0 0x562171955270 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 --2- 172.21.15.113:0/1674227137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621719566c0 0x562171954d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 >> 172.21.15.113:0/1674227137 conn(0x56217195b3b0 msgr2=0x56217195d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 shutdown_connections 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 1 -- 172.21.15.113:0/1674227137 wait complete. 2024-04-25T00:01:10.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9c17b8740 20 client.6000 trim_cache size 0 max 16384 2024-04-25T00:01:10.427 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:10.427 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": -2 2024-04-25T00:01:10.427 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9a406be30 msgr2=0x7fe9a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9a406be30 0x7fe9a406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80017f0 tx=0x7fe9a8009040 comp rx=0 tx=0).stop 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b815e2d0 msgr2=0x7fe9b8155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b815e2d0 0x7fe9b8155860 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac000c00 tx=0x7fe9ac0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 shutdown_connections 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9a406be30 0x7fe9a406e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9b815e6b0 0x7fe9b8155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b815e2d0 0x7fe9b8155860 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 --2- 172.21.15.113:0/2907130524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b80b3080 0x7fe9b81583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.419+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 >> 172.21.15.113:0/2907130524 conn(0x7fe9b806b0f0 msgr2=0x7fe9b80d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.420+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 shutdown_connections 2024-04-25T00:01:10.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.420+0000 7fe9be9ce640 1 -- 172.21.15.113:0/2907130524 wait complete. 2024-04-25T00:01:10.430 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T00:01:10.430 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T00:01:10.431 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.530+0000 7f81e0509640 1 Processor -- start 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.530+0000 7f81e0509640 1 -- start start 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81e0509640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f81d813f050 0x7f81d80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81e0509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81e0509640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d815d6b0 con 0x7f81d80d04f0 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81e0509640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f81d80d09a0 con 0x7f81d813f050 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81e0509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81d815c6a0 con 0x7f81d81606a0 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81deaa6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81de2a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f81d813f050 0x7f81d80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81deaa6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:57976/0 (socket says 172.21.15.113:57976) 2024-04-25T00:01:10.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.531+0000 7f81deaa6640 1 -- 172.21.15.113:0/1073671511 learned_addr learned my addr 172.21.15.113:0/1073671511 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81dd2a3640 1 -- 172.21.15.113:0/1073671511 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202489513 0 0) 0x7f81d815d6b0 con 0x7f81d80d04f0 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81dd2a3640 1 -- 172.21.15.113:0/1073671511 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81c4003520 con 0x7f81d80d04f0 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81de2a5640 1 -- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 msgr2=0x7f81d80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81de2a5640 1 --2- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81de2a5640 1 -- 172.21.15.113:0/1073671511 >> v1:172.21.15.178:6789/0 conn(0x7f81d80d04f0 legacy=0x7f81d815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81de2a5640 1 -- 172.21.15.113:0/1073671511 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d815c8f0 con 0x7f81d813f050 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81deaa6640 1 --2- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.532+0000 7f81de2a5640 1 --2- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 0x7f81d80cffb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81d40097a0 tx=0x7f81d402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c255c16f8ec9508f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81dd2a3640 1 -- 172.21.15.113:0/1073671511 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4037070 con 0x7f81d813f050 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81dd2a3640 1 -- 172.21.15.113:0/1073671511 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d402ae50 con 0x7f81d813f050 2024-04-25T00:01:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81dd2a3640 1 -- 172.21.15.113:0/1073671511 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4032640 con 0x7f81d813f050 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 -- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 msgr2=0x7f81d80cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 --2- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 0x7f81d80cffb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81d40097a0 tx=0x7f81d402a100 comp rx=0 tx=0).stop 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 -- 172.21.15.113:0/1073671511 shutdown_connections 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 --2- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 --2- 172.21.15.113:0/1073671511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 0x7f81d80cffb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 -- 172.21.15.113:0/1073671511 >> 172.21.15.113:0/1073671511 conn(0x7f81d806b0f0 msgr2=0x7f81d806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 -- 172.21.15.113:0/1073671511 shutdown_connections 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.533+0000 7f81e0509640 1 -- 172.21.15.113:0/1073671511 wait complete. 2024-04-25T00:01:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.534+0000 7f81e0509640 1 Processor -- start 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.534+0000 7f81e0509640 1 -- start start 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81e0509640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80d04f0 0x7f81d80afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81e0509640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 0x7f81d815b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81e0509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d8150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81e0509640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81d815d190 con 0x7f81d80d04f0 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81e0509640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81d815ceb0 con 0x7f81d813f050 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81e0509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81d813f430 con 0x7f81d81606a0 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80d04f0 0x7f81d80afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80d04f0 0x7f81d80afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33204/0 (socket says 172.21.15.113:33204) 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 -- 172.21.15.113:0/3579218145 learned_addr learned my addr 172.21.15.113:0/3579218145 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81deaa6640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d8150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 -- 172.21.15.113:0/3579218145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 msgr2=0x7f81d8150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d8150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 -- 172.21.15.113:0/3579218145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 msgr2=0x7f81d815b420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 0x7f81d815b420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.535+0000 7f81de2a5640 1 -- 172.21.15.113:0/3579218145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d80b0720 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.536+0000 7f81de2a5640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80d04f0 0x7f81d80afe10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f81d4035040 tx=0x7f81d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.536+0000 7f81ceffd640 1 -- 172.21.15.113:0/3579218145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4041020 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.536+0000 7f81ceffd640 1 -- 172.21.15.113:0/3579218145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d403c670 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.536+0000 7f81ceffd640 1 -- 172.21.15.113:0/3579218145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4037040 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.536+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d81f7920 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.536+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d81f8400 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.537+0000 7f81ceffd640 1 -- 172.21.15.113:0/3579218145 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81d4044020 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.538+0000 7f81ceffd640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.538+0000 7f81ceffd640 1 -- 172.21.15.113:0/3579218145 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f81d40ab4e0 con 0x7f81d80d04f0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.538+0000 7f81ddaa4640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.538+0000 7f81ddaa4640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f81d8156850 tx=0x7f81c8007840 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.539+0000 7f81e3214740 1 Processor -- start 2024-04-25T00:01:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.539+0000 7f81e3214740 1 -- start start 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602030220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x55660204dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81e3214740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556602005b20 0x55660204bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81e3214740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556601f694b0 con 0x556602005b20 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556601f11b30 con 0x55660204d6e0 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556601f65470 con 0x55660202fe40 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556602005b20 0x55660204bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556602005b20 0x55660204bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33206/0 (socket says 172.21.15.113:33206) 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 -- 172.21.15.113:0/3891417988 learned_addr learned my addr 172.21.15.113:0/3891417988 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81ddaa4640 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x55660204dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 -- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 msgr2=0x556602030220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81de2a5640 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602030220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602030220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 -- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 msgr2=0x55660204dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x55660204dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 -- 172.21.15.113:0/3891417988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556601d49b00 con 0x556602005b20 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81deaa6640 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556602005b20 0x55660204bda0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009d80 tx=0x7f81d0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cecd1eb0be154d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.540+0000 7f81bbfff640 1 -- 172.21.15.113:0/3891417988 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00147f0 con 0x556602005b20 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81bbfff640 1 -- 172.21.15.113:0/3891417988 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d001a070 con 0x556602005b20 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81bbfff640 1 -- 172.21.15.113:0/3891417988 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0014b90 con 0x556602005b20 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 -- 172.21.15.113:0/3891417988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556602005b20 msgr2=0x55660204bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556602005b20 0x55660204bda0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009d80 tx=0x7f81d0009810 comp rx=0 tx=0).stop 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 -- 172.21.15.113:0/3891417988 shutdown_connections 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x55660204dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556602005b20 0x55660204bda0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 --2- 172.21.15.113:0/3891417988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602030220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 -- 172.21.15.113:0/3891417988 >> 172.21.15.113:0/3891417988 conn(0x556602002450 msgr2=0x55660204ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 -- 172.21.15.113:0/3891417988 shutdown_connections 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.541+0000 7f81e3214740 1 -- 172.21.15.113:0/3891417988 wait complete. 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.542+0000 7f81e3214740 1 -- start start 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.542+0000 7f81e3214740 1 Processor -- start 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81e3214740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202fe40 0x556602018880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x556601fff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202c600 0x556601fff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81e3214740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5566020068c0 con 0x55660202fe40 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556601f250d0 con 0x55660204d6e0 2024-04-25T00:01:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556602000380 con 0x55660202c600 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81e3214740 10 client.0 _ll_register_callbacks cb 0x556601c7cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202fe40 0x556602018880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202fe40 0x556602018880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33208/0 (socket says 172.21.15.113:33208) 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 -- 172.21.15.113:0/1135165860 learned_addr learned my addr 172.21.15.113:0/1135165860 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81deaa6640 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202c600 0x556601fff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81ddaa4640 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x556601fff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 -- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202c600 msgr2=0x556601fff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202c600 0x556601fff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 -- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 msgr2=0x556601fff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x556601fff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 -- 172.21.15.113:0/1135165860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556601c3d3f0 con 0x55660202fe40 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81de2a5640 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202fe40 0x556602018880 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f81d40794e0 tx=0x7f81d4079200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d22ba2711611aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.543+0000 7f81b97fa640 1 -- 172.21.15.113:0/1135165860 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40797e0 con 0x55660202fe40 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81b97fa640 1 -- 172.21.15.113:0/1135165860 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f81d40ab0e0 con 0x55660202fe40 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d001a070 con 0x55660202fe40 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81b97fa640 1 -- 172.21.15.113:0/1135165860 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f81d40ab300 con 0x55660202fe40 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 10 client.6009 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x556601daa290 con 0x55660202fe40 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81b97fa640 1 -- 172.21.15.113:0/1135165860 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d001a070 con 0x55660202fe40 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81b97fa640 1 -- 172.21.15.113:0/1135165860 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f81d4081f30 con 0x55660202fe40 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 10 client.6009 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 10 client.6009 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 20 client.6009 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204c0a0 0x55660204c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 4 client.6009 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556601f769f0 con 0x55660204c0a0 2024-04-25T00:01:10.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81ddaa4640 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204c0a0 0x55660204c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81b97fa640 10 client.6009 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.544+0000 7f81ddaa4640 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204c0a0 0x55660204c4f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9e05f4c0bce990 server_cookie=d70f1d1666445917 in_seq=0 out_seq=0 2024-04-25T00:01:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81b97fa640 1 -- 172.21.15.113:0/1135165860 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556601f769f0 con 0x55660204c0a0 2024-04-25T00:01:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81b97fa640 10 client.6009 handle_command_reply: tid=0 2024-04-25T00:01:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 client.6009 shutdown 2024-04-25T00:01:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202fe40 msgr2=0x556602018880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202fe40 0x556602018880 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f81d40794e0 tx=0x7f81d4079200 comp rx=0 tx=0).stop 2024-04-25T00:01:10.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 shutdown_connections 2024-04-25T00:01:10.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204c0a0 0x55660204c4f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660204d6e0 0x556601fff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81b97fa640 0 client.6009 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202c600 0x556601fff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 --2- 172.21.15.113:0/1135165860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202fe40 0x556602018880 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 >> 172.21.15.113:0/1135165860 conn(0x556602005b20 msgr2=0x55660204e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 shutdown_connections 2024-04-25T00:01:10.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 1 -- 172.21.15.113:0/1135165860 wait complete. 2024-04-25T00:01:10.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.545+0000 7f81e3214740 20 client.6009 trim_cache size 0 max 16384 2024-04-25T00:01:10.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 Processor -- start 2024-04-25T00:01:10.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 -- start start 2024-04-25T00:01:10.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202c600 0x556602032d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202fe40 0x556602017020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660204c0a0 0x556602017560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8200750 con 0x55660202c600 2024-04-25T00:01:10.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8071ef0 con 0x55660202fe40 2024-04-25T00:01:10.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81d8071250 con 0x55660204c0a0 2024-04-25T00:01:10.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81de2a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202c600 0x556602032d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81de2a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202c600 0x556602032d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33224/0 (socket says 172.21.15.113:33224) 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.563+0000 7f81de2a5640 1 -- 172.21.15.113:0/3890440378 learned_addr learned my addr 172.21.15.113:0/3890440378 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81de2a5640 1 -- 172.21.15.113:0/3890440378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660204c0a0 msgr2=0x556602017560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81de2a5640 1 --2- 172.21.15.113:0/3890440378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660204c0a0 0x556602017560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81de2a5640 1 -- 172.21.15.113:0/3890440378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202fe40 msgr2=0x556602017020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81de2a5640 1 --2- 172.21.15.113:0/3890440378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202fe40 0x556602017020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81de2a5640 1 -- 172.21.15.113:0/3890440378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556601c164b0 con 0x55660202c600 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81de2a5640 1 --2- 172.21.15.113:0/3890440378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202c600 0x556602032d00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f81d4006f80 tx=0x7f81d40ab000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7eef066501868ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81bb7fe640 1 -- 172.21.15.113:0/3890440378 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40795b0 con 0x55660202c600 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81bb7fe640 1 -- 172.21.15.113:0/3890440378 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d402ae50 con 0x55660202c600 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81bb7fe640 1 -- 172.21.15.113:0/3890440378 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4079320 con 0x55660202c600 2024-04-25T00:01:10.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81e3214740 1 -- 172.21.15.113:0/3890440378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202c600 msgr2=0x556602032d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.564+0000 7f81e3214740 1 --2- 172.21.15.113:0/3890440378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202c600 0x556602032d00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f81d4006f80 tx=0x7f81d40ab000 comp rx=0 tx=0).stop 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 -- 172.21.15.113:0/3890440378 shutdown_connections 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 --2- 172.21.15.113:0/3890440378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202fe40 0x556602017020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 --2- 172.21.15.113:0/3890440378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660204c0a0 0x556602017560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 --2- 172.21.15.113:0/3890440378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660202c600 0x556602032d00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 -- 172.21.15.113:0/3890440378 >> 172.21.15.113:0/3890440378 conn(0x556602005b20 msgr2=0x556602005f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 -- 172.21.15.113:0/3890440378 shutdown_connections 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 -- 172.21.15.113:0/3890440378 wait complete. 2024-04-25T00:01:10.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 -- start start 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.565+0000 7f81e3214740 1 Processor -- start 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202c600 0x5566020198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81e3214740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602019e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81e3214740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660204c0a0 0x5566020176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81e3214740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55660201aad0 con 0x55660204c0a0 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81d80718a0 con 0x55660202c600 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81e3214740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5566020003b0 con 0x55660202fe40 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81e3214740 10 client.0 _ll_register_callbacks cb 0x7f81d4006ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660204c0a0 0x5566020176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660204c0a0 0x5566020176e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33230/0 (socket says 172.21.15.113:33230) 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 -- 172.21.15.113:0/2120851235 learned_addr learned my addr 172.21.15.113:0/2120851235 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81ddaa4640 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602019e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 -- 172.21.15.113:0/2120851235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 msgr2=0x556602019e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602019e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 -- 172.21.15.113:0/2120851235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202c600 msgr2=0x5566020198c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202c600 0x5566020198c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.566+0000 7f81deaa6640 1 -- 172.21.15.113:0/2120851235 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556602004360 con 0x55660204c0a0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81deaa6640 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660204c0a0 0x5566020176e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009d50 tx=0x7f81d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec70021b29783177 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81b97fa640 1 -- 172.21.15.113:0/2120851235 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00094d0 con 0x55660204c0a0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81b97fa640 1 -- 172.21.15.113:0/2120851235 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f81d0014dd0 con 0x55660204c0a0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d402ae50 con 0x55660204c0a0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81b97fa640 1 -- 172.21.15.113:0/2120851235 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f81d002a480 con 0x55660204c0a0 2024-04-25T00:01:10.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 10 client.6015 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x556602051770 con 0x55660204c0a0 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81b97fa640 1 -- 172.21.15.113:0/2120851235 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d402ae50 con 0x55660204c0a0 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81b97fa640 1 -- 172.21.15.113:0/2120851235 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f81d002a620 con 0x55660204c0a0 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 10 client.6015 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 10 client.6015 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 20 client.6015 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204d6e0 0x556602017c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 4 client.6015 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) -- 0x556601f70d40 con 0x55660204d6e0 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.567+0000 7f81de2a5640 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204d6e0 0x556602017c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81b97fa640 10 client.6015 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81de2a5640 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204d6e0 0x556602017c20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbb308421f3aa34 server_cookie=348696288dbeda62 in_seq=0 out_seq=0 2024-04-25T00:01:10.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81b97fa640 1 -- 172.21.15.113:0/2120851235 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556601f70d40 con 0x55660204d6e0 2024-04-25T00:01:10.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81b97fa640 10 client.6015 handle_command_reply: tid=0 2024-04-25T00:01:10.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 client.6015 shutdown 2024-04-25T00:01:10.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660204c0a0 msgr2=0x5566020176e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660204c0a0 0x5566020176e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009d50 tx=0x7f81d0004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:10.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 shutdown_connections 2024-04-25T00:01:10.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55660204d6e0 0x556602017c20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55660204c0a0 0x5566020176e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55660202fe40 0x556602019e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 --2- 172.21.15.113:0/2120851235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55660202c600 0x5566020198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81b97fa640 0 client.6015 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 >> 172.21.15.113:0/2120851235 conn(0x556602005b20 msgr2=0x556602016200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 shutdown_connections 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.568+0000 7f81e3214740 1 -- 172.21.15.113:0/2120851235 wait complete. 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e3214740 20 client.6015 trim_cache size 0 max 16384 2024-04-25T00:01:10.579 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:10.580 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:10.580 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "7ab989a0-a0d0-4884-83a6-5bca9588ee53", 2024-04-25T00:01:10.580 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:10.580 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:10.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 msgr2=0x7f81bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f81d8156850 tx=0x7f81c8007840 comp rx=0 tx=0).stop 2024-04-25T00:01:10.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80d04f0 msgr2=0x7f81d80afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80d04f0 0x7f81d80afe10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f81d4035040 tx=0x7f81d40046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 shutdown_connections 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d813f050 0x7f81d815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d81606a0 0x7f81d8150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 --2- 172.21.15.113:0/3579218145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80d04f0 0x7f81d80afe10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 >> 172.21.15.113:0/3579218145 conn(0x7f81d806b0f0 msgr2=0x7f81d80b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 shutdown_connections 2024-04-25T00:01:10.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.569+0000 7f81e0509640 1 -- 172.21.15.113:0/3579218145 wait complete. 2024-04-25T00:01:10.583 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '7ab989a0-a0d0-4884-83a6-5bca9588ee53', 'mode': 'asynchronous'}' 2024-04-25T00:01:10.584 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T00:01:10.584 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T00:01:10.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.683+0000 7f92bedb0640 1 Processor -- start 2024-04-25T00:01:10.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.683+0000 7f92bedb0640 1 -- start start 2024-04-25T00:01:10.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.683+0000 7f92bedb0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 0x7f92b80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.683+0000 7f92bedb0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f92b80d28c0 0x7f92b80b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.683+0000 7f92bedb0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b80d2e00 con 0x7f92b80b61e0 2024-04-25T00:01:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.683+0000 7f92bedb0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f92b80d30e0 con 0x7f92b80d28c0 2024-04-25T00:01:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.683+0000 7f92bedb0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80d33c0 con 0x7f92b815e510 2024-04-25T00:01:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92bd34d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f92b80b61e0 0x7f92b8071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52334/0 (socket says 172.21.15.113:52334) 2024-04-25T00:01:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92bd34d640 1 -- 172.21.15.113:0/2892080092 learned_addr learned my addr 172.21.15.113:0/2892080092 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b7fff640 1 --2- 172.21.15.113:0/2892080092 >> v2:172.21.15.178:3300/0 conn(0x7f92b80d28c0 0x7f92b80b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 0x7f92b80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2394411540 0 0) 0x7f92b80d2e00 con 0x7f92b80b61e0 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92a0003520 con 0x7f92b80b61e0 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1640755373 0 0) 0x7f92a0003520 con 0x7f92b80b61e0 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 msgr2=0x7f92b80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 0x7f92b80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 >> v2:172.21.15.178:3300/0 conn(0x7f92b80d28c0 msgr2=0x7f92b80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 --2- 172.21.15.113:0/2892080092 >> v2:172.21.15.178:3300/0 conn(0x7f92b80d28c0 0x7f92b80b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b80d36a0 con 0x7f92b80b61e0 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f92b00019f0 con 0x7f92b80b61e0 2024-04-25T00:01:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 >> v1:172.21.15.113:6789/0 conn(0x7f92b80b61e0 legacy=0x7f92b8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a00039b0 0x7f92a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 0x7f92a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a0004d40 0x7f92a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80d30e0 con 0x7f92a0004d40 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80d33c0 con 0x7f92a00039b0 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92a000b9c0 con 0x7f92b815e510 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a0004d40 0x7f92a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a00039b0 0x7f92a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.684+0000 7f92b7fff640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 0x7f92a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92bd34d640 1 -- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 msgr2=0x7f92a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b815e510 0x7f92a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92bd34d640 1 -- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a00039b0 msgr2=0x7f92a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a00039b0 0x7f92a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92bd34d640 1 -- 172.21.15.113:0/2892080092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a0003520 con 0x7f92a0004d40 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a0004d40 0x7f92a00051c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f92b000a920 tx=0x7f92a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d88587daeb3a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b000a4c0 con 0x7f92a0004d40 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b002f070 con 0x7f92a0004d40 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.685+0000 7f92b77fe640 1 -- 172.21.15.113:0/2892080092 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b002bd20 con 0x7f92a0004d40 2024-04-25T00:01:10.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.686+0000 7f92bedb0640 1 -- 172.21.15.113:0/2892080092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a0004d40 msgr2=0x7f92a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.686+0000 7f92bedb0640 1 --2- 172.21.15.113:0/2892080092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a0004d40 0x7f92a00051c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f92b000a920 tx=0x7f92a0005830 comp rx=0 tx=0).stop 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.686+0000 7f92bd34d640 1 -- 172.21.15.113:0/2892080092 reap_dead start 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.686+0000 7f92bedb0640 1 -- 172.21.15.113:0/2892080092 shutdown_connections 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.686+0000 7f92bedb0640 1 -- 172.21.15.113:0/2892080092 >> 172.21.15.113:0/2892080092 conn(0x7f92b806b720 msgr2=0x7f92b80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.686+0000 7f92bedb0640 1 -- 172.21.15.113:0/2892080092 shutdown_connections 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.686+0000 7f92bedb0640 1 -- 172.21.15.113:0/2892080092 wait complete. 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.687+0000 7f92bedb0640 1 Processor -- start 2024-04-25T00:01:10.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.687+0000 7f92bedb0640 1 -- start start 2024-04-25T00:01:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bedb0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b80c6f90 0x7f92b80c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bedb0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92b8072330 0x7f92b81611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bedb0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b815c600 0x7f92b815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bedb0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80b9dd0 con 0x7f92b8072330 2024-04-25T00:01:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bedb0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80d3ea0 con 0x7f92b815c600 2024-04-25T00:01:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bedb0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80b83d0 con 0x7f92b80c6f90 2024-04-25T00:01:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92b8072330 0x7f92b81611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92b8072330 0x7f92b81611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33242/0 (socket says 172.21.15.113:33242) 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 -- 172.21.15.113:0/3141092925 learned_addr learned my addr 172.21.15.113:0/3141092925 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b80c6f90 0x7f92b80c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 -- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b80c6f90 msgr2=0x7f92b80c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b80c6f90 0x7f92b80c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 -- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b815c600 msgr2=0x7f92b815cab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b815c600 0x7f92b815cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 -- 172.21.15.113:0/3141092925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b80c7370 con 0x7f92b8072330 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b80c6f90 0x7f92b80c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.688+0000 7f92b7fff640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92b8072330 0x7f92b81611a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f92ac011a70 tx=0x7f92ac011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.689+0000 7f92b57fa640 1 -- 172.21.15.113:0/3141092925 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00fad0 con 0x7f92b8072330 2024-04-25T00:01:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.689+0000 7f92b57fa640 1 -- 172.21.15.113:0/3141092925 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ac00fc70 con 0x7f92b8072330 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.689+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b8148620 con 0x7f92b8072330 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.689+0000 7f92b57fa640 1 -- 172.21.15.113:0/3141092925 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac014440 con 0x7f92b8072330 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.689+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b8148be0 con 0x7f92b8072330 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.690+0000 7f92b57fa640 1 -- 172.21.15.113:0/3141092925 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92ac012040 con 0x7f92b8072330 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.691+0000 7f92b57fa640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92a406be30 0x7f92a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.691+0000 7f92b57fa640 1 -- 172.21.15.113:0/3141092925 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f92ac08b4c0 con 0x7f92b8072330 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.691+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92a406be30 0x7f92a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.691+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92a406be30 0x7f92a406e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009ab0 tx=0x7f92a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 Processor -- start 2024-04-25T00:01:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 -- start start 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd6d80 0x55984cfd7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d0017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d001ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55984cf3a730 con 0x55984cfd6d80 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55984cee2de0 con 0x55984cfd3560 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55984cf366f0 con 0x55984cffefc0 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.692+0000 7f92bcb4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd6d80 0x55984cfd7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd6d80 0x55984cfd7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33246/0 (socket says 172.21.15.113:33246) 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 -- 172.21.15.113:0/3700325522 learned_addr learned my addr 172.21.15.113:0/3700325522 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92b7fff640 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d0017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bd34d640 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d001ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 -- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 msgr2=0x55984d001ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d001ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 -- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 msgr2=0x55984d0017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d0017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 -- 172.21.15.113:0/3700325522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984cd1af00 con 0x55984cfd6d80 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92b7fff640 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d0017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd6d80 0x55984cfd7160 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f92a8013dc0 tx=0x7f92a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ffd2c43ad147bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f929e7fc640 1 -- 172.21.15.113:0/3700325522 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8029070 con 0x55984cfd6d80 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f929e7fc640 1 -- 172.21.15.113:0/3700325522 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a801f050 con 0x55984cfd6d80 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.693+0000 7f929e7fc640 1 -- 172.21.15.113:0/3700325522 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8003d90 con 0x55984cfd6d80 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 -- 172.21.15.113:0/3700325522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd6d80 msgr2=0x55984cfd7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd6d80 0x55984cfd7160 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f92a8013dc0 tx=0x7f92a8016000 comp rx=0 tx=0).stop 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 -- 172.21.15.113:0/3700325522 shutdown_connections 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d0017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d001ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/3700325522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd6d80 0x55984cfd7160 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 -- 172.21.15.113:0/3700325522 >> 172.21.15.113:0/3700325522 conn(0x55984cf9e3c0 msgr2=0x55984d01c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 -- 172.21.15.113:0/3700325522 shutdown_connections 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 -- 172.21.15.113:0/3700325522 wait complete. 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 -- start start 2024-04-25T00:01:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.694+0000 7f92c1b9d740 1 Processor -- start 2024-04-25T00:01:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd3560 0x55984d01bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984d0202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d030970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55984cfd79a0 con 0x55984cfd3560 2024-04-25T00:01:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55984cef6360 con 0x55984cfd6d80 2024-04-25T00:01:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55984cfd7cd0 con 0x55984cffefc0 2024-04-25T00:01:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92c1b9d740 10 client.0 _ll_register_callbacks cb 0x55984cf355b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92bcb4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd3560 0x55984d01bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92bcb4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd3560 0x55984d01bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33248/0 (socket says 172.21.15.113:33248) 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92bcb4c640 1 -- 172.21.15.113:0/2073730415 learned_addr learned my addr 172.21.15.113:0/2073730415 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92b7fff640 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984d0202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.695+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d030970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92bcb4c640 1 -- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 msgr2=0x55984d030970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d030970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92bcb4c640 1 -- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 msgr2=0x55984d0202c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984d0202c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92bcb4c640 1 -- 172.21.15.113:0/2073730415 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984cc0e7f0 con 0x55984cfd3560 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd3560 0x55984d01bd30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009130 tx=0x7f92a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbdc6ab0985f4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92977fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92977fe640 1 -- 172.21.15.113:0/2073730415 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8033020 con 0x55984cfd3560 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92977fe640 1 -- 172.21.15.113:0/2073730415 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f92a802f470 con 0x55984cfd3560 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a801f050 con 0x55984cfd3560 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92977fe640 1 -- 172.21.15.113:0/2073730415 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f92a803d050 con 0x55984cfd3560 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 10 client.6027 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55984cd7b690 con 0x55984cfd3560 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92977fe640 1 -- 172.21.15.113:0/2073730415 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a801f050 con 0x55984cfd3560 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92977fe640 1 -- 172.21.15.113:0/2073730415 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f92a8040020 con 0x55984cfd3560 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 10 client.6027 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 10 client.6027 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 20 client.6027 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d000080 0x55984d000460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 4 client.6027 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55984cf47c70 con 0x55984d000080 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.696+0000 7f92b7fff640 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d000080 0x55984d000460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.697+0000 7f92977fe640 10 client.6027 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.697+0000 7f92b7fff640 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d000080 0x55984d000460 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3d581841d071aa server_cookie=4253fb2325cacbc0 in_seq=0 out_seq=0 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.697+0000 7f92977fe640 1 -- 172.21.15.113:0/2073730415 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55984cf47c70 con 0x55984d000080 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.697+0000 7f92977fe640 10 client.6027 handle_command_reply: tid=0 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.697+0000 7f92c1b9d740 1 client.6027 shutdown 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.697+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd3560 msgr2=0x55984d01bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.697+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd3560 0x55984d01bd30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009130 tx=0x7f92a8004730 comp rx=0 tx=0).stop 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 shutdown_connections 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d000080 0x55984d000460 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984d0202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cffefc0 0x55984d030970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92977fe640 0 client.6027 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2073730415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cfd3560 0x55984d01bd30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 >> 172.21.15.113:0/2073730415 conn(0x55984cf9e3c0 msgr2=0x55984cfd5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 shutdown_connections 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2073730415 wait complete. 2024-04-25T00:01:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.698+0000 7f92c1b9d740 20 client.6027 trim_cache size 0 max 16384 2024-04-25T00:01:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.715+0000 7f92c1b9d740 1 Processor -- start 2024-04-25T00:01:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.715+0000 7f92c1b9d740 1 -- start start 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d01fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd6d80 0x55984d020210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d01f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92b82016c0 con 0x55984cffefc0 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92b82007f0 con 0x55984cfd3560 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80cddc0 con 0x55984cfd6d80 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d01f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d01f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33250/0 (socket says 172.21.15.113:33250) 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 -- 172.21.15.113:0/80820837 learned_addr learned my addr 172.21.15.113:0/80820837 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d01fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 -- 172.21.15.113:0/80820837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd6d80 msgr2=0x55984d020210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd6d80 0x55984d020210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 -- 172.21.15.113:0/80820837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 msgr2=0x55984d01fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d01fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 -- 172.21.15.113:0/80820837 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984cbe78b0 con 0x55984cffefc0 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f92bd34d640 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d01f010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f92b000aa40 tx=0x7f92b002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2f92b25dd41a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f929dffb640 1 -- 172.21.15.113:0/80820837 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0044070 con 0x55984cffefc0 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f929dffb640 1 -- 172.21.15.113:0/80820837 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b003a050 con 0x55984cffefc0 2024-04-25T00:01:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.716+0000 7f929dffb640 1 -- 172.21.15.113:0/80820837 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0002000 con 0x55984cffefc0 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 -- 172.21.15.113:0/80820837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 msgr2=0x55984d01f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d01f010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f92b000aa40 tx=0x7f92b002d000 comp rx=0 tx=0).stop 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 -- 172.21.15.113:0/80820837 shutdown_connections 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d01f010 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd6d80 0x55984d020210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/80820837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd3560 0x55984d01fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 -- 172.21.15.113:0/80820837 >> 172.21.15.113:0/80820837 conn(0x55984cf9e3c0 msgr2=0x55984cfd53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 -- 172.21.15.113:0/80820837 shutdown_connections 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 -- 172.21.15.113:0/80820837 wait complete. 2024-04-25T00:01:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.717+0000 7f92c1b9d740 1 -- start start 2024-04-25T00:01:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 1 Processor -- start 2024-04-25T00:01:10.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd3560 0x55984cffda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984cffdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d000460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92b82001a0 con 0x55984cffefc0 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55984cf185e0 con 0x55984cfd6d80 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55984d097520 con 0x55984cfd3560 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.718+0000 7f92c1b9d740 10 client.0 _ll_register_callbacks cb 0x7f92a8027070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d000460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d000460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33260/0 (socket says 172.21.15.113:33260) 2024-04-25T00:01:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 -- 172.21.15.113:0/2250187775 learned_addr learned my addr 172.21.15.113:0/2250187775 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd3560 0x55984cffda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92b7fff640 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984cffdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 -- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd3560 msgr2=0x55984cffda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd3560 0x55984cffda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 -- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 msgr2=0x55984cffdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984cffdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 -- 172.21.15.113:0/2250187775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984cfd16f0 con 0x55984cffefc0 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92bd34d640 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d000460 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f92b0001cf0 tx=0x7f92b0001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0f84172b180d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92977fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92977fe640 1 -- 172.21.15.113:0/2250187775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b004e060 con 0x55984cffefc0 2024-04-25T00:01:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92977fe640 1 -- 172.21.15.113:0/2250187775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f92b002baa0 con 0x55984cffefc0 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92b003a050 con 0x55984cffefc0 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92977fe640 1 -- 172.21.15.113:0/2250187775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f92b0044070 con 0x55984cffefc0 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92c1b9d740 10 client.6033 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55984cffd230 con 0x55984cffefc0 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.719+0000 7f92977fe640 1 -- 172.21.15.113:0/2250187775 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b003a050 con 0x55984cffefc0 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92977fe640 1 -- 172.21.15.113:0/2250187775 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f92b002bca0 con 0x55984cffefc0 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92c1b9d740 10 client.6033 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92c1b9d740 10 client.6033 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92c1b9d740 20 client.6033 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d020370 0x55984d0009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92c1b9d740 4 client.6033 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) -- 0x55984cfd3060 con 0x55984d020370 2024-04-25T00:01:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d020370 0x55984d0009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92977fe640 10 client.6033 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.720+0000 7f92bcb4c640 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d020370 0x55984d0009a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b720910899ad08b server_cookie=3f7de859c84bfc71 in_seq=0 out_seq=0 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92977fe640 1 -- 172.21.15.113:0/2250187775 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55984cfd3060 con 0x55984d020370 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92977fe640 10 client.6033 handle_command_reply: tid=0 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 client.6033 shutdown 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 msgr2=0x55984d000460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d000460 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f92b0001cf0 tx=0x7f92b0001dc0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 shutdown_connections 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55984d020370 0x55984d0009a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55984cfd6d80 0x55984cffdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984cffefc0 0x55984d000460 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 --2- 172.21.15.113:0/2250187775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55984cfd3560 0x55984cffda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 >> 172.21.15.113:0/2250187775 conn(0x55984cf9e3c0 msgr2=0x55984d01fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92977fe640 0 client.6033 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 shutdown_connections 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 1 -- 172.21.15.113:0/2250187775 wait complete. 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92c1b9d740 20 client.6033 trim_cache size 0 max 16384 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:10.729 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "93e6ad6f-6c5d-469c-b8f8-944e944fbbb5", 2024-04-25T00:01:10.730 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:10.730 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:10.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92a406be30 msgr2=0x7f92a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92bedb0640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92a406be30 0x7f92a406e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009ab0 tx=0x7f92a8008040 comp rx=0 tx=0).stop 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92b8072330 msgr2=0x7f92b81611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.721+0000 7f92bedb0640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92b8072330 0x7f92b81611a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f92ac011a70 tx=0x7f92ac011f30 comp rx=0 tx=0).stop 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 shutdown_connections 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92a406be30 0x7f92a406e2f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b815c600 0x7f92b815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92b8072330 0x7f92b81611a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 --2- 172.21.15.113:0/3141092925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b80c6f90 0x7f92b80c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 >> 172.21.15.113:0/3141092925 conn(0x7f92b806b720 msgr2=0x7f92b80b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 shutdown_connections 2024-04-25T00:01:10.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.722+0000 7f92bedb0640 1 -- 172.21.15.113:0/3141092925 wait complete. 2024-04-25T00:01:10.734 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '93e6ad6f-6c5d-469c-b8f8-944e944fbbb5', 'mode': 'asynchronous'}' 2024-04-25T00:01:10.734 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T00:01:10.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T00:01:10.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.834+0000 7fa98aa5d640 1 Processor -- start 2024-04-25T00:01:10.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.834+0000 7fa98aa5d640 1 -- start start 2024-04-25T00:01:10.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.834+0000 7fa98aa5d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa98415e2d0 0x7fa98415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.834+0000 7fa98aa5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b4a40 0x7fa9840ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.834+0000 7fa98aa5d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98415f4f0 con 0x7fa9840b3080 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.834+0000 7fa98aa5d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa9840cde00 con 0x7fa98415e2d0 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.834+0000 7fa98aa5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa98415f7d0 con 0x7fa9840b4a40 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa983fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa98415e2d0 0x7fa98415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa9837fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa9840b3080 0x7fa9840b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52338/0 (socket says 172.21.15.113:52338) 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa9837fe640 1 -- 172.21.15.113:0/1883014568 learned_addr learned my addr 172.21.15.113:0/1883014568 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa983fff640 1 -- 172.21.15.113:0/1883014568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b4a40 msgr2=0x7fa9840ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa983fff640 1 --2- 172.21.15.113:0/1883014568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b4a40 0x7fa9840ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa983fff640 1 -- 172.21.15.113:0/1883014568 >> v1:172.21.15.113:6789/0 conn(0x7fa9840b3080 legacy=0x7fa9840b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa983fff640 1 -- 172.21.15.113:0/1883014568 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98415fab0 con 0x7fa98415e2d0 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.835+0000 7fa982ffd640 1 -- 172.21.15.113:0/1883014568 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259402937 0 0) 0x7fa98415f4f0 con 0x7fa9840b3080 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa983fff640 1 --2- 172.21.15.113:0/1883014568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa98415e2d0 0x7fa98415ebf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa97402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7561b680f10d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa982ffd640 1 -- 172.21.15.113:0/1883014568 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974037070 con 0x7fa98415e2d0 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa982ffd640 1 -- 172.21.15.113:0/1883014568 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974032440 con 0x7fa98415e2d0 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa982ffd640 1 -- 172.21.15.113:0/1883014568 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974032810 con 0x7fa98415e2d0 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 -- 172.21.15.113:0/1883014568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa98415e2d0 msgr2=0x7fa98415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/1883014568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa98415e2d0 0x7fa98415ebf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa97402a180 comp rx=0 tx=0).stop 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 -- 172.21.15.113:0/1883014568 shutdown_connections 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/1883014568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b4a40 0x7fa9840ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/1883014568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa98415e2d0 0x7fa98415ebf0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 -- 172.21.15.113:0/1883014568 >> 172.21.15.113:0/1883014568 conn(0x7fa98406b0f0 msgr2=0x7fa98406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 -- 172.21.15.113:0/1883014568 shutdown_connections 2024-04-25T00:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.836+0000 7fa98aa5d640 1 -- 172.21.15.113:0/1883014568 wait complete. 2024-04-25T00:01:10.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.837+0000 7fa98aa5d640 1 Processor -- start 2024-04-25T00:01:10.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa98aa5d640 1 -- start start 2024-04-25T00:01:10.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa98aa5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b3080 0x7fa9841f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa98aa5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9840b4a40 0x7fa9841f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa98aa5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa98415e2d0 0x7fa984153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa98aa5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa98415f2b0 con 0x7fa9840b4a40 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa98aa5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa98415f130 con 0x7fa98415e2d0 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa98aa5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa98415e6b0 con 0x7fa9840b3080 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9840b4a40 0x7fa9841f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9840b4a40 0x7fa9841f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33274/0 (socket says 172.21.15.113:33274) 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.838+0000 7fa9837fe640 1 -- 172.21.15.113:0/133882839 learned_addr learned my addr 172.21.15.113:0/133882839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa983fff640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b3080 0x7fa9841f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa988ffa640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa98415e2d0 0x7fa984153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa9837fe640 1 -- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b3080 msgr2=0x7fa9841f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa9837fe640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b3080 0x7fa9841f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa9837fe640 1 -- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa98415e2d0 msgr2=0x7fa984153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa9837fe640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa98415e2d0 0x7fa984153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa9837fe640 1 -- 172.21.15.113:0/133882839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa984154610 con 0x7fa9840b4a40 2024-04-25T00:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa9837fe640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9840b4a40 0x7fa9841f4d80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa97800c4d0 tx=0x7fa97800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa980ff9640 1 -- 172.21.15.113:0/133882839 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978018070 con 0x7fa9840b4a40 2024-04-25T00:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa980ff9640 1 -- 172.21.15.113:0/133882839 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978007500 con 0x7fa9840b4a40 2024-04-25T00:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa980ff9640 1 -- 172.21.15.113:0/133882839 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978004a20 con 0x7fa9840b4a40 2024-04-25T00:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa984151670 con 0x7fa9840b4a40 2024-04-25T00:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.839+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa984151c30 con 0x7fa9840b4a40 2024-04-25T00:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.841+0000 7fa980ff9640 1 -- 172.21.15.113:0/133882839 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa97801b070 con 0x7fa9840b4a40 2024-04-25T00:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.841+0000 7fa980ff9640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa97006be30 0x7fa97006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.841+0000 7fa980ff9640 1 -- 172.21.15.113:0/133882839 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa97808a170 con 0x7fa9840b4a40 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.841+0000 7fa983fff640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa97006be30 0x7fa97006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.841+0000 7fa983fff640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa97006be30 0x7fa97006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa974034000 tx=0x7fa97402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.842+0000 7fa98d76c740 1 Processor -- start 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.842+0000 7fa98d76c740 1 -- start start 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa18df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18d9ec0 0x55cfa18ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa18de170 0x55cfa18de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa98d76c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cfa1842220 con 0x55cfa18d9ec0 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cfa17ea8b0 con 0x55cfa18de170 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cfa183e1e0 con 0x55cfa18df030 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18d9ec0 0x55cfa18ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18d9ec0 0x55cfa18ddc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33280/0 (socket says 172.21.15.113:33280) 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 -- 172.21.15.113:0/1882647158 learned_addr learned my addr 172.21.15.113:0/1882647158 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa988ffa640 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa18de170 0x55cfa18de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa983fff640 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa18df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 -- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 msgr2=0x55cfa18df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa18df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 -- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa18de170 msgr2=0x55cfa18de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa18de170 0x55cfa18de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 -- 172.21.15.113:0/1882647158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa1622690 con 0x55cfa18d9ec0 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa983fff640 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa18df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa9837fe640 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18d9ec0 0x55cfa18ddc30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa9780548f0 tx=0x7fa978058490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aff85f80f6c05a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.843+0000 7fa969ffb640 1 -- 172.21.15.113:0/1882647158 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978060fc0 con 0x55cfa18d9ec0 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa969ffb640 1 -- 172.21.15.113:0/1882647158 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978056790 con 0x55cfa18d9ec0 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa969ffb640 1 -- 172.21.15.113:0/1882647158 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978004bc0 con 0x55cfa18d9ec0 2024-04-25T00:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 -- 172.21.15.113:0/1882647158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18d9ec0 msgr2=0x55cfa18ddc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18d9ec0 0x55cfa18ddc30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa9780548f0 tx=0x7fa978058490 comp rx=0 tx=0).stop 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 -- 172.21.15.113:0/1882647158 shutdown_connections 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa18de170 0x55cfa18de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18d9ec0 0x55cfa18ddc30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 --2- 172.21.15.113:0/1882647158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa18df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 -- 172.21.15.113:0/1882647158 >> 172.21.15.113:0/1882647158 conn(0x55cfa18dc130 msgr2=0x55cfa1924bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 -- 172.21.15.113:0/1882647158 shutdown_connections 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.844+0000 7fa98d76c740 1 -- 172.21.15.113:0/1882647158 wait complete. 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.845+0000 7fa98d76c740 1 -- start start 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.845+0000 7fa98d76c740 1 Processor -- start 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18de170 0x55cfa18feee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18df030 0x55cfa18ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa192a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cfa190abc0 con 0x55cfa18df030 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cfa17fde40 con 0x55cfa19000f0 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cfa190a040 con 0x55cfa18de170 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 10 client.0 _ll_register_callbacks cb 0x55cfa1802080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18df030 0x55cfa18ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18df030 0x55cfa18ff420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33292/0 (socket says 172.21.15.113:33292) 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 -- 172.21.15.113:0/2337525404 learned_addr learned my addr 172.21.15.113:0/2337525404 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa983fff640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18de170 0x55cfa18feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa988ffa640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa192a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 -- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18de170 msgr2=0x55cfa18feee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18de170 0x55cfa18feee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 -- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 msgr2=0x55cfa192a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa192a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 -- 172.21.15.113:0/2337525404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa1682e20 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa988ffa640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa192a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa9837fe640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18df030 0x55cfa18ff420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa978058190 tx=0x7fa978000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a53a4fe29a2508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa962ffd640 1 -- 172.21.15.113:0/2337525404 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978070370 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa962ffd640 1 -- 172.21.15.113:0/2337525404 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa9780578e0 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa978056790 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa962ffd640 1 -- 172.21.15.113:0/2337525404 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa978057ae0 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 10 client.6048 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55cfa1515f80 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.846+0000 7fa962ffd640 1 -- 172.21.15.113:0/2337525404 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978056790 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa962ffd640 1 -- 172.21.15.113:0/2337525404 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fa978088120 con 0x55cfa18df030 2024-04-25T00:01:10.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa98d76c740 10 client.6048 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa98d76c740 10 client.6048 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa98d76c740 20 client.6048 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa1903750 0x55cfa1903b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa98d76c740 4 client.6048 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfa17ffcc0 con 0x55cfa1903750 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa983fff640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa1903750 0x55cfa1903b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa962ffd640 10 client.6048 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.847+0000 7fa983fff640 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa1903750 0x55cfa1903b60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d52bde6f118cb90 server_cookie=12dda325de8deffd in_seq=0 out_seq=0 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa962ffd640 1 -- 172.21.15.113:0/2337525404 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfa17ffcc0 con 0x55cfa1903750 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa962ffd640 10 client.6048 handle_command_reply: tid=0 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 client.6048 shutdown 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18df030 msgr2=0x55cfa18ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18df030 0x55cfa18ff420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa978058190 tx=0x7fa978000ed0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 shutdown_connections 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa1903750 0x55cfa1903b60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa192a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18df030 0x55cfa18ff420 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa962ffd640 0 client.6048 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2337525404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18de170 0x55cfa18feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 >> 172.21.15.113:0/2337525404 conn(0x55cfa18d9ec0 msgr2=0x55cfa18d95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 shutdown_connections 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 1 -- 172.21.15.113:0/2337525404 wait complete. 2024-04-25T00:01:10.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.848+0000 7fa98d76c740 20 client.6048 trim_cache size 0 max 16384 2024-04-25T00:01:10.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.865+0000 7fa98d76c740 1 Processor -- start 2024-04-25T00:01:10.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.865+0000 7fa98d76c740 1 -- start start 2024-04-25T00:01:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1948de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1949320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa1904ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa98d76c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9841fe6d0 con 0x55cfa18de170 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9841f6400 con 0x55cfa19000f0 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9841f5760 con 0x55cfa18df030 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1948de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1948de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33304/0 (socket says 172.21.15.113:33304) 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 -- 172.21.15.113:0/882837934 learned_addr learned my addr 172.21.15.113:0/882837934 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 -- 172.21.15.113:0/882837934 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 msgr2=0x55cfa1949320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 --2- 172.21.15.113:0/882837934 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1949320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 -- 172.21.15.113:0/882837934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 msgr2=0x55cfa1904ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 --2- 172.21.15.113:0/882837934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa1904ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.866+0000 7fa983fff640 1 -- 172.21.15.113:0/882837934 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa1928d20 con 0x55cfa18de170 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa983fff640 1 --2- 172.21.15.113:0/882837934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1948de0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa97403f040 tx=0x7fa97402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10124558fec4e8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa9697fa640 1 -- 172.21.15.113:0/882837934 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974037040 con 0x55cfa18de170 2024-04-25T00:01:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa9697fa640 1 -- 172.21.15.113:0/882837934 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974032d80 con 0x55cfa18de170 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa9697fa640 1 -- 172.21.15.113:0/882837934 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9740535c0 con 0x55cfa18de170 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 -- 172.21.15.113:0/882837934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 msgr2=0x55cfa1948de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 --2- 172.21.15.113:0/882837934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1948de0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa97403f040 tx=0x7fa97402c000 comp rx=0 tx=0).stop 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 -- 172.21.15.113:0/882837934 shutdown_connections 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 --2- 172.21.15.113:0/882837934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa1904ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 --2- 172.21.15.113:0/882837934 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1949320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 --2- 172.21.15.113:0/882837934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1948de0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 -- 172.21.15.113:0/882837934 >> 172.21.15.113:0/882837934 conn(0x55cfa18d9ec0 msgr2=0x55cfa18fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 -- 172.21.15.113:0/882837934 shutdown_connections 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.867+0000 7fa98d76c740 1 -- 172.21.15.113:0/882837934 wait complete. 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.868+0000 7fa98d76c740 1 -- start start 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.868+0000 7fa98d76c740 1 Processor -- start 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1923ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1906590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa1900500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9841f5db0 con 0x55cfa18de170 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cfa19288f0 con 0x55cfa19000f0 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cfa18e0a60 con 0x55cfa18df030 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 10 client.0 _ll_register_callbacks cb 0x7fa97806f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1923ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1923ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33306/0 (socket says 172.21.15.113:33306) 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 -- 172.21.15.113:0/2265287817 learned_addr learned my addr 172.21.15.113:0/2265287817 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa988ffa640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa1900500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa9837fe640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1906590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 -- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 msgr2=0x55cfa1906590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1906590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 -- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 msgr2=0x55cfa1900500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa1900500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 -- 172.21.15.113:0/2265287817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa1924720 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa9837fe640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1906590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa983fff640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1923ef0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa974006f80 tx=0x7fa974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af24068a0fd6b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa962ffd640 1 -- 172.21.15.113:0/2265287817 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974063060 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa962ffd640 1 -- 172.21.15.113:0/2265287817 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa9740534c0 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa974032d80 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.869+0000 7fa962ffd640 1 -- 172.21.15.113:0/2265287817 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa974069050 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 10 client.6054 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55cfa18f9cb0 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa962ffd640 1 -- 172.21.15.113:0/2265287817 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974032d80 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa962ffd640 1 -- 172.21.15.113:0/2265287817 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fa97406c020 con 0x55cfa18de170 2024-04-25T00:01:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 10 client.6054 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 10 client.6054 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 20 client.6054 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa18dfca0 0x55cfa1900a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 4 client.6054 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) -- 0x55cfa184f760 con 0x55cfa18dfca0 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa9837fe640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa18dfca0 0x55cfa1900a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa962ffd640 10 client.6054 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.870+0000 7fa9837fe640 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa18dfca0 0x55cfa1900a40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6989e7ce13a8f9f server_cookie=643f65816b48584 in_seq=0 out_seq=0 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa962ffd640 1 -- 172.21.15.113:0/2265287817 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cfa184f760 con 0x55cfa18dfca0 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa962ffd640 10 client.6054 handle_command_reply: tid=0 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 client.6054 shutdown 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 msgr2=0x55cfa1923ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1923ef0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa974006f80 tx=0x7fa974004730 comp rx=0 tx=0).stop 2024-04-25T00:01:10.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 shutdown_connections 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55cfa18dfca0 0x55cfa1900a40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfa19000f0 0x55cfa1900500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfa18df030 0x55cfa1906590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa962ffd640 0 client.6054 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 --2- 172.21.15.113:0/2265287817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfa18de170 0x55cfa1923ef0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 >> 172.21.15.113:0/2265287817 conn(0x55cfa18d9ec0 msgr2=0x55cfa1929f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 shutdown_connections 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 1 -- 172.21.15.113:0/2265287817 wait complete. 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.871+0000 7fa98d76c740 20 client.6054 trim_cache size 0 max 16384 2024-04-25T00:01:10.880 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:10.881 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:10.881 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "7f93056b-d382-48bd-b672-da086de52ce2", 2024-04-25T00:01:10.881 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:10.881 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa97006be30 msgr2=0x7fa97006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa97006be30 0x7fa97006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa974034000 tx=0x7fa97402e000 comp rx=0 tx=0).stop 2024-04-25T00:01:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9840b4a40 msgr2=0x7fa9841f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9840b4a40 0x7fa9841f4d80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa97800c4d0 tx=0x7fa97800c990 comp rx=0 tx=0).stop 2024-04-25T00:01:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 shutdown_connections 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa97006be30 0x7fa97006e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa98415e2d0 0x7fa984153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9840b4a40 0x7fa9841f4d80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 --2- 172.21.15.113:0/133882839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9840b3080 0x7fa9841f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 >> 172.21.15.113:0/133882839 conn(0x7fa98406b0f0 msgr2=0x7fa9840afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 shutdown_connections 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:10.872+0000 7fa98aa5d640 1 -- 172.21.15.113:0/133882839 wait complete. 2024-04-25T00:01:10.883 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '7f93056b-d382-48bd-b672-da086de52ce2', 'mode': 'asynchronous'}' 2024-04-25T00:01:10.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:10.883 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T00:01:10.899 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T00:01:10.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.037+0000 7fd88e189640 1 Processor -- start 2024-04-25T00:01:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.037+0000 7fd88e189640 1 -- start start 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd88e189640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888144d60 0x7fd8880b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd88e189640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd88806e6e0 0x7fd888071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd88e189640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8880cfad0 con 0x7fd8881453c0 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd88e189640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd888145c40 con 0x7fd88806e6e0 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd88e189640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880cf230 con 0x7fd888144d60 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd886ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd8881453c0 0x7fd88806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52352/0 (socket says 172.21.15.113:52352) 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd886ffd640 1 -- 172.21.15.113:0/3308863819 learned_addr learned my addr 172.21.15.113:0/3308863819 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2816454330 0 0) 0x7fd8880cfad0 con 0x7fd8881453c0 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd870003520 con 0x7fd8881453c0 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3476919173 0 0) 0x7fd870003520 con 0x7fd8881453c0 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888144d60 msgr2=0x7fd8880b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888144d60 0x7fd8880b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 >> v2:172.21.15.178:3300/0 conn(0x7fd88806e6e0 msgr2=0x7fd888071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 --2- 172.21.15.113:0/3308863819 >> v2:172.21.15.178:3300/0 conn(0x7fd88806e6e0 0x7fd888071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.038+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8880cf510 con 0x7fd8881453c0 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fd87c001670 con 0x7fd8881453c0 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 >> v1:172.21.15.113:6789/0 conn(0x7fd8881453c0 legacy=0x7fd88806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888144d60 0x7fd8700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8700040c0 0x7fd870004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd870004d90 0x7fd870005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd888145c40 con 0x7fd870004d90 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880cf230 con 0x7fd8700040c0 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd87000bd20 con 0x7fd888144d60 2024-04-25T00:01:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd887fff640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd870004d90 0x7fd870005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd886ffd640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8700040c0 0x7fd870004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd8877fe640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888144d60 0x7fd8700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd887fff640 1 -- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888144d60 msgr2=0x7fd8700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd887fff640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888144d60 0x7fd8700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd887fff640 1 -- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8700040c0 msgr2=0x7fd870004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd887fff640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8700040c0 0x7fd870004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.039+0000 7fd887fff640 1 -- 172.21.15.113:0/3308863819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd870003520 con 0x7fd870004d90 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd887fff640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd870004d90 0x7fd870005210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd880009e10 tx=0x7fd870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20aa10a877e4410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880033070 con 0x7fd870004d90 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd880006e50 con 0x7fd870004d90 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd8867fc640 1 -- 172.21.15.113:0/3308863819 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880009a10 con 0x7fd870004d90 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd88e189640 1 -- 172.21.15.113:0/3308863819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd870004d90 msgr2=0x7fd870005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd88e189640 1 --2- 172.21.15.113:0/3308863819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd870004d90 0x7fd870005210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd880009e10 tx=0x7fd870005830 comp rx=0 tx=0).stop 2024-04-25T00:01:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd887fff640 1 -- 172.21.15.113:0/3308863819 reap_dead start 2024-04-25T00:01:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd88e189640 1 -- 172.21.15.113:0/3308863819 shutdown_connections 2024-04-25T00:01:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd88e189640 1 -- 172.21.15.113:0/3308863819 >> 172.21.15.113:0/3308863819 conn(0x7fd8880b1a50 msgr2=0x7fd8880b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd88e189640 1 -- 172.21.15.113:0/3308863819 shutdown_connections 2024-04-25T00:01:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.040+0000 7fd88e189640 1 -- 172.21.15.113:0/3308863819 wait complete. 2024-04-25T00:01:11.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.041+0000 7fd88e189640 1 Processor -- start 2024-04-25T00:01:11.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd88e189640 1 -- start start 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd88e189640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888139df0 0x7fd88813c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd88e189640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88813a1d0 0x7fd8881e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd88e189640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8881e8160 0x7fd888137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd88e189640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd888145140 con 0x7fd8881e8160 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd88e189640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd888070e10 con 0x7fd88813a1d0 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd88e189640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880b5390 con 0x7fd888139df0 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd887fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8881e8160 0x7fd888137680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd887fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8881e8160 0x7fd888137680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33326/0 (socket says 172.21.15.113:33326) 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.042+0000 7fd887fff640 1 -- 172.21.15.113:0/3743761001 learned_addr learned my addr 172.21.15.113:0/3743761001 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd886ffd640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88813a1d0 0x7fd8881e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd8877fe640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888139df0 0x7fd88813c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd887fff640 1 -- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888139df0 msgr2=0x7fd88813c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd887fff640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888139df0 0x7fd88813c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd887fff640 1 -- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88813a1d0 msgr2=0x7fd8881e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd887fff640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88813a1d0 0x7fd8881e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd887fff640 1 -- 172.21.15.113:0/3743761001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88813a730 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd887fff640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8881e8160 0x7fd888137680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd880009f00 tx=0x7fd880004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd86ffff640 1 -- 172.21.15.113:0/3743761001 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880033070 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd86ffff640 1 -- 172.21.15.113:0/3743761001 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd880004570 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88813aa10 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd86ffff640 1 -- 172.21.15.113:0/3743761001 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800092f0 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.043+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88813afd0 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.044+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8880691a0 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.044+0000 7fd86ffff640 1 -- 172.21.15.113:0/3743761001 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd880036070 con 0x7fd8881e8160 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.045+0000 7fd86ffff640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd87406be30 0x7fd87406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.045+0000 7fd8877fe640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd87406be30 0x7fd87406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.045+0000 7fd86ffff640 1 -- 172.21.15.113:0/3743761001 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fd8800aba50 con 0x7fd8881e8160 2024-04-25T00:01:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.045+0000 7fd8877fe640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd87406be30 0x7fd87406e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd878004700 tx=0x7fd878008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.049+0000 7fd86ffff640 1 -- 172.21.15.113:0/3743761001 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd880048060 con 0x7fd8881e8160 2024-04-25T00:01:11.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.259+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8880b27e0 con 0x7fd8881e8160 2024-04-25T00:01:11.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.260+0000 7fd86ffff640 1 -- 172.21.15.113:0/3743761001 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5190 (secure 0 0 0) 0x7fd880008030 con 0x7fd8881e8160 2024-04-25T00:01:11.260 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:11.260 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:10.191614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:11.261 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 23 2024-04-25T00:01:11.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd87406be30 msgr2=0x7fd87406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd87406be30 0x7fd87406e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd878004700 tx=0x7fd878008040 comp rx=0 tx=0).stop 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8881e8160 msgr2=0x7fd888137680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8881e8160 0x7fd888137680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd880009f00 tx=0x7fd880004b50 comp rx=0 tx=0).stop 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 shutdown_connections 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd87406be30 0x7fd87406e2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88813a1d0 0x7fd8881e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8881e8160 0x7fd888137680 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 --2- 172.21.15.113:0/3743761001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd888139df0 0x7fd88813c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 >> 172.21.15.113:0/3743761001 conn(0x7fd8880b1a50 msgr2=0x7fd888140ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 shutdown_connections 2024-04-25T00:01:11.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.262+0000 7fd88e189640 1 -- 172.21.15.113:0/3743761001 wait complete. 2024-04-25T00:01:11.273 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T00:01:11.382 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:11.382 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:11.382 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:11.392 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:11.392 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:11.392 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:11.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:11.408 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T00:01:11.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:01:11.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:01:11.469 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644157 s, 163 MB/s 2024-04-25T00:01:11.470 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T00:01:11.470 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:11.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.607+0000 7f50ddfce640 1 Processor -- start 2024-04-25T00:01:11.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.607+0000 7f50ddfce640 1 -- start start 2024-04-25T00:01:11.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50ddfce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f50d80d2290 0x7f50d80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50ddfce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f50d80d0b00 0x7f50d80d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50ddfce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d80d1480 con 0x7f50d815e320 2024-04-25T00:01:11.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50ddfce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f50d80d3410 con 0x7f50d80d2290 2024-04-25T00:01:11.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50ddfce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f50d80d2800 con 0x7f50d80d0b00 2024-04-25T00:01:11.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50d6ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f50d80d2290 0x7f50d80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50d77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f50d815e320 0x7f50d80d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52366/0 (socket says 172.21.15.113:52366) 2024-04-25T00:01:11.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50d77fe640 1 -- 172.21.15.113:0/837972766 learned_addr learned my addr 172.21.15.113:0/837972766 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:11.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.608+0000 7f50d7fff640 1 --2- 172.21.15.113:0/837972766 >> v2:172.21.15.178:3300/0 conn(0x7f50d80d0b00 0x7f50d80d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d6ffd640 1 -- 172.21.15.113:0/837972766 >> v2:172.21.15.178:3300/0 conn(0x7f50d80d0b00 msgr2=0x7f50d80d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:11.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d6ffd640 1 --2- 172.21.15.113:0/837972766 >> v2:172.21.15.178:3300/0 conn(0x7f50d80d0b00 0x7f50d80d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d6ffd640 1 -- 172.21.15.113:0/837972766 >> v1:172.21.15.113:6789/0 conn(0x7f50d815e320 legacy=0x7f50d80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d6ffd640 1 -- 172.21.15.113:0/837972766 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d80d2ae0 con 0x7f50d80d2290 2024-04-25T00:01:11.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d67fc640 1 -- 172.21.15.113:0/837972766 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 337334209 0 0) 0x7f50d80d1480 con 0x7f50d815e320 2024-04-25T00:01:11.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d6ffd640 1 --2- 172.21.15.113:0/837972766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d80d2290 0x7f50d80b9aa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f50d80b3970 tx=0x7f50cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24dd268052a14a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:11.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d67fc640 1 -- 172.21.15.113:0/837972766 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc037070 con 0x7f50d80d2290 2024-04-25T00:01:11.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d67fc640 1 -- 172.21.15.113:0/837972766 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50cc032440 con 0x7f50d80d2290 2024-04-25T00:01:11.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.609+0000 7f50d67fc640 1 -- 172.21.15.113:0/837972766 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc032810 con 0x7f50d80d2290 2024-04-25T00:01:11.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 -- 172.21.15.113:0/837972766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d80d2290 msgr2=0x7f50d80b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 --2- 172.21.15.113:0/837972766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d80d2290 0x7f50d80b9aa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f50d80b3970 tx=0x7f50cc02a230 comp rx=0 tx=0).stop 2024-04-25T00:01:11.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 -- 172.21.15.113:0/837972766 shutdown_connections 2024-04-25T00:01:11.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 --2- 172.21.15.113:0/837972766 >> v2:172.21.15.178:3300/0 conn(0x7f50d80d0b00 0x7f50d80d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 --2- 172.21.15.113:0/837972766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d80d2290 0x7f50d80b9aa0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 -- 172.21.15.113:0/837972766 >> 172.21.15.113:0/837972766 conn(0x7f50d80b5c20 msgr2=0x7f50d80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:11.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 -- 172.21.15.113:0/837972766 shutdown_connections 2024-04-25T00:01:11.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.610+0000 7f50ddfce640 1 -- 172.21.15.113:0/837972766 wait complete. 2024-04-25T00:01:11.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.611+0000 7f50ddfce640 1 Processor -- start 2024-04-25T00:01:11.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.611+0000 7f50ddfce640 1 -- start start 2024-04-25T00:01:11.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50ddfce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50d80d0b00 0x7f50d8151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50ddfce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50d80d2290 0x7f50d80ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50ddfce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d815e320 0x7f50d80b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50ddfce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50d80b9fe0 con 0x7f50d815e320 2024-04-25T00:01:11.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50ddfce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50d80d30a0 con 0x7f50d80d0b00 2024-04-25T00:01:11.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50ddfce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f50d815e700 con 0x7f50d80d2290 2024-04-25T00:01:11.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d815e320 0x7f50d80b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d815e320 0x7f50d80b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33334/0 (socket says 172.21.15.113:33334) 2024-04-25T00:01:11.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 -- 172.21.15.113:0/2337509899 learned_addr learned my addr 172.21.15.113:0/2337509899 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:11.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 -- 172.21.15.113:0/2337509899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50d80d2290 msgr2=0x7f50d80ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50d80d2290 0x7f50d80ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 -- 172.21.15.113:0/2337509899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50d80d0b00 msgr2=0x7f50d8151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50d80d0b00 0x7f50d8151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.612+0000 7f50d7fff640 1 -- 172.21.15.113:0/2337509899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d81609c0 con 0x7f50d815e320 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.613+0000 7f50d7fff640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d815e320 0x7f50d80b45a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f50d0007e60 tx=0x7f50d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.613+0000 7f50bffff640 1 -- 172.21.15.113:0/2337509899 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0014420 con 0x7f50d815e320 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.613+0000 7f50bffff640 1 -- 172.21.15.113:0/2337509899 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d0019070 con 0x7f50d815e320 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.613+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d8160ca0 con 0x7f50d815e320 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.613+0000 7f50bffff640 1 -- 172.21.15.113:0/2337509899 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0014820 con 0x7f50d815e320 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.613+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d8161260 con 0x7f50d815e320 2024-04-25T00:01:11.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.614+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d80691a0 con 0x7f50d815e320 2024-04-25T00:01:11.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.614+0000 7f50bffff640 1 -- 172.21.15.113:0/2337509899 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50d00149c0 con 0x7f50d815e320 2024-04-25T00:01:11.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.615+0000 7f50bffff640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50c406be30 0x7f50c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:11.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.615+0000 7f50bffff640 1 -- 172.21.15.113:0/2337509899 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f50d008b6c0 con 0x7f50d815e320 2024-04-25T00:01:11.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.615+0000 7f50d77fe640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50c406be30 0x7f50c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:11.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.615+0000 7f50d77fe640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50c406be30 0x7f50c406e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f50c8007fd0 tx=0x7f50c8007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:11.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.619+0000 7f50bffff640 1 -- 172.21.15.113:0/2337509899 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50d005a030 con 0x7f50d815e320 2024-04-25T00:01:11.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.827+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50d80b7680 con 0x7f50d815e320 2024-04-25T00:01:11.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.828+0000 7f50bffff640 1 -- 172.21.15.113:0/2337509899 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5190 (secure 0 0 0) 0x7f50d0059930 con 0x7f50d815e320 2024-04-25T00:01:11.828 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:11.828 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:10.191614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:11.829 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 23 2024-04-25T00:01:11.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50c406be30 msgr2=0x7f50c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50c406be30 0x7f50c406e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f50c8007fd0 tx=0x7f50c8007920 comp rx=0 tx=0).stop 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d815e320 msgr2=0x7f50d80b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d815e320 0x7f50d80b45a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f50d0007e60 tx=0x7f50d000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 shutdown_connections 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50c406be30 0x7f50c406e2f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d815e320 0x7f50d80b45a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50d80d2290 0x7f50d80ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 --2- 172.21.15.113:0/2337509899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50d80d0b00 0x7f50d8151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 >> 172.21.15.113:0/2337509899 conn(0x7f50d80b5c20 msgr2=0x7f50d80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 shutdown_connections 2024-04-25T00:01:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:11.830+0000 7f50ddfce640 1 -- 172.21.15.113:0/2337509899 wait complete. 2024-04-25T00:01:11.842 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T00:01:11.949 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:11.949 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:11.949 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:11.958 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:11.958 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:01:11.958 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:01:11.959 DEBUG:teuthology.orchestra.run.smithi113:> import stat 2024-04-25T00:01:11.959 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:01:11.959 DEBUG:teuthology.orchestra.run.smithi113:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T00:01:11.959 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:01:12.023 INFO:teuthology.orchestra.run.smithi113.stdout:1099511630414 2024-04-25T00:01:12.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:12.032 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T00:01:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 Processor -- start 2024-04-25T00:01:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 -- start start 2024-04-25T00:01:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc63815e510 0x7fc6380d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc6380b61e0 0x7fc638071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc6380d2e00 con 0x7fc6380b61e0 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc6380d30e0 con 0x7fc63815e510 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.171+0000 7fc63eae1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc6380d33c0 con 0x7fc6380d28c0 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc6380b61e0 0x7fc638071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc6380b61e0 0x7fc638071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33350/0 (socket says 172.21.15.113:33350) 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 -- 172.21.15.113:0/2924174979 learned_addr learned my addr 172.21.15.113:0/2924174979 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc637fff640 1 --2- 172.21.15.113:0/2924174979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63c87d640 1 --2- 172.21.15.113:0/2924174979 >> v2:172.21.15.178:3300/0 conn(0x7fc63815e510 0x7fc6380d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 -- 172.21.15.113:0/2924174979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 msgr2=0x7fc6380b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 --2- 172.21.15.113:0/2924174979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 -- 172.21.15.113:0/2924174979 >> v2:172.21.15.178:3300/0 conn(0x7fc63815e510 msgr2=0x7fc6380d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 --2- 172.21.15.113:0/2924174979 >> v2:172.21.15.178:3300/0 conn(0x7fc63815e510 0x7fc6380d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 -- 172.21.15.113:0/2924174979 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6380d36a0 con 0x7fc6380b61e0 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63c87d640 1 --2- 172.21.15.113:0/2924174979 >> v2:172.21.15.178:3300/0 conn(0x7fc63815e510 0x7fc6380d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.172+0000 7fc63d07e640 1 --2- 172.21.15.113:0/2924174979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc638071710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d52a0 tx=0x7fc63002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c86be879274c55b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc6377fe640 1 -- 172.21.15.113:0/2924174979 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630036070 con 0x7fc6380b61e0 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc6377fe640 1 -- 172.21.15.113:0/2924174979 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc630031600 con 0x7fc6380b61e0 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc6377fe640 1 -- 172.21.15.113:0/2924174979 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6300319d0 con 0x7fc6380b61e0 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 -- 172.21.15.113:0/2924174979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 msgr2=0x7fc638071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 --2- 172.21.15.113:0/2924174979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc638071710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d52a0 tx=0x7fc63002a610 comp rx=0 tx=0).stop 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 -- 172.21.15.113:0/2924174979 shutdown_connections 2024-04-25T00:01:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 --2- 172.21.15.113:0/2924174979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc638071710 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 --2- 172.21.15.113:0/2924174979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 --2- 172.21.15.113:0/2924174979 >> v2:172.21.15.178:3300/0 conn(0x7fc63815e510 0x7fc6380d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 -- 172.21.15.113:0/2924174979 >> 172.21.15.113:0/2924174979 conn(0x7fc63806b720 msgr2=0x7fc6380d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 -- 172.21.15.113:0/2924174979 shutdown_connections 2024-04-25T00:01:12.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.173+0000 7fc63eae1640 1 -- 172.21.15.113:0/2924174979 wait complete. 2024-04-25T00:01:12.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.174+0000 7fc63eae1640 1 Processor -- start 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.174+0000 7fc63eae1640 1 -- start start 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63eae1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc6380c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63eae1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63eae1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63815e510 0x7fc63815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63eae1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380b9dd0 con 0x7fc6380b61e0 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63eae1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380d3ea0 con 0x7fc63815e510 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63eae1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc638071c50 con 0x7fc6380d28c0 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63c87d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc6380c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63c87d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc6380c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33352/0 (socket says 172.21.15.113:33352) 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63c87d640 1 -- 172.21.15.113:0/4021618978 learned_addr learned my addr 172.21.15.113:0/4021618978 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc63d07e640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63815e510 0x7fc63815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.175+0000 7fc637fff640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63c87d640 1 -- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 msgr2=0x7fc6380ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63c87d640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63c87d640 1 -- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63815e510 msgr2=0x7fc63815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63c87d640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63815e510 0x7fc63815d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63c87d640 1 -- 172.21.15.113:0/4021618978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6380cc8d0 con 0x7fc6380b61e0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63c87d640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc6380c8680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00c7f0 tx=0x7fc62c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc6357fa640 1 -- 172.21.15.113:0/4021618978 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c015070 con 0x7fc6380b61e0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc6357fa640 1 -- 172.21.15.113:0/4021618978 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62c007820 con 0x7fc6380b61e0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6380c8d20 con 0x7fc6380b61e0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc6357fa640 1 -- 172.21.15.113:0/4021618978 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c007bd0 con 0x7fc6380b61e0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.176+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6380c92e0 con 0x7fc6380b61e0 2024-04-25T00:01:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.177+0000 7fc6357fa640 1 -- 172.21.15.113:0/4021618978 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc62c018070 con 0x7fc6380b61e0 2024-04-25T00:01:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.178+0000 7fc6357fa640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc61c06be30 0x7fc61c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.178+0000 7fc6357fa640 1 -- 172.21.15.113:0/4021618978 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fc62c08ace0 con 0x7fc6380b61e0 2024-04-25T00:01:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.178+0000 7fc637fff640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc61c06be30 0x7fc61c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.178+0000 7fc637fff640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc61c06be30 0x7fc61c06e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc6381f7080 tx=0x7fc624008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 Processor -- start 2024-04-25T00:01:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 -- start start 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770285950 0x55c7702382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770267df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268330 0x55c77023d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c7701a1670 con 0x55c770268a70 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c770149970 con 0x55c770268330 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c77019d630 con 0x55c770285950 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770267df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770267df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33356/0 (socket says 172.21.15.113:33356) 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.179+0000 7fc637fff640 1 -- 172.21.15.113:0/4292666685 learned_addr learned my addr 172.21.15.113:0/4292666685 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc637fff640 1 -- 172.21.15.113:0/4292666685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770285950 msgr2=0x55c7702382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc637fff640 1 --2- 172.21.15.113:0/4292666685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770285950 0x55c7702382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc637fff640 1 -- 172.21.15.113:0/4292666685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268330 msgr2=0x55c77023d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc637fff640 1 --2- 172.21.15.113:0/4292666685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268330 0x55c77023d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc637fff640 1 -- 172.21.15.113:0/4292666685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77016d400 con 0x55c770268a70 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc637fff640 1 --2- 172.21.15.113:0/4292666685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770267df0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc624011ce0 tx=0x7fc624014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3a67d4380793bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc6227fc640 1 -- 172.21.15.113:0/4292666685 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624011d10 con 0x55c770268a70 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc6227fc640 1 -- 172.21.15.113:0/4292666685 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc624026070 con 0x55c770268a70 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.180+0000 7fc6227fc640 1 -- 172.21.15.113:0/4292666685 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624021640 con 0x55c770268a70 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 -- 172.21.15.113:0/4292666685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 msgr2=0x55c770267df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 --2- 172.21.15.113:0/4292666685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770267df0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc624011ce0 tx=0x7fc624014000 comp rx=0 tx=0).stop 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 -- 172.21.15.113:0/4292666685 shutdown_connections 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 --2- 172.21.15.113:0/4292666685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268330 0x55c77023d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 --2- 172.21.15.113:0/4292666685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770267df0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 --2- 172.21.15.113:0/4292666685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770285950 0x55c7702382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 -- 172.21.15.113:0/4292666685 >> 172.21.15.113:0/4292666685 conn(0x55c77023a550 msgr2=0x55c770282df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 -- 172.21.15.113:0/4292666685 shutdown_connections 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 -- 172.21.15.113:0/4292666685 wait complete. 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 -- start start 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.181+0000 7fc6417f5740 1 Processor -- start 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770268a70 0x55c7702524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c77026b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770255970 0x55c770283f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc6417f5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c77023c700 con 0x55c770285950 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c77015d050 con 0x55c770255970 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c77023baf0 con 0x55c770268a70 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc6417f5740 10 client.0 _ll_register_callbacks cb 0x55c76feb4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c77026b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c77026b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33370/0 (socket says 172.21.15.113:33370) 2024-04-25T00:01:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc637fff640 1 -- 172.21.15.113:0/3377753246 learned_addr learned my addr 172.21.15.113:0/3377753246 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc63c87d640 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770268a70 0x55c7702524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.182+0000 7fc63d07e640 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770255970 0x55c770283f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc637fff640 1 -- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770268a70 msgr2=0x55c7702524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc637fff640 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770268a70 0x55c7702524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc637fff640 1 -- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770255970 msgr2=0x55c770283f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc637fff640 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770255970 0x55c770283f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc637fff640 1 -- 172.21.15.113:0/3377753246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77012edc0 con 0x55c770285950 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc637fff640 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c77026b690 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc624006a70 tx=0x7fc6240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f6f3720bf87ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6177fe640 1 -- 172.21.15.113:0/3377753246 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6240217d0 con 0x55c770285950 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6177fe640 1 -- 172.21.15.113:0/3377753246 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fc624021970 con 0x55c770285950 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc624026070 con 0x55c770285950 2024-04-25T00:01:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6177fe640 1 -- 172.21.15.113:0/3377753246 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc62403e050 con 0x55c770285950 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 10 client.6081 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55c76feb53f0 con 0x55c770285950 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6177fe640 1 -- 172.21.15.113:0/3377753246 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc624026070 con 0x55c770285950 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6177fe640 1 -- 172.21.15.113:0/3377753246 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fc624041020 con 0x55c770285950 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 10 client.6081 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 10 client.6081 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 20 client.6081 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c770254140 0x55c770254550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 4 client.6081 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7701aebb0 con 0x55c770254140 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.183+0000 7fc63c87d640 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c770254140 0x55c770254550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6177fe640 10 client.6081 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc63c87d640 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c770254140 0x55c770254550 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2237d79bf779cd7e server_cookie=71ebd66c9ce7c15 in_seq=0 out_seq=0 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6177fe640 1 -- 172.21.15.113:0/3377753246 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7701aebb0 con 0x55c770254140 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6177fe640 10 client.6081 handle_command_reply: tid=0 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6417f5740 1 client.6081 shutdown 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 msgr2=0x55c77026b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c77026b690 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc624006a70 tx=0x7fc6240046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 shutdown_connections 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.184+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c770254140 0x55c770254550 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770255970 0x55c770283f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c77026b690 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6177fe640 0 client.6081 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3377753246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770268a70 0x55c7702524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 >> 172.21.15.113:0/3377753246 conn(0x55c770268330 msgr2=0x55c770282df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 shutdown_connections 2024-04-25T00:01:12.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6417f5740 1 -- 172.21.15.113:0/3377753246 wait complete. 2024-04-25T00:01:12.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.185+0000 7fc6417f5740 20 client.6081 trim_cache size 0 max 16384 2024-04-25T00:01:12.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.198+0000 7fc6417f5740 1 Processor -- start 2024-04-25T00:01:12.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.198+0000 7fc6417f5740 1 -- start start 2024-04-25T00:01:12.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 0x55c7702522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268a70 0x55c770254710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c770262de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc6417f5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc638200830 con 0x55c770285950 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381ff810 con 0x55c770268a70 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc6380cdd40 con 0x55c770255970 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c770262de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c770262de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33382/0 (socket says 172.21.15.113:33382) 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 -- 172.21.15.113:0/3542197345 learned_addr learned my addr 172.21.15.113:0/3542197345 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63c87d640 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 0x55c7702522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc637fff640 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268a70 0x55c770254710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 -- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 msgr2=0x55c7702522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 0x55c7702522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 -- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268a70 msgr2=0x55c770254710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268a70 0x55c770254710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 -- 172.21.15.113:0/3542197345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c76ff7f7d0 con 0x55c770285950 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc63d07e640 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c770262de0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc630009f40 tx=0x7fc63002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a5e71053360854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc621ffb640 1 -- 172.21.15.113:0/3542197345 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630036040 con 0x55c770285950 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc621ffb640 1 -- 172.21.15.113:0/3542197345 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63003d6f0 con 0x55c770285950 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.199+0000 7fc621ffb640 1 -- 172.21.15.113:0/3542197345 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63003daa0 con 0x55c770285950 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 -- 172.21.15.113:0/3542197345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 msgr2=0x55c770262de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c770262de0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc630009f40 tx=0x7fc63002c000 comp rx=0 tx=0).stop 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 -- 172.21.15.113:0/3542197345 shutdown_connections 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770268a70 0x55c770254710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770285950 0x55c770262de0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3542197345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 0x55c7702522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 -- 172.21.15.113:0/3542197345 >> 172.21.15.113:0/3542197345 conn(0x55c770268330 msgr2=0x55c770238100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 -- 172.21.15.113:0/3542197345 shutdown_connections 2024-04-25T00:01:12.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 -- 172.21.15.113:0/3542197345 wait complete. 2024-04-25T00:01:12.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.200+0000 7fc6417f5740 1 -- start start 2024-04-25T00:01:12.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 1 Processor -- start 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 0x55c7702e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770282b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770285950 0x55c770283090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c770252810 con 0x55c770268a70 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380ce390 con 0x55c770285950 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c77023f740 con 0x55c770255970 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc6417f5740 10 client.0 _ll_register_callbacks cb 0x7fc624011bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770282b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770282b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33386/0 (socket says 172.21.15.113:33386) 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.201+0000 7fc637fff640 1 -- 172.21.15.113:0/3989764804 learned_addr learned my addr 172.21.15.113:0/3989764804 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc637fff640 1 -- 172.21.15.113:0/3989764804 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 msgr2=0x55c7702e6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc637fff640 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 0x55c7702e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc637fff640 1 -- 172.21.15.113:0/3989764804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770285950 msgr2=0x55c770283090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc637fff640 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770285950 0x55c770283090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc637fff640 1 -- 172.21.15.113:0/3989764804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c770278780 con 0x55c770268a70 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc637fff640 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770282b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc624004790 tx=0x7fc624017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c8f74dd8aa203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6177fe640 1 -- 172.21.15.113:0/3989764804 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62403b070 con 0x55c770268a70 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6177fe640 1 -- 172.21.15.113:0/3989764804 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fc624036a40 con 0x55c770268a70 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63003d6f0 con 0x55c770268a70 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6177fe640 1 -- 172.21.15.113:0/3989764804 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc624051050 con 0x55c770268a70 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 10 client.6087 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55c7702a6530 con 0x55c770268a70 2024-04-25T00:01:12.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6177fe640 1 -- 172.21.15.113:0/3989764804 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63003d6f0 con 0x55c770268a70 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6177fe640 1 -- 172.21.15.113:0/3989764804 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fc624054020 con 0x55c770268a70 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 10 client.6087 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 10 client.6087 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 20 client.6087 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c7702522d0 0x55c770280980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 4 client.6087 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.202+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c77027af70 con 0x55c7702522d0 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc63c87d640 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c7702522d0 0x55c770280980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6177fe640 10 client.6087 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc63c87d640 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c7702522d0 0x55c770280980 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1493bcf18c9f00f server_cookie=fa5205e4d4bdb619 in_seq=0 out_seq=0 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6177fe640 1 -- 172.21.15.113:0/3989764804 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c77027af70 con 0x55c7702522d0 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6177fe640 10 client.6087 handle_command_reply: tid=0 2024-04-25T00:01:12.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 client.6087 shutdown 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 msgr2=0x55c770282b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770282b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc624004790 tx=0x7fc624017000 comp rx=0 tx=0).stop 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 shutdown_connections 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55c7702522d0 0x55c770280980 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c770285950 0x55c770283090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c770268a70 0x55c770282b50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6177fe640 0 client.6087 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 --2- 172.21.15.113:0/3989764804 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c770255970 0x55c7702e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 >> 172.21.15.113:0/3989764804 conn(0x55c770268330 msgr2=0x55c770265130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 shutdown_connections 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 1 -- 172.21.15.113:0/3989764804 wait complete. 2024-04-25T00:01:12.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.203+0000 7fc6417f5740 20 client.6087 trim_cache size 0 max 16384 2024-04-25T00:01:12.211 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc61c06be30 msgr2=0x7fc61c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc61c06be30 0x7fc61c06e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc6381f7080 tx=0x7fc624008040 comp rx=0 tx=0).stop 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 msgr2=0x7fc6380c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc6380c8680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00c7f0 tx=0x7fc62c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 shutdown_connections 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc61c06be30 0x7fc61c06e2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63815e510 0x7fc63815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6380d28c0 0x7fc6380ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 --2- 172.21.15.113:0/4021618978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6380b61e0 0x7fc6380c8680 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 >> 172.21.15.113:0/4021618978 conn(0x7fc63806b720 msgr2=0x7fc6380b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 shutdown_connections 2024-04-25T00:01:12.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.204+0000 7fc63eae1640 1 -- 172.21.15.113:0/4021618978 wait complete. 2024-04-25T00:01:12.215 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4e.00000000 parent 2024-04-25T00:01:12.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.256+0000 7f36ad07af40 1 Processor -- start 2024-04-25T00:01:12.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.256+0000 7f36ad07af40 1 -- start start 2024-04-25T00:01:12.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ad07af40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 0x555d992abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ad07af40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 0x555d992a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ad07af40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d992a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ad07af40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555d99194640 con 0x555d992ada10 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ad07af40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555d99198680 con 0x555d992368f0 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ad07af40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x555d992ae2b0 con 0x555d992317a0 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d992a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d992a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33400/0 (socket says 172.21.15.113:33400) 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 -- 172.21.15.113:0/2035146495 learned_addr learned my addr 172.21.15.113:0/2035146495 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36aa37a640 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 0x555d992a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36aab7b640 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 0x555d992abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 -- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 msgr2=0x555d992abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 0x555d992abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 -- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 msgr2=0x555d992a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 0x555d992a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 -- 172.21.15.113:0/2035146495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d992a5b60 con 0x555d992ada10 2024-04-25T00:01:12.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.257+0000 7f36ab37c640 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d992a4e80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f36a400a3b0 tx=0x7f36a400a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7c465ddff233d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.259+0000 7f36a9378640 1 -- 172.21.15.113:0/2035146495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a400b8c0 con 0x555d992ada10 2024-04-25T00:01:12.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36a9378640 1 -- 172.21.15.113:0/2035146495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a400f030 con 0x555d992ada10 2024-04-25T00:01:12.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36a9378640 1 -- 172.21.15.113:0/2035146495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a400bd00 con 0x555d992ada10 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 -- 172.21.15.113:0/2035146495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 msgr2=0x555d992a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d992a4e80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f36a400a3b0 tx=0x7f36a400a870 comp rx=0 tx=0).stop 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 -- 172.21.15.113:0/2035146495 shutdown_connections 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 0x555d992a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d992a4e80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 --2- 172.21.15.113:0/2035146495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 0x555d992abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 -- 172.21.15.113:0/2035146495 >> 172.21.15.113:0/2035146495 conn(0x555d992181d0 msgr2=0x555d9921d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 -- 172.21.15.113:0/2035146495 shutdown_connections 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.260+0000 7f36ad07af40 1 -- 172.21.15.113:0/2035146495 wait complete. 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 Processor -- start 2024-04-25T00:01:12.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 -- start start 2024-04-25T00:01:12.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 0x555d992c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 0x555d992a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d991eb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555d992ac7a0 con 0x555d992ada10 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555d992c4390 con 0x555d992368f0 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ad07af40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x555d99235d60 con 0x555d992317a0 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ab37c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d991eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.261+0000 7f36ab37c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d991eb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33414/0 (socket says 172.21.15.113:33414) 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ab37c640 1 -- 172.21.15.113:0/1625449775 learned_addr learned my addr 172.21.15.113:0/1625449775 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36aab7b640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 0x555d992c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36aa37a640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 0x555d992a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ab37c640 1 -- 172.21.15.113:0/1625449775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 msgr2=0x555d992c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ab37c640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x555d992317a0 0x555d992c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ab37c640 1 -- 172.21.15.113:0/1625449775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 msgr2=0x555d992a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ab37c640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d992368f0 0x555d992a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ab37c640 1 -- 172.21.15.113:0/1625449775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d991eed90 con 0x555d992ada10 2024-04-25T00:01:12.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ab37c640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d991eb790 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f36a400ad70 tx=0x7f36a4002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36a3fff640 1 -- 172.21.15.113:0/1625449775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4024070 con 0x555d992ada10 2024-04-25T00:01:12.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36a3fff640 1 -- 172.21.15.113:0/1625449775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a4014420 con 0x555d992ada10 2024-04-25T00:01:12.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d991ef070 con 0x555d992ada10 2024-04-25T00:01:12.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36a3fff640 1 -- 172.21.15.113:0/1625449775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4014810 con 0x555d992ada10 2024-04-25T00:01:12.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.262+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9921c790 con 0x555d992ada10 2024-04-25T00:01:12.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.263+0000 7f36a3fff640 1 -- 172.21.15.113:0/1625449775 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36a4027070 con 0x555d992ada10 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36a3fff640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f368c06a810 0x7f368c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36aab7b640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f368c06a810 0x7f368c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36a3fff640 1 -- 172.21.15.113:0/1625449775 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f36a408a920 con 0x555d992ada10 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36ad07af40 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x555d9921ccb0 0x555d992a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 --> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] -- osd_op(unknown.0.0:1 7.2 7:43adcc81:::10000000a4e.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) -- 0x555d992a55b0 con 0x555d9921ccb0 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36aa37a640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x555d9921ccb0 0x555d992a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36aab7b640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f368c06a810 0x7f368c06ccd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f369c057000 tx=0x7f369c007030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.264+0000 7f36aa37a640 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x555d9921ccb0 0x555d992a6d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.265+0000 7f36aa37a640 1 -- 172.21.15.113:0/1625449775 <== osd.3 v2:172.21.15.113:6824/1975200730 1 ==== osd_op_reply(1 10000000a4e.00000000 [rmxattr] v48'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f36940107c0 con 0x555d9921ccb0 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x555d9921ccb0 msgr2=0x555d992a6d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x555d9921ccb0 0x555d992a6d80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f368c06a810 msgr2=0x7f368c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f368c06a810 0x7f368c06ccd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f369c057000 tx=0x7f369c007030 comp rx=0 tx=0).stop 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 msgr2=0x555d991eb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 --2- 172.21.15.113:0/1625449775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d992ada10 0x555d991eb790 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f36a400ad70 tx=0x7f36a4002f70 comp rx=0 tx=0).stop 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ab37c640 1 -- 172.21.15.113:0/1625449775 reap_dead start 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 shutdown_connections 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 >> 172.21.15.113:0/1625449775 conn(0x555d992181d0 msgr2=0x555d992a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 shutdown_connections 2024-04-25T00:01:12.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.266+0000 7f36ad07af40 1 -- 172.21.15.113:0/1625449775 wait complete. 2024-04-25T00:01:12.269 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T00:01:12.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T00:01:12.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.409+0000 7f5acdfdc640 1 Processor -- start 2024-04-25T00:01:12.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.409+0000 7f5acdfdc640 1 -- start start 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5acdfdc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 0x7f5ac815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5acdfdc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac8071170 con 0x7f5ac806cea0 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5acdfdc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac8071450 con 0x7f5ac815bff0 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5acdfdc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac815c470 con 0x7f5ac815b670 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5ac7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5ac806cea0 0x7f5ac8072210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52390/0 (socket says 172.21.15.113:52390) 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5ac7fff640 1 -- 172.21.15.113:0/56588381 learned_addr learned my addr 172.21.15.113:0/56588381 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 0x7f5ac815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4160188452 0 0) 0x7f5ac8071170 con 0x7f5ac806cea0 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.410+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ab4003520 con 0x7f5ac806cea0 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3647640095 0 0) 0x7f5ab4003520 con 0x7f5ac806cea0 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 msgr2=0x7f5ac815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 0x7f5ac815ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 >> v1:172.21.15.178:6789/0 conn(0x7f5ac815bff0 legacy=0x7f5ac80cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac8071730 con 0x7f5ac806cea0 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f5ac00019f0 con 0x7f5ac806cea0 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 >> v1:172.21.15.113:6789/0 conn(0x7f5ac806cea0 legacy=0x7f5ac8072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 0x7f5ab40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4003f60 0x7f5ab4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4004d40 0x7f5ab40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac815c470 con 0x7f5ab4004d40 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab400de10 con 0x7f5ab4003f60 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab400e010 con 0x7f5ac815b670 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4004d40 0x7f5ab40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 0x7f5ab40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4003f60 0x7f5ab4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac7fff640 1 -- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 msgr2=0x7f5ab40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac815b670 0x7f5ab40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac7fff640 1 -- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4003f60 msgr2=0x7f5ab4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4003f60 0x7f5ab4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac7fff640 1 -- 172.21.15.113:0/56588381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab4003520 con 0x7f5ab4004d40 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.411+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4003f60 0x7f5ab4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4004d40 0x7f5ab40051c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000a860 tx=0x7f5ab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe7ba8c4d72347e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000a480 con 0x7f5ab4004d40 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac002f070 con 0x7f5ab4004d40 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5ac67fc640 1 -- 172.21.15.113:0/56588381 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac002bd40 con 0x7f5ab4004d40 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5acdfdc640 1 -- 172.21.15.113:0/56588381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4004d40 msgr2=0x7f5ab40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5acdfdc640 1 --2- 172.21.15.113:0/56588381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4004d40 0x7f5ab40051c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000a860 tx=0x7f5ab4005830 comp rx=0 tx=0).stop 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5ac7fff640 1 -- 172.21.15.113:0/56588381 reap_dead start 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5acdfdc640 1 -- 172.21.15.113:0/56588381 shutdown_connections 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5acdfdc640 1 -- 172.21.15.113:0/56588381 >> 172.21.15.113:0/56588381 conn(0x7f5ac806b0f0 msgr2=0x7f5ac806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.412+0000 7f5acdfdc640 1 -- 172.21.15.113:0/56588381 shutdown_connections 2024-04-25T00:01:12.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.413+0000 7f5acdfdc640 1 -- 172.21.15.113:0/56588381 wait complete. 2024-04-25T00:01:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.414+0000 7f5acdfdc640 1 Processor -- start 2024-04-25T00:01:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.414+0000 7f5acdfdc640 1 -- start start 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4004d40 0x7f5ac81e4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac81e2da0 0x7f5ac80c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ac81e3220 0x7f5ac80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80cca90 con 0x7f5ac81e3220 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80cc910 con 0x7f5ab4004d40 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac815ba50 con 0x7f5ac81e2da0 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ac81e3220 0x7f5ac80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ac81e3220 0x7f5ac80ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33424/0 (socket says 172.21.15.113:33424) 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 -- 172.21.15.113:0/648426414 learned_addr learned my addr 172.21.15.113:0/648426414 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4004d40 0x7f5ac81e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac81e2da0 0x7f5ac80c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 -- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac81e2da0 msgr2=0x7f5ac80c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac81e2da0 0x7f5ac80c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 -- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4004d40 msgr2=0x7f5ac81e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4004d40 0x7f5ac81e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 -- 172.21.15.113:0/648426414 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac8148700 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ac81e3220 0x7f5ac80ca2e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5ac002b530 tx=0x7f5ac815ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5abffff640 1 -- 172.21.15.113:0/648426414 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000a480 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5abffff640 1 -- 172.21.15.113:0/648426414 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0004b60 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81e37f0 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5abffff640 1 -- 172.21.15.113:0/648426414 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0034470 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.415+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac80c95a0 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.417+0000 7f5abffff640 1 -- 172.21.15.113:0/648426414 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ac0034610 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.417+0000 7f5abffff640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5aa806be60 0x7f5aa806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.417+0000 7f5abffff640 1 -- 172.21.15.113:0/648426414 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f5ac00aad30 con 0x7f5ac81e3220 2024-04-25T00:01:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.417+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5aa806be60 0x7f5aa806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.418+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5aa806be60 0x7f5aa806e320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009bc0 tx=0x7f5ab0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.418+0000 7f5ad0dc9740 1 Processor -- start 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.418+0000 7f5ad0dc9740 1 -- start start 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8b849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8ba23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b5a2b0 0x55e9a8ba0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e9a8abc660 con 0x55e9a8b5a2b0 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e9a8a64800 con 0x55e9a8ba1e70 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55e9a8ab8620 con 0x55e9a8b845d0 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b5a2b0 0x55e9a8ba0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b5a2b0 0x55e9a8ba0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33436/0 (socket says 172.21.15.113:33436) 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ac7fff640 1 -- 172.21.15.113:0/3261877003 learned_addr learned my addr 172.21.15.113:0/3261877003 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8ba23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.419+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8b849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ac7fff640 1 -- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 msgr2=0x55e9a8b849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8b849b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ac7fff640 1 -- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 msgr2=0x55e9a8ba23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8ba23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ac7fff640 1 -- 172.21.15.113:0/3261877003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a889cc00 con 0x55e9a8b5a2b0 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b5a2b0 0x55e9a8ba0530 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0081650 tx=0x7f5ac0078e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6904ec93e32fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5abd7fa640 1 -- 172.21.15.113:0/3261877003 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0078fa0 con 0x55e9a8b5a2b0 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5abd7fa640 1 -- 172.21.15.113:0/3261877003 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac002f040 con 0x55e9a8b5a2b0 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5abd7fa640 1 -- 172.21.15.113:0/3261877003 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac007d350 con 0x55e9a8b5a2b0 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/3261877003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b5a2b0 msgr2=0x55e9a8ba0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.420+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b5a2b0 0x55e9a8ba0530 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0081650 tx=0x7f5ac0078e80 comp rx=0 tx=0).stop 2024-04-25T00:01:12.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/3261877003 shutdown_connections 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8ba23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b5a2b0 0x55e9a8ba0530 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/3261877003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8b849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/3261877003 >> 172.21.15.113:0/3261877003 conn(0x55e9a8b56be0 msgr2=0x55e9a8b9f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/3261877003 shutdown_connections 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/3261877003 wait complete. 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 -- start start 2024-04-25T00:01:12.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.421+0000 7f5ad0dc9740 1 Processor -- start 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b845d0 0x55e9a8b983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b53860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b75f50 0x55e9a8b53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e9a8ab8620 con 0x55e9a8ba1e70 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e9a8a77f10 con 0x55e9a8b845d0 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55e9a8abc660 con 0x55e9a8b75f50 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ad0dc9740 10 client.0 _ll_register_callbacks cb 0x55e9a87cffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b53860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b53860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33448/0 (socket says 172.21.15.113:33448) 2024-04-25T00:01:12.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 -- 172.21.15.113:0/1696432802 learned_addr learned my addr 172.21.15.113:0/1696432802 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b845d0 0x55e9a8b983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b75f50 0x55e9a8b53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 -- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b75f50 msgr2=0x55e9a8b53da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b75f50 0x55e9a8b53da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 -- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b845d0 msgr2=0x55e9a8b983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b845d0 0x55e9a8b983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 -- 172.21.15.113:0/1696432802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a87904f0 con 0x55e9a8ba1e70 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b75f50 0x55e9a8b53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b845d0 0x55e9a8b983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b53860 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8003d40 tx=0x7f5ab8001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccc615d28e7bf31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.422+0000 7f5aa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5aa67fc640 1 -- 172.21.15.113:0/1696432802 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80155c0 con 0x55e9a8ba1e70 2024-04-25T00:01:12.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5aa67fc640 1 -- 172.21.15.113:0/1696432802 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f5ab80197c0 con 0x55e9a8ba1e70 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac002f040 con 0x55e9a8ba1e70 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5aa67fc640 1 -- 172.21.15.113:0/1696432802 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5ab80199c0 con 0x55e9a8ba1e70 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 10 client.6111 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55e9a88fd390 con 0x55e9a8ba1e70 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5aa67fc640 1 -- 172.21.15.113:0/1696432802 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac002f040 con 0x55e9a8ba1e70 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5aa67fc640 1 -- 172.21.15.113:0/1696432802 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f5ab8019b60 con 0x55e9a8ba1e70 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 10 client.6111 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 10 client.6111 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 20 client.6111 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8baf9f0 0x55e9a8bafdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 4 client.6111 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9a8ac9ba0 con 0x55e9a8baf9f0 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8baf9f0 0x55e9a8bafdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5aa67fc640 10 client.6111 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.423+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8baf9f0 0x55e9a8bafdf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10abe71648f1a2c2 server_cookie=f120eff90e559273 in_seq=0 out_seq=0 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5aa67fc640 1 -- 172.21.15.113:0/1696432802 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9a8ac9ba0 con 0x55e9a8baf9f0 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5aa67fc640 10 client.6111 handle_command_reply: tid=0 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 client.6111 shutdown 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 msgr2=0x55e9a8b53860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b53860 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8003d40 tx=0x7f5ab8001680 comp rx=0 tx=0).stop 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 shutdown_connections 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8baf9f0 0x55e9a8bafdf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b75f50 0x55e9a8b53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5aa67fc640 0 client.6111 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b53860 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1696432802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b845d0 0x55e9a8b983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 >> 172.21.15.113:0/1696432802 conn(0x55e9a8b5a2b0 msgr2=0x55e9a8ba31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 shutdown_connections 2024-04-25T00:01:12.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1696432802 wait complete. 2024-04-25T00:01:12.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.424+0000 7f5ad0dc9740 20 client.6111 trim_cache size 0 max 16384 2024-04-25T00:01:12.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 Processor -- start 2024-04-25T00:01:12.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 -- start start 2024-04-25T00:01:12.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b75f50 0x55e9a8b80c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8baffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b98110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8200fa0 con 0x55e9a8ba1e70 2024-04-25T00:01:12.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81fff80 con 0x55e9a8b75f50 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.442+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8147d80 con 0x55e9a8b845d0 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b98110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b98110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33464/0 (socket says 172.21.15.113:33464) 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 -- 172.21.15.113:0/1295691080 learned_addr learned my addr 172.21.15.113:0/1295691080 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8baffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b75f50 0x55e9a8b80c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 -- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 msgr2=0x55e9a8baffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8baffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 -- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b75f50 msgr2=0x55e9a8b80c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b75f50 0x55e9a8b80c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 -- 172.21.15.113:0/1295691080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a8769670 con 0x55e9a8ba1e70 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b98110 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0078ca0 tx=0x7f5ac0037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a3dd9233c17f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5abcff9640 1 -- 172.21.15.113:0/1295691080 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac007d350 con 0x55e9a8ba1e70 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5abcff9640 1 -- 172.21.15.113:0/1295691080 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac002f040 con 0x55e9a8ba1e70 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.443+0000 7f5abcff9640 1 -- 172.21.15.113:0/1295691080 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0078fa0 con 0x55e9a8ba1e70 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1295691080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 msgr2=0x55e9a8b98110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b98110 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0078ca0 tx=0x7f5ac0037000 comp rx=0 tx=0).stop 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1295691080 shutdown_connections 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b98110 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8baffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/1295691080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8b75f50 0x55e9a8b80c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1295691080 >> 172.21.15.113:0/1295691080 conn(0x55e9a8b5a2b0 msgr2=0x55e9a8b4fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1295691080 shutdown_connections 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/1295691080 wait complete. 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 -- start start 2024-04-25T00:01:12.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.444+0000 7f5ad0dc9740 1 Processor -- start 2024-04-25T00:01:12.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b75f50 0x55e9a8c18510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8ba2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ad0dc9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e9a8a64800 con 0x55e9a8b75f50 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81ff910 con 0x55e9a8ba1e70 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ad0dc9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55e9a8b83480 con 0x55e9a8b845d0 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ad0dc9740 10 client.0 _ll_register_callbacks cb 0x7f5ab8005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b75f50 0x55e9a8c18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b75f50 0x55e9a8c18510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33472/0 (socket says 172.21.15.113:33472) 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ac77fe640 1 -- 172.21.15.113:0/966365911 learned_addr learned my addr 172.21.15.113:0/966365911 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8ba2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ac7fff640 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.445+0000 7f5ac77fe640 1 -- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 msgr2=0x55e9a8ba2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8ba2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ac77fe640 1 -- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 msgr2=0x55e9a8b743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ac77fe640 1 -- 172.21.15.113:0/966365911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a8b557b0 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ac77fe640 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b75f50 0x55e9a8c18510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55e9a8b96c50 tx=0x7f5ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea20b9006c30cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5aa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5aa67fc640 1 -- 172.21.15.113:0/966365911 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab001b070 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5aa67fc640 1 -- 172.21.15.113:0/966365911 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f5ab002d730 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac002f040 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5aa67fc640 1 -- 172.21.15.113:0/966365911 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5ab003e050 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 10 client.6117 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55e9a8b967f0 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5aa67fc640 1 -- 172.21.15.113:0/966365911 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac002f040 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5aa67fc640 1 -- 172.21.15.113:0/966365911 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f5ab0003b90 con 0x55e9a8b75f50 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 10 client.6117 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:12.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 10 client.6117 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 20 client.6117 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8ba32f0 0x55e9a8b74900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 4 client.6117 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) -- 0x55e9a8ac3ef0 con 0x55e9a8ba32f0 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.446+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8ba32f0 0x55e9a8b74900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5aa67fc640 10 client.6117 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ac6ffd640 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8ba32f0 0x55e9a8b74900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93280e6a08f6cb21 server_cookie=bb4aa3695535064d in_seq=0 out_seq=0 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5aa67fc640 1 -- 172.21.15.113:0/966365911 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e9a8ac3ef0 con 0x55e9a8ba32f0 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5aa67fc640 10 client.6117 handle_command_reply: tid=0 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 client.6117 shutdown 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b75f50 msgr2=0x55e9a8c18510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b75f50 0x55e9a8c18510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55e9a8b96c50 tx=0x7f5ab00046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 shutdown_connections 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55e9a8ba32f0 0x55e9a8b74900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9a8ba1e70 0x55e9a8b743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55e9a8b845d0 0x55e9a8ba2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 --2- 172.21.15.113:0/966365911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9a8b75f50 0x55e9a8c18510 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 >> 172.21.15.113:0/966365911 conn(0x55e9a8b5a2b0 msgr2=0x55e9a8b4ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5aa67fc640 0 client.6117 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 shutdown_connections 2024-04-25T00:01:12.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 1 -- 172.21.15.113:0/966365911 wait complete. 2024-04-25T00:01:12.454 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:01:12.447+0000 7fb09ec87640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4e [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v466 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5715=0-4194304@1} caps={5715=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5715 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55916b876000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000a4e:[<0x10000000001/new_file_1 v466>,<0x10000000000/ceph-qa-suite v51>,<0x1/scrub_test_path v19>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T00:01:12.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.447+0000 7f5ad0dc9740 20 client.6117 trim_cache size 0 max 16384 2024-04-25T00:01:12.455 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:12.455 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:12.455 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "3910791e-560c-4a32-b96e-d859b6c15273", 2024-04-25T00:01:12.455 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:12.455 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:12.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5aa806be60 msgr2=0x7f5aa806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5aa806be60 0x7f5aa806e320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009bc0 tx=0x7f5ab0009190 comp rx=0 tx=0).stop 2024-04-25T00:01:12.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ac81e3220 msgr2=0x7f5ac80ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ac81e3220 0x7f5ac80ca2e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5ac002b530 tx=0x7f5ac815ae30 comp rx=0 tx=0).stop 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 shutdown_connections 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5aa806be60 0x7f5aa806e320 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ac81e3220 0x7f5ac80ca2e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac81e2da0 0x7f5ac80c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 --2- 172.21.15.113:0/648426414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab4004d40 0x7f5ac81e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 >> 172.21.15.113:0/648426414 conn(0x7f5ac806b0f0 msgr2=0x7f5ac80b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 shutdown_connections 2024-04-25T00:01:12.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.448+0000 7f5acdfdc640 1 -- 172.21.15.113:0/648426414 wait complete. 2024-04-25T00:01:12.459 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '3910791e-560c-4a32-b96e-d859b6c15273', 'mode': 'asynchronous'}' 2024-04-25T00:01:12.459 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T00:01:12.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T00:01:12.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.557+0000 7f56f7d29640 1 Processor -- start 2024-04-25T00:01:12.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.557+0000 7f56f7d29640 1 -- start start 2024-04-25T00:01:12.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f7d29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f015e2d0 0x7f56f015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f7d29640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56f00b4a40 0x7f56f00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f7d29640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f015f4f0 con 0x7f56f00b3080 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f7d29640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f56f00cde00 con 0x7f56f00b4a40 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f7d29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56f015f7d0 con 0x7f56f015e2d0 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f62c6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56f00b4a40 0x7f56f00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f52c4640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f56f00b3080 0x7f56f00b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52406/0 (socket says 172.21.15.113:52406) 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f52c4640 1 -- 172.21.15.113:0/1214565832 learned_addr learned my addr 172.21.15.113:0/1214565832 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f62c6640 1 -- 172.21.15.113:0/1214565832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f015e2d0 msgr2=0x7f56f015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/1214565832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f015e2d0 0x7f56f015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f62c6640 1 --2- 172.21.15.113:0/1214565832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f015e2d0 0x7f56f015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f62c6640 1 -- 172.21.15.113:0/1214565832 >> v1:172.21.15.113:6789/0 conn(0x7f56f00b3080 legacy=0x7f56f00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f4ac3640 1 -- 172.21.15.113:0/1214565832 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009028734 0 0) 0x7f56f015f4f0 con 0x7f56f00b3080 2024-04-25T00:01:12.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.558+0000 7f56f62c6640 1 -- 172.21.15.113:0/1214565832 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f015fab0 con 0x7f56f00b4a40 2024-04-25T00:01:12.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.559+0000 7f56f62c6640 1 --2- 172.21.15.113:0/1214565832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f00b4a40 0x7f56f00ce290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f56e80097a0 tx=0x7f56e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0be23d3c385ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.559+0000 7f56f4ac3640 1 -- 172.21.15.113:0/1214565832 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8037070 con 0x7f56f00b4a40 2024-04-25T00:01:12.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.559+0000 7f56f4ac3640 1 -- 172.21.15.113:0/1214565832 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e8032440 con 0x7f56f00b4a40 2024-04-25T00:01:12.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.559+0000 7f56f4ac3640 1 -- 172.21.15.113:0/1214565832 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8032810 con 0x7f56f00b4a40 2024-04-25T00:01:12.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 -- 172.21.15.113:0/1214565832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f00b4a40 msgr2=0x7f56f00ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 --2- 172.21.15.113:0/1214565832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f00b4a40 0x7f56f00ce290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f56e80097a0 tx=0x7f56e802a180 comp rx=0 tx=0).stop 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 -- 172.21.15.113:0/1214565832 shutdown_connections 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 --2- 172.21.15.113:0/1214565832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f00b4a40 0x7f56f00ce290 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 --2- 172.21.15.113:0/1214565832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f015e2d0 0x7f56f015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 -- 172.21.15.113:0/1214565832 >> 172.21.15.113:0/1214565832 conn(0x7f56f006b0f0 msgr2=0x7f56f006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 -- 172.21.15.113:0/1214565832 shutdown_connections 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.560+0000 7f56f7d29640 1 -- 172.21.15.113:0/1214565832 wait complete. 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.561+0000 7f56f7d29640 1 Processor -- start 2024-04-25T00:01:12.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.561+0000 7f56f7d29640 1 -- start start 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f7d29640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f00b3080 0x7f56f01f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f7d29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f00b4a40 0x7f56f014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f7d29640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f015e2d0 0x7f56f015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f7d29640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56f015f2b0 con 0x7f56f015e2d0 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f7d29640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56f015f130 con 0x7f56f00b3080 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f7d29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56f015e6b0 con 0x7f56f00b4a40 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f015e2d0 0x7f56f015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f015e2d0 0x7f56f015c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33502/0 (socket says 172.21.15.113:33502) 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 -- 172.21.15.113:0/605175206 learned_addr learned my addr 172.21.15.113:0/605175206 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f00b3080 0x7f56f01f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f52c4640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f00b4a40 0x7f56f014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 -- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f00b4a40 msgr2=0x7f56f014eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f00b4a40 0x7f56f014eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 -- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f00b3080 msgr2=0x7f56f01f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f00b3080 0x7f56f01f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 -- 172.21.15.113:0/605175206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f01f5920 con 0x7f56f015e2d0 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f52c4640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f00b4a40 0x7f56f014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f62c6640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f015e2d0 0x7f56f015c0c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f56e80097a0 tx=0x7f56e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56e67fc640 1 -- 172.21.15.113:0/605175206 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8037040 con 0x7f56f015e2d0 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56e67fc640 1 -- 172.21.15.113:0/605175206 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e8042070 con 0x7f56f015e2d0 2024-04-25T00:01:12.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f01f5c00 con 0x7f56f015e2d0 2024-04-25T00:01:12.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.562+0000 7f56e67fc640 1 -- 172.21.15.113:0/605175206 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8030b30 con 0x7f56f015e2d0 2024-04-25T00:01:12.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.563+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f0150aa0 con 0x7f56f015e2d0 2024-04-25T00:01:12.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.564+0000 7f56e67fc640 1 -- 172.21.15.113:0/605175206 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56e8030cd0 con 0x7f56f015e2d0 2024-04-25T00:01:12.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.564+0000 7f56e67fc640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56d406be30 0x7f56d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.564+0000 7f56e67fc640 1 -- 172.21.15.113:0/605175206 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f56e80aba50 con 0x7f56f015e2d0 2024-04-25T00:01:12.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.564+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56d406be30 0x7f56d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.565+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56d406be30 0x7f56d406e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00f360 tx=0x7f56ec007710 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.565+0000 7f56faa3b740 1 Processor -- start 2024-04-25T00:01:12.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.565+0000 7f56faa3b740 1 -- start start 2024-04-25T00:01:12.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a29e3d60 0x55f1a29e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a2a01600 0x55f1a2a01b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29b9a40 0x55f1a29ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56faa3b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f1a291d6b0 con 0x55f1a29e3d60 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f1a28c5880 con 0x55f1a29b9a40 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f1a2919670 con 0x55f1a2a01600 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a29e3d60 0x55f1a29e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a29e3d60 0x55f1a29e4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33510/0 (socket says 172.21.15.113:33510) 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 -- 172.21.15.113:0/3806696295 learned_addr learned my addr 172.21.15.113:0/3806696295 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f62c6640 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29b9a40 0x55f1a29ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 -- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a2a01600 msgr2=0x55f1a2a01b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f52c4640 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a2a01600 0x55f1a2a01b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a2a01600 0x55f1a2a01b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 -- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29b9a40 msgr2=0x55f1a29ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29b9a40 0x55f1a29ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f5ac5640 1 -- 172.21.15.113:0/3806696295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a26fb7d0 con 0x55f1a29e3d60 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.566+0000 7f56f62c6640 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29b9a40 0x55f1a29ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a29e3d60 0x55f1a29e4140 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f56ec011520 tx=0x7f56ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea84cf6dc08cc0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56d37fe640 1 -- 172.21.15.113:0/3806696295 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec022070 con 0x55f1a29e3d60 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56d37fe640 1 -- 172.21.15.113:0/3806696295 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec010690 con 0x55f1a29e3d60 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56d37fe640 1 -- 172.21.15.113:0/3806696295 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec010a40 con 0x55f1a29e3d60 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 -- 172.21.15.113:0/3806696295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a29e3d60 msgr2=0x55f1a29e4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a29e3d60 0x55f1a29e4140 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f56ec011520 tx=0x7f56ec012000 comp rx=0 tx=0).stop 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 -- 172.21.15.113:0/3806696295 shutdown_connections 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29b9a40 0x55f1a29ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a2a01600 0x55f1a2a01b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 --2- 172.21.15.113:0/3806696295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a29e3d60 0x55f1a29e4140 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 -- 172.21.15.113:0/3806696295 >> 172.21.15.113:0/3806696295 conn(0x55f1a29b6600 msgr2=0x55f1a29feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 -- 172.21.15.113:0/3806696295 shutdown_connections 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.567+0000 7f56faa3b740 1 -- 172.21.15.113:0/3806696295 wait complete. 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.568+0000 7f56faa3b740 1 -- start start 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.568+0000 7f56faa3b740 1 Processor -- start 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a29e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56faa3b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f1a29ba7e0 con 0x55f1a2a01600 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f1a28d8f50 con 0x55f1a29e3d60 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f1a29b4530 con 0x55f1a29cf160 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56faa3b740 10 client.0 _ll_register_callbacks cb 0x55f1a2914000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33516/0 (socket says 172.21.15.113:33516) 2024-04-25T00:01:12.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 -- 172.21.15.113:0/2465385148 learned_addr learned my addr 172.21.15.113:0/2465385148 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a29e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f62c6640 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 -- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 msgr2=0x55f1a29cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 -- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 msgr2=0x55f1a29e6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a29e6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.569+0000 7f56f52c4640 1 -- 172.21.15.113:0/2465385148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a26313f0 con 0x55f1a2a01600 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56f52c4640 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29e2f20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f56e0007fd0 tx=0x7f56e000e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c04df8177c93482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2465385148 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0019060 con 0x55f1a2a01600 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2465385148 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f56e0014b30 con 0x55f1a2a01600 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56ec010690 con 0x55f1a2a01600 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2465385148 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56e0014d30 con 0x55f1a2a01600 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 10 client.6132 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f1a25f10f0 con 0x55f1a2a01600 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2465385148 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec010690 con 0x55f1a2a01600 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2465385148 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f56e00072d0 con 0x55f1a2a01600 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 10 client.6132 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 10 client.6132 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 20 client.6132 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29ce3f0 0x55f1a29ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 4 client.6132 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:12.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1a292abf0 con 0x55f1a29ce3f0 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29ce3f0 0x55f1a29ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56d0ff9640 10 client.6132 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.570+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29ce3f0 0x55f1a29ce840 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34759c4a58fa0625 server_cookie=18dd560372ff02ab in_seq=0 out_seq=0 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2465385148 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1a292abf0 con 0x55f1a29ce3f0 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56d0ff9640 10 client.6132 handle_command_reply: tid=0 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 client.6132 shutdown 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 msgr2=0x55f1a29e2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29e2f20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f56e0007fd0 tx=0x7f56e000e590 comp rx=0 tx=0).stop 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 shutdown_connections 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29ce3f0 0x55f1a29ce840 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29e2f20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2465385148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a29e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56d0ff9640 0 client.6132 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 >> 172.21.15.113:0/2465385148 conn(0x55f1a29b9a40 msgr2=0x55f1a2a020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 shutdown_connections 2024-04-25T00:01:12.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 1 -- 172.21.15.113:0/2465385148 wait complete. 2024-04-25T00:01:12.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.571+0000 7f56faa3b740 20 client.6132 trim_cache size 0 max 16384 2024-04-25T00:01:12.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 Processor -- start 2024-04-25T00:01:12.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 -- start start 2024-04-25T00:01:12.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29cf160 0x55f1a29ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29e3d60 0x55f1a2a229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29df590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01fe5a0 con 0x55f1a2a01600 2024-04-25T00:01:12.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01fd5c0 con 0x55f1a29cf160 2024-04-25T00:01:12.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56f0151ca0 con 0x55f1a29e3d60 2024-04-25T00:01:12.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56f62c6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29df590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56f62c6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29df590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33522/0 (socket says 172.21.15.113:33522) 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.589+0000 7f56f62c6640 1 -- 172.21.15.113:0/143813525 learned_addr learned my addr 172.21.15.113:0/143813525 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56f62c6640 1 -- 172.21.15.113:0/143813525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29e3d60 msgr2=0x55f1a2a229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56f62c6640 1 --2- 172.21.15.113:0/143813525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29e3d60 0x55f1a2a229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56f62c6640 1 -- 172.21.15.113:0/143813525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29cf160 msgr2=0x55f1a29ff3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56f62c6640 1 --2- 172.21.15.113:0/143813525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29cf160 0x55f1a29ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56f62c6640 1 -- 172.21.15.113:0/143813525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a29cf930 con 0x55f1a2a01600 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56f62c6640 1 --2- 172.21.15.113:0/143813525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29df590 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f56e8076120 tx=0x7f56e8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f320d59014dbe8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56d2ffd640 1 -- 172.21.15.113:0/143813525 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8079cc0 con 0x55f1a2a01600 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56d2ffd640 1 -- 172.21.15.113:0/143813525 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e80a8070 con 0x55f1a2a01600 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56d2ffd640 1 -- 172.21.15.113:0/143813525 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e807e070 con 0x55f1a2a01600 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 -- 172.21.15.113:0/143813525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 msgr2=0x55f1a29df590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 --2- 172.21.15.113:0/143813525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29df590 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f56e8076120 tx=0x7f56e8009190 comp rx=0 tx=0).stop 2024-04-25T00:01:12.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 -- 172.21.15.113:0/143813525 shutdown_connections 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 --2- 172.21.15.113:0/143813525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29df590 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 --2- 172.21.15.113:0/143813525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29e3d60 0x55f1a2a229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 --2- 172.21.15.113:0/143813525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29cf160 0x55f1a29ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 -- 172.21.15.113:0/143813525 >> 172.21.15.113:0/143813525 conn(0x55f1a29b9a40 msgr2=0x55f1a29e2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.590+0000 7f56faa3b740 1 -- 172.21.15.113:0/143813525 shutdown_connections 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.591+0000 7f56faa3b740 1 -- 172.21.15.113:0/143813525 wait complete. 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.591+0000 7f56faa3b740 1 -- start start 2024-04-25T00:01:12.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.591+0000 7f56faa3b740 1 Processor -- start 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29f5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a2a78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56faa3b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56faa3b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56f006daa0 con 0x55f1a2a01600 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f1a29b6dd0 con 0x55f1a29e3d60 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56faa3b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f1a29de860 con 0x55f1a29cf160 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56faa3b740 10 client.0 _ll_register_callbacks cb 0x7f56e0007b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33538/0 (socket says 172.21.15.113:33538) 2024-04-25T00:01:12.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 -- 172.21.15.113:0/2395736755 learned_addr learned my addr 172.21.15.113:0/2395736755 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f52c4640 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a2a78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 -- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 msgr2=0x55f1a29f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 -- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 msgr2=0x55f1a2a78580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a2a78580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 -- 172.21.15.113:0/2395736755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a29df870 con 0x55f1a2a01600 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56f62c6640 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29b9e20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f56e80a9f90 tx=0x7f56e802a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f31db52c7c989f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.592+0000 7f56d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2395736755 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8097070 con 0x55f1a2a01600 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2395736755 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f56e807a120 con 0x55f1a2a01600 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e80a8070 con 0x55f1a2a01600 2024-04-25T00:01:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2395736755 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56e807e070 con 0x55f1a2a01600 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 10 client.6138 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f1a29de660 con 0x55f1a2a01600 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2395736755 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e80a8070 con 0x55f1a2a01600 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2395736755 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f56e803a030 con 0x55f1a2a01600 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 10 client.6138 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 10 client.6138 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 20 client.6138 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29f5de0 0x55f1a29ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 4 client.6138 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1a2924f40 con 0x55f1a29f5de0 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29f5de0 0x55f1a29ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 10 client.6138 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56f5ac5640 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29f5de0 0x55f1a29ba360 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee50322d6cdf903d server_cookie=87b63fc87d546035 in_seq=0 out_seq=0 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 1 -- 172.21.15.113:0/2395736755 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+903 (crc 0 0 0) 0x55f1a2924f40 con 0x55f1a29f5de0 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56d0ff9640 10 client.6138 handle_command_reply: tid=0 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.593+0000 7f56faa3b740 1 client.6138 shutdown 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 msgr2=0x55f1a29b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29b9e20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f56e80a9f90 tx=0x7f56e802a6e0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 shutdown_connections 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x55f1a29f5de0 0x55f1a29ba360 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a29e3d60 0x55f1a2a78580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56d0ff9640 0 client.6138 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f1a2a01600 0x55f1a29b9e20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 --2- 172.21.15.113:0/2395736755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f1a29cf160 0x55f1a29f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 >> 172.21.15.113:0/2395736755 conn(0x55f1a29b9a40 msgr2=0x55f1a29fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 shutdown_connections 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 1 -- 172.21.15.113:0/2395736755 wait complete. 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56faa3b740 20 client.6138 trim_cache size 0 max 16384 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stdout: "3910791e-560c-4a32-b96e-d859b6c15273": { 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T00:01:12.601 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "3910791e-560c-4a32-b96e-d859b6c15273", 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "7ab989a0-a0d0-4884-83a6-5bca9588ee53": { 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "7ab989a0-a0d0-4884-83a6-5bca9588ee53", 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "7f93056b-d382-48bd-b672-da086de52ce2": { 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "7f93056b-d382-48bd-b672-da086de52ce2", 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:12.602 INFO:teuthology.orchestra.run.smithi113.stdout: "93e6ad6f-6c5d-469c-b8f8-944e944fbbb5": { 2024-04-25T00:01:12.603 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T00:01:12.603 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "93e6ad6f-6c5d-469c-b8f8-944e944fbbb5", 2024-04-25T00:01:12.603 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "" 2024-04-25T00:01:12.603 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:12.603 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:12.603 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56d406be30 msgr2=0x7f56d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56d406be30 0x7f56d406e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00f360 tx=0x7f56ec007710 comp rx=0 tx=0).stop 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f015e2d0 msgr2=0x7f56f015c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f015e2d0 0x7f56f015c0c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f56e80097a0 tx=0x7f56e80046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 shutdown_connections 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56d406be30 0x7f56d406e2f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f015e2d0 0x7f56f015c0c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f00b4a40 0x7f56f014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 --2- 172.21.15.113:0/605175206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f00b3080 0x7f56f01f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 >> 172.21.15.113:0/605175206 conn(0x7f56f006b0f0 msgr2=0x7f56f00d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 shutdown_connections 2024-04-25T00:01:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:12.594+0000 7f56f7d29640 1 -- 172.21.15.113:0/605175206 wait complete. 2024-04-25T00:01:17.607 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T00:01:17.607 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T00:01:17.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.717+0000 7f353f7d5640 1 Processor -- start 2024-04-25T00:01:17.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.717+0000 7f353f7d5640 1 -- start start 2024-04-25T00:01:17.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353f7d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f353815e2d0 0x7f353815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353f7d5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f35380b3080 0x7f35380b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353f7d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e6b0 0x7f35380b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353f7d5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f35380cde00 con 0x7f353815e2d0 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353f7d5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f353815f4f0 con 0x7f35380b3080 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353f7d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f353815f7d0 con 0x7f353815e6b0 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f353815e2d0 0x7f353815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f353815e2d0 0x7f353815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44140/0 (socket says 172.21.15.113:44140) 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 -- 172.21.15.113:0/4102980599 learned_addr learned my addr 172.21.15.113:0/4102980599 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353dd72640 1 --2- 172.21.15.113:0/4102980599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e6b0 0x7f35380b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 -- 172.21.15.113:0/4102980599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e6b0 msgr2=0x7f35380b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353cd70640 1 --2- 172.21.15.113:0/4102980599 >> v2:172.21.15.178:3300/0 conn(0x7f35380b3080 0x7f35380b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 --2- 172.21.15.113:0/4102980599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e6b0 0x7f35380b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 -- 172.21.15.113:0/4102980599 >> v2:172.21.15.178:3300/0 conn(0x7f35380b3080 msgr2=0x7f35380b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 --2- 172.21.15.113:0/4102980599 >> v2:172.21.15.178:3300/0 conn(0x7f35380b3080 0x7f35380b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.718+0000 7f353d571640 1 -- 172.21.15.113:0/4102980599 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353815fab0 con 0x7f353815e2d0 2024-04-25T00:01:17.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.719+0000 7f353d571640 1 --2- 172.21.15.113:0/4102980599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e2d0 0x7f353815ebf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3538070ff0 tx=0x7f353002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e3b602b1f94c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.719+0000 7f352ffff640 1 -- 172.21.15.113:0/4102980599 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530036070 con 0x7f353815e2d0 2024-04-25T00:01:17.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.719+0000 7f352ffff640 1 -- 172.21.15.113:0/4102980599 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530031600 con 0x7f353815e2d0 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.719+0000 7f352ffff640 1 -- 172.21.15.113:0/4102980599 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35300319d0 con 0x7f353815e2d0 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.719+0000 7f353f7d5640 1 -- 172.21.15.113:0/4102980599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e2d0 msgr2=0x7f353815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.719+0000 7f353f7d5640 1 --2- 172.21.15.113:0/4102980599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e2d0 0x7f353815ebf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3538070ff0 tx=0x7f353002a6a0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.720+0000 7f353f7d5640 1 -- 172.21.15.113:0/4102980599 shutdown_connections 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.720+0000 7f353f7d5640 1 --2- 172.21.15.113:0/4102980599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e6b0 0x7f35380b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.720+0000 7f353f7d5640 1 --2- 172.21.15.113:0/4102980599 >> v2:172.21.15.178:3300/0 conn(0x7f35380b3080 0x7f35380b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.720+0000 7f353f7d5640 1 --2- 172.21.15.113:0/4102980599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e2d0 0x7f353815ebf0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.720+0000 7f353f7d5640 1 -- 172.21.15.113:0/4102980599 >> 172.21.15.113:0/4102980599 conn(0x7f353806b0f0 msgr2=0x7f353806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.720+0000 7f353f7d5640 1 -- 172.21.15.113:0/4102980599 shutdown_connections 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.720+0000 7f353f7d5640 1 -- 172.21.15.113:0/4102980599 wait complete. 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.721+0000 7f353f7d5640 1 Processor -- start 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.721+0000 7f353f7d5640 1 -- start start 2024-04-25T00:01:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353f7d5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35380b3080 0x7f3538145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353f7d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e2d0 0x7f35381e4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353f7d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e6b0 0x7f353815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353f7d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f353815f2b0 con 0x7f353815e6b0 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353f7d5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f353815f130 con 0x7f35380b3080 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353f7d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3538071040 con 0x7f353815e2d0 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e6b0 0x7f353815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e6b0 0x7f353815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44144/0 (socket says 172.21.15.113:44144) 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 -- 172.21.15.113:0/286228632 learned_addr learned my addr 172.21.15.113:0/286228632 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353d571640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35380b3080 0x7f3538145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353cd70640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e2d0 0x7f35381e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 -- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e2d0 msgr2=0x7f35381e4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e2d0 0x7f35381e4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 -- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35380b3080 msgr2=0x7f3538145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35380b3080 0x7f3538145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 -- 172.21.15.113:0/286228632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3538146040 con 0x7f353815e6b0 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353dd72640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e6b0 0x7f353815cc60 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f353400c550 tx=0x7f353400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f352dffb640 1 -- 172.21.15.113:0/286228632 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534015070 con 0x7f353815e6b0 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f352dffb640 1 -- 172.21.15.113:0/286228632 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3534004220 con 0x7f353815e6b0 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35381e5450 con 0x7f353815e6b0 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.722+0000 7f352dffb640 1 -- 172.21.15.113:0/286228632 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534003610 con 0x7f353815e6b0 2024-04-25T00:01:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.723+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35381e5a10 con 0x7f353815e6b0 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.724+0000 7f352dffb640 1 -- 172.21.15.113:0/286228632 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35340037b0 con 0x7f353815e6b0 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.725+0000 7f352dffb640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f351c06be60 0x7f351c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.725+0000 7f352dffb640 1 -- 172.21.15.113:0/286228632 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f353408aac0 con 0x7f353815e6b0 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.725+0000 7f353d571640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f351c06be60 0x7f351c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.725+0000 7f353d571640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f351c06be60 0x7f351c06e320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3530033000 tx=0x7f3530009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.726+0000 7f35425c2740 1 Processor -- start 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.726+0000 7f35425c2740 1 -- start start 2024-04-25T00:01:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f35425c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6076d60 0x5648d6077140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6094b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d604ca40 0x5648d6092cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f35425c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5648d5fb06b0 con 0x5648d6076d60 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648d5f58880 con 0x5648d604ca40 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5648d5fac670 con 0x5648d6094600 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6076d60 0x5648d6077140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6076d60 0x5648d6077140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44150/0 (socket says 172.21.15.113:44150) 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 -- 172.21.15.113:0/2237797243 learned_addr learned my addr 172.21.15.113:0/2237797243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353dd72640 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d604ca40 0x5648d6092cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353cd70640 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6094b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 -- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 msgr2=0x5648d6094b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6094b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 -- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d604ca40 msgr2=0x5648d6092cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d604ca40 0x5648d6092cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 -- 172.21.15.113:0/2237797243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d5d8e7d0 con 0x5648d6076d60 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f353d571640 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6076d60 0x5648d6077140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3530009c90 tx=0x7f353002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b64688b4cbee69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f3522ffd640 1 -- 172.21.15.113:0/2237797243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353004e070 con 0x5648d6076d60 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f3522ffd640 1 -- 172.21.15.113:0/2237797243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530037040 con 0x5648d6076d60 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.727+0000 7f3522ffd640 1 -- 172.21.15.113:0/2237797243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530036040 con 0x5648d6076d60 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 -- 172.21.15.113:0/2237797243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6076d60 msgr2=0x5648d6077140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6076d60 0x5648d6077140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3530009c90 tx=0x7f353002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 -- 172.21.15.113:0/2237797243 shutdown_connections 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d604ca40 0x5648d6092cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6094b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 --2- 172.21.15.113:0/2237797243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6076d60 0x5648d6077140 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 -- 172.21.15.113:0/2237797243 >> 172.21.15.113:0/2237797243 conn(0x5648d6049600 msgr2=0x5648d6091b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 -- 172.21.15.113:0/2237797243 shutdown_connections 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.728+0000 7f35425c2740 1 -- 172.21.15.113:0/2237797243 wait complete. 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.729+0000 7f35425c2740 1 -- start start 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.729+0000 7f35425c2740 1 Processor -- start 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 0x5648d606f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6094600 0x5648d6074350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6074890 0x5648d6079610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5648d604d7e0 con 0x5648d6094600 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648d5f6bf50 con 0x5648d6074890 2024-04-25T00:01:17.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5648d6047530 con 0x5648d6076d60 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 10 client.0 _ll_register_callbacks cb 0x5648d5fa7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6094600 0x5648d6074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6094600 0x5648d6074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44152/0 (socket says 172.21.15.113:44152) 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 -- 172.21.15.113:0/1111114355 learned_addr learned my addr 172.21.15.113:0/1111114355 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 -- 172.21.15.113:0/1111114355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 msgr2=0x5648d606f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 0x5648d606f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 -- 172.21.15.113:0/1111114355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6074890 msgr2=0x5648d6079610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6074890 0x5648d6079610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 -- 172.21.15.113:0/1111114355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d5cc43f0 con 0x5648d6094600 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f353cd70640 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6094600 0x5648d6074350 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f352800aed0 tx=0x7f3528011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef90258602f5f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f3517fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:17.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f3517fff640 1 -- 172.21.15.113:0/1111114355 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528019070 con 0x5648d6094600 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f3517fff640 1 -- 172.21.15.113:0/1111114355 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f3528004220 con 0x5648d6094600 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3530037040 con 0x5648d6094600 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.730+0000 7f3517fff640 1 -- 172.21.15.113:0/1111114355 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3528023050 con 0x5648d6094600 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 10 client.6150 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5648d5c840f0 con 0x5648d6094600 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f3517fff640 1 -- 172.21.15.113:0/1111114355 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530037040 con 0x5648d6094600 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f3517fff640 1 -- 172.21.15.113:0/1111114355 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f3528026020 con 0x5648d6094600 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 10 client.6150 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 10 client.6150 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 20 client.6150 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6073730 0x5648d6073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 4 client.6150 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648d5fbdbf0 con 0x5648d6073730 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f353d571640 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6073730 0x5648d6073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f3517fff640 10 client.6150 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.731+0000 7f353d571640 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6073730 0x5648d6073bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf24b2c7084e20d server_cookie=a080c1ccc8665c70 in_seq=0 out_seq=0 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f3517fff640 1 -- 172.21.15.113:0/1111114355 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648d5fbdbf0 con 0x5648d6073730 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f3517fff640 10 client.6150 handle_command_reply: tid=0 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 client.6150 shutdown 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6094600 msgr2=0x5648d6074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6094600 0x5648d6074350 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f352800aed0 tx=0x7f3528011630 comp rx=0 tx=0).stop 2024-04-25T00:01:17.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 shutdown_connections 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6073730 0x5648d6073bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6074890 0x5648d6079610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6094600 0x5648d6074350 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f3517fff640 0 client.6150 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 --2- 172.21.15.113:0/1111114355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 0x5648d606f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 >> 172.21.15.113:0/1111114355 conn(0x5648d604ca40 msgr2=0x5648d6095980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 shutdown_connections 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 1 -- 172.21.15.113:0/1111114355 wait complete. 2024-04-25T00:01:17.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.732+0000 7f35425c2740 20 client.6150 trim_cache size 0 max 16384 2024-04-25T00:01:17.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 Processor -- start 2024-04-25T00:01:17.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 -- start start 2024-04-25T00:01:17.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d60b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6076d60 0x5648d6047860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6046450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f35382008f0 con 0x5648d6074890 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ff930 con 0x5648d6076d60 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35381fecc0 con 0x5648d6094600 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.750+0000 7f353d571640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d60b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d60b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44168/0 (socket says 172.21.15.113:44168) 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 -- 172.21.15.113:0/1967233048 learned_addr learned my addr 172.21.15.113:0/1967233048 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353cd70640 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6076d60 0x5648d6047860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353dd72640 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6046450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 -- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 msgr2=0x5648d6046450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6046450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 -- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6076d60 msgr2=0x5648d6047860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6076d60 0x5648d6047860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 -- 172.21.15.113:0/1967233048 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d5f36490 con 0x5648d6074890 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353dd72640 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6046450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f353d571640 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d60b5ae0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3530034040 tx=0x7f353003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc692851862c66a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f35227fc640 1 -- 172.21.15.113:0/1967233048 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353004e040 con 0x5648d6074890 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f35227fc640 1 -- 172.21.15.113:0/1967233048 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353003db50 con 0x5648d6074890 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.751+0000 7f35227fc640 1 -- 172.21.15.113:0/1967233048 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530036040 con 0x5648d6074890 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 -- 172.21.15.113:0/1967233048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 msgr2=0x5648d60b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d60b5ae0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3530034040 tx=0x7f353003f040 comp rx=0 tx=0).stop 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 -- 172.21.15.113:0/1967233048 shutdown_connections 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6076d60 0x5648d6047860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6094600 0x5648d6046450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 --2- 172.21.15.113:0/1967233048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d60b5ae0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 -- 172.21.15.113:0/1967233048 >> 172.21.15.113:0/1967233048 conn(0x5648d604ca40 msgr2=0x5648d606f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 -- 172.21.15.113:0/1967233048 shutdown_connections 2024-04-25T00:01:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 -- 172.21.15.113:0/1967233048 wait complete. 2024-04-25T00:01:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 -- start start 2024-04-25T00:01:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.752+0000 7f35425c2740 1 Processor -- start 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f35425c2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d6074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 0x5648d6092920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f35425c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6094600 0x5648d6092e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f35425c2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ff2e0 con 0x5648d6074890 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648d60b6280 con 0x5648d6094600 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f35425c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5648d6091250 con 0x5648d6076d60 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f35425c2740 10 client.0 _ll_register_callbacks cb 0x7f3528006e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353d571640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d6074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353d571640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d6074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44180/0 (socket says 172.21.15.113:44180) 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353d571640 1 -- 172.21.15.113:0/1193027295 learned_addr learned my addr 172.21.15.113:0/1193027295 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353cd70640 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 0x5648d6092920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353dd72640 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6094600 0x5648d6092e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353d571640 1 -- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 msgr2=0x5648d6092920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353d571640 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 0x5648d6092920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353d571640 1 -- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6094600 msgr2=0x5648d6092e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.753+0000 7f353d571640 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6094600 0x5648d6092e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f353d571640 1 -- 172.21.15.113:0/1193027295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d604aed0 con 0x5648d6074890 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f353d571640 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d6074fb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3530009f40 tx=0x7f3530004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1828bf53d5afa4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f3517fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f3517fff640 1 -- 172.21.15.113:0/1193027295 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530067060 con 0x5648d6074890 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f3517fff640 1 -- 172.21.15.113:0/1193027295 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f3530058120 con 0x5648d6074890 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353003db50 con 0x5648d6074890 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f3517fff640 1 -- 172.21.15.113:0/1193027295 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f353006d050 con 0x5648d6074890 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 10 client.6156 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5648d6090f90 con 0x5648d6074890 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f3517fff640 1 -- 172.21.15.113:0/1193027295 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353003db50 con 0x5648d6074890 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f3517fff640 1 -- 172.21.15.113:0/1193027295 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f3530070020 con 0x5648d6074890 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 10 client.6156 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 10 client.6156 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 20 client.6156 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6090ae0 0x5648d604be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 4 client.6156 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648d5fb7f40 con 0x5648d6090ae0 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.754+0000 7f353cd70640 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6090ae0 0x5648d604be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f3517fff640 10 client.6156 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f353cd70640 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6090ae0 0x5648d604be70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9342c0f1bde60e server_cookie=c4aec1e5c7bfd713 in_seq=0 out_seq=0 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f3517fff640 1 -- 172.21.15.113:0/1193027295 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5648d5fb7f40 con 0x5648d6090ae0 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f3517fff640 10 client.6156 handle_command_reply: tid=0 2024-04-25T00:01:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 client.6156 shutdown 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 msgr2=0x5648d6074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d6074fb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3530009f40 tx=0x7f3530004ab0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 shutdown_connections 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5648d6090ae0 0x5648d604be70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648d6094600 0x5648d6092e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5648d6076d60 0x5648d6092920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 --2- 172.21.15.113:0/1193027295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648d6074890 0x5648d6074fb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f3517fff640 0 client.6156 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 >> 172.21.15.113:0/1193027295 conn(0x5648d604ca40 msgr2=0x5648d6047860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 shutdown_connections 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 1 -- 172.21.15.113:0/1193027295 wait complete. 2024-04-25T00:01:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.755+0000 7f35425c2740 20 client.6156 trim_cache size 0 max 16384 2024-04-25T00:01:17.763 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:17.763 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:01:17.763 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:01:17.763 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f351c06be60 msgr2=0x7f351c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f351c06be60 0x7f351c06e320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3530033000 tx=0x7f3530009000 comp rx=0 tx=0).stop 2024-04-25T00:01:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e6b0 msgr2=0x7f353815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e6b0 0x7f353815cc60 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f353400c550 tx=0x7f353400ca10 comp rx=0 tx=0).stop 2024-04-25T00:01:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 shutdown_connections 2024-04-25T00:01:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f351c06be60 0x7f351c06e320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f353815e6b0 0x7f353815cc60 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f353815e2d0 0x7f35381e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 --2- 172.21.15.113:0/286228632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35380b3080 0x7f3538145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 >> 172.21.15.113:0/286228632 conn(0x7f353806b0f0 msgr2=0x7f35380726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 shutdown_connections 2024-04-25T00:01:17.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.756+0000 7f353f7d5640 1 -- 172.21.15.113:0/286228632 wait complete. 2024-04-25T00:01:17.767 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:01:17.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T00:01:17.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.866+0000 7f0ac6990640 1 Processor -- start 2024-04-25T00:01:17.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.866+0000 7f0ac6990640 1 -- start start 2024-04-25T00:01:17.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0ac6990640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac013f050 0x7f0ac00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0ac6990640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ac00d04f0 0x7f0ac015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0ac6990640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0ac01604d0 0x7f0ac006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0ac6990640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac015c6a0 con 0x7f0ac00d04f0 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0ac6990640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac00d0970 con 0x7f0ac01604d0 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0ac6990640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac015d6b0 con 0x7f0ac013f050 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ac00d04f0 0x7f0ac015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ac00d04f0 0x7f0ac015ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44186/0 (socket says 172.21.15.113:44186) 2024-04-25T00:01:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 -- 172.21.15.113:0/1942753959 learned_addr learned my addr 172.21.15.113:0/1942753959 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abffff640 1 --2- 172.21.15.113:0/1942753959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac013f050 0x7f0ac00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/1942753959 >> v2:172.21.15.178:3300/0 conn(0x7f0ac01604d0 0x7f0ac006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 -- 172.21.15.113:0/1942753959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac013f050 msgr2=0x7f0ac00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/1942753959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac013f050 0x7f0ac00cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 -- 172.21.15.113:0/1942753959 >> v2:172.21.15.178:3300/0 conn(0x7f0ac01604d0 msgr2=0x7f0ac006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/1942753959 >> v2:172.21.15.178:3300/0 conn(0x7f0ac01604d0 0x7f0ac006cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.867+0000 7f0abf7fe640 1 -- 172.21.15.113:0/1942753959 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac015c8f0 con 0x7f0ac00d04f0 2024-04-25T00:01:17.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/1942753959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac00d04f0 0x7f0ac015ff90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0ac006f460 tx=0x7f0ab002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f0ec9af8b809dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0abeffd640 1 -- 172.21.15.113:0/1942753959 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0037070 con 0x7f0ac00d04f0 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0abeffd640 1 -- 172.21.15.113:0/1942753959 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab0032440 con 0x7f0ac00d04f0 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0abeffd640 1 -- 172.21.15.113:0/1942753959 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0032810 con 0x7f0ac00d04f0 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 -- 172.21.15.113:0/1942753959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac00d04f0 msgr2=0x7f0ac015ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 --2- 172.21.15.113:0/1942753959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac00d04f0 0x7f0ac015ff90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0ac006f460 tx=0x7f0ab002a140 comp rx=0 tx=0).stop 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 -- 172.21.15.113:0/1942753959 shutdown_connections 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 --2- 172.21.15.113:0/1942753959 >> v2:172.21.15.178:3300/0 conn(0x7f0ac01604d0 0x7f0ac006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 --2- 172.21.15.113:0/1942753959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac00d04f0 0x7f0ac015ff90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 --2- 172.21.15.113:0/1942753959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac013f050 0x7f0ac00cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 -- 172.21.15.113:0/1942753959 >> 172.21.15.113:0/1942753959 conn(0x7f0ac006b0f0 msgr2=0x7f0ac006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.868+0000 7f0ac6990640 1 -- 172.21.15.113:0/1942753959 shutdown_connections 2024-04-25T00:01:17.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.869+0000 7f0ac6990640 1 -- 172.21.15.113:0/1942753959 wait complete. 2024-04-25T00:01:17.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.870+0000 7f0ac6990640 1 Processor -- start 2024-04-25T00:01:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.870+0000 7f0ac6990640 1 -- start start 2024-04-25T00:01:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.870+0000 7f0ac6990640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac00d04f0 0x7f0ac013b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.870+0000 7f0ac6990640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac013f050 0x7f0ac013ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac6990640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac01604d0 0x7f0ac0154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac6990640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac015d190 con 0x7f0ac013f050 2024-04-25T00:01:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac6990640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac015ceb0 con 0x7f0ac00d04f0 2024-04-25T00:01:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac6990640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac013f430 con 0x7f0ac01604d0 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac013f050 0x7f0ac013ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac013f050 0x7f0ac013ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44194/0 (socket says 172.21.15.113:44194) 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 -- 172.21.15.113:0/235542360 learned_addr learned my addr 172.21.15.113:0/235542360 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abffff640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac00d04f0 0x7f0ac013b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac01604d0 0x7f0ac0154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 -- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac01604d0 msgr2=0x7f0ac0154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac01604d0 0x7f0ac0154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 -- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac00d04f0 msgr2=0x7f0ac013b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac00d04f0 0x7f0ac013b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 -- 172.21.15.113:0/235542360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac01597b0 con 0x7f0ac013f050 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac01604d0 0x7f0ac0154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac013f050 0x7f0ac013ac50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0ab002a180 tx=0x7f0ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abcff9640 1 -- 172.21.15.113:0/235542360 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0041020 con 0x7f0ac013f050 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abcff9640 1 -- 172.21.15.113:0/235542360 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab00306c0 con 0x7f0ac013f050 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac0159a90 con 0x7f0ac013f050 2024-04-25T00:01:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0abcff9640 1 -- 172.21.15.113:0/235542360 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0037040 con 0x7f0ac013f050 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.871+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac015a050 con 0x7f0ac013f050 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.873+0000 7f0abcff9640 1 -- 172.21.15.113:0/235542360 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ab0044020 con 0x7f0ac013f050 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.873+0000 7f0abcff9640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0aac06be30 0x7f0aac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.873+0000 7f0abcff9640 1 -- 172.21.15.113:0/235542360 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f0ab00ab710 con 0x7f0ac013f050 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.873+0000 7f0abffff640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0aac06be30 0x7f0aac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.874+0000 7f0abffff640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0aac06be30 0x7f0aac06e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006fd0 tx=0x7f0ab4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.874+0000 7f0ac977a740 1 Processor -- start 2024-04-25T00:01:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.874+0000 7f0ac977a740 1 -- start start 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beac02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beaefdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaf0330 0x5560beac5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0ac977a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5560bea29670 con 0x5560beb0d950 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5560be9d1970 con 0x5560beaf0a70 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5560bea25630 con 0x5560beaf0330 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beac02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beac02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44198/0 (socket says 172.21.15.113:44198) 2024-04-25T00:01:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 -- 172.21.15.113:0/1592612259 learned_addr learned my addr 172.21.15.113:0/1592612259 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beaefdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaf0330 0x5560beac5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 -- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaf0330 msgr2=0x5560beac5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaf0330 0x5560beac5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 -- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 msgr2=0x5560beaefdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beaefdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 -- 172.21.15.113:0/1592612259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560be9f5400 con 0x5560beb0d950 2024-04-25T00:01:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0abffff640 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beac02e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4010bb0 tx=0x7f0ab4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6141671114a729eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.875+0000 7f0aa5ffb640 1 -- 172.21.15.113:0/1592612259 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab401c050 con 0x5560beb0d950 2024-04-25T00:01:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0aa5ffb640 1 -- 172.21.15.113:0/1592612259 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab4026070 con 0x5560beb0d950 2024-04-25T00:01:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0aa5ffb640 1 -- 172.21.15.113:0/1592612259 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4021440 con 0x5560beb0d950 2024-04-25T00:01:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 -- 172.21.15.113:0/1592612259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 msgr2=0x5560beac02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beac02e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4010bb0 tx=0x7f0ab4013000 comp rx=0 tx=0).stop 2024-04-25T00:01:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 -- 172.21.15.113:0/1592612259 shutdown_connections 2024-04-25T00:01:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beaefdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaf0330 0x5560beac5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 --2- 172.21.15.113:0/1592612259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beac02e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 -- 172.21.15.113:0/1592612259 >> 172.21.15.113:0/1592612259 conn(0x5560beac2550 msgr2=0x5560beb0adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 -- 172.21.15.113:0/1592612259 shutdown_connections 2024-04-25T00:01:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.876+0000 7f0ac977a740 1 -- 172.21.15.113:0/1592612259 wait complete. 2024-04-25T00:01:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.877+0000 7f0ac977a740 1 -- start start 2024-04-25T00:01:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.877+0000 7f0ac977a740 1 Processor -- start 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beb1a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beb0d950 0x5560beb03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaebb10 0x5560beb03590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5560beac4700 con 0x5560beaebb10 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5560be9e5050 con 0x5560beaf0a70 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5560beac3af0 con 0x5560beb0d950 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 10 client.0 _ll_register_callbacks cb 0x5560be73cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaebb10 0x5560beb03590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaebb10 0x5560beb03590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44206/0 (socket says 172.21.15.113:44206) 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3922676835 learned_addr learned my addr 172.21.15.113:0/3922676835 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0abffff640 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beb1a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beb0d950 0x5560beb03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beb0d950 msgr2=0x5560beb03050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beb0d950 0x5560beb03050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 msgr2=0x5560beb1a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beb1a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3922676835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560be9b6dc0 con 0x5560beaebb10 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaebb10 0x5560beb03590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8007a70 tx=0x7f0ab8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dff71a74301ac78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0a9effd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0a9effd640 1 -- 172.21.15.113:0/3922676835 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab801b020 con 0x5560beaebb10 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0a9effd640 1 -- 172.21.15.113:0/3922676835 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f0ab8011ae0 con 0x5560beaebb10 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.878+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab4026070 con 0x5560beaebb10 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0a9effd640 1 -- 172.21.15.113:0/3922676835 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0ab8025050 con 0x5560beaebb10 2024-04-25T00:01:17.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 10 client.6168 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:17.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5560be73d3f0 con 0x5560beaebb10 2024-04-25T00:01:17.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0a9effd640 1 -- 172.21.15.113:0/3922676835 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab4026070 con 0x5560beaebb10 2024-04-25T00:01:17.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0a9effd640 1 -- 172.21.15.113:0/3922676835 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f0ab8028020 con 0x5560beaebb10 2024-04-25T00:01:17.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 10 client.6168 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:17.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 10 client.6168 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:17.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 20 client.6168 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:17.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beaf1430 0x5560beaf1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 4 client.6168 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560bea36bb0 con 0x5560beaf1430 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0abffff640 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beaf1430 0x5560beaf1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0abffff640 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beaf1430 0x5560beaf1840 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9fa613ec228e6f server_cookie=85aabaa480de75b4 in_seq=0 out_seq=0 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.879+0000 7f0a9effd640 10 client.6168 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0a9effd640 1 -- 172.21.15.113:0/3922676835 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560bea36bb0 con 0x5560beaf1430 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0a9effd640 10 client.6168 handle_command_reply: tid=0 2024-04-25T00:01:17.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 client.6168 shutdown 2024-04-25T00:01:17.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaebb10 msgr2=0x5560beb03590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaebb10 0x5560beb03590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8007a70 tx=0x7f0ab8007f30 comp rx=0 tx=0).stop 2024-04-25T00:01:17.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 shutdown_connections 2024-04-25T00:01:17.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beaf1430 0x5560beaf1840 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaebb10 0x5560beb03590 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beb0d950 0x5560beb03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0a9effd640 0 client.6168 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3922676835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beb1a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 >> 172.21.15.113:0/3922676835 conn(0x5560beaf0330 msgr2=0x5560beb0e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 shutdown_connections 2024-04-25T00:01:17.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 1 -- 172.21.15.113:0/3922676835 wait complete. 2024-04-25T00:01:17.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.880+0000 7f0ac977a740 20 client.6168 trim_cache size 0 max 16384 2024-04-25T00:01:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.893+0000 7f0ac977a740 1 Processor -- start 2024-04-25T00:01:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.893+0000 7f0ac977a740 1 -- start start 2024-04-25T00:01:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 0x5560beb0a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beb24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beaf2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac977a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01fe770 con 0x5560beb0d950 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac014df20 con 0x5560beaf0a70 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac014d2b0 con 0x5560beaebb10 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beaf2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beaf2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44210/0 (socket says 172.21.15.113:44210) 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3671991593 learned_addr learned my addr 172.21.15.113:0/3671991593 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3671991593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 msgr2=0x5560beb0a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/3671991593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 0x5560beb0a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3671991593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 msgr2=0x5560beb24800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/3671991593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beb24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.894+0000 7f0ac4f2d640 1 -- 172.21.15.113:0/3671991593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560be8077d0 con 0x5560beb0d950 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/3671991593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beaf2630 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009f70 tx=0x7f0ab8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657279bce764564a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0aa57fa640 1 -- 172.21.15.113:0/3671991593 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab802e070 con 0x5560beb0d950 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0aa57fa640 1 -- 172.21.15.113:0/3671991593 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab8016660 con 0x5560beb0d950 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0aa57fa640 1 -- 172.21.15.113:0/3671991593 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab801b070 con 0x5560beb0d950 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 -- 172.21.15.113:0/3671991593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 msgr2=0x5560beaf2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3671991593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beaf2630 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009f70 tx=0x7f0ab8017000 comp rx=0 tx=0).stop 2024-04-25T00:01:17.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 -- 172.21.15.113:0/3671991593 shutdown_connections 2024-04-25T00:01:17.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3671991593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beaf0a70 0x5560beb24800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3671991593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beb0d950 0x5560beaf2630 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 --2- 172.21.15.113:0/3671991593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 0x5560beb0a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 -- 172.21.15.113:0/3671991593 >> 172.21.15.113:0/3671991593 conn(0x5560beaf0330 msgr2=0x5560beaea840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 -- 172.21.15.113:0/3671991593 shutdown_connections 2024-04-25T00:01:17.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.895+0000 7f0ac977a740 1 -- 172.21.15.113:0/3671991593 wait complete. 2024-04-25T00:01:17.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.896+0000 7f0ac977a740 1 -- start start 2024-04-25T00:01:17.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.896+0000 7f0ac977a740 1 Processor -- start 2024-04-25T00:01:17.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 0x5560beb015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaf0a70 0x5560beb838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beb0d950 0x5560beaded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5560beac1a40 con 0x5560beaf0a70 2024-04-25T00:01:17.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac014d8d0 con 0x5560beb0d950 2024-04-25T00:01:17.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5560beb033b0 con 0x5560beaebb10 2024-04-25T00:01:17.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 10 client.0 _ll_register_callbacks cb 0x7f0ab80097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:17.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaf0a70 0x5560beb838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaf0a70 0x5560beb838f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44220/0 (socket says 172.21.15.113:44220) 2024-04-25T00:01:17.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 -- 172.21.15.113:0/2989594639 learned_addr learned my addr 172.21.15.113:0/2989594639 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:17.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abffff640 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 0x5560beb015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac4f2d640 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beb0d950 0x5560beaded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 -- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 msgr2=0x5560beb015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 0x5560beb015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 -- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beb0d950 msgr2=0x5560beaded10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:17.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beb0d950 0x5560beaded10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 -- 172.21.15.113:0/2989594639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560beaf3a70 con 0x5560beaf0a70 2024-04-25T00:01:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0abf7fe640 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaf0a70 0x5560beb838f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00a9b40 tx=0x7f0ab0075cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534735b10310742f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0a9effd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0a9effd640 1 -- 172.21.15.113:0/2989594639 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab007dcb0 con 0x5560beaf0a70 2024-04-25T00:01:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0a9effd640 1 -- 172.21.15.113:0/2989594639 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f0ab0082050 con 0x5560beaf0a70 2024-04-25T00:01:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab8016660 con 0x5560beaf0a70 2024-04-25T00:01:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0a9effd640 1 -- 172.21.15.113:0/2989594639 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0ab00824e0 con 0x5560beaf0a70 2024-04-25T00:01:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 10 client.6174 fetch_fsmap learned FSMap version 23 2024-04-25T00:01:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.897+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5560beabf5b0 con 0x5560beaf0a70 2024-04-25T00:01:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0a9effd640 1 -- 172.21.15.113:0/2989594639 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab8016660 con 0x5560beaf0a70 2024-04-25T00:01:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0a9effd640 1 -- 172.21.15.113:0/2989594639 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f0ab003a030 con 0x5560beaf0a70 2024-04-25T00:01:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 10 client.6174 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:01:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 10 client.6174 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T00:01:17.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 20 client.6174 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:17.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beac2000 0x5560beadf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:17.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 4 client.6174 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:01:17.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560bea30f00 con 0x5560beac2000 2024-04-25T00:01:17.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0abffff640 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beac2000 0x5560beadf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:17.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0a9effd640 10 client.6174 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0abffff640 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beac2000 0x5560beadf250 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d5e9d0a67c9600 server_cookie=9638f2f910ce0c9c in_seq=0 out_seq=0 2024-04-25T00:01:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0a9effd640 1 -- 172.21.15.113:0/2989594639 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+168 (crc 0 0 0) 0x5560bea30f00 con 0x5560beac2000 2024-04-25T00:01:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0a9effd640 10 client.6174 handle_command_reply: tid=0 2024-04-25T00:01:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 client.6174 shutdown 2024-04-25T00:01:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaf0a70 msgr2=0x5560beb838f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaf0a70 0x5560beb838f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00a9b40 tx=0x7f0ab0075cb0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 shutdown_connections 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5560beac2000 0x5560beadf250 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560beb0d950 0x5560beaded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5560beaf0a70 0x5560beb838f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 --2- 172.21.15.113:0/2989594639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5560beaebb10 0x5560beb015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0a9effd640 0 client.6174 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.898+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 >> 172.21.15.113:0/2989594639 conn(0x5560beaf0330 msgr2=0x5560beb0a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 shutdown_connections 2024-04-25T00:01:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac977a740 1 -- 172.21.15.113:0/2989594639 wait complete. 2024-04-25T00:01:17.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac977a740 20 client.6174 trim_cache size 0 max 16384 2024-04-25T00:01:17.924 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:01:17.924 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:17.924 INFO:teuthology.orchestra.run.smithi113.stdout: "damage_type": "backtrace", 2024-04-25T00:01:17.924 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 2640221702, 2024-04-25T00:01:17.925 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 1099511630414, 2024-04-25T00:01:17.925 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T00:01:17.925 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:17.925 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:01:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0aac06be30 msgr2=0x7f0aac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0aac06be30 0x7f0aac06e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006fd0 tx=0x7f0ab4008040 comp rx=0 tx=0).stop 2024-04-25T00:01:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac013f050 msgr2=0x7f0ac013ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac013f050 0x7f0ac013ac50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0ab002a180 tx=0x7f0ab00046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 shutdown_connections 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0aac06be30 0x7f0aac06e2f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ac01604d0 0x7f0ac0154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ac013f050 0x7f0ac013ac50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 --2- 172.21.15.113:0/235542360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ac00d04f0 0x7f0ac013b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 >> 172.21.15.113:0/235542360 conn(0x7f0ac006b0f0 msgr2=0x7f0ac013cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 shutdown_connections 2024-04-25T00:01:17.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:17.899+0000 7f0ac6990640 1 -- 172.21.15.113:0/235542360 wait complete. 2024-04-25T00:01:17.929 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:18.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.025+0000 7fdd86109640 1 Processor -- start 2024-04-25T00:01:18.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.025+0000 7fdd86109640 1 -- start start 2024-04-25T00:01:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.026+0000 7fdd86109640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 0x7fdd800d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.026+0000 7fdd86109640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd800d2800 con 0x7fdd8015e320 2024-04-25T00:01:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.026+0000 7fdd86109640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd800d2ae0 con 0x7fdd800d2290 2024-04-25T00:01:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.026+0000 7fdd86109640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd800d14d0 con 0x7fdd800d0b00 2024-04-25T00:01:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.026+0000 7fdd7f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdd8015e320 0x7fdd800d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34220/0 (socket says 172.21.15.113:34220) 2024-04-25T00:01:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.026+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/1474041799 learned_addr learned my addr 172.21.15.113:0/1474041799 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526926697 0 0) 0x7fdd800d2800 con 0x7fdd8015e320 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd6c003520 con 0x7fdd8015e320 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7ffff640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 0x7fdd800d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4054611435 0 0) 0x7fdd6c003520 con 0x7fdd8015e320 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 msgr2=0x7fdd800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 0x7fdd800d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 >> v1:172.21.15.178:6789/0 conn(0x7fdd800d2290 legacy=0x7fdd800b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd800d2dc0 con 0x7fdd8015e320 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fdd70001690 con 0x7fdd8015e320 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 >> v1:172.21.15.113:6789/0 conn(0x7fdd8015e320 legacy=0x7fdd800d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 0x7fdd6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c004d40 0x7fdd6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800d14d0 con 0x7fdd6c0039e0 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c00bb00 con 0x7fdd6c004d40 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6c00bd00 con 0x7fdd800d0b00 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd76ffd640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 0x7fdd6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7ffff640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c004d40 0x7fdd6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 msgr2=0x7fdd6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd800d0b00 0x7fdd6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c004d40 msgr2=0x7fdd6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c004d40 0x7fdd6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.027+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/1474041799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c003520 con 0x7fdd6c0039e0 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.028+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdd800b39a0 tx=0x7fdd6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0efa40646378010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.028+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70007d50 con 0x7fdd6c0039e0 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.028+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd700058d0 con 0x7fdd6c0039e0 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.028+0000 7fdd7effd640 1 -- 172.21.15.113:0/1474041799 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70005ca0 con 0x7fdd6c0039e0 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.028+0000 7fdd86109640 1 -- 172.21.15.113:0/1474041799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c0039e0 msgr2=0x7fdd6c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.029+0000 7fdd86109640 1 --2- 172.21.15.113:0/1474041799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c0039e0 0x7fdd6c003e00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdd800b39a0 tx=0x7fdd6c005830 comp rx=0 tx=0).stop 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.029+0000 7fdd7ffff640 1 -- 172.21.15.113:0/1474041799 reap_dead start 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.029+0000 7fdd86109640 1 -- 172.21.15.113:0/1474041799 shutdown_connections 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.029+0000 7fdd86109640 1 -- 172.21.15.113:0/1474041799 >> 172.21.15.113:0/1474041799 conn(0x7fdd800b5c20 msgr2=0x7fdd800b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.029+0000 7fdd86109640 1 -- 172.21.15.113:0/1474041799 shutdown_connections 2024-04-25T00:01:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.029+0000 7fdd86109640 1 -- 172.21.15.113:0/1474041799 wait complete. 2024-04-25T00:01:18.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.030+0000 7fdd86109640 1 Processor -- start 2024-04-25T00:01:18.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.030+0000 7fdd86109640 1 -- start start 2024-04-25T00:01:18.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c0039e0 0x7fdd800aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd801f7c70 0x7fdd8014bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd8014c180 0x7fdd800b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8015e700 con 0x7fdd8014c180 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800d3410 con 0x7fdd6c0039e0 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8015e880 con 0x7fdd801f7c70 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd8014c180 0x7fdd800b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd8014c180 0x7fdd800b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44232/0 (socket says 172.21.15.113:44232) 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 -- 172.21.15.113:0/1946239063 learned_addr learned my addr 172.21.15.113:0/1946239063 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c0039e0 0x7fdd800aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd76ffd640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd801f7c70 0x7fdd8014bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 -- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd801f7c70 msgr2=0x7fdd8014bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd801f7c70 0x7fdd8014bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 -- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c0039e0 msgr2=0x7fdd800aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c0039e0 0x7fdd800aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 -- 172.21.15.113:0/1946239063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8014f720 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7ffff640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd8014c180 0x7fdd800b5040 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdd800d51b0 tx=0x7fdd780115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7cff9640 1 -- 172.21.15.113:0/1946239063 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78011e10 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7cff9640 1 -- 172.21.15.113:0/1946239063 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7801b070 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8014eba0 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd7cff9640 1 -- 172.21.15.113:0/1946239063 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78003990 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.031+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8014f160 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.032+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd801f3ca0 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.033+0000 7fdd7cff9640 1 -- 172.21.15.113:0/1946239063 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd78005d10 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.033+0000 7fdd7cff9640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd6006be30 0x7fdd6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.033+0000 7fdd7cff9640 1 -- 172.21.15.113:0/1946239063 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fdd7808b060 con 0x7fdd8014c180 2024-04-25T00:01:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.033+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd6006be30 0x7fdd6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.034+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd6006be30 0x7fdd6006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdd70033000 tx=0x7fdd70035040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.037+0000 7fdd7cff9640 1 -- 172.21.15.113:0/1946239063 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd801f3ca0 con 0x7fdd8014c180 2024-04-25T00:01:18.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.247+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd800691a0 con 0x7fdd8014c180 2024-04-25T00:01:18.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.248+0000 7fdd7cff9640 1 -- 172.21.15.113:0/1946239063 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5190 (secure 0 0 0) 0x7fdd78059410 con 0x7fdd8014c180 2024-04-25T00:01:18.248 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:18.248 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:10.191614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:18.249 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 23 2024-04-25T00:01:18.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd6006be30 msgr2=0x7fdd6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd6006be30 0x7fdd6006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdd70033000 tx=0x7fdd70035040 comp rx=0 tx=0).stop 2024-04-25T00:01:18.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd8014c180 msgr2=0x7fdd800b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd8014c180 0x7fdd800b5040 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdd800d51b0 tx=0x7fdd780115a0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 shutdown_connections 2024-04-25T00:01:18.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd6006be30 0x7fdd6006e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd8014c180 0x7fdd800b5040 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd801f7c70 0x7fdd8014bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 --2- 172.21.15.113:0/1946239063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c0039e0 0x7fdd800aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 >> 172.21.15.113:0/1946239063 conn(0x7fdd800b5c20 msgr2=0x7fdd800b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 shutdown_connections 2024-04-25T00:01:18.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.250+0000 7fdd86109640 1 -- 172.21.15.113:0/1946239063 wait complete. 2024-04-25T00:01:18.261 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2640221702 2024-04-25T00:01:18.266 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:01:18.266+0000 7f15dd321640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T00:01:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.360+0000 7fbb1fa12640 1 Processor -- start 2024-04-25T00:01:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.360+0000 7fbb1fa12640 1 -- start start 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1fa12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e2d0 0x7fbb1815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1fa12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb180b3080 0x7fbb180b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1fa12640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb1815f4f0 con 0x7fbb1815e6b0 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1fa12640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbb180cde00 con 0x7fbb180b3080 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1fa12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1815f7d0 con 0x7fbb1815e2d0 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1cfad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb180b3080 0x7fbb180b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1cfad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb180b3080 0x7fbb180b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44246/0 (socket says 172.21.15.113:44246) 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1dfaf640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbb1815e6b0 0x7fbb18071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34228/0 (socket says 172.21.15.113:34228) 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1cfad640 1 -- 172.21.15.113:0/448933213 learned_addr learned my addr 172.21.15.113:0/448933213 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/448933213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e2d0 0x7fbb1815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1cfad640 1 -- 172.21.15.113:0/448933213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e2d0 msgr2=0x7fbb1815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/448933213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e2d0 0x7fbb1815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1cfad640 1 -- 172.21.15.113:0/448933213 >> v1:172.21.15.113:6789/0 conn(0x7fbb1815e6b0 legacy=0x7fbb18071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1cfad640 1 -- 172.21.15.113:0/448933213 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1815fab0 con 0x7fbb180b3080 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb0ffff640 1 -- 172.21.15.113:0/448933213 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128423217 0 0) 0x7fbb1815f4f0 con 0x7fbb1815e6b0 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.361+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/448933213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e2d0 0x7fbb1815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/448933213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb180b4330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009870 tx=0x7fbb0802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f139a0bab7ebaefb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb0ffff640 1 -- 172.21.15.113:0/448933213 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08037070 con 0x7fbb180b3080 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb0ffff640 1 -- 172.21.15.113:0/448933213 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08032440 con 0x7fbb180b3080 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb0ffff640 1 -- 172.21.15.113:0/448933213 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08032810 con 0x7fbb180b3080 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb1fa12640 1 -- 172.21.15.113:0/448933213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 msgr2=0x7fbb180b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/448933213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb180b4330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009870 tx=0x7fbb0802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb1fa12640 1 -- 172.21.15.113:0/448933213 shutdown_connections 2024-04-25T00:01:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.362+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/448933213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb180b4330 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.363+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/448933213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e2d0 0x7fbb1815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.363+0000 7fbb1fa12640 1 -- 172.21.15.113:0/448933213 >> 172.21.15.113:0/448933213 conn(0x7fbb1806b0f0 msgr2=0x7fbb1806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.363+0000 7fbb1fa12640 1 -- 172.21.15.113:0/448933213 shutdown_connections 2024-04-25T00:01:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.363+0000 7fbb1fa12640 1 -- 172.21.15.113:0/448933213 wait complete. 2024-04-25T00:01:18.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 Processor -- start 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 -- start start 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb18141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb1815e2d0 0x7fbb1814ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e6b0 0x7fbb18153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1815f2b0 con 0x7fbb180b3080 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1815f130 con 0x7fbb1815e2d0 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1fa12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbb180cf2a0 con 0x7fbb1815e6b0 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.364+0000 7fbb1d7ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb18141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb18141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44258/0 (socket says 172.21.15.113:44258) 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/2888164850 learned_addr learned my addr 172.21.15.113:0/2888164850 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb1815e2d0 0x7fbb1814ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e6b0 0x7fbb18153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e6b0 msgr2=0x7fbb18153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e6b0 0x7fbb18153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb1815e2d0 msgr2=0x7fbb1814ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb1815e2d0 0x7fbb1814ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/2888164850 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb181542e0 con 0x7fbb180b3080 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb18141ac0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c7f0 tx=0x7fbb1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb0dffb640 1 -- 172.21.15.113:0/2888164850 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14015070 con 0x7fbb180b3080 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb0dffb640 1 -- 172.21.15.113:0/2888164850 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb14007a30 con 0x7fbb180b3080 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb181545c0 con 0x7fbb180b3080 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb0dffb640 1 -- 172.21.15.113:0/2888164850 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14007e00 con 0x7fbb180b3080 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.365+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1814ee90 con 0x7fbb180b3080 2024-04-25T00:01:18.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.367+0000 7fbb0dffb640 1 -- 172.21.15.113:0/2888164850 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb14018070 con 0x7fbb180b3080 2024-04-25T00:01:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.367+0000 7fbb0dffb640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbafc06be30 0x7fbafc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.367+0000 7fbb0dffb640 1 -- 172.21.15.113:0/2888164850 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fbb1408add0 con 0x7fbb180b3080 2024-04-25T00:01:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.367+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbafc06be30 0x7fbafc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.367+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbafc06be30 0x7fbafc06e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb08034000 tx=0x7fbb0802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.368+0000 7fbb227fa740 1 Processor -- start 2024-04-25T00:01:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.368+0000 7fbb227fa740 1 -- start start 2024-04-25T00:01:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f0edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3ee40 0x5639b2f13c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb227fa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639b2e73230 con 0x5639b2f3ee40 2024-04-25T00:01:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639b2e1b890 con 0x5639b2f3f580 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5639b2e6f1f0 con 0x5639b2f5c460 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3ee40 0x5639b2f13c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3ee40 0x5639b2f13c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44274/0 (socket says 172.21.15.113:44274) 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/3486838392 learned_addr learned my addr 172.21.15.113:0/3486838392 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f0edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 msgr2=0x5639b2f0edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f0edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 msgr2=0x5639b2f3e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f3e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/3486838392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b2e33250 con 0x5639b2f3ee40 2024-04-25T00:01:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3ee40 0x5639b2f13c50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbb10007480 tx=0x7fbb10007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200bb073beacc00e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb02ffd640 1 -- 172.21.15.113:0/3486838392 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10019070 con 0x5639b2f3ee40 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb02ffd640 1 -- 172.21.15.113:0/3486838392 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10001c50 con 0x5639b2f3ee40 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.369+0000 7fbb02ffd640 1 -- 172.21.15.113:0/3486838392 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10002020 con 0x5639b2f3ee40 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 -- 172.21.15.113:0/3486838392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3ee40 msgr2=0x5639b2f13c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3ee40 0x5639b2f13c50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbb10007480 tx=0x7fbb10007940 comp rx=0 tx=0).stop 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 -- 172.21.15.113:0/3486838392 shutdown_connections 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3ee40 0x5639b2f13c50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 --2- 172.21.15.113:0/3486838392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f0edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 -- 172.21.15.113:0/3486838392 >> 172.21.15.113:0/3486838392 conn(0x5639b2f11060 msgr2=0x5639b2f59900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 -- 172.21.15.113:0/3486838392 shutdown_connections 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.370+0000 7fbb227fa740 1 -- 172.21.15.113:0/3486838392 wait complete. 2024-04-25T00:01:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 -- start start 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 Processor -- start 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f52330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f2f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f3b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639b2f13210 con 0x5639b2f42630 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639b2e2ee20 con 0x5639b2f3f580 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.371+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5639b2f12600 con 0x5639b2f5c460 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb227fa740 10 client.0 _ll_register_callbacks cb 0x5639b2e2e810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f3b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f3b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44282/0 (socket says 172.21.15.113:44282) 2024-04-25T00:01:18.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/2180390661 learned_addr learned my addr 172.21.15.113:0/2180390661 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f52330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f2f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 msgr2=0x5639b2f2f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f2f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 msgr2=0x5639b2f52330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f52330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 -- 172.21.15.113:0/2180390661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b2c53710 con 0x5639b2f42630 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f3b480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbb100012e0 tx=0x7fbb10007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52455e8d47a8be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbaf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbaf7fff640 1 -- 172.21.15.113:0/2180390661 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10024020 con 0x5639b2f42630 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbaf7fff640 1 -- 172.21.15.113:0/2180390661 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fbb10020750 con 0x5639b2f42630 2024-04-25T00:01:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb10001c50 con 0x5639b2f42630 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbaf7fff640 1 -- 172.21.15.113:0/2180390661 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbb1002e050 con 0x5639b2f42630 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb227fa740 10 client.6195 fetch_fsmap learned FSMap version 24 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5639b2cb3ea0 con 0x5639b2f42630 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.372+0000 7fbaf7fff640 1 -- 172.21.15.113:0/2180390661 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10001c50 con 0x5639b2f42630 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbaf7fff640 1 -- 172.21.15.113:0/2180390661 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7fbb10031020 con 0x5639b2f42630 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb227fa740 10 client.6195 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb227fa740 10 client.6195 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb227fa740 20 client.6195 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb227fa740 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2f3fbe0 0x5639b2f3ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb227fa740 4 client.6195 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639b2e80770 con 0x5639b2f3fbe0 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2f3fbe0 0x5639b2f3ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbaf7fff640 10 client.6195 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.373+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2f3fbe0 0x5639b2f3ffc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed597021e03f956d server_cookie=6d896489f82b667e in_seq=0 out_seq=0 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbaf7fff640 1 -- 172.21.15.113:0/2180390661 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639b2e80770 con 0x5639b2f3fbe0 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbaf7fff640 10 client.6195 handle_command_reply: tid=0 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 client.6195 shutdown 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 msgr2=0x5639b2f3b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f3b480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbb100012e0 tx=0x7fbb10007ec0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 shutdown_connections 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2f3fbe0 0x5639b2f3ffc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f3b480 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f2f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbaf7fff640 0 client.6195 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 --2- 172.21.15.113:0/2180390661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f3f580 0x5639b2f52330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 >> 172.21.15.113:0/2180390661 conn(0x5639b2f3ee40 msgr2=0x5639b2f0fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 shutdown_connections 2024-04-25T00:01:18.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 1 -- 172.21.15.113:0/2180390661 wait complete. 2024-04-25T00:01:18.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.374+0000 7fbb227fa740 20 client.6195 trim_cache size 0 max 16384 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.387+0000 7fbb227fa740 1 Processor -- start 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.387+0000 7fbb227fa740 1 -- start start 2024-04-25T00:01:18.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f3f580 0x5639b2f3e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f15e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f5c460 0x5639b2f14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb227fa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18150670 con 0x5639b2f42630 2024-04-25T00:01:18.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18157260 con 0x5639b2f5c460 2024-04-25T00:01:18.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbb18150020 con 0x5639b2f3f580 2024-04-25T00:01:18.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f15e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f15e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44290/0 (socket says 172.21.15.113:44290) 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 -- 172.21.15.113:0/1297698447 learned_addr learned my addr 172.21.15.113:0/1297698447 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 -- 172.21.15.113:0/1297698447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f3f580 msgr2=0x5639b2f3e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/1297698447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f3f580 0x5639b2f3e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 -- 172.21.15.113:0/1297698447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f5c460 msgr2=0x5639b2f14bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/1297698447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f5c460 0x5639b2f14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 -- 172.21.15.113:0/1297698447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b2b47000 con 0x5639b2f42630 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/1297698447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f15e10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009630 tx=0x7fbb0802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eac1701ed861449 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.388+0000 7fbb027fc640 1 -- 172.21.15.113:0/1297698447 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08037040 con 0x5639b2f42630 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb027fc640 1 -- 172.21.15.113:0/1297698447 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0804e070 con 0x5639b2f42630 2024-04-25T00:01:18.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb027fc640 1 -- 172.21.15.113:0/1297698447 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08032e10 con 0x5639b2f42630 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 -- 172.21.15.113:0/1297698447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 msgr2=0x5639b2f15e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1297698447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f15e10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009630 tx=0x7fbb0802c000 comp rx=0 tx=0).stop 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 -- 172.21.15.113:0/1297698447 shutdown_connections 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1297698447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f5c460 0x5639b2f14bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1297698447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f42630 0x5639b2f15e10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1297698447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f3f580 0x5639b2f3e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 -- 172.21.15.113:0/1297698447 >> 172.21.15.113:0/1297698447 conn(0x5639b2f3ee40 msgr2=0x5639b2f69440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 -- 172.21.15.113:0/1297698447 shutdown_connections 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.389+0000 7fbb227fa740 1 -- 172.21.15.113:0/1297698447 wait complete. 2024-04-25T00:01:18.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.390+0000 7fbb227fa740 1 -- start start 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.390+0000 7fbb227fa740 1 Processor -- start 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3f580 0x5639b2f68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f42630 0x5639b2f57f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639b2f51800 con 0x5639b2f3f580 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639b2f51340 con 0x5639b2f42630 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5639b2f2ce20 con 0x5639b2f5c460 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 10 client.0 _ll_register_callbacks cb 0x7fbb10001e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3f580 0x5639b2f68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3f580 0x5639b2f68b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44296/0 (socket says 172.21.15.113:44296) 2024-04-25T00:01:18.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/1291445193 learned_addr learned my addr 172.21.15.113:0/1291445193 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1dfaf640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f42630 0x5639b2f57f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 msgr2=0x5639b2f584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f42630 msgr2=0x5639b2f57f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f42630 0x5639b2f57f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 -- 172.21.15.113:0/1291445193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b2f690c0 con 0x5639b2f3f580 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f42630 0x5639b2f57f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb1d7ae640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3f580 0x5639b2f68b80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbb14058dd0 tx=0x7fbb1400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe46883d981ef2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbaf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:18.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbaf7fff640 1 -- 172.21.15.113:0/1291445193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1401f040 con 0x5639b2f3f580 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbaf7fff640 1 -- 172.21.15.113:0/1291445193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fbb1405d120 con 0x5639b2f3f580 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0804e070 con 0x5639b2f3f580 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbaf7fff640 1 -- 172.21.15.113:0/1291445193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbb14055740 con 0x5639b2f3f580 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 10 client.6201 fetch_fsmap learned FSMap version 24 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5639b2f2ea40 con 0x5639b2f3f580 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbaf7fff640 1 -- 172.21.15.113:0/1291445193 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0804e070 con 0x5639b2f3f580 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbaf7fff640 1 -- 172.21.15.113:0/1291445193 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7fbb14058520 con 0x5639b2f3f580 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 10 client.6201 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 10 client.6201 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.391+0000 7fbb227fa740 20 client.6201 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2fd6f30 0x5639b2fd7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 4 client.6201 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2640221702}] 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 --> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2640221702}) -- 0x5639b2e7aac0 con 0x5639b2fd6f30 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2fd6f30 0x5639b2fd7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbaf7fff640 10 client.6201 ms_handle_connect on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb1cfad640 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2fd6f30 0x5639b2fd7310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabb1d817a333cc9 server_cookie=8c385ad5dec81f3c in_seq=0 out_seq=0 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbaf7fff640 1 -- 172.21.15.113:0/1291445193 <== mds.0 v2:172.21.15.113:6834/1715251400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5639b2e7aac0 con 0x5639b2fd6f30 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbaf7fff640 10 client.6201 handle_command_reply: tid=0 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 client.6201 shutdown 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3f580 msgr2=0x5639b2f68b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3f580 0x5639b2f68b80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbb14058dd0 tx=0x7fbb1400cf60 comp rx=0 tx=0).stop 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 shutdown_connections 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:6834/1715251400,v1:172.21.15.113:6836/1715251400] conn(0x5639b2fd6f30 0x5639b2fd7310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639b2f42630 0x5639b2f57f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbaf7fff640 0 client.6201 ms_handle_reset on v2:172.21.15.113:6834/1715251400 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639b2f5c460 0x5639b2f584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 --2- 172.21.15.113:0/1291445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639b2f3f580 0x5639b2f68b80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 >> 172.21.15.113:0/1291445193 conn(0x5639b2f3ee40 msgr2=0x5639b2f0fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 shutdown_connections 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 1 -- 172.21.15.113:0/1291445193 wait complete. 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.392+0000 7fbb227fa740 20 client.6201 trim_cache size 0 max 16384 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbafc06be30 msgr2=0x7fbafc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbafc06be30 0x7fbafc06e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb08034000 tx=0x7fbb0802e000 comp rx=0 tx=0).stop 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 msgr2=0x7fbb18141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb18141ac0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c7f0 tx=0x7fbb1400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 shutdown_connections 2024-04-25T00:01:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbafc06be30 0x7fbafc06e2f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb1815e2d0 0x7fbb1814ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb1815e6b0 0x7fbb18153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 --2- 172.21.15.113:0/2888164850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb180b3080 0x7fbb18141ac0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 >> 172.21.15.113:0/2888164850 conn(0x7fbb1806b0f0 msgr2=0x7fbb180b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 shutdown_connections 2024-04-25T00:01:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.393+0000 7fbb1fa12640 1 -- 172.21.15.113:0/2888164850 wait complete. 2024-04-25T00:01:18.405 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T00:01:18.405 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:18.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.541+0000 7fa3fd163640 1 Processor -- start 2024-04-25T00:01:18.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.541+0000 7fa3fd163640 1 -- start start 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3fd163640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3f806e6e0 0x7fa3f80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3fd163640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f8144cf0 con 0x7fa3f80b3fd0 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3fd163640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f80b2f30 con 0x7fa3f8144910 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3fd163640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f80b3210 con 0x7fa3f806e6e0 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3f759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3f806e6e0 0x7fa3f80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3f659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa3f80b3fd0 0x7fa3f806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34230/0 (socket says 172.21.15.113:34230) 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3f759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3f806e6e0 0x7fa3f80cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44298/0 (socket says 172.21.15.113:44298) 2024-04-25T00:01:18.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.542+0000 7fa3f659c640 1 -- 172.21.15.113:0/981337221 learned_addr learned my addr 172.21.15.113:0/981337221 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:18.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.543+0000 7fa3f759e640 1 -- 172.21.15.113:0/981337221 >> v1:172.21.15.178:6789/0 conn(0x7fa3f8144910 legacy=0x7fa3f80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.543+0000 7fa3f759e640 1 -- 172.21.15.113:0/981337221 >> v1:172.21.15.113:6789/0 conn(0x7fa3f80b3fd0 legacy=0x7fa3f806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.543+0000 7fa3f5d9b640 1 -- 172.21.15.113:0/981337221 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490126295 0 0) 0x7fa3f8144cf0 con 0x7fa3f80b3fd0 2024-04-25T00:01:18.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.543+0000 7fa3f759e640 1 -- 172.21.15.113:0/981337221 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f80b34f0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.543+0000 7fa3f759e640 1 --2- 172.21.15.113:0/981337221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80cd500 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009dc0 tx=0x7fa3e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96722337e5063c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3f5d9b640 1 -- 172.21.15.113:0/981337221 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8007740 con 0x7fa3f806e6e0 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3f5d9b640 1 -- 172.21.15.113:0/981337221 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e80078e0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3f5d9b640 1 -- 172.21.15.113:0/981337221 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8007cb0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3fd163640 1 -- 172.21.15.113:0/981337221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 msgr2=0x7fa3f80cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3fd163640 1 --2- 172.21.15.113:0/981337221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80cd500 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009dc0 tx=0x7fa3e802a730 comp rx=0 tx=0).stop 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3fd163640 1 -- 172.21.15.113:0/981337221 shutdown_connections 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3fd163640 1 --2- 172.21.15.113:0/981337221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80cd500 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3fd163640 1 -- 172.21.15.113:0/981337221 >> 172.21.15.113:0/981337221 conn(0x7fa3f80b1a50 msgr2=0x7fa3f80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3fd163640 1 -- 172.21.15.113:0/981337221 shutdown_connections 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.544+0000 7fa3fd163640 1 -- 172.21.15.113:0/981337221 wait complete. 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 Processor -- start 2024-04-25T00:01:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 -- start start 2024-04-25T00:01:18.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f80b3fd0 0x7fa3f8141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f8144910 0x7fa3f80ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80b4da0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f80b4440 con 0x7fa3f80b3fd0 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3fd163640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f80b4f20 con 0x7fa3f8144910 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3f6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3f6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44314/0 (socket says 172.21.15.113:44314) 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.546+0000 7fa3f6d9d640 1 -- 172.21.15.113:0/3841563836 learned_addr learned my addr 172.21.15.113:0/3841563836 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3f6d9d640 1 -- 172.21.15.113:0/3841563836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f8144910 msgr2=0x7fa3f80ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3f6d9d640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f8144910 0x7fa3f80ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3f6d9d640 1 -- 172.21.15.113:0/3841563836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f80b3fd0 msgr2=0x7fa3f8141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3f6d9d640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f80b3fd0 0x7fa3f8141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3f6d9d640 1 -- 172.21.15.113:0/3841563836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f81f4620 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3f6d9d640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80c4cc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c400 tx=0x7fa3f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3df7fe640 1 -- 172.21.15.113:0/3841563836 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0017070 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3df7fe640 1 -- 172.21.15.113:0/3841563836 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f00052d0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3df7fe640 1 -- 172.21.15.113:0/3841563836 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0003830 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f81f4900 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.547+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f81f4ec0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.548+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f80691a0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.548+0000 7fa3df7fe640 1 -- 172.21.15.113:0/3841563836 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3f001a070 con 0x7fa3f806e6e0 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.549+0000 7fa3df7fe640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa3e406be30 0x7fa3e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.549+0000 7fa3df7fe640 1 -- 172.21.15.113:0/3841563836 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa3f008a290 con 0x7fa3f806e6e0 2024-04-25T00:01:18.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.549+0000 7fa3f659c640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa3e406be30 0x7fa3e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.549+0000 7fa3f659c640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa3e406be30 0x7fa3e406e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec007fd0 tx=0x7fa3ec007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.553+0000 7fa3df7fe640 1 -- 172.21.15.113:0/3841563836 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3f0058c00 con 0x7fa3f806e6e0 2024-04-25T00:01:18.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.761+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f80731a0 con 0x7fa3f806e6e0 2024-04-25T00:01:18.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.761+0000 7fa3df7fe640 1 -- 172.21.15.113:0/3841563836 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5191 (secure 0 0 0) 0x7fa3f0058500 con 0x7fa3f806e6e0 2024-04-25T00:01:18.761 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:18.762 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:18.192070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:18.762 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 24 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa3e406be30 msgr2=0x7fa3e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa3e406be30 0x7fa3e406e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec007fd0 tx=0x7fa3ec007900 comp rx=0 tx=0).stop 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 msgr2=0x7fa3f80c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80c4cc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c400 tx=0x7fa3f000c8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 shutdown_connections 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa3e406be30 0x7fa3e406e2f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f80b3fd0 0x7fa3f8141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f8144910 0x7fa3f80ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 --2- 172.21.15.113:0/3841563836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f806e6e0 0x7fa3f80c4cc0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 >> 172.21.15.113:0/3841563836 conn(0x7fa3f80b1a50 msgr2=0x7fa3f80cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 shutdown_connections 2024-04-25T00:01:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:18.763+0000 7fa3fd163640 1 -- 172.21.15.113:0/3841563836 wait complete. 2024-04-25T00:01:18.775 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T00:01:18.880 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:18.880 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:01:18.880 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:18.888 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T00:01:18.888 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:01:18.888 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:01:18.923 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:01:18.983 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:01:18.983 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:01:18.983 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:18.983 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:18.998 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:18.998 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:19.054 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:19.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.192+0000 7f3c7ab5a640 1 Processor -- start 2024-04-25T00:01:19.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.192+0000 7f3c7ab5a640 1 -- start start 2024-04-25T00:01:19.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c7ab5a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c7406e6e0 0x7f3c740cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c7ab5a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c74144cf0 con 0x7f3c740b3fd0 2024-04-25T00:01:19.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c7ab5a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c740b2f30 con 0x7f3c74144910 2024-04-25T00:01:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c7ab5a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3c740b3210 con 0x7f3c7406e6e0 2024-04-25T00:01:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c790f7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c7406e6e0 0x7f3c740cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c73fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3c740b3fd0 0x7f3c7406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34232/0 (socket says 172.21.15.113:34232) 2024-04-25T00:01:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c73fff640 1 -- 172.21.15.113:0/3666016433 learned_addr learned my addr 172.21.15.113:0/3666016433 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c790f7640 1 -- 172.21.15.113:0/3666016433 >> v1:172.21.15.178:6789/0 conn(0x7f3c74144910 legacy=0x7f3c740b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c790f7640 1 -- 172.21.15.113:0/3666016433 >> v1:172.21.15.113:6789/0 conn(0x7f3c740b3fd0 legacy=0x7f3c7406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c737fe640 1 -- 172.21.15.113:0/3666016433 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522608806 0 0) 0x7f3c74144cf0 con 0x7f3c740b3fd0 2024-04-25T00:01:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.193+0000 7f3c790f7640 1 -- 172.21.15.113:0/3666016433 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c740b34f0 con 0x7f3c7406e6e0 2024-04-25T00:01:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.194+0000 7f3c790f7640 1 --2- 172.21.15.113:0/3666016433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c7406e6e0 0x7f3c740cd500 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009dc0 tx=0x7f3c6c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e0f0f8c255cf98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.194+0000 7f3c737fe640 1 -- 172.21.15.113:0/3666016433 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c007740 con 0x7f3c7406e6e0 2024-04-25T00:01:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.194+0000 7f3c737fe640 1 -- 172.21.15.113:0/3666016433 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0078e0 con 0x7f3c7406e6e0 2024-04-25T00:01:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.194+0000 7f3c737fe640 1 -- 172.21.15.113:0/3666016433 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c007cb0 con 0x7f3c7406e6e0 2024-04-25T00:01:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.195+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/3666016433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c7406e6e0 msgr2=0x7f3c740cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.195+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/3666016433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c7406e6e0 0x7f3c740cd500 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009dc0 tx=0x7f3c6c02a6d0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.195+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/3666016433 shutdown_connections 2024-04-25T00:01:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.195+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/3666016433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c7406e6e0 0x7f3c740cd500 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.195+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/3666016433 >> 172.21.15.113:0/3666016433 conn(0x7f3c740b1a50 msgr2=0x7f3c740b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.195+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/3666016433 shutdown_connections 2024-04-25T00:01:19.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.195+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/3666016433 wait complete. 2024-04-25T00:01:19.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.196+0000 7f3c7ab5a640 1 Processor -- start 2024-04-25T00:01:19.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.196+0000 7f3c7ab5a640 1 -- start start 2024-04-25T00:01:19.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c7406e6e0 0x7f3c7413f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c740b3fd0 0x7f3c740b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c74144910 0x7f3c740b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3c740b4da0 con 0x7f3c74144910 2024-04-25T00:01:19.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c740b4440 con 0x7f3c7406e6e0 2024-04-25T00:01:19.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c740b4f20 con 0x7f3c740b3fd0 2024-04-25T00:01:19.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c74144910 0x7f3c740b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c74144910 0x7f3c740b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44330/0 (socket says 172.21.15.113:44330) 2024-04-25T00:01:19.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 -- 172.21.15.113:0/1702415431 learned_addr learned my addr 172.21.15.113:0/1702415431 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:19.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c788f6640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c7406e6e0 0x7f3c7413f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c73fff640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c740b3fd0 0x7f3c740b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 -- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c740b3fd0 msgr2=0x7f3c740b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c740b3fd0 0x7f3c740b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 -- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c7406e6e0 msgr2=0x7f3c7413f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c7406e6e0 0x7f3c7413f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 -- 172.21.15.113:0/1702415431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7414b190 con 0x7f3c74144910 2024-04-25T00:01:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c73fff640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c740b3fd0 0x7f3c740b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c790f7640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c74144910 0x7f3c740b0980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009ee0 tx=0x7f3c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c717fa640 1 -- 172.21.15.113:0/1702415431 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c040070 con 0x7f3c74144910 2024-04-25T00:01:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c717fa640 1 -- 172.21.15.113:0/1702415431 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c031bc0 con 0x7f3c74144910 2024-04-25T00:01:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7414b470 con 0x7f3c74144910 2024-04-25T00:01:19.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c717fa640 1 -- 172.21.15.113:0/1702415431 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c003990 con 0x7f3c74144910 2024-04-25T00:01:19.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.197+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7414ba30 con 0x7f3c74144910 2024-04-25T00:01:19.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.199+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c740691a0 con 0x7f3c74144910 2024-04-25T00:01:19.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.199+0000 7f3c717fa640 1 -- 172.21.15.113:0/1702415431 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c6c039040 con 0x7f3c74144910 2024-04-25T00:01:19.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.199+0000 7f3c717fa640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c5806be60 0x7f3c5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.199+0000 7f3c717fa640 1 -- 172.21.15.113:0/1702415431 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f3c6c0ac3b0 con 0x7f3c74144910 2024-04-25T00:01:19.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.199+0000 7f3c788f6640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c5806be60 0x7f3c5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.200+0000 7f3c788f6640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c5806be60 0x7f3c5806e320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006fd0 tx=0x7f3c68009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.204+0000 7f3c717fa640 1 -- 172.21.15.113:0/1702415431 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c6c07ade0 con 0x7f3c74144910 2024-04-25T00:01:19.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.378+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c740731a0 con 0x7f3c74144910 2024-04-25T00:01:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.379+0000 7f3c717fa640 1 -- 172.21.15.113:0/1702415431 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16812 (secure 0 0 0) 0x7f3c6c07a6e0 con 0x7f3c74144910 2024-04-25T00:01:19.380 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:19.380 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":48,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:01.211634+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:47.724388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:48.476831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.381+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c5806be60 msgr2=0x7f3c5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.381+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c5806be60 0x7f3c5806e320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006fd0 tx=0x7f3c68009060 comp rx=0 tx=0).stop 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.381+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c74144910 msgr2=0x7f3c740b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.381+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c74144910 0x7f3c740b0980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009ee0 tx=0x7f3c6c004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.381+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 shutdown_connections 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.382+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c5806be60 0x7f3c5806e320 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.382+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c74144910 0x7f3c740b0980 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.382+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c740b3fd0 0x7f3c740b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.382+0000 7f3c7ab5a640 1 --2- 172.21.15.113:0/1702415431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c7406e6e0 0x7f3c7413f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.382+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 >> 172.21.15.113:0/1702415431 conn(0x7f3c740b1a50 msgr2=0x7f3c740b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.382+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 shutdown_connections 2024-04-25T00:01:19.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.382+0000 7f3c7ab5a640 1 -- 172.21.15.113:0/1702415431 wait complete. 2024-04-25T00:01:19.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:47.724388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:48.476831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:19.393 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:19.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.492+0000 7f313af3b640 1 Processor -- start 2024-04-25T00:01:19.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.492+0000 7f313af3b640 1 -- start start 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f313af3b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f313415e2d0 0x7f313415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f313af3b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313415f4f0 con 0x7f31340b4a40 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f313af3b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313415f7d0 con 0x7f31340b3080 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f313af3b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f31340cde00 con 0x7f313415e2d0 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f3138cd7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f313415e2d0 0x7f313415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f31394d8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f31340b4a40 0x7f31340ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34248/0 (socket says 172.21.15.113:34248) 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f31394d8640 1 -- 172.21.15.113:0/1884647711 learned_addr learned my addr 172.21.15.113:0/1884647711 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f3138cd7640 1 -- 172.21.15.113:0/1884647711 >> v1:172.21.15.178:6789/0 conn(0x7f31340b3080 legacy=0x7f31340b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f3138cd7640 1 -- 172.21.15.113:0/1884647711 >> v1:172.21.15.113:6789/0 conn(0x7f31340b4a40 legacy=0x7f31340ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f3138cd7640 1 -- 172.21.15.113:0/1884647711 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313415fab0 con 0x7f313415e2d0 2024-04-25T00:01:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.493+0000 7f31337fe640 1 -- 172.21.15.113:0/1884647711 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868504427 0 0) 0x7f313415f4f0 con 0x7f31340b4a40 2024-04-25T00:01:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.494+0000 7f3138cd7640 1 --2- 172.21.15.113:0/1884647711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313415e2d0 0x7f313415ebf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f31340cdf80 tx=0x7f312802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7962c08e9f6afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.494+0000 7f31337fe640 1 -- 172.21.15.113:0/1884647711 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128037070 con 0x7f313415e2d0 2024-04-25T00:01:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.494+0000 7f31337fe640 1 -- 172.21.15.113:0/1884647711 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3128032440 con 0x7f313415e2d0 2024-04-25T00:01:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.494+0000 7f31337fe640 1 -- 172.21.15.113:0/1884647711 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128032810 con 0x7f313415e2d0 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.494+0000 7f313af3b640 1 -- 172.21.15.113:0/1884647711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313415e2d0 msgr2=0x7f313415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.494+0000 7f313af3b640 1 --2- 172.21.15.113:0/1884647711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313415e2d0 0x7f313415ebf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f31340cdf80 tx=0x7f312802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.495+0000 7f313af3b640 1 -- 172.21.15.113:0/1884647711 shutdown_connections 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.495+0000 7f313af3b640 1 --2- 172.21.15.113:0/1884647711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313415e2d0 0x7f313415ebf0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.495+0000 7f313af3b640 1 -- 172.21.15.113:0/1884647711 >> 172.21.15.113:0/1884647711 conn(0x7f313406b0f0 msgr2=0x7f313406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.495+0000 7f313af3b640 1 -- 172.21.15.113:0/1884647711 shutdown_connections 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.495+0000 7f313af3b640 1 -- 172.21.15.113:0/1884647711 wait complete. 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.496+0000 7f313af3b640 1 Processor -- start 2024-04-25T00:01:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.496+0000 7f313af3b640 1 -- start start 2024-04-25T00:01:19.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31340b3080 0x7f31341414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31340b4a40 0x7f313413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f313415e2d0 0x7f31341400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f313415f2b0 con 0x7f31340b3080 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f313415f130 con 0x7f31340b4a40 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f313415e6b0 con 0x7f313415e2d0 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31340b3080 0x7f31341414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31340b3080 0x7f31341414e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44348/0 (socket says 172.21.15.113:44348) 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 -- 172.21.15.113:0/153368310 learned_addr learned my addr 172.21.15.113:0/153368310 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f31394d8640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f313415e2d0 0x7f31341400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 -- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f313415e2d0 msgr2=0x7f31341400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3133fff640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31340b4a40 0x7f313413e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f313415e2d0 0x7f31341400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 -- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31340b4a40 msgr2=0x7f313413e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31340b4a40 0x7f313413e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 -- 172.21.15.113:0/153368310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3134141a20 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f31394d8640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f313415e2d0 0x7f31341400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3133fff640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31340b4a40 0x7f313413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f3138cd7640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31340b3080 0x7f31341414e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3128035040 tx=0x7f31280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f31317fa640 1 -- 172.21.15.113:0/153368310 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128037040 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f31317fa640 1 -- 172.21.15.113:0/153368310 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3128004230 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313413f000 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f31317fa640 1 -- 172.21.15.113:0/153368310 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128032e20 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.497+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313413f8a0 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.498+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31340691a0 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.499+0000 7f31317fa640 1 -- 172.21.15.113:0/153368310 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3128030770 con 0x7f31340b3080 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.499+0000 7f31317fa640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f311806be30 0x7f311806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.499+0000 7f31317fa640 1 -- 172.21.15.113:0/153368310 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f31280ac2b0 con 0x7f31340b3080 2024-04-25T00:01:19.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.499+0000 7f3133fff640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f311806be30 0x7f311806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.500+0000 7f3133fff640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f311806be30 0x7f311806e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3120007fd0 tx=0x7f3120009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.503+0000 7f31317fa640 1 -- 172.21.15.113:0/153368310 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3134141a20 con 0x7f31340b3080 2024-04-25T00:01:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.674+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31340b2b00 con 0x7f31340b3080 2024-04-25T00:01:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.674+0000 7f31317fa640 1 -- 172.21.15.113:0/153368310 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f312807ad40 con 0x7f31340b3080 2024-04-25T00:01:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:01:19.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f311806be30 msgr2=0x7f311806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f311806be30 0x7f311806e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3120007fd0 tx=0x7f3120009040 comp rx=0 tx=0).stop 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31340b3080 msgr2=0x7f31341414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31340b3080 0x7f31341414e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3128035040 tx=0x7f31280046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 shutdown_connections 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f311806be30 0x7f311806e2f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31340b4a40 0x7f313413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f313415e2d0 0x7f31341400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 --2- 172.21.15.113:0/153368310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31340b3080 0x7f31341414e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 >> 172.21.15.113:0/153368310 conn(0x7f313406b0f0 msgr2=0x7f31340b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 shutdown_connections 2024-04-25T00:01:19.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.676+0000 7f313af3b640 1 -- 172.21.15.113:0/153368310 wait complete. 2024-04-25T00:01:19.687 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:01:19.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.788+0000 7f82f85f4640 1 Processor -- start 2024-04-25T00:01:19.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.788+0000 7f82f85f4640 1 -- start start 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f85f4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f015e2d0 0x7f82f015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f85f4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82f00b3080 0x7f82f00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f85f4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f015f4f0 con 0x7f82f015e6b0 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f85f4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f82f00cde00 con 0x7f82f00b3080 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f85f4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82f015f7d0 con 0x7f82f015e2d0 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f5b8f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82f00b3080 0x7f82f00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f6390640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f015e2d0 0x7f82f015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f5b8f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82f00b3080 0x7f82f00b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:52986/0 (socket says 172.21.15.113:52986) 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f5b8f640 1 -- 172.21.15.113:0/594955614 learned_addr learned my addr 172.21.15.113:0/594955614 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:19.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f6390640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f015e2d0 0x7f82f015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:55650/0 (socket says 172.21.15.113:55650) 2024-04-25T00:01:19.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f6b91640 1 --1- 172.21.15.113:0/594955614 >> v1:172.21.15.178:6789/0 conn(0x7f82f015e6b0 0x7f82f0071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:50456/0 (socket says 172.21.15.113:50456) 2024-04-25T00:01:19.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f5b8f640 1 -- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f015e2d0 msgr2=0x7f82f015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:19.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f5b8f640 1 --2- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f015e2d0 0x7f82f015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f5b8f640 1 -- 172.21.15.113:0/594955614 >> v1:172.21.15.178:6789/0 conn(0x7f82f015e6b0 legacy=0x7f82f0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f538e640 1 -- 172.21.15.113:0/594955614 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186699440 0 0) 0x7f82f015f4f0 con 0x7f82f015e6b0 2024-04-25T00:01:19.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f5b8f640 1 -- 172.21.15.113:0/594955614 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f015fab0 con 0x7f82f00b3080 2024-04-25T00:01:19.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.789+0000 7f82f6390640 1 --2- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f015e2d0 0x7f82f015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.790+0000 7f82f5b8f640 1 --2- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f00b3080 0x7f82f00b4330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009870 tx=0x7f82e002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e241bf370bde251e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.790+0000 7f82f538e640 1 -- 172.21.15.113:0/594955614 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0037070 con 0x7f82f00b3080 2024-04-25T00:01:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.790+0000 7f82f538e640 1 -- 172.21.15.113:0/594955614 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e0032440 con 0x7f82f00b3080 2024-04-25T00:01:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.790+0000 7f82f538e640 1 -- 172.21.15.113:0/594955614 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0032810 con 0x7f82f00b3080 2024-04-25T00:01:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 -- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f00b3080 msgr2=0x7f82f00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 --2- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f00b3080 0x7f82f00b4330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009870 tx=0x7f82e002a1c0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 -- 172.21.15.113:0/594955614 shutdown_connections 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 --2- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f00b3080 0x7f82f00b4330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 --2- 172.21.15.113:0/594955614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f015e2d0 0x7f82f015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 -- 172.21.15.113:0/594955614 >> 172.21.15.113:0/594955614 conn(0x7f82f006b0f0 msgr2=0x7f82f006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 -- 172.21.15.113:0/594955614 shutdown_connections 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.791+0000 7f82f85f4640 1 -- 172.21.15.113:0/594955614 wait complete. 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.792+0000 7f82f85f4640 1 Processor -- start 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.792+0000 7f82f85f4640 1 -- start start 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f85f4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f00b3080 0x7f82f01470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f85f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f015e2d0 0x7f82f01424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f85f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f015e6b0 0x7f82f014bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f85f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f82f015f2b0 con 0x7f82f015e6b0 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f85f4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82f015f130 con 0x7f82f015e2d0 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f85f4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82f00cf2a0 con 0x7f82f00b3080 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f015e6b0 0x7f82f014bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f015e6b0 0x7f82f014bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44354/0 (socket says 172.21.15.113:44354) 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 -- 172.21.15.113:0/3452651028 learned_addr learned my addr 172.21.15.113:0/3452651028 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f5b8f640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f015e2d0 0x7f82f01424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6390640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f00b3080 0x7f82f01470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 -- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f00b3080 msgr2=0x7f82f01470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f00b3080 0x7f82f01470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 -- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f015e2d0 msgr2=0x7f82f01424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f015e2d0 0x7f82f01424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 -- 172.21.15.113:0/3452651028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f014c550 con 0x7f82f015e6b0 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f5b8f640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f015e2d0 0x7f82f01424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82f6b91640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f015e6b0 0x7f82f014bc40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c730 tx=0x7f82e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82e6ffd640 1 -- 172.21.15.113:0/3452651028 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8018070 con 0x7f82f015e6b0 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.793+0000 7f82e6ffd640 1 -- 172.21.15.113:0/3452651028 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e800ce40 con 0x7f82f015e6b0 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.794+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f014c830 con 0x7f82f015e6b0 2024-04-25T00:01:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.794+0000 7f82e6ffd640 1 -- 172.21.15.113:0/3452651028 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e800a860 con 0x7f82f015e6b0 2024-04-25T00:01:19.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.794+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f014cdf0 con 0x7f82f015e6b0 2024-04-25T00:01:19.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.795+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f00691a0 con 0x7f82f015e6b0 2024-04-25T00:01:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.795+0000 7f82e6ffd640 1 -- 172.21.15.113:0/3452651028 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82e801b070 con 0x7f82f015e6b0 2024-04-25T00:01:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.795+0000 7f82e6ffd640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f82d406be30 0x7f82d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.795+0000 7f82e6ffd640 1 -- 172.21.15.113:0/3452651028 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f82e808a420 con 0x7f82f015e6b0 2024-04-25T00:01:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.795+0000 7f82f6390640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f82d406be30 0x7f82d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.796+0000 7f82f6390640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f82d406be30 0x7f82d406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009560 tx=0x7f82ec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.799+0000 7f82e6ffd640 1 -- 172.21.15.113:0/3452651028 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82e8058e20 con 0x7f82f015e6b0 2024-04-25T00:01:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.970+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82f00afce0 con 0x7f82f015e6b0 2024-04-25T00:01:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.971+0000 7f82e6ffd640 1 -- 172.21.15.113:0/3452651028 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7f82e8058720 con 0x7f82f015e6b0 2024-04-25T00:01:19.971 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:01:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f82d406be30 msgr2=0x7f82d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f82d406be30 0x7f82d406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009560 tx=0x7f82ec008040 comp rx=0 tx=0).stop 2024-04-25T00:01:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f015e6b0 msgr2=0x7f82f014bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f015e6b0 0x7f82f014bc40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c730 tx=0x7f82e800cbf0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 shutdown_connections 2024-04-25T00:01:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f82d406be30 0x7f82d406e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f015e2d0 0x7f82f01424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f015e6b0 0x7f82f014bc40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 --2- 172.21.15.113:0/3452651028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f00b3080 0x7f82f01470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 >> 172.21.15.113:0/3452651028 conn(0x7f82f006b0f0 msgr2=0x7f82f00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 shutdown_connections 2024-04-25T00:01:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:19.973+0000 7f82f85f4640 1 -- 172.21.15.113:0/3452651028 wait complete. 2024-04-25T00:01:19.984 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:01:20.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.083+0000 7fe2816c3640 1 Processor -- start 2024-04-25T00:01:20.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.083+0000 7fe2816c3640 1 -- start start 2024-04-25T00:01:20.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe2816c3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe27c15e510 0x7fe27c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe2816c3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe27c0d2e00 con 0x7fe27c0b63b0 2024-04-25T00:01:20.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe2816c3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe27c0d30e0 con 0x7fe27c0d28c0 2024-04-25T00:01:20.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe2816c3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe27c0d33c0 con 0x7fe27c15e510 2024-04-25T00:01:20.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe27affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe27c15e510 0x7fe27c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe27b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe27c0b63b0 0x7fe27c0b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34250/0 (socket says 172.21.15.113:34250) 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe27b7fe640 1 -- 172.21.15.113:0/1533419171 learned_addr learned my addr 172.21.15.113:0/1533419171 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe27affd640 1 -- 172.21.15.113:0/1533419171 >> v1:172.21.15.178:6789/0 conn(0x7fe27c0d28c0 legacy=0x7fe27c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe27affd640 1 -- 172.21.15.113:0/1533419171 >> v1:172.21.15.113:6789/0 conn(0x7fe27c0b63b0 legacy=0x7fe27c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe27affd640 1 -- 172.21.15.113:0/1533419171 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c0d36a0 con 0x7fe27c15e510 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.084+0000 7fe279ffb640 1 -- 172.21.15.113:0/1533419171 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 631826683 0 0) 0x7fe27c0d2e00 con 0x7fe27c0b63b0 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.085+0000 7fe27affd640 1 --2- 172.21.15.113:0/1533419171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c15e510 0x7fe27c0d2380 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0097a0 tx=0x7fe26c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f1d0141562c721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.085+0000 7fe279ffb640 1 -- 172.21.15.113:0/1533419171 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c037070 con 0x7fe27c15e510 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.085+0000 7fe279ffb640 1 -- 172.21.15.113:0/1533419171 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26c032440 con 0x7fe27c15e510 2024-04-25T00:01:20.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.085+0000 7fe279ffb640 1 -- 172.21.15.113:0/1533419171 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c032810 con 0x7fe27c15e510 2024-04-25T00:01:20.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.085+0000 7fe2816c3640 1 -- 172.21.15.113:0/1533419171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c15e510 msgr2=0x7fe27c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.086+0000 7fe2816c3640 1 --2- 172.21.15.113:0/1533419171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c15e510 0x7fe27c0d2380 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0097a0 tx=0x7fe26c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.086+0000 7fe2816c3640 1 -- 172.21.15.113:0/1533419171 shutdown_connections 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.086+0000 7fe2816c3640 1 --2- 172.21.15.113:0/1533419171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c15e510 0x7fe27c0d2380 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.086+0000 7fe2816c3640 1 -- 172.21.15.113:0/1533419171 >> 172.21.15.113:0/1533419171 conn(0x7fe27c06b720 msgr2=0x7fe27c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.086+0000 7fe2816c3640 1 -- 172.21.15.113:0/1533419171 shutdown_connections 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.086+0000 7fe2816c3640 1 -- 172.21.15.113:0/1533419171 wait complete. 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.087+0000 7fe2816c3640 1 Processor -- start 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.087+0000 7fe2816c3640 1 -- start start 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0b63b0 0x7fe27c153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe27c0d28c0 0x7fe27c0cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe27c15e510 0x7fe27c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c0b9dd0 con 0x7fe27c0b63b0 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c0d3ea0 con 0x7fe27c15e510 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe27c0b7ad0 con 0x7fe27c0d28c0 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0b63b0 0x7fe27c153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0b63b0 0x7fe27c153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44378/0 (socket says 172.21.15.113:44378) 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 -- 172.21.15.113:0/2544813739 learned_addr learned my addr 172.21.15.113:0/2544813739 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27b7fe640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe27c15e510 0x7fe27c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27a7fc640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe27c0d28c0 0x7fe27c0cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 -- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe27c0d28c0 msgr2=0x7fe27c0cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe27c0d28c0 0x7fe27c0cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 -- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe27c15e510 msgr2=0x7fe27c15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe27c15e510 0x7fe27c15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 -- 172.21.15.113:0/2544813739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c1542b0 con 0x7fe27c0b63b0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe27affd640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0b63b0 0x7fe27c153d70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2637fe640 1 -- 172.21.15.113:0/2544813739 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c037040 con 0x7fe27c0b63b0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2637fe640 1 -- 172.21.15.113:0/2544813739 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26c042070 con 0x7fe27c0b63b0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c0ce3e0 con 0x7fe27c0b63b0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2637fe640 1 -- 172.21.15.113:0/2544813739 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c030a70 con 0x7fe27c0b63b0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.088+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c0c9220 con 0x7fe27c0b63b0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.089+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe27c0691a0 con 0x7fe27c0b63b0 2024-04-25T00:01:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.090+0000 7fe2637fe640 1 -- 172.21.15.113:0/2544813739 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe26c030c10 con 0x7fe27c0b63b0 2024-04-25T00:01:20.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.090+0000 7fe2637fe640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe26806be30 0x7fe26806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.090+0000 7fe2637fe640 1 -- 172.21.15.113:0/2544813739 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe26c0aba80 con 0x7fe27c0b63b0 2024-04-25T00:01:20.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.090+0000 7fe27a7fc640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe26806be30 0x7fe26806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.090+0000 7fe27a7fc640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe26806be30 0x7fe26806e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0b84c0 tx=0x7fe270009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:20.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.094+0000 7fe2637fe640 1 -- 172.21.15.113:0/2544813739 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe26c07a360 con 0x7fe27c0b63b0 2024-04-25T00:01:20.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.269+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe27c0b93d0 con 0x7fe27c0b63b0 2024-04-25T00:01:20.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.270+0000 7fe2637fe640 1 -- 172.21.15.113:0/2544813739 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7fe26c079c60 con 0x7fe27c0b63b0 2024-04-25T00:01:20.271 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:01:20.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe26806be30 msgr2=0x7fe26806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe26806be30 0x7fe26806e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0b84c0 tx=0x7fe270009040 comp rx=0 tx=0).stop 2024-04-25T00:01:20.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0b63b0 msgr2=0x7fe27c153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0b63b0 0x7fe27c153d70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 shutdown_connections 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe26806be30 0x7fe26806e2f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe27c15e510 0x7fe27c15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe27c0d28c0 0x7fe27c0cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 --2- 172.21.15.113:0/2544813739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0b63b0 0x7fe27c153d70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 >> 172.21.15.113:0/2544813739 conn(0x7fe27c06b720 msgr2=0x7fe27c15c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 shutdown_connections 2024-04-25T00:01:20.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.272+0000 7fe2816c3640 1 -- 172.21.15.113:0/2544813739 wait complete. 2024-04-25T00:01:20.285 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:01:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.380+0000 7faebf5e3640 1 Processor -- start 2024-04-25T00:01:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.380+0000 7faebf5e3640 1 -- start start 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebf5e3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faeb815e510 0x7faeb80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebf5e3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faeb80b63b0 0x7faeb80b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebf5e3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeb80d2e00 con 0x7faeb80d28c0 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebf5e3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7faeb80d30e0 con 0x7faeb815e510 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebf5e3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faeb80d33c0 con 0x7faeb80b63b0 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebd37f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faeb815e510 0x7faeb80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebd37f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faeb815e510 0x7faeb80d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44388/0 (socket says 172.21.15.113:44388) 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.381+0000 7faebd37f640 1 -- 172.21.15.113:0/1874025085 learned_addr learned my addr 172.21.15.113:0/1874025085 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.382+0000 7faebd37f640 1 -- 172.21.15.113:0/1874025085 >> v2:172.21.15.178:3300/0 conn(0x7faeb80b63b0 msgr2=0x7faeb80b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.382+0000 7faebdb80640 1 --2- 172.21.15.113:0/1874025085 >> v2:172.21.15.178:3300/0 conn(0x7faeb80b63b0 0x7faeb80b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.382+0000 7faebd37f640 1 --2- 172.21.15.113:0/1874025085 >> v2:172.21.15.178:3300/0 conn(0x7faeb80b63b0 0x7faeb80b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.382+0000 7faebd37f640 1 -- 172.21.15.113:0/1874025085 >> v1:172.21.15.178:6789/0 conn(0x7faeb80d28c0 legacy=0x7faeb80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:20.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.382+0000 7faebd37f640 1 -- 172.21.15.113:0/1874025085 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb80d36a0 con 0x7faeb815e510 2024-04-25T00:01:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.382+0000 7faebd37f640 1 --2- 172.21.15.113:0/1874025085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80d2380 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faeb80b7190 tx=0x7faeb402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffc057475a5fcb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faeaffff640 1 -- 172.21.15.113:0/1874025085 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4036070 con 0x7faeb815e510 2024-04-25T00:01:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faeaffff640 1 -- 172.21.15.113:0/1874025085 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb4031600 con 0x7faeb815e510 2024-04-25T00:01:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faeaffff640 1 -- 172.21.15.113:0/1874025085 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb40319d0 con 0x7faeb815e510 2024-04-25T00:01:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 -- 172.21.15.113:0/1874025085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 msgr2=0x7faeb80d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1874025085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80d2380 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faeb80b7190 tx=0x7faeb402a660 comp rx=0 tx=0).stop 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 -- 172.21.15.113:0/1874025085 shutdown_connections 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1874025085 >> v2:172.21.15.178:3300/0 conn(0x7faeb80b63b0 0x7faeb80b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1874025085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80d2380 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 -- 172.21.15.113:0/1874025085 >> 172.21.15.113:0/1874025085 conn(0x7faeb806b720 msgr2=0x7faeb80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 -- 172.21.15.113:0/1874025085 shutdown_connections 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.383+0000 7faebf5e3640 1 -- 172.21.15.113:0/1874025085 wait complete. 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.384+0000 7faebf5e3640 1 Processor -- start 2024-04-25T00:01:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.384+0000 7faebf5e3640 1 -- start start 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebf5e3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faeb80b63b0 0x7faeb80ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebf5e3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faeb80d28c0 0x7faeb81496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebf5e3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebf5e3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faeb80b9dd0 con 0x7faeb815e510 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebf5e3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faeb80d3ea0 con 0x7faeb80d28c0 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebf5e3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faeb80b7900 con 0x7faeb80b63b0 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebdb80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebdb80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80b7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44394/0 (socket says 172.21.15.113:44394) 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebdb80640 1 -- 172.21.15.113:0/1854915363 learned_addr learned my addr 172.21.15.113:0/1854915363 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebd37f640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faeb80b63b0 0x7faeb80ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.385+0000 7faebcb7e640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faeb80d28c0 0x7faeb81496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebdb80640 1 -- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faeb80b63b0 msgr2=0x7faeb80ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebdb80640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faeb80b63b0 0x7faeb80ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebdb80640 1 -- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faeb80d28c0 msgr2=0x7faeb81496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebdb80640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faeb80d28c0 0x7faeb81496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebdb80640 1 -- 172.21.15.113:0/1854915363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb80cda20 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebcb7e640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faeb80d28c0 0x7faeb81496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebdb80640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80b7f50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c7c0 tx=0x7faeb000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faeadffb640 1 -- 172.21.15.113:0/1854915363 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0015070 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faeadffb640 1 -- 172.21.15.113:0/1854915363 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb0004220 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb80cdd00 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faeadffb640 1 -- 172.21.15.113:0/1854915363 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0007620 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.386+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb80ce2c0 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.387+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb80691a0 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.387+0000 7faeadffb640 1 -- 172.21.15.113:0/1854915363 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faeb00077c0 con 0x7faeb815e510 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.388+0000 7faeadffb640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae9c06be60 0x7fae9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.388+0000 7faeadffb640 1 -- 172.21.15.113:0/1854915363 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7faeb008acd0 con 0x7faeb815e510 2024-04-25T00:01:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.388+0000 7faebd37f640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae9c06be60 0x7fae9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.388+0000 7faebd37f640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae9c06be60 0x7fae9c06e320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faeb4033000 tx=0x7faeb4009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.392+0000 7faeadffb640 1 -- 172.21.15.113:0/1854915363 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faeb0059700 con 0x7faeb815e510 2024-04-25T00:01:20.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.577+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faeb80c76c0 con 0x7faeb815e510 2024-04-25T00:01:20.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.578+0000 7faeadffb640 1 -- 172.21.15.113:0/1854915363 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7faeb0059000 con 0x7faeb815e510 2024-04-25T00:01:20.579 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:01:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae9c06be60 msgr2=0x7fae9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae9c06be60 0x7fae9c06e320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faeb4033000 tx=0x7faeb4009000 comp rx=0 tx=0).stop 2024-04-25T00:01:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 msgr2=0x7faeb80b7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80b7f50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c7c0 tx=0x7faeb000cc80 comp rx=0 tx=0).stop 2024-04-25T00:01:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 shutdown_connections 2024-04-25T00:01:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae9c06be60 0x7fae9c06e320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faeb80d28c0 0x7faeb81496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb815e510 0x7faeb80b7f50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 --2- 172.21.15.113:0/1854915363 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faeb80b63b0 0x7faeb80ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 >> 172.21.15.113:0/1854915363 conn(0x7faeb806b720 msgr2=0x7faeb815c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 shutdown_connections 2024-04-25T00:01:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:20.580+0000 7faebf5e3640 1 -- 172.21.15.113:0/1854915363 wait complete. 2024-04-25T00:01:20.591 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:01:20.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:20.592 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:21.866 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:01:21.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:21.866 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:01:21.881 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:01:21.881 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:01:21.881 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:21.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:01:21.899 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:01:21.899 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:21.899 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:21.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:01:21.936 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:01:21.936 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:01:21.936 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:21.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:01:21.956 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:22.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.071+0000 7f1e8eb4f640 1 Processor -- start 2024-04-25T00:01:22.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.071+0000 7f1e8eb4f640 1 -- start start 2024-04-25T00:01:22.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e8eb4f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1e880b3fd0 0x7f1e8806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e8eb4f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e88144cf0 con 0x7f1e8806e510 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e8eb4f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e880b2f30 con 0x7f1e88144910 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e8eb4f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8806e980 con 0x7f1e880b3fd0 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e8d0ec640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1e8806e510 0x7f1e880cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34256/0 (socket says 172.21.15.113:34256) 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e8d0ec640 1 -- 172.21.15.113:0/2483975771 learned_addr learned my addr 172.21.15.113:0/2483975771 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2483975771 >> v2:172.21.15.178:3300/0 conn(0x7f1e880b3fd0 0x7f1e8806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861949883 0 0) 0x7f1e88144cf0 con 0x7f1e8806e510 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e70003520 con 0x7f1e8806e510 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3835004936 0 0) 0x7f1e70003520 con 0x7f1e8806e510 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 >> v2:172.21.15.178:3300/0 conn(0x7f1e880b3fd0 msgr2=0x7f1e8806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:22.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 --2- 172.21.15.113:0/2483975771 >> v2:172.21.15.178:3300/0 conn(0x7f1e880b3fd0 0x7f1e8806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 >> v1:172.21.15.178:6789/0 conn(0x7f1e88144910 legacy=0x7f1e880b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880b3210 con 0x7f1e8806e510 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1e800019f0 con 0x7f1e8806e510 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 >> v1:172.21.15.113:6789/0 conn(0x7f1e8806e510 legacy=0x7f1e880cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e700039e0 0x7f1e70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e700051c0 0x7f1e7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8806e980 con 0x7f1e70004800 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7000c090 con 0x7f1e700051c0 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.072+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7000c290 con 0x7f1e700039e0 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e8c8eb640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e700039e0 0x7f1e70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e8d0ec640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e700051c0 0x7f1e7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e7ffff640 1 -- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e700039e0 msgr2=0x7f1e70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e700039e0 0x7f1e70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e7ffff640 1 -- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e700051c0 msgr2=0x7f1e7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e700051c0 0x7f1e7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e8d0ec640 1 -- 172.21.15.113:0/2483975771 reap_dead start 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.073+0000 7f1e7ffff640 1 -- 172.21.15.113:0/2483975771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e70003520 con 0x7f1e70004800 2024-04-25T00:01:22.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e70004c80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009ac0 tx=0x7f1e70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a31bac424f2f73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78036070 con 0x7f1e70004800 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e78031400 con 0x7f1e70004800 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e7f7fe640 1 -- 172.21.15.113:0/2483975771 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78009690 con 0x7f1e70004800 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2483975771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 msgr2=0x7f1e70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e70004c80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009ac0 tx=0x7f1e70005830 comp rx=0 tx=0).stop 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2483975771 shutdown_connections 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2483975771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e70004c80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2483975771 >> 172.21.15.113:0/2483975771 conn(0x7f1e880b1a50 msgr2=0x7f1e880b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2483975771 shutdown_connections 2024-04-25T00:01:22.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.074+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2483975771 wait complete. 2024-04-25T00:01:22.080 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:01:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.075+0000 7f1e8eb4f640 1 Processor -- start 2024-04-25T00:01:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.075+0000 7f1e8eb4f640 1 -- start start 2024-04-25T00:01:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8eb4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e8815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8eb4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8813b820 0x7f1e880ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8eb4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e881f4390 0x7f1e880ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8eb4f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e880b4440 con 0x7f1e70004800 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8eb4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e880b37d0 con 0x7f1e881f4390 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8eb4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e880b4da0 con 0x7f1e8813b820 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8c8eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e8815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8c8eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e8815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44422/0 (socket says 172.21.15.113:44422) 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e8c8eb640 1 -- 172.21.15.113:0/2705615570 learned_addr learned my addr 172.21.15.113:0/2705615570 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.076+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8813b820 0x7f1e880ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8d0ec640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e881f4390 0x7f1e880ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8c8eb640 1 -- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8813b820 msgr2=0x7f1e880ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8c8eb640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8813b820 0x7f1e880ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8c8eb640 1 -- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e881f4390 msgr2=0x7f1e880ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8c8eb640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e881f4390 0x7f1e880ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8c8eb640 1 -- 172.21.15.113:0/2705615570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e881f4d00 con 0x7f1e70004800 2024-04-25T00:01:22.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8d0ec640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e881f4390 0x7f1e880ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8c8eb640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e8815d9e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400f850 tx=0x7f1e8400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e7d7fa640 1 -- 172.21.15.113:0/2705615570 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400fe40 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e7d7fa640 1 -- 172.21.15.113:0/2705615570 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e84018070 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e881f4fe0 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e7d7fa640 1 -- 172.21.15.113:0/2705615570 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400d730 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.077+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8813bd20 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.078+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e880691a0 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.078+0000 7f1e7d7fa640 1 -- 172.21.15.113:0/2705615570 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e8400d8d0 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.079+0000 7f1e7d7fa640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e7406be60 0x7f1e7406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.079+0000 7f1e7d7fa640 1 -- 172.21.15.113:0/2705615570 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f1e8408acb0 con 0x7f1e70004800 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.079+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e7406be60 0x7f1e7406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.079+0000 7f1e7ffff640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e7406be60 0x7f1e7406e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1e78033000 tx=0x7f1e78008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.083+0000 7f1e7d7fa640 1 -- 172.21.15.113:0/2705615570 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e84059620 con 0x7f1e70004800 2024-04-25T00:01:22.087 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:01:22.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.304+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e880731a0 con 0x7f1e70004800 2024-04-25T00:01:22.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.305+0000 7f1e7d7fa640 1 -- 172.21.15.113:0/2705615570 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5191 (secure 0 0 0) 0x7f1e84058f20 con 0x7f1e70004800 2024-04-25T00:01:22.306 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:22.307 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:22.191795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:22.307 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-04-25T00:01:22.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e7406be60 msgr2=0x7f1e7406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e7406be60 0x7f1e7406e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1e78033000 tx=0x7f1e78008030 comp rx=0 tx=0).stop 2024-04-25T00:01:22.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 msgr2=0x7f1e8815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e8815d9e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400f850 tx=0x7f1e8400fd10 comp rx=0 tx=0).stop 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 shutdown_connections 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e7406be60 0x7f1e7406e320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e881f4390 0x7f1e880ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8813b820 0x7f1e880ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 --2- 172.21.15.113:0/2705615570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70004800 0x7f1e8815d9e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 >> 172.21.15.113:0/2705615570 conn(0x7f1e880b1a50 msgr2=0x7f1e88071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 shutdown_connections 2024-04-25T00:01:22.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.307+0000 7f1e8eb4f640 1 -- 172.21.15.113:0/2705615570 wait complete. 2024-04-25T00:01:22.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:22.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.414+0000 7f6dbb16a640 1 Processor -- start 2024-04-25T00:01:22.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.414+0000 7f6dbb16a640 1 -- start start 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.414+0000 7f6dbb16a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db406e6e0 0x7f6db40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.414+0000 7f6dbb16a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6db41453c0 0x7f6db4071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.414+0000 7f6dbb16a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db40cfad0 con 0x7f6db4144d60 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6dbb16a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6db4145c40 con 0x7f6db41453c0 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6dbb16a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6db40cf230 con 0x7f6db406e6e0 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db9707640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6db41453c0 0x7f6db4071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db9707640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6db41453c0 0x7f6db4071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44430/0 (socket says 172.21.15.113:44430) 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db9707640 1 -- 172.21.15.113:0/717277314 learned_addr learned my addr 172.21.15.113:0/717277314 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db9707640 1 -- 172.21.15.113:0/717277314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db406e6e0 msgr2=0x7f6db40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db3fff640 1 --2- 172.21.15.113:0/717277314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db406e6e0 0x7f6db40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db9707640 1 --2- 172.21.15.113:0/717277314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db406e6e0 0x7f6db40b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db9707640 1 -- 172.21.15.113:0/717277314 >> v1:172.21.15.178:6789/0 conn(0x7f6db4144d60 legacy=0x7f6db406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:22.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db9707640 1 -- 172.21.15.113:0/717277314 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db40cf510 con 0x7f6db41453c0 2024-04-25T00:01:22.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.415+0000 7f6db3fff640 1 --2- 172.21.15.113:0/717277314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db406e6e0 0x7f6db40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:22.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6db9707640 1 --2- 172.21.15.113:0/717277314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db41453c0 0x7f6db4071330 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009e50 tx=0x7f6dac02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614a491622e10e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6db37fe640 1 -- 172.21.15.113:0/717277314 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac036070 con 0x7f6db41453c0 2024-04-25T00:01:22.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6db37fe640 1 -- 172.21.15.113:0/717277314 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dac031600 con 0x7f6db41453c0 2024-04-25T00:01:22.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6db37fe640 1 -- 172.21.15.113:0/717277314 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0319d0 con 0x7f6db41453c0 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 -- 172.21.15.113:0/717277314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db41453c0 msgr2=0x7f6db4071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/717277314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db41453c0 0x7f6db4071330 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009e50 tx=0x7f6dac02a630 comp rx=0 tx=0).stop 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 -- 172.21.15.113:0/717277314 shutdown_connections 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/717277314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db406e6e0 0x7f6db40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/717277314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db41453c0 0x7f6db4071330 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 -- 172.21.15.113:0/717277314 >> 172.21.15.113:0/717277314 conn(0x7f6db40b1a50 msgr2=0x7f6db40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 -- 172.21.15.113:0/717277314 shutdown_connections 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.416+0000 7f6dbb16a640 1 -- 172.21.15.113:0/717277314 wait complete. 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.417+0000 7f6dbb16a640 1 Processor -- start 2024-04-25T00:01:22.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6dbb16a640 1 -- start start 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6dbb16a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db406e6e0 0x7f6db413c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6dbb16a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db4144d60 0x7f6db41f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6dbb16a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6db41453c0 0x7f6db415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6dbb16a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4145140 con 0x7f6db406e6e0 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6dbb16a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4070e10 con 0x7f6db41453c0 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6dbb16a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6db4071870 con 0x7f6db4144d60 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6db8f06640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db406e6e0 0x7f6db413c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.418+0000 7f6db8f06640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db406e6e0 0x7f6db413c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44442/0 (socket says 172.21.15.113:44442) 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db8f06640 1 -- 172.21.15.113:0/842038638 learned_addr learned my addr 172.21.15.113:0/842038638 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db3fff640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db4144d60 0x7f6db41f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db9707640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6db41453c0 0x7f6db415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db8f06640 1 -- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db4144d60 msgr2=0x7f6db41f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db8f06640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db4144d60 0x7f6db41f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db8f06640 1 -- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6db41453c0 msgr2=0x7f6db415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db8f06640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6db41453c0 0x7f6db415adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db8f06640 1 -- 172.21.15.113:0/842038638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db41f30a0 con 0x7f6db406e6e0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db3fff640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db4144d60 0x7f6db41f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db8f06640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db406e6e0 0x7f6db413c4e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c430 tx=0x7f6da800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db17fa640 1 -- 172.21.15.113:0/842038638 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8011790 con 0x7f6db406e6e0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db17fa640 1 -- 172.21.15.113:0/842038638 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da8017070 con 0x7f6db406e6e0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db413b440 con 0x7f6db406e6e0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6db17fa640 1 -- 172.21.15.113:0/842038638 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8011b70 con 0x7f6db406e6e0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.419+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db413ba00 con 0x7f6db406e6e0 2024-04-25T00:01:22.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.420+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db40691a0 con 0x7f6db406e6e0 2024-04-25T00:01:22.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.420+0000 7f6db17fa640 1 -- 172.21.15.113:0/842038638 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6da8011d10 con 0x7f6db406e6e0 2024-04-25T00:01:22.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.421+0000 7f6db17fa640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6d9806be30 0x7f6d9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.421+0000 7f6db17fa640 1 -- 172.21.15.113:0/842038638 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f6da808ace0 con 0x7f6db406e6e0 2024-04-25T00:01:22.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.421+0000 7f6db3fff640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6d9806be30 0x7f6d9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.421+0000 7f6db3fff640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6d9806be30 0x7f6d9806e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006fd0 tx=0x7f6da0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.425+0000 7f6db17fa640 1 -- 172.21.15.113:0/842038638 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6da80596e0 con 0x7f6db406e6e0 2024-04-25T00:01:22.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.599+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6db40b27e0 con 0x7f6db406e6e0 2024-04-25T00:01:22.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.600+0000 7f6db17fa640 1 -- 172.21.15.113:0/842038638 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16812 (secure 0 0 0) 0x7f6da8058fe0 con 0x7f6db406e6e0 2024-04-25T00:01:22.601 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:22.601 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":48,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:01.211634+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:47.724388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:48.476831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:22.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6d9806be30 msgr2=0x7f6d9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6d9806be30 0x7f6d9806e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006fd0 tx=0x7f6da0008040 comp rx=0 tx=0).stop 2024-04-25T00:01:22.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db406e6e0 msgr2=0x7f6db413c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db406e6e0 0x7f6db413c4e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c430 tx=0x7f6da800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 shutdown_connections 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6d9806be30 0x7f6d9806e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6db41453c0 0x7f6db415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6db4144d60 0x7f6db41f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 --2- 172.21.15.113:0/842038638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6db406e6e0 0x7f6db413c4e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 >> 172.21.15.113:0/842038638 conn(0x7f6db40b1a50 msgr2=0x7f6db4071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 shutdown_connections 2024-04-25T00:01:22.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.602+0000 7f6dbb16a640 1 -- 172.21.15.113:0/842038638 wait complete. 2024-04-25T00:01:22.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:47.724388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:48.476831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:22.614 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:22.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.712+0000 7f81f3aa5640 1 Processor -- start 2024-04-25T00:01:22.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.712+0000 7f81f3aa5640 1 -- start start 2024-04-25T00:01:22.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f3aa5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f81ec15e2d0 0x7f81ec15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f3aa5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81ec0b3080 0x7f81ec0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f3aa5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81ec15f4f0 con 0x7f81ec15e6b0 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f3aa5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f81ec0cde00 con 0x7f81ec0b3080 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f3aa5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f81ec15f7d0 con 0x7f81ec15e2d0 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1040640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81ec0b3080 0x7f81ec0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1040640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81ec0b3080 0x7f81ec0b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44446/0 (socket says 172.21.15.113:44446) 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1040640 1 -- 172.21.15.113:0/3748020764 learned_addr learned my addr 172.21.15.113:0/3748020764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1841640 1 --2- 172.21.15.113:0/3748020764 >> v2:172.21.15.178:3300/0 conn(0x7f81ec15e2d0 0x7f81ec15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1040640 1 -- 172.21.15.113:0/3748020764 >> v2:172.21.15.178:3300/0 conn(0x7f81ec15e2d0 msgr2=0x7f81ec15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1040640 1 --2- 172.21.15.113:0/3748020764 >> v2:172.21.15.178:3300/0 conn(0x7f81ec15e2d0 0x7f81ec15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1040640 1 -- 172.21.15.113:0/3748020764 >> v1:172.21.15.178:6789/0 conn(0x7f81ec15e6b0 legacy=0x7f81ec071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.713+0000 7f81f1040640 1 -- 172.21.15.113:0/3748020764 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec15fab0 con 0x7f81ec0b3080 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.714+0000 7f81f1040640 1 --2- 172.21.15.113:0/3748020764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec0b3080 0x7f81ec0b4330 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0b2d40 tx=0x7f81dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addeb9c93b764ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.714+0000 7f81f083f640 1 -- 172.21.15.113:0/3748020764 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc037070 con 0x7f81ec0b3080 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.714+0000 7f81f083f640 1 -- 172.21.15.113:0/3748020764 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81dc032440 con 0x7f81ec0b3080 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.714+0000 7f81f083f640 1 -- 172.21.15.113:0/3748020764 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc032810 con 0x7f81ec0b3080 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3748020764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec0b3080 msgr2=0x7f81ec0b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3748020764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec0b3080 0x7f81ec0b4330 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0b2d40 tx=0x7f81dc02a230 comp rx=0 tx=0).stop 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3748020764 shutdown_connections 2024-04-25T00:01:22.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3748020764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec0b3080 0x7f81ec0b4330 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3748020764 >> v2:172.21.15.178:3300/0 conn(0x7f81ec15e2d0 0x7f81ec15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3748020764 >> 172.21.15.113:0/3748020764 conn(0x7f81ec06b0f0 msgr2=0x7f81ec06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3748020764 shutdown_connections 2024-04-25T00:01:22.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.715+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3748020764 wait complete. 2024-04-25T00:01:22.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.716+0000 7f81f3aa5640 1 Processor -- start 2024-04-25T00:01:22.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.716+0000 7f81f3aa5640 1 -- start start 2024-04-25T00:01:22.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f3aa5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81ec0b3080 0x7f81ec145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f3aa5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec15e2d0 0x7f81ec0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f3aa5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81ec15e6b0 0x7f81ec0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f3aa5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec15f2b0 con 0x7f81ec15e2d0 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f3aa5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec15f130 con 0x7f81ec0b3080 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f3aa5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81ec0cf2a0 con 0x7f81ec15e6b0 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec15e2d0 0x7f81ec0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec15e2d0 0x7f81ec0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44448/0 (socket says 172.21.15.113:44448) 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 -- 172.21.15.113:0/3622069067 learned_addr learned my addr 172.21.15.113:0/3622069067 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 -- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81ec15e6b0 msgr2=0x7f81ec0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1841640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81ec0b3080 0x7f81ec145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f2042640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81ec15e6b0 0x7f81ec0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81ec15e6b0 0x7f81ec0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 -- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81ec0b3080 msgr2=0x7f81ec145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81ec0b3080 0x7f81ec145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 -- 172.21.15.113:0/3622069067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec155fd0 con 0x7f81ec15e2d0 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1841640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81ec0b3080 0x7f81ec145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f2042640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81ec15e6b0 0x7f81ec0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.717+0000 7f81f1040640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec15e2d0 0x7f81ec0ce290 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f81dc02a270 tx=0x7f81dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.718+0000 7f81e27fc640 1 -- 172.21.15.113:0/3622069067 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc037040 con 0x7f81ec15e2d0 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.718+0000 7f81e27fc640 1 -- 172.21.15.113:0/3622069067 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81dc042070 con 0x7f81ec15e2d0 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.718+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec15ce40 con 0x7f81ec15e2d0 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.718+0000 7f81e27fc640 1 -- 172.21.15.113:0/3622069067 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc030bb0 con 0x7f81ec15e2d0 2024-04-25T00:01:22.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.718+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec15d400 con 0x7f81ec15e2d0 2024-04-25T00:01:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.719+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ec0691a0 con 0x7f81ec15e2d0 2024-04-25T00:01:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.719+0000 7f81e27fc640 1 -- 172.21.15.113:0/3622069067 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81dc047460 con 0x7f81ec15e2d0 2024-04-25T00:01:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.719+0000 7f81e27fc640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81d006be30 0x7f81d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.719+0000 7f81e27fc640 1 -- 172.21.15.113:0/3622069067 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f81dc0abaf0 con 0x7f81ec15e2d0 2024-04-25T00:01:22.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.720+0000 7f81f1841640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81d006be30 0x7f81d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.720+0000 7f81f1841640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81d006be30 0x7f81d006e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81ec152bc0 tx=0x7f81e80074c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.724+0000 7f81e27fc640 1 -- 172.21.15.113:0/3622069067 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81dc045360 con 0x7f81ec15e2d0 2024-04-25T00:01:22.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.896+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81ec072050 con 0x7f81ec15e2d0 2024-04-25T00:01:22.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.896+0000 7f81e27fc640 1 -- 172.21.15.113:0/3622069067 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f81dc07a4f0 con 0x7f81ec15e2d0 2024-04-25T00:01:22.898 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81d006be30 msgr2=0x7f81d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81d006be30 0x7f81d006e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81ec152bc0 tx=0x7f81e80074c0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec15e2d0 msgr2=0x7f81ec0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec15e2d0 0x7f81ec0ce290 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f81dc02a270 tx=0x7f81dc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 shutdown_connections 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81d006be30 0x7f81d006e2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81ec15e6b0 0x7f81ec0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ec15e2d0 0x7f81ec0ce290 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 --2- 172.21.15.113:0/3622069067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81ec0b3080 0x7f81ec145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 >> 172.21.15.113:0/3622069067 conn(0x7f81ec06b0f0 msgr2=0x7f81ec0aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 shutdown_connections 2024-04-25T00:01:22.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:22.898+0000 7f81f3aa5640 1 -- 172.21.15.113:0/3622069067 wait complete. 2024-04-25T00:01:22.910 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:01:23.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.007+0000 7f7c26d89640 1 Processor -- start 2024-04-25T00:01:23.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.007+0000 7f7c26d89640 1 -- start start 2024-04-25T00:01:23.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c26d89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c2015ba50 0x7f7c2015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c26d89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c2006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c26d89640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c2006d7e0 con 0x7f7c2015b670 2024-04-25T00:01:23.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c26d89640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7c20071170 con 0x7f7c2015ba50 2024-04-25T00:01:23.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c26d89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c20071450 con 0x7f7c2006cea0 2024-04-25T00:01:23.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c1ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c2015ba50 0x7f7c2015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c1ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c2015ba50 0x7f7c2015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44454/0 (socket says 172.21.15.113:44454) 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c1ffff640 1 -- 172.21.15.113:0/921117250 learned_addr learned my addr 172.21.15.113:0/921117250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c25326640 1 --2- 172.21.15.113:0/921117250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c2006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c1ffff640 1 -- 172.21.15.113:0/921117250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 msgr2=0x7f7c2006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c1ffff640 1 --2- 172.21.15.113:0/921117250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c2006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c1ffff640 1 -- 172.21.15.113:0/921117250 >> v1:172.21.15.178:6789/0 conn(0x7f7c2015b670 legacy=0x7f7c200cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.008+0000 7f7c1ffff640 1 -- 172.21.15.113:0/921117250 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c20071730 con 0x7f7c2015ba50 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.009+0000 7f7c1ffff640 1 --2- 172.21.15.113:0/921117250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c2015bff0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400bb10 tx=0x7f7c1402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d66f771508e622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.009+0000 7f7c1f7fe640 1 -- 172.21.15.113:0/921117250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14031710 con 0x7f7c2015ba50 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.009+0000 7f7c1f7fe640 1 -- 172.21.15.113:0/921117250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c140318b0 con 0x7f7c2015ba50 2024-04-25T00:01:23.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.009+0000 7f7c1f7fe640 1 -- 172.21.15.113:0/921117250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14031c80 con 0x7f7c2015ba50 2024-04-25T00:01:23.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 -- 172.21.15.113:0/921117250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 msgr2=0x7f7c2015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 --2- 172.21.15.113:0/921117250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c2015bff0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400bb10 tx=0x7f7c1402c7a0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 -- 172.21.15.113:0/921117250 shutdown_connections 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 --2- 172.21.15.113:0/921117250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c2006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 --2- 172.21.15.113:0/921117250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c2015bff0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 -- 172.21.15.113:0/921117250 >> 172.21.15.113:0/921117250 conn(0x7f7c2006b0f0 msgr2=0x7f7c2006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 -- 172.21.15.113:0/921117250 shutdown_connections 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.010+0000 7f7c26d89640 1 -- 172.21.15.113:0/921117250 wait complete. 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.011+0000 7f7c26d89640 1 Processor -- start 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.011+0000 7f7c26d89640 1 -- start start 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c26d89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c201498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c26d89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c2015b670 0x7f7c20151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c26d89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c20149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c26d89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200cca90 con 0x7f7c2015ba50 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c26d89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200cc910 con 0x7f7c2015b670 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c26d89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c200d0080 con 0x7f7c2006cea0 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c20149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c20149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44462/0 (socket says 172.21.15.113:44462) 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 -- 172.21.15.113:0/3624150016 learned_addr learned my addr 172.21.15.113:0/3624150016 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c1ffff640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c2015b670 0x7f7c20151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c24b25640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c201498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 -- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 msgr2=0x7f7c201498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c201498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 -- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c2015b670 msgr2=0x7f7c20151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c2015b670 0x7f7c20151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 -- 172.21.15.113:0/3624150016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201575b0 con 0x7f7c2015ba50 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c1ffff640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c2015b670 0x7f7c20151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c25326640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c20149210 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7c18007e40 tx=0x7f7c1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.012+0000 7f7c1d7fa640 1 -- 172.21.15.113:0/3624150016 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18014620 con 0x7f7c2015ba50 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.013+0000 7f7c1d7fa640 1 -- 172.21.15.113:0/3624150016 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c18019070 con 0x7f7c2015ba50 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.013+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c20151090 con 0x7f7c2015ba50 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.013+0000 7f7c1d7fa640 1 -- 172.21.15.113:0/3624150016 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18014a20 con 0x7f7c2015ba50 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.013+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c20151650 con 0x7f7c2015ba50 2024-04-25T00:01:23.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.014+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c20151850 con 0x7f7c2015ba50 2024-04-25T00:01:23.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.014+0000 7f7c1d7fa640 1 -- 172.21.15.113:0/3624150016 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c18014bc0 con 0x7f7c2015ba50 2024-04-25T00:01:23.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.014+0000 7f7c1d7fa640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.015+0000 7f7c24b25640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.015+0000 7f7c1d7fa640 1 -- 172.21.15.113:0/3624150016 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f7c1808b7c0 con 0x7f7c2015ba50 2024-04-25T00:01:23.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.015+0000 7f7c24b25640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7c10007fd0 tx=0x7f7c10009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.019+0000 7f7c1d7fa640 1 -- 172.21.15.113:0/3624150016 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c20151850 con 0x7f7c2015ba50 2024-04-25T00:01:23.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.191+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c200691a0 con 0x7f7c2015ba50 2024-04-25T00:01:23.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.191+0000 7f7c1d7fa640 1 -- 172.21.15.113:0/3624150016 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7f7c1805a130 con 0x7f7c2015ba50 2024-04-25T00:01:23.193 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:01:23.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c0c06be30 msgr2=0x7f7c0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7c10007fd0 tx=0x7f7c10009040 comp rx=0 tx=0).stop 2024-04-25T00:01:23.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 msgr2=0x7f7c20149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c20149210 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7c18007e40 tx=0x7f7c1800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 shutdown_connections 2024-04-25T00:01:23.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c2015b670 0x7f7c20151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c2015ba50 0x7f7c20149210 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 --2- 172.21.15.113:0/3624150016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c2006cea0 0x7f7c201498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 >> 172.21.15.113:0/3624150016 conn(0x7f7c2006b0f0 msgr2=0x7f7c20154ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:23.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 shutdown_connections 2024-04-25T00:01:23.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.194+0000 7f7c26d89640 1 -- 172.21.15.113:0/3624150016 wait complete. 2024-04-25T00:01:23.205 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:01:23.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.304+0000 7f8f8602a640 1 Processor -- start 2024-04-25T00:01:23.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.304+0000 7f8f8602a640 1 -- start start 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.304+0000 7f8f8602a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 0x7f8f8015e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.304+0000 7f8f8602a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8f8006fe70 0x7f8f800d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.304+0000 7f8f8602a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8f800d23c0 0x7f8f800b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.304+0000 7f8f8602a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8006b420 con 0x7f8f800d23c0 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8015ee60 con 0x7f8f8006fe70 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8015f140 con 0x7f8f80147640 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8f800d23c0 0x7f8f800b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8f800d23c0 0x7f8f800b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44472/0 (socket says 172.21.15.113:44472) 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 -- 172.21.15.113:0/1048042540 learned_addr learned my addr 172.21.15.113:0/1048042540 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 -- 172.21.15.113:0/1048042540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 msgr2=0x7f8f8015e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 --2- 172.21.15.113:0/1048042540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 0x7f8f8015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 -- 172.21.15.113:0/1048042540 >> v2:172.21.15.178:3300/0 conn(0x7f8f8006fe70 msgr2=0x7f8f800d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 --2- 172.21.15.113:0/1048042540 >> v2:172.21.15.178:3300/0 conn(0x7f8f8006fe70 0x7f8f800d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 -- 172.21.15.113:0/1048042540 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8015f420 con 0x7f8f800d23c0 2024-04-25T00:01:23.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7ffff640 1 --2- 172.21.15.113:0/1048042540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f800d23c0 0x7f8f800b3ae0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8f800b4be0 tx=0x7f8f7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe8105d999d078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7e7fc640 1 -- 172.21.15.113:0/1048042540 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78036070 con 0x7f8f800d23c0 2024-04-25T00:01:23.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7e7fc640 1 -- 172.21.15.113:0/1048042540 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f78031600 con 0x7f8f800d23c0 2024-04-25T00:01:23.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f7e7fc640 1 -- 172.21.15.113:0/1048042540 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f780319d0 con 0x7f8f800d23c0 2024-04-25T00:01:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 -- 172.21.15.113:0/1048042540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f800d23c0 msgr2=0x7f8f800b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 --2- 172.21.15.113:0/1048042540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f800d23c0 0x7f8f800b3ae0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8f800b4be0 tx=0x7f8f7802a6a0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 -- 172.21.15.113:0/1048042540 shutdown_connections 2024-04-25T00:01:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 --2- 172.21.15.113:0/1048042540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f800d23c0 0x7f8f800b3ae0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 --2- 172.21.15.113:0/1048042540 >> v2:172.21.15.178:3300/0 conn(0x7f8f8006fe70 0x7f8f800d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 --2- 172.21.15.113:0/1048042540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 0x7f8f8015e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 -- 172.21.15.113:0/1048042540 >> 172.21.15.113:0/1048042540 conn(0x7f8f800b7370 msgr2=0x7f8f800b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 -- 172.21.15.113:0/1048042540 shutdown_connections 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.306+0000 7f8f8602a640 1 -- 172.21.15.113:0/1048042540 wait complete. 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.307+0000 7f8f8602a640 1 Processor -- start 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f8602a640 1 -- start start 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f8602a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8006fe70 0x7f8f800cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f8602a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f800d23c0 0x7f8f800cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f8602a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 0x7f8f8013e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f8602a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8015fbc0 con 0x7f8f8006fe70 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f8602a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8015fa40 con 0x7f8f800d23c0 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f8602a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8f800702d0 con 0x7f8f80147640 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8006fe70 0x7f8f800cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8006fe70 0x7f8f800cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44480/0 (socket says 172.21.15.113:44480) 2024-04-25T00:01:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.308+0000 7f8f7f7fe640 1 -- 172.21.15.113:0/3338795437 learned_addr learned my addr 172.21.15.113:0/3338795437 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7ffff640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 0x7f8f8013e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7effd640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f800d23c0 0x7f8f800cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7f7fe640 1 -- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 msgr2=0x7f8f8013e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7f7fe640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 0x7f8f8013e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7f7fe640 1 -- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f800d23c0 msgr2=0x7f8f800cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7f7fe640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f800d23c0 0x7f8f800cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7f7fe640 1 -- 172.21.15.113:0/3338795437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f800cb1d0 con 0x7f8f8006fe70 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f7f7fe640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8006fe70 0x7f8f800cf850 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000b4a0 tx=0x7f8f7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f67fff640 1 -- 172.21.15.113:0/3338795437 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70014070 con 0x7f8f8006fe70 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f67fff640 1 -- 172.21.15.113:0/3338795437 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f70010610 con 0x7f8f8006fe70 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f67fff640 1 -- 172.21.15.113:0/3338795437 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70010a00 con 0x7f8f8006fe70 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f800d00b0 con 0x7f8f8006fe70 2024-04-25T00:01:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.309+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f800cb490 con 0x7f8f8006fe70 2024-04-25T00:01:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.310+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f800691a0 con 0x7f8f8006fe70 2024-04-25T00:01:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.311+0000 7f8f67fff640 1 -- 172.21.15.113:0/3338795437 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f70017070 con 0x7f8f8006fe70 2024-04-25T00:01:23.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.311+0000 7f8f67fff640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f6c06be60 0x7f8f6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.311+0000 7f8f67fff640 1 -- 172.21.15.113:0/3338795437 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f8f7008a7c0 con 0x7f8f8006fe70 2024-04-25T00:01:23.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.311+0000 7f8f7effd640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f6c06be60 0x7f8f6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.311+0000 7f8f7effd640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f6c06be60 0x7f8f6c06e320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f80068740 tx=0x7f8f74009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.316+0000 7f8f67fff640 1 -- 172.21.15.113:0/3338795437 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f70058b70 con 0x7f8f8006fe70 2024-04-25T00:01:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.487+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f8006e310 con 0x7f8f8006fe70 2024-04-25T00:01:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.488+0000 7f8f67fff640 1 -- 172.21.15.113:0/3338795437 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7f8f7005ce70 con 0x7f8f8006fe70 2024-04-25T00:01:23.489 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:01:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f6c06be60 msgr2=0x7f8f6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f6c06be60 0x7f8f6c06e320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f80068740 tx=0x7f8f74009060 comp rx=0 tx=0).stop 2024-04-25T00:01:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8006fe70 msgr2=0x7f8f800cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8006fe70 0x7f8f800cf850 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000b4a0 tx=0x7f8f7000b960 comp rx=0 tx=0).stop 2024-04-25T00:01:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 shutdown_connections 2024-04-25T00:01:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8f6c06be60 0x7f8f6c06e320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f800d23c0 0x7f8f800cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f80147640 0x7f8f8013e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 --2- 172.21.15.113:0/3338795437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8006fe70 0x7f8f800cf850 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 >> 172.21.15.113:0/3338795437 conn(0x7f8f800b7370 msgr2=0x7f8f801464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 shutdown_connections 2024-04-25T00:01:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.490+0000 7f8f8602a640 1 -- 172.21.15.113:0/3338795437 wait complete. 2024-04-25T00:01:23.502 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:23.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.597+0000 7efd5ccd4640 1 Processor -- start 2024-04-25T00:01:23.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.597+0000 7efd5ccd4640 1 -- start start 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5ccd4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd58144d60 0x7efd580b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5ccd4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd58071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5ccd4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd580cfad0 con 0x7efd581453c0 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5ccd4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efd58145c40 con 0x7efd58144d60 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5ccd4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd580cf230 con 0x7efd5806e6e0 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd58144d60 0x7efd580b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd4ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efd581453c0 0x7efd5806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34258/0 (socket says 172.21.15.113:34258) 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd58144d60 0x7efd580b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44494/0 (socket says 172.21.15.113:44494) 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd4ffff640 1 -- 172.21.15.113:0/4049808773 learned_addr learned my addr 172.21.15.113:0/4049808773 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:23.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd56d9d640 1 --2- 172.21.15.113:0/4049808773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd58071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5659c640 1 -- 172.21.15.113:0/4049808773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 msgr2=0x7efd58071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5659c640 1 --2- 172.21.15.113:0/4049808773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd58071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5659c640 1 -- 172.21.15.113:0/4049808773 >> v1:172.21.15.113:6789/0 conn(0x7efd581453c0 legacy=0x7efd5806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd5659c640 1 -- 172.21.15.113:0/4049808773 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd580cf510 con 0x7efd58144d60 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.598+0000 7efd55d9b640 1 -- 172.21.15.113:0/4049808773 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91317472 0 0) 0x7efd580cfad0 con 0x7efd581453c0 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5659c640 1 --2- 172.21.15.113:0/4049808773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd580b5390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7efd480097a0 tx=0x7efd4802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de16eba86d3acc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd55d9b640 1 -- 172.21.15.113:0/4049808773 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48037070 con 0x7efd58144d60 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd55d9b640 1 -- 172.21.15.113:0/4049808773 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4802ae50 con 0x7efd58144d60 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd55d9b640 1 -- 172.21.15.113:0/4049808773 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48032660 con 0x7efd58144d60 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 -- 172.21.15.113:0/4049808773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 msgr2=0x7efd580b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/4049808773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd580b5390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7efd480097a0 tx=0x7efd4802a100 comp rx=0 tx=0).stop 2024-04-25T00:01:23.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 -- 172.21.15.113:0/4049808773 shutdown_connections 2024-04-25T00:01:23.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/4049808773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd58071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/4049808773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd580b5390 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 -- 172.21.15.113:0/4049808773 >> 172.21.15.113:0/4049808773 conn(0x7efd580b1a50 msgr2=0x7efd580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:23.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 -- 172.21.15.113:0/4049808773 shutdown_connections 2024-04-25T00:01:23.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.599+0000 7efd5ccd4640 1 -- 172.21.15.113:0/4049808773 wait complete. 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.601+0000 7efd5ccd4640 1 Processor -- start 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.601+0000 7efd5ccd4640 1 -- start start 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd581438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd5813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd581453c0 0x7efd5813e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd58145140 con 0x7efd58144d60 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd58070e10 con 0x7efd581453c0 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd58071870 con 0x7efd5806e6e0 2024-04-25T00:01:23.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd5813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd5813b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44502/0 (socket says 172.21.15.113:44502) 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 -- 172.21.15.113:0/3090827342 learned_addr learned my addr 172.21.15.113:0/3090827342 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5659c640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd581438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd56d9d640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd581453c0 0x7efd5813e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 -- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 msgr2=0x7efd581438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd581438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 -- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd581453c0 msgr2=0x7efd5813e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd581453c0 0x7efd5813e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 -- 172.21.15.113:0/3090827342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd580a97d0 con 0x7efd58144d60 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4ffff640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd5813b930 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c460 tx=0x7efd4000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4effd640 1 -- 172.21.15.113:0/3090827342 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40018070 con 0x7efd58144d60 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4effd640 1 -- 172.21.15.113:0/3090827342 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd40007500 con 0x7efd58144d60 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd580aedd0 con 0x7efd58144d60 2024-04-25T00:01:23.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd4effd640 1 -- 172.21.15.113:0/3090827342 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40004a20 con 0x7efd58144d60 2024-04-25T00:01:23.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.602+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd580a9480 con 0x7efd58144d60 2024-04-25T00:01:23.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.603+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd580af2f0 con 0x7efd58144d60 2024-04-25T00:01:23.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.604+0000 7efd4effd640 1 -- 172.21.15.113:0/3090827342 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd4001b070 con 0x7efd58144d60 2024-04-25T00:01:23.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.604+0000 7efd4effd640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd3806be30 0x7efd3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.604+0000 7efd4effd640 1 -- 172.21.15.113:0/3090827342 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7efd4008a0a0 con 0x7efd58144d60 2024-04-25T00:01:23.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.604+0000 7efd5659c640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd3806be30 0x7efd3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.605+0000 7efd5659c640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd3806be30 0x7efd3806e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd48034000 tx=0x7efd4802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.608+0000 7efd4effd640 1 -- 172.21.15.113:0/3090827342 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd400583c0 con 0x7efd58144d60 2024-04-25T00:01:23.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.813+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd580691a0 con 0x7efd58144d60 2024-04-25T00:01:23.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.814+0000 7efd4effd640 1 -- 172.21.15.113:0/3090827342 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5191 (secure 0 0 0) 0x7efd4005c6c0 con 0x7efd58144d60 2024-04-25T00:01:23.815 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:23.815 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:22.191795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:23.816 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-04-25T00:01:23.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd3806be30 msgr2=0x7efd3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd3806be30 0x7efd3806e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd48034000 tx=0x7efd4802e000 comp rx=0 tx=0).stop 2024-04-25T00:01:23.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 msgr2=0x7efd5813b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd5813b930 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c460 tx=0x7efd4000c920 comp rx=0 tx=0).stop 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 shutdown_connections 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd3806be30 0x7efd3806e2f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd581453c0 0x7efd5813e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58144d60 0x7efd5813b930 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 --2- 172.21.15.113:0/3090827342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd5806e6e0 0x7efd581438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 >> 172.21.15.113:0/3090827342 conn(0x7efd580b1a50 msgr2=0x7efd580b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 shutdown_connections 2024-04-25T00:01:23.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.817+0000 7efd5ccd4640 1 -- 172.21.15.113:0/3090827342 wait complete. 2024-04-25T00:01:23.828 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e4619d640 1 Processor -- start 2024-04-25T00:01:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e4619d640 1 -- start start 2024-04-25T00:01:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e4619d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3e400b4b00 0x7f3e401656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e4619d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e40071b20 con 0x7f3e40072ea0 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e4619d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e400b2d50 con 0x7f3e4015a6b0 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e4619d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3e40072a70 con 0x7f3e400b4b00 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3e40072ea0 0x7f3e40071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34264/0 (socket says 172.21.15.113:34264) 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3effd640 1 -- 172.21.15.113:0/1964510278 learned_addr learned my addr 172.21.15.113:0/1964510278 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3ffff640 1 --2- 172.21.15.113:0/1964510278 >> v2:172.21.15.178:3300/0 conn(0x7f3e400b4b00 0x7f3e401656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158775065 0 0) 0x7f3e40071b20 con 0x7f3e40072ea0 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e28003520 con 0x7f3e40072ea0 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 59531476 0 0) 0x7f3e28003520 con 0x7f3e40072ea0 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 >> v2:172.21.15.178:3300/0 conn(0x7f3e400b4b00 msgr2=0x7f3e401656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 --2- 172.21.15.113:0/1964510278 >> v2:172.21.15.178:3300/0 conn(0x7f3e400b4b00 0x7f3e401656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 >> v1:172.21.15.178:6789/0 conn(0x7f3e4015a6b0 legacy=0x7f3e40070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e400b3010 con 0x7f3e40072ea0 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f3e30001670 con 0x7f3e40072ea0 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 >> v1:172.21.15.113:6789/0 conn(0x7f3e40072ea0 legacy=0x7f3e40071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e280039b0 0x7f3e28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e28004800 0x7f3e28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40072a70 con 0x7f3e280051c0 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2800c060 con 0x7f3e28004800 2024-04-25T00:01:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2800c260 con 0x7f3e280039b0 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3ffff640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.924+0000 7f3e3f7fe640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e280039b0 0x7f3e28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3effd640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e28004800 0x7f3e28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3ffff640 1 -- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e280039b0 msgr2=0x7f3e28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3ffff640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e280039b0 0x7f3e28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3ffff640 1 -- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e28004800 msgr2=0x7f3e28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3ffff640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e28004800 0x7f3e28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3ffff640 1 -- 172.21.15.113:0/1964510278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e28003520 con 0x7f3e280051c0 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3ffff640 1 -- 172.21.15.113:0/1964510278 reap_dead start 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3ffff640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e2800b9c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800a250 tx=0x7f3e28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a16a72d62195a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800adf0 con 0x7f3e280051c0 2024-04-25T00:01:23.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.925+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e38003990 con 0x7f3e280051c0 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e3e7fc640 1 -- 172.21.15.113:0/1964510278 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38003da0 con 0x7f3e280051c0 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e4619d640 1 -- 172.21.15.113:0/1964510278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 msgr2=0x7f3e2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e4619d640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e2800b9c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800a250 tx=0x7f3e28005830 comp rx=0 tx=0).stop 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e4619d640 1 -- 172.21.15.113:0/1964510278 shutdown_connections 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e4619d640 1 --2- 172.21.15.113:0/1964510278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e2800b9c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e4619d640 1 -- 172.21.15.113:0/1964510278 >> 172.21.15.113:0/1964510278 conn(0x7f3e400b1a50 msgr2=0x7f3e400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e4619d640 1 -- 172.21.15.113:0/1964510278 shutdown_connections 2024-04-25T00:01:23.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.926+0000 7f3e4619d640 1 -- 172.21.15.113:0/1964510278 wait complete. 2024-04-25T00:01:23.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.927+0000 7f3e4619d640 1 Processor -- start 2024-04-25T00:01:23.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.927+0000 7f3e4619d640 1 -- start start 2024-04-25T00:01:23.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e4619d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e40147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e4619d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e40148c00 0x7f3e40146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e4619d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e40149080 0x7f3e4013a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e4619d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40072720 con 0x7f3e280051c0 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e4619d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400b35d0 con 0x7f3e40149080 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e4619d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4015aa90 con 0x7f3e40148c00 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e40147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e40147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44518/0 (socket says 172.21.15.113:44518) 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 -- 172.21.15.113:0/2784917632 learned_addr learned my addr 172.21.15.113:0/2784917632 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3ffff640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e40149080 0x7f3e4013a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3effd640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e40148c00 0x7f3e40146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 -- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e40148c00 msgr2=0x7f3e40146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e40148c00 0x7f3e40146200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 -- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e40149080 msgr2=0x7f3e4013a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e40149080 0x7f3e4013a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3f7fe640 1 -- 172.21.15.113:0/2784917632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e40146cc0 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.928+0000 7f3e3ffff640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e40149080 0x7f3e4013a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.929+0000 7f3e3f7fe640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e40147d60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f620 tx=0x7f3e3400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.929+0000 7f3e27fff640 1 -- 172.21.15.113:0/2784917632 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e340052e0 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.929+0000 7f3e27fff640 1 -- 172.21.15.113:0/2784917632 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e34018070 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.929+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e40146fa0 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.929+0000 7f3e27fff640 1 -- 172.21.15.113:0/2784917632 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34003810 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.929+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e40147560 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.930+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e400691a0 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.930+0000 7f3e27fff640 1 -- 172.21.15.113:0/2784917632 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e340039b0 con 0x7f3e280051c0 2024-04-25T00:01:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.931+0000 7f3e27fff640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:23.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.931+0000 7f3e3effd640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:23.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.931+0000 7f3e27fff640 1 -- 172.21.15.113:0/2784917632 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f3e3408ac70 con 0x7f3e280051c0 2024-04-25T00:01:23.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.931+0000 7f3e3effd640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3e30001050 tx=0x7f3e30009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:23.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:23.936+0000 7f3e27fff640 1 -- 172.21.15.113:0/2784917632 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3e340595e0 con 0x7f3e280051c0 2024-04-25T00:01:24.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.112+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e401e4ea0 con 0x7f3e280051c0 2024-04-25T00:01:24.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.113+0000 7f3e27fff640 1 -- 172.21.15.113:0/2784917632 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16812 (secure 0 0 0) 0x7f3e34058ee0 con 0x7f3e280051c0 2024-04-25T00:01:24.114 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:24.114 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":48,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:01.211634+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:47.724388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:48.476831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3e2c06be30 msgr2=0x7f3e2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3e30001050 tx=0x7f3e30009040 comp rx=0 tx=0).stop 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 msgr2=0x7f3e40147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e40147d60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f620 tx=0x7f3e3400fae0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 shutdown_connections 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3e2c06be30 0x7f3e2c06e2f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e40149080 0x7f3e4013a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e40148c00 0x7f3e40146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 --2- 172.21.15.113:0/2784917632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e280051c0 0x7f3e40147d60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.115+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 >> 172.21.15.113:0/2784917632 conn(0x7f3e400b1a50 msgr2=0x7f3e401507d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:24.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.116+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 shutdown_connections 2024-04-25T00:01:24.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.116+0000 7f3e4619d640 1 -- 172.21.15.113:0/2784917632 wait complete. 2024-04-25T00:01:24.127 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:01:24.127 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:24.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.223+0000 7fc1eb45d640 1 Processor -- start 2024-04-25T00:01:24.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.223+0000 7fc1eb45d640 1 -- start start 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.224+0000 7fc1eb45d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1e4144910 0x7fc1e40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.224+0000 7fc1eb45d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.224+0000 7fc1eb45d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e4144cf0 con 0x7fc1e40b5240 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.224+0000 7fc1eb45d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e40b4510 con 0x7fc1e4144910 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.224+0000 7fc1eb45d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e40b2f30 con 0x7fc1e40b56b0 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.224+0000 7fc1e91f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1e4144910 0x7fc1e40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e91f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1e4144910 0x7fc1e40b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44534/0 (socket says 172.21.15.113:44534) 2024-04-25T00:01:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e91f9640 1 -- 172.21.15.113:0/553522854 learned_addr learned my addr 172.21.15.113:0/553522854 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:24.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e99fa640 1 --2- 172.21.15.113:0/553522854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e91f9640 1 -- 172.21.15.113:0/553522854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 msgr2=0x7fc1e40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:24.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e91f9640 1 --2- 172.21.15.113:0/553522854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e91f9640 1 -- 172.21.15.113:0/553522854 >> v1:172.21.15.178:6789/0 conn(0x7fc1e40b5240 legacy=0x7fc1e406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e91f9640 1 -- 172.21.15.113:0/553522854 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e40b3210 con 0x7fc1e4144910 2024-04-25T00:01:24.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.225+0000 7fc1e91f9640 1 --2- 172.21.15.113:0/553522854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40b3fd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009e80 tx=0x7fc1e002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19483dbb46792649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1dbfff640 1 -- 172.21.15.113:0/553522854 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0036070 con 0x7fc1e4144910 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1dbfff640 1 -- 172.21.15.113:0/553522854 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e0031600 con 0x7fc1e4144910 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1dbfff640 1 -- 172.21.15.113:0/553522854 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e00319d0 con 0x7fc1e4144910 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 -- 172.21.15.113:0/553522854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 msgr2=0x7fc1e40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/553522854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40b3fd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009e80 tx=0x7fc1e002a750 comp rx=0 tx=0).stop 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 -- 172.21.15.113:0/553522854 shutdown_connections 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/553522854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/553522854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40b3fd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 -- 172.21.15.113:0/553522854 >> 172.21.15.113:0/553522854 conn(0x7fc1e40b1a50 msgr2=0x7fc1e40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 -- 172.21.15.113:0/553522854 shutdown_connections 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.226+0000 7fc1eb45d640 1 -- 172.21.15.113:0/553522854 wait complete. 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.227+0000 7fc1eb45d640 1 Processor -- start 2024-04-25T00:01:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1eb45d640 1 -- start start 2024-04-25T00:01:24.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1eb45d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1e40b5240 0x7fc1e4156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1eb45d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1eb45d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1eb45d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e40b4da0 con 0x7fc1e4144910 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1eb45d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e40b37d0 con 0x7fc1e40b5240 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1eb45d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e40b4f20 con 0x7fc1e40b56b0 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1e99fa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1e99fa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44548/0 (socket says 172.21.15.113:44548) 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1e99fa640 1 -- 172.21.15.113:0/4139862491 learned_addr learned my addr 172.21.15.113:0/4139862491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1e91f9640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1e40b5240 0x7fc1e4156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.228+0000 7fc1e89f8640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1e99fa640 1 -- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 msgr2=0x7fc1e40c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1e99fa640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1e99fa640 1 -- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1e40b5240 msgr2=0x7fc1e4156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1e99fa640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1e40b5240 0x7fc1e4156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1e99fa640 1 -- 172.21.15.113:0/4139862491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e40ce6f0 con 0x7fc1e4144910 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1e89f8640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1e99fa640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40c5e40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00e7a0 tx=0x7fc1dc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1d9ffb640 1 -- 172.21.15.113:0/4139862491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc017070 con 0x7fc1e4144910 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1d9ffb640 1 -- 172.21.15.113:0/4139862491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1dc00c730 con 0x7fc1e4144910 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e40ce9d0 con 0x7fc1e4144910 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1d9ffb640 1 -- 172.21.15.113:0/4139862491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc00cb20 con 0x7fc1e4144910 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.229+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e40cef90 con 0x7fc1e4144910 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.230+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e40691a0 con 0x7fc1e4144910 2024-04-25T00:01:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.231+0000 7fc1d9ffb640 1 -- 172.21.15.113:0/4139862491 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1dc01a070 con 0x7fc1e4144910 2024-04-25T00:01:24.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.231+0000 7fc1d9ffb640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1c806be30 0x7fc1c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.231+0000 7fc1d9ffb640 1 -- 172.21.15.113:0/4139862491 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fc1dc08b340 con 0x7fc1e4144910 2024-04-25T00:01:24.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.231+0000 7fc1e91f9640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1c806be30 0x7fc1c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.231+0000 7fc1e91f9640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1c806be30 0x7fc1c806e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0033000 tx=0x7fc1e0009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.235+0000 7fc1d9ffb640 1 -- 172.21.15.113:0/4139862491 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc1dc0595d0 con 0x7fc1e4144910 2024-04-25T00:01:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.445+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1e4157050 con 0x7fc1e4144910 2024-04-25T00:01:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.445+0000 7fc1d9ffb640 1 -- 172.21.15.113:0/4139862491 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5191 (secure 0 0 0) 0x7fc1dc05d8d0 con 0x7fc1e4144910 2024-04-25T00:01:24.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:24.446 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:22.191795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:24.446 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-04-25T00:01:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1c806be30 msgr2=0x7fc1c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1c806be30 0x7fc1c806e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0033000 tx=0x7fc1e0009000 comp rx=0 tx=0).stop 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 msgr2=0x7fc1e40c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40c5e40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00e7a0 tx=0x7fc1dc00ec60 comp rx=0 tx=0).stop 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 shutdown_connections 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1c806be30 0x7fc1c806e2f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1e4144910 0x7fc1e40c5e40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1e40b56b0 0x7fc1e40c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 --2- 172.21.15.113:0/4139862491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1e40b5240 0x7fc1e4156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 >> 172.21.15.113:0/4139862491 conn(0x7fc1e40b1a50 msgr2=0x7fc1e4070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 shutdown_connections 2024-04-25T00:01:24.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.447+0000 7fc1eb45d640 1 -- 172.21.15.113:0/4139862491 wait complete. 2024-04-25T00:01:24.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:24.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.553+0000 7f851ca95640 1 Processor -- start 2024-04-25T00:01:24.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.553+0000 7f851ca95640 1 -- start start 2024-04-25T00:01:24.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.554+0000 7f851ca95640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8518144910 0x7f85180b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.554+0000 7f851ca95640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85180b5240 0x7f851806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.554+0000 7f851ca95640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8518144cf0 con 0x7f85180b5680 2024-04-25T00:01:24.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.554+0000 7f851ca95640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f85180b4510 con 0x7f85180b5240 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.554+0000 7f851ca95640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85180b2f30 con 0x7f8518144910 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f851659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8518144910 0x7f85180b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f8515d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85180b5240 0x7f851806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f851659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8518144910 0x7f85180b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:55826/0 (socket says 172.21.15.113:55826) 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f851659c640 1 -- 172.21.15.113:0/1103672566 learned_addr learned my addr 172.21.15.113:0/1103672566 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f8515d9b640 1 -- 172.21.15.113:0/1103672566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8518144910 msgr2=0x7f85180b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f8515d9b640 1 --2- 172.21.15.113:0/1103672566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8518144910 0x7f85180b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f8515d9b640 1 -- 172.21.15.113:0/1103672566 >> v1:172.21.15.178:6789/0 conn(0x7f85180b5680 legacy=0x7f85180cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.555+0000 7f8515d9b640 1 -- 172.21.15.113:0/1103672566 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85180b3210 con 0x7f85180b5240 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.556+0000 7f8515d9b640 1 --2- 172.21.15.113:0/1103672566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85180b5240 0x7f851806dfd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f85080097a0 tx=0x7f850802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e107288035ceedf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.556+0000 7f851559a640 1 -- 172.21.15.113:0/1103672566 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508037070 con 0x7f85180b5240 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.556+0000 7f851559a640 1 -- 172.21.15.113:0/1103672566 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8508032440 con 0x7f85180b5240 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.556+0000 7f851559a640 1 -- 172.21.15.113:0/1103672566 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508032810 con 0x7f85180b5240 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.556+0000 7f851ca95640 1 -- 172.21.15.113:0/1103672566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85180b5240 msgr2=0x7f851806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.556+0000 7f851ca95640 1 --2- 172.21.15.113:0/1103672566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85180b5240 0x7f851806dfd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f85080097a0 tx=0x7f850802a180 comp rx=0 tx=0).stop 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.557+0000 7f851ca95640 1 -- 172.21.15.113:0/1103672566 shutdown_connections 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.557+0000 7f851ca95640 1 --2- 172.21.15.113:0/1103672566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85180b5240 0x7f851806dfd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.557+0000 7f851ca95640 1 --2- 172.21.15.113:0/1103672566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8518144910 0x7f85180b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.557+0000 7f851ca95640 1 -- 172.21.15.113:0/1103672566 >> 172.21.15.113:0/1103672566 conn(0x7f85180b1a50 msgr2=0x7f85180b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:24.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.557+0000 7f851ca95640 1 -- 172.21.15.113:0/1103672566 shutdown_connections 2024-04-25T00:01:24.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.557+0000 7f851ca95640 1 -- 172.21.15.113:0/1103672566 wait complete. 2024-04-25T00:01:24.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.558+0000 7f851ca95640 1 Processor -- start 2024-04-25T00:01:24.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.558+0000 7f851ca95640 1 -- start start 2024-04-25T00:01:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85180b5240 0x7f85181383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85180b5680 0x7f851815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8518144910 0x7f85181f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85180b4da0 con 0x7f85180b5680 2024-04-25T00:01:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85180b37d0 con 0x7f8518144910 2024-04-25T00:01:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85180b4f20 con 0x7f85180b5240 2024-04-25T00:01:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85180b5680 0x7f851815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85180b5680 0x7f851815db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44552/0 (socket says 172.21.15.113:44552) 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 -- 172.21.15.113:0/1113664686 learned_addr learned my addr 172.21.15.113:0/1113664686 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8516d9d640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8518144910 0x7f85181f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851659c640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85180b5240 0x7f85181383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 -- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85180b5240 msgr2=0x7f85181383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85180b5240 0x7f85181383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 -- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8518144910 msgr2=0x7f85181f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8518144910 0x7f85181f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 -- 172.21.15.113:0/1113664686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851815ca80 con 0x7f85180b5680 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8516d9d640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8518144910 0x7f85181f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f8515d9b640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85180b5680 0x7f851815db30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8508035040 tx=0x7f85080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f850effd640 1 -- 172.21.15.113:0/1113664686 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508037040 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f850effd640 1 -- 172.21.15.113:0/1113664686 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8508030500 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851815cd60 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f850effd640 1 -- 172.21.15.113:0/1113664686 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508030910 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.559+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85181388f0 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.560+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85180691a0 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.561+0000 7f850effd640 1 -- 172.21.15.113:0/1113664686 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8508044020 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.561+0000 7f850effd640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84f806be30 0x7f84f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.561+0000 7f850effd640 1 -- 172.21.15.113:0/1113664686 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f85080ab100 con 0x7f85180b5680 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.561+0000 7f851659c640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84f806be30 0x7f84f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.562+0000 7f851659c640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84f806be30 0x7f84f806e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8500009b60 tx=0x7f8500008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.565+0000 7f850effd640 1 -- 172.21.15.113:0/1113664686 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8508079b00 con 0x7f85180b5680 2024-04-25T00:01:24.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.771+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85180b1810 con 0x7f85180b5680 2024-04-25T00:01:24.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.771+0000 7f850effd640 1 -- 172.21.15.113:0/1113664686 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5191 (secure 0 0 0) 0x7f8508079400 con 0x7f85180b5680 2024-04-25T00:01:24.771 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:24.772 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:00:49.512476+0000","modified":"2024-04-25T00:01:22.191795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6836/1715251400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1715251400},{"type":"v1","addr":"172.21.15.113:6836","nonce":1715251400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5556,"qdb_cluster":[5556]},"id":3}]} 2024-04-25T00:01:24.772 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-04-25T00:01:24.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84f806be30 msgr2=0x7f84f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84f806be30 0x7f84f806e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8500009b60 tx=0x7f8500008040 comp rx=0 tx=0).stop 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85180b5680 msgr2=0x7f851815db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85180b5680 0x7f851815db30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8508035040 tx=0x7f85080046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 shutdown_connections 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f84f806be30 0x7f84f806e2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8518144910 0x7f85181f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85180b5680 0x7f851815db30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 --2- 172.21.15.113:0/1113664686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85180b5240 0x7f85181383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 >> 172.21.15.113:0/1113664686 conn(0x7f85180b1a50 msgr2=0x7f85180af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 shutdown_connections 2024-04-25T00:01:24.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.773+0000 7f851ca95640 1 -- 172.21.15.113:0/1113664686 wait complete. 2024-04-25T00:01:24.784 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:24.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.880+0000 7f25d6fc8640 1 Processor -- start 2024-04-25T00:01:24.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.880+0000 7f25d6fc8640 1 -- start start 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d6fc8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25d0144d60 0x7f25d00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d6fc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d0071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d6fc8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d00cfad0 con 0x7f25d01453c0 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d6fc8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f25d0145c40 con 0x7f25d0144d60 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d6fc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f25d00cf230 con 0x7f25d006e6e0 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d4d64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25d0144d60 0x7f25d00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d4d64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25d0144d60 0x7f25d00b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44566/0 (socket says 172.21.15.113:44566) 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d4d64640 1 -- 172.21.15.113:0/3140674890 learned_addr learned my addr 172.21.15.113:0/3140674890 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d5565640 1 --2- 172.21.15.113:0/3140674890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d4d64640 1 -- 172.21.15.113:0/3140674890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 msgr2=0x7f25d0071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d4d64640 1 --2- 172.21.15.113:0/3140674890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d0071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d4d64640 1 -- 172.21.15.113:0/3140674890 >> v1:172.21.15.178:6789/0 conn(0x7f25d01453c0 legacy=0x7f25d006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d4d64640 1 -- 172.21.15.113:0/3140674890 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d00cf510 con 0x7f25d0144d60 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.881+0000 7f25d5565640 1 --2- 172.21.15.113:0/3140674890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d0071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.882+0000 7f25d4d64640 1 --2- 172.21.15.113:0/3140674890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d00b5390 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009870 tx=0x7f25c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35853384a1de6bff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.882+0000 7f25cf7fe640 1 -- 172.21.15.113:0/3140674890 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4037070 con 0x7f25d0144d60 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.882+0000 7f25cf7fe640 1 -- 172.21.15.113:0/3140674890 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c4032440 con 0x7f25d0144d60 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.882+0000 7f25cf7fe640 1 -- 172.21.15.113:0/3140674890 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4032810 con 0x7f25d0144d60 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.882+0000 7f25d6fc8640 1 -- 172.21.15.113:0/3140674890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 msgr2=0x7f25d00b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.882+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/3140674890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d00b5390 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009870 tx=0x7f25c402a180 comp rx=0 tx=0).stop 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.883+0000 7f25d6fc8640 1 -- 172.21.15.113:0/3140674890 shutdown_connections 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.883+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/3140674890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d0071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.883+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/3140674890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d00b5390 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.883+0000 7f25d6fc8640 1 -- 172.21.15.113:0/3140674890 >> 172.21.15.113:0/3140674890 conn(0x7f25d00b1a50 msgr2=0x7f25d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:24.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.883+0000 7f25d6fc8640 1 -- 172.21.15.113:0/3140674890 shutdown_connections 2024-04-25T00:01:24.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.883+0000 7f25d6fc8640 1 -- 172.21.15.113:0/3140674890 wait complete. 2024-04-25T00:01:24.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.884+0000 7f25d6fc8640 1 Processor -- start 2024-04-25T00:01:24.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.884+0000 7f25d6fc8640 1 -- start start 2024-04-25T00:01:24.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d01565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d013c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d01453c0 0x7f25d015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0145140 con 0x7f25d0144d60 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0070e10 con 0x7f25d01453c0 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f25d0071870 con 0x7f25d006e6e0 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d013c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d013c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44574/0 (socket says 172.21.15.113:44574) 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 -- 172.21.15.113:0/1411089813 learned_addr learned my addr 172.21.15.113:0/1411089813 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d4d64640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d01565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d5565640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d01453c0 0x7f25d015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 -- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 msgr2=0x7f25d01565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d01565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 -- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d01453c0 msgr2=0x7f25d015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d01453c0 0x7f25d015add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 -- 172.21.15.113:0/1411089813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d00b3080 con 0x7f25d0144d60 2024-04-25T00:01:24.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d4d64640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d01565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d5565640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d01453c0 0x7f25d015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cffff640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d013c010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b3f0 tx=0x7f25bc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cd7fa640 1 -- 172.21.15.113:0/1411089813 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc015020 con 0x7f25d0144d60 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cd7fa640 1 -- 172.21.15.113:0/1411089813 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25bc011830 con 0x7f25d0144d60 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d00b3320 con 0x7f25d0144d60 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25cd7fa640 1 -- 172.21.15.113:0/1411089813 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011c20 con 0x7f25d0144d60 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.885+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d00b38e0 con 0x7f25d0144d60 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.886+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d00691a0 con 0x7f25d0144d60 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.887+0000 7f25cd7fa640 1 -- 172.21.15.113:0/1411089813 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25bc018020 con 0x7f25d0144d60 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.887+0000 7f25cd7fa640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25b406be30 0x7f25b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.887+0000 7f25d4d64640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25b406be30 0x7f25b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:24.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.887+0000 7f25cd7fa640 1 -- 172.21.15.113:0/1411089813 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f25bc08ada0 con 0x7f25d0144d60 2024-04-25T00:01:24.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.888+0000 7f25d4d64640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25b406be30 0x7f25b406e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f25c4034000 tx=0x7f25c402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:24.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:24.892+0000 7f25cd7fa640 1 -- 172.21.15.113:0/1411089813 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25bc01f020 con 0x7f25d0144d60 2024-04-25T00:01:25.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.065+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25d013c550 con 0x7f25d0144d60 2024-04-25T00:01:25.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.066+0000 7f25cd7fa640 1 -- 172.21.15.113:0/1411089813 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16812 (secure 0 0 0) 0x7f25bc0172c0 con 0x7f25d0144d60 2024-04-25T00:01:25.066 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:25.067 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":48,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:01.211634+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:47.724388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:48.476831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.16","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25b406be30 msgr2=0x7f25b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25b406be30 0x7f25b406e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f25c4034000 tx=0x7f25c402e000 comp rx=0 tx=0).stop 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 msgr2=0x7f25d013c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d013c010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b3f0 tx=0x7f25bc00b8b0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 shutdown_connections 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25b406be30 0x7f25b406e2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d01453c0 0x7f25d015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d0144d60 0x7f25d013c010 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 --2- 172.21.15.113:0/1411089813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d006e6e0 0x7f25d01565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 >> 172.21.15.113:0/1411089813 conn(0x7f25d00b1a50 msgr2=0x7f25d00cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 shutdown_connections 2024-04-25T00:01:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.068+0000 7f25d6fc8640 1 -- 172.21.15.113:0/1411089813 wait complete. 2024-04-25T00:01:25.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:01:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.179+0000 7f7536b34640 1 Processor -- start 2024-04-25T00:01:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.179+0000 7f7536b34640 1 -- start start 2024-04-25T00:01:25.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.179+0000 7f7536b34640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75300d06c0 0x7f753015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.179+0000 7f7536b34640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75301604d0 0x7f75300cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.179+0000 7f7536b34640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f753015d6b0 con 0x7f753013f050 2024-04-25T00:01:25.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.179+0000 7f7536b34640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f753015c6a0 con 0x7f75301604d0 2024-04-25T00:01:25.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.179+0000 7f7536b34640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f753015c8f0 con 0x7f75300d06c0 2024-04-25T00:01:25.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75301604d0 0x7f75300cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75301604d0 0x7f75300cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44578/0 (socket says 172.21.15.113:44578) 2024-04-25T00:01:25.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 -- 172.21.15.113:0/799002714 learned_addr learned my addr 172.21.15.113:0/799002714 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:25.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f752ffff640 1 --2- 172.21.15.113:0/799002714 >> v2:172.21.15.178:3300/0 conn(0x7f75300d06c0 0x7f753015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 -- 172.21.15.113:0/799002714 >> v2:172.21.15.178:3300/0 conn(0x7f75300d06c0 msgr2=0x7f753015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:25.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 --2- 172.21.15.113:0/799002714 >> v2:172.21.15.178:3300/0 conn(0x7f75300d06c0 0x7f753015ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 -- 172.21.15.113:0/799002714 >> v1:172.21.15.178:6789/0 conn(0x7f753013f050 legacy=0x7f75300cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:25.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 -- 172.21.15.113:0/799002714 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753015cbd0 con 0x7f75301604d0 2024-04-25T00:01:25.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.180+0000 7f75350d1640 1 --2- 172.21.15.113:0/799002714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75301604d0 0x7f75300cea60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f753006f490 tx=0x7f752802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3ce684fd920745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f752f7fe640 1 -- 172.21.15.113:0/799002714 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528036070 con 0x7f75301604d0 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f752f7fe640 1 -- 172.21.15.113:0/799002714 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7528031600 con 0x7f75301604d0 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f752f7fe640 1 -- 172.21.15.113:0/799002714 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75280319d0 con 0x7f75301604d0 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 -- 172.21.15.113:0/799002714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75301604d0 msgr2=0x7f75300cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 --2- 172.21.15.113:0/799002714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75301604d0 0x7f75300cea60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f753006f490 tx=0x7f752802a660 comp rx=0 tx=0).stop 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 -- 172.21.15.113:0/799002714 shutdown_connections 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 --2- 172.21.15.113:0/799002714 >> v2:172.21.15.178:3300/0 conn(0x7f75300d06c0 0x7f753015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 --2- 172.21.15.113:0/799002714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75301604d0 0x7f75300cea60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 -- 172.21.15.113:0/799002714 >> 172.21.15.113:0/799002714 conn(0x7f753006b0f0 msgr2=0x7f753006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 -- 172.21.15.113:0/799002714 shutdown_connections 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.181+0000 7f7536b34640 1 -- 172.21.15.113:0/799002714 wait complete. 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.182+0000 7f7536b34640 1 Processor -- start 2024-04-25T00:01:25.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.182+0000 7f7536b34640 1 -- start start 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f7536b34640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75300d06c0 0x7f7530147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f7536b34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f753013f050 0x7f753014ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f7536b34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75301604d0 0x7f75301f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f7536b34640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f753013f430 con 0x7f75300d06c0 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f7536b34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f753015d190 con 0x7f753013f050 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f7536b34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7530160960 con 0x7f75301604d0 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f75348d0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75300d06c0 0x7f7530147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f75348d0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75300d06c0 0x7f7530147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44582/0 (socket says 172.21.15.113:44582) 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.183+0000 7f75348d0640 1 -- 172.21.15.113:0/4036968658 learned_addr learned my addr 172.21.15.113:0/4036968658 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:25.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f752ffff640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f753013f050 0x7f753014ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f75350d1640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75301604d0 0x7f75301f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f75348d0640 1 -- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75301604d0 msgr2=0x7f75301f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f75348d0640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75301604d0 0x7f75301f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f75348d0640 1 -- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f753013f050 msgr2=0x7f753014ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f75348d0640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f753013f050 0x7f753014ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f75348d0640 1 -- 172.21.15.113:0/4036968658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75301f3010 con 0x7f75300d06c0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f752ffff640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f753013f050 0x7f753014ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f75348d0640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75300d06c0 0x7f7530147ad0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f752400c430 tx=0x7f752400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f752d7fa640 1 -- 172.21.15.113:0/4036968658 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524011790 con 0x7f75300d06c0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f752d7fa640 1 -- 172.21.15.113:0/4036968658 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524017070 con 0x7f75300d06c0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753014e490 con 0x7f75300d06c0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f752d7fa640 1 -- 172.21.15.113:0/4036968658 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524011bb0 con 0x7f75300d06c0 2024-04-25T00:01:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.184+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753014ea50 con 0x7f75300d06c0 2024-04-25T00:01:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.185+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75300691a0 con 0x7f75300d06c0 2024-04-25T00:01:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.185+0000 7f752d7fa640 1 -- 172.21.15.113:0/4036968658 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f752401c460 con 0x7f75300d06c0 2024-04-25T00:01:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.186+0000 7f752d7fa640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f751406be60 0x7f751406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.186+0000 7f752d7fa640 1 -- 172.21.15.113:0/4036968658 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f752408aca0 con 0x7f75300d06c0 2024-04-25T00:01:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.186+0000 7f752ffff640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f751406be60 0x7f751406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.186+0000 7f752ffff640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f751406be60 0x7f751406e320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7530148310 tx=0x7f751c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.190+0000 7f752d7fa640 1 -- 172.21.15.113:0/4036968658 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75301f3010 con 0x7f75300d06c0 2024-04-25T00:01:25.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.401+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f75300d0e50 con 0x7f75300d06c0 2024-04-25T00:01:25.401 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:01:25.402+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:01:25.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.403+0000 7f752d7fa640 1 -- 172.21.15.113:0/4036968658 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f75240596a0 con 0x7f75300d06c0 2024-04-25T00:01:25.403 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:01:25.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f751406be60 msgr2=0x7f751406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f751406be60 0x7f751406e320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7530148310 tx=0x7f751c008040 comp rx=0 tx=0).stop 2024-04-25T00:01:25.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75300d06c0 msgr2=0x7f7530147ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75300d06c0 0x7f7530147ad0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f752400c430 tx=0x7f752400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 shutdown_connections 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f751406be60 0x7f751406e320 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f753013f050 0x7f753014ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75301604d0 0x7f75301f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 --2- 172.21.15.113:0/4036968658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75300d06c0 0x7f7530147ad0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 >> 172.21.15.113:0/4036968658 conn(0x7f753006b0f0 msgr2=0x7f753013d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 shutdown_connections 2024-04-25T00:01:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.407+0000 7f7536b34640 1 -- 172.21.15.113:0/4036968658 wait complete. 2024-04-25T00:01:25.418 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:01:25.444 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.518+0000 7fdb4135a640 1 Processor -- start 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.518+0000 7fdb4135a640 1 -- start start 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb4135a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdb3c161fd0 0x7fdb3c06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb4135a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb3c15ea00 con 0x7fdb3c061b80 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb4135a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb3c162490 con 0x7fdb3c0cd4b0 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb4135a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdb3c0cfc30 con 0x7fdb3c161fd0 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb3b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdb3c061b80 0x7fdb3c15eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34268/0 (socket says 172.21.15.113:34268) 2024-04-25T00:01:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb3b7fe640 1 -- 172.21.15.113:0/1519350093 learned_addr learned my addr 172.21.15.113:0/1519350093 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb3a7fc640 1 --2- 172.21.15.113:0/1519350093 >> v2:172.21.15.178:3300/0 conn(0x7fdb3c161fd0 0x7fdb3c06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3335957489 0 0) 0x7fdb3c15ea00 con 0x7fdb3c061b80 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb24003520 con 0x7fdb3c061b80 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.519+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285976404 0 0) 0x7fdb3c162490 con 0x7fdb3c0cd4b0 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.520+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb3c15ea00 con 0x7fdb3c0cd4b0 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.520+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 533496743 0 0) 0x7fdb24003520 con 0x7fdb3c061b80 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.520+0000 7fdb3a7fc640 1 -- 172.21.15.113:0/1519350093 >> v1:172.21.15.178:6789/0 conn(0x7fdb3c0cd4b0 legacy=0x7fdb3c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.520+0000 7fdb3a7fc640 1 -- 172.21.15.113:0/1519350093 >> v1:172.21.15.113:6789/0 conn(0x7fdb3c061b80 legacy=0x7fdb3c15eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.520+0000 7fdb3a7fc640 1 -- 172.21.15.113:0/1519350093 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c162770 con 0x7fdb3c161fd0 2024-04-25T00:01:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.520+0000 7fdb3a7fc640 1 --2- 172.21.15.113:0/1519350093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c06c210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009790 tx=0x7fdb3002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb6a1bab78f3833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30037070 con 0x7fdb3c161fd0 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3002ac30 con 0x7fdb3c161fd0 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb39ffb640 1 -- 172.21.15.113:0/1519350093 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb340019f0 con 0x7fdb3c161fd0 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb4135a640 1 -- 172.21.15.113:0/1519350093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 msgr2=0x7fdb3c06c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1519350093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c06c210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009790 tx=0x7fdb3002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb4135a640 1 -- 172.21.15.113:0/1519350093 shutdown_connections 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1519350093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c06c210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb4135a640 1 -- 172.21.15.113:0/1519350093 >> 172.21.15.113:0/1519350093 conn(0x7fdb3c0d0e30 msgr2=0x7fdb3c0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb4135a640 1 -- 172.21.15.113:0/1519350093 shutdown_connections 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.521+0000 7fdb4135a640 1 -- 172.21.15.113:0/1519350093 wait complete. 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.522+0000 7fdb4135a640 1 Processor -- start 2024-04-25T00:01:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.522+0000 7fdb4135a640 1 -- start start 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb4135a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb3c061b80 0x7fdb3c0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb4135a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb3c0cd4b0 0x7fdb3c1498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb4135a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb4135a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c0cf1d0 con 0x7fdb3c061b80 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb4135a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c162d30 con 0x7fdb3c161fd0 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb4135a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3c15f5f0 con 0x7fdb3c0cd4b0 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb3c061b80 0x7fdb3c0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb3c061b80 0x7fdb3c0b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44590/0 (socket says 172.21.15.113:44590) 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3affd640 1 -- 172.21.15.113:0/1382974993 learned_addr learned my addr 172.21.15.113:0/1382974993 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3b7fe640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3a7fc640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb3c0cd4b0 0x7fdb3c1498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3affd640 1 -- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb3c0cd4b0 msgr2=0x7fdb3c1498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3affd640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb3c0cd4b0 0x7fdb3c1498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3affd640 1 -- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 msgr2=0x7fdb3c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.523+0000 7fdb3affd640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb3affd640 1 -- 172.21.15.113:0/1382974993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c0afbc0 con 0x7fdb3c061b80 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb3b7fe640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb3a7fc640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb3c0cd4b0 0x7fdb3c1498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb3affd640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb3c061b80 0x7fdb3c0b5e90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00c4c0 tx=0x7fdb2c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb237fe640 1 -- 172.21.15.113:0/1382974993 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0052d0 con 0x7fdb3c061b80 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb237fe640 1 -- 172.21.15.113:0/1382974993 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2c005470 con 0x7fdb3c061b80 2024-04-25T00:01:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c0af320 con 0x7fdb3c061b80 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb237fe640 1 -- 172.21.15.113:0/1382974993 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c012870 con 0x7fdb3c061b80 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.524+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c0af8e0 con 0x7fdb3c061b80 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.525+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3c0691a0 con 0x7fdb3c061b80 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.525+0000 7fdb237fe640 1 -- 172.21.15.113:0/1382974993 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb2c012a10 con 0x7fdb3c061b80 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.526+0000 7fdb237fe640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb2806be30 0x7fdb2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.526+0000 7fdb237fe640 1 -- 172.21.15.113:0/1382974993 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7fdb2c08ac20 con 0x7fdb3c061b80 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.526+0000 7fdb3a7fc640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb2806be30 0x7fdb2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.526+0000 7fdb3a7fc640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb2806be30 0x7fdb2806e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdb30034000 tx=0x7fdb3002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.530+0000 7fdb237fe640 1 -- 172.21.15.113:0/1382974993 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdb2c0594b0 con 0x7fdb3c061b80 2024-04-25T00:01:25.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:25.739+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdb3c0b27d0 con 0x7fdb3c061b80 2024-04-25T00:01:26.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.405+0000 7fdb237fe640 1 -- 172.21.15.113:0/1382974993 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fdb2c058db0 con 0x7fdb3c061b80 2024-04-25T00:01:26.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb2806be30 msgr2=0x7fdb2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:26.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb2806be30 0x7fdb2806e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdb30034000 tx=0x7fdb3002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:26.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb3c061b80 msgr2=0x7fdb3c0b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb3c061b80 0x7fdb3c0b5e90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00c4c0 tx=0x7fdb2c00c980 comp rx=0 tx=0).stop 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 shutdown_connections 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb2806be30 0x7fdb2806e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb3c161fd0 0x7fdb3c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb3c0cd4b0 0x7fdb3c1498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 --2- 172.21.15.113:0/1382974993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb3c061b80 0x7fdb3c0b5e90 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 >> 172.21.15.113:0/1382974993 conn(0x7fdb3c0d0e30 msgr2=0x7fdb3c15de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 shutdown_connections 2024-04-25T00:01:26.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.407+0000 7fdb4135a640 1 -- 172.21.15.113:0/1382974993 wait complete. 2024-04-25T00:01:26.420 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:01:26.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.562+0000 7fd52c210640 1 Processor -- start 2024-04-25T00:01:26.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.562+0000 7fd52c210640 1 -- start start 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd52c210640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd52415e510 0x7fd524161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd52c210640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd5240b9bb0 0x7fd5240b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd52c210640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5241626f0 con 0x7fd5240b61e0 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd52c210640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd5241629d0 con 0x7fd5240b9bb0 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd52c210640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd524162cb0 con 0x7fd52415e510 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd529fac640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd52415e510 0x7fd524161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd5297ab640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd5240b9bb0 0x7fd5240b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:26.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd5297ab640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd5240b9bb0 0x7fd5240b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:53200/0 (socket says 172.21.15.113:53200) 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd5297ab640 1 -- 172.21.15.113:0/451465393 learned_addr learned my addr 172.21.15.113:0/451465393 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd52a7ad640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd5240b61e0 0x7fd524071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:50582/0 (socket says 172.21.15.113:50582) 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.563+0000 7fd529fac640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd52415e510 0x7fd524161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:55918/0 (socket says 172.21.15.113:55918) 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.564+0000 7fd5297ab640 1 -- 172.21.15.113:0/451465393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd52415e510 msgr2=0x7fd524161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.564+0000 7fd5297ab640 1 --2- 172.21.15.113:0/451465393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd52415e510 0x7fd524161ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.564+0000 7fd5297ab640 1 -- 172.21.15.113:0/451465393 >> v1:172.21.15.178:6789/0 conn(0x7fd5240b61e0 legacy=0x7fd524071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.564+0000 7fd5297ab640 1 -- 172.21.15.113:0/451465393 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5240d2660 con 0x7fd5240b9bb0 2024-04-25T00:01:26.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.564+0000 7fd528faa640 1 -- 172.21.15.113:0/451465393 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639706185 0 0) 0x7fd5241626f0 con 0x7fd5240b61e0 2024-04-25T00:01:26.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.564+0000 7fd5297ab640 1 --2- 172.21.15.113:0/451465393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5240b9bb0 0x7fd5240b5ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd514009870 tx=0x7fd51402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbdd6c8d3c93fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:26.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd528faa640 1 -- 172.21.15.113:0/451465393 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514037070 con 0x7fd5240b9bb0 2024-04-25T00:01:26.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd528faa640 1 -- 172.21.15.113:0/451465393 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd514032440 con 0x7fd5240b9bb0 2024-04-25T00:01:26.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd528faa640 1 -- 172.21.15.113:0/451465393 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514032810 con 0x7fd5240b9bb0 2024-04-25T00:01:26.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 -- 172.21.15.113:0/451465393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5240b9bb0 msgr2=0x7fd5240b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 --2- 172.21.15.113:0/451465393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5240b9bb0 0x7fd5240b5ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd514009870 tx=0x7fd51402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 -- 172.21.15.113:0/451465393 shutdown_connections 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 --2- 172.21.15.113:0/451465393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5240b9bb0 0x7fd5240b5ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 --2- 172.21.15.113:0/451465393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd52415e510 0x7fd524161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 -- 172.21.15.113:0/451465393 >> 172.21.15.113:0/451465393 conn(0x7fd52406b720 msgr2=0x7fd5240d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 -- 172.21.15.113:0/451465393 shutdown_connections 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.565+0000 7fd52c210640 1 -- 172.21.15.113:0/451465393 wait complete. 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.566+0000 7fd52c210640 1 Processor -- start 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.566+0000 7fd52c210640 1 -- start start 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd52c210640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5240b61e0 0x7fd524154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd52c210640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5240b9bb0 0x7fd5240ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd52c210640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd52415e510 0x7fd52414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd52c210640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd5240d3e70 con 0x7fd5240b61e0 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd52c210640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5241623e0 con 0x7fd52415e510 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd52c210640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5240b73c0 con 0x7fd5240b9bb0 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd529fac640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5240b61e0 0x7fd524154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd529fac640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5240b61e0 0x7fd524154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44602/0 (socket says 172.21.15.113:44602) 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd529fac640 1 -- 172.21.15.113:0/2585656998 learned_addr learned my addr 172.21.15.113:0/2585656998 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd52a7ad640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd52415e510 0x7fd52414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.567+0000 7fd5297ab640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5240b9bb0 0x7fd5240ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd529fac640 1 -- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5240b9bb0 msgr2=0x7fd5240ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd529fac640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5240b9bb0 0x7fd5240ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd529fac640 1 -- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd52415e510 msgr2=0x7fd52414d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd529fac640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd52415e510 0x7fd52414d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd529fac640 1 -- 172.21.15.113:0/2585656998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5240caab0 con 0x7fd5240b61e0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd52a7ad640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd52415e510 0x7fd52414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd529fac640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5240b61e0 0x7fd524154500 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd52000e870 tx=0x7fd52000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd51a7fc640 1 -- 172.21.15.113:0/2585656998 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520017070 con 0x7fd5240b61e0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd51a7fc640 1 -- 172.21.15.113:0/2585656998 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd52000c8e0 con 0x7fd5240b61e0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52415ec00 con 0x7fd5240b61e0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd51a7fc640 1 -- 172.21.15.113:0/2585656998 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52000ccd0 con 0x7fd5240b61e0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.568+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52415f1c0 con 0x7fd5240b61e0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.569+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5240691a0 con 0x7fd5240b61e0 2024-04-25T00:01:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.569+0000 7fd51a7fc640 1 -- 172.21.15.113:0/2585656998 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd52001a070 con 0x7fd5240b61e0 2024-04-25T00:01:26.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.570+0000 7fd51a7fc640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be60 0x7fd50806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:26.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.570+0000 7fd51a7fc640 1 -- 172.21.15.113:0/2585656998 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7fd52008b1d0 con 0x7fd5240b61e0 2024-04-25T00:01:26.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.570+0000 7fd5297ab640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be60 0x7fd50806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:26.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.570+0000 7fd5297ab640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be60 0x7fd50806e320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd514034000 tx=0x7fd51402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:26.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.574+0000 7fd51a7fc640 1 -- 172.21.15.113:0/2585656998 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd5200599e0 con 0x7fd5240b61e0 2024-04-25T00:01:26.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:26.747+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd5240d1f30 con 0x7fd5240b61e0 2024-04-25T00:01:27.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.408+0000 7fd51a7fc640 1 -- 172.21.15.113:0/2585656998 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) ==== 181+0+0 (secure 0 0 0) 0x7fd5200592e0 con 0x7fd5240b61e0 2024-04-25T00:01:27.409 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:01:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be60 msgr2=0x7fd50806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be60 0x7fd50806e320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd514034000 tx=0x7fd51402e000 comp rx=0 tx=0).stop 2024-04-25T00:01:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5240b61e0 msgr2=0x7fd524154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5240b61e0 0x7fd524154500 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd52000e870 tx=0x7fd52000ed30 comp rx=0 tx=0).stop 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 shutdown_connections 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be60 0x7fd50806e320 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd52415e510 0x7fd52414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5240b9bb0 0x7fd5240ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 --2- 172.21.15.113:0/2585656998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5240b61e0 0x7fd524154500 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 >> 172.21.15.113:0/2585656998 conn(0x7fd52406b720 msgr2=0x7fd5240b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 shutdown_connections 2024-04-25T00:01:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.413+0000 7fd52c210640 1 -- 172.21.15.113:0/2585656998 wait complete. 2024-04-25T00:01:27.426 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:01:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.527+0000 7fb78dfe7640 1 Processor -- start 2024-04-25T00:01:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.527+0000 7fb78dfe7640 1 -- start start 2024-04-25T00:01:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb78dfe7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb7880b47a0 0x7fb78815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb78dfe7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880cc0b0 0x7fb788154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb78dfe7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78815bcf0 con 0x7fb7880b4b80 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb78dfe7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb7880b2f40 con 0x7fb7880b47a0 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb78dfe7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880b31e0 con 0x7fb7880cc0b0 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb786ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880cc0b0 0x7fb788154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb7877fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb7880b47a0 0x7fb78815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb786ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880cc0b0 0x7fb788154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:40702/0 (socket says 172.21.15.113:40702) 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb786ffd640 1 -- 172.21.15.113:0/705946939 learned_addr learned my addr 172.21.15.113:0/705946939 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb786ffd640 1 -- 172.21.15.113:0/705946939 >> v2:172.21.15.178:3300/0 conn(0x7fb7880b47a0 msgr2=0x7fb78815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb786ffd640 1 --2- 172.21.15.113:0/705946939 >> v2:172.21.15.178:3300/0 conn(0x7fb7880b47a0 0x7fb78815b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb786ffd640 1 -- 172.21.15.113:0/705946939 >> v1:172.21.15.178:6789/0 conn(0x7fb7880b4b80 legacy=0x7fb78815bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb786ffd640 1 -- 172.21.15.113:0/705946939 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7880b34c0 con 0x7fb7880cc0b0 2024-04-25T00:01:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.528+0000 7fb7877fe640 1 --2- 172.21.15.113:0/705946939 >> v2:172.21.15.178:3300/0 conn(0x7fb7880b47a0 0x7fb78815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.529+0000 7fb786ffd640 1 --2- 172.21.15.113:0/705946939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880cc0b0 0x7fb788154830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb778009870 tx=0x7fb77802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19acedcb69ac3b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.529+0000 7fb7867fc640 1 -- 172.21.15.113:0/705946939 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778037070 con 0x7fb7880cc0b0 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.529+0000 7fb7867fc640 1 -- 172.21.15.113:0/705946939 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb778032440 con 0x7fb7880cc0b0 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.529+0000 7fb7867fc640 1 -- 172.21.15.113:0/705946939 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778032810 con 0x7fb7880cc0b0 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 -- 172.21.15.113:0/705946939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880cc0b0 msgr2=0x7fb788154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/705946939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880cc0b0 0x7fb788154830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb778009870 tx=0x7fb77802a270 comp rx=0 tx=0).stop 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 -- 172.21.15.113:0/705946939 shutdown_connections 2024-04-25T00:01:27.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/705946939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880cc0b0 0x7fb788154830 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/705946939 >> v2:172.21.15.178:3300/0 conn(0x7fb7880b47a0 0x7fb78815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 -- 172.21.15.113:0/705946939 >> 172.21.15.113:0/705946939 conn(0x7fb7880b1b30 msgr2=0x7fb7880b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:27.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 -- 172.21.15.113:0/705946939 shutdown_connections 2024-04-25T00:01:27.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.530+0000 7fb78dfe7640 1 -- 172.21.15.113:0/705946939 wait complete. 2024-04-25T00:01:27.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.531+0000 7fb78dfe7640 1 Processor -- start 2024-04-25T00:01:27.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.531+0000 7fb78dfe7640 1 -- start start 2024-04-25T00:01:27.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb78dfe7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880b47a0 0x7fb78814bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:27.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb78dfe7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880b4b80 0x7fb78815c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:27.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb78dfe7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7880cc0b0 0x7fb7881f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:27.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb78dfe7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880b3f80 con 0x7fb7880cc0b0 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb78dfe7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880b3a80 con 0x7fb7880b4b80 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb78dfe7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880b4100 con 0x7fb7880b47a0 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7880cc0b0 0x7fb7881f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7880cc0b0 0x7fb7881f01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38722/0 (socket says 172.21.15.113:38722) 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 -- 172.21.15.113:0/4118529842 learned_addr learned my addr 172.21.15.113:0/4118529842 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb7877fe640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880b47a0 0x7fb78814bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb786ffd640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880b4b80 0x7fb78815c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 -- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880b47a0 msgr2=0x7fb78814bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880b47a0 0x7fb78814bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 -- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880b4b80 msgr2=0x7fb78815c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880b4b80 0x7fb78815c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 -- 172.21.15.113:0/4118529842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78814c4c0 con 0x7fb7880cc0b0 2024-04-25T00:01:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb786ffd640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880b4b80 0x7fb78815c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.532+0000 7fb787fff640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7880cc0b0 0x7fb7881f01f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c7d0 tx=0x7fb78000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.533+0000 7fb76ffff640 1 -- 172.21.15.113:0/4118529842 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780003b70 con 0x7fb7880cc0b0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.533+0000 7fb76ffff640 1 -- 172.21.15.113:0/4118529842 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780003d30 con 0x7fb7880cc0b0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.533+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78815d3a0 con 0x7fb7880cc0b0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.533+0000 7fb76ffff640 1 -- 172.21.15.113:0/4118529842 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78000aa90 con 0x7fb7880cc0b0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.533+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78815d960 con 0x7fb7880cc0b0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.534+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7880691a0 con 0x7fb7880cc0b0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.534+0000 7fb76ffff640 1 -- 172.21.15.113:0/4118529842 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb78000ac30 con 0x7fb7880cc0b0 2024-04-25T00:01:27.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.534+0000 7fb76ffff640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb77406be30 0x7fb77406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:27.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.534+0000 7fb76ffff640 1 -- 172.21.15.113:0/4118529842 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5415+0+0 (secure 0 0 0) 0x7fb78008ad10 con 0x7fb7880cc0b0 2024-04-25T00:01:27.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.534+0000 7fb7877fe640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb77406be30 0x7fb77406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:27.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.535+0000 7fb7877fe640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb77406be30 0x7fb77406e2f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0045b0 tx=0x7fb77c00f040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:27.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.539+0000 7fb76ffff640 1 -- 172.21.15.113:0/4118529842 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb78815bcf0 con 0x7fb7880cc0b0 2024-04-25T00:01:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:27.716+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb788072da0 con 0x7fb7880cc0b0 2024-04-25T00:01:28.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.409+0000 7fb76ffff640 1 -- 172.21.15.113:0/4118529842 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) ==== 169+0+0 (secure 0 0 0) 0x7fb780059760 con 0x7fb7880cc0b0 2024-04-25T00:01:28.410 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:01:28.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.416+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb77406be30 msgr2=0x7fb77406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.416+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb77406be30 0x7fb77406e2f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0045b0 tx=0x7fb77c00f040 comp rx=0 tx=0).stop 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.416+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7880cc0b0 msgr2=0x7fb7881f01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.416+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7880cc0b0 0x7fb7881f01f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c7d0 tx=0x7fb78000cc90 comp rx=0 tx=0).stop 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.417+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 shutdown_connections 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.417+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb77406be30 0x7fb77406e2f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.417+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880b4b80 0x7fb78815c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.417+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7880cc0b0 0x7fb7881f01f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.417+0000 7fb78dfe7640 1 --2- 172.21.15.113:0/4118529842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7880b47a0 0x7fb78814bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.417+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 >> 172.21.15.113:0/4118529842 conn(0x7fb7880b1b30 msgr2=0x7fb78814cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:28.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.419+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 shutdown_connections 2024-04-25T00:01:28.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.419+0000 7fb78dfe7640 1 -- 172.21.15.113:0/4118529842 wait complete. 2024-04-25T00:01:28.438 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:01:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.538+0000 7fb2a1879640 1 Processor -- start 2024-04-25T00:01:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.538+0000 7fb2a1879640 1 -- start start 2024-04-25T00:01:28.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb2a1879640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb29c0b5ca0 0x7fb29c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb2a1879640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 0x7fb29c0b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb2a1879640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb29c0d2e00 con 0x7fb29c15e510 2024-04-25T00:01:28.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb2a1879640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb29c0b6640 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb2a1879640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c0d30e0 con 0x7fb29c0b73c0 2024-04-25T00:01:28.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb29a7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb29c0b5ca0 0x7fb29c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb29a7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb29c0b5ca0 0x7fb29c0b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38742/0 (socket says 172.21.15.113:38742) 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb29a7fc640 1 -- 172.21.15.113:0/3966417314 learned_addr learned my addr 172.21.15.113:0/3966417314 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb29a7fc640 1 -- 172.21.15.113:0/3966417314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 msgr2=0x7fb29c0b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb29a7fc640 1 --2- 172.21.15.113:0/3966417314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 0x7fb29c0b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb29a7fc640 1 -- 172.21.15.113:0/3966417314 >> v1:172.21.15.178:6789/0 conn(0x7fb29c15e510 legacy=0x7fb29c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.539+0000 7fb29a7fc640 1 -- 172.21.15.113:0/3966417314 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c0d33c0 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb29a7fc640 1 --2- 172.21.15.113:0/3966417314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c0b6100 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb290009870 tx=0x7fb29002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3757b1e6df635dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb299ffb640 1 -- 172.21.15.113:0/3966417314 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290037070 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb299ffb640 1 -- 172.21.15.113:0/3966417314 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290032440 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb299ffb640 1 -- 172.21.15.113:0/3966417314 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290032810 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb2a1879640 1 -- 172.21.15.113:0/3966417314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 msgr2=0x7fb29c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb2a1879640 1 --2- 172.21.15.113:0/3966417314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c0b6100 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb290009870 tx=0x7fb29002a180 comp rx=0 tx=0).stop 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb2a1879640 1 -- 172.21.15.113:0/3966417314 shutdown_connections 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb2a1879640 1 --2- 172.21.15.113:0/3966417314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 0x7fb29c0b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.540+0000 7fb2a1879640 1 --2- 172.21.15.113:0/3966417314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c0b6100 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.541+0000 7fb2a1879640 1 -- 172.21.15.113:0/3966417314 >> 172.21.15.113:0/3966417314 conn(0x7fb29c06b720 msgr2=0x7fb29c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.541+0000 7fb2a1879640 1 -- 172.21.15.113:0/3966417314 shutdown_connections 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.541+0000 7fb2a1879640 1 -- 172.21.15.113:0/3966417314 wait complete. 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.542+0000 7fb2a1879640 1 Processor -- start 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.542+0000 7fb2a1879640 1 -- start start 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.542+0000 7fb2a1879640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c14c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb2a1879640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 0x7fb29c1610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb2a1879640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb29c15e510 0x7fb29c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb2a1879640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0d3ea0 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb2a1879640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0d3980 con 0x7fb29c15e510 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb2a1879640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c0d2a90 con 0x7fb29c0b73c0 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c14c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c14c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38748/0 (socket says 172.21.15.113:38748) 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 -- 172.21.15.113:0/2610676597 learned_addr learned my addr 172.21.15.113:0/2610676597 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29a7fc640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 0x7fb29c1610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29b7fe640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb29c15e510 0x7fb29c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 -- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 msgr2=0x7fb29c1610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 0x7fb29c1610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 -- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb29c15e510 msgr2=0x7fb29c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb29c15e510 0x7fb29c158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 -- 172.21.15.113:0/2610676597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c0cab80 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29b7fe640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb29c15e510 0x7fb29c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb29affd640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c14c720 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00b410 tx=0x7fb28c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb283fff640 1 -- 172.21.15.113:0/2610676597 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c015020 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb283fff640 1 -- 172.21.15.113:0/2610676597 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c011830 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c0c9ae0 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb283fff640 1 -- 172.21.15.113:0/2610676597 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c011c20 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.543+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c0ca0a0 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.544+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29c0691a0 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.545+0000 7fb283fff640 1 -- 172.21.15.113:0/2610676597 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb28c018020 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.545+0000 7fb283fff640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb28806be30 0x7fb28806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.545+0000 7fb283fff640 1 -- 172.21.15.113:0/2610676597 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fb28c08a9b0 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.545+0000 7fb29a7fc640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb28806be30 0x7fb28806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.546+0000 7fb29a7fc640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb28806be30 0x7fb28806e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb290034000 tx=0x7fb29002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.550+0000 7fb283fff640 1 -- 172.21.15.113:0/2610676597 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb28c01f020 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.722+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb29c0ca760 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.726+0000 7fb283fff640 1 -- 172.21.15.113:0/2610676597 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fb28c0596f0 con 0x7fb29c0b5ca0 2024-04-25T00:01:28.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb28806be30 msgr2=0x7fb28806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb28806be30 0x7fb28806e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb290034000 tx=0x7fb29002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:28.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 msgr2=0x7fb29c14c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c14c720 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00b410 tx=0x7fb28c00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 shutdown_connections 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb28806be30 0x7fb28806e2f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb29c15e510 0x7fb29c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb29c0b73c0 0x7fb29c1610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 --2- 172.21.15.113:0/2610676597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29c0b5ca0 0x7fb29c14c720 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 >> 172.21.15.113:0/2610676597 conn(0x7fb29c06b720 msgr2=0x7fb29c15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 shutdown_connections 2024-04-25T00:01:28.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.728+0000 7fb2a1879640 1 -- 172.21.15.113:0/2610676597 wait complete. 2024-04-25T00:01:28.739 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T00:01:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.835+0000 7f2826d7e640 1 Processor -- start 2024-04-25T00:01:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.835+0000 7f2826d7e640 1 -- start start 2024-04-25T00:01:28.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f2826d7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 0x7f2820070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f2826d7e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28200b2d50 con 0x7f28200b3af0 2024-04-25T00:01:28.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f2826d7e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2820072a70 con 0x7f2820072ea0 2024-04-25T00:01:28.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f2826d7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f28200b3010 con 0x7f282015a6b0 2024-04-25T00:01:28.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f282531b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f28200b3af0 0x7f28200b3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43216/0 (socket says 172.21.15.113:43216) 2024-04-25T00:01:28.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f282531b640 1 -- 172.21.15.113:0/4219456949 learned_addr learned my addr 172.21.15.113:0/4219456949 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f2824b1a640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 0x7f2820070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223974477 0 0) 0x7f28200b2d50 con 0x7f28200b3af0 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.836+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2808003520 con 0x7f28200b3af0 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 298819517 0 0) 0x7f2808003520 con 0x7f28200b3af0 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 msgr2=0x7f2820070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 0x7f2820070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 >> v1:172.21.15.178:6789/0 conn(0x7f2820072ea0 legacy=0x7f2820071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28200b32f0 con 0x7f28200b3af0 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f28180019f0 con 0x7f28200b3af0 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 >> v1:172.21.15.113:6789/0 conn(0x7f28200b3af0 legacy=0x7f28200b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080039e0 0x7f2808003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 0x7f2808004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2808004d40 0x7f28080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f28200b3010 con 0x7f2808004d40 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f280800b9c0 con 0x7f28080039e0 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f280800bbc0 con 0x7f282015a6b0 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f282531b640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2808004d40 0x7f28080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f2824b1a640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080039e0 0x7f2808003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f281ffff640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 0x7f2808004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f282531b640 1 -- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 msgr2=0x7f2808004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f282531b640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f282015a6b0 0x7f2808004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f282531b640 1 -- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080039e0 msgr2=0x7f2808003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f282531b640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080039e0 0x7f2808003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.837+0000 7f282531b640 1 -- 172.21.15.113:0/4219456949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2808003520 con 0x7f2808004d40 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.838+0000 7f282531b640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2808004d40 0x7f28080051c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f281800a860 tx=0x7f2808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c929b641fe55d3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.838+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281800a480 con 0x7f2808004d40 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.838+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281802f070 con 0x7f2808004d40 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.838+0000 7f281f7fe640 1 -- 172.21.15.113:0/4219456949 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281802bcb0 con 0x7f2808004d40 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.838+0000 7f2826d7e640 1 -- 172.21.15.113:0/4219456949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2808004d40 msgr2=0x7f28080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.838+0000 7f2826d7e640 1 --2- 172.21.15.113:0/4219456949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2808004d40 0x7f28080051c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f281800a860 tx=0x7f2808005830 comp rx=0 tx=0).stop 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.839+0000 7f282531b640 1 -- 172.21.15.113:0/4219456949 reap_dead start 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.839+0000 7f2826d7e640 1 -- 172.21.15.113:0/4219456949 shutdown_connections 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.839+0000 7f2826d7e640 1 -- 172.21.15.113:0/4219456949 >> 172.21.15.113:0/4219456949 conn(0x7f28200b1a50 msgr2=0x7f28200b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.839+0000 7f2826d7e640 1 -- 172.21.15.113:0/4219456949 shutdown_connections 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.839+0000 7f2826d7e640 1 -- 172.21.15.113:0/4219456949 wait complete. 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.840+0000 7f2826d7e640 1 Processor -- start 2024-04-25T00:01:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.840+0000 7f2826d7e640 1 -- start start 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2808004d40 0x7f28201414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28201419e0 0x7f2820144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2820144b70 0x7f2820168700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f282015aa90 con 0x7f2820144b70 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2820072720 con 0x7f2808004d40 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f282015ac10 con 0x7f28201419e0 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2820144b70 0x7f2820168700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2820144b70 0x7f2820168700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38768/0 (socket says 172.21.15.113:38768) 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 -- 172.21.15.113:0/1073760051 learned_addr learned my addr 172.21.15.113:0/1073760051 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2824b1a640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2808004d40 0x7f28201414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f281ffff640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28201419e0 0x7f2820144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 -- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28201419e0 msgr2=0x7f2820144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28201419e0 0x7f2820144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 -- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2808004d40 msgr2=0x7f28201414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2808004d40 0x7f28201414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 -- 172.21.15.113:0/1073760051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2820141ef0 con 0x7f2820144b70 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f282531b640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2820144b70 0x7f2820168700 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2818001cf0 tx=0x7f28200b34c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f281d7fa640 1 -- 172.21.15.113:0/1073760051 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818045070 con 0x7f2820144b70 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f281d7fa640 1 -- 172.21.15.113:0/1073760051 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28180019e0 con 0x7f2820144b70 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282014c420 con 0x7f2820144b70 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f281d7fa640 1 -- 172.21.15.113:0/1073760051 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281800a480 con 0x7f2820144b70 2024-04-25T00:01:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.841+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282014c9e0 con 0x7f2820144b70 2024-04-25T00:01:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.842+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28200691a0 con 0x7f2820144b70 2024-04-25T00:01:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.843+0000 7f281d7fa640 1 -- 172.21.15.113:0/1073760051 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2818048070 con 0x7f2820144b70 2024-04-25T00:01:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.843+0000 7f281d7fa640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f280c06be60 0x7f280c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.843+0000 7f281d7fa640 1 -- 172.21.15.113:0/1073760051 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f28180aa6e0 con 0x7f2820144b70 2024-04-25T00:01:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.843+0000 7f2824b1a640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f280c06be60 0x7f280c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.844+0000 7f2824b1a640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f280c06be60 0x7f280c06e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f282014d9e0 tx=0x7f2810009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:28.847+0000 7f281d7fa640 1 -- 172.21.15.113:0/1073760051 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2818078e00 con 0x7f2820144b70 2024-04-25T00:01:29.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.097+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) -- 0x7f28200b2900 con 0x7f2820144b70 2024-04-25T00:01:29.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.416+0000 7f281d7fa640 1 -- 172.21.15.113:0/1073760051 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v167) ==== 143+0+0 (secure 0 0 0) 0x7f281807d100 con 0x7f2820144b70 2024-04-25T00:01:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.418+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f280c06be60 msgr2=0x7f280c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.418+0000 7f2826d7e640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f280c06be60 0x7f280c06e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f282014d9e0 tx=0x7f2810009190 comp rx=0 tx=0).stop 2024-04-25T00:01:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.418+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2820144b70 msgr2=0x7f2820168700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.418+0000 7f2826d7e640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2820144b70 0x7f2820168700 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2818001cf0 tx=0x7f28200b34c0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 shutdown_connections 2024-04-25T00:01:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f280c06be60 0x7f280c06e320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2820144b70 0x7f2820168700 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28201419e0 0x7f2820144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 --2- 172.21.15.113:0/1073760051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2808004d40 0x7f28201414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 >> 172.21.15.113:0/1073760051 conn(0x7f28200b1a50 msgr2=0x7f28200718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:29.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 shutdown_connections 2024-04-25T00:01:29.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.419+0000 7f2826d7e640 1 -- 172.21.15.113:0/1073760051 wait complete. 2024-04-25T00:01:29.431 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T00:01:29.431 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T00:01:29.431 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:29.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.529+0000 7fc89074f640 1 Processor -- start 2024-04-25T00:01:29.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.530+0000 7fc89074f640 1 -- start start 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.530+0000 7fc89074f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880b3fd0 0x7fc88806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.530+0000 7fc89074f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc88806e510 0x7fc8880cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.530+0000 7fc89074f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc888144cf0 con 0x7fc888144910 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.530+0000 7fc89074f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc8880b2f30 con 0x7fc88806e510 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.530+0000 7fc89074f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8880b3210 con 0x7fc8880b3fd0 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88e4eb640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc888144910 0x7fc8880b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43224/0 (socket says 172.21.15.113:43224) 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88ecec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc88806e510 0x7fc8880cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88e4eb640 1 -- 172.21.15.113:0/4267205055 learned_addr learned my addr 172.21.15.113:0/4267205055 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88d4e9640 1 -- 172.21.15.113:0/4267205055 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1524456493 0 0) 0x7fc888144cf0 con 0x7fc888144910 2024-04-25T00:01:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88dcea640 1 --2- 172.21.15.113:0/4267205055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880b3fd0 0x7fc88806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88d4e9640 1 -- 172.21.15.113:0/4267205055 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc874003520 con 0x7fc888144910 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88ecec640 1 -- 172.21.15.113:0/4267205055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880b3fd0 msgr2=0x7fc88806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88ecec640 1 --2- 172.21.15.113:0/4267205055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880b3fd0 0x7fc88806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88ecec640 1 -- 172.21.15.113:0/4267205055 >> v1:172.21.15.113:6789/0 conn(0x7fc888144910 legacy=0x7fc8880b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.531+0000 7fc88ecec640 1 -- 172.21.15.113:0/4267205055 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8880b34f0 con 0x7fc88806e510 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc88ecec640 1 --2- 172.21.15.113:0/4267205055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc8880cd500 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc880009d90 tx=0x7fc88002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3111272475fdb3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc88d4e9640 1 -- 172.21.15.113:0/4267205055 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880007740 con 0x7fc88806e510 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc88d4e9640 1 -- 172.21.15.113:0/4267205055 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8800078e0 con 0x7fc88806e510 2024-04-25T00:01:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc88d4e9640 1 -- 172.21.15.113:0/4267205055 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880007cb0 con 0x7fc88806e510 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 -- 172.21.15.113:0/4267205055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 msgr2=0x7fc8880cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 --2- 172.21.15.113:0/4267205055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc8880cd500 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc880009d90 tx=0x7fc88002a660 comp rx=0 tx=0).stop 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 -- 172.21.15.113:0/4267205055 shutdown_connections 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 --2- 172.21.15.113:0/4267205055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc8880cd500 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 --2- 172.21.15.113:0/4267205055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880b3fd0 0x7fc88806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 -- 172.21.15.113:0/4267205055 >> 172.21.15.113:0/4267205055 conn(0x7fc8880b1a50 msgr2=0x7fc8880b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 -- 172.21.15.113:0/4267205055 shutdown_connections 2024-04-25T00:01:29.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.532+0000 7fc89074f640 1 -- 172.21.15.113:0/4267205055 wait complete. 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.533+0000 7fc89074f640 1 Processor -- start 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.533+0000 7fc89074f640 1 -- start start 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc89074f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc88813b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc89074f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880b3fd0 0x7fc88813d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc89074f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc888144910 0x7fc88815c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc89074f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880b4da0 con 0x7fc88806e510 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc89074f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880b4410 con 0x7fc8880b3fd0 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc89074f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8880b4f20 con 0x7fc888144910 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc88e4eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc88813b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc88e4eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc88813b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38778/0 (socket says 172.21.15.113:38778) 2024-04-25T00:01:29.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc88e4eb640 1 -- 172.21.15.113:0/833837018 learned_addr learned my addr 172.21.15.113:0/833837018 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc88ecec640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc888144910 0x7fc88815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.534+0000 7fc88dcea640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880b3fd0 0x7fc88813d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc88e4eb640 1 -- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc888144910 msgr2=0x7fc88815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc88e4eb640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc888144910 0x7fc88815c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc88e4eb640 1 -- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880b3fd0 msgr2=0x7fc88813d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc88e4eb640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880b3fd0 0x7fc88813d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc88e4eb640 1 -- 172.21.15.113:0/833837018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88813b0b0 con 0x7fc88806e510 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc88e4eb640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc88813b950 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c390 tx=0x7fc88400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc87affd640 1 -- 172.21.15.113:0/833837018 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884017070 con 0x7fc88806e510 2024-04-25T00:01:29.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc87affd640 1 -- 172.21.15.113:0/833837018 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc884005ce0 con 0x7fc88806e510 2024-04-25T00:01:29.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88813e130 con 0x7fc88806e510 2024-04-25T00:01:29.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc87affd640 1 -- 172.21.15.113:0/833837018 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8840072f0 con 0x7fc88806e510 2024-04-25T00:01:29.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.535+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88813e6f0 con 0x7fc88806e510 2024-04-25T00:01:29.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.536+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8880691a0 con 0x7fc88806e510 2024-04-25T00:01:29.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.536+0000 7fc87affd640 1 -- 172.21.15.113:0/833837018 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc88401a070 con 0x7fc88806e510 2024-04-25T00:01:29.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.537+0000 7fc87affd640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc86c06be60 0x7fc86c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.537+0000 7fc87affd640 1 -- 172.21.15.113:0/833837018 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fc884089f20 con 0x7fc88806e510 2024-04-25T00:01:29.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.537+0000 7fc88dcea640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc86c06be60 0x7fc86c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.537+0000 7fc88dcea640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc86c06be60 0x7fc86c06e320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc88815da80 tx=0x7fc87c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.541+0000 7fc87affd640 1 -- 172.21.15.113:0/833837018 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc884058bd0 con 0x7fc88806e510 2024-04-25T00:01:29.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.714+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8880b2320 con 0x7fc88806e510 2024-04-25T00:01:29.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.715+0000 7fc87affd640 1 -- 172.21.15.113:0/833837018 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13070 (secure 0 0 0) 0x7fc8840584d0 con 0x7fc88806e510 2024-04-25T00:01:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:29.715 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":51,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:28.409593+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6836/1715251400":"2024-04-26T00:01:25.402683+0000","172.21.15.113:6834/1715251400":"2024-04-26T00:01:25.402683+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:29.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc86c06be60 msgr2=0x7fc86c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc86c06be60 0x7fc86c06e320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc88815da80 tx=0x7fc87c008040 comp rx=0 tx=0).stop 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 msgr2=0x7fc88813b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc88813b950 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c390 tx=0x7fc88400c850 comp rx=0 tx=0).stop 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 shutdown_connections 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc86c06be60 0x7fc86c06e320 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880b3fd0 0x7fc88813d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc888144910 0x7fc88815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 --2- 172.21.15.113:0/833837018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88806e510 0x7fc88813b950 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 >> 172.21.15.113:0/833837018 conn(0x7fc8880b1a50 msgr2=0x7fc888071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 shutdown_connections 2024-04-25T00:01:29.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.717+0000 7fc89074f640 1 -- 172.21.15.113:0/833837018 wait complete. 2024-04-25T00:01:29.728 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:29.729 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:29.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.829+0000 7f81aaa4d640 1 Processor -- start 2024-04-25T00:01:29.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.829+0000 7f81aaa4d640 1 -- start start 2024-04-25T00:01:29.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81aaa4d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81a415ba50 0x7f81a415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81aaa4d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a4071170 con 0x7f81a415b670 2024-04-25T00:01:29.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81aaa4d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a4071450 con 0x7f81a406cea0 2024-04-25T00:01:29.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81aaa4d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f81a4071730 con 0x7f81a415ba50 2024-04-25T00:01:29.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81a415ba50 0x7f81a415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f81a415b670 0x7f81a40cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43240/0 (socket says 172.21.15.113:43240) 2024-04-25T00:01:29.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81a415ba50 0x7f81a415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38784/0 (socket says 172.21.15.113:38784) 2024-04-25T00:01:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a3fff640 1 -- 172.21.15.113:0/943185700 learned_addr learned my addr 172.21.15.113:0/943185700 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a37fe640 1 -- 172.21.15.113:0/943185700 >> v1:172.21.15.178:6789/0 conn(0x7f81a406cea0 legacy=0x7f81a406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a37fe640 1 -- 172.21.15.113:0/943185700 >> v1:172.21.15.113:6789/0 conn(0x7f81a415b670 legacy=0x7f81a40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a2ffd640 1 -- 172.21.15.113:0/943185700 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3369728314 0 0) 0x7f81a4071170 con 0x7f81a415b670 2024-04-25T00:01:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.830+0000 7f81a37fe640 1 -- 172.21.15.113:0/943185700 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a4071a10 con 0x7f81a415ba50 2024-04-25T00:01:29.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81a37fe640 1 --2- 172.21.15.113:0/943185700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415ba50 0x7f81a415bff0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f819800a940 tx=0x7f819802b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a4d8cfd49c3199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81a2ffd640 1 -- 172.21.15.113:0/943185700 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800b040 con 0x7f81a415ba50 2024-04-25T00:01:29.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81a2ffd640 1 -- 172.21.15.113:0/943185700 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8198030710 con 0x7f81a415ba50 2024-04-25T00:01:29.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81a2ffd640 1 -- 172.21.15.113:0/943185700 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198030ae0 con 0x7f81a415ba50 2024-04-25T00:01:29.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81aaa4d640 1 -- 172.21.15.113:0/943185700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415ba50 msgr2=0x7f81a415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/943185700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415ba50 0x7f81a415bff0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f819800a940 tx=0x7f819802b480 comp rx=0 tx=0).stop 2024-04-25T00:01:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81aaa4d640 1 -- 172.21.15.113:0/943185700 shutdown_connections 2024-04-25T00:01:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/943185700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415ba50 0x7f81a415bff0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81aaa4d640 1 -- 172.21.15.113:0/943185700 >> 172.21.15.113:0/943185700 conn(0x7f81a406b0f0 msgr2=0x7f81a406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81aaa4d640 1 -- 172.21.15.113:0/943185700 shutdown_connections 2024-04-25T00:01:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.831+0000 7f81aaa4d640 1 -- 172.21.15.113:0/943185700 wait complete. 2024-04-25T00:01:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 Processor -- start 2024-04-25T00:01:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 -- start start 2024-04-25T00:01:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a406cea0 0x7f81a413ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415b670 0x7f81a4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a415ba50 0x7f81a413d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81a40d0200 con 0x7f81a415b670 2024-04-25T00:01:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81a40d0080 con 0x7f81a415ba50 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.833+0000 7f81aaa4d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81a40d0380 con 0x7f81a406cea0 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415b670 0x7f81a4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415b670 0x7f81a4157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38798/0 (socket says 172.21.15.113:38798) 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 -- 172.21.15.113:0/2344680261 learned_addr learned my addr 172.21.15.113:0/2344680261 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a8fea640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a415ba50 0x7f81a413d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a3fff640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a406cea0 0x7f81a413ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 -- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a406cea0 msgr2=0x7f81a413ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a406cea0 0x7f81a413ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 -- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a415ba50 msgr2=0x7f81a413d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a415ba50 0x7f81a413d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 -- 172.21.15.113:0/2344680261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a41406d0 con 0x7f81a415b670 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a3fff640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a406cea0 0x7f81a413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a37fe640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415b670 0x7f81a4157210 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f819800a940 tx=0x7f8198004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a0ff9640 1 -- 172.21.15.113:0/2344680261 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198040070 con 0x7f81a415b670 2024-04-25T00:01:29.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a0ff9640 1 -- 172.21.15.113:0/2344680261 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8198033d50 con 0x7f81a415b670 2024-04-25T00:01:29.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a4150bd0 con 0x7f81a415b670 2024-04-25T00:01:29.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81a0ff9640 1 -- 172.21.15.113:0/2344680261 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198006e10 con 0x7f81a415b670 2024-04-25T00:01:29.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.834+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a4151190 con 0x7f81a415b670 2024-04-25T00:01:29.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.835+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a4150610 con 0x7f81a415b670 2024-04-25T00:01:29.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.836+0000 7f81a0ff9640 1 -- 172.21.15.113:0/2344680261 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8198043070 con 0x7f81a415b670 2024-04-25T00:01:29.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.836+0000 7f81a0ff9640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f819006be30 0x7f819006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.836+0000 7f81a3fff640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f819006be30 0x7f819006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.836+0000 7f81a0ff9640 1 -- 172.21.15.113:0/2344680261 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f81980ab810 con 0x7f81a415b670 2024-04-25T00:01:29.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.837+0000 7f81a3fff640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f819006be30 0x7f819006e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8194000f60 tx=0x7f8194007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:29.841+0000 7f81a0ff9640 1 -- 172.21.15.113:0/2344680261 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81a4150610 con 0x7f81a415b670 2024-04-25T00:01:30.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.012+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81a415c470 con 0x7f81a415b670 2024-04-25T00:01:30.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.013+0000 7f81a0ff9640 1 -- 172.21.15.113:0/2344680261 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f81a41406d0 con 0x7f81a415b670 2024-04-25T00:01:30.013 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:01:30.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f819006be30 msgr2=0x7f819006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f819006be30 0x7f819006e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8194000f60 tx=0x7f8194007c10 comp rx=0 tx=0).stop 2024-04-25T00:01:30.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415b670 msgr2=0x7f81a4157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415b670 0x7f81a4157210 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f819800a940 tx=0x7f8198004730 comp rx=0 tx=0).stop 2024-04-25T00:01:30.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 shutdown_connections 2024-04-25T00:01:30.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f819006be30 0x7f819006e2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a415ba50 0x7f81a413d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a415b670 0x7f81a4157210 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 --2- 172.21.15.113:0/2344680261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a406cea0 0x7f81a413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 >> 172.21.15.113:0/2344680261 conn(0x7f81a406b0f0 msgr2=0x7f81a414ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 shutdown_connections 2024-04-25T00:01:30.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.015+0000 7f81aaa4d640 1 -- 172.21.15.113:0/2344680261 wait complete. 2024-04-25T00:01:30.026 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T00:01:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.123+0000 7f1948f34640 1 Processor -- start 2024-04-25T00:01:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.123+0000 7f1948f34640 1 -- start start 2024-04-25T00:01:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.123+0000 7f1948f34640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1944144910 0x7f19440b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1948f34640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19440b56b0 0x7f19440cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1948f34640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1944144cf0 con 0x7f19440b5240 2024-04-25T00:01:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1948f34640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f19440b4510 con 0x7f19440b56b0 2024-04-25T00:01:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1948f34640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f19440b2f30 con 0x7f1944144910 2024-04-25T00:01:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1942d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19440b56b0 0x7f19440cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1941d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f19440b5240 0x7f194406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43250/0 (socket says 172.21.15.113:43250) 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1941d9b640 1 -- 172.21.15.113:0/2692947513 learned_addr learned my addr 172.21.15.113:0/2692947513 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f194259c640 1 --2- 172.21.15.113:0/2692947513 >> v2:172.21.15.178:3300/0 conn(0x7f1944144910 0x7f19440b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1942d9d640 1 -- 172.21.15.113:0/2692947513 >> v2:172.21.15.178:3300/0 conn(0x7f1944144910 msgr2=0x7f19440b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1942d9d640 1 --2- 172.21.15.113:0/2692947513 >> v2:172.21.15.178:3300/0 conn(0x7f1944144910 0x7f19440b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1942d9d640 1 -- 172.21.15.113:0/2692947513 >> v1:172.21.15.113:6789/0 conn(0x7f19440b5240 legacy=0x7f194406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f194159a640 1 -- 172.21.15.113:0/2692947513 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 995021204 0 0) 0x7f1944144cf0 con 0x7f19440b5240 2024-04-25T00:01:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.124+0000 7f1942d9d640 1 -- 172.21.15.113:0/2692947513 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19440b3210 con 0x7f19440b56b0 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1942d9d640 1 --2- 172.21.15.113:0/2692947513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19440b56b0 0x7f19440cd500 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f193400bfa0 tx=0x7f193402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9014f6cde05d20e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f194159a640 1 -- 172.21.15.113:0/2692947513 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934039070 con 0x7f19440b56b0 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f194159a640 1 -- 172.21.15.113:0/2692947513 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1934009540 con 0x7f19440b56b0 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f194159a640 1 -- 172.21.15.113:0/2692947513 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934009910 con 0x7f19440b56b0 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1948f34640 1 -- 172.21.15.113:0/2692947513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19440b56b0 msgr2=0x7f19440cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1948f34640 1 --2- 172.21.15.113:0/2692947513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19440b56b0 0x7f19440cd500 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f193400bfa0 tx=0x7f193402c8b0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1948f34640 1 -- 172.21.15.113:0/2692947513 shutdown_connections 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1948f34640 1 --2- 172.21.15.113:0/2692947513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19440b56b0 0x7f19440cd500 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1948f34640 1 --2- 172.21.15.113:0/2692947513 >> v2:172.21.15.178:3300/0 conn(0x7f1944144910 0x7f19440b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1948f34640 1 -- 172.21.15.113:0/2692947513 >> 172.21.15.113:0/2692947513 conn(0x7f19440b1a50 msgr2=0x7f19440b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.125+0000 7f1948f34640 1 -- 172.21.15.113:0/2692947513 shutdown_connections 2024-04-25T00:01:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.126+0000 7f1948f34640 1 -- 172.21.15.113:0/2692947513 wait complete. 2024-04-25T00:01:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.127+0000 7f1948f34640 1 Processor -- start 2024-04-25T00:01:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.127+0000 7f1948f34640 1 -- start start 2024-04-25T00:01:30.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.127+0000 7f1948f34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19440b5240 0x7f194406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.127+0000 7f1948f34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19440b56b0 0x7f19441e6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.127+0000 7f1948f34640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1944144910 0x7f19441501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.127+0000 7f1948f34640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f19440b4da0 con 0x7f1944144910 2024-04-25T00:01:30.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1948f34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19440b37d0 con 0x7f19440b5240 2024-04-25T00:01:30.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1948f34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f19440b4f20 con 0x7f19440b56b0 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1944144910 0x7f19441501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1944144910 0x7f19441501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38820/0 (socket says 172.21.15.113:38820) 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 -- 172.21.15.113:0/4147751258 learned_addr learned my addr 172.21.15.113:0/4147751258 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1941d9b640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19440b56b0 0x7f19441e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f194259c640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19440b5240 0x7f194406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 -- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19440b56b0 msgr2=0x7f19441e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19440b56b0 0x7f19441e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 -- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19440b5240 msgr2=0x7f194406e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19440b5240 0x7f194406e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 -- 172.21.15.113:0/4147751258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19441e7b70 con 0x7f1944144910 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1942d9d640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1944144910 0x7f19441501f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1934037040 tx=0x7f1934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1932ffd640 1 -- 172.21.15.113:0/4147751258 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934039040 con 0x7f1944144910 2024-04-25T00:01:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1932ffd640 1 -- 172.21.15.113:0/4147751258 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19340092b0 con 0x7f1944144910 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19441e72d0 con 0x7f1944144910 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1932ffd640 1 -- 172.21.15.113:0/4147751258 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19340348a0 con 0x7f1944144910 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.128+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19441e7890 con 0x7f1944144910 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.129+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19440691a0 con 0x7f1944144910 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.130+0000 7f1932ffd640 1 -- 172.21.15.113:0/4147751258 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1934032070 con 0x7f1944144910 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.130+0000 7f1932ffd640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f192406be30 0x7f192406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.130+0000 7f194259c640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f192406be30 0x7f192406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.130+0000 7f1932ffd640 1 -- 172.21.15.113:0/4147751258 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f19340ab660 con 0x7f1944144910 2024-04-25T00:01:30.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.131+0000 7f194259c640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f192406be30 0x7f192406e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f193c00aa20 tx=0x7f193c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.135+0000 7f1932ffd640 1 -- 172.21.15.113:0/4147751258 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1934079cc0 con 0x7f1944144910 2024-04-25T00:01:30.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.376+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) -- 0x7f194406c100 con 0x7f1944144910 2024-04-25T00:01:30.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.419+0000 7f1932ffd640 1 -- 172.21.15.113:0/4147751258 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v168) ==== 146+0+0 (secure 0 0 0) 0x7f193407dfc0 con 0x7f1944144910 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f192406be30 msgr2=0x7f192406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f192406be30 0x7f192406e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f193c00aa20 tx=0x7f193c008040 comp rx=0 tx=0).stop 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1944144910 msgr2=0x7f19441501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1944144910 0x7f19441501f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1934037040 tx=0x7f1934004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 shutdown_connections 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f192406be30 0x7f192406e2f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1944144910 0x7f19441501f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19440b56b0 0x7f19441e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 --2- 172.21.15.113:0/4147751258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19440b5240 0x7f194406e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 >> 172.21.15.113:0/4147751258 conn(0x7f19440b1a50 msgr2=0x7f19440b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 shutdown_connections 2024-04-25T00:01:30.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.421+0000 7f1948f34640 1 -- 172.21.15.113:0/4147751258 wait complete. 2024-04-25T00:01:30.432 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:01:30.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.531+0000 7feed7e2c640 1 Processor -- start 2024-04-25T00:01:30.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.532+0000 7feed7e2c640 1 -- start start 2024-04-25T00:01:30.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.532+0000 7feed7e2c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed00b3080 0x7feed00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.532+0000 7feed7e2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feed015e6b0 0x7feed00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.532+0000 7feed7e2c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed015f4f0 con 0x7feed015e2d0 2024-04-25T00:01:30.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.532+0000 7feed7e2c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feed00cde00 con 0x7feed015e6b0 2024-04-25T00:01:30.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.532+0000 7feed7e2c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feed015f7d0 con 0x7feed00b3080 2024-04-25T00:01:30.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed63c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feed015e6b0 0x7feed00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed5bc8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feed015e2d0 0x7feed015ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43258/0 (socket says 172.21.15.113:43258) 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed5bc8640 1 -- 172.21.15.113:0/3984874573 learned_addr learned my addr 172.21.15.113:0/3984874573 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed53c7640 1 --2- 172.21.15.113:0/3984874573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed00b3080 0x7feed00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed63c9640 1 -- 172.21.15.113:0/3984874573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed00b3080 msgr2=0x7feed00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed63c9640 1 --2- 172.21.15.113:0/3984874573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed00b3080 0x7feed00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed63c9640 1 -- 172.21.15.113:0/3984874573 >> v1:172.21.15.113:6789/0 conn(0x7feed015e2d0 legacy=0x7feed015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed4bc6640 1 -- 172.21.15.113:0/3984874573 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794041648 0 0) 0x7feed015f4f0 con 0x7feed015e2d0 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed63c9640 1 -- 172.21.15.113:0/3984874573 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed015fab0 con 0x7feed015e6b0 2024-04-25T00:01:30.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.533+0000 7feed53c7640 1 --2- 172.21.15.113:0/3984874573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed00b3080 0x7feed00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed63c9640 1 --2- 172.21.15.113:0/3984874573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed015e6b0 0x7feed00b4870 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feec800bfa0 tx=0x7feec802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f05efa9619029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed4bc6640 1 -- 172.21.15.113:0/3984874573 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8038070 con 0x7feed015e6b0 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed4bc6640 1 -- 172.21.15.113:0/3984874573 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8009540 con 0x7feed015e6b0 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed4bc6640 1 -- 172.21.15.113:0/3984874573 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8009910 con 0x7feed015e6b0 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 -- 172.21.15.113:0/3984874573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed015e6b0 msgr2=0x7feed00b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 --2- 172.21.15.113:0/3984874573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed015e6b0 0x7feed00b4870 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feec800bfa0 tx=0x7feec802c7e0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 -- 172.21.15.113:0/3984874573 shutdown_connections 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 --2- 172.21.15.113:0/3984874573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed015e6b0 0x7feed00b4870 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 --2- 172.21.15.113:0/3984874573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed00b3080 0x7feed00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 -- 172.21.15.113:0/3984874573 >> 172.21.15.113:0/3984874573 conn(0x7feed006b0f0 msgr2=0x7feed006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 -- 172.21.15.113:0/3984874573 shutdown_connections 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.534+0000 7feed7e2c640 1 -- 172.21.15.113:0/3984874573 wait complete. 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.535+0000 7feed7e2c640 1 Processor -- start 2024-04-25T00:01:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.535+0000 7feed7e2c640 1 -- start start 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed7e2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed00b3080 0x7feed0154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed7e2c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed015e2d0 0x7feed01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed7e2c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed015e6b0 0x7feed0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed7e2c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feed006d020 con 0x7feed00b3080 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed7e2c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feed006cea0 con 0x7feed015e2d0 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed7e2c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feed006d1a0 con 0x7feed015e6b0 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed5bc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed00b3080 0x7feed0154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed5bc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed00b3080 0x7feed0154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38832/0 (socket says 172.21.15.113:38832) 2024-04-25T00:01:30.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed5bc8640 1 -- 172.21.15.113:0/749951110 learned_addr learned my addr 172.21.15.113:0/749951110 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed5bc8640 1 -- 172.21.15.113:0/749951110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed015e6b0 msgr2=0x7feed0156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed5bc8640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed015e6b0 0x7feed0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed5bc8640 1 -- 172.21.15.113:0/749951110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed015e2d0 msgr2=0x7feed01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.536+0000 7feed5bc8640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed015e2d0 0x7feed01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.537+0000 7feed5bc8640 1 -- 172.21.15.113:0/749951110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed015ab20 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.537+0000 7feed5bc8640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed00b3080 0x7feed0154130 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7feec000c4d0 tx=0x7feec000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.537+0000 7feec67fc640 1 -- 172.21.15.113:0/749951110 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0018070 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.537+0000 7feec67fc640 1 -- 172.21.15.113:0/749951110 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0007500 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.537+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed015ae00 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.537+0000 7feec67fc640 1 -- 172.21.15.113:0/749951110 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0004a20 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.537+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed015b3c0 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.538+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed00691a0 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.538+0000 7feec67fc640 1 -- 172.21.15.113:0/749951110 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feec001b070 con 0x7feed00b3080 2024-04-25T00:01:30.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.538+0000 7feec67fc640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feeb406be30 0x7feeb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.539+0000 7feed53c7640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feeb406be30 0x7feeb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.539+0000 7feec67fc640 1 -- 172.21.15.113:0/749951110 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7feec0089ee0 con 0x7feed00b3080 2024-04-25T00:01:30.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.539+0000 7feed53c7640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feeb406be30 0x7feeb406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feecc006fd0 tx=0x7feecc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.543+0000 7feec67fc640 1 -- 172.21.15.113:0/749951110 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feec0058b90 con 0x7feed00b3080 2024-04-25T00:01:30.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.719+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7feed00cca40 con 0x7feed00b3080 2024-04-25T00:01:30.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.724+0000 7feec67fc640 1 -- 172.21.15.113:0/749951110 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7feec0058490 con 0x7feed00b3080 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feeb406be30 msgr2=0x7feeb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feeb406be30 0x7feeb406e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feecc006fd0 tx=0x7feecc008040 comp rx=0 tx=0).stop 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed00b3080 msgr2=0x7feed0154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed00b3080 0x7feed0154130 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7feec000c4d0 tx=0x7feec000c990 comp rx=0 tx=0).stop 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 shutdown_connections 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feeb406be30 0x7feeb406e2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed015e2d0 0x7feed01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feed015e6b0 0x7feed0156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 --2- 172.21.15.113:0/749951110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed00b3080 0x7feed0154130 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.726+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 >> 172.21.15.113:0/749951110 conn(0x7feed006b0f0 msgr2=0x7feed00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.727+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 shutdown_connections 2024-04-25T00:01:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:30.727+0000 7feed7e2c640 1 -- 172.21.15.113:0/749951110 wait complete. 2024-04-25T00:01:30.737 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:01:30.738 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:01:30.780 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:01:30.820 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:01:30.821 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:30.821 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:30.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:01:30.836 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:01:30.836 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:30.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:01:30.878 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:30.878 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:30.893 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:01:30.893 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:01:30.894 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:30.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:01:30.935 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:01:30.936 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:31.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.032+0000 7f6a7a963640 1 Processor -- start 2024-04-25T00:01:31.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.032+0000 7f6a7a963640 1 -- start start 2024-04-25T00:01:31.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a7a963640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 0x7f6a740d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a7a963640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a740d2800 con 0x7f6a7415e320 2024-04-25T00:01:31.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a7a963640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a740d2ae0 con 0x7f6a740d2290 2024-04-25T00:01:31.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a7a963640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a740d14d0 con 0x7f6a740d0b00 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a73fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6a7415e320 0x7f6a740d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43274/0 (socket says 172.21.15.113:43274) 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a73fff640 1 -- 172.21.15.113:0/3572618064 learned_addr learned my addr 172.21.15.113:0/3572618064 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a78f00640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 0x7f6a740d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726048658 0 0) 0x7f6a740d2800 con 0x7f6a7415e320 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a5c003520 con 0x7f6a7415e320 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891257689 0 0) 0x7f6a740d2ae0 con 0x7f6a740d2290 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a740d2800 con 0x7f6a740d2290 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 997386211 0 0) 0x7f6a5c003520 con 0x7f6a7415e320 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 msgr2=0x7f6a740d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 0x7f6a740d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 >> v1:172.21.15.178:6789/0 conn(0x7f6a740d2290 legacy=0x7f6a740b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a740d2dc0 con 0x7f6a7415e320 2024-04-25T00:01:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6a68001690 con 0x7f6a7415e320 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 >> v1:172.21.15.113:6789/0 conn(0x7f6a7415e320 legacy=0x7f6a740d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a5c003b70 0x7f6a5c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a5c004800 0x7f6a5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 0x7f6a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740d14d0 con 0x7f6a5c003b70 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c00ba70 con 0x7f6a5c004800 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5c00bc40 con 0x7f6a740d0b00 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a5c003b70 0x7f6a5c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.033+0000 7f6a737fe640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a5c004800 0x7f6a5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a78f00640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 0x7f6a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a73fff640 1 -- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 msgr2=0x7f6a5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a740d0b00 0x7f6a5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a73fff640 1 -- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a5c004800 msgr2=0x7f6a5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a5c004800 0x7f6a5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a73fff640 1 -- 172.21.15.113:0/3572618064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c003520 con 0x7f6a5c003b70 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a737fe640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a5c004800 0x7f6a5c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a5c003b70 0x7f6a5c003f70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800c210 tx=0x7f6a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb43cbc7b2e54b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68007cf0 con 0x7f6a5c003b70 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a68005ad0 con 0x7f6a5c003b70 2024-04-25T00:01:31.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.034+0000 7f6a72ffd640 1 -- 172.21.15.113:0/3572618064 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68007070 con 0x7f6a5c003b70 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.035+0000 7f6a7a963640 1 -- 172.21.15.113:0/3572618064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a5c003b70 msgr2=0x7f6a5c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.035+0000 7f6a7a963640 1 --2- 172.21.15.113:0/3572618064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a5c003b70 0x7f6a5c003f70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800c210 tx=0x7f6a5c005830 comp rx=0 tx=0).stop 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.035+0000 7f6a78f00640 1 -- 172.21.15.113:0/3572618064 reap_dead start 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.035+0000 7f6a7a963640 1 -- 172.21.15.113:0/3572618064 shutdown_connections 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.035+0000 7f6a7a963640 1 -- 172.21.15.113:0/3572618064 >> 172.21.15.113:0/3572618064 conn(0x7f6a740b5c20 msgr2=0x7f6a740b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.035+0000 7f6a7a963640 1 -- 172.21.15.113:0/3572618064 shutdown_connections 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.035+0000 7f6a7a963640 1 -- 172.21.15.113:0/3572618064 wait complete. 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.036+0000 7f6a7a963640 1 Processor -- start 2024-04-25T00:01:31.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.036+0000 7f6a7a963640 1 -- start start 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a7a963640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a5c003b70 0x7f6a741f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a7a963640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a7414e550 0x7f6a740ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a7a963640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a740ceed0 0x7f6a740cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a7a963640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7415e700 con 0x7f6a740ceed0 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a7a963640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740d3410 con 0x7f6a7414e550 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a7a963640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7415e880 con 0x7f6a5c003b70 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a740ceed0 0x7f6a740cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a740ceed0 0x7f6a740cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38854/0 (socket says 172.21.15.113:38854) 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 -- 172.21.15.113:0/3854483838 learned_addr learned my addr 172.21.15.113:0/3854483838 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a737fe640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a7414e550 0x7f6a740ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 -- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a5c003b70 msgr2=0x7f6a741f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a5c003b70 0x7f6a741f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a5c003b70 0x7f6a741f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 -- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a7414e550 msgr2=0x7f6a740ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a7414e550 0x7f6a740ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a78f00640 1 -- 172.21.15.113:0/3854483838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7415f890 con 0x7f6a740ceed0 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a737fe640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a7414e550 0x7f6a740ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.037+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a5c003b70 0x7f6a741f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.038+0000 7f6a78f00640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a740ceed0 0x7f6a740cd1f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6a740d51b0 tx=0x7f6a6c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.038+0000 7f6a70ff9640 1 -- 172.21.15.113:0/3854483838 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003990 con 0x7f6a740ceed0 2024-04-25T00:01:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.038+0000 7f6a70ff9640 1 -- 172.21.15.113:0/3854483838 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6c01b070 con 0x7f6a740ceed0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.038+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a740ab800 con 0x7f6a740ceed0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.038+0000 7f6a70ff9640 1 -- 172.21.15.113:0/3854483838 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003b70 con 0x7f6a740ceed0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.038+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7415fe10 con 0x7f6a740ceed0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.039+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a740691a0 con 0x7f6a740ceed0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.039+0000 7f6a70ff9640 1 -- 172.21.15.113:0/3854483838 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a6c005d10 con 0x7f6a740ceed0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.040+0000 7f6a70ff9640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a6006be30 0x7f6a6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.040+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a6006be30 0x7f6a6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.040+0000 7f6a70ff9640 1 -- 172.21.15.113:0/3854483838 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f6a6c08b070 con 0x7f6a740ceed0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.040+0000 7f6a73fff640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a6006be30 0x7f6a6006e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a68032000 tx=0x7f6a68007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.044+0000 7f6a70ff9640 1 -- 172.21.15.113:0/3854483838 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a6c059d20 con 0x7f6a740ceed0 2024-04-25T00:01:31.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.252+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a740caa80 con 0x7f6a740ceed0 2024-04-25T00:01:31.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.252+0000 7f6a70ff9640 1 -- 172.21.15.113:0/3854483838 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3878 (secure 0 0 0) 0x7f6a6c059620 con 0x7f6a740ceed0 2024-04-25T00:01:31.253 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:31.253 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/189449016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":189449016},{"type":"v1","addr":"172.21.15.113:6837","nonce":189449016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2671615374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2671615374},{"type":"v1","addr":"172.21.15.178:6834","nonce":2671615374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3871137944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3871137944},{"type":"v1","addr":"172.21.15.178:6835","nonce":3871137944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":6315,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1447351993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1447351993},{"type":"v1","addr":"172.21.15.113:6836","nonce":1447351993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-04-25T00:01:31.253 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 27 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a6006be30 msgr2=0x7f6a6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a6006be30 0x7f6a6006e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a68032000 tx=0x7f6a68007000 comp rx=0 tx=0).stop 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a740ceed0 msgr2=0x7f6a740cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a740ceed0 0x7f6a740cd1f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6a740d51b0 tx=0x7f6a6c0115a0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 shutdown_connections 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a6006be30 0x7f6a6006e2f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a7414e550 0x7f6a740ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a740ceed0 0x7f6a740cd1f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 --2- 172.21.15.113:0/3854483838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a5c003b70 0x7f6a741f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.254+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 >> 172.21.15.113:0/3854483838 conn(0x7f6a740b5c20 msgr2=0x7f6a740b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.255+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 shutdown_connections 2024-04-25T00:01:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.255+0000 7f6a7a963640 1 -- 172.21.15.113:0/3854483838 wait complete. 2024-04-25T00:01:31.266 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:01:31.266 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:01:31.267 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:01:31.267 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:31.267 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:01:31.267 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:01:31.267 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:01:31.267 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:31.267 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:01:31.268 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:01:31.268 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:01:31.268 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:31.268 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:01:31.268 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:01:31.268 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:01:31.269 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:31.277 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:01:31.277 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:01:31.278 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:01:31.278 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:01:31.280 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:01:31.280 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:01:31.281 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:01:31.281 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:01:31.319 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:01:31.320 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:31.321 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:01:31.321 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:31.324 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:01:31.324 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:31.325 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:01:31.325 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:31.325 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:01:31.391 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:01:31.392 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:01:31.405 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:01:31.405 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:01:31.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.428+0000 7f396ac3f640 1 Processor -- start 2024-04-25T00:01:31.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.428+0000 7f396ac3f640 1 -- start start 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f396ac3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f396415e480 0x7f39640c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f396ac3f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f396406f610 0x7f3964061b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f396ac3f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39640620c0 0x7f396415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f396ac3f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f39640c7360 con 0x7f39640620c0 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f396ac3f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f396406faf0 con 0x7f396406f610 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f396ac3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f396406fdd0 con 0x7f396415e480 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39640620c0 0x7f396415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39640620c0 0x7f396415e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38892/0 (socket says 172.21.15.113:38892) 2024-04-25T00:01:31.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 -- 172.21.15.113:0/105111256 learned_addr learned my addr 172.21.15.113:0/105111256 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39689db640 1 --2- 172.21.15.113:0/105111256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f396415e480 0x7f39640c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f3963fff640 1 --2- 172.21.15.113:0/105111256 >> v2:172.21.15.178:3300/0 conn(0x7f396406f610 0x7f3964061b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 -- 172.21.15.113:0/105111256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f396415e480 msgr2=0x7f39640c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 --2- 172.21.15.113:0/105111256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f396415e480 0x7f39640c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 -- 172.21.15.113:0/105111256 >> v2:172.21.15.178:3300/0 conn(0x7f396406f610 msgr2=0x7f3964061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 --2- 172.21.15.113:0/105111256 >> v2:172.21.15.178:3300/0 conn(0x7f396406f610 0x7f3964061b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.429+0000 7f39691dc640 1 -- 172.21.15.113:0/105111256 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39640c6880 con 0x7f39640620c0 2024-04-25T00:01:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.430+0000 7f3963fff640 1 --2- 172.21.15.113:0/105111256 >> v2:172.21.15.178:3300/0 conn(0x7f396406f610 0x7f3964061b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:31.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.430+0000 7f39691dc640 1 --2- 172.21.15.113:0/105111256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39640620c0 0x7f396415e860 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f396409ef90 tx=0x7f395c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252a1f864c4f5827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.431+0000 7f39637fe640 1 -- 172.21.15.113:0/105111256 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c036070 con 0x7f39640620c0 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.431+0000 7f39637fe640 1 -- 172.21.15.113:0/105111256 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c031600 con 0x7f39640620c0 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.431+0000 7f39637fe640 1 -- 172.21.15.113:0/105111256 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0319d0 con 0x7f39640620c0 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 -- 172.21.15.113:0/105111256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39640620c0 msgr2=0x7f396415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 --2- 172.21.15.113:0/105111256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39640620c0 0x7f396415e860 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f396409ef90 tx=0x7f395c02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 -- 172.21.15.113:0/105111256 shutdown_connections 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 --2- 172.21.15.113:0/105111256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39640620c0 0x7f396415e860 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 --2- 172.21.15.113:0/105111256 >> v2:172.21.15.178:3300/0 conn(0x7f396406f610 0x7f3964061b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 --2- 172.21.15.113:0/105111256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f396415e480 0x7f39640c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 -- 172.21.15.113:0/105111256 >> 172.21.15.113:0/105111256 conn(0x7f39640cd9b0 msgr2=0x7f39640cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 -- 172.21.15.113:0/105111256 shutdown_connections 2024-04-25T00:01:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.432+0000 7f396ac3f640 1 -- 172.21.15.113:0/105111256 wait complete. 2024-04-25T00:01:31.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.434+0000 7f396ac3f640 1 Processor -- start 2024-04-25T00:01:31.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.435+0000 7f396ac3f640 1 -- start start 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f396ac3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39640620c0 0x7f3964155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f396ac3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f396406f610 0x7f39640cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f396ac3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f396415e480 0x7f39640b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f396ac3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f39640c7c10 con 0x7f396406f610 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f396ac3f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f39640c6e40 con 0x7f396415e480 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f396ac3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f39640c7d90 con 0x7f39640620c0 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f3963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f396406f610 0x7f39640cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f3963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f396406f610 0x7f39640cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38908/0 (socket says 172.21.15.113:38908) 2024-04-25T00:01:31.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f3963fff640 1 -- 172.21.15.113:0/1360077491 learned_addr learned my addr 172.21.15.113:0/1360077491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f39691dc640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f396415e480 0x7f39640b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.436+0000 7f39689db640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39640620c0 0x7f3964155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f3963fff640 1 -- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39640620c0 msgr2=0x7f3964155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f3963fff640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39640620c0 0x7f3964155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f3963fff640 1 -- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f396415e480 msgr2=0x7f39640b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f3963fff640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f396415e480 0x7f39640b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f3963fff640 1 -- 172.21.15.113:0/1360077491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3964155170 con 0x7f396406f610 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f39691dc640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f396415e480 0x7f39640b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:31.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f3963fff640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f396406f610 0x7f39640cf2e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f395000d7d0 tx=0x7f395000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f39617fa640 1 -- 172.21.15.113:0/1360077491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950004030 con 0x7f396406f610 2024-04-25T00:01:31.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f39617fa640 1 -- 172.21.15.113:0/1360077491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39500041d0 con 0x7f396406f610 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.437+0000 7f39617fa640 1 -- 172.21.15.113:0/1360077491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395000b6c0 con 0x7f396406f610 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.438+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39640cfcd0 con 0x7f396406f610 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.438+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39641553e0 con 0x7f396406f610 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.439+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3964068220 con 0x7f396406f610 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.439+0000 7f39617fa640 1 -- 172.21.15.113:0/1360077491 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f395000b860 con 0x7f396406f610 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.439+0000 7f39617fa640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f394806be30 0x7f394806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.440+0000 7f39617fa640 1 -- 172.21.15.113:0/1360077491 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f395008ac60 con 0x7f396406f610 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.440+0000 7f39689db640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f394806be30 0x7f394806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.440+0000 7f39689db640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f394806be30 0x7f394806e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f39640ce6d0 tx=0x7f3958009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.444+0000 7f39617fa640 1 -- 172.21.15.113:0/1360077491 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3950059350 con 0x7f396406f610 2024-04-25T00:01:31.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:31.628+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f396406df50 con 0x7f396406f610 2024-04-25T00:01:32.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.436+0000 7f39617fa640 1 -- 172.21.15.113:0/1360077491 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) ==== 97+0+0 (secure 0 0 0) 0x7f395005d650 con 0x7f396406f610 2024-04-25T00:01:32.437 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:01:32.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f394806be30 msgr2=0x7f394806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f394806be30 0x7f394806e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f39640ce6d0 tx=0x7f3958009190 comp rx=0 tx=0).stop 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f396406f610 msgr2=0x7f39640cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f396406f610 0x7f39640cf2e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f395000d7d0 tx=0x7f395000dc90 comp rx=0 tx=0).stop 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 shutdown_connections 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f394806be30 0x7f394806e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f396415e480 0x7f39640b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f396406f610 0x7f39640cf2e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 --2- 172.21.15.113:0/1360077491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39640620c0 0x7f3964155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 >> 172.21.15.113:0/1360077491 conn(0x7f39640cd9b0 msgr2=0x7f39640af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 shutdown_connections 2024-04-25T00:01:32.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.439+0000 7f396ac3f640 1 -- 172.21.15.113:0/1360077491 wait complete. 2024-04-25T00:01:32.452 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:01:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.552+0000 7ff09ac9a640 1 Processor -- start 2024-04-25T00:01:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff09ac9a640 1 -- start start 2024-04-25T00:01:32.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff09ac9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09415e320 0x7ff0940d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff09ac9a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0940d2290 0x7ff0940b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff09ac9a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0940d0b00 0x7ff0940d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff09ac9a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff0940d14d0 con 0x7ff0940d0b00 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff09ac9a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff0940d3410 con 0x7ff0940d2290 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff09ac9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff0940d3de0 con 0x7ff09415e320 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff099237640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0940d0b00 0x7ff0940d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.553+0000 7ff099237640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0940d0b00 0x7ff0940d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38912/0 (socket says 172.21.15.113:38912) 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff099237640 1 -- 172.21.15.113:0/22368013 learned_addr learned my addr 172.21.15.113:0/22368013 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff098a36640 1 --2- 172.21.15.113:0/22368013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09415e320 0x7ff0940d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff099237640 1 -- 172.21.15.113:0/22368013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09415e320 msgr2=0x7ff0940d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:32.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff099237640 1 --2- 172.21.15.113:0/22368013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09415e320 0x7ff0940d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff099237640 1 -- 172.21.15.113:0/22368013 >> v2:172.21.15.178:3300/0 conn(0x7ff0940d2290 msgr2=0x7ff0940b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff093fff640 1 --2- 172.21.15.113:0/22368013 >> v2:172.21.15.178:3300/0 conn(0x7ff0940d2290 0x7ff0940b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff099237640 1 --2- 172.21.15.113:0/22368013 >> v2:172.21.15.178:3300/0 conn(0x7ff0940d2290 0x7ff0940b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff099237640 1 -- 172.21.15.113:0/22368013 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0940d2800 con 0x7ff0940d0b00 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.554+0000 7ff093fff640 1 --2- 172.21.15.113:0/22368013 >> v2:172.21.15.178:3300/0 conn(0x7ff0940d2290 0x7ff0940b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff099237640 1 --2- 172.21.15.113:0/22368013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0940d0b00 0x7ff0940d0f90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff094072720 tx=0x7ff07c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96605168c1b6b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff0937fe640 1 -- 172.21.15.113:0/22368013 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c037070 con 0x7ff0940d0b00 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff0937fe640 1 -- 172.21.15.113:0/22368013 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c032440 con 0x7ff0940d0b00 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff0937fe640 1 -- 172.21.15.113:0/22368013 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c032810 con 0x7ff0940d0b00 2024-04-25T00:01:32.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 -- 172.21.15.113:0/22368013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0940d0b00 msgr2=0x7ff0940d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/22368013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0940d0b00 0x7ff0940d0f90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff094072720 tx=0x7ff07c02a270 comp rx=0 tx=0).stop 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 -- 172.21.15.113:0/22368013 shutdown_connections 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/22368013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0940d0b00 0x7ff0940d0f90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/22368013 >> v2:172.21.15.178:3300/0 conn(0x7ff0940d2290 0x7ff0940b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/22368013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09415e320 0x7ff0940d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 -- 172.21.15.113:0/22368013 >> 172.21.15.113:0/22368013 conn(0x7ff0940b5c20 msgr2=0x7ff0940b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 -- 172.21.15.113:0/22368013 shutdown_connections 2024-04-25T00:01:32.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.555+0000 7ff09ac9a640 1 -- 172.21.15.113:0/22368013 wait complete. 2024-04-25T00:01:32.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.556+0000 7ff09ac9a640 1 Processor -- start 2024-04-25T00:01:32.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.556+0000 7ff09ac9a640 1 -- start start 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.557+0000 7ff09ac9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0940d0b00 0x7ff09415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.557+0000 7ff09ac9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0940d2290 0x7ff0941f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.557+0000 7ff09ac9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09415e320 0x7ff0941f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.557+0000 7ff09ac9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff0940d30a0 con 0x7ff09415e320 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.557+0000 7ff09ac9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0940d2dc0 con 0x7ff0940d0b00 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.557+0000 7ff09ac9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff0940b9fe0 con 0x7ff0940d2290 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.557+0000 7ff099237640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09415e320 0x7ff0941f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09415e320 0x7ff0941f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38924/0 (socket says 172.21.15.113:38924) 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 -- 172.21.15.113:0/1779347619 learned_addr learned my addr 172.21.15.113:0/1779347619 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff093fff640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0940d2290 0x7ff0941f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff098a36640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0940d0b00 0x7ff09415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 -- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0940d2290 msgr2=0x7ff0941f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0940d2290 0x7ff0941f13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 -- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0940d0b00 msgr2=0x7ff09415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0940d0b00 0x7ff09415e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 -- 172.21.15.113:0/1779347619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09415ed70 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff099237640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09415e320 0x7ff0941f47a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0090b0 tx=0x7ff07c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff0917fa640 1 -- 172.21.15.113:0/1779347619 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c037040 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff0917fa640 1 -- 172.21.15.113:0/1779347619 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c042070 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff0917fa640 1 -- 172.21.15.113:0/1779347619 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c030bd0 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09415f050 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.558+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09415f610 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.559+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0940691a0 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.560+0000 7ff0917fa640 1 -- 172.21.15.113:0/1779347619 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff07c047460 con 0x7ff09415e320 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.560+0000 7ff0917fa640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff07406be60 0x7ff07406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.560+0000 7ff098a36640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff07406be60 0x7ff07406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.560+0000 7ff0917fa640 1 -- 172.21.15.113:0/1779347619 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7ff07c0ab730 con 0x7ff09415e320 2024-04-25T00:01:32.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.561+0000 7ff098a36640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff07406be60 0x7ff07406e320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff094161790 tx=0x7ff08c0083c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.565+0000 7ff0917fa640 1 -- 172.21.15.113:0/1779347619 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff07c07a430 con 0x7ff09415e320 2024-04-25T00:01:32.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.800+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff094158d60 con 0x7ff09415e320 2024-04-25T00:01:32.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.801+0000 7ff0917fa640 1 -- 172.21.15.113:0/1779347619 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7ff07c079d30 con 0x7ff09415e320 2024-04-25T00:01:32.801 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:32.801 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:32.801 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:01:32.801 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.801 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:01:32.801 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.803 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:01:32.804 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:01:32.804 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.804 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.804 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.804 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.805 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.806 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:01:32.808 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.809 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:01:32.810 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:01:32.811 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.812 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.813 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.814 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:01:32.815 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.802+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff07406be60 msgr2=0x7ff07406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.802+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff07406be60 0x7ff07406e320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff094161790 tx=0x7ff08c0083c0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.802+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09415e320 msgr2=0x7ff0941f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.802+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09415e320 0x7ff0941f47a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0090b0 tx=0x7ff07c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.802+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 shutdown_connections 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.802+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff07406be60 0x7ff07406e320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.802+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09415e320 0x7ff0941f47a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.803+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0940d2290 0x7ff0941f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.803+0000 7ff09ac9a640 1 --2- 172.21.15.113:0/1779347619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0940d0b00 0x7ff09415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.803+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 >> 172.21.15.113:0/1779347619 conn(0x7ff0940b5c20 msgr2=0x7ff0940b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.803+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 shutdown_connections 2024-04-25T00:01:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.803+0000 7ff09ac9a640 1 -- 172.21.15.113:0/1779347619 wait complete. 2024-04-25T00:01:32.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:32.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.915+0000 7fe9602b3640 1 Processor -- start 2024-04-25T00:01:32.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.915+0000 7fe9602b3640 1 -- start start 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe9602b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 0x7fe9580b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe9602b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe9581453c0 0x7fe95806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe9602b3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9580cfad0 con 0x7fe95806e510 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe9602b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe958145c40 con 0x7fe9581453c0 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe9602b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe95806e940 con 0x7fe958144d60 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe95e850640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe95806e510 0x7fe95815fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43300/0 (socket says 172.21.15.113:43300) 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe95e850640 1 -- 172.21.15.113:0/1384626292 learned_addr learned my addr 172.21.15.113:0/1384626292 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe95e04f640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 0x7fe9580b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe95d84e640 1 --2- 172.21.15.113:0/1384626292 >> v2:172.21.15.178:3300/0 conn(0x7fe9581453c0 0x7fe95806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189684367 0 0) 0x7fe9580cfad0 con 0x7fe95806e510 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.916+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe944003520 con 0x7fe95806e510 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 314740997 0 0) 0x7fe944003520 con 0x7fe95806e510 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 msgr2=0x7fe9580b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 0x7fe9580b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 >> v2:172.21.15.178:3300/0 conn(0x7fe9581453c0 msgr2=0x7fe95806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 --2- 172.21.15.113:0/1384626292 >> v2:172.21.15.178:3300/0 conn(0x7fe9581453c0 0x7fe95806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9580cf230 con 0x7fe95806e510 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe950001670 con 0x7fe95806e510 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 >> v1:172.21.15.113:6789/0 conn(0x7fe95806e510 legacy=0x7fe95815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 0x7fe9440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe944003f30 0x7fe944004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe944004d40 0x7fe9440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe958145c40 con 0x7fe944004d40 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe95806e940 con 0x7fe944003f30 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe94400ba10 con 0x7fe958144d60 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95e850640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe944004d40 0x7fe9440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95d84e640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe944003f30 0x7fe944004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95e04f640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 0x7fe9440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95e850640 1 -- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 msgr2=0x7fe9440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95e850640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe958144d60 0x7fe9440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95e850640 1 -- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe944003f30 msgr2=0x7fe944004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95e850640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe944003f30 0x7fe944004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.917+0000 7fe95e850640 1 -- 172.21.15.113:0/1384626292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe944003520 con 0x7fe944004d40 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.918+0000 7fe95e850640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe944004d40 0x7fe9440051c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c270 tx=0x7fe944005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2f88016256c695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.918+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950007d30 con 0x7fe944004d40 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.918+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe950005ad0 con 0x7fe944004d40 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.918+0000 7fe95d04d640 1 -- 172.21.15.113:0/1384626292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950007070 con 0x7fe944004d40 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.918+0000 7fe9602b3640 1 -- 172.21.15.113:0/1384626292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe944004d40 msgr2=0x7fe9440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.918+0000 7fe9602b3640 1 --2- 172.21.15.113:0/1384626292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe944004d40 0x7fe9440051c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c270 tx=0x7fe944005830 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.918+0000 7fe95e850640 1 -- 172.21.15.113:0/1384626292 reap_dead start 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.919+0000 7fe9602b3640 1 -- 172.21.15.113:0/1384626292 shutdown_connections 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.919+0000 7fe9602b3640 1 -- 172.21.15.113:0/1384626292 >> 172.21.15.113:0/1384626292 conn(0x7fe9580b1a50 msgr2=0x7fe9580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.919+0000 7fe9602b3640 1 -- 172.21.15.113:0/1384626292 shutdown_connections 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.919+0000 7fe9602b3640 1 -- 172.21.15.113:0/1384626292 wait complete. 2024-04-25T00:01:32.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.920+0000 7fe9602b3640 1 Processor -- start 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.920+0000 7fe9602b3640 1 -- start start 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.920+0000 7fe9602b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944004d40 0x7fe95815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.920+0000 7fe9602b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe958071bd0 0x7fe958072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.920+0000 7fe9602b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95815b290 0x7fe95815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe9602b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe958070e10 con 0x7fe958071bd0 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe9602b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9580cf7f0 con 0x7fe95815b290 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe9602b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe958145140 con 0x7fe944004d40 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe958071bd0 0x7fe958072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe958071bd0 0x7fe958072050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38944/0 (socket says 172.21.15.113:38944) 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 -- 172.21.15.113:0/423348288 learned_addr learned my addr 172.21.15.113:0/423348288 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95e04f640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944004d40 0x7fe95815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95e850640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95815b290 0x7fe95815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 -- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944004d40 msgr2=0x7fe95815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944004d40 0x7fe95815ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 -- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95815b290 msgr2=0x7fe95815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95815b290 0x7fe95815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 -- 172.21.15.113:0/423348288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95815f670 con 0x7fe958071bd0 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95e04f640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944004d40 0x7fe95815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95e850640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95815b290 0x7fe95815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe95d84e640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe958071bd0 0x7fe958072050 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe948011960 tx=0x7fe948011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe94effd640 1 -- 172.21.15.113:0/423348288 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94800faf0 con 0x7fe958071bd0 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe94effd640 1 -- 172.21.15.113:0/423348288 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94800fc90 con 0x7fe958071bd0 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95815f950 con 0x7fe958071bd0 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe94effd640 1 -- 172.21.15.113:0/423348288 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948014440 con 0x7fe958071bd0 2024-04-25T00:01:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.921+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95815ff10 con 0x7fe958071bd0 2024-04-25T00:01:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.922+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe958072590 con 0x7fe958071bd0 2024-04-25T00:01:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.923+0000 7fe94effd640 1 -- 172.21.15.113:0/423348288 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe948012040 con 0x7fe958071bd0 2024-04-25T00:01:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.923+0000 7fe94effd640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe93c06be60 0x7fe93c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.923+0000 7fe94effd640 1 -- 172.21.15.113:0/423348288 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7fe94808b1c0 con 0x7fe958071bd0 2024-04-25T00:01:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.923+0000 7fe95e04f640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe93c06be60 0x7fe93c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.924+0000 7fe95e04f640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe93c06be60 0x7fe93c06e320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe9581f7330 tx=0x7fe95400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:32.928+0000 7fe94effd640 1 -- 172.21.15.113:0/423348288 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe948019030 con 0x7fe958071bd0 2024-04-25T00:01:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.101+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9580691a0 con 0x7fe958071bd0 2024-04-25T00:01:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.101+0000 7fe94effd640 1 -- 172.21.15.113:0/423348288 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+12939 (secure 0 0 0) 0x7fe948059ec0 con 0x7fe958071bd0 2024-04-25T00:01:33.102 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:33.102 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:32.436467+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe93c06be60 msgr2=0x7fe93c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe93c06be60 0x7fe93c06e320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe9581f7330 tx=0x7fe95400b040 comp rx=0 tx=0).stop 2024-04-25T00:01:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe958071bd0 msgr2=0x7fe958072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe958071bd0 0x7fe958072050 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe948011960 tx=0x7fe948011e20 comp rx=0 tx=0).stop 2024-04-25T00:01:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 shutdown_connections 2024-04-25T00:01:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe93c06be60 0x7fe93c06e320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95815b290 0x7fe95815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe958071bd0 0x7fe958072050 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.103+0000 7fe9602b3640 1 --2- 172.21.15.113:0/423348288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944004d40 0x7fe95815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.104+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 >> 172.21.15.113:0/423348288 conn(0x7fe9580b1a50 msgr2=0x7fe958143b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.104+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 shutdown_connections 2024-04-25T00:01:33.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.104+0000 7fe9602b3640 1 -- 172.21.15.113:0/423348288 wait complete. 2024-04-25T00:01:33.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:33.115 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:33.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.215+0000 7effb4db7640 1 Processor -- start 2024-04-25T00:01:33.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.215+0000 7effb4db7640 1 -- start start 2024-04-25T00:01:33.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effb4db7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e2d0 0x7effb015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effb4db7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7effb00b3080 0x7effb00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effb4db7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb015f4f0 con 0x7effb015e6b0 2024-04-25T00:01:33.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effb4db7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7effb00cde00 con 0x7effb00b3080 2024-04-25T00:01:33.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effb4db7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7effb015f7d0 con 0x7effb015e2d0 2024-04-25T00:01:33.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effadd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7effb00b3080 0x7effb00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effadd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7effb00b3080 0x7effb00b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38952/0 (socket says 172.21.15.113:38952) 2024-04-25T00:01:33.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effadd9b640 1 -- 172.21.15.113:0/248468074 learned_addr learned my addr 172.21.15.113:0/248468074 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:33.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effae59c640 1 --2- 172.21.15.113:0/248468074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e2d0 0x7effb015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effadd9b640 1 -- 172.21.15.113:0/248468074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e2d0 msgr2=0x7effb015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effadd9b640 1 --2- 172.21.15.113:0/248468074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e2d0 0x7effb015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effadd9b640 1 -- 172.21.15.113:0/248468074 >> v1:172.21.15.178:6789/0 conn(0x7effb015e6b0 legacy=0x7effb0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.216+0000 7effadd9b640 1 -- 172.21.15.113:0/248468074 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb015fab0 con 0x7effb00b3080 2024-04-25T00:01:33.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.217+0000 7effadd9b640 1 --2- 172.21.15.113:0/248468074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb00b3080 0x7effb00b4330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7effa4009870 tx=0x7effa402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe376aa7579c61d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.217+0000 7effad59a640 1 -- 172.21.15.113:0/248468074 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4037070 con 0x7effb00b3080 2024-04-25T00:01:33.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.217+0000 7effad59a640 1 -- 172.21.15.113:0/248468074 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa4032440 con 0x7effb00b3080 2024-04-25T00:01:33.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.217+0000 7effad59a640 1 -- 172.21.15.113:0/248468074 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4032810 con 0x7effb00b3080 2024-04-25T00:01:33.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.217+0000 7effb4db7640 1 -- 172.21.15.113:0/248468074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb00b3080 msgr2=0x7effb00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.217+0000 7effb4db7640 1 --2- 172.21.15.113:0/248468074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb00b3080 0x7effb00b4330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7effa4009870 tx=0x7effa402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.218+0000 7effb4db7640 1 -- 172.21.15.113:0/248468074 shutdown_connections 2024-04-25T00:01:33.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.218+0000 7effb4db7640 1 --2- 172.21.15.113:0/248468074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb00b3080 0x7effb00b4330 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.218+0000 7effb4db7640 1 --2- 172.21.15.113:0/248468074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e2d0 0x7effb015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.218+0000 7effb4db7640 1 -- 172.21.15.113:0/248468074 >> 172.21.15.113:0/248468074 conn(0x7effb006b0f0 msgr2=0x7effb006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.218+0000 7effb4db7640 1 -- 172.21.15.113:0/248468074 shutdown_connections 2024-04-25T00:01:33.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.218+0000 7effb4db7640 1 -- 172.21.15.113:0/248468074 wait complete. 2024-04-25T00:01:33.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.219+0000 7effb4db7640 1 Processor -- start 2024-04-25T00:01:33.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.219+0000 7effb4db7640 1 -- start start 2024-04-25T00:01:33.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effb00b3080 0x7effb014dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb015e2d0 0x7effb01f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e6b0 0x7effb0152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7effb015f2b0 con 0x7effb015e2d0 2024-04-25T00:01:33.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effb015f130 con 0x7effb00b3080 2024-04-25T00:01:33.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7effb00cf2a0 con 0x7effb015e6b0 2024-04-25T00:01:33.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb015e2d0 0x7effb01f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb015e2d0 0x7effb01f2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38958/0 (socket says 172.21.15.113:38958) 2024-04-25T00:01:33.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 -- 172.21.15.113:0/3813650780 learned_addr learned my addr 172.21.15.113:0/3813650780 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:33.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effae59c640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effb00b3080 0x7effb014dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effaed9d640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e6b0 0x7effb0152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 -- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e6b0 msgr2=0x7effb0152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e6b0 0x7effb0152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 -- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effb00b3080 msgr2=0x7effb014dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effb00b3080 0x7effb014dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 -- 172.21.15.113:0/3813650780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb01f1980 con 0x7effb015e2d0 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effadd9b640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb015e2d0 0x7effb01f2a20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7effa4000c00 tx=0x7effa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7eff9effd640 1 -- 172.21.15.113:0/3813650780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4037040 con 0x7effb015e2d0 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7eff9effd640 1 -- 172.21.15.113:0/3813650780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa4042070 con 0x7effb015e2d0 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb01f1c60 con 0x7effb015e2d0 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7eff9effd640 1 -- 172.21.15.113:0/3813650780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4030880 con 0x7effb015e2d0 2024-04-25T00:01:33.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.220+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb01f2220 con 0x7effb015e2d0 2024-04-25T00:01:33.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.221+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb00691a0 con 0x7effb015e2d0 2024-04-25T00:01:33.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.222+0000 7eff9effd640 1 -- 172.21.15.113:0/3813650780 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effa402ad10 con 0x7effb015e2d0 2024-04-25T00:01:33.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.222+0000 7eff9effd640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff9006be30 0x7eff9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.222+0000 7effae59c640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff9006be30 0x7eff9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.222+0000 7eff9effd640 1 -- 172.21.15.113:0/3813650780 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7effa40aaee0 con 0x7effb015e2d0 2024-04-25T00:01:33.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.223+0000 7effae59c640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff9006be30 0x7eff9006e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effa0009b40 tx=0x7effa0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.227+0000 7eff9effd640 1 -- 172.21.15.113:0/3813650780 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7effa4079620 con 0x7effb015e2d0 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.399+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7effb00b4030 con 0x7effb015e2d0 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.399+0000 7eff9effd640 1 -- 172.21.15.113:0/3813650780 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7effa407d920 con 0x7effb015e2d0 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff9006be30 msgr2=0x7eff9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff9006be30 0x7eff9006e2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effa0009b40 tx=0x7effa0009190 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb015e2d0 msgr2=0x7effb01f2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb015e2d0 0x7effb01f2a20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7effa4000c00 tx=0x7effa40046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 shutdown_connections 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7eff9006be30 0x7eff9006e2f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effb015e6b0 0x7effb0152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effb015e2d0 0x7effb01f2a20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 --2- 172.21.15.113:0/3813650780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effb00b3080 0x7effb014dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 >> 172.21.15.113:0/3813650780 conn(0x7effb006b0f0 msgr2=0x7effb00b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 shutdown_connections 2024-04-25T00:01:33.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.401+0000 7effb4db7640 1 -- 172.21.15.113:0/3813650780 wait complete. 2024-04-25T00:01:33.413 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:01:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.510+0000 7f2e9e0f5640 1 Processor -- start 2024-04-25T00:01:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.510+0000 7f2e9e0f5640 1 -- start start 2024-04-25T00:01:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e9e0f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e98144910 0x7f2e980b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e9e0f5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e9806e6e0 0x7f2e980cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e9e0f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e98144cf0 con 0x7f2e980b5240 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e9e0f5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2e980b4510 con 0x7f2e9806e6e0 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e9e0f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e980b2f30 con 0x7f2e98144910 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e97fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e9806e6e0 0x7f2e980cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e97fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e9806e6e0 0x7f2e980cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38964/0 (socket says 172.21.15.113:38964) 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e97fff640 1 -- 172.21.15.113:0/1671084144 learned_addr learned my addr 172.21.15.113:0/1671084144 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e96ffd640 1 --1- 172.21.15.113:0/1671084144 >> v1:172.21.15.113:6789/0 conn(0x7f2e980b5240 0x7f2e9806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43306/0 (socket says 172.21.15.113:43306) 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e977fe640 1 --2- 172.21.15.113:0/1671084144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e98144910 0x7f2e980b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e97fff640 1 -- 172.21.15.113:0/1671084144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e98144910 msgr2=0x7f2e980b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e97fff640 1 --2- 172.21.15.113:0/1671084144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e98144910 0x7f2e980b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e967fc640 1 -- 172.21.15.113:0/1671084144 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224648281 0 0) 0x7f2e98144cf0 con 0x7f2e980b5240 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e97fff640 1 -- 172.21.15.113:0/1671084144 >> v1:172.21.15.113:6789/0 conn(0x7f2e980b5240 legacy=0x7f2e9806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.511+0000 7f2e97fff640 1 -- 172.21.15.113:0/1671084144 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e980b3210 con 0x7f2e9806e6e0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.512+0000 7f2e97fff640 1 --2- 172.21.15.113:0/1671084144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806e6e0 0x7f2e980cd500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000d150 tx=0x7f2e9002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9162550757f1cb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.512+0000 7f2e967fc640 1 -- 172.21.15.113:0/1671084144 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90009600 con 0x7f2e9806e6e0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.512+0000 7f2e967fc640 1 -- 172.21.15.113:0/1671084144 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e90037070 con 0x7f2e9806e6e0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.512+0000 7f2e967fc640 1 -- 172.21.15.113:0/1671084144 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e900099c0 con 0x7f2e9806e6e0 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1671084144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806e6e0 msgr2=0x7f2e980cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1671084144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806e6e0 0x7f2e980cd500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000d150 tx=0x7f2e9002c930 comp rx=0 tx=0).stop 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1671084144 shutdown_connections 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1671084144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806e6e0 0x7f2e980cd500 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1671084144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e98144910 0x7f2e980b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1671084144 >> 172.21.15.113:0/1671084144 conn(0x7f2e980b1a50 msgr2=0x7f2e980b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1671084144 shutdown_connections 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.513+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1671084144 wait complete. 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.514+0000 7f2e9e0f5640 1 Processor -- start 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.514+0000 7f2e9e0f5640 1 -- start start 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e9e0f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e9806e6e0 0x7f2e980cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e9e0f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e980b5240 0x7f2e980ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e9e0f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e98144910 0x7f2e9813ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e9e0f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e980b4da0 con 0x7f2e98144910 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e9e0f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e980b37d0 con 0x7f2e980b5240 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e9e0f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e980b4f20 con 0x7f2e9806e6e0 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e98144910 0x7f2e9813ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e98144910 0x7f2e9813ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38980/0 (socket says 172.21.15.113:38980) 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 -- 172.21.15.113:0/1091994092 learned_addr learned my addr 172.21.15.113:0/1091994092 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e977fe640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e9806e6e0 0x7f2e980cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 -- 172.21.15.113:0/1091994092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e9806e6e0 msgr2=0x7f2e980cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e9806e6e0 0x7f2e980cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 -- 172.21.15.113:0/1091994092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e980b5240 msgr2=0x7f2e980ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e980b5240 0x7f2e980ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.515+0000 7f2e97fff640 1 -- 172.21.15.113:0/1091994092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e980ce1c0 con 0x7f2e98144910 2024-04-25T00:01:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.516+0000 7f2e97fff640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e98144910 0x7f2e9813ffb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000d270 tx=0x7f2e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.516+0000 7f2e7ffff640 1 -- 172.21.15.113:0/1091994092 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90009dc0 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.516+0000 7f2e7ffff640 1 -- 172.21.15.113:0/1091994092 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e90032420 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.516+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e980ae270 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.516+0000 7f2e7ffff640 1 -- 172.21.15.113:0/1091994092 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90032810 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.516+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e980ae830 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.517+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e980691a0 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.517+0000 7f2e7ffff640 1 -- 172.21.15.113:0/1091994092 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e900329b0 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.518+0000 7f2e7ffff640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e8406be30 0x7f2e8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.518+0000 7f2e977fe640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e8406be30 0x7f2e8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.518+0000 7f2e7ffff640 1 -- 172.21.15.113:0/1091994092 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f2e900ab910 con 0x7f2e98144910 2024-04-25T00:01:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.518+0000 7f2e977fe640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e8406be30 0x7f2e8406e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2e98140bc0 tx=0x7f2e88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.522+0000 7f2e7ffff640 1 -- 172.21.15.113:0/1091994092 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e9007a6a0 con 0x7f2e98144910 2024-04-25T00:01:33.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.698+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2e9806e460 con 0x7f2e98144910 2024-04-25T00:01:33.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.698+0000 7f2e7ffff640 1 -- 172.21.15.113:0/1091994092 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7f2e90079fa0 con 0x7f2e98144910 2024-04-25T00:01:33.699 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:33.699 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:01:33.699 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:01:33.699 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:01:33.699 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:01:33.700 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:01:33.700 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:01:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e8406be30 msgr2=0x7f2e8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e8406be30 0x7f2e8406e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2e98140bc0 tx=0x7f2e88008040 comp rx=0 tx=0).stop 2024-04-25T00:01:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e98144910 msgr2=0x7f2e9813ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e98144910 0x7f2e9813ffb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000d270 tx=0x7f2e90004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:33.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 shutdown_connections 2024-04-25T00:01:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e8406be30 0x7f2e8406e2f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e980b5240 0x7f2e980ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e98144910 0x7f2e9813ffb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 --2- 172.21.15.113:0/1091994092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e9806e6e0 0x7f2e980cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 >> 172.21.15.113:0/1091994092 conn(0x7f2e980b1a50 msgr2=0x7f2e98071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 shutdown_connections 2024-04-25T00:01:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.700+0000 7f2e9e0f5640 1 -- 172.21.15.113:0/1091994092 wait complete. 2024-04-25T00:01:33.711 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:01:33.712 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.813+0000 7f81decc3640 1 Processor -- start 2024-04-25T00:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.813+0000 7f81decc3640 1 -- start start 2024-04-25T00:01:33.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.813+0000 7f81decc3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d8161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.813+0000 7f81decc3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81d80b63b0 0x7f81d80d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.813+0000 7f81decc3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d81626f0 con 0x7f81d80b9bb0 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.813+0000 7f81decc3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f81d81629d0 con 0x7f81d80b63b0 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.813+0000 7f81decc3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81d8162cb0 con 0x7f81d815e510 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dd260640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81d80b63b0 0x7f81d80d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dd260640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81d80b63b0 0x7f81d80d2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38982/0 (socket says 172.21.15.113:38982) 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dd260640 1 -- 172.21.15.113:0/1924271547 learned_addr learned my addr 172.21.15.113:0/1924271547 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dca5f640 1 --2- 172.21.15.113:0/1924271547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d8161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dd260640 1 -- 172.21.15.113:0/1924271547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 msgr2=0x7f81d8161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dd260640 1 --2- 172.21.15.113:0/1924271547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d8161ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dd260640 1 -- 172.21.15.113:0/1924271547 >> v1:172.21.15.178:6789/0 conn(0x7f81d80b9bb0 legacy=0x7f81d80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dd260640 1 -- 172.21.15.113:0/1924271547 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d80d2660 con 0x7f81d80b63b0 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.814+0000 7f81dca5f640 1 --2- 172.21.15.113:0/1924271547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d8161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81dd260640 1 --2- 172.21.15.113:0/1924271547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b63b0 0x7f81d80d2e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009e50 tx=0x7f81d002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a32d86bf7abd24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81d77fe640 1 -- 172.21.15.113:0/1924271547 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0036070 con 0x7f81d80b63b0 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81d77fe640 1 -- 172.21.15.113:0/1924271547 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d0031600 con 0x7f81d80b63b0 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81d77fe640 1 -- 172.21.15.113:0/1924271547 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00319d0 con 0x7f81d80b63b0 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 -- 172.21.15.113:0/1924271547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b63b0 msgr2=0x7f81d80d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 --2- 172.21.15.113:0/1924271547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b63b0 0x7f81d80d2e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009e50 tx=0x7f81d002a680 comp rx=0 tx=0).stop 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 -- 172.21.15.113:0/1924271547 shutdown_connections 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 --2- 172.21.15.113:0/1924271547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b63b0 0x7f81d80d2e60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 --2- 172.21.15.113:0/1924271547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d8161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 -- 172.21.15.113:0/1924271547 >> 172.21.15.113:0/1924271547 conn(0x7f81d806b720 msgr2=0x7f81d80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 -- 172.21.15.113:0/1924271547 shutdown_connections 2024-04-25T00:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.815+0000 7f81decc3640 1 -- 172.21.15.113:0/1924271547 wait complete. 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.816+0000 7f81decc3640 1 Processor -- start 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.816+0000 7f81decc3640 1 -- start start 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81decc3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d80b63b0 0x7f81d80c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81decc3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b9bb0 0x7f81d815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81decc3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81decc3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81d80b73c0 con 0x7f81d80b9bb0 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81decc3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81623e0 con 0x7f81d80b63b0 2024-04-25T00:01:33.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81decc3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81d80b7540 con 0x7f81d815e510 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b9bb0 0x7f81d815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b9bb0 0x7f81d815a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38988/0 (socket says 172.21.15.113:38988) 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.817+0000 7f81d7fff640 1 -- 172.21.15.113:0/3662784204 learned_addr learned my addr 172.21.15.113:0/3662784204 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81dca5f640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d80b63b0 0x7f81d80c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81dd260640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d7fff640 1 -- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 msgr2=0x7f81d815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d7fff640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d7fff640 1 -- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d80b63b0 msgr2=0x7f81d80c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d7fff640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d80b63b0 0x7f81d80c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d7fff640 1 -- 172.21.15.113:0/3662784204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d8161780 con 0x7f81d80b9bb0 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d7fff640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b9bb0 0x7f81d815a540 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f81c400c440 tx=0x7f81c4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d57fa640 1 -- 172.21.15.113:0/3662784204 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40052d0 con 0x7f81d80b9bb0 2024-04-25T00:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d57fa640 1 -- 172.21.15.113:0/3662784204 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c4005470 con 0x7f81d80b9bb0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81d57fa640 1 -- 172.21.15.113:0/3662784204 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4007260 con 0x7f81d80b9bb0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d81611c0 con 0x7f81d80b9bb0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.818+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d8140210 con 0x7f81d80b9bb0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.819+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d80691a0 con 0x7f81d80b9bb0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.819+0000 7f81d57fa640 1 -- 172.21.15.113:0/3662784204 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81c4007400 con 0x7f81d80b9bb0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.820+0000 7f81d57fa640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.820+0000 7f81d57fa640 1 -- 172.21.15.113:0/3662784204 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f81c408a090 con 0x7f81d80b9bb0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.820+0000 7f81dca5f640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.820+0000 7f81dca5f640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00d8b0 tx=0x7f81cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.824+0000 7f81d57fa640 1 -- 172.21.15.113:0/3662784204 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81c4058d30 con 0x7f81d80b9bb0 2024-04-25T00:01:33.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:33.995+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f81d80c7210 con 0x7f81d80b9bb0 2024-04-25T00:01:34.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.445+0000 7f81d57fa640 1 -- 172.21.15.113:0/3662784204 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7f81c4058630 con 0x7f81d80b9bb0 2024-04-25T00:01:34.446 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:01:34.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 msgr2=0x7f81bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:34.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00d8b0 tx=0x7f81cc008040 comp rx=0 tx=0).stop 2024-04-25T00:01:34.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b9bb0 msgr2=0x7f81d815a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:34.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b9bb0 0x7f81d815a540 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f81c400c440 tx=0x7f81c4001e70 comp rx=0 tx=0).stop 2024-04-25T00:01:34.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 shutdown_connections 2024-04-25T00:01:34.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81bc06be30 0x7f81bc06e2f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:34.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81d815e510 0x7f81d815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:34.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81d80b9bb0 0x7f81d815a540 secure :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f81c400c440 tx=0x7f81c4001e70 comp rx=0 tx=0).stop 2024-04-25T00:01:34.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 --2- 172.21.15.113:0/3662784204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81d80b63b0 0x7f81d80c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:34.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.449+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 >> 172.21.15.113:0/3662784204 conn(0x7f81d806b720 msgr2=0x7f81d80cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:34.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.450+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 shutdown_connections 2024-04-25T00:01:34.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.451+0000 7f81decc3640 1 -- 172.21.15.113:0/3662784204 wait complete. 2024-04-25T00:01:34.464 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:01:34.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.568+0000 7f40baa6b640 1 Processor -- start 2024-04-25T00:01:34.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.568+0000 7f40baa6b640 1 -- start start 2024-04-25T00:01:34.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40baa6b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40b406d5b0 0x7f40b415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:34.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40baa6b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b4071170 con 0x7f40b415ba50 2024-04-25T00:01:34.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40baa6b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b4071450 con 0x7f40b415b670 2024-04-25T00:01:34.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40baa6b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f40b4071730 con 0x7f40b406d5b0 2024-04-25T00:01:34.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b9008640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40b406d5b0 0x7f40b415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:34.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f40b415ba50 0x7f40b406cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43314/0 (socket says 172.21.15.113:43314) 2024-04-25T00:01:34.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b3fff640 1 -- 172.21.15.113:0/3259274923 learned_addr learned my addr 172.21.15.113:0/3259274923 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:34.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b9008640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40b406d5b0 0x7f40b415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38990/0 (socket says 172.21.15.113:38990) 2024-04-25T00:01:34.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b9008640 1 -- 172.21.15.113:0/3259274923 >> v1:172.21.15.178:6789/0 conn(0x7f40b415b670 legacy=0x7f40b40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:34.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b9008640 1 -- 172.21.15.113:0/3259274923 >> v1:172.21.15.113:6789/0 conn(0x7f40b415ba50 legacy=0x7f40b406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:34.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b9008640 1 -- 172.21.15.113:0/3259274923 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b4071a10 con 0x7f40b406d5b0 2024-04-25T00:01:34.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.569+0000 7f40b37fe640 1 -- 172.21.15.113:0/3259274923 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1656447604 0 0) 0x7f40b4071170 con 0x7f40b415ba50 2024-04-25T00:01:34.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40b9008640 1 --2- 172.21.15.113:0/3259274923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b406d5b0 0x7f40b415bff0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f40a40097a0 tx=0x7f40a402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fa7259c2e9d6af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40b37fe640 1 -- 172.21.15.113:0/3259274923 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4037070 con 0x7f40b406d5b0 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40b37fe640 1 -- 172.21.15.113:0/3259274923 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a402adc0 con 0x7f40b406d5b0 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40b37fe640 1 -- 172.21.15.113:0/3259274923 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4032660 con 0x7f40b406d5b0 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40baa6b640 1 -- 172.21.15.113:0/3259274923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b406d5b0 msgr2=0x7f40b415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3259274923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b406d5b0 0x7f40b415bff0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f40a40097a0 tx=0x7f40a402a0b0 comp rx=0 tx=0).stop 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40baa6b640 1 -- 172.21.15.113:0/3259274923 shutdown_connections 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3259274923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b406d5b0 0x7f40b415bff0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40baa6b640 1 -- 172.21.15.113:0/3259274923 >> 172.21.15.113:0/3259274923 conn(0x7f40b406b0f0 msgr2=0x7f40b406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40baa6b640 1 -- 172.21.15.113:0/3259274923 shutdown_connections 2024-04-25T00:01:34.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.570+0000 7f40baa6b640 1 -- 172.21.15.113:0/3259274923 wait complete. 2024-04-25T00:01:34.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 Processor -- start 2024-04-25T00:01:34.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 -- start start 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b406d5b0 0x7f40b40cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b415b670 0x7f40b413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b415ba50 0x7f40b413e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40d0200 con 0x7f40b415b670 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40d0080 con 0x7f40b415ba50 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40baa6b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40d0380 con 0x7f40b406d5b0 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.572+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b415b670 0x7f40b413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b415b670 0x7f40b413d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39002/0 (socket says 172.21.15.113:39002) 2024-04-25T00:01:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 -- 172.21.15.113:0/3730463858 learned_addr learned my addr 172.21.15.113:0/3730463858 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 -- 172.21.15.113:0/3730463858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b406d5b0 msgr2=0x7f40b40cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b406d5b0 0x7f40b40cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 -- 172.21.15.113:0/3730463858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b415ba50 msgr2=0x7f40b413e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b415ba50 0x7f40b413e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 -- 172.21.15.113:0/3730463858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b40cd580 con 0x7f40b415b670 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b3fff640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b415b670 0x7f40b413d670 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c430 tx=0x7f409c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b17fa640 1 -- 172.21.15.113:0/3730463858 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0074e0 con 0x7f40b415b670 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b17fa640 1 -- 172.21.15.113:0/3730463858 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f409c0047e0 con 0x7f40b415b670 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b413fee0 con 0x7f40b415b670 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40b17fa640 1 -- 172.21.15.113:0/3730463858 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c004bd0 con 0x7f40b415b670 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.573+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41404a0 con 0x7f40b415b670 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.574+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b40691a0 con 0x7f40b415b670 2024-04-25T00:01:34.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.575+0000 7f40b17fa640 1 -- 172.21.15.113:0/3730463858 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f409c011070 con 0x7f40b415b670 2024-04-25T00:01:34.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.575+0000 7f40b17fa640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f409406be30 0x7f409406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:34.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.575+0000 7f40b17fa640 1 -- 172.21.15.113:0/3730463858 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5227+0+0 (secure 0 0 0) 0x7f409c08a8f0 con 0x7f40b415b670 2024-04-25T00:01:34.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.575+0000 7f40b8807640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f409406be30 0x7f409406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:34.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.575+0000 7f40b8807640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f409406be30 0x7f409406e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40b415b060 tx=0x7f40ac007d10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:34.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.579+0000 7f40b17fa640 1 -- 172.21.15.113:0/3730463858 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f409c059480 con 0x7f40b415b670 2024-04-25T00:01:34.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:34.756+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f40b413fc90 con 0x7f40b415b670 2024-04-25T00:01:35.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.451+0000 7f40b17fa640 1 -- 172.21.15.113:0/3730463858 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f409c058d80 con 0x7f40b415b670 2024-04-25T00:01:35.451 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:01:35.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f409406be30 msgr2=0x7f409406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f409406be30 0x7f409406e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40b415b060 tx=0x7f40ac007d10 comp rx=0 tx=0).stop 2024-04-25T00:01:35.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b415b670 msgr2=0x7f40b413d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b415b670 0x7f40b413d670 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c430 tx=0x7f409c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 shutdown_connections 2024-04-25T00:01:35.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f409406be30 0x7f409406e2f0 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40b415b060 tx=0x7f40ac007d10 comp rx=0 tx=0).stop 2024-04-25T00:01:35.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b415ba50 0x7f40b413e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40b415b670 0x7f40b413d670 secure :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c430 tx=0x7f409c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 --2- 172.21.15.113:0/3730463858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b406d5b0 0x7f40b40cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.455+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 >> 172.21.15.113:0/3730463858 conn(0x7f40b406b0f0 msgr2=0x7f40b415a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:35.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.460+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 shutdown_connections 2024-04-25T00:01:35.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.463+0000 7f40baa6b640 1 -- 172.21.15.113:0/3730463858 wait complete. 2024-04-25T00:01:35.483 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:01:35.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.587+0000 7fbad297d640 1 Processor -- start 2024-04-25T00:01:35.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.587+0000 7fbad297d640 1 -- start start 2024-04-25T00:01:35.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbad297d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbacc15b670 0x7fbacc15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbad297d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15bff0 0x7fbacc0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbad297d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc071170 con 0x7fbacc0cc740 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbad297d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc15c470 con 0x7fbacc15b670 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbad297d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc071450 con 0x7fbacc15bff0 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbad0f1a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbacc0cc740 0x7fbacc06cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43318/0 (socket says 172.21.15.113:43318) 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbad0f1a640 1 -- 172.21.15.113:0/3505775061 learned_addr learned my addr 172.21.15.113:0/3505775061 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbacb7fe640 1 --2- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15bff0 0x7fbacc0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbacbfff640 1 --2- 172.21.15.113:0/3505775061 >> v2:172.21.15.178:3300/0 conn(0x7fbacc15b670 0x7fbacc15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.588+0000 7fbacaffd640 1 -- 172.21.15.113:0/3505775061 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635535513 0 0) 0x7fbacc071170 con 0x7fbacc0cc740 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.589+0000 7fbacaffd640 1 -- 172.21.15.113:0/3505775061 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbab4003520 con 0x7fbacc0cc740 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.589+0000 7fbacbfff640 1 -- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15bff0 msgr2=0x7fbacc0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.589+0000 7fbacbfff640 1 --2- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15bff0 0x7fbacc0cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.589+0000 7fbacbfff640 1 -- 172.21.15.113:0/3505775061 >> v1:172.21.15.113:6789/0 conn(0x7fbacc0cc740 legacy=0x7fbacc06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.589+0000 7fbacbfff640 1 -- 172.21.15.113:0/3505775061 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc071730 con 0x7fbacc15b670 2024-04-25T00:01:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.589+0000 7fbacb7fe640 1 --2- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15bff0 0x7fbacc0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.589+0000 7fbacbfff640 1 --2- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15b670 0x7fbacc15ba50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0097a0 tx=0x7fbabc02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd4621ee39d9c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbacaffd640 1 -- 172.21.15.113:0/3505775061 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc037070 con 0x7fbacc15b670 2024-04-25T00:01:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbacaffd640 1 -- 172.21.15.113:0/3505775061 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc02adb0 con 0x7fbacc15b670 2024-04-25T00:01:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbacaffd640 1 -- 172.21.15.113:0/3505775061 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc032660 con 0x7fbacc15b670 2024-04-25T00:01:35.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 -- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15b670 msgr2=0x7fbacc15ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 --2- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15b670 0x7fbacc15ba50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0097a0 tx=0x7fbabc02a060 comp rx=0 tx=0).stop 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 -- 172.21.15.113:0/3505775061 shutdown_connections 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 --2- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15bff0 0x7fbacc0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 --2- 172.21.15.113:0/3505775061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15b670 0x7fbacc15ba50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 -- 172.21.15.113:0/3505775061 >> 172.21.15.113:0/3505775061 conn(0x7fbacc06b0f0 msgr2=0x7fbacc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 -- 172.21.15.113:0/3505775061 shutdown_connections 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.590+0000 7fbad297d640 1 -- 172.21.15.113:0/3505775061 wait complete. 2024-04-25T00:01:35.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.591+0000 7fbad297d640 1 Processor -- start 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.591+0000 7fbad297d640 1 -- start start 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbad297d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbacc0cc740 0x7fbacc1487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbad297d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15b670 0x7fbacc14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbad297d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15bff0 0x7fbacc156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbad297d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc06d730 con 0x7fbacc0cc740 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbad297d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc06d5b0 con 0x7fbacc15bff0 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbad297d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc0d0080 con 0x7fbacc15b670 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbacbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbacc0cc740 0x7fbacc1487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbacbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbacc0cc740 0x7fbacc1487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39018/0 (socket says 172.21.15.113:39018) 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.592+0000 7fbacbfff640 1 -- 172.21.15.113:0/1047533890 learned_addr learned my addr 172.21.15.113:0/1047533890 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbad0f1a640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15bff0 0x7fbacc156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacb7fe640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15b670 0x7fbacc14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacbfff640 1 -- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15b670 msgr2=0x7fbacc14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacbfff640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15b670 0x7fbacc14f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacbfff640 1 -- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15bff0 msgr2=0x7fbacc156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacbfff640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15bff0 0x7fbacc156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacbfff640 1 -- 172.21.15.113:0/1047533890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc1506f0 con 0x7fbacc0cc740 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacb7fe640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15b670 0x7fbacc14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbad0f1a640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15bff0 0x7fbacc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbacbfff640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbacc0cc740 0x7fbacc1487e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbabc035040 tx=0x7fbabc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbac8ff9640 1 -- 172.21.15.113:0/1047533890 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc041070 con 0x7fbacc0cc740 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbac8ff9640 1 -- 172.21.15.113:0/1047533890 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc03c680 con 0x7fbacc0cc740 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc155640 con 0x7fbacc0cc740 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbac8ff9640 1 -- 172.21.15.113:0/1047533890 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc037040 con 0x7fbacc0cc740 2024-04-25T00:01:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.593+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc148d20 con 0x7fbacc0cc740 2024-04-25T00:01:35.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.594+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc0691a0 con 0x7fbacc0cc740 2024-04-25T00:01:35.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.594+0000 7fbac8ff9640 1 -- 172.21.15.113:0/1047533890 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbabc03a070 con 0x7fbacc0cc740 2024-04-25T00:01:35.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.595+0000 7fbac8ff9640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbab806be30 0x7fbab806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.595+0000 7fbacb7fe640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbab806be30 0x7fbab806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.595+0000 7fbac8ff9640 1 -- 172.21.15.113:0/1047533890 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5609+0+0 (secure 0 0 0) 0x7fbabc0ab050 con 0x7fbacc0cc740 2024-04-25T00:01:35.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.595+0000 7fbacb7fe640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbab806be30 0x7fbab806e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbacc1551b0 tx=0x7fbac0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:35.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.599+0000 7fbac8ff9640 1 -- 172.21.15.113:0/1047533890 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbabc079a60 con 0x7fbacc0cc740 2024-04-25T00:01:35.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:35.810+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbacc155d00 con 0x7fbacc0cc740 2024-04-25T00:01:36.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.452+0000 7fbac8ff9640 1 -- 172.21.15.113:0/1047533890 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) ==== 172+0+0 (secure 0 0 0) 0x7fbabc079360 con 0x7fbacc0cc740 2024-04-25T00:01:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.457+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbab806be30 msgr2=0x7fbab806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.457+0000 7fbad297d640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbab806be30 0x7fbab806e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbacc1551b0 tx=0x7fbac0008040 comp rx=0 tx=0).stop 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.457+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbacc0cc740 msgr2=0x7fbacc1487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.457+0000 7fbad297d640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbacc0cc740 0x7fbacc1487e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbabc035040 tx=0x7fbabc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.458+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 shutdown_connections 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.458+0000 7fbad297d640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbab806be30 0x7fbab806e2f0 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbacc1551b0 tx=0x7fbac0008040 comp rx=0 tx=0).stop 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.458+0000 7fbad297d640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbacc15bff0 0x7fbacc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.458+0000 7fbad297d640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbacc15b670 0x7fbacc14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.458+0000 7fbad297d640 1 --2- 172.21.15.113:0/1047533890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbacc0cc740 0x7fbacc1487e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.458+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 >> 172.21.15.113:0/1047533890 conn(0x7fbacc06b0f0 msgr2=0x7fbacc165b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:36.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.459+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 shutdown_connections 2024-04-25T00:01:36.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.460+0000 7fbad297d640 1 -- 172.21.15.113:0/1047533890 wait complete. 2024-04-25T00:01:36.484 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:36.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.581+0000 7f0bd0afc640 1 Processor -- start 2024-04-25T00:01:36.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.581+0000 7f0bd0afc640 1 -- start start 2024-04-25T00:01:36.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bd0afc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0bcc072ea0 0x7f0bcc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bd0afc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0bcc0b3af0 0x7f0bcc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bd0afc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bcc0b2d50 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bd0afc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0bcc072a70 con 0x7f0bcc072ea0 2024-04-25T00:01:36.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bd0afc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0bcc0b3010 con 0x7f0bcc0b3af0 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc9d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0bcc072ea0 0x7f0bcc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc9d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0bcc072ea0 0x7f0bcc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39022/0 (socket says 172.21.15.113:39022) 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bca59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0bcc15a6b0 0x7f0bcc070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43322/0 (socket says 172.21.15.113:43322) 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc9d9b640 1 -- 172.21.15.113:0/2658519189 learned_addr learned my addr 172.21.15.113:0/2658519189 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bcad9d640 1 --2- 172.21.15.113:0/2658519189 >> v2:172.21.15.178:3300/0 conn(0x7f0bcc0b3af0 0x7f0bcc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc9d9b640 1 -- 172.21.15.113:0/2658519189 >> v2:172.21.15.178:3300/0 conn(0x7f0bcc0b3af0 msgr2=0x7f0bcc0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc9d9b640 1 --2- 172.21.15.113:0/2658519189 >> v2:172.21.15.178:3300/0 conn(0x7f0bcc0b3af0 0x7f0bcc0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc959a640 1 -- 172.21.15.113:0/2658519189 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963232583 0 0) 0x7f0bcc0b2d50 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc9d9b640 1 -- 172.21.15.113:0/2658519189 >> v1:172.21.15.113:6789/0 conn(0x7f0bcc15a6b0 legacy=0x7f0bcc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.582+0000 7f0bc9d9b640 1 -- 172.21.15.113:0/2658519189 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc0b32f0 con 0x7f0bcc072ea0 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.583+0000 7f0bc9d9b640 1 --2- 172.21.15.113:0/2658519189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc071410 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0cc5a0 tx=0x7f0bc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ea83646508c8be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.583+0000 7f0bc959a640 1 -- 172.21.15.113:0/2658519189 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0037070 con 0x7f0bcc072ea0 2024-04-25T00:01:36.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.583+0000 7f0bc959a640 1 -- 172.21.15.113:0/2658519189 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0032440 con 0x7f0bcc072ea0 2024-04-25T00:01:36.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.583+0000 7f0bc959a640 1 -- 172.21.15.113:0/2658519189 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0032810 con 0x7f0bcc072ea0 2024-04-25T00:01:36.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2658519189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc072ea0 msgr2=0x7f0bcc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2658519189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc071410 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0cc5a0 tx=0x7f0bc002a270 comp rx=0 tx=0).stop 2024-04-25T00:01:36.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2658519189 shutdown_connections 2024-04-25T00:01:36.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2658519189 >> v2:172.21.15.178:3300/0 conn(0x7f0bcc0b3af0 0x7f0bcc0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2658519189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc071410 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2658519189 >> 172.21.15.113:0/2658519189 conn(0x7f0bcc0b1a50 msgr2=0x7f0bcc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2658519189 shutdown_connections 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.584+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2658519189 wait complete. 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.585+0000 7f0bd0afc640 1 Processor -- start 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.585+0000 7f0bd0afc640 1 -- start start 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bd0afc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bd0afc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b3af0 0x7f0bcc1f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bd0afc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc15a6b0 0x7f0bcc0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bd0afc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc15aa90 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bd0afc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc072720 con 0x7f0bcc072ea0 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bd0afc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc15ac10 con 0x7f0bcc0b3af0 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc15a6b0 0x7f0bcc0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc15a6b0 0x7f0bcc0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39032/0 (socket says 172.21.15.113:39032) 2024-04-25T00:01:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 -- 172.21.15.113:0/2861455096 learned_addr learned my addr 172.21.15.113:0/2861455096 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bc9d9b640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b3af0 0x7f0bcc1f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bca59c640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 -- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b3af0 msgr2=0x7f0bcc1f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b3af0 0x7f0bcc1f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 -- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc072ea0 msgr2=0x7f0bcc1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 -- 172.21.15.113:0/2861455096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc0ceee0 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bca59c640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bc9d9b640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b3af0 0x7f0bcc1f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.586+0000 7f0bcad9d640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc15a6b0 0x7f0bcc0cd700 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400d580 tx=0x7f0bc400da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.587+0000 7f0bbaffd640 1 -- 172.21.15.113:0/2861455096 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40125f0 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.587+0000 7f0bbaffd640 1 -- 172.21.15.113:0/2861455096 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc4019070 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.587+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc1f41c0 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.587+0000 7f0bbaffd640 1 -- 172.21.15.113:0/2861455096 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40129f0 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.587+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1f4780 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.588+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc0691a0 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.588+0000 7f0bbaffd640 1 -- 172.21.15.113:0/2861455096 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bc4012b90 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.588+0000 7f0bbaffd640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bac06be30 0x7f0bac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.589+0000 7f0bbaffd640 1 -- 172.21.15.113:0/2861455096 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f0bc408b1b0 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.589+0000 7f0bca59c640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bac06be30 0x7f0bac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.589+0000 7f0bca59c640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bac06be30 0x7f0bac06e2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0012e0 tx=0x7f0bbc005bb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.593+0000 7f0bbaffd640 1 -- 172.21.15.113:0/2861455096 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0bcc0ceee0 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.767+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bcc0b2900 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.768+0000 7f0bbaffd640 1 -- 172.21.15.113:0/2861455096 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16726 (secure 0 0 0) 0x7f0bc4059b70 con 0x7f0bcc15a6b0 2024-04-25T00:01:36.768 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":59,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:35.811400+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:36.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bac06be30 msgr2=0x7f0bac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bac06be30 0x7f0bac06e2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0012e0 tx=0x7f0bbc005bb0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc15a6b0 msgr2=0x7f0bcc0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc15a6b0 0x7f0bcc0cd700 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400d580 tx=0x7f0bc400da40 comp rx=0 tx=0).stop 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 shutdown_connections 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bac06be30 0x7f0bac06e2f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc15a6b0 0x7f0bcc0cd700 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b3af0 0x7f0bcc1f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 --2- 172.21.15.113:0/2861455096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc072ea0 0x7f0bcc1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 >> 172.21.15.113:0/2861455096 conn(0x7f0bcc0b1a50 msgr2=0x7f0bcc157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 shutdown_connections 2024-04-25T00:01:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.770+0000 7f0bd0afc640 1 -- 172.21.15.113:0/2861455096 wait complete. 2024-04-25T00:01:36.781 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:36.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.877+0000 7fc0cee76640 1 Processor -- start 2024-04-25T00:01:36.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.877+0000 7fc0cee76640 1 -- start start 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.878+0000 7fc0cee76640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc0c8144d60 0x7fc0c80b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.878+0000 7fc0cee76640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c8071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.878+0000 7fc0cee76640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c80cfad0 con 0x7fc0c81453c0 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.878+0000 7fc0cee76640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c8145c40 con 0x7fc0c8144d60 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.878+0000 7fc0cee76640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80cf230 con 0x7fc0c806e6e0 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0ccc12640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc0c8144d60 0x7fc0c80b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0cd413640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c8071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0c7fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc0c81453c0 0x7fc0c806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:39992/0 (socket says 172.21.15.113:39992) 2024-04-25T00:01:36.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0cd413640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c8071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:40982/0 (socket says 172.21.15.113:40982) 2024-04-25T00:01:36.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0c7fff640 1 -- 172.21.15.113:0/2205013228 learned_addr learned my addr 172.21.15.113:0/2205013228 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:36.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0ccc12640 1 -- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 msgr2=0x7fc0c8071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:36.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0ccc12640 1 --2- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c8071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0ccc12640 1 -- 172.21.15.113:0/2205013228 >> v1:172.21.15.178:6789/0 conn(0x7fc0c81453c0 legacy=0x7fc0c806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0ccc12640 1 -- 172.21.15.113:0/2205013228 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80cf510 con 0x7fc0c8144d60 2024-04-25T00:01:36.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.879+0000 7fc0cd413640 1 --2- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c8071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:36.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0ccc12640 1 --2- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 0x7fc0c80b5390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95cd3d2953cbdac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:36.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0c77fe640 1 -- 172.21.15.113:0/2205013228 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8037070 con 0x7fc0c8144d60 2024-04-25T00:01:36.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0c77fe640 1 -- 172.21.15.113:0/2205013228 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b802ae50 con 0x7fc0c8144d60 2024-04-25T00:01:36.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0c77fe640 1 -- 172.21.15.113:0/2205013228 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8032660 con 0x7fc0c8144d60 2024-04-25T00:01:36.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0cee76640 1 -- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 msgr2=0x7fc0c80b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 0x7fc0c80b5390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802a100 comp rx=0 tx=0).stop 2024-04-25T00:01:36.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0cee76640 1 -- 172.21.15.113:0/2205013228 shutdown_connections 2024-04-25T00:01:36.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c8071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2205013228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 0x7fc0c80b5390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.880+0000 7fc0cee76640 1 -- 172.21.15.113:0/2205013228 >> 172.21.15.113:0/2205013228 conn(0x7fc0c80b1a50 msgr2=0x7fc0c80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:36.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.881+0000 7fc0cee76640 1 -- 172.21.15.113:0/2205013228 shutdown_connections 2024-04-25T00:01:36.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.881+0000 7fc0cee76640 1 -- 172.21.15.113:0/2205013228 wait complete. 2024-04-25T00:01:36.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.882+0000 7fc0cee76640 1 Processor -- start 2024-04-25T00:01:36.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.882+0000 7fc0cee76640 1 -- start start 2024-04-25T00:01:36.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c813c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 0x7fc0c81e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c81453c0 0x7fc0c81e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8145140 con 0x7fc0c81453c0 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8070e10 con 0x7fc0c8144d60 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c8071870 con 0x7fc0c806e6e0 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c81453c0 0x7fc0c81e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c81453c0 0x7fc0c81e6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39040/0 (socket says 172.21.15.113:39040) 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 -- 172.21.15.113:0/2214478783 learned_addr learned my addr 172.21.15.113:0/2214478783 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0ccc12640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c813c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0c7fff640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 0x7fc0c81e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 -- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 msgr2=0x7fc0c813c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c813c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 -- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 msgr2=0x7fc0c81e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 0x7fc0c81e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 -- 172.21.15.113:0/2214478783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c8160540 con 0x7fc0c81453c0 2024-04-25T00:01:36.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cd413640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c81453c0 0x7fc0c81e6bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000efc0 tx=0x7fc0c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:36.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0c57fa640 1 -- 172.21.15.113:0/2214478783 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0018030 con 0x7fc0c81453c0 2024-04-25T00:01:36.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0c57fa640 1 -- 172.21.15.113:0/2214478783 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c000cd30 con 0x7fc0c81453c0 2024-04-25T00:01:36.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c8160820 con 0x7fc0c81453c0 2024-04-25T00:01:36.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0c57fa640 1 -- 172.21.15.113:0/2214478783 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0003bb0 con 0x7fc0c81453c0 2024-04-25T00:01:36.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.883+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c8160160 con 0x7fc0c81453c0 2024-04-25T00:01:36.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.884+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c81609f0 con 0x7fc0c81453c0 2024-04-25T00:01:36.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.885+0000 7fc0c57fa640 1 -- 172.21.15.113:0/2214478783 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0c001b070 con 0x7fc0c81453c0 2024-04-25T00:01:36.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.885+0000 7fc0c57fa640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0b406be30 0x7fc0b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:36.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.885+0000 7fc0c57fa640 1 -- 172.21.15.113:0/2214478783 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fc0c008b750 con 0x7fc0c81453c0 2024-04-25T00:01:36.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.885+0000 7fc0ccc12640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0b406be30 0x7fc0b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:36.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.885+0000 7fc0ccc12640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0b406be30 0x7fc0b406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:36.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:36.889+0000 7fc0c57fa640 1 -- 172.21.15.113:0/2214478783 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0c0010040 con 0x7fc0c81453c0 2024-04-25T00:01:37.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.062+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0c80b27e0 con 0x7fc0c81453c0 2024-04-25T00:01:37.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.063+0000 7fc0c57fa640 1 -- 172.21.15.113:0/2214478783 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16726 (secure 0 0 0) 0x7fc0c0059a20 con 0x7fc0c81453c0 2024-04-25T00:01:37.066 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":59,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:35.811400+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:37.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.065+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0b406be30 msgr2=0x7fc0b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.065+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0b406be30 0x7fc0b406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).stop 2024-04-25T00:01:37.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.065+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c81453c0 msgr2=0x7fc0c81e6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.065+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c81453c0 0x7fc0c81e6bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000efc0 tx=0x7fc0c000c490 comp rx=0 tx=0).stop 2024-04-25T00:01:37.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 shutdown_connections 2024-04-25T00:01:37.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0b406be30 0x7fc0b406e2f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c8144d60 0x7fc0c81e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c81453c0 0x7fc0c81e6bf0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 --2- 172.21.15.113:0/2214478783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c806e6e0 0x7fc0c813c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 >> 172.21.15.113:0/2214478783 conn(0x7fc0c80b1a50 msgr2=0x7fc0c80ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:37.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 shutdown_connections 2024-04-25T00:01:37.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.066+0000 7fc0cee76640 1 -- 172.21.15.113:0/2214478783 wait complete. 2024-04-25T00:01:37.077 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:01:37.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.177+0000 7f45b6a64640 1 Processor -- start 2024-04-25T00:01:37.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.177+0000 7f45b6a64640 1 -- start start 2024-04-25T00:01:37.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b6a64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45b013f050 0x7f45b00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b6a64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b6a64640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f45b01604d0 0x7f45b006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b6a64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f45b00d09a0 con 0x7f45b013f050 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b6a64640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f45b015c6a0 con 0x7f45b01604d0 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b6a64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45b015d6b0 con 0x7f45b00d04f0 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45b013f050 0x7f45b00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45b013f050 0x7f45b00cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39054/0 (socket says 172.21.15.113:39054) 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 -- 172.21.15.113:0/3125216384 learned_addr learned my addr 172.21.15.113:0/3125216384 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45a7fff640 1 --2- 172.21.15.113:0/3125216384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 -- 172.21.15.113:0/3125216384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 msgr2=0x7f45b015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 --2- 172.21.15.113:0/3125216384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b015ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 -- 172.21.15.113:0/3125216384 >> v2:172.21.15.178:3300/0 conn(0x7f45b01604d0 msgr2=0x7f45b006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:37.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b5001640 1 --2- 172.21.15.113:0/3125216384 >> v2:172.21.15.178:3300/0 conn(0x7f45b01604d0 0x7f45b006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 --2- 172.21.15.113:0/3125216384 >> v2:172.21.15.178:3300/0 conn(0x7f45b01604d0 0x7f45b006cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b4800640 1 -- 172.21.15.113:0/3125216384 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b015c8f0 con 0x7f45b013f050 2024-04-25T00:01:37.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.178+0000 7f45b5001640 1 --2- 172.21.15.113:0/3125216384 >> v2:172.21.15.178:3300/0 conn(0x7f45b01604d0 0x7f45b006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:37.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.179+0000 7f45b4800640 1 --2- 172.21.15.113:0/3125216384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b013f050 0x7f45b00cffb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45b006e210 tx=0x7f45ac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d723ed8594af59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:37.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.179+0000 7f45a77fe640 1 -- 172.21.15.113:0/3125216384 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac037070 con 0x7f45b013f050 2024-04-25T00:01:37.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.179+0000 7f45a77fe640 1 -- 172.21.15.113:0/3125216384 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ac032440 con 0x7f45b013f050 2024-04-25T00:01:37.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.179+0000 7f45a77fe640 1 -- 172.21.15.113:0/3125216384 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac032810 con 0x7f45b013f050 2024-04-25T00:01:37.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 -- 172.21.15.113:0/3125216384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b013f050 msgr2=0x7f45b00cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3125216384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b013f050 0x7f45b00cffb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45b006e210 tx=0x7f45ac02a140 comp rx=0 tx=0).stop 2024-04-25T00:01:37.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 -- 172.21.15.113:0/3125216384 shutdown_connections 2024-04-25T00:01:37.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3125216384 >> v2:172.21.15.178:3300/0 conn(0x7f45b01604d0 0x7f45b006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3125216384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3125216384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b013f050 0x7f45b00cffb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 -- 172.21.15.113:0/3125216384 >> 172.21.15.113:0/3125216384 conn(0x7f45b006b0f0 msgr2=0x7f45b006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 -- 172.21.15.113:0/3125216384 shutdown_connections 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.180+0000 7f45b6a64640 1 -- 172.21.15.113:0/3125216384 wait complete. 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.181+0000 7f45b6a64640 1 Processor -- start 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.181+0000 7f45b6a64640 1 -- start start 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b6a64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b0149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b6a64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b013f050 0x7f45b0071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b6a64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b01604d0 0x7f45b0072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b6a64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45b015d190 con 0x7f45b01604d0 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b6a64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45b015ceb0 con 0x7f45b013f050 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b6a64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45b013f430 con 0x7f45b00d04f0 2024-04-25T00:01:37.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b01604d0 0x7f45b0072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b01604d0 0x7f45b0072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39062/0 (socket says 172.21.15.113:39062) 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 -- 172.21.15.113:0/3065397300 learned_addr learned my addr 172.21.15.113:0/3065397300 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b4800640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b0149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45a7fff640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b013f050 0x7f45b0071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 -- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 msgr2=0x7f45b0149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b0149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 -- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b013f050 msgr2=0x7f45b0071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b013f050 0x7f45b0071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 -- 172.21.15.113:0/3065397300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b00b1d70 con 0x7f45b01604d0 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45a7fff640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b013f050 0x7f45b0071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.182+0000 7f45b5001640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b01604d0 0x7f45b0072400 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f45a800cd50 tx=0x7f45a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.183+0000 7f45a57fa640 1 -- 172.21.15.113:0/3065397300 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800f680 con 0x7f45b01604d0 2024-04-25T00:01:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.183+0000 7f45a57fa640 1 -- 172.21.15.113:0/3065397300 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a800f840 con 0x7f45b01604d0 2024-04-25T00:01:37.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.183+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b00b2050 con 0x7f45b01604d0 2024-04-25T00:01:37.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.183+0000 7f45a57fa640 1 -- 172.21.15.113:0/3065397300 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800fc30 con 0x7f45b01604d0 2024-04-25T00:01:37.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.183+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b00b2610 con 0x7f45b01604d0 2024-04-25T00:01:37.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.184+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b00691a0 con 0x7f45b01604d0 2024-04-25T00:01:37.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.184+0000 7f45a57fa640 1 -- 172.21.15.113:0/3065397300 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45a80213f0 con 0x7f45b01604d0 2024-04-25T00:01:37.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.184+0000 7f45a57fa640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f459c06be30 0x7f459c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.185+0000 7f45a57fa640 1 -- 172.21.15.113:0/3065397300 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f45a808bc30 con 0x7f45b01604d0 2024-04-25T00:01:37.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.185+0000 7f45b4800640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f459c06be30 0x7f459c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.185+0000 7f45b4800640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f459c06be30 0x7f459c06e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f45ac034000 tx=0x7f45ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:37.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.189+0000 7f45a57fa640 1 -- 172.21.15.113:0/3065397300 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45a8014030 con 0x7f45b01604d0 2024-04-25T00:01:37.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.397+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f45b00d0e50 con 0x7f45b01604d0 2024-04-25T00:01:37.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.460+0000 7f45a57fa640 1 -- 172.21.15.113:0/3065397300 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f45a8059f00 con 0x7f45b01604d0 2024-04-25T00:01:37.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f459c06be30 msgr2=0x7f459c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f459c06be30 0x7f459c06e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f45ac034000 tx=0x7f45ac02e000 comp rx=0 tx=0).stop 2024-04-25T00:01:37.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b01604d0 msgr2=0x7f45b0072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b01604d0 0x7f45b0072400 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f45a800cd50 tx=0x7f45a800a650 comp rx=0 tx=0).stop 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 shutdown_connections 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f459c06be30 0x7f459c06e2f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b013f050 0x7f45b0071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b01604d0 0x7f45b0072400 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 --2- 172.21.15.113:0/3065397300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00d04f0 0x7f45b0149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 >> 172.21.15.113:0/3065397300 conn(0x7f45b006b0f0 msgr2=0x7f45b0151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 shutdown_connections 2024-04-25T00:01:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.462+0000 7f45b6a64640 1 -- 172.21.15.113:0/3065397300 wait complete. 2024-04-25T00:01:37.478 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:01:37.478 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:01:37.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.577+0000 7f70313ce640 1 Processor -- start 2024-04-25T00:01:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.577+0000 7f70313ce640 1 -- start start 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f70313ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f702c0b5ca0 0x7f702c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f70313ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702c0d2e00 con 0x7f702c15e510 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f70313ce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702c0d30e0 con 0x7f702c0b73c0 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f70313ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f702c0b6640 con 0x7f702c0b5ca0 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f702a7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f702c0b5ca0 0x7f702c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f702affd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f702c15e510 0x7f702c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53460/0 (socket says 172.21.15.113:53460) 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f702affd640 1 -- 172.21.15.113:0/61784988 learned_addr learned my addr 172.21.15.113:0/61784988 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f702a7fc640 1 -- 172.21.15.113:0/61784988 >> v1:172.21.15.178:6789/0 conn(0x7f702c0b73c0 legacy=0x7f702c0b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.578+0000 7f702a7fc640 1 -- 172.21.15.113:0/61784988 >> v1:172.21.15.113:6789/0 conn(0x7f702c15e510 legacy=0x7f702c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.579+0000 7f702a7fc640 1 -- 172.21.15.113:0/61784988 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c0d33c0 con 0x7f702c0b5ca0 2024-04-25T00:01:37.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.579+0000 7f7029ffb640 1 -- 172.21.15.113:0/61784988 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1170066054 0 0) 0x7f702c0d2e00 con 0x7f702c15e510 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.579+0000 7f702a7fc640 1 --2- 172.21.15.113:0/61784988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c0b6100 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f70200097a0 tx=0x7f702002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad410ee4df703c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.579+0000 7f7029ffb640 1 -- 172.21.15.113:0/61784988 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020037070 con 0x7f702c0b5ca0 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.579+0000 7f7029ffb640 1 -- 172.21.15.113:0/61784988 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702002ae50 con 0x7f702c0b5ca0 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.579+0000 7f7029ffb640 1 -- 172.21.15.113:0/61784988 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020032660 con 0x7f702c0b5ca0 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.580+0000 7f70313ce640 1 -- 172.21.15.113:0/61784988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 msgr2=0x7f702c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.580+0000 7f70313ce640 1 --2- 172.21.15.113:0/61784988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c0b6100 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f70200097a0 tx=0x7f702002a140 comp rx=0 tx=0).stop 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.580+0000 7f70313ce640 1 -- 172.21.15.113:0/61784988 shutdown_connections 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.580+0000 7f70313ce640 1 --2- 172.21.15.113:0/61784988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c0b6100 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.580+0000 7f70313ce640 1 -- 172.21.15.113:0/61784988 >> 172.21.15.113:0/61784988 conn(0x7f702c06b720 msgr2=0x7f702c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.580+0000 7f70313ce640 1 -- 172.21.15.113:0/61784988 shutdown_connections 2024-04-25T00:01:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.580+0000 7f70313ce640 1 -- 172.21.15.113:0/61784988 wait complete. 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.581+0000 7f70313ce640 1 Processor -- start 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.581+0000 7f70313ce640 1 -- start start 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f70313ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f70313ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702c0b73c0 0x7f702c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f70313ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f702c15e510 0x7f702c0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f70313ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f702c0d3ea0 con 0x7f702c0b5ca0 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f70313ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f702c0d3980 con 0x7f702c15e510 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f70313ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f702c0d2a90 con 0x7f702c0b73c0 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c1520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59738/0 (socket says 172.21.15.113:59738) 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 -- 172.21.15.113:0/2166705646 learned_addr learned my addr 172.21.15.113:0/2166705646 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:37.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702a7fc640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702c0b73c0 0x7f702c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702b7fe640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f702c15e510 0x7f702c0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 -- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702c0b73c0 msgr2=0x7f702c0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702c0b73c0 0x7f702c0c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 -- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f702c15e510 msgr2=0x7f702c0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f702c15e510 0x7f702c0b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 -- 172.21.15.113:0/2166705646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c160e20 con 0x7f702c0b5ca0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.582+0000 7f702affd640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c1520c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c890 tx=0x7f701c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.583+0000 7f70137fe640 1 -- 172.21.15.113:0/2166705646 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c018070 con 0x7f702c0b5ca0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.583+0000 7f70137fe640 1 -- 172.21.15.113:0/2166705646 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701c00a640 con 0x7f702c0b5ca0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.583+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c161160 con 0x7f702c0b5ca0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.583+0000 7f70137fe640 1 -- 172.21.15.113:0/2166705646 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c00aa30 con 0x7f702c0b5ca0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.583+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c0cb2f0 con 0x7f702c0b5ca0 2024-04-25T00:01:37.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.584+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702c0691a0 con 0x7f702c0b5ca0 2024-04-25T00:01:37.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.584+0000 7f70137fe640 1 -- 172.21.15.113:0/2166705646 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f701c00abd0 con 0x7f702c0b5ca0 2024-04-25T00:01:37.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.584+0000 7f70137fe640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f701806be60 0x7f701806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:37.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.585+0000 7f702a7fc640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f701806be60 0x7f701806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:37.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.585+0000 7f70137fe640 1 -- 172.21.15.113:0/2166705646 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f701c08a510 con 0x7f702c0b5ca0 2024-04-25T00:01:37.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.585+0000 7f702a7fc640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f701806be60 0x7f701806e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7020034000 tx=0x7f702002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:37.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.589+0000 7f70137fe640 1 -- 172.21.15.113:0/2166705646 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f702c161160 con 0x7f702c0b5ca0 2024-04-25T00:01:37.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:37.797+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f702c0cb530 con 0x7f702c0b5ca0 2024-04-25T00:01:38.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.473+0000 7f70137fe640 1 -- 172.21.15.113:0/2166705646 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f702c0cb530 con 0x7f702c0b5ca0 2024-04-25T00:01:38.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f701806be60 msgr2=0x7f701806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f701806be60 0x7f701806e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7020034000 tx=0x7f702002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:38.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 msgr2=0x7f702c1520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c1520c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c890 tx=0x7f701c00cd50 comp rx=0 tx=0).stop 2024-04-25T00:01:38.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 shutdown_connections 2024-04-25T00:01:38.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f701806be60 0x7f701806e320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f702c15e510 0x7f702c0b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702c0b73c0 0x7f702c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 --2- 172.21.15.113:0/2166705646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702c0b5ca0 0x7f702c1520c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 >> 172.21.15.113:0/2166705646 conn(0x7f702c06b720 msgr2=0x7f702c15cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 shutdown_connections 2024-04-25T00:01:38.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.476+0000 7f70313ce640 1 -- 172.21.15.113:0/2166705646 wait complete. 2024-04-25T00:01:38.487 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:01:38.488 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:38.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.585+0000 7f2df95ce640 1 Processor -- start 2024-04-25T00:01:38.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.585+0000 7f2df95ce640 1 -- start start 2024-04-25T00:01:38.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df95ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2df415e570 0x7f2df40b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df95ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df40b2de0 0x7f2df40b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df95ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df40ba060 con 0x7f2df40d0ff0 2024-04-25T00:01:38.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df95ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2df40b3630 con 0x7f2df415e570 2024-04-25T00:01:38.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df95ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2df40b8320 con 0x7f2df40b2de0 2024-04-25T00:01:38.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2dea7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2df415e570 0x7f2df40b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2dea7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2df415e570 0x7f2df40b9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59740/0 (socket says 172.21.15.113:59740) 2024-04-25T00:01:38.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2dea7fc640 1 -- 172.21.15.113:0/382484047 learned_addr learned my addr 172.21.15.113:0/382484047 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:38.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df2ffd640 1 --1- 172.21.15.113:0/382484047 >> v1:172.21.15.113:6789/0 conn(0x7f2df40d0ff0 0x7f2df40d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53464/0 (socket says 172.21.15.113:53464) 2024-04-25T00:01:38.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df37fe640 1 --2- 172.21.15.113:0/382484047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df40b2de0 0x7f2df40b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2dea7fc640 1 -- 172.21.15.113:0/382484047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df40b2de0 msgr2=0x7f2df40b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:38.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2dea7fc640 1 --2- 172.21.15.113:0/382484047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df40b2de0 0x7f2df40b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2dea7fc640 1 -- 172.21.15.113:0/382484047 >> v1:172.21.15.113:6789/0 conn(0x7f2df40d0ff0 legacy=0x7f2df40d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2dea7fc640 1 -- 172.21.15.113:0/382484047 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df40b8600 con 0x7f2df415e570 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df27fc640 1 -- 172.21.15.113:0/382484047 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661816777 0 0) 0x7f2df40ba060 con 0x7f2df40d0ff0 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.586+0000 7f2df37fe640 1 --2- 172.21.15.113:0/382484047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df40b2de0 0x7f2df40b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.587+0000 7f2dea7fc640 1 --2- 172.21.15.113:0/382484047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df415e570 0x7f2df40b9960 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009870 tx=0x7f2ddc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84305c0ef5189c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.587+0000 7f2df27fc640 1 -- 172.21.15.113:0/382484047 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc037070 con 0x7f2df415e570 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.587+0000 7f2df27fc640 1 -- 172.21.15.113:0/382484047 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ddc02ae50 con 0x7f2df415e570 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.587+0000 7f2df27fc640 1 -- 172.21.15.113:0/382484047 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc032860 con 0x7f2df415e570 2024-04-25T00:01:38.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 -- 172.21.15.113:0/382484047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df415e570 msgr2=0x7f2df40b9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 --2- 172.21.15.113:0/382484047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df415e570 0x7f2df40b9960 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009870 tx=0x7f2ddc02a300 comp rx=0 tx=0).stop 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 -- 172.21.15.113:0/382484047 shutdown_connections 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 --2- 172.21.15.113:0/382484047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df40b2de0 0x7f2df40b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 --2- 172.21.15.113:0/382484047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df415e570 0x7f2df40b9960 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 -- 172.21.15.113:0/382484047 >> 172.21.15.113:0/382484047 conn(0x7f2df4071b00 msgr2=0x7f2df4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 -- 172.21.15.113:0/382484047 shutdown_connections 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.588+0000 7f2df95ce640 1 -- 172.21.15.113:0/382484047 wait complete. 2024-04-25T00:01:38.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.589+0000 7f2df95ce640 1 Processor -- start 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.589+0000 7f2df95ce640 1 -- start start 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df40b2de0 0x7f2df40cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2df40d0ff0 0x7f2df40c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df415e570 0x7f2df4146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40d2140 con 0x7f2df40b2de0 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40b8bc0 con 0x7f2df40d0ff0 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2df40d22c0 con 0x7f2df415e570 2024-04-25T00:01:38.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df40b2de0 0x7f2df40cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df40b2de0 0x7f2df40cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59752/0 (socket says 172.21.15.113:59752) 2024-04-25T00:01:38.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 -- 172.21.15.113:0/4272248473 learned_addr learned my addr 172.21.15.113:0/4272248473 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:38.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df37fe640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df415e570 0x7f2df4146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2dea7fc640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2df40d0ff0 0x7f2df40c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 -- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df415e570 msgr2=0x7f2df4146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df415e570 0x7f2df4146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 -- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2df40d0ff0 msgr2=0x7f2df40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2df40d0ff0 0x7f2df40c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 -- 172.21.15.113:0/4272248473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df415f700 con 0x7f2df40b2de0 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2dea7fc640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2df40d0ff0 0x7f2df40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df37fe640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df415e570 0x7f2df4146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df2ffd640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df40b2de0 0x7f2df40cbac0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c4d0 tx=0x7f2de400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2debfff640 1 -- 172.21.15.113:0/4272248473 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4018070 con 0x7f2df40b2de0 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2debfff640 1 -- 172.21.15.113:0/4272248473 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de4007500 con 0x7f2df40b2de0 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df415f9e0 con 0x7f2df40b2de0 2024-04-25T00:01:38.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2debfff640 1 -- 172.21.15.113:0/4272248473 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4004a20 con 0x7f2df40b2de0 2024-04-25T00:01:38.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.590+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df415ffa0 con 0x7f2df40b2de0 2024-04-25T00:01:38.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.591+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df40691a0 con 0x7f2df40b2de0 2024-04-25T00:01:38.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.592+0000 7f2debfff640 1 -- 172.21.15.113:0/4272248473 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2de401b070 con 0x7f2df40b2de0 2024-04-25T00:01:38.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.592+0000 7f2debfff640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dd406be60 0x7f2dd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.593+0000 7f2debfff640 1 -- 172.21.15.113:0/4272248473 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f2de408a1b0 con 0x7f2df40b2de0 2024-04-25T00:01:38.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.593+0000 7f2dea7fc640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dd406be60 0x7f2dd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.593+0000 7f2dea7fc640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dd406be60 0x7f2dd406e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc034000 tx=0x7f2ddc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.597+0000 7f2debfff640 1 -- 172.21.15.113:0/4272248473 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2de4058b60 con 0x7f2df40b2de0 2024-04-25T00:01:38.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.804+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2df415b4f0 con 0x7f2df40b2de0 2024-04-25T00:01:38.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.805+0000 7f2debfff640 1 -- 172.21.15.113:0/4272248473 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5193 (secure 0 0 0) 0x7f2de4058460 con 0x7f2df40b2de0 2024-04-25T00:01:38.805 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3783503244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3783503244},{"type":"v1","addr":"172.21.15.178:6835","nonce":3783503244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3613161476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3613161476},{"type":"v1","addr":"172.21.15.113:6835","nonce":3613161476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2692423657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2692423657},{"type":"v1","addr":"172.21.15.113:6837","nonce":2692423657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:37.799020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/3704852657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3704852657},{"type":"v1","addr":"172.21.15.178:6834","nonce":3704852657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":4}]} 2024-04-25T00:01:38.805 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.806+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dd406be60 msgr2=0x7f2dd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.806+0000 7f2df95ce640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dd406be60 0x7f2dd406e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc034000 tx=0x7f2ddc02e000 comp rx=0 tx=0).stop 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.806+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df40b2de0 msgr2=0x7f2df40cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.806+0000 7f2df95ce640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df40b2de0 0x7f2df40cbac0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c4d0 tx=0x7f2de400c990 comp rx=0 tx=0).stop 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 shutdown_connections 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2dd406be60 0x7f2dd406e320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2df40d0ff0 0x7f2df40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df415e570 0x7f2df4146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 --2- 172.21.15.113:0/4272248473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2df40b2de0 0x7f2df40cbac0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 >> 172.21.15.113:0/4272248473 conn(0x7f2df4071b00 msgr2=0x7f2df40b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 shutdown_connections 2024-04-25T00:01:38.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.807+0000 7f2df95ce640 1 -- 172.21.15.113:0/4272248473 wait complete. 2024-04-25T00:01:38.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:38.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.914+0000 7ff58b977640 1 Processor -- start 2024-04-25T00:01:38.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.914+0000 7ff58b977640 1 -- start start 2024-04-25T00:01:38.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff58b977640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff584144d60 0x7ff5840b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff58b977640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5841453c0 0x7ff58406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff58b977640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5840cfad0 con 0x7ff58406e510 2024-04-25T00:01:38.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff58b977640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff584145c40 con 0x7ff584144d60 2024-04-25T00:01:38.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff58b977640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff58406e920 con 0x7ff5841453c0 2024-04-25T00:01:38.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589713640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff584144d60 0x7ff5840b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589f14640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff58406e510 0x7ff58415fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53468/0 (socket says 172.21.15.113:53468) 2024-04-25T00:01:38.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589713640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff584144d60 0x7ff5840b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59758/0 (socket says 172.21.15.113:59758) 2024-04-25T00:01:38.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589f14640 1 -- 172.21.15.113:0/919905173 learned_addr learned my addr 172.21.15.113:0/919905173 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:38.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589713640 1 -- 172.21.15.113:0/919905173 >> v2:172.21.15.178:3300/0 conn(0x7ff5841453c0 msgr2=0x7ff58406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:38.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589713640 1 --2- 172.21.15.113:0/919905173 >> v2:172.21.15.178:3300/0 conn(0x7ff5841453c0 0x7ff58406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589713640 1 -- 172.21.15.113:0/919905173 >> v1:172.21.15.113:6789/0 conn(0x7ff58406e510 legacy=0x7ff58415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.915+0000 7ff589713640 1 -- 172.21.15.113:0/919905173 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5840cf230 con 0x7ff584144d60 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff589713640 1 --2- 172.21.15.113:0/919905173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff584144d60 0x7ff5840b5390 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ce060 tx=0x7ff58002a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614961532c054707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff57bfff640 1 -- 172.21.15.113:0/919905173 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580037070 con 0x7ff584144d60 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff57bfff640 1 -- 172.21.15.113:0/919905173 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58002aca0 con 0x7ff584144d60 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff57bfff640 1 -- 172.21.15.113:0/919905173 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580032660 con 0x7ff584144d60 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff58b977640 1 -- 172.21.15.113:0/919905173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff584144d60 msgr2=0x7ff5840b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff58b977640 1 --2- 172.21.15.113:0/919905173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff584144d60 0x7ff5840b5390 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ce060 tx=0x7ff58002a010 comp rx=0 tx=0).stop 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff58b977640 1 -- 172.21.15.113:0/919905173 shutdown_connections 2024-04-25T00:01:38.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff58b977640 1 --2- 172.21.15.113:0/919905173 >> v2:172.21.15.178:3300/0 conn(0x7ff5841453c0 0x7ff58406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff58b977640 1 --2- 172.21.15.113:0/919905173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff584144d60 0x7ff5840b5390 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.916+0000 7ff58b977640 1 -- 172.21.15.113:0/919905173 >> 172.21.15.113:0/919905173 conn(0x7ff5840b1a50 msgr2=0x7ff5840b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.917+0000 7ff58b977640 1 -- 172.21.15.113:0/919905173 shutdown_connections 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.917+0000 7ff58b977640 1 -- 172.21.15.113:0/919905173 wait complete. 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 Processor -- start 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 -- start start 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff58406e510 0x7ff58415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff584144d60 0x7ff5840b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5841453c0 0x7ff58413a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff584070e10 con 0x7ff5841453c0 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5840cf7f0 con 0x7ff58406e510 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.918+0000 7ff58b977640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff584145140 con 0x7ff584144d60 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5841453c0 0x7ff58413a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5841453c0 0x7ff58413a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59772/0 (socket says 172.21.15.113:59772) 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 -- 172.21.15.113:0/2265741867 learned_addr learned my addr 172.21.15.113:0/2265741867 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589713640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff58406e510 0x7ff58415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff588f12640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff584144d60 0x7ff5840b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 -- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff584144d60 msgr2=0x7ff5840b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff584144d60 0x7ff5840b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 -- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff58406e510 msgr2=0x7ff58415b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff58406e510 0x7ff58415b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 -- 172.21.15.113:0/2265741867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58413c500 con 0x7ff5841453c0 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff588f12640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff584144d60 0x7ff5840b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589713640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff58406e510 0x7ff58415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff589f14640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5841453c0 0x7ff58413a800 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c7f0 tx=0x7ff57c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff579ffb640 1 -- 172.21.15.113:0/2265741867 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c018070 con 0x7ff5841453c0 2024-04-25T00:01:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff579ffb640 1 -- 172.21.15.113:0/2265741867 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c00a640 con 0x7ff5841453c0 2024-04-25T00:01:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff579ffb640 1 -- 172.21.15.113:0/2265741867 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00aab0 con 0x7ff5841453c0 2024-04-25T00:01:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5840b3150 con 0x7ff5841453c0 2024-04-25T00:01:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.919+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58413ca50 con 0x7ff5841453c0 2024-04-25T00:01:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.921+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5840691a0 con 0x7ff5841453c0 2024-04-25T00:01:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.921+0000 7ff579ffb640 1 -- 172.21.15.113:0/2265741867 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff57c01b070 con 0x7ff5841453c0 2024-04-25T00:01:38.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.921+0000 7ff579ffb640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff56806be60 0x7ff56806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.921+0000 7ff579ffb640 1 -- 172.21.15.113:0/2265741867 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7ff57c08a450 con 0x7ff5841453c0 2024-04-25T00:01:38.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.921+0000 7ff589713640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff56806be60 0x7ff56806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.921+0000 7ff589713640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff56806be60 0x7ff56806e320 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff580034000 tx=0x7ff58002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:38.925+0000 7ff579ffb640 1 -- 172.21.15.113:0/2265741867 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff57c058e90 con 0x7ff5841453c0 2024-04-25T00:01:39.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.098+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff58415c1c0 con 0x7ff5841453c0 2024-04-25T00:01:39.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.099+0000 7ff579ffb640 1 -- 172.21.15.113:0/2265741867 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16713 (secure 0 0 0) 0x7ff57c058790 con 0x7ff5841453c0 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":60,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:37.460157+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff56806be60 msgr2=0x7ff56806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff56806be60 0x7ff56806e320 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff580034000 tx=0x7ff58002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5841453c0 msgr2=0x7ff58413a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5841453c0 0x7ff58413a800 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c7f0 tx=0x7ff57c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 shutdown_connections 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff56806be60 0x7ff56806e320 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5841453c0 0x7ff58413a800 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff584144d60 0x7ff5840b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 --2- 172.21.15.113:0/2265741867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff58406e510 0x7ff58415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 >> 172.21.15.113:0/2265741867 conn(0x7ff5840b1a50 msgr2=0x7ff5840ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.101+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 shutdown_connections 2024-04-25T00:01:39.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.102+0000 7ff58b977640 1 -- 172.21.15.113:0/2265741867 wait complete. 2024-04-25T00:01:39.113 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:01:39.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:01:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.210+0000 7f416bbe9640 1 Processor -- start 2024-04-25T00:01:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.210+0000 7f416bbe9640 1 -- start start 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f416bbe9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f416415e320 0x7f41640d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f416bbe9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f41640d0b00 0x7f41640d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f416bbe9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41640d3de0 con 0x7f41640d2290 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f416bbe9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f41640d14b0 con 0x7f416415e320 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f416bbe9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f41640d3410 con 0x7f41640d0b00 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4169985640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f416415e320 0x7f41640d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4169184640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f41640d2290 0x7f41640b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53474/0 (socket says 172.21.15.113:53474) 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4169184640 1 -- 172.21.15.113:0/2234306619 learned_addr learned my addr 172.21.15.113:0/2234306619 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4169985640 1 -- 172.21.15.113:0/2234306619 >> v2:172.21.15.178:3300/0 conn(0x7f41640d0b00 msgr2=0x7f41640d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4169985640 1 --2- 172.21.15.113:0/2234306619 >> v2:172.21.15.178:3300/0 conn(0x7f41640d0b00 0x7f41640d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4169985640 1 -- 172.21.15.113:0/2234306619 >> v1:172.21.15.113:6789/0 conn(0x7f41640d2290 legacy=0x7f41640b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4169985640 1 -- 172.21.15.113:0/2234306619 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41640d2800 con 0x7f416415e320 2024-04-25T00:01:39.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.211+0000 7f4168983640 1 -- 172.21.15.113:0/2234306619 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2224814710 0 0) 0x7f41640d3de0 con 0x7f41640d2290 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.212+0000 7f4169985640 1 --2- 172.21.15.113:0/2234306619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416415e320 0x7f41640d38a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f41640d2030 tx=0x7f416002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dec6f93837211aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.212+0000 7f4168983640 1 -- 172.21.15.113:0/2234306619 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160037070 con 0x7f416415e320 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.212+0000 7f4168983640 1 -- 172.21.15.113:0/2234306619 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4160032440 con 0x7f416415e320 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.212+0000 7f4168983640 1 -- 172.21.15.113:0/2234306619 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160032810 con 0x7f416415e320 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 -- 172.21.15.113:0/2234306619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416415e320 msgr2=0x7f41640d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2234306619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416415e320 0x7f41640d38a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f41640d2030 tx=0x7f416002a230 comp rx=0 tx=0).stop 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 -- 172.21.15.113:0/2234306619 shutdown_connections 2024-04-25T00:01:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2234306619 >> v2:172.21.15.178:3300/0 conn(0x7f41640d0b00 0x7f41640d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2234306619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416415e320 0x7f41640d38a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 -- 172.21.15.113:0/2234306619 >> 172.21.15.113:0/2234306619 conn(0x7f41640b5c20 msgr2=0x7f41640b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 -- 172.21.15.113:0/2234306619 shutdown_connections 2024-04-25T00:01:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.213+0000 7f416bbe9640 1 -- 172.21.15.113:0/2234306619 wait complete. 2024-04-25T00:01:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.214+0000 7f416bbe9640 1 Processor -- start 2024-04-25T00:01:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.214+0000 7f416bbe9640 1 -- start start 2024-04-25T00:01:39.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41640d0b00 0x7f41640b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41640d2290 0x7f4164151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f416415e320 0x7f4164157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f41640d30a0 con 0x7f41640d2290 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41640d2dc0 con 0x7f416415e320 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f416415e700 con 0x7f41640d0b00 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41640d2290 0x7f4164151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41640d2290 0x7f4164151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59790/0 (socket says 172.21.15.113:59790) 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 -- 172.21.15.113:0/2621604457 learned_addr learned my addr 172.21.15.113:0/2621604457 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169985640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41640d0b00 0x7f41640b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416a186640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f416415e320 0x7f4164157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 -- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41640d0b00 msgr2=0x7f41640b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41640d0b00 0x7f41640b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 -- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f416415e320 msgr2=0x7f4164157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f416415e320 0x7f4164157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 -- 172.21.15.113:0/2621604457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41640b81c0 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416a186640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f416415e320 0x7f4164157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f4169184640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41640d2290 0x7f4164151a40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f415400c4d0 tx=0x7f415400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f415a7fc640 1 -- 172.21.15.113:0/2621604457 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154018070 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f415a7fc640 1 -- 172.21.15.113:0/2621604457 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4154007500 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41640b84a0 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f415a7fc640 1 -- 172.21.15.113:0/2621604457 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154004a20 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.215+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41640b8a60 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.216+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41640691a0 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.217+0000 7f415a7fc640 1 -- 172.21.15.113:0/2621604457 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f415401b070 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.217+0000 7f415a7fc640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f414806be30 0x7f414806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.217+0000 7f415a7fc640 1 -- 172.21.15.113:0/2621604457 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f415408a1e0 con 0x7f41640d2290 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.217+0000 7f4169985640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f414806be30 0x7f414806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.217+0000 7f4169985640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f414806be30 0x7f414806e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4160034000 tx=0x7f416002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.221+0000 7f415a7fc640 1 -- 172.21.15.113:0/2621604457 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4154058b90 con 0x7f41640d2290 2024-04-25T00:01:39.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.390+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41640d1230 con 0x7f414806be30 2024-04-25T00:01:39.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.396+0000 7f415a7fc640 1 -- 172.21.15.113:0/2621604457 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246627 (secure 0 0 0) 0x7f41640d1230 con 0x7f414806be30 2024-04-25T00:01:39.399 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:39.400 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:01:39.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f414806be30 msgr2=0x7f414806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f414806be30 0x7f414806e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4160034000 tx=0x7f416002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:39.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41640d2290 msgr2=0x7f4164151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41640d2290 0x7f4164151a40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f415400c4d0 tx=0x7f415400c990 comp rx=0 tx=0).stop 2024-04-25T00:01:39.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 shutdown_connections 2024-04-25T00:01:39.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f414806be30 0x7f414806e2f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f416415e320 0x7f4164157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41640d2290 0x7f4164151a40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 --2- 172.21.15.113:0/2621604457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41640d0b00 0x7f41640b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 >> 172.21.15.113:0/2621604457 conn(0x7f41640b5c20 msgr2=0x7f41640b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 shutdown_connections 2024-04-25T00:01:39.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.399+0000 7f416bbe9640 1 -- 172.21.15.113:0/2621604457 wait complete. 2024-04-25T00:01:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-25T00:01:39.150358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289320,"kb_used_data":6576,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685400,"statfs":{"total":773094113280,"available":772797849600,"internally_reserved":0,"allocated":6733824,"data_stored":3480370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.703398"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463581+0000","last_change":"2024-04-25T00:01:35.463581+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463581+0000","last_undegraded":"2024-04-25T00:01:35.463581+0000","last_fullsized":"2024-04-25T00:01:35.463581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457980+0000","last_change":"2024-04-25T00:01:35.457980+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457980+0000","last_undegraded":"2024-04-25T00:01:35.457980+0000","last_fullsized":"2024-04-25T00:01:35.457980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460964+0000","last_change":"2024-04-25T00:01:35.460964+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460964+0000","last_undegraded":"2024-04-25T00:01:35.460964+0000","last_fullsized":"2024-04-25T00:01:35.460964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465826+0000","last_change":"2024-04-25T00:01:35.465826+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465826+0000","last_undegraded":"2024-04-25T00:01:35.465826+0000","last_fullsized":"2024-04-25T00:01:35.465826+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454449+0000","last_change":"2024-04-25T00:01:35.454449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454449+0000","last_undegraded":"2024-04-25T00:01:35.454449+0000","last_fullsized":"2024-04-25T00:01:35.454449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461127+0000","last_change":"2024-04-25T00:01:35.461127+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461127+0000","last_undegraded":"2024-04-25T00:01:35.461127+0000","last_fullsized":"2024-04-25T00:01:35.461127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463139+0000","last_change":"2024-04-25T00:01:35.463139+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463139+0000","last_undegraded":"2024-04-25T00:01:35.463139+0000","last_fullsized":"2024-04-25T00:01:35.463139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462009+0000","last_change":"2024-04-25T00:01:35.462009+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462009+0000","last_undegraded":"2024-04-25T00:01:35.462009+0000","last_fullsized":"2024-04-25T00:01:35.462009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464866+0000","last_change":"2024-04-25T00:01:35.464866+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464866+0000","last_undegraded":"2024-04-25T00:01:35.464866+0000","last_fullsized":"2024-04-25T00:01:35.464866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.466538+0000","last_change":"2024-04-25T00:01:35.466538+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.466538+0000","last_undegraded":"2024-04-25T00:01:35.466538+0000","last_fullsized":"2024-04-25T00:01:35.466538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460560+0000","last_change":"2024-04-25T00:01:35.460560+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460560+0000","last_undegraded":"2024-04-25T00:01:35.460560+0000","last_fullsized":"2024-04-25T00:01:35.460560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454445+0000","last_change":"2024-04-25T00:01:35.454445+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454445+0000","last_undegraded":"2024-04-25T00:01:35.454445+0000","last_fullsized":"2024-04-25T00:01:35.454445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460130+0000","last_change":"2024-04-25T00:01:35.460130+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460130+0000","last_undegraded":"2024-04-25T00:01:35.460130+0000","last_fullsized":"2024-04-25T00:01:35.460130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460355+0000","last_change":"2024-04-25T00:01:35.460355+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460355+0000","last_undegraded":"2024-04-25T00:01:35.460355+0000","last_fullsized":"2024-04-25T00:01:35.460355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455420+0000","last_change":"2024-04-25T00:01:35.455420+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455420+0000","last_undegraded":"2024-04-25T00:01:35.455420+0000","last_fullsized":"2024-04-25T00:01:35.455420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461735+0000","last_change":"2024-04-25T00:01:35.461735+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461735+0000","last_undegraded":"2024-04-25T00:01:35.461735+0000","last_fullsized":"2024-04-25T00:01:35.461735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457312+0000","last_change":"2024-04-25T00:01:35.457312+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457312+0000","last_undegraded":"2024-04-25T00:01:35.457312+0000","last_fullsized":"2024-04-25T00:01:35.457312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461284+0000","last_change":"2024-04-25T00:01:35.461284+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461284+0000","last_undegraded":"2024-04-25T00:01:35.461284+0000","last_fullsized":"2024-04-25T00:01:35.461284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458433+0000","last_change":"2024-04-25T00:01:35.458433+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458433+0000","last_undegraded":"2024-04-25T00:01:35.458433+0000","last_fullsized":"2024-04-25T00:01:35.458433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461883+0000","last_change":"2024-04-25T00:01:35.461883+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461883+0000","last_undegraded":"2024-04-25T00:01:35.461883+0000","last_fullsized":"2024-04-25T00:01:35.461883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457341+0000","last_change":"2024-04-25T00:01:35.457341+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457341+0000","last_undegraded":"2024-04-25T00:01:35.457341+0000","last_fullsized":"2024-04-25T00:01:35.457341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455162+0000","last_change":"2024-04-25T00:01:35.455162+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455162+0000","last_undegraded":"2024-04-25T00:01:35.455162+0000","last_fullsized":"2024-04-25T00:01:35.455162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460808+0000","last_change":"2024-04-25T00:01:35.460808+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460808+0000","last_undegraded":"2024-04-25T00:01:35.460808+0000","last_fullsized":"2024-04-25T00:01:35.460808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459589+0000","last_change":"2024-04-25T00:01:35.459589+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459589+0000","last_undegraded":"2024-04-25T00:01:35.459589+0000","last_fullsized":"2024-04-25T00:01:35.459589+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460726+0000","last_change":"2024-04-25T00:01:35.460726+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460726+0000","last_undegraded":"2024-04-25T00:01:35.460726+0000","last_fullsized":"2024-04-25T00:01:35.460726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462561+0000","last_change":"2024-04-25T00:01:35.462561+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462561+0000","last_undegraded":"2024-04-25T00:01:35.462561+0000","last_fullsized":"2024-04-25T00:01:35.462561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462201+0000","last_change":"2024-04-25T00:01:35.462201+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462201+0000","last_undegraded":"2024-04-25T00:01:35.462201+0000","last_fullsized":"2024-04-25T00:01:35.462201+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458578+0000","last_change":"2024-04-25T00:01:35.458578+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458578+0000","last_undegraded":"2024-04-25T00:01:35.458578+0000","last_fullsized":"2024-04-25T00:01:35.458578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460240+0000","last_change":"2024-04-25T00:01:35.460240+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460240+0000","last_undegraded":"2024-04-25T00:01:35.460240+0000","last_fullsized":"2024-04-25T00:01:35.460240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454470+0000","last_change":"2024-04-25T00:01:35.454470+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454470+0000","last_undegraded":"2024-04-25T00:01:35.454470+0000","last_fullsized":"2024-04-25T00:01:35.454470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455791+0000","last_change":"2024-04-25T00:01:35.455791+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455791+0000","last_undegraded":"2024-04-25T00:01:35.455791+0000","last_fullsized":"2024-04-25T00:01:35.455791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471527+0000","last_change":"2024-04-25T00:01:35.469190+0000","last_active":"2024-04-25T00:01:35.471527+0000","last_peered":"2024-04-25T00:01:35.471527+0000","last_clean":"2024-04-25T00:01:35.471527+0000","last_became_active":"2024-04-25T00:01:35.468910+0000","last_became_peered":"2024-04-25T00:01:35.468910+0000","last_unstale":"2024-04-25T00:01:35.471527+0000","last_undegraded":"2024-04-25T00:01:35.471527+0000","last_fullsized":"2024-04-25T00:01:35.471527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.183879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465124+0000","last_change":"2024-04-25T00:01:35.465124+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465124+0000","last_undegraded":"2024-04-25T00:01:35.465124+0000","last_fullsized":"2024-04-25T00:01:35.465124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466862+0000","last_change":"2024-04-25T00:01:35.465786+0000","last_active":"2024-04-25T00:01:35.466862+0000","last_peered":"2024-04-25T00:01:35.466862+0000","last_clean":"2024-04-25T00:01:35.466862+0000","last_became_active":"2024-04-25T00:01:35.465527+0000","last_became_peered":"2024-04-25T00:01:35.465527+0000","last_unstale":"2024-04-25T00:01:35.466862+0000","last_undegraded":"2024-04-25T00:01:35.466862+0000","last_fullsized":"2024-04-25T00:01:35.466862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:48.237734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454877+0000","last_change":"2024-04-25T00:01:35.454877+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454877+0000","last_undegraded":"2024-04-25T00:01:35.454877+0000","last_fullsized":"2024-04-25T00:01:35.454877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471535+0000","last_change":"2024-04-25T00:01:35.469185+0000","last_active":"2024-04-25T00:01:35.471535+0000","last_peered":"2024-04-25T00:01:35.471535+0000","last_clean":"2024-04-25T00:01:35.471535+0000","last_became_active":"2024-04-25T00:01:35.468886+0000","last_became_peered":"2024-04-25T00:01:35.468886+0000","last_unstale":"2024-04-25T00:01:35.471535+0000","last_undegraded":"2024-04-25T00:01:35.471535+0000","last_fullsized":"2024-04-25T00:01:35.471535+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:16.817987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460218+0000","last_change":"2024-04-25T00:01:35.460218+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460218+0000","last_undegraded":"2024-04-25T00:01:35.460218+0000","last_fullsized":"2024-04-25T00:01:35.460218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469474+0000","last_change":"2024-04-25T00:01:35.467838+0000","last_active":"2024-04-25T00:01:35.469474+0000","last_peered":"2024-04-25T00:01:35.469474+0000","last_clean":"2024-04-25T00:01:35.469474+0000","last_became_active":"2024-04-25T00:01:35.467216+0000","last_became_peered":"2024-04-25T00:01:35.467216+0000","last_unstale":"2024-04-25T00:01:35.469474+0000","last_undegraded":"2024-04-25T00:01:35.469474+0000","last_fullsized":"2024-04-25T00:01:35.469474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461449+0000","last_change":"2024-04-25T00:01:35.461449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461449+0000","last_undegraded":"2024-04-25T00:01:35.461449+0000","last_fullsized":"2024-04-25T00:01:35.461449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470832+0000","last_change":"2024-04-25T00:01:35.469530+0000","last_active":"2024-04-25T00:01:35.470832+0000","last_peered":"2024-04-25T00:01:35.470832+0000","last_clean":"2024-04-25T00:01:35.470832+0000","last_became_active":"2024-04-25T00:01:35.468953+0000","last_became_peered":"2024-04-25T00:01:35.468953+0000","last_unstale":"2024-04-25T00:01:35.470832+0000","last_undegraded":"2024-04-25T00:01:35.470832+0000","last_fullsized":"2024-04-25T00:01:35.470832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461820+0000","last_change":"2024-04-25T00:01:35.461820+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461820+0000","last_undegraded":"2024-04-25T00:01:35.461820+0000","last_fullsized":"2024-04-25T00:01:35.461820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465278+0000","last_change":"2024-04-25T00:01:35.463716+0000","last_active":"2024-04-25T00:01:35.465278+0000","last_peered":"2024-04-25T00:01:35.465278+0000","last_clean":"2024-04-25T00:01:35.465278+0000","last_became_active":"2024-04-25T00:01:35.463153+0000","last_became_peered":"2024-04-25T00:01:35.463153+0000","last_unstale":"2024-04-25T00:01:35.465278+0000","last_undegraded":"2024-04-25T00:01:35.465278+0000","last_fullsized":"2024-04-25T00:01:35.465278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454489+0000","last_change":"2024-04-25T00:01:35.454489+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454489+0000","last_undegraded":"2024-04-25T00:01:35.454489+0000","last_fullsized":"2024-04-25T00:01:35.454489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465858+0000","last_change":"2024-04-25T00:01:35.465127+0000","last_active":"2024-04-25T00:01:35.465858+0000","last_peered":"2024-04-25T00:01:35.465858+0000","last_clean":"2024-04-25T00:01:35.465858+0000","last_became_active":"2024-04-25T00:01:35.464804+0000","last_became_peered":"2024-04-25T00:01:35.464804+0000","last_unstale":"2024-04-25T00:01:35.465858+0000","last_undegraded":"2024-04-25T00:01:35.465858+0000","last_fullsized":"2024-04-25T00:01:35.465858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:33.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461392+0000","last_change":"2024-04-25T00:01:35.461392+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461392+0000","last_undegraded":"2024-04-25T00:01:35.461392+0000","last_fullsized":"2024-04-25T00:01:35.461392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471287+0000","last_change":"2024-04-25T00:01:35.470685+0000","last_active":"2024-04-25T00:01:35.471287+0000","last_peered":"2024-04-25T00:01:35.471287+0000","last_clean":"2024-04-25T00:01:35.471287+0000","last_became_active":"2024-04-25T00:01:35.470365+0000","last_became_peered":"2024-04-25T00:01:35.470365+0000","last_unstale":"2024-04-25T00:01:35.471287+0000","last_undegraded":"2024-04-25T00:01:35.471287+0000","last_fullsized":"2024-04-25T00:01:35.471287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461419+0000","last_change":"2024-04-25T00:01:35.461419+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461419+0000","last_undegraded":"2024-04-25T00:01:35.461419+0000","last_fullsized":"2024-04-25T00:01:35.461419+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471453+0000","last_change":"2024-04-25T00:01:35.470524+0000","last_active":"2024-04-25T00:01:35.471453+0000","last_peered":"2024-04-25T00:01:35.471453+0000","last_clean":"2024-04-25T00:01:35.471453+0000","last_became_active":"2024-04-25T00:01:35.470224+0000","last_became_peered":"2024-04-25T00:01:35.470224+0000","last_unstale":"2024-04-25T00:01:35.471453+0000","last_undegraded":"2024-04-25T00:01:35.471453+0000","last_fullsized":"2024-04-25T00:01:35.471453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456965+0000","last_change":"2024-04-25T00:01:35.456965+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456965+0000","last_undegraded":"2024-04-25T00:01:35.456965+0000","last_fullsized":"2024-04-25T00:01:35.456965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470019+0000","last_change":"2024-04-25T00:01:35.466689+0000","last_active":"2024-04-25T00:01:35.470019+0000","last_peered":"2024-04-25T00:01:35.470019+0000","last_clean":"2024-04-25T00:01:35.470019+0000","last_became_active":"2024-04-25T00:01:35.466390+0000","last_became_peered":"2024-04-25T00:01:35.466390+0000","last_unstale":"2024-04-25T00:01:35.470019+0000","last_undegraded":"2024-04-25T00:01:35.470019+0000","last_fullsized":"2024-04-25T00:01:35.470019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:21.027526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464083+0000","last_change":"2024-04-25T00:01:35.464083+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464083+0000","last_undegraded":"2024-04-25T00:01:35.464083+0000","last_fullsized":"2024-04-25T00:01:35.464083+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470655+0000","last_change":"2024-04-25T00:01:35.469419+0000","last_active":"2024-04-25T00:01:35.470655+0000","last_peered":"2024-04-25T00:01:35.470655+0000","last_clean":"2024-04-25T00:01:35.470655+0000","last_became_active":"2024-04-25T00:01:35.468979+0000","last_became_peered":"2024-04-25T00:01:35.468979+0000","last_unstale":"2024-04-25T00:01:35.470655+0000","last_undegraded":"2024-04-25T00:01:35.470655+0000","last_fullsized":"2024-04-25T00:01:35.470655+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457481+0000","last_change":"2024-04-25T00:01:35.457481+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457481+0000","last_undegraded":"2024-04-25T00:01:35.457481+0000","last_fullsized":"2024-04-25T00:01:35.457481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468042+0000","last_change":"2024-04-25T00:01:35.466590+0000","last_active":"2024-04-25T00:01:35.468042+0000","last_peered":"2024-04-25T00:01:35.468042+0000","last_clean":"2024-04-25T00:01:35.468042+0000","last_became_active":"2024-04-25T00:01:35.466216+0000","last_became_peered":"2024-04-25T00:01:35.466216+0000","last_unstale":"2024-04-25T00:01:35.468042+0000","last_undegraded":"2024-04-25T00:01:35.468042+0000","last_fullsized":"2024-04-25T00:01:35.468042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457950+0000","last_change":"2024-04-25T00:01:35.457950+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457950+0000","last_undegraded":"2024-04-25T00:01:35.457950+0000","last_fullsized":"2024-04-25T00:01:35.457950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470984+0000","last_change":"2024-04-25T00:01:35.469270+0000","last_active":"2024-04-25T00:01:35.470984+0000","last_peered":"2024-04-25T00:01:35.470984+0000","last_clean":"2024-04-25T00:01:35.470984+0000","last_became_active":"2024-04-25T00:01:35.468644+0000","last_became_peered":"2024-04-25T00:01:35.468644+0000","last_unstale":"2024-04-25T00:01:35.470984+0000","last_undegraded":"2024-04-25T00:01:35.470984+0000","last_fullsized":"2024-04-25T00:01:35.470984+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:02.120913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458008+0000","last_change":"2024-04-25T00:01:35.458008+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458008+0000","last_undegraded":"2024-04-25T00:01:35.458008+0000","last_fullsized":"2024-04-25T00:01:35.458008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467827+0000","last_change":"2024-04-25T00:01:35.465772+0000","last_active":"2024-04-25T00:01:35.467827+0000","last_peered":"2024-04-25T00:01:35.467827+0000","last_clean":"2024-04-25T00:01:35.467827+0000","last_became_active":"2024-04-25T00:01:35.465310+0000","last_became_peered":"2024-04-25T00:01:35.465310+0000","last_unstale":"2024-04-25T00:01:35.467827+0000","last_undegraded":"2024-04-25T00:01:35.467827+0000","last_fullsized":"2024-04-25T00:01:35.467827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:34.269657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459893+0000","last_change":"2024-04-25T00:01:35.459893+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459893+0000","last_undegraded":"2024-04-25T00:01:35.459893+0000","last_fullsized":"2024-04-25T00:01:35.459893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468569+0000","last_change":"2024-04-25T00:01:35.467537+0000","last_active":"2024-04-25T00:01:35.468569+0000","last_peered":"2024-04-25T00:01:35.468569+0000","last_clean":"2024-04-25T00:01:35.468569+0000","last_became_active":"2024-04-25T00:01:35.467005+0000","last_became_peered":"2024-04-25T00:01:35.467005+0000","last_unstale":"2024-04-25T00:01:35.468569+0000","last_undegraded":"2024-04-25T00:01:35.468569+0000","last_fullsized":"2024-04-25T00:01:35.468569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:39.054815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456599+0000","last_change":"2024-04-25T00:01:35.456599+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456599+0000","last_undegraded":"2024-04-25T00:01:35.456599+0000","last_fullsized":"2024-04-25T00:01:35.456599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471280+0000","last_change":"2024-04-25T00:01:35.469625+0000","last_active":"2024-04-25T00:01:35.471280+0000","last_peered":"2024-04-25T00:01:35.471280+0000","last_clean":"2024-04-25T00:01:35.471280+0000","last_became_active":"2024-04-25T00:01:35.469302+0000","last_became_peered":"2024-04-25T00:01:35.469302+0000","last_unstale":"2024-04-25T00:01:35.471280+0000","last_undegraded":"2024-04-25T00:01:35.471280+0000","last_fullsized":"2024-04-25T00:01:35.471280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456805+0000","last_change":"2024-04-25T00:01:35.456805+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456805+0000","last_undegraded":"2024-04-25T00:01:35.456805+0000","last_fullsized":"2024-04-25T00:01:35.456805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470624+0000","last_change":"2024-04-25T00:01:35.468884+0000","last_active":"2024-04-25T00:01:35.470624+0000","last_peered":"2024-04-25T00:01:35.470624+0000","last_clean":"2024-04-25T00:01:35.470624+0000","last_became_active":"2024-04-25T00:01:35.468603+0000","last_became_peered":"2024-04-25T00:01:35.468603+0000","last_unstale":"2024-04-25T00:01:35.470624+0000","last_undegraded":"2024-04-25T00:01:35.470624+0000","last_fullsized":"2024-04-25T00:01:35.470624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:39.405188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458803+0000","last_change":"2024-04-25T00:01:35.458803+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458803+0000","last_undegraded":"2024-04-25T00:01:35.458803+0000","last_fullsized":"2024-04-25T00:01:35.458803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.462144+0000","last_change":"2024-04-25T00:01:35.458456+0000","last_active":"2024-04-25T00:01:35.462144+0000","last_peered":"2024-04-25T00:01:35.462144+0000","last_clean":"2024-04-25T00:01:35.462144+0000","last_became_active":"2024-04-25T00:01:35.458035+0000","last_became_peered":"2024-04-25T00:01:35.458035+0000","last_unstale":"2024-04-25T00:01:35.462144+0000","last_undegraded":"2024-04-25T00:01:35.462144+0000","last_fullsized":"2024-04-25T00:01:35.462144+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460400+0000","last_change":"2024-04-25T00:01:35.460400+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460400+0000","last_undegraded":"2024-04-25T00:01:35.460400+0000","last_fullsized":"2024-04-25T00:01:35.460400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467672+0000","last_change":"2024-04-25T00:01:35.466637+0000","last_active":"2024-04-25T00:01:35.467672+0000","last_peered":"2024-04-25T00:01:35.467672+0000","last_clean":"2024-04-25T00:01:35.467672+0000","last_became_active":"2024-04-25T00:01:35.466330+0000","last_became_peered":"2024-04-25T00:01:35.466330+0000","last_unstale":"2024-04-25T00:01:35.467672+0000","last_undegraded":"2024-04-25T00:01:35.467672+0000","last_fullsized":"2024-04-25T00:01:35.467672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:04.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456985+0000","last_change":"2024-04-25T00:01:35.456985+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456985+0000","last_undegraded":"2024-04-25T00:01:35.456985+0000","last_fullsized":"2024-04-25T00:01:35.456985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.463998+0000","last_change":"2024-04-25T00:01:35.460727+0000","last_active":"2024-04-25T00:01:35.463998+0000","last_peered":"2024-04-25T00:01:35.463998+0000","last_clean":"2024-04-25T00:01:35.463998+0000","last_became_active":"2024-04-25T00:01:35.460439+0000","last_became_peered":"2024-04-25T00:01:35.460439+0000","last_unstale":"2024-04-25T00:01:35.463998+0000","last_undegraded":"2024-04-25T00:01:35.463998+0000","last_fullsized":"2024-04-25T00:01:35.463998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457477+0000","last_change":"2024-04-25T00:01:35.457477+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457477+0000","last_undegraded":"2024-04-25T00:01:35.457477+0000","last_fullsized":"2024-04-25T00:01:35.457477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469402+0000","last_change":"2024-04-25T00:01:35.467452+0000","last_active":"2024-04-25T00:01:35.469402+0000","last_peered":"2024-04-25T00:01:35.469402+0000","last_clean":"2024-04-25T00:01:35.469402+0000","last_became_active":"2024-04-25T00:01:35.466644+0000","last_became_peered":"2024-04-25T00:01:35.466644+0000","last_unstale":"2024-04-25T00:01:35.469402+0000","last_undegraded":"2024-04-25T00:01:35.469402+0000","last_fullsized":"2024-04-25T00:01:35.469402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:08.725990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454546+0000","last_change":"2024-04-25T00:01:35.454546+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454546+0000","last_undegraded":"2024-04-25T00:01:35.454546+0000","last_fullsized":"2024-04-25T00:01:35.454546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.464899+0000","last_change":"2024-04-25T00:01:35.462553+0000","last_active":"2024-04-25T00:01:35.464899+0000","last_peered":"2024-04-25T00:01:35.464899+0000","last_clean":"2024-04-25T00:01:35.464899+0000","last_became_active":"2024-04-25T00:01:35.462284+0000","last_became_peered":"2024-04-25T00:01:35.462284+0000","last_unstale":"2024-04-25T00:01:35.464899+0000","last_undegraded":"2024-04-25T00:01:35.464899+0000","last_fullsized":"2024-04-25T00:01:35.464899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:05.412281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456125+0000","last_change":"2024-04-25T00:01:35.456125+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456125+0000","last_undegraded":"2024-04-25T00:01:35.456125+0000","last_fullsized":"2024-04-25T00:01:35.456125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471239+0000","last_change":"2024-04-25T00:01:35.470548+0000","last_active":"2024-04-25T00:01:35.471239+0000","last_peered":"2024-04-25T00:01:35.471239+0000","last_clean":"2024-04-25T00:01:35.471239+0000","last_became_active":"2024-04-25T00:01:35.470230+0000","last_became_peered":"2024-04-25T00:01:35.470230+0000","last_unstale":"2024-04-25T00:01:35.471239+0000","last_undegraded":"2024-04-25T00:01:35.471239+0000","last_fullsized":"2024-04-25T00:01:35.471239+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459032+0000","last_change":"2024-04-25T00:01:35.459032+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459032+0000","last_undegraded":"2024-04-25T00:01:35.459032+0000","last_fullsized":"2024-04-25T00:01:35.459032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470245+0000","last_change":"2024-04-25T00:01:35.468183+0000","last_active":"2024-04-25T00:01:35.470245+0000","last_peered":"2024-04-25T00:01:35.470245+0000","last_clean":"2024-04-25T00:01:35.470245+0000","last_became_active":"2024-04-25T00:01:35.467905+0000","last_became_peered":"2024-04-25T00:01:35.467905+0000","last_unstale":"2024-04-25T00:01:35.470245+0000","last_undegraded":"2024-04-25T00:01:35.470245+0000","last_fullsized":"2024-04-25T00:01:35.470245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:50.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458100+0000","last_change":"2024-04-25T00:01:35.458100+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458100+0000","last_undegraded":"2024-04-25T00:01:35.458100+0000","last_fullsized":"2024-04-25T00:01:35.458100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471197+0000","last_change":"2024-04-25T00:01:35.468807+0000","last_active":"2024-04-25T00:01:35.471197+0000","last_peered":"2024-04-25T00:01:35.471197+0000","last_clean":"2024-04-25T00:01:35.471197+0000","last_became_active":"2024-04-25T00:01:35.468521+0000","last_became_peered":"2024-04-25T00:01:35.468521+0000","last_unstale":"2024-04-25T00:01:35.471197+0000","last_undegraded":"2024-04-25T00:01:35.471197+0000","last_fullsized":"2024-04-25T00:01:35.471197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463586+0000","last_change":"2024-04-25T00:01:35.463586+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463586+0000","last_undegraded":"2024-04-25T00:01:35.463586+0000","last_fullsized":"2024-04-25T00:01:35.463586+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466296+0000","last_change":"2024-04-25T00:01:35.463380+0000","last_active":"2024-04-25T00:01:35.466296+0000","last_peered":"2024-04-25T00:01:35.466296+0000","last_clean":"2024-04-25T00:01:35.466296+0000","last_became_active":"2024-04-25T00:01:35.463052+0000","last_became_peered":"2024-04-25T00:01:35.463052+0000","last_unstale":"2024-04-25T00:01:35.466296+0000","last_undegraded":"2024-04-25T00:01:35.466296+0000","last_fullsized":"2024-04-25T00:01:35.466296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:29.474425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454529+0000","last_change":"2024-04-25T00:01:35.454529+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454529+0000","last_undegraded":"2024-04-25T00:01:35.454529+0000","last_fullsized":"2024-04-25T00:01:35.454529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469453+0000","last_change":"2024-04-25T00:01:35.468164+0000","last_active":"2024-04-25T00:01:35.469453+0000","last_peered":"2024-04-25T00:01:35.469453+0000","last_clean":"2024-04-25T00:01:35.469453+0000","last_became_active":"2024-04-25T00:01:35.467823+0000","last_became_peered":"2024-04-25T00:01:35.467823+0000","last_unstale":"2024-04-25T00:01:35.469453+0000","last_undegraded":"2024-04-25T00:01:35.469453+0000","last_fullsized":"2024-04-25T00:01:35.469453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460509+0000","last_change":"2024-04-25T00:01:35.460509+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460509+0000","last_undegraded":"2024-04-25T00:01:35.460509+0000","last_fullsized":"2024-04-25T00:01:35.460509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467004+0000","last_change":"2024-04-25T00:01:35.465879+0000","last_active":"2024-04-25T00:01:35.467004+0000","last_peered":"2024-04-25T00:01:35.467004+0000","last_clean":"2024-04-25T00:01:35.467004+0000","last_became_active":"2024-04-25T00:01:35.465602+0000","last_became_peered":"2024-04-25T00:01:35.465602+0000","last_unstale":"2024-04-25T00:01:35.467004+0000","last_undegraded":"2024-04-25T00:01:35.467004+0000","last_fullsized":"2024-04-25T00:01:35.467004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:32.955486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457579+0000","last_change":"2024-04-25T00:01:35.457579+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457579+0000","last_undegraded":"2024-04-25T00:01:35.457579+0000","last_fullsized":"2024-04-25T00:01:35.457579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468169+0000","last_change":"2024-04-25T00:01:35.467484+0000","last_active":"2024-04-25T00:01:35.468169+0000","last_peered":"2024-04-25T00:01:35.468169+0000","last_clean":"2024-04-25T00:01:35.468169+0000","last_became_active":"2024-04-25T00:01:35.466391+0000","last_became_peered":"2024-04-25T00:01:35.466391+0000","last_unstale":"2024-04-25T00:01:35.468169+0000","last_undegraded":"2024-04-25T00:01:35.468169+0000","last_fullsized":"2024-04-25T00:01:35.468169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463048+0000","last_change":"2024-04-25T00:01:35.463048+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463048+0000","last_undegraded":"2024-04-25T00:01:35.463048+0000","last_fullsized":"2024-04-25T00:01:35.463048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470685+0000","last_change":"2024-04-25T00:01:35.469390+0000","last_active":"2024-04-25T00:01:35.470685+0000","last_peered":"2024-04-25T00:01:35.470685+0000","last_clean":"2024-04-25T00:01:35.470685+0000","last_became_active":"2024-04-25T00:01:35.469128+0000","last_became_peered":"2024-04-25T00:01:35.469128+0000","last_unstale":"2024-04-25T00:01:35.470685+0000","last_undegraded":"2024-04-25T00:01:35.470685+0000","last_fullsized":"2024-04-25T00:01:35.470685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.684750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460347+0000","last_change":"2024-04-25T00:01:35.460347+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460347+0000","last_undegraded":"2024-04-25T00:01:35.460347+0000","last_fullsized":"2024-04-25T00:01:35.460347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470837+0000","last_change":"2024-04-25T00:01:35.468981+0000","last_active":"2024-04-25T00:01:35.470837+0000","last_peered":"2024-04-25T00:01:35.470837+0000","last_clean":"2024-04-25T00:01:35.470837+0000","last_became_active":"2024-04-25T00:01:35.468687+0000","last_became_peered":"2024-04-25T00:01:35.468687+0000","last_unstale":"2024-04-25T00:01:35.470837+0000","last_undegraded":"2024-04-25T00:01:35.470837+0000","last_fullsized":"2024-04-25T00:01:35.470837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:56.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460678+0000","last_change":"2024-04-25T00:01:35.460678+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460678+0000","last_undegraded":"2024-04-25T00:01:35.460678+0000","last_fullsized":"2024-04-25T00:01:35.460678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468470+0000","last_change":"2024-04-25T00:01:35.467166+0000","last_active":"2024-04-25T00:01:35.468470+0000","last_peered":"2024-04-25T00:01:35.468470+0000","last_clean":"2024-04-25T00:01:35.468470+0000","last_became_active":"2024-04-25T00:01:35.466904+0000","last_became_peered":"2024-04-25T00:01:35.466904+0000","last_unstale":"2024-04-25T00:01:35.468470+0000","last_undegraded":"2024-04-25T00:01:35.468470+0000","last_fullsized":"2024-04-25T00:01:35.468470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:43.861535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.461537+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:01:35.461537+0000","last_peered":"2024-04-25T00:01:35.461537+0000","last_clean":"2024-04-25T00:01:35.461537+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:01:35.461537+0000","last_undegraded":"2024-04-25T00:01:35.461537+0000","last_fullsized":"2024-04-25T00:01:35.461537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44924,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326916,"statfs":{"total":96636764160,"available":96590761984,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:39.414 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:01:39.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.547+0000 7f674bcd2640 1 Processor -- start 2024-04-25T00:01:39.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.547+0000 7f674bcd2640 1 -- start start 2024-04-25T00:01:39.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674bcd2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67441453c0 0x7f67440b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674bcd2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67440cfad0 con 0x7f674406e6e0 2024-04-25T00:01:39.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674bcd2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67440cf230 con 0x7f6744144d60 2024-04-25T00:01:39.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674bcd2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6744145c40 con 0x7f67441453c0 2024-04-25T00:01:39.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674a26f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67441453c0 0x7f67440b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674a26f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67441453c0 0x7f67440b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59802/0 (socket says 172.21.15.113:59802) 2024-04-25T00:01:39.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674926d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f674406e6e0 0x7f6744071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53490/0 (socket says 172.21.15.113:53490) 2024-04-25T00:01:39.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674a26f640 1 -- 172.21.15.113:0/1941783968 learned_addr learned my addr 172.21.15.113:0/1941783968 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:39.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674a26f640 1 -- 172.21.15.113:0/1941783968 >> v1:172.21.15.178:6789/0 conn(0x7f6744144d60 legacy=0x7f674406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674a26f640 1 -- 172.21.15.113:0/1941783968 >> v1:172.21.15.113:6789/0 conn(0x7f674406e6e0 legacy=0x7f6744071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f674a26f640 1 -- 172.21.15.113:0/1941783968 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67440cf510 con 0x7f67441453c0 2024-04-25T00:01:39.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.548+0000 7f6748a6c640 1 -- 172.21.15.113:0/1941783968 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2870188470 0 0) 0x7f67440cfad0 con 0x7f674406e6e0 2024-04-25T00:01:39.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f674a26f640 1 --2- 172.21.15.113:0/1941783968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67441453c0 0x7f67440b5390 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f673c009e50 tx=0x7f673c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626b3943d14dc535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f6748a6c640 1 -- 172.21.15.113:0/1941783968 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c036070 con 0x7f67441453c0 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f6748a6c640 1 -- 172.21.15.113:0/1941783968 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c031600 con 0x7f67441453c0 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f6748a6c640 1 -- 172.21.15.113:0/1941783968 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0319d0 con 0x7f67441453c0 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f674bcd2640 1 -- 172.21.15.113:0/1941783968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67441453c0 msgr2=0x7f67440b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f674bcd2640 1 --2- 172.21.15.113:0/1941783968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67441453c0 0x7f67440b5390 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f673c009e50 tx=0x7f673c02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f674bcd2640 1 -- 172.21.15.113:0/1941783968 shutdown_connections 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f674bcd2640 1 --2- 172.21.15.113:0/1941783968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67441453c0 0x7f67440b5390 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.549+0000 7f674bcd2640 1 -- 172.21.15.113:0/1941783968 >> 172.21.15.113:0/1941783968 conn(0x7f67440b1a50 msgr2=0x7f67440b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.550+0000 7f674bcd2640 1 -- 172.21.15.113:0/1941783968 shutdown_connections 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.550+0000 7f674bcd2640 1 -- 172.21.15.113:0/1941783968 wait complete. 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 Processor -- start 2024-04-25T00:01:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 -- start start 2024-04-25T00:01:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674406e6e0 0x7f67440b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6744144d60 0x7f674415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67441453c0 0x7f674415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6744145140 con 0x7f6744144d60 2024-04-25T00:01:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6744070e10 con 0x7f67441453c0 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674bcd2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6744071a40 con 0x7f674406e6e0 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674926d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6744144d60 0x7f674415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674926d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6744144d60 0x7f674415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59818/0 (socket says 172.21.15.113:59818) 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.551+0000 7f674926d640 1 -- 172.21.15.113:0/2711368724 learned_addr learned my addr 172.21.15.113:0/2711368724 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674a26f640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67441453c0 0x7f674415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f6749a6e640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674406e6e0 0x7f67440b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674926d640 1 -- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674406e6e0 msgr2=0x7f67440b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674926d640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674406e6e0 0x7f67440b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674926d640 1 -- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67441453c0 msgr2=0x7f674415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674926d640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67441453c0 0x7f674415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674926d640 1 -- 172.21.15.113:0/2711368724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67441f4440 con 0x7f6744144d60 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674926d640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6744144d60 0x7f674415aa80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f673400c460 tx=0x7f673400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f673a7fc640 1 -- 172.21.15.113:0/2711368724 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734018070 con 0x7f6744144d60 2024-04-25T00:01:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f673a7fc640 1 -- 172.21.15.113:0/2711368724 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6734007520 con 0x7f6744144d60 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67441f4720 con 0x7f6744144d60 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f673a7fc640 1 -- 172.21.15.113:0/2711368724 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734004a20 con 0x7f6744144d60 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.552+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67441f51e0 con 0x7f6744144d60 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.553+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67441f48f0 con 0x7f6744144d60 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.553+0000 7f673a7fc640 1 -- 172.21.15.113:0/2711368724 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f673401b070 con 0x7f6744144d60 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.554+0000 7f673a7fc640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f672806be30 0x7f672806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.554+0000 7f673a7fc640 1 -- 172.21.15.113:0/2711368724 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f673408a260 con 0x7f6744144d60 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.554+0000 7f6749a6e640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f672806be30 0x7f672806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.554+0000 7f6749a6e640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f672806be30 0x7f672806e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6740000f60 tx=0x7f6740009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.558+0000 7f673a7fc640 1 -- 172.21.15.113:0/2711368724 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6734058c10 con 0x7f6744144d60 2024-04-25T00:01:39.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.728+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67440691a0 con 0x7f672806be30 2024-04-25T00:01:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.734+0000 7f673a7fc640 1 -- 172.21.15.113:0/2711368724 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246627 (secure 0 0 0) 0x7f67440691a0 con 0x7f672806be30 2024-04-25T00:01:39.737 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:39.738 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f672806be30 msgr2=0x7f672806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f672806be30 0x7f672806e2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6740000f60 tx=0x7f6740009040 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6744144d60 msgr2=0x7f674415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6744144d60 0x7f674415aa80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f673400c460 tx=0x7f673400c920 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 shutdown_connections 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f672806be30 0x7f672806e2f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67441453c0 0x7f674415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6744144d60 0x7f674415aa80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 --2- 172.21.15.113:0/2711368724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674406e6e0 0x7f67440b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 >> 172.21.15.113:0/2711368724 conn(0x7f67440b1a50 msgr2=0x7f674406cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 shutdown_connections 2024-04-25T00:01:39.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.737+0000 7f674bcd2640 1 -- 172.21.15.113:0/2711368724 wait complete. 2024-04-25T00:01:39.748 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-25T00:01:39.150358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289320,"kb_used_data":6576,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685400,"statfs":{"total":773094113280,"available":772797849600,"internally_reserved":0,"allocated":6733824,"data_stored":3480370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.703398"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463581+0000","last_change":"2024-04-25T00:01:35.463581+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463581+0000","last_undegraded":"2024-04-25T00:01:35.463581+0000","last_fullsized":"2024-04-25T00:01:35.463581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457980+0000","last_change":"2024-04-25T00:01:35.457980+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457980+0000","last_undegraded":"2024-04-25T00:01:35.457980+0000","last_fullsized":"2024-04-25T00:01:35.457980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460964+0000","last_change":"2024-04-25T00:01:35.460964+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460964+0000","last_undegraded":"2024-04-25T00:01:35.460964+0000","last_fullsized":"2024-04-25T00:01:35.460964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465826+0000","last_change":"2024-04-25T00:01:35.465826+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465826+0000","last_undegraded":"2024-04-25T00:01:35.465826+0000","last_fullsized":"2024-04-25T00:01:35.465826+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454449+0000","last_change":"2024-04-25T00:01:35.454449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454449+0000","last_undegraded":"2024-04-25T00:01:35.454449+0000","last_fullsized":"2024-04-25T00:01:35.454449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461127+0000","last_change":"2024-04-25T00:01:35.461127+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461127+0000","last_undegraded":"2024-04-25T00:01:35.461127+0000","last_fullsized":"2024-04-25T00:01:35.461127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463139+0000","last_change":"2024-04-25T00:01:35.463139+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463139+0000","last_undegraded":"2024-04-25T00:01:35.463139+0000","last_fullsized":"2024-04-25T00:01:35.463139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462009+0000","last_change":"2024-04-25T00:01:35.462009+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462009+0000","last_undegraded":"2024-04-25T00:01:35.462009+0000","last_fullsized":"2024-04-25T00:01:35.462009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464866+0000","last_change":"2024-04-25T00:01:35.464866+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464866+0000","last_undegraded":"2024-04-25T00:01:35.464866+0000","last_fullsized":"2024-04-25T00:01:35.464866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.466538+0000","last_change":"2024-04-25T00:01:35.466538+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.466538+0000","last_undegraded":"2024-04-25T00:01:35.466538+0000","last_fullsized":"2024-04-25T00:01:35.466538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460560+0000","last_change":"2024-04-25T00:01:35.460560+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460560+0000","last_undegraded":"2024-04-25T00:01:35.460560+0000","last_fullsized":"2024-04-25T00:01:35.460560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454445+0000","last_change":"2024-04-25T00:01:35.454445+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454445+0000","last_undegraded":"2024-04-25T00:01:35.454445+0000","last_fullsized":"2024-04-25T00:01:35.454445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460130+0000","last_change":"2024-04-25T00:01:35.460130+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460130+0000","last_undegraded":"2024-04-25T00:01:35.460130+0000","last_fullsized":"2024-04-25T00:01:35.460130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460355+0000","last_change":"2024-04-25T00:01:35.460355+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460355+0000","last_undegraded":"2024-04-25T00:01:35.460355+0000","last_fullsized":"2024-04-25T00:01:35.460355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455420+0000","last_change":"2024-04-25T00:01:35.455420+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455420+0000","last_undegraded":"2024-04-25T00:01:35.455420+0000","last_fullsized":"2024-04-25T00:01:35.455420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461735+0000","last_change":"2024-04-25T00:01:35.461735+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461735+0000","last_undegraded":"2024-04-25T00:01:35.461735+0000","last_fullsized":"2024-04-25T00:01:35.461735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457312+0000","last_change":"2024-04-25T00:01:35.457312+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457312+0000","last_undegraded":"2024-04-25T00:01:35.457312+0000","last_fullsized":"2024-04-25T00:01:35.457312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461284+0000","last_change":"2024-04-25T00:01:35.461284+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461284+0000","last_undegraded":"2024-04-25T00:01:35.461284+0000","last_fullsized":"2024-04-25T00:01:35.461284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458433+0000","last_change":"2024-04-25T00:01:35.458433+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458433+0000","last_undegraded":"2024-04-25T00:01:35.458433+0000","last_fullsized":"2024-04-25T00:01:35.458433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461883+0000","last_change":"2024-04-25T00:01:35.461883+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461883+0000","last_undegraded":"2024-04-25T00:01:35.461883+0000","last_fullsized":"2024-04-25T00:01:35.461883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457341+0000","last_change":"2024-04-25T00:01:35.457341+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457341+0000","last_undegraded":"2024-04-25T00:01:35.457341+0000","last_fullsized":"2024-04-25T00:01:35.457341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455162+0000","last_change":"2024-04-25T00:01:35.455162+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455162+0000","last_undegraded":"2024-04-25T00:01:35.455162+0000","last_fullsized":"2024-04-25T00:01:35.455162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460808+0000","last_change":"2024-04-25T00:01:35.460808+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460808+0000","last_undegraded":"2024-04-25T00:01:35.460808+0000","last_fullsized":"2024-04-25T00:01:35.460808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459589+0000","last_change":"2024-04-25T00:01:35.459589+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459589+0000","last_undegraded":"2024-04-25T00:01:35.459589+0000","last_fullsized":"2024-04-25T00:01:35.459589+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460726+0000","last_change":"2024-04-25T00:01:35.460726+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460726+0000","last_undegraded":"2024-04-25T00:01:35.460726+0000","last_fullsized":"2024-04-25T00:01:35.460726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462561+0000","last_change":"2024-04-25T00:01:35.462561+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462561+0000","last_undegraded":"2024-04-25T00:01:35.462561+0000","last_fullsized":"2024-04-25T00:01:35.462561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462201+0000","last_change":"2024-04-25T00:01:35.462201+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462201+0000","last_undegraded":"2024-04-25T00:01:35.462201+0000","last_fullsized":"2024-04-25T00:01:35.462201+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458578+0000","last_change":"2024-04-25T00:01:35.458578+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458578+0000","last_undegraded":"2024-04-25T00:01:35.458578+0000","last_fullsized":"2024-04-25T00:01:35.458578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460240+0000","last_change":"2024-04-25T00:01:35.460240+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460240+0000","last_undegraded":"2024-04-25T00:01:35.460240+0000","last_fullsized":"2024-04-25T00:01:35.460240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454470+0000","last_change":"2024-04-25T00:01:35.454470+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454470+0000","last_undegraded":"2024-04-25T00:01:35.454470+0000","last_fullsized":"2024-04-25T00:01:35.454470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455791+0000","last_change":"2024-04-25T00:01:35.455791+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455791+0000","last_undegraded":"2024-04-25T00:01:35.455791+0000","last_fullsized":"2024-04-25T00:01:35.455791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471527+0000","last_change":"2024-04-25T00:01:35.469190+0000","last_active":"2024-04-25T00:01:35.471527+0000","last_peered":"2024-04-25T00:01:35.471527+0000","last_clean":"2024-04-25T00:01:35.471527+0000","last_became_active":"2024-04-25T00:01:35.468910+0000","last_became_peered":"2024-04-25T00:01:35.468910+0000","last_unstale":"2024-04-25T00:01:35.471527+0000","last_undegraded":"2024-04-25T00:01:35.471527+0000","last_fullsized":"2024-04-25T00:01:35.471527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.183879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465124+0000","last_change":"2024-04-25T00:01:35.465124+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465124+0000","last_undegraded":"2024-04-25T00:01:35.465124+0000","last_fullsized":"2024-04-25T00:01:35.465124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466862+0000","last_change":"2024-04-25T00:01:35.465786+0000","last_active":"2024-04-25T00:01:35.466862+0000","last_peered":"2024-04-25T00:01:35.466862+0000","last_clean":"2024-04-25T00:01:35.466862+0000","last_became_active":"2024-04-25T00:01:35.465527+0000","last_became_peered":"2024-04-25T00:01:35.465527+0000","last_unstale":"2024-04-25T00:01:35.466862+0000","last_undegraded":"2024-04-25T00:01:35.466862+0000","last_fullsized":"2024-04-25T00:01:35.466862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:48.237734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454877+0000","last_change":"2024-04-25T00:01:35.454877+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454877+0000","last_undegraded":"2024-04-25T00:01:35.454877+0000","last_fullsized":"2024-04-25T00:01:35.454877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471535+0000","last_change":"2024-04-25T00:01:35.469185+0000","last_active":"2024-04-25T00:01:35.471535+0000","last_peered":"2024-04-25T00:01:35.471535+0000","last_clean":"2024-04-25T00:01:35.471535+0000","last_became_active":"2024-04-25T00:01:35.468886+0000","last_became_peered":"2024-04-25T00:01:35.468886+0000","last_unstale":"2024-04-25T00:01:35.471535+0000","last_undegraded":"2024-04-25T00:01:35.471535+0000","last_fullsized":"2024-04-25T00:01:35.471535+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:16.817987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460218+0000","last_change":"2024-04-25T00:01:35.460218+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460218+0000","last_undegraded":"2024-04-25T00:01:35.460218+0000","last_fullsized":"2024-04-25T00:01:35.460218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469474+0000","last_change":"2024-04-25T00:01:35.467838+0000","last_active":"2024-04-25T00:01:35.469474+0000","last_peered":"2024-04-25T00:01:35.469474+0000","last_clean":"2024-04-25T00:01:35.469474+0000","last_became_active":"2024-04-25T00:01:35.467216+0000","last_became_peered":"2024-04-25T00:01:35.467216+0000","last_unstale":"2024-04-25T00:01:35.469474+0000","last_undegraded":"2024-04-25T00:01:35.469474+0000","last_fullsized":"2024-04-25T00:01:35.469474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461449+0000","last_change":"2024-04-25T00:01:35.461449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461449+0000","last_undegraded":"2024-04-25T00:01:35.461449+0000","last_fullsized":"2024-04-25T00:01:35.461449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470832+0000","last_change":"2024-04-25T00:01:35.469530+0000","last_active":"2024-04-25T00:01:35.470832+0000","last_peered":"2024-04-25T00:01:35.470832+0000","last_clean":"2024-04-25T00:01:35.470832+0000","last_became_active":"2024-04-25T00:01:35.468953+0000","last_became_peered":"2024-04-25T00:01:35.468953+0000","last_unstale":"2024-04-25T00:01:35.470832+0000","last_undegraded":"2024-04-25T00:01:35.470832+0000","last_fullsized":"2024-04-25T00:01:35.470832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461820+0000","last_change":"2024-04-25T00:01:35.461820+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461820+0000","last_undegraded":"2024-04-25T00:01:35.461820+0000","last_fullsized":"2024-04-25T00:01:35.461820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465278+0000","last_change":"2024-04-25T00:01:35.463716+0000","last_active":"2024-04-25T00:01:35.465278+0000","last_peered":"2024-04-25T00:01:35.465278+0000","last_clean":"2024-04-25T00:01:35.465278+0000","last_became_active":"2024-04-25T00:01:35.463153+0000","last_became_peered":"2024-04-25T00:01:35.463153+0000","last_unstale":"2024-04-25T00:01:35.465278+0000","last_undegraded":"2024-04-25T00:01:35.465278+0000","last_fullsized":"2024-04-25T00:01:35.465278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454489+0000","last_change":"2024-04-25T00:01:35.454489+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454489+0000","last_undegraded":"2024-04-25T00:01:35.454489+0000","last_fullsized":"2024-04-25T00:01:35.454489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465858+0000","last_change":"2024-04-25T00:01:35.465127+0000","last_active":"2024-04-25T00:01:35.465858+0000","last_peered":"2024-04-25T00:01:35.465858+0000","last_clean":"2024-04-25T00:01:35.465858+0000","last_became_active":"2024-04-25T00:01:35.464804+0000","last_became_peered":"2024-04-25T00:01:35.464804+0000","last_unstale":"2024-04-25T00:01:35.465858+0000","last_undegraded":"2024-04-25T00:01:35.465858+0000","last_fullsized":"2024-04-25T00:01:35.465858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:33.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461392+0000","last_change":"2024-04-25T00:01:35.461392+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461392+0000","last_undegraded":"2024-04-25T00:01:35.461392+0000","last_fullsized":"2024-04-25T00:01:35.461392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471287+0000","last_change":"2024-04-25T00:01:35.470685+0000","last_active":"2024-04-25T00:01:35.471287+0000","last_peered":"2024-04-25T00:01:35.471287+0000","last_clean":"2024-04-25T00:01:35.471287+0000","last_became_active":"2024-04-25T00:01:35.470365+0000","last_became_peered":"2024-04-25T00:01:35.470365+0000","last_unstale":"2024-04-25T00:01:35.471287+0000","last_undegraded":"2024-04-25T00:01:35.471287+0000","last_fullsized":"2024-04-25T00:01:35.471287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461419+0000","last_change":"2024-04-25T00:01:35.461419+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461419+0000","last_undegraded":"2024-04-25T00:01:35.461419+0000","last_fullsized":"2024-04-25T00:01:35.461419+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471453+0000","last_change":"2024-04-25T00:01:35.470524+0000","last_active":"2024-04-25T00:01:35.471453+0000","last_peered":"2024-04-25T00:01:35.471453+0000","last_clean":"2024-04-25T00:01:35.471453+0000","last_became_active":"2024-04-25T00:01:35.470224+0000","last_became_peered":"2024-04-25T00:01:35.470224+0000","last_unstale":"2024-04-25T00:01:35.471453+0000","last_undegraded":"2024-04-25T00:01:35.471453+0000","last_fullsized":"2024-04-25T00:01:35.471453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456965+0000","last_change":"2024-04-25T00:01:35.456965+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456965+0000","last_undegraded":"2024-04-25T00:01:35.456965+0000","last_fullsized":"2024-04-25T00:01:35.456965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470019+0000","last_change":"2024-04-25T00:01:35.466689+0000","last_active":"2024-04-25T00:01:35.470019+0000","last_peered":"2024-04-25T00:01:35.470019+0000","last_clean":"2024-04-25T00:01:35.470019+0000","last_became_active":"2024-04-25T00:01:35.466390+0000","last_became_peered":"2024-04-25T00:01:35.466390+0000","last_unstale":"2024-04-25T00:01:35.470019+0000","last_undegraded":"2024-04-25T00:01:35.470019+0000","last_fullsized":"2024-04-25T00:01:35.470019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:21.027526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464083+0000","last_change":"2024-04-25T00:01:35.464083+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464083+0000","last_undegraded":"2024-04-25T00:01:35.464083+0000","last_fullsized":"2024-04-25T00:01:35.464083+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470655+0000","last_change":"2024-04-25T00:01:35.469419+0000","last_active":"2024-04-25T00:01:35.470655+0000","last_peered":"2024-04-25T00:01:35.470655+0000","last_clean":"2024-04-25T00:01:35.470655+0000","last_became_active":"2024-04-25T00:01:35.468979+0000","last_became_peered":"2024-04-25T00:01:35.468979+0000","last_unstale":"2024-04-25T00:01:35.470655+0000","last_undegraded":"2024-04-25T00:01:35.470655+0000","last_fullsized":"2024-04-25T00:01:35.470655+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457481+0000","last_change":"2024-04-25T00:01:35.457481+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457481+0000","last_undegraded":"2024-04-25T00:01:35.457481+0000","last_fullsized":"2024-04-25T00:01:35.457481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468042+0000","last_change":"2024-04-25T00:01:35.466590+0000","last_active":"2024-04-25T00:01:35.468042+0000","last_peered":"2024-04-25T00:01:35.468042+0000","last_clean":"2024-04-25T00:01:35.468042+0000","last_became_active":"2024-04-25T00:01:35.466216+0000","last_became_peered":"2024-04-25T00:01:35.466216+0000","last_unstale":"2024-04-25T00:01:35.468042+0000","last_undegraded":"2024-04-25T00:01:35.468042+0000","last_fullsized":"2024-04-25T00:01:35.468042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457950+0000","last_change":"2024-04-25T00:01:35.457950+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457950+0000","last_undegraded":"2024-04-25T00:01:35.457950+0000","last_fullsized":"2024-04-25T00:01:35.457950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470984+0000","last_change":"2024-04-25T00:01:35.469270+0000","last_active":"2024-04-25T00:01:35.470984+0000","last_peered":"2024-04-25T00:01:35.470984+0000","last_clean":"2024-04-25T00:01:35.470984+0000","last_became_active":"2024-04-25T00:01:35.468644+0000","last_became_peered":"2024-04-25T00:01:35.468644+0000","last_unstale":"2024-04-25T00:01:35.470984+0000","last_undegraded":"2024-04-25T00:01:35.470984+0000","last_fullsized":"2024-04-25T00:01:35.470984+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:02.120913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458008+0000","last_change":"2024-04-25T00:01:35.458008+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458008+0000","last_undegraded":"2024-04-25T00:01:35.458008+0000","last_fullsized":"2024-04-25T00:01:35.458008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467827+0000","last_change":"2024-04-25T00:01:35.465772+0000","last_active":"2024-04-25T00:01:35.467827+0000","last_peered":"2024-04-25T00:01:35.467827+0000","last_clean":"2024-04-25T00:01:35.467827+0000","last_became_active":"2024-04-25T00:01:35.465310+0000","last_became_peered":"2024-04-25T00:01:35.465310+0000","last_unstale":"2024-04-25T00:01:35.467827+0000","last_undegraded":"2024-04-25T00:01:35.467827+0000","last_fullsized":"2024-04-25T00:01:35.467827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:34.269657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459893+0000","last_change":"2024-04-25T00:01:35.459893+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459893+0000","last_undegraded":"2024-04-25T00:01:35.459893+0000","last_fullsized":"2024-04-25T00:01:35.459893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468569+0000","last_change":"2024-04-25T00:01:35.467537+0000","last_active":"2024-04-25T00:01:35.468569+0000","last_peered":"2024-04-25T00:01:35.468569+0000","last_clean":"2024-04-25T00:01:35.468569+0000","last_became_active":"2024-04-25T00:01:35.467005+0000","last_became_peered":"2024-04-25T00:01:35.467005+0000","last_unstale":"2024-04-25T00:01:35.468569+0000","last_undegraded":"2024-04-25T00:01:35.468569+0000","last_fullsized":"2024-04-25T00:01:35.468569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:39.054815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456599+0000","last_change":"2024-04-25T00:01:35.456599+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456599+0000","last_undegraded":"2024-04-25T00:01:35.456599+0000","last_fullsized":"2024-04-25T00:01:35.456599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471280+0000","last_change":"2024-04-25T00:01:35.469625+0000","last_active":"2024-04-25T00:01:35.471280+0000","last_peered":"2024-04-25T00:01:35.471280+0000","last_clean":"2024-04-25T00:01:35.471280+0000","last_became_active":"2024-04-25T00:01:35.469302+0000","last_became_peered":"2024-04-25T00:01:35.469302+0000","last_unstale":"2024-04-25T00:01:35.471280+0000","last_undegraded":"2024-04-25T00:01:35.471280+0000","last_fullsized":"2024-04-25T00:01:35.471280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456805+0000","last_change":"2024-04-25T00:01:35.456805+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456805+0000","last_undegraded":"2024-04-25T00:01:35.456805+0000","last_fullsized":"2024-04-25T00:01:35.456805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470624+0000","last_change":"2024-04-25T00:01:35.468884+0000","last_active":"2024-04-25T00:01:35.470624+0000","last_peered":"2024-04-25T00:01:35.470624+0000","last_clean":"2024-04-25T00:01:35.470624+0000","last_became_active":"2024-04-25T00:01:35.468603+0000","last_became_peered":"2024-04-25T00:01:35.468603+0000","last_unstale":"2024-04-25T00:01:35.470624+0000","last_undegraded":"2024-04-25T00:01:35.470624+0000","last_fullsized":"2024-04-25T00:01:35.470624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:39.405188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458803+0000","last_change":"2024-04-25T00:01:35.458803+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458803+0000","last_undegraded":"2024-04-25T00:01:35.458803+0000","last_fullsized":"2024-04-25T00:01:35.458803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.462144+0000","last_change":"2024-04-25T00:01:35.458456+0000","last_active":"2024-04-25T00:01:35.462144+0000","last_peered":"2024-04-25T00:01:35.462144+0000","last_clean":"2024-04-25T00:01:35.462144+0000","last_became_active":"2024-04-25T00:01:35.458035+0000","last_became_peered":"2024-04-25T00:01:35.458035+0000","last_unstale":"2024-04-25T00:01:35.462144+0000","last_undegraded":"2024-04-25T00:01:35.462144+0000","last_fullsized":"2024-04-25T00:01:35.462144+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460400+0000","last_change":"2024-04-25T00:01:35.460400+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460400+0000","last_undegraded":"2024-04-25T00:01:35.460400+0000","last_fullsized":"2024-04-25T00:01:35.460400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467672+0000","last_change":"2024-04-25T00:01:35.466637+0000","last_active":"2024-04-25T00:01:35.467672+0000","last_peered":"2024-04-25T00:01:35.467672+0000","last_clean":"2024-04-25T00:01:35.467672+0000","last_became_active":"2024-04-25T00:01:35.466330+0000","last_became_peered":"2024-04-25T00:01:35.466330+0000","last_unstale":"2024-04-25T00:01:35.467672+0000","last_undegraded":"2024-04-25T00:01:35.467672+0000","last_fullsized":"2024-04-25T00:01:35.467672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:04.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456985+0000","last_change":"2024-04-25T00:01:35.456985+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456985+0000","last_undegraded":"2024-04-25T00:01:35.456985+0000","last_fullsized":"2024-04-25T00:01:35.456985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.463998+0000","last_change":"2024-04-25T00:01:35.460727+0000","last_active":"2024-04-25T00:01:35.463998+0000","last_peered":"2024-04-25T00:01:35.463998+0000","last_clean":"2024-04-25T00:01:35.463998+0000","last_became_active":"2024-04-25T00:01:35.460439+0000","last_became_peered":"2024-04-25T00:01:35.460439+0000","last_unstale":"2024-04-25T00:01:35.463998+0000","last_undegraded":"2024-04-25T00:01:35.463998+0000","last_fullsized":"2024-04-25T00:01:35.463998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457477+0000","last_change":"2024-04-25T00:01:35.457477+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457477+0000","last_undegraded":"2024-04-25T00:01:35.457477+0000","last_fullsized":"2024-04-25T00:01:35.457477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469402+0000","last_change":"2024-04-25T00:01:35.467452+0000","last_active":"2024-04-25T00:01:35.469402+0000","last_peered":"2024-04-25T00:01:35.469402+0000","last_clean":"2024-04-25T00:01:35.469402+0000","last_became_active":"2024-04-25T00:01:35.466644+0000","last_became_peered":"2024-04-25T00:01:35.466644+0000","last_unstale":"2024-04-25T00:01:35.469402+0000","last_undegraded":"2024-04-25T00:01:35.469402+0000","last_fullsized":"2024-04-25T00:01:35.469402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:08.725990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454546+0000","last_change":"2024-04-25T00:01:35.454546+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454546+0000","last_undegraded":"2024-04-25T00:01:35.454546+0000","last_fullsized":"2024-04-25T00:01:35.454546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.464899+0000","last_change":"2024-04-25T00:01:35.462553+0000","last_active":"2024-04-25T00:01:35.464899+0000","last_peered":"2024-04-25T00:01:35.464899+0000","last_clean":"2024-04-25T00:01:35.464899+0000","last_became_active":"2024-04-25T00:01:35.462284+0000","last_became_peered":"2024-04-25T00:01:35.462284+0000","last_unstale":"2024-04-25T00:01:35.464899+0000","last_undegraded":"2024-04-25T00:01:35.464899+0000","last_fullsized":"2024-04-25T00:01:35.464899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:05.412281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456125+0000","last_change":"2024-04-25T00:01:35.456125+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456125+0000","last_undegraded":"2024-04-25T00:01:35.456125+0000","last_fullsized":"2024-04-25T00:01:35.456125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471239+0000","last_change":"2024-04-25T00:01:35.470548+0000","last_active":"2024-04-25T00:01:35.471239+0000","last_peered":"2024-04-25T00:01:35.471239+0000","last_clean":"2024-04-25T00:01:35.471239+0000","last_became_active":"2024-04-25T00:01:35.470230+0000","last_became_peered":"2024-04-25T00:01:35.470230+0000","last_unstale":"2024-04-25T00:01:35.471239+0000","last_undegraded":"2024-04-25T00:01:35.471239+0000","last_fullsized":"2024-04-25T00:01:35.471239+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459032+0000","last_change":"2024-04-25T00:01:35.459032+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459032+0000","last_undegraded":"2024-04-25T00:01:35.459032+0000","last_fullsized":"2024-04-25T00:01:35.459032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470245+0000","last_change":"2024-04-25T00:01:35.468183+0000","last_active":"2024-04-25T00:01:35.470245+0000","last_peered":"2024-04-25T00:01:35.470245+0000","last_clean":"2024-04-25T00:01:35.470245+0000","last_became_active":"2024-04-25T00:01:35.467905+0000","last_became_peered":"2024-04-25T00:01:35.467905+0000","last_unstale":"2024-04-25T00:01:35.470245+0000","last_undegraded":"2024-04-25T00:01:35.470245+0000","last_fullsized":"2024-04-25T00:01:35.470245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:50.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458100+0000","last_change":"2024-04-25T00:01:35.458100+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458100+0000","last_undegraded":"2024-04-25T00:01:35.458100+0000","last_fullsized":"2024-04-25T00:01:35.458100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471197+0000","last_change":"2024-04-25T00:01:35.468807+0000","last_active":"2024-04-25T00:01:35.471197+0000","last_peered":"2024-04-25T00:01:35.471197+0000","last_clean":"2024-04-25T00:01:35.471197+0000","last_became_active":"2024-04-25T00:01:35.468521+0000","last_became_peered":"2024-04-25T00:01:35.468521+0000","last_unstale":"2024-04-25T00:01:35.471197+0000","last_undegraded":"2024-04-25T00:01:35.471197+0000","last_fullsized":"2024-04-25T00:01:35.471197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463586+0000","last_change":"2024-04-25T00:01:35.463586+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463586+0000","last_undegraded":"2024-04-25T00:01:35.463586+0000","last_fullsized":"2024-04-25T00:01:35.463586+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466296+0000","last_change":"2024-04-25T00:01:35.463380+0000","last_active":"2024-04-25T00:01:35.466296+0000","last_peered":"2024-04-25T00:01:35.466296+0000","last_clean":"2024-04-25T00:01:35.466296+0000","last_became_active":"2024-04-25T00:01:35.463052+0000","last_became_peered":"2024-04-25T00:01:35.463052+0000","last_unstale":"2024-04-25T00:01:35.466296+0000","last_undegraded":"2024-04-25T00:01:35.466296+0000","last_fullsized":"2024-04-25T00:01:35.466296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:29.474425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454529+0000","last_change":"2024-04-25T00:01:35.454529+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454529+0000","last_undegraded":"2024-04-25T00:01:35.454529+0000","last_fullsized":"2024-04-25T00:01:35.454529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469453+0000","last_change":"2024-04-25T00:01:35.468164+0000","last_active":"2024-04-25T00:01:35.469453+0000","last_peered":"2024-04-25T00:01:35.469453+0000","last_clean":"2024-04-25T00:01:35.469453+0000","last_became_active":"2024-04-25T00:01:35.467823+0000","last_became_peered":"2024-04-25T00:01:35.467823+0000","last_unstale":"2024-04-25T00:01:35.469453+0000","last_undegraded":"2024-04-25T00:01:35.469453+0000","last_fullsized":"2024-04-25T00:01:35.469453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460509+0000","last_change":"2024-04-25T00:01:35.460509+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460509+0000","last_undegraded":"2024-04-25T00:01:35.460509+0000","last_fullsized":"2024-04-25T00:01:35.460509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467004+0000","last_change":"2024-04-25T00:01:35.465879+0000","last_active":"2024-04-25T00:01:35.467004+0000","last_peered":"2024-04-25T00:01:35.467004+0000","last_clean":"2024-04-25T00:01:35.467004+0000","last_became_active":"2024-04-25T00:01:35.465602+0000","last_became_peered":"2024-04-25T00:01:35.465602+0000","last_unstale":"2024-04-25T00:01:35.467004+0000","last_undegraded":"2024-04-25T00:01:35.467004+0000","last_fullsized":"2024-04-25T00:01:35.467004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:32.955486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457579+0000","last_change":"2024-04-25T00:01:35.457579+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457579+0000","last_undegraded":"2024-04-25T00:01:35.457579+0000","last_fullsized":"2024-04-25T00:01:35.457579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468169+0000","last_change":"2024-04-25T00:01:35.467484+0000","last_active":"2024-04-25T00:01:35.468169+0000","last_peered":"2024-04-25T00:01:35.468169+0000","last_clean":"2024-04-25T00:01:35.468169+0000","last_became_active":"2024-04-25T00:01:35.466391+0000","last_became_peered":"2024-04-25T00:01:35.466391+0000","last_unstale":"2024-04-25T00:01:35.468169+0000","last_undegraded":"2024-04-25T00:01:35.468169+0000","last_fullsized":"2024-04-25T00:01:35.468169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463048+0000","last_change":"2024-04-25T00:01:35.463048+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463048+0000","last_undegraded":"2024-04-25T00:01:35.463048+0000","last_fullsized":"2024-04-25T00:01:35.463048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470685+0000","last_change":"2024-04-25T00:01:35.469390+0000","last_active":"2024-04-25T00:01:35.470685+0000","last_peered":"2024-04-25T00:01:35.470685+0000","last_clean":"2024-04-25T00:01:35.470685+0000","last_became_active":"2024-04-25T00:01:35.469128+0000","last_became_peered":"2024-04-25T00:01:35.469128+0000","last_unstale":"2024-04-25T00:01:35.470685+0000","last_undegraded":"2024-04-25T00:01:35.470685+0000","last_fullsized":"2024-04-25T00:01:35.470685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.684750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460347+0000","last_change":"2024-04-25T00:01:35.460347+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460347+0000","last_undegraded":"2024-04-25T00:01:35.460347+0000","last_fullsized":"2024-04-25T00:01:35.460347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470837+0000","last_change":"2024-04-25T00:01:35.468981+0000","last_active":"2024-04-25T00:01:35.470837+0000","last_peered":"2024-04-25T00:01:35.470837+0000","last_clean":"2024-04-25T00:01:35.470837+0000","last_became_active":"2024-04-25T00:01:35.468687+0000","last_became_peered":"2024-04-25T00:01:35.468687+0000","last_unstale":"2024-04-25T00:01:35.470837+0000","last_undegraded":"2024-04-25T00:01:35.470837+0000","last_fullsized":"2024-04-25T00:01:35.470837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:56.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460678+0000","last_change":"2024-04-25T00:01:35.460678+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460678+0000","last_undegraded":"2024-04-25T00:01:35.460678+0000","last_fullsized":"2024-04-25T00:01:35.460678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468470+0000","last_change":"2024-04-25T00:01:35.467166+0000","last_active":"2024-04-25T00:01:35.468470+0000","last_peered":"2024-04-25T00:01:35.468470+0000","last_clean":"2024-04-25T00:01:35.468470+0000","last_became_active":"2024-04-25T00:01:35.466904+0000","last_became_peered":"2024-04-25T00:01:35.466904+0000","last_unstale":"2024-04-25T00:01:35.468470+0000","last_undegraded":"2024-04-25T00:01:35.468470+0000","last_fullsized":"2024-04-25T00:01:35.468470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:43.861535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.461537+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:01:35.461537+0000","last_peered":"2024-04-25T00:01:35.461537+0000","last_clean":"2024-04-25T00:01:35.461537+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:01:35.461537+0000","last_undegraded":"2024-04-25T00:01:35.461537+0000","last_fullsized":"2024-04-25T00:01:35.461537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44924,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326916,"statfs":{"total":96636764160,"available":96590761984,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:39.752 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:01:39.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.885+0000 7f45c7d5a640 1 Processor -- start 2024-04-25T00:01:39.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.885+0000 7f45c7d5a640 1 -- start start 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.886+0000 7f45c7d5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 0x7f45c00b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.886+0000 7f45c7d5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f45c00b2de0 0x7f45c00b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.886+0000 7f45c7d5a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c00ba060 con 0x7f45c00d0ff0 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.886+0000 7f45c7d5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f45c00b3630 con 0x7f45c00b2de0 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.886+0000 7f45c7d5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45c00b8320 con 0x7f45c015e570 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.886+0000 7f45c5af6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f45c00d0ff0 0x7f45c00d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53492/0 (socket says 172.21.15.113:53492) 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.886+0000 7f45c5af6640 1 -- 172.21.15.113:0/3005765972 learned_addr learned my addr 172.21.15.113:0/3005765972 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348090459 0 0) 0x7f45c00ba060 con 0x7f45c00d0ff0 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a8003520 con 0x7f45c00d0ff0 2024-04-25T00:01:39.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c62f7640 1 --2- 172.21.15.113:0/3005765972 >> v2:172.21.15.178:3300/0 conn(0x7f45c00b2de0 0x7f45c00b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 0x7f45c00b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1862721202 0 0) 0x7f45a8003520 con 0x7f45c00d0ff0 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 msgr2=0x7f45c00b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 0x7f45c00b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 >> v2:172.21.15.178:3300/0 conn(0x7f45c00b2de0 msgr2=0x7f45c00b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 --2- 172.21.15.113:0/3005765972 >> v2:172.21.15.178:3300/0 conn(0x7f45c00b2de0 0x7f45c00b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c00b8600 con 0x7f45c00d0ff0 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f45b8001670 con 0x7f45c00d0ff0 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c62f7640 1 --2- 172.21.15.113:0/3005765972 >> v2:172.21.15.178:3300/0 conn(0x7f45c00b2de0 0x7f45c00b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 >> v1:172.21.15.113:6789/0 conn(0x7f45c00d0ff0 legacy=0x7f45c00d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 0x7f45a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8003fe0 0x7f45a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b3630 con 0x7f45a8003fe0 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b8320 con 0x7f45a8004d40 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45a800bb50 con 0x7f45c015e570 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8003fe0 0x7f45a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c5af6640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 0x7f45a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c62f7640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c52f5640 1 -- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 msgr2=0x7f45a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45c015e570 0x7f45a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c52f5640 1 -- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a8004d40 msgr2=0x7f45a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.887+0000 7f45c52f5640 1 -- 172.21.15.113:0/3005765972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a8003520 con 0x7f45a8003fe0 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.888+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8003fe0 0x7f45a8004800 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f45b400ba10 tx=0x7f45a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d2b36cc2bb06a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.888+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4030070 con 0x7f45a8003fe0 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.888+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b402adb0 con 0x7f45a8003fe0 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.888+0000 7f45c4af4640 1 -- 172.21.15.113:0/3005765972 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400b610 con 0x7f45a8003fe0 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.889+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3005765972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8003fe0 msgr2=0x7f45a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.889+0000 7f45c7d5a640 1 --2- 172.21.15.113:0/3005765972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8003fe0 0x7f45a8004800 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f45b400ba10 tx=0x7f45a8005830 comp rx=0 tx=0).stop 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.889+0000 7f45c62f7640 1 -- 172.21.15.113:0/3005765972 reap_dead start 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.889+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3005765972 shutdown_connections 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.889+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3005765972 >> 172.21.15.113:0/3005765972 conn(0x7f45c0071b00 msgr2=0x7f45c0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.889+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3005765972 shutdown_connections 2024-04-25T00:01:39.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.889+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3005765972 wait complete. 2024-04-25T00:01:39.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.890+0000 7f45c7d5a640 1 Processor -- start 2024-04-25T00:01:39.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.890+0000 7f45c7d5a640 1 -- start start 2024-04-25T00:01:39.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a8003fe0 0x7f45c00c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45c00b4b30 0x7f45c00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45c01604f0 0x7f45c0160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b9960 con 0x7f45c00b4b30 2024-04-25T00:01:39.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b8bc0 con 0x7f45c01604f0 2024-04-25T00:01:39.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45c00b9ae0 con 0x7f45a8003fe0 2024-04-25T00:01:39.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45c00b4b30 0x7f45c00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45c00b4b30 0x7f45c00b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59838/0 (socket says 172.21.15.113:59838) 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 -- 172.21.15.113:0/3460974568 learned_addr learned my addr 172.21.15.113:0/3460974568 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c5af6640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a8003fe0 0x7f45c00c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c62f7640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45c01604f0 0x7f45c0160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 -- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a8003fe0 msgr2=0x7f45c00c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a8003fe0 0x7f45c00c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 -- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45c01604f0 msgr2=0x7f45c0160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45c01604f0 0x7f45c0160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 -- 172.21.15.113:0/3460974568 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c015e950 con 0x7f45c00b4b30 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c52f5640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45c00b4b30 0x7f45c00b4fb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f45b400bb30 tx=0x7f45b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45be7fc640 1 -- 172.21.15.113:0/3460974568 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4043020 con 0x7f45c00b4b30 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45be7fc640 1 -- 172.21.15.113:0/3460974568 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b403f470 con 0x7f45c00b4b30 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c015ebf0 con 0x7f45c00b4b30 2024-04-25T00:01:39.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45be7fc640 1 -- 172.21.15.113:0/3460974568 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b403f860 con 0x7f45c00b4b30 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.891+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c015f1b0 con 0x7f45c00b4b30 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.892+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c00b54f0 con 0x7f45c00b4b30 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.893+0000 7f45be7fc640 1 -- 172.21.15.113:0/3460974568 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45b4046020 con 0x7f45c00b4b30 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.893+0000 7f45be7fc640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.893+0000 7f45c5af6640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.893+0000 7f45be7fc640 1 -- 172.21.15.113:0/3460974568 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f45b40ab6f0 con 0x7f45c00b4b30 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.894+0000 7f45c5af6640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45ac06be30 0x7f45ac06e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f45b8001d30 tx=0x7f45b8007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:39.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:39.898+0000 7f45be7fc640 1 -- 172.21.15.113:0/3460974568 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45b404d030 con 0x7f45c00b4b30 2024-04-25T00:01:40.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.067+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45c00691a0 con 0x7f45ac06be30 2024-04-25T00:01:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.072+0000 7f45be7fc640 1 -- 172.21.15.113:0/3460974568 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246627 (secure 0 0 0) 0x7f45c00691a0 con 0x7f45ac06be30 2024-04-25T00:01:40.075 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:40.076 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45ac06be30 msgr2=0x7f45ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45ac06be30 0x7f45ac06e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f45b8001d30 tx=0x7f45b8007970 comp rx=0 tx=0).stop 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45c00b4b30 msgr2=0x7f45c00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45c00b4b30 0x7f45c00b4fb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f45b400bb30 tx=0x7f45b4004730 comp rx=0 tx=0).stop 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 shutdown_connections 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45c01604f0 0x7f45c0160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45c00b4b30 0x7f45c00b4fb0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 --2- 172.21.15.113:0/3460974568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a8003fe0 0x7f45c00c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 >> 172.21.15.113:0/3460974568 conn(0x7f45c0071b00 msgr2=0x7f45c00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:40.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 shutdown_connections 2024-04-25T00:01:40.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.075+0000 7f45c7d5a640 1 -- 172.21.15.113:0/3460974568 wait complete. 2024-04-25T00:01:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-25T00:01:39.150358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289320,"kb_used_data":6576,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685400,"statfs":{"total":773094113280,"available":772797849600,"internally_reserved":0,"allocated":6733824,"data_stored":3480370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.703398"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463581+0000","last_change":"2024-04-25T00:01:35.463581+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463581+0000","last_undegraded":"2024-04-25T00:01:35.463581+0000","last_fullsized":"2024-04-25T00:01:35.463581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457980+0000","last_change":"2024-04-25T00:01:35.457980+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457980+0000","last_undegraded":"2024-04-25T00:01:35.457980+0000","last_fullsized":"2024-04-25T00:01:35.457980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460964+0000","last_change":"2024-04-25T00:01:35.460964+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460964+0000","last_undegraded":"2024-04-25T00:01:35.460964+0000","last_fullsized":"2024-04-25T00:01:35.460964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465826+0000","last_change":"2024-04-25T00:01:35.465826+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465826+0000","last_undegraded":"2024-04-25T00:01:35.465826+0000","last_fullsized":"2024-04-25T00:01:35.465826+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454449+0000","last_change":"2024-04-25T00:01:35.454449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454449+0000","last_undegraded":"2024-04-25T00:01:35.454449+0000","last_fullsized":"2024-04-25T00:01:35.454449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461127+0000","last_change":"2024-04-25T00:01:35.461127+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461127+0000","last_undegraded":"2024-04-25T00:01:35.461127+0000","last_fullsized":"2024-04-25T00:01:35.461127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463139+0000","last_change":"2024-04-25T00:01:35.463139+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463139+0000","last_undegraded":"2024-04-25T00:01:35.463139+0000","last_fullsized":"2024-04-25T00:01:35.463139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462009+0000","last_change":"2024-04-25T00:01:35.462009+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462009+0000","last_undegraded":"2024-04-25T00:01:35.462009+0000","last_fullsized":"2024-04-25T00:01:35.462009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464866+0000","last_change":"2024-04-25T00:01:35.464866+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464866+0000","last_undegraded":"2024-04-25T00:01:35.464866+0000","last_fullsized":"2024-04-25T00:01:35.464866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.466538+0000","last_change":"2024-04-25T00:01:35.466538+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.466538+0000","last_undegraded":"2024-04-25T00:01:35.466538+0000","last_fullsized":"2024-04-25T00:01:35.466538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460560+0000","last_change":"2024-04-25T00:01:35.460560+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460560+0000","last_undegraded":"2024-04-25T00:01:35.460560+0000","last_fullsized":"2024-04-25T00:01:35.460560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454445+0000","last_change":"2024-04-25T00:01:35.454445+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454445+0000","last_undegraded":"2024-04-25T00:01:35.454445+0000","last_fullsized":"2024-04-25T00:01:35.454445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460130+0000","last_change":"2024-04-25T00:01:35.460130+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460130+0000","last_undegraded":"2024-04-25T00:01:35.460130+0000","last_fullsized":"2024-04-25T00:01:35.460130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460355+0000","last_change":"2024-04-25T00:01:35.460355+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460355+0000","last_undegraded":"2024-04-25T00:01:35.460355+0000","last_fullsized":"2024-04-25T00:01:35.460355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455420+0000","last_change":"2024-04-25T00:01:35.455420+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455420+0000","last_undegraded":"2024-04-25T00:01:35.455420+0000","last_fullsized":"2024-04-25T00:01:35.455420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461735+0000","last_change":"2024-04-25T00:01:35.461735+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461735+0000","last_undegraded":"2024-04-25T00:01:35.461735+0000","last_fullsized":"2024-04-25T00:01:35.461735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457312+0000","last_change":"2024-04-25T00:01:35.457312+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457312+0000","last_undegraded":"2024-04-25T00:01:35.457312+0000","last_fullsized":"2024-04-25T00:01:35.457312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461284+0000","last_change":"2024-04-25T00:01:35.461284+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461284+0000","last_undegraded":"2024-04-25T00:01:35.461284+0000","last_fullsized":"2024-04-25T00:01:35.461284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458433+0000","last_change":"2024-04-25T00:01:35.458433+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458433+0000","last_undegraded":"2024-04-25T00:01:35.458433+0000","last_fullsized":"2024-04-25T00:01:35.458433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461883+0000","last_change":"2024-04-25T00:01:35.461883+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461883+0000","last_undegraded":"2024-04-25T00:01:35.461883+0000","last_fullsized":"2024-04-25T00:01:35.461883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457341+0000","last_change":"2024-04-25T00:01:35.457341+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457341+0000","last_undegraded":"2024-04-25T00:01:35.457341+0000","last_fullsized":"2024-04-25T00:01:35.457341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455162+0000","last_change":"2024-04-25T00:01:35.455162+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455162+0000","last_undegraded":"2024-04-25T00:01:35.455162+0000","last_fullsized":"2024-04-25T00:01:35.455162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460808+0000","last_change":"2024-04-25T00:01:35.460808+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460808+0000","last_undegraded":"2024-04-25T00:01:35.460808+0000","last_fullsized":"2024-04-25T00:01:35.460808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459589+0000","last_change":"2024-04-25T00:01:35.459589+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459589+0000","last_undegraded":"2024-04-25T00:01:35.459589+0000","last_fullsized":"2024-04-25T00:01:35.459589+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460726+0000","last_change":"2024-04-25T00:01:35.460726+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460726+0000","last_undegraded":"2024-04-25T00:01:35.460726+0000","last_fullsized":"2024-04-25T00:01:35.460726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462561+0000","last_change":"2024-04-25T00:01:35.462561+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462561+0000","last_undegraded":"2024-04-25T00:01:35.462561+0000","last_fullsized":"2024-04-25T00:01:35.462561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462201+0000","last_change":"2024-04-25T00:01:35.462201+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462201+0000","last_undegraded":"2024-04-25T00:01:35.462201+0000","last_fullsized":"2024-04-25T00:01:35.462201+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458578+0000","last_change":"2024-04-25T00:01:35.458578+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458578+0000","last_undegraded":"2024-04-25T00:01:35.458578+0000","last_fullsized":"2024-04-25T00:01:35.458578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460240+0000","last_change":"2024-04-25T00:01:35.460240+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460240+0000","last_undegraded":"2024-04-25T00:01:35.460240+0000","last_fullsized":"2024-04-25T00:01:35.460240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454470+0000","last_change":"2024-04-25T00:01:35.454470+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454470+0000","last_undegraded":"2024-04-25T00:01:35.454470+0000","last_fullsized":"2024-04-25T00:01:35.454470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455791+0000","last_change":"2024-04-25T00:01:35.455791+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455791+0000","last_undegraded":"2024-04-25T00:01:35.455791+0000","last_fullsized":"2024-04-25T00:01:35.455791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471527+0000","last_change":"2024-04-25T00:01:35.469190+0000","last_active":"2024-04-25T00:01:35.471527+0000","last_peered":"2024-04-25T00:01:35.471527+0000","last_clean":"2024-04-25T00:01:35.471527+0000","last_became_active":"2024-04-25T00:01:35.468910+0000","last_became_peered":"2024-04-25T00:01:35.468910+0000","last_unstale":"2024-04-25T00:01:35.471527+0000","last_undegraded":"2024-04-25T00:01:35.471527+0000","last_fullsized":"2024-04-25T00:01:35.471527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.183879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465124+0000","last_change":"2024-04-25T00:01:35.465124+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465124+0000","last_undegraded":"2024-04-25T00:01:35.465124+0000","last_fullsized":"2024-04-25T00:01:35.465124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466862+0000","last_change":"2024-04-25T00:01:35.465786+0000","last_active":"2024-04-25T00:01:35.466862+0000","last_peered":"2024-04-25T00:01:35.466862+0000","last_clean":"2024-04-25T00:01:35.466862+0000","last_became_active":"2024-04-25T00:01:35.465527+0000","last_became_peered":"2024-04-25T00:01:35.465527+0000","last_unstale":"2024-04-25T00:01:35.466862+0000","last_undegraded":"2024-04-25T00:01:35.466862+0000","last_fullsized":"2024-04-25T00:01:35.466862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:48.237734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454877+0000","last_change":"2024-04-25T00:01:35.454877+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454877+0000","last_undegraded":"2024-04-25T00:01:35.454877+0000","last_fullsized":"2024-04-25T00:01:35.454877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471535+0000","last_change":"2024-04-25T00:01:35.469185+0000","last_active":"2024-04-25T00:01:35.471535+0000","last_peered":"2024-04-25T00:01:35.471535+0000","last_clean":"2024-04-25T00:01:35.471535+0000","last_became_active":"2024-04-25T00:01:35.468886+0000","last_became_peered":"2024-04-25T00:01:35.468886+0000","last_unstale":"2024-04-25T00:01:35.471535+0000","last_undegraded":"2024-04-25T00:01:35.471535+0000","last_fullsized":"2024-04-25T00:01:35.471535+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:16.817987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460218+0000","last_change":"2024-04-25T00:01:35.460218+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460218+0000","last_undegraded":"2024-04-25T00:01:35.460218+0000","last_fullsized":"2024-04-25T00:01:35.460218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469474+0000","last_change":"2024-04-25T00:01:35.467838+0000","last_active":"2024-04-25T00:01:35.469474+0000","last_peered":"2024-04-25T00:01:35.469474+0000","last_clean":"2024-04-25T00:01:35.469474+0000","last_became_active":"2024-04-25T00:01:35.467216+0000","last_became_peered":"2024-04-25T00:01:35.467216+0000","last_unstale":"2024-04-25T00:01:35.469474+0000","last_undegraded":"2024-04-25T00:01:35.469474+0000","last_fullsized":"2024-04-25T00:01:35.469474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461449+0000","last_change":"2024-04-25T00:01:35.461449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461449+0000","last_undegraded":"2024-04-25T00:01:35.461449+0000","last_fullsized":"2024-04-25T00:01:35.461449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470832+0000","last_change":"2024-04-25T00:01:35.469530+0000","last_active":"2024-04-25T00:01:35.470832+0000","last_peered":"2024-04-25T00:01:35.470832+0000","last_clean":"2024-04-25T00:01:35.470832+0000","last_became_active":"2024-04-25T00:01:35.468953+0000","last_became_peered":"2024-04-25T00:01:35.468953+0000","last_unstale":"2024-04-25T00:01:35.470832+0000","last_undegraded":"2024-04-25T00:01:35.470832+0000","last_fullsized":"2024-04-25T00:01:35.470832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461820+0000","last_change":"2024-04-25T00:01:35.461820+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461820+0000","last_undegraded":"2024-04-25T00:01:35.461820+0000","last_fullsized":"2024-04-25T00:01:35.461820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465278+0000","last_change":"2024-04-25T00:01:35.463716+0000","last_active":"2024-04-25T00:01:35.465278+0000","last_peered":"2024-04-25T00:01:35.465278+0000","last_clean":"2024-04-25T00:01:35.465278+0000","last_became_active":"2024-04-25T00:01:35.463153+0000","last_became_peered":"2024-04-25T00:01:35.463153+0000","last_unstale":"2024-04-25T00:01:35.465278+0000","last_undegraded":"2024-04-25T00:01:35.465278+0000","last_fullsized":"2024-04-25T00:01:35.465278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454489+0000","last_change":"2024-04-25T00:01:35.454489+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454489+0000","last_undegraded":"2024-04-25T00:01:35.454489+0000","last_fullsized":"2024-04-25T00:01:35.454489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465858+0000","last_change":"2024-04-25T00:01:35.465127+0000","last_active":"2024-04-25T00:01:35.465858+0000","last_peered":"2024-04-25T00:01:35.465858+0000","last_clean":"2024-04-25T00:01:35.465858+0000","last_became_active":"2024-04-25T00:01:35.464804+0000","last_became_peered":"2024-04-25T00:01:35.464804+0000","last_unstale":"2024-04-25T00:01:35.465858+0000","last_undegraded":"2024-04-25T00:01:35.465858+0000","last_fullsized":"2024-04-25T00:01:35.465858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:33.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461392+0000","last_change":"2024-04-25T00:01:35.461392+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461392+0000","last_undegraded":"2024-04-25T00:01:35.461392+0000","last_fullsized":"2024-04-25T00:01:35.461392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471287+0000","last_change":"2024-04-25T00:01:35.470685+0000","last_active":"2024-04-25T00:01:35.471287+0000","last_peered":"2024-04-25T00:01:35.471287+0000","last_clean":"2024-04-25T00:01:35.471287+0000","last_became_active":"2024-04-25T00:01:35.470365+0000","last_became_peered":"2024-04-25T00:01:35.470365+0000","last_unstale":"2024-04-25T00:01:35.471287+0000","last_undegraded":"2024-04-25T00:01:35.471287+0000","last_fullsized":"2024-04-25T00:01:35.471287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461419+0000","last_change":"2024-04-25T00:01:35.461419+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461419+0000","last_undegraded":"2024-04-25T00:01:35.461419+0000","last_fullsized":"2024-04-25T00:01:35.461419+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471453+0000","last_change":"2024-04-25T00:01:35.470524+0000","last_active":"2024-04-25T00:01:35.471453+0000","last_peered":"2024-04-25T00:01:35.471453+0000","last_clean":"2024-04-25T00:01:35.471453+0000","last_became_active":"2024-04-25T00:01:35.470224+0000","last_became_peered":"2024-04-25T00:01:35.470224+0000","last_unstale":"2024-04-25T00:01:35.471453+0000","last_undegraded":"2024-04-25T00:01:35.471453+0000","last_fullsized":"2024-04-25T00:01:35.471453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456965+0000","last_change":"2024-04-25T00:01:35.456965+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456965+0000","last_undegraded":"2024-04-25T00:01:35.456965+0000","last_fullsized":"2024-04-25T00:01:35.456965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470019+0000","last_change":"2024-04-25T00:01:35.466689+0000","last_active":"2024-04-25T00:01:35.470019+0000","last_peered":"2024-04-25T00:01:35.470019+0000","last_clean":"2024-04-25T00:01:35.470019+0000","last_became_active":"2024-04-25T00:01:35.466390+0000","last_became_peered":"2024-04-25T00:01:35.466390+0000","last_unstale":"2024-04-25T00:01:35.470019+0000","last_undegraded":"2024-04-25T00:01:35.470019+0000","last_fullsized":"2024-04-25T00:01:35.470019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:21.027526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464083+0000","last_change":"2024-04-25T00:01:35.464083+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464083+0000","last_undegraded":"2024-04-25T00:01:35.464083+0000","last_fullsized":"2024-04-25T00:01:35.464083+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470655+0000","last_change":"2024-04-25T00:01:35.469419+0000","last_active":"2024-04-25T00:01:35.470655+0000","last_peered":"2024-04-25T00:01:35.470655+0000","last_clean":"2024-04-25T00:01:35.470655+0000","last_became_active":"2024-04-25T00:01:35.468979+0000","last_became_peered":"2024-04-25T00:01:35.468979+0000","last_unstale":"2024-04-25T00:01:35.470655+0000","last_undegraded":"2024-04-25T00:01:35.470655+0000","last_fullsized":"2024-04-25T00:01:35.470655+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457481+0000","last_change":"2024-04-25T00:01:35.457481+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457481+0000","last_undegraded":"2024-04-25T00:01:35.457481+0000","last_fullsized":"2024-04-25T00:01:35.457481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468042+0000","last_change":"2024-04-25T00:01:35.466590+0000","last_active":"2024-04-25T00:01:35.468042+0000","last_peered":"2024-04-25T00:01:35.468042+0000","last_clean":"2024-04-25T00:01:35.468042+0000","last_became_active":"2024-04-25T00:01:35.466216+0000","last_became_peered":"2024-04-25T00:01:35.466216+0000","last_unstale":"2024-04-25T00:01:35.468042+0000","last_undegraded":"2024-04-25T00:01:35.468042+0000","last_fullsized":"2024-04-25T00:01:35.468042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457950+0000","last_change":"2024-04-25T00:01:35.457950+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457950+0000","last_undegraded":"2024-04-25T00:01:35.457950+0000","last_fullsized":"2024-04-25T00:01:35.457950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470984+0000","last_change":"2024-04-25T00:01:35.469270+0000","last_active":"2024-04-25T00:01:35.470984+0000","last_peered":"2024-04-25T00:01:35.470984+0000","last_clean":"2024-04-25T00:01:35.470984+0000","last_became_active":"2024-04-25T00:01:35.468644+0000","last_became_peered":"2024-04-25T00:01:35.468644+0000","last_unstale":"2024-04-25T00:01:35.470984+0000","last_undegraded":"2024-04-25T00:01:35.470984+0000","last_fullsized":"2024-04-25T00:01:35.470984+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:02.120913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458008+0000","last_change":"2024-04-25T00:01:35.458008+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458008+0000","last_undegraded":"2024-04-25T00:01:35.458008+0000","last_fullsized":"2024-04-25T00:01:35.458008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467827+0000","last_change":"2024-04-25T00:01:35.465772+0000","last_active":"2024-04-25T00:01:35.467827+0000","last_peered":"2024-04-25T00:01:35.467827+0000","last_clean":"2024-04-25T00:01:35.467827+0000","last_became_active":"2024-04-25T00:01:35.465310+0000","last_became_peered":"2024-04-25T00:01:35.465310+0000","last_unstale":"2024-04-25T00:01:35.467827+0000","last_undegraded":"2024-04-25T00:01:35.467827+0000","last_fullsized":"2024-04-25T00:01:35.467827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:34.269657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459893+0000","last_change":"2024-04-25T00:01:35.459893+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459893+0000","last_undegraded":"2024-04-25T00:01:35.459893+0000","last_fullsized":"2024-04-25T00:01:35.459893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468569+0000","last_change":"2024-04-25T00:01:35.467537+0000","last_active":"2024-04-25T00:01:35.468569+0000","last_peered":"2024-04-25T00:01:35.468569+0000","last_clean":"2024-04-25T00:01:35.468569+0000","last_became_active":"2024-04-25T00:01:35.467005+0000","last_became_peered":"2024-04-25T00:01:35.467005+0000","last_unstale":"2024-04-25T00:01:35.468569+0000","last_undegraded":"2024-04-25T00:01:35.468569+0000","last_fullsized":"2024-04-25T00:01:35.468569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:39.054815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456599+0000","last_change":"2024-04-25T00:01:35.456599+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456599+0000","last_undegraded":"2024-04-25T00:01:35.456599+0000","last_fullsized":"2024-04-25T00:01:35.456599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471280+0000","last_change":"2024-04-25T00:01:35.469625+0000","last_active":"2024-04-25T00:01:35.471280+0000","last_peered":"2024-04-25T00:01:35.471280+0000","last_clean":"2024-04-25T00:01:35.471280+0000","last_became_active":"2024-04-25T00:01:35.469302+0000","last_became_peered":"2024-04-25T00:01:35.469302+0000","last_unstale":"2024-04-25T00:01:35.471280+0000","last_undegraded":"2024-04-25T00:01:35.471280+0000","last_fullsized":"2024-04-25T00:01:35.471280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456805+0000","last_change":"2024-04-25T00:01:35.456805+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456805+0000","last_undegraded":"2024-04-25T00:01:35.456805+0000","last_fullsized":"2024-04-25T00:01:35.456805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470624+0000","last_change":"2024-04-25T00:01:35.468884+0000","last_active":"2024-04-25T00:01:35.470624+0000","last_peered":"2024-04-25T00:01:35.470624+0000","last_clean":"2024-04-25T00:01:35.470624+0000","last_became_active":"2024-04-25T00:01:35.468603+0000","last_became_peered":"2024-04-25T00:01:35.468603+0000","last_unstale":"2024-04-25T00:01:35.470624+0000","last_undegraded":"2024-04-25T00:01:35.470624+0000","last_fullsized":"2024-04-25T00:01:35.470624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:39.405188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458803+0000","last_change":"2024-04-25T00:01:35.458803+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458803+0000","last_undegraded":"2024-04-25T00:01:35.458803+0000","last_fullsized":"2024-04-25T00:01:35.458803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.462144+0000","last_change":"2024-04-25T00:01:35.458456+0000","last_active":"2024-04-25T00:01:35.462144+0000","last_peered":"2024-04-25T00:01:35.462144+0000","last_clean":"2024-04-25T00:01:35.462144+0000","last_became_active":"2024-04-25T00:01:35.458035+0000","last_became_peered":"2024-04-25T00:01:35.458035+0000","last_unstale":"2024-04-25T00:01:35.462144+0000","last_undegraded":"2024-04-25T00:01:35.462144+0000","last_fullsized":"2024-04-25T00:01:35.462144+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460400+0000","last_change":"2024-04-25T00:01:35.460400+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460400+0000","last_undegraded":"2024-04-25T00:01:35.460400+0000","last_fullsized":"2024-04-25T00:01:35.460400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467672+0000","last_change":"2024-04-25T00:01:35.466637+0000","last_active":"2024-04-25T00:01:35.467672+0000","last_peered":"2024-04-25T00:01:35.467672+0000","last_clean":"2024-04-25T00:01:35.467672+0000","last_became_active":"2024-04-25T00:01:35.466330+0000","last_became_peered":"2024-04-25T00:01:35.466330+0000","last_unstale":"2024-04-25T00:01:35.467672+0000","last_undegraded":"2024-04-25T00:01:35.467672+0000","last_fullsized":"2024-04-25T00:01:35.467672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:04.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456985+0000","last_change":"2024-04-25T00:01:35.456985+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456985+0000","last_undegraded":"2024-04-25T00:01:35.456985+0000","last_fullsized":"2024-04-25T00:01:35.456985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.463998+0000","last_change":"2024-04-25T00:01:35.460727+0000","last_active":"2024-04-25T00:01:35.463998+0000","last_peered":"2024-04-25T00:01:35.463998+0000","last_clean":"2024-04-25T00:01:35.463998+0000","last_became_active":"2024-04-25T00:01:35.460439+0000","last_became_peered":"2024-04-25T00:01:35.460439+0000","last_unstale":"2024-04-25T00:01:35.463998+0000","last_undegraded":"2024-04-25T00:01:35.463998+0000","last_fullsized":"2024-04-25T00:01:35.463998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457477+0000","last_change":"2024-04-25T00:01:35.457477+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457477+0000","last_undegraded":"2024-04-25T00:01:35.457477+0000","last_fullsized":"2024-04-25T00:01:35.457477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469402+0000","last_change":"2024-04-25T00:01:35.467452+0000","last_active":"2024-04-25T00:01:35.469402+0000","last_peered":"2024-04-25T00:01:35.469402+0000","last_clean":"2024-04-25T00:01:35.469402+0000","last_became_active":"2024-04-25T00:01:35.466644+0000","last_became_peered":"2024-04-25T00:01:35.466644+0000","last_unstale":"2024-04-25T00:01:35.469402+0000","last_undegraded":"2024-04-25T00:01:35.469402+0000","last_fullsized":"2024-04-25T00:01:35.469402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:08.725990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454546+0000","last_change":"2024-04-25T00:01:35.454546+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454546+0000","last_undegraded":"2024-04-25T00:01:35.454546+0000","last_fullsized":"2024-04-25T00:01:35.454546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.464899+0000","last_change":"2024-04-25T00:01:35.462553+0000","last_active":"2024-04-25T00:01:35.464899+0000","last_peered":"2024-04-25T00:01:35.464899+0000","last_clean":"2024-04-25T00:01:35.464899+0000","last_became_active":"2024-04-25T00:01:35.462284+0000","last_became_peered":"2024-04-25T00:01:35.462284+0000","last_unstale":"2024-04-25T00:01:35.464899+0000","last_undegraded":"2024-04-25T00:01:35.464899+0000","last_fullsized":"2024-04-25T00:01:35.464899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:05.412281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456125+0000","last_change":"2024-04-25T00:01:35.456125+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456125+0000","last_undegraded":"2024-04-25T00:01:35.456125+0000","last_fullsized":"2024-04-25T00:01:35.456125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471239+0000","last_change":"2024-04-25T00:01:35.470548+0000","last_active":"2024-04-25T00:01:35.471239+0000","last_peered":"2024-04-25T00:01:35.471239+0000","last_clean":"2024-04-25T00:01:35.471239+0000","last_became_active":"2024-04-25T00:01:35.470230+0000","last_became_peered":"2024-04-25T00:01:35.470230+0000","last_unstale":"2024-04-25T00:01:35.471239+0000","last_undegraded":"2024-04-25T00:01:35.471239+0000","last_fullsized":"2024-04-25T00:01:35.471239+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459032+0000","last_change":"2024-04-25T00:01:35.459032+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459032+0000","last_undegraded":"2024-04-25T00:01:35.459032+0000","last_fullsized":"2024-04-25T00:01:35.459032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470245+0000","last_change":"2024-04-25T00:01:35.468183+0000","last_active":"2024-04-25T00:01:35.470245+0000","last_peered":"2024-04-25T00:01:35.470245+0000","last_clean":"2024-04-25T00:01:35.470245+0000","last_became_active":"2024-04-25T00:01:35.467905+0000","last_became_peered":"2024-04-25T00:01:35.467905+0000","last_unstale":"2024-04-25T00:01:35.470245+0000","last_undegraded":"2024-04-25T00:01:35.470245+0000","last_fullsized":"2024-04-25T00:01:35.470245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:50.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458100+0000","last_change":"2024-04-25T00:01:35.458100+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458100+0000","last_undegraded":"2024-04-25T00:01:35.458100+0000","last_fullsized":"2024-04-25T00:01:35.458100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471197+0000","last_change":"2024-04-25T00:01:35.468807+0000","last_active":"2024-04-25T00:01:35.471197+0000","last_peered":"2024-04-25T00:01:35.471197+0000","last_clean":"2024-04-25T00:01:35.471197+0000","last_became_active":"2024-04-25T00:01:35.468521+0000","last_became_peered":"2024-04-25T00:01:35.468521+0000","last_unstale":"2024-04-25T00:01:35.471197+0000","last_undegraded":"2024-04-25T00:01:35.471197+0000","last_fullsized":"2024-04-25T00:01:35.471197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463586+0000","last_change":"2024-04-25T00:01:35.463586+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463586+0000","last_undegraded":"2024-04-25T00:01:35.463586+0000","last_fullsized":"2024-04-25T00:01:35.463586+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466296+0000","last_change":"2024-04-25T00:01:35.463380+0000","last_active":"2024-04-25T00:01:35.466296+0000","last_peered":"2024-04-25T00:01:35.466296+0000","last_clean":"2024-04-25T00:01:35.466296+0000","last_became_active":"2024-04-25T00:01:35.463052+0000","last_became_peered":"2024-04-25T00:01:35.463052+0000","last_unstale":"2024-04-25T00:01:35.466296+0000","last_undegraded":"2024-04-25T00:01:35.466296+0000","last_fullsized":"2024-04-25T00:01:35.466296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:29.474425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454529+0000","last_change":"2024-04-25T00:01:35.454529+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454529+0000","last_undegraded":"2024-04-25T00:01:35.454529+0000","last_fullsized":"2024-04-25T00:01:35.454529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469453+0000","last_change":"2024-04-25T00:01:35.468164+0000","last_active":"2024-04-25T00:01:35.469453+0000","last_peered":"2024-04-25T00:01:35.469453+0000","last_clean":"2024-04-25T00:01:35.469453+0000","last_became_active":"2024-04-25T00:01:35.467823+0000","last_became_peered":"2024-04-25T00:01:35.467823+0000","last_unstale":"2024-04-25T00:01:35.469453+0000","last_undegraded":"2024-04-25T00:01:35.469453+0000","last_fullsized":"2024-04-25T00:01:35.469453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460509+0000","last_change":"2024-04-25T00:01:35.460509+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460509+0000","last_undegraded":"2024-04-25T00:01:35.460509+0000","last_fullsized":"2024-04-25T00:01:35.460509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467004+0000","last_change":"2024-04-25T00:01:35.465879+0000","last_active":"2024-04-25T00:01:35.467004+0000","last_peered":"2024-04-25T00:01:35.467004+0000","last_clean":"2024-04-25T00:01:35.467004+0000","last_became_active":"2024-04-25T00:01:35.465602+0000","last_became_peered":"2024-04-25T00:01:35.465602+0000","last_unstale":"2024-04-25T00:01:35.467004+0000","last_undegraded":"2024-04-25T00:01:35.467004+0000","last_fullsized":"2024-04-25T00:01:35.467004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:32.955486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457579+0000","last_change":"2024-04-25T00:01:35.457579+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457579+0000","last_undegraded":"2024-04-25T00:01:35.457579+0000","last_fullsized":"2024-04-25T00:01:35.457579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468169+0000","last_change":"2024-04-25T00:01:35.467484+0000","last_active":"2024-04-25T00:01:35.468169+0000","last_peered":"2024-04-25T00:01:35.468169+0000","last_clean":"2024-04-25T00:01:35.468169+0000","last_became_active":"2024-04-25T00:01:35.466391+0000","last_became_peered":"2024-04-25T00:01:35.466391+0000","last_unstale":"2024-04-25T00:01:35.468169+0000","last_undegraded":"2024-04-25T00:01:35.468169+0000","last_fullsized":"2024-04-25T00:01:35.468169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463048+0000","last_change":"2024-04-25T00:01:35.463048+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463048+0000","last_undegraded":"2024-04-25T00:01:35.463048+0000","last_fullsized":"2024-04-25T00:01:35.463048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470685+0000","last_change":"2024-04-25T00:01:35.469390+0000","last_active":"2024-04-25T00:01:35.470685+0000","last_peered":"2024-04-25T00:01:35.470685+0000","last_clean":"2024-04-25T00:01:35.470685+0000","last_became_active":"2024-04-25T00:01:35.469128+0000","last_became_peered":"2024-04-25T00:01:35.469128+0000","last_unstale":"2024-04-25T00:01:35.470685+0000","last_undegraded":"2024-04-25T00:01:35.470685+0000","last_fullsized":"2024-04-25T00:01:35.470685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.684750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460347+0000","last_change":"2024-04-25T00:01:35.460347+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460347+0000","last_undegraded":"2024-04-25T00:01:35.460347+0000","last_fullsized":"2024-04-25T00:01:35.460347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470837+0000","last_change":"2024-04-25T00:01:35.468981+0000","last_active":"2024-04-25T00:01:35.470837+0000","last_peered":"2024-04-25T00:01:35.470837+0000","last_clean":"2024-04-25T00:01:35.470837+0000","last_became_active":"2024-04-25T00:01:35.468687+0000","last_became_peered":"2024-04-25T00:01:35.468687+0000","last_unstale":"2024-04-25T00:01:35.470837+0000","last_undegraded":"2024-04-25T00:01:35.470837+0000","last_fullsized":"2024-04-25T00:01:35.470837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:56.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460678+0000","last_change":"2024-04-25T00:01:35.460678+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460678+0000","last_undegraded":"2024-04-25T00:01:35.460678+0000","last_fullsized":"2024-04-25T00:01:35.460678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468470+0000","last_change":"2024-04-25T00:01:35.467166+0000","last_active":"2024-04-25T00:01:35.468470+0000","last_peered":"2024-04-25T00:01:35.468470+0000","last_clean":"2024-04-25T00:01:35.468470+0000","last_became_active":"2024-04-25T00:01:35.466904+0000","last_became_peered":"2024-04-25T00:01:35.466904+0000","last_unstale":"2024-04-25T00:01:35.468470+0000","last_undegraded":"2024-04-25T00:01:35.468470+0000","last_fullsized":"2024-04-25T00:01:35.468470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:43.861535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.461537+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:01:35.461537+0000","last_peered":"2024-04-25T00:01:35.461537+0000","last_clean":"2024-04-25T00:01:35.461537+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:01:35.461537+0000","last_undegraded":"2024-04-25T00:01:35.461537+0000","last_fullsized":"2024-04-25T00:01:35.461537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44924,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326916,"statfs":{"total":96636764160,"available":96590761984,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:40.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.465794+0000', 'last_change': '2024-04-25T00:01:35.465794+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.465794+0000', 'last_undegraded': '2024-04-25T00:01:35.465794+0000', 'last_fullsized': '2024-04-25T00:01:35.465794+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.463581+0000', 'last_change': '2024-04-25T00:01:35.463581+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.463581+0000', 'last_undegraded': '2024-04-25T00:01:35.463581+0000', 'last_fullsized': '2024-04-25T00:01:35.463581+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.457980+0000', 'last_change': '2024-04-25T00:01:35.457980+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.457980+0000', 'last_undegraded': '2024-04-25T00:01:35.457980+0000', 'last_fullsized': '2024-04-25T00:01:35.457980+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460964+0000', 'last_change': '2024-04-25T00:01:35.460964+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460964+0000', 'last_undegraded': '2024-04-25T00:01:35.460964+0000', 'last_fullsized': '2024-04-25T00:01:35.460964+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-04-25T00:01:40.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.465826+0000', 'last_change': '2024-04-25T00:01:35.465826+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.465826+0000', 'last_undegraded': '2024-04-25T00:01:35.465826+0000', 'last_fullsized': '2024-04-25T00:01:35.465826+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.454449+0000', 'last_change': '2024-04-25T00:01:35.454449+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.454449+0000', 'last_undegraded': '2024-04-25T00:01:35.454449+0000', 'last_fullsized': '2024-04-25T00:01:35.454449+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461127+0000', 'last_change': '2024-04-25T00:01:35.461127+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461127+0000', 'last_undegraded': '2024-04-25T00:01:35.461127+0000', 'last_fullsized': '2024-04-25T00:01:35.461127+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.463139+0000', 'last_change': '2024-04-25T00:01:35.463139+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.463139+0000', 'last_undegraded': '2024-04-25T00:01:35.463139+0000', 'last_fullsized': '2024-04-25T00:01:35.463139+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.462009+0000', 'last_change': '2024-04-25T00:01:35.462009+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.462009+0000', 'last_undegraded': '2024-04-25T00:01:35.462009+0000', 'last_fullsized': '2024-04-25T00:01:35.462009+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.464866+0000', 'last_change': '2024-04-25T00:01:35.464866+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.464866+0000', 'last_undegraded': '2024-04-25T00:01:35.464866+0000', 'last_fullsized': '2024-04-25T00:01:35.464866+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2024-04-25T00:01:40.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.466538+0000', 'last_change': '2024-04-25T00:01:35.466538+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.466538+0000', 'last_undegraded': '2024-04-25T00:01:35.466538+0000', 'last_fullsized': '2024-04-25T00:01:35.466538+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460560+0000', 'last_change': '2024-04-25T00:01:35.460560+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460560+0000', 'last_undegraded': '2024-04-25T00:01:35.460560+0000', 'last_fullsized': '2024-04-25T00:01:35.460560+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.454445+0000', 'last_change': '2024-04-25T00:01:35.454445+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.454445+0000', 'last_undegraded': '2024-04-25T00:01:35.454445+0000', 'last_fullsized': '2024-04-25T00:01:35.454445+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460130+0000', 'last_change': '2024-04-25T00:01:35.460130+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460130+0000', 'last_undegraded': '2024-04-25T00:01:35.460130+0000', 'last_fullsized': '2024-04-25T00:01:35.460130+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460355+0000', 'last_change': '2024-04-25T00:01:35.460355+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460355+0000', 'last_undegraded': '2024-04-25T00:01:35.460355+0000', 'last_fullsized': '2024-04-25T00:01:35.460355+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2024-04-25T00:01:40.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.455420+0000', 'last_change': '2024-04-25T00:01:35.455420+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.455420+0000', 'last_undegraded': '2024-04-25T00:01:35.455420+0000', 'last_fullsized': '2024-04-25T00:01:35.455420+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461735+0000', 'last_change': '2024-04-25T00:01:35.461735+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461735+0000', 'last_undegraded': '2024-04-25T00:01:35.461735+0000', 'last_fullsized': '2024-04-25T00:01:35.461735+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.457312+0000', 'last_change': '2024-04-25T00:01:35.457312+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.457312+0000', 'last_undegraded': '2024-04-25T00:01:35.457312+0000', 'last_fullsized': '2024-04-25T00:01:35.457312+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.465794+0000', 'last_change': '2024-04-25T00:01:35.465794+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.465794+0000', 'last_undegraded': '2024-04-25T00:01:35.465794+0000', 'last_fullsized': '2024-04-25T00:01:35.465794+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461284+0000', 'last_change': '2024-04-25T00:01:35.461284+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461284+0000', 'last_undegraded': '2024-04-25T00:01:35.461284+0000', 'last_fullsized': '2024-04-25T00:01:35.461284+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.458433+0000', 'last_change': '2024-04-25T00:01:35.458433+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.458433+0000', 'last_undegraded': '2024-04-25T00:01:35.458433+0000', 'last_fullsized': '2024-04-25T00:01:35.458433+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461883+0000', 'last_change': '2024-04-25T00:01:35.461883+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461883+0000', 'last_undegraded': '2024-04-25T00:01:35.461883+0000', 'last_fullsized': '2024-04-25T00:01:35.461883+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.457341+0000', 'last_change': '2024-04-25T00:01:35.457341+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.457341+0000', 'last_undegraded': '2024-04-25T00:01:35.457341+0000', 'last_fullsized': '2024-04-25T00:01:35.457341+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.455162+0000', 'last_change': '2024-04-25T00:01:35.455162+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.455162+0000', 'last_undegraded': '2024-04-25T00:01:35.455162+0000', 'last_fullsized': '2024-04-25T00:01:35.455162+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460808+0000', 'last_change': '2024-04-25T00:01:35.460808+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460808+0000', 'last_undegraded': '2024-04-25T00:01:35.460808+0000', 'last_fullsized': '2024-04-25T00:01:35.460808+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.459589+0000', 'last_change': '2024-04-25T00:01:35.459589+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.459589+0000', 'last_undegraded': '2024-04-25T00:01:35.459589+0000', 'last_fullsized': '2024-04-25T00:01:35.459589+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2024-04-25T00:01:40.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460726+0000', 'last_change': '2024-04-25T00:01:35.460726+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460726+0000', 'last_undegraded': '2024-04-25T00:01:35.460726+0000', 'last_fullsized': '2024-04-25T00:01:35.460726+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.462561+0000', 'last_change': '2024-04-25T00:01:35.462561+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.462561+0000', 'last_undegraded': '2024-04-25T00:01:35.462561+0000', 'last_fullsized': '2024-04-25T00:01:35.462561+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.462201+0000', 'last_change': '2024-04-25T00:01:35.462201+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.462201+0000', 'last_undegraded': '2024-04-25T00:01:35.462201+0000', 'last_fullsized': '2024-04-25T00:01:35.462201+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.458578+0000', 'last_change': '2024-04-25T00:01:35.458578+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.458578+0000', 'last_undegraded': '2024-04-25T00:01:35.458578+0000', 'last_fullsized': '2024-04-25T00:01:35.458578+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460240+0000', 'last_change': '2024-04-25T00:01:35.460240+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460240+0000', 'last_undegraded': '2024-04-25T00:01:35.460240+0000', 'last_fullsized': '2024-04-25T00:01:35.460240+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.454470+0000', 'last_change': '2024-04-25T00:01:35.454470+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.454470+0000', 'last_undegraded': '2024-04-25T00:01:35.454470+0000', 'last_fullsized': '2024-04-25T00:01:35.454470+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-04-25T00:01:40.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.455791+0000', 'last_change': '2024-04-25T00:01:35.455791+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.455791+0000', 'last_undegraded': '2024-04-25T00:01:35.455791+0000', 'last_fullsized': '2024-04-25T00:01:35.455791+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.465124+0000', 'last_change': '2024-04-25T00:01:35.465124+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.465124+0000', 'last_undegraded': '2024-04-25T00:01:35.465124+0000', 'last_fullsized': '2024-04-25T00:01:35.465124+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.454877+0000', 'last_change': '2024-04-25T00:01:35.454877+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.454877+0000', 'last_undegraded': '2024-04-25T00:01:35.454877+0000', 'last_fullsized': '2024-04-25T00:01:35.454877+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460218+0000', 'last_change': '2024-04-25T00:01:35.460218+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460218+0000', 'last_undegraded': '2024-04-25T00:01:35.460218+0000', 'last_fullsized': '2024-04-25T00:01:35.460218+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461449+0000', 'last_change': '2024-04-25T00:01:35.461449+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461449+0000', 'last_undegraded': '2024-04-25T00:01:35.461449+0000', 'last_fullsized': '2024-04-25T00:01:35.461449+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-04-25T00:01:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461820+0000', 'last_change': '2024-04-25T00:01:35.461820+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461820+0000', 'last_undegraded': '2024-04-25T00:01:35.461820+0000', 'last_fullsized': '2024-04-25T00:01:35.461820+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.454489+0000', 'last_change': '2024-04-25T00:01:35.454489+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.454489+0000', 'last_undegraded': '2024-04-25T00:01:35.454489+0000', 'last_fullsized': '2024-04-25T00:01:35.454489+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461392+0000', 'last_change': '2024-04-25T00:01:35.461392+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461392+0000', 'last_undegraded': '2024-04-25T00:01:35.461392+0000', 'last_fullsized': '2024-04-25T00:01:35.461392+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.461419+0000', 'last_change': '2024-04-25T00:01:35.461419+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.461419+0000', 'last_undegraded': '2024-04-25T00:01:35.461419+0000', 'last_fullsized': '2024-04-25T00:01:35.461419+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.456965+0000', 'last_change': '2024-04-25T00:01:35.456965+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.456965+0000', 'last_undegraded': '2024-04-25T00:01:35.456965+0000', 'last_fullsized': '2024-04-25T00:01:35.456965+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.464083+0000', 'last_change': '2024-04-25T00:01:35.464083+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.464083+0000', 'last_undegraded': '2024-04-25T00:01:35.464083+0000', 'last_fullsized': '2024-04-25T00:01:35.464083+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:01:40.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.457481+0000', 'last_change': '2024-04-25T00:01:35.457481+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.457481+0000', 'last_undegraded': '2024-04-25T00:01:35.457481+0000', 'last_fullsized': '2024-04-25T00:01:35.457481+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.457950+0000', 'last_change': '2024-04-25T00:01:35.457950+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.457950+0000', 'last_undegraded': '2024-04-25T00:01:35.457950+0000', 'last_fullsized': '2024-04-25T00:01:35.457950+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.458008+0000', 'last_change': '2024-04-25T00:01:35.458008+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.458008+0000', 'last_undegraded': '2024-04-25T00:01:35.458008+0000', 'last_fullsized': '2024-04-25T00:01:35.458008+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.459893+0000', 'last_change': '2024-04-25T00:01:35.459893+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.459893+0000', 'last_undegraded': '2024-04-25T00:01:35.459893+0000', 'last_fullsized': '2024-04-25T00:01:35.459893+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.456599+0000', 'last_change': '2024-04-25T00:01:35.456599+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.456599+0000', 'last_undegraded': '2024-04-25T00:01:35.456599+0000', 'last_fullsized': '2024-04-25T00:01:35.456599+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.456805+0000', 'last_change': '2024-04-25T00:01:35.456805+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.456805+0000', 'last_undegraded': '2024-04-25T00:01:35.456805+0000', 'last_fullsized': '2024-04-25T00:01:35.456805+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.458803+0000', 'last_change': '2024-04-25T00:01:35.458803+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.458803+0000', 'last_undegraded': '2024-04-25T00:01:35.458803+0000', 'last_fullsized': '2024-04-25T00:01:35.458803+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460400+0000', 'last_change': '2024-04-25T00:01:35.460400+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460400+0000', 'last_undegraded': '2024-04-25T00:01:35.460400+0000', 'last_fullsized': '2024-04-25T00:01:35.460400+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.456985+0000', 'last_change': '2024-04-25T00:01:35.456985+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.456985+0000', 'last_undegraded': '2024-04-25T00:01:35.456985+0000', 'last_fullsized': '2024-04-25T00:01:35.456985+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.457477+0000', 'last_change': '2024-04-25T00:01:35.457477+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.457477+0000', 'last_undegraded': '2024-04-25T00:01:35.457477+0000', 'last_fullsized': '2024-04-25T00:01:35.457477+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.454546+0000', 'last_change': '2024-04-25T00:01:35.454546+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.454546+0000', 'last_undegraded': '2024-04-25T00:01:35.454546+0000', 'last_fullsized': '2024-04-25T00:01:35.454546+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.456125+0000', 'last_change': '2024-04-25T00:01:35.456125+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.456125+0000', 'last_undegraded': '2024-04-25T00:01:35.456125+0000', 'last_fullsized': '2024-04-25T00:01:35.456125+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.459032+0000', 'last_change': '2024-04-25T00:01:35.459032+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.459032+0000', 'last_undegraded': '2024-04-25T00:01:35.459032+0000', 'last_fullsized': '2024-04-25T00:01:35.459032+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.458100+0000', 'last_change': '2024-04-25T00:01:35.458100+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.458100+0000', 'last_undegraded': '2024-04-25T00:01:35.458100+0000', 'last_fullsized': '2024-04-25T00:01:35.458100+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.463586+0000', 'last_change': '2024-04-25T00:01:35.463586+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.463586+0000', 'last_undegraded': '2024-04-25T00:01:35.463586+0000', 'last_fullsized': '2024-04-25T00:01:35.463586+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.454529+0000', 'last_change': '2024-04-25T00:01:35.454529+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.454529+0000', 'last_undegraded': '2024-04-25T00:01:35.454529+0000', 'last_fullsized': '2024-04-25T00:01:35.454529+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-04-25T00:01:40.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460509+0000', 'last_change': '2024-04-25T00:01:35.460509+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460509+0000', 'last_undegraded': '2024-04-25T00:01:35.460509+0000', 'last_fullsized': '2024-04-25T00:01:35.460509+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.457579+0000', 'last_change': '2024-04-25T00:01:35.457579+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.457579+0000', 'last_undegraded': '2024-04-25T00:01:35.457579+0000', 'last_fullsized': '2024-04-25T00:01:35.457579+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.463048+0000', 'last_change': '2024-04-25T00:01:35.463048+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.463048+0000', 'last_undegraded': '2024-04-25T00:01:35.463048+0000', 'last_fullsized': '2024-04-25T00:01:35.463048+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460347+0000', 'last_change': '2024-04-25T00:01:35.460347+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460347+0000', 'last_undegraded': '2024-04-25T00:01:35.460347+0000', 'last_fullsized': '2024-04-25T00:01:35.460347+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-04-25T00:01:40.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:01:35.460678+0000', 'last_change': '2024-04-25T00:01:35.460678+0000', 'last_active': '2024-04-25T00:01:35.447078+0000', 'last_peered': '2024-04-25T00:01:35.447078+0000', 'last_clean': '2024-04-25T00:01:35.447078+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:01:35.460678+0000', 'last_undegraded': '2024-04-25T00:01:35.460678+0000', 'last_fullsized': '2024-04-25T00:01:35.460678+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'last_clean_scrub_stamp': '2024-04-25T00:01:35.447078+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:01:40.103 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:01:40.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.222+0000 7f00e7d6a640 1 Processor -- start 2024-04-25T00:01:40.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.223+0000 7f00e7d6a640 1 -- start start 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.223+0000 7f00e7d6a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f00e015e320 0x7f00e00d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.223+0000 7f00e7d6a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00e00d0b00 0x7f00e00d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.223+0000 7f00e7d6a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e00d3de0 con 0x7f00e00d2290 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.223+0000 7f00e7d6a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f00e00d14b0 con 0x7f00e00d0b00 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.223+0000 7f00e7d6a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f00e00d3410 con 0x7f00e015e320 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.223+0000 7f00e6307640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00e00d0b00 0x7f00e00d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e6307640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00e00d0b00 0x7f00e00d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59848/0 (socket says 172.21.15.113:59848) 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e5305640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f00e00d2290 0x7f00e00b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53496/0 (socket says 172.21.15.113:53496) 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e6307640 1 -- 172.21.15.113:0/3430301462 learned_addr learned my addr 172.21.15.113:0/3430301462 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e5b06640 1 --2- 172.21.15.113:0/3430301462 >> v2:172.21.15.178:3300/0 conn(0x7f00e015e320 0x7f00e00d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e6307640 1 -- 172.21.15.113:0/3430301462 >> v2:172.21.15.178:3300/0 conn(0x7f00e015e320 msgr2=0x7f00e00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e6307640 1 --2- 172.21.15.113:0/3430301462 >> v2:172.21.15.178:3300/0 conn(0x7f00e015e320 0x7f00e00d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e6307640 1 -- 172.21.15.113:0/3430301462 >> v1:172.21.15.113:6789/0 conn(0x7f00e00d2290 legacy=0x7f00e00b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e6307640 1 -- 172.21.15.113:0/3430301462 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00d2800 con 0x7f00e00d0b00 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e4b04640 1 -- 172.21.15.113:0/3430301462 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1192268204 0 0) 0x7f00e00d3de0 con 0x7f00e00d2290 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.224+0000 7f00e6307640 1 --2- 172.21.15.113:0/3430301462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e00d0b00 0x7f00e00d0f70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f00e00b6d00 tx=0x7f00d802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bb00a67165d511 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e4b04640 1 -- 172.21.15.113:0/3430301462 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037070 con 0x7f00e00d0b00 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e4b04640 1 -- 172.21.15.113:0/3430301462 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d802ae50 con 0x7f00e00d0b00 2024-04-25T00:01:40.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e4b04640 1 -- 172.21.15.113:0/3430301462 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8032860 con 0x7f00e00d0b00 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 -- 172.21.15.113:0/3430301462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e00d0b00 msgr2=0x7f00e00d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/3430301462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e00d0b00 0x7f00e00d0f70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f00e00b6d00 tx=0x7f00d802a300 comp rx=0 tx=0).stop 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 -- 172.21.15.113:0/3430301462 shutdown_connections 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/3430301462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e00d0b00 0x7f00e00d0f70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/3430301462 >> v2:172.21.15.178:3300/0 conn(0x7f00e015e320 0x7f00e00d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 -- 172.21.15.113:0/3430301462 >> 172.21.15.113:0/3430301462 conn(0x7f00e00b5c20 msgr2=0x7f00e00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 -- 172.21.15.113:0/3430301462 shutdown_connections 2024-04-25T00:01:40.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.225+0000 7f00e7d6a640 1 -- 172.21.15.113:0/3430301462 wait complete. 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.226+0000 7f00e7d6a640 1 Processor -- start 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.226+0000 7f00e7d6a640 1 -- start start 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e7d6a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e00d0b00 0x7f00e015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e7d6a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e00d2290 0x7f00e00ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e7d6a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e015e320 0x7f00e00ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e7d6a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00e00d30a0 con 0x7f00e015e320 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e7d6a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00e00d2dc0 con 0x7f00e00d0b00 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e7d6a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f00e015e700 con 0x7f00e00d2290 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e6307640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e015e320 0x7f00e00ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e6307640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e015e320 0x7f00e00ab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59858/0 (socket says 172.21.15.113:59858) 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e6307640 1 -- 172.21.15.113:0/806940500 learned_addr learned my addr 172.21.15.113:0/806940500 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:40.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e6307640 1 -- 172.21.15.113:0/806940500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e00d2290 msgr2=0x7f00e00ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e6307640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e00d2290 0x7f00e00ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e6307640 1 -- 172.21.15.113:0/806940500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e00d0b00 msgr2=0x7f00e015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.227+0000 7f00e6307640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e00d0b00 0x7f00e015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.228+0000 7f00e6307640 1 -- 172.21.15.113:0/806940500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00abb80 con 0x7f00e015e320 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.228+0000 7f00e6307640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e015e320 0x7f00e00ab640 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f00d80090b0 tx=0x7f00d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.228+0000 7f00d67fc640 1 -- 172.21.15.113:0/806940500 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8041070 con 0x7f00e015e320 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.228+0000 7f00d67fc640 1 -- 172.21.15.113:0/806940500 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d803c890 con 0x7f00e015e320 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.228+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e00abdf0 con 0x7f00e015e320 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.228+0000 7f00d67fc640 1 -- 172.21.15.113:0/806940500 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037040 con 0x7f00e015e320 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.228+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e0160370 con 0x7f00e015e320 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.229+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e00b6c30 con 0x7f00e015e320 2024-04-25T00:01:40.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.229+0000 7f00d67fc640 1 -- 172.21.15.113:0/806940500 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00d8044070 con 0x7f00e015e320 2024-04-25T00:01:40.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.229+0000 7f00d67fc640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f00c406be30 0x7f00c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.230+0000 7f00e5b06640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f00c406be30 0x7f00c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.230+0000 7f00d67fc640 1 -- 172.21.15.113:0/806940500 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f00d80ab900 con 0x7f00e015e320 2024-04-25T00:01:40.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.230+0000 7f00e5b06640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f00c406be30 0x7f00c406e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f00dc007dc0 tx=0x7f00dc0072f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.234+0000 7f00d67fc640 1 -- 172.21.15.113:0/806940500 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00e00b6c30 con 0x7f00e015e320 2024-04-25T00:01:40.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.477+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f00e00691a0 con 0x7f00e015e320 2024-04-25T00:01:40.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.477+0000 7f00d67fc640 1 -- 172.21.15.113:0/806940500 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1583 (secure 0 0 0) 0x7f00d807a440 con 0x7f00e015e320 2024-04-25T00:01:40.477 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":225,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":296263680,"bytes_avail":772797849600,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":31,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":6402}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-25T00:01:21.147056+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:01:40.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f00c406be30 msgr2=0x7f00c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f00c406be30 0x7f00c406e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f00dc007dc0 tx=0x7f00dc0072f0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e015e320 msgr2=0x7f00e00ab640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e015e320 0x7f00e00ab640 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f00d80090b0 tx=0x7f00d80046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 shutdown_connections 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f00c406be30 0x7f00c406e2f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00e015e320 0x7f00e00ab640 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e00d2290 0x7f00e00ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 --2- 172.21.15.113:0/806940500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e00d0b00 0x7f00e015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 >> 172.21.15.113:0/806940500 conn(0x7f00e00b5c20 msgr2=0x7f00e015c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 shutdown_connections 2024-04-25T00:01:40.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.479+0000 7f00e7d6a640 1 -- 172.21.15.113:0/806940500 wait complete. 2024-04-25T00:01:40.491 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:01:40.491 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:01:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.586+0000 7fb01629e640 1 Processor -- start 2024-04-25T00:01:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.586+0000 7fb01629e640 1 -- start start 2024-04-25T00:01:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb01629e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb010144910 0x7fb0100b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb01629e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb0100b56b0 0x7fb0100cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb01629e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb010144cf0 con 0x7fb0100b5240 2024-04-25T00:01:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb01629e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb0100b4510 con 0x7fb010144910 2024-04-25T00:01:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb01629e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb0100b2f30 con 0x7fb0100b56b0 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb00ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb010144910 0x7fb0100b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb00ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb010144910 0x7fb0100b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59864/0 (socket says 172.21.15.113:59864) 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb00ffff640 1 -- 172.21.15.113:0/1365952649 learned_addr learned my addr 172.21.15.113:0/1365952649 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb01483b640 1 --2- 172.21.15.113:0/1365952649 >> v2:172.21.15.178:3300/0 conn(0x7fb0100b56b0 0x7fb0100cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb00ffff640 1 -- 172.21.15.113:0/1365952649 >> v2:172.21.15.178:3300/0 conn(0x7fb0100b56b0 msgr2=0x7fb0100cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb00ffff640 1 --2- 172.21.15.113:0/1365952649 >> v2:172.21.15.178:3300/0 conn(0x7fb0100b56b0 0x7fb0100cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb00ffff640 1 -- 172.21.15.113:0/1365952649 >> v1:172.21.15.178:6789/0 conn(0x7fb0100b5240 legacy=0x7fb01006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.587+0000 7fb00ffff640 1 -- 172.21.15.113:0/1365952649 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0100b3210 con 0x7fb010144910 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.588+0000 7fb00ffff640 1 --2- 172.21.15.113:0/1365952649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb010144910 0x7fb0100b3fd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb000009870 tx=0x7fb00002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65d756d6a1db1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.588+0000 7fb00effd640 1 -- 172.21.15.113:0/1365952649 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000037070 con 0x7fb010144910 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.588+0000 7fb00effd640 1 -- 172.21.15.113:0/1365952649 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb000032440 con 0x7fb010144910 2024-04-25T00:01:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.588+0000 7fb00effd640 1 -- 172.21.15.113:0/1365952649 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000032810 con 0x7fb010144910 2024-04-25T00:01:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 -- 172.21.15.113:0/1365952649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb010144910 msgr2=0x7fb0100b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 --2- 172.21.15.113:0/1365952649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb010144910 0x7fb0100b3fd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb000009870 tx=0x7fb00002a270 comp rx=0 tx=0).stop 2024-04-25T00:01:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 -- 172.21.15.113:0/1365952649 shutdown_connections 2024-04-25T00:01:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 --2- 172.21.15.113:0/1365952649 >> v2:172.21.15.178:3300/0 conn(0x7fb0100b56b0 0x7fb0100cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 --2- 172.21.15.113:0/1365952649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb010144910 0x7fb0100b3fd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 -- 172.21.15.113:0/1365952649 >> 172.21.15.113:0/1365952649 conn(0x7fb0100b1a50 msgr2=0x7fb0100b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 -- 172.21.15.113:0/1365952649 shutdown_connections 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.589+0000 7fb01629e640 1 -- 172.21.15.113:0/1365952649 wait complete. 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.590+0000 7fb01629e640 1 Processor -- start 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.590+0000 7fb01629e640 1 -- start start 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb01629e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0100b5240 0x7fb0100cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb01629e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0100b56b0 0x7fb010140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb01629e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb010144910 0x7fb0100c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb01629e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100b4da0 con 0x7fb0100b56b0 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb01629e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100b37d0 con 0x7fb0100b5240 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb01629e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb0100b4f20 con 0x7fb010144910 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0100b56b0 0x7fb010140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0100b56b0 0x7fb010140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59878/0 (socket says 172.21.15.113:59878) 2024-04-25T00:01:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 -- 172.21.15.113:0/1685394842 learned_addr learned my addr 172.21.15.113:0/1685394842 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00ffff640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0100b5240 0x7fb0100cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 -- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb010144910 msgr2=0x7fb0100c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb01483b640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb010144910 0x7fb0100c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb010144910 0x7fb0100c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 -- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0100b5240 msgr2=0x7fb0100cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0100b5240 0x7fb0100cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 -- 172.21.15.113:0/1685394842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0100c9700 con 0x7fb0100b56b0 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00ffff640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0100b5240 0x7fb0100cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00f7fe640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0100b56b0 0x7fb010140c20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb00400c410 tx=0x7fb00400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.591+0000 7fb00cff9640 1 -- 172.21.15.113:0/1685394842 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004017070 con 0x7fb0100b56b0 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.592+0000 7fb00cff9640 1 -- 172.21.15.113:0/1685394842 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0040035f0 con 0x7fb0100b56b0 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.592+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0100cc350 con 0x7fb0100b56b0 2024-04-25T00:01:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.592+0000 7fb00cff9640 1 -- 172.21.15.113:0/1685394842 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004003a20 con 0x7fb0100b56b0 2024-04-25T00:01:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.592+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0100cc070 con 0x7fb0100b56b0 2024-04-25T00:01:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.593+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0100691a0 con 0x7fb0100b56b0 2024-04-25T00:01:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.593+0000 7fb00cff9640 1 -- 172.21.15.113:0/1685394842 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb00401a070 con 0x7fb0100b56b0 2024-04-25T00:01:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.593+0000 7fb00cff9640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faffc06be30 0x7faffc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.594+0000 7fb00ffff640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faffc06be30 0x7faffc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.594+0000 7fb00cff9640 1 -- 172.21.15.113:0/1685394842 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fb00408a300 con 0x7fb0100b56b0 2024-04-25T00:01:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.594+0000 7fb00ffff640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faffc06be30 0x7faffc06e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb000034000 tx=0x7fb00002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:40.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.598+0000 7fb00cff9640 1 -- 172.21.15.113:0/1685394842 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb004058cb0 con 0x7fb0100b56b0 2024-04-25T00:01:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.765+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0100af160 con 0x7faffc06be30 2024-04-25T00:01:40.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.771+0000 7fb00cff9640 1 -- 172.21.15.113:0/1685394842 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246627 (secure 0 0 0) 0x7fb0100af160 con 0x7faffc06be30 2024-04-25T00:01:40.774 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:40.775 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:01:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faffc06be30 msgr2=0x7faffc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faffc06be30 0x7faffc06e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb000034000 tx=0x7fb00002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0100b56b0 msgr2=0x7fb010140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0100b56b0 0x7fb010140c20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb00400c410 tx=0x7fb00400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 shutdown_connections 2024-04-25T00:01:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faffc06be30 0x7faffc06e2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb010144910 0x7fb0100c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0100b56b0 0x7fb010140c20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 --2- 172.21.15.113:0/1685394842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0100b5240 0x7fb0100cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 >> 172.21.15.113:0/1685394842 conn(0x7fb0100b1a50 msgr2=0x7fb0100b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 shutdown_connections 2024-04-25T00:01:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:40.774+0000 7fb01629e640 1 -- 172.21.15.113:0/1685394842 wait complete. 2024-04-25T00:01:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-25T00:01:39.150358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289320,"kb_used_data":6576,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685400,"statfs":{"total":773094113280,"available":772797849600,"internally_reserved":0,"allocated":6733824,"data_stored":3480370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.703398"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463581+0000","last_change":"2024-04-25T00:01:35.463581+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463581+0000","last_undegraded":"2024-04-25T00:01:35.463581+0000","last_fullsized":"2024-04-25T00:01:35.463581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457980+0000","last_change":"2024-04-25T00:01:35.457980+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457980+0000","last_undegraded":"2024-04-25T00:01:35.457980+0000","last_fullsized":"2024-04-25T00:01:35.457980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460964+0000","last_change":"2024-04-25T00:01:35.460964+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460964+0000","last_undegraded":"2024-04-25T00:01:35.460964+0000","last_fullsized":"2024-04-25T00:01:35.460964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465826+0000","last_change":"2024-04-25T00:01:35.465826+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465826+0000","last_undegraded":"2024-04-25T00:01:35.465826+0000","last_fullsized":"2024-04-25T00:01:35.465826+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454449+0000","last_change":"2024-04-25T00:01:35.454449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454449+0000","last_undegraded":"2024-04-25T00:01:35.454449+0000","last_fullsized":"2024-04-25T00:01:35.454449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461127+0000","last_change":"2024-04-25T00:01:35.461127+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461127+0000","last_undegraded":"2024-04-25T00:01:35.461127+0000","last_fullsized":"2024-04-25T00:01:35.461127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463139+0000","last_change":"2024-04-25T00:01:35.463139+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463139+0000","last_undegraded":"2024-04-25T00:01:35.463139+0000","last_fullsized":"2024-04-25T00:01:35.463139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462009+0000","last_change":"2024-04-25T00:01:35.462009+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462009+0000","last_undegraded":"2024-04-25T00:01:35.462009+0000","last_fullsized":"2024-04-25T00:01:35.462009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464866+0000","last_change":"2024-04-25T00:01:35.464866+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464866+0000","last_undegraded":"2024-04-25T00:01:35.464866+0000","last_fullsized":"2024-04-25T00:01:35.464866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.466538+0000","last_change":"2024-04-25T00:01:35.466538+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.466538+0000","last_undegraded":"2024-04-25T00:01:35.466538+0000","last_fullsized":"2024-04-25T00:01:35.466538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460560+0000","last_change":"2024-04-25T00:01:35.460560+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460560+0000","last_undegraded":"2024-04-25T00:01:35.460560+0000","last_fullsized":"2024-04-25T00:01:35.460560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454445+0000","last_change":"2024-04-25T00:01:35.454445+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454445+0000","last_undegraded":"2024-04-25T00:01:35.454445+0000","last_fullsized":"2024-04-25T00:01:35.454445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460130+0000","last_change":"2024-04-25T00:01:35.460130+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460130+0000","last_undegraded":"2024-04-25T00:01:35.460130+0000","last_fullsized":"2024-04-25T00:01:35.460130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460355+0000","last_change":"2024-04-25T00:01:35.460355+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460355+0000","last_undegraded":"2024-04-25T00:01:35.460355+0000","last_fullsized":"2024-04-25T00:01:35.460355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455420+0000","last_change":"2024-04-25T00:01:35.455420+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455420+0000","last_undegraded":"2024-04-25T00:01:35.455420+0000","last_fullsized":"2024-04-25T00:01:35.455420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461735+0000","last_change":"2024-04-25T00:01:35.461735+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461735+0000","last_undegraded":"2024-04-25T00:01:35.461735+0000","last_fullsized":"2024-04-25T00:01:35.461735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457312+0000","last_change":"2024-04-25T00:01:35.457312+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457312+0000","last_undegraded":"2024-04-25T00:01:35.457312+0000","last_fullsized":"2024-04-25T00:01:35.457312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465794+0000","last_change":"2024-04-25T00:01:35.465794+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465794+0000","last_undegraded":"2024-04-25T00:01:35.465794+0000","last_fullsized":"2024-04-25T00:01:35.465794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461284+0000","last_change":"2024-04-25T00:01:35.461284+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461284+0000","last_undegraded":"2024-04-25T00:01:35.461284+0000","last_fullsized":"2024-04-25T00:01:35.461284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458433+0000","last_change":"2024-04-25T00:01:35.458433+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458433+0000","last_undegraded":"2024-04-25T00:01:35.458433+0000","last_fullsized":"2024-04-25T00:01:35.458433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461883+0000","last_change":"2024-04-25T00:01:35.461883+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461883+0000","last_undegraded":"2024-04-25T00:01:35.461883+0000","last_fullsized":"2024-04-25T00:01:35.461883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457341+0000","last_change":"2024-04-25T00:01:35.457341+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457341+0000","last_undegraded":"2024-04-25T00:01:35.457341+0000","last_fullsized":"2024-04-25T00:01:35.457341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455162+0000","last_change":"2024-04-25T00:01:35.455162+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455162+0000","last_undegraded":"2024-04-25T00:01:35.455162+0000","last_fullsized":"2024-04-25T00:01:35.455162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460808+0000","last_change":"2024-04-25T00:01:35.460808+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460808+0000","last_undegraded":"2024-04-25T00:01:35.460808+0000","last_fullsized":"2024-04-25T00:01:35.460808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459589+0000","last_change":"2024-04-25T00:01:35.459589+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459589+0000","last_undegraded":"2024-04-25T00:01:35.459589+0000","last_fullsized":"2024-04-25T00:01:35.459589+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460726+0000","last_change":"2024-04-25T00:01:35.460726+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460726+0000","last_undegraded":"2024-04-25T00:01:35.460726+0000","last_fullsized":"2024-04-25T00:01:35.460726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462561+0000","last_change":"2024-04-25T00:01:35.462561+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462561+0000","last_undegraded":"2024-04-25T00:01:35.462561+0000","last_fullsized":"2024-04-25T00:01:35.462561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.462201+0000","last_change":"2024-04-25T00:01:35.462201+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.462201+0000","last_undegraded":"2024-04-25T00:01:35.462201+0000","last_fullsized":"2024-04-25T00:01:35.462201+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458578+0000","last_change":"2024-04-25T00:01:35.458578+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458578+0000","last_undegraded":"2024-04-25T00:01:35.458578+0000","last_fullsized":"2024-04-25T00:01:35.458578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460240+0000","last_change":"2024-04-25T00:01:35.460240+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460240+0000","last_undegraded":"2024-04-25T00:01:35.460240+0000","last_fullsized":"2024-04-25T00:01:35.460240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454470+0000","last_change":"2024-04-25T00:01:35.454470+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454470+0000","last_undegraded":"2024-04-25T00:01:35.454470+0000","last_fullsized":"2024-04-25T00:01:35.454470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.455791+0000","last_change":"2024-04-25T00:01:35.455791+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.455791+0000","last_undegraded":"2024-04-25T00:01:35.455791+0000","last_fullsized":"2024-04-25T00:01:35.455791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471527+0000","last_change":"2024-04-25T00:01:35.469190+0000","last_active":"2024-04-25T00:01:35.471527+0000","last_peered":"2024-04-25T00:01:35.471527+0000","last_clean":"2024-04-25T00:01:35.471527+0000","last_became_active":"2024-04-25T00:01:35.468910+0000","last_became_peered":"2024-04-25T00:01:35.468910+0000","last_unstale":"2024-04-25T00:01:35.471527+0000","last_undegraded":"2024-04-25T00:01:35.471527+0000","last_fullsized":"2024-04-25T00:01:35.471527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.183879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.465124+0000","last_change":"2024-04-25T00:01:35.465124+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.465124+0000","last_undegraded":"2024-04-25T00:01:35.465124+0000","last_fullsized":"2024-04-25T00:01:35.465124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466862+0000","last_change":"2024-04-25T00:01:35.465786+0000","last_active":"2024-04-25T00:01:35.466862+0000","last_peered":"2024-04-25T00:01:35.466862+0000","last_clean":"2024-04-25T00:01:35.466862+0000","last_became_active":"2024-04-25T00:01:35.465527+0000","last_became_peered":"2024-04-25T00:01:35.465527+0000","last_unstale":"2024-04-25T00:01:35.466862+0000","last_undegraded":"2024-04-25T00:01:35.466862+0000","last_fullsized":"2024-04-25T00:01:35.466862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:48.237734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454877+0000","last_change":"2024-04-25T00:01:35.454877+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454877+0000","last_undegraded":"2024-04-25T00:01:35.454877+0000","last_fullsized":"2024-04-25T00:01:35.454877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471535+0000","last_change":"2024-04-25T00:01:35.469185+0000","last_active":"2024-04-25T00:01:35.471535+0000","last_peered":"2024-04-25T00:01:35.471535+0000","last_clean":"2024-04-25T00:01:35.471535+0000","last_became_active":"2024-04-25T00:01:35.468886+0000","last_became_peered":"2024-04-25T00:01:35.468886+0000","last_unstale":"2024-04-25T00:01:35.471535+0000","last_undegraded":"2024-04-25T00:01:35.471535+0000","last_fullsized":"2024-04-25T00:01:35.471535+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:16.817987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460218+0000","last_change":"2024-04-25T00:01:35.460218+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460218+0000","last_undegraded":"2024-04-25T00:01:35.460218+0000","last_fullsized":"2024-04-25T00:01:35.460218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469474+0000","last_change":"2024-04-25T00:01:35.467838+0000","last_active":"2024-04-25T00:01:35.469474+0000","last_peered":"2024-04-25T00:01:35.469474+0000","last_clean":"2024-04-25T00:01:35.469474+0000","last_became_active":"2024-04-25T00:01:35.467216+0000","last_became_peered":"2024-04-25T00:01:35.467216+0000","last_unstale":"2024-04-25T00:01:35.469474+0000","last_undegraded":"2024-04-25T00:01:35.469474+0000","last_fullsized":"2024-04-25T00:01:35.469474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461449+0000","last_change":"2024-04-25T00:01:35.461449+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461449+0000","last_undegraded":"2024-04-25T00:01:35.461449+0000","last_fullsized":"2024-04-25T00:01:35.461449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470832+0000","last_change":"2024-04-25T00:01:35.469530+0000","last_active":"2024-04-25T00:01:35.470832+0000","last_peered":"2024-04-25T00:01:35.470832+0000","last_clean":"2024-04-25T00:01:35.470832+0000","last_became_active":"2024-04-25T00:01:35.468953+0000","last_became_peered":"2024-04-25T00:01:35.468953+0000","last_unstale":"2024-04-25T00:01:35.470832+0000","last_undegraded":"2024-04-25T00:01:35.470832+0000","last_fullsized":"2024-04-25T00:01:35.470832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461820+0000","last_change":"2024-04-25T00:01:35.461820+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461820+0000","last_undegraded":"2024-04-25T00:01:35.461820+0000","last_fullsized":"2024-04-25T00:01:35.461820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465278+0000","last_change":"2024-04-25T00:01:35.463716+0000","last_active":"2024-04-25T00:01:35.465278+0000","last_peered":"2024-04-25T00:01:35.465278+0000","last_clean":"2024-04-25T00:01:35.465278+0000","last_became_active":"2024-04-25T00:01:35.463153+0000","last_became_peered":"2024-04-25T00:01:35.463153+0000","last_unstale":"2024-04-25T00:01:35.465278+0000","last_undegraded":"2024-04-25T00:01:35.465278+0000","last_fullsized":"2024-04-25T00:01:35.465278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:44.826461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454489+0000","last_change":"2024-04-25T00:01:35.454489+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454489+0000","last_undegraded":"2024-04-25T00:01:35.454489+0000","last_fullsized":"2024-04-25T00:01:35.454489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.465858+0000","last_change":"2024-04-25T00:01:35.465127+0000","last_active":"2024-04-25T00:01:35.465858+0000","last_peered":"2024-04-25T00:01:35.465858+0000","last_clean":"2024-04-25T00:01:35.465858+0000","last_became_active":"2024-04-25T00:01:35.464804+0000","last_became_peered":"2024-04-25T00:01:35.464804+0000","last_unstale":"2024-04-25T00:01:35.465858+0000","last_undegraded":"2024-04-25T00:01:35.465858+0000","last_fullsized":"2024-04-25T00:01:35.465858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:33.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461392+0000","last_change":"2024-04-25T00:01:35.461392+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461392+0000","last_undegraded":"2024-04-25T00:01:35.461392+0000","last_fullsized":"2024-04-25T00:01:35.461392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471287+0000","last_change":"2024-04-25T00:01:35.470685+0000","last_active":"2024-04-25T00:01:35.471287+0000","last_peered":"2024-04-25T00:01:35.471287+0000","last_clean":"2024-04-25T00:01:35.471287+0000","last_became_active":"2024-04-25T00:01:35.470365+0000","last_became_peered":"2024-04-25T00:01:35.470365+0000","last_unstale":"2024-04-25T00:01:35.471287+0000","last_undegraded":"2024-04-25T00:01:35.471287+0000","last_fullsized":"2024-04-25T00:01:35.471287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.461419+0000","last_change":"2024-04-25T00:01:35.461419+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.461419+0000","last_undegraded":"2024-04-25T00:01:35.461419+0000","last_fullsized":"2024-04-25T00:01:35.461419+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471453+0000","last_change":"2024-04-25T00:01:35.470524+0000","last_active":"2024-04-25T00:01:35.471453+0000","last_peered":"2024-04-25T00:01:35.471453+0000","last_clean":"2024-04-25T00:01:35.471453+0000","last_became_active":"2024-04-25T00:01:35.470224+0000","last_became_peered":"2024-04-25T00:01:35.470224+0000","last_unstale":"2024-04-25T00:01:35.471453+0000","last_undegraded":"2024-04-25T00:01:35.471453+0000","last_fullsized":"2024-04-25T00:01:35.471453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456965+0000","last_change":"2024-04-25T00:01:35.456965+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456965+0000","last_undegraded":"2024-04-25T00:01:35.456965+0000","last_fullsized":"2024-04-25T00:01:35.456965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470019+0000","last_change":"2024-04-25T00:01:35.466689+0000","last_active":"2024-04-25T00:01:35.470019+0000","last_peered":"2024-04-25T00:01:35.470019+0000","last_clean":"2024-04-25T00:01:35.470019+0000","last_became_active":"2024-04-25T00:01:35.466390+0000","last_became_peered":"2024-04-25T00:01:35.466390+0000","last_unstale":"2024-04-25T00:01:35.470019+0000","last_undegraded":"2024-04-25T00:01:35.470019+0000","last_fullsized":"2024-04-25T00:01:35.470019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:21.027526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.464083+0000","last_change":"2024-04-25T00:01:35.464083+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.464083+0000","last_undegraded":"2024-04-25T00:01:35.464083+0000","last_fullsized":"2024-04-25T00:01:35.464083+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470655+0000","last_change":"2024-04-25T00:01:35.469419+0000","last_active":"2024-04-25T00:01:35.470655+0000","last_peered":"2024-04-25T00:01:35.470655+0000","last_clean":"2024-04-25T00:01:35.470655+0000","last_became_active":"2024-04-25T00:01:35.468979+0000","last_became_peered":"2024-04-25T00:01:35.468979+0000","last_unstale":"2024-04-25T00:01:35.470655+0000","last_undegraded":"2024-04-25T00:01:35.470655+0000","last_fullsized":"2024-04-25T00:01:35.470655+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:34.145189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457481+0000","last_change":"2024-04-25T00:01:35.457481+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457481+0000","last_undegraded":"2024-04-25T00:01:35.457481+0000","last_fullsized":"2024-04-25T00:01:35.457481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468042+0000","last_change":"2024-04-25T00:01:35.466590+0000","last_active":"2024-04-25T00:01:35.468042+0000","last_peered":"2024-04-25T00:01:35.468042+0000","last_clean":"2024-04-25T00:01:35.468042+0000","last_became_active":"2024-04-25T00:01:35.466216+0000","last_became_peered":"2024-04-25T00:01:35.466216+0000","last_unstale":"2024-04-25T00:01:35.468042+0000","last_undegraded":"2024-04-25T00:01:35.468042+0000","last_fullsized":"2024-04-25T00:01:35.468042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457950+0000","last_change":"2024-04-25T00:01:35.457950+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457950+0000","last_undegraded":"2024-04-25T00:01:35.457950+0000","last_fullsized":"2024-04-25T00:01:35.457950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470984+0000","last_change":"2024-04-25T00:01:35.469270+0000","last_active":"2024-04-25T00:01:35.470984+0000","last_peered":"2024-04-25T00:01:35.470984+0000","last_clean":"2024-04-25T00:01:35.470984+0000","last_became_active":"2024-04-25T00:01:35.468644+0000","last_became_peered":"2024-04-25T00:01:35.468644+0000","last_unstale":"2024-04-25T00:01:35.470984+0000","last_undegraded":"2024-04-25T00:01:35.470984+0000","last_fullsized":"2024-04-25T00:01:35.470984+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:02.120913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458008+0000","last_change":"2024-04-25T00:01:35.458008+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458008+0000","last_undegraded":"2024-04-25T00:01:35.458008+0000","last_fullsized":"2024-04-25T00:01:35.458008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467827+0000","last_change":"2024-04-25T00:01:35.465772+0000","last_active":"2024-04-25T00:01:35.467827+0000","last_peered":"2024-04-25T00:01:35.467827+0000","last_clean":"2024-04-25T00:01:35.467827+0000","last_became_active":"2024-04-25T00:01:35.465310+0000","last_became_peered":"2024-04-25T00:01:35.465310+0000","last_unstale":"2024-04-25T00:01:35.467827+0000","last_undegraded":"2024-04-25T00:01:35.467827+0000","last_fullsized":"2024-04-25T00:01:35.467827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:34.269657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459893+0000","last_change":"2024-04-25T00:01:35.459893+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459893+0000","last_undegraded":"2024-04-25T00:01:35.459893+0000","last_fullsized":"2024-04-25T00:01:35.459893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468569+0000","last_change":"2024-04-25T00:01:35.467537+0000","last_active":"2024-04-25T00:01:35.468569+0000","last_peered":"2024-04-25T00:01:35.468569+0000","last_clean":"2024-04-25T00:01:35.468569+0000","last_became_active":"2024-04-25T00:01:35.467005+0000","last_became_peered":"2024-04-25T00:01:35.467005+0000","last_unstale":"2024-04-25T00:01:35.468569+0000","last_undegraded":"2024-04-25T00:01:35.468569+0000","last_fullsized":"2024-04-25T00:01:35.468569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:39.054815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456599+0000","last_change":"2024-04-25T00:01:35.456599+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456599+0000","last_undegraded":"2024-04-25T00:01:35.456599+0000","last_fullsized":"2024-04-25T00:01:35.456599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471280+0000","last_change":"2024-04-25T00:01:35.469625+0000","last_active":"2024-04-25T00:01:35.471280+0000","last_peered":"2024-04-25T00:01:35.471280+0000","last_clean":"2024-04-25T00:01:35.471280+0000","last_became_active":"2024-04-25T00:01:35.469302+0000","last_became_peered":"2024-04-25T00:01:35.469302+0000","last_unstale":"2024-04-25T00:01:35.471280+0000","last_undegraded":"2024-04-25T00:01:35.471280+0000","last_fullsized":"2024-04-25T00:01:35.471280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456805+0000","last_change":"2024-04-25T00:01:35.456805+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456805+0000","last_undegraded":"2024-04-25T00:01:35.456805+0000","last_fullsized":"2024-04-25T00:01:35.456805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470624+0000","last_change":"2024-04-25T00:01:35.468884+0000","last_active":"2024-04-25T00:01:35.470624+0000","last_peered":"2024-04-25T00:01:35.470624+0000","last_clean":"2024-04-25T00:01:35.470624+0000","last_became_active":"2024-04-25T00:01:35.468603+0000","last_became_peered":"2024-04-25T00:01:35.468603+0000","last_unstale":"2024-04-25T00:01:35.470624+0000","last_undegraded":"2024-04-25T00:01:35.470624+0000","last_fullsized":"2024-04-25T00:01:35.470624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:39.405188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458803+0000","last_change":"2024-04-25T00:01:35.458803+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458803+0000","last_undegraded":"2024-04-25T00:01:35.458803+0000","last_fullsized":"2024-04-25T00:01:35.458803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.462144+0000","last_change":"2024-04-25T00:01:35.458456+0000","last_active":"2024-04-25T00:01:35.462144+0000","last_peered":"2024-04-25T00:01:35.462144+0000","last_clean":"2024-04-25T00:01:35.462144+0000","last_became_active":"2024-04-25T00:01:35.458035+0000","last_became_peered":"2024-04-25T00:01:35.458035+0000","last_unstale":"2024-04-25T00:01:35.462144+0000","last_undegraded":"2024-04-25T00:01:35.462144+0000","last_fullsized":"2024-04-25T00:01:35.462144+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:28.823118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460400+0000","last_change":"2024-04-25T00:01:35.460400+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460400+0000","last_undegraded":"2024-04-25T00:01:35.460400+0000","last_fullsized":"2024-04-25T00:01:35.460400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467672+0000","last_change":"2024-04-25T00:01:35.466637+0000","last_active":"2024-04-25T00:01:35.467672+0000","last_peered":"2024-04-25T00:01:35.467672+0000","last_clean":"2024-04-25T00:01:35.467672+0000","last_became_active":"2024-04-25T00:01:35.466330+0000","last_became_peered":"2024-04-25T00:01:35.466330+0000","last_unstale":"2024-04-25T00:01:35.467672+0000","last_undegraded":"2024-04-25T00:01:35.467672+0000","last_fullsized":"2024-04-25T00:01:35.467672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:04.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456985+0000","last_change":"2024-04-25T00:01:35.456985+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456985+0000","last_undegraded":"2024-04-25T00:01:35.456985+0000","last_fullsized":"2024-04-25T00:01:35.456985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.463998+0000","last_change":"2024-04-25T00:01:35.460727+0000","last_active":"2024-04-25T00:01:35.463998+0000","last_peered":"2024-04-25T00:01:35.463998+0000","last_clean":"2024-04-25T00:01:35.463998+0000","last_became_active":"2024-04-25T00:01:35.460439+0000","last_became_peered":"2024-04-25T00:01:35.460439+0000","last_unstale":"2024-04-25T00:01:35.463998+0000","last_undegraded":"2024-04-25T00:01:35.463998+0000","last_fullsized":"2024-04-25T00:01:35.463998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457477+0000","last_change":"2024-04-25T00:01:35.457477+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457477+0000","last_undegraded":"2024-04-25T00:01:35.457477+0000","last_fullsized":"2024-04-25T00:01:35.457477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469402+0000","last_change":"2024-04-25T00:01:35.467452+0000","last_active":"2024-04-25T00:01:35.469402+0000","last_peered":"2024-04-25T00:01:35.469402+0000","last_clean":"2024-04-25T00:01:35.469402+0000","last_became_active":"2024-04-25T00:01:35.466644+0000","last_became_peered":"2024-04-25T00:01:35.466644+0000","last_unstale":"2024-04-25T00:01:35.469402+0000","last_undegraded":"2024-04-25T00:01:35.469402+0000","last_fullsized":"2024-04-25T00:01:35.469402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:08.725990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454546+0000","last_change":"2024-04-25T00:01:35.454546+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454546+0000","last_undegraded":"2024-04-25T00:01:35.454546+0000","last_fullsized":"2024-04-25T00:01:35.454546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.464899+0000","last_change":"2024-04-25T00:01:35.462553+0000","last_active":"2024-04-25T00:01:35.464899+0000","last_peered":"2024-04-25T00:01:35.464899+0000","last_clean":"2024-04-25T00:01:35.464899+0000","last_became_active":"2024-04-25T00:01:35.462284+0000","last_became_peered":"2024-04-25T00:01:35.462284+0000","last_unstale":"2024-04-25T00:01:35.464899+0000","last_undegraded":"2024-04-25T00:01:35.464899+0000","last_fullsized":"2024-04-25T00:01:35.464899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:05.412281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.456125+0000","last_change":"2024-04-25T00:01:35.456125+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.456125+0000","last_undegraded":"2024-04-25T00:01:35.456125+0000","last_fullsized":"2024-04-25T00:01:35.456125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471239+0000","last_change":"2024-04-25T00:01:35.470548+0000","last_active":"2024-04-25T00:01:35.471239+0000","last_peered":"2024-04-25T00:01:35.471239+0000","last_clean":"2024-04-25T00:01:35.471239+0000","last_became_active":"2024-04-25T00:01:35.470230+0000","last_became_peered":"2024-04-25T00:01:35.470230+0000","last_unstale":"2024-04-25T00:01:35.471239+0000","last_undegraded":"2024-04-25T00:01:35.471239+0000","last_fullsized":"2024-04-25T00:01:35.471239+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.882339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.459032+0000","last_change":"2024-04-25T00:01:35.459032+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.459032+0000","last_undegraded":"2024-04-25T00:01:35.459032+0000","last_fullsized":"2024-04-25T00:01:35.459032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470245+0000","last_change":"2024-04-25T00:01:35.468183+0000","last_active":"2024-04-25T00:01:35.470245+0000","last_peered":"2024-04-25T00:01:35.470245+0000","last_clean":"2024-04-25T00:01:35.470245+0000","last_became_active":"2024-04-25T00:01:35.467905+0000","last_became_peered":"2024-04-25T00:01:35.467905+0000","last_unstale":"2024-04-25T00:01:35.470245+0000","last_undegraded":"2024-04-25T00:01:35.470245+0000","last_fullsized":"2024-04-25T00:01:35.470245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:50.884149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.458100+0000","last_change":"2024-04-25T00:01:35.458100+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.458100+0000","last_undegraded":"2024-04-25T00:01:35.458100+0000","last_fullsized":"2024-04-25T00:01:35.458100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.471197+0000","last_change":"2024-04-25T00:01:35.468807+0000","last_active":"2024-04-25T00:01:35.471197+0000","last_peered":"2024-04-25T00:01:35.471197+0000","last_clean":"2024-04-25T00:01:35.471197+0000","last_became_active":"2024-04-25T00:01:35.468521+0000","last_became_peered":"2024-04-25T00:01:35.468521+0000","last_unstale":"2024-04-25T00:01:35.471197+0000","last_undegraded":"2024-04-25T00:01:35.471197+0000","last_fullsized":"2024-04-25T00:01:35.471197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:50.112049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463586+0000","last_change":"2024-04-25T00:01:35.463586+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463586+0000","last_undegraded":"2024-04-25T00:01:35.463586+0000","last_fullsized":"2024-04-25T00:01:35.463586+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.466296+0000","last_change":"2024-04-25T00:01:35.463380+0000","last_active":"2024-04-25T00:01:35.466296+0000","last_peered":"2024-04-25T00:01:35.466296+0000","last_clean":"2024-04-25T00:01:35.466296+0000","last_became_active":"2024-04-25T00:01:35.463052+0000","last_became_peered":"2024-04-25T00:01:35.463052+0000","last_unstale":"2024-04-25T00:01:35.466296+0000","last_undegraded":"2024-04-25T00:01:35.466296+0000","last_fullsized":"2024-04-25T00:01:35.466296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:29.474425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.454529+0000","last_change":"2024-04-25T00:01:35.454529+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.454529+0000","last_undegraded":"2024-04-25T00:01:35.454529+0000","last_fullsized":"2024-04-25T00:01:35.454529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.469453+0000","last_change":"2024-04-25T00:01:35.468164+0000","last_active":"2024-04-25T00:01:35.469453+0000","last_peered":"2024-04-25T00:01:35.469453+0000","last_clean":"2024-04-25T00:01:35.469453+0000","last_became_active":"2024-04-25T00:01:35.467823+0000","last_became_peered":"2024-04-25T00:01:35.467823+0000","last_unstale":"2024-04-25T00:01:35.469453+0000","last_undegraded":"2024-04-25T00:01:35.469453+0000","last_fullsized":"2024-04-25T00:01:35.469453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:59.698287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460509+0000","last_change":"2024-04-25T00:01:35.460509+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460509+0000","last_undegraded":"2024-04-25T00:01:35.460509+0000","last_fullsized":"2024-04-25T00:01:35.460509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.467004+0000","last_change":"2024-04-25T00:01:35.465879+0000","last_active":"2024-04-25T00:01:35.467004+0000","last_peered":"2024-04-25T00:01:35.467004+0000","last_clean":"2024-04-25T00:01:35.467004+0000","last_became_active":"2024-04-25T00:01:35.465602+0000","last_became_peered":"2024-04-25T00:01:35.465602+0000","last_unstale":"2024-04-25T00:01:35.467004+0000","last_undegraded":"2024-04-25T00:01:35.467004+0000","last_fullsized":"2024-04-25T00:01:35.467004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:32.955486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.457579+0000","last_change":"2024-04-25T00:01:35.457579+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.457579+0000","last_undegraded":"2024-04-25T00:01:35.457579+0000","last_fullsized":"2024-04-25T00:01:35.457579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468169+0000","last_change":"2024-04-25T00:01:35.467484+0000","last_active":"2024-04-25T00:01:35.468169+0000","last_peered":"2024-04-25T00:01:35.468169+0000","last_clean":"2024-04-25T00:01:35.468169+0000","last_became_active":"2024-04-25T00:01:35.466391+0000","last_became_peered":"2024-04-25T00:01:35.466391+0000","last_unstale":"2024-04-25T00:01:35.468169+0000","last_undegraded":"2024-04-25T00:01:35.468169+0000","last_fullsized":"2024-04-25T00:01:35.468169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:05.916615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.463048+0000","last_change":"2024-04-25T00:01:35.463048+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.463048+0000","last_undegraded":"2024-04-25T00:01:35.463048+0000","last_fullsized":"2024-04-25T00:01:35.463048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470685+0000","last_change":"2024-04-25T00:01:35.469390+0000","last_active":"2024-04-25T00:01:35.470685+0000","last_peered":"2024-04-25T00:01:35.470685+0000","last_clean":"2024-04-25T00:01:35.470685+0000","last_became_active":"2024-04-25T00:01:35.469128+0000","last_became_peered":"2024-04-25T00:01:35.469128+0000","last_unstale":"2024-04-25T00:01:35.470685+0000","last_undegraded":"2024-04-25T00:01:35.470685+0000","last_fullsized":"2024-04-25T00:01:35.470685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.684750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460347+0000","last_change":"2024-04-25T00:01:35.460347+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460347+0000","last_undegraded":"2024-04-25T00:01:35.460347+0000","last_fullsized":"2024-04-25T00:01:35.460347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.470837+0000","last_change":"2024-04-25T00:01:35.468981+0000","last_active":"2024-04-25T00:01:35.470837+0000","last_peered":"2024-04-25T00:01:35.470837+0000","last_clean":"2024-04-25T00:01:35.470837+0000","last_became_active":"2024-04-25T00:01:35.468687+0000","last_became_peered":"2024-04-25T00:01:35.468687+0000","last_unstale":"2024-04-25T00:01:35.470837+0000","last_undegraded":"2024-04-25T00:01:35.470837+0000","last_fullsized":"2024-04-25T00:01:35.470837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:56.015718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-04-25T00:01:35.460678+0000","last_change":"2024-04-25T00:01:35.460678+0000","last_active":"2024-04-25T00:01:35.447078+0000","last_peered":"2024-04-25T00:01:35.447078+0000","last_clean":"2024-04-25T00:01:35.447078+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:01:35.460678+0000","last_undegraded":"2024-04-25T00:01:35.460678+0000","last_fullsized":"2024-04-25T00:01:35.460678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.468470+0000","last_change":"2024-04-25T00:01:35.467166+0000","last_active":"2024-04-25T00:01:35.468470+0000","last_peered":"2024-04-25T00:01:35.468470+0000","last_clean":"2024-04-25T00:01:35.468470+0000","last_became_active":"2024-04-25T00:01:35.466904+0000","last_became_peered":"2024-04-25T00:01:35.466904+0000","last_unstale":"2024-04-25T00:01:35.468470+0000","last_undegraded":"2024-04-25T00:01:35.468470+0000","last_fullsized":"2024-04-25T00:01:35.468470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:43.861535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":239,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T00:01:35.461537+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:01:35.461537+0000","last_peered":"2024-04-25T00:01:35.461537+0000","last_clean":"2024-04-25T00:01:35.461537+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:01:35.461537+0000","last_undegraded":"2024-04-25T00:01:35.461537+0000","last_fullsized":"2024-04-25T00:01:35.461537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44924,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326916,"statfs":{"total":96636764160,"available":96590761984,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":289021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":684032,"data_stored":282754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1290240,"data_stored":879389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:41.732 INFO:tasks.ceph.mds.d.smithi178.stderr:2024-04-25T00:01:41.732+0000 7f7b95489640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:01:43.792 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:01:43.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.900+0000 7f8337cbd640 1 Processor -- start 2024-04-25T00:01:43.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.900+0000 7f8337cbd640 1 -- start start 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8337cbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 0x7f8330070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8337cbd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f83300b3af0 0x7f83300b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8337cbd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83300b2d50 con 0x7f8330072ea0 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8337cbd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8330072a70 con 0x7f83300b3af0 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8337cbd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83300b3010 con 0x7f833015a6b0 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8335258640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8330072ea0 0x7f8330071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53506/0 (socket says 172.21.15.113:53506) 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8335258640 1 -- 172.21.15.113:0/337416633 learned_addr learned my addr 172.21.15.113:0/337416633 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4090581672 0 0) 0x7f83300b2d50 con 0x7f8330072ea0 2024-04-25T00:01:43.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f833625a640 1 --2- 172.21.15.113:0/337416633 >> v2:172.21.15.178:3300/0 conn(0x7f83300b3af0 0x7f83300b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8335a59640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 0x7f8330070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f831c005730 con 0x7f8330072ea0 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4084897784 0 0) 0x7f831c005730 con 0x7f8330072ea0 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 msgr2=0x7f8330070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 0x7f8330070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 >> v2:172.21.15.178:3300/0 conn(0x7f83300b3af0 msgr2=0x7f83300b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 --2- 172.21.15.113:0/337416633 >> v2:172.21.15.178:3300/0 conn(0x7f83300b3af0 0x7f83300b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83300b32f0 con 0x7f8330072ea0 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8320001670 con 0x7f8330072ea0 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.901+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 >> v1:172.21.15.113:6789/0 conn(0x7f8330072ea0 legacy=0x7f8330071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8334a57640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 0x7f831c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8334a57640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f831c006140 0x7f831c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8334a57640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f831c006f50 0x7f831c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8330072a70 con 0x7f831c006f50 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83300b3010 con 0x7f831c006140 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f831c00dbd0 con 0x7f833015a6b0 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f833625a640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f831c006f50 0x7f831c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8335258640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f831c006140 0x7f831c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f8335a59640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 0x7f831c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f833625a640 1 -- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 msgr2=0x7f831c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f833625a640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f833015a6b0 0x7f831c005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f833625a640 1 -- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f831c006140 msgr2=0x7f831c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f833625a640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f831c006140 0x7f831c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.902+0000 7f833625a640 1 -- 172.21.15.113:0/337416633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c005730 con 0x7f831c006f50 2024-04-25T00:01:43.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f833625a640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f831c006f50 0x7f831c0073d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8328009fc0 tx=0x7f831c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81881316131c16c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:43.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328009c10 con 0x7f831c006f50 2024-04-25T00:01:43.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8328036070 con 0x7f831c006f50 2024-04-25T00:01:43.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f8334a57640 1 -- 172.21.15.113:0/337416633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832802d070 con 0x7f831c006f50 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f8337cbd640 1 -- 172.21.15.113:0/337416633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f831c006f50 msgr2=0x7f831c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f8337cbd640 1 --2- 172.21.15.113:0/337416633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f831c006f50 0x7f831c0073d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8328009fc0 tx=0x7f831c007a40 comp rx=0 tx=0).stop 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f833625a640 1 -- 172.21.15.113:0/337416633 reap_dead start 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f8337cbd640 1 -- 172.21.15.113:0/337416633 shutdown_connections 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.903+0000 7f8337cbd640 1 -- 172.21.15.113:0/337416633 >> 172.21.15.113:0/337416633 conn(0x7f83300b1a50 msgr2=0x7f83300b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.904+0000 7f8337cbd640 1 -- 172.21.15.113:0/337416633 shutdown_connections 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.904+0000 7f8337cbd640 1 -- 172.21.15.113:0/337416633 wait complete. 2024-04-25T00:01:43.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 Processor -- start 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 -- start start 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f831c006f50 0x7f8330158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8330152110 0x7f83300b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f833014e360 0x7f833014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f833015aa90 con 0x7f8330152110 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8330072720 con 0x7f833014e360 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.905+0000 7f8337cbd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f833015ac10 con 0x7f831c006f50 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8330152110 0x7f83300b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8330152110 0x7f83300b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59884/0 (socket says 172.21.15.113:59884) 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 -- 172.21.15.113:0/4158221905 learned_addr learned my addr 172.21.15.113:0/4158221905 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f833625a640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f833014e360 0x7f833014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335a59640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f831c006f50 0x7f8330158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 -- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f831c006f50 msgr2=0x7f8330158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f831c006f50 0x7f8330158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 -- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f833014e360 msgr2=0x7f833014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f833014e360 0x7f833014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 -- 172.21.15.113:0/4158221905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83300724a0 con 0x7f8330152110 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f833625a640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f833014e360 0x7f833014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8335258640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8330152110 0x7f83300b4140 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8320007340 tx=0x7f8320007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f83267fc640 1 -- 172.21.15.113:0/4158221905 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832000f8c0 con 0x7f8330152110 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f83267fc640 1 -- 172.21.15.113:0/4158221905 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8320018070 con 0x7f8330152110 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8330158bb0 con 0x7f8330152110 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f83267fc640 1 -- 172.21.15.113:0/4158221905 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832000fcc0 con 0x7f8330152110 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.906+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8330159170 con 0x7f8330152110 2024-04-25T00:01:43.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.907+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83300691a0 con 0x7f8330152110 2024-04-25T00:01:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.908+0000 7f83267fc640 1 -- 172.21.15.113:0/4158221905 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f832001d460 con 0x7f8330152110 2024-04-25T00:01:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.908+0000 7f83267fc640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f831406be60 0x7f831406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.908+0000 7f83267fc640 1 -- 172.21.15.113:0/4158221905 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f832008afc0 con 0x7f8330152110 2024-04-25T00:01:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.908+0000 7f8335a59640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f831406be60 0x7f831406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.909+0000 7f8335a59640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f831406be60 0x7f831406e320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f832c0099e0 tx=0x7f832c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:43.912+0000 7f83267fc640 1 -- 172.21.15.113:0/4158221905 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f832000fe60 con 0x7f8330152110 2024-04-25T00:01:44.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.084+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83300713f0 con 0x7f831406be60 2024-04-25T00:01:44.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.089+0000 7f83267fc640 1 -- 172.21.15.113:0/4158221905 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f83300713f0 con 0x7f831406be60 2024-04-25T00:01:44.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:44.093 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f831406be60 msgr2=0x7f831406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f831406be60 0x7f831406e320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f832c0099e0 tx=0x7f832c008040 comp rx=0 tx=0).stop 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8330152110 msgr2=0x7f83300b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8330152110 0x7f83300b4140 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8320007340 tx=0x7f8320007800 comp rx=0 tx=0).stop 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 shutdown_connections 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f831406be60 0x7f831406e320 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f833014e360 0x7f833014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8330152110 0x7f83300b4140 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 --2- 172.21.15.113:0/4158221905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f831c006f50 0x7f8330158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 >> 172.21.15.113:0/4158221905 conn(0x7f83300b1a50 msgr2=0x7f83300cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:44.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 shutdown_connections 2024-04-25T00:01:44.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.093+0000 7f8337cbd640 1 -- 172.21.15.113:0/4158221905 wait complete. 2024-04-25T00:01:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-04-25T00:01:43.151499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289592,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685128,"statfs":{"total":773094113280,"available":772797571072,"internally_reserved":0,"allocated":7020544,"data_stored":3602324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.698634"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300203+0000","last_change":"2024-04-25T00:01:36.472413+0000","last_active":"2024-04-25T00:01:38.300203+0000","last_peered":"2024-04-25T00:01:38.300203+0000","last_clean":"2024-04-25T00:01:38.300203+0000","last_became_active":"2024-04-25T00:01:36.472183+0000","last_became_peered":"2024-04-25T00:01:36.472183+0000","last_unstale":"2024-04-25T00:01:38.300203+0000","last_undegraded":"2024-04-25T00:01:38.300203+0000","last_fullsized":"2024-04-25T00:01:38.300203+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:37.057733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467304+0000","last_change":"2024-04-25T00:01:36.480779+0000","last_active":"2024-04-25T00:01:37.467304+0000","last_peered":"2024-04-25T00:01:37.467304+0000","last_clean":"2024-04-25T00:01:37.467304+0000","last_became_active":"2024-04-25T00:01:36.480539+0000","last_became_peered":"2024-04-25T00:01:36.480539+0000","last_unstale":"2024-04-25T00:01:37.467304+0000","last_undegraded":"2024-04-25T00:01:37.467304+0000","last_fullsized":"2024-04-25T00:01:37.467304+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:36.358262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.302008+0000","last_change":"2024-04-25T00:01:36.478168+0000","last_active":"2024-04-25T00:01:38.302008+0000","last_peered":"2024-04-25T00:01:38.302008+0000","last_clean":"2024-04-25T00:01:38.302008+0000","last_became_active":"2024-04-25T00:01:36.477897+0000","last_became_peered":"2024-04-25T00:01:36.477897+0000","last_unstale":"2024-04-25T00:01:38.302008+0000","last_undegraded":"2024-04-25T00:01:38.302008+0000","last_fullsized":"2024-04-25T00:01:38.302008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:59.137415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301191+0000","last_change":"2024-04-25T00:01:36.468916+0000","last_active":"2024-04-25T00:01:38.301191+0000","last_peered":"2024-04-25T00:01:38.301191+0000","last_clean":"2024-04-25T00:01:38.301191+0000","last_became_active":"2024-04-25T00:01:36.468454+0000","last_became_peered":"2024-04-25T00:01:36.468454+0000","last_unstale":"2024-04-25T00:01:38.301191+0000","last_undegraded":"2024-04-25T00:01:38.301191+0000","last_fullsized":"2024-04-25T00:01:38.301191+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:56.381452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300338+0000","last_change":"2024-04-25T00:01:36.468840+0000","last_active":"2024-04-25T00:01:38.300338+0000","last_peered":"2024-04-25T00:01:38.300338+0000","last_clean":"2024-04-25T00:01:38.300338+0000","last_became_active":"2024-04-25T00:01:36.468503+0000","last_became_peered":"2024-04-25T00:01:36.468503+0000","last_unstale":"2024-04-25T00:01:38.300338+0000","last_undegraded":"2024-04-25T00:01:38.300338+0000","last_fullsized":"2024-04-25T00:01:38.300338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:09.205206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468463+0000","last_change":"2024-04-25T00:01:36.462228+0000","last_active":"2024-04-25T00:01:37.468463+0000","last_peered":"2024-04-25T00:01:37.468463+0000","last_clean":"2024-04-25T00:01:37.468463+0000","last_became_active":"2024-04-25T00:01:36.461964+0000","last_became_peered":"2024-04-25T00:01:36.461964+0000","last_unstale":"2024-04-25T00:01:37.468463+0000","last_undegraded":"2024-04-25T00:01:37.468463+0000","last_fullsized":"2024-04-25T00:01:37.468463+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:24.906969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467829+0000","last_change":"2024-04-25T00:01:36.470048+0000","last_active":"2024-04-25T00:01:37.467829+0000","last_peered":"2024-04-25T00:01:37.467829+0000","last_clean":"2024-04-25T00:01:37.467829+0000","last_became_active":"2024-04-25T00:01:36.469285+0000","last_became_peered":"2024-04-25T00:01:36.469285+0000","last_unstale":"2024-04-25T00:01:37.467829+0000","last_undegraded":"2024-04-25T00:01:37.467829+0000","last_fullsized":"2024-04-25T00:01:37.467829+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:44.569579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467056+0000","last_change":"2024-04-25T00:01:36.473037+0000","last_active":"2024-04-25T00:01:37.467056+0000","last_peered":"2024-04-25T00:01:37.467056+0000","last_clean":"2024-04-25T00:01:37.467056+0000","last_became_active":"2024-04-25T00:01:36.472788+0000","last_became_peered":"2024-04-25T00:01:36.472788+0000","last_unstale":"2024-04-25T00:01:37.467056+0000","last_undegraded":"2024-04-25T00:01:37.467056+0000","last_fullsized":"2024-04-25T00:01:37.467056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:21.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468138+0000","last_change":"2024-04-25T00:01:36.461686+0000","last_active":"2024-04-25T00:01:37.468138+0000","last_peered":"2024-04-25T00:01:37.468138+0000","last_clean":"2024-04-25T00:01:37.468138+0000","last_became_active":"2024-04-25T00:01:36.461388+0000","last_became_peered":"2024-04-25T00:01:36.461388+0000","last_unstale":"2024-04-25T00:01:37.468138+0000","last_undegraded":"2024-04-25T00:01:37.468138+0000","last_fullsized":"2024-04-25T00:01:37.468138+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:18.327687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300260+0000","last_change":"2024-04-25T00:01:36.474877+0000","last_active":"2024-04-25T00:01:38.300260+0000","last_peered":"2024-04-25T00:01:38.300260+0000","last_clean":"2024-04-25T00:01:38.300260+0000","last_became_active":"2024-04-25T00:01:36.474613+0000","last_became_peered":"2024-04-25T00:01:36.474613+0000","last_unstale":"2024-04-25T00:01:38.300260+0000","last_undegraded":"2024-04-25T00:01:38.300260+0000","last_fullsized":"2024-04-25T00:01:38.300260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:17.822953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300400+0000","last_change":"2024-04-25T00:01:36.475358+0000","last_active":"2024-04-25T00:01:38.300400+0000","last_peered":"2024-04-25T00:01:38.300400+0000","last_clean":"2024-04-25T00:01:38.300400+0000","last_became_active":"2024-04-25T00:01:36.474773+0000","last_became_peered":"2024-04-25T00:01:36.474773+0000","last_unstale":"2024-04-25T00:01:38.300400+0000","last_undegraded":"2024-04-25T00:01:38.300400+0000","last_fullsized":"2024-04-25T00:01:38.300400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:56.491363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301220+0000","last_change":"2024-04-25T00:01:36.476992+0000","last_active":"2024-04-25T00:01:38.301220+0000","last_peered":"2024-04-25T00:01:38.301220+0000","last_clean":"2024-04-25T00:01:38.301220+0000","last_became_active":"2024-04-25T00:01:36.476648+0000","last_became_peered":"2024-04-25T00:01:36.476648+0000","last_unstale":"2024-04-25T00:01:38.301220+0000","last_undegraded":"2024-04-25T00:01:38.301220+0000","last_fullsized":"2024-04-25T00:01:38.301220+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:51.225405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301006+0000","last_change":"2024-04-25T00:01:36.475442+0000","last_active":"2024-04-25T00:01:38.301006+0000","last_peered":"2024-04-25T00:01:38.301006+0000","last_clean":"2024-04-25T00:01:38.301006+0000","last_became_active":"2024-04-25T00:01:36.474839+0000","last_became_peered":"2024-04-25T00:01:36.474839+0000","last_unstale":"2024-04-25T00:01:38.301006+0000","last_undegraded":"2024-04-25T00:01:38.301006+0000","last_fullsized":"2024-04-25T00:01:38.301006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:26.614552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468315+0000","last_change":"2024-04-25T00:01:36.471052+0000","last_active":"2024-04-25T00:01:37.468315+0000","last_peered":"2024-04-25T00:01:37.468315+0000","last_clean":"2024-04-25T00:01:37.468315+0000","last_became_active":"2024-04-25T00:01:36.470732+0000","last_became_peered":"2024-04-25T00:01:36.470732+0000","last_unstale":"2024-04-25T00:01:37.468315+0000","last_undegraded":"2024-04-25T00:01:37.468315+0000","last_fullsized":"2024-04-25T00:01:37.468315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:40.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467836+0000","last_change":"2024-04-25T00:01:36.465952+0000","last_active":"2024-04-25T00:01:37.467836+0000","last_peered":"2024-04-25T00:01:37.467836+0000","last_clean":"2024-04-25T00:01:37.467836+0000","last_became_active":"2024-04-25T00:01:36.465665+0000","last_became_peered":"2024-04-25T00:01:36.465665+0000","last_unstale":"2024-04-25T00:01:37.467836+0000","last_undegraded":"2024-04-25T00:01:37.467836+0000","last_fullsized":"2024-04-25T00:01:37.467836+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.029913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468135+0000","last_change":"2024-04-25T00:01:36.475851+0000","last_active":"2024-04-25T00:01:37.468135+0000","last_peered":"2024-04-25T00:01:37.468135+0000","last_clean":"2024-04-25T00:01:37.468135+0000","last_became_active":"2024-04-25T00:01:36.475317+0000","last_became_peered":"2024-04-25T00:01:36.475317+0000","last_unstale":"2024-04-25T00:01:37.468135+0000","last_undegraded":"2024-04-25T00:01:37.468135+0000","last_fullsized":"2024-04-25T00:01:37.468135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:51.861048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467430+0000","last_change":"2024-04-25T00:01:36.473740+0000","last_active":"2024-04-25T00:01:37.467430+0000","last_peered":"2024-04-25T00:01:37.467430+0000","last_clean":"2024-04-25T00:01:37.467430+0000","last_became_active":"2024-04-25T00:01:36.473475+0000","last_became_peered":"2024-04-25T00:01:36.473475+0000","last_unstale":"2024-04-25T00:01:37.467430+0000","last_undegraded":"2024-04-25T00:01:37.467430+0000","last_fullsized":"2024-04-25T00:01:37.467430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:13.382691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301602+0000","last_change":"2024-04-25T00:01:36.484181+0000","last_active":"2024-04-25T00:01:38.301602+0000","last_peered":"2024-04-25T00:01:38.301602+0000","last_clean":"2024-04-25T00:01:38.301602+0000","last_became_active":"2024-04-25T00:01:36.483920+0000","last_became_peered":"2024-04-25T00:01:36.483920+0000","last_unstale":"2024-04-25T00:01:38.301602+0000","last_undegraded":"2024-04-25T00:01:38.301602+0000","last_fullsized":"2024-04-25T00:01:38.301602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:56.381452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300045+0000","last_change":"2024-04-25T00:01:36.471885+0000","last_active":"2024-04-25T00:01:38.300045+0000","last_peered":"2024-04-25T00:01:38.300045+0000","last_clean":"2024-04-25T00:01:38.300045+0000","last_became_active":"2024-04-25T00:01:36.471614+0000","last_became_peered":"2024-04-25T00:01:36.471614+0000","last_unstale":"2024-04-25T00:01:38.300045+0000","last_undegraded":"2024-04-25T00:01:38.300045+0000","last_fullsized":"2024-04-25T00:01:38.300045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:44.866502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467544+0000","last_change":"2024-04-25T00:01:36.477297+0000","last_active":"2024-04-25T00:01:37.467544+0000","last_peered":"2024-04-25T00:01:37.467544+0000","last_clean":"2024-04-25T00:01:37.467544+0000","last_became_active":"2024-04-25T00:01:36.476918+0000","last_became_peered":"2024-04-25T00:01:36.476918+0000","last_unstale":"2024-04-25T00:01:37.467544+0000","last_undegraded":"2024-04-25T00:01:37.467544+0000","last_fullsized":"2024-04-25T00:01:37.467544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:05.390899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301788+0000","last_change":"2024-04-25T00:01:36.480270+0000","last_active":"2024-04-25T00:01:38.301788+0000","last_peered":"2024-04-25T00:01:38.301788+0000","last_clean":"2024-04-25T00:01:38.301788+0000","last_became_active":"2024-04-25T00:01:36.477032+0000","last_became_peered":"2024-04-25T00:01:36.477032+0000","last_unstale":"2024-04-25T00:01:38.301788+0000","last_undegraded":"2024-04-25T00:01:38.301788+0000","last_fullsized":"2024-04-25T00:01:38.301788+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:38.757347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467175+0000","last_change":"2024-04-25T00:01:36.470356+0000","last_active":"2024-04-25T00:01:37.467175+0000","last_peered":"2024-04-25T00:01:37.467175+0000","last_clean":"2024-04-25T00:01:37.467175+0000","last_became_active":"2024-04-25T00:01:36.470018+0000","last_became_peered":"2024-04-25T00:01:36.470018+0000","last_unstale":"2024-04-25T00:01:37.467175+0000","last_undegraded":"2024-04-25T00:01:37.467175+0000","last_fullsized":"2024-04-25T00:01:37.467175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:14.060045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301756+0000","last_change":"2024-04-25T00:01:36.482844+0000","last_active":"2024-04-25T00:01:38.301756+0000","last_peered":"2024-04-25T00:01:38.301756+0000","last_clean":"2024-04-25T00:01:38.301756+0000","last_became_active":"2024-04-25T00:01:36.482375+0000","last_became_peered":"2024-04-25T00:01:36.482375+0000","last_unstale":"2024-04-25T00:01:38.301756+0000","last_undegraded":"2024-04-25T00:01:38.301756+0000","last_fullsized":"2024-04-25T00:01:38.301756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300727+0000","last_change":"2024-04-25T00:01:36.471898+0000","last_active":"2024-04-25T00:01:38.300727+0000","last_peered":"2024-04-25T00:01:38.300727+0000","last_clean":"2024-04-25T00:01:38.300727+0000","last_became_active":"2024-04-25T00:01:36.471644+0000","last_became_peered":"2024-04-25T00:01:36.471644+0000","last_unstale":"2024-04-25T00:01:38.300727+0000","last_undegraded":"2024-04-25T00:01:38.300727+0000","last_fullsized":"2024-04-25T00:01:38.300727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:17.468234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300346+0000","last_change":"2024-04-25T00:01:36.467391+0000","last_active":"2024-04-25T00:01:38.300346+0000","last_peered":"2024-04-25T00:01:38.300346+0000","last_clean":"2024-04-25T00:01:38.300346+0000","last_became_active":"2024-04-25T00:01:36.467142+0000","last_became_peered":"2024-04-25T00:01:36.467142+0000","last_unstale":"2024-04-25T00:01:38.300346+0000","last_undegraded":"2024-04-25T00:01:38.300346+0000","last_fullsized":"2024-04-25T00:01:38.300346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:06.417247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301512+0000","last_change":"2024-04-25T00:01:36.472621+0000","last_active":"2024-04-25T00:01:38.301512+0000","last_peered":"2024-04-25T00:01:38.301512+0000","last_clean":"2024-04-25T00:01:38.301512+0000","last_became_active":"2024-04-25T00:01:36.472183+0000","last_became_peered":"2024-04-25T00:01:36.472183+0000","last_unstale":"2024-04-25T00:01:38.301512+0000","last_undegraded":"2024-04-25T00:01:38.301512+0000","last_fullsized":"2024-04-25T00:01:38.301512+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:00.967152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468245+0000","last_change":"2024-04-25T00:01:36.475162+0000","last_active":"2024-04-25T00:01:37.468245+0000","last_peered":"2024-04-25T00:01:37.468245+0000","last_clean":"2024-04-25T00:01:37.468245+0000","last_became_active":"2024-04-25T00:01:36.474860+0000","last_became_peered":"2024-04-25T00:01:36.474860+0000","last_unstale":"2024-04-25T00:01:37.468245+0000","last_undegraded":"2024-04-25T00:01:37.468245+0000","last_fullsized":"2024-04-25T00:01:37.468245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:40.410154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300493+0000","last_change":"2024-04-25T00:01:36.470617+0000","last_active":"2024-04-25T00:01:38.300493+0000","last_peered":"2024-04-25T00:01:38.300493+0000","last_clean":"2024-04-25T00:01:38.300493+0000","last_became_active":"2024-04-25T00:01:36.470202+0000","last_became_peered":"2024-04-25T00:01:36.470202+0000","last_unstale":"2024-04-25T00:01:38.300493+0000","last_undegraded":"2024-04-25T00:01:38.300493+0000","last_fullsized":"2024-04-25T00:01:38.300493+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:33.960452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467784+0000","last_change":"2024-04-25T00:01:36.471317+0000","last_active":"2024-04-25T00:01:37.467784+0000","last_peered":"2024-04-25T00:01:37.467784+0000","last_clean":"2024-04-25T00:01:37.467784+0000","last_became_active":"2024-04-25T00:01:36.471054+0000","last_became_peered":"2024-04-25T00:01:36.471054+0000","last_unstale":"2024-04-25T00:01:37.467784+0000","last_undegraded":"2024-04-25T00:01:37.467784+0000","last_fullsized":"2024-04-25T00:01:37.467784+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:15.345349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301449+0000","last_change":"2024-04-25T00:01:36.478636+0000","last_active":"2024-04-25T00:01:38.301449+0000","last_peered":"2024-04-25T00:01:38.301449+0000","last_clean":"2024-04-25T00:01:38.301449+0000","last_became_active":"2024-04-25T00:01:36.477679+0000","last_became_peered":"2024-04-25T00:01:36.477679+0000","last_unstale":"2024-04-25T00:01:38.301449+0000","last_undegraded":"2024-04-25T00:01:38.301449+0000","last_fullsized":"2024-04-25T00:01:38.301449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.242034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300077+0000","last_change":"2024-04-25T00:01:36.464145+0000","last_active":"2024-04-25T00:01:38.300077+0000","last_peered":"2024-04-25T00:01:38.300077+0000","last_clean":"2024-04-25T00:01:38.300077+0000","last_became_active":"2024-04-25T00:01:36.463901+0000","last_became_peered":"2024-04-25T00:01:36.463901+0000","last_unstale":"2024-04-25T00:01:38.300077+0000","last_undegraded":"2024-04-25T00:01:38.300077+0000","last_fullsized":"2024-04-25T00:01:38.300077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:53.773737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468164+0000","last_change":"2024-04-25T00:01:36.469736+0000","last_active":"2024-04-25T00:01:37.468164+0000","last_peered":"2024-04-25T00:01:37.468164+0000","last_clean":"2024-04-25T00:01:37.468164+0000","last_became_active":"2024-04-25T00:01:36.469405+0000","last_became_peered":"2024-04-25T00:01:36.469405+0000","last_unstale":"2024-04-25T00:01:37.468164+0000","last_undegraded":"2024-04-25T00:01:37.468164+0000","last_fullsized":"2024-04-25T00:01:37.468164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:05:22.032492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468840+0000","last_change":"2024-04-25T00:01:36.482140+0000","last_active":"2024-04-25T00:01:37.468840+0000","last_peered":"2024-04-25T00:01:37.468840+0000","last_clean":"2024-04-25T00:01:37.468840+0000","last_became_active":"2024-04-25T00:01:36.481838+0000","last_became_peered":"2024-04-25T00:01:36.481838+0000","last_unstale":"2024-04-25T00:01:37.468840+0000","last_undegraded":"2024-04-25T00:01:37.468840+0000","last_fullsized":"2024-04-25T00:01:37.468840+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:56.381452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468802+0000","last_change":"2024-04-25T00:01:35.469190+0000","last_active":"2024-04-25T00:01:37.468802+0000","last_peered":"2024-04-25T00:01:37.468802+0000","last_clean":"2024-04-25T00:01:37.468802+0000","last_became_active":"2024-04-25T00:01:35.468910+0000","last_became_peered":"2024-04-25T00:01:35.468910+0000","last_unstale":"2024-04-25T00:01:37.468802+0000","last_undegraded":"2024-04-25T00:01:37.468802+0000","last_fullsized":"2024-04-25T00:01:37.468802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:38.427211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300525+0000","last_change":"2024-04-25T00:01:36.474795+0000","last_active":"2024-04-25T00:01:38.300525+0000","last_peered":"2024-04-25T00:01:38.300525+0000","last_clean":"2024-04-25T00:01:38.300525+0000","last_became_active":"2024-04-25T00:01:36.474513+0000","last_became_peered":"2024-04-25T00:01:36.474513+0000","last_unstale":"2024-04-25T00:01:38.300525+0000","last_undegraded":"2024-04-25T00:01:38.300525+0000","last_fullsized":"2024-04-25T00:01:38.300525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:39.432178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467272+0000","last_change":"2024-04-25T00:01:35.465786+0000","last_active":"2024-04-25T00:01:37.467272+0000","last_peered":"2024-04-25T00:01:37.467272+0000","last_clean":"2024-04-25T00:01:37.467272+0000","last_became_active":"2024-04-25T00:01:35.465527+0000","last_became_peered":"2024-04-25T00:01:35.465527+0000","last_unstale":"2024-04-25T00:01:37.467272+0000","last_undegraded":"2024-04-25T00:01:37.467272+0000","last_fullsized":"2024-04-25T00:01:37.467272+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:45.237068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468398+0000","last_change":"2024-04-25T00:01:36.463202+0000","last_active":"2024-04-25T00:01:37.468398+0000","last_peered":"2024-04-25T00:01:37.468398+0000","last_clean":"2024-04-25T00:01:37.468398+0000","last_became_active":"2024-04-25T00:01:36.462901+0000","last_became_peered":"2024-04-25T00:01:36.462901+0000","last_unstale":"2024-04-25T00:01:37.468398+0000","last_undegraded":"2024-04-25T00:01:37.468398+0000","last_fullsized":"2024-04-25T00:01:37.468398+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:03.125879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468659+0000","last_change":"2024-04-25T00:01:35.469185+0000","last_active":"2024-04-25T00:01:37.468659+0000","last_peered":"2024-04-25T00:01:37.468659+0000","last_clean":"2024-04-25T00:01:37.468659+0000","last_became_active":"2024-04-25T00:01:35.468886+0000","last_became_peered":"2024-04-25T00:01:35.468886+0000","last_unstale":"2024-04-25T00:01:37.468659+0000","last_undegraded":"2024-04-25T00:01:37.468659+0000","last_fullsized":"2024-04-25T00:01:37.468659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:32.612355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468389+0000","last_change":"2024-04-25T00:01:36.469391+0000","last_active":"2024-04-25T00:01:37.468389+0000","last_peered":"2024-04-25T00:01:37.468389+0000","last_clean":"2024-04-25T00:01:37.468389+0000","last_became_active":"2024-04-25T00:01:36.469050+0000","last_became_peered":"2024-04-25T00:01:36.469050+0000","last_unstale":"2024-04-25T00:01:37.468389+0000","last_undegraded":"2024-04-25T00:01:37.468389+0000","last_fullsized":"2024-04-25T00:01:37.468389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:57.020684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301347+0000","last_change":"2024-04-25T00:01:35.467838+0000","last_active":"2024-04-25T00:01:38.301347+0000","last_peered":"2024-04-25T00:01:38.301347+0000","last_clean":"2024-04-25T00:01:38.301347+0000","last_became_active":"2024-04-25T00:01:35.467216+0000","last_became_peered":"2024-04-25T00:01:35.467216+0000","last_unstale":"2024-04-25T00:01:38.301347+0000","last_undegraded":"2024-04-25T00:01:38.301347+0000","last_fullsized":"2024-04-25T00:01:38.301347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:39.006389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467345+0000","last_change":"2024-04-25T00:01:36.470600+0000","last_active":"2024-04-25T00:01:37.467345+0000","last_peered":"2024-04-25T00:01:37.467345+0000","last_clean":"2024-04-25T00:01:37.467345+0000","last_became_active":"2024-04-25T00:01:36.469752+0000","last_became_peered":"2024-04-25T00:01:36.469752+0000","last_unstale":"2024-04-25T00:01:37.467345+0000","last_undegraded":"2024-04-25T00:01:37.467345+0000","last_fullsized":"2024-04-25T00:01:37.467345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:18.805486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467866+0000","last_change":"2024-04-25T00:01:35.469530+0000","last_active":"2024-04-25T00:01:37.467866+0000","last_peered":"2024-04-25T00:01:37.467866+0000","last_clean":"2024-04-25T00:01:37.467866+0000","last_became_active":"2024-04-25T00:01:35.468953+0000","last_became_peered":"2024-04-25T00:01:35.468953+0000","last_unstale":"2024-04-25T00:01:37.467866+0000","last_undegraded":"2024-04-25T00:01:37.467866+0000","last_fullsized":"2024-04-25T00:01:37.467866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:38.427211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467571+0000","last_change":"2024-04-25T00:01:36.477620+0000","last_active":"2024-04-25T00:01:37.467571+0000","last_peered":"2024-04-25T00:01:37.467571+0000","last_clean":"2024-04-25T00:01:37.467571+0000","last_became_active":"2024-04-25T00:01:36.477381+0000","last_became_peered":"2024-04-25T00:01:36.477381+0000","last_unstale":"2024-04-25T00:01:37.467571+0000","last_undegraded":"2024-04-25T00:01:37.467571+0000","last_fullsized":"2024-04-25T00:01:37.467571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:47.666569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468546+0000","last_change":"2024-04-25T00:01:35.463716+0000","last_active":"2024-04-25T00:01:37.468546+0000","last_peered":"2024-04-25T00:01:37.468546+0000","last_clean":"2024-04-25T00:01:37.468546+0000","last_became_active":"2024-04-25T00:01:35.463153+0000","last_became_peered":"2024-04-25T00:01:35.463153+0000","last_unstale":"2024-04-25T00:01:37.468546+0000","last_undegraded":"2024-04-25T00:01:37.468546+0000","last_fullsized":"2024-04-25T00:01:37.468546+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:33.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300757+0000","last_change":"2024-04-25T00:01:36.481155+0000","last_active":"2024-04-25T00:01:38.300757+0000","last_peered":"2024-04-25T00:01:38.300757+0000","last_clean":"2024-04-25T00:01:38.300757+0000","last_became_active":"2024-04-25T00:01:36.480867+0000","last_became_peered":"2024-04-25T00:01:36.480867+0000","last_unstale":"2024-04-25T00:01:38.300757+0000","last_undegraded":"2024-04-25T00:01:38.300757+0000","last_fullsized":"2024-04-25T00:01:38.300757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:10.408310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301601+0000","last_change":"2024-04-25T00:01:35.465127+0000","last_active":"2024-04-25T00:01:38.301601+0000","last_peered":"2024-04-25T00:01:38.301601+0000","last_clean":"2024-04-25T00:01:38.301601+0000","last_became_active":"2024-04-25T00:01:35.464804+0000","last_became_peered":"2024-04-25T00:01:35.464804+0000","last_unstale":"2024-04-25T00:01:38.301601+0000","last_undegraded":"2024-04-25T00:01:38.301601+0000","last_fullsized":"2024-04-25T00:01:38.301601+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:40.493015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467740+0000","last_change":"2024-04-25T00:01:36.474116+0000","last_active":"2024-04-25T00:01:37.467740+0000","last_peered":"2024-04-25T00:01:37.467740+0000","last_clean":"2024-04-25T00:01:37.467740+0000","last_became_active":"2024-04-25T00:01:36.473849+0000","last_became_peered":"2024-04-25T00:01:36.473849+0000","last_unstale":"2024-04-25T00:01:37.467740+0000","last_undegraded":"2024-04-25T00:01:37.467740+0000","last_fullsized":"2024-04-25T00:01:37.467740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:26.964485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467794+0000","last_change":"2024-04-25T00:01:35.470685+0000","last_active":"2024-04-25T00:01:37.467794+0000","last_peered":"2024-04-25T00:01:37.467794+0000","last_clean":"2024-04-25T00:01:37.467794+0000","last_became_active":"2024-04-25T00:01:35.470365+0000","last_became_peered":"2024-04-25T00:01:35.470365+0000","last_unstale":"2024-04-25T00:01:37.467794+0000","last_undegraded":"2024-04-25T00:01:37.467794+0000","last_fullsized":"2024-04-25T00:01:37.467794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.183879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468261+0000","last_change":"2024-04-25T00:01:36.470457+0000","last_active":"2024-04-25T00:01:37.468261+0000","last_peered":"2024-04-25T00:01:37.468261+0000","last_clean":"2024-04-25T00:01:37.468261+0000","last_became_active":"2024-04-25T00:01:36.470099+0000","last_became_peered":"2024-04-25T00:01:36.470099+0000","last_unstale":"2024-04-25T00:01:37.468261+0000","last_undegraded":"2024-04-25T00:01:37.468261+0000","last_fullsized":"2024-04-25T00:01:37.468261+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:50.000062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.302145+0000","last_change":"2024-04-25T00:01:35.470524+0000","last_active":"2024-04-25T00:01:38.302145+0000","last_peered":"2024-04-25T00:01:38.302145+0000","last_clean":"2024-04-25T00:01:38.302145+0000","last_became_active":"2024-04-25T00:01:35.470224+0000","last_became_peered":"2024-04-25T00:01:35.470224+0000","last_unstale":"2024-04-25T00:01:38.302145+0000","last_undegraded":"2024-04-25T00:01:38.302145+0000","last_fullsized":"2024-04-25T00:01:38.302145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:18.024947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.302203+0000","last_change":"2024-04-25T00:01:36.481829+0000","last_active":"2024-04-25T00:01:38.302203+0000","last_peered":"2024-04-25T00:01:38.302203+0000","last_clean":"2024-04-25T00:01:38.302203+0000","last_became_active":"2024-04-25T00:01:36.481173+0000","last_became_peered":"2024-04-25T00:01:36.481173+0000","last_unstale":"2024-04-25T00:01:38.302203+0000","last_undegraded":"2024-04-25T00:01:38.302203+0000","last_fullsized":"2024-04-25T00:01:38.302203+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:03.750098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467367+0000","last_change":"2024-04-25T00:01:35.466689+0000","last_active":"2024-04-25T00:01:37.467367+0000","last_peered":"2024-04-25T00:01:37.467367+0000","last_clean":"2024-04-25T00:01:37.467367+0000","last_became_active":"2024-04-25T00:01:35.466390+0000","last_became_peered":"2024-04-25T00:01:35.466390+0000","last_unstale":"2024-04-25T00:01:37.467367+0000","last_undegraded":"2024-04-25T00:01:37.467367+0000","last_fullsized":"2024-04-25T00:01:37.467367+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:50.220439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300098+0000","last_change":"2024-04-25T00:01:36.462105+0000","last_active":"2024-04-25T00:01:38.300098+0000","last_peered":"2024-04-25T00:01:38.300098+0000","last_clean":"2024-04-25T00:01:38.300098+0000","last_became_active":"2024-04-25T00:01:36.461770+0000","last_became_peered":"2024-04-25T00:01:36.461770+0000","last_unstale":"2024-04-25T00:01:38.300098+0000","last_undegraded":"2024-04-25T00:01:38.300098+0000","last_fullsized":"2024-04-25T00:01:38.300098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:51.117015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.302136+0000","last_change":"2024-04-25T00:01:35.469419+0000","last_active":"2024-04-25T00:01:38.302136+0000","last_peered":"2024-04-25T00:01:38.302136+0000","last_clean":"2024-04-25T00:01:38.302136+0000","last_became_active":"2024-04-25T00:01:35.468979+0000","last_became_peered":"2024-04-25T00:01:35.468979+0000","last_unstale":"2024-04-25T00:01:38.302136+0000","last_undegraded":"2024-04-25T00:01:38.302136+0000","last_fullsized":"2024-04-25T00:01:38.302136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:40.493015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301996+0000","last_change":"2024-04-25T00:01:36.480764+0000","last_active":"2024-04-25T00:01:38.301996+0000","last_peered":"2024-04-25T00:01:38.301996+0000","last_clean":"2024-04-25T00:01:38.301996+0000","last_became_active":"2024-04-25T00:01:36.480456+0000","last_became_peered":"2024-04-25T00:01:36.480456+0000","last_unstale":"2024-04-25T00:01:38.301996+0000","last_undegraded":"2024-04-25T00:01:38.301996+0000","last_fullsized":"2024-04-25T00:01:38.301996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:42.742426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468291+0000","last_change":"2024-04-25T00:01:35.466590+0000","last_active":"2024-04-25T00:01:37.468291+0000","last_peered":"2024-04-25T00:01:37.468291+0000","last_clean":"2024-04-25T00:01:37.468291+0000","last_became_active":"2024-04-25T00:01:35.466216+0000","last_became_peered":"2024-04-25T00:01:35.466216+0000","last_unstale":"2024-04-25T00:01:37.468291+0000","last_undegraded":"2024-04-25T00:01:37.468291+0000","last_fullsized":"2024-04-25T00:01:37.468291+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:55.486397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300734+0000","last_change":"2024-04-25T00:01:36.481200+0000","last_active":"2024-04-25T00:01:38.300734+0000","last_peered":"2024-04-25T00:01:38.300734+0000","last_clean":"2024-04-25T00:01:38.300734+0000","last_became_active":"2024-04-25T00:01:36.479943+0000","last_became_peered":"2024-04-25T00:01:36.479943+0000","last_unstale":"2024-04-25T00:01:38.300734+0000","last_undegraded":"2024-04-25T00:01:38.300734+0000","last_fullsized":"2024-04-25T00:01:38.300734+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:40.841056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467648+0000","last_change":"2024-04-25T00:01:35.469270+0000","last_active":"2024-04-25T00:01:37.467648+0000","last_peered":"2024-04-25T00:01:37.467648+0000","last_clean":"2024-04-25T00:01:37.467648+0000","last_became_active":"2024-04-25T00:01:35.468644+0000","last_became_peered":"2024-04-25T00:01:35.468644+0000","last_unstale":"2024-04-25T00:01:37.467648+0000","last_undegraded":"2024-04-25T00:01:37.467648+0000","last_fullsized":"2024-04-25T00:01:37.467648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:55.376486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301859+0000","last_change":"2024-04-25T00:01:36.480408+0000","last_active":"2024-04-25T00:01:38.301859+0000","last_peered":"2024-04-25T00:01:38.301859+0000","last_clean":"2024-04-25T00:01:38.301859+0000","last_became_active":"2024-04-25T00:01:36.480079+0000","last_became_peered":"2024-04-25T00:01:36.480079+0000","last_unstale":"2024-04-25T00:01:38.301859+0000","last_undegraded":"2024-04-25T00:01:38.301859+0000","last_fullsized":"2024-04-25T00:01:38.301859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:00.967152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300787+0000","last_change":"2024-04-25T00:01:35.465772+0000","last_active":"2024-04-25T00:01:38.300787+0000","last_peered":"2024-04-25T00:01:38.300787+0000","last_clean":"2024-04-25T00:01:38.300787+0000","last_became_active":"2024-04-25T00:01:35.465310+0000","last_became_peered":"2024-04-25T00:01:35.465310+0000","last_unstale":"2024-04-25T00:01:38.300787+0000","last_undegraded":"2024-04-25T00:01:38.300787+0000","last_fullsized":"2024-04-25T00:01:38.300787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:52.768771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467908+0000","last_change":"2024-04-25T00:01:36.478144+0000","last_active":"2024-04-25T00:01:37.467908+0000","last_peered":"2024-04-25T00:01:37.467908+0000","last_clean":"2024-04-25T00:01:37.467908+0000","last_became_active":"2024-04-25T00:01:36.477838+0000","last_became_peered":"2024-04-25T00:01:36.477838+0000","last_unstale":"2024-04-25T00:01:37.467908+0000","last_undegraded":"2024-04-25T00:01:37.467908+0000","last_fullsized":"2024-04-25T00:01:37.467908+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:41.497981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301700+0000","last_change":"2024-04-25T00:01:35.467537+0000","last_active":"2024-04-25T00:01:38.301700+0000","last_peered":"2024-04-25T00:01:38.301700+0000","last_clean":"2024-04-25T00:01:38.301700+0000","last_became_active":"2024-04-25T00:01:35.467005+0000","last_became_peered":"2024-04-25T00:01:35.467005+0000","last_unstale":"2024-04-25T00:01:38.301700+0000","last_undegraded":"2024-04-25T00:01:38.301700+0000","last_fullsized":"2024-04-25T00:01:38.301700+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:17.322721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301953+0000","last_change":"2024-04-25T00:01:36.483192+0000","last_active":"2024-04-25T00:01:38.301953+0000","last_peered":"2024-04-25T00:01:38.301953+0000","last_clean":"2024-04-25T00:01:38.301953+0000","last_became_active":"2024-04-25T00:01:36.482909+0000","last_became_peered":"2024-04-25T00:01:36.482909+0000","last_unstale":"2024-04-25T00:01:38.301953+0000","last_undegraded":"2024-04-25T00:01:38.301953+0000","last_fullsized":"2024-04-25T00:01:38.301953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:22.956256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301065+0000","last_change":"2024-04-25T00:01:35.469625+0000","last_active":"2024-04-25T00:01:38.301065+0000","last_peered":"2024-04-25T00:01:38.301065+0000","last_clean":"2024-04-25T00:01:38.301065+0000","last_became_active":"2024-04-25T00:01:35.469302+0000","last_became_peered":"2024-04-25T00:01:35.469302+0000","last_unstale":"2024-04-25T00:01:38.301065+0000","last_undegraded":"2024-04-25T00:01:38.301065+0000","last_fullsized":"2024-04-25T00:01:38.301065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:43.564613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301752+0000","last_change":"2024-04-25T00:01:36.479440+0000","last_active":"2024-04-25T00:01:38.301752+0000","last_peered":"2024-04-25T00:01:38.301752+0000","last_clean":"2024-04-25T00:01:38.301752+0000","last_became_active":"2024-04-25T00:01:36.479194+0000","last_became_peered":"2024-04-25T00:01:36.479194+0000","last_unstale":"2024-04-25T00:01:38.301752+0000","last_undegraded":"2024-04-25T00:01:38.301752+0000","last_fullsized":"2024-04-25T00:01:38.301752+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:51.225405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467390+0000","last_change":"2024-04-25T00:01:35.468884+0000","last_active":"2024-04-25T00:01:37.467390+0000","last_peered":"2024-04-25T00:01:37.467390+0000","last_clean":"2024-04-25T00:01:37.467390+0000","last_became_active":"2024-04-25T00:01:35.468603+0000","last_became_peered":"2024-04-25T00:01:35.468603+0000","last_unstale":"2024-04-25T00:01:37.467390+0000","last_undegraded":"2024-04-25T00:01:37.467390+0000","last_fullsized":"2024-04-25T00:01:37.467390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:23.902002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301284+0000","last_change":"2024-04-25T00:01:36.482207+0000","last_active":"2024-04-25T00:01:38.301284+0000","last_peered":"2024-04-25T00:01:38.301284+0000","last_clean":"2024-04-25T00:01:38.301284+0000","last_became_active":"2024-04-25T00:01:36.480747+0000","last_became_peered":"2024-04-25T00:01:36.480747+0000","last_unstale":"2024-04-25T00:01:38.301284+0000","last_undegraded":"2024-04-25T00:01:38.301284+0000","last_fullsized":"2024-04-25T00:01:38.301284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468938+0000","last_change":"2024-04-25T00:01:35.458456+0000","last_active":"2024-04-25T00:01:37.468938+0000","last_peered":"2024-04-25T00:01:37.468938+0000","last_clean":"2024-04-25T00:01:37.468938+0000","last_became_active":"2024-04-25T00:01:35.458035+0000","last_became_peered":"2024-04-25T00:01:35.458035+0000","last_unstale":"2024-04-25T00:01:37.468938+0000","last_undegraded":"2024-04-25T00:01:37.468938+0000","last_fullsized":"2024-04-25T00:01:37.468938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:04.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467677+0000","last_change":"2024-04-25T00:01:36.477094+0000","last_active":"2024-04-25T00:01:37.467677+0000","last_peered":"2024-04-25T00:01:37.467677+0000","last_clean":"2024-04-25T00:01:37.467677+0000","last_became_active":"2024-04-25T00:01:36.476794+0000","last_became_peered":"2024-04-25T00:01:36.476794+0000","last_unstale":"2024-04-25T00:01:37.467677+0000","last_undegraded":"2024-04-25T00:01:37.467677+0000","last_fullsized":"2024-04-25T00:01:37.467677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:55.116940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301335+0000","last_change":"2024-04-25T00:01:35.466637+0000","last_active":"2024-04-25T00:01:38.301335+0000","last_peered":"2024-04-25T00:01:38.301335+0000","last_clean":"2024-04-25T00:01:38.301335+0000","last_became_active":"2024-04-25T00:01:35.466330+0000","last_became_peered":"2024-04-25T00:01:35.466330+0000","last_unstale":"2024-04-25T00:01:38.301335+0000","last_undegraded":"2024-04-25T00:01:38.301335+0000","last_fullsized":"2024-04-25T00:01:38.301335+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:48.995096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.302190+0000","last_change":"2024-04-25T00:01:36.481508+0000","last_active":"2024-04-25T00:01:38.302190+0000","last_peered":"2024-04-25T00:01:38.302190+0000","last_clean":"2024-04-25T00:01:38.302190+0000","last_became_active":"2024-04-25T00:01:36.481185+0000","last_became_peered":"2024-04-25T00:01:36.481185+0000","last_unstale":"2024-04-25T00:01:38.302190+0000","last_undegraded":"2024-04-25T00:01:38.302190+0000","last_fullsized":"2024-04-25T00:01:38.302190+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:12.966288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468486+0000","last_change":"2024-04-25T00:01:35.460727+0000","last_active":"2024-04-25T00:01:37.468486+0000","last_peered":"2024-04-25T00:01:37.468486+0000","last_clean":"2024-04-25T00:01:37.468486+0000","last_became_active":"2024-04-25T00:01:35.460439+0000","last_became_peered":"2024-04-25T00:01:35.460439+0000","last_unstale":"2024-04-25T00:01:37.468486+0000","last_undegraded":"2024-04-25T00:01:37.468486+0000","last_fullsized":"2024-04-25T00:01:37.468486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:55.486397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300900+0000","last_change":"2024-04-25T00:01:36.481574+0000","last_active":"2024-04-25T00:01:38.300900+0000","last_peered":"2024-04-25T00:01:38.300900+0000","last_clean":"2024-04-25T00:01:38.300900+0000","last_became_active":"2024-04-25T00:01:36.481084+0000","last_became_peered":"2024-04-25T00:01:36.481084+0000","last_unstale":"2024-04-25T00:01:38.300900+0000","last_undegraded":"2024-04-25T00:01:38.300900+0000","last_fullsized":"2024-04-25T00:01:38.300900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.242034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467842+0000","last_change":"2024-04-25T00:01:35.467452+0000","last_active":"2024-04-25T00:01:37.467842+0000","last_peered":"2024-04-25T00:01:37.467842+0000","last_clean":"2024-04-25T00:01:37.467842+0000","last_became_active":"2024-04-25T00:01:35.466644+0000","last_became_peered":"2024-04-25T00:01:35.466644+0000","last_unstale":"2024-04-25T00:01:37.467842+0000","last_undegraded":"2024-04-25T00:01:37.467842+0000","last_fullsized":"2024-04-25T00:01:37.467842+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:39.836090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468647+0000","last_change":"2024-04-25T00:01:36.473007+0000","last_active":"2024-04-25T00:01:37.468647+0000","last_peered":"2024-04-25T00:01:37.468647+0000","last_clean":"2024-04-25T00:01:37.468647+0000","last_became_active":"2024-04-25T00:01:36.471843+0000","last_became_peered":"2024-04-25T00:01:36.471843+0000","last_unstale":"2024-04-25T00:01:37.468647+0000","last_undegraded":"2024-04-25T00:01:37.468647+0000","last_fullsized":"2024-04-25T00:01:37.468647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:40.841056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300127+0000","last_change":"2024-04-25T00:01:35.462553+0000","last_active":"2024-04-25T00:01:38.300127+0000","last_peered":"2024-04-25T00:01:38.300127+0000","last_clean":"2024-04-25T00:01:38.300127+0000","last_became_active":"2024-04-25T00:01:35.462284+0000","last_became_peered":"2024-04-25T00:01:35.462284+0000","last_unstale":"2024-04-25T00:01:38.300127+0000","last_undegraded":"2024-04-25T00:01:38.300127+0000","last_fullsized":"2024-04-25T00:01:38.300127+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:09.403344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468224+0000","last_change":"2024-04-25T00:01:36.480260+0000","last_active":"2024-04-25T00:01:37.468224+0000","last_peered":"2024-04-25T00:01:37.468224+0000","last_clean":"2024-04-25T00:01:37.468224+0000","last_became_active":"2024-04-25T00:01:36.479504+0000","last_became_peered":"2024-04-25T00:01:36.479504+0000","last_unstale":"2024-04-25T00:01:37.468224+0000","last_undegraded":"2024-04-25T00:01:37.468224+0000","last_fullsized":"2024-04-25T00:01:37.468224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:09.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300951+0000","last_change":"2024-04-25T00:01:35.470548+0000","last_active":"2024-04-25T00:01:38.300951+0000","last_peered":"2024-04-25T00:01:38.300951+0000","last_clean":"2024-04-25T00:01:38.300951+0000","last_became_active":"2024-04-25T00:01:35.470230+0000","last_became_peered":"2024-04-25T00:01:35.470230+0000","last_unstale":"2024-04-25T00:01:38.300951+0000","last_undegraded":"2024-04-25T00:01:38.300951+0000","last_fullsized":"2024-04-25T00:01:38.300951+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:18.024947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301673+0000","last_change":"2024-04-25T00:01:36.468790+0000","last_active":"2024-04-25T00:01:38.301673+0000","last_peered":"2024-04-25T00:01:38.301673+0000","last_clean":"2024-04-25T00:01:38.301673+0000","last_became_active":"2024-04-25T00:01:36.468467+0000","last_became_peered":"2024-04-25T00:01:36.468467+0000","last_unstale":"2024-04-25T00:01:38.301673+0000","last_undegraded":"2024-04-25T00:01:38.301673+0000","last_fullsized":"2024-04-25T00:01:38.301673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:22.956256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300501+0000","last_change":"2024-04-25T00:01:35.468183+0000","last_active":"2024-04-25T00:01:38.300501+0000","last_peered":"2024-04-25T00:01:38.300501+0000","last_clean":"2024-04-25T00:01:38.300501+0000","last_became_active":"2024-04-25T00:01:35.467905+0000","last_became_peered":"2024-04-25T00:01:35.467905+0000","last_unstale":"2024-04-25T00:01:38.300501+0000","last_undegraded":"2024-04-25T00:01:38.300501+0000","last_fullsized":"2024-04-25T00:01:38.300501+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:25.609586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301320+0000","last_change":"2024-04-25T00:01:36.479383+0000","last_active":"2024-04-25T00:01:38.301320+0000","last_peered":"2024-04-25T00:01:38.301320+0000","last_clean":"2024-04-25T00:01:38.301320+0000","last_became_active":"2024-04-25T00:01:36.478933+0000","last_became_peered":"2024-04-25T00:01:36.478933+0000","last_unstale":"2024-04-25T00:01:38.301320+0000","last_undegraded":"2024-04-25T00:01:38.301320+0000","last_fullsized":"2024-04-25T00:01:38.301320+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:38.757347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.469065+0000","last_change":"2024-04-25T00:01:35.468807+0000","last_active":"2024-04-25T00:01:37.469065+0000","last_peered":"2024-04-25T00:01:37.469065+0000","last_clean":"2024-04-25T00:01:37.469065+0000","last_became_active":"2024-04-25T00:01:35.468521+0000","last_became_peered":"2024-04-25T00:01:35.468521+0000","last_unstale":"2024-04-25T00:01:37.469065+0000","last_undegraded":"2024-04-25T00:01:37.469065+0000","last_fullsized":"2024-04-25T00:01:37.469065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:39.054815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300373+0000","last_change":"2024-04-25T00:01:36.462679+0000","last_active":"2024-04-25T00:01:38.300373+0000","last_peered":"2024-04-25T00:01:38.300373+0000","last_clean":"2024-04-25T00:01:38.300373+0000","last_became_active":"2024-04-25T00:01:36.462324+0000","last_became_peered":"2024-04-25T00:01:36.462324+0000","last_unstale":"2024-04-25T00:01:38.300373+0000","last_undegraded":"2024-04-25T00:01:38.300373+0000","last_fullsized":"2024-04-25T00:01:38.300373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:45.188845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301964+0000","last_change":"2024-04-25T00:01:35.463380+0000","last_active":"2024-04-25T00:01:38.301964+0000","last_peered":"2024-04-25T00:01:38.301964+0000","last_clean":"2024-04-25T00:01:38.301964+0000","last_became_active":"2024-04-25T00:01:35.463052+0000","last_became_peered":"2024-04-25T00:01:35.463052+0000","last_unstale":"2024-04-25T00:01:38.301964+0000","last_undegraded":"2024-04-25T00:01:38.301964+0000","last_fullsized":"2024-04-25T00:01:38.301964+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:36.052767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468648+0000","last_change":"2024-04-25T00:01:36.469253+0000","last_active":"2024-04-25T00:01:37.468648+0000","last_peered":"2024-04-25T00:01:37.468648+0000","last_clean":"2024-04-25T00:01:37.468648+0000","last_became_active":"2024-04-25T00:01:36.468986+0000","last_became_peered":"2024-04-25T00:01:36.468986+0000","last_unstale":"2024-04-25T00:01:37.468648+0000","last_undegraded":"2024-04-25T00:01:37.468648+0000","last_fullsized":"2024-04-25T00:01:37.468648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.242034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301352+0000","last_change":"2024-04-25T00:01:35.468164+0000","last_active":"2024-04-25T00:01:38.301352+0000","last_peered":"2024-04-25T00:01:38.301352+0000","last_clean":"2024-04-25T00:01:38.301352+0000","last_became_active":"2024-04-25T00:01:35.467823+0000","last_became_peered":"2024-04-25T00:01:35.467823+0000","last_unstale":"2024-04-25T00:01:38.301352+0000","last_undegraded":"2024-04-25T00:01:38.301352+0000","last_fullsized":"2024-04-25T00:01:38.301352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:33.764777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467462+0000","last_change":"2024-04-25T00:01:36.480699+0000","last_active":"2024-04-25T00:01:37.467462+0000","last_peered":"2024-04-25T00:01:37.467462+0000","last_clean":"2024-04-25T00:01:37.467462+0000","last_became_active":"2024-04-25T00:01:36.480436+0000","last_became_peered":"2024-04-25T00:01:36.480436+0000","last_unstale":"2024-04-25T00:01:37.467462+0000","last_undegraded":"2024-04-25T00:01:37.467462+0000","last_fullsized":"2024-04-25T00:01:37.467462+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:50.153864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.300038+0000","last_change":"2024-04-25T00:01:35.465879+0000","last_active":"2024-04-25T00:01:38.300038+0000","last_peered":"2024-04-25T00:01:38.300038+0000","last_clean":"2024-04-25T00:01:38.300038+0000","last_became_active":"2024-04-25T00:01:35.465602+0000","last_became_peered":"2024-04-25T00:01:35.465602+0000","last_unstale":"2024-04-25T00:01:38.300038+0000","last_undegraded":"2024-04-25T00:01:38.300038+0000","last_fullsized":"2024-04-25T00:01:38.300038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:09.430304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.302029+0000","last_change":"2024-04-25T00:01:36.483030+0000","last_active":"2024-04-25T00:01:38.302029+0000","last_peered":"2024-04-25T00:01:38.302029+0000","last_clean":"2024-04-25T00:01:38.302029+0000","last_became_active":"2024-04-25T00:01:36.482120+0000","last_became_peered":"2024-04-25T00:01:36.482120+0000","last_unstale":"2024-04-25T00:01:38.302029+0000","last_undegraded":"2024-04-25T00:01:38.302029+0000","last_fullsized":"2024-04-25T00:01:38.302029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:22.353859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.468262+0000","last_change":"2024-04-25T00:01:35.467484+0000","last_active":"2024-04-25T00:01:37.468262+0000","last_peered":"2024-04-25T00:01:37.468262+0000","last_clean":"2024-04-25T00:01:37.468262+0000","last_became_active":"2024-04-25T00:01:35.466391+0000","last_became_peered":"2024-04-25T00:01:35.466391+0000","last_unstale":"2024-04-25T00:01:37.468262+0000","last_undegraded":"2024-04-25T00:01:37.468262+0000","last_fullsized":"2024-04-25T00:01:37.468262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:32.612355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.299939+0000","last_change":"2024-04-25T00:01:36.473882+0000","last_active":"2024-04-25T00:01:38.299939+0000","last_peered":"2024-04-25T00:01:38.299939+0000","last_clean":"2024-04-25T00:01:38.299939+0000","last_became_active":"2024-04-25T00:01:36.473619+0000","last_became_peered":"2024-04-25T00:01:36.473619+0000","last_unstale":"2024-04-25T00:01:38.299939+0000","last_undegraded":"2024-04-25T00:01:38.299939+0000","last_fullsized":"2024-04-25T00:01:38.299939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:45.831427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.299977+0000","last_change":"2024-04-25T00:01:35.469390+0000","last_active":"2024-04-25T00:01:38.299977+0000","last_peered":"2024-04-25T00:01:38.299977+0000","last_clean":"2024-04-25T00:01:38.299977+0000","last_became_active":"2024-04-25T00:01:35.469128+0000","last_became_peered":"2024-04-25T00:01:35.469128+0000","last_unstale":"2024-04-25T00:01:38.299977+0000","last_undegraded":"2024-04-25T00:01:38.299977+0000","last_fullsized":"2024-04-25T00:01:38.299977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:16.463268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467852+0000","last_change":"2024-04-25T00:01:36.473629+0000","last_active":"2024-04-25T00:01:37.467852+0000","last_peered":"2024-04-25T00:01:37.467852+0000","last_clean":"2024-04-25T00:01:37.467852+0000","last_became_active":"2024-04-25T00:01:36.473373+0000","last_became_peered":"2024-04-25T00:01:36.473373+0000","last_unstale":"2024-04-25T00:01:37.467852+0000","last_undegraded":"2024-04-25T00:01:37.467852+0000","last_fullsized":"2024-04-25T00:01:37.467852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:11.530910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467218+0000","last_change":"2024-04-25T00:01:35.468981+0000","last_active":"2024-04-25T00:01:37.467218+0000","last_peered":"2024-04-25T00:01:37.467218+0000","last_clean":"2024-04-25T00:01:37.467218+0000","last_became_active":"2024-04-25T00:01:35.468687+0000","last_became_peered":"2024-04-25T00:01:35.468687+0000","last_unstale":"2024-04-25T00:01:37.467218+0000","last_undegraded":"2024-04-25T00:01:37.467218+0000","last_fullsized":"2024-04-25T00:01:37.467218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:50.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:37.467140+0000","last_change":"2024-04-25T00:01:36.478025+0000","last_active":"2024-04-25T00:01:37.467140+0000","last_peered":"2024-04-25T00:01:37.467140+0000","last_clean":"2024-04-25T00:01:37.467140+0000","last_became_active":"2024-04-25T00:01:36.477655+0000","last_became_peered":"2024-04-25T00:01:36.477655+0000","last_unstale":"2024-04-25T00:01:37.467140+0000","last_undegraded":"2024-04-25T00:01:37.467140+0000","last_fullsized":"2024-04-25T00:01:37.467140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:35.447078+0000","last_clean_scrub_stamp":"2024-04-25T00:01:35.447078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:14.241229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.302160+0000","last_change":"2024-04-25T00:01:35.467166+0000","last_active":"2024-04-25T00:01:38.302160+0000","last_peered":"2024-04-25T00:01:38.302160+0000","last_clean":"2024-04-25T00:01:38.302160+0000","last_became_active":"2024-04-25T00:01:35.466904+0000","last_became_peered":"2024-04-25T00:01:35.466904+0000","last_unstale":"2024-04-25T00:01:38.302160+0000","last_undegraded":"2024-04-25T00:01:38.302160+0000","last_fullsized":"2024-04-25T00:01:38.302160+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:01:34.442112+0000","last_clean_scrub_stamp":"2024-04-25T00:01:34.442112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:08.200240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":243,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T00:01:38.301175+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:01:38.301175+0000","last_peered":"2024-04-25T00:01:38.301175+0000","last_clean":"2024-04-25T00:01:38.301175+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:01:38.301175+0000","last_undegraded":"2024-04-25T00:01:38.301175+0000","last_fullsized":"2024-04-25T00:01:38.301175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44956,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326884,"statfs":{"total":96636764160,"available":96590729216,"internally_reserved":0,"allocated":1323008,"data_stored":892624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":720896,"data_stored":302096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40212,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331628,"statfs":{"total":96636764160,"available":96595587072,"internally_reserved":0,"allocated":729088,"data_stored":302748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":303708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":729088,"data_stored":302152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":729088,"data_stored":302152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":724992,"data_stored":302186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":894658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:44.108 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:01:44.108 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:01:44.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.247+0000 7fa12c0d2640 1 Processor -- start 2024-04-25T00:01:44.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.248+0000 7fa12c0d2640 1 -- start start 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.248+0000 7fa12c0d2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa12415e2d0 0x7fa12415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12c0d2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1240b3080 0x7fa1240b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12c0d2640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa12415e6b0 0x7fa1240b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12c0d2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa1240cde00 con 0x7fa1240b3080 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12c0d2640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa12415f4f0 con 0x7fa12415e6b0 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12c0d2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa12415f7d0 con 0x7fa12415e2d0 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1240b3080 0x7fa1240b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1240b3080 0x7fa1240b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59894/0 (socket says 172.21.15.113:59894) 2024-04-25T00:01:44.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 -- 172.21.15.113:0/3445724278 learned_addr learned my addr 172.21.15.113:0/3445724278 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12a66f640 1 --2- 172.21.15.113:0/3445724278 >> v2:172.21.15.178:3300/0 conn(0x7fa12415e6b0 0x7fa1240b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa129e6e640 1 --2- 172.21.15.113:0/3445724278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa12415e2d0 0x7fa12415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 -- 172.21.15.113:0/3445724278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa12415e2d0 msgr2=0x7fa12415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 --2- 172.21.15.113:0/3445724278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa12415e2d0 0x7fa12415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 -- 172.21.15.113:0/3445724278 >> v2:172.21.15.178:3300/0 conn(0x7fa12415e6b0 msgr2=0x7fa1240b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 --2- 172.21.15.113:0/3445724278 >> v2:172.21.15.178:3300/0 conn(0x7fa12415e6b0 0x7fa1240b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12966d640 1 -- 172.21.15.113:0/3445724278 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12415fab0 con 0x7fa1240b3080 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa12a66f640 1 --2- 172.21.15.113:0/3445724278 >> v2:172.21.15.178:3300/0 conn(0x7fa12415e6b0 0x7fa1240b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.249+0000 7fa129e6e640 1 --2- 172.21.15.113:0/3445724278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa12415e2d0 0x7fa12415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12966d640 1 --2- 172.21.15.113:0/3445724278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1240b3080 0x7fa1240b4330 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa12406f500 tx=0x7fa11402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7854ac781e156d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa128e6c640 1 -- 172.21.15.113:0/3445724278 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114037070 con 0x7fa1240b3080 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa128e6c640 1 -- 172.21.15.113:0/3445724278 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa114032440 con 0x7fa1240b3080 2024-04-25T00:01:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa128e6c640 1 -- 172.21.15.113:0/3445724278 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114032810 con 0x7fa1240b3080 2024-04-25T00:01:44.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3445724278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1240b3080 msgr2=0x7fa1240b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3445724278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1240b3080 0x7fa1240b4330 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa12406f500 tx=0x7fa11402a140 comp rx=0 tx=0).stop 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3445724278 shutdown_connections 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3445724278 >> v2:172.21.15.178:3300/0 conn(0x7fa12415e6b0 0x7fa1240b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3445724278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1240b3080 0x7fa1240b4330 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3445724278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa12415e2d0 0x7fa12415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.250+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3445724278 >> 172.21.15.113:0/3445724278 conn(0x7fa12406b0f0 msgr2=0x7fa12406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.251+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3445724278 shutdown_connections 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.251+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3445724278 wait complete. 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 Processor -- start 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 -- start start 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1240b3080 0x7fa1241f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa12415e2d0 0x7fa1241f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa12415e6b0 0x7fa124154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa12415f2b0 con 0x7fa12415e6b0 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa12415f130 con 0x7fa12415e2d0 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12c0d2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa124071040 con 0x7fa1240b3080 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12a66f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa12415e6b0 0x7fa124154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12a66f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa12415e6b0 0x7fa124154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59908/0 (socket says 172.21.15.113:59908) 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.252+0000 7fa12a66f640 1 -- 172.21.15.113:0/3192735757 learned_addr learned my addr 172.21.15.113:0/3192735757 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa129e6e640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1240b3080 0x7fa1241f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12966d640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa12415e2d0 0x7fa1241f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12a66f640 1 -- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1240b3080 msgr2=0x7fa1241f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12a66f640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1240b3080 0x7fa1241f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12a66f640 1 -- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa12415e2d0 msgr2=0x7fa1241f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12a66f640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa12415e2d0 0x7fa1241f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12a66f640 1 -- 172.21.15.113:0/3192735757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12414ff40 con 0x7fa12415e6b0 2024-04-25T00:01:44.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa129e6e640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1240b3080 0x7fa1241f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12a66f640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa12415e6b0 0x7fa124154900 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00cd50 tx=0x7fa11c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa11a7fc640 1 -- 172.21.15.113:0/3192735757 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00f680 con 0x7fa12415e6b0 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa11a7fc640 1 -- 172.21.15.113:0/3192735757 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c00f840 con 0x7fa12415e6b0 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa124150220 con 0x7fa12415e6b0 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa11a7fc640 1 -- 172.21.15.113:0/3192735757 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00fc30 con 0x7fa12415e6b0 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.253+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241507e0 con 0x7fa12415e6b0 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.254+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa124150cd0 con 0x7fa12415e6b0 2024-04-25T00:01:44.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.254+0000 7fa11a7fc640 1 -- 172.21.15.113:0/3192735757 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa11c0213f0 con 0x7fa12415e6b0 2024-04-25T00:01:44.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.255+0000 7fa11a7fc640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa10806be30 0x7fa10806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.255+0000 7fa129e6e640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa10806be30 0x7fa10806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.255+0000 7fa11a7fc640 1 -- 172.21.15.113:0/3192735757 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fa11c08bb60 con 0x7fa12415e6b0 2024-04-25T00:01:44.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.255+0000 7fa129e6e640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa10806be30 0x7fa10806e2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:44.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.259+0000 7fa11a7fc640 1 -- 172.21.15.113:0/3192735757 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa11c014030 con 0x7fa12415e6b0 2024-04-25T00:01:44.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.503+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa1240691a0 con 0x7fa12415e6b0 2024-04-25T00:01:44.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.505+0000 7fa11a7fc640 1 -- 172.21.15.113:0/3192735757 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fa11c059f50 con 0x7fa12415e6b0 2024-04-25T00:01:44.505 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:01:44.505 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:01:44.505 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:01:44.505 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:01:44.506 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:01:44.506 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:01:44.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa10806be30 msgr2=0x7fa10806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa10806be30 0x7fa10806e2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).stop 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa12415e6b0 msgr2=0x7fa124154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa12415e6b0 0x7fa124154900 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00cd50 tx=0x7fa11c00a650 comp rx=0 tx=0).stop 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 shutdown_connections 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa10806be30 0x7fa10806e2f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa12415e2d0 0x7fa1241f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa12415e6b0 0x7fa124154900 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 --2- 172.21.15.113:0/3192735757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1240b3080 0x7fa1241f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 >> 172.21.15.113:0/3192735757 conn(0x7fa12406b0f0 msgr2=0x7fa1240cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 shutdown_connections 2024-04-25T00:01:44.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.507+0000 7fa12c0d2640 1 -- 172.21.15.113:0/3192735757 wait complete. 2024-04-25T00:01:44.518 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:44.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.615+0000 7feebcd9e640 1 Processor -- start 2024-04-25T00:01:44.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.615+0000 7feebcd9e640 1 -- start start 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feebcd9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb815e550 0x7feeb80d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feebcd9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feeb80cf3f0 0x7feeb80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feebcd9e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeb815e930 con 0x7feeb80d0a40 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feebcd9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feeb80d3810 con 0x7feeb80cf3f0 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feebcd9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feeb80d13e0 con 0x7feeb815e550 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feeb7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feeb80cf3f0 0x7feeb80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feeb7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feeb80cf3f0 0x7feeb80d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59920/0 (socket says 172.21.15.113:59920) 2024-04-25T00:01:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.616+0000 7feeb7fff640 1 -- 172.21.15.113:0/3514368147 learned_addr learned my addr 172.21.15.113:0/3514368147 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:44.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.617+0000 7feeb77fe640 1 --2- 172.21.15.113:0/3514368147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb815e550 0x7feeb80d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.617+0000 7feeb7fff640 1 -- 172.21.15.113:0/3514368147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb815e550 msgr2=0x7feeb80d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:44.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.617+0000 7feeb7fff640 1 --2- 172.21.15.113:0/3514368147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb815e550 0x7feeb80d32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.617+0000 7feeb7fff640 1 -- 172.21.15.113:0/3514368147 >> v1:172.21.15.178:6789/0 conn(0x7feeb80d0a40 legacy=0x7feeb80cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:44.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.617+0000 7feeb7fff640 1 -- 172.21.15.113:0/3514368147 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb80d2510 con 0x7feeb80cf3f0 2024-04-25T00:01:44.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.617+0000 7feeb7fff640 1 --2- 172.21.15.113:0/3514368147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb80d0ea0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009e50 tx=0x7feeb002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ab0a7c67d08182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:44.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.617+0000 7feeb67fc640 1 -- 172.21.15.113:0/3514368147 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0036070 con 0x7feeb80cf3f0 2024-04-25T00:01:44.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feeb67fc640 1 -- 172.21.15.113:0/3514368147 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb0031600 con 0x7feeb80cf3f0 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feeb67fc640 1 -- 172.21.15.113:0/3514368147 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00319d0 con 0x7feeb80cf3f0 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 -- 172.21.15.113:0/3514368147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 msgr2=0x7feeb80d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 --2- 172.21.15.113:0/3514368147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb80d0ea0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009e50 tx=0x7feeb002a6c0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 -- 172.21.15.113:0/3514368147 shutdown_connections 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 --2- 172.21.15.113:0/3514368147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb80d0ea0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 --2- 172.21.15.113:0/3514368147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb815e550 0x7feeb80d32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 -- 172.21.15.113:0/3514368147 >> 172.21.15.113:0/3514368147 conn(0x7feeb806b2d0 msgr2=0x7feeb806b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 -- 172.21.15.113:0/3514368147 shutdown_connections 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.618+0000 7feebcd9e640 1 -- 172.21.15.113:0/3514368147 wait complete. 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.619+0000 7feebcd9e640 1 Processor -- start 2024-04-25T00:01:44.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feebcd9e640 1 -- start start 2024-04-25T00:01:44.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feebcd9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb81f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feebcd9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb80d0a40 0x7feeb80cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feebcd9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb815e550 0x7feeb80b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feebcd9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80d2db0 con 0x7feeb80cf3f0 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feebcd9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80d2ad0 con 0x7feeb815e550 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feebcd9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feeb80cfa10 con 0x7feeb80d0a40 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feeb77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb81f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feeb77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb81f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59928/0 (socket says 172.21.15.113:59928) 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.620+0000 7feeb77fe640 1 -- 172.21.15.113:0/2154947458 learned_addr learned my addr 172.21.15.113:0/2154947458 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb7fff640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb815e550 0x7feeb80b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb6ffd640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb80d0a40 0x7feeb80cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb77fe640 1 -- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb80d0a40 msgr2=0x7feeb80cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb77fe640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb80d0a40 0x7feeb80cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb77fe640 1 -- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb815e550 msgr2=0x7feeb80b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb77fe640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb815e550 0x7feeb80b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb77fe640 1 -- 172.21.15.113:0/2154947458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb81f4560 con 0x7feeb80cf3f0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feeb77fe640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb81f4020 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feea800d5f0 tx=0x7feea800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7fee9ffff640 1 -- 172.21.15.113:0/2154947458 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea80125f0 con 0x7feeb80cf3f0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7fee9ffff640 1 -- 172.21.15.113:0/2154947458 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea8019070 con 0x7feeb80cf3f0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7fee9ffff640 1 -- 172.21.15.113:0/2154947458 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea80129d0 con 0x7feeb80cf3f0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb80ccac0 con 0x7feeb80cf3f0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.621+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb80cd080 con 0x7feeb80cf3f0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.622+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb80691a0 con 0x7feeb80cf3f0 2024-04-25T00:01:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.622+0000 7fee9ffff640 1 -- 172.21.15.113:0/2154947458 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feea8012b70 con 0x7feeb80cf3f0 2024-04-25T00:01:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.623+0000 7fee9ffff640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.623+0000 7fee9ffff640 1 -- 172.21.15.113:0/2154947458 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7feea808b270 con 0x7feeb80cf3f0 2024-04-25T00:01:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.623+0000 7feeb6ffd640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.623+0000 7feeb6ffd640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feeb8161cb0 tx=0x7feeac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:44.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.627+0000 7fee9ffff640 1 -- 172.21.15.113:0/2154947458 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feea8059c20 con 0x7feeb80cf3f0 2024-04-25T00:01:44.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.835+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feeb80d2250 con 0x7feeb80cf3f0 2024-04-25T00:01:44.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.836+0000 7fee9ffff640 1 -- 172.21.15.113:0/2154947458 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5193 (secure 0 0 0) 0x7feea8059520 con 0x7feeb80cf3f0 2024-04-25T00:01:44.836 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3783503244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3783503244},{"type":"v1","addr":"172.21.15.178:6835","nonce":3783503244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3613161476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3613161476},{"type":"v1","addr":"172.21.15.113:6835","nonce":3613161476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2692423657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2692423657},{"type":"v1","addr":"172.21.15.113:6837","nonce":2692423657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:37.799020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/3704852657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3704852657},{"type":"v1","addr":"172.21.15.178:6834","nonce":3704852657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":4}]} 2024-04-25T00:01:44.836 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.837+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 msgr2=0x7feea406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.837+0000 7feebcd9e640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feeb8161cb0 tx=0x7feeac009040 comp rx=0 tx=0).stop 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.837+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 msgr2=0x7feeb81f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.837+0000 7feebcd9e640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb81f4020 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feea800d5f0 tx=0x7feea800dab0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 shutdown_connections 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb815e550 0x7feeb80b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeb80d0a40 0x7feeb80cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 --2- 172.21.15.113:0/2154947458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeb80cf3f0 0x7feeb81f4020 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:44.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 >> 172.21.15.113:0/2154947458 conn(0x7feeb806b2d0 msgr2=0x7feeb815a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:44.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 shutdown_connections 2024-04-25T00:01:44.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:44.838+0000 7feebcd9e640 1 -- 172.21.15.113:0/2154947458 wait complete. 2024-04-25T00:01:45.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:01:35.811705+0000', 'modified': '2024-04-25T00:01:37.799020+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6402': {'gid': 6402, 'name': 'd', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6834/3704852657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 3704852657}, {'type': 'v1', 'addr': '172.21.15.178:6834', 'nonce': 3704852657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6402, 'qdb_cluster': [6402]} 2024-04-25T00:01:45.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:01:45.252 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T00:01:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.350+0000 7f812aa41640 1 Processor -- start 2024-04-25T00:01:45.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.350+0000 7f812aa41640 1 -- start start 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f812aa41640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f812415ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f812aa41640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f81241604d0 0x7f81240cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f812aa41640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f812415d6b0 con 0x7f812413f050 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f812aa41640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f812415c6a0 con 0x7f81241604d0 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f812aa41640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f812415c8f0 con 0x7f81240d06c0 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f8123fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f812413f050 0x7f81240cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53520/0 (socket says 172.21.15.113:53520) 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f8123fff640 1 -- 172.21.15.113:0/2633512536 learned_addr learned my addr 172.21.15.113:0/2633512536 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f8128fde640 1 --2- 172.21.15.113:0/2633512536 >> v2:172.21.15.178:3300/0 conn(0x7f81241604d0 0x7f81240cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f81237fe640 1 --2- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f812415ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f8122ffd640 1 -- 172.21.15.113:0/2633512536 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2216126308 0 0) 0x7f812415d6b0 con 0x7f812413f050 2024-04-25T00:01:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.351+0000 7f8122ffd640 1 -- 172.21.15.113:0/2633512536 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f810c003520 con 0x7f812413f050 2024-04-25T00:01:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.352+0000 7f8128fde640 1 -- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 msgr2=0x7f812415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.352+0000 7f8128fde640 1 --2- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f812415ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.352+0000 7f8128fde640 1 -- 172.21.15.113:0/2633512536 >> v1:172.21.15.113:6789/0 conn(0x7f812413f050 legacy=0x7f81240cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.352+0000 7f8122ffd640 1 -- 172.21.15.113:0/2633512536 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1772885392 0 0) 0x7f810c003520 con 0x7f812413f050 2024-04-25T00:01:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.352+0000 7f8128fde640 1 -- 172.21.15.113:0/2633512536 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812415cbd0 con 0x7f81241604d0 2024-04-25T00:01:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.352+0000 7f81237fe640 1 --2- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f812415ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.352+0000 7f8128fde640 1 --2- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 0x7f81240cea60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f811c009d90 tx=0x7f811c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3acfdb4d5fe21b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f8122ffd640 1 -- 172.21.15.113:0/2633512536 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c031600 con 0x7f81241604d0 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f8122ffd640 1 -- 172.21.15.113:0/2633512536 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c0317a0 con 0x7f81241604d0 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f8122ffd640 1 -- 172.21.15.113:0/2633512536 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c031b50 con 0x7f81241604d0 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 -- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 msgr2=0x7f81240cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 --2- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 0x7f81240cea60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f811c009d90 tx=0x7f811c02a5d0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 -- 172.21.15.113:0/2633512536 shutdown_connections 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 --2- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 0x7f81240cea60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 --2- 172.21.15.113:0/2633512536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f812415ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 -- 172.21.15.113:0/2633512536 >> 172.21.15.113:0/2633512536 conn(0x7f812406b0f0 msgr2=0x7f812406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 -- 172.21.15.113:0/2633512536 shutdown_connections 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.353+0000 7f812aa41640 1 -- 172.21.15.113:0/2633512536 wait complete. 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.354+0000 7f812aa41640 1 Processor -- start 2024-04-25T00:01:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.354+0000 7f812aa41640 1 -- start start 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f812aa41640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f81241f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f812aa41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f812413f050 0x7f812413b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f812aa41640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 0x7f8124150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f812aa41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f812413f430 con 0x7f812413f050 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f812aa41640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f812415d190 con 0x7f81241604d0 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f812aa41640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81241608d0 con 0x7f81240d06c0 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f81237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f812413f050 0x7f812413b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f81237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f812413f050 0x7f812413b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59930/0 (socket says 172.21.15.113:59930) 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f81237fe640 1 -- 172.21.15.113:0/4559269 learned_addr learned my addr 172.21.15.113:0/4559269 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f8128fde640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 0x7f8124150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.355+0000 7f8123fff640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f81241f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f81237fe640 1 -- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 msgr2=0x7f81241f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f81237fe640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f81241f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f81237fe640 1 -- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 msgr2=0x7f8124150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f81237fe640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 0x7f8124150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f81237fe640 1 -- 172.21.15.113:0/4559269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812414cdd0 con 0x7f812413f050 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f8123fff640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f81241f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f81237fe640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f812413f050 0x7f812413b9f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f811800e930 tx=0x7f811800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f8120ff9640 1 -- 172.21.15.113:0/4559269 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118017070 con 0x7f812413f050 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f8120ff9640 1 -- 172.21.15.113:0/4559269 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811800ca00 con 0x7f812413f050 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812414d0b0 con 0x7f812413f050 2024-04-25T00:01:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f8120ff9640 1 -- 172.21.15.113:0/4559269 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800cdf0 con 0x7f812413f050 2024-04-25T00:01:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.356+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812414d670 con 0x7f812413f050 2024-04-25T00:01:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.357+0000 7f8120ff9640 1 -- 172.21.15.113:0/4559269 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f811801a070 con 0x7f812413f050 2024-04-25T00:01:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.358+0000 7f8120ff9640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811006be30 0x7f811006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.358+0000 7f8120ff9640 1 -- 172.21.15.113:0/4559269 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f811808acb0 con 0x7f812413f050 2024-04-25T00:01:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.358+0000 7f8123fff640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811006be30 0x7f811006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.358+0000 7f8123fff640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811006be30 0x7f811006e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8114001700 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 Processor -- start 2024-04-25T00:01:45.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 -- start start 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafa84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aafd8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8540 0x5631aafad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5631aaf10f00 con 0x5631aafd8540 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631aaeb9750 con 0x5631aafd8c80 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5631aaf0cec0 con 0x5631aaff5b60 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f8128fde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8540 0x5631aafad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.359+0000 7f8128fde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8540 0x5631aafad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59946/0 (socket says 172.21.15.113:59946) 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8128fde640 1 -- 172.21.15.113:0/2546452975 learned_addr learned my addr 172.21.15.113:0/2546452975 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8123fff640 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafa84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f81237fe640 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aafd8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8128fde640 1 -- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 msgr2=0x5631aafa84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8128fde640 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafa84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8128fde640 1 -- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 msgr2=0x5631aafd8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8128fde640 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aafd8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8128fde640 1 -- 172.21.15.113:0/2546452975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631aac252d0 con 0x5631aafd8540 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8128fde640 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8540 0x5631aafad350 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f811c02abc0 tx=0x7f811c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb76476336bb5abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8109ffb640 1 -- 172.21.15.113:0/2546452975 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c031780 con 0x5631aafd8540 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8109ffb640 1 -- 172.21.15.113:0/2546452975 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c041070 con 0x5631aafd8540 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.360+0000 7f8109ffb640 1 -- 172.21.15.113:0/2546452975 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c031b40 con 0x5631aafd8540 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 -- 172.21.15.113:0/2546452975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8540 msgr2=0x5631aafad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8540 0x5631aafad350 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f811c02abc0 tx=0x7f811c02e000 comp rx=0 tx=0).stop 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 -- 172.21.15.113:0/2546452975 shutdown_connections 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aafd8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8540 0x5631aafad350 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 --2- 172.21.15.113:0/2546452975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafa84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 -- 172.21.15.113:0/2546452975 >> 172.21.15.113:0/2546452975 conn(0x5631aafaa760 msgr2=0x5631aaff3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 -- 172.21.15.113:0/2546452975 shutdown_connections 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 -- 172.21.15.113:0/2546452975 wait complete. 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 -- start start 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.361+0000 7f812d820740 1 Processor -- start 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafdb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f812d820740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafdbd70 0x5631aaff2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f812d820740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5631aafac910 con 0x5631aafdbd70 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631aaecccf0 con 0x5631aafd8c80 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5631aafabd00 con 0x5631aaff5b60 2024-04-25T00:01:45.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f812d820740 10 client.0 _ll_register_callbacks cb 0x5631aaecc6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f8128fde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafdbd70 0x5631aaff2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f8128fde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafdbd70 0x5631aaff2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59952/0 (socket says 172.21.15.113:59952) 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f8128fde640 1 -- 172.21.15.113:0/779702435 learned_addr learned my addr 172.21.15.113:0/779702435 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f8123fff640 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.362+0000 7f81237fe640 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafdb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8128fde640 1 -- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 msgr2=0x5631aafdb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8128fde640 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafdb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8128fde640 1 -- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 msgr2=0x5631aaff8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8128fde640 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8128fde640 1 -- 172.21.15.113:0/779702435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631aacef6b0 con 0x5631aafdbd70 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8128fde640 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafdbd70 0x5631aaff2fb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f811c009e80 tx=0x7f811c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aed56ff1c2593e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8102ffd640 1 -- 172.21.15.113:0/779702435 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c004310 con 0x5631aafdbd70 2024-04-25T00:01:45.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8102ffd640 1 -- 172.21.15.113:0/779702435 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f811c0044b0 con 0x5631aafdbd70 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f811c041070 con 0x5631aafdbd70 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8102ffd640 1 -- 172.21.15.113:0/779702435 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f811c0046b0 con 0x5631aafdbd70 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 10 client.6573 fetch_fsmap learned FSMap version 31 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x5631aaff8450 con 0x5631aafdbd70 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8102ffd640 1 -- 172.21.15.113:0/779702435 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c041070 con 0x5631aafdbd70 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f8102ffd640 1 -- 172.21.15.113:0/779702435 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f811c031580 con 0x5631aafdbd70 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 10 client.6573 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 10 client.6573 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 20 client.6573 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631aaff34f0 0x5631ab006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 4 client.6573 mds_command: new command op to 6402 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.363+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 --> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631aaf1e440 con 0x5631aaff34f0 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.364+0000 7f8123fff640 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631aaff34f0 0x5631ab006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.364+0000 7f8102ffd640 10 client.6573 ms_handle_connect on v2:172.21.15.178:6832/3704852657 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.364+0000 7f8123fff640 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631aaff34f0 0x5631ab006cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bee3fd77d8b1fb0 server_cookie=63596acd7981b60e in_seq=0 out_seq=0 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f8102ffd640 1 -- 172.21.15.113:0/779702435 <== mds.0 v2:172.21.15.178:6832/3704852657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631aaf1e440 con 0x5631aaff34f0 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f8102ffd640 10 client.6573 handle_command_reply: tid=0 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 client.6573 shutdown 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafdbd70 msgr2=0x5631aaff2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafdbd70 0x5631aaff2fb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f811c009e80 tx=0x7f811c004ab0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 shutdown_connections 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631aaff34f0 0x5631ab006cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafdbd70 0x5631aaff2fb0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f8102ffd640 0 client.6573 ms_handle_reset on v2:172.21.15.178:6832/3704852657 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631aafdb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 --2- 172.21.15.113:0/779702435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 >> 172.21.15.113:0/779702435 conn(0x5631aafd8540 msgr2=0x5631aafa9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 shutdown_connections 2024-04-25T00:01:45.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 1 -- 172.21.15.113:0/779702435 wait complete. 2024-04-25T00:01:45.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.365+0000 7f812d820740 20 client.6573 trim_cache size 0 max 16384 2024-04-25T00:01:45.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.383+0000 7f812d820740 1 Processor -- start 2024-04-25T00:01:45.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.383+0000 7f812d820740 1 -- start start 2024-04-25T00:01:45.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aafdbd70 0x5631aaff6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f812d820740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aaff5b60 0x5631ab00cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f812d820740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81241ff790 con 0x5631aaff5b60 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f812414b4a0 con 0x5631aafd8c80 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f812414ea10 con 0x5631aafdbd70 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aaff5b60 0x5631ab00cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aaff5b60 0x5631ab00cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59960/0 (socket says 172.21.15.113:59960) 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 -- 172.21.15.113:0/221655348 learned_addr learned my addr 172.21.15.113:0/221655348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8123fff640 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f81237fe640 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aafdbd70 0x5631aaff6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 -- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aafdbd70 msgr2=0x5631aaff6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aafdbd70 0x5631aaff6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 -- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 msgr2=0x5631aaff6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 -- 172.21.15.113:0/221655348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631aafafb30 con 0x5631aaff5b60 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f8128fde640 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aaff5b60 0x5631ab00cc90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f811c009d90 tx=0x7f811c047070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d82e720041c8f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f81097fa640 1 -- 172.21.15.113:0/221655348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c031e60 con 0x5631aaff5b60 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f81097fa640 1 -- 172.21.15.113:0/221655348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c041040 con 0x5631aaff5b60 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.384+0000 7f81097fa640 1 -- 172.21.15.113:0/221655348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c004450 con 0x5631aaff5b60 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 -- 172.21.15.113:0/221655348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aaff5b60 msgr2=0x5631ab00cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aaff5b60 0x5631ab00cc90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f811c009d90 tx=0x7f811c047070 comp rx=0 tx=0).stop 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 -- 172.21.15.113:0/221655348 shutdown_connections 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aaff5b60 0x5631ab00cc90 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aafdbd70 0x5631aaff6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 --2- 172.21.15.113:0/221655348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafd8c80 0x5631aaff6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 -- 172.21.15.113:0/221655348 >> 172.21.15.113:0/221655348 conn(0x5631aafd8540 msgr2=0x5631aafa3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 -- 172.21.15.113:0/221655348 shutdown_connections 2024-04-25T00:01:45.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 -- 172.21.15.113:0/221655348 wait complete. 2024-04-25T00:01:45.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.385+0000 7f812d820740 1 -- start start 2024-04-25T00:01:45.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 1 Processor -- start 2024-04-25T00:01:45.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8c80 0x5631ab002d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafdbd70 0x5631ab051bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631ab052130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5631aaff3a90 con 0x5631aafd8c80 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631aaff76d0 con 0x5631aafdbd70 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5631aaff9b10 con 0x5631aaff5b60 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f812d820740 10 client.0 _ll_register_callbacks cb 0x7f811c003cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f8123fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8c80 0x5631ab002d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f8123fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8c80 0x5631ab002d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59966/0 (socket says 172.21.15.113:59966) 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.386+0000 7f8123fff640 1 -- 172.21.15.113:0/660807320 learned_addr learned my addr 172.21.15.113:0/660807320 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f81237fe640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafdbd70 0x5631ab051bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8128fde640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631ab052130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8123fff640 1 -- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 msgr2=0x5631ab052130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8123fff640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631ab052130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8123fff640 1 -- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafdbd70 msgr2=0x5631ab051bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8123fff640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafdbd70 0x5631ab051bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8123fff640 1 -- 172.21.15.113:0/660807320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631ab003290 con 0x5631aafd8c80 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8128fde640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631ab052130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f81237fe640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafdbd70 0x5631ab051bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8123fff640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8c80 0x5631ab002d50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8114005f80 tx=0x7f8114000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bb23296319ce1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8102ffd640 1 -- 172.21.15.113:0/660807320 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811400dcd0 con 0x5631aafd8c80 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8102ffd640 1 -- 172.21.15.113:0/660807320 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f81140070d0 con 0x5631aafd8c80 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f811c041040 con 0x5631aafd8c80 2024-04-25T00:01:45.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8102ffd640 1 -- 172.21.15.113:0/660807320 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8114037050 con 0x5631aafd8c80 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 10 client.6579 fetch_fsmap learned FSMap version 31 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x5631aafd09c0 con 0x5631aafd8c80 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8102ffd640 1 -- 172.21.15.113:0/660807320 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c041040 con 0x5631aafd8c80 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f8102ffd640 1 -- 172.21.15.113:0/660807320 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f811402d020 con 0x5631aafd8c80 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 10 client.6579 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 10 client.6579 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 20 client.6579 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631ab0704d0 0x5631aafee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 4 client.6579 mds_command: new command op to 6402 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.387+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 --> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] -- command(tid 0: {"prefix": "status"}) -- 0x5631aaf18790 con 0x5631ab0704d0 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f81237fe640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631ab0704d0 0x5631aafee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f8102ffd640 10 client.6579 ms_handle_connect on v2:172.21.15.178:6832/3704852657 2024-04-25T00:01:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f81237fe640 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631ab0704d0 0x5631aafee9c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b739baf8015fbcd2 server_cookie=b050ea665f58f72d in_seq=0 out_seq=0 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f8102ffd640 1 -- 172.21.15.113:0/660807320 <== mds.0 v2:172.21.15.178:6832/3704852657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5631aaf18790 con 0x5631ab0704d0 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f8102ffd640 10 client.6579 handle_command_reply: tid=0 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 client.6579 shutdown 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8c80 msgr2=0x5631ab002d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8c80 0x5631ab002d50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8114005f80 tx=0x7f8114000ed0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 shutdown_connections 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:6832/3704852657,v1:172.21.15.178:6834/3704852657] conn(0x5631ab0704d0 0x5631aafee9c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631aafdbd70 0x5631ab051bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5631aaff5b60 0x5631ab052130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f8102ffd640 0 client.6579 ms_handle_reset on v2:172.21.15.178:6832/3704852657 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 --2- 172.21.15.113:0/660807320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5631aafd8c80 0x5631ab002d50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.388+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 >> 172.21.15.113:0/660807320 conn(0x5631aafd8540 msgr2=0x5631aafdac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 shutdown_connections 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812d820740 1 -- 172.21.15.113:0/660807320 wait complete. 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812d820740 20 client.6579 trim_cache size 0 max 16384 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 6402, 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:01:45.396 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 8.9351116529999999, 2024-04-25T00:01:45.397 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 31, 2024-04-25T00:01:45.397 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 60, 2024-04-25T00:01:45.397 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 60, 2024-04-25T00:01:45.397 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 13.981823952999999 2024-04-25T00:01:45.397 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811006be30 msgr2=0x7f811006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811006be30 0x7f811006e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8114001700 tx=0x7f8114009040 comp rx=0 tx=0).stop 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f812413f050 msgr2=0x7f812413b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f812413f050 0x7f812413b9f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f811800e930 tx=0x7f811800edf0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 shutdown_connections 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f811006be30 0x7f811006e2f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81241604d0 0x7f8124150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f812413f050 0x7f812413b9f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 --2- 172.21.15.113:0/4559269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81240d06c0 0x7f81241f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 >> 172.21.15.113:0/4559269 conn(0x7f812406b0f0 msgr2=0x7f812415de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 shutdown_connections 2024-04-25T00:01:45.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.389+0000 7f812aa41640 1 -- 172.21.15.113:0/4559269 wait complete. 2024-04-25T00:01:45.400 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:01:45.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.497+0000 7f83d5914640 1 Processor -- start 2024-04-25T00:01:45.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.497+0000 7f83d5914640 1 -- start start 2024-04-25T00:01:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83d5914640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d00cb4d0 0x7f83d006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83d5914640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d006bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83d5914640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d006ee00 con 0x7f83d00cea50 2024-04-25T00:01:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83d5914640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f83d006b0f0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83d5914640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83d006f8e0 con 0x7f83d006e950 2024-04-25T00:01:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ce7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d00cb4d0 0x7f83d006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ce7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d00cb4d0 0x7f83d006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59968/0 (socket says 172.21.15.113:59968) 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ceffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f83d00cea50 0x7f83d00cee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53536/0 (socket says 172.21.15.113:53536) 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ce7fc640 1 -- 172.21.15.113:0/3186092262 learned_addr learned my addr 172.21.15.113:0/3186092262 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83cf7fe640 1 --2- 172.21.15.113:0/3186092262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d006bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ce7fc640 1 -- 172.21.15.113:0/3186092262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 msgr2=0x7f83d006bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ce7fc640 1 --2- 172.21.15.113:0/3186092262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d006bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ce7fc640 1 -- 172.21.15.113:0/3186092262 >> v1:172.21.15.113:6789/0 conn(0x7f83d00cea50 legacy=0x7f83d00cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83ce7fc640 1 -- 172.21.15.113:0/3186092262 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d006fbc0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83cdffb640 1 -- 172.21.15.113:0/3186092262 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2289082326 0 0) 0x7f83d006ee00 con 0x7f83d00cea50 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.498+0000 7f83cf7fe640 1 --2- 172.21.15.113:0/3186092262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d006bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.499+0000 7f83ce7fc640 1 --2- 172.21.15.113:0/3186092262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d006f2c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f83c400bb10 tx=0x7f83c402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266635db671b8ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.499+0000 7f83cdffb640 1 -- 172.21.15.113:0/3186092262 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4032790 con 0x7f83d00cb4d0 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.499+0000 7f83cdffb640 1 -- 172.21.15.113:0/3186092262 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c4038070 con 0x7f83d00cb4d0 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.499+0000 7f83cdffb640 1 -- 172.21.15.113:0/3186092262 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4032b50 con 0x7f83d00cb4d0 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 -- 172.21.15.113:0/3186092262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 msgr2=0x7f83d006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 --2- 172.21.15.113:0/3186092262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d006f2c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f83c400bb10 tx=0x7f83c402c890 comp rx=0 tx=0).stop 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 -- 172.21.15.113:0/3186092262 shutdown_connections 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 --2- 172.21.15.113:0/3186092262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d006bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 --2- 172.21.15.113:0/3186092262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d006f2c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 -- 172.21.15.113:0/3186092262 >> 172.21.15.113:0/3186092262 conn(0x7f83d00cd900 msgr2=0x7f83d00cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 -- 172.21.15.113:0/3186092262 shutdown_connections 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.500+0000 7f83d5914640 1 -- 172.21.15.113:0/3186092262 wait complete. 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.501+0000 7f83d5914640 1 Processor -- start 2024-04-25T00:01:45.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.501+0000 7f83d5914640 1 -- start start 2024-04-25T00:01:45.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d00c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d00c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d00cea50 0x7f83d01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83d015e6e0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83d006b620 con 0x7f83d00cea50 2024-04-25T00:01:45.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83d015e860 con 0x7f83d006e950 2024-04-25T00:01:45.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d00c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d00c2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59972/0 (socket says 172.21.15.113:59972) 2024-04-25T00:01:45.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 -- 172.21.15.113:0/1273753541 learned_addr learned my addr 172.21.15.113:0/1273753541 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83cf7fe640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d00cea50 0x7f83d01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ceffd640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d00c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 -- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 msgr2=0x7f83d00c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d00c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 -- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d00cea50 msgr2=0x7f83d01587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d00cea50 0x7f83d01587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 -- 172.21.15.113:0/1273753541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d0158d20 con 0x7f83d00cb4d0 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83cf7fe640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d00cea50 0x7f83d01587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ceffd640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d00c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83ce7fc640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d00c2b50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83b7fff640 1 -- 172.21.15.113:0/1273753541 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4033620 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83b7fff640 1 -- 172.21.15.113:0/1273753541 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c40337e0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d015cd80 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83b7fff640 1 -- 172.21.15.113:0/1273753541 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4033bd0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.502+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d00c73d0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.503+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d015d2a0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.504+0000 7f83b7fff640 1 -- 172.21.15.113:0/1273753541 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83c404d060 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.504+0000 7f83b7fff640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83bc06be30 0x7f83bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.504+0000 7f83b7fff640 1 -- 172.21.15.113:0/1273753541 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f83c40ada00 con 0x7f83d00cb4d0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.504+0000 7f83ceffd640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83bc06be30 0x7f83bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.505+0000 7f83ceffd640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83bc06be30 0x7f83bc06e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000f60 tx=0x7f83c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.508+0000 7f83b7fff640 1 -- 172.21.15.113:0/1273753541 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83d015d2a0 con 0x7f83d00cb4d0 2024-04-25T00:01:45.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.717+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f83d00ce560 con 0x7f83d00cb4d0 2024-04-25T00:01:45.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.718+0000 7f83b7fff640 1 -- 172.21.15.113:0/1273753541 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) ==== 53+0+73 (secure 0 0 0) 0x7f83c4041070 con 0x7f83d00cb4d0 2024-04-25T00:01:45.718 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:01:45.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.719+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83bc06be30 msgr2=0x7f83bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.719+0000 7f83d5914640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83bc06be30 0x7f83bc06e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000f60 tx=0x7f83c0009040 comp rx=0 tx=0).stop 2024-04-25T00:01:45.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.719+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 msgr2=0x7f83d00c2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.719+0000 7f83d5914640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d00c2b50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c40046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 shutdown_connections 2024-04-25T00:01:45.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83bc06be30 0x7f83bc06e2f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d00cea50 0x7f83d01587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d00cb4d0 0x7f83d00c2b50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 --2- 172.21.15.113:0/1273753541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d006e950 0x7f83d00c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 >> 172.21.15.113:0/1273753541 conn(0x7f83d00cd900 msgr2=0x7f83d00cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 shutdown_connections 2024-04-25T00:01:45.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45.720+0000 7f83d5914640 1 -- 172.21.15.113:0/1273753541 wait complete. 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:01:45.731 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:01:45.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:45.731 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:01:45.747 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42575sec preferred_lft 42575sec 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:45.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:45.749 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:01:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:01:45.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:45.811 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:01:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:01:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:01:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:01:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:01:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:01:45.868 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:01:45.868 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:01:45.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:01:45.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:01:45.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:01:45.868 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:01:45.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:01:45.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:45.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:45.991 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:01:46.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:01:46.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:01:46.006 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:01:46.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:46.006 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:01:46.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:01:46.069 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:46.124 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:01:46.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:01:46.124 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:46.178 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:01:46.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:46.179 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:46.260 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:46.260 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:01:46.260 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:01:46.268 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:46.268 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:01:46.268 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:01:46.268 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:01:46.268 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:01:46.279 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:46.279 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:01:46.280 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:01:46.281 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:01:46.281 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:01:46.282 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:01:46.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:46.282 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:46.346 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:01:46.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:01:46.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:46.445 INFO:teuthology.orchestra.run.smithi113.stdout:client6591 2024-04-25T00:01:46.445 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:46.445 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:46.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:46.461 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T00:01:46.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:46.519 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T00:01:46.576 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:01:46.576 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:01:46.576 DEBUG:teuthology.orchestra.run.smithi113:> import stat 2024-04-25T00:01:46.576 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:01:46.576 DEBUG:teuthology.orchestra.run.smithi113:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T00:01:46.576 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:01:46.680 INFO:teuthology.orchestra.run.smithi113.stdout:1099511627776 2024-04-25T00:01:46.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:46.688 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:46.689 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:46.745 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:46.745 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:46.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:46.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.935+0000 7fe557eee640 1 Processor -- start 2024-04-25T00:01:46.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.935+0000 7fe557eee640 1 -- start start 2024-04-25T00:01:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.936+0000 7fe557eee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe55015e320 0x7fe5500d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.936+0000 7fe557eee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5500d0b00 0x7fe5500d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.936+0000 7fe557eee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5500d3de0 con 0x7fe5500d2290 2024-04-25T00:01:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.936+0000 7fe557eee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe5500d14d0 con 0x7fe5500d0b00 2024-04-25T00:01:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.936+0000 7fe557eee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5500d3410 con 0x7fe55015e320 2024-04-25T00:01:46.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe55648b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5500d0b00 0x7fe5500d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe555c8a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe55015e320 0x7fe5500d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe555489640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe5500d2290 0x7fe5500b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:45146/0 (socket says 172.21.15.113:45146) 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe555489640 1 -- 172.21.15.113:0/1158685240 learned_addr learned my addr 172.21.15.113:0/1158685240 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe55648b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5500d0b00 0x7fe5500d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:35578/0 (socket says 172.21.15.113:35578) 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe554c88640 1 -- 172.21.15.113:0/1158685240 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 623698277 0 0) 0x7fe5500d3de0 con 0x7fe5500d2290 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe554c88640 1 -- 172.21.15.113:0/1158685240 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe538003520 con 0x7fe5500d2290 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe555c8a640 1 -- 172.21.15.113:0/1158685240 >> v2:172.21.15.178:3300/0 conn(0x7fe5500d0b00 msgr2=0x7fe5500d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe555c8a640 1 --2- 172.21.15.113:0/1158685240 >> v2:172.21.15.178:3300/0 conn(0x7fe5500d0b00 0x7fe5500d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe555c8a640 1 -- 172.21.15.113:0/1158685240 >> v1:172.21.15.178:6789/0 conn(0x7fe5500d2290 legacy=0x7fe5500b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe555c8a640 1 -- 172.21.15.113:0/1158685240 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5500d2800 con 0x7fe55015e320 2024-04-25T00:01:46.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.937+0000 7fe55648b640 1 --2- 172.21.15.113:0/1158685240 >> v2:172.21.15.178:3300/0 conn(0x7fe5500d0b00 0x7fe5500d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.938+0000 7fe555c8a640 1 --2- 172.21.15.113:0/1158685240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe55015e320 0x7fe5500d38a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe540009d90 tx=0x7fe54002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c8b489e06a295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.938+0000 7fe554c88640 1 -- 172.21.15.113:0/1158685240 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540007740 con 0x7fe55015e320 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.938+0000 7fe554c88640 1 -- 172.21.15.113:0/1158685240 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5400078e0 con 0x7fe55015e320 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.938+0000 7fe554c88640 1 -- 172.21.15.113:0/1158685240 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540007c90 con 0x7fe55015e320 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.938+0000 7fe557eee640 1 -- 172.21.15.113:0/1158685240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe55015e320 msgr2=0x7fe5500d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.938+0000 7fe557eee640 1 --2- 172.21.15.113:0/1158685240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe55015e320 0x7fe5500d38a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe540009d90 tx=0x7fe54002a6f0 comp rx=0 tx=0).stop 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.939+0000 7fe557eee640 1 -- 172.21.15.113:0/1158685240 shutdown_connections 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.939+0000 7fe557eee640 1 --2- 172.21.15.113:0/1158685240 >> v2:172.21.15.178:3300/0 conn(0x7fe5500d0b00 0x7fe5500d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.939+0000 7fe557eee640 1 --2- 172.21.15.113:0/1158685240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe55015e320 0x7fe5500d38a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.939+0000 7fe557eee640 1 -- 172.21.15.113:0/1158685240 >> 172.21.15.113:0/1158685240 conn(0x7fe5500b5c20 msgr2=0x7fe5500b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.939+0000 7fe557eee640 1 -- 172.21.15.113:0/1158685240 shutdown_connections 2024-04-25T00:01:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.939+0000 7fe557eee640 1 -- 172.21.15.113:0/1158685240 wait complete. 2024-04-25T00:01:46.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.940+0000 7fe557eee640 1 Processor -- start 2024-04-25T00:01:46.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.940+0000 7fe557eee640 1 -- start start 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5500d0b00 0x7fe5500c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5500d2290 0x7fe5500b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe55015e320 0x7fe5500b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5500d30a0 con 0x7fe55015e320 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5500d2dc0 con 0x7fe5500d0b00 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe55015e700 con 0x7fe5500d2290 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe55015e320 0x7fe5500b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:46.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe55015e320 0x7fe5500b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59988/0 (socket says 172.21.15.113:59988) 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 -- 172.21.15.113:0/1249016777 learned_addr learned my addr 172.21.15.113:0/1249016777 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe555489640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5500d2290 0x7fe5500b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe555c8a640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5500d0b00 0x7fe5500c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 -- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5500d2290 msgr2=0x7fe5500b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5500d2290 0x7fe5500b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 -- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5500d0b00 msgr2=0x7fe5500c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5500d0b00 0x7fe5500c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 -- 172.21.15.113:0/1249016777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5500ca210 con 0x7fe55015e320 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe55648b640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe55015e320 0x7fe5500b8c30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe54800e930 tx=0x7fe54800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe54e7fc640 1 -- 172.21.15.113:0/1249016777 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548017070 con 0x7fe55015e320 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe54e7fc640 1 -- 172.21.15.113:0/1249016777 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54800ca00 con 0x7fe55015e320 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5500ca4f0 con 0x7fe55015e320 2024-04-25T00:01:46.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe54e7fc640 1 -- 172.21.15.113:0/1249016777 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54800cdd0 con 0x7fe55015e320 2024-04-25T00:01:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.941+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5500caab0 con 0x7fe55015e320 2024-04-25T00:01:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.942+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5500691a0 con 0x7fe55015e320 2024-04-25T00:01:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.943+0000 7fe54e7fc640 1 -- 172.21.15.113:0/1249016777 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe548013470 con 0x7fe55015e320 2024-04-25T00:01:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.943+0000 7fe54e7fc640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe53c06be30 0x7fe53c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.943+0000 7fe54e7fc640 1 -- 172.21.15.113:0/1249016777 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fe54808ac50 con 0x7fe55015e320 2024-04-25T00:01:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.943+0000 7fe555c8a640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe53c06be30 0x7fe53c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:46.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.944+0000 7fe555c8a640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe53c06be30 0x7fe53c06e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe540034000 tx=0x7fe54002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:46.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:46.947+0000 7fe54e7fc640 1 -- 172.21.15.113:0/1249016777 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe548090020 con 0x7fe55015e320 2024-04-25T00:01:47.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.120+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5500d1990 con 0x7fe55015e320 2024-04-25T00:01:47.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.122+0000 7fe54e7fc640 1 -- 172.21.15.113:0/1249016777 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16713 (secure 0 0 0) 0x7fe548059100 con 0x7fe55015e320 2024-04-25T00:01:47.122 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:47.122 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":60,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:37.460157+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:47.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.123+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe53c06be30 msgr2=0x7fe53c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe53c06be30 0x7fe53c06e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe540034000 tx=0x7fe54002f000 comp rx=0 tx=0).stop 2024-04-25T00:01:47.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe55015e320 msgr2=0x7fe5500b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe55015e320 0x7fe5500b8c30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe54800e930 tx=0x7fe54800edf0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 shutdown_connections 2024-04-25T00:01:47.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe53c06be30 0x7fe53c06e2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe55015e320 0x7fe5500b8c30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5500d2290 0x7fe5500b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 --2- 172.21.15.113:0/1249016777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5500d0b00 0x7fe5500c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 >> 172.21.15.113:0/1249016777 conn(0x7fe5500b5c20 msgr2=0x7fe5500cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:47.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 shutdown_connections 2024-04-25T00:01:47.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.124+0000 7fe557eee640 1 -- 172.21.15.113:0/1249016777 wait complete. 2024-04-25T00:01:47.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:01:34.014634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:01:34.775167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:47.135 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:47.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.236+0000 7ff2a1ba2640 1 Processor -- start 2024-04-25T00:01:47.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.236+0000 7ff2a1ba2640 1 -- start start 2024-04-25T00:01:47.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff2a1ba2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff29c0b6100 0x7ff29c0b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff2a1ba2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29c0d2e00 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff2a1ba2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29c0d30e0 con 0x7ff29c15e510 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff2a1ba2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff29c0d33c0 con 0x7ff29c0b6100 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff29bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff29c0b6100 0x7ff29c0b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff29bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff29c0b6100 0x7ff29c0b7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59994/0 (socket says 172.21.15.113:59994) 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff29bfff640 1 -- 172.21.15.113:0/2246313832 learned_addr learned my addr 172.21.15.113:0/2246313832 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff293fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff29c0b5ca0 0x7ff29c0b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53540/0 (socket says 172.21.15.113:53540) 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff29bfff640 1 -- 172.21.15.113:0/2246313832 >> v1:172.21.15.178:6789/0 conn(0x7ff29c15e510 legacy=0x7ff29c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff29bfff640 1 -- 172.21.15.113:0/2246313832 >> v1:172.21.15.113:6789/0 conn(0x7ff29c0b5ca0 legacy=0x7ff29c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff29bfff640 1 -- 172.21.15.113:0/2246313832 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c0d36a0 con 0x7ff29c0b6100 2024-04-25T00:01:47.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.237+0000 7ff29affd640 1 -- 172.21.15.113:0/2246313832 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931515515 0 0) 0x7ff29c0d2e00 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff29bfff640 1 --2- 172.21.15.113:0/2246313832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b6100 0x7ff29c0b7ad0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff294009e50 tx=0x7ff29402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90a80dd3cfd1177 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff29affd640 1 -- 172.21.15.113:0/2246313832 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294036070 con 0x7ff29c0b6100 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff29affd640 1 -- 172.21.15.113:0/2246313832 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294031600 con 0x7ff29c0b6100 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff29affd640 1 -- 172.21.15.113:0/2246313832 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940319d0 con 0x7ff29c0b6100 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/2246313832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b6100 msgr2=0x7ff29c0b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/2246313832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b6100 0x7ff29c0b7ad0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff294009e50 tx=0x7ff29402a6d0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/2246313832 shutdown_connections 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/2246313832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b6100 0x7ff29c0b7ad0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.238+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/2246313832 >> 172.21.15.113:0/2246313832 conn(0x7ff29c06b720 msgr2=0x7ff29c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.239+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/2246313832 shutdown_connections 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.239+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/2246313832 wait complete. 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 Processor -- start 2024-04-25T00:01:47.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 -- start start 2024-04-25T00:01:47.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b5ca0 0x7ff29c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff29c0b6100 0x7ff29c158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff29c15e510 0x7ff29c160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c0b6510 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c0d3ea0 con 0x7ff29c0b6100 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff2a1ba2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff29c0d2a90 con 0x7ff29c15e510 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff29b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b5ca0 0x7ff29c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff29b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b5ca0 0x7ff29c1526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60004/0 (socket says 172.21.15.113:60004) 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.240+0000 7ff29b7fe640 1 -- 172.21.15.113:0/528040050 learned_addr learned my addr 172.21.15.113:0/528040050 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff29b7fe640 1 -- 172.21.15.113:0/528040050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff29c15e510 msgr2=0x7ff29c160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff29b7fe640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff29c15e510 0x7ff29c160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff29b7fe640 1 -- 172.21.15.113:0/528040050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff29c0b6100 msgr2=0x7ff29c158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff29b7fe640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff29c0b6100 0x7ff29c158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff29b7fe640 1 -- 172.21.15.113:0/528040050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c160310 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff29b7fe640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b5ca0 0x7ff29c1526a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c460 tx=0x7ff28c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff298ff9640 1 -- 172.21.15.113:0/528040050 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c016020 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff298ff9640 1 -- 172.21.15.113:0/528040050 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff28c012870 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c15f270 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff298ff9640 1 -- 172.21.15.113:0/528040050 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c012c60 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.241+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c15f830 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.242+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c0691a0 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.242+0000 7ff298ff9640 1 -- 172.21.15.113:0/528040050 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff28c019020 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.243+0000 7ff298ff9640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff27c06be60 0x7ff27c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.243+0000 7ff293fff640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff27c06be60 0x7ff27c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.243+0000 7ff298ff9640 1 -- 172.21.15.113:0/528040050 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7ff28c08a900 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.243+0000 7ff293fff640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff27c06be60 0x7ff27c06e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff284007fd0 tx=0x7ff284007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.247+0000 7ff298ff9640 1 -- 172.21.15.113:0/528040050 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff28c020070 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.419+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff29c06c100 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.419+0000 7ff298ff9640 1 -- 172.21.15.113:0/528040050 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7ff29c160310 con 0x7ff29c0b5ca0 2024-04-25T00:01:47.420 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff27c06be60 msgr2=0x7ff27c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff27c06be60 0x7ff27c06e320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff284007fd0 tx=0x7ff284007960 comp rx=0 tx=0).stop 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b5ca0 msgr2=0x7ff29c1526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b5ca0 0x7ff29c1526a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c460 tx=0x7ff28c00c920 comp rx=0 tx=0).stop 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 shutdown_connections 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff27c06be60 0x7ff27c06e320 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff29c0b6100 0x7ff29c158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff29c15e510 0x7ff29c160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 --2- 172.21.15.113:0/528040050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff29c0b5ca0 0x7ff29c1526a0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 >> 172.21.15.113:0/528040050 conn(0x7ff29c06b720 msgr2=0x7ff29c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:47.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 shutdown_connections 2024-04-25T00:01:47.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.421+0000 7ff2a1ba2640 1 -- 172.21.15.113:0/528040050 wait complete. 2024-04-25T00:01:47.433 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:01:47.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.533+0000 7f2ebee9a640 1 Processor -- start 2024-04-25T00:01:47.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.533+0000 7f2ebee9a640 1 -- start start 2024-04-25T00:01:47.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2ebee9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 0x7f2eb80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2ebee9a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb80d0970 con 0x7f2eb80d04f0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2ebee9a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb815d6b0 con 0x7f2eb81606a0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2ebee9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb815c6a0 con 0x7f2eb813f050 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2eb7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2eb80d04f0 0x7f2eb815ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34158/0 (socket says 172.21.15.113:34158) 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2eb7fff640 1 -- 172.21.15.113:0/2816157355 learned_addr learned my addr 172.21.15.113:0/2816157355 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2ebcc36640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 0x7f2eb80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.534+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819355815 0 0) 0x7f2eb80d0970 con 0x7f2eb80d04f0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ea0003520 con 0x7f2eb80d04f0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933156133 0 0) 0x7f2eb815d6b0 con 0x7f2eb81606a0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2eb80d0970 con 0x7f2eb81606a0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2933429198 0 0) 0x7f2ea0003520 con 0x7f2eb80d04f0 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 msgr2=0x7f2eb80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:47.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 0x7f2eb80cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 >> v1:172.21.15.178:6789/0 conn(0x7f2eb81606a0 legacy=0x7f2eb80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb815c8f0 con 0x7f2eb80d04f0 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f2eac001670 con 0x7f2eb80d04f0 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 >> v1:172.21.15.113:6789/0 conn(0x7f2eb80d04f0 legacy=0x7f2eb815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0003b70 0x7f2ea0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea0004800 0x7f2ea0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 0x7f2ea0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb815c6a0 con 0x7f2ea0004800 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea000bb50 con 0x7f2ea0003b70 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea000bd20 con 0x7f2eb813f050 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb7fff640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea0004800 0x7f2ea0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2ebcc36640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0003b70 0x7f2ea0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2ebd437640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 0x7f2ea0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb7fff640 1 -- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 msgr2=0x7f2ea0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb7fff640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb813f050 0x7f2ea0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb7fff640 1 -- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0003b70 msgr2=0x7f2ea0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb7fff640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0003b70 0x7f2ea0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.535+0000 7f2eb7fff640 1 -- 172.21.15.113:0/2816157355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea0003520 con 0x7f2ea0004800 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.536+0000 7f2eb7fff640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea0004800 0x7f2ea0004c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00c110 tx=0x7f2ea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c295885a4402d019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.536+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac007c00 con 0x7f2ea0004800 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.536+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac005ad0 con 0x7f2ea0004800 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.536+0000 7f2eb77fe640 1 -- 172.21.15.113:0/2816157355 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac007070 con 0x7f2ea0004800 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.536+0000 7f2ebee9a640 1 -- 172.21.15.113:0/2816157355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea0004800 msgr2=0x7f2ea0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.537+0000 7f2ebee9a640 1 --2- 172.21.15.113:0/2816157355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea0004800 0x7f2ea0004c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00c110 tx=0x7f2ea0005830 comp rx=0 tx=0).stop 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.537+0000 7f2ebd437640 1 -- 172.21.15.113:0/2816157355 reap_dead start 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.537+0000 7f2ebee9a640 1 -- 172.21.15.113:0/2816157355 shutdown_connections 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.537+0000 7f2ebee9a640 1 -- 172.21.15.113:0/2816157355 >> 172.21.15.113:0/2816157355 conn(0x7f2eb806b0f0 msgr2=0x7f2eb806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.537+0000 7f2ebee9a640 1 -- 172.21.15.113:0/2816157355 shutdown_connections 2024-04-25T00:01:47.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.537+0000 7f2ebee9a640 1 -- 172.21.15.113:0/2816157355 wait complete. 2024-04-25T00:01:47.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.538+0000 7f2ebee9a640 1 Processor -- start 2024-04-25T00:01:47.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.538+0000 7f2ebee9a640 1 -- start start 2024-04-25T00:01:47.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0004800 0x7f2eb81511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb8138330 0x7f2eb8139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb815b190 0x7f2eb815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb815ceb0 con 0x7f2eb815b190 2024-04-25T00:01:47.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb8160aa0 con 0x7f2ea0004800 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb815d190 con 0x7f2eb8138330 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb815b190 0x7f2eb815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb815b190 0x7f2eb815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43872/0 (socket says 172.21.15.113:43872) 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 -- 172.21.15.113:0/162729969 learned_addr learned my addr 172.21.15.113:0/162729969 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2eb7fff640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb8138330 0x7f2eb8139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebcc36640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0004800 0x7f2eb81511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 -- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb8138330 msgr2=0x7f2eb8139030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb8138330 0x7f2eb8139030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 -- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0004800 msgr2=0x7f2eb81511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0004800 0x7f2eb81511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 -- 172.21.15.113:0/162729969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb81399a0 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebd437640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb815b190 0x7f2eb815b640 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806f490 tx=0x7f2eb000feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2eb57fa640 1 -- 172.21.15.113:0/162729969 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0019070 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2eb57fa640 1 -- 172.21.15.113:0/162729969 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb0006e20 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb81f43f0 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2eb57fa640 1 -- 172.21.15.113:0/162729969 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0003b90 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.539+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb81f49b0 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.540+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb80691a0 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.541+0000 7f2eb57fa640 1 -- 172.21.15.113:0/162729969 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2eb001c070 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.541+0000 7f2eb57fa640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ea406be60 0x7f2ea406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.541+0000 7f2ebcc36640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ea406be60 0x7f2ea406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.541+0000 7f2eb57fa640 1 -- 172.21.15.113:0/162729969 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f2eb008a070 con 0x7f2eb815b190 2024-04-25T00:01:47.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.542+0000 7f2ebcc36640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ea406be60 0x7f2ea406e320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80b2740 tx=0x7f2ea8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.545+0000 7f2eb57fa640 1 -- 172.21.15.113:0/162729969 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2eb00583d0 con 0x7f2eb815b190 2024-04-25T00:01:47.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.720+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2eb80d0e50 con 0x7f2eb815b190 2024-04-25T00:01:47.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.720+0000 7f2eb57fa640 1 -- 172.21.15.113:0/162729969 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) ==== 104+0+11 (secure 0 0 0) 0x7f2eb005c6d0 con 0x7f2eb815b190 2024-04-25T00:01:47.720 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:01:47.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ea406be60 msgr2=0x7f2ea406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ea406be60 0x7f2ea406e320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80b2740 tx=0x7f2ea8009190 comp rx=0 tx=0).stop 2024-04-25T00:01:47.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb815b190 msgr2=0x7f2eb815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb815b190 0x7f2eb815b640 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806f490 tx=0x7f2eb000feb0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 shutdown_connections 2024-04-25T00:01:47.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ea406be60 0x7f2ea406e320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb815b190 0x7f2eb815b640 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2eb8138330 0x7f2eb8139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 --2- 172.21.15.113:0/162729969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ea0004800 0x7f2eb81511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 >> 172.21.15.113:0/162729969 conn(0x7f2eb806b0f0 msgr2=0x7f2eb80b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:47.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 shutdown_connections 2024-04-25T00:01:47.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.722+0000 7f2ebee9a640 1 -- 172.21.15.113:0/162729969 wait complete. 2024-04-25T00:01:47.733 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:01:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.833+0000 7fd4331e7640 1 Processor -- start 2024-04-25T00:01:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.833+0000 7fd4331e7640 1 -- start start 2024-04-25T00:01:47.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd4331e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd42c13f050 0x7fd42c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd4331e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd42c15d6b0 con 0x7fd42c0d04f0 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd4331e7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd42c15c8f0 con 0x7fd42c1606a0 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd4331e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd42c0d09a0 con 0x7fd42c13f050 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd430f83640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd42c13f050 0x7fd42c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd42bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd42c0d04f0 0x7fd42c15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34168/0 (socket says 172.21.15.113:34168) 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd42bfff640 1 -- 172.21.15.113:0/3893288924 learned_addr learned my addr 172.21.15.113:0/3893288924 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd430f83640 1 -- 172.21.15.113:0/3893288924 >> v1:172.21.15.178:6789/0 conn(0x7fd42c1606a0 legacy=0x7fd42c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd430f83640 1 -- 172.21.15.113:0/3893288924 >> v1:172.21.15.113:6789/0 conn(0x7fd42c0d04f0 legacy=0x7fd42c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd430f83640 1 -- 172.21.15.113:0/3893288924 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c15cbd0 con 0x7fd42c13f050 2024-04-25T00:01:47.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.834+0000 7fd42b7fe640 1 -- 172.21.15.113:0/3893288924 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 602404102 0 0) 0x7fd42c15d6b0 con 0x7fd42c0d04f0 2024-04-25T00:01:47.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.835+0000 7fd430f83640 1 --2- 172.21.15.113:0/3893288924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c13f050 0x7fd42c0cffb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd4200097a0 tx=0x7fd42002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3284f44a23713282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.835+0000 7fd42b7fe640 1 -- 172.21.15.113:0/3893288924 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420037070 con 0x7fd42c13f050 2024-04-25T00:01:47.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.835+0000 7fd42b7fe640 1 -- 172.21.15.113:0/3893288924 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42002ae50 con 0x7fd42c13f050 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.835+0000 7fd42b7fe640 1 -- 172.21.15.113:0/3893288924 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420032660 con 0x7fd42c13f050 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.836+0000 7fd4331e7640 1 -- 172.21.15.113:0/3893288924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c13f050 msgr2=0x7fd42c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.836+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3893288924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c13f050 0x7fd42c0cffb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd4200097a0 tx=0x7fd42002a140 comp rx=0 tx=0).stop 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.836+0000 7fd4331e7640 1 -- 172.21.15.113:0/3893288924 shutdown_connections 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.836+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3893288924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c13f050 0x7fd42c0cffb0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.836+0000 7fd4331e7640 1 -- 172.21.15.113:0/3893288924 >> 172.21.15.113:0/3893288924 conn(0x7fd42c06b0f0 msgr2=0x7fd42c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.836+0000 7fd4331e7640 1 -- 172.21.15.113:0/3893288924 shutdown_connections 2024-04-25T00:01:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.836+0000 7fd4331e7640 1 -- 172.21.15.113:0/3893288924 wait complete. 2024-04-25T00:01:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.837+0000 7fd4331e7640 1 Processor -- start 2024-04-25T00:01:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.837+0000 7fd4331e7640 1 -- start start 2024-04-25T00:01:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42c0d04f0 0x7fd42c157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd42c13f050 0x7fd42c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c1606a0 0x7fd42c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c15d190 con 0x7fd42c1606a0 2024-04-25T00:01:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c15c6a0 con 0x7fd42c13f050 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd42c13f430 con 0x7fd42c0d04f0 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c1606a0 0x7fd42c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c1606a0 0x7fd42c155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43894/0 (socket says 172.21.15.113:43894) 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 -- 172.21.15.113:0/3806482747 learned_addr learned my addr 172.21.15.113:0/3806482747 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd430f83640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42c0d04f0 0x7fd42c157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd42bfff640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd42c13f050 0x7fd42c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 -- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42c0d04f0 msgr2=0x7fd42c157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42c0d04f0 0x7fd42c157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 -- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd42c13f050 msgr2=0x7fd42c14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd42c13f050 0x7fd42c14f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 -- 172.21.15.113:0/3806482747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c155c80 con 0x7fd42c1606a0 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd431784640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c1606a0 0x7fd42c155330 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd42400b990 tx=0x7fd42400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4297fa640 1 -- 172.21.15.113:0/3806482747 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424019070 con 0x7fd42c1606a0 2024-04-25T00:01:47.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4297fa640 1 -- 172.21.15.113:0/3806482747 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42400cba0 con 0x7fd42c1606a0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42c155f60 con 0x7fd42c1606a0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4297fa640 1 -- 172.21.15.113:0/3806482747 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424014440 con 0x7fd42c1606a0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.838+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42c156520 con 0x7fd42c1606a0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.839+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42c068b50 con 0x7fd42c1606a0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.840+0000 7fd4297fa640 1 -- 172.21.15.113:0/3806482747 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd42401c070 con 0x7fd42c1606a0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.840+0000 7fd4297fa640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd41006be30 0x7fd41006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.840+0000 7fd4297fa640 1 -- 172.21.15.113:0/3806482747 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fd42408ac90 con 0x7fd42c1606a0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.840+0000 7fd430f83640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd41006be30 0x7fd41006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.840+0000 7fd430f83640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd41006be30 0x7fd41006e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd420034000 tx=0x7fd42002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:47.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:47.844+0000 7fd4297fa640 1 -- 172.21.15.113:0/3806482747 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd42c068b50 con 0x7fd42c1606a0 2024-04-25T00:01:48.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.017+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd42c0d0e50 con 0x7fd42c1606a0 2024-04-25T00:01:48.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.017+0000 7fd4297fa640 1 -- 172.21.15.113:0/3806482747 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) ==== 100+0+11 (secure 0 0 0) 0x7fd4240595b0 con 0x7fd42c1606a0 2024-04-25T00:01:48.018 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:01:48.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd41006be30 msgr2=0x7fd41006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd41006be30 0x7fd41006e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd420034000 tx=0x7fd42002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c1606a0 msgr2=0x7fd42c155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c1606a0 0x7fd42c155330 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd42400b990 tx=0x7fd42400be50 comp rx=0 tx=0).stop 2024-04-25T00:01:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 shutdown_connections 2024-04-25T00:01:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd41006be30 0x7fd41006e2f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd42c13f050 0x7fd42c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c1606a0 0x7fd42c155330 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 --2- 172.21.15.113:0/3806482747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42c0d04f0 0x7fd42c157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 >> 172.21.15.113:0/3806482747 conn(0x7fd42c06b0f0 msgr2=0x7fd42c13cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:48.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 shutdown_connections 2024-04-25T00:01:48.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.019+0000 7fd4331e7640 1 -- 172.21.15.113:0/3806482747 wait complete. 2024-04-25T00:01:48.031 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:01:48.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.127+0000 7faf36a8c640 1 Processor -- start 2024-04-25T00:01:48.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.127+0000 7faf36a8c640 1 -- start start 2024-04-25T00:01:48.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf36a8c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faf300719d0 0x7faf3015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf36a8c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf3006cea0 0x7faf300cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf36a8c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf30071580 con 0x7faf30071db0 2024-04-25T00:01:48.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf36a8c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7faf30141620 con 0x7faf3006cea0 2024-04-25T00:01:48.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf36a8c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faf30141fc0 con 0x7faf300719d0 2024-04-25T00:01:48.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf35029640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf3006cea0 0x7faf300cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf35029640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf3006cea0 0x7faf300cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43902/0 (socket says 172.21.15.113:43902) 2024-04-25T00:01:48.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf35029640 1 -- 172.21.15.113:0/300711220 learned_addr learned my addr 172.21.15.113:0/300711220 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:48.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.128+0000 7faf34828640 1 --2- 172.21.15.113:0/300711220 >> v2:172.21.15.178:3300/0 conn(0x7faf300719d0 0x7faf3015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf35029640 1 -- 172.21.15.113:0/300711220 >> v2:172.21.15.178:3300/0 conn(0x7faf300719d0 msgr2=0x7faf3015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:48.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf35029640 1 --2- 172.21.15.113:0/300711220 >> v2:172.21.15.178:3300/0 conn(0x7faf300719d0 0x7faf3015f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf35029640 1 -- 172.21.15.113:0/300711220 >> v1:172.21.15.178:6789/0 conn(0x7faf30071db0 legacy=0x7faf301418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:48.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf35029640 1 -- 172.21.15.113:0/300711220 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3006d3f0 con 0x7faf3006cea0 2024-04-25T00:01:48.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf35029640 1 --2- 172.21.15.113:0/300711220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf300cf020 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faf3006e150 tx=0x7faf2802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526e509cc724ff89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf2f7fe640 1 -- 172.21.15.113:0/300711220 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28036070 con 0x7faf3006cea0 2024-04-25T00:01:48.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf2f7fe640 1 -- 172.21.15.113:0/300711220 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf28031600 con 0x7faf3006cea0 2024-04-25T00:01:48.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.129+0000 7faf2f7fe640 1 -- 172.21.15.113:0/300711220 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280319f0 con 0x7faf3006cea0 2024-04-25T00:01:48.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 -- 172.21.15.113:0/300711220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 msgr2=0x7faf300cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 --2- 172.21.15.113:0/300711220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf300cf020 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faf3006e150 tx=0x7faf2802a7c0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 -- 172.21.15.113:0/300711220 shutdown_connections 2024-04-25T00:01:48.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 --2- 172.21.15.113:0/300711220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf300cf020 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 --2- 172.21.15.113:0/300711220 >> v2:172.21.15.178:3300/0 conn(0x7faf300719d0 0x7faf3015f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 -- 172.21.15.113:0/300711220 >> 172.21.15.113:0/300711220 conn(0x7faf3006b0f0 msgr2=0x7faf3006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:48.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 -- 172.21.15.113:0/300711220 shutdown_connections 2024-04-25T00:01:48.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.130+0000 7faf36a8c640 1 -- 172.21.15.113:0/300711220 wait complete. 2024-04-25T00:01:48.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.131+0000 7faf36a8c640 1 Processor -- start 2024-04-25T00:01:48.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.131+0000 7faf36a8c640 1 -- start start 2024-04-25T00:01:48.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf36a8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf30157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf36a8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf300719d0 0x7faf300b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf36a8c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf30071db0 0x7faf301548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf36a8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faf301439e0 con 0x7faf3006cea0 2024-04-25T00:01:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf36a8c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf3006d9b0 con 0x7faf30071db0 2024-04-25T00:01:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf36a8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faf3006dc90 con 0x7faf300719d0 2024-04-25T00:01:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf30157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf30157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43908/0 (socket says 172.21.15.113:43908) 2024-04-25T00:01:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 -- 172.21.15.113:0/3168211880 learned_addr learned my addr 172.21.15.113:0/3168211880 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf2ffff640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf300719d0 0x7faf300b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf35029640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf30071db0 0x7faf301548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 -- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf300719d0 msgr2=0x7faf300b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf300719d0 0x7faf300b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 -- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf30071db0 msgr2=0x7faf301548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf30071db0 0x7faf301548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 -- 172.21.15.113:0/3168211880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf301564f0 con 0x7faf3006cea0 2024-04-25T00:01:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.132+0000 7faf34828640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf30157590 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faf2000d800 tx=0x7faf2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.133+0000 7faf2d7fa640 1 -- 172.21.15.113:0/3168211880 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2000b6c0 con 0x7faf3006cea0 2024-04-25T00:01:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.133+0000 7faf2d7fa640 1 -- 172.21.15.113:0/3168211880 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf20019070 con 0x7faf3006cea0 2024-04-25T00:01:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.133+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf301567d0 con 0x7faf3006cea0 2024-04-25T00:01:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.133+0000 7faf2d7fa640 1 -- 172.21.15.113:0/3168211880 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2000ba80 con 0x7faf3006cea0 2024-04-25T00:01:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.133+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf30156d90 con 0x7faf3006cea0 2024-04-25T00:01:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.134+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf300691a0 con 0x7faf3006cea0 2024-04-25T00:01:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.134+0000 7faf2d7fa640 1 -- 172.21.15.113:0/3168211880 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf2000bc20 con 0x7faf3006cea0 2024-04-25T00:01:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.134+0000 7faf2d7fa640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf1c06be30 0x7faf1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.134+0000 7faf2ffff640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf1c06be30 0x7faf1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.134+0000 7faf2d7fa640 1 -- 172.21.15.113:0/3168211880 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7faf2008b1e0 con 0x7faf3006cea0 2024-04-25T00:01:48.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.135+0000 7faf2ffff640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf1c06be30 0x7faf1c06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faf30157ad0 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.139+0000 7faf2d7fa640 1 -- 172.21.15.113:0/3168211880 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faf20059540 con 0x7faf3006cea0 2024-04-25T00:01:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.323+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf300cfdf0 con 0x7faf3006cea0 2024-04-25T00:01:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.323+0000 7faf2d7fa640 1 -- 172.21.15.113:0/3168211880 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7faf2005d840 con 0x7faf3006cea0 2024-04-25T00:01:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:01:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf1c06be30 msgr2=0x7faf1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf1c06be30 0x7faf1c06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faf30157ad0 tx=0x7faf24009040 comp rx=0 tx=0).stop 2024-04-25T00:01:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 msgr2=0x7faf30157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf30157590 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faf2000d800 tx=0x7faf2000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 shutdown_connections 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf1c06be30 0x7faf1c06e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf30071db0 0x7faf301548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf300719d0 0x7faf300b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 --2- 172.21.15.113:0/3168211880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3006cea0 0x7faf30157590 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 >> 172.21.15.113:0/3168211880 conn(0x7faf3006b0f0 msgr2=0x7faf300b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 shutdown_connections 2024-04-25T00:01:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.325+0000 7faf36a8c640 1 -- 172.21.15.113:0/3168211880 wait complete. 2024-04-25T00:01:48.338 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:01:48.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:48.339 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:48.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:01:48.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:48.392 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:01:48.412 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:48.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.549+0000 7f8ffb89a640 1 Processor -- start 2024-04-25T00:01:48.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ffb89a640 1 -- start start 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ffb89a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ff4144d60 0x7f8ff40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ffb89a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ffb89a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8ff406e510 0x7f8ff415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ffb89a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff4145c40 con 0x7f8ff4144d60 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ffb89a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff406e940 con 0x7f8ff406e510 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ffb89a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff40cfad0 con 0x7f8ff41453c0 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.550+0000 7f8ff9636640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ff4144d60 0x7f8ff40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ff4144d60 0x7f8ff40b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43924/0 (socket says 172.21.15.113:43924) 2024-04-25T00:01:48.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 -- 172.21.15.113:0/2159241282 learned_addr learned my addr 172.21.15.113:0/2159241282 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff8e35640 1 --2- 172.21.15.113:0/2159241282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9e37640 1 --2- 172.21.15.113:0/2159241282 >> v2:172.21.15.178:3300/0 conn(0x7f8ff406e510 0x7f8ff415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 -- 172.21.15.113:0/2159241282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 msgr2=0x7f8ff406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 --2- 172.21.15.113:0/2159241282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 -- 172.21.15.113:0/2159241282 >> v2:172.21.15.178:3300/0 conn(0x7f8ff406e510 msgr2=0x7f8ff415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 --2- 172.21.15.113:0/2159241282 >> v2:172.21.15.178:3300/0 conn(0x7f8ff406e510 0x7f8ff415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 -- 172.21.15.113:0/2159241282 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff40cf230 con 0x7f8ff4144d60 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff8e35640 1 --2- 172.21.15.113:0/2159241282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9e37640 1 --2- 172.21.15.113:0/2159241282 >> v2:172.21.15.178:3300/0 conn(0x7f8ff406e510 0x7f8ff415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.551+0000 7f8ff9636640 1 --2- 172.21.15.113:0/2159241282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff40b5390 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40b25f0 tx=0x7f8fec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4274bdb0f2fb3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8febfff640 1 -- 172.21.15.113:0/2159241282 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec037070 con 0x7f8ff4144d60 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8febfff640 1 -- 172.21.15.113:0/2159241282 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec032440 con 0x7f8ff4144d60 2024-04-25T00:01:48.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8febfff640 1 -- 172.21.15.113:0/2159241282 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec032810 con 0x7f8ff4144d60 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2159241282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 msgr2=0x7f8ff40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2159241282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff40b5390 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40b25f0 tx=0x7f8fec02a180 comp rx=0 tx=0).stop 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2159241282 shutdown_connections 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2159241282 >> v2:172.21.15.178:3300/0 conn(0x7f8ff406e510 0x7f8ff415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2159241282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2159241282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff40b5390 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2159241282 >> 172.21.15.113:0/2159241282 conn(0x7f8ff40b1a50 msgr2=0x7f8ff40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2159241282 shutdown_connections 2024-04-25T00:01:48.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.552+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2159241282 wait complete. 2024-04-25T00:01:48.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.553+0000 7f8ffb89a640 1 Processor -- start 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.553+0000 7f8ffb89a640 1 -- start start 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ffb89a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff406e510 0x7f8ff415fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ffb89a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ffb89a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ffb89a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4070e10 con 0x7f8ff4144d60 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ffb89a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40cf7f0 con 0x7f8ff406e510 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ffb89a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff4145140 con 0x7f8ff41453c0 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff415b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43928/0 (socket says 172.21.15.113:43928) 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 -- 172.21.15.113:0/2204120051 learned_addr learned my addr 172.21.15.113:0/2204120051 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 -- 172.21.15.113:0/2204120051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 msgr2=0x7f8ff415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 -- 172.21.15.113:0/2204120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff406e510 msgr2=0x7f8ff415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff406e510 0x7f8ff415fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.554+0000 7f8ff8e35640 1 -- 172.21.15.113:0/2204120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff40ae4b0 con 0x7f8ff4144d60 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.555+0000 7f8ff8e35640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff415b820 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c7f0 tx=0x7f8fe400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.555+0000 7f8fe9ffb640 1 -- 172.21.15.113:0/2204120051 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4015070 con 0x7f8ff4144d60 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.555+0000 7f8fe9ffb640 1 -- 172.21.15.113:0/2204120051 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe40079d0 con 0x7f8ff4144d60 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.555+0000 7f8fe9ffb640 1 -- 172.21.15.113:0/2204120051 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4007da0 con 0x7f8ff4144d60 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.555+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff40ae790 con 0x7f8ff4144d60 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.555+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff40aed50 con 0x7f8ff4144d60 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.556+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff40aef50 con 0x7f8ff4144d60 2024-04-25T00:01:48.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.556+0000 7f8fe9ffb640 1 -- 172.21.15.113:0/2204120051 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fe4018070 con 0x7f8ff4144d60 2024-04-25T00:01:48.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.556+0000 7f8fe9ffb640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8fd806be30 0x7f8fd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.557+0000 7f8fe9ffb640 1 -- 172.21.15.113:0/2204120051 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe408ad50 con 0x7f8ff4144d60 2024-04-25T00:01:48.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.557+0000 7f8ff9636640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8fd806be30 0x7f8fd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.557+0000 7f8ff9636640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8fd806be30 0x7f8fd806e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fec034000 tx=0x7f8fec02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.561+0000 7f8fe9ffb640 1 -- 172.21.15.113:0/2204120051 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ff40aef50 con 0x7f8ff4144d60 2024-04-25T00:01:48.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.769+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ff40691a0 con 0x7f8ff4144d60 2024-04-25T00:01:48.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.770+0000 7f8fe9ffb640 1 -- 172.21.15.113:0/2204120051 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5193 (secure 0 0 0) 0x7f8fe408e030 con 0x7f8ff4144d60 2024-04-25T00:01:48.770 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:48.770 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3783503244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3783503244},{"type":"v1","addr":"172.21.15.178:6835","nonce":3783503244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3613161476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3613161476},{"type":"v1","addr":"172.21.15.113:6835","nonce":3613161476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2692423657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2692423657},{"type":"v1","addr":"172.21.15.113:6837","nonce":2692423657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:37.799020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/3704852657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3704852657},{"type":"v1","addr":"172.21.15.178:6834","nonce":3704852657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":4}]} 2024-04-25T00:01:48.770 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-04-25T00:01:48.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8fd806be30 msgr2=0x7f8fd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8fd806be30 0x7f8fd806e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fec034000 tx=0x7f8fec02c000 comp rx=0 tx=0).stop 2024-04-25T00:01:48.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 msgr2=0x7f8ff415b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff415b820 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c7f0 tx=0x7f8fe400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 shutdown_connections 2024-04-25T00:01:48.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8fd806be30 0x7f8fd806e2f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ff41453c0 0x7f8ff415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff4144d60 0x7f8ff415b820 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 --2- 172.21.15.113:0/2204120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ff406e510 0x7f8ff415fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 >> 172.21.15.113:0/2204120051 conn(0x7f8ff40b1a50 msgr2=0x7f8ff4071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:48.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 shutdown_connections 2024-04-25T00:01:48.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:48.772+0000 7f8ffb89a640 1 -- 172.21.15.113:0/2204120051 wait complete. 2024-04-25T00:01:48.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T00:01:48.918 INFO:teuthology.orchestra.run.smithi178.stdout:{"message":"","return_code":0} 2024-04-25T00:01:48.918 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T00:01:48.919 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:01:48.919 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:48.919 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:01:48.920 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:48.920 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:01:48.920 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:48.920 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:01:48.920 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:01:48.928 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:01:48.929 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:48.929 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:01:48.929 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:48.930 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:01:48.931 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:48.931 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:01:48.931 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:48.931 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T00:01:49.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 Processor -- start 2024-04-25T00:01:49.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 -- start start 2024-04-25T00:01:49.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5e7f70 0x562e0c5d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5d4a90 0x562e0c5f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562e0c4c1640 con 0x562e0c5d4a90 2024-04-25T00:01:49.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e0c4c5680 con 0x562e0c5e7f70 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f249591ff40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562e0c51cd90 con 0x562e0c5ee580 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.013+0000 7f24931a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5d4a90 0x562e0c5f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5d4a90 0x562e0c5f10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43950/0 (socket says 172.21.15.113:43950) 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 -- 172.21.15.113:0/4168150937 learned_addr learned my addr 172.21.15.113:0/4168150937 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24929a8640 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5e7f70 0x562e0c5d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24921a7640 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 -- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 msgr2=0x562e0c5f0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5f0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 -- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5e7f70 msgr2=0x562e0c5d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5e7f70 0x562e0c5d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 -- 172.21.15.113:0/4168150937 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0c545c10 con 0x562e0c5d4a90 2024-04-25T00:01:49.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24921a7640 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24929a8640 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5e7f70 0x562e0c5d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24931a9640 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5d4a90 0x562e0c5f10b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f248c025990 tx=0x7f248c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de42a264d86014ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24911a5640 1 -- 172.21.15.113:0/4168150937 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c055040 con 0x562e0c5d4a90 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24911a5640 1 -- 172.21.15.113:0/4168150937 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248c051440 con 0x562e0c5d4a90 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f24911a5640 1 -- 172.21.15.113:0/4168150937 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c006250 con 0x562e0c5d4a90 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.014+0000 7f249591ff40 1 -- 172.21.15.113:0/4168150937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5d4a90 msgr2=0x562e0c5f10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5d4a90 0x562e0c5f10b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f248c025990 tx=0x7f248c008030 comp rx=0 tx=0).stop 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 -- 172.21.15.113:0/4168150937 shutdown_connections 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5d4a90 0x562e0c5f10b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 --2- 172.21.15.113:0/4168150937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5e7f70 0x562e0c5d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 -- 172.21.15.113:0/4168150937 >> 172.21.15.113:0/4168150937 conn(0x562e0c5650e0 msgr2=0x562e0c5f18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 -- 172.21.15.113:0/4168150937 shutdown_connections 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 -- 172.21.15.113:0/4168150937 wait complete. 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 Processor -- start 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.015+0000 7f249591ff40 1 -- start start 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f249591ff40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5d4a90 0x562e0c5e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f249591ff40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5e7f70 0x562e0c543d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f249591ff40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f249591ff40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562e0c5e9630 con 0x562e0c5e7f70 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f249591ff40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e0c5e93e0 con 0x562e0c5d4a90 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f249591ff40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562e0c564230 con 0x562e0c5ee580 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5e7f70 0x562e0c543d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5e7f70 0x562e0c543d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43966/0 (socket says 172.21.15.113:43966) 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 -- 172.21.15.113:0/3001658380 learned_addr learned my addr 172.21.15.113:0/3001658380 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24929a8640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5d4a90 0x562e0c5e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24931a9640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 -- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 msgr2=0x562e0c5d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562e0c5ee580 0x562e0c5d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 -- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5d4a90 msgr2=0x562e0c5e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e0c5d4a90 0x562e0c5e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 -- 172.21.15.113:0/3001658380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0c5e86c0 con 0x562e0c5e7f70 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f24921a7640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5e7f70 0x562e0c543d90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2488009f00 tx=0x7f2488005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.016+0000 7f247bfff640 1 -- 172.21.15.113:0/3001658380 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248801a070 con 0x562e0c5e7f70 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.017+0000 7f247bfff640 1 -- 172.21.15.113:0/3001658380 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2488016610 con 0x562e0c5e7f70 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.017+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562e0c5403e0 con 0x562e0c5e7f70 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.017+0000 7f247bfff640 1 -- 172.21.15.113:0/3001658380 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24880169e0 con 0x562e0c5e7f70 2024-04-25T00:01:49.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.017+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0c544ff0 con 0x562e0c5e7f70 2024-04-25T00:01:49.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.018+0000 7f247bfff640 1 -- 172.21.15.113:0/3001658380 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f248801d070 con 0x562e0c5e7f70 2024-04-25T00:01:49.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.018+0000 7f247bfff640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248006a840 0x7f248006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.018+0000 7f247bfff640 1 -- 172.21.15.113:0/3001658380 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f2488089d70 con 0x562e0c5e7f70 2024-04-25T00:01:49.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.018+0000 7f24929a8640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248006a840 0x7f248006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.018+0000 7f249591ff40 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x562e0c5ebf50 0x562e0c5ec390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.018+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 --> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] -- osd_op(unknown.0.0:1 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) -- 0x562e0c5d0bd0 con 0x562e0c5ebf50 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.019+0000 7f24931a9640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x562e0c5ebf50 0x562e0c5ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.019+0000 7f24929a8640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248006a840 0x7f248006cd00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2484003130 tx=0x7f24840011c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.019+0000 7f24931a9640 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x562e0c5ebf50 0x562e0c5ec390 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f24931a9640 1 -- 172.21.15.113:0/3001658380 <== osd.7 v2:172.21.15.178:6824/3779386153 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v60'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f248c003070 con 0x562e0c5ebf50 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x562e0c5ebf50 msgr2=0x562e0c5ec390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x562e0c5ebf50 0x562e0c5ec390 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248006a840 msgr2=0x7f248006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248006a840 0x7f248006cd00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2484003130 tx=0x7f24840011c0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5e7f70 msgr2=0x562e0c543d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 --2- 172.21.15.113:0/3001658380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e0c5e7f70 0x562e0c543d90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2488009f00 tx=0x7f2488005e90 comp rx=0 tx=0).stop 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f24931a9640 1 -- 172.21.15.113:0/3001658380 reap_dead start 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 shutdown_connections 2024-04-25T00:01:49.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 >> 172.21.15.113:0/3001658380 conn(0x562e0c5650e0 msgr2=0x562e0c5f0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 shutdown_connections 2024-04-25T00:01:49.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.020+0000 7f249591ff40 1 -- 172.21.15.113:0/3001658380 wait complete. 2024-04-25T00:01:49.026 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T00:01:49.026 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T00:01:49.027 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T00:01:49.027 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T00:01:49.027 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T00:01:49.028 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T00:01:49.028 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T00:01:49.028 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T00:01:49.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.130+0000 7f1273561640 1 Processor -- start 2024-04-25T00:01:49.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.130+0000 7fa7fb5ee640 1 Processor -- start 2024-04-25T00:01:49.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.130+0000 7f17d0eba640 1 Processor -- start 2024-04-25T00:01:49.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.130+0000 7f1273561640 1 -- start start 2024-04-25T00:01:49.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.130+0000 7fa7fb5ee640 1 -- start start 2024-04-25T00:01:49.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.130+0000 7f17d0eba640 1 -- start start 2024-04-25T00:01:49.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1b3189e640 1 Processor -- start 2024-04-25T00:01:49.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1b3189e640 1 -- start start 2024-04-25T00:01:49.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1273561640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c13f050 0x7f126c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1273561640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f126c0d04f0 0x7f126c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1273561640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f126c1604d0 0x7f126c06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1273561640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f126c15c6a0 con 0x7f126c0d04f0 2024-04-25T00:01:49.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1273561640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f126c0d0970 con 0x7f126c1604d0 2024-04-25T00:01:49.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f1273561640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f126c15d6b0 con 0x7f126c13f050 2024-04-25T00:01:49.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f17d0eba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17cc15e510 0x7f17cc0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f17d0eba640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f17cc0b63b0 0x7f17cc0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f17d0eba640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17cc0d2e00 con 0x7f17cc0d28c0 2024-04-25T00:01:49.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f17d0eba640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f17cc0d30e0 con 0x7f17cc15e510 2024-04-25T00:01:49.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.131+0000 7f17d0eba640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f17cc0d33c0 con 0x7f17cc0b63b0 2024-04-25T00:01:49.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1270afc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f126c0d04f0 0x7f126c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17c3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f17cc0d28c0 0x7f17cc0b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34188/0 (socket says 172.21.15.113:34188) 2024-04-25T00:01:49.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17c3fff640 1 -- 172.21.15.113:0/3705894538 learned_addr learned my addr 172.21.15.113:0/3705894538 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:49.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17ca59c640 1 --2- 172.21.15.113:0/3705894538 >> v2:172.21.15.113:3300/0 conn(0x7f17cc15e510 0x7f17cc0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17cad9d640 1 --2- 172.21.15.113:0/3705894538 >> v2:172.21.15.178:3300/0 conn(0x7f17cc0b63b0 0x7f17cc0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c13f050 0x7f126c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c13f050 0x7f126c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:60822/0 (socket says 172.21.15.113:60822) 2024-04-25T00:01:49.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 -- 172.21.15.113:0/904696242 learned_addr learned my addr 172.21.15.113:0/904696242 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1271afe640 1 --2- 172.21.15.113:0/904696242 >> v2:172.21.15.178:3300/0 conn(0x7f126c1604d0 0x7f126c06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7fb5ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7f415ba50 0x7fa7f415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b3189e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b2c151fb0 0x7f1b2c15f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b3189e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1b2c15c140 0x7f1b2c0b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b3189e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c0716e0 0x7f1b2c071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b2c151fb0 0x7f1b2c15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17cad9d640 1 -- 172.21.15.113:0/3705894538 >> v2:172.21.15.113:3300/0 conn(0x7f17cc15e510 msgr2=0x7f17cc0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17cad9d640 1 --2- 172.21.15.113:0/3705894538 >> v2:172.21.15.113:3300/0 conn(0x7f17cc15e510 0x7f17cc0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17cad9d640 1 -- 172.21.15.113:0/3705894538 >> v1:172.21.15.113:6789/0 conn(0x7f17cc0d28c0 legacy=0x7f17cc0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f17cad9d640 1 -- 172.21.15.113:0/3705894538 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc0d36a0 con 0x7f17cc0b63b0 2024-04-25T00:01:49.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b2c151fb0 0x7f1b2c15f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44002/0 (socket says 172.21.15.113:44002) 2024-04-25T00:01:49.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2a7fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1b2c15c140 0x7f1b2c0b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2affd640 1 -- 172.21.15.113:0/1697692250 learned_addr learned my addr 172.21.15.113:0/1697692250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 -- 172.21.15.113:0/904696242 >> v2:172.21.15.178:3300/0 conn(0x7f126c1604d0 msgr2=0x7f126c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 --2- 172.21.15.113:0/904696242 >> v2:172.21.15.178:3300/0 conn(0x7f126c1604d0 0x7f126c06cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 -- 172.21.15.113:0/904696242 >> v2:172.21.15.113:3300/0 conn(0x7f126c0d04f0 msgr2=0x7f126c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 --2- 172.21.15.113:0/904696242 >> v2:172.21.15.113:3300/0 conn(0x7f126c0d04f0 0x7f126c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f12712fd640 1 -- 172.21.15.113:0/904696242 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c15c8f0 con 0x7f126c13f050 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c15f9d0 con 0x7f1b2c151fb0 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c152390 con 0x7f1b2c15c140 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c0c5830 con 0x7f1b2c0716e0 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2b7fe640 1 --2- 172.21.15.113:0/1697692250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c0716e0 0x7f1b2c071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2b7fe640 1 -- 172.21.15.113:0/1697692250 >> v2:172.21.15.178:3300/0 conn(0x7f1b2c15c140 msgr2=0x7f1b2c0b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2b7fe640 1 --2- 172.21.15.113:0/1697692250 >> v2:172.21.15.178:3300/0 conn(0x7f1b2c15c140 0x7f1b2c0b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2b7fe640 1 -- 172.21.15.113:0/1697692250 >> v2:172.21.15.113:3300/0 conn(0x7f1b2c151fb0 msgr2=0x7f1b2c15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7f1b2b7fe640 1 --2- 172.21.15.113:0/1697692250 >> v2:172.21.15.113:3300/0 conn(0x7f1b2c151fb0 0x7f1b2c15f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f1b2b7fe640 1 -- 172.21.15.113:0/1697692250 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c0c5ad0 con 0x7f1b2c0716e0 2024-04-25T00:01:49.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7fb5ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f4071170 con 0x7fa7f415b670 2024-04-25T00:01:49.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7fb5ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f4071450 con 0x7fa7f406cea0 2024-04-25T00:01:49.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7fb5ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f4071730 con 0x7fa7f415ba50 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7f8b89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7f415ba50 0x7fa7f415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7f8b89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7f415ba50 0x7fa7f415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43996/0 (socket says 172.21.15.113:43996) 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7f8b89640 1 -- 172.21.15.113:0/411245474 learned_addr learned my addr 172.21.15.113:0/411245474 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7f8b89640 1 -- 172.21.15.113:0/411245474 >> v1:172.21.15.178:6789/0 conn(0x7fa7f406cea0 legacy=0x7fa7f406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7f8b89640 1 -- 172.21.15.113:0/411245474 >> v1:172.21.15.113:6789/0 conn(0x7fa7f415b670 legacy=0x7fa7f40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.132+0000 7fa7f8b89640 1 -- 172.21.15.113:0/411245474 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f4071a10 con 0x7fa7f415ba50 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f12712fd640 1 --2- 172.21.15.113:0/904696242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c13f050 0x7f126c0cffb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f126c06f460 tx=0x7f126802a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b0632147b88d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f17cad9d640 1 --2- 172.21.15.113:0/3705894538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc0b73c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0b7190 tx=0x7f17c402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c36c6aa0face3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7fa7f8b89640 1 --2- 172.21.15.113:0/411245474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f415ba50 0x7fa7f415bff0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400bb00 tx=0x7fa7e402e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a703ff7754377a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f1263fff640 1 -- 172.21.15.113:0/904696242 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268037070 con 0x7f126c13f050 2024-04-25T00:01:49.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f1263fff640 1 -- 172.21.15.113:0/904696242 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1268032440 con 0x7f126c13f050 2024-04-25T00:01:49.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f1263fff640 1 -- 172.21.15.113:0/904696242 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268032810 con 0x7f126c13f050 2024-04-25T00:01:49.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f17c9d9b640 1 -- 172.21.15.113:0/3705894538 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4007740 con 0x7f17cc0b63b0 2024-04-25T00:01:49.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f1b2b7fe640 1 --2- 172.21.15.113:0/1697692250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c0716e0 0x7f1b2c071b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0d0470 tx=0x7f1b2402a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb63828816cc989 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f17c9d9b640 1 -- 172.21.15.113:0/3705894538 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c40078e0 con 0x7f17cc0b63b0 2024-04-25T00:01:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f17c9d9b640 1 -- 172.21.15.113:0/3705894538 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4007cb0 con 0x7f17cc0b63b0 2024-04-25T00:01:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7fa7ebfff640 1 -- 172.21.15.113:0/411245474 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4035770 con 0x7fa7f415ba50 2024-04-25T00:01:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7fa7ebfff640 1 -- 172.21.15.113:0/411245474 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e403b070 con 0x7fa7f415ba50 2024-04-25T00:01:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7fa7ebfff640 1 -- 172.21.15.113:0/411245474 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4035b30 con 0x7fa7f415ba50 2024-04-25T00:01:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 -- 172.21.15.113:0/904696242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c13f050 msgr2=0x7f126c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 --2- 172.21.15.113:0/904696242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c13f050 0x7f126c0cffb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f126c06f460 tx=0x7f126802a140 comp rx=0 tx=0).stop 2024-04-25T00:01:49.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.133+0000 7f1b29ffb640 1 -- 172.21.15.113:0/1697692250 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24007740 con 0x7f1b2c0716e0 2024-04-25T00:01:49.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b29ffb640 1 -- 172.21.15.113:0/1697692250 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b240078e0 con 0x7f1b2c0716e0 2024-04-25T00:01:49.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b29ffb640 1 -- 172.21.15.113:0/1697692250 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24007cb0 con 0x7f1b2c0716e0 2024-04-25T00:01:49.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 -- 172.21.15.113:0/3705894538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 msgr2=0x7f17cc0b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 --2- 172.21.15.113:0/3705894538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc0b73c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0b7190 tx=0x7f17c402a5f0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/411245474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f415ba50 msgr2=0x7fa7f415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 -- 172.21.15.113:0/3705894538 shutdown_connections 2024-04-25T00:01:49.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 --2- 172.21.15.113:0/3705894538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc0b73c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 -- 172.21.15.113:0/904696242 shutdown_connections 2024-04-25T00:01:49.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 --2- 172.21.15.113:0/904696242 >> v2:172.21.15.178:3300/0 conn(0x7f126c1604d0 0x7f126c06cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 --2- 172.21.15.113:0/904696242 >> v2:172.21.15.113:3300/0 conn(0x7f126c0d04f0 0x7f126c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 --2- 172.21.15.113:0/904696242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c13f050 0x7f126c0cffb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 -- 172.21.15.113:0/904696242 >> 172.21.15.113:0/904696242 conn(0x7f126c06b0f0 msgr2=0x7f126c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/411245474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f415ba50 0x7fa7f415bff0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400bb00 tx=0x7fa7e402e940 comp rx=0 tx=0).stop 2024-04-25T00:01:49.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 --2- 172.21.15.113:0/3705894538 >> v2:172.21.15.113:3300/0 conn(0x7f17cc15e510 0x7f17cc0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 -- 172.21.15.113:0/3705894538 >> 172.21.15.113:0/3705894538 conn(0x7f17cc06b720 msgr2=0x7f17cc0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 -- 172.21.15.113:0/3705894538 shutdown_connections 2024-04-25T00:01:49.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/411245474 shutdown_connections 2024-04-25T00:01:49.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/411245474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f415ba50 0x7fa7f415bff0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c0716e0 msgr2=0x7f1b2c071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 --2- 172.21.15.113:0/1697692250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c0716e0 0x7f1b2c071b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0d0470 tx=0x7f1b2402a540 comp rx=0 tx=0).stop 2024-04-25T00:01:49.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f17d0eba640 1 -- 172.21.15.113:0/3705894538 wait complete. 2024-04-25T00:01:49.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/411245474 >> 172.21.15.113:0/411245474 conn(0x7fa7f406b0f0 msgr2=0x7fa7f406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 shutdown_connections 2024-04-25T00:01:49.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 --2- 172.21.15.113:0/1697692250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c0716e0 0x7f1b2c071b70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 --2- 172.21.15.113:0/1697692250 >> v2:172.21.15.178:3300/0 conn(0x7f1b2c15c140 0x7f1b2c0b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 -- 172.21.15.113:0/904696242 shutdown_connections 2024-04-25T00:01:49.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1273561640 1 -- 172.21.15.113:0/904696242 wait complete. 2024-04-25T00:01:49.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 --2- 172.21.15.113:0/1697692250 >> v2:172.21.15.113:3300/0 conn(0x7f1b2c151fb0 0x7f1b2c15f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 >> 172.21.15.113:0/1697692250 conn(0x7f1b2c061f70 msgr2=0x7f1b2c0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 shutdown_connections 2024-04-25T00:01:49.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/411245474 shutdown_connections 2024-04-25T00:01:49.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7f1b3189e640 1 -- 172.21.15.113:0/1697692250 wait complete. 2024-04-25T00:01:49.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.134+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/411245474 wait complete. 2024-04-25T00:01:49.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 Processor -- start 2024-04-25T00:01:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f17d0eba640 1 Processor -- start 2024-04-25T00:01:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1b3189e640 1 Processor -- start 2024-04-25T00:01:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f17d0eba640 1 -- start start 2024-04-25T00:01:49.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 -- start start 2024-04-25T00:01:49.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7fa7fb5ee640 1 Processor -- start 2024-04-25T00:01:49.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7fa7fb5ee640 1 -- start start 2024-04-25T00:01:49.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1b3189e640 1 -- start start 2024-04-25T00:01:49.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c0d04f0 0x7f126c13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126c13f050 0x7f126c150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126c1604d0 0x7f126c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f126c15d190 con 0x7f126c1604d0 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f126c15ceb0 con 0x7f126c13f050 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.136+0000 7f1273561640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f126c13f430 con 0x7f126c0d04f0 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126c1604d0 0x7f126c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126c1604d0 0x7f126c150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44016/0 (socket says 172.21.15.113:44016) 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 -- 172.21.15.113:0/1437612385 learned_addr learned my addr 172.21.15.113:0/1437612385 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f12712fd640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c0d04f0 0x7f126c13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b3189e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b2c0716e0 0x7f1b2c1388d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17d0eba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1270afc640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126c13f050 0x7f126c150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b3189e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b2c151fb0 0x7f1b2c1f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17d0eba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17cc0d28c0 0x7f17cc0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 -- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c0d04f0 msgr2=0x7f126c13ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b3189e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c15c140 0x7f1b2c14c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b2c0716e0 0x7f1b2c1388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c0d04f0 0x7f126c13ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7fb5ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f406cea0 0x7fa7f414bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b2c0716e0 0x7f1b2c1388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55784/0 (socket says 172.21.15.113:55784) 2024-04-25T00:01:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 -- 172.21.15.113:0/2599364858 learned_addr learned my addr 172.21.15.113:0/2599364858 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b3240 con 0x7f1b2c151fb0 2024-04-25T00:01:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c15ef70 con 0x7f1b2c0716e0 2024-04-25T00:01:49.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c06d7b0 con 0x7f1b2c15c140 2024-04-25T00:01:49.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17d0eba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc15e510 0x7f17cc071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc1506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55796/0 (socket says 172.21.15.113:55796) 2024-04-25T00:01:49.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17ca59c640 1 -- 172.21.15.113:0/2447031511 learned_addr learned my addr 172.21.15.113:0/2447031511 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 -- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126c13f050 msgr2=0x7f126c150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126c13f050 0x7f126c150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 -- 172.21.15.113:0/1437612385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c149cd0 con 0x7f126c1604d0 2024-04-25T00:01:49.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17c3fff640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17cc0d28c0 0x7f17cc0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc0b9dd0 con 0x7f17cc0d28c0 2024-04-25T00:01:49.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7f938a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f406cea0 0x7fa7f414bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7f938a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f406cea0 0x7fa7f414bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44036/0 (socket says 172.21.15.113:44036) 2024-04-25T00:01:49.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2b7fe640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c15c140 0x7f1b2c14c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2a7fc640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b2c151fb0 0x7f1b2c1f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7fb5ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f415b670 0x7fa7f4155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7f938a640 1 -- 172.21.15.113:0/2181206851 learned_addr learned my addr 172.21.15.113:0/2181206851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7f415ba50 0x7fa7f40b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f40d0200 con 0x7fa7f406cea0 2024-04-25T00:01:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f40d0080 con 0x7fa7f415b670 2024-04-25T00:01:49.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f40d0380 con 0x7fa7f415ba50 2024-04-25T00:01:49.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1271afe640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126c1604d0 0x7f126c150aa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f126400efc0 tx=0x7f126400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc0d3ea0 con 0x7f17cc0b63b0 2024-04-25T00:01:49.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc0b7900 con 0x7f17cc15e510 2024-04-25T00:01:49.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 -- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c15c140 msgr2=0x7f1b2c14c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c15c140 0x7f1b2c14c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 -- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b2c151fb0 msgr2=0x7f1b2c1f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17cad9d640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc15e510 0x7f17cc071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17c3fff640 1 -- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc15e510 msgr2=0x7f17cc071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b2c151fb0 0x7f1b2c1f3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f1b2affd640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c1f4350 con 0x7f1b2c0716e0 2024-04-25T00:01:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17c3fff640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc15e510 0x7f17cc071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f9b8b640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7f415ba50 0x7fa7f40b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f8b89640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f415b670 0x7fa7f4155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1b2affd640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b2c0716e0 0x7f1b2c1388d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00dfd0 tx=0x7f1b1c010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17c3fff640 1 -- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 msgr2=0x7f17cc1506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17c3fff640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc1506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1261ffb640 1 -- 172.21.15.113:0/1437612385 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264018020 con 0x7f126c1604d0 2024-04-25T00:01:49.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17c3fff640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc0cce50 con 0x7f17cc0d28c0 2024-04-25T00:01:49.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.137+0000 7f17cad9d640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc15e510 0x7f17cc071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:49.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1261ffb640 1 -- 172.21.15.113:0/1437612385 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1264014480 con 0x7f126c1604d0 2024-04-25T00:01:49.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f938a640 1 -- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7f415ba50 msgr2=0x7fa7f40b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f938a640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7f415ba50 0x7fa7f40b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f938a640 1 -- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f415b670 msgr2=0x7fa7f4155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1b13fff640 1 -- 172.21.15.113:0/2599364858 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c00d2d0 con 0x7f1b2c0716e0 2024-04-25T00:01:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f938a640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f415b670 0x7fa7f4155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f938a640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f414ea90 con 0x7fa7f406cea0 2024-04-25T00:01:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1b13fff640 1 -- 172.21.15.113:0/2599364858 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c01d070 con 0x7f1b2c0716e0 2024-04-25T00:01:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1b13fff640 1 -- 172.21.15.113:0/2599364858 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c010de0 con 0x7f1b2c0716e0 2024-04-25T00:01:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c149fb0 con 0x7f126c1604d0 2024-04-25T00:01:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1261ffb640 1 -- 172.21.15.113:0/1437612385 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264014870 con 0x7f126c1604d0 2024-04-25T00:01:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c14d200 con 0x7f1b2c0716e0 2024-04-25T00:01:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f17c3fff640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17cc0d28c0 0x7f17cc0cc390 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000f60 tx=0x7f17b400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f17c2ffd640 1 -- 172.21.15.113:0/2447031511 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4017070 con 0x7f17cc0d28c0 2024-04-25T00:01:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f17c2ffd640 1 -- 172.21.15.113:0/2447031511 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b4005ce0 con 0x7f17cc0d28c0 2024-04-25T00:01:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f17c2ffd640 1 -- 172.21.15.113:0/2447031511 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b40072f0 con 0x7f17cc0d28c0 2024-04-25T00:01:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc0cd130 con 0x7f17cc0d28c0 2024-04-25T00:01:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7fa7f938a640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f406cea0 0x7fa7f414bf80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c410 tx=0x7fa7f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c14d7c0 con 0x7f1b2c0716e0 2024-04-25T00:01:49.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc0cd6f0 con 0x7f17cc0d28c0 2024-04-25T00:01:49.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.138+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c14a570 con 0x7f126c1604d0 2024-04-25T00:01:49.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.139+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc0691a0 con 0x7f17cc0d28c0 2024-04-25T00:01:49.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c068220 con 0x7f1b2c0716e0 2024-04-25T00:01:49.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c0691a0 con 0x7f126c1604d0 2024-04-25T00:01:49.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7fa7e9ffb640 1 -- 172.21.15.113:0/2181206851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0017070 con 0x7fa7f406cea0 2024-04-25T00:01:49.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7fa7e9ffb640 1 -- 172.21.15.113:0/2181206851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f00035f0 con 0x7fa7f406cea0 2024-04-25T00:01:49.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7fa7e9ffb640 1 -- 172.21.15.113:0/2181206851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00039e0 con 0x7fa7f406cea0 2024-04-25T00:01:49.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f4156950 con 0x7fa7f406cea0 2024-04-25T00:01:49.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f4156f10 con 0x7fa7f406cea0 2024-04-25T00:01:49.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7f1b13fff640 1 -- 172.21.15.113:0/2599364858 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b1c016050 con 0x7f1b2c0716e0 2024-04-25T00:01:49.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7f17c2ffd640 1 -- 172.21.15.113:0/2447031511 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17b401a070 con 0x7f17cc0d28c0 2024-04-25T00:01:49.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.140+0000 7f1b13fff640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1806be60 0x7f1b1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f1b13fff640 1 -- 172.21.15.113:0/2599364858 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f1b1c08eba0 con 0x7f1b2c0716e0 2024-04-25T00:01:49.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f1b2a7fc640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1806be60 0x7f1b1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f17c2ffd640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17ac06be80 0x7f17ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f1261ffb640 1 -- 172.21.15.113:0/1437612385 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1264005d10 con 0x7f126c1604d0 2024-04-25T00:01:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f1b2a7fc640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1806be60 0x7f1b1806e320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c14c610 tx=0x7f1b20009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f17c2ffd640 1 -- 172.21.15.113:0/2447031511 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f17b408a400 con 0x7f17cc0d28c0 2024-04-25T00:01:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f17ca59c640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17ac06be80 0x7f17ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f40691a0 con 0x7fa7f406cea0 2024-04-25T00:01:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.141+0000 7f1261ffb640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f125006bf30 0x7f125006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.142+0000 7f12712fd640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f125006bf30 0x7f125006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.142+0000 7f1261ffb640 1 -- 172.21.15.113:0/1437612385 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f126408b7c0 con 0x7f126c1604d0 2024-04-25T00:01:49.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.142+0000 7f17ca59c640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17ac06be80 0x7f17ac06e340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009b60 tx=0x7f17bc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.142+0000 7f12712fd640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f125006bf30 0x7f125006e3f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1268034000 tx=0x7f126802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.145+0000 7f1b13fff640 1 -- 172.21.15.113:0/2599364858 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b1c026020 con 0x7f1b2c0716e0 2024-04-25T00:01:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.145+0000 7f17c2ffd640 1 -- 172.21.15.113:0/2447031511 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f17b4021020 con 0x7f17cc0d28c0 2024-04-25T00:01:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.147+0000 7fa7e9ffb640 1 -- 172.21.15.113:0/2181206851 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7f0003b80 con 0x7fa7f406cea0 2024-04-25T00:01:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.147+0000 7fa7e9ffb640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7d806be30 0x7fa7d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.147+0000 7fa7f8b89640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7d806be30 0x7fa7d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.147+0000 7fa7f8b89640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7d806be30 0x7fa7d806e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7e403a000 tx=0x7fa7e4033000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.154+0000 7fa7e9ffb640 1 -- 172.21.15.113:0/2181206851 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7fa7f0056810 con 0x7fa7f406cea0 2024-04-25T00:01:49.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.155+0000 7f1261ffb640 1 -- 172.21.15.113:0/1437612385 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1264022070 con 0x7f126c1604d0 2024-04-25T00:01:49.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.164+0000 7fa7e9ffb640 1 -- 172.21.15.113:0/2181206851 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa7f0020080 con 0x7fa7f406cea0 2024-04-25T00:01:49.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.398+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7f1b2c0b5010 con 0x7f1b2c0716e0 2024-04-25T00:01:49.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.412+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7f17cc150c30 con 0x7f17cc0d28c0 2024-04-25T00:01:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.421+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7f126c0d0e50 con 0x7f126c1604d0 2024-04-25T00:01:49.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.427+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7fa7f414ec60 con 0x7fa7f406cea0 2024-04-25T00:01:49.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.527+0000 7f1b13fff640 1 -- 172.21.15.113:0/2599364858 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6393 v32) ==== 95+0+0 (secure 0 0 0) 0x7f1b2c1f4350 con 0x7f1b2c0716e0 2024-04-25T00:01:49.528 INFO:teuthology.orchestra.run.smithi113.stderr:failed mds gid 6393 2024-04-25T00:01:49.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1806be60 msgr2=0x7f1b1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1806be60 0x7f1b1806e320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c14c610 tx=0x7f1b20009190 comp rx=0 tx=0).stop 2024-04-25T00:01:49.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b2c0716e0 msgr2=0x7f1b2c1388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b2c0716e0 0x7f1b2c1388d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00dfd0 tx=0x7f1b1c010490 comp rx=0 tx=0).stop 2024-04-25T00:01:49.531 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f15d8317640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:01:49.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 shutdown_connections 2024-04-25T00:01:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1806be60 0x7f1b1806e320 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b2c15c140 0x7f1b2c14c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b2c151fb0 0x7f1b2c1f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 --2- 172.21.15.113:0/2599364858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b2c0716e0 0x7f1b2c1388d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.530+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 >> 172.21.15.113:0/2599364858 conn(0x7f1b2c061f70 msgr2=0x7f1b2c15c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.531+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 shutdown_connections 2024-04-25T00:01:49.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.531+0000 7f1b3189e640 1 -- 172.21.15.113:0/2599364858 wait complete. 2024-04-25T00:01:49.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.532+0000 7f17c2ffd640 1 -- 172.21.15.113:0/2447031511 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5656 v33) ==== 95+0+0 (secure 0 0 0) 0x7f17b40588d0 con 0x7f17cc0d28c0 2024-04-25T00:01:49.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.532+0000 7f1261ffb640 1 -- 172.21.15.113:0/1437612385 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 6402 v33) ==== 95+0+0 (secure 0 0 0) 0x7f1264010040 con 0x7f126c1604d0 2024-04-25T00:01:49.535 INFO:teuthology.orchestra.run.smithi113.stderr:failed mds gid 5656 2024-04-25T00:01:49.536 INFO:teuthology.orchestra.run.smithi113.stderr:failed mds gid 6402 2024-04-25T00:01:49.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17ac06be80 msgr2=0x7f17ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17ac06be80 0x7f17ac06e340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009b60 tx=0x7f17bc009190 comp rx=0 tx=0).stop 2024-04-25T00:01:49.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17cc0d28c0 msgr2=0x7f17cc0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17cc0d28c0 0x7f17cc0cc390 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000f60 tx=0x7f17b400c800 comp rx=0 tx=0).stop 2024-04-25T00:01:49.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 shutdown_connections 2024-04-25T00:01:49.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17ac06be80 0x7f17ac06e340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc15e510 0x7f17cc071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17cc0d28c0 0x7f17cc0cc390 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 --2- 172.21.15.113:0/2447031511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0b63b0 0x7f17cc1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 >> 172.21.15.113:0/2447031511 conn(0x7f17cc06b720 msgr2=0x7f17cc15cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f125006bf30 msgr2=0x7f125006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f125006bf30 0x7f125006e3f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1268034000 tx=0x7f126802e000 comp rx=0 tx=0).stop 2024-04-25T00:01:49.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126c1604d0 msgr2=0x7f126c150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126c1604d0 0x7f126c150aa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f126400efc0 tx=0x7f126400c490 comp rx=0 tx=0).stop 2024-04-25T00:01:49.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 shutdown_connections 2024-04-25T00:01:49.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f125006bf30 0x7f125006e3f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f126c13f050 0x7f126c150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f126c1604d0 0x7f126c150aa0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 --2- 172.21.15.113:0/1437612385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f126c0d04f0 0x7f126c13ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.535+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 >> 172.21.15.113:0/1437612385 conn(0x7f126c06b0f0 msgr2=0x7f126c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.536+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 shutdown_connections 2024-04-25T00:01:49.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.536+0000 7f1273561640 1 -- 172.21.15.113:0/1437612385 wait complete. 2024-04-25T00:01:49.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.536+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 shutdown_connections 2024-04-25T00:01:49.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.537+0000 7f17d0eba640 1 -- 172.21.15.113:0/2447031511 wait complete. 2024-04-25T00:01:49.548 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:01:49.548 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:01:49.550 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:01:49.550 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:49.551 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:01:49.552 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:01:49.553 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:01:49.553 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:01:49.555 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:01:49.556 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:49.556 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:01:49.556 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:49.621 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:01:49.637 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:01:49.638 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:01:49.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.812+0000 7fa7e9ffb640 1 -- 172.21.15.113:0/2181206851 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6390 v34) ==== 95+0+0 (secure 0 0 0) 0x7fa7f0058770 con 0x7fa7f406cea0 2024-04-25T00:01:49.813 INFO:teuthology.orchestra.run.smithi113.stderr:failed mds gid 6390 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7d806be30 msgr2=0x7fa7d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7d806be30 0x7fa7d806e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7e403a000 tx=0x7fa7e4033000 comp rx=0 tx=0).stop 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f406cea0 msgr2=0x7fa7f414bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f406cea0 0x7fa7f414bf80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c410 tx=0x7fa7f000c8d0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 shutdown_connections 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa7d806be30 0x7fa7d806e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f415b670 0x7fa7f4155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7f415ba50 0x7fa7f40b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 --2- 172.21.15.113:0/2181206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f406cea0 0x7fa7f414bf80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 >> 172.21.15.113:0/2181206851 conn(0x7fa7f406b0f0 msgr2=0x7fa7f406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 shutdown_connections 2024-04-25T00:01:49.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.816+0000 7fa7fb5ee640 1 -- 172.21.15.113:0/2181206851 wait complete. 2024-04-25T00:01:49.829 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:01:49.829 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:01:49.871 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:01:49.872 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:49.872 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:49.942 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:01:49.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.968+0000 7f3985c38640 1 Processor -- start 2024-04-25T00:01:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.968+0000 7f3985c38640 1 -- start start 2024-04-25T00:01:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f3985c38640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39800d0ad0 0x7f39800d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f3985c38640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f398015e590 0x7f39801628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f3985c38640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39800d13f0 con 0x7f39800b99e0 2024-04-25T00:01:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f3985c38640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f39800cf040 con 0x7f398015e590 2024-04-25T00:01:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f3985c38640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3980162de0 con 0x7f39800d0ad0 2024-04-25T00:01:49.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f397effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f398015e590 0x7f39801628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f397effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f398015e590 0x7f39801628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44066/0 (socket says 172.21.15.113:44066) 2024-04-25T00:01:49.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.969+0000 7f397effd640 1 -- 172.21.15.113:0/130540541 learned_addr learned my addr 172.21.15.113:0/130540541 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.970+0000 7f397f7fe640 1 --2- 172.21.15.113:0/130540541 >> v2:172.21.15.178:3300/0 conn(0x7f39800d0ad0 0x7f39800d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.970+0000 7f397effd640 1 -- 172.21.15.113:0/130540541 >> v2:172.21.15.178:3300/0 conn(0x7f39800d0ad0 msgr2=0x7f39800d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.970+0000 7f397effd640 1 --2- 172.21.15.113:0/130540541 >> v2:172.21.15.178:3300/0 conn(0x7f39800d0ad0 0x7f39800d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.970+0000 7f397effd640 1 -- 172.21.15.113:0/130540541 >> v1:172.21.15.178:6789/0 conn(0x7f39800b99e0 legacy=0x7f39800b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.970+0000 7f397effd640 1 -- 172.21.15.113:0/130540541 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39800b9f30 con 0x7f398015e590 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.970+0000 7f397effd640 1 --2- 172.21.15.113:0/130540541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f398015e590 0x7f39801628a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f39800d0aa0 tx=0x7f396802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec00fb56b903539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.970+0000 7f397e7fc640 1 -- 172.21.15.113:0/130540541 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968037070 con 0x7f398015e590 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f397e7fc640 1 -- 172.21.15.113:0/130540541 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3968032440 con 0x7f398015e590 2024-04-25T00:01:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f397e7fc640 1 -- 172.21.15.113:0/130540541 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968032810 con 0x7f398015e590 2024-04-25T00:01:49.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 -- 172.21.15.113:0/130540541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f398015e590 msgr2=0x7f39801628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 --2- 172.21.15.113:0/130540541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f398015e590 0x7f39801628a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f39800d0aa0 tx=0x7f396802a230 comp rx=0 tx=0).stop 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 -- 172.21.15.113:0/130540541 shutdown_connections 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 --2- 172.21.15.113:0/130540541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f398015e590 0x7f39801628a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 --2- 172.21.15.113:0/130540541 >> v2:172.21.15.178:3300/0 conn(0x7f39800d0ad0 0x7f39800d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 -- 172.21.15.113:0/130540541 >> 172.21.15.113:0/130540541 conn(0x7f398006b0f0 msgr2=0x7f398006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 -- 172.21.15.113:0/130540541 shutdown_connections 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.971+0000 7f3985c38640 1 -- 172.21.15.113:0/130540541 wait complete. 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.972+0000 7f3985c38640 1 Processor -- start 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.972+0000 7f3985c38640 1 -- start start 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f3985c38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39800b99e0 0x7f3980159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f3985c38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39800d0ad0 0x7f39801610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f3985c38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f398015e590 0x7f39800ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f3985c38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f39800cecd0 con 0x7f39800d0ad0 2024-04-25T00:01:49.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f3985c38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f39800ce9f0 con 0x7f398015e590 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f3985c38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f39800cf4d0 con 0x7f39800b99e0 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f397effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39800d0ad0 0x7f39801610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f397effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39800d0ad0 0x7f39801610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44080/0 (socket says 172.21.15.113:44080) 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f397effd640 1 -- 172.21.15.113:0/4206220606 learned_addr learned my addr 172.21.15.113:0/4206220606 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f397ffff640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f398015e590 0x7f39800ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f397effd640 1 -- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39800b99e0 msgr2=0x7f3980159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.973+0000 7f397f7fe640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39800b99e0 0x7f3980159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f397effd640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39800b99e0 0x7f3980159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f397effd640 1 -- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f398015e590 msgr2=0x7f39800ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f397effd640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f398015e590 0x7f39800ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f397effd640 1 -- 172.21.15.113:0/4206220606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39800b3c80 con 0x7f39800d0ad0 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f397ffff640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f398015e590 0x7f39800ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:49.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f397effd640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39800d0ad0 0x7f39801610d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f396802a270 tx=0x7f39680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f3977fff640 1 -- 172.21.15.113:0/4206220606 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968037040 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f3977fff640 1 -- 172.21.15.113:0/4206220606 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3968042070 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39800b3f60 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f3977fff640 1 -- 172.21.15.113:0/4206220606 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968030bb0 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.974+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39800b4520 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.975+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39800691a0 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.975+0000 7f3977fff640 1 -- 172.21.15.113:0/4206220606 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3968047460 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.976+0000 7f3977fff640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f396006be60 0x7f396006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.976+0000 7f397f7fe640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f396006be60 0x7f396006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.976+0000 7f3977fff640 1 -- 172.21.15.113:0/4206220606 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f39680aba80 con 0x7f39800d0ad0 2024-04-25T00:01:49.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.976+0000 7f397f7fe640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f396006be60 0x7f396006e320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3970006fd0 tx=0x7f3970008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:49.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:49.980+0000 7f3977fff640 1 -- 172.21.15.113:0/4206220606 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3968045360 con 0x7f39800d0ad0 2024-04-25T00:01:50.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.190+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39800d1d60 con 0x7f39800d0ad0 2024-04-25T00:01:50.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.191+0000 7f3977fff640 1 -- 172.21.15.113:0/4206220606 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+4359 (secure 0 0 0) 0x7f396807a460 con 0x7f39800d0ad0 2024-04-25T00:01:50.191 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:50.191 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:49.812498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"d","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/1818573977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1818573977},{"type":"v1","addr":"172.21.15.178:6835","nonce":1818573977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-25T00:01:50.192 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 34 2024-04-25T00:01:50.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f396006be60 msgr2=0x7f396006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:50.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f396006be60 0x7f396006e320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3970006fd0 tx=0x7f3970008040 comp rx=0 tx=0).stop 2024-04-25T00:01:50.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39800d0ad0 msgr2=0x7f39801610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:50.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39800d0ad0 0x7f39801610d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f396802a270 tx=0x7f39680046c0 comp rx=0 tx=0).stop 2024-04-25T00:01:50.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 shutdown_connections 2024-04-25T00:01:50.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f396006be60 0x7f396006e320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:50.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f398015e590 0x7f39800ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39800d0ad0 0x7f39801610d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 --2- 172.21.15.113:0/4206220606 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39800b99e0 0x7f3980159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 >> 172.21.15.113:0/4206220606 conn(0x7f398006b0f0 msgr2=0x7f39800d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 shutdown_connections 2024-04-25T00:01:50.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:50.193+0000 7f3985c38640 1 -- 172.21.15.113:0/4206220606 wait complete. 2024-04-25T00:01:50.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:01:35.811705+0000', 'modified': '2024-04-25T00:01:49.812498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6690}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6690': {'gid': 6690, 'name': 'd', 'rank': 0, 'incarnation': 34, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/1818573977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6833', 'nonce': 1818573977}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 1818573977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T00:01:50.608 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6690:up:replay 2024-04-25T00:01:51.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.718+0000 7f09d48cd640 1 Processor -- start 2024-04-25T00:01:51.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.718+0000 7f09d48cd640 1 -- start start 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09d48cd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d0144d60 0x7f09d00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09d48cd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f09d01453c0 0x7f09d006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09d48cd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d00cfad0 con 0x7f09d006e510 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09d48cd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f09d0145c40 con 0x7f09d01453c0 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09d48cd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09d006e940 con 0x7f09d0144d60 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09cdd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f09d01453c0 0x7f09d006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09cdd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f09d01453c0 0x7f09d006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44084/0 (socket says 172.21.15.113:44084) 2024-04-25T00:01:51.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.719+0000 7f09cdd9b640 1 -- 172.21.15.113:0/143342257 learned_addr learned my addr 172.21.15.113:0/143342257 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:51.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.720+0000 7f09ce59c640 1 --2- 172.21.15.113:0/143342257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d0144d60 0x7f09d00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.720+0000 7f09cdd9b640 1 -- 172.21.15.113:0/143342257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d0144d60 msgr2=0x7f09d00b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:51.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.720+0000 7f09cdd9b640 1 --2- 172.21.15.113:0/143342257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d0144d60 0x7f09d00b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.720+0000 7f09cdd9b640 1 -- 172.21.15.113:0/143342257 >> v1:172.21.15.178:6789/0 conn(0x7f09d006e510 legacy=0x7f09d015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.720+0000 7f09cdd9b640 1 -- 172.21.15.113:0/143342257 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d00cf230 con 0x7f09d01453c0 2024-04-25T00:01:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.720+0000 7f09cdd9b640 1 --2- 172.21.15.113:0/143342257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d006dfd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009870 tx=0x7f09c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c8c9c5154df138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09cd59a640 1 -- 172.21.15.113:0/143342257 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4037070 con 0x7f09d01453c0 2024-04-25T00:01:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09cd59a640 1 -- 172.21.15.113:0/143342257 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c4032440 con 0x7f09d01453c0 2024-04-25T00:01:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09cd59a640 1 -- 172.21.15.113:0/143342257 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4032810 con 0x7f09d01453c0 2024-04-25T00:01:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 -- 172.21.15.113:0/143342257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 msgr2=0x7f09d006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 --2- 172.21.15.113:0/143342257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d006dfd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009870 tx=0x7f09c402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 -- 172.21.15.113:0/143342257 shutdown_connections 2024-04-25T00:01:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 --2- 172.21.15.113:0/143342257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d006dfd0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 --2- 172.21.15.113:0/143342257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d0144d60 0x7f09d00b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 -- 172.21.15.113:0/143342257 >> 172.21.15.113:0/143342257 conn(0x7f09d00b1a50 msgr2=0x7f09d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 -- 172.21.15.113:0/143342257 shutdown_connections 2024-04-25T00:01:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.721+0000 7f09d48cd640 1 -- 172.21.15.113:0/143342257 wait complete. 2024-04-25T00:01:51.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 Processor -- start 2024-04-25T00:01:51.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 -- start start 2024-04-25T00:01:51.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d006e510 0x7f09d013c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0144d60 0x7f09d01e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0070e10 con 0x7f09d01453c0 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09d00cf7f0 con 0x7f09d0144d60 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.723+0000 7f09d48cd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09d0145140 con 0x7f09d006e510 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d015c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44086/0 (socket says 172.21.15.113:44086) 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 -- 172.21.15.113:0/3808428280 learned_addr learned my addr 172.21.15.113:0/3808428280 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ce59c640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d006e510 0x7f09d013c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09cdd9b640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0144d60 0x7f09d01e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 -- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d006e510 msgr2=0x7f09d013c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d006e510 0x7f09d013c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 -- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0144d60 msgr2=0x7f09d01e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0144d60 0x7f09d01e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 -- 172.21.15.113:0/3808428280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d013cc40 con 0x7f09d01453c0 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09ced9d640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d015c680 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c3c0 tx=0x7f09c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09beffd640 1 -- 172.21.15.113:0/3808428280 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c80052d0 con 0x7f09d01453c0 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09beffd640 1 -- 172.21.15.113:0/3808428280 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c8005470 con 0x7f09d01453c0 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d01493f0 con 0x7f09d01453c0 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09beffd640 1 -- 172.21.15.113:0/3808428280 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8007260 con 0x7f09d01453c0 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.724+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01499b0 con 0x7f09d01453c0 2024-04-25T00:01:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.725+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d00691a0 con 0x7f09d01453c0 2024-04-25T00:01:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.726+0000 7f09beffd640 1 -- 172.21.15.113:0/3808428280 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09c8007400 con 0x7f09d01453c0 2024-04-25T00:01:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.726+0000 7f09beffd640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09b006be30 0x7f09b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.726+0000 7f09beffd640 1 -- 172.21.15.113:0/3808428280 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f09c808a070 con 0x7f09d01453c0 2024-04-25T00:01:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.726+0000 7f09ce59c640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09b006be30 0x7f09b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.727+0000 7f09ce59c640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09b006be30 0x7f09b006e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009fd0 tx=0x7f09c0009800 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:51.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.731+0000 7f09beffd640 1 -- 172.21.15.113:0/3808428280 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09d013cc40 con 0x7f09d01453c0 2024-04-25T00:01:51.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.940+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09d00afe50 con 0x7f09d01453c0 2024-04-25T00:01:51.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.940+0000 7f09beffd640 1 -- 172.21.15.113:0/3808428280 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5187 (secure 0 0 0) 0x7f09c80584c0 con 0x7f09d01453c0 2024-04-25T00:01:51.941 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:51.941 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3327449639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3327449639},{"type":"v1","addr":"172.21.15.113:6837","nonce":3327449639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:50.816889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"d","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.178:6835/1818573977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1818573977},{"type":"v1","addr":"172.21.15.178:6835","nonce":1818573977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-25T00:01:51.941 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 36 2024-04-25T00:01:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09b006be30 msgr2=0x7f09b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09b006be30 0x7f09b006e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009fd0 tx=0x7f09c0009800 comp rx=0 tx=0).stop 2024-04-25T00:01:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 msgr2=0x7f09d015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d015c680 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c3c0 tx=0x7f09c800c880 comp rx=0 tx=0).stop 2024-04-25T00:01:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 shutdown_connections 2024-04-25T00:01:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09b006be30 0x7f09b006e2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d0144d60 0x7f09d01e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.942+0000 7f09d48cd640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d01453c0 0x7f09d015c680 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.943+0000 7f09d48cd640 1 --2- 172.21.15.113:0/3808428280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d006e510 0x7f09d013c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.943+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 >> 172.21.15.113:0/3808428280 conn(0x7f09d00b1a50 msgr2=0x7f09d00ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:51.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.943+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 shutdown_connections 2024-04-25T00:01:51.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:51.943+0000 7f09d48cd640 1 -- 172.21.15.113:0/3808428280 wait complete. 2024-04-25T00:01:52.209 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:01:52.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:01:52.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:01:35.811705+0000', 'modified': '2024-04-25T00:01:50.816889+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6690}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6690': {'gid': 6690, 'name': 'd', 'rank': 0, 'incarnation': 34, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.178:6835/1818573977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6833', 'nonce': 1818573977}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 1818573977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T00:01:52.356 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6690:up:rejoin 2024-04-25T00:01:53.357 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:53.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.465+0000 7ff3dc03f640 1 Processor -- start 2024-04-25T00:01:53.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.465+0000 7ff3dc03f640 1 -- start start 2024-04-25T00:01:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3dc03f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff3d4144d60 0x7ff3d40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3dc03f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d4071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3dc03f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d40cfad0 con 0x7ff3d41453c0 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3dc03f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d4145c40 con 0x7ff3d4144d60 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3dc03f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d40cf230 con 0x7ff3d406e6e0 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3d9ddb640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff3d4144d60 0x7ff3d40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3da5dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d4071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3da5dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d4071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:60932/0 (socket says 172.21.15.113:60932) 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3d9ddb640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff3d4144d60 0x7ff3d40b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55860/0 (socket says 172.21.15.113:55860) 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.466+0000 7ff3da5dc640 1 -- 172.21.15.113:0/1697605020 learned_addr learned my addr 172.21.15.113:0/1697605020 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.467+0000 7ff3d9ddb640 1 -- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 msgr2=0x7ff3d4071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.467+0000 7ff3d9ddb640 1 --2- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d4071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.467+0000 7ff3d9ddb640 1 -- 172.21.15.113:0/1697605020 >> v1:172.21.15.178:6789/0 conn(0x7ff3d41453c0 legacy=0x7ff3d406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.467+0000 7ff3d9ddb640 1 -- 172.21.15.113:0/1697605020 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d40cf510 con 0x7ff3d4144d60 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.467+0000 7ff3d8dd9640 1 -- 172.21.15.113:0/1697605020 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649306713 0 0) 0x7ff3d40cfad0 con 0x7ff3d41453c0 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.467+0000 7ff3da5dc640 1 --2- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d4071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.467+0000 7ff3d9ddb640 1 --2- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d4144d60 0x7ff3d40b5390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00097a0 tx=0x7ff3d002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4390aea511323699 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3d8dd9640 1 -- 172.21.15.113:0/1697605020 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0037070 con 0x7ff3d4144d60 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3d8dd9640 1 -- 172.21.15.113:0/1697605020 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d002ae50 con 0x7ff3d4144d60 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3d8dd9640 1 -- 172.21.15.113:0/1697605020 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0032660 con 0x7ff3d4144d60 2024-04-25T00:01:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 -- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d4144d60 msgr2=0x7ff3d40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d4144d60 0x7ff3d40b5390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00097a0 tx=0x7ff3d002a100 comp rx=0 tx=0).stop 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 -- 172.21.15.113:0/1697605020 shutdown_connections 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d4071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/1697605020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d4144d60 0x7ff3d40b5390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 -- 172.21.15.113:0/1697605020 >> 172.21.15.113:0/1697605020 conn(0x7ff3d40b1a50 msgr2=0x7ff3d40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 -- 172.21.15.113:0/1697605020 shutdown_connections 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.468+0000 7ff3dc03f640 1 -- 172.21.15.113:0/1697605020 wait complete. 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.469+0000 7ff3dc03f640 1 Processor -- start 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.469+0000 7ff3dc03f640 1 -- start start 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3dc03f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d41368a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3dc03f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4144d60 0x7ff3d4136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3dc03f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d41453c0 0x7ff3d41f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3dc03f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4145140 con 0x7ff3d4144d60 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3dc03f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4070e10 con 0x7ff3d41453c0 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3dc03f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d4071870 con 0x7ff3d406e6e0 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3d95da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4144d60 0x7ff3d4136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3d95da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4144d60 0x7ff3d4136de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44094/0 (socket says 172.21.15.113:44094) 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3d95da640 1 -- 172.21.15.113:0/2713751376 learned_addr learned my addr 172.21.15.113:0/2713751376 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3d9ddb640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d41368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.470+0000 7ff3da5dc640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d41453c0 0x7ff3d41f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3d95da640 1 -- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 msgr2=0x7ff3d41368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3d95da640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d41368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3d95da640 1 -- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d41453c0 msgr2=0x7ff3d41f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3d95da640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d41453c0 0x7ff3d41f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3d95da640 1 -- 172.21.15.113:0/2713751376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d415be70 con 0x7ff3d4144d60 2024-04-25T00:01:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3d95da640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4144d60 0x7ff3d4136de0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c410 tx=0x7ff3c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3ca7fc640 1 -- 172.21.15.113:0/2713751376 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4017070 con 0x7ff3d4144d60 2024-04-25T00:01:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3ca7fc640 1 -- 172.21.15.113:0/2713751376 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c40035f0 con 0x7ff3d4144d60 2024-04-25T00:01:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d415bb90 con 0x7ff3d4144d60 2024-04-25T00:01:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3ca7fc640 1 -- 172.21.15.113:0/2713751376 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40039e0 con 0x7ff3d4144d60 2024-04-25T00:01:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.471+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d4134680 con 0x7ff3d4144d60 2024-04-25T00:01:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.472+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d40691a0 con 0x7ff3d4144d60 2024-04-25T00:01:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.473+0000 7ff3ca7fc640 1 -- 172.21.15.113:0/2713751376 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3c401a070 con 0x7ff3d4144d60 2024-04-25T00:01:53.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.473+0000 7ff3ca7fc640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff3b806be30 0x7ff3b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:53.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.473+0000 7ff3ca7fc640 1 -- 172.21.15.113:0/2713751376 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7ff3c408a2f0 con 0x7ff3d4144d60 2024-04-25T00:01:53.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.473+0000 7ff3d9ddb640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff3b806be30 0x7ff3b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:53.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.473+0000 7ff3d9ddb640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff3b806be30 0x7ff3b806e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0034000 tx=0x7ff3d002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:53.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.477+0000 7ff3ca7fc640 1 -- 172.21.15.113:0/2713751376 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3c4058c40 con 0x7ff3d4144d60 2024-04-25T00:01:53.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.685+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3d40b5a10 con 0x7ff3d4144d60 2024-04-25T00:01:53.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.686+0000 7ff3ca7fc640 1 -- 172.21.15.113:0/2713751376 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5194 (secure 0 0 0) 0x7ff3c4058540 con 0x7ff3d4144d60 2024-04-25T00:01:53.687 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:01:53.687 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3327449639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3327449639},{"type":"v1","addr":"172.21.15.113:6837","nonce":3327449639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:52.819867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6835/1818573977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1818573977},{"type":"v1","addr":"172.21.15.178:6835","nonce":1818573977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6690,"qdb_cluster":[6690]},"id":4}]} 2024-04-25T00:01:53.687 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 37 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff3b806be30 msgr2=0x7ff3b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff3b806be30 0x7ff3b806e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0034000 tx=0x7ff3d002e000 comp rx=0 tx=0).stop 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4144d60 msgr2=0x7ff3d4136de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4144d60 0x7ff3d4136de0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c410 tx=0x7ff3c400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 shutdown_connections 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff3b806be30 0x7ff3b806e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3d41453c0 0x7ff3d41f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4144d60 0x7ff3d4136de0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 --2- 172.21.15.113:0/2713751376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3d406e6e0 0x7ff3d41368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 >> 172.21.15.113:0/2713751376 conn(0x7ff3d40b1a50 msgr2=0x7ff3d406d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:53.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 shutdown_connections 2024-04-25T00:01:53.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:53.688+0000 7ff3dc03f640 1 -- 172.21.15.113:0/2713751376 wait complete. 2024-04-25T00:01:54.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:01:35.811705+0000', 'modified': '2024-04-25T00:01:52.819867+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6690}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6690': {'gid': 6690, 'name': 'd', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.178:6835/1818573977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6833', 'nonce': 1818573977}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 1818573977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6690, 'qdb_cluster': [6690]} 2024-04-25T00:01:54.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:01:54.102 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T00:01:54.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.202+0000 7f0d93a61640 1 Processor -- start 2024-04-25T00:01:54.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.202+0000 7f0d93a61640 1 -- start start 2024-04-25T00:01:54.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d93a61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 0x7f0d8c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d93a61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d8c0cf660 0x7f0d8c15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d93a61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d8c0cfad0 con 0x7f0d8c15c360 2024-04-25T00:01:54.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d93a61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8c0cc960 con 0x7f0d8c0cf660 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d93a61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c0ccc40 con 0x7f0d8c0d2910 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d91ffe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d8c0cf660 0x7f0d8c15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d917fd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0d8c15c360 0x7f0d8c0d2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34214/0 (socket says 172.21.15.113:34214) 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d917fd640 1 -- 172.21.15.113:0/2552637179 learned_addr learned my addr 172.21.15.113:0/2552637179 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d91ffe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d8c0cf660 0x7f0d8c15fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44096/0 (socket says 172.21.15.113:44096) 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2552637179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 msgr2=0x7f0d8c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2552637179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 0x7f0d8c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2552637179 >> v1:172.21.15.113:6789/0 conn(0x7f0d8c15c360 legacy=0x7f0d8c0d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2552637179 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c0d1550 con 0x7f0d8c0cf660 2024-04-25T00:01:54.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.203+0000 7f0d83fff640 1 -- 172.21.15.113:0/2552637179 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637788781 0 0) 0x7f0d8c0cfad0 con 0x7f0d8c15c360 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2552637179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c0cf660 0x7f0d8c15fda0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009d90 tx=0x7f0d8402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6114797d8a3efd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d83fff640 1 -- 172.21.15.113:0/2552637179 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84031600 con 0x7f0d8c0cf660 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d83fff640 1 -- 172.21.15.113:0/2552637179 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d840317a0 con 0x7f0d8c0cf660 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d83fff640 1 -- 172.21.15.113:0/2552637179 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84031b70 con 0x7f0d8c0cf660 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 -- 172.21.15.113:0/2552637179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c0cf660 msgr2=0x7f0d8c15fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 --2- 172.21.15.113:0/2552637179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c0cf660 0x7f0d8c15fda0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009d90 tx=0x7f0d8402a5d0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 -- 172.21.15.113:0/2552637179 shutdown_connections 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 --2- 172.21.15.113:0/2552637179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c0cf660 0x7f0d8c15fda0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 --2- 172.21.15.113:0/2552637179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 0x7f0d8c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 -- 172.21.15.113:0/2552637179 >> 172.21.15.113:0/2552637179 conn(0x7f0d8c0b3d80 msgr2=0x7f0d8c0b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 -- 172.21.15.113:0/2552637179 shutdown_connections 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.204+0000 7f0d93a61640 1 -- 172.21.15.113:0/2552637179 wait complete. 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 Processor -- start 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 -- start start 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8c0cf660 0x7f0d8c0cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 0x7f0d8c0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c15c360 0x7f0d8c0c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0d1a70 con 0x7f0d8c15c360 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0ce990 con 0x7f0d8c0cf660 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d93a61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c1602e0 con 0x7f0d8c0d2910 2024-04-25T00:01:54.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c15c360 0x7f0d8c0c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c15c360 0x7f0d8c0c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44100/0 (socket says 172.21.15.113:44100) 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.206+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1709565458 learned_addr learned my addr 172.21.15.113:0/1709565458 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1709565458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 msgr2=0x7f0d8c0c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 0x7f0d8c0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1709565458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8c0cf660 msgr2=0x7f0d8c0cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d917fd640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8c0cf660 0x7f0d8c0cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8c0cf660 0x7f0d8c0cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1709565458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c0c7de0 con 0x7f0d8c15c360 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c15c360 0x7f0d8c0c7370 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009d90 tx=0x7f0d84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d81ffb640 1 -- 172.21.15.113:0/1709565458 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84040070 con 0x7f0d8c15c360 2024-04-25T00:01:54.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d81ffb640 1 -- 172.21.15.113:0/1709565458 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d84030b60 con 0x7f0d8c15c360 2024-04-25T00:01:54.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c0c80c0 con 0x7f0d8c15c360 2024-04-25T00:01:54.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d81ffb640 1 -- 172.21.15.113:0/1709565458 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8403c420 con 0x7f0d8c15c360 2024-04-25T00:01:54.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.207+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c15dba0 con 0x7f0d8c15c360 2024-04-25T00:01:54.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.209+0000 7f0d81ffb640 1 -- 172.21.15.113:0/1709565458 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d84039040 con 0x7f0d8c15c360 2024-04-25T00:01:54.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.209+0000 7f0d81ffb640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0d7006be30 0x7f0d7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.209+0000 7f0d81ffb640 1 -- 172.21.15.113:0/1709565458 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f0d840ac2d0 con 0x7f0d8c15c360 2024-04-25T00:01:54.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.209+0000 7f0d917fd640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0d7006be30 0x7f0d7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.209+0000 7f0d917fd640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0d7006be30 0x7f0d7006e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000f60 tx=0x7f0d88009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.210+0000 7f0d96768740 1 Processor -- start 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.210+0000 7f0d96768740 1 -- start start 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d96768740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af18cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cb540 0x559af18a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d96768740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559af1803f00 con 0x559af18e8b60 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559af17ac750 con 0x559af18cbc80 2024-04-25T00:01:54.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559af17ffec0 con 0x559af18cb540 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44102/0 (socket says 172.21.15.113:44102) 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 -- 172.21.15.113:0/3104627040 learned_addr learned my addr 172.21.15.113:0/3104627040 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d90ffc640 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af18cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cb540 0x559af18a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 -- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cb540 msgr2=0x559af18a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cb540 0x559af18a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 -- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 msgr2=0x559af18cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af18cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 -- 172.21.15.113:0/3104627040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af15182d0 con 0x559af18e8b60 2024-04-25T00:01:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d90ffc640 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af18cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cb540 0x559af18a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d917fd640 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189b4f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0d880074f0 tx=0x7f0d88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70beecc64a5bc0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d76ffd640 1 -- 172.21.15.113:0/3104627040 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800da70 con 0x559af18e8b60 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d76ffd640 1 -- 172.21.15.113:0/3104627040 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8800dc10 con 0x559af18e8b60 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.211+0000 7f0d76ffd640 1 -- 172.21.15.113:0/3104627040 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88020450 con 0x559af18e8b60 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 -- 172.21.15.113:0/3104627040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 msgr2=0x559af189b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189b4f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0d880074f0 tx=0x7f0d88010000 comp rx=0 tx=0).stop 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 -- 172.21.15.113:0/3104627040 shutdown_connections 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af18cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cb540 0x559af18a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 --2- 172.21.15.113:0/3104627040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189b4f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 -- 172.21.15.113:0/3104627040 >> 172.21.15.113:0/3104627040 conn(0x559af189d760 msgr2=0x559af18e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 -- 172.21.15.113:0/3104627040 shutdown_connections 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.212+0000 7f0d96768740 1 -- 172.21.15.113:0/3104627040 wait complete. 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.213+0000 7f0d96768740 1 -- start start 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.213+0000 7f0d96768740 1 Processor -- start 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.213+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af18c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18e8b60 0x559af18fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18a0d60 0x559af18a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559af189f910 con 0x559af18a0d60 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559af17bfcf0 con 0x559af18e8b60 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559af189ed00 con 0x559af18cbc80 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 10 client.0 _ll_register_callbacks cb 0x559af17bf6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18a0d60 0x559af18a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18a0d60 0x559af18a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44104/0 (socket says 172.21.15.113:44104) 2024-04-25T00:01:54.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2884933113 learned_addr learned my addr 172.21.15.113:0/2884933113 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d90ffc640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18e8b60 0x559af18fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d917fd640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af18c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 msgr2=0x559af18c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af18c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18e8b60 msgr2=0x559af18fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18e8b60 0x559af18fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2884933113 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af15e26b0 con 0x559af18a0d60 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d917fd640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af18c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18a0d60 0x559af18a11e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0d84082d20 tx=0x7f0d8407a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8914e86ad2fc0670 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d6bfff640 1 -- 172.21.15.113:0/2884933113 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84083250 con 0x559af18a0d60 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d6bfff640 1 -- 172.21.15.113:0/2884933113 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f0d8407a980 con 0x559af18a0d60 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8800dc10 con 0x559af18a0d60 2024-04-25T00:01:54.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d6bfff640 1 -- 172.21.15.113:0/2884933113 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0d8407ab80 con 0x559af18a0d60 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 10 client.6726 fetch_fsmap learned FSMap version 37 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x559af18f77c0 con 0x559af18a0d60 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d6bfff640 1 -- 172.21.15.113:0/2884933113 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8800dc10 con 0x559af18a0d60 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.214+0000 7f0d6bfff640 1 -- 172.21.15.113:0/2884933113 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f0d84082dc0 con 0x559af18a0d60 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d96768740 10 client.6726 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d96768740 10 client.6726 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d96768740 20 client.6726 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d96768740 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18cc170 0x559af18cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d96768740 4 client.6726 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 --> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af1811440 con 0x559af18cc170 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d917fd640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18cc170 0x559af18cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d917fd640 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18cc170 0x559af18cc550 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec22591dd5b52a7 server_cookie=63040c45b174ca66 in_seq=0 out_seq=0 2024-04-25T00:01:54.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.215+0000 7f0d6bfff640 10 client.6726 ms_handle_connect on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:54.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d6bfff640 1 -- 172.21.15.113:0/2884933113 <== mds.0 v2:172.21.15.178:6833/1818573977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559af1811440 con 0x559af18cc170 2024-04-25T00:01:54.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d6bfff640 10 client.6726 handle_command_reply: tid=0 2024-04-25T00:01:54.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 client.6726 shutdown 2024-04-25T00:01:54.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18a0d60 msgr2=0x559af18a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18a0d60 0x559af18a11e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0d84082d20 tx=0x7f0d8407a720 comp rx=0 tx=0).stop 2024-04-25T00:01:54.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 shutdown_connections 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18cc170 0x559af18cc550 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18e8b60 0x559af18fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18a0d60 0x559af18a11e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 --2- 172.21.15.113:0/2884933113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af18c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d6bfff640 0 client.6726 ms_handle_reset on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 >> 172.21.15.113:0/2884933113 conn(0x559af18cb540 msgr2=0x559af18e7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 shutdown_connections 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 1 -- 172.21.15.113:0/2884933113 wait complete. 2024-04-25T00:01:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.216+0000 7f0d96768740 20 client.6726 trim_cache size 0 max 16384 2024-04-25T00:01:54.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 Processor -- start 2024-04-25T00:01:54.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 -- start start 2024-04-25T00:01:54.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18a0d60 0x559af18ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af1899f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1fd640 con 0x559af18e8b60 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c15f010 con 0x559af18a0d60 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.234+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c1fd020 con 0x559af18cbc80 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44116/0 (socket says 172.21.15.113:44116) 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2434382536 learned_addr learned my addr 172.21.15.113:0/2434382536 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d90ffc640 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af1899f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d917fd640 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18a0d60 0x559af18ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 msgr2=0x559af1899f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af1899f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18a0d60 msgr2=0x559af18ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18a0d60 0x559af18ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 -- 172.21.15.113:0/2434382536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af18e9fb0 con 0x559af18e8b60 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d90ffc640 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af1899f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189a4b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0d84076b80 tx=0x7f0d84046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edb2c0d52e76ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d767fc640 1 -- 172.21.15.113:0/2434382536 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84082dc0 con 0x559af18e8b60 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d767fc640 1 -- 172.21.15.113:0/2434382536 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8407fbb0 con 0x559af18e8b60 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.235+0000 7f0d767fc640 1 -- 172.21.15.113:0/2434382536 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8407fda0 con 0x559af18e8b60 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 -- 172.21.15.113:0/2434382536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 msgr2=0x559af189a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189a4b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0d84076b80 tx=0x7f0d84046000 comp rx=0 tx=0).stop 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 -- 172.21.15.113:0/2434382536 shutdown_connections 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af189a4b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18cbc80 0x559af1899f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 --2- 172.21.15.113:0/2434382536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18a0d60 0x559af18ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 -- 172.21.15.113:0/2434382536 >> 172.21.15.113:0/2434382536 conn(0x559af18cb540 msgr2=0x559af18cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 -- 172.21.15.113:0/2434382536 shutdown_connections 2024-04-25T00:01:54.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 -- 172.21.15.113:0/2434382536 wait complete. 2024-04-25T00:01:54.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 -- start start 2024-04-25T00:01:54.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.236+0000 7f0d96768740 1 Processor -- start 2024-04-25T00:01:54.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18a0d60 0x559af18deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d96768740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af189efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d96768740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af18dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d96768740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559af18f8060 con 0x559af18e8b60 2024-04-25T00:01:54.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559af18f7aa0 con 0x559af18cbc80 2024-04-25T00:01:54.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d96768740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559af18f9380 con 0x559af18a0d60 2024-04-25T00:01:54.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d96768740 10 client.0 _ll_register_callbacks cb 0x7f0d8407a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:54.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af18dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d91ffe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af18dc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44118/0 (socket says 172.21.15.113:44118) 2024-04-25T00:01:54.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1941730781 learned_addr learned my addr 172.21.15.113:0/1941730781 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:54.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d90ffc640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af189efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d917fd640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18a0d60 0x559af18deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18a0d60 msgr2=0x559af18deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18a0d60 0x559af18deb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 msgr2=0x559af189efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.237+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af189efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d91ffe640 1 -- 172.21.15.113:0/1941730781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af18a0560 con 0x559af18e8b60 2024-04-25T00:01:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d90ffc640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af189efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d917fd640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18a0d60 0x559af18deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d91ffe640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af18dc100 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0d84078c70 tx=0x7f0d840aa700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b270877a29dd5b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d6bfff640 1 -- 172.21.15.113:0/1941730781 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84079d80 con 0x559af18e8b60 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d6bfff640 1 -- 172.21.15.113:0/1941730781 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f0d8407a980 con 0x559af18e8b60 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8407fbb0 con 0x559af18e8b60 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d6bfff640 1 -- 172.21.15.113:0/1941730781 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0d84076470 con 0x559af18e8b60 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 10 client.6732 fetch_fsmap learned FSMap version 37 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x559af18a0560 con 0x559af18e8b60 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d6bfff640 1 -- 172.21.15.113:0/1941730781 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8407fbb0 con 0x559af18e8b60 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d6bfff640 1 -- 172.21.15.113:0/1941730781 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f0d84076610 con 0x559af18e8b60 2024-04-25T00:01:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 10 client.6732 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T00:01:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 10 client.6732 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:01:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 20 client.6732 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18e62e0 0x559af18e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 4 client.6732 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 --> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] -- command(tid 0: {"prefix": "status"}) -- 0x559af180b790 con 0x559af18e62e0 2024-04-25T00:01:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.238+0000 7f0d917fd640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18e62e0 0x559af18e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d6bfff640 10 client.6732 ms_handle_connect on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d917fd640 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18e62e0 0x559af18e66c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61905e0f1409e04 server_cookie=f8f7e7b5d33eee20 in_seq=0 out_seq=0 2024-04-25T00:01:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d6bfff640 1 -- 172.21.15.113:0/1941730781 <== mds.0 v2:172.21.15.178:6833/1818573977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x559af180b790 con 0x559af18e62e0 2024-04-25T00:01:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d6bfff640 10 client.6732 handle_command_reply: tid=0 2024-04-25T00:01:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 client.6732 shutdown 2024-04-25T00:01:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 msgr2=0x559af18dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af18dc100 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0d84078c70 tx=0x7f0d840aa700 comp rx=0 tx=0).stop 2024-04-25T00:01:54.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 shutdown_connections 2024-04-25T00:01:54.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x559af18e62e0 0x559af18e66c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559af18cbc80 0x559af189efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af18e8b60 0x559af18dc100 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 --2- 172.21.15.113:0/1941730781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559af18a0d60 0x559af18deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d6bfff640 0 client.6732 ms_handle_reset on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:54.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 >> 172.21.15.113:0/1941730781 conn(0x559af18cb540 msgr2=0x559af1899f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 shutdown_connections 2024-04-25T00:01:54.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 1 -- 172.21.15.113:0/1941730781 wait complete. 2024-04-25T00:01:54.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.239+0000 7f0d96768740 20 client.6732 trim_cache size 0 max 16384 2024-04-25T00:01:54.252 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 6690, 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.426089964, 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 37, 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 64, 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 64, 2024-04-25T00:01:54.253 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 4.6002904759999996 2024-04-25T00:01:54.254 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:54.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0d7006be30 msgr2=0x7f0d7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0d7006be30 0x7f0d7006e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d88000f60 tx=0x7f0d88009040 comp rx=0 tx=0).stop 2024-04-25T00:01:54.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c15c360 msgr2=0x7f0d8c0c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c15c360 0x7f0d8c0c7370 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009d90 tx=0x7f0d84004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:54.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 shutdown_connections 2024-04-25T00:01:54.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0d7006be30 0x7f0d7006e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d8c15c360 0x7f0d8c0c7370 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d8c0d2910 0x7f0d8c0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 --2- 172.21.15.113:0/1709565458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8c0cf660 0x7f0d8c0cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 >> 172.21.15.113:0/1709565458 conn(0x7f0d8c0b3d80 msgr2=0x7f0d8c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 shutdown_connections 2024-04-25T00:01:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.240+0000 7f0d93a61640 1 -- 172.21.15.113:0/1709565458 wait complete. 2024-04-25T00:01:54.256 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:01:54.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.352+0000 7f4e0defc640 1 Processor -- start 2024-04-25T00:01:54.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.352+0000 7f4e0defc640 1 -- start start 2024-04-25T00:01:54.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e0defc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e08144d60 0x7f4e080b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e0defc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4e081453c0 0x7f4e0806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e0defc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e080cfad0 con 0x7f4e0806e510 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e0defc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4e08145c40 con 0x7f4e08144d60 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e0defc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4e0806e920 con 0x7f4e081453c0 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e077fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e08144d60 0x7f4e080b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e077fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e08144d60 0x7f4e080b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44124/0 (socket says 172.21.15.113:44124) 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e077fe640 1 -- 172.21.15.113:0/478209066 learned_addr learned my addr 172.21.15.113:0/478209066 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e077fe640 1 -- 172.21.15.113:0/478209066 >> v2:172.21.15.178:3300/0 conn(0x7f4e081453c0 msgr2=0x7f4e0806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e077fe640 1 --2- 172.21.15.113:0/478209066 >> v2:172.21.15.178:3300/0 conn(0x7f4e081453c0 0x7f4e0806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e077fe640 1 -- 172.21.15.113:0/478209066 >> v1:172.21.15.178:6789/0 conn(0x7f4e0806e510 legacy=0x7f4e0815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.353+0000 7f4e077fe640 1 -- 172.21.15.113:0/478209066 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e080cf230 con 0x7f4e08144d60 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.354+0000 7f4e077fe640 1 --2- 172.21.15.113:0/478209066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e08144d60 0x7f4e080b5390 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4e080ce060 tx=0x7f4df802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f1a985619c6461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.354+0000 7f4e067fc640 1 -- 172.21.15.113:0/478209066 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8037070 con 0x7f4e08144d60 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.354+0000 7f4e067fc640 1 -- 172.21.15.113:0/478209066 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df8032440 con 0x7f4e08144d60 2024-04-25T00:01:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.354+0000 7f4e067fc640 1 -- 172.21.15.113:0/478209066 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8032810 con 0x7f4e08144d60 2024-04-25T00:01:54.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 -- 172.21.15.113:0/478209066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e08144d60 msgr2=0x7f4e080b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 --2- 172.21.15.113:0/478209066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e08144d60 0x7f4e080b5390 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4e080ce060 tx=0x7f4df802a1a0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 -- 172.21.15.113:0/478209066 shutdown_connections 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 --2- 172.21.15.113:0/478209066 >> v2:172.21.15.178:3300/0 conn(0x7f4e081453c0 0x7f4e0806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 --2- 172.21.15.113:0/478209066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e08144d60 0x7f4e080b5390 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 -- 172.21.15.113:0/478209066 >> 172.21.15.113:0/478209066 conn(0x7f4e080b1a50 msgr2=0x7f4e080b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 -- 172.21.15.113:0/478209066 shutdown_connections 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.355+0000 7f4e0defc640 1 -- 172.21.15.113:0/478209066 wait complete. 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.356+0000 7f4e0defc640 1 Processor -- start 2024-04-25T00:01:54.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.356+0000 7f4e0defc640 1 -- start start 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e0defc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e0806e510 0x7f4e0813eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e0defc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e08144d60 0x7f4e081e8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e0defc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e081453c0 0x7f4e081e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e0defc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08070e10 con 0x7f4e081453c0 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e0defc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e080cf7f0 con 0x7f4e08144d60 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e0defc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4e08145140 con 0x7f4e0806e510 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e081453c0 0x7f4e081e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e081453c0 0x7f4e081e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44134/0 (socket says 172.21.15.113:44134) 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 -- 172.21.15.113:0/4037924815 learned_addr learned my addr 172.21.15.113:0/4037924815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e077fe640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e0806e510 0x7f4e0813eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e06ffd640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e08144d60 0x7f4e081e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 -- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e0806e510 msgr2=0x7f4e0813eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e0806e510 0x7f4e0813eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 -- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e08144d60 msgr2=0x7f4e081e8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e08144d60 0x7f4e081e8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 -- 172.21.15.113:0/4037924815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e081601d0 con 0x7f4e081453c0 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e077fe640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e0806e510 0x7f4e0813eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e06ffd640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e08144d60 0x7f4e081e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.357+0000 7f4e07fff640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e081453c0 0x7f4e081e6fd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b9e0 tx=0x7f4e0000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.358+0000 7f4deffff640 1 -- 172.21.15.113:0/4037924815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00019070 con 0x7f4e081453c0 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.358+0000 7f4deffff640 1 -- 172.21.15.113:0/4037924815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e0000caa0 con 0x7f4e081453c0 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.358+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e081604b0 con 0x7f4e081453c0 2024-04-25T00:01:54.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.358+0000 7f4deffff640 1 -- 172.21.15.113:0/4037924815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00014440 con 0x7f4e081453c0 2024-04-25T00:01:54.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.358+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0815d9c0 con 0x7f4e081453c0 2024-04-25T00:01:54.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.359+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e080691a0 con 0x7f4e081453c0 2024-04-25T00:01:54.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.359+0000 7f4deffff640 1 -- 172.21.15.113:0/4037924815 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e0001c070 con 0x7f4e081453c0 2024-04-25T00:01:54.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.360+0000 7f4deffff640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4df406be30 0x7f4df406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.360+0000 7f4deffff640 1 -- 172.21.15.113:0/4037924815 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f4e0008aaf0 con 0x7f4e081453c0 2024-04-25T00:01:54.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.360+0000 7f4e077fe640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4df406be30 0x7f4df406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.360+0000 7f4e077fe640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4df406be30 0x7f4df406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4df8034000 tx=0x7f4df802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.364+0000 7f4deffff640 1 -- 172.21.15.113:0/4037924815 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e000594d0 con 0x7f4e081453c0 2024-04-25T00:01:54.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.573+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e080b27e0 con 0x7f4e081453c0 2024-04-25T00:01:54.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.573+0000 7f4deffff640 1 -- 172.21.15.113:0/4037924815 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) ==== 53+0+73 (secure 0 0 0) 0x7f4e00058dd0 con 0x7f4e081453c0 2024-04-25T00:01:54.574 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4df406be30 msgr2=0x7f4df406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4df406be30 0x7f4df406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4df8034000 tx=0x7f4df802e000 comp rx=0 tx=0).stop 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e081453c0 msgr2=0x7f4e081e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e081453c0 0x7f4e081e6fd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b9e0 tx=0x7f4e0000bea0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 shutdown_connections 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4df406be30 0x7f4df406e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e08144d60 0x7f4e081e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e081453c0 0x7f4e081e6fd0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 --2- 172.21.15.113:0/4037924815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e0806e510 0x7f4e0813eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.575+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 >> 172.21.15.113:0/4037924815 conn(0x7f4e080b1a50 msgr2=0x7f4e0813f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.576+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 shutdown_connections 2024-04-25T00:01:54.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54.576+0000 7f4e0defc640 1 -- 172.21.15.113:0/4037924815 wait complete. 2024-04-25T00:01:54.586 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:01:54.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:01:54.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:54.587 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:54.603 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42566sec preferred_lft 42566sec 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:54.604 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:01:54.605 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:01:54.605 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:01:54.605 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:54.605 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:01:54.605 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:54.605 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:01:54.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:54.607 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:01:54.665 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:01:54.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:54.666 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:01:54.723 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:01:54.723 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:01:54.723 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:01:54.723 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:01:54.723 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:01:54.724 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:01:54.724 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:01:54.724 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:01:54.724 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:01:54.724 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:01:54.724 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:01:54.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:01:54.820 INFO:tasks.ceph.mds.d.smithi178.stderr:2024-04-25T00:01:54.820+0000 7f7758728640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:01:54.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:54.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:54.847 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:01:54.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:01:54.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:01:54.862 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:01:54.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:54.862 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:01:54.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:01:54.927 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:54.981 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:01:54.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:01:54.982 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:55.036 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:01:55.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:55.037 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:55.118 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:55.118 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:01:55.118 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:01:55.126 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:55.126 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:01:55.126 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:01:55.126 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:01:55.126 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:01:55.131 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:01:55.132 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:01:55.133 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:01:55.133 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:01:55.133 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:01:55.134 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:01:55.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:55.134 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:01:55.198 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:01:55.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:01:55.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:01:55.293 INFO:teuthology.orchestra.run.smithi113.stdout:client5468 2024-04-25T00:01:55.294 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:01:55.294 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:01:55.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:55.309 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T00:01:55.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:01:55.366 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T00:01:55.367 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T00:01:55.367 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T00:01:55.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.507+0000 7f49f04d6640 1 Processor -- start 2024-04-25T00:01:55.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.507+0000 7f49f04d6640 1 -- start start 2024-04-25T00:01:55.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49f04d6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49e815e510 0x7f49e8161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49f04d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b9bb0 0x7f49e80b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49f04d6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f49e80b61e0 0x7f49e8071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49f04d6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f49e81626f0 con 0x7f49e815e510 2024-04-25T00:01:55.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49f04d6640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f49e81629d0 con 0x7f49e80b61e0 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49f04d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49e8162cb0 con 0x7f49e80b9bb0 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49ee272640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49e815e510 0x7f49e8161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49ee272640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49e815e510 0x7f49e8161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44154/0 (socket says 172.21.15.113:44154) 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.508+0000 7f49ee272640 1 -- 172.21.15.113:0/3975667224 learned_addr learned my addr 172.21.15.113:0/3975667224 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49eea73640 1 --2- 172.21.15.113:0/3975667224 >> v2:172.21.15.178:3300/0 conn(0x7f49e80b61e0 0x7f49e8071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49eda71640 1 --2- 172.21.15.113:0/3975667224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b9bb0 0x7f49e80b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49ee272640 1 -- 172.21.15.113:0/3975667224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b9bb0 msgr2=0x7f49e80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49ee272640 1 --2- 172.21.15.113:0/3975667224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b9bb0 0x7f49e80b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49ee272640 1 -- 172.21.15.113:0/3975667224 >> v2:172.21.15.178:3300/0 conn(0x7f49e80b61e0 msgr2=0x7f49e8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49ee272640 1 --2- 172.21.15.113:0/3975667224 >> v2:172.21.15.178:3300/0 conn(0x7f49e80b61e0 0x7f49e8071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49ee272640 1 -- 172.21.15.113:0/3975667224 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80d2660 con 0x7f49e815e510 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49eea73640 1 --2- 172.21.15.113:0/3975667224 >> v2:172.21.15.178:3300/0 conn(0x7f49e80b61e0 0x7f49e8071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:55.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49eda71640 1 --2- 172.21.15.113:0/3975667224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b9bb0 0x7f49e80b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49ee272640 1 --2- 172.21.15.113:0/3975667224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e8161ea0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f49e80ceb70 tx=0x7f49e402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2522518f4bfc87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.509+0000 7f49ed270640 1 -- 172.21.15.113:0/3975667224 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4036070 con 0x7f49e815e510 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49ed270640 1 -- 172.21.15.113:0/3975667224 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e4031600 con 0x7f49e815e510 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49ed270640 1 -- 172.21.15.113:0/3975667224 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40319d0 con 0x7f49e815e510 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 -- 172.21.15.113:0/3975667224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 msgr2=0x7f49e8161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 --2- 172.21.15.113:0/3975667224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e8161ea0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f49e80ceb70 tx=0x7f49e402a610 comp rx=0 tx=0).stop 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 -- 172.21.15.113:0/3975667224 shutdown_connections 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 --2- 172.21.15.113:0/3975667224 >> v2:172.21.15.178:3300/0 conn(0x7f49e80b61e0 0x7f49e8071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 --2- 172.21.15.113:0/3975667224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b9bb0 0x7f49e80b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 --2- 172.21.15.113:0/3975667224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e8161ea0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 -- 172.21.15.113:0/3975667224 >> 172.21.15.113:0/3975667224 conn(0x7f49e806b720 msgr2=0x7f49e80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 -- 172.21.15.113:0/3975667224 shutdown_connections 2024-04-25T00:01:55.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.510+0000 7f49f04d6640 1 -- 172.21.15.113:0/3975667224 wait complete. 2024-04-25T00:01:55.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.511+0000 7f49f04d6640 1 Processor -- start 2024-04-25T00:01:55.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.511+0000 7f49f04d6640 1 -- start start 2024-04-25T00:01:55.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49f04d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b61e0 0x7f49e80cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49f04d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e80b9bb0 0x7f49e80ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49f04d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e815fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49f04d6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80b73c0 con 0x7f49e815e510 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49f04d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81623e0 con 0x7f49e80b9bb0 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49f04d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49e80b7540 con 0x7f49e80b61e0 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49eea73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e815fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49eea73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e815fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44156/0 (socket says 172.21.15.113:44156) 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49eea73640 1 -- 172.21.15.113:0/734109617 learned_addr learned my addr 172.21.15.113:0/734109617 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49eda71640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e80b9bb0 0x7f49e80ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.512+0000 7f49ee272640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b61e0 0x7f49e80cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49eea73640 1 -- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b61e0 msgr2=0x7f49e80cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49eea73640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b61e0 0x7f49e80cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49eea73640 1 -- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e80b9bb0 msgr2=0x7f49e80ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49eea73640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e80b9bb0 0x7f49e80ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49eea73640 1 -- 172.21.15.113:0/734109617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80c94d0 con 0x7f49e815e510 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49eea73640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e815fc40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f49e000afd0 tx=0x7f49e000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49deffd640 1 -- 172.21.15.113:0/734109617 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0015020 con 0x7f49e815e510 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49deffd640 1 -- 172.21.15.113:0/734109617 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0004440 con 0x7f49e815e510 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49deffd640 1 -- 172.21.15.113:0/734109617 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0003850 con 0x7f49e815e510 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e80c97b0 con 0x7f49e815e510 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.513+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e80c9d70 con 0x7f49e815e510 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.514+0000 7f49deffd640 1 -- 172.21.15.113:0/734109617 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49e0018020 con 0x7f49e815e510 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.515+0000 7f49deffd640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49cc06be60 0x7f49cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.515+0000 7f49deffd640 1 -- 172.21.15.113:0/734109617 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f49e008af70 con 0x7f49e815e510 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.515+0000 7f49ee272640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49cc06be60 0x7f49cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.515+0000 7f49ee272640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49cc06be60 0x7f49cc06e320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49e4033000 tx=0x7f49e4009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.516+0000 7f49f31e2740 1 Processor -- start 2024-04-25T00:01:55.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.516+0000 7f49f31e2740 1 -- start start 2024-04-25T00:01:55.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367868e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678995f0 0x563367898970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563367898eb0 0x56336786dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49f31e2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5633677d1690 con 0x5633678b64d0 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56336776fd20 con 0x563367898eb0 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5633677cd650 con 0x5633678995f0 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367868e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367868e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44170/0 (socket says 172.21.15.113:44170) 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 -- 172.21.15.113:0/1965972565 learned_addr learned my addr 172.21.15.113:0/1965972565 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 -- 172.21.15.113:0/1965972565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678995f0 msgr2=0x563367898970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 --2- 172.21.15.113:0/1965972565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678995f0 0x563367898970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 -- 172.21.15.113:0/1965972565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563367898eb0 msgr2=0x56336786dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 --2- 172.21.15.113:0/1965972565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563367898eb0 0x56336786dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 -- 172.21.15.113:0/1965972565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633675b1af0 con 0x5633678b64d0 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49ee272640 1 --2- 172.21.15.113:0/1965972565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367868e60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f49e40317f0 tx=0x7f49e402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1026aae474a0a5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49cbfff640 1 -- 172.21.15.113:0/1965972565 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4036040 con 0x5633678b64d0 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49cbfff640 1 -- 172.21.15.113:0/1965972565 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e404c070 con 0x5633678b64d0 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.517+0000 7f49cbfff640 1 -- 172.21.15.113:0/1965972565 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e403cb70 con 0x5633678b64d0 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 -- 172.21.15.113:0/1965972565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 msgr2=0x563367868e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1965972565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367868e60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f49e40317f0 tx=0x7f49e402e000 comp rx=0 tx=0).stop 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 -- 172.21.15.113:0/1965972565 shutdown_connections 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1965972565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563367898eb0 0x56336786dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1965972565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678995f0 0x563367898970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1965972565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367868e60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 -- 172.21.15.113:0/1965972565 >> 172.21.15.113:0/1965972565 conn(0x56336786b0d0 msgr2=0x5633678b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 -- 172.21.15.113:0/1965972565 shutdown_connections 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.518+0000 7f49f31e2740 1 -- 172.21.15.113:0/1965972565 wait complete. 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.519+0000 7f49f31e2740 1 -- start start 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.519+0000 7f49f31e2740 1 Processor -- start 2024-04-25T00:01:55.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x5633678c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b64d0 0x5633678c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336789b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56336786d280 con 0x5633678995f0 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56336776d770 con 0x5633678b64d0 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56336786c670 con 0x5633678b2d60 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 10 client.0 _ll_register_callbacks cb 0x563367791210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x5633678c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x5633678c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44172/0 (socket says 172.21.15.113:44172) 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 -- 172.21.15.113:0/1916006949 learned_addr learned my addr 172.21.15.113:0/1916006949 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49eda71640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b64d0 0x5633678c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49eea73640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336789b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 -- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 msgr2=0x56336789b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336789b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 -- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b64d0 msgr2=0x5633678c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b64d0 0x5633678c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 -- 172.21.15.113:0/1916006949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633674a53e0 con 0x5633678995f0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49eea73640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336789b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49ee272640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x5633678c3a20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f49e402ac00 tx=0x7f49e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca3d7ae1ebe16c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49c97fa640 1 -- 172.21.15.113:0/1916006949 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4057020 con 0x5633678995f0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49c97fa640 1 -- 172.21.15.113:0/1916006949 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f49e403cd80 con 0x5633678995f0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e404c070 con 0x5633678995f0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49c97fa640 1 -- 172.21.15.113:0/1916006949 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f49e4061050 con 0x5633678995f0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 10 client.6750 fetch_fsmap learned FSMap version 37 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.520+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x563367612280 con 0x5633678995f0 2024-04-25T00:01:55.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49c97fa640 1 -- 172.21.15.113:0/1916006949 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e404c070 con 0x5633678995f0 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49c97fa640 1 -- 172.21.15.113:0/1916006949 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f49e4064020 con 0x5633678995f0 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49f31e2740 10 client.6750 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49f31e2740 10 client.6750 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49f31e2740 20 client.6750 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x5633678942c0 0x5633678b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49f31e2740 4 client.6750 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 --> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633677e5080 con 0x5633678942c0 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49eda71640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x5633678942c0 0x5633678b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49eda71640 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x5633678942c0 0x5633678b50b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8d8a7ac4c80b01 server_cookie=14d840c2afa6d867 in_seq=0 out_seq=0 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.521+0000 7f49c97fa640 10 client.6750 ms_handle_connect on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49c97fa640 1 -- 172.21.15.113:0/1916006949 <== mds.0 v2:172.21.15.178:6833/1818573977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633677e5080 con 0x5633678942c0 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49c97fa640 10 client.6750 handle_command_reply: tid=0 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 client.6750 shutdown 2024-04-25T00:01:55.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 msgr2=0x5633678c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x5633678c3a20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f49e402ac00 tx=0x7f49e4004ab0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 shutdown_connections 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x5633678942c0 0x5633678b50b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b64d0 0x5633678c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336789b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 --2- 172.21.15.113:0/1916006949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x5633678c3a20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49c97fa640 0 client.6750 ms_handle_reset on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.522+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 >> 172.21.15.113:0/1916006949 conn(0x563367898eb0 msgr2=0x5633678b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.523+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 shutdown_connections 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.523+0000 7f49f31e2740 1 -- 172.21.15.113:0/1916006949 wait complete. 2024-04-25T00:01:55.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.523+0000 7f49f31e2740 20 client.6750 trim_cache size 0 max 16384 2024-04-25T00:01:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.540+0000 7f49f31e2740 1 Processor -- start 2024-04-25T00:01:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.540+0000 7f49f31e2740 1 -- start start 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678995f0 0x5633678b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336786c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367867e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49f31e2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49e82008b0 con 0x5633678b64d0 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80c89f0 con 0x5633678995f0 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49e80c7d50 con 0x5633678b2d60 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367867e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367867e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44178/0 (socket says 172.21.15.113:44178) 2024-04-25T00:01:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 -- 172.21.15.113:0/2903136646 learned_addr learned my addr 172.21.15.113:0/2903136646 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49ee272640 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678995f0 0x5633678b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eda71640 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336786c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 -- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 msgr2=0x56336786c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336786c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 -- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678995f0 msgr2=0x5633678b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678995f0 0x5633678b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 -- 172.21.15.113:0/2903136646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56336747e560 con 0x5633678b64d0 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49eea73640 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367867e20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f49e000cfd0 tx=0x7f49e005d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2202e43351de37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49cb7fe640 1 -- 172.21.15.113:0/2903136646 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0087070 con 0x5633678b64d0 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49cb7fe640 1 -- 172.21.15.113:0/2903136646 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e00710d0 con 0x5633678b64d0 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.541+0000 7f49cb7fe640 1 -- 172.21.15.113:0/2903136646 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0021d70 con 0x5633678b64d0 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 -- 172.21.15.113:0/2903136646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 msgr2=0x563367867e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367867e20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f49e000cfd0 tx=0x7f49e005d4d0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 -- 172.21.15.113:0/2903136646 shutdown_connections 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678b64d0 0x563367867e20 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b2d60 0x56336786c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2903136646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678995f0 0x5633678b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 -- 172.21.15.113:0/2903136646 >> 172.21.15.113:0/2903136646 conn(0x563367898eb0 msgr2=0x563367868e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 -- 172.21.15.113:0/2903136646 shutdown_connections 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 -- 172.21.15.113:0/2903136646 wait complete. 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.542+0000 7f49f31e2740 1 -- start start 2024-04-25T00:01:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 1 Processor -- start 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x56336786e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b2d60 0x56336790d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b64d0 0x56336786c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80c83a0 con 0x5633678995f0 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56336790db60 con 0x5633678b2d60 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5633678d7500 con 0x5633678b64d0 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49f31e2740 10 client.0 _ll_register_callbacks cb 0x7f49e4009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.543+0000 7f49ee272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x56336786e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x56336786e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44190/0 (socket says 172.21.15.113:44190) 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 -- 172.21.15.113:0/2573895763 learned_addr learned my addr 172.21.15.113:0/2573895763 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49eea73640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b64d0 0x56336786c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49eda71640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b2d60 0x56336790d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 -- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b64d0 msgr2=0x56336786c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b64d0 0x56336786c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 -- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b2d60 msgr2=0x56336790d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b2d60 0x56336790d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 -- 172.21.15.113:0/2573895763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633679278e0 con 0x5633678995f0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49eea73640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b64d0 0x56336786c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49ee272640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x56336786e270 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000f80 tx=0x7f49e402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e55254af6d5bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49c97fa640 1 -- 172.21.15.113:0/2573895763 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e406a060 con 0x5633678995f0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49c97fa640 1 -- 172.21.15.113:0/2573895763 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f49e4004540 con 0x5633678995f0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e00710d0 con 0x5633678995f0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49c97fa640 1 -- 172.21.15.113:0/2573895763 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f49e4070050 con 0x5633678995f0 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 10 client.6756 fetch_fsmap learned FSMap version 37 2024-04-25T00:01:55.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x5633679325e0 con 0x5633678995f0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49c97fa640 1 -- 172.21.15.113:0/2573895763 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e00710d0 con 0x5633678995f0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49c97fa640 1 -- 172.21.15.113:0/2573895763 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f49e4073020 con 0x5633678995f0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 10 client.6756 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 10 client.6756 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 20 client.6756 populate_metadata read hostname 'smithi113' 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x56336786cdc0 0x5633678d64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 4 client.6756 mds_command: new command op to 6690 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.544+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 --> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) -- 0x5633677debd0 con 0x56336786cdc0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.545+0000 7f49eda71640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x56336786cdc0 0x5633678d64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.545+0000 7f49c97fa640 10 client.6756 ms_handle_connect on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.545+0000 7f49eda71640 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x56336786cdc0 0x5633678d64a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf83db827590944 server_cookie=caa31ed38bf083fb in_seq=0 out_seq=0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49c97fa640 1 -- 172.21.15.113:0/2573895763 <== mds.0 v2:172.21.15.178:6833/1818573977 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5633677debd0 con 0x56336786cdc0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49c97fa640 10 client.6756 handle_command_reply: tid=0 2024-04-25T00:01:55.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 client.6756 shutdown 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 msgr2=0x56336786e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x56336786e270 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000f80 tx=0x7f49e402f000 comp rx=0 tx=0).stop 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 shutdown_connections 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:6833/1818573977,v1:172.21.15.178:6835/1818573977] conn(0x56336786cdc0 0x5633678d64a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5633678b2d60 0x56336790d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5633678b64d0 0x56336786c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49c97fa640 0 client.6756 ms_handle_reset on v2:172.21.15.178:6833/1818573977 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 --2- 172.21.15.113:0/2573895763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633678995f0 0x56336786e270 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 >> 172.21.15.113:0/2573895763 conn(0x563367898eb0 msgr2=0x5633678b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 shutdown_connections 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 1 -- 172.21.15.113:0/2573895763 wait complete. 2024-04-25T00:01:55.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f31e2740 20 client.6756 trim_cache size 0 max 16384 2024-04-25T00:01:55.553 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:01:55.553 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:01:55.553 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "dd4a10d1-5a0b-41e3-871b-b1ba452b2779", 2024-04-25T00:01:55.553 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:01:55.553 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49cc06be60 msgr2=0x7f49cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49cc06be60 0x7f49cc06e320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49e4033000 tx=0x7f49e4009000 comp rx=0 tx=0).stop 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 msgr2=0x7f49e815fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e815fc40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f49e000afd0 tx=0x7f49e000ca20 comp rx=0 tx=0).stop 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 shutdown_connections 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49cc06be60 0x7f49cc06e320 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e80b9bb0 0x7f49e80ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e815e510 0x7f49e815fc40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 --2- 172.21.15.113:0/734109617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49e80b61e0 0x7f49e80cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.546+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 >> 172.21.15.113:0/734109617 conn(0x7f49e806b720 msgr2=0x7f49e80b89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:55.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.547+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 shutdown_connections 2024-04-25T00:01:55.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:01:55.547+0000 7f49f04d6640 1 -- 172.21.15.113:0/734109617 wait complete. 2024-04-25T00:01:55.557 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': 'dd4a10d1-5a0b-41e3-871b-b1ba452b2779', 'mode': 'asynchronous'}' 2024-04-25T00:02:05.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:05.558 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T00:02:05.580 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:02:05.581 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:02:05.663 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:02:05.705 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:02:05.705 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:02:05.705 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:02:05.705 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:05.723 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:02:05.723 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:05.779 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:05.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.920+0000 7f949646b640 1 Processor -- start 2024-04-25T00:02:05.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.920+0000 7f949646b640 1 -- start start 2024-04-25T00:02:05.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f949646b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9490144d60 0x7f94900b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:05.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f949646b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f949006e6e0 0x7f9490071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:05.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f949646b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94900cfad0 con 0x7f94901453c0 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f949646b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9490145c40 con 0x7f9490144d60 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f949646b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f94900cf230 con 0x7f949006e6e0 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9490144d60 0x7f94900b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f94901453c0 0x7f949006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55220/0 (socket says 172.21.15.113:55220) 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948f7fe640 1 -- 172.21.15.113:0/4193595687 learned_addr learned my addr 172.21.15.113:0/4193595687 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f9494a08640 1 --2- 172.21.15.113:0/4193595687 >> v2:172.21.15.178:3300/0 conn(0x7f949006e6e0 0x7f9490071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948ffff640 1 -- 172.21.15.113:0/4193595687 >> v2:172.21.15.178:3300/0 conn(0x7f949006e6e0 msgr2=0x7f9490071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948ffff640 1 --2- 172.21.15.113:0/4193595687 >> v2:172.21.15.178:3300/0 conn(0x7f949006e6e0 0x7f9490071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948ffff640 1 -- 172.21.15.113:0/4193595687 >> v1:172.21.15.113:6789/0 conn(0x7f94901453c0 legacy=0x7f949006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948ffff640 1 -- 172.21.15.113:0/4193595687 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94900cf510 con 0x7f9490144d60 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f948effd640 1 -- 172.21.15.113:0/4193595687 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909779870 0 0) 0x7f94900cfad0 con 0x7f94901453c0 2024-04-25T00:02:05.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.921+0000 7f9494a08640 1 --2- 172.21.15.113:0/4193595687 >> v2:172.21.15.178:3300/0 conn(0x7f949006e6e0 0x7f9490071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f948ffff640 1 --2- 172.21.15.113:0/4193595687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9490144d60 0x7f94900b5390 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9490145850 tx=0x7f948402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ef89d5f8c7b37a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f948effd640 1 -- 172.21.15.113:0/4193595687 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484037070 con 0x7f9490144d60 2024-04-25T00:02:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f948effd640 1 -- 172.21.15.113:0/4193595687 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9484032440 con 0x7f9490144d60 2024-04-25T00:02:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f948effd640 1 -- 172.21.15.113:0/4193595687 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948402ae60 con 0x7f9490144d60 2024-04-25T00:02:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 -- 172.21.15.113:0/4193595687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9490144d60 msgr2=0x7f94900b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 --2- 172.21.15.113:0/4193595687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9490144d60 0x7f94900b5390 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9490145850 tx=0x7f948402a110 comp rx=0 tx=0).stop 2024-04-25T00:02:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 -- 172.21.15.113:0/4193595687 shutdown_connections 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 --2- 172.21.15.113:0/4193595687 >> v2:172.21.15.178:3300/0 conn(0x7f949006e6e0 0x7f9490071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 --2- 172.21.15.113:0/4193595687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9490144d60 0x7f94900b5390 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 -- 172.21.15.113:0/4193595687 >> 172.21.15.113:0/4193595687 conn(0x7f94900b1a50 msgr2=0x7f94900b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 -- 172.21.15.113:0/4193595687 shutdown_connections 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.922+0000 7f949646b640 1 -- 172.21.15.113:0/4193595687 wait complete. 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 Processor -- start 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 -- start start 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f949006e6e0 0x7f9490151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490144d60 0x7f949015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94901453c0 0x7f949015e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9490145140 con 0x7f949006e6e0 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9490070e10 con 0x7f94901453c0 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.924+0000 7f949646b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9490071870 con 0x7f9490144d60 2024-04-25T00:02:05.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f949006e6e0 0x7f9490151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f949006e6e0 0x7f9490151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44828/0 (socket says 172.21.15.113:44828) 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 -- 172.21.15.113:0/2451678066 learned_addr learned my addr 172.21.15.113:0/2451678066 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f9494a08640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94901453c0 0x7f949015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948f7fe640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490144d60 0x7f949015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 -- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490144d60 msgr2=0x7f949015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490144d60 0x7f949015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 -- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94901453c0 msgr2=0x7f949015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94901453c0 0x7f949015e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 -- 172.21.15.113:0/2451678066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9490151b80 con 0x7f949006e6e0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948ffff640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f949006e6e0 0x7f9490151640 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9484000c00 tx=0x7f94840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948cff9640 1 -- 172.21.15.113:0/2451678066 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484037040 con 0x7f949006e6e0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948cff9640 1 -- 172.21.15.113:0/2451678066 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9484042070 con 0x7f949006e6e0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949015ef80 con 0x7f949006e6e0 2024-04-25T00:02:05.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f948cff9640 1 -- 172.21.15.113:0/2451678066 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484030a50 con 0x7f949006e6e0 2024-04-25T00:02:05.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.925+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949015f540 con 0x7f949006e6e0 2024-04-25T00:02:05.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.926+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94900691a0 con 0x7f949006e6e0 2024-04-25T00:02:05.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.927+0000 7f948cff9640 1 -- 172.21.15.113:0/2451678066 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9484030bf0 con 0x7f949006e6e0 2024-04-25T00:02:05.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.927+0000 7f948cff9640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:05.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.927+0000 7f948cff9640 1 -- 172.21.15.113:0/2451678066 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f94840ab8b0 con 0x7f949006e6e0 2024-04-25T00:02:05.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.927+0000 7f948f7fe640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:05.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.927+0000 7f948f7fe640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f94901f54d0 tx=0x7f9480007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:05.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:05.931+0000 7f948cff9640 1 -- 172.21.15.113:0/2451678066 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9484045360 con 0x7f949006e6e0 2024-04-25T00:02:06.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.106+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94900b27e0 con 0x7f949006e6e0 2024-04-25T00:02:06.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.107+0000 7f948cff9640 1 -- 172.21.15.113:0/2451678066 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16844 (secure 0 0 0) 0x7f948407a290 con 0x7f949006e6e0 2024-04-25T00:02:06.108 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:06.108 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:49.536503+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6834/3704852657":"2024-04-26T00:01:49.531195+0000","172.21.15.178:6832/3704852657":"2024-04-26T00:01:49.531195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:06.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 msgr2=0x7f947c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f94901f54d0 tx=0x7f9480007c00 comp rx=0 tx=0).stop 2024-04-25T00:02:06.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f949006e6e0 msgr2=0x7f9490151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f949006e6e0 0x7f9490151640 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9484000c00 tx=0x7f94840046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 shutdown_connections 2024-04-25T00:02:06.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94901453c0 0x7f949015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490144d60 0x7f949015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 --2- 172.21.15.113:0/2451678066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f949006e6e0 0x7f9490151640 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 >> 172.21.15.113:0/2451678066 conn(0x7f94900b1a50 msgr2=0x7f949013e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 shutdown_connections 2024-04-25T00:02:06.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.110+0000 7f949646b640 1 -- 172.21.15.113:0/2451678066 wait complete. 2024-04-25T00:02:06.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:01:34.014634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:01:34.775167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:02:06.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:02:06.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.220+0000 7f3f16cba640 1 Processor -- start 2024-04-25T00:02:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.220+0000 7f3f16cba640 1 -- start start 2024-04-25T00:02:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f16cba640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3f1015e510 0x7f3f100d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f16cba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100d28c0 0x7f3f100b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f16cba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f100b61e0 0x7f3f10071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f16cba640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3f100d2e00 con 0x7f3f100b61e0 2024-04-25T00:02:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f16cba640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3f100d30e0 con 0x7f3f1015e510 2024-04-25T00:02:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f16cba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3f100d33c0 con 0x7f3f100d28c0 2024-04-25T00:02:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f15257640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f100b61e0 0x7f3f10071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f15257640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f100b61e0 0x7f3f10071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44830/0 (socket says 172.21.15.113:44830) 2024-04-25T00:02:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.221+0000 7f3f15257640 1 -- 172.21.15.113:0/2256896942 learned_addr learned my addr 172.21.15.113:0/2256896942 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f14a56640 1 --2- 172.21.15.113:0/2256896942 >> v2:172.21.15.178:3300/0 conn(0x7f3f1015e510 0x7f3f100d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f15257640 1 -- 172.21.15.113:0/2256896942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100d28c0 msgr2=0x7f3f100b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f0ffff640 1 --2- 172.21.15.113:0/2256896942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100d28c0 0x7f3f100b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f15257640 1 --2- 172.21.15.113:0/2256896942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100d28c0 0x7f3f100b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f15257640 1 -- 172.21.15.113:0/2256896942 >> v2:172.21.15.178:3300/0 conn(0x7f3f1015e510 msgr2=0x7f3f100d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f15257640 1 --2- 172.21.15.113:0/2256896942 >> v2:172.21.15.178:3300/0 conn(0x7f3f1015e510 0x7f3f100d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f15257640 1 -- 172.21.15.113:0/2256896942 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f100d36a0 con 0x7f3f100b61e0 2024-04-25T00:02:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f14a56640 1 --2- 172.21.15.113:0/2256896942 >> v2:172.21.15.178:3300/0 conn(0x7f3f1015e510 0x7f3f100d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f15257640 1 --2- 172.21.15.113:0/2256896942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f100b61e0 0x7f3f10071710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3f100d52a0 tx=0x7f3f0802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadaef0fcba1bb9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.222+0000 7f3f0f7fe640 1 -- 172.21.15.113:0/2256896942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08036070 con 0x7f3f100b61e0 2024-04-25T00:02:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f0f7fe640 1 -- 172.21.15.113:0/2256896942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f08031600 con 0x7f3f100b61e0 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f0f7fe640 1 -- 172.21.15.113:0/2256896942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080319d0 con 0x7f3f100b61e0 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 -- 172.21.15.113:0/2256896942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f100b61e0 msgr2=0x7f3f10071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2256896942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f100b61e0 0x7f3f10071710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3f100d52a0 tx=0x7f3f0802a610 comp rx=0 tx=0).stop 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 -- 172.21.15.113:0/2256896942 shutdown_connections 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2256896942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f100b61e0 0x7f3f10071710 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2256896942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100d28c0 0x7f3f100b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2256896942 >> v2:172.21.15.178:3300/0 conn(0x7f3f1015e510 0x7f3f100d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 -- 172.21.15.113:0/2256896942 >> 172.21.15.113:0/2256896942 conn(0x7f3f1006b720 msgr2=0x7f3f100d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 -- 172.21.15.113:0/2256896942 shutdown_connections 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.223+0000 7f3f16cba640 1 -- 172.21.15.113:0/2256896942 wait complete. 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.224+0000 7f3f16cba640 1 Processor -- start 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.224+0000 7f3f16cba640 1 -- start start 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f16cba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100b61e0 0x7f3f1014d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f16cba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f100d28c0 0x7f3f100c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f16cba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f1015e510 0x7f3f1015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f16cba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f100b9dd0 con 0x7f3f1015e510 2024-04-25T00:02:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f16cba640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f100d3ea0 con 0x7f3f100d28c0 2024-04-25T00:02:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f16cba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3f10071c50 con 0x7f3f100b61e0 2024-04-25T00:02:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f1015e510 0x7f3f1015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f1015e510 0x7f3f1015d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44832/0 (socket says 172.21.15.113:44832) 2024-04-25T00:02:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 -- 172.21.15.113:0/2096057095 learned_addr learned my addr 172.21.15.113:0/2096057095 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f14a56640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100b61e0 0x7f3f1014d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f0ffff640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f100d28c0 0x7f3f100c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 -- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100b61e0 msgr2=0x7f3f1014d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100b61e0 0x7f3f1014d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 -- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f100d28c0 msgr2=0x7f3f100c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f100d28c0 0x7f3f100c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.225+0000 7f3f15257640 1 -- 172.21.15.113:0/2096057095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f100c9800 con 0x7f3f1015e510 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.226+0000 7f3f15257640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f1015e510 0x7f3f1015d1a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3f08034040 tx=0x7f3f08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.226+0000 7f3f0d7fa640 1 -- 172.21.15.113:0/2096057095 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08041020 con 0x7f3f1015e510 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.226+0000 7f3f0d7fa640 1 -- 172.21.15.113:0/2096057095 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f08007780 con 0x7f3f1015e510 2024-04-25T00:02:06.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.226+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f100c9ae0 con 0x7f3f1015e510 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.226+0000 7f3f0d7fa640 1 -- 172.21.15.113:0/2096057095 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08036040 con 0x7f3f1015e510 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.226+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f100ca0a0 con 0x7f3f1015e510 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.227+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f100691a0 con 0x7f3f1015e510 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.227+0000 7f3f0d7fa640 1 -- 172.21.15.113:0/2096057095 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f08044020 con 0x7f3f1015e510 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.228+0000 7f3f0d7fa640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3efc06be30 0x7f3efc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.228+0000 7f3f0d7fa640 1 -- 172.21.15.113:0/2096057095 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f3f080abcd0 con 0x7f3f1015e510 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.228+0000 7f3f14a56640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3efc06be30 0x7f3efc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.228+0000 7f3f14a56640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3efc06be30 0x7f3efc06e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000b840 tx=0x7f3f00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.232+0000 7f3f0d7fa640 1 -- 172.21.15.113:0/2096057095 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f0804b030 con 0x7f3f1015e510 2024-04-25T00:02:06.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.402+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f100d0580 con 0x7f3f1015e510 2024-04-25T00:02:06.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.402+0000 7f3f0d7fa640 1 -- 172.21.15.113:0/2096057095 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) ==== 93+0+10 (secure 0 0 0) 0x7f3f0807a6e0 con 0x7f3f1015e510 2024-04-25T00:02:06.404 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:02:06.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3efc06be30 msgr2=0x7f3efc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3efc06be30 0x7f3efc06e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000b840 tx=0x7f3f00008040 comp rx=0 tx=0).stop 2024-04-25T00:02:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f1015e510 msgr2=0x7f3f1015d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f1015e510 0x7f3f1015d1a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3f08034040 tx=0x7f3f08004a40 comp rx=0 tx=0).stop 2024-04-25T00:02:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 shutdown_connections 2024-04-25T00:02:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3efc06be30 0x7f3efc06e2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f100d28c0 0x7f3f100c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f1015e510 0x7f3f1015d1a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 --2- 172.21.15.113:0/2096057095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f100b61e0 0x7f3f1014d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 >> 172.21.15.113:0/2096057095 conn(0x7f3f1006b720 msgr2=0x7f3f100d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 shutdown_connections 2024-04-25T00:02:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.404+0000 7f3f16cba640 1 -- 172.21.15.113:0/2096057095 wait complete. 2024-04-25T00:02:06.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:02:06.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.514+0000 7efe5619e640 1 Processor -- start 2024-04-25T00:02:06.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.515+0000 7efe5619e640 1 -- start start 2024-04-25T00:02:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.515+0000 7efe5619e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efe5015e240 0x7efe50072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.515+0000 7efe5619e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe50062290 0x7efe5006b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.515+0000 7efe5619e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe500b0400 con 0x7efe5006c810 2024-04-25T00:02:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.515+0000 7efe5619e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efe50071e80 con 0x7efe50062290 2024-04-25T00:02:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.515+0000 7efe5619e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efe50072160 con 0x7efe5015e240 2024-04-25T00:02:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5599d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe50062290 0x7efe5006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5599d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe50062290 0x7efe5006b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44848/0 (socket says 172.21.15.113:44848) 2024-04-25T00:02:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5599d640 1 -- 172.21.15.113:0/117293937 learned_addr learned my addr 172.21.15.113:0/117293937 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5519c640 1 --2- 172.21.15.113:0/117293937 >> v2:172.21.15.178:3300/0 conn(0x7efe5015e240 0x7efe50072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5599d640 1 -- 172.21.15.113:0/117293937 >> v2:172.21.15.178:3300/0 conn(0x7efe5015e240 msgr2=0x7efe50072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5599d640 1 --2- 172.21.15.113:0/117293937 >> v2:172.21.15.178:3300/0 conn(0x7efe5015e240 0x7efe50072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5599d640 1 -- 172.21.15.113:0/117293937 >> v1:172.21.15.178:6789/0 conn(0x7efe5006c810 legacy=0x7efe50061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5599d640 1 -- 172.21.15.113:0/117293937 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe500b06e0 con 0x7efe50062290 2024-04-25T00:02:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.516+0000 7efe5519c640 1 --2- 172.21.15.113:0/117293937 >> v2:172.21.15.178:3300/0 conn(0x7efe5015e240 0x7efe50072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5599d640 1 --2- 172.21.15.113:0/117293937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50062290 0x7efe5006b310 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efe50073280 tx=0x7efe4802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaefb40df29b38c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe47fff640 1 -- 172.21.15.113:0/117293937 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48036070 con 0x7efe50062290 2024-04-25T00:02:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe47fff640 1 -- 172.21.15.113:0/117293937 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe48031600 con 0x7efe50062290 2024-04-25T00:02:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe47fff640 1 -- 172.21.15.113:0/117293937 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe480319d0 con 0x7efe50062290 2024-04-25T00:02:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 -- 172.21.15.113:0/117293937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50062290 msgr2=0x7efe5006b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 --2- 172.21.15.113:0/117293937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50062290 0x7efe5006b310 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efe50073280 tx=0x7efe4802a6f0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 -- 172.21.15.113:0/117293937 shutdown_connections 2024-04-25T00:02:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 --2- 172.21.15.113:0/117293937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50062290 0x7efe5006b310 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 --2- 172.21.15.113:0/117293937 >> v2:172.21.15.178:3300/0 conn(0x7efe5015e240 0x7efe50072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 -- 172.21.15.113:0/117293937 >> 172.21.15.113:0/117293937 conn(0x7efe500d0d60 msgr2=0x7efe500d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 -- 172.21.15.113:0/117293937 shutdown_connections 2024-04-25T00:02:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.517+0000 7efe5619e640 1 -- 172.21.15.113:0/117293937 wait complete. 2024-04-25T00:02:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 Processor -- start 2024-04-25T00:02:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 -- start start 2024-04-25T00:02:06.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe50062290 0x7efe50156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe5006c810 0x7efe50154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe5015e240 0x7efe5014a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe500b0110 con 0x7efe5006c810 2024-04-25T00:02:06.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe500aff90 con 0x7efe5015e240 2024-04-25T00:02:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5619e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe500cec60 con 0x7efe50062290 2024-04-25T00:02:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5499b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe5006c810 0x7efe50154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5499b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe5006c810 0x7efe50154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44856/0 (socket says 172.21.15.113:44856) 2024-04-25T00:02:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.519+0000 7efe5499b640 1 -- 172.21.15.113:0/2797749451 learned_addr learned my addr 172.21.15.113:0/2797749451 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5599d640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe5015e240 0x7efe5014a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5519c640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe50062290 0x7efe50156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5499b640 1 -- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe50062290 msgr2=0x7efe50156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5499b640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe50062290 0x7efe50156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5499b640 1 -- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe5015e240 msgr2=0x7efe5014a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5499b640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe5015e240 0x7efe5014a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5499b640 1 -- 172.21.15.113:0/2797749451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50157860 con 0x7efe5006c810 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5499b640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe5006c810 0x7efe50154400 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c430 tx=0x7efe4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe45ffb640 1 -- 172.21.15.113:0/2797749451 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011790 con 0x7efe5006c810 2024-04-25T00:02:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe45ffb640 1 -- 172.21.15.113:0/2797749451 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c017070 con 0x7efe5006c810 2024-04-25T00:02:06.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe501573e0 con 0x7efe5006c810 2024-04-25T00:02:06.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe45ffb640 1 -- 172.21.15.113:0/2797749451 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011bb0 con 0x7efe5006c810 2024-04-25T00:02:06.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.520+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe50160130 con 0x7efe5006c810 2024-04-25T00:02:06.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.521+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5015fc70 con 0x7efe5006c810 2024-04-25T00:02:06.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.522+0000 7efe45ffb640 1 -- 172.21.15.113:0/2797749451 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe4c01c460 con 0x7efe5006c810 2024-04-25T00:02:06.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.522+0000 7efe45ffb640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe3406be30 0x7efe3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.522+0000 7efe45ffb640 1 -- 172.21.15.113:0/2797749451 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7efe4c08ad90 con 0x7efe5006c810 2024-04-25T00:02:06.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.522+0000 7efe5519c640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe3406be30 0x7efe3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.522+0000 7efe5519c640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe3406be30 0x7efe3406e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efe4000a950 tx=0x7efe40008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.526+0000 7efe45ffb640 1 -- 172.21.15.113:0/2797749451 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe5015fc70 con 0x7efe5006c810 2024-04-25T00:02:06.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.697+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe500691a0 con 0x7efe5006c810 2024-04-25T00:02:06.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.698+0000 7efe45ffb640 1 -- 172.21.15.113:0/2797749451 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) ==== 104+0+11 (secure 0 0 0) 0x7efe4c0596e0 con 0x7efe5006c810 2024-04-25T00:02:06.700 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:02:06.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.699+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe3406be30 msgr2=0x7efe3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.699+0000 7efe5619e640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe3406be30 0x7efe3406e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efe4000a950 tx=0x7efe40008040 comp rx=0 tx=0).stop 2024-04-25T00:02:06.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.699+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe5006c810 msgr2=0x7efe50154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.699+0000 7efe5619e640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe5006c810 0x7efe50154400 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c430 tx=0x7efe4c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 shutdown_connections 2024-04-25T00:02:06.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe3406be30 0x7efe3406e2f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe5015e240 0x7efe5014a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe5006c810 0x7efe50154400 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 --2- 172.21.15.113:0/2797749451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe50062290 0x7efe50156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 >> 172.21.15.113:0/2797749451 conn(0x7efe500d0d60 msgr2=0x7efe500cd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 shutdown_connections 2024-04-25T00:02:06.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.700+0000 7efe5619e640 1 -- 172.21.15.113:0/2797749451 wait complete. 2024-04-25T00:02:06.712 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:02:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.810+0000 7f0195d14640 1 Processor -- start 2024-04-25T00:02:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.810+0000 7f0195d14640 1 -- start start 2024-04-25T00:02:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f0195d14640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f01900b5ca0 0x7f01900b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f0195d14640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900d2e00 con 0x7f01900b73c0 2024-04-25T00:02:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f0195d14640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900d30e0 con 0x7f019015e510 2024-04-25T00:02:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f0195d14640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f01900b6640 con 0x7f01900b5ca0 2024-04-25T00:02:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f018ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f01900b73c0 0x7f01900b77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55226/0 (socket says 172.21.15.113:55226) 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f018ffff640 1 -- 172.21.15.113:0/2045941587 learned_addr learned my addr 172.21.15.113:0/2045941587 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f018effd640 1 --2- 172.21.15.113:0/2045941587 >> v2:172.21.15.178:3300/0 conn(0x7f01900b5ca0 0x7f01900b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080814040 0 0) 0x7f01900d2e00 con 0x7f01900b73c0 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.811+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0178003520 con 0x7f01900b73c0 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2668523426 0 0) 0x7f01900d30e0 con 0x7f019015e510 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01900d2e00 con 0x7f019015e510 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 473814410 0 0) 0x7f0178003520 con 0x7f01900b73c0 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 >> v2:172.21.15.178:3300/0 conn(0x7f01900b5ca0 msgr2=0x7f01900b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 --2- 172.21.15.113:0/2045941587 >> v2:172.21.15.178:3300/0 conn(0x7f01900b5ca0 0x7f01900b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 >> v1:172.21.15.178:6789/0 conn(0x7f019015e510 legacy=0x7f01900d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01900d33c0 con 0x7f01900b73c0 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f01880019f0 con 0x7f01900b73c0 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 >> v1:172.21.15.113:6789/0 conn(0x7f01900b73c0 legacy=0x7f01900b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f0178003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0178004800 0x7f0178004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01780051c0 0x7f017800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f017800c0c0 con 0x7f0178003b40 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f017800c270 con 0x7f01780051c0 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f017800c470 con 0x7f0178004800 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018f7fe640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f0178003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018effd640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0178004800 0x7f0178004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018ffff640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01780051c0 0x7f017800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018f7fe640 1 -- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0178004800 msgr2=0x7f0178004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018f7fe640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0178004800 0x7f0178004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018f7fe640 1 -- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01780051c0 msgr2=0x7f017800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018f7fe640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01780051c0 0x7f017800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018f7fe640 1 -- 172.21.15.113:0/2045941587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0178003520 con 0x7f0178003b40 2024-04-25T00:02:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018effd640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0178004800 0x7f0178004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018ffff640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01780051c0 0x7f017800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.812+0000 7f018ffff640 1 -- 172.21.15.113:0/2045941587 reap_dead start 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f018f7fe640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f0178003f40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f01800018d0 tx=0x7f0178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57723156b6432872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01800014f0 con 0x7f0178003b40 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01800052d0 con 0x7f0178003b40 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f018e7fc640 1 -- 172.21.15.113:0/2045941587 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018002ac70 con 0x7f0178003b40 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f0195d14640 1 -- 172.21.15.113:0/2045941587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 msgr2=0x7f0178003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f0195d14640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f0178003f40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f01800018d0 tx=0x7f0178005830 comp rx=0 tx=0).stop 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f0195d14640 1 -- 172.21.15.113:0/2045941587 shutdown_connections 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f0195d14640 1 --2- 172.21.15.113:0/2045941587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f0178003f40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f0195d14640 1 -- 172.21.15.113:0/2045941587 >> 172.21.15.113:0/2045941587 conn(0x7f019006b720 msgr2=0x7f01900d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.813+0000 7f0195d14640 1 -- 172.21.15.113:0/2045941587 shutdown_connections 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.814+0000 7f0195d14640 1 -- 172.21.15.113:0/2045941587 wait complete. 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.815+0000 7f0195d14640 1 Processor -- start 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.815+0000 7f0195d14640 1 -- start start 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f019014b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01900b3c80 0x7f01900b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01900c8760 0x7f01900c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f01900d3ea0 con 0x7f0178003b40 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01900d3980 con 0x7f01900c8760 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f01900b9dd0 con 0x7f01900b3c80 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f019014b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f019014b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44880/0 (socket says 172.21.15.113:44880) 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 -- 172.21.15.113:0/1510266787 learned_addr learned my addr 172.21.15.113:0/1510266787 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018effd640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01900b3c80 0x7f01900b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018ffff640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01900c8760 0x7f01900c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 -- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01900b3c80 msgr2=0x7f01900b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01900b3c80 0x7f01900b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 -- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01900c8760 msgr2=0x7f01900c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01900c8760 0x7f01900c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 -- 172.21.15.113:0/1510266787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01900c9120 con 0x7f0178003b40 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f018f7fe640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f019014b7c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f01800019f0 tx=0x7f01800012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0177fff640 1 -- 172.21.15.113:0/1510266787 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180040030 con 0x7f0178003b40 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0177fff640 1 -- 172.21.15.113:0/1510266787 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0180005480 con 0x7f0178003b40 2024-04-25T00:02:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01900c93e0 con 0x7f0178003b40 2024-04-25T00:02:06.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0177fff640 1 -- 172.21.15.113:0/1510266787 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01800014f0 con 0x7f0178003b40 2024-04-25T00:02:06.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.816+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01900c99a0 con 0x7f0178003b40 2024-04-25T00:02:06.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.817+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01900691a0 con 0x7f0178003b40 2024-04-25T00:02:06.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.818+0000 7f0177fff640 1 -- 172.21.15.113:0/1510266787 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0180043070 con 0x7f0178003b40 2024-04-25T00:02:06.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.818+0000 7f0177fff640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f017c06be60 0x7f017c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.818+0000 7f0177fff640 1 -- 172.21.15.113:0/1510266787 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f01800aaa10 con 0x7f0178003b40 2024-04-25T00:02:06.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.818+0000 7f018effd640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f017c06be60 0x7f017c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.818+0000 7f018effd640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f017c06be60 0x7f017c06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f01900d2880 tx=0x7f0184008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.822+0000 7f0177fff640 1 -- 172.21.15.113:0/1510266787 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01800793f0 con 0x7f0178003b40 2024-04-25T00:02:06.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.995+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0190155a70 con 0x7f0178003b40 2024-04-25T00:02:06.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.996+0000 7f0177fff640 1 -- 172.21.15.113:0/1510266787 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) ==== 100+0+11 (secure 0 0 0) 0x7f0180078cf0 con 0x7f0178003b40 2024-04-25T00:02:06.997 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:02:06.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f017c06be60 msgr2=0x7f017c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f017c06be60 0x7f017c06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f01900d2880 tx=0x7f0184008040 comp rx=0 tx=0).stop 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 msgr2=0x7f019014b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f019014b7c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f01800019f0 tx=0x7f01800012d0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 shutdown_connections 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f017c06be60 0x7f017c06e320 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01900c8760 0x7f01900c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01900b3c80 0x7f01900b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 --2- 172.21.15.113:0/1510266787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0178003b40 0x7f019014b7c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 >> 172.21.15.113:0/1510266787 conn(0x7f019006b720 msgr2=0x7f01900b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 shutdown_connections 2024-04-25T00:02:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:06.998+0000 7f0195d14640 1 -- 172.21.15.113:0/1510266787 wait complete. 2024-04-25T00:02:07.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:02:07.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.106+0000 7fe72c2cd640 1 Processor -- start 2024-04-25T00:02:07.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.106+0000 7fe72c2cd640 1 -- start start 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72c2cd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7240b3080 0x7fe7240b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72c2cd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe72415e6b0 0x7fe7240b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72c2cd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72415f4f0 con 0x7fe72415e2d0 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72c2cd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe7240cde00 con 0x7fe72415e6b0 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72c2cd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe72415f7d0 con 0x7fe7240b3080 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72a86a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe72415e6b0 0x7fe7240b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe729868640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7240b3080 0x7fe7240b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72a86a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe72415e6b0 0x7fe7240b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:39080/0 (socket says 172.21.15.113:39080) 2024-04-25T00:02:07.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe729868640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7240b3080 0x7fe7240b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:49710/0 (socket says 172.21.15.113:49710) 2024-04-25T00:02:07.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72a86a640 1 -- 172.21.15.113:0/1270410481 learned_addr learned my addr 172.21.15.113:0/1270410481 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:07.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72a86a640 1 -- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7240b3080 msgr2=0x7fe7240b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:07.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72a86a640 1 --2- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7240b3080 0x7fe7240b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72a86a640 1 -- 172.21.15.113:0/1270410481 >> v1:172.21.15.178:6789/0 conn(0x7fe72415e2d0 legacy=0x7fe72415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe72a86a640 1 -- 172.21.15.113:0/1270410481 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72415fab0 con 0x7fe72415e6b0 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe729067640 1 -- 172.21.15.113:0/1270410481 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286249701 0 0) 0x7fe72415f4f0 con 0x7fe72415e2d0 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.107+0000 7fe729868640 1 --2- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7240b3080 0x7fe7240b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.108+0000 7fe72a86a640 1 --2- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe72415e6b0 0x7fe7240b4870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009e50 tx=0x7fe71c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335683409c2caa1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.108+0000 7fe729067640 1 -- 172.21.15.113:0/1270410481 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c036070 con 0x7fe72415e6b0 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.108+0000 7fe729067640 1 -- 172.21.15.113:0/1270410481 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71c031600 con 0x7fe72415e6b0 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.108+0000 7fe729067640 1 -- 172.21.15.113:0/1270410481 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0319d0 con 0x7fe72415e6b0 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 -- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe72415e6b0 msgr2=0x7fe7240b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe72415e6b0 0x7fe7240b4870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009e50 tx=0x7fe71c02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 -- 172.21.15.113:0/1270410481 shutdown_connections 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe72415e6b0 0x7fe7240b4870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/1270410481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7240b3080 0x7fe7240b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 -- 172.21.15.113:0/1270410481 >> 172.21.15.113:0/1270410481 conn(0x7fe72406b0f0 msgr2=0x7fe72406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:07.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 -- 172.21.15.113:0/1270410481 shutdown_connections 2024-04-25T00:02:07.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.109+0000 7fe72c2cd640 1 -- 172.21.15.113:0/1270410481 wait complete. 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.110+0000 7fe72c2cd640 1 Processor -- start 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.110+0000 7fe72c2cd640 1 -- start start 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72c2cd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7240b3080 0x7fe724159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72c2cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72415e2d0 0x7fe72414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72c2cd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe72415e6b0 0x7fe7240adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72c2cd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7240ce410 con 0x7fe72415e2d0 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72c2cd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7240ce290 con 0x7fe7240b3080 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72c2cd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe7240ce590 con 0x7fe72415e6b0 2024-04-25T00:02:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72415e2d0 0x7fe72414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72415e2d0 0x7fe72414dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44886/0 (socket says 172.21.15.113:44886) 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 -- 172.21.15.113:0/654427388 learned_addr learned my addr 172.21.15.113:0/654427388 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72a86a640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe72415e6b0 0x7fe7240adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72a069640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7240b3080 0x7fe724159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 -- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe72415e6b0 msgr2=0x7fe7240adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe72415e6b0 0x7fe7240adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 -- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7240b3080 msgr2=0x7fe724159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7240b3080 0x7fe724159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 -- 172.21.15.113:0/654427388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72414e4f0 con 0x7fe72415e2d0 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72a069640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7240b3080 0x7fe724159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe72a86a640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe72415e6b0 0x7fe7240adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe729868640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72415e2d0 0x7fe72414dfb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9b0 tx=0x7fe71400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.111+0000 7fe71affd640 1 -- 172.21.15.113:0/654427388 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714017070 con 0x7fe72415e2d0 2024-04-25T00:02:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.112+0000 7fe71affd640 1 -- 172.21.15.113:0/654427388 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71400ca00 con 0x7fe72415e2d0 2024-04-25T00:02:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.112+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72414e780 con 0x7fe72415e2d0 2024-04-25T00:02:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.112+0000 7fe71affd640 1 -- 172.21.15.113:0/654427388 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400cdf0 con 0x7fe72415e2d0 2024-04-25T00:02:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.112+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72414ed40 con 0x7fe72415e2d0 2024-04-25T00:02:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.112+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7240691a0 con 0x7fe72415e2d0 2024-04-25T00:02:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.113+0000 7fe71affd640 1 -- 172.21.15.113:0/654427388 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe71401a070 con 0x7fe72415e2d0 2024-04-25T00:02:07.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.113+0000 7fe71affd640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe70806be30 0x7fe70806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.114+0000 7fe72a069640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe70806be30 0x7fe70806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.114+0000 7fe71affd640 1 -- 172.21.15.113:0/654427388 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fe71408aa60 con 0x7fe72415e2d0 2024-04-25T00:02:07.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.114+0000 7fe72a069640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe70806be30 0x7fe70806e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe720000f60 tx=0x7fe720009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.118+0000 7fe71affd640 1 -- 172.21.15.113:0/654427388 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7140593b0 con 0x7fe72415e2d0 2024-04-25T00:02:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.302+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe724072050 con 0x7fe72415e2d0 2024-04-25T00:02:07.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.302+0000 7fe71affd640 1 -- 172.21.15.113:0/654427388 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v64) ==== 80+0+124 (secure 0 0 0) 0x7fe714058cb0 con 0x7fe72415e2d0 2024-04-25T00:02:07.304 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.178:6834/3704852657 2024-04-26T00:01:49.531195+0000 2024-04-25T00:02:07.304 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.178:6832/3704852657 2024-04-26T00:01:49.531195+0000 2024-04-25T00:02:07.304 INFO:teuthology.orchestra.run.smithi113.stderr:listed 2 entries 2024-04-25T00:02:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe70806be30 msgr2=0x7fe70806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe70806be30 0x7fe70806e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe720000f60 tx=0x7fe720009040 comp rx=0 tx=0).stop 2024-04-25T00:02:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72415e2d0 msgr2=0x7fe72414dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72415e2d0 0x7fe72414dfb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9b0 tx=0x7fe71400ee70 comp rx=0 tx=0).stop 2024-04-25T00:02:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 shutdown_connections 2024-04-25T00:02:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe70806be30 0x7fe70806e2f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe72415e6b0 0x7fe7240adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72415e2d0 0x7fe72414dfb0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 --2- 172.21.15.113:0/654427388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7240b3080 0x7fe724159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 >> 172.21.15.113:0/654427388 conn(0x7fe72406b0f0 msgr2=0x7fe72415ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 shutdown_connections 2024-04-25T00:02:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.304+0000 7fe72c2cd640 1 -- 172.21.15.113:0/654427388 wait complete. 2024-04-25T00:02:07.316 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:02:07.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:07.316 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:02:07.345 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:02:07.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:07.345 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:02:07.400 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:02:07.400 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:02:07.400 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:07.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:02:07.419 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:02:07.419 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:02:07.419 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:07.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:02:07.455 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:02:07.455 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:02:07.455 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:07.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:02:07.478 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:02:07.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.590+0000 7ff6aca9a640 1 Processor -- start 2024-04-25T00:02:07.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.590+0000 7ff6aca9a640 1 -- start start 2024-04-25T00:02:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6aca9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a81453c0 0x7ff6a80b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6aca9a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a80cfad0 con 0x7ff6a8144d60 2024-04-25T00:02:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6aca9a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a80cf230 con 0x7ff6a806e6e0 2024-04-25T00:02:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6aca9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a8145c40 con 0x7ff6a81453c0 2024-04-25T00:02:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff6a8144d60 0x7ff6a806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54724/0 (socket says 172.21.15.113:54724) 2024-04-25T00:02:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a659c640 1 -- 172.21.15.113:0/3583198569 learned_addr learned my addr 172.21.15.113:0/3583198569 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657859290 0 0) 0x7ff6a80cfad0 con 0x7ff6a8144d60 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff694003520 con 0x7ff6a8144d60 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a6d9d640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a81453c0 0x7ff6a80b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1011367333 0 0) 0x7ff694003520 con 0x7ff6a8144d60 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a81453c0 msgr2=0x7ff6a80b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a81453c0 0x7ff6a80b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 >> v1:172.21.15.178:6789/0 conn(0x7ff6a806e6e0 legacy=0x7ff6a8071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a80cf510 con 0x7ff6a8144d60 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff698001670 con 0x7ff6a8144d60 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 >> v1:172.21.15.113:6789/0 conn(0x7ff6a8144d60 legacy=0x7ff6a806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6940039e0 0x7ff694003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.591+0000 7ff6a5d9b640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff694004800 0x7ff694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a5d9b640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff694005370 0x7ff69400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8145c40 con 0x7ff694005370 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff69400c280 con 0x7ff6940039e0 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff69400c480 con 0x7ff694004800 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a6d9d640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a81453c0 0x7ff6a80b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a6d9d640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff694005370 0x7ff69400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a659c640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6940039e0 0x7ff694003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff69ffff640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff694004800 0x7ff694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a6d9d640 1 -- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff694004800 msgr2=0x7ff694004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a6d9d640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff694004800 0x7ff694004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a6d9d640 1 -- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6940039e0 msgr2=0x7ff694003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a6d9d640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6940039e0 0x7ff694003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff6a6d9d640 1 -- 172.21.15.113:0/3583198569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff694003520 con 0x7ff694005370 2024-04-25T00:02:07.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.592+0000 7ff69ffff640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff694004800 0x7ff694004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6a6d9d640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff694005370 0x7ff69400b9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0007e20 tx=0x7ff694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8863a349695e2731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0038070 con 0x7ff694005370 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a002b910 con 0x7ff694005370 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6a5d9b640 1 -- 172.21.15.113:0/3583198569 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0007a70 con 0x7ff694005370 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6aca9a640 1 -- 172.21.15.113:0/3583198569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff694005370 msgr2=0x7ff69400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6aca9a640 1 --2- 172.21.15.113:0/3583198569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff694005370 0x7ff69400b9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0007e20 tx=0x7ff694005830 comp rx=0 tx=0).stop 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6a6d9d640 1 -- 172.21.15.113:0/3583198569 reap_dead start 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6aca9a640 1 -- 172.21.15.113:0/3583198569 shutdown_connections 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6aca9a640 1 -- 172.21.15.113:0/3583198569 >> 172.21.15.113:0/3583198569 conn(0x7ff6a80b1a50 msgr2=0x7ff6a80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6aca9a640 1 -- 172.21.15.113:0/3583198569 shutdown_connections 2024-04-25T00:02:07.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.593+0000 7ff6aca9a640 1 -- 172.21.15.113:0/3583198569 wait complete. 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 Processor -- start 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 -- start start 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6a815baa0 0x7ff6a813a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a813abb0 0x7ff6a813e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a813ebf0 0x7ff6a81366e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8145140 con 0x7ff6a813abb0 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8070e10 con 0x7ff6a815baa0 2024-04-25T00:02:07.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff6aca9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a80b5390 con 0x7ff6a813ebf0 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a813abb0 0x7ff6a813e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a813abb0 0x7ff6a813e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40052/0 (socket says 172.21.15.113:40052) 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.595+0000 7ff69ffff640 1 -- 172.21.15.113:0/275612011 learned_addr learned my addr 172.21.15.113:0/275612011 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69ffff640 1 -- 172.21.15.113:0/275612011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a813ebf0 msgr2=0x7ff6a81366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69ffff640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a813ebf0 0x7ff6a81366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69ffff640 1 -- 172.21.15.113:0/275612011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6a815baa0 msgr2=0x7ff6a813a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69ffff640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6a815baa0 0x7ff6a813a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69ffff640 1 -- 172.21.15.113:0/275612011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a813bd40 con 0x7ff6a813abb0 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69ffff640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a813abb0 0x7ff6a813e6b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d520 tx=0x7ff69000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69effd640 1 -- 172.21.15.113:0/275612011 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690005480 con 0x7ff6a813abb0 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69effd640 1 -- 172.21.15.113:0/275612011 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff690017070 con 0x7ff6a813abb0 2024-04-25T00:02:07.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a813b130 con 0x7ff6a813abb0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff69effd640 1 -- 172.21.15.113:0/275612011 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690003850 con 0x7ff6a813abb0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.596+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a813b6f0 con 0x7ff6a813abb0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.597+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a813f010 con 0x7ff6a813abb0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.598+0000 7ff69effd640 1 -- 172.21.15.113:0/275612011 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6900039f0 con 0x7ff6a813abb0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.598+0000 7ff69effd640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff68806be30 0x7ff68806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.598+0000 7ff6a659c640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff68806be30 0x7ff68806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.598+0000 7ff69effd640 1 -- 172.21.15.113:0/275612011 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7ff69008ac90 con 0x7ff6a813abb0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.598+0000 7ff6a659c640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff68806be30 0x7ff68806e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff698005970 tx=0x7ff698001df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.602+0000 7ff69effd640 1 -- 172.21.15.113:0/275612011 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff690059550 con 0x7ff6a813abb0 2024-04-25T00:02:07.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.811+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6a80691a0 con 0x7ff6a813abb0 2024-04-25T00:02:07.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.811+0000 7ff69effd640 1 -- 172.21.15.113:0/275612011 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5194 (secure 0 0 0) 0x7ff690058e50 con 0x7ff6a813abb0 2024-04-25T00:02:07.813 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:07.813 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3327449639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3327449639},{"type":"v1","addr":"172.21.15.113:6837","nonce":3327449639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:52.819867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6835/1818573977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1818573977},{"type":"v1","addr":"172.21.15.178:6835","nonce":1818573977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6690,"qdb_cluster":[6690]},"id":4}]} 2024-04-25T00:02:07.813 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 37 2024-04-25T00:02:07.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff68806be30 msgr2=0x7ff68806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff68806be30 0x7ff68806e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff698005970 tx=0x7ff698001df0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a813abb0 msgr2=0x7ff6a813e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a813abb0 0x7ff6a813e6b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d520 tx=0x7ff69000d9e0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 shutdown_connections 2024-04-25T00:02:07.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff68806be30 0x7ff68806e2f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6a813ebf0 0x7ff6a81366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a813abb0 0x7ff6a813e6b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 --2- 172.21.15.113:0/275612011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6a815baa0 0x7ff6a813a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 >> 172.21.15.113:0/275612011 conn(0x7ff6a80b1a50 msgr2=0x7ff6a8071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:07.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 shutdown_connections 2024-04-25T00:02:07.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.813+0000 7ff6aca9a640 1 -- 172.21.15.113:0/275612011 wait complete. 2024-04-25T00:02:07.827 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:07.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.921+0000 7f24a8ba5640 1 Processor -- start 2024-04-25T00:02:07.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.921+0000 7f24a8ba5640 1 -- start start 2024-04-25T00:02:07.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a8ba5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24a415e320 0x7f24a40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a8ba5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f24a40d0b00 0x7f24a40d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a8ba5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a40d3de0 con 0x7f24a40d2290 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a8ba5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f24a40d14b0 con 0x7f24a415e320 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a8ba5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f24a40d3410 con 0x7f24a40d0b00 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24a415e320 0x7f24a40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24a415e320 0x7f24a40d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40060/0 (socket says 172.21.15.113:40060) 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a259c640 1 -- 172.21.15.113:0/4164494835 learned_addr learned my addr 172.21.15.113:0/4164494835 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.922+0000 7f24a1d9b640 1 --1- 172.21.15.113:0/4164494835 >> v1:172.21.15.113:6789/0 conn(0x7f24a40d2290 0x7f24a40b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54740/0 (socket says 172.21.15.113:54740) 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.923+0000 7f24a2d9d640 1 --2- 172.21.15.113:0/4164494835 >> v2:172.21.15.178:3300/0 conn(0x7f24a40d0b00 0x7f24a40d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.923+0000 7f24a259c640 1 -- 172.21.15.113:0/4164494835 >> v2:172.21.15.178:3300/0 conn(0x7f24a40d0b00 msgr2=0x7f24a40d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.923+0000 7f24a259c640 1 --2- 172.21.15.113:0/4164494835 >> v2:172.21.15.178:3300/0 conn(0x7f24a40d0b00 0x7f24a40d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.923+0000 7f24a259c640 1 -- 172.21.15.113:0/4164494835 >> v1:172.21.15.113:6789/0 conn(0x7f24a40d2290 legacy=0x7f24a40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.923+0000 7f24a259c640 1 -- 172.21.15.113:0/4164494835 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a40d2800 con 0x7f24a415e320 2024-04-25T00:02:07.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.923+0000 7f24a259c640 1 --2- 172.21.15.113:0/4164494835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a415e320 0x7f24a40d38a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f24a40d2030 tx=0x7f249402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7ebd86839f7c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.923+0000 7f24a159a640 1 -- 172.21.15.113:0/4164494835 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494036070 con 0x7f24a415e320 2024-04-25T00:02:07.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a159a640 1 -- 172.21.15.113:0/4164494835 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2494031600 con 0x7f24a415e320 2024-04-25T00:02:07.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a159a640 1 -- 172.21.15.113:0/4164494835 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940319f0 con 0x7f24a415e320 2024-04-25T00:02:07.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 -- 172.21.15.113:0/4164494835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a415e320 msgr2=0x7f24a40d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/4164494835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a415e320 0x7f24a40d38a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f24a40d2030 tx=0x7f249402a7c0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 -- 172.21.15.113:0/4164494835 shutdown_connections 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/4164494835 >> v2:172.21.15.178:3300/0 conn(0x7f24a40d0b00 0x7f24a40d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/4164494835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a415e320 0x7f24a40d38a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 -- 172.21.15.113:0/4164494835 >> 172.21.15.113:0/4164494835 conn(0x7f24a40b5c20 msgr2=0x7f24a40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 -- 172.21.15.113:0/4164494835 shutdown_connections 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.924+0000 7f24a8ba5640 1 -- 172.21.15.113:0/4164494835 wait complete. 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.925+0000 7f24a8ba5640 1 Processor -- start 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.925+0000 7f24a8ba5640 1 -- start start 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a8ba5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24a40d0b00 0x7f24a41613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a8ba5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a40d2290 0x7f24a4154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a8ba5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a415e320 0x7f24a40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a8ba5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f24a40d30a0 con 0x7f24a40d2290 2024-04-25T00:02:07.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a8ba5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24a40d2dc0 con 0x7f24a415e320 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a8ba5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24a415e700 con 0x7f24a40d0b00 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a40d2290 0x7f24a4154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a40d2290 0x7f24a4154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40062/0 (socket says 172.21.15.113:40062) 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a1d9b640 1 -- 172.21.15.113:0/1052356293 learned_addr learned my addr 172.21.15.113:0/1052356293 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a1d9b640 1 -- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24a40d0b00 msgr2=0x7f24a41613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a1d9b640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24a40d0b00 0x7f24a41613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a2d9d640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a415e320 0x7f24a40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.926+0000 7f24a1d9b640 1 -- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a415e320 msgr2=0x7f24a40cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f24a1d9b640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a415e320 0x7f24a40cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f24a1d9b640 1 -- 172.21.15.113:0/1052356293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a4161900 con 0x7f24a40d2290 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f24a2d9d640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a415e320 0x7f24a40cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f24a1d9b640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a40d2290 0x7f24a4154a20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f249800c500 tx=0x7f249800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f2492ffd640 1 -- 172.21.15.113:0/1052356293 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498018070 con 0x7f24a40d2290 2024-04-25T00:02:07.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f2492ffd640 1 -- 172.21.15.113:0/1052356293 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2498007500 con 0x7f24a40d2290 2024-04-25T00:02:07.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4148ac0 con 0x7f24a40d2290 2024-04-25T00:02:07.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f2492ffd640 1 -- 172.21.15.113:0/1052356293 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24980049e0 con 0x7f24a40d2290 2024-04-25T00:02:07.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.927+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a4149080 con 0x7f24a40d2290 2024-04-25T00:02:07.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.928+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a40691a0 con 0x7f24a40d2290 2024-04-25T00:02:07.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.928+0000 7f2492ffd640 1 -- 172.21.15.113:0/1052356293 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f249801b070 con 0x7f24a40d2290 2024-04-25T00:02:07.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.929+0000 7f2492ffd640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248406be30 0x7f248406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:07.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.929+0000 7f2492ffd640 1 -- 172.21.15.113:0/1052356293 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f249808a250 con 0x7f24a40d2290 2024-04-25T00:02:07.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.929+0000 7f24a259c640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248406be30 0x7f248406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:07.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.929+0000 7f24a259c640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248406be30 0x7f248406e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2494033000 tx=0x7f249402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:07.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:07.933+0000 7f2492ffd640 1 -- 172.21.15.113:0/1052356293 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2498058c10 con 0x7f24a40d2290 2024-04-25T00:02:08.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.108+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24a40b7860 con 0x7f24a40d2290 2024-04-25T00:02:08.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.109+0000 7f2492ffd640 1 -- 172.21.15.113:0/1052356293 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16844 (secure 0 0 0) 0x7f2498058510 con 0x7f24a40d2290 2024-04-25T00:02:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:08.111 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:49.536503+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6834/3704852657":"2024-04-26T00:01:49.531195+0000","172.21.15.178:6832/3704852657":"2024-04-26T00:01:49.531195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:08.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248406be30 msgr2=0x7f248406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248406be30 0x7f248406e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2494033000 tx=0x7f249402e000 comp rx=0 tx=0).stop 2024-04-25T00:02:08.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a40d2290 msgr2=0x7f24a4154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a40d2290 0x7f24a4154a20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f249800c500 tx=0x7f249800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 shutdown_connections 2024-04-25T00:02:08.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f248406be30 0x7f248406e2f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a415e320 0x7f24a40cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a40d2290 0x7f24a4154a20 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 --2- 172.21.15.113:0/1052356293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24a40d0b00 0x7f24a41613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 >> 172.21.15.113:0/1052356293 conn(0x7f24a40b5c20 msgr2=0x7f24a40b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:08.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 shutdown_connections 2024-04-25T00:02:08.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.111+0000 7f24a8ba5640 1 -- 172.21.15.113:0/1052356293 wait complete. 2024-04-25T00:02:08.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:01:34.014634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:01:34.775167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:02:08.124 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:02:08.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.223+0000 7f60c55a0640 1 Processor -- start 2024-04-25T00:02:08.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.223+0000 7f60c55a0640 1 -- start start 2024-04-25T00:02:08.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60c55a0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60c015e310 0x7f60c006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60c55a0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60c55a0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c015e6f0 con 0x7f60c006bb00 2024-04-25T00:02:08.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60c55a0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f60c00d1140 con 0x7f60c015e310 2024-04-25T00:02:08.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60c55a0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60c00b1650 con 0x7f60c00d20e0 2024-04-25T00:02:08.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60bffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60c015e310 0x7f60c006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60bffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60c015e310 0x7f60c006b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40076/0 (socket says 172.21.15.113:40076) 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60bffff640 1 -- 172.21.15.113:0/1957814081 learned_addr learned my addr 172.21.15.113:0/1957814081 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60bffff640 1 -- 172.21.15.113:0/1957814081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 msgr2=0x7f60c00b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60bffff640 1 --2- 172.21.15.113:0/1957814081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60bffff640 1 -- 172.21.15.113:0/1957814081 >> v1:172.21.15.178:6789/0 conn(0x7f60c006bb00 legacy=0x7f60c00d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.224+0000 7f60bffff640 1 -- 172.21.15.113:0/1957814081 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c006b840 con 0x7f60c015e310 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.225+0000 7f60bffff640 1 --2- 172.21.15.113:0/1957814081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c015e310 0x7f60c006b300 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009870 tx=0x7f60a802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdb8a052f181bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.225+0000 7f60beffd640 1 -- 172.21.15.113:0/1957814081 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8037070 con 0x7f60c015e310 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.225+0000 7f60beffd640 1 -- 172.21.15.113:0/1957814081 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a8032440 con 0x7f60c015e310 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.225+0000 7f60beffd640 1 -- 172.21.15.113:0/1957814081 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8032810 con 0x7f60c015e310 2024-04-25T00:02:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.225+0000 7f60c55a0640 1 -- 172.21.15.113:0/1957814081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c015e310 msgr2=0x7f60c006b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.226+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1957814081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c015e310 0x7f60c006b300 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009870 tx=0x7f60a802a2c0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.226+0000 7f60c55a0640 1 -- 172.21.15.113:0/1957814081 shutdown_connections 2024-04-25T00:02:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.226+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1957814081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.226+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1957814081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c015e310 0x7f60c006b300 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.226+0000 7f60c55a0640 1 -- 172.21.15.113:0/1957814081 >> 172.21.15.113:0/1957814081 conn(0x7f60c009aa20 msgr2=0x7f60c009ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.226+0000 7f60c55a0640 1 -- 172.21.15.113:0/1957814081 shutdown_connections 2024-04-25T00:02:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.226+0000 7f60c55a0640 1 -- 172.21.15.113:0/1957814081 wait complete. 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.227+0000 7f60c55a0640 1 Processor -- start 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.227+0000 7f60c55a0640 1 -- start start 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c015e310 0x7f60c015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c00b5b80 0x7f60c00b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60c00d0af0 con 0x7f60c00b5b80 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60c006c270 con 0x7f60c015e310 2024-04-25T00:02:08.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60c00d0dd0 con 0x7f60c00d20e0 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c00b5b80 0x7f60c00b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c00b5b80 0x7f60c00b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40090/0 (socket says 172.21.15.113:40090) 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 -- 172.21.15.113:0/1966933229 learned_addr learned my addr 172.21.15.113:0/1966933229 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60bffff640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60bf7fe640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c015e310 0x7f60c015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 -- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 msgr2=0x7f60c00b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 -- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c015e310 msgr2=0x7f60c015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c015e310 0x7f60c015a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 -- 172.21.15.113:0/1966933229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c00b6680 con 0x7f60c00b5b80 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60bf7fe640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c015e310 0x7f60c015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60bffff640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c4d9f640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c00b5b80 0x7f60c00b6030 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ba10 tx=0x7f60b800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60bcff9640 1 -- 172.21.15.113:0/1966933229 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8019070 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60bcff9640 1 -- 172.21.15.113:0/1966933229 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b800cb40 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c00b6960 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60bcff9640 1 -- 172.21.15.113:0/1966933229 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8014440 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.228+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c00b6f20 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.229+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c00691a0 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.230+0000 7f60bcff9640 1 -- 172.21.15.113:0/1966933229 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60b801c070 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.230+0000 7f60bcff9640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60a006be30 0x7f60a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.230+0000 7f60bcff9640 1 -- 172.21.15.113:0/1966933229 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f60b808ab90 con 0x7f60c00b5b80 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.230+0000 7f60bffff640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60a006be30 0x7f60a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.231+0000 7f60bffff640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60a006be30 0x7f60a006e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f60a8034000 tx=0x7f60a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.234+0000 7f60bcff9640 1 -- 172.21.15.113:0/1966933229 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60b80594e0 con 0x7f60c00b5b80 2024-04-25T00:02:08.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.405+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60c00c6540 con 0x7f60c00b5b80 2024-04-25T00:02:08.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.405+0000 7f60bcff9640 1 -- 172.21.15.113:0/1966933229 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) ==== 93+0+10 (secure 0 0 0) 0x7f60b8058de0 con 0x7f60c00b5b80 2024-04-25T00:02:08.407 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60a006be30 msgr2=0x7f60a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60a006be30 0x7f60a006e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f60a8034000 tx=0x7f60a802e000 comp rx=0 tx=0).stop 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c00b5b80 msgr2=0x7f60c00b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c00b5b80 0x7f60c00b6030 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ba10 tx=0x7f60b800bed0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 shutdown_connections 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60a006be30 0x7f60a006e2f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c015e310 0x7f60c015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c00b5b80 0x7f60c00b6030 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 --2- 172.21.15.113:0/1966933229 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60c00d20e0 0x7f60c00b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.407+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 >> 172.21.15.113:0/1966933229 conn(0x7f60c009aa20 msgr2=0x7f60c00aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:08.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.408+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 shutdown_connections 2024-04-25T00:02:08.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.408+0000 7f60c55a0640 1 -- 172.21.15.113:0/1966933229 wait complete. 2024-04-25T00:02:08.420 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:02:08.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.520+0000 7fde94864640 1 Processor -- start 2024-04-25T00:02:08.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.520+0000 7fde94864640 1 -- start start 2024-04-25T00:02:08.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.521+0000 7fde94864640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fde9015e510 0x7fde900d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.521+0000 7fde94864640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde900d2e00 con 0x7fde900d28c0 2024-04-25T00:02:08.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.521+0000 7fde94864640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde900d30e0 con 0x7fde900b63b0 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.521+0000 7fde94864640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fde900d33c0 con 0x7fde9015e510 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.521+0000 7fde8dd9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fde900d28c0 0x7fde900b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54754/0 (socket says 172.21.15.113:54754) 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.521+0000 7fde8dd9b640 1 -- 172.21.15.113:0/424336625 learned_addr learned my addr 172.21.15.113:0/424336625 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.522+0000 7fde8e59c640 1 --2- 172.21.15.113:0/424336625 >> v2:172.21.15.178:3300/0 conn(0x7fde9015e510 0x7fde900d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.522+0000 7fde8d59a640 1 -- 172.21.15.113:0/424336625 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669803243 0 0) 0x7fde900d2e00 con 0x7fde900d28c0 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.522+0000 7fde8d59a640 1 -- 172.21.15.113:0/424336625 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde7c003520 con 0x7fde900d28c0 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.522+0000 7fde8d59a640 1 -- 172.21.15.113:0/424336625 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2211251289 0 0) 0x7fde7c003520 con 0x7fde900d28c0 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.522+0000 7fde8e59c640 1 -- 172.21.15.113:0/424336625 >> v1:172.21.15.178:6789/0 conn(0x7fde900b63b0 legacy=0x7fde900b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.522+0000 7fde8e59c640 1 -- 172.21.15.113:0/424336625 >> v1:172.21.15.113:6789/0 conn(0x7fde900d28c0 legacy=0x7fde900b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.522+0000 7fde8e59c640 1 -- 172.21.15.113:0/424336625 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde900d36a0 con 0x7fde9015e510 2024-04-25T00:02:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.523+0000 7fde8e59c640 1 --2- 172.21.15.113:0/424336625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9015e510 0x7fde900d2380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d90 tx=0x7fde8002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576359d819b2e0ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.523+0000 7fde8d59a640 1 -- 172.21.15.113:0/424336625 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78001670 con 0x7fde9015e510 2024-04-25T00:02:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.523+0000 7fde8d59a640 1 -- 172.21.15.113:0/424336625 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde80007540 con 0x7fde9015e510 2024-04-25T00:02:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.523+0000 7fde8d59a640 1 -- 172.21.15.113:0/424336625 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde800078f0 con 0x7fde9015e510 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.523+0000 7fde94864640 1 -- 172.21.15.113:0/424336625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9015e510 msgr2=0x7fde900d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.523+0000 7fde94864640 1 --2- 172.21.15.113:0/424336625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9015e510 0x7fde900d2380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d90 tx=0x7fde8002a5f0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.523+0000 7fde94864640 1 -- 172.21.15.113:0/424336625 shutdown_connections 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.524+0000 7fde94864640 1 --2- 172.21.15.113:0/424336625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9015e510 0x7fde900d2380 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.524+0000 7fde94864640 1 -- 172.21.15.113:0/424336625 >> 172.21.15.113:0/424336625 conn(0x7fde9006b720 msgr2=0x7fde900d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.524+0000 7fde94864640 1 -- 172.21.15.113:0/424336625 shutdown_connections 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.524+0000 7fde94864640 1 -- 172.21.15.113:0/424336625 wait complete. 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.525+0000 7fde94864640 1 Processor -- start 2024-04-25T00:02:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.525+0000 7fde94864640 1 -- start start 2024-04-25T00:02:08.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde900b63b0 0x7fde90154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde900d28c0 0x7fde900c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9015e510 0x7fde9015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fde900b9dd0 con 0x7fde900d28c0 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde900d3ea0 con 0x7fde900b63b0 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde900b7ad0 con 0x7fde9015e510 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde900d28c0 0x7fde900c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde900d28c0 0x7fde900c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40096/0 (socket says 172.21.15.113:40096) 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 -- 172.21.15.113:0/3165930735 learned_addr learned my addr 172.21.15.113:0/3165930735 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8e59c640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde900b63b0 0x7fde90154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8ed9d640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9015e510 0x7fde9015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 -- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9015e510 msgr2=0x7fde9015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9015e510 0x7fde9015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 -- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde900b63b0 msgr2=0x7fde90154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde900b63b0 0x7fde90154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 -- 172.21.15.113:0/3165930735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde900c96a0 con 0x7fde900d28c0 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8ed9d640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9015e510 0x7fde9015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde8dd9b640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde900d28c0 0x7fde900c8da0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fde7800c750 tx=0x7fde7800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde86ffd640 1 -- 172.21.15.113:0/3165930735 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78017070 con 0x7fde900d28c0 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde86ffd640 1 -- 172.21.15.113:0/3165930735 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde78005ad0 con 0x7fde900d28c0 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde900c9980 con 0x7fde900d28c0 2024-04-25T00:02:08.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde86ffd640 1 -- 172.21.15.113:0/3165930735 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78007070 con 0x7fde900d28c0 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.526+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde900c9f40 con 0x7fde900d28c0 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.527+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde900691a0 con 0x7fde900d28c0 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.528+0000 7fde86ffd640 1 -- 172.21.15.113:0/3165930735 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde7801a070 con 0x7fde900d28c0 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.528+0000 7fde86ffd640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde7006be60 0x7fde7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.528+0000 7fde86ffd640 1 -- 172.21.15.113:0/3165930735 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fde7808a3c0 con 0x7fde900d28c0 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.528+0000 7fde8e59c640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde7006be60 0x7fde7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.529+0000 7fde8e59c640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde7006be60 0x7fde7006e320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fde80034000 tx=0x7fde8002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.533+0000 7fde86ffd640 1 -- 172.21.15.113:0/3165930735 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde78058d10 con 0x7fde900d28c0 2024-04-25T00:02:08.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.705+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde900b8680 con 0x7fde900d28c0 2024-04-25T00:02:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.705+0000 7fde86ffd640 1 -- 172.21.15.113:0/3165930735 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) ==== 104+0+11 (secure 0 0 0) 0x7fde78058610 con 0x7fde900d28c0 2024-04-25T00:02:08.707 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:02:08.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde7006be60 msgr2=0x7fde7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde7006be60 0x7fde7006e320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fde80034000 tx=0x7fde8002f000 comp rx=0 tx=0).stop 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde900d28c0 msgr2=0x7fde900c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde900d28c0 0x7fde900c8da0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fde7800c750 tx=0x7fde7800cc10 comp rx=0 tx=0).stop 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 shutdown_connections 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde7006be60 0x7fde7006e320 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9015e510 0x7fde9015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde900d28c0 0x7fde900c8da0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 --2- 172.21.15.113:0/3165930735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde900b63b0 0x7fde90154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 >> 172.21.15.113:0/3165930735 conn(0x7fde9006b720 msgr2=0x7fde90071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 shutdown_connections 2024-04-25T00:02:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.707+0000 7fde94864640 1 -- 172.21.15.113:0/3165930735 wait complete. 2024-04-25T00:02:08.719 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:02:08.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.817+0000 7fd50f4b3640 1 Processor -- start 2024-04-25T00:02:08.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.817+0000 7fd50f4b3640 1 -- start start 2024-04-25T00:02:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50f4b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd50815b670 0x7fd50815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50f4b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd50815bff0 0x7fd5080cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50f4b3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd508071170 con 0x7fd5080cc740 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50f4b3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd50815c4a0 con 0x7fd50815b670 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50f4b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd508071450 con 0x7fd50815bff0 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50d24f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd50815b670 0x7fd50815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50d24f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd50815b670 0x7fd50815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40102/0 (socket says 172.21.15.113:40102) 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50da50640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd5080cc740 0x7fd50806cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54756/0 (socket says 172.21.15.113:54756) 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50d24f640 1 -- 172.21.15.113:0/2507926449 learned_addr learned my addr 172.21.15.113:0/2507926449 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50ca4e640 1 --2- 172.21.15.113:0/2507926449 >> v2:172.21.15.178:3300/0 conn(0x7fd50815bff0 0x7fd5080cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50d24f640 1 -- 172.21.15.113:0/2507926449 >> v2:172.21.15.178:3300/0 conn(0x7fd50815bff0 msgr2=0x7fd5080cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50d24f640 1 --2- 172.21.15.113:0/2507926449 >> v2:172.21.15.178:3300/0 conn(0x7fd50815bff0 0x7fd5080cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50d24f640 1 -- 172.21.15.113:0/2507926449 >> v1:172.21.15.113:6789/0 conn(0x7fd5080cc740 legacy=0x7fd50806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd50d24f640 1 -- 172.21.15.113:0/2507926449 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd508071730 con 0x7fd50815b670 2024-04-25T00:02:08.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.818+0000 7fd4fffff640 1 -- 172.21.15.113:0/2507926449 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075889627 0 0) 0x7fd508071170 con 0x7fd5080cc740 2024-04-25T00:02:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.819+0000 7fd50d24f640 1 --2- 172.21.15.113:0/2507926449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815b670 0x7fd50815ba50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd5081554a0 tx=0x7fd50402c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d18ec9aea4877c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.819+0000 7fd4fffff640 1 -- 172.21.15.113:0/2507926449 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50403a070 con 0x7fd50815b670 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.819+0000 7fd4fffff640 1 -- 172.21.15.113:0/2507926449 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd504032480 con 0x7fd50815b670 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.819+0000 7fd4fffff640 1 -- 172.21.15.113:0/2507926449 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504032850 con 0x7fd50815b670 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2507926449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815b670 msgr2=0x7fd50815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2507926449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815b670 0x7fd50815ba50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd5081554a0 tx=0x7fd50402c600 comp rx=0 tx=0).stop 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2507926449 shutdown_connections 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2507926449 >> v2:172.21.15.178:3300/0 conn(0x7fd50815bff0 0x7fd5080cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2507926449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815b670 0x7fd50815ba50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2507926449 >> 172.21.15.113:0/2507926449 conn(0x7fd50806b0f0 msgr2=0x7fd50806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2507926449 shutdown_connections 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.820+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2507926449 wait complete. 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.821+0000 7fd50f4b3640 1 Processor -- start 2024-04-25T00:02:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.821+0000 7fd50f4b3640 1 -- start start 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50f4b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5080cc740 0x7fd5081459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50f4b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd50815b670 0x7fd5081e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50f4b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815bff0 0x7fd5080c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50f4b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd50806d730 con 0x7fd50815bff0 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50f4b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd50806d5b0 con 0x7fd5080cc740 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50f4b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5080d0080 con 0x7fd50815b670 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815bff0 0x7fd5080c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815bff0 0x7fd5080c4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40110/0 (socket says 172.21.15.113:40110) 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 -- 172.21.15.113:0/2157058492 learned_addr learned my addr 172.21.15.113:0/2157058492 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50d24f640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5080cc740 0x7fd5081459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50ca4e640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd50815b670 0x7fd5081e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 -- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd50815b670 msgr2=0x7fd5081e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd50815b670 0x7fd5081e1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 -- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5080cc740 msgr2=0x7fd5081459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5080cc740 0x7fd5081459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 -- 172.21.15.113:0/2157058492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5081e27a0 con 0x7fd50815bff0 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50da50640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815bff0 0x7fd5080c4f80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c7e0 tx=0x7fd50000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd4fdffb640 1 -- 172.21.15.113:0/2157058492 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500018070 con 0x7fd50815bff0 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd4fdffb640 1 -- 172.21.15.113:0/2157058492 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd50000a640 con 0x7fd50815bff0 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5081e2a80 con 0x7fd50815bff0 2024-04-25T00:02:08.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.822+0000 7fd4fdffb640 1 -- 172.21.15.113:0/2157058492 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50000aa30 con 0x7fd50815bff0 2024-04-25T00:02:08.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.823+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd508145f10 con 0x7fd50815bff0 2024-04-25T00:02:08.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.823+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5081504d0 con 0x7fd50815bff0 2024-04-25T00:02:08.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.824+0000 7fd4fdffb640 1 -- 172.21.15.113:0/2157058492 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd50001b070 con 0x7fd50815bff0 2024-04-25T00:02:08.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.824+0000 7fd4fdffb640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4ec06be30 0x7fd4ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:08.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.824+0000 7fd4fdffb640 1 -- 172.21.15.113:0/2157058492 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fd50008a4d0 con 0x7fd50815bff0 2024-04-25T00:02:08.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.824+0000 7fd50d24f640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4ec06be30 0x7fd4ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:08.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.825+0000 7fd50d24f640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4ec06be30 0x7fd4ec06e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd504037000 tx=0x7fd504030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:08.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:08.829+0000 7fd4fdffb640 1 -- 172.21.15.113:0/2157058492 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd500058eb0 con 0x7fd50815bff0 2024-04-25T00:02:09.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.002+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd5080691a0 con 0x7fd50815bff0 2024-04-25T00:02:09.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.003+0000 7fd4fdffb640 1 -- 172.21.15.113:0/2157058492 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) ==== 100+0+11 (secure 0 0 0) 0x7fd50005f080 con 0x7fd50815bff0 2024-04-25T00:02:09.004 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:02:09.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4ec06be30 msgr2=0x7fd4ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4ec06be30 0x7fd4ec06e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd504037000 tx=0x7fd504030000 comp rx=0 tx=0).stop 2024-04-25T00:02:09.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815bff0 msgr2=0x7fd5080c4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815bff0 0x7fd5080c4f80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c7e0 tx=0x7fd50000cca0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 shutdown_connections 2024-04-25T00:02:09.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd4ec06be30 0x7fd4ec06e2f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50815bff0 0x7fd5080c4f80 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd50815b670 0x7fd5081e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 --2- 172.21.15.113:0/2157058492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5080cc740 0x7fd5081459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 >> 172.21.15.113:0/2157058492 conn(0x7fd50806b0f0 msgr2=0x7fd5080cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:09.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 shutdown_connections 2024-04-25T00:02:09.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.005+0000 7fd50f4b3640 1 -- 172.21.15.113:0/2157058492 wait complete. 2024-04-25T00:02:09.017 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:02:09.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.112+0000 7f6f3806b640 1 Processor -- start 2024-04-25T00:02:09.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.112+0000 7f6f3806b640 1 -- start start 2024-04-25T00:02:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f3806b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6f300b3fd0 0x7f6f3006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f3806b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f3006e510 0x7f6f300cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f3806b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f30144cf0 con 0x7f6f30144910 2024-04-25T00:02:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f3806b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3006e980 con 0x7f6f3006e510 2024-04-25T00:02:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f3806b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6f300b2f30 con 0x7f6f300b3fd0 2024-04-25T00:02:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f36608640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f3006e510 0x7f6f300cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f36608640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f3006e510 0x7f6f300cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40112/0 (socket says 172.21.15.113:40112) 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f35e07640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6f30144910 0x7f6f300b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54762/0 (socket says 172.21.15.113:54762) 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f36608640 1 -- 172.21.15.113:0/2985386515 learned_addr learned my addr 172.21.15.113:0/2985386515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f35606640 1 --2- 172.21.15.113:0/2985386515 >> v2:172.21.15.178:3300/0 conn(0x7f6f300b3fd0 0x7f6f3006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f36608640 1 -- 172.21.15.113:0/2985386515 >> v2:172.21.15.178:3300/0 conn(0x7f6f300b3fd0 msgr2=0x7f6f3006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f36608640 1 --2- 172.21.15.113:0/2985386515 >> v2:172.21.15.178:3300/0 conn(0x7f6f300b3fd0 0x7f6f3006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f36608640 1 -- 172.21.15.113:0/2985386515 >> v1:172.21.15.113:6789/0 conn(0x7f6f30144910 legacy=0x7f6f300b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f36608640 1 -- 172.21.15.113:0/2985386515 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f300b3210 con 0x7f6f3006e510 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.113+0000 7f6f34e05640 1 -- 172.21.15.113:0/2985386515 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760499315 0 0) 0x7f6f30144cf0 con 0x7f6f30144910 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.114+0000 7f6f36608640 1 --2- 172.21.15.113:0/2985386515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f300cd500 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009e50 tx=0x7f6f2802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff64a5ac53e71be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.114+0000 7f6f34e05640 1 -- 172.21.15.113:0/2985386515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28036070 con 0x7f6f3006e510 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.114+0000 7f6f34e05640 1 -- 172.21.15.113:0/2985386515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f28031600 con 0x7f6f3006e510 2024-04-25T00:02:09.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.114+0000 7f6f34e05640 1 -- 172.21.15.113:0/2985386515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f280319d0 con 0x7f6f3006e510 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 -- 172.21.15.113:0/2985386515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 msgr2=0x7f6f300cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2985386515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f300cd500 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009e50 tx=0x7f6f2802a760 comp rx=0 tx=0).stop 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 -- 172.21.15.113:0/2985386515 shutdown_connections 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2985386515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f300cd500 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2985386515 >> v2:172.21.15.178:3300/0 conn(0x7f6f300b3fd0 0x7f6f3006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 -- 172.21.15.113:0/2985386515 >> 172.21.15.113:0/2985386515 conn(0x7f6f300b1a50 msgr2=0x7f6f300b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 -- 172.21.15.113:0/2985386515 shutdown_connections 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.115+0000 7f6f3806b640 1 -- 172.21.15.113:0/2985386515 wait complete. 2024-04-25T00:02:09.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.116+0000 7f6f3806b640 1 Processor -- start 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.116+0000 7f6f3806b640 1 -- start start 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.116+0000 7f6f3806b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f3015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f3806b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f300b3fd0 0x7f6f30138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f3806b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f30144910 0x7f6f3015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f3806b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6f300b4440 con 0x7f6f3006e510 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f3806b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f300b37d0 con 0x7f6f30144910 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f3806b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6f300b4da0 con 0x7f6f300b3fd0 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f3015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f3015bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40128/0 (socket says 172.21.15.113:40128) 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 -- 172.21.15.113:0/2163568451 learned_addr learned my addr 172.21.15.113:0/2163568451 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35606640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f300b3fd0 0x7f6f30138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f36608640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f30144910 0x7f6f3015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 -- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f300b3fd0 msgr2=0x7f6f30138770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f300b3fd0 0x7f6f30138770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 -- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f30144910 msgr2=0x7f6f3015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f30144910 0x7f6f3015d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 -- 172.21.15.113:0/2163568451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3015c860 con 0x7f6f3006e510 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35606640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f300b3fd0 0x7f6f30138770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f36608640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f30144910 0x7f6f3015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f35e07640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f3015bfa0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c430 tx=0x7f6f2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f267fc640 1 -- 172.21.15.113:0/2163568451 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0074e0 con 0x7f6f3006e510 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f267fc640 1 -- 172.21.15.113:0/2163568451 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c0047e0 con 0x7f6f3006e510 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3015c580 con 0x7f6f3006e510 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f267fc640 1 -- 172.21.15.113:0/2163568451 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c004bd0 con 0x7f6f3006e510 2024-04-25T00:02:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.117+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f300adc10 con 0x7f6f3006e510 2024-04-25T00:02:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.118+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f300691a0 con 0x7f6f3006e510 2024-04-25T00:02:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.119+0000 7f6f267fc640 1 -- 172.21.15.113:0/2163568451 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f2c011070 con 0x7f6f3006e510 2024-04-25T00:02:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.119+0000 7f6f267fc640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f1406be30 0x7f6f1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.119+0000 7f6f267fc640 1 -- 172.21.15.113:0/2163568451 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f6f2c08aa90 con 0x7f6f3006e510 2024-04-25T00:02:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.119+0000 7f6f35606640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f1406be30 0x7f6f1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.119+0000 7f6f35606640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f1406be30 0x7f6f1406e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6f3015d9b0 tx=0x7f6f20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.123+0000 7f6f267fc640 1 -- 172.21.15.113:0/2163568451 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f2c00e280 con 0x7f6f3006e510 2024-04-25T00:02:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.334+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f30143250 con 0x7f6f3006e510 2024-04-25T00:02:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.334+0000 7f6f267fc640 1 -- 172.21.15.113:0/2163568451 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5194 (secure 0 0 0) 0x7f6f2c059470 con 0x7f6f3006e510 2024-04-25T00:02:09.336 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:09.336 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3327449639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3327449639},{"type":"v1","addr":"172.21.15.113:6837","nonce":3327449639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:52.819867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6835/1818573977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1818573977},{"type":"v1","addr":"172.21.15.178:6835","nonce":1818573977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6690,"qdb_cluster":[6690]},"id":4}]} 2024-04-25T00:02:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 37 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f1406be30 msgr2=0x7f6f1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f1406be30 0x7f6f1406e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6f3015d9b0 tx=0x7f6f20008040 comp rx=0 tx=0).stop 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 msgr2=0x7f6f3015bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f3015bfa0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c430 tx=0x7f6f2c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 shutdown_connections 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f1406be30 0x7f6f1406e2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f30144910 0x7f6f3015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f300b3fd0 0x7f6f30138770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 --2- 172.21.15.113:0/2163568451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3006e510 0x7f6f3015bfa0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 >> 172.21.15.113:0/2163568451 conn(0x7f6f300b1a50 msgr2=0x7f6f300cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 shutdown_connections 2024-04-25T00:02:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.336+0000 7f6f3806b640 1 -- 172.21.15.113:0/2163568451 wait complete. 2024-04-25T00:02:09.348 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:09.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.443+0000 7f6bb763f640 1 Processor -- start 2024-04-25T00:02:09.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.443+0000 7f6bb763f640 1 -- start start 2024-04-25T00:02:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6bb763f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6bb015a6b0 0x7f6bb0070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6bb763f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb00b2d50 con 0x7f6bb0072ea0 2024-04-25T00:02:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6bb763f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb0072a70 con 0x7f6bb00b3af0 2024-04-25T00:02:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6bb763f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb00b3010 con 0x7f6bb015a6b0 2024-04-25T00:02:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6bb4bda640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6bb0072ea0 0x7f6bb0071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54772/0 (socket says 172.21.15.113:54772) 2024-04-25T00:02:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6bb4bda640 1 -- 172.21.15.113:0/988335963 learned_addr learned my addr 172.21.15.113:0/988335963 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6bb53db640 1 --2- 172.21.15.113:0/988335963 >> v2:172.21.15.178:3300/0 conn(0x7f6bb015a6b0 0x7f6bb0070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865123103 0 0) 0x7f6bb00b2d50 con 0x7f6bb0072ea0 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.444+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b9c003520 con 0x7f6bb0072ea0 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3176831206 0 0) 0x7f6b9c003520 con 0x7f6bb0072ea0 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 >> v2:172.21.15.178:3300/0 conn(0x7f6bb015a6b0 msgr2=0x7f6bb0070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 --2- 172.21.15.113:0/988335963 >> v2:172.21.15.178:3300/0 conn(0x7f6bb015a6b0 0x7f6bb0070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 >> v1:172.21.15.178:6789/0 conn(0x7f6bb00b3af0 legacy=0x7f6bb00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb00b32f0 con 0x7f6bb0072ea0 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6ba0001670 con 0x7f6bb0072ea0 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 >> v1:172.21.15.113:6789/0 conn(0x7f6bb0072ea0 legacy=0x7f6bb0071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c0039b0 0x7f6b9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9c004800 0x7f6b9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9c0051c0 0x7f6b9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb00b3010 con 0x7f6b9c004800 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c00c060 con 0x7f6b9c0039b0 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9c00c260 con 0x7f6b9c0051c0 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb4bda640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9c004800 0x7f6b9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb53db640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c0039b0 0x7f6b9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb5bdc640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9c0051c0 0x7f6b9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb4bda640 1 -- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9c0051c0 msgr2=0x7f6b9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb4bda640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9c0051c0 0x7f6b9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb4bda640 1 -- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c0039b0 msgr2=0x7f6b9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb4bda640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c0039b0 0x7f6b9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb4bda640 1 -- 172.21.15.113:0/988335963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c003520 con 0x7f6b9c004800 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb5bdc640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9c0051c0 0x7f6b9c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.445+0000 7f6bb5bdc640 1 -- 172.21.15.113:0/988335963 reap_dead start 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.446+0000 7f6bb4bda640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9c004800 0x7f6b9c004c80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c240 tx=0x7f6b9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c50ca946706e7e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.446+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0007cf0 con 0x7f6b9c004800 2024-04-25T00:02:09.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.446+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba00058d0 con 0x7f6b9c004800 2024-04-25T00:02:09.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.446+0000 7f6ba7fff640 1 -- 172.21.15.113:0/988335963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0005ca0 con 0x7f6b9c004800 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.447+0000 7f6bb763f640 1 -- 172.21.15.113:0/988335963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9c004800 msgr2=0x7f6b9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.447+0000 7f6bb763f640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9c004800 0x7f6b9c004c80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c240 tx=0x7f6b9c005830 comp rx=0 tx=0).stop 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.447+0000 7f6bb763f640 1 -- 172.21.15.113:0/988335963 shutdown_connections 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.447+0000 7f6bb763f640 1 --2- 172.21.15.113:0/988335963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9c004800 0x7f6b9c004c80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.447+0000 7f6bb763f640 1 -- 172.21.15.113:0/988335963 >> 172.21.15.113:0/988335963 conn(0x7f6bb00b1a50 msgr2=0x7f6bb00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.447+0000 7f6bb763f640 1 -- 172.21.15.113:0/988335963 shutdown_connections 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.447+0000 7f6bb763f640 1 -- 172.21.15.113:0/988335963 wait complete. 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.448+0000 7f6bb763f640 1 Processor -- start 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.448+0000 7f6bb763f640 1 -- start start 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c004800 0x7f6bb014b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bb014ee10 0x7f6bb01e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6bb0154ef0 0x7f6bb01e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb015aa90 con 0x7f6bb014ee10 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0072720 con 0x7f6b9c004800 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb015ac10 con 0x7f6bb0154ef0 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bb014ee10 0x7f6bb01e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bb014ee10 0x7f6bb01e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40138/0 (socket says 172.21.15.113:40138) 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 -- 172.21.15.113:0/2632646396 learned_addr learned my addr 172.21.15.113:0/2632646396 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb5bdc640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6bb0154ef0 0x7f6bb01e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb53db640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c004800 0x7f6bb014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 -- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6bb0154ef0 msgr2=0x7f6bb01e6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6bb0154ef0 0x7f6bb01e6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 -- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c004800 msgr2=0x7f6bb014b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c004800 0x7f6bb014b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 -- 172.21.15.113:0/2632646396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb014f2a0 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb53db640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c004800 0x7f6bb014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb5bdc640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6bb0154ef0 0x7f6bb01e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb4bda640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bb014ee10 0x7f6bb01e6580 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0001970 tx=0x7f6bb00b34c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6ba5ffb640 1 -- 172.21.15.113:0/2632646396 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0040030 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6ba5ffb640 1 -- 172.21.15.113:0/2632646396 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba00049f0 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb014f580 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6ba5ffb640 1 -- 172.21.15.113:0/2632646396 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0007cf0 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.449+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb014fb40 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.450+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb00691a0 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.451+0000 7f6ba5ffb640 1 -- 172.21.15.113:0/2632646396 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ba0043070 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.451+0000 7f6ba5ffb640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b9406be60 0x7f6b9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.451+0000 7f6ba5ffb640 1 -- 172.21.15.113:0/2632646396 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f6ba0038070 con 0x7f6bb014ee10 2024-04-25T00:02:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.451+0000 7f6bb53db640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b9406be60 0x7f6b9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.452+0000 7f6bb53db640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b9406be60 0x7f6b9406e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00a270 tx=0x7f6bac00c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.455+0000 7f6ba5ffb640 1 -- 172.21.15.113:0/2632646396 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ba004a020 con 0x7f6bb014ee10 2024-04-25T00:02:09.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.630+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bb00b2900 con 0x7f6bb014ee10 2024-04-25T00:02:09.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.631+0000 7f6ba5ffb640 1 -- 172.21.15.113:0/2632646396 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16844 (secure 0 0 0) 0x7f6ba0078e30 con 0x7f6bb014ee10 2024-04-25T00:02:09.633 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:09.633 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:49.536503+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6834/3704852657":"2024-04-26T00:01:49.531195+0000","172.21.15.178:6832/3704852657":"2024-04-26T00:01:49.531195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:09.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b9406be60 msgr2=0x7f6b9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b9406be60 0x7f6b9406e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00a270 tx=0x7f6bac00c040 comp rx=0 tx=0).stop 2024-04-25T00:02:09.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bb014ee10 msgr2=0x7f6bb01e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bb014ee10 0x7f6bb01e6580 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0001970 tx=0x7f6bb00b34c0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 shutdown_connections 2024-04-25T00:02:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b9406be60 0x7f6b9406e320 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6bb0154ef0 0x7f6bb01e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bb014ee10 0x7f6bb01e6580 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 --2- 172.21.15.113:0/2632646396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b9c004800 0x7f6bb014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 >> 172.21.15.113:0/2632646396 conn(0x7f6bb00b1a50 msgr2=0x7f6bb0151a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 shutdown_connections 2024-04-25T00:02:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.634+0000 7f6bb763f640 1 -- 172.21.15.113:0/2632646396 wait complete. 2024-04-25T00:02:09.646 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:02:09.646 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:02:09.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.742+0000 7f21aec72640 1 Processor -- start 2024-04-25T00:02:09.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.742+0000 7f21aec72640 1 -- start start 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.743+0000 7f21aec72640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f21a815a6b0 0x7f21a8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.743+0000 7f21aec72640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a80b2d50 con 0x7f21a8072ea0 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.743+0000 7f21aec72640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a8072a70 con 0x7f21a80b3af0 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.743+0000 7f21aec72640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f21a80b3010 con 0x7f21a815a6b0 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.743+0000 7f21a7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f21a8072ea0 0x7f21a8071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54774/0 (socket says 172.21.15.113:54774) 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.743+0000 7f21a7fff640 1 -- 172.21.15.113:0/2067368081 learned_addr learned my addr 172.21.15.113:0/2067368081 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.743+0000 7f21aca0e640 1 --2- 172.21.15.113:0/2067368081 >> v2:172.21.15.178:3300/0 conn(0x7f21a815a6b0 0x7f21a8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.744+0000 7f21a77fe640 1 -- 172.21.15.113:0/2067368081 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214445319 0 0) 0x7f21a80b2d50 con 0x7f21a8072ea0 2024-04-25T00:02:09.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.744+0000 7f21a77fe640 1 -- 172.21.15.113:0/2067368081 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2190003520 con 0x7f21a8072ea0 2024-04-25T00:02:09.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.744+0000 7f21aca0e640 1 -- 172.21.15.113:0/2067368081 >> v1:172.21.15.178:6789/0 conn(0x7f21a80b3af0 legacy=0x7f21a80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.744+0000 7f21aca0e640 1 -- 172.21.15.113:0/2067368081 >> v1:172.21.15.113:6789/0 conn(0x7f21a8072ea0 legacy=0x7f21a8071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.744+0000 7f21aca0e640 1 -- 172.21.15.113:0/2067368081 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a80b32f0 con 0x7f21a815a6b0 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aca0e640 1 --2- 172.21.15.113:0/2067368081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a815a6b0 0x7f21a8070d00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21980097a0 tx=0x7f219802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ceeb2a722825bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21a77fe640 1 -- 172.21.15.113:0/2067368081 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198037070 con 0x7f21a815a6b0 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21a77fe640 1 -- 172.21.15.113:0/2067368081 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2198032440 con 0x7f21a815a6b0 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21a77fe640 1 -- 172.21.15.113:0/2067368081 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21980327f0 con 0x7f21a815a6b0 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aec72640 1 -- 172.21.15.113:0/2067368081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a815a6b0 msgr2=0x7f21a8070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aec72640 1 --2- 172.21.15.113:0/2067368081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a815a6b0 0x7f21a8070d00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21980097a0 tx=0x7f219802a230 comp rx=0 tx=0).stop 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aec72640 1 -- 172.21.15.113:0/2067368081 shutdown_connections 2024-04-25T00:02:09.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aec72640 1 --2- 172.21.15.113:0/2067368081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a815a6b0 0x7f21a8070d00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aec72640 1 -- 172.21.15.113:0/2067368081 >> 172.21.15.113:0/2067368081 conn(0x7f21a80b1a50 msgr2=0x7f21a80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:09.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aec72640 1 -- 172.21.15.113:0/2067368081 shutdown_connections 2024-04-25T00:02:09.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.745+0000 7f21aec72640 1 -- 172.21.15.113:0/2067368081 wait complete. 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.746+0000 7f21aec72640 1 Processor -- start 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.746+0000 7f21aec72640 1 -- start start 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aec72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a8072ea0 0x7f21a80a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aec72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a80b3af0 0x7f21a8142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aec72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21a815a6b0 0x7f21a813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aec72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f21a815aa90 con 0x7f21a8072ea0 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aec72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8072720 con 0x7f21a80b3af0 2024-04-25T00:02:09.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aec72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f21a815ac10 con 0x7f21a815a6b0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aca0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a8072ea0 0x7f21a80a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aca0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a8072ea0 0x7f21a80a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40144/0 (socket says 172.21.15.113:40144) 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.747+0000 7f21aca0e640 1 -- 172.21.15.113:0/3019578089 learned_addr learned my addr 172.21.15.113:0/3019578089 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21a7fff640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a80b3af0 0x7f21a8142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21ad20f640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21a815a6b0 0x7f21a813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aca0e640 1 -- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21a815a6b0 msgr2=0x7f21a813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aca0e640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21a815a6b0 0x7f21a813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aca0e640 1 -- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a80b3af0 msgr2=0x7f21a8142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aca0e640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a80b3af0 0x7f21a8142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aca0e640 1 -- 172.21.15.113:0/3019578089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a8143140 con 0x7f21a8072ea0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aca0e640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a8072ea0 0x7f21a80a9480 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f21980097a0 tx=0x7f21980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21a57fa640 1 -- 172.21.15.113:0/3019578089 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198037050 con 0x7f21a8072ea0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21a57fa640 1 -- 172.21.15.113:0/3019578089 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2198038040 con 0x7f21a8072ea0 2024-04-25T00:02:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a815c180 con 0x7f21a8072ea0 2024-04-25T00:02:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21a57fa640 1 -- 172.21.15.113:0/3019578089 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198004470 con 0x7f21a8072ea0 2024-04-25T00:02:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.748+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a815c740 con 0x7f21a8072ea0 2024-04-25T00:02:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.749+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a80691a0 con 0x7f21a8072ea0 2024-04-25T00:02:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.749+0000 7f21a57fa640 1 -- 172.21.15.113:0/3019578089 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f219803c6d0 con 0x7f21a8072ea0 2024-04-25T00:02:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.750+0000 7f21a57fa640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f219406be30 0x7f219406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.750+0000 7f21a7fff640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f219406be30 0x7f219406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.750+0000 7f21a57fa640 1 -- 172.21.15.113:0/3019578089 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f21980ab800 con 0x7f21a8072ea0 2024-04-25T00:02:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.750+0000 7f21a7fff640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f219406be30 0x7f219406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f21a80ceaa0 tx=0x7f219c007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:09.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.754+0000 7f21a57fa640 1 -- 172.21.15.113:0/3019578089 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f219807a1e0 con 0x7f21a8072ea0 2024-04-25T00:02:09.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.963+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21a80ce7f0 con 0x7f21a8072ea0 2024-04-25T00:02:09.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.963+0000 7f21a57fa640 1 -- 172.21.15.113:0/3019578089 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5194 (secure 0 0 0) 0x7f2198079ae0 con 0x7f21a8072ea0 2024-04-25T00:02:09.965 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:09.965 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3327449639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3327449639},{"type":"v1","addr":"172.21.15.113:6837","nonce":3327449639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:52.819867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6835/1818573977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1818573977},{"type":"v1","addr":"172.21.15.178:6835","nonce":1818573977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6690,"qdb_cluster":[6690]},"id":4}]} 2024-04-25T00:02:09.965 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 37 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f219406be30 msgr2=0x7f219406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f219406be30 0x7f219406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f21a80ceaa0 tx=0x7f219c007d30 comp rx=0 tx=0).stop 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a8072ea0 msgr2=0x7f21a80a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a8072ea0 0x7f21a80a9480 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f21980097a0 tx=0x7f21980046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 shutdown_connections 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f219406be30 0x7f219406e2f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a80b3af0 0x7f21a8142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21a815a6b0 0x7f21a813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 --2- 172.21.15.113:0/3019578089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a8072ea0 0x7f21a80a9480 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:09.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 >> 172.21.15.113:0/3019578089 conn(0x7f21a80b1a50 msgr2=0x7f21a80ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:09.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.965+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 shutdown_connections 2024-04-25T00:02:09.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:09.966+0000 7f21aec72640 1 -- 172.21.15.113:0/3019578089 wait complete. 2024-04-25T00:02:09.978 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:02:10.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.074+0000 7f5c4074f640 1 Processor -- start 2024-04-25T00:02:10.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.074+0000 7f5c4074f640 1 -- start start 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.074+0000 7f5c4074f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c3815e320 0x7f5c380d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.074+0000 7f5c4074f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d2290 0x7f5c380b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.074+0000 7f5c4074f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c380d3de0 con 0x7f5c380d0b00 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.074+0000 7f5c4074f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5c380d3410 con 0x7f5c3815e320 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.074+0000 7f5c4074f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5c380d2800 con 0x7f5c380d2290 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3e4eb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c3815e320 0x7f5c380d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3e4eb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c3815e320 0x7f5c380d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40150/0 (socket says 172.21.15.113:40150) 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3e4eb640 1 -- 172.21.15.113:0/4189297353 learned_addr learned my addr 172.21.15.113:0/4189297353 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3dcea640 1 --2- 172.21.15.113:0/4189297353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d2290 0x7f5c380b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3e4eb640 1 -- 172.21.15.113:0/4189297353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d2290 msgr2=0x7f5c380b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:10.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3e4eb640 1 --2- 172.21.15.113:0/4189297353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d2290 0x7f5c380b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3e4eb640 1 -- 172.21.15.113:0/4189297353 >> v1:172.21.15.178:6789/0 conn(0x7f5c380d0b00 legacy=0x7f5c380b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:10.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.075+0000 7f5c3e4eb640 1 -- 172.21.15.113:0/4189297353 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c380d2ae0 con 0x7f5c3815e320 2024-04-25T00:02:10.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c3e4eb640 1 --2- 172.21.15.113:0/4189297353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c380d38a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009870 tx=0x7f5c2802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1a8f598b0b4e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c3d4e9640 1 -- 172.21.15.113:0/4189297353 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28037070 con 0x7f5c3815e320 2024-04-25T00:02:10.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c3d4e9640 1 -- 172.21.15.113:0/4189297353 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2802ae50 con 0x7f5c3815e320 2024-04-25T00:02:10.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c3d4e9640 1 -- 172.21.15.113:0/4189297353 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28032860 con 0x7f5c3815e320 2024-04-25T00:02:10.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 -- 172.21.15.113:0/4189297353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 msgr2=0x7f5c380d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4189297353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c380d38a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009870 tx=0x7f5c2802a300 comp rx=0 tx=0).stop 2024-04-25T00:02:10.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 -- 172.21.15.113:0/4189297353 shutdown_connections 2024-04-25T00:02:10.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4189297353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d2290 0x7f5c380b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4189297353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c380d38a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 -- 172.21.15.113:0/4189297353 >> 172.21.15.113:0/4189297353 conn(0x7f5c380b5c20 msgr2=0x7f5c380b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 -- 172.21.15.113:0/4189297353 shutdown_connections 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.076+0000 7f5c4074f640 1 -- 172.21.15.113:0/4189297353 wait complete. 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.077+0000 7f5c4074f640 1 Processor -- start 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c4074f640 1 -- start start 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c4074f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d0b00 0x7f5c380c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c4074f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c380d2290 0x7f5c38154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c4074f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c38154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c4074f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5c380b9fe0 con 0x7f5c3815e320 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c4074f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c380d30a0 con 0x7f5c380d2290 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c4074f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3815e700 con 0x7f5c380d0b00 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c3ecec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c38154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c3ecec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c38154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40152/0 (socket says 172.21.15.113:40152) 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c3ecec640 1 -- 172.21.15.113:0/4021606175 learned_addr learned my addr 172.21.15.113:0/4021606175 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c3dcea640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c380d2290 0x7f5c38154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.078+0000 7f5c3e4eb640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d0b00 0x7f5c380c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3ecec640 1 -- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d0b00 msgr2=0x7f5c380c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3ecec640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d0b00 0x7f5c380c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3ecec640 1 -- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c380d2290 msgr2=0x7f5c38154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3ecec640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c380d2290 0x7f5c38154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3ecec640 1 -- 172.21.15.113:0/4021606175 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c380c9810 con 0x7f5c3815e320 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3dcea640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c380d2290 0x7f5c38154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3e4eb640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d0b00 0x7f5c380c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c3ecec640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c38154750 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c7d0 tx=0x7f5c3000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c2effd640 1 -- 172.21.15.113:0/4021606175 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30003b70 con 0x7f5c3815e320 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c2effd640 1 -- 172.21.15.113:0/4021606175 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c30003d30 con 0x7f5c3815e320 2024-04-25T00:02:10.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c380ccdc0 con 0x7f5c3815e320 2024-04-25T00:02:10.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c2effd640 1 -- 172.21.15.113:0/4021606175 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000aa90 con 0x7f5c3815e320 2024-04-25T00:02:10.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.079+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c380cd380 con 0x7f5c3815e320 2024-04-25T00:02:10.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.080+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c380691a0 con 0x7f5c3815e320 2024-04-25T00:02:10.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.080+0000 7f5c2effd640 1 -- 172.21.15.113:0/4021606175 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c3000ac30 con 0x7f5c3815e320 2024-04-25T00:02:10.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.081+0000 7f5c2effd640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c1c06be30 0x7f5c1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.081+0000 7f5c2effd640 1 -- 172.21.15.113:0/4021606175 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f5c3008add0 con 0x7f5c3815e320 2024-04-25T00:02:10.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.081+0000 7f5c3e4eb640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c1c06be30 0x7f5c1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.081+0000 7f5c3e4eb640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c1c06be30 0x7f5c1c06e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5c28034000 tx=0x7f5c2802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.086+0000 7f5c2effd640 1 -- 172.21.15.113:0/4021606175 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c30059720 con 0x7f5c3815e320 2024-04-25T00:02:10.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.293+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c3815d970 con 0x7f5c3815e320 2024-04-25T00:02:10.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.294+0000 7f5c2effd640 1 -- 172.21.15.113:0/4021606175 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5194 (secure 0 0 0) 0x7f5c3008c070 con 0x7f5c3815e320 2024-04-25T00:02:10.296 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:10.296 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3327449639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3327449639},{"type":"v1","addr":"172.21.15.113:6837","nonce":3327449639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:01:35.811705+0000","modified":"2024-04-25T00:01:52.819867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6690":{"gid":6690,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6835/1818573977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1818573977},{"type":"v1","addr":"172.21.15.178:6835","nonce":1818573977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6690,"qdb_cluster":[6690]},"id":4}]} 2024-04-25T00:02:10.296 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 37 2024-04-25T00:02:10.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c1c06be30 msgr2=0x7f5c1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c1c06be30 0x7f5c1c06e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5c28034000 tx=0x7f5c2802e000 comp rx=0 tx=0).stop 2024-04-25T00:02:10.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 msgr2=0x7f5c38154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c38154750 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c7d0 tx=0x7f5c3000cc90 comp rx=0 tx=0).stop 2024-04-25T00:02:10.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 shutdown_connections 2024-04-25T00:02:10.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c1c06be30 0x7f5c1c06e2f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c380d2290 0x7f5c38154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3815e320 0x7f5c38154750 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 --2- 172.21.15.113:0/4021606175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c380d0b00 0x7f5c380c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 >> 172.21.15.113:0/4021606175 conn(0x7f5c380b5c20 msgr2=0x7f5c380b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:10.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 shutdown_connections 2024-04-25T00:02:10.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.296+0000 7f5c4074f640 1 -- 172.21.15.113:0/4021606175 wait complete. 2024-04-25T00:02:10.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:10.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.403+0000 7f43b922a640 1 Processor -- start 2024-04-25T00:02:10.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.403+0000 7f43b922a640 1 -- start start 2024-04-25T00:02:10.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b922a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f43b415e320 0x7f43b40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b922a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43b40d2290 0x7f43b40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b922a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40d0b00 0x7f43b40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b922a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f43b40d14d0 con 0x7f43b40d2290 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b922a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f43b40d3410 con 0x7f43b415e320 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b922a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43b40d3de0 con 0x7f43b40d0b00 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43b40d2290 0x7f43b40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43b40d2290 0x7f43b40b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40158/0 (socket says 172.21.15.113:40158) 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b259c640 1 -- 172.21.15.113:0/2833570342 learned_addr learned my addr 172.21.15.113:0/2833570342 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b259c640 1 -- 172.21.15.113:0/2833570342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40d0b00 msgr2=0x7f43b40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b259c640 1 --2- 172.21.15.113:0/2833570342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40d0b00 0x7f43b40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b259c640 1 -- 172.21.15.113:0/2833570342 >> v2:172.21.15.178:3300/0 conn(0x7f43b415e320 msgr2=0x7f43b40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.404+0000 7f43b259c640 1 --2- 172.21.15.113:0/2833570342 >> v2:172.21.15.178:3300/0 conn(0x7f43b415e320 0x7f43b40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.405+0000 7f43b259c640 1 -- 172.21.15.113:0/2833570342 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b40d2800 con 0x7f43b40d2290 2024-04-25T00:02:10.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.405+0000 7f43b259c640 1 --2- 172.21.15.113:0/2833570342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d2290 0x7f43b40b9aa0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b6d00 tx=0x7f439c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd54baa256340c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.405+0000 7f43b1d9b640 1 -- 172.21.15.113:0/2833570342 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c037070 con 0x7f43b40d2290 2024-04-25T00:02:10.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.405+0000 7f43b1d9b640 1 -- 172.21.15.113:0/2833570342 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c032440 con 0x7f43b40d2290 2024-04-25T00:02:10.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.405+0000 7f43b1d9b640 1 -- 172.21.15.113:0/2833570342 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c032810 con 0x7f43b40d2290 2024-04-25T00:02:10.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 -- 172.21.15.113:0/2833570342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d2290 msgr2=0x7f43b40b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 --2- 172.21.15.113:0/2833570342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d2290 0x7f43b40b9aa0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b6d00 tx=0x7f439c02a270 comp rx=0 tx=0).stop 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 -- 172.21.15.113:0/2833570342 shutdown_connections 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 --2- 172.21.15.113:0/2833570342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40d0b00 0x7f43b40d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 --2- 172.21.15.113:0/2833570342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d2290 0x7f43b40b9aa0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 --2- 172.21.15.113:0/2833570342 >> v2:172.21.15.178:3300/0 conn(0x7f43b415e320 0x7f43b40d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 -- 172.21.15.113:0/2833570342 >> 172.21.15.113:0/2833570342 conn(0x7f43b40b5c20 msgr2=0x7f43b40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 -- 172.21.15.113:0/2833570342 shutdown_connections 2024-04-25T00:02:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.406+0000 7f43b922a640 1 -- 172.21.15.113:0/2833570342 wait complete. 2024-04-25T00:02:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.407+0000 7f43b922a640 1 Processor -- start 2024-04-25T00:02:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.407+0000 7f43b922a640 1 -- start start 2024-04-25T00:02:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b922a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d0b00 0x7f43b4154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b922a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b40d2290 0x7f43b4158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b922a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b415e320 0x7f43b40cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b922a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f43b40d30a0 con 0x7f43b40d0b00 2024-04-25T00:02:10.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b922a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43b40d2dc0 con 0x7f43b40d2290 2024-04-25T00:02:10.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b922a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43b40b9fe0 con 0x7f43b415e320 2024-04-25T00:02:10.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d0b00 0x7f43b4154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d0b00 0x7f43b4154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40162/0 (socket says 172.21.15.113:40162) 2024-04-25T00:02:10.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.408+0000 7f43b2d9d640 1 -- 172.21.15.113:0/4069740632 learned_addr learned my addr 172.21.15.113:0/4069740632 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:10.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b2d9d640 1 -- 172.21.15.113:0/4069740632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b415e320 msgr2=0x7f43b40cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b2d9d640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b415e320 0x7f43b40cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b2d9d640 1 -- 172.21.15.113:0/4069740632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b40d2290 msgr2=0x7f43b4158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b2d9d640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b40d2290 0x7f43b4158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b2d9d640 1 -- 172.21.15.113:0/4069740632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b41f41d0 con 0x7f43b40d0b00 2024-04-25T00:02:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b2d9d640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d0b00 0x7f43b4154160 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00b410 tx=0x7f43ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43ab7fe640 1 -- 172.21.15.113:0/4069740632 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac015020 con 0x7f43b40d0b00 2024-04-25T00:02:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43ab7fe640 1 -- 172.21.15.113:0/4069740632 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ac011850 con 0x7f43b40d0b00 2024-04-25T00:02:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b41f44b0 con 0x7f43b40d0b00 2024-04-25T00:02:10.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43ab7fe640 1 -- 172.21.15.113:0/4069740632 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac011c40 con 0x7f43b40d0b00 2024-04-25T00:02:10.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.409+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b415f8c0 con 0x7f43b40d0b00 2024-04-25T00:02:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.410+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b40691a0 con 0x7f43b40d0b00 2024-04-25T00:02:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.410+0000 7f43ab7fe640 1 -- 172.21.15.113:0/4069740632 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43ac018020 con 0x7f43b40d0b00 2024-04-25T00:02:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.411+0000 7f43ab7fe640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.411+0000 7f43ab7fe640 1 -- 172.21.15.113:0/4069740632 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f43ac08adf0 con 0x7f43b40d0b00 2024-04-25T00:02:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.411+0000 7f43b259c640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.411+0000 7f43b259c640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f439406be30 0x7f439406e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f439c034000 tx=0x7f439c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.416+0000 7f43ab7fe640 1 -- 172.21.15.113:0/4069740632 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f43ac01f020 con 0x7f43b40d0b00 2024-04-25T00:02:10.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.590+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43b40ce5a0 con 0x7f43b40d0b00 2024-04-25T00:02:10.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.591+0000 7f43ab7fe640 1 -- 172.21.15.113:0/4069740632 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16844 (secure 0 0 0) 0x7f43ac0172c0 con 0x7f43b40d0b00 2024-04-25T00:02:10.592 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:10.592 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:01:49.536503+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:01:34.014634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T00:01:34.775167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6834/3704852657":"2024-04-26T00:01:49.531195+0000","172.21.15.178:6832/3704852657":"2024-04-26T00:01:49.531195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:10.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f439406be30 msgr2=0x7f439406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f439406be30 0x7f439406e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f439c034000 tx=0x7f439c02c000 comp rx=0 tx=0).stop 2024-04-25T00:02:10.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d0b00 msgr2=0x7f43b4154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d0b00 0x7f43b4154160 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00b410 tx=0x7f43ac00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 shutdown_connections 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43b40d2290 0x7f43b4158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b415e320 0x7f43b40cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 --2- 172.21.15.113:0/4069740632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43b40d0b00 0x7f43b4154160 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.593+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 >> 172.21.15.113:0/4069740632 conn(0x7f43b40b5c20 msgr2=0x7f43b40ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.594+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 shutdown_connections 2024-04-25T00:02:10.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.594+0000 7f43b922a640 1 -- 172.21.15.113:0/4069740632 wait complete. 2024-04-25T00:02:10.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:02:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.702+0000 7f078e82c640 1 Processor -- start 2024-04-25T00:02:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.702+0000 7f078e82c640 1 -- start start 2024-04-25T00:02:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f078e82c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f07880b4a40 0x7f0788165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f078e82c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f078815f4f0 con 0x7f078815e2d0 2024-04-25T00:02:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f078e82c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f078815f7d0 con 0x7f07880b3080 2024-04-25T00:02:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f078e82c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f07880cde00 con 0x7f07880b4a40 2024-04-25T00:02:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0787fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f078815e2d0 0x7f078815ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54778/0 (socket says 172.21.15.113:54778) 2024-04-25T00:02:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0787fff640 1 -- 172.21.15.113:0/2482908057 learned_addr learned my addr 172.21.15.113:0/2482908057 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f078cdc9640 1 --2- 172.21.15.113:0/2482908057 >> v2:172.21.15.178:3300/0 conn(0x7f07880b4a40 0x7f0788165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125380623 0 0) 0x7f078815f4f0 con 0x7f078815e2d0 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0770003520 con 0x7f078815e2d0 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 150730618 0 0) 0x7f0770003520 con 0x7f078815e2d0 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 >> v2:172.21.15.178:3300/0 conn(0x7f07880b4a40 msgr2=0x7f0788165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 --2- 172.21.15.113:0/2482908057 >> v2:172.21.15.178:3300/0 conn(0x7f07880b4a40 0x7f0788165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 >> v1:172.21.15.178:6789/0 conn(0x7f07880b3080 legacy=0x7f07880b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078815fab0 con 0x7f078815e2d0 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0778001670 con 0x7f078815e2d0 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 >> v1:172.21.15.113:6789/0 conn(0x7f078815e2d0 legacy=0x7f078815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07700039b0 0x7f0770003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0770004800 0x7f0770004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07700051c0 0x7f077000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f07880cde00 con 0x7f07700051c0 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f077000c090 con 0x7f07700039b0 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f077000c290 con 0x7f0770004800 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.703+0000 7f078cdc9640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07700051c0 0x7f077000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f0787fff640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07700039b0 0x7f0770003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f07877fe640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0770004800 0x7f0770004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f078cdc9640 1 -- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0770004800 msgr2=0x7f0770004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f078cdc9640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0770004800 0x7f0770004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f078cdc9640 1 -- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07700039b0 msgr2=0x7f0770003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f078cdc9640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07700039b0 0x7f0770003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f078cdc9640 1 -- 172.21.15.113:0/2482908057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0770003520 con 0x7f07700051c0 2024-04-25T00:02:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f078cdc9640 1 -- 172.21.15.113:0/2482908057 reap_dead start 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.704+0000 7f078cdc9640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07700051c0 0x7f077000b9c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0780009f90 tx=0x7f0770005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7179ef6532c19ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780009c10 con 0x7f07700051c0 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0780036070 con 0x7f07700051c0 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f0786ffd640 1 -- 172.21.15.113:0/2482908057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780031860 con 0x7f07700051c0 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f078e82c640 1 -- 172.21.15.113:0/2482908057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07700051c0 msgr2=0x7f077000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f078e82c640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07700051c0 0x7f077000b9c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0780009f90 tx=0x7f0770005830 comp rx=0 tx=0).stop 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f078e82c640 1 -- 172.21.15.113:0/2482908057 shutdown_connections 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f078e82c640 1 --2- 172.21.15.113:0/2482908057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07700051c0 0x7f077000b9c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f078e82c640 1 -- 172.21.15.113:0/2482908057 >> 172.21.15.113:0/2482908057 conn(0x7f078806b0f0 msgr2=0x7f078806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f078e82c640 1 -- 172.21.15.113:0/2482908057 shutdown_connections 2024-04-25T00:02:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.705+0000 7f078e82c640 1 -- 172.21.15.113:0/2482908057 wait complete. 2024-04-25T00:02:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.706+0000 7f078e82c640 1 Processor -- start 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.706+0000 7f078e82c640 1 -- start start 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f078e82c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07700051c0 0x7f07881f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f078e82c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f078815c500 0x7f0788151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f078e82c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0788152170 0x7f078806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f078e82c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f078815e830 con 0x7f078815c500 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f078e82c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f078815e6b0 con 0x7f0788152170 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f078e82c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f078815e9b0 con 0x7f07700051c0 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f07877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f078815c500 0x7f0788151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f07877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f078815c500 0x7f0788151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40184/0 (socket says 172.21.15.113:40184) 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f07877fe640 1 -- 172.21.15.113:0/2267848158 learned_addr learned my addr 172.21.15.113:0/2267848158 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f078cdc9640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0788152170 0x7f078806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.707+0000 7f0787fff640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07700051c0 0x7f07881f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f07877fe640 1 -- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07700051c0 msgr2=0x7f07881f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f07877fe640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07700051c0 0x7f07881f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f07877fe640 1 -- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0788152170 msgr2=0x7f078806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f07877fe640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0788152170 0x7f078806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f07877fe640 1 -- 172.21.15.113:0/2267848158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07881f6a00 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f07877fe640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f078815c500 0x7f0788151c30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f077c00f620 tx=0x7f077c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f0784ff9640 1 -- 172.21.15.113:0/2267848158 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0052e0 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f0784ff9640 1 -- 172.21.15.113:0/2267848158 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c018070 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078815abd0 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f0784ff9640 1 -- 172.21.15.113:0/2267848158 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c003810 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.708+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07881530d0 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.709+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07880691a0 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.709+0000 7f0784ff9640 1 -- 172.21.15.113:0/2267848158 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f077c0039b0 con 0x7f078815c500 2024-04-25T00:02:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.710+0000 7f0784ff9640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f077406be60 0x7f077406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.710+0000 7f0787fff640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f077406be60 0x7f077406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.710+0000 7f0784ff9640 1 -- 172.21.15.113:0/2267848158 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f077c08ab70 con 0x7f078815c500 2024-04-25T00:02:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.710+0000 7f0787fff640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f077406be60 0x7f077406e320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0778001050 tx=0x7f0778009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.714+0000 7f0784ff9640 1 -- 172.21.15.113:0/2267848158 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07881f6a00 con 0x7f078815c500 2024-04-25T00:02:10.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.923+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0788072050 con 0x7f078815c500 2024-04-25T00:02:10.923 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:02:10.923+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:02:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.925+0000 7f0784ff9640 1 -- 172.21.15.113:0/2267848158 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f077c059550 con 0x7f078815c500 2024-04-25T00:02:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f077406be60 msgr2=0x7f077406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f077406be60 0x7f077406e320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0778001050 tx=0x7f0778009040 comp rx=0 tx=0).stop 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f078815c500 msgr2=0x7f0788151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f078815c500 0x7f0788151c30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f077c00f620 tx=0x7f077c00fae0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 shutdown_connections 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f077406be60 0x7f077406e320 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0788152170 0x7f078806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f078815c500 0x7f0788151c30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 --2- 172.21.15.113:0/2267848158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07700051c0 0x7f07881f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:10.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 >> 172.21.15.113:0/2267848158 conn(0x7f078806b0f0 msgr2=0x7f07880b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:10.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 shutdown_connections 2024-04-25T00:02:10.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:10.928+0000 7f078e82c640 1 -- 172.21.15.113:0/2267848158 wait complete. 2024-04-25T00:02:10.931 INFO:tasks.ceph.mds.d.smithi178.stderr: -2442> 2024-04-25T00:01:54.820+0000 7f7758728640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:02:10.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:02:10.960 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:02:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.037+0000 7f3d87db7640 1 Processor -- start 2024-04-25T00:02:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.038+0000 7f3d87db7640 1 -- start start 2024-04-25T00:02:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.038+0000 7f3d87db7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d8006fa40 0x7f3d800cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.038+0000 7f3d87db7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8006ca40 con 0x7f3d800cf3d0 2024-04-25T00:02:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.038+0000 7f3d87db7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8006b290 con 0x7f3d8015b070 2024-04-25T00:02:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.038+0000 7f3d87db7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8006b570 con 0x7f3d8006fa40 2024-04-25T00:02:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.038+0000 7f3d86354640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d8006fa40 0x7f3d800cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.039+0000 7f3d86354640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d8006fa40 0x7f3d800cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40192/0 (socket says 172.21.15.113:40192) 2024-04-25T00:02:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.039+0000 7f3d85352640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3d800cf3d0 0x7f3d8006f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54790/0 (socket says 172.21.15.113:54790) 2024-04-25T00:02:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.039+0000 7f3d86354640 1 -- 172.21.15.113:0/2232139440 learned_addr learned my addr 172.21.15.113:0/2232139440 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.039+0000 7f3d86354640 1 -- 172.21.15.113:0/2232139440 >> v1:172.21.15.178:6789/0 conn(0x7f3d8015b070 legacy=0x7f3d8006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:11.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.039+0000 7f3d86354640 1 -- 172.21.15.113:0/2232139440 >> v1:172.21.15.113:6789/0 conn(0x7f3d800cf3d0 legacy=0x7f3d8006f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.039+0000 7f3d86354640 1 -- 172.21.15.113:0/2232139440 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8006b850 con 0x7f3d8006fa40 2024-04-25T00:02:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.039+0000 7f3d86354640 1 --2- 172.21.15.113:0/2232139440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8006fa40 0x7f3d800cf860 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009e50 tx=0x7f3d7802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1df7c72076d0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d84b51640 1 -- 172.21.15.113:0/2232139440 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78036070 con 0x7f3d8006fa40 2024-04-25T00:02:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d84b51640 1 -- 172.21.15.113:0/2232139440 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d78031600 con 0x7f3d8006fa40 2024-04-25T00:02:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d84b51640 1 -- 172.21.15.113:0/2232139440 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780319f0 con 0x7f3d8006fa40 2024-04-25T00:02:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d87db7640 1 -- 172.21.15.113:0/2232139440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8006fa40 msgr2=0x7f3d800cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d87db7640 1 --2- 172.21.15.113:0/2232139440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8006fa40 0x7f3d800cf860 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009e50 tx=0x7f3d7802a7c0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d87db7640 1 -- 172.21.15.113:0/2232139440 shutdown_connections 2024-04-25T00:02:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d87db7640 1 --2- 172.21.15.113:0/2232139440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8006fa40 0x7f3d800cf860 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d87db7640 1 -- 172.21.15.113:0/2232139440 >> 172.21.15.113:0/2232139440 conn(0x7f3d800cd870 msgr2=0x7f3d800cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d87db7640 1 -- 172.21.15.113:0/2232139440 shutdown_connections 2024-04-25T00:02:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.040+0000 7f3d87db7640 1 -- 172.21.15.113:0/2232139440 wait complete. 2024-04-25T00:02:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.041+0000 7f3d87db7640 1 Processor -- start 2024-04-25T00:02:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.041+0000 7f3d87db7640 1 -- start start 2024-04-25T00:02:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d87db7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d8006fa40 0x7f3d80158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d87db7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d800cf3d0 0x7f3d800c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d87db7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8015b070 0x7f3d801e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d87db7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8015b450 con 0x7f3d8015b070 2024-04-25T00:02:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d87db7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8006be10 con 0x7f3d8006fa40 2024-04-25T00:02:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d87db7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d800c8910 con 0x7f3d800cf3d0 2024-04-25T00:02:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8015b070 0x7f3d801e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:11.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8015b070 0x7f3d801e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40196/0 (socket says 172.21.15.113:40196) 2024-04-25T00:02:11.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 -- 172.21.15.113:0/883391475 learned_addr learned my addr 172.21.15.113:0/883391475 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:11.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d85352640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d800cf3d0 0x7f3d800c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:11.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d85b53640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d8006fa40 0x7f3d80158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:11.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 -- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d800cf3d0 msgr2=0x7f3d800c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:11.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d800cf3d0 0x7f3d800c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 -- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d8006fa40 msgr2=0x7f3d80158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:11.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d8006fa40 0x7f3d80158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d86354640 1 -- 172.21.15.113:0/883391475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d800c8bf0 con 0x7f3d8015b070 2024-04-25T00:02:11.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.042+0000 7f3d85b53640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d8006fa40 0x7f3d80158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:11.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.043+0000 7f3d86354640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8015b070 0x7f3d801e7d10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3d7802a800 tx=0x7f3d78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:11.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.043+0000 7f3d767fc640 1 -- 172.21.15.113:0/883391475 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78036050 con 0x7f3d8015b070 2024-04-25T00:02:11.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.043+0000 7f3d767fc640 1 -- 172.21.15.113:0/883391475 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d78042070 con 0x7f3d8015b070 2024-04-25T00:02:11.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.043+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d800c8ed0 con 0x7f3d8015b070 2024-04-25T00:02:11.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.043+0000 7f3d767fc640 1 -- 172.21.15.113:0/883391475 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78009510 con 0x7f3d8015b070 2024-04-25T00:02:11.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.043+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d800c9f10 con 0x7f3d8015b070 2024-04-25T00:02:11.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.044+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d80068220 con 0x7f3d8015b070 2024-04-25T00:02:11.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.044+0000 7f3d767fc640 1 -- 172.21.15.113:0/883391475 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d78004470 con 0x7f3d8015b070 2024-04-25T00:02:11.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.045+0000 7f3d767fc640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d6406be60 0x7f3d6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.045+0000 7f3d85b53640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d6406be60 0x7f3d6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.045+0000 7f3d767fc640 1 -- 172.21.15.113:0/883391475 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7f3d780abc40 con 0x7f3d8015b070 2024-04-25T00:02:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.045+0000 7f3d85b53640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d6406be60 0x7f3d6406e320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c001660 tx=0x7f3d7c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.049+0000 7f3d767fc640 1 -- 172.21.15.113:0/883391475 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d7807a5d0 con 0x7f3d8015b070 2024-04-25T00:02:11.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.260+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3d800c4200 con 0x7f3d8015b070 2024-04-25T00:02:11.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.929+0000 7f3d767fc640 1 -- 172.21.15.113:0/883391475 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f3d78079ed0 con 0x7f3d8015b070 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d6406be60 msgr2=0x7f3d6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d6406be60 0x7f3d6406e320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c001660 tx=0x7f3d7c009040 comp rx=0 tx=0).stop 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8015b070 msgr2=0x7f3d801e7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8015b070 0x7f3d801e7d10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3d7802a800 tx=0x7f3d78004a40 comp rx=0 tx=0).stop 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 shutdown_connections 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d6406be60 0x7f3d6406e320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d8015b070 0x7f3d801e7d10 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d800cf3d0 0x7f3d800c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 --2- 172.21.15.113:0/883391475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d8006fa40 0x7f3d80158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:11.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.932+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 >> 172.21.15.113:0/883391475 conn(0x7f3d800cd870 msgr2=0x7f3d800af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:11.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.933+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 shutdown_connections 2024-04-25T00:02:11.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:11.933+0000 7f3d87db7640 1 -- 172.21.15.113:0/883391475 wait complete. 2024-04-25T00:02:11.946 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:02:12.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.049+0000 7fb8086e1640 1 Processor -- start 2024-04-25T00:02:12.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.049+0000 7fb8086e1640 1 -- start start 2024-04-25T00:02:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb8086e1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8000cc0b0 0x7fb800154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb8086e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800154d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb8086e1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8000b2f40 con 0x7fb8000b47a0 2024-04-25T00:02:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb8086e1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb8000b31e0 con 0x7fb8000cc0b0 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb8086e1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000b34c0 con 0x7fb8000b4b80 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb805c7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8000cc0b0 0x7fb800154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb80647d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb8000b47a0 0x7fb80015b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54800/0 (socket says 172.21.15.113:54800) 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb80647d640 1 -- 172.21.15.113:0/3555174823 learned_addr learned my addr 172.21.15.113:0/3555174823 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb806c7e640 1 --2- 172.21.15.113:0/3555174823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb805c7c640 1 -- 172.21.15.113:0/3555174823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 msgr2=0x7fb800154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb805c7c640 1 --2- 172.21.15.113:0/3555174823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb805c7c640 1 -- 172.21.15.113:0/3555174823 >> v1:172.21.15.113:6789/0 conn(0x7fb8000b47a0 legacy=0x7fb80015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb80547b640 1 -- 172.21.15.113:0/3555174823 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1754158492 0 0) 0x7fb8000b2f40 con 0x7fb8000b47a0 2024-04-25T00:02:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.050+0000 7fb805c7c640 1 -- 172.21.15.113:0/3555174823 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8000b37a0 con 0x7fb8000cc0b0 2024-04-25T00:02:12.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.051+0000 7fb805c7c640 1 --2- 172.21.15.113:0/3555174823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb800154830 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00097a0 tx=0x7fb7f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f51433ba2ed784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:12.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.051+0000 7fb80547b640 1 -- 172.21.15.113:0/3555174823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0037070 con 0x7fb8000cc0b0 2024-04-25T00:02:12.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.051+0000 7fb80547b640 1 -- 172.21.15.113:0/3555174823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f0032440 con 0x7fb8000cc0b0 2024-04-25T00:02:12.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.051+0000 7fb80547b640 1 -- 172.21.15.113:0/3555174823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0032810 con 0x7fb8000cc0b0 2024-04-25T00:02:12.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 -- 172.21.15.113:0/3555174823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 msgr2=0x7fb800154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 --2- 172.21.15.113:0/3555174823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb800154830 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00097a0 tx=0x7fb7f002a180 comp rx=0 tx=0).stop 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 -- 172.21.15.113:0/3555174823 shutdown_connections 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 --2- 172.21.15.113:0/3555174823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 --2- 172.21.15.113:0/3555174823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb800154830 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 -- 172.21.15.113:0/3555174823 >> 172.21.15.113:0/3555174823 conn(0x7fb8000b1b30 msgr2=0x7fb8000b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 -- 172.21.15.113:0/3555174823 shutdown_connections 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.052+0000 7fb8086e1640 1 -- 172.21.15.113:0/3555174823 wait complete. 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.053+0000 7fb8086e1640 1 Processor -- start 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.053+0000 7fb8086e1640 1 -- start start 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000b47a0 0x7fb800141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb8001597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000b4100 con 0x7fb8000cc0b0 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000b3f80 con 0x7fb8000b47a0 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000b4280 con 0x7fb8000b4b80 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb8001597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb8001597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40216/0 (socket says 172.21.15.113:40216) 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 -- 172.21.15.113:0/651434814 learned_addr learned my addr 172.21.15.113:0/651434814 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb80647d640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000b47a0 0x7fb800141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb805c7c640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 -- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 msgr2=0x7fb800145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 -- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000b47a0 msgr2=0x7fb800141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000b47a0 0x7fb800141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 -- 172.21.15.113:0/651434814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb800146250 con 0x7fb8000cc0b0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb806c7e640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb8001597e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009fd0 tx=0x7fb7f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb7f6ffd640 1 -- 172.21.15.113:0/651434814 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8014420 con 0x7fb8000cc0b0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb7f6ffd640 1 -- 172.21.15.113:0/651434814 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f8019070 con 0x7fb8000cc0b0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb7f6ffd640 1 -- 172.21.15.113:0/651434814 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8014800 con 0x7fb8000cc0b0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001464f0 con 0x7fb8000cc0b0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.054+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb800146ab0 con 0x7fb8000cc0b0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.055+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb800147290 con 0x7fb8000cc0b0 2024-04-25T00:02:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.056+0000 7fb7f6ffd640 1 -- 172.21.15.113:0/651434814 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7f80149a0 con 0x7fb8000cc0b0 2024-04-25T00:02:12.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.056+0000 7fb7f6ffd640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb7e406be30 0x7fb7e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:12.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.056+0000 7fb80647d640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb7e406be30 0x7fb7e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:12.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.056+0000 7fb7f6ffd640 1 -- 172.21.15.113:0/651434814 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7fb7f808b700 con 0x7fb8000cc0b0 2024-04-25T00:02:12.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.057+0000 7fb80647d640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb7e406be30 0x7fb7e406e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc000f60 tx=0x7fb7fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:12.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.061+0000 7fb7f6ffd640 1 -- 172.21.15.113:0/651434814 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb800147290 con 0x7fb8000cc0b0 2024-04-25T00:02:12.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.234+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb8000691a0 con 0x7fb8000cc0b0 2024-04-25T00:02:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.931+0000 7fb7f6ffd640 1 -- 172.21.15.113:0/651434814 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) ==== 181+0+0 (secure 0 0 0) 0x7fb7f805a090 con 0x7fb8000cc0b0 2024-04-25T00:02:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:02:12.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.935+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb7e406be30 msgr2=0x7fb7e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:12.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.935+0000 7fb8086e1640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb7e406be30 0x7fb7e406e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc000f60 tx=0x7fb7fc009040 comp rx=0 tx=0).stop 2024-04-25T00:02:12.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 msgr2=0x7fb8001597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:12.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb8001597e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009fd0 tx=0x7fb7f800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 shutdown_connections 2024-04-25T00:02:12.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb7e406be30 0x7fb7e406e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8000cc0b0 0x7fb8001597e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8000b4b80 0x7fb800145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 --2- 172.21.15.113:0/651434814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000b47a0 0x7fb800141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:12.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 >> 172.21.15.113:0/651434814 conn(0x7fb8000b1b30 msgr2=0x7fb80006df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:12.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 shutdown_connections 2024-04-25T00:02:12.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:12.936+0000 7fb8086e1640 1 -- 172.21.15.113:0/651434814 wait complete. 2024-04-25T00:02:12.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:02:13.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.058+0000 7fe157963640 1 Processor -- start 2024-04-25T00:02:13.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.058+0000 7fe157963640 1 -- start start 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe157963640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 0x7fe15006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe157963640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe150072f70 con 0x7fe150144a80 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe157963640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1500b2fd0 con 0x7fe1500cd630 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe157963640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe15006ea70 con 0x7fe15006e0d0 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe1556ff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe150144a80 0x7fe1500b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54810/0 (socket says 172.21.15.113:54810) 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe1556ff640 1 -- 172.21.15.113:0/2104435130 learned_addr learned my addr 172.21.15.113:0/2104435130 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe154efe640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 0x7fe15006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537968922 0 0) 0x7fe150072f70 con 0x7fe150144a80 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe13c003520 con 0x7fe150144a80 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 593117488 0 0) 0x7fe13c003520 con 0x7fe150144a80 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 msgr2=0x7fe15006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:13.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 0x7fe15006e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 >> v1:172.21.15.178:6789/0 conn(0x7fe1500cd630 legacy=0x7fe150072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1500b32b0 con 0x7fe150144a80 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe14c001670 con 0x7fe150144a80 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 >> v1:172.21.15.113:6789/0 conn(0x7fe150144a80 legacy=0x7fe1500b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe13c0039e0 0x7fe13c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 0x7fe13c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe13c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c00ba70 con 0x7fe13c004d40 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c00bbf0 con 0x7fe13c0039e0 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe13c00e000 con 0x7fe15006e0d0 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.059+0000 7fe155f00640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe13c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe1556ff640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe13c0039e0 0x7fe13c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe154efe640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 0x7fe13c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe155f00640 1 -- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 msgr2=0x7fe13c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe155f00640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15006e0d0 0x7fe13c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe155f00640 1 -- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe13c0039e0 msgr2=0x7fe13c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe155f00640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe13c0039e0 0x7fe13c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe155f00640 1 -- 172.21.15.113:0/2104435130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c003520 con 0x7fe13c004d40 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe155f00640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe13c0051c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe148001f60 tx=0x7fe13c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18af7b96eef8599f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.060+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148001b30 con 0x7fe13c004d40 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe148031620 con 0x7fe13c004d40 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe147fff640 1 -- 172.21.15.113:0/2104435130 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148031a10 con 0x7fe13c004d40 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe157963640 1 -- 172.21.15.113:0/2104435130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 msgr2=0x7fe13c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe157963640 1 --2- 172.21.15.113:0/2104435130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe13c0051c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe148001f60 tx=0x7fe13c005830 comp rx=0 tx=0).stop 2024-04-25T00:02:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe155f00640 1 -- 172.21.15.113:0/2104435130 reap_dead start 2024-04-25T00:02:13.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe157963640 1 -- 172.21.15.113:0/2104435130 shutdown_connections 2024-04-25T00:02:13.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe157963640 1 -- 172.21.15.113:0/2104435130 >> 172.21.15.113:0/2104435130 conn(0x7fe1500b1b30 msgr2=0x7fe1500b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:13.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe157963640 1 -- 172.21.15.113:0/2104435130 shutdown_connections 2024-04-25T00:02:13.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.061+0000 7fe157963640 1 -- 172.21.15.113:0/2104435130 wait complete. 2024-04-25T00:02:13.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.062+0000 7fe157963640 1 Processor -- start 2024-04-25T00:02:13.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.062+0000 7fe157963640 1 -- start start 2024-04-25T00:02:13.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe157963640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe15015cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe157963640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15013a7c0 0x7fe15015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe157963640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15015b9f0 0x7fe150157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe157963640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1500b4340 con 0x7fe13c004d40 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe157963640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1500b41c0 con 0x7fe15013a7c0 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe157963640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe15006e530 con 0x7fe15015b9f0 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe15015cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe15015cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40228/0 (socket says 172.21.15.113:40228) 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 -- 172.21.15.113:0/328477771 learned_addr learned my addr 172.21.15.113:0/328477771 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe154efe640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15013a7c0 0x7fe15015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe155f00640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15015b9f0 0x7fe150157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 -- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15015b9f0 msgr2=0x7fe150157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15015b9f0 0x7fe150157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 -- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15013a7c0 msgr2=0x7fe15015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15013a7c0 0x7fe15015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe1556ff640 1 -- 172.21.15.113:0/328477771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15015bee0 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.063+0000 7fe155f00640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15015b9f0 0x7fe150157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.064+0000 7fe1556ff640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe15015cf80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe14c007370 tx=0x7fe14c007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.064+0000 7fe145ffb640 1 -- 172.21.15.113:0/328477771 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c001630 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.064+0000 7fe145ffb640 1 -- 172.21.15.113:0/328477771 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14c018070 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.064+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15015c1c0 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.064+0000 7fe145ffb640 1 -- 172.21.15.113:0/328477771 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00fc00 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.064+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15015c780 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.065+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1500691a0 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.065+0000 7fe145ffb640 1 -- 172.21.15.113:0/328477771 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe14c01d460 con 0x7fe13c004d40 2024-04-25T00:02:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.066+0000 7fe145ffb640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe13406be30 0x7fe13406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.066+0000 7fe145ffb640 1 -- 172.21.15.113:0/328477771 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5443+0+0 (secure 0 0 0) 0x7fe14c08ace0 con 0x7fe13c004d40 2024-04-25T00:02:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.066+0000 7fe154efe640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe13406be30 0x7fe13406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.066+0000 7fe154efe640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe13406be30 0x7fe13406e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe140009bc0 tx=0x7fe140009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.070+0000 7fe145ffb640 1 -- 172.21.15.113:0/328477771 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe14c059820 con 0x7fe13c004d40 2024-04-25T00:02:13.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.241+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe1500b29a0 con 0x7fe13c004d40 2024-04-25T00:02:13.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.936+0000 7fe145ffb640 1 -- 172.21.15.113:0/328477771 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) ==== 169+0+0 (secure 0 0 0) 0x7fe14c059120 con 0x7fe13c004d40 2024-04-25T00:02:13.937 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe13406be30 msgr2=0x7fe13406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe13406be30 0x7fe13406e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe140009bc0 tx=0x7fe140009190 comp rx=0 tx=0).stop 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 msgr2=0x7fe15015cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe15015cf80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe14c007370 tx=0x7fe14c007770 comp rx=0 tx=0).stop 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 shutdown_connections 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe13406be30 0x7fe13406e2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15013a7c0 0x7fe15015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15015b9f0 0x7fe150157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 --2- 172.21.15.113:0/328477771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c004d40 0x7fe15015cf80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 >> 172.21.15.113:0/328477771 conn(0x7fe1500b1b30 msgr2=0x7fe1500b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 shutdown_connections 2024-04-25T00:02:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:13.939+0000 7fe157963640 1 -- 172.21.15.113:0/328477771 wait complete. 2024-04-25T00:02:13.956 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:02:14.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.058+0000 7f4e90aee640 1 Processor -- start 2024-04-25T00:02:14.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.058+0000 7f4e90aee640 1 -- start start 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e90aee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e8c13f050 0x7f4e8c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e90aee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4e8c0d04f0 0x7f4e8c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e90aee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e8c15d6b0 con 0x7f4e8c1604d0 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e90aee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8c0d09a0 con 0x7f4e8c13f050 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e90aee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8c15c6a0 con 0x7f4e8c0d04f0 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e8a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e8c13f050 0x7f4e8c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e8a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e8c13f050 0x7f4e8c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40234/0 (socket says 172.21.15.113:40234) 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e8a59c640 1 -- 172.21.15.113:0/1282923652 learned_addr learned my addr 172.21.15.113:0/1282923652 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e89d9b640 1 --2- 172.21.15.113:0/1282923652 >> v2:172.21.15.178:3300/0 conn(0x7f4e8c0d04f0 0x7f4e8c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e8a59c640 1 -- 172.21.15.113:0/1282923652 >> v2:172.21.15.178:3300/0 conn(0x7f4e8c0d04f0 msgr2=0x7f4e8c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:14.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e8a59c640 1 --2- 172.21.15.113:0/1282923652 >> v2:172.21.15.178:3300/0 conn(0x7f4e8c0d04f0 0x7f4e8c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e8a59c640 1 -- 172.21.15.113:0/1282923652 >> v1:172.21.15.178:6789/0 conn(0x7f4e8c1604d0 legacy=0x7f4e8c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:14.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.059+0000 7f4e8a59c640 1 -- 172.21.15.113:0/1282923652 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c15c8f0 con 0x7f4e8c13f050 2024-04-25T00:02:14.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.060+0000 7f4e8a59c640 1 --2- 172.21.15.113:0/1282923652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c13f050 0x7f4e8c0cffb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c06f460 tx=0x7f4e7c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf077ea4ae34db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:14.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.060+0000 7f4e8959a640 1 -- 172.21.15.113:0/1282923652 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c036070 con 0x7f4e8c13f050 2024-04-25T00:02:14.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.060+0000 7f4e8959a640 1 -- 172.21.15.113:0/1282923652 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e7c031600 con 0x7f4e8c13f050 2024-04-25T00:02:14.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.060+0000 7f4e8959a640 1 -- 172.21.15.113:0/1282923652 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0319d0 con 0x7f4e8c13f050 2024-04-25T00:02:14.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 -- 172.21.15.113:0/1282923652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c13f050 msgr2=0x7f4e8c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1282923652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c13f050 0x7f4e8c0cffb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c06f460 tx=0x7f4e7c02a660 comp rx=0 tx=0).stop 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 -- 172.21.15.113:0/1282923652 shutdown_connections 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1282923652 >> v2:172.21.15.178:3300/0 conn(0x7f4e8c0d04f0 0x7f4e8c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1282923652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c13f050 0x7f4e8c0cffb0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 -- 172.21.15.113:0/1282923652 >> 172.21.15.113:0/1282923652 conn(0x7f4e8c06b0f0 msgr2=0x7f4e8c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 -- 172.21.15.113:0/1282923652 shutdown_connections 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.061+0000 7f4e90aee640 1 -- 172.21.15.113:0/1282923652 wait complete. 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.062+0000 7f4e90aee640 1 Processor -- start 2024-04-25T00:02:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.062+0000 7f4e90aee640 1 -- start start 2024-04-25T00:02:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e90aee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8c0d04f0 0x7f4e8c0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e90aee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e8c13f050 0x7f4e8c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e90aee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c1604d0 0x7f4e8c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e90aee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c15d190 con 0x7f4e8c1604d0 2024-04-25T00:02:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e90aee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c15ceb0 con 0x7f4e8c0d04f0 2024-04-25T00:02:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e90aee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8c13f430 con 0x7f4e8c13f050 2024-04-25T00:02:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c1604d0 0x7f4e8c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c1604d0 0x7f4e8c156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40246/0 (socket says 172.21.15.113:40246) 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 -- 172.21.15.113:0/1380965295 learned_addr learned my addr 172.21.15.113:0/1380965295 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8a59c640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8c0d04f0 0x7f4e8c0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e89d9b640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e8c13f050 0x7f4e8c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 -- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e8c13f050 msgr2=0x7f4e8c155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e8c13f050 0x7f4e8c155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 -- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8c0d04f0 msgr2=0x7f4e8c0b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8c0d04f0 0x7f4e8c0b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.063+0000 7f4e8ad9d640 1 -- 172.21.15.113:0/1380965295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c156b30 con 0x7f4e8c1604d0 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.064+0000 7f4e8ad9d640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c1604d0 0x7f4e8c156160 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400b410 tx=0x7f4e8400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.064+0000 7f4e7affd640 1 -- 172.21.15.113:0/1380965295 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e84015030 con 0x7f4e8c1604d0 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.064+0000 7f4e7affd640 1 -- 172.21.15.113:0/1380965295 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e84004020 con 0x7f4e8c1604d0 2024-04-25T00:02:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.064+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c1e7510 con 0x7f4e8c1604d0 2024-04-25T00:02:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.064+0000 7f4e7affd640 1 -- 172.21.15.113:0/1380965295 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e84004410 con 0x7f4e8c1604d0 2024-04-25T00:02:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.064+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c1e7ad0 con 0x7f4e8c1604d0 2024-04-25T00:02:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.065+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8c0691a0 con 0x7f4e8c1604d0 2024-04-25T00:02:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.065+0000 7f4e7affd640 1 -- 172.21.15.113:0/1380965295 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e84018070 con 0x7f4e8c1604d0 2024-04-25T00:02:14.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.066+0000 7f4e7affd640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e6c06be30 0x7f4e6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.066+0000 7f4e7affd640 1 -- 172.21.15.113:0/1380965295 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5029+0+0 (secure 0 0 0) 0x7f4e8408a6a0 con 0x7f4e8c1604d0 2024-04-25T00:02:14.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.066+0000 7f4e8a59c640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e6c06be30 0x7f4e6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.066+0000 7f4e8a59c640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e6c06be30 0x7f4e6c06e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c033000 tx=0x7f4e7c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.070+0000 7f4e7affd640 1 -- 172.21.15.113:0/1380965295 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e840592f0 con 0x7f4e8c1604d0 2024-04-25T00:02:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.244+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4e8c0d0e50 con 0x7f4e8c1604d0 2024-04-25T00:02:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.247+0000 7f4e7affd640 1 -- 172.21.15.113:0/1380965295 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f4e84058bf0 con 0x7f4e8c1604d0 2024-04-25T00:02:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e6c06be30 msgr2=0x7f4e6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e6c06be30 0x7f4e6c06e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c033000 tx=0x7f4e7c009000 comp rx=0 tx=0).stop 2024-04-25T00:02:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c1604d0 msgr2=0x7f4e8c156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c1604d0 0x7f4e8c156160 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400b410 tx=0x7f4e8400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 shutdown_connections 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e6c06be30 0x7f4e6c06e2f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e8c1604d0 0x7f4e8c156160 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e8c13f050 0x7f4e8c155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 --2- 172.21.15.113:0/1380965295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8c0d04f0 0x7f4e8c0b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 >> 172.21.15.113:0/1380965295 conn(0x7f4e8c06b0f0 msgr2=0x7f4e8c151e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 shutdown_connections 2024-04-25T00:02:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.250+0000 7f4e90aee640 1 -- 172.21.15.113:0/1380965295 wait complete. 2024-04-25T00:02:14.261 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T00:02:14.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.356+0000 7ffbbc15a640 1 Processor -- start 2024-04-25T00:02:14.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.357+0000 7ffbbc15a640 1 -- start start 2024-04-25T00:02:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.357+0000 7ffbbc15a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbbc15a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffbb406e510 0x7ffbb40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbbc15a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb4144cf0 con 0x7ffbb4144910 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbbc15a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40b2f30 con 0x7ffbb406e510 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbbc15a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40b3210 con 0x7ffbb40b3fd0 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbba6f7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffbb406e510 0x7ffbb40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbb96f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbb9ef6640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ffbb4144910 0x7ffbb40b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:57922/0 (socket says 172.21.15.113:57922) 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbb9ef6640 1 -- 172.21.15.113:0/4104105472 learned_addr learned my addr 172.21.15.113:0/4104105472 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbb8ef4640 1 -- 172.21.15.113:0/4104105472 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 148641226 0 0) 0x7ffbb4144cf0 con 0x7ffbb4144910 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbb8ef4640 1 -- 172.21.15.113:0/4104105472 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba0005730 con 0x7ffbb4144910 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbba6f7640 1 -- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 msgr2=0x7ffbb406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbba6f7640 1 --2- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbba6f7640 1 -- 172.21.15.113:0/4104105472 >> v1:172.21.15.178:6789/0 conn(0x7ffbb4144910 legacy=0x7ffbb40b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbba6f7640 1 -- 172.21.15.113:0/4104105472 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b34f0 con 0x7ffbb406e510 2024-04-25T00:02:14.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.358+0000 7ffbb96f5640 1 --2- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.359+0000 7ffbba6f7640 1 --2- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 0x7ffbb40cd500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009d90 tx=0x7ffbac02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3473113d0b15ebbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.359+0000 7ffbb8ef4640 1 -- 172.21.15.113:0/4104105472 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac007740 con 0x7ffbb406e510 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.359+0000 7ffbb8ef4640 1 -- 172.21.15.113:0/4104105472 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac0078e0 con 0x7ffbb406e510 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.359+0000 7ffbb8ef4640 1 -- 172.21.15.113:0/4104105472 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac007cb0 con 0x7ffbb406e510 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 -- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 msgr2=0x7ffbb40cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 0x7ffbb40cd500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009d90 tx=0x7ffbac02a660 comp rx=0 tx=0).stop 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 -- 172.21.15.113:0/4104105472 shutdown_connections 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 0x7ffbb40cd500 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/4104105472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 -- 172.21.15.113:0/4104105472 >> 172.21.15.113:0/4104105472 conn(0x7ffbb40b1a50 msgr2=0x7ffbb40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 -- 172.21.15.113:0/4104105472 shutdown_connections 2024-04-25T00:02:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.360+0000 7ffbbc15a640 1 -- 172.21.15.113:0/4104105472 wait complete. 2024-04-25T00:02:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.361+0000 7ffbbc15a640 1 Processor -- start 2024-04-25T00:02:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.361+0000 7ffbbc15a640 1 -- start start 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.361+0000 7ffbbc15a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 0x7ffbb413e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.361+0000 7ffbbc15a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb40a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbbc15a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4144910 0x7ffbb40a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbbc15a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b4da0 con 0x7ffbb4144910 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbbc15a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b4410 con 0x7ffbb406e510 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbbc15a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40b4f20 con 0x7ffbb40b3fd0 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4144910 0x7ffbb40a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4144910 0x7ffbb40a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40262/0 (socket says 172.21.15.113:40262) 2024-04-25T00:02:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 -- 172.21.15.113:0/3109806172 learned_addr learned my addr 172.21.15.113:0/3109806172 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbb9ef6640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 0x7ffbb413e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbb96f5640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb40a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 -- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 msgr2=0x7ffbb40a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb40a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 -- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 msgr2=0x7ffbb413e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 0x7ffbb413e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 -- 172.21.15.113:0/3109806172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb415a6c0 con 0x7ffbb4144910 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbba6f7640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4144910 0x7ffbb40a99c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009eb0 tx=0x7ffbac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffba67fc640 1 -- 172.21.15.113:0/3109806172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac040070 con 0x7ffbb4144910 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffba67fc640 1 -- 172.21.15.113:0/3109806172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac031bc0 con 0x7ffbb4144910 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb415a9a0 con 0x7ffbb4144910 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffba67fc640 1 -- 172.21.15.113:0/3109806172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac009450 con 0x7ffbb4144910 2024-04-25T00:02:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.362+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb415af60 con 0x7ffbb4144910 2024-04-25T00:02:14.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.363+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4071ff0 con 0x7ffbb4144910 2024-04-25T00:02:14.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.364+0000 7ffba67fc640 1 -- 172.21.15.113:0/3109806172 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbac039040 con 0x7ffbb4144910 2024-04-25T00:02:14.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.364+0000 7ffba67fc640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb9806be30 0x7ffb9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:14.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.365+0000 7ffbb9ef6640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb9806be30 0x7ffb9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:14.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.365+0000 7ffba67fc640 1 -- 172.21.15.113:0/3109806172 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5029+0+0 (secure 0 0 0) 0x7ffbac0ac1c0 con 0x7ffbb4144910 2024-04-25T00:02:14.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.365+0000 7ffbb9ef6640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb9806be30 0x7ffb9806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005880 tx=0x7ffbb0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:14.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.369+0000 7ffba67fc640 1 -- 172.21.15.113:0/3109806172 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffbb4071ff0 con 0x7ffbb4144910 2024-04-25T00:02:14.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.615+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) -- 0x7ffbb40691a0 con 0x7ffbb4144910 2024-04-25T00:02:14.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.938+0000 7ffba67fc640 1 -- 172.21.15.113:0/3109806172 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v207) ==== 143+0+0 (secure 0 0 0) 0x7ffbac07a7c0 con 0x7ffbb4144910 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb9806be30 msgr2=0x7ffb9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb9806be30 0x7ffb9806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005880 tx=0x7ffbb0009040 comp rx=0 tx=0).stop 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4144910 msgr2=0x7ffbb40a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4144910 0x7ffbb40a99c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009eb0 tx=0x7ffbac004a40 comp rx=0 tx=0).stop 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 shutdown_connections 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb9806be30 0x7ffb9806e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4144910 0x7ffbb40a99c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb40b3fd0 0x7ffbb40a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 --2- 172.21.15.113:0/3109806172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb406e510 0x7ffbb413e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 >> 172.21.15.113:0/3109806172 conn(0x7ffbb40b1a50 msgr2=0x7ffbb4143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 shutdown_connections 2024-04-25T00:02:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:14.940+0000 7ffbbc15a640 1 -- 172.21.15.113:0/3109806172 wait complete. 2024-04-25T00:02:14.952 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T00:02:14.952 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T00:02:14.952 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T00:02:14.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:15.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.049+0000 7fbcf675a640 1 Processor -- start 2024-04-25T00:02:15.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.049+0000 7fbcf675a640 1 -- start start 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbcf675a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbcf675a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbcf006e510 0x7fbcf00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbcf675a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf0144cf0 con 0x7fbcf0144910 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbcf675a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf00b2f30 con 0x7fbcf006e510 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbcf675a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf00b3210 con 0x7fbcf00b3fd0 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbcf4cf7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbcf006e510 0x7fbcf00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbceffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbcf0144910 0x7fbcf00b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54814/0 (socket says 172.21.15.113:54814) 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbceffff640 1 -- 172.21.15.113:0/1322414717 learned_addr learned my addr 172.21.15.113:0/1322414717 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.050+0000 7fbcef7fe640 1 --2- 172.21.15.113:0/1322414717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbcf4cf7640 1 -- 172.21.15.113:0/1322414717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 msgr2=0x7fbcf006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbcf4cf7640 1 --2- 172.21.15.113:0/1322414717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbcf4cf7640 1 -- 172.21.15.113:0/1322414717 >> v1:172.21.15.113:6789/0 conn(0x7fbcf0144910 legacy=0x7fbcf00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbceeffd640 1 -- 172.21.15.113:0/1322414717 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310220482 0 0) 0x7fbcf0144cf0 con 0x7fbcf0144910 2024-04-25T00:02:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbcf4cf7640 1 -- 172.21.15.113:0/1322414717 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf00b34f0 con 0x7fbcf006e510 2024-04-25T00:02:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbcef7fe640 1 --2- 172.21.15.113:0/1322414717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbcf4cf7640 1 --2- 172.21.15.113:0/1322414717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf00cd500 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009d90 tx=0x7fbce802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914ab8913c0e2f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbceeffd640 1 -- 172.21.15.113:0/1322414717 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8007740 con 0x7fbcf006e510 2024-04-25T00:02:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbceeffd640 1 -- 172.21.15.113:0/1322414717 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce80078e0 con 0x7fbcf006e510 2024-04-25T00:02:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.051+0000 7fbceeffd640 1 -- 172.21.15.113:0/1322414717 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8007cb0 con 0x7fbcf006e510 2024-04-25T00:02:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 -- 172.21.15.113:0/1322414717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 msgr2=0x7fbcf00cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1322414717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf00cd500 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009d90 tx=0x7fbce802a660 comp rx=0 tx=0).stop 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 -- 172.21.15.113:0/1322414717 shutdown_connections 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1322414717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf00cd500 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1322414717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 -- 172.21.15.113:0/1322414717 >> 172.21.15.113:0/1322414717 conn(0x7fbcf00b1a50 msgr2=0x7fbcf00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 -- 172.21.15.113:0/1322414717 shutdown_connections 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.052+0000 7fbcf675a640 1 -- 172.21.15.113:0/1322414717 wait complete. 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.053+0000 7fbcf675a640 1 Processor -- start 2024-04-25T00:02:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.053+0000 7fbcf675a640 1 -- start start 2024-04-25T00:02:15.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcf675a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf013f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcf675a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf0157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcf675a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf0144910 0x7fbcf0072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcf675a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf00b4da0 con 0x7fbcf006e510 2024-04-25T00:02:15.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcf675a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf00b4410 con 0x7fbcf0144910 2024-04-25T00:02:15.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcf675a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf00b4f20 con 0x7fbcf00b3fd0 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf013f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf013f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40288/0 (socket says 172.21.15.113:40288) 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 -- 172.21.15.113:0/1676340243 learned_addr learned my addr 172.21.15.113:0/1676340243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcef7fe640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf0157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcf4cf7640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf0144910 0x7fbcf0072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 -- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 msgr2=0x7fbcf0157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf0157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 -- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf0144910 msgr2=0x7fbcf0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf0144910 0x7fbcf0072260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 -- 172.21.15.113:0/1676340243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf0159500 con 0x7fbcf006e510 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbcef7fe640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf0157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.054+0000 7fbceffff640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf013f850 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c430 tx=0x7fbce000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.055+0000 7fbcecff9640 1 -- 172.21.15.113:0/1676340243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00074e0 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.055+0000 7fbcecff9640 1 -- 172.21.15.113:0/1676340243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce00047e0 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.055+0000 7fbcecff9640 1 -- 172.21.15.113:0/1676340243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004bd0 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.055+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf0158980 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.055+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf0158f40 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.056+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf00691a0 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.056+0000 7fbcecff9640 1 -- 172.21.15.113:0/1676340243 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbce0011070 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.057+0000 7fbcecff9640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.057+0000 7fbcecff9640 1 -- 172.21.15.113:0/1676340243 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5029+0+0 (secure 0 0 0) 0x7fbce008a750 con 0x7fbcf006e510 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.057+0000 7fbcef7fe640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.057+0000 7fbcef7fe640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbce4006fd0 tx=0x7fbce4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.061+0000 7fbcecff9640 1 -- 172.21.15.113:0/1676340243 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbce0059430 con 0x7fbcf006e510 2024-04-25T00:02:15.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.236+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcf00cc2c0 con 0x7fbcf006e510 2024-04-25T00:02:15.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.237+0000 7fbcecff9640 1 -- 172.21.15.113:0/1676340243 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+13202 (secure 0 0 0) 0x7fbce0058d30 con 0x7fbcf006e510 2024-04-25T00:02:15.237 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:15.237 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":67,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:02:13.931996+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/1818573977":"2024-04-26T00:02:10.924570+0000","172.21.15.178:6833/1818573977":"2024-04-26T00:02:10.924570+0000","172.21.15.178:6834/3704852657":"2024-04-26T00:01:49.531195+0000","172.21.15.178:6832/3704852657":"2024-04-26T00:01:49.531195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:15.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcdc06be30 msgr2=0x7fbcdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbce4006fd0 tx=0x7fbce4008040 comp rx=0 tx=0).stop 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 msgr2=0x7fbcf013f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf013f850 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c430 tx=0x7fbce000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 shutdown_connections 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf0144910 0x7fbcf0072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcf00b3fd0 0x7fbcf0157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 --2- 172.21.15.113:0/1676340243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf006e510 0x7fbcf013f850 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 >> 172.21.15.113:0/1676340243 conn(0x7fbcf00b1a50 msgr2=0x7fbcf00b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 shutdown_connections 2024-04-25T00:02:15.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.239+0000 7fbcf675a640 1 -- 172.21.15.113:0/1676340243 wait complete. 2024-04-25T00:02:15.251 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:02:15.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:02:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.350+0000 7f09ed2d6640 1 Processor -- start 2024-04-25T00:02:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.350+0000 7f09ed2d6640 1 -- start start 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09ed2d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09ed2d6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e806bdc0 con 0x7f09e80cf3f0 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09ed2d6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e80c5700 con 0x7f09e8156600 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09ed2d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80b2280 con 0x7f09e806b800 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09e77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f09e80cf3f0 0x7f09e815fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54828/0 (socket says 172.21.15.113:54828) 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09e77fe640 1 -- 172.21.15.113:0/600579293 learned_addr learned my addr 172.21.15.113:0/600579293 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09e67fc640 1 --2- 172.21.15.113:0/600579293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09e5ffb640 1 -- 172.21.15.113:0/600579293 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787031761 0 0) 0x7f09e806bdc0 con 0x7f09e80cf3f0 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.351+0000 7f09e5ffb640 1 -- 172.21.15.113:0/600579293 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09d0003520 con 0x7f09e80cf3f0 2024-04-25T00:02:15.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.352+0000 7f09e5ffb640 1 -- 172.21.15.113:0/600579293 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218273517 0 0) 0x7f09e80c5700 con 0x7f09e8156600 2024-04-25T00:02:15.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.352+0000 7f09e5ffb640 1 -- 172.21.15.113:0/600579293 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09e806bdc0 con 0x7f09e8156600 2024-04-25T00:02:15.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.352+0000 7f09e67fc640 1 -- 172.21.15.113:0/600579293 >> v1:172.21.15.178:6789/0 conn(0x7f09e8156600 legacy=0x7f09e806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.352+0000 7f09e67fc640 1 -- 172.21.15.113:0/600579293 >> v1:172.21.15.113:6789/0 conn(0x7f09e80cf3f0 legacy=0x7f09e815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.352+0000 7f09e67fc640 1 -- 172.21.15.113:0/600579293 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e80c59e0 con 0x7f09e806b800 2024-04-25T00:02:15.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.352+0000 7f09e67fc640 1 --2- 172.21.15.113:0/600579293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e815d0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009800 tx=0x7f09dc02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e606b16eeabed52c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09e5ffb640 1 -- 172.21.15.113:0/600579293 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc037070 con 0x7f09e806b800 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09e5ffb640 1 -- 172.21.15.113:0/600579293 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09dc032440 con 0x7f09e806b800 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09e5ffb640 1 -- 172.21.15.113:0/600579293 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0327f0 con 0x7f09e806b800 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09ed2d6640 1 -- 172.21.15.113:0/600579293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 msgr2=0x7f09e815d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/600579293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e815d0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009800 tx=0x7f09dc02a230 comp rx=0 tx=0).stop 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09ed2d6640 1 -- 172.21.15.113:0/600579293 shutdown_connections 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/600579293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e815d0b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09ed2d6640 1 -- 172.21.15.113:0/600579293 >> 172.21.15.113:0/600579293 conn(0x7f09e806fb60 msgr2=0x7f09e806b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09ed2d6640 1 -- 172.21.15.113:0/600579293 shutdown_connections 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.353+0000 7f09ed2d6640 1 -- 172.21.15.113:0/600579293 wait complete. 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.354+0000 7f09ed2d6640 1 Processor -- start 2024-04-25T00:02:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.354+0000 7f09ed2d6640 1 -- start start 2024-04-25T00:02:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09ed2d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09ed2d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e80cf3f0 0x7f09e8155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09ed2d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e8156600 0x7f09e8141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09ed2d6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8160700 con 0x7f09e80cf3f0 2024-04-25T00:02:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09ed2d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81569e0 con 0x7f09e8156600 2024-04-25T00:02:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09ed2d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09e8160880 con 0x7f09e806b800 2024-04-25T00:02:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e80cf3f0 0x7f09e8155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e80cf3f0 0x7f09e8155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40292/0 (socket says 172.21.15.113:40292) 2024-04-25T00:02:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 -- 172.21.15.113:0/2675327623 learned_addr learned my addr 172.21.15.113:0/2675327623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e77fe640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e8156600 0x7f09e8141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e6ffd640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 -- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 msgr2=0x7f09e814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e814ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 -- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e8156600 msgr2=0x7f09e8141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e8156600 0x7f09e8141eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.355+0000 7f09e67fc640 1 -- 172.21.15.113:0/2675327623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e814cd50 con 0x7f09e80cf3f0 2024-04-25T00:02:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.356+0000 7f09e67fc640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e80cf3f0 0x7f09e8155400 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02a270 tx=0x7f09dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.356+0000 7f09cf7fe640 1 -- 172.21.15.113:0/2675327623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc041020 con 0x7f09e80cf3f0 2024-04-25T00:02:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.356+0000 7f09cf7fe640 1 -- 172.21.15.113:0/2675327623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09dc03d6e0 con 0x7f09e80cf3f0 2024-04-25T00:02:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.356+0000 7f09cf7fe640 1 -- 172.21.15.113:0/2675327623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc037040 con 0x7f09e80cf3f0 2024-04-25T00:02:15.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.356+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e814d030 con 0x7f09e80cf3f0 2024-04-25T00:02:15.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.356+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e814d5f0 con 0x7f09e80cf3f0 2024-04-25T00:02:15.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.357+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e80691a0 con 0x7f09e80cf3f0 2024-04-25T00:02:15.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.357+0000 7f09cf7fe640 1 -- 172.21.15.113:0/2675327623 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09dc044020 con 0x7f09e80cf3f0 2024-04-25T00:02:15.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.357+0000 7f09cf7fe640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09d406be30 0x7f09d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.358+0000 7f09cf7fe640 1 -- 172.21.15.113:0/2675327623 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5029+0+0 (secure 0 0 0) 0x7f09dc0ab330 con 0x7f09e80cf3f0 2024-04-25T00:02:15.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.358+0000 7f09e6ffd640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09d406be30 0x7f09d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.358+0000 7f09e6ffd640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09d406be30 0x7f09d406e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f09d8001c20 tx=0x7f09d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.362+0000 7f09cf7fe640 1 -- 172.21.15.113:0/2675327623 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09dc04b030 con 0x7f09e80cf3f0 2024-04-25T00:02:15.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.534+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09e8074ea0 con 0x7f09e80cf3f0 2024-04-25T00:02:15.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.535+0000 7f09cf7fe640 1 -- 172.21.15.113:0/2675327623 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f09dc079f90 con 0x7f09e80cf3f0 2024-04-25T00:02:15.537 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:02:15.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09d406be30 msgr2=0x7f09d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09d406be30 0x7f09d406e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f09d8001c20 tx=0x7f09d8009040 comp rx=0 tx=0).stop 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e80cf3f0 msgr2=0x7f09e8155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e80cf3f0 0x7f09e8155400 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02a270 tx=0x7f09dc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 shutdown_connections 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09d406be30 0x7f09d406e2f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e8156600 0x7f09e8141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e80cf3f0 0x7f09e8155400 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 --2- 172.21.15.113:0/2675327623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09e806b800 0x7f09e814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 >> 172.21.15.113:0/2675327623 conn(0x7f09e806fb60 msgr2=0x7f09e806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 shutdown_connections 2024-04-25T00:02:15.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.537+0000 7f09ed2d6640 1 -- 172.21.15.113:0/2675327623 wait complete. 2024-04-25T00:02:15.548 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T00:02:15.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.644+0000 7f95047e7640 1 Processor -- start 2024-04-25T00:02:15.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.644+0000 7f95047e7640 1 -- start start 2024-04-25T00:02:15.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f95047e7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f94fc144910 0x7f94fc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f95047e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f95047e7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94fc144cf0 con 0x7f94fc0b5680 2024-04-25T00:02:15.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f95047e7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f94fc0b4510 con 0x7f94fc144910 2024-04-25T00:02:15.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f95047e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94fc0b2f30 con 0x7f94fc0b5240 2024-04-25T00:02:15.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f9501d82640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f9502583640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f94fc144910 0x7f94fc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f9501d82640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:44790/0 (socket says 172.21.15.113:44790) 2024-04-25T00:02:15.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.645+0000 7f9501d82640 1 -- 172.21.15.113:0/1880292517 learned_addr learned my addr 172.21.15.113:0/1880292517 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.646+0000 7f9501d82640 1 -- 172.21.15.113:0/1880292517 >> v2:172.21.15.178:3300/0 conn(0x7f94fc144910 msgr2=0x7f94fc0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.646+0000 7f9501d82640 1 --2- 172.21.15.113:0/1880292517 >> v2:172.21.15.178:3300/0 conn(0x7f94fc144910 0x7f94fc0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.646+0000 7f9501d82640 1 -- 172.21.15.113:0/1880292517 >> v1:172.21.15.178:6789/0 conn(0x7f94fc0b5680 legacy=0x7f94fc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.646+0000 7f9501d82640 1 -- 172.21.15.113:0/1880292517 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc0b3210 con 0x7f94fc0b5240 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.646+0000 7f9501d82640 1 --2- 172.21.15.113:0/1880292517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc06dfd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009870 tx=0x7f94ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29a6be3a5db5ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f9501581640 1 -- 172.21.15.113:0/1880292517 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec037070 con 0x7f94fc0b5240 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f9501581640 1 -- 172.21.15.113:0/1880292517 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec032440 con 0x7f94fc0b5240 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f9501581640 1 -- 172.21.15.113:0/1880292517 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec032810 con 0x7f94fc0b5240 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 -- 172.21.15.113:0/1880292517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 msgr2=0x7f94fc06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 --2- 172.21.15.113:0/1880292517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc06dfd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009870 tx=0x7f94ec02a270 comp rx=0 tx=0).stop 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 -- 172.21.15.113:0/1880292517 shutdown_connections 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 --2- 172.21.15.113:0/1880292517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc06dfd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 --2- 172.21.15.113:0/1880292517 >> v2:172.21.15.178:3300/0 conn(0x7f94fc144910 0x7f94fc0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 -- 172.21.15.113:0/1880292517 >> 172.21.15.113:0/1880292517 conn(0x7f94fc0b1a50 msgr2=0x7f94fc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 -- 172.21.15.113:0/1880292517 shutdown_connections 2024-04-25T00:02:15.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.647+0000 7f95047e7640 1 -- 172.21.15.113:0/1880292517 wait complete. 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.648+0000 7f95047e7640 1 Processor -- start 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.648+0000 7f95047e7640 1 -- start start 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f95047e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f95047e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94fc0b5680 0x7f94fc0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f95047e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94fc144910 0x7f94fc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f95047e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc0b4da0 con 0x7f94fc0b5680 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f95047e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc0b37d0 con 0x7f94fc144910 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f95047e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94fc0b4f20 con 0x7f94fc0b5240 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f9501d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94fc0b5680 0x7f94fc0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f9501d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94fc0b5680 0x7f94fc0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40308/0 (socket says 172.21.15.113:40308) 2024-04-25T00:02:15.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f9501d82640 1 -- 172.21.15.113:0/452179574 learned_addr learned my addr 172.21.15.113:0/452179574 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f9502583640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.649+0000 7f9502d84640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94fc144910 0x7f94fc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f9501d82640 1 -- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 msgr2=0x7f94fc138770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f9501d82640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc138770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f9501d82640 1 -- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94fc144910 msgr2=0x7f94fc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f9501d82640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94fc144910 0x7f94fc15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f9501d82640 1 -- 172.21.15.113:0/452179574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc0ce7a0 con 0x7f94fc0b5680 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f9501d82640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94fc0b5680 0x7f94fc0ce030 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f94ec02a2b0 tx=0x7f94ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f94f2ffd640 1 -- 172.21.15.113:0/452179574 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec037040 con 0x7f94fc0b5680 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f94f2ffd640 1 -- 172.21.15.113:0/452179574 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec042070 con 0x7f94fc0b5680 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc0cea80 con 0x7f94fc0b5680 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f94f2ffd640 1 -- 172.21.15.113:0/452179574 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec030880 con 0x7f94fc0b5680 2024-04-25T00:02:15.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.650+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc0cf040 con 0x7f94fc0b5680 2024-04-25T00:02:15.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.651+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc0691a0 con 0x7f94fc0b5680 2024-04-25T00:02:15.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.651+0000 7f94f2ffd640 1 -- 172.21.15.113:0/452179574 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94ec030a20 con 0x7f94fc0b5680 2024-04-25T00:02:15.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.652+0000 7f94f2ffd640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f94e006be30 0x7f94e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:15.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.652+0000 7f94f2ffd640 1 -- 172.21.15.113:0/452179574 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5029+0+0 (secure 0 0 0) 0x7f94ec0ab1f0 con 0x7f94fc0b5680 2024-04-25T00:02:15.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.652+0000 7f9502583640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f94e006be30 0x7f94e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.652+0000 7f9502583640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f94e006be30 0x7f94e006e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f94f80113e0 tx=0x7f94f800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.656+0000 7f94f2ffd640 1 -- 172.21.15.113:0/452179574 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f94ec079880 con 0x7f94fc0b5680 2024-04-25T00:02:15.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.896+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) -- 0x7f94fc0b4a10 con 0x7f94fc0b5680 2024-04-25T00:02:15.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.944+0000 7f94f2ffd640 1 -- 172.21.15.113:0/452179574 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v208) ==== 161+0+0 (secure 0 0 0) 0x7f94ec07db80 con 0x7f94fc0b5680 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f94e006be30 msgr2=0x7f94e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f94e006be30 0x7f94e006e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f94f80113e0 tx=0x7f94f800b040 comp rx=0 tx=0).stop 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94fc0b5680 msgr2=0x7f94fc0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94fc0b5680 0x7f94fc0ce030 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f94ec02a2b0 tx=0x7f94ec0046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 shutdown_connections 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f94e006be30 0x7f94e006e2f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94fc144910 0x7f94fc15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94fc0b5680 0x7f94fc0ce030 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 --2- 172.21.15.113:0/452179574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94fc0b5240 0x7f94fc138770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 >> 172.21.15.113:0/452179574 conn(0x7f94fc0b1a50 msgr2=0x7f94fc15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:15.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 shutdown_connections 2024-04-25T00:02:15.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:15.946+0000 7f95047e7640 1 -- 172.21.15.113:0/452179574 wait complete. 2024-04-25T00:02:15.958 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:02:16.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.057+0000 7fdb54612640 1 Processor -- start 2024-04-25T00:02:16.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.057+0000 7fdb54612640 1 -- start start 2024-04-25T00:02:16.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb54612640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb4c0b3080 0x7fdb4c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb54612640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdb4c15e6b0 0x7fdb4c0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb54612640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4c15f4f0 con 0x7fdb4c15e2d0 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb54612640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdb4c0cde00 con 0x7fdb4c0b3080 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb54612640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdb4c15f7d0 con 0x7fdb4c15e6b0 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb51bad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb4c0b3080 0x7fdb4c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb523ae640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdb4c15e2d0 0x7fdb4c15ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54838/0 (socket says 172.21.15.113:54838) 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb523ae640 1 -- 172.21.15.113:0/2139696869 learned_addr learned my addr 172.21.15.113:0/2139696869 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb51bad640 1 -- 172.21.15.113:0/2139696869 >> v2:172.21.15.178:3300/0 conn(0x7fdb4c15e6b0 msgr2=0x7fdb4c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb51bad640 1 --2- 172.21.15.113:0/2139696869 >> v2:172.21.15.178:3300/0 conn(0x7fdb4c15e6b0 0x7fdb4c0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb51bad640 1 -- 172.21.15.113:0/2139696869 >> v1:172.21.15.113:6789/0 conn(0x7fdb4c15e2d0 legacy=0x7fdb4c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb51bad640 1 -- 172.21.15.113:0/2139696869 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c15fab0 con 0x7fdb4c0b3080 2024-04-25T00:02:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.058+0000 7fdb513ac640 1 -- 172.21.15.113:0/2139696869 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328654654 0 0) 0x7fdb4c15f4f0 con 0x7fdb4c15e2d0 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.059+0000 7fdb51bad640 1 --2- 172.21.15.113:0/2139696869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c0b4330 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0d0f20 tx=0x7fdb3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1898f04161c10a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.059+0000 7fdb513ac640 1 -- 172.21.15.113:0/2139696869 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c037070 con 0x7fdb4c0b3080 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.059+0000 7fdb513ac640 1 -- 172.21.15.113:0/2139696869 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3c032440 con 0x7fdb4c0b3080 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.059+0000 7fdb513ac640 1 -- 172.21.15.113:0/2139696869 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c032810 con 0x7fdb4c0b3080 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 -- 172.21.15.113:0/2139696869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 msgr2=0x7fdb4c0b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 --2- 172.21.15.113:0/2139696869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c0b4330 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0d0f20 tx=0x7fdb3c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 -- 172.21.15.113:0/2139696869 shutdown_connections 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 --2- 172.21.15.113:0/2139696869 >> v2:172.21.15.178:3300/0 conn(0x7fdb4c15e6b0 0x7fdb4c0b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 --2- 172.21.15.113:0/2139696869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c0b4330 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 -- 172.21.15.113:0/2139696869 >> 172.21.15.113:0/2139696869 conn(0x7fdb4c06b0f0 msgr2=0x7fdb4c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 -- 172.21.15.113:0/2139696869 shutdown_connections 2024-04-25T00:02:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.060+0000 7fdb54612640 1 -- 172.21.15.113:0/2139696869 wait complete. 2024-04-25T00:02:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.061+0000 7fdb54612640 1 Processor -- start 2024-04-25T00:02:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.061+0000 7fdb54612640 1 -- start start 2024-04-25T00:02:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb4c15e2d0 0x7fdb4c0aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb4c15e6b0 0x7fdb4c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c06d020 con 0x7fdb4c0b3080 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c06cea0 con 0x7fdb4c15e6b0 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4c06d1a0 con 0x7fdb4c15e2d0 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40330/0 (socket says 172.21.15.113:40330) 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 -- 172.21.15.113:0/3268414909 learned_addr learned my addr 172.21.15.113:0/3268414909 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb51bad640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb4c15e2d0 0x7fdb4c0aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb52baf640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb4c15e6b0 0x7fdb4c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 -- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb4c15e2d0 msgr2=0x7fdb4c0aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb4c15e2d0 0x7fdb4c0aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 -- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb4c15e6b0 msgr2=0x7fdb4c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb4c15e6b0 0x7fdb4c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 -- 172.21.15.113:0/3268414909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c0af2c0 con 0x7fdb4c0b3080 2024-04-25T00:02:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb51bad640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb4c15e2d0 0x7fdb4c0aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb523ae640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c159f80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c500 tx=0x7fdb4400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb42ffd640 1 -- 172.21.15.113:0/3268414909 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44018070 con 0x7fdb4c0b3080 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb42ffd640 1 -- 172.21.15.113:0/3268414909 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb44007580 con 0x7fdb4c0b3080 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c15a5a0 con 0x7fdb4c0b3080 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb42ffd640 1 -- 172.21.15.113:0/3268414909 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44004a20 con 0x7fdb4c0b3080 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.062+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c15ab60 con 0x7fdb4c0b3080 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.063+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4c0691a0 con 0x7fdb4c0b3080 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.064+0000 7fdb42ffd640 1 -- 172.21.15.113:0/3268414909 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb4401b070 con 0x7fdb4c0b3080 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.064+0000 7fdb42ffd640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb3006be30 0x7fdb3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.064+0000 7fdb51bad640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb3006be30 0x7fdb3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.064+0000 7fdb42ffd640 1 -- 172.21.15.113:0/3268414909 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5029+0+0 (secure 0 0 0) 0x7fdb44089e30 con 0x7fdb4c0b3080 2024-04-25T00:02:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.065+0000 7fdb51bad640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb3006be30 0x7fdb3006e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c034000 tx=0x7fdb3c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.068+0000 7fdb42ffd640 1 -- 172.21.15.113:0/3268414909 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdb44058b10 con 0x7fdb4c0b3080 2024-04-25T00:02:16.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.242+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdb4c072050 con 0x7fdb4c0b3080 2024-04-25T00:02:16.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.247+0000 7fdb42ffd640 1 -- 172.21.15.113:0/3268414909 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fdb44058410 con 0x7fdb4c0b3080 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb3006be30 msgr2=0x7fdb3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb3006be30 0x7fdb3006e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c034000 tx=0x7fdb3c02e000 comp rx=0 tx=0).stop 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 msgr2=0x7fdb4c159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c159f80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c500 tx=0x7fdb4400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 shutdown_connections 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb3006be30 0x7fdb3006e2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb4c15e6b0 0x7fdb4c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb4c15e2d0 0x7fdb4c0aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 --2- 172.21.15.113:0/3268414909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb4c0b3080 0x7fdb4c159f80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 >> 172.21.15.113:0/3268414909 conn(0x7fdb4c06b0f0 msgr2=0x7fdb4c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:16.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 shutdown_connections 2024-04-25T00:02:16.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.249+0000 7fdb54612640 1 -- 172.21.15.113:0/3268414909 wait complete. 2024-04-25T00:02:16.260 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:02:16.261 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:02:16.302 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:02:16.345 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:02:16.345 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:02:16.345 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:16.362 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:02:16.363 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:02:16.363 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:16.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:02:16.403 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:02:16.403 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:16.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:02:16.418 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:02:16.418 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:16.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:02:16.460 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:02:16.460 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:02:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.556+0000 7f11064d4640 1 Processor -- start 2024-04-25T00:02:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.556+0000 7f11064d4640 1 -- start start 2024-04-25T00:02:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f11064d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 0x7f11000cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f11064d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1100144cf0 con 0x7f11000b3fd0 2024-04-25T00:02:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f11064d4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11000b2f30 con 0x7f1100144910 2024-04-25T00:02:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f11064d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11000b3210 con 0x7f110006e6e0 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10ff7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f11000b3fd0 0x7f110006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54842/0 (socket says 172.21.15.113:54842) 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10ff7fe640 1 -- 172.21.15.113:0/3355339786 learned_addr learned my addr 172.21.15.113:0/3355339786 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f1104a71640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 0x7f11000cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2177898544 0 0) 0x7f1100144cf0 con 0x7f11000b3fd0 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10e8003520 con 0x7f11000b3fd0 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176221687 0 0) 0x7f11000b2f30 con 0x7f1100144910 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1100144cf0 con 0x7f1100144910 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1398876760 0 0) 0x7f10e8003520 con 0x7f11000b3fd0 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 msgr2=0x7f11000cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 0x7f11000cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 >> v1:172.21.15.178:6789/0 conn(0x7f1100144910 legacy=0x7f11000b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11000b34f0 con 0x7f11000b3fd0 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.557+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f10f4001670 con 0x7f11000b3fd0 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 >> v1:172.21.15.113:6789/0 conn(0x7f11000b3fd0 legacy=0x7f110006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10feffd640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e8003b70 0x7f10e8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10feffd640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e8004800 0x7f10e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10feffd640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 0x7f10e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f11000b3210 con 0x7f10e8004800 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10e800baa0 con 0x7f10e8003b70 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10e800bca0 con 0x7f110006e6e0 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10ff7fe640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e8004800 0x7f10e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10fffff640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e8003b70 0x7f10e8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f1104a71640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 0x7f10e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10ff7fe640 1 -- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 msgr2=0x7f10e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10ff7fe640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f110006e6e0 0x7f10e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10ff7fe640 1 -- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e8003b70 msgr2=0x7f10e8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10ff7fe640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e8003b70 0x7f10e8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.558+0000 7f10ff7fe640 1 -- 172.21.15.113:0/3355339786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e8003520 con 0x7f10e8004800 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f10ff7fe640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e8004800 0x7f10e8004c80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f10f4007fc0 tx=0x7f10e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fcc175f21008c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4038070 con 0x7f10e8004800 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f402f070 con 0x7f10e8004800 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f10feffd640 1 -- 172.21.15.113:0/3355339786 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4007be0 con 0x7f10e8004800 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f11064d4640 1 -- 172.21.15.113:0/3355339786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e8004800 msgr2=0x7f10e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f11064d4640 1 --2- 172.21.15.113:0/3355339786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e8004800 0x7f10e8004c80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f10f4007fc0 tx=0x7f10e8005830 comp rx=0 tx=0).stop 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f1104a71640 1 -- 172.21.15.113:0/3355339786 reap_dead start 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f11064d4640 1 -- 172.21.15.113:0/3355339786 shutdown_connections 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f11064d4640 1 -- 172.21.15.113:0/3355339786 >> 172.21.15.113:0/3355339786 conn(0x7f11000b1a50 msgr2=0x7f11000b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:16.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f11064d4640 1 -- 172.21.15.113:0/3355339786 shutdown_connections 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.559+0000 7f11064d4640 1 -- 172.21.15.113:0/3355339786 wait complete. 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.560+0000 7f11064d4640 1 Processor -- start 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f11064d4640 1 -- start start 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f11064d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e8004800 0x7f11001e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f11064d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11000ae130 0x7f11000ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f11064d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11000aead0 0x7f110013a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f11064d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f11000b4da0 con 0x7f11000aead0 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f11064d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11000b4440 con 0x7f11000ae130 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f11064d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11000b4f20 con 0x7f10e8004800 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f1104a71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11000aead0 0x7f110013a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f1104a71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11000aead0 0x7f110013a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40348/0 (socket says 172.21.15.113:40348) 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.561+0000 7f1104a71640 1 -- 172.21.15.113:0/1637854208 learned_addr learned my addr 172.21.15.113:0/1637854208 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f10fffff640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e8004800 0x7f11001e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f10ff7fe640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11000ae130 0x7f11000ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f1104a71640 1 -- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e8004800 msgr2=0x7f11001e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f1104a71640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e8004800 0x7f11001e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f1104a71640 1 -- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11000ae130 msgr2=0x7f11000ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f1104a71640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11000ae130 0x7f11000ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f1104a71640 1 -- 172.21.15.113:0/1637854208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110013cd00 con 0x7f11000aead0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f10fffff640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e8004800 0x7f11001e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f1104a71640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11000aead0 0x7f110013a780 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f11000cfc70 tx=0x7f10f80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f10fcff9640 1 -- 172.21.15.113:0/1637854208 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8003990 con 0x7f11000aead0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f10fcff9640 1 -- 172.21.15.113:0/1637854208 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f801b070 con 0x7f11000aead0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110013bf50 con 0x7f11000aead0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f10fcff9640 1 -- 172.21.15.113:0/1637854208 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8003b70 con 0x7f11000aead0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.562+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11000ad930 con 0x7f11000aead0 2024-04-25T00:02:16.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.563+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11000691a0 con 0x7f11000aead0 2024-04-25T00:02:16.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.564+0000 7f10fcff9640 1 -- 172.21.15.113:0/1637854208 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10f8005d10 con 0x7f11000aead0 2024-04-25T00:02:16.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.564+0000 7f10fcff9640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10ec06be30 0x7f10ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.564+0000 7f10fcff9640 1 -- 172.21.15.113:0/1637854208 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5029+0+0 (secure 0 0 0) 0x7f10f808afd0 con 0x7f11000aead0 2024-04-25T00:02:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.564+0000 7f10fffff640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10ec06be30 0x7f10ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.564+0000 7f10fffff640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10ec06be30 0x7f10ec06e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f10f00035f0 tx=0x7f10f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.568+0000 7f10fcff9640 1 -- 172.21.15.113:0/1637854208 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f110013cd00 con 0x7f11000aead0 2024-04-25T00:02:16.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.778+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11000731a0 con 0x7f11000aead0 2024-04-25T00:02:16.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.778+0000 7f10fcff9640 1 -- 172.21.15.113:0/1637854208 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3881 (secure 0 0 0) 0x7f10f8014350 con 0x7f11000aead0 2024-04-25T00:02:16.779 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:16.779 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5441,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3883289312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3883289312},{"type":"v1","addr":"172.21.15.178:6834","nonce":3883289312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2397796719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2397796719},{"type":"v1","addr":"172.21.15.113:6835","nonce":2397796719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3327449639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3327449639},{"type":"v1","addr":"172.21.15.113:6837","nonce":3327449639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2954431085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2954431085},{"type":"v1","addr":"172.21.15.178:6835","nonce":2954431085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-04-25T00:02:16.779 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 39 2024-04-25T00:02:16.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10ec06be30 msgr2=0x7f10ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10ec06be30 0x7f10ec06e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f10f00035f0 tx=0x7f10f0009040 comp rx=0 tx=0).stop 2024-04-25T00:02:16.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11000aead0 msgr2=0x7f110013a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11000aead0 0x7f110013a780 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f11000cfc70 tx=0x7f10f80115a0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 shutdown_connections 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f10ec06be30 0x7f10ec06e2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11000ae130 0x7f11000ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11000aead0 0x7f110013a780 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 --2- 172.21.15.113:0/1637854208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e8004800 0x7f11001e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 >> 172.21.15.113:0/1637854208 conn(0x7f11000b1a50 msgr2=0x7f110006dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 shutdown_connections 2024-04-25T00:02:16.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.780+0000 7f11064d4640 1 -- 172.21.15.113:0/1637854208 wait complete. 2024-04-25T00:02:16.792 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:02:16.792 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:02:16.793 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:02:16.793 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:16.793 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:02:16.793 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:02:16.793 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:02:16.793 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:16.793 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:02:16.794 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:02:16.794 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:02:16.794 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:16.794 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:02:16.794 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:02:16.794 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:02:16.794 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:02:16.802 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:02:16.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:02:16.803 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:02:16.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:02:16.806 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:02:16.806 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:02:16.807 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:02:16.807 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:02:16.845 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:02:16.846 DEBUG:teuthology.parallel:result is None 2024-04-25T00:02:16.846 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:02:16.846 DEBUG:teuthology.parallel:result is None 2024-04-25T00:02:16.849 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:02:16.849 DEBUG:teuthology.parallel:result is None 2024-04-25T00:02:16.850 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:02:16.850 DEBUG:teuthology.parallel:result is None 2024-04-25T00:02:16.850 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:02:16.918 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:02:16.918 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:02:16.931 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:02:16.931 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:02:16.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.953+0000 7f08e882a640 1 Processor -- start 2024-04-25T00:02:16.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.953+0000 7f08e882a640 1 -- start start 2024-04-25T00:02:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e882a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08e415e510 0x7f08e40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e882a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e40b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e882a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e40d2e00 con 0x7f08e40d28c0 2024-04-25T00:02:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e882a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f08e40d30e0 con 0x7f08e415e510 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e882a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08e40d33c0 con 0x7f08e40b63b0 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08e415e510 0x7f08e40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08e415e510 0x7f08e40d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40392/0 (socket says 172.21.15.113:40392) 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e259c640 1 -- 172.21.15.113:0/2444425497 learned_addr learned my addr 172.21.15.113:0/2444425497 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.954+0000 7f08e2d9d640 1 --2- 172.21.15.113:0/2444425497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e40b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.955+0000 7f08e259c640 1 -- 172.21.15.113:0/2444425497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 msgr2=0x7f08e40b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.955+0000 7f08e259c640 1 --2- 172.21.15.113:0/2444425497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e40b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.955+0000 7f08e259c640 1 -- 172.21.15.113:0/2444425497 >> v1:172.21.15.178:6789/0 conn(0x7f08e40d28c0 legacy=0x7f08e40b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.955+0000 7f08e259c640 1 -- 172.21.15.113:0/2444425497 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e40d36a0 con 0x7f08e415e510 2024-04-25T00:02:16.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.955+0000 7f08e259c640 1 --2- 172.21.15.113:0/2444425497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e415e510 0x7f08e40d2380 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009870 tx=0x7f08d402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515116707fed6807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.960+0000 7f08e1d9b640 1 -- 172.21.15.113:0/2444425497 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4037070 con 0x7f08e415e510 2024-04-25T00:02:16.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.960+0000 7f08e1d9b640 1 -- 172.21.15.113:0/2444425497 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d4032440 con 0x7f08e415e510 2024-04-25T00:02:16.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.960+0000 7f08e1d9b640 1 -- 172.21.15.113:0/2444425497 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4032810 con 0x7f08e415e510 2024-04-25T00:02:16.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 -- 172.21.15.113:0/2444425497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e415e510 msgr2=0x7f08e40d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 --2- 172.21.15.113:0/2444425497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e415e510 0x7f08e40d2380 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009870 tx=0x7f08d402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 -- 172.21.15.113:0/2444425497 shutdown_connections 2024-04-25T00:02:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 --2- 172.21.15.113:0/2444425497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e40b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 --2- 172.21.15.113:0/2444425497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e415e510 0x7f08e40d2380 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 -- 172.21.15.113:0/2444425497 >> 172.21.15.113:0/2444425497 conn(0x7f08e406b720 msgr2=0x7f08e40d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 -- 172.21.15.113:0/2444425497 shutdown_connections 2024-04-25T00:02:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.961+0000 7f08e882a640 1 -- 172.21.15.113:0/2444425497 wait complete. 2024-04-25T00:02:16.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.962+0000 7f08e882a640 1 Processor -- start 2024-04-25T00:02:16.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.962+0000 7f08e882a640 1 -- start start 2024-04-25T00:02:16.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e882a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e41528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e882a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e40d28c0 0x7f08e41f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e882a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e415e510 0x7f08e4161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e882a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40b9dd0 con 0x7f08e40d28c0 2024-04-25T00:02:16.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e882a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40d3ea0 con 0x7f08e415e510 2024-04-25T00:02:16.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e882a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08e40b7900 con 0x7f08e40b63b0 2024-04-25T00:02:16.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e40d28c0 0x7f08e41f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e40d28c0 0x7f08e41f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40406/0 (socket says 172.21.15.113:40406) 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08dbfff640 1 -- 172.21.15.113:0/3217482471 learned_addr learned my addr 172.21.15.113:0/3217482471 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e259c640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e41528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.963+0000 7f08e2d9d640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e415e510 0x7f08e4161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.964+0000 7f08dbfff640 1 -- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 msgr2=0x7f08e41528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.964+0000 7f08dbfff640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e41528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.964+0000 7f08dbfff640 1 -- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e415e510 msgr2=0x7f08e4161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.964+0000 7f08dbfff640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e415e510 0x7f08e4161000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.964+0000 7f08dbfff640 1 -- 172.21.15.113:0/3217482471 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e415ff60 con 0x7f08e40d28c0 2024-04-25T00:02:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.964+0000 7f08dbfff640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e40d28c0 0x7f08e41f4f80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c400 tx=0x7f08cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.967+0000 7f08daffd640 1 -- 172.21.15.113:0/3217482471 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0052d0 con 0x7f08e40d28c0 2024-04-25T00:02:16.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.967+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e4160240 con 0x7f08e40d28c0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.968+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e4160800 con 0x7f08e40d28c0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.968+0000 7f08daffd640 1 -- 172.21.15.113:0/3217482471 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc005470 con 0x7f08e40d28c0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.968+0000 7f08daffd640 1 -- 172.21.15.113:0/3217482471 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc007240 con 0x7f08e40d28c0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.969+0000 7f08daffd640 1 -- 172.21.15.113:0/3217482471 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08cc0073e0 con 0x7f08e40d28c0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.969+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e40691a0 con 0x7f08e40d28c0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.969+0000 7f08daffd640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08c406be30 0x7f08c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.969+0000 7f08daffd640 1 -- 172.21.15.113:0/3217482471 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5029+0+0 (secure 0 0 0) 0x7f08cc089f90 con 0x7f08e40d28c0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.970+0000 7f08e259c640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08c406be30 0x7f08c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:16.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.970+0000 7f08e259c640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08c406be30 0x7f08c406e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08d4034000 tx=0x7f08d402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:16.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:16.974+0000 7f08daffd640 1 -- 172.21.15.113:0/3217482471 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f08cc0585a0 con 0x7f08e40d28c0 2024-04-25T00:02:17.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.158+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f08e40d47c0 con 0x7f08e40d28c0 2024-04-25T00:02:17.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.976+0000 7f08daffd640 1 -- 172.21.15.113:0/3217482471 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) ==== 97+0+0 (secure 0 0 0) 0x7f08cc05c8a0 con 0x7f08e40d28c0 2024-04-25T00:02:17.976 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:02:17.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08c406be30 msgr2=0x7f08c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:17.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08c406be30 0x7f08c406e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08d4034000 tx=0x7f08d402e000 comp rx=0 tx=0).stop 2024-04-25T00:02:17.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e40d28c0 msgr2=0x7f08e41f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e40d28c0 0x7f08e41f4f80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c400 tx=0x7f08cc00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 shutdown_connections 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f08c406be30 0x7f08c406e2f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e415e510 0x7f08e4161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08e40d28c0 0x7f08e41f4f80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 --2- 172.21.15.113:0/3217482471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08e40b63b0 0x7f08e41528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 >> 172.21.15.113:0/3217482471 conn(0x7f08e406b720 msgr2=0x7f08e415d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 shutdown_connections 2024-04-25T00:02:17.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:17.979+0000 7f08e882a640 1 -- 172.21.15.113:0/3217482471 wait complete. 2024-04-25T00:02:17.994 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:02:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.097+0000 7f885e34f640 1 Processor -- start 2024-04-25T00:02:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.097+0000 7f885e34f640 1 -- start start 2024-04-25T00:02:18.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f885e34f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8858144910 0x7f88580b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f885e34f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88580b5240 0x7f885806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f885e34f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8858144cf0 con 0x7f88580b5680 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f885e34f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f88580b4510 con 0x7f88580b5240 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f885e34f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f88580b2f30 con 0x7f8858144910 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f88577fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88580b5240 0x7f885806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f88577fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88580b5240 0x7f885806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54262/0 (socket says 172.21.15.113:54262) 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f88577fe640 1 -- 172.21.15.113:0/3261496113 learned_addr learned my addr 172.21.15.113:0/3261496113 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f8857fff640 1 --2- 172.21.15.113:0/3261496113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8858144910 0x7f88580b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f88577fe640 1 -- 172.21.15.113:0/3261496113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8858144910 msgr2=0x7f88580b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f88577fe640 1 --2- 172.21.15.113:0/3261496113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8858144910 0x7f88580b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f88577fe640 1 -- 172.21.15.113:0/3261496113 >> v1:172.21.15.178:6789/0 conn(0x7f88580b5680 legacy=0x7f88580cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.098+0000 7f88577fe640 1 -- 172.21.15.113:0/3261496113 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88580b3210 con 0x7f88580b5240 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f88577fe640 1 --2- 172.21.15.113:0/3261496113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f885806dfd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f884c009870 tx=0x7f884c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ada785c0ee7439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f8856ffd640 1 -- 172.21.15.113:0/3261496113 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c037070 con 0x7f88580b5240 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f8856ffd640 1 -- 172.21.15.113:0/3261496113 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f884c032440 con 0x7f88580b5240 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f8856ffd640 1 -- 172.21.15.113:0/3261496113 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c032810 con 0x7f88580b5240 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f885e34f640 1 -- 172.21.15.113:0/3261496113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 msgr2=0x7f885806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f885e34f640 1 --2- 172.21.15.113:0/3261496113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f885806dfd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f884c009870 tx=0x7f884c02a210 comp rx=0 tx=0).stop 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f885e34f640 1 -- 172.21.15.113:0/3261496113 shutdown_connections 2024-04-25T00:02:18.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f885e34f640 1 --2- 172.21.15.113:0/3261496113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f885806dfd0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f885e34f640 1 --2- 172.21.15.113:0/3261496113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8858144910 0x7f88580b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.099+0000 7f885e34f640 1 -- 172.21.15.113:0/3261496113 >> 172.21.15.113:0/3261496113 conn(0x7f88580b1a50 msgr2=0x7f88580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:18.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.100+0000 7f885e34f640 1 -- 172.21.15.113:0/3261496113 shutdown_connections 2024-04-25T00:02:18.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.100+0000 7f885e34f640 1 -- 172.21.15.113:0/3261496113 wait complete. 2024-04-25T00:02:18.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.101+0000 7f885e34f640 1 Processor -- start 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.101+0000 7f885e34f640 1 -- start start 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f88580ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88580b5680 0x7f88580a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8858144910 0x7f88580ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88580b4da0 con 0x7f88580b5240 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88580b37d0 con 0x7f8858144910 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f88580b4f20 con 0x7f88580b5680 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f88580ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f88580ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54278/0 (socket says 172.21.15.113:54278) 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 -- 172.21.15.113:0/4272536745 learned_addr learned my addr 172.21.15.113:0/4272536745 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f88577fe640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88580b5680 0x7f88580a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885c8ec640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8858144910 0x7f88580ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 -- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88580b5680 msgr2=0x7f88580a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88580b5680 0x7f88580a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 -- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8858144910 msgr2=0x7f88580ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8858144910 0x7f88580ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 -- 172.21.15.113:0/4272536745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885815de90 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8857fff640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f88580ad020 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f884800d830 tx=0x7f884800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8854ff9640 1 -- 172.21.15.113:0/4272536745 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884800b6c0 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8854ff9640 1 -- 172.21.15.113:0/4272536745 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8848019070 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88580ad700 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f8854ff9640 1 -- 172.21.15.113:0/4272536745 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884800ba80 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.102+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88580adcc0 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.103+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88580691a0 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.104+0000 7f8854ff9640 1 -- 172.21.15.113:0/4272536745 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f884801e460 con 0x7f88580b5240 2024-04-25T00:02:18.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.104+0000 7f8854ff9640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f884406be30 0x7f884406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.104+0000 7f8854ff9640 1 -- 172.21.15.113:0/4272536745 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4857+0+0 (secure 0 0 0) 0x7f884808b1f0 con 0x7f88580b5240 2024-04-25T00:02:18.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.104+0000 7f88577fe640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f884406be30 0x7f884406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.105+0000 7f88577fe640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f884406be30 0x7f884406e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f884c034000 tx=0x7f884c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.108+0000 7f8854ff9640 1 -- 172.21.15.113:0/4272536745 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8848059ef0 con 0x7f88580b5240 2024-04-25T00:02:18.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.346+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f88581438e0 con 0x7f88580b5240 2024-04-25T00:02:18.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.347+0000 7f8854ff9640 1 -- 172.21.15.113:0/4272536745 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f88480597f0 con 0x7f88580b5240 2024-04-25T00:02:18.347 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:18.347 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:02:18.347 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:02:18.348 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:02:18.349 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.351 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.352 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.353 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:02:18.354 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:02:18.355 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.355 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.355 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.355 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.355 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.356 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:02:18.356 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:02:18.356 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.356 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:02:18.356 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:02:18.356 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:02:18.357 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:02:18.358 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:02:18.360 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:02:18.361 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.362 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:18.362 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:02:18.362 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:02:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f884406be30 msgr2=0x7f884406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f884406be30 0x7f884406e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f884c034000 tx=0x7f884c02e000 comp rx=0 tx=0).stop 2024-04-25T00:02:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 msgr2=0x7f88580ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f88580ad020 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f884800d830 tx=0x7f884800dcf0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 shutdown_connections 2024-04-25T00:02:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f884406be30 0x7f884406e2f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8858144910 0x7f88580ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88580b5680 0x7f88580a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 --2- 172.21.15.113:0/4272536745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88580b5240 0x7f88580ad020 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 >> 172.21.15.113:0/4272536745 conn(0x7f88580b1a50 msgr2=0x7f88580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 shutdown_connections 2024-04-25T00:02:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.349+0000 7f885e34f640 1 -- 172.21.15.113:0/4272536745 wait complete. 2024-04-25T00:02:18.365 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:18.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 Processor -- start 2024-04-25T00:02:18.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 -- start start 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f548c15e320 0x7f548c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f548c0d0b00 0x7f548c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f548c0d14d0 con 0x7f548c0d0b00 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f548c0d3410 con 0x7f548c15e320 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.461+0000 7f549213b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f548c0d3de0 con 0x7f548c0d2290 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f548c0d0b00 0x7f548c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f548c0d0b00 0x7f548c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54294/0 (socket says 172.21.15.113:54294) 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 -- 172.21.15.113:0/420078024 learned_addr learned my addr 172.21.15.113:0/420078024 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:18.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548b7fe640 1 --2- 172.21.15.113:0/420078024 >> v2:172.21.15.178:3300/0 conn(0x7f548c15e320 0x7f548c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548affd640 1 --2- 172.21.15.113:0/420078024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 -- 172.21.15.113:0/420078024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 msgr2=0x7f548c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 --2- 172.21.15.113:0/420078024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c0b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 -- 172.21.15.113:0/420078024 >> v2:172.21.15.178:3300/0 conn(0x7f548c15e320 msgr2=0x7f548c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 --2- 172.21.15.113:0/420078024 >> v2:172.21.15.178:3300/0 conn(0x7f548c15e320 0x7f548c0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.462+0000 7f548bfff640 1 -- 172.21.15.113:0/420078024 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c0d2800 con 0x7f548c0d0b00 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f548bfff640 1 --2- 172.21.15.113:0/420078024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c0d0f90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f548c0b6d00 tx=0x7f548402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e0b7e91314189d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f548a7fc640 1 -- 172.21.15.113:0/420078024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484036070 con 0x7f548c0d0b00 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f548a7fc640 1 -- 172.21.15.113:0/420078024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5484031600 con 0x7f548c0d0b00 2024-04-25T00:02:18.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f548a7fc640 1 -- 172.21.15.113:0/420078024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840319d0 con 0x7f548c0d0b00 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 -- 172.21.15.113:0/420078024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 msgr2=0x7f548c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 --2- 172.21.15.113:0/420078024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c0d0f90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f548c0b6d00 tx=0x7f548402a730 comp rx=0 tx=0).stop 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 -- 172.21.15.113:0/420078024 shutdown_connections 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 --2- 172.21.15.113:0/420078024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c0d0f90 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 --2- 172.21.15.113:0/420078024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 --2- 172.21.15.113:0/420078024 >> v2:172.21.15.178:3300/0 conn(0x7f548c15e320 0x7f548c0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 -- 172.21.15.113:0/420078024 >> 172.21.15.113:0/420078024 conn(0x7f548c0b5c20 msgr2=0x7f548c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 -- 172.21.15.113:0/420078024 shutdown_connections 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.463+0000 7f549213b640 1 -- 172.21.15.113:0/420078024 wait complete. 2024-04-25T00:02:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.464+0000 7f549213b640 1 Processor -- start 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f549213b640 1 -- start start 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f549213b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f549213b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f549213b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f548c15e320 0x7f548c0b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f549213b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f548c0d30a0 con 0x7f548c0d0b00 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f549213b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f548c0d2dc0 con 0x7f548c15e320 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f549213b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f548c0b9fe0 con 0x7f548c0d2290 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f548b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f548b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54300/0 (socket says 172.21.15.113:54300) 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.465+0000 7f548b7fe640 1 -- 172.21.15.113:0/248076626 learned_addr learned my addr 172.21.15.113:0/248076626 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548bfff640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f548c15e320 0x7f548c0b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548affd640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548b7fe640 1 -- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 msgr2=0x7f548c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548b7fe640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548b7fe640 1 -- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f548c15e320 msgr2=0x7f548c0b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548b7fe640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f548c15e320 0x7f548c0b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548b7fe640 1 -- 172.21.15.113:0/248076626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c15eb90 con 0x7f548c0d0b00 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f548b7fe640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c156bd0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f547c00d5f0 tx=0x7f547c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f5473fff640 1 -- 172.21.15.113:0/248076626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c0125f0 con 0x7f548c0d0b00 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f5473fff640 1 -- 172.21.15.113:0/248076626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547c019070 con 0x7f548c0d0b00 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c15ee70 con 0x7f548c0d0b00 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f5473fff640 1 -- 172.21.15.113:0/248076626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c0129d0 con 0x7f548c0d0b00 2024-04-25T00:02:18.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.466+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c15f430 con 0x7f548c0d0b00 2024-04-25T00:02:18.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.467+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548c0691a0 con 0x7f548c0d0b00 2024-04-25T00:02:18.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.468+0000 7f5473fff640 1 -- 172.21.15.113:0/248076626 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f547c012b70 con 0x7f548c0d0b00 2024-04-25T00:02:18.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.468+0000 7f5473fff640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547806be30 0x7f547806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.468+0000 7f5473fff640 1 -- 172.21.15.113:0/248076626 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4857+0+0 (secure 0 0 0) 0x7f547c08ad20 con 0x7f548c0d0b00 2024-04-25T00:02:18.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.468+0000 7f548affd640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547806be30 0x7f547806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.468+0000 7f548affd640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547806be30 0x7f547806e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f548c0b4410 tx=0x7f5480008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.472+0000 7f5473fff640 1 -- 172.21.15.113:0/248076626 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f547c059460 con 0x7f548c0d0b00 2024-04-25T00:02:18.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.648+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f548c15c9f0 con 0x7f548c0d0b00 2024-04-25T00:02:18.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.649+0000 7f5473fff640 1 -- 172.21.15.113:0/248076626 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+12939 (secure 0 0 0) 0x7f547c05d760 con 0x7f548c0d0b00 2024-04-25T00:02:18.649 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:18.649 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:02:17.975315+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:18.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547806be30 msgr2=0x7f547806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547806be30 0x7f547806e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f548c0b4410 tx=0x7f5480008040 comp rx=0 tx=0).stop 2024-04-25T00:02:18.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 msgr2=0x7f548c156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c156bd0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f547c00d5f0 tx=0x7f547c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 shutdown_connections 2024-04-25T00:02:18.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547806be30 0x7f547806e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f548c15e320 0x7f548c0b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f548c0d2290 0x7f548c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 --2- 172.21.15.113:0/248076626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f548c0d0b00 0x7f548c156bd0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 >> 172.21.15.113:0/248076626 conn(0x7f548c0b5c20 msgr2=0x7f548c0b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:18.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 shutdown_connections 2024-04-25T00:02:18.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.651+0000 7f549213b640 1 -- 172.21.15.113:0/248076626 wait complete. 2024-04-25T00:02:18.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:02:18.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:02:18.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.762+0000 7f426cb4f640 1 Processor -- start 2024-04-25T00:02:18.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.762+0000 7f426cb4f640 1 -- start start 2024-04-25T00:02:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.763+0000 7f426cb4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f426815e510 0x7f42680d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.763+0000 7f426cb4f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f42680d28c0 0x7f42680b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.763+0000 7f426cb4f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42680d2e00 con 0x7f42680b61e0 2024-04-25T00:02:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.763+0000 7f426cb4f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f42680d30e0 con 0x7f42680d28c0 2024-04-25T00:02:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.763+0000 7f426cb4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42680d33c0 con 0x7f426815e510 2024-04-25T00:02:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f426815e510 0x7f42680d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f4265d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f42680d28c0 0x7f42680b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f426815e510 0x7f42680d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:50890/0 (socket says 172.21.15.113:50890) 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426659c640 1 -- 172.21.15.113:0/1384084489 learned_addr learned my addr 172.21.15.113:0/1384084489 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f4265d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f42680d28c0 0x7f42680b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:49906/0 (socket says 172.21.15.113:49906) 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f4266d9d640 1 --1- 172.21.15.113:0/1384084489 >> v1:172.21.15.178:6789/0 conn(0x7f42680b61e0 0x7f4268071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33712/0 (socket says 172.21.15.113:33712) 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426659c640 1 -- 172.21.15.113:0/1384084489 >> v2:172.21.15.178:3300/0 conn(0x7f42680d28c0 msgr2=0x7f42680b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426659c640 1 --2- 172.21.15.113:0/1384084489 >> v2:172.21.15.178:3300/0 conn(0x7f42680d28c0 0x7f42680b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426659c640 1 -- 172.21.15.113:0/1384084489 >> v1:172.21.15.178:6789/0 conn(0x7f42680b61e0 legacy=0x7f4268071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426659c640 1 -- 172.21.15.113:0/1384084489 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42680d36a0 con 0x7f426815e510 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f426559a640 1 -- 172.21.15.113:0/1384084489 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689525116 0 0) 0x7f42680d2e00 con 0x7f42680b61e0 2024-04-25T00:02:18.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.764+0000 7f4265d9b640 1 --2- 172.21.15.113:0/1384084489 >> v2:172.21.15.178:3300/0 conn(0x7f42680d28c0 0x7f42680b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.765+0000 7f426659c640 1 --2- 172.21.15.113:0/1384084489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f426815e510 0x7f42680d2380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4258009e50 tx=0x7f425802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693015dc8317db61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.765+0000 7f426559a640 1 -- 172.21.15.113:0/1384084489 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258036070 con 0x7f426815e510 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.765+0000 7f426559a640 1 -- 172.21.15.113:0/1384084489 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4258031600 con 0x7f426815e510 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.765+0000 7f426559a640 1 -- 172.21.15.113:0/1384084489 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580319d0 con 0x7f426815e510 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.765+0000 7f426cb4f640 1 -- 172.21.15.113:0/1384084489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f426815e510 msgr2=0x7f42680d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.765+0000 7f426cb4f640 1 --2- 172.21.15.113:0/1384084489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f426815e510 0x7f42680d2380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4258009e50 tx=0x7f425802a680 comp rx=0 tx=0).stop 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.766+0000 7f426cb4f640 1 -- 172.21.15.113:0/1384084489 shutdown_connections 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.766+0000 7f426cb4f640 1 --2- 172.21.15.113:0/1384084489 >> v2:172.21.15.178:3300/0 conn(0x7f42680d28c0 0x7f42680b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.766+0000 7f426cb4f640 1 --2- 172.21.15.113:0/1384084489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f426815e510 0x7f42680d2380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.766+0000 7f426cb4f640 1 -- 172.21.15.113:0/1384084489 >> 172.21.15.113:0/1384084489 conn(0x7f426806b720 msgr2=0x7f42680d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.766+0000 7f426cb4f640 1 -- 172.21.15.113:0/1384084489 shutdown_connections 2024-04-25T00:02:18.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.766+0000 7f426cb4f640 1 -- 172.21.15.113:0/1384084489 wait complete. 2024-04-25T00:02:18.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.767+0000 7f426cb4f640 1 Processor -- start 2024-04-25T00:02:18.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.767+0000 7f426cb4f640 1 -- start start 2024-04-25T00:02:18.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42680b61e0 0x7f426814d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42680d28c0 0x7f42680c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f426815e510 0x7f4268160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f42680b9dd0 con 0x7f42680b61e0 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42680d3ea0 con 0x7f426815e510 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42680b83d0 con 0x7f42680d28c0 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42680b61e0 0x7f426814d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42680b61e0 0x7f426814d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54312/0 (socket says 172.21.15.113:54312) 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 -- 172.21.15.113:0/2183554258 learned_addr learned my addr 172.21.15.113:0/2183554258 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f4265d9b640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42680d28c0 0x7f42680c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f4266d9d640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f426815e510 0x7f4268160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 -- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42680d28c0 msgr2=0x7f42680c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42680d28c0 0x7f42680c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 -- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f426815e510 msgr2=0x7f4268160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f426815e510 0x7f4268160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 -- 172.21.15.113:0/2183554258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426814dc90 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426659c640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42680b61e0 0x7f426814d750 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4258034040 tx=0x7f4258004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f4256ffd640 1 -- 172.21.15.113:0/2183554258 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258041020 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f4256ffd640 1 -- 172.21.15.113:0/2183554258 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4258037040 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42680c8290 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f4256ffd640 1 -- 172.21.15.113:0/2183554258 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258036050 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.768+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42680c8810 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.769+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42680691a0 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.770+0000 7f4256ffd640 1 -- 172.21.15.113:0/2183554258 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4258007c50 con 0x7f42680b61e0 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.770+0000 7f4256ffd640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f424806be30 0x7f424806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:18.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.770+0000 7f4256ffd640 1 -- 172.21.15.113:0/2183554258 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4857+0+0 (secure 0 0 0) 0x7f42580ab120 con 0x7f42680b61e0 2024-04-25T00:02:18.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.770+0000 7f4265d9b640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f424806be30 0x7f424806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:18.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.771+0000 7f4265d9b640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f424806be30 0x7f424806e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42680d0a30 tx=0x7f425c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:18.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.774+0000 7f4256ffd640 1 -- 172.21.15.113:0/2183554258 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f425804b060 con 0x7f42680b61e0 2024-04-25T00:02:18.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.946+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42680d43a0 con 0x7f42680b61e0 2024-04-25T00:02:18.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.946+0000 7f4256ffd640 1 -- 172.21.15.113:0/2183554258 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f426814dc90 con 0x7f42680b61e0 2024-04-25T00:02:18.946 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:02:18.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f424806be30 msgr2=0x7f424806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f424806be30 0x7f424806e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42680d0a30 tx=0x7f425c009190 comp rx=0 tx=0).stop 2024-04-25T00:02:18.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42680b61e0 msgr2=0x7f426814d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:18.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42680b61e0 0x7f426814d750 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4258034040 tx=0x7f4258004a40 comp rx=0 tx=0).stop 2024-04-25T00:02:18.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 shutdown_connections 2024-04-25T00:02:18.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f424806be30 0x7f424806e2f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f426815e510 0x7f4268160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42680d28c0 0x7f42680c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 --2- 172.21.15.113:0/2183554258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42680b61e0 0x7f426814d750 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 >> 172.21.15.113:0/2183554258 conn(0x7f426806b720 msgr2=0x7f42680d48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 shutdown_connections 2024-04-25T00:02:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:18.948+0000 7f426cb4f640 1 -- 172.21.15.113:0/2183554258 wait complete. 2024-04-25T00:02:18.959 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:02:19.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.056+0000 7f35e6da5640 1 Processor -- start 2024-04-25T00:02:19.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.056+0000 7f35e6da5640 1 -- start start 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e6da5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f35e0144d60 0x7f35e00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e6da5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f35e006e6e0 0x7f35e0071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e6da5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e00cfad0 con 0x7f35e01453c0 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e6da5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f35e0145c40 con 0x7f35e006e6e0 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e6da5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f35e00cf230 con 0x7f35e0144d60 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e5342640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f35e006e6e0 0x7f35e0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e5342640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f35e006e6e0 0x7f35e0071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54322/0 (socket says 172.21.15.113:54322) 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e5342640 1 -- 172.21.15.113:0/2845749443 learned_addr learned my addr 172.21.15.113:0/2845749443 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e4b41640 1 --2- 172.21.15.113:0/2845749443 >> v2:172.21.15.178:3300/0 conn(0x7f35e0144d60 0x7f35e00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e5342640 1 -- 172.21.15.113:0/2845749443 >> v2:172.21.15.178:3300/0 conn(0x7f35e0144d60 msgr2=0x7f35e00b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e5342640 1 --2- 172.21.15.113:0/2845749443 >> v2:172.21.15.178:3300/0 conn(0x7f35e0144d60 0x7f35e00b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e5342640 1 -- 172.21.15.113:0/2845749443 >> v1:172.21.15.178:6789/0 conn(0x7f35e01453c0 legacy=0x7f35e006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e5342640 1 -- 172.21.15.113:0/2845749443 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e00cf510 con 0x7f35e006e6e0 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.057+0000 7f35e4b41640 1 --2- 172.21.15.113:0/2845749443 >> v2:172.21.15.178:3300/0 conn(0x7f35e0144d60 0x7f35e00b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e5342640 1 --2- 172.21.15.113:0/2845749443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e0071330 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f35e006f970 tx=0x7f35d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c06fa9a0c4a02f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35df7fe640 1 -- 172.21.15.113:0/2845749443 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8036070 con 0x7f35e006e6e0 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35df7fe640 1 -- 172.21.15.113:0/2845749443 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d8031600 con 0x7f35e006e6e0 2024-04-25T00:02:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35df7fe640 1 -- 172.21.15.113:0/2845749443 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80319d0 con 0x7f35e006e6e0 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 -- 172.21.15.113:0/2845749443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 msgr2=0x7f35e0071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2845749443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e0071330 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f35e006f970 tx=0x7f35d802a660 comp rx=0 tx=0).stop 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 -- 172.21.15.113:0/2845749443 shutdown_connections 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2845749443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e0071330 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2845749443 >> v2:172.21.15.178:3300/0 conn(0x7f35e0144d60 0x7f35e00b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 -- 172.21.15.113:0/2845749443 >> 172.21.15.113:0/2845749443 conn(0x7f35e00b1a50 msgr2=0x7f35e00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 -- 172.21.15.113:0/2845749443 shutdown_connections 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.058+0000 7f35e6da5640 1 -- 172.21.15.113:0/2845749443 wait complete. 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.059+0000 7f35e6da5640 1 Processor -- start 2024-04-25T00:02:19.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e6da5640 1 -- start start 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e6da5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e01f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e6da5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e0144d60 0x7f35e01357b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e6da5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e01453c0 0x7f35e013db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e6da5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0145140 con 0x7f35e006e6e0 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e6da5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0070e10 con 0x7f35e01453c0 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e6da5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35e0071870 con 0x7f35e0144d60 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e4b41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e01f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.060+0000 7f35e4b41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e01f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54330/0 (socket says 172.21.15.113:54330) 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e4b41640 1 -- 172.21.15.113:0/2691099174 learned_addr learned my addr 172.21.15.113:0/2691099174 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e5342640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e01453c0 0x7f35e013db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35dffff640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e0144d60 0x7f35e01357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e4b41640 1 -- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e0144d60 msgr2=0x7f35e01357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e4b41640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e0144d60 0x7f35e01357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e4b41640 1 -- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e01453c0 msgr2=0x7f35e013db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e4b41640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e01453c0 0x7f35e013db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e4b41640 1 -- 172.21.15.113:0/2691099174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e01373a0 con 0x7f35e006e6e0 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35dffff640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e0144d60 0x7f35e01357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e4b41640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e01f2100 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c7f0 tx=0x7f35d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35dd7fa640 1 -- 172.21.15.113:0/2691099174 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4015070 con 0x7f35e006e6e0 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35dd7fa640 1 -- 172.21.15.113:0/2691099174 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4007a30 con 0x7f35e006e6e0 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e013ac30 con 0x7f35e006e6e0 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35dd7fa640 1 -- 172.21.15.113:0/2691099174 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4007e00 con 0x7f35e006e6e0 2024-04-25T00:02:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.061+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e013b1f0 con 0x7f35e006e6e0 2024-04-25T00:02:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.062+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e006e500 con 0x7f35e006e6e0 2024-04-25T00:02:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.063+0000 7f35dd7fa640 1 -- 172.21.15.113:0/2691099174 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35d4018070 con 0x7f35e006e6e0 2024-04-25T00:02:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.063+0000 7f35dd7fa640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35c406be30 0x7f35c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.063+0000 7f35dd7fa640 1 -- 172.21.15.113:0/2691099174 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4857+0+0 (secure 0 0 0) 0x7f35d408a9c0 con 0x7f35e006e6e0 2024-04-25T00:02:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.063+0000 7f35dffff640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35c406be30 0x7f35c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.063+0000 7f35dffff640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35c406be30 0x7f35c406e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f35cc006fd0 tx=0x7f35cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.067+0000 7f35dd7fa640 1 -- 172.21.15.113:0/2691099174 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f35e006e500 con 0x7f35e006e6e0 2024-04-25T00:02:19.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.244+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f35e00691a0 con 0x7f35e006e6e0 2024-04-25T00:02:19.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.245+0000 7f35dd7fa640 1 -- 172.21.15.113:0/2691099174 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f35d408ce20 con 0x7f35e006e6e0 2024-04-25T00:02:19.245 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:19.245 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:02:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:02:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:02:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:02:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:02:19.246 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:02:19.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35c406be30 msgr2=0x7f35c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35c406be30 0x7f35c406e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f35cc006fd0 tx=0x7f35cc008040 comp rx=0 tx=0).stop 2024-04-25T00:02:19.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 msgr2=0x7f35e01f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e01f2100 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c7f0 tx=0x7f35d400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 shutdown_connections 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35c406be30 0x7f35c406e2f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e01453c0 0x7f35e013db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e0144d60 0x7f35e01357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 --2- 172.21.15.113:0/2691099174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35e006e6e0 0x7f35e01f2100 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 >> 172.21.15.113:0/2691099174 conn(0x7f35e00b1a50 msgr2=0x7f35e0072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 shutdown_connections 2024-04-25T00:02:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.247+0000 7f35e6da5640 1 -- 172.21.15.113:0/2691099174 wait complete. 2024-04-25T00:02:19.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:02:19.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:02:19.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.359+0000 7f87fae8c640 1 Processor -- start 2024-04-25T00:02:19.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.359+0000 7f87fae8c640 1 -- start start 2024-04-25T00:02:19.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87fae8c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f87f4145890 0x7f87f40ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87fae8c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87f40ced20 0x7f87f406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87fae8c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f406ea20 con 0x7f87f406e610 2024-04-25T00:02:19.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87fae8c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f87f4145c70 con 0x7f87f40ced20 2024-04-25T00:02:19.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87fae8c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f87f40cf3a0 con 0x7f87f4145890 2024-04-25T00:02:19.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87f40ced20 0x7f87f406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f9429640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f87f406e610 0x7f87f415fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58126/0 (socket says 172.21.15.113:58126) 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f9429640 1 -- 172.21.15.113:0/1056737387 learned_addr learned my addr 172.21.15.113:0/1056737387 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f8c28640 1 --2- 172.21.15.113:0/1056737387 >> v2:172.21.15.178:3300/0 conn(0x7f87f4145890 0x7f87f40ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f3fff640 1 -- 172.21.15.113:0/1056737387 >> v2:172.21.15.178:3300/0 conn(0x7f87f4145890 msgr2=0x7f87f40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f3fff640 1 --2- 172.21.15.113:0/1056737387 >> v2:172.21.15.178:3300/0 conn(0x7f87f4145890 0x7f87f40ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f3fff640 1 -- 172.21.15.113:0/1056737387 >> v1:172.21.15.113:6789/0 conn(0x7f87f406e610 legacy=0x7f87f415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f3fff640 1 -- 172.21.15.113:0/1056737387 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f40cf680 con 0x7f87f40ced20 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f37fe640 1 -- 172.21.15.113:0/1056737387 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696761997 0 0) 0x7f87ec001490 con 0x7f87f406e610 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.360+0000 7f87f8c28640 1 --2- 172.21.15.113:0/1056737387 >> v2:172.21.15.178:3300/0 conn(0x7f87f4145890 0x7f87f40ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.361+0000 7f87f3fff640 1 --2- 172.21.15.113:0/1056737387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f406e0d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f87f406c0d0 tx=0x7f87e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561d567967bc8a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.361+0000 7f87f37fe640 1 -- 172.21.15.113:0/1056737387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0037070 con 0x7f87f40ced20 2024-04-25T00:02:19.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.361+0000 7f87f37fe640 1 -- 172.21.15.113:0/1056737387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e0032440 con 0x7f87f40ced20 2024-04-25T00:02:19.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.361+0000 7f87f37fe640 1 -- 172.21.15.113:0/1056737387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0032810 con 0x7f87f40ced20 2024-04-25T00:02:19.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 -- 172.21.15.113:0/1056737387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 msgr2=0x7f87f406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 --2- 172.21.15.113:0/1056737387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f406e0d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f87f406c0d0 tx=0x7f87e002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 -- 172.21.15.113:0/1056737387 shutdown_connections 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 --2- 172.21.15.113:0/1056737387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f406e0d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 --2- 172.21.15.113:0/1056737387 >> v2:172.21.15.178:3300/0 conn(0x7f87f4145890 0x7f87f40ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 -- 172.21.15.113:0/1056737387 >> 172.21.15.113:0/1056737387 conn(0x7f87f40b1b30 msgr2=0x7f87f40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 -- 172.21.15.113:0/1056737387 shutdown_connections 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.362+0000 7f87fae8c640 1 -- 172.21.15.113:0/1056737387 wait complete. 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.363+0000 7f87fae8c640 1 Processor -- start 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.363+0000 7f87fae8c640 1 -- start start 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87f406e610 0x7f87f415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f4160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f4145890 0x7f87f41f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87f40b53b0 con 0x7f87f40ced20 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4070f10 con 0x7f87f4145890 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87f4072440 con 0x7f87f406e610 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f4160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54348/0 (socket says 172.21.15.113:54348) 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 -- 172.21.15.113:0/2054719512 learned_addr learned my addr 172.21.15.113:0/2054719512 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f8c28640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87f406e610 0x7f87f415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f9429640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f4145890 0x7f87f41f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 -- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87f406e610 msgr2=0x7f87f415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87f406e610 0x7f87f415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 -- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f4145890 msgr2=0x7f87f41f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f4145890 0x7f87f41f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 -- 172.21.15.113:0/2054719512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f415ee00 con 0x7f87f40ced20 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f8c28640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87f406e610 0x7f87f415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f9429640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f4145890 0x7f87f41f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f3fff640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f4160080 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009890 tx=0x7f87e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f17fa640 1 -- 172.21.15.113:0/2054719512 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0037040 con 0x7f87f40ced20 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f17fa640 1 -- 172.21.15.113:0/2054719512 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e0042070 con 0x7f87f40ced20 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f413a800 con 0x7f87f40ced20 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87f17fa640 1 -- 172.21.15.113:0/2054719512 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0030bd0 con 0x7f87f40ced20 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.364+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f413ad80 con 0x7f87f40ced20 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.365+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f40691a0 con 0x7f87f40ced20 2024-04-25T00:02:19.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.366+0000 7f87f17fa640 1 -- 172.21.15.113:0/2054719512 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87e0047460 con 0x7f87f40ced20 2024-04-25T00:02:19.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.366+0000 7f87f17fa640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87d806be30 0x7f87d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.366+0000 7f87f17fa640 1 -- 172.21.15.113:0/2054719512 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4857+0+0 (secure 0 0 0) 0x7f87e00ab750 con 0x7f87f40ced20 2024-04-25T00:02:19.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.366+0000 7f87f8c28640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87d806be30 0x7f87d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.367+0000 7f87f8c28640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87d806be30 0x7f87d806e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.370+0000 7f87f17fa640 1 -- 172.21.15.113:0/2054719512 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f87e0032e20 con 0x7f87f40ced20 2024-04-25T00:02:19.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.543+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f87f4072bc0 con 0x7f87f40ced20 2024-04-25T00:02:19.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.983+0000 7f87f17fa640 1 -- 172.21.15.113:0/2054719512 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f87e007a450 con 0x7f87f40ced20 2024-04-25T00:02:19.984 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:02:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87d806be30 msgr2=0x7f87d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87d806be30 0x7f87d806e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8008040 comp rx=0 tx=0).stop 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 msgr2=0x7f87f4160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f4160080 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009890 tx=0x7f87e00046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 shutdown_connections 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f87d806be30 0x7f87d806e2f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f4145890 0x7f87f41f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87f40ced20 0x7f87f4160080 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 --2- 172.21.15.113:0/2054719512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87f406e610 0x7f87f415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 >> 172.21.15.113:0/2054719512 conn(0x7f87f40b1b30 msgr2=0x7f87f40741e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 shutdown_connections 2024-04-25T00:02:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:19.987+0000 7f87fae8c640 1 -- 172.21.15.113:0/2054719512 wait complete. 2024-04-25T00:02:20.003 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:02:20.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.111+0000 7f52e0bc1640 1 Processor -- start 2024-04-25T00:02:20.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.111+0000 7f52e0bc1640 1 -- start start 2024-04-25T00:02:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52e0bc1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 0x7f52dc0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52e0bc1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52dc1606a0 0x7f52dc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52e0bc1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc0d0970 con 0x7f52dc0d04f0 2024-04-25T00:02:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52e0bc1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f52dc15c6a0 con 0x7f52dc1606a0 2024-04-25T00:02:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52e0bc1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc160aa0 con 0x7f52dc13f050 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52d9d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f52dc0d04f0 0x7f52dc15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58130/0 (socket says 172.21.15.113:58130) 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52d9d9b640 1 -- 172.21.15.113:0/493301029 learned_addr learned my addr 172.21.15.113:0/493301029 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2298591343 0 0) 0x7f52dc0d0970 con 0x7f52dc0d04f0 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52c4003520 con 0x7f52dc0d04f0 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52dad9d640 1 --2- 172.21.15.113:0/493301029 >> v2:172.21.15.178:3300/0 conn(0x7f52dc1606a0 0x7f52dc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.112+0000 7f52da59c640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 0x7f52dc0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1028391086 0 0) 0x7f52c4003520 con 0x7f52dc0d04f0 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 msgr2=0x7f52dc0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 0x7f52dc0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 >> v2:172.21.15.178:3300/0 conn(0x7f52dc1606a0 msgr2=0x7f52dc0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 --2- 172.21.15.113:0/493301029 >> v2:172.21.15.178:3300/0 conn(0x7f52dc1606a0 0x7f52dc0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc15d6b0 con 0x7f52dc0d04f0 2024-04-25T00:02:20.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f52d0001670 con 0x7f52dc0d04f0 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 >> v1:172.21.15.113:6789/0 conn(0x7f52dc0d04f0 legacy=0x7f52dc15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 0x7f52c40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4003fe0 0x7f52c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c4004d40 0x7f52c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc15c6a0 con 0x7f52c4004d40 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc160aa0 con 0x7f52c4003fe0 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52c400bb80 con 0x7f52dc13f050 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52dad9d640 1 --2- 172.21.15.113:0/493301029 >> v2:172.21.15.178:3300/0 conn(0x7f52dc1606a0 0x7f52dc0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:20.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52da59c640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 0x7f52c40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52dad9d640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c4004d40 0x7f52c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52d9d9b640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4003fe0 0x7f52c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52dad9d640 1 -- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 msgr2=0x7f52c40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52dad9d640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc13f050 0x7f52c40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52dad9d640 1 -- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4003fe0 msgr2=0x7f52c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52dad9d640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4003fe0 0x7f52c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.113+0000 7f52dad9d640 1 -- 172.21.15.113:0/493301029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c4003520 con 0x7f52c4004d40 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.114+0000 7f52dad9d640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c4004d40 0x7f52c40051c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098a0 tx=0x7f52c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59120aa5fadca48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.114+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4037070 con 0x7f52c4004d40 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.114+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d402ad20 con 0x7f52c4004d40 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.114+0000 7f52d959a640 1 -- 172.21.15.113:0/493301029 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d400f040 con 0x7f52c4004d40 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.115+0000 7f52e0bc1640 1 -- 172.21.15.113:0/493301029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c4004d40 msgr2=0x7f52c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.115+0000 7f52e0bc1640 1 --2- 172.21.15.113:0/493301029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c4004d40 0x7f52c40051c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098a0 tx=0x7f52c4005830 comp rx=0 tx=0).stop 2024-04-25T00:02:20.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.115+0000 7f52dad9d640 1 -- 172.21.15.113:0/493301029 reap_dead start 2024-04-25T00:02:20.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.115+0000 7f52e0bc1640 1 -- 172.21.15.113:0/493301029 shutdown_connections 2024-04-25T00:02:20.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.115+0000 7f52e0bc1640 1 -- 172.21.15.113:0/493301029 >> 172.21.15.113:0/493301029 conn(0x7f52dc06b0f0 msgr2=0x7f52dc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:20.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.115+0000 7f52e0bc1640 1 -- 172.21.15.113:0/493301029 shutdown_connections 2024-04-25T00:02:20.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.115+0000 7f52e0bc1640 1 -- 172.21.15.113:0/493301029 wait complete. 2024-04-25T00:02:20.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.116+0000 7f52e0bc1640 1 Processor -- start 2024-04-25T00:02:20.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.116+0000 7f52e0bc1640 1 -- start start 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52e0bc1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4004d40 0x7f52dc14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52e0bc1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc0b1110 0x7f52dc0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52e0bc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc1f0ac0 0x7f52dc1f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52e0bc1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc15ceb0 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52e0bc1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc15cbd0 con 0x7f52c4004d40 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52e0bc1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc15d190 con 0x7f52dc0b1110 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc1f0ac0 0x7f52dc1f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc1f0ac0 0x7f52dc1f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54372/0 (socket says 172.21.15.113:54372) 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 -- 172.21.15.113:0/554488404 learned_addr learned my addr 172.21.15.113:0/554488404 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52da59c640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4004d40 0x7f52dc14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52d9d9b640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc0b1110 0x7f52dc0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 -- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc0b1110 msgr2=0x7f52dc0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc0b1110 0x7f52dc0b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 -- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4004d40 msgr2=0x7f52dc14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4004d40 0x7f52dc14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 -- 172.21.15.113:0/554488404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc0b1e40 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.117+0000 7f52dad9d640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc1f0ac0 0x7f52dc1f0f40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f52d40099c0 tx=0x7f52d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.118+0000 7f52caffd640 1 -- 172.21.15.113:0/554488404 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4037030 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.118+0000 7f52caffd640 1 -- 172.21.15.113:0/554488404 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d4041070 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.118+0000 7f52caffd640 1 -- 172.21.15.113:0/554488404 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d403b7f0 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.118+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc0b2120 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.118+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc0b26e0 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.119+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc0691a0 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.119+0000 7f52caffd640 1 -- 172.21.15.113:0/554488404 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52d403b990 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.119+0000 7f52caffd640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be60 0x7f52bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:20.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.120+0000 7f52da59c640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be60 0x7f52bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:20.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.120+0000 7f52caffd640 1 -- 172.21.15.113:0/554488404 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5227+0+0 (secure 0 0 0) 0x7f52d40aae60 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.120+0000 7f52da59c640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be60 0x7f52bc06e320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52cc003b80 tx=0x7f52cc009740 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:20.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.124+0000 7f52caffd640 1 -- 172.21.15.113:0/554488404 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52d400f000 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.296+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f52dc0d0e50 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.993+0000 7f52caffd640 1 -- 172.21.15.113:0/554488404 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) ==== 170+0+0 (secure 0 0 0) 0x7f52d4079310 con 0x7f52dc1f0ac0 2024-04-25T00:02:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:02:20.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.996+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be60 msgr2=0x7f52bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.996+0000 7f52e0bc1640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be60 0x7f52bc06e320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52cc003b80 tx=0x7f52cc009740 comp rx=0 tx=0).stop 2024-04-25T00:02:20.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.996+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc1f0ac0 msgr2=0x7f52dc1f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.996+0000 7f52e0bc1640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc1f0ac0 0x7f52dc1f0f40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f52d40099c0 tx=0x7f52d4004730 comp rx=0 tx=0).stop 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.997+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 shutdown_connections 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.997+0000 7f52e0bc1640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be60 0x7f52bc06e320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.997+0000 7f52e0bc1640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc1f0ac0 0x7f52dc1f0f40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.997+0000 7f52e0bc1640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc0b1110 0x7f52dc0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.997+0000 7f52e0bc1640 1 --2- 172.21.15.113:0/554488404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c4004d40 0x7f52dc14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.997+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 >> 172.21.15.113:0/554488404 conn(0x7f52dc06b0f0 msgr2=0x7f52dc15d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.998+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 shutdown_connections 2024-04-25T00:02:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:20.998+0000 7f52e0bc1640 1 -- 172.21.15.113:0/554488404 wait complete. 2024-04-25T00:02:21.021 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:02:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.123+0000 7fe7fcfc8640 1 Processor -- start 2024-04-25T00:02:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.123+0000 7fe7fcfc8640 1 -- start start 2024-04-25T00:02:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.124+0000 7fe7fcfc8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe7f815e510 0x7fe7f80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.124+0000 7fe7fcfc8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f80d2e00 con 0x7fe7f80d28c0 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.124+0000 7fe7fcfc8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f80d30e0 con 0x7fe7f80b63b0 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.124+0000 7fe7fcfc8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80d33c0 con 0x7fe7f815e510 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.124+0000 7fe7f5d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe7f80d28c0 0x7fe7f80b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58140/0 (socket says 172.21.15.113:58140) 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.124+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/463979942 learned_addr learned my addr 172.21.15.113:0/463979942 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f659c640 1 --2- 172.21.15.113:0/463979942 >> v2:172.21.15.178:3300/0 conn(0x7fe7f815e510 0x7fe7f80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 282798568 0 0) 0x7fe7f80d2e00 con 0x7fe7f80d28c0 2024-04-25T00:02:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7e0003520 con 0x7fe7f80d28c0 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 118270053 0 0) 0x7fe7e0003520 con 0x7fe7f80d28c0 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 >> v2:172.21.15.178:3300/0 conn(0x7fe7f815e510 msgr2=0x7fe7f80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 --2- 172.21.15.113:0/463979942 >> v2:172.21.15.178:3300/0 conn(0x7fe7f815e510 0x7fe7f80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 >> v1:172.21.15.178:6789/0 conn(0x7fe7f80b63b0 legacy=0x7fe7f80b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f80d36a0 con 0x7fe7f80d28c0 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe7ec001670 con 0x7fe7f80d28c0 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 >> v1:172.21.15.113:6789/0 conn(0x7fe7f80d28c0 legacy=0x7fe7f80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e00039b0 0x7fe7e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7e0004800 0x7fe7e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7e00051c0 0x7fe7e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80d33c0 con 0x7fe7e0004800 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e000c160 con 0x7fe7e00051c0 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e000c360 con 0x7fe7e00039b0 2024-04-25T00:02:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f5d9b640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7e0004800 0x7fe7e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f659c640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e00039b0 0x7fe7e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f6d9d640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7e00051c0 0x7fe7e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e00039b0 msgr2=0x7fe7e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f5d9b640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e00039b0 0x7fe7e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7e00051c0 msgr2=0x7fe7e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f5d9b640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7e00051c0 0x7fe7e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/463979942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e0003520 con 0x7fe7e0004800 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f6d9d640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7e00051c0 0x7fe7e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.125+0000 7fe7f6d9d640 1 -- 172.21.15.113:0/463979942 reap_dead start 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.126+0000 7fe7f5d9b640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7e0004800 0x7fe7e0004c80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c240 tx=0x7fe7e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b26ab57c2dbfd5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.126+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007d10 con 0x7fe7e0004800 2024-04-25T00:02:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.126+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec005ad0 con 0x7fe7e0004800 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.126+0000 7fe7f559a640 1 -- 172.21.15.113:0/463979942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007070 con 0x7fe7e0004800 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.127+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/463979942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7e0004800 msgr2=0x7fe7e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.127+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7e0004800 0x7fe7e0004c80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c240 tx=0x7fe7e0005830 comp rx=0 tx=0).stop 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.127+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/463979942 shutdown_connections 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.127+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/463979942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7e0004800 0x7fe7e0004c80 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.127+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/463979942 >> 172.21.15.113:0/463979942 conn(0x7fe7f806b720 msgr2=0x7fe7f80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.127+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/463979942 shutdown_connections 2024-04-25T00:02:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.127+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/463979942 wait complete. 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.128+0000 7fe7fcfc8640 1 Processor -- start 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.128+0000 7fe7fcfc8640 1 -- start start 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7fcfc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e0004800 0x7fe7f81f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7fcfc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f80ccd50 0x7fe7f80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7fcfc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f80cd1d0 0x7fe7f80b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7fcfc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80b9dd0 con 0x7fe7f80ccd50 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7fcfc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80d3ea0 con 0x7fe7f80cd1d0 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7fcfc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f80d2380 con 0x7fe7e0004800 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f80ccd50 0x7fe7f80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f80ccd50 0x7fe7f80b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54386/0 (socket says 172.21.15.113:54386) 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/119683970 learned_addr learned my addr 172.21.15.113:0/119683970 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f659c640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e0004800 0x7fe7f81f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e0004800 msgr2=0x7fe7f81f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e0004800 0x7fe7f81f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f6d9d640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f80cd1d0 0x7fe7f80b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f80cd1d0 msgr2=0x7fe7f80b8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f80cd1d0 0x7fe7f80b8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 -- 172.21.15.113:0/119683970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f81f4250 con 0x7fe7f80ccd50 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f659c640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e0004800 0x7fe7f81f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7f5d9b640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f80ccd50 0x7fe7f80b8a50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c240 tx=0x7fe7f80d2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.129+0000 7fe7e6ffd640 1 -- 172.21.15.113:0/119683970 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec041020 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.130+0000 7fe7e6ffd640 1 -- 172.21.15.113:0/119683970 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec000ed0 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.130+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f80c8a80 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.130+0000 7fe7e6ffd640 1 -- 172.21.15.113:0/119683970 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007d10 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.130+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f80c9040 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.131+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f80691a0 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.131+0000 7fe7e6ffd640 1 -- 172.21.15.113:0/119683970 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7ec044020 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.132+0000 7fe7e6ffd640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7d806be30 0x7fe7d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.132+0000 7fe7e6ffd640 1 -- 172.21.15.113:0/119683970 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5609+0+0 (secure 0 0 0) 0x7fe7ec0abb90 con 0x7fe7f80ccd50 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.132+0000 7fe7f659c640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7d806be30 0x7fe7d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:21.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.132+0000 7fe7f659c640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7d806be30 0x7fe7d806e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8007440 tx=0x7fe7e800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:21.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.136+0000 7fe7e6ffd640 1 -- 172.21.15.113:0/119683970 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7ec07a680 con 0x7fe7f80ccd50 2024-04-25T00:02:21.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:21.348+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe7f8159360 con 0x7fe7f80ccd50 2024-04-25T00:02:22.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.000+0000 7fe7e6ffd640 1 -- 172.21.15.113:0/119683970 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v41) ==== 174+0+0 (secure 0 0 0) 0x7fe7ec079f80 con 0x7fe7f80ccd50 2024-04-25T00:02:22.000 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T00:02:22.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.004+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7d806be30 msgr2=0x7fe7d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.004+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7d806be30 0x7fe7d806e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8007440 tx=0x7fe7e800b040 comp rx=0 tx=0).stop 2024-04-25T00:02:22.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.004+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f80ccd50 msgr2=0x7fe7f80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.004+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f80ccd50 0x7fe7f80b8a50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c240 tx=0x7fe7f80d2fd0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.005+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 shutdown_connections 2024-04-25T00:02:22.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.005+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7d806be30 0x7fe7d806e2f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.005+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f80cd1d0 0x7fe7f80b8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.005+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f80ccd50 0x7fe7f80b8a50 secure :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c240 tx=0x7fe7f80d2fd0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.005+0000 7fe7fcfc8640 1 --2- 172.21.15.113:0/119683970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7e0004800 0x7fe7f81f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.005+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 >> 172.21.15.113:0/119683970 conn(0x7fe7f806b720 msgr2=0x7fe7f815c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:22.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.007+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 shutdown_connections 2024-04-25T00:02:22.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.007+0000 7fe7fcfc8640 1 -- 172.21.15.113:0/119683970 wait complete. 2024-04-25T00:02:22.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:22.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.127+0000 7ff9b579e640 1 Processor -- start 2024-04-25T00:02:22.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.127+0000 7ff9b579e640 1 -- start start 2024-04-25T00:02:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9b579e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9b00d2290 0x7ff9b00b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9b579e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff9b00d0b00 0x7ff9b00d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9b579e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b00d1480 con 0x7ff9b015e320 2024-04-25T00:02:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9b579e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00d3410 con 0x7ff9b00d2290 2024-04-25T00:02:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9b579e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00d2800 con 0x7ff9b00d0b00 2024-04-25T00:02:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9aeffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff9b015e320 0x7ff9b00d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58144/0 (socket says 172.21.15.113:58144) 2024-04-25T00:02:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9aeffd640 1 -- 172.21.15.113:0/1840695528 learned_addr learned my addr 172.21.15.113:0/1840695528 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9ae7fc640 1 --2- 172.21.15.113:0/1840695528 >> v2:172.21.15.113:3300/0 conn(0x7ff9b00d2290 0x7ff9b00b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9adffb640 1 -- 172.21.15.113:0/1840695528 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3483256182 0 0) 0x7ff9b00d1480 con 0x7ff9b015e320 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9adffb640 1 -- 172.21.15.113:0/1840695528 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff998003520 con 0x7ff9b015e320 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9ae7fc640 1 -- 172.21.15.113:0/1840695528 >> v2:172.21.15.178:3300/0 conn(0x7ff9b00d0b00 msgr2=0x7ff9b00d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9ae7fc640 1 --2- 172.21.15.113:0/1840695528 >> v2:172.21.15.178:3300/0 conn(0x7ff9b00d0b00 0x7ff9b00d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.128+0000 7ff9ae7fc640 1 -- 172.21.15.113:0/1840695528 >> v1:172.21.15.113:6789/0 conn(0x7ff9b015e320 legacy=0x7ff9b00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.129+0000 7ff9ae7fc640 1 -- 172.21.15.113:0/1840695528 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00d2ae0 con 0x7ff9b00d2290 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.129+0000 7ff9ae7fc640 1 --2- 172.21.15.113:0/1840695528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b00b9aa0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00b3970 tx=0x7ff9a402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658965af28c1a1fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.129+0000 7ff9adffb640 1 -- 172.21.15.113:0/1840695528 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4037070 con 0x7ff9b00d2290 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.129+0000 7ff9adffb640 1 -- 172.21.15.113:0/1840695528 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a4032440 con 0x7ff9b00d2290 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.129+0000 7ff9adffb640 1 -- 172.21.15.113:0/1840695528 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40327f0 con 0x7ff9b00d2290 2024-04-25T00:02:22.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 -- 172.21.15.113:0/1840695528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 msgr2=0x7ff9b00b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1840695528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b00b9aa0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00b3970 tx=0x7ff9a402a250 comp rx=0 tx=0).stop 2024-04-25T00:02:22.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 -- 172.21.15.113:0/1840695528 shutdown_connections 2024-04-25T00:02:22.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1840695528 >> v2:172.21.15.178:3300/0 conn(0x7ff9b00d0b00 0x7ff9b00d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1840695528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b00b9aa0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 -- 172.21.15.113:0/1840695528 >> 172.21.15.113:0/1840695528 conn(0x7ff9b00b5c20 msgr2=0x7ff9b00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:22.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 -- 172.21.15.113:0/1840695528 shutdown_connections 2024-04-25T00:02:22.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.130+0000 7ff9b579e640 1 -- 172.21.15.113:0/1840695528 wait complete. 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.131+0000 7ff9b579e640 1 Processor -- start 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.131+0000 7ff9b579e640 1 -- start start 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9b579e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b00d0b00 0x7ff9b00d4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9b579e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b0158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9b579e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b015e320 0x7ff9b01587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9b579e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00b9fe0 con 0x7ff9b00d2290 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9b579e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00d30a0 con 0x7ff9b015e320 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9b579e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b015e700 con 0x7ff9b00d0b00 2024-04-25T00:02:22.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b0158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54402/0 (socket says 172.21.15.113:54402) 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 -- 172.21.15.113:0/1093186368 learned_addr learned my addr 172.21.15.113:0/1093186368 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9af7fe640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b015e320 0x7ff9b01587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9aeffd640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b00d0b00 0x7ff9b00d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 -- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b00d0b00 msgr2=0x7ff9b00d4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b00d0b00 0x7ff9b00d4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 -- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b015e320 msgr2=0x7ff9b01587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b015e320 0x7ff9b01587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9ae7fc640 1 -- 172.21.15.113:0/1093186368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b01f48a0 con 0x7ff9b00d2290 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9af7fe640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b015e320 0x7ff9b01587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.132+0000 7ff9aeffd640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b00d0b00 0x7ff9b00d4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.133+0000 7ff9ae7fc640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b0158270 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff9a402a290 tx=0x7ff9a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.133+0000 7ff9977fe640 1 -- 172.21.15.113:0/1093186368 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4037040 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.133+0000 7ff9977fe640 1 -- 172.21.15.113:0/1093186368 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a4042070 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.133+0000 7ff9977fe640 1 -- 172.21.15.113:0/1093186368 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4032810 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.133+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b01f7340 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.133+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b01f7900 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.134+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b00691a0 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.134+0000 7ff9977fe640 1 -- 172.21.15.113:0/1093186368 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9a40329b0 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.134+0000 7ff9977fe640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.135+0000 7ff9977fe640 1 -- 172.21.15.113:0/1093186368 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7ff9a40ab790 con 0x7ff9b00d2290 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.135+0000 7ff9aeffd640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.135+0000 7ff9aeffd640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0005c00 tx=0x7ff9a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.139+0000 7ff9977fe640 1 -- 172.21.15.113:0/1093186368 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9a407a140 con 0x7ff9b00d2290 2024-04-25T00:02:22.266 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:02:22.269 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:02:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.319+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9b00d4d70 con 0x7ff9b00d2290 2024-04-25T00:02:22.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.320+0000 7ff9977fe640 1 -- 172.21.15.113:0/1093186368 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16729 (secure 0 0 0) 0x7ff9a4079a40 con 0x7ff9b00d2290 2024-04-25T00:02:22.321 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:22.321 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":75,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:02:21.349871+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:02:19.562437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:02:20.315341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:22.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 msgr2=0x7ff99c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0005c00 tx=0x7ff9a0009040 comp rx=0 tx=0).stop 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 msgr2=0x7ff9b0158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b0158270 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff9a402a290 tx=0x7ff9a40046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 shutdown_connections 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b015e320 0x7ff9b01587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9b00d2290 0x7ff9b0158270 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 --2- 172.21.15.113:0/1093186368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b00d0b00 0x7ff9b00d4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 >> 172.21.15.113:0/1093186368 conn(0x7ff9b00b5c20 msgr2=0x7ff9b00b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 shutdown_connections 2024-04-25T00:02:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.322+0000 7ff9b579e640 1 -- 172.21.15.113:0/1093186368 wait complete. 2024-04-25T00:02:22.335 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:22.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.472+0000 7f544bcca640 1 Processor -- start 2024-04-25T00:02:22.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544bcca640 1 -- start start 2024-04-25T00:02:22.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544bcca640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f544415e320 0x7f54440d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544bcca640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54440d2290 0x7f54440b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544bcca640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54440d3de0 con 0x7f54440d0b00 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544bcca640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f54440d3410 con 0x7f54440d2290 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544bcca640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f54440d2800 con 0x7f544415e320 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f5449265640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54440d2290 0x7f54440b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544a267640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f54440d0b00 0x7f54440b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58148/0 (socket says 172.21.15.113:58148) 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.473+0000 7f544a267640 1 -- 172.21.15.113:0/2961927009 learned_addr learned my addr 172.21.15.113:0/2961927009 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.474+0000 7f5449265640 1 -- 172.21.15.113:0/2961927009 >> v2:172.21.15.178:3300/0 conn(0x7f544415e320 msgr2=0x7f54440d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.474+0000 7f5449265640 1 --2- 172.21.15.113:0/2961927009 >> v2:172.21.15.178:3300/0 conn(0x7f544415e320 0x7f54440d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.474+0000 7f5449265640 1 -- 172.21.15.113:0/2961927009 >> v1:172.21.15.113:6789/0 conn(0x7f54440d0b00 legacy=0x7f54440b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.474+0000 7f5449265640 1 -- 172.21.15.113:0/2961927009 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54440d2ae0 con 0x7f54440d2290 2024-04-25T00:02:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.474+0000 7f5448a64640 1 -- 172.21.15.113:0/2961927009 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71942684 0 0) 0x7f54440d3de0 con 0x7f54440d0b00 2024-04-25T00:02:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.474+0000 7f5449265640 1 --2- 172.21.15.113:0/2961927009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f54440b9aa0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f54440b6d00 tx=0x7f543402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a69381a1a203451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f5448a64640 1 -- 172.21.15.113:0/2961927009 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434037070 con 0x7f54440d2290 2024-04-25T00:02:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f5448a64640 1 -- 172.21.15.113:0/2961927009 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5434032440 con 0x7f54440d2290 2024-04-25T00:02:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f5448a64640 1 -- 172.21.15.113:0/2961927009 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434032810 con 0x7f54440d2290 2024-04-25T00:02:22.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 -- 172.21.15.113:0/2961927009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 msgr2=0x7f54440b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 --2- 172.21.15.113:0/2961927009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f54440b9aa0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f54440b6d00 tx=0x7f543402a230 comp rx=0 tx=0).stop 2024-04-25T00:02:22.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 -- 172.21.15.113:0/2961927009 shutdown_connections 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 --2- 172.21.15.113:0/2961927009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f54440b9aa0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 --2- 172.21.15.113:0/2961927009 >> v2:172.21.15.178:3300/0 conn(0x7f544415e320 0x7f54440d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 -- 172.21.15.113:0/2961927009 >> 172.21.15.113:0/2961927009 conn(0x7f54440b5c20 msgr2=0x7f54440b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 -- 172.21.15.113:0/2961927009 shutdown_connections 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.475+0000 7f544bcca640 1 -- 172.21.15.113:0/2961927009 wait complete. 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.476+0000 7f544bcca640 1 Processor -- start 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.476+0000 7f544bcca640 1 -- start start 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f544bcca640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54440d0b00 0x7f54441f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f544bcca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f544415f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f544bcca640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544415e320 0x7f54440b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f544bcca640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54440b9fe0 con 0x7f54440d2290 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f544bcca640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54440d30a0 con 0x7f544415e320 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f544bcca640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f544415e700 con 0x7f54440d0b00 2024-04-25T00:02:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f5449265640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f544415f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f5449265640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f544415f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54412/0 (socket says 172.21.15.113:54412) 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f5449265640 1 -- 172.21.15.113:0/3097542090 learned_addr learned my addr 172.21.15.113:0/3097542090 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f5449265640 1 -- 172.21.15.113:0/3097542090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54440d0b00 msgr2=0x7f54441f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f5449265640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54440d0b00 0x7f54441f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.477+0000 7f5449265640 1 -- 172.21.15.113:0/3097542090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544415e320 msgr2=0x7f54440b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f5449265640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544415e320 0x7f54440b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f5449265640 1 -- 172.21.15.113:0/3097542090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54440cc870 con 0x7f54440d2290 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f5449265640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f544415f260 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f5434035040 tx=0x7f54340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f543a7fc640 1 -- 172.21.15.113:0/3097542090 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434037040 con 0x7f54440d2290 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f543a7fc640 1 -- 172.21.15.113:0/3097542090 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5434004230 con 0x7f54440d2290 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544415fd20 con 0x7f54440d2290 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f543a7fc640 1 -- 172.21.15.113:0/3097542090 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434032e20 con 0x7f54440d2290 2024-04-25T00:02:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.478+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54441602e0 con 0x7f54440d2290 2024-04-25T00:02:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.479+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54440691a0 con 0x7f54440d2290 2024-04-25T00:02:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.479+0000 7f543a7fc640 1 -- 172.21.15.113:0/3097542090 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5434030770 con 0x7f54440d2290 2024-04-25T00:02:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.479+0000 7f543a7fc640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.480+0000 7f543a7fc640 1 -- 172.21.15.113:0/3097542090 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f54340ac0f0 con 0x7f54440d2290 2024-04-25T00:02:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.480+0000 7f5449a66640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.480+0000 7f5449a66640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f542806be30 0x7f542806e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f54440b9290 tx=0x7f5440009250 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.484+0000 7f543a7fc640 1 -- 172.21.15.113:0/3097542090 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f543407ac50 con 0x7f54440d2290 2024-04-25T00:02:22.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.656+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54440b7730 con 0x7f54440d2290 2024-04-25T00:02:22.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.657+0000 7f543a7fc640 1 -- 172.21.15.113:0/3097542090 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16729 (secure 0 0 0) 0x7f543407a550 con 0x7f54440d2290 2024-04-25T00:02:22.658 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:22.658 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":75,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:02:21.349871+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:02:19.562437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:02:20.315341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f542806be30 msgr2=0x7f542806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f542806be30 0x7f542806e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f54440b9290 tx=0x7f5440009250 comp rx=0 tx=0).stop 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 msgr2=0x7f544415f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f544415f260 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f5434035040 tx=0x7f54340046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 shutdown_connections 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f542806be30 0x7f542806e2f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544415e320 0x7f54440b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54440d2290 0x7f544415f260 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 --2- 172.21.15.113:0/3097542090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54440d0b00 0x7f54441f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 >> 172.21.15.113:0/3097542090 conn(0x7f54440b5c20 msgr2=0x7f54440b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:22.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 shutdown_connections 2024-04-25T00:02:22.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.659+0000 7f544bcca640 1 -- 172.21.15.113:0/3097542090 wait complete. 2024-04-25T00:02:22.671 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:02:22.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.769+0000 7fcfff509640 1 Processor -- start 2024-04-25T00:02:22.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.770+0000 7fcfff509640 1 -- start start 2024-04-25T00:02:22.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.770+0000 7fcfff509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff80b5ca0 0x7fcff80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.770+0000 7fcfff509640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcff80b73c0 0x7fcff80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.770+0000 7fcfff509640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff80d2e00 con 0x7fcff815e510 2024-04-25T00:02:22.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.770+0000 7fcfff509640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcff80b6640 con 0x7fcff80b73c0 2024-04-25T00:02:22.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.770+0000 7fcfff509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcff80d30e0 con 0x7fcff80b5ca0 2024-04-25T00:02:22.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffdaa6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcff80b73c0 0x7fcff80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffdaa6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcff80b73c0 0x7fcff80b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54416/0 (socket says 172.21.15.113:54416) 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffdaa6640 1 -- 172.21.15.113:0/615683590 learned_addr learned my addr 172.21.15.113:0/615683590 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffd2a5640 1 --1- 172.21.15.113:0/615683590 >> v1:172.21.15.113:6789/0 conn(0x7fcff815e510 0x7fcff80d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58154/0 (socket says 172.21.15.113:58154) 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffcaa4640 1 --2- 172.21.15.113:0/615683590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff80b5ca0 0x7fcff80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffdaa6640 1 -- 172.21.15.113:0/615683590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff80b5ca0 msgr2=0x7fcff80b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffdaa6640 1 --2- 172.21.15.113:0/615683590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff80b5ca0 0x7fcff80b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffdaa6640 1 -- 172.21.15.113:0/615683590 >> v1:172.21.15.113:6789/0 conn(0x7fcff815e510 legacy=0x7fcff80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcfebfff640 1 -- 172.21.15.113:0/615683590 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078700431 0 0) 0x7fcff80d2e00 con 0x7fcff815e510 2024-04-25T00:02:22.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.771+0000 7fcffdaa6640 1 -- 172.21.15.113:0/615683590 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff80d33c0 con 0x7fcff80b73c0 2024-04-25T00:02:22.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcffdaa6640 1 --2- 172.21.15.113:0/615683590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80b7830 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009e50 tx=0x7fcff002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa1b4f38459aa3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfebfff640 1 -- 172.21.15.113:0/615683590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0036070 con 0x7fcff80b73c0 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfebfff640 1 -- 172.21.15.113:0/615683590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff0031600 con 0x7fcff80b73c0 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfebfff640 1 -- 172.21.15.113:0/615683590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00319d0 con 0x7fcff80b73c0 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 -- 172.21.15.113:0/615683590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 msgr2=0x7fcff80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 --2- 172.21.15.113:0/615683590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80b7830 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009e50 tx=0x7fcff002a680 comp rx=0 tx=0).stop 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 -- 172.21.15.113:0/615683590 shutdown_connections 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 --2- 172.21.15.113:0/615683590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80b7830 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 --2- 172.21.15.113:0/615683590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff80b5ca0 0x7fcff80b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 -- 172.21.15.113:0/615683590 >> 172.21.15.113:0/615683590 conn(0x7fcff806b720 msgr2=0x7fcff80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 -- 172.21.15.113:0/615683590 shutdown_connections 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.772+0000 7fcfff509640 1 -- 172.21.15.113:0/615683590 wait complete. 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.773+0000 7fcfff509640 1 Processor -- start 2024-04-25T00:02:22.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcfff509640 1 -- start start 2024-04-25T00:02:22.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcfff509640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff80b5ca0 0x7fcff815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcfff509640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcfff509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff815e510 0x7fcff8160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcfff509640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcff80d3ea0 con 0x7fcff80b73c0 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcfff509640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcff80d3980 con 0x7fcff80b5ca0 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcfff509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcff80d2a90 con 0x7fcff815e510 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcffcaa4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcffcaa4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54432/0 (socket says 172.21.15.113:54432) 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.774+0000 7fcffcaa4640 1 -- 172.21.15.113:0/3369993589 learned_addr learned my addr 172.21.15.113:0/3369993589 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcffcaa4640 1 -- 172.21.15.113:0/3369993589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff815e510 msgr2=0x7fcff8160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcffcaa4640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff815e510 0x7fcff8160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcffcaa4640 1 -- 172.21.15.113:0/3369993589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff80b5ca0 msgr2=0x7fcff815c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcffcaa4640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff80b5ca0 0x7fcff815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcffcaa4640 1 -- 172.21.15.113:0/3369993589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff80c8290 con 0x7fcff80b73c0 2024-04-25T00:02:22.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcffcaa4640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80c7d50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00d5f0 tx=0x7fcfec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcfe9ffb640 1 -- 172.21.15.113:0/3369993589 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0125f0 con 0x7fcff80b73c0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcfe9ffb640 1 -- 172.21.15.113:0/3369993589 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec019070 con 0x7fcff80b73c0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcfe9ffb640 1 -- 172.21.15.113:0/3369993589 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0129d0 con 0x7fcff80b73c0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81587c0 con 0x7fcff80b73c0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.775+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff8158d80 con 0x7fcff80b73c0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.776+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff80691a0 con 0x7fcff80b73c0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.776+0000 7fcfe9ffb640 1 -- 172.21.15.113:0/3369993589 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcfec012b70 con 0x7fcff80b73c0 2024-04-25T00:02:22.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.777+0000 7fcfe9ffb640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfdc06be30 0x7fcfdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:22.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.777+0000 7fcffd2a5640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfdc06be30 0x7fcfdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:22.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.777+0000 7fcfe9ffb640 1 -- 172.21.15.113:0/3369993589 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7fcfec08b270 con 0x7fcff80b73c0 2024-04-25T00:02:22.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.777+0000 7fcffd2a5640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfdc06be30 0x7fcfdc06e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcff4007fd0 tx=0x7fcff4007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:22.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.781+0000 7fcfe9ffb640 1 -- 172.21.15.113:0/3369993589 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcfec059c20 con 0x7fcff80b73c0 2024-04-25T00:02:22.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:22.991+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcff80c9010 con 0x7fcff80b73c0 2024-04-25T00:02:23.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.001+0000 7fcfe9ffb640 1 -- 172.21.15.113:0/3369993589 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7fcfec059520 con 0x7fcff80b73c0 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfdc06be30 msgr2=0x7fcfdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfdc06be30 0x7fcfdc06e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcff4007fd0 tx=0x7fcff4007920 comp rx=0 tx=0).stop 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 msgr2=0x7fcff80c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80c7d50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00d5f0 tx=0x7fcfec00dab0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 shutdown_connections 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcfdc06be30 0x7fcfdc06e2f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff815e510 0x7fcff8160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcff80b73c0 0x7fcff80c7d50 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 --2- 172.21.15.113:0/3369993589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff80b5ca0 0x7fcff815c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 >> 172.21.15.113:0/3369993589 conn(0x7fcff806b720 msgr2=0x7fcff8071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 shutdown_connections 2024-04-25T00:02:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.003+0000 7fcfff509640 1 -- 172.21.15.113:0/3369993589 wait complete. 2024-04-25T00:02:23.016 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:02:23.016 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:02:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.114+0000 7f37535f9640 1 Processor -- start 2024-04-25T00:02:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.114+0000 7f37535f9640 1 -- start start 2024-04-25T00:02:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f37535f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f374c15ba50 0x7f374c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f37535f9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f374c06cea0 0x7f374c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f37535f9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374c06d7e0 con 0x7f374c15b670 2024-04-25T00:02:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f37535f9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f374c071170 con 0x7f374c15ba50 2024-04-25T00:02:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f37535f9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f374c071450 con 0x7f374c06cea0 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3750b94640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f374c15ba50 0x7f374c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3751395640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f374c15b670 0x7f374c0cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58158/0 (socket says 172.21.15.113:58158) 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3750b94640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f374c15ba50 0x7f374c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54444/0 (socket says 172.21.15.113:54444) 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3751395640 1 -- 172.21.15.113:0/4238170043 learned_addr learned my addr 172.21.15.113:0/4238170043 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3751b96640 1 --2- 172.21.15.113:0/4238170043 >> v2:172.21.15.178:3300/0 conn(0x7f374c06cea0 0x7f374c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3750b94640 1 -- 172.21.15.113:0/4238170043 >> v2:172.21.15.178:3300/0 conn(0x7f374c06cea0 msgr2=0x7f374c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3750b94640 1 --2- 172.21.15.113:0/4238170043 >> v2:172.21.15.178:3300/0 conn(0x7f374c06cea0 0x7f374c06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3750b94640 1 -- 172.21.15.113:0/4238170043 >> v1:172.21.15.113:6789/0 conn(0x7f374c15b670 legacy=0x7f374c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3750b94640 1 -- 172.21.15.113:0/4238170043 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c071730 con 0x7f374c15ba50 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.115+0000 7f3743fff640 1 -- 172.21.15.113:0/4238170043 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189532869 0 0) 0x7f374c06d7e0 con 0x7f374c15b670 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.116+0000 7f3750b94640 1 --2- 172.21.15.113:0/4238170043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c15bff0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f374c1589f0 tx=0x7f373c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54030733d5ce997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.116+0000 7f3743fff640 1 -- 172.21.15.113:0/4238170043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c037070 con 0x7f374c15ba50 2024-04-25T00:02:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.116+0000 7f3743fff640 1 -- 172.21.15.113:0/4238170043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373c02add0 con 0x7f374c15ba50 2024-04-25T00:02:23.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.116+0000 7f3743fff640 1 -- 172.21.15.113:0/4238170043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c032660 con 0x7f374c15ba50 2024-04-25T00:02:23.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 -- 172.21.15.113:0/4238170043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 msgr2=0x7f374c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:23.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 --2- 172.21.15.113:0/4238170043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c15bff0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f374c1589f0 tx=0x7f373c02a080 comp rx=0 tx=0).stop 2024-04-25T00:02:23.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 -- 172.21.15.113:0/4238170043 shutdown_connections 2024-04-25T00:02:23.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 --2- 172.21.15.113:0/4238170043 >> v2:172.21.15.178:3300/0 conn(0x7f374c06cea0 0x7f374c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 --2- 172.21.15.113:0/4238170043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c15bff0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 -- 172.21.15.113:0/4238170043 >> 172.21.15.113:0/4238170043 conn(0x7f374c06b0f0 msgr2=0x7f374c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 -- 172.21.15.113:0/4238170043 shutdown_connections 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.117+0000 7f37535f9640 1 -- 172.21.15.113:0/4238170043 wait complete. 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.118+0000 7f37535f9640 1 Processor -- start 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.118+0000 7f37535f9640 1 -- start start 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f37535f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f374c06cea0 0x7f374c145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f37535f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f374c15b670 0x7f374c1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f37535f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f37535f9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f374c0cca90 con 0x7f374c15ba50 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f37535f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f374c0cc910 con 0x7f374c15b670 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f37535f9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f374c0d0080 con 0x7f374c06cea0 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54452/0 (socket says 172.21.15.113:54452) 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 -- 172.21.15.113:0/1270315745 learned_addr learned my addr 172.21.15.113:0/1270315745 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751395640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f374c06cea0 0x7f374c145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3750b94640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f374c15b670 0x7f374c1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 -- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f374c06cea0 msgr2=0x7f374c145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f374c06cea0 0x7f374c145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 -- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f374c15b670 msgr2=0x7f374c1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f374c15b670 0x7f374c1470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 -- 172.21.15.113:0/1270315745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c14a160 con 0x7f374c15ba50 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751395640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f374c06cea0 0x7f374c145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3750b94640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f374c15b670 0x7f374c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3751b96640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c150530 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f374400d800 tx=0x7f374400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3741ffb640 1 -- 172.21.15.113:0/1270315745 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374400b6c0 con 0x7f374c15ba50 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3741ffb640 1 -- 172.21.15.113:0/1270315745 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3744019070 con 0x7f374c15ba50 2024-04-25T00:02:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c14a440 con 0x7f374c15ba50 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.119+0000 7f3741ffb640 1 -- 172.21.15.113:0/1270315745 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374400bac0 con 0x7f374c15ba50 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.120+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c14aa00 con 0x7f374c15ba50 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.120+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374c0691a0 con 0x7f374c15ba50 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.121+0000 7f3741ffb640 1 -- 172.21.15.113:0/1270315745 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f374400bc60 con 0x7f374c15ba50 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.121+0000 7f3741ffb640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f373006be30 0x7f373006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.121+0000 7f3741ffb640 1 -- 172.21.15.113:0/1270315745 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f374408b280 con 0x7f374c15ba50 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.121+0000 7f3751395640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f373006be30 0x7f373006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.121+0000 7f3751395640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f373006be30 0x7f373006e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3748007fd0 tx=0x7f3748005b70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.125+0000 7f3741ffb640 1 -- 172.21.15.113:0/1270315745 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3744059c30 con 0x7f374c15ba50 2024-04-25T00:02:23.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:23.335+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f374c158300 con 0x7f374c15ba50 2024-04-25T00:02:24.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.010+0000 7f3741ffb640 1 -- 172.21.15.113:0/1270315745 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7f3744059530 con 0x7f374c15ba50 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f373006be30 msgr2=0x7f373006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f373006be30 0x7f373006e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3748007fd0 tx=0x7f3748005b70 comp rx=0 tx=0).stop 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 msgr2=0x7f374c150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c150530 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f374400d800 tx=0x7f374400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 shutdown_connections 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f373006be30 0x7f373006e2f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f374c15b670 0x7f374c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f374c15ba50 0x7f374c150530 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 --2- 172.21.15.113:0/1270315745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f374c06cea0 0x7f374c145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.012+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 >> 172.21.15.113:0/1270315745 conn(0x7f374c06b0f0 msgr2=0x7f374c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.013+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 shutdown_connections 2024-04-25T00:02:24.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.013+0000 7f37535f9640 1 -- 172.21.15.113:0/1270315745 wait complete. 2024-04-25T00:02:24.024 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:02:24.025 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:02:24.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.122+0000 7f3fed0ee640 1 Processor -- start 2024-04-25T00:02:24.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.122+0000 7f3fed0ee640 1 -- start start 2024-04-25T00:02:24.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fed0ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe815a6b0 0x7f3fe8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fed0ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3fe8072ea0 0x7f3fe8071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fed0ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe80b2d50 con 0x7f3fe8071780 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fed0ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe8072a70 con 0x7f3fe8072ea0 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fed0ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe80b3010 con 0x7f3fe815a6b0 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fe659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3fe8072ea0 0x7f3fe8071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fe759e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3fe8071780 0x7f3fe815e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58166/0 (socket says 172.21.15.113:58166) 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.123+0000 7f3fe759e640 1 -- 172.21.15.113:0/193723737 learned_addr learned my addr 172.21.15.113:0/193723737 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.124+0000 7f3fe6d9d640 1 --2- 172.21.15.113:0/193723737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe815a6b0 0x7f3fe8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.124+0000 7f3fe659c640 1 -- 172.21.15.113:0/193723737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe815a6b0 msgr2=0x7f3fe8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.124+0000 7f3fe659c640 1 --2- 172.21.15.113:0/193723737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe815a6b0 0x7f3fe8070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.124+0000 7f3fe659c640 1 -- 172.21.15.113:0/193723737 >> v1:172.21.15.113:6789/0 conn(0x7f3fe8071780 legacy=0x7f3fe815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.124+0000 7f3fe659c640 1 -- 172.21.15.113:0/193723737 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe80b32f0 con 0x7f3fe8072ea0 2024-04-25T00:02:24.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.124+0000 7f3fe5d9b640 1 -- 172.21.15.113:0/193723737 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1297584337 0 0) 0x7f3fe80b2d50 con 0x7f3fe8071780 2024-04-25T00:02:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.124+0000 7f3fe659c640 1 --2- 172.21.15.113:0/193723737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe8072ea0 0x7f3fe8071240 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009d90 tx=0x7f3fdc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9946db17109a826a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fe5d9b640 1 -- 172.21.15.113:0/193723737 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc007740 con 0x7f3fe8072ea0 2024-04-25T00:02:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fe5d9b640 1 -- 172.21.15.113:0/193723737 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc0078e0 con 0x7f3fe8072ea0 2024-04-25T00:02:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fe5d9b640 1 -- 172.21.15.113:0/193723737 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc007cb0 con 0x7f3fe8072ea0 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 -- 172.21.15.113:0/193723737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe8072ea0 msgr2=0x7f3fe8071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/193723737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe8072ea0 0x7f3fe8071240 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009d90 tx=0x7f3fdc02a660 comp rx=0 tx=0).stop 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 -- 172.21.15.113:0/193723737 shutdown_connections 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/193723737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe8072ea0 0x7f3fe8071240 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/193723737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe815a6b0 0x7f3fe8070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 -- 172.21.15.113:0/193723737 >> 172.21.15.113:0/193723737 conn(0x7f3fe80b1a50 msgr2=0x7f3fe80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 -- 172.21.15.113:0/193723737 shutdown_connections 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.125+0000 7f3fed0ee640 1 -- 172.21.15.113:0/193723737 wait complete. 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 Processor -- start 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 -- start start 2024-04-25T00:02:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe8071780 0x7f3fe814c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe8072ea0 0x7f3fe81569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe815a6b0 0x7f3fe80acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe815aa90 con 0x7f3fe815a6b0 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe8072720 con 0x7f3fe8071780 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fed0ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe815ac10 con 0x7f3fe8072ea0 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fe759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe815a6b0 0x7f3fe80acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fe759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe815a6b0 0x7f3fe80acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54462/0 (socket says 172.21.15.113:54462) 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.127+0000 7f3fe759e640 1 -- 172.21.15.113:0/1114147691 learned_addr learned my addr 172.21.15.113:0/1114147691 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe6d9d640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe8071780 0x7f3fe814c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe659c640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe8072ea0 0x7f3fe81569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe759e640 1 -- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe8072ea0 msgr2=0x7f3fe81569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe759e640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe8072ea0 0x7f3fe81569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe759e640 1 -- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe8071780 msgr2=0x7f3fe814c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe759e640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe8071780 0x7f3fe814c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe759e640 1 -- 172.21.15.113:0/1114147691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe8155910 con 0x7f3fe815a6b0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fe759e640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe815a6b0 0x7f3fe80acec0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800c3c0 tx=0x7f3fd800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fcf7fe640 1 -- 172.21.15.113:0/1114147691 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80074e0 con 0x7f3fe815a6b0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fcf7fe640 1 -- 172.21.15.113:0/1114147691 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd80047e0 con 0x7f3fe815a6b0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe8155bf0 con 0x7f3fe815a6b0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fcf7fe640 1 -- 172.21.15.113:0/1114147691 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8004c50 con 0x7f3fe815a6b0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.128+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe81561b0 con 0x7f3fe815a6b0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.129+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe80691a0 con 0x7f3fe815a6b0 2024-04-25T00:02:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.129+0000 7f3fcf7fe640 1 -- 172.21.15.113:0/1114147691 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fd8011070 con 0x7f3fe815a6b0 2024-04-25T00:02:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.130+0000 7f3fcf7fe640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3fd406be30 0x7f3fd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.130+0000 7f3fe6d9d640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3fd406be30 0x7f3fd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.130+0000 7f3fcf7fe640 1 -- 172.21.15.113:0/1114147691 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f3fd808a9c0 con 0x7f3fe815a6b0 2024-04-25T00:02:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.130+0000 7f3fe6d9d640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3fd406be30 0x7f3fd406e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006fd0 tx=0x7f3fe0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.134+0000 7f3fcf7fe640 1 -- 172.21.15.113:0/1114147691 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3fd808ce20 con 0x7f3fe815a6b0 2024-04-25T00:02:24.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.345+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fe8156640 con 0x7f3fe815a6b0 2024-04-25T00:02:24.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.345+0000 7f3fcf7fe640 1 -- 172.21.15.113:0/1114147691 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5192 (secure 0 0 0) 0x7f3fd801a2b0 con 0x7f3fe815a6b0 2024-04-25T00:02:24.346 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:24.346 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4243948985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4243948985},{"type":"v1","addr":"172.21.15.178:6835","nonce":4243948985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/665506344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":665506344},{"type":"v1","addr":"172.21.15.178:6834","nonce":665506344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4064308986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4064308986},{"type":"v1","addr":"172.21.15.113:6836","nonce":4064308986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:02:21.350179+0000","modified":"2024-04-25T00:02:23.337147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3530277358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3530277358},{"type":"v1","addr":"172.21.15.113:6837","nonce":3530277358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6951,"qdb_cluster":[6951]},"id":5}]} 2024-04-25T00:02:24.347 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 43 2024-04-25T00:02:24.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3fd406be30 msgr2=0x7f3fd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3fd406be30 0x7f3fd406e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006fd0 tx=0x7f3fe0008040 comp rx=0 tx=0).stop 2024-04-25T00:02:24.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe815a6b0 msgr2=0x7f3fe80acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe815a6b0 0x7f3fe80acec0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800c3c0 tx=0x7f3fd800c880 comp rx=0 tx=0).stop 2024-04-25T00:02:24.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 shutdown_connections 2024-04-25T00:02:24.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3fd406be30 0x7f3fd406e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fe815a6b0 0x7f3fe80acec0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe8072ea0 0x7f3fe81569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.347+0000 7f3fed0ee640 1 --2- 172.21.15.113:0/1114147691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe8071780 0x7f3fe814c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.348+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 >> 172.21.15.113:0/1114147691 conn(0x7f3fe80b1a50 msgr2=0x7f3fe80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:24.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.348+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 shutdown_connections 2024-04-25T00:02:24.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.348+0000 7f3fed0ee640 1 -- 172.21.15.113:0/1114147691 wait complete. 2024-04-25T00:02:24.359 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:02:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.454+0000 7f0cdae3f640 1 Processor -- start 2024-04-25T00:02:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.454+0000 7f0cdae3f640 1 -- start start 2024-04-25T00:02:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.455+0000 7f0cdae3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 0x7f0cd4070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.455+0000 7f0cdae3f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0cd40b3af0 0x7f0cd40b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.455+0000 7f0cdae3f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd40b2d50 con 0x7f0cd4072ea0 2024-04-25T00:02:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.455+0000 7f0cdae3f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd4072a70 con 0x7f0cd40b3af0 2024-04-25T00:02:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.455+0000 7f0cdae3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd40b3010 con 0x7f0cd415a6b0 2024-04-25T00:02:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.455+0000 7f0cd3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0cd4072ea0 0x7f0cd4071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58174/0 (socket says 172.21.15.113:58174) 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.455+0000 7f0cd3fff640 1 -- 172.21.15.113:0/2822452035 learned_addr learned my addr 172.21.15.113:0/2822452035 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd93dc640 1 --2- 172.21.15.113:0/2822452035 >> v2:172.21.15.178:3300/0 conn(0x7f0cd40b3af0 0x7f0cd40b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 0x7f0cd4070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829707846 0 0) 0x7f0cd40b2d50 con 0x7f0cd4072ea0 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cbc003520 con 0x7f0cd4072ea0 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3138246382 0 0) 0x7f0cbc003520 con 0x7f0cd4072ea0 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 msgr2=0x7f0cd4070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 0x7f0cd4070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 >> v2:172.21.15.178:3300/0 conn(0x7f0cd40b3af0 msgr2=0x7f0cd40b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 --2- 172.21.15.113:0/2822452035 >> v2:172.21.15.178:3300/0 conn(0x7f0cd40b3af0 0x7f0cd40b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd40b32f0 con 0x7f0cd4072ea0 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0cc8001670 con 0x7f0cd4072ea0 2024-04-25T00:02:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 >> v1:172.21.15.113:6789/0 conn(0x7f0cd4072ea0 legacy=0x7f0cd4071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cbc0039b0 0x7f0cbc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc004360 0x7f0cbc004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 0x7f0cbc004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4072a70 con 0x7f0cbc0039b0 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd40b3010 con 0x7f0cbc004360 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0cbc00b9c0 con 0x7f0cd415a6b0 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cbc0039b0 0x7f0cbc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd3fff640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc004360 0x7f0cbc004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd93dc640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 0x7f0cbc004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd8bdb640 1 -- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 msgr2=0x7f0cbc004f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.456+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd415a6b0 0x7f0cbc004f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd8bdb640 1 -- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc004360 msgr2=0x7f0cbc004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc004360 0x7f0cbc004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd8bdb640 1 -- 172.21.15.113:0/2822452035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc003520 con 0x7f0cbc0039b0 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd3fff640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc004360 0x7f0cbc004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cbc0039b0 0x7f0cbc003d90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009ac0 tx=0x7f0cbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5c4846e9ae8b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4036070 con 0x7f0cbc0039b0 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc4031400 con 0x7f0cbc0039b0 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.457+0000 7f0cd37fe640 1 -- 172.21.15.113:0/2822452035 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4009690 con 0x7f0cbc0039b0 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.458+0000 7f0cdae3f640 1 -- 172.21.15.113:0/2822452035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cbc0039b0 msgr2=0x7f0cbc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.458+0000 7f0cdae3f640 1 --2- 172.21.15.113:0/2822452035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cbc0039b0 0x7f0cbc003d90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009ac0 tx=0x7f0cbc005830 comp rx=0 tx=0).stop 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.458+0000 7f0cd93dc640 1 -- 172.21.15.113:0/2822452035 reap_dead start 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.458+0000 7f0cdae3f640 1 -- 172.21.15.113:0/2822452035 shutdown_connections 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.458+0000 7f0cdae3f640 1 -- 172.21.15.113:0/2822452035 >> 172.21.15.113:0/2822452035 conn(0x7f0cd40b1a50 msgr2=0x7f0cd40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.458+0000 7f0cdae3f640 1 -- 172.21.15.113:0/2822452035 shutdown_connections 2024-04-25T00:02:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.458+0000 7f0cdae3f640 1 -- 172.21.15.113:0/2822452035 wait complete. 2024-04-25T00:02:24.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.459+0000 7f0cdae3f640 1 Processor -- start 2024-04-25T00:02:24.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.459+0000 7f0cdae3f640 1 -- start start 2024-04-25T00:02:24.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cdae3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc0039b0 0x7f0cd413bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cdae3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd413e610 0x7f0cd4142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cdae3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd41432a0 0x7f0cd4143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cdae3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd415aa90 con 0x7f0cd413e610 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cdae3f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4072720 con 0x7f0cbc0039b0 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cdae3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd415ac10 con 0x7f0cd41432a0 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd413e610 0x7f0cd4142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd413e610 0x7f0cd4142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54478/0 (socket says 172.21.15.113:54478) 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 -- 172.21.15.113:0/4240300220 learned_addr learned my addr 172.21.15.113:0/4240300220 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 -- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd41432a0 msgr2=0x7f0cd4143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc0039b0 0x7f0cd413bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd93dc640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd41432a0 0x7f0cd4143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd41432a0 0x7f0cd4143720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 -- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc0039b0 msgr2=0x7f0cd413bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc0039b0 0x7f0cd413bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd3fff640 1 -- 172.21.15.113:0/4240300220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd413fab0 con 0x7f0cd413e610 2024-04-25T00:02:24.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.460+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc0039b0 0x7f0cd413bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:24.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.461+0000 7f0cd3fff640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd413e610 0x7f0cd4142d60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8001750 tx=0x7f0cc8008430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.461+0000 7f0cd17fa640 1 -- 172.21.15.113:0/4240300220 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800eb40 con 0x7f0cd413e610 2024-04-25T00:02:24.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.461+0000 7f0cd17fa640 1 -- 172.21.15.113:0/4240300220 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc8018070 con 0x7f0cd413e610 2024-04-25T00:02:24.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.461+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd414cac0 con 0x7f0cd413e610 2024-04-25T00:02:24.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.461+0000 7f0cd17fa640 1 -- 172.21.15.113:0/4240300220 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8008740 con 0x7f0cd413e610 2024-04-25T00:02:24.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.461+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd414d080 con 0x7f0cd413e610 2024-04-25T00:02:24.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.462+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd41654f0 con 0x7f0cd413e610 2024-04-25T00:02:24.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.462+0000 7f0cd17fa640 1 -- 172.21.15.113:0/4240300220 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cc801d460 con 0x7f0cd413e610 2024-04-25T00:02:24.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.462+0000 7f0cd17fa640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cc006be60 0x7f0cc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.463+0000 7f0cd17fa640 1 -- 172.21.15.113:0/4240300220 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f0cc808ada0 con 0x7f0cd413e610 2024-04-25T00:02:24.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.463+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cc006be60 0x7f0cc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.463+0000 7f0cd8bdb640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cc006be60 0x7f0cc006e320 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4033000 tx=0x7f0cc4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.467+0000 7f0cd17fa640 1 -- 172.21.15.113:0/4240300220 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0cd41654f0 con 0x7f0cd413e610 2024-04-25T00:02:24.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.639+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cd414d280 con 0x7f0cd413e610 2024-04-25T00:02:24.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.640+0000 7f0cd17fa640 1 -- 172.21.15.113:0/4240300220 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16716 (secure 0 0 0) 0x7f0cd414d280 con 0x7f0cd413e610 2024-04-25T00:02:24.641 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:24.641 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:02:23.001110+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:02:19.562437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:02:20.315341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:02:24.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cc006be60 msgr2=0x7f0cc006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cc006be60 0x7f0cc006e320 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4033000 tx=0x7f0cc4008030 comp rx=0 tx=0).stop 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd413e610 msgr2=0x7f0cd4142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd413e610 0x7f0cd4142d60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8001750 tx=0x7f0cc8008430 comp rx=0 tx=0).stop 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 shutdown_connections 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cc006be60 0x7f0cc006e320 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd41432a0 0x7f0cd4143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd413e610 0x7f0cd4142d60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 --2- 172.21.15.113:0/4240300220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cbc0039b0 0x7f0cd413bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 >> 172.21.15.113:0/4240300220 conn(0x7f0cd40b1a50 msgr2=0x7f0cd40ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 shutdown_connections 2024-04-25T00:02:24.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.642+0000 7f0cdae3f640 1 -- 172.21.15.113:0/4240300220 wait complete. 2024-04-25T00:02:24.654 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:02:24.654 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:02:24.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.751+0000 7f0282eb3640 1 Processor -- start 2024-04-25T00:02:24.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.751+0000 7f0282eb3640 1 -- start start 2024-04-25T00:02:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.752+0000 7f0282eb3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f027c15e320 0x7f027c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.752+0000 7f0282eb3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f027c0d3de0 con 0x7f027c0d2290 2024-04-25T00:02:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.752+0000 7f0282eb3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f027c0d2800 con 0x7f027c0d0b00 2024-04-25T00:02:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.752+0000 7f0282eb3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f027c0d3410 con 0x7f027c15e320 2024-04-25T00:02:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.752+0000 7f027bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f027c0d2290 0x7f027c0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58178/0 (socket says 172.21.15.113:58178) 2024-04-25T00:02:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.752+0000 7f027bfff640 1 -- 172.21.15.113:0/3919347630 learned_addr learned my addr 172.21.15.113:0/3919347630 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f0280c4f640 1 --2- 172.21.15.113:0/3919347630 >> v2:172.21.15.178:3300/0 conn(0x7f027c15e320 0x7f027c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605418598 0 0) 0x7f027c0d3de0 con 0x7f027c0d2290 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0264003520 con 0x7f027c0d2290 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2242480903 0 0) 0x7f0264003520 con 0x7f027c0d2290 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 >> v2:172.21.15.178:3300/0 conn(0x7f027c15e320 msgr2=0x7f027c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 --2- 172.21.15.113:0/3919347630 >> v2:172.21.15.178:3300/0 conn(0x7f027c15e320 0x7f027c0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 >> v1:172.21.15.178:6789/0 conn(0x7f027c0d0b00 legacy=0x7f027c0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c0d2ae0 con 0x7f027c0d2290 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0270001670 con 0x7f027c0d2290 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f0280c4f640 1 --2- 172.21.15.113:0/3919347630 >> v2:172.21.15.178:3300/0 conn(0x7f027c15e320 0x7f027c0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 >> v1:172.21.15.113:6789/0 conn(0x7f027c0d2290 legacy=0x7f027c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02640039b0 0x7f0264003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0264004800 0x7f0264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02640051c0 0x7f026400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f027c0d3410 con 0x7f0264004800 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f026400c070 con 0x7f02640039b0 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f026400c270 con 0x7f02640051c0 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027bfff640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0264004800 0x7f0264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f0280c4f640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02640039b0 0x7f0264003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f0281450640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02640051c0 0x7f026400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027bfff640 1 -- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02640051c0 msgr2=0x7f026400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027bfff640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02640051c0 0x7f026400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027bfff640 1 -- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02640039b0 msgr2=0x7f0264003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027bfff640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02640039b0 0x7f0264003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f027bfff640 1 -- 172.21.15.113:0/3919347630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0264003520 con 0x7f0264004800 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.753+0000 7f0281450640 1 -- 172.21.15.113:0/3919347630 reap_dead start 2024-04-25T00:02:24.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.754+0000 7f027bfff640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0264004800 0x7f0264004c80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f027000c210 tx=0x7f0264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b760101b2ba848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.754+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270007d30 con 0x7f0264004800 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.754+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0270005ad0 con 0x7f0264004800 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.754+0000 7f027b7fe640 1 -- 172.21.15.113:0/3919347630 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270007070 con 0x7f0264004800 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.755+0000 7f0282eb3640 1 -- 172.21.15.113:0/3919347630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0264004800 msgr2=0x7f0264004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.755+0000 7f0282eb3640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0264004800 0x7f0264004c80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f027000c210 tx=0x7f0264005830 comp rx=0 tx=0).stop 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.755+0000 7f0282eb3640 1 -- 172.21.15.113:0/3919347630 shutdown_connections 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.755+0000 7f0282eb3640 1 --2- 172.21.15.113:0/3919347630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0264004800 0x7f0264004c80 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.755+0000 7f0282eb3640 1 -- 172.21.15.113:0/3919347630 >> 172.21.15.113:0/3919347630 conn(0x7f027c0b5c20 msgr2=0x7f027c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.755+0000 7f0282eb3640 1 -- 172.21.15.113:0/3919347630 shutdown_connections 2024-04-25T00:02:24.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.755+0000 7f0282eb3640 1 -- 172.21.15.113:0/3919347630 wait complete. 2024-04-25T00:02:24.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.756+0000 7f0282eb3640 1 Processor -- start 2024-04-25T00:02:24.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.756+0000 7f0282eb3640 1 -- start start 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0282eb3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0264004800 0x7f027c160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0282eb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f027c1f3b00 0x7f027c0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0282eb3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f027c1f77f0 0x7f027c1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0282eb3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f027c15e700 con 0x7f027c1f77f0 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0282eb3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f027c0d30a0 con 0x7f027c1f3b00 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0282eb3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f027c15e880 con 0x7f0264004800 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f027c1f77f0 0x7f027c1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f027c1f77f0 0x7f027c1f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54498/0 (socket says 172.21.15.113:54498) 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 -- 172.21.15.113:0/1012007304 learned_addr learned my addr 172.21.15.113:0/1012007304 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 -- 172.21.15.113:0/1012007304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0264004800 msgr2=0x7f027c160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0264004800 0x7f027c160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 -- 172.21.15.113:0/1012007304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f027c1f3b00 msgr2=0x7f027c0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f027c1f3b00 0x7f027c0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 -- 172.21.15.113:0/1012007304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c1f42d0 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f0281450640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f027c1f77f0 0x7f027c1f7c70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f027c072720 tx=0x7f027400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.757+0000 7f02797fa640 1 -- 172.21.15.113:0/1012007304 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274003970 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.758+0000 7f02797fa640 1 -- 172.21.15.113:0/1012007304 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0274018070 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.758+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c1f45b0 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.758+0000 7f02797fa640 1 -- 172.21.15.113:0/1012007304 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274003d70 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.758+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c0cdfd0 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.759+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f027c0691a0 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.759+0000 7f02797fa640 1 -- 172.21.15.113:0/1012007304 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02740050d0 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.759+0000 7f02797fa640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f026806be60 0x7f026806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.760+0000 7f0280c4f640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f026806be60 0x7f026806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.760+0000 7f02797fa640 1 -- 172.21.15.113:0/1012007304 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f027408aa50 con 0x7f027c1f77f0 2024-04-25T00:02:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.760+0000 7f0280c4f640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f026806be60 0x7f026806e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f026c009cc0 tx=0x7f026c009780 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.764+0000 7f02797fa640 1 -- 172.21.15.113:0/1012007304 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0274059490 con 0x7f027c1f77f0 2024-04-25T00:02:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.933+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f027c1f4ad0 con 0x7f026806be60 2024-04-25T00:02:24.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.939+0000 7f02797fa640 1 -- 172.21.15.113:0/1012007304 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7f026c009e10 con 0x7f026806be60 2024-04-25T00:02:24.942 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:24.943 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:02:24.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f026806be60 msgr2=0x7f026806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f026806be60 0x7f026806e320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f026c009cc0 tx=0x7f026c009780 comp rx=0 tx=0).stop 2024-04-25T00:02:24.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f027c1f77f0 msgr2=0x7f027c1f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:24.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f027c1f77f0 0x7f027c1f7c70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f027c072720 tx=0x7f027400eb90 comp rx=0 tx=0).stop 2024-04-25T00:02:24.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 shutdown_connections 2024-04-25T00:02:24.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f026806be60 0x7f026806e320 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f027c1f3b00 0x7f027c0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f027c1f77f0 0x7f027c1f7c70 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 --2- 172.21.15.113:0/1012007304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0264004800 0x7f027c160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:24.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 >> 172.21.15.113:0/1012007304 conn(0x7f027c0b5c20 msgr2=0x7f027c0b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:24.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 shutdown_connections 2024-04-25T00:02:24.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:24.942+0000 7f0282eb3640 1 -- 172.21.15.113:0/1012007304 wait complete. 2024-04-25T00:02:24.953 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-04-25T00:02:23.159663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274216,"kb_used_data":8116,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700504,"statfs":{"total":773094113280,"available":772813316096,"internally_reserved":0,"allocated":8310784,"data_stored":4193627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000675"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004482+0000","last_change":"2024-04-25T00:02:21.004482+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004482+0000","last_undegraded":"2024-04-25T00:02:21.004482+0000","last_fullsized":"2024-04-25T00:02:21.004482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004023+0000","last_change":"2024-04-25T00:02:21.004023+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004023+0000","last_undegraded":"2024-04-25T00:02:21.004023+0000","last_fullsized":"2024-04-25T00:02:21.004023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006263+0000","last_change":"2024-04-25T00:02:21.006263+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006263+0000","last_undegraded":"2024-04-25T00:02:21.006263+0000","last_fullsized":"2024-04-25T00:02:21.006263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002424+0000","last_change":"2024-04-25T00:02:21.002424+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002424+0000","last_undegraded":"2024-04-25T00:02:21.002424+0000","last_fullsized":"2024-04-25T00:02:21.002424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002669+0000","last_change":"2024-04-25T00:02:21.002669+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002669+0000","last_undegraded":"2024-04-25T00:02:21.002669+0000","last_fullsized":"2024-04-25T00:02:21.002669+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001702+0000","last_change":"2024-04-25T00:02:21.001702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001702+0000","last_undegraded":"2024-04-25T00:02:21.001702+0000","last_fullsized":"2024-04-25T00:02:21.001702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002415+0000","last_change":"2024-04-25T00:02:21.002415+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002415+0000","last_undegraded":"2024-04-25T00:02:21.002415+0000","last_fullsized":"2024-04-25T00:02:21.002415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003841+0000","last_change":"2024-04-25T00:02:21.003841+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003841+0000","last_undegraded":"2024-04-25T00:02:21.003841+0000","last_fullsized":"2024-04-25T00:02:21.003841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004039+0000","last_change":"2024-04-25T00:02:21.004039+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004039+0000","last_undegraded":"2024-04-25T00:02:21.004039+0000","last_fullsized":"2024-04-25T00:02:21.004039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998227+0000","last_change":"2024-04-25T00:02:20.998227+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998227+0000","last_undegraded":"2024-04-25T00:02:20.998227+0000","last_fullsized":"2024-04-25T00:02:20.998227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003622+0000","last_change":"2024-04-25T00:02:21.003622+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003622+0000","last_undegraded":"2024-04-25T00:02:21.003622+0000","last_fullsized":"2024-04-25T00:02:21.003622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001910+0000","last_change":"2024-04-25T00:02:21.001910+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001910+0000","last_undegraded":"2024-04-25T00:02:21.001910+0000","last_fullsized":"2024-04-25T00:02:21.001910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997224+0000","last_change":"2024-04-25T00:02:20.997224+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997224+0000","last_undegraded":"2024-04-25T00:02:20.997224+0000","last_fullsized":"2024-04-25T00:02:20.997224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005347+0000","last_change":"2024-04-25T00:02:21.005347+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005347+0000","last_undegraded":"2024-04-25T00:02:21.005347+0000","last_fullsized":"2024-04-25T00:02:21.005347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997702+0000","last_change":"2024-04-25T00:02:20.997702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997702+0000","last_undegraded":"2024-04-25T00:02:20.997702+0000","last_fullsized":"2024-04-25T00:02:20.997702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997729+0000","last_change":"2024-04-25T00:02:20.997729+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997729+0000","last_undegraded":"2024-04-25T00:02:20.997729+0000","last_fullsized":"2024-04-25T00:02:20.997729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003670+0000","last_change":"2024-04-25T00:02:21.003670+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003670+0000","last_undegraded":"2024-04-25T00:02:21.003670+0000","last_fullsized":"2024-04-25T00:02:21.003670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000719+0000","last_change":"2024-04-25T00:02:21.000719+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000719+0000","last_undegraded":"2024-04-25T00:02:21.000719+0000","last_fullsized":"2024-04-25T00:02:21.000719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003140+0000","last_change":"2024-04-25T00:02:21.003140+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003140+0000","last_undegraded":"2024-04-25T00:02:21.003140+0000","last_fullsized":"2024-04-25T00:02:21.003140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997349+0000","last_change":"2024-04-25T00:02:20.997349+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997349+0000","last_undegraded":"2024-04-25T00:02:20.997349+0000","last_fullsized":"2024-04-25T00:02:20.997349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998024+0000","last_change":"2024-04-25T00:02:20.998024+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998024+0000","last_undegraded":"2024-04-25T00:02:20.998024+0000","last_fullsized":"2024-04-25T00:02:20.998024+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997322+0000","last_change":"2024-04-25T00:02:20.997322+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997322+0000","last_undegraded":"2024-04-25T00:02:20.997322+0000","last_fullsized":"2024-04-25T00:02:20.997322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002667+0000","last_change":"2024-04-25T00:02:21.002667+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002667+0000","last_undegraded":"2024-04-25T00:02:21.002667+0000","last_fullsized":"2024-04-25T00:02:21.002667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002165+0000","last_change":"2024-04-25T00:02:21.002165+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002165+0000","last_undegraded":"2024-04-25T00:02:21.002165+0000","last_fullsized":"2024-04-25T00:02:21.002165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.999243+0000","last_change":"2024-04-25T00:02:20.999243+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.999243+0000","last_undegraded":"2024-04-25T00:02:20.999243+0000","last_fullsized":"2024-04-25T00:02:20.999243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003685+0000","last_change":"2024-04-25T00:02:21.003685+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003685+0000","last_undegraded":"2024-04-25T00:02:21.003685+0000","last_fullsized":"2024-04-25T00:02:21.003685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004843+0000","last_change":"2024-04-25T00:02:21.004843+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004843+0000","last_undegraded":"2024-04-25T00:02:21.004843+0000","last_fullsized":"2024-04-25T00:02:21.004843+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998059+0000","last_change":"2024-04-25T00:02:20.998059+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998059+0000","last_undegraded":"2024-04-25T00:02:20.998059+0000","last_fullsized":"2024-04-25T00:02:20.998059+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007663+0000","last_change":"2024-04-25T00:02:21.007663+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007663+0000","last_undegraded":"2024-04-25T00:02:21.007663+0000","last_fullsized":"2024-04-25T00:02:21.007663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000548+0000","last_change":"2024-04-25T00:02:21.000548+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000548+0000","last_undegraded":"2024-04-25T00:02:21.000548+0000","last_fullsized":"2024-04-25T00:02:21.000548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001720+0000","last_change":"2024-04-25T00:02:21.001720+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001720+0000","last_undegraded":"2024-04-25T00:02:21.001720+0000","last_fullsized":"2024-04-25T00:02:21.001720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000890+0000","last_change":"2024-04-25T00:02:21.000890+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000890+0000","last_undegraded":"2024-04-25T00:02:21.000890+0000","last_fullsized":"2024-04-25T00:02:21.000890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004905+0000","last_change":"2024-04-25T00:02:21.004905+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004905+0000","last_undegraded":"2024-04-25T00:02:21.004905+0000","last_fullsized":"2024-04-25T00:02:21.004905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014554+0000","last_change":"2024-04-25T00:02:21.012893+0000","last_active":"2024-04-25T00:02:21.014554+0000","last_peered":"2024-04-25T00:02:21.014554+0000","last_clean":"2024-04-25T00:02:21.014554+0000","last_became_active":"2024-04-25T00:02:21.012601+0000","last_became_peered":"2024-04-25T00:02:21.012601+0000","last_unstale":"2024-04-25T00:02:21.014554+0000","last_undegraded":"2024-04-25T00:02:21.014554+0000","last_fullsized":"2024-04-25T00:02:21.014554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.876656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003284+0000","last_change":"2024-04-25T00:02:21.003284+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003284+0000","last_undegraded":"2024-04-25T00:02:21.003284+0000","last_fullsized":"2024-04-25T00:02:21.003284+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013760+0000","last_change":"2024-04-25T00:02:21.011558+0000","last_active":"2024-04-25T00:02:21.013760+0000","last_peered":"2024-04-25T00:02:21.013760+0000","last_clean":"2024-04-25T00:02:21.013760+0000","last_became_active":"2024-04-25T00:02:21.011287+0000","last_became_peered":"2024-04-25T00:02:21.011287+0000","last_unstale":"2024-04-25T00:02:21.013760+0000","last_undegraded":"2024-04-25T00:02:21.013760+0000","last_fullsized":"2024-04-25T00:02:21.013760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:20.889569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002102+0000","last_change":"2024-04-25T00:02:21.002102+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002102+0000","last_undegraded":"2024-04-25T00:02:21.002102+0000","last_fullsized":"2024-04-25T00:02:21.002102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013400+0000","last_change":"2024-04-25T00:02:21.007131+0000","last_active":"2024-04-25T00:02:21.013400+0000","last_peered":"2024-04-25T00:02:21.013400+0000","last_clean":"2024-04-25T00:02:21.013400+0000","last_became_active":"2024-04-25T00:02:21.006720+0000","last_became_peered":"2024-04-25T00:02:21.006720+0000","last_unstale":"2024-04-25T00:02:21.013400+0000","last_undegraded":"2024-04-25T00:02:21.013400+0000","last_fullsized":"2024-04-25T00:02:21.013400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:23.288653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002741+0000","last_change":"2024-04-25T00:02:21.002741+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002741+0000","last_undegraded":"2024-04-25T00:02:21.002741+0000","last_fullsized":"2024-04-25T00:02:21.002741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014923+0000","last_change":"2024-04-25T00:02:21.012185+0000","last_active":"2024-04-25T00:02:21.014923+0000","last_peered":"2024-04-25T00:02:21.014923+0000","last_clean":"2024-04-25T00:02:21.014923+0000","last_became_active":"2024-04-25T00:02:21.011910+0000","last_became_peered":"2024-04-25T00:02:21.011910+0000","last_unstale":"2024-04-25T00:02:21.014923+0000","last_undegraded":"2024-04-25T00:02:21.014923+0000","last_fullsized":"2024-04-25T00:02:21.014923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:24.418612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000075+0000","last_change":"2024-04-25T00:02:21.000075+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000075+0000","last_undegraded":"2024-04-25T00:02:21.000075+0000","last_fullsized":"2024-04-25T00:02:21.000075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016721+0000","last_change":"2024-04-25T00:02:21.015042+0000","last_active":"2024-04-25T00:02:21.016721+0000","last_peered":"2024-04-25T00:02:21.016721+0000","last_clean":"2024-04-25T00:02:21.016721+0000","last_became_active":"2024-04-25T00:02:21.014741+0000","last_became_peered":"2024-04-25T00:02:21.014741+0000","last_unstale":"2024-04-25T00:02:21.016721+0000","last_undegraded":"2024-04-25T00:02:21.016721+0000","last_fullsized":"2024-04-25T00:02:21.016721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:54.347862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997330+0000","last_change":"2024-04-25T00:02:20.997330+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997330+0000","last_undegraded":"2024-04-25T00:02:20.997330+0000","last_fullsized":"2024-04-25T00:02:20.997330+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.010476+0000","last_change":"2024-04-25T00:02:21.008726+0000","last_active":"2024-04-25T00:02:21.010476+0000","last_peered":"2024-04-25T00:02:21.010476+0000","last_clean":"2024-04-25T00:02:21.010476+0000","last_became_active":"2024-04-25T00:02:21.008442+0000","last_became_peered":"2024-04-25T00:02:21.008442+0000","last_unstale":"2024-04-25T00:02:21.010476+0000","last_undegraded":"2024-04-25T00:02:21.010476+0000","last_fullsized":"2024-04-25T00:02:21.010476+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:24.591088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005709+0000","last_change":"2024-04-25T00:02:21.005709+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005709+0000","last_undegraded":"2024-04-25T00:02:21.005709+0000","last_fullsized":"2024-04-25T00:02:21.005709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013907+0000","last_change":"2024-04-25T00:02:21.011239+0000","last_active":"2024-04-25T00:02:21.013907+0000","last_peered":"2024-04-25T00:02:21.013907+0000","last_clean":"2024-04-25T00:02:21.013907+0000","last_became_active":"2024-04-25T00:02:21.010824+0000","last_became_peered":"2024-04-25T00:02:21.010824+0000","last_unstale":"2024-04-25T00:02:21.013907+0000","last_undegraded":"2024-04-25T00:02:21.013907+0000","last_fullsized":"2024-04-25T00:02:21.013907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:24.542662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997196+0000","last_change":"2024-04-25T00:02:20.997196+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997196+0000","last_undegraded":"2024-04-25T00:02:20.997196+0000","last_fullsized":"2024-04-25T00:02:20.997196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014422+0000","last_change":"2024-04-25T00:02:21.013287+0000","last_active":"2024-04-25T00:02:21.014422+0000","last_peered":"2024-04-25T00:02:21.014422+0000","last_clean":"2024-04-25T00:02:21.014422+0000","last_became_active":"2024-04-25T00:02:21.013026+0000","last_became_peered":"2024-04-25T00:02:21.013026+0000","last_unstale":"2024-04-25T00:02:21.014422+0000","last_undegraded":"2024-04-25T00:02:21.014422+0000","last_fullsized":"2024-04-25T00:02:21.014422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:23.963484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002079+0000","last_change":"2024-04-25T00:02:21.002079+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002079+0000","last_undegraded":"2024-04-25T00:02:21.002079+0000","last_fullsized":"2024-04-25T00:02:21.002079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014442+0000","last_change":"2024-04-25T00:02:21.013431+0000","last_active":"2024-04-25T00:02:21.014442+0000","last_peered":"2024-04-25T00:02:21.014442+0000","last_clean":"2024-04-25T00:02:21.014442+0000","last_became_active":"2024-04-25T00:02:21.012907+0000","last_became_peered":"2024-04-25T00:02:21.012907+0000","last_unstale":"2024-04-25T00:02:21.014442+0000","last_undegraded":"2024-04-25T00:02:21.014442+0000","last_fullsized":"2024-04-25T00:02:21.014442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:43.668722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004081+0000","last_change":"2024-04-25T00:02:21.004081+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004081+0000","last_undegraded":"2024-04-25T00:02:21.004081+0000","last_fullsized":"2024-04-25T00:02:21.004081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013959+0000","last_change":"2024-04-25T00:02:21.008299+0000","last_active":"2024-04-25T00:02:21.013959+0000","last_peered":"2024-04-25T00:02:21.013959+0000","last_clean":"2024-04-25T00:02:21.013959+0000","last_became_active":"2024-04-25T00:02:21.008024+0000","last_became_peered":"2024-04-25T00:02:21.008024+0000","last_unstale":"2024-04-25T00:02:21.013959+0000","last_undegraded":"2024-04-25T00:02:21.013959+0000","last_fullsized":"2024-04-25T00:02:21.013959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:50.202267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001633+0000","last_change":"2024-04-25T00:02:21.001633+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001633+0000","last_undegraded":"2024-04-25T00:02:21.001633+0000","last_fullsized":"2024-04-25T00:02:21.001633+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014517+0000","last_change":"2024-04-25T00:02:21.011826+0000","last_active":"2024-04-25T00:02:21.014517+0000","last_peered":"2024-04-25T00:02:21.014517+0000","last_clean":"2024-04-25T00:02:21.014517+0000","last_became_active":"2024-04-25T00:02:21.011515+0000","last_became_peered":"2024-04-25T00:02:21.011515+0000","last_unstale":"2024-04-25T00:02:21.014517+0000","last_undegraded":"2024-04-25T00:02:21.014517+0000","last_fullsized":"2024-04-25T00:02:21.014517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:38.562371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004537+0000","last_change":"2024-04-25T00:02:21.004537+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004537+0000","last_undegraded":"2024-04-25T00:02:21.004537+0000","last_fullsized":"2024-04-25T00:02:21.004537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014276+0000","last_change":"2024-04-25T00:02:21.012403+0000","last_active":"2024-04-25T00:02:21.014276+0000","last_peered":"2024-04-25T00:02:21.014276+0000","last_clean":"2024-04-25T00:02:21.014276+0000","last_became_active":"2024-04-25T00:02:21.012150+0000","last_became_peered":"2024-04-25T00:02:21.012150+0000","last_unstale":"2024-04-25T00:02:21.014276+0000","last_undegraded":"2024-04-25T00:02:21.014276+0000","last_fullsized":"2024-04-25T00:02:21.014276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:19.301050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002303+0000","last_change":"2024-04-25T00:02:21.002303+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002303+0000","last_undegraded":"2024-04-25T00:02:21.002303+0000","last_fullsized":"2024-04-25T00:02:21.002303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012131+0000","last_change":"2024-04-25T00:02:21.009088+0000","last_active":"2024-04-25T00:02:21.012131+0000","last_peered":"2024-04-25T00:02:21.012131+0000","last_clean":"2024-04-25T00:02:21.012131+0000","last_became_active":"2024-04-25T00:02:21.008821+0000","last_became_peered":"2024-04-25T00:02:21.008821+0000","last_unstale":"2024-04-25T00:02:21.012131+0000","last_undegraded":"2024-04-25T00:02:21.012131+0000","last_fullsized":"2024-04-25T00:02:21.012131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002805+0000","last_change":"2024-04-25T00:02:21.002805+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002805+0000","last_undegraded":"2024-04-25T00:02:21.002805+0000","last_fullsized":"2024-04-25T00:02:21.002805+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015993+0000","last_change":"2024-04-25T00:02:21.015109+0000","last_active":"2024-04-25T00:02:21.015993+0000","last_peered":"2024-04-25T00:02:21.015993+0000","last_clean":"2024-04-25T00:02:21.015993+0000","last_became_active":"2024-04-25T00:02:21.014777+0000","last_became_peered":"2024-04-25T00:02:21.014777+0000","last_unstale":"2024-04-25T00:02:21.015993+0000","last_undegraded":"2024-04-25T00:02:21.015993+0000","last_fullsized":"2024-04-25T00:02:21.015993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997520+0000","last_change":"2024-04-25T00:02:20.997520+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997520+0000","last_undegraded":"2024-04-25T00:02:20.997520+0000","last_fullsized":"2024-04-25T00:02:20.997520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011475+0000","last_change":"2024-04-25T00:02:21.010049+0000","last_active":"2024-04-25T00:02:21.011475+0000","last_peered":"2024-04-25T00:02:21.011475+0000","last_clean":"2024-04-25T00:02:21.011475+0000","last_became_active":"2024-04-25T00:02:21.009715+0000","last_became_peered":"2024-04-25T00:02:21.009715+0000","last_unstale":"2024-04-25T00:02:21.011475+0000","last_undegraded":"2024-04-25T00:02:21.011475+0000","last_fullsized":"2024-04-25T00:02:21.011475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:57.913997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002070+0000","last_change":"2024-04-25T00:02:21.002070+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002070+0000","last_undegraded":"2024-04-25T00:02:21.002070+0000","last_fullsized":"2024-04-25T00:02:21.002070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014496+0000","last_change":"2024-04-25T00:02:21.010313+0000","last_active":"2024-04-25T00:02:21.014496+0000","last_peered":"2024-04-25T00:02:21.014496+0000","last_clean":"2024-04-25T00:02:21.014496+0000","last_became_active":"2024-04-25T00:02:21.010052+0000","last_became_peered":"2024-04-25T00:02:21.010052+0000","last_unstale":"2024-04-25T00:02:21.014496+0000","last_undegraded":"2024-04-25T00:02:21.014496+0000","last_fullsized":"2024-04-25T00:02:21.014496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:27.273733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997694+0000","last_change":"2024-04-25T00:02:20.997694+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997694+0000","last_undegraded":"2024-04-25T00:02:20.997694+0000","last_fullsized":"2024-04-25T00:02:20.997694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014955+0000","last_change":"2024-04-25T00:02:21.013942+0000","last_active":"2024-04-25T00:02:21.014955+0000","last_peered":"2024-04-25T00:02:21.014955+0000","last_clean":"2024-04-25T00:02:21.014955+0000","last_became_active":"2024-04-25T00:02:21.013529+0000","last_became_peered":"2024-04-25T00:02:21.013529+0000","last_unstale":"2024-04-25T00:02:21.014955+0000","last_undegraded":"2024-04-25T00:02:21.014955+0000","last_fullsized":"2024-04-25T00:02:21.014955+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:03.561219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003026+0000","last_change":"2024-04-25T00:02:21.003026+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003026+0000","last_undegraded":"2024-04-25T00:02:21.003026+0000","last_fullsized":"2024-04-25T00:02:21.003026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013509+0000","last_change":"2024-04-25T00:02:21.007387+0000","last_active":"2024-04-25T00:02:21.013509+0000","last_peered":"2024-04-25T00:02:21.013509+0000","last_clean":"2024-04-25T00:02:21.013509+0000","last_became_active":"2024-04-25T00:02:21.007015+0000","last_became_peered":"2024-04-25T00:02:21.007015+0000","last_unstale":"2024-04-25T00:02:21.013509+0000","last_undegraded":"2024-04-25T00:02:21.013509+0000","last_fullsized":"2024-04-25T00:02:21.013509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:45.498458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000822+0000","last_change":"2024-04-25T00:02:21.000822+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000822+0000","last_undegraded":"2024-04-25T00:02:21.000822+0000","last_fullsized":"2024-04-25T00:02:21.000822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015260+0000","last_change":"2024-04-25T00:02:21.014230+0000","last_active":"2024-04-25T00:02:21.015260+0000","last_peered":"2024-04-25T00:02:21.015260+0000","last_clean":"2024-04-25T00:02:21.015260+0000","last_became_active":"2024-04-25T00:02:21.013946+0000","last_became_peered":"2024-04-25T00:02:21.013946+0000","last_unstale":"2024-04-25T00:02:21.015260+0000","last_undegraded":"2024-04-25T00:02:21.015260+0000","last_fullsized":"2024-04-25T00:02:21.015260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:26.029288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002416+0000","last_change":"2024-04-25T00:02:21.002416+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002416+0000","last_undegraded":"2024-04-25T00:02:21.002416+0000","last_fullsized":"2024-04-25T00:02:21.002416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016174+0000","last_change":"2024-04-25T00:02:21.015322+0000","last_active":"2024-04-25T00:02:21.016174+0000","last_peered":"2024-04-25T00:02:21.016174+0000","last_clean":"2024-04-25T00:02:21.016174+0000","last_became_active":"2024-04-25T00:02:21.014852+0000","last_became_peered":"2024-04-25T00:02:21.014852+0000","last_unstale":"2024-04-25T00:02:21.016174+0000","last_undegraded":"2024-04-25T00:02:21.016174+0000","last_fullsized":"2024-04-25T00:02:21.016174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:56.062217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007261+0000","last_change":"2024-04-25T00:02:21.007261+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007261+0000","last_undegraded":"2024-04-25T00:02:21.007261+0000","last_fullsized":"2024-04-25T00:02:21.007261+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.009434+0000","last_change":"2024-04-25T00:02:21.007937+0000","last_active":"2024-04-25T00:02:21.009434+0000","last_peered":"2024-04-25T00:02:21.009434+0000","last_clean":"2024-04-25T00:02:21.009434+0000","last_became_active":"2024-04-25T00:02:21.007605+0000","last_became_peered":"2024-04-25T00:02:21.007605+0000","last_unstale":"2024-04-25T00:02:21.009434+0000","last_undegraded":"2024-04-25T00:02:21.009434+0000","last_fullsized":"2024-04-25T00:02:21.009434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:19.375987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004626+0000","last_change":"2024-04-25T00:02:21.004626+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004626+0000","last_undegraded":"2024-04-25T00:02:21.004626+0000","last_fullsized":"2024-04-25T00:02:21.004626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013944+0000","last_change":"2024-04-25T00:02:21.013272+0000","last_active":"2024-04-25T00:02:21.013944+0000","last_peered":"2024-04-25T00:02:21.013944+0000","last_clean":"2024-04-25T00:02:21.013944+0000","last_became_active":"2024-04-25T00:02:21.012935+0000","last_became_peered":"2024-04-25T00:02:21.012935+0000","last_unstale":"2024-04-25T00:02:21.013944+0000","last_undegraded":"2024-04-25T00:02:21.013944+0000","last_fullsized":"2024-04-25T00:02:21.013944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:49.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005277+0000","last_change":"2024-04-25T00:02:21.005277+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005277+0000","last_undegraded":"2024-04-25T00:02:21.005277+0000","last_fullsized":"2024-04-25T00:02:21.005277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011879+0000","last_change":"2024-04-25T00:02:21.010717+0000","last_active":"2024-04-25T00:02:21.011879+0000","last_peered":"2024-04-25T00:02:21.011879+0000","last_clean":"2024-04-25T00:02:21.011879+0000","last_became_active":"2024-04-25T00:02:21.010435+0000","last_became_peered":"2024-04-25T00:02:21.010435+0000","last_unstale":"2024-04-25T00:02:21.011879+0000","last_undegraded":"2024-04-25T00:02:21.011879+0000","last_fullsized":"2024-04-25T00:02:21.011879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:58.591352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997469+0000","last_change":"2024-04-25T00:02:20.997469+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997469+0000","last_undegraded":"2024-04-25T00:02:20.997469+0000","last_fullsized":"2024-04-25T00:02:20.997469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014893+0000","last_change":"2024-04-25T00:02:21.012474+0000","last_active":"2024-04-25T00:02:21.014893+0000","last_peered":"2024-04-25T00:02:21.014893+0000","last_clean":"2024-04-25T00:02:21.014893+0000","last_became_active":"2024-04-25T00:02:21.012207+0000","last_became_peered":"2024-04-25T00:02:21.012207+0000","last_unstale":"2024-04-25T00:02:21.014893+0000","last_undegraded":"2024-04-25T00:02:21.014893+0000","last_fullsized":"2024-04-25T00:02:21.014893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:29.100886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003155+0000","last_change":"2024-04-25T00:02:21.003155+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003155+0000","last_undegraded":"2024-04-25T00:02:21.003155+0000","last_fullsized":"2024-04-25T00:02:21.003155+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011553+0000","last_change":"2024-04-25T00:02:21.009993+0000","last_active":"2024-04-25T00:02:21.011553+0000","last_peered":"2024-04-25T00:02:21.011553+0000","last_clean":"2024-04-25T00:02:21.011553+0000","last_became_active":"2024-04-25T00:02:21.009614+0000","last_became_peered":"2024-04-25T00:02:21.009614+0000","last_unstale":"2024-04-25T00:02:21.011553+0000","last_undegraded":"2024-04-25T00:02:21.011553+0000","last_fullsized":"2024-04-25T00:02:21.011553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:34.685171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003163+0000","last_change":"2024-04-25T00:02:21.003163+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003163+0000","last_undegraded":"2024-04-25T00:02:21.003163+0000","last_fullsized":"2024-04-25T00:02:21.003163+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012000+0000","last_change":"2024-04-25T00:02:21.010823+0000","last_active":"2024-04-25T00:02:21.012000+0000","last_peered":"2024-04-25T00:02:21.012000+0000","last_clean":"2024-04-25T00:02:21.012000+0000","last_became_active":"2024-04-25T00:02:21.009880+0000","last_became_peered":"2024-04-25T00:02:21.009880+0000","last_unstale":"2024-04-25T00:02:21.012000+0000","last_undegraded":"2024-04-25T00:02:21.012000+0000","last_fullsized":"2024-04-25T00:02:21.012000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:41.022670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002260+0000","last_change":"2024-04-25T00:02:21.002260+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002260+0000","last_undegraded":"2024-04-25T00:02:21.002260+0000","last_fullsized":"2024-04-25T00:02:21.002260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.008835+0000","last_change":"2024-04-25T00:02:21.003729+0000","last_active":"2024-04-25T00:02:21.008835+0000","last_peered":"2024-04-25T00:02:21.008835+0000","last_clean":"2024-04-25T00:02:21.008835+0000","last_became_active":"2024-04-25T00:02:21.003453+0000","last_became_peered":"2024-04-25T00:02:21.003453+0000","last_unstale":"2024-04-25T00:02:21.008835+0000","last_undegraded":"2024-04-25T00:02:21.008835+0000","last_fullsized":"2024-04-25T00:02:21.008835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:58.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004996+0000","last_change":"2024-04-25T00:02:21.004996+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004996+0000","last_undegraded":"2024-04-25T00:02:21.004996+0000","last_fullsized":"2024-04-25T00:02:21.004996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012025+0000","last_change":"2024-04-25T00:02:21.010404+0000","last_active":"2024-04-25T00:02:21.012025+0000","last_peered":"2024-04-25T00:02:21.012025+0000","last_clean":"2024-04-25T00:02:21.012025+0000","last_became_active":"2024-04-25T00:02:21.010105+0000","last_became_peered":"2024-04-25T00:02:21.010105+0000","last_unstale":"2024-04-25T00:02:21.012025+0000","last_undegraded":"2024-04-25T00:02:21.012025+0000","last_fullsized":"2024-04-25T00:02:21.012025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:29.720152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003057+0000","last_change":"2024-04-25T00:02:21.003057+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003057+0000","last_undegraded":"2024-04-25T00:02:21.003057+0000","last_fullsized":"2024-04-25T00:02:21.003057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013880+0000","last_change":"2024-04-25T00:02:21.011175+0000","last_active":"2024-04-25T00:02:21.013880+0000","last_peered":"2024-04-25T00:02:21.013880+0000","last_clean":"2024-04-25T00:02:21.013880+0000","last_became_active":"2024-04-25T00:02:21.010864+0000","last_became_peered":"2024-04-25T00:02:21.010864+0000","last_unstale":"2024-04-25T00:02:21.013880+0000","last_undegraded":"2024-04-25T00:02:21.013880+0000","last_fullsized":"2024-04-25T00:02:21.013880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:34.531368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000723+0000","last_change":"2024-04-25T00:02:21.000723+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000723+0000","last_undegraded":"2024-04-25T00:02:21.000723+0000","last_fullsized":"2024-04-25T00:02:21.000723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014057+0000","last_change":"2024-04-25T00:02:21.009271+0000","last_active":"2024-04-25T00:02:21.014057+0000","last_peered":"2024-04-25T00:02:21.014057+0000","last_clean":"2024-04-25T00:02:21.014057+0000","last_became_active":"2024-04-25T00:02:21.009010+0000","last_became_peered":"2024-04-25T00:02:21.009010+0000","last_unstale":"2024-04-25T00:02:21.014057+0000","last_undegraded":"2024-04-25T00:02:21.014057+0000","last_fullsized":"2024-04-25T00:02:21.014057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:07.487563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003376+0000","last_change":"2024-04-25T00:02:21.003376+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003376+0000","last_undegraded":"2024-04-25T00:02:21.003376+0000","last_fullsized":"2024-04-25T00:02:21.003376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011285+0000","last_change":"2024-04-25T00:02:21.009814+0000","last_active":"2024-04-25T00:02:21.011285+0000","last_peered":"2024-04-25T00:02:21.011285+0000","last_clean":"2024-04-25T00:02:21.011285+0000","last_became_active":"2024-04-25T00:02:21.009197+0000","last_became_peered":"2024-04-25T00:02:21.009197+0000","last_unstale":"2024-04-25T00:02:21.011285+0000","last_undegraded":"2024-04-25T00:02:21.011285+0000","last_fullsized":"2024-04-25T00:02:21.011285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:19.681462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006708+0000","last_change":"2024-04-25T00:02:21.006708+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006708+0000","last_undegraded":"2024-04-25T00:02:21.006708+0000","last_fullsized":"2024-04-25T00:02:21.006708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013290+0000","last_change":"2024-04-25T00:02:21.011072+0000","last_active":"2024-04-25T00:02:21.013290+0000","last_peered":"2024-04-25T00:02:21.013290+0000","last_clean":"2024-04-25T00:02:21.013290+0000","last_became_active":"2024-04-25T00:02:21.010789+0000","last_became_peered":"2024-04-25T00:02:21.010789+0000","last_unstale":"2024-04-25T00:02:21.013290+0000","last_undegraded":"2024-04-25T00:02:21.013290+0000","last_fullsized":"2024-04-25T00:02:21.013290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:45.234560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.004389+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:02:21.004389+0000","last_peered":"2024-04-25T00:02:21.004389+0000","last_clean":"2024-04-25T00:02:21.004389+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:02:21.004389+0000","last_undegraded":"2024-04-25T00:02:21.004389+0000","last_fullsized":"2024-04-25T00:02:21.004389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":382102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:02:24.958 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:02:25.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.092+0000 7fdfa13dc640 1 Processor -- start 2024-04-25T00:02:25.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.092+0000 7fdfa13dc640 1 -- start start 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdfa13dc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdf9c15c280 0x7fdf9c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdfa13dc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c0cb210 con 0x7fdf9c062290 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdfa13dc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c0cbb30 con 0x7fdf9c09eaf0 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdfa13dc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c06b590 con 0x7fdf9c15c280 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdf9c062290 0x7fdf9c15fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58186/0 (socket says 172.21.15.113:58186) 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9b7fe640 1 -- 172.21.15.113:0/4020857602 learned_addr learned my addr 172.21.15.113:0/4020857602 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9affd640 1 --2- 172.21.15.113:0/4020857602 >> v2:172.21.15.178:3300/0 conn(0x7fdf9c15c280 0x7fdf9c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141924410 0 0) 0x7fdf9c0cb210 con 0x7fdf9c062290 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf88003520 con 0x7fdf9c062290 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 969838345 0 0) 0x7fdf88003520 con 0x7fdf9c062290 2024-04-25T00:02:25.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 >> v2:172.21.15.178:3300/0 conn(0x7fdf9c15c280 msgr2=0x7fdf9c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 --2- 172.21.15.113:0/4020857602 >> v2:172.21.15.178:3300/0 conn(0x7fdf9c15c280 0x7fdf9c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 >> v1:172.21.15.178:6789/0 conn(0x7fdf9c09eaf0 legacy=0x7fdf9c061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c0cbe10 con 0x7fdf9c062290 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fdf940019f0 con 0x7fdf9c062290 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 >> v1:172.21.15.113:6789/0 conn(0x7fdf9c062290 legacy=0x7fdf9c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf880039b0 0x7fdf88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf88004800 0x7fdf88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf880051c0 0x7fdf8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c06b590 con 0x7fdf880039b0 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.093+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8800c160 con 0x7fdf880051c0 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8800c360 con 0x7fdf88004800 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9affd640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf880039b0 0x7fdf88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9b7fe640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf880051c0 0x7fdf8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf927fc640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf88004800 0x7fdf88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9affd640 1 -- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf88004800 msgr2=0x7fdf88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9affd640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf88004800 0x7fdf88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9affd640 1 -- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf880051c0 msgr2=0x7fdf8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9affd640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf880051c0 0x7fdf8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9b7fe640 1 -- 172.21.15.113:0/4020857602 reap_dead start 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9affd640 1 -- 172.21.15.113:0/4020857602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf88003520 con 0x7fdf880039b0 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.094+0000 7fdf9affd640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf880039b0 0x7fdf88003dd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009ac0 tx=0x7fdf88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdb4d808e0ee3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c036070 con 0x7fdf880039b0 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c031400 con 0x7fdf880039b0 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdf9a7fc640 1 -- 172.21.15.113:0/4020857602 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c009690 con 0x7fdf880039b0 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdfa13dc640 1 -- 172.21.15.113:0/4020857602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf880039b0 msgr2=0x7fdf88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf880039b0 0x7fdf88003dd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009ac0 tx=0x7fdf88005830 comp rx=0 tx=0).stop 2024-04-25T00:02:25.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdfa13dc640 1 -- 172.21.15.113:0/4020857602 shutdown_connections 2024-04-25T00:02:25.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/4020857602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf880039b0 0x7fdf88003dd0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdfa13dc640 1 -- 172.21.15.113:0/4020857602 >> 172.21.15.113:0/4020857602 conn(0x7fdf9c0cd860 msgr2=0x7fdf9c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:25.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdfa13dc640 1 -- 172.21.15.113:0/4020857602 shutdown_connections 2024-04-25T00:02:25.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.095+0000 7fdfa13dc640 1 -- 172.21.15.113:0/4020857602 wait complete. 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.096+0000 7fdfa13dc640 1 Processor -- start 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdfa13dc640 1 -- start start 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdfa13dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf880039b0 0x7fdf9c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdfa13dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c147b80 0x7fdf9c1e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdfa13dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf9c155830 0x7fdf9c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdfa13dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c0cb430 con 0x7fdf9c147b80 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdfa13dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c0cc0f0 con 0x7fdf9c155830 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdfa13dc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c0cb5b0 con 0x7fdf880039b0 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdf927fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c147b80 0x7fdf9c1e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdf927fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c147b80 0x7fdf9c1e78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54502/0 (socket says 172.21.15.113:54502) 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdf927fc640 1 -- 172.21.15.113:0/100840914 learned_addr learned my addr 172.21.15.113:0/100840914 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.097+0000 7fdf9affd640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf880039b0 0x7fdf9c0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf9b7fe640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf9c155830 0x7fdf9c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf927fc640 1 -- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf880039b0 msgr2=0x7fdf9c0c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf927fc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf880039b0 0x7fdf9c0c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf927fc640 1 -- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf9c155830 msgr2=0x7fdf9c155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf927fc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf9c155830 0x7fdf9c155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf927fc640 1 -- 172.21.15.113:0/100840914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1e7e30 con 0x7fdf9c147b80 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf927fc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c147b80 0x7fdf9c1e78f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdf84007fd0 tx=0x7fdf8400e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf93fff640 1 -- 172.21.15.113:0/100840914 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84019070 con 0x7fdf9c147b80 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf93fff640 1 -- 172.21.15.113:0/100840914 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf840052e0 con 0x7fdf9c147b80 2024-04-25T00:02:25.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdf93fff640 1 -- 172.21.15.113:0/100840914 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84003810 con 0x7fdf9c147b80 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c0c8490 con 0x7fdf9c147b80 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.098+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c0c8a50 con 0x7fdf9c147b80 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.099+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c0691a0 con 0x7fdf9c147b80 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.099+0000 7fdf93fff640 1 -- 172.21.15.113:0/100840914 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf8401c070 con 0x7fdf9c147b80 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.100+0000 7fdf93fff640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.100+0000 7fdf93fff640 1 -- 172.21.15.113:0/100840914 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7fdf8408a410 con 0x7fdf9c147b80 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.100+0000 7fdf9affd640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.100+0000 7fdf9affd640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c033000 tx=0x7fdf8c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.104+0000 7fdf93fff640 1 -- 172.21.15.113:0/100840914 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf84058770 con 0x7fdf9c147b80 2024-04-25T00:02:25.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.274+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf9c0b0720 con 0x7fdf7c06be30 2024-04-25T00:02:25.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.280+0000 7fdf93fff640 1 -- 172.21.15.113:0/100840914 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7fdf9c0b0720 con 0x7fdf7c06be30 2024-04-25T00:02:25.283 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:25.284 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:02:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf7c06be30 msgr2=0x7fdf7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c033000 tx=0x7fdf8c008030 comp rx=0 tx=0).stop 2024-04-25T00:02:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c147b80 msgr2=0x7fdf9c1e78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c147b80 0x7fdf9c1e78f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdf84007fd0 tx=0x7fdf8400e8a0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 shutdown_connections 2024-04-25T00:02:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf7c06be30 0x7fdf7c06e2f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf9c155830 0x7fdf9c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c147b80 0x7fdf9c1e78f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 --2- 172.21.15.113:0/100840914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf880039b0 0x7fdf9c0c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 >> 172.21.15.113:0/100840914 conn(0x7fdf9c0cd860 msgr2=0x7fdf9c156c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:25.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 shutdown_connections 2024-04-25T00:02:25.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.283+0000 7fdfa13dc640 1 -- 172.21.15.113:0/100840914 wait complete. 2024-04-25T00:02:25.295 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-25T00:02:25.159984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274216,"kb_used_data":8116,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700504,"statfs":{"total":773094113280,"available":772813316096,"internally_reserved":0,"allocated":8310784,"data_stored":4193627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.172535"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004482+0000","last_change":"2024-04-25T00:02:21.004482+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004482+0000","last_undegraded":"2024-04-25T00:02:21.004482+0000","last_fullsized":"2024-04-25T00:02:21.004482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004023+0000","last_change":"2024-04-25T00:02:21.004023+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004023+0000","last_undegraded":"2024-04-25T00:02:21.004023+0000","last_fullsized":"2024-04-25T00:02:21.004023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006263+0000","last_change":"2024-04-25T00:02:21.006263+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006263+0000","last_undegraded":"2024-04-25T00:02:21.006263+0000","last_fullsized":"2024-04-25T00:02:21.006263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002424+0000","last_change":"2024-04-25T00:02:21.002424+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002424+0000","last_undegraded":"2024-04-25T00:02:21.002424+0000","last_fullsized":"2024-04-25T00:02:21.002424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002669+0000","last_change":"2024-04-25T00:02:21.002669+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002669+0000","last_undegraded":"2024-04-25T00:02:21.002669+0000","last_fullsized":"2024-04-25T00:02:21.002669+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001702+0000","last_change":"2024-04-25T00:02:21.001702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001702+0000","last_undegraded":"2024-04-25T00:02:21.001702+0000","last_fullsized":"2024-04-25T00:02:21.001702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002415+0000","last_change":"2024-04-25T00:02:21.002415+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002415+0000","last_undegraded":"2024-04-25T00:02:21.002415+0000","last_fullsized":"2024-04-25T00:02:21.002415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003841+0000","last_change":"2024-04-25T00:02:21.003841+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003841+0000","last_undegraded":"2024-04-25T00:02:21.003841+0000","last_fullsized":"2024-04-25T00:02:21.003841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004039+0000","last_change":"2024-04-25T00:02:21.004039+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004039+0000","last_undegraded":"2024-04-25T00:02:21.004039+0000","last_fullsized":"2024-04-25T00:02:21.004039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998227+0000","last_change":"2024-04-25T00:02:20.998227+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998227+0000","last_undegraded":"2024-04-25T00:02:20.998227+0000","last_fullsized":"2024-04-25T00:02:20.998227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003622+0000","last_change":"2024-04-25T00:02:21.003622+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003622+0000","last_undegraded":"2024-04-25T00:02:21.003622+0000","last_fullsized":"2024-04-25T00:02:21.003622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001910+0000","last_change":"2024-04-25T00:02:21.001910+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001910+0000","last_undegraded":"2024-04-25T00:02:21.001910+0000","last_fullsized":"2024-04-25T00:02:21.001910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997224+0000","last_change":"2024-04-25T00:02:20.997224+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997224+0000","last_undegraded":"2024-04-25T00:02:20.997224+0000","last_fullsized":"2024-04-25T00:02:20.997224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005347+0000","last_change":"2024-04-25T00:02:21.005347+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005347+0000","last_undegraded":"2024-04-25T00:02:21.005347+0000","last_fullsized":"2024-04-25T00:02:21.005347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997702+0000","last_change":"2024-04-25T00:02:20.997702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997702+0000","last_undegraded":"2024-04-25T00:02:20.997702+0000","last_fullsized":"2024-04-25T00:02:20.997702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997729+0000","last_change":"2024-04-25T00:02:20.997729+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997729+0000","last_undegraded":"2024-04-25T00:02:20.997729+0000","last_fullsized":"2024-04-25T00:02:20.997729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003670+0000","last_change":"2024-04-25T00:02:21.003670+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003670+0000","last_undegraded":"2024-04-25T00:02:21.003670+0000","last_fullsized":"2024-04-25T00:02:21.003670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000719+0000","last_change":"2024-04-25T00:02:21.000719+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000719+0000","last_undegraded":"2024-04-25T00:02:21.000719+0000","last_fullsized":"2024-04-25T00:02:21.000719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003140+0000","last_change":"2024-04-25T00:02:21.003140+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003140+0000","last_undegraded":"2024-04-25T00:02:21.003140+0000","last_fullsized":"2024-04-25T00:02:21.003140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997349+0000","last_change":"2024-04-25T00:02:20.997349+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997349+0000","last_undegraded":"2024-04-25T00:02:20.997349+0000","last_fullsized":"2024-04-25T00:02:20.997349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998024+0000","last_change":"2024-04-25T00:02:20.998024+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998024+0000","last_undegraded":"2024-04-25T00:02:20.998024+0000","last_fullsized":"2024-04-25T00:02:20.998024+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997322+0000","last_change":"2024-04-25T00:02:20.997322+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997322+0000","last_undegraded":"2024-04-25T00:02:20.997322+0000","last_fullsized":"2024-04-25T00:02:20.997322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002667+0000","last_change":"2024-04-25T00:02:21.002667+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002667+0000","last_undegraded":"2024-04-25T00:02:21.002667+0000","last_fullsized":"2024-04-25T00:02:21.002667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002165+0000","last_change":"2024-04-25T00:02:21.002165+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002165+0000","last_undegraded":"2024-04-25T00:02:21.002165+0000","last_fullsized":"2024-04-25T00:02:21.002165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.999243+0000","last_change":"2024-04-25T00:02:20.999243+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.999243+0000","last_undegraded":"2024-04-25T00:02:20.999243+0000","last_fullsized":"2024-04-25T00:02:20.999243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003685+0000","last_change":"2024-04-25T00:02:21.003685+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003685+0000","last_undegraded":"2024-04-25T00:02:21.003685+0000","last_fullsized":"2024-04-25T00:02:21.003685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004843+0000","last_change":"2024-04-25T00:02:21.004843+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004843+0000","last_undegraded":"2024-04-25T00:02:21.004843+0000","last_fullsized":"2024-04-25T00:02:21.004843+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998059+0000","last_change":"2024-04-25T00:02:20.998059+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998059+0000","last_undegraded":"2024-04-25T00:02:20.998059+0000","last_fullsized":"2024-04-25T00:02:20.998059+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007663+0000","last_change":"2024-04-25T00:02:21.007663+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007663+0000","last_undegraded":"2024-04-25T00:02:21.007663+0000","last_fullsized":"2024-04-25T00:02:21.007663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000548+0000","last_change":"2024-04-25T00:02:21.000548+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000548+0000","last_undegraded":"2024-04-25T00:02:21.000548+0000","last_fullsized":"2024-04-25T00:02:21.000548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001720+0000","last_change":"2024-04-25T00:02:21.001720+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001720+0000","last_undegraded":"2024-04-25T00:02:21.001720+0000","last_fullsized":"2024-04-25T00:02:21.001720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000890+0000","last_change":"2024-04-25T00:02:21.000890+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000890+0000","last_undegraded":"2024-04-25T00:02:21.000890+0000","last_fullsized":"2024-04-25T00:02:21.000890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004905+0000","last_change":"2024-04-25T00:02:21.004905+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004905+0000","last_undegraded":"2024-04-25T00:02:21.004905+0000","last_fullsized":"2024-04-25T00:02:21.004905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014554+0000","last_change":"2024-04-25T00:02:21.012893+0000","last_active":"2024-04-25T00:02:21.014554+0000","last_peered":"2024-04-25T00:02:21.014554+0000","last_clean":"2024-04-25T00:02:21.014554+0000","last_became_active":"2024-04-25T00:02:21.012601+0000","last_became_peered":"2024-04-25T00:02:21.012601+0000","last_unstale":"2024-04-25T00:02:21.014554+0000","last_undegraded":"2024-04-25T00:02:21.014554+0000","last_fullsized":"2024-04-25T00:02:21.014554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.876656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003284+0000","last_change":"2024-04-25T00:02:21.003284+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003284+0000","last_undegraded":"2024-04-25T00:02:21.003284+0000","last_fullsized":"2024-04-25T00:02:21.003284+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013760+0000","last_change":"2024-04-25T00:02:21.011558+0000","last_active":"2024-04-25T00:02:21.013760+0000","last_peered":"2024-04-25T00:02:21.013760+0000","last_clean":"2024-04-25T00:02:21.013760+0000","last_became_active":"2024-04-25T00:02:21.011287+0000","last_became_peered":"2024-04-25T00:02:21.011287+0000","last_unstale":"2024-04-25T00:02:21.013760+0000","last_undegraded":"2024-04-25T00:02:21.013760+0000","last_fullsized":"2024-04-25T00:02:21.013760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:20.889569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002102+0000","last_change":"2024-04-25T00:02:21.002102+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002102+0000","last_undegraded":"2024-04-25T00:02:21.002102+0000","last_fullsized":"2024-04-25T00:02:21.002102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013400+0000","last_change":"2024-04-25T00:02:21.007131+0000","last_active":"2024-04-25T00:02:21.013400+0000","last_peered":"2024-04-25T00:02:21.013400+0000","last_clean":"2024-04-25T00:02:21.013400+0000","last_became_active":"2024-04-25T00:02:21.006720+0000","last_became_peered":"2024-04-25T00:02:21.006720+0000","last_unstale":"2024-04-25T00:02:21.013400+0000","last_undegraded":"2024-04-25T00:02:21.013400+0000","last_fullsized":"2024-04-25T00:02:21.013400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:23.288653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002741+0000","last_change":"2024-04-25T00:02:21.002741+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002741+0000","last_undegraded":"2024-04-25T00:02:21.002741+0000","last_fullsized":"2024-04-25T00:02:21.002741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014923+0000","last_change":"2024-04-25T00:02:21.012185+0000","last_active":"2024-04-25T00:02:21.014923+0000","last_peered":"2024-04-25T00:02:21.014923+0000","last_clean":"2024-04-25T00:02:21.014923+0000","last_became_active":"2024-04-25T00:02:21.011910+0000","last_became_peered":"2024-04-25T00:02:21.011910+0000","last_unstale":"2024-04-25T00:02:21.014923+0000","last_undegraded":"2024-04-25T00:02:21.014923+0000","last_fullsized":"2024-04-25T00:02:21.014923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:24.418612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000075+0000","last_change":"2024-04-25T00:02:21.000075+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000075+0000","last_undegraded":"2024-04-25T00:02:21.000075+0000","last_fullsized":"2024-04-25T00:02:21.000075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016721+0000","last_change":"2024-04-25T00:02:21.015042+0000","last_active":"2024-04-25T00:02:21.016721+0000","last_peered":"2024-04-25T00:02:21.016721+0000","last_clean":"2024-04-25T00:02:21.016721+0000","last_became_active":"2024-04-25T00:02:21.014741+0000","last_became_peered":"2024-04-25T00:02:21.014741+0000","last_unstale":"2024-04-25T00:02:21.016721+0000","last_undegraded":"2024-04-25T00:02:21.016721+0000","last_fullsized":"2024-04-25T00:02:21.016721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:54.347862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997330+0000","last_change":"2024-04-25T00:02:20.997330+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997330+0000","last_undegraded":"2024-04-25T00:02:20.997330+0000","last_fullsized":"2024-04-25T00:02:20.997330+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.010476+0000","last_change":"2024-04-25T00:02:21.008726+0000","last_active":"2024-04-25T00:02:21.010476+0000","last_peered":"2024-04-25T00:02:21.010476+0000","last_clean":"2024-04-25T00:02:21.010476+0000","last_became_active":"2024-04-25T00:02:21.008442+0000","last_became_peered":"2024-04-25T00:02:21.008442+0000","last_unstale":"2024-04-25T00:02:21.010476+0000","last_undegraded":"2024-04-25T00:02:21.010476+0000","last_fullsized":"2024-04-25T00:02:21.010476+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:24.591088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005709+0000","last_change":"2024-04-25T00:02:21.005709+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005709+0000","last_undegraded":"2024-04-25T00:02:21.005709+0000","last_fullsized":"2024-04-25T00:02:21.005709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013907+0000","last_change":"2024-04-25T00:02:21.011239+0000","last_active":"2024-04-25T00:02:21.013907+0000","last_peered":"2024-04-25T00:02:21.013907+0000","last_clean":"2024-04-25T00:02:21.013907+0000","last_became_active":"2024-04-25T00:02:21.010824+0000","last_became_peered":"2024-04-25T00:02:21.010824+0000","last_unstale":"2024-04-25T00:02:21.013907+0000","last_undegraded":"2024-04-25T00:02:21.013907+0000","last_fullsized":"2024-04-25T00:02:21.013907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:24.542662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997196+0000","last_change":"2024-04-25T00:02:20.997196+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997196+0000","last_undegraded":"2024-04-25T00:02:20.997196+0000","last_fullsized":"2024-04-25T00:02:20.997196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014422+0000","last_change":"2024-04-25T00:02:21.013287+0000","last_active":"2024-04-25T00:02:21.014422+0000","last_peered":"2024-04-25T00:02:21.014422+0000","last_clean":"2024-04-25T00:02:21.014422+0000","last_became_active":"2024-04-25T00:02:21.013026+0000","last_became_peered":"2024-04-25T00:02:21.013026+0000","last_unstale":"2024-04-25T00:02:21.014422+0000","last_undegraded":"2024-04-25T00:02:21.014422+0000","last_fullsized":"2024-04-25T00:02:21.014422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:23.963484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002079+0000","last_change":"2024-04-25T00:02:21.002079+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002079+0000","last_undegraded":"2024-04-25T00:02:21.002079+0000","last_fullsized":"2024-04-25T00:02:21.002079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014442+0000","last_change":"2024-04-25T00:02:21.013431+0000","last_active":"2024-04-25T00:02:21.014442+0000","last_peered":"2024-04-25T00:02:21.014442+0000","last_clean":"2024-04-25T00:02:21.014442+0000","last_became_active":"2024-04-25T00:02:21.012907+0000","last_became_peered":"2024-04-25T00:02:21.012907+0000","last_unstale":"2024-04-25T00:02:21.014442+0000","last_undegraded":"2024-04-25T00:02:21.014442+0000","last_fullsized":"2024-04-25T00:02:21.014442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:43.668722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004081+0000","last_change":"2024-04-25T00:02:21.004081+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004081+0000","last_undegraded":"2024-04-25T00:02:21.004081+0000","last_fullsized":"2024-04-25T00:02:21.004081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013959+0000","last_change":"2024-04-25T00:02:21.008299+0000","last_active":"2024-04-25T00:02:21.013959+0000","last_peered":"2024-04-25T00:02:21.013959+0000","last_clean":"2024-04-25T00:02:21.013959+0000","last_became_active":"2024-04-25T00:02:21.008024+0000","last_became_peered":"2024-04-25T00:02:21.008024+0000","last_unstale":"2024-04-25T00:02:21.013959+0000","last_undegraded":"2024-04-25T00:02:21.013959+0000","last_fullsized":"2024-04-25T00:02:21.013959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:50.202267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001633+0000","last_change":"2024-04-25T00:02:21.001633+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001633+0000","last_undegraded":"2024-04-25T00:02:21.001633+0000","last_fullsized":"2024-04-25T00:02:21.001633+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014517+0000","last_change":"2024-04-25T00:02:21.011826+0000","last_active":"2024-04-25T00:02:21.014517+0000","last_peered":"2024-04-25T00:02:21.014517+0000","last_clean":"2024-04-25T00:02:21.014517+0000","last_became_active":"2024-04-25T00:02:21.011515+0000","last_became_peered":"2024-04-25T00:02:21.011515+0000","last_unstale":"2024-04-25T00:02:21.014517+0000","last_undegraded":"2024-04-25T00:02:21.014517+0000","last_fullsized":"2024-04-25T00:02:21.014517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:38.562371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004537+0000","last_change":"2024-04-25T00:02:21.004537+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004537+0000","last_undegraded":"2024-04-25T00:02:21.004537+0000","last_fullsized":"2024-04-25T00:02:21.004537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014276+0000","last_change":"2024-04-25T00:02:21.012403+0000","last_active":"2024-04-25T00:02:21.014276+0000","last_peered":"2024-04-25T00:02:21.014276+0000","last_clean":"2024-04-25T00:02:21.014276+0000","last_became_active":"2024-04-25T00:02:21.012150+0000","last_became_peered":"2024-04-25T00:02:21.012150+0000","last_unstale":"2024-04-25T00:02:21.014276+0000","last_undegraded":"2024-04-25T00:02:21.014276+0000","last_fullsized":"2024-04-25T00:02:21.014276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:19.301050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002303+0000","last_change":"2024-04-25T00:02:21.002303+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002303+0000","last_undegraded":"2024-04-25T00:02:21.002303+0000","last_fullsized":"2024-04-25T00:02:21.002303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012131+0000","last_change":"2024-04-25T00:02:21.009088+0000","last_active":"2024-04-25T00:02:21.012131+0000","last_peered":"2024-04-25T00:02:21.012131+0000","last_clean":"2024-04-25T00:02:21.012131+0000","last_became_active":"2024-04-25T00:02:21.008821+0000","last_became_peered":"2024-04-25T00:02:21.008821+0000","last_unstale":"2024-04-25T00:02:21.012131+0000","last_undegraded":"2024-04-25T00:02:21.012131+0000","last_fullsized":"2024-04-25T00:02:21.012131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002805+0000","last_change":"2024-04-25T00:02:21.002805+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002805+0000","last_undegraded":"2024-04-25T00:02:21.002805+0000","last_fullsized":"2024-04-25T00:02:21.002805+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015993+0000","last_change":"2024-04-25T00:02:21.015109+0000","last_active":"2024-04-25T00:02:21.015993+0000","last_peered":"2024-04-25T00:02:21.015993+0000","last_clean":"2024-04-25T00:02:21.015993+0000","last_became_active":"2024-04-25T00:02:21.014777+0000","last_became_peered":"2024-04-25T00:02:21.014777+0000","last_unstale":"2024-04-25T00:02:21.015993+0000","last_undegraded":"2024-04-25T00:02:21.015993+0000","last_fullsized":"2024-04-25T00:02:21.015993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997520+0000","last_change":"2024-04-25T00:02:20.997520+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997520+0000","last_undegraded":"2024-04-25T00:02:20.997520+0000","last_fullsized":"2024-04-25T00:02:20.997520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011475+0000","last_change":"2024-04-25T00:02:21.010049+0000","last_active":"2024-04-25T00:02:21.011475+0000","last_peered":"2024-04-25T00:02:21.011475+0000","last_clean":"2024-04-25T00:02:21.011475+0000","last_became_active":"2024-04-25T00:02:21.009715+0000","last_became_peered":"2024-04-25T00:02:21.009715+0000","last_unstale":"2024-04-25T00:02:21.011475+0000","last_undegraded":"2024-04-25T00:02:21.011475+0000","last_fullsized":"2024-04-25T00:02:21.011475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:57.913997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002070+0000","last_change":"2024-04-25T00:02:21.002070+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002070+0000","last_undegraded":"2024-04-25T00:02:21.002070+0000","last_fullsized":"2024-04-25T00:02:21.002070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014496+0000","last_change":"2024-04-25T00:02:21.010313+0000","last_active":"2024-04-25T00:02:21.014496+0000","last_peered":"2024-04-25T00:02:21.014496+0000","last_clean":"2024-04-25T00:02:21.014496+0000","last_became_active":"2024-04-25T00:02:21.010052+0000","last_became_peered":"2024-04-25T00:02:21.010052+0000","last_unstale":"2024-04-25T00:02:21.014496+0000","last_undegraded":"2024-04-25T00:02:21.014496+0000","last_fullsized":"2024-04-25T00:02:21.014496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:27.273733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997694+0000","last_change":"2024-04-25T00:02:20.997694+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997694+0000","last_undegraded":"2024-04-25T00:02:20.997694+0000","last_fullsized":"2024-04-25T00:02:20.997694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014955+0000","last_change":"2024-04-25T00:02:21.013942+0000","last_active":"2024-04-25T00:02:21.014955+0000","last_peered":"2024-04-25T00:02:21.014955+0000","last_clean":"2024-04-25T00:02:21.014955+0000","last_became_active":"2024-04-25T00:02:21.013529+0000","last_became_peered":"2024-04-25T00:02:21.013529+0000","last_unstale":"2024-04-25T00:02:21.014955+0000","last_undegraded":"2024-04-25T00:02:21.014955+0000","last_fullsized":"2024-04-25T00:02:21.014955+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:03.561219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003026+0000","last_change":"2024-04-25T00:02:21.003026+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003026+0000","last_undegraded":"2024-04-25T00:02:21.003026+0000","last_fullsized":"2024-04-25T00:02:21.003026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013509+0000","last_change":"2024-04-25T00:02:21.007387+0000","last_active":"2024-04-25T00:02:21.013509+0000","last_peered":"2024-04-25T00:02:21.013509+0000","last_clean":"2024-04-25T00:02:21.013509+0000","last_became_active":"2024-04-25T00:02:21.007015+0000","last_became_peered":"2024-04-25T00:02:21.007015+0000","last_unstale":"2024-04-25T00:02:21.013509+0000","last_undegraded":"2024-04-25T00:02:21.013509+0000","last_fullsized":"2024-04-25T00:02:21.013509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:45.498458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000822+0000","last_change":"2024-04-25T00:02:21.000822+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000822+0000","last_undegraded":"2024-04-25T00:02:21.000822+0000","last_fullsized":"2024-04-25T00:02:21.000822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015260+0000","last_change":"2024-04-25T00:02:21.014230+0000","last_active":"2024-04-25T00:02:21.015260+0000","last_peered":"2024-04-25T00:02:21.015260+0000","last_clean":"2024-04-25T00:02:21.015260+0000","last_became_active":"2024-04-25T00:02:21.013946+0000","last_became_peered":"2024-04-25T00:02:21.013946+0000","last_unstale":"2024-04-25T00:02:21.015260+0000","last_undegraded":"2024-04-25T00:02:21.015260+0000","last_fullsized":"2024-04-25T00:02:21.015260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:26.029288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002416+0000","last_change":"2024-04-25T00:02:21.002416+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002416+0000","last_undegraded":"2024-04-25T00:02:21.002416+0000","last_fullsized":"2024-04-25T00:02:21.002416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016174+0000","last_change":"2024-04-25T00:02:21.015322+0000","last_active":"2024-04-25T00:02:21.016174+0000","last_peered":"2024-04-25T00:02:21.016174+0000","last_clean":"2024-04-25T00:02:21.016174+0000","last_became_active":"2024-04-25T00:02:21.014852+0000","last_became_peered":"2024-04-25T00:02:21.014852+0000","last_unstale":"2024-04-25T00:02:21.016174+0000","last_undegraded":"2024-04-25T00:02:21.016174+0000","last_fullsized":"2024-04-25T00:02:21.016174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:56.062217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007261+0000","last_change":"2024-04-25T00:02:21.007261+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007261+0000","last_undegraded":"2024-04-25T00:02:21.007261+0000","last_fullsized":"2024-04-25T00:02:21.007261+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.009434+0000","last_change":"2024-04-25T00:02:21.007937+0000","last_active":"2024-04-25T00:02:21.009434+0000","last_peered":"2024-04-25T00:02:21.009434+0000","last_clean":"2024-04-25T00:02:21.009434+0000","last_became_active":"2024-04-25T00:02:21.007605+0000","last_became_peered":"2024-04-25T00:02:21.007605+0000","last_unstale":"2024-04-25T00:02:21.009434+0000","last_undegraded":"2024-04-25T00:02:21.009434+0000","last_fullsized":"2024-04-25T00:02:21.009434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:19.375987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004626+0000","last_change":"2024-04-25T00:02:21.004626+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004626+0000","last_undegraded":"2024-04-25T00:02:21.004626+0000","last_fullsized":"2024-04-25T00:02:21.004626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013944+0000","last_change":"2024-04-25T00:02:21.013272+0000","last_active":"2024-04-25T00:02:21.013944+0000","last_peered":"2024-04-25T00:02:21.013944+0000","last_clean":"2024-04-25T00:02:21.013944+0000","last_became_active":"2024-04-25T00:02:21.012935+0000","last_became_peered":"2024-04-25T00:02:21.012935+0000","last_unstale":"2024-04-25T00:02:21.013944+0000","last_undegraded":"2024-04-25T00:02:21.013944+0000","last_fullsized":"2024-04-25T00:02:21.013944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:49.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005277+0000","last_change":"2024-04-25T00:02:21.005277+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005277+0000","last_undegraded":"2024-04-25T00:02:21.005277+0000","last_fullsized":"2024-04-25T00:02:21.005277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011879+0000","last_change":"2024-04-25T00:02:21.010717+0000","last_active":"2024-04-25T00:02:21.011879+0000","last_peered":"2024-04-25T00:02:21.011879+0000","last_clean":"2024-04-25T00:02:21.011879+0000","last_became_active":"2024-04-25T00:02:21.010435+0000","last_became_peered":"2024-04-25T00:02:21.010435+0000","last_unstale":"2024-04-25T00:02:21.011879+0000","last_undegraded":"2024-04-25T00:02:21.011879+0000","last_fullsized":"2024-04-25T00:02:21.011879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:58.591352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997469+0000","last_change":"2024-04-25T00:02:20.997469+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997469+0000","last_undegraded":"2024-04-25T00:02:20.997469+0000","last_fullsized":"2024-04-25T00:02:20.997469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014893+0000","last_change":"2024-04-25T00:02:21.012474+0000","last_active":"2024-04-25T00:02:21.014893+0000","last_peered":"2024-04-25T00:02:21.014893+0000","last_clean":"2024-04-25T00:02:21.014893+0000","last_became_active":"2024-04-25T00:02:21.012207+0000","last_became_peered":"2024-04-25T00:02:21.012207+0000","last_unstale":"2024-04-25T00:02:21.014893+0000","last_undegraded":"2024-04-25T00:02:21.014893+0000","last_fullsized":"2024-04-25T00:02:21.014893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:29.100886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003155+0000","last_change":"2024-04-25T00:02:21.003155+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003155+0000","last_undegraded":"2024-04-25T00:02:21.003155+0000","last_fullsized":"2024-04-25T00:02:21.003155+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011553+0000","last_change":"2024-04-25T00:02:21.009993+0000","last_active":"2024-04-25T00:02:21.011553+0000","last_peered":"2024-04-25T00:02:21.011553+0000","last_clean":"2024-04-25T00:02:21.011553+0000","last_became_active":"2024-04-25T00:02:21.009614+0000","last_became_peered":"2024-04-25T00:02:21.009614+0000","last_unstale":"2024-04-25T00:02:21.011553+0000","last_undegraded":"2024-04-25T00:02:21.011553+0000","last_fullsized":"2024-04-25T00:02:21.011553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:34.685171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003163+0000","last_change":"2024-04-25T00:02:21.003163+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003163+0000","last_undegraded":"2024-04-25T00:02:21.003163+0000","last_fullsized":"2024-04-25T00:02:21.003163+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012000+0000","last_change":"2024-04-25T00:02:21.010823+0000","last_active":"2024-04-25T00:02:21.012000+0000","last_peered":"2024-04-25T00:02:21.012000+0000","last_clean":"2024-04-25T00:02:21.012000+0000","last_became_active":"2024-04-25T00:02:21.009880+0000","last_became_peered":"2024-04-25T00:02:21.009880+0000","last_unstale":"2024-04-25T00:02:21.012000+0000","last_undegraded":"2024-04-25T00:02:21.012000+0000","last_fullsized":"2024-04-25T00:02:21.012000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:41.022670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002260+0000","last_change":"2024-04-25T00:02:21.002260+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002260+0000","last_undegraded":"2024-04-25T00:02:21.002260+0000","last_fullsized":"2024-04-25T00:02:21.002260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.008835+0000","last_change":"2024-04-25T00:02:21.003729+0000","last_active":"2024-04-25T00:02:21.008835+0000","last_peered":"2024-04-25T00:02:21.008835+0000","last_clean":"2024-04-25T00:02:21.008835+0000","last_became_active":"2024-04-25T00:02:21.003453+0000","last_became_peered":"2024-04-25T00:02:21.003453+0000","last_unstale":"2024-04-25T00:02:21.008835+0000","last_undegraded":"2024-04-25T00:02:21.008835+0000","last_fullsized":"2024-04-25T00:02:21.008835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:58.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004996+0000","last_change":"2024-04-25T00:02:21.004996+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004996+0000","last_undegraded":"2024-04-25T00:02:21.004996+0000","last_fullsized":"2024-04-25T00:02:21.004996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012025+0000","last_change":"2024-04-25T00:02:21.010404+0000","last_active":"2024-04-25T00:02:21.012025+0000","last_peered":"2024-04-25T00:02:21.012025+0000","last_clean":"2024-04-25T00:02:21.012025+0000","last_became_active":"2024-04-25T00:02:21.010105+0000","last_became_peered":"2024-04-25T00:02:21.010105+0000","last_unstale":"2024-04-25T00:02:21.012025+0000","last_undegraded":"2024-04-25T00:02:21.012025+0000","last_fullsized":"2024-04-25T00:02:21.012025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:29.720152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003057+0000","last_change":"2024-04-25T00:02:21.003057+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003057+0000","last_undegraded":"2024-04-25T00:02:21.003057+0000","last_fullsized":"2024-04-25T00:02:21.003057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013880+0000","last_change":"2024-04-25T00:02:21.011175+0000","last_active":"2024-04-25T00:02:21.013880+0000","last_peered":"2024-04-25T00:02:21.013880+0000","last_clean":"2024-04-25T00:02:21.013880+0000","last_became_active":"2024-04-25T00:02:21.010864+0000","last_became_peered":"2024-04-25T00:02:21.010864+0000","last_unstale":"2024-04-25T00:02:21.013880+0000","last_undegraded":"2024-04-25T00:02:21.013880+0000","last_fullsized":"2024-04-25T00:02:21.013880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:34.531368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000723+0000","last_change":"2024-04-25T00:02:21.000723+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000723+0000","last_undegraded":"2024-04-25T00:02:21.000723+0000","last_fullsized":"2024-04-25T00:02:21.000723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014057+0000","last_change":"2024-04-25T00:02:21.009271+0000","last_active":"2024-04-25T00:02:21.014057+0000","last_peered":"2024-04-25T00:02:21.014057+0000","last_clean":"2024-04-25T00:02:21.014057+0000","last_became_active":"2024-04-25T00:02:21.009010+0000","last_became_peered":"2024-04-25T00:02:21.009010+0000","last_unstale":"2024-04-25T00:02:21.014057+0000","last_undegraded":"2024-04-25T00:02:21.014057+0000","last_fullsized":"2024-04-25T00:02:21.014057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:07.487563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003376+0000","last_change":"2024-04-25T00:02:21.003376+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003376+0000","last_undegraded":"2024-04-25T00:02:21.003376+0000","last_fullsized":"2024-04-25T00:02:21.003376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011285+0000","last_change":"2024-04-25T00:02:21.009814+0000","last_active":"2024-04-25T00:02:21.011285+0000","last_peered":"2024-04-25T00:02:21.011285+0000","last_clean":"2024-04-25T00:02:21.011285+0000","last_became_active":"2024-04-25T00:02:21.009197+0000","last_became_peered":"2024-04-25T00:02:21.009197+0000","last_unstale":"2024-04-25T00:02:21.011285+0000","last_undegraded":"2024-04-25T00:02:21.011285+0000","last_fullsized":"2024-04-25T00:02:21.011285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:19.681462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006708+0000","last_change":"2024-04-25T00:02:21.006708+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006708+0000","last_undegraded":"2024-04-25T00:02:21.006708+0000","last_fullsized":"2024-04-25T00:02:21.006708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013290+0000","last_change":"2024-04-25T00:02:21.011072+0000","last_active":"2024-04-25T00:02:21.013290+0000","last_peered":"2024-04-25T00:02:21.013290+0000","last_clean":"2024-04-25T00:02:21.013290+0000","last_became_active":"2024-04-25T00:02:21.010789+0000","last_became_peered":"2024-04-25T00:02:21.010789+0000","last_unstale":"2024-04-25T00:02:21.013290+0000","last_undegraded":"2024-04-25T00:02:21.013290+0000","last_fullsized":"2024-04-25T00:02:21.013290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:45.234560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.004389+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:02:21.004389+0000","last_peered":"2024-04-25T00:02:21.004389+0000","last_clean":"2024-04-25T00:02:21.004389+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:02:21.004389+0000","last_undegraded":"2024-04-25T00:02:21.004389+0000","last_fullsized":"2024-04-25T00:02:21.004389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":382102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:02:25.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:02:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.432+0000 7f406d91d640 1 Processor -- start 2024-04-25T00:02:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.432+0000 7f406d91d640 1 -- start start 2024-04-25T00:02:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f406d91d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f406815e320 0x7f40680d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f406d91d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40680d2290 0x7f40680b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f406d91d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40680d3de0 con 0x7f40680d0b00 2024-04-25T00:02:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f406d91d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f40680d3410 con 0x7f406815e320 2024-04-25T00:02:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f406d91d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f40680d2800 con 0x7f40680d2290 2024-04-25T00:02:25.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f40667fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40680d2290 0x7f40680b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f40667fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40680d2290 0x7f40680b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:51116/0 (socket says 172.21.15.113:51116) 2024-04-25T00:02:25.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f40667fc640 1 -- 172.21.15.113:0/4286577725 learned_addr learned my addr 172.21.15.113:0/4286577725 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:25.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.433+0000 7f4066ffd640 1 --2- 172.21.15.113:0/4286577725 >> v2:172.21.15.178:3300/0 conn(0x7f406815e320 0x7f40680d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.434+0000 7f40667fc640 1 -- 172.21.15.113:0/4286577725 >> v2:172.21.15.178:3300/0 conn(0x7f406815e320 msgr2=0x7f40680d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:25.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.434+0000 7f40667fc640 1 --2- 172.21.15.113:0/4286577725 >> v2:172.21.15.178:3300/0 conn(0x7f406815e320 0x7f40680d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.434+0000 7f40667fc640 1 -- 172.21.15.113:0/4286577725 >> v1:172.21.15.178:6789/0 conn(0x7f40680d0b00 legacy=0x7f40680b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.434+0000 7f40667fc640 1 -- 172.21.15.113:0/4286577725 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40680d2ae0 con 0x7f40680d2290 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.434+0000 7f40667fc640 1 --2- 172.21.15.113:0/4286577725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40680d2290 0x7f40680b9aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f405c009870 tx=0x7f405c02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e72e86eda49938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.434+0000 7f4065ffb640 1 -- 172.21.15.113:0/4286577725 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c037070 con 0x7f40680d2290 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f4065ffb640 1 -- 172.21.15.113:0/4286577725 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405c02ae50 con 0x7f40680d2290 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f4065ffb640 1 -- 172.21.15.113:0/4286577725 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c032860 con 0x7f40680d2290 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 -- 172.21.15.113:0/4286577725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40680d2290 msgr2=0x7f40680b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 --2- 172.21.15.113:0/4286577725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40680d2290 0x7f40680b9aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f405c009870 tx=0x7f405c02a300 comp rx=0 tx=0).stop 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 -- 172.21.15.113:0/4286577725 shutdown_connections 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 --2- 172.21.15.113:0/4286577725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40680d2290 0x7f40680b9aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 --2- 172.21.15.113:0/4286577725 >> v2:172.21.15.178:3300/0 conn(0x7f406815e320 0x7f40680d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 -- 172.21.15.113:0/4286577725 >> 172.21.15.113:0/4286577725 conn(0x7f40680b5c20 msgr2=0x7f40680b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 -- 172.21.15.113:0/4286577725 shutdown_connections 2024-04-25T00:02:25.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.435+0000 7f406d91d640 1 -- 172.21.15.113:0/4286577725 wait complete. 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.436+0000 7f406d91d640 1 Processor -- start 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.436+0000 7f406d91d640 1 -- start start 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f406d91d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40680d0b00 0x7f4068160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f406d91d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40680d2290 0x7f40681f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f406d91d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406815e320 0x7f40680ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f406d91d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f40680b9fe0 con 0x7f40680d2290 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f406d91d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40680d30a0 con 0x7f40680d0b00 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f406d91d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f406815e700 con 0x7f406815e320 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f40667fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40680d2290 0x7f40681f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f40667fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40680d2290 0x7f40681f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54506/0 (socket says 172.21.15.113:54506) 2024-04-25T00:02:25.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f40667fc640 1 -- 172.21.15.113:0/1982111060 learned_addr learned my addr 172.21.15.113:0/1982111060 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f4066ffd640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40680d0b00 0x7f4068160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.437+0000 7f40677fe640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406815e320 0x7f40680ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f40667fc640 1 -- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406815e320 msgr2=0x7f40680ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f40667fc640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406815e320 0x7f40680ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f40667fc640 1 -- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40680d0b00 msgr2=0x7f4068160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f40667fc640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40680d0b00 0x7f4068160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f40667fc640 1 -- 172.21.15.113:0/1982111060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40681f4ca0 con 0x7f40680d2290 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f4066ffd640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40680d0b00 0x7f4068160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f40677fe640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406815e320 0x7f40680ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f40667fc640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40680d2290 0x7f40681f4760 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f405c000c00 tx=0x7f405c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f404ffff640 1 -- 172.21.15.113:0/1982111060 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c041070 con 0x7f40680d2290 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f404ffff640 1 -- 172.21.15.113:0/1982111060 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405c03c660 con 0x7f40680d2290 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40680ab070 con 0x7f40680d2290 2024-04-25T00:02:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f404ffff640 1 -- 172.21.15.113:0/1982111060 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c037040 con 0x7f40680d2290 2024-04-25T00:02:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.438+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40680ab630 con 0x7f40680d2290 2024-04-25T00:02:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.439+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40680691a0 con 0x7f40680d2290 2024-04-25T00:02:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.439+0000 7f404ffff640 1 -- 172.21.15.113:0/1982111060 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f405c044070 con 0x7f40680d2290 2024-04-25T00:02:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.440+0000 7f404ffff640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f405406be30 0x7f405406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.440+0000 7f404ffff640 1 -- 172.21.15.113:0/1982111060 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f405c0ab670 con 0x7f40680d2290 2024-04-25T00:02:25.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.440+0000 7f4066ffd640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f405406be30 0x7f405406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.440+0000 7f4066ffd640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f405406be30 0x7f405406e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4058009af0 tx=0x7f4058008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.444+0000 7f404ffff640 1 -- 172.21.15.113:0/1982111060 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40681f4ca0 con 0x7f40680d2290 2024-04-25T00:02:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.613+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40680d4c50 con 0x7f405406be30 2024-04-25T00:02:25.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.619+0000 7f404ffff640 1 -- 172.21.15.113:0/1982111060 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7f40680d4c50 con 0x7f405406be30 2024-04-25T00:02:25.623 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:02:25.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f405406be30 msgr2=0x7f405406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f405406be30 0x7f405406e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4058009af0 tx=0x7f4058008040 comp rx=0 tx=0).stop 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40680d2290 msgr2=0x7f40681f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40680d2290 0x7f40681f4760 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f405c000c00 tx=0x7f405c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 shutdown_connections 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f405406be30 0x7f405406e2f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406815e320 0x7f40680ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40680d2290 0x7f40681f4760 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 --2- 172.21.15.113:0/1982111060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40680d0b00 0x7f4068160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 >> 172.21.15.113:0/1982111060 conn(0x7f40680b5c20 msgr2=0x7f40680b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:25.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 shutdown_connections 2024-04-25T00:02:25.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.622+0000 7f406d91d640 1 -- 172.21.15.113:0/1982111060 wait complete. 2024-04-25T00:02:25.633 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-25T00:02:25.159984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274216,"kb_used_data":8116,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700504,"statfs":{"total":773094113280,"available":772813316096,"internally_reserved":0,"allocated":8310784,"data_stored":4193627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.172535"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004482+0000","last_change":"2024-04-25T00:02:21.004482+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004482+0000","last_undegraded":"2024-04-25T00:02:21.004482+0000","last_fullsized":"2024-04-25T00:02:21.004482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004023+0000","last_change":"2024-04-25T00:02:21.004023+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004023+0000","last_undegraded":"2024-04-25T00:02:21.004023+0000","last_fullsized":"2024-04-25T00:02:21.004023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006263+0000","last_change":"2024-04-25T00:02:21.006263+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006263+0000","last_undegraded":"2024-04-25T00:02:21.006263+0000","last_fullsized":"2024-04-25T00:02:21.006263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002424+0000","last_change":"2024-04-25T00:02:21.002424+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002424+0000","last_undegraded":"2024-04-25T00:02:21.002424+0000","last_fullsized":"2024-04-25T00:02:21.002424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002669+0000","last_change":"2024-04-25T00:02:21.002669+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002669+0000","last_undegraded":"2024-04-25T00:02:21.002669+0000","last_fullsized":"2024-04-25T00:02:21.002669+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001702+0000","last_change":"2024-04-25T00:02:21.001702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001702+0000","last_undegraded":"2024-04-25T00:02:21.001702+0000","last_fullsized":"2024-04-25T00:02:21.001702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002415+0000","last_change":"2024-04-25T00:02:21.002415+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002415+0000","last_undegraded":"2024-04-25T00:02:21.002415+0000","last_fullsized":"2024-04-25T00:02:21.002415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003841+0000","last_change":"2024-04-25T00:02:21.003841+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003841+0000","last_undegraded":"2024-04-25T00:02:21.003841+0000","last_fullsized":"2024-04-25T00:02:21.003841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004039+0000","last_change":"2024-04-25T00:02:21.004039+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004039+0000","last_undegraded":"2024-04-25T00:02:21.004039+0000","last_fullsized":"2024-04-25T00:02:21.004039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998227+0000","last_change":"2024-04-25T00:02:20.998227+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998227+0000","last_undegraded":"2024-04-25T00:02:20.998227+0000","last_fullsized":"2024-04-25T00:02:20.998227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003622+0000","last_change":"2024-04-25T00:02:21.003622+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003622+0000","last_undegraded":"2024-04-25T00:02:21.003622+0000","last_fullsized":"2024-04-25T00:02:21.003622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001910+0000","last_change":"2024-04-25T00:02:21.001910+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001910+0000","last_undegraded":"2024-04-25T00:02:21.001910+0000","last_fullsized":"2024-04-25T00:02:21.001910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997224+0000","last_change":"2024-04-25T00:02:20.997224+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997224+0000","last_undegraded":"2024-04-25T00:02:20.997224+0000","last_fullsized":"2024-04-25T00:02:20.997224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005347+0000","last_change":"2024-04-25T00:02:21.005347+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005347+0000","last_undegraded":"2024-04-25T00:02:21.005347+0000","last_fullsized":"2024-04-25T00:02:21.005347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997702+0000","last_change":"2024-04-25T00:02:20.997702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997702+0000","last_undegraded":"2024-04-25T00:02:20.997702+0000","last_fullsized":"2024-04-25T00:02:20.997702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997729+0000","last_change":"2024-04-25T00:02:20.997729+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997729+0000","last_undegraded":"2024-04-25T00:02:20.997729+0000","last_fullsized":"2024-04-25T00:02:20.997729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003670+0000","last_change":"2024-04-25T00:02:21.003670+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003670+0000","last_undegraded":"2024-04-25T00:02:21.003670+0000","last_fullsized":"2024-04-25T00:02:21.003670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000719+0000","last_change":"2024-04-25T00:02:21.000719+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000719+0000","last_undegraded":"2024-04-25T00:02:21.000719+0000","last_fullsized":"2024-04-25T00:02:21.000719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003140+0000","last_change":"2024-04-25T00:02:21.003140+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003140+0000","last_undegraded":"2024-04-25T00:02:21.003140+0000","last_fullsized":"2024-04-25T00:02:21.003140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997349+0000","last_change":"2024-04-25T00:02:20.997349+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997349+0000","last_undegraded":"2024-04-25T00:02:20.997349+0000","last_fullsized":"2024-04-25T00:02:20.997349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998024+0000","last_change":"2024-04-25T00:02:20.998024+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998024+0000","last_undegraded":"2024-04-25T00:02:20.998024+0000","last_fullsized":"2024-04-25T00:02:20.998024+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997322+0000","last_change":"2024-04-25T00:02:20.997322+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997322+0000","last_undegraded":"2024-04-25T00:02:20.997322+0000","last_fullsized":"2024-04-25T00:02:20.997322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002667+0000","last_change":"2024-04-25T00:02:21.002667+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002667+0000","last_undegraded":"2024-04-25T00:02:21.002667+0000","last_fullsized":"2024-04-25T00:02:21.002667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002165+0000","last_change":"2024-04-25T00:02:21.002165+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002165+0000","last_undegraded":"2024-04-25T00:02:21.002165+0000","last_fullsized":"2024-04-25T00:02:21.002165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.999243+0000","last_change":"2024-04-25T00:02:20.999243+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.999243+0000","last_undegraded":"2024-04-25T00:02:20.999243+0000","last_fullsized":"2024-04-25T00:02:20.999243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003685+0000","last_change":"2024-04-25T00:02:21.003685+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003685+0000","last_undegraded":"2024-04-25T00:02:21.003685+0000","last_fullsized":"2024-04-25T00:02:21.003685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004843+0000","last_change":"2024-04-25T00:02:21.004843+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004843+0000","last_undegraded":"2024-04-25T00:02:21.004843+0000","last_fullsized":"2024-04-25T00:02:21.004843+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998059+0000","last_change":"2024-04-25T00:02:20.998059+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998059+0000","last_undegraded":"2024-04-25T00:02:20.998059+0000","last_fullsized":"2024-04-25T00:02:20.998059+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007663+0000","last_change":"2024-04-25T00:02:21.007663+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007663+0000","last_undegraded":"2024-04-25T00:02:21.007663+0000","last_fullsized":"2024-04-25T00:02:21.007663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000548+0000","last_change":"2024-04-25T00:02:21.000548+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000548+0000","last_undegraded":"2024-04-25T00:02:21.000548+0000","last_fullsized":"2024-04-25T00:02:21.000548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001720+0000","last_change":"2024-04-25T00:02:21.001720+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001720+0000","last_undegraded":"2024-04-25T00:02:21.001720+0000","last_fullsized":"2024-04-25T00:02:21.001720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000890+0000","last_change":"2024-04-25T00:02:21.000890+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000890+0000","last_undegraded":"2024-04-25T00:02:21.000890+0000","last_fullsized":"2024-04-25T00:02:21.000890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004905+0000","last_change":"2024-04-25T00:02:21.004905+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004905+0000","last_undegraded":"2024-04-25T00:02:21.004905+0000","last_fullsized":"2024-04-25T00:02:21.004905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014554+0000","last_change":"2024-04-25T00:02:21.012893+0000","last_active":"2024-04-25T00:02:21.014554+0000","last_peered":"2024-04-25T00:02:21.014554+0000","last_clean":"2024-04-25T00:02:21.014554+0000","last_became_active":"2024-04-25T00:02:21.012601+0000","last_became_peered":"2024-04-25T00:02:21.012601+0000","last_unstale":"2024-04-25T00:02:21.014554+0000","last_undegraded":"2024-04-25T00:02:21.014554+0000","last_fullsized":"2024-04-25T00:02:21.014554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.876656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003284+0000","last_change":"2024-04-25T00:02:21.003284+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003284+0000","last_undegraded":"2024-04-25T00:02:21.003284+0000","last_fullsized":"2024-04-25T00:02:21.003284+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013760+0000","last_change":"2024-04-25T00:02:21.011558+0000","last_active":"2024-04-25T00:02:21.013760+0000","last_peered":"2024-04-25T00:02:21.013760+0000","last_clean":"2024-04-25T00:02:21.013760+0000","last_became_active":"2024-04-25T00:02:21.011287+0000","last_became_peered":"2024-04-25T00:02:21.011287+0000","last_unstale":"2024-04-25T00:02:21.013760+0000","last_undegraded":"2024-04-25T00:02:21.013760+0000","last_fullsized":"2024-04-25T00:02:21.013760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:20.889569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002102+0000","last_change":"2024-04-25T00:02:21.002102+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002102+0000","last_undegraded":"2024-04-25T00:02:21.002102+0000","last_fullsized":"2024-04-25T00:02:21.002102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013400+0000","last_change":"2024-04-25T00:02:21.007131+0000","last_active":"2024-04-25T00:02:21.013400+0000","last_peered":"2024-04-25T00:02:21.013400+0000","last_clean":"2024-04-25T00:02:21.013400+0000","last_became_active":"2024-04-25T00:02:21.006720+0000","last_became_peered":"2024-04-25T00:02:21.006720+0000","last_unstale":"2024-04-25T00:02:21.013400+0000","last_undegraded":"2024-04-25T00:02:21.013400+0000","last_fullsized":"2024-04-25T00:02:21.013400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:23.288653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002741+0000","last_change":"2024-04-25T00:02:21.002741+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002741+0000","last_undegraded":"2024-04-25T00:02:21.002741+0000","last_fullsized":"2024-04-25T00:02:21.002741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014923+0000","last_change":"2024-04-25T00:02:21.012185+0000","last_active":"2024-04-25T00:02:21.014923+0000","last_peered":"2024-04-25T00:02:21.014923+0000","last_clean":"2024-04-25T00:02:21.014923+0000","last_became_active":"2024-04-25T00:02:21.011910+0000","last_became_peered":"2024-04-25T00:02:21.011910+0000","last_unstale":"2024-04-25T00:02:21.014923+0000","last_undegraded":"2024-04-25T00:02:21.014923+0000","last_fullsized":"2024-04-25T00:02:21.014923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:24.418612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000075+0000","last_change":"2024-04-25T00:02:21.000075+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000075+0000","last_undegraded":"2024-04-25T00:02:21.000075+0000","last_fullsized":"2024-04-25T00:02:21.000075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016721+0000","last_change":"2024-04-25T00:02:21.015042+0000","last_active":"2024-04-25T00:02:21.016721+0000","last_peered":"2024-04-25T00:02:21.016721+0000","last_clean":"2024-04-25T00:02:21.016721+0000","last_became_active":"2024-04-25T00:02:21.014741+0000","last_became_peered":"2024-04-25T00:02:21.014741+0000","last_unstale":"2024-04-25T00:02:21.016721+0000","last_undegraded":"2024-04-25T00:02:21.016721+0000","last_fullsized":"2024-04-25T00:02:21.016721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:54.347862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997330+0000","last_change":"2024-04-25T00:02:20.997330+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997330+0000","last_undegraded":"2024-04-25T00:02:20.997330+0000","last_fullsized":"2024-04-25T00:02:20.997330+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.010476+0000","last_change":"2024-04-25T00:02:21.008726+0000","last_active":"2024-04-25T00:02:21.010476+0000","last_peered":"2024-04-25T00:02:21.010476+0000","last_clean":"2024-04-25T00:02:21.010476+0000","last_became_active":"2024-04-25T00:02:21.008442+0000","last_became_peered":"2024-04-25T00:02:21.008442+0000","last_unstale":"2024-04-25T00:02:21.010476+0000","last_undegraded":"2024-04-25T00:02:21.010476+0000","last_fullsized":"2024-04-25T00:02:21.010476+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:24.591088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005709+0000","last_change":"2024-04-25T00:02:21.005709+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005709+0000","last_undegraded":"2024-04-25T00:02:21.005709+0000","last_fullsized":"2024-04-25T00:02:21.005709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013907+0000","last_change":"2024-04-25T00:02:21.011239+0000","last_active":"2024-04-25T00:02:21.013907+0000","last_peered":"2024-04-25T00:02:21.013907+0000","last_clean":"2024-04-25T00:02:21.013907+0000","last_became_active":"2024-04-25T00:02:21.010824+0000","last_became_peered":"2024-04-25T00:02:21.010824+0000","last_unstale":"2024-04-25T00:02:21.013907+0000","last_undegraded":"2024-04-25T00:02:21.013907+0000","last_fullsized":"2024-04-25T00:02:21.013907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:24.542662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997196+0000","last_change":"2024-04-25T00:02:20.997196+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997196+0000","last_undegraded":"2024-04-25T00:02:20.997196+0000","last_fullsized":"2024-04-25T00:02:20.997196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014422+0000","last_change":"2024-04-25T00:02:21.013287+0000","last_active":"2024-04-25T00:02:21.014422+0000","last_peered":"2024-04-25T00:02:21.014422+0000","last_clean":"2024-04-25T00:02:21.014422+0000","last_became_active":"2024-04-25T00:02:21.013026+0000","last_became_peered":"2024-04-25T00:02:21.013026+0000","last_unstale":"2024-04-25T00:02:21.014422+0000","last_undegraded":"2024-04-25T00:02:21.014422+0000","last_fullsized":"2024-04-25T00:02:21.014422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:23.963484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002079+0000","last_change":"2024-04-25T00:02:21.002079+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002079+0000","last_undegraded":"2024-04-25T00:02:21.002079+0000","last_fullsized":"2024-04-25T00:02:21.002079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014442+0000","last_change":"2024-04-25T00:02:21.013431+0000","last_active":"2024-04-25T00:02:21.014442+0000","last_peered":"2024-04-25T00:02:21.014442+0000","last_clean":"2024-04-25T00:02:21.014442+0000","last_became_active":"2024-04-25T00:02:21.012907+0000","last_became_peered":"2024-04-25T00:02:21.012907+0000","last_unstale":"2024-04-25T00:02:21.014442+0000","last_undegraded":"2024-04-25T00:02:21.014442+0000","last_fullsized":"2024-04-25T00:02:21.014442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:43.668722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004081+0000","last_change":"2024-04-25T00:02:21.004081+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004081+0000","last_undegraded":"2024-04-25T00:02:21.004081+0000","last_fullsized":"2024-04-25T00:02:21.004081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013959+0000","last_change":"2024-04-25T00:02:21.008299+0000","last_active":"2024-04-25T00:02:21.013959+0000","last_peered":"2024-04-25T00:02:21.013959+0000","last_clean":"2024-04-25T00:02:21.013959+0000","last_became_active":"2024-04-25T00:02:21.008024+0000","last_became_peered":"2024-04-25T00:02:21.008024+0000","last_unstale":"2024-04-25T00:02:21.013959+0000","last_undegraded":"2024-04-25T00:02:21.013959+0000","last_fullsized":"2024-04-25T00:02:21.013959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:50.202267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001633+0000","last_change":"2024-04-25T00:02:21.001633+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001633+0000","last_undegraded":"2024-04-25T00:02:21.001633+0000","last_fullsized":"2024-04-25T00:02:21.001633+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014517+0000","last_change":"2024-04-25T00:02:21.011826+0000","last_active":"2024-04-25T00:02:21.014517+0000","last_peered":"2024-04-25T00:02:21.014517+0000","last_clean":"2024-04-25T00:02:21.014517+0000","last_became_active":"2024-04-25T00:02:21.011515+0000","last_became_peered":"2024-04-25T00:02:21.011515+0000","last_unstale":"2024-04-25T00:02:21.014517+0000","last_undegraded":"2024-04-25T00:02:21.014517+0000","last_fullsized":"2024-04-25T00:02:21.014517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:38.562371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004537+0000","last_change":"2024-04-25T00:02:21.004537+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004537+0000","last_undegraded":"2024-04-25T00:02:21.004537+0000","last_fullsized":"2024-04-25T00:02:21.004537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014276+0000","last_change":"2024-04-25T00:02:21.012403+0000","last_active":"2024-04-25T00:02:21.014276+0000","last_peered":"2024-04-25T00:02:21.014276+0000","last_clean":"2024-04-25T00:02:21.014276+0000","last_became_active":"2024-04-25T00:02:21.012150+0000","last_became_peered":"2024-04-25T00:02:21.012150+0000","last_unstale":"2024-04-25T00:02:21.014276+0000","last_undegraded":"2024-04-25T00:02:21.014276+0000","last_fullsized":"2024-04-25T00:02:21.014276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:19.301050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002303+0000","last_change":"2024-04-25T00:02:21.002303+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002303+0000","last_undegraded":"2024-04-25T00:02:21.002303+0000","last_fullsized":"2024-04-25T00:02:21.002303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012131+0000","last_change":"2024-04-25T00:02:21.009088+0000","last_active":"2024-04-25T00:02:21.012131+0000","last_peered":"2024-04-25T00:02:21.012131+0000","last_clean":"2024-04-25T00:02:21.012131+0000","last_became_active":"2024-04-25T00:02:21.008821+0000","last_became_peered":"2024-04-25T00:02:21.008821+0000","last_unstale":"2024-04-25T00:02:21.012131+0000","last_undegraded":"2024-04-25T00:02:21.012131+0000","last_fullsized":"2024-04-25T00:02:21.012131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002805+0000","last_change":"2024-04-25T00:02:21.002805+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002805+0000","last_undegraded":"2024-04-25T00:02:21.002805+0000","last_fullsized":"2024-04-25T00:02:21.002805+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015993+0000","last_change":"2024-04-25T00:02:21.015109+0000","last_active":"2024-04-25T00:02:21.015993+0000","last_peered":"2024-04-25T00:02:21.015993+0000","last_clean":"2024-04-25T00:02:21.015993+0000","last_became_active":"2024-04-25T00:02:21.014777+0000","last_became_peered":"2024-04-25T00:02:21.014777+0000","last_unstale":"2024-04-25T00:02:21.015993+0000","last_undegraded":"2024-04-25T00:02:21.015993+0000","last_fullsized":"2024-04-25T00:02:21.015993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997520+0000","last_change":"2024-04-25T00:02:20.997520+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997520+0000","last_undegraded":"2024-04-25T00:02:20.997520+0000","last_fullsized":"2024-04-25T00:02:20.997520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011475+0000","last_change":"2024-04-25T00:02:21.010049+0000","last_active":"2024-04-25T00:02:21.011475+0000","last_peered":"2024-04-25T00:02:21.011475+0000","last_clean":"2024-04-25T00:02:21.011475+0000","last_became_active":"2024-04-25T00:02:21.009715+0000","last_became_peered":"2024-04-25T00:02:21.009715+0000","last_unstale":"2024-04-25T00:02:21.011475+0000","last_undegraded":"2024-04-25T00:02:21.011475+0000","last_fullsized":"2024-04-25T00:02:21.011475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:57.913997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002070+0000","last_change":"2024-04-25T00:02:21.002070+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002070+0000","last_undegraded":"2024-04-25T00:02:21.002070+0000","last_fullsized":"2024-04-25T00:02:21.002070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014496+0000","last_change":"2024-04-25T00:02:21.010313+0000","last_active":"2024-04-25T00:02:21.014496+0000","last_peered":"2024-04-25T00:02:21.014496+0000","last_clean":"2024-04-25T00:02:21.014496+0000","last_became_active":"2024-04-25T00:02:21.010052+0000","last_became_peered":"2024-04-25T00:02:21.010052+0000","last_unstale":"2024-04-25T00:02:21.014496+0000","last_undegraded":"2024-04-25T00:02:21.014496+0000","last_fullsized":"2024-04-25T00:02:21.014496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:27.273733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997694+0000","last_change":"2024-04-25T00:02:20.997694+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997694+0000","last_undegraded":"2024-04-25T00:02:20.997694+0000","last_fullsized":"2024-04-25T00:02:20.997694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014955+0000","last_change":"2024-04-25T00:02:21.013942+0000","last_active":"2024-04-25T00:02:21.014955+0000","last_peered":"2024-04-25T00:02:21.014955+0000","last_clean":"2024-04-25T00:02:21.014955+0000","last_became_active":"2024-04-25T00:02:21.013529+0000","last_became_peered":"2024-04-25T00:02:21.013529+0000","last_unstale":"2024-04-25T00:02:21.014955+0000","last_undegraded":"2024-04-25T00:02:21.014955+0000","last_fullsized":"2024-04-25T00:02:21.014955+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:03.561219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003026+0000","last_change":"2024-04-25T00:02:21.003026+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003026+0000","last_undegraded":"2024-04-25T00:02:21.003026+0000","last_fullsized":"2024-04-25T00:02:21.003026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013509+0000","last_change":"2024-04-25T00:02:21.007387+0000","last_active":"2024-04-25T00:02:21.013509+0000","last_peered":"2024-04-25T00:02:21.013509+0000","last_clean":"2024-04-25T00:02:21.013509+0000","last_became_active":"2024-04-25T00:02:21.007015+0000","last_became_peered":"2024-04-25T00:02:21.007015+0000","last_unstale":"2024-04-25T00:02:21.013509+0000","last_undegraded":"2024-04-25T00:02:21.013509+0000","last_fullsized":"2024-04-25T00:02:21.013509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:45.498458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000822+0000","last_change":"2024-04-25T00:02:21.000822+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000822+0000","last_undegraded":"2024-04-25T00:02:21.000822+0000","last_fullsized":"2024-04-25T00:02:21.000822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015260+0000","last_change":"2024-04-25T00:02:21.014230+0000","last_active":"2024-04-25T00:02:21.015260+0000","last_peered":"2024-04-25T00:02:21.015260+0000","last_clean":"2024-04-25T00:02:21.015260+0000","last_became_active":"2024-04-25T00:02:21.013946+0000","last_became_peered":"2024-04-25T00:02:21.013946+0000","last_unstale":"2024-04-25T00:02:21.015260+0000","last_undegraded":"2024-04-25T00:02:21.015260+0000","last_fullsized":"2024-04-25T00:02:21.015260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:26.029288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002416+0000","last_change":"2024-04-25T00:02:21.002416+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002416+0000","last_undegraded":"2024-04-25T00:02:21.002416+0000","last_fullsized":"2024-04-25T00:02:21.002416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016174+0000","last_change":"2024-04-25T00:02:21.015322+0000","last_active":"2024-04-25T00:02:21.016174+0000","last_peered":"2024-04-25T00:02:21.016174+0000","last_clean":"2024-04-25T00:02:21.016174+0000","last_became_active":"2024-04-25T00:02:21.014852+0000","last_became_peered":"2024-04-25T00:02:21.014852+0000","last_unstale":"2024-04-25T00:02:21.016174+0000","last_undegraded":"2024-04-25T00:02:21.016174+0000","last_fullsized":"2024-04-25T00:02:21.016174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:56.062217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007261+0000","last_change":"2024-04-25T00:02:21.007261+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007261+0000","last_undegraded":"2024-04-25T00:02:21.007261+0000","last_fullsized":"2024-04-25T00:02:21.007261+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.009434+0000","last_change":"2024-04-25T00:02:21.007937+0000","last_active":"2024-04-25T00:02:21.009434+0000","last_peered":"2024-04-25T00:02:21.009434+0000","last_clean":"2024-04-25T00:02:21.009434+0000","last_became_active":"2024-04-25T00:02:21.007605+0000","last_became_peered":"2024-04-25T00:02:21.007605+0000","last_unstale":"2024-04-25T00:02:21.009434+0000","last_undegraded":"2024-04-25T00:02:21.009434+0000","last_fullsized":"2024-04-25T00:02:21.009434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:19.375987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004626+0000","last_change":"2024-04-25T00:02:21.004626+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004626+0000","last_undegraded":"2024-04-25T00:02:21.004626+0000","last_fullsized":"2024-04-25T00:02:21.004626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013944+0000","last_change":"2024-04-25T00:02:21.013272+0000","last_active":"2024-04-25T00:02:21.013944+0000","last_peered":"2024-04-25T00:02:21.013944+0000","last_clean":"2024-04-25T00:02:21.013944+0000","last_became_active":"2024-04-25T00:02:21.012935+0000","last_became_peered":"2024-04-25T00:02:21.012935+0000","last_unstale":"2024-04-25T00:02:21.013944+0000","last_undegraded":"2024-04-25T00:02:21.013944+0000","last_fullsized":"2024-04-25T00:02:21.013944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:49.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005277+0000","last_change":"2024-04-25T00:02:21.005277+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005277+0000","last_undegraded":"2024-04-25T00:02:21.005277+0000","last_fullsized":"2024-04-25T00:02:21.005277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011879+0000","last_change":"2024-04-25T00:02:21.010717+0000","last_active":"2024-04-25T00:02:21.011879+0000","last_peered":"2024-04-25T00:02:21.011879+0000","last_clean":"2024-04-25T00:02:21.011879+0000","last_became_active":"2024-04-25T00:02:21.010435+0000","last_became_peered":"2024-04-25T00:02:21.010435+0000","last_unstale":"2024-04-25T00:02:21.011879+0000","last_undegraded":"2024-04-25T00:02:21.011879+0000","last_fullsized":"2024-04-25T00:02:21.011879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:58.591352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997469+0000","last_change":"2024-04-25T00:02:20.997469+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997469+0000","last_undegraded":"2024-04-25T00:02:20.997469+0000","last_fullsized":"2024-04-25T00:02:20.997469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014893+0000","last_change":"2024-04-25T00:02:21.012474+0000","last_active":"2024-04-25T00:02:21.014893+0000","last_peered":"2024-04-25T00:02:21.014893+0000","last_clean":"2024-04-25T00:02:21.014893+0000","last_became_active":"2024-04-25T00:02:21.012207+0000","last_became_peered":"2024-04-25T00:02:21.012207+0000","last_unstale":"2024-04-25T00:02:21.014893+0000","last_undegraded":"2024-04-25T00:02:21.014893+0000","last_fullsized":"2024-04-25T00:02:21.014893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:29.100886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003155+0000","last_change":"2024-04-25T00:02:21.003155+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003155+0000","last_undegraded":"2024-04-25T00:02:21.003155+0000","last_fullsized":"2024-04-25T00:02:21.003155+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011553+0000","last_change":"2024-04-25T00:02:21.009993+0000","last_active":"2024-04-25T00:02:21.011553+0000","last_peered":"2024-04-25T00:02:21.011553+0000","last_clean":"2024-04-25T00:02:21.011553+0000","last_became_active":"2024-04-25T00:02:21.009614+0000","last_became_peered":"2024-04-25T00:02:21.009614+0000","last_unstale":"2024-04-25T00:02:21.011553+0000","last_undegraded":"2024-04-25T00:02:21.011553+0000","last_fullsized":"2024-04-25T00:02:21.011553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:34.685171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003163+0000","last_change":"2024-04-25T00:02:21.003163+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003163+0000","last_undegraded":"2024-04-25T00:02:21.003163+0000","last_fullsized":"2024-04-25T00:02:21.003163+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012000+0000","last_change":"2024-04-25T00:02:21.010823+0000","last_active":"2024-04-25T00:02:21.012000+0000","last_peered":"2024-04-25T00:02:21.012000+0000","last_clean":"2024-04-25T00:02:21.012000+0000","last_became_active":"2024-04-25T00:02:21.009880+0000","last_became_peered":"2024-04-25T00:02:21.009880+0000","last_unstale":"2024-04-25T00:02:21.012000+0000","last_undegraded":"2024-04-25T00:02:21.012000+0000","last_fullsized":"2024-04-25T00:02:21.012000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:41.022670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002260+0000","last_change":"2024-04-25T00:02:21.002260+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002260+0000","last_undegraded":"2024-04-25T00:02:21.002260+0000","last_fullsized":"2024-04-25T00:02:21.002260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.008835+0000","last_change":"2024-04-25T00:02:21.003729+0000","last_active":"2024-04-25T00:02:21.008835+0000","last_peered":"2024-04-25T00:02:21.008835+0000","last_clean":"2024-04-25T00:02:21.008835+0000","last_became_active":"2024-04-25T00:02:21.003453+0000","last_became_peered":"2024-04-25T00:02:21.003453+0000","last_unstale":"2024-04-25T00:02:21.008835+0000","last_undegraded":"2024-04-25T00:02:21.008835+0000","last_fullsized":"2024-04-25T00:02:21.008835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:58.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004996+0000","last_change":"2024-04-25T00:02:21.004996+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004996+0000","last_undegraded":"2024-04-25T00:02:21.004996+0000","last_fullsized":"2024-04-25T00:02:21.004996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012025+0000","last_change":"2024-04-25T00:02:21.010404+0000","last_active":"2024-04-25T00:02:21.012025+0000","last_peered":"2024-04-25T00:02:21.012025+0000","last_clean":"2024-04-25T00:02:21.012025+0000","last_became_active":"2024-04-25T00:02:21.010105+0000","last_became_peered":"2024-04-25T00:02:21.010105+0000","last_unstale":"2024-04-25T00:02:21.012025+0000","last_undegraded":"2024-04-25T00:02:21.012025+0000","last_fullsized":"2024-04-25T00:02:21.012025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:29.720152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003057+0000","last_change":"2024-04-25T00:02:21.003057+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003057+0000","last_undegraded":"2024-04-25T00:02:21.003057+0000","last_fullsized":"2024-04-25T00:02:21.003057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013880+0000","last_change":"2024-04-25T00:02:21.011175+0000","last_active":"2024-04-25T00:02:21.013880+0000","last_peered":"2024-04-25T00:02:21.013880+0000","last_clean":"2024-04-25T00:02:21.013880+0000","last_became_active":"2024-04-25T00:02:21.010864+0000","last_became_peered":"2024-04-25T00:02:21.010864+0000","last_unstale":"2024-04-25T00:02:21.013880+0000","last_undegraded":"2024-04-25T00:02:21.013880+0000","last_fullsized":"2024-04-25T00:02:21.013880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:34.531368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000723+0000","last_change":"2024-04-25T00:02:21.000723+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000723+0000","last_undegraded":"2024-04-25T00:02:21.000723+0000","last_fullsized":"2024-04-25T00:02:21.000723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014057+0000","last_change":"2024-04-25T00:02:21.009271+0000","last_active":"2024-04-25T00:02:21.014057+0000","last_peered":"2024-04-25T00:02:21.014057+0000","last_clean":"2024-04-25T00:02:21.014057+0000","last_became_active":"2024-04-25T00:02:21.009010+0000","last_became_peered":"2024-04-25T00:02:21.009010+0000","last_unstale":"2024-04-25T00:02:21.014057+0000","last_undegraded":"2024-04-25T00:02:21.014057+0000","last_fullsized":"2024-04-25T00:02:21.014057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:07.487563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003376+0000","last_change":"2024-04-25T00:02:21.003376+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003376+0000","last_undegraded":"2024-04-25T00:02:21.003376+0000","last_fullsized":"2024-04-25T00:02:21.003376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011285+0000","last_change":"2024-04-25T00:02:21.009814+0000","last_active":"2024-04-25T00:02:21.011285+0000","last_peered":"2024-04-25T00:02:21.011285+0000","last_clean":"2024-04-25T00:02:21.011285+0000","last_became_active":"2024-04-25T00:02:21.009197+0000","last_became_peered":"2024-04-25T00:02:21.009197+0000","last_unstale":"2024-04-25T00:02:21.011285+0000","last_undegraded":"2024-04-25T00:02:21.011285+0000","last_fullsized":"2024-04-25T00:02:21.011285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:19.681462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006708+0000","last_change":"2024-04-25T00:02:21.006708+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006708+0000","last_undegraded":"2024-04-25T00:02:21.006708+0000","last_fullsized":"2024-04-25T00:02:21.006708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013290+0000","last_change":"2024-04-25T00:02:21.011072+0000","last_active":"2024-04-25T00:02:21.013290+0000","last_peered":"2024-04-25T00:02:21.013290+0000","last_clean":"2024-04-25T00:02:21.013290+0000","last_became_active":"2024-04-25T00:02:21.010789+0000","last_became_peered":"2024-04-25T00:02:21.010789+0000","last_unstale":"2024-04-25T00:02:21.013290+0000","last_undegraded":"2024-04-25T00:02:21.013290+0000","last_fullsized":"2024-04-25T00:02:21.013290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:45.234560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.004389+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:02:21.004389+0000","last_peered":"2024-04-25T00:02:21.004389+0000","last_clean":"2024-04-25T00:02:21.004389+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:02:21.004389+0000","last_undegraded":"2024-04-25T00:02:21.004389+0000","last_fullsized":"2024-04-25T00:02:21.004389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":382102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:02:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-04-25T00:02:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004482+0000', 'last_change': '2024-04-25T00:02:21.004482+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004482+0000', 'last_undegraded': '2024-04-25T00:02:21.004482+0000', 'last_fullsized': '2024-04-25T00:02:21.004482+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-04-25T00:02:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004023+0000', 'last_change': '2024-04-25T00:02:21.004023+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004023+0000', 'last_undegraded': '2024-04-25T00:02:21.004023+0000', 'last_fullsized': '2024-04-25T00:02:21.004023+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.006263+0000', 'last_change': '2024-04-25T00:02:21.006263+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.006263+0000', 'last_undegraded': '2024-04-25T00:02:21.006263+0000', 'last_fullsized': '2024-04-25T00:02:21.006263+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002424+0000', 'last_change': '2024-04-25T00:02:21.002424+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002424+0000', 'last_undegraded': '2024-04-25T00:02:21.002424+0000', 'last_fullsized': '2024-04-25T00:02:21.002424+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002669+0000', 'last_change': '2024-04-25T00:02:21.002669+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002669+0000', 'last_undegraded': '2024-04-25T00:02:21.002669+0000', 'last_fullsized': '2024-04-25T00:02:21.002669+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.001702+0000', 'last_change': '2024-04-25T00:02:21.001702+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.001702+0000', 'last_undegraded': '2024-04-25T00:02:21.001702+0000', 'last_fullsized': '2024-04-25T00:02:21.001702+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-04-25T00:02:25.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002415+0000', 'last_change': '2024-04-25T00:02:21.002415+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002415+0000', 'last_undegraded': '2024-04-25T00:02:21.002415+0000', 'last_fullsized': '2024-04-25T00:02:21.002415+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003841+0000', 'last_change': '2024-04-25T00:02:21.003841+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003841+0000', 'last_undegraded': '2024-04-25T00:02:21.003841+0000', 'last_fullsized': '2024-04-25T00:02:21.003841+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004039+0000', 'last_change': '2024-04-25T00:02:21.004039+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004039+0000', 'last_undegraded': '2024-04-25T00:02:21.004039+0000', 'last_fullsized': '2024-04-25T00:02:21.004039+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.998227+0000', 'last_change': '2024-04-25T00:02:20.998227+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.998227+0000', 'last_undegraded': '2024-04-25T00:02:20.998227+0000', 'last_fullsized': '2024-04-25T00:02:20.998227+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003622+0000', 'last_change': '2024-04-25T00:02:21.003622+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003622+0000', 'last_undegraded': '2024-04-25T00:02:21.003622+0000', 'last_fullsized': '2024-04-25T00:02:21.003622+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.001910+0000', 'last_change': '2024-04-25T00:02:21.001910+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.001910+0000', 'last_undegraded': '2024-04-25T00:02:21.001910+0000', 'last_fullsized': '2024-04-25T00:02:21.001910+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:02:25.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997224+0000', 'last_change': '2024-04-25T00:02:20.997224+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997224+0000', 'last_undegraded': '2024-04-25T00:02:20.997224+0000', 'last_fullsized': '2024-04-25T00:02:20.997224+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.005347+0000', 'last_change': '2024-04-25T00:02:21.005347+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.005347+0000', 'last_undegraded': '2024-04-25T00:02:21.005347+0000', 'last_fullsized': '2024-04-25T00:02:21.005347+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997702+0000', 'last_change': '2024-04-25T00:02:20.997702+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997702+0000', 'last_undegraded': '2024-04-25T00:02:20.997702+0000', 'last_fullsized': '2024-04-25T00:02:20.997702+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997729+0000', 'last_change': '2024-04-25T00:02:20.997729+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997729+0000', 'last_undegraded': '2024-04-25T00:02:20.997729+0000', 'last_fullsized': '2024-04-25T00:02:20.997729+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003670+0000', 'last_change': '2024-04-25T00:02:21.003670+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003670+0000', 'last_undegraded': '2024-04-25T00:02:21.003670+0000', 'last_fullsized': '2024-04-25T00:02:21.003670+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-04-25T00:02:25.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.000719+0000', 'last_change': '2024-04-25T00:02:21.000719+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.000719+0000', 'last_undegraded': '2024-04-25T00:02:21.000719+0000', 'last_fullsized': '2024-04-25T00:02:21.000719+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003140+0000', 'last_change': '2024-04-25T00:02:21.003140+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003140+0000', 'last_undegraded': '2024-04-25T00:02:21.003140+0000', 'last_fullsized': '2024-04-25T00:02:21.003140+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997349+0000', 'last_change': '2024-04-25T00:02:20.997349+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997349+0000', 'last_undegraded': '2024-04-25T00:02:20.997349+0000', 'last_fullsized': '2024-04-25T00:02:20.997349+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.998024+0000', 'last_change': '2024-04-25T00:02:20.998024+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.998024+0000', 'last_undegraded': '2024-04-25T00:02:20.998024+0000', 'last_fullsized': '2024-04-25T00:02:20.998024+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997322+0000', 'last_change': '2024-04-25T00:02:20.997322+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997322+0000', 'last_undegraded': '2024-04-25T00:02:20.997322+0000', 'last_fullsized': '2024-04-25T00:02:20.997322+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-04-25T00:02:25.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002667+0000', 'last_change': '2024-04-25T00:02:21.002667+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002667+0000', 'last_undegraded': '2024-04-25T00:02:21.002667+0000', 'last_fullsized': '2024-04-25T00:02:21.002667+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002165+0000', 'last_change': '2024-04-25T00:02:21.002165+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002165+0000', 'last_undegraded': '2024-04-25T00:02:21.002165+0000', 'last_fullsized': '2024-04-25T00:02:21.002165+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.999243+0000', 'last_change': '2024-04-25T00:02:20.999243+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.999243+0000', 'last_undegraded': '2024-04-25T00:02:20.999243+0000', 'last_fullsized': '2024-04-25T00:02:20.999243+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003685+0000', 'last_change': '2024-04-25T00:02:21.003685+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003685+0000', 'last_undegraded': '2024-04-25T00:02:21.003685+0000', 'last_fullsized': '2024-04-25T00:02:21.003685+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004843+0000', 'last_change': '2024-04-25T00:02:21.004843+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004843+0000', 'last_undegraded': '2024-04-25T00:02:21.004843+0000', 'last_fullsized': '2024-04-25T00:02:21.004843+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-04-25T00:02:25.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.998059+0000', 'last_change': '2024-04-25T00:02:20.998059+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.998059+0000', 'last_undegraded': '2024-04-25T00:02:20.998059+0000', 'last_fullsized': '2024-04-25T00:02:20.998059+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.007663+0000', 'last_change': '2024-04-25T00:02:21.007663+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.007663+0000', 'last_undegraded': '2024-04-25T00:02:21.007663+0000', 'last_fullsized': '2024-04-25T00:02:21.007663+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.000548+0000', 'last_change': '2024-04-25T00:02:21.000548+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.000548+0000', 'last_undegraded': '2024-04-25T00:02:21.000548+0000', 'last_fullsized': '2024-04-25T00:02:21.000548+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.001720+0000', 'last_change': '2024-04-25T00:02:21.001720+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.001720+0000', 'last_undegraded': '2024-04-25T00:02:21.001720+0000', 'last_fullsized': '2024-04-25T00:02:21.001720+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.000890+0000', 'last_change': '2024-04-25T00:02:21.000890+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.000890+0000', 'last_undegraded': '2024-04-25T00:02:21.000890+0000', 'last_fullsized': '2024-04-25T00:02:21.000890+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004905+0000', 'last_change': '2024-04-25T00:02:21.004905+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004905+0000', 'last_undegraded': '2024-04-25T00:02:21.004905+0000', 'last_fullsized': '2024-04-25T00:02:21.004905+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-04-25T00:02:25.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003284+0000', 'last_change': '2024-04-25T00:02:21.003284+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003284+0000', 'last_undegraded': '2024-04-25T00:02:21.003284+0000', 'last_fullsized': '2024-04-25T00:02:21.003284+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002102+0000', 'last_change': '2024-04-25T00:02:21.002102+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002102+0000', 'last_undegraded': '2024-04-25T00:02:21.002102+0000', 'last_fullsized': '2024-04-25T00:02:21.002102+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002741+0000', 'last_change': '2024-04-25T00:02:21.002741+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002741+0000', 'last_undegraded': '2024-04-25T00:02:21.002741+0000', 'last_fullsized': '2024-04-25T00:02:21.002741+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.000075+0000', 'last_change': '2024-04-25T00:02:21.000075+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.000075+0000', 'last_undegraded': '2024-04-25T00:02:21.000075+0000', 'last_fullsized': '2024-04-25T00:02:21.000075+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997330+0000', 'last_change': '2024-04-25T00:02:20.997330+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997330+0000', 'last_undegraded': '2024-04-25T00:02:20.997330+0000', 'last_fullsized': '2024-04-25T00:02:20.997330+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.005709+0000', 'last_change': '2024-04-25T00:02:21.005709+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.005709+0000', 'last_undegraded': '2024-04-25T00:02:21.005709+0000', 'last_fullsized': '2024-04-25T00:02:21.005709+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997196+0000', 'last_change': '2024-04-25T00:02:20.997196+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997196+0000', 'last_undegraded': '2024-04-25T00:02:20.997196+0000', 'last_fullsized': '2024-04-25T00:02:20.997196+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002079+0000', 'last_change': '2024-04-25T00:02:21.002079+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002079+0000', 'last_undegraded': '2024-04-25T00:02:21.002079+0000', 'last_fullsized': '2024-04-25T00:02:21.002079+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004081+0000', 'last_change': '2024-04-25T00:02:21.004081+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004081+0000', 'last_undegraded': '2024-04-25T00:02:21.004081+0000', 'last_fullsized': '2024-04-25T00:02:21.004081+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.001633+0000', 'last_change': '2024-04-25T00:02:21.001633+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.001633+0000', 'last_undegraded': '2024-04-25T00:02:21.001633+0000', 'last_fullsized': '2024-04-25T00:02:21.001633+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004537+0000', 'last_change': '2024-04-25T00:02:21.004537+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004537+0000', 'last_undegraded': '2024-04-25T00:02:21.004537+0000', 'last_fullsized': '2024-04-25T00:02:21.004537+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-04-25T00:02:25.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002303+0000', 'last_change': '2024-04-25T00:02:21.002303+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002303+0000', 'last_undegraded': '2024-04-25T00:02:21.002303+0000', 'last_fullsized': '2024-04-25T00:02:21.002303+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002805+0000', 'last_change': '2024-04-25T00:02:21.002805+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002805+0000', 'last_undegraded': '2024-04-25T00:02:21.002805+0000', 'last_fullsized': '2024-04-25T00:02:21.002805+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997520+0000', 'last_change': '2024-04-25T00:02:20.997520+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997520+0000', 'last_undegraded': '2024-04-25T00:02:20.997520+0000', 'last_fullsized': '2024-04-25T00:02:20.997520+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002070+0000', 'last_change': '2024-04-25T00:02:21.002070+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002070+0000', 'last_undegraded': '2024-04-25T00:02:21.002070+0000', 'last_fullsized': '2024-04-25T00:02:21.002070+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997694+0000', 'last_change': '2024-04-25T00:02:20.997694+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997694+0000', 'last_undegraded': '2024-04-25T00:02:20.997694+0000', 'last_fullsized': '2024-04-25T00:02:20.997694+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-04-25T00:02:25.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003026+0000', 'last_change': '2024-04-25T00:02:21.003026+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003026+0000', 'last_undegraded': '2024-04-25T00:02:21.003026+0000', 'last_fullsized': '2024-04-25T00:02:21.003026+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.000822+0000', 'last_change': '2024-04-25T00:02:21.000822+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.000822+0000', 'last_undegraded': '2024-04-25T00:02:21.000822+0000', 'last_fullsized': '2024-04-25T00:02:21.000822+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002416+0000', 'last_change': '2024-04-25T00:02:21.002416+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002416+0000', 'last_undegraded': '2024-04-25T00:02:21.002416+0000', 'last_fullsized': '2024-04-25T00:02:21.002416+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.007261+0000', 'last_change': '2024-04-25T00:02:21.007261+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.007261+0000', 'last_undegraded': '2024-04-25T00:02:21.007261+0000', 'last_fullsized': '2024-04-25T00:02:21.007261+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004626+0000', 'last_change': '2024-04-25T00:02:21.004626+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004626+0000', 'last_undegraded': '2024-04-25T00:02:21.004626+0000', 'last_fullsized': '2024-04-25T00:02:21.004626+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.005277+0000', 'last_change': '2024-04-25T00:02:21.005277+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.005277+0000', 'last_undegraded': '2024-04-25T00:02:21.005277+0000', 'last_fullsized': '2024-04-25T00:02:21.005277+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-04-25T00:02:25.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:20.997469+0000', 'last_change': '2024-04-25T00:02:20.997469+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:20.997469+0000', 'last_undegraded': '2024-04-25T00:02:20.997469+0000', 'last_fullsized': '2024-04-25T00:02:20.997469+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003155+0000', 'last_change': '2024-04-25T00:02:21.003155+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003155+0000', 'last_undegraded': '2024-04-25T00:02:21.003155+0000', 'last_fullsized': '2024-04-25T00:02:21.003155+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003163+0000', 'last_change': '2024-04-25T00:02:21.003163+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003163+0000', 'last_undegraded': '2024-04-25T00:02:21.003163+0000', 'last_fullsized': '2024-04-25T00:02:21.003163+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.002260+0000', 'last_change': '2024-04-25T00:02:21.002260+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.002260+0000', 'last_undegraded': '2024-04-25T00:02:21.002260+0000', 'last_fullsized': '2024-04-25T00:02:21.002260+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.004996+0000', 'last_change': '2024-04-25T00:02:21.004996+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.004996+0000', 'last_undegraded': '2024-04-25T00:02:21.004996+0000', 'last_fullsized': '2024-04-25T00:02:21.004996+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-04-25T00:02:25.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003057+0000', 'last_change': '2024-04-25T00:02:21.003057+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003057+0000', 'last_undegraded': '2024-04-25T00:02:21.003057+0000', 'last_fullsized': '2024-04-25T00:02:21.003057+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-04-25T00:02:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.000723+0000', 'last_change': '2024-04-25T00:02:21.000723+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.000723+0000', 'last_undegraded': '2024-04-25T00:02:21.000723+0000', 'last_fullsized': '2024-04-25T00:02:21.000723+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:02:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-04-25T00:02:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.003376+0000', 'last_change': '2024-04-25T00:02:21.003376+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.003376+0000', 'last_undegraded': '2024-04-25T00:02:21.003376+0000', 'last_fullsized': '2024-04-25T00:02:21.003376+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:02:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-04-25T00:02:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:02:21.006708+0000', 'last_change': '2024-04-25T00:02:21.006708+0000', 'last_active': '2024-04-25T00:02:20.985123+0000', 'last_peered': '2024-04-25T00:02:20.985123+0000', 'last_clean': '2024-04-25T00:02:20.985123+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:02:21.006708+0000', 'last_undegraded': '2024-04-25T00:02:21.006708+0000', 'last_fullsized': '2024-04-25T00:02:21.006708+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'last_clean_scrub_stamp': '2024-04-25T00:02:20.985123+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:02:25.651 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:02:25.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.768+0000 7f58634ba640 1 Processor -- start 2024-04-25T00:02:25.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.768+0000 7f58634ba640 1 -- start start 2024-04-25T00:02:25.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f58634ba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f585c0b3fd0 0x7f585c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f58634ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c06e510 0x7f585c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f58634ba640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585c144cf0 con 0x7f585c144910 2024-04-25T00:02:25.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f58634ba640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f585c0b2f30 con 0x7f585c0b3fd0 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f58634ba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f585c0b3210 con 0x7f585c06e510 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5860a55640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f585c0b3fd0 0x7f585c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5860a55640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f585c0b3fd0 0x7f585c06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54510/0 (socket says 172.21.15.113:54510) 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5860a55640 1 -- 172.21.15.113:0/3112593053 learned_addr learned my addr 172.21.15.113:0/3112593053 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5861a57640 1 --2- 172.21.15.113:0/3112593053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c06e510 0x7f585c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5860a55640 1 -- 172.21.15.113:0/3112593053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c06e510 msgr2=0x7f585c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5860a55640 1 --2- 172.21.15.113:0/3112593053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c06e510 0x7f585c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5860a55640 1 -- 172.21.15.113:0/3112593053 >> v1:172.21.15.178:6789/0 conn(0x7f585c144910 legacy=0x7f585c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5860a55640 1 -- 172.21.15.113:0/3112593053 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c0b34f0 con 0x7f585c0b3fd0 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.769+0000 7f5861a57640 1 --2- 172.21.15.113:0/3112593053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c06e510 0x7f585c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.770+0000 7f5860a55640 1 --2- 172.21.15.113:0/3112593053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c0b3fd0 0x7f585c06dfd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f584c0098a0 tx=0x7f584c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d01a215387c2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.770+0000 7f5853fff640 1 -- 172.21.15.113:0/3112593053 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c037070 con 0x7f585c0b3fd0 2024-04-25T00:02:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.770+0000 7f5853fff640 1 -- 172.21.15.113:0/3112593053 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584c032440 con 0x7f585c0b3fd0 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.770+0000 7f5853fff640 1 -- 172.21.15.113:0/3112593053 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c032810 con 0x7f585c0b3fd0 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 -- 172.21.15.113:0/3112593053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c0b3fd0 msgr2=0x7f585c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 --2- 172.21.15.113:0/3112593053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c0b3fd0 0x7f585c06dfd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f584c0098a0 tx=0x7f584c02a290 comp rx=0 tx=0).stop 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 -- 172.21.15.113:0/3112593053 shutdown_connections 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 --2- 172.21.15.113:0/3112593053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c06e510 0x7f585c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 --2- 172.21.15.113:0/3112593053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c0b3fd0 0x7f585c06dfd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 -- 172.21.15.113:0/3112593053 >> 172.21.15.113:0/3112593053 conn(0x7f585c0b1a50 msgr2=0x7f585c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 -- 172.21.15.113:0/3112593053 shutdown_connections 2024-04-25T00:02:25.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.771+0000 7f58634ba640 1 -- 172.21.15.113:0/3112593053 wait complete. 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.772+0000 7f58634ba640 1 Processor -- start 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.772+0000 7f58634ba640 1 -- start start 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f58634ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c06e510 0x7f585c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f58634ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c0b3fd0 0x7f585c133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f58634ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f585c144910 0x7f585c13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f58634ba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f585c0b4da0 con 0x7f585c06e510 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f58634ba640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f585c0b4440 con 0x7f585c144910 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f58634ba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f585c0b4f20 con 0x7f585c0b3fd0 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c06e510 0x7f585c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c06e510 0x7f585c15a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54512/0 (socket says 172.21.15.113:54512) 2024-04-25T00:02:25.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 -- 172.21.15.113:0/3525980822 learned_addr learned my addr 172.21.15.113:0/3525980822 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5860a55640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c0b3fd0 0x7f585c133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 -- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c0b3fd0 msgr2=0x7f585c133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861a57640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f585c144910 0x7f585c13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c0b3fd0 0x7f585c133830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 -- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f585c144910 msgr2=0x7f585c13a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f585c144910 0x7f585c13a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 -- 172.21.15.113:0/3525980822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c1341a0 con 0x7f585c06e510 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5861256640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c06e510 0x7f585c15a4e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f585800c790 tx=0x7f585800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5851ffb640 1 -- 172.21.15.113:0/3525980822 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858016020 con 0x7f585c06e510 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5851ffb640 1 -- 172.21.15.113:0/3525980822 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5858006d50 con 0x7f585c06e510 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f5851ffb640 1 -- 172.21.15.113:0/3525980822 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858003bb0 con 0x7f585c06e510 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.773+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c134480 con 0x7f585c06e510 2024-04-25T00:02:25.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.774+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c134a40 con 0x7f585c06e510 2024-04-25T00:02:25.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.775+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c136330 con 0x7f585c06e510 2024-04-25T00:02:25.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.775+0000 7f5851ffb640 1 -- 172.21.15.113:0/3525980822 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f585800a850 con 0x7f585c06e510 2024-04-25T00:02:25.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.775+0000 7f5851ffb640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f584006be30 0x7f584006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:25.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.775+0000 7f5860a55640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f584006be30 0x7f584006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:25.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.775+0000 7f5851ffb640 1 -- 172.21.15.113:0/3525980822 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f585808aaf0 con 0x7f585c06e510 2024-04-25T00:02:25.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.776+0000 7f5860a55640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f584006be30 0x7f584006e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f584c034000 tx=0x7f584c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:25.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:25.780+0000 7f5851ffb640 1 -- 172.21.15.113:0/3525980822 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5858020020 con 0x7f585c06e510 2024-04-25T00:02:26.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.018+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f585c0691a0 con 0x7f585c06e510 2024-04-25T00:02:26.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.018+0000 7f5851ffb640 1 -- 172.21.15.113:0/3525980822 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1454 (secure 0 0 0) 0x7f5858019310 con 0x7f585c06e510 2024-04-25T00:02:26.019 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:26.019 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":271,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":76,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":280797184,"bytes_avail":772813316096,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":43,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6951}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-04-25T00:02:11.157835+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:02:26.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f584006be30 msgr2=0x7f584006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f584006be30 0x7f584006e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f584c034000 tx=0x7f584c02e000 comp rx=0 tx=0).stop 2024-04-25T00:02:26.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c06e510 msgr2=0x7f585c15a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c06e510 0x7f585c15a4e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f585800c790 tx=0x7f585800cc50 comp rx=0 tx=0).stop 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 shutdown_connections 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f584006be30 0x7f584006e2f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f585c144910 0x7f585c13a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f585c0b3fd0 0x7f585c133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 --2- 172.21.15.113:0/3525980822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f585c06e510 0x7f585c15a4e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 >> 172.21.15.113:0/3525980822 conn(0x7f585c0b1a50 msgr2=0x7f585c072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 shutdown_connections 2024-04-25T00:02:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.021+0000 7f58634ba640 1 -- 172.21.15.113:0/3525980822 wait complete. 2024-04-25T00:02:26.032 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:02:26.032 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:02:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.127+0000 7fd628f16640 1 Processor -- start 2024-04-25T00:02:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.127+0000 7fd628f16640 1 -- start start 2024-04-25T00:02:26.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.128+0000 7fd628f16640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd62415e320 0x7fd6240d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:26.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.128+0000 7fd628f16640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:26.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.128+0000 7fd628f16640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6240d3de0 con 0x7fd6240d2290 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.128+0000 7fd628f16640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd6240d14d0 con 0x7fd62415e320 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.128+0000 7fd628f16640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd6240d3410 con 0x7fd6240d0b00 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd62259c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd62415e320 0x7fd6240d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd622d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd62259c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd62415e320 0x7fd6240d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:50186/0 (socket says 172.21.15.113:50186) 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd622d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:51142/0 (socket says 172.21.15.113:51142) 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd62259c640 1 -- 172.21.15.113:0/1234756746 learned_addr learned my addr 172.21.15.113:0/1234756746 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd621d9b640 1 --1- 172.21.15.113:0/1234756746 >> v1:172.21.15.178:6789/0 conn(0x7fd6240d2290 0x7fd6240b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33778/0 (socket says 172.21.15.113:33778) 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd622d9d640 1 -- 172.21.15.113:0/1234756746 >> v2:172.21.15.178:3300/0 conn(0x7fd62415e320 msgr2=0x7fd6240d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd622d9d640 1 --2- 172.21.15.113:0/1234756746 >> v2:172.21.15.178:3300/0 conn(0x7fd62415e320 0x7fd6240d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd622d9d640 1 -- 172.21.15.113:0/1234756746 >> v1:172.21.15.178:6789/0 conn(0x7fd6240d2290 legacy=0x7fd6240b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd622d9d640 1 -- 172.21.15.113:0/1234756746 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6240d2800 con 0x7fd6240d0b00 2024-04-25T00:02:26.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.129+0000 7fd62259c640 1 --2- 172.21.15.113:0/1234756746 >> v2:172.21.15.178:3300/0 conn(0x7fd62415e320 0x7fd6240d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.130+0000 7fd622d9d640 1 --2- 172.21.15.113:0/1234756746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240d0f90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0097a0 tx=0x7fd61c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e64413b5685d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.130+0000 7fd62159a640 1 -- 172.21.15.113:0/1234756746 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c037070 con 0x7fd6240d0b00 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.130+0000 7fd62159a640 1 -- 172.21.15.113:0/1234756746 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c032440 con 0x7fd6240d0b00 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.130+0000 7fd62159a640 1 -- 172.21.15.113:0/1234756746 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c032810 con 0x7fd6240d0b00 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 -- 172.21.15.113:0/1234756746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 msgr2=0x7fd6240d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 --2- 172.21.15.113:0/1234756746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240d0f90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0097a0 tx=0x7fd61c02a210 comp rx=0 tx=0).stop 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 -- 172.21.15.113:0/1234756746 shutdown_connections 2024-04-25T00:02:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 --2- 172.21.15.113:0/1234756746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240d0f90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 --2- 172.21.15.113:0/1234756746 >> v2:172.21.15.178:3300/0 conn(0x7fd62415e320 0x7fd6240d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 -- 172.21.15.113:0/1234756746 >> 172.21.15.113:0/1234756746 conn(0x7fd6240b5c20 msgr2=0x7fd6240b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 -- 172.21.15.113:0/1234756746 shutdown_connections 2024-04-25T00:02:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.131+0000 7fd628f16640 1 -- 172.21.15.113:0/1234756746 wait complete. 2024-04-25T00:02:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.132+0000 7fd628f16640 1 Processor -- start 2024-04-25T00:02:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.132+0000 7fd628f16640 1 -- start start 2024-04-25T00:02:26.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd628f16640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:26.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd628f16640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6240d2290 0x7fd6241543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:26.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd628f16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd62415e320 0x7fd624146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd628f16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6240d30a0 con 0x7fd62415e320 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd628f16640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6240d2dc0 con 0x7fd6240d2290 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd628f16640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd62415e700 con 0x7fd6240d0b00 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd62415e320 0x7fd624146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd62415e320 0x7fd624146660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54522/0 (socket says 172.21.15.113:54522) 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 -- 172.21.15.113:0/2920983370 learned_addr learned my addr 172.21.15.113:0/2920983370 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd621d9b640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6240d2290 0x7fd6241543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd62259c640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 -- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 msgr2=0x7fd6240cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 -- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6240d2290 msgr2=0x7fd6241543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6240d2290 0x7fd6241543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 -- 172.21.15.113:0/2920983370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6240cc940 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd621d9b640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6240d2290 0x7fd6241543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd622d9d640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd62415e320 0x7fd624146660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd61c035040 tx=0x7fd61c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.133+0000 7fd612ffd640 1 -- 172.21.15.113:0/2920983370 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c037040 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.134+0000 7fd612ffd640 1 -- 172.21.15.113:0/2920983370 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c042070 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.134+0000 7fd612ffd640 1 -- 172.21.15.113:0/2920983370 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0308a0 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.134+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6240ccc80 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.134+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6240cd4c0 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.135+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6240691a0 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.135+0000 7fd612ffd640 1 -- 172.21.15.113:0/2920983370 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd61c030a40 con 0x7fd62415e320 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.136+0000 7fd612ffd640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd60406be30 0x7fd60406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.136+0000 7fd62259c640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd60406be30 0x7fd60406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.136+0000 7fd612ffd640 1 -- 172.21.15.113:0/2920983370 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7fd61c0ab830 con 0x7fd62415e320 2024-04-25T00:02:26.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.136+0000 7fd62259c640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd60406be30 0x7fd60406e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ce0 tx=0x7fd61400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:26.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.140+0000 7fd612ffd640 1 -- 172.21.15.113:0/2920983370 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd6240ccc80 con 0x7fd62415e320 2024-04-25T00:02:26.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.312+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd62415cc90 con 0x7fd60406be30 2024-04-25T00:02:26.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.317+0000 7fd612ffd640 1 -- 172.21.15.113:0/2920983370 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7fd62415cc90 con 0x7fd60406be30 2024-04-25T00:02:26.320 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:26.321 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:02:26.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd60406be30 msgr2=0x7fd60406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd60406be30 0x7fd60406e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ce0 tx=0x7fd61400b040 comp rx=0 tx=0).stop 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd62415e320 msgr2=0x7fd624146660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd62415e320 0x7fd624146660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd61c035040 tx=0x7fd61c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 shutdown_connections 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd60406be30 0x7fd60406e2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6240d2290 0x7fd6241543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd62415e320 0x7fd624146660 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 --2- 172.21.15.113:0/2920983370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6240d0b00 0x7fd6240cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 >> 172.21.15.113:0/2920983370 conn(0x7fd6240b5c20 msgr2=0x7fd6240b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.320+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 shutdown_connections 2024-04-25T00:02:26.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:26.321+0000 7fd628f16640 1 -- 172.21.15.113:0/2920983370 wait complete. 2024-04-25T00:02:26.331 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-25T00:02:25.159984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274216,"kb_used_data":8116,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700504,"statfs":{"total":773094113280,"available":772813316096,"internally_reserved":0,"allocated":8310784,"data_stored":4193627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.172535"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004482+0000","last_change":"2024-04-25T00:02:21.004482+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004482+0000","last_undegraded":"2024-04-25T00:02:21.004482+0000","last_fullsized":"2024-04-25T00:02:21.004482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004023+0000","last_change":"2024-04-25T00:02:21.004023+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004023+0000","last_undegraded":"2024-04-25T00:02:21.004023+0000","last_fullsized":"2024-04-25T00:02:21.004023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006263+0000","last_change":"2024-04-25T00:02:21.006263+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006263+0000","last_undegraded":"2024-04-25T00:02:21.006263+0000","last_fullsized":"2024-04-25T00:02:21.006263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002424+0000","last_change":"2024-04-25T00:02:21.002424+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002424+0000","last_undegraded":"2024-04-25T00:02:21.002424+0000","last_fullsized":"2024-04-25T00:02:21.002424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002669+0000","last_change":"2024-04-25T00:02:21.002669+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002669+0000","last_undegraded":"2024-04-25T00:02:21.002669+0000","last_fullsized":"2024-04-25T00:02:21.002669+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001702+0000","last_change":"2024-04-25T00:02:21.001702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001702+0000","last_undegraded":"2024-04-25T00:02:21.001702+0000","last_fullsized":"2024-04-25T00:02:21.001702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002415+0000","last_change":"2024-04-25T00:02:21.002415+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002415+0000","last_undegraded":"2024-04-25T00:02:21.002415+0000","last_fullsized":"2024-04-25T00:02:21.002415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003841+0000","last_change":"2024-04-25T00:02:21.003841+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003841+0000","last_undegraded":"2024-04-25T00:02:21.003841+0000","last_fullsized":"2024-04-25T00:02:21.003841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004039+0000","last_change":"2024-04-25T00:02:21.004039+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004039+0000","last_undegraded":"2024-04-25T00:02:21.004039+0000","last_fullsized":"2024-04-25T00:02:21.004039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998227+0000","last_change":"2024-04-25T00:02:20.998227+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998227+0000","last_undegraded":"2024-04-25T00:02:20.998227+0000","last_fullsized":"2024-04-25T00:02:20.998227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003622+0000","last_change":"2024-04-25T00:02:21.003622+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003622+0000","last_undegraded":"2024-04-25T00:02:21.003622+0000","last_fullsized":"2024-04-25T00:02:21.003622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001910+0000","last_change":"2024-04-25T00:02:21.001910+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001910+0000","last_undegraded":"2024-04-25T00:02:21.001910+0000","last_fullsized":"2024-04-25T00:02:21.001910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997224+0000","last_change":"2024-04-25T00:02:20.997224+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997224+0000","last_undegraded":"2024-04-25T00:02:20.997224+0000","last_fullsized":"2024-04-25T00:02:20.997224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005347+0000","last_change":"2024-04-25T00:02:21.005347+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005347+0000","last_undegraded":"2024-04-25T00:02:21.005347+0000","last_fullsized":"2024-04-25T00:02:21.005347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997702+0000","last_change":"2024-04-25T00:02:20.997702+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997702+0000","last_undegraded":"2024-04-25T00:02:20.997702+0000","last_fullsized":"2024-04-25T00:02:20.997702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997729+0000","last_change":"2024-04-25T00:02:20.997729+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997729+0000","last_undegraded":"2024-04-25T00:02:20.997729+0000","last_fullsized":"2024-04-25T00:02:20.997729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003670+0000","last_change":"2024-04-25T00:02:21.003670+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003670+0000","last_undegraded":"2024-04-25T00:02:21.003670+0000","last_fullsized":"2024-04-25T00:02:21.003670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000719+0000","last_change":"2024-04-25T00:02:21.000719+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000719+0000","last_undegraded":"2024-04-25T00:02:21.000719+0000","last_fullsized":"2024-04-25T00:02:21.000719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003140+0000","last_change":"2024-04-25T00:02:21.003140+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003140+0000","last_undegraded":"2024-04-25T00:02:21.003140+0000","last_fullsized":"2024-04-25T00:02:21.003140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997349+0000","last_change":"2024-04-25T00:02:20.997349+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997349+0000","last_undegraded":"2024-04-25T00:02:20.997349+0000","last_fullsized":"2024-04-25T00:02:20.997349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998024+0000","last_change":"2024-04-25T00:02:20.998024+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998024+0000","last_undegraded":"2024-04-25T00:02:20.998024+0000","last_fullsized":"2024-04-25T00:02:20.998024+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997322+0000","last_change":"2024-04-25T00:02:20.997322+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997322+0000","last_undegraded":"2024-04-25T00:02:20.997322+0000","last_fullsized":"2024-04-25T00:02:20.997322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002667+0000","last_change":"2024-04-25T00:02:21.002667+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002667+0000","last_undegraded":"2024-04-25T00:02:21.002667+0000","last_fullsized":"2024-04-25T00:02:21.002667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002165+0000","last_change":"2024-04-25T00:02:21.002165+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002165+0000","last_undegraded":"2024-04-25T00:02:21.002165+0000","last_fullsized":"2024-04-25T00:02:21.002165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.999243+0000","last_change":"2024-04-25T00:02:20.999243+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.999243+0000","last_undegraded":"2024-04-25T00:02:20.999243+0000","last_fullsized":"2024-04-25T00:02:20.999243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003685+0000","last_change":"2024-04-25T00:02:21.003685+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003685+0000","last_undegraded":"2024-04-25T00:02:21.003685+0000","last_fullsized":"2024-04-25T00:02:21.003685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004843+0000","last_change":"2024-04-25T00:02:21.004843+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004843+0000","last_undegraded":"2024-04-25T00:02:21.004843+0000","last_fullsized":"2024-04-25T00:02:21.004843+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.998059+0000","last_change":"2024-04-25T00:02:20.998059+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.998059+0000","last_undegraded":"2024-04-25T00:02:20.998059+0000","last_fullsized":"2024-04-25T00:02:20.998059+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007663+0000","last_change":"2024-04-25T00:02:21.007663+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007663+0000","last_undegraded":"2024-04-25T00:02:21.007663+0000","last_fullsized":"2024-04-25T00:02:21.007663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000548+0000","last_change":"2024-04-25T00:02:21.000548+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000548+0000","last_undegraded":"2024-04-25T00:02:21.000548+0000","last_fullsized":"2024-04-25T00:02:21.000548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001720+0000","last_change":"2024-04-25T00:02:21.001720+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001720+0000","last_undegraded":"2024-04-25T00:02:21.001720+0000","last_fullsized":"2024-04-25T00:02:21.001720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000890+0000","last_change":"2024-04-25T00:02:21.000890+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000890+0000","last_undegraded":"2024-04-25T00:02:21.000890+0000","last_fullsized":"2024-04-25T00:02:21.000890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004905+0000","last_change":"2024-04-25T00:02:21.004905+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004905+0000","last_undegraded":"2024-04-25T00:02:21.004905+0000","last_fullsized":"2024-04-25T00:02:21.004905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014554+0000","last_change":"2024-04-25T00:02:21.012893+0000","last_active":"2024-04-25T00:02:21.014554+0000","last_peered":"2024-04-25T00:02:21.014554+0000","last_clean":"2024-04-25T00:02:21.014554+0000","last_became_active":"2024-04-25T00:02:21.012601+0000","last_became_peered":"2024-04-25T00:02:21.012601+0000","last_unstale":"2024-04-25T00:02:21.014554+0000","last_undegraded":"2024-04-25T00:02:21.014554+0000","last_fullsized":"2024-04-25T00:02:21.014554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.876656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003284+0000","last_change":"2024-04-25T00:02:21.003284+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003284+0000","last_undegraded":"2024-04-25T00:02:21.003284+0000","last_fullsized":"2024-04-25T00:02:21.003284+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013760+0000","last_change":"2024-04-25T00:02:21.011558+0000","last_active":"2024-04-25T00:02:21.013760+0000","last_peered":"2024-04-25T00:02:21.013760+0000","last_clean":"2024-04-25T00:02:21.013760+0000","last_became_active":"2024-04-25T00:02:21.011287+0000","last_became_peered":"2024-04-25T00:02:21.011287+0000","last_unstale":"2024-04-25T00:02:21.013760+0000","last_undegraded":"2024-04-25T00:02:21.013760+0000","last_fullsized":"2024-04-25T00:02:21.013760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:20.889569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002102+0000","last_change":"2024-04-25T00:02:21.002102+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002102+0000","last_undegraded":"2024-04-25T00:02:21.002102+0000","last_fullsized":"2024-04-25T00:02:21.002102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013400+0000","last_change":"2024-04-25T00:02:21.007131+0000","last_active":"2024-04-25T00:02:21.013400+0000","last_peered":"2024-04-25T00:02:21.013400+0000","last_clean":"2024-04-25T00:02:21.013400+0000","last_became_active":"2024-04-25T00:02:21.006720+0000","last_became_peered":"2024-04-25T00:02:21.006720+0000","last_unstale":"2024-04-25T00:02:21.013400+0000","last_undegraded":"2024-04-25T00:02:21.013400+0000","last_fullsized":"2024-04-25T00:02:21.013400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:23.288653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002741+0000","last_change":"2024-04-25T00:02:21.002741+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002741+0000","last_undegraded":"2024-04-25T00:02:21.002741+0000","last_fullsized":"2024-04-25T00:02:21.002741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014923+0000","last_change":"2024-04-25T00:02:21.012185+0000","last_active":"2024-04-25T00:02:21.014923+0000","last_peered":"2024-04-25T00:02:21.014923+0000","last_clean":"2024-04-25T00:02:21.014923+0000","last_became_active":"2024-04-25T00:02:21.011910+0000","last_became_peered":"2024-04-25T00:02:21.011910+0000","last_unstale":"2024-04-25T00:02:21.014923+0000","last_undegraded":"2024-04-25T00:02:21.014923+0000","last_fullsized":"2024-04-25T00:02:21.014923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:24.418612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000075+0000","last_change":"2024-04-25T00:02:21.000075+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000075+0000","last_undegraded":"2024-04-25T00:02:21.000075+0000","last_fullsized":"2024-04-25T00:02:21.000075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016721+0000","last_change":"2024-04-25T00:02:21.015042+0000","last_active":"2024-04-25T00:02:21.016721+0000","last_peered":"2024-04-25T00:02:21.016721+0000","last_clean":"2024-04-25T00:02:21.016721+0000","last_became_active":"2024-04-25T00:02:21.014741+0000","last_became_peered":"2024-04-25T00:02:21.014741+0000","last_unstale":"2024-04-25T00:02:21.016721+0000","last_undegraded":"2024-04-25T00:02:21.016721+0000","last_fullsized":"2024-04-25T00:02:21.016721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:54.347862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997330+0000","last_change":"2024-04-25T00:02:20.997330+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997330+0000","last_undegraded":"2024-04-25T00:02:20.997330+0000","last_fullsized":"2024-04-25T00:02:20.997330+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.010476+0000","last_change":"2024-04-25T00:02:21.008726+0000","last_active":"2024-04-25T00:02:21.010476+0000","last_peered":"2024-04-25T00:02:21.010476+0000","last_clean":"2024-04-25T00:02:21.010476+0000","last_became_active":"2024-04-25T00:02:21.008442+0000","last_became_peered":"2024-04-25T00:02:21.008442+0000","last_unstale":"2024-04-25T00:02:21.010476+0000","last_undegraded":"2024-04-25T00:02:21.010476+0000","last_fullsized":"2024-04-25T00:02:21.010476+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:24.591088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005709+0000","last_change":"2024-04-25T00:02:21.005709+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005709+0000","last_undegraded":"2024-04-25T00:02:21.005709+0000","last_fullsized":"2024-04-25T00:02:21.005709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013907+0000","last_change":"2024-04-25T00:02:21.011239+0000","last_active":"2024-04-25T00:02:21.013907+0000","last_peered":"2024-04-25T00:02:21.013907+0000","last_clean":"2024-04-25T00:02:21.013907+0000","last_became_active":"2024-04-25T00:02:21.010824+0000","last_became_peered":"2024-04-25T00:02:21.010824+0000","last_unstale":"2024-04-25T00:02:21.013907+0000","last_undegraded":"2024-04-25T00:02:21.013907+0000","last_fullsized":"2024-04-25T00:02:21.013907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:24.542662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997196+0000","last_change":"2024-04-25T00:02:20.997196+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997196+0000","last_undegraded":"2024-04-25T00:02:20.997196+0000","last_fullsized":"2024-04-25T00:02:20.997196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014422+0000","last_change":"2024-04-25T00:02:21.013287+0000","last_active":"2024-04-25T00:02:21.014422+0000","last_peered":"2024-04-25T00:02:21.014422+0000","last_clean":"2024-04-25T00:02:21.014422+0000","last_became_active":"2024-04-25T00:02:21.013026+0000","last_became_peered":"2024-04-25T00:02:21.013026+0000","last_unstale":"2024-04-25T00:02:21.014422+0000","last_undegraded":"2024-04-25T00:02:21.014422+0000","last_fullsized":"2024-04-25T00:02:21.014422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:23.963484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002079+0000","last_change":"2024-04-25T00:02:21.002079+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002079+0000","last_undegraded":"2024-04-25T00:02:21.002079+0000","last_fullsized":"2024-04-25T00:02:21.002079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014442+0000","last_change":"2024-04-25T00:02:21.013431+0000","last_active":"2024-04-25T00:02:21.014442+0000","last_peered":"2024-04-25T00:02:21.014442+0000","last_clean":"2024-04-25T00:02:21.014442+0000","last_became_active":"2024-04-25T00:02:21.012907+0000","last_became_peered":"2024-04-25T00:02:21.012907+0000","last_unstale":"2024-04-25T00:02:21.014442+0000","last_undegraded":"2024-04-25T00:02:21.014442+0000","last_fullsized":"2024-04-25T00:02:21.014442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:43.668722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004081+0000","last_change":"2024-04-25T00:02:21.004081+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004081+0000","last_undegraded":"2024-04-25T00:02:21.004081+0000","last_fullsized":"2024-04-25T00:02:21.004081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013959+0000","last_change":"2024-04-25T00:02:21.008299+0000","last_active":"2024-04-25T00:02:21.013959+0000","last_peered":"2024-04-25T00:02:21.013959+0000","last_clean":"2024-04-25T00:02:21.013959+0000","last_became_active":"2024-04-25T00:02:21.008024+0000","last_became_peered":"2024-04-25T00:02:21.008024+0000","last_unstale":"2024-04-25T00:02:21.013959+0000","last_undegraded":"2024-04-25T00:02:21.013959+0000","last_fullsized":"2024-04-25T00:02:21.013959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:50.202267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.001633+0000","last_change":"2024-04-25T00:02:21.001633+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.001633+0000","last_undegraded":"2024-04-25T00:02:21.001633+0000","last_fullsized":"2024-04-25T00:02:21.001633+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014517+0000","last_change":"2024-04-25T00:02:21.011826+0000","last_active":"2024-04-25T00:02:21.014517+0000","last_peered":"2024-04-25T00:02:21.014517+0000","last_clean":"2024-04-25T00:02:21.014517+0000","last_became_active":"2024-04-25T00:02:21.011515+0000","last_became_peered":"2024-04-25T00:02:21.011515+0000","last_unstale":"2024-04-25T00:02:21.014517+0000","last_undegraded":"2024-04-25T00:02:21.014517+0000","last_fullsized":"2024-04-25T00:02:21.014517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:38.562371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004537+0000","last_change":"2024-04-25T00:02:21.004537+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004537+0000","last_undegraded":"2024-04-25T00:02:21.004537+0000","last_fullsized":"2024-04-25T00:02:21.004537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014276+0000","last_change":"2024-04-25T00:02:21.012403+0000","last_active":"2024-04-25T00:02:21.014276+0000","last_peered":"2024-04-25T00:02:21.014276+0000","last_clean":"2024-04-25T00:02:21.014276+0000","last_became_active":"2024-04-25T00:02:21.012150+0000","last_became_peered":"2024-04-25T00:02:21.012150+0000","last_unstale":"2024-04-25T00:02:21.014276+0000","last_undegraded":"2024-04-25T00:02:21.014276+0000","last_fullsized":"2024-04-25T00:02:21.014276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:19.301050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002303+0000","last_change":"2024-04-25T00:02:21.002303+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002303+0000","last_undegraded":"2024-04-25T00:02:21.002303+0000","last_fullsized":"2024-04-25T00:02:21.002303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012131+0000","last_change":"2024-04-25T00:02:21.009088+0000","last_active":"2024-04-25T00:02:21.012131+0000","last_peered":"2024-04-25T00:02:21.012131+0000","last_clean":"2024-04-25T00:02:21.012131+0000","last_became_active":"2024-04-25T00:02:21.008821+0000","last_became_peered":"2024-04-25T00:02:21.008821+0000","last_unstale":"2024-04-25T00:02:21.012131+0000","last_undegraded":"2024-04-25T00:02:21.012131+0000","last_fullsized":"2024-04-25T00:02:21.012131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002805+0000","last_change":"2024-04-25T00:02:21.002805+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002805+0000","last_undegraded":"2024-04-25T00:02:21.002805+0000","last_fullsized":"2024-04-25T00:02:21.002805+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015993+0000","last_change":"2024-04-25T00:02:21.015109+0000","last_active":"2024-04-25T00:02:21.015993+0000","last_peered":"2024-04-25T00:02:21.015993+0000","last_clean":"2024-04-25T00:02:21.015993+0000","last_became_active":"2024-04-25T00:02:21.014777+0000","last_became_peered":"2024-04-25T00:02:21.014777+0000","last_unstale":"2024-04-25T00:02:21.015993+0000","last_undegraded":"2024-04-25T00:02:21.015993+0000","last_fullsized":"2024-04-25T00:02:21.015993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.858993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997520+0000","last_change":"2024-04-25T00:02:20.997520+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997520+0000","last_undegraded":"2024-04-25T00:02:20.997520+0000","last_fullsized":"2024-04-25T00:02:20.997520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011475+0000","last_change":"2024-04-25T00:02:21.010049+0000","last_active":"2024-04-25T00:02:21.011475+0000","last_peered":"2024-04-25T00:02:21.011475+0000","last_clean":"2024-04-25T00:02:21.011475+0000","last_became_active":"2024-04-25T00:02:21.009715+0000","last_became_peered":"2024-04-25T00:02:21.009715+0000","last_unstale":"2024-04-25T00:02:21.011475+0000","last_undegraded":"2024-04-25T00:02:21.011475+0000","last_fullsized":"2024-04-25T00:02:21.011475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:57.913997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002070+0000","last_change":"2024-04-25T00:02:21.002070+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002070+0000","last_undegraded":"2024-04-25T00:02:21.002070+0000","last_fullsized":"2024-04-25T00:02:21.002070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014496+0000","last_change":"2024-04-25T00:02:21.010313+0000","last_active":"2024-04-25T00:02:21.014496+0000","last_peered":"2024-04-25T00:02:21.014496+0000","last_clean":"2024-04-25T00:02:21.014496+0000","last_became_active":"2024-04-25T00:02:21.010052+0000","last_became_peered":"2024-04-25T00:02:21.010052+0000","last_unstale":"2024-04-25T00:02:21.014496+0000","last_undegraded":"2024-04-25T00:02:21.014496+0000","last_fullsized":"2024-04-25T00:02:21.014496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:27.273733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997694+0000","last_change":"2024-04-25T00:02:20.997694+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997694+0000","last_undegraded":"2024-04-25T00:02:20.997694+0000","last_fullsized":"2024-04-25T00:02:20.997694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014955+0000","last_change":"2024-04-25T00:02:21.013942+0000","last_active":"2024-04-25T00:02:21.014955+0000","last_peered":"2024-04-25T00:02:21.014955+0000","last_clean":"2024-04-25T00:02:21.014955+0000","last_became_active":"2024-04-25T00:02:21.013529+0000","last_became_peered":"2024-04-25T00:02:21.013529+0000","last_unstale":"2024-04-25T00:02:21.014955+0000","last_undegraded":"2024-04-25T00:02:21.014955+0000","last_fullsized":"2024-04-25T00:02:21.014955+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:03.561219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003026+0000","last_change":"2024-04-25T00:02:21.003026+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003026+0000","last_undegraded":"2024-04-25T00:02:21.003026+0000","last_fullsized":"2024-04-25T00:02:21.003026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013509+0000","last_change":"2024-04-25T00:02:21.007387+0000","last_active":"2024-04-25T00:02:21.013509+0000","last_peered":"2024-04-25T00:02:21.013509+0000","last_clean":"2024-04-25T00:02:21.013509+0000","last_became_active":"2024-04-25T00:02:21.007015+0000","last_became_peered":"2024-04-25T00:02:21.007015+0000","last_unstale":"2024-04-25T00:02:21.013509+0000","last_undegraded":"2024-04-25T00:02:21.013509+0000","last_fullsized":"2024-04-25T00:02:21.013509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:45.498458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000822+0000","last_change":"2024-04-25T00:02:21.000822+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000822+0000","last_undegraded":"2024-04-25T00:02:21.000822+0000","last_fullsized":"2024-04-25T00:02:21.000822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.015260+0000","last_change":"2024-04-25T00:02:21.014230+0000","last_active":"2024-04-25T00:02:21.015260+0000","last_peered":"2024-04-25T00:02:21.015260+0000","last_clean":"2024-04-25T00:02:21.015260+0000","last_became_active":"2024-04-25T00:02:21.013946+0000","last_became_peered":"2024-04-25T00:02:21.013946+0000","last_unstale":"2024-04-25T00:02:21.015260+0000","last_undegraded":"2024-04-25T00:02:21.015260+0000","last_fullsized":"2024-04-25T00:02:21.015260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:26.029288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002416+0000","last_change":"2024-04-25T00:02:21.002416+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002416+0000","last_undegraded":"2024-04-25T00:02:21.002416+0000","last_fullsized":"2024-04-25T00:02:21.002416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.016174+0000","last_change":"2024-04-25T00:02:21.015322+0000","last_active":"2024-04-25T00:02:21.016174+0000","last_peered":"2024-04-25T00:02:21.016174+0000","last_clean":"2024-04-25T00:02:21.016174+0000","last_became_active":"2024-04-25T00:02:21.014852+0000","last_became_peered":"2024-04-25T00:02:21.014852+0000","last_unstale":"2024-04-25T00:02:21.016174+0000","last_undegraded":"2024-04-25T00:02:21.016174+0000","last_fullsized":"2024-04-25T00:02:21.016174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:56.062217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.007261+0000","last_change":"2024-04-25T00:02:21.007261+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.007261+0000","last_undegraded":"2024-04-25T00:02:21.007261+0000","last_fullsized":"2024-04-25T00:02:21.007261+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.009434+0000","last_change":"2024-04-25T00:02:21.007937+0000","last_active":"2024-04-25T00:02:21.009434+0000","last_peered":"2024-04-25T00:02:21.009434+0000","last_clean":"2024-04-25T00:02:21.009434+0000","last_became_active":"2024-04-25T00:02:21.007605+0000","last_became_peered":"2024-04-25T00:02:21.007605+0000","last_unstale":"2024-04-25T00:02:21.009434+0000","last_undegraded":"2024-04-25T00:02:21.009434+0000","last_fullsized":"2024-04-25T00:02:21.009434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:19.375987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004626+0000","last_change":"2024-04-25T00:02:21.004626+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004626+0000","last_undegraded":"2024-04-25T00:02:21.004626+0000","last_fullsized":"2024-04-25T00:02:21.004626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013944+0000","last_change":"2024-04-25T00:02:21.013272+0000","last_active":"2024-04-25T00:02:21.013944+0000","last_peered":"2024-04-25T00:02:21.013944+0000","last_clean":"2024-04-25T00:02:21.013944+0000","last_became_active":"2024-04-25T00:02:21.012935+0000","last_became_peered":"2024-04-25T00:02:21.012935+0000","last_unstale":"2024-04-25T00:02:21.013944+0000","last_undegraded":"2024-04-25T00:02:21.013944+0000","last_fullsized":"2024-04-25T00:02:21.013944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:49.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.005277+0000","last_change":"2024-04-25T00:02:21.005277+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.005277+0000","last_undegraded":"2024-04-25T00:02:21.005277+0000","last_fullsized":"2024-04-25T00:02:21.005277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011879+0000","last_change":"2024-04-25T00:02:21.010717+0000","last_active":"2024-04-25T00:02:21.011879+0000","last_peered":"2024-04-25T00:02:21.011879+0000","last_clean":"2024-04-25T00:02:21.011879+0000","last_became_active":"2024-04-25T00:02:21.010435+0000","last_became_peered":"2024-04-25T00:02:21.010435+0000","last_unstale":"2024-04-25T00:02:21.011879+0000","last_undegraded":"2024-04-25T00:02:21.011879+0000","last_fullsized":"2024-04-25T00:02:21.011879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:58.591352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:20.997469+0000","last_change":"2024-04-25T00:02:20.997469+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:20.997469+0000","last_undegraded":"2024-04-25T00:02:20.997469+0000","last_fullsized":"2024-04-25T00:02:20.997469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014893+0000","last_change":"2024-04-25T00:02:21.012474+0000","last_active":"2024-04-25T00:02:21.014893+0000","last_peered":"2024-04-25T00:02:21.014893+0000","last_clean":"2024-04-25T00:02:21.014893+0000","last_became_active":"2024-04-25T00:02:21.012207+0000","last_became_peered":"2024-04-25T00:02:21.012207+0000","last_unstale":"2024-04-25T00:02:21.014893+0000","last_undegraded":"2024-04-25T00:02:21.014893+0000","last_fullsized":"2024-04-25T00:02:21.014893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:29.100886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003155+0000","last_change":"2024-04-25T00:02:21.003155+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003155+0000","last_undegraded":"2024-04-25T00:02:21.003155+0000","last_fullsized":"2024-04-25T00:02:21.003155+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011553+0000","last_change":"2024-04-25T00:02:21.009993+0000","last_active":"2024-04-25T00:02:21.011553+0000","last_peered":"2024-04-25T00:02:21.011553+0000","last_clean":"2024-04-25T00:02:21.011553+0000","last_became_active":"2024-04-25T00:02:21.009614+0000","last_became_peered":"2024-04-25T00:02:21.009614+0000","last_unstale":"2024-04-25T00:02:21.011553+0000","last_undegraded":"2024-04-25T00:02:21.011553+0000","last_fullsized":"2024-04-25T00:02:21.011553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:34.685171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003163+0000","last_change":"2024-04-25T00:02:21.003163+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003163+0000","last_undegraded":"2024-04-25T00:02:21.003163+0000","last_fullsized":"2024-04-25T00:02:21.003163+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012000+0000","last_change":"2024-04-25T00:02:21.010823+0000","last_active":"2024-04-25T00:02:21.012000+0000","last_peered":"2024-04-25T00:02:21.012000+0000","last_clean":"2024-04-25T00:02:21.012000+0000","last_became_active":"2024-04-25T00:02:21.009880+0000","last_became_peered":"2024-04-25T00:02:21.009880+0000","last_unstale":"2024-04-25T00:02:21.012000+0000","last_undegraded":"2024-04-25T00:02:21.012000+0000","last_fullsized":"2024-04-25T00:02:21.012000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:41.022670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.002260+0000","last_change":"2024-04-25T00:02:21.002260+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.002260+0000","last_undegraded":"2024-04-25T00:02:21.002260+0000","last_fullsized":"2024-04-25T00:02:21.002260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.008835+0000","last_change":"2024-04-25T00:02:21.003729+0000","last_active":"2024-04-25T00:02:21.008835+0000","last_peered":"2024-04-25T00:02:21.008835+0000","last_clean":"2024-04-25T00:02:21.008835+0000","last_became_active":"2024-04-25T00:02:21.003453+0000","last_became_peered":"2024-04-25T00:02:21.003453+0000","last_unstale":"2024-04-25T00:02:21.008835+0000","last_undegraded":"2024-04-25T00:02:21.008835+0000","last_fullsized":"2024-04-25T00:02:21.008835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:58.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.004996+0000","last_change":"2024-04-25T00:02:21.004996+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.004996+0000","last_undegraded":"2024-04-25T00:02:21.004996+0000","last_fullsized":"2024-04-25T00:02:21.004996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.012025+0000","last_change":"2024-04-25T00:02:21.010404+0000","last_active":"2024-04-25T00:02:21.012025+0000","last_peered":"2024-04-25T00:02:21.012025+0000","last_clean":"2024-04-25T00:02:21.012025+0000","last_became_active":"2024-04-25T00:02:21.010105+0000","last_became_peered":"2024-04-25T00:02:21.010105+0000","last_unstale":"2024-04-25T00:02:21.012025+0000","last_undegraded":"2024-04-25T00:02:21.012025+0000","last_fullsized":"2024-04-25T00:02:21.012025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:29.720152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003057+0000","last_change":"2024-04-25T00:02:21.003057+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003057+0000","last_undegraded":"2024-04-25T00:02:21.003057+0000","last_fullsized":"2024-04-25T00:02:21.003057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013880+0000","last_change":"2024-04-25T00:02:21.011175+0000","last_active":"2024-04-25T00:02:21.013880+0000","last_peered":"2024-04-25T00:02:21.013880+0000","last_clean":"2024-04-25T00:02:21.013880+0000","last_became_active":"2024-04-25T00:02:21.010864+0000","last_became_peered":"2024-04-25T00:02:21.010864+0000","last_unstale":"2024-04-25T00:02:21.013880+0000","last_undegraded":"2024-04-25T00:02:21.013880+0000","last_fullsized":"2024-04-25T00:02:21.013880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:34.531368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.000723+0000","last_change":"2024-04-25T00:02:21.000723+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.000723+0000","last_undegraded":"2024-04-25T00:02:21.000723+0000","last_fullsized":"2024-04-25T00:02:21.000723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.014057+0000","last_change":"2024-04-25T00:02:21.009271+0000","last_active":"2024-04-25T00:02:21.014057+0000","last_peered":"2024-04-25T00:02:21.014057+0000","last_clean":"2024-04-25T00:02:21.014057+0000","last_became_active":"2024-04-25T00:02:21.009010+0000","last_became_peered":"2024-04-25T00:02:21.009010+0000","last_unstale":"2024-04-25T00:02:21.014057+0000","last_undegraded":"2024-04-25T00:02:21.014057+0000","last_fullsized":"2024-04-25T00:02:21.014057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:07.487563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.003376+0000","last_change":"2024-04-25T00:02:21.003376+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.003376+0000","last_undegraded":"2024-04-25T00:02:21.003376+0000","last_fullsized":"2024-04-25T00:02:21.003376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.011285+0000","last_change":"2024-04-25T00:02:21.009814+0000","last_active":"2024-04-25T00:02:21.011285+0000","last_peered":"2024-04-25T00:02:21.011285+0000","last_clean":"2024-04-25T00:02:21.011285+0000","last_became_active":"2024-04-25T00:02:21.009197+0000","last_became_peered":"2024-04-25T00:02:21.009197+0000","last_unstale":"2024-04-25T00:02:21.011285+0000","last_undegraded":"2024-04-25T00:02:21.011285+0000","last_fullsized":"2024-04-25T00:02:21.011285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:19.681462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T00:02:21.006708+0000","last_change":"2024-04-25T00:02:21.006708+0000","last_active":"2024-04-25T00:02:20.985123+0000","last_peered":"2024-04-25T00:02:20.985123+0000","last_clean":"2024-04-25T00:02:20.985123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:02:21.006708+0000","last_undegraded":"2024-04-25T00:02:21.006708+0000","last_fullsized":"2024-04-25T00:02:21.006708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.013290+0000","last_change":"2024-04-25T00:02:21.011072+0000","last_active":"2024-04-25T00:02:21.013290+0000","last_peered":"2024-04-25T00:02:21.013290+0000","last_clean":"2024-04-25T00:02:21.013290+0000","last_became_active":"2024-04-25T00:02:21.010789+0000","last_became_peered":"2024-04-25T00:02:21.010789+0000","last_unstale":"2024-04-25T00:02:21.013290+0000","last_undegraded":"2024-04-25T00:02:21.013290+0000","last_fullsized":"2024-04-25T00:02:21.013290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:45.234560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T00:02:21.004389+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:02:21.004389+0000","last_peered":"2024-04-25T00:02:21.004389+0000","last_clean":"2024-04-25T00:02:21.004389+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:02:21.004389+0000","last_undegraded":"2024-04-25T00:02:21.004389+0000","last_fullsized":"2024-04-25T00:02:21.004389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":901120,"data_stored":382102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335628,"statfs":{"total":96636764160,"available":96599683072,"internally_reserved":0,"allocated":888832,"data_stored":375827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1482752,"data_stored":966195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:02:27.255 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:02:27.255+0000 7f1032662640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:02:29.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:02:29.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.444+0000 7f7a233c4640 1 Processor -- start 2024-04-25T00:02:29.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.444+0000 7f7a233c4640 1 -- start start 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a233c4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 0x7f7a1c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a233c4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a1c0b2d50 con 0x7f7a1c15a6b0 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a233c4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a1c072a70 con 0x7f7a1c0b3af0 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a233c4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1c0b3010 con 0x7f7a1c072ea0 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a21160640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7a1c15a6b0 0x7f7a1c070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52892/0 (socket says 172.21.15.113:52892) 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a21160640 1 -- 172.21.15.113:0/2748224386 learned_addr learned my addr 172.21.15.113:0/2748224386 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103115791 0 0) 0x7f7a1c0b2d50 con 0x7f7a1c15a6b0 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a08003520 con 0x7f7a1c15a6b0 2024-04-25T00:02:29.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a2095f640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 0x7f7a1c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1002671002 0 0) 0x7f7a08003520 con 0x7f7a1c15a6b0 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 msgr2=0x7f7a1c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a13fff640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 0x7f7a1c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.445+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 >> v1:172.21.15.178:6789/0 conn(0x7f7a1c0b3af0 legacy=0x7f7a1c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c0b32f0 con 0x7f7a1c15a6b0 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f7a18001670 con 0x7f7a1c15a6b0 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 >> v1:172.21.15.113:6789/0 conn(0x7f7a1c15a6b0 legacy=0x7f7a1c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a080039e0 0x7f7a08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 0x7f7a08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a08004d40 0x7f7a080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c0b3010 con 0x7f7a080039e0 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0800bb50 con 0x7f7a08004d40 2024-04-25T00:02:29.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0800bd20 con 0x7f7a1c072ea0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a21160640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a080039e0 0x7f7a08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a2095f640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 0x7f7a08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a21961640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a08004d40 0x7f7a080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a21160640 1 -- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 msgr2=0x7f7a08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a21160640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c072ea0 0x7f7a08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a21160640 1 -- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a08004d40 msgr2=0x7f7a080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a21160640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a08004d40 0x7f7a080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.446+0000 7f7a21160640 1 -- 172.21.15.113:0/2748224386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a08003520 con 0x7f7a080039e0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a21160640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a080039e0 0x7f7a08003e00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c270 tx=0x7f7a08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6228fb435895da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18007cf0 con 0x7f7a080039e0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a18005af0 con 0x7f7a080039e0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a13fff640 1 -- 172.21.15.113:0/2748224386 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18007070 con 0x7f7a080039e0 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a233c4640 1 -- 172.21.15.113:0/2748224386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a080039e0 msgr2=0x7f7a08003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a233c4640 1 --2- 172.21.15.113:0/2748224386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a080039e0 0x7f7a08003e00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c270 tx=0x7f7a08005830 comp rx=0 tx=0).stop 2024-04-25T00:02:29.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a21961640 1 -- 172.21.15.113:0/2748224386 reap_dead start 2024-04-25T00:02:29.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a233c4640 1 -- 172.21.15.113:0/2748224386 shutdown_connections 2024-04-25T00:02:29.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.447+0000 7f7a233c4640 1 -- 172.21.15.113:0/2748224386 >> 172.21.15.113:0/2748224386 conn(0x7f7a1c0b1a50 msgr2=0x7f7a1c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:29.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.448+0000 7f7a233c4640 1 -- 172.21.15.113:0/2748224386 shutdown_connections 2024-04-25T00:02:29.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.448+0000 7f7a233c4640 1 -- 172.21.15.113:0/2748224386 wait complete. 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 Processor -- start 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 -- start start 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a080039e0 0x7f7a1c148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a1c169080 0x7f7a1c1f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c1f5bb0 0x7f7a1c1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c15aa90 con 0x7f7a1c169080 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c072720 con 0x7f7a080039e0 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.449+0000 7f7a233c4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1c15ac10 con 0x7f7a1c1f5bb0 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a1c169080 0x7f7a1c1f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a1c169080 0x7f7a1c1f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40092/0 (socket says 172.21.15.113:40092) 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 -- 172.21.15.113:0/3837351506 learned_addr learned my addr 172.21.15.113:0/3837351506 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:29.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a21160640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a080039e0 0x7f7a1c148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a21961640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c1f5bb0 0x7f7a1c1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 -- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c1f5bb0 msgr2=0x7f7a1c1f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c1f5bb0 0x7f7a1c1f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 -- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a080039e0 msgr2=0x7f7a1c148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a080039e0 0x7f7a1c148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 -- 172.21.15.113:0/3837351506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c155800 con 0x7f7a1c169080 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a21160640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a080039e0 0x7f7a1c148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a2095f640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a1c169080 0x7f7a1c1f5670 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00b7d0 tx=0x7f7a0c00bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a11ffb640 1 -- 172.21.15.113:0/3837351506 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c018070 con 0x7f7a1c169080 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a11ffb640 1 -- 172.21.15.113:0/3837351506 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a0c0107d0 con 0x7f7a1c169080 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a11ffb640 1 -- 172.21.15.113:0/3837351506 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c010ba0 con 0x7f7a1c169080 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.450+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c155ae0 con 0x7f7a1c169080 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.451+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c151300 con 0x7f7a1c169080 2024-04-25T00:02:29.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.451+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c155570 con 0x7f7a1c169080 2024-04-25T00:02:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.452+0000 7f7a11ffb640 1 -- 172.21.15.113:0/3837351506 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a0c01b070 con 0x7f7a1c169080 2024-04-25T00:02:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.452+0000 7f7a11ffb640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a0006be30 0x7f7a0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.452+0000 7f7a11ffb640 1 -- 172.21.15.113:0/3837351506 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f7a0c08b1a0 con 0x7f7a1c169080 2024-04-25T00:02:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.452+0000 7f7a21160640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a0006be30 0x7f7a0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.453+0000 7f7a21160640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a0006be30 0x7f7a0006e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7a18033000 tx=0x7f7a18009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.457+0000 7f7a11ffb640 1 -- 172.21.15.113:0/3837351506 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a0c059b50 con 0x7f7a1c169080 2024-04-25T00:02:29.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.628+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a1c0691a0 con 0x7f7a0006be30 2024-04-25T00:02:29.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.633+0000 7f7a11ffb640 1 -- 172.21.15.113:0/3837351506 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f7a1c0691a0 con 0x7f7a0006be30 2024-04-25T00:02:29.637 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:29.639 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:02:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a0006be30 msgr2=0x7f7a0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a0006be30 0x7f7a0006e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7a18033000 tx=0x7f7a18009030 comp rx=0 tx=0).stop 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a1c169080 msgr2=0x7f7a1c1f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a1c169080 0x7f7a1c1f5670 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00b7d0 tx=0x7f7a0c00bc90 comp rx=0 tx=0).stop 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 shutdown_connections 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a0006be30 0x7f7a0006e2f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c1f5bb0 0x7f7a1c1f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a1c169080 0x7f7a1c1f5670 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 --2- 172.21.15.113:0/3837351506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a080039e0 0x7f7a1c148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 >> 172.21.15.113:0/3837351506 conn(0x7f7a1c0b1a50 msgr2=0x7f7a1c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 shutdown_connections 2024-04-25T00:02:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.637+0000 7f7a233c4640 1 -- 172.21.15.113:0/3837351506 wait complete. 2024-04-25T00:02:29.648 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-04-25T00:02:29.161040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274584,"kb_used_data":8456,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700136,"statfs":{"total":773094113280,"available":772812939264,"internally_reserved":0,"allocated":8658944,"data_stored":4346972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001773"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509617+0000","last_change":"2024-04-25T00:02:22.016902+0000","last_active":"2024-04-25T00:02:23.509617+0000","last_peered":"2024-04-25T00:02:23.509617+0000","last_clean":"2024-04-25T00:02:23.509617+0000","last_became_active":"2024-04-25T00:02:22.016591+0000","last_became_peered":"2024-04-25T00:02:22.016591+0000","last_unstale":"2024-04-25T00:02:23.509617+0000","last_undegraded":"2024-04-25T00:02:23.509617+0000","last_fullsized":"2024-04-25T00:02:23.509617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:00.155148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510228+0000","last_change":"2024-04-25T00:02:22.017585+0000","last_active":"2024-04-25T00:02:23.510228+0000","last_peered":"2024-04-25T00:02:23.510228+0000","last_clean":"2024-04-25T00:02:23.510228+0000","last_became_active":"2024-04-25T00:02:22.016839+0000","last_became_peered":"2024-04-25T00:02:22.016839+0000","last_unstale":"2024-04-25T00:02:23.510228+0000","last_undegraded":"2024-04-25T00:02:23.510228+0000","last_fullsized":"2024-04-25T00:02:23.510228+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:23.760489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508792+0000","last_change":"2024-04-25T00:02:22.001929+0000","last_active":"2024-04-25T00:02:23.508792+0000","last_peered":"2024-04-25T00:02:23.508792+0000","last_clean":"2024-04-25T00:02:23.508792+0000","last_became_active":"2024-04-25T00:02:22.001588+0000","last_became_peered":"2024-04-25T00:02:22.001588+0000","last_unstale":"2024-04-25T00:02:23.508792+0000","last_undegraded":"2024-04-25T00:02:23.508792+0000","last_fullsized":"2024-04-25T00:02:23.508792+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:59.753282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007243+0000","last_change":"2024-04-25T00:02:22.013443+0000","last_active":"2024-04-25T00:02:23.007243+0000","last_peered":"2024-04-25T00:02:23.007243+0000","last_clean":"2024-04-25T00:02:23.007243+0000","last_became_active":"2024-04-25T00:02:22.013180+0000","last_became_peered":"2024-04-25T00:02:22.013180+0000","last_unstale":"2024-04-25T00:02:23.007243+0000","last_undegraded":"2024-04-25T00:02:23.007243+0000","last_fullsized":"2024-04-25T00:02:23.007243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:47.450016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202342+0000","last_change":"2024-04-25T00:02:22.013395+0000","last_active":"2024-04-25T00:02:23.202342+0000","last_peered":"2024-04-25T00:02:23.202342+0000","last_clean":"2024-04-25T00:02:23.202342+0000","last_became_active":"2024-04-25T00:02:22.013099+0000","last_became_peered":"2024-04-25T00:02:22.013099+0000","last_unstale":"2024-04-25T00:02:23.202342+0000","last_undegraded":"2024-04-25T00:02:23.202342+0000","last_fullsized":"2024-04-25T00:02:23.202342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:55.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509596+0000","last_change":"2024-04-25T00:02:22.016664+0000","last_active":"2024-04-25T00:02:23.509596+0000","last_peered":"2024-04-25T00:02:23.509596+0000","last_clean":"2024-04-25T00:02:23.509596+0000","last_became_active":"2024-04-25T00:02:22.015919+0000","last_became_peered":"2024-04-25T00:02:22.015919+0000","last_unstale":"2024-04-25T00:02:23.509596+0000","last_undegraded":"2024-04-25T00:02:23.509596+0000","last_fullsized":"2024-04-25T00:02:23.509596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:07.891904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013631+0000","last_change":"2024-04-25T00:02:22.011167+0000","last_active":"2024-04-25T00:02:23.013631+0000","last_peered":"2024-04-25T00:02:23.013631+0000","last_clean":"2024-04-25T00:02:23.013631+0000","last_became_active":"2024-04-25T00:02:22.010869+0000","last_became_peered":"2024-04-25T00:02:22.010869+0000","last_unstale":"2024-04-25T00:02:23.013631+0000","last_undegraded":"2024-04-25T00:02:23.013631+0000","last_fullsized":"2024-04-25T00:02:23.013631+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:47.575753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510147+0000","last_change":"2024-04-25T00:02:22.020363+0000","last_active":"2024-04-25T00:02:23.510147+0000","last_peered":"2024-04-25T00:02:23.510147+0000","last_clean":"2024-04-25T00:02:23.510147+0000","last_became_active":"2024-04-25T00:02:22.020088+0000","last_became_peered":"2024-04-25T00:02:22.020088+0000","last_unstale":"2024-04-25T00:02:23.510147+0000","last_undegraded":"2024-04-25T00:02:23.510147+0000","last_fullsized":"2024-04-25T00:02:23.510147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:07.594981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202629+0000","last_change":"2024-04-25T00:02:22.028756+0000","last_active":"2024-04-25T00:02:23.202629+0000","last_peered":"2024-04-25T00:02:23.202629+0000","last_clean":"2024-04-25T00:02:23.202629+0000","last_became_active":"2024-04-25T00:02:22.028439+0000","last_became_peered":"2024-04-25T00:02:22.028439+0000","last_unstale":"2024-04-25T00:02:23.202629+0000","last_undegraded":"2024-04-25T00:02:23.202629+0000","last_fullsized":"2024-04-25T00:02:23.202629+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:10.445014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008044+0000","last_change":"2024-04-25T00:02:22.023743+0000","last_active":"2024-04-25T00:02:23.008044+0000","last_peered":"2024-04-25T00:02:23.008044+0000","last_clean":"2024-04-25T00:02:23.008044+0000","last_became_active":"2024-04-25T00:02:22.023387+0000","last_became_peered":"2024-04-25T00:02:22.023387+0000","last_unstale":"2024-04-25T00:02:23.008044+0000","last_undegraded":"2024-04-25T00:02:23.008044+0000","last_fullsized":"2024-04-25T00:02:23.008044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:40.654985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509562+0000","last_change":"2024-04-25T00:02:22.004707+0000","last_active":"2024-04-25T00:02:23.509562+0000","last_peered":"2024-04-25T00:02:23.509562+0000","last_clean":"2024-04-25T00:02:23.509562+0000","last_became_active":"2024-04-25T00:02:22.004400+0000","last_became_peered":"2024-04-25T00:02:22.004400+0000","last_unstale":"2024-04-25T00:02:23.509562+0000","last_undegraded":"2024-04-25T00:02:23.509562+0000","last_fullsized":"2024-04-25T00:02:23.509562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:51.086980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007625+0000","last_change":"2024-04-25T00:02:22.015718+0000","last_active":"2024-04-25T00:02:23.007625+0000","last_peered":"2024-04-25T00:02:23.007625+0000","last_clean":"2024-04-25T00:02:23.007625+0000","last_became_active":"2024-04-25T00:02:22.015262+0000","last_became_peered":"2024-04-25T00:02:22.015262+0000","last_unstale":"2024-04-25T00:02:23.007625+0000","last_undegraded":"2024-04-25T00:02:23.007625+0000","last_fullsized":"2024-04-25T00:02:23.007625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:24.295392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007575+0000","last_change":"2024-04-25T00:02:22.012790+0000","last_active":"2024-04-25T00:02:23.007575+0000","last_peered":"2024-04-25T00:02:23.007575+0000","last_clean":"2024-04-25T00:02:23.007575+0000","last_became_active":"2024-04-25T00:02:22.012517+0000","last_became_peered":"2024-04-25T00:02:22.012517+0000","last_unstale":"2024-04-25T00:02:23.007575+0000","last_undegraded":"2024-04-25T00:02:23.007575+0000","last_fullsized":"2024-04-25T00:02:23.007575+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:26.207063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202480+0000","last_change":"2024-04-25T00:02:22.015309+0000","last_active":"2024-04-25T00:02:23.202480+0000","last_peered":"2024-04-25T00:02:23.202480+0000","last_clean":"2024-04-25T00:02:23.202480+0000","last_became_active":"2024-04-25T00:02:22.014994+0000","last_became_peered":"2024-04-25T00:02:22.014994+0000","last_unstale":"2024-04-25T00:02:23.202480+0000","last_undegraded":"2024-04-25T00:02:23.202480+0000","last_fullsized":"2024-04-25T00:02:23.202480+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:48.663924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007160+0000","last_change":"2024-04-25T00:02:22.017469+0000","last_active":"2024-04-25T00:02:23.007160+0000","last_peered":"2024-04-25T00:02:23.007160+0000","last_clean":"2024-04-25T00:02:23.007160+0000","last_became_active":"2024-04-25T00:02:22.017218+0000","last_became_peered":"2024-04-25T00:02:22.017218+0000","last_unstale":"2024-04-25T00:02:23.007160+0000","last_undegraded":"2024-04-25T00:02:23.007160+0000","last_fullsized":"2024-04-25T00:02:23.007160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:06.277373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007577+0000","last_change":"2024-04-25T00:02:22.021707+0000","last_active":"2024-04-25T00:02:23.007577+0000","last_peered":"2024-04-25T00:02:23.007577+0000","last_clean":"2024-04-25T00:02:23.007577+0000","last_became_active":"2024-04-25T00:02:22.021449+0000","last_became_peered":"2024-04-25T00:02:22.021449+0000","last_unstale":"2024-04-25T00:02:23.007577+0000","last_undegraded":"2024-04-25T00:02:23.007577+0000","last_fullsized":"2024-04-25T00:02:23.007577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:35.691909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202671+0000","last_change":"2024-04-25T00:02:22.014422+0000","last_active":"2024-04-25T00:02:23.202671+0000","last_peered":"2024-04-25T00:02:23.202671+0000","last_clean":"2024-04-25T00:02:23.202671+0000","last_became_active":"2024-04-25T00:02:22.014092+0000","last_became_peered":"2024-04-25T00:02:22.014092+0000","last_unstale":"2024-04-25T00:02:23.202671+0000","last_undegraded":"2024-04-25T00:02:23.202671+0000","last_fullsized":"2024-04-25T00:02:23.202671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:26.379101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007216+0000","last_change":"2024-04-25T00:02:22.012718+0000","last_active":"2024-04-25T00:02:23.007216+0000","last_peered":"2024-04-25T00:02:23.007216+0000","last_clean":"2024-04-25T00:02:23.007216+0000","last_became_active":"2024-04-25T00:02:22.012483+0000","last_became_peered":"2024-04-25T00:02:22.012483+0000","last_unstale":"2024-04-25T00:02:23.007216+0000","last_undegraded":"2024-04-25T00:02:23.007216+0000","last_fullsized":"2024-04-25T00:02:23.007216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:36.763450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509581+0000","last_change":"2024-04-25T00:02:22.012565+0000","last_active":"2024-04-25T00:02:23.509581+0000","last_peered":"2024-04-25T00:02:23.509581+0000","last_clean":"2024-04-25T00:02:23.509581+0000","last_became_active":"2024-04-25T00:02:22.012298+0000","last_became_peered":"2024-04-25T00:02:22.012298+0000","last_unstale":"2024-04-25T00:02:23.509581+0000","last_undegraded":"2024-04-25T00:02:23.509581+0000","last_fullsized":"2024-04-25T00:02:23.509581+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:44.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007488+0000","last_change":"2024-04-25T00:02:22.010272+0000","last_active":"2024-04-25T00:02:23.007488+0000","last_peered":"2024-04-25T00:02:23.007488+0000","last_clean":"2024-04-25T00:02:23.007488+0000","last_became_active":"2024-04-25T00:02:22.009998+0000","last_became_peered":"2024-04-25T00:02:22.009998+0000","last_unstale":"2024-04-25T00:02:23.007488+0000","last_undegraded":"2024-04-25T00:02:23.007488+0000","last_fullsized":"2024-04-25T00:02:23.007488+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:34.328805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007853+0000","last_change":"2024-04-25T00:02:22.015221+0000","last_active":"2024-04-25T00:02:23.007853+0000","last_peered":"2024-04-25T00:02:23.007853+0000","last_clean":"2024-04-25T00:02:23.007853+0000","last_became_active":"2024-04-25T00:02:22.014976+0000","last_became_peered":"2024-04-25T00:02:22.014976+0000","last_unstale":"2024-04-25T00:02:23.007853+0000","last_undegraded":"2024-04-25T00:02:23.007853+0000","last_fullsized":"2024-04-25T00:02:23.007853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:33.204614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007817+0000","last_change":"2024-04-25T00:02:22.025473+0000","last_active":"2024-04-25T00:02:23.007817+0000","last_peered":"2024-04-25T00:02:23.007817+0000","last_clean":"2024-04-25T00:02:23.007817+0000","last_became_active":"2024-04-25T00:02:22.025073+0000","last_became_peered":"2024-04-25T00:02:22.025073+0000","last_unstale":"2024-04-25T00:02:23.007817+0000","last_undegraded":"2024-04-25T00:02:23.007817+0000","last_fullsized":"2024-04-25T00:02:23.007817+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:50.928945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509112+0000","last_change":"2024-04-25T00:02:22.007202+0000","last_active":"2024-04-25T00:02:23.509112+0000","last_peered":"2024-04-25T00:02:23.509112+0000","last_clean":"2024-04-25T00:02:23.509112+0000","last_became_active":"2024-04-25T00:02:22.006945+0000","last_became_peered":"2024-04-25T00:02:22.006945+0000","last_unstale":"2024-04-25T00:02:23.509112+0000","last_undegraded":"2024-04-25T00:02:23.509112+0000","last_fullsized":"2024-04-25T00:02:23.509112+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:55.354601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509020+0000","last_change":"2024-04-25T00:02:22.013713+0000","last_active":"2024-04-25T00:02:23.509020+0000","last_peered":"2024-04-25T00:02:23.509020+0000","last_clean":"2024-04-25T00:02:23.509020+0000","last_became_active":"2024-04-25T00:02:22.013453+0000","last_became_peered":"2024-04-25T00:02:22.013453+0000","last_unstale":"2024-04-25T00:02:23.509020+0000","last_undegraded":"2024-04-25T00:02:23.509020+0000","last_fullsized":"2024-04-25T00:02:23.509020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:39.569110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008227+0000","last_change":"2024-04-25T00:02:22.015235+0000","last_active":"2024-04-25T00:02:23.008227+0000","last_peered":"2024-04-25T00:02:23.008227+0000","last_clean":"2024-04-25T00:02:23.008227+0000","last_became_active":"2024-04-25T00:02:22.015015+0000","last_became_peered":"2024-04-25T00:02:22.015015+0000","last_unstale":"2024-04-25T00:02:23.008227+0000","last_undegraded":"2024-04-25T00:02:23.008227+0000","last_fullsized":"2024-04-25T00:02:23.008227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:00.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.014012+0000","last_change":"2024-04-25T00:02:22.015309+0000","last_active":"2024-04-25T00:02:23.014012+0000","last_peered":"2024-04-25T00:02:23.014012+0000","last_clean":"2024-04-25T00:02:23.014012+0000","last_became_active":"2024-04-25T00:02:22.015062+0000","last_became_peered":"2024-04-25T00:02:22.015062+0000","last_unstale":"2024-04-25T00:02:23.014012+0000","last_undegraded":"2024-04-25T00:02:23.014012+0000","last_fullsized":"2024-04-25T00:02:23.014012+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:39.569110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508820+0000","last_change":"2024-04-25T00:02:22.006835+0000","last_active":"2024-04-25T00:02:23.508820+0000","last_peered":"2024-04-25T00:02:23.508820+0000","last_clean":"2024-04-25T00:02:23.508820+0000","last_became_active":"2024-04-25T00:02:22.006570+0000","last_became_peered":"2024-04-25T00:02:22.006570+0000","last_unstale":"2024-04-25T00:02:23.508820+0000","last_undegraded":"2024-04-25T00:02:23.508820+0000","last_fullsized":"2024-04-25T00:02:23.508820+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:36.903401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008413+0000","last_change":"2024-04-25T00:02:22.023906+0000","last_active":"2024-04-25T00:02:23.008413+0000","last_peered":"2024-04-25T00:02:23.008413+0000","last_clean":"2024-04-25T00:02:23.008413+0000","last_became_active":"2024-04-25T00:02:22.023268+0000","last_became_peered":"2024-04-25T00:02:22.023268+0000","last_unstale":"2024-04-25T00:02:23.008413+0000","last_undegraded":"2024-04-25T00:02:23.008413+0000","last_fullsized":"2024-04-25T00:02:23.008413+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:01.962421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508825+0000","last_change":"2024-04-25T00:02:22.006450+0000","last_active":"2024-04-25T00:02:23.508825+0000","last_peered":"2024-04-25T00:02:23.508825+0000","last_clean":"2024-04-25T00:02:23.508825+0000","last_became_active":"2024-04-25T00:02:22.005809+0000","last_became_peered":"2024-04-25T00:02:22.005809+0000","last_unstale":"2024-04-25T00:02:23.508825+0000","last_undegraded":"2024-04-25T00:02:23.508825+0000","last_fullsized":"2024-04-25T00:02:23.508825+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:45.045511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013680+0000","last_change":"2024-04-25T00:02:22.014706+0000","last_active":"2024-04-25T00:02:23.013680+0000","last_peered":"2024-04-25T00:02:23.013680+0000","last_clean":"2024-04-25T00:02:23.013680+0000","last_became_active":"2024-04-25T00:02:22.014427+0000","last_became_peered":"2024-04-25T00:02:22.014427+0000","last_unstale":"2024-04-25T00:02:23.013680+0000","last_undegraded":"2024-04-25T00:02:23.013680+0000","last_fullsized":"2024-04-25T00:02:23.013680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:33.502411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013848+0000","last_change":"2024-04-25T00:02:22.017488+0000","last_active":"2024-04-25T00:02:23.013848+0000","last_peered":"2024-04-25T00:02:23.013848+0000","last_clean":"2024-04-25T00:02:23.013848+0000","last_became_active":"2024-04-25T00:02:22.017180+0000","last_became_peered":"2024-04-25T00:02:22.017180+0000","last_unstale":"2024-04-25T00:02:23.013848+0000","last_undegraded":"2024-04-25T00:02:23.013848+0000","last_fullsized":"2024-04-25T00:02:23.013848+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:08.744763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510085+0000","last_change":"2024-04-25T00:02:22.013398+0000","last_active":"2024-04-25T00:02:23.510085+0000","last_peered":"2024-04-25T00:02:23.510085+0000","last_clean":"2024-04-25T00:02:23.510085+0000","last_became_active":"2024-04-25T00:02:22.013147+0000","last_became_peered":"2024-04-25T00:02:22.013147+0000","last_unstale":"2024-04-25T00:02:23.510085+0000","last_undegraded":"2024-04-25T00:02:23.510085+0000","last_fullsized":"2024-04-25T00:02:23.510085+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:49.288143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509874+0000","last_change":"2024-04-25T00:02:22.012997+0000","last_active":"2024-04-25T00:02:23.509874+0000","last_peered":"2024-04-25T00:02:23.509874+0000","last_clean":"2024-04-25T00:02:23.509874+0000","last_became_active":"2024-04-25T00:02:22.012745+0000","last_became_peered":"2024-04-25T00:02:22.012745+0000","last_unstale":"2024-04-25T00:02:23.509874+0000","last_undegraded":"2024-04-25T00:02:23.509874+0000","last_fullsized":"2024-04-25T00:02:23.509874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:34.464895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"75'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509941+0000","last_change":"2024-04-25T00:02:21.012893+0000","last_active":"2024-04-25T00:02:23.509941+0000","last_peered":"2024-04-25T00:02:23.509941+0000","last_clean":"2024-04-25T00:02:23.509941+0000","last_became_active":"2024-04-25T00:02:21.012601+0000","last_became_peered":"2024-04-25T00:02:21.012601+0000","last_unstale":"2024-04-25T00:02:23.509941+0000","last_undegraded":"2024-04-25T00:02:23.509941+0000","last_fullsized":"2024-04-25T00:02:23.509941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:00.955682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202712+0000","last_change":"2024-04-25T00:02:22.014377+0000","last_active":"2024-04-25T00:02:23.202712+0000","last_peered":"2024-04-25T00:02:23.202712+0000","last_clean":"2024-04-25T00:02:23.202712+0000","last_became_active":"2024-04-25T00:02:22.013771+0000","last_became_peered":"2024-04-25T00:02:22.013771+0000","last_unstale":"2024-04-25T00:02:23.202712+0000","last_undegraded":"2024-04-25T00:02:23.202712+0000","last_fullsized":"2024-04-25T00:02:23.202712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:31.780079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007706+0000","last_change":"2024-04-25T00:02:21.011558+0000","last_active":"2024-04-25T00:02:23.007706+0000","last_peered":"2024-04-25T00:02:23.007706+0000","last_clean":"2024-04-25T00:02:23.007706+0000","last_became_active":"2024-04-25T00:02:21.011287+0000","last_became_peered":"2024-04-25T00:02:21.011287+0000","last_unstale":"2024-04-25T00:02:23.007706+0000","last_undegraded":"2024-04-25T00:02:23.007706+0000","last_fullsized":"2024-04-25T00:02:23.007706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:34.685171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510228+0000","last_change":"2024-04-25T00:02:22.011118+0000","last_active":"2024-04-25T00:02:23.510228+0000","last_peered":"2024-04-25T00:02:23.510228+0000","last_clean":"2024-04-25T00:02:23.510228+0000","last_became_active":"2024-04-25T00:02:22.010789+0000","last_became_peered":"2024-04-25T00:02:22.010789+0000","last_unstale":"2024-04-25T00:02:23.510228+0000","last_undegraded":"2024-04-25T00:02:23.510228+0000","last_fullsized":"2024-04-25T00:02:23.510228+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:53.728371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008029+0000","last_change":"2024-04-25T00:02:21.007131+0000","last_active":"2024-04-25T00:02:23.008029+0000","last_peered":"2024-04-25T00:02:23.008029+0000","last_clean":"2024-04-25T00:02:23.008029+0000","last_became_active":"2024-04-25T00:02:21.006720+0000","last_became_peered":"2024-04-25T00:02:21.006720+0000","last_unstale":"2024-04-25T00:02:23.008029+0000","last_undegraded":"2024-04-25T00:02:23.008029+0000","last_fullsized":"2024-04-25T00:02:23.008029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:48.281404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509721+0000","last_change":"2024-04-25T00:02:22.009451+0000","last_active":"2024-04-25T00:02:23.509721+0000","last_peered":"2024-04-25T00:02:23.509721+0000","last_clean":"2024-04-25T00:02:23.509721+0000","last_became_active":"2024-04-25T00:02:22.009163+0000","last_became_peered":"2024-04-25T00:02:22.009163+0000","last_unstale":"2024-04-25T00:02:23.509721+0000","last_undegraded":"2024-04-25T00:02:23.509721+0000","last_fullsized":"2024-04-25T00:02:23.509721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:51.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510373+0000","last_change":"2024-04-25T00:02:21.012185+0000","last_active":"2024-04-25T00:02:23.510373+0000","last_peered":"2024-04-25T00:02:23.510373+0000","last_clean":"2024-04-25T00:02:23.510373+0000","last_became_active":"2024-04-25T00:02:21.011910+0000","last_became_peered":"2024-04-25T00:02:21.011910+0000","last_unstale":"2024-04-25T00:02:23.510373+0000","last_undegraded":"2024-04-25T00:02:23.510373+0000","last_fullsized":"2024-04-25T00:02:23.510373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:06.563798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013833+0000","last_change":"2024-04-25T00:02:22.009809+0000","last_active":"2024-04-25T00:02:23.013833+0000","last_peered":"2024-04-25T00:02:23.013833+0000","last_clean":"2024-04-25T00:02:23.013833+0000","last_became_active":"2024-04-25T00:02:22.009545+0000","last_became_peered":"2024-04-25T00:02:22.009545+0000","last_unstale":"2024-04-25T00:02:23.013833+0000","last_undegraded":"2024-04-25T00:02:23.013833+0000","last_fullsized":"2024-04-25T00:02:23.013833+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:21.837983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508874+0000","last_change":"2024-04-25T00:02:21.015042+0000","last_active":"2024-04-25T00:02:23.508874+0000","last_peered":"2024-04-25T00:02:23.508874+0000","last_clean":"2024-04-25T00:02:23.508874+0000","last_became_active":"2024-04-25T00:02:21.014741+0000","last_became_peered":"2024-04-25T00:02:21.014741+0000","last_unstale":"2024-04-25T00:02:23.508874+0000","last_undegraded":"2024-04-25T00:02:23.508874+0000","last_fullsized":"2024-04-25T00:02:23.508874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:50.080241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008046+0000","last_change":"2024-04-25T00:02:22.014512+0000","last_active":"2024-04-25T00:02:23.008046+0000","last_peered":"2024-04-25T00:02:23.008046+0000","last_clean":"2024-04-25T00:02:23.008046+0000","last_became_active":"2024-04-25T00:02:22.014019+0000","last_became_peered":"2024-04-25T00:02:22.014019+0000","last_unstale":"2024-04-25T00:02:23.008046+0000","last_undegraded":"2024-04-25T00:02:23.008046+0000","last_fullsized":"2024-04-25T00:02:23.008046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:12.502530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509702+0000","last_change":"2024-04-25T00:02:21.008726+0000","last_active":"2024-04-25T00:02:23.509702+0000","last_peered":"2024-04-25T00:02:23.509702+0000","last_clean":"2024-04-25T00:02:23.509702+0000","last_became_active":"2024-04-25T00:02:21.008442+0000","last_became_peered":"2024-04-25T00:02:21.008442+0000","last_unstale":"2024-04-25T00:02:23.509702+0000","last_undegraded":"2024-04-25T00:02:23.509702+0000","last_fullsized":"2024-04-25T00:02:23.509702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:34.531368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202216+0000","last_change":"2024-04-25T00:02:22.020206+0000","last_active":"2024-04-25T00:02:23.202216+0000","last_peered":"2024-04-25T00:02:23.202216+0000","last_clean":"2024-04-25T00:02:23.202216+0000","last_became_active":"2024-04-25T00:02:22.019969+0000","last_became_peered":"2024-04-25T00:02:22.019969+0000","last_unstale":"2024-04-25T00:02:23.202216+0000","last_undegraded":"2024-04-25T00:02:23.202216+0000","last_fullsized":"2024-04-25T00:02:23.202216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:34.780745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.006748+0000","last_change":"2024-04-25T00:02:21.011239+0000","last_active":"2024-04-25T00:02:23.006748+0000","last_peered":"2024-04-25T00:02:23.006748+0000","last_clean":"2024-04-25T00:02:23.006748+0000","last_became_active":"2024-04-25T00:02:21.010824+0000","last_became_peered":"2024-04-25T00:02:21.010824+0000","last_unstale":"2024-04-25T00:02:23.006748+0000","last_undegraded":"2024-04-25T00:02:23.006748+0000","last_fullsized":"2024-04-25T00:02:23.006748+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:33.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007370+0000","last_change":"2024-04-25T00:02:22.001529+0000","last_active":"2024-04-25T00:02:23.007370+0000","last_peered":"2024-04-25T00:02:23.007370+0000","last_clean":"2024-04-25T00:02:23.007370+0000","last_became_active":"2024-04-25T00:02:22.001203+0000","last_became_peered":"2024-04-25T00:02:22.001203+0000","last_unstale":"2024-04-25T00:02:23.007370+0000","last_undegraded":"2024-04-25T00:02:23.007370+0000","last_fullsized":"2024-04-25T00:02:23.007370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:22.896815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202158+0000","last_change":"2024-04-25T00:02:21.013287+0000","last_active":"2024-04-25T00:02:23.202158+0000","last_peered":"2024-04-25T00:02:23.202158+0000","last_clean":"2024-04-25T00:02:23.202158+0000","last_became_active":"2024-04-25T00:02:21.013026+0000","last_became_peered":"2024-04-25T00:02:21.013026+0000","last_unstale":"2024-04-25T00:02:23.202158+0000","last_undegraded":"2024-04-25T00:02:23.202158+0000","last_fullsized":"2024-04-25T00:02:23.202158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:50.202267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.014038+0000","last_change":"2024-04-25T00:02:22.006215+0000","last_active":"2024-04-25T00:02:23.014038+0000","last_peered":"2024-04-25T00:02:23.014038+0000","last_clean":"2024-04-25T00:02:23.014038+0000","last_became_active":"2024-04-25T00:02:22.005915+0000","last_became_peered":"2024-04-25T00:02:22.005915+0000","last_unstale":"2024-04-25T00:02:23.014038+0000","last_undegraded":"2024-04-25T00:02:23.014038+0000","last_fullsized":"2024-04-25T00:02:23.014038+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:19:55.354601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008079+0000","last_change":"2024-04-25T00:02:21.013431+0000","last_active":"2024-04-25T00:02:23.008079+0000","last_peered":"2024-04-25T00:02:23.008079+0000","last_clean":"2024-04-25T00:02:23.008079+0000","last_became_active":"2024-04-25T00:02:21.012907+0000","last_became_peered":"2024-04-25T00:02:21.012907+0000","last_unstale":"2024-04-25T00:02:23.008079+0000","last_undegraded":"2024-04-25T00:02:23.008079+0000","last_fullsized":"2024-04-25T00:02:23.008079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:06.588243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509779+0000","last_change":"2024-04-25T00:02:22.014131+0000","last_active":"2024-04-25T00:02:23.509779+0000","last_peered":"2024-04-25T00:02:23.509779+0000","last_clean":"2024-04-25T00:02:23.509779+0000","last_became_active":"2024-04-25T00:02:22.013860+0000","last_became_peered":"2024-04-25T00:02:22.013860+0000","last_unstale":"2024-04-25T00:02:23.509779+0000","last_undegraded":"2024-04-25T00:02:23.509779+0000","last_fullsized":"2024-04-25T00:02:23.509779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:01.066332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510291+0000","last_change":"2024-04-25T00:02:21.008299+0000","last_active":"2024-04-25T00:02:23.510291+0000","last_peered":"2024-04-25T00:02:23.510291+0000","last_clean":"2024-04-25T00:02:23.510291+0000","last_became_active":"2024-04-25T00:02:21.008024+0000","last_became_peered":"2024-04-25T00:02:21.008024+0000","last_unstale":"2024-04-25T00:02:23.510291+0000","last_undegraded":"2024-04-25T00:02:23.510291+0000","last_fullsized":"2024-04-25T00:02:23.510291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:24.941460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510356+0000","last_change":"2024-04-25T00:02:22.018965+0000","last_active":"2024-04-25T00:02:23.510356+0000","last_peered":"2024-04-25T00:02:23.510356+0000","last_clean":"2024-04-25T00:02:23.510356+0000","last_became_active":"2024-04-25T00:02:22.017660+0000","last_became_peered":"2024-04-25T00:02:22.017660+0000","last_unstale":"2024-04-25T00:02:23.510356+0000","last_undegraded":"2024-04-25T00:02:23.510356+0000","last_fullsized":"2024-04-25T00:02:23.510356+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:16.717482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"75'3","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508901+0000","last_change":"2024-04-25T00:02:21.011826+0000","last_active":"2024-04-25T00:02:23.508901+0000","last_peered":"2024-04-25T00:02:23.508901+0000","last_clean":"2024-04-25T00:02:23.508901+0000","last_became_active":"2024-04-25T00:02:21.011515+0000","last_became_peered":"2024-04-25T00:02:21.011515+0000","last_unstale":"2024-04-25T00:02:23.508901+0000","last_undegraded":"2024-04-25T00:02:23.508901+0000","last_fullsized":"2024-04-25T00:02:23.508901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:11.305619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202462+0000","last_change":"2024-04-25T00:02:22.014394+0000","last_active":"2024-04-25T00:02:23.202462+0000","last_peered":"2024-04-25T00:02:23.202462+0000","last_clean":"2024-04-25T00:02:23.202462+0000","last_became_active":"2024-04-25T00:02:22.013561+0000","last_became_peered":"2024-04-25T00:02:22.013561+0000","last_unstale":"2024-04-25T00:02:23.202462+0000","last_undegraded":"2024-04-25T00:02:23.202462+0000","last_fullsized":"2024-04-25T00:02:23.202462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:25.948199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"75'6","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007714+0000","last_change":"2024-04-25T00:02:21.012403+0000","last_active":"2024-04-25T00:02:23.007714+0000","last_peered":"2024-04-25T00:02:23.007714+0000","last_clean":"2024-04-25T00:02:23.007714+0000","last_became_active":"2024-04-25T00:02:21.012150+0000","last_became_peered":"2024-04-25T00:02:21.012150+0000","last_unstale":"2024-04-25T00:02:23.007714+0000","last_undegraded":"2024-04-25T00:02:23.007714+0000","last_fullsized":"2024-04-25T00:02:23.007714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:24.941460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509139+0000","last_change":"2024-04-25T00:02:22.014440+0000","last_active":"2024-04-25T00:02:23.509139+0000","last_peered":"2024-04-25T00:02:23.509139+0000","last_clean":"2024-04-25T00:02:23.509139+0000","last_became_active":"2024-04-25T00:02:22.014168+0000","last_became_peered":"2024-04-25T00:02:22.014168+0000","last_unstale":"2024-04-25T00:02:23.509139+0000","last_undegraded":"2024-04-25T00:02:23.509139+0000","last_fullsized":"2024-04-25T00:02:23.509139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:19:13.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007001+0000","last_change":"2024-04-25T00:02:21.009088+0000","last_active":"2024-04-25T00:02:23.007001+0000","last_peered":"2024-04-25T00:02:23.007001+0000","last_clean":"2024-04-25T00:02:23.007001+0000","last_became_active":"2024-04-25T00:02:21.008821+0000","last_became_peered":"2024-04-25T00:02:21.008821+0000","last_unstale":"2024-04-25T00:02:23.007001+0000","last_undegraded":"2024-04-25T00:02:23.007001+0000","last_fullsized":"2024-04-25T00:02:23.007001+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:54.262263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013861+0000","last_change":"2024-04-25T00:02:22.016300+0000","last_active":"2024-04-25T00:02:23.013861+0000","last_peered":"2024-04-25T00:02:23.013861+0000","last_clean":"2024-04-25T00:02:23.013861+0000","last_became_active":"2024-04-25T00:02:22.016024+0000","last_became_peered":"2024-04-25T00:02:22.016024+0000","last_unstale":"2024-04-25T00:02:23.013861+0000","last_undegraded":"2024-04-25T00:02:23.013861+0000","last_fullsized":"2024-04-25T00:02:23.013861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:11.266070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509805+0000","last_change":"2024-04-25T00:02:21.015109+0000","last_active":"2024-04-25T00:02:23.509805+0000","last_peered":"2024-04-25T00:02:23.509805+0000","last_clean":"2024-04-25T00:02:23.509805+0000","last_became_active":"2024-04-25T00:02:21.014777+0000","last_became_peered":"2024-04-25T00:02:21.014777+0000","last_unstale":"2024-04-25T00:02:23.509805+0000","last_undegraded":"2024-04-25T00:02:23.509805+0000","last_fullsized":"2024-04-25T00:02:23.509805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:24.542662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007293+0000","last_change":"2024-04-25T00:02:22.012830+0000","last_active":"2024-04-25T00:02:23.007293+0000","last_peered":"2024-04-25T00:02:23.007293+0000","last_clean":"2024-04-25T00:02:23.007293+0000","last_became_active":"2024-04-25T00:02:22.012569+0000","last_became_peered":"2024-04-25T00:02:22.012569+0000","last_unstale":"2024-04-25T00:02:23.007293+0000","last_undegraded":"2024-04-25T00:02:23.007293+0000","last_fullsized":"2024-04-25T00:02:23.007293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:07.863912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013637+0000","last_change":"2024-04-25T00:02:21.010049+0000","last_active":"2024-04-25T00:02:23.013637+0000","last_peered":"2024-04-25T00:02:23.013637+0000","last_clean":"2024-04-25T00:02:23.013637+0000","last_became_active":"2024-04-25T00:02:21.009715+0000","last_became_peered":"2024-04-25T00:02:21.009715+0000","last_unstale":"2024-04-25T00:02:23.013637+0000","last_undegraded":"2024-04-25T00:02:23.013637+0000","last_fullsized":"2024-04-25T00:02:23.013637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:39.648247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202303+0000","last_change":"2024-04-25T00:02:22.013601+0000","last_active":"2024-04-25T00:02:23.202303+0000","last_peered":"2024-04-25T00:02:23.202303+0000","last_clean":"2024-04-25T00:02:23.202303+0000","last_became_active":"2024-04-25T00:02:22.013229+0000","last_became_peered":"2024-04-25T00:02:22.013229+0000","last_unstale":"2024-04-25T00:02:23.202303+0000","last_undegraded":"2024-04-25T00:02:23.202303+0000","last_fullsized":"2024-04-25T00:02:23.202303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:07.570537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"75'8","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007589+0000","last_change":"2024-04-25T00:02:21.010313+0000","last_active":"2024-04-25T00:02:23.007589+0000","last_peered":"2024-04-25T00:02:23.007589+0000","last_clean":"2024-04-25T00:02:23.007589+0000","last_became_active":"2024-04-25T00:02:21.010052+0000","last_became_peered":"2024-04-25T00:02:21.010052+0000","last_unstale":"2024-04-25T00:02:23.007589+0000","last_undegraded":"2024-04-25T00:02:23.007589+0000","last_fullsized":"2024-04-25T00:02:23.007589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:57.497595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008264+0000","last_change":"2024-04-25T00:02:22.020673+0000","last_active":"2024-04-25T00:02:23.008264+0000","last_peered":"2024-04-25T00:02:23.008264+0000","last_clean":"2024-04-25T00:02:23.008264+0000","last_became_active":"2024-04-25T00:02:22.020410+0000","last_became_peered":"2024-04-25T00:02:22.020410+0000","last_unstale":"2024-04-25T00:02:23.008264+0000","last_undegraded":"2024-04-25T00:02:23.008264+0000","last_fullsized":"2024-04-25T00:02:23.008264+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:59:57.068955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510035+0000","last_change":"2024-04-25T00:02:21.013942+0000","last_active":"2024-04-25T00:02:23.510035+0000","last_peered":"2024-04-25T00:02:23.510035+0000","last_clean":"2024-04-25T00:02:23.510035+0000","last_became_active":"2024-04-25T00:02:21.013529+0000","last_became_peered":"2024-04-25T00:02:21.013529+0000","last_unstale":"2024-04-25T00:02:23.510035+0000","last_undegraded":"2024-04-25T00:02:23.510035+0000","last_fullsized":"2024-04-25T00:02:23.510035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:47.657186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509950+0000","last_change":"2024-04-25T00:02:22.010853+0000","last_active":"2024-04-25T00:02:23.509950+0000","last_peered":"2024-04-25T00:02:23.509950+0000","last_clean":"2024-04-25T00:02:23.509950+0000","last_became_active":"2024-04-25T00:02:22.010540+0000","last_became_peered":"2024-04-25T00:02:22.010540+0000","last_unstale":"2024-04-25T00:02:23.509950+0000","last_undegraded":"2024-04-25T00:02:23.509950+0000","last_fullsized":"2024-04-25T00:02:23.509950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:07.635540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.008145+0000","last_change":"2024-04-25T00:02:21.007387+0000","last_active":"2024-04-25T00:02:23.008145+0000","last_peered":"2024-04-25T00:02:23.008145+0000","last_clean":"2024-04-25T00:02:23.008145+0000","last_became_active":"2024-04-25T00:02:21.007015+0000","last_became_peered":"2024-04-25T00:02:21.007015+0000","last_unstale":"2024-04-25T00:02:23.008145+0000","last_undegraded":"2024-04-25T00:02:23.008145+0000","last_fullsized":"2024-04-25T00:02:23.008145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:22.753750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007658+0000","last_change":"2024-04-25T00:02:22.015520+0000","last_active":"2024-04-25T00:02:23.007658+0000","last_peered":"2024-04-25T00:02:23.007658+0000","last_clean":"2024-04-25T00:02:23.007658+0000","last_became_active":"2024-04-25T00:02:22.014922+0000","last_became_peered":"2024-04-25T00:02:22.014922+0000","last_unstale":"2024-04-25T00:02:23.007658+0000","last_undegraded":"2024-04-25T00:02:23.007658+0000","last_fullsized":"2024-04-25T00:02:23.007658+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:46.505197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509775+0000","last_change":"2024-04-25T00:02:21.014230+0000","last_active":"2024-04-25T00:02:23.509775+0000","last_peered":"2024-04-25T00:02:23.509775+0000","last_clean":"2024-04-25T00:02:23.509775+0000","last_became_active":"2024-04-25T00:02:21.013946+0000","last_became_peered":"2024-04-25T00:02:21.013946+0000","last_unstale":"2024-04-25T00:02:23.509775+0000","last_undegraded":"2024-04-25T00:02:23.509775+0000","last_fullsized":"2024-04-25T00:02:23.509775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:54.262263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202567+0000","last_change":"2024-04-25T00:02:22.018654+0000","last_active":"2024-04-25T00:02:23.202567+0000","last_peered":"2024-04-25T00:02:23.202567+0000","last_clean":"2024-04-25T00:02:23.202567+0000","last_became_active":"2024-04-25T00:02:22.018279+0000","last_became_peered":"2024-04-25T00:02:22.018279+0000","last_unstale":"2024-04-25T00:02:23.202567+0000","last_undegraded":"2024-04-25T00:02:23.202567+0000","last_fullsized":"2024-04-25T00:02:23.202567+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:37.399094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509331+0000","last_change":"2024-04-25T00:02:21.015322+0000","last_active":"2024-04-25T00:02:23.509331+0000","last_peered":"2024-04-25T00:02:23.509331+0000","last_clean":"2024-04-25T00:02:23.509331+0000","last_became_active":"2024-04-25T00:02:21.014852+0000","last_became_peered":"2024-04-25T00:02:21.014852+0000","last_unstale":"2024-04-25T00:02:23.509331+0000","last_undegraded":"2024-04-25T00:02:23.509331+0000","last_fullsized":"2024-04-25T00:02:23.509331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:05.270634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508779+0000","last_change":"2024-04-25T00:02:22.009714+0000","last_active":"2024-04-25T00:02:23.508779+0000","last_peered":"2024-04-25T00:02:23.508779+0000","last_clean":"2024-04-25T00:02:23.508779+0000","last_became_active":"2024-04-25T00:02:22.009214+0000","last_became_peered":"2024-04-25T00:02:22.009214+0000","last_unstale":"2024-04-25T00:02:23.508779+0000","last_undegraded":"2024-04-25T00:02:23.508779+0000","last_fullsized":"2024-04-25T00:02:23.508779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:55.926104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509919+0000","last_change":"2024-04-25T00:02:21.007937+0000","last_active":"2024-04-25T00:02:23.509919+0000","last_peered":"2024-04-25T00:02:23.509919+0000","last_clean":"2024-04-25T00:02:23.509919+0000","last_became_active":"2024-04-25T00:02:21.007605+0000","last_became_peered":"2024-04-25T00:02:21.007605+0000","last_unstale":"2024-04-25T00:02:23.509919+0000","last_undegraded":"2024-04-25T00:02:23.509919+0000","last_fullsized":"2024-04-25T00:02:23.509919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:19.301050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508707+0000","last_change":"2024-04-25T00:02:22.015593+0000","last_active":"2024-04-25T00:02:23.508707+0000","last_peered":"2024-04-25T00:02:23.508707+0000","last_clean":"2024-04-25T00:02:23.508707+0000","last_became_active":"2024-04-25T00:02:22.015328+0000","last_became_peered":"2024-04-25T00:02:22.015328+0000","last_unstale":"2024-04-25T00:02:23.508707+0000","last_undegraded":"2024-04-25T00:02:23.508707+0000","last_fullsized":"2024-04-25T00:02:23.508707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:41:19.855010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013960+0000","last_change":"2024-04-25T00:02:21.013272+0000","last_active":"2024-04-25T00:02:23.013960+0000","last_peered":"2024-04-25T00:02:23.013960+0000","last_clean":"2024-04-25T00:02:23.013960+0000","last_became_active":"2024-04-25T00:02:21.012935+0000","last_became_peered":"2024-04-25T00:02:21.012935+0000","last_unstale":"2024-04-25T00:02:23.013960+0000","last_undegraded":"2024-04-25T00:02:23.013960+0000","last_fullsized":"2024-04-25T00:02:23.013960+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:59.876656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508720+0000","last_change":"2024-04-25T00:02:22.014699+0000","last_active":"2024-04-25T00:02:23.508720+0000","last_peered":"2024-04-25T00:02:23.508720+0000","last_clean":"2024-04-25T00:02:23.508720+0000","last_became_active":"2024-04-25T00:02:22.014451+0000","last_became_peered":"2024-04-25T00:02:22.014451+0000","last_unstale":"2024-04-25T00:02:23.508720+0000","last_undegraded":"2024-04-25T00:02:23.508720+0000","last_fullsized":"2024-04-25T00:02:23.508720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:11.526812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007141+0000","last_change":"2024-04-25T00:02:21.010717+0000","last_active":"2024-04-25T00:02:23.007141+0000","last_peered":"2024-04-25T00:02:23.007141+0000","last_clean":"2024-04-25T00:02:23.007141+0000","last_became_active":"2024-04-25T00:02:21.010435+0000","last_became_peered":"2024-04-25T00:02:21.010435+0000","last_unstale":"2024-04-25T00:02:23.007141+0000","last_undegraded":"2024-04-25T00:02:23.007141+0000","last_fullsized":"2024-04-25T00:02:23.007141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:57.913997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007461+0000","last_change":"2024-04-25T00:02:22.016231+0000","last_active":"2024-04-25T00:02:23.007461+0000","last_peered":"2024-04-25T00:02:23.007461+0000","last_clean":"2024-04-25T00:02:23.007461+0000","last_became_active":"2024-04-25T00:02:22.015976+0000","last_became_peered":"2024-04-25T00:02:22.015976+0000","last_unstale":"2024-04-25T00:02:23.007461+0000","last_undegraded":"2024-04-25T00:02:23.007461+0000","last_fullsized":"2024-04-25T00:02:23.007461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:22.055700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509998+0000","last_change":"2024-04-25T00:02:21.012474+0000","last_active":"2024-04-25T00:02:23.509998+0000","last_peered":"2024-04-25T00:02:23.509998+0000","last_clean":"2024-04-25T00:02:23.509998+0000","last_became_active":"2024-04-25T00:02:21.012207+0000","last_became_peered":"2024-04-25T00:02:21.012207+0000","last_unstale":"2024-04-25T00:02:23.509998+0000","last_undegraded":"2024-04-25T00:02:23.509998+0000","last_fullsized":"2024-04-25T00:02:23.509998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:36.392355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.013966+0000","last_change":"2024-04-25T00:02:22.010495+0000","last_active":"2024-04-25T00:02:23.013966+0000","last_peered":"2024-04-25T00:02:23.013966+0000","last_clean":"2024-04-25T00:02:23.013966+0000","last_became_active":"2024-04-25T00:02:22.010122+0000","last_became_peered":"2024-04-25T00:02:22.010122+0000","last_unstale":"2024-04-25T00:02:23.013966+0000","last_undegraded":"2024-04-25T00:02:23.013966+0000","last_fullsized":"2024-04-25T00:02:23.013966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:11.021602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.014129+0000","last_change":"2024-04-25T00:02:21.009993+0000","last_active":"2024-04-25T00:02:23.014129+0000","last_peered":"2024-04-25T00:02:23.014129+0000","last_clean":"2024-04-25T00:02:23.014129+0000","last_became_active":"2024-04-25T00:02:21.009614+0000","last_became_peered":"2024-04-25T00:02:21.009614+0000","last_unstale":"2024-04-25T00:02:23.014129+0000","last_undegraded":"2024-04-25T00:02:23.014129+0000","last_fullsized":"2024-04-25T00:02:23.014129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:11.495792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509350+0000","last_change":"2024-04-25T00:02:22.013205+0000","last_active":"2024-04-25T00:02:23.509350+0000","last_peered":"2024-04-25T00:02:23.509350+0000","last_clean":"2024-04-25T00:02:23.509350+0000","last_became_active":"2024-04-25T00:02:22.012710+0000","last_became_peered":"2024-04-25T00:02:22.012710+0000","last_unstale":"2024-04-25T00:02:23.509350+0000","last_undegraded":"2024-04-25T00:02:23.509350+0000","last_fullsized":"2024-04-25T00:02:23.509350+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:12.312357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202543+0000","last_change":"2024-04-25T00:02:21.010823+0000","last_active":"2024-04-25T00:02:23.202543+0000","last_peered":"2024-04-25T00:02:23.202543+0000","last_clean":"2024-04-25T00:02:23.202543+0000","last_became_active":"2024-04-25T00:02:21.009880+0000","last_became_peered":"2024-04-25T00:02:21.009880+0000","last_unstale":"2024-04-25T00:02:23.202543+0000","last_undegraded":"2024-04-25T00:02:23.202543+0000","last_fullsized":"2024-04-25T00:02:23.202543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:19.375987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509751+0000","last_change":"2024-04-25T00:02:22.007397+0000","last_active":"2024-04-25T00:02:23.509751+0000","last_peered":"2024-04-25T00:02:23.509751+0000","last_clean":"2024-04-25T00:02:23.509751+0000","last_became_active":"2024-04-25T00:02:22.006892+0000","last_became_peered":"2024-04-25T00:02:22.006892+0000","last_unstale":"2024-04-25T00:02:23.509751+0000","last_undegraded":"2024-04-25T00:02:23.509751+0000","last_fullsized":"2024-04-25T00:02:23.509751+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:29.699927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007581+0000","last_change":"2024-04-25T00:02:21.003729+0000","last_active":"2024-04-25T00:02:23.007581+0000","last_peered":"2024-04-25T00:02:23.007581+0000","last_clean":"2024-04-25T00:02:23.007581+0000","last_became_active":"2024-04-25T00:02:21.003453+0000","last_became_peered":"2024-04-25T00:02:21.003453+0000","last_unstale":"2024-04-25T00:02:23.007581+0000","last_undegraded":"2024-04-25T00:02:23.007581+0000","last_fullsized":"2024-04-25T00:02:23.007581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:49.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202472+0000","last_change":"2024-04-25T00:02:22.014004+0000","last_active":"2024-04-25T00:02:23.202472+0000","last_peered":"2024-04-25T00:02:23.202472+0000","last_clean":"2024-04-25T00:02:23.202472+0000","last_became_active":"2024-04-25T00:02:22.013732+0000","last_became_peered":"2024-04-25T00:02:22.013732+0000","last_unstale":"2024-04-25T00:02:23.202472+0000","last_undegraded":"2024-04-25T00:02:23.202472+0000","last_fullsized":"2024-04-25T00:02:23.202472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:42.558729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.202327+0000","last_change":"2024-04-25T00:02:21.010404+0000","last_active":"2024-04-25T00:02:23.202327+0000","last_peered":"2024-04-25T00:02:23.202327+0000","last_clean":"2024-04-25T00:02:23.202327+0000","last_became_active":"2024-04-25T00:02:21.010105+0000","last_became_peered":"2024-04-25T00:02:21.010105+0000","last_unstale":"2024-04-25T00:02:23.202327+0000","last_undegraded":"2024-04-25T00:02:23.202327+0000","last_fullsized":"2024-04-25T00:02:23.202327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:18.148628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509814+0000","last_change":"2024-04-25T00:02:22.010855+0000","last_active":"2024-04-25T00:02:23.509814+0000","last_peered":"2024-04-25T00:02:23.509814+0000","last_clean":"2024-04-25T00:02:23.509814+0000","last_became_active":"2024-04-25T00:02:22.010518+0000","last_became_peered":"2024-04-25T00:02:22.010518+0000","last_unstale":"2024-04-25T00:02:23.509814+0000","last_undegraded":"2024-04-25T00:02:23.509814+0000","last_fullsized":"2024-04-25T00:02:23.509814+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:58.504334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.006860+0000","last_change":"2024-04-25T00:02:21.011175+0000","last_active":"2024-04-25T00:02:23.006860+0000","last_peered":"2024-04-25T00:02:23.006860+0000","last_clean":"2024-04-25T00:02:23.006860+0000","last_became_active":"2024-04-25T00:02:21.010864+0000","last_became_peered":"2024-04-25T00:02:21.010864+0000","last_unstale":"2024-04-25T00:02:23.006860+0000","last_undegraded":"2024-04-25T00:02:23.006860+0000","last_fullsized":"2024-04-25T00:02:23.006860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:06.563798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.006811+0000","last_change":"2024-04-25T00:02:22.013997+0000","last_active":"2024-04-25T00:02:23.006811+0000","last_peered":"2024-04-25T00:02:23.006811+0000","last_clean":"2024-04-25T00:02:23.006811+0000","last_became_active":"2024-04-25T00:02:22.011669+0000","last_became_peered":"2024-04-25T00:02:22.011669+0000","last_unstale":"2024-04-25T00:02:23.006811+0000","last_undegraded":"2024-04-25T00:02:23.006811+0000","last_fullsized":"2024-04-25T00:02:23.006811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:25:41.919497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.007517+0000","last_change":"2024-04-25T00:02:21.009271+0000","last_active":"2024-04-25T00:02:23.007517+0000","last_peered":"2024-04-25T00:02:23.007517+0000","last_clean":"2024-04-25T00:02:23.007517+0000","last_became_active":"2024-04-25T00:02:21.009010+0000","last_became_peered":"2024-04-25T00:02:21.009010+0000","last_unstale":"2024-04-25T00:02:23.007517+0000","last_undegraded":"2024-04-25T00:02:23.007517+0000","last_fullsized":"2024-04-25T00:02:23.007517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:06.885165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509562+0000","last_change":"2024-04-25T00:02:22.012031+0000","last_active":"2024-04-25T00:02:23.509562+0000","last_peered":"2024-04-25T00:02:23.509562+0000","last_clean":"2024-04-25T00:02:23.509562+0000","last_became_active":"2024-04-25T00:02:22.011756+0000","last_became_peered":"2024-04-25T00:02:22.011756+0000","last_unstale":"2024-04-25T00:02:23.509562+0000","last_undegraded":"2024-04-25T00:02:23.509562+0000","last_fullsized":"2024-04-25T00:02:23.509562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:28.280471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510221+0000","last_change":"2024-04-25T00:02:21.009814+0000","last_active":"2024-04-25T00:02:23.510221+0000","last_peered":"2024-04-25T00:02:23.510221+0000","last_clean":"2024-04-25T00:02:23.510221+0000","last_became_active":"2024-04-25T00:02:21.009197+0000","last_became_peered":"2024-04-25T00:02:21.009197+0000","last_unstale":"2024-04-25T00:02:23.510221+0000","last_undegraded":"2024-04-25T00:02:23.510221+0000","last_fullsized":"2024-04-25T00:02:23.510221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:41.551991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.508715+0000","last_change":"2024-04-25T00:02:22.016400+0000","last_active":"2024-04-25T00:02:23.508715+0000","last_peered":"2024-04-25T00:02:23.508715+0000","last_clean":"2024-04-25T00:02:23.508715+0000","last_became_active":"2024-04-25T00:02:22.016160+0000","last_became_peered":"2024-04-25T00:02:22.016160+0000","last_unstale":"2024-04-25T00:02:23.508715+0000","last_undegraded":"2024-04-25T00:02:23.508715+0000","last_fullsized":"2024-04-25T00:02:23.508715+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:20.985123+0000","last_clean_scrub_stamp":"2024-04-25T00:02:20.985123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:35.442980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.510296+0000","last_change":"2024-04-25T00:02:21.011072+0000","last_active":"2024-04-25T00:02:23.510296+0000","last_peered":"2024-04-25T00:02:23.510296+0000","last_clean":"2024-04-25T00:02:23.510296+0000","last_became_active":"2024-04-25T00:02:21.010789+0000","last_became_peered":"2024-04-25T00:02:21.010789+0000","last_unstale":"2024-04-25T00:02:23.510296+0000","last_undegraded":"2024-04-25T00:02:23.510296+0000","last_fullsized":"2024-04-25T00:02:23.510296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:02:19.978385+0000","last_clean_scrub_stamp":"2024-04-25T00:02:19.978385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:33.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":271,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T00:02:23.509049+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:02:23.509049+0000","last_peered":"2024-04-25T00:02:23.509049+0000","last_clean":"2024-04-25T00:02:23.509049+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:02:23.509049+0000","last_undegraded":"2024-04-25T00:02:23.509049+0000","last_fullsized":"2024-04-25T00:02:23.509049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28520,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343320,"statfs":{"total":96636764160,"available":96607559680,"internally_reserved":0,"allocated":1527808,"data_stored":986161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331424,"statfs":{"total":96636764160,"available":96595378176,"internally_reserved":0,"allocated":933888,"data_stored":395269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40412,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331428,"statfs":{"total":96636764160,"available":96595382272,"internally_reserved":0,"allocated":929792,"data_stored":395267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":395247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":933888,"data_stored":395289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":937984,"data_stored":396789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36272,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335568,"statfs":{"total":96636764160,"available":96599621632,"internally_reserved":0,"allocated":946176,"data_stored":397405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1519616,"data_stored":985545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:02:29.653 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:02:29.653 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:02:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.792+0000 7f466e1b6640 1 Processor -- start 2024-04-25T00:02:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.792+0000 7f466e1b6640 1 -- start start 2024-04-25T00:02:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f466e1b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f466815e510 0x7f46680d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f466e1b6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680d28c0 0x7f46680b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f466e1b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46680d2e00 con 0x7f46680b61e0 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f466e1b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f46680d30e0 con 0x7f466815e510 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f466e1b6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46680d33c0 con 0x7f46680d28c0 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f46677fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f466815e510 0x7f46680d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f4667fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f46680b61e0 0x7f4668071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52902/0 (socket says 172.21.15.113:52902) 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.793+0000 7f4667fff640 1 -- 172.21.15.113:0/1659851634 learned_addr learned my addr 172.21.15.113:0/1659851634 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f4666ffd640 1 --2- 172.21.15.113:0/1659851634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680d28c0 0x7f46680b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f46677fe640 1 -- 172.21.15.113:0/1659851634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680d28c0 msgr2=0x7f46680b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f46677fe640 1 --2- 172.21.15.113:0/1659851634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680d28c0 0x7f46680b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f46677fe640 1 -- 172.21.15.113:0/1659851634 >> v1:172.21.15.113:6789/0 conn(0x7f46680b61e0 legacy=0x7f4668071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f46667fc640 1 -- 172.21.15.113:0/1659851634 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414819322 0 0) 0x7f46680d2e00 con 0x7f46680b61e0 2024-04-25T00:02:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f46677fe640 1 -- 172.21.15.113:0/1659851634 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46680d36a0 con 0x7f466815e510 2024-04-25T00:02:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f46677fe640 1 --2- 172.21.15.113:0/1659851634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680d2380 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f46580097a0 tx=0x7f465802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db8986bad5cca5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.794+0000 7f46667fc640 1 -- 172.21.15.113:0/1659851634 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658037070 con 0x7f466815e510 2024-04-25T00:02:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f46667fc640 1 -- 172.21.15.113:0/1659851634 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465802ae50 con 0x7f466815e510 2024-04-25T00:02:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f46667fc640 1 -- 172.21.15.113:0/1659851634 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658032660 con 0x7f466815e510 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 -- 172.21.15.113:0/1659851634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 msgr2=0x7f46680d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 --2- 172.21.15.113:0/1659851634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680d2380 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f46580097a0 tx=0x7f465802a100 comp rx=0 tx=0).stop 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 -- 172.21.15.113:0/1659851634 shutdown_connections 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 --2- 172.21.15.113:0/1659851634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680d28c0 0x7f46680b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 --2- 172.21.15.113:0/1659851634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680d2380 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 -- 172.21.15.113:0/1659851634 >> 172.21.15.113:0/1659851634 conn(0x7f466806b720 msgr2=0x7f46680d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 -- 172.21.15.113:0/1659851634 shutdown_connections 2024-04-25T00:02:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.795+0000 7f466e1b6640 1 -- 172.21.15.113:0/1659851634 wait complete. 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.796+0000 7f466e1b6640 1 Processor -- start 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.796+0000 7f466e1b6640 1 -- start start 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f466e1b6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680b61e0 0x7f4668142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f466e1b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46680d28c0 0x7f46680c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f466e1b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f466e1b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f46680b9dd0 con 0x7f466815e510 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f466e1b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46680d3ea0 con 0x7f46680d28c0 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f466e1b6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46680b83d0 con 0x7f46680b61e0 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f4667fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f4667fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40114/0 (socket says 172.21.15.113:40114) 2024-04-25T00:02:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f4667fff640 1 -- 172.21.15.113:0/217998992 learned_addr learned my addr 172.21.15.113:0/217998992 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:29.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f46677fe640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680b61e0 0x7f4668142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.797+0000 7f4666ffd640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46680d28c0 0x7f46680c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f4667fff640 1 -- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680b61e0 msgr2=0x7f4668142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f4667fff640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680b61e0 0x7f4668142ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f4667fff640 1 -- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46680d28c0 msgr2=0x7f46680c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f4667fff640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46680d28c0 0x7f46680c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f4667fff640 1 -- 172.21.15.113:0/217998992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46680c8e00 con 0x7f466815e510 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f4667fff640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680cd9c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f466000c7a0 tx=0x7f466000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f464ffff640 1 -- 172.21.15.113:0/217998992 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660018070 con 0x7f466815e510 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f464ffff640 1 -- 172.21.15.113:0/217998992 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466000a640 con 0x7f466815e510 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f464ffff640 1 -- 172.21.15.113:0/217998992 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466000aab0 con 0x7f466815e510 2024-04-25T00:02:29.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46680c90a0 con 0x7f466815e510 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.798+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46680c9660 con 0x7f466815e510 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.799+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46680691a0 con 0x7f466815e510 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.799+0000 7f464ffff640 1 -- 172.21.15.113:0/217998992 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f466000ac50 con 0x7f466815e510 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.800+0000 7f464ffff640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465406be30 0x7f465406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.800+0000 7f46677fe640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465406be30 0x7f465406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.800+0000 7f464ffff640 1 -- 172.21.15.113:0/217998992 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f466008a4f0 con 0x7f466815e510 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.800+0000 7f46677fe640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465406be30 0x7f465406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4658034000 tx=0x7f465802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:29.804+0000 7f464ffff640 1 -- 172.21.15.113:0/217998992 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4660058ea0 con 0x7f466815e510 2024-04-25T00:02:30.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.051+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f46680ce980 con 0x7f466815e510 2024-04-25T00:02:30.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.052+0000 7f464ffff640 1 -- 172.21.15.113:0/217998992 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f46600587a0 con 0x7f466815e510 2024-04-25T00:02:30.053 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:02:30.053 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:02:30.053 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:02:30.053 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:02:30.053 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:02:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:02:30.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465406be30 msgr2=0x7f465406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465406be30 0x7f465406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4658034000 tx=0x7f465802e000 comp rx=0 tx=0).stop 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 msgr2=0x7f46680cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680cd9c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f466000c7a0 tx=0x7f466000cc60 comp rx=0 tx=0).stop 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 shutdown_connections 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465406be30 0x7f465406e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46680d28c0 0x7f46680c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466815e510 0x7f46680cd9c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 --2- 172.21.15.113:0/217998992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46680b61e0 0x7f4668142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 >> 172.21.15.113:0/217998992 conn(0x7f466806b720 msgr2=0x7f46680b7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 shutdown_connections 2024-04-25T00:02:30.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.055+0000 7f466e1b6640 1 -- 172.21.15.113:0/217998992 wait complete. 2024-04-25T00:02:30.066 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:02:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.163+0000 7f73fb85d640 1 Processor -- start 2024-04-25T00:02:30.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.163+0000 7f73fb85d640 1 -- start start 2024-04-25T00:02:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73fb85d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f73f40b3fd0 0x7f73f406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73fb85d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f4144cf0 con 0x7f73f406e510 2024-04-25T00:02:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73fb85d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f40b2f30 con 0x7f73f4144910 2024-04-25T00:02:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73fb85d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f73f406e980 con 0x7f73f40b3fd0 2024-04-25T00:02:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73f9dfa640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f73f406e510 0x7f73f40cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52906/0 (socket says 172.21.15.113:52906) 2024-04-25T00:02:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73f9dfa640 1 -- 172.21.15.113:0/752206442 learned_addr learned my addr 172.21.15.113:0/752206442 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709287107 0 0) 0x7f73f4144cf0 con 0x7f73f406e510 2024-04-25T00:02:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73e0003520 con 0x7f73f406e510 2024-04-25T00:02:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.164+0000 7f73f8df8640 1 --2- 172.21.15.113:0/752206442 >> v2:172.21.15.178:3300/0 conn(0x7f73f40b3fd0 0x7f73f406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 744144348 0 0) 0x7f73e0003520 con 0x7f73f406e510 2024-04-25T00:02:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 >> v2:172.21.15.178:3300/0 conn(0x7f73f40b3fd0 msgr2=0x7f73f406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 --2- 172.21.15.113:0/752206442 >> v2:172.21.15.178:3300/0 conn(0x7f73f40b3fd0 0x7f73f406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 >> v1:172.21.15.178:6789/0 conn(0x7f73f4144910 legacy=0x7f73f40b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f40b3210 con 0x7f73f406e510 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f73ec001670 con 0x7f73f406e510 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 >> v1:172.21.15.113:6789/0 conn(0x7f73f406e510 legacy=0x7f73f40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73e00039e0 0x7f73e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e00051c0 0x7f73e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73f406e980 con 0x7f73e0004800 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73e000c090 con 0x7f73e00039e0 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73e000c290 con 0x7f73e00051c0 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f8df8640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f95f9640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73e00039e0 0x7f73e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f9dfa640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e00051c0 0x7f73e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f8df8640 1 -- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e00051c0 msgr2=0x7f73e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f8df8640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e00051c0 0x7f73e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f8df8640 1 -- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73e00039e0 msgr2=0x7f73e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f8df8640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73e00039e0 0x7f73e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f8df8640 1 -- 172.21.15.113:0/752206442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e0003520 con 0x7f73e0004800 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.165+0000 7f73f9dfa640 1 -- 172.21.15.113:0/752206442 reap_dead start 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73f8df8640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ac0 tx=0x7f73e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14058de121357c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4036070 con 0x7f73e0004800 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e4031400 con 0x7f73e0004800 2024-04-25T00:02:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73ebfff640 1 -- 172.21.15.113:0/752206442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4009690 con 0x7f73e0004800 2024-04-25T00:02:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73fb85d640 1 -- 172.21.15.113:0/752206442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73e0004800 msgr2=0x7f73e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73fb85d640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ac0 tx=0x7f73e0005830 comp rx=0 tx=0).stop 2024-04-25T00:02:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73fb85d640 1 -- 172.21.15.113:0/752206442 shutdown_connections 2024-04-25T00:02:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73fb85d640 1 --2- 172.21.15.113:0/752206442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.166+0000 7f73fb85d640 1 -- 172.21.15.113:0/752206442 >> 172.21.15.113:0/752206442 conn(0x7f73f40b1a50 msgr2=0x7f73f40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.167+0000 7f73fb85d640 1 -- 172.21.15.113:0/752206442 shutdown_connections 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.167+0000 7f73fb85d640 1 -- 172.21.15.113:0/752206442 wait complete. 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 Processor -- start 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 -- start start 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e0004800 0x7f73f41f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73f415bf70 0x7f73f41406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73f41f1e40 0x7f73f415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73f40b4440 con 0x7f73f415bf70 2024-04-25T00:02:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73f40b37d0 con 0x7f73f41f1e40 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73fb85d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73f40b4da0 con 0x7f73e0004800 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73f8df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73f415bf70 0x7f73f41406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73f8df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73f415bf70 0x7f73f41406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40140/0 (socket says 172.21.15.113:40140) 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.168+0000 7f73f8df8640 1 -- 172.21.15.113:0/448790265 learned_addr learned my addr 172.21.15.113:0/448790265 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f95f9640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e0004800 0x7f73f41f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f9dfa640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73f41f1e40 0x7f73f415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f8df8640 1 -- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e0004800 msgr2=0x7f73f41f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f8df8640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e0004800 0x7f73f41f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f8df8640 1 -- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73f41f1e40 msgr2=0x7f73f415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f8df8640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73f41f1e40 0x7f73f415eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f8df8640 1 -- 172.21.15.113:0/448790265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f41f3850 con 0x7f73f415bf70 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f9dfa640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73f41f1e40 0x7f73f415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73f8df8640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73f415bf70 0x7f73f41406b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f73e4034040 tx=0x7f73e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73e9ffb640 1 -- 172.21.15.113:0/448790265 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e403f020 con 0x7f73f415bf70 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73e9ffb640 1 -- 172.21.15.113:0/448790265 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e4037040 con 0x7f73f415bf70 2024-04-25T00:02:30.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f41f2810 con 0x7f73f415bf70 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73e9ffb640 1 -- 172.21.15.113:0/448790265 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4036070 con 0x7f73f415bf70 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.169+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f41f2dd0 con 0x7f73f415bf70 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.170+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f4075680 con 0x7f73f415bf70 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.171+0000 7f73e9ffb640 1 -- 172.21.15.113:0/448790265 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73e4042020 con 0x7f73f415bf70 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.171+0000 7f73e9ffb640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f73d806be30 0x7f73d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.171+0000 7f73e9ffb640 1 -- 172.21.15.113:0/448790265 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f73e40aaf80 con 0x7f73f415bf70 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.171+0000 7f73f95f9640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f73d806be30 0x7f73d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.172+0000 7f73f95f9640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f73d806be30 0x7f73d806e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f73f0000f80 tx=0x7f73f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.175+0000 7f73e9ffb640 1 -- 172.21.15.113:0/448790265 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f73f4075680 con 0x7f73f415bf70 2024-04-25T00:02:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.382+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73f40691a0 con 0x7f73f415bf70 2024-04-25T00:02:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.382+0000 7f73e9ffb640 1 -- 172.21.15.113:0/448790265 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5192 (secure 0 0 0) 0x7f73e4049030 con 0x7f73f415bf70 2024-04-25T00:02:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:02:30.384 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4243948985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4243948985},{"type":"v1","addr":"172.21.15.178:6835","nonce":4243948985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/665506344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":665506344},{"type":"v1","addr":"172.21.15.178:6834","nonce":665506344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4064308986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4064308986},{"type":"v1","addr":"172.21.15.113:6836","nonce":4064308986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:02:21.350179+0000","modified":"2024-04-25T00:02:23.337147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3530277358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3530277358},{"type":"v1","addr":"172.21.15.113:6837","nonce":3530277358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6951,"qdb_cluster":[6951]},"id":5}]} 2024-04-25T00:02:30.385 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 43 2024-04-25T00:02:30.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f73d806be30 msgr2=0x7f73d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f73d806be30 0x7f73d806e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f73f0000f80 tx=0x7f73f0009040 comp rx=0 tx=0).stop 2024-04-25T00:02:30.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73f415bf70 msgr2=0x7f73f41406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73f415bf70 0x7f73f41406b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f73e4034040 tx=0x7f73e4004730 comp rx=0 tx=0).stop 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 shutdown_connections 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f73d806be30 0x7f73d806e2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73f41f1e40 0x7f73f415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73f415bf70 0x7f73f41406b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 --2- 172.21.15.113:0/448790265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73e0004800 0x7f73f41f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 >> 172.21.15.113:0/448790265 conn(0x7f73f40b1a50 msgr2=0x7f73f40cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.384+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 shutdown_connections 2024-04-25T00:02:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.385+0000 7f73fb85d640 1 -- 172.21.15.113:0/448790265 wait complete. 2024-04-25T00:02:30.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:02:21.350179+0000', 'modified': '2024-04-25T00:02:23.337147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6951}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6951': {'gid': 6951, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6837/3530277358', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 3530277358}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3530277358}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6951, 'qdb_cluster': [6951]} 2024-04-25T00:02:30.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:02:30.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:02:30.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.899+0000 7f4347a4d640 1 Processor -- start 2024-04-25T00:02:30.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.899+0000 7f4347a4d640 1 -- start start 2024-04-25T00:02:30.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f4347a4d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f434015e2d0 0x7f434015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f4347a4d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f43400b4a40 0x7f43400ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f4347a4d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f434015f4f0 con 0x7f43400b3080 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f4347a4d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f43400cde00 con 0x7f434015e2d0 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f4347a4d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f434015f7d0 con 0x7f43400b4a40 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f43457e9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f434015e2d0 0x7f434015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f43457e9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f434015e2d0 0x7f434015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40154/0 (socket says 172.21.15.113:40154) 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f43457e9640 1 -- 172.21.15.113:0/252330700 learned_addr learned my addr 172.21.15.113:0/252330700 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f43457e9640 1 -- 172.21.15.113:0/252330700 >> v2:172.21.15.178:3300/0 conn(0x7f43400b4a40 msgr2=0x7f43400ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f43457e9640 1 --2- 172.21.15.113:0/252330700 >> v2:172.21.15.178:3300/0 conn(0x7f43400b4a40 0x7f43400ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f43457e9640 1 -- 172.21.15.113:0/252330700 >> v1:172.21.15.178:6789/0 conn(0x7f43400b3080 legacy=0x7f43400b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.900+0000 7f43457e9640 1 -- 172.21.15.113:0/252330700 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434015fab0 con 0x7f434015e2d0 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.901+0000 7f43457e9640 1 --2- 172.21.15.113:0/252330700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f434015e2d0 0x7f434015ebf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f43400d0f20 tx=0x7f433c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e448096f59aa4361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.901+0000 7f4337fff640 1 -- 172.21.15.113:0/252330700 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c036070 con 0x7f434015e2d0 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.901+0000 7f4337fff640 1 -- 172.21.15.113:0/252330700 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433c031600 con 0x7f434015e2d0 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.901+0000 7f4337fff640 1 -- 172.21.15.113:0/252330700 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c0319d0 con 0x7f434015e2d0 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 -- 172.21.15.113:0/252330700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f434015e2d0 msgr2=0x7f434015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 --2- 172.21.15.113:0/252330700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f434015e2d0 0x7f434015ebf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f43400d0f20 tx=0x7f433c02a6f0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 -- 172.21.15.113:0/252330700 shutdown_connections 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 --2- 172.21.15.113:0/252330700 >> v2:172.21.15.178:3300/0 conn(0x7f43400b4a40 0x7f43400ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 --2- 172.21.15.113:0/252330700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f434015e2d0 0x7f434015ebf0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 -- 172.21.15.113:0/252330700 >> 172.21.15.113:0/252330700 conn(0x7f434006b0f0 msgr2=0x7f434006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 -- 172.21.15.113:0/252330700 shutdown_connections 2024-04-25T00:02:30.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.902+0000 7f4347a4d640 1 -- 172.21.15.113:0/252330700 wait complete. 2024-04-25T00:02:30.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.903+0000 7f4347a4d640 1 Processor -- start 2024-04-25T00:02:30.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.903+0000 7f4347a4d640 1 -- start start 2024-04-25T00:02:30.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4347a4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43400b3080 0x7f434015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4347a4d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43400b4a40 0x7f434014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4347a4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f434015e2d0 0x7f43401541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4347a4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f434015f2b0 con 0x7f43400b3080 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4347a4d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f434015f130 con 0x7f434015e2d0 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4347a4d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f434015e6b0 con 0x7f43400b4a40 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43400b3080 0x7f434015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43400b3080 0x7f434015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40156/0 (socket says 172.21.15.113:40156) 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 -- 172.21.15.113:0/2206120187 learned_addr learned my addr 172.21.15.113:0/2206120187 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 -- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43400b4a40 msgr2=0x7f434014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43400b4a40 0x7f434014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 -- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f434015e2d0 msgr2=0x7f43401541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4345fea640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f434015e2d0 0x7f43401541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f434015e2d0 0x7f43401541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 -- 172.21.15.113:0/2206120187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434015be90 con 0x7f43400b3080 2024-04-25T00:02:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4345fea640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f434015e2d0 0x7f43401541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f43457e9640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43400b3080 0x7f434015b950 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f433c034040 tx=0x7f433c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4335ffb640 1 -- 172.21.15.113:0/2206120187 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c041020 con 0x7f43400b3080 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4335ffb640 1 -- 172.21.15.113:0/2206120187 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433c007800 con 0x7f43400b3080 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4335ffb640 1 -- 172.21.15.113:0/2206120187 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c036040 con 0x7f43400b3080 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.904+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4340153450 con 0x7f43400b3080 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.905+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4340153710 con 0x7f43400b3080 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.906+0000 7f4335ffb640 1 -- 172.21.15.113:0/2206120187 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f433c044020 con 0x7f43400b3080 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.906+0000 7f4335ffb640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432406be30 0x7f432406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.906+0000 7f4335ffb640 1 -- 172.21.15.113:0/2206120187 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f433c0abdc0 con 0x7f43400b3080 2024-04-25T00:02:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.906+0000 7f4344fe8640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432406be30 0x7f432406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.907+0000 7f4344fe8640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432406be30 0x7f432406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f43401f5bb0 tx=0x7f4330007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.907+0000 7f434a3fd740 1 Processor -- start 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.907+0000 7f434a3fd740 1 -- start start 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e94534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 0x5651e9483000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483540 0x5651e9458350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f434a3fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5651e93bbf00 con 0x5651e94a0b60 2024-04-25T00:02:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5651e9364750 con 0x5651e9483c80 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5651e93b7ec0 con 0x5651e9483540 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e94534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e94534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40160/0 (socket says 172.21.15.113:40160) 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 -- 172.21.15.113:0/2389770671 learned_addr learned my addr 172.21.15.113:0/2389770671 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 -- 172.21.15.113:0/2389770671 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483540 msgr2=0x5651e9458350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 --2- 172.21.15.113:0/2389770671 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483540 0x5651e9458350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 -- 172.21.15.113:0/2389770671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 msgr2=0x5651e9483000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 --2- 172.21.15.113:0/2389770671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 0x5651e9483000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.908+0000 7f43457e9640 1 -- 172.21.15.113:0/2389770671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e90d02d0 con 0x5651e94a0b60 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f43457e9640 1 --2- 172.21.15.113:0/2389770671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e94534f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f433c0aa230 tx=0x7f433c009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4471a94ef27faaa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f4322ffd640 1 -- 172.21.15.113:0/2389770671 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c082d10 con 0x5651e94a0b60 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f4322ffd640 1 -- 172.21.15.113:0/2389770671 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433c0a9070 con 0x5651e94a0b60 2024-04-25T00:02:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f4322ffd640 1 -- 172.21.15.113:0/2389770671 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c04dde0 con 0x5651e94a0b60 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 -- 172.21.15.113:0/2389770671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 msgr2=0x5651e94534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2389770671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e94534f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f433c0aa230 tx=0x7f433c009610 comp rx=0 tx=0).stop 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 -- 172.21.15.113:0/2389770671 shutdown_connections 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2389770671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 0x5651e9483000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2389770671 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483540 0x5651e9458350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2389770671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e94534f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 -- 172.21.15.113:0/2389770671 >> 172.21.15.113:0/2389770671 conn(0x5651e9455760 msgr2=0x5651e949e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 -- 172.21.15.113:0/2389770671 shutdown_connections 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.909+0000 7f434a3fd740 1 -- 172.21.15.113:0/2389770671 wait complete. 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.910+0000 7f434a3fd740 1 -- start start 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.910+0000 7f434a3fd740 1 Processor -- start 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e9483c80 0x5651e9458470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94a0b60 0x5651e947deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94acff0 0x5651e947e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5651e93bbf00 con 0x5651e9483c80 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5651e9377cf0 con 0x5651e94acff0 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5651e9457910 con 0x5651e94a0b60 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 10 client.0 _ll_register_callbacks cb 0x5651e93776e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e9483c80 0x5651e9458470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e9483c80 0x5651e9458470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40176/0 (socket says 172.21.15.113:40176) 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 -- 172.21.15.113:0/1119992618 learned_addr learned my addr 172.21.15.113:0/1119992618 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f4344fe8640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94a0b60 0x5651e947deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f4345fea640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94acff0 0x5651e947e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 -- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94a0b60 msgr2=0x5651e947deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94a0b60 0x5651e947deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 -- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94acff0 msgr2=0x5651e947e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94acff0 0x5651e947e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 -- 172.21.15.113:0/1119992618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e919a6b0 con 0x5651e9483c80 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f4345fea640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94acff0 0x5651e947e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f43457e9640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e9483c80 0x5651e9458470 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f433c0093e0 tx=0x7f433c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ddc89eefc70bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f431bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f431bfff640 1 -- 172.21.15.113:0/1119992618 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c082d10 con 0x5651e9483c80 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f431bfff640 1 -- 172.21.15.113:0/1119992618 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f433c0a39b0 con 0x5651e9483c80 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.911+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c0a9070 con 0x5651e9483c80 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f431bfff640 1 -- 172.21.15.113:0/1119992618 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f433c0767a0 con 0x5651e9483c80 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 10 client.7119 fetch_fsmap learned FSMap version 43 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5651e9484cf0 con 0x5651e9483c80 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f431bfff640 1 -- 172.21.15.113:0/1119992618 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433c0a9070 con 0x5651e9483c80 2024-04-25T00:02:30.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f431bfff640 1 -- 172.21.15.113:0/1119992618 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f433c082880 con 0x5651e9483c80 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 10 client.7119 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 10 client.7119 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 20 client.7119 populate_metadata read hostname 'smithi113' 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9495330 0x5651e9495710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 4 client.7119 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651e93c9440 con 0x5651e9495330 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f4344fe8640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9495330 0x5651e9495710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f431bfff640 10 client.7119 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.912+0000 7f4344fe8640 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9495330 0x5651e9495710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aceee4ae3c2024d server_cookie=ed38514063df439c in_seq=0 out_seq=0 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f431bfff640 1 -- 172.21.15.113:0/1119992618 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651e93c9440 con 0x5651e9495330 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f431bfff640 10 client.7119 handle_command_reply: tid=0 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 client.7119 shutdown 2024-04-25T00:02:30.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e9483c80 msgr2=0x5651e9458470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e9483c80 0x5651e9458470 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f433c0093e0 tx=0x7f433c009810 comp rx=0 tx=0).stop 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 shutdown_connections 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9495330 0x5651e9495710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94acff0 0x5651e947e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f431bfff640 0 client.7119 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94a0b60 0x5651e947deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 --2- 172.21.15.113:0/1119992618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e9483c80 0x5651e9458470 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 >> 172.21.15.113:0/1119992618 conn(0x5651e9483540 msgr2=0x5651e9452c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 shutdown_connections 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 1 -- 172.21.15.113:0/1119992618 wait complete. 2024-04-25T00:02:30.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.913+0000 7f434a3fd740 20 client.7119 trim_cache size 0 max 16384 2024-04-25T00:02:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.931+0000 7f434a3fd740 1 Processor -- start 2024-04-25T00:02:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.931+0000 7f434a3fd740 1 -- start start 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 0x5651e947d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e9452fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94acff0 0x5651e94534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f434a3fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f43401f5420 con 0x5651e94a0b60 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43401527b0 con 0x5651e9483c80 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43401f4dd0 con 0x5651e94acff0 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e9452fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e9452fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40186/0 (socket says 172.21.15.113:40186) 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 -- 172.21.15.113:0/2629257548 learned_addr learned my addr 172.21.15.113:0/2629257548 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 -- 172.21.15.113:0/2629257548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94acff0 msgr2=0x5651e94534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 --2- 172.21.15.113:0/2629257548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94acff0 0x5651e94534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 -- 172.21.15.113:0/2629257548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 msgr2=0x5651e947d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 --2- 172.21.15.113:0/2629257548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 0x5651e947d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 -- 172.21.15.113:0/2629257548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e945abb0 con 0x5651e94a0b60 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.932+0000 7f4344fe8640 1 --2- 172.21.15.113:0/2629257548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e9452fa0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4340155280 tx=0x7f4330010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79995e4d2198afe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f43227fc640 1 -- 172.21.15.113:0/2629257548 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330028070 con 0x5651e94a0b60 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f43227fc640 1 -- 172.21.15.113:0/2629257548 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330007110 con 0x5651e94a0b60 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f43227fc640 1 -- 172.21.15.113:0/2629257548 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300074e0 con 0x5651e94a0b60 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 -- 172.21.15.113:0/2629257548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 msgr2=0x5651e9452fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2629257548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e9452fa0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4340155280 tx=0x7f4330010000 comp rx=0 tx=0).stop 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 -- 172.21.15.113:0/2629257548 shutdown_connections 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2629257548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e94acff0 0x5651e94534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2629257548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94a0b60 0x5651e9452fa0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2629257548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e9483c80 0x5651e947d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 -- 172.21.15.113:0/2629257548 >> 172.21.15.113:0/2629257548 conn(0x5651e9483540 msgr2=0x5651e9452250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 -- 172.21.15.113:0/2629257548 shutdown_connections 2024-04-25T00:02:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.933+0000 7f434a3fd740 1 -- 172.21.15.113:0/2629257548 wait complete. 2024-04-25T00:02:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.934+0000 7f434a3fd740 1 -- start start 2024-04-25T00:02:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.934+0000 7f434a3fd740 1 Processor -- start 2024-04-25T00:02:30.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483c80 0x5651e9505290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94a0b60 0x5651e9452d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94acff0 0x5651e94532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5651e93bbf00 con 0x5651e94acff0 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5651e9457910 con 0x5651e94a0b60 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5651e9377cf0 con 0x5651e9483c80 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 10 client.0 _ll_register_callbacks cb 0x7f433c07a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94acff0 0x5651e94532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94acff0 0x5651e94532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40188/0 (socket says 172.21.15.113:40188) 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 -- 172.21.15.113:0/2387547702 learned_addr learned my addr 172.21.15.113:0/2387547702 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 -- 172.21.15.113:0/2387547702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483c80 msgr2=0x5651e9505290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483c80 0x5651e9505290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 -- 172.21.15.113:0/2387547702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94a0b60 msgr2=0x5651e9452d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94a0b60 0x5651e9452d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 -- 172.21.15.113:0/2387547702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e9453a20 con 0x5651e94acff0 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f4345fea640 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94acff0 0x5651e94532b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f433800cef0 tx=0x7f433800e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec28e3b13091519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f431bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f431bfff640 1 -- 172.21.15.113:0/2387547702 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338018020 con 0x5651e94acff0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f431bfff640 1 -- 172.21.15.113:0/2387547702 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f4338013450 con 0x5651e94acff0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4330007110 con 0x5651e94acff0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f431bfff640 1 -- 172.21.15.113:0/2387547702 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4338022050 con 0x5651e94acff0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 10 client.7125 fetch_fsmap learned FSMap version 43 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.935+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5651e949beb0 con 0x5651e94acff0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f431bfff640 1 -- 172.21.15.113:0/2387547702 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330007110 con 0x5651e94acff0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f431bfff640 1 -- 172.21.15.113:0/2387547702 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f4338025020 con 0x5651e94acff0 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 10 client.7125 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 10 client.7125 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 20 client.7125 populate_metadata read hostname 'smithi113' 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9496550 0x5651e949f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 4 client.7125 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "status"}) -- 0x5651e9497a00 con 0x5651e9496550 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f43457e9640 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9496550 0x5651e949f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f431bfff640 10 client.7125 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f43457e9640 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9496550 0x5651e949f710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24369601e68ac960 server_cookie=37a090243e2d1469 in_seq=0 out_seq=0 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f431bfff640 1 -- 172.21.15.113:0/2387547702 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5651e9497a00 con 0x5651e9496550 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f431bfff640 10 client.7125 handle_command_reply: tid=0 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 client.7125 shutdown 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94acff0 msgr2=0x5651e94532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94acff0 0x5651e94532b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f433800cef0 tx=0x7f433800e5a0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 shutdown_connections 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x5651e9496550 0x5651e949f710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651e94a0b60 0x5651e9452d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5651e94acff0 0x5651e94532b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 --2- 172.21.15.113:0/2387547702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651e9483c80 0x5651e9505290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 >> 172.21.15.113:0/2387547702 conn(0x5651e9483540 msgr2=0x5651e94ac5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.936+0000 7f431bfff640 0 client.7125 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 shutdown_connections 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f434a3fd740 1 -- 172.21.15.113:0/2387547702 wait complete. 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f434a3fd740 20 client.7125 trim_cache size 0 max 16384 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:02:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 6951, 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 8.9357451460000004, 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 43, 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 76, 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 14.017297057 2024-04-25T00:02:30.944 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432406be30 msgr2=0x7f432406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432406be30 0x7f432406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f43401f5bb0 tx=0x7f4330007990 comp rx=0 tx=0).stop 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43400b3080 msgr2=0x7f434015b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43400b3080 0x7f434015b950 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f433c034040 tx=0x7f433c004a40 comp rx=0 tx=0).stop 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 shutdown_connections 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432406be30 0x7f432406e2f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f434015e2d0 0x7f43401541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43400b4a40 0x7f434014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 --2- 172.21.15.113:0/2206120187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43400b3080 0x7f434015b950 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 >> 172.21.15.113:0/2206120187 conn(0x7f434006b0f0 msgr2=0x7f43400b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 shutdown_connections 2024-04-25T00:02:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:30.937+0000 7f4347a4d640 1 -- 172.21.15.113:0/2206120187 wait complete. 2024-04-25T00:02:30.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:02:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.046+0000 7f8a2ed29640 1 Processor -- start 2024-04-25T00:02:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.046+0000 7f8a2ed29640 1 -- start start 2024-04-25T00:02:31.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a2ed29640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8a2815e320 0x7f8a280d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a2ed29640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a280d3de0 con 0x7f8a280d2290 2024-04-25T00:02:31.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a2ed29640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a280d2800 con 0x7f8a280d0b00 2024-04-25T00:02:31.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a2ed29640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8a280d3410 con 0x7f8a2815e320 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a27fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8a280d2290 0x7f8a280b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52920/0 (socket says 172.21.15.113:52920) 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a27fff640 1 -- 172.21.15.113:0/3374961548 learned_addr learned my addr 172.21.15.113:0/3374961548 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2611007797 0 0) 0x7f8a280d3de0 con 0x7f8a280d2290 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a18003520 con 0x7f8a280d2290 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a2cac5640 1 --2- 172.21.15.113:0/3374961548 >> v2:172.21.15.178:3300/0 conn(0x7f8a2815e320 0x7f8a280d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2081360733 0 0) 0x7f8a18003520 con 0x7f8a280d2290 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 >> v2:172.21.15.178:3300/0 conn(0x7f8a2815e320 msgr2=0x7f8a280d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 --2- 172.21.15.113:0/3374961548 >> v2:172.21.15.178:3300/0 conn(0x7f8a2815e320 0x7f8a280d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 >> v1:172.21.15.178:6789/0 conn(0x7f8a280d0b00 legacy=0x7f8a280b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a280d2ae0 con 0x7f8a280d2290 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8a14001670 con 0x7f8a280d2290 2024-04-25T00:02:31.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 >> v1:172.21.15.113:6789/0 conn(0x7f8a280d2290 legacy=0x7f8a280b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a180039b0 0x7f8a18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a18004800 0x7f8a18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180051c0 0x7f8a1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a280d3410 con 0x7f8a180039b0 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1800c070 con 0x7f8a180051c0 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1800c270 con 0x7f8a18004800 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.047+0000 7f8a2cac5640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a180039b0 0x7f8a18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2d2c6640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180051c0 0x7f8a1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a27fff640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a18004800 0x7f8a18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2cac5640 1 -- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a18004800 msgr2=0x7f8a18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2cac5640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a18004800 0x7f8a18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2cac5640 1 -- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180051c0 msgr2=0x7f8a1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:31.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2cac5640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180051c0 0x7f8a1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2cac5640 1 -- 172.21.15.113:0/3374961548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a18003520 con 0x7f8a180039b0 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2d2c6640 1 -- 172.21.15.113:0/3374961548 reap_dead start 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a27fff640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a18004800 0x7f8a18004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.048+0000 7f8a2cac5640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a180039b0 0x7f8a18003dd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009ac0 tx=0x7f8a18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c063501c6eecad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c036070 con 0x7f8a180039b0 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c031400 con 0x7f8a180039b0 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a277fe640 1 -- 172.21.15.113:0/3374961548 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c009690 con 0x7f8a180039b0 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3374961548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a180039b0 msgr2=0x7f8a18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a180039b0 0x7f8a18003dd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009ac0 tx=0x7f8a18005830 comp rx=0 tx=0).stop 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3374961548 shutdown_connections 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3374961548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a180039b0 0x7f8a18003dd0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3374961548 >> 172.21.15.113:0/3374961548 conn(0x7f8a280b5c20 msgr2=0x7f8a280b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3374961548 shutdown_connections 2024-04-25T00:02:31.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.049+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3374961548 wait complete. 2024-04-25T00:02:31.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.050+0000 7f8a2ed29640 1 Processor -- start 2024-04-25T00:02:31.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.050+0000 7f8a2ed29640 1 -- start start 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a2ed29640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180039b0 0x7f8a2814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a2ed29640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a2814ea70 0x7f8a280cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a2ed29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a280cd730 0x7f8a281f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a2ed29640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2815e700 con 0x7f8a2814ea70 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a2ed29640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a280d30a0 con 0x7f8a180039b0 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a2ed29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2815e880 con 0x7f8a280cd730 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a2814ea70 0x7f8a280cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a2814ea70 0x7f8a280cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40202/0 (socket says 172.21.15.113:40202) 2024-04-25T00:02:31.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 -- 172.21.15.113:0/3303491852 learned_addr learned my addr 172.21.15.113:0/3303491852 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 -- 172.21.15.113:0/3303491852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a280cd730 msgr2=0x7f8a281f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a280cd730 0x7f8a281f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 -- 172.21.15.113:0/3303491852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180039b0 msgr2=0x7f8a2814e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180039b0 0x7f8a2814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.051+0000 7f8a27fff640 1 -- 172.21.15.113:0/3303491852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2814eef0 con 0x7f8a2814ea70 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.052+0000 7f8a27fff640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a2814ea70 0x7f8a280cd1f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e780 tx=0x7f8a1400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.052+0000 7f8a257fa640 1 -- 172.21.15.113:0/3303491852 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400c8d0 con 0x7f8a2814ea70 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.052+0000 7f8a257fa640 1 -- 172.21.15.113:0/3303491852 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a14018070 con 0x7f8a2814ea70 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.052+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2814f1d0 con 0x7f8a2814ea70 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.052+0000 7f8a257fa640 1 -- 172.21.15.113:0/3303491852 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400ccb0 con 0x7f8a2814ea70 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.052+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2814f790 con 0x7f8a2814ea70 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.053+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2814f990 con 0x7f8a2814ea70 2024-04-25T00:02:31.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.053+0000 7f8a257fa640 1 -- 172.21.15.113:0/3303491852 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a1401d460 con 0x7f8a2814ea70 2024-04-25T00:02:31.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.054+0000 7f8a257fa640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a0c06be30 0x7f8a0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:31.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.054+0000 7f8a2cac5640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a0c06be30 0x7f8a0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:31.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.054+0000 7f8a257fa640 1 -- 172.21.15.113:0/3303491852 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f8a1408ad40 con 0x7f8a2814ea70 2024-04-25T00:02:31.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.054+0000 7f8a2cac5640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a0c06be30 0x7f8a0c06e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c033000 tx=0x7f8a1c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:31.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.058+0000 7f8a257fa640 1 -- 172.21.15.113:0/3303491852 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a2814f990 con 0x7f8a2814ea70 2024-04-25T00:02:31.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.267+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8a280691a0 con 0x7f8a2814ea70 2024-04-25T00:02:31.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.268+0000 7f8a257fa640 1 -- 172.21.15.113:0/3303491852 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f8a14059780 con 0x7f8a2814ea70 2024-04-25T00:02:31.268 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:02:31.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a0c06be30 msgr2=0x7f8a0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:31.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a0c06be30 0x7f8a0c06e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c033000 tx=0x7f8a1c02e000 comp rx=0 tx=0).stop 2024-04-25T00:02:31.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a2814ea70 msgr2=0x7f8a280cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:31.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a2814ea70 0x7f8a280cd1f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e780 tx=0x7f8a1400ec40 comp rx=0 tx=0).stop 2024-04-25T00:02:31.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 shutdown_connections 2024-04-25T00:02:31.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a0c06be30 0x7f8a0c06e2f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a280cd730 0x7f8a281f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a2814ea70 0x7f8a280cd1f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 --2- 172.21.15.113:0/3303491852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a180039b0 0x7f8a2814e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:31.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 >> 172.21.15.113:0/3303491852 conn(0x7f8a280b5c20 msgr2=0x7f8a280b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:31.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 shutdown_connections 2024-04-25T00:02:31.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31.270+0000 7f8a2ed29640 1 -- 172.21.15.113:0/3303491852 wait complete. 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:02:31.281 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:02:31.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:31.282 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:02:31.297 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42529sec preferred_lft 42529sec 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:02:31.298 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:02:31.299 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:31.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:31.300 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:02:31.359 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:02:31.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:31.361 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:02:31.416 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:02:31.416 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:02:31.416 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:02:31.416 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:02:31.416 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:02:31.417 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:02:31.417 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:02:31.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:02:31.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:02:31.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:02:31.417 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:02:31.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:02:31.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:31.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:31.540 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:02:31.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:02:31.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:02:31.555 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:02:31.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:31.555 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:02:31.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:02:31.619 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:02:31.674 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:02:31.674 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:02:31.674 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:02:31.728 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:02:31.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:31.728 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:02:31.809 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:02:31.809 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:02:31.809 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:02:31.818 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:02:31.819 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:02:31.819 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:02:31.819 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:02:31.819 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:02:31.822 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:02:31.822 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:02:31.822 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:02:31.822 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:02:31.822 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:02:31.822 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:02:31.823 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:02:31.823 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:02:31.823 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:02:31.823 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:02:31.823 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:02:31.823 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:02:31.823 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:02:31.825 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:02:31.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:31.825 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:02:31.889 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:02:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:02:31.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:31.984 INFO:teuthology.orchestra.run.smithi113.stdout:client6103 2024-04-25T00:02:31.984 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:02:31.985 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:32.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T00:02:32.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.140+0000 7f4ce3d63640 1 Processor -- start 2024-04-25T00:02:32.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.140+0000 7f4ce3d63640 1 -- start start 2024-04-25T00:02:32.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce3d63640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4cdc06e0d0 0x7f4cdc06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:32.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce3d63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:32.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce3d63640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cdc072f70 con 0x7f4cdc144a80 2024-04-25T00:02:32.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce3d63640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4cdc06ea70 con 0x7f4cdc06e0d0 2024-04-25T00:02:32.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce3d63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc0ce000 con 0x7f4cdc0cd630 2024-04-25T00:02:32.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce12fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4cdc06e0d0 0x7f4cdc06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:32.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce2300640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:32.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce1aff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f4cdc144a80 0x7f4cdc0b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:54282/0 (socket says 172.21.15.113:54282) 2024-04-25T00:02:32.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.141+0000 7f4ce1aff640 1 -- 172.21.15.113:0/2325173987 learned_addr learned my addr 172.21.15.113:0/2325173987 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:02:32.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.142+0000 7f4ce12fe640 1 -- 172.21.15.113:0/2325173987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 msgr2=0x7f4cdc0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.142+0000 7f4ce12fe640 1 --2- 172.21.15.113:0/2325173987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc0cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.142+0000 7f4ce12fe640 1 -- 172.21.15.113:0/2325173987 >> v1:172.21.15.178:6789/0 conn(0x7f4cdc144a80 legacy=0x7f4cdc0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.142+0000 7f4ce12fe640 1 -- 172.21.15.113:0/2325173987 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc0b2fd0 con 0x7f4cdc06e0d0 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.142+0000 7f4ce12fe640 1 --2- 172.21.15.113:0/2325173987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 0x7f4cdc06e530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0097a0 tx=0x7f4ccc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c7dabbbeb3733ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce0afd640 1 -- 172.21.15.113:0/2325173987 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc037070 con 0x7f4cdc06e0d0 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce0afd640 1 -- 172.21.15.113:0/2325173987 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ccc02ae50 con 0x7f4cdc06e0d0 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce0afd640 1 -- 172.21.15.113:0/2325173987 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc032660 con 0x7f4cdc06e0d0 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2325173987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 msgr2=0x7f4cdc06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2325173987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 0x7f4cdc06e530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0097a0 tx=0x7f4ccc02a100 comp rx=0 tx=0).stop 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2325173987 shutdown_connections 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2325173987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2325173987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 0x7f4cdc06e530 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:32.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2325173987 >> 172.21.15.113:0/2325173987 conn(0x7f4cdc0b1b30 msgr2=0x7f4cdc0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:32.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2325173987 shutdown_connections 2024-04-25T00:02:32.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.143+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2325173987 wait complete. 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.144+0000 7f4ce3d63640 1 Processor -- start 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.144+0000 7f4ce3d63640 1 -- start start 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce3d63640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 0x7f4cdc15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce3d63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc13a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce3d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cdc144a80 0x7f4cdc13cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce3d63640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc0b41c0 con 0x7f4cdc144a80 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce3d63640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc0b3590 con 0x7f4cdc06e0d0 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce3d63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc0b4340 con 0x7f4cdc0cd630 2024-04-25T00:02:32.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce2300640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cdc144a80 0x7f4cdc13cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce2300640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cdc144a80 0x7f4cdc13cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40210/0 (socket says 172.21.15.113:40210) 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce2300640 1 -- 172.21.15.113:0/2304278018 learned_addr learned my addr 172.21.15.113:0/2304278018 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce1aff640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 0x7f4cdc15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.145+0000 7f4ce12fe640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc13a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce2300640 1 -- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 msgr2=0x7f4cdc13a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce2300640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc13a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce2300640 1 -- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 msgr2=0x7f4cdc15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce2300640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 0x7f4cdc15e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce2300640 1 -- 172.21.15.113:0/2304278018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc13ae80 con 0x7f4cdc144a80 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce2300640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cdc144a80 0x7f4cdc13cc70 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e960 tx=0x7f4cd400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4cd27fc640 1 -- 172.21.15.113:0/2304278018 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4017070 con 0x7f4cdc144a80 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4cd27fc640 1 -- 172.21.15.113:0/2304278018 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd400c980 con 0x7f4cdc144a80 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc15c500 con 0x7f4cdc144a80 2024-04-25T00:02:32.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4cd27fc640 1 -- 172.21.15.113:0/2304278018 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400cd70 con 0x7f4cdc144a80 2024-04-25T00:02:32.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.146+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc15ca80 con 0x7f4cdc144a80 2024-04-25T00:02:32.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.147+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cdc13fcd0 con 0x7f4cdc144a80 2024-04-25T00:02:32.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.148+0000 7f4cd27fc640 1 -- 172.21.15.113:0/2304278018 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4cd401a070 con 0x7f4cdc144a80 2024-04-25T00:02:32.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.148+0000 7f4cd27fc640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc006be30 0x7f4cc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:32.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.148+0000 7f4cd27fc640 1 -- 172.21.15.113:0/2304278018 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7f4cd408ad20 con 0x7f4cdc144a80 2024-04-25T00:02:32.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.148+0000 7f4ce1aff640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc006be30 0x7f4cc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:32.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.148+0000 7f4ce1aff640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc006be30 0x7f4cc006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc13b520 tx=0x7f4cd8001990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:32.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.152+0000 7f4cd27fc640 1 -- 172.21.15.113:0/2304278018 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4cdc13fcd0 con 0x7f4cdc144a80 2024-04-25T00:02:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:32.367+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f4cdc0b29a0 con 0x7f4cdc144a80 2024-04-25T00:02:33.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.033+0000 7f4cd27fc640 1 -- 172.21.15.113:0/2304278018 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v44) ==== 167+0+0 (secure 0 0 0) 0x7f4cd4059110 con 0x7f4cdc144a80 2024-04-25T00:02:33.033 INFO:teuthology.orchestra.run.smithi113.stderr:enabled new snapshots 2024-04-25T00:02:33.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.036+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc006be30 msgr2=0x7f4cc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.036+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc006be30 0x7f4cc006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc13b520 tx=0x7f4cd8001990 comp rx=0 tx=0).stop 2024-04-25T00:02:33.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.036+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cdc144a80 msgr2=0x7f4cdc13cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.036+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cdc144a80 0x7f4cdc13cc70 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e960 tx=0x7f4cd400ee20 comp rx=0 tx=0).stop 2024-04-25T00:02:33.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 shutdown_connections 2024-04-25T00:02:33.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc006be30 0x7f4cc006e2f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cdc144a80 0x7f4cdc13cc70 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4cdc0cd630 0x7f4cdc13a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 --2- 172.21.15.113:0/2304278018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cdc06e0d0 0x7f4cdc15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 >> 172.21.15.113:0/2304278018 conn(0x7f4cdc0b1b30 msgr2=0x7f4cdc072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 shutdown_connections 2024-04-25T00:02:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:33.037+0000 7f4ce3d63640 1 -- 172.21.15.113:0/2304278018 wait complete. 2024-04-25T00:02:33.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:33.049 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T00:02:33.067 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:02:33.068 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:02:33.124 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> import uuid 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> n = 2000 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> create_hard_links = False 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> if 3 > 0: 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> create_hard_links = True 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> dpath = os.path.dirname(path) 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> fnameprefix = os.path.basename(path) 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> os.makedirs(dpath, exist_ok=True) 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> try: 2024-04-25T00:02:33.125 DEBUG:teuthology.orchestra.run.smithi113:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> for i in range(n): 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> f.write(f"{i}") 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> f.flush() 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(f.fileno()) 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> os.unlink(fpath) 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(dirfd) 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> if create_hard_links: 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> for j in range(3): 2024-04-25T00:02:33.126 DEBUG:teuthology.orchestra.run.smithi113:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T00:02:33.127 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:02:33.127 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(dirfd) 2024-04-25T00:02:33.127 DEBUG:teuthology.orchestra.run.smithi113:> finally: 2024-04-25T00:02:33.127 DEBUG:teuthology.orchestra.run.smithi113:> os.close(dirfd) 2024-04-25T00:02:33.127 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:02:51.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:51.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:51.594 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T00:02:51.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:51.851 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T00:02:52.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:02:52.328 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:03:19.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:03:19.270 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T00:03:19.325 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T00:03:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.453+0000 7ff554831640 1 Processor -- start 2024-04-25T00:03:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.454+0000 7ff554831640 1 -- start start 2024-04-25T00:03:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.454+0000 7ff554831640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5500b5ca0 0x7ff5500b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.454+0000 7ff554831640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b73c0 0x7ff5500b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.454+0000 7ff554831640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5500d2e00 con 0x7ff55015e510 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.454+0000 7ff554831640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff5500b6640 con 0x7ff5500b5ca0 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.454+0000 7ff554831640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5500d30e0 con 0x7ff5500b73c0 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff547fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5500b5ca0 0x7ff5500b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b73c0 0x7ff5500b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff547fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5500b5ca0 0x7ff5500b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:45450/0 (socket says 172.21.15.113:45450) 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b73c0 0x7ff5500b7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:35948/0 (socket says 172.21.15.113:35948) 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff54e59c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff55015e510 0x7ff5500d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:43428/0 (socket says 172.21.15.113:43428) 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff547fff640 1 -- 172.21.15.113:0/706561752 learned_addr learned my addr 172.21.15.113:0/706561752 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff547fff640 1 -- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b73c0 msgr2=0x7ff5500b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff547fff640 1 --2- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b73c0 0x7ff5500b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff547fff640 1 -- 172.21.15.113:0/706561752 >> v1:172.21.15.178:6789/0 conn(0x7ff55015e510 legacy=0x7ff5500d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff547fff640 1 -- 172.21.15.113:0/706561752 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5500d33c0 con 0x7ff5500b5ca0 2024-04-25T00:03:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff54dd9b640 1 -- 172.21.15.113:0/706561752 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2186752019 0 0) 0x7ff5500d2e00 con 0x7ff55015e510 2024-04-25T00:03:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.455+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b73c0 0x7ff5500b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff547fff640 1 --2- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5500b5ca0 0x7ff5500b6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff538009870 tx=0x7ff53802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8aae720d5d801a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff54dd9b640 1 -- 172.21.15.113:0/706561752 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538037070 con 0x7ff5500b5ca0 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff54dd9b640 1 -- 172.21.15.113:0/706561752 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff538032440 con 0x7ff5500b5ca0 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff54dd9b640 1 -- 172.21.15.113:0/706561752 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538032810 con 0x7ff5500b5ca0 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 -- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5500b5ca0 msgr2=0x7ff5500b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 --2- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5500b5ca0 0x7ff5500b6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff538009870 tx=0x7ff53802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 -- 172.21.15.113:0/706561752 shutdown_connections 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 --2- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b73c0 0x7ff5500b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 --2- 172.21.15.113:0/706561752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5500b5ca0 0x7ff5500b6100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 -- 172.21.15.113:0/706561752 >> 172.21.15.113:0/706561752 conn(0x7ff55006b720 msgr2=0x7ff5500d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 -- 172.21.15.113:0/706561752 shutdown_connections 2024-04-25T00:03:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.456+0000 7ff554831640 1 -- 172.21.15.113:0/706561752 wait complete. 2024-04-25T00:03:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 Processor -- start 2024-04-25T00:03:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 -- start start 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b5ca0 0x7ff550150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5500b73c0 0x7ff55015fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff55015e510 0x7ff5500ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff5500d3ea0 con 0x7ff5500b73c0 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5500d3980 con 0x7ff55015e510 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.458+0000 7ff554831640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5500d2a90 con 0x7ff5500b5ca0 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5500b73c0 0x7ff55015fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5500b73c0 0x7ff55015fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54534/0 (socket says 172.21.15.113:54534) 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 -- 172.21.15.113:0/2028881501 learned_addr learned my addr 172.21.15.113:0/2028881501 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff55015e510 0x7ff5500ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff54e59c640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b5ca0 0x7ff550150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 -- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b5ca0 msgr2=0x7ff550150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b5ca0 0x7ff550150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 -- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff55015e510 msgr2=0x7ff5500ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff55015e510 0x7ff5500ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 -- 172.21.15.113:0/2028881501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55015eb20 con 0x7ff5500b73c0 2024-04-25T00:03:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff55015e510 0x7ff5500ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff547fff640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5500b73c0 0x7ff55015fbc0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff538035040 tx=0x7ff5380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff546ffd640 1 -- 172.21.15.113:0/2028881501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538037040 con 0x7ff5500b73c0 2024-04-25T00:03:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff546ffd640 1 -- 172.21.15.113:0/2028881501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff538004230 con 0x7ff5500b73c0 2024-04-25T00:03:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff546ffd640 1 -- 172.21.15.113:0/2028881501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538003610 con 0x7ff5500b73c0 2024-04-25T00:03:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55015ee00 con 0x7ff5500b73c0 2024-04-25T00:03:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.459+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55015f3c0 con 0x7ff5500b73c0 2024-04-25T00:03:19.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.461+0000 7ff546ffd640 1 -- 172.21.15.113:0/2028881501 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff53802ad10 con 0x7ff5500b73c0 2024-04-25T00:03:19.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.461+0000 7ff546ffd640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff53006be30 0x7ff53006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.461+0000 7ff546ffd640 1 -- 172.21.15.113:0/2028881501 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7ff5380ab790 con 0x7ff5500b73c0 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.461+0000 7ff54e59c640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff53006be30 0x7ff53006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.461+0000 7ff54e59c640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff53006be30 0x7ff53006e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff540000f60 tx=0x7ff540007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.462+0000 7ff557619740 1 Processor -- start 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.462+0000 7ff557619740 1 -- start start 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a57b80 0x55ba46a0a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a3aca0 0x55ba46a3a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff557619740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a3a560 0x55ba46a0f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff557619740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ba46973330 con 0x55ba46a3a560 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba4691bab0 con 0x55ba46a3aca0 2024-04-25T00:03:19.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba4696f2f0 con 0x55ba46a57b80 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a3a560 0x55ba46a0f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a3a560 0x55ba46a0f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54542/0 (socket says 172.21.15.113:54542) 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 -- 172.21.15.113:0/802218882 learned_addr learned my addr 172.21.15.113:0/802218882 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54e59c640 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a57b80 0x55ba46a0a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff547fff640 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a3aca0 0x55ba46a3a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 -- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a57b80 msgr2=0x55ba46a0a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a57b80 0x55ba46a0a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 -- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a3aca0 msgr2=0x55ba46a3a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a3aca0 0x55ba46a3a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 -- 172.21.15.113:0/802218882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba467517d0 con 0x55ba46a3a560 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.463+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a3a560 0x55ba46a0f370 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff548003c60 tx=0x7ff548009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2832fab5b508057f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff52ffff640 1 -- 172.21.15.113:0/802218882 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548019070 con 0x55ba46a3a560 2024-04-25T00:03:19.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff52ffff640 1 -- 172.21.15.113:0/802218882 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54800e900 con 0x55ba46a3a560 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff52ffff640 1 -- 172.21.15.113:0/802218882 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54800ecf0 con 0x55ba46a3a560 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff557619740 1 -- 172.21.15.113:0/802218882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a3a560 msgr2=0x55ba46a0f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff557619740 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a3a560 0x55ba46a0f370 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff548003c60 tx=0x7ff548009750 comp rx=0 tx=0).stop 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff557619740 1 -- 172.21.15.113:0/802218882 shutdown_connections 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff557619740 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a3aca0 0x55ba46a3a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff557619740 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a3a560 0x55ba46a0f370 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff557619740 1 --2- 172.21.15.113:0/802218882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a57b80 0x55ba46a0a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.464+0000 7ff557619740 1 -- 172.21.15.113:0/802218882 >> 172.21.15.113:0/802218882 conn(0x55ba46a0c780 msgr2=0x55ba46a55020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.465+0000 7ff557619740 1 -- 172.21.15.113:0/802218882 shutdown_connections 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.465+0000 7ff557619740 1 -- 172.21.15.113:0/802218882 wait complete. 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.465+0000 7ff557619740 1 -- start start 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.465+0000 7ff557619740 1 Processor -- start 2024-04-25T00:03:19.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a35650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46a3d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff557619740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff557619740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ba46a0e930 con 0x55ba46a35cc0 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba4692ef40 con 0x55ba46a57b80 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba46a0dd20 con 0x55ba46a3aca0 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff557619740 10 client.0 _ll_register_callbacks cb 0x55ba46686be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54544/0 (socket says 172.21.15.113:54544) 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 -- 172.21.15.113:0/533641366 learned_addr learned my addr 172.21.15.113:0/533641366 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff547fff640 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46a3d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54e59c640 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a35650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 -- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 msgr2=0x55ba46a35650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a35650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 -- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 msgr2=0x55ba46a3d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46a3d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.466+0000 7ff54ed9d640 1 -- 172.21.15.113:0/533641366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba466470f0 con 0x55ba46a35cc0 2024-04-25T00:03:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3dd50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff548018000 tx=0x7ff548004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f067957dff747927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff52d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff52d7fa640 1 -- 172.21.15.113:0/533641366 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54802d060 con 0x55ba46a35cc0 2024-04-25T00:03:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff52d7fa640 1 -- 172.21.15.113:0/533641366 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7ff54800e700 con 0x55ba46a35cc0 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54800e900 con 0x55ba46a35cc0 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff52d7fa640 1 -- 172.21.15.113:0/533641366 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff548033050 con 0x55ba46a35cc0 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 10 client.7143 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55ba467b1f60 con 0x55ba46a35cc0 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff52d7fa640 1 -- 172.21.15.113:0/533641366 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54800e900 con 0x55ba46a35cc0 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff52d7fa640 1 -- 172.21.15.113:0/533641366 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7ff548036020 con 0x55ba46a35cc0 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 10 client.7143 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 10 client.7143 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 20 client.7143 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a3b660 0x55ba46a3ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 4 client.7143 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba46980870 con 0x55ba46a3b660 2024-04-25T00:03:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.467+0000 7ff54e59c640 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a3b660 0x55ba46a3ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff54e59c640 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a3b660 0x55ba46a3ba40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a849189e108c4 server_cookie=d7bbd623fae3c02f in_seq=0 out_seq=0 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff52d7fa640 10 client.7143 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff52d7fa640 1 -- 172.21.15.113:0/533641366 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba46980870 con 0x55ba46a3b660 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff52d7fa640 10 client.7143 handle_command_reply: tid=0 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 client.7143 shutdown 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 msgr2=0x55ba46a3dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3dd50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff548018000 tx=0x7ff548004ab0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 shutdown_connections 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a3b660 0x55ba46a3ba40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46a3d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3dd50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 --2- 172.21.15.113:0/533641366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a35650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff52d7fa640 0 client.7143 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 >> 172.21.15.113:0/533641366 conn(0x55ba46a3a560 msgr2=0x55ba46a58440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 shutdown_connections 2024-04-25T00:03:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.468+0000 7ff557619740 1 -- 172.21.15.113:0/533641366 wait complete. 2024-04-25T00:03:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.469+0000 7ff557619740 20 client.7143 trim_cache size 0 max 16384 2024-04-25T00:03:19.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.484+0000 7ff557619740 1 Processor -- start 2024-04-25T00:03:19.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.484+0000 7ff557619740 1 -- start start 2024-04-25T00:03:19.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a35cc0 0x55ba46a396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a11690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff557619740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a57b80 0x55ba46a4d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff557619740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff550200800 con 0x55ba46a57b80 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501ff850 con 0x55ba46a35cc0 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5501f7f70 con 0x55ba46a3aca0 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a57b80 0x55ba46a4d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff54ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a57b80 0x55ba46a4d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54556/0 (socket says 172.21.15.113:54556) 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff54ed9d640 1 -- 172.21.15.113:0/1009657613 learned_addr learned my addr 172.21.15.113:0/1009657613 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff547fff640 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a11690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff54e59c640 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a35cc0 0x55ba46a396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff54ed9d640 1 -- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 msgr2=0x55ba46a11690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.485+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a11690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff54ed9d640 1 -- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a35cc0 msgr2=0x55ba46a396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a35cc0 0x55ba46a396b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff54ed9d640 1 -- 172.21.15.113:0/1009657613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba46a3bdb0 con 0x55ba46a57b80 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff54e59c640 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a35cc0 0x55ba46a396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff547fff640 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a11690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a57b80 0x55ba46a4d4d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff548003c50 tx=0x7ff548020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf7522d388e7dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff52f7fe640 1 -- 172.21.15.113:0/1009657613 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54803c070 con 0x55ba46a57b80 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff52f7fe640 1 -- 172.21.15.113:0/1009657613 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54800ee70 con 0x55ba46a57b80 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff52f7fe640 1 -- 172.21.15.113:0/1009657613 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548019040 con 0x55ba46a57b80 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 -- 172.21.15.113:0/1009657613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a57b80 msgr2=0x55ba46a4d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a57b80 0x55ba46a4d4d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff548003c50 tx=0x7ff548020070 comp rx=0 tx=0).stop 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 -- 172.21.15.113:0/1009657613 shutdown_connections 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a57b80 0x55ba46a4d4d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a11690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 --2- 172.21.15.113:0/1009657613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a35cc0 0x55ba46a396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 -- 172.21.15.113:0/1009657613 >> 172.21.15.113:0/1009657613 conn(0x55ba46a3a560 msgr2=0x55ba46a35460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 -- 172.21.15.113:0/1009657613 shutdown_connections 2024-04-25T00:03:19.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.486+0000 7ff557619740 1 -- 172.21.15.113:0/1009657613 wait complete. 2024-04-25T00:03:19.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.487+0000 7ff557619740 1 -- start start 2024-04-25T00:03:19.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.487+0000 7ff557619740 1 Processor -- start 2024-04-25T00:03:19.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a37f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46ad2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ba46a39df0 con 0x55ba46a35cc0 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff55015be60 con 0x55ba46a57b80 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba4697f9b0 con 0x55ba46a3aca0 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 10 client.0 _ll_register_callbacks cb 0x7ff538075d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54566/0 (socket says 172.21.15.113:54566) 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 -- 172.21.15.113:0/953270654 learned_addr learned my addr 172.21.15.113:0/953270654 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff547fff640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a37f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46ad2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 -- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 msgr2=0x55ba46a37f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a37f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 -- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 msgr2=0x55ba46ad2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46ad2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 -- 172.21.15.113:0/953270654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba466201b0 con 0x55ba46a35cc0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54ed9d640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46ad2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff54e59c640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3c750 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x55ba46a4d150 tx=0x7ff540012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b10b3c7bed461c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff52d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff52d7fa640 1 -- 172.21.15.113:0/953270654 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400298c0 con 0x55ba46a35cc0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff52d7fa640 1 -- 172.21.15.113:0/953270654 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7ff5400133f0 con 0x55ba46a35cc0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.488+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54800ee70 con 0x55ba46a35cc0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff52d7fa640 1 -- 172.21.15.113:0/953270654 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff540039050 con 0x55ba46a35cc0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 10 client.7149 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55ba46ad2940 con 0x55ba46a35cc0 2024-04-25T00:03:19.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff52d7fa640 1 -- 172.21.15.113:0/953270654 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54800ee70 con 0x55ba46a35cc0 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff52d7fa640 1 -- 172.21.15.113:0/953270654 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7ff54003c020 con 0x55ba46a35cc0 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 10 client.7149 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 10 client.7149 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 20 client.7149 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a649e0 0x55ba46ab4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 4 client.7149 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55ba4697abc0 con 0x55ba46a649e0 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff547fff640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a649e0 0x55ba46ab4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff52d7fa640 10 client.7149 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.489+0000 7ff547fff640 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a649e0 0x55ba46ab4a80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465f4251967a2ed7 server_cookie=6c6d873be3de82cf in_seq=0 out_seq=0 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff52d7fa640 1 -- 172.21.15.113:0/953270654 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17999 (crc 0 0 0) 0x55ba4697abc0 con 0x55ba46a649e0 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff52d7fa640 10 client.7149 handle_command_reply: tid=0 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 client.7149 shutdown 2024-04-25T00:03:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 msgr2=0x55ba46a3c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3c750 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x55ba46a4d150 tx=0x7ff540012000 comp rx=0 tx=0).stop 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 shutdown_connections 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba46a649e0 0x55ba46ab4a80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba46a57b80 0x55ba46ad2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba46a3aca0 0x55ba46a37f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 --2- 172.21.15.113:0/953270654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba46a35cc0 0x55ba46a3c750 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff52d7fa640 0 client.7149 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 >> 172.21.15.113:0/953270654 conn(0x55ba46a3a560 msgr2=0x55ba46a4e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 shutdown_connections 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 1 -- 172.21.15.113:0/953270654 wait complete. 2024-04-25T00:03:19.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.490+0000 7ff557619740 20 client.7149 trim_cache size 0 max 16384 2024-04-25T00:03:19.498 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:03:19.498 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:03:19.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 1015, 2024-04-25T00:03:19.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 1009, 2024-04-25T00:03:19.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 561079, 2024-04-25T00:03:19.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 13723021, 2024-04-25T00:03:19.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 8, 2024-04-25T00:03:19.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-25T00:03:19.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.130970017, 2024-04-25T00:03:19.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.061526003, 2024-04-25T00:03:19.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.030057123, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.034231742, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1009, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.246528739, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000244329 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000243, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000121 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 116496, 2024-04-25T00:03:19.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 18480 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 28714, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 51194, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 8923022, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 67250083, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 8, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 2.779184641, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 2.137482224, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.875992690, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.055791385, 2024-04-25T00:03:19.501 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 51193, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.263770882, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000024686 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 49027, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.007430760, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000151 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 108992 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.502 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2179, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 2179, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 544227, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 39224195, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.296511586, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.171162092, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.044967927, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.068051354, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2178, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.315868091, 2024-04-25T00:03:19.503 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000145026 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-25T00:03:19.504 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1250, 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.150010372, 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.014520008 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1844, 2024-04-25T00:03:19.505 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.027360394, 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000557136 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 25169, 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 25169, 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 25169, 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 421.270824930, 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.016737686 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.506 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 12, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 19677, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 19677, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-04-25T00:03:19.507 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 19676, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 133, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 133, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 2, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 45378, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 44940, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 82, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-25T00:03:19.508 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 0, 2024-04-25T00:03:19.509 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:03:19.510 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 1188, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 4036, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 4051, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 3348, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 15, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 13953 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-25T00:03:19.511 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 2000, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 4076, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 1869, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 2152, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-25T00:03:19.512 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 30330, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-04-25T00:03:19.513 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 30330, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 50, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 50, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 4, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 105876939, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 0, 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-25T00:03:19.514 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2641, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 121.755300818, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.046101969 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 0 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 2014, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 2014, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 0, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 13, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 13, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 0, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 19677, 2024-04-25T00:03:19.515 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 19678, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 1, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 133, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 2135, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 2002, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 116308, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 231788 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 27920, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 25169, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 3, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-04-25T00:03:19.516 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2000, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.764252795, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000882126 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000991419, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000165236 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6000, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 7.354427411, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001225737 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6997, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 5.497591400, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000785706 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000039846, 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000039846 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.520 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.002177280, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002177280 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-25T00:03:19.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.190413593, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.190413593 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000080605, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000080605 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.670037872, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.208754734 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.522 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2152, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 386.372331253, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.179541046 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.039018293, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.039018293 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.524 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000270906, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000270906 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.525 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.526 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8000, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.379192257, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002297399 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.527 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 1, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 0, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 18313, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 18313, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 45, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.528 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:03:19.529 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:03:19.530 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 17188281, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 6329, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 440, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 5, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-25T00:03:19.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 5504, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 79, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 21126935, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 873966, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-25T00:03:19.532 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 4700, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 101654773, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 33, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 4700, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4700, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.048612172, 2024-04-25T00:03:19.533 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.003840130 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4700, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 4791 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 4700, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 31, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 4669, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 0, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 11, 2024-04-25T00:03:19.534 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 31, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 2672, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 22, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 1869, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 36, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 22, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:03:19.535 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-25T00:03:19.536 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 78, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 3, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 8, 2024-04-25T00:03:19.537 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 8, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 98, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 0, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 30 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 1, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-25T00:03:19.538 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 32, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 1869, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 1869, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:03:19.539 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 31903, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 7586395, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 31903, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 31903, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 7586395, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.540 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 4700, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 101858494, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 4700, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 101858494, 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:19.541 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 4700, 2024-04-25T00:03:19.542 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 4700, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 4700, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 4700, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 1869, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 203721, 2024-04-25T00:03:19.543 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 1869, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 31, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 203721, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x55780bb46500": { 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 507092, 2024-04-25T00:03:19.544 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 30330, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 101682635, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 30330, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2641, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 101175543, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.546 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:03:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff53006be30 msgr2=0x7ff53006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff53006be30 0x7ff53006e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff540000f60 tx=0x7ff540007d50 comp rx=0 tx=0).stop 2024-04-25T00:03:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5500b73c0 msgr2=0x7ff55015fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5500b73c0 0x7ff55015fbc0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff538035040 tx=0x7ff5380046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 shutdown_connections 2024-04-25T00:03:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff53006be30 0x7ff53006e2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff55015e510 0x7ff5500ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5500b73c0 0x7ff55015fbc0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 --2- 172.21.15.113:0/2028881501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5500b5ca0 0x7ff550150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 >> 172.21.15.113:0/2028881501 conn(0x7ff55006b720 msgr2=0x7ff5500d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 shutdown_connections 2024-04-25T00:03:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.491+0000 7ff554831640 1 -- 172.21.15.113:0/2028881501 wait complete. 2024-04-25T00:03:19.548 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T00:03:19.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:03:19.549 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T00:03:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.649+0000 7f35df4e1640 1 Processor -- start 2024-04-25T00:03:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.650+0000 7f35df4e1640 1 -- start start 2024-04-25T00:03:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.650+0000 7f35df4e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.650+0000 7f35df4e1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d81626f0 con 0x7f35d815e510 2024-04-25T00:03:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.650+0000 7f35df4e1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d81629d0 con 0x7f35d80b61e0 2024-04-25T00:03:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.650+0000 7f35df4e1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35d8162cb0 con 0x7f35d80b9bb0 2024-04-25T00:03:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.650+0000 7f35dd27d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f35d815e510 0x7f35d8161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38876/0 (socket says 172.21.15.113:38876) 2024-04-25T00:03:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.650+0000 7f35dd27d640 1 -- 172.21.15.113:0/2425239100 learned_addr learned my addr 172.21.15.113:0/2425239100 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.651+0000 7f35dca7c640 1 --2- 172.21.15.113:0/2425239100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.651+0000 7f35cffff640 1 -- 172.21.15.113:0/2425239100 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384498466 0 0) 0x7f35d81626f0 con 0x7f35d815e510 2024-04-25T00:03:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.651+0000 7f35cffff640 1 -- 172.21.15.113:0/2425239100 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35c4003520 con 0x7f35d815e510 2024-04-25T00:03:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.651+0000 7f35dca7c640 1 -- 172.21.15.113:0/2425239100 >> v1:172.21.15.178:6789/0 conn(0x7f35d80b61e0 legacy=0x7f35d80d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.651+0000 7f35cffff640 1 -- 172.21.15.113:0/2425239100 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185428517 0 0) 0x7f35d81629d0 con 0x7f35d80b61e0 2024-04-25T00:03:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.651+0000 7f35dca7c640 1 -- 172.21.15.113:0/2425239100 >> v1:172.21.15.113:6789/0 conn(0x7f35d815e510 legacy=0x7f35d8161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.651+0000 7f35dca7c640 1 -- 172.21.15.113:0/2425239100 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80d2660 con 0x7f35d80b9bb0 2024-04-25T00:03:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35dca7c640 1 --2- 172.21.15.113:0/2425239100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80b5ca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35c80097d0 tx=0x7f35c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=815bd2e1e176a9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35cffff640 1 -- 172.21.15.113:0/2425239100 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8037070 con 0x7f35d80b9bb0 2024-04-25T00:03:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35cffff640 1 -- 172.21.15.113:0/2425239100 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c8032440 con 0x7f35d80b9bb0 2024-04-25T00:03:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35cffff640 1 -- 172.21.15.113:0/2425239100 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80327f0 con 0x7f35d80b9bb0 2024-04-25T00:03:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35df4e1640 1 -- 172.21.15.113:0/2425239100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 msgr2=0x7f35d80b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2425239100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80b5ca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35c80097d0 tx=0x7f35c802a180 comp rx=0 tx=0).stop 2024-04-25T00:03:19.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35df4e1640 1 -- 172.21.15.113:0/2425239100 shutdown_connections 2024-04-25T00:03:19.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2425239100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80b5ca0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35df4e1640 1 -- 172.21.15.113:0/2425239100 >> 172.21.15.113:0/2425239100 conn(0x7f35d806b720 msgr2=0x7f35d80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35df4e1640 1 -- 172.21.15.113:0/2425239100 shutdown_connections 2024-04-25T00:03:19.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.652+0000 7f35df4e1640 1 -- 172.21.15.113:0/2425239100 wait complete. 2024-04-25T00:03:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.653+0000 7f35df4e1640 1 Processor -- start 2024-04-25T00:03:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35df4e1640 1 -- start start 2024-04-25T00:03:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35df4e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b61e0 0x7f35d81499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35df4e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35df4e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35d815e510 0x7f35d80cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35df4e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f35d80b7540 con 0x7f35d80b61e0 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35df4e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35d80b73c0 con 0x7f35d815e510 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35df4e1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35d80d3570 con 0x7f35d80b9bb0 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35dd27d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b61e0 0x7f35d81499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35dd27d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b61e0 0x7f35d81499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54582/0 (socket says 172.21.15.113:54582) 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.654+0000 7f35dd27d640 1 -- 172.21.15.113:0/2194428662 learned_addr learned my addr 172.21.15.113:0/2194428662 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dca7c640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dda7e640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35d815e510 0x7f35d80cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dd27d640 1 -- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 msgr2=0x7f35d80cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dd27d640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dd27d640 1 -- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35d815e510 msgr2=0x7f35d80cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dd27d640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35d815e510 0x7f35d80cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dd27d640 1 -- 172.21.15.113:0/2194428662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d815cb20 con 0x7f35d80b61e0 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35dd27d640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b61e0 0x7f35d81499c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c390 tx=0x7f35d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35cdffb640 1 -- 172.21.15.113:0/2194428662 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0017070 con 0x7f35d80b61e0 2024-04-25T00:03:19.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35cdffb640 1 -- 172.21.15.113:0/2194428662 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d0005ce0 con 0x7f35d80b61e0 2024-04-25T00:03:19.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d80cce30 con 0x7f35d80b61e0 2024-04-25T00:03:19.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35cdffb640 1 -- 172.21.15.113:0/2194428662 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00072f0 con 0x7f35d80b61e0 2024-04-25T00:03:19.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.655+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d80cc1a0 con 0x7f35d80b61e0 2024-04-25T00:03:19.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.656+0000 7f35cdffb640 1 -- 172.21.15.113:0/2194428662 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35d001a070 con 0x7f35d80b61e0 2024-04-25T00:03:19.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.657+0000 7f35cdffb640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35bc06be30 0x7f35bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.657+0000 7f35cdffb640 1 -- 172.21.15.113:0/2194428662 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7f35d008a270 con 0x7f35d80b61e0 2024-04-25T00:03:19.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.657+0000 7f35dca7c640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35bc06be30 0x7f35bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.657+0000 7f35dca7c640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35bc06be30 0x7f35bc06e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f35c8034000 tx=0x7f35c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.658+0000 7f35e21f4740 1 Processor -- start 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.658+0000 7f35e21f4740 1 -- start start 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2fd9f0 0x56009e31b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e2d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2d2640 0x56009e2fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35e21f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56009e236940 con 0x56009e2d2640 2024-04-25T00:03:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56009e1de920 con 0x56009e2fd9f0 2024-04-25T00:03:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56009e232900 con 0x56009e2f9ee0 2024-04-25T00:03:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2d2640 0x56009e2fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2d2640 0x56009e2fddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54590/0 (socket says 172.21.15.113:54590) 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 -- 172.21.15.113:0/1536992028 learned_addr learned my addr 172.21.15.113:0/1536992028 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dd27d640 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2fd9f0 0x56009e31b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dca7c640 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e2d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 -- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 msgr2=0x56009e2d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e2d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 -- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2fd9f0 msgr2=0x56009e31b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2fd9f0 0x56009e31b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 -- 172.21.15.113:0/1536992028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009e016a20 con 0x56009e2d2640 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dd27d640 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2fd9f0 0x56009e31b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35dda7e640 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2d2640 0x56009e2fddd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f35d4001e00 tx=0x7f35d400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1a6c78f928162a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35c2ffd640 1 -- 172.21.15.113:0/1536992028 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4018070 con 0x56009e2d2640 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35c2ffd640 1 -- 172.21.15.113:0/1536992028 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d400de30 con 0x56009e2d2640 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.659+0000 7f35c2ffd640 1 -- 172.21.15.113:0/1536992028 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4003bb0 con 0x56009e2d2640 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 -- 172.21.15.113:0/1536992028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2d2640 msgr2=0x56009e2fddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2d2640 0x56009e2fddd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f35d4001e00 tx=0x7f35d400d520 comp rx=0 tx=0).stop 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 -- 172.21.15.113:0/1536992028 shutdown_connections 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2d2640 0x56009e2fddd0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e2d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1536992028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2fd9f0 0x56009e31b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 -- 172.21.15.113:0/1536992028 >> 172.21.15.113:0/1536992028 conn(0x56009e2d0150 msgr2=0x56009e31bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 -- 172.21.15.113:0/1536992028 shutdown_connections 2024-04-25T00:03:19.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.660+0000 7f35e21f4740 1 -- 172.21.15.113:0/1536992028 wait complete. 2024-04-25T00:03:19.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 -- start start 2024-04-25T00:03:19.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 Processor -- start 2024-04-25T00:03:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 0x56009e2ec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e2fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e319a40 0x56009e319ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56009e31ce40 con 0x56009e2fd9f0 2024-04-25T00:03:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56009e1f2030 con 0x56009e2f9ee0 2024-04-25T00:03:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.661+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56009e31c680 con 0x56009e319a40 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35e21f4740 10 client.0 _ll_register_callbacks cb 0x56009e1ece00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e2fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dd27d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 0x56009e2ec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e2fd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54600/0 (socket says 172.21.15.113:54600) 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 -- 172.21.15.113:0/1905209508 learned_addr learned my addr 172.21.15.113:0/1905209508 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dda7e640 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e319a40 0x56009e319ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 -- 172.21.15.113:0/1905209508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e319a40 msgr2=0x56009e319ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e319a40 0x56009e319ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 -- 172.21.15.113:0/1905209508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 msgr2=0x56009e2ec2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 0x56009e2ec2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 -- 172.21.15.113:0/1905209508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009df0a310 con 0x56009e2fd9f0 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35dca7c640 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e2fd020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f35c8032af0 tx=0x7f35c802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfeb2b60d137b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:19.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35b7fff640 1 -- 172.21.15.113:0/1905209508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c804e060 con 0x56009e2fd9f0 2024-04-25T00:03:19.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35b7fff640 1 -- 172.21.15.113:0/1905209508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7f35c803ca70 con 0x56009e2fd9f0 2024-04-25T00:03:19.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d400de30 con 0x56009e2fd9f0 2024-04-25T00:03:19.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35b7fff640 1 -- 172.21.15.113:0/1905209508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f35c8054050 con 0x56009e2fd9f0 2024-04-25T00:03:19.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35e21f4740 10 client.7161 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:19.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56009e0771b0 con 0x56009e2fd9f0 2024-04-25T00:03:19.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35b7fff640 1 -- 172.21.15.113:0/1905209508 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d400de30 con 0x56009e2fd9f0 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.662+0000 7f35b7fff640 1 -- 172.21.15.113:0/1905209508 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f35c8057020 con 0x56009e2fd9f0 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35e21f4740 10 client.7161 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35e21f4740 10 client.7161 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35e21f4740 20 client.7161 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fb910 0x56009e2fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35e21f4740 4 client.7161 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56009e243e80 con 0x56009e2fb910 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35dd27d640 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fb910 0x56009e2fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35dd27d640 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fb910 0x56009e2fbd70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768bc1b3c886dbe0 server_cookie=ef50de650f31bfcf in_seq=0 out_seq=0 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35b7fff640 10 client.7161 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35b7fff640 1 -- 172.21.15.113:0/1905209508 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56009e243e80 con 0x56009e2fb910 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35b7fff640 10 client.7161 handle_command_reply: tid=0 2024-04-25T00:03:19.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.663+0000 7f35e21f4740 1 client.7161 shutdown 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 msgr2=0x56009e2fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e2fd020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f35c8032af0 tx=0x7f35c802c070 comp rx=0 tx=0).stop 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 shutdown_connections 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fb910 0x56009e2fbd70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e319a40 0x56009e319ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e2fd020 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1905209508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 0x56009e2ec2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35b7fff640 0 client.7161 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 >> 172.21.15.113:0/1905209508 conn(0x56009e2d2640 msgr2=0x56009e2fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 shutdown_connections 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 1 -- 172.21.15.113:0/1905209508 wait complete. 2024-04-25T00:03:19.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.664+0000 7f35e21f4740 20 client.7161 trim_cache size 0 max 16384 2024-04-25T00:03:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.681+0000 7f35e21f4740 1 Processor -- start 2024-04-25T00:03:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.681+0000 7f35e21f4740 1 -- start start 2024-04-25T00:03:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 0x56009e2fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2fd9f0 0x56009e317b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e319a40 0x56009e31bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35e21f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8200600 con 0x56009e319a40 2024-04-25T00:03:19.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81598e0 con 0x56009e2f9ee0 2024-04-25T00:03:19.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35d8160d40 con 0x56009e2fd9f0 2024-04-25T00:03:19.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e319a40 0x56009e31bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e319a40 0x56009e31bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54610/0 (socket says 172.21.15.113:54610) 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 -- 172.21.15.113:0/1733665498 learned_addr learned my addr 172.21.15.113:0/1733665498 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 -- 172.21.15.113:0/1733665498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2fd9f0 msgr2=0x56009e317b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dca7c640 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2fd9f0 0x56009e317b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2fd9f0 0x56009e317b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 -- 172.21.15.113:0/1733665498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 msgr2=0x56009e2fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 0x56009e2fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.682+0000 7f35dda7e640 1 -- 172.21.15.113:0/1733665498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009e328dc0 con 0x56009e319a40 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35dda7e640 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e319a40 0x56009e31bf70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x56009e2d4ca0 tx=0x7f35d400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549f44f476546993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35c27fc640 1 -- 172.21.15.113:0/1733665498 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4018040 con 0x56009e319a40 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35c27fc640 1 -- 172.21.15.113:0/1733665498 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4024070 con 0x56009e319a40 2024-04-25T00:03:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35c27fc640 1 -- 172.21.15.113:0/1733665498 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4003970 con 0x56009e319a40 2024-04-25T00:03:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 -- 172.21.15.113:0/1733665498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e319a40 msgr2=0x56009e31bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e319a40 0x56009e31bf70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x56009e2d4ca0 tx=0x7f35d400b000 comp rx=0 tx=0).stop 2024-04-25T00:03:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 -- 172.21.15.113:0/1733665498 shutdown_connections 2024-04-25T00:03:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e319a40 0x56009e31bf70 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2fd9f0 0x56009e317b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 --2- 172.21.15.113:0/1733665498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e2f9ee0 0x56009e2fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 -- 172.21.15.113:0/1733665498 >> 172.21.15.113:0/1733665498 conn(0x56009e2d2640 msgr2=0x56009e31b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 -- 172.21.15.113:0/1733665498 shutdown_connections 2024-04-25T00:03:19.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.683+0000 7f35e21f4740 1 -- 172.21.15.113:0/1733665498 wait complete. 2024-04-25T00:03:19.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.684+0000 7f35e21f4740 1 -- start start 2024-04-25T00:03:19.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.684+0000 7f35e21f4740 1 Processor -- start 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e392640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e3189f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e319a40 0x56009e318f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56009e2c88a0 con 0x56009e2fd9f0 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8159290 con 0x56009e319a40 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56009e2d2310 con 0x56009e2f9ee0 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 10 client.0 _ll_register_callbacks cb 0x7f35c40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e3189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e3189f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54614/0 (socket says 172.21.15.113:54614) 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 -- 172.21.15.113:0/798506458 learned_addr learned my addr 172.21.15.113:0/798506458 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dd27d640 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e392640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dda7e640 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e319a40 0x56009e318f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 -- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 msgr2=0x56009e392640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e392640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 -- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e319a40 msgr2=0x56009e318f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e319a40 0x56009e318f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 -- 172.21.15.113:0/798506458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009e2d1290 con 0x56009e2fd9f0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35dca7c640 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e3189f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006f80 tx=0x7f35c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08e99b941e5c28a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35b7fff640 1 -- 172.21.15.113:0/798506458 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c805d060 con 0x56009e2fd9f0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35b7fff640 1 -- 172.21.15.113:0/798506458 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7f35c8004450 con 0x56009e2fd9f0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.685+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d4024070 con 0x56009e2fd9f0 2024-04-25T00:03:19.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35b7fff640 1 -- 172.21.15.113:0/798506458 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f35c8063050 con 0x56009e2fd9f0 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 10 client.7167 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56009dee3490 con 0x56009e2fd9f0 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35b7fff640 1 -- 172.21.15.113:0/798506458 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4024070 con 0x56009e2fd9f0 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35b7fff640 1 -- 172.21.15.113:0/798506458 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f35c8066020 con 0x56009e2fd9f0 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 10 client.7167 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 10 client.7167 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 20 client.7167 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fe670 0x56009e3004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 4 client.7167 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) -- 0x56009e23e1d0 con 0x56009e2fe670 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35dd27d640 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fe670 0x56009e3004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35b7fff640 10 client.7167 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.686+0000 7f35dd27d640 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fe670 0x56009e3004f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa60ab18d7831330 server_cookie=53f1a5dd6bbba73b in_seq=0 out_seq=0 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35b7fff640 1 -- 172.21.15.113:0/798506458 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56009e23e1d0 con 0x56009e2fe670 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35b7fff640 10 client.7167 handle_command_reply: tid=0 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 client.7167 shutdown 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 msgr2=0x56009e3189f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e3189f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006f80 tx=0x7f35c8004730 comp rx=0 tx=0).stop 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 shutdown_connections 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56009e2fe670 0x56009e3004f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56009e319a40 0x56009e318f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35b7fff640 0 client.7167 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56009e2fd9f0 0x56009e3189f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 --2- 172.21.15.113:0/798506458 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56009e2f9ee0 0x56009e392640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 >> 172.21.15.113:0/798506458 conn(0x56009e2d2640 msgr2=0x56009e317b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 shutdown_connections 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.696+0000 7f35e21f4740 1 -- 172.21.15.113:0/798506458 wait complete. 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35e21f4740 20 client.7167 trim_cache size 0 max 16384 2024-04-25T00:03:19.698 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:03:19.699 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:03:19.699 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "748c4e7b-8c1c-4bf7-a1d3-817a1f157b37", 2024-04-25T00:03:19.699 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:03:19.699 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35bc06be30 msgr2=0x7f35bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35bc06be30 0x7f35bc06e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f35c8034000 tx=0x7f35c802e000 comp rx=0 tx=0).stop 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b61e0 msgr2=0x7f35d81499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b61e0 0x7f35d81499c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c390 tx=0x7f35d000c850 comp rx=0 tx=0).stop 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 shutdown_connections 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f35bc06be30 0x7f35bc06e2f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35d815e510 0x7f35d80cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35d80b9bb0 0x7f35d80cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 --2- 172.21.15.113:0/2194428662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35d80b61e0 0x7f35d81499c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 >> 172.21.15.113:0/2194428662 conn(0x7f35d806b720 msgr2=0x7f35d80ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 shutdown_connections 2024-04-25T00:03:19.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.697+0000 7f35df4e1640 1 -- 172.21.15.113:0/2194428662 wait complete. 2024-04-25T00:03:19.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:03:19.710 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T00:03:19.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.859+0000 7f36cd748640 1 Processor -- start 2024-04-25T00:03:19.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.859+0000 7f36cd748640 1 -- start start 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36cd748640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c815c470 0x7f36c806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36cd748640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f36c8062290 0x7f36c806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36cd748640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c8160320 con 0x7f36c815c090 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36cd748640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f36c806e050 con 0x7f36c8062290 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36cd748640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f36c8160600 con 0x7f36c815c470 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36c6ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f36c815c090 0x7f36c8061b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38890/0 (socket says 172.21.15.113:38890) 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1469793923 learned_addr learned my addr 172.21.15.113:0/1469793923 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:19.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1469793923 >> v2:172.21.15.178:3300/0 conn(0x7f36c8062290 0x7f36c806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.860+0000 7f36c67fc640 1 --2- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c815c470 0x7f36c806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.861+0000 7f36c5ffb640 1 -- 172.21.15.113:0/1469793923 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661505027 0 0) 0x7f36c8160320 con 0x7f36c815c090 2024-04-25T00:03:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.861+0000 7f36c5ffb640 1 -- 172.21.15.113:0/1469793923 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36b0003520 con 0x7f36c815c090 2024-04-25T00:03:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.861+0000 7f36c77fe640 1 -- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c815c470 msgr2=0x7f36c806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.861+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c815c470 0x7f36c806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.861+0000 7f36c77fe640 1 -- 172.21.15.113:0/1469793923 >> v1:172.21.15.113:6789/0 conn(0x7f36c815c090 legacy=0x7f36c8061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.861+0000 7f36c77fe640 1 -- 172.21.15.113:0/1469793923 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c81608e0 con 0x7f36c8062290 2024-04-25T00:03:19.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.862+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c8062290 0x7f36c806e4e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36c00097a0 tx=0x7f36c002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637437f1b83a6e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.862+0000 7f36c5ffb640 1 -- 172.21.15.113:0/1469793923 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0037070 con 0x7f36c8062290 2024-04-25T00:03:19.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.862+0000 7f36c5ffb640 1 -- 172.21.15.113:0/1469793923 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c0032440 con 0x7f36c8062290 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.862+0000 7f36c5ffb640 1 -- 172.21.15.113:0/1469793923 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c00327f0 con 0x7f36c8062290 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 -- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c8062290 msgr2=0x7f36c806e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 --2- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c8062290 0x7f36c806e4e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36c00097a0 tx=0x7f36c002a230 comp rx=0 tx=0).stop 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 -- 172.21.15.113:0/1469793923 shutdown_connections 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 --2- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c8062290 0x7f36c806e4e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 --2- 172.21.15.113:0/1469793923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c815c470 0x7f36c806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 -- 172.21.15.113:0/1469793923 >> 172.21.15.113:0/1469793923 conn(0x7f36c80cd8f0 msgr2=0x7f36c80cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 -- 172.21.15.113:0/1469793923 shutdown_connections 2024-04-25T00:03:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.863+0000 7f36cd748640 1 -- 172.21.15.113:0/1469793923 wait complete. 2024-04-25T00:03:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.865+0000 7f36cd748640 1 Processor -- start 2024-04-25T00:03:19.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.865+0000 7f36cd748640 1 -- start start 2024-04-25T00:03:19.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36cd748640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c8062290 0x7f36c81588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36cd748640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c815c090 0x7f36c8154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36cd748640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c8157020 0x7f36c8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36cd748640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36c815fee0 con 0x7f36c8062290 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36cd748640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36c815fd60 con 0x7f36c815c090 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36cd748640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f36c8160060 con 0x7f36c8157020 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c8062290 0x7f36c81588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.866+0000 7f36c67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c815c090 0x7f36c8154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c8062290 0x7f36c81588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54628/0 (socket says 172.21.15.113:54628) 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c815c090 0x7f36c8154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:45596/0 (socket says 172.21.15.113:45596) 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 -- 172.21.15.113:0/994635505 learned_addr learned my addr 172.21.15.113:0/994635505 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c77fe640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c8157020 0x7f36c8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 -- 172.21.15.113:0/994635505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c8157020 msgr2=0x7f36c8156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c8157020 0x7f36c8156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 -- 172.21.15.113:0/994635505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c815c090 msgr2=0x7f36c8154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c815c090 0x7f36c8154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 -- 172.21.15.113:0/994635505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c80c8a80 con 0x7f36c8062290 2024-04-25T00:03:19.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.867+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c8062290 0x7f36c81588a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f36b800cb70 tx=0x7f36b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.868+0000 7f36af7fe640 1 -- 172.21.15.113:0/994635505 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8017070 con 0x7f36c8062290 2024-04-25T00:03:19.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.868+0000 7f36af7fe640 1 -- 172.21.15.113:0/994635505 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b800ad40 con 0x7f36c8062290 2024-04-25T00:03:19.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.868+0000 7f36af7fe640 1 -- 172.21.15.113:0/994635505 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8012620 con 0x7f36c8062290 2024-04-25T00:03:19.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.868+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c80c8d60 con 0x7f36c8062290 2024-04-25T00:03:19.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.868+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c80c9320 con 0x7f36c8062290 2024-04-25T00:03:19.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.870+0000 7f36af7fe640 1 -- 172.21.15.113:0/994635505 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36b801a070 con 0x7f36c8062290 2024-04-25T00:03:19.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.871+0000 7f36af7fe640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f36b4070240 0x7f36b4072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.871+0000 7f36af7fe640 1 -- 172.21.15.113:0/994635505 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7f36b808acc0 con 0x7f36c8062290 2024-04-25T00:03:19.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.871+0000 7f36c67fc640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f36b4070240 0x7f36b4072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.872+0000 7f36d044c740 1 Processor -- start 2024-04-25T00:03:19.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.872+0000 7f36c67fc640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f36b4070240 0x7f36b4072700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cdde0 tx=0x7f36bc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.872+0000 7f36d044c740 1 -- start start 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba86279140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86296b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36d044c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8624ea40 0x55ba86294cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba86279140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba86279140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:45598/0 (socket says 172.21.15.113:45598) 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36c67fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86296b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36c77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8624ea40 0x55ba86294cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36c67fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86296b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:36054/0 (socket says 172.21.15.113:36054) 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.873+0000 7f36c77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8624ea40 0x55ba86294cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54642/0 (socket says 172.21.15.113:54642) 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36c6ffd640 1 -- 172.21.15.113:0/4099212770 learned_addr learned my addr 172.21.15.113:0/4099212770 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ba861b26b0 con 0x55ba8624ea40 2024-04-25T00:03:19.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba8615a880 con 0x55ba86278d60 2024-04-25T00:03:19.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba861ae670 con 0x55ba86296600 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36c6ffd640 1 -- 172.21.15.113:0/4099212770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 msgr2=0x55ba86296b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/4099212770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86296b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36c6ffd640 1 -- 172.21.15.113:0/4099212770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8624ea40 msgr2=0x55ba86294cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/4099212770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8624ea40 0x55ba86294cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36c6ffd640 1 -- 172.21.15.113:0/4099212770 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba85f907d0 con 0x55ba86278d60 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/4099212770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba86279140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36b80615e0 tx=0x7f36b8058c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=117875ba31108956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.874+0000 7f36acff9640 1 -- 172.21.15.113:0/4099212770 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80572e0 con 0x55ba86278d60 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.875+0000 7f36acff9640 1 -- 172.21.15.113:0/4099212770 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b8055650 con 0x55ba86278d60 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.875+0000 7f36acff9640 1 -- 172.21.15.113:0/4099212770 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80618a0 con 0x55ba86278d60 2024-04-25T00:03:19.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.875+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 msgr2=0x55ba86279140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.875+0000 7f36d044c740 1 --2- 172.21.15.113:0/4099212770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba86279140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36b80615e0 tx=0x7f36b8058c50 comp rx=0 tx=0).stop 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.876+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 shutdown_connections 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.876+0000 7f36d044c740 1 --2- 172.21.15.113:0/4099212770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8624ea40 0x55ba86294cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.876+0000 7f36d044c740 1 --2- 172.21.15.113:0/4099212770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86296b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.876+0000 7f36d044c740 1 --2- 172.21.15.113:0/4099212770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba86279140 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.876+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 >> 172.21.15.113:0/4099212770 conn(0x55ba8624b600 msgr2=0x55ba86293b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.876+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 shutdown_connections 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.876+0000 7f36d044c740 1 -- 172.21.15.113:0/4099212770 wait complete. 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.877+0000 7f36d044c740 1 -- start start 2024-04-25T00:03:19.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.877+0000 7f36d044c740 1 Processor -- start 2024-04-25T00:03:19.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.877+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba862a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86292790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36d044c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36d044c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ba8624f7e0 con 0x55ba8628c090 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36d044c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba8616df50 con 0x55ba86278d60 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36d044c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba86249530 con 0x55ba86296600 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36d044c740 10 client.0 _ll_register_callbacks cb 0x55ba861a9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba862a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba862a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:45602/0 (socket says 172.21.15.113:45602) 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1143765653 learned_addr learned my addr 172.21.15.113:0/1143765653 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c67fc640 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86292790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 msgr2=0x55ba86292790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86292790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1143765653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 msgr2=0x55ba862774b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862774b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1143765653 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba85ec63f0 con 0x55ba86278d60 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c67fc640 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86292790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba862a5600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36b80012b0 tx=0x7f36b8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668f13f3dc931054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.878+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1143765653 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8088070 con 0x55ba86278d60 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1143765653 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7f36b8082b10 con 0x55ba86278d60 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b8055650 con 0x55ba86278d60 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1143765653 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f36b8055280 con 0x55ba86278d60 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 10 client.6157 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:19.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55ba85e860f0 con 0x55ba86278d60 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1143765653 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b8055650 con 0x55ba86278d60 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1143765653 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f36b8054d30 con 0x55ba86278d60 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 10 client.6157 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 10 client.6157 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 20 client.6157 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba862a4800 0x55ba862a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 4 client.6157 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.879+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba861bfbf0 con 0x55ba862a4800 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.880+0000 7f36c67fc640 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba862a4800 0x55ba862a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.880+0000 7f36c67fc640 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba862a4800 0x55ba862a4c10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d47d8b57adc48a server_cookie=e579f5aabdfb004e in_seq=0 out_seq=0 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.880+0000 7f36a5ffb640 10 client.6157 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1143765653 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba861bfbf0 con 0x55ba862a4800 2024-04-25T00:03:19.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36a5ffb640 10 client.6157 handle_command_reply: tid=0 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 client.6157 shutdown 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 msgr2=0x55ba862a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba862a5600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36b80012b0 tx=0x7f36b8000f40 comp rx=0 tx=0).stop 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 shutdown_connections 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba862a4800 0x55ba862a4c10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86296600 0x55ba86292790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36a5ffb640 0 client.6157 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 --2- 172.21.15.113:0/1143765653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86278d60 0x55ba862a5600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 >> 172.21.15.113:0/1143765653 conn(0x55ba8624ea40 msgr2=0x55ba86297980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 shutdown_connections 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.881+0000 7f36d044c740 1 -- 172.21.15.113:0/1143765653 wait complete. 2024-04-25T00:03:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.882+0000 7f36d044c740 20 client.6157 trim_cache size 0 max 16384 2024-04-25T00:03:19.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.908+0000 7f36d044c740 1 Processor -- start 2024-04-25T00:03:19.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.908+0000 7f36d044c740 1 -- start start 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36d044c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba86278d60 0x55ba8624ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba8628c090 0x55ba8624f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba8624c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36d044c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81fc300 con 0x55ba86278d60 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36d044c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36c80cff10 con 0x55ba86296600 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36d044c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f36c80ca5d0 con 0x55ba8628c090 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36c67fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba8628c090 0x55ba8624f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba86278d60 0x55ba8624ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba86278d60 0x55ba8624ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54674/0 (socket says 172.21.15.113:54674) 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36c6ffd640 1 -- 172.21.15.113:0/3730167147 learned_addr learned my addr 172.21.15.113:0/3730167147 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.909+0000 7f36c77fe640 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba8624c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36c77fe640 1 -- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba8628c090 msgr2=0x55ba8624f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36c77fe640 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba8628c090 0x55ba8624f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36c77fe640 1 -- 172.21.15.113:0/3730167147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba86278d60 msgr2=0x55ba8624ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36c77fe640 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba86278d60 0x55ba8624ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36c77fe640 1 -- 172.21.15.113:0/3730167147 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8623b090 con 0x55ba86296600 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba86278d60 0x55ba8624ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36c77fe640 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba8624c910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f36c80b3960 tx=0x7f36c002c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4dded7e98b1cce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36a7fff640 1 -- 172.21.15.113:0/3730167147 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0037040 con 0x55ba86296600 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.910+0000 7f36a7fff640 1 -- 172.21.15.113:0/3730167147 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c0032580 con 0x55ba86296600 2024-04-25T00:03:19.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36a7fff640 1 -- 172.21.15.113:0/3730167147 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0032950 con 0x55ba86296600 2024-04-25T00:03:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 -- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 msgr2=0x55ba8624c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba8624c910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f36c80b3960 tx=0x7f36c002c000 comp rx=0 tx=0).stop 2024-04-25T00:03:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 -- 172.21.15.113:0/3730167147 shutdown_connections 2024-04-25T00:03:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba8624c910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba8628c090 0x55ba8624f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 --2- 172.21.15.113:0/3730167147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba86278d60 0x55ba8624ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 -- 172.21.15.113:0/3730167147 >> 172.21.15.113:0/3730167147 conn(0x55ba8624ea40 msgr2=0x55ba862925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 -- 172.21.15.113:0/3730167147 shutdown_connections 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.911+0000 7f36d044c740 1 -- 172.21.15.113:0/3730167147 wait complete. 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.912+0000 7f36d044c740 1 -- start start 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.912+0000 7f36d044c740 1 Processor -- start 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86278d60 0x55ba862a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36d044c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36d044c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba862b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36c67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36d044c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36c80cac20 con 0x55ba8628c090 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36c67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54690/0 (socket says 172.21.15.113:54690) 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36c67fc640 1 -- 172.21.15.113:0/1467691060 learned_addr learned my addr 172.21.15.113:0/1467691060 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86278d60 0x55ba862a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba862b6750 con 0x55ba86296600 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba8624d2f0 con 0x55ba86278d60 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36d044c740 10 client.0 _ll_register_callbacks cb 0x7f36b00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.913+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba862b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36c77fe640 1 -- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86278d60 msgr2=0x55ba862a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86278d60 0x55ba862a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36c77fe640 1 -- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 msgr2=0x55ba862a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36c77fe640 1 -- 172.21.15.113:0/1467691060 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba86277670 con 0x55ba86296600 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36c67fc640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba862b2d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36c0031000 tx=0x7f36c0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b547bebc1d70ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-04-25T00:03:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1467691060 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c004c070 con 0x55ba86296600 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1467691060 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5767+0+0 (secure 0 0 0) 0x7f36c0032940 con 0x55ba86296600 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.914+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c0032580 con 0x55ba86296600 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1467691060 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f36c0057050 con 0x55ba86296600 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 10 client.6163 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55ba8624f390 con 0x55ba86296600 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1467691060 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c0006a20 con 0x55ba86296600 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1467691060 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f36c005a020 con 0x55ba86296600 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 10 client.6163 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 10 client.6163 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 20 client.6163 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba86310710 0x55ba862b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 4 client.6163 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:03:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f36c0032580 con 0x55ba86310710 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba86310710 0x55ba862b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.915+0000 7f36a5ffb640 10 client.6163 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba86310710 0x55ba862b32c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d46ca8d101e099f server_cookie=c3a844ad29cbba31 in_seq=0 out_seq=0 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36a5ffb640 1 -- 172.21.15.113:0/1467691060 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x7f36c0032580 con 0x55ba86310710 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36a5ffb640 10 client.6163 handle_command_reply: tid=0 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 client.6163 shutdown 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 msgr2=0x55ba862b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba862b2d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36c0031000 tx=0x7f36c0004730 comp rx=0 tx=0).stop 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 shutdown_connections 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55ba86310710 0x55ba862b32c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba86296600 0x55ba862b2d80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba8628c090 0x55ba862a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 --2- 172.21.15.113:0/1467691060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba86278d60 0x55ba862a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 >> 172.21.15.113:0/1467691060 conn(0x55ba8624ea40 msgr2=0x55ba862769b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36a5ffb640 0 client.6163 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.916+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 shutdown_connections 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36d044c740 1 -- 172.21.15.113:0/1467691060 wait complete. 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36d044c740 20 client.6163 trim_cache size 0 max 16384 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "scrub active (346 inodes in the stack)", 2024-04-25T00:03:19.923 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:03:19.924 INFO:teuthology.orchestra.run.smithi113.stdout: "748c4e7b-8c1c-4bf7-a1d3-817a1f157b37": { 2024-04-25T00:03:19.924 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "~mds0", 2024-04-25T00:03:19.924 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "748c4e7b-8c1c-4bf7-a1d3-817a1f157b37", 2024-04-25T00:03:19.924 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive" 2024-04-25T00:03:19.924 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.924 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:19.924 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:03:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f36b4070240 msgr2=0x7f36b4072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f36b4070240 0x7f36b4072700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cdde0 tx=0x7f36bc009190 comp rx=0 tx=0).stop 2024-04-25T00:03:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c8062290 msgr2=0x7f36c81588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c8062290 0x7f36c81588a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f36b800cb70 tx=0x7f36b800a430 comp rx=0 tx=0).stop 2024-04-25T00:03:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 shutdown_connections 2024-04-25T00:03:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f36b4070240 0x7f36b4072700 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c815c090 0x7f36c8154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c8157020 0x7f36c8156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 --2- 172.21.15.113:0/994635505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c8062290 0x7f36c81588a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:19.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 >> 172.21.15.113:0/994635505 conn(0x7f36c80cd8f0 msgr2=0x7f36c80cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:19.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.917+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 shutdown_connections 2024-04-25T00:03:19.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:19.918+0000 7f36cd748640 1 -- 172.21.15.113:0/994635505 wait complete. 2024-04-25T00:03:19.931 INFO:tasks.cephfs.filesystem:scrub status for tag:748c4e7b-8c1c-4bf7-a1d3-817a1f157b37 - {'path': '~mds0', 'tag': '748c4e7b-8c1c-4bf7-a1d3-817a1f157b37', 'options': 'recursive'} 2024-04-25T00:03:22.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:03:22.394 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:03:49.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:03:49.933 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T00:03:50.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.045+0000 7f86d583f640 1 Processor -- start 2024-04-25T00:03:50.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.045+0000 7f86d583f640 1 -- start start 2024-04-25T00:03:50.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86d583f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86d015ba50 0x7f86d015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86d583f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86d583f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d006d7e0 con 0x7f86d015b670 2024-04-25T00:03:50.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86d583f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f86d0071170 con 0x7f86d015ba50 2024-04-25T00:03:50.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86d583f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f86d0071450 con 0x7f86d006cea0 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86ce7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86d015ba50 0x7f86d015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86ce7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86d015ba50 0x7f86d015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39616/0 (socket says 172.21.15.113:39616) 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3274142739 learned_addr learned my addr 172.21.15.113:0/3274142739 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/3274142739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3274142739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 msgr2=0x7f86d006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/3274142739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3274142739 >> v1:172.21.15.178:6789/0 conn(0x7f86d015b670 legacy=0x7f86d00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.046+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3274142739 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d0071730 con 0x7f86d015ba50 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.047+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/3274142739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015ba50 0x7f86d015bff0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2962461a40483f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.047+0000 7f86cdffb640 1 -- 172.21.15.113:0/3274142739 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4037070 con 0x7f86d015ba50 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.047+0000 7f86cdffb640 1 -- 172.21.15.113:0/3274142739 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c402ae10 con 0x7f86d015ba50 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.047+0000 7f86cdffb640 1 -- 172.21.15.113:0/3274142739 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4032660 con 0x7f86d015ba50 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 -- 172.21.15.113:0/3274142739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015ba50 msgr2=0x7f86d015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 --2- 172.21.15.113:0/3274142739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015ba50 0x7f86d015bff0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c402a100 comp rx=0 tx=0).stop 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 -- 172.21.15.113:0/3274142739 shutdown_connections 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 --2- 172.21.15.113:0/3274142739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 --2- 172.21.15.113:0/3274142739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015ba50 0x7f86d015bff0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 -- 172.21.15.113:0/3274142739 >> 172.21.15.113:0/3274142739 conn(0x7f86d006b0f0 msgr2=0x7f86d006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 -- 172.21.15.113:0/3274142739 shutdown_connections 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.048+0000 7f86d583f640 1 -- 172.21.15.113:0/3274142739 wait complete. 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.049+0000 7f86d583f640 1 Processor -- start 2024-04-25T00:03:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.049+0000 7f86d583f640 1 -- start start 2024-04-25T00:03:50.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86d583f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d013fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86d583f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015b670 0x7f86d014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86d583f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d015ba50 0x7f86d00b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86d583f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86d00cca90 con 0x7f86d015b670 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86d583f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86d00cc910 con 0x7f86d015ba50 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86d583f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f86d00d0060 con 0x7f86d006cea0 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86ce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015b670 0x7f86d014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86ce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015b670 0x7f86d014a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39632/0 (socket says 172.21.15.113:39632) 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86ce7fc640 1 -- 172.21.15.113:0/4086586305 learned_addr learned my addr 172.21.15.113:0/4086586305 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86ceffd640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d013fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.050+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d015ba50 0x7f86d00b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86ce7fc640 1 -- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 msgr2=0x7f86d013fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d013fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86ce7fc640 1 -- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d015ba50 msgr2=0x7f86d00b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d015ba50 0x7f86d00b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86ce7fc640 1 -- 172.21.15.113:0/4086586305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d01562c0 con 0x7f86d015b670 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015b670 0x7f86d014a740 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86b7fff640 1 -- 172.21.15.113:0/4086586305 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4041070 con 0x7f86d015b670 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86b7fff640 1 -- 172.21.15.113:0/4086586305 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c403c670 con 0x7f86d015b670 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d01565a0 con 0x7f86d015b670 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86b7fff640 1 -- 172.21.15.113:0/4086586305 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4037040 con 0x7f86d015b670 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.051+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d0156b60 con 0x7f86d015b670 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.053+0000 7f86b7fff640 1 -- 172.21.15.113:0/4086586305 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86c4044070 con 0x7f86d015b670 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.053+0000 7f86b7fff640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86bc06be30 0x7f86bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.053+0000 7f86ceffd640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86bc06be30 0x7f86bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.053+0000 7f86b7fff640 1 -- 172.21.15.113:0/4086586305 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f86c40ab400 con 0x7f86d015b670 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.054+0000 7f86ceffd640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86bc06be30 0x7f86bc06e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f86c0004f60 tx=0x7f86c0007e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.054+0000 7f86d862e740 1 Processor -- start 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.054+0000 7f86d862e740 1 -- start start 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86d862e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 0x55fced4d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced489a40 0x55fced4cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86d862e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fced3ed6b0 con 0x55fced4b3d60 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fced395880 con 0x55fced4d1600 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fced3e9670 con 0x55fced489a40 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86ceffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86ceffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39640/0 (socket says 172.21.15.113:39640) 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86ceffd640 1 -- 172.21.15.113:0/2015561822 learned_addr learned my addr 172.21.15.113:0/2015561822 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced489a40 0x55fced4cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.055+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 0x55fced4d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86ceffd640 1 -- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced489a40 msgr2=0x55fced4cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86ceffd640 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced489a40 0x55fced4cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86ceffd640 1 -- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 msgr2=0x55fced4d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86ceffd640 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 0x55fced4d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86ceffd640 1 -- 172.21.15.113:0/2015561822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fced1cb7d0 con 0x55fced4b3d60 2024-04-25T00:03:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86ceffd640 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4b4140 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f86c0003ca0 tx=0x7f86c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3f76d7982afe3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86b57fa640 1 -- 172.21.15.113:0/2015561822 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0025070 con 0x55fced4b3d60 2024-04-25T00:03:50.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86b57fa640 1 -- 172.21.15.113:0/2015561822 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c001b050 con 0x55fced4b3d60 2024-04-25T00:03:50.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.056+0000 7f86b57fa640 1 -- 172.21.15.113:0/2015561822 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c00045d0 con 0x55fced4b3d60 2024-04-25T00:03:50.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 -- 172.21.15.113:0/2015561822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 msgr2=0x55fced4b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4b4140 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f86c0003ca0 tx=0x7f86c0012000 comp rx=0 tx=0).stop 2024-04-25T00:03:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 -- 172.21.15.113:0/2015561822 shutdown_connections 2024-04-25T00:03:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 0x55fced4d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced489a40 0x55fced4cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 --2- 172.21.15.113:0/2015561822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4b4140 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 -- 172.21.15.113:0/2015561822 >> 172.21.15.113:0/2015561822 conn(0x55fced486600 msgr2=0x55fced4ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 -- 172.21.15.113:0/2015561822 shutdown_connections 2024-04-25T00:03:50.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 -- 172.21.15.113:0/2015561822 wait complete. 2024-04-25T00:03:50.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 -- start start 2024-04-25T00:03:50.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.057+0000 7f86d862e740 1 Processor -- start 2024-04-25T00:03:50.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4b3d60 0x55fced4b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86d862e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4d1600 0x55fced4b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4defd0 0x55fced4b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86d862e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fced48a7e0 con 0x55fced4d1600 2024-04-25T00:03:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fced3a8f50 con 0x55fced4defd0 2024-04-25T00:03:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fced484530 con 0x55fced4b3d60 2024-04-25T00:03:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86d862e740 10 client.0 _ll_register_callbacks cb 0x55fced3e4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4d1600 0x55fced4b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4d1600 0x55fced4b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39648/0 (socket says 172.21.15.113:39648) 2024-04-25T00:03:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3494129759 learned_addr learned my addr 172.21.15.113:0/3494129759 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ceffd640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4b3d60 0x55fced4b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4defd0 0x55fced4b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4b3d60 msgr2=0x55fced4b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4b3d60 0x55fced4b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4defd0 msgr2=0x55fced4b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4defd0 0x55fced4b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ce7fc640 1 -- 172.21.15.113:0/3494129759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fced1013f0 con 0x55fced4d1600 2024-04-25T00:03:50.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.058+0000 7f86ceffd640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4b3d60 0x55fced4b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:50.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4d1600 0x55fced4b2b80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f86c40a9860 tx=0x7f86c4075a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a271cc0b33a535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:50.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ae7fc640 1 -- 172.21.15.113:0/3494129759 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c404b040 con 0x55fced4d1600 2024-04-25T00:03:50.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ae7fc640 1 -- 172.21.15.113:0/3494129759 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f86c4081d90 con 0x55fced4d1600 2024-04-25T00:03:50.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c001b050 con 0x55fced4d1600 2024-04-25T00:03:50.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ae7fc640 1 -- 172.21.15.113:0/3494129759 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f86c4081f90 con 0x55fced4d1600 2024-04-25T00:03:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 10 client.7197 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55fced0c10f0 con 0x55fced4d1600 2024-04-25T00:03:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ae7fc640 1 -- 172.21.15.113:0/3494129759 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c001b050 con 0x55fced4d1600 2024-04-25T00:03:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ae7fc640 1 -- 172.21.15.113:0/3494129759 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f86c407eb80 con 0x55fced4d1600 2024-04-25T00:03:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 10 client.7197 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 10 client.7197 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 20 client.7197 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced4b4f60 0x55fced4b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 4 client.7197 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fced3fabf0 con 0x55fced4b4f60 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ceffd640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced4b4f60 0x55fced4b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ceffd640 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced4b4f60 0x55fced4b53f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9af9b93096303be server_cookie=b8c3144e3bcad4e7 in_seq=0 out_seq=0 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.059+0000 7f86ae7fc640 10 client.7197 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86ae7fc640 1 -- 172.21.15.113:0/3494129759 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fced3fabf0 con 0x55fced4b4f60 2024-04-25T00:03:50.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86ae7fc640 10 client.7197 handle_command_reply: tid=0 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 client.7197 shutdown 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4d1600 msgr2=0x55fced4b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4d1600 0x55fced4b2b80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f86c40a9860 tx=0x7f86c4075a50 comp rx=0 tx=0).stop 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 shutdown_connections 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced4b4f60 0x55fced4b53f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4defd0 0x55fced4b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4d1600 0x55fced4b2b80 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86ae7fc640 0 client.7197 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 --2- 172.21.15.113:0/3494129759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4b3d60 0x55fced4b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 >> 172.21.15.113:0/3494129759 conn(0x55fced489a40 msgr2=0x55fced4d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 shutdown_connections 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 1 -- 172.21.15.113:0/3494129759 wait complete. 2024-04-25T00:03:50.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.060+0000 7f86d862e740 20 client.7197 trim_cache size 0 max 16384 2024-04-25T00:03:50.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.078+0000 7f86d862e740 1 Processor -- start 2024-04-25T00:03:50.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.078+0000 7f86d862e740 1 -- start start 2024-04-25T00:03:50.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86d862e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4de950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 0x55fced4d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4defd0 0x55fced4d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86d862e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01fe8a0 con 0x55fced4b3d60 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01fda80 con 0x55fced4d1600 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f86d01f64c0 con 0x55fced4defd0 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4de950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4de950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39654/0 (socket says 172.21.15.113:39654) 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 -- 172.21.15.113:0/4017018625 learned_addr learned my addr 172.21.15.113:0/4017018625 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4defd0 0x55fced4d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 -- 172.21.15.113:0/4017018625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4defd0 msgr2=0x55fced4d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4defd0 0x55fced4d2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 -- 172.21.15.113:0/4017018625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 msgr2=0x55fced4d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 0x55fced4d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 -- 172.21.15.113:0/4017018625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fced4aa1f0 con 0x55fced4b3d60 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86ceffd640 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4de950 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f86c0020000 tx=0x7f86c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f0352962a61040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86b4ff9640 1 -- 172.21.15.113:0/4017018625 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0025040 con 0x55fced4b3d60 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86b4ff9640 1 -- 172.21.15.113:0/4017018625 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c001b070 con 0x55fced4b3d60 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.079+0000 7f86b4ff9640 1 -- 172.21.15.113:0/4017018625 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0014650 con 0x55fced4b3d60 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 -- 172.21.15.113:0/4017018625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 msgr2=0x55fced4de950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4de950 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f86c0020000 tx=0x7f86c0014000 comp rx=0 tx=0).stop 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 -- 172.21.15.113:0/4017018625 shutdown_connections 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4d1600 0x55fced4d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4defd0 0x55fced4d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 --2- 172.21.15.113:0/4017018625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4b3d60 0x55fced4de950 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 -- 172.21.15.113:0/4017018625 >> 172.21.15.113:0/4017018625 conn(0x55fced489a40 msgr2=0x55fced47fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 -- 172.21.15.113:0/4017018625 shutdown_connections 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.080+0000 7f86d862e740 1 -- 172.21.15.113:0/4017018625 wait complete. 2024-04-25T00:03:50.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 -- start start 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 Processor -- start 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4b3d60 0x55fced4d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4d1600 0x55fced4d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4defd0 0x55fced4c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fced48a340 con 0x55fced4defd0 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01489b0 con 0x55fced4b3d60 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.081+0000 7f86d862e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fced4abdc0 con 0x55fced4d1600 2024-04-25T00:03:50.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86d862e740 10 client.0 _ll_register_callbacks cb 0x7f86c40794c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:50.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4defd0 0x55fced4c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4defd0 0x55fced4c4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39662/0 (socket says 172.21.15.113:39662) 2024-04-25T00:03:50.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 -- 172.21.15.113:0/776813080 learned_addr learned my addr 172.21.15.113:0/776813080 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ce7fc640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4d1600 0x55fced4d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ceffd640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4b3d60 0x55fced4d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 -- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4d1600 msgr2=0x55fced4d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4d1600 0x55fced4d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 -- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4b3d60 msgr2=0x55fced4d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4b3d60 0x55fced4d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 -- 172.21.15.113:0/776813080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fced489280 con 0x55fced4defd0 2024-04-25T00:03:50.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ceffd640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4b3d60 0x55fced4d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:50.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86cf7fe640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4defd0 0x55fced4c4590 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f86c8004460 tx=0x7f86c8004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ae930b0fb5db84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:50.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ae7fc640 1 -- 172.21.15.113:0/776813080 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c801c070 con 0x55fced4defd0 2024-04-25T00:03:50.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ae7fc640 1 -- 172.21.15.113:0/776813080 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f86c8007bc0 con 0x55fced4defd0 2024-04-25T00:03:50.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c001b070 con 0x55fced4defd0 2024-04-25T00:03:50.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ae7fc640 1 -- 172.21.15.113:0/776813080 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f86c8027050 con 0x55fced4defd0 2024-04-25T00:03:50.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86d862e740 10 client.7203 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:50.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55fced487e40 con 0x55fced4defd0 2024-04-25T00:03:50.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.082+0000 7f86ae7fc640 1 -- 172.21.15.113:0/776813080 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c001b070 con 0x55fced4defd0 2024-04-25T00:03:50.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86ae7fc640 1 -- 172.21.15.113:0/776813080 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f86c802a020 con 0x55fced4defd0 2024-04-25T00:03:50.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 10 client.7203 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:50.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 10 client.7203 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:50.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 20 client.7203 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:50.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced54c9e0 0x55fced4c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 4 client.7203 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:03:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fced3f4f40 con 0x55fced54c9e0 2024-04-25T00:03:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86ceffd640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced54c9e0 0x55fced4c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86ae7fc640 10 client.7203 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86ceffd640 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced54c9e0 0x55fced4c4ad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74033bf1309da78e server_cookie=b198978e6bac7e0d in_seq=0 out_seq=0 2024-04-25T00:03:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86ae7fc640 1 -- 172.21.15.113:0/776813080 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fced3f4f40 con 0x55fced54c9e0 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86ae7fc640 10 client.7203 handle_command_reply: tid=0 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 client.7203 shutdown 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4defd0 msgr2=0x55fced4c4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4defd0 0x55fced4c4590 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f86c8004460 tx=0x7f86c8004920 comp rx=0 tx=0).stop 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 shutdown_connections 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x55fced54c9e0 0x55fced4c4ad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fced4defd0 0x55fced4c4590 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fced4d1600 0x55fced4d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 --2- 172.21.15.113:0/776813080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fced4b3d60 0x55fced4d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 >> 172.21.15.113:0/776813080 conn(0x55fced489a40 msgr2=0x55fced485320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86ae7fc640 0 client.7203 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.083+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 shutdown_connections 2024-04-25T00:03:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d862e740 1 -- 172.21.15.113:0/776813080 wait complete. 2024-04-25T00:03:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d862e740 20 client.7203 trim_cache size 0 max 16384 2024-04-25T00:03:50.097 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:03:50.097 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:03:50.097 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:03:50.097 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:03:50.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86bc06be30 msgr2=0x7f86bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86bc06be30 0x7f86bc06e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f86c0004f60 tx=0x7f86c0007e30 comp rx=0 tx=0).stop 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015b670 msgr2=0x7f86d014a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015b670 0x7f86d014a740 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c40046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 shutdown_connections 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f86bc06be30 0x7f86bc06e2f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d015ba50 0x7f86d00b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d015b670 0x7f86d014a740 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 --2- 172.21.15.113:0/4086586305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86d006cea0 0x7f86d013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 >> 172.21.15.113:0/4086586305 conn(0x7f86d006b0f0 msgr2=0x7f86d01591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 shutdown_connections 2024-04-25T00:03:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.084+0000 7f86d583f640 1 -- 172.21.15.113:0/4086586305 wait complete. 2024-04-25T00:03:50.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:03:50.101 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T00:03:50.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.199+0000 7f48f77e7640 1 Processor -- start 2024-04-25T00:03:50.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.199+0000 7f48f77e7640 1 -- start start 2024-04-25T00:03:50.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f77e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48f015e2d0 0x7f48f015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f77e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f015f4f0 con 0x7f48f00b3080 2024-04-25T00:03:50.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f77e7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f015f7d0 con 0x7f48f00b4a40 2024-04-25T00:03:50.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f77e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f48f00cde00 con 0x7f48f015e2d0 2024-04-25T00:03:50.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f5583640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48f015e2d0 0x7f48f015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f5583640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48f015e2d0 0x7f48f015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39664/0 (socket says 172.21.15.113:39664) 2024-04-25T00:03:50.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f5583640 1 -- 172.21.15.113:0/98331901 learned_addr learned my addr 172.21.15.113:0/98331901 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f4d82640 1 --1- 172.21.15.113:0/98331901 >> v1:172.21.15.113:6789/0 conn(0x7f48f00b3080 0x7f48f00b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38084/0 (socket says 172.21.15.113:38084) 2024-04-25T00:03:50.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f5583640 1 -- 172.21.15.113:0/98331901 >> v1:172.21.15.178:6789/0 conn(0x7f48f00b4a40 legacy=0x7f48f00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f5583640 1 -- 172.21.15.113:0/98331901 >> v1:172.21.15.113:6789/0 conn(0x7f48f00b3080 legacy=0x7f48f00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48effff640 1 -- 172.21.15.113:0/98331901 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2311537989 0 0) 0x7f48f015f4f0 con 0x7f48f00b3080 2024-04-25T00:03:50.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.200+0000 7f48f5583640 1 -- 172.21.15.113:0/98331901 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f015fab0 con 0x7f48f015e2d0 2024-04-25T00:03:50.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.201+0000 7f48f5583640 1 --2- 172.21.15.113:0/98331901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f015e2d0 0x7f48f015ebf0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f48f00cdf80 tx=0x7f48e802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7e952298ea1e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.201+0000 7f48effff640 1 -- 172.21.15.113:0/98331901 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8037070 con 0x7f48f015e2d0 2024-04-25T00:03:50.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.201+0000 7f48effff640 1 -- 172.21.15.113:0/98331901 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e8032640 con 0x7f48f015e2d0 2024-04-25T00:03:50.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.201+0000 7f48effff640 1 -- 172.21.15.113:0/98331901 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8032a10 con 0x7f48f015e2d0 2024-04-25T00:03:50.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.202+0000 7f48f77e7640 1 -- 172.21.15.113:0/98331901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f015e2d0 msgr2=0x7f48f015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.202+0000 7f48f77e7640 1 --2- 172.21.15.113:0/98331901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f015e2d0 0x7f48f015ebf0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f48f00cdf80 tx=0x7f48e802a2e0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.202+0000 7f48f77e7640 1 -- 172.21.15.113:0/98331901 shutdown_connections 2024-04-25T00:03:50.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.202+0000 7f48f77e7640 1 --2- 172.21.15.113:0/98331901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f015e2d0 0x7f48f015ebf0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.202+0000 7f48f77e7640 1 -- 172.21.15.113:0/98331901 >> 172.21.15.113:0/98331901 conn(0x7f48f006b0f0 msgr2=0x7f48f006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.202+0000 7f48f77e7640 1 -- 172.21.15.113:0/98331901 shutdown_connections 2024-04-25T00:03:50.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.202+0000 7f48f77e7640 1 -- 172.21.15.113:0/98331901 wait complete. 2024-04-25T00:03:50.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.203+0000 7f48f77e7640 1 Processor -- start 2024-04-25T00:03:50.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.203+0000 7f48f77e7640 1 -- start start 2024-04-25T00:03:50.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f77e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f00b3080 0x7f48f0147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f77e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48f00b4a40 0x7f48f015b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f77e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48f015e2d0 0x7f48f015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f77e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f48f015f2b0 con 0x7f48f00b3080 2024-04-25T00:03:50.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f77e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48f015f130 con 0x7f48f015e2d0 2024-04-25T00:03:50.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f77e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48f015e6b0 con 0x7f48f00b4a40 2024-04-25T00:03:50.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f00b3080 0x7f48f0147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f00b3080 0x7f48f0147c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39672/0 (socket says 172.21.15.113:39672) 2024-04-25T00:03:50.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 -- 172.21.15.113:0/4117981461 learned_addr learned my addr 172.21.15.113:0/4117981461 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 -- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48f00b4a40 msgr2=0x7f48f015b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f4d82640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48f00b4a40 0x7f48f015b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5d84640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48f015e2d0 0x7f48f015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48f00b4a40 0x7f48f015b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 -- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48f015e2d0 msgr2=0x7f48f015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48f015e2d0 0x7f48f015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 -- 172.21.15.113:0/4117981461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f015bcb0 con 0x7f48f00b3080 2024-04-25T00:03:50.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5d84640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48f015e2d0 0x7f48f015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:50.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f4d82640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48f00b4a40 0x7f48f015b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:50.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f5583640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f00b3080 0x7f48f0147c20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000c00 tx=0x7f48e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48edffb640 1 -- 172.21.15.113:0/4117981461 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8037040 con 0x7f48f00b3080 2024-04-25T00:03:50.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48edffb640 1 -- 172.21.15.113:0/4117981461 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e8004230 con 0x7f48f00b3080 2024-04-25T00:03:50.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48edffb640 1 -- 172.21.15.113:0/4117981461 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e80306f0 con 0x7f48f00b3080 2024-04-25T00:03:50.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.204+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f015bf80 con 0x7f48f00b3080 2024-04-25T00:03:50.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.205+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f015c540 con 0x7f48f00b3080 2024-04-25T00:03:50.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.206+0000 7f48edffb640 1 -- 172.21.15.113:0/4117981461 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48e8032500 con 0x7f48f00b3080 2024-04-25T00:03:50.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.206+0000 7f48edffb640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f48dc06be30 0x7f48dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.206+0000 7f48edffb640 1 -- 172.21.15.113:0/4117981461 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f48e80abe20 con 0x7f48f00b3080 2024-04-25T00:03:50.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.206+0000 7f48f4d82640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f48dc06be30 0x7f48dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.207+0000 7f48f4d82640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f48dc06be30 0x7f48dc06e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f48f015ca90 tx=0x7f48e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.207+0000 7f48fa5d1740 1 Processor -- start 2024-04-25T00:03:50.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.207+0000 7f48fa5d1740 1 -- start start 2024-04-25T00:03:50.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a711510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a741020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741560 0x56072a716370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56072a67a330 con 0x56072a741560 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56072a622ab0 con 0x56072a741ca0 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56072a6762f0 con 0x56072a75eb80 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741560 0x56072a716370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741560 0x56072a716370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39688/0 (socket says 172.21.15.113:39688) 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 -- 172.21.15.113:0/3088539088 learned_addr learned my addr 172.21.15.113:0/3088539088 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f4d82640 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a741020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 -- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 msgr2=0x56072a711510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5583640 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a711510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a711510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 -- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 msgr2=0x56072a741020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a741020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 -- 172.21.15.113:0/3088539088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56072a4587d0 con 0x56072a741560 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5583640 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a711510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48f5d84640 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741560 0x56072a716370 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f48e0001e10 tx=0x7f48e000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a25b8953c7442c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.208+0000 7f48d6ffd640 1 -- 172.21.15.113:0/3088539088 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0018070 con 0x56072a741560 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48d6ffd640 1 -- 172.21.15.113:0/3088539088 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e000de30 con 0x56072a741560 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48d6ffd640 1 -- 172.21.15.113:0/3088539088 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0003b90 con 0x56072a741560 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 -- 172.21.15.113:0/3088539088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741560 msgr2=0x56072a716370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741560 0x56072a716370 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f48e0001e10 tx=0x7f48e000d520 comp rx=0 tx=0).stop 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 -- 172.21.15.113:0/3088539088 shutdown_connections 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a741020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741560 0x56072a716370 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/3088539088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a711510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 -- 172.21.15.113:0/3088539088 >> 172.21.15.113:0/3088539088 conn(0x56072a713780 msgr2=0x56072a75c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 -- 172.21.15.113:0/3088539088 shutdown_connections 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.209+0000 7f48fa5d1740 1 -- 172.21.15.113:0/3088539088 wait complete. 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.210+0000 7f48fa5d1740 1 -- start start 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.210+0000 7f48fa5d1740 1 Processor -- start 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a73cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a759030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a76c240 0x56072a7438c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56072a715930 con 0x56072a76c240 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56072a635f40 con 0x56072a741ca0 2024-04-25T00:03:50.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56072a714d20 con 0x56072a75eb80 2024-04-25T00:03:50.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 10 client.0 _ll_register_callbacks cb 0x56072a38dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:50.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a76c240 0x56072a7438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a76c240 0x56072a7438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39696/0 (socket says 172.21.15.113:39696) 2024-04-25T00:03:50.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 -- 172.21.15.113:0/882523069 learned_addr learned my addr 172.21.15.113:0/882523069 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5583640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a73cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f4d82640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a759030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 -- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 msgr2=0x56072a759030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a759030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 -- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 msgr2=0x56072a73cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a73cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 -- 172.21.15.113:0/882523069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56072a34e0f0 con 0x56072a76c240 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f4d82640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a759030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48f5d84640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a76c240 0x56072a7438c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f48e0016040 tx=0x7f48e0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce208c07bd13b497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48cbfff640 1 -- 172.21.15.113:0/882523069 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0021030 con 0x56072a76c240 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48cbfff640 1 -- 172.21.15.113:0/882523069 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f48e000dc30 con 0x56072a76c240 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e000de30 con 0x56072a76c240 2024-04-25T00:03:50.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48cbfff640 1 -- 172.21.15.113:0/882523069 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f48e002a050 con 0x56072a76c240 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 10 client.7218 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.211+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56072a4b8f60 con 0x56072a76c240 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48cbfff640 1 -- 172.21.15.113:0/882523069 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e000de30 con 0x56072a76c240 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48cbfff640 1 -- 172.21.15.113:0/882523069 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f48e002d020 con 0x56072a76c240 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48fa5d1740 10 client.7218 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48fa5d1740 10 client.7218 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48fa5d1740 20 client.7218 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a716d80 0x56072a717160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48fa5d1740 4 client.7218 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56072a687870 con 0x56072a716d80 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48f5583640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a716d80 0x56072a717160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48f5583640 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a716d80 0x56072a717160 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b560aacce55a8a4 server_cookie=ea0a83d68884a1b5 in_seq=0 out_seq=0 2024-04-25T00:03:50.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.212+0000 7f48cbfff640 10 client.7218 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48cbfff640 1 -- 172.21.15.113:0/882523069 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56072a687870 con 0x56072a716d80 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48cbfff640 10 client.7218 handle_command_reply: tid=0 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 client.7218 shutdown 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a76c240 msgr2=0x56072a7438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a76c240 0x56072a7438c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f48e0016040 tx=0x7f48e0001250 comp rx=0 tx=0).stop 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 shutdown_connections 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a716d80 0x56072a717160 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a76c240 0x56072a7438c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a759030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48cbfff640 0 client.7218 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/882523069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741ca0 0x56072a73cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 >> 172.21.15.113:0/882523069 conn(0x56072a741560 msgr2=0x56072a75d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 shutdown_connections 2024-04-25T00:03:50.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 1 -- 172.21.15.113:0/882523069 wait complete. 2024-04-25T00:03:50.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.213+0000 7f48fa5d1740 20 client.7218 trim_cache size 0 max 16384 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.229+0000 7f48fa5d1740 1 Processor -- start 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.229+0000 7f48fa5d1740 1 -- start start 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741560 0x56072a76b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a710c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a7111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f48f02008d0 con 0x56072a741ca0 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0158e50 con 0x56072a741560 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48f01f8520 con 0x56072a75eb80 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a710c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a710c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39698/0 (socket says 172.21.15.113:39698) 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 -- 172.21.15.113:0/1631664548 learned_addr learned my addr 172.21.15.113:0/1631664548 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 -- 172.21.15.113:0/1631664548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 msgr2=0x56072a7111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 --2- 172.21.15.113:0/1631664548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a7111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 -- 172.21.15.113:0/1631664548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741560 msgr2=0x56072a76b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 --2- 172.21.15.113:0/1631664548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741560 0x56072a76b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 -- 172.21.15.113:0/1631664548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56072a3271b0 con 0x56072a741ca0 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48f4d82640 1 --2- 172.21.15.113:0/1631664548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a710c70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c00 tx=0x7f48e4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494ba6cee63f0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48d67fc640 1 -- 172.21.15.113:0/1631664548 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4027070 con 0x56072a741ca0 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48d67fc640 1 -- 172.21.15.113:0/1631664548 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e400d7a0 con 0x56072a741ca0 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.230+0000 7f48d67fc640 1 -- 172.21.15.113:0/1631664548 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e401a070 con 0x56072a741ca0 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 -- 172.21.15.113:0/1631664548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 msgr2=0x56072a710c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/1631664548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a710c70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c00 tx=0x7f48e4011000 comp rx=0 tx=0).stop 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 -- 172.21.15.113:0/1631664548 shutdown_connections 2024-04-25T00:03:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/1631664548 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a75eb80 0x56072a7111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/1631664548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a710c70 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/1631664548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a741560 0x56072a76b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 -- 172.21.15.113:0/1631664548 >> 172.21.15.113:0/1631664548 conn(0x56072a758870 msgr2=0x56072a75c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 -- 172.21.15.113:0/1631664548 shutdown_connections 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 -- 172.21.15.113:0/1631664548 wait complete. 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.231+0000 7f48fa5d1740 1 -- start start 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.232+0000 7f48fa5d1740 1 Processor -- start 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.232+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a741560 0x56072a76a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.232+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a76ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.232+0000 7f48fa5d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a75eb80 0x56072a7d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.232+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56072a743810 con 0x56072a741ca0 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.232+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0158800 con 0x56072a75eb80 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.232+0000 7f48fa5d1740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56072a73ed90 con 0x56072a741560 2024-04-25T00:03:50.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48fa5d1740 10 client.0 _ll_register_callbacks cb 0x7f48e0016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a76ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a76ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39704/0 (socket says 172.21.15.113:39704) 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 -- 172.21.15.113:0/4235098303 learned_addr learned my addr 172.21.15.113:0/4235098303 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f5d84640 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a75eb80 0x56072a7d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f5583640 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a741560 0x56072a76a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 -- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a741560 msgr2=0x56072a76a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a741560 0x56072a76a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 -- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a75eb80 msgr2=0x56072a7d6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a75eb80 0x56072a7d6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 -- 172.21.15.113:0/4235098303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56072a76bca0 con 0x56072a741ca0 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48f4d82640 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a76ac80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f48e4020000 tx=0x7f48e40012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d484ac1a526eb9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48cbfff640 1 -- 172.21.15.113:0/4235098303 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4030060 con 0x56072a741ca0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48cbfff640 1 -- 172.21.15.113:0/4235098303 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f48e4012410 con 0x56072a741ca0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e400d7a0 con 0x56072a741ca0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48cbfff640 1 -- 172.21.15.113:0/4235098303 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f48e4027040 con 0x56072a741ca0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48fa5d1740 10 client.7224 fetch_fsmap learned FSMap version 44 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x56072a7d7ba0 con 0x56072a741ca0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.233+0000 7f48cbfff640 1 -- 172.21.15.113:0/4235098303 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e400d7a0 con 0x56072a741ca0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48cbfff640 1 -- 172.21.15.113:0/4235098303 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f48e402a070 con 0x56072a741ca0 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48fa5d1740 10 client.7224 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48fa5d1740 10 client.7224 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48fa5d1740 20 client.7224 populate_metadata read hostname 'smithi113' 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a76c240 0x56072a7d7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48fa5d1740 4 client.7224 mds_command: new command op to 6951 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T00:03:50.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 --> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] -- command(tid 0: {"prefix": "perf dump"}) -- 0x56072a681bc0 con 0x56072a76c240 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48f5583640 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a76c240 0x56072a7d7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48cbfff640 10 client.7224 ms_handle_connect on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48f5583640 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a76c240 0x56072a7d7390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed916e5e178a15e server_cookie=669d4f72a9224019 in_seq=0 out_seq=0 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48cbfff640 1 -- 172.21.15.113:0/4235098303 <== mds.0 v2:172.21.15.113:6835/3530277358 1 ==== command_reply(tid 0: 0 ) ==== 8+0+18008 (crc 0 0 0) 0x56072a681bc0 con 0x56072a76c240 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.234+0000 7f48cbfff640 10 client.7224 handle_command_reply: tid=0 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 client.7224 shutdown 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 msgr2=0x56072a76ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a76ac80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f48e4020000 tx=0x7f48e40012b0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 shutdown_connections 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:6835/3530277358,v1:172.21.15.113:6837/3530277358] conn(0x56072a76c240 0x56072a7d7390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56072a75eb80 0x56072a7d6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48cbfff640 0 client.7224 ms_handle_reset on v2:172.21.15.113:6835/3530277358 2024-04-25T00:03:50.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56072a741ca0 0x56072a76ac80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 --2- 172.21.15.113:0/4235098303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56072a741560 0x56072a76a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 >> 172.21.15.113:0/4235098303 conn(0x56072a758870 msgr2=0x56072a73d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 shutdown_connections 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 1 -- 172.21.15.113:0/4235098303 wait complete. 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48fa5d1740 20 client.7224 trim_cache size 0 max 16384 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:03:50.249 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2069, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 2063, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1030527, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 14383151, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 16, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 6, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.196495058, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.081780203, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.047344974, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.057932241, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2063, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.346062468, 2024-04-25T00:03:50.250 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000167747 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 10, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000001288, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000128 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 120800, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 25648 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 29665, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 52119, 2024-04-25T00:03:50.251 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 9171930, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 69272543, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 8, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 2.856194174, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 2.171757048, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.891191525, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.078081724, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 52118, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.343718017, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000025782 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:03:50.252 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 49106, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.007455263, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000151 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 119840 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 4019, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 4019, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1012197, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 39829196, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-04-25T00:03:50.253 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.413638461, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.223598306, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.067386804, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.104745605, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4018, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.474054992, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000117982 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.254 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2420, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.538743056, 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.007660637 2024-04-25T00:03:50.255 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3728, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.300894418, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000348952 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 25169, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 25169, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 25169, 2024-04-25T00:03:50.256 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 421.270824930, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.016737686 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 12, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 17677, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 17677, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-04-25T00:03:50.257 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 17676, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 2, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 2, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 2, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 45379, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 44941, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 82, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-04-25T00:03:50.258 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 1732, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 1732, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:03:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 11, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 1721, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 1188, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 4036, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 4051, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 3348, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 16, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 13953 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.260 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 4076, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 3869, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 2152, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-04-25T00:03:50.261 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-04-25T00:03:50.262 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 32337, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 32337, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 52, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 52, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 4, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-04-25T00:03:50.263 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 107079495, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 0, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2645, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 121.790030609, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.046045380 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 0 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 14, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 2015, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 2001, 2024-04-25T00:03:50.264 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 13, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 13, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 0, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 17677, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 19678, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 2001, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 2, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 2135, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 2133, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 103800, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 231788 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 27920, 2024-04-25T00:03:50.265 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 25169, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 5, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2000, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.764252795, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000882126 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000991419, 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000165236 2024-04-25T00:03:50.266 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6000, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 7.354427411, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001225737 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 6997, 2024-04-25T00:03:50.267 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 5.497591400, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000785706 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.268 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000039846, 2024-04-25T00:03:50.269 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000039846 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.002177280, 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002177280 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.270 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.190413593, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.190413593 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000080605, 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000080605 2024-04-25T00:03:50.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.670037872, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.208754734 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2152, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 386.372331253, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.179541046 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.272 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.039018293, 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.039018293 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-04-25T00:03:50.274 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000270906, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000270906 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.275 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.276 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8000, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 18.379192257, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002297399 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 1, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 0, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 18313, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 18313, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 75, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-25T00:03:50.278 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:03:50.279 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:03:50.280 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 250233, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 92, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 440, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 5, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-25T00:03:50.281 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 5608, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 81, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 13314341, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 659599, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-25T00:03:50.282 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 8488, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 103577491, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 33, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 8488, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8488, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 28.073397114, 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.003307421 2024-04-25T00:03:50.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8488, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 8581 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 8488, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 1786, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 6702, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 0, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 11, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 53, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 2697, 2024-04-25T00:03:50.284 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 27, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 1869, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 2036, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 1733, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 22, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:03:50.285 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-04-25T00:03:50.286 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 80, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 5, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 8, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 8, 2024-04-25T00:03:50.287 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 102, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 0, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 31 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 1, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-04-25T00:03:50.288 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 32, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 3869, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 3869, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:50.289 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 35748, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 8426500, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 35748, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 35748, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 8426500, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-04-25T00:03:50.290 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 8488, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 103983212, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 8488, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 103983212, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.291 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 8488, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 8488, 2024-04-25T00:03:50.292 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 8488, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 8488, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 3869, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 405721, 2024-04-25T00:03:50.293 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 3869, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 52, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 405721, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x55780bb46500": { 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-25T00:03:50.294 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 32337, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 102885191, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 32337, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 2645, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 102885191, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-25T00:03:50.295 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.296 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:03:50.296 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:03:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f48dc06be30 msgr2=0x7f48dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48f77e7640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f48dc06be30 0x7f48dc06e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f48f015ca90 tx=0x7f48e4009040 comp rx=0 tx=0).stop 2024-04-25T00:03:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f00b3080 msgr2=0x7f48f0147c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.235+0000 7f48f77e7640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f00b3080 0x7f48f0147c20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000c00 tx=0x7f48e80046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 shutdown_connections 2024-04-25T00:03:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f48dc06be30 0x7f48dc06e2f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48f015e2d0 0x7f48f015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48f00b4a40 0x7f48f015b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 --2- 172.21.15.113:0/4117981461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48f00b3080 0x7f48f0147c20 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 >> 172.21.15.113:0/4117981461 conn(0x7f48f006b0f0 msgr2=0x7f48f00ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 shutdown_connections 2024-04-25T00:03:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.236+0000 7f48f77e7640 1 -- 172.21.15.113:0/4117981461 wait complete. 2024-04-25T00:03:50.299 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:03:50.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:03:50.333 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:03:50.382 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:03:50.382 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:03:50.382 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:03:50.382 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:03:50.397 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:03:50.397 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:03:50.453 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:03:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.588+0000 7f50a36d6640 1 Processor -- start 2024-04-25T00:03:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.588+0000 7f50a36d6640 1 -- start start 2024-04-25T00:03:50.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a36d6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f509c144910 0x7f509c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a36d6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f509c144cf0 con 0x7f509c0b56b0 2024-04-25T00:03:50.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a36d6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f509c0b2f30 con 0x7f509c0b5240 2024-04-25T00:03:50.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a36d6640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f509c0b4510 con 0x7f509c144910 2024-04-25T00:03:50.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a1c73640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f509c0b56b0 0x7f509c0cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38096/0 (socket says 172.21.15.113:38096) 2024-04-25T00:03:50.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a1c73640 1 -- 172.21.15.113:0/298464750 learned_addr learned my addr 172.21.15.113:0/298464750 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1272410469 0 0) 0x7f509c144cf0 con 0x7f509c0b56b0 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a1472640 1 --2- 172.21.15.113:0/298464750 >> v2:172.21.15.178:3300/0 conn(0x7f509c144910 0x7f509c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5088003520 con 0x7f509c0b56b0 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2776351499 0 0) 0x7f5088003520 con 0x7f509c0b56b0 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 >> v2:172.21.15.178:3300/0 conn(0x7f509c144910 msgr2=0x7f509c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 --2- 172.21.15.113:0/298464750 >> v2:172.21.15.178:3300/0 conn(0x7f509c144910 0x7f509c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 >> v1:172.21.15.178:6789/0 conn(0x7f509c0b5240 legacy=0x7f509c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c0b3210 con 0x7f509c0b56b0 2024-04-25T00:03:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f50940019f0 con 0x7f509c0b56b0 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f50a1472640 1 --2- 172.21.15.113:0/298464750 >> v2:172.21.15.178:3300/0 conn(0x7f509c144910 0x7f509c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 >> v1:172.21.15.113:6789/0 conn(0x7f509c0b56b0 legacy=0x7f509c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5088003a00 0x7f5088003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088004800 0x7f5088004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.589+0000 7f5093fff640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5088005240 0x7f508800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f509c0b4510 con 0x7f5088005240 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f508800c190 con 0x7f5088003a00 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f508800c390 con 0x7f5088004800 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1c73640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5088005240 0x7f508800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1472640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5088003a00 0x7f5088003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a0c71640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088004800 0x7f5088004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1c73640 1 -- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088004800 msgr2=0x7f5088004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1c73640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088004800 0x7f5088004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1c73640 1 -- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5088003a00 msgr2=0x7f5088003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1c73640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5088003a00 0x7f5088003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1c73640 1 -- 172.21.15.113:0/298464750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5088003520 con 0x7f5088005240 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.590+0000 7f50a1c73640 1 -- 172.21.15.113:0/298464750 reap_dead start 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a1c73640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5088005240 0x7f508800b9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f509c06ed60 tx=0x7f5088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e7f558f19b2887 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509400a3b0 con 0x7f5088005240 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509402f070 con 0x7f5088005240 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f5093fff640 1 -- 172.21.15.113:0/298464750 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509402bb50 con 0x7f5088005240 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a36d6640 1 -- 172.21.15.113:0/298464750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5088005240 msgr2=0x7f508800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a36d6640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5088005240 0x7f508800b9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f509c06ed60 tx=0x7f5088005830 comp rx=0 tx=0).stop 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a36d6640 1 -- 172.21.15.113:0/298464750 shutdown_connections 2024-04-25T00:03:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a36d6640 1 --2- 172.21.15.113:0/298464750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5088005240 0x7f508800b9c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a36d6640 1 -- 172.21.15.113:0/298464750 >> 172.21.15.113:0/298464750 conn(0x7f509c0b1a50 msgr2=0x7f509c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a36d6640 1 -- 172.21.15.113:0/298464750 shutdown_connections 2024-04-25T00:03:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.591+0000 7f50a36d6640 1 -- 172.21.15.113:0/298464750 wait complete. 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.592+0000 7f50a36d6640 1 Processor -- start 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.592+0000 7f50a36d6640 1 -- start start 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a36d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088005240 0x7f509c13cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a36d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f509c0ab600 0x7f509c15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a36d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c13d910 0x7f509c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a36d6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f509c0b4da0 con 0x7f509c13d910 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a36d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f509c0b37d0 con 0x7f509c0ab600 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a36d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f509c0b4f20 con 0x7f5088005240 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c13d910 0x7f509c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c13d910 0x7f509c15b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39718/0 (socket says 172.21.15.113:39718) 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 -- 172.21.15.113:0/2311550114 learned_addr learned my addr 172.21.15.113:0/2311550114 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a0c71640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f509c0ab600 0x7f509c15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 -- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088005240 msgr2=0x7f509c13cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088005240 0x7f509c13cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 -- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f509c0ab600 msgr2=0x7f509c15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f509c0ab600 0x7f509c15b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.593+0000 7f50a1c73640 1 -- 172.21.15.113:0/2311550114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c0abae0 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.594+0000 7f50a0c71640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f509c0ab600 0x7f509c15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.594+0000 7f50a1c73640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c13d910 0x7f509c15b890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5094000f80 tx=0x7f509402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.594+0000 7f5091ffb640 1 -- 172.21.15.113:0/2311550114 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094044070 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.594+0000 7f5091ffb640 1 -- 172.21.15.113:0/2311550114 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509402bc00 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.594+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509c0abdc0 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.594+0000 7f5091ffb640 1 -- 172.21.15.113:0/2311550114 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094034470 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.594+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509c0ac380 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.595+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509c0691a0 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.595+0000 7f5091ffb640 1 -- 172.21.15.113:0/2311550114 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5094047070 con 0x7f509c13d910 2024-04-25T00:03:50.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.596+0000 7f5091ffb640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f508006be30 0x7f508006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.596+0000 7f50a1472640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f508006be30 0x7f508006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.596+0000 7f5091ffb640 1 -- 172.21.15.113:0/2311550114 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f50940aaad0 con 0x7f509c13d910 2024-04-25T00:03:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.596+0000 7f50a1472640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f508006be30 0x7f508006e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fa0 tx=0x7f5098006f10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.600+0000 7f5091ffb640 1 -- 172.21.15.113:0/2311550114 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5094078f00 con 0x7f509c13d910 2024-04-25T00:03:50.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.774+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f509c0691a0 con 0x7f509c13d910 2024-04-25T00:03:50.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.775+0000 7f5091ffb640 1 -- 172.21.15.113:0/2311550114 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16879 (secure 0 0 0) 0x7f509407d200 con 0x7f509c13d910 2024-04-25T00:03:50.775 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:50.775 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:03:29.825357+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:02:19.562437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:02:20.315341+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:03:50.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f508006be30 msgr2=0x7f508006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f508006be30 0x7f508006e2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fa0 tx=0x7f5098006f10 comp rx=0 tx=0).stop 2024-04-25T00:03:50.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c13d910 msgr2=0x7f509c15b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c13d910 0x7f509c15b890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5094000f80 tx=0x7f509402d070 comp rx=0 tx=0).stop 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 shutdown_connections 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f508006be30 0x7f508006e2f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f509c0ab600 0x7f509c15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509c13d910 0x7f509c15b890 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 --2- 172.21.15.113:0/2311550114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5088005240 0x7f509c13cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 >> 172.21.15.113:0/2311550114 conn(0x7f509c0b1a50 msgr2=0x7f509c06cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 shutdown_connections 2024-04-25T00:03:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.777+0000 7f50a36d6640 1 -- 172.21.15.113:0/2311550114 wait complete. 2024-04-25T00:03:50.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:02:19.562437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:02:20.315341+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 79, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:03:50.789 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:03:50.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.889+0000 7f724dcf5640 1 Processor -- start 2024-04-25T00:03:50.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.889+0000 7f724dcf5640 1 -- start start 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f724dcf5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f724815e2d0 0x7f724815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f724dcf5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 0x7f72480b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f724dcf5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724815f4f0 con 0x7f724815e6b0 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f724dcf5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f72480cde00 con 0x7f724815e2d0 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f724dcf5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f724815f7d0 con 0x7f72480b3080 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f7247fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f724815e6b0 0x7f7248071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38112/0 (socket says 172.21.15.113:38112) 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f7247fff640 1 -- 172.21.15.113:0/2500050611 learned_addr learned my addr 172.21.15.113:0/2500050611 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819178110 0 0) 0x7f724815f4f0 con 0x7f724815e6b0 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7230005730 con 0x7f724815e6b0 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f72477fe640 1 --2- 172.21.15.113:0/2500050611 >> v2:172.21.15.178:3300/0 conn(0x7f724815e2d0 0x7f724815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.890+0000 7f7246ffd640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 0x7f72480b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 294358380 0 0) 0x7f7230005730 con 0x7f724815e6b0 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 msgr2=0x7f72480b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 0x7f72480b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 >> v2:172.21.15.178:3300/0 conn(0x7f724815e2d0 msgr2=0x7f724815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 --2- 172.21.15.113:0/2500050611 >> v2:172.21.15.178:3300/0 conn(0x7f724815e2d0 0x7f724815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724815fab0 con 0x7f724815e6b0 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f72400019f0 con 0x7f724815e6b0 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 >> v1:172.21.15.113:6789/0 conn(0x7f724815e6b0 legacy=0x7f7248071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f7230005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7230006a10 0x7f7230006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 0x7f72300073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f72480cde00 con 0x7f7230005bc0 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f724815f7d0 con 0x7f7230006a10 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f723000dbd0 con 0x7f72480b3080 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72477fe640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f7230005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f7246ffd640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7230006a10 0x7f7230006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f7247fff640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 0x7f72300073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72477fe640 1 -- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 msgr2=0x7f72300073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72477fe640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480b3080 0x7f72300073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72477fe640 1 -- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7230006a10 msgr2=0x7f7230006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72477fe640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7230006a10 0x7f7230006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f72477fe640 1 -- 172.21.15.113:0/2500050611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7230005730 con 0x7f7230005bc0 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.891+0000 7f7246ffd640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7230006a10 0x7f7230006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f72477fe640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f7230005fe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7238009af0 tx=0x7f7230007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc272f59b0c9112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238036070 con 0x7f7230005bc0 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7238031400 con 0x7f7230005bc0 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f72467fc640 1 -- 172.21.15.113:0/2500050611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238009690 con 0x7f7230005bc0 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f724dcf5640 1 -- 172.21.15.113:0/2500050611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 msgr2=0x7f7230005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f724dcf5640 1 --2- 172.21.15.113:0/2500050611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f7230005fe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7238009af0 tx=0x7f7230007a40 comp rx=0 tx=0).stop 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f7247fff640 1 -- 172.21.15.113:0/2500050611 reap_dead start 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f724dcf5640 1 -- 172.21.15.113:0/2500050611 shutdown_connections 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f724dcf5640 1 -- 172.21.15.113:0/2500050611 >> 172.21.15.113:0/2500050611 conn(0x7f724806b0f0 msgr2=0x7f724806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.892+0000 7f724dcf5640 1 -- 172.21.15.113:0/2500050611 shutdown_connections 2024-04-25T00:03:50.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.893+0000 7f724dcf5640 1 -- 172.21.15.113:0/2500050611 wait complete. 2024-04-25T00:03:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 Processor -- start 2024-04-25T00:03:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 -- start start 2024-04-25T00:03:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f72480cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f724815c4b0 0x7f72480cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480adb00 0x7f72480adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f724815f2b0 con 0x7f7230005bc0 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f724815f130 con 0x7f72480adb00 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.894+0000 7f724dcf5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f72480cf2a0 con 0x7f724815c4b0 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f72480cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f72480cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39728/0 (socket says 172.21.15.113:39728) 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 -- 172.21.15.113:0/2875178952 learned_addr learned my addr 172.21.15.113:0/2875178952 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f7246ffd640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f724815c4b0 0x7f72480cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 -- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f724815c4b0 msgr2=0x7f72480cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f7247fff640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480adb00 0x7f72480adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f724815c4b0 0x7f72480cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 -- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480adb00 msgr2=0x7f72480adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480adb00 0x7f72480adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 -- 172.21.15.113:0/2875178952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72480b2a50 con 0x7f7230005bc0 2024-04-25T00:03:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f7246ffd640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f724815c4b0 0x7f72480cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f7247fff640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480adb00 0x7f72480adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f72477fe640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f72480cc320 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7238009af0 tx=0x7f7238004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f722ffff640 1 -- 172.21.15.113:0/2875178952 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238041020 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f722ffff640 1 -- 172.21.15.113:0/2875178952 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f723802ade0 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72480b19b0 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f722ffff640 1 -- 172.21.15.113:0/2875178952 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238036040 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.895+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72480b1f70 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.896+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72480691a0 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.896+0000 7f722ffff640 1 -- 172.21.15.113:0/2875178952 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7238044020 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.897+0000 7f722ffff640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f723406be30 0x7f723406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.897+0000 7f722ffff640 1 -- 172.21.15.113:0/2875178952 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f72380ab3e0 con 0x7f7230005bc0 2024-04-25T00:03:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.897+0000 7f7246ffd640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f723406be30 0x7f723406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.897+0000 7f7246ffd640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f723406be30 0x7f723406e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f72480d0dc0 tx=0x7f723c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:50.901+0000 7f722ffff640 1 -- 172.21.15.113:0/2875178952 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f723804b030 con 0x7f7230005bc0 2024-04-25T00:03:51.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.072+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7248072050 con 0x7f7230005bc0 2024-04-25T00:03:51.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.072+0000 7f722ffff640 1 -- 172.21.15.113:0/2875178952 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f7238079d30 con 0x7f7230005bc0 2024-04-25T00:03:51.073 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:03:51.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f723406be30 msgr2=0x7f723406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f723406be30 0x7f723406e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f72480d0dc0 tx=0x7f723c008040 comp rx=0 tx=0).stop 2024-04-25T00:03:51.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 msgr2=0x7f72480cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f72480cc320 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7238009af0 tx=0x7f7238004730 comp rx=0 tx=0).stop 2024-04-25T00:03:51.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 shutdown_connections 2024-04-25T00:03:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f723406be30 0x7f723406e2f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480adb00 0x7f72480adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f724815c4b0 0x7f72480cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 --2- 172.21.15.113:0/2875178952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230005bc0 0x7f72480cc320 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.074+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 >> 172.21.15.113:0/2875178952 conn(0x7f724806b0f0 msgr2=0x7f72480b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.075+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 shutdown_connections 2024-04-25T00:03:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.075+0000 7f724dcf5640 1 -- 172.21.15.113:0/2875178952 wait complete. 2024-04-25T00:03:51.085 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:03:51.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.185+0000 7f2bebb63640 1 Processor -- start 2024-04-25T00:03:51.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.185+0000 7f2bebb63640 1 -- start start 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2bebb63640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2be40afa40 0x7f2be40cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2bebb63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2bebb63640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2be415f060 0x7f2be415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2bebb63640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2be4071ac0 con 0x7f2be40afa40 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2bebb63640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2be415fa50 con 0x7f2be415f060 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2bebb63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2be40cd290 con 0x7f2be40afe20 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2be40afa40 0x7f2be40cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2be40afa40 0x7f2be40cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39738/0 (socket says 172.21.15.113:39738) 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 -- 172.21.15.113:0/4275682084 learned_addr learned my addr 172.21.15.113:0/4275682084 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2bea100640 1 --2- 172.21.15.113:0/4275682084 >> v2:172.21.15.178:3300/0 conn(0x7f2be415f060 0x7f2be415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be90fe640 1 --2- 172.21.15.113:0/4275682084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 -- 172.21.15.113:0/4275682084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 msgr2=0x7f2be415eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 --2- 172.21.15.113:0/4275682084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 -- 172.21.15.113:0/4275682084 >> v2:172.21.15.178:3300/0 conn(0x7f2be415f060 msgr2=0x7f2be415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 --2- 172.21.15.113:0/4275682084 >> v2:172.21.15.178:3300/0 conn(0x7f2be415f060 0x7f2be415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.186+0000 7f2be98ff640 1 -- 172.21.15.113:0/4275682084 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be415f4f0 con 0x7f2be40afa40 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2be98ff640 1 --2- 172.21.15.113:0/4275682084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be40cd720 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2be41631d0 tx=0x7f2be002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf43a9550938d6a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2be88fd640 1 -- 172.21.15.113:0/4275682084 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0037070 con 0x7f2be40afa40 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2be88fd640 1 -- 172.21.15.113:0/4275682084 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be0032440 con 0x7f2be40afa40 2024-04-25T00:03:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2be88fd640 1 -- 172.21.15.113:0/4275682084 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0032810 con 0x7f2be40afa40 2024-04-25T00:03:51.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 -- 172.21.15.113:0/4275682084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 msgr2=0x7f2be40cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 --2- 172.21.15.113:0/4275682084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be40cd720 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2be41631d0 tx=0x7f2be002a140 comp rx=0 tx=0).stop 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 -- 172.21.15.113:0/4275682084 shutdown_connections 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 --2- 172.21.15.113:0/4275682084 >> v2:172.21.15.178:3300/0 conn(0x7f2be415f060 0x7f2be415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 --2- 172.21.15.113:0/4275682084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 --2- 172.21.15.113:0/4275682084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be40cd720 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 -- 172.21.15.113:0/4275682084 >> 172.21.15.113:0/4275682084 conn(0x7f2be40b57e0 msgr2=0x7f2be4072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 -- 172.21.15.113:0/4275682084 shutdown_connections 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.187+0000 7f2bebb63640 1 -- 172.21.15.113:0/4275682084 wait complete. 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 Processor -- start 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 -- start start 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be41f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be415f060 0x7f2be415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2be40ccc40 con 0x7f2be40afa40 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4158f60 con 0x7f2be415f060 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2bebb63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2be40ccf20 con 0x7f2be40afe20 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2be98ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be41f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2be98ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be41f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39740/0 (socket says 172.21.15.113:39740) 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.189+0000 7f2be98ff640 1 -- 172.21.15.113:0/1373334145 learned_addr learned my addr 172.21.15.113:0/1373334145 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be90fe640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2bea100640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be415f060 0x7f2be415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be98ff640 1 -- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 msgr2=0x7f2be415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be98ff640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be98ff640 1 -- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be415f060 msgr2=0x7f2be415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be98ff640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be415f060 0x7f2be415c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be98ff640 1 -- 172.21.15.113:0/1373334145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be41ee750 con 0x7f2be40afa40 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2bea100640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be415f060 0x7f2be415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:51.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be90fe640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2be98ff640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be41f4cc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000c00 tx=0x7f2be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2bda7fc640 1 -- 172.21.15.113:0/1373334145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0041070 con 0x7f2be40afa40 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2bda7fc640 1 -- 172.21.15.113:0/1373334145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be003c870 con 0x7f2be40afa40 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be41f5200 con 0x7f2be40afa40 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2bda7fc640 1 -- 172.21.15.113:0/1373334145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0037040 con 0x7f2be40afa40 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.190+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be40ad490 con 0x7f2be40afa40 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.191+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be40691a0 con 0x7f2be40afa40 2024-04-25T00:03:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.191+0000 7f2bda7fc640 1 -- 172.21.15.113:0/1373334145 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2be0044070 con 0x7f2be40afa40 2024-04-25T00:03:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.192+0000 7f2bda7fc640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2bc806be30 0x7f2bc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.192+0000 7f2be90fe640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2bc806be30 0x7f2bc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.192+0000 7f2bda7fc640 1 -- 172.21.15.113:0/1373334145 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f2be00ab9f0 con 0x7f2be40afa40 2024-04-25T00:03:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.192+0000 7f2be90fe640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2bc806be30 0x7f2bc806e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2be415d270 tx=0x7f2bd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.196+0000 7f2bda7fc640 1 -- 172.21.15.113:0/1373334145 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2be41ee750 con 0x7f2be40afa40 2024-04-25T00:03:51.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.366+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2be40cd7c0 con 0x7f2be40afa40 2024-04-25T00:03:51.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.367+0000 7f2bda7fc640 1 -- 172.21.15.113:0/1373334145 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f2be415f4f0 con 0x7f2be40afa40 2024-04-25T00:03:51.367 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:03:51.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2bc806be30 msgr2=0x7f2bc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2bc806be30 0x7f2bc806e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2be415d270 tx=0x7f2bd4008040 comp rx=0 tx=0).stop 2024-04-25T00:03:51.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 msgr2=0x7f2be41f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be41f4cc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000c00 tx=0x7f2be00046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 shutdown_connections 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2bc806be30 0x7f2bc806e2f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be415f060 0x7f2be415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be40afe20 0x7f2be415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 --2- 172.21.15.113:0/1373334145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be40afa40 0x7f2be41f4cc0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 >> 172.21.15.113:0/1373334145 conn(0x7f2be40b57e0 msgr2=0x7f2be406dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 shutdown_connections 2024-04-25T00:03:51.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.369+0000 7f2bebb63640 1 -- 172.21.15.113:0/1373334145 wait complete. 2024-04-25T00:03:51.380 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:03:51.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77dc909640 1 Processor -- start 2024-04-25T00:03:51.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77dc909640 1 -- start start 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77dc909640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77d815e510 0x7f77d80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77dc909640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f77d80d28c0 0x7f77d80b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77dc909640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d80d2e00 con 0x7f77d80b61e0 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77dc909640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f77d80b6610 con 0x7f77d815e510 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77dc909640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f77d80d30e0 con 0x7f77d80d28c0 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.479+0000 7f77d659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77d815e510 0x7f77d80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77d659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77d815e510 0x7f77d80d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39744/0 (socket says 172.21.15.113:39744) 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77d659c640 1 -- 172.21.15.113:0/76209250 learned_addr learned my addr 172.21.15.113:0/76209250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77cffff640 1 --2- 172.21.15.113:0/76209250 >> v2:172.21.15.178:3300/0 conn(0x7f77d80d28c0 0x7f77d80b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77d659c640 1 -- 172.21.15.113:0/76209250 >> v2:172.21.15.178:3300/0 conn(0x7f77d80d28c0 msgr2=0x7f77d80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77d659c640 1 --2- 172.21.15.113:0/76209250 >> v2:172.21.15.178:3300/0 conn(0x7f77d80d28c0 0x7f77d80b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77d659c640 1 -- 172.21.15.113:0/76209250 >> v1:172.21.15.178:6789/0 conn(0x7f77d80b61e0 legacy=0x7f77d8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77d659c640 1 -- 172.21.15.113:0/76209250 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80d33c0 con 0x7f77d815e510 2024-04-25T00:03:51.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.480+0000 7f77d659c640 1 --2- 172.21.15.113:0/76209250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d815e510 0x7f77d80d2380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f77d80d52f0 tx=0x7f77c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78087afd099124b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77d5d9b640 1 -- 172.21.15.113:0/76209250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8037070 con 0x7f77d815e510 2024-04-25T00:03:51.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77d5d9b640 1 -- 172.21.15.113:0/76209250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8032440 con 0x7f77d815e510 2024-04-25T00:03:51.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77d5d9b640 1 -- 172.21.15.113:0/76209250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8032810 con 0x7f77d815e510 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 -- 172.21.15.113:0/76209250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d815e510 msgr2=0x7f77d80d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 --2- 172.21.15.113:0/76209250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d815e510 0x7f77d80d2380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f77d80d52f0 tx=0x7f77c802a1e0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 -- 172.21.15.113:0/76209250 shutdown_connections 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 --2- 172.21.15.113:0/76209250 >> v2:172.21.15.178:3300/0 conn(0x7f77d80d28c0 0x7f77d80b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 --2- 172.21.15.113:0/76209250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d815e510 0x7f77d80d2380 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 -- 172.21.15.113:0/76209250 >> 172.21.15.113:0/76209250 conn(0x7f77d806b720 msgr2=0x7f77d80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 -- 172.21.15.113:0/76209250 shutdown_connections 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.481+0000 7f77dc909640 1 -- 172.21.15.113:0/76209250 wait complete. 2024-04-25T00:03:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.482+0000 7f77dc909640 1 Processor -- start 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77dc909640 1 -- start start 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77dc909640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d80b61e0 0x7f77d813e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77dc909640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d80d28c0 0x7f77d8151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77dc909640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815e510 0x7f77d80b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77dc909640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d3ea0 con 0x7f77d80b61e0 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77dc909640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d3980 con 0x7f77d80d28c0 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77dc909640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80b9dd0 con 0x7f77d815e510 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d80b61e0 0x7f77d813e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d80b61e0 0x7f77d813e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39760/0 (socket says 172.21.15.113:39760) 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77d659c640 1 -- 172.21.15.113:0/311141433 learned_addr learned my addr 172.21.15.113:0/311141433 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77d6d9d640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815e510 0x7f77d80b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.483+0000 7f77cffff640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d80d28c0 0x7f77d8151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77d659c640 1 -- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815e510 msgr2=0x7f77d80b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77d659c640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815e510 0x7f77d80b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77d659c640 1 -- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d80d28c0 msgr2=0x7f77d8151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77d659c640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d80d28c0 0x7f77d8151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77d659c640 1 -- 172.21.15.113:0/311141433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d8141020 con 0x7f77d80b61e0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77d659c640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d80b61e0 0x7f77d813e030 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000c00 tx=0x7f77c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77ceffd640 1 -- 172.21.15.113:0/311141433 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8041020 con 0x7f77d80b61e0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77ceffd640 1 -- 172.21.15.113:0/311141433 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c80306c0 con 0x7f77d80b61e0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d81403a0 con 0x7f77d80b61e0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77ceffd640 1 -- 172.21.15.113:0/311141433 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8037040 con 0x7f77d80b61e0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.484+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d8140960 con 0x7f77d80b61e0 2024-04-25T00:03:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.485+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d80691a0 con 0x7f77d80b61e0 2024-04-25T00:03:51.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.486+0000 7f77ceffd640 1 -- 172.21.15.113:0/311141433 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77c8044020 con 0x7f77d80b61e0 2024-04-25T00:03:51.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.486+0000 7f77ceffd640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f77b806be30 0x7f77b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.486+0000 7f77ceffd640 1 -- 172.21.15.113:0/311141433 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f77c80ab4f0 con 0x7f77d80b61e0 2024-04-25T00:03:51.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.486+0000 7f77cffff640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f77b806be30 0x7f77b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.486+0000 7f77cffff640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f77b806be30 0x7f77b806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f77d80b50b0 tx=0x7f77c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.490+0000 7f77ceffd640 1 -- 172.21.15.113:0/311141433 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77c804b030 con 0x7f77d80b61e0 2024-04-25T00:03:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.660+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77d80b9700 con 0x7f77d80b61e0 2024-04-25T00:03:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.660+0000 7f77ceffd640 1 -- 172.21.15.113:0/311141433 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f77c8079880 con 0x7f77d80b61e0 2024-04-25T00:03:51.661 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:03:51.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f77b806be30 msgr2=0x7f77b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f77b806be30 0x7f77b806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f77d80b50b0 tx=0x7f77c0008040 comp rx=0 tx=0).stop 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d80b61e0 msgr2=0x7f77d813e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d80b61e0 0x7f77d813e030 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000c00 tx=0x7f77c80046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 shutdown_connections 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f77b806be30 0x7f77b806e2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d80d28c0 0x7f77d8151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815e510 0x7f77d80b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 --2- 172.21.15.113:0/311141433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d80b61e0 0x7f77d813e030 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 >> 172.21.15.113:0/311141433 conn(0x7f77d806b720 msgr2=0x7f77d80d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 shutdown_connections 2024-04-25T00:03:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.662+0000 7f77dc909640 1 -- 172.21.15.113:0/311141433 wait complete. 2024-04-25T00:03:51.673 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:03:51.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.771+0000 7f21f78bd640 1 Processor -- start 2024-04-25T00:03:51.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.771+0000 7f21f78bd640 1 -- start start 2024-04-25T00:03:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f78bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00b3080 0x7f21f00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f78bd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21f015e6b0 0x7f21f00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f78bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f015f4f0 con 0x7f21f015e2d0 2024-04-25T00:03:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f78bd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f21f00cde00 con 0x7f21f015e6b0 2024-04-25T00:03:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f78bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f21f015f7d0 con 0x7f21f00b3080 2024-04-25T00:03:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f5e5a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21f015e6b0 0x7f21f00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f5659640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f21f015e2d0 0x7f21f015ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38116/0 (socket says 172.21.15.113:38116) 2024-04-25T00:03:51.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f5659640 1 -- 172.21.15.113:0/1061410919 learned_addr learned my addr 172.21.15.113:0/1061410919 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:51.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f5e5a640 1 -- 172.21.15.113:0/1061410919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00b3080 msgr2=0x7f21f00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:51.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f5e5a640 1 --2- 172.21.15.113:0/1061410919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00b3080 0x7f21f00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f5e5a640 1 -- 172.21.15.113:0/1061410919 >> v1:172.21.15.113:6789/0 conn(0x7f21f015e2d0 legacy=0x7f21f015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21e7fff640 1 -- 172.21.15.113:0/1061410919 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 776619438 0 0) 0x7f21f015f4f0 con 0x7f21f015e2d0 2024-04-25T00:03:51.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.772+0000 7f21f5e5a640 1 -- 172.21.15.113:0/1061410919 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f015fab0 con 0x7f21f015e6b0 2024-04-25T00:03:51.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21f5e5a640 1 --2- 172.21.15.113:0/1061410919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f015e6b0 0x7f21f00b4870 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f21e80097a0 tx=0x7f21e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6e438a4e70bb8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21e7fff640 1 -- 172.21.15.113:0/1061410919 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8037070 con 0x7f21f015e6b0 2024-04-25T00:03:51.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21e7fff640 1 -- 172.21.15.113:0/1061410919 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e802ae50 con 0x7f21f015e6b0 2024-04-25T00:03:51.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21e7fff640 1 -- 172.21.15.113:0/1061410919 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8032660 con 0x7f21f015e6b0 2024-04-25T00:03:51.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21f78bd640 1 -- 172.21.15.113:0/1061410919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f015e6b0 msgr2=0x7f21f00b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1061410919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f015e6b0 0x7f21f00b4870 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f21e80097a0 tx=0x7f21e802a100 comp rx=0 tx=0).stop 2024-04-25T00:03:51.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21f78bd640 1 -- 172.21.15.113:0/1061410919 shutdown_connections 2024-04-25T00:03:51.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1061410919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f015e6b0 0x7f21f00b4870 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1061410919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00b3080 0x7f21f00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.773+0000 7f21f78bd640 1 -- 172.21.15.113:0/1061410919 >> 172.21.15.113:0/1061410919 conn(0x7f21f006b0f0 msgr2=0x7f21f006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:51.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.774+0000 7f21f78bd640 1 -- 172.21.15.113:0/1061410919 shutdown_connections 2024-04-25T00:03:51.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.774+0000 7f21f78bd640 1 -- 172.21.15.113:0/1061410919 wait complete. 2024-04-25T00:03:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 Processor -- start 2024-04-25T00:03:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 -- start start 2024-04-25T00:03:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f00b3080 0x7f21f0150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f015e2d0 0x7f21f00ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f015e6b0 0x7f21f00b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f21f006d020 con 0x7f21f00b3080 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f21f006cea0 con 0x7f21f015e6b0 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f78bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f21f006d1a0 con 0x7f21f015e2d0 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f5659640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f00b3080 0x7f21f0150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f5659640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f00b3080 0x7f21f0150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39780/0 (socket says 172.21.15.113:39780) 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.775+0000 7f21f5659640 1 -- 172.21.15.113:0/1978070112 learned_addr learned my addr 172.21.15.113:0/1978070112 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f5659640 1 -- 172.21.15.113:0/1978070112 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f015e2d0 msgr2=0x7f21f00ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f5659640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f015e2d0 0x7f21f00ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f5659640 1 -- 172.21.15.113:0/1978070112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f015e6b0 msgr2=0x7f21f00b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f5659640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f015e6b0 0x7f21f00b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f5659640 1 -- 172.21.15.113:0/1978070112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f00a9620 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f5659640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f00b3080 0x7f21f0150d40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c890 tx=0x7f21ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21e5ffb640 1 -- 172.21.15.113:0/1978070112 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec018070 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21e5ffb640 1 -- 172.21.15.113:0/1978070112 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec00a640 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f00a9900 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21e5ffb640 1 -- 172.21.15.113:0/1978070112 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00aa30 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.776+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f00a9ec0 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.777+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f00691a0 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.777+0000 7f21e5ffb640 1 -- 172.21.15.113:0/1978070112 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21ec01b070 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.778+0000 7f21e5ffb640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f21d406be30 0x7f21d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.778+0000 7f21f4e58640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f21d406be30 0x7f21d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.778+0000 7f21e5ffb640 1 -- 172.21.15.113:0/1978070112 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f21ec08a4e0 con 0x7f21f00b3080 2024-04-25T00:03:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.778+0000 7f21f4e58640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f21d406be30 0x7f21d406e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f21f00d09b0 tx=0x7f21e0006c90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:51.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.782+0000 7f21e5ffb640 1 -- 172.21.15.113:0/1978070112 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f21ec058ec0 con 0x7f21f00b3080 2024-04-25T00:03:51.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.969+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f21f01556a0 con 0x7f21f00b3080 2024-04-25T00:03:51.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.969+0000 7f21e5ffb640 1 -- 172.21.15.113:0/1978070112 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7f21ec0587c0 con 0x7f21f00b3080 2024-04-25T00:03:51.970 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:03:51.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f21d406be30 msgr2=0x7f21d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f21d406be30 0x7f21d406e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f21f00d09b0 tx=0x7f21e0006c90 comp rx=0 tx=0).stop 2024-04-25T00:03:51.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f00b3080 msgr2=0x7f21f0150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:51.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f00b3080 0x7f21f0150d40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c890 tx=0x7f21ec00cd50 comp rx=0 tx=0).stop 2024-04-25T00:03:51.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 shutdown_connections 2024-04-25T00:03:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f21d406be30 0x7f21d406e2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f015e6b0 0x7f21f00b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f015e2d0 0x7f21f00ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.971+0000 7f21f78bd640 1 --2- 172.21.15.113:0/1978070112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f00b3080 0x7f21f0150d40 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.972+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 >> 172.21.15.113:0/1978070112 conn(0x7f21f006b0f0 msgr2=0x7f21f0154a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.972+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 shutdown_connections 2024-04-25T00:03:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:51.972+0000 7f21f78bd640 1 -- 172.21.15.113:0/1978070112 wait complete. 2024-04-25T00:03:51.983 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:03:51.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:03:51.983 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:03:52.032 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:03:52.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:03:52.032 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:03:52.053 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:03:52.053 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:03:52.053 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:03:52.072 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:03:52.072 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:03:52.073 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:03:52.073 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:03:52.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:03:52.109 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:03:52.109 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:03:52.109 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:03:52.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:03:52.130 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:03:52.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.246+0000 7ff0d7431640 1 Processor -- start 2024-04-25T00:03:52.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.246+0000 7ff0d7431640 1 -- start start 2024-04-25T00:03:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d7431640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0d015b820 0x7ff0d00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d7431640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d015da40 0x7ff0d0164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d7431640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d0155a10 con 0x7ff0d00b3ed0 2024-04-25T00:03:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d7431640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d015e930 con 0x7ff0d015b820 2024-04-25T00:03:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d7431640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d00b4f10 con 0x7ff0d015da40 2024-04-25T00:03:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d49cc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff0d00b3ed0 0x7ff0d015bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38118/0 (socket says 172.21.15.113:38118) 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d49cc640 1 -- 172.21.15.113:0/3971623506 learned_addr learned my addr 172.21.15.113:0/3971623506 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d51cd640 1 --2- 172.21.15.113:0/3971623506 >> v2:172.21.15.178:3300/0 conn(0x7ff0d015b820 0x7ff0d00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0d59ce640 1 --2- 172.21.15.113:0/3971623506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d015da40 0x7ff0d0164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0c7fff640 1 -- 172.21.15.113:0/3971623506 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239979978 0 0) 0x7ff0d0155a10 con 0x7ff0d00b3ed0 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.247+0000 7ff0c7fff640 1 -- 172.21.15.113:0/3971623506 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0bc003520 con 0x7ff0d00b3ed0 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0c7fff640 1 -- 172.21.15.113:0/3971623506 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2119776484 0 0) 0x7ff0bc003520 con 0x7ff0d00b3ed0 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0d59ce640 1 -- 172.21.15.113:0/3971623506 >> v2:172.21.15.178:3300/0 conn(0x7ff0d015b820 msgr2=0x7ff0d00d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0d59ce640 1 --2- 172.21.15.113:0/3971623506 >> v2:172.21.15.178:3300/0 conn(0x7ff0d015b820 0x7ff0d00d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0d59ce640 1 -- 172.21.15.113:0/3971623506 >> v1:172.21.15.113:6789/0 conn(0x7ff0d00b3ed0 legacy=0x7ff0d015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0d59ce640 1 -- 172.21.15.113:0/3971623506 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d00b4390 con 0x7ff0d015da40 2024-04-25T00:03:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0d51cd640 1 --2- 172.21.15.113:0/3971623506 >> v2:172.21.15.178:3300/0 conn(0x7ff0d015b820 0x7ff0d00d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0d59ce640 1 --2- 172.21.15.113:0/3971623506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d015da40 0x7ff0d0164b60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009d90 tx=0x7ff0c802a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797cd7730057e51a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.248+0000 7ff0c7fff640 1 -- 172.21.15.113:0/3971623506 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc001670 con 0x7ff0d015da40 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0c7fff640 1 -- 172.21.15.113:0/3971623506 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c8007540 con 0x7ff0d015da40 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0c7fff640 1 -- 172.21.15.113:0/3971623506 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80078f0 con 0x7ff0d015da40 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 -- 172.21.15.113:0/3971623506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d015da40 msgr2=0x7ff0d0164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 --2- 172.21.15.113:0/3971623506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d015da40 0x7ff0d0164b60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009d90 tx=0x7ff0c802a560 comp rx=0 tx=0).stop 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 -- 172.21.15.113:0/3971623506 shutdown_connections 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 --2- 172.21.15.113:0/3971623506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d015da40 0x7ff0d0164b60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 --2- 172.21.15.113:0/3971623506 >> v2:172.21.15.178:3300/0 conn(0x7ff0d015b820 0x7ff0d00d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 -- 172.21.15.113:0/3971623506 >> 172.21.15.113:0/3971623506 conn(0x7ff0d00b27f0 msgr2=0x7ff0d00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 -- 172.21.15.113:0/3971623506 shutdown_connections 2024-04-25T00:03:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.249+0000 7ff0d7431640 1 -- 172.21.15.113:0/3971623506 wait complete. 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.250+0000 7ff0d7431640 1 Processor -- start 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.250+0000 7ff0d7431640 1 -- start start 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d7431640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d00b3ed0 0x7ff0d0149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d7431640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0d015b820 0x7ff0d014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d7431640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d015da40 0x7ff0d00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d7431640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d00b4c30 con 0x7ff0d015b820 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d7431640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d00b4950 con 0x7ff0d015da40 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d7431640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d015c310 con 0x7ff0d00b3ed0 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0d015b820 0x7ff0d014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0d015b820 0x7ff0d014f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39792/0 (socket says 172.21.15.113:39792) 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 -- 172.21.15.113:0/1736408225 learned_addr learned my addr 172.21.15.113:0/1736408225 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d59ce640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d015da40 0x7ff0d00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d51cd640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d00b3ed0 0x7ff0d0149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 -- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d00b3ed0 msgr2=0x7ff0d0149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d00b3ed0 0x7ff0d0149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 -- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d015da40 msgr2=0x7ff0d00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d015da40 0x7ff0d00ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.251+0000 7ff0d49cc640 1 -- 172.21.15.113:0/1736408225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d0152ce0 con 0x7ff0d015b820 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.252+0000 7ff0d49cc640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0d015b820 0x7ff0d014f760 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c750 tx=0x7ff0cc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.252+0000 7ff0c5ffb640 1 -- 172.21.15.113:0/1736408225 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc017070 con 0x7ff0d015b820 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.252+0000 7ff0c5ffb640 1 -- 172.21.15.113:0/1736408225 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0cc005ad0 con 0x7ff0d015b820 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.252+0000 7ff0c5ffb640 1 -- 172.21.15.113:0/1736408225 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc007070 con 0x7ff0d015b820 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.252+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d0152fc0 con 0x7ff0d015b820 2024-04-25T00:03:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.252+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d0153580 con 0x7ff0d015b820 2024-04-25T00:03:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.253+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d00691a0 con 0x7ff0d015b820 2024-04-25T00:03:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.253+0000 7ff0c5ffb640 1 -- 172.21.15.113:0/1736408225 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0cc01a070 con 0x7ff0d015b820 2024-04-25T00:03:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.253+0000 7ff0c5ffb640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0b406be30 0x7ff0b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.254+0000 7ff0c5ffb640 1 -- 172.21.15.113:0/1736408225 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7ff0cc08a280 con 0x7ff0d015b820 2024-04-25T00:03:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.254+0000 7ff0d51cd640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0b406be30 0x7ff0b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.254+0000 7ff0d51cd640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0b406be30 0x7ff0b406e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009560 tx=0x7ff0c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.258+0000 7ff0c5ffb640 1 -- 172.21.15.113:0/1736408225 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff0d00691a0 con 0x7ff0d015b820 2024-04-25T00:03:52.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.463+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0d00cffc0 con 0x7ff0d015b820 2024-04-25T00:03:52.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.464+0000 7ff0c5ffb640 1 -- 172.21.15.113:0/1736408225 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5192 (secure 0 0 0) 0x7ff0cc058580 con 0x7ff0d015b820 2024-04-25T00:03:52.465 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:52.465 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4243948985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4243948985},{"type":"v1","addr":"172.21.15.178:6835","nonce":4243948985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/665506344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":665506344},{"type":"v1","addr":"172.21.15.178:6834","nonce":665506344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4064308986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4064308986},{"type":"v1","addr":"172.21.15.113:6836","nonce":4064308986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T00:02:21.350179+0000","modified":"2024-04-25T00:02:32.368592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3530277358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3530277358},{"type":"v1","addr":"172.21.15.113:6837","nonce":3530277358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6951,"qdb_cluster":[6951]},"id":5}]} 2024-04-25T00:03:52.465 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-04-25T00:03:52.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0b406be30 msgr2=0x7ff0b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0b406be30 0x7ff0b406e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009560 tx=0x7ff0c0008040 comp rx=0 tx=0).stop 2024-04-25T00:03:52.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0d015b820 msgr2=0x7ff0d014f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0d015b820 0x7ff0d014f760 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c750 tx=0x7ff0cc00cc10 comp rx=0 tx=0).stop 2024-04-25T00:03:52.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 shutdown_connections 2024-04-25T00:03:52.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff0b406be30 0x7ff0b406e2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d015da40 0x7ff0d00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0d015b820 0x7ff0d014f760 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 --2- 172.21.15.113:0/1736408225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d00b3ed0 0x7ff0d0149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 >> 172.21.15.113:0/1736408225 conn(0x7ff0d00b27f0 msgr2=0x7ff0d00741e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 shutdown_connections 2024-04-25T00:03:52.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.466+0000 7ff0d7431640 1 -- 172.21.15.113:0/1736408225 wait complete. 2024-04-25T00:03:52.477 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:03:52.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.573+0000 7f837d476640 1 Processor -- start 2024-04-25T00:03:52.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.573+0000 7f837d476640 1 -- start start 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f837d476640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f837815e320 0x7f83780d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f837d476640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d2290 0x7f83780b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f837d476640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f83780d0b00 0x7f83780d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f837d476640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f83780d14d0 con 0x7f837815e320 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f837d476640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f83780d3410 con 0x7f83780d0b00 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f837d476640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83780d3de0 con 0x7f83780d2290 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f8376ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f837815e320 0x7f83780d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f8376ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f837815e320 0x7f83780d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39796/0 (socket says 172.21.15.113:39796) 2024-04-25T00:03:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f8376ffd640 1 -- 172.21.15.113:0/998233137 learned_addr learned my addr 172.21.15.113:0/998233137 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f8376ffd640 1 -- 172.21.15.113:0/998233137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d2290 msgr2=0x7f83780b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.574+0000 7f8376ffd640 1 --2- 172.21.15.113:0/998233137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d2290 0x7f83780b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.575+0000 7f8376ffd640 1 -- 172.21.15.113:0/998233137 >> v2:172.21.15.178:3300/0 conn(0x7f83780d0b00 msgr2=0x7f83780d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.575+0000 7f8376ffd640 1 --2- 172.21.15.113:0/998233137 >> v2:172.21.15.178:3300/0 conn(0x7f83780d0b00 0x7f83780d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.575+0000 7f8376ffd640 1 -- 172.21.15.113:0/998233137 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780d2800 con 0x7f837815e320 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.575+0000 7f8376ffd640 1 --2- 172.21.15.113:0/998233137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780d38a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f83780b7190 tx=0x7f836802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b7ab50b9c68a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.575+0000 7f8375ffb640 1 -- 172.21.15.113:0/998233137 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368036070 con 0x7f837815e320 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.575+0000 7f8375ffb640 1 -- 172.21.15.113:0/998233137 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8368031600 con 0x7f837815e320 2024-04-25T00:03:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.575+0000 7f8375ffb640 1 -- 172.21.15.113:0/998233137 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680319d0 con 0x7f837815e320 2024-04-25T00:03:52.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 -- 172.21.15.113:0/998233137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 msgr2=0x7f83780d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 --2- 172.21.15.113:0/998233137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780d38a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f83780b7190 tx=0x7f836802a730 comp rx=0 tx=0).stop 2024-04-25T00:03:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 -- 172.21.15.113:0/998233137 shutdown_connections 2024-04-25T00:03:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 --2- 172.21.15.113:0/998233137 >> v2:172.21.15.178:3300/0 conn(0x7f83780d0b00 0x7f83780d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 --2- 172.21.15.113:0/998233137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d2290 0x7f83780b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 --2- 172.21.15.113:0/998233137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780d38a0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 -- 172.21.15.113:0/998233137 >> 172.21.15.113:0/998233137 conn(0x7f83780b5c20 msgr2=0x7f83780b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 -- 172.21.15.113:0/998233137 shutdown_connections 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.576+0000 7f837d476640 1 -- 172.21.15.113:0/998233137 wait complete. 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.577+0000 7f837d476640 1 Processor -- start 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.577+0000 7f837d476640 1 -- start start 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f837d476640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d0b00 0x7f83780cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f837d476640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83780d2290 0x7f83780cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f837d476640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f837d476640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83780d30a0 con 0x7f837815e320 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f837d476640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83780d2dc0 con 0x7f83780d2290 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f837d476640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83780b9fe0 con 0x7f83780d0b00 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39812/0 (socket says 172.21.15.113:39812) 2024-04-25T00:03:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 -- 172.21.15.113:0/1505954873 learned_addr learned my addr 172.21.15.113:0/1505954873 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 -- 172.21.15.113:0/1505954873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d0b00 msgr2=0x7f83780cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d0b00 0x7f83780cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 -- 172.21.15.113:0/1505954873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83780d2290 msgr2=0x7f83780cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83780d2290 0x7f83780cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.578+0000 7f83777fe640 1 -- 172.21.15.113:0/1505954873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8378161580 con 0x7f837815e320 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.579+0000 7f83777fe640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780cccb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8370009fd0 tx=0x7f837000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.579+0000 7f835f7fe640 1 -- 172.21.15.113:0/1505954873 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370011830 con 0x7f837815e320 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.579+0000 7f835f7fe640 1 -- 172.21.15.113:0/1505954873 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8370016070 con 0x7f837815e320 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.579+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8378156980 con 0x7f837815e320 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.579+0000 7f835f7fe640 1 -- 172.21.15.113:0/1505954873 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370011bf0 con 0x7f837815e320 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.579+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8378156f40 con 0x7f837815e320 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.580+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8378157140 con 0x7f837815e320 2024-04-25T00:03:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.580+0000 7f835f7fe640 1 -- 172.21.15.113:0/1505954873 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f837001b460 con 0x7f837815e320 2024-04-25T00:03:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.580+0000 7f835f7fe640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f836406be30 0x7f836406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.581+0000 7f835f7fe640 1 -- 172.21.15.113:0/1505954873 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f837008ab70 con 0x7f837815e320 2024-04-25T00:03:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.581+0000 7f8376ffd640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f836406be30 0x7f836406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.581+0000 7f8376ffd640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f836406be30 0x7f836406e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8368033000 tx=0x7f836802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.585+0000 7f835f7fe640 1 -- 172.21.15.113:0/1505954873 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8378157140 con 0x7f837815e320 2024-04-25T00:03:52.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:03:52.684 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:03:52.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.763+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83780691a0 con 0x7f837815e320 2024-04-25T00:03:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.764+0000 7f835f7fe640 1 -- 172.21.15.113:0/1505954873 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16879 (secure 0 0 0) 0x7f8370059550 con 0x7f837815e320 2024-04-25T00:03:52.766 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:52.766 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:03:29.825357+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:02:19.562437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:02:20.315341+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.766+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f836406be30 msgr2=0x7f836406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.766+0000 7f837d476640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f836406be30 0x7f836406e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8368033000 tx=0x7f836802e000 comp rx=0 tx=0).stop 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.766+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 msgr2=0x7f83780cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.766+0000 7f837d476640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780cccb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8370009fd0 tx=0x7f837000b8e0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.766+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 shutdown_connections 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.767+0000 7f837d476640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f836406be30 0x7f836406e2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.767+0000 7f837d476640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83780d2290 0x7f83780cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.767+0000 7f837d476640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815e320 0x7f83780cccb0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.767+0000 7f837d476640 1 --2- 172.21.15.113:0/1505954873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83780d0b00 0x7f83780cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.767+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 >> 172.21.15.113:0/1505954873 conn(0x7f83780b5c20 msgr2=0x7f83780b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.767+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 shutdown_connections 2024-04-25T00:03:52.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.767+0000 7f837d476640 1 -- 172.21.15.113:0/1505954873 wait complete. 2024-04-25T00:03:52.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:02:19.562437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:02:20.315341+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 79, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:03:52.779 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:03:52.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.917+0000 7fb3f2509640 1 Processor -- start 2024-04-25T00:03:52.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.917+0000 7fb3f2509640 1 -- start start 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3f2509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 0x7fb3ec0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3f2509640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec15f4f0 con 0x7fb3ec15e2d0 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3f2509640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec15f7d0 con 0x7fb3ec15e6b0 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3f2509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ec0cde00 con 0x7fb3ec0b3080 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3ebfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb3ec15e2d0 0x7fb3ec15ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38122/0 (socket says 172.21.15.113:38122) 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3ebfff640 1 -- 172.21.15.113:0/4194929240 learned_addr learned my addr 172.21.15.113:0/4194929240 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744378621 0 0) 0x7fb3ec15f4f0 con 0x7fb3ec15e2d0 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3d4003520 con 0x7fb3ec15e2d0 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.918+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 0x7fb3ec0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 511748604 0 0) 0x7fb3d4003520 con 0x7fb3ec15e2d0 2024-04-25T00:03:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 msgr2=0x7fb3ec0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 0x7fb3ec0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 >> v1:172.21.15.178:6789/0 conn(0x7fb3ec15e6b0 legacy=0x7fb3ec0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec15fab0 con 0x7fb3ec15e2d0 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fb3dc001670 con 0x7fb3ec15e2d0 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 >> v1:172.21.15.113:6789/0 conn(0x7fb3ec15e2d0 legacy=0x7fb3ec15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3d40039e0 0x7fb3d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 0x7fb3d400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec0cde00 con 0x7fb3d4004800 2024-04-25T00:03:52.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d4005520 con 0x7fb3d40039e0 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d400e110 con 0x7fb3ec0b3080 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3ebfff640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3d40039e0 0x7fb3d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3f0aa6640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 0x7fb3d400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eb7fe640 1 -- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 msgr2=0x7fb3d400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 0x7fb3d400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eb7fe640 1 -- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3d40039e0 msgr2=0x7fb3d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3d40039e0 0x7fb3d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3eb7fe640 1 -- 172.21.15.113:0/4194929240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d4003520 con 0x7fb3d4004800 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3ebfff640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3d40039e0 0x7fb3d4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.919+0000 7fb3f0aa6640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec0b3080 0x7fb3d400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3d4004c80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000ba00 tx=0x7fb3d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb54925bed5b3688 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0030070 con 0x7fb3d4004800 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e002adb0 con 0x7fb3d4004800 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3eaffd640 1 -- 172.21.15.113:0/4194929240 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000b600 con 0x7fb3d4004800 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3f2509640 1 -- 172.21.15.113:0/4194929240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 msgr2=0x7fb3d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3f2509640 1 --2- 172.21.15.113:0/4194929240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3d4004c80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000ba00 tx=0x7fb3d4005830 comp rx=0 tx=0).stop 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3f0aa6640 1 -- 172.21.15.113:0/4194929240 reap_dead start 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3f2509640 1 -- 172.21.15.113:0/4194929240 shutdown_connections 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3f2509640 1 -- 172.21.15.113:0/4194929240 >> 172.21.15.113:0/4194929240 conn(0x7fb3ec06b0f0 msgr2=0x7fb3ec06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3f2509640 1 -- 172.21.15.113:0/4194929240 shutdown_connections 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.920+0000 7fb3f2509640 1 -- 172.21.15.113:0/4194929240 wait complete. 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 Processor -- start 2024-04-25T00:03:52.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 -- start start 2024-04-25T00:03:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3ec14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec14da50 0x7fb3ec151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3ec153270 0x7fb3ec0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec0b44b0 con 0x7fb3d4004800 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec0b4330 con 0x7fb3ec153270 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.922+0000 7fb3f2509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ec0b4630 con 0x7fb3ec14da50 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3ec14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3ec14fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39836/0 (socket says 172.21.15.113:39836) 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 -- 172.21.15.113:0/1473879474 learned_addr learned my addr 172.21.15.113:0/1473879474 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3f0aa6640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3ec153270 0x7fb3ec0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec14da50 0x7fb3ec151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 -- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec14da50 msgr2=0x7fb3ec151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec14da50 0x7fb3ec151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 -- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3ec153270 msgr2=0x7fb3ec0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3ec153270 0x7fb3ec0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 -- 172.21.15.113:0/1473879474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec1505d0 con 0x7fb3d4004800 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec14da50 0x7fb3ec151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3f0aa6640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3ec153270 0x7fb3ec0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3ebfff640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3ec14fff0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e920 tx=0x7fb3dc00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3e8ff9640 1 -- 172.21.15.113:0/1473879474 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00c870 con 0x7fb3d4004800 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3e8ff9640 1 -- 172.21.15.113:0/1473879474 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc018070 con 0x7fb3d4004800 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ec152790 con 0x7fb3d4004800 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3e8ff9640 1 -- 172.21.15.113:0/1473879474 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00cc50 con 0x7fb3d4004800 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.923+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ec152d50 con 0x7fb3d4004800 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.924+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ec14de30 con 0x7fb3d4004800 2024-04-25T00:03:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.925+0000 7fb3e8ff9640 1 -- 172.21.15.113:0/1473879474 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3dc013420 con 0x7fb3d4004800 2024-04-25T00:03:52.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.925+0000 7fb3e8ff9640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb3d806be60 0x7fb3d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.925+0000 7fb3e8ff9640 1 -- 172.21.15.113:0/1473879474 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fb3dc08aa90 con 0x7fb3d4004800 2024-04-25T00:03:52.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.925+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb3d806be60 0x7fb3d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.925+0000 7fb3eb7fe640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb3d806be60 0x7fb3d806e320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0b34c0 tx=0x7fb3e0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:52.929+0000 7fb3e8ff9640 1 -- 172.21.15.113:0/1473879474 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3dc0594a0 con 0x7fb3d4004800 2024-04-25T00:03:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.100+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3ec072050 con 0x7fb3d4004800 2024-04-25T00:03:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.101+0000 7fb3e8ff9640 1 -- 172.21.15.113:0/1473879474 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fb3dc058da0 con 0x7fb3d4004800 2024-04-25T00:03:53.102 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:03:53.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb3d806be60 msgr2=0x7fb3d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb3d806be60 0x7fb3d806e320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0b34c0 tx=0x7fb3e0030000 comp rx=0 tx=0).stop 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 msgr2=0x7fb3ec14fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3ec14fff0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00e920 tx=0x7fb3dc00ede0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 shutdown_connections 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb3d806be60 0x7fb3d806e320 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3ec153270 0x7fb3ec0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3ec14da50 0x7fb3ec151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 --2- 172.21.15.113:0/1473879474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3d4004800 0x7fb3ec14fff0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 >> 172.21.15.113:0/1473879474 conn(0x7fb3ec06b0f0 msgr2=0x7fb3ec0d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:53.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 shutdown_connections 2024-04-25T00:03:53.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.103+0000 7fb3f2509640 1 -- 172.21.15.113:0/1473879474 wait complete. 2024-04-25T00:03:53.114 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:03:53.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.213+0000 7f0aae815640 1 Processor -- start 2024-04-25T00:03:53.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.213+0000 7f0aae815640 1 -- start start 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aae815640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0aa815e2d0 0x7f0aa815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aae815640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa80b3080 0x7f0aa80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aae815640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa815f4f0 con 0x7f0aa815e6b0 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aae815640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa80cde00 con 0x7f0aa815e2d0 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aae815640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa815f7d0 con 0x7f0aa80b3080 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aa7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0aa815e2d0 0x7f0aa815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aa77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa80b3080 0x7f0aa80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aa77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa80b3080 0x7f0aa80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:50388/0 (socket says 172.21.15.113:50388) 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aa77fe640 1 -- 172.21.15.113:0/1398692786 learned_addr learned my addr 172.21.15.113:0/1398692786 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:53.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aa7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0aa815e2d0 0x7f0aa815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:44778/0 (socket says 172.21.15.113:44778) 2024-04-25T00:03:53.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.214+0000 7f0aacdb2640 1 --1- 172.21.15.113:0/1398692786 >> v1:172.21.15.178:6789/0 conn(0x7f0aa815e6b0 0x7f0aa8071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:38476/0 (socket says 172.21.15.113:38476) 2024-04-25T00:03:53.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.215+0000 7f0aa7fff640 1 -- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa80b3080 msgr2=0x7f0aa80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:53.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.215+0000 7f0aa7fff640 1 --2- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa80b3080 0x7f0aa80b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.215+0000 7f0aa7fff640 1 -- 172.21.15.113:0/1398692786 >> v1:172.21.15.178:6789/0 conn(0x7f0aa815e6b0 legacy=0x7f0aa8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.215+0000 7f0aa7fff640 1 -- 172.21.15.113:0/1398692786 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa815fab0 con 0x7f0aa815e2d0 2024-04-25T00:03:53.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.215+0000 7f0aa6ffd640 1 -- 172.21.15.113:0/1398692786 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1132147758 0 0) 0x7f0aa815f4f0 con 0x7f0aa815e6b0 2024-04-25T00:03:53.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.215+0000 7f0aa77fe640 1 --2- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa80b3080 0x7f0aa80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:53.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.215+0000 7f0aa7fff640 1 --2- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa815ebf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0a980097a0 tx=0x7f0a9802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afd9ba613f54d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aa6ffd640 1 -- 172.21.15.113:0/1398692786 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98037070 con 0x7f0aa815e2d0 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aa6ffd640 1 -- 172.21.15.113:0/1398692786 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9802ae50 con 0x7f0aa815e2d0 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aa6ffd640 1 -- 172.21.15.113:0/1398692786 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98032660 con 0x7f0aa815e2d0 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 -- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 msgr2=0x7f0aa815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 --2- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa815ebf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0a980097a0 tx=0x7f0a9802a100 comp rx=0 tx=0).stop 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 -- 172.21.15.113:0/1398692786 shutdown_connections 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 --2- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa80b3080 0x7f0aa80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 --2- 172.21.15.113:0/1398692786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa815ebf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 -- 172.21.15.113:0/1398692786 >> 172.21.15.113:0/1398692786 conn(0x7f0aa806b0f0 msgr2=0x7f0aa806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 -- 172.21.15.113:0/1398692786 shutdown_connections 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.216+0000 7f0aae815640 1 -- 172.21.15.113:0/1398692786 wait complete. 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.217+0000 7f0aae815640 1 Processor -- start 2024-04-25T00:03:53.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aae815640 1 -- start start 2024-04-25T00:03:53.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aae815640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aa80b3080 0x7f0aa8154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aae815640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa813f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aae815640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa815e6b0 0x7f0aa813e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aae815640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa815f2b0 con 0x7f0aa80b3080 2024-04-25T00:03:53.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aae815640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa815f130 con 0x7f0aa815e2d0 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aae815640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa80cf2a0 con 0x7f0aa815e6b0 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aa7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aa80b3080 0x7f0aa8154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aa7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aa80b3080 0x7f0aa8154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39842/0 (socket says 172.21.15.113:39842) 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.218+0000 7f0aa7fff640 1 -- 172.21.15.113:0/1896912751 learned_addr learned my addr 172.21.15.113:0/1896912751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aacdb2640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa815e6b0 0x7f0aa813e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa77fe640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa813f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa7fff640 1 -- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa815e6b0 msgr2=0x7f0aa813e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa7fff640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa815e6b0 0x7f0aa813e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa7fff640 1 -- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 msgr2=0x7f0aa813f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa7fff640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa813f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa7fff640 1 -- 172.21.15.113:0/1896912751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa8140510 con 0x7f0aa80b3080 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa77fe640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa813f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aacdb2640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa815e6b0 0x7f0aa813e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa7fff640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aa80b3080 0x7f0aa8154f00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0a980097a0 tx=0x7f0a980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa4ff9640 1 -- 172.21.15.113:0/1896912751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98042070 con 0x7f0aa80b3080 2024-04-25T00:03:53.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa4ff9640 1 -- 172.21.15.113:0/1896912751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9803e460 con 0x7f0aa80b3080 2024-04-25T00:03:53.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa813fc70 con 0x7f0aa80b3080 2024-04-25T00:03:53.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aa4ff9640 1 -- 172.21.15.113:0/1896912751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98037040 con 0x7f0aa80b3080 2024-04-25T00:03:53.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.219+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa8140230 con 0x7f0aa80b3080 2024-04-25T00:03:53.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.220+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa80691a0 con 0x7f0aa80b3080 2024-04-25T00:03:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.221+0000 7f0aa4ff9640 1 -- 172.21.15.113:0/1896912751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a98045070 con 0x7f0aa80b3080 2024-04-25T00:03:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.221+0000 7f0aa4ff9640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a9406be30 0x7f0a9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.221+0000 7f0aa4ff9640 1 -- 172.21.15.113:0/1896912751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f0a980ab700 con 0x7f0aa80b3080 2024-04-25T00:03:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.221+0000 7f0aa77fe640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a9406be30 0x7f0a9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.222+0000 7f0aa77fe640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a9406be30 0x7f0a9406e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009630 tx=0x7f0a9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.225+0000 7f0aa4ff9640 1 -- 172.21.15.113:0/1896912751 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a9807a200 con 0x7f0aa80b3080 2024-04-25T00:03:53.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.397+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0aa8072050 con 0x7f0aa80b3080 2024-04-25T00:03:53.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.398+0000 7f0aa4ff9640 1 -- 172.21.15.113:0/1896912751 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f0a98079b00 con 0x7f0aa80b3080 2024-04-25T00:03:53.400 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:03:53.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.399+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a9406be30 msgr2=0x7f0a9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.399+0000 7f0aae815640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a9406be30 0x7f0a9406e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009630 tx=0x7f0a9c008040 comp rx=0 tx=0).stop 2024-04-25T00:03:53.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.399+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aa80b3080 msgr2=0x7f0aa8154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.399+0000 7f0aae815640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aa80b3080 0x7f0aa8154f00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0a980097a0 tx=0x7f0a980046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 shutdown_connections 2024-04-25T00:03:53.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a9406be30 0x7f0a9406e2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa815e2d0 0x7f0aa813f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aa815e6b0 0x7f0aa813e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 --2- 172.21.15.113:0/1896912751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aa80b3080 0x7f0aa8154f00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 >> 172.21.15.113:0/1896912751 conn(0x7f0aa806b0f0 msgr2=0x7f0aa80afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:53.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 shutdown_connections 2024-04-25T00:03:53.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.400+0000 7f0aae815640 1 -- 172.21.15.113:0/1896912751 wait complete. 2024-04-25T00:03:53.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:03:53.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.510+0000 7f9262f77640 1 Processor -- start 2024-04-25T00:03:53.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.510+0000 7f9262f77640 1 -- start start 2024-04-25T00:03:53.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f9262f77640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f925c1606a0 0x7f925c167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f9262f77640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c15d6b0 con 0x7f925c0d06c0 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f9262f77640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c15c8f0 con 0x7f925c13f050 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f9262f77640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f925c15c6a0 con 0x7f925c1606a0 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f925bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f925c0d06c0 0x7f925c15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38136/0 (socket says 172.21.15.113:38136) 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f925bfff640 1 -- 172.21.15.113:0/3373956985 learned_addr learned my addr 172.21.15.113:0/3373956985 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f9261514640 1 --2- 172.21.15.113:0/3373956985 >> v2:172.21.15.178:3300/0 conn(0x7f925c1606a0 0x7f925c167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f925b7fe640 1 -- 172.21.15.113:0/3373956985 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897963261 0 0) 0x7f925c15d6b0 con 0x7f925c0d06c0 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.511+0000 7f925b7fe640 1 -- 172.21.15.113:0/3373956985 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9244003520 con 0x7f925c0d06c0 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.512+0000 7f925b7fe640 1 -- 172.21.15.113:0/3373956985 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 673976630 0 0) 0x7f9244003520 con 0x7f925c0d06c0 2024-04-25T00:03:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.512+0000 7f9261514640 1 -- 172.21.15.113:0/3373956985 >> v1:172.21.15.178:6789/0 conn(0x7f925c13f050 legacy=0x7f925c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.512+0000 7f9261514640 1 -- 172.21.15.113:0/3373956985 >> v1:172.21.15.113:6789/0 conn(0x7f925c0d06c0 legacy=0x7f925c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.512+0000 7f9261514640 1 -- 172.21.15.113:0/3373956985 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c15cbd0 con 0x7f925c1606a0 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.512+0000 7f9261514640 1 --2- 172.21.15.113:0/3373956985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c1606a0 0x7f925c167080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f925402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e885779382ded99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f925b7fe640 1 -- 172.21.15.113:0/3373956985 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250001670 con 0x7f925c1606a0 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f925b7fe640 1 -- 172.21.15.113:0/3373956985 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9254007540 con 0x7f925c1606a0 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f925b7fe640 1 -- 172.21.15.113:0/3373956985 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254007910 con 0x7f925c1606a0 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f9262f77640 1 -- 172.21.15.113:0/3373956985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c1606a0 msgr2=0x7f925c167080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f9262f77640 1 --2- 172.21.15.113:0/3373956985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c1606a0 0x7f925c167080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f925402a5f0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f9262f77640 1 -- 172.21.15.113:0/3373956985 shutdown_connections 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f9262f77640 1 --2- 172.21.15.113:0/3373956985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c1606a0 0x7f925c167080 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f9262f77640 1 -- 172.21.15.113:0/3373956985 >> 172.21.15.113:0/3373956985 conn(0x7f925c06b0f0 msgr2=0x7f925c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f9262f77640 1 -- 172.21.15.113:0/3373956985 shutdown_connections 2024-04-25T00:03:53.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.513+0000 7f9262f77640 1 -- 172.21.15.113:0/3373956985 wait complete. 2024-04-25T00:03:53.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.514+0000 7f9262f77640 1 Processor -- start 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.514+0000 7f9262f77640 1 -- start start 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9262f77640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f925c0d06c0 0x7f925c154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9262f77640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c13f050 0x7f925c13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9262f77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f925c1606a0 0x7f925c072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9262f77640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f925c13f430 con 0x7f925c1606a0 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9262f77640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f925c15d190 con 0x7f925c13f050 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9262f77640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0cfa70 con 0x7f925c0d06c0 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9261514640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f925c1606a0 0x7f925c072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9261514640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f925c1606a0 0x7f925c072a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39852/0 (socket says 172.21.15.113:39852) 2024-04-25T00:03:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9261514640 1 -- 172.21.15.113:0/835431403 learned_addr learned my addr 172.21.15.113:0/835431403 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f925bfff640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c13f050 0x7f925c13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.515+0000 7f9260d13640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f925c0d06c0 0x7f925c154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9261514640 1 -- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f925c0d06c0 msgr2=0x7f925c154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9261514640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f925c0d06c0 0x7f925c154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9261514640 1 -- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c13f050 msgr2=0x7f925c13b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9261514640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c13f050 0x7f925c13b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9261514640 1 -- 172.21.15.113:0/835431403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c1537f0 con 0x7f925c1606a0 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f925bfff640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c13f050 0x7f925c13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9260d13640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f925c0d06c0 0x7f925c154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9261514640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f925c1606a0 0x7f925c072a10 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f9254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f92597fa640 1 -- 172.21.15.113:0/835431403 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254040070 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f92597fa640 1 -- 172.21.15.113:0/835431403 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9254031bc0 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f92597fa640 1 -- 172.21.15.113:0/835431403 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254009450 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c153ad0 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.516+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c154090 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.517+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c0691a0 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.517+0000 7f92597fa640 1 -- 172.21.15.113:0/835431403 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9254039040 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.518+0000 7f92597fa640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f924806be30 0x7f924806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.518+0000 7f92597fa640 1 -- 172.21.15.113:0/835431403 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f92540ac380 con 0x7f925c1606a0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.518+0000 7f9260d13640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f924806be30 0x7f924806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.518+0000 7f9260d13640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f924806be30 0x7f924806e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f924c007fd0 tx=0x7f924c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.522+0000 7f92597fa640 1 -- 172.21.15.113:0/835431403 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f92540303a0 con 0x7f925c1606a0 2024-04-25T00:03:53.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.696+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f925c156a50 con 0x7f925c1606a0 2024-04-25T00:03:53.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.696+0000 7f92597fa640 1 -- 172.21.15.113:0/835431403 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f925407acd0 con 0x7f925c1606a0 2024-04-25T00:03:53.697 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f924806be30 msgr2=0x7f924806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f924806be30 0x7f924806e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f924c007fd0 tx=0x7f924c009040 comp rx=0 tx=0).stop 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f925c1606a0 msgr2=0x7f925c072a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f925c1606a0 0x7f925c072a10 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f9254004a40 comp rx=0 tx=0).stop 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 shutdown_connections 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f924806be30 0x7f924806e2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f925c13f050 0x7f925c13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f925c1606a0 0x7f925c072a10 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 --2- 172.21.15.113:0/835431403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f925c0d06c0 0x7f925c154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.698+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 >> 172.21.15.113:0/835431403 conn(0x7f925c06b0f0 msgr2=0x7f925c06d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.699+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 shutdown_connections 2024-04-25T00:03:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.699+0000 7f9262f77640 1 -- 172.21.15.113:0/835431403 wait complete. 2024-04-25T00:03:53.709 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:03:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.805+0000 7f74cff9e640 1 Processor -- start 2024-04-25T00:03:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.805+0000 7f74cff9e640 1 -- start start 2024-04-25T00:03:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cff9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f74c806e6e0 0x7f74c80b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cff9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c81453c0 0x7f74c8071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cff9e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c80cfad0 con 0x7f74c8144d60 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cff9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f74c8145c40 con 0x7f74c806e6e0 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cff9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f74c80cf230 con 0x7f74c81453c0 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cd539640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f74c806e6e0 0x7f74c80b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cd539640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f74c806e6e0 0x7f74c80b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39858/0 (socket says 172.21.15.113:39858) 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cd539640 1 -- 172.21.15.113:0/3969222938 learned_addr learned my addr 172.21.15.113:0/3969222938 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74ce53b640 1 --2- 172.21.15.113:0/3969222938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c81453c0 0x7f74c8071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cd539640 1 -- 172.21.15.113:0/3969222938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c81453c0 msgr2=0x7f74c8071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cd539640 1 --2- 172.21.15.113:0/3969222938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c81453c0 0x7f74c8071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cd539640 1 -- 172.21.15.113:0/3969222938 >> v1:172.21.15.178:6789/0 conn(0x7f74c8144d60 legacy=0x7f74c806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.806+0000 7f74cd539640 1 -- 172.21.15.113:0/3969222938 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c80cf510 con 0x7f74c806e6e0 2024-04-25T00:03:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.807+0000 7f74cd539640 1 --2- 172.21.15.113:0/3969222938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80b5390 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009870 tx=0x7f74b802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce02ecf70c30bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.807+0000 7f74ccd38640 1 -- 172.21.15.113:0/3969222938 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8037070 con 0x7f74c806e6e0 2024-04-25T00:03:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.807+0000 7f74ccd38640 1 -- 172.21.15.113:0/3969222938 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b802ae10 con 0x7f74c806e6e0 2024-04-25T00:03:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.807+0000 7f74ccd38640 1 -- 172.21.15.113:0/3969222938 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8032660 con 0x7f74c806e6e0 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.807+0000 7f74cff9e640 1 -- 172.21.15.113:0/3969222938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 msgr2=0x7f74c80b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.807+0000 7f74cff9e640 1 --2- 172.21.15.113:0/3969222938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80b5390 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009870 tx=0x7f74b802a100 comp rx=0 tx=0).stop 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.808+0000 7f74cff9e640 1 -- 172.21.15.113:0/3969222938 shutdown_connections 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.808+0000 7f74cff9e640 1 --2- 172.21.15.113:0/3969222938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c81453c0 0x7f74c8071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.808+0000 7f74cff9e640 1 --2- 172.21.15.113:0/3969222938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80b5390 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.808+0000 7f74cff9e640 1 -- 172.21.15.113:0/3969222938 >> 172.21.15.113:0/3969222938 conn(0x7f74c80b1a50 msgr2=0x7f74c80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.808+0000 7f74cff9e640 1 -- 172.21.15.113:0/3969222938 shutdown_connections 2024-04-25T00:03:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.808+0000 7f74cff9e640 1 -- 172.21.15.113:0/3969222938 wait complete. 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 Processor -- start 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 -- start start 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c8144d60 0x7f74c815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c81453c0 0x7f74c815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8145140 con 0x7f74c806e6e0 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8070e10 con 0x7f74c81453c0 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cff9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f74c8071870 con 0x7f74c8144d60 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.809+0000 7f74cdd3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39874/0 (socket says 172.21.15.113:39874) 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 -- 172.21.15.113:0/4051364954 learned_addr learned my addr 172.21.15.113:0/4051364954 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cd539640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c8144d60 0x7f74c815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74ce53b640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c81453c0 0x7f74c815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 -- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c8144d60 msgr2=0x7f74c815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c8144d60 0x7f74c815d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 -- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c81453c0 msgr2=0x7f74c815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c81453c0 0x7f74c815d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 -- 172.21.15.113:0/4051364954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c814c280 con 0x7f74c806e6e0 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cd539640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c8144d60 0x7f74c815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74ce53b640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c81453c0 0x7f74c815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cdd3a640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80aea90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c460 tx=0x7f74c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74be7fc640 1 -- 172.21.15.113:0/4051364954 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4016020 con 0x7f74c806e6e0 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74be7fc640 1 -- 172.21.15.113:0/4051364954 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4012870 con 0x7f74c806e6e0 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74be7fc640 1 -- 172.21.15.113:0/4051364954 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4012c60 con 0x7f74c806e6e0 2024-04-25T00:03:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c814c560 con 0x7f74c806e6e0 2024-04-25T00:03:53.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.810+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c814cb20 con 0x7f74c806e6e0 2024-04-25T00:03:53.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.811+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c80691a0 con 0x7f74c806e6e0 2024-04-25T00:03:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.812+0000 7f74be7fc640 1 -- 172.21.15.113:0/4051364954 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f74c4019020 con 0x7f74c806e6e0 2024-04-25T00:03:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.812+0000 7f74be7fc640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f74ac06be30 0x7f74ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.812+0000 7f74cd539640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f74ac06be30 0x7f74ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.812+0000 7f74be7fc640 1 -- 172.21.15.113:0/4051364954 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f74c408ab30 con 0x7f74c806e6e0 2024-04-25T00:03:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.813+0000 7f74cd539640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f74ac06be30 0x7f74ac06e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f74b8034000 tx=0x7f74b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:53.816+0000 7f74be7fc640 1 -- 172.21.15.113:0/4051364954 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f74c4020030 con 0x7f74c806e6e0 2024-04-25T00:03:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.027+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74c80afac0 con 0x7f74c806e6e0 2024-04-25T00:03:54.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.027+0000 7f74be7fc640 1 -- 172.21.15.113:0/4051364954 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5192 (secure 0 0 0) 0x7f74c4059510 con 0x7f74c806e6e0 2024-04-25T00:03:54.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:54.028 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4243948985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4243948985},{"type":"v1","addr":"172.21.15.178:6835","nonce":4243948985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/665506344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":665506344},{"type":"v1","addr":"172.21.15.178:6834","nonce":665506344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4064308986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4064308986},{"type":"v1","addr":"172.21.15.113:6836","nonce":4064308986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T00:02:21.350179+0000","modified":"2024-04-25T00:02:32.368592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3530277358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3530277358},{"type":"v1","addr":"172.21.15.113:6837","nonce":3530277358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6951,"qdb_cluster":[6951]},"id":5}]} 2024-04-25T00:03:54.028 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f74ac06be30 msgr2=0x7f74ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f74ac06be30 0x7f74ac06e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f74b8034000 tx=0x7f74b802e000 comp rx=0 tx=0).stop 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 msgr2=0x7f74c80aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80aea90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c460 tx=0x7f74c400c920 comp rx=0 tx=0).stop 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 shutdown_connections 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f74ac06be30 0x7f74ac06e2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c81453c0 0x7f74c815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c8144d60 0x7f74c815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 --2- 172.21.15.113:0/4051364954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c806e6e0 0x7f74c80aea90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 >> 172.21.15.113:0/4051364954 conn(0x7f74c80b1a50 msgr2=0x7f74c8071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 shutdown_connections 2024-04-25T00:03:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.029+0000 7f74cff9e640 1 -- 172.21.15.113:0/4051364954 wait complete. 2024-04-25T00:03:54.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:03:54.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.137+0000 7f503b3b6640 1 Processor -- start 2024-04-25T00:03:54.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.137+0000 7f503b3b6640 1 -- start start 2024-04-25T00:03:54.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.137+0000 7f503b3b6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f50340b3fd0 0x7f503406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f503b3b6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f503406e510 0x7f50340cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f503b3b6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5034144cf0 con 0x7f5034144910 2024-04-25T00:03:54.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f503b3b6640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f50340b2f30 con 0x7f50340b3fd0 2024-04-25T00:03:54.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f503b3b6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f50340b3210 con 0x7f503406e510 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5038951640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f50340b3fd0 0x7f503406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5039953640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f503406e510 0x7f50340cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5039152640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5034144910 0x7f50340b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:38492/0 (socket says 172.21.15.113:38492) 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5039152640 1 -- 172.21.15.113:0/344214796 learned_addr learned my addr 172.21.15.113:0/344214796 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5039953640 1 -- 172.21.15.113:0/344214796 >> v2:172.21.15.178:3300/0 conn(0x7f50340b3fd0 msgr2=0x7f503406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5039953640 1 --2- 172.21.15.113:0/344214796 >> v2:172.21.15.178:3300/0 conn(0x7f50340b3fd0 0x7f503406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5039953640 1 -- 172.21.15.113:0/344214796 >> v1:172.21.15.178:6789/0 conn(0x7f5034144910 legacy=0x7f50340b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5039953640 1 -- 172.21.15.113:0/344214796 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50340b34f0 con 0x7f503406e510 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.138+0000 7f5038951640 1 --2- 172.21.15.113:0/344214796 >> v2:172.21.15.178:3300/0 conn(0x7f50340b3fd0 0x7f503406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.139+0000 7f5039953640 1 --2- 172.21.15.113:0/344214796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f503406e510 0x7f50340cd500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50240097d0 tx=0x7f502402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67d3d4456908bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.139+0000 7f5033fff640 1 -- 172.21.15.113:0/344214796 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024037070 con 0x7f503406e510 2024-04-25T00:03:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.139+0000 7f5033fff640 1 -- 172.21.15.113:0/344214796 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5024032440 con 0x7f503406e510 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.139+0000 7f5033fff640 1 -- 172.21.15.113:0/344214796 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024032810 con 0x7f503406e510 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 -- 172.21.15.113:0/344214796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f503406e510 msgr2=0x7f50340cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 --2- 172.21.15.113:0/344214796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f503406e510 0x7f50340cd500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50240097d0 tx=0x7f502402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 -- 172.21.15.113:0/344214796 shutdown_connections 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 --2- 172.21.15.113:0/344214796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f503406e510 0x7f50340cd500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 --2- 172.21.15.113:0/344214796 >> v2:172.21.15.178:3300/0 conn(0x7f50340b3fd0 0x7f503406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 -- 172.21.15.113:0/344214796 >> 172.21.15.113:0/344214796 conn(0x7f50340b1a50 msgr2=0x7f50340b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 -- 172.21.15.113:0/344214796 shutdown_connections 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.140+0000 7f503b3b6640 1 -- 172.21.15.113:0/344214796 wait complete. 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.141+0000 7f503b3b6640 1 Processor -- start 2024-04-25T00:03:54.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.141+0000 7f503b3b6640 1 -- start start 2024-04-25T00:03:54.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f503b3b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f503406e510 0x7f5034159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f503b3b6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50340b3fd0 0x7f503413eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f503b3b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5034144910 0x7f503413f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f503b3b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50340b4da0 con 0x7f5034144910 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f503b3b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50340b4440 con 0x7f503406e510 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f503b3b6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f50340b4f20 con 0x7f50340b3fd0 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5034144910 0x7f503413f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5034144910 0x7f503413f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39878/0 (socket says 172.21.15.113:39878) 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 -- 172.21.15.113:0/1077602579 learned_addr learned my addr 172.21.15.113:0/1077602579 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039152640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f503406e510 0x7f5034159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5038951640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50340b3fd0 0x7f503413eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 -- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50340b3fd0 msgr2=0x7f503413eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50340b3fd0 0x7f503413eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 -- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f503406e510 msgr2=0x7f5034159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f503406e510 0x7f5034159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 -- 172.21.15.113:0/1077602579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503413dbf0 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5039953640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5034144910 0x7f503413f3e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f50240098f0 tx=0x7f50240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5031ffb640 1 -- 172.21.15.113:0/1077602579 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024041030 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5031ffb640 1 -- 172.21.15.113:0/1077602579 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502403d420 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f5031ffb640 1 -- 172.21.15.113:0/1077602579 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024037040 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.142+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503415a750 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.143+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503415aff0 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.144+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50340691a0 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.144+0000 7f5031ffb640 1 -- 172.21.15.113:0/1077602579 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5024044070 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.145+0000 7f5031ffb640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f502006be30 0x7f502006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.145+0000 7f5039152640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f502006be30 0x7f502006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.145+0000 7f5031ffb640 1 -- 172.21.15.113:0/1077602579 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f50240ab380 con 0x7f5034144910 2024-04-25T00:03:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.145+0000 7f5039152640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f502006be30 0x7f502006e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f502c0012e0 tx=0x7f502c001a30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.149+0000 7f5031ffb640 1 -- 172.21.15.113:0/1077602579 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50240797b0 con 0x7f5034144910 2024-04-25T00:03:54.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.322+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50340b54e0 con 0x7f5034144910 2024-04-25T00:03:54.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.323+0000 7f5031ffb640 1 -- 172.21.15.113:0/1077602579 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16879 (secure 0 0 0) 0x7f502407dab0 con 0x7f5034144910 2024-04-25T00:03:54.323 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:54.323 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:03:29.825357+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:02:19.562437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:02:20.315341+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f502006be30 msgr2=0x7f502006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f502006be30 0x7f502006e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f502c0012e0 tx=0x7f502c001a30 comp rx=0 tx=0).stop 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5034144910 msgr2=0x7f503413f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5034144910 0x7f503413f3e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f50240098f0 tx=0x7f50240046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 shutdown_connections 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f502006be30 0x7f502006e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5034144910 0x7f503413f3e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50340b3fd0 0x7f503413eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 --2- 172.21.15.113:0/1077602579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f503406e510 0x7f5034159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 >> 172.21.15.113:0/1077602579 conn(0x7f50340b1a50 msgr2=0x7f50340af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:54.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 shutdown_connections 2024-04-25T00:03:54.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.325+0000 7f503b3b6640 1 -- 172.21.15.113:0/1077602579 wait complete. 2024-04-25T00:03:54.337 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:03:54.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:03:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.433+0000 7f2388c72640 1 Processor -- start 2024-04-25T00:03:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.433+0000 7f2388c72640 1 -- start start 2024-04-25T00:03:54.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2388c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2388c72640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23840d2290 0x7f23840b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2388c72640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23840d3de0 con 0x7f23840d0b00 2024-04-25T00:03:54.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2388c72640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f23840d3410 con 0x7f23840d2290 2024-04-25T00:03:54.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2388c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f23840d2800 con 0x7f238415e320 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2381d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23840d2290 0x7f23840b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2381d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23840d2290 0x7f23840b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39886/0 (socket says 172.21.15.113:39886) 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.434+0000 7f2381d9b640 1 -- 172.21.15.113:0/3036671008 learned_addr learned my addr 172.21.15.113:0/3036671008 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.435+0000 7f238259c640 1 --2- 172.21.15.113:0/3036671008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.435+0000 7f2381d9b640 1 -- 172.21.15.113:0/3036671008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 msgr2=0x7f23840d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.435+0000 7f2381d9b640 1 --2- 172.21.15.113:0/3036671008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.435+0000 7f2381d9b640 1 -- 172.21.15.113:0/3036671008 >> v1:172.21.15.178:6789/0 conn(0x7f23840d0b00 legacy=0x7f23840b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.435+0000 7f2381d9b640 1 -- 172.21.15.113:0/3036671008 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23840d2ae0 con 0x7f23840d2290 2024-04-25T00:03:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.435+0000 7f2381d9b640 1 --2- 172.21.15.113:0/3036671008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d2290 0x7f23840b9aa0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2378009870 tx=0x7f237802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4dfdad155cd72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.435+0000 7f238159a640 1 -- 172.21.15.113:0/3036671008 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378037070 con 0x7f23840d2290 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f238159a640 1 -- 172.21.15.113:0/3036671008 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237802ae50 con 0x7f23840d2290 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f238159a640 1 -- 172.21.15.113:0/3036671008 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378032860 con 0x7f23840d2290 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 -- 172.21.15.113:0/3036671008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d2290 msgr2=0x7f23840b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 --2- 172.21.15.113:0/3036671008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d2290 0x7f23840b9aa0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2378009870 tx=0x7f237802a300 comp rx=0 tx=0).stop 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 -- 172.21.15.113:0/3036671008 shutdown_connections 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 --2- 172.21.15.113:0/3036671008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d2290 0x7f23840b9aa0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 --2- 172.21.15.113:0/3036671008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 -- 172.21.15.113:0/3036671008 >> 172.21.15.113:0/3036671008 conn(0x7f23840b5c20 msgr2=0x7f23840b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 -- 172.21.15.113:0/3036671008 shutdown_connections 2024-04-25T00:03:54.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.436+0000 7f2388c72640 1 -- 172.21.15.113:0/3036671008 wait complete. 2024-04-25T00:03:54.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.437+0000 7f2388c72640 1 Processor -- start 2024-04-25T00:03:54.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.437+0000 7f2388c72640 1 -- start start 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2388c72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d0b00 0x7f23840c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2388c72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23840d2290 0x7f2384152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2388c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2388c72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23840b9fe0 con 0x7f23840d0b00 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2388c72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23840d30a0 con 0x7f23840d2290 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2388c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f238415e700 con 0x7f238415e320 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f238259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d0b00 0x7f23840c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f238259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d0b00 0x7f23840c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39900/0 (socket says 172.21.15.113:39900) 2024-04-25T00:03:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f238259c640 1 -- 172.21.15.113:0/141898182 learned_addr learned my addr 172.21.15.113:0/141898182 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2381d9b640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23840d2290 0x7f2384152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.438+0000 7f2382d9d640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f238259c640 1 -- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 msgr2=0x7f23840c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f238259c640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f238259c640 1 -- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23840d2290 msgr2=0x7f2384152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f238259c640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23840d2290 0x7f2384152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f238259c640 1 -- 172.21.15.113:0/141898182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238415fa50 con 0x7f23840d0b00 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f238259c640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d0b00 0x7f23840c83c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2374007e60 tx=0x7f237400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f2372ffd640 1 -- 172.21.15.113:0/141898182 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374014620 con 0x7f23840d0b00 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f2372ffd640 1 -- 172.21.15.113:0/141898182 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2374019070 con 0x7f23840d0b00 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238415fd30 con 0x7f23840d0b00 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f2372ffd640 1 -- 172.21.15.113:0/141898182 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374014a20 con 0x7f23840d0b00 2024-04-25T00:03:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.439+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23841602f0 con 0x7f23840d0b00 2024-04-25T00:03:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.440+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23840cf410 con 0x7f23840d0b00 2024-04-25T00:03:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.440+0000 7f2372ffd640 1 -- 172.21.15.113:0/141898182 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2374014bc0 con 0x7f23840d0b00 2024-04-25T00:03:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.441+0000 7f2372ffd640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406be30 0x7f236406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.441+0000 7f2381d9b640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406be30 0x7f236406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.441+0000 7f2372ffd640 1 -- 172.21.15.113:0/141898182 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f237408b760 con 0x7f23840d0b00 2024-04-25T00:03:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.441+0000 7f2381d9b640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406be30 0x7f236406e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2378034000 tx=0x7f237802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.445+0000 7f2372ffd640 1 -- 172.21.15.113:0/141898182 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23840cf410 con 0x7f23840d0b00 2024-04-25T00:03:54.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.657+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23840b4cc0 con 0x7f23840d0b00 2024-04-25T00:03:54.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.657+0000 7f2372ffd640 1 -- 172.21.15.113:0/141898182 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5192 (secure 0 0 0) 0x7f237405a0b0 con 0x7f23840d0b00 2024-04-25T00:03:54.658 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:54.658 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4243948985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4243948985},{"type":"v1","addr":"172.21.15.178:6835","nonce":4243948985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/665506344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":665506344},{"type":"v1","addr":"172.21.15.178:6834","nonce":665506344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4064308986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4064308986},{"type":"v1","addr":"172.21.15.113:6836","nonce":4064308986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T00:02:21.350179+0000","modified":"2024-04-25T00:02:32.368592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3530277358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3530277358},{"type":"v1","addr":"172.21.15.113:6837","nonce":3530277358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6951,"qdb_cluster":[6951]},"id":5}]} 2024-04-25T00:03:54.658 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-04-25T00:03:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406be30 msgr2=0x7f236406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406be30 0x7f236406e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2378034000 tx=0x7f237802e000 comp rx=0 tx=0).stop 2024-04-25T00:03:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d0b00 msgr2=0x7f23840c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d0b00 0x7f23840c83c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2374007e60 tx=0x7f237400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 shutdown_connections 2024-04-25T00:03:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f236406be30 0x7f236406e2f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23840d2290 0x7f2384152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f238415e320 0x7f23840c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 --2- 172.21.15.113:0/141898182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23840d0b00 0x7f23840c83c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 >> 172.21.15.113:0/141898182 conn(0x7f23840b5c20 msgr2=0x7f23840b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:54.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 shutdown_connections 2024-04-25T00:03:54.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.659+0000 7f2388c72640 1 -- 172.21.15.113:0/141898182 wait complete. 2024-04-25T00:03:54.671 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:03:54.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.766+0000 7fc77629e640 1 Processor -- start 2024-04-25T00:03:54.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.766+0000 7fc77629e640 1 -- start start 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77629e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc770072ea0 0x7fc770071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77629e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7700b3af0 0x7fc7700b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77629e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7700b2d50 con 0x7fc77015a6b0 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77629e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc770072a70 con 0x7fc770072ea0 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77629e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc7700b3010 con 0x7fc7700b3af0 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77483b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7700b3af0 0x7fc7700b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc76f7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc770072ea0 0x7fc770071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77483b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7700b3af0 0x7fc7700b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:50456/0 (socket says 172.21.15.113:50456) 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc77483b640 1 -- 172.21.15.113:0/2447181841 learned_addr learned my addr 172.21.15.113:0/2447181841 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:54.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.767+0000 7fc76ffff640 1 --1- 172.21.15.113:0/2447181841 >> v1:172.21.15.178:6789/0 conn(0x7fc77015a6b0 0x7fc770070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:38510/0 (socket says 172.21.15.113:38510) 2024-04-25T00:03:54.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.768+0000 7fc77483b640 1 -- 172.21.15.113:0/2447181841 >> v2:172.21.15.178:3300/0 conn(0x7fc770072ea0 msgr2=0x7fc770071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:54.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.768+0000 7fc77483b640 1 --2- 172.21.15.113:0/2447181841 >> v2:172.21.15.178:3300/0 conn(0x7fc770072ea0 0x7fc770071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.768+0000 7fc77483b640 1 -- 172.21.15.113:0/2447181841 >> v1:172.21.15.178:6789/0 conn(0x7fc77015a6b0 legacy=0x7fc770070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.768+0000 7fc77483b640 1 -- 172.21.15.113:0/2447181841 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7700b32f0 con 0x7fc7700b3af0 2024-04-25T00:03:54.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.768+0000 7fc77483b640 1 --2- 172.21.15.113:0/2447181841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7700b3af0 0x7fc7700b3f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc768009e50 tx=0x7fc76802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b0ffa759feacdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc76effd640 1 -- 172.21.15.113:0/2447181841 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768036070 con 0x7fc7700b3af0 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc76effd640 1 -- 172.21.15.113:0/2447181841 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc768031600 con 0x7fc7700b3af0 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc76effd640 1 -- 172.21.15.113:0/2447181841 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7680319d0 con 0x7fc7700b3af0 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 -- 172.21.15.113:0/2447181841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7700b3af0 msgr2=0x7fc7700b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 --2- 172.21.15.113:0/2447181841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7700b3af0 0x7fc7700b3f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc768009e50 tx=0x7fc76802a710 comp rx=0 tx=0).stop 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 -- 172.21.15.113:0/2447181841 shutdown_connections 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 --2- 172.21.15.113:0/2447181841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7700b3af0 0x7fc7700b3f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 --2- 172.21.15.113:0/2447181841 >> v2:172.21.15.178:3300/0 conn(0x7fc770072ea0 0x7fc770071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 -- 172.21.15.113:0/2447181841 >> 172.21.15.113:0/2447181841 conn(0x7fc7700b1a50 msgr2=0x7fc7700b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 -- 172.21.15.113:0/2447181841 shutdown_connections 2024-04-25T00:03:54.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.769+0000 7fc77629e640 1 -- 172.21.15.113:0/2447181841 wait complete. 2024-04-25T00:03:54.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.770+0000 7fc77629e640 1 Processor -- start 2024-04-25T00:03:54.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc77629e640 1 -- start start 2024-04-25T00:03:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc77629e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc770072ea0 0x7fc7700cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc77629e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7700b3af0 0x7fc770070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc77629e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc77015a6b0 0x7fc770071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc77629e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc77015aa90 con 0x7fc770072ea0 2024-04-25T00:03:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc77629e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc770072720 con 0x7fc7700b3af0 2024-04-25T00:03:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc77629e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc77015ac10 con 0x7fc77015a6b0 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc76ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc770072ea0 0x7fc7700cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc76ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc770072ea0 0x7fc7700cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39912/0 (socket says 172.21.15.113:39912) 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc76ffff640 1 -- 172.21.15.113:0/4139122123 learned_addr learned my addr 172.21.15.113:0/4139122123 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.771+0000 7fc76f7fe640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7700b3af0 0x7fc770070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc77483b640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc77015a6b0 0x7fc770071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76ffff640 1 -- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc77015a6b0 msgr2=0x7fc770071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76ffff640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc77015a6b0 0x7fc770071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76ffff640 1 -- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7700b3af0 msgr2=0x7fc770070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76ffff640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7700b3af0 0x7fc770070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76ffff640 1 -- 172.21.15.113:0/4139122123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7700cdf50 con 0x7fc770072ea0 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76ffff640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc770072ea0 0x7fc7700cd6b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c3c0 tx=0x7fc76000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76cff9640 1 -- 172.21.15.113:0/4139122123 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7600074e0 con 0x7fc770072ea0 2024-04-25T00:03:54.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76cff9640 1 -- 172.21.15.113:0/4139122123 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc760007680 con 0x7fc770072ea0 2024-04-25T00:03:54.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7700ce230 con 0x7fc770072ea0 2024-04-25T00:03:54.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc76cff9640 1 -- 172.21.15.113:0/4139122123 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760004a00 con 0x7fc770072ea0 2024-04-25T00:03:54.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.772+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7700ce7f0 con 0x7fc770072ea0 2024-04-25T00:03:54.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.773+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7700691a0 con 0x7fc770072ea0 2024-04-25T00:03:54.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.773+0000 7fc76cff9640 1 -- 172.21.15.113:0/4139122123 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc76001a020 con 0x7fc770072ea0 2024-04-25T00:03:54.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.774+0000 7fc76cff9640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc75c06be30 0x7fc75c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:54.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.774+0000 7fc76cff9640 1 -- 172.21.15.113:0/4139122123 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fc76008a4f0 con 0x7fc770072ea0 2024-04-25T00:03:54.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.774+0000 7fc76f7fe640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc75c06be30 0x7fc75c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:54.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.774+0000 7fc76f7fe640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc75c06be30 0x7fc75c06e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc764009b40 tx=0x7fc764009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:54.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.778+0000 7fc76cff9640 1 -- 172.21.15.113:0/4139122123 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc7600587f0 con 0x7fc770072ea0 2024-04-25T00:03:54.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.988+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7701572e0 con 0x7fc770072ea0 2024-04-25T00:03:54.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.988+0000 7fc76cff9640 1 -- 172.21.15.113:0/4139122123 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5192 (secure 0 0 0) 0x7fc76005caf0 con 0x7fc770072ea0 2024-04-25T00:03:54.989 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:54.989 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4243948985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4243948985},{"type":"v1","addr":"172.21.15.178:6835","nonce":4243948985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/665506344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":665506344},{"type":"v1","addr":"172.21.15.178:6834","nonce":665506344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4064308986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4064308986},{"type":"v1","addr":"172.21.15.113:6836","nonce":4064308986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T00:02:21.350179+0000","modified":"2024-04-25T00:02:32.368592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6951},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6951":{"gid":6951,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/3530277358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3530277358},{"type":"v1","addr":"172.21.15.113:6837","nonce":3530277358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6951,"qdb_cluster":[6951]},"id":5}]} 2024-04-25T00:03:54.989 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-04-25T00:03:54.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc75c06be30 msgr2=0x7fc75c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc75c06be30 0x7fc75c06e2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc764009b40 tx=0x7fc764009190 comp rx=0 tx=0).stop 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc770072ea0 msgr2=0x7fc7700cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc770072ea0 0x7fc7700cd6b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c3c0 tx=0x7fc76000c880 comp rx=0 tx=0).stop 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 shutdown_connections 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc75c06be30 0x7fc75c06e2f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7700b3af0 0x7fc770070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc77015a6b0 0x7fc770071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 --2- 172.21.15.113:0/4139122123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc770072ea0 0x7fc7700cd6b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 >> 172.21.15.113:0/4139122123 conn(0x7fc7700b1a50 msgr2=0x7fc7700b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 shutdown_connections 2024-04-25T00:03:54.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:54.990+0000 7fc77629e640 1 -- 172.21.15.113:0/4139122123 wait complete. 2024-04-25T00:03:55.001 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:03:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.098+0000 7fefeef29640 1 Processor -- start 2024-04-25T00:03:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.098+0000 7fefeef29640 1 -- start start 2024-04-25T00:03:55.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.098+0000 7fefeef29640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fefe8072ea0 0x7fefe8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.098+0000 7fefeef29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe80b3af0 0x7fefe80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.098+0000 7fefeef29640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe80b2d50 con 0x7fefe815a6b0 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.098+0000 7fefeef29640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fefe8072a70 con 0x7fefe8072ea0 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.098+0000 7fefeef29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fefe80b3010 con 0x7fefe80b3af0 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefe7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fefe8072ea0 0x7fefe8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefed4c6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe80b3af0 0x7fefe80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefe7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fefe8072ea0 0x7fefe8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:44850/0 (socket says 172.21.15.113:44850) 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefe7fff640 1 -- 172.21.15.113:0/3711589830 learned_addr learned my addr 172.21.15.113:0/3711589830 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefeccc5640 1 --1- 172.21.15.113:0/3711589830 >> v1:172.21.15.178:6789/0 conn(0x7fefe815a6b0 0x7fefe8070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:38516/0 (socket says 172.21.15.113:38516) 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefed4c6640 1 -- 172.21.15.113:0/3711589830 >> v2:172.21.15.178:3300/0 conn(0x7fefe8072ea0 msgr2=0x7fefe8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefed4c6640 1 --2- 172.21.15.113:0/3711589830 >> v2:172.21.15.178:3300/0 conn(0x7fefe8072ea0 0x7fefe8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefed4c6640 1 -- 172.21.15.113:0/3711589830 >> v1:172.21.15.178:6789/0 conn(0x7fefe815a6b0 legacy=0x7fefe8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefed4c6640 1 -- 172.21.15.113:0/3711589830 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe80b32f0 con 0x7fefe80b3af0 2024-04-25T00:03:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.099+0000 7fefe7fff640 1 --2- 172.21.15.113:0/3711589830 >> v2:172.21.15.178:3300/0 conn(0x7fefe8072ea0 0x7fefe8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:55.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.100+0000 7fefed4c6640 1 --2- 172.21.15.113:0/3711589830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe80b3af0 0x7fefe80b3f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefd800a940 tx=0x7fefd802b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ca1daf5ba0cbf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.100+0000 7fefe77fe640 1 -- 172.21.15.113:0/3711589830 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd800b040 con 0x7fefe80b3af0 2024-04-25T00:03:55.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.100+0000 7fefe77fe640 1 -- 172.21.15.113:0/3711589830 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd8030710 con 0x7fefe80b3af0 2024-04-25T00:03:55.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.100+0000 7fefe77fe640 1 -- 172.21.15.113:0/3711589830 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8030ae0 con 0x7fefe80b3af0 2024-04-25T00:03:55.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.100+0000 7fefeef29640 1 -- 172.21.15.113:0/3711589830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe80b3af0 msgr2=0x7fefe80b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.100+0000 7fefeef29640 1 --2- 172.21.15.113:0/3711589830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe80b3af0 0x7fefe80b3f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefd800a940 tx=0x7fefd802b550 comp rx=0 tx=0).stop 2024-04-25T00:03:55.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.101+0000 7fefeef29640 1 -- 172.21.15.113:0/3711589830 shutdown_connections 2024-04-25T00:03:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.101+0000 7fefeef29640 1 --2- 172.21.15.113:0/3711589830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe80b3af0 0x7fefe80b3f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.101+0000 7fefeef29640 1 --2- 172.21.15.113:0/3711589830 >> v2:172.21.15.178:3300/0 conn(0x7fefe8072ea0 0x7fefe8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.101+0000 7fefeef29640 1 -- 172.21.15.113:0/3711589830 >> 172.21.15.113:0/3711589830 conn(0x7fefe80b1a50 msgr2=0x7fefe80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.101+0000 7fefeef29640 1 -- 172.21.15.113:0/3711589830 shutdown_connections 2024-04-25T00:03:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.101+0000 7fefeef29640 1 -- 172.21.15.113:0/3711589830 wait complete. 2024-04-25T00:03:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.102+0000 7fefeef29640 1 Processor -- start 2024-04-25T00:03:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.102+0000 7fefeef29640 1 -- start start 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe8072ea0 0x7fefe814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80b3af0 0x7fefe8153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe815a6b0 0x7fefe8153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fefe815aa90 con 0x7fefe80b3af0 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8072720 con 0x7fefe815a6b0 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fefe815ac10 con 0x7fefe8072ea0 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80b3af0 0x7fefe8153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80b3af0 0x7fefe8153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39920/0 (socket says 172.21.15.113:39920) 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 -- 172.21.15.113:0/2857307138 learned_addr learned my addr 172.21.15.113:0/2857307138 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeccc5640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe8072ea0 0x7fefe814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefed4c6640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe815a6b0 0x7fefe8153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 -- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe8072ea0 msgr2=0x7fefe814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe8072ea0 0x7fefe814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 -- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe815a6b0 msgr2=0x7fefe8153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe815a6b0 0x7fefe8153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 -- 172.21.15.113:0/2857307138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe814c9b0 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe7fff640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80b3af0 0x7fefe8153660 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e9d0 tx=0x7fefd000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe57fa640 1 -- 172.21.15.113:0/2857307138 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0017070 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe57fa640 1 -- 172.21.15.113:0/2857307138 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd000ca60 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe81512d0 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefe57fa640 1 -- 172.21.15.113:0/2857307138 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd000ce10 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.103+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe8151890 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.104+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe80691a0 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.105+0000 7fefe57fa640 1 -- 172.21.15.113:0/2857307138 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefd001a070 con 0x7fefe80b3af0 2024-04-25T00:03:55.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.105+0000 7fefe57fa640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fefc806be30 0x7fefc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.105+0000 7fefeccc5640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fefc806be30 0x7fefc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.105+0000 7fefe57fa640 1 -- 172.21.15.113:0/2857307138 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fefd008acd0 con 0x7fefe80b3af0 2024-04-25T00:03:55.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.106+0000 7fefeccc5640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fefc806be30 0x7fefc806e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefe0007fd0 tx=0x7fefe0001630 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.110+0000 7fefe57fa640 1 -- 172.21.15.113:0/2857307138 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fefd0059060 con 0x7fefe80b3af0 2024-04-25T00:03:55.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.286+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefe80b2900 con 0x7fefe80b3af0 2024-04-25T00:03:55.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.287+0000 7fefe57fa640 1 -- 172.21.15.113:0/2857307138 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16879 (secure 0 0 0) 0x7fefd005d360 con 0x7fefe80b3af0 2024-04-25T00:03:55.288 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:03:55.288 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:03:29.825357+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:02:19.562437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T00:02:20.315341+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:03:55.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fefc806be30 msgr2=0x7fefc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fefc806be30 0x7fefc806e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefe0007fd0 tx=0x7fefe0001630 comp rx=0 tx=0).stop 2024-04-25T00:03:55.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80b3af0 msgr2=0x7fefe8153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80b3af0 0x7fefe8153660 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e9d0 tx=0x7fefd000ee90 comp rx=0 tx=0).stop 2024-04-25T00:03:55.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 shutdown_connections 2024-04-25T00:03:55.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fefc806be30 0x7fefc806e2f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefe815a6b0 0x7fefe8153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80b3af0 0x7fefe8153660 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 --2- 172.21.15.113:0/2857307138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefe8072ea0 0x7fefe814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 >> 172.21.15.113:0/2857307138 conn(0x7fefe80b1a50 msgr2=0x7fefe814f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:55.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 shutdown_connections 2024-04-25T00:03:55.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.290+0000 7fefeef29640 1 -- 172.21.15.113:0/2857307138 wait complete. 2024-04-25T00:03:55.301 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:03:55.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.398+0000 7f68bdd29640 1 Processor -- start 2024-04-25T00:03:55.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.398+0000 7f68bdd29640 1 -- start start 2024-04-25T00:03:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.398+0000 7f68bdd29640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f68b815e510 0x7f68b80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.398+0000 7f68bdd29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 0x7f68b80b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.398+0000 7f68bdd29640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b80d2e00 con 0x7f68b80d28c0 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.398+0000 7f68bdd29640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f68b80d30e0 con 0x7f68b815e510 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.398+0000 7f68bdd29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f68b80d33c0 con 0x7f68b80b63b0 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b6ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f68b80d28c0 0x7f68b80b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38138/0 (socket says 172.21.15.113:38138) 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b6ffd640 1 -- 172.21.15.113:0/3082930725 learned_addr learned my addr 172.21.15.113:0/3082930725 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b77fe640 1 --2- 172.21.15.113:0/3082930725 >> v2:172.21.15.178:3300/0 conn(0x7f68b815e510 0x7f68b80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116463722 0 0) 0x7f68b80d2e00 con 0x7f68b80d28c0 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b7fff640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 0x7f68b80b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68a0003520 con 0x7f68b80d28c0 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3143342285 0 0) 0x7f68a0003520 con 0x7f68b80d28c0 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 msgr2=0x7f68b80b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 0x7f68b80b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 >> v2:172.21.15.178:3300/0 conn(0x7f68b815e510 msgr2=0x7f68b80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 --2- 172.21.15.113:0/3082930725 >> v2:172.21.15.178:3300/0 conn(0x7f68b815e510 0x7f68b80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b80d36a0 con 0x7f68b80d28c0 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f68ac001670 con 0x7f68b80d28c0 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 >> v1:172.21.15.113:6789/0 conn(0x7f68b80d28c0 legacy=0x7f68b80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 0x7f68a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a0004d40 0x7f68a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80d30e0 con 0x7f68a00039b0 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80d33c0 con 0x7f68a0004d40 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f68a000b9f0 con 0x7f68b80b63b0 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b77fe640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b7fff640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a0004d40 0x7f68a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b6ffd640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 0x7f68a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b77fe640 1 -- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 msgr2=0x7f68a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.399+0000 7f68b77fe640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 0x7f68a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b77fe640 1 -- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a0004d40 msgr2=0x7f68a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b77fe640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a0004d40 0x7f68a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b77fe640 1 -- 172.21.15.113:0/3082930725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a0003520 con 0x7f68a00039b0 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b7fff640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a0004d40 0x7f68a00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b6ffd640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b80b63b0 0x7f68a0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b77fe640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68a0003dd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ac0 tx=0x7f68a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a4bb89c597e2c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8036070 con 0x7f68a00039b0 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a8031400 con 0x7f68a00039b0 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.400+0000 7f68b67fc640 1 -- 172.21.15.113:0/3082930725 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8009690 con 0x7f68a00039b0 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.401+0000 7f68bdd29640 1 -- 172.21.15.113:0/3082930725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 msgr2=0x7f68a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.401+0000 7f68bdd29640 1 --2- 172.21.15.113:0/3082930725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68a0003dd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ac0 tx=0x7f68a0005830 comp rx=0 tx=0).stop 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.401+0000 7f68b7fff640 1 -- 172.21.15.113:0/3082930725 reap_dead start 2024-04-25T00:03:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.401+0000 7f68bdd29640 1 -- 172.21.15.113:0/3082930725 shutdown_connections 2024-04-25T00:03:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.401+0000 7f68bdd29640 1 -- 172.21.15.113:0/3082930725 >> 172.21.15.113:0/3082930725 conn(0x7f68b806b720 msgr2=0x7f68b80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.401+0000 7f68bdd29640 1 -- 172.21.15.113:0/3082930725 shutdown_connections 2024-04-25T00:03:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.401+0000 7f68bdd29640 1 -- 172.21.15.113:0/3082930725 wait complete. 2024-04-25T00:03:55.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.402+0000 7f68bdd29640 1 Processor -- start 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.402+0000 7f68bdd29640 1 -- start start 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68bdd29640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68b80c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68bdd29640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b8155ae0 0x7f68b8156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68bdd29640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b8156650 0x7f68b8157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68bdd29640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80b9dd0 con 0x7f68a00039b0 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68bdd29640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80d3ea0 con 0x7f68b8155ae0 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68bdd29640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f68b80b73c0 con 0x7f68b8156650 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68b80c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68b80c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39930/0 (socket says 172.21.15.113:39930) 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 -- 172.21.15.113:0/1029407298 learned_addr learned my addr 172.21.15.113:0/1029407298 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b7fff640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b8156650 0x7f68b8157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b6ffd640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b8155ae0 0x7f68b8156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 -- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b8156650 msgr2=0x7f68b8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b8156650 0x7f68b8157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 -- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b8155ae0 msgr2=0x7f68b8156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b8155ae0 0x7f68b8156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 -- 172.21.15.113:0/1029407298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b8158a50 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.403+0000 7f68b77fe640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68b80c9e90 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009be0 tx=0x7f68a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.404+0000 7f689ffff640 1 -- 172.21.15.113:0/1029407298 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8040020 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.404+0000 7f689ffff640 1 -- 172.21.15.113:0/1029407298 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a8003950 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.404+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b8158d30 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.404+0000 7f689ffff640 1 -- 172.21.15.113:0/1029407298 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8036040 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.404+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81592f0 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.405+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b80691a0 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.405+0000 7f689ffff640 1 -- 172.21.15.113:0/1029407298 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68a8043020 con 0x7f68a00039b0 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.405+0000 7f689ffff640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68a406be60 0x7f68a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.405+0000 7f689ffff640 1 -- 172.21.15.113:0/1029407298 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f68a80aafa0 con 0x7f68a00039b0 2024-04-25T00:03:55.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.406+0000 7f68b6ffd640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68a406be60 0x7f68a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.406+0000 7f68b6ffd640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68a406be60 0x7f68a406e320 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f68b80cb3a0 tx=0x7f68ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.410+0000 7f689ffff640 1 -- 172.21.15.113:0/1029407298 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f68a804a070 con 0x7f68a00039b0 2024-04-25T00:03:55.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.616+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f68b80b3e20 con 0x7f68a00039b0 2024-04-25T00:03:55.617 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:03:55.617+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:03:55.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.618+0000 7f689ffff640 1 -- 172.21.15.113:0/1029407298 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7f68a80798f0 con 0x7f68a00039b0 2024-04-25T00:03:55.618 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:03:55.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68a406be60 msgr2=0x7f68a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68a406be60 0x7f68a406e320 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f68b80cb3a0 tx=0x7f68ac009040 comp rx=0 tx=0).stop 2024-04-25T00:03:55.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 msgr2=0x7f68b80c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68b80c9e90 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009be0 tx=0x7f68a8004730 comp rx=0 tx=0).stop 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 shutdown_connections 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68a406be60 0x7f68a406e320 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68b8155ae0 0x7f68b8156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68b8156650 0x7f68b8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 --2- 172.21.15.113:0/1029407298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68a00039b0 0x7f68b80c9e90 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 >> 172.21.15.113:0/1029407298 conn(0x7f68b806b720 msgr2=0x7f68b815d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 shutdown_connections 2024-04-25T00:03:55.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.622+0000 7f68bdd29640 1 -- 172.21.15.113:0/1029407298 wait complete. 2024-04-25T00:03:55.633 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:03:55.660 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:03:55.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.734+0000 7f20b51d5640 1 Processor -- start 2024-04-25T00:03:55.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.734+0000 7f20b51d5640 1 -- start start 2024-04-25T00:03:55.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.734+0000 7f20b51d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e2d0 0x7f20b015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.734+0000 7f20b51d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20b00b3080 0x7f20b00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.734+0000 7f20b51d5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f20b015e6b0 0x7f20b00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.734+0000 7f20b51d5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f20b00cde00 con 0x7f20b00b3080 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20b51d5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f20b015f4f0 con 0x7f20b015e6b0 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20b51d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f20b015f7d0 con 0x7f20b015e2d0 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20b00b3080 0x7f20b00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20b00b3080 0x7f20b00b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39944/0 (socket says 172.21.15.113:39944) 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 -- 172.21.15.113:0/680921315 learned_addr learned my addr 172.21.15.113:0/680921315 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20af59e640 1 --2- 172.21.15.113:0/680921315 >> v2:172.21.15.178:3300/0 conn(0x7f20b015e6b0 0x7f20b00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20aed9d640 1 --2- 172.21.15.113:0/680921315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e2d0 0x7f20b015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 -- 172.21.15.113:0/680921315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e2d0 msgr2=0x7f20b015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 --2- 172.21.15.113:0/680921315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e2d0 0x7f20b015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 -- 172.21.15.113:0/680921315 >> v2:172.21.15.178:3300/0 conn(0x7f20b015e6b0 msgr2=0x7f20b00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 --2- 172.21.15.113:0/680921315 >> v2:172.21.15.178:3300/0 conn(0x7f20b015e6b0 0x7f20b00b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.735+0000 7f20a7fff640 1 -- 172.21.15.113:0/680921315 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b015fab0 con 0x7f20b00b3080 2024-04-25T00:03:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20a7fff640 1 --2- 172.21.15.113:0/680921315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b00b3080 0x7f20b00b4330 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f20b006f500 tx=0x7f209802c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221df3ed307b6fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20ae59c640 1 -- 172.21.15.113:0/680921315 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098032700 con 0x7f20b00b3080 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20ae59c640 1 -- 172.21.15.113:0/680921315 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20980328a0 con 0x7f20b00b3080 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20ae59c640 1 -- 172.21.15.113:0/680921315 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098032c70 con 0x7f20b00b3080 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 -- 172.21.15.113:0/680921315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b00b3080 msgr2=0x7f20b00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 --2- 172.21.15.113:0/680921315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b00b3080 0x7f20b00b4330 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f20b006f500 tx=0x7f209802c800 comp rx=0 tx=0).stop 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 -- 172.21.15.113:0/680921315 shutdown_connections 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 --2- 172.21.15.113:0/680921315 >> v2:172.21.15.178:3300/0 conn(0x7f20b015e6b0 0x7f20b00b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 --2- 172.21.15.113:0/680921315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b00b3080 0x7f20b00b4330 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 --2- 172.21.15.113:0/680921315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e2d0 0x7f20b015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 -- 172.21.15.113:0/680921315 >> 172.21.15.113:0/680921315 conn(0x7f20b006b0f0 msgr2=0x7f20b006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 -- 172.21.15.113:0/680921315 shutdown_connections 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.736+0000 7f20b51d5640 1 -- 172.21.15.113:0/680921315 wait complete. 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 Processor -- start 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 -- start start 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b00b3080 0x7f20b00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b015e2d0 0x7f20b013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e6b0 0x7f20b013cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20b015f2b0 con 0x7f20b015e2d0 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20b015f130 con 0x7f20b00b3080 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20b51d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f20b0071040 con 0x7f20b015e6b0 2024-04-25T00:03:55.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.738+0000 7f20a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b015e2d0 0x7f20b013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b015e2d0 0x7f20b013c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39950/0 (socket says 172.21.15.113:39950) 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 -- 172.21.15.113:0/212235959 learned_addr learned my addr 172.21.15.113:0/212235959 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20aed9d640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b00b3080 0x7f20b00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20af59e640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e6b0 0x7f20b013cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 -- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e6b0 msgr2=0x7f20b013cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e6b0 0x7f20b013cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 -- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b00b3080 msgr2=0x7f20b00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b00b3080 0x7f20b00b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 -- 172.21.15.113:0/212235959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b0150f10 con 0x7f20b015e2d0 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20af59e640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e6b0 0x7f20b013cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a7fff640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b015e2d0 0x7f20b013c620 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f209802c840 tx=0x7f20980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a77fe640 1 -- 172.21.15.113:0/212235959 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098042070 con 0x7f20b015e2d0 2024-04-25T00:03:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a77fe640 1 -- 172.21.15.113:0/212235959 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2098034bd0 con 0x7f20b015e2d0 2024-04-25T00:03:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b01511f0 con 0x7f20b015e2d0 2024-04-25T00:03:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20a77fe640 1 -- 172.21.15.113:0/212235959 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098047460 con 0x7f20b015e2d0 2024-04-25T00:03:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.739+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b015a080 con 0x7f20b015e2d0 2024-04-25T00:03:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.740+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b00cf360 con 0x7f20b015e2d0 2024-04-25T00:03:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.741+0000 7f20a77fe640 1 -- 172.21.15.113:0/212235959 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f209803e410 con 0x7f20b015e2d0 2024-04-25T00:03:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.741+0000 7f20a77fe640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f209006be30 0x7f209006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.741+0000 7f20a77fe640 1 -- 172.21.15.113:0/212235959 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5853+0+0 (secure 0 0 0) 0x7f20980ae4c0 con 0x7f20b015e2d0 2024-04-25T00:03:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.741+0000 7f20aed9d640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f209006be30 0x7f209006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.742+0000 7f20aed9d640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f209006be30 0x7f209006e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006fd0 tx=0x7f20a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:55.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.745+0000 7f20a77fe640 1 -- 172.21.15.113:0/212235959 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20b00cf360 con 0x7f20b015e2d0 2024-04-25T00:03:55.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:55.956+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f20b015a280 con 0x7f20b015e2d0 2024-04-25T00:03:56.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.620+0000 7f20a77fe640 1 -- 172.21.15.113:0/212235959 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7f20b015a280 con 0x7f20b015e2d0 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f209006be30 msgr2=0x7f209006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f209006be30 0x7f209006e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006fd0 tx=0x7f20a0008040 comp rx=0 tx=0).stop 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b015e2d0 msgr2=0x7f20b013c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b015e2d0 0x7f20b013c620 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f209802c840 tx=0x7f20980046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 shutdown_connections 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f209006be30 0x7f209006e2f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20b015e6b0 0x7f20b013cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20b015e2d0 0x7f20b013c620 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 --2- 172.21.15.113:0/212235959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20b00b3080 0x7f20b00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 >> 172.21.15.113:0/212235959 conn(0x7f20b006b0f0 msgr2=0x7f20b006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 shutdown_connections 2024-04-25T00:03:56.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.622+0000 7f20b51d5640 1 -- 172.21.15.113:0/212235959 wait complete. 2024-04-25T00:03:56.636 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:03:56.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.780+0000 7f1fe017e640 1 Processor -- start 2024-04-25T00:03:56.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.780+0000 7f1fe017e640 1 -- start start 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fe017e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8144a80 0x7f1fd80b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fe017e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd8072f70 con 0x7f1fd80b3ff0 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fe017e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd80b2fd0 con 0x7f1fd806e7e0 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fe017e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd80b32b0 con 0x7f1fd8144a80 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdd719640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1fd80b3ff0 0x7f1fd806e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38150/0 (socket says 172.21.15.113:38150) 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdd719640 1 -- 172.21.15.113:0/99802716 learned_addr learned my addr 172.21.15.113:0/99802716 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fddf1a640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8144a80 0x7f1fd80b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1044512213 0 0) 0x7f1fd8072f70 con 0x7f1fd80b3ff0 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fc4003520 con 0x7f1fd80b3ff0 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681736241 0 0) 0x7f1fd80b2fd0 con 0x7f1fd806e7e0 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fd8072f70 con 0x7f1fd806e7e0 2024-04-25T00:03:56.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3192243606 0 0) 0x7f1fc4003520 con 0x7f1fd80b3ff0 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8144a80 msgr2=0x7f1fd80b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8144a80 0x7f1fd80b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 >> v1:172.21.15.178:6789/0 conn(0x7f1fd806e7e0 legacy=0x7f1fd80cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd80b3590 con 0x7f1fd80b3ff0 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1fc8001670 con 0x7f1fd80b3ff0 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 >> v1:172.21.15.113:6789/0 conn(0x7f1fd80b3ff0 legacy=0x7f1fd806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fc4003b70 0x7f1fc4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fc4004800 0x7f1fc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.781+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd80b32b0 con 0x7f1fc40051c0 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc400c090 con 0x7f1fc4003b70 2024-04-25T00:03:56.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc400c290 con 0x7f1fc4004800 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fddf1a640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8144a80 0x7f1fd80b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fde71b640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fdd719640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fc4004800 0x7f1fc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fddf1a640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fc4003b70 0x7f1fc4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fde71b640 1 -- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fc4004800 msgr2=0x7f1fc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fde71b640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fc4004800 0x7f1fc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fde71b640 1 -- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fc4003b70 msgr2=0x7f1fc4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fde71b640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fc4003b70 0x7f1fc4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fde71b640 1 -- 172.21.15.113:0/99802716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc4003520 con 0x7f1fc40051c0 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.782+0000 7f1fddf1a640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fc4003b70 0x7f1fc4003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fde71b640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fc400b9c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0001f90 tx=0x7f1fc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5b159d986686a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0001be0 con 0x7f1fc40051c0 2024-04-25T00:03:56.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd0003b90 con 0x7f1fc40051c0 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fdcf18640 1 -- 172.21.15.113:0/99802716 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00050d0 con 0x7f1fc40051c0 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fe017e640 1 -- 172.21.15.113:0/99802716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 msgr2=0x7f1fc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fe017e640 1 --2- 172.21.15.113:0/99802716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fc400b9c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0001f90 tx=0x7f1fc4005830 comp rx=0 tx=0).stop 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fde71b640 1 -- 172.21.15.113:0/99802716 reap_dead start 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fe017e640 1 -- 172.21.15.113:0/99802716 shutdown_connections 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fe017e640 1 -- 172.21.15.113:0/99802716 >> 172.21.15.113:0/99802716 conn(0x7f1fd80b1b30 msgr2=0x7f1fd80b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fe017e640 1 -- 172.21.15.113:0/99802716 shutdown_connections 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.783+0000 7f1fe017e640 1 -- 172.21.15.113:0/99802716 wait complete. 2024-04-25T00:03:56.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.784+0000 7f1fe017e640 1 Processor -- start 2024-04-25T00:03:56.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fe017e640 1 -- start start 2024-04-25T00:03:56.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fe017e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fd815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fe017e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd815ac40 0x7f1fd8142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fe017e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8143080 0x7f1fd8140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fe017e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd80b3ab0 con 0x7f1fc40051c0 2024-04-25T00:03:56.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fe017e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd80cdf90 con 0x7f1fd815ac40 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fe017e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd80b3c30 con 0x7f1fd8143080 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fddf1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fd815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fddf1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fd815daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39958/0 (socket says 172.21.15.113:39958) 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.785+0000 7f1fddf1a640 1 -- 172.21.15.113:0/3866855460 learned_addr learned my addr 172.21.15.113:0/3866855460 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fdd719640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd815ac40 0x7f1fd8142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fddf1a640 1 -- 172.21.15.113:0/3866855460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8143080 msgr2=0x7f1fd8140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fddf1a640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8143080 0x7f1fd8140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fddf1a640 1 -- 172.21.15.113:0/3866855460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd815ac40 msgr2=0x7f1fd8142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fddf1a640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd815ac40 0x7f1fd8142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fddf1a640 1 -- 172.21.15.113:0/3866855460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd8143540 con 0x7f1fc40051c0 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fddf1a640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fd815daf0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4011830 tx=0x7f1fd4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fce7fc640 1 -- 172.21.15.113:0/3866855460 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd400f960 con 0x7f1fc40051c0 2024-04-25T00:03:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fce7fc640 1 -- 172.21.15.113:0/3866855460 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd400fb00 con 0x7f1fc40051c0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd8143820 con 0x7f1fc40051c0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fce7fc640 1 -- 172.21.15.113:0/3866855460 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4015460 con 0x7f1fc40051c0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.786+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd8140ed0 con 0x7f1fc40051c0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.787+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd80691a0 con 0x7f1fc40051c0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.787+0000 7f1fce7fc640 1 -- 172.21.15.113:0/3866855460 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fd4015810 con 0x7f1fc40051c0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.788+0000 7f1fce7fc640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1fbc06be30 0x7f1fbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.788+0000 7f1fdd719640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1fbc06be30 0x7f1fbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.788+0000 7f1fce7fc640 1 -- 172.21.15.113:0/3866855460 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5853+0+0 (secure 0 0 0) 0x7f1fd408bbb0 con 0x7f1fc40051c0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.788+0000 7f1fdd719640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1fbc06be30 0x7f1fbc06e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80b5de0 tx=0x7f1fc8000f50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.792+0000 7f1fce7fc640 1 -- 172.21.15.113:0/3866855460 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fd4019030 con 0x7f1fc40051c0 2024-04-25T00:03:56.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:56.966+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1fd80b29a0 con 0x7f1fc40051c0 2024-04-25T00:03:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.879+0000 7f1fce7fc640 1 -- 172.21.15.113:0/3866855460 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f1fd405a4d0 con 0x7f1fc40051c0 2024-04-25T00:03:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:03:57.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1fbc06be30 msgr2=0x7f1fbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:57.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1fbc06be30 0x7f1fbc06e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80b5de0 tx=0x7f1fc8000f50 comp rx=0 tx=0).stop 2024-04-25T00:03:57.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 msgr2=0x7f1fd815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:57.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fd815daf0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4011830 tx=0x7f1fd4011cf0 comp rx=0 tx=0).stop 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 shutdown_connections 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1fbc06be30 0x7f1fbc06e2f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd815ac40 0x7f1fd8142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8143080 0x7f1fd8140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 --2- 172.21.15.113:0/3866855460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc40051c0 0x7f1fd815daf0 secure :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4011830 tx=0x7f1fd4011cf0 comp rx=0 tx=0).stop 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 >> 172.21.15.113:0/3866855460 conn(0x7f1fd80b1b30 msgr2=0x7f1fd80ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.884+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 shutdown_connections 2024-04-25T00:03:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:57.885+0000 7f1fe017e640 1 -- 172.21.15.113:0/3866855460 wait complete. 2024-04-25T00:03:57.902 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:03:58.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.009+0000 7f5200161640 1 Processor -- start 2024-04-25T00:03:58.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.009+0000 7f5200161640 1 -- start start 2024-04-25T00:03:58.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f5200161640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 0x7f51f806e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f5200161640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f8072f70 con 0x7f51f8144a80 2024-04-25T00:03:58.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f5200161640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f80b2fd0 con 0x7f51f80cd630 2024-04-25T00:03:58.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f5200161640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51f806ea70 con 0x7f51f806e0d0 2024-04-25T00:03:58.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fdefd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f51f8144a80 0x7f51f80b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56426/0 (socket says 172.21.15.113:56426) 2024-04-25T00:03:58.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fdefd640 1 -- 172.21.15.113:0/1953712649 learned_addr learned my addr 172.21.15.113:0/1953712649 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:58.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fd6fc640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 0x7f51f806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223085315 0 0) 0x7f51f8072f70 con 0x7f51f8144a80 2024-04-25T00:03:58.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e4003420 con 0x7f51f8144a80 2024-04-25T00:03:58.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2328457836 0 0) 0x7f51e4003420 con 0x7f51f8144a80 2024-04-25T00:03:58.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 msgr2=0x7f51f806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 0x7f51f806e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 >> v1:172.21.15.178:6789/0 conn(0x7f51f80cd630 legacy=0x7f51f8072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f80b32b0 con 0x7f51f8144a80 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.010+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f51f4001670 con 0x7f51f8144a80 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 >> v1:172.21.15.113:6789/0 conn(0x7f51f8144a80 legacy=0x7f51f80b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fcefb640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e4004ac0 0x7f51e4004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fcefb640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e40036a0 0x7f51e4003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fcefb640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 0x7f51e400fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51e40044c0 con 0x7f51e40036a0 2024-04-25T00:03:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51e40100b0 con 0x7f51e4004ac0 2024-04-25T00:03:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51e4010280 con 0x7f51f806e0d0 2024-04-25T00:03:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fd6fc640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e40036a0 0x7f51e4003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fdefd640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e4004ac0 0x7f51e4004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fe6fe640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 0x7f51e400fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fd6fc640 1 -- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 msgr2=0x7f51e400fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fd6fc640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f806e0d0 0x7f51e400fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fd6fc640 1 -- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e4004ac0 msgr2=0x7f51e4004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fd6fc640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e4004ac0 0x7f51e4004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fd6fc640 1 -- 172.21.15.113:0/1953712649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e4003420 con 0x7f51e40036a0 2024-04-25T00:03:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.011+0000 7f51fdefd640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e4004ac0 0x7f51e4004ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.012+0000 7f51fd6fc640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e40036a0 0x7f51e4003b20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009af0 tx=0x7f51e4004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7044cf1929df63a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.012+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8036070 con 0x7f51e40036a0 2024-04-25T00:03:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.012+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e8031400 con 0x7f51e40036a0 2024-04-25T00:03:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.012+0000 7f51fcefb640 1 -- 172.21.15.113:0/1953712649 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8009690 con 0x7f51e40036a0 2024-04-25T00:03:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.013+0000 7f5200161640 1 -- 172.21.15.113:0/1953712649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e40036a0 msgr2=0x7f51e4003b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.013+0000 7f5200161640 1 --2- 172.21.15.113:0/1953712649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e40036a0 0x7f51e4003b20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009af0 tx=0x7f51e4004720 comp rx=0 tx=0).stop 2024-04-25T00:03:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.013+0000 7f51fe6fe640 1 -- 172.21.15.113:0/1953712649 reap_dead start 2024-04-25T00:03:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.013+0000 7f5200161640 1 -- 172.21.15.113:0/1953712649 shutdown_connections 2024-04-25T00:03:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.013+0000 7f5200161640 1 -- 172.21.15.113:0/1953712649 >> 172.21.15.113:0/1953712649 conn(0x7f51f80b1b30 msgr2=0x7f51f80b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.013+0000 7f5200161640 1 -- 172.21.15.113:0/1953712649 shutdown_connections 2024-04-25T00:03:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.013+0000 7f5200161640 1 -- 172.21.15.113:0/1953712649 wait complete. 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.014+0000 7f5200161640 1 Processor -- start 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.014+0000 7f5200161640 1 -- start start 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f5200161640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e40036a0 0x7f51f81e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f5200161640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f815deb0 0x7f51f8156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f5200161640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f81e45a0 0x7f51f81e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f5200161640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51f80b4340 con 0x7f51f81e45a0 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f5200161640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51f80b41c0 con 0x7f51e40036a0 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f5200161640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51f806e530 con 0x7f51f815deb0 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f51fe6fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f81e45a0 0x7f51f81e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f51fe6fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f81e45a0 0x7f51f81e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50462/0 (socket says 172.21.15.113:50462) 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f51fe6fe640 1 -- 172.21.15.113:0/2245585753 learned_addr learned my addr 172.21.15.113:0/2245585753 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f51fd6fc640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f815deb0 0x7f51f8156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f51fdefd640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e40036a0 0x7f51f81e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f51fe6fe640 1 -- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f815deb0 msgr2=0x7f51f8156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.015+0000 7f51fe6fe640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f815deb0 0x7f51f8156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f51fe6fe640 1 -- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e40036a0 msgr2=0x7f51f81e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f51fe6fe640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e40036a0 0x7f51f81e61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f51fe6fe640 1 -- 172.21.15.113:0/2245585753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f81e37e0 con 0x7f51f81e45a0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f51fe6fe640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f81e45a0 0x7f51f81e4a30 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f51f8074d30 tx=0x7f51f000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f51ee7fc640 1 -- 172.21.15.113:0/2245585753 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0003970 con 0x7f51f81e45a0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f51ee7fc640 1 -- 172.21.15.113:0/2245585753 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0018070 con 0x7f51f81e45a0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f81e3ac0 con 0x7f51f81e45a0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f51ee7fc640 1 -- 172.21.15.113:0/2245585753 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0003d50 con 0x7f51f81e45a0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.016+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f81e4080 con 0x7f51f81e45a0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.017+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f80691a0 con 0x7f51f81e45a0 2024-04-25T00:03:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.018+0000 7f51ee7fc640 1 -- 172.21.15.113:0/2245585753 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51f00050d0 con 0x7f51f81e45a0 2024-04-25T00:03:58.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.018+0000 7f51ee7fc640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51dc06be60 0x7f51dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:58.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.018+0000 7f51ee7fc640 1 -- 172.21.15.113:0/2245585753 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5415+0+0 (secure 0 0 0) 0x7f51f008a810 con 0x7f51f81e45a0 2024-04-25T00:03:58.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.018+0000 7f51fdefd640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51dc06be60 0x7f51dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:58.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.018+0000 7f51fdefd640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51dc06be60 0x7f51dc06e320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000c00 tx=0x7f51f4007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:58.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.022+0000 7f51ee7fc640 1 -- 172.21.15.113:0/2245585753 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f51f0058d20 con 0x7f51f81e45a0 2024-04-25T00:03:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.197+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f51f80cee70 con 0x7f51f81e45a0 2024-04-25T00:03:58.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.880+0000 7f51ee7fc640 1 -- 172.21.15.113:0/2245585753 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f51f005f030 con 0x7f51f81e45a0 2024-04-25T00:03:58.880 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:03:58.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51dc06be60 msgr2=0x7f51dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:58.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51dc06be60 0x7f51dc06e320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000c00 tx=0x7f51f4007040 comp rx=0 tx=0).stop 2024-04-25T00:03:58.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f81e45a0 msgr2=0x7f51f81e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:58.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f81e45a0 0x7f51f81e4a30 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f51f8074d30 tx=0x7f51f000eb90 comp rx=0 tx=0).stop 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 shutdown_connections 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51dc06be60 0x7f51dc06e320 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000c00 tx=0x7f51f4007040 comp rx=0 tx=0).stop 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f81e45a0 0x7f51f81e4a30 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51f815deb0 0x7f51f8156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 --2- 172.21.15.113:0/2245585753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e40036a0 0x7f51f81e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.885+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 >> 172.21.15.113:0/2245585753 conn(0x7f51f80b1b30 msgr2=0x7f51f80cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.887+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 shutdown_connections 2024-04-25T00:03:58.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:58.887+0000 7f5200161640 1 -- 172.21.15.113:0/2245585753 wait complete. 2024-04-25T00:03:58.905 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:03:59.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.006+0000 7f1c3049e640 1 Processor -- start 2024-04-25T00:03:59.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.006+0000 7f1c3049e640 1 -- start start 2024-04-25T00:03:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.007+0000 7f1c3049e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c2815e2d0 0x7f1c2815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.007+0000 7f1c3049e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c280b3080 0x7f1c280b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.007+0000 7f1c3049e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2815f4f0 con 0x7f1c2815e6b0 2024-04-25T00:03:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.007+0000 7f1c3049e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1c280cde00 con 0x7f1c2815e2d0 2024-04-25T00:03:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.007+0000 7f1c3049e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2815f7d0 con 0x7f1c280b3080 2024-04-25T00:03:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2e23a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c2815e2d0 0x7f1c2815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2ea3b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1c2815e6b0 0x7f1c28071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56438/0 (socket says 172.21.15.113:56438) 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2ea3b640 1 -- 172.21.15.113:0/416313708 learned_addr learned my addr 172.21.15.113:0/416313708 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2e23a640 1 -- 172.21.15.113:0/416313708 >> v2:172.21.15.178:3300/0 conn(0x7f1c280b3080 msgr2=0x7f1c280b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2da39640 1 --2- 172.21.15.113:0/416313708 >> v2:172.21.15.178:3300/0 conn(0x7f1c280b3080 0x7f1c280b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2e23a640 1 --2- 172.21.15.113:0/416313708 >> v2:172.21.15.178:3300/0 conn(0x7f1c280b3080 0x7f1c280b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2e23a640 1 -- 172.21.15.113:0/416313708 >> v1:172.21.15.113:6789/0 conn(0x7f1c2815e6b0 legacy=0x7f1c28071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2d238640 1 -- 172.21.15.113:0/416313708 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2657757706 0 0) 0x7f1c2815f4f0 con 0x7f1c2815e6b0 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2e23a640 1 -- 172.21.15.113:0/416313708 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2815fab0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2da39640 1 --2- 172.21.15.113:0/416313708 >> v2:172.21.15.178:3300/0 conn(0x7f1c280b3080 0x7f1c280b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.008+0000 7f1c2e23a640 1 --2- 172.21.15.113:0/416313708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c2815ebf0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1c28070ff0 tx=0x7f1c2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6b4f81e0b64972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c2d238640 1 -- 172.21.15.113:0/416313708 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20007740 con 0x7f1c2815e2d0 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c2d238640 1 -- 172.21.15.113:0/416313708 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c200078e0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c2d238640 1 -- 172.21.15.113:0/416313708 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20007cb0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 -- 172.21.15.113:0/416313708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 msgr2=0x7f1c2815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 --2- 172.21.15.113:0/416313708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c2815ebf0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1c28070ff0 tx=0x7f1c2002a680 comp rx=0 tx=0).stop 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 -- 172.21.15.113:0/416313708 shutdown_connections 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 --2- 172.21.15.113:0/416313708 >> v2:172.21.15.178:3300/0 conn(0x7f1c280b3080 0x7f1c280b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 --2- 172.21.15.113:0/416313708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c2815ebf0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 -- 172.21.15.113:0/416313708 >> 172.21.15.113:0/416313708 conn(0x7f1c2806b0f0 msgr2=0x7f1c2806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 -- 172.21.15.113:0/416313708 shutdown_connections 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.009+0000 7f1c3049e640 1 -- 172.21.15.113:0/416313708 wait complete. 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.010+0000 7f1c3049e640 1 Processor -- start 2024-04-25T00:03:59.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.010+0000 7f1c3049e640 1 -- start start 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c3049e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c280b3080 0x7f1c2814ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c3049e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c28154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c3049e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2815e6b0 0x7f1c280b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c3049e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2815f2b0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c3049e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2815f130 con 0x7f1c280b3080 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c3049e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1c280cf2a0 con 0x7f1c2815e6b0 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c28154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c28154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50470/0 (socket says 172.21.15.113:50470) 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 -- 172.21.15.113:0/1909272902 learned_addr learned my addr 172.21.15.113:0/1909272902 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2e23a640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c280b3080 0x7f1c2814ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2ea3b640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2815e6b0 0x7f1c280b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 -- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2815e6b0 msgr2=0x7f1c280b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2815e6b0 0x7f1c280b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 -- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c280b3080 msgr2=0x7f1c2814ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c280b3080 0x7f1c2814ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2da39640 1 -- 172.21.15.113:0/1909272902 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2814f120 con 0x7f1c2815e2d0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.011+0000 7f1c2ea3b640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2815e6b0 0x7f1c280b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.012+0000 7f1c2da39640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c28154ef0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c7f0 tx=0x7f1c1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.012+0000 7f1c1effd640 1 -- 172.21.15.113:0/1909272902 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18015070 con 0x7f1c2815e2d0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.012+0000 7f1c1effd640 1 -- 172.21.15.113:0/1909272902 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c18007a30 con 0x7f1c2815e2d0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.012+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2815a220 con 0x7f1c2815e2d0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.012+0000 7f1c1effd640 1 -- 172.21.15.113:0/1909272902 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18007e00 con 0x7f1c2815e2d0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.012+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2815a7e0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.013+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2815c3c0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.013+0000 7f1c1effd640 1 -- 172.21.15.113:0/1909272902 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c18018070 con 0x7f1c2815e2d0 2024-04-25T00:03:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.013+0000 7f1c1effd640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c0c06be30 0x7f1c0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.014+0000 7f1c1effd640 1 -- 172.21.15.113:0/1909272902 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4943+0+0 (secure 0 0 0) 0x7f1c1808aae0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.014+0000 7f1c2e23a640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c0c06be30 0x7f1c0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.014+0000 7f1c2e23a640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c0c06be30 0x7f1c0c06e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1c20034000 tx=0x7f1c2002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:59.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.018+0000 7f1c1effd640 1 -- 172.21.15.113:0/1909272902 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c18059790 con 0x7f1c2815e2d0 2024-04-25T00:03:59.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.192+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1c280691a0 con 0x7f1c2815e2d0 2024-04-25T00:03:59.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.195+0000 7f1c1effd640 1 -- 172.21.15.113:0/1909272902 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f1c18059090 con 0x7f1c2815e2d0 2024-04-25T00:03:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c0c06be30 msgr2=0x7f1c0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c0c06be30 0x7f1c0c06e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1c20034000 tx=0x7f1c2002f000 comp rx=0 tx=0).stop 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 msgr2=0x7f1c28154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c28154ef0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c7f0 tx=0x7f1c1800cbf0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 shutdown_connections 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c0c06be30 0x7f1c0c06e2f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2815e6b0 0x7f1c280b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2815e2d0 0x7f1c28154ef0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 --2- 172.21.15.113:0/1909272902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c280b3080 0x7f1c2814ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 >> 172.21.15.113:0/1909272902 conn(0x7f1c2806b0f0 msgr2=0x7f1c280cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 shutdown_connections 2024-04-25T00:03:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.197+0000 7f1c3049e640 1 -- 172.21.15.113:0/1909272902 wait complete. 2024-04-25T00:03:59.208 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T00:03:59.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.306+0000 7f60df5ed640 1 Processor -- start 2024-04-25T00:03:59.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.306+0000 7f60df5ed640 1 -- start start 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60df5ed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 0x7f60d8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60df5ed640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f60d80b3af0 0x7f60d80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60df5ed640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d80b2d50 con 0x7f60d815a6b0 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60df5ed640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f60d8072a70 con 0x7f60d80b3af0 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60df5ed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60d80b3010 con 0x7f60d8072ea0 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60dd389640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f60d815a6b0 0x7f60d8070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56444/0 (socket says 172.21.15.113:56444) 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60dd389640 1 -- 172.21.15.113:0/3159191975 learned_addr learned my addr 172.21.15.113:0/3159191975 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160839377 0 0) 0x7f60d80b2d50 con 0x7f60d815a6b0 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c0003520 con 0x7f60d815a6b0 2024-04-25T00:03:59.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.307+0000 7f60ddb8a640 1 --2- 172.21.15.113:0/3159191975 >> v2:172.21.15.178:3300/0 conn(0x7f60d80b3af0 0x7f60d80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60dcb88640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 0x7f60d8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3547918024 0 0) 0x7f60c0003520 con 0x7f60d815a6b0 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 msgr2=0x7f60d8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 0x7f60d8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 >> v2:172.21.15.178:3300/0 conn(0x7f60d80b3af0 msgr2=0x7f60d80b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 --2- 172.21.15.113:0/3159191975 >> v2:172.21.15.178:3300/0 conn(0x7f60d80b3af0 0x7f60d80b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d80b32f0 con 0x7f60d815a6b0 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f60c80019f0 con 0x7f60d815a6b0 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 >> v1:172.21.15.113:6789/0 conn(0x7f60d815a6b0 legacy=0x7f60d8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 0x7f60c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c0003fe0 0x7f60c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c0004d40 0x7f60c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8072a70 con 0x7f60c0004d40 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60d80b3010 con 0x7f60c0003fe0 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60c000bac0 con 0x7f60d8072ea0 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60ddb8a640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c0004d40 0x7f60c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60dd389640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 0x7f60c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60dcb88640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c0003fe0 0x7f60c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60ddb8a640 1 -- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 msgr2=0x7f60c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60ddb8a640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d8072ea0 0x7f60c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60ddb8a640 1 -- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c0003fe0 msgr2=0x7f60c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60ddb8a640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c0003fe0 0x7f60c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.308+0000 7f60ddb8a640 1 -- 172.21.15.113:0/3159191975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c0003520 con 0x7f60c0004d40 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60ddb8a640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c0004d40 0x7f60c00051c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f60d000ba30 tx=0x7f60c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191e1dc42bf083b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0030070 con 0x7f60c0004d40 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d002ae10 con 0x7f60c0004d40 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60d7fff640 1 -- 172.21.15.113:0/3159191975 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000b600 con 0x7f60c0004d40 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60df5ed640 1 -- 172.21.15.113:0/3159191975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c0004d40 msgr2=0x7f60c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60df5ed640 1 --2- 172.21.15.113:0/3159191975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c0004d40 0x7f60c00051c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f60d000ba30 tx=0x7f60c0005830 comp rx=0 tx=0).stop 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60ddb8a640 1 -- 172.21.15.113:0/3159191975 reap_dead start 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60df5ed640 1 -- 172.21.15.113:0/3159191975 shutdown_connections 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.309+0000 7f60df5ed640 1 -- 172.21.15.113:0/3159191975 >> 172.21.15.113:0/3159191975 conn(0x7f60d80b1a50 msgr2=0x7f60d80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.310+0000 7f60df5ed640 1 -- 172.21.15.113:0/3159191975 shutdown_connections 2024-04-25T00:03:59.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.310+0000 7f60df5ed640 1 -- 172.21.15.113:0/3159191975 wait complete. 2024-04-25T00:03:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 Processor -- start 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 -- start start 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d81e3b30 0x7f60d815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d814dde0 0x7f60d81509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60d81e3f10 0x7f60d815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60d815aa90 con 0x7f60d81e3b30 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8072720 con 0x7f60d81e3f10 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60df5ed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60d815ac10 con 0x7f60d814dde0 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60dd389640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d81e3b30 0x7f60d815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60dd389640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d81e3b30 0x7f60d815cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50484/0 (socket says 172.21.15.113:50484) 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.311+0000 7f60dd389640 1 -- 172.21.15.113:0/3987020515 learned_addr learned my addr 172.21.15.113:0/3987020515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60dcb88640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d814dde0 0x7f60d81509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60ddb8a640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60d81e3f10 0x7f60d815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60dd389640 1 -- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d814dde0 msgr2=0x7f60d81509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60dd389640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d814dde0 0x7f60d81509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60dd389640 1 -- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60d81e3f10 msgr2=0x7f60d815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60dd389640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60d81e3f10 0x7f60d815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60dd389640 1 -- 172.21.15.113:0/3987020515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d815d270 con 0x7f60d81e3b30 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60dd389640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d81e3b30 0x7f60d815cc20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f60c800a630 tx=0x7f60c800aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60d5ffb640 1 -- 172.21.15.113:0/3987020515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8017020 con 0x7f60d81e3b30 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60d5ffb640 1 -- 172.21.15.113:0/3987020515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c8013600 con 0x7f60d81e3b30 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d815d550 con 0x7f60d81e3b30 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60d5ffb640 1 -- 172.21.15.113:0/3987020515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8013a10 con 0x7f60d81e3b30 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.312+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d815b2d0 con 0x7f60d81e3b30 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.313+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d80691a0 con 0x7f60d81e3b30 2024-04-25T00:03:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.313+0000 7f60d5ffb640 1 -- 172.21.15.113:0/3987020515 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60c801a020 con 0x7f60d81e3b30 2024-04-25T00:03:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.314+0000 7f60d5ffb640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60c406bdd0 0x7f60c406e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.314+0000 7f60d5ffb640 1 -- 172.21.15.113:0/3987020515 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4943+0+0 (secure 0 0 0) 0x7f60c808bbc0 con 0x7f60d81e3b30 2024-04-25T00:03:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.314+0000 7f60dcb88640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60c406bdd0 0x7f60c406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.314+0000 7f60dcb88640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60c406bdd0 0x7f60c406e290 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60d81e4570 tx=0x7f60cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.318+0000 7f60d5ffb640 1 -- 172.21.15.113:0/3987020515 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60c8021020 con 0x7f60d81e3b30 2024-04-25T00:03:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.560+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) -- 0x7f60d80b2d50 con 0x7f60d81e3b30 2024-04-25T00:03:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.891+0000 7f60d5ffb640 1 -- 172.21.15.113:0/3987020515 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v277) ==== 158+0+0 (secure 0 0 0) 0x7f60c805a860 con 0x7f60d81e3b30 2024-04-25T00:03:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.893+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60c406bdd0 msgr2=0x7f60c406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.893+0000 7f60df5ed640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60c406bdd0 0x7f60c406e290 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60d81e4570 tx=0x7f60cc008040 comp rx=0 tx=0).stop 2024-04-25T00:03:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.893+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d81e3b30 msgr2=0x7f60d815cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.893+0000 7f60df5ed640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d81e3b30 0x7f60d815cc20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f60c800a630 tx=0x7f60c800aaf0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 shutdown_connections 2024-04-25T00:03:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f60c406bdd0 0x7f60c406e290 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60d81e3f10 0x7f60d815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60d814dde0 0x7f60d81509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 --2- 172.21.15.113:0/3987020515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d81e3b30 0x7f60d815cc20 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 >> 172.21.15.113:0/3987020515 conn(0x7f60d80b1a50 msgr2=0x7f60d80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 shutdown_connections 2024-04-25T00:03:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:03:59.894+0000 7f60df5ed640 1 -- 172.21.15.113:0/3987020515 wait complete. 2024-04-25T00:03:59.906 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T00:03:59.906 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T00:03:59.907 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:00.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f251cf09640 1 Processor -- start 2024-04-25T00:04:00.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f251cf09640 1 -- start start 2024-04-25T00:04:00.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f251cf09640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f251815e320 0x7f25180d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f251cf09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25180d0b00 0x7f25180d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f251cf09640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25180d3de0 con 0x7f25180d2290 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f251cf09640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f25180d14b0 con 0x7f25180d0b00 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f251cf09640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f25180d3410 con 0x7f251815e320 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f2516d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25180d0b00 0x7f25180d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.005+0000 7f250ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f25180d2290 0x7f25180b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56460/0 (socket says 172.21.15.113:56460) 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f250ffff640 1 -- 172.21.15.113:0/401129759 learned_addr learned my addr 172.21.15.113:0/401129759 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f2516d9d640 1 -- 172.21.15.113:0/401129759 >> v2:172.21.15.178:3300/0 conn(0x7f251815e320 msgr2=0x7f25180d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f251659c640 1 --2- 172.21.15.113:0/401129759 >> v2:172.21.15.178:3300/0 conn(0x7f251815e320 0x7f25180d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f2516d9d640 1 --2- 172.21.15.113:0/401129759 >> v2:172.21.15.178:3300/0 conn(0x7f251815e320 0x7f25180d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f2515d9b640 1 -- 172.21.15.113:0/401129759 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637067122 0 0) 0x7f25180d3de0 con 0x7f25180d2290 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f2516d9d640 1 -- 172.21.15.113:0/401129759 >> v1:172.21.15.113:6789/0 conn(0x7f25180d2290 legacy=0x7f25180b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f2516d9d640 1 -- 172.21.15.113:0/401129759 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25180d2800 con 0x7f25180d0b00 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f251659c640 1 --2- 172.21.15.113:0/401129759 >> v2:172.21.15.178:3300/0 conn(0x7f251815e320 0x7f25180d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.006+0000 7f2516d9d640 1 --2- 172.21.15.113:0/401129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d0b00 0x7f25180d0f70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f25180b6d00 tx=0x7f251002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc15a6d6b2a7bea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f2515d9b640 1 -- 172.21.15.113:0/401129759 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510007740 con 0x7f25180d0b00 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f2515d9b640 1 -- 172.21.15.113:0/401129759 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25100078e0 con 0x7f25180d0b00 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f2515d9b640 1 -- 172.21.15.113:0/401129759 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510007cb0 con 0x7f25180d0b00 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 -- 172.21.15.113:0/401129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d0b00 msgr2=0x7f25180d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 --2- 172.21.15.113:0/401129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d0b00 0x7f25180d0f70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f25180b6d00 tx=0x7f251002a710 comp rx=0 tx=0).stop 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 -- 172.21.15.113:0/401129759 shutdown_connections 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 --2- 172.21.15.113:0/401129759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d0b00 0x7f25180d0f70 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 --2- 172.21.15.113:0/401129759 >> v2:172.21.15.178:3300/0 conn(0x7f251815e320 0x7f25180d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 -- 172.21.15.113:0/401129759 >> 172.21.15.113:0/401129759 conn(0x7f25180b5c20 msgr2=0x7f25180b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 -- 172.21.15.113:0/401129759 shutdown_connections 2024-04-25T00:04:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.007+0000 7f251cf09640 1 -- 172.21.15.113:0/401129759 wait complete. 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 Processor -- start 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 -- start start 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25180d0b00 0x7f25181f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d2290 0x7f25180b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f251815e320 0x7f2518145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f25180d30a0 con 0x7f25180d2290 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25180d2dc0 con 0x7f25180d0b00 2024-04-25T00:04:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f251cf09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f251815e700 con 0x7f251815e320 2024-04-25T00:04:00.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f250ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d2290 0x7f25180b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f250ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d2290 0x7f25180b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50510/0 (socket says 172.21.15.113:50510) 2024-04-25T00:04:00.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.009+0000 7f250ffff640 1 -- 172.21.15.113:0/3179789248 learned_addr learned my addr 172.21.15.113:0/3179789248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:00.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f2516d9d640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f251815e320 0x7f2518145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f251659c640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25180d0b00 0x7f25181f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250ffff640 1 -- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f251815e320 msgr2=0x7f2518145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250ffff640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f251815e320 0x7f2518145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250ffff640 1 -- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25180d0b00 msgr2=0x7f25181f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250ffff640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25180d0b00 0x7f25181f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250ffff640 1 -- 172.21.15.113:0/3179789248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25181604a0 con 0x7f25180d2290 2024-04-25T00:04:00.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f2516d9d640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f251815e320 0x7f2518145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:00.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f251659c640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25180d0b00 0x7f25181f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:00.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250ffff640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d2290 0x7f25180b6270 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f250000c500 tx=0x7f250000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250effd640 1 -- 172.21.15.113:0/3179789248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500018070 con 0x7f25180d2290 2024-04-25T00:04:00.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250effd640 1 -- 172.21.15.113:0/3179789248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2500007580 con 0x7f25180d2290 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2518146b40 con 0x7f25180d2290 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f250effd640 1 -- 172.21.15.113:0/3179789248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500004a20 con 0x7f25180d2290 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.010+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2518160a20 con 0x7f25180d2290 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.011+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25181f5590 con 0x7f25180d2290 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.011+0000 7f250effd640 1 -- 172.21.15.113:0/3179789248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f250001b070 con 0x7f25180d2290 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.012+0000 7f250effd640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24f806be30 0x7f24f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.012+0000 7f250effd640 1 -- 172.21.15.113:0/3179789248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4943+0+0 (secure 0 0 0) 0x7f2500089dd0 con 0x7f25180d2290 2024-04-25T00:04:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.012+0000 7f251659c640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24f806be30 0x7f24f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.012+0000 7f251659c640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24f806be30 0x7f24f806e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.016+0000 7f250effd640 1 -- 172.21.15.113:0/3179789248 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2500058b10 con 0x7f25180d2290 2024-04-25T00:04:00.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.191+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2518146860 con 0x7f25180d2290 2024-04-25T00:04:00.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.192+0000 7f250effd640 1 -- 172.21.15.113:0/3179789248 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13071 (secure 0 0 0) 0x7f250008b090 con 0x7f25180d2290 2024-04-25T00:04:00.192 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:00.192 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":83,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:03:58.879880+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6837/3530277358":"2024-04-26T00:03:55.617844+0000","172.21.15.113:6835/3530277358":"2024-04-26T00:03:55.617844+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:00.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24f806be30 msgr2=0x7f24f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24f806be30 0x7f24f806e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508008040 comp rx=0 tx=0).stop 2024-04-25T00:04:00.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d2290 msgr2=0x7f25180b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d2290 0x7f25180b6270 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f250000c500 tx=0x7f250000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 shutdown_connections 2024-04-25T00:04:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24f806be30 0x7f24f806e2f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f251815e320 0x7f2518145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25180d2290 0x7f25180b6270 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 --2- 172.21.15.113:0/3179789248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25180d0b00 0x7f25181f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 >> 172.21.15.113:0/3179789248 conn(0x7f25180b5c20 msgr2=0x7f25180b86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 shutdown_connections 2024-04-25T00:04:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.194+0000 7f251cf09640 1 -- 172.21.15.113:0/3179789248 wait complete. 2024-04-25T00:04:00.205 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:04:00.205 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:04:00.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.310+0000 7f2cff99e640 1 Processor -- start 2024-04-25T00:04:00.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.310+0000 7f2cff99e640 1 -- start start 2024-04-25T00:04:00.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cff99e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 0x7f2cf815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cff99e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2cf80b3080 0x7f2cf80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cff99e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2cf815e6b0 0x7f2cf80b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cff99e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf80cde00 con 0x7f2cf815e6b0 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cff99e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf815f4f0 con 0x7f2cf80b3080 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cff99e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf815f7d0 con 0x7f2cf815e2d0 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2cf815e6b0 0x7f2cf80b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2cf815e6b0 0x7f2cf80b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50524/0 (socket says 172.21.15.113:50524) 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 -- 172.21.15.113:0/3769355213 learned_addr learned my addr 172.21.15.113:0/3769355213 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 -- 172.21.15.113:0/3769355213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 msgr2=0x7f2cf815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 --2- 172.21.15.113:0/3769355213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 0x7f2cf815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 -- 172.21.15.113:0/3769355213 >> v2:172.21.15.178:3300/0 conn(0x7f2cf80b3080 msgr2=0x7f2cf80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 --2- 172.21.15.113:0/3769355213 >> v2:172.21.15.178:3300/0 conn(0x7f2cf80b3080 0x7f2cf80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.311+0000 7f2cfdf3b640 1 -- 172.21.15.113:0/3769355213 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf815fab0 con 0x7f2cf815e6b0 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.312+0000 7f2cfdf3b640 1 --2- 172.21.15.113:0/3769355213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf815e6b0 0x7f2cf80b4870 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80d1180 tx=0x7f2cf002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545331c3f2850bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.312+0000 7f2ceffff640 1 -- 172.21.15.113:0/3769355213 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0036070 con 0x7f2cf815e6b0 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.312+0000 7f2ceffff640 1 -- 172.21.15.113:0/3769355213 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf0031600 con 0x7f2cf815e6b0 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.312+0000 7f2ceffff640 1 -- 172.21.15.113:0/3769355213 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00319d0 con 0x7f2cf815e6b0 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.312+0000 7f2cff99e640 1 -- 172.21.15.113:0/3769355213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf815e6b0 msgr2=0x7f2cf80b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.312+0000 7f2cff99e640 1 --2- 172.21.15.113:0/3769355213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf815e6b0 0x7f2cf80b4870 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80d1180 tx=0x7f2cf002a610 comp rx=0 tx=0).stop 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.313+0000 7f2cff99e640 1 -- 172.21.15.113:0/3769355213 shutdown_connections 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.313+0000 7f2cff99e640 1 --2- 172.21.15.113:0/3769355213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf815e6b0 0x7f2cf80b4870 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.313+0000 7f2cff99e640 1 --2- 172.21.15.113:0/3769355213 >> v2:172.21.15.178:3300/0 conn(0x7f2cf80b3080 0x7f2cf80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.313+0000 7f2cff99e640 1 --2- 172.21.15.113:0/3769355213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 0x7f2cf815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.313+0000 7f2cff99e640 1 -- 172.21.15.113:0/3769355213 >> 172.21.15.113:0/3769355213 conn(0x7f2cf806b0f0 msgr2=0x7f2cf806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:00.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.313+0000 7f2cff99e640 1 -- 172.21.15.113:0/3769355213 shutdown_connections 2024-04-25T00:04:00.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.313+0000 7f2cff99e640 1 -- 172.21.15.113:0/3769355213 wait complete. 2024-04-25T00:04:00.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 Processor -- start 2024-04-25T00:04:00.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 -- start start 2024-04-25T00:04:00.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf80b3080 0x7f2cf81489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 0x7f2cf8150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cf815e6b0 0x7f2cf8071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf815f2b0 con 0x7f2cf80b3080 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf815f130 con 0x7f2cf815e6b0 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.314+0000 7f2cff99e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf8071040 con 0x7f2cf815e2d0 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf80b3080 0x7f2cf81489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf80b3080 0x7f2cf81489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50534/0 (socket says 172.21.15.113:50534) 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 -- 172.21.15.113:0/2499610440 learned_addr learned my addr 172.21.15.113:0/2499610440 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfdf3b640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cf815e6b0 0x7f2cf8071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfcf39640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 0x7f2cf8150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 -- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 msgr2=0x7f2cf8150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 0x7f2cf8150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 -- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cf815e6b0 msgr2=0x7f2cf8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cf815e6b0 0x7f2cf8071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 -- 172.21.15.113:0/2499610440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf8149020 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cfd73a640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf80b3080 0x7f2cf81489b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400b4d0 tx=0x7f2cf400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cedffb640 1 -- 172.21.15.113:0/2499610440 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4015020 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cedffb640 1 -- 172.21.15.113:0/2499610440 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4011850 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf8149300 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cedffb640 1 -- 172.21.15.113:0/2499610440 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4011c40 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.315+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf81498c0 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.316+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf80691a0 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.317+0000 7f2cedffb640 1 -- 172.21.15.113:0/2499610440 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cf4018020 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.317+0000 7f2cedffb640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cdc06be30 0x7f2cdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.317+0000 7f2cedffb640 1 -- 172.21.15.113:0/2499610440 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4943+0+0 (secure 0 0 0) 0x7f2cf408a960 con 0x7f2cf80b3080 2024-04-25T00:04:00.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.317+0000 7f2cfcf39640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cdc06be30 0x7f2cdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.318+0000 7f2cfcf39640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cdc06be30 0x7f2cdc06e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2cf81529b0 tx=0x7f2ce8009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.321+0000 7f2cedffb640 1 -- 172.21.15.113:0/2499610440 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2cf401f020 con 0x7f2cf80b3080 2024-04-25T00:04:00.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.492+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cf8072050 con 0x7f2cf80b3080 2024-04-25T00:04:00.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.492+0000 7f2cedffb640 1 -- 172.21.15.113:0/2499610440 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f2cf40596a0 con 0x7f2cf80b3080 2024-04-25T00:04:00.492 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:04:00.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cdc06be30 msgr2=0x7f2cdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cdc06be30 0x7f2cdc06e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2cf81529b0 tx=0x7f2ce8009110 comp rx=0 tx=0).stop 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf80b3080 msgr2=0x7f2cf81489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf80b3080 0x7f2cf81489b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400b4d0 tx=0x7f2cf400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 shutdown_connections 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cdc06be30 0x7f2cdc06e2f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cf815e6b0 0x7f2cf8071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cf815e2d0 0x7f2cf8150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 --2- 172.21.15.113:0/2499610440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf80b3080 0x7f2cf81489b0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 >> 172.21.15.113:0/2499610440 conn(0x7f2cf806b0f0 msgr2=0x7f2cf815d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 shutdown_connections 2024-04-25T00:04:00.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.494+0000 7f2cff99e640 1 -- 172.21.15.113:0/2499610440 wait complete. 2024-04-25T00:04:00.505 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T00:04:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.601+0000 7f6a6c4d4640 1 Processor -- start 2024-04-25T00:04:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.601+0000 7f6a6c4d4640 1 -- start start 2024-04-25T00:04:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6c4d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 0x7f6a640b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6c4d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a64144cf0 con 0x7f6a6406e6e0 2024-04-25T00:04:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6c4d4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a640b2f30 con 0x7f6a640b5240 2024-04-25T00:04:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6c4d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a640b4510 con 0x7f6a64144910 2024-04-25T00:04:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6aa71640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6a6406e6e0 0x7f6a640cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56472/0 (socket says 172.21.15.113:56472) 2024-04-25T00:04:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6aa71640 1 -- 172.21.15.113:0/2630101688 learned_addr learned my addr 172.21.15.113:0/2630101688 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6a270640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 0x7f6a640b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2998113894 0 0) 0x7f6a64144cf0 con 0x7f6a6406e6e0 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.602+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a50003520 con 0x7f6a6406e6e0 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 586978900 0 0) 0x7f6a50003520 con 0x7f6a6406e6e0 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 msgr2=0x7f6a640b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 0x7f6a640b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 >> v1:172.21.15.178:6789/0 conn(0x7f6a640b5240 legacy=0x7f6a6406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a640b3210 con 0x7f6a6406e6e0 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6a5c001670 con 0x7f6a6406e6e0 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 >> v1:172.21.15.113:6789/0 conn(0x7f6a6406e6e0 legacy=0x7f6a640cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 0x7f6a500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a50003f60 0x7f6a50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a50004d40 0x7f6a500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a640b4510 con 0x7f6a50004d40 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5000baa0 con 0x7f6a50003f60 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5000bca0 con 0x7f6a64144910 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6aa71640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a50004d40 0x7f6a500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a69a6f640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a50003f60 0x7f6a50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6a270640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 0x7f6a500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6aa71640 1 -- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 msgr2=0x7f6a500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6aa71640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a64144910 0x7f6a500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6aa71640 1 -- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a50003f60 msgr2=0x7f6a50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6aa71640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a50003f60 0x7f6a50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.603+0000 7f6a6aa71640 1 -- 172.21.15.113:0/2630101688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a50003520 con 0x7f6a50004d40 2024-04-25T00:04:00.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.604+0000 7f6a6aa71640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a50004d40 0x7f6a500051c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c240 tx=0x7f6a50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219c642bbe77dcd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.604+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c007cf0 con 0x7f6a50004d40 2024-04-25T00:04:00.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.604+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5c0058d0 con 0x7f6a50004d40 2024-04-25T00:04:00.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.604+0000 7f6a6926e640 1 -- 172.21.15.113:0/2630101688 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c005d20 con 0x7f6a50004d40 2024-04-25T00:04:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.604+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/2630101688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a50004d40 msgr2=0x7f6a500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.604+0000 7f6a6c4d4640 1 --2- 172.21.15.113:0/2630101688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a50004d40 0x7f6a500051c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c240 tx=0x7f6a50005830 comp rx=0 tx=0).stop 2024-04-25T00:04:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.604+0000 7f6a6aa71640 1 -- 172.21.15.113:0/2630101688 reap_dead start 2024-04-25T00:04:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.605+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/2630101688 shutdown_connections 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.605+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/2630101688 >> 172.21.15.113:0/2630101688 conn(0x7f6a640b1a50 msgr2=0x7f6a640b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.605+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/2630101688 shutdown_connections 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.605+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/2630101688 wait complete. 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.606+0000 7f6a6c4d4640 1 Processor -- start 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.606+0000 7f6a6c4d4640 1 -- start start 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.606+0000 7f6a6c4d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a50004d40 0x7f6a6414c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.606+0000 7f6a6c4d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a6413e7b0 0x7f6a641f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6c4d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a6413ec30 0x7f6a640af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6c4d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a640b4da0 con 0x7f6a6413ec30 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6c4d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a640b37d0 con 0x7f6a6413e7b0 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6c4d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a640b4f20 con 0x7f6a50004d40 2024-04-25T00:04:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a6413ec30 0x7f6a640af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a6413ec30 0x7f6a640af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50554/0 (socket says 172.21.15.113:50554) 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 -- 172.21.15.113:0/70568309 learned_addr learned my addr 172.21.15.113:0/70568309 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a69a6f640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a6413e7b0 0x7f6a641f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6a270640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a50004d40 0x7f6a6414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 -- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a50004d40 msgr2=0x7f6a6414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a50004d40 0x7f6a6414c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 -- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a6413e7b0 msgr2=0x7f6a641f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a6413e7b0 0x7f6a641f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 -- 172.21.15.113:0/70568309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6414c5c0 con 0x7f6a6413ec30 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a69a6f640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a6413e7b0 0x7f6a641f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6aa71640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a6413ec30 0x7f6a640af910 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c360 tx=0x7f6a640b3100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a5affd640 1 -- 172.21.15.113:0/70568309 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c043070 con 0x7f6a6413ec30 2024-04-25T00:04:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a5affd640 1 -- 172.21.15.113:0/70568309 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5c0047e0 con 0x7f6a6413ec30 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a641f5d40 con 0x7f6a6413ec30 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a5affd640 1 -- 172.21.15.113:0/70568309 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c038040 con 0x7f6a6413ec30 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.607+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a640a9cc0 con 0x7f6a6413ec30 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.608+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a640691a0 con 0x7f6a6413ec30 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.609+0000 7f6a5affd640 1 -- 172.21.15.113:0/70568309 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a5c046070 con 0x7f6a6413ec30 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.609+0000 7f6a5affd640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a4806be30 0x7f6a4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.609+0000 7f6a5affd640 1 -- 172.21.15.113:0/70568309 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4943+0+0 (secure 0 0 0) 0x7f6a5c0aa240 con 0x7f6a6413ec30 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.609+0000 7f6a6a270640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a4806be30 0x7f6a4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:00.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.610+0000 7f6a6a270640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a4806be30 0x7f6a4806e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a60007520 tx=0x7f6a6000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:00.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.613+0000 7f6a5affd640 1 -- 172.21.15.113:0/70568309 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a5c0788a0 con 0x7f6a6413ec30 2024-04-25T00:04:00.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.851+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) -- 0x7f6a640731a0 con 0x7f6a6413ec30 2024-04-25T00:04:00.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.892+0000 7f6a5affd640 1 -- 172.21.15.113:0/70568309 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v278) ==== 147+0+0 (secure 0 0 0) 0x7f6a5c07cba0 con 0x7f6a6413ec30 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.894+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a4806be30 msgr2=0x7f6a4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.894+0000 7f6a6c4d4640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a4806be30 0x7f6a4806e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a60007520 tx=0x7f6a6000b040 comp rx=0 tx=0).stop 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a6413ec30 msgr2=0x7f6a640af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a6413ec30 0x7f6a640af910 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c360 tx=0x7f6a640b3100 comp rx=0 tx=0).stop 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 shutdown_connections 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a4806be30 0x7f6a4806e2f0 secure :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a60007520 tx=0x7f6a6000b040 comp rx=0 tx=0).stop 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a6413e7b0 0x7f6a641f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a6413ec30 0x7f6a640af910 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 --2- 172.21.15.113:0/70568309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a50004d40 0x7f6a6414c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:00.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 >> 172.21.15.113:0/70568309 conn(0x7f6a640b1a50 msgr2=0x7f6a6406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:00.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 shutdown_connections 2024-04-25T00:04:00.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:00.895+0000 7f6a6c4d4640 1 -- 172.21.15.113:0/70568309 wait complete. 2024-04-25T00:04:00.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:04:01.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.005+0000 7f341b3ab640 1 Processor -- start 2024-04-25T00:04:01.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.006+0000 7f341b3ab640 1 -- start start 2024-04-25T00:04:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.006+0000 7f341b3ab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341415ba50 0x7f341415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.006+0000 7f341b3ab640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f341406cea0 0x7f341406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.006+0000 7f341b3ab640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3414071170 con 0x7f341415b670 2024-04-25T00:04:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.006+0000 7f341b3ab640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f341406d810 con 0x7f341406cea0 2024-04-25T00:04:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.006+0000 7f341b3ab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3414071450 con 0x7f341415ba50 2024-04-25T00:04:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419948640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f341406cea0 0x7f341406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3418946640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341415ba50 0x7f341415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419147640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f341415b670 0x7f34140cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:51168/0 (socket says 172.21.15.113:51168) 2024-04-25T00:04:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419948640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f341406cea0 0x7f341406d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:53302/0 (socket says 172.21.15.113:53302) 2024-04-25T00:04:01.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419147640 1 -- 172.21.15.113:0/3265621451 learned_addr learned my addr 172.21.15.113:0/3265621451 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:01.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419948640 1 -- 172.21.15.113:0/3265621451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341415ba50 msgr2=0x7f341415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419948640 1 --2- 172.21.15.113:0/3265621451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341415ba50 0x7f341415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419948640 1 -- 172.21.15.113:0/3265621451 >> v1:172.21.15.178:6789/0 conn(0x7f341415b670 legacy=0x7f34140cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3419948640 1 -- 172.21.15.113:0/3265621451 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3414071730 con 0x7f341406cea0 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.007+0000 7f3407fff640 1 -- 172.21.15.113:0/3265621451 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735810282 0 0) 0x7f3414071170 con 0x7f341415b670 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.008+0000 7f3419948640 1 --2- 172.21.15.113:0/3265621451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341406cea0 0x7f341406d2d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f340c009d90 tx=0x7f340c02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc788ad849e80ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.008+0000 7f3407fff640 1 -- 172.21.15.113:0/3265621451 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c007740 con 0x7f341406cea0 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.008+0000 7f3407fff640 1 -- 172.21.15.113:0/3265621451 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c0078e0 con 0x7f341406cea0 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.008+0000 7f3407fff640 1 -- 172.21.15.113:0/3265621451 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c007cb0 con 0x7f341406cea0 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 -- 172.21.15.113:0/3265621451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341406cea0 msgr2=0x7f341406d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 --2- 172.21.15.113:0/3265621451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341406cea0 0x7f341406d2d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f340c009d90 tx=0x7f340c02a530 comp rx=0 tx=0).stop 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 -- 172.21.15.113:0/3265621451 shutdown_connections 2024-04-25T00:04:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 --2- 172.21.15.113:0/3265621451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341406cea0 0x7f341406d2d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 --2- 172.21.15.113:0/3265621451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341415ba50 0x7f341415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 -- 172.21.15.113:0/3265621451 >> 172.21.15.113:0/3265621451 conn(0x7f341406b0f0 msgr2=0x7f341406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 -- 172.21.15.113:0/3265621451 shutdown_connections 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.009+0000 7f341b3ab640 1 -- 172.21.15.113:0/3265621451 wait complete. 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.010+0000 7f341b3ab640 1 Processor -- start 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.010+0000 7f341b3ab640 1 -- start start 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f341b3ab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341406cea0 0x7f34141578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f341b3ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f341415b670 0x7f34140aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f341b3ab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341415ba50 0x7f34140aa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f341b3ab640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f34140cca90 con 0x7f341415b670 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f341b3ab640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34140cc910 con 0x7f341415ba50 2024-04-25T00:04:01.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f341b3ab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f34140d0080 con 0x7f341406cea0 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f341415b670 0x7f34140aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f341415b670 0x7f34140aa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50570/0 (socket says 172.21.15.113:50570) 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 -- 172.21.15.113:0/2873780594 learned_addr learned my addr 172.21.15.113:0/2873780594 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3419948640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341415ba50 0x7f34140aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3419147640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341406cea0 0x7f34141578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 -- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341406cea0 msgr2=0x7f34141578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341406cea0 0x7f34141578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 -- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341415ba50 msgr2=0x7f34140aa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341415ba50 0x7f34140aa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 -- 172.21.15.113:0/2873780594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34141f3f70 con 0x7f341415b670 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3419948640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341415ba50 0x7f34140aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3418946640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f341415b670 0x7f34140aa250 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f341000e870 tx=0x7f341000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3405ffb640 1 -- 172.21.15.113:0/2873780594 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410017070 con 0x7f341415b670 2024-04-25T00:04:01.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3405ffb640 1 -- 172.21.15.113:0/2873780594 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341000c940 con 0x7f341415b670 2024-04-25T00:04:01.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34141f4250 con 0x7f341415b670 2024-04-25T00:04:01.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.011+0000 7f3405ffb640 1 -- 172.21.15.113:0/2873780594 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341000cd30 con 0x7f341415b670 2024-04-25T00:04:01.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.012+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34141f4d30 con 0x7f341415b670 2024-04-25T00:04:01.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.012+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34140691a0 con 0x7f341415b670 2024-04-25T00:04:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.013+0000 7f3405ffb640 1 -- 172.21.15.113:0/2873780594 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f341001a070 con 0x7f341415b670 2024-04-25T00:04:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.014+0000 7f3405ffb640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33f806be30 0x7f33f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.014+0000 7f3419147640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33f806be30 0x7f33f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.014+0000 7f3405ffb640 1 -- 172.21.15.113:0/2873780594 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4943+0+0 (secure 0 0 0) 0x7f341008ad40 con 0x7f341415b670 2024-04-25T00:04:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.014+0000 7f3419147640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33f806be30 0x7f33f806e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3408000f60 tx=0x7f3408007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.018+0000 7f3405ffb640 1 -- 172.21.15.113:0/2873780594 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3410059a80 con 0x7f341415b670 2024-04-25T00:04:01.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.191+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f34140cf2c0 con 0x7f341415b670 2024-04-25T00:04:01.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.194+0000 7f3405ffb640 1 -- 172.21.15.113:0/2873780594 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f3410059380 con 0x7f341415b670 2024-04-25T00:04:01.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33f806be30 msgr2=0x7f33f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33f806be30 0x7f33f806e2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3408000f60 tx=0x7f3408007b50 comp rx=0 tx=0).stop 2024-04-25T00:04:01.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f341415b670 msgr2=0x7f34140aa250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f341415b670 0x7f34140aa250 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f341000e870 tx=0x7f341000ed30 comp rx=0 tx=0).stop 2024-04-25T00:04:01.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 shutdown_connections 2024-04-25T00:04:01.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33f806be30 0x7f33f806e2f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341415ba50 0x7f34140aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f341415b670 0x7f34140aa250 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 --2- 172.21.15.113:0/2873780594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f341406cea0 0x7f34141578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 >> 172.21.15.113:0/2873780594 conn(0x7f341406b0f0 msgr2=0x7f341415d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 shutdown_connections 2024-04-25T00:04:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.196+0000 7f341b3ab640 1 -- 172.21.15.113:0/2873780594 wait complete. 2024-04-25T00:04:01.207 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:04:01.208 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:04:01.250 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:04:01.300 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:04:01.300 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:01.300 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:01.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:04:01.316 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:04:01.316 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:01.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:04:01.333 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:01.333 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:01.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:04:01.372 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:04:01.372 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:01.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:04:01.391 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:04:01.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:01.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.508+0000 7f0aa3528640 1 Processor -- start 2024-04-25T00:04:01.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.509+0000 7f0aa3528640 1 -- start start 2024-04-25T00:04:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.509+0000 7f0aa3528640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a9c144910 0x7f0a9c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.509+0000 7f0aa3528640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b56b0 0x7f0a9c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.509+0000 7f0aa3528640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9c144cf0 con 0x7f0a9c0b5240 2024-04-25T00:04:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.509+0000 7f0aa3528640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9c0b4510 con 0x7f0a9c144910 2024-04-25T00:04:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.509+0000 7f0aa3528640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9c0b2f30 con 0x7f0a9c0b56b0 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa12c4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a9c144910 0x7f0a9c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa0ac3640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0a9c0b5240 0x7f0a9c06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56488/0 (socket says 172.21.15.113:56488) 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa0ac3640 1 -- 172.21.15.113:0/4186795385 learned_addr learned my addr 172.21.15.113:0/4186795385 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa1ac5640 1 --2- 172.21.15.113:0/4186795385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b56b0 0x7f0a9c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa12c4640 1 -- 172.21.15.113:0/4186795385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b56b0 msgr2=0x7f0a9c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa12c4640 1 --2- 172.21.15.113:0/4186795385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b56b0 0x7f0a9c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa12c4640 1 -- 172.21.15.113:0/4186795385 >> v1:172.21.15.113:6789/0 conn(0x7f0a9c0b5240 legacy=0x7f0a9c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0a93fff640 1 -- 172.21.15.113:0/4186795385 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157614579 0 0) 0x7f0a9c144cf0 con 0x7f0a9c0b5240 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa12c4640 1 -- 172.21.15.113:0/4186795385 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c0b3210 con 0x7f0a9c144910 2024-04-25T00:04:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.510+0000 7f0aa1ac5640 1 --2- 172.21.15.113:0/4186795385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b56b0 0x7f0a9c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa12c4640 1 --2- 172.21.15.113:0/4186795385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c0b3fd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0097d0 tx=0x7f0a8c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f559122f829d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0a93fff640 1 -- 172.21.15.113:0/4186795385 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c037070 con 0x7f0a9c144910 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0a93fff640 1 -- 172.21.15.113:0/4186795385 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8c032440 con 0x7f0a9c144910 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0a93fff640 1 -- 172.21.15.113:0/4186795385 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c032810 con 0x7f0a9c144910 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 -- 172.21.15.113:0/4186795385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 msgr2=0x7f0a9c0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 --2- 172.21.15.113:0/4186795385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c0b3fd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0097d0 tx=0x7f0a8c02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 -- 172.21.15.113:0/4186795385 shutdown_connections 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 --2- 172.21.15.113:0/4186795385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b56b0 0x7f0a9c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 --2- 172.21.15.113:0/4186795385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c0b3fd0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 -- 172.21.15.113:0/4186795385 >> 172.21.15.113:0/4186795385 conn(0x7f0a9c0b1a50 msgr2=0x7f0a9c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 -- 172.21.15.113:0/4186795385 shutdown_connections 2024-04-25T00:04:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.511+0000 7f0aa3528640 1 -- 172.21.15.113:0/4186795385 wait complete. 2024-04-25T00:04:01.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.512+0000 7f0aa3528640 1 Processor -- start 2024-04-25T00:04:01.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa3528640 1 -- start start 2024-04-25T00:04:01.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa3528640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b5240 0x7f0a9c1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa3528640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a9c0b56b0 0x7f0a9c1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa3528640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa3528640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c0b4da0 con 0x7f0a9c144910 2024-04-25T00:04:01.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa3528640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c0b37d0 con 0x7f0a9c0b56b0 2024-04-25T00:04:01.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa3528640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9c0b4f20 con 0x7f0a9c0b5240 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa1ac5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa1ac5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50596/0 (socket says 172.21.15.113:50596) 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa1ac5640 1 -- 172.21.15.113:0/3466750590 learned_addr learned my addr 172.21.15.113:0/3466750590 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa12c4640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b5240 0x7f0a9c1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.513+0000 7f0aa0ac3640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a9c0b56b0 0x7f0a9c1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa1ac5640 1 -- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b5240 msgr2=0x7f0a9c1f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa1ac5640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b5240 0x7f0a9c1f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa1ac5640 1 -- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a9c0b56b0 msgr2=0x7f0a9c1406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa1ac5640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a9c0b56b0 0x7f0a9c1406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa1ac5640 1 -- 172.21.15.113:0/3466750590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c1f4b40 con 0x7f0a9c144910 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa1ac5640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c150600 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400cdd0 tx=0x7f0a9400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0a91ffb640 1 -- 172.21.15.113:0/3466750590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9401a070 con 0x7f0a9c144910 2024-04-25T00:04:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0a91ffb640 1 -- 172.21.15.113:0/3466750590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9400d840 con 0x7f0a9c144910 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c1f7760 con 0x7f0a9c144910 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0a91ffb640 1 -- 172.21.15.113:0/3466750590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400dc70 con 0x7f0a9c144910 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.514+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c1f7d20 con 0x7f0a9c144910 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.515+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9c0691a0 con 0x7f0a9c144910 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.515+0000 7f0a91ffb640 1 -- 172.21.15.113:0/3466750590 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a9401d070 con 0x7f0a9c144910 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.516+0000 7f0a91ffb640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a8006be30 0x7f0a8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.516+0000 7f0a91ffb640 1 -- 172.21.15.113:0/3466750590 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4943+0+0 (secure 0 0 0) 0x7f0a9408b260 con 0x7f0a9c144910 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.516+0000 7f0aa12c4640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a8006be30 0x7f0a8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.516+0000 7f0aa12c4640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a8006be30 0x7f0a8006e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c034000 tx=0x7f0a8c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.520+0000 7f0a91ffb640 1 -- 172.21.15.113:0/3466750590 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a940598c0 con 0x7f0a9c144910 2024-04-25T00:04:01.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.729+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a9c0cf920 con 0x7f0a9c144910 2024-04-25T00:04:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.730+0000 7f0a91ffb640 1 -- 172.21.15.113:0/3466750590 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3878 (secure 0 0 0) 0x7f0a940600e0 con 0x7f0a9c144910 2024-04-25T00:04:01.730 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:01.730 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4243948985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4243948985},{"type":"v1","addr":"172.21.15.178:6835","nonce":4243948985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/665506344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":665506344},{"type":"v1","addr":"172.21.15.178:6834","nonce":665506344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4064308986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4064308986},{"type":"v1","addr":"172.21.15.113:6836","nonce":4064308986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3743847466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3743847466},{"type":"v1","addr":"172.21.15.113:6837","nonce":3743847466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[]} 2024-04-25T00:04:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 46 2024-04-25T00:04:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a8006be30 msgr2=0x7f0a8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a8006be30 0x7f0a8006e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c034000 tx=0x7f0a8c02e000 comp rx=0 tx=0).stop 2024-04-25T00:04:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 msgr2=0x7f0a9c150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c150600 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400cdd0 tx=0x7f0a9400a710 comp rx=0 tx=0).stop 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 shutdown_connections 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a8006be30 0x7f0a8006e2f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a9c0b56b0 0x7f0a9c1406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a9c144910 0x7f0a9c150600 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 --2- 172.21.15.113:0/3466750590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a9c0b5240 0x7f0a9c1f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 >> 172.21.15.113:0/3466750590 conn(0x7f0a9c0b1a50 msgr2=0x7f0a9c0b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 shutdown_connections 2024-04-25T00:04:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.732+0000 7f0aa3528640 1 -- 172.21.15.113:0/3466750590 wait complete. 2024-04-25T00:04:01.744 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:04:01.745 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:04:01.745 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:04:01.745 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:01.745 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:04:01.745 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:04:01.745 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:04:01.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:01.746 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:04:01.746 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:04:01.746 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:04:01.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:01.746 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:04:01.747 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:04:01.747 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:04:01.747 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:01.753 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:04:01.754 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:04:01.755 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:04:01.755 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:04:01.758 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:04:01.758 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:04:01.759 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:04:01.759 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:04:01.797 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:04:01.798 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:01.798 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:04:01.799 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:01.800 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:04:01.801 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:01.801 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:04:01.801 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:01.801 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:04:01.870 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:04:01.870 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:04:01.878 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:04:01.878 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:04:01.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.905+0000 7fe682934640 1 Processor -- start 2024-04-25T00:04:01.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.905+0000 7fe682934640 1 -- start start 2024-04-25T00:04:01.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe682934640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe67c15e510 0x7fe67c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe682934640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe67c0d2e00 con 0x7fe67c0b63b0 2024-04-25T00:04:01.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe682934640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe67c0d30e0 con 0x7fe67c0d28c0 2024-04-25T00:04:01.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe682934640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe67c0d33c0 con 0x7fe67c15e510 2024-04-25T00:04:01.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe680ed1640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe67c0b63b0 0x7fe67c0b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56524/0 (socket says 172.21.15.113:56524) 2024-04-25T00:04:01.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe680ed1640 1 -- 172.21.15.113:0/2509308074 learned_addr learned my addr 172.21.15.113:0/2509308074 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:01.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe67bfff640 1 --2- 172.21.15.113:0/2509308074 >> v2:172.21.15.178:3300/0 conn(0x7fe67c15e510 0x7fe67c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe67bfff640 1 -- 172.21.15.113:0/2509308074 >> v1:172.21.15.178:6789/0 conn(0x7fe67c0d28c0 legacy=0x7fe67c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe67bfff640 1 -- 172.21.15.113:0/2509308074 >> v1:172.21.15.113:6789/0 conn(0x7fe67c0b63b0 legacy=0x7fe67c0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.906+0000 7fe67bfff640 1 -- 172.21.15.113:0/2509308074 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c0d36a0 con 0x7fe67c15e510 2024-04-25T00:04:01.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.907+0000 7fe67bfff640 1 --2- 172.21.15.113:0/2509308074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c15e510 0x7fe67c0d2380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0097a0 tx=0x7fe66c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e705191063690da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.907+0000 7fe67affd640 1 -- 172.21.15.113:0/2509308074 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c037070 con 0x7fe67c15e510 2024-04-25T00:04:01.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.907+0000 7fe67affd640 1 -- 172.21.15.113:0/2509308074 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66c032440 con 0x7fe67c15e510 2024-04-25T00:04:01.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.907+0000 7fe67affd640 1 -- 172.21.15.113:0/2509308074 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c032810 con 0x7fe67c15e510 2024-04-25T00:04:01.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.908+0000 7fe682934640 1 -- 172.21.15.113:0/2509308074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c15e510 msgr2=0x7fe67c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.908+0000 7fe682934640 1 --2- 172.21.15.113:0/2509308074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c15e510 0x7fe67c0d2380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0097a0 tx=0x7fe66c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.908+0000 7fe682934640 1 -- 172.21.15.113:0/2509308074 shutdown_connections 2024-04-25T00:04:01.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.908+0000 7fe682934640 1 --2- 172.21.15.113:0/2509308074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c15e510 0x7fe67c0d2380 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.908+0000 7fe682934640 1 -- 172.21.15.113:0/2509308074 >> 172.21.15.113:0/2509308074 conn(0x7fe67c06b720 msgr2=0x7fe67c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:01.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.908+0000 7fe682934640 1 -- 172.21.15.113:0/2509308074 shutdown_connections 2024-04-25T00:04:01.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.908+0000 7fe682934640 1 -- 172.21.15.113:0/2509308074 wait complete. 2024-04-25T00:04:01.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.909+0000 7fe682934640 1 Processor -- start 2024-04-25T00:04:01.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.909+0000 7fe682934640 1 -- start start 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe682934640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67c0b63b0 0x7fe67c1f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe682934640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c0d28c0 0x7fe67c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe682934640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe67c15e510 0x7fe67c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe682934640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0b9dd0 con 0x7fe67c0b63b0 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe682934640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0d3ea0 con 0x7fe67c0d28c0 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe682934640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe67c0b7ad0 con 0x7fe67c15e510 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67c0b63b0 0x7fe67c1f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67c0b63b0 0x7fe67c1f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50622/0 (socket says 172.21.15.113:50622) 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 -- 172.21.15.113:0/3118427630 learned_addr learned my addr 172.21.15.113:0/3118427630 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe680ed1640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe67c15e510 0x7fe67c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67b7fe640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c0d28c0 0x7fe67c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 -- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe67c15e510 msgr2=0x7fe67c15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe67c15e510 0x7fe67c15b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 -- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c0d28c0 msgr2=0x7fe67c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:01.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c0d28c0 0x7fe67c15b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:01.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67bfff640 1 -- 172.21.15.113:0/3118427630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c1f7d10 con 0x7fe67c0b63b0 2024-04-25T00:04:01.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.910+0000 7fe67b7fe640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c0d28c0 0x7fe67c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:01.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.911+0000 7fe67bfff640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67c0b63b0 0x7fe67c1f77d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe66c035040 tx=0x7fe66c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.913+0000 7fe678ff9640 1 -- 172.21.15.113:0/3118427630 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c037040 con 0x7fe67c0b63b0 2024-04-25T00:04:01.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.913+0000 7fe678ff9640 1 -- 172.21.15.113:0/3118427630 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66c030770 con 0x7fe67c0b63b0 2024-04-25T00:04:01.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.913+0000 7fe678ff9640 1 -- 172.21.15.113:0/3118427630 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c030b40 con 0x7fe67c0b63b0 2024-04-25T00:04:01.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.913+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67c1e8b80 con 0x7fe67c0b63b0 2024-04-25T00:04:01.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.913+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67c1e6c90 con 0x7fe67c0b63b0 2024-04-25T00:04:01.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.914+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67c0691a0 con 0x7fe67c0b63b0 2024-04-25T00:04:01.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.914+0000 7fe678ff9640 1 -- 172.21.15.113:0/3118427630 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe66c044020 con 0x7fe67c0b63b0 2024-04-25T00:04:01.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.915+0000 7fe678ff9640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66806be80 0x7fe66806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:01.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.915+0000 7fe678ff9640 1 -- 172.21.15.113:0/3118427630 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4943+0+0 (secure 0 0 0) 0x7fe66c0aaf20 con 0x7fe67c0b63b0 2024-04-25T00:04:01.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.915+0000 7fe67b7fe640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66806be80 0x7fe66806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:01.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.915+0000 7fe67b7fe640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66806be80 0x7fe66806e340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0b61d0 tx=0x7fe670009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:01.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:01.919+0000 7fe678ff9640 1 -- 172.21.15.113:0/3118427630 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe66c032e20 con 0x7fe67c0b63b0 2024-04-25T00:04:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.104+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe67c0683e0 con 0x7fe67c0b63b0 2024-04-25T00:04:02.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.921+0000 7fe678ff9640 1 -- 172.21.15.113:0/3118427630 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7fe66c079d40 con 0x7fe67c0b63b0 2024-04-25T00:04:02.921 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:04:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66806be80 msgr2=0x7fe66806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66806be80 0x7fe66806e340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0b61d0 tx=0x7fe670009040 comp rx=0 tx=0).stop 2024-04-25T00:04:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67c0b63b0 msgr2=0x7fe67c1f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67c0b63b0 0x7fe67c1f77d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe66c035040 tx=0x7fe66c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 shutdown_connections 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66806be80 0x7fe66806e340 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe67c0d28c0 0x7fe67c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe67c15e510 0x7fe67c15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 --2- 172.21.15.113:0/3118427630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67c0b63b0 0x7fe67c1f77d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 >> 172.21.15.113:0/3118427630 conn(0x7fe67c06b720 msgr2=0x7fe67c071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 shutdown_connections 2024-04-25T00:04:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:02.925+0000 7fe682934640 1 -- 172.21.15.113:0/3118427630 wait complete. 2024-04-25T00:04:02.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:04:03.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.040+0000 7f4466f5a640 1 Processor -- start 2024-04-25T00:04:03.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.040+0000 7f4466f5a640 1 -- start start 2024-04-25T00:04:03.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f4466f5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4460144910 0x7f44600b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f4466f5a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44600b56b0 0x7f44600cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f4466f5a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4460144cf0 con 0x7f44600b5240 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f4466f5a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f44600b4510 con 0x7f44600b56b0 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f4466f5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f44600b2f30 con 0x7f4460144910 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f44654f7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44600b56b0 0x7f44600cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f44654f7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44600b56b0 0x7f44600cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50628/0 (socket says 172.21.15.113:50628) 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f44654f7640 1 -- 172.21.15.113:0/948608782 learned_addr learned my addr 172.21.15.113:0/948608782 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f44654f7640 1 -- 172.21.15.113:0/948608782 >> v2:172.21.15.178:3300/0 conn(0x7f4460144910 msgr2=0x7f44600b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f44654f7640 1 --2- 172.21.15.113:0/948608782 >> v2:172.21.15.178:3300/0 conn(0x7f4460144910 0x7f44600b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f44654f7640 1 -- 172.21.15.113:0/948608782 >> v1:172.21.15.178:6789/0 conn(0x7f44600b5240 legacy=0x7f446006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.041+0000 7f44654f7640 1 -- 172.21.15.113:0/948608782 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44600b3210 con 0x7f44600b56b0 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.042+0000 7f44654f7640 1 --2- 172.21.15.113:0/948608782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b56b0 0x7f44600cd500 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f445800d150 tx=0x7f445802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ce30a3e48062aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.042+0000 7f445f7fe640 1 -- 172.21.15.113:0/948608782 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458009600 con 0x7f44600b56b0 2024-04-25T00:04:03.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.042+0000 7f445f7fe640 1 -- 172.21.15.113:0/948608782 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445802f070 con 0x7f44600b56b0 2024-04-25T00:04:03.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.042+0000 7f445f7fe640 1 -- 172.21.15.113:0/948608782 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44580099c0 con 0x7f44600b56b0 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 -- 172.21.15.113:0/948608782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b56b0 msgr2=0x7f44600cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 --2- 172.21.15.113:0/948608782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b56b0 0x7f44600cd500 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f445800d150 tx=0x7f445802c930 comp rx=0 tx=0).stop 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 -- 172.21.15.113:0/948608782 shutdown_connections 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 --2- 172.21.15.113:0/948608782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b56b0 0x7f44600cd500 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 --2- 172.21.15.113:0/948608782 >> v2:172.21.15.178:3300/0 conn(0x7f4460144910 0x7f44600b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 -- 172.21.15.113:0/948608782 >> 172.21.15.113:0/948608782 conn(0x7f44600b1a50 msgr2=0x7f44600b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 -- 172.21.15.113:0/948608782 shutdown_connections 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.043+0000 7f4466f5a640 1 -- 172.21.15.113:0/948608782 wait complete. 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.044+0000 7f4466f5a640 1 Processor -- start 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.044+0000 7f4466f5a640 1 -- start start 2024-04-25T00:04:03.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4466f5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b5240 0x7f44601f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4466f5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44600b56b0 0x7f44601f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4466f5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4460144910 0x7f446013aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4466f5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44600b4da0 con 0x7f44600b5240 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4466f5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44600b37d0 con 0x7f44600b56b0 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4466f5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44600b4f20 con 0x7f4460144910 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b5240 0x7f44601f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b5240 0x7f44601f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50640/0 (socket says 172.21.15.113:50640) 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 -- 172.21.15.113:0/3922413470 learned_addr learned my addr 172.21.15.113:0/3922413470 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f445ffff640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44600b56b0 0x7f44601f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f44654f7640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4460144910 0x7f446013aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 -- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4460144910 msgr2=0x7f446013aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4460144910 0x7f446013aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 -- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44600b56b0 msgr2=0x7f44601f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44600b56b0 0x7f44601f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 -- 172.21.15.113:0/3922413470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44601f5ae0 con 0x7f44600b5240 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f44654f7640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4460144910 0x7f446013aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f445ffff640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44600b56b0 0x7f44601f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.045+0000 7f4464cf6640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b5240 0x7f44601f7340 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f445000b410 tx=0x7f445000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.046+0000 7f445d7fa640 1 -- 172.21.15.113:0/3922413470 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450015020 con 0x7f44600b5240 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.046+0000 7f445d7fa640 1 -- 172.21.15.113:0/3922413470 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4450011850 con 0x7f44600b5240 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.046+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44600cd830 con 0x7f44600b5240 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.046+0000 7f445d7fa640 1 -- 172.21.15.113:0/3922413470 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450011c40 con 0x7f44600b5240 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.046+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44600cddf0 con 0x7f44600b5240 2024-04-25T00:04:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.047+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44600691a0 con 0x7f44600b5240 2024-04-25T00:04:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.047+0000 7f445d7fa640 1 -- 172.21.15.113:0/3922413470 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4450018020 con 0x7f44600b5240 2024-04-25T00:04:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.047+0000 7f445d7fa640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f444c06be30 0x7f444c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.048+0000 7f445d7fa640 1 -- 172.21.15.113:0/3922413470 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f445008aa70 con 0x7f44600b5240 2024-04-25T00:04:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.048+0000 7f445ffff640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f444c06be30 0x7f444c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.048+0000 7f445ffff640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f444c06be30 0x7f444c06e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4454007fd0 tx=0x7f4454009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.052+0000 7f445d7fa640 1 -- 172.21.15.113:0/3922413470 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f445001f020 con 0x7f44600b5240 2024-04-25T00:04:03.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.291+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f44600731a0 con 0x7f44600b5240 2024-04-25T00:04:03.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.292+0000 7f445d7fa640 1 -- 172.21.15.113:0/3922413470 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f4450059770 con 0x7f44600b5240 2024-04-25T00:04:03.292 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:03.292 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:03.292 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:04:03.292 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.292 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.294 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:04:03.295 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.296 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.297 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.298 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.299 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:04:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:04:03.301 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.302 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:03.303 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:04:03.304 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:04:03.305 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.293+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f444c06be30 msgr2=0x7f444c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.293+0000 7f4466f5a640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f444c06be30 0x7f444c06e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4454007fd0 tx=0x7f4454009040 comp rx=0 tx=0).stop 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b5240 msgr2=0x7f44601f7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b5240 0x7f44601f7340 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f445000b410 tx=0x7f445000b8d0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 shutdown_connections 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f444c06be30 0x7f444c06e2f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44600b56b0 0x7f44601f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4460144910 0x7f446013aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 --2- 172.21.15.113:0/3922413470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44600b5240 0x7f44601f7340 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 >> 172.21.15.113:0/3922413470 conn(0x7f44600b1a50 msgr2=0x7f4460071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 shutdown_connections 2024-04-25T00:04:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.294+0000 7f4466f5a640 1 -- 172.21.15.113:0/3922413470 wait complete. 2024-04-25T00:04:03.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:03.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.404+0000 7f24c091b640 1 Processor -- start 2024-04-25T00:04:03.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.404+0000 7f24c091b640 1 -- start start 2024-04-25T00:04:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24c091b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24bc06e6e0 0x7f24bc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24c091b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc1453c0 0x7f24bc071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24c091b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24bc0cfad0 con 0x7f24bc144d60 2024-04-25T00:04:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24c091b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f24bc145c40 con 0x7f24bc06e6e0 2024-04-25T00:04:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24c091b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc0cf230 con 0x7f24bc1453c0 2024-04-25T00:04:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24b9d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24bc06e6e0 0x7f24bc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24b9d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24bc06e6e0 0x7f24bc0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50648/0 (socket says 172.21.15.113:50648) 2024-04-25T00:04:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.405+0000 7f24b9d9b640 1 -- 172.21.15.113:0/1224754026 learned_addr learned my addr 172.21.15.113:0/1224754026 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.406+0000 7f24bad9d640 1 --2- 172.21.15.113:0/1224754026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc1453c0 0x7f24bc071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.406+0000 7f24b9d9b640 1 -- 172.21.15.113:0/1224754026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc1453c0 msgr2=0x7f24bc071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.406+0000 7f24b9d9b640 1 --2- 172.21.15.113:0/1224754026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc1453c0 0x7f24bc071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.406+0000 7f24b9d9b640 1 -- 172.21.15.113:0/1224754026 >> v1:172.21.15.178:6789/0 conn(0x7f24bc144d60 legacy=0x7f24bc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.406+0000 7f24b9d9b640 1 -- 172.21.15.113:0/1224754026 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc0cf510 con 0x7f24bc06e6e0 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.406+0000 7f24b9d9b640 1 --2- 172.21.15.113:0/1224754026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc06e6e0 0x7f24bc0b5390 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009870 tx=0x7f24b002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd24e4472da19b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24b959a640 1 -- 172.21.15.113:0/1224754026 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0037070 con 0x7f24bc06e6e0 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24b959a640 1 -- 172.21.15.113:0/1224754026 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b002ae10 con 0x7f24bc06e6e0 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24b959a640 1 -- 172.21.15.113:0/1224754026 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0032660 con 0x7f24bc06e6e0 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 -- 172.21.15.113:0/1224754026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc06e6e0 msgr2=0x7f24bc0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 --2- 172.21.15.113:0/1224754026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc06e6e0 0x7f24bc0b5390 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009870 tx=0x7f24b002a100 comp rx=0 tx=0).stop 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 -- 172.21.15.113:0/1224754026 shutdown_connections 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 --2- 172.21.15.113:0/1224754026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc1453c0 0x7f24bc071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 --2- 172.21.15.113:0/1224754026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc06e6e0 0x7f24bc0b5390 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 -- 172.21.15.113:0/1224754026 >> 172.21.15.113:0/1224754026 conn(0x7f24bc0b1a50 msgr2=0x7f24bc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:03.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 -- 172.21.15.113:0/1224754026 shutdown_connections 2024-04-25T00:04:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.407+0000 7f24c091b640 1 -- 172.21.15.113:0/1224754026 wait complete. 2024-04-25T00:04:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.408+0000 7f24c091b640 1 Processor -- start 2024-04-25T00:04:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24c091b640 1 -- start start 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24c091b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc06e6e0 0x7f24bc0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24c091b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc144d60 0x7f24bc15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24c091b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc1453c0 0x7f24bc15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24c091b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc145140 con 0x7f24bc1453c0 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24c091b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc070e10 con 0x7f24bc06e6e0 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24c091b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc071870 con 0x7f24bc144d60 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24bad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc1453c0 0x7f24bc15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24bad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc1453c0 0x7f24bc15d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50650/0 (socket says 172.21.15.113:50650) 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.409+0000 7f24bad9d640 1 -- 172.21.15.113:0/1866895326 learned_addr learned my addr 172.21.15.113:0/1866895326 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:03.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24b9d9b640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc144d60 0x7f24bc15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24ba59c640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc06e6e0 0x7f24bc0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24bad9d640 1 -- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc144d60 msgr2=0x7f24bc15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24bad9d640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc144d60 0x7f24bc15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24bad9d640 1 -- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc06e6e0 msgr2=0x7f24bc0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24bad9d640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc06e6e0 0x7f24bc0aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24bad9d640 1 -- 172.21.15.113:0/1866895326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc14c280 con 0x7f24bc1453c0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24bad9d640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc1453c0 0x7f24bc15d960 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f24b400ccb0 tx=0x7f24b400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24aaffd640 1 -- 172.21.15.113:0/1866895326 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b400ce60 con 0x7f24bc1453c0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24aaffd640 1 -- 172.21.15.113:0/1866895326 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b4003990 con 0x7f24bc1453c0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc14c560 con 0x7f24bc1453c0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24aaffd640 1 -- 172.21.15.113:0/1866895326 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003d60 con 0x7f24bc1453c0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.410+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc14cb20 con 0x7f24bc1453c0 2024-04-25T00:04:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.411+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24bc0691a0 con 0x7f24bc1453c0 2024-04-25T00:04:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.411+0000 7f24aaffd640 1 -- 172.21.15.113:0/1866895326 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24b4003fc0 con 0x7f24bc1453c0 2024-04-25T00:04:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.412+0000 7f24aaffd640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.412+0000 7f24aaffd640 1 -- 172.21.15.113:0/1866895326 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f24b408aee0 con 0x7f24bc1453c0 2024-04-25T00:04:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.412+0000 7f24ba59c640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.412+0000 7f24ba59c640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f249c06be30 0x7f249c06e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f24bc1f5480 tx=0x7f24ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.416+0000 7f24aaffd640 1 -- 172.21.15.113:0/1866895326 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24b4014030 con 0x7f24bc1453c0 2024-04-25T00:04:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.589+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24bc0afac0 con 0x7f24bc1453c0 2024-04-25T00:04:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.589+0000 7f24aaffd640 1 -- 172.21.15.113:0/1866895326 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+12940 (secure 0 0 0) 0x7f24b4059c50 con 0x7f24bc1453c0 2024-04-25T00:04:03.590 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:03.590 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:02.920299+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f249c06be30 msgr2=0x7f249c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f249c06be30 0x7f249c06e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f24bc1f5480 tx=0x7f24ac009040 comp rx=0 tx=0).stop 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc1453c0 msgr2=0x7f24bc15d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc1453c0 0x7f24bc15d960 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f24b400ccb0 tx=0x7f24b400a5c0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 shutdown_connections 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24bc1453c0 0x7f24bc15d960 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24bc144d60 0x7f24bc15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 --2- 172.21.15.113:0/1866895326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc06e6e0 0x7f24bc0aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 >> 172.21.15.113:0/1866895326 conn(0x7f24bc0b1a50 msgr2=0x7f24bc071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:03.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.591+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 shutdown_connections 2024-04-25T00:04:03.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.592+0000 7f24c091b640 1 -- 172.21.15.113:0/1866895326 wait complete. 2024-04-25T00:04:03.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:04:03.603 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:04:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.703+0000 7fa16bcc3640 1 Processor -- start 2024-04-25T00:04:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.704+0000 7fa16bcc3640 1 -- start start 2024-04-25T00:04:03.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.704+0000 7fa16bcc3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa1640b5ca0 0x7fa1640b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.704+0000 7fa16bcc3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 0x7fa1640b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.704+0000 7fa16bcc3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1640d2e00 con 0x7fa16415e510 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.704+0000 7fa16bcc3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa1640b6640 con 0x7fa1640b5ca0 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.704+0000 7fa16bcc3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1640d30e0 con 0x7fa1640b73c0 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa169a5f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa16415e510 0x7fa1640d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56534/0 (socket says 172.21.15.113:56534) 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa169a5f640 1 -- 172.21.15.113:0/2295228685 learned_addr learned my addr 172.21.15.113:0/2295228685 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3211510350 0 0) 0x7fa1640d2e00 con 0x7fa16415e510 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa150003520 con 0x7fa16415e510 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa16925e640 1 --2- 172.21.15.113:0/2295228685 >> v2:172.21.15.178:3300/0 conn(0x7fa1640b5ca0 0x7fa1640b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa16a260640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 0x7fa1640b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3031734605 0 0) 0x7fa150003520 con 0x7fa16415e510 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 msgr2=0x7fa1640b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 0x7fa1640b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 >> v2:172.21.15.178:3300/0 conn(0x7fa1640b5ca0 msgr2=0x7fa1640b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 --2- 172.21.15.113:0/2295228685 >> v2:172.21.15.178:3300/0 conn(0x7fa1640b5ca0 0x7fa1640b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1640d33c0 con 0x7fa16415e510 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fa160001670 con 0x7fa16415e510 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 >> v1:172.21.15.113:6789/0 conn(0x7fa16415e510 legacy=0x7fa1640d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 0x7fa1500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa150003fe0 0x7fa150004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150004d40 0x7fa1500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1640d30e0 con 0x7fa150003fe0 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa15000bb00 con 0x7fa150004d40 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa15000bd00 con 0x7fa1640b73c0 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa16925e640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa150003fe0 0x7fa150004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa169a5f640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 0x7fa1500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.705+0000 7fa16a260640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150004d40 0x7fa1500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa16925e640 1 -- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 msgr2=0x7fa1500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa16925e640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 0x7fa1500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa16925e640 1 -- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150004d40 msgr2=0x7fa1500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa16925e640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150004d40 0x7fa1500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa16925e640 1 -- 172.21.15.113:0/2295228685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa150003520 con 0x7fa150003fe0 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa169a5f640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b73c0 0x7fa1500039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa16a260640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150004d40 0x7fa1500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa16925e640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa150003fe0 0x7fa150004800 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa15400ba30 tx=0x7fa150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec40e4ff7ffd6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154030070 con 0x7fa150003fe0 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa15402ae10 con 0x7fa150003fe0 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.706+0000 7fa168a5d640 1 -- 172.21.15.113:0/2295228685 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15400b600 con 0x7fa150003fe0 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.707+0000 7fa16bcc3640 1 -- 172.21.15.113:0/2295228685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa150003fe0 msgr2=0x7fa150004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.707+0000 7fa16bcc3640 1 --2- 172.21.15.113:0/2295228685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa150003fe0 0x7fa150004800 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa15400ba30 tx=0x7fa150005830 comp rx=0 tx=0).stop 2024-04-25T00:04:03.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.707+0000 7fa16a260640 1 -- 172.21.15.113:0/2295228685 reap_dead start 2024-04-25T00:04:03.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.707+0000 7fa16bcc3640 1 -- 172.21.15.113:0/2295228685 shutdown_connections 2024-04-25T00:04:03.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.707+0000 7fa16bcc3640 1 -- 172.21.15.113:0/2295228685 >> 172.21.15.113:0/2295228685 conn(0x7fa16406b720 msgr2=0x7fa1640d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:03.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.707+0000 7fa16bcc3640 1 -- 172.21.15.113:0/2295228685 shutdown_connections 2024-04-25T00:04:03.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.707+0000 7fa16bcc3640 1 -- 172.21.15.113:0/2295228685 wait complete. 2024-04-25T00:04:03.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.708+0000 7fa16bcc3640 1 Processor -- start 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.708+0000 7fa16bcc3640 1 -- start start 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16bcc3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150003fe0 0x7fa164161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16bcc3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1640b4f40 0x7fa1641f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16bcc3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b53c0 0x7fa1641f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16bcc3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1640d3ea0 con 0x7fa1640b4f40 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16bcc3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1640d3980 con 0x7fa150003fe0 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16bcc3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1640d2a90 con 0x7fa1640b53c0 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1640b4f40 0x7fa1641f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1640b4f40 0x7fa1641f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50672/0 (socket says 172.21.15.113:50672) 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 -- 172.21.15.113:0/3284569337 learned_addr learned my addr 172.21.15.113:0/3284569337 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 -- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b53c0 msgr2=0x7fa1641f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16a260640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b53c0 0x7fa1641f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa169a5f640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150003fe0 0x7fa164161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b53c0 0x7fa1641f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 -- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150003fe0 msgr2=0x7fa164161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150003fe0 0x7fa164161270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 -- 172.21.15.113:0/3284569337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16415b370 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.709+0000 7fa16925e640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1640b4f40 0x7fa1641f65d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa15400bb50 tx=0x7fa154004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.710+0000 7fa15a7fc640 1 -- 172.21.15.113:0/3284569337 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154042030 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.710+0000 7fa15a7fc640 1 -- 172.21.15.113:0/3284569337 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa15402ac10 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.710+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1641617b0 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.710+0000 7fa15a7fc640 1 -- 172.21.15.113:0/3284569337 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154004260 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.710+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16415b8f0 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.711+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1640691a0 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.711+0000 7fa15a7fc640 1 -- 172.21.15.113:0/3284569337 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa154045070 con 0x7fa1640b4f40 2024-04-25T00:04:03.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.711+0000 7fa15a7fc640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14806be60 0x7fa14806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:03.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.712+0000 7fa15a7fc640 1 -- 172.21.15.113:0/3284569337 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7fa1540aac40 con 0x7fa1640b4f40 2024-04-25T00:04:03.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.712+0000 7fa169a5f640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14806be60 0x7fa14806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:03.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.712+0000 7fa169a5f640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14806be60 0x7fa14806e320 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa160001050 tx=0x7fa160001e50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:03.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.716+0000 7fa15a7fc640 1 -- 172.21.15.113:0/3284569337 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1540798b0 con 0x7fa1640b4f40 2024-04-25T00:04:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.891+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa16406c100 con 0x7fa1640b4f40 2024-04-25T00:04:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.891+0000 7fa15a7fc640 1 -- 172.21.15.113:0/3284569337 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fa1540791b0 con 0x7fa1640b4f40 2024-04-25T00:04:03.892 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:04:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14806be60 msgr2=0x7fa14806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14806be60 0x7fa14806e320 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa160001050 tx=0x7fa160001e50 comp rx=0 tx=0).stop 2024-04-25T00:04:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1640b4f40 msgr2=0x7fa1641f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1640b4f40 0x7fa1641f65d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa15400bb50 tx=0x7fa154004730 comp rx=0 tx=0).stop 2024-04-25T00:04:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 shutdown_connections 2024-04-25T00:04:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14806be60 0x7fa14806e320 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1640b53c0 0x7fa1641f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1640b4f40 0x7fa1641f65d0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.893+0000 7fa16bcc3640 1 --2- 172.21.15.113:0/3284569337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150003fe0 0x7fa164161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.894+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 >> 172.21.15.113:0/3284569337 conn(0x7fa16406b720 msgr2=0x7fa1640cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.894+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 shutdown_connections 2024-04-25T00:04:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:03.894+0000 7fa16bcc3640 1 -- 172.21.15.113:0/3284569337 wait complete. 2024-04-25T00:04:03.905 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:04:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.002+0000 7f68333ee640 1 Processor -- start 2024-04-25T00:04:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.002+0000 7f68333ee640 1 -- start start 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f68333ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f682c144910 0x7f682c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f68333ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5240 0x7f682c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f68333ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f682c0b5680 0x7f682c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f68333ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f682c0b4510 con 0x7f682c0b5680 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f68333ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f682c144cf0 con 0x7f682c144910 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f68333ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f682c0b2f30 con 0x7f682c0b5240 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f683198b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f682c0b5680 0x7f682c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f683198b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f682c0b5680 0x7f682c06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50686/0 (socket says 172.21.15.113:50686) 2024-04-25T00:04:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f683198b640 1 -- 172.21.15.113:0/2682232309 learned_addr learned my addr 172.21.15.113:0/2682232309 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:04.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f683118a640 1 --2- 172.21.15.113:0/2682232309 >> v2:172.21.15.178:3300/0 conn(0x7f682c144910 0x7f682c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.003+0000 7f6830989640 1 --2- 172.21.15.113:0/2682232309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5240 0x7f682c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f683198b640 1 -- 172.21.15.113:0/2682232309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5240 msgr2=0x7f682c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:04.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f683198b640 1 --2- 172.21.15.113:0/2682232309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5240 0x7f682c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f683198b640 1 -- 172.21.15.113:0/2682232309 >> v2:172.21.15.178:3300/0 conn(0x7f682c144910 msgr2=0x7f682c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:04.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f683198b640 1 --2- 172.21.15.113:0/2682232309 >> v2:172.21.15.178:3300/0 conn(0x7f682c144910 0x7f682c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f683198b640 1 -- 172.21.15.113:0/2682232309 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c0b3210 con 0x7f682c0b5680 2024-04-25T00:04:04.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f683198b640 1 --2- 172.21.15.113:0/2682232309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5680 0x7f682c06e510 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f682c070af0 tx=0x7f682402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706705e5b5cde061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f6823fff640 1 -- 172.21.15.113:0/2682232309 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824032790 con 0x7f682c0b5680 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f6823fff640 1 -- 172.21.15.113:0/2682232309 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6824038070 con 0x7f682c0b5680 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.004+0000 7f6823fff640 1 -- 172.21.15.113:0/2682232309 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824032b50 con 0x7f682c0b5680 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 -- 172.21.15.113:0/2682232309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5680 msgr2=0x7f682c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 --2- 172.21.15.113:0/2682232309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5680 0x7f682c06e510 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f682c070af0 tx=0x7f682402c890 comp rx=0 tx=0).stop 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 -- 172.21.15.113:0/2682232309 shutdown_connections 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 --2- 172.21.15.113:0/2682232309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5680 0x7f682c06e510 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 --2- 172.21.15.113:0/2682232309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5240 0x7f682c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 --2- 172.21.15.113:0/2682232309 >> v2:172.21.15.178:3300/0 conn(0x7f682c144910 0x7f682c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 -- 172.21.15.113:0/2682232309 >> 172.21.15.113:0/2682232309 conn(0x7f682c0b1a50 msgr2=0x7f682c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 -- 172.21.15.113:0/2682232309 shutdown_connections 2024-04-25T00:04:04.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.005+0000 7f68333ee640 1 -- 172.21.15.113:0/2682232309 wait complete. 2024-04-25T00:04:04.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.006+0000 7f68333ee640 1 Processor -- start 2024-04-25T00:04:04.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.006+0000 7f68333ee640 1 -- start start 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f68333ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5240 0x7f682c0ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f68333ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5680 0x7f682c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f68333ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f682c144910 0x7f682c142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f68333ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f682c0b4da0 con 0x7f682c0b5240 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f68333ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f682c0b37d0 con 0x7f682c144910 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f68333ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f682c0b4f20 con 0x7f682c0b5680 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5240 0x7f682c0ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5240 0x7f682c0ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50694/0 (socket says 172.21.15.113:50694) 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 -- 172.21.15.113:0/3719902050 learned_addr learned my addr 172.21.15.113:0/3719902050 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 -- 172.21.15.113:0/3719902050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5680 msgr2=0x7f682c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5680 0x7f682c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 -- 172.21.15.113:0/3719902050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f682c144910 msgr2=0x7f682c142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f682c144910 0x7f682c142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 -- 172.21.15.113:0/3719902050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c1f83f0 con 0x7f682c0b5240 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.007+0000 7f683118a640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5240 0x7f682c0ae6b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6828007ef0 tx=0x7f682800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:04.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.008+0000 7f6821ffb640 1 -- 172.21.15.113:0/3719902050 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828014620 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.008+0000 7f6821ffb640 1 -- 172.21.15.113:0/3719902050 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828019070 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.008+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c1f7870 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.008+0000 7f6821ffb640 1 -- 172.21.15.113:0/3719902050 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828014a00 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.008+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c1f7e30 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.009+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682c0691a0 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.009+0000 7f6821ffb640 1 -- 172.21.15.113:0/3719902050 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6828014ba0 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.010+0000 7f6821ffb640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f681006be30 0x7f681006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.010+0000 7f6821ffb640 1 -- 172.21.15.113:0/3719902050 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f682808b480 con 0x7f682c0b5240 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.010+0000 7f6830989640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f681006be30 0x7f681006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.010+0000 7f6830989640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f681006be30 0x7f681006e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f681c006a90 tx=0x7f681c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:04.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.014+0000 7f6821ffb640 1 -- 172.21.15.113:0/3719902050 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f682805a180 con 0x7f682c0b5240 2024-04-25T00:04:04.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.189+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f682c143fc0 con 0x7f682c0b5240 2024-04-25T00:04:04.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.189+0000 7f6821ffb640 1 -- 172.21.15.113:0/3719902050 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7f6828059a80 con 0x7f682c0b5240 2024-04-25T00:04:04.190 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:04.190 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:04:04.190 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:04.190 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:04:04.190 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:04:04.190 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:04.191 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:04:04.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f681006be30 msgr2=0x7f681006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f681006be30 0x7f681006e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f681c006a90 tx=0x7f681c008040 comp rx=0 tx=0).stop 2024-04-25T00:04:04.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5240 msgr2=0x7f682c0ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5240 0x7f682c0ae6b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6828007ef0 tx=0x7f682800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 shutdown_connections 2024-04-25T00:04:04.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f681006be30 0x7f681006e2f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f682c144910 0x7f682c142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f682c0b5680 0x7f682c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.191+0000 7f68333ee640 1 --2- 172.21.15.113:0/3719902050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f682c0b5240 0x7f682c0ae6b0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.192+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 >> 172.21.15.113:0/3719902050 conn(0x7f682c0b1a50 msgr2=0x7f682c15f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:04.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.192+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 shutdown_connections 2024-04-25T00:04:04.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.192+0000 7f68333ee640 1 -- 172.21.15.113:0/3719902050 wait complete. 2024-04-25T00:04:04.203 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:04:04.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:04:04.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.302+0000 7f55adf63640 1 Processor -- start 2024-04-25T00:04:04.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55adf63640 1 -- start start 2024-04-25T00:04:04.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55adf63640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55a815e510 0x7f55a8161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55adf63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b63b0 0x7f55a80d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55adf63640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a81626f0 con 0x7f55a80b9bb0 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55adf63640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f55a81629d0 con 0x7f55a815e510 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55adf63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f55a8162cb0 con 0x7f55a80b63b0 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55a77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55a815e510 0x7f55a8161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.303+0000 7f55a77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55a815e510 0x7f55a8161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50700/0 (socket says 172.21.15.113:50700) 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.304+0000 7f55a77fe640 1 -- 172.21.15.113:0/1529530821 learned_addr learned my addr 172.21.15.113:0/1529530821 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.304+0000 7f55a7fff640 1 --2- 172.21.15.113:0/1529530821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b63b0 0x7f55a80d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.304+0000 7f55a77fe640 1 -- 172.21.15.113:0/1529530821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b63b0 msgr2=0x7f55a80d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.304+0000 7f55a77fe640 1 --2- 172.21.15.113:0/1529530821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b63b0 0x7f55a80d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.304+0000 7f55a77fe640 1 -- 172.21.15.113:0/1529530821 >> v1:172.21.15.178:6789/0 conn(0x7f55a80b9bb0 legacy=0x7f55a80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.304+0000 7f55a77fe640 1 -- 172.21.15.113:0/1529530821 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a80d2660 con 0x7f55a815e510 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.304+0000 7f55a77fe640 1 --2- 172.21.15.113:0/1529530821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a8161ea0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5598009870 tx=0x7f559802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782d34f05f2a2289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55a67fc640 1 -- 172.21.15.113:0/1529530821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598037070 con 0x7f55a815e510 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55a67fc640 1 -- 172.21.15.113:0/1529530821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5598032440 con 0x7f55a815e510 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55a67fc640 1 -- 172.21.15.113:0/1529530821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598032810 con 0x7f55a815e510 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 -- 172.21.15.113:0/1529530821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 msgr2=0x7f55a8161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 --2- 172.21.15.113:0/1529530821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a8161ea0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5598009870 tx=0x7f559802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 -- 172.21.15.113:0/1529530821 shutdown_connections 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 --2- 172.21.15.113:0/1529530821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b63b0 0x7f55a80d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 --2- 172.21.15.113:0/1529530821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a8161ea0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 -- 172.21.15.113:0/1529530821 >> 172.21.15.113:0/1529530821 conn(0x7f55a806b720 msgr2=0x7f55a80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 -- 172.21.15.113:0/1529530821 shutdown_connections 2024-04-25T00:04:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.305+0000 7f55adf63640 1 -- 172.21.15.113:0/1529530821 wait complete. 2024-04-25T00:04:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.306+0000 7f55adf63640 1 Processor -- start 2024-04-25T00:04:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.306+0000 7f55adf63640 1 -- start start 2024-04-25T00:04:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55adf63640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55a80b63b0 0x7f55a80c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55adf63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b9bb0 0x7f55a8150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55adf63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a81612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55adf63640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f55a80b73c0 con 0x7f55a815e510 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55adf63640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81623e0 con 0x7f55a80b63b0 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55adf63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f55a80b7540 con 0x7f55a80b9bb0 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a81612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a81612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50706/0 (socket says 172.21.15.113:50706) 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55a7fff640 1 -- 172.21.15.113:0/2262016202 learned_addr learned my addr 172.21.15.113:0/2262016202 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55a77fe640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55a80b63b0 0x7f55a80c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.307+0000 7f55a6ffd640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b9bb0 0x7f55a8150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55a7fff640 1 -- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b9bb0 msgr2=0x7f55a8150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55a7fff640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b9bb0 0x7f55a8150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55a7fff640 1 -- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55a80b63b0 msgr2=0x7f55a80c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55a7fff640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55a80b63b0 0x7f55a80c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55a7fff640 1 -- 172.21.15.113:0/2262016202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a81617e0 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55a7fff640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a81612a0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f55a0007e90 tx=0x7f55a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f558ffff640 1 -- 172.21.15.113:0/2262016202 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0014420 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f558ffff640 1 -- 172.21.15.113:0/2262016202 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a0019070 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a80c9020 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f558ffff640 1 -- 172.21.15.113:0/2262016202 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0014800 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.308+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a815ea20 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.309+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a80691a0 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.309+0000 7f558ffff640 1 -- 172.21.15.113:0/2262016202 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55a00149a0 con 0x7f55a815e510 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.310+0000 7f558ffff640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f559406be30 0x7f559406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.310+0000 7f558ffff640 1 -- 172.21.15.113:0/2262016202 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f55a008b130 con 0x7f55a815e510 2024-04-25T00:04:04.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.310+0000 7f55a77fe640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f559406be30 0x7f559406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:04.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.310+0000 7f55a77fe640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f559406be30 0x7f559406e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5598034000 tx=0x7f559802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.314+0000 7f558ffff640 1 -- 172.21.15.113:0/2262016202 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f55a0059870 con 0x7f55a815e510 2024-04-25T00:04:04.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.490+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f55a80d1f30 con 0x7f55a815e510 2024-04-25T00:04:04.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.930+0000 7f558ffff640 1 -- 172.21.15.113:0/2262016202 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7f55a005db70 con 0x7f55a815e510 2024-04-25T00:04:04.930 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:04:04.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f559406be30 msgr2=0x7f559406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f559406be30 0x7f559406e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5598034000 tx=0x7f559802e000 comp rx=0 tx=0).stop 2024-04-25T00:04:04.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 msgr2=0x7f55a81612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:04.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a81612a0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f55a0007e90 tx=0x7f55a000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 shutdown_connections 2024-04-25T00:04:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f559406be30 0x7f559406e2f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a815e510 0x7f55a81612a0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55a80b9bb0 0x7f55a8150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 --2- 172.21.15.113:0/2262016202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55a80b63b0 0x7f55a80c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.934+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 >> 172.21.15.113:0/2262016202 conn(0x7f55a806b720 msgr2=0x7f55a80d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.935+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 shutdown_connections 2024-04-25T00:04:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:04.935+0000 7f55adf63640 1 -- 172.21.15.113:0/2262016202 wait complete. 2024-04-25T00:04:04.949 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:04:05.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.055+0000 7f3c95ca5640 1 Processor -- start 2024-04-25T00:04:05.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.055+0000 7f3c95ca5640 1 -- start start 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c95ca5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3c90071df0 0x7f3c900d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c95ca5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c900b1e00 0x7f3c9015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c95ca5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c9006d900 con 0x7f3c9015e310 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c95ca5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3c9015fc40 con 0x7f3c900b1e00 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c95ca5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3c9006e9a0 con 0x7f3c90071df0 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c900b1e00 0x7f3c9015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c900b1e00 0x7f3c9015e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50716/0 (socket says 172.21.15.113:50716) 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8ffff640 1 -- 172.21.15.113:0/4008398289 learned_addr learned my addr 172.21.15.113:0/4008398289 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8effd640 1 --2- 172.21.15.113:0/4008398289 >> v2:172.21.15.178:3300/0 conn(0x7f3c90071df0 0x7f3c900d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8ffff640 1 -- 172.21.15.113:0/4008398289 >> v2:172.21.15.178:3300/0 conn(0x7f3c90071df0 msgr2=0x7f3c900d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8ffff640 1 --2- 172.21.15.113:0/4008398289 >> v2:172.21.15.178:3300/0 conn(0x7f3c90071df0 0x7f3c900d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8ffff640 1 -- 172.21.15.113:0/4008398289 >> v1:172.21.15.178:6789/0 conn(0x7f3c9015e310 legacy=0x7f3c900716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.056+0000 7f3c8ffff640 1 -- 172.21.15.113:0/4008398289 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9006dbe0 con 0x7f3c900b1e00 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c8ffff640 1 --2- 172.21.15.113:0/4008398289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c900b1e00 0x7f3c9015e6f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f3c900b5be0 tx=0x7f3c8802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e788dd13f6b4e36a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c8e7fc640 1 -- 172.21.15.113:0/4008398289 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88036070 con 0x7f3c900b1e00 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c8e7fc640 1 -- 172.21.15.113:0/4008398289 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c88031600 con 0x7f3c900b1e00 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c8e7fc640 1 -- 172.21.15.113:0/4008398289 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c880319d0 con 0x7f3c900b1e00 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 -- 172.21.15.113:0/4008398289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c900b1e00 msgr2=0x7f3c9015e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/4008398289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c900b1e00 0x7f3c9015e6f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f3c900b5be0 tx=0x7f3c8802a5d0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 -- 172.21.15.113:0/4008398289 shutdown_connections 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/4008398289 >> v2:172.21.15.178:3300/0 conn(0x7f3c90071df0 0x7f3c900d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/4008398289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c900b1e00 0x7f3c9015e6f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 -- 172.21.15.113:0/4008398289 >> 172.21.15.113:0/4008398289 conn(0x7f3c9006fb60 msgr2=0x7f3c9006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 -- 172.21.15.113:0/4008398289 shutdown_connections 2024-04-25T00:04:05.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.057+0000 7f3c95ca5640 1 -- 172.21.15.113:0/4008398289 wait complete. 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.058+0000 7f3c95ca5640 1 Processor -- start 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c95ca5640 1 -- start start 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c95ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c90071df0 0x7f3c90148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c95ca5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c900b1e00 0x7f3c901537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c95ca5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c9015e310 0x7f3c9015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c95ca5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9006e1a0 con 0x7f3c90071df0 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c95ca5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9006e510 con 0x7f3c900b1e00 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c95ca5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c900ceb20 con 0x7f3c9015e310 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c8f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c90071df0 0x7f3c90148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.059+0000 7f3c8f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c90071df0 0x7f3c90148600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50732/0 (socket says 172.21.15.113:50732) 2024-04-25T00:04:05.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8f7fe640 1 -- 172.21.15.113:0/765852067 learned_addr learned my addr 172.21.15.113:0/765852067 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8effd640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c900b1e00 0x7f3c901537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8ffff640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c9015e310 0x7f3c9015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8f7fe640 1 -- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c9015e310 msgr2=0x7f3c9015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8f7fe640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c9015e310 0x7f3c9015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8f7fe640 1 -- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c900b1e00 msgr2=0x7f3c901537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8f7fe640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c900b1e00 0x7f3c901537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8f7fe640 1 -- 172.21.15.113:0/765852067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9014fc90 con 0x7f3c90071df0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8ffff640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c9015e310 0x7f3c9015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c8f7fe640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c90071df0 0x7f3c90148600 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b420 tx=0x7f3c8000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c77fff640 1 -- 172.21.15.113:0/765852067 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80011830 con 0x7f3c90071df0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c77fff640 1 -- 172.21.15.113:0/765852067 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c80016070 con 0x7f3c90071df0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9014ff70 con 0x7f3c90071df0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c77fff640 1 -- 172.21.15.113:0/765852067 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80011c10 con 0x7f3c90071df0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.060+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c90150530 con 0x7f3c90071df0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.061+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9006c080 con 0x7f3c90071df0 2024-04-25T00:04:05.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.062+0000 7f3c77fff640 1 -- 172.21.15.113:0/765852067 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c8001b460 con 0x7f3c90071df0 2024-04-25T00:04:05.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.062+0000 7f3c77fff640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c7c06be30 0x7f3c7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.062+0000 7f3c77fff640 1 -- 172.21.15.113:0/765852067 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5227+0+0 (secure 0 0 0) 0x7f3c8008a6e0 con 0x7f3c90071df0 2024-04-25T00:04:05.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.062+0000 7f3c8effd640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c7c06be30 0x7f3c7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.063+0000 7f3c8effd640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c7c06be30 0x7f3c7c06e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3c90068290 tx=0x7f3c84008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.067+0000 7f3c77fff640 1 -- 172.21.15.113:0/765852067 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c80020020 con 0x7f3c90071df0 2024-04-25T00:04:05.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.239+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3c900691a0 con 0x7f3c90071df0 2024-04-25T00:04:05.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.935+0000 7f3c77fff640 1 -- 172.21.15.113:0/765852067 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7f3c80059270 con 0x7f3c90071df0 2024-04-25T00:04:05.936 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:04:05.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c7c06be30 msgr2=0x7f3c7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c7c06be30 0x7f3c7c06e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3c90068290 tx=0x7f3c84008040 comp rx=0 tx=0).stop 2024-04-25T00:04:05.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c90071df0 msgr2=0x7f3c90148600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c90071df0 0x7f3c90148600 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b420 tx=0x7f3c8000b8e0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 shutdown_connections 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3c7c06be30 0x7f3c7c06e2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c900b1e00 0x7f3c901537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c9015e310 0x7f3c9015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 --2- 172.21.15.113:0/765852067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c90071df0 0x7f3c90148600 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 >> 172.21.15.113:0/765852067 conn(0x7f3c9006fb60 msgr2=0x7f3c900b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 shutdown_connections 2024-04-25T00:04:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:05.938+0000 7f3c95ca5640 1 -- 172.21.15.113:0/765852067 wait complete. 2024-04-25T00:04:05.962 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:04:06.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.064+0000 7f23c3a64640 1 Processor -- start 2024-04-25T00:04:06.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.064+0000 7f23c3a64640 1 -- start start 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c3a64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23bc15ba50 0x7f23bc15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c3a64640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f23bc06cea0 0x7f23bc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c3a64640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23bc06d7e0 con 0x7f23bc15b670 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c3a64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f23bc071170 con 0x7f23bc15ba50 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c3a64640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f23bc071450 con 0x7f23bc06cea0 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c0fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23bc15ba50 0x7f23bc15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c0fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23bc15ba50 0x7f23bc15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50744/0 (socket says 172.21.15.113:50744) 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c0fff640 1 -- 172.21.15.113:0/411666792 learned_addr learned my addr 172.21.15.113:0/411666792 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c2001640 1 --2- 172.21.15.113:0/411666792 >> v2:172.21.15.178:3300/0 conn(0x7f23bc06cea0 0x7f23bc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:06.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c0fff640 1 -- 172.21.15.113:0/411666792 >> v2:172.21.15.178:3300/0 conn(0x7f23bc06cea0 msgr2=0x7f23bc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:06.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c0fff640 1 --2- 172.21.15.113:0/411666792 >> v2:172.21.15.178:3300/0 conn(0x7f23bc06cea0 0x7f23bc06d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c0fff640 1 -- 172.21.15.113:0/411666792 >> v1:172.21.15.178:6789/0 conn(0x7f23bc15b670 legacy=0x7f23bc0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:06.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.065+0000 7f23c0fff640 1 -- 172.21.15.113:0/411666792 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc071730 con 0x7f23bc15ba50 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.066+0000 7f23c0fff640 1 --2- 172.21.15.113:0/411666792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15ba50 0x7f23bc15bff0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f23bc1589f0 tx=0x7f23ac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14835d7da6d09578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.066+0000 7f23b3fff640 1 -- 172.21.15.113:0/411666792 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac037070 con 0x7f23bc15ba50 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.066+0000 7f23b3fff640 1 -- 172.21.15.113:0/411666792 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ac02ae20 con 0x7f23bc15ba50 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.066+0000 7f23b3fff640 1 -- 172.21.15.113:0/411666792 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac032660 con 0x7f23bc15ba50 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 -- 172.21.15.113:0/411666792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15ba50 msgr2=0x7f23bc15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 --2- 172.21.15.113:0/411666792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15ba50 0x7f23bc15bff0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f23bc1589f0 tx=0x7f23ac02a110 comp rx=0 tx=0).stop 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 -- 172.21.15.113:0/411666792 shutdown_connections 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 --2- 172.21.15.113:0/411666792 >> v2:172.21.15.178:3300/0 conn(0x7f23bc06cea0 0x7f23bc06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 --2- 172.21.15.113:0/411666792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15ba50 0x7f23bc15bff0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 -- 172.21.15.113:0/411666792 >> 172.21.15.113:0/411666792 conn(0x7f23bc06b0f0 msgr2=0x7f23bc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 -- 172.21.15.113:0/411666792 shutdown_connections 2024-04-25T00:04:06.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.067+0000 7f23c3a64640 1 -- 172.21.15.113:0/411666792 wait complete. 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.068+0000 7f23c3a64640 1 Processor -- start 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.068+0000 7f23c3a64640 1 -- start start 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c3a64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23bc06cea0 0x7f23bc159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c3a64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15b670 0x7f23bc13b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c3a64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23bc15ba50 0x7f23bc168c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c3a64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc0cca90 con 0x7f23bc15b670 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c3a64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc0cc910 con 0x7f23bc06cea0 2024-04-25T00:04:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c3a64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f23bc0d0080 con 0x7f23bc15ba50 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15b670 0x7f23bc13b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15b670 0x7f23bc13b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50746/0 (socket says 172.21.15.113:50746) 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 -- 172.21.15.113:0/1014737122 learned_addr learned my addr 172.21.15.113:0/1014737122 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c2001640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23bc15ba50 0x7f23bc168c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c1800640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23bc06cea0 0x7f23bc159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 -- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23bc15ba50 msgr2=0x7f23bc168c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23bc15ba50 0x7f23bc168c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 -- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23bc06cea0 msgr2=0x7f23bc159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23bc06cea0 0x7f23bc159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c0fff640 1 -- 172.21.15.113:0/1014737122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc14e7e0 con 0x7f23bc15b670 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.069+0000 7f23c1800640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23bc06cea0 0x7f23bc159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:06.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.070+0000 7f23c0fff640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15b670 0x7f23bc13b380 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02a150 tx=0x7f23ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.070+0000 7f23b1ffb640 1 -- 172.21.15.113:0/1014737122 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac041070 con 0x7f23bc15b670 2024-04-25T00:04:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.070+0000 7f23b1ffb640 1 -- 172.21.15.113:0/1014737122 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ac03c670 con 0x7f23bc15b670 2024-04-25T00:04:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.070+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc167e80 con 0x7f23bc15b670 2024-04-25T00:04:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.070+0000 7f23b1ffb640 1 -- 172.21.15.113:0/1014737122 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac037040 con 0x7f23bc15b670 2024-04-25T00:04:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.070+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc168720 con 0x7f23bc15b670 2024-04-25T00:04:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.071+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23bc168050 con 0x7f23bc15b670 2024-04-25T00:04:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.072+0000 7f23b1ffb640 1 -- 172.21.15.113:0/1014737122 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23ac044070 con 0x7f23bc15b670 2024-04-25T00:04:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.072+0000 7f23b1ffb640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23a006be60 0x7f23a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.072+0000 7f23b1ffb640 1 -- 172.21.15.113:0/1014737122 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5609+0+0 (secure 0 0 0) 0x7f23ac0ab4b0 con 0x7f23bc15b670 2024-04-25T00:04:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.072+0000 7f23c1800640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23a006be60 0x7f23a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.073+0000 7f23c1800640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23a006be60 0x7f23a006e320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f23b8003eb0 tx=0x7f23b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:06.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.077+0000 7f23b1ffb640 1 -- 172.21.15.113:0/1014737122 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23ac079f50 con 0x7f23bc15b670 2024-04-25T00:04:06.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.287+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f23bc0691a0 con 0x7f23bc15b670 2024-04-25T00:04:06.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.937+0000 7f23b1ffb640 1 -- 172.21.15.113:0/1014737122 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v48) ==== 174+0+0 (secure 0 0 0) 0x7f23ac02ae20 con 0x7f23bc15b670 2024-04-25T00:04:06.937 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T00:04:06.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.940+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23a006be60 msgr2=0x7f23a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:06.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.940+0000 7f23c3a64640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23a006be60 0x7f23a006e320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f23b8003eb0 tx=0x7f23b8009040 comp rx=0 tx=0).stop 2024-04-25T00:04:06.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.940+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15b670 msgr2=0x7f23bc13b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:06.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.940+0000 7f23c3a64640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15b670 0x7f23bc13b380 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02a150 tx=0x7f23ac0046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 shutdown_connections 2024-04-25T00:04:06.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f23a006be60 0x7f23a006e320 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23bc15ba50 0x7f23bc168c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc15b670 0x7f23bc13b380 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 --2- 172.21.15.113:0/1014737122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23bc06cea0 0x7f23bc159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:06.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 >> 172.21.15.113:0/1014737122 conn(0x7f23bc06b0f0 msgr2=0x7f23bc154c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:06.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 shutdown_connections 2024-04-25T00:04:06.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:06.941+0000 7f23c3a64640 1 -- 172.21.15.113:0/1014737122 wait complete. 2024-04-25T00:04:06.970 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:07.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.068+0000 7f5bddaba640 1 Processor -- start 2024-04-25T00:04:07.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.068+0000 7f5bddaba640 1 -- start start 2024-04-25T00:04:07.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bddaba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5bd80b4b00 0x7f5bd81656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bddaba640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd8071b20 con 0x7f5bd815a6b0 2024-04-25T00:04:07.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bddaba640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd80b2d50 con 0x7f5bd8072ea0 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bddaba640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd8072a70 con 0x7f5bd80b4b00 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bd7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5bd80b4b00 0x7f5bd81656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bd77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5bd815a6b0 0x7f5bd8070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56550/0 (socket says 172.21.15.113:56550) 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bd77fe640 1 -- 172.21.15.113:0/4113274335 learned_addr learned my addr 172.21.15.113:0/4113274335 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.069+0000 7f5bd7fff640 1 -- 172.21.15.113:0/4113274335 >> v1:172.21.15.178:6789/0 conn(0x7f5bd8072ea0 legacy=0x7f5bd8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.070+0000 7f5bd7fff640 1 -- 172.21.15.113:0/4113274335 >> v1:172.21.15.113:6789/0 conn(0x7f5bd815a6b0 legacy=0x7f5bd8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.070+0000 7f5bd7fff640 1 -- 172.21.15.113:0/4113274335 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd80b3010 con 0x7f5bd80b4b00 2024-04-25T00:04:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.070+0000 7f5bd7fff640 1 --2- 172.21.15.113:0/4113274335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd81656d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009d90 tx=0x7f5bd002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba85b585568c959b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.070+0000 7f5bd67fc640 1 -- 172.21.15.113:0/4113274335 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0007740 con 0x7f5bd80b4b00 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.070+0000 7f5bd67fc640 1 -- 172.21.15.113:0/4113274335 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd00078e0 con 0x7f5bd80b4b00 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.070+0000 7f5bd67fc640 1 -- 172.21.15.113:0/4113274335 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0007cb0 con 0x7f5bd80b4b00 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.071+0000 7f5bddaba640 1 -- 172.21.15.113:0/4113274335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 msgr2=0x7f5bd81656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.071+0000 7f5bddaba640 1 --2- 172.21.15.113:0/4113274335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd81656d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009d90 tx=0x7f5bd002a6a0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.071+0000 7f5bddaba640 1 -- 172.21.15.113:0/4113274335 shutdown_connections 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.071+0000 7f5bddaba640 1 --2- 172.21.15.113:0/4113274335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd81656d0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.071+0000 7f5bddaba640 1 -- 172.21.15.113:0/4113274335 >> 172.21.15.113:0/4113274335 conn(0x7f5bd80b1a50 msgr2=0x7f5bd80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.071+0000 7f5bddaba640 1 -- 172.21.15.113:0/4113274335 shutdown_connections 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.071+0000 7f5bddaba640 1 -- 172.21.15.113:0/4113274335 wait complete. 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.072+0000 7f5bddaba640 1 Processor -- start 2024-04-25T00:04:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.072+0000 7f5bddaba640 1 -- start start 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bddaba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8072ea0 0x7f5bd80ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bddaba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd80aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bddaba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd815a6b0 0x7f5bd8152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bddaba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8072720 con 0x7f5bd80b4b00 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bddaba640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80b35d0 con 0x7f5bd815a6b0 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bddaba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd815aa90 con 0x7f5bd8072ea0 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd80aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd80aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50754/0 (socket says 172.21.15.113:50754) 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 -- 172.21.15.113:0/3571319841 learned_addr learned my addr 172.21.15.113:0/3571319841 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd77fe640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8072ea0 0x7f5bd80ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd7fff640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd815a6b0 0x7f5bd8152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 -- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8072ea0 msgr2=0x7f5bd80ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8072ea0 0x7f5bd80ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 -- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd815a6b0 msgr2=0x7f5bd8152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd815a6b0 0x7f5bd8152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 -- 172.21.15.113:0/3571319841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd80addb0 con 0x7f5bd80b4b00 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bd6ffd640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd80aee50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c430 tx=0x7f5bcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.073+0000 7f5bbffff640 1 -- 172.21.15.113:0/3571319841 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc011790 con 0x7f5bd80b4b00 2024-04-25T00:04:07.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.074+0000 7f5bbffff640 1 -- 172.21.15.113:0/3571319841 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc017070 con 0x7f5bd80b4b00 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.074+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd80ae090 con 0x7f5bd80b4b00 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.074+0000 7f5bbffff640 1 -- 172.21.15.113:0/3571319841 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc011bb0 con 0x7f5bd80b4b00 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.074+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd80ae650 con 0x7f5bd80b4b00 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.074+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd80691a0 con 0x7f5bd80b4b00 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.075+0000 7f5bbffff640 1 -- 172.21.15.113:0/3571319841 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bcc01c460 con 0x7f5bd80b4b00 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.075+0000 7f5bbffff640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5bc406be30 0x7f5bc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.075+0000 7f5bbffff640 1 -- 172.21.15.113:0/3571319841 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f5bcc08acd0 con 0x7f5bd80b4b00 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.075+0000 7f5bd77fe640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5bc406be30 0x7f5bc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.076+0000 7f5bd77fe640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5bc406be30 0x7f5bc406e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8000f60 tx=0x7f5bc8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.080+0000 7f5bbffff640 1 -- 172.21.15.113:0/3571319841 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5bcc059710 con 0x7f5bd80b4b00 2024-04-25T00:04:07.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.257+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bd80b2900 con 0x7f5bd80b4b00 2024-04-25T00:04:07.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.258+0000 7f5bbffff640 1 -- 172.21.15.113:0/3571319841 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16721 (secure 0 0 0) 0x7f5bcc059010 con 0x7f5bd80b4b00 2024-04-25T00:04:07.258 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:07.258 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":91,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:06.288563+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:04.509800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:05.258783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:07.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5bc406be30 msgr2=0x7f5bc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5bc406be30 0x7f5bc406e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8000f60 tx=0x7f5bc8009040 comp rx=0 tx=0).stop 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 msgr2=0x7f5bd80aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd80aee50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c430 tx=0x7f5bcc00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 shutdown_connections 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5bc406be30 0x7f5bc406e2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd815a6b0 0x7f5bd8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bd80b4b00 0x7f5bd80aee50 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 --2- 172.21.15.113:0/3571319841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8072ea0 0x7f5bd80ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 >> 172.21.15.113:0/3571319841 conn(0x7f5bd80b1a50 msgr2=0x7f5bd80b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 shutdown_connections 2024-04-25T00:04:07.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.260+0000 7f5bddaba640 1 -- 172.21.15.113:0/3571319841 wait complete. 2024-04-25T00:04:07.272 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.368+0000 7f52e1114640 1 Processor -- start 2024-04-25T00:04:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.368+0000 7f52e1114640 1 -- start start 2024-04-25T00:04:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52e1114640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52dc144d60 0x7f52dc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52e1114640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52dc06e6e0 0x7f52dc071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52e1114640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc0cfad0 con 0x7f52dc1453c0 2024-04-25T00:04:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52e1114640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f52dc145c40 con 0x7f52dc144d60 2024-04-25T00:04:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52e1114640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f52dc0cf230 con 0x7f52dc06e6e0 2024-04-25T00:04:07.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52dad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52dc144d60 0x7f52dc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52d3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f52dc1453c0 0x7f52dc06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56562/0 (socket says 172.21.15.113:56562) 2024-04-25T00:04:07.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52d3fff640 1 -- 172.21.15.113:0/2612031669 learned_addr learned my addr 172.21.15.113:0/2612031669 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52dad9d640 1 -- 172.21.15.113:0/2612031669 >> v2:172.21.15.178:3300/0 conn(0x7f52dc06e6e0 msgr2=0x7f52dc071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52dad9d640 1 --2- 172.21.15.113:0/2612031669 >> v2:172.21.15.178:3300/0 conn(0x7f52dc06e6e0 0x7f52dc071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52dad9d640 1 -- 172.21.15.113:0/2612031669 >> v1:172.21.15.113:6789/0 conn(0x7f52dc1453c0 legacy=0x7f52dc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52dad9d640 1 -- 172.21.15.113:0/2612031669 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc0cf510 con 0x7f52dc144d60 2024-04-25T00:04:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.369+0000 7f52da59c640 1 -- 172.21.15.113:0/2612031669 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443451516 0 0) 0x7f52dc0cfad0 con 0x7f52dc1453c0 2024-04-25T00:04:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52dad9d640 1 --2- 172.21.15.113:0/2612031669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc0b5390 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f52dc145850 tx=0x7f52cc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04de84e5468143a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52da59c640 1 -- 172.21.15.113:0/2612031669 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc037070 con 0x7f52dc144d60 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52da59c640 1 -- 172.21.15.113:0/2612031669 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52cc032440 con 0x7f52dc144d60 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52da59c640 1 -- 172.21.15.113:0/2612031669 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc02ae60 con 0x7f52dc144d60 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 -- 172.21.15.113:0/2612031669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 msgr2=0x7f52dc0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 --2- 172.21.15.113:0/2612031669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc0b5390 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f52dc145850 tx=0x7f52cc02a110 comp rx=0 tx=0).stop 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 -- 172.21.15.113:0/2612031669 shutdown_connections 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 --2- 172.21.15.113:0/2612031669 >> v2:172.21.15.178:3300/0 conn(0x7f52dc06e6e0 0x7f52dc071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 --2- 172.21.15.113:0/2612031669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc0b5390 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 -- 172.21.15.113:0/2612031669 >> 172.21.15.113:0/2612031669 conn(0x7f52dc0b1a50 msgr2=0x7f52dc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 -- 172.21.15.113:0/2612031669 shutdown_connections 2024-04-25T00:04:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.370+0000 7f52e1114640 1 -- 172.21.15.113:0/2612031669 wait complete. 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 Processor -- start 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 -- start start 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52dc06e6e0 0x7f52dc0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc1f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc1453c0 0x7f52dc15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc145140 con 0x7f52dc144d60 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc070e10 con 0x7f52dc06e6e0 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.372+0000 7f52e1114640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc071870 con 0x7f52dc1453c0 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc1f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc1f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50774/0 (socket says 172.21.15.113:50774) 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 -- 172.21.15.113:0/1131115814 learned_addr learned my addr 172.21.15.113:0/1131115814 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:07.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52dad9d640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52dc06e6e0 0x7f52dc0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52db59e640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc1453c0 0x7f52dc15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 -- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc1453c0 msgr2=0x7f52dc15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc1453c0 0x7f52dc15d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 -- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52dc06e6e0 msgr2=0x7f52dc0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52dc06e6e0 0x7f52dc0aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 -- 172.21.15.113:0/1131115814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc15c770 con 0x7f52dc144d60 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d3fff640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc1f2530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c430 tx=0x7f52c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d37fe640 1 -- 172.21.15.113:0/1131115814 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c40074e0 con 0x7f52dc144d60 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d37fe640 1 -- 172.21.15.113:0/1131115814 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c40047e0 con 0x7f52dc144d60 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc15ca50 con 0x7f52dc144d60 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52d37fe640 1 -- 172.21.15.113:0/1131115814 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4004bd0 con 0x7f52dc144d60 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.373+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc15d010 con 0x7f52dc144d60 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.374+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc0691a0 con 0x7f52dc144d60 2024-04-25T00:04:07.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.375+0000 7f52d37fe640 1 -- 172.21.15.113:0/1131115814 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52c4011070 con 0x7f52dc144d60 2024-04-25T00:04:07.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.375+0000 7f52d37fe640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be30 0x7f52bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.375+0000 7f52dad9d640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be30 0x7f52bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.375+0000 7f52d37fe640 1 -- 172.21.15.113:0/1131115814 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f52c408a940 con 0x7f52dc144d60 2024-04-25T00:04:07.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.376+0000 7f52dad9d640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be30 0x7f52bc06e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f52cc034000 tx=0x7f52cc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.379+0000 7f52d37fe640 1 -- 172.21.15.113:0/1131115814 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52c4058ca0 con 0x7f52dc144d60 2024-04-25T00:04:07.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.552+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52dc0b27e0 con 0x7f52dc144d60 2024-04-25T00:04:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.554+0000 7f52d37fe640 1 -- 172.21.15.113:0/1131115814 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16721 (secure 0 0 0) 0x7f52c405cfa0 con 0x7f52dc144d60 2024-04-25T00:04:07.554 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:07.554 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":91,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:06.288563+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:04.509800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:05.258783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:07.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.555+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be30 msgr2=0x7f52bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.555+0000 7f52e1114640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be30 0x7f52bc06e2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f52cc034000 tx=0x7f52cc02e000 comp rx=0 tx=0).stop 2024-04-25T00:04:07.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 msgr2=0x7f52dc1f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc1f2530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c430 tx=0x7f52c400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 shutdown_connections 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f52bc06be30 0x7f52bc06e2f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52dc1453c0 0x7f52dc15d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc144d60 0x7f52dc1f2530 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 --2- 172.21.15.113:0/1131115814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52dc06e6e0 0x7f52dc0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 >> 172.21.15.113:0/1131115814 conn(0x7f52dc0b1a50 msgr2=0x7f52dc0ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 shutdown_connections 2024-04-25T00:04:07.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.556+0000 7f52e1114640 1 -- 172.21.15.113:0/1131115814 wait complete. 2024-04-25T00:04:07.567 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:04:07.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.667+0000 7f2ca6b30640 1 Processor -- start 2024-04-25T00:04:07.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.667+0000 7f2ca6b30640 1 -- start start 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.668+0000 7f2ca6b30640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2ca015e510 0x7f2ca00d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.668+0000 7f2ca6b30640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca00d2e00 con 0x7f2ca00d28c0 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.668+0000 7f2ca6b30640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca00d30e0 con 0x7f2ca00b63b0 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.668+0000 7f2ca6b30640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca00d33c0 con 0x7f2ca015e510 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.668+0000 7f2c9ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2ca00d28c0 0x7f2ca00b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34704/0 (socket says 172.21.15.113:34704) 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.668+0000 7f2c9ffff640 1 -- 172.21.15.113:0/2108438256 learned_addr learned my addr 172.21.15.113:0/2108438256 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407451566 0 0) 0x7f2ca00d2e00 con 0x7f2ca00d28c0 2024-04-25T00:04:07.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c88003520 con 0x7f2ca00d28c0 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/2108438256 >> v2:172.21.15.178:3300/0 conn(0x7f2ca015e510 0x7f2ca00d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 820403393 0 0) 0x7f2c88003520 con 0x7f2ca00d28c0 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 >> v2:172.21.15.178:3300/0 conn(0x7f2ca015e510 msgr2=0x7f2ca00d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 --2- 172.21.15.113:0/2108438256 >> v2:172.21.15.178:3300/0 conn(0x7f2ca015e510 0x7f2ca00d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 >> v1:172.21.15.178:6789/0 conn(0x7f2ca00b63b0 legacy=0x7f2ca00b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca00d36a0 con 0x7f2ca00d28c0 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f2c94001670 con 0x7f2ca00d28c0 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 >> v1:172.21.15.113:6789/0 conn(0x7f2ca00d28c0 legacy=0x7f2ca00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2c88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c88004800 0x7f2c88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c880051c0 0x7f2c8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca00d33c0 con 0x7f2c880039b0 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8800c160 con 0x7f2c880051c0 2024-04-25T00:04:07.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8800c360 con 0x7f2c88004800 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2c88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2c9ffff640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c88004800 0x7f2c88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca50cd640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c880051c0 0x7f2c8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca48cc640 1 -- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c88004800 msgr2=0x7f2c88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c88004800 0x7f2c88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca48cc640 1 -- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c880051c0 msgr2=0x7f2c8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c880051c0 0x7f2c8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca50cd640 1 -- 172.21.15.113:0/2108438256 reap_dead start 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.669+0000 7f2ca48cc640 1 -- 172.21.15.113:0/2108438256 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c88003520 con 0x7f2c880039b0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.670+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2c88003dd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ac0 tx=0x7f2c88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b177f17b2d4081c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.670+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90036070 con 0x7f2c880039b0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.670+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c90031400 con 0x7f2c880039b0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.670+0000 7f2c9f7fe640 1 -- 172.21.15.113:0/2108438256 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90009690 con 0x7f2c880039b0 2024-04-25T00:04:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.671+0000 7f2ca6b30640 1 -- 172.21.15.113:0/2108438256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 msgr2=0x7f2c88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.671+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2c88003dd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ac0 tx=0x7f2c88005830 comp rx=0 tx=0).stop 2024-04-25T00:04:07.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.671+0000 7f2ca6b30640 1 -- 172.21.15.113:0/2108438256 shutdown_connections 2024-04-25T00:04:07.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.671+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/2108438256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2c88003dd0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.671+0000 7f2ca6b30640 1 -- 172.21.15.113:0/2108438256 >> 172.21.15.113:0/2108438256 conn(0x7f2ca006b720 msgr2=0x7f2ca00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:07.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.671+0000 7f2ca6b30640 1 -- 172.21.15.113:0/2108438256 shutdown_connections 2024-04-25T00:04:07.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.671+0000 7f2ca6b30640 1 -- 172.21.15.113:0/2108438256 wait complete. 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.672+0000 7f2ca6b30640 1 Processor -- start 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.672+0000 7f2ca6b30640 1 -- start start 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2ca01612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ca0151340 0x7f2ca00c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ca00c9c50 0x7f2ca00ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca00b9dd0 con 0x7f2c880039b0 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca00d3ea0 con 0x7f2ca00c9c50 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca00d2380 con 0x7f2ca0151340 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2ca01612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2ca01612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53862/0 (socket says 172.21.15.113:53862) 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 -- 172.21.15.113:0/519698146 learned_addr learned my addr 172.21.15.113:0/519698146 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2c9ffff640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ca0151340 0x7f2ca00c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca50cd640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ca00c9c50 0x7f2ca00ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 -- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ca0151340 msgr2=0x7f2ca00c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ca0151340 0x7f2ca00c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 -- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ca00c9c50 msgr2=0x7f2ca00ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ca00c9c50 0x7f2ca00ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 -- 172.21.15.113:0/519698146 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca00c8f80 con 0x7f2c880039b0 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2c9ffff640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ca0151340 0x7f2ca00c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca48cc640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2ca01612f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009be0 tx=0x7f2c90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2c9d7fa640 1 -- 172.21.15.113:0/519698146 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9003f020 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2c9d7fa640 1 -- 172.21.15.113:0/519698146 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c90037040 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca0153ce0 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2c9d7fa640 1 -- 172.21.15.113:0/519698146 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90036070 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.673+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca01542a0 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.674+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca00691a0 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.675+0000 7f2c9d7fa640 1 -- 172.21.15.113:0/519698146 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c90042020 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.675+0000 7f2c9d7fa640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c8c06be60 0x7f2c8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.675+0000 7f2c9d7fa640 1 -- 172.21.15.113:0/519698146 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f2c900aae50 con 0x7f2c880039b0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.675+0000 7f2c9ffff640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c8c06be60 0x7f2c8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.676+0000 7f2c9ffff640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c8c06be60 0x7f2c8c06e320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ca015a3a0 tx=0x7f2c94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:07.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.679+0000 7f2c9d7fa640 1 -- 172.21.15.113:0/519698146 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c90049030 con 0x7f2c880039b0 2024-04-25T00:04:07.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.888+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2ca00b4240 con 0x7f2c880039b0 2024-04-25T00:04:07.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.953+0000 7f2c9d7fa640 1 -- 172.21.15.113:0/519698146 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) ==== 118+0+0 (secure 0 0 0) 0x7f2c900798c0 con 0x7f2c880039b0 2024-04-25T00:04:07.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c8c06be60 msgr2=0x7f2c8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c8c06be60 0x7f2c8c06e320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ca015a3a0 tx=0x7f2c94009040 comp rx=0 tx=0).stop 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 msgr2=0x7f2ca01612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2ca01612f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009be0 tx=0x7f2c90004730 comp rx=0 tx=0).stop 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 shutdown_connections 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c8c06be60 0x7f2c8c06e320 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ca00c9c50 0x7f2ca00ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ca0151340 0x7f2ca00c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 --2- 172.21.15.113:0/519698146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c880039b0 0x7f2ca01612f0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 >> 172.21.15.113:0/519698146 conn(0x7f2ca006b720 msgr2=0x7f2ca015d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 shutdown_connections 2024-04-25T00:04:07.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:07.955+0000 7f2ca6b30640 1 -- 172.21.15.113:0/519698146 wait complete. 2024-04-25T00:04:07.966 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:04:07.966 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:04:08.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.066+0000 7f3db58a3640 1 Processor -- start 2024-04-25T00:04:08.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.066+0000 7f3db58a3640 1 -- start start 2024-04-25T00:04:08.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3db58a3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3db00cf3c0 0x7f3db00d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:08.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3db58a3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3db0162560 con 0x7f3db0161fc0 2024-04-25T00:04:08.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3db58a3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3db0162840 con 0x7f3db00cf7a0 2024-04-25T00:04:08.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3db58a3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3db0162b20 con 0x7f3db00cf3c0 2024-04-25T00:04:08.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3daeffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3db00cf3c0 0x7f3db00d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:08.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3daf7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3db0161fc0 0x7f3db00d1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34718/0 (socket says 172.21.15.113:34718) 2024-04-25T00:04:08.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3daf7fe640 1 -- 172.21.15.113:0/3501908371 learned_addr learned my addr 172.21.15.113:0/3501908371 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:08.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3daeffd640 1 -- 172.21.15.113:0/3501908371 >> v1:172.21.15.178:6789/0 conn(0x7f3db00cf7a0 legacy=0x7f3db00d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:08.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3daeffd640 1 -- 172.21.15.113:0/3501908371 >> v1:172.21.15.113:6789/0 conn(0x7f3db0161fc0 legacy=0x7f3db00d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3daeffd640 1 -- 172.21.15.113:0/3501908371 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db00d40f0 con 0x7f3db00cf3c0 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.067+0000 7f3dae7fc640 1 -- 172.21.15.113:0/3501908371 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1824478203 0 0) 0x7f3db0162560 con 0x7f3db0161fc0 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.068+0000 7f3daeffd640 1 --2- 172.21.15.113:0/3501908371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf3c0 0x7f3db00d3bb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3da00097a0 tx=0x7f3da002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54747aec8493eb17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.068+0000 7f3dae7fc640 1 -- 172.21.15.113:0/3501908371 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0037070 con 0x7f3db00cf3c0 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.068+0000 7f3dae7fc640 1 -- 172.21.15.113:0/3501908371 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da002ae50 con 0x7f3db00cf3c0 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.068+0000 7f3dae7fc640 1 -- 172.21.15.113:0/3501908371 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0032660 con 0x7f3db00cf3c0 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.069+0000 7f3db58a3640 1 -- 172.21.15.113:0/3501908371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf3c0 msgr2=0x7f3db00d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.069+0000 7f3db58a3640 1 --2- 172.21.15.113:0/3501908371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf3c0 0x7f3db00d3bb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3da00097a0 tx=0x7f3da002a140 comp rx=0 tx=0).stop 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.069+0000 7f3db58a3640 1 -- 172.21.15.113:0/3501908371 shutdown_connections 2024-04-25T00:04:08.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.069+0000 7f3db58a3640 1 --2- 172.21.15.113:0/3501908371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf3c0 0x7f3db00d3bb0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.069+0000 7f3db58a3640 1 -- 172.21.15.113:0/3501908371 >> 172.21.15.113:0/3501908371 conn(0x7f3db00718f0 msgr2=0x7f3db0071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:08.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.069+0000 7f3db58a3640 1 -- 172.21.15.113:0/3501908371 shutdown_connections 2024-04-25T00:04:08.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.069+0000 7f3db58a3640 1 -- 172.21.15.113:0/3501908371 wait complete. 2024-04-25T00:04:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 Processor -- start 2024-04-25T00:04:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 -- start start 2024-04-25T00:04:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3db00cf3c0 0x7f3db01514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf7a0 0x7f3db015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db0161fc0 0x7f3db01600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:08.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3db00d1920 con 0x7f3db00cf7a0 2024-04-25T00:04:08.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3db00d2bf0 con 0x7f3db00cf3c0 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.070+0000 7f3db58a3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3db00d1aa0 con 0x7f3db0161fc0 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf7a0 0x7f3db015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf7a0 0x7f3db015fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53888/0 (socket says 172.21.15.113:53888) 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 -- 172.21.15.113:0/2650790676 learned_addr learned my addr 172.21.15.113:0/2650790676 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3daeffd640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3db00cf3c0 0x7f3db01514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3daf7fe640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db0161fc0 0x7f3db01600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 -- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db0161fc0 msgr2=0x7f3db01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db0161fc0 0x7f3db01600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 -- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3db00cf3c0 msgr2=0x7f3db01514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3db00cf3c0 0x7f3db01514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 -- 172.21.15.113:0/2650790676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db014c5e0 con 0x7f3db00cf7a0 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3daf7fe640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db0161fc0 0x7f3db01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3da7fff640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf7a0 0x7f3db015fb90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c430 tx=0x7f3d9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3dacff9640 1 -- 172.21.15.113:0/2650790676 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98011790 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3dacff9640 1 -- 172.21.15.113:0/2650790676 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d98017070 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3dacff9640 1 -- 172.21.15.113:0/2650790676 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98011bb0 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db014c8c0 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.071+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db014ce80 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.072+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db00691a0 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.073+0000 7f3dacff9640 1 -- 172.21.15.113:0/2650790676 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d9801c460 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.073+0000 7f3dacff9640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9006be30 0x7f3d9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.073+0000 7f3dacff9640 1 -- 172.21.15.113:0/2650790676 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f3d9808ad40 con 0x7f3db00cf7a0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.073+0000 7f3daeffd640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9006be30 0x7f3d9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:08.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.074+0000 7f3daeffd640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9006be30 0x7f3d9006e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3da0034000 tx=0x7f3da002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:08.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.077+0000 7f3dacff9640 1 -- 172.21.15.113:0/2650790676 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d98059780 con 0x7f3db00cf7a0 2024-04-25T00:04:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.289+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3db015c380 con 0x7f3db00cf7a0 2024-04-25T00:04:08.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.954+0000 7f3dacff9640 1 -- 172.21.15.113:0/2650790676 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7f3d98059080 con 0x7f3db00cf7a0 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9006be30 msgr2=0x7f3d9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9006be30 0x7f3d9006e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3da0034000 tx=0x7f3da002e000 comp rx=0 tx=0).stop 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf7a0 msgr2=0x7f3db015fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf7a0 0x7f3db015fb90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c430 tx=0x7f3d9800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 shutdown_connections 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d9006be30 0x7f3d9006e2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db0161fc0 0x7f3db01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db00cf7a0 0x7f3db015fb90 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 --2- 172.21.15.113:0/2650790676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3db00cf3c0 0x7f3db01514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:08.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 >> 172.21.15.113:0/2650790676 conn(0x7f3db00718f0 msgr2=0x7f3db00b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:08.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.957+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 shutdown_connections 2024-04-25T00:04:08.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:08.958+0000 7f3db58a3640 1 -- 172.21.15.113:0/2650790676 wait complete. 2024-04-25T00:04:08.971 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:04:08.972 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:09.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.075+0000 7ff435495640 1 Processor -- start 2024-04-25T00:04:09.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.075+0000 7ff435495640 1 -- start start 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.075+0000 7ff435495640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff43006bc50 0x7ff43006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.075+0000 7ff435495640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff430161fe0 0x7ff43006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff435495640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4301624e0 con 0x7ff43006b0f0 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff435495640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff4301627c0 con 0x7ff43006bc50 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff435495640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff430162aa0 con 0x7ff430161fe0 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff427fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff430161fe0 0x7ff43006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff42effd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff43006bc50 0x7ff43006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff427fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff430161fe0 0x7ff43006c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:53808/0 (socket says 172.21.15.113:53808) 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff427fff640 1 -- 172.21.15.113:0/4237386178 learned_addr learned my addr 172.21.15.113:0/4237386178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff427fff640 1 -- 172.21.15.113:0/4237386178 >> v2:172.21.15.178:3300/0 conn(0x7ff43006bc50 msgr2=0x7ff43006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:09.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff427fff640 1 --2- 172.21.15.113:0/4237386178 >> v2:172.21.15.178:3300/0 conn(0x7ff43006bc50 0x7ff43006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff427fff640 1 -- 172.21.15.113:0/4237386178 >> v1:172.21.15.178:6789/0 conn(0x7ff43006b0f0 legacy=0x7ff4300709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff427fff640 1 -- 172.21.15.113:0/4237386178 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff430162d80 con 0x7ff430161fe0 2024-04-25T00:04:09.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.076+0000 7ff42effd640 1 --2- 172.21.15.113:0/4237386178 >> v2:172.21.15.178:3300/0 conn(0x7ff43006bc50 0x7ff43006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:09.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.077+0000 7ff427fff640 1 --2- 172.21.15.113:0/4237386178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff430161fe0 0x7ff43006c030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff418009870 tx=0x7ff41802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ab44c0d5b2b0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.077+0000 7ff42e7fc640 1 -- 172.21.15.113:0/4237386178 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418037070 con 0x7ff430161fe0 2024-04-25T00:04:09.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.077+0000 7ff42e7fc640 1 -- 172.21.15.113:0/4237386178 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff418032440 con 0x7ff430161fe0 2024-04-25T00:04:09.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.077+0000 7ff42e7fc640 1 -- 172.21.15.113:0/4237386178 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418032810 con 0x7ff430161fe0 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 -- 172.21.15.113:0/4237386178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff430161fe0 msgr2=0x7ff43006c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 --2- 172.21.15.113:0/4237386178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff430161fe0 0x7ff43006c030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff418009870 tx=0x7ff41802a270 comp rx=0 tx=0).stop 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 -- 172.21.15.113:0/4237386178 shutdown_connections 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 --2- 172.21.15.113:0/4237386178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff430161fe0 0x7ff43006c030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 --2- 172.21.15.113:0/4237386178 >> v2:172.21.15.178:3300/0 conn(0x7ff43006bc50 0x7ff43006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 -- 172.21.15.113:0/4237386178 >> 172.21.15.113:0/4237386178 conn(0x7ff4300b3a70 msgr2=0x7ff4300b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 -- 172.21.15.113:0/4237386178 shutdown_connections 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.078+0000 7ff435495640 1 -- 172.21.15.113:0/4237386178 wait complete. 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.079+0000 7ff435495640 1 Processor -- start 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.079+0000 7ff435495640 1 -- start start 2024-04-25T00:04:09.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff435495640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43006b0f0 0x7ff4300c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff435495640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff43006bc50 0x7ff4300b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff435495640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff430161fe0 0x7ff4300b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff435495640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff4300710b0 con 0x7ff43006bc50 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff435495640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff430071940 con 0x7ff430161fe0 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff435495640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff430071230 con 0x7ff43006b0f0 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff43006bc50 0x7ff4300b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff43006bc50 0x7ff4300b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53900/0 (socket says 172.21.15.113:53900) 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 -- 172.21.15.113:0/2202576166 learned_addr learned my addr 172.21.15.113:0/2202576166 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff42effd640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43006b0f0 0x7ff4300c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff42f7fe640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff430161fe0 0x7ff4300b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 -- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43006b0f0 msgr2=0x7ff4300c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43006b0f0 0x7ff4300c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 -- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff430161fe0 msgr2=0x7ff4300b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff430161fe0 0x7ff4300b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 -- 172.21.15.113:0/2202576166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff430159cd0 con 0x7ff43006bc50 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff42effd640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43006b0f0 0x7ff4300c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff427fff640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff43006bc50 0x7ff4300b4e90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff41802a2b0 tx=0x7ff4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff4277fe640 1 -- 172.21.15.113:0/2202576166 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418041030 con 0x7ff43006bc50 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff4277fe640 1 -- 172.21.15.113:0/2202576166 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff418030440 con 0x7ff43006bc50 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff430159fb0 con 0x7ff43006bc50 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.080+0000 7ff4277fe640 1 -- 172.21.15.113:0/2202576166 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418037040 con 0x7ff43006bc50 2024-04-25T00:04:09.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.081+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43015a570 con 0x7ff43006bc50 2024-04-25T00:04:09.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.082+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4300691a0 con 0x7ff43006bc50 2024-04-25T00:04:09.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.082+0000 7ff4277fe640 1 -- 172.21.15.113:0/2202576166 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff418044070 con 0x7ff43006bc50 2024-04-25T00:04:09.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.082+0000 7ff4277fe640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff41006be60 0x7ff41006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.082+0000 7ff4277fe640 1 -- 172.21.15.113:0/2202576166 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7ff4180ab130 con 0x7ff43006bc50 2024-04-25T00:04:09.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.082+0000 7ff42effd640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff41006be60 0x7ff41006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.083+0000 7ff42effd640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff41006be60 0x7ff41006e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff420009af0 tx=0x7ff420008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.086+0000 7ff4277fe640 1 -- 172.21.15.113:0/2202576166 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff41804b070 con 0x7ff43006bc50 2024-04-25T00:04:09.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.296+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4300714d0 con 0x7ff43006bc50 2024-04-25T00:04:09.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.296+0000 7ff4277fe640 1 -- 172.21.15.113:0/2202576166 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5195 (secure 0 0 0) 0x7ff418079b70 con 0x7ff43006bc50 2024-04-25T00:04:09.297 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:09.297 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:06.288869+0000","modified":"2024-04-25T00:04:08.290392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2796979045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2796979045},{"type":"v1","addr":"172.21.15.113:6837","nonce":2796979045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7395,"qdb_cluster":[7395]},"id":6}]} 2024-04-25T00:04:09.297 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff41006be60 msgr2=0x7ff41006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff41006be60 0x7ff41006e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff420009af0 tx=0x7ff420008040 comp rx=0 tx=0).stop 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff43006bc50 msgr2=0x7ff4300b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff43006bc50 0x7ff4300b4e90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff41802a2b0 tx=0x7ff4180046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 shutdown_connections 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff41006be60 0x7ff41006e320 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff430161fe0 0x7ff4300b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff43006bc50 0x7ff4300b4e90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 --2- 172.21.15.113:0/2202576166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43006b0f0 0x7ff4300c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 >> 172.21.15.113:0/2202576166 conn(0x7ff4300b3a70 msgr2=0x7ff4300b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 shutdown_connections 2024-04-25T00:04:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.298+0000 7ff435495640 1 -- 172.21.15.113:0/2202576166 wait complete. 2024-04-25T00:04:09.310 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:09.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.406+0000 7ff5d7b3b640 1 Processor -- start 2024-04-25T00:04:09.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.406+0000 7ff5d7b3b640 1 -- start start 2024-04-25T00:04:09.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.407+0000 7ff5d7b3b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5d0144d60 0x7ff5d00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.407+0000 7ff5d7b3b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 0x7ff5d0071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.407+0000 7ff5d7b3b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5d00cfad0 con 0x7ff5d01453c0 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.407+0000 7ff5d7b3b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d0145c40 con 0x7ff5d0144d60 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.407+0000 7ff5d7b3b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d00cf230 con 0x7ff5d006e6e0 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.407+0000 7ff5d50d6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff5d01453c0 0x7ff5d006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34728/0 (socket says 172.21.15.113:34728) 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.407+0000 7ff5d50d6640 1 -- 172.21.15.113:0/4057996221 learned_addr learned my addr 172.21.15.113:0/4057996221 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634169479 0 0) 0x7ff5d00cfad0 con 0x7ff5d01453c0 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4057996221 >> v2:172.21.15.178:3300/0 conn(0x7ff5d0144d60 0x7ff5d00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5bc003520 con 0x7ff5d01453c0 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d60d8640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 0x7ff5d0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2351050860 0 0) 0x7ff5bc003520 con 0x7ff5d01453c0 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 msgr2=0x7ff5d0071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 0x7ff5d0071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 >> v2:172.21.15.178:3300/0 conn(0x7ff5d0144d60 msgr2=0x7ff5d00b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:09.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 --2- 172.21.15.113:0/4057996221 >> v2:172.21.15.178:3300/0 conn(0x7ff5d0144d60 0x7ff5d00b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d00cf510 con 0x7ff5d01453c0 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff5c0001670 con 0x7ff5d01453c0 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 >> v1:172.21.15.113:6789/0 conn(0x7ff5d01453c0 legacy=0x7ff5d006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5bc0039b0 0x7ff5bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5bc004800 0x7ff5bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 0x7ff5bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0145c40 con 0x7ff5bc0039b0 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d00cf230 con 0x7ff5bc004800 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc00b9f0 con 0x7ff5d006e6e0 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5bc0039b0 0x7ff5bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d50d6640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5bc004800 0x7ff5bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d60d8640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 0x7ff5bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d58d7640 1 -- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 msgr2=0x7ff5bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 0x7ff5bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d58d7640 1 -- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5bc004800 msgr2=0x7ff5bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5bc004800 0x7ff5bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d58d7640 1 -- 172.21.15.113:0/4057996221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc003520 con 0x7ff5bc0039b0 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.408+0000 7ff5d60d8640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d006e6e0 0x7ff5bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.409+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5bc0039b0 0x7ff5bc003dd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009fc0 tx=0x7ff5bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53d0abf402a404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.409+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc009c10 con 0x7ff5bc0039b0 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.409+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc036070 con 0x7ff5bc0039b0 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.409+0000 7ff5d48d5640 1 -- 172.21.15.113:0/4057996221 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc02d070 con 0x7ff5bc0039b0 2024-04-25T00:04:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.410+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4057996221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5bc0039b0 msgr2=0x7ff5bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.410+0000 7ff5d7b3b640 1 --2- 172.21.15.113:0/4057996221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5bc0039b0 0x7ff5bc003dd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009fc0 tx=0x7ff5bc005830 comp rx=0 tx=0).stop 2024-04-25T00:04:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.410+0000 7ff5d60d8640 1 -- 172.21.15.113:0/4057996221 reap_dead start 2024-04-25T00:04:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.410+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4057996221 shutdown_connections 2024-04-25T00:04:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.410+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4057996221 >> 172.21.15.113:0/4057996221 conn(0x7ff5d00b1a50 msgr2=0x7ff5d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.410+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4057996221 shutdown_connections 2024-04-25T00:04:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.410+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4057996221 wait complete. 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.411+0000 7ff5d7b3b640 1 Processor -- start 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.411+0000 7ff5d7b3b640 1 -- start start 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5bc0039b0 0x7ff5d015b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d013bd90 0x7ff5d0138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d01392a0 0x7ff5d0071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0145140 con 0x7ff5d01392a0 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0070e10 con 0x7ff5d013bd90 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d0071330 con 0x7ff5bc0039b0 2024-04-25T00:04:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d01392a0 0x7ff5d0071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d01392a0 0x7ff5d0071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53910/0 (socket says 172.21.15.113:53910) 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 -- 172.21.15.113:0/4225535707 learned_addr learned my addr 172.21.15.113:0/4225535707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5bc0039b0 0x7ff5d015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d50d6640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d013bd90 0x7ff5d0138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 -- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5bc0039b0 msgr2=0x7ff5d015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5bc0039b0 0x7ff5d015b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 -- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d013bd90 msgr2=0x7ff5d0138d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d013bd90 0x7ff5d0138d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 -- 172.21.15.113:0/4225535707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d015baa0 con 0x7ff5d01392a0 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d60d8640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d01392a0 0x7ff5d0071c70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0062850 tx=0x7ff5c8011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5c67fc640 1 -- 172.21.15.113:0/4225535707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8019020 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5c67fc640 1 -- 172.21.15.113:0/4225535707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c8004440 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d013c270 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5c67fc640 1 -- 172.21.15.113:0/4225535707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8003850 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.412+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d013c830 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.413+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d013ca30 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.414+0000 7ff5c67fc640 1 -- 172.21.15.113:0/4225535707 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5c801c020 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.414+0000 7ff5c67fc640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff5b406be30 0x7ff5b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.414+0000 7ff5c67fc640 1 -- 172.21.15.113:0/4225535707 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7ff5c808adf0 con 0x7ff5d01392a0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.414+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff5b406be30 0x7ff5b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.415+0000 7ff5d58d7640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff5b406be30 0x7ff5b406e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc035000 tx=0x7ff5cc02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.418+0000 7ff5c67fc640 1 -- 172.21.15.113:0/4225535707 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff5c8023030 con 0x7ff5d01392a0 2024-04-25T00:04:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.591+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5d00691a0 con 0x7ff5d01392a0 2024-04-25T00:04:09.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.592+0000 7ff5c67fc640 1 -- 172.21.15.113:0/4225535707 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16708 (secure 0 0 0) 0x7ff5c8059830 con 0x7ff5d01392a0 2024-04-25T00:04:09.592 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:09.593 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:07.938101+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:04.509800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:05.258783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff5b406be30 msgr2=0x7ff5b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff5b406be30 0x7ff5b406e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc035000 tx=0x7ff5cc02f000 comp rx=0 tx=0).stop 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d01392a0 msgr2=0x7ff5d0071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d01392a0 0x7ff5d0071c70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0062850 tx=0x7ff5c8011860 comp rx=0 tx=0).stop 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 shutdown_connections 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff5b406be30 0x7ff5b406e2f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d013bd90 0x7ff5d0138d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d01392a0 0x7ff5d0071c70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 --2- 172.21.15.113:0/4225535707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5bc0039b0 0x7ff5d015b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 >> 172.21.15.113:0/4225535707 conn(0x7ff5d00b1a50 msgr2=0x7ff5d0140cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 shutdown_connections 2024-04-25T00:04:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.594+0000 7ff5d7b3b640 1 -- 172.21.15.113:0/4225535707 wait complete. 2024-04-25T00:04:09.606 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:04:09.606 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:04:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.702+0000 7f16a86f5640 1 Processor -- start 2024-04-25T00:04:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.702+0000 7f16a86f5640 1 -- start start 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a86f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a86f5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16a006e6e0 0x7f16a00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a86f5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16a0144cf0 con 0x7f16a00b5240 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a86f5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f16a00b4510 con 0x7f16a006e6e0 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a86f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f16a00b2f30 con 0x7f16a0144910 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6c92640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16a006e6e0 0x7f16a00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6c92640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16a006e6e0 0x7f16a00cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53914/0 (socket says 172.21.15.113:53914) 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6c92640 1 -- 172.21.15.113:0/3229537111 learned_addr learned my addr 172.21.15.113:0/3229537111 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6491640 1 --2- 172.21.15.113:0/3229537111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6c92640 1 -- 172.21.15.113:0/3229537111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 msgr2=0x7f16a00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6c92640 1 --2- 172.21.15.113:0/3229537111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a00b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6c92640 1 -- 172.21.15.113:0/3229537111 >> v1:172.21.15.178:6789/0 conn(0x7f16a00b5240 legacy=0x7f16a006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6c92640 1 -- 172.21.15.113:0/3229537111 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a00b3210 con 0x7f16a006e6e0 2024-04-25T00:04:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.703+0000 7f16a6491640 1 --2- 172.21.15.113:0/3229537111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.704+0000 7f16a6c92640 1 --2- 172.21.15.113:0/3229537111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a006e6e0 0x7f16a00cd500 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1698009870 tx=0x7f169802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cad3e297108c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.704+0000 7f16a548f640 1 -- 172.21.15.113:0/3229537111 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698037070 con 0x7f16a006e6e0 2024-04-25T00:04:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.704+0000 7f16a548f640 1 -- 172.21.15.113:0/3229537111 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1698032440 con 0x7f16a006e6e0 2024-04-25T00:04:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.704+0000 7f16a548f640 1 -- 172.21.15.113:0/3229537111 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698032810 con 0x7f16a006e6e0 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 -- 172.21.15.113:0/3229537111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a006e6e0 msgr2=0x7f16a00cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 --2- 172.21.15.113:0/3229537111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a006e6e0 0x7f16a00cd500 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1698009870 tx=0x7f169802a210 comp rx=0 tx=0).stop 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 -- 172.21.15.113:0/3229537111 shutdown_connections 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 --2- 172.21.15.113:0/3229537111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a006e6e0 0x7f16a00cd500 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 --2- 172.21.15.113:0/3229537111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 -- 172.21.15.113:0/3229537111 >> 172.21.15.113:0/3229537111 conn(0x7f16a00b1a50 msgr2=0x7f16a00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 -- 172.21.15.113:0/3229537111 shutdown_connections 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.705+0000 7f16a86f5640 1 -- 172.21.15.113:0/3229537111 wait complete. 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.706+0000 7f16a86f5640 1 Processor -- start 2024-04-25T00:04:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.706+0000 7f16a86f5640 1 -- start start 2024-04-25T00:04:09.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16a006e6e0 0x7f16a013d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a00b5240 0x7f16a01f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a01f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16a00b4da0 con 0x7f16a00b5240 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16a00b37d0 con 0x7f16a006e6e0 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f16a00b4f20 con 0x7f16a0144910 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a00b5240 0x7f16a01f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a00b5240 0x7f16a01f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53920/0 (socket says 172.21.15.113:53920) 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 -- 172.21.15.113:0/1545108645 learned_addr learned my addr 172.21.15.113:0/1545108645 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a6491640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16a006e6e0 0x7f16a013d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a6c92640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a01f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 -- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 msgr2=0x7f16a01f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a01f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 -- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16a006e6e0 msgr2=0x7f16a013d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16a006e6e0 0x7f16a013d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 -- 172.21.15.113:0/1545108645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a015a590 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a6c92640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a01f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a5c90640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a00b5240 0x7f16a01f1ca0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f169000c430 tx=0x7f169000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f1696ffd640 1 -- 172.21.15.113:0/1545108645 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690011790 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f1696ffd640 1 -- 172.21.15.113:0/1545108645 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1690017070 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a015a870 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f1696ffd640 1 -- 172.21.15.113:0/1545108645 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690011bb0 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.707+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a015ae30 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.708+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a00691a0 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.709+0000 7f1696ffd640 1 -- 172.21.15.113:0/1545108645 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f169001c460 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.709+0000 7f1696ffd640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f168406be30 0x7f168406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.709+0000 7f1696ffd640 1 -- 172.21.15.113:0/1545108645 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f169008acb0 con 0x7f16a00b5240 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.709+0000 7f16a6491640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f168406be30 0x7f168406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:09.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.710+0000 7f16a6491640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f168406be30 0x7f168406e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f169c00d620 tx=0x7f169c007810 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.713+0000 7f1696ffd640 1 -- 172.21.15.113:0/1545108645 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1690059660 con 0x7f16a00b5240 2024-04-25T00:04:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.882+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16a0071b40 con 0x7f168406be30 2024-04-25T00:04:09.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.887+0000 7f1696ffd640 1 -- 172.21.15.113:0/1545108645 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246765 (secure 0 0 0) 0x7f16a0071b40 con 0x7f168406be30 2024-04-25T00:04:09.890 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:09.892 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:04:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f168406be30 msgr2=0x7f168406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f168406be30 0x7f168406e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f169c00d620 tx=0x7f169c007810 comp rx=0 tx=0).stop 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a00b5240 msgr2=0x7f16a01f1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a00b5240 0x7f16a01f1ca0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f169000c430 tx=0x7f169000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 shutdown_connections 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f168406be30 0x7f168406e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16a0144910 0x7f16a01f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a00b5240 0x7f16a01f1ca0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 --2- 172.21.15.113:0/1545108645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16a006e6e0 0x7f16a013d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 >> 172.21.15.113:0/1545108645 conn(0x7f16a00b1a50 msgr2=0x7f16a00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 shutdown_connections 2024-04-25T00:04:09.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:09.890+0000 7f16a86f5640 1 -- 172.21.15.113:0/1545108645 wait complete. 2024-04-25T00:04:09.902 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-04-25T00:04:09.180995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":382712,"kb_used_data":9716,"kb_used_omap":12,"kb_used_meta":372851,"kb_avail":754592008,"statfs":{"total":773094113280,"available":772702216192,"internally_reserved":0,"allocated":9949184,"data_stored":4938731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":381799984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.249291"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944580+0000","last_change":"2024-04-25T00:04:05.944580+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944580+0000","last_undegraded":"2024-04-25T00:04:05.944580+0000","last_fullsized":"2024-04-25T00:04:05.944580+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949172+0000","last_change":"2024-04-25T00:04:05.949172+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949172+0000","last_undegraded":"2024-04-25T00:04:05.949172+0000","last_fullsized":"2024-04-25T00:04:05.949172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948454+0000","last_change":"2024-04-25T00:04:05.948454+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948454+0000","last_undegraded":"2024-04-25T00:04:05.948454+0000","last_fullsized":"2024-04-25T00:04:05.948454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949735+0000","last_change":"2024-04-25T00:04:05.949735+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949735+0000","last_undegraded":"2024-04-25T00:04:05.949735+0000","last_fullsized":"2024-04-25T00:04:05.949735+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941249+0000","last_change":"2024-04-25T00:04:05.941249+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941249+0000","last_undegraded":"2024-04-25T00:04:05.941249+0000","last_fullsized":"2024-04-25T00:04:05.941249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949527+0000","last_change":"2024-04-25T00:04:05.949527+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949527+0000","last_undegraded":"2024-04-25T00:04:05.949527+0000","last_fullsized":"2024-04-25T00:04:05.949527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.954626+0000","last_change":"2024-04-25T00:04:05.954626+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.954626+0000","last_undegraded":"2024-04-25T00:04:05.954626+0000","last_fullsized":"2024-04-25T00:04:05.954626+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942745+0000","last_change":"2024-04-25T00:04:05.942745+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942745+0000","last_undegraded":"2024-04-25T00:04:05.942745+0000","last_fullsized":"2024-04-25T00:04:05.942745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.945401+0000","last_change":"2024-04-25T00:04:05.945401+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.945401+0000","last_undegraded":"2024-04-25T00:04:05.945401+0000","last_fullsized":"2024-04-25T00:04:05.945401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942883+0000","last_change":"2024-04-25T00:04:05.942883+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942883+0000","last_undegraded":"2024-04-25T00:04:05.942883+0000","last_fullsized":"2024-04-25T00:04:05.942883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949288+0000","last_change":"2024-04-25T00:04:05.949288+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949288+0000","last_undegraded":"2024-04-25T00:04:05.949288+0000","last_fullsized":"2024-04-25T00:04:05.949288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.945295+0000","last_change":"2024-04-25T00:04:05.945295+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.945295+0000","last_undegraded":"2024-04-25T00:04:05.945295+0000","last_fullsized":"2024-04-25T00:04:05.945295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948113+0000","last_change":"2024-04-25T00:04:05.948113+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948113+0000","last_undegraded":"2024-04-25T00:04:05.948113+0000","last_fullsized":"2024-04-25T00:04:05.948113+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949697+0000","last_change":"2024-04-25T00:04:05.949697+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949697+0000","last_undegraded":"2024-04-25T00:04:05.949697+0000","last_fullsized":"2024-04-25T00:04:05.949697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949882+0000","last_change":"2024-04-25T00:04:05.949882+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949882+0000","last_undegraded":"2024-04-25T00:04:05.949882+0000","last_fullsized":"2024-04-25T00:04:05.949882+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939754+0000","last_change":"2024-04-25T00:04:05.939754+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939754+0000","last_undegraded":"2024-04-25T00:04:05.939754+0000","last_fullsized":"2024-04-25T00:04:05.939754+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939922+0000","last_change":"2024-04-25T00:04:05.939922+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939922+0000","last_undegraded":"2024-04-25T00:04:05.939922+0000","last_fullsized":"2024-04-25T00:04:05.939922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950188+0000","last_change":"2024-04-25T00:04:05.950188+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950188+0000","last_undegraded":"2024-04-25T00:04:05.950188+0000","last_fullsized":"2024-04-25T00:04:05.950188+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939943+0000","last_change":"2024-04-25T00:04:05.939943+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939943+0000","last_undegraded":"2024-04-25T00:04:05.939943+0000","last_fullsized":"2024-04-25T00:04:05.939943+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949340+0000","last_change":"2024-04-25T00:04:05.949340+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949340+0000","last_undegraded":"2024-04-25T00:04:05.949340+0000","last_fullsized":"2024-04-25T00:04:05.949340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.954404+0000","last_change":"2024-04-25T00:04:05.954404+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.954404+0000","last_undegraded":"2024-04-25T00:04:05.954404+0000","last_fullsized":"2024-04-25T00:04:05.954404+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950060+0000","last_change":"2024-04-25T00:04:05.950060+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950060+0000","last_undegraded":"2024-04-25T00:04:05.950060+0000","last_fullsized":"2024-04-25T00:04:05.950060+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943371+0000","last_change":"2024-04-25T00:04:05.943371+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943371+0000","last_undegraded":"2024-04-25T00:04:05.943371+0000","last_fullsized":"2024-04-25T00:04:05.943371+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.953326+0000","last_change":"2024-04-25T00:04:05.953326+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.953326+0000","last_undegraded":"2024-04-25T00:04:05.953326+0000","last_fullsized":"2024-04-25T00:04:05.953326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950169+0000","last_change":"2024-04-25T00:04:05.950169+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950169+0000","last_undegraded":"2024-04-25T00:04:05.950169+0000","last_fullsized":"2024-04-25T00:04:05.950169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948459+0000","last_change":"2024-04-25T00:04:05.948459+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948459+0000","last_undegraded":"2024-04-25T00:04:05.948459+0000","last_fullsized":"2024-04-25T00:04:05.948459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942111+0000","last_change":"2024-04-25T00:04:05.942111+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942111+0000","last_undegraded":"2024-04-25T00:04:05.942111+0000","last_fullsized":"2024-04-25T00:04:05.942111+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944848+0000","last_change":"2024-04-25T00:04:05.944848+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944848+0000","last_undegraded":"2024-04-25T00:04:05.944848+0000","last_fullsized":"2024-04-25T00:04:05.944848+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948830+0000","last_change":"2024-04-25T00:04:05.948830+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948830+0000","last_undegraded":"2024-04-25T00:04:05.948830+0000","last_fullsized":"2024-04-25T00:04:05.948830+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949664+0000","last_change":"2024-04-25T00:04:05.949664+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949664+0000","last_undegraded":"2024-04-25T00:04:05.949664+0000","last_fullsized":"2024-04-25T00:04:05.949664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939777+0000","last_change":"2024-04-25T00:04:05.939777+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939777+0000","last_undegraded":"2024-04-25T00:04:05.939777+0000","last_fullsized":"2024-04-25T00:04:05.939777+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943940+0000","last_change":"2024-04-25T00:04:05.943940+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943940+0000","last_undegraded":"2024-04-25T00:04:05.943940+0000","last_fullsized":"2024-04-25T00:04:05.943940+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941083+0000","last_change":"2024-04-25T00:04:05.941083+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941083+0000","last_undegraded":"2024-04-25T00:04:05.941083+0000","last_fullsized":"2024-04-25T00:04:05.941083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952826+0000","last_change":"2024-04-25T00:04:05.949584+0000","last_active":"2024-04-25T00:04:05.952826+0000","last_peered":"2024-04-25T00:04:05.952826+0000","last_clean":"2024-04-25T00:04:05.952826+0000","last_became_active":"2024-04-25T00:04:05.948893+0000","last_became_peered":"2024-04-25T00:04:05.948893+0000","last_unstale":"2024-04-25T00:04:05.952826+0000","last_undegraded":"2024-04-25T00:04:05.952826+0000","last_fullsized":"2024-04-25T00:04:05.952826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:42.347304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943649+0000","last_change":"2024-04-25T00:04:05.943649+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943649+0000","last_undegraded":"2024-04-25T00:04:05.943649+0000","last_fullsized":"2024-04-25T00:04:05.943649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949676+0000","last_change":"2024-04-25T00:04:05.943990+0000","last_active":"2024-04-25T00:04:05.949676+0000","last_peered":"2024-04-25T00:04:05.949676+0000","last_clean":"2024-04-25T00:04:05.949676+0000","last_became_active":"2024-04-25T00:04:05.943718+0000","last_became_peered":"2024-04-25T00:04:05.943718+0000","last_unstale":"2024-04-25T00:04:05.949676+0000","last_undegraded":"2024-04-25T00:04:05.949676+0000","last_fullsized":"2024-04-25T00:04:05.949676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:23.344286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943771+0000","last_change":"2024-04-25T00:04:05.943771+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943771+0000","last_undegraded":"2024-04-25T00:04:05.943771+0000","last_fullsized":"2024-04-25T00:04:05.943771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958211+0000","last_change":"2024-04-25T00:04:05.954463+0000","last_active":"2024-04-25T00:04:05.958211+0000","last_peered":"2024-04-25T00:04:05.958211+0000","last_clean":"2024-04-25T00:04:05.958211+0000","last_became_active":"2024-04-25T00:04:05.954215+0000","last_became_peered":"2024-04-25T00:04:05.954215+0000","last_unstale":"2024-04-25T00:04:05.958211+0000","last_undegraded":"2024-04-25T00:04:05.958211+0000","last_fullsized":"2024-04-25T00:04:05.958211+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:23.865237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947794+0000","last_change":"2024-04-25T00:04:05.947794+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947794+0000","last_undegraded":"2024-04-25T00:04:05.947794+0000","last_fullsized":"2024-04-25T00:04:05.947794+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957333+0000","last_change":"2024-04-25T00:04:05.948415+0000","last_active":"2024-04-25T00:04:05.957333+0000","last_peered":"2024-04-25T00:04:05.957333+0000","last_clean":"2024-04-25T00:04:05.957333+0000","last_became_active":"2024-04-25T00:04:05.948148+0000","last_became_peered":"2024-04-25T00:04:05.948148+0000","last_unstale":"2024-04-25T00:04:05.957333+0000","last_undegraded":"2024-04-25T00:04:05.957333+0000","last_fullsized":"2024-04-25T00:04:05.957333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:40.441664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949243+0000","last_change":"2024-04-25T00:04:05.949243+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949243+0000","last_undegraded":"2024-04-25T00:04:05.949243+0000","last_fullsized":"2024-04-25T00:04:05.949243+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949445+0000","last_change":"2024-04-25T00:04:05.947803+0000","last_active":"2024-04-25T00:04:05.949445+0000","last_peered":"2024-04-25T00:04:05.949445+0000","last_clean":"2024-04-25T00:04:05.949445+0000","last_became_active":"2024-04-25T00:04:05.947378+0000","last_became_peered":"2024-04-25T00:04:05.947378+0000","last_unstale":"2024-04-25T00:04:05.949445+0000","last_undegraded":"2024-04-25T00:04:05.949445+0000","last_fullsized":"2024-04-25T00:04:05.949445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:02.514311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940520+0000","last_change":"2024-04-25T00:04:05.940520+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940520+0000","last_undegraded":"2024-04-25T00:04:05.940520+0000","last_fullsized":"2024-04-25T00:04:05.940520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949414+0000","last_change":"2024-04-25T00:04:05.944427+0000","last_active":"2024-04-25T00:04:05.949414+0000","last_peered":"2024-04-25T00:04:05.949414+0000","last_clean":"2024-04-25T00:04:05.949414+0000","last_became_active":"2024-04-25T00:04:05.944008+0000","last_became_peered":"2024-04-25T00:04:05.944008+0000","last_unstale":"2024-04-25T00:04:05.949414+0000","last_undegraded":"2024-04-25T00:04:05.949414+0000","last_fullsized":"2024-04-25T00:04:05.949414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:56.840548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949045+0000","last_change":"2024-04-25T00:04:05.949045+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949045+0000","last_undegraded":"2024-04-25T00:04:05.949045+0000","last_fullsized":"2024-04-25T00:04:05.949045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958180+0000","last_change":"2024-04-25T00:04:05.955068+0000","last_active":"2024-04-25T00:04:05.958180+0000","last_peered":"2024-04-25T00:04:05.958180+0000","last_clean":"2024-04-25T00:04:05.958180+0000","last_became_active":"2024-04-25T00:04:05.954821+0000","last_became_peered":"2024-04-25T00:04:05.954821+0000","last_unstale":"2024-04-25T00:04:05.958180+0000","last_undegraded":"2024-04-25T00:04:05.958180+0000","last_fullsized":"2024-04-25T00:04:05.958180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:43.009886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949448+0000","last_change":"2024-04-25T00:04:05.949448+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949448+0000","last_undegraded":"2024-04-25T00:04:05.949448+0000","last_fullsized":"2024-04-25T00:04:05.949448+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.955763+0000","last_change":"2024-04-25T00:04:05.951451+0000","last_active":"2024-04-25T00:04:05.955763+0000","last_peered":"2024-04-25T00:04:05.955763+0000","last_clean":"2024-04-25T00:04:05.955763+0000","last_became_active":"2024-04-25T00:04:05.951147+0000","last_became_peered":"2024-04-25T00:04:05.951147+0000","last_unstale":"2024-04-25T00:04:05.955763+0000","last_undegraded":"2024-04-25T00:04:05.955763+0000","last_fullsized":"2024-04-25T00:04:05.955763+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:13.641630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948977+0000","last_change":"2024-04-25T00:04:05.948977+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948977+0000","last_undegraded":"2024-04-25T00:04:05.948977+0000","last_fullsized":"2024-04-25T00:04:05.948977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953341+0000","last_change":"2024-04-25T00:04:05.949805+0000","last_active":"2024-04-25T00:04:05.953341+0000","last_peered":"2024-04-25T00:04:05.953341+0000","last_clean":"2024-04-25T00:04:05.953341+0000","last_became_active":"2024-04-25T00:04:05.949244+0000","last_became_peered":"2024-04-25T00:04:05.949244+0000","last_unstale":"2024-04-25T00:04:05.953341+0000","last_undegraded":"2024-04-25T00:04:05.953341+0000","last_fullsized":"2024-04-25T00:04:05.953341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:22.756776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943259+0000","last_change":"2024-04-25T00:04:05.943259+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943259+0000","last_undegraded":"2024-04-25T00:04:05.943259+0000","last_fullsized":"2024-04-25T00:04:05.943259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953423+0000","last_change":"2024-04-25T00:04:05.952083+0000","last_active":"2024-04-25T00:04:05.953423+0000","last_peered":"2024-04-25T00:04:05.953423+0000","last_clean":"2024-04-25T00:04:05.953423+0000","last_became_active":"2024-04-25T00:04:05.951758+0000","last_became_peered":"2024-04-25T00:04:05.951758+0000","last_unstale":"2024-04-25T00:04:05.953423+0000","last_undegraded":"2024-04-25T00:04:05.953423+0000","last_fullsized":"2024-04-25T00:04:05.953423+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:55.690985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940486+0000","last_change":"2024-04-25T00:04:05.940486+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940486+0000","last_undegraded":"2024-04-25T00:04:05.940486+0000","last_fullsized":"2024-04-25T00:04:05.940486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953947+0000","last_change":"2024-04-25T00:04:05.951264+0000","last_active":"2024-04-25T00:04:05.953947+0000","last_peered":"2024-04-25T00:04:05.953947+0000","last_clean":"2024-04-25T00:04:05.953947+0000","last_became_active":"2024-04-25T00:04:05.949917+0000","last_became_peered":"2024-04-25T00:04:05.949917+0000","last_unstale":"2024-04-25T00:04:05.953947+0000","last_undegraded":"2024-04-25T00:04:05.953947+0000","last_fullsized":"2024-04-25T00:04:05.953947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:30.057771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941657+0000","last_change":"2024-04-25T00:04:05.941657+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941657+0000","last_undegraded":"2024-04-25T00:04:05.941657+0000","last_fullsized":"2024-04-25T00:04:05.941657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.951957+0000","last_change":"2024-04-25T00:04:05.951475+0000","last_active":"2024-04-25T00:04:05.951957+0000","last_peered":"2024-04-25T00:04:05.951957+0000","last_clean":"2024-04-25T00:04:05.951957+0000","last_became_active":"2024-04-25T00:04:05.951195+0000","last_became_peered":"2024-04-25T00:04:05.951195+0000","last_unstale":"2024-04-25T00:04:05.951957+0000","last_undegraded":"2024-04-25T00:04:05.951957+0000","last_fullsized":"2024-04-25T00:04:05.951957+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:21.778025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940395+0000","last_change":"2024-04-25T00:04:05.940395+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940395+0000","last_undegraded":"2024-04-25T00:04:05.940395+0000","last_fullsized":"2024-04-25T00:04:05.940395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.950535+0000","last_change":"2024-04-25T00:04:05.949976+0000","last_active":"2024-04-25T00:04:05.950535+0000","last_peered":"2024-04-25T00:04:05.950535+0000","last_clean":"2024-04-25T00:04:05.950535+0000","last_became_active":"2024-04-25T00:04:05.948413+0000","last_became_peered":"2024-04-25T00:04:05.948413+0000","last_unstale":"2024-04-25T00:04:05.950535+0000","last_undegraded":"2024-04-25T00:04:05.950535+0000","last_fullsized":"2024-04-25T00:04:05.950535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:44.096851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947569+0000","last_change":"2024-04-25T00:04:05.947569+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947569+0000","last_undegraded":"2024-04-25T00:04:05.947569+0000","last_fullsized":"2024-04-25T00:04:05.947569+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952039+0000","last_change":"2024-04-25T00:04:05.946979+0000","last_active":"2024-04-25T00:04:05.952039+0000","last_peered":"2024-04-25T00:04:05.952039+0000","last_clean":"2024-04-25T00:04:05.952039+0000","last_became_active":"2024-04-25T00:04:05.946660+0000","last_became_peered":"2024-04-25T00:04:05.946660+0000","last_unstale":"2024-04-25T00:04:05.952039+0000","last_undegraded":"2024-04-25T00:04:05.952039+0000","last_fullsized":"2024-04-25T00:04:05.952039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:45.279979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948793+0000","last_change":"2024-04-25T00:04:05.948793+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948793+0000","last_undegraded":"2024-04-25T00:04:05.948793+0000","last_fullsized":"2024-04-25T00:04:05.948793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.955067+0000","last_change":"2024-04-25T00:04:05.953311+0000","last_active":"2024-04-25T00:04:05.955067+0000","last_peered":"2024-04-25T00:04:05.955067+0000","last_clean":"2024-04-25T00:04:05.955067+0000","last_became_active":"2024-04-25T00:04:05.953035+0000","last_became_peered":"2024-04-25T00:04:05.953035+0000","last_unstale":"2024-04-25T00:04:05.955067+0000","last_undegraded":"2024-04-25T00:04:05.955067+0000","last_fullsized":"2024-04-25T00:04:05.955067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:42.383041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949653+0000","last_change":"2024-04-25T00:04:05.949653+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949653+0000","last_undegraded":"2024-04-25T00:04:05.949653+0000","last_fullsized":"2024-04-25T00:04:05.949653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957482+0000","last_change":"2024-04-25T00:04:05.953176+0000","last_active":"2024-04-25T00:04:05.957482+0000","last_peered":"2024-04-25T00:04:05.957482+0000","last_clean":"2024-04-25T00:04:05.957482+0000","last_became_active":"2024-04-25T00:04:05.952925+0000","last_became_peered":"2024-04-25T00:04:05.952925+0000","last_unstale":"2024-04-25T00:04:05.957482+0000","last_undegraded":"2024-04-25T00:04:05.957482+0000","last_fullsized":"2024-04-25T00:04:05.957482+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.139999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947102+0000","last_change":"2024-04-25T00:04:05.947102+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947102+0000","last_undegraded":"2024-04-25T00:04:05.947102+0000","last_fullsized":"2024-04-25T00:04:05.947102+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953397+0000","last_change":"2024-04-25T00:04:05.952219+0000","last_active":"2024-04-25T00:04:05.953397+0000","last_peered":"2024-04-25T00:04:05.953397+0000","last_clean":"2024-04-25T00:04:05.953397+0000","last_became_active":"2024-04-25T00:04:05.951940+0000","last_became_peered":"2024-04-25T00:04:05.951940+0000","last_unstale":"2024-04-25T00:04:05.953397+0000","last_undegraded":"2024-04-25T00:04:05.953397+0000","last_fullsized":"2024-04-25T00:04:05.953397+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:27.978716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944150+0000","last_change":"2024-04-25T00:04:05.944150+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944150+0000","last_undegraded":"2024-04-25T00:04:05.944150+0000","last_fullsized":"2024-04-25T00:04:05.944150+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.954027+0000","last_change":"2024-04-25T00:04:05.945666+0000","last_active":"2024-04-25T00:04:05.954027+0000","last_peered":"2024-04-25T00:04:05.954027+0000","last_clean":"2024-04-25T00:04:05.954027+0000","last_became_active":"2024-04-25T00:04:05.945410+0000","last_became_peered":"2024-04-25T00:04:05.945410+0000","last_unstale":"2024-04-25T00:04:05.954027+0000","last_undegraded":"2024-04-25T00:04:05.954027+0000","last_fullsized":"2024-04-25T00:04:05.954027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:42.595520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940325+0000","last_change":"2024-04-25T00:04:05.940325+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940325+0000","last_undegraded":"2024-04-25T00:04:05.940325+0000","last_fullsized":"2024-04-25T00:04:05.940325+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960618+0000","last_change":"2024-04-25T00:04:05.957759+0000","last_active":"2024-04-25T00:04:05.960618+0000","last_peered":"2024-04-25T00:04:05.960618+0000","last_clean":"2024-04-25T00:04:05.960618+0000","last_became_active":"2024-04-25T00:04:05.957505+0000","last_became_peered":"2024-04-25T00:04:05.957505+0000","last_unstale":"2024-04-25T00:04:05.960618+0000","last_undegraded":"2024-04-25T00:04:05.960618+0000","last_fullsized":"2024-04-25T00:04:05.960618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:51.841352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949229+0000","last_change":"2024-04-25T00:04:05.949229+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949229+0000","last_undegraded":"2024-04-25T00:04:05.949229+0000","last_fullsized":"2024-04-25T00:04:05.949229+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960796+0000","last_change":"2024-04-25T00:04:05.958193+0000","last_active":"2024-04-25T00:04:05.960796+0000","last_peered":"2024-04-25T00:04:05.960796+0000","last_clean":"2024-04-25T00:04:05.960796+0000","last_became_active":"2024-04-25T00:04:05.957902+0000","last_became_peered":"2024-04-25T00:04:05.957902+0000","last_unstale":"2024-04-25T00:04:05.960796+0000","last_undegraded":"2024-04-25T00:04:05.960796+0000","last_fullsized":"2024-04-25T00:04:05.960796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:15.198528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946063+0000","last_change":"2024-04-25T00:04:05.946063+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946063+0000","last_undegraded":"2024-04-25T00:04:05.946063+0000","last_fullsized":"2024-04-25T00:04:05.946063+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960707+0000","last_change":"2024-04-25T00:04:05.956393+0000","last_active":"2024-04-25T00:04:05.960707+0000","last_peered":"2024-04-25T00:04:05.960707+0000","last_clean":"2024-04-25T00:04:05.960707+0000","last_became_active":"2024-04-25T00:04:05.955523+0000","last_became_peered":"2024-04-25T00:04:05.955523+0000","last_unstale":"2024-04-25T00:04:05.960707+0000","last_undegraded":"2024-04-25T00:04:05.960707+0000","last_fullsized":"2024-04-25T00:04:05.960707+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:57.738671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.952933+0000","last_change":"2024-04-25T00:04:05.952933+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.952933+0000","last_undegraded":"2024-04-25T00:04:05.952933+0000","last_fullsized":"2024-04-25T00:04:05.952933+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958421+0000","last_change":"2024-04-25T00:04:05.955280+0000","last_active":"2024-04-25T00:04:05.958421+0000","last_peered":"2024-04-25T00:04:05.958421+0000","last_clean":"2024-04-25T00:04:05.958421+0000","last_became_active":"2024-04-25T00:04:05.955021+0000","last_became_peered":"2024-04-25T00:04:05.955021+0000","last_unstale":"2024-04-25T00:04:05.958421+0000","last_undegraded":"2024-04-25T00:04:05.958421+0000","last_fullsized":"2024-04-25T00:04:05.958421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:39.296304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946745+0000","last_change":"2024-04-25T00:04:05.946745+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946745+0000","last_undegraded":"2024-04-25T00:04:05.946745+0000","last_fullsized":"2024-04-25T00:04:05.946745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960555+0000","last_change":"2024-04-25T00:04:05.955799+0000","last_active":"2024-04-25T00:04:05.960555+0000","last_peered":"2024-04-25T00:04:05.960555+0000","last_clean":"2024-04-25T00:04:05.960555+0000","last_became_active":"2024-04-25T00:04:05.955566+0000","last_became_peered":"2024-04-25T00:04:05.955566+0000","last_unstale":"2024-04-25T00:04:05.960555+0000","last_undegraded":"2024-04-25T00:04:05.960555+0000","last_fullsized":"2024-04-25T00:04:05.960555+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:55.307410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940520+0000","last_change":"2024-04-25T00:04:05.940520+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940520+0000","last_undegraded":"2024-04-25T00:04:05.940520+0000","last_fullsized":"2024-04-25T00:04:05.940520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.956548+0000","last_change":"2024-04-25T00:04:05.953036+0000","last_active":"2024-04-25T00:04:05.956548+0000","last_peered":"2024-04-25T00:04:05.956548+0000","last_clean":"2024-04-25T00:04:05.956548+0000","last_became_active":"2024-04-25T00:04:05.952742+0000","last_became_peered":"2024-04-25T00:04:05.952742+0000","last_unstale":"2024-04-25T00:04:05.956548+0000","last_undegraded":"2024-04-25T00:04:05.956548+0000","last_fullsized":"2024-04-25T00:04:05.956548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:45.008035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949861+0000","last_change":"2024-04-25T00:04:05.949861+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949861+0000","last_undegraded":"2024-04-25T00:04:05.949861+0000","last_fullsized":"2024-04-25T00:04:05.949861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.951271+0000","last_change":"2024-04-25T00:04:05.950505+0000","last_active":"2024-04-25T00:04:05.951271+0000","last_peered":"2024-04-25T00:04:05.951271+0000","last_clean":"2024-04-25T00:04:05.951271+0000","last_became_active":"2024-04-25T00:04:05.948719+0000","last_became_peered":"2024-04-25T00:04:05.948719+0000","last_unstale":"2024-04-25T00:04:05.951271+0000","last_undegraded":"2024-04-25T00:04:05.951271+0000","last_fullsized":"2024-04-25T00:04:05.951271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:35.048191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943304+0000","last_change":"2024-04-25T00:04:05.943304+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943304+0000","last_undegraded":"2024-04-25T00:04:05.943304+0000","last_fullsized":"2024-04-25T00:04:05.943304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953915+0000","last_change":"2024-04-25T00:04:05.949348+0000","last_active":"2024-04-25T00:04:05.953915+0000","last_peered":"2024-04-25T00:04:05.953915+0000","last_clean":"2024-04-25T00:04:05.953915+0000","last_became_active":"2024-04-25T00:04:05.948760+0000","last_became_peered":"2024-04-25T00:04:05.948760+0000","last_unstale":"2024-04-25T00:04:05.953915+0000","last_undegraded":"2024-04-25T00:04:05.953915+0000","last_fullsized":"2024-04-25T00:04:05.953915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:28.987214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939814+0000","last_change":"2024-04-25T00:04:05.939814+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939814+0000","last_undegraded":"2024-04-25T00:04:05.939814+0000","last_fullsized":"2024-04-25T00:04:05.939814+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.954516+0000","last_change":"2024-04-25T00:04:05.953709+0000","last_active":"2024-04-25T00:04:05.954516+0000","last_peered":"2024-04-25T00:04:05.954516+0000","last_clean":"2024-04-25T00:04:05.954516+0000","last_became_active":"2024-04-25T00:04:05.953453+0000","last_became_peered":"2024-04-25T00:04:05.953453+0000","last_unstale":"2024-04-25T00:04:05.954516+0000","last_undegraded":"2024-04-25T00:04:05.954516+0000","last_fullsized":"2024-04-25T00:04:05.954516+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:56.648421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946625+0000","last_change":"2024-04-25T00:04:05.946625+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946625+0000","last_undegraded":"2024-04-25T00:04:05.946625+0000","last_fullsized":"2024-04-25T00:04:05.946625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957426+0000","last_change":"2024-04-25T00:04:05.953267+0000","last_active":"2024-04-25T00:04:05.957426+0000","last_peered":"2024-04-25T00:04:05.957426+0000","last_clean":"2024-04-25T00:04:05.957426+0000","last_became_active":"2024-04-25T00:04:05.953021+0000","last_became_peered":"2024-04-25T00:04:05.953021+0000","last_unstale":"2024-04-25T00:04:05.957426+0000","last_undegraded":"2024-04-25T00:04:05.957426+0000","last_fullsized":"2024-04-25T00:04:05.957426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:37.383495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939924+0000","last_change":"2024-04-25T00:04:05.939924+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939924+0000","last_undegraded":"2024-04-25T00:04:05.939924+0000","last_fullsized":"2024-04-25T00:04:05.939924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949496+0000","last_change":"2024-04-25T00:04:05.944307+0000","last_active":"2024-04-25T00:04:05.949496+0000","last_peered":"2024-04-25T00:04:05.949496+0000","last_clean":"2024-04-25T00:04:05.949496+0000","last_became_active":"2024-04-25T00:04:05.943618+0000","last_became_peered":"2024-04-25T00:04:05.943618+0000","last_unstale":"2024-04-25T00:04:05.949496+0000","last_undegraded":"2024-04-25T00:04:05.949496+0000","last_fullsized":"2024-04-25T00:04:05.949496+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:48.774779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948590+0000","last_change":"2024-04-25T00:04:05.948590+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948590+0000","last_undegraded":"2024-04-25T00:04:05.948590+0000","last_fullsized":"2024-04-25T00:04:05.948590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952860+0000","last_change":"2024-04-25T00:04:05.949003+0000","last_active":"2024-04-25T00:04:05.952860+0000","last_peered":"2024-04-25T00:04:05.952860+0000","last_clean":"2024-04-25T00:04:05.952860+0000","last_became_active":"2024-04-25T00:04:05.948731+0000","last_became_peered":"2024-04-25T00:04:05.948731+0000","last_unstale":"2024-04-25T00:04:05.952860+0000","last_undegraded":"2024-04-25T00:04:05.952860+0000","last_fullsized":"2024-04-25T00:04:05.952860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:20.845105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942891+0000","last_change":"2024-04-25T00:04:05.942891+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942891+0000","last_undegraded":"2024-04-25T00:04:05.942891+0000","last_fullsized":"2024-04-25T00:04:05.942891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949915+0000","last_change":"2024-04-25T00:04:05.949157+0000","last_active":"2024-04-25T00:04:05.949915+0000","last_peered":"2024-04-25T00:04:05.949915+0000","last_clean":"2024-04-25T00:04:05.949915+0000","last_became_active":"2024-04-25T00:04:05.948879+0000","last_became_peered":"2024-04-25T00:04:05.948879+0000","last_unstale":"2024-04-25T00:04:05.949915+0000","last_undegraded":"2024-04-25T00:04:05.949915+0000","last_fullsized":"2024-04-25T00:04:05.949915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:26.961912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948850+0000","last_change":"2024-04-25T00:04:05.948850+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948850+0000","last_undegraded":"2024-04-25T00:04:05.948850+0000","last_fullsized":"2024-04-25T00:04:05.948850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957511+0000","last_change":"2024-04-25T00:04:05.957149+0000","last_active":"2024-04-25T00:04:05.957511+0000","last_peered":"2024-04-25T00:04:05.957511+0000","last_clean":"2024-04-25T00:04:05.957511+0000","last_became_active":"2024-04-25T00:04:05.956870+0000","last_became_peered":"2024-04-25T00:04:05.956870+0000","last_unstale":"2024-04-25T00:04:05.957511+0000","last_undegraded":"2024-04-25T00:04:05.957511+0000","last_fullsized":"2024-04-25T00:04:05.957511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:41.561274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":297,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.950293+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:04:05.950293+0000","last_peered":"2024-04-25T00:04:05.950293+0000","last_clean":"2024-04-25T00:04:05.950293+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:04:05.950293+0000","last_undegraded":"2024-04-25T00:04:05.950293+0000","last_fullsized":"2024-04-25T00:04:05.950293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55040,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316800,"statfs":{"total":96636764160,"available":96580403200,"internally_reserved":0,"allocated":1687552,"data_stored":1059333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58620,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313220,"statfs":{"total":96636764160,"available":96576737280,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40576,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331264,"statfs":{"total":96636764160,"available":96595214336,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50304,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321536,"statfs":{"total":96636764160,"available":96585252864,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41164,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330676,"statfs":{"total":96636764160,"available":96594612224,"internally_reserved":0,"allocated":1699840,"data_stored":1065608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:04:09.907 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:04:10.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.037+0000 7fe67d7b0640 1 Processor -- start 2024-04-25T00:04:10.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.037+0000 7fe67d7b0640 1 -- start start 2024-04-25T00:04:10.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.038+0000 7fe67d7b0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6780d2290 0x7fe6780b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.038+0000 7fe67d7b0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6780d2800 con 0x7fe67815e320 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.038+0000 7fe67d7b0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6780d2ae0 con 0x7fe6780d0b00 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.038+0000 7fe67d7b0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe6780d3410 con 0x7fe6780d2290 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.038+0000 7fe676ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe67815e320 0x7fe6780d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34734/0 (socket says 172.21.15.113:34734) 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.038+0000 7fe676ffd640 1 -- 172.21.15.113:0/1053028921 learned_addr learned my addr 172.21.15.113:0/1053028921 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.038+0000 7fe6767fc640 1 --2- 172.21.15.113:0/1053028921 >> v2:172.21.15.113:3300/0 conn(0x7fe6780d2290 0x7fe6780b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.039+0000 7fe675ffb640 1 -- 172.21.15.113:0/1053028921 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10166103 0 0) 0x7fe6780d2800 con 0x7fe67815e320 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.039+0000 7fe675ffb640 1 -- 172.21.15.113:0/1053028921 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe660003520 con 0x7fe67815e320 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.039+0000 7fe6767fc640 1 -- 172.21.15.113:0/1053028921 >> v1:172.21.15.178:6789/0 conn(0x7fe6780d0b00 legacy=0x7fe6780d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.039+0000 7fe6767fc640 1 -- 172.21.15.113:0/1053028921 >> v1:172.21.15.113:6789/0 conn(0x7fe67815e320 legacy=0x7fe6780d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.039+0000 7fe6767fc640 1 -- 172.21.15.113:0/1053028921 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6780d2dc0 con 0x7fe6780d2290 2024-04-25T00:04:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.039+0000 7fe6767fc640 1 --2- 172.21.15.113:0/1053028921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6780d2290 0x7fe6780b9aa0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0097a0 tx=0x7fe66c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d26e86a9658c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe675ffb640 1 -- 172.21.15.113:0/1053028921 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c037070 con 0x7fe6780d2290 2024-04-25T00:04:10.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe675ffb640 1 -- 172.21.15.113:0/1053028921 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66c032440 con 0x7fe6780d2290 2024-04-25T00:04:10.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe675ffb640 1 -- 172.21.15.113:0/1053028921 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c032810 con 0x7fe6780d2290 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe67d7b0640 1 -- 172.21.15.113:0/1053028921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6780d2290 msgr2=0x7fe6780b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/1053028921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6780d2290 0x7fe6780b9aa0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0097a0 tx=0x7fe66c02a270 comp rx=0 tx=0).stop 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe67d7b0640 1 -- 172.21.15.113:0/1053028921 shutdown_connections 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/1053028921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6780d2290 0x7fe6780b9aa0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe67d7b0640 1 -- 172.21.15.113:0/1053028921 >> 172.21.15.113:0/1053028921 conn(0x7fe6780b5c20 msgr2=0x7fe6780b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe67d7b0640 1 -- 172.21.15.113:0/1053028921 shutdown_connections 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.040+0000 7fe67d7b0640 1 -- 172.21.15.113:0/1053028921 wait complete. 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.041+0000 7fe67d7b0640 1 Processor -- start 2024-04-25T00:04:10.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.041+0000 7fe67d7b0640 1 -- start start 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe67d7b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6780d0b00 0x7fe678154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe67d7b0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6780d2290 0x7fe678157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe67d7b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67815e320 0x7fe6780b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe67d7b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe67815e700 con 0x7fe67815e320 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe67d7b0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6780b9fe0 con 0x7fe6780d0b00 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe67d7b0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe67815e880 con 0x7fe6780d2290 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67815e320 0x7fe6780b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67815e320 0x7fe6780b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53942/0 (socket says 172.21.15.113:53942) 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 -- 172.21.15.113:0/3338088707 learned_addr learned my addr 172.21.15.113:0/3338088707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe676ffd640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6780d0b00 0x7fe678154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6767fc640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6780d2290 0x7fe678157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 -- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6780d2290 msgr2=0x7fe678157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6780d2290 0x7fe678157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 -- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6780d0b00 msgr2=0x7fe678154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6780d0b00 0x7fe678154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.042+0000 7fe6777fe640 1 -- 172.21.15.113:0/3338088707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe678154580 con 0x7fe67815e320 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.043+0000 7fe6777fe640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67815e320 0x7fe6780b7c90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c410 tx=0x7fe67000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.043+0000 7fe65f7fe640 1 -- 172.21.15.113:0/3338088707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670011790 con 0x7fe67815e320 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.043+0000 7fe65f7fe640 1 -- 172.21.15.113:0/3338088707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670017070 con 0x7fe67815e320 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.043+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe678158a80 con 0x7fe67815e320 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.043+0000 7fe65f7fe640 1 -- 172.21.15.113:0/3338088707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670011b70 con 0x7fe67815e320 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.043+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe678159040 con 0x7fe67815e320 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.044+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6780691a0 con 0x7fe67815e320 2024-04-25T00:04:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.044+0000 7fe65f7fe640 1 -- 172.21.15.113:0/3338088707 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe670011d10 con 0x7fe67815e320 2024-04-25T00:04:10.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.045+0000 7fe65f7fe640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66406be30 0x7fe66406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.045+0000 7fe65f7fe640 1 -- 172.21.15.113:0/3338088707 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fe67008aa30 con 0x7fe67815e320 2024-04-25T00:04:10.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.045+0000 7fe676ffd640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66406be30 0x7fe66406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.045+0000 7fe676ffd640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66406be30 0x7fe66406e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe668005c00 tx=0x7fe668009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.049+0000 7fe65f7fe640 1 -- 172.21.15.113:0/3338088707 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe670021020 con 0x7fe67815e320 2024-04-25T00:04:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.217+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6780d3b50 con 0x7fe66406be30 2024-04-25T00:04:10.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.222+0000 7fe65f7fe640 1 -- 172.21.15.113:0/3338088707 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246765 (secure 0 0 0) 0x7fe6780d3b50 con 0x7fe66406be30 2024-04-25T00:04:10.225 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:10.226 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66406be30 msgr2=0x7fe66406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66406be30 0x7fe66406e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe668005c00 tx=0x7fe668009040 comp rx=0 tx=0).stop 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67815e320 msgr2=0x7fe6780b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67815e320 0x7fe6780b7c90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c410 tx=0x7fe67000c8d0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 shutdown_connections 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe66406be30 0x7fe66406e2f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe67815e320 0x7fe6780b7c90 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6780d2290 0x7fe678157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 --2- 172.21.15.113:0/3338088707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6780d0b00 0x7fe678154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 >> 172.21.15.113:0/3338088707 conn(0x7fe6780b5c20 msgr2=0x7fe6780b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:10.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.225+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 shutdown_connections 2024-04-25T00:04:10.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.226+0000 7fe67d7b0640 1 -- 172.21.15.113:0/3338088707 wait complete. 2024-04-25T00:04:10.237 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-04-25T00:04:09.180995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":382712,"kb_used_data":9716,"kb_used_omap":12,"kb_used_meta":372851,"kb_avail":754592008,"statfs":{"total":773094113280,"available":772702216192,"internally_reserved":0,"allocated":9949184,"data_stored":4938731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":381799984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.249291"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944580+0000","last_change":"2024-04-25T00:04:05.944580+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944580+0000","last_undegraded":"2024-04-25T00:04:05.944580+0000","last_fullsized":"2024-04-25T00:04:05.944580+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949172+0000","last_change":"2024-04-25T00:04:05.949172+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949172+0000","last_undegraded":"2024-04-25T00:04:05.949172+0000","last_fullsized":"2024-04-25T00:04:05.949172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948454+0000","last_change":"2024-04-25T00:04:05.948454+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948454+0000","last_undegraded":"2024-04-25T00:04:05.948454+0000","last_fullsized":"2024-04-25T00:04:05.948454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949735+0000","last_change":"2024-04-25T00:04:05.949735+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949735+0000","last_undegraded":"2024-04-25T00:04:05.949735+0000","last_fullsized":"2024-04-25T00:04:05.949735+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941249+0000","last_change":"2024-04-25T00:04:05.941249+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941249+0000","last_undegraded":"2024-04-25T00:04:05.941249+0000","last_fullsized":"2024-04-25T00:04:05.941249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949527+0000","last_change":"2024-04-25T00:04:05.949527+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949527+0000","last_undegraded":"2024-04-25T00:04:05.949527+0000","last_fullsized":"2024-04-25T00:04:05.949527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.954626+0000","last_change":"2024-04-25T00:04:05.954626+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.954626+0000","last_undegraded":"2024-04-25T00:04:05.954626+0000","last_fullsized":"2024-04-25T00:04:05.954626+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942745+0000","last_change":"2024-04-25T00:04:05.942745+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942745+0000","last_undegraded":"2024-04-25T00:04:05.942745+0000","last_fullsized":"2024-04-25T00:04:05.942745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.945401+0000","last_change":"2024-04-25T00:04:05.945401+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.945401+0000","last_undegraded":"2024-04-25T00:04:05.945401+0000","last_fullsized":"2024-04-25T00:04:05.945401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942883+0000","last_change":"2024-04-25T00:04:05.942883+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942883+0000","last_undegraded":"2024-04-25T00:04:05.942883+0000","last_fullsized":"2024-04-25T00:04:05.942883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949288+0000","last_change":"2024-04-25T00:04:05.949288+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949288+0000","last_undegraded":"2024-04-25T00:04:05.949288+0000","last_fullsized":"2024-04-25T00:04:05.949288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.945295+0000","last_change":"2024-04-25T00:04:05.945295+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.945295+0000","last_undegraded":"2024-04-25T00:04:05.945295+0000","last_fullsized":"2024-04-25T00:04:05.945295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948113+0000","last_change":"2024-04-25T00:04:05.948113+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948113+0000","last_undegraded":"2024-04-25T00:04:05.948113+0000","last_fullsized":"2024-04-25T00:04:05.948113+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949697+0000","last_change":"2024-04-25T00:04:05.949697+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949697+0000","last_undegraded":"2024-04-25T00:04:05.949697+0000","last_fullsized":"2024-04-25T00:04:05.949697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949882+0000","last_change":"2024-04-25T00:04:05.949882+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949882+0000","last_undegraded":"2024-04-25T00:04:05.949882+0000","last_fullsized":"2024-04-25T00:04:05.949882+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939754+0000","last_change":"2024-04-25T00:04:05.939754+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939754+0000","last_undegraded":"2024-04-25T00:04:05.939754+0000","last_fullsized":"2024-04-25T00:04:05.939754+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939922+0000","last_change":"2024-04-25T00:04:05.939922+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939922+0000","last_undegraded":"2024-04-25T00:04:05.939922+0000","last_fullsized":"2024-04-25T00:04:05.939922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950188+0000","last_change":"2024-04-25T00:04:05.950188+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950188+0000","last_undegraded":"2024-04-25T00:04:05.950188+0000","last_fullsized":"2024-04-25T00:04:05.950188+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939943+0000","last_change":"2024-04-25T00:04:05.939943+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939943+0000","last_undegraded":"2024-04-25T00:04:05.939943+0000","last_fullsized":"2024-04-25T00:04:05.939943+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949340+0000","last_change":"2024-04-25T00:04:05.949340+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949340+0000","last_undegraded":"2024-04-25T00:04:05.949340+0000","last_fullsized":"2024-04-25T00:04:05.949340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.954404+0000","last_change":"2024-04-25T00:04:05.954404+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.954404+0000","last_undegraded":"2024-04-25T00:04:05.954404+0000","last_fullsized":"2024-04-25T00:04:05.954404+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950060+0000","last_change":"2024-04-25T00:04:05.950060+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950060+0000","last_undegraded":"2024-04-25T00:04:05.950060+0000","last_fullsized":"2024-04-25T00:04:05.950060+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943371+0000","last_change":"2024-04-25T00:04:05.943371+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943371+0000","last_undegraded":"2024-04-25T00:04:05.943371+0000","last_fullsized":"2024-04-25T00:04:05.943371+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.953326+0000","last_change":"2024-04-25T00:04:05.953326+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.953326+0000","last_undegraded":"2024-04-25T00:04:05.953326+0000","last_fullsized":"2024-04-25T00:04:05.953326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950169+0000","last_change":"2024-04-25T00:04:05.950169+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950169+0000","last_undegraded":"2024-04-25T00:04:05.950169+0000","last_fullsized":"2024-04-25T00:04:05.950169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948459+0000","last_change":"2024-04-25T00:04:05.948459+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948459+0000","last_undegraded":"2024-04-25T00:04:05.948459+0000","last_fullsized":"2024-04-25T00:04:05.948459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942111+0000","last_change":"2024-04-25T00:04:05.942111+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942111+0000","last_undegraded":"2024-04-25T00:04:05.942111+0000","last_fullsized":"2024-04-25T00:04:05.942111+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944848+0000","last_change":"2024-04-25T00:04:05.944848+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944848+0000","last_undegraded":"2024-04-25T00:04:05.944848+0000","last_fullsized":"2024-04-25T00:04:05.944848+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948830+0000","last_change":"2024-04-25T00:04:05.948830+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948830+0000","last_undegraded":"2024-04-25T00:04:05.948830+0000","last_fullsized":"2024-04-25T00:04:05.948830+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949664+0000","last_change":"2024-04-25T00:04:05.949664+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949664+0000","last_undegraded":"2024-04-25T00:04:05.949664+0000","last_fullsized":"2024-04-25T00:04:05.949664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939777+0000","last_change":"2024-04-25T00:04:05.939777+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939777+0000","last_undegraded":"2024-04-25T00:04:05.939777+0000","last_fullsized":"2024-04-25T00:04:05.939777+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943940+0000","last_change":"2024-04-25T00:04:05.943940+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943940+0000","last_undegraded":"2024-04-25T00:04:05.943940+0000","last_fullsized":"2024-04-25T00:04:05.943940+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941083+0000","last_change":"2024-04-25T00:04:05.941083+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941083+0000","last_undegraded":"2024-04-25T00:04:05.941083+0000","last_fullsized":"2024-04-25T00:04:05.941083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952826+0000","last_change":"2024-04-25T00:04:05.949584+0000","last_active":"2024-04-25T00:04:05.952826+0000","last_peered":"2024-04-25T00:04:05.952826+0000","last_clean":"2024-04-25T00:04:05.952826+0000","last_became_active":"2024-04-25T00:04:05.948893+0000","last_became_peered":"2024-04-25T00:04:05.948893+0000","last_unstale":"2024-04-25T00:04:05.952826+0000","last_undegraded":"2024-04-25T00:04:05.952826+0000","last_fullsized":"2024-04-25T00:04:05.952826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:42.347304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943649+0000","last_change":"2024-04-25T00:04:05.943649+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943649+0000","last_undegraded":"2024-04-25T00:04:05.943649+0000","last_fullsized":"2024-04-25T00:04:05.943649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949676+0000","last_change":"2024-04-25T00:04:05.943990+0000","last_active":"2024-04-25T00:04:05.949676+0000","last_peered":"2024-04-25T00:04:05.949676+0000","last_clean":"2024-04-25T00:04:05.949676+0000","last_became_active":"2024-04-25T00:04:05.943718+0000","last_became_peered":"2024-04-25T00:04:05.943718+0000","last_unstale":"2024-04-25T00:04:05.949676+0000","last_undegraded":"2024-04-25T00:04:05.949676+0000","last_fullsized":"2024-04-25T00:04:05.949676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:23.344286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943771+0000","last_change":"2024-04-25T00:04:05.943771+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943771+0000","last_undegraded":"2024-04-25T00:04:05.943771+0000","last_fullsized":"2024-04-25T00:04:05.943771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958211+0000","last_change":"2024-04-25T00:04:05.954463+0000","last_active":"2024-04-25T00:04:05.958211+0000","last_peered":"2024-04-25T00:04:05.958211+0000","last_clean":"2024-04-25T00:04:05.958211+0000","last_became_active":"2024-04-25T00:04:05.954215+0000","last_became_peered":"2024-04-25T00:04:05.954215+0000","last_unstale":"2024-04-25T00:04:05.958211+0000","last_undegraded":"2024-04-25T00:04:05.958211+0000","last_fullsized":"2024-04-25T00:04:05.958211+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:23.865237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947794+0000","last_change":"2024-04-25T00:04:05.947794+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947794+0000","last_undegraded":"2024-04-25T00:04:05.947794+0000","last_fullsized":"2024-04-25T00:04:05.947794+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957333+0000","last_change":"2024-04-25T00:04:05.948415+0000","last_active":"2024-04-25T00:04:05.957333+0000","last_peered":"2024-04-25T00:04:05.957333+0000","last_clean":"2024-04-25T00:04:05.957333+0000","last_became_active":"2024-04-25T00:04:05.948148+0000","last_became_peered":"2024-04-25T00:04:05.948148+0000","last_unstale":"2024-04-25T00:04:05.957333+0000","last_undegraded":"2024-04-25T00:04:05.957333+0000","last_fullsized":"2024-04-25T00:04:05.957333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:40.441664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949243+0000","last_change":"2024-04-25T00:04:05.949243+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949243+0000","last_undegraded":"2024-04-25T00:04:05.949243+0000","last_fullsized":"2024-04-25T00:04:05.949243+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949445+0000","last_change":"2024-04-25T00:04:05.947803+0000","last_active":"2024-04-25T00:04:05.949445+0000","last_peered":"2024-04-25T00:04:05.949445+0000","last_clean":"2024-04-25T00:04:05.949445+0000","last_became_active":"2024-04-25T00:04:05.947378+0000","last_became_peered":"2024-04-25T00:04:05.947378+0000","last_unstale":"2024-04-25T00:04:05.949445+0000","last_undegraded":"2024-04-25T00:04:05.949445+0000","last_fullsized":"2024-04-25T00:04:05.949445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:02.514311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940520+0000","last_change":"2024-04-25T00:04:05.940520+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940520+0000","last_undegraded":"2024-04-25T00:04:05.940520+0000","last_fullsized":"2024-04-25T00:04:05.940520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949414+0000","last_change":"2024-04-25T00:04:05.944427+0000","last_active":"2024-04-25T00:04:05.949414+0000","last_peered":"2024-04-25T00:04:05.949414+0000","last_clean":"2024-04-25T00:04:05.949414+0000","last_became_active":"2024-04-25T00:04:05.944008+0000","last_became_peered":"2024-04-25T00:04:05.944008+0000","last_unstale":"2024-04-25T00:04:05.949414+0000","last_undegraded":"2024-04-25T00:04:05.949414+0000","last_fullsized":"2024-04-25T00:04:05.949414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:56.840548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949045+0000","last_change":"2024-04-25T00:04:05.949045+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949045+0000","last_undegraded":"2024-04-25T00:04:05.949045+0000","last_fullsized":"2024-04-25T00:04:05.949045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958180+0000","last_change":"2024-04-25T00:04:05.955068+0000","last_active":"2024-04-25T00:04:05.958180+0000","last_peered":"2024-04-25T00:04:05.958180+0000","last_clean":"2024-04-25T00:04:05.958180+0000","last_became_active":"2024-04-25T00:04:05.954821+0000","last_became_peered":"2024-04-25T00:04:05.954821+0000","last_unstale":"2024-04-25T00:04:05.958180+0000","last_undegraded":"2024-04-25T00:04:05.958180+0000","last_fullsized":"2024-04-25T00:04:05.958180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:43.009886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949448+0000","last_change":"2024-04-25T00:04:05.949448+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949448+0000","last_undegraded":"2024-04-25T00:04:05.949448+0000","last_fullsized":"2024-04-25T00:04:05.949448+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.955763+0000","last_change":"2024-04-25T00:04:05.951451+0000","last_active":"2024-04-25T00:04:05.955763+0000","last_peered":"2024-04-25T00:04:05.955763+0000","last_clean":"2024-04-25T00:04:05.955763+0000","last_became_active":"2024-04-25T00:04:05.951147+0000","last_became_peered":"2024-04-25T00:04:05.951147+0000","last_unstale":"2024-04-25T00:04:05.955763+0000","last_undegraded":"2024-04-25T00:04:05.955763+0000","last_fullsized":"2024-04-25T00:04:05.955763+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:13.641630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948977+0000","last_change":"2024-04-25T00:04:05.948977+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948977+0000","last_undegraded":"2024-04-25T00:04:05.948977+0000","last_fullsized":"2024-04-25T00:04:05.948977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953341+0000","last_change":"2024-04-25T00:04:05.949805+0000","last_active":"2024-04-25T00:04:05.953341+0000","last_peered":"2024-04-25T00:04:05.953341+0000","last_clean":"2024-04-25T00:04:05.953341+0000","last_became_active":"2024-04-25T00:04:05.949244+0000","last_became_peered":"2024-04-25T00:04:05.949244+0000","last_unstale":"2024-04-25T00:04:05.953341+0000","last_undegraded":"2024-04-25T00:04:05.953341+0000","last_fullsized":"2024-04-25T00:04:05.953341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:22.756776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943259+0000","last_change":"2024-04-25T00:04:05.943259+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943259+0000","last_undegraded":"2024-04-25T00:04:05.943259+0000","last_fullsized":"2024-04-25T00:04:05.943259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953423+0000","last_change":"2024-04-25T00:04:05.952083+0000","last_active":"2024-04-25T00:04:05.953423+0000","last_peered":"2024-04-25T00:04:05.953423+0000","last_clean":"2024-04-25T00:04:05.953423+0000","last_became_active":"2024-04-25T00:04:05.951758+0000","last_became_peered":"2024-04-25T00:04:05.951758+0000","last_unstale":"2024-04-25T00:04:05.953423+0000","last_undegraded":"2024-04-25T00:04:05.953423+0000","last_fullsized":"2024-04-25T00:04:05.953423+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:55.690985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940486+0000","last_change":"2024-04-25T00:04:05.940486+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940486+0000","last_undegraded":"2024-04-25T00:04:05.940486+0000","last_fullsized":"2024-04-25T00:04:05.940486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953947+0000","last_change":"2024-04-25T00:04:05.951264+0000","last_active":"2024-04-25T00:04:05.953947+0000","last_peered":"2024-04-25T00:04:05.953947+0000","last_clean":"2024-04-25T00:04:05.953947+0000","last_became_active":"2024-04-25T00:04:05.949917+0000","last_became_peered":"2024-04-25T00:04:05.949917+0000","last_unstale":"2024-04-25T00:04:05.953947+0000","last_undegraded":"2024-04-25T00:04:05.953947+0000","last_fullsized":"2024-04-25T00:04:05.953947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:30.057771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941657+0000","last_change":"2024-04-25T00:04:05.941657+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941657+0000","last_undegraded":"2024-04-25T00:04:05.941657+0000","last_fullsized":"2024-04-25T00:04:05.941657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.951957+0000","last_change":"2024-04-25T00:04:05.951475+0000","last_active":"2024-04-25T00:04:05.951957+0000","last_peered":"2024-04-25T00:04:05.951957+0000","last_clean":"2024-04-25T00:04:05.951957+0000","last_became_active":"2024-04-25T00:04:05.951195+0000","last_became_peered":"2024-04-25T00:04:05.951195+0000","last_unstale":"2024-04-25T00:04:05.951957+0000","last_undegraded":"2024-04-25T00:04:05.951957+0000","last_fullsized":"2024-04-25T00:04:05.951957+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:21.778025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940395+0000","last_change":"2024-04-25T00:04:05.940395+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940395+0000","last_undegraded":"2024-04-25T00:04:05.940395+0000","last_fullsized":"2024-04-25T00:04:05.940395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.950535+0000","last_change":"2024-04-25T00:04:05.949976+0000","last_active":"2024-04-25T00:04:05.950535+0000","last_peered":"2024-04-25T00:04:05.950535+0000","last_clean":"2024-04-25T00:04:05.950535+0000","last_became_active":"2024-04-25T00:04:05.948413+0000","last_became_peered":"2024-04-25T00:04:05.948413+0000","last_unstale":"2024-04-25T00:04:05.950535+0000","last_undegraded":"2024-04-25T00:04:05.950535+0000","last_fullsized":"2024-04-25T00:04:05.950535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:44.096851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947569+0000","last_change":"2024-04-25T00:04:05.947569+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947569+0000","last_undegraded":"2024-04-25T00:04:05.947569+0000","last_fullsized":"2024-04-25T00:04:05.947569+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952039+0000","last_change":"2024-04-25T00:04:05.946979+0000","last_active":"2024-04-25T00:04:05.952039+0000","last_peered":"2024-04-25T00:04:05.952039+0000","last_clean":"2024-04-25T00:04:05.952039+0000","last_became_active":"2024-04-25T00:04:05.946660+0000","last_became_peered":"2024-04-25T00:04:05.946660+0000","last_unstale":"2024-04-25T00:04:05.952039+0000","last_undegraded":"2024-04-25T00:04:05.952039+0000","last_fullsized":"2024-04-25T00:04:05.952039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:45.279979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948793+0000","last_change":"2024-04-25T00:04:05.948793+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948793+0000","last_undegraded":"2024-04-25T00:04:05.948793+0000","last_fullsized":"2024-04-25T00:04:05.948793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.955067+0000","last_change":"2024-04-25T00:04:05.953311+0000","last_active":"2024-04-25T00:04:05.955067+0000","last_peered":"2024-04-25T00:04:05.955067+0000","last_clean":"2024-04-25T00:04:05.955067+0000","last_became_active":"2024-04-25T00:04:05.953035+0000","last_became_peered":"2024-04-25T00:04:05.953035+0000","last_unstale":"2024-04-25T00:04:05.955067+0000","last_undegraded":"2024-04-25T00:04:05.955067+0000","last_fullsized":"2024-04-25T00:04:05.955067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:42.383041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949653+0000","last_change":"2024-04-25T00:04:05.949653+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949653+0000","last_undegraded":"2024-04-25T00:04:05.949653+0000","last_fullsized":"2024-04-25T00:04:05.949653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957482+0000","last_change":"2024-04-25T00:04:05.953176+0000","last_active":"2024-04-25T00:04:05.957482+0000","last_peered":"2024-04-25T00:04:05.957482+0000","last_clean":"2024-04-25T00:04:05.957482+0000","last_became_active":"2024-04-25T00:04:05.952925+0000","last_became_peered":"2024-04-25T00:04:05.952925+0000","last_unstale":"2024-04-25T00:04:05.957482+0000","last_undegraded":"2024-04-25T00:04:05.957482+0000","last_fullsized":"2024-04-25T00:04:05.957482+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.139999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947102+0000","last_change":"2024-04-25T00:04:05.947102+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947102+0000","last_undegraded":"2024-04-25T00:04:05.947102+0000","last_fullsized":"2024-04-25T00:04:05.947102+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953397+0000","last_change":"2024-04-25T00:04:05.952219+0000","last_active":"2024-04-25T00:04:05.953397+0000","last_peered":"2024-04-25T00:04:05.953397+0000","last_clean":"2024-04-25T00:04:05.953397+0000","last_became_active":"2024-04-25T00:04:05.951940+0000","last_became_peered":"2024-04-25T00:04:05.951940+0000","last_unstale":"2024-04-25T00:04:05.953397+0000","last_undegraded":"2024-04-25T00:04:05.953397+0000","last_fullsized":"2024-04-25T00:04:05.953397+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:27.978716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944150+0000","last_change":"2024-04-25T00:04:05.944150+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944150+0000","last_undegraded":"2024-04-25T00:04:05.944150+0000","last_fullsized":"2024-04-25T00:04:05.944150+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.954027+0000","last_change":"2024-04-25T00:04:05.945666+0000","last_active":"2024-04-25T00:04:05.954027+0000","last_peered":"2024-04-25T00:04:05.954027+0000","last_clean":"2024-04-25T00:04:05.954027+0000","last_became_active":"2024-04-25T00:04:05.945410+0000","last_became_peered":"2024-04-25T00:04:05.945410+0000","last_unstale":"2024-04-25T00:04:05.954027+0000","last_undegraded":"2024-04-25T00:04:05.954027+0000","last_fullsized":"2024-04-25T00:04:05.954027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:42.595520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940325+0000","last_change":"2024-04-25T00:04:05.940325+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940325+0000","last_undegraded":"2024-04-25T00:04:05.940325+0000","last_fullsized":"2024-04-25T00:04:05.940325+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960618+0000","last_change":"2024-04-25T00:04:05.957759+0000","last_active":"2024-04-25T00:04:05.960618+0000","last_peered":"2024-04-25T00:04:05.960618+0000","last_clean":"2024-04-25T00:04:05.960618+0000","last_became_active":"2024-04-25T00:04:05.957505+0000","last_became_peered":"2024-04-25T00:04:05.957505+0000","last_unstale":"2024-04-25T00:04:05.960618+0000","last_undegraded":"2024-04-25T00:04:05.960618+0000","last_fullsized":"2024-04-25T00:04:05.960618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:51.841352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949229+0000","last_change":"2024-04-25T00:04:05.949229+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949229+0000","last_undegraded":"2024-04-25T00:04:05.949229+0000","last_fullsized":"2024-04-25T00:04:05.949229+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960796+0000","last_change":"2024-04-25T00:04:05.958193+0000","last_active":"2024-04-25T00:04:05.960796+0000","last_peered":"2024-04-25T00:04:05.960796+0000","last_clean":"2024-04-25T00:04:05.960796+0000","last_became_active":"2024-04-25T00:04:05.957902+0000","last_became_peered":"2024-04-25T00:04:05.957902+0000","last_unstale":"2024-04-25T00:04:05.960796+0000","last_undegraded":"2024-04-25T00:04:05.960796+0000","last_fullsized":"2024-04-25T00:04:05.960796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:15.198528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946063+0000","last_change":"2024-04-25T00:04:05.946063+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946063+0000","last_undegraded":"2024-04-25T00:04:05.946063+0000","last_fullsized":"2024-04-25T00:04:05.946063+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960707+0000","last_change":"2024-04-25T00:04:05.956393+0000","last_active":"2024-04-25T00:04:05.960707+0000","last_peered":"2024-04-25T00:04:05.960707+0000","last_clean":"2024-04-25T00:04:05.960707+0000","last_became_active":"2024-04-25T00:04:05.955523+0000","last_became_peered":"2024-04-25T00:04:05.955523+0000","last_unstale":"2024-04-25T00:04:05.960707+0000","last_undegraded":"2024-04-25T00:04:05.960707+0000","last_fullsized":"2024-04-25T00:04:05.960707+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:57.738671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.952933+0000","last_change":"2024-04-25T00:04:05.952933+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.952933+0000","last_undegraded":"2024-04-25T00:04:05.952933+0000","last_fullsized":"2024-04-25T00:04:05.952933+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958421+0000","last_change":"2024-04-25T00:04:05.955280+0000","last_active":"2024-04-25T00:04:05.958421+0000","last_peered":"2024-04-25T00:04:05.958421+0000","last_clean":"2024-04-25T00:04:05.958421+0000","last_became_active":"2024-04-25T00:04:05.955021+0000","last_became_peered":"2024-04-25T00:04:05.955021+0000","last_unstale":"2024-04-25T00:04:05.958421+0000","last_undegraded":"2024-04-25T00:04:05.958421+0000","last_fullsized":"2024-04-25T00:04:05.958421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:39.296304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946745+0000","last_change":"2024-04-25T00:04:05.946745+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946745+0000","last_undegraded":"2024-04-25T00:04:05.946745+0000","last_fullsized":"2024-04-25T00:04:05.946745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960555+0000","last_change":"2024-04-25T00:04:05.955799+0000","last_active":"2024-04-25T00:04:05.960555+0000","last_peered":"2024-04-25T00:04:05.960555+0000","last_clean":"2024-04-25T00:04:05.960555+0000","last_became_active":"2024-04-25T00:04:05.955566+0000","last_became_peered":"2024-04-25T00:04:05.955566+0000","last_unstale":"2024-04-25T00:04:05.960555+0000","last_undegraded":"2024-04-25T00:04:05.960555+0000","last_fullsized":"2024-04-25T00:04:05.960555+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:55.307410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940520+0000","last_change":"2024-04-25T00:04:05.940520+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940520+0000","last_undegraded":"2024-04-25T00:04:05.940520+0000","last_fullsized":"2024-04-25T00:04:05.940520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.956548+0000","last_change":"2024-04-25T00:04:05.953036+0000","last_active":"2024-04-25T00:04:05.956548+0000","last_peered":"2024-04-25T00:04:05.956548+0000","last_clean":"2024-04-25T00:04:05.956548+0000","last_became_active":"2024-04-25T00:04:05.952742+0000","last_became_peered":"2024-04-25T00:04:05.952742+0000","last_unstale":"2024-04-25T00:04:05.956548+0000","last_undegraded":"2024-04-25T00:04:05.956548+0000","last_fullsized":"2024-04-25T00:04:05.956548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:45.008035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949861+0000","last_change":"2024-04-25T00:04:05.949861+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949861+0000","last_undegraded":"2024-04-25T00:04:05.949861+0000","last_fullsized":"2024-04-25T00:04:05.949861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.951271+0000","last_change":"2024-04-25T00:04:05.950505+0000","last_active":"2024-04-25T00:04:05.951271+0000","last_peered":"2024-04-25T00:04:05.951271+0000","last_clean":"2024-04-25T00:04:05.951271+0000","last_became_active":"2024-04-25T00:04:05.948719+0000","last_became_peered":"2024-04-25T00:04:05.948719+0000","last_unstale":"2024-04-25T00:04:05.951271+0000","last_undegraded":"2024-04-25T00:04:05.951271+0000","last_fullsized":"2024-04-25T00:04:05.951271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:35.048191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943304+0000","last_change":"2024-04-25T00:04:05.943304+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943304+0000","last_undegraded":"2024-04-25T00:04:05.943304+0000","last_fullsized":"2024-04-25T00:04:05.943304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953915+0000","last_change":"2024-04-25T00:04:05.949348+0000","last_active":"2024-04-25T00:04:05.953915+0000","last_peered":"2024-04-25T00:04:05.953915+0000","last_clean":"2024-04-25T00:04:05.953915+0000","last_became_active":"2024-04-25T00:04:05.948760+0000","last_became_peered":"2024-04-25T00:04:05.948760+0000","last_unstale":"2024-04-25T00:04:05.953915+0000","last_undegraded":"2024-04-25T00:04:05.953915+0000","last_fullsized":"2024-04-25T00:04:05.953915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:28.987214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939814+0000","last_change":"2024-04-25T00:04:05.939814+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939814+0000","last_undegraded":"2024-04-25T00:04:05.939814+0000","last_fullsized":"2024-04-25T00:04:05.939814+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.954516+0000","last_change":"2024-04-25T00:04:05.953709+0000","last_active":"2024-04-25T00:04:05.954516+0000","last_peered":"2024-04-25T00:04:05.954516+0000","last_clean":"2024-04-25T00:04:05.954516+0000","last_became_active":"2024-04-25T00:04:05.953453+0000","last_became_peered":"2024-04-25T00:04:05.953453+0000","last_unstale":"2024-04-25T00:04:05.954516+0000","last_undegraded":"2024-04-25T00:04:05.954516+0000","last_fullsized":"2024-04-25T00:04:05.954516+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:56.648421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946625+0000","last_change":"2024-04-25T00:04:05.946625+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946625+0000","last_undegraded":"2024-04-25T00:04:05.946625+0000","last_fullsized":"2024-04-25T00:04:05.946625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957426+0000","last_change":"2024-04-25T00:04:05.953267+0000","last_active":"2024-04-25T00:04:05.957426+0000","last_peered":"2024-04-25T00:04:05.957426+0000","last_clean":"2024-04-25T00:04:05.957426+0000","last_became_active":"2024-04-25T00:04:05.953021+0000","last_became_peered":"2024-04-25T00:04:05.953021+0000","last_unstale":"2024-04-25T00:04:05.957426+0000","last_undegraded":"2024-04-25T00:04:05.957426+0000","last_fullsized":"2024-04-25T00:04:05.957426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:37.383495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939924+0000","last_change":"2024-04-25T00:04:05.939924+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939924+0000","last_undegraded":"2024-04-25T00:04:05.939924+0000","last_fullsized":"2024-04-25T00:04:05.939924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949496+0000","last_change":"2024-04-25T00:04:05.944307+0000","last_active":"2024-04-25T00:04:05.949496+0000","last_peered":"2024-04-25T00:04:05.949496+0000","last_clean":"2024-04-25T00:04:05.949496+0000","last_became_active":"2024-04-25T00:04:05.943618+0000","last_became_peered":"2024-04-25T00:04:05.943618+0000","last_unstale":"2024-04-25T00:04:05.949496+0000","last_undegraded":"2024-04-25T00:04:05.949496+0000","last_fullsized":"2024-04-25T00:04:05.949496+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:48.774779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948590+0000","last_change":"2024-04-25T00:04:05.948590+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948590+0000","last_undegraded":"2024-04-25T00:04:05.948590+0000","last_fullsized":"2024-04-25T00:04:05.948590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952860+0000","last_change":"2024-04-25T00:04:05.949003+0000","last_active":"2024-04-25T00:04:05.952860+0000","last_peered":"2024-04-25T00:04:05.952860+0000","last_clean":"2024-04-25T00:04:05.952860+0000","last_became_active":"2024-04-25T00:04:05.948731+0000","last_became_peered":"2024-04-25T00:04:05.948731+0000","last_unstale":"2024-04-25T00:04:05.952860+0000","last_undegraded":"2024-04-25T00:04:05.952860+0000","last_fullsized":"2024-04-25T00:04:05.952860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:20.845105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942891+0000","last_change":"2024-04-25T00:04:05.942891+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942891+0000","last_undegraded":"2024-04-25T00:04:05.942891+0000","last_fullsized":"2024-04-25T00:04:05.942891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949915+0000","last_change":"2024-04-25T00:04:05.949157+0000","last_active":"2024-04-25T00:04:05.949915+0000","last_peered":"2024-04-25T00:04:05.949915+0000","last_clean":"2024-04-25T00:04:05.949915+0000","last_became_active":"2024-04-25T00:04:05.948879+0000","last_became_peered":"2024-04-25T00:04:05.948879+0000","last_unstale":"2024-04-25T00:04:05.949915+0000","last_undegraded":"2024-04-25T00:04:05.949915+0000","last_fullsized":"2024-04-25T00:04:05.949915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:26.961912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948850+0000","last_change":"2024-04-25T00:04:05.948850+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948850+0000","last_undegraded":"2024-04-25T00:04:05.948850+0000","last_fullsized":"2024-04-25T00:04:05.948850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957511+0000","last_change":"2024-04-25T00:04:05.957149+0000","last_active":"2024-04-25T00:04:05.957511+0000","last_peered":"2024-04-25T00:04:05.957511+0000","last_clean":"2024-04-25T00:04:05.957511+0000","last_became_active":"2024-04-25T00:04:05.956870+0000","last_became_peered":"2024-04-25T00:04:05.956870+0000","last_unstale":"2024-04-25T00:04:05.957511+0000","last_undegraded":"2024-04-25T00:04:05.957511+0000","last_fullsized":"2024-04-25T00:04:05.957511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:41.561274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":297,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.950293+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:04:05.950293+0000","last_peered":"2024-04-25T00:04:05.950293+0000","last_clean":"2024-04-25T00:04:05.950293+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:04:05.950293+0000","last_undegraded":"2024-04-25T00:04:05.950293+0000","last_fullsized":"2024-04-25T00:04:05.950293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55040,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316800,"statfs":{"total":96636764160,"available":96580403200,"internally_reserved":0,"allocated":1687552,"data_stored":1059333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58620,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313220,"statfs":{"total":96636764160,"available":96576737280,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40576,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331264,"statfs":{"total":96636764160,"available":96595214336,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50304,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321536,"statfs":{"total":96636764160,"available":96585252864,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41164,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330676,"statfs":{"total":96636764160,"available":96594612224,"internally_reserved":0,"allocated":1699840,"data_stored":1065608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:04:10.241 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:04:10.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.375+0000 7f6eda2d6640 1 Processor -- start 2024-04-25T00:04:10.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.375+0000 7f6eda2d6640 1 -- start start 2024-04-25T00:04:10.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6eda2d6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ed4144d60 0x7f6ed40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6eda2d6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed40cfad0 con 0x7f6ed406e6e0 2024-04-25T00:04:10.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6eda2d6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed40cf230 con 0x7f6ed41453c0 2024-04-25T00:04:10.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6eda2d6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed4145c40 con 0x7f6ed4144d60 2024-04-25T00:04:10.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6ed3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ed4144d60 0x7f6ed40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6ed8873640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6ed406e6e0 0x7f6ed4071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34746/0 (socket says 172.21.15.113:34746) 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6ed8873640 1 -- 172.21.15.113:0/1497335679 learned_addr learned my addr 172.21.15.113:0/1497335679 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6ed3fff640 1 -- 172.21.15.113:0/1497335679 >> v1:172.21.15.178:6789/0 conn(0x7f6ed41453c0 legacy=0x7f6ed406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6ed3fff640 1 -- 172.21.15.113:0/1497335679 >> v1:172.21.15.113:6789/0 conn(0x7f6ed406e6e0 legacy=0x7f6ed4071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6ed2ffd640 1 -- 172.21.15.113:0/1497335679 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292790555 0 0) 0x7f6ed40cfad0 con 0x7f6ed406e6e0 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.376+0000 7f6ed3fff640 1 -- 172.21.15.113:0/1497335679 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed40cf510 con 0x7f6ed4144d60 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6ed3fff640 1 --2- 172.21.15.113:0/1497335679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed4144d60 0x7f6ed40b5390 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40097a0 tx=0x7f6ec402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298707aa19f4e309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6ed2ffd640 1 -- 172.21.15.113:0/1497335679 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4037070 con 0x7f6ed4144d60 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6ed2ffd640 1 -- 172.21.15.113:0/1497335679 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4032440 con 0x7f6ed4144d60 2024-04-25T00:04:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6ed2ffd640 1 -- 172.21.15.113:0/1497335679 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4032810 con 0x7f6ed4144d60 2024-04-25T00:04:10.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1497335679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed4144d60 msgr2=0x7f6ed40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1497335679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed4144d60 0x7f6ed40b5390 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40097a0 tx=0x7f6ec402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1497335679 shutdown_connections 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.377+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1497335679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed4144d60 0x7f6ed40b5390 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.378+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1497335679 >> 172.21.15.113:0/1497335679 conn(0x7f6ed40b1a50 msgr2=0x7f6ed40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.378+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1497335679 shutdown_connections 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.378+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1497335679 wait complete. 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 Processor -- start 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 -- start start 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed406e6e0 0x7f6ed413c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ed4144d60 0x7f6ed414d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ed41453c0 0x7f6ed4142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4145140 con 0x7f6ed406e6e0 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4070e10 con 0x7f6ed4144d60 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.379+0000 7f6eda2d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4071a40 con 0x7f6ed41453c0 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed406e6e0 0x7f6ed413c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed406e6e0 0x7f6ed413c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53952/0 (socket says 172.21.15.113:53952) 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 -- 172.21.15.113:0/1547827357 learned_addr learned my addr 172.21.15.113:0/1547827357 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed8873640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ed41453c0 0x7f6ed4142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed37fe640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ed4144d60 0x7f6ed414d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 -- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ed41453c0 msgr2=0x7f6ed4142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ed41453c0 0x7f6ed4142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 -- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ed4144d60 msgr2=0x7f6ed414d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ed4144d60 0x7f6ed414d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 -- 172.21.15.113:0/1547827357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed413cbd0 con 0x7f6ed406e6e0 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed8873640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ed41453c0 0x7f6ed4142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed37fe640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ed4144d60 0x7f6ed414d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed3fff640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed406e6e0 0x7f6ed413c690 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4000c00 tx=0x7f6ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed0ff9640 1 -- 172.21.15.113:0/1547827357 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4037040 con 0x7f6ed406e6e0 2024-04-25T00:04:10.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed0ff9640 1 -- 172.21.15.113:0/1547827357 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4042070 con 0x7f6ed406e6e0 2024-04-25T00:04:10.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed414dea0 con 0x7f6ed406e6e0 2024-04-25T00:04:10.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6ed0ff9640 1 -- 172.21.15.113:0/1547827357 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4030a70 con 0x7f6ed406e6e0 2024-04-25T00:04:10.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.380+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed415cd70 con 0x7f6ed406e6e0 2024-04-25T00:04:10.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.381+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed40691a0 con 0x7f6ed406e6e0 2024-04-25T00:04:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.382+0000 7f6ed0ff9640 1 -- 172.21.15.113:0/1547827357 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ec4030c10 con 0x7f6ed406e6e0 2024-04-25T00:04:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.382+0000 7f6ed0ff9640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6ec006be30 0x7f6ec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.382+0000 7f6ed0ff9640 1 -- 172.21.15.113:0/1547827357 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f6ec40ab920 con 0x7f6ed406e6e0 2024-04-25T00:04:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.382+0000 7f6ed37fe640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6ec006be30 0x7f6ec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.383+0000 7f6ed37fe640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6ec006be30 0x7f6ec006e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8007fd0 tx=0x7f6ec8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.387+0000 7f6ed0ff9640 1 -- 172.21.15.113:0/1547827357 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ed413cbd0 con 0x7f6ed406e6e0 2024-04-25T00:04:10.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.556+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ed40b27e0 con 0x7f6ec006be30 2024-04-25T00:04:10.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.562+0000 7f6ed0ff9640 1 -- 172.21.15.113:0/1547827357 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246765 (secure 0 0 0) 0x7f6ed40b27e0 con 0x7f6ec006be30 2024-04-25T00:04:10.565 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:10.566 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6ec006be30 msgr2=0x7f6ec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6ec006be30 0x7f6ec006e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8007fd0 tx=0x7f6ec8009040 comp rx=0 tx=0).stop 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed406e6e0 msgr2=0x7f6ed413c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed406e6e0 0x7f6ed413c690 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4000c00 tx=0x7f6ec40046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 shutdown_connections 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6ec006be30 0x7f6ec006e2f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ed4144d60 0x7f6ed414d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ed41453c0 0x7f6ed4142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 --2- 172.21.15.113:0/1547827357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ed406e6e0 0x7f6ed413c690 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.565+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 >> 172.21.15.113:0/1547827357 conn(0x7f6ed40b1a50 msgr2=0x7f6ed40ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:10.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.566+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 shutdown_connections 2024-04-25T00:04:10.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.566+0000 7f6eda2d6640 1 -- 172.21.15.113:0/1547827357 wait complete. 2024-04-25T00:04:10.577 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-04-25T00:04:09.180995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":382712,"kb_used_data":9716,"kb_used_omap":12,"kb_used_meta":372851,"kb_avail":754592008,"statfs":{"total":773094113280,"available":772702216192,"internally_reserved":0,"allocated":9949184,"data_stored":4938731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":381799984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.249291"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944580+0000","last_change":"2024-04-25T00:04:05.944580+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944580+0000","last_undegraded":"2024-04-25T00:04:05.944580+0000","last_fullsized":"2024-04-25T00:04:05.944580+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949172+0000","last_change":"2024-04-25T00:04:05.949172+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949172+0000","last_undegraded":"2024-04-25T00:04:05.949172+0000","last_fullsized":"2024-04-25T00:04:05.949172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948454+0000","last_change":"2024-04-25T00:04:05.948454+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948454+0000","last_undegraded":"2024-04-25T00:04:05.948454+0000","last_fullsized":"2024-04-25T00:04:05.948454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949735+0000","last_change":"2024-04-25T00:04:05.949735+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949735+0000","last_undegraded":"2024-04-25T00:04:05.949735+0000","last_fullsized":"2024-04-25T00:04:05.949735+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941249+0000","last_change":"2024-04-25T00:04:05.941249+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941249+0000","last_undegraded":"2024-04-25T00:04:05.941249+0000","last_fullsized":"2024-04-25T00:04:05.941249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949527+0000","last_change":"2024-04-25T00:04:05.949527+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949527+0000","last_undegraded":"2024-04-25T00:04:05.949527+0000","last_fullsized":"2024-04-25T00:04:05.949527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.954626+0000","last_change":"2024-04-25T00:04:05.954626+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.954626+0000","last_undegraded":"2024-04-25T00:04:05.954626+0000","last_fullsized":"2024-04-25T00:04:05.954626+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942745+0000","last_change":"2024-04-25T00:04:05.942745+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942745+0000","last_undegraded":"2024-04-25T00:04:05.942745+0000","last_fullsized":"2024-04-25T00:04:05.942745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.945401+0000","last_change":"2024-04-25T00:04:05.945401+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.945401+0000","last_undegraded":"2024-04-25T00:04:05.945401+0000","last_fullsized":"2024-04-25T00:04:05.945401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942883+0000","last_change":"2024-04-25T00:04:05.942883+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942883+0000","last_undegraded":"2024-04-25T00:04:05.942883+0000","last_fullsized":"2024-04-25T00:04:05.942883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949288+0000","last_change":"2024-04-25T00:04:05.949288+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949288+0000","last_undegraded":"2024-04-25T00:04:05.949288+0000","last_fullsized":"2024-04-25T00:04:05.949288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.945295+0000","last_change":"2024-04-25T00:04:05.945295+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.945295+0000","last_undegraded":"2024-04-25T00:04:05.945295+0000","last_fullsized":"2024-04-25T00:04:05.945295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948113+0000","last_change":"2024-04-25T00:04:05.948113+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948113+0000","last_undegraded":"2024-04-25T00:04:05.948113+0000","last_fullsized":"2024-04-25T00:04:05.948113+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949697+0000","last_change":"2024-04-25T00:04:05.949697+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949697+0000","last_undegraded":"2024-04-25T00:04:05.949697+0000","last_fullsized":"2024-04-25T00:04:05.949697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949882+0000","last_change":"2024-04-25T00:04:05.949882+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949882+0000","last_undegraded":"2024-04-25T00:04:05.949882+0000","last_fullsized":"2024-04-25T00:04:05.949882+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939754+0000","last_change":"2024-04-25T00:04:05.939754+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939754+0000","last_undegraded":"2024-04-25T00:04:05.939754+0000","last_fullsized":"2024-04-25T00:04:05.939754+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939922+0000","last_change":"2024-04-25T00:04:05.939922+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939922+0000","last_undegraded":"2024-04-25T00:04:05.939922+0000","last_fullsized":"2024-04-25T00:04:05.939922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950188+0000","last_change":"2024-04-25T00:04:05.950188+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950188+0000","last_undegraded":"2024-04-25T00:04:05.950188+0000","last_fullsized":"2024-04-25T00:04:05.950188+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939943+0000","last_change":"2024-04-25T00:04:05.939943+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939943+0000","last_undegraded":"2024-04-25T00:04:05.939943+0000","last_fullsized":"2024-04-25T00:04:05.939943+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949340+0000","last_change":"2024-04-25T00:04:05.949340+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949340+0000","last_undegraded":"2024-04-25T00:04:05.949340+0000","last_fullsized":"2024-04-25T00:04:05.949340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.954404+0000","last_change":"2024-04-25T00:04:05.954404+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.954404+0000","last_undegraded":"2024-04-25T00:04:05.954404+0000","last_fullsized":"2024-04-25T00:04:05.954404+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950060+0000","last_change":"2024-04-25T00:04:05.950060+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950060+0000","last_undegraded":"2024-04-25T00:04:05.950060+0000","last_fullsized":"2024-04-25T00:04:05.950060+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943371+0000","last_change":"2024-04-25T00:04:05.943371+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943371+0000","last_undegraded":"2024-04-25T00:04:05.943371+0000","last_fullsized":"2024-04-25T00:04:05.943371+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.953326+0000","last_change":"2024-04-25T00:04:05.953326+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.953326+0000","last_undegraded":"2024-04-25T00:04:05.953326+0000","last_fullsized":"2024-04-25T00:04:05.953326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.950169+0000","last_change":"2024-04-25T00:04:05.950169+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.950169+0000","last_undegraded":"2024-04-25T00:04:05.950169+0000","last_fullsized":"2024-04-25T00:04:05.950169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948459+0000","last_change":"2024-04-25T00:04:05.948459+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948459+0000","last_undegraded":"2024-04-25T00:04:05.948459+0000","last_fullsized":"2024-04-25T00:04:05.948459+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942111+0000","last_change":"2024-04-25T00:04:05.942111+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942111+0000","last_undegraded":"2024-04-25T00:04:05.942111+0000","last_fullsized":"2024-04-25T00:04:05.942111+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944848+0000","last_change":"2024-04-25T00:04:05.944848+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944848+0000","last_undegraded":"2024-04-25T00:04:05.944848+0000","last_fullsized":"2024-04-25T00:04:05.944848+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948830+0000","last_change":"2024-04-25T00:04:05.948830+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948830+0000","last_undegraded":"2024-04-25T00:04:05.948830+0000","last_fullsized":"2024-04-25T00:04:05.948830+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949664+0000","last_change":"2024-04-25T00:04:05.949664+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949664+0000","last_undegraded":"2024-04-25T00:04:05.949664+0000","last_fullsized":"2024-04-25T00:04:05.949664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939777+0000","last_change":"2024-04-25T00:04:05.939777+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939777+0000","last_undegraded":"2024-04-25T00:04:05.939777+0000","last_fullsized":"2024-04-25T00:04:05.939777+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943940+0000","last_change":"2024-04-25T00:04:05.943940+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943940+0000","last_undegraded":"2024-04-25T00:04:05.943940+0000","last_fullsized":"2024-04-25T00:04:05.943940+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941083+0000","last_change":"2024-04-25T00:04:05.941083+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941083+0000","last_undegraded":"2024-04-25T00:04:05.941083+0000","last_fullsized":"2024-04-25T00:04:05.941083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952826+0000","last_change":"2024-04-25T00:04:05.949584+0000","last_active":"2024-04-25T00:04:05.952826+0000","last_peered":"2024-04-25T00:04:05.952826+0000","last_clean":"2024-04-25T00:04:05.952826+0000","last_became_active":"2024-04-25T00:04:05.948893+0000","last_became_peered":"2024-04-25T00:04:05.948893+0000","last_unstale":"2024-04-25T00:04:05.952826+0000","last_undegraded":"2024-04-25T00:04:05.952826+0000","last_fullsized":"2024-04-25T00:04:05.952826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:42.347304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943649+0000","last_change":"2024-04-25T00:04:05.943649+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943649+0000","last_undegraded":"2024-04-25T00:04:05.943649+0000","last_fullsized":"2024-04-25T00:04:05.943649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949676+0000","last_change":"2024-04-25T00:04:05.943990+0000","last_active":"2024-04-25T00:04:05.949676+0000","last_peered":"2024-04-25T00:04:05.949676+0000","last_clean":"2024-04-25T00:04:05.949676+0000","last_became_active":"2024-04-25T00:04:05.943718+0000","last_became_peered":"2024-04-25T00:04:05.943718+0000","last_unstale":"2024-04-25T00:04:05.949676+0000","last_undegraded":"2024-04-25T00:04:05.949676+0000","last_fullsized":"2024-04-25T00:04:05.949676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:23.344286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943771+0000","last_change":"2024-04-25T00:04:05.943771+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943771+0000","last_undegraded":"2024-04-25T00:04:05.943771+0000","last_fullsized":"2024-04-25T00:04:05.943771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958211+0000","last_change":"2024-04-25T00:04:05.954463+0000","last_active":"2024-04-25T00:04:05.958211+0000","last_peered":"2024-04-25T00:04:05.958211+0000","last_clean":"2024-04-25T00:04:05.958211+0000","last_became_active":"2024-04-25T00:04:05.954215+0000","last_became_peered":"2024-04-25T00:04:05.954215+0000","last_unstale":"2024-04-25T00:04:05.958211+0000","last_undegraded":"2024-04-25T00:04:05.958211+0000","last_fullsized":"2024-04-25T00:04:05.958211+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:23.865237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947794+0000","last_change":"2024-04-25T00:04:05.947794+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947794+0000","last_undegraded":"2024-04-25T00:04:05.947794+0000","last_fullsized":"2024-04-25T00:04:05.947794+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957333+0000","last_change":"2024-04-25T00:04:05.948415+0000","last_active":"2024-04-25T00:04:05.957333+0000","last_peered":"2024-04-25T00:04:05.957333+0000","last_clean":"2024-04-25T00:04:05.957333+0000","last_became_active":"2024-04-25T00:04:05.948148+0000","last_became_peered":"2024-04-25T00:04:05.948148+0000","last_unstale":"2024-04-25T00:04:05.957333+0000","last_undegraded":"2024-04-25T00:04:05.957333+0000","last_fullsized":"2024-04-25T00:04:05.957333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:40.441664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949243+0000","last_change":"2024-04-25T00:04:05.949243+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949243+0000","last_undegraded":"2024-04-25T00:04:05.949243+0000","last_fullsized":"2024-04-25T00:04:05.949243+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949445+0000","last_change":"2024-04-25T00:04:05.947803+0000","last_active":"2024-04-25T00:04:05.949445+0000","last_peered":"2024-04-25T00:04:05.949445+0000","last_clean":"2024-04-25T00:04:05.949445+0000","last_became_active":"2024-04-25T00:04:05.947378+0000","last_became_peered":"2024-04-25T00:04:05.947378+0000","last_unstale":"2024-04-25T00:04:05.949445+0000","last_undegraded":"2024-04-25T00:04:05.949445+0000","last_fullsized":"2024-04-25T00:04:05.949445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:02.514311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940520+0000","last_change":"2024-04-25T00:04:05.940520+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940520+0000","last_undegraded":"2024-04-25T00:04:05.940520+0000","last_fullsized":"2024-04-25T00:04:05.940520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949414+0000","last_change":"2024-04-25T00:04:05.944427+0000","last_active":"2024-04-25T00:04:05.949414+0000","last_peered":"2024-04-25T00:04:05.949414+0000","last_clean":"2024-04-25T00:04:05.949414+0000","last_became_active":"2024-04-25T00:04:05.944008+0000","last_became_peered":"2024-04-25T00:04:05.944008+0000","last_unstale":"2024-04-25T00:04:05.949414+0000","last_undegraded":"2024-04-25T00:04:05.949414+0000","last_fullsized":"2024-04-25T00:04:05.949414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:56.840548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949045+0000","last_change":"2024-04-25T00:04:05.949045+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949045+0000","last_undegraded":"2024-04-25T00:04:05.949045+0000","last_fullsized":"2024-04-25T00:04:05.949045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958180+0000","last_change":"2024-04-25T00:04:05.955068+0000","last_active":"2024-04-25T00:04:05.958180+0000","last_peered":"2024-04-25T00:04:05.958180+0000","last_clean":"2024-04-25T00:04:05.958180+0000","last_became_active":"2024-04-25T00:04:05.954821+0000","last_became_peered":"2024-04-25T00:04:05.954821+0000","last_unstale":"2024-04-25T00:04:05.958180+0000","last_undegraded":"2024-04-25T00:04:05.958180+0000","last_fullsized":"2024-04-25T00:04:05.958180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:43.009886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949448+0000","last_change":"2024-04-25T00:04:05.949448+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949448+0000","last_undegraded":"2024-04-25T00:04:05.949448+0000","last_fullsized":"2024-04-25T00:04:05.949448+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.955763+0000","last_change":"2024-04-25T00:04:05.951451+0000","last_active":"2024-04-25T00:04:05.955763+0000","last_peered":"2024-04-25T00:04:05.955763+0000","last_clean":"2024-04-25T00:04:05.955763+0000","last_became_active":"2024-04-25T00:04:05.951147+0000","last_became_peered":"2024-04-25T00:04:05.951147+0000","last_unstale":"2024-04-25T00:04:05.955763+0000","last_undegraded":"2024-04-25T00:04:05.955763+0000","last_fullsized":"2024-04-25T00:04:05.955763+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:13.641630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948977+0000","last_change":"2024-04-25T00:04:05.948977+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948977+0000","last_undegraded":"2024-04-25T00:04:05.948977+0000","last_fullsized":"2024-04-25T00:04:05.948977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953341+0000","last_change":"2024-04-25T00:04:05.949805+0000","last_active":"2024-04-25T00:04:05.953341+0000","last_peered":"2024-04-25T00:04:05.953341+0000","last_clean":"2024-04-25T00:04:05.953341+0000","last_became_active":"2024-04-25T00:04:05.949244+0000","last_became_peered":"2024-04-25T00:04:05.949244+0000","last_unstale":"2024-04-25T00:04:05.953341+0000","last_undegraded":"2024-04-25T00:04:05.953341+0000","last_fullsized":"2024-04-25T00:04:05.953341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:22.756776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943259+0000","last_change":"2024-04-25T00:04:05.943259+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943259+0000","last_undegraded":"2024-04-25T00:04:05.943259+0000","last_fullsized":"2024-04-25T00:04:05.943259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953423+0000","last_change":"2024-04-25T00:04:05.952083+0000","last_active":"2024-04-25T00:04:05.953423+0000","last_peered":"2024-04-25T00:04:05.953423+0000","last_clean":"2024-04-25T00:04:05.953423+0000","last_became_active":"2024-04-25T00:04:05.951758+0000","last_became_peered":"2024-04-25T00:04:05.951758+0000","last_unstale":"2024-04-25T00:04:05.953423+0000","last_undegraded":"2024-04-25T00:04:05.953423+0000","last_fullsized":"2024-04-25T00:04:05.953423+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:55.690985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940486+0000","last_change":"2024-04-25T00:04:05.940486+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940486+0000","last_undegraded":"2024-04-25T00:04:05.940486+0000","last_fullsized":"2024-04-25T00:04:05.940486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953947+0000","last_change":"2024-04-25T00:04:05.951264+0000","last_active":"2024-04-25T00:04:05.953947+0000","last_peered":"2024-04-25T00:04:05.953947+0000","last_clean":"2024-04-25T00:04:05.953947+0000","last_became_active":"2024-04-25T00:04:05.949917+0000","last_became_peered":"2024-04-25T00:04:05.949917+0000","last_unstale":"2024-04-25T00:04:05.953947+0000","last_undegraded":"2024-04-25T00:04:05.953947+0000","last_fullsized":"2024-04-25T00:04:05.953947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:30.057771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.941657+0000","last_change":"2024-04-25T00:04:05.941657+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.941657+0000","last_undegraded":"2024-04-25T00:04:05.941657+0000","last_fullsized":"2024-04-25T00:04:05.941657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.951957+0000","last_change":"2024-04-25T00:04:05.951475+0000","last_active":"2024-04-25T00:04:05.951957+0000","last_peered":"2024-04-25T00:04:05.951957+0000","last_clean":"2024-04-25T00:04:05.951957+0000","last_became_active":"2024-04-25T00:04:05.951195+0000","last_became_peered":"2024-04-25T00:04:05.951195+0000","last_unstale":"2024-04-25T00:04:05.951957+0000","last_undegraded":"2024-04-25T00:04:05.951957+0000","last_fullsized":"2024-04-25T00:04:05.951957+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:21.778025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940395+0000","last_change":"2024-04-25T00:04:05.940395+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940395+0000","last_undegraded":"2024-04-25T00:04:05.940395+0000","last_fullsized":"2024-04-25T00:04:05.940395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.950535+0000","last_change":"2024-04-25T00:04:05.949976+0000","last_active":"2024-04-25T00:04:05.950535+0000","last_peered":"2024-04-25T00:04:05.950535+0000","last_clean":"2024-04-25T00:04:05.950535+0000","last_became_active":"2024-04-25T00:04:05.948413+0000","last_became_peered":"2024-04-25T00:04:05.948413+0000","last_unstale":"2024-04-25T00:04:05.950535+0000","last_undegraded":"2024-04-25T00:04:05.950535+0000","last_fullsized":"2024-04-25T00:04:05.950535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:44.096851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947569+0000","last_change":"2024-04-25T00:04:05.947569+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947569+0000","last_undegraded":"2024-04-25T00:04:05.947569+0000","last_fullsized":"2024-04-25T00:04:05.947569+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952039+0000","last_change":"2024-04-25T00:04:05.946979+0000","last_active":"2024-04-25T00:04:05.952039+0000","last_peered":"2024-04-25T00:04:05.952039+0000","last_clean":"2024-04-25T00:04:05.952039+0000","last_became_active":"2024-04-25T00:04:05.946660+0000","last_became_peered":"2024-04-25T00:04:05.946660+0000","last_unstale":"2024-04-25T00:04:05.952039+0000","last_undegraded":"2024-04-25T00:04:05.952039+0000","last_fullsized":"2024-04-25T00:04:05.952039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:45.279979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948793+0000","last_change":"2024-04-25T00:04:05.948793+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948793+0000","last_undegraded":"2024-04-25T00:04:05.948793+0000","last_fullsized":"2024-04-25T00:04:05.948793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.955067+0000","last_change":"2024-04-25T00:04:05.953311+0000","last_active":"2024-04-25T00:04:05.955067+0000","last_peered":"2024-04-25T00:04:05.955067+0000","last_clean":"2024-04-25T00:04:05.955067+0000","last_became_active":"2024-04-25T00:04:05.953035+0000","last_became_peered":"2024-04-25T00:04:05.953035+0000","last_unstale":"2024-04-25T00:04:05.955067+0000","last_undegraded":"2024-04-25T00:04:05.955067+0000","last_fullsized":"2024-04-25T00:04:05.955067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:42.383041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949653+0000","last_change":"2024-04-25T00:04:05.949653+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949653+0000","last_undegraded":"2024-04-25T00:04:05.949653+0000","last_fullsized":"2024-04-25T00:04:05.949653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957482+0000","last_change":"2024-04-25T00:04:05.953176+0000","last_active":"2024-04-25T00:04:05.957482+0000","last_peered":"2024-04-25T00:04:05.957482+0000","last_clean":"2024-04-25T00:04:05.957482+0000","last_became_active":"2024-04-25T00:04:05.952925+0000","last_became_peered":"2024-04-25T00:04:05.952925+0000","last_unstale":"2024-04-25T00:04:05.957482+0000","last_undegraded":"2024-04-25T00:04:05.957482+0000","last_fullsized":"2024-04-25T00:04:05.957482+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.139999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.947102+0000","last_change":"2024-04-25T00:04:05.947102+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.947102+0000","last_undegraded":"2024-04-25T00:04:05.947102+0000","last_fullsized":"2024-04-25T00:04:05.947102+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953397+0000","last_change":"2024-04-25T00:04:05.952219+0000","last_active":"2024-04-25T00:04:05.953397+0000","last_peered":"2024-04-25T00:04:05.953397+0000","last_clean":"2024-04-25T00:04:05.953397+0000","last_became_active":"2024-04-25T00:04:05.951940+0000","last_became_peered":"2024-04-25T00:04:05.951940+0000","last_unstale":"2024-04-25T00:04:05.953397+0000","last_undegraded":"2024-04-25T00:04:05.953397+0000","last_fullsized":"2024-04-25T00:04:05.953397+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:27.978716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.944150+0000","last_change":"2024-04-25T00:04:05.944150+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.944150+0000","last_undegraded":"2024-04-25T00:04:05.944150+0000","last_fullsized":"2024-04-25T00:04:05.944150+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.954027+0000","last_change":"2024-04-25T00:04:05.945666+0000","last_active":"2024-04-25T00:04:05.954027+0000","last_peered":"2024-04-25T00:04:05.954027+0000","last_clean":"2024-04-25T00:04:05.954027+0000","last_became_active":"2024-04-25T00:04:05.945410+0000","last_became_peered":"2024-04-25T00:04:05.945410+0000","last_unstale":"2024-04-25T00:04:05.954027+0000","last_undegraded":"2024-04-25T00:04:05.954027+0000","last_fullsized":"2024-04-25T00:04:05.954027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:42.595520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940325+0000","last_change":"2024-04-25T00:04:05.940325+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940325+0000","last_undegraded":"2024-04-25T00:04:05.940325+0000","last_fullsized":"2024-04-25T00:04:05.940325+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960618+0000","last_change":"2024-04-25T00:04:05.957759+0000","last_active":"2024-04-25T00:04:05.960618+0000","last_peered":"2024-04-25T00:04:05.960618+0000","last_clean":"2024-04-25T00:04:05.960618+0000","last_became_active":"2024-04-25T00:04:05.957505+0000","last_became_peered":"2024-04-25T00:04:05.957505+0000","last_unstale":"2024-04-25T00:04:05.960618+0000","last_undegraded":"2024-04-25T00:04:05.960618+0000","last_fullsized":"2024-04-25T00:04:05.960618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:51.841352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949229+0000","last_change":"2024-04-25T00:04:05.949229+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949229+0000","last_undegraded":"2024-04-25T00:04:05.949229+0000","last_fullsized":"2024-04-25T00:04:05.949229+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960796+0000","last_change":"2024-04-25T00:04:05.958193+0000","last_active":"2024-04-25T00:04:05.960796+0000","last_peered":"2024-04-25T00:04:05.960796+0000","last_clean":"2024-04-25T00:04:05.960796+0000","last_became_active":"2024-04-25T00:04:05.957902+0000","last_became_peered":"2024-04-25T00:04:05.957902+0000","last_unstale":"2024-04-25T00:04:05.960796+0000","last_undegraded":"2024-04-25T00:04:05.960796+0000","last_fullsized":"2024-04-25T00:04:05.960796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:15.198528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946063+0000","last_change":"2024-04-25T00:04:05.946063+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946063+0000","last_undegraded":"2024-04-25T00:04:05.946063+0000","last_fullsized":"2024-04-25T00:04:05.946063+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960707+0000","last_change":"2024-04-25T00:04:05.956393+0000","last_active":"2024-04-25T00:04:05.960707+0000","last_peered":"2024-04-25T00:04:05.960707+0000","last_clean":"2024-04-25T00:04:05.960707+0000","last_became_active":"2024-04-25T00:04:05.955523+0000","last_became_peered":"2024-04-25T00:04:05.955523+0000","last_unstale":"2024-04-25T00:04:05.960707+0000","last_undegraded":"2024-04-25T00:04:05.960707+0000","last_fullsized":"2024-04-25T00:04:05.960707+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:57.738671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.952933+0000","last_change":"2024-04-25T00:04:05.952933+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.952933+0000","last_undegraded":"2024-04-25T00:04:05.952933+0000","last_fullsized":"2024-04-25T00:04:05.952933+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.958421+0000","last_change":"2024-04-25T00:04:05.955280+0000","last_active":"2024-04-25T00:04:05.958421+0000","last_peered":"2024-04-25T00:04:05.958421+0000","last_clean":"2024-04-25T00:04:05.958421+0000","last_became_active":"2024-04-25T00:04:05.955021+0000","last_became_peered":"2024-04-25T00:04:05.955021+0000","last_unstale":"2024-04-25T00:04:05.958421+0000","last_undegraded":"2024-04-25T00:04:05.958421+0000","last_fullsized":"2024-04-25T00:04:05.958421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:39.296304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946745+0000","last_change":"2024-04-25T00:04:05.946745+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946745+0000","last_undegraded":"2024-04-25T00:04:05.946745+0000","last_fullsized":"2024-04-25T00:04:05.946745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.960555+0000","last_change":"2024-04-25T00:04:05.955799+0000","last_active":"2024-04-25T00:04:05.960555+0000","last_peered":"2024-04-25T00:04:05.960555+0000","last_clean":"2024-04-25T00:04:05.960555+0000","last_became_active":"2024-04-25T00:04:05.955566+0000","last_became_peered":"2024-04-25T00:04:05.955566+0000","last_unstale":"2024-04-25T00:04:05.960555+0000","last_undegraded":"2024-04-25T00:04:05.960555+0000","last_fullsized":"2024-04-25T00:04:05.960555+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:55.307410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.940520+0000","last_change":"2024-04-25T00:04:05.940520+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.940520+0000","last_undegraded":"2024-04-25T00:04:05.940520+0000","last_fullsized":"2024-04-25T00:04:05.940520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.956548+0000","last_change":"2024-04-25T00:04:05.953036+0000","last_active":"2024-04-25T00:04:05.956548+0000","last_peered":"2024-04-25T00:04:05.956548+0000","last_clean":"2024-04-25T00:04:05.956548+0000","last_became_active":"2024-04-25T00:04:05.952742+0000","last_became_peered":"2024-04-25T00:04:05.952742+0000","last_unstale":"2024-04-25T00:04:05.956548+0000","last_undegraded":"2024-04-25T00:04:05.956548+0000","last_fullsized":"2024-04-25T00:04:05.956548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:45.008035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.949861+0000","last_change":"2024-04-25T00:04:05.949861+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.949861+0000","last_undegraded":"2024-04-25T00:04:05.949861+0000","last_fullsized":"2024-04-25T00:04:05.949861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.951271+0000","last_change":"2024-04-25T00:04:05.950505+0000","last_active":"2024-04-25T00:04:05.951271+0000","last_peered":"2024-04-25T00:04:05.951271+0000","last_clean":"2024-04-25T00:04:05.951271+0000","last_became_active":"2024-04-25T00:04:05.948719+0000","last_became_peered":"2024-04-25T00:04:05.948719+0000","last_unstale":"2024-04-25T00:04:05.951271+0000","last_undegraded":"2024-04-25T00:04:05.951271+0000","last_fullsized":"2024-04-25T00:04:05.951271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:35.048191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.943304+0000","last_change":"2024-04-25T00:04:05.943304+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.943304+0000","last_undegraded":"2024-04-25T00:04:05.943304+0000","last_fullsized":"2024-04-25T00:04:05.943304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.953915+0000","last_change":"2024-04-25T00:04:05.949348+0000","last_active":"2024-04-25T00:04:05.953915+0000","last_peered":"2024-04-25T00:04:05.953915+0000","last_clean":"2024-04-25T00:04:05.953915+0000","last_became_active":"2024-04-25T00:04:05.948760+0000","last_became_peered":"2024-04-25T00:04:05.948760+0000","last_unstale":"2024-04-25T00:04:05.953915+0000","last_undegraded":"2024-04-25T00:04:05.953915+0000","last_fullsized":"2024-04-25T00:04:05.953915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:28.987214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939814+0000","last_change":"2024-04-25T00:04:05.939814+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939814+0000","last_undegraded":"2024-04-25T00:04:05.939814+0000","last_fullsized":"2024-04-25T00:04:05.939814+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.954516+0000","last_change":"2024-04-25T00:04:05.953709+0000","last_active":"2024-04-25T00:04:05.954516+0000","last_peered":"2024-04-25T00:04:05.954516+0000","last_clean":"2024-04-25T00:04:05.954516+0000","last_became_active":"2024-04-25T00:04:05.953453+0000","last_became_peered":"2024-04-25T00:04:05.953453+0000","last_unstale":"2024-04-25T00:04:05.954516+0000","last_undegraded":"2024-04-25T00:04:05.954516+0000","last_fullsized":"2024-04-25T00:04:05.954516+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:56.648421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.946625+0000","last_change":"2024-04-25T00:04:05.946625+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.946625+0000","last_undegraded":"2024-04-25T00:04:05.946625+0000","last_fullsized":"2024-04-25T00:04:05.946625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957426+0000","last_change":"2024-04-25T00:04:05.953267+0000","last_active":"2024-04-25T00:04:05.957426+0000","last_peered":"2024-04-25T00:04:05.957426+0000","last_clean":"2024-04-25T00:04:05.957426+0000","last_became_active":"2024-04-25T00:04:05.953021+0000","last_became_peered":"2024-04-25T00:04:05.953021+0000","last_unstale":"2024-04-25T00:04:05.957426+0000","last_undegraded":"2024-04-25T00:04:05.957426+0000","last_fullsized":"2024-04-25T00:04:05.957426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:37.383495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.939924+0000","last_change":"2024-04-25T00:04:05.939924+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.939924+0000","last_undegraded":"2024-04-25T00:04:05.939924+0000","last_fullsized":"2024-04-25T00:04:05.939924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949496+0000","last_change":"2024-04-25T00:04:05.944307+0000","last_active":"2024-04-25T00:04:05.949496+0000","last_peered":"2024-04-25T00:04:05.949496+0000","last_clean":"2024-04-25T00:04:05.949496+0000","last_became_active":"2024-04-25T00:04:05.943618+0000","last_became_peered":"2024-04-25T00:04:05.943618+0000","last_unstale":"2024-04-25T00:04:05.949496+0000","last_undegraded":"2024-04-25T00:04:05.949496+0000","last_fullsized":"2024-04-25T00:04:05.949496+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:48.774779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948590+0000","last_change":"2024-04-25T00:04:05.948590+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948590+0000","last_undegraded":"2024-04-25T00:04:05.948590+0000","last_fullsized":"2024-04-25T00:04:05.948590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.952860+0000","last_change":"2024-04-25T00:04:05.949003+0000","last_active":"2024-04-25T00:04:05.952860+0000","last_peered":"2024-04-25T00:04:05.952860+0000","last_clean":"2024-04-25T00:04:05.952860+0000","last_became_active":"2024-04-25T00:04:05.948731+0000","last_became_peered":"2024-04-25T00:04:05.948731+0000","last_unstale":"2024-04-25T00:04:05.952860+0000","last_undegraded":"2024-04-25T00:04:05.952860+0000","last_fullsized":"2024-04-25T00:04:05.952860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:20.845105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.942891+0000","last_change":"2024-04-25T00:04:05.942891+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.942891+0000","last_undegraded":"2024-04-25T00:04:05.942891+0000","last_fullsized":"2024-04-25T00:04:05.942891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.949915+0000","last_change":"2024-04-25T00:04:05.949157+0000","last_active":"2024-04-25T00:04:05.949915+0000","last_peered":"2024-04-25T00:04:05.949915+0000","last_clean":"2024-04-25T00:04:05.949915+0000","last_became_active":"2024-04-25T00:04:05.948879+0000","last_became_peered":"2024-04-25T00:04:05.948879+0000","last_unstale":"2024-04-25T00:04:05.949915+0000","last_undegraded":"2024-04-25T00:04:05.949915+0000","last_fullsized":"2024-04-25T00:04:05.949915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:26.961912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-04-25T00:04:05.948850+0000","last_change":"2024-04-25T00:04:05.948850+0000","last_active":"2024-04-25T00:04:05.931537+0000","last_peered":"2024-04-25T00:04:05.931537+0000","last_clean":"2024-04-25T00:04:05.931537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:04:05.948850+0000","last_undegraded":"2024-04-25T00:04:05.948850+0000","last_fullsized":"2024-04-25T00:04:05.948850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.957511+0000","last_change":"2024-04-25T00:04:05.957149+0000","last_active":"2024-04-25T00:04:05.957511+0000","last_peered":"2024-04-25T00:04:05.957511+0000","last_clean":"2024-04-25T00:04:05.957511+0000","last_became_active":"2024-04-25T00:04:05.956870+0000","last_became_peered":"2024-04-25T00:04:05.956870+0000","last_unstale":"2024-04-25T00:04:05.957511+0000","last_undegraded":"2024-04-25T00:04:05.957511+0000","last_fullsized":"2024-04-25T00:04:05.957511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:41.561274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":297,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T00:04:05.950293+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:04:05.950293+0000","last_peered":"2024-04-25T00:04:05.950293+0000","last_clean":"2024-04-25T00:04:05.950293+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:04:05.950293+0000","last_undegraded":"2024-04-25T00:04:05.950293+0000","last_fullsized":"2024-04-25T00:04:05.950293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55040,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316800,"statfs":{"total":96636764160,"available":96580403200,"internally_reserved":0,"allocated":1687552,"data_stored":1059333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58620,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313220,"statfs":{"total":96636764160,"available":96576737280,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40576,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331264,"statfs":{"total":96636764160,"available":96595214336,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50304,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321536,"statfs":{"total":96636764160,"available":96585252864,"internally_reserved":0,"allocated":1093632,"data_stored":468965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41164,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330676,"statfs":{"total":96636764160,"available":96594612224,"internally_reserved":0,"allocated":1699840,"data_stored":1065608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.944580+0000', 'last_change': '2024-04-25T00:04:05.944580+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.944580+0000', 'last_undegraded': '2024-04-25T00:04:05.944580+0000', 'last_fullsized': '2024-04-25T00:04:05.944580+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949172+0000', 'last_change': '2024-04-25T00:04:05.949172+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949172+0000', 'last_undegraded': '2024-04-25T00:04:05.949172+0000', 'last_fullsized': '2024-04-25T00:04:05.949172+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948454+0000', 'last_change': '2024-04-25T00:04:05.948454+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948454+0000', 'last_undegraded': '2024-04-25T00:04:05.948454+0000', 'last_fullsized': '2024-04-25T00:04:05.948454+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-04-25T00:04:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949735+0000', 'last_change': '2024-04-25T00:04:05.949735+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949735+0000', 'last_undegraded': '2024-04-25T00:04:05.949735+0000', 'last_fullsized': '2024-04-25T00:04:05.949735+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.941249+0000', 'last_change': '2024-04-25T00:04:05.941249+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.941249+0000', 'last_undegraded': '2024-04-25T00:04:05.941249+0000', 'last_fullsized': '2024-04-25T00:04:05.941249+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949527+0000', 'last_change': '2024-04-25T00:04:05.949527+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949527+0000', 'last_undegraded': '2024-04-25T00:04:05.949527+0000', 'last_fullsized': '2024-04-25T00:04:05.949527+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.954626+0000', 'last_change': '2024-04-25T00:04:05.954626+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.954626+0000', 'last_undegraded': '2024-04-25T00:04:05.954626+0000', 'last_fullsized': '2024-04-25T00:04:05.954626+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.942745+0000', 'last_change': '2024-04-25T00:04:05.942745+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.942745+0000', 'last_undegraded': '2024-04-25T00:04:05.942745+0000', 'last_fullsized': '2024-04-25T00:04:05.942745+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2024-04-25T00:04:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.945401+0000', 'last_change': '2024-04-25T00:04:05.945401+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.945401+0000', 'last_undegraded': '2024-04-25T00:04:05.945401+0000', 'last_fullsized': '2024-04-25T00:04:05.945401+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.942883+0000', 'last_change': '2024-04-25T00:04:05.942883+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.942883+0000', 'last_undegraded': '2024-04-25T00:04:05.942883+0000', 'last_fullsized': '2024-04-25T00:04:05.942883+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949288+0000', 'last_change': '2024-04-25T00:04:05.949288+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949288+0000', 'last_undegraded': '2024-04-25T00:04:05.949288+0000', 'last_fullsized': '2024-04-25T00:04:05.949288+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.945295+0000', 'last_change': '2024-04-25T00:04:05.945295+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.945295+0000', 'last_undegraded': '2024-04-25T00:04:05.945295+0000', 'last_fullsized': '2024-04-25T00:04:05.945295+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948113+0000', 'last_change': '2024-04-25T00:04:05.948113+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948113+0000', 'last_undegraded': '2024-04-25T00:04:05.948113+0000', 'last_fullsized': '2024-04-25T00:04:05.948113+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949697+0000', 'last_change': '2024-04-25T00:04:05.949697+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949697+0000', 'last_undegraded': '2024-04-25T00:04:05.949697+0000', 'last_fullsized': '2024-04-25T00:04:05.949697+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949882+0000', 'last_change': '2024-04-25T00:04:05.949882+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949882+0000', 'last_undegraded': '2024-04-25T00:04:05.949882+0000', 'last_fullsized': '2024-04-25T00:04:05.949882+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.939754+0000', 'last_change': '2024-04-25T00:04:05.939754+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.939754+0000', 'last_undegraded': '2024-04-25T00:04:05.939754+0000', 'last_fullsized': '2024-04-25T00:04:05.939754+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.939922+0000', 'last_change': '2024-04-25T00:04:05.939922+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.939922+0000', 'last_undegraded': '2024-04-25T00:04:05.939922+0000', 'last_fullsized': '2024-04-25T00:04:05.939922+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.950188+0000', 'last_change': '2024-04-25T00:04:05.950188+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.950188+0000', 'last_undegraded': '2024-04-25T00:04:05.950188+0000', 'last_fullsized': '2024-04-25T00:04:05.950188+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.939943+0000', 'last_change': '2024-04-25T00:04:05.939943+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.939943+0000', 'last_undegraded': '2024-04-25T00:04:05.939943+0000', 'last_fullsized': '2024-04-25T00:04:05.939943+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-04-25T00:04:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949340+0000', 'last_change': '2024-04-25T00:04:05.949340+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949340+0000', 'last_undegraded': '2024-04-25T00:04:05.949340+0000', 'last_fullsized': '2024-04-25T00:04:05.949340+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.954404+0000', 'last_change': '2024-04-25T00:04:05.954404+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.954404+0000', 'last_undegraded': '2024-04-25T00:04:05.954404+0000', 'last_fullsized': '2024-04-25T00:04:05.954404+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.950060+0000', 'last_change': '2024-04-25T00:04:05.950060+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.950060+0000', 'last_undegraded': '2024-04-25T00:04:05.950060+0000', 'last_fullsized': '2024-04-25T00:04:05.950060+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.943371+0000', 'last_change': '2024-04-25T00:04:05.943371+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.943371+0000', 'last_undegraded': '2024-04-25T00:04:05.943371+0000', 'last_fullsized': '2024-04-25T00:04:05.943371+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.953326+0000', 'last_change': '2024-04-25T00:04:05.953326+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.953326+0000', 'last_undegraded': '2024-04-25T00:04:05.953326+0000', 'last_fullsized': '2024-04-25T00:04:05.953326+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2024-04-25T00:04:10.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.950169+0000', 'last_change': '2024-04-25T00:04:05.950169+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.950169+0000', 'last_undegraded': '2024-04-25T00:04:05.950169+0000', 'last_fullsized': '2024-04-25T00:04:05.950169+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948459+0000', 'last_change': '2024-04-25T00:04:05.948459+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948459+0000', 'last_undegraded': '2024-04-25T00:04:05.948459+0000', 'last_fullsized': '2024-04-25T00:04:05.948459+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.942111+0000', 'last_change': '2024-04-25T00:04:05.942111+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.942111+0000', 'last_undegraded': '2024-04-25T00:04:05.942111+0000', 'last_fullsized': '2024-04-25T00:04:05.942111+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.944848+0000', 'last_change': '2024-04-25T00:04:05.944848+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.944848+0000', 'last_undegraded': '2024-04-25T00:04:05.944848+0000', 'last_fullsized': '2024-04-25T00:04:05.944848+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948830+0000', 'last_change': '2024-04-25T00:04:05.948830+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948830+0000', 'last_undegraded': '2024-04-25T00:04:05.948830+0000', 'last_fullsized': '2024-04-25T00:04:05.948830+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949664+0000', 'last_change': '2024-04-25T00:04:05.949664+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949664+0000', 'last_undegraded': '2024-04-25T00:04:05.949664+0000', 'last_fullsized': '2024-04-25T00:04:05.949664+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2024-04-25T00:04:10.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.939777+0000', 'last_change': '2024-04-25T00:04:05.939777+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.939777+0000', 'last_undegraded': '2024-04-25T00:04:05.939777+0000', 'last_fullsized': '2024-04-25T00:04:05.939777+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.943940+0000', 'last_change': '2024-04-25T00:04:05.943940+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.943940+0000', 'last_undegraded': '2024-04-25T00:04:05.943940+0000', 'last_fullsized': '2024-04-25T00:04:05.943940+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.941083+0000', 'last_change': '2024-04-25T00:04:05.941083+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.941083+0000', 'last_undegraded': '2024-04-25T00:04:05.941083+0000', 'last_fullsized': '2024-04-25T00:04:05.941083+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.943649+0000', 'last_change': '2024-04-25T00:04:05.943649+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.943649+0000', 'last_undegraded': '2024-04-25T00:04:05.943649+0000', 'last_fullsized': '2024-04-25T00:04:05.943649+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.943771+0000', 'last_change': '2024-04-25T00:04:05.943771+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.943771+0000', 'last_undegraded': '2024-04-25T00:04:05.943771+0000', 'last_fullsized': '2024-04-25T00:04:05.943771+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.947794+0000', 'last_change': '2024-04-25T00:04:05.947794+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.947794+0000', 'last_undegraded': '2024-04-25T00:04:05.947794+0000', 'last_fullsized': '2024-04-25T00:04:05.947794+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949243+0000', 'last_change': '2024-04-25T00:04:05.949243+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949243+0000', 'last_undegraded': '2024-04-25T00:04:05.949243+0000', 'last_fullsized': '2024-04-25T00:04:05.949243+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.940520+0000', 'last_change': '2024-04-25T00:04:05.940520+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.940520+0000', 'last_undegraded': '2024-04-25T00:04:05.940520+0000', 'last_fullsized': '2024-04-25T00:04:05.940520+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949045+0000', 'last_change': '2024-04-25T00:04:05.949045+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949045+0000', 'last_undegraded': '2024-04-25T00:04:05.949045+0000', 'last_fullsized': '2024-04-25T00:04:05.949045+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949448+0000', 'last_change': '2024-04-25T00:04:05.949448+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949448+0000', 'last_undegraded': '2024-04-25T00:04:05.949448+0000', 'last_fullsized': '2024-04-25T00:04:05.949448+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948977+0000', 'last_change': '2024-04-25T00:04:05.948977+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948977+0000', 'last_undegraded': '2024-04-25T00:04:05.948977+0000', 'last_fullsized': '2024-04-25T00:04:05.948977+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T00:04:10.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.943259+0000', 'last_change': '2024-04-25T00:04:05.943259+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.943259+0000', 'last_undegraded': '2024-04-25T00:04:05.943259+0000', 'last_fullsized': '2024-04-25T00:04:05.943259+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.940486+0000', 'last_change': '2024-04-25T00:04:05.940486+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.940486+0000', 'last_undegraded': '2024-04-25T00:04:05.940486+0000', 'last_fullsized': '2024-04-25T00:04:05.940486+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.941657+0000', 'last_change': '2024-04-25T00:04:05.941657+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.941657+0000', 'last_undegraded': '2024-04-25T00:04:05.941657+0000', 'last_fullsized': '2024-04-25T00:04:05.941657+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.940395+0000', 'last_change': '2024-04-25T00:04:05.940395+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.940395+0000', 'last_undegraded': '2024-04-25T00:04:05.940395+0000', 'last_fullsized': '2024-04-25T00:04:05.940395+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.947569+0000', 'last_change': '2024-04-25T00:04:05.947569+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.947569+0000', 'last_undegraded': '2024-04-25T00:04:05.947569+0000', 'last_fullsized': '2024-04-25T00:04:05.947569+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-04-25T00:04:10.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948793+0000', 'last_change': '2024-04-25T00:04:05.948793+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948793+0000', 'last_undegraded': '2024-04-25T00:04:05.948793+0000', 'last_fullsized': '2024-04-25T00:04:05.948793+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949653+0000', 'last_change': '2024-04-25T00:04:05.949653+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949653+0000', 'last_undegraded': '2024-04-25T00:04:05.949653+0000', 'last_fullsized': '2024-04-25T00:04:05.949653+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.947102+0000', 'last_change': '2024-04-25T00:04:05.947102+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.947102+0000', 'last_undegraded': '2024-04-25T00:04:05.947102+0000', 'last_fullsized': '2024-04-25T00:04:05.947102+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.944150+0000', 'last_change': '2024-04-25T00:04:05.944150+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.944150+0000', 'last_undegraded': '2024-04-25T00:04:05.944150+0000', 'last_fullsized': '2024-04-25T00:04:05.944150+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.940325+0000', 'last_change': '2024-04-25T00:04:05.940325+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.940325+0000', 'last_undegraded': '2024-04-25T00:04:05.940325+0000', 'last_fullsized': '2024-04-25T00:04:05.940325+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-04-25T00:04:10.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949229+0000', 'last_change': '2024-04-25T00:04:05.949229+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949229+0000', 'last_undegraded': '2024-04-25T00:04:05.949229+0000', 'last_fullsized': '2024-04-25T00:04:05.949229+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.946063+0000', 'last_change': '2024-04-25T00:04:05.946063+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.946063+0000', 'last_undegraded': '2024-04-25T00:04:05.946063+0000', 'last_fullsized': '2024-04-25T00:04:05.946063+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.952933+0000', 'last_change': '2024-04-25T00:04:05.952933+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.952933+0000', 'last_undegraded': '2024-04-25T00:04:05.952933+0000', 'last_fullsized': '2024-04-25T00:04:05.952933+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.946745+0000', 'last_change': '2024-04-25T00:04:05.946745+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.946745+0000', 'last_undegraded': '2024-04-25T00:04:05.946745+0000', 'last_fullsized': '2024-04-25T00:04:05.946745+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.940520+0000', 'last_change': '2024-04-25T00:04:05.940520+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.940520+0000', 'last_undegraded': '2024-04-25T00:04:05.940520+0000', 'last_fullsized': '2024-04-25T00:04:05.940520+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.949861+0000', 'last_change': '2024-04-25T00:04:05.949861+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.949861+0000', 'last_undegraded': '2024-04-25T00:04:05.949861+0000', 'last_fullsized': '2024-04-25T00:04:05.949861+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.943304+0000', 'last_change': '2024-04-25T00:04:05.943304+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.943304+0000', 'last_undegraded': '2024-04-25T00:04:05.943304+0000', 'last_fullsized': '2024-04-25T00:04:05.943304+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.939814+0000', 'last_change': '2024-04-25T00:04:05.939814+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.939814+0000', 'last_undegraded': '2024-04-25T00:04:05.939814+0000', 'last_fullsized': '2024-04-25T00:04:05.939814+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.946625+0000', 'last_change': '2024-04-25T00:04:05.946625+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.946625+0000', 'last_undegraded': '2024-04-25T00:04:05.946625+0000', 'last_fullsized': '2024-04-25T00:04:05.946625+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.939924+0000', 'last_change': '2024-04-25T00:04:05.939924+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.939924+0000', 'last_undegraded': '2024-04-25T00:04:05.939924+0000', 'last_fullsized': '2024-04-25T00:04:05.939924+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948590+0000', 'last_change': '2024-04-25T00:04:05.948590+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948590+0000', 'last_undegraded': '2024-04-25T00:04:05.948590+0000', 'last_fullsized': '2024-04-25T00:04:05.948590+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2024-04-25T00:04:10.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.942891+0000', 'last_change': '2024-04-25T00:04:05.942891+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.942891+0000', 'last_undegraded': '2024-04-25T00:04:05.942891+0000', 'last_fullsized': '2024-04-25T00:04:05.942891+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T00:04:10.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2024-04-25T00:04:10.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-04-25T00:04:05.948850+0000', 'last_change': '2024-04-25T00:04:05.948850+0000', 'last_active': '2024-04-25T00:04:05.931537+0000', 'last_peered': '2024-04-25T00:04:05.931537+0000', 'last_clean': '2024-04-25T00:04:05.931537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T00:04:05.948850+0000', 'last_undegraded': '2024-04-25T00:04:05.948850+0000', 'last_fullsized': '2024-04-25T00:04:05.948850+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'last_clean_scrub_stamp': '2024-04-25T00:04:05.931537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T00:04:10.593 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:04:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.712+0000 7fc1d3c45640 1 Processor -- start 2024-04-25T00:04:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d3c45640 1 -- start start 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d3c45640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1cc144910 0x7fc1cc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d3c45640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc1cc0b56b0 0x7fc1cc0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d3c45640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1cc144cf0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d3c45640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc1cc0b4510 con 0x7fc1cc144910 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d3c45640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc1cc0b2f30 con 0x7fc1cc0b56b0 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d19e1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1cc144910 0x7fc1cc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.713+0000 7fc1d11e0640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc1cc0b5240 0x7fc1cc06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34748/0 (socket says 172.21.15.113:34748) 2024-04-25T00:04:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.714+0000 7fc1d11e0640 1 -- 172.21.15.113:0/2402011777 learned_addr learned my addr 172.21.15.113:0/2402011777 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.714+0000 7fc1d21e2640 1 --2- 172.21.15.113:0/2402011777 >> v2:172.21.15.178:3300/0 conn(0x7fc1cc0b56b0 0x7fc1cc0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.714+0000 7fc1d19e1640 1 -- 172.21.15.113:0/2402011777 >> v2:172.21.15.178:3300/0 conn(0x7fc1cc0b56b0 msgr2=0x7fc1cc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.714+0000 7fc1d19e1640 1 --2- 172.21.15.113:0/2402011777 >> v2:172.21.15.178:3300/0 conn(0x7fc1cc0b56b0 0x7fc1cc0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.714+0000 7fc1d19e1640 1 -- 172.21.15.113:0/2402011777 >> v1:172.21.15.113:6789/0 conn(0x7fc1cc0b5240 legacy=0x7fc1cc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.714+0000 7fc1d19e1640 1 -- 172.21.15.113:0/2402011777 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc0b3210 con 0x7fc1cc144910 2024-04-25T00:04:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.714+0000 7fc1d19e1640 1 --2- 172.21.15.113:0/2402011777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc144910 0x7fc1cc0b3fd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400ba40 tx=0x7fc1c402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3000e05253350d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d09df640 1 -- 172.21.15.113:0/2402011777 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4032770 con 0x7fc1cc144910 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d09df640 1 -- 172.21.15.113:0/2402011777 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4038070 con 0x7fc1cc144910 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d09df640 1 -- 172.21.15.113:0/2402011777 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4032b30 con 0x7fc1cc144910 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 -- 172.21.15.113:0/2402011777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc144910 msgr2=0x7fc1cc0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/2402011777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc144910 0x7fc1cc0b3fd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400ba40 tx=0x7fc1c402c8b0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 -- 172.21.15.113:0/2402011777 shutdown_connections 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/2402011777 >> v2:172.21.15.178:3300/0 conn(0x7fc1cc0b56b0 0x7fc1cc0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/2402011777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc144910 0x7fc1cc0b3fd0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 -- 172.21.15.113:0/2402011777 >> 172.21.15.113:0/2402011777 conn(0x7fc1cc0b1a50 msgr2=0x7fc1cc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 -- 172.21.15.113:0/2402011777 shutdown_connections 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.715+0000 7fc1d3c45640 1 -- 172.21.15.113:0/2402011777 wait complete. 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.716+0000 7fc1d3c45640 1 Processor -- start 2024-04-25T00:04:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.716+0000 7fc1d3c45640 1 -- start start 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d3c45640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc0b5240 0x7fc1cc141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d3c45640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1cc0b56b0 0x7fc1cc0c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d3c45640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc144910 0x7fc1cc0c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d3c45640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0b4da0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d3c45640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0b37d0 con 0x7fc1cc144910 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d3c45640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc0b4f20 con 0x7fc1cc0b56b0 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc0b5240 0x7fc1cc141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc0b5240 0x7fc1cc141060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53956/0 (socket says 172.21.15.113:53956) 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 -- 172.21.15.113:0/258333872 learned_addr learned my addr 172.21.15.113:0/258333872 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 -- 172.21.15.113:0/258333872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1cc0b56b0 msgr2=0x7fc1cc0c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1cc0b56b0 0x7fc1cc0c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 -- 172.21.15.113:0/258333872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc144910 msgr2=0x7fc1cc0c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc144910 0x7fc1cc0c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.717+0000 7fc1d19e1640 1 -- 172.21.15.113:0/258333872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc14c340 con 0x7fc1cc0b5240 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.718+0000 7fc1d19e1640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc0b5240 0x7fc1cc141060 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400bb60 tx=0x7fc1c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.718+0000 7fc1c27fc640 1 -- 172.21.15.113:0/258333872 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c404a070 con 0x7fc1cc0b5240 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.718+0000 7fc1c27fc640 1 -- 172.21.15.113:0/258333872 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4033610 con 0x7fc1cc0b5240 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.718+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc0abbd0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.718+0000 7fc1c27fc640 1 -- 172.21.15.113:0/258333872 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40339e0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.718+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc14c5e0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.719+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1cc0691a0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.719+0000 7fc1c27fc640 1 -- 172.21.15.113:0/258333872 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1c404d070 con 0x7fc1cc0b5240 2024-04-25T00:04:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.720+0000 7fc1c27fc640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.720+0000 7fc1c27fc640 1 -- 172.21.15.113:0/258333872 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fc1c40ad740 con 0x7fc1cc0b5240 2024-04-25T00:04:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.720+0000 7fc1d11e0640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.720+0000 7fc1d11e0640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1b006be30 0x7fc1b006e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000f60 tx=0x7fc1bc007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:10.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.724+0000 7fc1c27fc640 1 -- 172.21.15.113:0/258333872 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc1c407c2a0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.962+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc1cc0b1540 con 0x7fc1cc0b5240 2024-04-25T00:04:10.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.962+0000 7fc1c27fc640 1 -- 172.21.15.113:0/258333872 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1196 (secure 0 0 0) 0x7fc1c407bba0 con 0x7fc1cc0b5240 2024-04-25T00:04:10.963 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:10.963 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":375,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":391897088,"bytes_avail":772702216192,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":50,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7395}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T00:03:19.169889+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:04:10.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1b006be30 msgr2=0x7fc1b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1b006be30 0x7fc1b006e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000f60 tx=0x7fc1bc007a80 comp rx=0 tx=0).stop 2024-04-25T00:04:10.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc0b5240 msgr2=0x7fc1cc141060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:10.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc0b5240 0x7fc1cc141060 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400bb60 tx=0x7fc1c40046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 shutdown_connections 2024-04-25T00:04:10.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc144910 0x7fc1cc0c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1cc0b56b0 0x7fc1cc0c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 --2- 172.21.15.113:0/258333872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1cc0b5240 0x7fc1cc141060 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:10.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.964+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 >> 172.21.15.113:0/258333872 conn(0x7fc1cc0b1a50 msgr2=0x7fc1cc070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:10.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.965+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 shutdown_connections 2024-04-25T00:04:10.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:10.965+0000 7fc1d3c45640 1 -- 172.21.15.113:0/258333872 wait complete. 2024-04-25T00:04:10.976 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:04:10.976 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:04:11.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.072+0000 7f8263334640 1 Processor -- start 2024-04-25T00:04:11.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.072+0000 7f8263334640 1 -- start start 2024-04-25T00:04:11.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f8263334640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f825c15f590 0x7f825c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:11.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f8263334640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f825c0d0050 0x7f825c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:11.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f8263334640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f825c161fa0 con 0x7f825c06c760 2024-04-25T00:04:11.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f8263334640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f825c162280 con 0x7f825c15f590 2024-04-25T00:04:11.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f8263334640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f825c162560 con 0x7f825c0d0050 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f82608cf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f825c15f590 0x7f825c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f82608cf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f825c15f590 0x7f825c0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53964/0 (socket says 172.21.15.113:53964) 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.073+0000 7f82608cf640 1 -- 172.21.15.113:0/873991145 learned_addr learned my addr 172.21.15.113:0/873991145 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.074+0000 7f82618d1640 1 --2- 172.21.15.113:0/873991145 >> v2:172.21.15.178:3300/0 conn(0x7f825c0d0050 0x7f825c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.074+0000 7f82608cf640 1 -- 172.21.15.113:0/873991145 >> v2:172.21.15.178:3300/0 conn(0x7f825c0d0050 msgr2=0x7f825c06f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.074+0000 7f82608cf640 1 --2- 172.21.15.113:0/873991145 >> v2:172.21.15.178:3300/0 conn(0x7f825c0d0050 0x7f825c06f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.074+0000 7f82608cf640 1 -- 172.21.15.113:0/873991145 >> v1:172.21.15.178:6789/0 conn(0x7f825c06c760 legacy=0x7f825c06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.074+0000 7f82608cf640 1 -- 172.21.15.113:0/873991145 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c162840 con 0x7f825c15f590 2024-04-25T00:04:11.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.074+0000 7f82608cf640 1 --2- 172.21.15.113:0/873991145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c15f590 0x7f825c0cfb10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f825c161e40 tx=0x7f824c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a7c493c416eac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:11.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.074+0000 7f8253fff640 1 -- 172.21.15.113:0/873991145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c037070 con 0x7f825c15f590 2024-04-25T00:04:11.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8253fff640 1 -- 172.21.15.113:0/873991145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c032440 con 0x7f825c15f590 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8253fff640 1 -- 172.21.15.113:0/873991145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c032810 con 0x7f825c15f590 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 -- 172.21.15.113:0/873991145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c15f590 msgr2=0x7f825c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 --2- 172.21.15.113:0/873991145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c15f590 0x7f825c0cfb10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f825c161e40 tx=0x7f824c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 -- 172.21.15.113:0/873991145 shutdown_connections 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 --2- 172.21.15.113:0/873991145 >> v2:172.21.15.178:3300/0 conn(0x7f825c0d0050 0x7f825c06f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 --2- 172.21.15.113:0/873991145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c15f590 0x7f825c0cfb10 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 -- 172.21.15.113:0/873991145 >> 172.21.15.113:0/873991145 conn(0x7f825c0d1f40 msgr2=0x7f825c0d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 -- 172.21.15.113:0/873991145 shutdown_connections 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.075+0000 7f8263334640 1 -- 172.21.15.113:0/873991145 wait complete. 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.076+0000 7f8263334640 1 Processor -- start 2024-04-25T00:04:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.076+0000 7f8263334640 1 -- start start 2024-04-25T00:04:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f8263334640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f825c06c760 0x7f825c0b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f8263334640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c0d0050 0x7f825c0b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f8263334640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f825c15f590 0x7f825c0b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f8263334640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f825c158d30 con 0x7f825c0d0050 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f8263334640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f825c158bb0 con 0x7f825c15f590 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f8263334640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f825c06f7a0 con 0x7f825c06c760 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c0d0050 0x7f825c0b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c0d0050 0x7f825c0b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53974/0 (socket says 172.21.15.113:53974) 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 -- 172.21.15.113:0/1926926476 learned_addr learned my addr 172.21.15.113:0/1926926476 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82618d1640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f825c15f590 0x7f825c0b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82610d0640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f825c06c760 0x7f825c0b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 -- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f825c06c760 msgr2=0x7f825c0b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f825c06c760 0x7f825c0b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 -- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f825c15f590 msgr2=0x7f825c0b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f825c15f590 0x7f825c0b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.077+0000 7f82608cf640 1 -- 172.21.15.113:0/1926926476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1579e0 con 0x7f825c0d0050 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.078+0000 7f82608cf640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c0d0050 0x7f825c0b6380 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f824c000c00 tx=0x7f824c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.078+0000 7f8251ffb640 1 -- 172.21.15.113:0/1926926476 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c041020 con 0x7f825c0d0050 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.078+0000 7f8251ffb640 1 -- 172.21.15.113:0/1926926476 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c0306c0 con 0x7f825c0d0050 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.078+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c0c5e80 con 0x7f825c0d0050 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.078+0000 7f8251ffb640 1 -- 172.21.15.113:0/1926926476 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c037040 con 0x7f825c0d0050 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.078+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c0c5b90 con 0x7f825c0d0050 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.079+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c0691a0 con 0x7f825c0d0050 2024-04-25T00:04:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.079+0000 7f8251ffb640 1 -- 172.21.15.113:0/1926926476 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f824c044020 con 0x7f825c0d0050 2024-04-25T00:04:11.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.080+0000 7f8251ffb640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f824006be30 0x7f824006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.080+0000 7f8251ffb640 1 -- 172.21.15.113:0/1926926476 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f824c0ab670 con 0x7f825c0d0050 2024-04-25T00:04:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.080+0000 7f82610d0640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f824006be30 0x7f824006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.080+0000 7f82610d0640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f824006be30 0x7f824006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8258005520 tx=0x7f8258009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:11.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.084+0000 7f8251ffb640 1 -- 172.21.15.113:0/1926926476 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f824c04b030 con 0x7f825c0d0050 2024-04-25T00:04:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.256+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f825c0b6e00 con 0x7f824006be30 2024-04-25T00:04:11.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.262+0000 7f8251ffb640 1 -- 172.21.15.113:0/1926926476 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252088 (secure 0 0 0) 0x7f825c0b6e00 con 0x7f824006be30 2024-04-25T00:04:11.265 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:11.266 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f824006be30 msgr2=0x7f824006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f824006be30 0x7f824006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8258005520 tx=0x7f8258009040 comp rx=0 tx=0).stop 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c0d0050 msgr2=0x7f825c0b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c0d0050 0x7f825c0b6380 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f824c000c00 tx=0x7f824c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 shutdown_connections 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f824006be30 0x7f824006e2f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f825c15f590 0x7f825c0b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f825c0d0050 0x7f825c0b6380 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 --2- 172.21.15.113:0/1926926476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f825c06c760 0x7f825c0b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 >> 172.21.15.113:0/1926926476 conn(0x7f825c0d1f40 msgr2=0x7f825c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:11.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 shutdown_connections 2024-04-25T00:04:11.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:11.265+0000 7f8263334640 1 -- 172.21.15.113:0/1926926476 wait complete. 2024-04-25T00:04:11.276 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-04-25T00:04:11.181299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383040,"kb_used_data":10056,"kb_used_omap":12,"kb_used_meta":372851,"kb_avail":754591680,"statfs":{"total":773094113280,"available":772701880320,"internally_reserved":0,"allocated":10297344,"data_stored":5092076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":381799984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000911"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730415+0000","last_change":"2024-04-25T00:04:06.952353+0000","last_active":"2024-04-25T00:04:08.730415+0000","last_peered":"2024-04-25T00:04:08.730415+0000","last_clean":"2024-04-25T00:04:08.730415+0000","last_became_active":"2024-04-25T00:04:06.949248+0000","last_became_peered":"2024-04-25T00:04:06.949248+0000","last_unstale":"2024-04-25T00:04:08.730415+0000","last_undegraded":"2024-04-25T00:04:08.730415+0000","last_fullsized":"2024-04-25T00:04:08.730415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:11.974068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082207+0000","last_change":"2024-04-25T00:04:06.959519+0000","last_active":"2024-04-25T00:04:08.082207+0000","last_peered":"2024-04-25T00:04:08.082207+0000","last_clean":"2024-04-25T00:04:08.082207+0000","last_became_active":"2024-04-25T00:04:06.959247+0000","last_became_peered":"2024-04-25T00:04:06.959247+0000","last_unstale":"2024-04-25T00:04:08.082207+0000","last_undegraded":"2024-04-25T00:04:08.082207+0000","last_fullsized":"2024-04-25T00:04:08.082207+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:49.779489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942843+0000","last_change":"2024-04-25T00:04:06.956520+0000","last_active":"2024-04-25T00:04:07.942843+0000","last_peered":"2024-04-25T00:04:07.942843+0000","last_clean":"2024-04-25T00:04:07.942843+0000","last_became_active":"2024-04-25T00:04:06.956166+0000","last_became_peered":"2024-04-25T00:04:06.956166+0000","last_unstale":"2024-04-25T00:04:07.942843+0000","last_undegraded":"2024-04-25T00:04:07.942843+0000","last_fullsized":"2024-04-25T00:04:07.942843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:10.830589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728945+0000","last_change":"2024-04-25T00:04:06.965710+0000","last_active":"2024-04-25T00:04:08.728945+0000","last_peered":"2024-04-25T00:04:08.728945+0000","last_clean":"2024-04-25T00:04:08.728945+0000","last_became_active":"2024-04-25T00:04:06.965463+0000","last_became_peered":"2024-04-25T00:04:06.965463+0000","last_unstale":"2024-04-25T00:04:08.728945+0000","last_undegraded":"2024-04-25T00:04:08.728945+0000","last_fullsized":"2024-04-25T00:04:08.728945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:23.761487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729734+0000","last_change":"2024-04-25T00:04:06.959847+0000","last_active":"2024-04-25T00:04:08.729734+0000","last_peered":"2024-04-25T00:04:08.729734+0000","last_clean":"2024-04-25T00:04:08.729734+0000","last_became_active":"2024-04-25T00:04:06.959557+0000","last_became_peered":"2024-04-25T00:04:06.959557+0000","last_unstale":"2024-04-25T00:04:08.729734+0000","last_undegraded":"2024-04-25T00:04:08.729734+0000","last_fullsized":"2024-04-25T00:04:08.729734+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:56.312120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729107+0000","last_change":"2024-04-25T00:04:06.969950+0000","last_active":"2024-04-25T00:04:08.729107+0000","last_peered":"2024-04-25T00:04:08.729107+0000","last_clean":"2024-04-25T00:04:08.729107+0000","last_became_active":"2024-04-25T00:04:06.969657+0000","last_became_peered":"2024-04-25T00:04:06.969657+0000","last_unstale":"2024-04-25T00:04:08.729107+0000","last_undegraded":"2024-04-25T00:04:08.729107+0000","last_fullsized":"2024-04-25T00:04:08.729107+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:46.284690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728707+0000","last_change":"2024-04-25T00:04:06.955896+0000","last_active":"2024-04-25T00:04:08.728707+0000","last_peered":"2024-04-25T00:04:08.728707+0000","last_clean":"2024-04-25T00:04:08.728707+0000","last_became_active":"2024-04-25T00:04:06.955627+0000","last_became_peered":"2024-04-25T00:04:06.955627+0000","last_unstale":"2024-04-25T00:04:08.728707+0000","last_undegraded":"2024-04-25T00:04:08.728707+0000","last_fullsized":"2024-04-25T00:04:08.728707+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:44.699696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728712+0000","last_change":"2024-04-25T00:04:06.956600+0000","last_active":"2024-04-25T00:04:08.728712+0000","last_peered":"2024-04-25T00:04:08.728712+0000","last_clean":"2024-04-25T00:04:08.728712+0000","last_became_active":"2024-04-25T00:04:06.956208+0000","last_became_peered":"2024-04-25T00:04:06.956208+0000","last_unstale":"2024-04-25T00:04:08.728712+0000","last_undegraded":"2024-04-25T00:04:08.728712+0000","last_fullsized":"2024-04-25T00:04:08.728712+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:45.698203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728967+0000","last_change":"2024-04-25T00:04:06.953287+0000","last_active":"2024-04-25T00:04:08.728967+0000","last_peered":"2024-04-25T00:04:08.728967+0000","last_clean":"2024-04-25T00:04:08.728967+0000","last_became_active":"2024-04-25T00:04:06.952994+0000","last_became_peered":"2024-04-25T00:04:06.952994+0000","last_unstale":"2024-04-25T00:04:08.728967+0000","last_undegraded":"2024-04-25T00:04:08.728967+0000","last_fullsized":"2024-04-25T00:04:08.728967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.785779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730517+0000","last_change":"2024-04-25T00:04:06.958676+0000","last_active":"2024-04-25T00:04:08.730517+0000","last_peered":"2024-04-25T00:04:08.730517+0000","last_clean":"2024-04-25T00:04:08.730517+0000","last_became_active":"2024-04-25T00:04:06.958134+0000","last_became_peered":"2024-04-25T00:04:06.958134+0000","last_unstale":"2024-04-25T00:04:08.730517+0000","last_undegraded":"2024-04-25T00:04:08.730517+0000","last_fullsized":"2024-04-25T00:04:08.730517+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.611026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081899+0000","last_change":"2024-04-25T00:04:06.954489+0000","last_active":"2024-04-25T00:04:08.081899+0000","last_peered":"2024-04-25T00:04:08.081899+0000","last_clean":"2024-04-25T00:04:08.081899+0000","last_became_active":"2024-04-25T00:04:06.954231+0000","last_became_peered":"2024-04-25T00:04:06.954231+0000","last_unstale":"2024-04-25T00:04:08.081899+0000","last_undegraded":"2024-04-25T00:04:08.081899+0000","last_fullsized":"2024-04-25T00:04:08.081899+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:06.518045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730042+0000","last_change":"2024-04-25T00:04:06.958176+0000","last_active":"2024-04-25T00:04:08.730042+0000","last_peered":"2024-04-25T00:04:08.730042+0000","last_clean":"2024-04-25T00:04:08.730042+0000","last_became_active":"2024-04-25T00:04:06.957880+0000","last_became_peered":"2024-04-25T00:04:06.957880+0000","last_unstale":"2024-04-25T00:04:08.730042+0000","last_undegraded":"2024-04-25T00:04:08.730042+0000","last_fullsized":"2024-04-25T00:04:08.730042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:43.387751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942465+0000","last_change":"2024-04-25T00:04:06.954690+0000","last_active":"2024-04-25T00:04:07.942465+0000","last_peered":"2024-04-25T00:04:07.942465+0000","last_clean":"2024-04-25T00:04:07.942465+0000","last_became_active":"2024-04-25T00:04:06.954359+0000","last_became_peered":"2024-04-25T00:04:06.954359+0000","last_unstale":"2024-04-25T00:04:07.942465+0000","last_undegraded":"2024-04-25T00:04:07.942465+0000","last_fullsized":"2024-04-25T00:04:07.942465+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.916084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728925+0000","last_change":"2024-04-25T00:04:06.966541+0000","last_active":"2024-04-25T00:04:08.728925+0000","last_peered":"2024-04-25T00:04:08.728925+0000","last_clean":"2024-04-25T00:04:08.728925+0000","last_became_active":"2024-04-25T00:04:06.964906+0000","last_became_peered":"2024-04-25T00:04:06.964906+0000","last_unstale":"2024-04-25T00:04:08.728925+0000","last_undegraded":"2024-04-25T00:04:08.728925+0000","last_fullsized":"2024-04-25T00:04:08.728925+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:29.991925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941438+0000","last_change":"2024-04-25T00:04:06.954911+0000","last_active":"2024-04-25T00:04:07.941438+0000","last_peered":"2024-04-25T00:04:07.941438+0000","last_clean":"2024-04-25T00:04:07.941438+0000","last_became_active":"2024-04-25T00:04:06.954623+0000","last_became_peered":"2024-04-25T00:04:06.954623+0000","last_unstale":"2024-04-25T00:04:07.941438+0000","last_undegraded":"2024-04-25T00:04:07.941438+0000","last_fullsized":"2024-04-25T00:04:07.941438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:11.936001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728663+0000","last_change":"2024-04-25T00:04:06.958454+0000","last_active":"2024-04-25T00:04:08.728663+0000","last_peered":"2024-04-25T00:04:08.728663+0000","last_clean":"2024-04-25T00:04:08.728663+0000","last_became_active":"2024-04-25T00:04:06.958168+0000","last_became_peered":"2024-04-25T00:04:06.958168+0000","last_unstale":"2024-04-25T00:04:08.728663+0000","last_undegraded":"2024-04-25T00:04:08.728663+0000","last_fullsized":"2024-04-25T00:04:08.728663+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.611026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729761+0000","last_change":"2024-04-25T00:04:06.965845+0000","last_active":"2024-04-25T00:04:08.729761+0000","last_peered":"2024-04-25T00:04:08.729761+0000","last_clean":"2024-04-25T00:04:08.729761+0000","last_became_active":"2024-04-25T00:04:06.965479+0000","last_became_peered":"2024-04-25T00:04:06.965479+0000","last_unstale":"2024-04-25T00:04:08.729761+0000","last_undegraded":"2024-04-25T00:04:08.729761+0000","last_fullsized":"2024-04-25T00:04:08.729761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:46.284690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729096+0000","last_change":"2024-04-25T00:04:06.956525+0000","last_active":"2024-04-25T00:04:08.729096+0000","last_peered":"2024-04-25T00:04:08.729096+0000","last_clean":"2024-04-25T00:04:08.729096+0000","last_became_active":"2024-04-25T00:04:06.956231+0000","last_became_peered":"2024-04-25T00:04:06.956231+0000","last_unstale":"2024-04-25T00:04:08.729096+0000","last_undegraded":"2024-04-25T00:04:08.729096+0000","last_fullsized":"2024-04-25T00:04:08.729096+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.695696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942694+0000","last_change":"2024-04-25T00:04:06.962729+0000","last_active":"2024-04-25T00:04:07.942694+0000","last_peered":"2024-04-25T00:04:07.942694+0000","last_clean":"2024-04-25T00:04:07.942694+0000","last_became_active":"2024-04-25T00:04:06.962401+0000","last_became_peered":"2024-04-25T00:04:06.962401+0000","last_unstale":"2024-04-25T00:04:07.942694+0000","last_undegraded":"2024-04-25T00:04:07.942694+0000","last_fullsized":"2024-04-25T00:04:07.942694+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:42.565984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728981+0000","last_change":"2024-04-25T00:04:06.954206+0000","last_active":"2024-04-25T00:04:08.728981+0000","last_peered":"2024-04-25T00:04:08.728981+0000","last_clean":"2024-04-25T00:04:08.728981+0000","last_became_active":"2024-04-25T00:04:06.953948+0000","last_became_peered":"2024-04-25T00:04:06.953948+0000","last_unstale":"2024-04-25T00:04:08.728981+0000","last_undegraded":"2024-04-25T00:04:08.728981+0000","last_fullsized":"2024-04-25T00:04:08.728981+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:31.062481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728890+0000","last_change":"2024-04-25T00:04:06.954187+0000","last_active":"2024-04-25T00:04:08.728890+0000","last_peered":"2024-04-25T00:04:08.728890+0000","last_clean":"2024-04-25T00:04:08.728890+0000","last_became_active":"2024-04-25T00:04:06.953910+0000","last_became_peered":"2024-04-25T00:04:06.953910+0000","last_unstale":"2024-04-25T00:04:08.728890+0000","last_undegraded":"2024-04-25T00:04:08.728890+0000","last_fullsized":"2024-04-25T00:04:08.728890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:21.849815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081731+0000","last_change":"2024-04-25T00:04:06.963317+0000","last_active":"2024-04-25T00:04:08.081731+0000","last_peered":"2024-04-25T00:04:08.081731+0000","last_clean":"2024-04-25T00:04:08.081731+0000","last_became_active":"2024-04-25T00:04:06.963056+0000","last_became_peered":"2024-04-25T00:04:06.963056+0000","last_unstale":"2024-04-25T00:04:08.081731+0000","last_undegraded":"2024-04-25T00:04:08.081731+0000","last_fullsized":"2024-04-25T00:04:08.081731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:47.934078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729961+0000","last_change":"2024-04-25T00:04:06.946140+0000","last_active":"2024-04-25T00:04:08.729961+0000","last_peered":"2024-04-25T00:04:08.729961+0000","last_clean":"2024-04-25T00:04:08.729961+0000","last_became_active":"2024-04-25T00:04:06.945857+0000","last_became_peered":"2024-04-25T00:04:06.945857+0000","last_unstale":"2024-04-25T00:04:08.729961+0000","last_undegraded":"2024-04-25T00:04:08.729961+0000","last_fullsized":"2024-04-25T00:04:08.729961+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:22.782736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728681+0000","last_change":"2024-04-25T00:04:06.955247+0000","last_active":"2024-04-25T00:04:08.728681+0000","last_peered":"2024-04-25T00:04:08.728681+0000","last_clean":"2024-04-25T00:04:08.728681+0000","last_became_active":"2024-04-25T00:04:06.954406+0000","last_became_peered":"2024-04-25T00:04:06.954406+0000","last_unstale":"2024-04-25T00:04:08.728681+0000","last_undegraded":"2024-04-25T00:04:08.728681+0000","last_fullsized":"2024-04-25T00:04:08.728681+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:40.872518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081857+0000","last_change":"2024-04-25T00:04:06.964118+0000","last_active":"2024-04-25T00:04:08.081857+0000","last_peered":"2024-04-25T00:04:08.081857+0000","last_clean":"2024-04-25T00:04:08.081857+0000","last_became_active":"2024-04-25T00:04:06.963878+0000","last_became_peered":"2024-04-25T00:04:06.963878+0000","last_unstale":"2024-04-25T00:04:08.081857+0000","last_undegraded":"2024-04-25T00:04:08.081857+0000","last_fullsized":"2024-04-25T00:04:08.081857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:24.348997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942845+0000","last_change":"2024-04-25T00:04:06.954948+0000","last_active":"2024-04-25T00:04:07.942845+0000","last_peered":"2024-04-25T00:04:07.942845+0000","last_clean":"2024-04-25T00:04:07.942845+0000","last_became_active":"2024-04-25T00:04:06.954653+0000","last_became_peered":"2024-04-25T00:04:06.954653+0000","last_unstale":"2024-04-25T00:04:07.942845+0000","last_undegraded":"2024-04-25T00:04:07.942845+0000","last_fullsized":"2024-04-25T00:04:07.942845+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:06.870260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729141+0000","last_change":"2024-04-25T00:04:06.955683+0000","last_active":"2024-04-25T00:04:08.729141+0000","last_peered":"2024-04-25T00:04:08.729141+0000","last_clean":"2024-04-25T00:04:08.729141+0000","last_became_active":"2024-04-25T00:04:06.955354+0000","last_became_peered":"2024-04-25T00:04:06.955354+0000","last_unstale":"2024-04-25T00:04:08.729141+0000","last_undegraded":"2024-04-25T00:04:08.729141+0000","last_fullsized":"2024-04-25T00:04:08.729141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:03.189346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730193+0000","last_change":"2024-04-25T00:04:06.953751+0000","last_active":"2024-04-25T00:04:08.730193+0000","last_peered":"2024-04-25T00:04:08.730193+0000","last_clean":"2024-04-25T00:04:08.730193+0000","last_became_active":"2024-04-25T00:04:06.953474+0000","last_became_peered":"2024-04-25T00:04:06.953474+0000","last_unstale":"2024-04-25T00:04:08.730193+0000","last_undegraded":"2024-04-25T00:04:08.730193+0000","last_fullsized":"2024-04-25T00:04:08.730193+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:46.843144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082095+0000","last_change":"2024-04-25T00:04:06.958480+0000","last_active":"2024-04-25T00:04:08.082095+0000","last_peered":"2024-04-25T00:04:08.082095+0000","last_clean":"2024-04-25T00:04:08.082095+0000","last_became_active":"2024-04-25T00:04:06.956972+0000","last_became_peered":"2024-04-25T00:04:06.956972+0000","last_unstale":"2024-04-25T00:04:08.082095+0000","last_undegraded":"2024-04-25T00:04:08.082095+0000","last_fullsized":"2024-04-25T00:04:08.082095+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:57.845258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081781+0000","last_change":"2024-04-25T00:04:06.959665+0000","last_active":"2024-04-25T00:04:08.081781+0000","last_peered":"2024-04-25T00:04:08.081781+0000","last_clean":"2024-04-25T00:04:08.081781+0000","last_became_active":"2024-04-25T00:04:06.959404+0000","last_became_peered":"2024-04-25T00:04:06.959404+0000","last_unstale":"2024-04-25T00:04:08.081781+0000","last_undegraded":"2024-04-25T00:04:08.081781+0000","last_fullsized":"2024-04-25T00:04:08.081781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:45.308175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728876+0000","last_change":"2024-04-25T00:04:06.964438+0000","last_active":"2024-04-25T00:04:08.728876+0000","last_peered":"2024-04-25T00:04:08.728876+0000","last_clean":"2024-04-25T00:04:08.728876+0000","last_became_active":"2024-04-25T00:04:06.964110+0000","last_became_peered":"2024-04-25T00:04:06.964110+0000","last_unstale":"2024-04-25T00:04:08.728876+0000","last_undegraded":"2024-04-25T00:04:08.728876+0000","last_fullsized":"2024-04-25T00:04:08.728876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:03.695662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729571+0000","last_change":"2024-04-25T00:04:06.962878+0000","last_active":"2024-04-25T00:04:08.729571+0000","last_peered":"2024-04-25T00:04:08.729571+0000","last_clean":"2024-04-25T00:04:08.729571+0000","last_became_active":"2024-04-25T00:04:06.962600+0000","last_became_peered":"2024-04-25T00:04:06.962600+0000","last_unstale":"2024-04-25T00:04:08.729571+0000","last_undegraded":"2024-04-25T00:04:08.729571+0000","last_fullsized":"2024-04-25T00:04:08.729571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.810929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729922+0000","last_change":"2024-04-25T00:04:06.960444+0000","last_active":"2024-04-25T00:04:08.729922+0000","last_peered":"2024-04-25T00:04:08.729922+0000","last_clean":"2024-04-25T00:04:08.729922+0000","last_became_active":"2024-04-25T00:04:06.959875+0000","last_became_peered":"2024-04-25T00:04:06.959875+0000","last_unstale":"2024-04-25T00:04:08.729922+0000","last_undegraded":"2024-04-25T00:04:08.729922+0000","last_fullsized":"2024-04-25T00:04:08.729922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:16.203238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730609+0000","last_change":"2024-04-25T00:04:05.949584+0000","last_active":"2024-04-25T00:04:08.730609+0000","last_peered":"2024-04-25T00:04:08.730609+0000","last_clean":"2024-04-25T00:04:08.730609+0000","last_became_active":"2024-04-25T00:04:05.948893+0000","last_became_peered":"2024-04-25T00:04:05.948893+0000","last_unstale":"2024-04-25T00:04:08.730609+0000","last_undegraded":"2024-04-25T00:04:08.730609+0000","last_fullsized":"2024-04-25T00:04:08.730609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:05.513334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730605+0000","last_change":"2024-04-25T00:04:06.952292+0000","last_active":"2024-04-25T00:04:08.730605+0000","last_peered":"2024-04-25T00:04:08.730605+0000","last_clean":"2024-04-25T00:04:08.730605+0000","last_became_active":"2024-04-25T00:04:06.950108+0000","last_became_peered":"2024-04-25T00:04:06.950108+0000","last_unstale":"2024-04-25T00:04:08.730605+0000","last_undegraded":"2024-04-25T00:04:08.730605+0000","last_fullsized":"2024-04-25T00:04:08.730605+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:03.695662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730032+0000","last_change":"2024-04-25T00:04:05.943990+0000","last_active":"2024-04-25T00:04:08.730032+0000","last_peered":"2024-04-25T00:04:08.730032+0000","last_clean":"2024-04-25T00:04:08.730032+0000","last_became_active":"2024-04-25T00:04:05.943718+0000","last_became_peered":"2024-04-25T00:04:05.943718+0000","last_unstale":"2024-04-25T00:04:08.730032+0000","last_undegraded":"2024-04-25T00:04:08.730032+0000","last_fullsized":"2024-04-25T00:04:08.730032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:37.383495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730389+0000","last_change":"2024-04-25T00:04:06.962771+0000","last_active":"2024-04-25T00:04:08.730389+0000","last_peered":"2024-04-25T00:04:08.730389+0000","last_clean":"2024-04-25T00:04:08.730389+0000","last_became_active":"2024-04-25T00:04:06.962517+0000","last_became_peered":"2024-04-25T00:04:06.962517+0000","last_unstale":"2024-04-25T00:04:08.730389+0000","last_undegraded":"2024-04-25T00:04:08.730389+0000","last_fullsized":"2024-04-25T00:04:08.730389+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:33.559156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942663+0000","last_change":"2024-04-25T00:04:05.954463+0000","last_active":"2024-04-25T00:04:07.942663+0000","last_peered":"2024-04-25T00:04:07.942663+0000","last_clean":"2024-04-25T00:04:07.942663+0000","last_became_active":"2024-04-25T00:04:05.954215+0000","last_became_peered":"2024-04-25T00:04:05.954215+0000","last_unstale":"2024-04-25T00:04:07.942663+0000","last_undegraded":"2024-04-25T00:04:07.942663+0000","last_fullsized":"2024-04-25T00:04:07.942663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:45.838433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942830+0000","last_change":"2024-04-25T00:04:06.970413+0000","last_active":"2024-04-25T00:04:07.942830+0000","last_peered":"2024-04-25T00:04:07.942830+0000","last_clean":"2024-04-25T00:04:07.942830+0000","last_became_active":"2024-04-25T00:04:06.970129+0000","last_became_peered":"2024-04-25T00:04:06.970129+0000","last_unstale":"2024-04-25T00:04:07.942830+0000","last_undegraded":"2024-04-25T00:04:07.942830+0000","last_fullsized":"2024-04-25T00:04:07.942830+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:50.019061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942683+0000","last_change":"2024-04-25T00:04:05.948415+0000","last_active":"2024-04-25T00:04:07.942683+0000","last_peered":"2024-04-25T00:04:07.942683+0000","last_clean":"2024-04-25T00:04:07.942683+0000","last_became_active":"2024-04-25T00:04:05.948148+0000","last_became_peered":"2024-04-25T00:04:05.948148+0000","last_unstale":"2024-04-25T00:04:07.942683+0000","last_undegraded":"2024-04-25T00:04:07.942683+0000","last_fullsized":"2024-04-25T00:04:07.942683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:19.803498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081881+0000","last_change":"2024-04-25T00:04:06.959495+0000","last_active":"2024-04-25T00:04:08.081881+0000","last_peered":"2024-04-25T00:04:08.081881+0000","last_clean":"2024-04-25T00:04:08.081881+0000","last_became_active":"2024-04-25T00:04:06.959226+0000","last_became_peered":"2024-04-25T00:04:06.959226+0000","last_unstale":"2024-04-25T00:04:08.081881+0000","last_undegraded":"2024-04-25T00:04:08.081881+0000","last_fullsized":"2024-04-25T00:04:08.081881+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:31.537173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729879+0000","last_change":"2024-04-25T00:04:05.947803+0000","last_active":"2024-04-25T00:04:08.729879+0000","last_peered":"2024-04-25T00:04:08.729879+0000","last_clean":"2024-04-25T00:04:08.729879+0000","last_became_active":"2024-04-25T00:04:05.947378+0000","last_became_peered":"2024-04-25T00:04:05.947378+0000","last_unstale":"2024-04-25T00:04:08.729879+0000","last_undegraded":"2024-04-25T00:04:08.729879+0000","last_fullsized":"2024-04-25T00:04:08.729879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.139999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729682+0000","last_change":"2024-04-25T00:04:06.966781+0000","last_active":"2024-04-25T00:04:08.729682+0000","last_peered":"2024-04-25T00:04:08.729682+0000","last_clean":"2024-04-25T00:04:08.729682+0000","last_became_active":"2024-04-25T00:04:06.966507+0000","last_became_peered":"2024-04-25T00:04:06.966507+0000","last_unstale":"2024-04-25T00:04:08.729682+0000","last_undegraded":"2024-04-25T00:04:08.729682+0000","last_fullsized":"2024-04-25T00:04:08.729682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.785779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728966+0000","last_change":"2024-04-25T00:04:05.944427+0000","last_active":"2024-04-25T00:04:08.728966+0000","last_peered":"2024-04-25T00:04:08.728966+0000","last_clean":"2024-04-25T00:04:08.728966+0000","last_became_active":"2024-04-25T00:04:05.944008+0000","last_became_peered":"2024-04-25T00:04:05.944008+0000","last_unstale":"2024-04-25T00:04:08.728966+0000","last_undegraded":"2024-04-25T00:04:08.728966+0000","last_fullsized":"2024-04-25T00:04:08.728966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:57.489583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942411+0000","last_change":"2024-04-25T00:04:06.956402+0000","last_active":"2024-04-25T00:04:07.942411+0000","last_peered":"2024-04-25T00:04:07.942411+0000","last_clean":"2024-04-25T00:04:07.942411+0000","last_became_active":"2024-04-25T00:04:06.956097+0000","last_became_peered":"2024-04-25T00:04:06.956097+0000","last_unstale":"2024-04-25T00:04:07.942411+0000","last_undegraded":"2024-04-25T00:04:07.942411+0000","last_fullsized":"2024-04-25T00:04:07.942411+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:35.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942240+0000","last_change":"2024-04-25T00:04:05.955068+0000","last_active":"2024-04-25T00:04:07.942240+0000","last_peered":"2024-04-25T00:04:07.942240+0000","last_clean":"2024-04-25T00:04:07.942240+0000","last_became_active":"2024-04-25T00:04:05.954821+0000","last_became_peered":"2024-04-25T00:04:05.954821+0000","last_unstale":"2024-04-25T00:04:07.942240+0000","last_undegraded":"2024-04-25T00:04:07.942240+0000","last_fullsized":"2024-04-25T00:04:07.942240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:21.778025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941439+0000","last_change":"2024-04-25T00:04:06.954403+0000","last_active":"2024-04-25T00:04:07.941439+0000","last_peered":"2024-04-25T00:04:07.941439+0000","last_clean":"2024-04-25T00:04:07.941439+0000","last_became_active":"2024-04-25T00:04:06.954129+0000","last_became_peered":"2024-04-25T00:04:06.954129+0000","last_unstale":"2024-04-25T00:04:07.941439+0000","last_undegraded":"2024-04-25T00:04:07.941439+0000","last_fullsized":"2024-04-25T00:04:07.941439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.916084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081748+0000","last_change":"2024-04-25T00:04:05.951451+0000","last_active":"2024-04-25T00:04:08.081748+0000","last_peered":"2024-04-25T00:04:08.081748+0000","last_clean":"2024-04-25T00:04:08.081748+0000","last_became_active":"2024-04-25T00:04:05.951147+0000","last_became_peered":"2024-04-25T00:04:05.951147+0000","last_unstale":"2024-04-25T00:04:08.081748+0000","last_undegraded":"2024-04-25T00:04:08.081748+0000","last_fullsized":"2024-04-25T00:04:08.081748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:48.350400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941578+0000","last_change":"2024-04-25T00:04:06.954596+0000","last_active":"2024-04-25T00:04:07.941578+0000","last_peered":"2024-04-25T00:04:07.941578+0000","last_clean":"2024-04-25T00:04:07.941578+0000","last_became_active":"2024-04-25T00:04:06.954331+0000","last_became_peered":"2024-04-25T00:04:06.954331+0000","last_unstale":"2024-04-25T00:04:07.941578+0000","last_undegraded":"2024-04-25T00:04:07.941578+0000","last_fullsized":"2024-04-25T00:04:07.941578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:54.686746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730713+0000","last_change":"2024-04-25T00:04:05.949805+0000","last_active":"2024-04-25T00:04:08.730713+0000","last_peered":"2024-04-25T00:04:08.730713+0000","last_clean":"2024-04-25T00:04:08.730713+0000","last_became_active":"2024-04-25T00:04:05.949244+0000","last_became_peered":"2024-04-25T00:04:05.949244+0000","last_unstale":"2024-04-25T00:04:08.730713+0000","last_undegraded":"2024-04-25T00:04:08.730713+0000","last_fullsized":"2024-04-25T00:04:08.730713+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.672942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729132+0000","last_change":"2024-04-25T00:04:06.965478+0000","last_active":"2024-04-25T00:04:08.729132+0000","last_peered":"2024-04-25T00:04:08.729132+0000","last_clean":"2024-04-25T00:04:08.729132+0000","last_became_active":"2024-04-25T00:04:06.965210+0000","last_became_peered":"2024-04-25T00:04:06.965210+0000","last_unstale":"2024-04-25T00:04:08.729132+0000","last_undegraded":"2024-04-25T00:04:08.729132+0000","last_fullsized":"2024-04-25T00:04:08.729132+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:56.312120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730469+0000","last_change":"2024-04-25T00:04:05.952083+0000","last_active":"2024-04-25T00:04:08.730469+0000","last_peered":"2024-04-25T00:04:08.730469+0000","last_clean":"2024-04-25T00:04:08.730469+0000","last_became_active":"2024-04-25T00:04:05.951758+0000","last_became_peered":"2024-04-25T00:04:05.951758+0000","last_unstale":"2024-04-25T00:04:08.730469+0000","last_undegraded":"2024-04-25T00:04:08.730469+0000","last_fullsized":"2024-04-25T00:04:08.730469+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:46.929367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942601+0000","last_change":"2024-04-25T00:04:06.944843+0000","last_active":"2024-04-25T00:04:07.942601+0000","last_peered":"2024-04-25T00:04:07.942601+0000","last_clean":"2024-04-25T00:04:07.942601+0000","last_became_active":"2024-04-25T00:04:06.944602+0000","last_became_peered":"2024-04-25T00:04:06.944602+0000","last_unstale":"2024-04-25T00:04:07.942601+0000","last_undegraded":"2024-04-25T00:04:07.942601+0000","last_fullsized":"2024-04-25T00:04:07.942601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:45.698203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730029+0000","last_change":"2024-04-25T00:04:05.951264+0000","last_active":"2024-04-25T00:04:08.730029+0000","last_peered":"2024-04-25T00:04:08.730029+0000","last_clean":"2024-04-25T00:04:08.730029+0000","last_became_active":"2024-04-25T00:04:05.949917+0000","last_became_peered":"2024-04-25T00:04:05.949917+0000","last_unstale":"2024-04-25T00:04:08.730029+0000","last_undegraded":"2024-04-25T00:04:08.730029+0000","last_fullsized":"2024-04-25T00:04:08.730029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:40.495984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729299+0000","last_change":"2024-04-25T00:04:06.964012+0000","last_active":"2024-04-25T00:04:08.729299+0000","last_peered":"2024-04-25T00:04:08.729299+0000","last_clean":"2024-04-25T00:04:08.729299+0000","last_became_active":"2024-04-25T00:04:06.963702+0000","last_became_peered":"2024-04-25T00:04:06.963702+0000","last_unstale":"2024-04-25T00:04:08.729299+0000","last_undegraded":"2024-04-25T00:04:08.729299+0000","last_fullsized":"2024-04-25T00:04:08.729299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:32.180163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729358+0000","last_change":"2024-04-25T00:04:05.951475+0000","last_active":"2024-04-25T00:04:08.729358+0000","last_peered":"2024-04-25T00:04:08.729358+0000","last_clean":"2024-04-25T00:04:08.729358+0000","last_became_active":"2024-04-25T00:04:05.951195+0000","last_became_peered":"2024-04-25T00:04:05.951195+0000","last_unstale":"2024-04-25T00:04:08.729358+0000","last_undegraded":"2024-04-25T00:04:08.729358+0000","last_fullsized":"2024-04-25T00:04:08.729358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:45.838433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942348+0000","last_change":"2024-04-25T00:04:06.958086+0000","last_active":"2024-04-25T00:04:07.942348+0000","last_peered":"2024-04-25T00:04:07.942348+0000","last_clean":"2024-04-25T00:04:07.942348+0000","last_became_active":"2024-04-25T00:04:06.957793+0000","last_became_peered":"2024-04-25T00:04:06.957793+0000","last_unstale":"2024-04-25T00:04:07.942348+0000","last_undegraded":"2024-04-25T00:04:07.942348+0000","last_fullsized":"2024-04-25T00:04:07.942348+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:03.189346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082069+0000","last_change":"2024-04-25T00:04:05.949976+0000","last_active":"2024-04-25T00:04:08.082069+0000","last_peered":"2024-04-25T00:04:08.082069+0000","last_clean":"2024-04-25T00:04:08.082069+0000","last_became_active":"2024-04-25T00:04:05.948413+0000","last_became_peered":"2024-04-25T00:04:05.948413+0000","last_unstale":"2024-04-25T00:04:08.082069+0000","last_undegraded":"2024-04-25T00:04:08.082069+0000","last_fullsized":"2024-04-25T00:04:08.082069+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:18.406599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942591+0000","last_change":"2024-04-25T00:04:06.967827+0000","last_active":"2024-04-25T00:04:07.942591+0000","last_peered":"2024-04-25T00:04:07.942591+0000","last_clean":"2024-04-25T00:04:07.942591+0000","last_became_active":"2024-04-25T00:04:06.967488+0000","last_became_peered":"2024-04-25T00:04:06.967488+0000","last_unstale":"2024-04-25T00:04:07.942591+0000","last_undegraded":"2024-04-25T00:04:07.942591+0000","last_fullsized":"2024-04-25T00:04:07.942591+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:54.686746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941490+0000","last_change":"2024-04-25T00:04:05.946979+0000","last_active":"2024-04-25T00:04:07.941490+0000","last_peered":"2024-04-25T00:04:07.941490+0000","last_clean":"2024-04-25T00:04:07.941490+0000","last_became_active":"2024-04-25T00:04:05.946660+0000","last_became_peered":"2024-04-25T00:04:05.946660+0000","last_unstale":"2024-04-25T00:04:07.941490+0000","last_undegraded":"2024-04-25T00:04:07.941490+0000","last_fullsized":"2024-04-25T00:04:07.941490+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:39.155362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081642+0000","last_change":"2024-04-25T00:04:06.970858+0000","last_active":"2024-04-25T00:04:08.081642+0000","last_peered":"2024-04-25T00:04:08.081642+0000","last_clean":"2024-04-25T00:04:08.081642+0000","last_became_active":"2024-04-25T00:04:06.970568+0000","last_became_peered":"2024-04-25T00:04:06.970568+0000","last_unstale":"2024-04-25T00:04:08.081642+0000","last_undegraded":"2024-04-25T00:04:08.081642+0000","last_fullsized":"2024-04-25T00:04:08.081642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:57.548535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728906+0000","last_change":"2024-04-25T00:04:05.953311+0000","last_active":"2024-04-25T00:04:08.728906+0000","last_peered":"2024-04-25T00:04:08.728906+0000","last_clean":"2024-04-25T00:04:08.728906+0000","last_became_active":"2024-04-25T00:04:05.953035+0000","last_became_peered":"2024-04-25T00:04:05.953035+0000","last_unstale":"2024-04-25T00:04:08.728906+0000","last_undegraded":"2024-04-25T00:04:08.728906+0000","last_fullsized":"2024-04-25T00:04:08.728906+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:14.724496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081649+0000","last_change":"2024-04-25T00:04:06.963511+0000","last_active":"2024-04-25T00:04:08.081649+0000","last_peered":"2024-04-25T00:04:08.081649+0000","last_clean":"2024-04-25T00:04:08.081649+0000","last_became_active":"2024-04-25T00:04:06.963262+0000","last_became_peered":"2024-04-25T00:04:06.963262+0000","last_unstale":"2024-04-25T00:04:08.081649+0000","last_undegraded":"2024-04-25T00:04:08.081649+0000","last_fullsized":"2024-04-25T00:04:08.081649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:41.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942196+0000","last_change":"2024-04-25T00:04:05.953176+0000","last_active":"2024-04-25T00:04:07.942196+0000","last_peered":"2024-04-25T00:04:07.942196+0000","last_clean":"2024-04-25T00:04:07.942196+0000","last_became_active":"2024-04-25T00:04:05.952925+0000","last_became_peered":"2024-04-25T00:04:05.952925+0000","last_unstale":"2024-04-25T00:04:07.942196+0000","last_undegraded":"2024-04-25T00:04:07.942196+0000","last_fullsized":"2024-04-25T00:04:07.942196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.806218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942890+0000","last_change":"2024-04-25T00:04:06.953792+0000","last_active":"2024-04-25T00:04:07.942890+0000","last_peered":"2024-04-25T00:04:07.942890+0000","last_clean":"2024-04-25T00:04:07.942890+0000","last_became_active":"2024-04-25T00:04:06.953524+0000","last_became_peered":"2024-04-25T00:04:06.953524+0000","last_unstale":"2024-04-25T00:04:07.942890+0000","last_undegraded":"2024-04-25T00:04:07.942890+0000","last_fullsized":"2024-04-25T00:04:07.942890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:48.566137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730787+0000","last_change":"2024-04-25T00:04:05.952219+0000","last_active":"2024-04-25T00:04:08.730787+0000","last_peered":"2024-04-25T00:04:08.730787+0000","last_clean":"2024-04-25T00:04:08.730787+0000","last_became_active":"2024-04-25T00:04:05.951940+0000","last_became_peered":"2024-04-25T00:04:05.951940+0000","last_unstale":"2024-04-25T00:04:08.730787+0000","last_undegraded":"2024-04-25T00:04:08.730787+0000","last_fullsized":"2024-04-25T00:04:08.730787+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:23.344286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730497+0000","last_change":"2024-04-25T00:04:06.952571+0000","last_active":"2024-04-25T00:04:08.730497+0000","last_peered":"2024-04-25T00:04:08.730497+0000","last_clean":"2024-04-25T00:04:08.730497+0000","last_became_active":"2024-04-25T00:04:06.951778+0000","last_became_peered":"2024-04-25T00:04:06.951778+0000","last_unstale":"2024-04-25T00:04:08.730497+0000","last_undegraded":"2024-04-25T00:04:08.730497+0000","last_fullsized":"2024-04-25T00:04:08.730497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:11.572368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942540+0000","last_change":"2024-04-25T00:04:05.945666+0000","last_active":"2024-04-25T00:04:07.942540+0000","last_peered":"2024-04-25T00:04:07.942540+0000","last_clean":"2024-04-25T00:04:07.942540+0000","last_became_active":"2024-04-25T00:04:05.945410+0000","last_became_peered":"2024-04-25T00:04:05.945410+0000","last_unstale":"2024-04-25T00:04:07.942540+0000","last_undegraded":"2024-04-25T00:04:07.942540+0000","last_fullsized":"2024-04-25T00:04:07.942540+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:42.383041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729470+0000","last_change":"2024-04-25T00:04:06.960071+0000","last_active":"2024-04-25T00:04:08.729470+0000","last_peered":"2024-04-25T00:04:08.729470+0000","last_clean":"2024-04-25T00:04:08.729470+0000","last_became_active":"2024-04-25T00:04:06.959553+0000","last_became_peered":"2024-04-25T00:04:06.959553+0000","last_unstale":"2024-04-25T00:04:08.729470+0000","last_undegraded":"2024-04-25T00:04:08.729470+0000","last_fullsized":"2024-04-25T00:04:08.729470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:45.698203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942737+0000","last_change":"2024-04-25T00:04:05.957759+0000","last_active":"2024-04-25T00:04:07.942737+0000","last_peered":"2024-04-25T00:04:07.942737+0000","last_clean":"2024-04-25T00:04:07.942737+0000","last_became_active":"2024-04-25T00:04:05.957505+0000","last_became_peered":"2024-04-25T00:04:05.957505+0000","last_unstale":"2024-04-25T00:04:07.942737+0000","last_undegraded":"2024-04-25T00:04:07.942737+0000","last_fullsized":"2024-04-25T00:04:07.942737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:32.554445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728942+0000","last_change":"2024-04-25T00:04:06.968219+0000","last_active":"2024-04-25T00:04:08.728942+0000","last_peered":"2024-04-25T00:04:08.728942+0000","last_clean":"2024-04-25T00:04:08.728942+0000","last_became_active":"2024-04-25T00:04:06.966241+0000","last_became_peered":"2024-04-25T00:04:06.966241+0000","last_unstale":"2024-04-25T00:04:08.728942+0000","last_undegraded":"2024-04-25T00:04:08.728942+0000","last_fullsized":"2024-04-25T00:04:08.728942+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:43.352015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941705+0000","last_change":"2024-04-25T00:04:05.958193+0000","last_active":"2024-04-25T00:04:07.941705+0000","last_peered":"2024-04-25T00:04:07.941705+0000","last_clean":"2024-04-25T00:04:07.941705+0000","last_became_active":"2024-04-25T00:04:05.957902+0000","last_became_peered":"2024-04-25T00:04:05.957902+0000","last_unstale":"2024-04-25T00:04:07.941705+0000","last_undegraded":"2024-04-25T00:04:07.941705+0000","last_fullsized":"2024-04-25T00:04:07.941705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:50.454685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728776+0000","last_change":"2024-04-25T00:04:06.957603+0000","last_active":"2024-04-25T00:04:08.728776+0000","last_peered":"2024-04-25T00:04:08.728776+0000","last_clean":"2024-04-25T00:04:08.728776+0000","last_became_active":"2024-04-25T00:04:06.957293+0000","last_became_peered":"2024-04-25T00:04:06.957293+0000","last_unstale":"2024-04-25T00:04:08.728776+0000","last_undegraded":"2024-04-25T00:04:08.728776+0000","last_fullsized":"2024-04-25T00:04:08.728776+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:42.565984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729050+0000","last_change":"2024-04-25T00:04:05.956393+0000","last_active":"2024-04-25T00:04:08.729050+0000","last_peered":"2024-04-25T00:04:08.729050+0000","last_clean":"2024-04-25T00:04:08.729050+0000","last_became_active":"2024-04-25T00:04:05.955523+0000","last_became_peered":"2024-04-25T00:04:05.955523+0000","last_unstale":"2024-04-25T00:04:08.729050+0000","last_undegraded":"2024-04-25T00:04:08.729050+0000","last_fullsized":"2024-04-25T00:04:08.729050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:35.028683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728768+0000","last_change":"2024-04-25T00:04:06.960013+0000","last_active":"2024-04-25T00:04:08.728768+0000","last_peered":"2024-04-25T00:04:08.728768+0000","last_clean":"2024-04-25T00:04:08.728768+0000","last_became_active":"2024-04-25T00:04:06.957037+0000","last_became_peered":"2024-04-25T00:04:06.957037+0000","last_unstale":"2024-04-25T00:04:08.728768+0000","last_undegraded":"2024-04-25T00:04:08.728768+0000","last_fullsized":"2024-04-25T00:04:08.728768+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:56.473226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728743+0000","last_change":"2024-04-25T00:04:05.955280+0000","last_active":"2024-04-25T00:04:08.728743+0000","last_peered":"2024-04-25T00:04:08.728743+0000","last_clean":"2024-04-25T00:04:08.728743+0000","last_became_active":"2024-04-25T00:04:05.955021+0000","last_became_peered":"2024-04-25T00:04:05.955021+0000","last_unstale":"2024-04-25T00:04:08.728743+0000","last_undegraded":"2024-04-25T00:04:08.728743+0000","last_fullsized":"2024-04-25T00:04:08.728743+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:56.254060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729251+0000","last_change":"2024-04-25T00:04:06.953963+0000","last_active":"2024-04-25T00:04:08.729251+0000","last_peered":"2024-04-25T00:04:08.729251+0000","last_clean":"2024-04-25T00:04:08.729251+0000","last_became_active":"2024-04-25T00:04:06.953697+0000","last_became_peered":"2024-04-25T00:04:06.953697+0000","last_unstale":"2024-04-25T00:04:08.729251+0000","last_undegraded":"2024-04-25T00:04:08.729251+0000","last_fullsized":"2024-04-25T00:04:08.729251+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:16.203238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941358+0000","last_change":"2024-04-25T00:04:05.955799+0000","last_active":"2024-04-25T00:04:07.941358+0000","last_peered":"2024-04-25T00:04:07.941358+0000","last_clean":"2024-04-25T00:04:07.941358+0000","last_became_active":"2024-04-25T00:04:05.955566+0000","last_became_peered":"2024-04-25T00:04:05.955566+0000","last_unstale":"2024-04-25T00:04:07.941358+0000","last_undegraded":"2024-04-25T00:04:07.941358+0000","last_fullsized":"2024-04-25T00:04:07.941358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:37.193721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729985+0000","last_change":"2024-04-25T00:04:06.955956+0000","last_active":"2024-04-25T00:04:08.729985+0000","last_peered":"2024-04-25T00:04:08.729985+0000","last_clean":"2024-04-25T00:04:08.729985+0000","last_became_active":"2024-04-25T00:04:06.955663+0000","last_became_peered":"2024-04-25T00:04:06.955663+0000","last_unstale":"2024-04-25T00:04:08.729985+0000","last_undegraded":"2024-04-25T00:04:08.729985+0000","last_fullsized":"2024-04-25T00:04:08.729985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:38.198432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081907+0000","last_change":"2024-04-25T00:04:05.953036+0000","last_active":"2024-04-25T00:04:08.081907+0000","last_peered":"2024-04-25T00:04:08.081907+0000","last_clean":"2024-04-25T00:04:08.081907+0000","last_became_active":"2024-04-25T00:04:05.952742+0000","last_became_peered":"2024-04-25T00:04:05.952742+0000","last_unstale":"2024-04-25T00:04:08.081907+0000","last_undegraded":"2024-04-25T00:04:08.081907+0000","last_fullsized":"2024-04-25T00:04:08.081907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:51.577243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942904+0000","last_change":"2024-04-25T00:04:06.962964+0000","last_active":"2024-04-25T00:04:07.942904+0000","last_peered":"2024-04-25T00:04:07.942904+0000","last_clean":"2024-04-25T00:04:07.942904+0000","last_became_active":"2024-04-25T00:04:06.962676+0000","last_became_peered":"2024-04-25T00:04:06.962676+0000","last_unstale":"2024-04-25T00:04:07.942904+0000","last_undegraded":"2024-04-25T00:04:07.942904+0000","last_fullsized":"2024-04-25T00:04:07.942904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082115+0000","last_change":"2024-04-25T00:04:05.950505+0000","last_active":"2024-04-25T00:04:08.082115+0000","last_peered":"2024-04-25T00:04:08.082115+0000","last_clean":"2024-04-25T00:04:08.082115+0000","last_became_active":"2024-04-25T00:04:05.948719+0000","last_became_peered":"2024-04-25T00:04:05.948719+0000","last_unstale":"2024-04-25T00:04:08.082115+0000","last_undegraded":"2024-04-25T00:04:08.082115+0000","last_fullsized":"2024-04-25T00:04:08.082115+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:59.714801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730385+0000","last_change":"2024-04-25T00:04:06.962187+0000","last_active":"2024-04-25T00:04:08.730385+0000","last_peered":"2024-04-25T00:04:08.730385+0000","last_clean":"2024-04-25T00:04:08.730385+0000","last_became_active":"2024-04-25T00:04:06.961735+0000","last_became_peered":"2024-04-25T00:04:06.961735+0000","last_unstale":"2024-04-25T00:04:08.730385+0000","last_undegraded":"2024-04-25T00:04:08.730385+0000","last_fullsized":"2024-04-25T00:04:08.730385+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:20.808209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730227+0000","last_change":"2024-04-25T00:04:05.949348+0000","last_active":"2024-04-25T00:04:08.730227+0000","last_peered":"2024-04-25T00:04:08.730227+0000","last_clean":"2024-04-25T00:04:08.730227+0000","last_became_active":"2024-04-25T00:04:05.948760+0000","last_became_peered":"2024-04-25T00:04:05.948760+0000","last_unstale":"2024-04-25T00:04:08.730227+0000","last_undegraded":"2024-04-25T00:04:08.730227+0000","last_fullsized":"2024-04-25T00:04:08.730227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:44.303465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729644+0000","last_change":"2024-04-25T00:04:06.962874+0000","last_active":"2024-04-25T00:04:08.729644+0000","last_peered":"2024-04-25T00:04:08.729644+0000","last_clean":"2024-04-25T00:04:08.729644+0000","last_became_active":"2024-04-25T00:04:06.962602+0000","last_became_peered":"2024-04-25T00:04:06.962602+0000","last_unstale":"2024-04-25T00:04:08.729644+0000","last_undegraded":"2024-04-25T00:04:08.729644+0000","last_fullsized":"2024-04-25T00:04:08.729644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:42.565984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730515+0000","last_change":"2024-04-25T00:04:05.953709+0000","last_active":"2024-04-25T00:04:08.730515+0000","last_peered":"2024-04-25T00:04:08.730515+0000","last_clean":"2024-04-25T00:04:08.730515+0000","last_became_active":"2024-04-25T00:04:05.953453+0000","last_became_peered":"2024-04-25T00:04:05.953453+0000","last_unstale":"2024-04-25T00:04:08.730515+0000","last_undegraded":"2024-04-25T00:04:08.730515+0000","last_fullsized":"2024-04-25T00:04:08.730515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:30.532462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729295+0000","last_change":"2024-04-25T00:04:06.957488+0000","last_active":"2024-04-25T00:04:08.729295+0000","last_peered":"2024-04-25T00:04:08.729295+0000","last_clean":"2024-04-25T00:04:08.729295+0000","last_became_active":"2024-04-25T00:04:06.957191+0000","last_became_peered":"2024-04-25T00:04:06.957191+0000","last_unstale":"2024-04-25T00:04:08.729295+0000","last_undegraded":"2024-04-25T00:04:08.729295+0000","last_fullsized":"2024-04-25T00:04:08.729295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:38.198432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942441+0000","last_change":"2024-04-25T00:04:05.953267+0000","last_active":"2024-04-25T00:04:07.942441+0000","last_peered":"2024-04-25T00:04:07.942441+0000","last_clean":"2024-04-25T00:04:07.942441+0000","last_became_active":"2024-04-25T00:04:05.953021+0000","last_became_peered":"2024-04-25T00:04:05.953021+0000","last_unstale":"2024-04-25T00:04:07.942441+0000","last_undegraded":"2024-04-25T00:04:07.942441+0000","last_fullsized":"2024-04-25T00:04:07.942441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:14.724496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942463+0000","last_change":"2024-04-25T00:04:06.961589+0000","last_active":"2024-04-25T00:04:07.942463+0000","last_peered":"2024-04-25T00:04:07.942463+0000","last_clean":"2024-04-25T00:04:07.942463+0000","last_became_active":"2024-04-25T00:04:06.961267+0000","last_became_peered":"2024-04-25T00:04:06.961267+0000","last_unstale":"2024-04-25T00:04:07.942463+0000","last_undegraded":"2024-04-25T00:04:07.942463+0000","last_fullsized":"2024-04-25T00:04:07.942463+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:32.180163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729078+0000","last_change":"2024-04-25T00:04:05.944307+0000","last_active":"2024-04-25T00:04:08.729078+0000","last_peered":"2024-04-25T00:04:08.729078+0000","last_clean":"2024-04-25T00:04:08.729078+0000","last_became_active":"2024-04-25T00:04:05.943618+0000","last_became_peered":"2024-04-25T00:04:05.943618+0000","last_unstale":"2024-04-25T00:04:08.729078+0000","last_undegraded":"2024-04-25T00:04:08.729078+0000","last_fullsized":"2024-04-25T00:04:08.729078+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:42.595520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942451+0000","last_change":"2024-04-25T00:04:06.958042+0000","last_active":"2024-04-25T00:04:07.942451+0000","last_peered":"2024-04-25T00:04:07.942451+0000","last_clean":"2024-04-25T00:04:07.942451+0000","last_became_active":"2024-04-25T00:04:06.957721+0000","last_became_peered":"2024-04-25T00:04:06.957721+0000","last_unstale":"2024-04-25T00:04:07.942451+0000","last_undegraded":"2024-04-25T00:04:07.942451+0000","last_fullsized":"2024-04-25T00:04:07.942451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:11.936001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730138+0000","last_change":"2024-04-25T00:04:05.949003+0000","last_active":"2024-04-25T00:04:08.730138+0000","last_peered":"2024-04-25T00:04:08.730138+0000","last_clean":"2024-04-25T00:04:08.730138+0000","last_became_active":"2024-04-25T00:04:05.948731+0000","last_became_peered":"2024-04-25T00:04:05.948731+0000","last_unstale":"2024-04-25T00:04:08.730138+0000","last_undegraded":"2024-04-25T00:04:08.730138+0000","last_fullsized":"2024-04-25T00:04:08.730138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:56.543825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730137+0000","last_change":"2024-04-25T00:04:06.962198+0000","last_active":"2024-04-25T00:04:08.730137+0000","last_peered":"2024-04-25T00:04:08.730137+0000","last_clean":"2024-04-25T00:04:08.730137+0000","last_became_active":"2024-04-25T00:04:06.961841+0000","last_became_peered":"2024-04-25T00:04:06.961841+0000","last_unstale":"2024-04-25T00:04:08.730137+0000","last_undegraded":"2024-04-25T00:04:08.730137+0000","last_fullsized":"2024-04-25T00:04:08.730137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:15.729207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728719+0000","last_change":"2024-04-25T00:04:05.949157+0000","last_active":"2024-04-25T00:04:08.728719+0000","last_peered":"2024-04-25T00:04:08.728719+0000","last_clean":"2024-04-25T00:04:08.728719+0000","last_became_active":"2024-04-25T00:04:05.948879+0000","last_became_peered":"2024-04-25T00:04:05.948879+0000","last_unstale":"2024-04-25T00:04:08.728719+0000","last_undegraded":"2024-04-25T00:04:08.728719+0000","last_fullsized":"2024-04-25T00:04:08.728719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:40.441664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081958+0000","last_change":"2024-04-25T00:04:06.954858+0000","last_active":"2024-04-25T00:04:08.081958+0000","last_peered":"2024-04-25T00:04:08.081958+0000","last_clean":"2024-04-25T00:04:08.081958+0000","last_became_active":"2024-04-25T00:04:06.954554+0000","last_became_peered":"2024-04-25T00:04:06.954554+0000","last_unstale":"2024-04-25T00:04:08.081958+0000","last_undegraded":"2024-04-25T00:04:08.081958+0000","last_fullsized":"2024-04-25T00:04:08.081958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:19.677653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942966+0000","last_change":"2024-04-25T00:04:05.957149+0000","last_active":"2024-04-25T00:04:07.942966+0000","last_peered":"2024-04-25T00:04:07.942966+0000","last_clean":"2024-04-25T00:04:07.942966+0000","last_became_active":"2024-04-25T00:04:05.956870+0000","last_became_peered":"2024-04-25T00:04:05.956870+0000","last_unstale":"2024-04-25T00:04:07.942966+0000","last_undegraded":"2024-04-25T00:04:07.942966+0000","last_fullsized":"2024-04-25T00:04:07.942966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:56.781068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":302,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942651+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:04:07.942651+0000","last_peered":"2024-04-25T00:04:07.942651+0000","last_clean":"2024-04-25T00:04:07.942651+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:04:07.942651+0000","last_undegraded":"2024-04-25T00:04:07.942651+0000","last_fullsized":"2024-04-25T00:04:07.942651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55084,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316756,"statfs":{"total":96636764160,"available":96580358144,"internally_reserved":0,"allocated":1728512,"data_stored":1078753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54504,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317336,"statfs":{"total":96636764160,"available":96580952064,"internally_reserved":0,"allocated":1142784,"data_stored":489009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1146880,"data_stored":489949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54492,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317348,"statfs":{"total":96636764160,"available":96580964352,"internally_reserved":0,"allocated":1130496,"data_stored":488315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58672,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313168,"statfs":{"total":96636764160,"available":96576684032,"internally_reserved":0,"allocated":1142784,"data_stored":489927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1134592,"data_stored":488897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":1142784,"data_stored":488509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41188,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330652,"statfs":{"total":96636764160,"available":96594587648,"internally_reserved":0,"allocated":1728512,"data_stored":1078717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:04:12.201 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:04:12.200+0000 7f5b95c55640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:04:14.281 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:04:14.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.388+0000 7fbd5875c640 1 Processor -- start 2024-04-25T00:04:14.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.388+0000 7fbd5875c640 1 -- start start 2024-04-25T00:04:14.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.389+0000 7fbd5875c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd500b3fd0 0x7fbd5006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.389+0000 7fbd5875c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd5006e510 0x7fbd500cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.389+0000 7fbd5875c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd50144cf0 con 0x7fbd50144910 2024-04-25T00:04:14.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.389+0000 7fbd5875c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbd500b2f30 con 0x7fbd5006e510 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.389+0000 7fbd5875c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd500b3210 con 0x7fbd500b3fd0 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd56cf9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd5006e510 0x7fbd500cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd564f8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbd50144910 0x7fbd500b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34756/0 (socket says 172.21.15.113:34756) 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd564f8640 1 -- 172.21.15.113:0/4179922262 learned_addr learned my addr 172.21.15.113:0/4179922262 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd55cf7640 1 --2- 172.21.15.113:0/4179922262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd500b3fd0 0x7fbd5006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd56cf9640 1 -- 172.21.15.113:0/4179922262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd500b3fd0 msgr2=0x7fbd5006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd56cf9640 1 --2- 172.21.15.113:0/4179922262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd500b3fd0 0x7fbd5006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd56cf9640 1 -- 172.21.15.113:0/4179922262 >> v1:172.21.15.113:6789/0 conn(0x7fbd50144910 legacy=0x7fbd500b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd56cf9640 1 -- 172.21.15.113:0/4179922262 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd500b34f0 con 0x7fbd5006e510 2024-04-25T00:04:14.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.390+0000 7fbd554f6640 1 -- 172.21.15.113:0/4179922262 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891219417 0 0) 0x7fbd50144cf0 con 0x7fbd50144910 2024-04-25T00:04:14.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd56cf9640 1 --2- 172.21.15.113:0/4179922262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd5006e510 0x7fbd500cd500 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009d90 tx=0x7fbd4802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2e1e153ebc1e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:14.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd554f6640 1 -- 172.21.15.113:0/4179922262 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48007740 con 0x7fbd5006e510 2024-04-25T00:04:14.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd554f6640 1 -- 172.21.15.113:0/4179922262 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd480078e0 con 0x7fbd5006e510 2024-04-25T00:04:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd554f6640 1 -- 172.21.15.113:0/4179922262 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48007cb0 con 0x7fbd5006e510 2024-04-25T00:04:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 -- 172.21.15.113:0/4179922262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd5006e510 msgr2=0x7fbd500cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 --2- 172.21.15.113:0/4179922262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd5006e510 0x7fbd500cd500 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009d90 tx=0x7fbd4802a660 comp rx=0 tx=0).stop 2024-04-25T00:04:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 -- 172.21.15.113:0/4179922262 shutdown_connections 2024-04-25T00:04:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 --2- 172.21.15.113:0/4179922262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd5006e510 0x7fbd500cd500 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 --2- 172.21.15.113:0/4179922262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd500b3fd0 0x7fbd5006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 -- 172.21.15.113:0/4179922262 >> 172.21.15.113:0/4179922262 conn(0x7fbd500b1a50 msgr2=0x7fbd500b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 -- 172.21.15.113:0/4179922262 shutdown_connections 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.391+0000 7fbd5875c640 1 -- 172.21.15.113:0/4179922262 wait complete. 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 Processor -- start 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 -- start start 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd5006e510 0x7fbd500a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd500b3fd0 0x7fbd501323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd50144910 0x7fbd500b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbd500b4da0 con 0x7fbd500b3fd0 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd500b4410 con 0x7fbd5006e510 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd5875c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd500b4f20 con 0x7fbd50144910 2024-04-25T00:04:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd55cf7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd500b3fd0 0x7fbd501323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd55cf7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd500b3fd0 0x7fbd501323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53988/0 (socket says 172.21.15.113:53988) 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.393+0000 7fbd55cf7640 1 -- 172.21.15.113:0/2709879822 learned_addr learned my addr 172.21.15.113:0/2709879822 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd564f8640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd5006e510 0x7fbd500a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd56cf9640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd50144910 0x7fbd500b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd55cf7640 1 -- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd50144910 msgr2=0x7fbd500b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd55cf7640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd50144910 0x7fbd500b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd55cf7640 1 -- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd5006e510 msgr2=0x7fbd500a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd55cf7640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd5006e510 0x7fbd500a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd55cf7640 1 -- 172.21.15.113:0/2709879822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd50132e80 con 0x7fbd500b3fd0 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd56cf9640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd50144910 0x7fbd500b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd55cf7640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd500b3fd0 0x7fbd501323e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d800 tx=0x7fbd4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd46ffd640 1 -- 172.21.15.113:0/2709879822 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00b6c0 con 0x7fbd500b3fd0 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd46ffd640 1 -- 172.21.15.113:0/2709879822 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4c019070 con 0x7fbd500b3fd0 2024-04-25T00:04:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd50133160 con 0x7fbd500b3fd0 2024-04-25T00:04:14.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd46ffd640 1 -- 172.21.15.113:0/2709879822 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00ba80 con 0x7fbd500b3fd0 2024-04-25T00:04:14.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.394+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5013f6a0 con 0x7fbd500b3fd0 2024-04-25T00:04:14.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.395+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd50130350 con 0x7fbd500b3fd0 2024-04-25T00:04:14.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.395+0000 7fbd46ffd640 1 -- 172.21.15.113:0/2709879822 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd4c00bc20 con 0x7fbd500b3fd0 2024-04-25T00:04:14.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.396+0000 7fbd46ffd640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbd3406be60 0x7fbd3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.396+0000 7fbd46ffd640 1 -- 172.21.15.113:0/2709879822 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fbd4c08b240 con 0x7fbd500b3fd0 2024-04-25T00:04:14.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.396+0000 7fbd564f8640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbd3406be60 0x7fbd3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.396+0000 7fbd564f8640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbd3406be60 0x7fbd3406e320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000f60 tx=0x7fbd40007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.400+0000 7fbd46ffd640 1 -- 172.21.15.113:0/2709879822 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbd4c01c360 con 0x7fbd500b3fd0 2024-04-25T00:04:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.569+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd500691a0 con 0x7fbd3406be60 2024-04-25T00:04:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.575+0000 7fbd46ffd640 1 -- 172.21.15.113:0/2709879822 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252088 (secure 0 0 0) 0x7fbd500691a0 con 0x7fbd3406be60 2024-04-25T00:04:14.578 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:14.579 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:04:14.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbd3406be60 msgr2=0x7fbd3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbd3406be60 0x7fbd3406e320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbd40000f60 tx=0x7fbd40007c00 comp rx=0 tx=0).stop 2024-04-25T00:04:14.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd500b3fd0 msgr2=0x7fbd501323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd500b3fd0 0x7fbd501323e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d800 tx=0x7fbd4c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 shutdown_connections 2024-04-25T00:04:14.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbd3406be60 0x7fbd3406e320 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd50144910 0x7fbd500b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd500b3fd0 0x7fbd501323e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 --2- 172.21.15.113:0/2709879822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd5006e510 0x7fbd500a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 >> 172.21.15.113:0/2709879822 conn(0x7fbd500b1a50 msgr2=0x7fbd500b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:14.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 shutdown_connections 2024-04-25T00:04:14.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.578+0000 7fbd5875c640 1 -- 172.21.15.113:0/2709879822 wait complete. 2024-04-25T00:04:14.589 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-04-25T00:04:13.181747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383040,"kb_used_data":10056,"kb_used_omap":12,"kb_used_meta":372851,"kb_avail":754591680,"statfs":{"total":773094113280,"available":772701880320,"internally_reserved":0,"allocated":10297344,"data_stored":5092076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":381799984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.242483"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730415+0000","last_change":"2024-04-25T00:04:06.952353+0000","last_active":"2024-04-25T00:04:08.730415+0000","last_peered":"2024-04-25T00:04:08.730415+0000","last_clean":"2024-04-25T00:04:08.730415+0000","last_became_active":"2024-04-25T00:04:06.949248+0000","last_became_peered":"2024-04-25T00:04:06.949248+0000","last_unstale":"2024-04-25T00:04:08.730415+0000","last_undegraded":"2024-04-25T00:04:08.730415+0000","last_fullsized":"2024-04-25T00:04:08.730415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:11.974068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082207+0000","last_change":"2024-04-25T00:04:06.959519+0000","last_active":"2024-04-25T00:04:08.082207+0000","last_peered":"2024-04-25T00:04:08.082207+0000","last_clean":"2024-04-25T00:04:08.082207+0000","last_became_active":"2024-04-25T00:04:06.959247+0000","last_became_peered":"2024-04-25T00:04:06.959247+0000","last_unstale":"2024-04-25T00:04:08.082207+0000","last_undegraded":"2024-04-25T00:04:08.082207+0000","last_fullsized":"2024-04-25T00:04:08.082207+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:49.779489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942843+0000","last_change":"2024-04-25T00:04:06.956520+0000","last_active":"2024-04-25T00:04:07.942843+0000","last_peered":"2024-04-25T00:04:07.942843+0000","last_clean":"2024-04-25T00:04:07.942843+0000","last_became_active":"2024-04-25T00:04:06.956166+0000","last_became_peered":"2024-04-25T00:04:06.956166+0000","last_unstale":"2024-04-25T00:04:07.942843+0000","last_undegraded":"2024-04-25T00:04:07.942843+0000","last_fullsized":"2024-04-25T00:04:07.942843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:10.830589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728945+0000","last_change":"2024-04-25T00:04:06.965710+0000","last_active":"2024-04-25T00:04:08.728945+0000","last_peered":"2024-04-25T00:04:08.728945+0000","last_clean":"2024-04-25T00:04:08.728945+0000","last_became_active":"2024-04-25T00:04:06.965463+0000","last_became_peered":"2024-04-25T00:04:06.965463+0000","last_unstale":"2024-04-25T00:04:08.728945+0000","last_undegraded":"2024-04-25T00:04:08.728945+0000","last_fullsized":"2024-04-25T00:04:08.728945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:23.761487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729734+0000","last_change":"2024-04-25T00:04:06.959847+0000","last_active":"2024-04-25T00:04:08.729734+0000","last_peered":"2024-04-25T00:04:08.729734+0000","last_clean":"2024-04-25T00:04:08.729734+0000","last_became_active":"2024-04-25T00:04:06.959557+0000","last_became_peered":"2024-04-25T00:04:06.959557+0000","last_unstale":"2024-04-25T00:04:08.729734+0000","last_undegraded":"2024-04-25T00:04:08.729734+0000","last_fullsized":"2024-04-25T00:04:08.729734+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:56.312120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729107+0000","last_change":"2024-04-25T00:04:06.969950+0000","last_active":"2024-04-25T00:04:08.729107+0000","last_peered":"2024-04-25T00:04:08.729107+0000","last_clean":"2024-04-25T00:04:08.729107+0000","last_became_active":"2024-04-25T00:04:06.969657+0000","last_became_peered":"2024-04-25T00:04:06.969657+0000","last_unstale":"2024-04-25T00:04:08.729107+0000","last_undegraded":"2024-04-25T00:04:08.729107+0000","last_fullsized":"2024-04-25T00:04:08.729107+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:46.284690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728707+0000","last_change":"2024-04-25T00:04:06.955896+0000","last_active":"2024-04-25T00:04:08.728707+0000","last_peered":"2024-04-25T00:04:08.728707+0000","last_clean":"2024-04-25T00:04:08.728707+0000","last_became_active":"2024-04-25T00:04:06.955627+0000","last_became_peered":"2024-04-25T00:04:06.955627+0000","last_unstale":"2024-04-25T00:04:08.728707+0000","last_undegraded":"2024-04-25T00:04:08.728707+0000","last_fullsized":"2024-04-25T00:04:08.728707+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:44.699696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728712+0000","last_change":"2024-04-25T00:04:06.956600+0000","last_active":"2024-04-25T00:04:08.728712+0000","last_peered":"2024-04-25T00:04:08.728712+0000","last_clean":"2024-04-25T00:04:08.728712+0000","last_became_active":"2024-04-25T00:04:06.956208+0000","last_became_peered":"2024-04-25T00:04:06.956208+0000","last_unstale":"2024-04-25T00:04:08.728712+0000","last_undegraded":"2024-04-25T00:04:08.728712+0000","last_fullsized":"2024-04-25T00:04:08.728712+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:45.698203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728967+0000","last_change":"2024-04-25T00:04:06.953287+0000","last_active":"2024-04-25T00:04:08.728967+0000","last_peered":"2024-04-25T00:04:08.728967+0000","last_clean":"2024-04-25T00:04:08.728967+0000","last_became_active":"2024-04-25T00:04:06.952994+0000","last_became_peered":"2024-04-25T00:04:06.952994+0000","last_unstale":"2024-04-25T00:04:08.728967+0000","last_undegraded":"2024-04-25T00:04:08.728967+0000","last_fullsized":"2024-04-25T00:04:08.728967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.785779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730517+0000","last_change":"2024-04-25T00:04:06.958676+0000","last_active":"2024-04-25T00:04:08.730517+0000","last_peered":"2024-04-25T00:04:08.730517+0000","last_clean":"2024-04-25T00:04:08.730517+0000","last_became_active":"2024-04-25T00:04:06.958134+0000","last_became_peered":"2024-04-25T00:04:06.958134+0000","last_unstale":"2024-04-25T00:04:08.730517+0000","last_undegraded":"2024-04-25T00:04:08.730517+0000","last_fullsized":"2024-04-25T00:04:08.730517+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.611026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081899+0000","last_change":"2024-04-25T00:04:06.954489+0000","last_active":"2024-04-25T00:04:08.081899+0000","last_peered":"2024-04-25T00:04:08.081899+0000","last_clean":"2024-04-25T00:04:08.081899+0000","last_became_active":"2024-04-25T00:04:06.954231+0000","last_became_peered":"2024-04-25T00:04:06.954231+0000","last_unstale":"2024-04-25T00:04:08.081899+0000","last_undegraded":"2024-04-25T00:04:08.081899+0000","last_fullsized":"2024-04-25T00:04:08.081899+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:06.518045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730042+0000","last_change":"2024-04-25T00:04:06.958176+0000","last_active":"2024-04-25T00:04:08.730042+0000","last_peered":"2024-04-25T00:04:08.730042+0000","last_clean":"2024-04-25T00:04:08.730042+0000","last_became_active":"2024-04-25T00:04:06.957880+0000","last_became_peered":"2024-04-25T00:04:06.957880+0000","last_unstale":"2024-04-25T00:04:08.730042+0000","last_undegraded":"2024-04-25T00:04:08.730042+0000","last_fullsized":"2024-04-25T00:04:08.730042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:43.387751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942465+0000","last_change":"2024-04-25T00:04:06.954690+0000","last_active":"2024-04-25T00:04:07.942465+0000","last_peered":"2024-04-25T00:04:07.942465+0000","last_clean":"2024-04-25T00:04:07.942465+0000","last_became_active":"2024-04-25T00:04:06.954359+0000","last_became_peered":"2024-04-25T00:04:06.954359+0000","last_unstale":"2024-04-25T00:04:07.942465+0000","last_undegraded":"2024-04-25T00:04:07.942465+0000","last_fullsized":"2024-04-25T00:04:07.942465+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.916084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728925+0000","last_change":"2024-04-25T00:04:06.966541+0000","last_active":"2024-04-25T00:04:08.728925+0000","last_peered":"2024-04-25T00:04:08.728925+0000","last_clean":"2024-04-25T00:04:08.728925+0000","last_became_active":"2024-04-25T00:04:06.964906+0000","last_became_peered":"2024-04-25T00:04:06.964906+0000","last_unstale":"2024-04-25T00:04:08.728925+0000","last_undegraded":"2024-04-25T00:04:08.728925+0000","last_fullsized":"2024-04-25T00:04:08.728925+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:51:29.991925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941438+0000","last_change":"2024-04-25T00:04:06.954911+0000","last_active":"2024-04-25T00:04:07.941438+0000","last_peered":"2024-04-25T00:04:07.941438+0000","last_clean":"2024-04-25T00:04:07.941438+0000","last_became_active":"2024-04-25T00:04:06.954623+0000","last_became_peered":"2024-04-25T00:04:06.954623+0000","last_unstale":"2024-04-25T00:04:07.941438+0000","last_undegraded":"2024-04-25T00:04:07.941438+0000","last_fullsized":"2024-04-25T00:04:07.941438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:11.936001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728663+0000","last_change":"2024-04-25T00:04:06.958454+0000","last_active":"2024-04-25T00:04:08.728663+0000","last_peered":"2024-04-25T00:04:08.728663+0000","last_clean":"2024-04-25T00:04:08.728663+0000","last_became_active":"2024-04-25T00:04:06.958168+0000","last_became_peered":"2024-04-25T00:04:06.958168+0000","last_unstale":"2024-04-25T00:04:08.728663+0000","last_undegraded":"2024-04-25T00:04:08.728663+0000","last_fullsized":"2024-04-25T00:04:08.728663+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.611026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729761+0000","last_change":"2024-04-25T00:04:06.965845+0000","last_active":"2024-04-25T00:04:08.729761+0000","last_peered":"2024-04-25T00:04:08.729761+0000","last_clean":"2024-04-25T00:04:08.729761+0000","last_became_active":"2024-04-25T00:04:06.965479+0000","last_became_peered":"2024-04-25T00:04:06.965479+0000","last_unstale":"2024-04-25T00:04:08.729761+0000","last_undegraded":"2024-04-25T00:04:08.729761+0000","last_fullsized":"2024-04-25T00:04:08.729761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:46.284690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729096+0000","last_change":"2024-04-25T00:04:06.956525+0000","last_active":"2024-04-25T00:04:08.729096+0000","last_peered":"2024-04-25T00:04:08.729096+0000","last_clean":"2024-04-25T00:04:08.729096+0000","last_became_active":"2024-04-25T00:04:06.956231+0000","last_became_peered":"2024-04-25T00:04:06.956231+0000","last_unstale":"2024-04-25T00:04:08.729096+0000","last_undegraded":"2024-04-25T00:04:08.729096+0000","last_fullsized":"2024-04-25T00:04:08.729096+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:56.695696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942694+0000","last_change":"2024-04-25T00:04:06.962729+0000","last_active":"2024-04-25T00:04:07.942694+0000","last_peered":"2024-04-25T00:04:07.942694+0000","last_clean":"2024-04-25T00:04:07.942694+0000","last_became_active":"2024-04-25T00:04:06.962401+0000","last_became_peered":"2024-04-25T00:04:06.962401+0000","last_unstale":"2024-04-25T00:04:07.942694+0000","last_undegraded":"2024-04-25T00:04:07.942694+0000","last_fullsized":"2024-04-25T00:04:07.942694+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:42.565984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728981+0000","last_change":"2024-04-25T00:04:06.954206+0000","last_active":"2024-04-25T00:04:08.728981+0000","last_peered":"2024-04-25T00:04:08.728981+0000","last_clean":"2024-04-25T00:04:08.728981+0000","last_became_active":"2024-04-25T00:04:06.953948+0000","last_became_peered":"2024-04-25T00:04:06.953948+0000","last_unstale":"2024-04-25T00:04:08.728981+0000","last_undegraded":"2024-04-25T00:04:08.728981+0000","last_fullsized":"2024-04-25T00:04:08.728981+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:31.062481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728890+0000","last_change":"2024-04-25T00:04:06.954187+0000","last_active":"2024-04-25T00:04:08.728890+0000","last_peered":"2024-04-25T00:04:08.728890+0000","last_clean":"2024-04-25T00:04:08.728890+0000","last_became_active":"2024-04-25T00:04:06.953910+0000","last_became_peered":"2024-04-25T00:04:06.953910+0000","last_unstale":"2024-04-25T00:04:08.728890+0000","last_undegraded":"2024-04-25T00:04:08.728890+0000","last_fullsized":"2024-04-25T00:04:08.728890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:21.849815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081731+0000","last_change":"2024-04-25T00:04:06.963317+0000","last_active":"2024-04-25T00:04:08.081731+0000","last_peered":"2024-04-25T00:04:08.081731+0000","last_clean":"2024-04-25T00:04:08.081731+0000","last_became_active":"2024-04-25T00:04:06.963056+0000","last_became_peered":"2024-04-25T00:04:06.963056+0000","last_unstale":"2024-04-25T00:04:08.081731+0000","last_undegraded":"2024-04-25T00:04:08.081731+0000","last_fullsized":"2024-04-25T00:04:08.081731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:47.934078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729961+0000","last_change":"2024-04-25T00:04:06.946140+0000","last_active":"2024-04-25T00:04:08.729961+0000","last_peered":"2024-04-25T00:04:08.729961+0000","last_clean":"2024-04-25T00:04:08.729961+0000","last_became_active":"2024-04-25T00:04:06.945857+0000","last_became_peered":"2024-04-25T00:04:06.945857+0000","last_unstale":"2024-04-25T00:04:08.729961+0000","last_undegraded":"2024-04-25T00:04:08.729961+0000","last_fullsized":"2024-04-25T00:04:08.729961+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:22.782736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728681+0000","last_change":"2024-04-25T00:04:06.955247+0000","last_active":"2024-04-25T00:04:08.728681+0000","last_peered":"2024-04-25T00:04:08.728681+0000","last_clean":"2024-04-25T00:04:08.728681+0000","last_became_active":"2024-04-25T00:04:06.954406+0000","last_became_peered":"2024-04-25T00:04:06.954406+0000","last_unstale":"2024-04-25T00:04:08.728681+0000","last_undegraded":"2024-04-25T00:04:08.728681+0000","last_fullsized":"2024-04-25T00:04:08.728681+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:40.872518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081857+0000","last_change":"2024-04-25T00:04:06.964118+0000","last_active":"2024-04-25T00:04:08.081857+0000","last_peered":"2024-04-25T00:04:08.081857+0000","last_clean":"2024-04-25T00:04:08.081857+0000","last_became_active":"2024-04-25T00:04:06.963878+0000","last_became_peered":"2024-04-25T00:04:06.963878+0000","last_unstale":"2024-04-25T00:04:08.081857+0000","last_undegraded":"2024-04-25T00:04:08.081857+0000","last_fullsized":"2024-04-25T00:04:08.081857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:24.348997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942845+0000","last_change":"2024-04-25T00:04:06.954948+0000","last_active":"2024-04-25T00:04:07.942845+0000","last_peered":"2024-04-25T00:04:07.942845+0000","last_clean":"2024-04-25T00:04:07.942845+0000","last_became_active":"2024-04-25T00:04:06.954653+0000","last_became_peered":"2024-04-25T00:04:06.954653+0000","last_unstale":"2024-04-25T00:04:07.942845+0000","last_undegraded":"2024-04-25T00:04:07.942845+0000","last_fullsized":"2024-04-25T00:04:07.942845+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:06.870260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729141+0000","last_change":"2024-04-25T00:04:06.955683+0000","last_active":"2024-04-25T00:04:08.729141+0000","last_peered":"2024-04-25T00:04:08.729141+0000","last_clean":"2024-04-25T00:04:08.729141+0000","last_became_active":"2024-04-25T00:04:06.955354+0000","last_became_peered":"2024-04-25T00:04:06.955354+0000","last_unstale":"2024-04-25T00:04:08.729141+0000","last_undegraded":"2024-04-25T00:04:08.729141+0000","last_fullsized":"2024-04-25T00:04:08.729141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:03.189346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730193+0000","last_change":"2024-04-25T00:04:06.953751+0000","last_active":"2024-04-25T00:04:08.730193+0000","last_peered":"2024-04-25T00:04:08.730193+0000","last_clean":"2024-04-25T00:04:08.730193+0000","last_became_active":"2024-04-25T00:04:06.953474+0000","last_became_peered":"2024-04-25T00:04:06.953474+0000","last_unstale":"2024-04-25T00:04:08.730193+0000","last_undegraded":"2024-04-25T00:04:08.730193+0000","last_fullsized":"2024-04-25T00:04:08.730193+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:46.843144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082095+0000","last_change":"2024-04-25T00:04:06.958480+0000","last_active":"2024-04-25T00:04:08.082095+0000","last_peered":"2024-04-25T00:04:08.082095+0000","last_clean":"2024-04-25T00:04:08.082095+0000","last_became_active":"2024-04-25T00:04:06.956972+0000","last_became_peered":"2024-04-25T00:04:06.956972+0000","last_unstale":"2024-04-25T00:04:08.082095+0000","last_undegraded":"2024-04-25T00:04:08.082095+0000","last_fullsized":"2024-04-25T00:04:08.082095+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:57.845258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081781+0000","last_change":"2024-04-25T00:04:06.959665+0000","last_active":"2024-04-25T00:04:08.081781+0000","last_peered":"2024-04-25T00:04:08.081781+0000","last_clean":"2024-04-25T00:04:08.081781+0000","last_became_active":"2024-04-25T00:04:06.959404+0000","last_became_peered":"2024-04-25T00:04:06.959404+0000","last_unstale":"2024-04-25T00:04:08.081781+0000","last_undegraded":"2024-04-25T00:04:08.081781+0000","last_fullsized":"2024-04-25T00:04:08.081781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:45.308175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728876+0000","last_change":"2024-04-25T00:04:06.964438+0000","last_active":"2024-04-25T00:04:08.728876+0000","last_peered":"2024-04-25T00:04:08.728876+0000","last_clean":"2024-04-25T00:04:08.728876+0000","last_became_active":"2024-04-25T00:04:06.964110+0000","last_became_peered":"2024-04-25T00:04:06.964110+0000","last_unstale":"2024-04-25T00:04:08.728876+0000","last_undegraded":"2024-04-25T00:04:08.728876+0000","last_fullsized":"2024-04-25T00:04:08.728876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:03.695662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729571+0000","last_change":"2024-04-25T00:04:06.962878+0000","last_active":"2024-04-25T00:04:08.729571+0000","last_peered":"2024-04-25T00:04:08.729571+0000","last_clean":"2024-04-25T00:04:08.729571+0000","last_became_active":"2024-04-25T00:04:06.962600+0000","last_became_peered":"2024-04-25T00:04:06.962600+0000","last_unstale":"2024-04-25T00:04:08.729571+0000","last_undegraded":"2024-04-25T00:04:08.729571+0000","last_fullsized":"2024-04-25T00:04:08.729571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.810929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729922+0000","last_change":"2024-04-25T00:04:06.960444+0000","last_active":"2024-04-25T00:04:08.729922+0000","last_peered":"2024-04-25T00:04:08.729922+0000","last_clean":"2024-04-25T00:04:08.729922+0000","last_became_active":"2024-04-25T00:04:06.959875+0000","last_became_peered":"2024-04-25T00:04:06.959875+0000","last_unstale":"2024-04-25T00:04:08.729922+0000","last_undegraded":"2024-04-25T00:04:08.729922+0000","last_fullsized":"2024-04-25T00:04:08.729922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:16.203238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730609+0000","last_change":"2024-04-25T00:04:05.949584+0000","last_active":"2024-04-25T00:04:08.730609+0000","last_peered":"2024-04-25T00:04:08.730609+0000","last_clean":"2024-04-25T00:04:08.730609+0000","last_became_active":"2024-04-25T00:04:05.948893+0000","last_became_peered":"2024-04-25T00:04:05.948893+0000","last_unstale":"2024-04-25T00:04:08.730609+0000","last_undegraded":"2024-04-25T00:04:08.730609+0000","last_fullsized":"2024-04-25T00:04:08.730609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:05.513334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730605+0000","last_change":"2024-04-25T00:04:06.952292+0000","last_active":"2024-04-25T00:04:08.730605+0000","last_peered":"2024-04-25T00:04:08.730605+0000","last_clean":"2024-04-25T00:04:08.730605+0000","last_became_active":"2024-04-25T00:04:06.950108+0000","last_became_peered":"2024-04-25T00:04:06.950108+0000","last_unstale":"2024-04-25T00:04:08.730605+0000","last_undegraded":"2024-04-25T00:04:08.730605+0000","last_fullsized":"2024-04-25T00:04:08.730605+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:03.695662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730032+0000","last_change":"2024-04-25T00:04:05.943990+0000","last_active":"2024-04-25T00:04:08.730032+0000","last_peered":"2024-04-25T00:04:08.730032+0000","last_clean":"2024-04-25T00:04:08.730032+0000","last_became_active":"2024-04-25T00:04:05.943718+0000","last_became_peered":"2024-04-25T00:04:05.943718+0000","last_unstale":"2024-04-25T00:04:08.730032+0000","last_undegraded":"2024-04-25T00:04:08.730032+0000","last_fullsized":"2024-04-25T00:04:08.730032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:37.383495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730389+0000","last_change":"2024-04-25T00:04:06.962771+0000","last_active":"2024-04-25T00:04:08.730389+0000","last_peered":"2024-04-25T00:04:08.730389+0000","last_clean":"2024-04-25T00:04:08.730389+0000","last_became_active":"2024-04-25T00:04:06.962517+0000","last_became_peered":"2024-04-25T00:04:06.962517+0000","last_unstale":"2024-04-25T00:04:08.730389+0000","last_undegraded":"2024-04-25T00:04:08.730389+0000","last_fullsized":"2024-04-25T00:04:08.730389+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:33.559156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942663+0000","last_change":"2024-04-25T00:04:05.954463+0000","last_active":"2024-04-25T00:04:07.942663+0000","last_peered":"2024-04-25T00:04:07.942663+0000","last_clean":"2024-04-25T00:04:07.942663+0000","last_became_active":"2024-04-25T00:04:05.954215+0000","last_became_peered":"2024-04-25T00:04:05.954215+0000","last_unstale":"2024-04-25T00:04:07.942663+0000","last_undegraded":"2024-04-25T00:04:07.942663+0000","last_fullsized":"2024-04-25T00:04:07.942663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:45.838433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942830+0000","last_change":"2024-04-25T00:04:06.970413+0000","last_active":"2024-04-25T00:04:07.942830+0000","last_peered":"2024-04-25T00:04:07.942830+0000","last_clean":"2024-04-25T00:04:07.942830+0000","last_became_active":"2024-04-25T00:04:06.970129+0000","last_became_peered":"2024-04-25T00:04:06.970129+0000","last_unstale":"2024-04-25T00:04:07.942830+0000","last_undegraded":"2024-04-25T00:04:07.942830+0000","last_fullsized":"2024-04-25T00:04:07.942830+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:50.019061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942683+0000","last_change":"2024-04-25T00:04:05.948415+0000","last_active":"2024-04-25T00:04:07.942683+0000","last_peered":"2024-04-25T00:04:07.942683+0000","last_clean":"2024-04-25T00:04:07.942683+0000","last_became_active":"2024-04-25T00:04:05.948148+0000","last_became_peered":"2024-04-25T00:04:05.948148+0000","last_unstale":"2024-04-25T00:04:07.942683+0000","last_undegraded":"2024-04-25T00:04:07.942683+0000","last_fullsized":"2024-04-25T00:04:07.942683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:19.803498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081881+0000","last_change":"2024-04-25T00:04:06.959495+0000","last_active":"2024-04-25T00:04:08.081881+0000","last_peered":"2024-04-25T00:04:08.081881+0000","last_clean":"2024-04-25T00:04:08.081881+0000","last_became_active":"2024-04-25T00:04:06.959226+0000","last_became_peered":"2024-04-25T00:04:06.959226+0000","last_unstale":"2024-04-25T00:04:08.081881+0000","last_undegraded":"2024-04-25T00:04:08.081881+0000","last_fullsized":"2024-04-25T00:04:08.081881+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:31.537173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729879+0000","last_change":"2024-04-25T00:04:05.947803+0000","last_active":"2024-04-25T00:04:08.729879+0000","last_peered":"2024-04-25T00:04:08.729879+0000","last_clean":"2024-04-25T00:04:08.729879+0000","last_became_active":"2024-04-25T00:04:05.947378+0000","last_became_peered":"2024-04-25T00:04:05.947378+0000","last_unstale":"2024-04-25T00:04:08.729879+0000","last_undegraded":"2024-04-25T00:04:08.729879+0000","last_fullsized":"2024-04-25T00:04:08.729879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:44.139999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729682+0000","last_change":"2024-04-25T00:04:06.966781+0000","last_active":"2024-04-25T00:04:08.729682+0000","last_peered":"2024-04-25T00:04:08.729682+0000","last_clean":"2024-04-25T00:04:08.729682+0000","last_became_active":"2024-04-25T00:04:06.966507+0000","last_became_peered":"2024-04-25T00:04:06.966507+0000","last_unstale":"2024-04-25T00:04:08.729682+0000","last_undegraded":"2024-04-25T00:04:08.729682+0000","last_fullsized":"2024-04-25T00:04:08.729682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:57.785779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728966+0000","last_change":"2024-04-25T00:04:05.944427+0000","last_active":"2024-04-25T00:04:08.728966+0000","last_peered":"2024-04-25T00:04:08.728966+0000","last_clean":"2024-04-25T00:04:08.728966+0000","last_became_active":"2024-04-25T00:04:05.944008+0000","last_became_peered":"2024-04-25T00:04:05.944008+0000","last_unstale":"2024-04-25T00:04:08.728966+0000","last_undegraded":"2024-04-25T00:04:08.728966+0000","last_fullsized":"2024-04-25T00:04:08.728966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:57.489583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942411+0000","last_change":"2024-04-25T00:04:06.956402+0000","last_active":"2024-04-25T00:04:07.942411+0000","last_peered":"2024-04-25T00:04:07.942411+0000","last_clean":"2024-04-25T00:04:07.942411+0000","last_became_active":"2024-04-25T00:04:06.956097+0000","last_became_peered":"2024-04-25T00:04:06.956097+0000","last_unstale":"2024-04-25T00:04:07.942411+0000","last_undegraded":"2024-04-25T00:04:07.942411+0000","last_fullsized":"2024-04-25T00:04:07.942411+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:35.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942240+0000","last_change":"2024-04-25T00:04:05.955068+0000","last_active":"2024-04-25T00:04:07.942240+0000","last_peered":"2024-04-25T00:04:07.942240+0000","last_clean":"2024-04-25T00:04:07.942240+0000","last_became_active":"2024-04-25T00:04:05.954821+0000","last_became_peered":"2024-04-25T00:04:05.954821+0000","last_unstale":"2024-04-25T00:04:07.942240+0000","last_undegraded":"2024-04-25T00:04:07.942240+0000","last_fullsized":"2024-04-25T00:04:07.942240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:21.778025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941439+0000","last_change":"2024-04-25T00:04:06.954403+0000","last_active":"2024-04-25T00:04:07.941439+0000","last_peered":"2024-04-25T00:04:07.941439+0000","last_clean":"2024-04-25T00:04:07.941439+0000","last_became_active":"2024-04-25T00:04:06.954129+0000","last_became_peered":"2024-04-25T00:04:06.954129+0000","last_unstale":"2024-04-25T00:04:07.941439+0000","last_undegraded":"2024-04-25T00:04:07.941439+0000","last_fullsized":"2024-04-25T00:04:07.941439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.916084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081748+0000","last_change":"2024-04-25T00:04:05.951451+0000","last_active":"2024-04-25T00:04:08.081748+0000","last_peered":"2024-04-25T00:04:08.081748+0000","last_clean":"2024-04-25T00:04:08.081748+0000","last_became_active":"2024-04-25T00:04:05.951147+0000","last_became_peered":"2024-04-25T00:04:05.951147+0000","last_unstale":"2024-04-25T00:04:08.081748+0000","last_undegraded":"2024-04-25T00:04:08.081748+0000","last_fullsized":"2024-04-25T00:04:08.081748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:48.350400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941578+0000","last_change":"2024-04-25T00:04:06.954596+0000","last_active":"2024-04-25T00:04:07.941578+0000","last_peered":"2024-04-25T00:04:07.941578+0000","last_clean":"2024-04-25T00:04:07.941578+0000","last_became_active":"2024-04-25T00:04:06.954331+0000","last_became_peered":"2024-04-25T00:04:06.954331+0000","last_unstale":"2024-04-25T00:04:07.941578+0000","last_undegraded":"2024-04-25T00:04:07.941578+0000","last_fullsized":"2024-04-25T00:04:07.941578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:54.686746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730713+0000","last_change":"2024-04-25T00:04:05.949805+0000","last_active":"2024-04-25T00:04:08.730713+0000","last_peered":"2024-04-25T00:04:08.730713+0000","last_clean":"2024-04-25T00:04:08.730713+0000","last_became_active":"2024-04-25T00:04:05.949244+0000","last_became_peered":"2024-04-25T00:04:05.949244+0000","last_unstale":"2024-04-25T00:04:08.730713+0000","last_undegraded":"2024-04-25T00:04:08.730713+0000","last_fullsized":"2024-04-25T00:04:08.730713+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.672942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729132+0000","last_change":"2024-04-25T00:04:06.965478+0000","last_active":"2024-04-25T00:04:08.729132+0000","last_peered":"2024-04-25T00:04:08.729132+0000","last_clean":"2024-04-25T00:04:08.729132+0000","last_became_active":"2024-04-25T00:04:06.965210+0000","last_became_peered":"2024-04-25T00:04:06.965210+0000","last_unstale":"2024-04-25T00:04:08.729132+0000","last_undegraded":"2024-04-25T00:04:08.729132+0000","last_fullsized":"2024-04-25T00:04:08.729132+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:56.312120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730469+0000","last_change":"2024-04-25T00:04:05.952083+0000","last_active":"2024-04-25T00:04:08.730469+0000","last_peered":"2024-04-25T00:04:08.730469+0000","last_clean":"2024-04-25T00:04:08.730469+0000","last_became_active":"2024-04-25T00:04:05.951758+0000","last_became_peered":"2024-04-25T00:04:05.951758+0000","last_unstale":"2024-04-25T00:04:08.730469+0000","last_undegraded":"2024-04-25T00:04:08.730469+0000","last_fullsized":"2024-04-25T00:04:08.730469+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:46.929367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942601+0000","last_change":"2024-04-25T00:04:06.944843+0000","last_active":"2024-04-25T00:04:07.942601+0000","last_peered":"2024-04-25T00:04:07.942601+0000","last_clean":"2024-04-25T00:04:07.942601+0000","last_became_active":"2024-04-25T00:04:06.944602+0000","last_became_peered":"2024-04-25T00:04:06.944602+0000","last_unstale":"2024-04-25T00:04:07.942601+0000","last_undegraded":"2024-04-25T00:04:07.942601+0000","last_fullsized":"2024-04-25T00:04:07.942601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:45.698203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730029+0000","last_change":"2024-04-25T00:04:05.951264+0000","last_active":"2024-04-25T00:04:08.730029+0000","last_peered":"2024-04-25T00:04:08.730029+0000","last_clean":"2024-04-25T00:04:08.730029+0000","last_became_active":"2024-04-25T00:04:05.949917+0000","last_became_peered":"2024-04-25T00:04:05.949917+0000","last_unstale":"2024-04-25T00:04:08.730029+0000","last_undegraded":"2024-04-25T00:04:08.730029+0000","last_fullsized":"2024-04-25T00:04:08.730029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:40.495984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729299+0000","last_change":"2024-04-25T00:04:06.964012+0000","last_active":"2024-04-25T00:04:08.729299+0000","last_peered":"2024-04-25T00:04:08.729299+0000","last_clean":"2024-04-25T00:04:08.729299+0000","last_became_active":"2024-04-25T00:04:06.963702+0000","last_became_peered":"2024-04-25T00:04:06.963702+0000","last_unstale":"2024-04-25T00:04:08.729299+0000","last_undegraded":"2024-04-25T00:04:08.729299+0000","last_fullsized":"2024-04-25T00:04:08.729299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:32.180163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729358+0000","last_change":"2024-04-25T00:04:05.951475+0000","last_active":"2024-04-25T00:04:08.729358+0000","last_peered":"2024-04-25T00:04:08.729358+0000","last_clean":"2024-04-25T00:04:08.729358+0000","last_became_active":"2024-04-25T00:04:05.951195+0000","last_became_peered":"2024-04-25T00:04:05.951195+0000","last_unstale":"2024-04-25T00:04:08.729358+0000","last_undegraded":"2024-04-25T00:04:08.729358+0000","last_fullsized":"2024-04-25T00:04:08.729358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:45.838433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942348+0000","last_change":"2024-04-25T00:04:06.958086+0000","last_active":"2024-04-25T00:04:07.942348+0000","last_peered":"2024-04-25T00:04:07.942348+0000","last_clean":"2024-04-25T00:04:07.942348+0000","last_became_active":"2024-04-25T00:04:06.957793+0000","last_became_peered":"2024-04-25T00:04:06.957793+0000","last_unstale":"2024-04-25T00:04:07.942348+0000","last_undegraded":"2024-04-25T00:04:07.942348+0000","last_fullsized":"2024-04-25T00:04:07.942348+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:03.189346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082069+0000","last_change":"2024-04-25T00:04:05.949976+0000","last_active":"2024-04-25T00:04:08.082069+0000","last_peered":"2024-04-25T00:04:08.082069+0000","last_clean":"2024-04-25T00:04:08.082069+0000","last_became_active":"2024-04-25T00:04:05.948413+0000","last_became_peered":"2024-04-25T00:04:05.948413+0000","last_unstale":"2024-04-25T00:04:08.082069+0000","last_undegraded":"2024-04-25T00:04:08.082069+0000","last_fullsized":"2024-04-25T00:04:08.082069+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:18.406599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942591+0000","last_change":"2024-04-25T00:04:06.967827+0000","last_active":"2024-04-25T00:04:07.942591+0000","last_peered":"2024-04-25T00:04:07.942591+0000","last_clean":"2024-04-25T00:04:07.942591+0000","last_became_active":"2024-04-25T00:04:06.967488+0000","last_became_peered":"2024-04-25T00:04:06.967488+0000","last_unstale":"2024-04-25T00:04:07.942591+0000","last_undegraded":"2024-04-25T00:04:07.942591+0000","last_fullsized":"2024-04-25T00:04:07.942591+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:54.686746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941490+0000","last_change":"2024-04-25T00:04:05.946979+0000","last_active":"2024-04-25T00:04:07.941490+0000","last_peered":"2024-04-25T00:04:07.941490+0000","last_clean":"2024-04-25T00:04:07.941490+0000","last_became_active":"2024-04-25T00:04:05.946660+0000","last_became_peered":"2024-04-25T00:04:05.946660+0000","last_unstale":"2024-04-25T00:04:07.941490+0000","last_undegraded":"2024-04-25T00:04:07.941490+0000","last_fullsized":"2024-04-25T00:04:07.941490+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:39.155362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081642+0000","last_change":"2024-04-25T00:04:06.970858+0000","last_active":"2024-04-25T00:04:08.081642+0000","last_peered":"2024-04-25T00:04:08.081642+0000","last_clean":"2024-04-25T00:04:08.081642+0000","last_became_active":"2024-04-25T00:04:06.970568+0000","last_became_peered":"2024-04-25T00:04:06.970568+0000","last_unstale":"2024-04-25T00:04:08.081642+0000","last_undegraded":"2024-04-25T00:04:08.081642+0000","last_fullsized":"2024-04-25T00:04:08.081642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:57.548535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728906+0000","last_change":"2024-04-25T00:04:05.953311+0000","last_active":"2024-04-25T00:04:08.728906+0000","last_peered":"2024-04-25T00:04:08.728906+0000","last_clean":"2024-04-25T00:04:08.728906+0000","last_became_active":"2024-04-25T00:04:05.953035+0000","last_became_peered":"2024-04-25T00:04:05.953035+0000","last_unstale":"2024-04-25T00:04:08.728906+0000","last_undegraded":"2024-04-25T00:04:08.728906+0000","last_fullsized":"2024-04-25T00:04:08.728906+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:14.724496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081649+0000","last_change":"2024-04-25T00:04:06.963511+0000","last_active":"2024-04-25T00:04:08.081649+0000","last_peered":"2024-04-25T00:04:08.081649+0000","last_clean":"2024-04-25T00:04:08.081649+0000","last_became_active":"2024-04-25T00:04:06.963262+0000","last_became_peered":"2024-04-25T00:04:06.963262+0000","last_unstale":"2024-04-25T00:04:08.081649+0000","last_undegraded":"2024-04-25T00:04:08.081649+0000","last_fullsized":"2024-04-25T00:04:08.081649+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:41.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942196+0000","last_change":"2024-04-25T00:04:05.953176+0000","last_active":"2024-04-25T00:04:07.942196+0000","last_peered":"2024-04-25T00:04:07.942196+0000","last_clean":"2024-04-25T00:04:07.942196+0000","last_became_active":"2024-04-25T00:04:05.952925+0000","last_became_peered":"2024-04-25T00:04:05.952925+0000","last_unstale":"2024-04-25T00:04:07.942196+0000","last_undegraded":"2024-04-25T00:04:07.942196+0000","last_fullsized":"2024-04-25T00:04:07.942196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.806218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942890+0000","last_change":"2024-04-25T00:04:06.953792+0000","last_active":"2024-04-25T00:04:07.942890+0000","last_peered":"2024-04-25T00:04:07.942890+0000","last_clean":"2024-04-25T00:04:07.942890+0000","last_became_active":"2024-04-25T00:04:06.953524+0000","last_became_peered":"2024-04-25T00:04:06.953524+0000","last_unstale":"2024-04-25T00:04:07.942890+0000","last_undegraded":"2024-04-25T00:04:07.942890+0000","last_fullsized":"2024-04-25T00:04:07.942890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:48.566137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730787+0000","last_change":"2024-04-25T00:04:05.952219+0000","last_active":"2024-04-25T00:04:08.730787+0000","last_peered":"2024-04-25T00:04:08.730787+0000","last_clean":"2024-04-25T00:04:08.730787+0000","last_became_active":"2024-04-25T00:04:05.951940+0000","last_became_peered":"2024-04-25T00:04:05.951940+0000","last_unstale":"2024-04-25T00:04:08.730787+0000","last_undegraded":"2024-04-25T00:04:08.730787+0000","last_fullsized":"2024-04-25T00:04:08.730787+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:23.344286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730497+0000","last_change":"2024-04-25T00:04:06.952571+0000","last_active":"2024-04-25T00:04:08.730497+0000","last_peered":"2024-04-25T00:04:08.730497+0000","last_clean":"2024-04-25T00:04:08.730497+0000","last_became_active":"2024-04-25T00:04:06.951778+0000","last_became_peered":"2024-04-25T00:04:06.951778+0000","last_unstale":"2024-04-25T00:04:08.730497+0000","last_undegraded":"2024-04-25T00:04:08.730497+0000","last_fullsized":"2024-04-25T00:04:08.730497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:11.572368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942540+0000","last_change":"2024-04-25T00:04:05.945666+0000","last_active":"2024-04-25T00:04:07.942540+0000","last_peered":"2024-04-25T00:04:07.942540+0000","last_clean":"2024-04-25T00:04:07.942540+0000","last_became_active":"2024-04-25T00:04:05.945410+0000","last_became_peered":"2024-04-25T00:04:05.945410+0000","last_unstale":"2024-04-25T00:04:07.942540+0000","last_undegraded":"2024-04-25T00:04:07.942540+0000","last_fullsized":"2024-04-25T00:04:07.942540+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:42.383041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729470+0000","last_change":"2024-04-25T00:04:06.960071+0000","last_active":"2024-04-25T00:04:08.729470+0000","last_peered":"2024-04-25T00:04:08.729470+0000","last_clean":"2024-04-25T00:04:08.729470+0000","last_became_active":"2024-04-25T00:04:06.959553+0000","last_became_peered":"2024-04-25T00:04:06.959553+0000","last_unstale":"2024-04-25T00:04:08.729470+0000","last_undegraded":"2024-04-25T00:04:08.729470+0000","last_fullsized":"2024-04-25T00:04:08.729470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:45.698203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942737+0000","last_change":"2024-04-25T00:04:05.957759+0000","last_active":"2024-04-25T00:04:07.942737+0000","last_peered":"2024-04-25T00:04:07.942737+0000","last_clean":"2024-04-25T00:04:07.942737+0000","last_became_active":"2024-04-25T00:04:05.957505+0000","last_became_peered":"2024-04-25T00:04:05.957505+0000","last_unstale":"2024-04-25T00:04:07.942737+0000","last_undegraded":"2024-04-25T00:04:07.942737+0000","last_fullsized":"2024-04-25T00:04:07.942737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:32.554445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728942+0000","last_change":"2024-04-25T00:04:06.968219+0000","last_active":"2024-04-25T00:04:08.728942+0000","last_peered":"2024-04-25T00:04:08.728942+0000","last_clean":"2024-04-25T00:04:08.728942+0000","last_became_active":"2024-04-25T00:04:06.966241+0000","last_became_peered":"2024-04-25T00:04:06.966241+0000","last_unstale":"2024-04-25T00:04:08.728942+0000","last_undegraded":"2024-04-25T00:04:08.728942+0000","last_fullsized":"2024-04-25T00:04:08.728942+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:43.352015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941705+0000","last_change":"2024-04-25T00:04:05.958193+0000","last_active":"2024-04-25T00:04:07.941705+0000","last_peered":"2024-04-25T00:04:07.941705+0000","last_clean":"2024-04-25T00:04:07.941705+0000","last_became_active":"2024-04-25T00:04:05.957902+0000","last_became_peered":"2024-04-25T00:04:05.957902+0000","last_unstale":"2024-04-25T00:04:07.941705+0000","last_undegraded":"2024-04-25T00:04:07.941705+0000","last_fullsized":"2024-04-25T00:04:07.941705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:50.454685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728776+0000","last_change":"2024-04-25T00:04:06.957603+0000","last_active":"2024-04-25T00:04:08.728776+0000","last_peered":"2024-04-25T00:04:08.728776+0000","last_clean":"2024-04-25T00:04:08.728776+0000","last_became_active":"2024-04-25T00:04:06.957293+0000","last_became_peered":"2024-04-25T00:04:06.957293+0000","last_unstale":"2024-04-25T00:04:08.728776+0000","last_undegraded":"2024-04-25T00:04:08.728776+0000","last_fullsized":"2024-04-25T00:04:08.728776+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:42.565984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729050+0000","last_change":"2024-04-25T00:04:05.956393+0000","last_active":"2024-04-25T00:04:08.729050+0000","last_peered":"2024-04-25T00:04:08.729050+0000","last_clean":"2024-04-25T00:04:08.729050+0000","last_became_active":"2024-04-25T00:04:05.955523+0000","last_became_peered":"2024-04-25T00:04:05.955523+0000","last_unstale":"2024-04-25T00:04:08.729050+0000","last_undegraded":"2024-04-25T00:04:08.729050+0000","last_fullsized":"2024-04-25T00:04:08.729050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:35.028683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728768+0000","last_change":"2024-04-25T00:04:06.960013+0000","last_active":"2024-04-25T00:04:08.728768+0000","last_peered":"2024-04-25T00:04:08.728768+0000","last_clean":"2024-04-25T00:04:08.728768+0000","last_became_active":"2024-04-25T00:04:06.957037+0000","last_became_peered":"2024-04-25T00:04:06.957037+0000","last_unstale":"2024-04-25T00:04:08.728768+0000","last_undegraded":"2024-04-25T00:04:08.728768+0000","last_fullsized":"2024-04-25T00:04:08.728768+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:56.473226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728743+0000","last_change":"2024-04-25T00:04:05.955280+0000","last_active":"2024-04-25T00:04:08.728743+0000","last_peered":"2024-04-25T00:04:08.728743+0000","last_clean":"2024-04-25T00:04:08.728743+0000","last_became_active":"2024-04-25T00:04:05.955021+0000","last_became_peered":"2024-04-25T00:04:05.955021+0000","last_unstale":"2024-04-25T00:04:08.728743+0000","last_undegraded":"2024-04-25T00:04:08.728743+0000","last_fullsized":"2024-04-25T00:04:08.728743+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:56.254060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729251+0000","last_change":"2024-04-25T00:04:06.953963+0000","last_active":"2024-04-25T00:04:08.729251+0000","last_peered":"2024-04-25T00:04:08.729251+0000","last_clean":"2024-04-25T00:04:08.729251+0000","last_became_active":"2024-04-25T00:04:06.953697+0000","last_became_peered":"2024-04-25T00:04:06.953697+0000","last_unstale":"2024-04-25T00:04:08.729251+0000","last_undegraded":"2024-04-25T00:04:08.729251+0000","last_fullsized":"2024-04-25T00:04:08.729251+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:16.203238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.941358+0000","last_change":"2024-04-25T00:04:05.955799+0000","last_active":"2024-04-25T00:04:07.941358+0000","last_peered":"2024-04-25T00:04:07.941358+0000","last_clean":"2024-04-25T00:04:07.941358+0000","last_became_active":"2024-04-25T00:04:05.955566+0000","last_became_peered":"2024-04-25T00:04:05.955566+0000","last_unstale":"2024-04-25T00:04:07.941358+0000","last_undegraded":"2024-04-25T00:04:07.941358+0000","last_fullsized":"2024-04-25T00:04:07.941358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:37.193721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729985+0000","last_change":"2024-04-25T00:04:06.955956+0000","last_active":"2024-04-25T00:04:08.729985+0000","last_peered":"2024-04-25T00:04:08.729985+0000","last_clean":"2024-04-25T00:04:08.729985+0000","last_became_active":"2024-04-25T00:04:06.955663+0000","last_became_peered":"2024-04-25T00:04:06.955663+0000","last_unstale":"2024-04-25T00:04:08.729985+0000","last_undegraded":"2024-04-25T00:04:08.729985+0000","last_fullsized":"2024-04-25T00:04:08.729985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:38.198432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081907+0000","last_change":"2024-04-25T00:04:05.953036+0000","last_active":"2024-04-25T00:04:08.081907+0000","last_peered":"2024-04-25T00:04:08.081907+0000","last_clean":"2024-04-25T00:04:08.081907+0000","last_became_active":"2024-04-25T00:04:05.952742+0000","last_became_peered":"2024-04-25T00:04:05.952742+0000","last_unstale":"2024-04-25T00:04:08.081907+0000","last_undegraded":"2024-04-25T00:04:08.081907+0000","last_fullsized":"2024-04-25T00:04:08.081907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:51.577243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942904+0000","last_change":"2024-04-25T00:04:06.962964+0000","last_active":"2024-04-25T00:04:07.942904+0000","last_peered":"2024-04-25T00:04:07.942904+0000","last_clean":"2024-04-25T00:04:07.942904+0000","last_became_active":"2024-04-25T00:04:06.962676+0000","last_became_peered":"2024-04-25T00:04:06.962676+0000","last_unstale":"2024-04-25T00:04:07.942904+0000","last_undegraded":"2024-04-25T00:04:07.942904+0000","last_fullsized":"2024-04-25T00:04:07.942904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:48.286783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.082115+0000","last_change":"2024-04-25T00:04:05.950505+0000","last_active":"2024-04-25T00:04:08.082115+0000","last_peered":"2024-04-25T00:04:08.082115+0000","last_clean":"2024-04-25T00:04:08.082115+0000","last_became_active":"2024-04-25T00:04:05.948719+0000","last_became_peered":"2024-04-25T00:04:05.948719+0000","last_unstale":"2024-04-25T00:04:08.082115+0000","last_undegraded":"2024-04-25T00:04:08.082115+0000","last_fullsized":"2024-04-25T00:04:08.082115+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:59.714801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730385+0000","last_change":"2024-04-25T00:04:06.962187+0000","last_active":"2024-04-25T00:04:08.730385+0000","last_peered":"2024-04-25T00:04:08.730385+0000","last_clean":"2024-04-25T00:04:08.730385+0000","last_became_active":"2024-04-25T00:04:06.961735+0000","last_became_peered":"2024-04-25T00:04:06.961735+0000","last_unstale":"2024-04-25T00:04:08.730385+0000","last_undegraded":"2024-04-25T00:04:08.730385+0000","last_fullsized":"2024-04-25T00:04:08.730385+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:20.808209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730227+0000","last_change":"2024-04-25T00:04:05.949348+0000","last_active":"2024-04-25T00:04:08.730227+0000","last_peered":"2024-04-25T00:04:08.730227+0000","last_clean":"2024-04-25T00:04:08.730227+0000","last_became_active":"2024-04-25T00:04:05.948760+0000","last_became_peered":"2024-04-25T00:04:05.948760+0000","last_unstale":"2024-04-25T00:04:08.730227+0000","last_undegraded":"2024-04-25T00:04:08.730227+0000","last_fullsized":"2024-04-25T00:04:08.730227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:44.303465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729644+0000","last_change":"2024-04-25T00:04:06.962874+0000","last_active":"2024-04-25T00:04:08.729644+0000","last_peered":"2024-04-25T00:04:08.729644+0000","last_clean":"2024-04-25T00:04:08.729644+0000","last_became_active":"2024-04-25T00:04:06.962602+0000","last_became_peered":"2024-04-25T00:04:06.962602+0000","last_unstale":"2024-04-25T00:04:08.729644+0000","last_undegraded":"2024-04-25T00:04:08.729644+0000","last_fullsized":"2024-04-25T00:04:08.729644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:42.565984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730515+0000","last_change":"2024-04-25T00:04:05.953709+0000","last_active":"2024-04-25T00:04:08.730515+0000","last_peered":"2024-04-25T00:04:08.730515+0000","last_clean":"2024-04-25T00:04:08.730515+0000","last_became_active":"2024-04-25T00:04:05.953453+0000","last_became_peered":"2024-04-25T00:04:05.953453+0000","last_unstale":"2024-04-25T00:04:08.730515+0000","last_undegraded":"2024-04-25T00:04:08.730515+0000","last_fullsized":"2024-04-25T00:04:08.730515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:30.532462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729295+0000","last_change":"2024-04-25T00:04:06.957488+0000","last_active":"2024-04-25T00:04:08.729295+0000","last_peered":"2024-04-25T00:04:08.729295+0000","last_clean":"2024-04-25T00:04:08.729295+0000","last_became_active":"2024-04-25T00:04:06.957191+0000","last_became_peered":"2024-04-25T00:04:06.957191+0000","last_unstale":"2024-04-25T00:04:08.729295+0000","last_undegraded":"2024-04-25T00:04:08.729295+0000","last_fullsized":"2024-04-25T00:04:08.729295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:38.198432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942441+0000","last_change":"2024-04-25T00:04:05.953267+0000","last_active":"2024-04-25T00:04:07.942441+0000","last_peered":"2024-04-25T00:04:07.942441+0000","last_clean":"2024-04-25T00:04:07.942441+0000","last_became_active":"2024-04-25T00:04:05.953021+0000","last_became_peered":"2024-04-25T00:04:05.953021+0000","last_unstale":"2024-04-25T00:04:07.942441+0000","last_undegraded":"2024-04-25T00:04:07.942441+0000","last_fullsized":"2024-04-25T00:04:07.942441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:14.724496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942463+0000","last_change":"2024-04-25T00:04:06.961589+0000","last_active":"2024-04-25T00:04:07.942463+0000","last_peered":"2024-04-25T00:04:07.942463+0000","last_clean":"2024-04-25T00:04:07.942463+0000","last_became_active":"2024-04-25T00:04:06.961267+0000","last_became_peered":"2024-04-25T00:04:06.961267+0000","last_unstale":"2024-04-25T00:04:07.942463+0000","last_undegraded":"2024-04-25T00:04:07.942463+0000","last_fullsized":"2024-04-25T00:04:07.942463+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:32.180163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.729078+0000","last_change":"2024-04-25T00:04:05.944307+0000","last_active":"2024-04-25T00:04:08.729078+0000","last_peered":"2024-04-25T00:04:08.729078+0000","last_clean":"2024-04-25T00:04:08.729078+0000","last_became_active":"2024-04-25T00:04:05.943618+0000","last_became_peered":"2024-04-25T00:04:05.943618+0000","last_unstale":"2024-04-25T00:04:08.729078+0000","last_undegraded":"2024-04-25T00:04:08.729078+0000","last_fullsized":"2024-04-25T00:04:08.729078+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:42.595520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942451+0000","last_change":"2024-04-25T00:04:06.958042+0000","last_active":"2024-04-25T00:04:07.942451+0000","last_peered":"2024-04-25T00:04:07.942451+0000","last_clean":"2024-04-25T00:04:07.942451+0000","last_became_active":"2024-04-25T00:04:06.957721+0000","last_became_peered":"2024-04-25T00:04:06.957721+0000","last_unstale":"2024-04-25T00:04:07.942451+0000","last_undegraded":"2024-04-25T00:04:07.942451+0000","last_fullsized":"2024-04-25T00:04:07.942451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:11.936001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730138+0000","last_change":"2024-04-25T00:04:05.949003+0000","last_active":"2024-04-25T00:04:08.730138+0000","last_peered":"2024-04-25T00:04:08.730138+0000","last_clean":"2024-04-25T00:04:08.730138+0000","last_became_active":"2024-04-25T00:04:05.948731+0000","last_became_peered":"2024-04-25T00:04:05.948731+0000","last_unstale":"2024-04-25T00:04:08.730138+0000","last_undegraded":"2024-04-25T00:04:08.730138+0000","last_fullsized":"2024-04-25T00:04:08.730138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:56.543825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.730137+0000","last_change":"2024-04-25T00:04:06.962198+0000","last_active":"2024-04-25T00:04:08.730137+0000","last_peered":"2024-04-25T00:04:08.730137+0000","last_clean":"2024-04-25T00:04:08.730137+0000","last_became_active":"2024-04-25T00:04:06.961841+0000","last_became_peered":"2024-04-25T00:04:06.961841+0000","last_unstale":"2024-04-25T00:04:08.730137+0000","last_undegraded":"2024-04-25T00:04:08.730137+0000","last_fullsized":"2024-04-25T00:04:08.730137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:15.729207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.728719+0000","last_change":"2024-04-25T00:04:05.949157+0000","last_active":"2024-04-25T00:04:08.728719+0000","last_peered":"2024-04-25T00:04:08.728719+0000","last_clean":"2024-04-25T00:04:08.728719+0000","last_became_active":"2024-04-25T00:04:05.948879+0000","last_became_peered":"2024-04-25T00:04:05.948879+0000","last_unstale":"2024-04-25T00:04:08.728719+0000","last_undegraded":"2024-04-25T00:04:08.728719+0000","last_fullsized":"2024-04-25T00:04:08.728719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:40.441664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:08.081958+0000","last_change":"2024-04-25T00:04:06.954858+0000","last_active":"2024-04-25T00:04:08.081958+0000","last_peered":"2024-04-25T00:04:08.081958+0000","last_clean":"2024-04-25T00:04:08.081958+0000","last_became_active":"2024-04-25T00:04:06.954554+0000","last_became_peered":"2024-04-25T00:04:06.954554+0000","last_unstale":"2024-04-25T00:04:08.081958+0000","last_undegraded":"2024-04-25T00:04:08.081958+0000","last_fullsized":"2024-04-25T00:04:08.081958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:05.931537+0000","last_clean_scrub_stamp":"2024-04-25T00:04:05.931537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:19.677653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942966+0000","last_change":"2024-04-25T00:04:05.957149+0000","last_active":"2024-04-25T00:04:07.942966+0000","last_peered":"2024-04-25T00:04:07.942966+0000","last_clean":"2024-04-25T00:04:07.942966+0000","last_became_active":"2024-04-25T00:04:05.956870+0000","last_became_peered":"2024-04-25T00:04:05.956870+0000","last_unstale":"2024-04-25T00:04:07.942966+0000","last_undegraded":"2024-04-25T00:04:07.942966+0000","last_fullsized":"2024-04-25T00:04:07.942966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:04.926826+0000","last_clean_scrub_stamp":"2024-04-25T00:04:04.926826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:56.781068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":302,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:04:07.942651+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:04:07.942651+0000","last_peered":"2024-04-25T00:04:07.942651+0000","last_clean":"2024-04-25T00:04:07.942651+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:04:07.942651+0000","last_undegraded":"2024-04-25T00:04:07.942651+0000","last_fullsized":"2024-04-25T00:04:07.942651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55084,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316756,"statfs":{"total":96636764160,"available":96580358144,"internally_reserved":0,"allocated":1728512,"data_stored":1078753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54504,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317336,"statfs":{"total":96636764160,"available":96580952064,"internally_reserved":0,"allocated":1142784,"data_stored":489009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1146880,"data_stored":489949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54492,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317348,"statfs":{"total":96636764160,"available":96580964352,"internally_reserved":0,"allocated":1130496,"data_stored":488315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58672,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313168,"statfs":{"total":96636764160,"available":96576684032,"internally_reserved":0,"allocated":1142784,"data_stored":489927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1134592,"data_stored":488897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":1142784,"data_stored":488509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41188,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330652,"statfs":{"total":96636764160,"available":96594587648,"internally_reserved":0,"allocated":1728512,"data_stored":1078717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:04:14.593 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:04:14.594 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:04:14.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.733+0000 7f4675bc1640 1 Processor -- start 2024-04-25T00:04:14.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.733+0000 7f4675bc1640 1 -- start start 2024-04-25T00:04:14.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f4675bc1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 0x7f4670165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f4675bc1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f467015f4f0 con 0x7f46700b3080 2024-04-25T00:04:14.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f4675bc1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f467015f7d0 con 0x7f467015e2d0 2024-04-25T00:04:14.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f4675bc1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46700cde00 con 0x7f46700b4a40 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f466effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f46700b3080 0x7f46700b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34762/0 (socket says 172.21.15.113:34762) 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f466effd640 1 -- 172.21.15.113:0/2284823747 learned_addr learned my addr 172.21.15.113:0/2284823747 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2100861270 0 0) 0x7f467015f4f0 con 0x7f46700b3080 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f466ffff640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 0x7f4670165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.734+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4658003520 con 0x7f46700b3080 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 986828096 0 0) 0x7f4658003520 con 0x7f46700b3080 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 msgr2=0x7f4670165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 0x7f4670165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 >> v1:172.21.15.178:6789/0 conn(0x7f467015e2d0 legacy=0x7f467015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467015fab0 con 0x7f46700b3080 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f4664001670 con 0x7f46700b3080 2024-04-25T00:04:14.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 >> v1:172.21.15.113:6789/0 conn(0x7f46700b3080 legacy=0x7f46700b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 0x7f46580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4658003f40 0x7f4658004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658004d40 0x7f46580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f46700cde00 con 0x7f4658003f40 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f465800baa0 con 0x7f4658004d40 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f465800bca0 con 0x7f46700b4a40 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466effd640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4658003f40 0x7f4658004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466ffff640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658004d40 0x7f46580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466f7fe640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 0x7f46580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466effd640 1 -- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 msgr2=0x7f46580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466effd640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46700b4a40 0x7f46580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466effd640 1 -- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658004d40 msgr2=0x7f46580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466effd640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658004d40 0x7f46580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.735+0000 7f466effd640 1 -- 172.21.15.113:0/2284823747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4658003520 con 0x7f4658003f40 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f466effd640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4658003f40 0x7f4658004800 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f466400c190 tx=0x7f4658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391d6e9c121c8afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664007c60 con 0x7f4658003f40 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4664005af0 con 0x7f4658003f40 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f466e7fc640 1 -- 172.21.15.113:0/2284823747 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664007070 con 0x7f4658003f40 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f4675bc1640 1 -- 172.21.15.113:0/2284823747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4658003f40 msgr2=0x7f4658004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f4675bc1640 1 --2- 172.21.15.113:0/2284823747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4658003f40 0x7f4658004800 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f466400c190 tx=0x7f4658005830 comp rx=0 tx=0).stop 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f466ffff640 1 -- 172.21.15.113:0/2284823747 reap_dead start 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f4675bc1640 1 -- 172.21.15.113:0/2284823747 shutdown_connections 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.736+0000 7f4675bc1640 1 -- 172.21.15.113:0/2284823747 >> 172.21.15.113:0/2284823747 conn(0x7f467006b0f0 msgr2=0x7f467006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.737+0000 7f4675bc1640 1 -- 172.21.15.113:0/2284823747 shutdown_connections 2024-04-25T00:04:14.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.737+0000 7f4675bc1640 1 -- 172.21.15.113:0/2284823747 wait complete. 2024-04-25T00:04:14.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.738+0000 7f4675bc1640 1 Processor -- start 2024-04-25T00:04:14.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.738+0000 7f4675bc1640 1 -- start start 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.738+0000 7f4675bc1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658003f40 0x7f46700aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4675bc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4670156ce0 0x7f4670151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4675bc1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46701519b0 0x7f46701f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4675bc1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f46700cf420 con 0x7f4670156ce0 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4675bc1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46700cf2a0 con 0x7f4658003f40 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4675bc1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46700cf5a0 con 0x7f46701519b0 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4670156ce0 0x7f4670151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4670156ce0 0x7f4670151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54010/0 (socket says 172.21.15.113:54010) 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 -- 172.21.15.113:0/4245931982 learned_addr learned my addr 172.21.15.113:0/4245931982 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466ffff640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46701519b0 0x7f46701f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466f7fe640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658003f40 0x7f46700aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 -- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46701519b0 msgr2=0x7f46701f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46701519b0 0x7f46701f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 -- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658003f40 msgr2=0x7f46700aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658003f40 0x7f46700aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 -- 172.21.15.113:0/4245931982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46700af270 con 0x7f4670156ce0 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466ffff640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46701519b0 0x7f46701f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466f7fe640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658003f40 0x7f46700aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f466effd640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4670156ce0 0x7f4670151470 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f466400c2b0 tx=0x7f467015f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4657fff640 1 -- 172.21.15.113:0/4245931982 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664041020 con 0x7f4670156ce0 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4657fff640 1 -- 172.21.15.113:0/4245931982 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46640013b0 con 0x7f4670156ce0 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4670155c40 con 0x7f4670156ce0 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4657fff640 1 -- 172.21.15.113:0/4245931982 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664007070 con 0x7f4670156ce0 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.739+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4670156200 con 0x7f4670156ce0 2024-04-25T00:04:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.740+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46700691a0 con 0x7f4670156ce0 2024-04-25T00:04:14.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.741+0000 7f4657fff640 1 -- 172.21.15.113:0/4245931982 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4664044020 con 0x7f4670156ce0 2024-04-25T00:04:14.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.741+0000 7f4657fff640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465c06be60 0x7f465c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:14.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.741+0000 7f4657fff640 1 -- 172.21.15.113:0/4245931982 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f46640ab180 con 0x7f4670156ce0 2024-04-25T00:04:14.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.741+0000 7f466f7fe640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465c06be60 0x7f465c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:14.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.742+0000 7f466f7fe640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465c06be60 0x7f465c06e320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:14.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.745+0000 7f4657fff640 1 -- 172.21.15.113:0/4245931982 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4664079bc0 con 0x7f4670156ce0 2024-04-25T00:04:14.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.988+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4670072050 con 0x7f4670156ce0 2024-04-25T00:04:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.991+0000 7f4657fff640 1 -- 172.21.15.113:0/4245931982 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f46640794c0 con 0x7f4670156ce0 2024-04-25T00:04:14.991 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:04:14.991 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:04:14.991 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:04:14.991 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:04:14.991 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:04:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:04:14.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.992+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465c06be60 msgr2=0x7f465c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.992+0000 7f4675bc1640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465c06be60 0x7f465c06e320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660009040 comp rx=0 tx=0).stop 2024-04-25T00:04:14.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.992+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4670156ce0 msgr2=0x7f4670151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:14.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.992+0000 7f4675bc1640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4670156ce0 0x7f4670151470 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f466400c2b0 tx=0x7f467015f9a0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 shutdown_connections 2024-04-25T00:04:14.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f465c06be60 0x7f465c06e320 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46701519b0 0x7f46701f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4670156ce0 0x7f4670151470 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 --2- 172.21.15.113:0/4245931982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4658003f40 0x7f46700aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:14.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 >> 172.21.15.113:0/4245931982 conn(0x7f467006b0f0 msgr2=0x7f46700729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:14.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 shutdown_connections 2024-04-25T00:04:14.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:14.993+0000 7f4675bc1640 1 -- 172.21.15.113:0/4245931982 wait complete. 2024-04-25T00:04:15.004 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:15.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.100+0000 7f12aa4bc640 1 Processor -- start 2024-04-25T00:04:15.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.101+0000 7f12aa4bc640 1 -- start start 2024-04-25T00:04:15.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.101+0000 7f12aa4bc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 0x7f12a40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.101+0000 7f12aa4bc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a40d2800 con 0x7f12a40d0b00 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.101+0000 7f12aa4bc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a40d2ae0 con 0x7f12a415e320 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.101+0000 7f12aa4bc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12a40d3410 con 0x7f12a40d2290 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.101+0000 7f12a8a59640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f12a40d0b00 0x7f12a40b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34772/0 (socket says 172.21.15.113:34772) 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.101+0000 7f12a8a59640 1 -- 172.21.15.113:0/913172612 learned_addr learned my addr 172.21.15.113:0/913172612 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a37fe640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 0x7f12a40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284195264 0 0) 0x7f12a40d2800 con 0x7f12a40d0b00 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f128c003520 con 0x7f12a40d0b00 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2752868232 0 0) 0x7f128c003520 con 0x7f12a40d0b00 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 msgr2=0x7f12a40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 0x7f12a40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 >> v1:172.21.15.178:6789/0 conn(0x7f12a415e320 legacy=0x7f12a40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a40d2dc0 con 0x7f12a40d0b00 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f129c0019f0 con 0x7f12a40d0b00 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a37fe640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 0x7f12a40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 >> v1:172.21.15.113:6789/0 conn(0x7f12a40d0b00 legacy=0x7f12a40b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0039e0 0x7f128c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 0x7f128c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c004d40 0x7f128c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40d3410 con 0x7f128c0039e0 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f128c00b9f0 con 0x7f128c004d40 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f128c00bbf0 con 0x7f12a40d2290 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a3fff640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0039e0 0x7f128c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a37fe640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 0x7f128c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a8a59640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c004d40 0x7f128c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a3fff640 1 -- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 msgr2=0x7f128c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a3fff640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12a40d2290 0x7f128c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a3fff640 1 -- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c004d40 msgr2=0x7f128c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a3fff640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c004d40 0x7f128c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a3fff640 1 -- 172.21.15.113:0/913172612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c003520 con 0x7f128c0039e0 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.102+0000 7f12a8a59640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c004d40 0x7f128c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.103+0000 7f12a3fff640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0039e0 0x7f128c003dc0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1294001860 tx=0x7f128c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ea4779cc4e7ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.103+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294037070 con 0x7f128c0039e0 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.103+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129402ac30 con 0x7f128c0039e0 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.103+0000 7f12a2ffd640 1 -- 172.21.15.113:0/913172612 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294001430 con 0x7f128c0039e0 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.104+0000 7f12aa4bc640 1 -- 172.21.15.113:0/913172612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0039e0 msgr2=0x7f128c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.104+0000 7f12aa4bc640 1 --2- 172.21.15.113:0/913172612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0039e0 0x7f128c003dc0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1294001860 tx=0x7f128c005830 comp rx=0 tx=0).stop 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.104+0000 7f12a8a59640 1 -- 172.21.15.113:0/913172612 reap_dead start 2024-04-25T00:04:15.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.104+0000 7f12aa4bc640 1 -- 172.21.15.113:0/913172612 shutdown_connections 2024-04-25T00:04:15.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.104+0000 7f12aa4bc640 1 -- 172.21.15.113:0/913172612 >> 172.21.15.113:0/913172612 conn(0x7f12a40b5c20 msgr2=0x7f12a40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.104+0000 7f12aa4bc640 1 -- 172.21.15.113:0/913172612 shutdown_connections 2024-04-25T00:04:15.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.104+0000 7f12aa4bc640 1 -- 172.21.15.113:0/913172612 wait complete. 2024-04-25T00:04:15.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.105+0000 7f12aa4bc640 1 Processor -- start 2024-04-25T00:04:15.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.105+0000 7f12aa4bc640 1 -- start start 2024-04-25T00:04:15.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12aa4bc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0039e0 0x7f12a415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12aa4bc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12a40c8190 0x7f12a415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12aa4bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a40ccc90 0x7f12a40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12aa4bc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12a415e880 con 0x7f12a40c8190 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12aa4bc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12a415e700 con 0x7f12a40ccc90 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12aa4bc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12a40d0f90 con 0x7f128c0039e0 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12a40c8190 0x7f12a415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12a40c8190 0x7f12a415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54024/0 (socket says 172.21.15.113:54024) 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 -- 172.21.15.113:0/3002786704 learned_addr learned my addr 172.21.15.113:0/3002786704 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a8a59640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a40ccc90 0x7f12a40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a3fff640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0039e0 0x7f12a415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 -- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0039e0 msgr2=0x7f12a415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0039e0 0x7f12a415ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 -- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a40ccc90 msgr2=0x7f12a40cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a40ccc90 0x7f12a40cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 -- 172.21.15.113:0/3002786704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a40c8690 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a37fe640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12a40c8190 0x7f12a415ef40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f12980119d0 tx=0x7f1298011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a0ff9640 1 -- 172.21.15.113:0/3002786704 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129800faf0 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a0ff9640 1 -- 172.21.15.113:0/3002786704 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129800fc90 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a40c8970 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.106+0000 7f12a0ff9640 1 -- 172.21.15.113:0/3002786704 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298004030 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.107+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a40c8f30 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.108+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a40691a0 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.108+0000 7f12a0ff9640 1 -- 172.21.15.113:0/3002786704 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1298012040 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.108+0000 7f12a0ff9640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f129006be60 0x7f129006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.108+0000 7f12a0ff9640 1 -- 172.21.15.113:0/3002786704 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f129808b130 con 0x7f12a40c8190 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.108+0000 7f12a3fff640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f129006be60 0x7f129006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.109+0000 7f12a3fff640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f129006be60 0x7f129006e320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1294034000 tx=0x7f1294009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.113+0000 7f12a0ff9640 1 -- 172.21.15.113:0/3002786704 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1298019070 con 0x7f12a40c8190 2024-04-25T00:04:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.321+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12a40c9310 con 0x7f12a40c8190 2024-04-25T00:04:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.321+0000 7f12a0ff9640 1 -- 172.21.15.113:0/3002786704 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5195 (secure 0 0 0) 0x7f1298059b70 con 0x7f12a40c8190 2024-04-25T00:04:15.321 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:15.322 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:06.288869+0000","modified":"2024-04-25T00:04:08.290392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2796979045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2796979045},{"type":"v1","addr":"172.21.15.113:6837","nonce":2796979045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7395,"qdb_cluster":[7395]},"id":6}]} 2024-04-25T00:04:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-04-25T00:04:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f129006be60 msgr2=0x7f129006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f129006be60 0x7f129006e320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1294034000 tx=0x7f1294009030 comp rx=0 tx=0).stop 2024-04-25T00:04:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12a40c8190 msgr2=0x7f12a415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12a40c8190 0x7f12a415ef40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f12980119d0 tx=0x7f1298011e90 comp rx=0 tx=0).stop 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 shutdown_connections 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f129006be60 0x7f129006e320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a40ccc90 0x7f12a40cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12a40c8190 0x7f12a415ef40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 --2- 172.21.15.113:0/3002786704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0039e0 0x7f12a415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 >> 172.21.15.113:0/3002786704 conn(0x7f12a40b5c20 msgr2=0x7f12a40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 shutdown_connections 2024-04-25T00:04:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.323+0000 7f12aa4bc640 1 -- 172.21.15.113:0/3002786704 wait complete. 2024-04-25T00:04:15.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:04:06.288869+0000', 'modified': '2024-04-25T00:04:08.290392+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7395': {'gid': 7395, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6837/2796979045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 2796979045}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 2796979045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7395, 'qdb_cluster': [7395]} 2024-04-25T00:04:15.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:04:15.737 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:04:15.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.835+0000 7fa9e5fee640 1 Processor -- start 2024-04-25T00:04:15.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.835+0000 7fa9e5fee640 1 -- start start 2024-04-25T00:04:15.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9e5fee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9e00b3080 0x7fa9e00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9e5fee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e015f4f0 con 0x7fa9e015e2d0 2024-04-25T00:04:15.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9e5fee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e015f7d0 con 0x7fa9e015e6b0 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9e5fee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e00cde00 con 0x7fa9e00b3080 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9deffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9e00b3080 0x7fa9e00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9df7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa9e015e2d0 0x7fa9e015ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34788/0 (socket says 172.21.15.113:34788) 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9df7fe640 1 -- 172.21.15.113:0/1535477885 learned_addr learned my addr 172.21.15.113:0/1535477885 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9deffd640 1 -- 172.21.15.113:0/1535477885 >> v1:172.21.15.178:6789/0 conn(0x7fa9e015e6b0 legacy=0x7fa9e00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9deffd640 1 -- 172.21.15.113:0/1535477885 >> v1:172.21.15.113:6789/0 conn(0x7fa9e015e2d0 legacy=0x7fa9e015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9de7fc640 1 -- 172.21.15.113:0/1535477885 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140218225 0 0) 0x7fa9e015f4f0 con 0x7fa9e015e2d0 2024-04-25T00:04:15.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.836+0000 7fa9deffd640 1 -- 172.21.15.113:0/1535477885 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e015fab0 con 0x7fa9e00b3080 2024-04-25T00:04:15.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9deffd640 1 --2- 172.21.15.113:0/1535477885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e00b4330 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00097a0 tx=0x7fa9d002c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c237091bcd950c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9de7fc640 1 -- 172.21.15.113:0/1535477885 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0039070 con 0x7fa9e00b3080 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9de7fc640 1 -- 172.21.15.113:0/1535477885 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d0034640 con 0x7fa9e00b3080 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9de7fc640 1 -- 172.21.15.113:0/1535477885 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0034a10 con 0x7fa9e00b3080 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9e5fee640 1 -- 172.21.15.113:0/1535477885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 msgr2=0x7fa9e00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/1535477885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e00b4330 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00097a0 tx=0x7fa9d002c3f0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9e5fee640 1 -- 172.21.15.113:0/1535477885 shutdown_connections 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/1535477885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e00b4330 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9e5fee640 1 -- 172.21.15.113:0/1535477885 >> 172.21.15.113:0/1535477885 conn(0x7fa9e006b0f0 msgr2=0x7fa9e006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9e5fee640 1 -- 172.21.15.113:0/1535477885 shutdown_connections 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.837+0000 7fa9e5fee640 1 -- 172.21.15.113:0/1535477885 wait complete. 2024-04-25T00:04:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 Processor -- start 2024-04-25T00:04:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 -- start start 2024-04-25T00:04:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e01f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9e015e2d0 0x7fa9e015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9e015e6b0 0x7fa9e01f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e00b49f0 con 0x7fa9e00b3080 2024-04-25T00:04:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e00b4870 con 0x7fa9e015e6b0 2024-04-25T00:04:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9e5fee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e00b4b70 con 0x7fa9e015e2d0 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.839+0000 7fa9df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e01f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e01f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54038/0 (socket says 172.21.15.113:54038) 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 -- 172.21.15.113:0/945534189 learned_addr learned my addr 172.21.15.113:0/945534189 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9dffff640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9e015e6b0 0x7fa9e01f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9deffd640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9e015e2d0 0x7fa9e015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 -- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9e015e2d0 msgr2=0x7fa9e015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9e015e2d0 0x7fa9e015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 -- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9e015e6b0 msgr2=0x7fa9e01f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9e015e6b0 0x7fa9e01f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 -- 172.21.15.113:0/945534189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e01f4900 con 0x7fa9e00b3080 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e01f43c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800c4d0 tx=0x7fa9c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9d7fff640 1 -- 172.21.15.113:0/945534189 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8018070 con 0x7fa9e00b3080 2024-04-25T00:04:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9d7fff640 1 -- 172.21.15.113:0/945534189 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c8007500 con 0x7fa9e00b3080 2024-04-25T00:04:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e01f4bb0 con 0x7fa9e00b3080 2024-04-25T00:04:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9d7fff640 1 -- 172.21.15.113:0/945534189 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8004a20 con 0x7fa9e00b3080 2024-04-25T00:04:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.840+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e01f5170 con 0x7fa9e00b3080 2024-04-25T00:04:15.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.842+0000 7fa9d7fff640 1 -- 172.21.15.113:0/945534189 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9c801b070 con 0x7fa9e00b3080 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.842+0000 7fa9d7fff640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa9c006be30 0x7fa9c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.842+0000 7fa9d7fff640 1 -- 172.21.15.113:0/945534189 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fa9c808a0a0 con 0x7fa9e00b3080 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.842+0000 7fa9deffd640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa9c006be30 0x7fa9c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.842+0000 7fa9deffd640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa9c006be30 0x7fa9c006e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0036000 tx=0x7fa9d0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.843+0000 7fa9e8ddc740 1 Processor -- start 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.843+0000 7fa9e8ddc740 1 -- start start 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135464b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56135467bc80 0x56135467b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467b540 0x561354650350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5613545b3f00 con 0x56135467bc80 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56135455c750 con 0x561354698b60 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5613545afec0 con 0x56135467b540 2024-04-25T00:04:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56135467bc80 0x56135467b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56135467bc80 0x56135467b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54042/0 (socket says 172.21.15.113:54042) 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 -- 172.21.15.113:0/1353544808 learned_addr learned my addr 172.21.15.113:0/1353544808 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9dffff640 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467b540 0x561354650350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 -- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467b540 msgr2=0x561354650350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135464b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467b540 0x561354650350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 -- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 msgr2=0x56135464b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135464b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.844+0000 7fa9deffd640 1 -- 172.21.15.113:0/1353544808 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613542c82d0 con 0x56135467bc80 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9deffd640 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56135467bc80 0x56135467b000 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009890 tx=0x7fa9d002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e3eec9e517fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9d57fa640 1 -- 172.21.15.113:0/1353544808 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0039040 con 0x56135467bc80 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9d57fa640 1 -- 172.21.15.113:0/1353544808 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d004e070 con 0x56135467bc80 2024-04-25T00:04:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9d57fa640 1 -- 172.21.15.113:0/1353544808 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d003e880 con 0x56135467bc80 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1353544808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56135467bc80 msgr2=0x56135467b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56135467bc80 0x56135467b000 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009890 tx=0x7fa9d002e000 comp rx=0 tx=0).stop 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1353544808 shutdown_connections 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467b540 0x561354650350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56135467bc80 0x56135467b000 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1353544808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135464b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1353544808 >> 172.21.15.113:0/1353544808 conn(0x56135464d760 msgr2=0x561354696000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1353544808 shutdown_connections 2024-04-25T00:04:15.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.845+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1353544808 wait complete. 2024-04-25T00:04:15.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.846+0000 7fa9e8ddc740 1 -- start start 2024-04-25T00:04:15.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.846+0000 7fa9e8ddc740 1 Processor -- start 2024-04-25T00:04:15.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x56135467e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135464a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56135464f910 con 0x561354698b60 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56135456fcf0 con 0x561354668aa0 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56135464ed00 con 0x56135467bc80 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 10 client.0 _ll_register_callbacks cb 0x56135456f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x56135467e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x56135467e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54054/0 (socket says 172.21.15.113:54054) 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 -- 172.21.15.113:0/377484421 learned_addr learned my addr 172.21.15.113:0/377484421 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9dffff640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135464a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 -- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 msgr2=0x56135467e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 -- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 msgr2=0x56135464a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135464a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 -- 172.21.15.113:0/377484421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613543926b0 con 0x561354698b60 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9deffd640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x56135467e570 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00097a0 tx=0x7fa9d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a0324bee0d2082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9be7fc640 1 -- 172.21.15.113:0/377484421 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0039040 con 0x561354698b60 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9be7fc640 1 -- 172.21.15.113:0/377484421 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fa9d0052420 con 0x561354698b60 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.847+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d004e070 con 0x561354698b60 2024-04-25T00:04:15.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9be7fc640 1 -- 172.21.15.113:0/377484421 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa9d0063050 con 0x561354698b60 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 10 client.7572 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5613546a9ff0 con 0x561354698b60 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9be7fc640 1 -- 172.21.15.113:0/377484421 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d004e070 con 0x561354698b60 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9be7fc640 1 -- 172.21.15.113:0/377484421 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fa9d0066020 con 0x561354698b60 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 10 client.7572 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 10 client.7572 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 20 client.7572 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x56135467a190 0x56135467c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 4 client.7572 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613545c1440 con 0x56135467a190 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x56135467a190 0x56135467c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x56135467a190 0x56135467c6c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8242649d20afbd8a server_cookie=3dbe6cc14ab33711 in_seq=0 out_seq=0 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.848+0000 7fa9be7fc640 10 client.7572 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:15.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9be7fc640 1 -- 172.21.15.113:0/377484421 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613545c1440 con 0x56135467a190 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9be7fc640 10 client.7572 handle_command_reply: tid=0 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 client.7572 shutdown 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 msgr2=0x56135467e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x56135467e570 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00097a0 tx=0x7fa9d0004730 comp rx=0 tx=0).stop 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 shutdown_connections 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x56135467a190 0x56135467c6c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135464a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x56135467e570 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/377484421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9be7fc640 0 client.7572 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 >> 172.21.15.113:0/377484421 conn(0x56135467b540 msgr2=0x561354696890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 shutdown_connections 2024-04-25T00:04:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/377484421 wait complete. 2024-04-25T00:04:15.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.849+0000 7fa9e8ddc740 20 client.7572 trim_cache size 0 max 16384 2024-04-25T00:04:15.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.867+0000 7fa9e8ddc740 1 Processor -- start 2024-04-25T00:04:15.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.867+0000 7fa9e8ddc740 1 -- start start 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354668aa0 0x5613546b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135467f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e00ae000 con 0x561354668aa0 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e015c3e0 con 0x561354698b60 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e00ad9b0 con 0x56135467bc80 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354668aa0 0x5613546b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354668aa0 0x5613546b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54068/0 (socket says 172.21.15.113:54068) 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 -- 172.21.15.113:0/1616760281 learned_addr learned my addr 172.21.15.113:0/1616760281 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9deffd640 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9dffff640 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135467f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 -- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 msgr2=0x56135467ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 -- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 msgr2=0x56135467f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135467f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9df7fe640 1 -- 172.21.15.113:0/1616760281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135464a5c0 con 0x561354668aa0 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.868+0000 7fa9dffff640 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135467f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354668aa0 0x5613546b88b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80548f0 tx=0x7fa9c805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d7ca2828d33d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9d4ff9640 1 -- 172.21.15.113:0/1616760281 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8086070 con 0x561354668aa0 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9d4ff9640 1 -- 172.21.15.113:0/1616760281 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c8070030 con 0x561354668aa0 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9d4ff9640 1 -- 172.21.15.113:0/1616760281 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80701e0 con 0x561354668aa0 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1616760281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354668aa0 msgr2=0x5613546b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354668aa0 0x5613546b88b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80548f0 tx=0x7fa9c805b040 comp rx=0 tx=0).stop 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1616760281 shutdown_connections 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354698b60 0x56135467f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x56135467ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1616760281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354668aa0 0x5613546b88b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1616760281 >> 172.21.15.113:0/1616760281 conn(0x56135467b540 msgr2=0x561354697e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1616760281 shutdown_connections 2024-04-25T00:04:15.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.869+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1616760281 wait complete. 2024-04-25T00:04:15.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.870+0000 7fa9e8ddc740 1 -- start start 2024-04-25T00:04:15.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.870+0000 7fa9e8ddc740 1 Processor -- start 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135467c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x561354665850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x5613546a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56135467a010 con 0x561354698b60 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56135464f910 con 0x561354668aa0 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56135467a630 con 0x56135467bc80 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 10 client.0 _ll_register_callbacks cb 0x7fa9d003e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x5613546a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x5613546a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54082/0 (socket says 172.21.15.113:54082) 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 -- 172.21.15.113:0/1509804761 learned_addr learned my addr 172.21.15.113:0/1509804761 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135467c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9deffd640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x561354665850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 -- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 msgr2=0x561354665850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x561354665850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 -- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 msgr2=0x56135467c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135467c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 -- 172.21.15.113:0/1509804761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135468b990 con 0x561354698b60 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135467c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9dffff640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x5613546a4e30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005980 tx=0x7fa9d8014770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f07fd374eed5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9be7fc640 1 -- 172.21.15.113:0/1509804761 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d801b070 con 0x561354698b60 2024-04-25T00:04:15.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9be7fc640 1 -- 172.21.15.113:0/1509804761 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fa9d8006d60 con 0x561354698b60 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.871+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c8070030 con 0x561354698b60 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9be7fc640 1 -- 172.21.15.113:0/1509804761 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa9d8026050 con 0x561354698b60 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 10 client.7578 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x561354679c40 con 0x561354698b60 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9be7fc640 1 -- 172.21.15.113:0/1509804761 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c8070030 con 0x561354698b60 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9be7fc640 1 -- 172.21.15.113:0/1509804761 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fa9d8029020 con 0x561354698b60 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 10 client.7578 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 10 client.7578 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 20 client.7578 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5613546a7fc0 0x5613546a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 4 client.7578 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "status"}) -- 0x56135464cbb0 con 0x5613546a7fc0 2024-04-25T00:04:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5613546a7fc0 0x5613546a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9be7fc640 10 client.7578 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9df7fe640 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5613546a7fc0 0x5613546a83a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8faf633cc87d4590 server_cookie=a52dca0bada30072 in_seq=0 out_seq=0 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9be7fc640 1 -- 172.21.15.113:0/1509804761 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56135464cbb0 con 0x5613546a7fc0 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9be7fc640 10 client.7578 handle_command_reply: tid=0 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.872+0000 7fa9e8ddc740 1 client.7578 shutdown 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 msgr2=0x5613546a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x5613546a4e30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005980 tx=0x7fa9d8014770 comp rx=0 tx=0).stop 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 shutdown_connections 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5613546a7fc0 0x5613546a83a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561354698b60 0x5613546a4e30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56135467bc80 0x561354665850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 --2- 172.21.15.113:0/1509804761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561354668aa0 0x56135467c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 >> 172.21.15.113:0/1509804761 conn(0x56135467b540 msgr2=0x5613546b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9be7fc640 0 client.7578 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 shutdown_connections 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 1 -- 172.21.15.113:0/1509804761 wait complete. 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e8ddc740 20 client.7578 trim_cache size 0 max 16384 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 7395, 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:04:15.881 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:04:15.882 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 8.9351759400000006, 2024-04-25T00:04:15.882 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 50, 2024-04-25T00:04:15.882 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 92, 2024-04-25T00:04:15.882 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T00:04:15.882 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 14.001301579 2024-04-25T00:04:15.882 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa9c006be30 msgr2=0x7fa9c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa9c006be30 0x7fa9c006e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0036000 tx=0x7fa9d0030000 comp rx=0 tx=0).stop 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 msgr2=0x7fa9e01f43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e01f43c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800c4d0 tx=0x7fa9c800c990 comp rx=0 tx=0).stop 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 shutdown_connections 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa9c006be30 0x7fa9c006e2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9e015e6b0 0x7fa9e01f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9e015e2d0 0x7fa9e015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 --2- 172.21.15.113:0/945534189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9e00b3080 0x7fa9e01f43c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 >> 172.21.15.113:0/945534189 conn(0x7fa9e006b0f0 msgr2=0x7fa9e00ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 shutdown_connections 2024-04-25T00:04:15.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.873+0000 7fa9e5fee640 1 -- 172.21.15.113:0/945534189 wait complete. 2024-04-25T00:04:15.885 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:04:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.982+0000 7f1efca9e640 1 Processor -- start 2024-04-25T00:04:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.982+0000 7f1efca9e640 1 -- start start 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1efca9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 0x7f1ef806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1efca9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef8144cf0 con 0x7f1ef8144910 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1efca9e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef80b2f30 con 0x7f1ef806e510 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1efca9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef80b3210 con 0x7f1ef80b3fd0 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1ef8144910 0x7f1ef80b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34804/0 (socket says 172.21.15.113:34804) 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef659c640 1 -- 172.21.15.113:0/740164220 learned_addr learned my addr 172.21.15.113:0/740164220 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1eeffff640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 0x7f1ef806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056904476 0 0) 0x7f1ef8144cf0 con 0x7f1ef8144910 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ee4003520 con 0x7f1ef8144910 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1651039362 0 0) 0x7f1ee4003520 con 0x7f1ef8144910 2024-04-25T00:04:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 msgr2=0x7f1ef806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 0x7f1ef806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 >> v1:172.21.15.178:6789/0 conn(0x7f1ef806e510 legacy=0x7f1ef80cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef80b34f0 con 0x7f1ef8144910 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1ee8001670 con 0x7f1ef8144910 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 >> v1:172.21.15.113:6789/0 conn(0x7f1ef8144910 legacy=0x7f1ef80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ee40039e0 0x7f1ee4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 0x7f1ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef80b3210 con 0x7f1ee4004800 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee400baa0 con 0x7f1ee40039e0 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.983+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee400deb0 con 0x7f1ef80b3fd0 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1eeffff640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1ef659c640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ee40039e0 0x7f1ee4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1ef6d9d640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 0x7f1ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1eeffff640 1 -- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 msgr2=0x7f1ee40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1eeffff640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 0x7f1ee40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1eeffff640 1 -- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ee40039e0 msgr2=0x7f1ee4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1eeffff640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ee40039e0 0x7f1ee4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1eeffff640 1 -- 172.21.15.113:0/740164220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee4003520 con 0x7f1ee4004800 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.984+0000 7f1ef6d9d640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80b3fd0 0x7f1ee40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1eeffff640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ee4004c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009a90 tx=0x7f1ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a78e4bc043d416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0036070 con 0x7f1ee4004800 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee0031400 con 0x7f1ee4004800 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1ef5d9b640 1 -- 172.21.15.113:0/740164220 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0009690 con 0x7f1ee4004800 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1efca9e640 1 -- 172.21.15.113:0/740164220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 msgr2=0x7f1ee4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1efca9e640 1 --2- 172.21.15.113:0/740164220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ee4004c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009a90 tx=0x7f1ee4005830 comp rx=0 tx=0).stop 2024-04-25T00:04:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1ef6d9d640 1 -- 172.21.15.113:0/740164220 reap_dead start 2024-04-25T00:04:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1efca9e640 1 -- 172.21.15.113:0/740164220 shutdown_connections 2024-04-25T00:04:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1efca9e640 1 -- 172.21.15.113:0/740164220 >> 172.21.15.113:0/740164220 conn(0x7f1ef80b1a50 msgr2=0x7f1ef80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1efca9e640 1 -- 172.21.15.113:0/740164220 shutdown_connections 2024-04-25T00:04:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.985+0000 7f1efca9e640 1 -- 172.21.15.113:0/740164220 wait complete. 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.986+0000 7f1efca9e640 1 Processor -- start 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1efca9e640 1 -- start start 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1efca9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ef815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1efca9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef80aee90 0x7f1ef80ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1efca9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80ab590 0x7f1ef80ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1efca9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef80b4da0 con 0x7f1ee4004800 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1efca9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef80b4410 con 0x7f1ef80aee90 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1efca9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef80b4f20 con 0x7f1ef80ab590 2024-04-25T00:04:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ef815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ef815c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54090/0 (socket says 172.21.15.113:54090) 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1ef659c640 1 -- 172.21.15.113:0/2903099285 learned_addr learned my addr 172.21.15.113:0/2903099285 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.987+0000 7f1eeffff640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef80aee90 0x7f1ef80ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef6d9d640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80ab590 0x7f1ef80ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef659c640 1 -- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80ab590 msgr2=0x7f1ef80ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef659c640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80ab590 0x7f1ef80ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef659c640 1 -- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef80aee90 msgr2=0x7f1ef80ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef659c640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef80aee90 0x7f1ef80ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef659c640 1 -- 172.21.15.113:0/2903099285 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef813cf60 con 0x7f1ee4004800 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1eeffff640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef80aee90 0x7f1ef80ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef6d9d640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80ab590 0x7f1ef80ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1ef659c640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ef815c350 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8007290 tx=0x7f1ee8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1eeeffd640 1 -- 172.21.15.113:0/2903099285 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800f960 con 0x7f1ee4004800 2024-04-25T00:04:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1eeeffd640 1 -- 172.21.15.113:0/2903099285 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee8018070 con 0x7f1ee4004800 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef815c890 con 0x7f1ee4004800 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1eeeffd640 1 -- 172.21.15.113:0/2903099285 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800fd40 con 0x7f1ee4004800 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.988+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef813fe80 con 0x7f1ee4004800 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.989+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef80ab9a0 con 0x7f1ee4004800 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.989+0000 7f1eeeffd640 1 -- 172.21.15.113:0/2903099285 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ee801d460 con 0x7f1ee4004800 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.990+0000 7f1eeeffd640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ed806be60 0x7f1ed806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.990+0000 7f1eeeffd640 1 -- 172.21.15.113:0/2903099285 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f1ee808aec0 con 0x7f1ee4004800 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.990+0000 7f1eeffff640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ed806be60 0x7f1ed806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.990+0000 7f1eeffff640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ed806be60 0x7f1ed806e320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0033000 tx=0x7f1ee0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:15.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:15.994+0000 7f1eeeffd640 1 -- 172.21.15.113:0/2903099285 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ef80ab9a0 con 0x7f1ee4004800 2024-04-25T00:04:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.209+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ef80691a0 con 0x7f1ee4004800 2024-04-25T00:04:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.209+0000 7f1eeeffd640 1 -- 172.21.15.113:0/2903099285 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) ==== 53+0+73 (secure 0 0 0) 0x7f1ef815c890 con 0x7f1ee4004800 2024-04-25T00:04:16.209 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:04:16.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ed806be60 msgr2=0x7f1ed806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ed806be60 0x7f1ed806e320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0033000 tx=0x7f1ee0008030 comp rx=0 tx=0).stop 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 msgr2=0x7f1ef815c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ef815c350 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8007290 tx=0x7f1ee8007750 comp rx=0 tx=0).stop 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 shutdown_connections 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ed806be60 0x7f1ed806e320 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef80aee90 0x7f1ef80ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef80ab590 0x7f1ef80ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 --2- 172.21.15.113:0/2903099285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ee4004800 0x7f1ef815c350 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 >> 172.21.15.113:0/2903099285 conn(0x7f1ef80b1a50 msgr2=0x7f1ef80cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 shutdown_connections 2024-04-25T00:04:16.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16.211+0000 7f1efca9e640 1 -- 172.21.15.113:0/2903099285 wait complete. 2024-04-25T00:04:16.222 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:04:16.222 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:04:16.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:04:16.222 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:04:16.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:16.223 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:04:16.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:04:16.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:04:16.223 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:04:16.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:16.223 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:04:16.238 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:16.239 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42424sec preferred_lft 42424sec 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:16.240 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:04:16.241 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:16.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:16.242 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:04:16.302 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:04:16.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:16.304 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:04:16.359 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:04:16.359 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:04:16.359 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:04:16.359 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:04:16.359 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:04:16.360 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:04:16.360 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:16.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:04:16.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:04:16.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:04:16.360 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:16.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:04:16.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:16.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:16.482 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:04:16.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:04:16.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:04:16.497 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:04:16.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:16.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:04:16.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:04:16.561 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:16.616 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:04:16.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:04:16.616 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:16.670 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:04:16.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:16.671 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:16.750 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:16.750 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:04:16.750 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:04:16.759 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:16.759 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:04:16.761 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:04:16.761 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:04:16.761 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:04:16.763 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:16.763 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:04:16.763 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:04:16.764 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:04:16.764 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:04:16.764 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:04:16.764 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:04:16.764 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:04:16.764 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:04:16.764 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:04:16.765 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:04:16.765 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:04:16.765 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:04:16.766 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:04:16.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:16.766 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:16.827 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:04:16.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:16.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:16.922 INFO:teuthology.orchestra.run.smithi113.stdout:client7587 2024-04-25T00:04:16.923 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:16.923 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:16.938 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:16.938 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:16.939 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T00:04:16.939 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:16.939 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:16.939 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:16.939 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:17.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:17.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880022 s, 119 MB/s 2024-04-25T00:04:17.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.052 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844598 s, 124 MB/s 2024-04-25T00:04:17.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.061 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738235 s, 142 MB/s 2024-04-25T00:04:17.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.070 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714549 s, 147 MB/s 2024-04-25T00:04:17.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.079 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713798 s, 147 MB/s 2024-04-25T00:04:17.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.089 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084654 s, 124 MB/s 2024-04-25T00:04:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810215 s, 129 MB/s 2024-04-25T00:04:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604922 s, 173 MB/s 2024-04-25T00:04:17.116 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.116 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.117 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092295 s, 114 MB/s 2024-04-25T00:04:17.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761562 s, 138 MB/s 2024-04-25T00:04:17.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.134 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682169 s, 154 MB/s 2024-04-25T00:04:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626661 s, 167 MB/s 2024-04-25T00:04:17.149 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058557 s, 179 MB/s 2024-04-25T00:04:17.157 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.157 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.157 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558749 s, 188 MB/s 2024-04-25T00:04:17.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.170 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757788 s, 138 MB/s 2024-04-25T00:04:17.174 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.175 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.175 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693599 s, 151 MB/s 2024-04-25T00:04:17.183 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.183 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.183 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736256 s, 142 MB/s 2024-04-25T00:04:17.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.192 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689959 s, 152 MB/s 2024-04-25T00:04:17.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.200 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709473 s, 148 MB/s 2024-04-25T00:04:17.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.208 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634686 s, 165 MB/s 2024-04-25T00:04:17.216 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.216 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.216 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598036 s, 175 MB/s 2024-04-25T00:04:17.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.225 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730346 s, 144 MB/s 2024-04-25T00:04:17.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.233 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694324 s, 151 MB/s 2024-04-25T00:04:17.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.242 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735824 s, 143 MB/s 2024-04-25T00:04:17.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.251 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698505 s, 150 MB/s 2024-04-25T00:04:17.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.261 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861381 s, 122 MB/s 2024-04-25T00:04:17.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.269 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660209 s, 159 MB/s 2024-04-25T00:04:17.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696954 s, 150 MB/s 2024-04-25T00:04:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741527 s, 141 MB/s 2024-04-25T00:04:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564415 s, 186 MB/s 2024-04-25T00:04:17.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766325 s, 137 MB/s 2024-04-25T00:04:17.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798035 s, 131 MB/s 2024-04-25T00:04:17.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:17.320 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:17.320 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:17.320 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T00:04:17.320 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:17.320 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:17.320 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:17.320 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:17.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:17.379 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.379 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.379 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793807 s, 132 MB/s 2024-04-25T00:04:17.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.386 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578469 s, 181 MB/s 2024-04-25T00:04:17.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748264 s, 140 MB/s 2024-04-25T00:04:17.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.403 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641525 s, 163 MB/s 2024-04-25T00:04:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646699 s, 162 MB/s 2024-04-25T00:04:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679238 s, 154 MB/s 2024-04-25T00:04:17.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.429 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075374 s, 139 MB/s 2024-04-25T00:04:17.437 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.437 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.437 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664482 s, 158 MB/s 2024-04-25T00:04:17.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.446 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716823 s, 146 MB/s 2024-04-25T00:04:17.455 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.455 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.455 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716895 s, 146 MB/s 2024-04-25T00:04:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654585 s, 160 MB/s 2024-04-25T00:04:17.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784173 s, 134 MB/s 2024-04-25T00:04:17.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.480 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578541 s, 181 MB/s 2024-04-25T00:04:17.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.488 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706731 s, 148 MB/s 2024-04-25T00:04:17.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.498 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801599 s, 131 MB/s 2024-04-25T00:04:17.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.505 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570882 s, 184 MB/s 2024-04-25T00:04:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699898 s, 150 MB/s 2024-04-25T00:04:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.523 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717716 s, 146 MB/s 2024-04-25T00:04:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.531 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662117 s, 158 MB/s 2024-04-25T00:04:17.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.538 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602246 s, 174 MB/s 2024-04-25T00:04:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715878 s, 146 MB/s 2024-04-25T00:04:17.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.556 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729363 s, 144 MB/s 2024-04-25T00:04:17.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.563 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575084 s, 182 MB/s 2024-04-25T00:04:17.571 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.571 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672812 s, 156 MB/s 2024-04-25T00:04:17.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.580 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709927 s, 148 MB/s 2024-04-25T00:04:17.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.589 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764189 s, 137 MB/s 2024-04-25T00:04:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681564 s, 154 MB/s 2024-04-25T00:04:17.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666326 s, 157 MB/s 2024-04-25T00:04:17.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.613 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057338 s, 183 MB/s 2024-04-25T00:04:17.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717103 s, 146 MB/s 2024-04-25T00:04:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568708 s, 184 MB/s 2024-04-25T00:04:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568537 s, 184 MB/s 2024-04-25T00:04:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:17.642 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:17.642 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:17.643 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T00:04:17.643 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:17.643 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:17.643 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:17.643 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:17.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:17.701 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.701 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.701 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737681 s, 142 MB/s 2024-04-25T00:04:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720023 s, 146 MB/s 2024-04-25T00:04:17.717 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.717 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.717 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564533 s, 186 MB/s 2024-04-25T00:04:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576867 s, 182 MB/s 2024-04-25T00:04:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752023 s, 139 MB/s 2024-04-25T00:04:17.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.741 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582759 s, 180 MB/s 2024-04-25T00:04:17.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.748 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590124 s, 178 MB/s 2024-04-25T00:04:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.756 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559653 s, 187 MB/s 2024-04-25T00:04:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.764 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721044 s, 145 MB/s 2024-04-25T00:04:17.773 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766801 s, 137 MB/s 2024-04-25T00:04:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687055 s, 153 MB/s 2024-04-25T00:04:17.791 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.791 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.791 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761691 s, 138 MB/s 2024-04-25T00:04:17.800 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.800 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.800 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660582 s, 159 MB/s 2024-04-25T00:04:17.806 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.807 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.807 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586327 s, 179 MB/s 2024-04-25T00:04:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923005 s, 114 MB/s 2024-04-25T00:04:17.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.827 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761422 s, 138 MB/s 2024-04-25T00:04:17.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674124 s, 156 MB/s 2024-04-25T00:04:17.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.845 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825806 s, 127 MB/s 2024-04-25T00:04:17.854 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.854 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.854 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735425 s, 143 MB/s 2024-04-25T00:04:17.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.863 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076686 s, 137 MB/s 2024-04-25T00:04:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567318 s, 185 MB/s 2024-04-25T00:04:17.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717547 s, 146 MB/s 2024-04-25T00:04:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.887 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678094 s, 155 MB/s 2024-04-25T00:04:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662292 s, 158 MB/s 2024-04-25T00:04:17.904 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.904 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.904 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663437 s, 158 MB/s 2024-04-25T00:04:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.912 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672032 s, 156 MB/s 2024-04-25T00:04:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570303 s, 184 MB/s 2024-04-25T00:04:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.929 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.929 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782601 s, 134 MB/s 2024-04-25T00:04:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775498 s, 135 MB/s 2024-04-25T00:04:17.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.947 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734204 s, 143 MB/s 2024-04-25T00:04:17.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.956 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732468 s, 143 MB/s 2024-04-25T00:04:17.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:17.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:17.963 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561611 s, 187 MB/s 2024-04-25T00:04:17.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:17.969 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:17.970 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:17.970 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T00:04:17.970 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:17.970 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:17.970 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:17.970 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:18.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:18.028 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751597 s, 140 MB/s 2024-04-25T00:04:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727874 s, 144 MB/s 2024-04-25T00:04:18.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.046 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720929 s, 145 MB/s 2024-04-25T00:04:18.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.054 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669521 s, 157 MB/s 2024-04-25T00:04:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066733 s, 157 MB/s 2024-04-25T00:04:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712361 s, 147 MB/s 2024-04-25T00:04:18.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691069 s, 152 MB/s 2024-04-25T00:04:18.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.088 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070656 s, 148 MB/s 2024-04-25T00:04:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666666 s, 157 MB/s 2024-04-25T00:04:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.105 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687724 s, 152 MB/s 2024-04-25T00:04:18.113 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657991 s, 159 MB/s 2024-04-25T00:04:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674062 s, 156 MB/s 2024-04-25T00:04:18.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.129 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576016 s, 182 MB/s 2024-04-25T00:04:18.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.137 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711362 s, 147 MB/s 2024-04-25T00:04:18.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.146 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659871 s, 159 MB/s 2024-04-25T00:04:18.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.154 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695805 s, 151 MB/s 2024-04-25T00:04:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655326 s, 160 MB/s 2024-04-25T00:04:18.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.178 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013459 s, 77.9 MB/s 2024-04-25T00:04:18.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.186 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714655 s, 147 MB/s 2024-04-25T00:04:18.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.195 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702246 s, 149 MB/s 2024-04-25T00:04:18.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.204 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733504 s, 143 MB/s 2024-04-25T00:04:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716612 s, 146 MB/s 2024-04-25T00:04:18.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.220 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581616 s, 180 MB/s 2024-04-25T00:04:18.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.229 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.229 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716462 s, 146 MB/s 2024-04-25T00:04:18.236 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.236 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.236 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638953 s, 164 MB/s 2024-04-25T00:04:18.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.244 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557935 s, 188 MB/s 2024-04-25T00:04:18.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.252 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715601 s, 147 MB/s 2024-04-25T00:04:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606665 s, 173 MB/s 2024-04-25T00:04:18.268 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.268 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.268 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611155 s, 172 MB/s 2024-04-25T00:04:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637597 s, 164 MB/s 2024-04-25T00:04:18.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.284 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684249 s, 153 MB/s 2024-04-25T00:04:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654706 s, 160 MB/s 2024-04-25T00:04:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:18.299 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:18.299 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:18.299 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T00:04:18.299 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:18.299 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:18.300 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:18.300 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:18.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:18.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.358 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721145 s, 145 MB/s 2024-04-25T00:04:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819985 s, 128 MB/s 2024-04-25T00:04:18.374 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.375 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554273 s, 189 MB/s 2024-04-25T00:04:18.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.384 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754193 s, 139 MB/s 2024-04-25T00:04:18.392 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.392 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.392 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668761 s, 157 MB/s 2024-04-25T00:04:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712672 s, 147 MB/s 2024-04-25T00:04:18.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.409 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672551 s, 156 MB/s 2024-04-25T00:04:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065935 s, 159 MB/s 2024-04-25T00:04:18.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.426 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732687 s, 143 MB/s 2024-04-25T00:04:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705475 s, 149 MB/s 2024-04-25T00:04:18.444 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.444 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.444 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748847 s, 140 MB/s 2024-04-25T00:04:18.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565569 s, 185 MB/s 2024-04-25T00:04:18.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.459 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656408 s, 160 MB/s 2024-04-25T00:04:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.467 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063635 s, 165 MB/s 2024-04-25T00:04:18.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.475 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661474 s, 159 MB/s 2024-04-25T00:04:18.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.484 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729721 s, 144 MB/s 2024-04-25T00:04:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.492 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574442 s, 183 MB/s 2024-04-25T00:04:18.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.499 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581421 s, 180 MB/s 2024-04-25T00:04:18.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.508 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751514 s, 140 MB/s 2024-04-25T00:04:18.515 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.515 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.515 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548608 s, 191 MB/s 2024-04-25T00:04:18.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.523 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557148 s, 188 MB/s 2024-04-25T00:04:18.529 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.529 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.530 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560218 s, 187 MB/s 2024-04-25T00:04:18.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.538 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683972 s, 153 MB/s 2024-04-25T00:04:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072487 s, 145 MB/s 2024-04-25T00:04:18.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.555 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641434 s, 163 MB/s 2024-04-25T00:04:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649475 s, 161 MB/s 2024-04-25T00:04:18.570 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.570 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.570 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577016 s, 182 MB/s 2024-04-25T00:04:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728221 s, 144 MB/s 2024-04-25T00:04:18.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.588 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754817 s, 139 MB/s 2024-04-25T00:04:18.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706034 s, 149 MB/s 2024-04-25T00:04:18.604 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.604 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.604 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571589 s, 183 MB/s 2024-04-25T00:04:18.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.613 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706841 s, 148 MB/s 2024-04-25T00:04:18.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:18.619 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:18.620 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:18.620 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T00:04:18.620 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:18.620 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:18.620 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:18.620 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:18.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071226 s, 147 MB/s 2024-04-25T00:04:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641926 s, 163 MB/s 2024-04-25T00:04:18.693 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.693 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.693 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577595 s, 182 MB/s 2024-04-25T00:04:18.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569405 s, 184 MB/s 2024-04-25T00:04:18.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.709 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.709 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650851 s, 161 MB/s 2024-04-25T00:04:18.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.716 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578657 s, 181 MB/s 2024-04-25T00:04:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078194 s, 134 MB/s 2024-04-25T00:04:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.735 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742755 s, 141 MB/s 2024-04-25T00:04:18.742 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.742 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.742 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631926 s, 166 MB/s 2024-04-25T00:04:18.749 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.749 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.750 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575863 s, 182 MB/s 2024-04-25T00:04:18.759 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.759 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.759 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812811 s, 129 MB/s 2024-04-25T00:04:18.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.768 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707157 s, 148 MB/s 2024-04-25T00:04:18.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.777 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715229 s, 147 MB/s 2024-04-25T00:04:18.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.785 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653266 s, 161 MB/s 2024-04-25T00:04:18.793 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.793 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.793 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694639 s, 151 MB/s 2024-04-25T00:04:18.800 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.801 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585903 s, 179 MB/s 2024-04-25T00:04:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791333 s, 133 MB/s 2024-04-25T00:04:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666117 s, 157 MB/s 2024-04-25T00:04:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747485 s, 140 MB/s 2024-04-25T00:04:18.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569028 s, 184 MB/s 2024-04-25T00:04:18.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.843 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639376 s, 164 MB/s 2024-04-25T00:04:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772442 s, 136 MB/s 2024-04-25T00:04:18.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.861 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071658 s, 146 MB/s 2024-04-25T00:04:18.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.868 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562562 s, 186 MB/s 2024-04-25T00:04:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556432 s, 188 MB/s 2024-04-25T00:04:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742643 s, 141 MB/s 2024-04-25T00:04:18.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.893 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713938 s, 147 MB/s 2024-04-25T00:04:18.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.902 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069421 s, 151 MB/s 2024-04-25T00:04:18.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.910 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644942 s, 163 MB/s 2024-04-25T00:04:18.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.919 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733227 s, 143 MB/s 2024-04-25T00:04:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760151 s, 138 MB/s 2024-04-25T00:04:18.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:18.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:18.937 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738472 s, 142 MB/s 2024-04-25T00:04:18.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:18.943 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:18.943 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:18.943 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T00:04:18.944 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:18.944 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:18.944 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:18.944 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:18.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740129 s, 142 MB/s 2024-04-25T00:04:19.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.011 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.011 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074163 s, 141 MB/s 2024-04-25T00:04:19.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.019 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715375 s, 147 MB/s 2024-04-25T00:04:19.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.027 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639233 s, 164 MB/s 2024-04-25T00:04:19.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.036 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714582 s, 147 MB/s 2024-04-25T00:04:19.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.044 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642258 s, 163 MB/s 2024-04-25T00:04:19.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.052 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654397 s, 160 MB/s 2024-04-25T00:04:19.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.060 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643936 s, 163 MB/s 2024-04-25T00:04:19.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.068 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646672 s, 162 MB/s 2024-04-25T00:04:19.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.077 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659806 s, 159 MB/s 2024-04-25T00:04:19.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.086 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748037 s, 140 MB/s 2024-04-25T00:04:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652506 s, 161 MB/s 2024-04-25T00:04:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745777 s, 141 MB/s 2024-04-25T00:04:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729834 s, 144 MB/s 2024-04-25T00:04:19.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.121 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741551 s, 141 MB/s 2024-04-25T00:04:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070468 s, 149 MB/s 2024-04-25T00:04:19.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.136 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575537 s, 182 MB/s 2024-04-25T00:04:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743222 s, 141 MB/s 2024-04-25T00:04:19.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.154 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728846 s, 144 MB/s 2024-04-25T00:04:19.163 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.163 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.163 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736075 s, 142 MB/s 2024-04-25T00:04:19.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.172 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751252 s, 140 MB/s 2024-04-25T00:04:19.181 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.182 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795296 s, 132 MB/s 2024-04-25T00:04:19.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.190 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698085 s, 150 MB/s 2024-04-25T00:04:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.199 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763599 s, 137 MB/s 2024-04-25T00:04:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.208 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731649 s, 143 MB/s 2024-04-25T00:04:19.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.217 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725723 s, 144 MB/s 2024-04-25T00:04:19.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.225 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647838 s, 162 MB/s 2024-04-25T00:04:19.234 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.234 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.234 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758889 s, 138 MB/s 2024-04-25T00:04:19.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.242 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702021 s, 149 MB/s 2024-04-25T00:04:19.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758821 s, 138 MB/s 2024-04-25T00:04:19.259 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.259 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.259 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588137 s, 178 MB/s 2024-04-25T00:04:19.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.267 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643717 s, 163 MB/s 2024-04-25T00:04:19.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:19.274 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:19.274 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:19.274 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T00:04:19.274 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:19.274 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:19.274 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:19.274 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:19.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:19.331 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.331 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.331 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639437 s, 164 MB/s 2024-04-25T00:04:19.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.340 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737412 s, 142 MB/s 2024-04-25T00:04:19.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.349 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071505 s, 147 MB/s 2024-04-25T00:04:19.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.356 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057936 s, 181 MB/s 2024-04-25T00:04:19.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.363 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555856 s, 189 MB/s 2024-04-25T00:04:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770827 s, 136 MB/s 2024-04-25T00:04:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558978 s, 188 MB/s 2024-04-25T00:04:19.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.388 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007126 s, 147 MB/s 2024-04-25T00:04:19.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.396 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058682 s, 179 MB/s 2024-04-25T00:04:19.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.404 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645449 s, 162 MB/s 2024-04-25T00:04:19.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634089 s, 165 MB/s 2024-04-25T00:04:19.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.421 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733938 s, 143 MB/s 2024-04-25T00:04:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709151 s, 148 MB/s 2024-04-25T00:04:19.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.438 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725997 s, 144 MB/s 2024-04-25T00:04:19.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706128 s, 148 MB/s 2024-04-25T00:04:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055956 s, 187 MB/s 2024-04-25T00:04:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672179 s, 156 MB/s 2024-04-25T00:04:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056772 s, 185 MB/s 2024-04-25T00:04:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729272 s, 144 MB/s 2024-04-25T00:04:19.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.487 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722123 s, 145 MB/s 2024-04-25T00:04:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.496 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739802 s, 142 MB/s 2024-04-25T00:04:19.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.504 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648851 s, 162 MB/s 2024-04-25T00:04:19.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.511 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569051 s, 184 MB/s 2024-04-25T00:04:19.520 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.521 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.521 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742547 s, 141 MB/s 2024-04-25T00:04:19.533 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.533 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.533 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108637 s, 96.5 MB/s 2024-04-25T00:04:19.542 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.542 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.542 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715298 s, 147 MB/s 2024-04-25T00:04:19.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.549 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562712 s, 186 MB/s 2024-04-25T00:04:19.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.558 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715328 s, 147 MB/s 2024-04-25T00:04:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563978 s, 186 MB/s 2024-04-25T00:04:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739876 s, 142 MB/s 2024-04-25T00:04:19.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.582 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057732 s, 182 MB/s 2024-04-25T00:04:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735084 s, 143 MB/s 2024-04-25T00:04:19.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:19.597 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:19.597 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:19.597 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T00:04:19.597 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:19.598 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:19.598 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:19.598 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:19.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:19.655 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.655 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748418 s, 140 MB/s 2024-04-25T00:04:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.664 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691814 s, 152 MB/s 2024-04-25T00:04:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592806 s, 177 MB/s 2024-04-25T00:04:19.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054111 s, 194 MB/s 2024-04-25T00:04:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.687 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692929 s, 151 MB/s 2024-04-25T00:04:19.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.699 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811267 s, 129 MB/s 2024-04-25T00:04:19.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.705 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716041 s, 146 MB/s 2024-04-25T00:04:19.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.713 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652493 s, 161 MB/s 2024-04-25T00:04:19.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708974 s, 148 MB/s 2024-04-25T00:04:19.729 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.729 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.729 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568916 s, 184 MB/s 2024-04-25T00:04:19.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.738 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699828 s, 150 MB/s 2024-04-25T00:04:19.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.746 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659459 s, 159 MB/s 2024-04-25T00:04:19.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.756 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729319 s, 144 MB/s 2024-04-25T00:04:19.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.762 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580535 s, 181 MB/s 2024-04-25T00:04:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708926 s, 148 MB/s 2024-04-25T00:04:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661497 s, 159 MB/s 2024-04-25T00:04:19.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.787 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580479 s, 181 MB/s 2024-04-25T00:04:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.796 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732679 s, 143 MB/s 2024-04-25T00:04:19.804 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.804 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.804 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581216 s, 180 MB/s 2024-04-25T00:04:19.811 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.811 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.811 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686628 s, 153 MB/s 2024-04-25T00:04:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782173 s, 134 MB/s 2024-04-25T00:04:19.829 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.829 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.829 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652619 s, 161 MB/s 2024-04-25T00:04:19.838 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.838 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.838 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748411 s, 140 MB/s 2024-04-25T00:04:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064557 s, 162 MB/s 2024-04-25T00:04:19.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.855 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760861 s, 138 MB/s 2024-04-25T00:04:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.864 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076643 s, 137 MB/s 2024-04-25T00:04:19.872 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.872 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.872 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658595 s, 159 MB/s 2024-04-25T00:04:19.880 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.880 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.880 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635572 s, 165 MB/s 2024-04-25T00:04:19.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.889 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728198 s, 144 MB/s 2024-04-25T00:04:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735127 s, 143 MB/s 2024-04-25T00:04:19.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.906 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653672 s, 160 MB/s 2024-04-25T00:04:19.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.915 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677404 s, 155 MB/s 2024-04-25T00:04:19.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:19.921 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:19.921 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:19.921 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T00:04:19.921 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:19.921 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:19.921 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:19.921 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.981 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647237 s, 162 MB/s 2024-04-25T00:04:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071437 s, 147 MB/s 2024-04-25T00:04:19.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:19.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:19.996 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714106 s, 147 MB/s 2024-04-25T00:04:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749705 s, 140 MB/s 2024-04-25T00:04:20.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.013 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650418 s, 161 MB/s 2024-04-25T00:04:20.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.022 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756119 s, 139 MB/s 2024-04-25T00:04:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561274 s, 187 MB/s 2024-04-25T00:04:20.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701328 s, 150 MB/s 2024-04-25T00:04:20.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.047 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724328 s, 145 MB/s 2024-04-25T00:04:20.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.054 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579545 s, 181 MB/s 2024-04-25T00:04:20.063 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.063 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.063 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759708 s, 138 MB/s 2024-04-25T00:04:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735055 s, 143 MB/s 2024-04-25T00:04:20.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.081 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074792 s, 140 MB/s 2024-04-25T00:04:20.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.090 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708621 s, 148 MB/s 2024-04-25T00:04:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693841 s, 151 MB/s 2024-04-25T00:04:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716721 s, 146 MB/s 2024-04-25T00:04:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.115 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683131 s, 153 MB/s 2024-04-25T00:04:20.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.124 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736632 s, 142 MB/s 2024-04-25T00:04:20.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.132 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572833 s, 183 MB/s 2024-04-25T00:04:20.141 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.141 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.141 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738567 s, 142 MB/s 2024-04-25T00:04:20.149 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710753 s, 148 MB/s 2024-04-25T00:04:20.157 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.158 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638367 s, 164 MB/s 2024-04-25T00:04:20.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.167 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733842 s, 143 MB/s 2024-04-25T00:04:20.174 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.175 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.175 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660456 s, 159 MB/s 2024-04-25T00:04:20.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.184 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762655 s, 137 MB/s 2024-04-25T00:04:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700049 s, 150 MB/s 2024-04-25T00:04:20.201 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.201 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.201 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702957 s, 149 MB/s 2024-04-25T00:04:20.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.208 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641425 s, 163 MB/s 2024-04-25T00:04:20.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.217 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693604 s, 151 MB/s 2024-04-25T00:04:20.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.228 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873457 s, 120 MB/s 2024-04-25T00:04:20.236 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.236 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.236 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694968 s, 151 MB/s 2024-04-25T00:04:20.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.243 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614364 s, 171 MB/s 2024-04-25T00:04:20.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:20.250 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:20.250 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:20.250 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T00:04:20.250 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:20.250 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:20.250 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:20.250 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:20.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.309 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078155 s, 134 MB/s 2024-04-25T00:04:20.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586418 s, 179 MB/s 2024-04-25T00:04:20.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.324 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564184 s, 186 MB/s 2024-04-25T00:04:20.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.332 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682946 s, 154 MB/s 2024-04-25T00:04:20.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.341 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682008 s, 154 MB/s 2024-04-25T00:04:20.349 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742574 s, 141 MB/s 2024-04-25T00:04:20.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.359 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.359 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740777 s, 142 MB/s 2024-04-25T00:04:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574043 s, 183 MB/s 2024-04-25T00:04:20.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620894 s, 169 MB/s 2024-04-25T00:04:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562753 s, 186 MB/s 2024-04-25T00:04:20.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.388 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559335 s, 187 MB/s 2024-04-25T00:04:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562222 s, 187 MB/s 2024-04-25T00:04:20.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.404 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066302 s, 158 MB/s 2024-04-25T00:04:20.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.411 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580611 s, 181 MB/s 2024-04-25T00:04:20.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.420 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756293 s, 139 MB/s 2024-04-25T00:04:20.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.429 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730099 s, 144 MB/s 2024-04-25T00:04:20.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.436 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558819 s, 188 MB/s 2024-04-25T00:04:20.445 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.445 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.445 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696744 s, 150 MB/s 2024-04-25T00:04:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728892 s, 144 MB/s 2024-04-25T00:04:20.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.463 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714451 s, 147 MB/s 2024-04-25T00:04:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068111 s, 154 MB/s 2024-04-25T00:04:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723132 s, 145 MB/s 2024-04-25T00:04:20.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.487 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557038 s, 188 MB/s 2024-04-25T00:04:20.496 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.496 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.496 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749724 s, 140 MB/s 2024-04-25T00:04:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073675 s, 142 MB/s 2024-04-25T00:04:20.512 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.512 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.512 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557979 s, 188 MB/s 2024-04-25T00:04:20.520 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.520 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.520 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632392 s, 166 MB/s 2024-04-25T00:04:20.528 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.529 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.529 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712508 s, 147 MB/s 2024-04-25T00:04:20.537 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.537 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.537 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661244 s, 159 MB/s 2024-04-25T00:04:20.545 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.545 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.546 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718209 s, 146 MB/s 2024-04-25T00:04:20.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.556 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651257 s, 161 MB/s 2024-04-25T00:04:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641189 s, 164 MB/s 2024-04-25T00:04:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:20.568 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:20.568 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:20.568 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T00:04:20.569 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:20.569 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:20.569 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:20.569 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:20.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:20.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.626 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665567 s, 158 MB/s 2024-04-25T00:04:20.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.635 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727489 s, 144 MB/s 2024-04-25T00:04:20.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.642 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575523 s, 182 MB/s 2024-04-25T00:04:20.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.650 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657828 s, 159 MB/s 2024-04-25T00:04:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005562 s, 189 MB/s 2024-04-25T00:04:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668996 s, 157 MB/s 2024-04-25T00:04:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641338 s, 163 MB/s 2024-04-25T00:04:20.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.683 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696535 s, 151 MB/s 2024-04-25T00:04:20.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.692 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740623 s, 142 MB/s 2024-04-25T00:04:20.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652404 s, 161 MB/s 2024-04-25T00:04:20.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.708 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671134 s, 156 MB/s 2024-04-25T00:04:20.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.717 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.717 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705095 s, 149 MB/s 2024-04-25T00:04:20.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.725 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707604 s, 148 MB/s 2024-04-25T00:04:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722149 s, 145 MB/s 2024-04-25T00:04:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698775 s, 150 MB/s 2024-04-25T00:04:20.751 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.751 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.751 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725377 s, 145 MB/s 2024-04-25T00:04:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.760 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733012 s, 143 MB/s 2024-04-25T00:04:20.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.768 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652843 s, 161 MB/s 2024-04-25T00:04:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661995 s, 158 MB/s 2024-04-25T00:04:20.783 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.784 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578787 s, 181 MB/s 2024-04-25T00:04:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646945 s, 162 MB/s 2024-04-25T00:04:20.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.801 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777553 s, 135 MB/s 2024-04-25T00:04:20.809 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.809 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.809 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646681 s, 162 MB/s 2024-04-25T00:04:20.817 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.817 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.817 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643521 s, 163 MB/s 2024-04-25T00:04:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664941 s, 158 MB/s 2024-04-25T00:04:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740973 s, 142 MB/s 2024-04-25T00:04:20.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.843 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721272 s, 145 MB/s 2024-04-25T00:04:20.854 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.854 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.854 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752103 s, 139 MB/s 2024-04-25T00:04:20.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.861 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711965 s, 147 MB/s 2024-04-25T00:04:20.869 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695766 s, 151 MB/s 2024-04-25T00:04:20.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.877 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579185 s, 181 MB/s 2024-04-25T00:04:20.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.885 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668284 s, 157 MB/s 2024-04-25T00:04:20.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:20.892 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:20.892 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:20.892 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T00:04:20.892 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:20.892 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:20.892 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:20.892 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:20.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:20.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.950 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688626 s, 152 MB/s 2024-04-25T00:04:20.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.958 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647215 s, 162 MB/s 2024-04-25T00:04:20.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.965 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571262 s, 184 MB/s 2024-04-25T00:04:20.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.973 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654909 s, 160 MB/s 2024-04-25T00:04:20.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.981 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651282 s, 161 MB/s 2024-04-25T00:04:20.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557365 s, 188 MB/s 2024-04-25T00:04:20.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:20.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:20.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713649 s, 147 MB/s 2024-04-25T00:04:21.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751469 s, 140 MB/s 2024-04-25T00:04:21.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.015 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710788 s, 148 MB/s 2024-04-25T00:04:21.024 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.024 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.024 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726178 s, 144 MB/s 2024-04-25T00:04:21.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.033 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750612 s, 140 MB/s 2024-04-25T00:04:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712709 s, 147 MB/s 2024-04-25T00:04:21.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.051 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763094 s, 137 MB/s 2024-04-25T00:04:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692279 s, 151 MB/s 2024-04-25T00:04:21.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.067 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635051 s, 165 MB/s 2024-04-25T00:04:21.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.075 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566144 s, 185 MB/s 2024-04-25T00:04:21.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.083 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073816 s, 142 MB/s 2024-04-25T00:04:21.091 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.091 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.091 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627722 s, 167 MB/s 2024-04-25T00:04:21.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.099 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689139 s, 152 MB/s 2024-04-25T00:04:21.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.107 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005771 s, 182 MB/s 2024-04-25T00:04:21.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.115 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658917 s, 159 MB/s 2024-04-25T00:04:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723405 s, 145 MB/s 2024-04-25T00:04:21.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.132 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676474 s, 155 MB/s 2024-04-25T00:04:21.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589638 s, 178 MB/s 2024-04-25T00:04:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664861 s, 158 MB/s 2024-04-25T00:04:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738226 s, 142 MB/s 2024-04-25T00:04:21.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.164 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570714 s, 184 MB/s 2024-04-25T00:04:21.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.172 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623025 s, 168 MB/s 2024-04-25T00:04:21.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072648 s, 144 MB/s 2024-04-25T00:04:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647498 s, 162 MB/s 2024-04-25T00:04:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654132 s, 160 MB/s 2024-04-25T00:04:21.205 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.205 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.205 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654389 s, 160 MB/s 2024-04-25T00:04:21.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:21.212 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:21.212 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:21.212 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T00:04:21.213 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:21.213 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:21.213 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:21.213 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:21.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:21.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.272 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719902 s, 146 MB/s 2024-04-25T00:04:21.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.279 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060038 s, 175 MB/s 2024-04-25T00:04:21.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.288 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662702 s, 158 MB/s 2024-04-25T00:04:21.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677426 s, 155 MB/s 2024-04-25T00:04:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701994 s, 149 MB/s 2024-04-25T00:04:21.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680116 s, 154 MB/s 2024-04-25T00:04:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709195 s, 148 MB/s 2024-04-25T00:04:21.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.330 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707956 s, 148 MB/s 2024-04-25T00:04:21.336 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.337 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550889 s, 190 MB/s 2024-04-25T00:04:21.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.346 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726337 s, 144 MB/s 2024-04-25T00:04:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715155 s, 147 MB/s 2024-04-25T00:04:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573234 s, 183 MB/s 2024-04-25T00:04:21.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724922 s, 145 MB/s 2024-04-25T00:04:21.377 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.377 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.377 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056546 s, 185 MB/s 2024-04-25T00:04:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722631 s, 145 MB/s 2024-04-25T00:04:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696425 s, 151 MB/s 2024-04-25T00:04:21.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.403 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662704 s, 158 MB/s 2024-04-25T00:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.412 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681231 s, 154 MB/s 2024-04-25T00:04:21.419 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.419 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.419 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554243 s, 189 MB/s 2024-04-25T00:04:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563707 s, 186 MB/s 2024-04-25T00:04:21.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.434 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640593 s, 164 MB/s 2024-04-25T00:04:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722902 s, 145 MB/s 2024-04-25T00:04:21.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635003 s, 165 MB/s 2024-04-25T00:04:21.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.459 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664334 s, 158 MB/s 2024-04-25T00:04:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542162 s, 193 MB/s 2024-04-25T00:04:21.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.475 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691828 s, 152 MB/s 2024-04-25T00:04:21.481 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.482 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.482 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568837 s, 184 MB/s 2024-04-25T00:04:21.490 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.490 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.490 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710511 s, 148 MB/s 2024-04-25T00:04:21.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724068 s, 145 MB/s 2024-04-25T00:04:21.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632533 s, 166 MB/s 2024-04-25T00:04:21.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.516 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707394 s, 148 MB/s 2024-04-25T00:04:21.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.523 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580749 s, 181 MB/s 2024-04-25T00:04:21.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:21.529 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:21.530 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:21.530 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T00:04:21.530 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:21.530 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:21.530 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:21.530 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:21.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:21.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688215 s, 152 MB/s 2024-04-25T00:04:21.595 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.595 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.595 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554079 s, 189 MB/s 2024-04-25T00:04:21.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.604 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697213 s, 150 MB/s 2024-04-25T00:04:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055317 s, 190 MB/s 2024-04-25T00:04:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732237 s, 143 MB/s 2024-04-25T00:04:21.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.628 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645798 s, 162 MB/s 2024-04-25T00:04:21.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722046 s, 145 MB/s 2024-04-25T00:04:21.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.645 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663508 s, 158 MB/s 2024-04-25T00:04:21.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659526 s, 159 MB/s 2024-04-25T00:04:21.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.661 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718893 s, 146 MB/s 2024-04-25T00:04:21.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711451 s, 147 MB/s 2024-04-25T00:04:21.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669035 s, 157 MB/s 2024-04-25T00:04:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718248 s, 146 MB/s 2024-04-25T00:04:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701771 s, 149 MB/s 2024-04-25T00:04:21.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.702 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556514 s, 188 MB/s 2024-04-25T00:04:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067551 s, 155 MB/s 2024-04-25T00:04:21.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.719 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584352 s, 179 MB/s 2024-04-25T00:04:21.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.727 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692665 s, 151 MB/s 2024-04-25T00:04:21.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.735 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667358 s, 157 MB/s 2024-04-25T00:04:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649375 s, 161 MB/s 2024-04-25T00:04:21.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.750 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584137 s, 180 MB/s 2024-04-25T00:04:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659815 s, 159 MB/s 2024-04-25T00:04:21.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.767 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723604 s, 145 MB/s 2024-04-25T00:04:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733879 s, 143 MB/s 2024-04-25T00:04:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557563 s, 188 MB/s 2024-04-25T00:04:21.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.790 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554855 s, 189 MB/s 2024-04-25T00:04:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674863 s, 155 MB/s 2024-04-25T00:04:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057114 s, 184 MB/s 2024-04-25T00:04:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071726 s, 146 MB/s 2024-04-25T00:04:21.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.822 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556732 s, 188 MB/s 2024-04-25T00:04:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765667 s, 137 MB/s 2024-04-25T00:04:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736514 s, 142 MB/s 2024-04-25T00:04:21.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:21.847 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:21.847 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:21.847 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T00:04:21.847 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:21.847 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:21.847 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:21.847 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:21.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:21.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.916 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716001 s, 146 MB/s 2024-04-25T00:04:21.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.923 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657159 s, 160 MB/s 2024-04-25T00:04:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753049 s, 139 MB/s 2024-04-25T00:04:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750176 s, 140 MB/s 2024-04-25T00:04:21.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.952 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826838 s, 127 MB/s 2024-04-25T00:04:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746595 s, 140 MB/s 2024-04-25T00:04:21.970 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.970 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.970 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756114 s, 139 MB/s 2024-04-25T00:04:21.979 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.979 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.979 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719935 s, 146 MB/s 2024-04-25T00:04:21.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729421 s, 144 MB/s 2024-04-25T00:04:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:21.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670598 s, 156 MB/s 2024-04-25T00:04:22.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737216 s, 142 MB/s 2024-04-25T00:04:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705268 s, 149 MB/s 2024-04-25T00:04:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666935 s, 157 MB/s 2024-04-25T00:04:22.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.031 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587015 s, 179 MB/s 2024-04-25T00:04:22.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.039 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637568 s, 164 MB/s 2024-04-25T00:04:22.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.048 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773833 s, 136 MB/s 2024-04-25T00:04:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.057 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708109 s, 148 MB/s 2024-04-25T00:04:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701902 s, 149 MB/s 2024-04-25T00:04:22.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.074 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729157 s, 144 MB/s 2024-04-25T00:04:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.083 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718295 s, 146 MB/s 2024-04-25T00:04:22.091 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.092 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.092 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752059 s, 139 MB/s 2024-04-25T00:04:22.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.101 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.101 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749976 s, 140 MB/s 2024-04-25T00:04:22.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.109 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065229 s, 161 MB/s 2024-04-25T00:04:22.118 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.118 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.118 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769664 s, 136 MB/s 2024-04-25T00:04:22.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639706 s, 164 MB/s 2024-04-25T00:04:22.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.134 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589515 s, 178 MB/s 2024-04-25T00:04:22.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.144 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783554 s, 134 MB/s 2024-04-25T00:04:22.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.152 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634697 s, 165 MB/s 2024-04-25T00:04:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668404 s, 157 MB/s 2024-04-25T00:04:22.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.171 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791385 s, 132 MB/s 2024-04-25T00:04:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729829 s, 144 MB/s 2024-04-25T00:04:22.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.189 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754382 s, 139 MB/s 2024-04-25T00:04:22.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:22.196 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:22.196 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:22.196 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T00:04:22.196 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:22.196 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:22.196 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:22.196 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:22.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:22.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.254 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682809 s, 154 MB/s 2024-04-25T00:04:22.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.262 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607969 s, 172 MB/s 2024-04-25T00:04:22.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.271 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721858 s, 145 MB/s 2024-04-25T00:04:22.280 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.280 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.280 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652444 s, 161 MB/s 2024-04-25T00:04:22.290 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.291 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654141 s, 160 MB/s 2024-04-25T00:04:22.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.298 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.298 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722916 s, 145 MB/s 2024-04-25T00:04:22.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.306 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006606 s, 159 MB/s 2024-04-25T00:04:22.315 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.315 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742941 s, 141 MB/s 2024-04-25T00:04:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596034 s, 176 MB/s 2024-04-25T00:04:22.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.333 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757845 s, 138 MB/s 2024-04-25T00:04:22.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673531 s, 156 MB/s 2024-04-25T00:04:22.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.351 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709104 s, 148 MB/s 2024-04-25T00:04:22.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.360 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709092 s, 148 MB/s 2024-04-25T00:04:22.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.369 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723779 s, 145 MB/s 2024-04-25T00:04:22.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.379 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071529 s, 147 MB/s 2024-04-25T00:04:22.385 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.385 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.386 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563134 s, 186 MB/s 2024-04-25T00:04:22.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696427 s, 151 MB/s 2024-04-25T00:04:22.402 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.402 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.405 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580128 s, 181 MB/s 2024-04-25T00:04:22.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.412 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719246 s, 146 MB/s 2024-04-25T00:04:22.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.420 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659813 s, 159 MB/s 2024-04-25T00:04:22.427 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.428 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580154 s, 181 MB/s 2024-04-25T00:04:22.437 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.437 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.437 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738822 s, 142 MB/s 2024-04-25T00:04:22.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.446 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720074 s, 146 MB/s 2024-04-25T00:04:22.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.454 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570512 s, 184 MB/s 2024-04-25T00:04:22.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.463 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736779 s, 142 MB/s 2024-04-25T00:04:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716324 s, 146 MB/s 2024-04-25T00:04:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589379 s, 178 MB/s 2024-04-25T00:04:22.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.488 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717086 s, 146 MB/s 2024-04-25T00:04:22.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.497 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733912 s, 143 MB/s 2024-04-25T00:04:22.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.504 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573433 s, 183 MB/s 2024-04-25T00:04:22.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.514 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706905 s, 148 MB/s 2024-04-25T00:04:22.521 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.521 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.521 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671094 s, 156 MB/s 2024-04-25T00:04:22.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:22.529 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:22.529 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:22.529 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T00:04:22.529 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:22.529 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:22.529 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:22.529 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:22.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.588 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730843 s, 143 MB/s 2024-04-25T00:04:22.596 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.596 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.596 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754129 s, 139 MB/s 2024-04-25T00:04:22.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719044 s, 146 MB/s 2024-04-25T00:04:22.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.614 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703855 s, 149 MB/s 2024-04-25T00:04:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719349 s, 146 MB/s 2024-04-25T00:04:22.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569149 s, 184 MB/s 2024-04-25T00:04:22.638 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.638 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.639 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721442 s, 145 MB/s 2024-04-25T00:04:22.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.647 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738225 s, 142 MB/s 2024-04-25T00:04:22.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.656 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721387 s, 145 MB/s 2024-04-25T00:04:22.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.664 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659381 s, 159 MB/s 2024-04-25T00:04:22.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.673 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688621 s, 152 MB/s 2024-04-25T00:04:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.682 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713493 s, 147 MB/s 2024-04-25T00:04:22.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.689 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630973 s, 166 MB/s 2024-04-25T00:04:22.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.698 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712021 s, 147 MB/s 2024-04-25T00:04:22.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.706 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652363 s, 161 MB/s 2024-04-25T00:04:22.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.715 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655411 s, 160 MB/s 2024-04-25T00:04:22.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.723 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747118 s, 140 MB/s 2024-04-25T00:04:22.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.732 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686276 s, 153 MB/s 2024-04-25T00:04:22.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.742 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.742 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007164 s, 146 MB/s 2024-04-25T00:04:22.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.748 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653381 s, 160 MB/s 2024-04-25T00:04:22.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.757 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661517 s, 159 MB/s 2024-04-25T00:04:22.764 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.764 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.764 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571053 s, 184 MB/s 2024-04-25T00:04:22.773 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.773 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.773 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753654 s, 139 MB/s 2024-04-25T00:04:22.781 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.782 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070223 s, 149 MB/s 2024-04-25T00:04:22.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.790 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643142 s, 163 MB/s 2024-04-25T00:04:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737549 s, 142 MB/s 2024-04-25T00:04:22.807 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.807 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.807 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728812 s, 144 MB/s 2024-04-25T00:04:22.815 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.815 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.815 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642692 s, 163 MB/s 2024-04-25T00:04:22.819 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:04:22.821 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:04:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577764 s, 181 MB/s 2024-04-25T00:04:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.832 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.832 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730188 s, 144 MB/s 2024-04-25T00:04:22.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.840 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690666 s, 152 MB/s 2024-04-25T00:04:22.848 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.848 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.848 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699148 s, 150 MB/s 2024-04-25T00:04:22.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:22.856 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:22.856 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:22.856 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T00:04:22.856 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:22.856 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:22.857 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:22.857 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:22.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.981 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576147 s, 182 MB/s 2024-04-25T00:04:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562722 s, 186 MB/s 2024-04-25T00:04:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666679 s, 157 MB/s 2024-04-25T00:04:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.004 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660929 s, 159 MB/s 2024-04-25T00:04:23.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.014 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780754 s, 134 MB/s 2024-04-25T00:04:23.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.022 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658373 s, 159 MB/s 2024-04-25T00:04:23.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.029 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566832 s, 185 MB/s 2024-04-25T00:04:23.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585141 s, 179 MB/s 2024-04-25T00:04:23.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.046 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755966 s, 139 MB/s 2024-04-25T00:04:23.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.055 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738474 s, 142 MB/s 2024-04-25T00:04:23.063 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.064 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072575 s, 144 MB/s 2024-04-25T00:04:23.073 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.073 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.073 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795066 s, 132 MB/s 2024-04-25T00:04:23.082 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.082 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.082 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715643 s, 147 MB/s 2024-04-25T00:04:23.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.090 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657224 s, 160 MB/s 2024-04-25T00:04:23.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.099 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723088 s, 145 MB/s 2024-04-25T00:04:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662648 s, 158 MB/s 2024-04-25T00:04:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570493 s, 184 MB/s 2024-04-25T00:04:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579061 s, 181 MB/s 2024-04-25T00:04:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007577 s, 138 MB/s 2024-04-25T00:04:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664972 s, 158 MB/s 2024-04-25T00:04:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649606 s, 161 MB/s 2024-04-25T00:04:23.155 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.155 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.156 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671855 s, 156 MB/s 2024-04-25T00:04:23.163 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.163 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.163 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658932 s, 159 MB/s 2024-04-25T00:04:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565949 s, 185 MB/s 2024-04-25T00:04:23.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.179 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751179 s, 140 MB/s 2024-04-25T00:04:23.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.189 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761725 s, 138 MB/s 2024-04-25T00:04:23.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.198 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747032 s, 140 MB/s 2024-04-25T00:04:23.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.206 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657806 s, 159 MB/s 2024-04-25T00:04:23.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.215 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.215 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710143 s, 148 MB/s 2024-04-25T00:04:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661819 s, 158 MB/s 2024-04-25T00:04:23.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.231 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659248 s, 159 MB/s 2024-04-25T00:04:23.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.240 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729518 s, 144 MB/s 2024-04-25T00:04:23.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:23.246 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:23.246 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:23.246 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T00:04:23.246 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:23.246 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:23.246 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:23.247 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:23.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:23.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595247 s, 176 MB/s 2024-04-25T00:04:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720499 s, 146 MB/s 2024-04-25T00:04:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057608 s, 182 MB/s 2024-04-25T00:04:23.328 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.328 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.328 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074486 s, 141 MB/s 2024-04-25T00:04:23.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.335 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565724 s, 185 MB/s 2024-04-25T00:04:23.344 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.345 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753731 s, 139 MB/s 2024-04-25T00:04:23.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.353 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703814 s, 149 MB/s 2024-04-25T00:04:23.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.362 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716611 s, 146 MB/s 2024-04-25T00:04:23.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.369 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554058 s, 189 MB/s 2024-04-25T00:04:23.377 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069612 s, 151 MB/s 2024-04-25T00:04:23.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.387 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741735 s, 141 MB/s 2024-04-25T00:04:23.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.396 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746311 s, 141 MB/s 2024-04-25T00:04:23.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.405 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.405 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738948 s, 142 MB/s 2024-04-25T00:04:23.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.413 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725158 s, 145 MB/s 2024-04-25T00:04:23.422 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.422 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.423 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715306 s, 147 MB/s 2024-04-25T00:04:23.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.431 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731831 s, 143 MB/s 2024-04-25T00:04:23.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.438 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574904 s, 182 MB/s 2024-04-25T00:04:23.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.447 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740534 s, 142 MB/s 2024-04-25T00:04:23.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.456 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007142 s, 147 MB/s 2024-04-25T00:04:23.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.464 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654788 s, 160 MB/s 2024-04-25T00:04:23.473 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.473 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.473 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714029 s, 147 MB/s 2024-04-25T00:04:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675128 s, 155 MB/s 2024-04-25T00:04:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662206 s, 158 MB/s 2024-04-25T00:04:23.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.497 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657013 s, 160 MB/s 2024-04-25T00:04:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747523 s, 140 MB/s 2024-04-25T00:04:23.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.517 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776023 s, 135 MB/s 2024-04-25T00:04:23.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.524 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624433 s, 168 MB/s 2024-04-25T00:04:23.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.532 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664264 s, 158 MB/s 2024-04-25T00:04:23.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.540 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641237 s, 164 MB/s 2024-04-25T00:04:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638767 s, 164 MB/s 2024-04-25T00:04:23.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.556 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665405 s, 158 MB/s 2024-04-25T00:04:23.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.564 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686357 s, 153 MB/s 2024-04-25T00:04:23.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:23.571 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:23.571 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:23.572 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T00:04:23.572 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:23.572 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:23.572 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:23.572 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:23.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:23.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584618 s, 179 MB/s 2024-04-25T00:04:23.638 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.638 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.638 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719957 s, 146 MB/s 2024-04-25T00:04:23.646 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.647 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678341 s, 155 MB/s 2024-04-25T00:04:23.654 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.654 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.654 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063981 s, 164 MB/s 2024-04-25T00:04:23.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.662 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066749 s, 157 MB/s 2024-04-25T00:04:23.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565781 s, 185 MB/s 2024-04-25T00:04:23.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654773 s, 160 MB/s 2024-04-25T00:04:23.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.685 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589503 s, 178 MB/s 2024-04-25T00:04:23.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.692 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562761 s, 186 MB/s 2024-04-25T00:04:23.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563078 s, 186 MB/s 2024-04-25T00:04:23.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.707 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573058 s, 183 MB/s 2024-04-25T00:04:23.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.718 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005839 s, 180 MB/s 2024-04-25T00:04:23.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.723 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068237 s, 154 MB/s 2024-04-25T00:04:23.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641874 s, 163 MB/s 2024-04-25T00:04:23.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.738 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582057 s, 180 MB/s 2024-04-25T00:04:23.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.747 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723061 s, 145 MB/s 2024-04-25T00:04:23.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.754 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569547 s, 184 MB/s 2024-04-25T00:04:23.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.763 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704344 s, 149 MB/s 2024-04-25T00:04:23.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.772 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.772 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717692 s, 146 MB/s 2024-04-25T00:04:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720105 s, 146 MB/s 2024-04-25T00:04:23.789 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.789 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.789 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691647 s, 152 MB/s 2024-04-25T00:04:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647592 s, 162 MB/s 2024-04-25T00:04:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703616 s, 149 MB/s 2024-04-25T00:04:23.814 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.814 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.814 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069074 s, 152 MB/s 2024-04-25T00:04:23.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.823 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680543 s, 154 MB/s 2024-04-25T00:04:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007087 s, 148 MB/s 2024-04-25T00:04:23.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.840 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730302 s, 144 MB/s 2024-04-25T00:04:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063572 s, 165 MB/s 2024-04-25T00:04:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574482 s, 183 MB/s 2024-04-25T00:04:23.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.863 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628991 s, 167 MB/s 2024-04-25T00:04:23.871 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.871 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.871 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656564 s, 160 MB/s 2024-04-25T00:04:23.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673459 s, 156 MB/s 2024-04-25T00:04:23.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:23.886 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:23.887 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:23.887 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T00:04:23.887 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:23.887 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:23.887 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:23.887 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:23.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.947 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764147 s, 137 MB/s 2024-04-25T00:04:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631835 s, 166 MB/s 2024-04-25T00:04:23.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:23.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:23.963 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720168 s, 146 MB/s 2024-04-25T00:04:23.970 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.034 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554539 s, 189 MB/s 2024-04-25T00:04:24.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.036 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714025 s, 147 MB/s 2024-04-25T00:04:24.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.037 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651901 s, 161 MB/s 2024-04-25T00:04:24.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.037 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723691 s, 145 MB/s 2024-04-25T00:04:24.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624957 s, 168 MB/s 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569882 s, 184 MB/s 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571903 s, 183 MB/s 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.039 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074679 s, 140 MB/s 2024-04-25T00:04:24.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.039 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674167 s, 156 MB/s 2024-04-25T00:04:24.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.044 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064772 s, 162 MB/s 2024-04-25T00:04:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648084 s, 162 MB/s 2024-04-25T00:04:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721935 s, 145 MB/s 2024-04-25T00:04:24.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.069 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.069 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662602 s, 158 MB/s 2024-04-25T00:04:24.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.077 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641565 s, 163 MB/s 2024-04-25T00:04:24.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.085 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686566 s, 153 MB/s 2024-04-25T00:04:24.093 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.093 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.093 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641676 s, 163 MB/s 2024-04-25T00:04:24.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.100 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562444 s, 186 MB/s 2024-04-25T00:04:24.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.109 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704209 s, 149 MB/s 2024-04-25T00:04:24.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.118 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.118 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707747 s, 148 MB/s 2024-04-25T00:04:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645552 s, 162 MB/s 2024-04-25T00:04:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676313 s, 155 MB/s 2024-04-25T00:04:24.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.143 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670274 s, 156 MB/s 2024-04-25T00:04:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645678 s, 162 MB/s 2024-04-25T00:04:24.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.159 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715021 s, 147 MB/s 2024-04-25T00:04:24.167 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.167 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.167 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070871 s, 148 MB/s 2024-04-25T00:04:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713075 s, 147 MB/s 2024-04-25T00:04:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574698 s, 182 MB/s 2024-04-25T00:04:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716516 s, 146 MB/s 2024-04-25T00:04:24.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706285 s, 148 MB/s 2024-04-25T00:04:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:24.207 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:24.207 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:24.207 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T00:04:24.208 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:24.208 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:24.208 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:24.208 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:24.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:24.266 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.266 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.266 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071711 s, 146 MB/s 2024-04-25T00:04:24.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.274 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619289 s, 169 MB/s 2024-04-25T00:04:24.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.283 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749095 s, 140 MB/s 2024-04-25T00:04:24.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.292 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733104 s, 143 MB/s 2024-04-25T00:04:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.301 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071917 s, 146 MB/s 2024-04-25T00:04:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708532 s, 148 MB/s 2024-04-25T00:04:24.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055908 s, 188 MB/s 2024-04-25T00:04:24.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.324 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583144 s, 180 MB/s 2024-04-25T00:04:24.331 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.331 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.331 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566414 s, 185 MB/s 2024-04-25T00:04:24.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.341 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.341 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065468 s, 160 MB/s 2024-04-25T00:04:24.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.348 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667483 s, 157 MB/s 2024-04-25T00:04:24.357 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.357 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.357 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756946 s, 139 MB/s 2024-04-25T00:04:24.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.364 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625644 s, 168 MB/s 2024-04-25T00:04:24.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.373 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069913 s, 150 MB/s 2024-04-25T00:04:24.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.382 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732233 s, 143 MB/s 2024-04-25T00:04:24.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.390 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674365 s, 155 MB/s 2024-04-25T00:04:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.399 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687924 s, 152 MB/s 2024-04-25T00:04:24.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.406 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639789 s, 164 MB/s 2024-04-25T00:04:24.415 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.415 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.415 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722985 s, 145 MB/s 2024-04-25T00:04:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624388 s, 168 MB/s 2024-04-25T00:04:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720158 s, 146 MB/s 2024-04-25T00:04:24.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.439 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568937 s, 184 MB/s 2024-04-25T00:04:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655455 s, 160 MB/s 2024-04-25T00:04:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579186 s, 181 MB/s 2024-04-25T00:04:24.462 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.463 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646468 s, 162 MB/s 2024-04-25T00:04:24.470 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.471 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066359 s, 158 MB/s 2024-04-25T00:04:24.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739391 s, 142 MB/s 2024-04-25T00:04:24.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.487 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576905 s, 182 MB/s 2024-04-25T00:04:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063792 s, 164 MB/s 2024-04-25T00:04:24.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.503 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637902 s, 164 MB/s 2024-04-25T00:04:24.510 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.511 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632347 s, 166 MB/s 2024-04-25T00:04:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739197 s, 142 MB/s 2024-04-25T00:04:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:24.526 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:24.527 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:24.527 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T00:04:24.527 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:24.527 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:24.527 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:24.527 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:24.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:24.585 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.585 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.585 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072867 s, 144 MB/s 2024-04-25T00:04:24.594 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.594 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.594 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767655 s, 137 MB/s 2024-04-25T00:04:24.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.603 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070994 s, 148 MB/s 2024-04-25T00:04:24.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.614 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904936 s, 116 MB/s 2024-04-25T00:04:24.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.621 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625048 s, 168 MB/s 2024-04-25T00:04:24.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558465 s, 188 MB/s 2024-04-25T00:04:24.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.637 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600563 s, 175 MB/s 2024-04-25T00:04:24.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.645 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694515 s, 151 MB/s 2024-04-25T00:04:24.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.652 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555376 s, 189 MB/s 2024-04-25T00:04:24.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.660 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646212 s, 162 MB/s 2024-04-25T00:04:24.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.669 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677184 s, 155 MB/s 2024-04-25T00:04:24.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751918 s, 139 MB/s 2024-04-25T00:04:24.686 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.686 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.686 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656236 s, 160 MB/s 2024-04-25T00:04:24.694 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.695 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.695 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071598 s, 146 MB/s 2024-04-25T00:04:24.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.702 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566997 s, 185 MB/s 2024-04-25T00:04:24.710 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.710 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.711 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715796 s, 146 MB/s 2024-04-25T00:04:24.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.719 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723221 s, 145 MB/s 2024-04-25T00:04:24.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.728 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665986 s, 157 MB/s 2024-04-25T00:04:24.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.735 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587362 s, 179 MB/s 2024-04-25T00:04:24.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.746 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575899 s, 182 MB/s 2024-04-25T00:04:24.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.751 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.751 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650799 s, 161 MB/s 2024-04-25T00:04:24.759 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.759 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.759 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708559 s, 148 MB/s 2024-04-25T00:04:24.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.768 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737886 s, 142 MB/s 2024-04-25T00:04:24.775 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.775 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.775 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551983 s, 190 MB/s 2024-04-25T00:04:24.783 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.783 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.783 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640626 s, 164 MB/s 2024-04-25T00:04:24.791 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.791 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.791 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666001 s, 157 MB/s 2024-04-25T00:04:24.800 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.800 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.800 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074296 s, 141 MB/s 2024-04-25T00:04:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703854 s, 149 MB/s 2024-04-25T00:04:24.817 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.817 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714927 s, 147 MB/s 2024-04-25T00:04:24.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.826 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715764 s, 146 MB/s 2024-04-25T00:04:24.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.836 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781036 s, 134 MB/s 2024-04-25T00:04:24.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.845 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762245 s, 138 MB/s 2024-04-25T00:04:24.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:24.851 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:24.851 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:24.851 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T00:04:24.852 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:24.852 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:24.852 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:24.852 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:24.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:24.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.909 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650345 s, 161 MB/s 2024-04-25T00:04:24.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.919 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659872 s, 159 MB/s 2024-04-25T00:04:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797598 s, 131 MB/s 2024-04-25T00:04:24.935 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.935 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.935 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710633 s, 148 MB/s 2024-04-25T00:04:24.944 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.944 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.944 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722057 s, 145 MB/s 2024-04-25T00:04:24.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.952 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059535 s, 176 MB/s 2024-04-25T00:04:24.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.959 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550512 s, 190 MB/s 2024-04-25T00:04:24.968 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.968 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.968 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737776 s, 142 MB/s 2024-04-25T00:04:24.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.975 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579514 s, 181 MB/s 2024-04-25T00:04:24.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.983 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563442 s, 186 MB/s 2024-04-25T00:04:24.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:24.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:24.992 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760399 s, 138 MB/s 2024-04-25T00:04:25.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.001 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071939 s, 146 MB/s 2024-04-25T00:04:25.007 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.008 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.008 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557989 s, 188 MB/s 2024-04-25T00:04:25.016 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.017 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073448 s, 143 MB/s 2024-04-25T00:04:25.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.025 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738982 s, 142 MB/s 2024-04-25T00:04:25.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.034 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702721 s, 149 MB/s 2024-04-25T00:04:25.043 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.043 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.043 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706522 s, 148 MB/s 2024-04-25T00:04:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657038 s, 160 MB/s 2024-04-25T00:04:25.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658189 s, 159 MB/s 2024-04-25T00:04:25.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737708 s, 142 MB/s 2024-04-25T00:04:25.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.075 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059716 s, 176 MB/s 2024-04-25T00:04:25.084 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.085 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738287 s, 142 MB/s 2024-04-25T00:04:25.092 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.092 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.092 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061619 s, 170 MB/s 2024-04-25T00:04:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701398 s, 149 MB/s 2024-04-25T00:04:25.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.109 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636394 s, 165 MB/s 2024-04-25T00:04:25.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.118 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721934 s, 145 MB/s 2024-04-25T00:04:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578747 s, 181 MB/s 2024-04-25T00:04:25.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.133 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659878 s, 159 MB/s 2024-04-25T00:04:25.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.142 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727756 s, 144 MB/s 2024-04-25T00:04:25.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665741 s, 158 MB/s 2024-04-25T00:04:25.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.159 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736678 s, 142 MB/s 2024-04-25T00:04:25.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.169 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723123 s, 145 MB/s 2024-04-25T00:04:25.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:25.174 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:25.175 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:25.175 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T00:04:25.175 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:25.175 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:25.175 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:25.175 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:25.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:25.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.233 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750516 s, 140 MB/s 2024-04-25T00:04:25.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.243 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782678 s, 134 MB/s 2024-04-25T00:04:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570773 s, 184 MB/s 2024-04-25T00:04:25.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.258 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666668 s, 157 MB/s 2024-04-25T00:04:25.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.267 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713897 s, 147 MB/s 2024-04-25T00:04:25.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.276 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739231 s, 142 MB/s 2024-04-25T00:04:25.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.285 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715192 s, 147 MB/s 2024-04-25T00:04:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554604 s, 189 MB/s 2024-04-25T00:04:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703489 s, 149 MB/s 2024-04-25T00:04:25.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709704 s, 148 MB/s 2024-04-25T00:04:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574495 s, 183 MB/s 2024-04-25T00:04:25.325 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.325 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.326 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072484 s, 145 MB/s 2024-04-25T00:04:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749284 s, 140 MB/s 2024-04-25T00:04:25.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583543 s, 180 MB/s 2024-04-25T00:04:25.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593844 s, 177 MB/s 2024-04-25T00:04:25.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.362 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072535 s, 145 MB/s 2024-04-25T00:04:25.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.367 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.367 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592964 s, 177 MB/s 2024-04-25T00:04:25.374 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.374 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.374 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578779 s, 181 MB/s 2024-04-25T00:04:25.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.382 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641882 s, 163 MB/s 2024-04-25T00:04:25.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.391 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676227 s, 155 MB/s 2024-04-25T00:04:25.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.399 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.399 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069527 s, 151 MB/s 2024-04-25T00:04:25.407 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.407 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.407 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716899 s, 146 MB/s 2024-04-25T00:04:25.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.416 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741246 s, 141 MB/s 2024-04-25T00:04:25.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.426 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716793 s, 146 MB/s 2024-04-25T00:04:25.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.434 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723174 s, 145 MB/s 2024-04-25T00:04:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565339 s, 185 MB/s 2024-04-25T00:04:25.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.449 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675869 s, 155 MB/s 2024-04-25T00:04:25.458 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.458 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.458 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713315 s, 147 MB/s 2024-04-25T00:04:25.467 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.467 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.467 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708594 s, 148 MB/s 2024-04-25T00:04:25.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.476 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.476 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723273 s, 145 MB/s 2024-04-25T00:04:25.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.484 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729741 s, 144 MB/s 2024-04-25T00:04:25.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.492 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577147 s, 182 MB/s 2024-04-25T00:04:25.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:25.498 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:25.498 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:25.498 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T00:04:25.498 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:25.499 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:25.499 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:25.499 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:25.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:25.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.555 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576872 s, 182 MB/s 2024-04-25T00:04:25.562 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.565 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563881 s, 186 MB/s 2024-04-25T00:04:25.571 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737193 s, 142 MB/s 2024-04-25T00:04:25.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.581 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734157 s, 143 MB/s 2024-04-25T00:04:25.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.588 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058125 s, 180 MB/s 2024-04-25T00:04:25.596 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702934 s, 149 MB/s 2024-04-25T00:04:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665888 s, 157 MB/s 2024-04-25T00:04:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737378 s, 142 MB/s 2024-04-25T00:04:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631133 s, 166 MB/s 2024-04-25T00:04:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592062 s, 177 MB/s 2024-04-25T00:04:25.638 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.638 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713496 s, 147 MB/s 2024-04-25T00:04:25.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.647 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742928 s, 141 MB/s 2024-04-25T00:04:25.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.656 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719587 s, 146 MB/s 2024-04-25T00:04:25.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.664 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065881 s, 159 MB/s 2024-04-25T00:04:25.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.672 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673134 s, 156 MB/s 2024-04-25T00:04:25.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.681 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722799 s, 145 MB/s 2024-04-25T00:04:25.690 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.690 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.690 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007149 s, 147 MB/s 2024-04-25T00:04:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563411 s, 186 MB/s 2024-04-25T00:04:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.706 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659123 s, 159 MB/s 2024-04-25T00:04:25.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.715 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697274 s, 150 MB/s 2024-04-25T00:04:25.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650423 s, 161 MB/s 2024-04-25T00:04:25.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.730 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665073 s, 158 MB/s 2024-04-25T00:04:25.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.739 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673132 s, 156 MB/s 2024-04-25T00:04:25.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.746 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057 s, 184 MB/s 2024-04-25T00:04:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650382 s, 161 MB/s 2024-04-25T00:04:25.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.763 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713463 s, 147 MB/s 2024-04-25T00:04:25.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.772 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741718 s, 141 MB/s 2024-04-25T00:04:25.780 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.780 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.780 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722431 s, 145 MB/s 2024-04-25T00:04:25.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.789 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.789 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671104 s, 156 MB/s 2024-04-25T00:04:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774628 s, 135 MB/s 2024-04-25T00:04:25.806 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.807 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.807 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069119 s, 152 MB/s 2024-04-25T00:04:25.815 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.815 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.816 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760241 s, 138 MB/s 2024-04-25T00:04:25.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:25.822 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:25.822 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:25.822 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T00:04:25.822 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:25.822 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:25.822 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:25.823 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:25.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:25.881 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.881 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778006 s, 135 MB/s 2024-04-25T00:04:25.891 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.891 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.891 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819192 s, 128 MB/s 2024-04-25T00:04:25.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.901 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769318 s, 136 MB/s 2024-04-25T00:04:25.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.909 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688708 s, 152 MB/s 2024-04-25T00:04:25.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.918 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735438 s, 143 MB/s 2024-04-25T00:04:25.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.926 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644516 s, 163 MB/s 2024-04-25T00:04:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559998 s, 187 MB/s 2024-04-25T00:04:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643034 s, 163 MB/s 2024-04-25T00:04:25.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.950 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730119 s, 144 MB/s 2024-04-25T00:04:25.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.959 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705993 s, 149 MB/s 2024-04-25T00:04:25.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.966 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555755 s, 189 MB/s 2024-04-25T00:04:25.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.974 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707414 s, 148 MB/s 2024-04-25T00:04:25.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.982 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590295 s, 178 MB/s 2024-04-25T00:04:25.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.991 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724052 s, 145 MB/s 2024-04-25T00:04:25.999 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:25.999 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:25.999 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066266 s, 158 MB/s 2024-04-25T00:04:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.007 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561504 s, 187 MB/s 2024-04-25T00:04:26.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.015 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602531 s, 174 MB/s 2024-04-25T00:04:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.022 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570762 s, 184 MB/s 2024-04-25T00:04:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690307 s, 152 MB/s 2024-04-25T00:04:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927866 s, 113 MB/s 2024-04-25T00:04:26.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.050 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579597 s, 181 MB/s 2024-04-25T00:04:26.058 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.058 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645285 s, 162 MB/s 2024-04-25T00:04:26.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.068 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748051 s, 140 MB/s 2024-04-25T00:04:26.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.077 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711421 s, 147 MB/s 2024-04-25T00:04:26.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.085 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646147 s, 162 MB/s 2024-04-25T00:04:26.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.095 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668714 s, 157 MB/s 2024-04-25T00:04:26.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.103 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756975 s, 139 MB/s 2024-04-25T00:04:26.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.112 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657297 s, 160 MB/s 2024-04-25T00:04:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640269 s, 164 MB/s 2024-04-25T00:04:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575945 s, 182 MB/s 2024-04-25T00:04:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.137 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692601 s, 151 MB/s 2024-04-25T00:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719415 s, 146 MB/s 2024-04-25T00:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:26.153 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:26.153 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:26.153 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T00:04:26.153 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:26.153 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:26.153 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:26.153 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:26.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:26.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621431 s, 169 MB/s 2024-04-25T00:04:26.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.221 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685653 s, 153 MB/s 2024-04-25T00:04:26.229 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.229 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.230 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718253 s, 146 MB/s 2024-04-25T00:04:26.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.238 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067136 s, 156 MB/s 2024-04-25T00:04:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761304 s, 138 MB/s 2024-04-25T00:04:26.256 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.256 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.256 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640442 s, 164 MB/s 2024-04-25T00:04:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659987 s, 159 MB/s 2024-04-25T00:04:26.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.272 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596564 s, 176 MB/s 2024-04-25T00:04:26.281 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.281 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.281 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631334 s, 166 MB/s 2024-04-25T00:04:26.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.289 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645177 s, 163 MB/s 2024-04-25T00:04:26.298 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.298 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.298 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650698 s, 161 MB/s 2024-04-25T00:04:26.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.306 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645644 s, 162 MB/s 2024-04-25T00:04:26.315 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.315 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.315 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697763 s, 150 MB/s 2024-04-25T00:04:26.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.324 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732502 s, 143 MB/s 2024-04-25T00:04:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715544 s, 147 MB/s 2024-04-25T00:04:26.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708479 s, 148 MB/s 2024-04-25T00:04:26.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614148 s, 171 MB/s 2024-04-25T00:04:26.359 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.359 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.359 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684926 s, 153 MB/s 2024-04-25T00:04:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575104 s, 182 MB/s 2024-04-25T00:04:26.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.375 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007243 s, 145 MB/s 2024-04-25T00:04:26.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.386 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696685 s, 151 MB/s 2024-04-25T00:04:26.392 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.392 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.392 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714361 s, 147 MB/s 2024-04-25T00:04:26.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.401 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743501 s, 141 MB/s 2024-04-25T00:04:26.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.409 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633858 s, 165 MB/s 2024-04-25T00:04:26.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.418 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717403 s, 146 MB/s 2024-04-25T00:04:26.427 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.427 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.427 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721677 s, 145 MB/s 2024-04-25T00:04:26.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.435 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063795 s, 164 MB/s 2024-04-25T00:04:26.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.443 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677092 s, 155 MB/s 2024-04-25T00:04:26.452 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.452 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.452 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691778 s, 152 MB/s 2024-04-25T00:04:26.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.461 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740707 s, 142 MB/s 2024-04-25T00:04:26.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.469 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072994 s, 144 MB/s 2024-04-25T00:04:26.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.478 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695157 s, 151 MB/s 2024-04-25T00:04:26.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:26.485 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:26.485 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:26.485 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T00:04:26.485 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:26.485 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:26.485 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:26.485 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:26.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:26.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.543 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719457 s, 146 MB/s 2024-04-25T00:04:26.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.551 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666664 s, 157 MB/s 2024-04-25T00:04:26.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.560 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738243 s, 142 MB/s 2024-04-25T00:04:26.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.569 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.569 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717165 s, 146 MB/s 2024-04-25T00:04:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709213 s, 148 MB/s 2024-04-25T00:04:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056205 s, 187 MB/s 2024-04-25T00:04:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576661 s, 182 MB/s 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564776 s, 186 MB/s 2024-04-25T00:04:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564208 s, 186 MB/s 2024-04-25T00:04:26.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681805 s, 154 MB/s 2024-04-25T00:04:26.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.623 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634479 s, 165 MB/s 2024-04-25T00:04:26.630 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.630 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.630 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572207 s, 183 MB/s 2024-04-25T00:04:26.638 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717035 s, 146 MB/s 2024-04-25T00:04:26.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.647 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699523 s, 150 MB/s 2024-04-25T00:04:26.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.656 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664621 s, 158 MB/s 2024-04-25T00:04:26.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.664 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708198 s, 148 MB/s 2024-04-25T00:04:26.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.673 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709325 s, 148 MB/s 2024-04-25T00:04:26.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.681 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717236 s, 146 MB/s 2024-04-25T00:04:26.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.690 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676187 s, 155 MB/s 2024-04-25T00:04:26.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.698 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745886 s, 141 MB/s 2024-04-25T00:04:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.708 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750582 s, 140 MB/s 2024-04-25T00:04:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702066 s, 149 MB/s 2024-04-25T00:04:26.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.724 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665548 s, 158 MB/s 2024-04-25T00:04:26.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.733 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708425 s, 148 MB/s 2024-04-25T00:04:26.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.740 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558642 s, 188 MB/s 2024-04-25T00:04:26.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.748 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648025 s, 162 MB/s 2024-04-25T00:04:26.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.757 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703359 s, 149 MB/s 2024-04-25T00:04:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639515 s, 164 MB/s 2024-04-25T00:04:26.773 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.774 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.774 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725896 s, 144 MB/s 2024-04-25T00:04:26.781 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.782 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634759 s, 165 MB/s 2024-04-25T00:04:26.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564438 s, 186 MB/s 2024-04-25T00:04:26.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.798 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.798 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724691 s, 145 MB/s 2024-04-25T00:04:26.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:26.804 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:26.804 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:26.804 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T00:04:26.804 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:26.804 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:26.804 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:26.804 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:26.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065348 s, 160 MB/s 2024-04-25T00:04:26.869 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.869 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657007 s, 160 MB/s 2024-04-25T00:04:26.878 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740363 s, 142 MB/s 2024-04-25T00:04:26.888 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.888 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.888 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804396 s, 130 MB/s 2024-04-25T00:04:26.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.896 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614374 s, 171 MB/s 2024-04-25T00:04:26.904 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.904 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.905 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666192 s, 157 MB/s 2024-04-25T00:04:26.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.912 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575257 s, 182 MB/s 2024-04-25T00:04:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662796 s, 158 MB/s 2024-04-25T00:04:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728264 s, 144 MB/s 2024-04-25T00:04:26.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.938 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672404 s, 156 MB/s 2024-04-25T00:04:26.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.946 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661784 s, 158 MB/s 2024-04-25T00:04:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844793 s, 124 MB/s 2024-04-25T00:04:26.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.965 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644345 s, 163 MB/s 2024-04-25T00:04:26.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.974 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707331 s, 148 MB/s 2024-04-25T00:04:26.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.983 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763192 s, 137 MB/s 2024-04-25T00:04:26.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:26.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:26.991 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666894 s, 157 MB/s 2024-04-25T00:04:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.001 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753107 s, 139 MB/s 2024-04-25T00:04:27.008 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.008 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.009 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621523 s, 169 MB/s 2024-04-25T00:04:27.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.018 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664494 s, 158 MB/s 2024-04-25T00:04:27.026 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.027 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745079 s, 141 MB/s 2024-04-25T00:04:27.035 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.035 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.035 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696523 s, 151 MB/s 2024-04-25T00:04:27.045 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.045 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.045 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821229 s, 128 MB/s 2024-04-25T00:04:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676078 s, 155 MB/s 2024-04-25T00:04:27.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.062 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659687 s, 159 MB/s 2024-04-25T00:04:27.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.071 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729372 s, 144 MB/s 2024-04-25T00:04:27.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.080 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712312 s, 147 MB/s 2024-04-25T00:04:27.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.088 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730945 s, 143 MB/s 2024-04-25T00:04:27.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.097 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708429 s, 148 MB/s 2024-04-25T00:04:27.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.106 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739151 s, 142 MB/s 2024-04-25T00:04:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706402 s, 148 MB/s 2024-04-25T00:04:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677498 s, 155 MB/s 2024-04-25T00:04:27.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.132 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724049 s, 145 MB/s 2024-04-25T00:04:27.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:27.139 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:27.139 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:27.139 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T00:04:27.139 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:27.139 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:27.139 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:27.140 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:27.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:27.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.198 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636767 s, 165 MB/s 2024-04-25T00:04:27.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.206 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623792 s, 168 MB/s 2024-04-25T00:04:27.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.214 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006873 s, 153 MB/s 2024-04-25T00:04:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.221 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558867 s, 188 MB/s 2024-04-25T00:04:27.230 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739161 s, 142 MB/s 2024-04-25T00:04:27.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.240 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713549 s, 147 MB/s 2024-04-25T00:04:27.247 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.247 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.247 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575555 s, 182 MB/s 2024-04-25T00:04:27.255 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.255 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.255 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656653 s, 160 MB/s 2024-04-25T00:04:27.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.262 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581167 s, 180 MB/s 2024-04-25T00:04:27.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.271 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688281 s, 152 MB/s 2024-04-25T00:04:27.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574204 s, 183 MB/s 2024-04-25T00:04:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705063 s, 149 MB/s 2024-04-25T00:04:27.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.295 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651611 s, 161 MB/s 2024-04-25T00:04:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587114 s, 179 MB/s 2024-04-25T00:04:27.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.310 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663372 s, 158 MB/s 2024-04-25T00:04:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063884 s, 164 MB/s 2024-04-25T00:04:27.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.328 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.328 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077511 s, 135 MB/s 2024-04-25T00:04:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780109 s, 134 MB/s 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663863 s, 158 MB/s 2024-04-25T00:04:27.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.354 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735966 s, 142 MB/s 2024-04-25T00:04:27.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.362 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634167 s, 165 MB/s 2024-04-25T00:04:27.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065244 s, 161 MB/s 2024-04-25T00:04:27.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065113 s, 161 MB/s 2024-04-25T00:04:27.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.388 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806041 s, 130 MB/s 2024-04-25T00:04:27.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.397 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.397 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698881 s, 150 MB/s 2024-04-25T00:04:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769941 s, 136 MB/s 2024-04-25T00:04:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763296 s, 137 MB/s 2024-04-25T00:04:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769448 s, 136 MB/s 2024-04-25T00:04:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.434 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077072 s, 136 MB/s 2024-04-25T00:04:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737034 s, 142 MB/s 2024-04-25T00:04:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681443 s, 154 MB/s 2024-04-25T00:04:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658902 s, 159 MB/s 2024-04-25T00:04:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:27.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:27.466 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T00:04:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.565+0000 7f9c9cf42640 1 Processor -- start 2024-04-25T00:04:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.565+0000 7f9c9cf42640 1 -- start start 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c9cf42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9c9814b5a0 0x7f9c9806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c9cf42640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9806e610 0x7f9c98070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c9cf42640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c980b4950 con 0x7f9c98071470 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c9cf42640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9814b980 con 0x7f9c9806e610 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c9cf42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9c98071ca0 con 0x7f9c9814b5a0 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c95d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9806e610 0x7f9c98070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c95d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9806e610 0x7f9c98070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35088/0 (socket says 172.21.15.113:35088) 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2940340829 learned_addr learned my addr 172.21.15.113:0/2940340829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c9659c640 1 --2- 172.21.15.113:0/2940340829 >> v2:172.21.15.178:3300/0 conn(0x7f9c9814b5a0 0x7f9c9806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2940340829 >> v2:172.21.15.178:3300/0 conn(0x7f9c9814b5a0 msgr2=0x7f9c9806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/2940340829 >> v2:172.21.15.178:3300/0 conn(0x7f9c9814b5a0 0x7f9c9806e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2940340829 >> v1:172.21.15.178:6789/0 conn(0x7f9c98071470 legacy=0x7f9c98072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.566+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2940340829 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c98167210 con 0x7f9c9806e610 2024-04-25T00:04:27.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.567+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/2940340829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9806e610 0x7f9c98070f30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9c9806c0d0 tx=0x7f9c8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e347e88f7ee056b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.567+0000 7f9c9559a640 1 -- 172.21.15.113:0/2940340829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c037070 con 0x7f9c9806e610 2024-04-25T00:04:27.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.567+0000 7f9c9559a640 1 -- 172.21.15.113:0/2940340829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c032440 con 0x7f9c9806e610 2024-04-25T00:04:27.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.567+0000 7f9c9559a640 1 -- 172.21.15.113:0/2940340829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c032810 con 0x7f9c9806e610 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 -- 172.21.15.113:0/2940340829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9806e610 msgr2=0x7f9c98070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/2940340829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9806e610 0x7f9c98070f30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9c9806c0d0 tx=0x7f9c8c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 -- 172.21.15.113:0/2940340829 shutdown_connections 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/2940340829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9806e610 0x7f9c98070f30 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/2940340829 >> v2:172.21.15.178:3300/0 conn(0x7f9c9814b5a0 0x7f9c9806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 -- 172.21.15.113:0/2940340829 >> 172.21.15.113:0/2940340829 conn(0x7f9c980b1b30 msgr2=0x7f9c980b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 -- 172.21.15.113:0/2940340829 shutdown_connections 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.568+0000 7f9c9cf42640 1 -- 172.21.15.113:0/2940340829 wait complete. 2024-04-25T00:04:27.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.569+0000 7f9c9cf42640 1 Processor -- start 2024-04-25T00:04:27.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.569+0000 7f9c9cf42640 1 -- start start 2024-04-25T00:04:27.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c9cf42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9806e610 0x7f9c981fc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c9cf42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c98071470 0x7f9c980af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c9cf42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c9814b5a0 0x7f9c98148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c9cf42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98166a10 con 0x7f9c98071470 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c9cf42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98166730 con 0x7f9c9814b5a0 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c9cf42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c98166cf0 con 0x7f9c9806e610 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c98071470 0x7f9c980af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c98071470 0x7f9c980af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35092/0 (socket says 172.21.15.113:35092) 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 -- 172.21.15.113:0/4184469514 learned_addr learned my addr 172.21.15.113:0/4184469514 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c9659c640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9806e610 0x7f9c981fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 -- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9806e610 msgr2=0x7f9c981fc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c9814b5a0 0x7f9c98148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9806e610 0x7f9c981fc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 -- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c9814b5a0 msgr2=0x7f9c98148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c9814b5a0 0x7f9c98148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 -- 172.21.15.113:0/4184469514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c980af850 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.570+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c98071470 0x7f9c980af310 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c000c00 tx=0x7f9c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.571+0000 7f9c86ffd640 1 -- 172.21.15.113:0/4184469514 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c041020 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.571+0000 7f9c86ffd640 1 -- 172.21.15.113:0/4184469514 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c030970 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.571+0000 7f9c86ffd640 1 -- 172.21.15.113:0/4184469514 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c037040 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.571+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c981f8f20 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.571+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981f94e0 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.572+0000 7f9c86ffd640 1 -- 172.21.15.113:0/4184469514 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c8c044020 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.572+0000 7f9c86ffd640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7806be30 0x7f9c7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.573+0000 7f9c86ffd640 1 -- 172.21.15.113:0/4184469514 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f9c8c0ab8a0 con 0x7f9c98071470 2024-04-25T00:04:27.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.573+0000 7f9c9659c640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7806be30 0x7f9c7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.573+0000 7f9c9659c640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7806be30 0x7f9c7806e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9c9806e400 tx=0x7f9c880083c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.573+0000 7f9c9fc55740 1 Processor -- start 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c9fc55740 1 -- start start 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70d3320 0x559de7085cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70b6440 0x559de70b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70b5d00 0x559de708ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559de6fed930 con 0x559de70b5d00 2024-04-25T00:04:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559de6f95cc0 con 0x559de70b6440 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559de6fe98f0 con 0x559de70d3320 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c96d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70b5d00 0x559de708ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c96d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70b5d00 0x559de708ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35096/0 (socket says 172.21.15.113:35096) 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.574+0000 7f9c96d9d640 1 -- 172.21.15.113:0/3563962336 learned_addr learned my addr 172.21.15.113:0/3563962336 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c9659c640 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70d3320 0x559de7085cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70b6440 0x559de70b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c96d9d640 1 -- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70d3320 msgr2=0x559de7085cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70d3320 0x559de7085cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c96d9d640 1 -- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70b6440 msgr2=0x559de70b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70b6440 0x559de70b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c96d9d640 1 -- 172.21.15.113:0/3563962336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de6dcdba0 con 0x559de70b5d00 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70b5d00 0x559de708ab10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9c90003e10 tx=0x7f9c900042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a5c5516de72598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c77fff640 1 -- 172.21.15.113:0/3563962336 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90016020 con 0x559de70b5d00 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c77fff640 1 -- 172.21.15.113:0/3563962336 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9000db60 con 0x559de70b5d00 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.575+0000 7f9c77fff640 1 -- 172.21.15.113:0/3563962336 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c900123f0 con 0x559de70b5d00 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3563962336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70b5d00 msgr2=0x559de708ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70b5d00 0x559de708ab10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9c90003e10 tx=0x7f9c900042d0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3563962336 shutdown_connections 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70b6440 0x559de70b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70b5d00 0x559de708ab10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3563962336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70d3320 0x559de7085cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3563962336 >> 172.21.15.113:0/3563962336 conn(0x559de7087f20 msgr2=0x559de70d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3563962336 shutdown_connections 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3563962336 wait complete. 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 -- start start 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.576+0000 7f9c9fc55740 1 Processor -- start 2024-04-25T00:04:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70d3320 0x559de70e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70e1500 0x559de70b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559de708a0d0 con 0x559de70e1500 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559de6fa9330 con 0x559de70d3320 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559de70894c0 con 0x559de70b6440 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9fc55740 10 client.0 _ll_register_callbacks cb 0x559de6fe5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c96d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70e1500 0x559de70b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c96d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70e1500 0x559de70b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35098/0 (socket says 172.21.15.113:35098) 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c96d9d640 1 -- 172.21.15.113:0/251211366 learned_addr learned my addr 172.21.15.113:0/251211366 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c9659c640 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.577+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70d3320 0x559de70e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c96d9d640 1 -- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 msgr2=0x559de70b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c96d9d640 1 -- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70d3320 msgr2=0x559de70e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70d3320 0x559de70e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c96d9d640 1 -- 172.21.15.113:0/251211366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de6cc1490 con 0x559de70e1500 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70e1500 0x559de70b8910 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9c90003af0 tx=0x7f9c900036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77c51d4a7d26ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c757fa640 1 -- 172.21.15.113:0/251211366 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90005450 con 0x559de70e1500 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c757fa640 1 -- 172.21.15.113:0/251211366 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f9c90027af0 con 0x559de70e1500 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c9000db60 con 0x559de70e1500 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c757fa640 1 -- 172.21.15.113:0/251211366 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9c9002e050 con 0x559de70e1500 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 10 client.7599 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x559de6e2e330 con 0x559de70e1500 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c757fa640 1 -- 172.21.15.113:0/251211366 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9000db60 con 0x559de70e1500 2024-04-25T00:04:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c757fa640 1 -- 172.21.15.113:0/251211366 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f9c90031020 con 0x559de70e1500 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 10 client.7599 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 10 client.7599 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 20 client.7599 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de70b6e00 0x559de70b7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 4 client.7599 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559de6ffae70 con 0x559de70b6e00 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.578+0000 7f9c9659c640 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de70b6e00 0x559de70b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.579+0000 7f9c757fa640 10 client.7599 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.579+0000 7f9c9659c640 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de70b6e00 0x559de70b7210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1588ddc8c30e4ed8 server_cookie=64da6e17ed7bcaa2 in_seq=0 out_seq=0 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.579+0000 7f9c757fa640 1 -- 172.21.15.113:0/251211366 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559de6ffae70 con 0x559de70b6e00 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.579+0000 7f9c757fa640 10 client.7599 handle_command_reply: tid=0 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.579+0000 7f9c9fc55740 1 client.7599 shutdown 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.579+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70e1500 msgr2=0x559de70b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.579+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70e1500 0x559de70b8910 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9c90003af0 tx=0x7f9c900036f0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 shutdown_connections 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de70b6e00 0x559de70b7210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70d3320 0x559de70e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70e1500 0x559de70b8910 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/251211366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c757fa640 0 client.7599 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 >> 172.21.15.113:0/251211366 conn(0x559de70b5d00 msgr2=0x559de70d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 shutdown_connections 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 1 -- 172.21.15.113:0/251211366 wait complete. 2024-04-25T00:04:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.580+0000 7f9c9fc55740 20 client.7599 trim_cache size 0 max 16384 2024-04-25T00:04:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.597+0000 7f9c9fc55740 1 Processor -- start 2024-04-25T00:04:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.597+0000 7f9c9fc55740 1 -- start start 2024-04-25T00:04:27.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.597+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.597+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de70c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 0x559de70c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98204d60 con 0x559de70d3320 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98203e10 con 0x559de70e1500 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c98203170 con 0x559de70b6440 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de70c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de70c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35110/0 (socket says 172.21.15.113:35110) 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 -- 172.21.15.113:0/3365128157 learned_addr learned my addr 172.21.15.113:0/3365128157 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 -- 172.21.15.113:0/3365128157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 msgr2=0x559de70d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/3365128157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 -- 172.21.15.113:0/3365128157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 msgr2=0x559de70c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/3365128157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 0x559de70c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 -- 172.21.15.113:0/3365128157 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de6c9a610 con 0x559de70d3320 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/3365128157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de70c8e10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c02a1e0 tx=0x7f9c8c0794b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170bd15435891938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c777fe640 1 -- 172.21.15.113:0/3365128157 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c07df00 con 0x559de70d3320 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c777fe640 1 -- 172.21.15.113:0/3365128157 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c082730 con 0x559de70d3320 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.598+0000 7f9c777fe640 1 -- 172.21.15.113:0/3365128157 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c077e50 con 0x559de70d3320 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3365128157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 msgr2=0x559de70c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3365128157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de70c8e10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c02a1e0 tx=0x7f9c8c0794b0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3365128157 shutdown_connections 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3365128157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 0x559de70c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3365128157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de70c8e10 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/3365128157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3365128157 >> 172.21.15.113:0/3365128157 conn(0x559de70b5d00 msgr2=0x559de7081420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3365128157 shutdown_connections 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 -- 172.21.15.113:0/3365128157 wait complete. 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 -- start start 2024-04-25T00:04:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.599+0000 7f9c9fc55740 1 Processor -- start 2024-04-25T00:04:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de714b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c9fc55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 0x559de714b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c982037c0 con 0x559de70d3320 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559de70e0670 con 0x559de70e1500 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c9fc55740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559de714c070 con 0x559de70b6440 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c9fc55740 10 client.0 _ll_register_callbacks cb 0x7f9c90003c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de714b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de714b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35126/0 (socket says 172.21.15.113:35126) 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.600+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2642048055 learned_addr learned my addr 172.21.15.113:0/2642048055 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2642048055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 msgr2=0x559de70d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c96d9d640 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 0x559de714b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2642048055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 msgr2=0x559de714b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 0x559de714b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c95d9b640 1 -- 172.21.15.113:0/2642048055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de70dfbc0 con 0x559de70d3320 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c95d9b640 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de714b2e0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c02bfd0 tx=0x7f9c8c0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a87be220189aee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c757fa640 1 -- 172.21.15.113:0/2642048055 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c097060 con 0x559de70d3320 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c757fa640 1 -- 172.21.15.113:0/2642048055 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f9c8c030d60 con 0x559de70d3320 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c8c082730 con 0x559de70d3320 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c757fa640 1 -- 172.21.15.113:0/2642048055 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9c8c07df00 con 0x559de70d3320 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 10 client.7605 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x559de714be70 con 0x559de70d3320 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c757fa640 1 -- 172.21.15.113:0/2642048055 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c082730 con 0x559de70d3320 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c757fa640 1 -- 172.21.15.113:0/2642048055 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f9c8c03a030 con 0x559de70d3320 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 10 client.7605 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 10 client.7605 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 20 client.7605 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de71348e0 0x559de70af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 4 client.7605 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x559de70e4ee0 con 0x559de71348e0 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.601+0000 7f9c9659c640 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de71348e0 0x559de70af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.602+0000 7f9c757fa640 10 client.7605 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.602+0000 7f9c9659c640 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de71348e0 0x559de70af6a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21576ee56105e2b1 server_cookie=f0472d7d60de2761 in_seq=0 out_seq=0 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c757fa640 1 -- 172.21.15.113:0/2642048055 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559de70e4ee0 con 0x559de71348e0 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c757fa640 10 client.7605 handle_command_reply: tid=0 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 client.7605 shutdown 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 msgr2=0x559de714b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de714b2e0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c02bfd0 tx=0x7f9c8c0094d0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 shutdown_connections 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x559de71348e0 0x559de70af6a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559de70e1500 0x559de714b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c757fa640 0 client.7605 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de70d3320 0x559de714b2e0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 --2- 172.21.15.113:0/2642048055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559de70b6440 0x559de70d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.604+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 >> 172.21.15.113:0/2642048055 conn(0x559de70b5d00 msgr2=0x559de7081420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 shutdown_connections 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9fc55740 1 -- 172.21.15.113:0/2642048055 wait complete. 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9fc55740 20 client.7605 trim_cache size 0 max 16384 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:04:27.610 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "7c9bc5c3-1434-4ded-9e0e-b2a88c4fd9c6", 2024-04-25T00:04:27.611 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:04:27.611 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7806be30 msgr2=0x7f9c7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7806be30 0x7f9c7806e2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9c9806e400 tx=0x7f9c880083c0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c98071470 msgr2=0x7f9c980af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c98071470 0x7f9c980af310 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c000c00 tx=0x7f9c8c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 shutdown_connections 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7806be30 0x7f9c7806e2f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c9814b5a0 0x7f9c98148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c98071470 0x7f9c980af310 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 --2- 172.21.15.113:0/4184469514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9806e610 0x7f9c981fc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 >> 172.21.15.113:0/4184469514 conn(0x7f9c980b1b30 msgr2=0x7f9c9806d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 shutdown_connections 2024-04-25T00:04:27.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.605+0000 7f9c9cf42640 1 -- 172.21.15.113:0/4184469514 wait complete. 2024-04-25T00:04:27.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:27.621 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T00:04:27.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.744+0000 7f44e9c31640 1 Processor -- start 2024-04-25T00:04:27.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.744+0000 7f44e9c31640 1 -- start start 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e9c31640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e40b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e9c31640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44e40b73c0 0x7f44e40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e9c31640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e40d2e00 con 0x7f44e415e510 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e9c31640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f44e40b6640 con 0x7f44e40b73c0 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e9c31640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44e40d30e0 con 0x7f44e40b5ca0 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f44e415e510 0x7f44e40d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39696/0 (socket says 172.21.15.113:39696) 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e37fe640 1 -- 172.21.15.113:0/3890118239 learned_addr learned my addr 172.21.15.113:0/3890118239 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.745+0000 7f44e3fff640 1 --2- 172.21.15.113:0/3890118239 >> v2:172.21.15.113:3300/0 conn(0x7f44e40b73c0 0x7f44e40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44daffd640 1 --2- 172.21.15.113:0/3890118239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e40b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e3fff640 1 -- 172.21.15.113:0/3890118239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 msgr2=0x7f44e40b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e3fff640 1 --2- 172.21.15.113:0/3890118239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e40b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e3fff640 1 -- 172.21.15.113:0/3890118239 >> v1:172.21.15.113:6789/0 conn(0x7f44e415e510 legacy=0x7f44e40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e3fff640 1 -- 172.21.15.113:0/3890118239 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e40d33c0 con 0x7f44e40b73c0 2024-04-25T00:04:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e2ffd640 1 -- 172.21.15.113:0/3890118239 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3080780121 0 0) 0x7f44e40d2e00 con 0x7f44e415e510 2024-04-25T00:04:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e3fff640 1 --2- 172.21.15.113:0/3890118239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e40b73c0 0x7f44e40b7830 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009d90 tx=0x7f44dc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea3a002562f525e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e2ffd640 1 -- 172.21.15.113:0/3890118239 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc031600 con 0x7f44e40b73c0 2024-04-25T00:04:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e2ffd640 1 -- 172.21.15.113:0/3890118239 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc0317a0 con 0x7f44e40b73c0 2024-04-25T00:04:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.746+0000 7f44e2ffd640 1 -- 172.21.15.113:0/3890118239 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc031b70 con 0x7f44e40b73c0 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 -- 172.21.15.113:0/3890118239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e40b73c0 msgr2=0x7f44e40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 --2- 172.21.15.113:0/3890118239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e40b73c0 0x7f44e40b7830 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009d90 tx=0x7f44dc02a5d0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 -- 172.21.15.113:0/3890118239 shutdown_connections 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 --2- 172.21.15.113:0/3890118239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e40b73c0 0x7f44e40b7830 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 --2- 172.21.15.113:0/3890118239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e40b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 -- 172.21.15.113:0/3890118239 >> 172.21.15.113:0/3890118239 conn(0x7f44e406b720 msgr2=0x7f44e40d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 -- 172.21.15.113:0/3890118239 shutdown_connections 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.747+0000 7f44e9c31640 1 -- 172.21.15.113:0/3890118239 wait complete. 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.748+0000 7f44e9c31640 1 Processor -- start 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.748+0000 7f44e9c31640 1 -- start start 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e9c31640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e414ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e9c31640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44e40b73c0 0x7f44e40ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e9c31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e415e510 0x7f44e4153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e9c31640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44e40d3ea0 con 0x7f44e415e510 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e9c31640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44e40d3980 con 0x7f44e40b73c0 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e9c31640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44e40d2a90 con 0x7f44e40b5ca0 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e415e510 0x7f44e4153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e415e510 0x7f44e4153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35138/0 (socket says 172.21.15.113:35138) 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 -- 172.21.15.113:0/2291650705 learned_addr learned my addr 172.21.15.113:0/2291650705 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e37fe640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e414ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44daffd640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44e40b73c0 0x7f44e40ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 -- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 msgr2=0x7f44e414ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e414ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 -- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44e40b73c0 msgr2=0x7f44e40ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44e40b73c0 0x7f44e40ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44e3fff640 1 -- 172.21.15.113:0/2291650705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e4160ce0 con 0x7f44e415e510 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.749+0000 7f44daffd640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44e40b73c0 0x7f44e40ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.750+0000 7f44e3fff640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e415e510 0x7f44e4153d00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009d90 tx=0x7f44dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.750+0000 7f44e0ff9640 1 -- 172.21.15.113:0/2291650705 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc03f070 con 0x7f44e415e510 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.750+0000 7f44e0ff9640 1 -- 172.21.15.113:0/2291650705 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc030be0 con 0x7f44e415e510 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.750+0000 7f44e0ff9640 1 -- 172.21.15.113:0/2291650705 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc009450 con 0x7f44e415e510 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.750+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e4160fc0 con 0x7f44e415e510 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.750+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e4161580 con 0x7f44e415e510 2024-04-25T00:04:27.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.751+0000 7f44e0ff9640 1 -- 172.21.15.113:0/2291650705 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44dc003990 con 0x7f44e415e510 2024-04-25T00:04:27.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.752+0000 7f44e0ff9640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44c406be30 0x7f44c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.752+0000 7f44e37fe640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44c406be30 0x7f44c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.752+0000 7f44e0ff9640 1 -- 172.21.15.113:0/2291650705 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f44dc0ab910 con 0x7f44e415e510 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.752+0000 7f44e37fe640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44c406be30 0x7f44c406e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f44d4007fd0 tx=0x7f44d4007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.753+0000 7f44eca1b740 1 Processor -- start 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.753+0000 7f44eca1b740 1 -- start start 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e7ab80 0x563c54e2d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e5d560 0x563c54e32370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44eca1b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563c54d96330 con 0x563c54e5dca0 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c54d3eab0 con 0x563c54e7ab80 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563c54d922f0 con 0x563c54e5d560 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e5d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35146/0 (socket says 172.21.15.113:35146) 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 -- 172.21.15.113:0/3683954125 learned_addr learned my addr 172.21.15.113:0/3683954125 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44e3fff640 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e5d560 0x563c54e32370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44e37fe640 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e7ab80 0x563c54e2d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 -- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e5d560 msgr2=0x563c54e32370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e5d560 0x563c54e32370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 -- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e7ab80 msgr2=0x563c54e2d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e7ab80 0x563c54e2d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 -- 172.21.15.113:0/3683954125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c54b747d0 con 0x563c54e5dca0 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44daffd640 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e5d020 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b950 tx=0x7f44cc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe5db9477f76db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44d97fa640 1 -- 172.21.15.113:0/3683954125 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc018020 con 0x563c54e5dca0 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44d97fa640 1 -- 172.21.15.113:0/3683954125 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc00fa80 con 0x563c54e5dca0 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.754+0000 7f44d97fa640 1 -- 172.21.15.113:0/3683954125 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00fe30 con 0x563c54e5dca0 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 -- 172.21.15.113:0/3683954125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 msgr2=0x563c54e5d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e5d020 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b950 tx=0x7f44cc00be10 comp rx=0 tx=0).stop 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 -- 172.21.15.113:0/3683954125 shutdown_connections 2024-04-25T00:04:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e5d560 0x563c54e32370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e5d020 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3683954125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e7ab80 0x563c54e2d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 -- 172.21.15.113:0/3683954125 >> 172.21.15.113:0/3683954125 conn(0x563c54e2f780 msgr2=0x563c54e78020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 -- 172.21.15.113:0/3683954125 shutdown_connections 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.755+0000 7f44eca1b740 1 -- 172.21.15.113:0/3683954125 wait complete. 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 -- start start 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 Processor -- start 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e4dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e2c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563c54e31930 con 0x563c54e5dca0 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c54d51f40 con 0x563c54e2c2b0 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563c54e30d20 con 0x563c54e7ab80 2024-04-25T00:04:27.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44eca1b740 10 client.0 _ll_register_callbacks cb 0x563c54aa9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.756+0000 7f44e37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e4dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e4dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35158/0 (socket says 172.21.15.113:35158) 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 -- 172.21.15.113:0/3232140802 learned_addr learned my addr 172.21.15.113:0/3232140802 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44daffd640 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e3fff640 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e2c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 -- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 msgr2=0x563c54e60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 -- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 msgr2=0x563c54e2c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e2c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 -- 172.21.15.113:0/3232140802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c54a6a0f0 con 0x563c54e5dca0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44e37fe640 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e4dbd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f44d4013440 tx=0x7f44d4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6788fc7f06fb02c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44c27fc640 1 -- 172.21.15.113:0/3232140802 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4026060 con 0x563c54e5dca0 2024-04-25T00:04:27.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44c27fc640 1 -- 172.21.15.113:0/3232140802 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f44d4021aa0 con 0x563c54e5dca0 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44cc00fa80 con 0x563c54e5dca0 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44c27fc640 1 -- 172.21.15.113:0/3232140802 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f44d4019070 con 0x563c54e5dca0 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44eca1b740 10 client.7620 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x563c54bd4f60 con 0x563c54e5dca0 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44c27fc640 1 -- 172.21.15.113:0/3232140802 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc00fa80 con 0x563c54e5dca0 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44c27fc640 1 -- 172.21.15.113:0/3232140802 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f44d401f070 con 0x563c54e5dca0 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44eca1b740 10 client.7620 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.757+0000 7f44eca1b740 10 client.7620 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44eca1b740 20 client.7620 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e4b750 0x563c54e4bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44eca1b740 4 client.7620 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c54da3870 con 0x563c54e4b750 2024-04-25T00:04:27.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44daffd640 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e4b750 0x563c54e4bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44c27fc640 10 client.7620 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44daffd640 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e4b750 0x563c54e4bba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74aa8caa7f49a8b3 server_cookie=9b5fa50edb1ece41 in_seq=0 out_seq=0 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44c27fc640 1 -- 172.21.15.113:0/3232140802 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c54da3870 con 0x563c54e4b750 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.758+0000 7f44c27fc640 10 client.7620 handle_command_reply: tid=0 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 client.7620 shutdown 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 msgr2=0x563c54e4dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e4dbd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f44d4013440 tx=0x7f44d4000ed0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 shutdown_connections 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e4b750 0x563c54e4bba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e2c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44c27fc640 0 client.7620 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 --2- 172.21.15.113:0/3232140802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e4dbd0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 >> 172.21.15.113:0/3232140802 conn(0x563c54e5d560 msgr2=0x563c54e5a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 shutdown_connections 2024-04-25T00:04:27.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 1 -- 172.21.15.113:0/3232140802 wait complete. 2024-04-25T00:04:27.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.759+0000 7f44eca1b740 20 client.7620 trim_cache size 0 max 16384 2024-04-25T00:04:27.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.776+0000 7f44eca1b740 1 Processor -- start 2024-04-25T00:04:27.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.776+0000 7f44eca1b740 1 -- start start 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e74db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e76bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44eca1b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4200920 con 0x563c54e5dca0 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41ffad0 con 0x563c54e2c2b0 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44e415fc50 con 0x563c54e7ab80 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e74db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e74db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35172/0 (socket says 172.21.15.113:35172) 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 -- 172.21.15.113:0/4032167009 learned_addr learned my addr 172.21.15.113:0/4032167009 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44e37fe640 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44e3fff640 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e76bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 -- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 msgr2=0x563c54e76bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e76bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 -- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 msgr2=0x563c54e4d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e4d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 -- 172.21.15.113:0/4032167009 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c54e308c0 con 0x563c54e5dca0 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44e3fff640 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44daffd640 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e74db0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f44cc017040 tx=0x7f44cc009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a2c753ef0e5185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44d8ff9640 1 -- 172.21.15.113:0/4032167009 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc018040 con 0x563c54e5dca0 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44d8ff9640 1 -- 172.21.15.113:0/4032167009 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc02f070 con 0x563c54e5dca0 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.777+0000 7f44d8ff9640 1 -- 172.21.15.113:0/4032167009 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc004040 con 0x563c54e5dca0 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 -- 172.21.15.113:0/4032167009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 msgr2=0x563c54e74db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e74db0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f44cc017040 tx=0x7f44cc009000 comp rx=0 tx=0).stop 2024-04-25T00:04:27.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 -- 172.21.15.113:0/4032167009 shutdown_connections 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e7ab80 0x563c54e76bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e5dca0 0x563c54e74db0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 --2- 172.21.15.113:0/4032167009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e2c2b0 0x563c54e4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 -- 172.21.15.113:0/4032167009 >> 172.21.15.113:0/4032167009 conn(0x563c54e5d560 msgr2=0x563c54e28790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 -- 172.21.15.113:0/4032167009 shutdown_connections 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 -- 172.21.15.113:0/4032167009 wait complete. 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.778+0000 7f44eca1b740 1 -- start start 2024-04-25T00:04:27.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 1 Processor -- start 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e2c2b0 0x563c54eec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e5dca0 0x563c54ef2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e7ab80 0x563c54e58920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563c54d39730 con 0x563c54e7ab80 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41602a0 con 0x563c54e5dca0 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563c54e2c8d0 con 0x563c54e2c2b0 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44eca1b740 10 client.0 _ll_register_callbacks cb 0x7f44d4007060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e7ab80 0x563c54e58920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e7ab80 0x563c54e58920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35180/0 (socket says 172.21.15.113:35180) 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44e3fff640 1 -- 172.21.15.113:0/1530753142 learned_addr learned my addr 172.21.15.113:0/1530753142 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44e37fe640 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e2c2b0 0x563c54eec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.779+0000 7f44daffd640 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e5dca0 0x563c54ef2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44e3fff640 1 -- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e2c2b0 msgr2=0x563c54eec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44e3fff640 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e2c2b0 0x563c54eec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44e3fff640 1 -- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e5dca0 msgr2=0x563c54ef2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44e3fff640 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e5dca0 0x563c54ef2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44e3fff640 1 -- 172.21.15.113:0/1530753142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c54a431b0 con 0x563c54e7ab80 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44e3fff640 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e7ab80 0x563c54e58920 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f44dc079a00 tx=0x7f44dc079d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d45bffa8790884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44c27fc640 1 -- 172.21.15.113:0/1530753142 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc0a7070 con 0x563c54e7ab80 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44c27fc640 1 -- 172.21.15.113:0/1530753142 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f44dc079f60 con 0x563c54e7ab80 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44cc02f070 con 0x563c54e7ab80 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44c27fc640 1 -- 172.21.15.113:0/1530753142 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f44dc048040 con 0x563c54e7ab80 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 10 client.7626 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x563c54ef2a60 con 0x563c54e7ab80 2024-04-25T00:04:27.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44c27fc640 1 -- 172.21.15.113:0/1530753142 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc02f070 con 0x563c54e7ab80 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44c27fc640 1 -- 172.21.15.113:0/1530753142 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f44dc04b060 con 0x563c54e7ab80 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 10 client.7626 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 10 client.7626 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 20 client.7626 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e58e60 0x563c54e76b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 4 client.7626 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x563c54d9dbc0 con 0x563c54e58e60 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.780+0000 7f44e37fe640 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e58e60 0x563c54e76b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44c27fc640 10 client.7626 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44e37fe640 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e58e60 0x563c54e76b60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68bc33ba951a051 server_cookie=dc9b8b045d1cef94 in_seq=0 out_seq=0 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44c27fc640 1 -- 172.21.15.113:0/1530753142 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563c54d9dbc0 con 0x563c54e58e60 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44c27fc640 10 client.7626 handle_command_reply: tid=0 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 client.7626 shutdown 2024-04-25T00:04:27.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e7ab80 msgr2=0x563c54e58920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e7ab80 0x563c54e58920 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f44dc079a00 tx=0x7f44dc079d00 comp rx=0 tx=0).stop 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 shutdown_connections 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x563c54e58e60 0x563c54e76b60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c54e5dca0 0x563c54ef2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c54e7ab80 0x563c54e58920 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44c27fc640 0 client.7626 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 --2- 172.21.15.113:0/1530753142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c54e2c2b0 0x563c54eec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 >> 172.21.15.113:0/1530753142 conn(0x563c54e5d560 msgr2=0x563c54e28790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 shutdown_connections 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 1 -- 172.21.15.113:0/1530753142 wait complete. 2024-04-25T00:04:27.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.781+0000 7f44eca1b740 20 client.7626 trim_cache size 0 max 16384 2024-04-25T00:04:27.790 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:04:27.790 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44c406be30 msgr2=0x7f44c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44c406be30 0x7f44c406e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f44d4007fd0 tx=0x7f44d4007de0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e415e510 msgr2=0x7f44e4153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e415e510 0x7f44e4153d00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009d90 tx=0x7f44dc004a40 comp rx=0 tx=0).stop 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 shutdown_connections 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44c406be30 0x7f44c406e2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44e40b73c0 0x7f44e40ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44e415e510 0x7f44e4153d00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 --2- 172.21.15.113:0/2291650705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44e40b5ca0 0x7f44e414ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 >> 172.21.15.113:0/2291650705 conn(0x7f44e406b720 msgr2=0x7f44e415b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 shutdown_connections 2024-04-25T00:04:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.782+0000 7f44e9c31640 1 -- 172.21.15.113:0/2291650705 wait complete. 2024-04-25T00:04:27.793 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T00:04:27.794 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T00:04:27.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.890+0000 7fe92fce1640 1 Processor -- start 2024-04-25T00:04:27.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.891+0000 7fe92fce1640 1 -- start start 2024-04-25T00:04:27.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.891+0000 7fe92fce1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe92815e2d0 0x7fe92815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.891+0000 7fe92fce1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92815f4f0 con 0x7fe9280b4a40 2024-04-25T00:04:27.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.891+0000 7fe92fce1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92815f7d0 con 0x7fe9280b3080 2024-04-25T00:04:27.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.891+0000 7fe92fce1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe9280cde00 con 0x7fe92815e2d0 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.891+0000 7fe92e27e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe9280b4a40 0x7fe9280ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39708/0 (socket says 172.21.15.113:39708) 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92e27e640 1 -- 172.21.15.113:0/2524153209 learned_addr learned my addr 172.21.15.113:0/2524153209 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92da7d640 1 --2- 172.21.15.113:0/2524153209 >> v2:172.21.15.178:3300/0 conn(0x7fe92815e2d0 0x7fe92815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92ca7b640 1 -- 172.21.15.113:0/2524153209 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283350741 0 0) 0x7fe92815f4f0 con 0x7fe9280b4a40 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92ca7b640 1 -- 172.21.15.113:0/2524153209 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe914003520 con 0x7fe9280b4a40 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92ca7b640 1 -- 172.21.15.113:0/2524153209 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4128237457 0 0) 0x7fe914003520 con 0x7fe9280b4a40 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92da7d640 1 -- 172.21.15.113:0/2524153209 >> v1:172.21.15.178:6789/0 conn(0x7fe9280b3080 legacy=0x7fe9280b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92da7d640 1 -- 172.21.15.113:0/2524153209 >> v1:172.21.15.113:6789/0 conn(0x7fe9280b4a40 legacy=0x7fe9280ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.892+0000 7fe92da7d640 1 -- 172.21.15.113:0/2524153209 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92815fab0 con 0x7fe92815e2d0 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92da7d640 1 --2- 172.21.15.113:0/2524153209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe92815e2d0 0x7fe92815ebf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe9280cdf80 tx=0x7fe92402a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7eaad2fa07817a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92ca7b640 1 -- 172.21.15.113:0/2524153209 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035020 con 0x7fe92815e2d0 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92ca7b640 1 -- 172.21.15.113:0/2524153209 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe924007540 con 0x7fe92815e2d0 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92ca7b640 1 -- 172.21.15.113:0/2524153209 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920001670 con 0x7fe92815e2d0 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92fce1640 1 -- 172.21.15.113:0/2524153209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe92815e2d0 msgr2=0x7fe92815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92fce1640 1 --2- 172.21.15.113:0/2524153209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe92815e2d0 0x7fe92815ebf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe9280cdf80 tx=0x7fe92402a590 comp rx=0 tx=0).stop 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92fce1640 1 -- 172.21.15.113:0/2524153209 shutdown_connections 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92fce1640 1 --2- 172.21.15.113:0/2524153209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe92815e2d0 0x7fe92815ebf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.893+0000 7fe92fce1640 1 -- 172.21.15.113:0/2524153209 >> 172.21.15.113:0/2524153209 conn(0x7fe92806b0f0 msgr2=0x7fe92806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.894+0000 7fe92fce1640 1 -- 172.21.15.113:0/2524153209 shutdown_connections 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.894+0000 7fe92fce1640 1 -- 172.21.15.113:0/2524153209 wait complete. 2024-04-25T00:04:27.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 Processor -- start 2024-04-25T00:04:27.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 -- start start 2024-04-25T00:04:27.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9280b3080 0x7fe928151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9280b4a40 0x7fe92814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92815e2d0 0x7fe9280ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe92815f2b0 con 0x7fe92815e2d0 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe92815f130 con 0x7fe9280b3080 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92fce1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe92815e6b0 con 0x7fe9280b4a40 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92e27e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92815e2d0 0x7fe9280ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92e27e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92815e2d0 0x7fe9280ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35184/0 (socket says 172.21.15.113:35184) 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92e27e640 1 -- 172.21.15.113:0/3101854382 learned_addr learned my addr 172.21.15.113:0/3101854382 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.895+0000 7fe92da7d640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9280b3080 0x7fe928151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92d27c640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9280b4a40 0x7fe92814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92e27e640 1 -- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9280b4a40 msgr2=0x7fe92814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92e27e640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9280b4a40 0x7fe92814ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92e27e640 1 -- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9280b3080 msgr2=0x7fe928151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92e27e640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9280b3080 0x7fe928151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92e27e640 1 -- 172.21.15.113:0/3101854382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9280aeeb0 con 0x7fe92815e2d0 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92e27e640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92815e2d0 0x7fe9280ae4f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c550 tx=0x7fe92000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe91e7fc640 1 -- 172.21.15.113:0/3101854382 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920018070 con 0x7fe92815e2d0 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe91e7fc640 1 -- 172.21.15.113:0/3101854382 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9200074e0 con 0x7fe92815e2d0 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9280af190 con 0x7fe92815e2d0 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe91e7fc640 1 -- 172.21.15.113:0/3101854382 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920004a20 con 0x7fe92815e2d0 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.896+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe928150e80 con 0x7fe92815e2d0 2024-04-25T00:04:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.897+0000 7fe91e7fc640 1 -- 172.21.15.113:0/3101854382 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe92001b070 con 0x7fe92815e2d0 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.898+0000 7fe91e7fc640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe90c06be30 0x7fe90c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.898+0000 7fe91e7fc640 1 -- 172.21.15.113:0/3101854382 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fe92008a1c0 con 0x7fe92815e2d0 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.898+0000 7fe92da7d640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe90c06be30 0x7fe90c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.898+0000 7fe92da7d640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe90c06be30 0x7fe90c06e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe924033000 tx=0x7fe92402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.899+0000 7fe9329f4740 1 Processor -- start 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.899+0000 7fe9329f4740 1 -- start start 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5c1600 0x562dcb5c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb579a40 0x562dcb5bfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe9329f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562dcb4dd6b0 con 0x562dcb579a40 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562dcb485880 con 0x562dcb5c1600 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562dcb4d9670 con 0x562dcb5a3d60 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb579a40 0x562dcb5bfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb579a40 0x562dcb5bfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35198/0 (socket says 172.21.15.113:35198) 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 -- 172.21.15.113:0/499561770 learned_addr learned my addr 172.21.15.113:0/499561770 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92da7d640 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 -- 172.21.15.113:0/499561770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 msgr2=0x562dcb5a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 -- 172.21.15.113:0/499561770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5c1600 msgr2=0x562dcb5c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5c1600 0x562dcb5c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 -- 172.21.15.113:0/499561770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcb2bb7d0 con 0x562dcb579a40 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe92e27e640 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb579a40 0x562dcb5bfcc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe9200584a0 tx=0x7fe920058510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da38c2f0207b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe90b7fe640 1 -- 172.21.15.113:0/499561770 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920056810 con 0x562dcb579a40 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe90b7fe640 1 -- 172.21.15.113:0/499561770 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe920054b80 con 0x562dcb579a40 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.900+0000 7fe90b7fe640 1 -- 172.21.15.113:0/499561770 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920004de0 con 0x562dcb579a40 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 -- 172.21.15.113:0/499561770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb579a40 msgr2=0x562dcb5bfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb579a40 0x562dcb5bfcc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe9200584a0 tx=0x7fe920058510 comp rx=0 tx=0).stop 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 -- 172.21.15.113:0/499561770 shutdown_connections 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5c1600 0x562dcb5c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb579a40 0x562dcb5bfcc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 --2- 172.21.15.113:0/499561770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 -- 172.21.15.113:0/499561770 >> 172.21.15.113:0/499561770 conn(0x562dcb576600 msgr2=0x562dcb5beb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 -- 172.21.15.113:0/499561770 shutdown_connections 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 -- 172.21.15.113:0/499561770 wait complete. 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.901+0000 7fe9329f4740 1 -- start start 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.902+0000 7fe9329f4740 1 Processor -- start 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.902+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.902+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5c1600 0x562dcb5bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.902+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a75d0 0x562dcb5a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.902+0000 7fe9329f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562dcb57a7e0 con 0x562dcb5c1600 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.902+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562dcb498f50 con 0x562dcb5a75d0 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.902+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562dcb574530 con 0x562dcb5a3d60 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe9329f4740 10 client.0 _ll_register_callbacks cb 0x562dcb4d4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5c1600 0x562dcb5bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92da7d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5c1600 0x562dcb5bd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35206/0 (socket says 172.21.15.113:35206) 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 -- 172.21.15.113:0/857156244 learned_addr learned my addr 172.21.15.113:0/857156244 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92e27e640 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a75d0 0x562dcb5a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 -- 172.21.15.113:0/857156244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 msgr2=0x562dcb5a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 -- 172.21.15.113:0/857156244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a75d0 msgr2=0x562dcb5a7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a75d0 0x562dcb5a7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 -- 172.21.15.113:0/857156244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcb1f13f0 con 0x562dcb5c1600 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe92d27c640 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5c1600 0x562dcb5bd6f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe918007a20 tx=0x7fe91800f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9939ebd5ee6f1693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe908ff9640 1 -- 172.21.15.113:0/857156244 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91801a060 con 0x562dcb5c1600 2024-04-25T00:04:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe908ff9640 1 -- 172.21.15.113:0/857156244 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fe918005320 con 0x562dcb5c1600 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe920054b80 con 0x562dcb5c1600 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe908ff9640 1 -- 172.21.15.113:0/857156244 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe918007030 con 0x562dcb5c1600 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe9329f4740 10 client.7638 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x562dcb1b10f0 con 0x562dcb5c1600 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe908ff9640 1 -- 172.21.15.113:0/857156244 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe920054b80 con 0x562dcb5c1600 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.903+0000 7fe908ff9640 1 -- 172.21.15.113:0/857156244 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fe9180071d0 con 0x562dcb5c1600 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe9329f4740 10 client.7638 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe9329f4740 10 client.7638 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe9329f4740 20 client.7638 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe9329f4740 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5b6640 0x562dcb5b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe9329f4740 4 client.7638 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dcb4eabf0 con 0x562dcb5b6640 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe92da7d640 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5b6640 0x562dcb5b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe908ff9640 10 client.7638 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.904+0000 7fe92da7d640 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5b6640 0x562dcb5b6a50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec2476f4eb715ef server_cookie=9bbef598bce2b487 in_seq=0 out_seq=0 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe908ff9640 1 -- 172.21.15.113:0/857156244 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dcb4eabf0 con 0x562dcb5b6640 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe908ff9640 10 client.7638 handle_command_reply: tid=0 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 client.7638 shutdown 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5c1600 msgr2=0x562dcb5bd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5c1600 0x562dcb5bd6f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe918007a20 tx=0x7fe91800f8e0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 shutdown_connections 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5b6640 0x562dcb5b6a50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a75d0 0x562dcb5a7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5c1600 0x562dcb5bd6f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe908ff9640 0 client.7638 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 --2- 172.21.15.113:0/857156244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5a3d60 0x562dcb5a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 >> 172.21.15.113:0/857156244 conn(0x562dcb579a40 msgr2=0x562dcb5c20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 shutdown_connections 2024-04-25T00:04:27.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 1 -- 172.21.15.113:0/857156244 wait complete. 2024-04-25T00:04:27.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.905+0000 7fe9329f4740 20 client.7638 trim_cache size 0 max 16384 2024-04-25T00:04:27.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.922+0000 7fe9329f4740 1 Processor -- start 2024-04-25T00:04:27.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.922+0000 7fe9329f4740 1 -- start start 2024-04-25T00:04:27.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb5e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe9329f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe928200790 con 0x562dcb5a75d0 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281ff810 con 0x562dcb5a3d60 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe92806d240 con 0x562dcb5b6640 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb5e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb5e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35218/0 (socket says 172.21.15.113:35218) 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 -- 172.21.15.113:0/1256330951 learned_addr learned my addr 172.21.15.113:0/1256330951 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92da7d640 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92e27e640 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 -- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 msgr2=0x562dcb5b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 -- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 msgr2=0x562dcb5d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92d27c640 1 -- 172.21.15.113:0/1256330951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcb5702a0 con 0x562dcb5a75d0 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.923+0000 7fe92da7d640 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe92d27c640 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb5e2d40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe9180077f0 tx=0x7fe91800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83573f559ca3617b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe90affd640 1 -- 172.21.15.113:0/1256330951 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180053e0 con 0x562dcb5a75d0 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe90affd640 1 -- 172.21.15.113:0/1256330951 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9180071d0 con 0x562dcb5a75d0 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe90affd640 1 -- 172.21.15.113:0/1256330951 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180069b0 con 0x562dcb5a75d0 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 -- 172.21.15.113:0/1256330951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 msgr2=0x562dcb5e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb5e2d40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe9180077f0 tx=0x7fe91800a000 comp rx=0 tx=0).stop 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 -- 172.21.15.113:0/1256330951 shutdown_connections 2024-04-25T00:04:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb5e2d40 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 --2- 172.21.15.113:0/1256330951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 -- 172.21.15.113:0/1256330951 >> 172.21.15.113:0/1256330951 conn(0x562dcb579a40 msgr2=0x562dcb578d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 -- 172.21.15.113:0/1256330951 shutdown_connections 2024-04-25T00:04:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.924+0000 7fe9329f4740 1 -- 172.21.15.113:0/1256330951 wait complete. 2024-04-25T00:04:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.925+0000 7fe9329f4740 1 -- start start 2024-04-25T00:04:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.925+0000 7fe9329f4740 1 Processor -- start 2024-04-25T00:04:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb621a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe9329f4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe9329f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562dcb5a6ba0 con 0x562dcb5a75d0 2024-04-25T00:04:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe92806d890 con 0x562dcb5a3d60 2024-04-25T00:04:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe9329f4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562dcb5c3010 con 0x562dcb5b6640 2024-04-25T00:04:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe9329f4740 10 client.0 _ll_register_callbacks cb 0x7fe918001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb621a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb621a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35234/0 (socket says 172.21.15.113:35234) 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 -- 172.21.15.113:0/3130804140 learned_addr learned my addr 172.21.15.113:0/3130804140 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92da7d640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92e27e640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 -- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 msgr2=0x562dcb5b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 -- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 msgr2=0x562dcb5d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 -- 172.21.15.113:0/3130804140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcb5a4a20 con 0x562dcb5a75d0 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92da7d640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:27.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe92d27c640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb621a40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe918018040 tx=0x7fe918000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b919e33dc0dab25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.926+0000 7fe908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe908ff9640 1 -- 172.21.15.113:0/3130804140 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918006e90 con 0x562dcb5a75d0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe908ff9640 1 -- 172.21.15.113:0/3130804140 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fe918025040 con 0x562dcb5a75d0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9180071d0 con 0x562dcb5a75d0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe908ff9640 1 -- 172.21.15.113:0/3130804140 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe918039050 con 0x562dcb5a75d0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 10 client.7644 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x562dcb5e2d90 con 0x562dcb5a75d0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe908ff9640 1 -- 172.21.15.113:0/3130804140 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9180071d0 con 0x562dcb5a75d0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe908ff9640 1 -- 172.21.15.113:0/3130804140 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fe91803c020 con 0x562dcb5a75d0 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 10 client.7644 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 10 client.7644 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 20 client.7644 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5c1600 0x562dcb5b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 4 client.7644 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562dcb575420 con 0x562dcb5c1600 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe92da7d640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5c1600 0x562dcb5b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe908ff9640 10 client.7644 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.927+0000 7fe92da7d640 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5c1600 0x562dcb5b4a40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8c7f4865c0ac75 server_cookie=4a718d302013b2d5 in_seq=0 out_seq=0 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe908ff9640 1 -- 172.21.15.113:0/3130804140 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+276 (crc 0 0 0) 0x562dcb575420 con 0x562dcb5c1600 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe908ff9640 10 client.7644 handle_command_reply: tid=0 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 client.7644 shutdown 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 msgr2=0x562dcb621a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb621a40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe918018040 tx=0x7fe918000ed0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 shutdown_connections 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x562dcb5c1600 0x562dcb5b4a40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562dcb5b6640 0x562dcb5b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe908ff9640 0 client.7644 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562dcb5a75d0 0x562dcb621a40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 --2- 172.21.15.113:0/3130804140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562dcb5a3d60 0x562dcb5d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 >> 172.21.15.113:0/3130804140 conn(0x562dcb579a40 msgr2=0x562dcb5b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 shutdown_connections 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 1 -- 172.21.15.113:0/3130804140 wait complete. 2024-04-25T00:04:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe9329f4740 20 client.7644 trim_cache size 0 max 16384 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: "7c9bc5c3-1434-4ded-9e0e-b2a88c4fd9c6": { 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_control_test_path", 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "7c9bc5c3-1434-4ded-9e0e-b2a88c4fd9c6", 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive" 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:27.938 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:27.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe90c06be30 msgr2=0x7fe90c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe90c06be30 0x7fe90c06e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe924033000 tx=0x7fe92402f000 comp rx=0 tx=0).stop 2024-04-25T00:04:27.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92815e2d0 msgr2=0x7fe9280ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:27.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92815e2d0 0x7fe9280ae4f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c550 tx=0x7fe92000ca10 comp rx=0 tx=0).stop 2024-04-25T00:04:27.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 shutdown_connections 2024-04-25T00:04:27.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe90c06be30 0x7fe90c06e2f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92815e2d0 0x7fe9280ae4f0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9280b4a40 0x7fe92814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 --2- 172.21.15.113:0/3101854382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9280b3080 0x7fe928151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 >> 172.21.15.113:0/3101854382 conn(0x7fe92806b0f0 msgr2=0x7fe9280afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.928+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 shutdown_connections 2024-04-25T00:04:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:27.929+0000 7fe92fce1640 1 -- 172.21.15.113:0/3101854382 wait complete. 2024-04-25T00:04:32.942 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T00:04:32.942 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T00:04:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.053+0000 7f8c80cbd640 1 Processor -- start 2024-04-25T00:04:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.053+0000 7f8c80cbd640 1 -- start start 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c80cbd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c7c13f050 0x7f8c7c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c80cbd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8c7c0d04f0 0x7f8c7c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c80cbd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c15d6b0 con 0x7f8c7c1604d0 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c80cbd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c0d09a0 con 0x7f8c7c13f050 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c80cbd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c15c6a0 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c7a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c7c13f050 0x7f8c7c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c7a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c7c13f050 0x7f8c7c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35238/0 (socket says 172.21.15.113:35238) 2024-04-25T00:04:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c7a59c640 1 -- 172.21.15.113:0/757152058 learned_addr learned my addr 172.21.15.113:0/757152058 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c73fff640 1 --2- 172.21.15.113:0/757152058 >> v2:172.21.15.178:3300/0 conn(0x7f8c7c0d04f0 0x7f8c7c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c7a59c640 1 -- 172.21.15.113:0/757152058 >> v2:172.21.15.178:3300/0 conn(0x7f8c7c0d04f0 msgr2=0x7f8c7c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/757152058 >> v2:172.21.15.178:3300/0 conn(0x7f8c7c0d04f0 0x7f8c7c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c7a59c640 1 -- 172.21.15.113:0/757152058 >> v1:172.21.15.113:6789/0 conn(0x7f8c7c1604d0 legacy=0x7f8c7c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c79d9b640 1 -- 172.21.15.113:0/757152058 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030200757 0 0) 0x7f8c7c15d6b0 con 0x7f8c7c1604d0 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.054+0000 7f8c7a59c640 1 -- 172.21.15.113:0/757152058 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c15c8f0 con 0x7f8c7c13f050 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.055+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/757152058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c13f050 0x7f8c7c0cffb0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c06f460 tx=0x7f8c6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850f6a9059fed1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.055+0000 7f8c79d9b640 1 -- 172.21.15.113:0/757152058 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c037070 con 0x7f8c7c13f050 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.055+0000 7f8c79d9b640 1 -- 172.21.15.113:0/757152058 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c032440 con 0x7f8c7c13f050 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.055+0000 7f8c79d9b640 1 -- 172.21.15.113:0/757152058 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c032810 con 0x7f8c7c13f050 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 -- 172.21.15.113:0/757152058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c13f050 msgr2=0x7f8c7c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/757152058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c13f050 0x7f8c7c0cffb0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c06f460 tx=0x7f8c6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 -- 172.21.15.113:0/757152058 shutdown_connections 2024-04-25T00:04:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/757152058 >> v2:172.21.15.178:3300/0 conn(0x7f8c7c0d04f0 0x7f8c7c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/757152058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c13f050 0x7f8c7c0cffb0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 -- 172.21.15.113:0/757152058 >> 172.21.15.113:0/757152058 conn(0x7f8c7c06b0f0 msgr2=0x7f8c7c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 -- 172.21.15.113:0/757152058 shutdown_connections 2024-04-25T00:04:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.056+0000 7f8c80cbd640 1 -- 172.21.15.113:0/757152058 wait complete. 2024-04-25T00:04:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.057+0000 7f8c80cbd640 1 Processor -- start 2024-04-25T00:04:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.057+0000 7f8c80cbd640 1 -- start start 2024-04-25T00:04:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c80cbd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c0d04f0 0x7f8c7c1511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c80cbd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7c13f050 0x7f8c7c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c80cbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c7c1604d0 0x7f8c7c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c80cbd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c15d190 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c80cbd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c15ceb0 con 0x7f8c7c13f050 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c80cbd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7c13f430 con 0x7f8c7c1604d0 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c0d04f0 0x7f8c7c1511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c0d04f0 0x7f8c7c1511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35242/0 (socket says 172.21.15.113:35242) 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3058154037 learned_addr learned my addr 172.21.15.113:0/3058154037 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c7c1604d0 0x7f8c7c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c73fff640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7c13f050 0x7f8c7c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c7c1604d0 msgr2=0x7f8c7c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c7c1604d0 0x7f8c7c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7c13f050 msgr2=0x7f8c7c15b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7c13f050 0x7f8c7c15b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3058154037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c15aae0 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c0d04f0 0x7f8c7c1511a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02a220 tx=0x7f8c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.058+0000 7f8c72ffd640 1 -- 172.21.15.113:0/3058154037 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c041020 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.059+0000 7f8c72ffd640 1 -- 172.21.15.113:0/3058154037 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c0306c0 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.059+0000 7f8c72ffd640 1 -- 172.21.15.113:0/3058154037 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c037040 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.059+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c1564f0 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.059+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c15b040 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.060+0000 7f8c72ffd640 1 -- 172.21.15.113:0/3058154037 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c6c044020 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.060+0000 7f8c72ffd640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c5c06be30 0x7f8c5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.060+0000 7f8c72ffd640 1 -- 172.21.15.113:0/3058154037 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8c6c0ab6a0 con 0x7f8c7c0d04f0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.060+0000 7f8c73fff640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c5c06be30 0x7f8c5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.061+0000 7f8c73fff640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c5c06be30 0x7f8c5c06e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c1f5060 tx=0x7f8c64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 Processor -- start 2024-04-25T00:04:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 -- start start 2024-04-25T00:04:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee43140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee60b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee18a40 0x5559bee5ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5559bed7c6b0 con 0x5559bee60600 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5559bed24880 con 0x5559bee42d60 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.062+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5559bed78670 con 0x5559bee18a40 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee60b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee60b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35244/0 (socket says 172.21.15.113:35244) 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 -- 172.21.15.113:0/1893975286 learned_addr learned my addr 172.21.15.113:0/1893975286 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee18a40 0x5559bee5ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee43140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 -- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee18a40 msgr2=0x5559bee5ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee18a40 0x5559bee5ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 -- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 msgr2=0x5559bee43140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee43140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 -- 172.21.15.113:0/1893975286 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559beb5a7d0 con 0x5559bee60600 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee43140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee60b50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b630 tx=0x7f8c64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81310e76f29721d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1893975286 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6401b050 con 0x5559bee60600 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1893975286 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c64025070 con 0x5559bee60600 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.063+0000 7f8c5bfff640 1 -- 172.21.15.113:0/1893975286 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64014de0 con 0x5559bee60600 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 -- 172.21.15.113:0/1893975286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 msgr2=0x5559bee60b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee60b50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b630 tx=0x7f8c64012000 comp rx=0 tx=0).stop 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 -- 172.21.15.113:0/1893975286 shutdown_connections 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee18a40 0x5559bee5ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee60b50 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1893975286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee43140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 -- 172.21.15.113:0/1893975286 >> 172.21.15.113:0/1893975286 conn(0x5559bee15600 msgr2=0x5559bee5db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 -- 172.21.15.113:0/1893975286 shutdown_connections 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.064+0000 7f8c839ce740 1 -- 172.21.15.113:0/1893975286 wait complete. 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.065+0000 7f8c839ce740 1 -- start start 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.065+0000 7f8c839ce740 1 Processor -- start 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.065+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee42d60 0x5559bee6e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559bee55610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee40e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5559bee197e0 con 0x5559bee42d60 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5559bed37f50 con 0x5559bee60600 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5559bee13530 con 0x5559bee3f250 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 10 client.0 _ll_register_callbacks cb 0x5559bed73000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee42d60 0x5559bee6e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee42d60 0x5559bee6e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35252/0 (socket says 172.21.15.113:35252) 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3498220153 learned_addr learned my addr 172.21.15.113:0/3498220153 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee40e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c73fff640 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559bee55610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 msgr2=0x5559bee40e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee40e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 msgr2=0x5559bee55610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559bee55610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 -- 172.21.15.113:0/3498220153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bea903f0 con 0x5559bee42d60 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee42d60 0x5559bee6e4b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0a9b40 tx=0x7f8c6c07a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ffa1184955def1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c597fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c597fa640 1 -- 172.21.15.113:0/3498220153 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c07dd00 con 0x5559bee42d60 2024-04-25T00:04:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c597fa640 1 -- 172.21.15.113:0/3498220153 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8c6c0a4be0 con 0x5559bee42d60 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c64025070 con 0x5559bee42d60 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c597fa640 1 -- 172.21.15.113:0/3498220153 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c082530 con 0x5559bee42d60 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 10 client.7659 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.066+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5559bea500f0 con 0x5559bee42d60 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c597fa640 1 -- 172.21.15.113:0/3498220153 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c64025070 con 0x5559bee42d60 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c597fa640 1 -- 172.21.15.113:0/3498220153 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f8c6c03a030 con 0x5559bee42d60 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c839ce740 10 client.7659 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c839ce740 10 client.7659 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c839ce740 20 client.7659 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c839ce740 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559bee46bf0 0x5559bee43d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c839ce740 4 client.7659 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559bed89bf0 con 0x5559bee46bf0 2024-04-25T00:04:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c73fff640 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559bee46bf0 0x5559bee43d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c597fa640 10 client.7659 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.067+0000 7f8c73fff640 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559bee46bf0 0x5559bee43d10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfccd99344903111 server_cookie=c1c4425f70b730f in_seq=0 out_seq=0 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c597fa640 1 -- 172.21.15.113:0/3498220153 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559bed89bf0 con 0x5559bee46bf0 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c597fa640 10 client.7659 handle_command_reply: tid=0 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 client.7659 shutdown 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee42d60 msgr2=0x5559bee6e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee42d60 0x5559bee6e4b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0a9b40 tx=0x7f8c6c07a110 comp rx=0 tx=0).stop 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 shutdown_connections 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559bee46bf0 0x5559bee43d10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559bee55610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee40e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c597fa640 0 client.7659 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 --2- 172.21.15.113:0/3498220153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee42d60 0x5559bee6e4b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 >> 172.21.15.113:0/3498220153 conn(0x5559bee18a40 msgr2=0x5559bee610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 shutdown_connections 2024-04-25T00:04:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 1 -- 172.21.15.113:0/3498220153 wait complete. 2024-04-25T00:04:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.068+0000 7f8c839ce740 20 client.7659 trim_cache size 0 max 16384 2024-04-25T00:04:33.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.085+0000 7f8c839ce740 1 Processor -- start 2024-04-25T00:04:33.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c839ce740 1 -- start start 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee45310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee6df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee6e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c839ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c1fe520 con 0x5559bee60600 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c1fd540 con 0x5559bee42d60 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7c12ffe0 con 0x5559bee3f250 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c7ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee6e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c7ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee6e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35262/0 (socket says 172.21.15.113:35262) 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.086+0000 7f8c7ad9d640 1 -- 172.21.15.113:0/1679068877 learned_addr learned my addr 172.21.15.113:0/1679068877 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee45310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee6df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c7ad9d640 1 -- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 msgr2=0x5559bee45310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee45310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c7ad9d640 1 -- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 msgr2=0x5559bee6df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee6df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c7ad9d640 1 -- 172.21.15.113:0/1679068877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bee54d30 con 0x5559bee60600 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee6e480 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005150 tx=0x7f8c74005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e58fd174f529de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c5b7fe640 1 -- 172.21.15.113:0/1679068877 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401a070 con 0x5559bee60600 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c5b7fe640 1 -- 172.21.15.113:0/1679068877 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74013830 con 0x5559bee60600 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.087+0000 7f8c5b7fe640 1 -- 172.21.15.113:0/1679068877 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74013c00 con 0x5559bee60600 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 -- 172.21.15.113:0/1679068877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 msgr2=0x5559bee6e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee6e480 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005150 tx=0x7f8c74005610 comp rx=0 tx=0).stop 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 -- 172.21.15.113:0/1679068877 shutdown_connections 2024-04-25T00:04:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee42d60 0x5559bee6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee60600 0x5559bee6e480 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1679068877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee3f250 0x5559bee45310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 -- 172.21.15.113:0/1679068877 >> 172.21.15.113:0/1679068877 conn(0x5559bee18a40 msgr2=0x5559bee5c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 -- 172.21.15.113:0/1679068877 shutdown_connections 2024-04-25T00:04:33.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 -- 172.21.15.113:0/1679068877 wait complete. 2024-04-25T00:04:33.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 -- start start 2024-04-25T00:04:33.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.088+0000 7f8c839ce740 1 Processor -- start 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee3f250 0x5559bee5d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee42d60 0x5559bee6e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c839ce740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559beeda570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c839ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c12da40 con 0x5559bee3f250 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5559bed89bf0 con 0x5559bee60600 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c839ce740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5559bee74170 con 0x5559bee42d60 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c839ce740 10 client.0 _ll_register_callbacks cb 0x7f8c6c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c7a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee3f250 0x5559bee5d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c7a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee3f250 0x5559bee5d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35266/0 (socket says 172.21.15.113:35266) 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c7a59c640 1 -- 172.21.15.113:0/1664341408 learned_addr learned my addr 172.21.15.113:0/1664341408 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee42d60 0x5559bee6e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.089+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559beeda570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c7a59c640 1 -- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee42d60 msgr2=0x5559bee6e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee42d60 0x5559bee6e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c7a59c640 1 -- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 msgr2=0x5559beeda570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559beeda570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c7a59c640 1 -- 172.21.15.113:0/1664341408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bee6e880 con 0x5559bee3f250 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c7ad9d640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559beeda570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c7a59c640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee3f250 0x5559bee5d350 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c079770 tx=0x7f8c6c008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5128e7e0e47c83e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c597fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:33.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c597fa640 1 -- 172.21.15.113:0/1664341408 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c07dd00 con 0x5559bee3f250 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c597fa640 1 -- 172.21.15.113:0/1664341408 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8c6c078da0 con 0x5559bee3f250 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c74013830 con 0x5559bee3f250 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c597fa640 1 -- 172.21.15.113:0/1664341408 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c0792b0 con 0x5559bee3f250 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 10 client.7665 fetch_fsmap learned FSMap version 50 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5559bee5f3f0 con 0x5559bee3f250 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c597fa640 1 -- 172.21.15.113:0/1664341408 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74013830 con 0x5559bee3f250 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c597fa640 1 -- 172.21.15.113:0/1664341408 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f8c6c0756a0 con 0x5559bee3f250 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 10 client.7665 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 10 client.7665 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 20 client.7665 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559beedaab0 0x5559bee81440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 4 client.7665 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:04:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 --> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559bed83f40 con 0x5559beedaab0 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.090+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559beedaab0 0x5559bee81440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c597fa640 10 client.7665 ms_handle_connect on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c73fff640 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559beedaab0 0x5559bee81440 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee1165f1b85c49f server_cookie=f26bb67c3187fea4 in_seq=0 out_seq=0 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c597fa640 1 -- 172.21.15.113:0/1664341408 <== mds.0 v2:172.21.15.113:6835/2796979045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5559bed83f40 con 0x5559beedaab0 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c597fa640 10 client.7665 handle_command_reply: tid=0 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 client.7665 shutdown 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee3f250 msgr2=0x5559bee5d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee3f250 0x5559bee5d350 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c079770 tx=0x7f8c6c008070 comp rx=0 tx=0).stop 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 shutdown_connections 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:6835/2796979045,v1:172.21.15.113:6837/2796979045] conn(0x5559beedaab0 0x5559bee81440 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559bee60600 0x5559beeda570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559bee42d60 0x5559bee6e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 --2- 172.21.15.113:0/1664341408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5559bee3f250 0x5559bee5d350 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c597fa640 0 client.7665 ms_handle_reset on v2:172.21.15.113:6835/2796979045 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 >> 172.21.15.113:0/1664341408 conn(0x5559bee18a40 msgr2=0x5559bee42610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 shutdown_connections 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 1 -- 172.21.15.113:0/1664341408 wait complete. 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.091+0000 7f8c839ce740 20 client.7665 trim_cache size 0 max 16384 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:04:33.099 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c5c06be30 msgr2=0x7f8c5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c5c06be30 0x7f8c5c06e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c1f5060 tx=0x7f8c64008040 comp rx=0 tx=0).stop 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c0d04f0 msgr2=0x7f8c7c1511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c0d04f0 0x7f8c7c1511a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02a220 tx=0x7f8c6c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 shutdown_connections 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8c5c06be30 0x7f8c5c06e2f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c7c13f050 0x7f8c7c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c7c1604d0 0x7f8c7c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 --2- 172.21.15.113:0/3058154037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7c0d04f0 0x7f8c7c1511a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 >> 172.21.15.113:0/3058154037 conn(0x7f8c7c06b0f0 msgr2=0x7f8c7c0b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 shutdown_connections 2024-04-25T00:04:33.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.092+0000 7f8c80cbd640 1 -- 172.21.15.113:0/3058154037 wait complete. 2024-04-25T00:04:33.103 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:04:33.103 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.199+0000 7fd10102a640 1 Processor -- start 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.199+0000 7fd10102a640 1 -- start start 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd10102a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd0fc0b3fd0 0x7fd0fc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd10102a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 0x7fd0fc0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd10102a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0fc144cf0 con 0x7fd0fc144910 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd10102a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd0fc0b2f30 con 0x7fd0fc0b3fd0 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd10102a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd0fc0b3210 con 0x7fd0fc06e510 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0fad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd0fc144910 0x7fd0fc0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39728/0 (socket says 172.21.15.113:39728) 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0fad9d640 1 -- 172.21.15.113:0/1418471099 learned_addr learned my addr 172.21.15.113:0/1418471099 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0fb59e640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 0x7fd0fc0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0fa59c640 1 --2- 172.21.15.113:0/1418471099 >> v2:172.21.15.178:3300/0 conn(0x7fd0fc0b3fd0 0x7fd0fc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961272957 0 0) 0x7fd0fc144cf0 con 0x7fd0fc144910 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0e4003520 con 0x7fd0fc144910 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3498493742 0 0) 0x7fd0e4003520 con 0x7fd0fc144910 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 msgr2=0x7fd0fc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 0x7fd0fc0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 >> v2:172.21.15.178:3300/0 conn(0x7fd0fc0b3fd0 msgr2=0x7fd0fc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 --2- 172.21.15.113:0/1418471099 >> v2:172.21.15.178:3300/0 conn(0x7fd0fc0b3fd0 0x7fd0fc06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc0b34f0 con 0x7fd0fc144910 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fd0ec001670 con 0x7fd0fc144910 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 >> v1:172.21.15.113:6789/0 conn(0x7fd0fc144910 legacy=0x7fd0fc0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 0x7fd0e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4003f30 0x7fd0e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e4004d40 0x7fd0e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0b2f30 con 0x7fd0e4004d40 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0b3210 con 0x7fd0e4003f30 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.200+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e400b9c0 con 0x7fd0fc06e510 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fb59e640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e4004d40 0x7fd0e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fad9d640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 0x7fd0e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fa59c640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4003f30 0x7fd0e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fb59e640 1 -- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 msgr2=0x7fd0e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fb59e640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc06e510 0x7fd0e40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fb59e640 1 -- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4003f30 msgr2=0x7fd0e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fb59e640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4003f30 0x7fd0e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fb59e640 1 -- 172.21.15.113:0/1418471099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e4003520 con 0x7fd0e4004d40 2024-04-25T00:04:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.201+0000 7fd0fb59e640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e4004d40 0x7fd0e40051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009fc0 tx=0x7fd0e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4e6c60e623517a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4009c10 con 0x7fd0e4004d40 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f4030660 con 0x7fd0e4004d40 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd0f9d9b640 1 -- 172.21.15.113:0/1418471099 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4030a50 con 0x7fd0e4004d40 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd10102a640 1 -- 172.21.15.113:0/1418471099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e4004d40 msgr2=0x7fd0e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd10102a640 1 --2- 172.21.15.113:0/1418471099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e4004d40 0x7fd0e40051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009fc0 tx=0x7fd0e4005830 comp rx=0 tx=0).stop 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd0fb59e640 1 -- 172.21.15.113:0/1418471099 reap_dead start 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd10102a640 1 -- 172.21.15.113:0/1418471099 shutdown_connections 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd10102a640 1 -- 172.21.15.113:0/1418471099 >> 172.21.15.113:0/1418471099 conn(0x7fd0fc0b1a50 msgr2=0x7fd0fc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd10102a640 1 -- 172.21.15.113:0/1418471099 shutdown_connections 2024-04-25T00:04:33.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.202+0000 7fd10102a640 1 -- 172.21.15.113:0/1418471099 wait complete. 2024-04-25T00:04:33.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.204+0000 7fd10102a640 1 Processor -- start 2024-04-25T00:04:33.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.204+0000 7fd10102a640 1 -- start start 2024-04-25T00:04:33.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4004d40 0x7fd0fc154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0fc0ad970 0x7fd0fc0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc139460 0x7fd0fc139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0b4da0 con 0x7fd0fc0ad970 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0b4440 con 0x7fd0e4004d40 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd0fc0b4f20 con 0x7fd0fc139460 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0fc0ad970 0x7fd0fc0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0fc0ad970 0x7fd0fc0addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35292/0 (socket says 172.21.15.113:35292) 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 -- 172.21.15.113:0/3162340046 learned_addr learned my addr 172.21.15.113:0/3162340046 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fad9d640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4004d40 0x7fd0fc154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fb59e640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc139460 0x7fd0fc139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 -- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc139460 msgr2=0x7fd0fc139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc139460 0x7fd0fc139910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 -- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4004d40 msgr2=0x7fd0fc154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4004d40 0x7fd0fc154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 -- 172.21.15.113:0/3162340046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc138f80 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fad9d640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4004d40 0x7fd0fc154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0fa59c640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0fc0ad970 0x7fd0fc0addf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0011990 tx=0x7fd0f0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0e37fe640 1 -- 172.21.15.113:0/3162340046 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000fad0 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0e37fe640 1 -- 172.21.15.113:0/3162340046 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f000fc70 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd0e37fe640 1 -- 172.21.15.113:0/3162340046 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0014440 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc0ae6e0 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.205+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc0aeca0 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.206+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0fc0691a0 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.207+0000 7fd0e37fe640 1 -- 172.21.15.113:0/3162340046 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0f0012040 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.207+0000 7fd0e37fe640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd0e806be60 0x7fd0e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.207+0000 7fd0e37fe640 1 -- 172.21.15.113:0/3162340046 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fd0f008b450 con 0x7fd0fc0ad970 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.207+0000 7fd0fad9d640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd0e806be60 0x7fd0e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.208+0000 7fd0fad9d640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd0e806be60 0x7fd0e806e320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007dd0 tx=0x7fd0ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.212+0000 7fd0e37fe640 1 -- 172.21.15.113:0/3162340046 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd0f0019030 con 0x7fd0fc0ad970 2024-04-25T00:04:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.423+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0fc144cf0 con 0x7fd0fc0ad970 2024-04-25T00:04:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.423+0000 7fd0e37fe640 1 -- 172.21.15.113:0/3162340046 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5195 (secure 0 0 0) 0x7fd0f0059e00 con 0x7fd0fc0ad970 2024-04-25T00:04:33.424 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:33.424 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:06.288869+0000","modified":"2024-04-25T00:04:08.290392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2796979045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2796979045},{"type":"v1","addr":"172.21.15.113:6837","nonce":2796979045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7395,"qdb_cluster":[7395]},"id":6}]} 2024-04-25T00:04:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-04-25T00:04:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd0e806be60 msgr2=0x7fd0e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd0e806be60 0x7fd0e806e320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007dd0 tx=0x7fd0ec009040 comp rx=0 tx=0).stop 2024-04-25T00:04:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0fc0ad970 msgr2=0x7fd0fc0addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0fc0ad970 0x7fd0fc0addf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0011990 tx=0x7fd0f0011e50 comp rx=0 tx=0).stop 2024-04-25T00:04:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 shutdown_connections 2024-04-25T00:04:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd0e806be60 0x7fd0e806e320 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0fc139460 0x7fd0fc139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0fc0ad970 0x7fd0fc0addf0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 --2- 172.21.15.113:0/3162340046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e4004d40 0x7fd0fc154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 >> 172.21.15.113:0/3162340046 conn(0x7fd0fc0b1a50 msgr2=0x7fd0fc143120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 shutdown_connections 2024-04-25T00:04:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.425+0000 7fd10102a640 1 -- 172.21.15.113:0/3162340046 wait complete. 2024-04-25T00:04:33.436 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:04:33.437 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:04:33.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.532+0000 7f8bd3370640 1 Processor -- start 2024-04-25T00:04:33.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.532+0000 7f8bd3370640 1 -- start start 2024-04-25T00:04:33.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd3370640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8bcc0b3fd0 0x7f8bcc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd3370640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 0x7f8bcc0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd3370640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc144cf0 con 0x7f8bcc144910 2024-04-25T00:04:33.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd3370640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc0b2f30 con 0x7f8bcc0b3fd0 2024-04-25T00:04:33.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd3370640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b3210 con 0x7f8bcc06e510 2024-04-25T00:04:33.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd110c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8bcc144910 0x7f8bcc0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39742/0 (socket says 172.21.15.113:39742) 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd110c640 1 -- 172.21.15.113:0/1238074377 learned_addr learned my addr 172.21.15.113:0/1238074377 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd090b640 1 --2- 172.21.15.113:0/1238074377 >> v2:172.21.15.178:3300/0 conn(0x7f8bcc0b3fd0 0x7f8bcc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bd190d640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 0x7f8bcc0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531058858 0 0) 0x7f8bcc144cf0 con 0x7f8bcc144910 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb8003520 con 0x7f8bcc144910 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 898408211 0 0) 0x7f8bb8003520 con 0x7f8bcc144910 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 msgr2=0x7f8bcc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 0x7f8bcc0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 >> v2:172.21.15.178:3300/0 conn(0x7f8bcc0b3fd0 msgr2=0x7f8bcc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 --2- 172.21.15.113:0/1238074377 >> v2:172.21.15.178:3300/0 conn(0x7f8bcc0b3fd0 0x7f8bcc06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0b34f0 con 0x7f8bcc144910 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8bc8001670 con 0x7f8bcc144910 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 >> v1:172.21.15.113:6789/0 conn(0x7f8bcc144910 legacy=0x7f8bcc0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 0x7f8bb80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb8004d40 0x7f8bb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b2f30 con 0x7f8bb8003f30 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b3210 con 0x7f8bb8004d40 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.533+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb800b9c0 con 0x7f8bcc06e510 2024-04-25T00:04:33.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd090b640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd110c640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 0x7f8bb80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd190d640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb8004d40 0x7f8bb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd090b640 1 -- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 msgr2=0x7f8bb80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd090b640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc06e510 0x7f8bb80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd090b640 1 -- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb8004d40 msgr2=0x7f8bb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd090b640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb8004d40 0x7f8bb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd090b640 1 -- 172.21.15.113:0/1238074377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb8003520 con 0x7f8bb8003f30 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.534+0000 7f8bd090b640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bb8004800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009ac0 tx=0x7f8bb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acf35bd68673715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc036070 con 0x7f8bb8003f30 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc031400 con 0x7f8bb8003f30 2024-04-25T00:04:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bc3fff640 1 -- 172.21.15.113:0/1238074377 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc009690 con 0x7f8bb8003f30 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bd3370640 1 -- 172.21.15.113:0/1238074377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 msgr2=0x7f8bb8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bd3370640 1 --2- 172.21.15.113:0/1238074377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bb8004800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009ac0 tx=0x7f8bb8005830 comp rx=0 tx=0).stop 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bd190d640 1 -- 172.21.15.113:0/1238074377 reap_dead start 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bd3370640 1 -- 172.21.15.113:0/1238074377 shutdown_connections 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bd3370640 1 -- 172.21.15.113:0/1238074377 >> 172.21.15.113:0/1238074377 conn(0x7f8bcc0b1a50 msgr2=0x7f8bcc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bd3370640 1 -- 172.21.15.113:0/1238074377 shutdown_connections 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.535+0000 7f8bd3370640 1 -- 172.21.15.113:0/1238074377 wait complete. 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.536+0000 7f8bd3370640 1 Processor -- start 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.536+0000 7f8bd3370640 1 -- start start 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd3370640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bcc14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd3370640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc0cf120 0x7f8bcc0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd3370640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc1f5300 0x7f8bcc1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd3370640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b4da0 con 0x7f8bb8003f30 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd3370640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b4440 con 0x7f8bcc0cf120 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd3370640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b4f20 con 0x7f8bcc1f5300 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bcc14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bcc14fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35314/0 (socket says 172.21.15.113:35314) 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 -- 172.21.15.113:0/50013599 learned_addr learned my addr 172.21.15.113:0/50013599 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd090b640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc0cf120 0x7f8bcc0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd190d640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc1f5300 0x7f8bcc1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 -- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc1f5300 msgr2=0x7f8bcc1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc1f5300 0x7f8bcc1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 -- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc0cf120 msgr2=0x7f8bcc0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc0cf120 0x7f8bcc0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd110c640 1 -- 172.21.15.113:0/50013599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc151350 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd090b640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc0cf120 0x7f8bcc0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.537+0000 7f8bd190d640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc1f5300 0x7f8bcc1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.538+0000 7f8bd110c640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bcc14fd80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8001b80 tx=0x7f8bc8007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.538+0000 7f8bc1ffb640 1 -- 172.21.15.113:0/50013599 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800f960 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.538+0000 7f8bc1ffb640 1 -- 172.21.15.113:0/50013599 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc8018070 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.538+0000 7f8bc1ffb640 1 -- 172.21.15.113:0/50013599 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800fd60 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.538+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc1502c0 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.538+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc150870 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.539+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc0691a0 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.539+0000 7f8bc1ffb640 1 -- 172.21.15.113:0/50013599 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bc801d460 con 0x7f8bb8003f30 2024-04-25T00:04:33.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.539+0000 7f8bc1ffb640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8bb006be60 0x7f8bb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.540+0000 7f8bc1ffb640 1 -- 172.21.15.113:0/50013599 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8bc808ad80 con 0x7f8bb8003f30 2024-04-25T00:04:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.540+0000 7f8bd090b640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8bb006be60 0x7f8bb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.540+0000 7f8bd090b640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8bb006be60 0x7f8bb006e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc033000 tx=0x7f8bbc008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.544+0000 7f8bc1ffb640 1 -- 172.21.15.113:0/50013599 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8bc80597c0 con 0x7f8bb8003f30 2024-04-25T00:04:33.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.780+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8bcc0731a0 con 0x7f8bb8003f30 2024-04-25T00:04:33.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.780+0000 7f8bc1ffb640 1 -- 172.21.15.113:0/50013599 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1187 (secure 0 0 0) 0x7f8bc80590c0 con 0x7f8bb8003f30 2024-04-25T00:04:33.781 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:33.781 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":398,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076472280,"bytes_used":2548195328,"bytes_avail":770545917952,"bytes_total":773094113280,"write_bytes_sec":89651547,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":50,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7395}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T00:03:19.169889+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:04:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8bb006be60 msgr2=0x7f8bb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8bb006be60 0x7f8bb006e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc033000 tx=0x7f8bbc008030 comp rx=0 tx=0).stop 2024-04-25T00:04:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 msgr2=0x7f8bcc14fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bcc14fd80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8001b80 tx=0x7f8bc8007720 comp rx=0 tx=0).stop 2024-04-25T00:04:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 shutdown_connections 2024-04-25T00:04:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8bb006be60 0x7f8bb006e320 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc0cf120 0x7f8bcc0cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc1f5300 0x7f8bcc1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 --2- 172.21.15.113:0/50013599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bb8003f30 0x7f8bcc14fd80 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:33.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.782+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 >> 172.21.15.113:0/50013599 conn(0x7f8bcc0b1a50 msgr2=0x7f8bcc15ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:33.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.783+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 shutdown_connections 2024-04-25T00:04:33.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:33.783+0000 7f8bd3370640 1 -- 172.21.15.113:0/50013599 wait complete. 2024-04-25T00:04:33.793 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:04:33.793 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:04:33.794 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:04:33.829 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:04:33.872 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:04:33.872 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:04:33.872 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:33.872 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:33.887 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:33.888 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:33.943 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.081+0000 7f37cdc3f640 1 Processor -- start 2024-04-25T00:04:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.081+0000 7f37cdc3f640 1 -- start start 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.081+0000 7f37cdc3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 0x7f37c80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.081+0000 7f37cdc3f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c80d2800 con 0x7f37c815e320 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.081+0000 7f37cdc3f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c80d2ae0 con 0x7f37c80d0b00 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.081+0000 7f37cdc3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37c80d3410 con 0x7f37c80d2290 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f37c815e320 0x7f37c80d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39746/0 (socket says 172.21.15.113:39746) 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c77fe640 1 -- 172.21.15.113:0/2057544760 learned_addr learned my addr 172.21.15.113:0/2057544760 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c6ffd640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 0x7f37c80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387632222 0 0) 0x7f37c80d2800 con 0x7f37c815e320 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37b0003520 con 0x7f37c815e320 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4082095214 0 0) 0x7f37b0003520 con 0x7f37c815e320 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 msgr2=0x7f37c80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 0x7f37c80b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 >> v1:172.21.15.178:6789/0 conn(0x7f37c80d0b00 legacy=0x7f37c80b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c80d2dc0 con 0x7f37c815e320 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f37bc001670 con 0x7f37c815e320 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 >> v1:172.21.15.113:6789/0 conn(0x7f37c815e320 legacy=0x7f37c80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b00039e0 0x7f37b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b0004800 0x7f37b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 0x7f37b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80d3410 con 0x7f37b0004800 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37b000ba70 con 0x7f37b00039e0 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37b000bc70 con 0x7f37c80d2290 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c6ffd640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b0004800 0x7f37b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c7fff640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 0x7f37b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.082+0000 7f37c77fe640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b00039e0 0x7f37b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c6ffd640 1 -- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 msgr2=0x7f37b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c6ffd640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37c80d2290 0x7f37b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c6ffd640 1 -- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b00039e0 msgr2=0x7f37b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c6ffd640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b00039e0 0x7f37b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c6ffd640 1 -- 172.21.15.113:0/2057544760 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b0003520 con 0x7f37b0004800 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c6ffd640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b0004800 0x7f37b0004c80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009ac0 tx=0x7f37b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632bde659d7de7cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8036070 con 0x7f37b0004800 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b8031400 con 0x7f37b0004800 2024-04-25T00:04:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.083+0000 7f37c67fc640 1 -- 172.21.15.113:0/2057544760 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8009690 con 0x7f37b0004800 2024-04-25T00:04:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.084+0000 7f37cdc3f640 1 -- 172.21.15.113:0/2057544760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b0004800 msgr2=0x7f37b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.084+0000 7f37cdc3f640 1 --2- 172.21.15.113:0/2057544760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b0004800 0x7f37b0004c80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009ac0 tx=0x7f37b0005830 comp rx=0 tx=0).stop 2024-04-25T00:04:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.084+0000 7f37c7fff640 1 -- 172.21.15.113:0/2057544760 reap_dead start 2024-04-25T00:04:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.084+0000 7f37cdc3f640 1 -- 172.21.15.113:0/2057544760 shutdown_connections 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.084+0000 7f37cdc3f640 1 -- 172.21.15.113:0/2057544760 >> 172.21.15.113:0/2057544760 conn(0x7f37c80b5c20 msgr2=0x7f37c80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.084+0000 7f37cdc3f640 1 -- 172.21.15.113:0/2057544760 shutdown_connections 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.084+0000 7f37cdc3f640 1 -- 172.21.15.113:0/2057544760 wait complete. 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.085+0000 7f37cdc3f640 1 Processor -- start 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.085+0000 7f37cdc3f640 1 -- start start 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37cdc3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b0004800 0x7f37c81f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37cdc3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c815ea00 0x7f37c8160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37cdc3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37c815ee80 0x7f37c8160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37cdc3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37c815e880 con 0x7f37c815ee80 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37cdc3f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37c815e700 con 0x7f37c815ea00 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37cdc3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37c80d0f90 con 0x7f37b0004800 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37c815ee80 0x7f37c8160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37c815ee80 0x7f37c8160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35334/0 (socket says 172.21.15.113:35334) 2024-04-25T00:04:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 -- 172.21.15.113:0/382989584 learned_addr learned my addr 172.21.15.113:0/382989584 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c77fe640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b0004800 0x7f37c81f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 -- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b0004800 msgr2=0x7f37c81f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b0004800 0x7f37c81f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 -- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c815ea00 msgr2=0x7f37c8160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c815ea00 0x7f37c8160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c7fff640 1 -- 172.21.15.113:0/382989584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c81613d0 con 0x7f37c815ee80 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.086+0000 7f37c77fe640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b0004800 0x7f37c81f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.087+0000 7f37c7fff640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37c815ee80 0x7f37c8160ad0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f37c80d51b0 tx=0x7f37c000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.087+0000 7f37affff640 1 -- 172.21.15.113:0/382989584 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c001a070 con 0x7f37c815ee80 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.087+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c81616b0 con 0x7f37c815ee80 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.087+0000 7f37affff640 1 -- 172.21.15.113:0/382989584 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c0006d60 con 0x7f37c815ee80 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.087+0000 7f37affff640 1 -- 172.21.15.113:0/382989584 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0003b90 con 0x7f37c815ee80 2024-04-25T00:04:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.087+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81f4060 con 0x7f37c815ee80 2024-04-25T00:04:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.088+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c80691a0 con 0x7f37c815ee80 2024-04-25T00:04:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.088+0000 7f37affff640 1 -- 172.21.15.113:0/382989584 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37c0005070 con 0x7f37c815ee80 2024-04-25T00:04:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.088+0000 7f37affff640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37b406be30 0x7f37b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.089+0000 7f37c77fe640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37b406be30 0x7f37b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.089+0000 7f37affff640 1 -- 172.21.15.113:0/382989584 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f37c008a480 con 0x7f37c815ee80 2024-04-25T00:04:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.089+0000 7f37c77fe640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37b406be30 0x7f37b406e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000c00 tx=0x7f37bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.093+0000 7f37affff640 1 -- 172.21.15.113:0/382989584 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37c0058db0 con 0x7f37c815ee80 2024-04-25T00:04:34.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.267+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37c815d470 con 0x7f37c815ee80 2024-04-25T00:04:34.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.268+0000 7f37affff640 1 -- 172.21.15.113:0/382989584 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16708 (secure 0 0 0) 0x7f37c00586b0 con 0x7f37c815ee80 2024-04-25T00:04:34.270 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:34.270 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:07.938101+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:04.509800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:05.258783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:34.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37b406be30 msgr2=0x7f37b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37b406be30 0x7f37b406e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000c00 tx=0x7f37bc009040 comp rx=0 tx=0).stop 2024-04-25T00:04:34.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37c815ee80 msgr2=0x7f37c8160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37c815ee80 0x7f37c8160ad0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f37c80d51b0 tx=0x7f37c000fd40 comp rx=0 tx=0).stop 2024-04-25T00:04:34.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 shutdown_connections 2024-04-25T00:04:34.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f37b406be30 0x7f37b406e2f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c815ea00 0x7f37c8160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37c815ee80 0x7f37c8160ad0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 --2- 172.21.15.113:0/382989584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b0004800 0x7f37c81f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 >> 172.21.15.113:0/382989584 conn(0x7f37c80b5c20 msgr2=0x7f37c80b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:34.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 shutdown_connections 2024-04-25T00:04:34.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.271+0000 7f37cdc3f640 1 -- 172.21.15.113:0/382989584 wait complete. 2024-04-25T00:04:34.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:04:04.509800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:04:05.258783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:04:34.283 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:04:34.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.382+0000 7fdf3424d640 1 Processor -- start 2024-04-25T00:04:34.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.383+0000 7fdf3424d640 1 -- start start 2024-04-25T00:04:34.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.383+0000 7fdf3424d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf2c15e510 0x7fdf2c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.383+0000 7fdf3424d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdf2c0d28c0 0x7fdf2c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.383+0000 7fdf3424d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf2c0d2e00 con 0x7fdf2c0b61e0 2024-04-25T00:04:34.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.383+0000 7fdf3424d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdf2c0b6610 con 0x7fdf2c15e510 2024-04-25T00:04:34.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.383+0000 7fdf3424d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdf2c0d30e0 con 0x7fdf2c0d28c0 2024-04-25T00:04:34.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.383+0000 7fdf31fe9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf2c15e510 0x7fdf2c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf327ea640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdf2c0b61e0 0x7fdf2c071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39760/0 (socket says 172.21.15.113:39760) 2024-04-25T00:04:34.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf31fe9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf2c15e510 0x7fdf2c0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35336/0 (socket says 172.21.15.113:35336) 2024-04-25T00:04:34.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf327ea640 1 -- 172.21.15.113:0/3507876741 learned_addr learned my addr 172.21.15.113:0/3507876741 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:34.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf317e8640 1 --2- 172.21.15.113:0/3507876741 >> v2:172.21.15.178:3300/0 conn(0x7fdf2c0d28c0 0x7fdf2c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf31fe9640 1 -- 172.21.15.113:0/3507876741 >> v2:172.21.15.178:3300/0 conn(0x7fdf2c0d28c0 msgr2=0x7fdf2c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf31fe9640 1 --2- 172.21.15.113:0/3507876741 >> v2:172.21.15.178:3300/0 conn(0x7fdf2c0d28c0 0x7fdf2c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf31fe9640 1 -- 172.21.15.113:0/3507876741 >> v1:172.21.15.113:6789/0 conn(0x7fdf2c0b61e0 legacy=0x7fdf2c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf31fe9640 1 -- 172.21.15.113:0/3507876741 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c0d33c0 con 0x7fdf2c15e510 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf30fe7640 1 -- 172.21.15.113:0/3507876741 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277803518 0 0) 0x7fdf2c0d2e00 con 0x7fdf2c0b61e0 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf317e8640 1 --2- 172.21.15.113:0/3507876741 >> v2:172.21.15.178:3300/0 conn(0x7fdf2c0d28c0 0x7fdf2c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.384+0000 7fdf31fe9640 1 --2- 172.21.15.113:0/3507876741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c0d2380 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0d52f0 tx=0x7fdf2802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf7e8d985d40f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf30fe7640 1 -- 172.21.15.113:0/3507876741 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28037070 con 0x7fdf2c15e510 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf30fe7640 1 -- 172.21.15.113:0/3507876741 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf28032440 con 0x7fdf2c15e510 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf30fe7640 1 -- 172.21.15.113:0/3507876741 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2802ae60 con 0x7fdf2c15e510 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 -- 172.21.15.113:0/3507876741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 msgr2=0x7fdf2c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 --2- 172.21.15.113:0/3507876741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c0d2380 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0d52f0 tx=0x7fdf2802a110 comp rx=0 tx=0).stop 2024-04-25T00:04:34.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 -- 172.21.15.113:0/3507876741 shutdown_connections 2024-04-25T00:04:34.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 --2- 172.21.15.113:0/3507876741 >> v2:172.21.15.178:3300/0 conn(0x7fdf2c0d28c0 0x7fdf2c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 --2- 172.21.15.113:0/3507876741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c0d2380 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 -- 172.21.15.113:0/3507876741 >> 172.21.15.113:0/3507876741 conn(0x7fdf2c06b720 msgr2=0x7fdf2c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:34.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 -- 172.21.15.113:0/3507876741 shutdown_connections 2024-04-25T00:04:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.385+0000 7fdf3424d640 1 -- 172.21.15.113:0/3507876741 wait complete. 2024-04-25T00:04:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.386+0000 7fdf3424d640 1 Processor -- start 2024-04-25T00:04:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.386+0000 7fdf3424d640 1 -- start start 2024-04-25T00:04:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf3424d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf2c0b61e0 0x7fdf2c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf3424d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf2c0d28c0 0x7fdf2c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf3424d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c1f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf3424d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c0d3ea0 con 0x7fdf2c15e510 2024-04-25T00:04:34.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf3424d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c0d3980 con 0x7fdf2c0b61e0 2024-04-25T00:04:34.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf3424d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2c0b9dd0 con 0x7fdf2c0d28c0 2024-04-25T00:04:34.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf327ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c1f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf327ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c1f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35344/0 (socket says 172.21.15.113:35344) 2024-04-25T00:04:34.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf327ea640 1 -- 172.21.15.113:0/1652893027 learned_addr learned my addr 172.21.15.113:0/1652893027 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:34.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf317e8640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf2c0d28c0 0x7fdf2c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf31fe9640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf2c0b61e0 0x7fdf2c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf327ea640 1 -- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf2c0d28c0 msgr2=0x7fdf2c0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf327ea640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf2c0d28c0 0x7fdf2c0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf327ea640 1 -- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf2c0b61e0 msgr2=0x7fdf2c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.387+0000 7fdf327ea640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf2c0b61e0 0x7fdf2c0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.388+0000 7fdf327ea640 1 -- 172.21.15.113:0/1652893027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c0cc450 con 0x7fdf2c15e510 2024-04-25T00:04:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.388+0000 7fdf327ea640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c1f3660 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c700 tx=0x7fdf2400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.388+0000 7fdf227fc640 1 -- 172.21.15.113:0/1652893027 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24018070 con 0x7fdf2c15e510 2024-04-25T00:04:34.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.388+0000 7fdf227fc640 1 -- 172.21.15.113:0/1652893027 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf2400ce10 con 0x7fdf2c15e510 2024-04-25T00:04:34.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.388+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c0ca630 con 0x7fdf2c15e510 2024-04-25T00:04:34.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.388+0000 7fdf227fc640 1 -- 172.21.15.113:0/1652893027 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400a860 con 0x7fdf2c15e510 2024-04-25T00:04:34.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.388+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c0cbe90 con 0x7fdf2c15e510 2024-04-25T00:04:34.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.389+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2c0691a0 con 0x7fdf2c15e510 2024-04-25T00:04:34.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.389+0000 7fdf227fc640 1 -- 172.21.15.113:0/1652893027 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf2401b070 con 0x7fdf2c15e510 2024-04-25T00:04:34.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.390+0000 7fdf227fc640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf1006be30 0x7fdf1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.390+0000 7fdf227fc640 1 -- 172.21.15.113:0/1652893027 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fdf2408a3f0 con 0x7fdf2c15e510 2024-04-25T00:04:34.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.390+0000 7fdf31fe9640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf1006be30 0x7fdf1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.390+0000 7fdf31fe9640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf1006be30 0x7fdf1006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdf28034000 tx=0x7fdf2802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.394+0000 7fdf227fc640 1 -- 172.21.15.113:0/1652893027 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf24058e30 con 0x7fdf2c15e510 2024-04-25T00:04:34.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.564+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf2c146e90 con 0x7fdf2c15e510 2024-04-25T00:04:34.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.564+0000 7fdf227fc640 1 -- 172.21.15.113:0/1652893027 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fdf24058730 con 0x7fdf2c15e510 2024-04-25T00:04:34.565 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf1006be30 msgr2=0x7fdf1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf1006be30 0x7fdf1006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdf28034000 tx=0x7fdf2802e000 comp rx=0 tx=0).stop 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 msgr2=0x7fdf2c1f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c1f3660 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c700 tx=0x7fdf2400cbc0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 shutdown_connections 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdf1006be30 0x7fdf1006e2f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf2c15e510 0x7fdf2c1f3660 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdf2c0d28c0 0x7fdf2c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 --2- 172.21.15.113:0/1652893027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf2c0b61e0 0x7fdf2c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 >> 172.21.15.113:0/1652893027 conn(0x7fdf2c06b720 msgr2=0x7fdf2c0b7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 shutdown_connections 2024-04-25T00:04:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.566+0000 7fdf3424d640 1 -- 172.21.15.113:0/1652893027 wait complete. 2024-04-25T00:04:34.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:04:34.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.676+0000 7f677499e640 1 Processor -- start 2024-04-25T00:04:34.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.676+0000 7f677499e640 1 -- start start 2024-04-25T00:04:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.677+0000 7f677499e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 0x7f6770167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.677+0000 7f677499e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677015d6b0 con 0x7f67700d06c0 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.677+0000 7f677499e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677015c8f0 con 0x7f677013f050 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.677+0000 7f677499e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f677015c6a0 con 0x7f67701606a0 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.677+0000 7f676dd9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f67700d06c0 0x7f677015ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39770/0 (socket says 172.21.15.113:39770) 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.677+0000 7f676dd9b640 1 -- 172.21.15.113:0/3732833815 learned_addr learned my addr 172.21.15.113:0/3732833815 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.677+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2656880060 0 0) 0x7f677015d6b0 con 0x7f67700d06c0 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6758003520 con 0x7f67700d06c0 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 0x7f6770167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1314869194 0 0) 0x7f6758003520 con 0x7f67700d06c0 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 msgr2=0x7f6770167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 0x7f6770167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 >> v1:172.21.15.178:6789/0 conn(0x7f677013f050 legacy=0x7f67700cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677015cbd0 con 0x7f67700d06c0 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6764001670 con 0x7f67700d06c0 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 0x7f6770167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 >> v1:172.21.15.113:6789/0 conn(0x7f67700d06c0 legacy=0x7f677015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67580039e0 0x7f6758003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 0x7f6758004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6758004d40 0x7f67580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f677015c6a0 con 0x7f6758004d40 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f675800bb50 con 0x7f67580039e0 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f675800bd50 con 0x7f67701606a0 2024-04-25T00:04:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6758004d40 0x7f67580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676e59c640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67580039e0 0x7f6758003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676dd9b640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 0x7f6758004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 -- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 msgr2=0x7f6758004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67701606a0 0x7f6758004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 -- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67580039e0 msgr2=0x7f6758003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67580039e0 0x7f6758003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.678+0000 7f676ed9d640 1 -- 172.21.15.113:0/3732833815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6758003520 con 0x7f6758004d40 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.679+0000 7f676ed9d640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6758004d40 0x7f67580051c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f67680098a0 tx=0x7f6758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbb7f006030a8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.679+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676800f040 con 0x7f6758004d40 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.679+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676802ace0 con 0x7f6758004d40 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.679+0000 7f676d59a640 1 -- 172.21.15.113:0/3732833815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768004250 con 0x7f6758004d40 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.680+0000 7f677499e640 1 -- 172.21.15.113:0/3732833815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6758004d40 msgr2=0x7f67580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.680+0000 7f677499e640 1 --2- 172.21.15.113:0/3732833815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6758004d40 0x7f67580051c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f67680098a0 tx=0x7f6758005830 comp rx=0 tx=0).stop 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.680+0000 7f676ed9d640 1 -- 172.21.15.113:0/3732833815 reap_dead start 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.680+0000 7f677499e640 1 -- 172.21.15.113:0/3732833815 shutdown_connections 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.680+0000 7f677499e640 1 -- 172.21.15.113:0/3732833815 >> 172.21.15.113:0/3732833815 conn(0x7f677006b0f0 msgr2=0x7f677006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.680+0000 7f677499e640 1 -- 172.21.15.113:0/3732833815 shutdown_connections 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.680+0000 7f677499e640 1 -- 172.21.15.113:0/3732833815 wait complete. 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.681+0000 7f677499e640 1 Processor -- start 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.681+0000 7f677499e640 1 -- start start 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f677499e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6758004d40 0x7f677013a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f677499e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67700b4370 0x7f67700b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f677499e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677015e730 0x7f677015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f677499e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f677013f430 con 0x7f677015e730 2024-04-25T00:04:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f677499e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f677015d190 con 0x7f6758004d40 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f677499e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f67700cfa70 con 0x7f67700b4370 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677015e730 0x7f677015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677015e730 0x7f677015ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35352/0 (socket says 172.21.15.113:35352) 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 -- 172.21.15.113:0/2468554411 learned_addr learned my addr 172.21.15.113:0/2468554411 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676dd9b640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67700b4370 0x7f67700b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676e59c640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6758004d40 0x7f677013a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 -- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67700b4370 msgr2=0x7f67700b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67700b4370 0x7f67700b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 -- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6758004d40 msgr2=0x7f677013a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6758004d40 0x7f677013a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 -- 172.21.15.113:0/2468554411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67700afc80 con 0x7f677015e730 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f676ed9d640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677015e730 0x7f677015ebb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f67680099c0 tx=0x7f6768004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f675effd640 1 -- 172.21.15.113:0/2468554411 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676800f000 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f675effd640 1 -- 172.21.15.113:0/2468554411 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768044070 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677013ac40 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.682+0000 7f675effd640 1 -- 172.21.15.113:0/2468554411 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768003b50 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.683+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6770148650 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.684+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677015f0f0 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.684+0000 7f675effd640 1 -- 172.21.15.113:0/2468554411 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f676803b040 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.685+0000 7f675effd640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675006be60 0x7f675006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.685+0000 7f675effd640 1 -- 172.21.15.113:0/2468554411 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f67680aa980 con 0x7f677015e730 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.685+0000 7f676e59c640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675006be60 0x7f675006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.685+0000 7f676e59c640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675006be60 0x7f675006e320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6760001ee0 tx=0x7f6760009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.689+0000 7f675effd640 1 -- 172.21.15.113:0/2468554411 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f676804d020 con 0x7f677015e730 2024-04-25T00:04:34.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.861+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67700691a0 con 0x7f677015e730 2024-04-25T00:04:34.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.862+0000 7f675effd640 1 -- 172.21.15.113:0/2468554411 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f6768078ce0 con 0x7f677015e730 2024-04-25T00:04:34.863 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:04:34.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.863+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675006be60 msgr2=0x7f675006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.863+0000 7f677499e640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675006be60 0x7f675006e320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6760001ee0 tx=0x7f6760009040 comp rx=0 tx=0).stop 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.863+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677015e730 msgr2=0x7f677015ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.863+0000 7f677499e640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677015e730 0x7f677015ebb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f67680099c0 tx=0x7f6768004730 comp rx=0 tx=0).stop 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 shutdown_connections 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f675006be60 0x7f675006e320 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677015e730 0x7f677015ebb0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67700b4370 0x7f67700b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 --2- 172.21.15.113:0/2468554411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6758004d40 0x7f677013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 >> 172.21.15.113:0/2468554411 conn(0x7f677006b0f0 msgr2=0x7f677013df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 shutdown_connections 2024-04-25T00:04:34.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.864+0000 7f677499e640 1 -- 172.21.15.113:0/2468554411 wait complete. 2024-04-25T00:04:34.875 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:04:34.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.975+0000 7f7bf0ba9640 1 Processor -- start 2024-04-25T00:04:34.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.975+0000 7f7bf0ba9640 1 -- start start 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bf0ba9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7bec0d06c0 0x7f7bec15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bf0ba9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec1604d0 0x7f7bec0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bf0ba9640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bec15d6b0 con 0x7f7bec13f050 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bf0ba9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7bec15c6a0 con 0x7f7bec0d06c0 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bf0ba9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec15c8f0 con 0x7f7bec1604d0 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bead9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec1604d0 0x7f7bec0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bead9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec1604d0 0x7f7bec0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:56282/0 (socket says 172.21.15.113:56282) 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7bead9d640 1 -- 172.21.15.113:0/3876707953 learned_addr learned my addr 172.21.15.113:0/3876707953 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.976+0000 7f7be9d9b640 1 --2- 172.21.15.113:0/3876707953 >> v2:172.21.15.178:3300/0 conn(0x7f7bec0d06c0 0x7f7bec15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7be959a640 1 -- 172.21.15.113:0/3876707953 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764235087 0 0) 0x7f7bec15d6b0 con 0x7f7bec13f050 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7be959a640 1 -- 172.21.15.113:0/3876707953 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bd4003520 con 0x7f7bec13f050 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7bead9d640 1 -- 172.21.15.113:0/3876707953 >> v2:172.21.15.178:3300/0 conn(0x7f7bec0d06c0 msgr2=0x7f7bec15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7bead9d640 1 --2- 172.21.15.113:0/3876707953 >> v2:172.21.15.178:3300/0 conn(0x7f7bec0d06c0 0x7f7bec15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7bead9d640 1 -- 172.21.15.113:0/3876707953 >> v1:172.21.15.178:6789/0 conn(0x7f7bec13f050 legacy=0x7f7bec0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7bead9d640 1 -- 172.21.15.113:0/3876707953 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec15cbd0 con 0x7f7bec1604d0 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7be9d9b640 1 --2- 172.21.15.113:0/3876707953 >> v2:172.21.15.178:3300/0 conn(0x7f7bec0d06c0 0x7f7bec15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.977+0000 7f7bead9d640 1 --2- 172.21.15.113:0/3876707953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec1604d0 0x7f7bec0cea60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009870 tx=0x7f7be402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e66700055232ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7be959a640 1 -- 172.21.15.113:0/3876707953 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4037070 con 0x7f7bec1604d0 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7be959a640 1 -- 172.21.15.113:0/3876707953 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be4032440 con 0x7f7bec1604d0 2024-04-25T00:04:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7be959a640 1 -- 172.21.15.113:0/3876707953 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be40327f0 con 0x7f7bec1604d0 2024-04-25T00:04:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3876707953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec1604d0 msgr2=0x7f7bec0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3876707953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec1604d0 0x7f7bec0cea60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009870 tx=0x7f7be402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3876707953 shutdown_connections 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3876707953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec1604d0 0x7f7bec0cea60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3876707953 >> v2:172.21.15.178:3300/0 conn(0x7f7bec0d06c0 0x7f7bec15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3876707953 >> 172.21.15.113:0/3876707953 conn(0x7f7bec06b0f0 msgr2=0x7f7bec06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3876707953 shutdown_connections 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.978+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3876707953 wait complete. 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 Processor -- start 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 -- start start 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec0d06c0 0x7f7bec138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec13f050 0x7f7bec1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bec1604d0 0x7f7bec13c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec13f430 con 0x7f7bec13f050 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec15d190 con 0x7f7bec1604d0 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bf0ba9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec160930 con 0x7f7bec0d06c0 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7be9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec13f050 0x7f7bec1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7be9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec13f050 0x7f7bec1562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35366/0 (socket says 172.21.15.113:35366) 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7be9d9b640 1 -- 172.21.15.113:0/3898432003 learned_addr learned my addr 172.21.15.113:0/3898432003 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.980+0000 7f7bead9d640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bec1604d0 0x7f7bec13c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7bea59c640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec0d06c0 0x7f7bec138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7be9d9b640 1 -- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec0d06c0 msgr2=0x7f7bec138100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7be9d9b640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec0d06c0 0x7f7bec138100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7be9d9b640 1 -- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bec1604d0 msgr2=0x7f7bec13c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7be9d9b640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bec1604d0 0x7f7bec13c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7be9d9b640 1 -- 172.21.15.113:0/3898432003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec138830 con 0x7f7bec13f050 2024-04-25T00:04:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7bead9d640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bec1604d0 0x7f7bec13c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7be9d9b640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec13f050 0x7f7bec1562b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7be000e980 tx=0x7f7be000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7bdaffd640 1 -- 172.21.15.113:0/3898432003 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0017070 con 0x7f7bec13f050 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7bdaffd640 1 -- 172.21.15.113:0/3898432003 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be000ca00 con 0x7f7bec13f050 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bec138b10 con 0x7f7bec13f050 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7bdaffd640 1 -- 172.21.15.113:0/3898432003 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be000cdf0 con 0x7f7bec13f050 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.981+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bec1390d0 con 0x7f7bec13f050 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.982+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bec0691a0 con 0x7f7bec13f050 2024-04-25T00:04:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.982+0000 7f7bdaffd640 1 -- 172.21.15.113:0/3898432003 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7be001a070 con 0x7f7bec13f050 2024-04-25T00:04:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.983+0000 7f7bdaffd640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bcc06be30 0x7f7bcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.983+0000 7f7bdaffd640 1 -- 172.21.15.113:0/3898432003 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f7be008ae90 con 0x7f7bec13f050 2024-04-25T00:04:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.983+0000 7f7bea59c640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bcc06be30 0x7f7bcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.983+0000 7f7bea59c640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bcc06be30 0x7f7bcc06e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc001d50 tx=0x7f7bdc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:34.987+0000 7f7bdaffd640 1 -- 172.21.15.113:0/3898432003 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7be00597b0 con 0x7f7bec13f050 2024-04-25T00:04:35.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.163+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7bec0d0e50 con 0x7f7bec13f050 2024-04-25T00:04:35.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.163+0000 7f7bdaffd640 1 -- 172.21.15.113:0/3898432003 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f7be00590b0 con 0x7f7bec13f050 2024-04-25T00:04:35.164 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:04:35.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bcc06be30 msgr2=0x7f7bcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bcc06be30 0x7f7bcc06e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc001d50 tx=0x7f7bdc009040 comp rx=0 tx=0).stop 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec13f050 msgr2=0x7f7bec1562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec13f050 0x7f7bec1562b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7be000e980 tx=0x7f7be000ee40 comp rx=0 tx=0).stop 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 shutdown_connections 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bcc06be30 0x7f7bcc06e2f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bec1604d0 0x7f7bec13c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec13f050 0x7f7bec1562b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 --2- 172.21.15.113:0/3898432003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bec0d06c0 0x7f7bec138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 >> 172.21.15.113:0/3898432003 conn(0x7f7bec06b0f0 msgr2=0x7f7bec15f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 shutdown_connections 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.165+0000 7f7bf0ba9640 1 -- 172.21.15.113:0/3898432003 wait complete. 2024-04-25T00:04:35.176 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:04:35.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.273+0000 7f24be43f640 1 Processor -- start 2024-04-25T00:04:35.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.273+0000 7f24be43f640 1 -- start start 2024-04-25T00:04:35.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24be43f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24b815e2d0 0x7f24b815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:35.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24be43f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b80b3080 0x7f24b80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:35.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24be43f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b815f4f0 con 0x7f24b815e6b0 2024-04-25T00:04:35.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24be43f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f24b80cde00 con 0x7f24b815e2d0 2024-04-25T00:04:35.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24be43f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24b815f7d0 con 0x7f24b80b3080 2024-04-25T00:04:35.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24b815e2d0 0x7f24b815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:35.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24b815e2d0 0x7f24b815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35376/0 (socket says 172.21.15.113:35376) 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b7fff640 1 -- 172.21.15.113:0/4141186636 learned_addr learned my addr 172.21.15.113:0/4141186636 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b7fff640 1 -- 172.21.15.113:0/4141186636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b80b3080 msgr2=0x7f24b80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b77fe640 1 --2- 172.21.15.113:0/4141186636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b80b3080 0x7f24b80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b7fff640 1 --2- 172.21.15.113:0/4141186636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b80b3080 0x7f24b80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b7fff640 1 -- 172.21.15.113:0/4141186636 >> v1:172.21.15.178:6789/0 conn(0x7f24b815e6b0 legacy=0x7f24b8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.274+0000 7f24b7fff640 1 -- 172.21.15.113:0/4141186636 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b815fab0 con 0x7f24b815e2d0 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.275+0000 7f24b7fff640 1 --2- 172.21.15.113:0/4141186636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b815e2d0 0x7f24b815ebf0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009870 tx=0x7f24a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da99aaa2d2c02cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.275+0000 7f24b6ffd640 1 -- 172.21.15.113:0/4141186636 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8037070 con 0x7f24b815e2d0 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.275+0000 7f24b6ffd640 1 -- 172.21.15.113:0/4141186636 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a8032440 con 0x7f24b815e2d0 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.275+0000 7f24b6ffd640 1 -- 172.21.15.113:0/4141186636 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8032810 con 0x7f24b815e2d0 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 -- 172.21.15.113:0/4141186636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b815e2d0 msgr2=0x7f24b815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 --2- 172.21.15.113:0/4141186636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b815e2d0 0x7f24b815ebf0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009870 tx=0x7f24a802a180 comp rx=0 tx=0).stop 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 -- 172.21.15.113:0/4141186636 shutdown_connections 2024-04-25T00:04:35.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 --2- 172.21.15.113:0/4141186636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b80b3080 0x7f24b80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 --2- 172.21.15.113:0/4141186636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b815e2d0 0x7f24b815ebf0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 -- 172.21.15.113:0/4141186636 >> 172.21.15.113:0/4141186636 conn(0x7f24b806b0f0 msgr2=0x7f24b806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:35.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 -- 172.21.15.113:0/4141186636 shutdown_connections 2024-04-25T00:04:35.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.276+0000 7f24be43f640 1 -- 172.21.15.113:0/4141186636 wait complete. 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.277+0000 7f24be43f640 1 Processor -- start 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.277+0000 7f24be43f640 1 -- start start 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24be43f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b80b3080 0x7f24b8157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24be43f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b815e2d0 0x7f24b814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24be43f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b815e6b0 0x7f24b813c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24be43f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f24b815f2b0 con 0x7f24b80b3080 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24be43f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24b815f130 con 0x7f24b815e6b0 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24be43f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24b80cf2a0 con 0x7f24b815e2d0 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b80b3080 0x7f24b8157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:35.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b80b3080 0x7f24b8157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35380/0 (socket says 172.21.15.113:35380) 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 -- 172.21.15.113:0/1157161807 learned_addr learned my addr 172.21.15.113:0/1157161807 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 -- 172.21.15.113:0/1157161807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b815e2d0 msgr2=0x7f24b814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24bc9dc640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b815e6b0 0x7f24b813c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b815e2d0 0x7f24b814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 -- 172.21.15.113:0/1157161807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b815e6b0 msgr2=0x7f24b813c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b815e6b0 0x7f24b813c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 -- 172.21.15.113:0/1157161807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b81593b0 con 0x7f24b80b3080 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.278+0000 7f24b7fff640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b80b3080 0x7f24b8157a10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009870 tx=0x7f24a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.279+0000 7f24b4ff9640 1 -- 172.21.15.113:0/1157161807 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8041070 con 0x7f24b80b3080 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.279+0000 7f24b4ff9640 1 -- 172.21.15.113:0/1157161807 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a803c870 con 0x7f24b80b3080 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.279+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b8158310 con 0x7f24b80b3080 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.279+0000 7f24b4ff9640 1 -- 172.21.15.113:0/1157161807 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8037040 con 0x7f24b80b3080 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.279+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81588d0 con 0x7f24b80b3080 2024-04-25T00:04:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.280+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b80691a0 con 0x7f24b80b3080 2024-04-25T00:04:35.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.280+0000 7f24b4ff9640 1 -- 172.21.15.113:0/1157161807 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24a8044070 con 0x7f24b80b3080 2024-04-25T00:04:35.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.280+0000 7f24b4ff9640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24a406be60 0x7f24a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:35.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.281+0000 7f24b4ff9640 1 -- 172.21.15.113:0/1157161807 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f24a80ab970 con 0x7f24b80b3080 2024-04-25T00:04:35.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.281+0000 7f24b77fe640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24a406be60 0x7f24a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:35.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.281+0000 7f24b77fe640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24a406be60 0x7f24a406e320 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006fd0 tx=0x7f24ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:35.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.285+0000 7f24b4ff9640 1 -- 172.21.15.113:0/1157161807 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24a807a3b0 con 0x7f24b80b3080 2024-04-25T00:04:35.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.468+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f24b80d03c0 con 0x7f24b80b3080 2024-04-25T00:04:35.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.468+0000 7f24b4ff9640 1 -- 172.21.15.113:0/1157161807 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) ==== 80+0+0 (secure 0 0 0) 0x7f24a8079cb0 con 0x7f24b80b3080 2024-04-25T00:04:35.469 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24a406be60 msgr2=0x7f24a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24a406be60 0x7f24a406e320 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006fd0 tx=0x7f24ac008040 comp rx=0 tx=0).stop 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b80b3080 msgr2=0x7f24b8157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b80b3080 0x7f24b8157a10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009870 tx=0x7f24a80046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 shutdown_connections 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24a406be60 0x7f24a406e320 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b815e6b0 0x7f24b813c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b815e2d0 0x7f24b814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 --2- 172.21.15.113:0/1157161807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24b80b3080 0x7f24b8157a10 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:35.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 >> 172.21.15.113:0/1157161807 conn(0x7f24b806b0f0 msgr2=0x7f24b80cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:35.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 shutdown_connections 2024-04-25T00:04:35.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:35.470+0000 7f24be43f640 1 -- 172.21.15.113:0/1157161807 wait complete. 2024-04-25T00:04:35.481 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:04:35.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:35.482 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:36.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:04:36.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:36.091 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:04:36.109 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:04:36.110 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:04:36.110 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:36.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:04:36.129 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:04:36.130 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:36.130 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:36.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:04:36.167 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:04:36.167 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:04:36.168 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:36.185 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:04:36.186 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:36.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.303+0000 7f3a9f9b7640 1 Processor -- start 2024-04-25T00:04:36.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.303+0000 7f3a9f9b7640 1 -- start start 2024-04-25T00:04:36.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.304+0000 7f3a9f9b7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3a9815e320 0x7f3a980d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.304+0000 7f3a9f9b7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a980d3de0 con 0x7f3a980d0b00 2024-04-25T00:04:36.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.304+0000 7f3a9f9b7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a980d2800 con 0x7f3a980d2290 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.304+0000 7f3a9f9b7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3a980d3410 con 0x7f3a9815e320 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.304+0000 7f3a9df54640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3a980d0b00 0x7f3a980b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39774/0 (socket says 172.21.15.113:39774) 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.304+0000 7f3a9df54640 1 -- 172.21.15.113:0/502083119 learned_addr learned my addr 172.21.15.113:0/502083119 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9d753640 1 --2- 172.21.15.113:0/502083119 >> v2:172.21.15.178:3300/0 conn(0x7f3a9815e320 0x7f3a980d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358837527 0 0) 0x7f3a980d3de0 con 0x7f3a980d0b00 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a84003520 con 0x7f3a980d0b00 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496263740 0 0) 0x7f3a980d2800 con 0x7f3a980d2290 2024-04-25T00:04:36.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a980d3de0 con 0x7f3a980d2290 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3098482500 0 0) 0x7f3a84003520 con 0x7f3a980d0b00 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 >> v2:172.21.15.178:3300/0 conn(0x7f3a9815e320 msgr2=0x7f3a980d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 --2- 172.21.15.113:0/502083119 >> v2:172.21.15.178:3300/0 conn(0x7f3a9815e320 0x7f3a980d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 >> v1:172.21.15.178:6789/0 conn(0x7f3a980d2290 legacy=0x7f3a980b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a980d2ae0 con 0x7f3a980d0b00 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f3a900015f0 con 0x7f3a980d0b00 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 >> v1:172.21.15.113:6789/0 conn(0x7f3a980d0b00 legacy=0x7f3a980b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a84003b90 0x7f3a84003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84004800 0x7f3a84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a840051c0 0x7f3a8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3a980d3410 con 0x7f3a84003b90 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400bfe0 con 0x7f3a840051c0 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8400c1e0 con 0x7f3a84004800 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9d753640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a84003b90 0x7f3a84003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9df54640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a840051c0 0x7f3a8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9cf52640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84004800 0x7f3a84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9d753640 1 -- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84004800 msgr2=0x7f3a84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9d753640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84004800 0x7f3a84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9d753640 1 -- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a840051c0 msgr2=0x7f3a8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9d753640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a840051c0 0x7f3a8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9d753640 1 -- 172.21.15.113:0/502083119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a84003520 con 0x7f3a84003b90 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9df54640 1 -- 172.21.15.113:0/502083119 reap_dead start 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.305+0000 7f3a9cf52640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84004800 0x7f3a84004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.306+0000 7f3a9d753640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a84003b90 0x7f3a84003f90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009fc0 tx=0x7f3a84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6be1bfd1b4956c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.306+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94009c10 con 0x7f3a84003b90 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.306+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94036070 con 0x7f3a84003b90 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.306+0000 7f3a8ffff640 1 -- 172.21.15.113:0/502083119 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94031880 con 0x7f3a84003b90 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.307+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/502083119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a84003b90 msgr2=0x7f3a84003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.307+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a84003b90 0x7f3a84003f90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009fc0 tx=0x7f3a84005830 comp rx=0 tx=0).stop 2024-04-25T00:04:36.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.307+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/502083119 shutdown_connections 2024-04-25T00:04:36.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.307+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/502083119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a84003b90 0x7f3a84003f90 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.307+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/502083119 >> 172.21.15.113:0/502083119 conn(0x7f3a980b5c20 msgr2=0x7f3a980b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:36.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.307+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/502083119 shutdown_connections 2024-04-25T00:04:36.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.307+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/502083119 wait complete. 2024-04-25T00:04:36.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.308+0000 7f3a9f9b7640 1 Processor -- start 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.308+0000 7f3a9f9b7640 1 -- start start 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9f9b7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003b90 0x7f3a9815f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9f9b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c7bc0 0x7f3a981f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9f9b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a980c8560 0x7f3a980cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9f9b7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9815e700 con 0x7f3a980c7bc0 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9f9b7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a980d30a0 con 0x7f3a84003b90 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9f9b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9815e880 con 0x7f3a980c8560 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c7bc0 0x7f3a981f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c7bc0 0x7f3a981f34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35396/0 (socket says 172.21.15.113:35396) 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 -- 172.21.15.113:0/740085115 learned_addr learned my addr 172.21.15.113:0/740085115 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9d753640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003b90 0x7f3a9815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9df54640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a980c8560 0x7f3a980cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 -- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a980c8560 msgr2=0x7f3a980cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a980c8560 0x7f3a980cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 -- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003b90 msgr2=0x7f3a9815f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003b90 0x7f3a9815f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 -- 172.21.15.113:0/740085115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a980c80a0 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9cf52640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c7bc0 0x7f3a981f34b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007fd0 tx=0x7f3a8800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a8dffb640 1 -- 172.21.15.113:0/740085115 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a880052e0 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a8dffb640 1 -- 172.21.15.113:0/740085115 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a88005480 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9815ec50 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.309+0000 7f3a8dffb640 1 -- 172.21.15.113:0/740085115 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88007020 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.310+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9815fc70 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.311+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a98161a30 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.311+0000 7f3a8dffb640 1 -- 172.21.15.113:0/740085115 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a8801a020 con 0x7f3a980c7bc0 2024-04-25T00:04:36.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.311+0000 7f3a8dffb640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3a7c06be60 0x7f3a7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.312+0000 7f3a8dffb640 1 -- 172.21.15.113:0/740085115 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f3a8808a170 con 0x7f3a980c7bc0 2024-04-25T00:04:36.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.312+0000 7f3a9d753640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3a7c06be60 0x7f3a7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.312+0000 7f3a9d753640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3a7c06be60 0x7f3a7c06e320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3a94035000 tx=0x7f3a9402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.316+0000 7f3a8dffb640 1 -- 172.21.15.113:0/740085115 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a88058560 con 0x7f3a980c7bc0 2024-04-25T00:04:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.524+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a980691a0 con 0x7f3a980c7bc0 2024-04-25T00:04:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.525+0000 7f3a8dffb640 1 -- 172.21.15.113:0/740085115 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5195 (secure 0 0 0) 0x7f3a8805c860 con 0x7f3a980c7bc0 2024-04-25T00:04:36.525 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:06.288869+0000","modified":"2024-04-25T00:04:08.290392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2796979045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2796979045},{"type":"v1","addr":"172.21.15.113:6837","nonce":2796979045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7395,"qdb_cluster":[7395]},"id":6}]} 2024-04-25T00:04:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3a7c06be60 msgr2=0x7f3a7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3a7c06be60 0x7f3a7c06e320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3a94035000 tx=0x7f3a9402f000 comp rx=0 tx=0).stop 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c7bc0 msgr2=0x7f3a981f34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c7bc0 0x7f3a981f34b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007fd0 tx=0x7f3a8800e960 comp rx=0 tx=0).stop 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 shutdown_connections 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3a7c06be60 0x7f3a7c06e320 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a980c8560 0x7f3a980cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c7bc0 0x7f3a981f34b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 --2- 172.21.15.113:0/740085115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003b90 0x7f3a9815f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 >> 172.21.15.113:0/740085115 conn(0x7f3a980b5c20 msgr2=0x7f3a980ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:36.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 shutdown_connections 2024-04-25T00:04:36.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.527+0000 7f3a9f9b7640 1 -- 172.21.15.113:0/740085115 wait complete. 2024-04-25T00:04:36.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.636+0000 7ff394eee640 1 Processor -- start 2024-04-25T00:04:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.636+0000 7ff394eee640 1 -- start start 2024-04-25T00:04:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff394eee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3900b3fd0 0x7ff39006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff394eee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff39006e510 0x7ff3900cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff394eee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff390144cf0 con 0x7ff390144910 2024-04-25T00:04:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff394eee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff3900b2f30 con 0x7ff39006e510 2024-04-25T00:04:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff394eee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff3900b3210 con 0x7ff3900b3fd0 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff38ed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff39006e510 0x7ff3900cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff38e59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff390144910 0x7ff3900b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39788/0 (socket says 172.21.15.113:39788) 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff38e59c640 1 -- 172.21.15.113:0/1702279037 learned_addr learned my addr 172.21.15.113:0/1702279037 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff387fff640 1 --2- 172.21.15.113:0/1702279037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3900b3fd0 0x7ff39006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff38ed9d640 1 -- 172.21.15.113:0/1702279037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3900b3fd0 msgr2=0x7ff39006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff38ed9d640 1 --2- 172.21.15.113:0/1702279037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3900b3fd0 0x7ff39006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff38ed9d640 1 -- 172.21.15.113:0/1702279037 >> v1:172.21.15.113:6789/0 conn(0x7ff390144910 legacy=0x7ff3900b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.637+0000 7ff38ed9d640 1 -- 172.21.15.113:0/1702279037 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3900b34f0 con 0x7ff39006e510 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff38ed9d640 1 --2- 172.21.15.113:0/1702279037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39006e510 0x7ff3900cd500 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff388009d90 tx=0x7ff38802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d592078b761be25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff38dd9b640 1 -- 172.21.15.113:0/1702279037 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388007740 con 0x7ff39006e510 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff38dd9b640 1 -- 172.21.15.113:0/1702279037 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3880078e0 con 0x7ff39006e510 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff38dd9b640 1 -- 172.21.15.113:0/1702279037 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388007cb0 con 0x7ff39006e510 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff394eee640 1 -- 172.21.15.113:0/1702279037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39006e510 msgr2=0x7ff3900cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff394eee640 1 --2- 172.21.15.113:0/1702279037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39006e510 0x7ff3900cd500 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff388009d90 tx=0x7ff38802a660 comp rx=0 tx=0).stop 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff394eee640 1 -- 172.21.15.113:0/1702279037 shutdown_connections 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff394eee640 1 --2- 172.21.15.113:0/1702279037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39006e510 0x7ff3900cd500 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff394eee640 1 --2- 172.21.15.113:0/1702279037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3900b3fd0 0x7ff39006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.638+0000 7ff394eee640 1 -- 172.21.15.113:0/1702279037 >> 172.21.15.113:0/1702279037 conn(0x7ff3900b1a50 msgr2=0x7ff3900b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.639+0000 7ff394eee640 1 -- 172.21.15.113:0/1702279037 shutdown_connections 2024-04-25T00:04:36.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.639+0000 7ff394eee640 1 -- 172.21.15.113:0/1702279037 wait complete. 2024-04-25T00:04:36.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.640+0000 7ff394eee640 1 Processor -- start 2024-04-25T00:04:36.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.640+0000 7ff394eee640 1 -- start start 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.640+0000 7ff394eee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff39006e510 0x7ff39015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.640+0000 7ff394eee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3900b3fd0 0x7ff390132ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.640+0000 7ff394eee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff390144910 0x7ff39013ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff394eee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900b4da0 con 0x7ff3900b3fd0 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff394eee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3900b43b0 con 0x7ff39006e510 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff394eee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff3900b4f20 con 0x7ff390144910 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3900b3fd0 0x7ff390132ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3900b3fd0 0x7ff390132ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35414/0 (socket says 172.21.15.113:35414) 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 -- 172.21.15.113:0/1518325158 learned_addr learned my addr 172.21.15.113:0/1518325158 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff38e59c640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff39006e510 0x7ff39015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff38ed9d640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff390144910 0x7ff39013ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 -- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff390144910 msgr2=0x7ff39013ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff390144910 0x7ff39013ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 -- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff39006e510 msgr2=0x7ff39015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff39006e510 0x7ff39015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 -- 172.21.15.113:0/1518325158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff390133770 con 0x7ff3900b3fd0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff387fff640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3900b3fd0 0x7ff390132ca0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff37800d830 tx=0x7ff37800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff386ffd640 1 -- 172.21.15.113:0/1518325158 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37800b6c0 con 0x7ff3900b3fd0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff386ffd640 1 -- 172.21.15.113:0/1518325158 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff378019070 con 0x7ff3900b3fd0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff390133a50 con 0x7ff3900b3fd0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff386ffd640 1 -- 172.21.15.113:0/1518325158 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37800ba80 con 0x7ff3900b3fd0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.641+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff390134010 con 0x7ff3900b3fd0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.642+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3900691a0 con 0x7ff3900b3fd0 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.643+0000 7ff386ffd640 1 -- 172.21.15.113:0/1518325158 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff37801e460 con 0x7ff3900b3fd0 2024-04-25T00:04:36.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.643+0000 7ff386ffd640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff37006be60 0x7ff37006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.643+0000 7ff38e59c640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff37006be60 0x7ff37006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.643+0000 7ff386ffd640 1 -- 172.21.15.113:0/1518325158 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7ff37808b390 con 0x7ff3900b3fd0 2024-04-25T00:04:36.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.644+0000 7ff38e59c640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff37006be60 0x7ff37006e320 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff380007fd0 tx=0x7ff380007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.647+0000 7ff386ffd640 1 -- 172.21.15.113:0/1518325158 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3780596f0 con 0x7ff3900b3fd0 2024-04-25T00:04:36.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.823+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff39015f870 con 0x7ff3900b3fd0 2024-04-25T00:04:36.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.824+0000 7ff386ffd640 1 -- 172.21.15.113:0/1518325158 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16708 (secure 0 0 0) 0x7ff37805d9f0 con 0x7ff3900b3fd0 2024-04-25T00:04:36.825 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:07.938101+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:04.509800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:05.258783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff37006be60 msgr2=0x7ff37006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff37006be60 0x7ff37006e320 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff380007fd0 tx=0x7ff380007e80 comp rx=0 tx=0).stop 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3900b3fd0 msgr2=0x7ff390132ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3900b3fd0 0x7ff390132ca0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff37800d830 tx=0x7ff37800dcf0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 shutdown_connections 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff37006be60 0x7ff37006e320 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff390144910 0x7ff39013ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3900b3fd0 0x7ff390132ca0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 --2- 172.21.15.113:0/1518325158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff39006e510 0x7ff39015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 >> 172.21.15.113:0/1518325158 conn(0x7ff3900b1a50 msgr2=0x7ff39015eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 shutdown_connections 2024-04-25T00:04:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.826+0000 7ff394eee640 1 -- 172.21.15.113:0/1518325158 wait complete. 2024-04-25T00:04:36.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:04:04.509800+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:04:05.258783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:04:36.839 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:04:36.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.938+0000 7f1a26f9e640 1 Processor -- start 2024-04-25T00:04:36.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a26f9e640 1 -- start start 2024-04-25T00:04:36.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a26f9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 0x7f1a200cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a26f9e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1a200b6160 0x7f1a201624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a26f9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a20162bb0 con 0x7f1a20162040 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a26f9e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1a200cf030 con 0x7f1a200b6160 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a26f9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1a20162e90 con 0x7f1a2015e3e0 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a2579b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1a20162040 0x7f1a200b5a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39790/0 (socket says 172.21.15.113:39790) 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.939+0000 7f1a2579b640 1 -- 172.21.15.113:0/3648251581 learned_addr learned my addr 172.21.15.113:0/3648251581 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2679d640 1 --2- 172.21.15.113:0/3648251581 >> v2:172.21.15.178:3300/0 conn(0x7f1a200b6160 0x7f1a201624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a25f9c640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 0x7f1a200cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3855271579 0 0) 0x7f1a20162bb0 con 0x7f1a20162040 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a0c003520 con 0x7f1a20162040 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 924936236 0 0) 0x7f1a0c003520 con 0x7f1a20162040 2024-04-25T00:04:36.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 msgr2=0x7f1a200cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 0x7f1a200cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 >> v2:172.21.15.178:3300/0 conn(0x7f1a200b6160 msgr2=0x7f1a201624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 --2- 172.21.15.113:0/3648251581 >> v2:172.21.15.178:3300/0 conn(0x7f1a200b6160 0x7f1a201624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a200ce9e0 con 0x7f1a20162040 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1a100016c0 con 0x7f1a20162040 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 >> v1:172.21.15.113:6789/0 conn(0x7f1a20162040 legacy=0x7f1a200b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a0c0039b0 0x7f1a0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 0x7f1a0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a0c004d40 0x7f1a0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a200cf030 con 0x7f1a0c004d40 2024-04-25T00:04:36.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1a20162e90 con 0x7f1a0c0039b0 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c00b9c0 con 0x7f1a2015e3e0 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2679d640 1 --2- 172.21.15.113:0/3648251581 >> v2:172.21.15.178:3300/0 conn(0x7f1a200b6160 0x7f1a201624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a25f9c640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a0c0039b0 0x7f1a0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2679d640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a0c004d40 0x7f1a0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2579b640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 0x7f1a0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2679d640 1 -- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 msgr2=0x7f1a0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2679d640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a2015e3e0 0x7f1a0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2679d640 1 -- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a0c0039b0 msgr2=0x7f1a0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.940+0000 7f1a2679d640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a0c0039b0 0x7f1a0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.941+0000 7f1a2679d640 1 -- 172.21.15.113:0/3648251581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c003520 con 0x7f1a0c004d40 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.941+0000 7f1a2679d640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a0c004d40 0x7f1a0c0051c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009c60 tx=0x7f1a0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3102f10bac334565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.941+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18036070 con 0x7f1a0c004d40 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.941+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18031400 con 0x7f1a0c004d40 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.941+0000 7f1a24f9a640 1 -- 172.21.15.113:0/3648251581 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18009860 con 0x7f1a0c004d40 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.942+0000 7f1a26f9e640 1 -- 172.21.15.113:0/3648251581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a0c004d40 msgr2=0x7f1a0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.942+0000 7f1a26f9e640 1 --2- 172.21.15.113:0/3648251581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a0c004d40 0x7f1a0c0051c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009c60 tx=0x7f1a0c005830 comp rx=0 tx=0).stop 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.942+0000 7f1a2679d640 1 -- 172.21.15.113:0/3648251581 reap_dead start 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.942+0000 7f1a26f9e640 1 -- 172.21.15.113:0/3648251581 shutdown_connections 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.942+0000 7f1a26f9e640 1 -- 172.21.15.113:0/3648251581 >> 172.21.15.113:0/3648251581 conn(0x7f1a20071da0 msgr2=0x7f1a200d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.942+0000 7f1a26f9e640 1 -- 172.21.15.113:0/3648251581 shutdown_connections 2024-04-25T00:04:36.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.942+0000 7f1a26f9e640 1 -- 172.21.15.113:0/3648251581 wait complete. 2024-04-25T00:04:36.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.943+0000 7f1a26f9e640 1 Processor -- start 2024-04-25T00:04:36.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.943+0000 7f1a26f9e640 1 -- start start 2024-04-25T00:04:36.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a26f9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a20150300 0x7f1a201f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a26f9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a201602a0 0x7f1a20160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a26f9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a201e92b0 0x7f1a201e9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a26f9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2015e7c0 con 0x7f1a201e92b0 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a26f9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1a200b37e0 con 0x7f1a201602a0 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a26f9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2015e940 con 0x7f1a20150300 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a201e92b0 0x7f1a201e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a201e92b0 0x7f1a201e9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35434/0 (socket says 172.21.15.113:35434) 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 -- 172.21.15.113:0/2668850797 learned_addr learned my addr 172.21.15.113:0/2668850797 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2579b640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a201602a0 0x7f1a20160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a25f9c640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a20150300 0x7f1a201f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 -- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a20150300 msgr2=0x7f1a201f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a20150300 0x7f1a201f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 -- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a201602a0 msgr2=0x7f1a20160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:36.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a201602a0 0x7f1a20160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 -- 172.21.15.113:0/2668850797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a201e8210 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.944+0000 7f1a2679d640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a201e92b0 0x7f1a201e9730 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009d80 tx=0x7f1a18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.945+0000 7f1a167fc640 1 -- 172.21.15.113:0/2668850797 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18036070 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.945+0000 7f1a167fc640 1 -- 172.21.15.113:0/2668850797 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18037040 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.945+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a201e84f0 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.945+0000 7f1a167fc640 1 -- 172.21.15.113:0/2668850797 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18009860 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.945+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a201e8ab0 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.946+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a200691a0 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.946+0000 7f1a167fc640 1 -- 172.21.15.113:0/2668850797 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a18043020 con 0x7f1a201e92b0 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.947+0000 7f1a167fc640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a0406be30 0x7f1a0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:36.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.947+0000 7f1a25f9c640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a0406be30 0x7f1a0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:36.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.947+0000 7f1a167fc640 1 -- 172.21.15.113:0/2668850797 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f1a180aa7b0 con 0x7f1a201e92b0 2024-04-25T00:04:36.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.947+0000 7f1a25f9c640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a0406be30 0x7f1a0406e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c014450 tx=0x7f1a1c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:36.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:36.951+0000 7f1a167fc640 1 -- 172.21.15.113:0/2668850797 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a18078a80 con 0x7f1a201e92b0 2024-04-25T00:04:37.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.124+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a200d1e90 con 0x7f1a201e92b0 2024-04-25T00:04:37.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.125+0000 7f1a167fc640 1 -- 172.21.15.113:0/2668850797 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f1a1807cd80 con 0x7f1a201e92b0 2024-04-25T00:04:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:04:37.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a0406be30 msgr2=0x7f1a0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a0406be30 0x7f1a0406e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c014450 tx=0x7f1a1c00b040 comp rx=0 tx=0).stop 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a201e92b0 msgr2=0x7f1a201e9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a201e92b0 0x7f1a201e9730 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009d80 tx=0x7f1a18004730 comp rx=0 tx=0).stop 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 shutdown_connections 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1a0406be30 0x7f1a0406e2f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a201602a0 0x7f1a20160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a201e92b0 0x7f1a201e9730 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 --2- 172.21.15.113:0/2668850797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1a20150300 0x7f1a201f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 >> 172.21.15.113:0/2668850797 conn(0x7f1a20071da0 msgr2=0x7f1a200d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 shutdown_connections 2024-04-25T00:04:37.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.127+0000 7f1a26f9e640 1 -- 172.21.15.113:0/2668850797 wait complete. 2024-04-25T00:04:37.139 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:04:37.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.239+0000 7fb645c72640 1 Processor -- start 2024-04-25T00:04:37.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.239+0000 7fb645c72640 1 -- start start 2024-04-25T00:04:37.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb645c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 0x7fb64015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb645c72640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6401604d0 0x7fb6400cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb645c72640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64015d6b0 con 0x7fb64013f050 2024-04-25T00:04:37.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb645c72640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb64015c6a0 con 0x7fb6401604d0 2024-04-25T00:04:37.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb645c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb64015c8f0 con 0x7fb6400d06c0 2024-04-25T00:04:37.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb64013f050 0x7fb6400cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39806/0 (socket says 172.21.15.113:39806) 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63f7fe640 1 -- 172.21.15.113:0/935026392 learned_addr learned my addr 172.21.15.113:0/935026392 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336923158 0 0) 0x7fb64015d6b0 con 0x7fb64013f050 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63ffff640 1 --2- 172.21.15.113:0/935026392 >> v2:172.21.15.178:3300/0 conn(0x7fb6401604d0 0x7fb6400cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb628003520 con 0x7fb64013f050 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63effd640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 0x7fb64015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 437883206 0 0) 0x7fb628003520 con 0x7fb64013f050 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 msgr2=0x7fb64015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 0x7fb64015ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 >> v2:172.21.15.178:3300/0 conn(0x7fb6401604d0 msgr2=0x7fb6400cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 --2- 172.21.15.113:0/935026392 >> v2:172.21.15.178:3300/0 conn(0x7fb6401604d0 0x7fb6400cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64015cbd0 con 0x7fb64013f050 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.240+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fb630001670 con 0x7fb64013f050 2024-04-25T00:04:37.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 >> v1:172.21.15.113:6789/0 conn(0x7fb64013f050 legacy=0x7fb6400cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63e7fc640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 0x7fb6280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63e7fc640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb628003f30 0x7fb628004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63e7fc640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb628004d40 0x7fb6280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb64015c6a0 con 0x7fb628004d40 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb64015c8f0 con 0x7fb628003f30 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb62800ba10 con 0x7fb6400d06c0 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63ffff640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb628004d40 0x7fb6280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63f7fe640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 0x7fb6280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63effd640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb628003f30 0x7fb628004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63ffff640 1 -- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 msgr2=0x7fb6280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63ffff640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6400d06c0 0x7fb6280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63ffff640 1 -- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb628003f30 msgr2=0x7fb628004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:37.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63ffff640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb628003f30 0x7fb628004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.241+0000 7fb63ffff640 1 -- 172.21.15.113:0/935026392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb628003520 con 0x7fb628004d40 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb63ffff640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb628004d40 0x7fb6280051c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e170 tx=0x7fb628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670193ad1507e8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638034020 con 0x7fb628004d40 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb638030430 con 0x7fb628004d40 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb63e7fc640 1 -- 172.21.15.113:0/935026392 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638009c10 con 0x7fb628004d40 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb645c72640 1 -- 172.21.15.113:0/935026392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb628004d40 msgr2=0x7fb6280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb645c72640 1 --2- 172.21.15.113:0/935026392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb628004d40 0x7fb6280051c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e170 tx=0x7fb628005830 comp rx=0 tx=0).stop 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb63ffff640 1 -- 172.21.15.113:0/935026392 reap_dead start 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb645c72640 1 -- 172.21.15.113:0/935026392 shutdown_connections 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb645c72640 1 -- 172.21.15.113:0/935026392 >> 172.21.15.113:0/935026392 conn(0x7fb64006b0f0 msgr2=0x7fb64006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb645c72640 1 -- 172.21.15.113:0/935026392 shutdown_connections 2024-04-25T00:04:37.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.242+0000 7fb645c72640 1 -- 172.21.15.113:0/935026392 wait complete. 2024-04-25T00:04:37.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.243+0000 7fb645c72640 1 Processor -- start 2024-04-25T00:04:37.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb645c72640 1 -- start start 2024-04-25T00:04:37.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb645c72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64013db80 0x7fb6401f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb645c72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb64014b800 0x7fb64015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb645c72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb64015b740 0x7fb640155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb645c72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb64013f430 con 0x7fb64014b800 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb645c72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb64015d190 con 0x7fb64015b740 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb645c72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6401608d0 con 0x7fb64013db80 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb63effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb64014b800 0x7fb64015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.244+0000 7fb63effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb64014b800 0x7fb64015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35448/0 (socket says 172.21.15.113:35448) 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63effd640 1 -- 172.21.15.113:0/1930120051 learned_addr learned my addr 172.21.15.113:0/1930120051 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63ffff640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb64015b740 0x7fb640155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63f7fe640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64013db80 0x7fb6401f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63effd640 1 -- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64013db80 msgr2=0x7fb6401f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63effd640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64013db80 0x7fb6401f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63effd640 1 -- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb64015b740 msgr2=0x7fb640155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63effd640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb64015b740 0x7fb640155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63effd640 1 -- 172.21.15.113:0/1930120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64014c030 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63ffff640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb64015b740 0x7fb640155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb63effd640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb64014b800 0x7fb64015b200 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb634011960 tx=0x7fb634011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb627fff640 1 -- 172.21.15.113:0/1930120051 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634019070 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb627fff640 1 -- 172.21.15.113:0/1930120051 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb634004240 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb640149310 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb627fff640 1 -- 172.21.15.113:0/1930120051 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63400f820 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.245+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6401498d0 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.246+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64015bb50 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.246+0000 7fb627fff640 1 -- 172.21.15.113:0/1930120051 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb63401c070 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.247+0000 7fb627fff640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62c06be30 0x7fb62c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.247+0000 7fb627fff640 1 -- 172.21.15.113:0/1930120051 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fb63408b1d0 con 0x7fb64014b800 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.247+0000 7fb63f7fe640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62c06be30 0x7fb62c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.247+0000 7fb63f7fe640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62c06be30 0x7fb62c06e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb630004840 tx=0x7fb630009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.251+0000 7fb627fff640 1 -- 172.21.15.113:0/1930120051 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb64015bb50 con 0x7fb64014b800 2024-04-25T00:04:37.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.423+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb640160250 con 0x7fb64014b800 2024-04-25T00:04:37.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.423+0000 7fb627fff640 1 -- 172.21.15.113:0/1930120051 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fb64014c030 con 0x7fb64014b800 2024-04-25T00:04:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62c06be30 msgr2=0x7fb62c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62c06be30 0x7fb62c06e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb630004840 tx=0x7fb630009040 comp rx=0 tx=0).stop 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb64014b800 msgr2=0x7fb64015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb64014b800 0x7fb64015b200 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb634011960 tx=0x7fb634011e20 comp rx=0 tx=0).stop 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 shutdown_connections 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62c06be30 0x7fb62c06e2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb64015b740 0x7fb640155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb64014b800 0x7fb64015b200 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 --2- 172.21.15.113:0/1930120051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64013db80 0x7fb6401f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 >> 172.21.15.113:0/1930120051 conn(0x7fb64006b0f0 msgr2=0x7fb6400cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 shutdown_connections 2024-04-25T00:04:37.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.425+0000 7fb645c72640 1 -- 172.21.15.113:0/1930120051 wait complete. 2024-04-25T00:04:37.437 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:04:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.536+0000 7f95cd99e640 1 Processor -- start 2024-04-25T00:04:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.536+0000 7f95cd99e640 1 -- start start 2024-04-25T00:04:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.536+0000 7f95cd99e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95c8161ff0 0x7f95c8162480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.536+0000 7f95cd99e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c80d2ce0 con 0x7f95c806bb40 2024-04-25T00:04:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.536+0000 7f95cd99e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c80d0170 con 0x7f95c8158a00 2024-04-25T00:04:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.536+0000 7f95cd99e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f95c806b0f0 con 0x7f95c8161ff0 2024-04-25T00:04:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.537+0000 7f95c67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95c8161ff0 0x7f95c8162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.537+0000 7f95c77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f95c806bb40 0x7f95c806bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37146/0 (socket says 172.21.15.113:37146) 2024-04-25T00:04:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.537+0000 7f95c77fe640 1 -- 172.21.15.113:0/368333559 learned_addr learned my addr 172.21.15.113:0/368333559 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.537+0000 7f95c67fc640 1 -- 172.21.15.113:0/368333559 >> v1:172.21.15.178:6789/0 conn(0x7f95c8158a00 legacy=0x7f95c80d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.537+0000 7f95c5ffb640 1 -- 172.21.15.113:0/368333559 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2493074537 0 0) 0x7f95c80d2ce0 con 0x7f95c806bb40 2024-04-25T00:04:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.537+0000 7f95c67fc640 1 -- 172.21.15.113:0/368333559 >> v1:172.21.15.113:6789/0 conn(0x7f95c806bb40 legacy=0x7f95c806bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.537+0000 7f95c67fc640 1 -- 172.21.15.113:0/368333559 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c80d0450 con 0x7f95c8161ff0 2024-04-25T00:04:37.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95c67fc640 1 --2- 172.21.15.113:0/368333559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8161ff0 0x7f95c8162480 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0097a0 tx=0x7f95bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c064fc5b4a56d924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95c5ffb640 1 -- 172.21.15.113:0/368333559 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc037070 con 0x7f95c8161ff0 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95c5ffb640 1 -- 172.21.15.113:0/368333559 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc032440 con 0x7f95c8161ff0 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95c5ffb640 1 -- 172.21.15.113:0/368333559 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc032810 con 0x7f95c8161ff0 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95cd99e640 1 -- 172.21.15.113:0/368333559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8161ff0 msgr2=0x7f95c8162480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95cd99e640 1 --2- 172.21.15.113:0/368333559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8161ff0 0x7f95c8162480 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0097a0 tx=0x7f95bc02a230 comp rx=0 tx=0).stop 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95cd99e640 1 -- 172.21.15.113:0/368333559 shutdown_connections 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95cd99e640 1 --2- 172.21.15.113:0/368333559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8161ff0 0x7f95c8162480 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95cd99e640 1 -- 172.21.15.113:0/368333559 >> 172.21.15.113:0/368333559 conn(0x7f95c80b3d80 msgr2=0x7f95c80b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95cd99e640 1 -- 172.21.15.113:0/368333559 shutdown_connections 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.538+0000 7f95cd99e640 1 -- 172.21.15.113:0/368333559 wait complete. 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.539+0000 7f95cd99e640 1 Processor -- start 2024-04-25T00:04:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95cd99e640 1 -- start start 2024-04-25T00:04:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95cd99e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95c806bb40 0x7f95c8144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95cd99e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8158a00 0x7f95c80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95cd99e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95c8161ff0 0x7f95c80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95cd99e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f95c80cfe40 con 0x7f95c8158a00 2024-04-25T00:04:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95cd99e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95c806b6b0 con 0x7f95c8161ff0 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95cd99e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f95c8158de0 con 0x7f95c806bb40 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95c67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8158a00 0x7f95c80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95c67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8158a00 0x7f95c80b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49540/0 (socket says 172.21.15.113:49540) 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.540+0000 7f95c67fc640 1 -- 172.21.15.113:0/3700093678 learned_addr learned my addr 172.21.15.113:0/3700093678 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c77fe640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95c8161ff0 0x7f95c80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c6ffd640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95c806bb40 0x7f95c8144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c67fc640 1 -- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95c806bb40 msgr2=0x7f95c8144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c67fc640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95c806bb40 0x7f95c8144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c67fc640 1 -- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95c8161ff0 msgr2=0x7f95c80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c67fc640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95c8161ff0 0x7f95c80ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c67fc640 1 -- 172.21.15.113:0/3700093678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c80c69e0 con 0x7f95c8158a00 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c77fe640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95c8161ff0 0x7f95c80ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c6ffd640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95c806bb40 0x7f95c8144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95c67fc640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8158a00 0x7f95c80b23f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000c00 tx=0x7f95bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95affff640 1 -- 172.21.15.113:0/3700093678 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc037040 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95affff640 1 -- 172.21.15.113:0/3700093678 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc042070 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c80c7450 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95affff640 1 -- 172.21.15.113:0/3700093678 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc030a70 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.541+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c8144640 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.542+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c80691a0 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.542+0000 7f95affff640 1 -- 172.21.15.113:0/3700093678 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95bc030c10 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.543+0000 7f95affff640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f95b406be30 0x7f95b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.543+0000 7f95affff640 1 -- 172.21.15.113:0/3700093678 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f95bc0ab9b0 con 0x7f95c8158a00 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.543+0000 7f95c6ffd640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f95b406be30 0x7f95b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.543+0000 7f95c6ffd640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f95b406be30 0x7f95b406e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000f60 tx=0x7f95b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.547+0000 7f95affff640 1 -- 172.21.15.113:0/3700093678 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f95bc07a360 con 0x7f95c8158a00 2024-04-25T00:04:37.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.718+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95c80b81f0 con 0x7f95c8158a00 2024-04-25T00:04:37.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.718+0000 7f95affff640 1 -- 172.21.15.113:0/3700093678 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f95bc079c60 con 0x7f95c8158a00 2024-04-25T00:04:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:04:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f95b406be30 msgr2=0x7f95b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f95b406be30 0x7f95b406e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000f60 tx=0x7f95b8009040 comp rx=0 tx=0).stop 2024-04-25T00:04:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8158a00 msgr2=0x7f95c80b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8158a00 0x7f95c80b23f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000c00 tx=0x7f95bc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 shutdown_connections 2024-04-25T00:04:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f95b406be30 0x7f95b406e2f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95c8161ff0 0x7f95c80ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c8158a00 0x7f95c80b23f0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 --2- 172.21.15.113:0/3700093678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95c806bb40 0x7f95c8144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 >> 172.21.15.113:0/3700093678 conn(0x7f95c80b3d80 msgr2=0x7f95c80b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 shutdown_connections 2024-04-25T00:04:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.720+0000 7f95cd99e640 1 -- 172.21.15.113:0/3700093678 wait complete. 2024-04-25T00:04:37.732 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:37.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.828+0000 7fe63e57e640 1 Processor -- start 2024-04-25T00:04:37.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.828+0000 7fe63e57e640 1 -- start start 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe63e57e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6380d0b00 0x7fe6380d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe63e57e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6380d1480 con 0x7fe6380d2290 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe63e57e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6380d2800 con 0x7fe63815e320 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe63e57e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe6380d3410 con 0x7fe6380d0b00 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe63cb1b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6380d0b00 0x7fe6380d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe6377fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe6380d2290 0x7fe6380b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37156/0 (socket says 172.21.15.113:37156) 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe63cb1b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6380d0b00 0x7fe6380d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49550/0 (socket says 172.21.15.113:49550) 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.829+0000 7fe6377fe640 1 -- 172.21.15.113:0/3209603243 learned_addr learned my addr 172.21.15.113:0/3209603243 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.830+0000 7fe63cb1b640 1 -- 172.21.15.113:0/3209603243 >> v1:172.21.15.178:6789/0 conn(0x7fe63815e320 legacy=0x7fe6380d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:37.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.830+0000 7fe63cb1b640 1 -- 172.21.15.113:0/3209603243 >> v1:172.21.15.113:6789/0 conn(0x7fe6380d2290 legacy=0x7fe6380b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.830+0000 7fe63cb1b640 1 -- 172.21.15.113:0/3209603243 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380d2ae0 con 0x7fe6380d0b00 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.830+0000 7fe63cb1b640 1 --2- 172.21.15.113:0/3209603243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380d0f40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe630009d90 tx=0x7fe63002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b060d5db896fe60f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.830+0000 7fe636ffd640 1 -- 172.21.15.113:0/3209603243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630007740 con 0x7fe6380d0b00 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.830+0000 7fe636ffd640 1 -- 172.21.15.113:0/3209603243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6300078e0 con 0x7fe6380d0b00 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.830+0000 7fe636ffd640 1 -- 172.21.15.113:0/3209603243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630007cb0 con 0x7fe6380d0b00 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.831+0000 7fe63e57e640 1 -- 172.21.15.113:0/3209603243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 msgr2=0x7fe6380d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.831+0000 7fe63e57e640 1 --2- 172.21.15.113:0/3209603243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380d0f40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe630009d90 tx=0x7fe63002a730 comp rx=0 tx=0).stop 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.831+0000 7fe63e57e640 1 -- 172.21.15.113:0/3209603243 shutdown_connections 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.831+0000 7fe63e57e640 1 --2- 172.21.15.113:0/3209603243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380d0f40 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.831+0000 7fe63e57e640 1 -- 172.21.15.113:0/3209603243 >> 172.21.15.113:0/3209603243 conn(0x7fe6380b5c20 msgr2=0x7fe6380b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:37.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.831+0000 7fe63e57e640 1 -- 172.21.15.113:0/3209603243 shutdown_connections 2024-04-25T00:04:37.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.831+0000 7fe63e57e640 1 -- 172.21.15.113:0/3209603243 wait complete. 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.832+0000 7fe63e57e640 1 Processor -- start 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.832+0000 7fe63e57e640 1 -- start start 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63e57e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63e57e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6380d2290 0x7fe638154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63e57e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe63815e320 0x7fe6381565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63e57e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe63815e700 con 0x7fe6380d0b00 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63e57e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6380d30a0 con 0x7fe63815e320 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63e57e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe63815e880 con 0x7fe6380d2290 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49560/0 (socket says 172.21.15.113:49560) 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 -- 172.21.15.113:0/978293426 learned_addr learned my addr 172.21.15.113:0/978293426 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 -- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6380d2290 msgr2=0x7fe638154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6380d2290 0x7fe638154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 -- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe63815e320 msgr2=0x7fe6381565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63cb1b640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe63815e320 0x7fe6381565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe63815e320 0x7fe6381565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe637fff640 1 -- 172.21.15.113:0/978293426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380cc8a0 con 0x7fe6380d0b00 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.833+0000 7fe63cb1b640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe63815e320 0x7fe6381565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.834+0000 7fe637fff640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380cc360 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fe62800c430 tx=0x7fe62800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.834+0000 7fe634ff9640 1 -- 172.21.15.113:0/978293426 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628011790 con 0x7fe6380d0b00 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.834+0000 7fe634ff9640 1 -- 172.21.15.113:0/978293426 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe628017070 con 0x7fe6380d0b00 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.834+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6380ccb40 con 0x7fe6380d0b00 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.834+0000 7fe634ff9640 1 -- 172.21.15.113:0/978293426 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628011b70 con 0x7fe6380d0b00 2024-04-25T00:04:37.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.834+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6380cd100 con 0x7fe6380d0b00 2024-04-25T00:04:37.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.835+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6380691a0 con 0x7fe6380d0b00 2024-04-25T00:04:37.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.835+0000 7fe634ff9640 1 -- 172.21.15.113:0/978293426 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe628011d10 con 0x7fe6380d0b00 2024-04-25T00:04:37.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.835+0000 7fe634ff9640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe62406be30 0x7fe62406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:37.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.836+0000 7fe634ff9640 1 -- 172.21.15.113:0/978293426 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fe62808adc0 con 0x7fe6380d0b00 2024-04-25T00:04:37.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.836+0000 7fe6377fe640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe62406be30 0x7fe62406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:37.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.836+0000 7fe6377fe640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe62406be30 0x7fe62406e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe62c000f60 tx=0x7fe62c007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:37.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:37.840+0000 7fe634ff9640 1 -- 172.21.15.113:0/978293426 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6280596e0 con 0x7fe6380d0b00 2024-04-25T00:04:38.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.053+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe63815cdb0 con 0x7fe6380d0b00 2024-04-25T00:04:38.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.054+0000 7fe634ff9640 1 -- 172.21.15.113:0/978293426 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5195 (secure 0 0 0) 0x7fe628058fe0 con 0x7fe6380d0b00 2024-04-25T00:04:38.055 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:06.288869+0000","modified":"2024-04-25T00:04:08.290392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2796979045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2796979045},{"type":"v1","addr":"172.21.15.113:6837","nonce":2796979045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7395,"qdb_cluster":[7395]},"id":6}]} 2024-04-25T00:04:38.056 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-04-25T00:04:38.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe62406be30 msgr2=0x7fe62406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe62406be30 0x7fe62406e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe62c000f60 tx=0x7fe62c007a80 comp rx=0 tx=0).stop 2024-04-25T00:04:38.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 msgr2=0x7fe6380cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380cc360 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fe62800c430 tx=0x7fe62800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 shutdown_connections 2024-04-25T00:04:38.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe62406be30 0x7fe62406e2f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe63815e320 0x7fe6381565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6380d2290 0x7fe638154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 --2- 172.21.15.113:0/978293426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6380d0b00 0x7fe6380cc360 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 >> 172.21.15.113:0/978293426 conn(0x7fe6380b5c20 msgr2=0x7fe6380b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:38.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 shutdown_connections 2024-04-25T00:04:38.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.056+0000 7fe63e57e640 1 -- 172.21.15.113:0/978293426 wait complete. 2024-04-25T00:04:38.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:38.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.164+0000 7f13a6bd5640 1 Processor -- start 2024-04-25T00:04:38.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.164+0000 7f13a6bd5640 1 -- start start 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f13a6bd5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f13a6bd5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f13a006e510 0x7f13a00b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f13a6bd5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a00b3c20 con 0x7f13a015e240 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f13a6bd5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f13a006e960 con 0x7f13a006e510 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f13a6bd5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13a0072d90 con 0x7f13a00b5620 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f139ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f139ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:36132/0 (socket says 172.21.15.113:36132) 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f139ffff640 1 -- 172.21.15.113:0/2080741749 learned_addr learned my addr 172.21.15.113:0/2080741749 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f13a5172640 1 --2- 172.21.15.113:0/2080741749 >> v2:172.21.15.178:3300/0 conn(0x7f13a006e510 0x7f13a00b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f139ffff640 1 -- 172.21.15.113:0/2080741749 >> v2:172.21.15.178:3300/0 conn(0x7f13a006e510 msgr2=0x7f13a00b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f139ffff640 1 --2- 172.21.15.113:0/2080741749 >> v2:172.21.15.178:3300/0 conn(0x7f13a006e510 0x7f13a00b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f139ffff640 1 -- 172.21.15.113:0/2080741749 >> v1:172.21.15.178:6789/0 conn(0x7f13a015e240 legacy=0x7f13a01627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.165+0000 7f139ffff640 1 -- 172.21.15.113:0/2080741749 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a015e620 con 0x7f13a00b5620 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.166+0000 7f139ffff640 1 --2- 172.21.15.113:0/2080741749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a006dfd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f138c009870 tx=0x7f138c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=304a3700ef9a1e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.166+0000 7f139f7fe640 1 -- 172.21.15.113:0/2080741749 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c037070 con 0x7f13a00b5620 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.166+0000 7f139f7fe640 1 -- 172.21.15.113:0/2080741749 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138c032440 con 0x7f13a00b5620 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.166+0000 7f139f7fe640 1 -- 172.21.15.113:0/2080741749 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c032810 con 0x7f13a00b5620 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 -- 172.21.15.113:0/2080741749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 msgr2=0x7f13a006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/2080741749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a006dfd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f138c009870 tx=0x7f138c02a270 comp rx=0 tx=0).stop 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 -- 172.21.15.113:0/2080741749 shutdown_connections 2024-04-25T00:04:38.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/2080741749 >> v2:172.21.15.178:3300/0 conn(0x7f13a006e510 0x7f13a00b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/2080741749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a006dfd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 -- 172.21.15.113:0/2080741749 >> 172.21.15.113:0/2080741749 conn(0x7f13a00b1a50 msgr2=0x7f13a00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:38.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 -- 172.21.15.113:0/2080741749 shutdown_connections 2024-04-25T00:04:38.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.167+0000 7f13a6bd5640 1 -- 172.21.15.113:0/2080741749 wait complete. 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.168+0000 7f13a6bd5640 1 Processor -- start 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.168+0000 7f13a6bd5640 1 -- start start 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a6bd5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a006e510 0x7f13a01f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a6bd5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a01427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a6bd5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015e240 0x7f13a01582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a6bd5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00727d0 con 0x7f13a015e240 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a6bd5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00724f0 con 0x7f13a006e510 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a6bd5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13a0072ab0 con 0x7f13a00b5620 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015e240 0x7f13a01582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015e240 0x7f13a01582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49574/0 (socket says 172.21.15.113:49574) 2024-04-25T00:04:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 -- 172.21.15.113:0/816988977 learned_addr learned my addr 172.21.15.113:0/816988977 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f139ffff640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a01427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a4971640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a006e510 0x7f13a01f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 -- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 msgr2=0x7f13a01427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a01427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 -- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a006e510 msgr2=0x7f13a01f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a006e510 0x7f13a01f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a5172640 1 -- 172.21.15.113:0/816988977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a0159ba0 con 0x7f13a015e240 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f139ffff640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a01427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.169+0000 7f13a4971640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a006e510 0x7f13a01f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.170+0000 7f13a5172640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015e240 0x7f13a01582b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f139800ec10 tx=0x7f139800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.170+0000 7f139d7fa640 1 -- 172.21.15.113:0/816988977 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398018020 con 0x7f13a015e240 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.170+0000 7f139d7fa640 1 -- 172.21.15.113:0/816988977 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139800cd20 con 0x7f13a015e240 2024-04-25T00:04:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.170+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a0158b00 con 0x7f13a015e240 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.170+0000 7f139d7fa640 1 -- 172.21.15.113:0/816988977 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13980146c0 con 0x7f13a015e240 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.170+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01590c0 con 0x7f13a015e240 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.171+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a00691a0 con 0x7f13a015e240 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.171+0000 7f139d7fa640 1 -- 172.21.15.113:0/816988977 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1398005d10 con 0x7f13a015e240 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.171+0000 7f139d7fa640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138406be30 0x7f138406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.172+0000 7f139d7fa640 1 -- 172.21.15.113:0/816988977 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f139808b800 con 0x7f13a015e240 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.172+0000 7f13a4971640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138406be30 0x7f138406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.172+0000 7f13a4971640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138406be30 0x7f138406e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f13a015f320 tx=0x7f1394009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.176+0000 7f139d7fa640 1 -- 172.21.15.113:0/816988977 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1398022070 con 0x7f13a015e240 2024-04-25T00:04:38.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.350+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13a0161b40 con 0x7f13a015e240 2024-04-25T00:04:38.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.352+0000 7f139d7fa640 1 -- 172.21.15.113:0/816988977 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16708 (secure 0 0 0) 0x7f1398010040 con 0x7f13a015e240 2024-04-25T00:04:38.352 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:07.938101+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:04.509800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:05.258783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:38.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138406be30 msgr2=0x7f138406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138406be30 0x7f138406e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f13a015f320 tx=0x7f1394009040 comp rx=0 tx=0).stop 2024-04-25T00:04:38.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015e240 msgr2=0x7f13a01582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015e240 0x7f13a01582b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f139800ec10 tx=0x7f139800c490 comp rx=0 tx=0).stop 2024-04-25T00:04:38.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 shutdown_connections 2024-04-25T00:04:38.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138406be30 0x7f138406e2f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a015e240 0x7f13a01582b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a00b5620 0x7f13a01427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 --2- 172.21.15.113:0/816988977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a006e510 0x7f13a01f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 >> 172.21.15.113:0/816988977 conn(0x7f13a00b1a50 msgr2=0x7f13a006cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:38.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 shutdown_connections 2024-04-25T00:04:38.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.354+0000 7f13a6bd5640 1 -- 172.21.15.113:0/816988977 wait complete. 2024-04-25T00:04:38.365 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:04:38.366 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.462+0000 7f646aa88640 1 Processor -- start 2024-04-25T00:04:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.462+0000 7f646aa88640 1 -- start start 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f646aa88640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64640b3fd0 0x7f646406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f646aa88640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64640cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f646aa88640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6464144cf0 con 0x7f6464144910 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f646aa88640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f64640b2f30 con 0x7f64640b3fd0 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f646aa88640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64640b3210 con 0x7f646406e510 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f6463fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64640b3fd0 0x7f646406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f6463fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64640b3fd0 0x7f646406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49586/0 (socket says 172.21.15.113:49586) 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f6463fff640 1 -- 172.21.15.113:0/527476243 learned_addr learned my addr 172.21.15.113:0/527476243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f6469025640 1 --2- 172.21.15.113:0/527476243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64640cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f6463fff640 1 -- 172.21.15.113:0/527476243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 msgr2=0x7f64640cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:38.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.463+0000 7f6463fff640 1 --2- 172.21.15.113:0/527476243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64640cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.464+0000 7f6463fff640 1 -- 172.21.15.113:0/527476243 >> v1:172.21.15.178:6789/0 conn(0x7f6464144910 legacy=0x7f64640b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.464+0000 7f6463fff640 1 -- 172.21.15.113:0/527476243 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64640b34f0 con 0x7f64640b3fd0 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.464+0000 7f6463fff640 1 --2- 172.21.15.113:0/527476243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64640b3fd0 0x7f646406dfd0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f645800bb40 tx=0x7f645802c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5335d92c9bddb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.464+0000 7f64637fe640 1 -- 172.21.15.113:0/527476243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458036020 con 0x7f64640b3fd0 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.464+0000 7f64637fe640 1 -- 172.21.15.113:0/527476243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458032820 con 0x7f64640b3fd0 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.464+0000 7f64637fe640 1 -- 172.21.15.113:0/527476243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458032bf0 con 0x7f64640b3fd0 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 -- 172.21.15.113:0/527476243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64640b3fd0 msgr2=0x7f646406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 --2- 172.21.15.113:0/527476243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64640b3fd0 0x7f646406dfd0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f645800bb40 tx=0x7f645802c960 comp rx=0 tx=0).stop 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 -- 172.21.15.113:0/527476243 shutdown_connections 2024-04-25T00:04:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 --2- 172.21.15.113:0/527476243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64640cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 --2- 172.21.15.113:0/527476243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64640b3fd0 0x7f646406dfd0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 -- 172.21.15.113:0/527476243 >> 172.21.15.113:0/527476243 conn(0x7f64640b1a50 msgr2=0x7f64640b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 -- 172.21.15.113:0/527476243 shutdown_connections 2024-04-25T00:04:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.465+0000 7f646aa88640 1 -- 172.21.15.113:0/527476243 wait complete. 2024-04-25T00:04:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.466+0000 7f646aa88640 1 Processor -- start 2024-04-25T00:04:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.466+0000 7f646aa88640 1 -- start start 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f646aa88640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64641f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f646aa88640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64640b3fd0 0x7f64641f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f646aa88640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6464144910 0x7f64640c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f646aa88640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f64640b4da0 con 0x7f6464144910 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f646aa88640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64640b4440 con 0x7f64640b3fd0 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f646aa88640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64640b4f20 con 0x7f646406e510 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6464144910 0x7f64640c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6464144910 0x7f64640c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49588/0 (socket says 172.21.15.113:49588) 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 -- 172.21.15.113:0/3692322089 learned_addr learned my addr 172.21.15.113:0/3692322089 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6463fff640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64640b3fd0 0x7f64641f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6468824640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64641f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 -- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 msgr2=0x7f64641f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64641f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 -- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64640b3fd0 msgr2=0x7f64641f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64640b3fd0 0x7f64641f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 -- 172.21.15.113:0/3692322089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6464134350 con 0x7f6464144910 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.467+0000 7f6469025640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6464144910 0x7f64640c5d40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f645c007cf0 tx=0x7f645c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.468+0000 7f64617fa640 1 -- 172.21.15.113:0/3692322089 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c014420 con 0x7f6464144910 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.468+0000 7f64617fa640 1 -- 172.21.15.113:0/3692322089 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645c019070 con 0x7f6464144910 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.468+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6464134630 con 0x7f6464144910 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.468+0000 7f64617fa640 1 -- 172.21.15.113:0/3692322089 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c014820 con 0x7f6464144910 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.468+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6464134bf0 con 0x7f6464144910 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.469+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64640691a0 con 0x7f6464144910 2024-04-25T00:04:38.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.469+0000 7f64617fa640 1 -- 172.21.15.113:0/3692322089 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f645c0149c0 con 0x7f6464144910 2024-04-25T00:04:38.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.470+0000 7f64617fa640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f645006be30 0x7f645006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.470+0000 7f64617fa640 1 -- 172.21.15.113:0/3692322089 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f645c08b400 con 0x7f6464144910 2024-04-25T00:04:38.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.470+0000 7f6468824640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f645006be30 0x7f645006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.470+0000 7f6468824640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f645006be30 0x7f645006e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6454001780 tx=0x7f6454009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.474+0000 7f64617fa640 1 -- 172.21.15.113:0/3692322089 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f645c0597f0 con 0x7f6464144910 2024-04-25T00:04:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.681+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64640afe90 con 0x7f6464144910 2024-04-25T00:04:38.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.681+0000 7f64617fa640 1 -- 172.21.15.113:0/3692322089 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5195 (secure 0 0 0) 0x7f645c05daf0 con 0x7f6464144910 2024-04-25T00:04:38.682 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:06.288869+0000","modified":"2024-04-25T00:04:08.290392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2796979045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2796979045},{"type":"v1","addr":"172.21.15.113:6837","nonce":2796979045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7395,"qdb_cluster":[7395]},"id":6}]} 2024-04-25T00:04:38.682 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.683+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f645006be30 msgr2=0x7f645006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.683+0000 7f646aa88640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f645006be30 0x7f645006e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6454001780 tx=0x7f6454009040 comp rx=0 tx=0).stop 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.683+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6464144910 msgr2=0x7f64640c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.683+0000 7f646aa88640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6464144910 0x7f64640c5d40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f645c007cf0 tx=0x7f645c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.683+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 shutdown_connections 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.684+0000 7f646aa88640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f645006be30 0x7f645006e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.684+0000 7f646aa88640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64640b3fd0 0x7f64641f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.684+0000 7f646aa88640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6464144910 0x7f64640c5d40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.684+0000 7f646aa88640 1 --2- 172.21.15.113:0/3692322089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f646406e510 0x7f64641f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.684+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 >> 172.21.15.113:0/3692322089 conn(0x7f64640b1a50 msgr2=0x7f6464071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:38.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.684+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 shutdown_connections 2024-04-25T00:04:38.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.684+0000 7f646aa88640 1 -- 172.21.15.113:0/3692322089 wait complete. 2024-04-25T00:04:38.695 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.791+0000 7f875cb89640 1 Processor -- start 2024-04-25T00:04:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.791+0000 7f875cb89640 1 -- start start 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f875cb89640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f87580b3fd0 0x7f875806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f875cb89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875806e510 0x7f87580cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f875cb89640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8758144cf0 con 0x7f8758144910 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f875cb89640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f87580b2f30 con 0x7f87580b3fd0 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f875cb89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87580b3210 con 0x7f875806e510 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8756d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875806e510 0x7f87580cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8755d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f87580b3fd0 0x7f875806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8756d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875806e510 0x7f87580cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:36160/0 (socket says 172.21.15.113:36160) 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8756d9d640 1 -- 172.21.15.113:0/4160313771 learned_addr learned my addr 172.21.15.113:0/4160313771 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f875659c640 1 --1- 172.21.15.113:0/4160313771 >> v1:172.21.15.178:6789/0 conn(0x7f8758144910 0x7f87580b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:40956/0 (socket says 172.21.15.113:40956) 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8756d9d640 1 -- 172.21.15.113:0/4160313771 >> v2:172.21.15.178:3300/0 conn(0x7f87580b3fd0 msgr2=0x7f875806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8756d9d640 1 --2- 172.21.15.113:0/4160313771 >> v2:172.21.15.178:3300/0 conn(0x7f87580b3fd0 0x7f875806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8756d9d640 1 -- 172.21.15.113:0/4160313771 >> v1:172.21.15.178:6789/0 conn(0x7f8758144910 legacy=0x7f87580b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.792+0000 7f8756d9d640 1 -- 172.21.15.113:0/4160313771 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87580b34f0 con 0x7f875806e510 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.793+0000 7f8756d9d640 1 --2- 172.21.15.113:0/4160313771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875806e510 0x7f87580cd500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8750009e80 tx=0x7f875002a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ab8a61b56733c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.793+0000 7f875559a640 1 -- 172.21.15.113:0/4160313771 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750036070 con 0x7f875806e510 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.793+0000 7f875559a640 1 -- 172.21.15.113:0/4160313771 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750031600 con 0x7f875806e510 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.793+0000 7f875559a640 1 -- 172.21.15.113:0/4160313771 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500319d0 con 0x7f875806e510 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 -- 172.21.15.113:0/4160313771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875806e510 msgr2=0x7f87580cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 --2- 172.21.15.113:0/4160313771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875806e510 0x7f87580cd500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8750009e80 tx=0x7f875002a750 comp rx=0 tx=0).stop 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 -- 172.21.15.113:0/4160313771 shutdown_connections 2024-04-25T00:04:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 --2- 172.21.15.113:0/4160313771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875806e510 0x7f87580cd500 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 --2- 172.21.15.113:0/4160313771 >> v2:172.21.15.178:3300/0 conn(0x7f87580b3fd0 0x7f875806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 -- 172.21.15.113:0/4160313771 >> 172.21.15.113:0/4160313771 conn(0x7f87580b1a50 msgr2=0x7f87580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:38.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 -- 172.21.15.113:0/4160313771 shutdown_connections 2024-04-25T00:04:38.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.794+0000 7f875cb89640 1 -- 172.21.15.113:0/4160313771 wait complete. 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.795+0000 7f875cb89640 1 Processor -- start 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.795+0000 7f875cb89640 1 -- start start 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f875cb89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f875806e510 0x7f87581f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f875cb89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87580b3fd0 0x7f87580a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f875cb89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8758144910 0x7f8758071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f875cb89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87580b4da0 con 0x7f87580b3fd0 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f875cb89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87580b4440 con 0x7f875806e510 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f875cb89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87580b4f20 con 0x7f8758144910 2024-04-25T00:04:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87580b3fd0 0x7f87580a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87580b3fd0 0x7f87580a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49596/0 (socket says 172.21.15.113:49596) 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 -- 172.21.15.113:0/171050147 learned_addr learned my addr 172.21.15.113:0/171050147 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8756d9d640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8758144910 0x7f8758071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f875659c640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f875806e510 0x7f87581f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 -- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8758144910 msgr2=0x7f8758071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8758144910 0x7f8758071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 -- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f875806e510 msgr2=0x7f87581f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f875806e510 0x7f87581f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.796+0000 7f8755d9b640 1 -- 172.21.15.113:0/171050147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87581f3850 con 0x7f87580b3fd0 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.797+0000 7f8755d9b640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87580b3fd0 0x7f87580a9db0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e9b0 tx=0x7f874c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.797+0000 7f8746ffd640 1 -- 172.21.15.113:0/171050147 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c017070 con 0x7f87580b3fd0 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.797+0000 7f8746ffd640 1 -- 172.21.15.113:0/171050147 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c00c9a0 con 0x7f87580b3fd0 2024-04-25T00:04:38.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.797+0000 7f8746ffd640 1 -- 172.21.15.113:0/171050147 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c00cd90 con 0x7f87580b3fd0 2024-04-25T00:04:38.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.797+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87581f3b30 con 0x7f87580b3fd0 2024-04-25T00:04:38.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.797+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581f43d0 con 0x7f87580b3fd0 2024-04-25T00:04:38.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.798+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87581f3f20 con 0x7f87580b3fd0 2024-04-25T00:04:38.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.798+0000 7f8746ffd640 1 -- 172.21.15.113:0/171050147 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f874c01a070 con 0x7f87580b3fd0 2024-04-25T00:04:38.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.799+0000 7f8746ffd640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873806be30 0x7f873806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:38.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.799+0000 7f8746ffd640 1 -- 172.21.15.113:0/171050147 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f874c08b260 con 0x7f87580b3fd0 2024-04-25T00:04:38.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.799+0000 7f875659c640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873806be30 0x7f873806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:38.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.799+0000 7f875659c640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873806be30 0x7f873806e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8748000f60 tx=0x7f8748009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:38.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:38.803+0000 7f8746ffd640 1 -- 172.21.15.113:0/171050147 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f874c059c10 con 0x7f87580b3fd0 2024-04-25T00:04:39.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.015+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8758072270 con 0x7f87580b3fd0 2024-04-25T00:04:39.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.016+0000 7f8746ffd640 1 -- 172.21.15.113:0/171050147 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5195 (secure 0 0 0) 0x7f874c059510 con 0x7f87580b3fd0 2024-04-25T00:04:39.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:06.288869+0000","modified":"2024-04-25T00:04:08.290392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2796979045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2796979045},{"type":"v1","addr":"172.21.15.113:6837","nonce":2796979045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7395,"qdb_cluster":[7395]},"id":6}]} 2024-04-25T00:04:39.017 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-04-25T00:04:39.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873806be30 msgr2=0x7f873806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873806be30 0x7f873806e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8748000f60 tx=0x7f8748009040 comp rx=0 tx=0).stop 2024-04-25T00:04:39.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87580b3fd0 msgr2=0x7f87580a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87580b3fd0 0x7f87580a9db0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e9b0 tx=0x7f874c00ee70 comp rx=0 tx=0).stop 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 shutdown_connections 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873806be30 0x7f873806e2f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8758144910 0x7f8758071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87580b3fd0 0x7f87580a9db0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 --2- 172.21.15.113:0/171050147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f875806e510 0x7f87581f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 >> 172.21.15.113:0/171050147 conn(0x7f87580b1a50 msgr2=0x7f87580cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 shutdown_connections 2024-04-25T00:04:39.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.018+0000 7f875cb89640 1 -- 172.21.15.113:0/171050147 wait complete. 2024-04-25T00:04:39.029 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:39.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.126+0000 7fec9e041640 1 Processor -- start 2024-04-25T00:04:39.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.126+0000 7fec9e041640 1 -- start start 2024-04-25T00:04:39.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec9e041640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec9815e240 0x7fec981627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec9e041640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec980b5620 0x7fec9806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec9e041640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fec9806e510 0x7fec98070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec9e041640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fec98072d90 con 0x7fec9815e240 2024-04-25T00:04:39.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec9e041640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fec9806e930 con 0x7fec9806e510 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec9e041640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec98162d20 con 0x7fec980b5620 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec9815e240 0x7fec981627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec9815e240 0x7fec981627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49610/0 (socket says 172.21.15.113:49610) 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 -- 172.21.15.113:0/54427567 learned_addr learned my addr 172.21.15.113:0/54427567 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec97fff640 1 --2- 172.21.15.113:0/54427567 >> v2:172.21.15.178:3300/0 conn(0x7fec9806e510 0x7fec98070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec96ffd640 1 --2- 172.21.15.113:0/54427567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec980b5620 0x7fec9806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 -- 172.21.15.113:0/54427567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec980b5620 msgr2=0x7fec9806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 --2- 172.21.15.113:0/54427567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec980b5620 0x7fec9806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 -- 172.21.15.113:0/54427567 >> v2:172.21.15.178:3300/0 conn(0x7fec9806e510 msgr2=0x7fec98070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 --2- 172.21.15.113:0/54427567 >> v2:172.21.15.178:3300/0 conn(0x7fec9806e510 0x7fec98070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.127+0000 7fec977fe640 1 -- 172.21.15.113:0/54427567 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec980b3c20 con 0x7fec9815e240 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.128+0000 7fec977fe640 1 --2- 172.21.15.113:0/54427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9815e240 0x7fec981627e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fec9806f980 tx=0x7fec8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105d84d896038285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.128+0000 7fec967fc640 1 -- 172.21.15.113:0/54427567 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88037070 con 0x7fec9815e240 2024-04-25T00:04:39.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.128+0000 7fec967fc640 1 -- 172.21.15.113:0/54427567 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88032440 con 0x7fec9815e240 2024-04-25T00:04:39.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.128+0000 7fec967fc640 1 -- 172.21.15.113:0/54427567 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88032810 con 0x7fec9815e240 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 -- 172.21.15.113:0/54427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9815e240 msgr2=0x7fec981627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 --2- 172.21.15.113:0/54427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9815e240 0x7fec981627e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fec9806f980 tx=0x7fec8802a210 comp rx=0 tx=0).stop 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 -- 172.21.15.113:0/54427567 shutdown_connections 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 --2- 172.21.15.113:0/54427567 >> v2:172.21.15.178:3300/0 conn(0x7fec9806e510 0x7fec98070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 --2- 172.21.15.113:0/54427567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec980b5620 0x7fec9806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 --2- 172.21.15.113:0/54427567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9815e240 0x7fec981627e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 -- 172.21.15.113:0/54427567 >> 172.21.15.113:0/54427567 conn(0x7fec980b1a50 msgr2=0x7fec980b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:39.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 -- 172.21.15.113:0/54427567 shutdown_connections 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.129+0000 7fec9e041640 1 -- 172.21.15.113:0/54427567 wait complete. 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.130+0000 7fec9e041640 1 Processor -- start 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.130+0000 7fec9e041640 1 -- start start 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec9e041640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9806e510 0x7fec9813e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec9e041640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec980b5620 0x7fec981601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec9e041640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9815e240 0x7fec9813fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec9e041640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fec980724f0 con 0x7fec980b5620 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec9e041640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec98072210 con 0x7fec9806e510 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec9e041640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec980727d0 con 0x7fec9815e240 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec980b5620 0x7fec981601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec980b5620 0x7fec981601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49624/0 (socket says 172.21.15.113:49624) 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 -- 172.21.15.113:0/1167632898 learned_addr learned my addr 172.21.15.113:0/1167632898 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec977fe640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9806e510 0x7fec9813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec97fff640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9815e240 0x7fec9813fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 -- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9815e240 msgr2=0x7fec9813fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9815e240 0x7fec9813fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 -- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9806e510 msgr2=0x7fec9813e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9806e510 0x7fec9813e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 -- 172.21.15.113:0/1167632898 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9813ed40 con 0x7fec980b5620 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec977fe640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9806e510 0x7fec9813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec96ffd640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec980b5620 0x7fec981601d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d7d0 tx=0x7fec8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec7ffff640 1 -- 172.21.15.113:0/1167632898 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c004020 con 0x7fec980b5620 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec7ffff640 1 -- 172.21.15.113:0/1167632898 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c0041e0 con 0x7fec980b5620 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9813f020 con 0x7fec980b5620 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.131+0000 7fec7ffff640 1 -- 172.21.15.113:0/1167632898 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0043d0 con 0x7fec980b5620 2024-04-25T00:04:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.132+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9813f8c0 con 0x7fec980b5620 2024-04-25T00:04:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.132+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec980691a0 con 0x7fec980b5620 2024-04-25T00:04:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.133+0000 7fec7ffff640 1 -- 172.21.15.113:0/1167632898 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec8c01b020 con 0x7fec980b5620 2024-04-25T00:04:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.133+0000 7fec7ffff640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec8406be30 0x7fec8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.133+0000 7fec7ffff640 1 -- 172.21.15.113:0/1167632898 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fec8c08ac10 con 0x7fec980b5620 2024-04-25T00:04:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.133+0000 7fec977fe640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec8406be30 0x7fec8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.134+0000 7fec977fe640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec8406be30 0x7fec8406e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec88034000 tx=0x7fec8802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.137+0000 7fec7ffff640 1 -- 172.21.15.113:0/1167632898 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec8c059650 con 0x7fec980b5620 2024-04-25T00:04:39.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.313+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec98161b40 con 0x7fec980b5620 2024-04-25T00:04:39.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.314+0000 7fec7ffff640 1 -- 172.21.15.113:0/1167632898 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16708 (secure 0 0 0) 0x7fec8c058f50 con 0x7fec980b5620 2024-04-25T00:04:39.314 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:07.938101+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:04.509800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:05.258783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:39.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec8406be30 msgr2=0x7fec8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec8406be30 0x7fec8406e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec88034000 tx=0x7fec8802e000 comp rx=0 tx=0).stop 2024-04-25T00:04:39.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec980b5620 msgr2=0x7fec981601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec980b5620 0x7fec981601d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d7d0 tx=0x7fec8c00dc90 comp rx=0 tx=0).stop 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 shutdown_connections 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec8406be30 0x7fec8406e2f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9815e240 0x7fec9813fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec980b5620 0x7fec981601d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 --2- 172.21.15.113:0/1167632898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9806e510 0x7fec9813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 >> 172.21.15.113:0/1167632898 conn(0x7fec980b1a50 msgr2=0x7fec980afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 shutdown_connections 2024-04-25T00:04:39.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.316+0000 7fec9e041640 1 -- 172.21.15.113:0/1167632898 wait complete. 2024-04-25T00:04:39.328 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:04:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.424+0000 7ff77ad61640 1 Processor -- start 2024-04-25T00:04:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.424+0000 7ff77ad61640 1 -- start start 2024-04-25T00:04:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff77ad61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff77415e2d0 0x7ff77415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff77ad61640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff7740b4a40 0x7ff7740ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff77ad61640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff77415f4f0 con 0x7ff7740b3080 2024-04-25T00:04:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff77ad61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff7740cde00 con 0x7ff77415e2d0 2024-04-25T00:04:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff77ad61640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff77415f7d0 con 0x7ff7740b4a40 2024-04-25T00:04:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff778afd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff77415e2d0 0x7ff77415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff778afd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff77415e2d0 0x7ff77415ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49636/0 (socket says 172.21.15.113:49636) 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff778afd640 1 -- 172.21.15.113:0/3410128485 learned_addr learned my addr 172.21.15.113:0/3410128485 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff778afd640 1 -- 172.21.15.113:0/3410128485 >> v2:172.21.15.178:3300/0 conn(0x7ff7740b4a40 msgr2=0x7ff7740ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff7792fe640 1 --2- 172.21.15.113:0/3410128485 >> v2:172.21.15.178:3300/0 conn(0x7ff7740b4a40 0x7ff7740ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff778afd640 1 --2- 172.21.15.113:0/3410128485 >> v2:172.21.15.178:3300/0 conn(0x7ff7740b4a40 0x7ff7740ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff778afd640 1 -- 172.21.15.113:0/3410128485 >> v1:172.21.15.178:6789/0 conn(0x7ff7740b3080 legacy=0x7ff7740b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff778afd640 1 -- 172.21.15.113:0/3410128485 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77415fab0 con 0x7ff77415e2d0 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.425+0000 7ff7792fe640 1 --2- 172.21.15.113:0/3410128485 >> v2:172.21.15.178:3300/0 conn(0x7ff7740b4a40 0x7ff7740ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.426+0000 7ff778afd640 1 --2- 172.21.15.113:0/3410128485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77415e2d0 0x7ff77415ebf0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff7740d0f20 tx=0x7ff76402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7d07c1a31ce0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.426+0000 7ff7737fe640 1 -- 172.21.15.113:0/3410128485 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764037070 con 0x7ff77415e2d0 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.426+0000 7ff7737fe640 1 -- 172.21.15.113:0/3410128485 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff764032440 con 0x7ff77415e2d0 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.426+0000 7ff7737fe640 1 -- 172.21.15.113:0/3410128485 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764032810 con 0x7ff77415e2d0 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 -- 172.21.15.113:0/3410128485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77415e2d0 msgr2=0x7ff77415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 --2- 172.21.15.113:0/3410128485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77415e2d0 0x7ff77415ebf0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff7740d0f20 tx=0x7ff76402a230 comp rx=0 tx=0).stop 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 -- 172.21.15.113:0/3410128485 shutdown_connections 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 --2- 172.21.15.113:0/3410128485 >> v2:172.21.15.178:3300/0 conn(0x7ff7740b4a40 0x7ff7740ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 --2- 172.21.15.113:0/3410128485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77415e2d0 0x7ff77415ebf0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 -- 172.21.15.113:0/3410128485 >> 172.21.15.113:0/3410128485 conn(0x7ff77406b0f0 msgr2=0x7ff77406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 -- 172.21.15.113:0/3410128485 shutdown_connections 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.427+0000 7ff77ad61640 1 -- 172.21.15.113:0/3410128485 wait complete. 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.428+0000 7ff77ad61640 1 Processor -- start 2024-04-25T00:04:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.428+0000 7ff77ad61640 1 -- start start 2024-04-25T00:04:39.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff77ad61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7740b3080 0x7ff77415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff77ad61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740b4a40 0x7ff7741558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff77ad61640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77415e2d0 0x7ff774145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff77ad61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff77415f2b0 con 0x7ff7740b3080 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff77ad61640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff77415f130 con 0x7ff77415e2d0 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff77ad61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff77415e6b0 con 0x7ff7740b4a40 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7740b3080 0x7ff77415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7740b3080 0x7ff77415a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49640/0 (socket says 172.21.15.113:49640) 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 -- 172.21.15.113:0/694124676 learned_addr learned my addr 172.21.15.113:0/694124676 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 -- 172.21.15.113:0/694124676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740b4a40 msgr2=0x7ff7741558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740b4a40 0x7ff7741558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 -- 172.21.15.113:0/694124676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77415e2d0 msgr2=0x7ff774145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77415e2d0 0x7ff774145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 -- 172.21.15.113:0/694124676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77415c910 con 0x7ff7740b3080 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff778afd640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7740b3080 0x7ff77415a100 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff764000c00 tx=0x7ff7640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.429+0000 7ff7717fa640 1 -- 172.21.15.113:0/694124676 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76403f070 con 0x7ff7740b3080 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.430+0000 7ff7717fa640 1 -- 172.21.15.113:0/694124676 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff764030aa0 con 0x7ff7740b3080 2024-04-25T00:04:39.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.430+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77415cbf0 con 0x7ff7740b3080 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.430+0000 7ff7717fa640 1 -- 172.21.15.113:0/694124676 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764037040 con 0x7ff7740b3080 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.430+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77415ba20 con 0x7ff7740b3080 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.431+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7740691a0 con 0x7ff7740b3080 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.431+0000 7ff7717fa640 1 -- 172.21.15.113:0/694124676 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff76404a060 con 0x7ff7740b3080 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.431+0000 7ff7717fa640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff76006be30 0x7ff76006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.432+0000 7ff7717fa640 1 -- 172.21.15.113:0/694124676 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7ff7640acd90 con 0x7ff7740b3080 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.432+0000 7ff773fff640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff76006be30 0x7ff76006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.432+0000 7ff773fff640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff76006be30 0x7ff76006e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff774152240 tx=0x7ff7680079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.436+0000 7ff7717fa640 1 -- 172.21.15.113:0/694124676 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff76407b7d0 con 0x7ff7740b3080 2024-04-25T00:04:39.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.644+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff774072050 con 0x7ff7740b3080 2024-04-25T00:04:39.645 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:04:39.645+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:04:39.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.646+0000 7ff7717fa640 1 -- 172.21.15.113:0/694124676 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7ff76407b0d0 con 0x7ff7740b3080 2024-04-25T00:04:39.646 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff76006be30 msgr2=0x7ff76006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff76006be30 0x7ff76006e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff774152240 tx=0x7ff7680079b0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7740b3080 msgr2=0x7ff77415a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7740b3080 0x7ff77415a100 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff764000c00 tx=0x7ff7640046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 shutdown_connections 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff76006be30 0x7ff76006e2f0 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff774152240 tx=0x7ff7680079b0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77415e2d0 0x7ff774145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740b4a40 0x7ff7741558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 --2- 172.21.15.113:0/694124676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7740b3080 0x7ff77415a100 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 >> 172.21.15.113:0/694124676 conn(0x7ff77406b0f0 msgr2=0x7ff77406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 shutdown_connections 2024-04-25T00:04:39.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.650+0000 7ff77ad61640 1 -- 172.21.15.113:0/694124676 wait complete. 2024-04-25T00:04:39.662 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:04:39.686 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:04:39.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.762+0000 7fb9cb370640 1 Processor -- start 2024-04-25T00:04:39.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.762+0000 7fb9cb370640 1 -- start start 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9cb370640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c413f050 0x7fb9c40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9cb370640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9c40d04f0 0x7fb9c415ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9cb370640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9c41604d0 0x7fb9c406cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9cb370640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb9c415c6a0 con 0x7fb9c40d04f0 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9cb370640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb9c40d0970 con 0x7fb9c41604d0 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9cb370640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c415d6b0 con 0x7fb9c413f050 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9c40d04f0 0x7fb9c415ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9c40d04f0 0x7fb9c415ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49670/0 (socket says 172.21.15.113:49670) 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 -- 172.21.15.113:0/4240378136 learned_addr learned my addr 172.21.15.113:0/4240378136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:39.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 -- 172.21.15.113:0/4240378136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c413f050 msgr2=0x7fb9c40cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:39.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 --2- 172.21.15.113:0/4240378136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c413f050 0x7fb9c40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 -- 172.21.15.113:0/4240378136 >> v2:172.21.15.178:3300/0 conn(0x7fb9c41604d0 msgr2=0x7fb9c406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:39.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 --2- 172.21.15.113:0/4240378136 >> v2:172.21.15.178:3300/0 conn(0x7fb9c41604d0 0x7fb9c406cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.763+0000 7fb9c890b640 1 -- 172.21.15.113:0/4240378136 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c415c8f0 con 0x7fb9c40d04f0 2024-04-25T00:04:39.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.764+0000 7fb9c890b640 1 --2- 172.21.15.113:0/4240378136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c40d04f0 0x7fb9c415ff90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fb9c406f460 tx=0x7fb9b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7143aa4c5db61c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.764+0000 7fb9bbfff640 1 -- 172.21.15.113:0/4240378136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4037070 con 0x7fb9c40d04f0 2024-04-25T00:04:39.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.764+0000 7fb9bbfff640 1 -- 172.21.15.113:0/4240378136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b4032440 con 0x7fb9c40d04f0 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.764+0000 7fb9bbfff640 1 -- 172.21.15.113:0/4240378136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4032810 con 0x7fb9c40d04f0 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 -- 172.21.15.113:0/4240378136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c40d04f0 msgr2=0x7fb9c415ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 --2- 172.21.15.113:0/4240378136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c40d04f0 0x7fb9c415ff90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fb9c406f460 tx=0x7fb9b402a140 comp rx=0 tx=0).stop 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 -- 172.21.15.113:0/4240378136 shutdown_connections 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 --2- 172.21.15.113:0/4240378136 >> v2:172.21.15.178:3300/0 conn(0x7fb9c41604d0 0x7fb9c406cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 --2- 172.21.15.113:0/4240378136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c40d04f0 0x7fb9c415ff90 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 --2- 172.21.15.113:0/4240378136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c413f050 0x7fb9c40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 -- 172.21.15.113:0/4240378136 >> 172.21.15.113:0/4240378136 conn(0x7fb9c406b0f0 msgr2=0x7fb9c406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 -- 172.21.15.113:0/4240378136 shutdown_connections 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.765+0000 7fb9cb370640 1 -- 172.21.15.113:0/4240378136 wait complete. 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.766+0000 7fb9cb370640 1 Processor -- start 2024-04-25T00:04:39.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.766+0000 7fb9cb370640 1 -- start start 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9cb370640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c40d04f0 0x7fb9c4154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9cb370640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c413f050 0x7fb9c413aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9cb370640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c41604d0 0x7fb9c413b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9cb370640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c415d190 con 0x7fb9c413f050 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9cb370640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c415ceb0 con 0x7fb9c40d04f0 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9cb370640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c413f430 con 0x7fb9c41604d0 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c413f050 0x7fb9c413aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c413f050 0x7fb9c413aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49678/0 (socket says 172.21.15.113:49678) 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 -- 172.21.15.113:0/1318105193 learned_addr learned my addr 172.21.15.113:0/1318105193 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 -- 172.21.15.113:0/1318105193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c41604d0 msgr2=0x7fb9c413b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c41604d0 0x7fb9c413b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 -- 172.21.15.113:0/1318105193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c40d04f0 msgr2=0x7fb9c4154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c40d04f0 0x7fb9c4154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 -- 172.21.15.113:0/1318105193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c4159fe0 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9c890b640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c413f050 0x7fb9c413aec0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4000c00 tx=0x7fb9b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9b9ffb640 1 -- 172.21.15.113:0/1318105193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4041070 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9b9ffb640 1 -- 172.21.15.113:0/1318105193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b403c870 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c415a2c0 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.767+0000 7fb9b9ffb640 1 -- 172.21.15.113:0/1318105193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4037040 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.768+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c41588d0 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.769+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c40691a0 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.769+0000 7fb9b9ffb640 1 -- 172.21.15.113:0/1318105193 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9b4044070 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.769+0000 7fb9b9ffb640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9a806be30 0x7fb9a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.770+0000 7fb9b9ffb640 1 -- 172.21.15.113:0/1318105193 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fb9b40abc20 con 0x7fb9c413f050 2024-04-25T00:04:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.770+0000 7fb9c910c640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9a806be30 0x7fb9a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:39.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.770+0000 7fb9c910c640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9a806be30 0x7fb9a806e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000ab90 tx=0x7fb9c0009250 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:39.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.774+0000 7fb9b9ffb640 1 -- 172.21.15.113:0/1318105193 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9b407a720 con 0x7fb9c413f050 2024-04-25T00:04:39.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:39.981+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb9c415c3e0 con 0x7fb9c413f050 2024-04-25T00:04:40.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.648+0000 7fb9b9ffb640 1 -- 172.21.15.113:0/1318105193 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7fb9b407a020 con 0x7fb9c413f050 2024-04-25T00:04:40.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9a806be30 msgr2=0x7fb9a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9a806be30 0x7fb9a806e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000ab90 tx=0x7fb9c0009250 comp rx=0 tx=0).stop 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c413f050 msgr2=0x7fb9c413aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c413f050 0x7fb9c413aec0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4000c00 tx=0x7fb9b40046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 shutdown_connections 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9a806be30 0x7fb9a806e2f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9c41604d0 0x7fb9c413b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c413f050 0x7fb9c413aec0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 --2- 172.21.15.113:0/1318105193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c40d04f0 0x7fb9c4154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 >> 172.21.15.113:0/1318105193 conn(0x7fb9c406b0f0 msgr2=0x7fb9c415d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 shutdown_connections 2024-04-25T00:04:40.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.650+0000 7fb9cb370640 1 -- 172.21.15.113:0/1318105193 wait complete. 2024-04-25T00:04:40.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:04:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.805+0000 7f18975ce640 1 Processor -- start 2024-04-25T00:04:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.805+0000 7f18975ce640 1 -- start start 2024-04-25T00:04:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f18975ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18900b47a0 0x7f189015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f18975ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f18975ce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189015bcf0 con 0x7f18900cc0b0 2024-04-25T00:04:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f18975ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f18900b2f40 con 0x7f18900b47a0 2024-04-25T00:04:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f18975ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18900b31e0 con 0x7f1890154f40 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f189536a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18900b47a0 0x7f189015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f189536a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18900b47a0 0x7f189015b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49690/0 (socket says 172.21.15.113:49690) 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f189536a640 1 -- 172.21.15.113:0/3985334418 learned_addr learned my addr 172.21.15.113:0/3985334418 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f189536a640 1 -- 172.21.15.113:0/3985334418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 msgr2=0x7f189015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f1895b6b640 1 --2- 172.21.15.113:0/3985334418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f189536a640 1 --2- 172.21.15.113:0/3985334418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f189536a640 1 -- 172.21.15.113:0/3985334418 >> v1:172.21.15.178:6789/0 conn(0x7f18900cc0b0 legacy=0x7f1890154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f189536a640 1 -- 172.21.15.113:0/3985334418 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18900b34c0 con 0x7f18900b47a0 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.806+0000 7f1895b6b640 1 --2- 172.21.15.113:0/3985334418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.807+0000 7f189536a640 1 --2- 172.21.15.113:0/3985334418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900b47a0 0x7f189015b7b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50da2d6b84507d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.807+0000 7f1887fff640 1 -- 172.21.15.113:0/3985334418 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c037070 con 0x7f18900b47a0 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.807+0000 7f1887fff640 1 -- 172.21.15.113:0/3985334418 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188c032440 con 0x7f18900b47a0 2024-04-25T00:04:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.807+0000 7f1887fff640 1 -- 172.21.15.113:0/3985334418 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c032810 con 0x7f18900b47a0 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 -- 172.21.15.113:0/3985334418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900b47a0 msgr2=0x7f189015b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 --2- 172.21.15.113:0/3985334418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900b47a0 0x7f189015b7b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c02a210 comp rx=0 tx=0).stop 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 -- 172.21.15.113:0/3985334418 shutdown_connections 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 --2- 172.21.15.113:0/3985334418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 --2- 172.21.15.113:0/3985334418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900b47a0 0x7f189015b7b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 -- 172.21.15.113:0/3985334418 >> 172.21.15.113:0/3985334418 conn(0x7f18900b1b30 msgr2=0x7f18900b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 -- 172.21.15.113:0/3985334418 shutdown_connections 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.808+0000 7f18975ce640 1 -- 172.21.15.113:0/3985334418 wait complete. 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.809+0000 7f18975ce640 1 Processor -- start 2024-04-25T00:04:40.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.809+0000 7f18975ce640 1 -- start start 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f18975ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18900b47a0 0x7f189015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f18975ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900cc0b0 0x7f189015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f18975ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f18975ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18900b4b80 con 0x7f18900cc0b0 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f18975ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18900b3a80 con 0x7f18900b47a0 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f18975ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18900b4d00 con 0x7f1890154f40 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900cc0b0 0x7f189015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900cc0b0 0x7f189015e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49704/0 (socket says 172.21.15.113:49704) 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 -- 172.21.15.113:0/1361000838 learned_addr learned my addr 172.21.15.113:0/1361000838 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f189536a640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18900b47a0 0x7f189015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1895b6b640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 -- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 msgr2=0x7f189015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:40.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 -- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18900b47a0 msgr2=0x7f189015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18900b47a0 0x7f189015a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 -- 172.21.15.113:0/1361000838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18900ce6a0 con 0x7f18900cc0b0 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1894b69640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900cc0b0 0x7f189015e520 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f188000c430 tx=0x7f188000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1885ffb640 1 -- 172.21.15.113:0/1361000838 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880016070 con 0x7f18900cc0b0 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1885ffb640 1 -- 172.21.15.113:0/1361000838 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18800108a0 con 0x7f18900cc0b0 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f1885ffb640 1 -- 172.21.15.113:0/1361000838 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880010cd0 con 0x7f18900cc0b0 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.810+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189013d680 con 0x7f18900cc0b0 2024-04-25T00:04:40.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.811+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189013dc40 con 0x7f18900cc0b0 2024-04-25T00:04:40.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.812+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18900691a0 con 0x7f18900cc0b0 2024-04-25T00:04:40.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.812+0000 7f1885ffb640 1 -- 172.21.15.113:0/1361000838 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1880019070 con 0x7f18900cc0b0 2024-04-25T00:04:40.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.812+0000 7f1885ffb640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f187406be60 0x7f187406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:40.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.812+0000 7f1885ffb640 1 -- 172.21.15.113:0/1361000838 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f188008ca60 con 0x7f18900cc0b0 2024-04-25T00:04:40.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.812+0000 7f189536a640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f187406be60 0x7f187406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:40.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.813+0000 7f189536a640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f187406be60 0x7f187406e320 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f188c034000 tx=0x7f188c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:40.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.817+0000 7f1885ffb640 1 -- 172.21.15.113:0/1361000838 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f188005b3b0 con 0x7f18900cc0b0 2024-04-25T00:04:40.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:40.991+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f18900b26b0 con 0x7f18900cc0b0 2024-04-25T00:04:41.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.649+0000 7f1885ffb640 1 -- 172.21.15.113:0/1361000838 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f188005acb0 con 0x7f18900cc0b0 2024-04-25T00:04:41.649 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:04:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.651+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f187406be60 msgr2=0x7f187406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f187406be60 0x7f187406e320 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f188c034000 tx=0x7f188c02e000 comp rx=0 tx=0).stop 2024-04-25T00:04:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900cc0b0 msgr2=0x7f189015e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900cc0b0 0x7f189015e520 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f188000c430 tx=0x7f188000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 shutdown_connections 2024-04-25T00:04:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f187406be60 0x7f187406e320 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1890154f40 0x7f189015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18900cc0b0 0x7f189015e520 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 --2- 172.21.15.113:0/1361000838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18900b47a0 0x7f189015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 >> 172.21.15.113:0/1361000838 conn(0x7f18900b1b30 msgr2=0x7f1890151400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:41.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 shutdown_connections 2024-04-25T00:04:41.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.652+0000 7f18975ce640 1 -- 172.21.15.113:0/1361000838 wait complete. 2024-04-25T00:04:41.669 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:04:41.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.769+0000 7f75c16f5640 1 Processor -- start 2024-04-25T00:04:41.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.769+0000 7f75c16f5640 1 -- start start 2024-04-25T00:04:41.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.770+0000 7f75c16f5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75bc0b9bb0 0x7f75bc0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.770+0000 7f75c16f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75bc1626f0 con 0x7f75bc15e510 2024-04-25T00:04:41.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.770+0000 7f75c16f5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75bc1629d0 con 0x7f75bc0b61e0 2024-04-25T00:04:41.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.770+0000 7f75c16f5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f75bc162cb0 con 0x7f75bc0b9bb0 2024-04-25T00:04:41.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.770+0000 7f75baffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f75bc15e510 0x7f75bc161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37166/0 (socket says 172.21.15.113:37166) 2024-04-25T00:04:41.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.770+0000 7f75baffd640 1 -- 172.21.15.113:0/4108637300 learned_addr learned my addr 172.21.15.113:0/4108637300 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75ba7fc640 1 --2- 172.21.15.113:0/4108637300 >> v2:172.21.15.178:3300/0 conn(0x7f75bc0b9bb0 0x7f75bc0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708663784 0 0) 0x7f75bc1626f0 con 0x7f75bc15e510 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75a4003520 con 0x7f75bc15e510 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3482465577 0 0) 0x7f75a4003520 con 0x7f75bc15e510 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 >> v2:172.21.15.178:3300/0 conn(0x7f75bc0b9bb0 msgr2=0x7f75bc0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 --2- 172.21.15.113:0/4108637300 >> v2:172.21.15.178:3300/0 conn(0x7f75bc0b9bb0 0x7f75bc0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 >> v1:172.21.15.178:6789/0 conn(0x7f75bc0b61e0 legacy=0x7f75bc0d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc0d2660 con 0x7f75bc15e510 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f75ac0019f0 con 0x7f75bc15e510 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 >> v1:172.21.15.113:6789/0 conn(0x7f75bc15e510 legacy=0x7f75bc161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75a4004800 0x7f75a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc162cb0 con 0x7f75a40039b0 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75a400c160 con 0x7f75a40051c0 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75a400c360 con 0x7f75a4004800 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75baffd640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75bb7fe640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75ba7fc640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75a4004800 0x7f75a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75baffd640 1 -- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75a4004800 msgr2=0x7f75a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75baffd640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75a4004800 0x7f75a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75baffd640 1 -- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75a40051c0 msgr2=0x7f75a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75baffd640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75baffd640 1 -- 172.21.15.113:0/4108637300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a4003520 con 0x7f75a40039b0 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75bb7fe640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75a40051c0 0x7f75a400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.771+0000 7f75bb7fe640 1 -- 172.21.15.113:0/4108637300 reap_dead start 2024-04-25T00:04:41.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.772+0000 7f75baffd640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75a4003dd0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00a850 tx=0x7f75a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e10ddcd128e2b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.772+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00a4a0 con 0x7f75a40039b0 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.772+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ac02f070 con 0x7f75a40039b0 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.772+0000 7f75b9ffb640 1 -- 172.21.15.113:0/4108637300 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac02bd20 con 0x7f75a40039b0 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.772+0000 7f75c16f5640 1 -- 172.21.15.113:0/4108637300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 msgr2=0x7f75a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.772+0000 7f75c16f5640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75a4003dd0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00a850 tx=0x7f75a4005830 comp rx=0 tx=0).stop 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.773+0000 7f75c16f5640 1 -- 172.21.15.113:0/4108637300 shutdown_connections 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.773+0000 7f75c16f5640 1 --2- 172.21.15.113:0/4108637300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75a4003dd0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.773+0000 7f75c16f5640 1 -- 172.21.15.113:0/4108637300 >> 172.21.15.113:0/4108637300 conn(0x7f75bc06b720 msgr2=0x7f75bc0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.773+0000 7f75c16f5640 1 -- 172.21.15.113:0/4108637300 shutdown_connections 2024-04-25T00:04:41.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.773+0000 7f75c16f5640 1 -- 172.21.15.113:0/4108637300 wait complete. 2024-04-25T00:04:41.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.774+0000 7f75c16f5640 1 Processor -- start 2024-04-25T00:04:41.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.774+0000 7f75c16f5640 1 -- start start 2024-04-25T00:04:41.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75bc161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75bc0c9450 0x7f75bc14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75bc0cde10 0x7f75bc0c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc0b7540 con 0x7f75a40039b0 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc0b73c0 con 0x7f75bc0c9450 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75bc0b5ca0 con 0x7f75bc0cde10 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75bc161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75bc161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49730/0 (socket says 172.21.15.113:49730) 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 -- 172.21.15.113:0/1678832987 learned_addr learned my addr 172.21.15.113:0/1678832987 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75bb7fe640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75bc0cde10 0x7f75bc0c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75ba7fc640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75bc0c9450 0x7f75bc14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 -- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75bc0cde10 msgr2=0x7f75bc0c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75bc0cde10 0x7f75bc0c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 -- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75bc0c9450 msgr2=0x7f75bc14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75bc0c9450 0x7f75bc14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 -- 172.21.15.113:0/1678832987 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc161620 con 0x7f75a40039b0 2024-04-25T00:04:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75ba7fc640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75bc0c9450 0x7f75bc14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75baffd640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75bc161040 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f75ac001cf0 tx=0x7f75ac02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75a37fe640 1 -- 172.21.15.113:0/1678832987 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac045070 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75a37fe640 1 -- 172.21.15.113:0/1678832987 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ac040680 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc0c8970 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75a37fe640 1 -- 172.21.15.113:0/1678832987 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac040a70 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.775+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc0c8f30 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.776+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75bc0691a0 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.777+0000 7f75a37fe640 1 -- 172.21.15.113:0/1678832987 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75ac048070 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.777+0000 7f75a37fe640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be60 0x7f75a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.777+0000 7f75a37fe640 1 -- 172.21.15.113:0/1678832987 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5357+0+0 (secure 0 0 0) 0x7f75ac0ab530 con 0x7f75a40039b0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.777+0000 7f75ba7fc640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be60 0x7f75a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.777+0000 7f75ba7fc640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be60 0x7f75a806e320 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f75bc160dc0 tx=0x7f75b0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:41.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.781+0000 7f75a37fe640 1 -- 172.21.15.113:0/1678832987 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75ac07a100 con 0x7f75a40039b0 2024-04-25T00:04:41.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:41.953+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75bc0cf850 con 0x7f75a40039b0 2024-04-25T00:04:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.655+0000 7f75a37fe640 1 -- 172.21.15.113:0/1678832987 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f75ac079a00 con 0x7f75a40039b0 2024-04-25T00:04:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be60 msgr2=0x7f75a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be60 0x7f75a806e320 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f75bc160dc0 tx=0x7f75b0008040 comp rx=0 tx=0).stop 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 msgr2=0x7f75bc161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75bc161040 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f75ac001cf0 tx=0x7f75ac02d070 comp rx=0 tx=0).stop 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 shutdown_connections 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be60 0x7f75a806e320 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75bc0c9450 0x7f75bc14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75bc0cde10 0x7f75bc0c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 --2- 172.21.15.113:0/1678832987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40039b0 0x7f75bc161040 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 >> 172.21.15.113:0/1678832987 conn(0x7f75bc06b720 msgr2=0x7f75bc071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:42.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.660+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 shutdown_connections 2024-04-25T00:04:42.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.662+0000 7f75c16f5640 1 -- 172.21.15.113:0/1678832987 wait complete. 2024-04-25T00:04:42.686 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:04:42.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.791+0000 7f232539e640 1 Processor -- start 2024-04-25T00:04:42.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.791+0000 7f232539e640 1 -- start start 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.791+0000 7f232539e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232015ba50 0x7f232015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.791+0000 7f232539e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2320071170 con 0x7f232015b670 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.791+0000 7f232539e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2320071450 con 0x7f232006cea0 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.791+0000 7f232539e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2320071730 con 0x7f232015ba50 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f232015b670 0x7f23200cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37178/0 (socket says 172.21.15.113:37178) 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231effd640 1 -- 172.21.15.113:0/2223269401 learned_addr learned my addr 172.21.15.113:0/2223269401 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3354976063 0 0) 0x7f2320071170 con 0x7f232015b670 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2308003520 con 0x7f232015b670 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4179155813 0 0) 0x7f2308003520 con 0x7f232015b670 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232015ba50 msgr2=0x7f232015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232015ba50 0x7f232015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 >> v1:172.21.15.178:6789/0 conn(0x7f232006cea0 legacy=0x7f2320072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2320071a10 con 0x7f232015b670 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f2310001670 con 0x7f232015b670 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 >> v1:172.21.15.113:6789/0 conn(0x7f232015b670 legacy=0x7f23200cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23080039e0 0x7f2308003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2308004800 0x7f2308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232015ba50 0x7f230800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2320071730 con 0x7f23080039e0 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2308005650 con 0x7f2308004800 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f230800bf50 con 0x7f232015ba50 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.792+0000 7f231effd640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23080039e0 0x7f2308003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231e7fc640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2308004800 0x7f2308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231f7fe640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232015ba50 0x7f230800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231effd640 1 -- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232015ba50 msgr2=0x7f230800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231effd640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232015ba50 0x7f230800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231effd640 1 -- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2308004800 msgr2=0x7f2308004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231effd640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2308004800 0x7f2308004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231effd640 1 -- 172.21.15.113:0/2223269401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2308003520 con 0x7f23080039e0 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231effd640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23080039e0 0x7f2308003e00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f231000c270 tx=0x7f2308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1d7bbf0adb940f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310007d10 con 0x7f23080039e0 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2310005ad0 con 0x7f23080039e0 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.793+0000 7f231dffb640 1 -- 172.21.15.113:0/2223269401 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310007070 con 0x7f23080039e0 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.794+0000 7f232539e640 1 -- 172.21.15.113:0/2223269401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23080039e0 msgr2=0x7f2308003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.794+0000 7f232539e640 1 --2- 172.21.15.113:0/2223269401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23080039e0 0x7f2308003e00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f231000c270 tx=0x7f2308005830 comp rx=0 tx=0).stop 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.794+0000 7f231f7fe640 1 -- 172.21.15.113:0/2223269401 reap_dead start 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.794+0000 7f232539e640 1 -- 172.21.15.113:0/2223269401 shutdown_connections 2024-04-25T00:04:42.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.794+0000 7f232539e640 1 -- 172.21.15.113:0/2223269401 >> 172.21.15.113:0/2223269401 conn(0x7f232006b0f0 msgr2=0x7f232006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:42.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.794+0000 7f232539e640 1 -- 172.21.15.113:0/2223269401 shutdown_connections 2024-04-25T00:04:42.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.794+0000 7f232539e640 1 -- 172.21.15.113:0/2223269401 wait complete. 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.795+0000 7f232539e640 1 Processor -- start 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.795+0000 7f232539e640 1 -- start start 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f232539e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23080039e0 0x7f23200c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f232539e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2320153940 0x7f232014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f232539e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2320153dc0 0x7f23200c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f232539e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f232015c170 con 0x7f2320153dc0 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f232539e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f232015bff0 con 0x7f23080039e0 2024-04-25T00:04:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f232539e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f232015c2f0 con 0x7f2320153940 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2320153dc0 0x7f23200c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2320153dc0 0x7f23200c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49754/0 (socket says 172.21.15.113:49754) 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 -- 172.21.15.113:0/2285797910 learned_addr learned my addr 172.21.15.113:0/2285797910 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231e7fc640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2320153940 0x7f232014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231effd640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23080039e0 0x7f23200c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 -- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2320153940 msgr2=0x7f232014f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2320153940 0x7f232014f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 -- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23080039e0 msgr2=0x7f23200c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23080039e0 0x7f23200c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 -- 172.21.15.113:0/2285797910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23200c4f80 con 0x7f2320153dc0 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231effd640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23080039e0 0x7f23200c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.796+0000 7f231f7fe640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2320153dc0 0x7f23200c9290 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2318000f80 tx=0x7f231800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.797+0000 7f23077fe640 1 -- 172.21.15.113:0/2285797910 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318003970 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.797+0000 7f23077fe640 1 -- 172.21.15.113:0/2285797910 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2318018070 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.797+0000 7f23077fe640 1 -- 172.21.15.113:0/2285797910 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318003d50 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.797+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23200c5260 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.797+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23200c5820 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.798+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23200691a0 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.798+0000 7f23077fe640 1 -- 172.21.15.113:0/2285797910 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23180050d0 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.798+0000 7f23077fe640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f230c06be30 0x7f230c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.798+0000 7f23077fe640 1 -- 172.21.15.113:0/2285797910 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f231808a770 con 0x7f2320153dc0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.798+0000 7f231effd640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f230c06be30 0x7f230c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.799+0000 7f231effd640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f230c06be30 0x7f230c06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2310033000 tx=0x7f2310009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.802+0000 7f23077fe640 1 -- 172.21.15.113:0/2285797910 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2318059420 con 0x7f2320153dc0 2024-04-25T00:04:42.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.976+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f23200683a0 con 0x7f2320153dc0 2024-04-25T00:04:42.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.979+0000 7f23077fe640 1 -- 172.21.15.113:0/2285797910 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f2318058d20 con 0x7f2320153dc0 2024-04-25T00:04:42.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f230c06be30 msgr2=0x7f230c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f230c06be30 0x7f230c06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2310033000 tx=0x7f2310009030 comp rx=0 tx=0).stop 2024-04-25T00:04:42.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2320153dc0 msgr2=0x7f23200c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:42.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2320153dc0 0x7f23200c9290 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2318000f80 tx=0x7f231800ec10 comp rx=0 tx=0).stop 2024-04-25T00:04:42.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 shutdown_connections 2024-04-25T00:04:42.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f230c06be30 0x7f230c06e2f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2320153dc0 0x7f23200c9290 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2320153940 0x7f232014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 --2- 172.21.15.113:0/2285797910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23080039e0 0x7f23200c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:42.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 >> 172.21.15.113:0/2285797910 conn(0x7f232006b0f0 msgr2=0x7f232015a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:42.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.981+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 shutdown_connections 2024-04-25T00:04:42.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:42.982+0000 7f232539e640 1 -- 172.21.15.113:0/2285797910 wait complete. 2024-04-25T00:04:42.994 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T00:04:43.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.107+0000 7f0bd2d61640 1 Processor -- start 2024-04-25T00:04:43.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.107+0000 7f0bd2d61640 1 -- start start 2024-04-25T00:04:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.107+0000 7f0bd2d61640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0bcc144910 0x7f0bcc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.107+0000 7f0bd2d61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 0x7f0bcc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.107+0000 7f0bd2d61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bcc144cf0 con 0x7f0bcc0b5680 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.107+0000 7f0bd2d61640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0bcc0b4510 con 0x7f0bcc144910 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.107+0000 7f0bd2d61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b2f30 con 0x7f0bcc0b5240 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bd12fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0bcc0b5680 0x7f0bcc0cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37186/0 (socket says 172.21.15.113:37186) 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bd12fe640 1 -- 172.21.15.113:0/96403759 learned_addr learned my addr 172.21.15.113:0/96403759 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757230100 0 0) 0x7f0bcc144cf0 con 0x7f0bcc0b5680 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bbc003520 con 0x7f0bcc0b5680 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 0x7f0bcc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bd0afd640 1 --2- 172.21.15.113:0/96403759 >> v2:172.21.15.178:3300/0 conn(0x7f0bcc144910 0x7f0bcc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2537859858 0 0) 0x7f0bbc003520 con 0x7f0bcc0b5680 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 msgr2=0x7f0bcc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 0x7f0bcc06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 >> v2:172.21.15.178:3300/0 conn(0x7f0bcc144910 msgr2=0x7f0bcc0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 --2- 172.21.15.113:0/96403759 >> v2:172.21.15.178:3300/0 conn(0x7f0bcc144910 0x7f0bcc0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc0b3210 con 0x7f0bcc0b5680 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0bc40019f0 con 0x7f0bcc0b5680 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 >> v1:172.21.15.113:6789/0 conn(0x7f0bcc0b5680 legacy=0x7f0bcc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bbc0039b0 0x7f0bbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc004800 0x7f0bbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 0x7f0bbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b4510 con 0x7f0bbc004800 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b2f30 con 0x7f0bbc0039b0 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc00b9c0 con 0x7f0bcc0b5240 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc004800 0x7f0bbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bd0afd640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bbc0039b0 0x7f0bbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.108+0000 7f0bd12fe640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 0x7f0bbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcbfff640 1 -- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 msgr2=0x7f0bbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bcc0b5240 0x7f0bbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcbfff640 1 -- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bbc0039b0 msgr2=0x7f0bbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bbc0039b0 0x7f0bbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcbfff640 1 -- 172.21.15.113:0/96403759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc003520 con 0x7f0bbc004800 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc004800 0x7f0bbc004c80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009ac0 tx=0x7f0bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76450e698657e88e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8036070 con 0x7f0bbc004800 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb8031400 con 0x7f0bbc004800 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.109+0000 7f0bcb7fe640 1 -- 172.21.15.113:0/96403759 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8009690 con 0x7f0bbc004800 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.110+0000 7f0bd2d61640 1 -- 172.21.15.113:0/96403759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc004800 msgr2=0x7f0bbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.110+0000 7f0bd2d61640 1 --2- 172.21.15.113:0/96403759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc004800 0x7f0bbc004c80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009ac0 tx=0x7f0bbc005830 comp rx=0 tx=0).stop 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.110+0000 7f0bd12fe640 1 -- 172.21.15.113:0/96403759 reap_dead start 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.110+0000 7f0bd2d61640 1 -- 172.21.15.113:0/96403759 shutdown_connections 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.110+0000 7f0bd2d61640 1 -- 172.21.15.113:0/96403759 >> 172.21.15.113:0/96403759 conn(0x7f0bcc0b1a50 msgr2=0x7f0bcc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.110+0000 7f0bd2d61640 1 -- 172.21.15.113:0/96403759 shutdown_connections 2024-04-25T00:04:43.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.110+0000 7f0bd2d61640 1 -- 172.21.15.113:0/96403759 wait complete. 2024-04-25T00:04:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.111+0000 7f0bd2d61640 1 Processor -- start 2024-04-25T00:04:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.111+0000 7f0bd2d61640 1 -- start start 2024-04-25T00:04:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd2d61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bbc004800 0x7f0bcc13bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd2d61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc13ecc0 0x7f0bcc13d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd2d61640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc15d5f0 0x7f0bcc13de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd2d61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b4da0 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd2d61640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b37d0 con 0x7f0bcc15d5f0 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd2d61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b4f20 con 0x7f0bbc004800 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc13ecc0 0x7f0bcc13d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc13ecc0 0x7f0bcc13d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49768/0 (socket says 172.21.15.113:49768) 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 -- 172.21.15.113:0/1210479438 learned_addr learned my addr 172.21.15.113:0/1210479438 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd0afd640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bbc004800 0x7f0bcc13bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bd12fe640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc15d5f0 0x7f0bcc13de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 -- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bbc004800 msgr2=0x7f0bcc13bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bbc004800 0x7f0bcc13bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 -- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc15d5f0 msgr2=0x7f0bcc13de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc15d5f0 0x7f0bcc13de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.112+0000 7f0bcbfff640 1 -- 172.21.15.113:0/1210479438 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc13b520 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.113+0000 7f0bcbfff640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc13ecc0 0x7f0bcc13d920 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8034040 tx=0x7f0bb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.113+0000 7f0bc97fa640 1 -- 172.21.15.113:0/1210479438 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8041020 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.113+0000 7f0bc97fa640 1 -- 172.21.15.113:0/1210479438 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb803c470 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.113+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc13b800 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.113+0000 7f0bc97fa640 1 -- 172.21.15.113:0/1210479438 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8036040 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.113+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc13e7a0 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.114+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc0691a0 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.114+0000 7f0bc97fa640 1 -- 172.21.15.113:0/1210479438 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bb8044020 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.114+0000 7f0bc97fa640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bb006bd70 0x7f0bb006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.115+0000 7f0bc97fa640 1 -- 172.21.15.113:0/1210479438 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f0bb80aae20 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.115+0000 7f0bd0afd640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bb006bd70 0x7f0bb006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.115+0000 7f0bd0afd640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bb006bd70 0x7f0bb006e230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc1f3270 tx=0x7f0bc0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.119+0000 7f0bc97fa640 1 -- 172.21.15.113:0/1210479438 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0bb804b070 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.358+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) -- 0x7f0bcc143e70 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.657+0000 7f0bc97fa640 1 -- 172.21.15.113:0/1210479438 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v313) ==== 144+0+0 (secure 0 0 0) 0x7f0bb8079510 con 0x7f0bcc13ecc0 2024-04-25T00:04:43.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bb006bd70 msgr2=0x7f0bb006e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bb006bd70 0x7f0bb006e230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc1f3270 tx=0x7f0bc0009190 comp rx=0 tx=0).stop 2024-04-25T00:04:43.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc13ecc0 msgr2=0x7f0bcc13d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc13ecc0 0x7f0bcc13d920 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8034040 tx=0x7f0bb8004730 comp rx=0 tx=0).stop 2024-04-25T00:04:43.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 shutdown_connections 2024-04-25T00:04:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0bb006bd70 0x7f0bb006e230 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bcc15d5f0 0x7f0bcc13de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bcc13ecc0 0x7f0bcc13d920 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 --2- 172.21.15.113:0/1210479438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bbc004800 0x7f0bcc13bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 >> 172.21.15.113:0/1210479438 conn(0x7f0bcc0b1a50 msgr2=0x7f0bcc0b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 shutdown_connections 2024-04-25T00:04:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.659+0000 7f0bd2d61640 1 -- 172.21.15.113:0/1210479438 wait complete. 2024-04-25T00:04:43.671 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T00:04:43.671 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T00:04:43.671 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:43.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.767+0000 7f3f0716a640 1 Processor -- start 2024-04-25T00:04:43.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.767+0000 7f3f0716a640 1 -- start start 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3f0716a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3f00144910 0x7f3f000b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3f0716a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 0x7f3f000cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3f0716a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f00144cf0 con 0x7f3f000b5240 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3f0716a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000b4510 con 0x7f3f00144910 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3f0716a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000b2f30 con 0x7f3f000b56b0 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3efffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3f000b5240 0x7f3f0006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37190/0 (socket says 172.21.15.113:37190) 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3efffff640 1 -- 172.21.15.113:0/4177959999 learned_addr learned my addr 172.21.15.113:0/4177959999 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3f04f06640 1 --2- 172.21.15.113:0/4177959999 >> v2:172.21.15.178:3300/0 conn(0x7f3f00144910 0x7f3f000b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3f05707640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 0x7f3f000cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939618234 0 0) 0x7f3f00144cf0 con 0x7f3f000b5240 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee8003520 con 0x7f3f000b5240 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.768+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2424081980 0 0) 0x7f3ee8003520 con 0x7f3f000b5240 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 msgr2=0x7f3f000cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 0x7f3f000cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 >> v2:172.21.15.178:3300/0 conn(0x7f3f00144910 msgr2=0x7f3f000b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 --2- 172.21.15.113:0/4177959999 >> v2:172.21.15.178:3300/0 conn(0x7f3f00144910 0x7f3f000b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000b3210 con 0x7f3f000b5240 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f3ef4001670 con 0x7f3f000b5240 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 >> v1:172.21.15.113:6789/0 conn(0x7f3f000b5240 legacy=0x7f3f0006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee80039b0 0x7f3ee8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 0x7f3ee8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b4510 con 0x7f3ee80039b0 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b2f30 con 0x7f3ee8004d40 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee800b9c0 con 0x7f3f000b56b0 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f04f06640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee80039b0 0x7f3ee8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3efffff640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 0x7f3ee8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f05707640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f04f06640 1 -- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 msgr2=0x7f3ee8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f04f06640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3f000b56b0 0x7f3ee8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f04f06640 1 -- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee8004d40 msgr2=0x7f3ee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f04f06640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f04f06640 1 -- 172.21.15.113:0/4177959999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee8003520 con 0x7f3ee80039b0 2024-04-25T00:04:43.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.769+0000 7f3f05707640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f04f06640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee80039b0 0x7f3ee8003dd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009ac0 tx=0x7f3ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4c96dffd5a0792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0036070 con 0x7f3ee80039b0 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0031400 con 0x7f3ee80039b0 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3eff7fe640 1 -- 172.21.15.113:0/4177959999 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0009690 con 0x7f3ee80039b0 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f0716a640 1 -- 172.21.15.113:0/4177959999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee80039b0 msgr2=0x7f3ee8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f0716a640 1 --2- 172.21.15.113:0/4177959999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee80039b0 0x7f3ee8003dd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009ac0 tx=0x7f3ee8005830 comp rx=0 tx=0).stop 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f05707640 1 -- 172.21.15.113:0/4177959999 reap_dead start 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f0716a640 1 -- 172.21.15.113:0/4177959999 shutdown_connections 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f0716a640 1 -- 172.21.15.113:0/4177959999 >> 172.21.15.113:0/4177959999 conn(0x7f3f000b1a50 msgr2=0x7f3f000b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f0716a640 1 -- 172.21.15.113:0/4177959999 shutdown_connections 2024-04-25T00:04:43.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.770+0000 7f3f0716a640 1 -- 172.21.15.113:0/4177959999 wait complete. 2024-04-25T00:04:43.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 Processor -- start 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 -- start start 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee80039b0 0x7f3f001540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f001f3cd0 0x7f3f001402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f00140830 0x7f3f000ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b4da0 con 0x7f3f001f3cd0 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b37d0 con 0x7f3f00140830 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.772+0000 7f3f0716a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000b4f20 con 0x7f3ee80039b0 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f001f3cd0 0x7f3f001402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f001f3cd0 0x7f3f001402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49780/0 (socket says 172.21.15.113:49780) 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 -- 172.21.15.113:0/3926192529 learned_addr learned my addr 172.21.15.113:0/3926192529 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3f04f06640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee80039b0 0x7f3f001540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3f05707640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f00140830 0x7f3f000ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 -- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee80039b0 msgr2=0x7f3f001540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee80039b0 0x7f3f001540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 -- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f00140830 msgr2=0x7f3f000ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f00140830 0x7f3f000ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 -- 172.21.15.113:0/3926192529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f00154610 con 0x7f3f001f3cd0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efffff640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f001f3cd0 0x7f3f001402f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007270 tx=0x7f3ef4007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efd7fa640 1 -- 172.21.15.113:0/3926192529 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4015070 con 0x7f3f001f3cd0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efd7fa640 1 -- 172.21.15.113:0/3926192529 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef400fd30 con 0x7f3f001f3cd0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f00138710 con 0x7f3f001f3cd0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3efd7fa640 1 -- 172.21.15.113:0/3926192529 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4011680 con 0x7f3f001f3cd0 2024-04-25T00:04:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.773+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f00138cd0 con 0x7f3f001f3cd0 2024-04-25T00:04:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.774+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f000691a0 con 0x7f3f001f3cd0 2024-04-25T00:04:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.775+0000 7f3efd7fa640 1 -- 172.21.15.113:0/3926192529 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ef4018070 con 0x7f3f001f3cd0 2024-04-25T00:04:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.775+0000 7f3efd7fa640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eec06be60 0x7f3eec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.775+0000 7f3efd7fa640 1 -- 172.21.15.113:0/3926192529 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f3ef408ac10 con 0x7f3f001f3cd0 2024-04-25T00:04:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.775+0000 7f3f04f06640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eec06be60 0x7f3eec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.775+0000 7f3f04f06640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eec06be60 0x7f3eec06e320 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0033000 tx=0x7f3ef0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.779+0000 7f3efd7fa640 1 -- 172.21.15.113:0/3926192529 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ef4059950 con 0x7f3f001f3cd0 2024-04-25T00:04:43.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.953+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f000731a0 con 0x7f3f001f3cd0 2024-04-25T00:04:43.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.954+0000 7f3efd7fa640 1 -- 172.21.15.113:0/3926192529 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+13071 (secure 0 0 0) 0x7f3ef4059250 con 0x7f3f001f3cd0 2024-04-25T00:04:43.955 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":95,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:42.650576+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6837/2796979045":"2024-04-26T00:04:39.645959+0000","172.21.15.113:6835/2796979045":"2024-04-26T00:04:39.645959+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:43.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eec06be60 msgr2=0x7f3eec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eec06be60 0x7f3eec06e320 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0033000 tx=0x7f3ef0008030 comp rx=0 tx=0).stop 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f001f3cd0 msgr2=0x7f3f001402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f001f3cd0 0x7f3f001402f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007270 tx=0x7f3ef4007730 comp rx=0 tx=0).stop 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 shutdown_connections 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eec06be60 0x7f3eec06e320 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f00140830 0x7f3f000ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f001f3cd0 0x7f3f001402f0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 --2- 172.21.15.113:0/3926192529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee80039b0 0x7f3f001540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 >> 172.21.15.113:0/3926192529 conn(0x7f3f000b1a50 msgr2=0x7f3f0006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 shutdown_connections 2024-04-25T00:04:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:43.956+0000 7f3f0716a640 1 -- 172.21.15.113:0/3926192529 wait complete. 2024-04-25T00:04:43.968 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:04:43.968 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:04:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.067+0000 7f702c031640 1 Processor -- start 2024-04-25T00:04:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.067+0000 7f702c031640 1 -- start start 2024-04-25T00:04:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f702c031640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702415e510 0x7f70240d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f702c031640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70240d28c0 0x7f70240b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f702c031640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70240d2e00 con 0x7f70240b61e0 2024-04-25T00:04:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f702c031640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f70240d30e0 con 0x7f70240d28c0 2024-04-25T00:04:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f702c031640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f70240d33c0 con 0x7f702415e510 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f70295cc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70240d28c0 0x7f70240b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f70295cc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70240d28c0 0x7f70240b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49782/0 (socket says 172.21.15.113:49782) 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.068+0000 7f70295cc640 1 -- 172.21.15.113:0/605232577 learned_addr learned my addr 172.21.15.113:0/605232577 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.069+0000 7f7029dcd640 1 --2- 172.21.15.113:0/605232577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702415e510 0x7f70240d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.069+0000 7f70295cc640 1 -- 172.21.15.113:0/605232577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702415e510 msgr2=0x7f70240d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.069+0000 7f70295cc640 1 --2- 172.21.15.113:0/605232577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702415e510 0x7f70240d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.069+0000 7f70295cc640 1 -- 172.21.15.113:0/605232577 >> v1:172.21.15.178:6789/0 conn(0x7f70240b61e0 legacy=0x7f7024071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.069+0000 7f70295cc640 1 -- 172.21.15.113:0/605232577 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70240d36a0 con 0x7f70240d28c0 2024-04-25T00:04:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.069+0000 7f70295cc640 1 --2- 172.21.15.113:0/605232577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70240d28c0 0x7f70240b5ca0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f7014009870 tx=0x7f701402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db9c6f6288406e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.069+0000 7f7028dcb640 1 -- 172.21.15.113:0/605232577 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014037070 con 0x7f70240d28c0 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f7028dcb640 1 -- 172.21.15.113:0/605232577 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7014032440 con 0x7f70240d28c0 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f7028dcb640 1 -- 172.21.15.113:0/605232577 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014032810 con 0x7f70240d28c0 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 -- 172.21.15.113:0/605232577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70240d28c0 msgr2=0x7f70240b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 --2- 172.21.15.113:0/605232577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70240d28c0 0x7f70240b5ca0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f7014009870 tx=0x7f701402a180 comp rx=0 tx=0).stop 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 -- 172.21.15.113:0/605232577 shutdown_connections 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 --2- 172.21.15.113:0/605232577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70240d28c0 0x7f70240b5ca0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 --2- 172.21.15.113:0/605232577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f702415e510 0x7f70240d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 -- 172.21.15.113:0/605232577 >> 172.21.15.113:0/605232577 conn(0x7f702406b720 msgr2=0x7f70240d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 -- 172.21.15.113:0/605232577 shutdown_connections 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.070+0000 7f702c031640 1 -- 172.21.15.113:0/605232577 wait complete. 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.071+0000 7f702c031640 1 Processor -- start 2024-04-25T00:04:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.071+0000 7f702c031640 1 -- start start 2024-04-25T00:04:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702c031640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70240b61e0 0x7f702414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702c031640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70240d28c0 0x7f70240ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702c031640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702415e510 0x7f7024156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702c031640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f70240b9dd0 con 0x7f702415e510 2024-04-25T00:04:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702c031640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70240d3ea0 con 0x7f70240d28c0 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702c031640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f70240b83d0 con 0x7f70240b61e0 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702415e510 0x7f7024156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702415e510 0x7f7024156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49792/0 (socket says 172.21.15.113:49792) 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 -- 172.21.15.113:0/1650428690 learned_addr learned my addr 172.21.15.113:0/1650428690 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f7029dcd640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70240b61e0 0x7f702414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f70295cc640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70240d28c0 0x7f70240ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 -- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70240b61e0 msgr2=0x7f702414c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70240b61e0 0x7f702414c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 -- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70240d28c0 msgr2=0x7f70240ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70240d28c0 0x7f70240ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.072+0000 7f702a5ce640 1 -- 172.21.15.113:0/1650428690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70240ca9b0 con 0x7f702415e510 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.073+0000 7f702a5ce640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702415e510 0x7f7024156fc0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c410 tx=0x7f701c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.073+0000 7f701a7fc640 1 -- 172.21.15.113:0/1650428690 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c011790 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.073+0000 7f701a7fc640 1 -- 172.21.15.113:0/1650428690 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701c017070 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.073+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70240c93d0 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.073+0000 7f701a7fc640 1 -- 172.21.15.113:0/1650428690 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c011bb0 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.073+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70240c9c70 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.074+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70240691a0 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.074+0000 7f701a7fc640 1 -- 172.21.15.113:0/1650428690 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f701c01c460 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.075+0000 7f701a7fc640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f700806be30 0x7f700806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.075+0000 7f7029dcd640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f700806be30 0x7f700806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.075+0000 7f701a7fc640 1 -- 172.21.15.113:0/1650428690 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f701c08a770 con 0x7f702415e510 2024-04-25T00:04:44.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.075+0000 7f7029dcd640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f700806be30 0x7f700806e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f70200070b0 tx=0x7f7020007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.079+0000 7f701a7fc640 1 -- 172.21.15.113:0/1650428690 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f701c021020 con 0x7f702415e510 2024-04-25T00:04:44.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.251+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f70240c9760 con 0x7f702415e510 2024-04-25T00:04:44.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.251+0000 7f701a7fc640 1 -- 172.21.15.113:0/1650428690 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f701c059420 con 0x7f702415e510 2024-04-25T00:04:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:04:44.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f700806be30 msgr2=0x7f700806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f700806be30 0x7f700806e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f70200070b0 tx=0x7f7020007040 comp rx=0 tx=0).stop 2024-04-25T00:04:44.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702415e510 msgr2=0x7f7024156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702415e510 0x7f7024156fc0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c410 tx=0x7f701c00c8d0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 shutdown_connections 2024-04-25T00:04:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f700806be30 0x7f700806e2f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70240d28c0 0x7f70240ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f702415e510 0x7f7024156fc0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 --2- 172.21.15.113:0/1650428690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70240b61e0 0x7f702414c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 >> 172.21.15.113:0/1650428690 conn(0x7f702406b720 msgr2=0x7f70240d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 shutdown_connections 2024-04-25T00:04:44.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.253+0000 7f702c031640 1 -- 172.21.15.113:0/1650428690 wait complete. 2024-04-25T00:04:44.265 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T00:04:44.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.361+0000 7efc7315c640 1 Processor -- start 2024-04-25T00:04:44.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.362+0000 7efc7315c640 1 -- start start 2024-04-25T00:04:44.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.362+0000 7efc7315c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c15e2c0 0x7efc6c0b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.362+0000 7efc7315c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efc6c0b6810 0x7efc6c0b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.362+0000 7efc7315c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc6c15e6a0 con 0x7efc6c0d24a0 2024-04-25T00:04:44.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.362+0000 7efc7315c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efc6c0d2c80 con 0x7efc6c0b6810 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.362+0000 7efc7315c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc6c0d2f60 con 0x7efc6c15e2c0 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc70ef8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c15e2c0 0x7efc6c0b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc716f9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efc6c0b6810 0x7efc6c0b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc716f9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efc6c0b6810 0x7efc6c0b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:58876/0 (socket says 172.21.15.113:58876) 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc6bfff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7efc6c0d24a0 0x7efc6c0b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:41004/0 (socket says 172.21.15.113:41004) 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc70ef8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c15e2c0 0x7efc6c0b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:36360/0 (socket says 172.21.15.113:36360) 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc716f9640 1 -- 172.21.15.113:0/3368594833 learned_addr learned my addr 172.21.15.113:0/3368594833 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc716f9640 1 -- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c15e2c0 msgr2=0x7efc6c0b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc716f9640 1 --2- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c15e2c0 0x7efc6c0b9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc716f9640 1 -- 172.21.15.113:0/3368594833 >> v1:172.21.15.178:6789/0 conn(0x7efc6c0d24a0 legacy=0x7efc6c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc716f9640 1 -- 172.21.15.113:0/3368594833 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c0d3240 con 0x7efc6c0b6810 2024-04-25T00:04:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.363+0000 7efc70ef8640 1 --2- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c15e2c0 0x7efc6c0b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc716f9640 1 --2- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c0b6810 0x7efc6c0b7200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efc64009e50 tx=0x7efc6402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49660c4c6fcc0580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc6b7fe640 1 -- 172.21.15.113:0/3368594833 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64036070 con 0x7efc6c0b6810 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc6b7fe640 1 -- 172.21.15.113:0/3368594833 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc64031600 con 0x7efc6c0b6810 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc6b7fe640 1 -- 172.21.15.113:0/3368594833 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc640319d0 con 0x7efc6c0b6810 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 -- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c0b6810 msgr2=0x7efc6c0b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 --2- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c0b6810 0x7efc6c0b7200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efc64009e50 tx=0x7efc6402a680 comp rx=0 tx=0).stop 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 -- 172.21.15.113:0/3368594833 shutdown_connections 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 --2- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c0b6810 0x7efc6c0b7200 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 --2- 172.21.15.113:0/3368594833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c15e2c0 0x7efc6c0b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 -- 172.21.15.113:0/3368594833 >> 172.21.15.113:0/3368594833 conn(0x7efc6c06b510 msgr2=0x7efc6c06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 -- 172.21.15.113:0/3368594833 shutdown_connections 2024-04-25T00:04:44.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.364+0000 7efc7315c640 1 -- 172.21.15.113:0/3368594833 wait complete. 2024-04-25T00:04:44.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 Processor -- start 2024-04-25T00:04:44.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 -- start start 2024-04-25T00:04:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c0b6810 0x7efc6c149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc6c0d24a0 0x7efc6c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c15e2c0 0x7efc6c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c0d2900 con 0x7efc6c0d24a0 2024-04-25T00:04:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c0d3d20 con 0x7efc6c15e2c0 2024-04-25T00:04:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.366+0000 7efc7315c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc6c0b7740 con 0x7efc6c0b6810 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc6c0d24a0 0x7efc6c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc6c0d24a0 0x7efc6c157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49800/0 (socket says 172.21.15.113:49800) 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 -- 172.21.15.113:0/2241552019 learned_addr learned my addr 172.21.15.113:0/2241552019 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc70ef8640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c0b6810 0x7efc6c149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc716f9640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c15e2c0 0x7efc6c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 -- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c0b6810 msgr2=0x7efc6c149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c0b6810 0x7efc6c149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 -- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c15e2c0 msgr2=0x7efc6c1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c15e2c0 0x7efc6c1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 -- 172.21.15.113:0/2241552019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c15a270 con 0x7efc6c0d24a0 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc716f9640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c15e2c0 0x7efc6c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc6bfff640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc6c0d24a0 0x7efc6c157f90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7efc5800c400 tx=0x7efc5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc697fa640 1 -- 172.21.15.113:0/2241552019 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580074e0 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc697fa640 1 -- 172.21.15.113:0/2241552019 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc580047e0 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c15a550 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc697fa640 1 -- 172.21.15.113:0/2241552019 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004c10 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.367+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c15ab10 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.368+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6c0691a0 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.369+0000 7efc697fa640 1 -- 172.21.15.113:0/2241552019 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc5801a020 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.369+0000 7efc697fa640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5006be60 0x7efc5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.369+0000 7efc70ef8640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5006be60 0x7efc5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.369+0000 7efc697fa640 1 -- 172.21.15.113:0/2241552019 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7efc5808a350 con 0x7efc6c0d24a0 2024-04-25T00:04:44.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.369+0000 7efc70ef8640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5006be60 0x7efc5006e320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efc60009a30 tx=0x7efc60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.373+0000 7efc697fa640 1 -- 172.21.15.113:0/2241552019 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc58059000 con 0x7efc6c0d24a0 2024-04-25T00:04:44.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.613+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) -- 0x7efc6c15d750 con 0x7efc6c0d24a0 2024-04-25T00:04:44.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.658+0000 7efc697fa640 1 -- 172.21.15.113:0/2241552019 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v314) ==== 158+0+0 (secure 0 0 0) 0x7efc58058900 con 0x7efc6c0d24a0 2024-04-25T00:04:44.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.660+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5006be60 msgr2=0x7efc5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.660+0000 7efc7315c640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5006be60 0x7efc5006e320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efc60009a30 tx=0x7efc60008040 comp rx=0 tx=0).stop 2024-04-25T00:04:44.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.660+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc6c0d24a0 msgr2=0x7efc6c157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.660+0000 7efc7315c640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc6c0d24a0 0x7efc6c157f90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7efc5800c400 tx=0x7efc5800c8c0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 shutdown_connections 2024-04-25T00:04:44.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc5006be60 0x7efc5006e320 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6c15e2c0 0x7efc6c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc6c0d24a0 0x7efc6c157f90 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 --2- 172.21.15.113:0/2241552019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc6c0b6810 0x7efc6c149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 >> 172.21.15.113:0/2241552019 conn(0x7efc6c06b510 msgr2=0x7efc6c0b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:44.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 shutdown_connections 2024-04-25T00:04:44.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.661+0000 7efc7315c640 1 -- 172.21.15.113:0/2241552019 wait complete. 2024-04-25T00:04:44.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:04:44.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.771+0000 7f3692714640 1 Processor -- start 2024-04-25T00:04:44.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.771+0000 7f3692714640 1 -- start start 2024-04-25T00:04:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3692714640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f368c15e2d0 0x7f368c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3692714640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 0x7f368c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3692714640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f368c15e6b0 0x7f368c0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3692714640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f368c0cde00 con 0x7f368c15e6b0 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3692714640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f368c15f4f0 con 0x7f368c15e2d0 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3692714640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f368c15f7d0 con 0x7f368c0b3080 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3690cb1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f368c15e6b0 0x7f368c0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3690cb1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f368c15e6b0 0x7f368c0b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49816/0 (socket says 172.21.15.113:49816) 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3690cb1640 1 -- 172.21.15.113:0/2088143635 learned_addr learned my addr 172.21.15.113:0/2088143635 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3690cb1640 1 -- 172.21.15.113:0/2088143635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 msgr2=0x7f368c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.772+0000 7f3690cb1640 1 --2- 172.21.15.113:0/2088143635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 0x7f368c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f3690cb1640 1 -- 172.21.15.113:0/2088143635 >> v2:172.21.15.178:3300/0 conn(0x7f368c15e2d0 msgr2=0x7f368c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f368bfff640 1 --2- 172.21.15.113:0/2088143635 >> v2:172.21.15.178:3300/0 conn(0x7f368c15e2d0 0x7f368c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f3690cb1640 1 --2- 172.21.15.113:0/2088143635 >> v2:172.21.15.178:3300/0 conn(0x7f368c15e2d0 0x7f368c15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f3690cb1640 1 -- 172.21.15.113:0/2088143635 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c15fab0 con 0x7f368c15e6b0 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f3690cb1640 1 --2- 172.21.15.113:0/2088143635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e6b0 0x7f368c0b4870 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f368c0b2d40 tx=0x7f368402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70ab17261e8b223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f368affd640 1 -- 172.21.15.113:0/2088143635 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684036070 con 0x7f368c15e6b0 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f368affd640 1 -- 172.21.15.113:0/2088143635 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3684031600 con 0x7f368c15e6b0 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.773+0000 7f368affd640 1 -- 172.21.15.113:0/2088143635 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36840319d0 con 0x7f368c15e6b0 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 -- 172.21.15.113:0/2088143635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e6b0 msgr2=0x7f368c0b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 --2- 172.21.15.113:0/2088143635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e6b0 0x7f368c0b4870 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f368c0b2d40 tx=0x7f368402a610 comp rx=0 tx=0).stop 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 -- 172.21.15.113:0/2088143635 shutdown_connections 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 --2- 172.21.15.113:0/2088143635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e6b0 0x7f368c0b4870 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 --2- 172.21.15.113:0/2088143635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 0x7f368c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 --2- 172.21.15.113:0/2088143635 >> v2:172.21.15.178:3300/0 conn(0x7f368c15e2d0 0x7f368c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 -- 172.21.15.113:0/2088143635 >> 172.21.15.113:0/2088143635 conn(0x7f368c06b0f0 msgr2=0x7f368c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:44.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 -- 172.21.15.113:0/2088143635 shutdown_connections 2024-04-25T00:04:44.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.774+0000 7f3692714640 1 -- 172.21.15.113:0/2088143635 wait complete. 2024-04-25T00:04:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.775+0000 7f3692714640 1 Processor -- start 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.775+0000 7f3692714640 1 -- start start 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3692714640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 0x7f368c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3692714640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e2d0 0x7f368c15c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3692714640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f368c15e6b0 0x7f368c15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3692714640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f368c15f2b0 con 0x7f368c15e2d0 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3692714640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f368c15f130 con 0x7f368c15e6b0 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3692714640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f368c071040 con 0x7f368c0b3080 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e2d0 0x7f368c15c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e2d0 0x7f368c15c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49824/0 (socket says 172.21.15.113:49824) 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 -- 172.21.15.113:0/3393133537 learned_addr learned my addr 172.21.15.113:0/3393133537 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368bfff640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 0x7f368c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3690cb1640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f368c15e6b0 0x7f368c15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 -- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 msgr2=0x7f368c1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 0x7f368c1494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 -- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f368c15e6b0 msgr2=0x7f368c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f368c15e6b0 0x7f368c15caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f368b7fe640 1 -- 172.21.15.113:0/3393133537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c14cd00 con 0x7f368c15e2d0 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.776+0000 7f3690cb1640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f368c15e6b0 0x7f368c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.777+0000 7f368b7fe640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e2d0 0x7f368c15c5b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f368000b450 tx=0x7f368000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.777+0000 7f3688ff9640 1 -- 172.21.15.113:0/3393133537 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680015020 con 0x7f368c15e2d0 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.777+0000 7f3688ff9640 1 -- 172.21.15.113:0/3393133537 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3680011890 con 0x7f368c15e2d0 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.777+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c149a00 con 0x7f368c15e2d0 2024-04-25T00:04:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.777+0000 7f3688ff9640 1 -- 172.21.15.113:0/3393133537 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680011c80 con 0x7f368c15e2d0 2024-04-25T00:04:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.777+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c0adc80 con 0x7f368c15e2d0 2024-04-25T00:04:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.778+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c0691a0 con 0x7f368c15e2d0 2024-04-25T00:04:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.778+0000 7f3688ff9640 1 -- 172.21.15.113:0/3393133537 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3680018020 con 0x7f368c15e2d0 2024-04-25T00:04:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.778+0000 7f3688ff9640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f367806be60 0x7f367806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.779+0000 7f3688ff9640 1 -- 172.21.15.113:0/3393133537 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f368008ad40 con 0x7f368c15e2d0 2024-04-25T00:04:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.779+0000 7f368bfff640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f367806be60 0x7f367806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.779+0000 7f368bfff640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f367806be60 0x7f367806e320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f367c006fd0 tx=0x7f367c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.783+0000 7f3688ff9640 1 -- 172.21.15.113:0/3393133537 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f368001f020 con 0x7f368c15e2d0 2024-04-25T00:04:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.958+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f368c0b2730 con 0x7f368c15e2d0 2024-04-25T00:04:44.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.961+0000 7f3688ff9640 1 -- 172.21.15.113:0/3393133537 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f368000d040 con 0x7f368c15e2d0 2024-04-25T00:04:44.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f367806be60 msgr2=0x7f367806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f367806be60 0x7f367806e320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f367c006fd0 tx=0x7f367c008040 comp rx=0 tx=0).stop 2024-04-25T00:04:44.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e2d0 msgr2=0x7f368c15c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:44.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e2d0 0x7f368c15c5b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f368000b450 tx=0x7f368000b910 comp rx=0 tx=0).stop 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 shutdown_connections 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f367806be60 0x7f367806e320 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f368c15e6b0 0x7f368c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f368c15e2d0 0x7f368c15c5b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 --2- 172.21.15.113:0/3393133537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f368c0b3080 0x7f368c1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 >> 172.21.15.113:0/3393133537 conn(0x7f368c06b0f0 msgr2=0x7f368c0729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 shutdown_connections 2024-04-25T00:04:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:44.964+0000 7f3692714640 1 -- 172.21.15.113:0/3393133537 wait complete. 2024-04-25T00:04:44.975 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:04:44.975 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:04:45.019 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:04:45.060 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:04:45.060 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:45.060 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:45.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:04:45.075 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:04:45.076 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:45.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:04:45.117 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:45.117 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:45.132 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:04:45.132 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:04:45.132 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:45.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:04:45.174 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:04:45.174 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:45.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.271+0000 7f11ee637640 1 Processor -- start 2024-04-25T00:04:45.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.271+0000 7f11ee637640 1 -- start start 2024-04-25T00:04:45.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.271+0000 7f11ee637640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 0x7f11e80d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.271+0000 7f11ee637640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11e80d3de0 con 0x7f11e80d0b00 2024-04-25T00:04:45.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.271+0000 7f11ee637640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11e80d2800 con 0x7f11e80d2290 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.271+0000 7f11ee637640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11e80d3410 con 0x7f11e815e320 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11ecbd4640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f11e80d0b00 0x7f11e80b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37200/0 (socket says 172.21.15.113:37200) 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11ecbd4640 1 -- 172.21.15.113:0/613949638 learned_addr learned my addr 172.21.15.113:0/613949638 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e7fff640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 0x7f11e80d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3206724398 0 0) 0x7f11e80d3de0 con 0x7f11e80d0b00 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11d4003520 con 0x7f11e80d0b00 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061161352 0 0) 0x7f11e80d2800 con 0x7f11e80d2290 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11e80d3de0 con 0x7f11e80d2290 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2986613428 0 0) 0x7f11d4003520 con 0x7f11e80d0b00 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 msgr2=0x7f11e80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 0x7f11e80d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 >> v1:172.21.15.178:6789/0 conn(0x7f11e80d2290 legacy=0x7f11e80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e80d2ae0 con 0x7f11e80d0b00 2024-04-25T00:04:45.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f11e00019f0 con 0x7f11e80d0b00 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 >> v1:172.21.15.113:6789/0 conn(0x7f11e80d0b00 legacy=0x7f11e80b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 0x7f11d4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40040d0 0x7f11d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d4004d40 0x7f11d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80d3410 con 0x7f11d4004d40 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11d400baa0 con 0x7f11d40040d0 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11d400bca0 con 0x7f11e815e320 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11ecbd4640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d4004d40 0x7f11d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e7fff640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 0x7f11d4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.272+0000 7f11e77fe640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40040d0 0x7f11d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11ecbd4640 1 -- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 msgr2=0x7f11d4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11ecbd4640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11e815e320 0x7f11d4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11ecbd4640 1 -- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40040d0 msgr2=0x7f11d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11ecbd4640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40040d0 0x7f11d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11ecbd4640 1 -- 172.21.15.113:0/613949638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d4003520 con 0x7f11d4004d40 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11ecbd4640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d4004d40 0x7f11d40051c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f11e000a820 tx=0x7f11d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88eb975e22bc89c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e000a450 con 0x7f11d4004d40 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e002f070 con 0x7f11d4004d40 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.273+0000 7f11e6ffd640 1 -- 172.21.15.113:0/613949638 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e002bcd0 con 0x7f11d4004d40 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.274+0000 7f11ee637640 1 -- 172.21.15.113:0/613949638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d4004d40 msgr2=0x7f11d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.274+0000 7f11ee637640 1 --2- 172.21.15.113:0/613949638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d4004d40 0x7f11d40051c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f11e000a820 tx=0x7f11d4005830 comp rx=0 tx=0).stop 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.274+0000 7f11ecbd4640 1 -- 172.21.15.113:0/613949638 reap_dead start 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.274+0000 7f11ee637640 1 -- 172.21.15.113:0/613949638 shutdown_connections 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.274+0000 7f11ee637640 1 -- 172.21.15.113:0/613949638 >> 172.21.15.113:0/613949638 conn(0x7f11e80b5c20 msgr2=0x7f11e80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.274+0000 7f11ee637640 1 -- 172.21.15.113:0/613949638 shutdown_connections 2024-04-25T00:04:45.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.274+0000 7f11ee637640 1 -- 172.21.15.113:0/613949638 wait complete. 2024-04-25T00:04:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.275+0000 7f11ee637640 1 Processor -- start 2024-04-25T00:04:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.275+0000 7f11ee637640 1 -- start start 2024-04-25T00:04:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ee637640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11d4004d40 0x7f11e8153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ee637640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11e80cb1e0 0x7f11e81604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ee637640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e80cd390 0x7f11e80cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ee637640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f11e815e700 con 0x7f11e80cd390 2024-04-25T00:04:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ee637640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80d30a0 con 0x7f11e80cb1e0 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ee637640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11e815e880 con 0x7f11d4004d40 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e80cd390 0x7f11e80cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e80cd390 0x7f11e80cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49830/0 (socket says 172.21.15.113:49830) 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 -- 172.21.15.113:0/3218824764 learned_addr learned my addr 172.21.15.113:0/3218824764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11e77fe640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11e80cb1e0 0x7f11e81604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11e7fff640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11d4004d40 0x7f11e8153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 -- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11d4004d40 msgr2=0x7f11e8153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11d4004d40 0x7f11e8153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 -- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11e80cb1e0 msgr2=0x7f11e81604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11e80cb1e0 0x7f11e81604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 -- 172.21.15.113:0/3218824764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e8154180 con 0x7f11e80cd390 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.276+0000 7f11ecbd4640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e80cd390 0x7f11e80cd810 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f11e000a940 tx=0x7f11e0001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.277+0000 7f11e4ff9640 1 -- 172.21.15.113:0/3218824764 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e000a450 con 0x7f11e80cd390 2024-04-25T00:04:45.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.277+0000 7f11e4ff9640 1 -- 172.21.15.113:0/3218824764 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e002bc20 con 0x7f11e80cd390 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.277+0000 7f11e4ff9640 1 -- 172.21.15.113:0/3218824764 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0001f90 con 0x7f11e80cd390 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.277+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e80cb8a0 con 0x7f11e80cd390 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.277+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e8158320 con 0x7f11e80cd390 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.278+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e80691a0 con 0x7f11e80cd390 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.278+0000 7f11e4ff9640 1 -- 172.21.15.113:0/3218824764 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11e0034470 con 0x7f11e80cd390 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.278+0000 7f11e4ff9640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f11c806be30 0x7f11c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.279+0000 7f11e4ff9640 1 -- 172.21.15.113:0/3218824764 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f11e00aa810 con 0x7f11e80cd390 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.279+0000 7f11e7fff640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f11c806be30 0x7f11c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.279+0000 7f11e7fff640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f11c806be30 0x7f11c806e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009bc0 tx=0x7f11d8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.283+0000 7f11e4ff9640 1 -- 172.21.15.113:0/3218824764 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f11e0079610 con 0x7f11e80cd390 2024-04-25T00:04:45.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.492+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e80b3750 con 0x7f11e80cd390 2024-04-25T00:04:45.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.492+0000 7f11e4ff9640 1 -- 172.21.15.113:0/3218824764 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+3881 (secure 0 0 0) 0x7f11e0078f10 con 0x7f11e80cd390 2024-04-25T00:04:45.493 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:45.494 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3263483278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3263483278},{"type":"v1","addr":"172.21.15.178:6834","nonce":3263483278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3807770563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3807770563},{"type":"v1","addr":"172.21.15.178:6835","nonce":3807770563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2145774724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2145774724},{"type":"v1","addr":"172.21.15.113:6836","nonce":2145774724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2305704869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2305704869},{"type":"v1","addr":"172.21.15.113:6837","nonce":2305704869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[]} 2024-04-25T00:04:45.494 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-25T00:04:45.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f11c806be30 msgr2=0x7f11c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f11c806be30 0x7f11c806e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009bc0 tx=0x7f11d8009190 comp rx=0 tx=0).stop 2024-04-25T00:04:45.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e80cd390 msgr2=0x7f11e80cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e80cd390 0x7f11e80cd810 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f11e000a940 tx=0x7f11e0001d40 comp rx=0 tx=0).stop 2024-04-25T00:04:45.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 shutdown_connections 2024-04-25T00:04:45.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f11c806be30 0x7f11c806e2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11e80cb1e0 0x7f11e81604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e80cd390 0x7f11e80cd810 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 --2- 172.21.15.113:0/3218824764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11d4004d40 0x7f11e8153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 >> 172.21.15.113:0/3218824764 conn(0x7f11e80b5c20 msgr2=0x7f11e80b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 shutdown_connections 2024-04-25T00:04:45.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.495+0000 7f11ee637640 1 -- 172.21.15.113:0/3218824764 wait complete. 2024-04-25T00:04:45.507 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:04:45.507 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:04:45.508 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:04:45.508 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:45.508 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:04:45.508 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:04:45.508 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:04:45.508 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:45.509 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:04:45.509 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:04:45.509 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:04:45.509 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:45.509 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:04:45.510 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:04:45.510 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:04:45.510 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:04:45.516 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:04:45.517 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:04:45.517 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:04:45.518 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:04:45.521 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:04:45.521 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:04:45.521 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:04:45.522 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:04:45.559 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:04:45.560 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:45.560 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:04:45.560 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:45.563 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:04:45.564 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:45.564 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:04:45.564 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:45.565 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:04:45.632 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:04:45.633 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:04:45.642 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:04:45.642 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:04:45.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.667+0000 7fd1e4d34640 1 Processor -- start 2024-04-25T00:04:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.667+0000 7fd1e4d34640 1 -- start start 2024-04-25T00:04:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.668+0000 7fd1e4d34640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1e015e510 0x7fd1e00d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.668+0000 7fd1e4d34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e00d28c0 0x7fd1e00b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.668+0000 7fd1e4d34640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1e00b61e0 0x7fd1e0071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.668+0000 7fd1e4d34640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00d2e00 con 0x7fd1e015e510 2024-04-25T00:04:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.668+0000 7fd1e4d34640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00d30e0 con 0x7fd1e00b61e0 2024-04-25T00:04:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.668+0000 7fd1e4d34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e00d33c0 con 0x7fd1e00d28c0 2024-04-25T00:04:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1e015e510 0x7fd1e00d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1e015e510 0x7fd1e00d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49878/0 (socket says 172.21.15.113:49878) 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 -- 172.21.15.113:0/4222929483 learned_addr learned my addr 172.21.15.113:0/4222929483 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1d7fff640 1 --2- 172.21.15.113:0/4222929483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e00d28c0 0x7fd1e00b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1ded9d640 1 --2- 172.21.15.113:0/4222929483 >> v2:172.21.15.178:3300/0 conn(0x7fd1e00b61e0 0x7fd1e0071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 -- 172.21.15.113:0/4222929483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e00d28c0 msgr2=0x7fd1e00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 --2- 172.21.15.113:0/4222929483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e00d28c0 0x7fd1e00b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 -- 172.21.15.113:0/4222929483 >> v2:172.21.15.178:3300/0 conn(0x7fd1e00b61e0 msgr2=0x7fd1e0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 --2- 172.21.15.113:0/4222929483 >> v2:172.21.15.178:3300/0 conn(0x7fd1e00b61e0 0x7fd1e0071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1de59c640 1 -- 172.21.15.113:0/4222929483 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e00d36a0 con 0x7fd1e015e510 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1d7fff640 1 --2- 172.21.15.113:0/4222929483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e00d28c0 0x7fd1e00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.669+0000 7fd1ded9d640 1 --2- 172.21.15.113:0/4222929483 >> v2:172.21.15.178:3300/0 conn(0x7fd1e00b61e0 0x7fd1e0071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.670+0000 7fd1de59c640 1 --2- 172.21.15.113:0/4222929483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e015e510 0x7fd1e00d2380 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00d1d80 tx=0x7fd1d002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f042c1d81c815905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.670+0000 7fd1ddd9b640 1 -- 172.21.15.113:0/4222929483 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0037070 con 0x7fd1e015e510 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.670+0000 7fd1ddd9b640 1 -- 172.21.15.113:0/4222929483 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d0032440 con 0x7fd1e015e510 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.670+0000 7fd1ddd9b640 1 -- 172.21.15.113:0/4222929483 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0032810 con 0x7fd1e015e510 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.671+0000 7fd1e4d34640 1 -- 172.21.15.113:0/4222929483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e015e510 msgr2=0x7fd1e00d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.671+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/4222929483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e015e510 0x7fd1e00d2380 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00d1d80 tx=0x7fd1d002a140 comp rx=0 tx=0).stop 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.671+0000 7fd1e4d34640 1 -- 172.21.15.113:0/4222929483 shutdown_connections 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.671+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/4222929483 >> v2:172.21.15.178:3300/0 conn(0x7fd1e00b61e0 0x7fd1e0071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.671+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/4222929483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e00d28c0 0x7fd1e00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.671+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/4222929483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e015e510 0x7fd1e00d2380 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.671+0000 7fd1e4d34640 1 -- 172.21.15.113:0/4222929483 >> 172.21.15.113:0/4222929483 conn(0x7fd1e006b720 msgr2=0x7fd1e00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.672+0000 7fd1e4d34640 1 -- 172.21.15.113:0/4222929483 shutdown_connections 2024-04-25T00:04:45.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.672+0000 7fd1e4d34640 1 -- 172.21.15.113:0/4222929483 wait complete. 2024-04-25T00:04:45.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.673+0000 7fd1e4d34640 1 Processor -- start 2024-04-25T00:04:45.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.673+0000 7fd1e4d34640 1 -- start start 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1e4d34640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e00b61e0 0x7fd1e00ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1e4d34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e00d28c0 0x7fd1e0144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1e4d34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e015e510 0x7fd1e01f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1e4d34640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00b9dd0 con 0x7fd1e00b61e0 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1e4d34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00d3ea0 con 0x7fd1e00d28c0 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1e4d34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e0071c50 con 0x7fd1e015e510 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1de59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e00b61e0 0x7fd1e00ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1de59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e00b61e0 0x7fd1e00ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49888/0 (socket says 172.21.15.113:49888) 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1de59c640 1 -- 172.21.15.113:0/2786277463 learned_addr learned my addr 172.21.15.113:0/2786277463 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1d7fff640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e00d28c0 0x7fd1e0144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.674+0000 7fd1ded9d640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e015e510 0x7fd1e01f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1de59c640 1 -- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e015e510 msgr2=0x7fd1e01f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1de59c640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e015e510 0x7fd1e01f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1de59c640 1 -- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e00d28c0 msgr2=0x7fd1e0144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1de59c640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e00d28c0 0x7fd1e0144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1de59c640 1 -- 172.21.15.113:0/2786277463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e00cd080 con 0x7fd1e00b61e0 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1de59c640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e00b61e0 0x7fd1e00ca5f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0000c00 tx=0x7fd1d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1d6ffd640 1 -- 172.21.15.113:0/2786277463 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0041070 con 0x7fd1e00b61e0 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1d6ffd640 1 -- 172.21.15.113:0/2786277463 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d003c870 con 0x7fd1e00b61e0 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1d6ffd640 1 -- 172.21.15.113:0/2786277463 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0037040 con 0x7fd1e00b61e0 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e01f4860 con 0x7fd1e00b61e0 2024-04-25T00:04:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.675+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e00cd300 con 0x7fd1e00b61e0 2024-04-25T00:04:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.676+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e00691a0 con 0x7fd1e00b61e0 2024-04-25T00:04:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.677+0000 7fd1d6ffd640 1 -- 172.21.15.113:0/2786277463 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1d0044070 con 0x7fd1e00b61e0 2024-04-25T00:04:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.677+0000 7fd1d6ffd640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1c006be30 0x7fd1c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.677+0000 7fd1d6ffd640 1 -- 172.21.15.113:0/2786277463 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4943+0+0 (secure 0 0 0) 0x7fd1d00ab6d0 con 0x7fd1e00b61e0 2024-04-25T00:04:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.678+0000 7fd1d7fff640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1c006be30 0x7fd1c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.678+0000 7fd1d7fff640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1c006be30 0x7fd1c006e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00d13f0 tx=0x7fd1c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.682+0000 7fd1d6ffd640 1 -- 172.21.15.113:0/2786277463 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1e00cd080 con 0x7fd1e00b61e0 2024-04-25T00:04:45.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:45.864+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd1e00b9020 con 0x7fd1e00b61e0 2024-04-25T00:04:46.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.672+0000 7fd1d6ffd640 1 -- 172.21.15.113:0/2786277463 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) ==== 97+0+0 (secure 0 0 0) 0x7fd1d0079ee0 con 0x7fd1e00b61e0 2024-04-25T00:04:46.673 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:04:46.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.675+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1c006be30 msgr2=0x7fd1c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:46.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.675+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1c006be30 0x7fd1c006e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00d13f0 tx=0x7fd1c8008040 comp rx=0 tx=0).stop 2024-04-25T00:04:46.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.675+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e00b61e0 msgr2=0x7fd1e00ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:46.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.675+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e00b61e0 0x7fd1e00ca5f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0000c00 tx=0x7fd1d00046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.675+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 shutdown_connections 2024-04-25T00:04:46.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.675+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1c006be30 0x7fd1c006e2f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.676+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e00d28c0 0x7fd1e0144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.676+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1e015e510 0x7fd1e01f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.676+0000 7fd1e4d34640 1 --2- 172.21.15.113:0/2786277463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1e00b61e0 0x7fd1e00ca5f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.676+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 >> 172.21.15.113:0/2786277463 conn(0x7fd1e006b720 msgr2=0x7fd1e015c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:46.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.676+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 shutdown_connections 2024-04-25T00:04:46.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.676+0000 7fd1e4d34640 1 -- 172.21.15.113:0/2786277463 wait complete. 2024-04-25T00:04:46.689 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:04:46.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.789+0000 7fd131b70640 1 Processor -- start 2024-04-25T00:04:46.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.789+0000 7fd131b70640 1 -- start start 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.789+0000 7fd131b70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c06e6e0 0x7fd12c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.789+0000 7fd131b70640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd12c1453c0 0x7fd12c071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.789+0000 7fd131b70640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd12c0cfad0 con 0x7fd12c144d60 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.789+0000 7fd131b70640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd12c145c40 con 0x7fd12c1453c0 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.789+0000 7fd131b70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c0cf230 con 0x7fd12c06e6e0 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd12c1453c0 0x7fd12c071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd12c1453c0 0x7fd12c071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49890/0 (socket says 172.21.15.113:49890) 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12bfff640 1 -- 172.21.15.113:0/2828848409 learned_addr learned my addr 172.21.15.113:0/2828848409 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12affd640 1 --2- 172.21.15.113:0/2828848409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c06e6e0 0x7fd12c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:46.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12bfff640 1 -- 172.21.15.113:0/2828848409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c06e6e0 msgr2=0x7fd12c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12bfff640 1 --2- 172.21.15.113:0/2828848409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c06e6e0 0x7fd12c0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12bfff640 1 -- 172.21.15.113:0/2828848409 >> v1:172.21.15.178:6789/0 conn(0x7fd12c144d60 legacy=0x7fd12c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.790+0000 7fd12bfff640 1 -- 172.21.15.113:0/2828848409 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c0cf510 con 0x7fd12c1453c0 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd12bfff640 1 --2- 172.21.15.113:0/2828848409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c1453c0 0x7fd12c071330 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd124009e50 tx=0x7fd12402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d3d93b376695be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd12a7fc640 1 -- 172.21.15.113:0/2828848409 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124036070 con 0x7fd12c1453c0 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd12a7fc640 1 -- 172.21.15.113:0/2828848409 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd124031600 con 0x7fd12c1453c0 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd12a7fc640 1 -- 172.21.15.113:0/2828848409 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240319d0 con 0x7fd12c1453c0 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 -- 172.21.15.113:0/2828848409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c1453c0 msgr2=0x7fd12c071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 --2- 172.21.15.113:0/2828848409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c1453c0 0x7fd12c071330 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd124009e50 tx=0x7fd12402a630 comp rx=0 tx=0).stop 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 -- 172.21.15.113:0/2828848409 shutdown_connections 2024-04-25T00:04:46.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 --2- 172.21.15.113:0/2828848409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c06e6e0 0x7fd12c0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 --2- 172.21.15.113:0/2828848409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c1453c0 0x7fd12c071330 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 -- 172.21.15.113:0/2828848409 >> 172.21.15.113:0/2828848409 conn(0x7fd12c0b1a50 msgr2=0x7fd12c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:46.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 -- 172.21.15.113:0/2828848409 shutdown_connections 2024-04-25T00:04:46.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.791+0000 7fd131b70640 1 -- 172.21.15.113:0/2828848409 wait complete. 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.792+0000 7fd131b70640 1 Processor -- start 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.793+0000 7fd131b70640 1 -- start start 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.793+0000 7fd131b70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c06e6e0 0x7fd12c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.793+0000 7fd131b70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c144d60 0x7fd12c143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.793+0000 7fd131b70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12c1453c0 0x7fd12c13bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.793+0000 7fd131b70640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c145140 con 0x7fd12c06e6e0 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.793+0000 7fd131b70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c070e10 con 0x7fd12c1453c0 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.793+0000 7fd131b70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c071870 con 0x7fd12c144d60 2024-04-25T00:04:46.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c06e6e0 0x7fd12c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c06e6e0 0x7fd12c15b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49900/0 (socket says 172.21.15.113:49900) 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 -- 172.21.15.113:0/454608533 learned_addr learned my addr 172.21.15.113:0/454608533 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12bfff640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12c1453c0 0x7fd12c13bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12affd640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c144d60 0x7fd12c143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 -- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c144d60 msgr2=0x7fd12c143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c144d60 0x7fd12c143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 -- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12c1453c0 msgr2=0x7fd12c13bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12c1453c0 0x7fd12c13bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 -- 172.21.15.113:0/454608533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c13cc20 con 0x7fd12c06e6e0 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12bfff640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12c1453c0 0x7fd12c13bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12affd640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c144d60 0x7fd12c143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd12b7fe640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c06e6e0 0x7fd12c15b780 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c430 tx=0x7fd12000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:46.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd113fff640 1 -- 172.21.15.113:0/454608533 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120011790 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd113fff640 1 -- 172.21.15.113:0/454608533 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd120017070 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c1e6010 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd113fff640 1 -- 172.21.15.113:0/454608533 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120011bb0 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.794+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c1e65d0 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.795+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12c0691a0 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.796+0000 7fd113fff640 1 -- 172.21.15.113:0/454608533 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd12001c460 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.796+0000 7fd113fff640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd11806be60 0x7fd11806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.796+0000 7fd113fff640 1 -- 172.21.15.113:0/454608533 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7fd12008a970 con 0x7fd12c06e6e0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.796+0000 7fd12affd640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd11806be60 0x7fd11806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.796+0000 7fd12affd640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd11806be60 0x7fd11806e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd12c068760 tx=0x7fd11c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:46.800+0000 7fd113fff640 1 -- 172.21.15.113:0/454608533 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd120059680 con 0x7fd12c06e6e0 2024-04-25T00:04:47.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.037+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd12c0afac0 con 0x7fd12c06e6e0 2024-04-25T00:04:47.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.038+0000 7fd113fff640 1 -- 172.21.15.113:0/454608533 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fd120058f80 con 0x7fd12c06e6e0 2024-04-25T00:04:47.038 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.039 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.040 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.041 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.042 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.044 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.045 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.046 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:04:47.047 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:04:47.048 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:04:47.049 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:04:47.050 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.051 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:04:47.052 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:04:47.053 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.054 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.055 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:04:47.056 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:04:47.057 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:47.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd11806be60 msgr2=0x7fd11806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd11806be60 0x7fd11806e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd12c068760 tx=0x7fd11c008040 comp rx=0 tx=0).stop 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c06e6e0 msgr2=0x7fd12c15b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c06e6e0 0x7fd12c15b780 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c430 tx=0x7fd12000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 shutdown_connections 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd11806be60 0x7fd11806e320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12c1453c0 0x7fd12c13bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c144d60 0x7fd12c143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 --2- 172.21.15.113:0/454608533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd12c06e6e0 0x7fd12c15b780 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 >> 172.21.15.113:0/454608533 conn(0x7fd12c0b1a50 msgr2=0x7fd12c071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 shutdown_connections 2024-04-25T00:04:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.040+0000 7fd131b70640 1 -- 172.21.15.113:0/454608533 wait complete. 2024-04-25T00:04:47.061 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:47.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.158+0000 7fa8a8c9a640 1 Processor -- start 2024-04-25T00:04:47.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.158+0000 7fa8a8c9a640 1 -- start start 2024-04-25T00:04:47.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a8c9a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa8a4144d60 0x7fa8a40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a8c9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a41453c0 0x7fa8a406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a8c9a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8a406e510 0x7fa8a415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a8c9a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a4145c40 con 0x7fa8a406e510 2024-04-25T00:04:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a8c9a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a406e940 con 0x7fa8a4144d60 2024-04-25T00:04:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a8c9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a40cfad0 con 0x7fa8a41453c0 2024-04-25T00:04:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8a406e510 0x7fa8a415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8a406e510 0x7fa8a415fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49912/0 (socket says 172.21.15.113:49912) 2024-04-25T00:04:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 -- 172.21.15.113:0/3457960354 learned_addr learned my addr 172.21.15.113:0/3457960354 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 -- 172.21.15.113:0/3457960354 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a41453c0 msgr2=0x7fa8a406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 --2- 172.21.15.113:0/3457960354 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a41453c0 0x7fa8a406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a259c640 1 --2- 172.21.15.113:0/3457960354 >> v2:172.21.15.178:3300/0 conn(0x7fa8a4144d60 0x7fa8a40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 -- 172.21.15.113:0/3457960354 >> v2:172.21.15.178:3300/0 conn(0x7fa8a4144d60 msgr2=0x7fa8a40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 --2- 172.21.15.113:0/3457960354 >> v2:172.21.15.178:3300/0 conn(0x7fa8a4144d60 0x7fa8a40b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.159+0000 7fa8a2d9d640 1 -- 172.21.15.113:0/3457960354 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a40cf230 con 0x7fa8a406e510 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.160+0000 7fa8a2d9d640 1 --2- 172.21.15.113:0/3457960354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a406e510 0x7fa8a415fc30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa8a406f970 tx=0x7fa89c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fe44dfa4c00654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.160+0000 7fa8a159a640 1 -- 172.21.15.113:0/3457960354 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c037070 con 0x7fa8a406e510 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.160+0000 7fa8a159a640 1 -- 172.21.15.113:0/3457960354 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa89c032440 con 0x7fa8a406e510 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.160+0000 7fa8a159a640 1 -- 172.21.15.113:0/3457960354 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c032810 con 0x7fa8a406e510 2024-04-25T00:04:47.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/3457960354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a406e510 msgr2=0x7fa8a415fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/3457960354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a406e510 0x7fa8a415fc30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa8a406f970 tx=0x7fa89c02a180 comp rx=0 tx=0).stop 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/3457960354 shutdown_connections 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/3457960354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a406e510 0x7fa8a415fc30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/3457960354 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a41453c0 0x7fa8a406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/3457960354 >> v2:172.21.15.178:3300/0 conn(0x7fa8a4144d60 0x7fa8a40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/3457960354 >> 172.21.15.113:0/3457960354 conn(0x7fa8a40b1a50 msgr2=0x7fa8a40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/3457960354 shutdown_connections 2024-04-25T00:04:47.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.161+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/3457960354 wait complete. 2024-04-25T00:04:47.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.162+0000 7fa8a8c9a640 1 Processor -- start 2024-04-25T00:04:47.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.162+0000 7fa8a8c9a640 1 -- start start 2024-04-25T00:04:47.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a8c9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a406e510 0x7fa8a415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a8c9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4144d60 0x7fa8a413c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a8c9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8a41453c0 0x7fa8a41433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a8c9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a4070e10 con 0x7fa8a4144d60 2024-04-25T00:04:47.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a8c9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a40cf7f0 con 0x7fa8a41453c0 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a8c9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a4145140 con 0x7fa8a406e510 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4144d60 0x7fa8a413c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4144d60 0x7fa8a413c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49924/0 (socket says 172.21.15.113:49924) 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 -- 172.21.15.113:0/2335906712 learned_addr learned my addr 172.21.15.113:0/2335906712 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 -- 172.21.15.113:0/2335906712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a406e510 msgr2=0x7fa8a415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a406e510 0x7fa8a415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 -- 172.21.15.113:0/2335906712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8a41453c0 msgr2=0x7fa8a41433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8a41453c0 0x7fa8a41433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 -- 172.21.15.113:0/2335906712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a40addc0 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa8a1d9b640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4144d60 0x7fa8a413c790 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b4d0 tx=0x7fa89800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.163+0000 7fa892ffd640 1 -- 172.21.15.113:0/2335906712 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898014070 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.164+0000 7fa892ffd640 1 -- 172.21.15.113:0/2335906712 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa898010610 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.164+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a40ae0a0 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.164+0000 7fa892ffd640 1 -- 172.21.15.113:0/2335906712 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898010a00 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.164+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a40ae660 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.165+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a40691a0 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.165+0000 7fa892ffd640 1 -- 172.21.15.113:0/2335906712 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa898017070 con 0x7fa8a4144d60 2024-04-25T00:04:47.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.165+0000 7fa892ffd640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa88406be30 0x7fa88406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.166+0000 7fa8a259c640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa88406be30 0x7fa88406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.166+0000 7fa892ffd640 1 -- 172.21.15.113:0/2335906712 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7fa89808a470 con 0x7fa8a4144d60 2024-04-25T00:04:47.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.166+0000 7fa8a259c640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa88406be30 0x7fa88406e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa894007dc0 tx=0x7fa8940072f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.170+0000 7fa892ffd640 1 -- 172.21.15.113:0/2335906712 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa898059200 con 0x7fa8a4144d60 2024-04-25T00:04:47.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.346+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8a4140220 con 0x7fa8a4144d60 2024-04-25T00:04:47.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.347+0000 7fa892ffd640 1 -- 172.21.15.113:0/2335906712 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+12941 (secure 0 0 0) 0x7fa898058b00 con 0x7fa8a4144d60 2024-04-25T00:04:47.347 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:47.347 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:46.672164+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.348+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa88406be30 msgr2=0x7fa88406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.348+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa88406be30 0x7fa88406e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa894007dc0 tx=0x7fa8940072f0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.348+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4144d60 msgr2=0x7fa8a413c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.348+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4144d60 0x7fa8a413c790 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b4d0 tx=0x7fa89800b990 comp rx=0 tx=0).stop 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 shutdown_connections 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa88406be30 0x7fa88406e2f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8a41453c0 0x7fa8a41433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4144d60 0x7fa8a413c790 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 --2- 172.21.15.113:0/2335906712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8a406e510 0x7fa8a415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 >> 172.21.15.113:0/2335906712 conn(0x7fa8a40b1a50 msgr2=0x7fa8a413f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:47.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 shutdown_connections 2024-04-25T00:04:47.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.349+0000 7fa8a8c9a640 1 -- 172.21.15.113:0/2335906712 wait complete. 2024-04-25T00:04:47.360 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:04:47.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:04:47.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.459+0000 7f56c909e640 1 Processor -- start 2024-04-25T00:04:47.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.459+0000 7f56c909e640 1 -- start start 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c909e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 0x7f56c415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c909e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c4071170 con 0x7f56c406cea0 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c909e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c4071450 con 0x7f56c415bff0 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c909e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56c415c470 con 0x7f56c415b670 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c359e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f56c406cea0 0x7f56c4072210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37220/0 (socket says 172.21.15.113:37220) 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c359e640 1 -- 172.21.15.113:0/3705491964 learned_addr learned my addr 172.21.15.113:0/3705491964 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c2d9d640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 0x7f56c415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383628733 0 0) 0x7f56c4071170 con 0x7f56c406cea0 2024-04-25T00:04:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56ac003520 con 0x7f56c406cea0 2024-04-25T00:04:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 13359871 0 0) 0x7f56ac003520 con 0x7f56c406cea0 2024-04-25T00:04:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 msgr2=0x7f56c415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 0x7f56c415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 >> v1:172.21.15.178:6789/0 conn(0x7f56c415bff0 legacy=0x7f56c40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c4071730 con 0x7f56c406cea0 2024-04-25T00:04:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.460+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f56bc001670 con 0x7f56c406cea0 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 >> v1:172.21.15.113:6789/0 conn(0x7f56c406cea0 legacy=0x7f56c4072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c1d9b640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 0x7f56ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c1d9b640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ac003f20 0x7f56ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c1d9b640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac004d40 0x7f56ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56c415c470 con 0x7f56ac004d40 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac00bb80 con 0x7f56ac003f20 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56ac00bd80 con 0x7f56c415b670 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c359e640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac004d40 0x7f56ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c2d9d640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 0x7f56ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c259c640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ac003f20 0x7f56ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c359e640 1 -- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 msgr2=0x7f56ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:47.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c359e640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c415b670 0x7f56ac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c359e640 1 -- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ac003f20 msgr2=0x7f56ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:47.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c359e640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ac003f20 0x7f56ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.461+0000 7f56c359e640 1 -- 172.21.15.113:0/3705491964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac003520 con 0x7f56ac004d40 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c359e640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac004d40 0x7f56ac0051c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c240 tx=0x7f56ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b47ae94f126dca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007d30 con 0x7f56ac004d40 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc005af0 con 0x7f56ac004d40 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c1d9b640 1 -- 172.21.15.113:0/3705491964 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007070 con 0x7f56ac004d40 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c909e640 1 -- 172.21.15.113:0/3705491964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac004d40 msgr2=0x7f56ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c909e640 1 --2- 172.21.15.113:0/3705491964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56ac004d40 0x7f56ac0051c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c240 tx=0x7f56ac005830 comp rx=0 tx=0).stop 2024-04-25T00:04:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c359e640 1 -- 172.21.15.113:0/3705491964 reap_dead start 2024-04-25T00:04:47.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c909e640 1 -- 172.21.15.113:0/3705491964 shutdown_connections 2024-04-25T00:04:47.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c909e640 1 -- 172.21.15.113:0/3705491964 >> 172.21.15.113:0/3705491964 conn(0x7f56c406b0f0 msgr2=0x7f56c406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:47.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c909e640 1 -- 172.21.15.113:0/3705491964 shutdown_connections 2024-04-25T00:04:47.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.462+0000 7f56c909e640 1 -- 172.21.15.113:0/3705491964 wait complete. 2024-04-25T00:04:47.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.464+0000 7f56c909e640 1 Processor -- start 2024-04-25T00:04:47.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.464+0000 7f56c909e640 1 -- start start 2024-04-25T00:04:47.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c909e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c4153220 0x7f56c40c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c909e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c4153a30 0x7f56c41e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c909e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56c41e24a0 0x7f56c41f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c909e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40cca90 con 0x7f56c41e24a0 2024-04-25T00:04:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c909e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40cc910 con 0x7f56c4153220 2024-04-25T00:04:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c909e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56c415ba50 con 0x7f56c4153a30 2024-04-25T00:04:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56c41e24a0 0x7f56c41f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56c41e24a0 0x7f56c41f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47768/0 (socket says 172.21.15.113:47768) 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 -- 172.21.15.113:0/1116833995 learned_addr learned my addr 172.21.15.113:0/1116833995 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c2d9d640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c4153220 0x7f56c40c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c259c640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c4153a30 0x7f56c41e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 -- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c4153a30 msgr2=0x7f56c41e1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c4153a30 0x7f56c41e1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 -- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c4153220 msgr2=0x7f56c40c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c4153220 0x7f56c40c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 -- 172.21.15.113:0/1116833995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c40c86a0 con 0x7f56c41e24a0 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c359e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56c41e24a0 0x7f56c41f4ec0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f56bc02ae20 tx=0x7f56c415ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1116833995 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007d30 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1116833995 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc043070 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c40c8980 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.465+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1116833995 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0076c0 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.466+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c40c8380 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.466+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c40691a0 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.467+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1116833995 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56bc03e470 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.467+0000 7f56ab7fe640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56b006be60 0x7f56b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.467+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1116833995 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7f56bc0aae90 con 0x7f56c41e24a0 2024-04-25T00:04:47.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.467+0000 7f56c2d9d640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56b006be60 0x7f56b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.468+0000 7f56c2d9d640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56b006be60 0x7f56b006e320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f56b4014510 tx=0x7f56b400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.472+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1116833995 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56bc0796f0 con 0x7f56c41e24a0 2024-04-25T00:04:47.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.644+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56c40ca260 con 0x7f56c41e24a0 2024-04-25T00:04:47.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.644+0000 7f56ab7fe640 1 -- 172.21.15.113:0/1116833995 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f56bc07d9f0 con 0x7f56c41e24a0 2024-04-25T00:04:47.644 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:04:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56b006be60 msgr2=0x7f56b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56b006be60 0x7f56b006e320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f56b4014510 tx=0x7f56b400b040 comp rx=0 tx=0).stop 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56c41e24a0 msgr2=0x7f56c41f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56c41e24a0 0x7f56c41f4ec0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f56bc02ae20 tx=0x7f56c415ae30 comp rx=0 tx=0).stop 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 shutdown_connections 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f56b006be60 0x7f56b006e320 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56c41e24a0 0x7f56c41f4ec0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56c4153a30 0x7f56c41e1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 --2- 172.21.15.113:0/1116833995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c4153220 0x7f56c40c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 >> 172.21.15.113:0/1116833995 conn(0x7f56c406b0f0 msgr2=0x7f56c40cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 shutdown_connections 2024-04-25T00:04:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.646+0000 7f56c909e640 1 -- 172.21.15.113:0/1116833995 wait complete. 2024-04-25T00:04:47.657 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:04:47.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.754+0000 7fca348b3640 1 Processor -- start 2024-04-25T00:04:47.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.754+0000 7fca348b3640 1 -- start start 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca348b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca300b3fd0 0x7fca3006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca348b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca348b3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca30144cf0 con 0x7fca30144910 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca348b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fca300b2f30 con 0x7fca300b3fd0 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca348b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca300b3210 con 0x7fca3006e510 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca2ed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca27fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca300b3fd0 0x7fca3006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca2ed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:46462/0 (socket says 172.21.15.113:46462) 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca2ed9d640 1 -- 172.21.15.113:0/2409510583 learned_addr learned my addr 172.21.15.113:0/2409510583 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca2ed9d640 1 -- 172.21.15.113:0/2409510583 >> v2:172.21.15.178:3300/0 conn(0x7fca300b3fd0 msgr2=0x7fca3006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:47.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca2ed9d640 1 --2- 172.21.15.113:0/2409510583 >> v2:172.21.15.178:3300/0 conn(0x7fca300b3fd0 0x7fca3006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca2ed9d640 1 -- 172.21.15.113:0/2409510583 >> v1:172.21.15.178:6789/0 conn(0x7fca30144910 legacy=0x7fca300b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca2ed9d640 1 -- 172.21.15.113:0/2409510583 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca300b34f0 con 0x7fca3006e510 2024-04-25T00:04:47.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.755+0000 7fca27fff640 1 --2- 172.21.15.113:0/2409510583 >> v2:172.21.15.178:3300/0 conn(0x7fca300b3fd0 0x7fca3006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.756+0000 7fca2ed9d640 1 --2- 172.21.15.113:0/2409510583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300cd500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fca28009e80 tx=0x7fca2802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2f06d10c18aeca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.756+0000 7fca2dd9b640 1 -- 172.21.15.113:0/2409510583 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28036070 con 0x7fca3006e510 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.756+0000 7fca2dd9b640 1 -- 172.21.15.113:0/2409510583 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca28031600 con 0x7fca3006e510 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.756+0000 7fca2dd9b640 1 -- 172.21.15.113:0/2409510583 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280319d0 con 0x7fca3006e510 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 -- 172.21.15.113:0/2409510583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 msgr2=0x7fca300cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 --2- 172.21.15.113:0/2409510583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300cd500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fca28009e80 tx=0x7fca2802a750 comp rx=0 tx=0).stop 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 -- 172.21.15.113:0/2409510583 shutdown_connections 2024-04-25T00:04:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 --2- 172.21.15.113:0/2409510583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300cd500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 --2- 172.21.15.113:0/2409510583 >> v2:172.21.15.178:3300/0 conn(0x7fca300b3fd0 0x7fca3006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 -- 172.21.15.113:0/2409510583 >> 172.21.15.113:0/2409510583 conn(0x7fca300b1a50 msgr2=0x7fca300b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 -- 172.21.15.113:0/2409510583 shutdown_connections 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.757+0000 7fca348b3640 1 -- 172.21.15.113:0/2409510583 wait complete. 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.758+0000 7fca348b3640 1 Processor -- start 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.758+0000 7fca348b3640 1 -- start start 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca348b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca348b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca300b3fd0 0x7fca3015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca348b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30144910 0x7fca301f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca348b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fca300b4da0 con 0x7fca300b3fd0 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca348b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca300b4440 con 0x7fca30144910 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca348b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca300b4f20 con 0x7fca3006e510 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca300b3fd0 0x7fca3015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca300b3fd0 0x7fca3015b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47778/0 (socket says 172.21.15.113:47778) 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 -- 172.21.15.113:0/1812485080 learned_addr learned my addr 172.21.15.113:0/1812485080 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca2e59c640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca2ed9d640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30144910 0x7fca301f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 -- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 msgr2=0x7fca300aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 -- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30144910 msgr2=0x7fca301f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30144910 0x7fca301f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 -- 172.21.15.113:0/1812485080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3015c060 con 0x7fca300b3fd0 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.759+0000 7fca27fff640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca300b3fd0 0x7fca3015b5c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fca1800ebe0 tx=0x7fca1800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.760+0000 7fca26ffd640 1 -- 172.21.15.113:0/1812485080 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18017070 con 0x7fca300b3fd0 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.760+0000 7fca26ffd640 1 -- 172.21.15.113:0/1812485080 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1800cc30 con 0x7fca300b3fd0 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.760+0000 7fca26ffd640 1 -- 172.21.15.113:0/1812485080 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18013470 con 0x7fca300b3fd0 2024-04-25T00:04:47.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.760+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3015c340 con 0x7fca300b3fd0 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.760+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3015c900 con 0x7fca300b3fd0 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.761+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca300691a0 con 0x7fca300b3fd0 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.761+0000 7fca26ffd640 1 -- 172.21.15.113:0/1812485080 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca1801a070 con 0x7fca300b3fd0 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.762+0000 7fca26ffd640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca1006be30 0x7fca1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.762+0000 7fca26ffd640 1 -- 172.21.15.113:0/1812485080 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7fca1808ad50 con 0x7fca300b3fd0 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.762+0000 7fca2e59c640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca1006be30 0x7fca1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.762+0000 7fca2e59c640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca1006be30 0x7fca1006e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca3014d950 tx=0x7fca20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:47.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.766+0000 7fca26ffd640 1 -- 172.21.15.113:0/1812485080 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fca1808e030 con 0x7fca300b3fd0 2024-04-25T00:04:47.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.939+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fca301f4e70 con 0x7fca300b3fd0 2024-04-25T00:04:47.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.940+0000 7fca26ffd640 1 -- 172.21.15.113:0/1812485080 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) ==== 84+0+65 (secure 0 0 0) 0x7fca18059400 con 0x7fca300b3fd0 2024-04-25T00:04:47.940 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:47.940 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:04:47.941 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:04:47.941 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:04:47.941 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:04:47.941 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:04:47.941 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:04:47.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca1006be30 msgr2=0x7fca1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca1006be30 0x7fca1006e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca3014d950 tx=0x7fca20009040 comp rx=0 tx=0).stop 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca300b3fd0 msgr2=0x7fca3015b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca300b3fd0 0x7fca3015b5c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fca1800ebe0 tx=0x7fca1800c520 comp rx=0 tx=0).stop 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 shutdown_connections 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca1006be30 0x7fca1006e2f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca30144910 0x7fca301f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca300b3fd0 0x7fca3015b5c0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 --2- 172.21.15.113:0/1812485080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca3006e510 0x7fca300aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 >> 172.21.15.113:0/1812485080 conn(0x7fca300b1a50 msgr2=0x7fca300cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 shutdown_connections 2024-04-25T00:04:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:47.942+0000 7fca348b3640 1 -- 172.21.15.113:0/1812485080 wait complete. 2024-04-25T00:04:47.953 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:04:47.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:04:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.053+0000 7f4ed6c7d640 1 Processor -- start 2024-04-25T00:04:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.053+0000 7f4ed6c7d640 1 -- start start 2024-04-25T00:04:48.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.054+0000 7f4ed6c7d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4ed015e510 0x7f4ed0161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.054+0000 7f4ed6c7d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed01626f0 con 0x7f4ed00b9bb0 2024-04-25T00:04:48.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.054+0000 7f4ed6c7d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed01629d0 con 0x7f4ed00b63b0 2024-04-25T00:04:48.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.054+0000 7f4ed6c7d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed0162cb0 con 0x7f4ed015e510 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.054+0000 7f4ecffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4ed00b9bb0 0x7f4ed00b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38172/0 (socket says 172.21.15.113:38172) 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.054+0000 7f4ecffff640 1 -- 172.21.15.113:0/4093528584 learned_addr learned my addr 172.21.15.113:0/4093528584 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ed4a19640 1 --2- 172.21.15.113:0/4093528584 >> v2:172.21.15.178:3300/0 conn(0x7f4ed015e510 0x7f4ed0161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 289749528 0 0) 0x7f4ed01626f0 con 0x7f4ed00b9bb0 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb8003520 con 0x7f4ed00b9bb0 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 579445935 0 0) 0x7f4eb8003520 con 0x7f4ed00b9bb0 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 >> v2:172.21.15.178:3300/0 conn(0x7f4ed015e510 msgr2=0x7f4ed0161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 --2- 172.21.15.113:0/4093528584 >> v2:172.21.15.178:3300/0 conn(0x7f4ed015e510 0x7f4ed0161ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 >> v1:172.21.15.178:6789/0 conn(0x7f4ed00b63b0 legacy=0x7f4ed00d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed00d2660 con 0x7f4ed00b9bb0 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f4ec4001670 con 0x7f4ed00b9bb0 2024-04-25T00:04:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 >> v1:172.21.15.113:6789/0 conn(0x7f4ed00b9bb0 legacy=0x7f4ed00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eb80039b0 0x7f4eb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb80043b0 0x7f4eb8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb8005120 0x7f4eb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0162cb0 con 0x7f4eb80043b0 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb800c060 con 0x7f4eb8005120 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb800c260 con 0x7f4eb80039b0 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecffff640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb80043b0 0x7f4eb8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ed4a19640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eb80039b0 0x7f4eb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ed521a640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb8005120 0x7f4eb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecffff640 1 -- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eb80039b0 msgr2=0x7f4eb8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecffff640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eb80039b0 0x7f4eb8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecffff640 1 -- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb8005120 msgr2=0x7f4eb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecffff640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb8005120 0x7f4eb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ed521a640 1 -- 172.21.15.113:0/4093528584 reap_dead start 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.055+0000 7f4ecffff640 1 -- 172.21.15.113:0/4093528584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb8003520 con 0x7f4eb80043b0 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.056+0000 7f4ecffff640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb80043b0 0x7f4eb8004880 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c210 tx=0x7f4eb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73b4666cc2bd7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.056+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4007d10 con 0x7f4eb80043b0 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.056+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec4005ad0 con 0x7f4eb80043b0 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.056+0000 7f4ecf7fe640 1 -- 172.21.15.113:0/4093528584 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4007070 con 0x7f4eb80043b0 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.057+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/4093528584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb80043b0 msgr2=0x7f4eb8004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.057+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb80043b0 0x7f4eb8004880 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c210 tx=0x7f4eb8005830 comp rx=0 tx=0).stop 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.057+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/4093528584 shutdown_connections 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.057+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/4093528584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4eb80043b0 0x7f4eb8004880 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.057+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/4093528584 >> 172.21.15.113:0/4093528584 conn(0x7f4ed006b720 msgr2=0x7f4ed00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.057+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/4093528584 shutdown_connections 2024-04-25T00:04:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.057+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/4093528584 wait complete. 2024-04-25T00:04:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.058+0000 7f4ed6c7d640 1 Processor -- start 2024-04-25T00:04:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.058+0000 7f4ed6c7d640 1 -- start start 2024-04-25T00:04:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed6c7d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb80043b0 0x7f4ed00ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed6c7d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed00cd2b0 0x7f4ed00ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed6c7d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed00cf400 0x7f4ed00cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed6c7d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00b7540 con 0x7f4ed00cf400 2024-04-25T00:04:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed6c7d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00b73c0 con 0x7f4eb80043b0 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed6c7d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed00b5ca0 con 0x7f4ed00cd2b0 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed00cf400 0x7f4ed00cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed00cf400 0x7f4ed00cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47788/0 (socket says 172.21.15.113:47788) 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 -- 172.21.15.113:0/3303287233 learned_addr learned my addr 172.21.15.113:0/3303287233 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ecffff640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed00cd2b0 0x7f4ed00ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed4a19640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb80043b0 0x7f4ed00ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 -- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed00cd2b0 msgr2=0x7f4ed00ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed00cd2b0 0x7f4ed00ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 -- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb80043b0 msgr2=0x7f4ed00ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb80043b0 0x7f4ed00ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 -- 172.21.15.113:0/3303287233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed00c8800 con 0x7f4ed00cf400 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ed521a640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed00cf400 0x7f4ed00cb2f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00728a0 tx=0x7f4ec800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.059+0000 7f4ecd7fa640 1 -- 172.21.15.113:0/3303287233 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8019070 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.060+0000 7f4ecd7fa640 1 -- 172.21.15.113:0/3303287233 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec8006d60 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.060+0000 7f4ecd7fa640 1 -- 172.21.15.113:0/3303287233 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8003b90 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.060+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed00c8aa0 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.060+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed00c9060 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.061+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed00691a0 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.061+0000 7f4ecd7fa640 1 -- 172.21.15.113:0/3303287233 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ec800d6b0 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.062+0000 7f4ecd7fa640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4ebc06be60 0x7f4ebc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.062+0000 7f4ed4a19640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4ebc06be60 0x7f4ebc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.062+0000 7f4ecd7fa640 1 -- 172.21.15.113:0/3303287233 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4857+0+0 (secure 0 0 0) 0x7f4ec8089b80 con 0x7f4ed00cf400 2024-04-25T00:04:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.062+0000 7f4ed4a19640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4ebc06be60 0x7f4ebc06e320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009bc0 tx=0x7f4ec0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:48.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.066+0000 7f4ecd7fa640 1 -- 172.21.15.113:0/3303287233 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ec8058230 con 0x7f4ed00cf400 2024-04-25T00:04:48.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.240+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4ed01563b0 con 0x7f4ed00cf400 2024-04-25T00:04:48.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.682+0000 7f4ecd7fa640 1 -- 172.21.15.113:0/3303287233 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) ==== 138+0+0 (secure 0 0 0) 0x7f4ec805c530 con 0x7f4ed00cf400 2024-04-25T00:04:48.682 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:04:48.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.686+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4ebc06be60 msgr2=0x7f4ebc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.686+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4ebc06be60 0x7f4ebc06e320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009bc0 tx=0x7f4ec0009190 comp rx=0 tx=0).stop 2024-04-25T00:04:48.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.686+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed00cf400 msgr2=0x7f4ed00cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.686+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed00cf400 0x7f4ed00cb2f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00728a0 tx=0x7f4ec800fc80 comp rx=0 tx=0).stop 2024-04-25T00:04:48.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.687+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 shutdown_connections 2024-04-25T00:04:48.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.687+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4ebc06be60 0x7f4ebc06e320 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.687+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed00cf400 0x7f4ed00cb2f0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.687+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed00cd2b0 0x7f4ed00ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.687+0000 7f4ed6c7d640 1 --2- 172.21.15.113:0/3303287233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eb80043b0 0x7f4ed00ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.687+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 >> 172.21.15.113:0/3303287233 conn(0x7f4ed006b720 msgr2=0x7f4ed00b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:48.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.687+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 shutdown_connections 2024-04-25T00:04:48.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.688+0000 7f4ed6c7d640 1 -- 172.21.15.113:0/3303287233 wait complete. 2024-04-25T00:04:48.702 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:04:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.808+0000 7f4a3cea5640 1 Processor -- start 2024-04-25T00:04:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.808+0000 7f4a3cea5640 1 -- start start 2024-04-25T00:04:48.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3cea5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a3815e2d0 0x7f4a3815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3cea5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4a380b3080 0x7f4a380b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3cea5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e6b0 0x7f4a380b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3cea5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4a380cde00 con 0x7f4a3815e2d0 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3cea5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4a3815f4f0 con 0x7f4a380b3080 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3cea5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3815f7d0 con 0x7f4a3815e6b0 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a3815e2d0 0x7f4a3815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a3815e2d0 0x7f4a3815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47794/0 (socket says 172.21.15.113:47794) 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 -- 172.21.15.113:0/3358792036 learned_addr learned my addr 172.21.15.113:0/3358792036 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a35d9b640 1 --2- 172.21.15.113:0/3358792036 >> v2:172.21.15.178:3300/0 conn(0x7f4a380b3080 0x7f4a380b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a36d9d640 1 --2- 172.21.15.113:0/3358792036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e6b0 0x7f4a380b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 -- 172.21.15.113:0/3358792036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e6b0 msgr2=0x7f4a380b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 --2- 172.21.15.113:0/3358792036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e6b0 0x7f4a380b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 -- 172.21.15.113:0/3358792036 >> v2:172.21.15.178:3300/0 conn(0x7f4a380b3080 msgr2=0x7f4a380b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 --2- 172.21.15.113:0/3358792036 >> v2:172.21.15.178:3300/0 conn(0x7f4a380b3080 0x7f4a380b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a3659c640 1 -- 172.21.15.113:0/3358792036 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3815fab0 con 0x7f4a3815e2d0 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a35d9b640 1 --2- 172.21.15.113:0/3358792036 >> v2:172.21.15.178:3300/0 conn(0x7f4a380b3080 0x7f4a380b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.809+0000 7f4a36d9d640 1 --2- 172.21.15.113:0/3358792036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e6b0 0x7f4a380b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.810+0000 7f4a3659c640 1 --2- 172.21.15.113:0/3358792036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e2d0 0x7f4a3815ebf0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f4a38070ff0 tx=0x7f4a2802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ad998d71c325c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.810+0000 7f4a3559a640 1 -- 172.21.15.113:0/3358792036 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28036070 con 0x7f4a3815e2d0 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.810+0000 7f4a3559a640 1 -- 172.21.15.113:0/3358792036 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a28031600 con 0x7f4a3815e2d0 2024-04-25T00:04:48.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.810+0000 7f4a3559a640 1 -- 172.21.15.113:0/3358792036 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a280319d0 con 0x7f4a3815e2d0 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 -- 172.21.15.113:0/3358792036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e2d0 msgr2=0x7f4a3815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/3358792036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e2d0 0x7f4a3815ebf0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f4a38070ff0 tx=0x7f4a2802a6a0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 -- 172.21.15.113:0/3358792036 shutdown_connections 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/3358792036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e6b0 0x7f4a380b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/3358792036 >> v2:172.21.15.178:3300/0 conn(0x7f4a380b3080 0x7f4a380b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/3358792036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e2d0 0x7f4a3815ebf0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 -- 172.21.15.113:0/3358792036 >> 172.21.15.113:0/3358792036 conn(0x7f4a3806b0f0 msgr2=0x7f4a3806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 -- 172.21.15.113:0/3358792036 shutdown_connections 2024-04-25T00:04:48.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.811+0000 7f4a3cea5640 1 -- 172.21.15.113:0/3358792036 wait complete. 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.812+0000 7f4a3cea5640 1 Processor -- start 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.812+0000 7f4a3cea5640 1 -- start start 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a3cea5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a380b3080 0x7f4a38145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a3cea5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e2d0 0x7f4a38143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a3cea5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e6b0 0x7f4a38142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a3cea5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3815f2b0 con 0x7f4a3815e6b0 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a3cea5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3815f130 con 0x7f4a380b3080 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a3cea5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4a38071040 con 0x7f4a3815e2d0 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e6b0 0x7f4a38142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e6b0 0x7f4a38142830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47802/0 (socket says 172.21.15.113:47802) 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 -- 172.21.15.113:0/1032140292 learned_addr learned my addr 172.21.15.113:0/1032140292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 -- 172.21.15.113:0/1032140292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e2d0 msgr2=0x7f4a38143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e2d0 0x7f4a38143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 -- 172.21.15.113:0/1032140292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a380b3080 msgr2=0x7f4a38145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a380b3080 0x7f4a38145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.813+0000 7f4a36d9d640 1 -- 172.21.15.113:0/1032140292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3815db00 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.814+0000 7f4a36d9d640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e6b0 0x7f4a38142830 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c7c0 tx=0x7f4a3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.814+0000 7f4a26ffd640 1 -- 172.21.15.113:0/1032140292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30015020 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.814+0000 7f4a26ffd640 1 -- 172.21.15.113:0/1032140292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a30004230 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.814+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a381539b0 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.814+0000 7f4a26ffd640 1 -- 172.21.15.113:0/1032140292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30007620 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.814+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a38153f70 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.815+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a380691a0 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.815+0000 7f4a26ffd640 1 -- 172.21.15.113:0/1032140292 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a30018020 con 0x7f4a3815e6b0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.816+0000 7f4a26ffd640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a1806be30 0x7f4a1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.816+0000 7f4a3659c640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a1806be30 0x7f4a1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.816+0000 7f4a26ffd640 1 -- 172.21.15.113:0/1032140292 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5227+0+0 (secure 0 0 0) 0x7f4a3008abf0 con 0x7f4a3815e6b0 2024-04-25T00:04:48.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.816+0000 7f4a3659c640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a1806be30 0x7f4a1806e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4a28033000 tx=0x7f4a2802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:48.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.820+0000 7f4a26ffd640 1 -- 172.21.15.113:0/1032140292 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a3001f020 con 0x7f4a3815e6b0 2024-04-25T00:04:48.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:48.995+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4a38072050 con 0x7f4a3815e6b0 2024-04-25T00:04:49.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.688+0000 7f4a26ffd640 1 -- 172.21.15.113:0/1032140292 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) ==== 170+0+0 (secure 0 0 0) 0x7f4a300598d0 con 0x7f4a3815e6b0 2024-04-25T00:04:49.688 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:04:49.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.690+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a1806be30 msgr2=0x7f4a1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:49.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.690+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a1806be30 0x7f4a1806e2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4a28033000 tx=0x7f4a2802e000 comp rx=0 tx=0).stop 2024-04-25T00:04:49.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.690+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e6b0 msgr2=0x7f4a38142830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:49.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.690+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e6b0 0x7f4a38142830 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c7c0 tx=0x7f4a3000cc80 comp rx=0 tx=0).stop 2024-04-25T00:04:49.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.691+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 shutdown_connections 2024-04-25T00:04:49.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.691+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a1806be30 0x7f4a1806e2f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.691+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a3815e6b0 0x7f4a38142830 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.691+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a3815e2d0 0x7f4a38143160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.691+0000 7f4a3cea5640 1 --2- 172.21.15.113:0/1032140292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a380b3080 0x7f4a38145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.691+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 >> 172.21.15.113:0/1032140292 conn(0x7f4a3806b0f0 msgr2=0x7f4a380b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:49.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.691+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 shutdown_connections 2024-04-25T00:04:49.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.693+0000 7f4a3cea5640 1 -- 172.21.15.113:0/1032140292 wait complete. 2024-04-25T00:04:49.721 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:04:49.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.823+0000 7f5109742640 1 Processor -- start 2024-04-25T00:04:49.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.823+0000 7f5109742640 1 -- start start 2024-04-25T00:04:49.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5109742640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f510415e510 0x7f51040d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5109742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51040b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5109742640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51040d2e00 con 0x7f51040d28c0 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5109742640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f51040d30e0 con 0x7f510415e510 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5109742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51040d33c0 con 0x7f51040b63b0 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5102ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f510415e510 0x7f51040d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5102ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f510415e510 0x7f51040d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47810/0 (socket says 172.21.15.113:47810) 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.824+0000 7f5102ffd640 1 -- 172.21.15.113:0/391450640 learned_addr learned my addr 172.21.15.113:0/391450640 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f51037fe640 1 --2- 172.21.15.113:0/391450640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51040b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5102ffd640 1 -- 172.21.15.113:0/391450640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 msgr2=0x7f51040b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5102ffd640 1 --2- 172.21.15.113:0/391450640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51040b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5102ffd640 1 -- 172.21.15.113:0/391450640 >> v1:172.21.15.178:6789/0 conn(0x7f51040d28c0 legacy=0x7f51040b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5102ffd640 1 -- 172.21.15.113:0/391450640 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51040d36a0 con 0x7f510415e510 2024-04-25T00:04:49.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f51037fe640 1 --2- 172.21.15.113:0/391450640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51040b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:49.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5102ffd640 1 --2- 172.21.15.113:0/391450640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51040d2380 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009e50 tx=0x7f50f402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5106563a29f5f479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:49.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5101ffb640 1 -- 172.21.15.113:0/391450640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4036070 con 0x7f510415e510 2024-04-25T00:04:49.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5101ffb640 1 -- 172.21.15.113:0/391450640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f4031600 con 0x7f510415e510 2024-04-25T00:04:49.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.825+0000 7f5101ffb640 1 -- 172.21.15.113:0/391450640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f40319d0 con 0x7f510415e510 2024-04-25T00:04:49.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 -- 172.21.15.113:0/391450640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 msgr2=0x7f51040d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:49.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 --2- 172.21.15.113:0/391450640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51040d2380 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009e50 tx=0x7f50f402a630 comp rx=0 tx=0).stop 2024-04-25T00:04:49.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 -- 172.21.15.113:0/391450640 shutdown_connections 2024-04-25T00:04:49.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 --2- 172.21.15.113:0/391450640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51040b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 --2- 172.21.15.113:0/391450640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51040d2380 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 -- 172.21.15.113:0/391450640 >> 172.21.15.113:0/391450640 conn(0x7f510406b720 msgr2=0x7f51040d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 -- 172.21.15.113:0/391450640 shutdown_connections 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.826+0000 7f5109742640 1 -- 172.21.15.113:0/391450640 wait complete. 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.827+0000 7f5109742640 1 Processor -- start 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.827+0000 7f5109742640 1 -- start start 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f5109742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51041f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f5109742640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51040d28c0 0x7f51041f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f5109742640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51041f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f5109742640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51040b9dd0 con 0x7f510415e510 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f5109742640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51040d3ea0 con 0x7f51040d28c0 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f5109742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51040b7900 con 0x7f51040b63b0 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51041f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51041f49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47824/0 (socket says 172.21.15.113:47824) 2024-04-25T00:04:49.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 -- 172.21.15.113:0/295886257 learned_addr learned my addr 172.21.15.113:0/295886257 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f5102ffd640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51041f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51027fc640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51040d28c0 0x7f51041f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 -- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 msgr2=0x7f51041f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51041f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 -- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51040d28c0 msgr2=0x7f51041f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51040d28c0 0x7f51041f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51037fe640 1 -- 172.21.15.113:0/295886257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51040cc2c0 con 0x7f510415e510 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.828+0000 7f51027fc640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51040d28c0 0x7f51041f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.829+0000 7f51037fe640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51041f49d0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c7f0 tx=0x7f50fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.829+0000 7f50eb7fe640 1 -- 172.21.15.113:0/295886257 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc015070 con 0x7f510415e510 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.829+0000 7f50eb7fe640 1 -- 172.21.15.113:0/295886257 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc004240 con 0x7f510415e510 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.829+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51040cc5a0 con 0x7f510415e510 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.829+0000 7f50eb7fe640 1 -- 172.21.15.113:0/295886257 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc007620 con 0x7f510415e510 2024-04-25T00:04:49.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.829+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51040ccb60 con 0x7f510415e510 2024-04-25T00:04:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.830+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51040691a0 con 0x7f510415e510 2024-04-25T00:04:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.830+0000 7f50eb7fe640 1 -- 172.21.15.113:0/295886257 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50fc0079d0 con 0x7f510415e510 2024-04-25T00:04:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.831+0000 7f50eb7fe640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50f006be60 0x7f50f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.831+0000 7f50eb7fe640 1 -- 172.21.15.113:0/295886257 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5609+0+0 (secure 0 0 0) 0x7f50fc08ae10 con 0x7f510415e510 2024-04-25T00:04:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.831+0000 7f5102ffd640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50f006be60 0x7f50f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.831+0000 7f5102ffd640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50f006be60 0x7f50f006e320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50f4033000 tx=0x7f50f4009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:49.835+0000 7f50eb7fe640 1 -- 172.21.15.113:0/295886257 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50fc0598b0 con 0x7f510415e510 2024-04-25T00:04:50.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.045+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f51040b80a0 con 0x7f510415e510 2024-04-25T00:04:50.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.689+0000 7f50eb7fe640 1 -- 172.21.15.113:0/295886257 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v54) ==== 174+0+0 (secure 0 0 0) 0x7f50fc0591b0 con 0x7f510415e510 2024-04-25T00:04:50.690 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T00:04:50.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.693+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50f006be60 msgr2=0x7f50f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:50.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.693+0000 7f5109742640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50f006be60 0x7f50f006e320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50f4033000 tx=0x7f50f4009000 comp rx=0 tx=0).stop 2024-04-25T00:04:50.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.693+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 msgr2=0x7f51041f49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:50.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.693+0000 7f5109742640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51041f49d0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c7f0 tx=0x7f50fc00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.696+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 shutdown_connections 2024-04-25T00:04:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.696+0000 7f5109742640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f50f006be60 0x7f50f006e320 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.696+0000 7f5109742640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51040d28c0 0x7f51041f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.696+0000 7f5109742640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510415e510 0x7f51041f49d0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.696+0000 7f5109742640 1 --2- 172.21.15.113:0/295886257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040b63b0 0x7f51041f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.697+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 >> 172.21.15.113:0/295886257 conn(0x7f510406b720 msgr2=0x7f510415d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.709+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 shutdown_connections 2024-04-25T00:04:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.710+0000 7f5109742640 1 -- 172.21.15.113:0/295886257 wait complete. 2024-04-25T00:04:50.732 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:50.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.831+0000 7ff9fd3a5640 1 Processor -- start 2024-04-25T00:04:50.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.831+0000 7ff9fd3a5640 1 -- start start 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9fd3a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9f8161fe0 0x7ff9f806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9fd3a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f806b800 con 0x7ff9f806c030 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9fd3a5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f81624e0 con 0x7ff9f806bc50 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9fd3a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f81627c0 con 0x7ff9f8161fe0 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9f77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff9f806c030 0x7ff9f80709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38186/0 (socket says 172.21.15.113:38186) 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9f77fe640 1 -- 172.21.15.113:0/3545123636 learned_addr learned my addr 172.21.15.113:0/3545123636 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9f67fc640 1 --2- 172.21.15.113:0/3545123636 >> v2:172.21.15.113:3300/0 conn(0x7ff9f8161fe0 0x7ff9f806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9f67fc640 1 -- 172.21.15.113:0/3545123636 >> v1:172.21.15.178:6789/0 conn(0x7ff9f806bc50 legacy=0x7ff9f806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9f67fc640 1 -- 172.21.15.113:0/3545123636 >> v1:172.21.15.113:6789/0 conn(0x7ff9f806c030 legacy=0x7ff9f80709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:50.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.832+0000 7ff9f67fc640 1 -- 172.21.15.113:0/3545123636 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8162aa0 con 0x7ff9f8161fe0 2024-04-25T00:04:50.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.833+0000 7ff9f67fc640 1 --2- 172.21.15.113:0/3545123636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f806cc70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0097a0 tx=0x7ff9ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353694cdcfed276e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:50.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.833+0000 7ff9f5ffb640 1 -- 172.21.15.113:0/3545123636 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec037070 con 0x7ff9f8161fe0 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.833+0000 7ff9f5ffb640 1 -- 172.21.15.113:0/3545123636 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ec032440 con 0x7ff9f8161fe0 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.833+0000 7ff9f5ffb640 1 -- 172.21.15.113:0/3545123636 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec032810 con 0x7ff9f8161fe0 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.834+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/3545123636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 msgr2=0x7ff9f806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.834+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/3545123636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f806cc70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0097a0 tx=0x7ff9ec02a230 comp rx=0 tx=0).stop 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.834+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/3545123636 shutdown_connections 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.834+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/3545123636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f806cc70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.834+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/3545123636 >> 172.21.15.113:0/3545123636 conn(0x7ff9f80b3a70 msgr2=0x7ff9f80b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.834+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/3545123636 shutdown_connections 2024-04-25T00:04:50.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.834+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/3545123636 wait complete. 2024-04-25T00:04:50.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.835+0000 7ff9fd3a5640 1 Processor -- start 2024-04-25T00:04:50.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.835+0000 7ff9fd3a5640 1 -- start start 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9fd3a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9f806bc50 0x7ff9f80b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9fd3a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9f806c030 0x7ff9f80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9fd3a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f80c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9fd3a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f8071940 con 0x7ff9f8161fe0 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9fd3a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f815ea30 con 0x7ff9f806c030 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9fd3a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f80710b0 con 0x7ff9f806bc50 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f80c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f80c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47832/0 (socket says 172.21.15.113:47832) 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 -- 172.21.15.113:0/4105495769 learned_addr learned my addr 172.21.15.113:0/4105495769 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 -- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9f806bc50 msgr2=0x7ff9f80b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f6ffd640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9f806bc50 0x7ff9f80b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:50.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9f806bc50 0x7ff9f80b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f67fc640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9f806c030 0x7ff9f80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 -- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9f806c030 msgr2=0x7ff9f80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9f806c030 0x7ff9f80b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 -- 172.21.15.113:0/4105495769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f80b51f0 con 0x7ff9f8161fe0 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f67fc640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9f806c030 0x7ff9f80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.836+0000 7ff9f77fe640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f80c9030 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000cbc0 tx=0x7ff9f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.837+0000 7ff9df7fe640 1 -- 172.21.15.113:0/4105495769 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0017070 con 0x7ff9f8161fe0 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.837+0000 7ff9df7fe640 1 -- 172.21.15.113:0/4105495769 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f000ace0 con 0x7ff9f8161fe0 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.837+0000 7ff9df7fe640 1 -- 172.21.15.113:0/4105495769 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0012600 con 0x7ff9f8161fe0 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.837+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f80b54d0 con 0x7ff9f8161fe0 2024-04-25T00:04:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.837+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f80b5a90 con 0x7ff9f8161fe0 2024-04-25T00:04:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.838+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f80691a0 con 0x7ff9f8161fe0 2024-04-25T00:04:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.838+0000 7ff9df7fe640 1 -- 172.21.15.113:0/4105495769 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9f001a070 con 0x7ff9f8161fe0 2024-04-25T00:04:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.839+0000 7ff9df7fe640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff9e406be30 0x7ff9e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.839+0000 7ff9df7fe640 1 -- 172.21.15.113:0/4105495769 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff9f008ab00 con 0x7ff9f8161fe0 2024-04-25T00:04:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.839+0000 7ff9f6ffd640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff9e406be30 0x7ff9e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.839+0000 7ff9f6ffd640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff9e406be30 0x7ff9e406e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80015e0 tx=0x7ff9e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:50.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:50.843+0000 7ff9df7fe640 1 -- 172.21.15.113:0/4105495769 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9f0058e60 con 0x7ff9f8161fe0 2024-04-25T00:04:51.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.016+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9f80714d0 con 0x7ff9f8161fe0 2024-04-25T00:04:51.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.017+0000 7ff9df7fe640 1 -- 172.21.15.113:0/4105495769 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16680 (secure 0 0 0) 0x7ff9f005d160 con 0x7ff9f8161fe0 2024-04-25T00:04:51.017 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:51.017 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":103,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:50.047141+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:48.259110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:49.014993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff9e406be30 msgr2=0x7ff9e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff9e406be30 0x7ff9e406e2f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80015e0 tx=0x7ff9e8009040 comp rx=0 tx=0).stop 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 msgr2=0x7ff9f80c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f80c9030 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000cbc0 tx=0x7ff9f000a430 comp rx=0 tx=0).stop 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 shutdown_connections 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff9e406be30 0x7ff9e406e2f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9f806c030 0x7ff9f80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9f8161fe0 0x7ff9f80c9030 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 --2- 172.21.15.113:0/4105495769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9f806bc50 0x7ff9f80b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 >> 172.21.15.113:0/4105495769 conn(0x7ff9f80b3a70 msgr2=0x7ff9f80cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 shutdown_connections 2024-04-25T00:04:51.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.019+0000 7ff9fd3a5640 1 -- 172.21.15.113:0/4105495769 wait complete. 2024-04-25T00:04:51.031 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:51.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.127+0000 7fa494590640 1 Processor -- start 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.127+0000 7fa494590640 1 -- start start 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa494590640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa48c15e320 0x7fa48c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa494590640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c0d0b00 0x7fa48c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa494590640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa48c0d3de0 con 0x7fa48c0d2290 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa494590640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c0d14d0 con 0x7fa48c15e320 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa494590640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c0d3410 con 0x7fa48c0d0b00 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa49232c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa48c15e320 0x7fa48c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa491b2b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa48c0d2290 0x7fa48c0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38194/0 (socket says 172.21.15.113:38194) 2024-04-25T00:04:51.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa49232c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa48c15e320 0x7fa48c0d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47836/0 (socket says 172.21.15.113:47836) 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.128+0000 7fa491b2b640 1 -- 172.21.15.113:0/2875251726 learned_addr learned my addr 172.21.15.113:0/2875251726 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa492b2d640 1 --2- 172.21.15.113:0/2875251726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c0d0b00 0x7fa48c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49232c640 1 -- 172.21.15.113:0/2875251726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c0d0b00 msgr2=0x7fa48c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49232c640 1 --2- 172.21.15.113:0/2875251726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c0d0b00 0x7fa48c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49232c640 1 -- 172.21.15.113:0/2875251726 >> v1:172.21.15.113:6789/0 conn(0x7fa48c0d2290 legacy=0x7fa48c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49232c640 1 -- 172.21.15.113:0/2875251726 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c0d2800 con 0x7fa48c15e320 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49132a640 1 -- 172.21.15.113:0/2875251726 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249349453 0 0) 0x7fa48c0d3de0 con 0x7fa48c0d2290 2024-04-25T00:04:51.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49232c640 1 --2- 172.21.15.113:0/2875251726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c15e320 0x7fa48c0d38a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa4840097a0 tx=0x7fa48402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761363d98db4eabb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49132a640 1 -- 172.21.15.113:0/2875251726 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484037070 con 0x7fa48c15e320 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49132a640 1 -- 172.21.15.113:0/2875251726 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa484032440 con 0x7fa48c15e320 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.129+0000 7fa49132a640 1 -- 172.21.15.113:0/2875251726 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484032810 con 0x7fa48c15e320 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 -- 172.21.15.113:0/2875251726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c15e320 msgr2=0x7fa48c0d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 --2- 172.21.15.113:0/2875251726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c15e320 0x7fa48c0d38a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa4840097a0 tx=0x7fa48402a210 comp rx=0 tx=0).stop 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 -- 172.21.15.113:0/2875251726 shutdown_connections 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 --2- 172.21.15.113:0/2875251726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c0d0b00 0x7fa48c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 --2- 172.21.15.113:0/2875251726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c15e320 0x7fa48c0d38a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 -- 172.21.15.113:0/2875251726 >> 172.21.15.113:0/2875251726 conn(0x7fa48c0b5c20 msgr2=0x7fa48c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 -- 172.21.15.113:0/2875251726 shutdown_connections 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.130+0000 7fa494590640 1 -- 172.21.15.113:0/2875251726 wait complete. 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.131+0000 7fa494590640 1 Processor -- start 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.131+0000 7fa494590640 1 -- start start 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa494590640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c0d0b00 0x7fa48c15f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa494590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0d2290 0x7fa48c1f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa494590640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c15e320 0x7fa48c160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa494590640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0d30a0 con 0x7fa48c0d2290 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa494590640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0d2dc0 con 0x7fa48c0d0b00 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa494590640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c15e700 con 0x7fa48c15e320 2024-04-25T00:04:51.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0d2290 0x7fa48c1f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0d2290 0x7fa48c1f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47840/0 (socket says 172.21.15.113:47840) 2024-04-25T00:04:51.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 -- 172.21.15.113:0/3182518288 learned_addr learned my addr 172.21.15.113:0/3182518288 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:51.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa492b2d640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c15e320 0x7fa48c160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa49232c640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c0d0b00 0x7fa48c15f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 -- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c15e320 msgr2=0x7fa48c160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c15e320 0x7fa48c160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 -- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c0d0b00 msgr2=0x7fa48c15f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c0d0b00 0x7fa48c15f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 -- 172.21.15.113:0/3182518288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c154f90 con 0x7fa48c0d2290 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa49232c640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c0d0b00 0x7fa48c15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.132+0000 7fa491b2b640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0d2290 0x7fa48c1f3180 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c4d0 tx=0x7fa47c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.133+0000 7fa482ffd640 1 -- 172.21.15.113:0/3182518288 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c018070 con 0x7fa48c0d2290 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.133+0000 7fa482ffd640 1 -- 172.21.15.113:0/3182518288 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c007500 con 0x7fa48c0d2290 2024-04-25T00:04:51.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.133+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c15fc50 con 0x7fa48c0d2290 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.133+0000 7fa482ffd640 1 -- 172.21.15.113:0/3182518288 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c004a20 con 0x7fa48c0d2290 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.133+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c0cb380 con 0x7fa48c0d2290 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.134+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c0691a0 con 0x7fa48c0d2290 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.134+0000 7fa482ffd640 1 -- 172.21.15.113:0/3182518288 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa47c01b070 con 0x7fa48c0d2290 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.135+0000 7fa482ffd640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa47006be60 0x7fa47006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.135+0000 7fa49232c640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa47006be60 0x7fa47006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.135+0000 7fa482ffd640 1 -- 172.21.15.113:0/3182518288 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fa47c08a100 con 0x7fa48c0d2290 2024-04-25T00:04:51.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.135+0000 7fa49232c640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa47006be60 0x7fa47006e320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa484034000 tx=0x7fa48402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.139+0000 7fa482ffd640 1 -- 172.21.15.113:0/3182518288 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa48c154f90 con 0x7fa48c0d2290 2024-04-25T00:04:51.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.315+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa48c074f50 con 0x7fa48c0d2290 2024-04-25T00:04:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.316+0000 7fa482ffd640 1 -- 172.21.15.113:0/3182518288 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16680 (secure 0 0 0) 0x7fa47c058ab0 con 0x7fa48c0d2290 2024-04-25T00:04:51.316 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:51.317 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":103,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:50.047141+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:48.259110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:49.014993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa47006be60 msgr2=0x7fa47006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa47006be60 0x7fa47006e320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa484034000 tx=0x7fa48402e000 comp rx=0 tx=0).stop 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0d2290 msgr2=0x7fa48c1f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0d2290 0x7fa48c1f3180 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c4d0 tx=0x7fa47c00c990 comp rx=0 tx=0).stop 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 shutdown_connections 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa47006be60 0x7fa47006e320 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa48c15e320 0x7fa48c160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0d2290 0x7fa48c1f3180 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 --2- 172.21.15.113:0/3182518288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c0d0b00 0x7fa48c15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 >> 172.21.15.113:0/3182518288 conn(0x7fa48c0b5c20 msgr2=0x7fa48c0b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 shutdown_connections 2024-04-25T00:04:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.318+0000 7fa494590640 1 -- 172.21.15.113:0/3182518288 wait complete. 2024-04-25T00:04:51.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:04:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.429+0000 7ff67f44c640 1 Processor -- start 2024-04-25T00:04:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.430+0000 7ff67f44c640 1 -- start start 2024-04-25T00:04:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.430+0000 7ff67f44c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 0x7ff67815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.430+0000 7ff67f44c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff678071170 con 0x7ff67815b670 2024-04-25T00:04:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.430+0000 7ff67f44c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff678071450 con 0x7ff67815ba50 2024-04-25T00:04:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.430+0000 7ff67f44c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff678071730 con 0x7ff67806d5b0 2024-04-25T00:04:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.430+0000 7ff67d1e8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff67815b670 0x7ff6780cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38198/0 (socket says 172.21.15.113:38198) 2024-04-25T00:04:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.430+0000 7ff67d1e8640 1 -- 172.21.15.113:0/3028198026 learned_addr learned my addr 172.21.15.113:0/3028198026 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d9e9640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 0x7ff67815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 604088400 0 0) 0x7ff678071170 con 0x7ff67815b670 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff664003440 con 0x7ff67815b670 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2628591949 0 0) 0x7ff664003440 con 0x7ff67815b670 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 msgr2=0x7ff67815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 0x7ff67815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 >> v1:172.21.15.178:6789/0 conn(0x7ff67815ba50 legacy=0x7ff67806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff678071a10 con 0x7ff67815b670 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff674001670 con 0x7ff67815b670 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 >> v1:172.21.15.113:6789/0 conn(0x7ff67815b670 legacy=0x7ff6780cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff664003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff664008850 0x7ff66400acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 0x7ff66400d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff678071730 con 0x7ff664003ad0 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff66400dd10 con 0x7ff664008850 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff66400df10 con 0x7ff67806d5b0 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d1e8640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff664003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67c9e7640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff664008850 0x7ff66400acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d9e9640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 0x7ff66400d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d1e8640 1 -- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 msgr2=0x7ff66400d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d1e8640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 0x7ff66400d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d1e8640 1 -- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff664008850 msgr2=0x7ff66400acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d1e8640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff664008850 0x7ff66400acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d1e8640 1 -- 172.21.15.113:0/3028198026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff664003440 con 0x7ff664003ad0 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.431+0000 7ff67d9e9640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff67806d5b0 0x7ff66400d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.432+0000 7ff67d1e8640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff664003eb0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff67400c240 tx=0x7ff664003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21be60b8ac8704be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.432+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67402e070 con 0x7ff664003ad0 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.432+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff674005af0 con 0x7ff664003ad0 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.432+0000 7ff66ffff640 1 -- 172.21.15.113:0/3028198026 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674007d30 con 0x7ff664003ad0 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.433+0000 7ff67f44c640 1 -- 172.21.15.113:0/3028198026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 msgr2=0x7ff664003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.433+0000 7ff67f44c640 1 --2- 172.21.15.113:0/3028198026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff664003eb0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff67400c240 tx=0x7ff664003730 comp rx=0 tx=0).stop 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.433+0000 7ff67d9e9640 1 -- 172.21.15.113:0/3028198026 reap_dead start 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.433+0000 7ff67f44c640 1 -- 172.21.15.113:0/3028198026 shutdown_connections 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.433+0000 7ff67f44c640 1 -- 172.21.15.113:0/3028198026 >> 172.21.15.113:0/3028198026 conn(0x7ff67806b0f0 msgr2=0x7ff67806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.433+0000 7ff67f44c640 1 -- 172.21.15.113:0/3028198026 shutdown_connections 2024-04-25T00:04:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.433+0000 7ff67f44c640 1 -- 172.21.15.113:0/3028198026 wait complete. 2024-04-25T00:04:51.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.434+0000 7ff67f44c640 1 Processor -- start 2024-04-25T00:04:51.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.434+0000 7ff67f44c640 1 -- start start 2024-04-25T00:04:51.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67f44c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff678147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67f44c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff678152d90 0x7ff678153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67f44c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781efa10 0x7ff6781efe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67f44c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff67815c170 con 0x7ff664003ad0 2024-04-25T00:04:51.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67f44c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff67815bff0 con 0x7ff6781efa10 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67f44c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff67815c2f0 con 0x7ff678152d90 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff678147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff678147bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47852/0 (socket says 172.21.15.113:47852) 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 -- 172.21.15.113:0/1657886958 learned_addr learned my addr 172.21.15.113:0/1657886958 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67c9e7640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff678152d90 0x7ff678153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d9e9640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781efa10 0x7ff6781efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 -- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff678152d90 msgr2=0x7ff678153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff678152d90 0x7ff678153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 -- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781efa10 msgr2=0x7ff6781efe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781efa10 0x7ff6781efe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 -- 172.21.15.113:0/1657886958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6781f0740 con 0x7ff664003ad0 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67d1e8640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff678147bb0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff67400c330 tx=0x7ff678071340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff66dffb640 1 -- 172.21.15.113:0/1657886958 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674042070 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff66dffb640 1 -- 172.21.15.113:0/1657886958 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff674007280 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6781507c0 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.435+0000 7ff66dffb640 1 -- 172.21.15.113:0/1657886958 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674007d30 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.436+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6781f03d0 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.436+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6780691a0 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.437+0000 7ff66dffb640 1 -- 172.21.15.113:0/1657886958 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff674045070 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.437+0000 7ff66dffb640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff65c06be30 0x7ff65c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.437+0000 7ff66dffb640 1 -- 172.21.15.113:0/1657886958 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff6740ab9a0 con 0x7ff664003ad0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.437+0000 7ff67c9e7640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff65c06be30 0x7ff65c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.438+0000 7ff67c9e7640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff65c06be30 0x7ff65c06e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff678152840 tx=0x7ff668007bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.441+0000 7ff66dffb640 1 -- 172.21.15.113:0/1657886958 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff67404c020 con 0x7ff664003ad0 2024-04-25T00:04:51.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.650+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff6780b42f0 con 0x7ff664003ad0 2024-04-25T00:04:51.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.709+0000 7ff66dffb640 1 -- 172.21.15.113:0/1657886958 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7ff67407a3e0 con 0x7ff664003ad0 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.711+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff65c06be30 msgr2=0x7ff65c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.711+0000 7ff67f44c640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff65c06be30 0x7ff65c06e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff678152840 tx=0x7ff668007bd0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.711+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 msgr2=0x7ff678147bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.711+0000 7ff67f44c640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff678147bb0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff67400c330 tx=0x7ff678071340 comp rx=0 tx=0).stop 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 shutdown_connections 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff65c06be30 0x7ff65c06e2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781efa10 0x7ff6781efe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff678152d90 0x7ff678153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 --2- 172.21.15.113:0/1657886958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff664003ad0 0x7ff678147bb0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 >> 172.21.15.113:0/1657886958 conn(0x7ff67806b0f0 msgr2=0x7ff67806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:51.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 shutdown_connections 2024-04-25T00:04:51.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.712+0000 7ff67f44c640 1 -- 172.21.15.113:0/1657886958 wait complete. 2024-04-25T00:04:51.723 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:04:51.723 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:04:51.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.822+0000 7efe228c4640 1 Processor -- start 2024-04-25T00:04:51.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.822+0000 7efe228c4640 1 -- start start 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe228c4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe1c0b3080 0x7efe1c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe228c4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1c15f4f0 con 0x7efe1c15e6b0 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe228c4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1c15f7d0 con 0x7efe1c15e2d0 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe228c4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efe1c0cde00 con 0x7efe1c0b3080 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe1b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe1c0b3080 0x7efe1c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe20e61640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efe1c15e6b0 0x7efe1c0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38206/0 (socket says 172.21.15.113:38206) 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe1b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe1c0b3080 0x7efe1c0b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47860/0 (socket says 172.21.15.113:47860) 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe20e61640 1 -- 172.21.15.113:0/3269250985 learned_addr learned my addr 172.21.15.113:0/3269250985 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe1b7fe640 1 -- 172.21.15.113:0/3269250985 >> v1:172.21.15.178:6789/0 conn(0x7efe1c15e2d0 legacy=0x7efe1c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe1b7fe640 1 -- 172.21.15.113:0/3269250985 >> v1:172.21.15.113:6789/0 conn(0x7efe1c15e6b0 legacy=0x7efe1c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe1b7fe640 1 -- 172.21.15.113:0/3269250985 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c15fab0 con 0x7efe1c0b3080 2024-04-25T00:04:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.823+0000 7efe1affd640 1 -- 172.21.15.113:0/3269250985 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2350712742 0 0) 0x7efe1c15f4f0 con 0x7efe1c15e6b0 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.824+0000 7efe1b7fe640 1 --2- 172.21.15.113:0/3269250985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c0b3080 0x7efe1c0b4330 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7efe100097a0 tx=0x7efe1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b14a855c633caa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.824+0000 7efe1affd640 1 -- 172.21.15.113:0/3269250985 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10037070 con 0x7efe1c0b3080 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.824+0000 7efe1affd640 1 -- 172.21.15.113:0/3269250985 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe10032440 con 0x7efe1c0b3080 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.824+0000 7efe1affd640 1 -- 172.21.15.113:0/3269250985 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10032810 con 0x7efe1c0b3080 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.825+0000 7efe228c4640 1 -- 172.21.15.113:0/3269250985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c0b3080 msgr2=0x7efe1c0b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.825+0000 7efe228c4640 1 --2- 172.21.15.113:0/3269250985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c0b3080 0x7efe1c0b4330 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7efe100097a0 tx=0x7efe1002a1e0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.825+0000 7efe228c4640 1 -- 172.21.15.113:0/3269250985 shutdown_connections 2024-04-25T00:04:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.825+0000 7efe228c4640 1 --2- 172.21.15.113:0/3269250985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c0b3080 0x7efe1c0b4330 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.825+0000 7efe228c4640 1 -- 172.21.15.113:0/3269250985 >> 172.21.15.113:0/3269250985 conn(0x7efe1c06b0f0 msgr2=0x7efe1c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.825+0000 7efe228c4640 1 -- 172.21.15.113:0/3269250985 shutdown_connections 2024-04-25T00:04:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.825+0000 7efe228c4640 1 -- 172.21.15.113:0/3269250985 wait complete. 2024-04-25T00:04:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.826+0000 7efe228c4640 1 Processor -- start 2024-04-25T00:04:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.826+0000 7efe228c4640 1 -- start start 2024-04-25T00:04:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe1c0b3080 0x7efe1c142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c15e2d0 0x7efe1c13f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe1c15e6b0 0x7efe1c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c0b49f0 con 0x7efe1c15e2d0 2024-04-25T00:04:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c0b4870 con 0x7efe1c0b3080 2024-04-25T00:04:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c0b4b70 con 0x7efe1c15e6b0 2024-04-25T00:04:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c15e2d0 0x7efe1c13f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c15e2d0 0x7efe1c13f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47870/0 (socket says 172.21.15.113:47870) 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 -- 172.21.15.113:0/3844190881 learned_addr learned my addr 172.21.15.113:0/3844190881 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe20e61640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe1c15e6b0 0x7efe1c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1bfff640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe1c0b3080 0x7efe1c142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 -- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe1c15e6b0 msgr2=0x7efe1c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe1c15e6b0 0x7efe1c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 -- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe1c0b3080 msgr2=0x7efe1c142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe1c0b3080 0x7efe1c142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 -- 172.21.15.113:0/3844190881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c142a70 con 0x7efe1c15e2d0 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe1b7fe640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c15e2d0 0x7efe1c13f1b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7efe100097a0 tx=0x7efe100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe18ff9640 1 -- 172.21.15.113:0/3844190881 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10041070 con 0x7efe1c15e2d0 2024-04-25T00:04:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe18ff9640 1 -- 172.21.15.113:0/3844190881 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1003c870 con 0x7efe1c15e2d0 2024-04-25T00:04:51.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c1553e0 con 0x7efe1c15e2d0 2024-04-25T00:04:51.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe18ff9640 1 -- 172.21.15.113:0/3844190881 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10037040 con 0x7efe1c15e2d0 2024-04-25T00:04:51.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.827+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c1559a0 con 0x7efe1c15e2d0 2024-04-25T00:04:51.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.828+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c0691a0 con 0x7efe1c15e2d0 2024-04-25T00:04:51.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.829+0000 7efe18ff9640 1 -- 172.21.15.113:0/3844190881 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe10044070 con 0x7efe1c15e2d0 2024-04-25T00:04:51.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.829+0000 7efe18ff9640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0806be60 0x7efe0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:51.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.829+0000 7efe1bfff640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0806be60 0x7efe0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:51.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.829+0000 7efe18ff9640 1 -- 172.21.15.113:0/3844190881 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7efe100abb20 con 0x7efe1c15e2d0 2024-04-25T00:04:51.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.830+0000 7efe1bfff640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0806be60 0x7efe0806e320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efe0c007fd0 tx=0x7efe0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:51.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:51.833+0000 7efe18ff9640 1 -- 172.21.15.113:0/3844190881 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe1007a680 con 0x7efe1c15e2d0 2024-04-25T00:04:52.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.042+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7efe1c072050 con 0x7efe1c15e2d0 2024-04-25T00:04:52.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.714+0000 7efe18ff9640 1 -- 172.21.15.113:0/3844190881 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7efe10079f80 con 0x7efe1c15e2d0 2024-04-25T00:04:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0806be60 msgr2=0x7efe0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0806be60 0x7efe0806e320 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efe0c007fd0 tx=0x7efe0c009040 comp rx=0 tx=0).stop 2024-04-25T00:04:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c15e2d0 msgr2=0x7efe1c13f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c15e2d0 0x7efe1c13f1b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7efe100097a0 tx=0x7efe100046c0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 shutdown_connections 2024-04-25T00:04:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe0806be60 0x7efe0806e320 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe1c15e6b0 0x7efe1c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe1c15e2d0 0x7efe1c13f1b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 --2- 172.21.15.113:0/3844190881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe1c0b3080 0x7efe1c142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 >> 172.21.15.113:0/3844190881 conn(0x7efe1c06b0f0 msgr2=0x7efe1c0b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 shutdown_connections 2024-04-25T00:04:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.717+0000 7efe228c4640 1 -- 172.21.15.113:0/3844190881 wait complete. 2024-04-25T00:04:52.730 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:04:52.730 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:52.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.829+0000 7f912523f640 1 Processor -- start 2024-04-25T00:04:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.829+0000 7f912523f640 1 -- start start 2024-04-25T00:04:52.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f912523f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f912015a6b0 0x7f9120070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:52.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f912523f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:52.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f912523f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9120071780 0x7f912015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:52.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f912523f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f91200b2d50 con 0x7f912015a6b0 2024-04-25T00:04:52.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f912523f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9120072a70 con 0x7f9120071780 2024-04-25T00:04:52.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f912523f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91200b3010 con 0x7f9120072ea0 2024-04-25T00:04:52.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f912015a6b0 0x7f9120070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:52.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f912015a6b0 0x7f9120070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47876/0 (socket says 172.21.15.113:47876) 2024-04-25T00:04:52.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 -- 172.21.15.113:0/3018762810 learned_addr learned my addr 172.21.15.113:0/3018762810 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:52.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 -- 172.21.15.113:0/3018762810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 msgr2=0x7f9120071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:52.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f9117fff640 1 --2- 172.21.15.113:0/3018762810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:52.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911f59e640 1 --2- 172.21.15.113:0/3018762810 >> v2:172.21.15.178:3300/0 conn(0x7f9120071780 0x7f912015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:52.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 --2- 172.21.15.113:0/3018762810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 -- 172.21.15.113:0/3018762810 >> v2:172.21.15.178:3300/0 conn(0x7f9120071780 msgr2=0x7f912015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:52.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 --2- 172.21.15.113:0/3018762810 >> v2:172.21.15.178:3300/0 conn(0x7f9120071780 0x7f912015e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.830+0000 7f911ed9d640 1 -- 172.21.15.113:0/3018762810 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91200b32f0 con 0x7f912015a6b0 2024-04-25T00:04:52.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.831+0000 7f911ed9d640 1 --2- 172.21.15.113:0/3018762810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120070d00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9120072c80 tx=0x7f911002b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392e1c589fd67aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:52.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.831+0000 7f911e59c640 1 -- 172.21.15.113:0/3018762810 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911000b040 con 0x7f912015a6b0 2024-04-25T00:04:52.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.831+0000 7f911e59c640 1 -- 172.21.15.113:0/3018762810 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9110030710 con 0x7f912015a6b0 2024-04-25T00:04:52.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.831+0000 7f911e59c640 1 -- 172.21.15.113:0/3018762810 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110030ae0 con 0x7f912015a6b0 2024-04-25T00:04:52.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.831+0000 7f912523f640 1 -- 172.21.15.113:0/3018762810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 msgr2=0x7f9120070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:52.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.831+0000 7f912523f640 1 --2- 172.21.15.113:0/3018762810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120070d00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9120072c80 tx=0x7f911002b580 comp rx=0 tx=0).stop 2024-04-25T00:04:52.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.832+0000 7f912523f640 1 -- 172.21.15.113:0/3018762810 shutdown_connections 2024-04-25T00:04:52.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.832+0000 7f912523f640 1 --2- 172.21.15.113:0/3018762810 >> v2:172.21.15.178:3300/0 conn(0x7f9120071780 0x7f912015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.832+0000 7f912523f640 1 --2- 172.21.15.113:0/3018762810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.832+0000 7f912523f640 1 --2- 172.21.15.113:0/3018762810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120070d00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.832+0000 7f912523f640 1 -- 172.21.15.113:0/3018762810 >> 172.21.15.113:0/3018762810 conn(0x7f91200b1a50 msgr2=0x7f91200b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.832+0000 7f912523f640 1 -- 172.21.15.113:0/3018762810 shutdown_connections 2024-04-25T00:04:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.832+0000 7f912523f640 1 -- 172.21.15.113:0/3018762810 wait complete. 2024-04-25T00:04:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.833+0000 7f912523f640 1 Processor -- start 2024-04-25T00:04:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.833+0000 7f912523f640 1 -- start start 2024-04-25T00:04:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.833+0000 7f912523f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9120071780 0x7f9120155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f912523f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f912523f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f912523f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f912015aa90 con 0x7f912015a6b0 2024-04-25T00:04:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f912523f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9120072720 con 0x7f9120071780 2024-04-25T00:04:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f912523f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f912015ac10 con 0x7f9120072ea0 2024-04-25T00:04:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47880/0 (socket says 172.21.15.113:47880) 2024-04-25T00:04:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 -- 172.21.15.113:0/1364921919 learned_addr learned my addr 172.21.15.113:0/1364921919 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911ed9d640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9120071780 0x7f9120155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f9117fff640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 -- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 msgr2=0x7f9120153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 -- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9120071780 msgr2=0x7f9120155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9120071780 0x7f9120155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 -- 172.21.15.113:0/1364921919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9120157f30 con 0x7f912015a6b0 2024-04-25T00:04:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f911f59e640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120153e70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f911800cd20 tx=0x7f911800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f91177fe640 1 -- 172.21.15.113:0/1364921919 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118003b90 con 0x7f912015a6b0 2024-04-25T00:04:52.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f91177fe640 1 -- 172.21.15.113:0/1364921919 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9118003d30 con 0x7f912015a6b0 2024-04-25T00:04:52.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9120158210 con 0x7f912015a6b0 2024-04-25T00:04:52.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f91177fe640 1 -- 172.21.15.113:0/1364921919 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118004240 con 0x7f912015a6b0 2024-04-25T00:04:52.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.834+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91201587d0 con 0x7f912015a6b0 2024-04-25T00:04:52.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.835+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91200691a0 con 0x7f912015a6b0 2024-04-25T00:04:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.836+0000 7f91177fe640 1 -- 172.21.15.113:0/1364921919 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91180043e0 con 0x7f912015a6b0 2024-04-25T00:04:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.836+0000 7f91177fe640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f910006be60 0x7f910006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.836+0000 7f91177fe640 1 -- 172.21.15.113:0/1364921919 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f911808b7c0 con 0x7f912015a6b0 2024-04-25T00:04:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.836+0000 7f911ed9d640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f910006be60 0x7f910006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.837+0000 7f911ed9d640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f910006be60 0x7f910006e320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9110035000 tx=0x7f911000c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:52.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:52.840+0000 7f91177fe640 1 -- 172.21.15.113:0/1364921919 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f911805a200 con 0x7f912015a6b0 2024-04-25T00:04:52.883 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:04:52.886 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:04:53.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.055+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91200b2900 con 0x7f912015a6b0 2024-04-25T00:04:53.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.056+0000 7f91177fe640 1 -- 172.21.15.113:0/1364921919 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7f9118059b00 con 0x7f912015a6b0 2024-04-25T00:04:53.056 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:53.057 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:04:53.057 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f910006be60 msgr2=0x7f910006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f910006be60 0x7f910006e320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9110035000 tx=0x7f911000c040 comp rx=0 tx=0).stop 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 msgr2=0x7f9120153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120153e70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f911800cd20 tx=0x7f911800a660 comp rx=0 tx=0).stop 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 shutdown_connections 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f910006be60 0x7f910006e320 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f912015a6b0 0x7f9120153e70 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9120072ea0 0x7f9120153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 --2- 172.21.15.113:0/1364921919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9120071780 0x7f9120155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 >> 172.21.15.113:0/1364921919 conn(0x7f91200b1a50 msgr2=0x7f91200b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:53.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 shutdown_connections 2024-04-25T00:04:53.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.058+0000 7f912523f640 1 -- 172.21.15.113:0/1364921919 wait complete. 2024-04-25T00:04:53.070 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:04:53.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.166+0000 7fe0d79dc640 1 Processor -- start 2024-04-25T00:04:53.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.166+0000 7fe0d79dc640 1 -- start start 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0d79dc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe0d00b4b00 0x7fe0d01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0d79dc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d0071b20 con 0x7fe0d015a6b0 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0d79dc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d00b2d50 con 0x7fe0d0072ea0 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0d79dc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d0072a70 con 0x7fe0d00b4b00 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0d5778640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe0d015a6b0 0x7fe0d0070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38214/0 (socket says 172.21.15.113:38214) 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0d5778640 1 -- 172.21.15.113:0/4074678547 learned_addr learned my addr 172.21.15.113:0/4074678547 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0d5f79640 1 --2- 172.21.15.113:0/4074678547 >> v2:172.21.15.178:3300/0 conn(0x7fe0d00b4b00 0x7fe0d01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744426730 0 0) 0x7fe0d0071b20 con 0x7fe0d015a6b0 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0bc003520 con 0x7fe0d015a6b0 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2267690021 0 0) 0x7fe0bc003520 con 0x7fe0d015a6b0 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 >> v2:172.21.15.178:3300/0 conn(0x7fe0d00b4b00 msgr2=0x7fe0d01656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 --2- 172.21.15.113:0/4074678547 >> v2:172.21.15.178:3300/0 conn(0x7fe0d00b4b00 0x7fe0d01656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 >> v1:172.21.15.178:6789/0 conn(0x7fe0d0072ea0 legacy=0x7fe0d0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d00b3010 con 0x7fe0d015a6b0 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe0cc001670 con 0x7fe0d015a6b0 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 >> v1:172.21.15.113:6789/0 conn(0x7fe0d015a6b0 legacy=0x7fe0d0070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0bc0039b0 0x7fe0bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc004800 0x7fe0bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0bc0051c0 0x7fe0bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0072a70 con 0x7fe0bc0039b0 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.167+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c060 con 0x7fe0bc0051c0 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c260 con 0x7fe0bc004800 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5778640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0bc0039b0 0x7fe0bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d4f77640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc004800 0x7fe0bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5f79640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0bc0051c0 0x7fe0bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5778640 1 -- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc004800 msgr2=0x7fe0bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5778640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc004800 0x7fe0bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5778640 1 -- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0bc0051c0 msgr2=0x7fe0bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5778640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0bc0051c0 0x7fe0bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5778640 1 -- 172.21.15.113:0/4074678547 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc003520 con 0x7fe0bc0039b0 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5f79640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0bc0051c0 0x7fe0bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5f79640 1 -- 172.21.15.113:0/4074678547 reap_dead start 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.168+0000 7fe0d5778640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0bc0039b0 0x7fe0bc003dd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c240 tx=0x7fe0bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856eb227b72c0a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc007d10 con 0x7fe0bc0039b0 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0cc0058d0 con 0x7fe0bc0039b0 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0c7fff640 1 -- 172.21.15.113:0/4074678547 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc005d40 con 0x7fe0bc0039b0 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0d79dc640 1 -- 172.21.15.113:0/4074678547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0bc0039b0 msgr2=0x7fe0bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0bc0039b0 0x7fe0bc003dd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c240 tx=0x7fe0bc005830 comp rx=0 tx=0).stop 2024-04-25T00:04:53.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0d79dc640 1 -- 172.21.15.113:0/4074678547 shutdown_connections 2024-04-25T00:04:53.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/4074678547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0bc0039b0 0x7fe0bc003dd0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0d79dc640 1 -- 172.21.15.113:0/4074678547 >> 172.21.15.113:0/4074678547 conn(0x7fe0d00b1a50 msgr2=0x7fe0d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:53.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0d79dc640 1 -- 172.21.15.113:0/4074678547 shutdown_connections 2024-04-25T00:04:53.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.169+0000 7fe0d79dc640 1 -- 172.21.15.113:0/4074678547 wait complete. 2024-04-25T00:04:53.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.170+0000 7fe0d79dc640 1 Processor -- start 2024-04-25T00:04:53.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.170+0000 7fe0d79dc640 1 -- start start 2024-04-25T00:04:53.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d79dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc0039b0 0x7fe0d0146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d79dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0d0143340 0x7fe0d0147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d79dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0d0145030 0x7fe0d01454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d79dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0072720 con 0x7fe0d0145030 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d79dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00b35d0 con 0x7fe0d0143340 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d79dc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d015aa90 con 0x7fe0bc0039b0 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0d0145030 0x7fe0d01454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0d0145030 0x7fe0d01454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47904/0 (socket says 172.21.15.113:47904) 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 -- 172.21.15.113:0/2175497598 learned_addr learned my addr 172.21.15.113:0/2175497598 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 -- 172.21.15.113:0/2175497598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc0039b0 msgr2=0x7fe0d0146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc0039b0 0x7fe0d0146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 -- 172.21.15.113:0/2175497598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0d0143340 msgr2=0x7fe0d0147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0d0143340 0x7fe0d0147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.171+0000 7fe0d5f79640 1 -- 172.21.15.113:0/2175497598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d01438b0 con 0x7fe0d0145030 2024-04-25T00:04:53.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.172+0000 7fe0d5f79640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0d0145030 0x7fe0d01454e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe0d006f2b0 tx=0x7fe0c8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.172+0000 7fe0c5ffb640 1 -- 172.21.15.113:0/2175497598 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c801a030 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.172+0000 7fe0c5ffb640 1 -- 172.21.15.113:0/2175497598 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8010bc0 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.172+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d0143b90 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.172+0000 7fe0c5ffb640 1 -- 172.21.15.113:0/2175497598 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8003990 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.172+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d0144150 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.173+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d00691a0 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.173+0000 7fe0c5ffb640 1 -- 172.21.15.113:0/2175497598 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0c801d070 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.174+0000 7fe0c5ffb640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe0b406be60 0x7fe0b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.174+0000 7fe0c5ffb640 1 -- 172.21.15.113:0/2175497598 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fe0c808b6e0 con 0x7fe0d0145030 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.174+0000 7fe0d5778640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe0b406be60 0x7fe0b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.174+0000 7fe0d5778640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe0b406be60 0x7fe0b406e320 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc033000 tx=0x7fe0cc009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.178+0000 7fe0c5ffb640 1 -- 172.21.15.113:0/2175497598 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0c805a120 con 0x7fe0d0145030 2024-04-25T00:04:53.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.352+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0d00b2900 con 0x7fe0d0145030 2024-04-25T00:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.353+0000 7fe0c5ffb640 1 -- 172.21.15.113:0/2175497598 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16667 (secure 0 0 0) 0x7fe0c8059a20 con 0x7fe0d0145030 2024-04-25T00:04:53.354 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:53.354 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:51.690687+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:48.259110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:49.014993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe0b406be60 msgr2=0x7fe0b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe0b406be60 0x7fe0b406e320 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc033000 tx=0x7fe0cc009030 comp rx=0 tx=0).stop 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0d0145030 msgr2=0x7fe0d01454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0d0145030 0x7fe0d01454e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe0d006f2b0 tx=0x7fe0c8010590 comp rx=0 tx=0).stop 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 shutdown_connections 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe0b406be60 0x7fe0b406e320 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0d0143340 0x7fe0d0147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0d0145030 0x7fe0d01454e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 --2- 172.21.15.113:0/2175497598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0bc0039b0 0x7fe0d0146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 >> 172.21.15.113:0/2175497598 conn(0x7fe0d00b1a50 msgr2=0x7fe0d00ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 shutdown_connections 2024-04-25T00:04:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.355+0000 7fe0d79dc640 1 -- 172.21.15.113:0/2175497598 wait complete. 2024-04-25T00:04:53.367 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:04:53.367 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:04:53.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.463+0000 7ff75c9e7640 1 Processor -- start 2024-04-25T00:04:53.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.463+0000 7ff75c9e7640 1 -- start start 2024-04-25T00:04:53.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff75c9e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 0x7ff758070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff75c9e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7580b2d50 con 0x7ff758072ea0 2024-04-25T00:04:53.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff75c9e7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff758072a70 con 0x7ff7580b3af0 2024-04-25T00:04:53.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff75c9e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff7580b3010 con 0x7ff75815a6b0 2024-04-25T00:04:53.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff755d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff758072ea0 0x7ff758071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38218/0 (socket says 172.21.15.113:38218) 2024-04-25T00:04:53.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff755d9b640 1 -- 172.21.15.113:0/3656725309 learned_addr learned my addr 172.21.15.113:0/3656725309 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:53.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff75659c640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 0x7ff758070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2570559391 0 0) 0x7ff7580b2d50 con 0x7ff758072ea0 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.464+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff740003520 con 0x7ff758072ea0 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1594565914 0 0) 0x7ff740003520 con 0x7ff758072ea0 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 msgr2=0x7ff758070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 0x7ff758070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 >> v1:172.21.15.178:6789/0 conn(0x7ff7580b3af0 legacy=0x7ff7580b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7580b32f0 con 0x7ff758072ea0 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff74c001670 con 0x7ff758072ea0 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 >> v1:172.21.15.113:6789/0 conn(0x7ff758072ea0 legacy=0x7ff758071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 0x7ff7400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff740003fb0 0x7ff740004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff7400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff7580b3010 con 0x7ff740004d40 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff74000bb00 con 0x7ff740003fb0 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff74000bd00 con 0x7ff75815a6b0 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff756d9d640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff7400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff75659c640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 0x7ff7400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff755d9b640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff740003fb0 0x7ff740004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff756d9d640 1 -- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 msgr2=0x7ff7400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff756d9d640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75815a6b0 0x7ff7400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff756d9d640 1 -- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff740003fb0 msgr2=0x7ff740004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff756d9d640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff740003fb0 0x7ff740004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.465+0000 7ff756d9d640 1 -- 172.21.15.113:0/3656725309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff740003520 con 0x7ff740004d40 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.466+0000 7ff756d9d640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff7400051c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff750001750 tx=0x7ff740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b0efcad9dcfa44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.466+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750001370 con 0x7ff740004d40 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.466+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7500052d0 con 0x7ff740004d40 2024-04-25T00:04:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.466+0000 7ff75559a640 1 -- 172.21.15.113:0/3656725309 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75002abe0 con 0x7ff740004d40 2024-04-25T00:04:53.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.467+0000 7ff75c9e7640 1 -- 172.21.15.113:0/3656725309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 msgr2=0x7ff7400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.467+0000 7ff75c9e7640 1 --2- 172.21.15.113:0/3656725309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff7400051c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff750001750 tx=0x7ff740005830 comp rx=0 tx=0).stop 2024-04-25T00:04:53.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.467+0000 7ff756d9d640 1 -- 172.21.15.113:0/3656725309 reap_dead start 2024-04-25T00:04:53.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.467+0000 7ff75c9e7640 1 -- 172.21.15.113:0/3656725309 shutdown_connections 2024-04-25T00:04:53.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.467+0000 7ff75c9e7640 1 -- 172.21.15.113:0/3656725309 >> 172.21.15.113:0/3656725309 conn(0x7ff7580b1a50 msgr2=0x7ff7580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:53.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.467+0000 7ff75c9e7640 1 -- 172.21.15.113:0/3656725309 shutdown_connections 2024-04-25T00:04:53.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.467+0000 7ff75c9e7640 1 -- 172.21.15.113:0/3656725309 wait complete. 2024-04-25T00:04:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.468+0000 7ff75c9e7640 1 Processor -- start 2024-04-25T00:04:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.468+0000 7ff75c9e7640 1 -- start start 2024-04-25T00:04:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff758137810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75813b0c0 0x7ff7580ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff75814e780 0x7ff75814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff75815aa90 con 0x7ff740004d40 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff758072720 con 0x7ff75814e780 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff75815ac10 con 0x7ff75813b0c0 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff758137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff758137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47922/0 (socket says 172.21.15.113:47922) 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 -- 172.21.15.113:0/2849313648 learned_addr learned my addr 172.21.15.113:0/2849313648 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff755d9b640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75813b0c0 0x7ff7580ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff756d9d640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff75814e780 0x7ff75814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 -- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75813b0c0 msgr2=0x7ff7580ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75813b0c0 0x7ff7580ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 -- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff75814e780 msgr2=0x7ff75814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff75814e780 0x7ff75814ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 -- 172.21.15.113:0/2849313648 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7580ceca0 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff755d9b640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75813b0c0 0x7ff7580ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75659c640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff758137810 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff748011e60 tx=0x7ff74800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff746ffd640 1 -- 172.21.15.113:0/2849313648 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748015630 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff746ffd640 1 -- 172.21.15.113:0/2849313648 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7480157d0 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75813d200 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff746ffd640 1 -- 172.21.15.113:0/2849313648 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748015ba0 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.469+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75813d780 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.471+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7580691a0 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.471+0000 7ff746ffd640 1 -- 172.21.15.113:0/2849313648 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff74801d400 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.471+0000 7ff746ffd640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff73806be60 0x7ff73806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.472+0000 7ff755d9b640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff73806be60 0x7ff73806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.472+0000 7ff746ffd640 1 -- 172.21.15.113:0/2849313648 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7ff74808bf50 con 0x7ff740004d40 2024-04-25T00:04:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.472+0000 7ff755d9b640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff73806be60 0x7ff73806e320 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff74c000ff0 tx=0x7ff74c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.476+0000 7ff746ffd640 1 -- 172.21.15.113:0/2849313648 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff748019030 con 0x7ff740004d40 2024-04-25T00:04:53.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.644+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7580b2900 con 0x7ff73806be60 2024-04-25T00:04:53.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.650+0000 7ff746ffd640 1 -- 172.21.15.113:0/2849313648 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252416 (secure 0 0 0) 0x7ff7580b2900 con 0x7ff73806be60 2024-04-25T00:04:53.653 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:53.655 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:04:53.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff73806be60 msgr2=0x7ff73806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff73806be60 0x7ff73806e320 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff74c000ff0 tx=0x7ff74c009040 comp rx=0 tx=0).stop 2024-04-25T00:04:53.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 msgr2=0x7ff758137810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff758137810 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff748011e60 tx=0x7ff74800f580 comp rx=0 tx=0).stop 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 shutdown_connections 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff73806be60 0x7ff73806e320 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff75814e780 0x7ff75814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75813b0c0 0x7ff7580ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 --2- 172.21.15.113:0/2849313648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff740004d40 0x7ff758137810 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 >> 172.21.15.113:0/2849313648 conn(0x7ff7580b1a50 msgr2=0x7ff758150210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 shutdown_connections 2024-04-25T00:04:53.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.653+0000 7ff75c9e7640 1 -- 172.21.15.113:0/2849313648 wait complete. 2024-04-25T00:04:53.665 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-25T00:04:53.189273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":396532,"kb_used_data":11064,"kb_used_omap":12,"kb_used_meta":385331,"kb_avail":754578188,"statfs":{"total":773094113280,"available":772688064512,"internally_reserved":0,"allocated":11329536,"data_stored":5551795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394579504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.505597"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.711316+0000","last_change":"2024-04-25T00:04:50.710708+0000","last_active":"2024-04-25T00:04:50.711316+0000","last_peered":"2024-04-25T00:04:50.711316+0000","last_clean":"2024-04-25T00:04:50.711316+0000","last_became_active":"2024-04-25T00:04:50.710152+0000","last_became_peered":"2024-04-25T00:04:50.710152+0000","last_unstale":"2024-04-25T00:04:50.711316+0000","last_undegraded":"2024-04-25T00:04:50.711316+0000","last_fullsized":"2024-04-25T00:04:50.711316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:40.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718747+0000","last_change":"2024-04-25T00:04:50.708247+0000","last_active":"2024-04-25T00:04:50.718747+0000","last_peered":"2024-04-25T00:04:50.718747+0000","last_clean":"2024-04-25T00:04:50.718747+0000","last_became_active":"2024-04-25T00:04:50.707274+0000","last_became_peered":"2024-04-25T00:04:50.707274+0000","last_unstale":"2024-04-25T00:04:50.718747+0000","last_undegraded":"2024-04-25T00:04:50.718747+0000","last_fullsized":"2024-04-25T00:04:50.718747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:51.057870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709187+0000","last_change":"2024-04-25T00:04:50.705854+0000","last_active":"2024-04-25T00:04:50.709187+0000","last_peered":"2024-04-25T00:04:50.709187+0000","last_clean":"2024-04-25T00:04:50.709187+0000","last_became_active":"2024-04-25T00:04:50.705584+0000","last_became_peered":"2024-04-25T00:04:50.705584+0000","last_unstale":"2024-04-25T00:04:50.709187+0000","last_undegraded":"2024-04-25T00:04:50.709187+0000","last_fullsized":"2024-04-25T00:04:50.709187+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:29.450392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714858+0000","last_change":"2024-04-25T00:04:50.712263+0000","last_active":"2024-04-25T00:04:50.714858+0000","last_peered":"2024-04-25T00:04:50.714858+0000","last_clean":"2024-04-25T00:04:50.714858+0000","last_became_active":"2024-04-25T00:04:50.712011+0000","last_became_peered":"2024-04-25T00:04:50.712011+0000","last_unstale":"2024-04-25T00:04:50.714858+0000","last_undegraded":"2024-04-25T00:04:50.714858+0000","last_fullsized":"2024-04-25T00:04:50.714858+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:55.460510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714834+0000","last_change":"2024-04-25T00:04:50.708165+0000","last_active":"2024-04-25T00:04:50.714834+0000","last_peered":"2024-04-25T00:04:50.714834+0000","last_clean":"2024-04-25T00:04:50.714834+0000","last_became_active":"2024-04-25T00:04:50.707885+0000","last_became_peered":"2024-04-25T00:04:50.707885+0000","last_unstale":"2024-04-25T00:04:50.714834+0000","last_undegraded":"2024-04-25T00:04:50.714834+0000","last_fullsized":"2024-04-25T00:04:50.714834+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:32.038973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.712536+0000","last_change":"2024-04-25T00:04:50.711592+0000","last_active":"2024-04-25T00:04:50.712536+0000","last_peered":"2024-04-25T00:04:50.712536+0000","last_clean":"2024-04-25T00:04:50.712536+0000","last_became_active":"2024-04-25T00:04:50.711290+0000","last_became_peered":"2024-04-25T00:04:50.711290+0000","last_unstale":"2024-04-25T00:04:50.712536+0000","last_undegraded":"2024-04-25T00:04:50.712536+0000","last_fullsized":"2024-04-25T00:04:50.712536+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:51.846162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710858+0000","last_change":"2024-04-25T00:04:50.708327+0000","last_active":"2024-04-25T00:04:50.710858+0000","last_peered":"2024-04-25T00:04:50.710858+0000","last_clean":"2024-04-25T00:04:50.710858+0000","last_became_active":"2024-04-25T00:04:50.708055+0000","last_became_peered":"2024-04-25T00:04:50.708055+0000","last_unstale":"2024-04-25T00:04:50.710858+0000","last_undegraded":"2024-04-25T00:04:50.710858+0000","last_fullsized":"2024-04-25T00:04:50.710858+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:17.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719481+0000","last_change":"2024-04-25T00:04:50.709416+0000","last_active":"2024-04-25T00:04:50.719481+0000","last_peered":"2024-04-25T00:04:50.719481+0000","last_clean":"2024-04-25T00:04:50.719481+0000","last_became_active":"2024-04-25T00:04:50.709137+0000","last_became_peered":"2024-04-25T00:04:50.709137+0000","last_unstale":"2024-04-25T00:04:50.719481+0000","last_undegraded":"2024-04-25T00:04:50.719481+0000","last_fullsized":"2024-04-25T00:04:50.719481+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:17.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717176+0000","last_change":"2024-04-25T00:04:50.714210+0000","last_active":"2024-04-25T00:04:50.717176+0000","last_peered":"2024-04-25T00:04:50.717176+0000","last_clean":"2024-04-25T00:04:50.717176+0000","last_became_active":"2024-04-25T00:04:50.713959+0000","last_became_peered":"2024-04-25T00:04:50.713959+0000","last_unstale":"2024-04-25T00:04:50.717176+0000","last_undegraded":"2024-04-25T00:04:50.717176+0000","last_fullsized":"2024-04-25T00:04:50.717176+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:49.814304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715274+0000","last_change":"2024-04-25T00:04:50.712606+0000","last_active":"2024-04-25T00:04:50.715274+0000","last_peered":"2024-04-25T00:04:50.715274+0000","last_clean":"2024-04-25T00:04:50.715274+0000","last_became_active":"2024-04-25T00:04:50.712332+0000","last_became_peered":"2024-04-25T00:04:50.712332+0000","last_unstale":"2024-04-25T00:04:50.715274+0000","last_undegraded":"2024-04-25T00:04:50.715274+0000","last_fullsized":"2024-04-25T00:04:50.715274+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:31.534036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718780+0000","last_change":"2024-04-25T00:04:50.714176+0000","last_active":"2024-04-25T00:04:50.718780+0000","last_peered":"2024-04-25T00:04:50.718780+0000","last_clean":"2024-04-25T00:04:50.718780+0000","last_became_active":"2024-04-25T00:04:50.713642+0000","last_became_peered":"2024-04-25T00:04:50.713642+0000","last_unstale":"2024-04-25T00:04:50.718780+0000","last_undegraded":"2024-04-25T00:04:50.718780+0000","last_fullsized":"2024-04-25T00:04:50.718780+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:07.941150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.702633+0000","last_change":"2024-04-25T00:04:50.698815+0000","last_active":"2024-04-25T00:04:50.702633+0000","last_peered":"2024-04-25T00:04:50.702633+0000","last_clean":"2024-04-25T00:04:50.702633+0000","last_became_active":"2024-04-25T00:04:50.698489+0000","last_became_peered":"2024-04-25T00:04:50.698489+0000","last_unstale":"2024-04-25T00:04:50.702633+0000","last_undegraded":"2024-04-25T00:04:50.702633+0000","last_fullsized":"2024-04-25T00:04:50.702633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:46.915644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.725414+0000","last_change":"2024-04-25T00:04:50.720321+0000","last_active":"2024-04-25T00:04:50.725414+0000","last_peered":"2024-04-25T00:04:50.725414+0000","last_clean":"2024-04-25T00:04:50.725414+0000","last_became_active":"2024-04-25T00:04:50.720044+0000","last_became_peered":"2024-04-25T00:04:50.720044+0000","last_unstale":"2024-04-25T00:04:50.725414+0000","last_undegraded":"2024-04-25T00:04:50.725414+0000","last_fullsized":"2024-04-25T00:04:50.725414+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:51.057870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720368+0000","last_change":"2024-04-25T00:04:50.718821+0000","last_active":"2024-04-25T00:04:50.720368+0000","last_peered":"2024-04-25T00:04:50.720368+0000","last_clean":"2024-04-25T00:04:50.720368+0000","last_became_active":"2024-04-25T00:04:50.718422+0000","last_became_peered":"2024-04-25T00:04:50.718422+0000","last_unstale":"2024-04-25T00:04:50.720368+0000","last_undegraded":"2024-04-25T00:04:50.720368+0000","last_fullsized":"2024-04-25T00:04:50.720368+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:46.941535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714897+0000","last_change":"2024-04-25T00:04:50.711323+0000","last_active":"2024-04-25T00:04:50.714897+0000","last_peered":"2024-04-25T00:04:50.714897+0000","last_clean":"2024-04-25T00:04:50.714897+0000","last_became_active":"2024-04-25T00:04:50.711045+0000","last_became_peered":"2024-04-25T00:04:50.711045+0000","last_unstale":"2024-04-25T00:04:50.714897+0000","last_undegraded":"2024-04-25T00:04:50.714897+0000","last_fullsized":"2024-04-25T00:04:50.714897+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:26.448731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719204+0000","last_change":"2024-04-25T00:04:50.718213+0000","last_active":"2024-04-25T00:04:50.719204+0000","last_peered":"2024-04-25T00:04:50.719204+0000","last_clean":"2024-04-25T00:04:50.719204+0000","last_became_active":"2024-04-25T00:04:50.717911+0000","last_became_peered":"2024-04-25T00:04:50.717911+0000","last_unstale":"2024-04-25T00:04:50.719204+0000","last_undegraded":"2024-04-25T00:04:50.719204+0000","last_fullsized":"2024-04-25T00:04:50.719204+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:45:45.068235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718837+0000","last_change":"2024-04-25T00:04:50.709213+0000","last_active":"2024-04-25T00:04:50.718837+0000","last_peered":"2024-04-25T00:04:50.718837+0000","last_clean":"2024-04-25T00:04:50.718837+0000","last_became_active":"2024-04-25T00:04:50.708855+0000","last_became_peered":"2024-04-25T00:04:50.708855+0000","last_unstale":"2024-04-25T00:04:50.718837+0000","last_undegraded":"2024-04-25T00:04:50.718837+0000","last_fullsized":"2024-04-25T00:04:50.718837+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:36.656777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717106+0000","last_change":"2024-04-25T00:04:50.713693+0000","last_active":"2024-04-25T00:04:50.717106+0000","last_peered":"2024-04-25T00:04:50.717106+0000","last_clean":"2024-04-25T00:04:50.717106+0000","last_became_active":"2024-04-25T00:04:50.713415+0000","last_became_peered":"2024-04-25T00:04:50.713415+0000","last_unstale":"2024-04-25T00:04:50.717106+0000","last_undegraded":"2024-04-25T00:04:50.717106+0000","last_fullsized":"2024-04-25T00:04:50.717106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:16.829314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709451+0000","last_change":"2024-04-25T00:04:50.704862+0000","last_active":"2024-04-25T00:04:50.709451+0000","last_peered":"2024-04-25T00:04:50.709451+0000","last_clean":"2024-04-25T00:04:50.709451+0000","last_became_active":"2024-04-25T00:04:50.704385+0000","last_became_peered":"2024-04-25T00:04:50.704385+0000","last_unstale":"2024-04-25T00:04:50.709451+0000","last_undegraded":"2024-04-25T00:04:50.709451+0000","last_fullsized":"2024-04-25T00:04:50.709451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:27.139941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717387+0000","last_change":"2024-04-25T00:04:50.714676+0000","last_active":"2024-04-25T00:04:50.717387+0000","last_peered":"2024-04-25T00:04:50.717387+0000","last_clean":"2024-04-25T00:04:50.717387+0000","last_became_active":"2024-04-25T00:04:50.713734+0000","last_became_peered":"2024-04-25T00:04:50.713734+0000","last_unstale":"2024-04-25T00:04:50.717387+0000","last_undegraded":"2024-04-25T00:04:50.717387+0000","last_fullsized":"2024-04-25T00:04:50.717387+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:26.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709515+0000","last_change":"2024-04-25T00:04:50.705417+0000","last_active":"2024-04-25T00:04:50.709515+0000","last_peered":"2024-04-25T00:04:50.709515+0000","last_clean":"2024-04-25T00:04:50.709515+0000","last_became_active":"2024-04-25T00:04:50.705153+0000","last_became_peered":"2024-04-25T00:04:50.705153+0000","last_unstale":"2024-04-25T00:04:50.709515+0000","last_undegraded":"2024-04-25T00:04:50.709515+0000","last_fullsized":"2024-04-25T00:04:50.709515+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:59.481396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718809+0000","last_change":"2024-04-25T00:04:50.709000+0000","last_active":"2024-04-25T00:04:50.718809+0000","last_peered":"2024-04-25T00:04:50.718809+0000","last_clean":"2024-04-25T00:04:50.718809+0000","last_became_active":"2024-04-25T00:04:50.708513+0000","last_became_peered":"2024-04-25T00:04:50.708513+0000","last_unstale":"2024-04-25T00:04:50.718809+0000","last_undegraded":"2024-04-25T00:04:50.718809+0000","last_fullsized":"2024-04-25T00:04:50.718809+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:22.029364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719073+0000","last_change":"2024-04-25T00:04:50.706975+0000","last_active":"2024-04-25T00:04:50.719073+0000","last_peered":"2024-04-25T00:04:50.719073+0000","last_clean":"2024-04-25T00:04:50.719073+0000","last_became_active":"2024-04-25T00:04:50.706695+0000","last_became_peered":"2024-04-25T00:04:50.706695+0000","last_unstale":"2024-04-25T00:04:50.719073+0000","last_undegraded":"2024-04-25T00:04:50.719073+0000","last_fullsized":"2024-04-25T00:04:50.719073+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:27.727405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717832+0000","last_change":"2024-04-25T00:04:50.715096+0000","last_active":"2024-04-25T00:04:50.717832+0000","last_peered":"2024-04-25T00:04:50.717832+0000","last_clean":"2024-04-25T00:04:50.717832+0000","last_became_active":"2024-04-25T00:04:50.714538+0000","last_became_peered":"2024-04-25T00:04:50.714538+0000","last_unstale":"2024-04-25T00:04:50.717832+0000","last_undegraded":"2024-04-25T00:04:50.717832+0000","last_fullsized":"2024-04-25T00:04:50.717832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:40.169163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719311+0000","last_change":"2024-04-25T00:04:50.717877+0000","last_active":"2024-04-25T00:04:50.719311+0000","last_peered":"2024-04-25T00:04:50.719311+0000","last_clean":"2024-04-25T00:04:50.719311+0000","last_became_active":"2024-04-25T00:04:50.717605+0000","last_became_peered":"2024-04-25T00:04:50.717605+0000","last_unstale":"2024-04-25T00:04:50.719311+0000","last_undegraded":"2024-04-25T00:04:50.719311+0000","last_fullsized":"2024-04-25T00:04:50.719311+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:37.576586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.708778+0000","last_change":"2024-04-25T00:04:50.699922+0000","last_active":"2024-04-25T00:04:50.708778+0000","last_peered":"2024-04-25T00:04:50.708778+0000","last_clean":"2024-04-25T00:04:50.708778+0000","last_became_active":"2024-04-25T00:04:50.699643+0000","last_became_peered":"2024-04-25T00:04:50.699643+0000","last_unstale":"2024-04-25T00:04:50.708778+0000","last_undegraded":"2024-04-25T00:04:50.708778+0000","last_fullsized":"2024-04-25T00:04:50.708778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:28.797591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718754+0000","last_change":"2024-04-25T00:04:50.715520+0000","last_active":"2024-04-25T00:04:50.718754+0000","last_peered":"2024-04-25T00:04:50.718754+0000","last_clean":"2024-04-25T00:04:50.718754+0000","last_became_active":"2024-04-25T00:04:50.715262+0000","last_became_peered":"2024-04-25T00:04:50.715262+0000","last_unstale":"2024-04-25T00:04:50.718754+0000","last_undegraded":"2024-04-25T00:04:50.718754+0000","last_fullsized":"2024-04-25T00:04:50.718754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:40.601371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719453+0000","last_change":"2024-04-25T00:04:50.718604+0000","last_active":"2024-04-25T00:04:50.719453+0000","last_peered":"2024-04-25T00:04:50.719453+0000","last_clean":"2024-04-25T00:04:50.719453+0000","last_became_active":"2024-04-25T00:04:50.717127+0000","last_became_peered":"2024-04-25T00:04:50.717127+0000","last_unstale":"2024-04-25T00:04:50.719453+0000","last_undegraded":"2024-04-25T00:04:50.719453+0000","last_fullsized":"2024-04-25T00:04:50.719453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:52.556475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.708819+0000","last_change":"2024-04-25T00:04:50.699874+0000","last_active":"2024-04-25T00:04:50.708819+0000","last_peered":"2024-04-25T00:04:50.708819+0000","last_clean":"2024-04-25T00:04:50.708819+0000","last_became_active":"2024-04-25T00:04:50.699585+0000","last_became_peered":"2024-04-25T00:04:50.699585+0000","last_unstale":"2024-04-25T00:04:50.708819+0000","last_undegraded":"2024-04-25T00:04:50.708819+0000","last_fullsized":"2024-04-25T00:04:50.708819+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:16.829314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.711867+0000","last_change":"2024-04-25T00:04:50.705759+0000","last_active":"2024-04-25T00:04:50.711867+0000","last_peered":"2024-04-25T00:04:50.711867+0000","last_clean":"2024-04-25T00:04:50.711867+0000","last_became_active":"2024-04-25T00:04:50.704542+0000","last_became_peered":"2024-04-25T00:04:50.704542+0000","last_unstale":"2024-04-25T00:04:50.711867+0000","last_undegraded":"2024-04-25T00:04:50.711867+0000","last_fullsized":"2024-04-25T00:04:50.711867+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:30.595333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720337+0000","last_change":"2024-04-25T00:04:50.717537+0000","last_active":"2024-04-25T00:04:50.720337+0000","last_peered":"2024-04-25T00:04:50.720337+0000","last_clean":"2024-04-25T00:04:50.720337+0000","last_became_active":"2024-04-25T00:04:50.717140+0000","last_became_peered":"2024-04-25T00:04:50.717140+0000","last_unstale":"2024-04-25T00:04:50.720337+0000","last_undegraded":"2024-04-25T00:04:50.720337+0000","last_fullsized":"2024-04-25T00:04:50.720337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:27.727405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709844+0000","last_change":"2024-04-25T00:04:50.706241+0000","last_active":"2024-04-25T00:04:50.709844+0000","last_peered":"2024-04-25T00:04:50.709844+0000","last_clean":"2024-04-25T00:04:50.709844+0000","last_became_active":"2024-04-25T00:04:50.702182+0000","last_became_peered":"2024-04-25T00:04:50.702182+0000","last_unstale":"2024-04-25T00:04:50.709844+0000","last_undegraded":"2024-04-25T00:04:50.709844+0000","last_fullsized":"2024-04-25T00:04:50.709844+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:30.036880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710448+0000","last_change":"2024-04-25T00:04:50.706712+0000","last_active":"2024-04-25T00:04:50.710448+0000","last_peered":"2024-04-25T00:04:50.710448+0000","last_clean":"2024-04-25T00:04:50.710448+0000","last_became_active":"2024-04-25T00:04:50.706461+0000","last_became_peered":"2024-04-25T00:04:50.706461+0000","last_unstale":"2024-04-25T00:04:50.710448+0000","last_undegraded":"2024-04-25T00:04:50.710448+0000","last_fullsized":"2024-04-25T00:04:50.710448+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:04.560398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721863+0000","last_change":"2024-04-25T00:04:49.700092+0000","last_active":"2024-04-25T00:04:50.721863+0000","last_peered":"2024-04-25T00:04:50.721863+0000","last_clean":"2024-04-25T00:04:50.721863+0000","last_became_active":"2024-04-25T00:04:49.699806+0000","last_became_peered":"2024-04-25T00:04:49.699806+0000","last_unstale":"2024-04-25T00:04:50.721863+0000","last_undegraded":"2024-04-25T00:04:50.721863+0000","last_fullsized":"2024-04-25T00:04:50.721863+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:36.571143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719208+0000","last_change":"2024-04-25T00:04:50.716389+0000","last_active":"2024-04-25T00:04:50.719208+0000","last_peered":"2024-04-25T00:04:50.719208+0000","last_clean":"2024-04-25T00:04:50.719208+0000","last_became_active":"2024-04-25T00:04:50.716129+0000","last_became_peered":"2024-04-25T00:04:50.716129+0000","last_unstale":"2024-04-25T00:04:50.719208+0000","last_undegraded":"2024-04-25T00:04:50.719208+0000","last_fullsized":"2024-04-25T00:04:50.719208+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:28.797591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720700+0000","last_change":"2024-04-25T00:04:49.705248+0000","last_active":"2024-04-25T00:04:50.720700+0000","last_peered":"2024-04-25T00:04:50.720700+0000","last_clean":"2024-04-25T00:04:50.720700+0000","last_became_active":"2024-04-25T00:04:49.704923+0000","last_became_peered":"2024-04-25T00:04:49.704923+0000","last_unstale":"2024-04-25T00:04:50.720700+0000","last_undegraded":"2024-04-25T00:04:50.720700+0000","last_fullsized":"2024-04-25T00:04:50.720700+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:50.052426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.702097+0000","last_change":"2024-04-25T00:04:50.700009+0000","last_active":"2024-04-25T00:04:50.702097+0000","last_peered":"2024-04-25T00:04:50.702097+0000","last_clean":"2024-04-25T00:04:50.702097+0000","last_became_active":"2024-04-25T00:04:50.699723+0000","last_became_peered":"2024-04-25T00:04:50.699723+0000","last_unstale":"2024-04-25T00:04:50.702097+0000","last_undegraded":"2024-04-25T00:04:50.702097+0000","last_fullsized":"2024-04-25T00:04:50.702097+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:15.932352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.722171+0000","last_change":"2024-04-25T00:04:49.707165+0000","last_active":"2024-04-25T00:04:50.722171+0000","last_peered":"2024-04-25T00:04:50.722171+0000","last_clean":"2024-04-25T00:04:50.722171+0000","last_became_active":"2024-04-25T00:04:49.706863+0000","last_became_peered":"2024-04-25T00:04:49.706863+0000","last_unstale":"2024-04-25T00:04:50.722171+0000","last_undegraded":"2024-04-25T00:04:50.722171+0000","last_fullsized":"2024-04-25T00:04:50.722171+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:25.443288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709528+0000","last_change":"2024-04-25T00:04:50.705884+0000","last_active":"2024-04-25T00:04:50.709528+0000","last_peered":"2024-04-25T00:04:50.709528+0000","last_clean":"2024-04-25T00:04:50.709528+0000","last_became_active":"2024-04-25T00:04:50.705594+0000","last_became_peered":"2024-04-25T00:04:50.705594+0000","last_unstale":"2024-04-25T00:04:50.709528+0000","last_undegraded":"2024-04-25T00:04:50.709528+0000","last_fullsized":"2024-04-25T00:04:50.709528+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:37.576586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718811+0000","last_change":"2024-04-25T00:04:49.701549+0000","last_active":"2024-04-25T00:04:50.718811+0000","last_peered":"2024-04-25T00:04:50.718811+0000","last_clean":"2024-04-25T00:04:50.718811+0000","last_became_active":"2024-04-25T00:04:49.700122+0000","last_became_peered":"2024-04-25T00:04:49.700122+0000","last_unstale":"2024-04-25T00:04:50.718811+0000","last_undegraded":"2024-04-25T00:04:50.718811+0000","last_fullsized":"2024-04-25T00:04:50.718811+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:51.551032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.711453+0000","last_change":"2024-04-25T00:04:50.709917+0000","last_active":"2024-04-25T00:04:50.711453+0000","last_peered":"2024-04-25T00:04:50.711453+0000","last_clean":"2024-04-25T00:04:50.711453+0000","last_became_active":"2024-04-25T00:04:50.709668+0000","last_became_peered":"2024-04-25T00:04:50.709668+0000","last_unstale":"2024-04-25T00:04:50.711453+0000","last_undegraded":"2024-04-25T00:04:50.711453+0000","last_fullsized":"2024-04-25T00:04:50.711453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:46.915644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719743+0000","last_change":"2024-04-25T00:04:49.698677+0000","last_active":"2024-04-25T00:04:50.719743+0000","last_peered":"2024-04-25T00:04:50.719743+0000","last_clean":"2024-04-25T00:04:50.719743+0000","last_became_active":"2024-04-25T00:04:49.698343+0000","last_became_peered":"2024-04-25T00:04:49.698343+0000","last_unstale":"2024-04-25T00:04:50.719743+0000","last_undegraded":"2024-04-25T00:04:50.719743+0000","last_fullsized":"2024-04-25T00:04:50.719743+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:46.442408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710634+0000","last_change":"2024-04-25T00:04:50.709182+0000","last_active":"2024-04-25T00:04:50.710634+0000","last_peered":"2024-04-25T00:04:50.710634+0000","last_clean":"2024-04-25T00:04:50.710634+0000","last_became_active":"2024-04-25T00:04:50.708917+0000","last_became_peered":"2024-04-25T00:04:50.708917+0000","last_unstale":"2024-04-25T00:04:50.710634+0000","last_undegraded":"2024-04-25T00:04:50.710634+0000","last_fullsized":"2024-04-25T00:04:50.710634+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:22.184636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720662+0000","last_change":"2024-04-25T00:04:49.697066+0000","last_active":"2024-04-25T00:04:50.720662+0000","last_peered":"2024-04-25T00:04:50.720662+0000","last_clean":"2024-04-25T00:04:50.720662+0000","last_became_active":"2024-04-25T00:04:49.696724+0000","last_became_peered":"2024-04-25T00:04:49.696724+0000","last_unstale":"2024-04-25T00:04:50.720662+0000","last_undegraded":"2024-04-25T00:04:50.720662+0000","last_fullsized":"2024-04-25T00:04:50.720662+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:08.738132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714849+0000","last_change":"2024-04-25T00:04:50.711416+0000","last_active":"2024-04-25T00:04:50.714849+0000","last_peered":"2024-04-25T00:04:50.714849+0000","last_clean":"2024-04-25T00:04:50.714849+0000","last_became_active":"2024-04-25T00:04:50.711151+0000","last_became_peered":"2024-04-25T00:04:50.711151+0000","last_unstale":"2024-04-25T00:04:50.714849+0000","last_undegraded":"2024-04-25T00:04:50.714849+0000","last_fullsized":"2024-04-25T00:04:50.714849+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:18.525910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.699988+0000","last_change":"2024-04-25T00:04:49.702715+0000","last_active":"2024-04-25T00:04:50.699988+0000","last_peered":"2024-04-25T00:04:50.699988+0000","last_clean":"2024-04-25T00:04:50.699988+0000","last_became_active":"2024-04-25T00:04:49.701768+0000","last_became_peered":"2024-04-25T00:04:49.701768+0000","last_unstale":"2024-04-25T00:04:50.699988+0000","last_undegraded":"2024-04-25T00:04:50.699988+0000","last_fullsized":"2024-04-25T00:04:50.699988+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.765807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715591+0000","last_change":"2024-04-25T00:04:50.707612+0000","last_active":"2024-04-25T00:04:50.715591+0000","last_peered":"2024-04-25T00:04:50.715591+0000","last_clean":"2024-04-25T00:04:50.715591+0000","last_became_active":"2024-04-25T00:04:50.706935+0000","last_became_peered":"2024-04-25T00:04:50.706935+0000","last_unstale":"2024-04-25T00:04:50.715591+0000","last_undegraded":"2024-04-25T00:04:50.715591+0000","last_fullsized":"2024-04-25T00:04:50.715591+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:52.556475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.697229+0000","last_change":"2024-04-25T00:04:49.702798+0000","last_active":"2024-04-25T00:04:50.697229+0000","last_peered":"2024-04-25T00:04:50.697229+0000","last_clean":"2024-04-25T00:04:50.697229+0000","last_became_active":"2024-04-25T00:04:49.702011+0000","last_became_peered":"2024-04-25T00:04:49.702011+0000","last_unstale":"2024-04-25T00:04:50.697229+0000","last_undegraded":"2024-04-25T00:04:50.697229+0000","last_fullsized":"2024-04-25T00:04:50.697229+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:45.936092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.699296+0000","last_change":"2024-04-25T00:04:50.697948+0000","last_active":"2024-04-25T00:04:50.699296+0000","last_peered":"2024-04-25T00:04:50.699296+0000","last_clean":"2024-04-25T00:04:50.699296+0000","last_became_active":"2024-04-25T00:04:50.697662+0000","last_became_peered":"2024-04-25T00:04:50.697662+0000","last_unstale":"2024-04-25T00:04:50.699296+0000","last_undegraded":"2024-04-25T00:04:50.699296+0000","last_fullsized":"2024-04-25T00:04:50.699296+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:55.460510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.696289+0000","last_change":"2024-04-25T00:04:49.704325+0000","last_active":"2024-04-25T00:04:50.696289+0000","last_peered":"2024-04-25T00:04:50.696289+0000","last_clean":"2024-04-25T00:04:50.696289+0000","last_became_active":"2024-04-25T00:04:49.703972+0000","last_became_peered":"2024-04-25T00:04:49.703972+0000","last_unstale":"2024-04-25T00:04:50.696289+0000","last_undegraded":"2024-04-25T00:04:50.696289+0000","last_fullsized":"2024-04-25T00:04:50.696289+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:39.357772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709680+0000","last_change":"2024-04-25T00:04:50.708479+0000","last_active":"2024-04-25T00:04:50.709680+0000","last_peered":"2024-04-25T00:04:50.709680+0000","last_clean":"2024-04-25T00:04:50.709680+0000","last_became_active":"2024-04-25T00:04:50.707209+0000","last_became_peered":"2024-04-25T00:04:50.707209+0000","last_unstale":"2024-04-25T00:04:50.709680+0000","last_undegraded":"2024-04-25T00:04:50.709680+0000","last_fullsized":"2024-04-25T00:04:50.709680+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:00.423063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.703672+0000","last_change":"2024-04-25T00:04:49.707114+0000","last_active":"2024-04-25T00:04:50.703672+0000","last_peered":"2024-04-25T00:04:50.703672+0000","last_clean":"2024-04-25T00:04:50.703672+0000","last_became_active":"2024-04-25T00:04:49.706823+0000","last_became_peered":"2024-04-25T00:04:49.706823+0000","last_unstale":"2024-04-25T00:04:50.703672+0000","last_undegraded":"2024-04-25T00:04:50.703672+0000","last_fullsized":"2024-04-25T00:04:50.703672+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:48.808861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719664+0000","last_change":"2024-04-25T00:04:50.719086+0000","last_active":"2024-04-25T00:04:50.719664+0000","last_peered":"2024-04-25T00:04:50.719664+0000","last_clean":"2024-04-25T00:04:50.719664+0000","last_became_active":"2024-04-25T00:04:50.718128+0000","last_became_peered":"2024-04-25T00:04:50.718128+0000","last_unstale":"2024-04-25T00:04:50.719664+0000","last_undegraded":"2024-04-25T00:04:50.719664+0000","last_fullsized":"2024-04-25T00:04:50.719664+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:00.423063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719971+0000","last_change":"2024-04-25T00:04:49.706365+0000","last_active":"2024-04-25T00:04:50.719971+0000","last_peered":"2024-04-25T00:04:50.719971+0000","last_clean":"2024-04-25T00:04:50.719971+0000","last_became_active":"2024-04-25T00:04:49.706063+0000","last_became_peered":"2024-04-25T00:04:49.706063+0000","last_unstale":"2024-04-25T00:04:50.719971+0000","last_undegraded":"2024-04-25T00:04:50.719971+0000","last_fullsized":"2024-04-25T00:04:50.719971+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:21.023921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721031+0000","last_change":"2024-04-25T00:04:50.720358+0000","last_active":"2024-04-25T00:04:50.721031+0000","last_peered":"2024-04-25T00:04:50.721031+0000","last_clean":"2024-04-25T00:04:50.721031+0000","last_became_active":"2024-04-25T00:04:50.720075+0000","last_became_peered":"2024-04-25T00:04:50.720075+0000","last_unstale":"2024-04-25T00:04:50.721031+0000","last_undegraded":"2024-04-25T00:04:50.721031+0000","last_fullsized":"2024-04-25T00:04:50.721031+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:23.235971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.727490+0000","last_change":"2024-04-25T00:04:49.704288+0000","last_active":"2024-04-25T00:04:50.727490+0000","last_peered":"2024-04-25T00:04:50.727490+0000","last_clean":"2024-04-25T00:04:50.727490+0000","last_became_active":"2024-04-25T00:04:49.703553+0000","last_became_peered":"2024-04-25T00:04:49.703553+0000","last_unstale":"2024-04-25T00:04:50.727490+0000","last_undegraded":"2024-04-25T00:04:50.727490+0000","last_fullsized":"2024-04-25T00:04:50.727490+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:30.528592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714795+0000","last_change":"2024-04-25T00:04:50.710809+0000","last_active":"2024-04-25T00:04:50.714795+0000","last_peered":"2024-04-25T00:04:50.714795+0000","last_clean":"2024-04-25T00:04:50.714795+0000","last_became_active":"2024-04-25T00:04:50.710551+0000","last_became_peered":"2024-04-25T00:04:50.710551+0000","last_unstale":"2024-04-25T00:04:50.714795+0000","last_undegraded":"2024-04-25T00:04:50.714795+0000","last_fullsized":"2024-04-25T00:04:50.714795+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:15.107357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717467+0000","last_change":"2024-04-25T00:04:49.708336+0000","last_active":"2024-04-25T00:04:50.717467+0000","last_peered":"2024-04-25T00:04:50.717467+0000","last_clean":"2024-04-25T00:04:50.717467+0000","last_became_active":"2024-04-25T00:04:49.707866+0000","last_became_peered":"2024-04-25T00:04:49.707866+0000","last_unstale":"2024-04-25T00:04:50.717467+0000","last_undegraded":"2024-04-25T00:04:50.717467+0000","last_fullsized":"2024-04-25T00:04:50.717467+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:42.557675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.712670+0000","last_change":"2024-04-25T00:04:50.707202+0000","last_active":"2024-04-25T00:04:50.712670+0000","last_peered":"2024-04-25T00:04:50.712670+0000","last_clean":"2024-04-25T00:04:50.712670+0000","last_became_active":"2024-04-25T00:04:50.706958+0000","last_became_peered":"2024-04-25T00:04:50.706958+0000","last_unstale":"2024-04-25T00:04:50.712670+0000","last_undegraded":"2024-04-25T00:04:50.712670+0000","last_fullsized":"2024-04-25T00:04:50.712670+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:41.537968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.700160+0000","last_change":"2024-04-25T00:04:49.711969+0000","last_active":"2024-04-25T00:04:50.700160+0000","last_peered":"2024-04-25T00:04:50.700160+0000","last_clean":"2024-04-25T00:04:50.700160+0000","last_became_active":"2024-04-25T00:04:49.711643+0000","last_became_peered":"2024-04-25T00:04:49.711643+0000","last_unstale":"2024-04-25T00:04:50.700160+0000","last_undegraded":"2024-04-25T00:04:50.700160+0000","last_fullsized":"2024-04-25T00:04:50.700160+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:26.761343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709250+0000","last_change":"2024-04-25T00:04:50.705531+0000","last_active":"2024-04-25T00:04:50.709250+0000","last_peered":"2024-04-25T00:04:50.709250+0000","last_clean":"2024-04-25T00:04:50.709250+0000","last_became_active":"2024-04-25T00:04:50.705148+0000","last_became_peered":"2024-04-25T00:04:50.705148+0000","last_unstale":"2024-04-25T00:04:50.709250+0000","last_undegraded":"2024-04-25T00:04:50.709250+0000","last_fullsized":"2024-04-25T00:04:50.709250+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:09.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.693939+0000","last_change":"2024-04-25T00:04:49.698185+0000","last_active":"2024-04-25T00:04:50.693939+0000","last_peered":"2024-04-25T00:04:50.693939+0000","last_clean":"2024-04-25T00:04:50.693939+0000","last_became_active":"2024-04-25T00:04:49.697860+0000","last_became_peered":"2024-04-25T00:04:49.697860+0000","last_unstale":"2024-04-25T00:04:50.693939+0000","last_undegraded":"2024-04-25T00:04:50.693939+0000","last_fullsized":"2024-04-25T00:04:50.693939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:31.033529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720269+0000","last_change":"2024-04-25T00:04:50.717124+0000","last_active":"2024-04-25T00:04:50.720269+0000","last_peered":"2024-04-25T00:04:50.720269+0000","last_clean":"2024-04-25T00:04:50.720269+0000","last_became_active":"2024-04-25T00:04:50.716863+0000","last_became_peered":"2024-04-25T00:04:50.716863+0000","last_unstale":"2024-04-25T00:04:50.720269+0000","last_undegraded":"2024-04-25T00:04:50.720269+0000","last_fullsized":"2024-04-25T00:04:50.720269+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:47.447851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721885+0000","last_change":"2024-04-25T00:04:49.706532+0000","last_active":"2024-04-25T00:04:50.721885+0000","last_peered":"2024-04-25T00:04:50.721885+0000","last_clean":"2024-04-25T00:04:50.721885+0000","last_became_active":"2024-04-25T00:04:49.704248+0000","last_became_peered":"2024-04-25T00:04:49.704248+0000","last_unstale":"2024-04-25T00:04:50.721885+0000","last_undegraded":"2024-04-25T00:04:50.721885+0000","last_fullsized":"2024-04-25T00:04:50.721885+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:54.319114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.708872+0000","last_change":"2024-04-25T00:04:50.705150+0000","last_active":"2024-04-25T00:04:50.708872+0000","last_peered":"2024-04-25T00:04:50.708872+0000","last_clean":"2024-04-25T00:04:50.708872+0000","last_became_active":"2024-04-25T00:04:50.704132+0000","last_became_peered":"2024-04-25T00:04:50.704132+0000","last_unstale":"2024-04-25T00:04:50.708872+0000","last_undegraded":"2024-04-25T00:04:50.708872+0000","last_fullsized":"2024-04-25T00:04:50.708872+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:32.318327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.722338+0000","last_change":"2024-04-25T00:04:49.707053+0000","last_active":"2024-04-25T00:04:50.722338+0000","last_peered":"2024-04-25T00:04:50.722338+0000","last_clean":"2024-04-25T00:04:50.722338+0000","last_became_active":"2024-04-25T00:04:49.706724+0000","last_became_peered":"2024-04-25T00:04:49.706724+0000","last_unstale":"2024-04-25T00:04:50.722338+0000","last_undegraded":"2024-04-25T00:04:50.722338+0000","last_fullsized":"2024-04-25T00:04:50.722338+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:41.110533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717795+0000","last_change":"2024-04-25T00:04:50.717245+0000","last_active":"2024-04-25T00:04:50.717795+0000","last_peered":"2024-04-25T00:04:50.717795+0000","last_clean":"2024-04-25T00:04:50.717795+0000","last_became_active":"2024-04-25T00:04:50.716962+0000","last_became_peered":"2024-04-25T00:04:50.716962+0000","last_unstale":"2024-04-25T00:04:50.717795+0000","last_undegraded":"2024-04-25T00:04:50.717795+0000","last_fullsized":"2024-04-25T00:04:50.717795+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:09.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715483+0000","last_change":"2024-04-25T00:04:49.701091+0000","last_active":"2024-04-25T00:04:50.715483+0000","last_peered":"2024-04-25T00:04:50.715483+0000","last_clean":"2024-04-25T00:04:50.715483+0000","last_became_active":"2024-04-25T00:04:49.700350+0000","last_became_peered":"2024-04-25T00:04:49.700350+0000","last_unstale":"2024-04-25T00:04:50.715483+0000","last_undegraded":"2024-04-25T00:04:50.715483+0000","last_fullsized":"2024-04-25T00:04:50.715483+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:53.577335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715655+0000","last_change":"2024-04-25T00:04:50.713931+0000","last_active":"2024-04-25T00:04:50.715655+0000","last_peered":"2024-04-25T00:04:50.715655+0000","last_clean":"2024-04-25T00:04:50.715655+0000","last_became_active":"2024-04-25T00:04:50.713279+0000","last_became_peered":"2024-04-25T00:04:50.713279+0000","last_unstale":"2024-04-25T00:04:50.715655+0000","last_undegraded":"2024-04-25T00:04:50.715655+0000","last_fullsized":"2024-04-25T00:04:50.715655+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:23.853626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695394+0000","last_change":"2024-04-25T00:04:49.704292+0000","last_active":"2024-04-25T00:04:50.695394+0000","last_peered":"2024-04-25T00:04:50.695394+0000","last_clean":"2024-04-25T00:04:50.695394+0000","last_became_active":"2024-04-25T00:04:49.704053+0000","last_became_peered":"2024-04-25T00:04:49.704053+0000","last_unstale":"2024-04-25T00:04:50.695394+0000","last_undegraded":"2024-04-25T00:04:50.695394+0000","last_fullsized":"2024-04-25T00:04:50.695394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:34.335248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717765+0000","last_change":"2024-04-25T00:04:50.713772+0000","last_active":"2024-04-25T00:04:50.717765+0000","last_peered":"2024-04-25T00:04:50.717765+0000","last_clean":"2024-04-25T00:04:50.717765+0000","last_became_active":"2024-04-25T00:04:50.713532+0000","last_became_peered":"2024-04-25T00:04:50.713532+0000","last_unstale":"2024-04-25T00:04:50.717765+0000","last_undegraded":"2024-04-25T00:04:50.717765+0000","last_fullsized":"2024-04-25T00:04:50.717765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:55.726257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.722678+0000","last_change":"2024-04-25T00:04:49.707370+0000","last_active":"2024-04-25T00:04:50.722678+0000","last_peered":"2024-04-25T00:04:50.722678+0000","last_clean":"2024-04-25T00:04:50.722678+0000","last_became_active":"2024-04-25T00:04:49.707113+0000","last_became_peered":"2024-04-25T00:04:49.707113+0000","last_unstale":"2024-04-25T00:04:50.722678+0000","last_undegraded":"2024-04-25T00:04:50.722678+0000","last_fullsized":"2024-04-25T00:04:50.722678+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:59.417620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.723260+0000","last_change":"2024-04-25T00:04:50.722655+0000","last_active":"2024-04-25T00:04:50.723260+0000","last_peered":"2024-04-25T00:04:50.723260+0000","last_clean":"2024-04-25T00:04:50.723260+0000","last_became_active":"2024-04-25T00:04:50.722353+0000","last_became_peered":"2024-04-25T00:04:50.722353+0000","last_unstale":"2024-04-25T00:04:50.723260+0000","last_undegraded":"2024-04-25T00:04:50.723260+0000","last_fullsized":"2024-04-25T00:04:50.723260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:15.932352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.701516+0000","last_change":"2024-04-25T00:04:49.709190+0000","last_active":"2024-04-25T00:04:50.701516+0000","last_peered":"2024-04-25T00:04:50.701516+0000","last_clean":"2024-04-25T00:04:50.701516+0000","last_became_active":"2024-04-25T00:04:49.708908+0000","last_became_peered":"2024-04-25T00:04:49.708908+0000","last_unstale":"2024-04-25T00:04:50.701516+0000","last_undegraded":"2024-04-25T00:04:50.701516+0000","last_fullsized":"2024-04-25T00:04:50.701516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:31.033529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719464+0000","last_change":"2024-04-25T00:04:50.715463+0000","last_active":"2024-04-25T00:04:50.719464+0000","last_peered":"2024-04-25T00:04:50.719464+0000","last_clean":"2024-04-25T00:04:50.719464+0000","last_became_active":"2024-04-25T00:04:50.715200+0000","last_became_peered":"2024-04-25T00:04:50.715200+0000","last_unstale":"2024-04-25T00:04:50.719464+0000","last_undegraded":"2024-04-25T00:04:50.719464+0000","last_fullsized":"2024-04-25T00:04:50.719464+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:40.363215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.703291+0000","last_change":"2024-04-25T00:04:49.704021+0000","last_active":"2024-04-25T00:04:50.703291+0000","last_peered":"2024-04-25T00:04:50.703291+0000","last_clean":"2024-04-25T00:04:50.703291+0000","last_became_active":"2024-04-25T00:04:49.703669+0000","last_became_peered":"2024-04-25T00:04:49.703669+0000","last_unstale":"2024-04-25T00:04:50.703291+0000","last_undegraded":"2024-04-25T00:04:50.703291+0000","last_fullsized":"2024-04-25T00:04:50.703291+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:30.528592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714899+0000","last_change":"2024-04-25T00:04:50.708746+0000","last_active":"2024-04-25T00:04:50.714899+0000","last_peered":"2024-04-25T00:04:50.714899+0000","last_clean":"2024-04-25T00:04:50.714899+0000","last_became_active":"2024-04-25T00:04:50.707499+0000","last_became_peered":"2024-04-25T00:04:50.707499+0000","last_unstale":"2024-04-25T00:04:50.714899+0000","last_undegraded":"2024-04-25T00:04:50.714899+0000","last_fullsized":"2024-04-25T00:04:50.714899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:42.115976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695455+0000","last_change":"2024-04-25T00:04:49.699269+0000","last_active":"2024-04-25T00:04:50.695455+0000","last_peered":"2024-04-25T00:04:50.695455+0000","last_clean":"2024-04-25T00:04:50.695455+0000","last_became_active":"2024-04-25T00:04:49.699014+0000","last_became_peered":"2024-04-25T00:04:49.699014+0000","last_unstale":"2024-04-25T00:04:50.695455+0000","last_undegraded":"2024-04-25T00:04:50.695455+0000","last_fullsized":"2024-04-25T00:04:50.695455+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:48.808861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718771+0000","last_change":"2024-04-25T00:04:50.710664+0000","last_active":"2024-04-25T00:04:50.718771+0000","last_peered":"2024-04-25T00:04:50.718771+0000","last_clean":"2024-04-25T00:04:50.718771+0000","last_became_active":"2024-04-25T00:04:50.710357+0000","last_became_peered":"2024-04-25T00:04:50.710357+0000","last_unstale":"2024-04-25T00:04:50.718771+0000","last_undegraded":"2024-04-25T00:04:50.718771+0000","last_fullsized":"2024-04-25T00:04:50.718771+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:22.184636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718605+0000","last_change":"2024-04-25T00:04:49.704259+0000","last_active":"2024-04-25T00:04:50.718605+0000","last_peered":"2024-04-25T00:04:50.718605+0000","last_clean":"2024-04-25T00:04:50.718605+0000","last_became_active":"2024-04-25T00:04:49.703737+0000","last_became_peered":"2024-04-25T00:04:49.703737+0000","last_unstale":"2024-04-25T00:04:50.718605+0000","last_undegraded":"2024-04-25T00:04:50.718605+0000","last_fullsized":"2024-04-25T00:04:50.718605+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:41.110533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710338+0000","last_change":"2024-04-25T00:04:50.706648+0000","last_active":"2024-04-25T00:04:50.710338+0000","last_peered":"2024-04-25T00:04:50.710338+0000","last_clean":"2024-04-25T00:04:50.710338+0000","last_became_active":"2024-04-25T00:04:50.706376+0000","last_became_peered":"2024-04-25T00:04:50.706376+0000","last_unstale":"2024-04-25T00:04:50.710338+0000","last_undegraded":"2024-04-25T00:04:50.710338+0000","last_fullsized":"2024-04-25T00:04:50.710338+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:26.318174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.696101+0000","last_change":"2024-04-25T00:04:49.706380+0000","last_active":"2024-04-25T00:04:50.696101+0000","last_peered":"2024-04-25T00:04:50.696101+0000","last_clean":"2024-04-25T00:04:50.696101+0000","last_became_active":"2024-04-25T00:04:49.703814+0000","last_became_peered":"2024-04-25T00:04:49.703814+0000","last_unstale":"2024-04-25T00:04:50.696101+0000","last_undegraded":"2024-04-25T00:04:50.696101+0000","last_fullsized":"2024-04-25T00:04:50.696101+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:17.520467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719311+0000","last_change":"2024-04-25T00:04:50.712815+0000","last_active":"2024-04-25T00:04:50.719311+0000","last_peered":"2024-04-25T00:04:50.719311+0000","last_clean":"2024-04-25T00:04:50.719311+0000","last_became_active":"2024-04-25T00:04:50.712440+0000","last_became_peered":"2024-04-25T00:04:50.712440+0000","last_unstale":"2024-04-25T00:04:50.719311+0000","last_undegraded":"2024-04-25T00:04:50.719311+0000","last_fullsized":"2024-04-25T00:04:50.719311+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:59.955428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695496+0000","last_change":"2024-04-25T00:04:49.707755+0000","last_active":"2024-04-25T00:04:50.695496+0000","last_peered":"2024-04-25T00:04:50.695496+0000","last_clean":"2024-04-25T00:04:50.695496+0000","last_became_active":"2024-04-25T00:04:49.707514+0000","last_became_peered":"2024-04-25T00:04:49.707514+0000","last_unstale":"2024-04-25T00:04:50.695496+0000","last_undegraded":"2024-04-25T00:04:50.695496+0000","last_fullsized":"2024-04-25T00:04:50.695496+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:25.666226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719221+0000","last_change":"2024-04-25T00:04:50.716292+0000","last_active":"2024-04-25T00:04:50.719221+0000","last_peered":"2024-04-25T00:04:50.719221+0000","last_clean":"2024-04-25T00:04:50.719221+0000","last_became_active":"2024-04-25T00:04:50.715979+0000","last_became_peered":"2024-04-25T00:04:50.715979+0000","last_unstale":"2024-04-25T00:04:50.719221+0000","last_undegraded":"2024-04-25T00:04:50.719221+0000","last_fullsized":"2024-04-25T00:04:50.719221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:09.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.707900+0000","last_change":"2024-04-25T00:04:49.704507+0000","last_active":"2024-04-25T00:04:50.707900+0000","last_peered":"2024-04-25T00:04:50.707900+0000","last_clean":"2024-04-25T00:04:50.707900+0000","last_became_active":"2024-04-25T00:04:49.701378+0000","last_became_peered":"2024-04-25T00:04:49.701378+0000","last_unstale":"2024-04-25T00:04:50.707900+0000","last_undegraded":"2024-04-25T00:04:50.707900+0000","last_fullsized":"2024-04-25T00:04:50.707900+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:49.617006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715120+0000","last_change":"2024-04-25T00:04:50.712219+0000","last_active":"2024-04-25T00:04:50.715120+0000","last_peered":"2024-04-25T00:04:50.715120+0000","last_clean":"2024-04-25T00:04:50.715120+0000","last_became_active":"2024-04-25T00:04:50.711656+0000","last_became_peered":"2024-04-25T00:04:50.711656+0000","last_unstale":"2024-04-25T00:04:50.715120+0000","last_undegraded":"2024-04-25T00:04:50.715120+0000","last_fullsized":"2024-04-25T00:04:50.715120+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:17.311346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695557+0000","last_change":"2024-04-25T00:04:49.701995+0000","last_active":"2024-04-25T00:04:50.695557+0000","last_peered":"2024-04-25T00:04:50.695557+0000","last_clean":"2024-04-25T00:04:50.695557+0000","last_became_active":"2024-04-25T00:04:49.701096+0000","last_became_peered":"2024-04-25T00:04:49.701096+0000","last_unstale":"2024-04-25T00:04:50.695557+0000","last_undegraded":"2024-04-25T00:04:50.695557+0000","last_fullsized":"2024-04-25T00:04:50.695557+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:14.101914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.712621+0000","last_change":"2024-04-25T00:04:50.710044+0000","last_active":"2024-04-25T00:04:50.712621+0000","last_peered":"2024-04-25T00:04:50.712621+0000","last_clean":"2024-04-25T00:04:50.712621+0000","last_became_active":"2024-04-25T00:04:50.709758+0000","last_became_peered":"2024-04-25T00:04:50.709758+0000","last_unstale":"2024-04-25T00:04:50.712621+0000","last_undegraded":"2024-04-25T00:04:50.712621+0000","last_fullsized":"2024-04-25T00:04:50.712621+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:35.340691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.699435+0000","last_change":"2024-04-25T00:04:49.698249+0000","last_active":"2024-04-25T00:04:50.699435+0000","last_peered":"2024-04-25T00:04:50.699435+0000","last_clean":"2024-04-25T00:04:50.699435+0000","last_became_active":"2024-04-25T00:04:49.697983+0000","last_became_peered":"2024-04-25T00:04:49.697983+0000","last_unstale":"2024-04-25T00:04:50.699435+0000","last_undegraded":"2024-04-25T00:04:50.699435+0000","last_fullsized":"2024-04-25T00:04:50.699435+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:37.433492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709484+0000","last_change":"2024-04-25T00:04:50.704693+0000","last_active":"2024-04-25T00:04:50.709484+0000","last_peered":"2024-04-25T00:04:50.709484+0000","last_clean":"2024-04-25T00:04:50.709484+0000","last_became_active":"2024-04-25T00:04:50.704412+0000","last_became_peered":"2024-04-25T00:04:50.704412+0000","last_unstale":"2024-04-25T00:04:50.709484+0000","last_undegraded":"2024-04-25T00:04:50.709484+0000","last_fullsized":"2024-04-25T00:04:50.709484+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:06.534925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.706502+0000","last_change":"2024-04-25T00:04:49.706055+0000","last_active":"2024-04-25T00:04:50.706502+0000","last_peered":"2024-04-25T00:04:50.706502+0000","last_clean":"2024-04-25T00:04:50.706502+0000","last_became_active":"2024-04-25T00:04:49.705790+0000","last_became_peered":"2024-04-25T00:04:49.705790+0000","last_unstale":"2024-04-25T00:04:50.706502+0000","last_undegraded":"2024-04-25T00:04:50.706502+0000","last_fullsized":"2024-04-25T00:04:50.706502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:25.666226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719268+0000","last_change":"2024-04-25T00:04:50.716267+0000","last_active":"2024-04-25T00:04:50.719268+0000","last_peered":"2024-04-25T00:04:50.719268+0000","last_clean":"2024-04-25T00:04:50.719268+0000","last_became_active":"2024-04-25T00:04:50.715954+0000","last_became_peered":"2024-04-25T00:04:50.715954+0000","last_unstale":"2024-04-25T00:04:50.719268+0000","last_undegraded":"2024-04-25T00:04:50.719268+0000","last_fullsized":"2024-04-25T00:04:50.719268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:46.915644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.700193+0000","last_change":"2024-04-25T00:04:49.708898+0000","last_active":"2024-04-25T00:04:50.700193+0000","last_peered":"2024-04-25T00:04:50.700193+0000","last_clean":"2024-04-25T00:04:50.700193+0000","last_became_active":"2024-04-25T00:04:49.708642+0000","last_became_peered":"2024-04-25T00:04:49.708642+0000","last_unstale":"2024-04-25T00:04:50.700193+0000","last_undegraded":"2024-04-25T00:04:50.700193+0000","last_fullsized":"2024-04-25T00:04:50.700193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:35.592808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.716257+0000","last_change":"2024-04-25T00:04:50.713843+0000","last_active":"2024-04-25T00:04:50.716257+0000","last_peered":"2024-04-25T00:04:50.716257+0000","last_clean":"2024-04-25T00:04:50.716257+0000","last_became_active":"2024-04-25T00:04:50.713611+0000","last_became_peered":"2024-04-25T00:04:50.713611+0000","last_unstale":"2024-04-25T00:04:50.716257+0000","last_undegraded":"2024-04-25T00:04:50.716257+0000","last_fullsized":"2024-04-25T00:04:50.716257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:55.324557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721146+0000","last_change":"2024-04-25T00:04:49.703373+0000","last_active":"2024-04-25T00:04:50.721146+0000","last_peered":"2024-04-25T00:04:50.721146+0000","last_clean":"2024-04-25T00:04:50.721146+0000","last_became_active":"2024-04-25T00:04:49.702658+0000","last_became_peered":"2024-04-25T00:04:49.702658+0000","last_unstale":"2024-04-25T00:04:50.721146+0000","last_undegraded":"2024-04-25T00:04:50.721146+0000","last_fullsized":"2024-04-25T00:04:50.721146+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:35.651334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.700263+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:04:50.700263+0000","last_peered":"2024-04-25T00:04:50.700263+0000","last_clean":"2024-04-25T00:04:50.700263+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:04:50.700263+0000","last_undegraded":"2024-04-25T00:04:50.700263+0000","last_fullsized":"2024-04-25T00:04:50.700263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55220,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316620,"statfs":{"total":96636764160,"available":96580218880,"internally_reserved":0,"allocated":1875968,"data_stored":1141010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58788,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313052,"statfs":{"total":96636764160,"available":96576565248,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58784,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313056,"statfs":{"total":96636764160,"available":96576569344,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58788,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313052,"statfs":{"total":96636764160,"available":96576565248,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40736,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331104,"statfs":{"total":96636764160,"available":96595050496,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50472,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321368,"statfs":{"total":96636764160,"available":96585080832,"internally_reserved":0,"allocated":1273856,"data_stored":550530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41332,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330508,"statfs":{"total":96636764160,"available":96594440192,"internally_reserved":0,"allocated":1871872,"data_stored":1141480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:04:53.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:04:53.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.802+0000 7f5919a10640 1 Processor -- start 2024-04-25T00:04:53.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.802+0000 7f5919a10640 1 -- start start 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.802+0000 7f5919a10640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59140d0ff0 0x7f59140b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.802+0000 7f5919a10640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f591415e570 0x7f59140b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.802+0000 7f5919a10640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59140ba060 con 0x7f59140b2de0 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.802+0000 7f5919a10640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f59140b3630 con 0x7f59140d0ff0 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.802+0000 7f5919a10640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f59140b8320 con 0x7f591415e570 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f5912ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59140d0ff0 0x7f59140b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f5912ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59140d0ff0 0x7f59140b9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47924/0 (socket says 172.21.15.113:47924) 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f5912ffd640 1 -- 172.21.15.113:0/415670868 learned_addr learned my addr 172.21.15.113:0/415670868 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f59127fc640 1 --2- 172.21.15.113:0/415670868 >> v2:172.21.15.178:3300/0 conn(0x7f591415e570 0x7f59140b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f5912ffd640 1 -- 172.21.15.113:0/415670868 >> v2:172.21.15.178:3300/0 conn(0x7f591415e570 msgr2=0x7f59140b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f5912ffd640 1 --2- 172.21.15.113:0/415670868 >> v2:172.21.15.178:3300/0 conn(0x7f591415e570 0x7f59140b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f5912ffd640 1 -- 172.21.15.113:0/415670868 >> v1:172.21.15.178:6789/0 conn(0x7f59140b2de0 legacy=0x7f59140d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.803+0000 7f5912ffd640 1 -- 172.21.15.113:0/415670868 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59140b8600 con 0x7f59140d0ff0 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5912ffd640 1 --2- 172.21.15.113:0/415670868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140b9960 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f59140b5c40 tx=0x7f590402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cea4d7ac160b5de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5911ffb640 1 -- 172.21.15.113:0/415670868 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904037070 con 0x7f59140d0ff0 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5911ffb640 1 -- 172.21.15.113:0/415670868 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5904032440 con 0x7f59140d0ff0 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5911ffb640 1 -- 172.21.15.113:0/415670868 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904032810 con 0x7f59140d0ff0 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 -- 172.21.15.113:0/415670868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 msgr2=0x7f59140b9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 --2- 172.21.15.113:0/415670868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140b9960 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f59140b5c40 tx=0x7f590402a2c0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 -- 172.21.15.113:0/415670868 shutdown_connections 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 --2- 172.21.15.113:0/415670868 >> v2:172.21.15.178:3300/0 conn(0x7f591415e570 0x7f59140b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 --2- 172.21.15.113:0/415670868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140b9960 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 -- 172.21.15.113:0/415670868 >> 172.21.15.113:0/415670868 conn(0x7f5914071b00 msgr2=0x7f5914071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 -- 172.21.15.113:0/415670868 shutdown_connections 2024-04-25T00:04:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.804+0000 7f5919a10640 1 -- 172.21.15.113:0/415670868 wait complete. 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.805+0000 7f5919a10640 1 Processor -- start 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f5919a10640 1 -- start start 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f5919a10640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59140b2de0 0x7f59141f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f5919a10640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f5919a10640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591415e570 0x7f59140d3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f5919a10640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59140d2140 con 0x7f59140d0ff0 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f5919a10640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59140b8bc0 con 0x7f591415e570 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f5919a10640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f59140d22c0 con 0x7f59140b2de0 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f59127fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f59127fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47936/0 (socket says 172.21.15.113:47936) 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.806+0000 7f59127fc640 1 -- 172.21.15.113:0/1258803177 learned_addr learned my addr 172.21.15.113:0/1258803177 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:53.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f59137fe640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591415e570 0x7f59140d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f5912ffd640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59140b2de0 0x7f59141f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f59127fc640 1 -- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59140b2de0 msgr2=0x7f59141f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f59127fc640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59140b2de0 0x7f59141f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f59127fc640 1 -- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591415e570 msgr2=0x7f59140d3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f59127fc640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591415e570 0x7f59140d3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f59127fc640 1 -- 172.21.15.113:0/1258803177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59141f5c50 con 0x7f59140d0ff0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f5912ffd640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59140b2de0 0x7f59141f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f59127fc640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140cc510 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f590800d5d0 tx=0x7f590800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f58fbfff640 1 -- 172.21.15.113:0/1258803177 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908004240 con 0x7f59140d0ff0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f58fbfff640 1 -- 172.21.15.113:0/1258803177 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59080043e0 con 0x7f59140d0ff0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59141f49e0 con 0x7f59140d0ff0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f58fbfff640 1 -- 172.21.15.113:0/1258803177 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908003830 con 0x7f59140d0ff0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.807+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59141f4fa0 con 0x7f59140d0ff0 2024-04-25T00:04:53.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.808+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591415ea00 con 0x7f59140d0ff0 2024-04-25T00:04:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.808+0000 7f58fbfff640 1 -- 172.21.15.113:0/1258803177 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59080039d0 con 0x7f59140d0ff0 2024-04-25T00:04:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.809+0000 7f58fbfff640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f590006be60 0x7f590006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.809+0000 7f58fbfff640 1 -- 172.21.15.113:0/1258803177 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f590808b170 con 0x7f59140d0ff0 2024-04-25T00:04:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.809+0000 7f5912ffd640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f590006be60 0x7f590006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.809+0000 7f5912ffd640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f590006be60 0x7f590006e320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.813+0000 7f58fbfff640 1 -- 172.21.15.113:0/1258803177 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5908059a90 con 0x7f59140d0ff0 2024-04-25T00:04:53.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.983+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59140691a0 con 0x7f590006be60 2024-04-25T00:04:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.989+0000 7f58fbfff640 1 -- 172.21.15.113:0/1258803177 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252416 (secure 0 0 0) 0x7f59140691a0 con 0x7f590006be60 2024-04-25T00:04:53.992 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f590006be60 msgr2=0x7f590006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f590006be60 0x7f590006e320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).stop 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 msgr2=0x7f59140cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140cc510 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f590800d5d0 tx=0x7f590800da90 comp rx=0 tx=0).stop 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 shutdown_connections 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f590006be60 0x7f590006e320 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591415e570 0x7f59140d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59140d0ff0 0x7f59140cc510 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 --2- 172.21.15.113:0/1258803177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59140b2de0 0x7f59141f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 >> 172.21.15.113:0/1258803177 conn(0x7f5914071b00 msgr2=0x7f59140ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 shutdown_connections 2024-04-25T00:04:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:53.992+0000 7f5919a10640 1 -- 172.21.15.113:0/1258803177 wait complete. 2024-04-25T00:04:54.004 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-25T00:04:53.189273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":396532,"kb_used_data":11064,"kb_used_omap":12,"kb_used_meta":385331,"kb_avail":754578188,"statfs":{"total":773094113280,"available":772688064512,"internally_reserved":0,"allocated":11329536,"data_stored":5551795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394579504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.505597"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.711316+0000","last_change":"2024-04-25T00:04:50.710708+0000","last_active":"2024-04-25T00:04:50.711316+0000","last_peered":"2024-04-25T00:04:50.711316+0000","last_clean":"2024-04-25T00:04:50.711316+0000","last_became_active":"2024-04-25T00:04:50.710152+0000","last_became_peered":"2024-04-25T00:04:50.710152+0000","last_unstale":"2024-04-25T00:04:50.711316+0000","last_undegraded":"2024-04-25T00:04:50.711316+0000","last_fullsized":"2024-04-25T00:04:50.711316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:40.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718747+0000","last_change":"2024-04-25T00:04:50.708247+0000","last_active":"2024-04-25T00:04:50.718747+0000","last_peered":"2024-04-25T00:04:50.718747+0000","last_clean":"2024-04-25T00:04:50.718747+0000","last_became_active":"2024-04-25T00:04:50.707274+0000","last_became_peered":"2024-04-25T00:04:50.707274+0000","last_unstale":"2024-04-25T00:04:50.718747+0000","last_undegraded":"2024-04-25T00:04:50.718747+0000","last_fullsized":"2024-04-25T00:04:50.718747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:51.057870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709187+0000","last_change":"2024-04-25T00:04:50.705854+0000","last_active":"2024-04-25T00:04:50.709187+0000","last_peered":"2024-04-25T00:04:50.709187+0000","last_clean":"2024-04-25T00:04:50.709187+0000","last_became_active":"2024-04-25T00:04:50.705584+0000","last_became_peered":"2024-04-25T00:04:50.705584+0000","last_unstale":"2024-04-25T00:04:50.709187+0000","last_undegraded":"2024-04-25T00:04:50.709187+0000","last_fullsized":"2024-04-25T00:04:50.709187+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:21:29.450392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714858+0000","last_change":"2024-04-25T00:04:50.712263+0000","last_active":"2024-04-25T00:04:50.714858+0000","last_peered":"2024-04-25T00:04:50.714858+0000","last_clean":"2024-04-25T00:04:50.714858+0000","last_became_active":"2024-04-25T00:04:50.712011+0000","last_became_peered":"2024-04-25T00:04:50.712011+0000","last_unstale":"2024-04-25T00:04:50.714858+0000","last_undegraded":"2024-04-25T00:04:50.714858+0000","last_fullsized":"2024-04-25T00:04:50.714858+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:55.460510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714834+0000","last_change":"2024-04-25T00:04:50.708165+0000","last_active":"2024-04-25T00:04:50.714834+0000","last_peered":"2024-04-25T00:04:50.714834+0000","last_clean":"2024-04-25T00:04:50.714834+0000","last_became_active":"2024-04-25T00:04:50.707885+0000","last_became_peered":"2024-04-25T00:04:50.707885+0000","last_unstale":"2024-04-25T00:04:50.714834+0000","last_undegraded":"2024-04-25T00:04:50.714834+0000","last_fullsized":"2024-04-25T00:04:50.714834+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:32.038973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.712536+0000","last_change":"2024-04-25T00:04:50.711592+0000","last_active":"2024-04-25T00:04:50.712536+0000","last_peered":"2024-04-25T00:04:50.712536+0000","last_clean":"2024-04-25T00:04:50.712536+0000","last_became_active":"2024-04-25T00:04:50.711290+0000","last_became_peered":"2024-04-25T00:04:50.711290+0000","last_unstale":"2024-04-25T00:04:50.712536+0000","last_undegraded":"2024-04-25T00:04:50.712536+0000","last_fullsized":"2024-04-25T00:04:50.712536+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:51.846162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710858+0000","last_change":"2024-04-25T00:04:50.708327+0000","last_active":"2024-04-25T00:04:50.710858+0000","last_peered":"2024-04-25T00:04:50.710858+0000","last_clean":"2024-04-25T00:04:50.710858+0000","last_became_active":"2024-04-25T00:04:50.708055+0000","last_became_peered":"2024-04-25T00:04:50.708055+0000","last_unstale":"2024-04-25T00:04:50.710858+0000","last_undegraded":"2024-04-25T00:04:50.710858+0000","last_fullsized":"2024-04-25T00:04:50.710858+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:17.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719481+0000","last_change":"2024-04-25T00:04:50.709416+0000","last_active":"2024-04-25T00:04:50.719481+0000","last_peered":"2024-04-25T00:04:50.719481+0000","last_clean":"2024-04-25T00:04:50.719481+0000","last_became_active":"2024-04-25T00:04:50.709137+0000","last_became_peered":"2024-04-25T00:04:50.709137+0000","last_unstale":"2024-04-25T00:04:50.719481+0000","last_undegraded":"2024-04-25T00:04:50.719481+0000","last_fullsized":"2024-04-25T00:04:50.719481+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:17.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717176+0000","last_change":"2024-04-25T00:04:50.714210+0000","last_active":"2024-04-25T00:04:50.717176+0000","last_peered":"2024-04-25T00:04:50.717176+0000","last_clean":"2024-04-25T00:04:50.717176+0000","last_became_active":"2024-04-25T00:04:50.713959+0000","last_became_peered":"2024-04-25T00:04:50.713959+0000","last_unstale":"2024-04-25T00:04:50.717176+0000","last_undegraded":"2024-04-25T00:04:50.717176+0000","last_fullsized":"2024-04-25T00:04:50.717176+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:49.814304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715274+0000","last_change":"2024-04-25T00:04:50.712606+0000","last_active":"2024-04-25T00:04:50.715274+0000","last_peered":"2024-04-25T00:04:50.715274+0000","last_clean":"2024-04-25T00:04:50.715274+0000","last_became_active":"2024-04-25T00:04:50.712332+0000","last_became_peered":"2024-04-25T00:04:50.712332+0000","last_unstale":"2024-04-25T00:04:50.715274+0000","last_undegraded":"2024-04-25T00:04:50.715274+0000","last_fullsized":"2024-04-25T00:04:50.715274+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:31.534036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718780+0000","last_change":"2024-04-25T00:04:50.714176+0000","last_active":"2024-04-25T00:04:50.718780+0000","last_peered":"2024-04-25T00:04:50.718780+0000","last_clean":"2024-04-25T00:04:50.718780+0000","last_became_active":"2024-04-25T00:04:50.713642+0000","last_became_peered":"2024-04-25T00:04:50.713642+0000","last_unstale":"2024-04-25T00:04:50.718780+0000","last_undegraded":"2024-04-25T00:04:50.718780+0000","last_fullsized":"2024-04-25T00:04:50.718780+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:07.941150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.702633+0000","last_change":"2024-04-25T00:04:50.698815+0000","last_active":"2024-04-25T00:04:50.702633+0000","last_peered":"2024-04-25T00:04:50.702633+0000","last_clean":"2024-04-25T00:04:50.702633+0000","last_became_active":"2024-04-25T00:04:50.698489+0000","last_became_peered":"2024-04-25T00:04:50.698489+0000","last_unstale":"2024-04-25T00:04:50.702633+0000","last_undegraded":"2024-04-25T00:04:50.702633+0000","last_fullsized":"2024-04-25T00:04:50.702633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:46.915644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.725414+0000","last_change":"2024-04-25T00:04:50.720321+0000","last_active":"2024-04-25T00:04:50.725414+0000","last_peered":"2024-04-25T00:04:50.725414+0000","last_clean":"2024-04-25T00:04:50.725414+0000","last_became_active":"2024-04-25T00:04:50.720044+0000","last_became_peered":"2024-04-25T00:04:50.720044+0000","last_unstale":"2024-04-25T00:04:50.725414+0000","last_undegraded":"2024-04-25T00:04:50.725414+0000","last_fullsized":"2024-04-25T00:04:50.725414+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:51.057870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720368+0000","last_change":"2024-04-25T00:04:50.718821+0000","last_active":"2024-04-25T00:04:50.720368+0000","last_peered":"2024-04-25T00:04:50.720368+0000","last_clean":"2024-04-25T00:04:50.720368+0000","last_became_active":"2024-04-25T00:04:50.718422+0000","last_became_peered":"2024-04-25T00:04:50.718422+0000","last_unstale":"2024-04-25T00:04:50.720368+0000","last_undegraded":"2024-04-25T00:04:50.720368+0000","last_fullsized":"2024-04-25T00:04:50.720368+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:46.941535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714897+0000","last_change":"2024-04-25T00:04:50.711323+0000","last_active":"2024-04-25T00:04:50.714897+0000","last_peered":"2024-04-25T00:04:50.714897+0000","last_clean":"2024-04-25T00:04:50.714897+0000","last_became_active":"2024-04-25T00:04:50.711045+0000","last_became_peered":"2024-04-25T00:04:50.711045+0000","last_unstale":"2024-04-25T00:04:50.714897+0000","last_undegraded":"2024-04-25T00:04:50.714897+0000","last_fullsized":"2024-04-25T00:04:50.714897+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:26.448731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719204+0000","last_change":"2024-04-25T00:04:50.718213+0000","last_active":"2024-04-25T00:04:50.719204+0000","last_peered":"2024-04-25T00:04:50.719204+0000","last_clean":"2024-04-25T00:04:50.719204+0000","last_became_active":"2024-04-25T00:04:50.717911+0000","last_became_peered":"2024-04-25T00:04:50.717911+0000","last_unstale":"2024-04-25T00:04:50.719204+0000","last_undegraded":"2024-04-25T00:04:50.719204+0000","last_fullsized":"2024-04-25T00:04:50.719204+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:45:45.068235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718837+0000","last_change":"2024-04-25T00:04:50.709213+0000","last_active":"2024-04-25T00:04:50.718837+0000","last_peered":"2024-04-25T00:04:50.718837+0000","last_clean":"2024-04-25T00:04:50.718837+0000","last_became_active":"2024-04-25T00:04:50.708855+0000","last_became_peered":"2024-04-25T00:04:50.708855+0000","last_unstale":"2024-04-25T00:04:50.718837+0000","last_undegraded":"2024-04-25T00:04:50.718837+0000","last_fullsized":"2024-04-25T00:04:50.718837+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:36.656777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717106+0000","last_change":"2024-04-25T00:04:50.713693+0000","last_active":"2024-04-25T00:04:50.717106+0000","last_peered":"2024-04-25T00:04:50.717106+0000","last_clean":"2024-04-25T00:04:50.717106+0000","last_became_active":"2024-04-25T00:04:50.713415+0000","last_became_peered":"2024-04-25T00:04:50.713415+0000","last_unstale":"2024-04-25T00:04:50.717106+0000","last_undegraded":"2024-04-25T00:04:50.717106+0000","last_fullsized":"2024-04-25T00:04:50.717106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:16.829314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709451+0000","last_change":"2024-04-25T00:04:50.704862+0000","last_active":"2024-04-25T00:04:50.709451+0000","last_peered":"2024-04-25T00:04:50.709451+0000","last_clean":"2024-04-25T00:04:50.709451+0000","last_became_active":"2024-04-25T00:04:50.704385+0000","last_became_peered":"2024-04-25T00:04:50.704385+0000","last_unstale":"2024-04-25T00:04:50.709451+0000","last_undegraded":"2024-04-25T00:04:50.709451+0000","last_fullsized":"2024-04-25T00:04:50.709451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:27.139941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717387+0000","last_change":"2024-04-25T00:04:50.714676+0000","last_active":"2024-04-25T00:04:50.717387+0000","last_peered":"2024-04-25T00:04:50.717387+0000","last_clean":"2024-04-25T00:04:50.717387+0000","last_became_active":"2024-04-25T00:04:50.713734+0000","last_became_peered":"2024-04-25T00:04:50.713734+0000","last_unstale":"2024-04-25T00:04:50.717387+0000","last_undegraded":"2024-04-25T00:04:50.717387+0000","last_fullsized":"2024-04-25T00:04:50.717387+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:26.671669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709515+0000","last_change":"2024-04-25T00:04:50.705417+0000","last_active":"2024-04-25T00:04:50.709515+0000","last_peered":"2024-04-25T00:04:50.709515+0000","last_clean":"2024-04-25T00:04:50.709515+0000","last_became_active":"2024-04-25T00:04:50.705153+0000","last_became_peered":"2024-04-25T00:04:50.705153+0000","last_unstale":"2024-04-25T00:04:50.709515+0000","last_undegraded":"2024-04-25T00:04:50.709515+0000","last_fullsized":"2024-04-25T00:04:50.709515+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:59.481396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718809+0000","last_change":"2024-04-25T00:04:50.709000+0000","last_active":"2024-04-25T00:04:50.718809+0000","last_peered":"2024-04-25T00:04:50.718809+0000","last_clean":"2024-04-25T00:04:50.718809+0000","last_became_active":"2024-04-25T00:04:50.708513+0000","last_became_peered":"2024-04-25T00:04:50.708513+0000","last_unstale":"2024-04-25T00:04:50.718809+0000","last_undegraded":"2024-04-25T00:04:50.718809+0000","last_fullsized":"2024-04-25T00:04:50.718809+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:22.029364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719073+0000","last_change":"2024-04-25T00:04:50.706975+0000","last_active":"2024-04-25T00:04:50.719073+0000","last_peered":"2024-04-25T00:04:50.719073+0000","last_clean":"2024-04-25T00:04:50.719073+0000","last_became_active":"2024-04-25T00:04:50.706695+0000","last_became_peered":"2024-04-25T00:04:50.706695+0000","last_unstale":"2024-04-25T00:04:50.719073+0000","last_undegraded":"2024-04-25T00:04:50.719073+0000","last_fullsized":"2024-04-25T00:04:50.719073+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:27.727405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717832+0000","last_change":"2024-04-25T00:04:50.715096+0000","last_active":"2024-04-25T00:04:50.717832+0000","last_peered":"2024-04-25T00:04:50.717832+0000","last_clean":"2024-04-25T00:04:50.717832+0000","last_became_active":"2024-04-25T00:04:50.714538+0000","last_became_peered":"2024-04-25T00:04:50.714538+0000","last_unstale":"2024-04-25T00:04:50.717832+0000","last_undegraded":"2024-04-25T00:04:50.717832+0000","last_fullsized":"2024-04-25T00:04:50.717832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:40.169163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719311+0000","last_change":"2024-04-25T00:04:50.717877+0000","last_active":"2024-04-25T00:04:50.719311+0000","last_peered":"2024-04-25T00:04:50.719311+0000","last_clean":"2024-04-25T00:04:50.719311+0000","last_became_active":"2024-04-25T00:04:50.717605+0000","last_became_peered":"2024-04-25T00:04:50.717605+0000","last_unstale":"2024-04-25T00:04:50.719311+0000","last_undegraded":"2024-04-25T00:04:50.719311+0000","last_fullsized":"2024-04-25T00:04:50.719311+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:37.576586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.708778+0000","last_change":"2024-04-25T00:04:50.699922+0000","last_active":"2024-04-25T00:04:50.708778+0000","last_peered":"2024-04-25T00:04:50.708778+0000","last_clean":"2024-04-25T00:04:50.708778+0000","last_became_active":"2024-04-25T00:04:50.699643+0000","last_became_peered":"2024-04-25T00:04:50.699643+0000","last_unstale":"2024-04-25T00:04:50.708778+0000","last_undegraded":"2024-04-25T00:04:50.708778+0000","last_fullsized":"2024-04-25T00:04:50.708778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:28.797591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718754+0000","last_change":"2024-04-25T00:04:50.715520+0000","last_active":"2024-04-25T00:04:50.718754+0000","last_peered":"2024-04-25T00:04:50.718754+0000","last_clean":"2024-04-25T00:04:50.718754+0000","last_became_active":"2024-04-25T00:04:50.715262+0000","last_became_peered":"2024-04-25T00:04:50.715262+0000","last_unstale":"2024-04-25T00:04:50.718754+0000","last_undegraded":"2024-04-25T00:04:50.718754+0000","last_fullsized":"2024-04-25T00:04:50.718754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:40.601371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719453+0000","last_change":"2024-04-25T00:04:50.718604+0000","last_active":"2024-04-25T00:04:50.719453+0000","last_peered":"2024-04-25T00:04:50.719453+0000","last_clean":"2024-04-25T00:04:50.719453+0000","last_became_active":"2024-04-25T00:04:50.717127+0000","last_became_peered":"2024-04-25T00:04:50.717127+0000","last_unstale":"2024-04-25T00:04:50.719453+0000","last_undegraded":"2024-04-25T00:04:50.719453+0000","last_fullsized":"2024-04-25T00:04:50.719453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:52.556475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.708819+0000","last_change":"2024-04-25T00:04:50.699874+0000","last_active":"2024-04-25T00:04:50.708819+0000","last_peered":"2024-04-25T00:04:50.708819+0000","last_clean":"2024-04-25T00:04:50.708819+0000","last_became_active":"2024-04-25T00:04:50.699585+0000","last_became_peered":"2024-04-25T00:04:50.699585+0000","last_unstale":"2024-04-25T00:04:50.708819+0000","last_undegraded":"2024-04-25T00:04:50.708819+0000","last_fullsized":"2024-04-25T00:04:50.708819+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:16.829314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.711867+0000","last_change":"2024-04-25T00:04:50.705759+0000","last_active":"2024-04-25T00:04:50.711867+0000","last_peered":"2024-04-25T00:04:50.711867+0000","last_clean":"2024-04-25T00:04:50.711867+0000","last_became_active":"2024-04-25T00:04:50.704542+0000","last_became_peered":"2024-04-25T00:04:50.704542+0000","last_unstale":"2024-04-25T00:04:50.711867+0000","last_undegraded":"2024-04-25T00:04:50.711867+0000","last_fullsized":"2024-04-25T00:04:50.711867+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:30.595333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720337+0000","last_change":"2024-04-25T00:04:50.717537+0000","last_active":"2024-04-25T00:04:50.720337+0000","last_peered":"2024-04-25T00:04:50.720337+0000","last_clean":"2024-04-25T00:04:50.720337+0000","last_became_active":"2024-04-25T00:04:50.717140+0000","last_became_peered":"2024-04-25T00:04:50.717140+0000","last_unstale":"2024-04-25T00:04:50.720337+0000","last_undegraded":"2024-04-25T00:04:50.720337+0000","last_fullsized":"2024-04-25T00:04:50.720337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:23:27.727405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709844+0000","last_change":"2024-04-25T00:04:50.706241+0000","last_active":"2024-04-25T00:04:50.709844+0000","last_peered":"2024-04-25T00:04:50.709844+0000","last_clean":"2024-04-25T00:04:50.709844+0000","last_became_active":"2024-04-25T00:04:50.702182+0000","last_became_peered":"2024-04-25T00:04:50.702182+0000","last_unstale":"2024-04-25T00:04:50.709844+0000","last_undegraded":"2024-04-25T00:04:50.709844+0000","last_fullsized":"2024-04-25T00:04:50.709844+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:30.036880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710448+0000","last_change":"2024-04-25T00:04:50.706712+0000","last_active":"2024-04-25T00:04:50.710448+0000","last_peered":"2024-04-25T00:04:50.710448+0000","last_clean":"2024-04-25T00:04:50.710448+0000","last_became_active":"2024-04-25T00:04:50.706461+0000","last_became_peered":"2024-04-25T00:04:50.706461+0000","last_unstale":"2024-04-25T00:04:50.710448+0000","last_undegraded":"2024-04-25T00:04:50.710448+0000","last_fullsized":"2024-04-25T00:04:50.710448+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:04.560398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721863+0000","last_change":"2024-04-25T00:04:49.700092+0000","last_active":"2024-04-25T00:04:50.721863+0000","last_peered":"2024-04-25T00:04:50.721863+0000","last_clean":"2024-04-25T00:04:50.721863+0000","last_became_active":"2024-04-25T00:04:49.699806+0000","last_became_peered":"2024-04-25T00:04:49.699806+0000","last_unstale":"2024-04-25T00:04:50.721863+0000","last_undegraded":"2024-04-25T00:04:50.721863+0000","last_fullsized":"2024-04-25T00:04:50.721863+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:36.571143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719208+0000","last_change":"2024-04-25T00:04:50.716389+0000","last_active":"2024-04-25T00:04:50.719208+0000","last_peered":"2024-04-25T00:04:50.719208+0000","last_clean":"2024-04-25T00:04:50.719208+0000","last_became_active":"2024-04-25T00:04:50.716129+0000","last_became_peered":"2024-04-25T00:04:50.716129+0000","last_unstale":"2024-04-25T00:04:50.719208+0000","last_undegraded":"2024-04-25T00:04:50.719208+0000","last_fullsized":"2024-04-25T00:04:50.719208+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:28.797591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720700+0000","last_change":"2024-04-25T00:04:49.705248+0000","last_active":"2024-04-25T00:04:50.720700+0000","last_peered":"2024-04-25T00:04:50.720700+0000","last_clean":"2024-04-25T00:04:50.720700+0000","last_became_active":"2024-04-25T00:04:49.704923+0000","last_became_peered":"2024-04-25T00:04:49.704923+0000","last_unstale":"2024-04-25T00:04:50.720700+0000","last_undegraded":"2024-04-25T00:04:50.720700+0000","last_fullsized":"2024-04-25T00:04:50.720700+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:50.052426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.702097+0000","last_change":"2024-04-25T00:04:50.700009+0000","last_active":"2024-04-25T00:04:50.702097+0000","last_peered":"2024-04-25T00:04:50.702097+0000","last_clean":"2024-04-25T00:04:50.702097+0000","last_became_active":"2024-04-25T00:04:50.699723+0000","last_became_peered":"2024-04-25T00:04:50.699723+0000","last_unstale":"2024-04-25T00:04:50.702097+0000","last_undegraded":"2024-04-25T00:04:50.702097+0000","last_fullsized":"2024-04-25T00:04:50.702097+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:15.932352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.722171+0000","last_change":"2024-04-25T00:04:49.707165+0000","last_active":"2024-04-25T00:04:50.722171+0000","last_peered":"2024-04-25T00:04:50.722171+0000","last_clean":"2024-04-25T00:04:50.722171+0000","last_became_active":"2024-04-25T00:04:49.706863+0000","last_became_peered":"2024-04-25T00:04:49.706863+0000","last_unstale":"2024-04-25T00:04:50.722171+0000","last_undegraded":"2024-04-25T00:04:50.722171+0000","last_fullsized":"2024-04-25T00:04:50.722171+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:25.443288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709528+0000","last_change":"2024-04-25T00:04:50.705884+0000","last_active":"2024-04-25T00:04:50.709528+0000","last_peered":"2024-04-25T00:04:50.709528+0000","last_clean":"2024-04-25T00:04:50.709528+0000","last_became_active":"2024-04-25T00:04:50.705594+0000","last_became_peered":"2024-04-25T00:04:50.705594+0000","last_unstale":"2024-04-25T00:04:50.709528+0000","last_undegraded":"2024-04-25T00:04:50.709528+0000","last_fullsized":"2024-04-25T00:04:50.709528+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:37.576586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718811+0000","last_change":"2024-04-25T00:04:49.701549+0000","last_active":"2024-04-25T00:04:50.718811+0000","last_peered":"2024-04-25T00:04:50.718811+0000","last_clean":"2024-04-25T00:04:50.718811+0000","last_became_active":"2024-04-25T00:04:49.700122+0000","last_became_peered":"2024-04-25T00:04:49.700122+0000","last_unstale":"2024-04-25T00:04:50.718811+0000","last_undegraded":"2024-04-25T00:04:50.718811+0000","last_fullsized":"2024-04-25T00:04:50.718811+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:51.551032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.711453+0000","last_change":"2024-04-25T00:04:50.709917+0000","last_active":"2024-04-25T00:04:50.711453+0000","last_peered":"2024-04-25T00:04:50.711453+0000","last_clean":"2024-04-25T00:04:50.711453+0000","last_became_active":"2024-04-25T00:04:50.709668+0000","last_became_peered":"2024-04-25T00:04:50.709668+0000","last_unstale":"2024-04-25T00:04:50.711453+0000","last_undegraded":"2024-04-25T00:04:50.711453+0000","last_fullsized":"2024-04-25T00:04:50.711453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:46.915644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719743+0000","last_change":"2024-04-25T00:04:49.698677+0000","last_active":"2024-04-25T00:04:50.719743+0000","last_peered":"2024-04-25T00:04:50.719743+0000","last_clean":"2024-04-25T00:04:50.719743+0000","last_became_active":"2024-04-25T00:04:49.698343+0000","last_became_peered":"2024-04-25T00:04:49.698343+0000","last_unstale":"2024-04-25T00:04:50.719743+0000","last_undegraded":"2024-04-25T00:04:50.719743+0000","last_fullsized":"2024-04-25T00:04:50.719743+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:46.442408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710634+0000","last_change":"2024-04-25T00:04:50.709182+0000","last_active":"2024-04-25T00:04:50.710634+0000","last_peered":"2024-04-25T00:04:50.710634+0000","last_clean":"2024-04-25T00:04:50.710634+0000","last_became_active":"2024-04-25T00:04:50.708917+0000","last_became_peered":"2024-04-25T00:04:50.708917+0000","last_unstale":"2024-04-25T00:04:50.710634+0000","last_undegraded":"2024-04-25T00:04:50.710634+0000","last_fullsized":"2024-04-25T00:04:50.710634+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:22.184636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720662+0000","last_change":"2024-04-25T00:04:49.697066+0000","last_active":"2024-04-25T00:04:50.720662+0000","last_peered":"2024-04-25T00:04:50.720662+0000","last_clean":"2024-04-25T00:04:50.720662+0000","last_became_active":"2024-04-25T00:04:49.696724+0000","last_became_peered":"2024-04-25T00:04:49.696724+0000","last_unstale":"2024-04-25T00:04:50.720662+0000","last_undegraded":"2024-04-25T00:04:50.720662+0000","last_fullsized":"2024-04-25T00:04:50.720662+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:08.738132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714849+0000","last_change":"2024-04-25T00:04:50.711416+0000","last_active":"2024-04-25T00:04:50.714849+0000","last_peered":"2024-04-25T00:04:50.714849+0000","last_clean":"2024-04-25T00:04:50.714849+0000","last_became_active":"2024-04-25T00:04:50.711151+0000","last_became_peered":"2024-04-25T00:04:50.711151+0000","last_unstale":"2024-04-25T00:04:50.714849+0000","last_undegraded":"2024-04-25T00:04:50.714849+0000","last_fullsized":"2024-04-25T00:04:50.714849+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:18.525910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.699988+0000","last_change":"2024-04-25T00:04:49.702715+0000","last_active":"2024-04-25T00:04:50.699988+0000","last_peered":"2024-04-25T00:04:50.699988+0000","last_clean":"2024-04-25T00:04:50.699988+0000","last_became_active":"2024-04-25T00:04:49.701768+0000","last_became_peered":"2024-04-25T00:04:49.701768+0000","last_unstale":"2024-04-25T00:04:50.699988+0000","last_undegraded":"2024-04-25T00:04:50.699988+0000","last_fullsized":"2024-04-25T00:04:50.699988+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:32.765807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715591+0000","last_change":"2024-04-25T00:04:50.707612+0000","last_active":"2024-04-25T00:04:50.715591+0000","last_peered":"2024-04-25T00:04:50.715591+0000","last_clean":"2024-04-25T00:04:50.715591+0000","last_became_active":"2024-04-25T00:04:50.706935+0000","last_became_peered":"2024-04-25T00:04:50.706935+0000","last_unstale":"2024-04-25T00:04:50.715591+0000","last_undegraded":"2024-04-25T00:04:50.715591+0000","last_fullsized":"2024-04-25T00:04:50.715591+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:52.556475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.697229+0000","last_change":"2024-04-25T00:04:49.702798+0000","last_active":"2024-04-25T00:04:50.697229+0000","last_peered":"2024-04-25T00:04:50.697229+0000","last_clean":"2024-04-25T00:04:50.697229+0000","last_became_active":"2024-04-25T00:04:49.702011+0000","last_became_peered":"2024-04-25T00:04:49.702011+0000","last_unstale":"2024-04-25T00:04:50.697229+0000","last_undegraded":"2024-04-25T00:04:50.697229+0000","last_fullsized":"2024-04-25T00:04:50.697229+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:45.936092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.699296+0000","last_change":"2024-04-25T00:04:50.697948+0000","last_active":"2024-04-25T00:04:50.699296+0000","last_peered":"2024-04-25T00:04:50.699296+0000","last_clean":"2024-04-25T00:04:50.699296+0000","last_became_active":"2024-04-25T00:04:50.697662+0000","last_became_peered":"2024-04-25T00:04:50.697662+0000","last_unstale":"2024-04-25T00:04:50.699296+0000","last_undegraded":"2024-04-25T00:04:50.699296+0000","last_fullsized":"2024-04-25T00:04:50.699296+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:55.460510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.696289+0000","last_change":"2024-04-25T00:04:49.704325+0000","last_active":"2024-04-25T00:04:50.696289+0000","last_peered":"2024-04-25T00:04:50.696289+0000","last_clean":"2024-04-25T00:04:50.696289+0000","last_became_active":"2024-04-25T00:04:49.703972+0000","last_became_peered":"2024-04-25T00:04:49.703972+0000","last_unstale":"2024-04-25T00:04:50.696289+0000","last_undegraded":"2024-04-25T00:04:50.696289+0000","last_fullsized":"2024-04-25T00:04:50.696289+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:39.357772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709680+0000","last_change":"2024-04-25T00:04:50.708479+0000","last_active":"2024-04-25T00:04:50.709680+0000","last_peered":"2024-04-25T00:04:50.709680+0000","last_clean":"2024-04-25T00:04:50.709680+0000","last_became_active":"2024-04-25T00:04:50.707209+0000","last_became_peered":"2024-04-25T00:04:50.707209+0000","last_unstale":"2024-04-25T00:04:50.709680+0000","last_undegraded":"2024-04-25T00:04:50.709680+0000","last_fullsized":"2024-04-25T00:04:50.709680+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:00.423063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.703672+0000","last_change":"2024-04-25T00:04:49.707114+0000","last_active":"2024-04-25T00:04:50.703672+0000","last_peered":"2024-04-25T00:04:50.703672+0000","last_clean":"2024-04-25T00:04:50.703672+0000","last_became_active":"2024-04-25T00:04:49.706823+0000","last_became_peered":"2024-04-25T00:04:49.706823+0000","last_unstale":"2024-04-25T00:04:50.703672+0000","last_undegraded":"2024-04-25T00:04:50.703672+0000","last_fullsized":"2024-04-25T00:04:50.703672+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:48.808861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719664+0000","last_change":"2024-04-25T00:04:50.719086+0000","last_active":"2024-04-25T00:04:50.719664+0000","last_peered":"2024-04-25T00:04:50.719664+0000","last_clean":"2024-04-25T00:04:50.719664+0000","last_became_active":"2024-04-25T00:04:50.718128+0000","last_became_peered":"2024-04-25T00:04:50.718128+0000","last_unstale":"2024-04-25T00:04:50.719664+0000","last_undegraded":"2024-04-25T00:04:50.719664+0000","last_fullsized":"2024-04-25T00:04:50.719664+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:00.423063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719971+0000","last_change":"2024-04-25T00:04:49.706365+0000","last_active":"2024-04-25T00:04:50.719971+0000","last_peered":"2024-04-25T00:04:50.719971+0000","last_clean":"2024-04-25T00:04:50.719971+0000","last_became_active":"2024-04-25T00:04:49.706063+0000","last_became_peered":"2024-04-25T00:04:49.706063+0000","last_unstale":"2024-04-25T00:04:50.719971+0000","last_undegraded":"2024-04-25T00:04:50.719971+0000","last_fullsized":"2024-04-25T00:04:50.719971+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:21.023921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721031+0000","last_change":"2024-04-25T00:04:50.720358+0000","last_active":"2024-04-25T00:04:50.721031+0000","last_peered":"2024-04-25T00:04:50.721031+0000","last_clean":"2024-04-25T00:04:50.721031+0000","last_became_active":"2024-04-25T00:04:50.720075+0000","last_became_peered":"2024-04-25T00:04:50.720075+0000","last_unstale":"2024-04-25T00:04:50.721031+0000","last_undegraded":"2024-04-25T00:04:50.721031+0000","last_fullsized":"2024-04-25T00:04:50.721031+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:23.235971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.727490+0000","last_change":"2024-04-25T00:04:49.704288+0000","last_active":"2024-04-25T00:04:50.727490+0000","last_peered":"2024-04-25T00:04:50.727490+0000","last_clean":"2024-04-25T00:04:50.727490+0000","last_became_active":"2024-04-25T00:04:49.703553+0000","last_became_peered":"2024-04-25T00:04:49.703553+0000","last_unstale":"2024-04-25T00:04:50.727490+0000","last_undegraded":"2024-04-25T00:04:50.727490+0000","last_fullsized":"2024-04-25T00:04:50.727490+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:30.528592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714795+0000","last_change":"2024-04-25T00:04:50.710809+0000","last_active":"2024-04-25T00:04:50.714795+0000","last_peered":"2024-04-25T00:04:50.714795+0000","last_clean":"2024-04-25T00:04:50.714795+0000","last_became_active":"2024-04-25T00:04:50.710551+0000","last_became_peered":"2024-04-25T00:04:50.710551+0000","last_unstale":"2024-04-25T00:04:50.714795+0000","last_undegraded":"2024-04-25T00:04:50.714795+0000","last_fullsized":"2024-04-25T00:04:50.714795+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:15.107357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717467+0000","last_change":"2024-04-25T00:04:49.708336+0000","last_active":"2024-04-25T00:04:50.717467+0000","last_peered":"2024-04-25T00:04:50.717467+0000","last_clean":"2024-04-25T00:04:50.717467+0000","last_became_active":"2024-04-25T00:04:49.707866+0000","last_became_peered":"2024-04-25T00:04:49.707866+0000","last_unstale":"2024-04-25T00:04:50.717467+0000","last_undegraded":"2024-04-25T00:04:50.717467+0000","last_fullsized":"2024-04-25T00:04:50.717467+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:42.557675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.712670+0000","last_change":"2024-04-25T00:04:50.707202+0000","last_active":"2024-04-25T00:04:50.712670+0000","last_peered":"2024-04-25T00:04:50.712670+0000","last_clean":"2024-04-25T00:04:50.712670+0000","last_became_active":"2024-04-25T00:04:50.706958+0000","last_became_peered":"2024-04-25T00:04:50.706958+0000","last_unstale":"2024-04-25T00:04:50.712670+0000","last_undegraded":"2024-04-25T00:04:50.712670+0000","last_fullsized":"2024-04-25T00:04:50.712670+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:41.537968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.700160+0000","last_change":"2024-04-25T00:04:49.711969+0000","last_active":"2024-04-25T00:04:50.700160+0000","last_peered":"2024-04-25T00:04:50.700160+0000","last_clean":"2024-04-25T00:04:50.700160+0000","last_became_active":"2024-04-25T00:04:49.711643+0000","last_became_peered":"2024-04-25T00:04:49.711643+0000","last_unstale":"2024-04-25T00:04:50.700160+0000","last_undegraded":"2024-04-25T00:04:50.700160+0000","last_fullsized":"2024-04-25T00:04:50.700160+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:26.761343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709250+0000","last_change":"2024-04-25T00:04:50.705531+0000","last_active":"2024-04-25T00:04:50.709250+0000","last_peered":"2024-04-25T00:04:50.709250+0000","last_clean":"2024-04-25T00:04:50.709250+0000","last_became_active":"2024-04-25T00:04:50.705148+0000","last_became_peered":"2024-04-25T00:04:50.705148+0000","last_unstale":"2024-04-25T00:04:50.709250+0000","last_undegraded":"2024-04-25T00:04:50.709250+0000","last_fullsized":"2024-04-25T00:04:50.709250+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:09.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.693939+0000","last_change":"2024-04-25T00:04:49.698185+0000","last_active":"2024-04-25T00:04:50.693939+0000","last_peered":"2024-04-25T00:04:50.693939+0000","last_clean":"2024-04-25T00:04:50.693939+0000","last_became_active":"2024-04-25T00:04:49.697860+0000","last_became_peered":"2024-04-25T00:04:49.697860+0000","last_unstale":"2024-04-25T00:04:50.693939+0000","last_undegraded":"2024-04-25T00:04:50.693939+0000","last_fullsized":"2024-04-25T00:04:50.693939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:31.033529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.720269+0000","last_change":"2024-04-25T00:04:50.717124+0000","last_active":"2024-04-25T00:04:50.720269+0000","last_peered":"2024-04-25T00:04:50.720269+0000","last_clean":"2024-04-25T00:04:50.720269+0000","last_became_active":"2024-04-25T00:04:50.716863+0000","last_became_peered":"2024-04-25T00:04:50.716863+0000","last_unstale":"2024-04-25T00:04:50.720269+0000","last_undegraded":"2024-04-25T00:04:50.720269+0000","last_fullsized":"2024-04-25T00:04:50.720269+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:47.447851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721885+0000","last_change":"2024-04-25T00:04:49.706532+0000","last_active":"2024-04-25T00:04:50.721885+0000","last_peered":"2024-04-25T00:04:50.721885+0000","last_clean":"2024-04-25T00:04:50.721885+0000","last_became_active":"2024-04-25T00:04:49.704248+0000","last_became_peered":"2024-04-25T00:04:49.704248+0000","last_unstale":"2024-04-25T00:04:50.721885+0000","last_undegraded":"2024-04-25T00:04:50.721885+0000","last_fullsized":"2024-04-25T00:04:50.721885+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:54.319114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.708872+0000","last_change":"2024-04-25T00:04:50.705150+0000","last_active":"2024-04-25T00:04:50.708872+0000","last_peered":"2024-04-25T00:04:50.708872+0000","last_clean":"2024-04-25T00:04:50.708872+0000","last_became_active":"2024-04-25T00:04:50.704132+0000","last_became_peered":"2024-04-25T00:04:50.704132+0000","last_unstale":"2024-04-25T00:04:50.708872+0000","last_undegraded":"2024-04-25T00:04:50.708872+0000","last_fullsized":"2024-04-25T00:04:50.708872+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:32.318327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.722338+0000","last_change":"2024-04-25T00:04:49.707053+0000","last_active":"2024-04-25T00:04:50.722338+0000","last_peered":"2024-04-25T00:04:50.722338+0000","last_clean":"2024-04-25T00:04:50.722338+0000","last_became_active":"2024-04-25T00:04:49.706724+0000","last_became_peered":"2024-04-25T00:04:49.706724+0000","last_unstale":"2024-04-25T00:04:50.722338+0000","last_undegraded":"2024-04-25T00:04:50.722338+0000","last_fullsized":"2024-04-25T00:04:50.722338+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:41.110533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717795+0000","last_change":"2024-04-25T00:04:50.717245+0000","last_active":"2024-04-25T00:04:50.717795+0000","last_peered":"2024-04-25T00:04:50.717795+0000","last_clean":"2024-04-25T00:04:50.717795+0000","last_became_active":"2024-04-25T00:04:50.716962+0000","last_became_peered":"2024-04-25T00:04:50.716962+0000","last_unstale":"2024-04-25T00:04:50.717795+0000","last_undegraded":"2024-04-25T00:04:50.717795+0000","last_fullsized":"2024-04-25T00:04:50.717795+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:09.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715483+0000","last_change":"2024-04-25T00:04:49.701091+0000","last_active":"2024-04-25T00:04:50.715483+0000","last_peered":"2024-04-25T00:04:50.715483+0000","last_clean":"2024-04-25T00:04:50.715483+0000","last_became_active":"2024-04-25T00:04:49.700350+0000","last_became_peered":"2024-04-25T00:04:49.700350+0000","last_unstale":"2024-04-25T00:04:50.715483+0000","last_undegraded":"2024-04-25T00:04:50.715483+0000","last_fullsized":"2024-04-25T00:04:50.715483+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:53.577335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715655+0000","last_change":"2024-04-25T00:04:50.713931+0000","last_active":"2024-04-25T00:04:50.715655+0000","last_peered":"2024-04-25T00:04:50.715655+0000","last_clean":"2024-04-25T00:04:50.715655+0000","last_became_active":"2024-04-25T00:04:50.713279+0000","last_became_peered":"2024-04-25T00:04:50.713279+0000","last_unstale":"2024-04-25T00:04:50.715655+0000","last_undegraded":"2024-04-25T00:04:50.715655+0000","last_fullsized":"2024-04-25T00:04:50.715655+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:23.853626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695394+0000","last_change":"2024-04-25T00:04:49.704292+0000","last_active":"2024-04-25T00:04:50.695394+0000","last_peered":"2024-04-25T00:04:50.695394+0000","last_clean":"2024-04-25T00:04:50.695394+0000","last_became_active":"2024-04-25T00:04:49.704053+0000","last_became_peered":"2024-04-25T00:04:49.704053+0000","last_unstale":"2024-04-25T00:04:50.695394+0000","last_undegraded":"2024-04-25T00:04:50.695394+0000","last_fullsized":"2024-04-25T00:04:50.695394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:34.335248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.717765+0000","last_change":"2024-04-25T00:04:50.713772+0000","last_active":"2024-04-25T00:04:50.717765+0000","last_peered":"2024-04-25T00:04:50.717765+0000","last_clean":"2024-04-25T00:04:50.717765+0000","last_became_active":"2024-04-25T00:04:50.713532+0000","last_became_peered":"2024-04-25T00:04:50.713532+0000","last_unstale":"2024-04-25T00:04:50.717765+0000","last_undegraded":"2024-04-25T00:04:50.717765+0000","last_fullsized":"2024-04-25T00:04:50.717765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:55.726257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.722678+0000","last_change":"2024-04-25T00:04:49.707370+0000","last_active":"2024-04-25T00:04:50.722678+0000","last_peered":"2024-04-25T00:04:50.722678+0000","last_clean":"2024-04-25T00:04:50.722678+0000","last_became_active":"2024-04-25T00:04:49.707113+0000","last_became_peered":"2024-04-25T00:04:49.707113+0000","last_unstale":"2024-04-25T00:04:50.722678+0000","last_undegraded":"2024-04-25T00:04:50.722678+0000","last_fullsized":"2024-04-25T00:04:50.722678+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:59.417620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.723260+0000","last_change":"2024-04-25T00:04:50.722655+0000","last_active":"2024-04-25T00:04:50.723260+0000","last_peered":"2024-04-25T00:04:50.723260+0000","last_clean":"2024-04-25T00:04:50.723260+0000","last_became_active":"2024-04-25T00:04:50.722353+0000","last_became_peered":"2024-04-25T00:04:50.722353+0000","last_unstale":"2024-04-25T00:04:50.723260+0000","last_undegraded":"2024-04-25T00:04:50.723260+0000","last_fullsized":"2024-04-25T00:04:50.723260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:15.932352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.701516+0000","last_change":"2024-04-25T00:04:49.709190+0000","last_active":"2024-04-25T00:04:50.701516+0000","last_peered":"2024-04-25T00:04:50.701516+0000","last_clean":"2024-04-25T00:04:50.701516+0000","last_became_active":"2024-04-25T00:04:49.708908+0000","last_became_peered":"2024-04-25T00:04:49.708908+0000","last_unstale":"2024-04-25T00:04:50.701516+0000","last_undegraded":"2024-04-25T00:04:50.701516+0000","last_fullsized":"2024-04-25T00:04:50.701516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:31.033529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719464+0000","last_change":"2024-04-25T00:04:50.715463+0000","last_active":"2024-04-25T00:04:50.719464+0000","last_peered":"2024-04-25T00:04:50.719464+0000","last_clean":"2024-04-25T00:04:50.719464+0000","last_became_active":"2024-04-25T00:04:50.715200+0000","last_became_peered":"2024-04-25T00:04:50.715200+0000","last_unstale":"2024-04-25T00:04:50.719464+0000","last_undegraded":"2024-04-25T00:04:50.719464+0000","last_fullsized":"2024-04-25T00:04:50.719464+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:40.363215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.703291+0000","last_change":"2024-04-25T00:04:49.704021+0000","last_active":"2024-04-25T00:04:50.703291+0000","last_peered":"2024-04-25T00:04:50.703291+0000","last_clean":"2024-04-25T00:04:50.703291+0000","last_became_active":"2024-04-25T00:04:49.703669+0000","last_became_peered":"2024-04-25T00:04:49.703669+0000","last_unstale":"2024-04-25T00:04:50.703291+0000","last_undegraded":"2024-04-25T00:04:50.703291+0000","last_fullsized":"2024-04-25T00:04:50.703291+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:30.528592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.714899+0000","last_change":"2024-04-25T00:04:50.708746+0000","last_active":"2024-04-25T00:04:50.714899+0000","last_peered":"2024-04-25T00:04:50.714899+0000","last_clean":"2024-04-25T00:04:50.714899+0000","last_became_active":"2024-04-25T00:04:50.707499+0000","last_became_peered":"2024-04-25T00:04:50.707499+0000","last_unstale":"2024-04-25T00:04:50.714899+0000","last_undegraded":"2024-04-25T00:04:50.714899+0000","last_fullsized":"2024-04-25T00:04:50.714899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:42.115976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695455+0000","last_change":"2024-04-25T00:04:49.699269+0000","last_active":"2024-04-25T00:04:50.695455+0000","last_peered":"2024-04-25T00:04:50.695455+0000","last_clean":"2024-04-25T00:04:50.695455+0000","last_became_active":"2024-04-25T00:04:49.699014+0000","last_became_peered":"2024-04-25T00:04:49.699014+0000","last_unstale":"2024-04-25T00:04:50.695455+0000","last_undegraded":"2024-04-25T00:04:50.695455+0000","last_fullsized":"2024-04-25T00:04:50.695455+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:48.808861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718771+0000","last_change":"2024-04-25T00:04:50.710664+0000","last_active":"2024-04-25T00:04:50.718771+0000","last_peered":"2024-04-25T00:04:50.718771+0000","last_clean":"2024-04-25T00:04:50.718771+0000","last_became_active":"2024-04-25T00:04:50.710357+0000","last_became_peered":"2024-04-25T00:04:50.710357+0000","last_unstale":"2024-04-25T00:04:50.718771+0000","last_undegraded":"2024-04-25T00:04:50.718771+0000","last_fullsized":"2024-04-25T00:04:50.718771+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:22.184636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.718605+0000","last_change":"2024-04-25T00:04:49.704259+0000","last_active":"2024-04-25T00:04:50.718605+0000","last_peered":"2024-04-25T00:04:50.718605+0000","last_clean":"2024-04-25T00:04:50.718605+0000","last_became_active":"2024-04-25T00:04:49.703737+0000","last_became_peered":"2024-04-25T00:04:49.703737+0000","last_unstale":"2024-04-25T00:04:50.718605+0000","last_undegraded":"2024-04-25T00:04:50.718605+0000","last_fullsized":"2024-04-25T00:04:50.718605+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:41.110533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.710338+0000","last_change":"2024-04-25T00:04:50.706648+0000","last_active":"2024-04-25T00:04:50.710338+0000","last_peered":"2024-04-25T00:04:50.710338+0000","last_clean":"2024-04-25T00:04:50.710338+0000","last_became_active":"2024-04-25T00:04:50.706376+0000","last_became_peered":"2024-04-25T00:04:50.706376+0000","last_unstale":"2024-04-25T00:04:50.710338+0000","last_undegraded":"2024-04-25T00:04:50.710338+0000","last_fullsized":"2024-04-25T00:04:50.710338+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:26.318174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.696101+0000","last_change":"2024-04-25T00:04:49.706380+0000","last_active":"2024-04-25T00:04:50.696101+0000","last_peered":"2024-04-25T00:04:50.696101+0000","last_clean":"2024-04-25T00:04:50.696101+0000","last_became_active":"2024-04-25T00:04:49.703814+0000","last_became_peered":"2024-04-25T00:04:49.703814+0000","last_unstale":"2024-04-25T00:04:50.696101+0000","last_undegraded":"2024-04-25T00:04:50.696101+0000","last_fullsized":"2024-04-25T00:04:50.696101+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:17.520467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719311+0000","last_change":"2024-04-25T00:04:50.712815+0000","last_active":"2024-04-25T00:04:50.719311+0000","last_peered":"2024-04-25T00:04:50.719311+0000","last_clean":"2024-04-25T00:04:50.719311+0000","last_became_active":"2024-04-25T00:04:50.712440+0000","last_became_peered":"2024-04-25T00:04:50.712440+0000","last_unstale":"2024-04-25T00:04:50.719311+0000","last_undegraded":"2024-04-25T00:04:50.719311+0000","last_fullsized":"2024-04-25T00:04:50.719311+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:09:59.955428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695496+0000","last_change":"2024-04-25T00:04:49.707755+0000","last_active":"2024-04-25T00:04:50.695496+0000","last_peered":"2024-04-25T00:04:50.695496+0000","last_clean":"2024-04-25T00:04:50.695496+0000","last_became_active":"2024-04-25T00:04:49.707514+0000","last_became_peered":"2024-04-25T00:04:49.707514+0000","last_unstale":"2024-04-25T00:04:50.695496+0000","last_undegraded":"2024-04-25T00:04:50.695496+0000","last_fullsized":"2024-04-25T00:04:50.695496+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:25.666226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719221+0000","last_change":"2024-04-25T00:04:50.716292+0000","last_active":"2024-04-25T00:04:50.719221+0000","last_peered":"2024-04-25T00:04:50.719221+0000","last_clean":"2024-04-25T00:04:50.719221+0000","last_became_active":"2024-04-25T00:04:50.715979+0000","last_became_peered":"2024-04-25T00:04:50.715979+0000","last_unstale":"2024-04-25T00:04:50.719221+0000","last_undegraded":"2024-04-25T00:04:50.719221+0000","last_fullsized":"2024-04-25T00:04:50.719221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:09.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.707900+0000","last_change":"2024-04-25T00:04:49.704507+0000","last_active":"2024-04-25T00:04:50.707900+0000","last_peered":"2024-04-25T00:04:50.707900+0000","last_clean":"2024-04-25T00:04:50.707900+0000","last_became_active":"2024-04-25T00:04:49.701378+0000","last_became_peered":"2024-04-25T00:04:49.701378+0000","last_unstale":"2024-04-25T00:04:50.707900+0000","last_undegraded":"2024-04-25T00:04:50.707900+0000","last_fullsized":"2024-04-25T00:04:50.707900+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:49.617006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.715120+0000","last_change":"2024-04-25T00:04:50.712219+0000","last_active":"2024-04-25T00:04:50.715120+0000","last_peered":"2024-04-25T00:04:50.715120+0000","last_clean":"2024-04-25T00:04:50.715120+0000","last_became_active":"2024-04-25T00:04:50.711656+0000","last_became_peered":"2024-04-25T00:04:50.711656+0000","last_unstale":"2024-04-25T00:04:50.715120+0000","last_undegraded":"2024-04-25T00:04:50.715120+0000","last_fullsized":"2024-04-25T00:04:50.715120+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:17.311346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.695557+0000","last_change":"2024-04-25T00:04:49.701995+0000","last_active":"2024-04-25T00:04:50.695557+0000","last_peered":"2024-04-25T00:04:50.695557+0000","last_clean":"2024-04-25T00:04:50.695557+0000","last_became_active":"2024-04-25T00:04:49.701096+0000","last_became_peered":"2024-04-25T00:04:49.701096+0000","last_unstale":"2024-04-25T00:04:50.695557+0000","last_undegraded":"2024-04-25T00:04:50.695557+0000","last_fullsized":"2024-04-25T00:04:50.695557+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:14.101914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.712621+0000","last_change":"2024-04-25T00:04:50.710044+0000","last_active":"2024-04-25T00:04:50.712621+0000","last_peered":"2024-04-25T00:04:50.712621+0000","last_clean":"2024-04-25T00:04:50.712621+0000","last_became_active":"2024-04-25T00:04:50.709758+0000","last_became_peered":"2024-04-25T00:04:50.709758+0000","last_unstale":"2024-04-25T00:04:50.712621+0000","last_undegraded":"2024-04-25T00:04:50.712621+0000","last_fullsized":"2024-04-25T00:04:50.712621+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:35.340691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.699435+0000","last_change":"2024-04-25T00:04:49.698249+0000","last_active":"2024-04-25T00:04:50.699435+0000","last_peered":"2024-04-25T00:04:50.699435+0000","last_clean":"2024-04-25T00:04:50.699435+0000","last_became_active":"2024-04-25T00:04:49.697983+0000","last_became_peered":"2024-04-25T00:04:49.697983+0000","last_unstale":"2024-04-25T00:04:50.699435+0000","last_undegraded":"2024-04-25T00:04:50.699435+0000","last_fullsized":"2024-04-25T00:04:50.699435+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:37.433492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.709484+0000","last_change":"2024-04-25T00:04:50.704693+0000","last_active":"2024-04-25T00:04:50.709484+0000","last_peered":"2024-04-25T00:04:50.709484+0000","last_clean":"2024-04-25T00:04:50.709484+0000","last_became_active":"2024-04-25T00:04:50.704412+0000","last_became_peered":"2024-04-25T00:04:50.704412+0000","last_unstale":"2024-04-25T00:04:50.709484+0000","last_undegraded":"2024-04-25T00:04:50.709484+0000","last_fullsized":"2024-04-25T00:04:50.709484+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:06.534925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.706502+0000","last_change":"2024-04-25T00:04:49.706055+0000","last_active":"2024-04-25T00:04:50.706502+0000","last_peered":"2024-04-25T00:04:50.706502+0000","last_clean":"2024-04-25T00:04:50.706502+0000","last_became_active":"2024-04-25T00:04:49.705790+0000","last_became_peered":"2024-04-25T00:04:49.705790+0000","last_unstale":"2024-04-25T00:04:50.706502+0000","last_undegraded":"2024-04-25T00:04:50.706502+0000","last_fullsized":"2024-04-25T00:04:50.706502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:25.666226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.719268+0000","last_change":"2024-04-25T00:04:50.716267+0000","last_active":"2024-04-25T00:04:50.719268+0000","last_peered":"2024-04-25T00:04:50.719268+0000","last_clean":"2024-04-25T00:04:50.719268+0000","last_became_active":"2024-04-25T00:04:50.715954+0000","last_became_peered":"2024-04-25T00:04:50.715954+0000","last_unstale":"2024-04-25T00:04:50.719268+0000","last_undegraded":"2024-04-25T00:04:50.719268+0000","last_fullsized":"2024-04-25T00:04:50.719268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:46.915644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.700193+0000","last_change":"2024-04-25T00:04:49.708898+0000","last_active":"2024-04-25T00:04:50.700193+0000","last_peered":"2024-04-25T00:04:50.700193+0000","last_clean":"2024-04-25T00:04:50.700193+0000","last_became_active":"2024-04-25T00:04:49.708642+0000","last_became_peered":"2024-04-25T00:04:49.708642+0000","last_unstale":"2024-04-25T00:04:50.700193+0000","last_undegraded":"2024-04-25T00:04:50.700193+0000","last_fullsized":"2024-04-25T00:04:50.700193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:35.592808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.716257+0000","last_change":"2024-04-25T00:04:50.713843+0000","last_active":"2024-04-25T00:04:50.716257+0000","last_peered":"2024-04-25T00:04:50.716257+0000","last_clean":"2024-04-25T00:04:50.716257+0000","last_became_active":"2024-04-25T00:04:50.713611+0000","last_became_peered":"2024-04-25T00:04:50.713611+0000","last_unstale":"2024-04-25T00:04:50.716257+0000","last_undegraded":"2024-04-25T00:04:50.716257+0000","last_fullsized":"2024-04-25T00:04:50.716257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:49.683726+0000","last_clean_scrub_stamp":"2024-04-25T00:04:49.683726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:55.324557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.721146+0000","last_change":"2024-04-25T00:04:49.703373+0000","last_active":"2024-04-25T00:04:50.721146+0000","last_peered":"2024-04-25T00:04:50.721146+0000","last_clean":"2024-04-25T00:04:50.721146+0000","last_became_active":"2024-04-25T00:04:49.702658+0000","last_became_peered":"2024-04-25T00:04:49.702658+0000","last_unstale":"2024-04-25T00:04:50.721146+0000","last_undegraded":"2024-04-25T00:04:50.721146+0000","last_fullsized":"2024-04-25T00:04:50.721146+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:04:48.678283+0000","last_clean_scrub_stamp":"2024-04-25T00:04:48.678283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:35.651334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T00:04:50.700263+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:04:50.700263+0000","last_peered":"2024-04-25T00:04:50.700263+0000","last_clean":"2024-04-25T00:04:50.700263+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:04:50.700263+0000","last_undegraded":"2024-04-25T00:04:50.700263+0000","last_fullsized":"2024-04-25T00:04:50.700263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55220,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316620,"statfs":{"total":96636764160,"available":96580218880,"internally_reserved":0,"allocated":1875968,"data_stored":1141010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58788,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313052,"statfs":{"total":96636764160,"available":96576565248,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58784,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313056,"statfs":{"total":96636764160,"available":96576569344,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58788,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313052,"statfs":{"total":96636764160,"available":96576565248,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40736,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331104,"statfs":{"total":96636764160,"available":96595050496,"internally_reserved":0,"allocated":1261568,"data_stored":543755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50472,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321368,"statfs":{"total":96636764160,"available":96585080832,"internally_reserved":0,"allocated":1273856,"data_stored":550530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41332,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330508,"statfs":{"total":96636764160,"available":96594440192,"internally_reserved":0,"allocated":1871872,"data_stored":1141480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:04:54.008 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:04:54.008 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:04:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.147+0000 7f693a45d640 1 Processor -- start 2024-04-25T00:04:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.147+0000 7f693a45d640 1 -- start start 2024-04-25T00:04:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.148+0000 7f693a45d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415ba50 0x7f693415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.148+0000 7f693a45d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f693406cea0 0x7f693406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.148+0000 7f693a45d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6934071170 con 0x7f693415b670 2024-04-25T00:04:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.148+0000 7f693a45d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f693406d810 con 0x7f693406cea0 2024-04-25T00:04:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.148+0000 7f693a45d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6934071450 con 0x7f693415ba50 2024-04-25T00:04:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69389fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f693406cea0 0x7f693406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69337fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415ba50 0x7f693415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69389fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f693406cea0 0x7f693406d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:34040/0 (socket says 172.21.15.113:34040) 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69389fa640 1 -- 172.21.15.113:0/276221976 learned_addr learned my addr 172.21.15.113:0/276221976 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69337fe640 1 -- 172.21.15.113:0/276221976 >> v2:172.21.15.178:3300/0 conn(0x7f693406cea0 msgr2=0x7f693406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69337fe640 1 --2- 172.21.15.113:0/276221976 >> v2:172.21.15.178:3300/0 conn(0x7f693406cea0 0x7f693406d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69337fe640 1 -- 172.21.15.113:0/276221976 >> v1:172.21.15.178:6789/0 conn(0x7f693415b670 legacy=0x7f69340cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69337fe640 1 -- 172.21.15.113:0/276221976 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6934071730 con 0x7f693415ba50 2024-04-25T00:04:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.149+0000 7f69389fa640 1 --2- 172.21.15.113:0/276221976 >> v2:172.21.15.178:3300/0 conn(0x7f693406cea0 0x7f693406d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f69337fe640 1 --2- 172.21.15.113:0/276221976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415ba50 0x7f693415bff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f69280097a0 tx=0x7f692802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7787be3f03ce96a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f6932ffd640 1 -- 172.21.15.113:0/276221976 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928037070 con 0x7f693415ba50 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f6932ffd640 1 -- 172.21.15.113:0/276221976 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692802adb0 con 0x7f693415ba50 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f6932ffd640 1 -- 172.21.15.113:0/276221976 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928032660 con 0x7f693415ba50 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f693a45d640 1 -- 172.21.15.113:0/276221976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415ba50 msgr2=0x7f693415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f693a45d640 1 --2- 172.21.15.113:0/276221976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415ba50 0x7f693415bff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f69280097a0 tx=0x7f692802a060 comp rx=0 tx=0).stop 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f693a45d640 1 -- 172.21.15.113:0/276221976 shutdown_connections 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f693a45d640 1 --2- 172.21.15.113:0/276221976 >> v2:172.21.15.178:3300/0 conn(0x7f693406cea0 0x7f693406d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f693a45d640 1 --2- 172.21.15.113:0/276221976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415ba50 0x7f693415bff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.150+0000 7f693a45d640 1 -- 172.21.15.113:0/276221976 >> 172.21.15.113:0/276221976 conn(0x7f693406b0f0 msgr2=0x7f693406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.151+0000 7f693a45d640 1 -- 172.21.15.113:0/276221976 shutdown_connections 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.151+0000 7f693a45d640 1 -- 172.21.15.113:0/276221976 wait complete. 2024-04-25T00:04:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 Processor -- start 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 -- start start 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f693406cea0 0x7f693414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415b670 0x7f6934154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f693415ba50 0x7f6934156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f69340cca90 con 0x7f693415ba50 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69340cc910 con 0x7f693406cea0 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.152+0000 7f693a45d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f69340d0080 con 0x7f693415b670 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f693415ba50 0x7f6934156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f693415ba50 0x7f6934156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47940/0 (socket says 172.21.15.113:47940) 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69337fe640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415b670 0x7f6934154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 -- 172.21.15.113:0/1372664691 learned_addr learned my addr 172.21.15.113:0/1372664691 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f6933fff640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f693406cea0 0x7f693414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 -- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415b670 msgr2=0x7f6934154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415b670 0x7f6934154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 -- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f693406cea0 msgr2=0x7f693414e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f693406cea0 0x7f693414e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 -- 172.21.15.113:0/1372664691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693414d600 con 0x7f693415ba50 2024-04-25T00:04:54.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f69389fa640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f693415ba50 0x7f6934156d60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f692c00efc0 tx=0x7f692c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:54.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f6930ff9640 1 -- 172.21.15.113:0/1372664691 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c00cbe0 con 0x7f693415ba50 2024-04-25T00:04:54.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f6930ff9640 1 -- 172.21.15.113:0/1372664691 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c00cd80 con 0x7f693415ba50 2024-04-25T00:04:54.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f6930ff9640 1 -- 172.21.15.113:0/1372664691 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c003bb0 con 0x7f693415ba50 2024-04-25T00:04:54.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6934154c60 con 0x7f693415ba50 2024-04-25T00:04:54.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.153+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693414db50 con 0x7f693415ba50 2024-04-25T00:04:54.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.154+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69340691a0 con 0x7f693415ba50 2024-04-25T00:04:54.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.155+0000 7f6930ff9640 1 -- 172.21.15.113:0/1372664691 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f692c01b030 con 0x7f693415ba50 2024-04-25T00:04:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.155+0000 7f6930ff9640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f692006be60 0x7f692006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.155+0000 7f6930ff9640 1 -- 172.21.15.113:0/1372664691 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f692c08b0c0 con 0x7f693415ba50 2024-04-25T00:04:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.155+0000 7f6933fff640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f692006be60 0x7f692006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.156+0000 7f6933fff640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f692006be60 0x7f692006e320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f69340cff70 tx=0x7f6924009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.159+0000 7f6930ff9640 1 -- 172.21.15.113:0/1372664691 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f692c022020 con 0x7f693415ba50 2024-04-25T00:04:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.397+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f693406c100 con 0x7f693415ba50 2024-04-25T00:04:54.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.399+0000 7f6930ff9640 1 -- 172.21.15.113:0/1372664691 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f692c010040 con 0x7f693415ba50 2024-04-25T00:04:54.400 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:04:54.400 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:04:54.400 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:04:54.400 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:04:54.400 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:04:54.400 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f692006be60 msgr2=0x7f692006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f692006be60 0x7f692006e320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f69340cff70 tx=0x7f6924009040 comp rx=0 tx=0).stop 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f693415ba50 msgr2=0x7f6934156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f693415ba50 0x7f6934156d60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f692c00efc0 tx=0x7f692c00c490 comp rx=0 tx=0).stop 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 shutdown_connections 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f692006be60 0x7f692006e320 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f693415ba50 0x7f6934156d60 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f693415b670 0x7f6934154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 --2- 172.21.15.113:0/1372664691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f693406cea0 0x7f693414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.401+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 >> 172.21.15.113:0/1372664691 conn(0x7f693406b0f0 msgr2=0x7f693415c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.402+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 shutdown_connections 2024-04-25T00:04:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.402+0000 7f693a45d640 1 -- 172.21.15.113:0/1372664691 wait complete. 2024-04-25T00:04:54.413 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:04:54.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.509+0000 7f61eb43f640 1 Processor -- start 2024-04-25T00:04:54.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.509+0000 7f61eb43f640 1 -- start start 2024-04-25T00:04:54.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.510+0000 7f61eb43f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 0x7f61e40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.510+0000 7f61eb43f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f61e40d0b00 0x7f61e40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.510+0000 7f61eb43f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e40d3de0 con 0x7f61e40d2290 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.510+0000 7f61eb43f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f61e40d14d0 con 0x7f61e40d0b00 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.510+0000 7f61eb43f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f61e40d3410 con 0x7f61e415e320 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.510+0000 7f61e89da640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f61e40d2290 0x7f61e40b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38222/0 (socket says 172.21.15.113:38222) 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.510+0000 7f61e89da640 1 -- 172.21.15.113:0/4191565512 learned_addr learned my addr 172.21.15.113:0/4191565512 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e91db640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 0x7f61e40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 862497075 0 0) 0x7f61e40d3de0 con 0x7f61e40d2290 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61d0003520 con 0x7f61e40d2290 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e99dc640 1 --2- 172.21.15.113:0/4191565512 >> v2:172.21.15.178:3300/0 conn(0x7f61e40d0b00 0x7f61e40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3159920986 0 0) 0x7f61d0003520 con 0x7f61e40d2290 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 msgr2=0x7f61e40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 0x7f61e40d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 >> v2:172.21.15.178:3300/0 conn(0x7f61e40d0b00 msgr2=0x7f61e40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:54.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 --2- 172.21.15.113:0/4191565512 >> v2:172.21.15.178:3300/0 conn(0x7f61e40d0b00 0x7f61e40d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e40d2800 con 0x7f61e40d2290 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f61d4001670 con 0x7f61e40d2290 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 >> v1:172.21.15.113:6789/0 conn(0x7f61e40d2290 legacy=0x7f61e40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 0x7f61d00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0003f30 0x7f61d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0004d40 0x7f61d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61e40d14d0 con 0x7f61d0004d40 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61e40d3410 con 0x7f61d0003f30 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f61d0014200 con 0x7f61e415e320 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e99dc640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0004d40 0x7f61d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e91db640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 0x7f61d00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e89da640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0003f30 0x7f61d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e99dc640 1 -- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 msgr2=0x7f61d00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e99dc640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415e320 0x7f61d00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e99dc640 1 -- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0003f30 msgr2=0x7f61d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e99dc640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0003f30 0x7f61d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e99dc640 1 -- 172.21.15.113:0/4191565512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d0003520 con 0x7f61d0004d40 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.511+0000 7f61e89da640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0003f30 0x7f61d0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.512+0000 7f61e99dc640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0004d40 0x7f61d00051c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009ac0 tx=0x7f61d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8130b1b14336485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.512+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc036070 con 0x7f61d0004d40 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.512+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc031400 con 0x7f61d0004d40 2024-04-25T00:04:54.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.512+0000 7f61dbfff640 1 -- 172.21.15.113:0/4191565512 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc009690 con 0x7f61d0004d40 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.513+0000 7f61eb43f640 1 -- 172.21.15.113:0/4191565512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0004d40 msgr2=0x7f61d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.513+0000 7f61eb43f640 1 --2- 172.21.15.113:0/4191565512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0004d40 0x7f61d00051c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009ac0 tx=0x7f61d0005830 comp rx=0 tx=0).stop 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.513+0000 7f61e99dc640 1 -- 172.21.15.113:0/4191565512 reap_dead start 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.513+0000 7f61eb43f640 1 -- 172.21.15.113:0/4191565512 shutdown_connections 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.513+0000 7f61eb43f640 1 -- 172.21.15.113:0/4191565512 >> 172.21.15.113:0/4191565512 conn(0x7f61e40b5c20 msgr2=0x7f61e40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.513+0000 7f61eb43f640 1 -- 172.21.15.113:0/4191565512 shutdown_connections 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.513+0000 7f61eb43f640 1 -- 172.21.15.113:0/4191565512 wait complete. 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.514+0000 7f61eb43f640 1 Processor -- start 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.514+0000 7f61eb43f640 1 -- start start 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61eb43f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0004d40 0x7f61e40cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61eb43f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415eb20 0x7f61e415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61eb43f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e41520f0 0x7f61e41525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61eb43f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61e40d30a0 con 0x7f61e41520f0 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61eb43f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61e40d2dc0 con 0x7f61d0004d40 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61eb43f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f61e415e700 con 0x7f61e415eb20 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e41520f0 0x7f61e41525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e41520f0 0x7f61e41525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47958/0 (socket says 172.21.15.113:47958) 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 -- 172.21.15.113:0/2520991455 learned_addr learned my addr 172.21.15.113:0/2520991455 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e89da640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415eb20 0x7f61e415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 -- 172.21.15.113:0/2520991455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415eb20 msgr2=0x7f61e415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415eb20 0x7f61e415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 -- 172.21.15.113:0/2520991455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0004d40 msgr2=0x7f61e40cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0004d40 0x7f61e40cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 -- 172.21.15.113:0/2520991455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e4160390 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61e99dc640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e41520f0 0x7f61e41525a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009ac0 tx=0x7f61dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61d9ffb640 1 -- 172.21.15.113:0/2520991455 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc036040 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61d9ffb640 1 -- 172.21.15.113:0/2520991455 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc041070 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e40cba30 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.515+0000 7f61d9ffb640 1 -- 172.21.15.113:0/2520991455 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc009690 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.516+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e40cbff0 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.517+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e40691a0 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.517+0000 7f61d9ffb640 1 -- 172.21.15.113:0/2520991455 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61dc039070 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.517+0000 7f61d9ffb640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f61c806be60 0x7f61c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.518+0000 7f61e91db640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f61c806be60 0x7f61c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.518+0000 7f61d9ffb640 1 -- 172.21.15.113:0/2520991455 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f61dc0aa6b0 con 0x7f61e41520f0 2024-04-25T00:04:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.518+0000 7f61e91db640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f61c806be60 0x7f61c806e320 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009fc0 tx=0x7f61e0009750 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:54.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.522+0000 7f61d9ffb640 1 -- 172.21.15.113:0/2520991455 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61dc04a020 con 0x7f61e41520f0 2024-04-25T00:04:54.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.733+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e4152ae0 con 0x7f61e41520f0 2024-04-25T00:04:54.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.733+0000 7f61d9ffb640 1 -- 172.21.15.113:0/2520991455 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7f61dc079060 con 0x7f61e41520f0 2024-04-25T00:04:54.771 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:04:54.771 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f61c806be60 msgr2=0x7f61c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f61c806be60 0x7f61c806e320 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009fc0 tx=0x7f61e0009750 comp rx=0 tx=0).stop 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e41520f0 msgr2=0x7f61e41525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e41520f0 0x7f61e41525a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009ac0 tx=0x7f61dc004730 comp rx=0 tx=0).stop 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 shutdown_connections 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f61c806be60 0x7f61c806e320 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e41520f0 0x7f61e41525a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e415eb20 0x7f61e415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 --2- 172.21.15.113:0/2520991455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61d0004d40 0x7f61e40cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 >> 172.21.15.113:0/2520991455 conn(0x7f61e40b5c20 msgr2=0x7f61e40b7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 shutdown_connections 2024-04-25T00:04:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:54.736+0000 7f61eb43f640 1 -- 172.21.15.113:0/2520991455 wait complete. 2024-04-25T00:04:55.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:04:50.047438+0000', 'modified': '2024-04-25T00:04:52.043934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7848}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7848': {'gid': 7848, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6836/677893386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 677893386}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 677893386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7848, 'qdb_cluster': [7848]} 2024-04-25T00:04:55.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:04:55.179 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:04:55.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.277+0000 7f779fe9e640 1 Processor -- start 2024-04-25T00:04:55.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.278+0000 7f779fe9e640 1 -- start start 2024-04-25T00:04:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.278+0000 7f779fe9e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f77980b6100 0x7f77980b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.278+0000 7f779fe9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77980d2e00 con 0x7f77980b5ca0 2024-04-25T00:04:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.278+0000 7f779fe9e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77980d30e0 con 0x7f779815e510 2024-04-25T00:04:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.278+0000 7f779fe9e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f77980d33c0 con 0x7f77980b6100 2024-04-25T00:04:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779d439640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f77980b5ca0 0x7f77980b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38232/0 (socket says 172.21.15.113:38232) 2024-04-25T00:04:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779d439640 1 -- 172.21.15.113:0/3581752172 learned_addr learned my addr 172.21.15.113:0/3581752172 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779e43b640 1 --2- 172.21.15.113:0/3581752172 >> v2:172.21.15.178:3300/0 conn(0x7f77980b6100 0x7f77980b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895716505 0 0) 0x7f77980d2e00 con 0x7f77980b5ca0 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7784003520 con 0x7f77980b5ca0 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 34940594 0 0) 0x7f7784003520 con 0x7f77980b5ca0 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 >> v2:172.21.15.178:3300/0 conn(0x7f77980b6100 msgr2=0x7f77980b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 --2- 172.21.15.113:0/3581752172 >> v2:172.21.15.178:3300/0 conn(0x7f77980b6100 0x7f77980b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 >> v1:172.21.15.178:6789/0 conn(0x7f779815e510 legacy=0x7f77980d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77980d36a0 con 0x7f77980b5ca0 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f7794001670 con 0x7f77980b5ca0 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 >> v1:172.21.15.113:6789/0 conn(0x7f77980b5ca0 legacy=0x7f77980b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7784003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784004800 0x7f7784004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77840051c0 0x7f778400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77980d33c0 con 0x7f77840039b0 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f778400c160 con 0x7f77840051c0 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f778400c360 con 0x7f7784004800 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.279+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7784003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779d439640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784004800 0x7f7784004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779e43b640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77840051c0 0x7f778400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779dc3a640 1 -- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784004800 msgr2=0x7f7784004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784004800 0x7f7784004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779dc3a640 1 -- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77840051c0 msgr2=0x7f778400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77840051c0 0x7f778400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779dc3a640 1 -- 172.21.15.113:0/3581752172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7784003520 con 0x7f77840039b0 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779e43b640 1 -- 172.21.15.113:0/3581752172 reap_dead start 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7784003dd0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7788001750 tx=0x7f7784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769740a6adcfe72a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788001370 con 0x7f77840039b0 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77880052d0 con 0x7f77840039b0 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.280+0000 7f779cc38640 1 -- 172.21.15.113:0/3581752172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778802abc0 con 0x7f77840039b0 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.281+0000 7f779fe9e640 1 -- 172.21.15.113:0/3581752172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 msgr2=0x7f7784003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.281+0000 7f779fe9e640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7784003dd0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7788001750 tx=0x7f7784005830 comp rx=0 tx=0).stop 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.281+0000 7f779fe9e640 1 -- 172.21.15.113:0/3581752172 shutdown_connections 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.281+0000 7f779fe9e640 1 --2- 172.21.15.113:0/3581752172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7784003dd0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.281+0000 7f779fe9e640 1 -- 172.21.15.113:0/3581752172 >> 172.21.15.113:0/3581752172 conn(0x7f779806b720 msgr2=0x7f77980d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.281+0000 7f779fe9e640 1 -- 172.21.15.113:0/3581752172 shutdown_connections 2024-04-25T00:04:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.281+0000 7f779fe9e640 1 -- 172.21.15.113:0/3581752172 wait complete. 2024-04-25T00:04:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.282+0000 7f779fe9e640 1 Processor -- start 2024-04-25T00:04:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.282+0000 7f779fe9e640 1 -- start start 2024-04-25T00:04:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779fe9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7798156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779fe9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77981612f0 0x7f7798151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779fe9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7798157d00 0x7f7798158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779fe9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77980b6510 con 0x7f77840039b0 2024-04-25T00:04:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779fe9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77980d3ea0 con 0x7f7798157d00 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779fe9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77980b9dd0 con 0x7f77981612f0 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7798156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7798156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47974/0 (socket says 172.21.15.113:47974) 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 -- 172.21.15.113:0/4132614913 learned_addr learned my addr 172.21.15.113:0/4132614913 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779d439640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77981612f0 0x7f7798151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779e43b640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7798157d00 0x7f7798158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 -- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77981612f0 msgr2=0x7f7798151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77981612f0 0x7f7798151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 -- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7798157d00 msgr2=0x7f7798158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7798157d00 0x7f7798158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 -- 172.21.15.113:0/4132614913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7798161730 con 0x7f77840039b0 2024-04-25T00:04:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779e43b640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7798157d00 0x7f7798158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779d439640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77981612f0 0x7f7798151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.283+0000 7f779dc3a640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7798156520 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7788001870 tx=0x7f7788001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.284+0000 7f778e7fc640 1 -- 172.21.15.113:0/4132614913 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788040030 con 0x7f77840039b0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.284+0000 7f778e7fc640 1 -- 172.21.15.113:0/4132614913 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778802be50 con 0x7f77840039b0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.284+0000 7f778e7fc640 1 -- 172.21.15.113:0/4132614913 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788001370 con 0x7f77840039b0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.284+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7798156ba0 con 0x7f77840039b0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.284+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7798157160 con 0x7f77840039b0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.285+0000 7f778e7fc640 1 -- 172.21.15.113:0/4132614913 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7788043070 con 0x7f77840039b0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.286+0000 7f778e7fc640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f777c06be60 0x7f777c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.286+0000 7f779d439640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f777c06be60 0x7f777c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.286+0000 7f778e7fc640 1 -- 172.21.15.113:0/4132614913 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f77880aa5a0 con 0x7f77840039b0 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.286+0000 7f779d439640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f777c06be60 0x7f777c06e320 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7794001eb0 tx=0x7f7794009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 Processor -- start 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 -- start start 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2bbe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2bee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2bee540 0x5634d2bc3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5634d2b26f00 con 0x5634d2beec80 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5634d2acf750 con 0x5634d2bee540 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5634d2b22ec0 con 0x5634d2c0bb60 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f779d439640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2bee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f779d439640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2bee000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47982/0 (socket says 172.21.15.113:47982) 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.287+0000 7f779d439640 1 -- 172.21.15.113:0/2756433769 learned_addr learned my addr 172.21.15.113:0/2756433769 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779dc3a640 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2bbe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779e43b640 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2bee540 0x5634d2bc3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779d439640 1 -- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 msgr2=0x5634d2bbe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779d439640 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2bbe4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779d439640 1 -- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2bee540 msgr2=0x5634d2bc3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779d439640 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2bee540 0x5634d2bc3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779d439640 1 -- 172.21.15.113:0/2756433769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d283b2d0 con 0x5634d2beec80 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779e43b640 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2bee540 0x5634d2bc3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f779d439640 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2bee000 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7794010c70 tx=0x7f779400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d29eafca165960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f777b7fe640 1 -- 172.21.15.113:0/2756433769 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794027070 con 0x5634d2beec80 2024-04-25T00:04:55.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f777b7fe640 1 -- 172.21.15.113:0/2756433769 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7794005250 con 0x5634d2beec80 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.288+0000 7f777b7fe640 1 -- 172.21.15.113:0/2756433769 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794005600 con 0x5634d2beec80 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 -- 172.21.15.113:0/2756433769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 msgr2=0x5634d2bee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2bee000 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7794010c70 tx=0x7f779400e000 comp rx=0 tx=0).stop 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 -- 172.21.15.113:0/2756433769 shutdown_connections 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2bee540 0x5634d2bc3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2bee000 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 --2- 172.21.15.113:0/2756433769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2bbe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 -- 172.21.15.113:0/2756433769 >> 172.21.15.113:0/2756433769 conn(0x5634d2bc0760 msgr2=0x5634d2c09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 -- 172.21.15.113:0/2756433769 shutdown_connections 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 -- 172.21.15.113:0/2756433769 wait complete. 2024-04-25T00:04:55.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 -- start start 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.289+0000 7f77a2baa740 1 Processor -- start 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2bf1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2be9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f77a2baa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5634d2bc2910 con 0x5634d2be9bc0 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5634d2ae2cf0 con 0x5634d2c0bb60 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5634d2bc1d00 con 0x5634d2beec80 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f77a2baa740 10 client.0 _ll_register_callbacks cb 0x5634d2ae26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f779e43b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f779e43b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c0c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47984/0 (socket says 172.21.15.113:47984) 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f779e43b640 1 -- 172.21.15.113:0/3437869762 learned_addr learned my addr 172.21.15.113:0/3437869762 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:55.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2bf1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.290+0000 7f779d439640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2be9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779e43b640 1 -- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 msgr2=0x5634d2bf1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779e43b640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2bf1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779e43b640 1 -- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 msgr2=0x5634d2be9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779e43b640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2be9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779e43b640 1 -- 172.21.15.113:0/3437869762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d29056b0 con 0x5634d2be9bc0 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779d439640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2be9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779e43b640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c0c880 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f77980d1b90 tx=0x7f7790011600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d11d3e5e6cd640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f7778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f7778ff9640 1 -- 172.21.15.113:0/3437869762 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790009e60 con 0x5634d2be9bc0 2024-04-25T00:04:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f7778ff9640 1 -- 172.21.15.113:0/3437869762 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f7790011e30 con 0x5634d2be9bc0 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7794005250 con 0x5634d2be9bc0 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f7778ff9640 1 -- 172.21.15.113:0/3437869762 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f7790026050 con 0x5634d2be9bc0 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 10 client.7986 fetch_fsmap learned FSMap version 56 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x5634d2bfff10 con 0x5634d2be9bc0 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f7778ff9640 1 -- 172.21.15.113:0/3437869762 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7794005250 con 0x5634d2be9bc0 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f7778ff9640 1 -- 172.21.15.113:0/3437869762 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7f7790029020 con 0x5634d2be9bc0 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 10 client.7986 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 10 client.7986 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 20 client.7986 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2bef640 0x5634d2c00a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 4 client.7986 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:55.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634d2b34440 con 0x5634d2bef640 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.291+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2bef640 0x5634d2c00a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f7778ff9640 10 client.7986 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2bef640 0x5634d2c00a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c6244505d5e45b8 server_cookie=d36cce32cafb1d1a in_seq=0 out_seq=0 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f7778ff9640 1 -- 172.21.15.113:0/3437869762 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634d2b34440 con 0x5634d2bef640 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f7778ff9640 10 client.7986 handle_command_reply: tid=0 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 client.7986 shutdown 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 msgr2=0x5634d2c0c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c0c880 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f77980d1b90 tx=0x7f7790011600 comp rx=0 tx=0).stop 2024-04-25T00:04:55.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 shutdown_connections 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2bef640 0x5634d2c00a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2be9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c0c880 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f7778ff9640 0 client.7986 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3437869762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2bf1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.292+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 >> 172.21.15.113:0/3437869762 conn(0x5634d2bee540 msgr2=0x5634d2bbd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.293+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 shutdown_connections 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.293+0000 7f77a2baa740 1 -- 172.21.15.113:0/3437869762 wait complete. 2024-04-25T00:04:55.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.293+0000 7f77a2baa740 20 client.7986 trim_cache size 0 max 16384 2024-04-25T00:04:55.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 Processor -- start 2024-04-25T00:04:55.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 -- start start 2024-04-25T00:04:55.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c09600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2c02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2bbd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7798200180 con 0x5634d2be9bc0 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7798152df0 con 0x5634d2c0bb60 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77981ffc60 con 0x5634d2beec80 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.311+0000 7f779dc3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c09600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c09600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47996/0 (socket says 172.21.15.113:47996) 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 -- 172.21.15.113:0/183115897 learned_addr learned my addr 172.21.15.113:0/183115897 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779d439640 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2c02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779e43b640 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2bbd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 -- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 msgr2=0x5634d2c02070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2c02070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 -- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 msgr2=0x5634d2bbd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2bbd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 -- 172.21.15.113:0/183115897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d2bc0390 con 0x5634d2be9bc0 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f779dc3a640 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c09600 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7788078750 tx=0x7f778807b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685f12cb521b02fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f777affd640 1 -- 172.21.15.113:0/183115897 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880787c0 con 0x5634d2be9bc0 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f777affd640 1 -- 172.21.15.113:0/183115897 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778807cb70 con 0x5634d2be9bc0 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.312+0000 7f777affd640 1 -- 172.21.15.113:0/183115897 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788081430 con 0x5634d2be9bc0 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 -- 172.21.15.113:0/183115897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 msgr2=0x5634d2c09600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c09600 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7788078750 tx=0x7f778807b040 comp rx=0 tx=0).stop 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 -- 172.21.15.113:0/183115897 shutdown_connections 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2c0bb60 0x5634d2bbd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2beec80 0x5634d2c02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 --2- 172.21.15.113:0/183115897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2be9bc0 0x5634d2c09600 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 -- 172.21.15.113:0/183115897 >> 172.21.15.113:0/183115897 conn(0x5634d2bee540 msgr2=0x5634d2c08e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 -- 172.21.15.113:0/183115897 shutdown_connections 2024-04-25T00:04:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 -- 172.21.15.113:0/183115897 wait complete. 2024-04-25T00:04:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 -- start start 2024-04-25T00:04:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.313+0000 7f77a2baa740 1 Processor -- start 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2be9bc0 0x5634d2c806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2c86b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f77a2baa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2c87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f77a2baa740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5634d2bef4c0 con 0x5634d2beec80 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5634d2c19380 con 0x5634d2be9bc0 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f77a2baa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5634d2befb10 con 0x5634d2c0bb60 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f77a2baa740 10 client.0 _ll_register_callbacks cb 0x7f7790013930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f779d439640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2c86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f779d439640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2c86b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48002/0 (socket says 172.21.15.113:48002) 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f779d439640 1 -- 172.21.15.113:0/3379475495 learned_addr learned my addr 172.21.15.113:0/3379475495 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2be9bc0 0x5634d2c806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.314+0000 7f779e43b640 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2c87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f779d439640 1 -- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 msgr2=0x5634d2c87080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f779d439640 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2c87080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f779d439640 1 -- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2be9bc0 msgr2=0x5634d2c806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f779d439640 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2be9bc0 0x5634d2c806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f779d439640 1 -- 172.21.15.113:0/3379475495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d2c191b0 con 0x5634d2beec80 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f779d439640 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2c86b40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7794006fd0 tx=0x7f77980d2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86897b86bdf5ddec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f7778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f7778ff9640 1 -- 172.21.15.113:0/3379475495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794027040 con 0x5634d2beec80 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f7778ff9640 1 -- 172.21.15.113:0/3379475495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f7794021620 con 0x5634d2beec80 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778807cb70 con 0x5634d2beec80 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f7778ff9640 1 -- 172.21.15.113:0/3379475495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f779403a050 con 0x5634d2beec80 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 10 client.7992 fetch_fsmap learned FSMap version 56 2024-04-25T00:04:55.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x5634d2c83e60 con 0x5634d2beec80 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f7778ff9640 1 -- 172.21.15.113:0/3379475495 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778807cb70 con 0x5634d2beec80 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f7778ff9640 1 -- 172.21.15.113:0/3379475495 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7f779403d020 con 0x5634d2beec80 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 10 client.7992 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 10 client.7992 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 20 client.7992 populate_metadata read hostname 'smithi113' 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2c181c0 0x5634d2c0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 4 client.7992 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "status"}) -- 0x5634d2bf2570 con 0x5634d2c181c0 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.315+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2c181c0 0x5634d2c0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f7778ff9640 10 client.7992 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f779dc3a640 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2c181c0 0x5634d2c0a130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f5127e795064ea server_cookie=db916b2be1e3f06e in_seq=0 out_seq=0 2024-04-25T00:04:55.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f7778ff9640 1 -- 172.21.15.113:0/3379475495 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5634d2bf2570 con 0x5634d2c181c0 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f7778ff9640 10 client.7992 handle_command_reply: tid=0 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 client.7992 shutdown 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 msgr2=0x5634d2c86b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2c86b40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7794006fd0 tx=0x7f77980d2fd0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 shutdown_connections 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x5634d2c181c0 0x5634d2c0a130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5634d2c0bb60 0x5634d2c87080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634d2beec80 0x5634d2c86b40 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f7778ff9640 0 client.7992 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 --2- 172.21.15.113:0/3379475495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5634d2be9bc0 0x5634d2c806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 >> 172.21.15.113:0/3379475495 conn(0x5634d2bee540 msgr2=0x5634d2bbd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 shutdown_connections 2024-04-25T00:04:55.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 1 -- 172.21.15.113:0/3379475495 wait complete. 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.316+0000 7f77a2baa740 20 client.7992 trim_cache size 0 max 16384 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 7848, 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.6257651190000004, 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 56, 2024-04-25T00:04:55.325 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 104, 2024-04-25T00:04:55.326 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 104, 2024-04-25T00:04:55.326 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 9.6825496350000009 2024-04-25T00:04:55.326 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:04:55.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f777c06be60 msgr2=0x7f777c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f777c06be60 0x7f777c06e320 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7794001eb0 tx=0x7f7794009040 comp rx=0 tx=0).stop 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 msgr2=0x7f7798156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7798156520 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7788001870 tx=0x7f7788001120 comp rx=0 tx=0).stop 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 shutdown_connections 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f777c06be60 0x7f777c06e320 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7798157d00 0x7f7798158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77981612f0 0x7f7798151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 --2- 172.21.15.113:0/4132614913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77840039b0 0x7f7798156520 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 >> 172.21.15.113:0/4132614913 conn(0x7f779806b720 msgr2=0x7f779815c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 shutdown_connections 2024-04-25T00:04:55.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.317+0000 7f779fe9e640 1 -- 172.21.15.113:0/4132614913 wait complete. 2024-04-25T00:04:55.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:04:55.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.424+0000 7f4740d34640 1 Processor -- start 2024-04-25T00:04:55.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.424+0000 7f4740d34640 1 -- start start 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f4740d34640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f473c0d2290 0x7f473c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f4740d34640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f473c0d0b00 0x7f473c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f4740d34640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f473c0d1480 con 0x7f473c15e320 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f4740d34640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f473c0d3410 con 0x7f473c0d0b00 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f4740d34640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f473c0d2800 con 0x7f473c0d2290 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f473a59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f473c15e320 0x7f473c0d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38234/0 (socket says 172.21.15.113:38234) 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f473ad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f473c0d0b00 0x7f473c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.425+0000 7f473a59c640 1 -- 172.21.15.113:0/4037062611 learned_addr learned my addr 172.21.15.113:0/4037062611 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f4739d9b640 1 --2- 172.21.15.113:0/4037062611 >> v2:172.21.15.178:3300/0 conn(0x7f473c0d2290 0x7f473c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f473ad9d640 1 -- 172.21.15.113:0/4037062611 >> v2:172.21.15.178:3300/0 conn(0x7f473c0d2290 msgr2=0x7f473c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f473ad9d640 1 --2- 172.21.15.113:0/4037062611 >> v2:172.21.15.178:3300/0 conn(0x7f473c0d2290 0x7f473c0b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f473ad9d640 1 -- 172.21.15.113:0/4037062611 >> v1:172.21.15.113:6789/0 conn(0x7f473c15e320 legacy=0x7f473c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f473ad9d640 1 -- 172.21.15.113:0/4037062611 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c0d2ae0 con 0x7f473c0d0b00 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f473959a640 1 -- 172.21.15.113:0/4037062611 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 296769594 0 0) 0x7f473c0d1480 con 0x7f473c15e320 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f4739d9b640 1 --2- 172.21.15.113:0/4037062611 >> v2:172.21.15.178:3300/0 conn(0x7f473c0d2290 0x7f473c0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.426+0000 7f473ad9d640 1 --2- 172.21.15.113:0/4037062611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c0d0f40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f473c072810 tx=0x7f473402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104d8f9466e4d0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f473959a640 1 -- 172.21.15.113:0/4037062611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734037070 con 0x7f473c0d0b00 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f473959a640 1 -- 172.21.15.113:0/4037062611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4734032440 con 0x7f473c0d0b00 2024-04-25T00:04:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f473959a640 1 -- 172.21.15.113:0/4037062611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734032810 con 0x7f473c0d0b00 2024-04-25T00:04:55.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 -- 172.21.15.113:0/4037062611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 msgr2=0x7f473c0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 --2- 172.21.15.113:0/4037062611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c0d0f40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f473c072810 tx=0x7f473402a230 comp rx=0 tx=0).stop 2024-04-25T00:04:55.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 -- 172.21.15.113:0/4037062611 shutdown_connections 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 --2- 172.21.15.113:0/4037062611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c0d0f40 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 --2- 172.21.15.113:0/4037062611 >> v2:172.21.15.178:3300/0 conn(0x7f473c0d2290 0x7f473c0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 -- 172.21.15.113:0/4037062611 >> 172.21.15.113:0/4037062611 conn(0x7f473c0b5c20 msgr2=0x7f473c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 -- 172.21.15.113:0/4037062611 shutdown_connections 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.427+0000 7f4740d34640 1 -- 172.21.15.113:0/4037062611 wait complete. 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.428+0000 7f4740d34640 1 Processor -- start 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.428+0000 7f4740d34640 1 -- start start 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f4740d34640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c1f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f4740d34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473c0d2290 0x7f473c15fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f4740d34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473c15e320 0x7f473c160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f4740d34640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f473c0b9fe0 con 0x7f473c0d0b00 2024-04-25T00:04:55.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f4740d34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f473c0d30a0 con 0x7f473c0d2290 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f4740d34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f473c15e700 con 0x7f473c15e320 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f473a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c1f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f473a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c1f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48028/0 (socket says 172.21.15.113:48028) 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f473a59c640 1 -- 172.21.15.113:0/3376053165 learned_addr learned my addr 172.21.15.113:0/3376053165 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f4739d9b640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473c0d2290 0x7f473c15fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.429+0000 7f473ad9d640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473c15e320 0x7f473c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f473a59c640 1 -- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473c15e320 msgr2=0x7f473c160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f473a59c640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473c15e320 0x7f473c160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f473a59c640 1 -- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473c0d2290 msgr2=0x7f473c15fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f473a59c640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473c0d2290 0x7f473c15fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f473a59c640 1 -- 172.21.15.113:0/3376053165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c1f4e80 con 0x7f473c0d0b00 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f473a59c640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c1f4940 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c8c0 tx=0x7f472c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f472affd640 1 -- 172.21.15.113:0/3376053165 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c018070 con 0x7f473c0d0b00 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f472affd640 1 -- 172.21.15.113:0/3376053165 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472c00a640 con 0x7f473c0d0b00 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c15eaa0 con 0x7f473c0d0b00 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f472affd640 1 -- 172.21.15.113:0/3376053165 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c00aa30 con 0x7f473c0d0b00 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.430+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c15f060 con 0x7f473c0d0b00 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.431+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c0691a0 con 0x7f473c0d0b00 2024-04-25T00:04:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.431+0000 7f472affd640 1 -- 172.21.15.113:0/3376053165 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f472c01b070 con 0x7f473c0d0b00 2024-04-25T00:04:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.432+0000 7f472affd640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f471c06be30 0x7f471c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.432+0000 7f472affd640 1 -- 172.21.15.113:0/3376053165 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f472c08a570 con 0x7f473c0d0b00 2024-04-25T00:04:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.432+0000 7f4739d9b640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f471c06be30 0x7f471c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.432+0000 7f4739d9b640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f471c06be30 0x7f471c06e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f473c0b5a80 tx=0x7f4730008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:55.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.436+0000 7f472affd640 1 -- 172.21.15.113:0/3376053165 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f472c058f20 con 0x7f473c0d0b00 2024-04-25T00:04:55.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.645+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f473c15cf30 con 0x7f473c0d0b00 2024-04-25T00:04:55.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.645+0000 7f472affd640 1 -- 172.21.15.113:0/3376053165 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7f472c058820 con 0x7f473c0d0b00 2024-04-25T00:04:55.645 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:04:55.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f471c06be30 msgr2=0x7f471c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f471c06be30 0x7f471c06e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f473c0b5a80 tx=0x7f4730008040 comp rx=0 tx=0).stop 2024-04-25T00:04:55.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 msgr2=0x7f473c1f4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:55.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c1f4940 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c8c0 tx=0x7f472c00cd80 comp rx=0 tx=0).stop 2024-04-25T00:04:55.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 shutdown_connections 2024-04-25T00:04:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f471c06be30 0x7f471c06e2f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473c0d2290 0x7f473c15fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473c15e320 0x7f473c160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 --2- 172.21.15.113:0/3376053165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f473c0d0b00 0x7f473c1f4940 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 >> 172.21.15.113:0/3376053165 conn(0x7f473c0b5c20 msgr2=0x7f473c0d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 shutdown_connections 2024-04-25T00:04:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55.647+0000 7f4740d34640 1 -- 172.21.15.113:0/3376053165 wait complete. 2024-04-25T00:04:55.658 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:04:55.658 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:04:55.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:04:55.658 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:04:55.658 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:55.659 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:04:55.659 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:04:55.659 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:04:55.659 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:04:55.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:55.659 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:04:55.674 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:55.675 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:04:55.676 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42385sec preferred_lft 42385sec 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:04:55.677 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:04:55.678 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:04:55.678 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:55.678 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:04:55.678 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:04:55.678 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:04:55.678 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:04:55.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:55.679 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:04:55.737 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:04:55.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:55.738 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:04:55.794 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:04:55.795 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:04:55.795 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:04:55.795 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:04:55.795 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:04:55.795 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:04:55.795 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:55.795 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:04:55.795 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:04:55.795 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:04:55.795 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:55.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:04:55.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:55.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:55.918 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:04:55.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:04:55.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:04:55.934 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:04:55.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:55.934 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:04:55.969 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:04:55.968+0000 7f98f517b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:04:55.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:04:55.992 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:56.047 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:04:56.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:04:56.047 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:56.101 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:04:56.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:56.102 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:56.183 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:56.183 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:04:56.183 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:04:56.191 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:56.191 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:04:56.191 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:04:56.192 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:04:56.192 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:04:56.196 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:04:56.196 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:04:56.196 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:04:56.196 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:04:56.196 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:04:56.196 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:04:56.196 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:04:56.197 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:04:56.197 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:04:56.197 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:04:56.197 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:04:56.197 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:04:56.197 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:04:56.198 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:04:56.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:04:56.198 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:56.263 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:04:56.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:56.363 INFO:teuthology.orchestra.run.smithi113.stdout:client8004 2024-04-25T00:04:56.363 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:04:56.363 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:04:56.378 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:04:56.378 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T00:04:56.379 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:56.379 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:56.379 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T00:04:56.380 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:56.380 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:56.380 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:56.380 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:56.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.485 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00947026 s, 111 MB/s 2024-04-25T00:04:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087836 s, 119 MB/s 2024-04-25T00:04:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843962 s, 124 MB/s 2024-04-25T00:04:56.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727081 s, 144 MB/s 2024-04-25T00:04:56.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754351 s, 139 MB/s 2024-04-25T00:04:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582477 s, 180 MB/s 2024-04-25T00:04:56.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.540 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840584 s, 125 MB/s 2024-04-25T00:04:56.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.549 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740198 s, 142 MB/s 2024-04-25T00:04:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077149 s, 136 MB/s 2024-04-25T00:04:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640673 s, 164 MB/s 2024-04-25T00:04:56.575 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.575 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.575 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735075 s, 143 MB/s 2024-04-25T00:04:56.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.585 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.585 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816803 s, 128 MB/s 2024-04-25T00:04:56.593 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.593 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.593 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713465 s, 147 MB/s 2024-04-25T00:04:56.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.603 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797235 s, 132 MB/s 2024-04-25T00:04:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723871 s, 145 MB/s 2024-04-25T00:04:56.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.620 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672237 s, 156 MB/s 2024-04-25T00:04:56.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695917 s, 151 MB/s 2024-04-25T00:04:56.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565809 s, 185 MB/s 2024-04-25T00:04:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750306 s, 140 MB/s 2024-04-25T00:04:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076406 s, 137 MB/s 2024-04-25T00:04:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711173 s, 147 MB/s 2024-04-25T00:04:56.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591308 s, 177 MB/s 2024-04-25T00:04:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699373 s, 150 MB/s 2024-04-25T00:04:56.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.688 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757879 s, 138 MB/s 2024-04-25T00:04:56.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739659 s, 142 MB/s 2024-04-25T00:04:56.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.704 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585599 s, 179 MB/s 2024-04-25T00:04:56.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.713 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722808 s, 145 MB/s 2024-04-25T00:04:56.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.723 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778509 s, 135 MB/s 2024-04-25T00:04:56.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067672 s, 155 MB/s 2024-04-25T00:04:56.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.740 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660681 s, 159 MB/s 2024-04-25T00:04:56.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.749 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692374 s, 151 MB/s 2024-04-25T00:04:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748237 s, 140 MB/s 2024-04-25T00:04:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:56.765 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:56.765 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:56.765 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T00:04:56.765 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:56.765 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:56.765 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:56.765 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:56.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.823 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582255 s, 180 MB/s 2024-04-25T00:04:56.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.831 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059352 s, 177 MB/s 2024-04-25T00:04:56.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.840 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670654 s, 156 MB/s 2024-04-25T00:04:56.849 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.849 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.849 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770908 s, 136 MB/s 2024-04-25T00:04:56.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.858 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072475 s, 145 MB/s 2024-04-25T00:04:56.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.866 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612043 s, 171 MB/s 2024-04-25T00:04:56.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.875 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717957 s, 146 MB/s 2024-04-25T00:04:56.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.882 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575511 s, 182 MB/s 2024-04-25T00:04:56.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.889 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573084 s, 183 MB/s 2024-04-25T00:04:56.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.898 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697643 s, 150 MB/s 2024-04-25T00:04:56.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.907 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728523 s, 144 MB/s 2024-04-25T00:04:56.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701368 s, 150 MB/s 2024-04-25T00:04:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.925 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.925 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730952 s, 143 MB/s 2024-04-25T00:04:56.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.933 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730238 s, 144 MB/s 2024-04-25T00:04:56.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.941 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055932 s, 187 MB/s 2024-04-25T00:04:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702996 s, 149 MB/s 2024-04-25T00:04:56.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.958 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727931 s, 144 MB/s 2024-04-25T00:04:56.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.966 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700065 s, 150 MB/s 2024-04-25T00:04:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646527 s, 162 MB/s 2024-04-25T00:04:56.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.984 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.984 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710948 s, 147 MB/s 2024-04-25T00:04:56.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:56.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:56.992 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672355 s, 156 MB/s 2024-04-25T00:04:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749058 s, 140 MB/s 2024-04-25T00:04:57.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.009 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692637 s, 151 MB/s 2024-04-25T00:04:57.018 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.018 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.018 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728017 s, 144 MB/s 2024-04-25T00:04:57.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.025 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581617 s, 180 MB/s 2024-04-25T00:04:57.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.034 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729385 s, 144 MB/s 2024-04-25T00:04:57.043 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.043 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.043 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747057 s, 140 MB/s 2024-04-25T00:04:57.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.051 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066186 s, 158 MB/s 2024-04-25T00:04:57.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.060 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708574 s, 148 MB/s 2024-04-25T00:04:57.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.069 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.069 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708628 s, 148 MB/s 2024-04-25T00:04:57.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.076 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567474 s, 185 MB/s 2024-04-25T00:04:57.084 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.085 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735356 s, 143 MB/s 2024-04-25T00:04:57.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:57.091 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:57.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:57.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719794 s, 146 MB/s 2024-04-25T00:04:57.157 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.157 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.157 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575899 s, 182 MB/s 2024-04-25T00:04:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.167 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705993 s, 149 MB/s 2024-04-25T00:04:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562862 s, 186 MB/s 2024-04-25T00:04:57.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.180 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562482 s, 186 MB/s 2024-04-25T00:04:57.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.188 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568511 s, 184 MB/s 2024-04-25T00:04:57.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.196 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685112 s, 153 MB/s 2024-04-25T00:04:57.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568859 s, 184 MB/s 2024-04-25T00:04:57.210 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.211 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584829 s, 179 MB/s 2024-04-25T00:04:57.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.218 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600857 s, 175 MB/s 2024-04-25T00:04:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568694 s, 184 MB/s 2024-04-25T00:04:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812718 s, 129 MB/s 2024-04-25T00:04:57.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.244 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731014 s, 143 MB/s 2024-04-25T00:04:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689066 s, 152 MB/s 2024-04-25T00:04:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708858 s, 148 MB/s 2024-04-25T00:04:57.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669225 s, 157 MB/s 2024-04-25T00:04:57.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694263 s, 151 MB/s 2024-04-25T00:04:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075376 s, 139 MB/s 2024-04-25T00:04:57.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.296 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668905 s, 157 MB/s 2024-04-25T00:04:57.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.306 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758908 s, 138 MB/s 2024-04-25T00:04:57.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606841 s, 173 MB/s 2024-04-25T00:04:57.322 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.322 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.322 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785894 s, 133 MB/s 2024-04-25T00:04:57.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.330 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683732 s, 153 MB/s 2024-04-25T00:04:57.339 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.339 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.339 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744837 s, 141 MB/s 2024-04-25T00:04:57.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.349 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753519 s, 139 MB/s 2024-04-25T00:04:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588969 s, 178 MB/s 2024-04-25T00:04:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006738 s, 156 MB/s 2024-04-25T00:04:57.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.373 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703669 s, 149 MB/s 2024-04-25T00:04:57.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.382 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718309 s, 146 MB/s 2024-04-25T00:04:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715555 s, 147 MB/s 2024-04-25T00:04:57.399 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.399 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.399 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701996 s, 149 MB/s 2024-04-25T00:04:57.408 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.408 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.409 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708878 s, 148 MB/s 2024-04-25T00:04:57.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:57.415 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:57.415 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:57.415 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T00:04:57.415 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:57.416 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:57.416 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:57.416 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:57.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579822 s, 181 MB/s 2024-04-25T00:04:57.480 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.480 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055225 s, 190 MB/s 2024-04-25T00:04:57.489 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.489 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.489 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709926 s, 148 MB/s 2024-04-25T00:04:57.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.498 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665556 s, 158 MB/s 2024-04-25T00:04:57.510 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.510 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.510 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736926 s, 142 MB/s 2024-04-25T00:04:57.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.515 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.515 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587567 s, 178 MB/s 2024-04-25T00:04:57.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.524 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708307 s, 148 MB/s 2024-04-25T00:04:57.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.532 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675157 s, 155 MB/s 2024-04-25T00:04:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574437 s, 183 MB/s 2024-04-25T00:04:57.550 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.550 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.550 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567025 s, 185 MB/s 2024-04-25T00:04:57.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.557 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694968 s, 151 MB/s 2024-04-25T00:04:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066022 s, 159 MB/s 2024-04-25T00:04:57.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.575 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729817 s, 144 MB/s 2024-04-25T00:04:57.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.584 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075325 s, 139 MB/s 2024-04-25T00:04:57.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.593 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069554 s, 151 MB/s 2024-04-25T00:04:57.601 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.601 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.601 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701487 s, 149 MB/s 2024-04-25T00:04:57.609 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006635 s, 158 MB/s 2024-04-25T00:04:57.618 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.619 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.619 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694459 s, 151 MB/s 2024-04-25T00:04:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652321 s, 161 MB/s 2024-04-25T00:04:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619888 s, 169 MB/s 2024-04-25T00:04:57.643 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.643 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.643 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563425 s, 186 MB/s 2024-04-25T00:04:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589069 s, 178 MB/s 2024-04-25T00:04:57.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.660 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715601 s, 147 MB/s 2024-04-25T00:04:57.668 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647383 s, 162 MB/s 2024-04-25T00:04:57.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.677 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683108 s, 154 MB/s 2024-04-25T00:04:57.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.685 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065014 s, 161 MB/s 2024-04-25T00:04:57.694 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.694 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.694 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070828 s, 148 MB/s 2024-04-25T00:04:57.703 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.703 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.704 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750205 s, 140 MB/s 2024-04-25T00:04:57.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.714 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577795 s, 181 MB/s 2024-04-25T00:04:57.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.719 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689716 s, 152 MB/s 2024-04-25T00:04:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657924 s, 159 MB/s 2024-04-25T00:04:57.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.735 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577619 s, 182 MB/s 2024-04-25T00:04:57.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:57.741 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:57.741 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:57.742 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T00:04:57.742 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:57.742 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:57.742 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:57.742 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:57.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:57.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.799 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066844 s, 157 MB/s 2024-04-25T00:04:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729673 s, 144 MB/s 2024-04-25T00:04:57.815 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.816 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.816 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591333 s, 177 MB/s 2024-04-25T00:04:57.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.823 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560262 s, 187 MB/s 2024-04-25T00:04:57.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721744 s, 145 MB/s 2024-04-25T00:04:57.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.840 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715753 s, 146 MB/s 2024-04-25T00:04:57.848 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.848 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.848 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654615 s, 160 MB/s 2024-04-25T00:04:57.857 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.857 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.857 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678149 s, 155 MB/s 2024-04-25T00:04:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059922 s, 175 MB/s 2024-04-25T00:04:57.872 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.872 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.872 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578119 s, 181 MB/s 2024-04-25T00:04:57.880 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.880 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.880 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661917 s, 158 MB/s 2024-04-25T00:04:57.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.887 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562303 s, 186 MB/s 2024-04-25T00:04:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639722 s, 164 MB/s 2024-04-25T00:04:57.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.903 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.903 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578779 s, 181 MB/s 2024-04-25T00:04:57.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.911 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.911 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063181 s, 166 MB/s 2024-04-25T00:04:57.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.918 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582347 s, 180 MB/s 2024-04-25T00:04:57.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.927 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666948 s, 157 MB/s 2024-04-25T00:04:57.935 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.935 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.935 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705904 s, 149 MB/s 2024-04-25T00:04:57.943 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.943 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.943 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066784 s, 157 MB/s 2024-04-25T00:04:57.951 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.951 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.951 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646485 s, 162 MB/s 2024-04-25T00:04:57.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.959 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552729 s, 190 MB/s 2024-04-25T00:04:57.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.966 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581689 s, 180 MB/s 2024-04-25T00:04:57.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.973 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568984 s, 184 MB/s 2024-04-25T00:04:57.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.982 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712615 s, 147 MB/s 2024-04-25T00:04:57.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005585 s, 188 MB/s 2024-04-25T00:04:57.998 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:57.998 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:57.998 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723558 s, 145 MB/s 2024-04-25T00:04:58.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656998 s, 160 MB/s 2024-04-25T00:04:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678658 s, 155 MB/s 2024-04-25T00:04:58.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.022 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670182 s, 156 MB/s 2024-04-25T00:04:58.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.030 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.030 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566134 s, 185 MB/s 2024-04-25T00:04:58.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.039 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702748 s, 149 MB/s 2024-04-25T00:04:58.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.047 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716304 s, 146 MB/s 2024-04-25T00:04:58.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:58.054 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:58.054 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:58.054 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T00:04:58.054 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:58.054 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:58.054 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:58.054 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:58.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:58.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.112 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674012 s, 156 MB/s 2024-04-25T00:04:58.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.123 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651371 s, 161 MB/s 2024-04-25T00:04:58.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.128 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679523 s, 154 MB/s 2024-04-25T00:04:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549915 s, 191 MB/s 2024-04-25T00:04:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.144 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667974 s, 157 MB/s 2024-04-25T00:04:58.151 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.151 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.151 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055574 s, 189 MB/s 2024-04-25T00:04:58.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.158 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582245 s, 180 MB/s 2024-04-25T00:04:58.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.166 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641311 s, 164 MB/s 2024-04-25T00:04:58.174 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.175 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.175 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673471 s, 156 MB/s 2024-04-25T00:04:58.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.182 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574876 s, 182 MB/s 2024-04-25T00:04:58.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.191 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068179 s, 154 MB/s 2024-04-25T00:04:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.198 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562744 s, 186 MB/s 2024-04-25T00:04:58.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.207 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.207 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726456 s, 144 MB/s 2024-04-25T00:04:58.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.214 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550026 s, 191 MB/s 2024-04-25T00:04:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719083 s, 146 MB/s 2024-04-25T00:04:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812825 s, 129 MB/s 2024-04-25T00:04:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660588 s, 159 MB/s 2024-04-25T00:04:58.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.249 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068175 s, 154 MB/s 2024-04-25T00:04:58.256 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.256 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.256 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619651 s, 169 MB/s 2024-04-25T00:04:58.264 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.264 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.264 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645749 s, 162 MB/s 2024-04-25T00:04:58.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.272 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580147 s, 181 MB/s 2024-04-25T00:04:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548355 s, 191 MB/s 2024-04-25T00:04:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653141 s, 161 MB/s 2024-04-25T00:04:58.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.295 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569714 s, 184 MB/s 2024-04-25T00:04:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673709 s, 156 MB/s 2024-04-25T00:04:58.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.311 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649137 s, 162 MB/s 2024-04-25T00:04:58.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.320 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723336 s, 145 MB/s 2024-04-25T00:04:58.328 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.328 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.328 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680798 s, 154 MB/s 2024-04-25T00:04:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740553 s, 142 MB/s 2024-04-25T00:04:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672947 s, 156 MB/s 2024-04-25T00:04:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575127 s, 182 MB/s 2024-04-25T00:04:58.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.362 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074389 s, 141 MB/s 2024-04-25T00:04:58.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:58.369 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:58.369 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:58.369 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T00:04:58.369 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:58.369 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:58.369 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:58.369 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:58.427 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.427 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.427 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668349 s, 157 MB/s 2024-04-25T00:04:58.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757854 s, 138 MB/s 2024-04-25T00:04:58.444 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.445 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.445 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751776 s, 139 MB/s 2024-04-25T00:04:58.453 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.453 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.453 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683092 s, 154 MB/s 2024-04-25T00:04:58.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.461 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712722 s, 147 MB/s 2024-04-25T00:04:58.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.469 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.469 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562157 s, 187 MB/s 2024-04-25T00:04:58.477 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.477 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.477 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689738 s, 152 MB/s 2024-04-25T00:04:58.485 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.486 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.486 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632471 s, 166 MB/s 2024-04-25T00:04:58.494 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.494 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.494 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659226 s, 159 MB/s 2024-04-25T00:04:58.501 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.501 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.502 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660692 s, 159 MB/s 2024-04-25T00:04:58.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.511 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728969 s, 144 MB/s 2024-04-25T00:04:58.519 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.519 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.519 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655644 s, 160 MB/s 2024-04-25T00:04:58.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644429 s, 163 MB/s 2024-04-25T00:04:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066017 s, 159 MB/s 2024-04-25T00:04:58.542 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.543 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626944 s, 167 MB/s 2024-04-25T00:04:58.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.551 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670707 s, 156 MB/s 2024-04-25T00:04:58.559 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.559 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.559 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649275 s, 161 MB/s 2024-04-25T00:04:58.566 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.566 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.567 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572717 s, 183 MB/s 2024-04-25T00:04:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687785 s, 152 MB/s 2024-04-25T00:04:58.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.582 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571578 s, 183 MB/s 2024-04-25T00:04:58.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.591 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751698 s, 139 MB/s 2024-04-25T00:04:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562717 s, 186 MB/s 2024-04-25T00:04:58.607 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.607 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.607 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698704 s, 150 MB/s 2024-04-25T00:04:58.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682495 s, 154 MB/s 2024-04-25T00:04:58.624 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.624 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.624 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731003 s, 143 MB/s 2024-04-25T00:04:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718359 s, 146 MB/s 2024-04-25T00:04:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664618 s, 158 MB/s 2024-04-25T00:04:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654104 s, 160 MB/s 2024-04-25T00:04:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668618 s, 157 MB/s 2024-04-25T00:04:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636756 s, 165 MB/s 2024-04-25T00:04:58.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.673 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646676 s, 162 MB/s 2024-04-25T00:04:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070063 s, 150 MB/s 2024-04-25T00:04:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:58.689 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:58.689 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:58.689 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T00:04:58.689 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:58.689 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:58.689 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:58.690 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:58.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.750 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719423 s, 146 MB/s 2024-04-25T00:04:58.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.757 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556067 s, 189 MB/s 2024-04-25T00:04:58.765 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.765 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.765 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663056 s, 158 MB/s 2024-04-25T00:04:58.773 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.774 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.774 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646832 s, 162 MB/s 2024-04-25T00:04:58.781 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.781 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.781 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571607 s, 183 MB/s 2024-04-25T00:04:58.789 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.789 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.789 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067046 s, 156 MB/s 2024-04-25T00:04:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644466 s, 163 MB/s 2024-04-25T00:04:58.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.805 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643671 s, 163 MB/s 2024-04-25T00:04:58.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.813 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663336 s, 158 MB/s 2024-04-25T00:04:58.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.822 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737951 s, 142 MB/s 2024-04-25T00:04:58.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.830 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636387 s, 165 MB/s 2024-04-25T00:04:58.837 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.837 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.837 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564774 s, 186 MB/s 2024-04-25T00:04:58.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.844 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557124 s, 188 MB/s 2024-04-25T00:04:58.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.854 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658847 s, 159 MB/s 2024-04-25T00:04:58.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.861 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683136 s, 153 MB/s 2024-04-25T00:04:58.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685049 s, 153 MB/s 2024-04-25T00:04:58.878 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.878 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670251 s, 156 MB/s 2024-04-25T00:04:58.888 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.888 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.888 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684889 s, 153 MB/s 2024-04-25T00:04:58.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.896 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713871 s, 147 MB/s 2024-04-25T00:04:58.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.906 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765698 s, 137 MB/s 2024-04-25T00:04:58.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.914 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649034 s, 162 MB/s 2024-04-25T00:04:58.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.923 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667086 s, 157 MB/s 2024-04-25T00:04:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569262 s, 184 MB/s 2024-04-25T00:04:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655688 s, 160 MB/s 2024-04-25T00:04:58.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.946 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571454 s, 183 MB/s 2024-04-25T00:04:58.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.954 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555974 s, 189 MB/s 2024-04-25T00:04:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736872 s, 142 MB/s 2024-04-25T00:04:58.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.972 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067197 s, 156 MB/s 2024-04-25T00:04:58.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.980 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644583 s, 163 MB/s 2024-04-25T00:04:58.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652632 s, 161 MB/s 2024-04-25T00:04:58.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:58.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:58.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679415 s, 154 MB/s 2024-04-25T00:04:59.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647443 s, 162 MB/s 2024-04-25T00:04:59.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:59.012 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:59.013 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:59.013 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T00:04:59.013 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:59.013 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:59.013 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:59.013 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:59.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647992 s, 162 MB/s 2024-04-25T00:04:59.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.080 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670128 s, 156 MB/s 2024-04-25T00:04:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655209 s, 160 MB/s 2024-04-25T00:04:59.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.095 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587113 s, 179 MB/s 2024-04-25T00:04:59.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.104 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.104 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669342 s, 157 MB/s 2024-04-25T00:04:59.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.111 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565559 s, 185 MB/s 2024-04-25T00:04:59.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.120 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735764 s, 143 MB/s 2024-04-25T00:04:59.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.129 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736536 s, 142 MB/s 2024-04-25T00:04:59.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.138 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751584 s, 140 MB/s 2024-04-25T00:04:59.148 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.148 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.148 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747211 s, 140 MB/s 2024-04-25T00:04:59.155 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.156 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674305 s, 156 MB/s 2024-04-25T00:04:59.163 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.164 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065588 s, 160 MB/s 2024-04-25T00:04:59.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.172 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641522 s, 163 MB/s 2024-04-25T00:04:59.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.179 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644766 s, 163 MB/s 2024-04-25T00:04:59.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.189 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739246 s, 142 MB/s 2024-04-25T00:04:59.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.196 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577467 s, 182 MB/s 2024-04-25T00:04:59.205 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.205 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.205 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704368 s, 149 MB/s 2024-04-25T00:04:59.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.213 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686619 s, 153 MB/s 2024-04-25T00:04:59.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600033 s, 175 MB/s 2024-04-25T00:04:59.229 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.229 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.229 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662871 s, 158 MB/s 2024-04-25T00:04:59.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.237 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682474 s, 154 MB/s 2024-04-25T00:04:59.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.245 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.245 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585569 s, 179 MB/s 2024-04-25T00:04:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070385 s, 149 MB/s 2024-04-25T00:04:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654405 s, 160 MB/s 2024-04-25T00:04:59.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705009 s, 149 MB/s 2024-04-25T00:04:59.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632659 s, 166 MB/s 2024-04-25T00:04:59.286 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.286 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.286 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679674 s, 154 MB/s 2024-04-25T00:04:59.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.295 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733415 s, 143 MB/s 2024-04-25T00:04:59.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552198 s, 190 MB/s 2024-04-25T00:04:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.311 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719736 s, 146 MB/s 2024-04-25T00:04:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659043 s, 159 MB/s 2024-04-25T00:04:59.328 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.328 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.328 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680594 s, 154 MB/s 2024-04-25T00:04:59.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:59.335 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:59.335 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:59.335 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T00:04:59.335 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:59.335 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:59.335 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:59.335 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:59.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:59.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.394 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667146 s, 157 MB/s 2024-04-25T00:04:59.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.402 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647569 s, 162 MB/s 2024-04-25T00:04:59.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.409 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576858 s, 182 MB/s 2024-04-25T00:04:59.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.419 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.419 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715626 s, 147 MB/s 2024-04-25T00:04:59.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.426 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648147 s, 162 MB/s 2024-04-25T00:04:59.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.435 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681861 s, 154 MB/s 2024-04-25T00:04:59.441 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.441 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.441 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546768 s, 192 MB/s 2024-04-25T00:04:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643868 s, 163 MB/s 2024-04-25T00:04:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726251 s, 144 MB/s 2024-04-25T00:04:59.465 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.465 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.465 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058252 s, 180 MB/s 2024-04-25T00:04:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661991 s, 158 MB/s 2024-04-25T00:04:59.482 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.482 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.482 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722371 s, 145 MB/s 2024-04-25T00:04:59.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.491 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648792 s, 162 MB/s 2024-04-25T00:04:59.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.499 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706215 s, 148 MB/s 2024-04-25T00:04:59.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.508 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699777 s, 150 MB/s 2024-04-25T00:04:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573305 s, 183 MB/s 2024-04-25T00:04:59.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.524 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698427 s, 150 MB/s 2024-04-25T00:04:59.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.532 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654545 s, 160 MB/s 2024-04-25T00:04:59.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.540 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648292 s, 162 MB/s 2024-04-25T00:04:59.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.549 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721766 s, 145 MB/s 2024-04-25T00:04:59.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.557 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.557 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649621 s, 161 MB/s 2024-04-25T00:04:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662974 s, 158 MB/s 2024-04-25T00:04:59.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587534 s, 178 MB/s 2024-04-25T00:04:59.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.581 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697108 s, 150 MB/s 2024-04-25T00:04:59.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.589 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692137 s, 151 MB/s 2024-04-25T00:04:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652391 s, 161 MB/s 2024-04-25T00:04:59.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638398 s, 164 MB/s 2024-04-25T00:04:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708888 s, 148 MB/s 2024-04-25T00:04:59.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.621 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579256 s, 181 MB/s 2024-04-25T00:04:59.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062788 s, 167 MB/s 2024-04-25T00:04:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670611 s, 156 MB/s 2024-04-25T00:04:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648891 s, 162 MB/s 2024-04-25T00:04:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:59.652 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:59.652 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:59.652 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T00:04:59.653 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:59.653 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:59.653 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:59.653 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:04:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:04:59.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.712 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778016 s, 135 MB/s 2024-04-25T00:04:59.720 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.720 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.720 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679838 s, 154 MB/s 2024-04-25T00:04:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590013 s, 178 MB/s 2024-04-25T00:04:59.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.735 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577443 s, 182 MB/s 2024-04-25T00:04:59.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.743 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682234 s, 154 MB/s 2024-04-25T00:04:59.751 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.751 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.751 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646944 s, 162 MB/s 2024-04-25T00:04:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682392 s, 154 MB/s 2024-04-25T00:04:59.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.768 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644944 s, 163 MB/s 2024-04-25T00:04:59.775 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.776 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650106 s, 161 MB/s 2024-04-25T00:04:59.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728032 s, 144 MB/s 2024-04-25T00:04:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.793 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636527 s, 165 MB/s 2024-04-25T00:04:59.800 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.800 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.800 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574108 s, 183 MB/s 2024-04-25T00:04:59.808 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.808 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.808 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067027 s, 156 MB/s 2024-04-25T00:04:59.816 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.816 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.816 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646439 s, 162 MB/s 2024-04-25T00:04:59.825 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.825 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.825 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717605 s, 146 MB/s 2024-04-25T00:04:59.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.834 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738217 s, 142 MB/s 2024-04-25T00:04:59.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.843 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675916 s, 155 MB/s 2024-04-25T00:04:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058283 s, 180 MB/s 2024-04-25T00:04:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660441 s, 159 MB/s 2024-04-25T00:04:59.865 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.865 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.865 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573328 s, 183 MB/s 2024-04-25T00:04:59.874 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.874 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.875 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764849 s, 137 MB/s 2024-04-25T00:04:59.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.883 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653403 s, 160 MB/s 2024-04-25T00:04:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661781 s, 158 MB/s 2024-04-25T00:04:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582702 s, 180 MB/s 2024-04-25T00:04:59.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.906 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632696 s, 166 MB/s 2024-04-25T00:04:59.913 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.913 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.913 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583052 s, 180 MB/s 2024-04-25T00:04:59.920 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.920 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.920 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556625 s, 188 MB/s 2024-04-25T00:04:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702128 s, 149 MB/s 2024-04-25T00:04:59.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.937 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637165 s, 165 MB/s 2024-04-25T00:04:59.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.946 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713417 s, 147 MB/s 2024-04-25T00:04:59.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.953 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576162 s, 182 MB/s 2024-04-25T00:04:59.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:04:59.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:04:59.962 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673115 s, 156 MB/s 2024-04-25T00:04:59.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:04:59.969 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:04:59.969 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:04:59.969 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T00:04:59.969 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:04:59.969 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:04:59.970 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:04:59.970 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:00.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668833 s, 157 MB/s 2024-04-25T00:05:00.035 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.035 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.035 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671538 s, 156 MB/s 2024-04-25T00:05:00.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.044 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733502 s, 143 MB/s 2024-04-25T00:05:00.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.052 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660855 s, 159 MB/s 2024-04-25T00:05:00.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.060 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065275 s, 161 MB/s 2024-04-25T00:05:00.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.068 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056321 s, 186 MB/s 2024-04-25T00:05:00.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.075 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581145 s, 180 MB/s 2024-04-25T00:05:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654416 s, 160 MB/s 2024-04-25T00:05:00.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.090 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564168 s, 186 MB/s 2024-04-25T00:05:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.099 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067734 s, 155 MB/s 2024-04-25T00:05:00.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.107 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709797 s, 148 MB/s 2024-04-25T00:05:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765886 s, 137 MB/s 2024-04-25T00:05:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663493 s, 158 MB/s 2024-04-25T00:05:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550032 s, 191 MB/s 2024-04-25T00:05:00.140 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.140 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.140 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654144 s, 160 MB/s 2024-04-25T00:05:00.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058257 s, 180 MB/s 2024-04-25T00:05:00.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.157 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067224 s, 156 MB/s 2024-04-25T00:05:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676832 s, 155 MB/s 2024-04-25T00:05:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651492 s, 161 MB/s 2024-04-25T00:05:00.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.180 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568385 s, 184 MB/s 2024-04-25T00:05:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.189 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065139 s, 161 MB/s 2024-04-25T00:05:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645632 s, 162 MB/s 2024-04-25T00:05:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569351 s, 184 MB/s 2024-04-25T00:05:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556856 s, 188 MB/s 2024-04-25T00:05:00.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.219 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660076 s, 159 MB/s 2024-04-25T00:05:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705831 s, 149 MB/s 2024-04-25T00:05:00.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637518 s, 164 MB/s 2024-04-25T00:05:00.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.244 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653911 s, 160 MB/s 2024-04-25T00:05:00.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.252 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657869 s, 159 MB/s 2024-04-25T00:05:00.259 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.260 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628242 s, 167 MB/s 2024-04-25T00:05:00.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.268 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.268 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686813 s, 153 MB/s 2024-04-25T00:05:00.277 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.277 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.277 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740549 s, 142 MB/s 2024-04-25T00:05:00.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:00.284 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:00.284 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:00.284 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T00:05:00.284 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:00.284 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:00.284 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:00.284 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:00.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:00.341 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.341 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609401 s, 172 MB/s 2024-04-25T00:05:00.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689331 s, 152 MB/s 2024-04-25T00:05:00.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.358 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673157 s, 156 MB/s 2024-04-25T00:05:00.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708801 s, 148 MB/s 2024-04-25T00:05:00.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.375 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654816 s, 160 MB/s 2024-04-25T00:05:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653125 s, 161 MB/s 2024-04-25T00:05:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.392 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719409 s, 146 MB/s 2024-04-25T00:05:00.400 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.400 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.400 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669296 s, 157 MB/s 2024-04-25T00:05:00.407 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.407 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.407 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559464 s, 187 MB/s 2024-04-25T00:05:00.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.416 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070955 s, 148 MB/s 2024-04-25T00:05:00.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.424 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638068 s, 164 MB/s 2024-04-25T00:05:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618408 s, 170 MB/s 2024-04-25T00:05:00.440 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.440 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.441 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734603 s, 143 MB/s 2024-04-25T00:05:00.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.450 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691126 s, 152 MB/s 2024-04-25T00:05:00.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.456 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571207 s, 184 MB/s 2024-04-25T00:05:00.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.464 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056055 s, 187 MB/s 2024-04-25T00:05:00.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690349 s, 152 MB/s 2024-04-25T00:05:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761968 s, 138 MB/s 2024-04-25T00:05:00.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.488 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588777 s, 178 MB/s 2024-04-25T00:05:00.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.497 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662477 s, 158 MB/s 2024-04-25T00:05:00.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.504 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584182 s, 179 MB/s 2024-04-25T00:05:00.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.514 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709325 s, 148 MB/s 2024-04-25T00:05:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678231 s, 155 MB/s 2024-04-25T00:05:00.529 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.529 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.529 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642679 s, 163 MB/s 2024-04-25T00:05:00.537 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.537 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.537 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643757 s, 163 MB/s 2024-04-25T00:05:00.546 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.546 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.546 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679702 s, 154 MB/s 2024-04-25T00:05:00.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.553 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566844 s, 185 MB/s 2024-04-25T00:05:00.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.560 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583246 s, 180 MB/s 2024-04-25T00:05:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669911 s, 157 MB/s 2024-04-25T00:05:00.577 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.578 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.578 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641887 s, 163 MB/s 2024-04-25T00:05:00.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.584 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571366 s, 184 MB/s 2024-04-25T00:05:00.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.591 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583336 s, 180 MB/s 2024-04-25T00:05:00.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:00.598 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:00.598 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:00.598 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T00:05:00.598 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:00.598 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:00.598 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:00.598 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:00.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:00.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.658 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708022 s, 148 MB/s 2024-04-25T00:05:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.668 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.668 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764448 s, 137 MB/s 2024-04-25T00:05:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683546 s, 153 MB/s 2024-04-25T00:05:00.684 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.684 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.684 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664936 s, 158 MB/s 2024-04-25T00:05:00.694 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.695 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.695 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803611 s, 130 MB/s 2024-04-25T00:05:00.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.706 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728433 s, 144 MB/s 2024-04-25T00:05:00.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.714 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769827 s, 136 MB/s 2024-04-25T00:05:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604847 s, 173 MB/s 2024-04-25T00:05:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619746 s, 169 MB/s 2024-04-25T00:05:00.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.740 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787498 s, 133 MB/s 2024-04-25T00:05:00.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.749 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663006 s, 158 MB/s 2024-04-25T00:05:00.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.757 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071748 s, 146 MB/s 2024-04-25T00:05:00.766 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.766 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.766 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728768 s, 144 MB/s 2024-04-25T00:05:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643778 s, 163 MB/s 2024-04-25T00:05:00.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.782 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589861 s, 178 MB/s 2024-04-25T00:05:00.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.790 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657285 s, 160 MB/s 2024-04-25T00:05:00.798 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702745 s, 149 MB/s 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573995 s, 183 MB/s 2024-04-25T00:05:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566099 s, 185 MB/s 2024-04-25T00:05:00.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.822 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719838 s, 146 MB/s 2024-04-25T00:05:00.829 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.829 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.829 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569331 s, 184 MB/s 2024-04-25T00:05:00.837 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.837 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.837 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056556 s, 185 MB/s 2024-04-25T00:05:00.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.844 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055571 s, 189 MB/s 2024-04-25T00:05:00.852 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.852 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.852 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675555 s, 155 MB/s 2024-04-25T00:05:00.860 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.860 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.860 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659656 s, 159 MB/s 2024-04-25T00:05:00.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.868 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649649 s, 161 MB/s 2024-04-25T00:05:00.876 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670752 s, 156 MB/s 2024-04-25T00:05:00.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.885 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727461 s, 144 MB/s 2024-04-25T00:05:00.894 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.894 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.894 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659286 s, 159 MB/s 2024-04-25T00:05:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663013 s, 158 MB/s 2024-04-25T00:05:00.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.909 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550026 s, 191 MB/s 2024-04-25T00:05:00.917 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.917 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.917 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651758 s, 161 MB/s 2024-04-25T00:05:00.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:00.924 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:00.924 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:00.924 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T00:05:00.924 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:00.924 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:00.924 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:00.924 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:00.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:00.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.981 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565157 s, 186 MB/s 2024-04-25T00:05:00.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588427 s, 178 MB/s 2024-04-25T00:05:00.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:00.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:00.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690581 s, 152 MB/s 2024-04-25T00:05:01.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.005 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632125 s, 166 MB/s 2024-04-25T00:05:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.013 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642412 s, 163 MB/s 2024-04-25T00:05:01.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719846 s, 146 MB/s 2024-04-25T00:05:01.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.031 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800834 s, 131 MB/s 2024-04-25T00:05:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735732 s, 143 MB/s 2024-04-25T00:05:01.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.048 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638574 s, 164 MB/s 2024-04-25T00:05:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645121 s, 163 MB/s 2024-04-25T00:05:01.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.064 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067857 s, 155 MB/s 2024-04-25T00:05:01.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.071 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556012 s, 189 MB/s 2024-04-25T00:05:01.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.080 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653377 s, 160 MB/s 2024-04-25T00:05:01.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.088 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676678 s, 155 MB/s 2024-04-25T00:05:01.096 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653502 s, 160 MB/s 2024-04-25T00:05:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.104 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581841 s, 180 MB/s 2024-04-25T00:05:01.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643714 s, 163 MB/s 2024-04-25T00:05:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717339 s, 146 MB/s 2024-04-25T00:05:01.127 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.128 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560931 s, 187 MB/s 2024-04-25T00:05:01.135 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655364 s, 160 MB/s 2024-04-25T00:05:01.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.144 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067391 s, 156 MB/s 2024-04-25T00:05:01.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.153 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637934 s, 164 MB/s 2024-04-25T00:05:01.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.159 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578302 s, 181 MB/s 2024-04-25T00:05:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707986 s, 148 MB/s 2024-04-25T00:05:01.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.177 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724777 s, 145 MB/s 2024-04-25T00:05:01.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.184 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606934 s, 173 MB/s 2024-04-25T00:05:01.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.194 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664903 s, 158 MB/s 2024-04-25T00:05:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558294 s, 188 MB/s 2024-04-25T00:05:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729177 s, 144 MB/s 2024-04-25T00:05:01.216 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.216 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.216 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585342 s, 179 MB/s 2024-04-25T00:05:01.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.224 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058292 s, 180 MB/s 2024-04-25T00:05:01.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.232 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662319 s, 158 MB/s 2024-04-25T00:05:01.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:01.239 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:01.239 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:01.239 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T00:05:01.239 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:01.239 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:01.239 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:01.239 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:01.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583404 s, 180 MB/s 2024-04-25T00:05:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579594 s, 181 MB/s 2024-04-25T00:05:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612995 s, 171 MB/s 2024-04-25T00:05:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563696 s, 186 MB/s 2024-04-25T00:05:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583841 s, 180 MB/s 2024-04-25T00:05:01.334 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.334 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.334 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716692 s, 146 MB/s 2024-04-25T00:05:01.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.343 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.343 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589507 s, 178 MB/s 2024-04-25T00:05:01.349 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.349 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.349 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00540542 s, 194 MB/s 2024-04-25T00:05:01.357 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.357 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.357 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663037 s, 158 MB/s 2024-04-25T00:05:01.365 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.366 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680443 s, 154 MB/s 2024-04-25T00:05:01.376 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.376 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.376 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863327 s, 121 MB/s 2024-04-25T00:05:01.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.385 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656724 s, 160 MB/s 2024-04-25T00:05:01.393 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748544 s, 140 MB/s 2024-04-25T00:05:01.400 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.400 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565563 s, 185 MB/s 2024-04-25T00:05:01.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.409 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652043 s, 161 MB/s 2024-04-25T00:05:01.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.418 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715581 s, 147 MB/s 2024-04-25T00:05:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745112 s, 141 MB/s 2024-04-25T00:05:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694435 s, 151 MB/s 2024-04-25T00:05:01.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.443 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562577 s, 186 MB/s 2024-04-25T00:05:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708768 s, 148 MB/s 2024-04-25T00:05:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841763 s, 125 MB/s 2024-04-25T00:05:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566799 s, 185 MB/s 2024-04-25T00:05:01.476 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.476 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.477 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636888 s, 165 MB/s 2024-04-25T00:05:01.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.484 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635119 s, 165 MB/s 2024-04-25T00:05:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710329 s, 148 MB/s 2024-04-25T00:05:01.501 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.502 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.502 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666895 s, 157 MB/s 2024-04-25T00:05:01.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.508 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553675 s, 189 MB/s 2024-04-25T00:05:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.517 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748678 s, 140 MB/s 2024-04-25T00:05:01.525 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.525 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.526 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692666 s, 151 MB/s 2024-04-25T00:05:01.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.534 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073426 s, 143 MB/s 2024-04-25T00:05:01.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.543 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067186 s, 156 MB/s 2024-04-25T00:05:01.550 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.550 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.551 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657417 s, 159 MB/s 2024-04-25T00:05:01.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:01.557 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:01.557 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:01.557 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T00:05:01.557 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:01.557 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:01.558 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:01.558 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661253 s, 159 MB/s 2024-04-25T00:05:01.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.622 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549346 s, 191 MB/s 2024-04-25T00:05:01.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.631 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719733 s, 146 MB/s 2024-04-25T00:05:01.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.639 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666317 s, 157 MB/s 2024-04-25T00:05:01.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.647 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657715 s, 159 MB/s 2024-04-25T00:05:01.654 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.655 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.655 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561688 s, 187 MB/s 2024-04-25T00:05:01.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.664 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669802 s, 157 MB/s 2024-04-25T00:05:01.671 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.671 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.671 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651735 s, 161 MB/s 2024-04-25T00:05:01.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.679 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578639 s, 181 MB/s 2024-04-25T00:05:01.686 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.687 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667651 s, 157 MB/s 2024-04-25T00:05:01.694 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.694 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.695 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658167 s, 159 MB/s 2024-04-25T00:05:01.703 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.703 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.703 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660475 s, 159 MB/s 2024-04-25T00:05:01.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.712 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732074 s, 143 MB/s 2024-04-25T00:05:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712995 s, 147 MB/s 2024-04-25T00:05:01.728 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.728 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.728 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627921 s, 167 MB/s 2024-04-25T00:05:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749816 s, 140 MB/s 2024-04-25T00:05:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057252 s, 183 MB/s 2024-04-25T00:05:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558279 s, 188 MB/s 2024-04-25T00:05:01.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.761 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644614 s, 163 MB/s 2024-04-25T00:05:01.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.768 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.768 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055997 s, 187 MB/s 2024-04-25T00:05:01.775 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.776 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567358 s, 185 MB/s 2024-04-25T00:05:01.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.784 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656025 s, 160 MB/s 2024-04-25T00:05:01.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781899 s, 134 MB/s 2024-04-25T00:05:01.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.803 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675254 s, 155 MB/s 2024-04-25T00:05:01.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565547 s, 185 MB/s 2024-04-25T00:05:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072804 s, 144 MB/s 2024-04-25T00:05:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655799 s, 160 MB/s 2024-04-25T00:05:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559029 s, 188 MB/s 2024-04-25T00:05:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732225 s, 143 MB/s 2024-04-25T00:05:01.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668373 s, 157 MB/s 2024-04-25T00:05:01.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.861 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566164 s, 185 MB/s 2024-04-25T00:05:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666219 s, 157 MB/s 2024-04-25T00:05:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:01.876 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:01.876 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:01.876 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T00:05:01.876 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:01.877 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:01.877 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:01.877 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:01.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:01.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.934 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578763 s, 181 MB/s 2024-04-25T00:05:01.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.942 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.942 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557952 s, 188 MB/s 2024-04-25T00:05:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.951 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066291 s, 158 MB/s 2024-04-25T00:05:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571338 s, 184 MB/s 2024-04-25T00:05:01.967 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.967 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.967 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684287 s, 153 MB/s 2024-04-25T00:05:01.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.974 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553415 s, 189 MB/s 2024-04-25T00:05:01.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.983 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666937 s, 157 MB/s 2024-04-25T00:05:01.990 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.990 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.990 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558013 s, 188 MB/s 2024-04-25T00:05:01.999 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:01.999 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:01.999 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638516 s, 164 MB/s 2024-04-25T00:05:02.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570665 s, 184 MB/s 2024-04-25T00:05:02.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.015 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.015 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655283 s, 160 MB/s 2024-04-25T00:05:02.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.023 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560339 s, 187 MB/s 2024-04-25T00:05:02.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.031 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064317 s, 163 MB/s 2024-04-25T00:05:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646068 s, 162 MB/s 2024-04-25T00:05:02.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.048 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650389 s, 161 MB/s 2024-04-25T00:05:02.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.056 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066143 s, 159 MB/s 2024-04-25T00:05:02.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.065 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.065 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570811 s, 184 MB/s 2024-04-25T00:05:02.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.074 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794741 s, 132 MB/s 2024-04-25T00:05:02.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.083 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687979 s, 152 MB/s 2024-04-25T00:05:02.091 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.091 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.091 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572576 s, 183 MB/s 2024-04-25T00:05:02.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.099 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659716 s, 159 MB/s 2024-04-25T00:05:02.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.109 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749637 s, 140 MB/s 2024-04-25T00:05:02.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.118 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660994 s, 159 MB/s 2024-04-25T00:05:02.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646987 s, 162 MB/s 2024-04-25T00:05:02.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.133 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563425 s, 186 MB/s 2024-04-25T00:05:02.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.143 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716039 s, 146 MB/s 2024-04-25T00:05:02.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551357 s, 190 MB/s 2024-04-25T00:05:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075582 s, 139 MB/s 2024-04-25T00:05:02.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.168 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650855 s, 161 MB/s 2024-04-25T00:05:02.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.178 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732647 s, 143 MB/s 2024-04-25T00:05:02.185 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.185 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.185 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574501 s, 183 MB/s 2024-04-25T00:05:02.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.194 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660608 s, 159 MB/s 2024-04-25T00:05:02.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:02.200 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:02.200 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:02.200 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T00:05:02.201 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:02.201 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:02.201 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:02.201 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:02.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:02.259 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.259 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.259 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660713 s, 159 MB/s 2024-04-25T00:05:02.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.267 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624197 s, 168 MB/s 2024-04-25T00:05:02.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.276 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654096 s, 160 MB/s 2024-04-25T00:05:02.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.285 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719646 s, 146 MB/s 2024-04-25T00:05:02.293 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.293 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.293 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602512 s, 174 MB/s 2024-04-25T00:05:02.301 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.302 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630973 s, 166 MB/s 2024-04-25T00:05:02.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.310 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068726 s, 153 MB/s 2024-04-25T00:05:02.319 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.319 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.319 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746958 s, 140 MB/s 2024-04-25T00:05:02.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.327 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638121 s, 164 MB/s 2024-04-25T00:05:02.336 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.336 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.336 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784049 s, 134 MB/s 2024-04-25T00:05:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.345 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070112 s, 150 MB/s 2024-04-25T00:05:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670002 s, 157 MB/s 2024-04-25T00:05:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645442 s, 162 MB/s 2024-04-25T00:05:02.368 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.369 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647429 s, 162 MB/s 2024-04-25T00:05:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737303 s, 142 MB/s 2024-04-25T00:05:02.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.386 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664916 s, 158 MB/s 2024-04-25T00:05:02.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.394 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669248 s, 157 MB/s 2024-04-25T00:05:02.402 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.402 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.402 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646839 s, 162 MB/s 2024-04-25T00:05:02.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.411 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702847 s, 149 MB/s 2024-04-25T00:05:02.419 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.419 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.419 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707547 s, 148 MB/s 2024-04-25T00:05:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.427 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577244 s, 182 MB/s 2024-04-25T00:05:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.435 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673051 s, 156 MB/s 2024-04-25T00:05:02.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.442 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599443 s, 175 MB/s 2024-04-25T00:05:02.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072717 s, 144 MB/s 2024-04-25T00:05:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072885 s, 144 MB/s 2024-04-25T00:05:02.468 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.468 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.468 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667768 s, 157 MB/s 2024-04-25T00:05:02.477 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.477 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.477 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714341 s, 147 MB/s 2024-04-25T00:05:02.485 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.485 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.485 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716947 s, 146 MB/s 2024-04-25T00:05:02.494 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.494 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.495 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574038 s, 183 MB/s 2024-04-25T00:05:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.501 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065908 s, 159 MB/s 2024-04-25T00:05:02.509 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.509 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.509 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657742 s, 159 MB/s 2024-04-25T00:05:02.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.517 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577658 s, 182 MB/s 2024-04-25T00:05:02.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:02.523 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:02.524 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:02.524 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T00:05:02.524 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:02.524 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:02.524 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:02.524 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:02.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:02.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.582 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666967 s, 157 MB/s 2024-04-25T00:05:02.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.589 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577918 s, 181 MB/s 2024-04-25T00:05:02.596 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.596 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.596 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574927 s, 182 MB/s 2024-04-25T00:05:02.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.606 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737976 s, 142 MB/s 2024-04-25T00:05:02.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.613 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057953 s, 181 MB/s 2024-04-25T00:05:02.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.621 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065528 s, 160 MB/s 2024-04-25T00:05:02.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.630 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.630 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636038 s, 165 MB/s 2024-04-25T00:05:02.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.637 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627767 s, 167 MB/s 2024-04-25T00:05:02.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.644 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560726 s, 187 MB/s 2024-04-25T00:05:02.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635183 s, 165 MB/s 2024-04-25T00:05:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733435 s, 143 MB/s 2024-04-25T00:05:02.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580714 s, 181 MB/s 2024-04-25T00:05:02.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.679 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693444 s, 151 MB/s 2024-04-25T00:05:02.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.692 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113221 s, 92.6 MB/s 2024-04-25T00:05:02.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649384 s, 161 MB/s 2024-04-25T00:05:02.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.709 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.709 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703763 s, 149 MB/s 2024-04-25T00:05:02.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.716 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561272 s, 187 MB/s 2024-04-25T00:05:02.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.724 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658067 s, 159 MB/s 2024-04-25T00:05:02.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.733 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695534 s, 151 MB/s 2024-04-25T00:05:02.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.742 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.742 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713804 s, 147 MB/s 2024-04-25T00:05:02.749 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.750 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675194 s, 155 MB/s 2024-04-25T00:05:02.756 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.757 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549856 s, 191 MB/s 2024-04-25T00:05:02.765 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.765 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.765 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657435 s, 159 MB/s 2024-04-25T00:05:02.772 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.772 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.772 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568098 s, 185 MB/s 2024-04-25T00:05:02.780 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.780 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.780 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624681 s, 168 MB/s 2024-04-25T00:05:02.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.789 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681926 s, 154 MB/s 2024-04-25T00:05:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549399 s, 191 MB/s 2024-04-25T00:05:02.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.805 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682256 s, 154 MB/s 2024-04-25T00:05:02.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.813 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674827 s, 155 MB/s 2024-04-25T00:05:02.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.822 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657719 s, 159 MB/s 2024-04-25T00:05:02.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.834 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740677 s, 142 MB/s 2024-04-25T00:05:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666547 s, 157 MB/s 2024-04-25T00:05:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:02.846 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:02.846 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:02.846 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T00:05:02.846 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:02.846 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:02.847 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:02.847 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:02.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:02.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.906 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716524 s, 146 MB/s 2024-04-25T00:05:02.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.915 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703723 s, 149 MB/s 2024-04-25T00:05:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652228 s, 161 MB/s 2024-04-25T00:05:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.931 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567228 s, 185 MB/s 2024-04-25T00:05:02.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.940 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718963 s, 146 MB/s 2024-04-25T00:05:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.950 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817466 s, 128 MB/s 2024-04-25T00:05:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644071 s, 163 MB/s 2024-04-25T00:05:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.966 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627603 s, 167 MB/s 2024-04-25T00:05:02.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.973 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560416 s, 187 MB/s 2024-04-25T00:05:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656565 s, 160 MB/s 2024-04-25T00:05:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063799 s, 164 MB/s 2024-04-25T00:05:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562773 s, 186 MB/s 2024-04-25T00:05:03.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.004 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651147 s, 161 MB/s 2024-04-25T00:05:03.011 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.011 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.011 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569387 s, 184 MB/s 2024-04-25T00:05:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562719 s, 186 MB/s 2024-04-25T00:05:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640732 s, 164 MB/s 2024-04-25T00:05:03.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.036 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731225 s, 143 MB/s 2024-04-25T00:05:03.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070897 s, 148 MB/s 2024-04-25T00:05:03.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.053 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673004 s, 156 MB/s 2024-04-25T00:05:03.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.060 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551438 s, 190 MB/s 2024-04-25T00:05:03.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.069 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.069 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698555 s, 150 MB/s 2024-04-25T00:05:03.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.076 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572246 s, 183 MB/s 2024-04-25T00:05:03.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.085 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715624 s, 147 MB/s 2024-04-25T00:05:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730598 s, 144 MB/s 2024-04-25T00:05:03.102 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.102 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.102 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703975 s, 149 MB/s 2024-04-25T00:05:03.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.110 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631181 s, 166 MB/s 2024-04-25T00:05:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635751 s, 165 MB/s 2024-04-25T00:05:03.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633341 s, 166 MB/s 2024-04-25T00:05:03.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.134 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643128 s, 163 MB/s 2024-04-25T00:05:03.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.142 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652704 s, 161 MB/s 2024-04-25T00:05:03.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635114 s, 165 MB/s 2024-04-25T00:05:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566218 s, 185 MB/s 2024-04-25T00:05:03.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:03.165 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:03.165 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:03.165 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T00:05:03.165 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:03.165 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:03.165 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:03.165 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:03.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715496 s, 147 MB/s 2024-04-25T00:05:03.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.231 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580577 s, 181 MB/s 2024-04-25T00:05:03.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.239 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059046 s, 178 MB/s 2024-04-25T00:05:03.246 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.246 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.247 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604065 s, 174 MB/s 2024-04-25T00:05:03.255 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.255 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.255 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666221 s, 157 MB/s 2024-04-25T00:05:03.263 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.263 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.263 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673341 s, 156 MB/s 2024-04-25T00:05:03.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.271 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650414 s, 161 MB/s 2024-04-25T00:05:03.280 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.280 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.280 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757891 s, 138 MB/s 2024-04-25T00:05:03.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.289 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070663 s, 148 MB/s 2024-04-25T00:05:03.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.297 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629169 s, 167 MB/s 2024-04-25T00:05:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668024 s, 157 MB/s 2024-04-25T00:05:03.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636093 s, 165 MB/s 2024-04-25T00:05:03.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641859 s, 163 MB/s 2024-04-25T00:05:03.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.331 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.331 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768248 s, 136 MB/s 2024-04-25T00:05:03.338 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.338 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.338 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626271 s, 167 MB/s 2024-04-25T00:05:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684058 s, 153 MB/s 2024-04-25T00:05:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570173 s, 184 MB/s 2024-04-25T00:05:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703136 s, 149 MB/s 2024-04-25T00:05:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651203 s, 161 MB/s 2024-04-25T00:05:03.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576695 s, 182 MB/s 2024-04-25T00:05:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.387 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066756 s, 157 MB/s 2024-04-25T00:05:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673756 s, 156 MB/s 2024-04-25T00:05:03.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.404 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773171 s, 136 MB/s 2024-04-25T00:05:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677855 s, 155 MB/s 2024-04-25T00:05:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650143 s, 161 MB/s 2024-04-25T00:05:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.429 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644201 s, 163 MB/s 2024-04-25T00:05:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735764 s, 143 MB/s 2024-04-25T00:05:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064387 s, 163 MB/s 2024-04-25T00:05:03.455 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.455 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.455 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760436 s, 138 MB/s 2024-04-25T00:05:03.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.463 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681987 s, 154 MB/s 2024-04-25T00:05:03.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007418 s, 141 MB/s 2024-04-25T00:05:03.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552723 s, 190 MB/s 2024-04-25T00:05:03.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:03.487 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:03.487 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:03.487 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T00:05:03.487 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:03.487 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:03.487 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:03.488 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702773 s, 149 MB/s 2024-04-25T00:05:03.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.555 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628224 s, 167 MB/s 2024-04-25T00:05:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.564 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686704 s, 153 MB/s 2024-04-25T00:05:03.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641062 s, 164 MB/s 2024-04-25T00:05:03.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.583 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886061 s, 118 MB/s 2024-04-25T00:05:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786906 s, 133 MB/s 2024-04-25T00:05:03.601 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.601 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.601 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702264 s, 149 MB/s 2024-04-25T00:05:03.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.610 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648017 s, 162 MB/s 2024-04-25T00:05:03.618 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.618 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.618 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652652 s, 161 MB/s 2024-04-25T00:05:03.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.627 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697626 s, 150 MB/s 2024-04-25T00:05:03.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686516 s, 153 MB/s 2024-04-25T00:05:03.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.646 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.646 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746584 s, 140 MB/s 2024-04-25T00:05:03.654 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.654 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.654 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063197 s, 166 MB/s 2024-04-25T00:05:03.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.661 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576341 s, 182 MB/s 2024-04-25T00:05:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066206 s, 158 MB/s 2024-04-25T00:05:03.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.677 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575548 s, 182 MB/s 2024-04-25T00:05:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555299 s, 189 MB/s 2024-04-25T00:05:03.693 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.693 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.694 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653953 s, 160 MB/s 2024-04-25T00:05:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701003 s, 150 MB/s 2024-04-25T00:05:03.711 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.711 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.711 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642175 s, 163 MB/s 2024-04-25T00:05:03.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.718 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584732 s, 179 MB/s 2024-04-25T00:05:03.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.728 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738991 s, 142 MB/s 2024-04-25T00:05:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701953 s, 149 MB/s 2024-04-25T00:05:03.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.744 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677855 s, 155 MB/s 2024-04-25T00:05:03.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.754 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075295 s, 139 MB/s 2024-04-25T00:05:03.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.761 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579913 s, 181 MB/s 2024-04-25T00:05:03.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.771 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796812 s, 132 MB/s 2024-04-25T00:05:03.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.780 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.780 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764823 s, 137 MB/s 2024-04-25T00:05:03.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.787 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056593 s, 185 MB/s 2024-04-25T00:05:03.796 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.796 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.796 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727109 s, 144 MB/s 2024-04-25T00:05:03.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.803 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056112 s, 187 MB/s 2024-04-25T00:05:03.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554039 s, 189 MB/s 2024-04-25T00:05:03.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:03.817 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:03.817 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:03.817 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T00:05:03.817 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:03.817 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:03.817 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:03.817 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:03.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.875 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678198 s, 155 MB/s 2024-04-25T00:05:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637928 s, 164 MB/s 2024-04-25T00:05:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737195 s, 142 MB/s 2024-04-25T00:05:03.899 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.899 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.899 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595908 s, 176 MB/s 2024-04-25T00:05:03.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.908 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.908 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699921 s, 150 MB/s 2024-04-25T00:05:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650175 s, 161 MB/s 2024-04-25T00:05:03.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.924 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064754 s, 162 MB/s 2024-04-25T00:05:03.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.931 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585053 s, 179 MB/s 2024-04-25T00:05:03.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.940 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649975 s, 161 MB/s 2024-04-25T00:05:03.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.949 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721526 s, 145 MB/s 2024-04-25T00:05:03.957 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.957 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.957 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642907 s, 163 MB/s 2024-04-25T00:05:03.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.965 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065669 s, 160 MB/s 2024-04-25T00:05:03.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.974 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665773 s, 157 MB/s 2024-04-25T00:05:03.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.983 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734657 s, 143 MB/s 2024-04-25T00:05:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670243 s, 156 MB/s 2024-04-25T00:05:04.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.001 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666162 s, 157 MB/s 2024-04-25T00:05:04.008 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.008 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.008 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571678 s, 183 MB/s 2024-04-25T00:05:04.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.017 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663804 s, 158 MB/s 2024-04-25T00:05:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656315 s, 160 MB/s 2024-04-25T00:05:04.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.033 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573628 s, 183 MB/s 2024-04-25T00:05:04.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718968 s, 146 MB/s 2024-04-25T00:05:04.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.050 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564186 s, 186 MB/s 2024-04-25T00:05:04.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067861 s, 155 MB/s 2024-04-25T00:05:04.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.067 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684176 s, 153 MB/s 2024-04-25T00:05:04.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.076 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711254 s, 147 MB/s 2024-04-25T00:05:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635312 s, 165 MB/s 2024-04-25T00:05:04.092 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.092 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.092 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065265 s, 161 MB/s 2024-04-25T00:05:04.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.100 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654658 s, 160 MB/s 2024-04-25T00:05:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706399 s, 148 MB/s 2024-04-25T00:05:04.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568224 s, 185 MB/s 2024-04-25T00:05:04.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.124 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649988 s, 161 MB/s 2024-04-25T00:05:04.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.132 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056543 s, 185 MB/s 2024-04-25T00:05:04.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:04.138 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:04.138 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:04.138 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T00:05:04.138 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:04.138 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:04.139 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:04.139 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:04.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:04.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.196 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644676 s, 163 MB/s 2024-04-25T00:05:04.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.204 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669638 s, 157 MB/s 2024-04-25T00:05:04.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.214 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711917 s, 147 MB/s 2024-04-25T00:05:04.221 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.221 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.221 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599996 s, 175 MB/s 2024-04-25T00:05:04.229 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.229 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.229 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654877 s, 160 MB/s 2024-04-25T00:05:04.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.237 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006822 s, 154 MB/s 2024-04-25T00:05:04.245 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.245 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.245 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640122 s, 164 MB/s 2024-04-25T00:05:04.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.254 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723757 s, 145 MB/s 2024-04-25T00:05:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574147 s, 183 MB/s 2024-04-25T00:05:04.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698831 s, 150 MB/s 2024-04-25T00:05:04.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.279 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701934 s, 149 MB/s 2024-04-25T00:05:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645906 s, 162 MB/s 2024-04-25T00:05:04.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.294 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575456 s, 182 MB/s 2024-04-25T00:05:04.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731333 s, 143 MB/s 2024-04-25T00:05:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734672 s, 143 MB/s 2024-04-25T00:05:04.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071175 s, 147 MB/s 2024-04-25T00:05:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738819 s, 142 MB/s 2024-04-25T00:05:04.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.338 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.338 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635736 s, 165 MB/s 2024-04-25T00:05:04.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.346 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643342 s, 163 MB/s 2024-04-25T00:05:04.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.354 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602006 s, 174 MB/s 2024-04-25T00:05:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594744 s, 176 MB/s 2024-04-25T00:05:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716321 s, 146 MB/s 2024-04-25T00:05:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741507 s, 141 MB/s 2024-04-25T00:05:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706164 s, 148 MB/s 2024-04-25T00:05:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723923 s, 145 MB/s 2024-04-25T00:05:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657788 s, 159 MB/s 2024-04-25T00:05:04.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.413 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711242 s, 147 MB/s 2024-04-25T00:05:04.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.421 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631386 s, 166 MB/s 2024-04-25T00:05:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575551 s, 182 MB/s 2024-04-25T00:05:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583356 s, 180 MB/s 2024-04-25T00:05:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671224 s, 156 MB/s 2024-04-25T00:05:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777205 s, 135 MB/s 2024-04-25T00:05:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:04.461 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:04.461 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:04.461 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T00:05:04.461 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:04.461 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:04.462 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:04.462 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:04.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:04.520 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.520 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.520 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726489 s, 144 MB/s 2024-04-25T00:05:04.528 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710956 s, 147 MB/s 2024-04-25T00:05:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578299 s, 181 MB/s 2024-04-25T00:05:04.545 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.545 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.545 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677452 s, 155 MB/s 2024-04-25T00:05:04.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.553 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.553 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572779 s, 183 MB/s 2024-04-25T00:05:04.562 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.562 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.562 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674179 s, 156 MB/s 2024-04-25T00:05:04.570 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.570 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.570 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698535 s, 150 MB/s 2024-04-25T00:05:04.577 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.577 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.577 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579847 s, 181 MB/s 2024-04-25T00:05:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071129 s, 147 MB/s 2024-04-25T00:05:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568479 s, 184 MB/s 2024-04-25T00:05:04.601 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.601 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.601 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645974 s, 162 MB/s 2024-04-25T00:05:04.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.610 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072815 s, 144 MB/s 2024-04-25T00:05:04.619 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.619 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.619 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720382 s, 146 MB/s 2024-04-25T00:05:04.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.626 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567311 s, 185 MB/s 2024-04-25T00:05:04.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.635 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690137 s, 152 MB/s 2024-04-25T00:05:04.643 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.643 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.643 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655198 s, 160 MB/s 2024-04-25T00:05:04.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.651 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.651 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595316 s, 176 MB/s 2024-04-25T00:05:04.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.658 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568507 s, 184 MB/s 2024-04-25T00:05:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660392 s, 159 MB/s 2024-04-25T00:05:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728843 s, 144 MB/s 2024-04-25T00:05:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057 s, 184 MB/s 2024-04-25T00:05:04.690 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.690 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.690 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600512 s, 175 MB/s 2024-04-25T00:05:04.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.698 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652705 s, 161 MB/s 2024-04-25T00:05:04.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.707 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066584 s, 157 MB/s 2024-04-25T00:05:04.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.715 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715463 s, 147 MB/s 2024-04-25T00:05:04.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549706 s, 191 MB/s 2024-04-25T00:05:04.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670873 s, 156 MB/s 2024-04-25T00:05:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676479 s, 155 MB/s 2024-04-25T00:05:04.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.747 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066363 s, 158 MB/s 2024-04-25T00:05:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663336 s, 158 MB/s 2024-04-25T00:05:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560343 s, 187 MB/s 2024-04-25T00:05:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648547 s, 162 MB/s 2024-04-25T00:05:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:04.777 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:04.777 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:04.777 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T00:05:04.777 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:04.777 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:04.778 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:04.778 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:04.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:04.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584441 s, 179 MB/s 2024-04-25T00:05:04.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.842 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060193 s, 174 MB/s 2024-04-25T00:05:04.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.850 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657515 s, 159 MB/s 2024-04-25T00:05:04.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.859 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715438 s, 147 MB/s 2024-04-25T00:05:04.867 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.867 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.867 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648097 s, 162 MB/s 2024-04-25T00:05:04.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.875 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654751 s, 160 MB/s 2024-04-25T00:05:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637673 s, 164 MB/s 2024-04-25T00:05:04.892 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.892 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.892 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727441 s, 144 MB/s 2024-04-25T00:05:04.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.901 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737184 s, 142 MB/s 2024-04-25T00:05:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667133 s, 157 MB/s 2024-04-25T00:05:04.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.918 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576052 s, 182 MB/s 2024-04-25T00:05:04.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.927 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.927 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638802 s, 164 MB/s 2024-04-25T00:05:04.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649615 s, 161 MB/s 2024-04-25T00:05:04.943 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.943 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.943 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637901 s, 164 MB/s 2024-04-25T00:05:04.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.952 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713414 s, 147 MB/s 2024-04-25T00:05:04.960 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.960 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.960 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636446 s, 165 MB/s 2024-04-25T00:05:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566469 s, 185 MB/s 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553225 s, 190 MB/s 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657724 s, 159 MB/s 2024-04-25T00:05:04.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:04.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:04.992 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663949 s, 158 MB/s 2024-04-25T00:05:05.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.001 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637053 s, 165 MB/s 2024-04-25T00:05:05.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.009 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639218 s, 164 MB/s 2024-04-25T00:05:05.016 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.016 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.016 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055444 s, 189 MB/s 2024-04-25T00:05:05.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.025 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662326 s, 158 MB/s 2024-04-25T00:05:05.035 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.035 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.035 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751731 s, 139 MB/s 2024-04-25T00:05:05.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587432 s, 179 MB/s 2024-04-25T00:05:05.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.050 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588199 s, 178 MB/s 2024-04-25T00:05:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063914 s, 164 MB/s 2024-04-25T00:05:05.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.067 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664862 s, 158 MB/s 2024-04-25T00:05:05.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.076 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707985 s, 148 MB/s 2024-04-25T00:05:05.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.085 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666891 s, 157 MB/s 2024-04-25T00:05:05.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.094 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725528 s, 145 MB/s 2024-04-25T00:05:05.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:05.101 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:05.101 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:05.101 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T00:05:05.101 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:05.101 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:05.101 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:05.101 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:05.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:05.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.159 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645028 s, 163 MB/s 2024-04-25T00:05:05.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.170 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761684 s, 138 MB/s 2024-04-25T00:05:05.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.178 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681187 s, 154 MB/s 2024-04-25T00:05:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663378 s, 158 MB/s 2024-04-25T00:05:05.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.195 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683173 s, 153 MB/s 2024-04-25T00:05:05.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063228 s, 166 MB/s 2024-04-25T00:05:05.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.211 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581644 s, 180 MB/s 2024-04-25T00:05:05.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.219 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057286 s, 183 MB/s 2024-04-25T00:05:05.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.228 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676296 s, 155 MB/s 2024-04-25T00:05:05.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580018 s, 181 MB/s 2024-04-25T00:05:05.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.245 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716137 s, 146 MB/s 2024-04-25T00:05:05.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.253 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063929 s, 164 MB/s 2024-04-25T00:05:05.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.261 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663426 s, 158 MB/s 2024-04-25T00:05:05.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.269 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588871 s, 178 MB/s 2024-04-25T00:05:05.277 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.277 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.277 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597341 s, 176 MB/s 2024-04-25T00:05:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559339 s, 187 MB/s 2024-04-25T00:05:05.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.294 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731491 s, 143 MB/s 2024-04-25T00:05:05.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665719 s, 158 MB/s 2024-04-25T00:05:05.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762639 s, 137 MB/s 2024-04-25T00:05:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616415 s, 170 MB/s 2024-04-25T00:05:05.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.330 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710337 s, 148 MB/s 2024-04-25T00:05:05.339 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.339 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.339 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703401 s, 149 MB/s 2024-04-25T00:05:05.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.347 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594933 s, 176 MB/s 2024-04-25T00:05:05.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.354 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566946 s, 185 MB/s 2024-04-25T00:05:05.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.363 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650991 s, 161 MB/s 2024-04-25T00:05:05.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.372 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704439 s, 149 MB/s 2024-04-25T00:05:05.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.381 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727621 s, 144 MB/s 2024-04-25T00:05:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654882 s, 160 MB/s 2024-04-25T00:05:05.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.398 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678694 s, 154 MB/s 2024-04-25T00:05:05.407 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.407 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.407 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659935 s, 159 MB/s 2024-04-25T00:05:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066526 s, 158 MB/s 2024-04-25T00:05:05.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.424 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649089 s, 162 MB/s 2024-04-25T00:05:05.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:05.431 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:05.431 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:05.431 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T00:05:05.431 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:05.431 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:05.431 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:05.431 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:05.489 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.489 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.489 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598369 s, 175 MB/s 2024-04-25T00:05:05.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.499 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747528 s, 140 MB/s 2024-04-25T00:05:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656626 s, 160 MB/s 2024-04-25T00:05:05.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.515 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646517 s, 162 MB/s 2024-04-25T00:05:05.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.522 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058586 s, 179 MB/s 2024-04-25T00:05:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652847 s, 161 MB/s 2024-04-25T00:05:05.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.538 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647361 s, 162 MB/s 2024-04-25T00:05:05.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.547 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068742 s, 153 MB/s 2024-04-25T00:05:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650912 s, 161 MB/s 2024-04-25T00:05:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618091 s, 170 MB/s 2024-04-25T00:05:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729728 s, 144 MB/s 2024-04-25T00:05:05.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.580 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643336 s, 163 MB/s 2024-04-25T00:05:05.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.587 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555417 s, 189 MB/s 2024-04-25T00:05:05.595 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.596 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.596 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714203 s, 147 MB/s 2024-04-25T00:05:05.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.603 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056714 s, 185 MB/s 2024-04-25T00:05:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651267 s, 161 MB/s 2024-04-25T00:05:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747643 s, 140 MB/s 2024-04-25T00:05:05.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.628 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643586 s, 163 MB/s 2024-04-25T00:05:05.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638118 s, 164 MB/s 2024-04-25T00:05:05.643 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.644 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597983 s, 175 MB/s 2024-04-25T00:05:05.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706137 s, 148 MB/s 2024-04-25T00:05:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.660 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594085 s, 177 MB/s 2024-04-25T00:05:05.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.669 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686536 s, 153 MB/s 2024-04-25T00:05:05.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.677 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066154 s, 159 MB/s 2024-04-25T00:05:05.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.687 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796368 s, 132 MB/s 2024-04-25T00:05:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698902 s, 150 MB/s 2024-04-25T00:05:05.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.705 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699317 s, 150 MB/s 2024-04-25T00:05:05.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.715 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838656 s, 125 MB/s 2024-04-25T00:05:05.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.723 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643296 s, 163 MB/s 2024-04-25T00:05:05.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.732 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.732 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667777 s, 157 MB/s 2024-04-25T00:05:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800574 s, 131 MB/s 2024-04-25T00:05:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712544 s, 147 MB/s 2024-04-25T00:05:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:05.759 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:05.759 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:05.759 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T00:05:05.760 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:05.760 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:05.760 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:05.760 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:05.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:05.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.819 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586419 s, 179 MB/s 2024-04-25T00:05:05.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.827 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635261 s, 165 MB/s 2024-04-25T00:05:05.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668317 s, 157 MB/s 2024-04-25T00:05:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672299 s, 156 MB/s 2024-04-25T00:05:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686143 s, 153 MB/s 2024-04-25T00:05:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.861 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780274 s, 134 MB/s 2024-04-25T00:05:05.869 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.869 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.869 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695919 s, 151 MB/s 2024-04-25T00:05:05.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.877 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661685 s, 158 MB/s 2024-04-25T00:05:05.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.885 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574628 s, 182 MB/s 2024-04-25T00:05:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639725 s, 164 MB/s 2024-04-25T00:05:05.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.900 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560223 s, 187 MB/s 2024-04-25T00:05:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573865 s, 183 MB/s 2024-04-25T00:05:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.916 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652446 s, 161 MB/s 2024-04-25T00:05:05.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.924 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655352 s, 160 MB/s 2024-04-25T00:05:05.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.932 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070681 s, 148 MB/s 2024-04-25T00:05:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.939 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584501 s, 179 MB/s 2024-04-25T00:05:05.948 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.948 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.948 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718504 s, 146 MB/s 2024-04-25T00:05:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597471 s, 176 MB/s 2024-04-25T00:05:05.964 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.964 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.964 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066782 s, 157 MB/s 2024-04-25T00:05:05.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.972 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680573 s, 154 MB/s 2024-04-25T00:05:05.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.981 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718335 s, 146 MB/s 2024-04-25T00:05:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640078 s, 164 MB/s 2024-04-25T00:05:05.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:05.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:05.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562836 s, 186 MB/s 2024-04-25T00:05:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057817 s, 181 MB/s 2024-04-25T00:05:06.012 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.013 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686176 s, 153 MB/s 2024-04-25T00:05:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676143 s, 155 MB/s 2024-04-25T00:05:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717744 s, 146 MB/s 2024-04-25T00:05:06.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.037 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570838 s, 184 MB/s 2024-04-25T00:05:06.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.046 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704159 s, 149 MB/s 2024-04-25T00:05:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677906 s, 155 MB/s 2024-04-25T00:05:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627675 s, 167 MB/s 2024-04-25T00:05:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718794 s, 146 MB/s 2024-04-25T00:05:06.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:06.077 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:06.077 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:06.077 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T00:05:06.077 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:06.078 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:06.078 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:06.078 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:06.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.137 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752639 s, 139 MB/s 2024-04-25T00:05:06.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679977 s, 154 MB/s 2024-04-25T00:05:06.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.152 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585429 s, 179 MB/s 2024-04-25T00:05:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562979 s, 186 MB/s 2024-04-25T00:05:06.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.168 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701321 s, 150 MB/s 2024-04-25T00:05:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704105 s, 149 MB/s 2024-04-25T00:05:06.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.184 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564166 s, 186 MB/s 2024-04-25T00:05:06.193 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.193 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.193 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693731 s, 151 MB/s 2024-04-25T00:05:06.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.200 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005739 s, 183 MB/s 2024-04-25T00:05:06.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.209 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.209 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694675 s, 151 MB/s 2024-04-25T00:05:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720743 s, 145 MB/s 2024-04-25T00:05:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604895 s, 173 MB/s 2024-04-25T00:05:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658692 s, 159 MB/s 2024-04-25T00:05:06.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.241 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566291 s, 185 MB/s 2024-04-25T00:05:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.250 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726986 s, 144 MB/s 2024-04-25T00:05:06.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.258 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711955 s, 147 MB/s 2024-04-25T00:05:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.267 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659449 s, 159 MB/s 2024-04-25T00:05:06.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.275 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725167 s, 145 MB/s 2024-04-25T00:05:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067391 s, 156 MB/s 2024-04-25T00:05:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670443 s, 156 MB/s 2024-04-25T00:05:06.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.300 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654633 s, 160 MB/s 2024-04-25T00:05:06.307 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.308 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619364 s, 169 MB/s 2024-04-25T00:05:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626545 s, 167 MB/s 2024-04-25T00:05:06.323 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.323 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.324 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622609 s, 168 MB/s 2024-04-25T00:05:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733386 s, 143 MB/s 2024-04-25T00:05:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594416 s, 176 MB/s 2024-04-25T00:05:06.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687077 s, 153 MB/s 2024-04-25T00:05:06.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.356 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587792 s, 178 MB/s 2024-04-25T00:05:06.365 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.365 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.365 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758599 s, 138 MB/s 2024-04-25T00:05:06.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.373 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675953 s, 155 MB/s 2024-04-25T00:05:06.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.382 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650647 s, 161 MB/s 2024-04-25T00:05:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636612 s, 165 MB/s 2024-04-25T00:05:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:06.396 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:06.397 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:06.397 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T00:05:06.397 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:06.397 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:06.397 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:06.397 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:06.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699976 s, 150 MB/s 2024-04-25T00:05:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068811 s, 152 MB/s 2024-04-25T00:05:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569051 s, 184 MB/s 2024-04-25T00:05:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713352 s, 147 MB/s 2024-04-25T00:05:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562925 s, 186 MB/s 2024-04-25T00:05:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.497 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759175 s, 138 MB/s 2024-04-25T00:05:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566051 s, 185 MB/s 2024-04-25T00:05:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675624 s, 155 MB/s 2024-04-25T00:05:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634664 s, 165 MB/s 2024-04-25T00:05:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735196 s, 143 MB/s 2024-04-25T00:05:06.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.539 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725024 s, 145 MB/s 2024-04-25T00:05:06.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.548 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643558 s, 163 MB/s 2024-04-25T00:05:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066301 s, 158 MB/s 2024-04-25T00:05:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578053 s, 181 MB/s 2024-04-25T00:05:06.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.574 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768809 s, 136 MB/s 2024-04-25T00:05:06.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.582 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652323 s, 161 MB/s 2024-04-25T00:05:06.590 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.590 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.590 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660352 s, 159 MB/s 2024-04-25T00:05:06.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.599 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064527 s, 163 MB/s 2024-04-25T00:05:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639985 s, 164 MB/s 2024-04-25T00:05:06.616 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.616 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.616 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720681 s, 145 MB/s 2024-04-25T00:05:06.624 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.624 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.624 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059086 s, 177 MB/s 2024-04-25T00:05:06.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.632 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564819 s, 186 MB/s 2024-04-25T00:05:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693239 s, 151 MB/s 2024-04-25T00:05:06.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.649 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709024 s, 148 MB/s 2024-04-25T00:05:06.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.658 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691218 s, 152 MB/s 2024-04-25T00:05:06.667 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.667 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.667 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691258 s, 152 MB/s 2024-04-25T00:05:06.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.675 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.675 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563981 s, 186 MB/s 2024-04-25T00:05:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.684 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.684 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700424 s, 150 MB/s 2024-04-25T00:05:06.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.692 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649161 s, 162 MB/s 2024-04-25T00:05:06.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653874 s, 160 MB/s 2024-04-25T00:05:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694679 s, 151 MB/s 2024-04-25T00:05:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607319 s, 173 MB/s 2024-04-25T00:05:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:06.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:06.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T00:05:06.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.826+0000 7febaaf09640 1 Processor -- start 2024-04-25T00:05:06.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.826+0000 7febaaf09640 1 -- start start 2024-04-25T00:05:06.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7febaaf09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feba406e0d0 0x7feba406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7febaaf09640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feba40cd630 0x7feba40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7febaaf09640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba40cdfc0 con 0x7feba4144a80 2024-04-25T00:05:06.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7febaaf09640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feba406ea70 con 0x7feba406e0d0 2024-04-25T00:05:06.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7febaaf09640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7feba4072f70 con 0x7feba40cd630 2024-04-25T00:05:06.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7feba3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feba406e0d0 0x7feba406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7feba3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feba406e0d0 0x7feba406e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39480/0 (socket says 172.21.15.113:39480) 2024-04-25T00:05:06.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7feba3fff640 1 -- 172.21.15.113:0/2760530952 learned_addr learned my addr 172.21.15.113:0/2760530952 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:06.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.827+0000 7feba94a6640 1 --2- 172.21.15.113:0/2760530952 >> v2:172.21.15.178:3300/0 conn(0x7feba40cd630 0x7feba40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba3fff640 1 -- 172.21.15.113:0/2760530952 >> v2:172.21.15.178:3300/0 conn(0x7feba40cd630 msgr2=0x7feba40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba3fff640 1 --2- 172.21.15.113:0/2760530952 >> v2:172.21.15.178:3300/0 conn(0x7feba40cd630 0x7feba40cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba3fff640 1 -- 172.21.15.113:0/2760530952 >> v1:172.21.15.178:6789/0 conn(0x7feba4144a80 legacy=0x7feba40b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba3fff640 1 -- 172.21.15.113:0/2760530952 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba40b2fd0 con 0x7feba406e0d0 2024-04-25T00:05:06.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba3fff640 1 --2- 172.21.15.113:0/2760530952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba406e0d0 0x7feba406e530 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7feba40d0db0 tx=0x7feb9802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7252a4c559d3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba37fe640 1 -- 172.21.15.113:0/2760530952 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98033640 con 0x7feba406e0d0 2024-04-25T00:05:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba37fe640 1 -- 172.21.15.113:0/2760530952 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb98038070 con 0x7feba406e0d0 2024-04-25T00:05:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.828+0000 7feba37fe640 1 -- 172.21.15.113:0/2760530952 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98033a00 con 0x7feba406e0d0 2024-04-25T00:05:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 -- 172.21.15.113:0/2760530952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba406e0d0 msgr2=0x7feba406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 --2- 172.21.15.113:0/2760530952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba406e0d0 0x7feba406e530 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7feba40d0db0 tx=0x7feb9802c8b0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 -- 172.21.15.113:0/2760530952 shutdown_connections 2024-04-25T00:05:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 --2- 172.21.15.113:0/2760530952 >> v2:172.21.15.178:3300/0 conn(0x7feba40cd630 0x7feba40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 --2- 172.21.15.113:0/2760530952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba406e0d0 0x7feba406e530 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 -- 172.21.15.113:0/2760530952 >> 172.21.15.113:0/2760530952 conn(0x7feba40b1b30 msgr2=0x7feba40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:06.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 -- 172.21.15.113:0/2760530952 shutdown_connections 2024-04-25T00:05:06.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.829+0000 7febaaf09640 1 -- 172.21.15.113:0/2760530952 wait complete. 2024-04-25T00:05:06.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.830+0000 7febaaf09640 1 Processor -- start 2024-04-25T00:05:06.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.830+0000 7febaaf09640 1 -- start start 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7febaaf09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba406e0d0 0x7feba41f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7febaaf09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba40cd630 0x7feba413db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7febaaf09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba4144a80 0x7feba413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7febaaf09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feba40b41c0 con 0x7feba4144a80 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7febaaf09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba40b3590 con 0x7feba406e0d0 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7febaaf09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feba40b4340 con 0x7feba40cd630 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba4144a80 0x7feba413e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba4144a80 0x7feba413e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39494/0 (socket says 172.21.15.113:39494) 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 -- 172.21.15.113:0/270764517 learned_addr learned my addr 172.21.15.113:0/270764517 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba3fff640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba40cd630 0x7feba413db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba8ca5640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba406e0d0 0x7feba41f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 -- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba40cd630 msgr2=0x7feba413db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba40cd630 0x7feba413db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 -- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba406e0d0 msgr2=0x7feba41f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba406e0d0 0x7feba41f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.831+0000 7feba94a6640 1 -- 172.21.15.113:0/270764517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba415b770 con 0x7feba4144a80 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.832+0000 7feba8ca5640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba406e0d0 0x7feba41f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.832+0000 7feba94a6640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba4144a80 0x7feba413e0d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00efc0 tx=0x7feb9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.832+0000 7feba17fa640 1 -- 172.21.15.113:0/270764517 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c018020 con 0x7feba4144a80 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.832+0000 7feba17fa640 1 -- 172.21.15.113:0/270764517 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c014480 con 0x7feba4144a80 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.832+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba415ba50 con 0x7feba4144a80 2024-04-25T00:05:06.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.832+0000 7feba17fa640 1 -- 172.21.15.113:0/270764517 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c014870 con 0x7feba4144a80 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.832+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba415c010 con 0x7feba4144a80 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.833+0000 7feba17fa640 1 -- 172.21.15.113:0/270764517 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb9c005d10 con 0x7feba4144a80 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.834+0000 7feba17fa640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9006be30 0x7feb9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.834+0000 7feba17fa640 1 -- 172.21.15.113:0/270764517 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7feb9c08bbc0 con 0x7feba4144a80 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.834+0000 7feba8ca5640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9006be30 0x7feb9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.834+0000 7feba8ca5640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9006be30 0x7feb9006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7feb94001660 tx=0x7feb94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.834+0000 7febadc0d740 1 Processor -- start 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7febadc0d740 1 -- start start 2024-04-25T00:05:06.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7febadc0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b45cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b76440 0x560f47b757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b75d00 0x560f47b4ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7febadc0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560f47aad930 con 0x560f47b93320 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560f47a55cc0 con 0x560f47b75d00 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560f47aa98f0 con 0x560f47b76440 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7feba8ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b45cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7feba8ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b45cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39506/0 (socket says 172.21.15.113:39506) 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7feba8ca5640 1 -- 172.21.15.113:0/2327345635 learned_addr learned my addr 172.21.15.113:0/2327345635 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7feba94a6640 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b75d00 0x560f47b4ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.835+0000 7feba3fff640 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b76440 0x560f47b757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feba8ca5640 1 -- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b76440 msgr2=0x560f47b757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feba8ca5640 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b76440 0x560f47b757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feba8ca5640 1 -- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b75d00 msgr2=0x560f47b4ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feba8ca5640 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b75d00 0x560f47b4ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feba8ca5640 1 -- 172.21.15.113:0/2327345635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4788dba0 con 0x560f47b93320 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feba8ca5640 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b45cb0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7feb94001eb0 tx=0x7feb94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95900163ec58f7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feb8a7fc640 1 -- 172.21.15.113:0/2327345635 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940035f0 con 0x560f47b93320 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feb8a7fc640 1 -- 172.21.15.113:0/2327345635 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94003790 con 0x560f47b93320 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7feb8a7fc640 1 -- 172.21.15.113:0/2327345635 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94003b80 con 0x560f47b93320 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 -- 172.21.15.113:0/2327345635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 msgr2=0x560f47b45cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b45cb0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7feb94001eb0 tx=0x7feb94011000 comp rx=0 tx=0).stop 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 -- 172.21.15.113:0/2327345635 shutdown_connections 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b75d00 0x560f47b4ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b76440 0x560f47b757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 --2- 172.21.15.113:0/2327345635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b45cb0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 -- 172.21.15.113:0/2327345635 >> 172.21.15.113:0/2327345635 conn(0x560f47b47f20 msgr2=0x560f47b907c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.836+0000 7febadc0d740 1 -- 172.21.15.113:0/2327345635 shutdown_connections 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.837+0000 7febadc0d740 1 -- 172.21.15.113:0/2327345635 wait complete. 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.837+0000 7febadc0d740 1 -- start start 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.837+0000 7febadc0d740 1 Processor -- start 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b76440 0x560f47b63160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7febadc0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b63ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b88710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7febadc0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560f47b4a0d0 con 0x560f47b93320 2024-04-25T00:05:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560f47a69330 con 0x560f47b76440 2024-04-25T00:05:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560f47b494c0 con 0x560f47b88290 2024-04-25T00:05:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7febadc0d740 10 client.0 _ll_register_callbacks cb 0x560f47aa5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b63ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba8ca5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b76440 0x560f47b63160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b63ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39508/0 (socket says 172.21.15.113:39508) 2024-04-25T00:05:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba94a6640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b88710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 -- 172.21.15.113:0/2322977425 learned_addr learned my addr 172.21.15.113:0/2322977425 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 -- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 msgr2=0x560f47b88710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b88710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 -- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b76440 msgr2=0x560f47b63160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b76440 0x560f47b63160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba3fff640 1 -- 172.21.15.113:0/2322977425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f47781490 con 0x560f47b93320 2024-04-25T00:05:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba8ca5640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b76440 0x560f47b63160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.838+0000 7feba94a6640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b88710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feba3fff640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b63ab0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7feb98008010 tx=0x7feb9802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb3425744c54c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feb837fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feb837fe640 1 -- 172.21.15.113:0/2322977425 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9804b060 con 0x560f47b93320 2024-04-25T00:05:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feb837fe640 1 -- 172.21.15.113:0/2322977425 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7feb98046410 con 0x560f47b93320 2024-04-25T00:05:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb94003790 con 0x560f47b93320 2024-04-25T00:05:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feb837fe640 1 -- 172.21.15.113:0/2322977425 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7feb98051050 con 0x560f47b93320 2024-04-25T00:05:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 10 client.8016 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x560f478ee330 con 0x560f47b93320 2024-04-25T00:05:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feb837fe640 1 -- 172.21.15.113:0/2322977425 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94003790 con 0x560f47b93320 2024-04-25T00:05:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feb837fe640 1 -- 172.21.15.113:0/2322977425 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7feb98054020 con 0x560f47b93320 2024-04-25T00:05:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 10 client.8016 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 10 client.8016 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 20 client.8016 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:06.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47b792d0 0x560f47b76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 4 client.8016 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:06.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f47abae70 con 0x560f47b792d0 2024-04-25T00:05:06.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.839+0000 7feba8ca5640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47b792d0 0x560f47b76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7feb837fe640 10 client.8016 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7feba8ca5640 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47b792d0 0x560f47b76930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610be86fa563ea85 server_cookie=bde8123da66dcfe9 in_seq=0 out_seq=0 2024-04-25T00:05:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7feb837fe640 1 -- 172.21.15.113:0/2322977425 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f47abae70 con 0x560f47b792d0 2024-04-25T00:05:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7feb837fe640 10 client.8016 handle_command_reply: tid=0 2024-04-25T00:05:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 client.8016 shutdown 2024-04-25T00:05:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 msgr2=0x560f47b63ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b63ab0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7feb98008010 tx=0x7feb9802f000 comp rx=0 tx=0).stop 2024-04-25T00:05:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 shutdown_connections 2024-04-25T00:05:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47b792d0 0x560f47b76930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b88710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b93320 0x560f47b63ab0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7feb837fe640 0 client.8016 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 --2- 172.21.15.113:0/2322977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b76440 0x560f47b63160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 >> 172.21.15.113:0/2322977425 conn(0x560f47b75d00 msgr2=0x560f47b4a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 shutdown_connections 2024-04-25T00:05:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.840+0000 7febadc0d740 1 -- 172.21.15.113:0/2322977425 wait complete. 2024-04-25T00:05:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.841+0000 7febadc0d740 20 client.8016 trim_cache size 0 max 16384 2024-04-25T00:05:06.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.858+0000 7febadc0d740 1 Processor -- start 2024-04-25T00:05:06.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.858+0000 7febadc0d740 1 -- start start 2024-04-25T00:05:06.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7febadc0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b62f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b88290 0x560f47b414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b93320 0x560f47b70db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7febadc0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feba4200850 con 0x560f47b76440 2024-04-25T00:05:06.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba4140bc0 con 0x560f47b88290 2024-04-25T00:05:06.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feba40ae760 con 0x560f47b93320 2024-04-25T00:05:06.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b62f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b62f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39522/0 (socket says 172.21.15.113:39522) 2024-04-25T00:05:06.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 -- 172.21.15.113:0/290515242 learned_addr learned my addr 172.21.15.113:0/290515242 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:06.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba94a6640 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b93320 0x560f47b70db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba3fff640 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b88290 0x560f47b414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 -- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b93320 msgr2=0x560f47b70db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b93320 0x560f47b70db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 -- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b88290 msgr2=0x560f47b414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b88290 0x560f47b414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 -- 172.21.15.113:0/290515242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4775a610 con 0x560f47b76440 2024-04-25T00:05:06.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba94a6640 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b93320 0x560f47b70db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:06.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feba8ca5640 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b62f70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7feb94014000 tx=0x7feb94017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352a7a569ae25a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:06.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feb89ffb640 1 -- 172.21.15.113:0/290515242 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94003c90 con 0x560f47b76440 2024-04-25T00:05:06.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feb89ffb640 1 -- 172.21.15.113:0/290515242 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94003e30 con 0x560f47b76440 2024-04-25T00:05:06.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.859+0000 7feb89ffb640 1 -- 172.21.15.113:0/290515242 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9402f370 con 0x560f47b76440 2024-04-25T00:05:06.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 -- 172.21.15.113:0/290515242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 msgr2=0x560f47b62f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b62f70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7feb94014000 tx=0x7feb94017070 comp rx=0 tx=0).stop 2024-04-25T00:05:06.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 -- 172.21.15.113:0/290515242 shutdown_connections 2024-04-25T00:05:06.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b88290 0x560f47b414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b93320 0x560f47b70db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 --2- 172.21.15.113:0/290515242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b62f70 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 -- 172.21.15.113:0/290515242 >> 172.21.15.113:0/290515242 conn(0x560f47b75d00 msgr2=0x560f47b44fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:06.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 -- 172.21.15.113:0/290515242 shutdown_connections 2024-04-25T00:05:06.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 -- 172.21.15.113:0/290515242 wait complete. 2024-04-25T00:05:06.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.860+0000 7febadc0d740 1 -- start start 2024-04-25T00:05:06.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 1 Processor -- start 2024-04-25T00:05:06.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b93320 0x560f47bb30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560f47b87fe0 con 0x560f47b76440 2024-04-25T00:05:06.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba4140570 con 0x560f47b93320 2024-04-25T00:05:06.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560f47b41c30 con 0x560f47b88290 2024-04-25T00:05:06.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7febadc0d740 10 client.0 _ll_register_callbacks cb 0x7feb9800b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:06.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7feba8ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7feba8ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b86150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39536/0 (socket says 172.21.15.113:39536) 2024-04-25T00:05:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.861+0000 7feba8ca5640 1 -- 172.21.15.113:0/3953374210 learned_addr learned my addr 172.21.15.113:0/3953374210 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba3fff640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba94a6640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b93320 0x560f47bb30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba8ca5640 1 -- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 msgr2=0x560f47b86690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba8ca5640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b86690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba8ca5640 1 -- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b93320 msgr2=0x560f47bb30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:06.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba8ca5640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b93320 0x560f47bb30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba8ca5640 1 -- 172.21.15.113:0/3953374210 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f47b71990 con 0x560f47b76440 2024-04-25T00:05:06.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba94a6640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b93320 0x560f47bb30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:06.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba3fff640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b86690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:06.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feba8ca5640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b86150 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7feb94009010 tx=0x7feb94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10f3af6f9963335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:06.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feb837fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:06.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feb837fe640 1 -- 172.21.15.113:0/3953374210 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94034070 con 0x560f47b76440 2024-04-25T00:05:06.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feb837fe640 1 -- 172.21.15.113:0/3953374210 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7feb9402f370 con 0x560f47b76440 2024-04-25T00:05:06.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb94003e30 con 0x560f47b76440 2024-04-25T00:05:06.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feb837fe640 1 -- 172.21.15.113:0/3953374210 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7feb94041050 con 0x560f47b76440 2024-04-25T00:05:06.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 10 client.8022 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:06.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x560f47ba04e0 con 0x560f47b76440 2024-04-25T00:05:06.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feb837fe640 1 -- 172.21.15.113:0/3953374210 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94003e30 con 0x560f47b76440 2024-04-25T00:05:06.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7feb837fe640 1 -- 172.21.15.113:0/3953374210 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7feb94044020 con 0x560f47b76440 2024-04-25T00:05:06.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 10 client.8022 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:06.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 10 client.8022 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:06.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 20 client.8022 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47c09d20 0x560f47bb3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 4 client.8022 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T00:05:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.862+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x560f47ab51c0 con 0x560f47c09d20 2024-04-25T00:05:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.863+0000 7feba3fff640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47c09d20 0x560f47bb3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.863+0000 7feb837fe640 10 client.8022 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.863+0000 7feba3fff640 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47c09d20 0x560f47bb3620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410d5c1323fb9fd1 server_cookie=42a861de6ec22f24 in_seq=0 out_seq=0 2024-04-25T00:05:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.865+0000 7feb837fe640 1 -- 172.21.15.113:0/3953374210 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f47ab51c0 con 0x560f47c09d20 2024-04-25T00:05:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.865+0000 7feb837fe640 10 client.8022 handle_command_reply: tid=0 2024-04-25T00:05:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.865+0000 7febadc0d740 1 client.8022 shutdown 2024-04-25T00:05:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.865+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 msgr2=0x560f47b86150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.865+0000 7febadc0d740 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b86150 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7feb94009010 tx=0x7feb94000bc0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 shutdown_connections 2024-04-25T00:05:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x560f47c09d20 0x560f47bb3620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560f47b93320 0x560f47bb30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560f47b88290 0x560f47b86690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7feb837fe640 0 client.8022 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 --2- 172.21.15.113:0/3953374210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560f47b76440 0x560f47b86150 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 >> 172.21.15.113:0/3953374210 conn(0x560f47b75d00 msgr2=0x560f47b70db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 shutdown_connections 2024-04-25T00:05:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 1 -- 172.21.15.113:0/3953374210 wait complete. 2024-04-25T00:05:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febadc0d740 20 client.8022 trim_cache size 0 max 16384 2024-04-25T00:05:06.889 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:06.889 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:05:06.889 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "bfa02794-118a-4fbc-8b32-9fb5063d3d27", 2024-04-25T00:05:06.889 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:05:06.890 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:06.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9006be30 msgr2=0x7feb9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febaaf09640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9006be30 0x7feb9006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7feb94001660 tx=0x7feb94009040 comp rx=0 tx=0).stop 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba4144a80 msgr2=0x7feba413e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.866+0000 7febaaf09640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba4144a80 0x7feba413e0d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00efc0 tx=0x7feb9c00c490 comp rx=0 tx=0).stop 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 shutdown_connections 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9006be30 0x7feb9006e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba4144a80 0x7feba413e0d0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba40cd630 0x7feba413db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 --2- 172.21.15.113:0/270764517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba406e0d0 0x7feba41f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 >> 172.21.15.113:0/270764517 conn(0x7feba40b1b30 msgr2=0x7feba40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 shutdown_connections 2024-04-25T00:05:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:06.867+0000 7febaaf09640 1 -- 172.21.15.113:0/270764517 wait complete. 2024-04-25T00:05:06.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:06.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T00:05:07.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.015+0000 7fe037bed640 1 Processor -- start 2024-04-25T00:05:07.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.015+0000 7fe037bed640 1 -- start start 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe037bed640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0300b5ca0 0x7fe0300b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe037bed640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe0300b73c0 0x7fe0300b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe037bed640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0300d2e00 con 0x7fe03015e510 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe037bed640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe0300b6640 con 0x7fe0300b5ca0 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe037bed640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe0300d30e0 con 0x7fe0300b73c0 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe035188640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0300b5ca0 0x7fe0300b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe035989640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe03015e510 0x7fe0300d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52790/0 (socket says 172.21.15.113:52790) 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.016+0000 7fe035989640 1 -- 172.21.15.113:0/1687977714 learned_addr learned my addr 172.21.15.113:0/1687977714 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe035188640 1 -- 172.21.15.113:0/1687977714 >> v2:172.21.15.178:3300/0 conn(0x7fe0300b73c0 msgr2=0x7fe0300b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe035188640 1 --2- 172.21.15.113:0/1687977714 >> v2:172.21.15.178:3300/0 conn(0x7fe0300b73c0 0x7fe0300b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe035188640 1 -- 172.21.15.113:0/1687977714 >> v1:172.21.15.113:6789/0 conn(0x7fe03015e510 legacy=0x7fe0300d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe035188640 1 -- 172.21.15.113:0/1687977714 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0300d33c0 con 0x7fe0300b5ca0 2024-04-25T00:05:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe034987640 1 -- 172.21.15.113:0/1687977714 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318779887 0 0) 0x7fe0300d2e00 con 0x7fe03015e510 2024-04-25T00:05:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe035188640 1 --2- 172.21.15.113:0/1687977714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b5ca0 0x7fe0300b6100 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe0300d1d80 tx=0x7fe02002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b912d46ea229001e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe034987640 1 -- 172.21.15.113:0/1687977714 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020037070 con 0x7fe0300b5ca0 2024-04-25T00:05:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe034987640 1 -- 172.21.15.113:0/1687977714 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe020032440 con 0x7fe0300b5ca0 2024-04-25T00:05:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.017+0000 7fe034987640 1 -- 172.21.15.113:0/1687977714 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02002ae60 con 0x7fe0300b5ca0 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 -- 172.21.15.113:0/1687977714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b5ca0 msgr2=0x7fe0300b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 --2- 172.21.15.113:0/1687977714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b5ca0 0x7fe0300b6100 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe0300d1d80 tx=0x7fe02002a110 comp rx=0 tx=0).stop 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 -- 172.21.15.113:0/1687977714 shutdown_connections 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 --2- 172.21.15.113:0/1687977714 >> v2:172.21.15.178:3300/0 conn(0x7fe0300b73c0 0x7fe0300b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 --2- 172.21.15.113:0/1687977714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b5ca0 0x7fe0300b6100 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 -- 172.21.15.113:0/1687977714 >> 172.21.15.113:0/1687977714 conn(0x7fe03006b720 msgr2=0x7fe0300d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 -- 172.21.15.113:0/1687977714 shutdown_connections 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.018+0000 7fe037bed640 1 -- 172.21.15.113:0/1687977714 wait complete. 2024-04-25T00:05:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.019+0000 7fe037bed640 1 Processor -- start 2024-04-25T00:05:07.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.019+0000 7fe037bed640 1 -- start start 2024-04-25T00:05:07.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe037bed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0300b5ca0 0x7fe0300c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe037bed640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b73c0 0x7fe0300c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe037bed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe03015e510 0x7fe03015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe037bed640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0300d3ea0 con 0x7fe0300b73c0 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe037bed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0300d3980 con 0x7fe03015e510 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe037bed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe0300d2a90 con 0x7fe0300b5ca0 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b73c0 0x7fe0300c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b73c0 0x7fe0300c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39550/0 (socket says 172.21.15.113:39550) 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 -- 172.21.15.113:0/2385695025 learned_addr learned my addr 172.21.15.113:0/2385695025 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe03618a640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe03015e510 0x7fe03015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035989640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0300b5ca0 0x7fe0300c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 -- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0300b5ca0 msgr2=0x7fe0300c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0300b5ca0 0x7fe0300c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 -- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe03015e510 msgr2=0x7fe03015cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe03015e510 0x7fe03015cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 -- 172.21.15.113:0/2385695025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0300ca670 con 0x7fe0300b73c0 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.020+0000 7fe035188640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b73c0 0x7fe0300c9d40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe020009890 tx=0x7fe0200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.021+0000 7fe0267fc640 1 -- 172.21.15.113:0/2385695025 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020037040 con 0x7fe0300b73c0 2024-04-25T00:05:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.021+0000 7fe0267fc640 1 -- 172.21.15.113:0/2385695025 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe020004230 con 0x7fe0300b73c0 2024-04-25T00:05:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.021+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0300ca950 con 0x7fe0300b73c0 2024-04-25T00:05:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.021+0000 7fe0267fc640 1 -- 172.21.15.113:0/2385695025 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020032e20 con 0x7fe0300b73c0 2024-04-25T00:05:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.021+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0300c8830 con 0x7fe0300b73c0 2024-04-25T00:05:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.022+0000 7fe0267fc640 1 -- 172.21.15.113:0/2385695025 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe020030770 con 0x7fe0300b73c0 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.023+0000 7fe0267fc640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01406be30 0x7fe01406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.023+0000 7fe035989640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01406be30 0x7fe01406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.023+0000 7fe0267fc640 1 -- 172.21.15.113:0/2385695025 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fe0200ac030 con 0x7fe0300b73c0 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.023+0000 7fe035989640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01406be30 0x7fe01406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe0280012e0 tx=0x7fe028001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 Processor -- start 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 -- start start 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837673bb80 0x5583766ee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x55837671e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671e560 0x5583766f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558376657330 con 0x55837671e560 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5583765ffab0 con 0x55837673bb80 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5583766532f0 con 0x55837671eca0 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03618a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671e560 0x5583766f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03618a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671e560 0x5583766f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39564/0 (socket says 172.21.15.113:39564) 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.024+0000 7fe03618a640 1 -- 172.21.15.113:0/3839528672 learned_addr learned my addr 172.21.15.113:0/3839528672 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe035188640 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x55837671e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe035989640 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837673bb80 0x5583766ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe03618a640 1 -- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 msgr2=0x55837671e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe03618a640 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x55837671e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe03618a640 1 -- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837673bb80 msgr2=0x5583766ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe03618a640 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837673bb80 0x5583766ee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe03618a640 1 -- 172.21.15.113:0/3839528672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583764357d0 con 0x55837671e560 2024-04-25T00:05:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe035989640 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837673bb80 0x5583766ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe03618a640 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671e560 0x5583766f3370 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0099a0 tx=0x7fe02c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b81dc51a7496ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe0137fe640 1 -- 172.21.15.113:0/3839528672 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c017070 con 0x55837671e560 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe0137fe640 1 -- 172.21.15.113:0/3839528672 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c00ea80 con 0x55837671e560 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.025+0000 7fe0137fe640 1 -- 172.21.15.113:0/3839528672 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00ee50 con 0x55837671e560 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3839528672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671e560 msgr2=0x5583766f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671e560 0x5583766f3370 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0099a0 tx=0x7fe02c009e60 comp rx=0 tx=0).stop 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3839528672 shutdown_connections 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671e560 0x5583766f3370 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x55837671e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3839528672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837673bb80 0x5583766ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3839528672 >> 172.21.15.113:0/3839528672 conn(0x5583766f0780 msgr2=0x558376739020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3839528672 shutdown_connections 2024-04-25T00:05:07.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3839528672 wait complete. 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 -- start start 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.026+0000 7fe03a9cf740 1 Processor -- start 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671eca0 0x55837671d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837673bb80 0x55837673ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837670a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5583766f2930 con 0x55837671eca0 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558376612f40 con 0x55837671fba0 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5583766f1d20 con 0x55837673bb80 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03a9cf740 10 client.0 _ll_register_callbacks cb 0x55837636abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe035989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671eca0 0x55837671d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe035989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671eca0 0x55837671d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39572/0 (socket says 172.21.15.113:39572) 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe035989640 1 -- 172.21.15.113:0/3901728991 learned_addr learned my addr 172.21.15.113:0/3901728991 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe03618a640 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837670a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.027+0000 7fe035188640 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837673bb80 0x55837673ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe035989640 1 -- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837673bb80 msgr2=0x55837673ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe035989640 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837673bb80 0x55837673ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe035989640 1 -- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 msgr2=0x55837670a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe035989640 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837670a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe035989640 1 -- 172.21.15.113:0/3901728991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837632b0f0 con 0x55837671eca0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe035989640 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671eca0 0x55837671d0f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe028013a80 tx=0x7fe028001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcdc733f66dee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe010ff9640 1 -- 172.21.15.113:0/3901728991 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028027060 con 0x55837671eca0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe010ff9640 1 -- 172.21.15.113:0/3901728991 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fe028012ae0 con 0x55837671eca0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02c00ea80 con 0x55837671eca0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe010ff9640 1 -- 172.21.15.113:0/3901728991 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe028012ce0 con 0x55837671eca0 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 10 client.8037 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x558376495f60 con 0x55837671eca0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe010ff9640 1 -- 172.21.15.113:0/3901728991 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c00ea80 con 0x55837671eca0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe010ff9640 1 -- 172.21.15.113:0/3901728991 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7fe02802c3b0 con 0x55837671eca0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 10 client.8037 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 10 client.8037 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 20 client.8037 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837670b7d0 0x558376739020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 4 client.8037 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558376664870 con 0x55837670b7d0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.028+0000 7fe035188640 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837670b7d0 0x558376739020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe010ff9640 10 client.8037 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe035188640 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837670b7d0 0x558376739020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f147eaf28c07cb3a server_cookie=f8c6621e6d91c9a2 in_seq=0 out_seq=0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe010ff9640 1 -- 172.21.15.113:0/3901728991 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558376664870 con 0x55837670b7d0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe010ff9640 10 client.8037 handle_command_reply: tid=0 2024-04-25T00:05:07.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 client.8037 shutdown 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671eca0 msgr2=0x55837671d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671eca0 0x55837671d0f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe028013a80 tx=0x7fe028001250 comp rx=0 tx=0).stop 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 shutdown_connections 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837670b7d0 0x558376739020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837670a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837673bb80 0x55837673ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe010ff9640 0 client.8037 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3901728991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837671eca0 0x55837671d0f0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.029+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 >> 172.21.15.113:0/3901728991 conn(0x55837671e560 msgr2=0x55837673c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.030+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 shutdown_connections 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.030+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3901728991 wait complete. 2024-04-25T00:05:07.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.030+0000 7fe03a9cf740 20 client.8037 trim_cache size 0 max 16384 2024-04-25T00:05:07.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 Processor -- start 2024-04-25T00:05:07.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 -- start start 2024-04-25T00:05:07.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671eca0 0x558376752ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671fba0 0x55837673a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x55837675be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe030200830 con 0x55837673bb80 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301ff890 con 0x55837671eca0 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.047+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe03015b000 con 0x55837671fba0 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x55837675be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x55837675be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39576/0 (socket says 172.21.15.113:39576) 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 -- 172.21.15.113:0/2304292148 learned_addr learned my addr 172.21.15.113:0/2304292148 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe035188640 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671fba0 0x55837673a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe035989640 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671eca0 0x558376752ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 -- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671fba0 msgr2=0x55837673a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671fba0 0x55837673a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 -- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671eca0 msgr2=0x558376752ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671eca0 0x558376752ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 -- 172.21.15.113:0/2304292148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583763041b0 con 0x55837673bb80 2024-04-25T00:05:07.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe03618a640 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x55837675be90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00e5a0 tx=0x7fe02c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d4e0be208dfc24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe012ffd640 1 -- 172.21.15.113:0/2304292148 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c017040 con 0x55837673bb80 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe012ffd640 1 -- 172.21.15.113:0/2304292148 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c00eb40 con 0x55837673bb80 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.048+0000 7fe012ffd640 1 -- 172.21.15.113:0/2304292148 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c005920 con 0x55837673bb80 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 -- 172.21.15.113:0/2304292148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 msgr2=0x55837675be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x55837675be90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00e5a0 tx=0x7fe02c009f90 comp rx=0 tx=0).stop 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 -- 172.21.15.113:0/2304292148 shutdown_connections 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x55837675be90 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671fba0 0x55837673a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/2304292148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671eca0 0x558376752ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 -- 172.21.15.113:0/2304292148 >> 172.21.15.113:0/2304292148 conn(0x55837671e560 msgr2=0x5583766f3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 -- 172.21.15.113:0/2304292148 shutdown_connections 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 -- 172.21.15.113:0/2304292148 wait complete. 2024-04-25T00:05:07.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 -- start start 2024-04-25T00:05:07.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.049+0000 7fe03a9cf740 1 Processor -- start 2024-04-25T00:05:07.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x558376739090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837671c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03a9cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x5583766edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5583766f1d20 con 0x55837673bb80 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe03015b650 con 0x55837671fba0 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03a9cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5583766f4fe0 con 0x55837671eca0 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03a9cf740 10 client.0 _ll_register_callbacks cb 0x7fe028004e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03618a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x5583766edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03618a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x5583766edc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39580/0 (socket says 172.21.15.113:39580) 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe03618a640 1 -- 172.21.15.113:0/3456088569 learned_addr learned my addr 172.21.15.113:0/3456088569 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe035188640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837671c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.050+0000 7fe035989640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x558376739090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03618a640 1 -- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 msgr2=0x558376739090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03618a640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x558376739090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03618a640 1 -- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 msgr2=0x55837671c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03618a640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837671c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03618a640 1 -- 172.21.15.113:0/3456088569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583767395d0 con 0x55837673bb80 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe035188640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837671c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03618a640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x5583766edc50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe02c000c00 tx=0x7fe02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d976a5e913359fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe010ff9640 1 -- 172.21.15.113:0/3456088569 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c02a070 con 0x55837673bb80 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe010ff9640 1 -- 172.21.15.113:0/3456088569 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fe02c006040 con 0x55837673bb80 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02c00eb40 con 0x55837673bb80 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe010ff9640 1 -- 172.21.15.113:0/3456088569 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe02c035050 con 0x55837673bb80 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 10 client.8043 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x5583767b5d20 con 0x55837673bb80 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe010ff9640 1 -- 172.21.15.113:0/3456088569 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c00eb40 con 0x55837673bb80 2024-04-25T00:05:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe010ff9640 1 -- 172.21.15.113:0/3456088569 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7fe02c038020 con 0x55837673bb80 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 10 client.8043 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 10 client.8043 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 20 client.8043 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837671cb10 0x5583766ee190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 4 client.8043 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55837665ebc0 con 0x55837671cb10 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.051+0000 7fe035989640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837671cb10 0x5583766ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe010ff9640 10 client.8043 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe035989640 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837671cb10 0x5583766ee190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d4b156fe3c22af server_cookie=4dcf25ae7e2a865a in_seq=0 out_seq=0 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe010ff9640 1 -- 172.21.15.113:0/3456088569 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55837665ebc0 con 0x55837671cb10 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe010ff9640 10 client.8043 handle_command_reply: tid=0 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 client.8043 shutdown 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 msgr2=0x5583766edc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x5583766edc50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe02c000c00 tx=0x7fe02c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 shutdown_connections 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55837671cb10 0x5583766ee190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55837671fba0 0x55837671c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe010ff9640 0 client.8043 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55837673bb80 0x5583766edc50 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 --2- 172.21.15.113:0/3456088569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55837671eca0 0x558376739090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 >> 172.21.15.113:0/3456088569 conn(0x55837671e560 msgr2=0x558376752d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 shutdown_connections 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 1 -- 172.21.15.113:0/3456088569 wait complete. 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.052+0000 7fe03a9cf740 20 client.8043 trim_cache size 0 max 16384 2024-04-25T00:05:07.059 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:07.060 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:05:07.060 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:07.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01406be30 msgr2=0x7fe01406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01406be30 0x7fe01406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe0280012e0 tx=0x7fe028001700 comp rx=0 tx=0).stop 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b73c0 msgr2=0x7fe0300c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b73c0 0x7fe0300c9d40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe020009890 tx=0x7fe0200046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 shutdown_connections 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe01406be30 0x7fe01406e2f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe03015e510 0x7fe03015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0300b73c0 0x7fe0300c9d40 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 --2- 172.21.15.113:0/2385695025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0300b5ca0 0x7fe0300c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 >> 172.21.15.113:0/2385695025 conn(0x7fe03006b720 msgr2=0x7fe030155880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 shutdown_connections 2024-04-25T00:05:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.053+0000 7fe037bed640 1 -- 172.21.15.113:0/2385695025 wait complete. 2024-04-25T00:05:07.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:07.064 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T00:05:07.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.162+0000 7f6dcfbed640 1 Processor -- start 2024-04-25T00:05:07.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.162+0000 7f6dcfbed640 1 -- start start 2024-04-25T00:05:07.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcfbed640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcfbed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc80ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcfbed640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc815f4f0 con 0x7f6dc80b3080 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcfbed640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc80cde00 con 0x7f6dc815e2d0 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcfbed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc815f7d0 con 0x7f6dc80b4a40 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dce18a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc80ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd989640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd989640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:39980/0 (socket says 172.21.15.113:39980) 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dce18a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc80ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:59490/0 (socket says 172.21.15.113:59490) 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd989640 1 -- 172.21.15.113:0/892748331 learned_addr learned my addr 172.21.15.113:0/892748331 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd188640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6dc80b3080 0x7f6dc80b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:49902/0 (socket says 172.21.15.113:49902) 2024-04-25T00:05:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd989640 1 -- 172.21.15.113:0/892748331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 msgr2=0x7f6dc80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd989640 1 --2- 172.21.15.113:0/892748331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc80ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd989640 1 -- 172.21.15.113:0/892748331 >> v1:172.21.15.178:6789/0 conn(0x7f6dc80b3080 legacy=0x7f6dc80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcd989640 1 -- 172.21.15.113:0/892748331 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc815fab0 con 0x7f6dc815e2d0 2024-04-25T00:05:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.163+0000 7f6dcc987640 1 -- 172.21.15.113:0/892748331 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2360073832 0 0) 0x7f6dc815f4f0 con 0x7f6dc80b3080 2024-04-25T00:05:07.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.164+0000 7f6dcd989640 1 --2- 172.21.15.113:0/892748331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 0x7f6dc815ebf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000bb10 tx=0x7f6dc002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc5deee1c4a52e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.164+0000 7f6dcc987640 1 -- 172.21.15.113:0/892748331 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0036020 con 0x7f6dc815e2d0 2024-04-25T00:05:07.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.164+0000 7f6dcc987640 1 -- 172.21.15.113:0/892748331 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0032820 con 0x7f6dc815e2d0 2024-04-25T00:05:07.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.164+0000 7f6dcc987640 1 -- 172.21.15.113:0/892748331 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0032bf0 con 0x7f6dc815e2d0 2024-04-25T00:05:07.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.164+0000 7f6dcfbed640 1 -- 172.21.15.113:0/892748331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 msgr2=0x7f6dc815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.164+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/892748331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 0x7f6dc815ebf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000bb10 tx=0x7f6dc002c920 comp rx=0 tx=0).stop 2024-04-25T00:05:07.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.165+0000 7f6dcfbed640 1 -- 172.21.15.113:0/892748331 shutdown_connections 2024-04-25T00:05:07.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.165+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/892748331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc80ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.165+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/892748331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 0x7f6dc815ebf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.165+0000 7f6dcfbed640 1 -- 172.21.15.113:0/892748331 >> 172.21.15.113:0/892748331 conn(0x7f6dc806b0f0 msgr2=0x7f6dc806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.165+0000 7f6dcfbed640 1 -- 172.21.15.113:0/892748331 shutdown_connections 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.165+0000 7f6dcfbed640 1 -- 172.21.15.113:0/892748331 wait complete. 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 Processor -- start 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 -- start start 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc80b3080 0x7f6dc8142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc8152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 0x7f6dc80b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc815f2b0 con 0x7f6dc80b3080 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc815f130 con 0x7f6dc815e2d0 2024-04-25T00:05:07.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.166+0000 7f6dcfbed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc815e6b0 con 0x7f6dc80b4a40 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc80b3080 0x7f6dc8142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc80b3080 0x7f6dc8142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39584/0 (socket says 172.21.15.113:39584) 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 -- 172.21.15.113:0/3759602280 learned_addr learned my addr 172.21.15.113:0/3759602280 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd188640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc8152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dce18a640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 0x7f6dc80b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 -- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 msgr2=0x7f6dc8152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc8152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 -- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 msgr2=0x7f6dc80b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 0x7f6dc80b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 -- 172.21.15.113:0/3759602280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc81501e0 con 0x7f6dc80b3080 2024-04-25T00:05:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcd989640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc80b3080 0x7f6dc8142bd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0035040 tx=0x7f6dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/3759602280 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0036040 con 0x7f6dc80b3080 2024-04-25T00:05:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/3759602280 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc003bb50 con 0x7f6dc80b3080 2024-04-25T00:05:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc8152f10 con 0x7f6dc80b3080 2024-04-25T00:05:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/3759602280 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0004030 con 0x7f6dc80b3080 2024-04-25T00:05:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.167+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc8150780 con 0x7f6dc80b3080 2024-04-25T00:05:07.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.169+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/3759602280 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dc004c060 con 0x7f6dc80b3080 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.169+0000 7f6dbe7fc640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.169+0000 7f6dbe7fc640 1 -- 172.21.15.113:0/3759602280 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f6dc00ada90 con 0x7f6dc80b3080 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.169+0000 7f6dcd188640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.170+0000 7f6dcd188640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80b2320 tx=0x7f6db8007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.170+0000 7f6dd29cf740 1 Processor -- start 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.170+0000 7f6dd29cf740 1 -- start start 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a22847140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22864600 0x561a22864b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a2281ca40 0x561a22862cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561a227806b0 con 0x561a22864600 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561a22728880 con 0x561a2281ca40 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561a2277c670 con 0x561a22846d60 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dcd188640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22864600 0x561a22864b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dcd188640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22864600 0x561a22864b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39588/0 (socket says 172.21.15.113:39588) 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dcd188640 1 -- 172.21.15.113:0/2607503670 learned_addr learned my addr 172.21.15.113:0/2607503670 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dcd188640 1 -- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 msgr2=0x561a22847140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dcd989640 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a22847140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dcd188640 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a22847140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dcd188640 1 -- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a2281ca40 msgr2=0x561a22862cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.171+0000 7f6dce18a640 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a2281ca40 0x561a22862cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dcd188640 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a2281ca40 0x561a22862cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dcd188640 1 -- 172.21.15.113:0/2607503670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2255e7d0 con 0x561a22864600 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dcd989640 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a22847140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dce18a640 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a2281ca40 0x561a22862cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dcd188640 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22864600 0x561a22864b50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f6db80011e0 tx=0x7f6db8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff39817833eb1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dab7fe640 1 -- 172.21.15.113:0/2607503670 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8026070 con 0x561a22864600 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dab7fe640 1 -- 172.21.15.113:0/2607503670 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8013b80 con 0x561a22864600 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dab7fe640 1 -- 172.21.15.113:0/2607503670 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8012350 con 0x561a22864600 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2607503670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22864600 msgr2=0x561a22864b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22864600 0x561a22864b50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f6db80011e0 tx=0x7f6db8013000 comp rx=0 tx=0).stop 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2607503670 shutdown_connections 2024-04-25T00:05:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a2281ca40 0x561a22862cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22864600 0x561a22864b50 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2607503670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a22847140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.172+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2607503670 >> 172.21.15.113:0/2607503670 conn(0x561a22819600 msgr2=0x561a22861b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.173+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2607503670 shutdown_connections 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.173+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2607503670 wait complete. 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.173+0000 7f6dd29cf740 1 -- start start 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.173+0000 7f6dd29cf740 1 Processor -- start 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a228606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22864600 0x561a228754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22849b50 0x561a22816260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561a2281d7e0 con 0x561a22846d60 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561a2273bf50 con 0x561a22864600 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561a22817530 con 0x561a22849b50 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dd29cf740 10 client.0 _ll_register_callbacks cb 0x561a22777000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a228606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a228606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39590/0 (socket says 172.21.15.113:39590) 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 -- 172.21.15.113:0/2970878390 learned_addr learned my addr 172.21.15.113:0/2970878390 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dce18a640 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22849b50 0x561a22816260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd188640 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22864600 0x561a228754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 -- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22849b50 msgr2=0x561a22816260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22849b50 0x561a22816260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 -- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22864600 msgr2=0x561a228754d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22864600 0x561a228754d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.174+0000 7f6dcd989640 1 -- 172.21.15.113:0/2970878390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a224943f0 con 0x561a22846d60 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dcd989640 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a228606d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000b2c0 tx=0x7f6dc007b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b67789abea1401a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6da8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6da8ff9640 1 -- 172.21.15.113:0/2970878390 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00a9020 con 0x561a22846d60 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db8013b80 con 0x561a22846d60 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6da8ff9640 1 -- 172.21.15.113:0/2970878390 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f6dc00043d0 con 0x561a22846d60 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6da8ff9640 1 -- 172.21.15.113:0/2970878390 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f6dc009c020 con 0x561a22846d60 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 10 client.8052 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x561a224540f0 con 0x561a22846d60 2024-04-25T00:05:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6da8ff9640 1 -- 172.21.15.113:0/2970878390 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8013b80 con 0x561a22846d60 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6da8ff9640 1 -- 172.21.15.113:0/2970878390 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7f6dc000e040 con 0x561a22846d60 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 10 client.8052 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 10 client.8052 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 20 client.8052 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a2285a200 0x561a2285a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 4 client.8052 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2278dbf0 con 0x561a2285a200 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.175+0000 7f6dcd188640 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a2285a200 0x561a2285a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6da8ff9640 10 client.8052 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6dcd188640 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a2285a200 0x561a2285a610 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d81bc874afc325d server_cookie=3de00c1988f59049 in_seq=0 out_seq=0 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6da8ff9640 1 -- 172.21.15.113:0/2970878390 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a2278dbf0 con 0x561a2285a200 2024-04-25T00:05:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6da8ff9640 10 client.8052 handle_command_reply: tid=0 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6dd29cf740 1 client.8052 shutdown 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 msgr2=0x561a228606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a228606d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000b2c0 tx=0x7f6dc007b8a0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 shutdown_connections 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.176+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a2285a200 0x561a2285a610 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6da8ff9640 0 client.8052 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22864600 0x561a228754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22849b50 0x561a22816260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/2970878390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a228606d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 >> 172.21.15.113:0/2970878390 conn(0x561a2281ca40 msgr2=0x561a228650f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 shutdown_connections 2024-04-25T00:05:07.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6dd29cf740 1 -- 172.21.15.113:0/2970878390 wait complete. 2024-04-25T00:05:07.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.177+0000 7f6dd29cf740 20 client.8052 trim_cache size 0 max 16384 2024-04-25T00:05:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.194+0000 7f6dd29cf740 1 Processor -- start 2024-04-25T00:05:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.194+0000 7f6dd29cf740 1 -- start start 2024-04-25T00:05:07.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a22861b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a228620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a2285a200 0x561a2284a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81fc350 con 0x561a22846d60 2024-04-25T00:05:07.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8151990 con 0x561a22849b50 2024-04-25T00:05:07.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc8150cf0 con 0x561a2285a200 2024-04-25T00:05:07.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a22861b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a22861b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39592/0 (socket says 172.21.15.113:39592) 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 -- 172.21.15.113:0/1121326536 learned_addr learned my addr 172.21.15.113:0/1121326536 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dce18a640 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a2285a200 0x561a2284a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd188640 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a228620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 -- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a2285a200 msgr2=0x561a2284a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a2285a200 0x561a2284a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 -- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 msgr2=0x561a228620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a228620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 -- 172.21.15.113:0/1121326536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a228165d0 con 0x561a22846d60 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6dcd989640 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a22861b70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00849d0 tx=0x7f6dc007e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b92b5f7cb9b846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6daaffd640 1 -- 172.21.15.113:0/1121326536 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00a9040 con 0x561a22846d60 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6daaffd640 1 -- 172.21.15.113:0/1121326536 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0032070 con 0x561a22846d60 2024-04-25T00:05:07.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.195+0000 7f6daaffd640 1 -- 172.21.15.113:0/1121326536 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00043f0 con 0x561a22846d60 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 -- 172.21.15.113:0/1121326536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 msgr2=0x561a22861b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a22861b70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00849d0 tx=0x7f6dc007e000 comp rx=0 tx=0).stop 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 -- 172.21.15.113:0/1121326536 shutdown_connections 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a228620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a2285a200 0x561a2284a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/1121326536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a22846d60 0x561a22861b70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 -- 172.21.15.113:0/1121326536 >> 172.21.15.113:0/1121326536 conn(0x561a2281ca40 msgr2=0x561a22861420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 -- 172.21.15.113:0/1121326536 shutdown_connections 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 -- 172.21.15.113:0/1121326536 wait complete. 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.196+0000 7f6dd29cf740 1 -- start start 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 1 Processor -- start 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a228dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a2284a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a2285a200 0x561a2284ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561a22849910 con 0x561a2285a200 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8151340 con 0x561a22849b50 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561a2284a4d0 con 0x561a22846d60 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.197+0000 7f6dd29cf740 10 client.0 _ll_register_callbacks cb 0x7f6dc007bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a2285a200 0x561a2284ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a2285a200 0x561a2284ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39600/0 (socket says 172.21.15.113:39600) 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 -- 172.21.15.113:0/3789267546 learned_addr learned my addr 172.21.15.113:0/3789267546 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dcd989640 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a228dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dcd188640 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a2284a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 -- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 msgr2=0x561a228dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a228dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 -- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 msgr2=0x561a2284a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a2284a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 -- 172.21.15.113:0/3789267546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2285fcb0 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dce18a640 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a2285a200 0x561a2284ab90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4004660 tx=0x7f6dc4017780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceff009b53092444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6da8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6da8ff9640 1 -- 172.21.15.113:0/3789267546 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4017e70 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6da8ff9640 1 -- 172.21.15.113:0/3789267546 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f6dc4022ae0 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc0032070 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6da8ff9640 1 -- 172.21.15.113:0/3789267546 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f6dc4022ce0 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 10 client.8058 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x561a2285fee0 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6da8ff9640 1 -- 172.21.15.113:0/3789267546 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0032070 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6da8ff9640 1 -- 172.21.15.113:0/3789267546 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7f6dc4029050 con 0x561a2285a200 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 10 client.8058 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 10 client.8058 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:07.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 20 client.8058 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a22864600 0x561a22858570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 4 client.8058 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.198+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a22787f40 con 0x561a22864600 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dcd989640 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a22864600 0x561a22858570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6da8ff9640 10 client.8058 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dcd989640 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a22864600 0x561a22858570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59843e968c6e575 server_cookie=711a39d39d2dade5 in_seq=0 out_seq=0 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6da8ff9640 1 -- 172.21.15.113:0/3789267546 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+276 (crc 0 0 0) 0x561a22787f40 con 0x561a22864600 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6da8ff9640 10 client.8058 handle_command_reply: tid=0 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 client.8058 shutdown 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a2285a200 msgr2=0x561a2284ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a2285a200 0x561a2284ab90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4004660 tx=0x7f6dc4017780 comp rx=0 tx=0).stop 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 shutdown_connections 2024-04-25T00:05:07.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x561a22864600 0x561a22858570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561a22849b50 0x561a2284a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561a2285a200 0x561a2284ab90 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6da8ff9640 0 client.8058 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 --2- 172.21.15.113:0/3789267546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561a22846d60 0x561a228dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 >> 172.21.15.113:0/3789267546 conn(0x561a2281ca40 msgr2=0x561a22861e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 shutdown_connections 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 1 -- 172.21.15.113:0/3789267546 wait complete. 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.199+0000 7f6dd29cf740 20 client.8058 trim_cache size 0 max 16384 2024-04-25T00:05:07.207 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bfa02794-118a-4fbc-8b32-9fb5063d3d27": { 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_control_test_path", 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "bfa02794-118a-4fbc-8b32-9fb5063d3d27", 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive" 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:07.208 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:07.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 msgr2=0x7f6dac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80b2320 tx=0x7f6db8007d50 comp rx=0 tx=0).stop 2024-04-25T00:05:07.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc80b3080 msgr2=0x7f6dc8142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc80b3080 0x7f6dc8142bd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0035040 tx=0x7f6dc00046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 shutdown_connections 2024-04-25T00:05:07.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6dac06be30 0x7f6dac06e2f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc815e2d0 0x7f6dc80b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6dc80b4a40 0x7f6dc8152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 --2- 172.21.15.113:0/3759602280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc80b3080 0x7f6dc8142bd0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 >> 172.21.15.113:0/3759602280 conn(0x7f6dc806b0f0 msgr2=0x7f6dc80b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 shutdown_connections 2024-04-25T00:05:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.200+0000 7f6dcfbed640 1 -- 172.21.15.113:0/3759602280 wait complete. 2024-04-25T00:05:07.211 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.307+0000 7fb151c37640 1 Processor -- start 2024-04-25T00:05:07.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.307+0000 7fb151c37640 1 -- start start 2024-04-25T00:05:07.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb151c37640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb14c0cf130 0x7fb14c0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb151c37640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb151c37640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb14c0cf510 con 0x7fb14c061b80 2024-04-25T00:05:07.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb151c37640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb14c15ecd0 con 0x7fb14c0cf130 2024-04-25T00:05:07.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb151c37640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c15efb0 con 0x7fb14c15e810 2024-04-25T00:05:07.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb14affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb14b7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb14c0cf130 0x7fb14c0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb14affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:59546/0 (socket says 172.21.15.113:59546) 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb14affd640 1 -- 172.21.15.113:0/1470552085 learned_addr learned my addr 172.21.15.113:0/1470552085 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.308+0000 7fb14bfff640 1 --1- 172.21.15.113:0/1470552085 >> v1:172.21.15.178:6789/0 conn(0x7fb14c061b80 0x7fb14c06b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:49910/0 (socket says 172.21.15.113:49910) 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.309+0000 7fb14b7fe640 1 -- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 msgr2=0x7fb14c0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.309+0000 7fb14b7fe640 1 --2- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.309+0000 7fb14b7fe640 1 -- 172.21.15.113:0/1470552085 >> v1:172.21.15.178:6789/0 conn(0x7fb14c061b80 legacy=0x7fb14c06b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.309+0000 7fb14b7fe640 1 -- 172.21.15.113:0/1470552085 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c15f290 con 0x7fb14c0cf130 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.309+0000 7fb14a7fc640 1 -- 172.21.15.113:0/1470552085 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1610057218 0 0) 0x7fb14c0cf510 con 0x7fb14c061b80 2024-04-25T00:05:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.309+0000 7fb14affd640 1 --2- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.309+0000 7fb14b7fe640 1 --2- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c5e90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0097a0 tx=0x7fb13c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d7da5e29235defe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb14a7fc640 1 -- 172.21.15.113:0/1470552085 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c037070 con 0x7fb14c0cf130 2024-04-25T00:05:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb14a7fc640 1 -- 172.21.15.113:0/1470552085 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13c032440 con 0x7fb14c0cf130 2024-04-25T00:05:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb14a7fc640 1 -- 172.21.15.113:0/1470552085 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c032810 con 0x7fb14c0cf130 2024-04-25T00:05:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 -- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c0cf130 msgr2=0x7fb14c0c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 --2- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c5e90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0097a0 tx=0x7fb13c02a180 comp rx=0 tx=0).stop 2024-04-25T00:05:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 -- 172.21.15.113:0/1470552085 shutdown_connections 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 --2- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 --2- 172.21.15.113:0/1470552085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c5e90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 -- 172.21.15.113:0/1470552085 >> 172.21.15.113:0/1470552085 conn(0x7fb14c0cd860 msgr2=0x7fb14c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 -- 172.21.15.113:0/1470552085 shutdown_connections 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.310+0000 7fb151c37640 1 -- 172.21.15.113:0/1470552085 wait complete. 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.311+0000 7fb151c37640 1 Processor -- start 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.311+0000 7fb151c37640 1 -- start start 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb151c37640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c061b80 0x7fb14c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb151c37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb151c37640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c1f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb151c37640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0c7c30 con 0x7fb14c0cf130 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb151c37640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0c7ab0 con 0x7fb14c061b80 2024-04-25T00:05:07.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb151c37640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c062010 con 0x7fb14c15e810 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb14affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb14affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39608/0 (socket says 172.21.15.113:39608) 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb14affd640 1 -- 172.21.15.113:0/409213733 learned_addr learned my addr 172.21.15.113:0/409213733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb14b7fe640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c061b80 0x7fb14c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb14bfff640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c1f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.312+0000 7fb14affd640 1 -- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 msgr2=0x7fb14c1f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb14affd640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c1f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb14affd640 1 -- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c061b80 msgr2=0x7fb14c0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb14affd640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c061b80 0x7fb14c0ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb14affd640 1 -- 172.21.15.113:0/409213733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c0af680 con 0x7fb14c0cf130 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb14b7fe640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c061b80 0x7fb14c0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:07.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb14bfff640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c1f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb14affd640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c8150 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9e0 tx=0x7fb14000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb133fff640 1 -- 172.21.15.113:0/409213733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140017070 con 0x7fb14c0cf130 2024-04-25T00:05:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb133fff640 1 -- 172.21.15.113:0/409213733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14000ca30 con 0x7fb14c0cf130 2024-04-25T00:05:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb133fff640 1 -- 172.21.15.113:0/409213733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14000ce20 con 0x7fb14c0cf130 2024-04-25T00:05:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c0aede0 con 0x7fb14c0cf130 2024-04-25T00:05:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.313+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c0af3a0 con 0x7fb14c0cf130 2024-04-25T00:05:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.314+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c0b1090 con 0x7fb14c0cf130 2024-04-25T00:05:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.314+0000 7fb133fff640 1 -- 172.21.15.113:0/409213733 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb14001a070 con 0x7fb14c0cf130 2024-04-25T00:05:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.315+0000 7fb133fff640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb13806be30 0x7fb13806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.315+0000 7fb133fff640 1 -- 172.21.15.113:0/409213733 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fb14008b080 con 0x7fb14c0cf130 2024-04-25T00:05:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.315+0000 7fb14b7fe640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb13806be30 0x7fb13806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.315+0000 7fb14b7fe640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb13806be30 0x7fb13806e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb13c034000 tx=0x7fb13c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.319+0000 7fb133fff640 1 -- 172.21.15.113:0/409213733 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb14c0b1090 con 0x7fb14c0cf130 2024-04-25T00:05:07.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.530+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb14c0691a0 con 0x7fb14c0cf130 2024-04-25T00:05:07.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.531+0000 7fb133fff640 1 -- 172.21.15.113:0/409213733 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7fb1400593f0 con 0x7fb14c0cf130 2024-04-25T00:05:07.532 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:07.532 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:07.532 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb13806be30 msgr2=0x7fb13806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb13806be30 0x7fb13806e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb13c034000 tx=0x7fb13c02e000 comp rx=0 tx=0).stop 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb14c0cf130 msgr2=0x7fb14c0c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c8150 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9e0 tx=0x7fb14000eea0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 shutdown_connections 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb13806be30 0x7fb13806e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb14c15e810 0x7fb14c1f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb14c0cf130 0x7fb14c0c8150 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 --2- 172.21.15.113:0/409213733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb14c061b80 0x7fb14c0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 >> 172.21.15.113:0/409213733 conn(0x7fb14c0cd860 msgr2=0x7fb14c0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 shutdown_connections 2024-04-25T00:05:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.533+0000 7fb151c37640 1 -- 172.21.15.113:0/409213733 wait complete. 2024-04-25T00:05:07.544 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:07.545 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:07.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.640+0000 7f3b78589640 1 Processor -- start 2024-04-25T00:05:07.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.640+0000 7f3b78589640 1 -- start start 2024-04-25T00:05:07.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b78589640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7015e320 0x7f3b700d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b78589640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b700d0b00 0x7f3b700d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b78589640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b700d3de0 con 0x7f3b700d2290 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b78589640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3b700d14d0 con 0x7f3b700d0b00 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b78589640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3b700d3410 con 0x7f3b7015e320 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b76b26640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b700d0b00 0x7f3b700d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b75b24640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3b700d2290 0x7f3b700b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47338/0 (socket says 172.21.15.113:47338) 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.641+0000 7f3b75b24640 1 -- 172.21.15.113:0/713703401 learned_addr learned my addr 172.21.15.113:0/713703401 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b76325640 1 --2- 172.21.15.113:0/713703401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7015e320 0x7f3b700d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b76b26640 1 -- 172.21.15.113:0/713703401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7015e320 msgr2=0x7f3b700d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b76b26640 1 --2- 172.21.15.113:0/713703401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7015e320 0x7f3b700d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b76b26640 1 -- 172.21.15.113:0/713703401 >> v1:172.21.15.113:6789/0 conn(0x7f3b700d2290 legacy=0x7f3b700b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b75323640 1 -- 172.21.15.113:0/713703401 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970310881 0 0) 0x7f3b700d3de0 con 0x7f3b700d2290 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b76b26640 1 -- 172.21.15.113:0/713703401 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b700d2800 con 0x7f3b700d0b00 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b76b26640 1 --2- 172.21.15.113:0/713703401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b700d0b00 0x7f3b700d0f90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3b680097a0 tx=0x7f3b6802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c903755ee07770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.642+0000 7f3b75323640 1 -- 172.21.15.113:0/713703401 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68037070 con 0x7f3b700d0b00 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b75323640 1 -- 172.21.15.113:0/713703401 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b68032440 con 0x7f3b700d0b00 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b75323640 1 -- 172.21.15.113:0/713703401 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68032810 con 0x7f3b700d0b00 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 -- 172.21.15.113:0/713703401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b700d0b00 msgr2=0x7f3b700d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 --2- 172.21.15.113:0/713703401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b700d0b00 0x7f3b700d0f90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3b680097a0 tx=0x7f3b6802a210 comp rx=0 tx=0).stop 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 -- 172.21.15.113:0/713703401 shutdown_connections 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 --2- 172.21.15.113:0/713703401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b700d0b00 0x7f3b700d0f90 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 --2- 172.21.15.113:0/713703401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7015e320 0x7f3b700d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 -- 172.21.15.113:0/713703401 >> 172.21.15.113:0/713703401 conn(0x7f3b700b5c20 msgr2=0x7f3b700b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 -- 172.21.15.113:0/713703401 shutdown_connections 2024-04-25T00:05:07.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.643+0000 7f3b78589640 1 -- 172.21.15.113:0/713703401 wait complete. 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.644+0000 7f3b78589640 1 Processor -- start 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.644+0000 7f3b78589640 1 -- start start 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b78589640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b700d0b00 0x7f3b70152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b78589640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b700d2290 0x7f3b700b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b78589640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b7015e320 0x7f3b700b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b78589640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3b700d30a0 con 0x7f3b7015e320 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b78589640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b700d2dc0 con 0x7f3b700d2290 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b78589640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7015e700 con 0x7f3b700d0b00 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b7015e320 0x7f3b700b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b7015e320 0x7f3b700b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40102/0 (socket says 172.21.15.113:40102) 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 -- 172.21.15.113:0/810871627 learned_addr learned my addr 172.21.15.113:0/810871627 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:07.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b75b24640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b700d2290 0x7f3b700b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76325640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b700d0b00 0x7f3b70152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 -- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b700d0b00 msgr2=0x7f3b70152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b700d0b00 0x7f3b70152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 -- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b700d2290 msgr2=0x7f3b700b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b700d2290 0x7f3b700b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 -- 172.21.15.113:0/810871627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b70153790 con 0x7f3b7015e320 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b75b24640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b700d2290 0x7f3b700b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.645+0000 7f3b76b26640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b7015e320 0x7f3b700b8930 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3b68035040 tx=0x7f3b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.646+0000 7f3b66ffd640 1 -- 172.21.15.113:0/810871627 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68037040 con 0x7f3b7015e320 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.646+0000 7f3b66ffd640 1 -- 172.21.15.113:0/810871627 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b68042070 con 0x7f3b7015e320 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.646+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b700b8f60 con 0x7f3b7015e320 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.646+0000 7f3b66ffd640 1 -- 172.21.15.113:0/810871627 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680308a0 con 0x7f3b7015e320 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.646+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b700b9520 con 0x7f3b7015e320 2024-04-25T00:05:07.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.647+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b700b9b30 con 0x7f3b7015e320 2024-04-25T00:05:07.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.647+0000 7f3b66ffd640 1 -- 172.21.15.113:0/810871627 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b68030a40 con 0x7f3b7015e320 2024-04-25T00:05:07.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.647+0000 7f3b66ffd640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b5406be30 0x7f3b5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:07.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.648+0000 7f3b76325640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b5406be30 0x7f3b5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:07.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.648+0000 7f3b66ffd640 1 -- 172.21.15.113:0/810871627 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f3b680ab6e0 con 0x7f3b7015e320 2024-04-25T00:05:07.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.648+0000 7f3b76325640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b5406be30 0x7f3b5406e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c011340 tx=0x7f3b6c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:07.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.652+0000 7f3b66ffd640 1 -- 172.21.15.113:0/810871627 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b6807a090 con 0x7f3b7015e320 2024-04-25T00:05:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.890+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3b700691a0 con 0x7f3b7015e320 2024-04-25T00:05:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.891+0000 7f3b66ffd640 1 -- 172.21.15.113:0/810871627 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1574 (secure 0 0 0) 0x7f3b68079990 con 0x7f3b7015e320 2024-04-25T00:05:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:07.891 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":988,"data_bytes":1013439650,"bytes_used":2374676480,"bytes_avail":770719436800,"bytes_total":773094113280,"write_bytes_sec":84398175,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T00:04:39.186609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b5406be30 msgr2=0x7f3b5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b5406be30 0x7f3b5406e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c011340 tx=0x7f3b6c00b040 comp rx=0 tx=0).stop 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b7015e320 msgr2=0x7f3b700b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b7015e320 0x7f3b700b8930 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3b68035040 tx=0x7f3b680046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 shutdown_connections 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3b5406be30 0x7f3b5406e2f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b700d2290 0x7f3b700b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b7015e320 0x7f3b700b8930 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 --2- 172.21.15.113:0/810871627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b700d0b00 0x7f3b70152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 >> 172.21.15.113:0/810871627 conn(0x7f3b700b5c20 msgr2=0x7f3b70071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 shutdown_connections 2024-04-25T00:05:07.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:07.893+0000 7f3b78589640 1 -- 172.21.15.113:0/810871627 wait complete. 2024-04-25T00:05:07.904 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:05:08.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.009+0000 7f433c902640 1 Processor -- start 2024-04-25T00:05:09.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.009+0000 7f433c902640 1 -- start start 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433c902640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f43381453c0 0x7f43380b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433c902640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43380cfad0 con 0x7f4338144d60 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433c902640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43380cf230 con 0x7f433806e6e0 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433c902640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4338145c40 con 0x7f43381453c0 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4338144d60 0x7f433806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47354/0 (socket says 172.21.15.113:47354) 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433659c640 1 -- 172.21.15.113:0/4048402929 learned_addr learned my addr 172.21.15.113:0/4048402929 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f4336d9d640 1 --2- 172.21.15.113:0/4048402929 >> v2:172.21.15.178:3300/0 conn(0x7f43381453c0 0x7f43380b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2008469320 0 0) 0x7f43380cfad0 con 0x7f4338144d60 2024-04-25T00:05:09.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4320003520 con 0x7f4338144d60 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938337196 0 0) 0x7f43380cf230 con 0x7f433806e6e0 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43380cfad0 con 0x7f433806e6e0 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3464529894 0 0) 0x7f4320003520 con 0x7f4338144d60 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 >> v2:172.21.15.178:3300/0 conn(0x7f43381453c0 msgr2=0x7f43380b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 --2- 172.21.15.113:0/4048402929 >> v2:172.21.15.178:3300/0 conn(0x7f43381453c0 0x7f43380b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 >> v1:172.21.15.178:6789/0 conn(0x7f433806e6e0 legacy=0x7f4338071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43380cf510 con 0x7f4338144d60 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f4328001670 con 0x7f4338144d60 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 >> v1:172.21.15.113:6789/0 conn(0x7f4338144d60 legacy=0x7f433806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4320003b90 0x7f4320003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.010+0000 7f433559a640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4320004800 0x7f4320004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433559a640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320005280 0x7f432000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4338145c40 con 0x7f4320003b90 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f432000c1c0 con 0x7f4320005280 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f432000c3c0 con 0x7f4320004800 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433659c640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4320003b90 0x7f4320003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f4335d9b640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4320004800 0x7f4320004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433659c640 1 -- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4320004800 msgr2=0x7f4320004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433659c640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4320004800 0x7f4320004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433659c640 1 -- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320005280 msgr2=0x7f432000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433659c640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320005280 0x7f432000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f433659c640 1 -- 172.21.15.113:0/4048402929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4320003520 con 0x7f4320003b90 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f4336d9d640 1 -- 172.21.15.113:0/4048402929 reap_dead start 2024-04-25T00:05:09.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.011+0000 7f4335d9b640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4320004800 0x7f4320004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433659c640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4320003b90 0x7f4320003f90 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f432800c0e0 tx=0x7f4320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdfd526b8c700ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328007c00 con 0x7f4320003b90 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4328005ad0 con 0x7f4320003b90 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433559a640 1 -- 172.21.15.113:0/4048402929 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328007070 con 0x7f4320003b90 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433c902640 1 -- 172.21.15.113:0/4048402929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4320003b90 msgr2=0x7f4320003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433c902640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4320003b90 0x7f4320003f90 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f432800c0e0 tx=0x7f4320005830 comp rx=0 tx=0).stop 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433c902640 1 -- 172.21.15.113:0/4048402929 shutdown_connections 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433c902640 1 --2- 172.21.15.113:0/4048402929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4320003b90 0x7f4320003f90 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433c902640 1 -- 172.21.15.113:0/4048402929 >> 172.21.15.113:0/4048402929 conn(0x7f43380b1a50 msgr2=0x7f43380b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433c902640 1 -- 172.21.15.113:0/4048402929 shutdown_connections 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.012+0000 7f433c902640 1 -- 172.21.15.113:0/4048402929 wait complete. 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 Processor -- start 2024-04-25T00:05:09.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 -- start start 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320003b90 0x7f433813c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4338158cf0 0x7f43380b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43380b4b50 0x7f43381564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4338145140 con 0x7f43380b4b50 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4338070e10 con 0x7f4320003b90 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.014+0000 7f433c902640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43380b5390 con 0x7f4338158cf0 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43380b4b50 0x7f43381564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43380b4b50 0x7f43381564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40126/0 (socket says 172.21.15.113:40126) 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 -- 172.21.15.113:0/447280900 learned_addr learned my addr 172.21.15.113:0/447280900 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f433659c640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320003b90 0x7f433813c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4335d9b640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4338158cf0 0x7f43380b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 -- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4338158cf0 msgr2=0x7f43380b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4338158cf0 0x7f43380b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 -- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320003b90 msgr2=0x7f433813c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320003b90 0x7f433813c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 -- 172.21.15.113:0/447280900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433813cd60 con 0x7f43380b4b50 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4336d9d640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43380b4b50 0x7f43381564c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f433000ea10 tx=0x7f433000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4326ffd640 1 -- 172.21.15.113:0/447280900 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330018020 con 0x7f43380b4b50 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4326ffd640 1 -- 172.21.15.113:0/447280900 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433000ba80 con 0x7f43380b4b50 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43380b3570 con 0x7f43380b4b50 2024-04-25T00:05:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f4326ffd640 1 -- 172.21.15.113:0/447280900 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330014400 con 0x7f43380b4b50 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.015+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43380b3b30 con 0x7f43380b4b50 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.016+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43380691a0 con 0x7f43380b4b50 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.017+0000 7f4326ffd640 1 -- 172.21.15.113:0/447280900 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4330005d10 con 0x7f43380b4b50 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.017+0000 7f4326ffd640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f431806be60 0x7f431806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.017+0000 7f433659c640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f431806be60 0x7f431806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.017+0000 7f4326ffd640 1 -- 172.21.15.113:0/447280900 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f433008b210 con 0x7f43380b4b50 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.018+0000 7f433659c640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f431806be60 0x7f431806e320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4328032000 tx=0x7f432802d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.022+0000 7f4326ffd640 1 -- 172.21.15.113:0/447280900 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4330022030 con 0x7f43380b4b50 2024-04-25T00:05:09.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.235+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4338071360 con 0x7f43380b4b50 2024-04-25T00:05:09.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.235+0000 7f4326ffd640 1 -- 172.21.15.113:0/447280900 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7f433813cd60 con 0x7f43380b4b50 2024-04-25T00:05:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:09.236 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:09.236 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f431806be60 msgr2=0x7f431806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f431806be60 0x7f431806e320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4328032000 tx=0x7f432802d000 comp rx=0 tx=0).stop 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43380b4b50 msgr2=0x7f43381564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43380b4b50 0x7f43381564c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f433000ea10 tx=0x7f433000eed0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 shutdown_connections 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f431806be60 0x7f431806e320 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43380b4b50 0x7f43381564c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4338158cf0 0x7f43380b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 --2- 172.21.15.113:0/447280900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4320003b90 0x7f433813c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 >> 172.21.15.113:0/447280900 conn(0x7f43380b1a50 msgr2=0x7f433806df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 shutdown_connections 2024-04-25T00:05:09.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.237+0000 7f433c902640 1 -- 172.21.15.113:0/447280900 wait complete. 2024-04-25T00:05:09.248 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:09.249 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:09.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.344+0000 7fd7c78f5640 1 Processor -- start 2024-04-25T00:05:09.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.344+0000 7fd7c78f5640 1 -- start start 2024-04-25T00:05:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c78f5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7c0144910 0x7fd7c00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c78f5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd7c00b56b0 0x7fd7c00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c78f5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c0144cf0 con 0x7fd7c00b5240 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c78f5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c00b4510 con 0x7fd7c0144910 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c78f5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c00b2f30 con 0x7fd7c00b56b0 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c5691640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7c0144910 0x7fd7c00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c5691640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7c0144910 0x7fd7c00b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40128/0 (socket says 172.21.15.113:40128) 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c5691640 1 -- 172.21.15.113:0/2002401259 learned_addr learned my addr 172.21.15.113:0/2002401259 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c5691640 1 -- 172.21.15.113:0/2002401259 >> v2:172.21.15.178:3300/0 conn(0x7fd7c00b56b0 msgr2=0x7fd7c00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c5691640 1 --2- 172.21.15.113:0/2002401259 >> v2:172.21.15.178:3300/0 conn(0x7fd7c00b56b0 0x7fd7c00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c5691640 1 -- 172.21.15.113:0/2002401259 >> v1:172.21.15.178:6789/0 conn(0x7fd7c00b5240 legacy=0x7fd7c006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.345+0000 7fd7c5691640 1 -- 172.21.15.113:0/2002401259 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c00b3210 con 0x7fd7c0144910 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7c5691640 1 --2- 172.21.15.113:0/2002401259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00b3fd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009870 tx=0x7fd7bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ea8a05c8700c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7b7fff640 1 -- 172.21.15.113:0/2002401259 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc037070 con 0x7fd7c0144910 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7b7fff640 1 -- 172.21.15.113:0/2002401259 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc032440 con 0x7fd7c0144910 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7b7fff640 1 -- 172.21.15.113:0/2002401259 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc032810 con 0x7fd7c0144910 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7c78f5640 1 -- 172.21.15.113:0/2002401259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 msgr2=0x7fd7c00b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/2002401259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00b3fd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009870 tx=0x7fd7bc02a270 comp rx=0 tx=0).stop 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7c78f5640 1 -- 172.21.15.113:0/2002401259 shutdown_connections 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/2002401259 >> v2:172.21.15.178:3300/0 conn(0x7fd7c00b56b0 0x7fd7c00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/2002401259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00b3fd0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.346+0000 7fd7c78f5640 1 -- 172.21.15.113:0/2002401259 >> 172.21.15.113:0/2002401259 conn(0x7fd7c00b1a50 msgr2=0x7fd7c00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.347+0000 7fd7c78f5640 1 -- 172.21.15.113:0/2002401259 shutdown_connections 2024-04-25T00:05:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.347+0000 7fd7c78f5640 1 -- 172.21.15.113:0/2002401259 wait complete. 2024-04-25T00:05:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 Processor -- start 2024-04-25T00:05:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 -- start start 2024-04-25T00:05:09.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7c00b5240 0x7fd7c015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c00b56b0 0x7fd7c01f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c00b4da0 con 0x7fd7c0144910 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c00b37d0 con 0x7fd7c00b56b0 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c78f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c00b4f20 con 0x7fd7c00b5240 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c5e92640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c5e92640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40132/0 (socket says 172.21.15.113:40132) 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.348+0000 7fd7c5e92640 1 -- 172.21.15.113:0/389119585 learned_addr learned my addr 172.21.15.113:0/389119585 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c4e90640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c00b56b0 0x7fd7c01f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c5e92640 1 -- 172.21.15.113:0/389119585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7c00b5240 msgr2=0x7fd7c015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c5e92640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7c00b5240 0x7fd7c015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c5e92640 1 -- 172.21.15.113:0/389119585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c00b56b0 msgr2=0x7fd7c01f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c5e92640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c00b56b0 0x7fd7c01f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c5e92640 1 -- 172.21.15.113:0/389119585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c01f3090 con 0x7fd7c0144910 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c5e92640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00a9480 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800b9e0 tx=0x7fd7b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7b5ffb640 1 -- 172.21.15.113:0/389119585 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8019070 con 0x7fd7c0144910 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7b5ffb640 1 -- 172.21.15.113:0/389119585 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b800ca20 con 0x7fd7c0144910 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01f33d0 con 0x7fd7c0144910 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7b5ffb640 1 -- 172.21.15.113:0/389119585 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800ce50 con 0x7fd7c0144910 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.349+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c01f3c10 con 0x7fd7c0144910 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.350+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c01f3e10 con 0x7fd7c0144910 2024-04-25T00:05:09.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.350+0000 7fd7b5ffb640 1 -- 172.21.15.113:0/389119585 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7b801c070 con 0x7fd7c0144910 2024-04-25T00:05:09.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.351+0000 7fd7b5ffb640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7a406bfd0 0x7fd7a406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:09.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.351+0000 7fd7c5691640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7a406bfd0 0x7fd7a406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:09.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.351+0000 7fd7b5ffb640 1 -- 172.21.15.113:0/389119585 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fd7b808aad0 con 0x7fd7c0144910 2024-04-25T00:05:09.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.351+0000 7fd7c5691640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7a406bfd0 0x7fd7a406e490 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc034000 tx=0x7fd7bc02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.355+0000 7fd7b5ffb640 1 -- 172.21.15.113:0/389119585 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7c01f3e10 con 0x7fd7c0144910 2024-04-25T00:05:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.591+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd7c00731a0 con 0x7fd7c0144910 2024-04-25T00:05:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.591+0000 7fd7b5ffb640 1 -- 172.21.15.113:0/389119585 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1574 (secure 0 0 0) 0x7fd7c01f33d0 con 0x7fd7c0144910 2024-04-25T00:05:09.592 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:09.592 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":434,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":988,"data_bytes":1013439650,"bytes_used":2374676480,"bytes_avail":770719436800,"bytes_total":773094113280,"write_bytes_sec":84398175,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T00:04:39.186609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7a406bfd0 msgr2=0x7fd7a406e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7a406bfd0 0x7fd7a406e490 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc034000 tx=0x7fd7bc02c000 comp rx=0 tx=0).stop 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 msgr2=0x7fd7c00a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00a9480 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800b9e0 tx=0x7fd7b800bea0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 shutdown_connections 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd7a406bfd0 0x7fd7a406e490 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c00b56b0 0x7fd7c01f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0144910 0x7fd7c00a9480 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 --2- 172.21.15.113:0/389119585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7c00b5240 0x7fd7c015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 >> 172.21.15.113:0/389119585 conn(0x7fd7c00b1a50 msgr2=0x7fd7c00cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 shutdown_connections 2024-04-25T00:05:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:09.593+0000 7fd7c78f5640 1 -- 172.21.15.113:0/389119585 wait complete. 2024-04-25T00:05:09.604 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:05:10.606 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.713+0000 7fa4b4ed4640 1 Processor -- start 2024-04-25T00:05:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.713+0000 7fa4b4ed4640 1 -- start start 2024-04-25T00:05:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4b4ed4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4b015a6b0 0x7fa4b0070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4b4ed4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa4b00b3af0 0x7fa4b00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4b4ed4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b00b2d50 con 0x7fa4b0072ea0 2024-04-25T00:05:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4b4ed4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b0072a70 con 0x7fa4b015a6b0 2024-04-25T00:05:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4b4ed4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b00b3010 con 0x7fa4b00b3af0 2024-04-25T00:05:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4ae59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4b015a6b0 0x7fa4b0070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4ae59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4b015a6b0 0x7fa4b0070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40136/0 (socket says 172.21.15.113:40136) 2024-04-25T00:05:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.714+0000 7fa4ae59c640 1 -- 172.21.15.113:0/3539071362 learned_addr learned my addr 172.21.15.113:0/3539071362 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.715+0000 7fa4aed9d640 1 --2- 172.21.15.113:0/3539071362 >> v2:172.21.15.178:3300/0 conn(0x7fa4b00b3af0 0x7fa4b00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.715+0000 7fa4ae59c640 1 -- 172.21.15.113:0/3539071362 >> v2:172.21.15.178:3300/0 conn(0x7fa4b00b3af0 msgr2=0x7fa4b00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.715+0000 7fa4ae59c640 1 --2- 172.21.15.113:0/3539071362 >> v2:172.21.15.178:3300/0 conn(0x7fa4b00b3af0 0x7fa4b00b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.715+0000 7fa4ae59c640 1 -- 172.21.15.113:0/3539071362 >> v1:172.21.15.178:6789/0 conn(0x7fa4b0072ea0 legacy=0x7fa4b0071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.715+0000 7fa4ae59c640 1 -- 172.21.15.113:0/3539071362 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b00b32f0 con 0x7fa4b015a6b0 2024-04-25T00:05:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.715+0000 7fa4ae59c640 1 --2- 172.21.15.113:0/3539071362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b0070d00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00cc5a0 tx=0x7fa4a402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f2cc420eb78463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4ad59a640 1 -- 172.21.15.113:0/3539071362 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4036020 con 0x7fa4b015a6b0 2024-04-25T00:05:10.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4ad59a640 1 -- 172.21.15.113:0/3539071362 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4032640 con 0x7fa4b015a6b0 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4ad59a640 1 -- 172.21.15.113:0/3539071362 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4032a10 con 0x7fa4b015a6b0 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/3539071362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 msgr2=0x7fa4b0070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/3539071362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b0070d00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00cc5a0 tx=0x7fa4a402c940 comp rx=0 tx=0).stop 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/3539071362 shutdown_connections 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/3539071362 >> v2:172.21.15.178:3300/0 conn(0x7fa4b00b3af0 0x7fa4b00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/3539071362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b0070d00 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/3539071362 >> 172.21.15.113:0/3539071362 conn(0x7fa4b00b1a50 msgr2=0x7fa4b00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:10.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/3539071362 shutdown_connections 2024-04-25T00:05:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.716+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/3539071362 wait complete. 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 Processor -- start 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 -- start start 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0072ea0 0x7fa4b01f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4b00b3af0 0x7fa4b01efa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b015aa90 con 0x7fa4b015a6b0 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0072720 con 0x7fa4b0072ea0 2024-04-25T00:05:10.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4b4ed4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b015ac10 con 0x7fa4b00b3af0 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4aed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4aed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b014a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40144/0 (socket says 172.21.15.113:40144) 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.718+0000 7fa4aed9d640 1 -- 172.21.15.113:0/4163887829 learned_addr learned my addr 172.21.15.113:0/4163887829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4add9b640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4b00b3af0 0x7fa4b01efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4ae59c640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0072ea0 0x7fa4b01f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4aed9d640 1 -- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4b00b3af0 msgr2=0x7fa4b01efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4aed9d640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4b00b3af0 0x7fa4b01efa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4aed9d640 1 -- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0072ea0 msgr2=0x7fa4b01f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4aed9d640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0072ea0 0x7fa4b01f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4aed9d640 1 -- 172.21.15.113:0/4163887829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b014a540 con 0x7fa4b015a6b0 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4add9b640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4b00b3af0 0x7fa4b01efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4aed9d640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b014a000 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8007dc0 tx=0x7fa4a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:10.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa49effd640 1 -- 172.21.15.113:0/4163887829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8014420 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa49effd640 1 -- 172.21.15.113:0/4163887829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a8019070 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa49effd640 1 -- 172.21.15.113:0/4163887829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8014820 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b01f4490 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.719+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b01f3ed0 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.720+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b00691a0 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.721+0000 7fa49effd640 1 -- 172.21.15.113:0/4163887829 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4a80149c0 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.721+0000 7fa49effd640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa49006be30 0x7fa49006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.721+0000 7fa49effd640 1 -- 172.21.15.113:0/4163887829 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fa4a808b630 con 0x7fa4b015a6b0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.721+0000 7fa4ae59c640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa49006be30 0x7fa49006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.721+0000 7fa4ae59c640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa49006be30 0x7fa49006e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4037000 tx=0x7fa4a403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:10.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.725+0000 7fa49effd640 1 -- 172.21.15.113:0/4163887829 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa4a8059fe0 con 0x7fa4b015a6b0 2024-04-25T00:05:10.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.934+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4b0144e80 con 0x7fa4b015a6b0 2024-04-25T00:05:10.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.934+0000 7fa49effd640 1 -- 172.21.15.113:0/4163887829 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7fa4a805ee20 con 0x7fa4b015a6b0 2024-04-25T00:05:10.935 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:10.935 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:10.935 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa49006be30 msgr2=0x7fa49006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa49006be30 0x7fa49006e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4037000 tx=0x7fa4a403d040 comp rx=0 tx=0).stop 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 msgr2=0x7fa4b014a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b014a000 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8007dc0 tx=0x7fa4a800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 shutdown_connections 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa49006be30 0x7fa49006e2f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b015a6b0 0x7fa4b014a000 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4b00b3af0 0x7fa4b01efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 --2- 172.21.15.113:0/4163887829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0072ea0 0x7fa4b01f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:10.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 >> 172.21.15.113:0/4163887829 conn(0x7fa4b00b1a50 msgr2=0x7fa4b00cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:10.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 shutdown_connections 2024-04-25T00:05:10.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:10.936+0000 7fa4b4ed4640 1 -- 172.21.15.113:0/4163887829 wait complete. 2024-04-25T00:05:10.947 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:10.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:11.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.042+0000 7fb82e7ce640 1 Processor -- start 2024-04-25T00:05:11.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.042+0000 7fb82e7ce640 1 -- start start 2024-04-25T00:05:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb82e7ce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb82815e320 0x7fb8280d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb82e7ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8280d2290 0x7fb8280b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb82e7ce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8280d3de0 con 0x7fb8280d0b00 2024-04-25T00:05:11.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb82e7ce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb8280d3410 con 0x7fb82815e320 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb82e7ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb8280d2800 con 0x7fb8280d2290 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb8277fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8280d2290 0x7fb8280b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb827fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb82815e320 0x7fb8280d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb827fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb82815e320 0x7fb8280d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:56254/0 (socket says 172.21.15.113:56254) 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb827fff640 1 -- 172.21.15.113:0/2462133966 learned_addr learned my addr 172.21.15.113:0/2462133966 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.043+0000 7fb82cd6b640 1 --1- 172.21.15.113:0/2462133966 >> v1:172.21.15.178:6789/0 conn(0x7fb8280d0b00 0x7fb8280b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:48196/0 (socket says 172.21.15.113:48196) 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.044+0000 7fb8277fe640 1 -- 172.21.15.113:0/2462133966 >> v2:172.21.15.178:3300/0 conn(0x7fb82815e320 msgr2=0x7fb8280d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.044+0000 7fb8277fe640 1 --2- 172.21.15.113:0/2462133966 >> v2:172.21.15.178:3300/0 conn(0x7fb82815e320 0x7fb8280d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.044+0000 7fb8277fe640 1 -- 172.21.15.113:0/2462133966 >> v1:172.21.15.178:6789/0 conn(0x7fb8280d0b00 legacy=0x7fb8280b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.044+0000 7fb8277fe640 1 -- 172.21.15.113:0/2462133966 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8280d2ae0 con 0x7fb8280d2290 2024-04-25T00:05:11.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.044+0000 7fb827fff640 1 --2- 172.21.15.113:0/2462133966 >> v2:172.21.15.178:3300/0 conn(0x7fb82815e320 0x7fb8280d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.044+0000 7fb8277fe640 1 --2- 172.21.15.113:0/2462133966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8280d2290 0x7fb8280b9aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0097a0 tx=0x7fb81c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6494a49b23349538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb826ffd640 1 -- 172.21.15.113:0/2462133966 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c037070 con 0x7fb8280d2290 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb826ffd640 1 -- 172.21.15.113:0/2462133966 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81c032440 con 0x7fb8280d2290 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb826ffd640 1 -- 172.21.15.113:0/2462133966 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c032810 con 0x7fb8280d2290 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 -- 172.21.15.113:0/2462133966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8280d2290 msgr2=0x7fb8280b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/2462133966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8280d2290 0x7fb8280b9aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0097a0 tx=0x7fb81c02a210 comp rx=0 tx=0).stop 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 -- 172.21.15.113:0/2462133966 shutdown_connections 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/2462133966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8280d2290 0x7fb8280b9aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/2462133966 >> v2:172.21.15.178:3300/0 conn(0x7fb82815e320 0x7fb8280d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 -- 172.21.15.113:0/2462133966 >> 172.21.15.113:0/2462133966 conn(0x7fb8280b5c20 msgr2=0x7fb8280b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 -- 172.21.15.113:0/2462133966 shutdown_connections 2024-04-25T00:05:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.045+0000 7fb82e7ce640 1 -- 172.21.15.113:0/2462133966 wait complete. 2024-04-25T00:05:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.046+0000 7fb82e7ce640 1 Processor -- start 2024-04-25T00:05:11.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb82e7ce640 1 -- start start 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb82e7ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8280d0b00 0x7fb828160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb82e7ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8280d2290 0x7fb8281f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb82e7ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb82815e320 0x7fb8280ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb82e7ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8280b9fe0 con 0x7fb8280d2290 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb82e7ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8280d30a0 con 0x7fb8280d0b00 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb82e7ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb82815e700 con 0x7fb82815e320 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb8277fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8280d2290 0x7fb8281f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb8277fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8280d2290 0x7fb8281f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40156/0 (socket says 172.21.15.113:40156) 2024-04-25T00:05:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb8277fe640 1 -- 172.21.15.113:0/1383717561 learned_addr learned my addr 172.21.15.113:0/1383717561 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.047+0000 7fb827fff640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8280d0b00 0x7fb828160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb82cd6b640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb82815e320 0x7fb8280ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb8277fe640 1 -- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb82815e320 msgr2=0x7fb8280ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb8277fe640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb82815e320 0x7fb8280ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb8277fe640 1 -- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8280d0b00 msgr2=0x7fb828160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb8277fe640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8280d0b00 0x7fb828160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb8277fe640 1 -- 172.21.15.113:0/1383717561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8281f4ca0 con 0x7fb8280d2290 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb8277fe640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8280d2290 0x7fb8281f4760 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb81c035040 tx=0x7fb81c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb824ff9640 1 -- 172.21.15.113:0/1383717561 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c037040 con 0x7fb8280d2290 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb824ff9640 1 -- 172.21.15.113:0/1383717561 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81c030500 con 0x7fb8280d2290 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8280ab070 con 0x7fb8280d2290 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb824ff9640 1 -- 172.21.15.113:0/1383717561 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c030910 con 0x7fb8280d2290 2024-04-25T00:05:11.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.048+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8280ab630 con 0x7fb8280d2290 2024-04-25T00:05:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.049+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8280691a0 con 0x7fb8280d2290 2024-04-25T00:05:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.050+0000 7fb824ff9640 1 -- 172.21.15.113:0/1383717561 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb81c044020 con 0x7fb8280d2290 2024-04-25T00:05:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.050+0000 7fb824ff9640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81406be30 0x7fb81406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.050+0000 7fb824ff9640 1 -- 172.21.15.113:0/1383717561 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fb81c0aafd0 con 0x7fb8280d2290 2024-04-25T00:05:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.050+0000 7fb827fff640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81406be30 0x7fb81406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.051+0000 7fb827fff640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81406be30 0x7fb81406e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb818006ca0 tx=0x7fb818009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.054+0000 7fb824ff9640 1 -- 172.21.15.113:0/1383717561 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb8281f4ca0 con 0x7fb8280d2290 2024-04-25T00:05:11.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.295+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb8280d4c50 con 0x7fb8280d2290 2024-04-25T00:05:11.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.295+0000 7fb824ff9640 1 -- 172.21.15.113:0/1383717561 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1574 (secure 0 0 0) 0x7fb81c079330 con 0x7fb8280d2290 2024-04-25T00:05:11.296 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:11.296 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":436,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":988,"data_bytes":1013439650,"bytes_used":2374676480,"bytes_avail":770719436800,"bytes_total":773094113280,"write_bytes_sec":84396741,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T00:04:39.186609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81406be30 msgr2=0x7fb81406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81406be30 0x7fb81406e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb818006ca0 tx=0x7fb818009240 comp rx=0 tx=0).stop 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8280d2290 msgr2=0x7fb8281f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8280d2290 0x7fb8281f4760 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb81c035040 tx=0x7fb81c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 shutdown_connections 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb81406be30 0x7fb81406e2f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb82815e320 0x7fb8280ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8280d2290 0x7fb8281f4760 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 --2- 172.21.15.113:0/1383717561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8280d0b00 0x7fb828160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 >> 172.21.15.113:0/1383717561 conn(0x7fb8280b5c20 msgr2=0x7fb8280b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 shutdown_connections 2024-04-25T00:05:11.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:11.297+0000 7fb82e7ce640 1 -- 172.21.15.113:0/1383717561 wait complete. 2024-04-25T00:05:11.308 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:05:12.309 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:12.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.411+0000 7f8b582fc640 1 Processor -- start 2024-04-25T00:05:12.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.411+0000 7f8b582fc640 1 -- start start 2024-04-25T00:05:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.412+0000 7f8b582fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b500b3fd0 0x7f8b5006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.412+0000 7f8b582fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b5006e510 0x7f8b500cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.412+0000 7f8b582fc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b50144cf0 con 0x7f8b50144910 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.412+0000 7f8b582fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8b5006e980 con 0x7f8b5006e510 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.412+0000 7f8b582fc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8b500b2f30 con 0x7f8b500b3fd0 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b56899640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b5006e510 0x7f8b500cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b56899640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b5006e510 0x7f8b500cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40172/0 (socket says 172.21.15.113:40172) 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b56899640 1 -- 172.21.15.113:0/141056774 learned_addr learned my addr 172.21.15.113:0/141056774 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b55897640 1 --2- 172.21.15.113:0/141056774 >> v2:172.21.15.178:3300/0 conn(0x7f8b500b3fd0 0x7f8b5006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b56899640 1 -- 172.21.15.113:0/141056774 >> v2:172.21.15.178:3300/0 conn(0x7f8b500b3fd0 msgr2=0x7f8b5006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b56899640 1 --2- 172.21.15.113:0/141056774 >> v2:172.21.15.178:3300/0 conn(0x7f8b500b3fd0 0x7f8b5006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b56899640 1 -- 172.21.15.113:0/141056774 >> v1:172.21.15.178:6789/0 conn(0x7f8b50144910 legacy=0x7f8b500b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.413+0000 7f8b56899640 1 -- 172.21.15.113:0/141056774 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b500b3210 con 0x7f8b5006e510 2024-04-25T00:05:12.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b56899640 1 --2- 172.21.15.113:0/141056774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b5006e510 0x7f8b500cd500 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800d150 tx=0x7f8b4802c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febbb0ccadbdd9b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:12.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b55096640 1 -- 172.21.15.113:0/141056774 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48009600 con 0x7f8b5006e510 2024-04-25T00:05:12.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b55096640 1 -- 172.21.15.113:0/141056774 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b48037070 con 0x7f8b5006e510 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b55096640 1 -- 172.21.15.113:0/141056774 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b480099c0 con 0x7f8b5006e510 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b582fc640 1 -- 172.21.15.113:0/141056774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b5006e510 msgr2=0x7f8b500cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b582fc640 1 --2- 172.21.15.113:0/141056774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b5006e510 0x7f8b500cd500 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800d150 tx=0x7f8b4802c970 comp rx=0 tx=0).stop 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b582fc640 1 -- 172.21.15.113:0/141056774 shutdown_connections 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b582fc640 1 --2- 172.21.15.113:0/141056774 >> v2:172.21.15.178:3300/0 conn(0x7f8b500b3fd0 0x7f8b5006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b582fc640 1 --2- 172.21.15.113:0/141056774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b5006e510 0x7f8b500cd500 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.414+0000 7f8b582fc640 1 -- 172.21.15.113:0/141056774 >> 172.21.15.113:0/141056774 conn(0x7f8b500b1a50 msgr2=0x7f8b500b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.415+0000 7f8b582fc640 1 -- 172.21.15.113:0/141056774 shutdown_connections 2024-04-25T00:05:12.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.415+0000 7f8b582fc640 1 -- 172.21.15.113:0/141056774 wait complete. 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 Processor -- start 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 -- start start 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b5006e510 0x7f8b5014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b500b3fd0 0x7f8b501385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50144910 0x7f8b50138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b500b4440 con 0x7f8b50144910 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b500b37d0 con 0x7f8b5006e510 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b582fc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b500b4da0 con 0x7f8b500b3fd0 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.416+0000 7f8b56899640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50144910 0x7f8b50138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50144910 0x7f8b50138b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40186/0 (socket says 172.21.15.113:40186) 2024-04-25T00:05:12.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 -- 172.21.15.113:0/2726199284 learned_addr learned my addr 172.21.15.113:0/2726199284 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56098640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b5006e510 0x7f8b5014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b55897640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b500b3fd0 0x7f8b501385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 -- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b500b3fd0 msgr2=0x7f8b501385d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b500b3fd0 0x7f8b501385d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 -- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b5006e510 msgr2=0x7f8b5014f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b5006e510 0x7f8b5014f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 -- 172.21.15.113:0/2726199284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5013b230 con 0x7f8b50144910 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b56899640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50144910 0x7f8b50138b10 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8b48034040 tx=0x7f8b48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b42ffd640 1 -- 172.21.15.113:0/2726199284 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48009dc0 con 0x7f8b50144910 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b42ffd640 1 -- 172.21.15.113:0/2726199284 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b48047480 con 0x7f8b50144910 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5013b510 con 0x7f8b50144910 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b42ffd640 1 -- 172.21.15.113:0/2726199284 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48047870 con 0x7f8b50144910 2024-04-25T00:05:12.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.417+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b50140120 con 0x7f8b50144910 2024-04-25T00:05:12.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.418+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b500691a0 con 0x7f8b50144910 2024-04-25T00:05:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.419+0000 7f8b42ffd640 1 -- 172.21.15.113:0/2726199284 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b48032420 con 0x7f8b50144910 2024-04-25T00:05:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.419+0000 7f8b42ffd640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b3406be30 0x7f8b3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.419+0000 7f8b42ffd640 1 -- 172.21.15.113:0/2726199284 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f8b480ae0c0 con 0x7f8b50144910 2024-04-25T00:05:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.419+0000 7f8b56098640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b3406be30 0x7f8b3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.419+0000 7f8b56098640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b3406be30 0x7f8b3406e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c007fd0 tx=0x7f8b4c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:12.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.423+0000 7f8b42ffd640 1 -- 172.21.15.113:0/2726199284 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8b4807ca70 con 0x7f8b50144910 2024-04-25T00:05:12.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.632+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b500722f0 con 0x7f8b50144910 2024-04-25T00:05:12.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.632+0000 7f8b42ffd640 1 -- 172.21.15.113:0/2726199284 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7f8b4807c370 con 0x7f8b50144910 2024-04-25T00:05:12.633 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:12.633 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:12.634 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b3406be30 msgr2=0x7f8b3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b3406be30 0x7f8b3406e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c007fd0 tx=0x7f8b4c009040 comp rx=0 tx=0).stop 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50144910 msgr2=0x7f8b50138b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50144910 0x7f8b50138b10 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8b48034040 tx=0x7f8b48004a40 comp rx=0 tx=0).stop 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 shutdown_connections 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b3406be30 0x7f8b3406e2f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50144910 0x7f8b50138b10 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b500b3fd0 0x7f8b501385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 --2- 172.21.15.113:0/2726199284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b5006e510 0x7f8b5014f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 >> 172.21.15.113:0/2726199284 conn(0x7f8b500b1a50 msgr2=0x7f8b500cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 shutdown_connections 2024-04-25T00:05:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.634+0000 7f8b582fc640 1 -- 172.21.15.113:0/2726199284 wait complete. 2024-04-25T00:05:12.645 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:12.645 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:12.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.740+0000 7fa379b41640 1 Processor -- start 2024-04-25T00:05:12.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.740+0000 7fa379b41640 1 -- start start 2024-04-25T00:05:12.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa379b41640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 0x7fa3740b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa379b41640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3740cfad0 con 0x7fa37406e6e0 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa379b41640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3740cf230 con 0x7fa374144d60 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa379b41640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa374145c40 con 0x7fa3741453c0 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa372ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa37406e6e0 0x7fa374071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47356/0 (socket says 172.21.15.113:47356) 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa372ffd640 1 -- 172.21.15.113:0/2912982118 learned_addr learned my addr 172.21.15.113:0/2912982118 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807021807 0 0) 0x7fa3740cfad0 con 0x7fa37406e6e0 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa35c003520 con 0x7fa37406e6e0 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa373fff640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 0x7fa3740b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 765545242 0 0) 0x7fa35c003520 con 0x7fa37406e6e0 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 msgr2=0x7fa3740b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa3727fc640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 0x7fa3740b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.741+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 >> v1:172.21.15.178:6789/0 conn(0x7fa374144d60 legacy=0x7fa37406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:12.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3740cf510 con 0x7fa37406e6e0 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fa368001670 con 0x7fa37406e6e0 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 >> v1:172.21.15.113:6789/0 conn(0x7fa37406e6e0 legacy=0x7fa374071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa35c0039e0 0x7fa35c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 0x7fa35c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c004d90 0x7fa35c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa374145c40 con 0x7fa35c004d90 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c00bc50 con 0x7fa35c0039e0 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c00be50 con 0x7fa3741453c0 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa373fff640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c004d90 0x7fa35c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3737fe640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa35c0039e0 0x7fa35c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa372ffd640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 0x7fa35c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa373fff640 1 -- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 msgr2=0x7fa35c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa373fff640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3741453c0 0x7fa35c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa373fff640 1 -- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa35c0039e0 msgr2=0x7fa35c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa373fff640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa35c0039e0 0x7fa35c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa373fff640 1 -- 172.21.15.113:0/2912982118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c003520 con 0x7fa35c004d90 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.742+0000 7fa3737fe640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa35c0039e0 0x7fa35c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa373fff640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c004d90 0x7fa35c005210 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa36c007330 tx=0x7fa35c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d184c5298154af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c033070 con 0x7fa35c004d90 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36c02f650 con 0x7fa35c004d90 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa3727fc640 1 -- 172.21.15.113:0/2912982118 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c009be0 con 0x7fa35c004d90 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa379b41640 1 -- 172.21.15.113:0/2912982118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c004d90 msgr2=0x7fa35c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa379b41640 1 --2- 172.21.15.113:0/2912982118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c004d90 0x7fa35c005210 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa36c007330 tx=0x7fa35c005830 comp rx=0 tx=0).stop 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa373fff640 1 -- 172.21.15.113:0/2912982118 reap_dead start 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa379b41640 1 -- 172.21.15.113:0/2912982118 shutdown_connections 2024-04-25T00:05:12.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa379b41640 1 -- 172.21.15.113:0/2912982118 >> 172.21.15.113:0/2912982118 conn(0x7fa3740b1a50 msgr2=0x7fa3740b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa379b41640 1 -- 172.21.15.113:0/2912982118 shutdown_connections 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.743+0000 7fa379b41640 1 -- 172.21.15.113:0/2912982118 wait complete. 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.744+0000 7fa379b41640 1 Processor -- start 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa379b41640 1 -- start start 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa379b41640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa35c004d90 0x7fa3741f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa379b41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa374143940 0x7fa374143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa379b41640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa374136490 0x7fa3741368f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa379b41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa374145140 con 0x7fa374143940 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa379b41640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa374070e10 con 0x7fa374136490 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa379b41640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa3740b5390 con 0x7fa35c004d90 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa372ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa374143940 0x7fa374143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa372ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa374143940 0x7fa374143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40200/0 (socket says 172.21.15.113:40200) 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.745+0000 7fa372ffd640 1 -- 172.21.15.113:0/947697666 learned_addr learned my addr 172.21.15.113:0/947697666 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:12.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa3737fe640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa35c004d90 0x7fa3741f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa373fff640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa374136490 0x7fa3741368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa372ffd640 1 -- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa35c004d90 msgr2=0x7fa3741f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa372ffd640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa35c004d90 0x7fa3741f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa372ffd640 1 -- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa374136490 msgr2=0x7fa3741368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa372ffd640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa374136490 0x7fa3741368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa372ffd640 1 -- 172.21.15.113:0/947697666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3741f3d40 con 0x7fa374143940 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa372ffd640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa374143940 0x7fa374143dc0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa368007370 tx=0x7fa368007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa35bfff640 1 -- 172.21.15.113:0/947697666 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36800f8e0 con 0x7fa374143940 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa35bfff640 1 -- 172.21.15.113:0/947697666 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368018070 con 0x7fa374143940 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3741f4020 con 0x7fa374143940 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa35bfff640 1 -- 172.21.15.113:0/947697666 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36800fcc0 con 0x7fa374143940 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.746+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3741f45e0 con 0x7fa374143940 2024-04-25T00:05:12.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.747+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa374144300 con 0x7fa374143940 2024-04-25T00:05:12.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.748+0000 7fa35bfff640 1 -- 172.21.15.113:0/947697666 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa36801d460 con 0x7fa374143940 2024-04-25T00:05:12.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.748+0000 7fa35bfff640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36006be60 0x7fa36006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:12.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.748+0000 7fa35bfff640 1 -- 172.21.15.113:0/947697666 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fa36808ada0 con 0x7fa374143940 2024-04-25T00:05:12.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.748+0000 7fa3737fe640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36006be60 0x7fa36006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:12.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.748+0000 7fa3737fe640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36006be60 0x7fa36006e320 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa364007fd0 tx=0x7fa364009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:12.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.752+0000 7fa35bfff640 1 -- 172.21.15.113:0/947697666 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa36800fe60 con 0x7fa374143940 2024-04-25T00:05:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.992+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa3741f4ad0 con 0x7fa374143940 2024-04-25T00:05:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.992+0000 7fa35bfff640 1 -- 172.21.15.113:0/947697666 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7fa3680597e0 con 0x7fa374143940 2024-04-25T00:05:12.993 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:12.994 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":437,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561799,"bytes_used":2568093696,"bytes_avail":770526019584,"bytes_total":773094113280,"write_bytes_sec":89659908,"read_op_per_sec":0,"write_op_per_sec":88},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T00:05:11.192556+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36006be60 msgr2=0x7fa36006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36006be60 0x7fa36006e320 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa364007fd0 tx=0x7fa364009040 comp rx=0 tx=0).stop 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa374143940 msgr2=0x7fa374143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa374143940 0x7fa374143dc0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa368007370 tx=0x7fa368007830 comp rx=0 tx=0).stop 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 shutdown_connections 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36006be60 0x7fa36006e320 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa374136490 0x7fa3741368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa374143940 0x7fa374143dc0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 --2- 172.21.15.113:0/947697666 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa35c004d90 0x7fa3741f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:12.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 >> 172.21.15.113:0/947697666 conn(0x7fa3740b1a50 msgr2=0x7fa3740afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:12.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 shutdown_connections 2024-04-25T00:05:12.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:12.994+0000 7fa379b41640 1 -- 172.21.15.113:0/947697666 wait complete. 2024-04-25T00:05:13.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T00:05:13.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:13.006 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T00:05:13.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.103+0000 7efe5a714640 1 Processor -- start 2024-04-25T00:05:13.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.104+0000 7efe5a714640 1 -- start start 2024-04-25T00:05:13.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.104+0000 7efe5a714640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efe540b5ca0 0x7efe540b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.104+0000 7efe5a714640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe540d2e00 con 0x7efe540b73c0 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.104+0000 7efe5a714640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe540d30e0 con 0x7efe5415e510 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.104+0000 7efe5a714640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efe540b6640 con 0x7efe540b5ca0 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe58cb1640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efe540b73c0 0x7efe540b77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47360/0 (socket says 172.21.15.113:47360) 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe58cb1640 1 -- 172.21.15.113:0/2070375590 learned_addr learned my addr 172.21.15.113:0/2070375590 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2263623958 0 0) 0x7efe540d2e00 con 0x7efe540b73c0 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe537fe640 1 --2- 172.21.15.113:0/2070375590 >> v2:172.21.15.178:3300/0 conn(0x7efe540b5ca0 0x7efe540b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe3c003520 con 0x7efe540b73c0 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1804098895 0 0) 0x7efe3c003520 con 0x7efe540b73c0 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 >> v2:172.21.15.178:3300/0 conn(0x7efe540b5ca0 msgr2=0x7efe540b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 --2- 172.21.15.113:0/2070375590 >> v2:172.21.15.178:3300/0 conn(0x7efe540b5ca0 0x7efe540b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 >> v1:172.21.15.178:6789/0 conn(0x7efe5415e510 legacy=0x7efe540d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe540d33c0 con 0x7efe540b73c0 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7efe4c0019f0 con 0x7efe540b73c0 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 >> v1:172.21.15.113:6789/0 conn(0x7efe540b73c0 legacy=0x7efe540b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe3c0039b0 0x7efe3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c004800 0x7efe3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe3c0051c0 0x7efe3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c00c130 con 0x7efe3c0051c0 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c00c2e0 con 0x7efe3c004800 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c00c4e0 con 0x7efe3c0039b0 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe58cb1640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe3c0051c0 0x7efe3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe537fe640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c004800 0x7efe3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe53fff640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe3c0039b0 0x7efe3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe58cb1640 1 -- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe3c0039b0 msgr2=0x7efe3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.105+0000 7efe58cb1640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe3c0039b0 0x7efe3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe58cb1640 1 -- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c004800 msgr2=0x7efe3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe58cb1640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c004800 0x7efe3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe58cb1640 1 -- 172.21.15.113:0/2070375590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c003520 con 0x7efe3c0051c0 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe58cb1640 1 -- 172.21.15.113:0/2070375590 reap_dead start 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe58cb1640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe3c0051c0 0x7efe3c00b9c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00a830 tx=0x7efe3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e9d71bacadf2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00a480 con 0x7efe3c0051c0 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c02f070 con 0x7efe3c0051c0 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.106+0000 7efe52ffd640 1 -- 172.21.15.113:0/2070375590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c02bcb0 con 0x7efe3c0051c0 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.107+0000 7efe5a714640 1 -- 172.21.15.113:0/2070375590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe3c0051c0 msgr2=0x7efe3c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.107+0000 7efe5a714640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe3c0051c0 0x7efe3c00b9c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00a830 tx=0x7efe3c005830 comp rx=0 tx=0).stop 2024-04-25T00:05:13.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.107+0000 7efe5a714640 1 -- 172.21.15.113:0/2070375590 shutdown_connections 2024-04-25T00:05:13.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.107+0000 7efe5a714640 1 --2- 172.21.15.113:0/2070375590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe3c0051c0 0x7efe3c00b9c0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.107+0000 7efe5a714640 1 -- 172.21.15.113:0/2070375590 >> 172.21.15.113:0/2070375590 conn(0x7efe5406b720 msgr2=0x7efe540d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.107+0000 7efe5a714640 1 -- 172.21.15.113:0/2070375590 shutdown_connections 2024-04-25T00:05:13.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.107+0000 7efe5a714640 1 -- 172.21.15.113:0/2070375590 wait complete. 2024-04-25T00:05:13.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.108+0000 7efe5a714640 1 Processor -- start 2024-04-25T00:05:13.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.108+0000 7efe5a714640 1 -- start start 2024-04-25T00:05:13.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe5a714640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c0051c0 0x7efe54147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe5a714640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe541f3d50 0x7efe540b8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe5a714640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe540c98e0 0x7efe540c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe5a714640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe540d3ea0 con 0x7efe541f3d50 2024-04-25T00:05:13.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe5a714640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe540d3980 con 0x7efe3c0051c0 2024-04-25T00:05:13.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe5a714640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe540b9dd0 con 0x7efe540c98e0 2024-04-25T00:05:13.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe541f3d50 0x7efe540b8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe541f3d50 0x7efe540b8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40226/0 (socket says 172.21.15.113:40226) 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 -- 172.21.15.113:0/3081689448 learned_addr learned my addr 172.21.15.113:0/3081689448 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe58cb1640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe540c98e0 0x7efe540c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe53fff640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c0051c0 0x7efe54147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 -- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe540c98e0 msgr2=0x7efe540c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe540c98e0 0x7efe540c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 -- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c0051c0 msgr2=0x7efe54147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c0051c0 0x7efe54147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 -- 172.21.15.113:0/3081689448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541f4d50 con 0x7efe541f3d50 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.109+0000 7efe537fe640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe541f3d50 0x7efe540b8cb0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7efe48011910 tx=0x7efe48011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.110+0000 7efe50ff9640 1 -- 172.21.15.113:0/3081689448 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4800faf0 con 0x7efe541f3d50 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.110+0000 7efe50ff9640 1 -- 172.21.15.113:0/3081689448 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4800fc90 con 0x7efe541f3d50 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.110+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe54145260 con 0x7efe541f3d50 2024-04-25T00:05:13.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.110+0000 7efe50ff9640 1 -- 172.21.15.113:0/3081689448 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48014440 con 0x7efe541f3d50 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.110+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541f44b0 con 0x7efe541f3d50 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.111+0000 7efe50ff9640 1 -- 172.21.15.113:0/3081689448 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe48012040 con 0x7efe541f3d50 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.111+0000 7efe50ff9640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.111+0000 7efe50ff9640 1 -- 172.21.15.113:0/3081689448 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7efe4808b310 con 0x7efe541f3d50 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.111+0000 7efe53fff640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.112+0000 7efe53fff640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe4006be30 0x7efe4006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7efe44001760 tx=0x7efe44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.112+0000 7efe5d426740 1 Processor -- start 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.112+0000 7efe5d426740 1 -- start start 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc172a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe5d426740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172bdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc17275a40 0x55dc172bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe5d426740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc171d96b0 con 0x55dc172bd600 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dc17181880 con 0x55dc1729fd60 2024-04-25T00:05:13.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dc171d5670 con 0x55dc17275a40 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172bdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172bdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40238/0 (socket says 172.21.15.113:40238) 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 -- 172.21.15.113:0/715602030 learned_addr learned my addr 172.21.15.113:0/715602030 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe58cb1640 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc17275a40 0x55dc172bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe53fff640 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc172a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 -- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc17275a40 msgr2=0x55dc172bbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc17275a40 0x55dc172bbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 -- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 msgr2=0x55dc172a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc172a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe537fe640 1 -- 172.21.15.113:0/715602030 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc16fb77d0 con 0x55dc172bd600 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.113+0000 7efe53fff640 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc172a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe537fe640 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172bdb50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efe480898f0 tx=0x7efe48059350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd65e91485e9973 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe39ffb640 1 -- 172.21.15.113:0/715602030 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4808b0c0 con 0x55dc172bd600 2024-04-25T00:05:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe39ffb640 1 -- 172.21.15.113:0/715602030 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe48057a00 con 0x55dc172bd600 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe39ffb640 1 -- 172.21.15.113:0/715602030 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48061da0 con 0x55dc172bd600 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 -- 172.21.15.113:0/715602030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 msgr2=0x55dc172bdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172bdb50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efe480898f0 tx=0x7efe48059350 comp rx=0 tx=0).stop 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 -- 172.21.15.113:0/715602030 shutdown_connections 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc17275a40 0x55dc172bbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172bdb50 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 --2- 172.21.15.113:0/715602030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc172a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 -- 172.21.15.113:0/715602030 >> 172.21.15.113:0/715602030 conn(0x55dc17272600 msgr2=0x55dc172bab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 -- 172.21.15.113:0/715602030 shutdown_connections 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.114+0000 7efe5d426740 1 -- 172.21.15.113:0/715602030 wait complete. 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.115+0000 7efe5d426740 1 -- start start 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.115+0000 7efe5d426740 1 Processor -- start 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe5d426740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1729fd60 0x55dc172a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc172bd600 0x55dc1726f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe5d426740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc172767e0 con 0x55dc1729fd60 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dc17194f50 con 0x55dc1726fab0 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dc17270530 con 0x55dc172bd600 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe5d426740 10 client.0 _ll_register_callbacks cb 0x55dc171d0000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1729fd60 0x55dc172a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1729fd60 0x55dc172a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40244/0 (socket says 172.21.15.113:40244) 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 -- 172.21.15.113:0/1376798874 learned_addr learned my addr 172.21.15.113:0/1376798874 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe537fe640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc172bd600 0x55dc1726f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe58cb1640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 -- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc172bd600 msgr2=0x55dc1726f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc172bd600 0x55dc1726f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 -- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 msgr2=0x55dc172a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 -- 172.21.15.113:0/1376798874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc16eed3f0 con 0x55dc1729fd60 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe58cb1640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe53fff640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1729fd60 0x55dc172a3340 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efe440157f0 tx=0x7efe44001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619d56a7bad7832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.116+0000 7efe32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe32ffd640 1 -- 172.21.15.113:0/1376798874 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44026020 con 0x55dc1729fd60 2024-04-25T00:05:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe32ffd640 1 -- 172.21.15.113:0/1376798874 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7efe44022690 con 0x55dc1729fd60 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe48057a00 con 0x55dc1729fd60 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe32ffd640 1 -- 172.21.15.113:0/1376798874 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7efe44030050 con 0x55dc1729fd60 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 10 client.8115 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55dc16ead0f0 con 0x55dc1729fd60 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe32ffd640 1 -- 172.21.15.113:0/1376798874 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe48057a00 con 0x55dc1729fd60 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe32ffd640 1 -- 172.21.15.113:0/1376798874 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7efe44033020 con 0x55dc1729fd60 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 10 client.8115 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 10 client.8115 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 20 client.8115 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cc1b0 0x55dc1729cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 4 client.8115 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc171e6bf0 con 0x55dc172cc1b0 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe537fe640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cc1b0 0x55dc1729cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe32ffd640 10 client.8115 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.117+0000 7efe537fe640 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cc1b0 0x55dc1729cdd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4edf354b04cc202e server_cookie=eb86c5f98c47aa59 in_seq=0 out_seq=0 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe32ffd640 1 -- 172.21.15.113:0/1376798874 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc171e6bf0 con 0x55dc172cc1b0 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe32ffd640 10 client.8115 handle_command_reply: tid=0 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 client.8115 shutdown 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1729fd60 msgr2=0x55dc172a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1729fd60 0x55dc172a3340 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efe440157f0 tx=0x7efe44001120 comp rx=0 tx=0).stop 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 shutdown_connections 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cc1b0 0x55dc1729cdd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe32ffd640 0 client.8115 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc172bd600 0x55dc1726f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 --2- 172.21.15.113:0/1376798874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1729fd60 0x55dc172a3340 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 >> 172.21.15.113:0/1376798874 conn(0x55dc17275a40 msgr2=0x55dc172be0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 shutdown_connections 2024-04-25T00:05:13.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 1 -- 172.21.15.113:0/1376798874 wait complete. 2024-04-25T00:05:13.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.118+0000 7efe5d426740 20 client.8115 trim_cache size 0 max 16384 2024-04-25T00:05:13.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.135+0000 7efe5d426740 1 Processor -- start 2024-04-25T00:05:13.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.135+0000 7efe5d426740 1 -- start start 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172bcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1729fd60 0x55dc172a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe5d426740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe5d426740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc172a0d90 con 0x55dc172bd600 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe54200de0 con 0x55dc1726fab0 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe541ffe20 con 0x55dc1729fd60 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172760a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40256/0 (socket says 172.21.15.113:40256) 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 -- 172.21.15.113:0/808660379 learned_addr learned my addr 172.21.15.113:0/808660379 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 -- 172.21.15.113:0/808660379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1729fd60 msgr2=0x55dc172a2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 --2- 172.21.15.113:0/808660379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1729fd60 0x55dc172a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 -- 172.21.15.113:0/808660379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 msgr2=0x55dc172bcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 --2- 172.21.15.113:0/808660379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172bcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 -- 172.21.15.113:0/808660379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc172a04b0 con 0x55dc172bd600 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe58cb1640 1 --2- 172.21.15.113:0/808660379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172760a0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efe4c02b500 tx=0x7efe4c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e7699eadf1ec7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe397fa640 1 -- 172.21.15.113:0/808660379 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0019f0 con 0x55dc172bd600 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe397fa640 1 -- 172.21.15.113:0/808660379 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c02f040 con 0x55dc172bd600 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.136+0000 7efe397fa640 1 -- 172.21.15.113:0/808660379 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c02ba50 con 0x55dc172bd600 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 -- 172.21.15.113:0/808660379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 msgr2=0x55dc172760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 --2- 172.21.15.113:0/808660379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172760a0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efe4c02b500 tx=0x7efe4c02d000 comp rx=0 tx=0).stop 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 -- 172.21.15.113:0/808660379 shutdown_connections 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 --2- 172.21.15.113:0/808660379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172760a0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 --2- 172.21.15.113:0/808660379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1729fd60 0x55dc172a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 --2- 172.21.15.113:0/808660379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1726fab0 0x55dc172bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 -- 172.21.15.113:0/808660379 >> 172.21.15.113:0/808660379 conn(0x55dc17275a40 msgr2=0x55dc172bab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 -- 172.21.15.113:0/808660379 shutdown_connections 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 -- 172.21.15.113:0/808660379 wait complete. 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.137+0000 7efe5d426740 1 -- start start 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 1 Processor -- start 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1726fab0 0x55dc172b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc1732eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ff7d0 con 0x55dc172bd600 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe541467b0 con 0x55dc1729fd60 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dc172ba590 con 0x55dc1726fab0 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.138+0000 7efe5d426740 10 client.0 _ll_register_callbacks cb 0x7efe44001da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40258/0 (socket says 172.21.15.113:40258) 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 -- 172.21.15.113:0/1523515188 learned_addr learned my addr 172.21.15.113:0/1523515188 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe53fff640 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1726fab0 0x55dc172b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe537fe640 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc1732eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 -- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1726fab0 msgr2=0x55dc172b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1726fab0 0x55dc172b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 -- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 msgr2=0x55dc1732eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc1732eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 -- 172.21.15.113:0/1523515188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1732f3e0 con 0x55dc172bd600 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe58cb1640 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172c7e90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0013d0 tx=0x7efe4c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab64b430137749b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe32ffd640 1 -- 172.21.15.113:0/1523515188 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c02bd90 con 0x55dc172bd600 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe32ffd640 1 -- 172.21.15.113:0/1523515188 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7efe4c034470 con 0x55dc172bd600 2024-04-25T00:05:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4c02f040 con 0x55dc172bd600 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe32ffd640 1 -- 172.21.15.113:0/1523515188 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7efe4c054050 con 0x55dc172bd600 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe5d426740 10 client.8121 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55dc172c8490 con 0x55dc172bd600 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.139+0000 7efe32ffd640 1 -- 172.21.15.113:0/1523515188 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c02f040 con 0x55dc172bd600 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe32ffd640 1 -- 172.21.15.113:0/1523515188 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7efe4c057020 con 0x55dc172bd600 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 10 client.8121 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 10 client.8121 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 20 client.8121 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cf4c0 0x55dc172cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 4 client.8121 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x55dc171e0f40 con 0x55dc172cf4c0 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe53fff640 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cf4c0 0x55dc172cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe32ffd640 10 client.8121 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe53fff640 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cf4c0 0x55dc172cf8d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79c3d3e7680c7c90 server_cookie=4d85366093566f82 in_seq=0 out_seq=0 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe32ffd640 1 -- 172.21.15.113:0/1523515188 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc171e0f40 con 0x55dc172cf4c0 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe32ffd640 10 client.8121 handle_command_reply: tid=0 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 client.8121 shutdown 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 msgr2=0x55dc172c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172c7e90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0013d0 tx=0x7efe4c001360 comp rx=0 tx=0).stop 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 shutdown_connections 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55dc172cf4c0 0x55dc172cf8d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc1729fd60 0x55dc1732eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc172bd600 0x55dc172c7e90 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe32ffd640 0 client.8121 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 --2- 172.21.15.113:0/1523515188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc1726fab0 0x55dc172b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 >> 172.21.15.113:0/1523515188 conn(0x55dc17275a40 msgr2=0x55dc172bcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 shutdown_connections 2024-04-25T00:05:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.140+0000 7efe5d426740 1 -- 172.21.15.113:0/1523515188 wait complete. 2024-04-25T00:05:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5d426740 20 client.8121 trim_cache size 0 max 16384 2024-04-25T00:05:13.150 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:13.150 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:05:13.150 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:13.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe4006be30 msgr2=0x7efe4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe4006be30 0x7efe4006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7efe44001760 tx=0x7efe44009040 comp rx=0 tx=0).stop 2024-04-25T00:05:13.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe541f3d50 msgr2=0x7efe540b8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe541f3d50 0x7efe540b8cb0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7efe48011910 tx=0x7efe48011dd0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 shutdown_connections 2024-04-25T00:05:13.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe4006be30 0x7efe4006e2f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe540c98e0 0x7efe540c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe541f3d50 0x7efe540b8cb0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 --2- 172.21.15.113:0/3081689448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c0051c0 0x7efe54147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 >> 172.21.15.113:0/3081689448 conn(0x7efe5406b720 msgr2=0x7efe540b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 shutdown_connections 2024-04-25T00:05:13.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.141+0000 7efe5a714640 1 -- 172.21.15.113:0/3081689448 wait complete. 2024-04-25T00:05:13.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:13.153 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T00:05:13.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.250+0000 7f04fd207640 1 Processor -- start 2024-04-25T00:05:13.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04fd207640 1 -- start start 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04fd207640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f815e2d0 0x7f04f815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04fd207640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04f80b4a40 0x7f04f80ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04fd207640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f815f4f0 con 0x7f04f80b3080 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04fd207640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f04f80cde00 con 0x7f04f80b4a40 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04fd207640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04f815f7d0 con 0x7f04f815e2d0 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04f759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04f80b4a40 0x7f04f80ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04f759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04f80b4a40 0x7f04f80ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40262/0 (socket says 172.21.15.113:40262) 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.251+0000 7f04f759e640 1 -- 172.21.15.113:0/2809889292 learned_addr learned my addr 172.21.15.113:0/2809889292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.252+0000 7f04f759e640 1 -- 172.21.15.113:0/2809889292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f815e2d0 msgr2=0x7f04f815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.252+0000 7f04f759e640 1 --2- 172.21.15.113:0/2809889292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f815e2d0 0x7f04f815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.252+0000 7f04f759e640 1 -- 172.21.15.113:0/2809889292 >> v1:172.21.15.178:6789/0 conn(0x7f04f80b3080 legacy=0x7f04f80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.252+0000 7f04f759e640 1 -- 172.21.15.113:0/2809889292 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f815fab0 con 0x7f04f80b4a40 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.252+0000 7f04f759e640 1 --2- 172.21.15.113:0/2809889292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b4a40 0x7f04f80ce290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009e50 tx=0x7f04f002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4de721ebcb5f2be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.252+0000 7f04f5d9b640 1 -- 172.21.15.113:0/2809889292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0036070 con 0x7f04f80b4a40 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04f5d9b640 1 -- 172.21.15.113:0/2809889292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f0031600 con 0x7f04f80b4a40 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04f5d9b640 1 -- 172.21.15.113:0/2809889292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00319d0 con 0x7f04f80b4a40 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 -- 172.21.15.113:0/2809889292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b4a40 msgr2=0x7f04f80ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 --2- 172.21.15.113:0/2809889292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b4a40 0x7f04f80ce290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009e50 tx=0x7f04f002a6c0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 -- 172.21.15.113:0/2809889292 shutdown_connections 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 --2- 172.21.15.113:0/2809889292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b4a40 0x7f04f80ce290 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 --2- 172.21.15.113:0/2809889292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f815e2d0 0x7f04f815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 -- 172.21.15.113:0/2809889292 >> 172.21.15.113:0/2809889292 conn(0x7f04f806b0f0 msgr2=0x7f04f806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 -- 172.21.15.113:0/2809889292 shutdown_connections 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.253+0000 7f04fd207640 1 -- 172.21.15.113:0/2809889292 wait complete. 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.254+0000 7f04fd207640 1 Processor -- start 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04fd207640 1 -- start start 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04fd207640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b3080 0x7f04f8147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04fd207640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f80b4a40 0x7f04f815c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04fd207640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f815e2d0 0x7f04f815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04fd207640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04f815f2b0 con 0x7f04f80b3080 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04fd207640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04f815f130 con 0x7f04f815e2d0 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04fd207640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04f815e6b0 con 0x7f04f80b4a40 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04f6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b3080 0x7f04f8147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04f6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b3080 0x7f04f8147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40278/0 (socket says 172.21.15.113:40278) 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.255+0000 7f04f6d9d640 1 -- 172.21.15.113:0/3799309236 learned_addr learned my addr 172.21.15.113:0/3799309236 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04f6d9d640 1 -- 172.21.15.113:0/3799309236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f80b4a40 msgr2=0x7f04f815c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f80b4a40 0x7f04f815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04f6d9d640 1 -- 172.21.15.113:0/3799309236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f815e2d0 msgr2=0x7f04f815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f815e2d0 0x7f04f815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04f6d9d640 1 -- 172.21.15.113:0/3799309236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f80a9520 con 0x7f04f80b3080 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b3080 0x7f04f8147ab0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f04e800b410 tx=0x7f04e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04df7fe640 1 -- 172.21.15.113:0/3799309236 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8015020 con 0x7f04f80b3080 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04df7fe640 1 -- 172.21.15.113:0/3799309236 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e8011850 con 0x7f04f80b3080 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f815b980 con 0x7f04f80b3080 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04df7fe640 1 -- 172.21.15.113:0/3799309236 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8011c40 con 0x7f04f80b3080 2024-04-25T00:05:13.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.256+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f80a9a70 con 0x7f04f80b3080 2024-04-25T00:05:13.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.257+0000 7f04df7fe640 1 -- 172.21.15.113:0/3799309236 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04e8018020 con 0x7f04f80b3080 2024-04-25T00:05:13.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.258+0000 7f04df7fe640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04e406be30 0x7f04e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.258+0000 7f04df7fe640 1 -- 172.21.15.113:0/3799309236 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f04e808ada0 con 0x7f04f80b3080 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.258+0000 7f04f659c640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04e406be30 0x7f04e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.258+0000 7f04f659c640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04e406be30 0x7f04e406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f04f80d0d40 tx=0x7f04ec007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.259+0000 7f04ffff2740 1 Processor -- start 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.259+0000 7f04ffff2740 1 -- start start 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569704d60 0x55f569705140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569722600 0x55f569722b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5696daa40 0x55f569720cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04ffff2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f56963e6b0 con 0x55f569722600 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f5695e6880 con 0x55f5696daa40 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f56963a670 con 0x55f569704d60 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569722600 0x55f569722b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569722600 0x55f569722b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40284/0 (socket says 172.21.15.113:40284) 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 -- 172.21.15.113:0/4224445951 learned_addr learned my addr 172.21.15.113:0/4224445951 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f759e640 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5696daa40 0x55f569720cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569704d60 0x55f569705140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 -- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569704d60 msgr2=0x55f569705140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569704d60 0x55f569705140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 -- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5696daa40 msgr2=0x55f569720cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5696daa40 0x55f569720cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 -- 172.21.15.113:0/4224445951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56941c7d0 con 0x55f569722600 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04f659c640 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569722600 0x55f569722b50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f04ec005560 tx=0x7f04ec00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855044ac9c5a73a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04dcff9640 1 -- 172.21.15.113:0/4224445951 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012650 con 0x55f569722600 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04dcff9640 1 -- 172.21.15.113:0/4224445951 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec012810 con 0x55f569722600 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.260+0000 7f04dcff9640 1 -- 172.21.15.113:0/4224445951 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec012c00 con 0x55f569722600 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 -- 172.21.15.113:0/4224445951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569722600 msgr2=0x55f569722b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569722600 0x55f569722b50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f04ec005560 tx=0x7f04ec00f000 comp rx=0 tx=0).stop 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 -- 172.21.15.113:0/4224445951 shutdown_connections 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5696daa40 0x55f569720cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569722600 0x55f569722b50 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 --2- 172.21.15.113:0/4224445951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569704d60 0x55f569705140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 -- 172.21.15.113:0/4224445951 >> 172.21.15.113:0/4224445951 conn(0x55f5696d7600 msgr2=0x55f56971fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 -- 172.21.15.113:0/4224445951 shutdown_connections 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.261+0000 7f04ffff2740 1 -- 172.21.15.113:0/4224445951 wait complete. 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 -- start start 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 Processor -- start 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f5696f7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f569706120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f569719580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f5696db7e0 con 0x55f569719100 2024-04-25T00:05:13.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f5695f9f50 con 0x55f569704d60 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f5696d5530 con 0x55f569722600 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04ffff2740 10 client.0 _ll_register_callbacks cb 0x55f569635000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04f759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f569719580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04f759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f569719580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40294/0 (socket says 172.21.15.113:40294) 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.262+0000 7f04f759e640 1 -- 172.21.15.113:0/3937614611 learned_addr learned my addr 172.21.15.113:0/3937614611 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f5696f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f659c640 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f569706120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f759e640 1 -- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 msgr2=0x55f569706120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f759e640 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f569706120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f759e640 1 -- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 msgr2=0x55f5696f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f759e640 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f5696f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f759e640 1 -- 172.21.15.113:0/3937614611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5693523f0 con 0x55f569719100 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04f759e640 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f569719580 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009f70 tx=0x7f04f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731fb8bd02b5dbf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04d5ffb640 1 -- 172.21.15.113:0/3937614611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0036040 con 0x55f569719100 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04d5ffb640 1 -- 172.21.15.113:0/3937614611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f04f0039b40 con 0x55f569719100 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ec012810 con 0x55f569719100 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04d5ffb640 1 -- 172.21.15.113:0/3937614611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f04f004e050 con 0x55f569719100 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04ffff2740 10 client.8133 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55f5693120f0 con 0x55f569719100 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04d5ffb640 1 -- 172.21.15.113:0/3937614611 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec012810 con 0x55f569719100 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04d5ffb640 1 -- 172.21.15.113:0/3937614611 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7f04f0051020 con 0x55f569719100 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04ffff2740 10 client.8133 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.263+0000 7f04ffff2740 10 client.8133 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04ffff2740 20 client.8133 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:13.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04ffff2740 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f569730b80 0x55f569730f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04ffff2740 4 client.8133 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f56964bbf0 con 0x55f569730b80 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f569730b80 0x55f569730f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04d5ffb640 10 client.8133 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f569730b80 0x55f569730f90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cabe665c09937bc server_cookie=4171ed68b76c636f in_seq=0 out_seq=0 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04d5ffb640 1 -- 172.21.15.113:0/3937614611 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f56964bbf0 con 0x55f569730b80 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04d5ffb640 10 client.8133 handle_command_reply: tid=0 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.264+0000 7f04ffff2740 1 client.8133 shutdown 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 msgr2=0x55f569719580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f569719580 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009f70 tx=0x7f04f0004a40 comp rx=0 tx=0).stop 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 shutdown_connections 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f569730b80 0x55f569730f90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f569719580 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f569706120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 --2- 172.21.15.113:0/3937614611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f5696f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04d5ffb640 0 client.8133 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 >> 172.21.15.113:0/3937614611 conn(0x55f5696daa40 msgr2=0x55f569723980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 shutdown_connections 2024-04-25T00:05:13.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 1 -- 172.21.15.113:0/3937614611 wait complete. 2024-04-25T00:05:13.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.265+0000 7f04ffff2740 20 client.8133 trim_cache size 0 max 16384 2024-04-25T00:05:13.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.282+0000 7f04ffff2740 1 Processor -- start 2024-04-25T00:05:13.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.282+0000 7f04ffff2740 1 -- start start 2024-04-25T00:05:13.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f5696d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f56972f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f56972f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04ffff2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81fc630 con 0x55f569719100 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81fb840 con 0x55f569704d60 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04f81f5120 con 0x55f569722600 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f56972f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f56972f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40304/0 (socket says 172.21.15.113:40304) 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 -- 172.21.15.113:0/1053223895 learned_addr learned my addr 172.21.15.113:0/1053223895 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 -- 172.21.15.113:0/1053223895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 msgr2=0x55f56972f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 --2- 172.21.15.113:0/1053223895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f56972f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 -- 172.21.15.113:0/1053223895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 msgr2=0x55f5696d8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 --2- 172.21.15.113:0/1053223895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f5696d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 -- 172.21.15.113:0/1053223895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56960bec0 con 0x55f569719100 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04f659c640 1 --2- 172.21.15.113:0/1053223895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f56972f0f0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f04ec005500 tx=0x7f04ec018050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d7fd2162fb8f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04d7fff640 1 -- 172.21.15.113:0/1053223895 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec013760 con 0x55f569719100 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04d7fff640 1 -- 172.21.15.113:0/1053223895 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec013900 con 0x55f569719100 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.283+0000 7f04d7fff640 1 -- 172.21.15.113:0/1053223895 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec013d10 con 0x55f569719100 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 -- 172.21.15.113:0/1053223895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 msgr2=0x55f56972f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 --2- 172.21.15.113:0/1053223895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f56972f0f0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f04ec005500 tx=0x7f04ec018050 comp rx=0 tx=0).stop 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 -- 172.21.15.113:0/1053223895 shutdown_connections 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 --2- 172.21.15.113:0/1053223895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f56972f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 --2- 172.21.15.113:0/1053223895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f56972f0f0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 --2- 172.21.15.113:0/1053223895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f5696d8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 -- 172.21.15.113:0/1053223895 >> 172.21.15.113:0/1053223895 conn(0x55f5696daa40 msgr2=0x55f5696f7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 -- 172.21.15.113:0/1053223895 shutdown_connections 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 -- 172.21.15.113:0/1053223895 wait complete. 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 -- start start 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.284+0000 7f04ffff2740 1 Processor -- start 2024-04-25T00:05:13.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f569795d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f5696d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04ffff2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f5696d9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04ffff2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f569708740 con 0x55f569719100 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04f806d800 con 0x55f569704d60 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04ffff2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f56971a530 con 0x55f569722600 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04ffff2740 10 client.0 _ll_register_callbacks cb 0x7f04f0034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04f659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f5696d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04f659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f5696d8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40308/0 (socket says 172.21.15.113:40308) 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04f659c640 1 -- 172.21.15.113:0/29575585 learned_addr learned my addr 172.21.15.113:0/29575585 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f569795d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.285+0000 7f04f759e640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f5696d9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f659c640 1 -- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 msgr2=0x55f5696d9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f659c640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f5696d9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f659c640 1 -- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 msgr2=0x55f569795d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f659c640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f569795d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f659c640 1 -- 172.21.15.113:0/29575585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f569717bf0 con 0x55f569719100 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f569795d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f659c640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f5696d8b00 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0078c0 tx=0x7f04ec001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2549e835be748 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04d5ffb640 1 -- 172.21.15.113:0/29575585 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec03a060 con 0x55f569719100 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04d5ffb640 1 -- 172.21.15.113:0/29575585 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f04ec01c040 con 0x55f569719100 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ec013900 con 0x55f569719100 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04d5ffb640 1 -- 172.21.15.113:0/29575585 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f04ec007020 con 0x55f569719100 2024-04-25T00:05:13.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 10 client.8139 fetch_fsmap learned FSMap version 56 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55f5697a3740 con 0x55f569719100 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04d5ffb640 1 -- 172.21.15.113:0/29575585 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec013900 con 0x55f569719100 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04d5ffb640 1 -- 172.21.15.113:0/29575585 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 56) ==== 3494+0+0 (secure 0 0 0) 0x7f04ec02a070 con 0x55f569719100 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 10 client.8139 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 10 client.8139 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 20 client.8139 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f5696da130 0x55f5697a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 4 client.8139 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 --> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f569645f40 con 0x55f5696da130 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.286+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f5696da130 0x55f5697a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04d5ffb640 10 client.8139 ms_handle_connect on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04f6d9d640 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f5696da130 0x55f5697a2760 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4aa18e1a3e94bc7 server_cookie=5daaa895679aa264 in_seq=0 out_seq=0 2024-04-25T00:05:13.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04d5ffb640 1 -- 172.21.15.113:0/29575585 <== mds.0 v2:172.21.15.113:6834/677893386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f569645f40 con 0x55f5696da130 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04d5ffb640 10 client.8139 handle_command_reply: tid=0 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 client.8139 shutdown 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 msgr2=0x55f5696d8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f5696d8b00 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0078c0 tx=0x7f04ec001090 comp rx=0 tx=0).stop 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 shutdown_connections 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:6834/677893386,v1:172.21.15.113:6836/677893386] conn(0x55f5696da130 0x55f5697a2760 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f569722600 0x55f5696d9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f569719100 0x55f5696d8b00 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04d5ffb640 0 client.8139 ms_handle_reset on v2:172.21.15.113:6834/677893386 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 --2- 172.21.15.113:0/29575585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f569704d60 0x55f569795d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 >> 172.21.15.113:0/29575585 conn(0x55f5696daa40 msgr2=0x55f5696dbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 shutdown_connections 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 1 -- 172.21.15.113:0/29575585 wait complete. 2024-04-25T00:05:13.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04ffff2740 20 client.8139 trim_cache size 0 max 16384 2024-04-25T00:05:13.296 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:13.296 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:05:13.296 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:05:13.296 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.287+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04e406be30 msgr2=0x7f04e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04e406be30 0x7f04e406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f04f80d0d40 tx=0x7f04ec007990 comp rx=0 tx=0).stop 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b3080 msgr2=0x7f04f8147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b3080 0x7f04f8147ab0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f04e800b410 tx=0x7f04e800b8d0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 shutdown_connections 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04e406be30 0x7f04e406e2f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f815e2d0 0x7f04f815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04f80b4a40 0x7f04f815c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 --2- 172.21.15.113:0/3799309236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f80b3080 0x7f04f8147ab0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 >> 172.21.15.113:0/3799309236 conn(0x7f04f806b0f0 msgr2=0x7f04f80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 shutdown_connections 2024-04-25T00:05:13.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.288+0000 7f04fd207640 1 -- 172.21.15.113:0/3799309236 wait complete. 2024-04-25T00:05:13.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.394+0000 7fbefb29e640 1 Processor -- start 2024-04-25T00:05:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.394+0000 7fbefb29e640 1 -- start start 2024-04-25T00:05:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbefb29e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbef406e6e0 0x7fbef40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbefb29e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef40cfad0 con 0x7fbef41453c0 2024-04-25T00:05:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbefb29e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef40cf230 con 0x7fbef4144d60 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbefb29e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbef4145c40 con 0x7fbef406e6e0 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbef8839640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbef406e6e0 0x7fbef40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbef983b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbef41453c0 0x7fbef4071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47372/0 (socket says 172.21.15.113:47372) 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbef983b640 1 -- 172.21.15.113:0/2106540007 learned_addr learned my addr 172.21.15.113:0/2106540007 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbef8839640 1 -- 172.21.15.113:0/2106540007 >> v1:172.21.15.178:6789/0 conn(0x7fbef4144d60 legacy=0x7fbef406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbef8839640 1 -- 172.21.15.113:0/2106540007 >> v1:172.21.15.113:6789/0 conn(0x7fbef41453c0 legacy=0x7fbef4071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbef8839640 1 -- 172.21.15.113:0/2106540007 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef40cf510 con 0x7fbef406e6e0 2024-04-25T00:05:13.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.395+0000 7fbeebfff640 1 -- 172.21.15.113:0/2106540007 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790035640 0 0) 0x7fbef40cfad0 con 0x7fbef41453c0 2024-04-25T00:05:13.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.396+0000 7fbef8839640 1 --2- 172.21.15.113:0/2106540007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef406e6e0 0x7fbef40b5390 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbee40097a0 tx=0x7fbee402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc30d2eb622f5d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.396+0000 7fbeebfff640 1 -- 172.21.15.113:0/2106540007 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4037070 con 0x7fbef406e6e0 2024-04-25T00:05:13.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.396+0000 7fbeebfff640 1 -- 172.21.15.113:0/2106540007 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee402adc0 con 0x7fbef406e6e0 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.396+0000 7fbeebfff640 1 -- 172.21.15.113:0/2106540007 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4032660 con 0x7fbef406e6e0 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.397+0000 7fbefb29e640 1 -- 172.21.15.113:0/2106540007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef406e6e0 msgr2=0x7fbef40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.397+0000 7fbefb29e640 1 --2- 172.21.15.113:0/2106540007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef406e6e0 0x7fbef40b5390 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbee40097a0 tx=0x7fbee402a0b0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.397+0000 7fbefb29e640 1 -- 172.21.15.113:0/2106540007 shutdown_connections 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.397+0000 7fbefb29e640 1 --2- 172.21.15.113:0/2106540007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef406e6e0 0x7fbef40b5390 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.397+0000 7fbefb29e640 1 -- 172.21.15.113:0/2106540007 >> 172.21.15.113:0/2106540007 conn(0x7fbef40b1a50 msgr2=0x7fbef40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.397+0000 7fbefb29e640 1 -- 172.21.15.113:0/2106540007 shutdown_connections 2024-04-25T00:05:13.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.397+0000 7fbefb29e640 1 -- 172.21.15.113:0/2106540007 wait complete. 2024-04-25T00:05:13.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.398+0000 7fbefb29e640 1 Processor -- start 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.398+0000 7fbefb29e640 1 -- start start 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.398+0000 7fbefb29e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef406e6e0 0x7fbef413c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.398+0000 7fbefb29e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef4144d60 0x7fbef4142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbefb29e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef41453c0 0x7fbef41f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbefb29e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4145140 con 0x7fbef4144d60 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbefb29e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4070e10 con 0x7fbef41453c0 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbefb29e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbef4071a40 con 0x7fbef406e6e0 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef4144d60 0x7fbef4142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef4144d60 0x7fbef4142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40318/0 (socket says 172.21.15.113:40318) 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 -- 172.21.15.113:0/321927832 learned_addr learned my addr 172.21.15.113:0/321927832 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef903a640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef406e6e0 0x7fbef413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef983b640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef41453c0 0x7fbef41f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 -- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef406e6e0 msgr2=0x7fbef413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef406e6e0 0x7fbef413c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 -- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef41453c0 msgr2=0x7fbef41f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef41453c0 0x7fbef41f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 -- 172.21.15.113:0/321927832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef41f2860 con 0x7fbef4144d60 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef983b640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef41453c0 0x7fbef41f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbef8839640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef4144d60 0x7fbef4142d60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbee40097a0 tx=0x7fbee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbee9ffb640 1 -- 172.21.15.113:0/321927832 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4037040 con 0x7fbef4144d60 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbee9ffb640 1 -- 172.21.15.113:0/321927832 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee4004250 con 0x7fbef4144d60 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbee9ffb640 1 -- 172.21.15.113:0/321927832 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4003610 con 0x7fbef4144d60 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef41f2b40 con 0x7fbef4144d60 2024-04-25T00:05:13.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.399+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef413cb40 con 0x7fbef4144d60 2024-04-25T00:05:13.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.401+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef4142350 con 0x7fbef4144d60 2024-04-25T00:05:13.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.401+0000 7fbee9ffb640 1 -- 172.21.15.113:0/321927832 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbee402ac40 con 0x7fbef4144d60 2024-04-25T00:05:13.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.401+0000 7fbee9ffb640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.402+0000 7fbef903a640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.402+0000 7fbee9ffb640 1 -- 172.21.15.113:0/321927832 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fbee40abd20 con 0x7fbef4144d60 2024-04-25T00:05:13.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.402+0000 7fbef903a640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbed806be30 0x7fbed806e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbef0003d90 tx=0x7fbef0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.406+0000 7fbee9ffb640 1 -- 172.21.15.113:0/321927832 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbee407a6d0 con 0x7fbef4144d60 2024-04-25T00:05:13.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.613+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef40691a0 con 0x7fbef4144d60 2024-04-25T00:05:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.613+0000 7fbee9ffb640 1 -- 172.21.15.113:0/321927832 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7fbee4079fd0 con 0x7fbef4144d60 2024-04-25T00:05:13.615 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:13.615 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:13.616 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbed806be30 msgr2=0x7fbed806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbed806be30 0x7fbed806e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbef0003d90 tx=0x7fbef0009040 comp rx=0 tx=0).stop 2024-04-25T00:05:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef4144d60 msgr2=0x7fbef4142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef4144d60 0x7fbef4142d60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbee40097a0 tx=0x7fbee40046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 shutdown_connections 2024-04-25T00:05:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef41453c0 0x7fbef41f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbef4144d60 0x7fbef4142d60 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 --2- 172.21.15.113:0/321927832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef406e6e0 0x7fbef413c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 >> 172.21.15.113:0/321927832 conn(0x7fbef40b1a50 msgr2=0x7fbef406cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 shutdown_connections 2024-04-25T00:05:13.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.615+0000 7fbefb29e640 1 -- 172.21.15.113:0/321927832 wait complete. 2024-04-25T00:05:13.627 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:13.627 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:13.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.724+0000 7f34630ee640 1 Processor -- start 2024-04-25T00:05:13.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.724+0000 7f34630ee640 1 -- start start 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f34630ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f345c0b4b00 0x7f345c1656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f34630ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c071b20 con 0x7f345c072ea0 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f34630ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c0b2d50 con 0x7f345c15a6b0 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f34630ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f345c072a70 con 0x7f345c0b4b00 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f345c072ea0 0x7f345c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47378/0 (socket says 172.21.15.113:47378) 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345bfff640 1 -- 172.21.15.113:0/3769848547 learned_addr learned my addr 172.21.15.113:0/3769848547 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f346168b640 1 --2- 172.21.15.113:0/3769848547 >> v2:172.21.15.178:3300/0 conn(0x7f345c0b4b00 0x7f345c1656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3539172863 0 0) 0x7f345c071b20 con 0x7f345c072ea0 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f344c003520 con 0x7f345c072ea0 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4152783372 0 0) 0x7f344c003520 con 0x7f345c072ea0 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 >> v2:172.21.15.178:3300/0 conn(0x7f345c0b4b00 msgr2=0x7f345c1656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 --2- 172.21.15.113:0/3769848547 >> v2:172.21.15.178:3300/0 conn(0x7f345c0b4b00 0x7f345c1656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 >> v1:172.21.15.178:6789/0 conn(0x7f345c15a6b0 legacy=0x7f345c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c0b3010 con 0x7f345c072ea0 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f3448001670 con 0x7f345c072ea0 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 >> v1:172.21.15.113:6789/0 conn(0x7f345c072ea0 legacy=0x7f345c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f344c0039b0 0x7f344c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c004800 0x7f344c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f344c0051c0 0x7f344c00cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f345c072a70 con 0x7f344c0051c0 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f344c00d170 con 0x7f344c004800 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.725+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f344c00d370 con 0x7f344c0039b0 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f344c0051c0 0x7f344c00cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f3460e8a640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f344c0039b0 0x7f344c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f345bfff640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c004800 0x7f344c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 -- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f344c0039b0 msgr2=0x7f344c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f344c0039b0 0x7f344c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 -- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c004800 msgr2=0x7f344c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c004800 0x7f344c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 -- 172.21.15.113:0/3769848547 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c003520 con 0x7f344c0051c0 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 -- 172.21.15.113:0/3769848547 reap_dead start 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.726+0000 7f346168b640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f344c0051c0 0x7f344c00cad0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f3454009fc0 tx=0x7f344c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae58e7e66460bc28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454035020 con 0x7f344c0051c0 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3454003bd0 con 0x7f344c0051c0 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f345b7fe640 1 -- 172.21.15.113:0/3769848547 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454009c10 con 0x7f344c0051c0 2024-04-25T00:05:13.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f34630ee640 1 -- 172.21.15.113:0/3769848547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f344c0051c0 msgr2=0x7f344c00cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f34630ee640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f344c0051c0 0x7f344c00cad0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f3454009fc0 tx=0x7f344c005830 comp rx=0 tx=0).stop 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f34630ee640 1 -- 172.21.15.113:0/3769848547 shutdown_connections 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f34630ee640 1 --2- 172.21.15.113:0/3769848547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f344c0051c0 0x7f344c00cad0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f34630ee640 1 -- 172.21.15.113:0/3769848547 >> 172.21.15.113:0/3769848547 conn(0x7f345c0b1a50 msgr2=0x7f345c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f34630ee640 1 -- 172.21.15.113:0/3769848547 shutdown_connections 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.727+0000 7f34630ee640 1 -- 172.21.15.113:0/3769848547 wait complete. 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.728+0000 7f34630ee640 1 Processor -- start 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.728+0000 7f34630ee640 1 -- start start 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f34630ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c0051c0 0x7f345c158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f34630ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c156400 0x7f345c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f34630ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345c148cc0 0x7f345c149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f34630ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f345c072720 con 0x7f345c148cc0 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f34630ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f345c0b35d0 con 0x7f344c0051c0 2024-04-25T00:05:13.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f34630ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f345c15aa90 con 0x7f345c156400 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f346168b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345c148cc0 0x7f345c149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f346168b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345c148cc0 0x7f345c149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40348/0 (socket says 172.21.15.113:40348) 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f346168b640 1 -- 172.21.15.113:0/4064601803 learned_addr learned my addr 172.21.15.113:0/4064601803 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.729+0000 7f3460e8a640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c0051c0 0x7f345c158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f345bfff640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c156400 0x7f345c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f346168b640 1 -- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c156400 msgr2=0x7f345c156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f346168b640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c156400 0x7f345c156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f346168b640 1 -- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c0051c0 msgr2=0x7f345c158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f346168b640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c0051c0 0x7f345c158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f346168b640 1 -- 172.21.15.113:0/4064601803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c149680 con 0x7f345c148cc0 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f3460e8a640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c0051c0 0x7f345c158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f345bfff640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c156400 0x7f345c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:13.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f346168b640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345c148cc0 0x7f345c149140 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f345402af10 tx=0x7f3454009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f34597fa640 1 -- 172.21.15.113:0/4064601803 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454040070 con 0x7f345c148cc0 2024-04-25T00:05:13.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f34597fa640 1 -- 172.21.15.113:0/4064601803 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345403c680 con 0x7f345c148cc0 2024-04-25T00:05:13.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f34597fa640 1 -- 172.21.15.113:0/4064601803 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454035030 con 0x7f345c148cc0 2024-04-25T00:05:13.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c149af0 con 0x7f345c148cc0 2024-04-25T00:05:13.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.730+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c157250 con 0x7f345c148cc0 2024-04-25T00:05:13.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.731+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c0cc370 con 0x7f345c148cc0 2024-04-25T00:05:13.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.731+0000 7f34597fa640 1 -- 172.21.15.113:0/4064601803 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3454043070 con 0x7f345c148cc0 2024-04-25T00:05:13.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.732+0000 7f34597fa640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f344006be30 0x7f344006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:13.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.732+0000 7f34597fa640 1 -- 172.21.15.113:0/4064601803 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f34540abf20 con 0x7f345c148cc0 2024-04-25T00:05:13.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.732+0000 7f3460e8a640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f344006be30 0x7f344006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:13.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.732+0000 7f3460e8a640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f344006be30 0x7f344006e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f345c1f35e0 tx=0x7f3450009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:13.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.736+0000 7f34597fa640 1 -- 172.21.15.113:0/4064601803 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f345c0cc370 con 0x7f345c148cc0 2024-04-25T00:05:13.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.973+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f345c157f10 con 0x7f345c148cc0 2024-04-25T00:05:13.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.973+0000 7f34597fa640 1 -- 172.21.15.113:0/4064601803 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7f345407a280 con 0x7f345c148cc0 2024-04-25T00:05:13.975 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:13.975 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":438,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561799,"bytes_used":2568093696,"bytes_avail":770526019584,"bytes_total":773094113280,"write_bytes_sec":48431362,"read_op_per_sec":0,"write_op_per_sec":47},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T00:05:11.192556+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f344006be30 msgr2=0x7f344006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f344006be30 0x7f344006e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f345c1f35e0 tx=0x7f3450009040 comp rx=0 tx=0).stop 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345c148cc0 msgr2=0x7f345c149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345c148cc0 0x7f345c149140 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f345402af10 tx=0x7f3454009750 comp rx=0 tx=0).stop 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 shutdown_connections 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f344006be30 0x7f344006e2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345c148cc0 0x7f345c149140 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c156400 0x7f345c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 --2- 172.21.15.113:0/4064601803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f344c0051c0 0x7f345c158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:13.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 >> 172.21.15.113:0/4064601803 conn(0x7f345c0b1a50 msgr2=0x7f345c0b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 shutdown_connections 2024-04-25T00:05:13.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:13.975+0000 7f34630ee640 1 -- 172.21.15.113:0/4064601803 wait complete. 2024-04-25T00:05:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T00:05:14.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.087+0000 7f6b38329640 1 Processor -- start 2024-04-25T00:05:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.087+0000 7f6b38329640 1 -- start start 2024-04-25T00:05:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.088+0000 7f6b38329640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6b3015e320 0x7f6b300d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.088+0000 7f6b38329640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b300d0b00 0x7f6b300d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.088+0000 7f6b38329640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b300d3de0 con 0x7f6b300d2290 2024-04-25T00:05:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.088+0000 7f6b38329640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6b300d14d0 con 0x7f6b3015e320 2024-04-25T00:05:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.088+0000 7f6b38329640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b300d3410 con 0x7f6b300d0b00 2024-04-25T00:05:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.088+0000 7f6b358c4640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6b300d2290 0x7f6b300b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47382/0 (socket says 172.21.15.113:47382) 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.088+0000 7f6b358c4640 1 -- 172.21.15.113:0/3528418483 learned_addr learned my addr 172.21.15.113:0/3528418483 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b368c6640 1 --2- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b300d0b00 0x7f6b300d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b360c5640 1 --2- 172.21.15.113:0/3528418483 >> v2:172.21.15.178:3300/0 conn(0x7f6b3015e320 0x7f6b300d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b350c3640 1 -- 172.21.15.113:0/3528418483 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359142291 0 0) 0x7f6b300d3de0 con 0x7f6b300d2290 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b350c3640 1 -- 172.21.15.113:0/3528418483 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b1c003520 con 0x7f6b300d2290 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b360c5640 1 -- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b300d0b00 msgr2=0x7f6b300d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b360c5640 1 --2- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b300d0b00 0x7f6b300d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b350c3640 1 -- 172.21.15.113:0/3528418483 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2025341879 0 0) 0x7f6b1c003520 con 0x7f6b300d2290 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b360c5640 1 -- 172.21.15.113:0/3528418483 >> v1:172.21.15.113:6789/0 conn(0x7f6b300d2290 legacy=0x7f6b300b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.089+0000 7f6b360c5640 1 -- 172.21.15.113:0/3528418483 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b300d2800 con 0x7f6b3015e320 2024-04-25T00:05:15.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b360c5640 1 --2- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3015e320 0x7f6b300d38a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b2802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a336753d8bbf10e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b350c3640 1 -- 172.21.15.113:0/3528418483 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28037070 con 0x7f6b3015e320 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b350c3640 1 -- 172.21.15.113:0/3528418483 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2802ac30 con 0x7f6b3015e320 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b350c3640 1 -- 172.21.15.113:0/3528418483 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20001670 con 0x7f6b3015e320 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 -- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3015e320 msgr2=0x7f6b300d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 --2- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3015e320 0x7f6b300d38a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b2802a1a0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 -- 172.21.15.113:0/3528418483 shutdown_connections 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 --2- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b300d0b00 0x7f6b300d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 --2- 172.21.15.113:0/3528418483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3015e320 0x7f6b300d38a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 -- 172.21.15.113:0/3528418483 >> 172.21.15.113:0/3528418483 conn(0x7f6b300b5c20 msgr2=0x7f6b300b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 -- 172.21.15.113:0/3528418483 shutdown_connections 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.090+0000 7f6b38329640 1 -- 172.21.15.113:0/3528418483 wait complete. 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 Processor -- start 2024-04-25T00:05:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 -- start start 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b300d0b00 0x7f6b3014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b300d2290 0x7f6b3015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b3015e320 0x7f6b300cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300d30a0 con 0x7f6b300d0b00 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300d2dc0 con 0x7f6b300d2290 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b38329640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3015e700 con 0x7f6b3015e320 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b360c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b300d0b00 0x7f6b3014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b360c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b300d0b00 0x7f6b3014a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40356/0 (socket says 172.21.15.113:40356) 2024-04-25T00:05:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.092+0000 7f6b360c5640 1 -- 172.21.15.113:0/756012034 learned_addr learned my addr 172.21.15.113:0/756012034 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b358c4640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b300d2290 0x7f6b3015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b360c5640 1 -- 172.21.15.113:0/756012034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b3015e320 msgr2=0x7f6b300cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b360c5640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b3015e320 0x7f6b300cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b360c5640 1 -- 172.21.15.113:0/756012034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b300d2290 msgr2=0x7f6b3015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b360c5640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b300d2290 0x7f6b3015f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b360c5640 1 -- 172.21.15.113:0/756012034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3014a920 con 0x7f6b300d0b00 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b360c5640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b300d0b00 0x7f6b3014a3e0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6b28035040 tx=0x7f6b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b26ffd640 1 -- 172.21.15.113:0/756012034 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28037050 con 0x7f6b300d0b00 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b26ffd640 1 -- 172.21.15.113:0/756012034 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b28043070 con 0x7f6b300d0b00 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b26ffd640 1 -- 172.21.15.113:0/756012034 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28006e10 con 0x7f6b300d0b00 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b30156e20 con 0x7f6b300d0b00 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.093+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b301573e0 con 0x7f6b300d0b00 2024-04-25T00:05:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.094+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b300691a0 con 0x7f6b300d0b00 2024-04-25T00:05:15.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.094+0000 7f6b26ffd640 1 -- 172.21.15.113:0/756012034 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b2803ca10 con 0x7f6b300d0b00 2024-04-25T00:05:15.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.095+0000 7f6b26ffd640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b1406be30 0x7f6b1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.095+0000 7f6b26ffd640 1 -- 172.21.15.113:0/756012034 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f6b280ab8b0 con 0x7f6b300d0b00 2024-04-25T00:05:15.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.095+0000 7f6b358c4640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b1406be30 0x7f6b1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.095+0000 7f6b358c4640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b1406be30 0x7f6b1406e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6b200017e0 tx=0x7f6b20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:15.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.099+0000 7f6b26ffd640 1 -- 172.21.15.113:0/756012034 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b28030e20 con 0x7f6b300d0b00 2024-04-25T00:05:15.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.307+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b300b4ad0 con 0x7f6b300d0b00 2024-04-25T00:05:15.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.308+0000 7f6b26ffd640 1 -- 172.21.15.113:0/756012034 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7f6b2807a380 con 0x7f6b300d0b00 2024-04-25T00:05:15.401 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:15.401 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.309+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b1406be30 msgr2=0x7f6b1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.309+0000 7f6b38329640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b1406be30 0x7f6b1406e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6b200017e0 tx=0x7f6b20009040 comp rx=0 tx=0).stop 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.309+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b300d0b00 msgr2=0x7f6b3014a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.309+0000 7f6b38329640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b300d0b00 0x7f6b3014a3e0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6b28035040 tx=0x7f6b280046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 shutdown_connections 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b1406be30 0x7f6b1406e2f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b300d2290 0x7f6b3015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b3015e320 0x7f6b300cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 --2- 172.21.15.113:0/756012034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b300d0b00 0x7f6b3014a3e0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 >> 172.21.15.113:0/756012034 conn(0x7f6b300b5c20 msgr2=0x7f6b300b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:15.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 shutdown_connections 2024-04-25T00:05:15.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.310+0000 7f6b38329640 1 -- 172.21.15.113:0/756012034 wait complete. 2024-04-25T00:05:15.404 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:15.404 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:15.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.500+0000 7fbcd7212640 1 Processor -- start 2024-04-25T00:05:15.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.500+0000 7fbcd7212640 1 -- start start 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbcd7212640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbcd0144910 0x7fbcd00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbcd7212640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 0x7fbcd00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbcd7212640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd0144cf0 con 0x7fbcd00b5240 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbcd7212640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd00b4510 con 0x7fbcd0144910 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbcd7212640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00b2f30 con 0x7fbcd00b56b0 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbcd00b5240 0x7fbcd006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47392/0 (socket says 172.21.15.113:47392) 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccffff640 1 -- 172.21.15.113:0/960504587 learned_addr learned my addr 172.21.15.113:0/960504587 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:15.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbcd57af640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 0x7fbcd00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbcd4fae640 1 --2- 172.21.15.113:0/960504587 >> v2:172.21.15.178:3300/0 conn(0x7fbcd0144910 0x7fbcd00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 825342446 0 0) 0x7fbcd0144cf0 con 0x7fbcd00b5240 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcc0005730 con 0x7fbcd00b5240 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1461326273 0 0) 0x7fbcc0005730 con 0x7fbcd00b5240 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 msgr2=0x7fbcd00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 0x7fbcd00cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 >> v2:172.21.15.178:3300/0 conn(0x7fbcd0144910 msgr2=0x7fbcd00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 --2- 172.21.15.113:0/960504587 >> v2:172.21.15.178:3300/0 conn(0x7fbcd0144910 0x7fbcd00b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd00b3210 con 0x7fbcd00b5240 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fbcbc001670 con 0x7fbcd00b5240 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 >> v1:172.21.15.113:6789/0 conn(0x7fbcd00b5240 legacy=0x7fbcd006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcc0005bc0 0x7fbcc0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 0x7fbcc0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcc0006f50 0x7fbcc00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00b4510 con 0x7fbcc0006f50 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00b2f30 con 0x7fbcc0005bc0 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.501+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc000dbd0 con 0x7fbcd00b56b0 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbcd57af640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcc0006f50 0x7fbcc00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbcd4fae640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcc0005bc0 0x7fbcc0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbccffff640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 0x7fbcc0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbcd57af640 1 -- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 msgr2=0x7fbcc0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbcd57af640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00b56b0 0x7fbcc0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbcd57af640 1 -- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcc0005bc0 msgr2=0x7fbcc0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbcd57af640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcc0005bc0 0x7fbcc0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.502+0000 7fbcd57af640 1 -- 172.21.15.113:0/960504587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc0005730 con 0x7fbcc0006f50 2024-04-25T00:05:15.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd57af640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcc0006f50 0x7fbcc00073d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009f90 tx=0x7fbcc0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf99a4fda2160c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:15.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8009c10 con 0x7fbcc0006f50 2024-04-25T00:05:15.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc8036070 con 0x7fbcc0006f50 2024-04-25T00:05:15.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbccf7fe640 1 -- 172.21.15.113:0/960504587 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8031850 con 0x7fbcc0006f50 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd7212640 1 -- 172.21.15.113:0/960504587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcc0006f50 msgr2=0x7fbcc00073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd7212640 1 --2- 172.21.15.113:0/960504587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcc0006f50 0x7fbcc00073d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009f90 tx=0x7fbcc0007a40 comp rx=0 tx=0).stop 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd57af640 1 -- 172.21.15.113:0/960504587 reap_dead start 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd7212640 1 -- 172.21.15.113:0/960504587 shutdown_connections 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd7212640 1 -- 172.21.15.113:0/960504587 >> 172.21.15.113:0/960504587 conn(0x7fbcd00b1a50 msgr2=0x7fbcd00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd7212640 1 -- 172.21.15.113:0/960504587 shutdown_connections 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.503+0000 7fbcd7212640 1 -- 172.21.15.113:0/960504587 wait complete. 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.504+0000 7fbcd7212640 1 Processor -- start 2024-04-25T00:05:15.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.504+0000 7fbcd7212640 1 -- start start 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd7212640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcd00ae920 0x7fbcd013e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd7212640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00aed00 0x7fbcd013c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd7212640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcd0142e30 0x7fbcd01432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd7212640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00b4da0 con 0x7fbcd00ae920 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd7212640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00b37d0 con 0x7fbcd0142e30 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd7212640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00b4f20 con 0x7fbcd00aed00 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd4fae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcd00ae920 0x7fbcd013e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd4fae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcd00ae920 0x7fbcd013e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40374/0 (socket says 172.21.15.113:40374) 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.505+0000 7fbcd4fae640 1 -- 172.21.15.113:0/851070096 learned_addr learned my addr 172.21.15.113:0/851070096 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbccffff640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00aed00 0x7fbcd013c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd57af640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcd0142e30 0x7fbcd01432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd4fae640 1 -- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00aed00 msgr2=0x7fbcd013c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd4fae640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00aed00 0x7fbcd013c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd4fae640 1 -- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcd0142e30 msgr2=0x7fbcd01432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd4fae640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcd0142e30 0x7fbcd01432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd4fae640 1 -- 172.21.15.113:0/851070096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd013eda0 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd4fae640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcd00ae920 0x7fbcd013e750 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4011960 tx=0x7fbcc4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbccd7fa640 1 -- 172.21.15.113:0/851070096 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400faf0 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbccd7fa640 1 -- 172.21.15.113:0/851070096 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc400fc90 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbccd7fa640 1 -- 172.21.15.113:0/851070096 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4014440 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd0138d90 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.506+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd0139350 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.507+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd01437f0 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.507+0000 7fbccd7fa640 1 -- 172.21.15.113:0/851070096 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcc4012040 con 0x7fbcd00ae920 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.508+0000 7fbccd7fa640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcb406be60 0x7fbcb406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.508+0000 7fbccffff640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcb406be60 0x7fbcb406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.508+0000 7fbccd7fa640 1 -- 172.21.15.113:0/851070096 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fbcc408b4d0 con 0x7fbcd00ae920 2024-04-25T00:05:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.508+0000 7fbccffff640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcb406be60 0x7fbcb406e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc004840 tx=0x7fbcbc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.512+0000 7fbccd7fa640 1 -- 172.21.15.113:0/851070096 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbcc4019030 con 0x7fbcd00ae920 2024-04-25T00:05:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.751+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbcd00691a0 con 0x7fbcd00ae920 2024-04-25T00:05:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.752+0000 7fbccd7fa640 1 -- 172.21.15.113:0/851070096 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7fbcc4089e20 con 0x7fbcd00ae920 2024-04-25T00:05:15.753 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:15.753 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":440,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561799,"bytes_used":2568093696,"bytes_avail":770526019584,"bytes_total":773094113280,"write_bytes_sec":48431362,"read_op_per_sec":0,"write_op_per_sec":47},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T00:05:11.192556+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:15.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcb406be60 msgr2=0x7fbcb406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcb406be60 0x7fbcb406e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc004840 tx=0x7fbcbc009040 comp rx=0 tx=0).stop 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcd00ae920 msgr2=0x7fbcd013e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcd00ae920 0x7fbcd013e750 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4011960 tx=0x7fbcc4011e20 comp rx=0 tx=0).stop 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 shutdown_connections 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbcb406be60 0x7fbcb406e320 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcd0142e30 0x7fbcd01432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcd00aed00 0x7fbcd013c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 --2- 172.21.15.113:0/851070096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcd00ae920 0x7fbcd013e750 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 >> 172.21.15.113:0/851070096 conn(0x7fbcd00b1a50 msgr2=0x7fbcd00b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 shutdown_connections 2024-04-25T00:05:15.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:15.754+0000 7fbcd7212640 1 -- 172.21.15.113:0/851070096 wait complete. 2024-04-25T00:05:15.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T00:05:16.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.874+0000 7f1e65914640 1 Processor -- start 2024-04-25T00:05:16.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.874+0000 7f1e65914640 1 -- start start 2024-04-25T00:05:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e65914640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1e60144910 0x7f1e600b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e65914640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e60144cf0 con 0x7f1e600b56b0 2024-04-25T00:05:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e65914640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e600b2f30 con 0x7f1e600b5240 2024-04-25T00:05:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e65914640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1e600b4510 con 0x7f1e60144910 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e5f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1e600b56b0 0x7f1e600cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47402/0 (socket says 172.21.15.113:47402) 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e5f7fe640 1 -- 172.21.15.113:0/2946630495 learned_addr learned my addr 172.21.15.113:0/2946630495 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2946630495 >> v2:172.21.15.178:3300/0 conn(0x7f1e60144910 0x7f1e600b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926063815 0 0) 0x7f1e60144cf0 con 0x7f1e600b56b0 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.875+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e48003520 con 0x7f1e600b56b0 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569159578 0 0) 0x7f1e600b2f30 con 0x7f1e600b5240 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e60144cf0 con 0x7f1e600b5240 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2731410944 0 0) 0x7f1e48003520 con 0x7f1e600b56b0 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 >> v2:172.21.15.178:3300/0 conn(0x7f1e60144910 msgr2=0x7f1e600b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 --2- 172.21.15.113:0/2946630495 >> v2:172.21.15.178:3300/0 conn(0x7f1e60144910 0x7f1e600b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 >> v1:172.21.15.178:6789/0 conn(0x7f1e600b5240 legacy=0x7f1e6006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e600b3210 con 0x7f1e600b56b0 2024-04-25T00:05:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1e580019f0 con 0x7f1e600b56b0 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 >> v1:172.21.15.113:6789/0 conn(0x7f1e600b56b0 legacy=0x7f1e600cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e48003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e48004800 0x7f1e48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e480051c0 0x7f1e4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e600b4510 con 0x7f1e48003b40 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4800c060 con 0x7f1e480051c0 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4800c260 con 0x7f1e48004800 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e48003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5f7fe640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e480051c0 0x7f1e4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5e7fc640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e48004800 0x7f1e48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5effd640 1 -- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e48004800 msgr2=0x7f1e48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e48004800 0x7f1e48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5effd640 1 -- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e480051c0 msgr2=0x7f1e4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e480051c0 0x7f1e4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5effd640 1 -- 172.21.15.113:0/2946630495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e48003520 con 0x7f1e48003b40 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.876+0000 7f1e5f7fe640 1 -- 172.21.15.113:0/2946630495 reap_dead start 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.877+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e48003f40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009ac0 tx=0x7f1e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a7d7c967f0e1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.877+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50036070 con 0x7f1e48003b40 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.877+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e50031400 con 0x7f1e48003b40 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.877+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2946630495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50009690 con 0x7f1e48003b40 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.878+0000 7f1e65914640 1 -- 172.21.15.113:0/2946630495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 msgr2=0x7f1e48003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.878+0000 7f1e65914640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e48003f40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009ac0 tx=0x7f1e48005830 comp rx=0 tx=0).stop 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.878+0000 7f1e65914640 1 -- 172.21.15.113:0/2946630495 shutdown_connections 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.878+0000 7f1e65914640 1 --2- 172.21.15.113:0/2946630495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e48003f40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.878+0000 7f1e65914640 1 -- 172.21.15.113:0/2946630495 >> 172.21.15.113:0/2946630495 conn(0x7f1e600b1a50 msgr2=0x7f1e600b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:16.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.878+0000 7f1e65914640 1 -- 172.21.15.113:0/2946630495 shutdown_connections 2024-04-25T00:05:16.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.878+0000 7f1e65914640 1 -- 172.21.15.113:0/2946630495 wait complete. 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.879+0000 7f1e65914640 1 Processor -- start 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.879+0000 7f1e65914640 1 -- start start 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e65914640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e600ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e65914640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6015d460 0x7f1e60141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e65914640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e600ac120 0x7f1e600ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e65914640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e600b4da0 con 0x7f1e48003b40 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e65914640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e600b37d0 con 0x7f1e600ac120 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e65914640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e600b4f20 con 0x7f1e6015d460 2024-04-25T00:05:16.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e600ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e600ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40396/0 (socket says 172.21.15.113:40396) 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 -- 172.21.15.113:0/2737929969 learned_addr learned my addr 172.21.15.113:0/2737929969 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5e7fc640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6015d460 0x7f1e60141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5f7fe640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e600ac120 0x7f1e600ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 -- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6015d460 msgr2=0x7f1e60141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6015d460 0x7f1e60141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 -- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e600ac120 msgr2=0x7f1e600ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e600ac120 0x7f1e600ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 -- 172.21.15.113:0/2737929969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e600ac6e0 con 0x7f1e48003b40 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e5effd640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e600ad780 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f1e5002abb0 tx=0x7f1e50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e47fff640 1 -- 172.21.15.113:0/2737929969 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5003f020 con 0x7f1e48003b40 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e47fff640 1 -- 172.21.15.113:0/2737929969 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e50037040 con 0x7f1e48003b40 2024-04-25T00:05:16.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e47fff640 1 -- 172.21.15.113:0/2737929969 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50036070 con 0x7f1e48003b40 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.880+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e600ac9c0 con 0x7f1e48003b40 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.881+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e600acf80 con 0x7f1e48003b40 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.882+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e600691a0 con 0x7f1e48003b40 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.882+0000 7f1e47fff640 1 -- 172.21.15.113:0/2737929969 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e50042020 con 0x7f1e48003b40 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.882+0000 7f1e47fff640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be60 0x7f1e4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.883+0000 7f1e47fff640 1 -- 172.21.15.113:0/2737929969 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f1e500aaee0 con 0x7f1e48003b40 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.883+0000 7f1e5e7fc640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be60 0x7f1e4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:16.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.883+0000 7f1e5e7fc640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be60 0x7f1e4c06e320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1e54000c00 tx=0x7f1e54009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:16.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:16.887+0000 7f1e47fff640 1 -- 172.21.15.113:0/2737929969 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e50049030 con 0x7f1e48003b40 2024-04-25T00:05:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.099+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e600731a0 con 0x7f1e48003b40 2024-04-25T00:05:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.100+0000 7f1e47fff640 1 -- 172.21.15.113:0/2737929969 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7f1e50079960 con 0x7f1e48003b40 2024-04-25T00:05:17.100 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:17.100 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:17.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be60 msgr2=0x7f1e4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be60 0x7f1e4c06e320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1e54000c00 tx=0x7f1e54009040 comp rx=0 tx=0).stop 2024-04-25T00:05:17.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 msgr2=0x7f1e600ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e600ad780 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f1e5002abb0 tx=0x7f1e50004730 comp rx=0 tx=0).stop 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 shutdown_connections 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be60 0x7f1e4c06e320 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e600ac120 0x7f1e600ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6015d460 0x7f1e60141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 --2- 172.21.15.113:0/2737929969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e48003b40 0x7f1e600ad780 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 >> 172.21.15.113:0/2737929969 conn(0x7f1e600b1a50 msgr2=0x7f1e6006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 shutdown_connections 2024-04-25T00:05:17.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.102+0000 7f1e65914640 1 -- 172.21.15.113:0/2737929969 wait complete. 2024-04-25T00:05:17.113 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:17.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:17.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.208+0000 7fcbd0076640 1 Processor -- start 2024-04-25T00:05:17.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.208+0000 7fcbd0076640 1 -- start start 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbd0076640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcbc806e6e0 0x7fcbc80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbd0076640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc8144cf0 con 0x7fcbc80b3fd0 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbd0076640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc80b2f30 con 0x7fcbc8144910 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbd0076640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc80b3210 con 0x7fcbc806e6e0 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcd611640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcbc80b3fd0 0x7fcbc806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47404/0 (socket says 172.21.15.113:47404) 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcd611640 1 -- 172.21.15.113:0/3150299701 learned_addr learned my addr 172.21.15.113:0/3150299701 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3070459135 0 0) 0x7fcbc8144cf0 con 0x7fcbc80b3fd0 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbce613640 1 --2- 172.21.15.113:0/3150299701 >> v2:172.21.15.178:3300/0 conn(0x7fcbc806e6e0 0x7fcbc80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbb4005730 con 0x7fcbc80b3fd0 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 764157063 0 0) 0x7fcbb4005730 con 0x7fcbc80b3fd0 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 >> v2:172.21.15.178:3300/0 conn(0x7fcbc806e6e0 msgr2=0x7fcbc80cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 --2- 172.21.15.113:0/3150299701 >> v2:172.21.15.178:3300/0 conn(0x7fcbc806e6e0 0x7fcbc80cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 >> v1:172.21.15.178:6789/0 conn(0x7fcbc8144910 legacy=0x7fcbc80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc80b34f0 con 0x7fcbc80b3fd0 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fcbb8001670 con 0x7fcbc80b3fd0 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.209+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 >> v1:172.21.15.113:6789/0 conn(0x7fcbc80b3fd0 legacy=0x7fcbc806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcce10640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb4005c10 0x7fcbb4006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcce10640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb4006a10 0x7fcbb4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcce10640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40073d0 0x7fcbb400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc80b3210 con 0x7fcbb4005c10 2024-04-25T00:05:17.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb400e300 con 0x7fcbb4006a10 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb400e500 con 0x7fcbb40073d0 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcde12640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb4005c10 0x7fcbb4006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbce613640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40073d0 0x7fcbb400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcd611640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb4006a10 0x7fcbb4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcde12640 1 -- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40073d0 msgr2=0x7fcbb400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcde12640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40073d0 0x7fcbb400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcde12640 1 -- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb4006a10 msgr2=0x7fcbb4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcde12640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb4006a10 0x7fcbb4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbce613640 1 -- 172.21.15.113:0/3150299701 reap_dead start 2024-04-25T00:05:17.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.210+0000 7fcbcde12640 1 -- 172.21.15.113:0/3150299701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb4005730 con 0x7fcbb4005c10 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbcde12640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb4005c10 0x7fcbb4006030 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4001750 tx=0x7fcbb4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6c460b2925eab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4001370 con 0x7fcbb4005c10 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc40052d0 con 0x7fcbb4005c10 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbcce10640 1 -- 172.21.15.113:0/3150299701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc402abe0 con 0x7fcbb4005c10 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbd0076640 1 -- 172.21.15.113:0/3150299701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb4005c10 msgr2=0x7fcbb4006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbd0076640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb4005c10 0x7fcbb4006030 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4001750 tx=0x7fcbb4007a40 comp rx=0 tx=0).stop 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbd0076640 1 -- 172.21.15.113:0/3150299701 shutdown_connections 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbd0076640 1 --2- 172.21.15.113:0/3150299701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb4005c10 0x7fcbb4006030 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbd0076640 1 -- 172.21.15.113:0/3150299701 >> 172.21.15.113:0/3150299701 conn(0x7fcbc80b1a50 msgr2=0x7fcbc80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbd0076640 1 -- 172.21.15.113:0/3150299701 shutdown_connections 2024-04-25T00:05:17.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.211+0000 7fcbd0076640 1 -- 172.21.15.113:0/3150299701 wait complete. 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 Processor -- start 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 -- start start 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb4005c10 0x7fcbc8134e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc8141fb0 0x7fcbc80ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc80cb9f0 0x7fcbc813a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc80b4da0 con 0x7fcbc8141fb0 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc80b4440 con 0x7fcbc80cb9f0 2024-04-25T00:05:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbd0076640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc80b4f20 con 0x7fcbb4005c10 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbcd611640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc8141fb0 0x7fcbc80ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbcd611640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc8141fb0 0x7fcbc80ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40406/0 (socket says 172.21.15.113:40406) 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.213+0000 7fcbcd611640 1 -- 172.21.15.113:0/2209693294 learned_addr learned my addr 172.21.15.113:0/2209693294 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcde12640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb4005c10 0x7fcbc8134e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbce613640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc80cb9f0 0x7fcbc813a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcd611640 1 -- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb4005c10 msgr2=0x7fcbc8134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcd611640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb4005c10 0x7fcbc8134e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcd611640 1 -- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc80cb9f0 msgr2=0x7fcbc813a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcd611640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc80cb9f0 0x7fcbc813a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcd611640 1 -- 172.21.15.113:0/2209693294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc81f87f0 con 0x7fcbc8141fb0 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcde12640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb4005c10 0x7fcbc8134e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:17.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbcd611640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc8141fb0 0x7fcbc80ce7f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80072c0 tx=0x7fcbb80076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbbe7fc640 1 -- 172.21.15.113:0/2209693294 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800f940 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbbe7fc640 1 -- 172.21.15.113:0/2209693294 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb8018070 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbbe7fc640 1 -- 172.21.15.113:0/2209693294 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800fd40 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc81f7c70 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.214+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81f8230 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.215+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc80691a0 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.215+0000 7fcbbe7fc640 1 -- 172.21.15.113:0/2209693294 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbb801d460 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.216+0000 7fcbbe7fc640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcbac06be60 0x7fcbac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.216+0000 7fcbbe7fc640 1 -- 172.21.15.113:0/2209693294 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fcbb808ada0 con 0x7fcbc8141fb0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.216+0000 7fcbcde12640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcbac06be60 0x7fcbac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.216+0000 7fcbcde12640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcbac06be60 0x7fcbac06e320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4032000 tx=0x7fcbc402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:17.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.220+0000 7fcbbe7fc640 1 -- 172.21.15.113:0/2209693294 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcbb80597e0 con 0x7fcbc8141fb0 2024-04-25T00:05:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.461+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fcbc80731a0 con 0x7fcbc8141fb0 2024-04-25T00:05:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.461+0000 7fcbbe7fc640 1 -- 172.21.15.113:0/2209693294 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7fcbb80590e0 con 0x7fcbc8141fb0 2024-04-25T00:05:17.461 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:17.462 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":442,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561799,"bytes_used":2568093696,"bytes_avail":770526019584,"bytes_total":773094113280,"write_bytes_sec":48432098,"read_op_per_sec":0,"write_op_per_sec":47},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T00:05:11.192556+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcbac06be60 msgr2=0x7fcbac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcbac06be60 0x7fcbac06e320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4032000 tx=0x7fcbc402e000 comp rx=0 tx=0).stop 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc8141fb0 msgr2=0x7fcbc80ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc8141fb0 0x7fcbc80ce7f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80072c0 tx=0x7fcbb80076c0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 shutdown_connections 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcbac06be60 0x7fcbac06e320 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc80cb9f0 0x7fcbc813a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc8141fb0 0x7fcbc80ce7f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 --2- 172.21.15.113:0/2209693294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb4005c10 0x7fcbc8134e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 >> 172.21.15.113:0/2209693294 conn(0x7fcbc80b1a50 msgr2=0x7fcbc8071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 shutdown_connections 2024-04-25T00:05:17.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:17.463+0000 7fcbd0076640 1 -- 172.21.15.113:0/2209693294 wait complete. 2024-04-25T00:05:17.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T00:05:18.475 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:18.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.584+0000 7fe3fff5a640 1 Processor -- start 2024-04-25T00:05:18.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.584+0000 7fe3fff5a640 1 -- start start 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fff5a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3f8144d60 0x7fe3f80b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fff5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe3f81453c0 0x7fe3f806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fff5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f806e510 0x7fe3f815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fff5a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe3f8145c40 con 0x7fe3f8144d60 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fff5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe3f80cfad0 con 0x7fe3f81453c0 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fff5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f806e920 con 0x7fe3f806e510 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fdcf6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3f8144d60 0x7fe3f80b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fdcf6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3f8144d60 0x7fe3f80b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53748/0 (socket says 172.21.15.113:53748) 2024-04-25T00:05:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fdcf6640 1 -- 172.21.15.113:0/593793294 learned_addr learned my addr 172.21.15.113:0/593793294 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:18.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fe4f7640 1 --2- 172.21.15.113:0/593793294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f806e510 0x7fe3f815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fd4f5640 1 --2- 172.21.15.113:0/593793294 >> v2:172.21.15.178:3300/0 conn(0x7fe3f81453c0 0x7fe3f806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fdcf6640 1 -- 172.21.15.113:0/593793294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f806e510 msgr2=0x7fe3f815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:18.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fdcf6640 1 --2- 172.21.15.113:0/593793294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f806e510 0x7fe3f815fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fdcf6640 1 -- 172.21.15.113:0/593793294 >> v2:172.21.15.178:3300/0 conn(0x7fe3f81453c0 msgr2=0x7fe3f806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:18.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.585+0000 7fe3fdcf6640 1 --2- 172.21.15.113:0/593793294 >> v2:172.21.15.178:3300/0 conn(0x7fe3f81453c0 0x7fe3f806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.586+0000 7fe3fdcf6640 1 -- 172.21.15.113:0/593793294 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f80cf230 con 0x7fe3f8144d60 2024-04-25T00:05:18.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.586+0000 7fe3fdcf6640 1 --2- 172.21.15.113:0/593793294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f80b5390 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80ce060 tx=0x7fe3f402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5e6924cd19ba11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.586+0000 7fe3fccf4640 1 -- 172.21.15.113:0/593793294 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4036070 con 0x7fe3f8144d60 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.586+0000 7fe3fccf4640 1 -- 172.21.15.113:0/593793294 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f4031600 con 0x7fe3f8144d60 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.586+0000 7fe3fccf4640 1 -- 172.21.15.113:0/593793294 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40319d0 con 0x7fe3f8144d60 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 -- 172.21.15.113:0/593793294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 msgr2=0x7fe3f80b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/593793294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f80b5390 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80ce060 tx=0x7fe3f402a610 comp rx=0 tx=0).stop 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 -- 172.21.15.113:0/593793294 shutdown_connections 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/593793294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f806e510 0x7fe3f815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/593793294 >> v2:172.21.15.178:3300/0 conn(0x7fe3f81453c0 0x7fe3f806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/593793294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f80b5390 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 -- 172.21.15.113:0/593793294 >> 172.21.15.113:0/593793294 conn(0x7fe3f80b1a50 msgr2=0x7fe3f80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:18.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 -- 172.21.15.113:0/593793294 shutdown_connections 2024-04-25T00:05:18.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.587+0000 7fe3fff5a640 1 -- 172.21.15.113:0/593793294 wait complete. 2024-04-25T00:05:18.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.588+0000 7fe3fff5a640 1 Processor -- start 2024-04-25T00:05:18.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.588+0000 7fe3fff5a640 1 -- start start 2024-04-25T00:05:18.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fff5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3f806e510 0x7fe3f8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fff5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fff5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f81453c0 0x7fe3f815dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fff5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8070e10 con 0x7fe3f8144d60 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fff5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f80cf7f0 con 0x7fe3f806e510 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fff5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f8145140 con 0x7fe3f81453c0 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f815ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53756/0 (socket says 172.21.15.113:53756) 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 -- 172.21.15.113:0/3546881877 learned_addr learned my addr 172.21.15.113:0/3546881877 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fe4f7640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f81453c0 0x7fe3f815dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fdcf6640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3f806e510 0x7fe3f8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 -- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f81453c0 msgr2=0x7fe3f815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f81453c0 0x7fe3f815dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 -- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3f806e510 msgr2=0x7fe3f8159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3f806e510 0x7fe3f8159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 -- 172.21.15.113:0/3546881877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f81e5c10 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.589+0000 7fe3fd4f5640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f815ad60 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d800 tx=0x7fe3e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.590+0000 7fe3ee7fc640 1 -- 172.21.15.113:0/3546881877 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800b660 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.590+0000 7fe3ee7fc640 1 -- 172.21.15.113:0/3546881877 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e8019070 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.590+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f815b8e0 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.590+0000 7fe3ee7fc640 1 -- 172.21.15.113:0/3546881877 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800ba20 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.590+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f81e5eb0 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.591+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f80691a0 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.591+0000 7fe3ee7fc640 1 -- 172.21.15.113:0/3546881877 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3e800bbc0 con 0x7fe3f8144d60 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.592+0000 7fe3ee7fc640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3dc06be30 0x7fe3dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.592+0000 7fe3fdcf6640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3dc06be30 0x7fe3dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.592+0000 7fe3ee7fc640 1 -- 172.21.15.113:0/3546881877 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fe3e808b2a0 con 0x7fe3f8144d60 2024-04-25T00:05:18.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.592+0000 7fe3fdcf6640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3dc06be30 0x7fe3dc06e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4033000 tx=0x7fe3f4009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:18.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.596+0000 7fe3ee7fc640 1 -- 172.21.15.113:0/3546881877 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3e8059c50 con 0x7fe3f8144d60 2024-04-25T00:05:18.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.806+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3f80b27e0 con 0x7fe3f8144d60 2024-04-25T00:05:18.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.807+0000 7fe3ee7fc640 1 -- 172.21.15.113:0/3546881877 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7fe3e808b0c0 con 0x7fe3f8144d60 2024-04-25T00:05:18.807 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:18.808 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:18.808 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3dc06be30 msgr2=0x7fe3dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3dc06be30 0x7fe3dc06e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4033000 tx=0x7fe3f4009000 comp rx=0 tx=0).stop 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 msgr2=0x7fe3f815ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f815ad60 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d800 tx=0x7fe3e800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 shutdown_connections 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3dc06be30 0x7fe3dc06e2f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f81453c0 0x7fe3f815dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8144d60 0x7fe3f815ad60 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 --2- 172.21.15.113:0/3546881877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3f806e510 0x7fe3f8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 >> 172.21.15.113:0/3546881877 conn(0x7fe3f80b1a50 msgr2=0x7fe3f8071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:18.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 shutdown_connections 2024-04-25T00:05:18.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.809+0000 7fe3fff5a640 1 -- 172.21.15.113:0/3546881877 wait complete. 2024-04-25T00:05:18.820 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:05:18.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.916+0000 7efc3b31b640 1 Processor -- start 2024-04-25T00:05:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.916+0000 7efc3b31b640 1 -- start start 2024-04-25T00:05:18.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc3b31b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 0x7efc340b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc3b31b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc340cfad0 con 0x7efc34144d60 2024-04-25T00:05:18.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc3b31b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc340cf230 con 0x7efc3406e6e0 2024-04-25T00:05:18.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc3b31b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc34145c40 con 0x7efc341453c0 2024-04-25T00:05:18.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc390b7640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efc34144d60 0x7efc3406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33050/0 (socket says 172.21.15.113:33050) 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc390b7640 1 -- 172.21.15.113:0/1318517140 learned_addr learned my addr 172.21.15.113:0/1318517140 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc398b8640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 0x7efc340b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851158954 0 0) 0x7efc340cfad0 con 0x7efc34144d60 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc20003520 con 0x7efc34144d60 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4178430545 0 0) 0x7efc20003520 con 0x7efc34144d60 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 msgr2=0x7efc340b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 0x7efc340b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 >> v1:172.21.15.178:6789/0 conn(0x7efc3406e6e0 legacy=0x7efc34071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc340cf510 con 0x7efc34144d60 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7efc24001670 con 0x7efc34144d60 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 >> v1:172.21.15.113:6789/0 conn(0x7efc34144d60 legacy=0x7efc3406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 0x7efc200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc20004d40 0x7efc200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efc34145c40 con 0x7efc20003f60 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc2000bc00 con 0x7efc20004d40 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc2000be00 con 0x7efc341453c0 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc388b6640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc390b7640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 0x7efc200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.917+0000 7efc398b8640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc20004d40 0x7efc200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc388b6640 1 -- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 msgr2=0x7efc200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc388b6640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc341453c0 0x7efc200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc388b6640 1 -- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc20004d40 msgr2=0x7efc200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc388b6640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc20004d40 0x7efc200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc388b6640 1 -- 172.21.15.113:0/1318517140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc20003520 con 0x7efc20003f60 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc388b6640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc20004800 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7efc30001720 tx=0x7efc20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe7f1f5c1e0452b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30001370 con 0x7efc20003f60 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc300052d0 con 0x7efc20003f60 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.918+0000 7efc2bfff640 1 -- 172.21.15.113:0/1318517140 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3002abc0 con 0x7efc20003f60 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.919+0000 7efc3b31b640 1 -- 172.21.15.113:0/1318517140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 msgr2=0x7efc20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.919+0000 7efc3b31b640 1 --2- 172.21.15.113:0/1318517140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc20004800 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7efc30001720 tx=0x7efc20005830 comp rx=0 tx=0).stop 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.919+0000 7efc398b8640 1 -- 172.21.15.113:0/1318517140 reap_dead start 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.919+0000 7efc3b31b640 1 -- 172.21.15.113:0/1318517140 shutdown_connections 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.919+0000 7efc3b31b640 1 -- 172.21.15.113:0/1318517140 >> 172.21.15.113:0/1318517140 conn(0x7efc340b1a50 msgr2=0x7efc340b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.919+0000 7efc3b31b640 1 -- 172.21.15.113:0/1318517140 shutdown_connections 2024-04-25T00:05:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.919+0000 7efc3b31b640 1 -- 172.21.15.113:0/1318517140 wait complete. 2024-04-25T00:05:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.920+0000 7efc3b31b640 1 Processor -- start 2024-04-25T00:05:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.920+0000 7efc3b31b640 1 -- start start 2024-04-25T00:05:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc3b31b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc3415b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc3b31b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc34137200 0x7efc34138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc3b31b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc34138640 0x7efc3413c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc3b31b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efc34145140 con 0x7efc20003f60 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc3b31b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc34070e10 con 0x7efc34137200 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc3b31b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc340b5390 con 0x7efc34138640 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc3415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc3415b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53780/0 (socket says 172.21.15.113:53780) 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 -- 172.21.15.113:0/4033324527 learned_addr learned my addr 172.21.15.113:0/4033324527 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc398b8640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc34138640 0x7efc3413c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc388b6640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc34137200 0x7efc34138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 -- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc34138640 msgr2=0x7efc3413c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc34138640 0x7efc3413c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 -- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc34137200 msgr2=0x7efc34138100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc34137200 0x7efc34138100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 -- 172.21.15.113:0/4033324527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc34136760 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc398b8640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc34138640 0x7efc3413c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.921+0000 7efc390b7640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc3415b800 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7efc24004f40 tx=0x7efc240074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.922+0000 7efc29ffb640 1 -- 172.21.15.113:0/4033324527 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2400f750 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.922+0000 7efc29ffb640 1 -- 172.21.15.113:0/4033324527 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc24018070 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.922+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc34138ae0 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.922+0000 7efc29ffb640 1 -- 172.21.15.113:0/4033324527 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2400fb70 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.922+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc34138dc0 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.923+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc340691a0 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.923+0000 7efc29ffb640 1 -- 172.21.15.113:0/4033324527 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc2400fd10 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.924+0000 7efc29ffb640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc1806be30 0x7efc1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.924+0000 7efc29ffb640 1 -- 172.21.15.113:0/4033324527 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7efc2408adf0 con 0x7efc20003f60 2024-04-25T00:05:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.924+0000 7efc388b6640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc1806be30 0x7efc1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.924+0000 7efc388b6640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc1806be30 0x7efc1806e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efc30032000 tx=0x7efc3002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:18.928+0000 7efc29ffb640 1 -- 172.21.15.113:0/4033324527 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc240597a0 con 0x7efc20003f60 2024-04-25T00:05:19.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.167+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7efc340b27e0 con 0x7efc20003f60 2024-04-25T00:05:19.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.167+0000 7efc29ffb640 1 -- 172.21.15.113:0/4033324527 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1575 (secure 0 0 0) 0x7efc240590a0 con 0x7efc20003f60 2024-04-25T00:05:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:19.168 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":444,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566146,"bytes_used":2568093696,"bytes_avail":770526019584,"bytes_total":773094113280,"write_bytes_sec":48432477,"read_op_per_sec":0,"write_op_per_sec":48},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7848}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T00:05:17.193610+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:19.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc1806be30 msgr2=0x7efc1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc1806be30 0x7efc1806e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efc30032000 tx=0x7efc3002e000 comp rx=0 tx=0).stop 2024-04-25T00:05:19.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 msgr2=0x7efc3415b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc3415b800 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7efc24004f40 tx=0x7efc240074f0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 shutdown_connections 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efc1806be30 0x7efc1806e2f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc34137200 0x7efc34138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc34138640 0x7efc3413c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 --2- 172.21.15.113:0/4033324527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc20003f60 0x7efc3415b800 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 >> 172.21.15.113:0/4033324527 conn(0x7efc340b1a50 msgr2=0x7efc3406cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 shutdown_connections 2024-04-25T00:05:19.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.169+0000 7efc3b31b640 1 -- 172.21.15.113:0/4033324527 wait complete. 2024-04-25T00:05:19.180 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:05:19.180 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:05:19.181 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:05:19.219 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:05:19.257 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:05:19.257 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:05:19.257 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:05:19.257 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:19.274 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:05:19.274 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:19.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:19.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.466+0000 7ffbbd49e640 1 Processor -- start 2024-04-25T00:05:19.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.467+0000 7ffbbd49e640 1 -- start start 2024-04-25T00:05:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.467+0000 7ffbbd49e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffbb80b3fd0 0x7ffbb806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.467+0000 7ffbbd49e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb806e510 0x7ffbb80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.467+0000 7ffbbd49e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb8144cf0 con 0x7ffbb8144910 2024-04-25T00:05:19.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.467+0000 7ffbbd49e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb80b2f30 con 0x7ffbb80b3fd0 2024-04-25T00:05:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.467+0000 7ffbbd49e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb80b3210 con 0x7ffbb806e510 2024-04-25T00:05:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb67fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffbb80b3fd0 0x7ffbb806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb806e510 0x7ffbb80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb6ffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ffbb8144910 0x7ffbb80b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:45838/0 (socket says 172.21.15.113:45838) 2024-04-25T00:05:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb806e510 0x7ffbb80cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:56708/0 (socket says 172.21.15.113:56708) 2024-04-25T00:05:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb6ffd640 1 -- 172.21.15.113:0/835083985 learned_addr learned my addr 172.21.15.113:0/835083985 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:19.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb5ffb640 1 -- 172.21.15.113:0/835083985 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 538063908 0 0) 0x7ffbb8144cf0 con 0x7ffbb8144910 2024-04-25T00:05:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb5ffb640 1 -- 172.21.15.113:0/835083985 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba0003520 con 0x7ffbb8144910 2024-04-25T00:05:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb67fc640 1 -- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb806e510 msgr2=0x7ffbb80cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb67fc640 1 --2- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb806e510 0x7ffbb80cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb67fc640 1 -- 172.21.15.113:0/835083985 >> v1:172.21.15.178:6789/0 conn(0x7ffbb8144910 legacy=0x7ffbb80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb67fc640 1 -- 172.21.15.113:0/835083985 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb80b34f0 con 0x7ffbb80b3fd0 2024-04-25T00:05:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.468+0000 7ffbb77fe640 1 --2- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb806e510 0x7ffbb80cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:19.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbb67fc640 1 --2- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb806dfd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00ba70 tx=0x7ffbac02c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=425e64fd7cf83474 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbb5ffb640 1 -- 172.21.15.113:0/835083985 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac034620 con 0x7ffbb80b3fd0 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbb5ffb640 1 -- 172.21.15.113:0/835083985 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac0347c0 con 0x7ffbb80b3fd0 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbb5ffb640 1 -- 172.21.15.113:0/835083985 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac034b70 con 0x7ffbb80b3fd0 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 -- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 msgr2=0x7ffbb806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb806dfd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00ba70 tx=0x7ffbac02c870 comp rx=0 tx=0).stop 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 -- 172.21.15.113:0/835083985 shutdown_connections 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb806e510 0x7ffbb80cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/835083985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb806dfd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 -- 172.21.15.113:0/835083985 >> 172.21.15.113:0/835083985 conn(0x7ffbb80b1a50 msgr2=0x7ffbb80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 -- 172.21.15.113:0/835083985 shutdown_connections 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.469+0000 7ffbbd49e640 1 -- 172.21.15.113:0/835083985 wait complete. 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 Processor -- start 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 -- start start 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb806e510 0x7ffbb8137db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb813d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb8144910 0x7ffbb813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb80b4da0 con 0x7ffbb806e510 2024-04-25T00:05:19.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb80b4440 con 0x7ffbb80b3fd0 2024-04-25T00:05:19.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbbd49e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb80b4f20 con 0x7ffbb8144910 2024-04-25T00:05:19.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbb6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb806e510 0x7ffbb8137db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbb6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb806e510 0x7ffbb8137db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53796/0 (socket says 172.21.15.113:53796) 2024-04-25T00:05:19.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.471+0000 7ffbb6ffd640 1 -- 172.21.15.113:0/208936520 learned_addr learned my addr 172.21.15.113:0/208936520 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:19.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb67fc640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb813d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb77fe640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb8144910 0x7ffbb813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb6ffd640 1 -- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb8144910 msgr2=0x7ffbb813df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb6ffd640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb8144910 0x7ffbb813df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb6ffd640 1 -- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 msgr2=0x7ffbb813d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb6ffd640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb813d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb6ffd640 1 -- 172.21.15.113:0/208936520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb80cd900 con 0x7ffbb806e510 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb67fc640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb813d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbb6ffd640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb806e510 0x7ffbb8137db0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ffba800c390 tx=0x7ffba800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffb9f7fe640 1 -- 172.21.15.113:0/208936520 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8017070 con 0x7ffbb806e510 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffb9f7fe640 1 -- 172.21.15.113:0/208936520 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba8005ce0 con 0x7ffbb806e510 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb813e510 con 0x7ffbb806e510 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffb9f7fe640 1 -- 172.21.15.113:0/208936520 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba80072f0 con 0x7ffbb806e510 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.472+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb813ead0 con 0x7ffbb806e510 2024-04-25T00:05:19.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.473+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb80691a0 con 0x7ffbb806e510 2024-04-25T00:05:19.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.473+0000 7ffb9f7fe640 1 -- 172.21.15.113:0/208936520 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffba801a070 con 0x7ffbb806e510 2024-04-25T00:05:19.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.474+0000 7ffb9f7fe640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffba406be30 0x7ffba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.474+0000 7ffb9f7fe640 1 -- 172.21.15.113:0/208936520 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7ffba808a190 con 0x7ffbb806e510 2024-04-25T00:05:19.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.474+0000 7ffbb67fc640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffba406be30 0x7ffba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.474+0000 7ffbb67fc640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffba406be30 0x7ffba406e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffbac037000 tx=0x7ffbac032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:19.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.478+0000 7ffb9f7fe640 1 -- 172.21.15.113:0/208936520 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffba8058bd0 con 0x7ffbb806e510 2024-04-25T00:05:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.652+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbb8143f30 con 0x7ffbb806e510 2024-04-25T00:05:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.653+0000 7ffb9f7fe640 1 -- 172.21.15.113:0/208936520 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16667 (secure 0 0 0) 0x7ffba80584d0 con 0x7ffbb806e510 2024-04-25T00:05:19.654 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:19.654 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:51.690687+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:48.259110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:49.014993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffba406be30 msgr2=0x7ffba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffba406be30 0x7ffba406e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffbac037000 tx=0x7ffbac032000 comp rx=0 tx=0).stop 2024-04-25T00:05:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb806e510 msgr2=0x7ffbb8137db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb806e510 0x7ffbb8137db0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ffba800c390 tx=0x7ffba800c850 comp rx=0 tx=0).stop 2024-04-25T00:05:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 shutdown_connections 2024-04-25T00:05:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffba406be30 0x7ffba406e2f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbb80b3fd0 0x7ffbb813d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbb8144910 0x7ffbb813df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 --2- 172.21.15.113:0/208936520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb806e510 0x7ffbb8137db0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 >> 172.21.15.113:0/208936520 conn(0x7ffbb80b1a50 msgr2=0x7ffbb8071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:19.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 shutdown_connections 2024-04-25T00:05:19.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.655+0000 7ffbbd49e640 1 -- 172.21.15.113:0/208936520 wait complete. 2024-04-25T00:05:19.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:04:48.259110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:04:49.014993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:05:19.667 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:05:19.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.766+0000 7fdd5e79e640 1 Processor -- start 2024-04-25T00:05:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.766+0000 7fdd5e79e640 1 -- start start 2024-04-25T00:05:19.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd5e79e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdd5806e770 0x7fdd5806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd5e79e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5815fb90 con 0x7fdd5806d410 2024-04-25T00:05:19.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd5e79e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5815fe70 con 0x7fdd5806b0f0 2024-04-25T00:05:19.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd5e79e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdd58160150 con 0x7fdd5806e770 2024-04-25T00:05:19.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd5df9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdd5806d410 0x7fdd5806d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33064/0 (socket says 172.21.15.113:33064) 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd5df9d640 1 -- 172.21.15.113:0/4261748192 learned_addr learned my addr 172.21.15.113:0/4261748192 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3354144017 0 0) 0x7fdd5815fb90 con 0x7fdd5806d410 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd44003520 con 0x7fdd5806d410 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.767+0000 7fdd5cf9b640 1 --2- 172.21.15.113:0/4261748192 >> v2:172.21.15.178:3300/0 conn(0x7fdd5806e770 0x7fdd5806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3104507345 0 0) 0x7fdd44003520 con 0x7fdd5806d410 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 >> v2:172.21.15.178:3300/0 conn(0x7fdd5806e770 msgr2=0x7fdd5806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 --2- 172.21.15.113:0/4261748192 >> v2:172.21.15.178:3300/0 conn(0x7fdd5806e770 0x7fdd5806ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 >> v1:172.21.15.178:6789/0 conn(0x7fdd5806b0f0 legacy=0x7fdd5806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd58160430 con 0x7fdd5806d410 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fdd50001670 con 0x7fdd5806d410 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 >> v1:172.21.15.113:6789/0 conn(0x7fdd5806d410 legacy=0x7fdd5806d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd440039b0 0x7fdd44003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd44004800 0x7fdd44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440051c0 0x7fdd4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58160150 con 0x7fdd440039b0 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4400c0f0 con 0x7fdd44004800 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4400c2f0 con 0x7fdd440051c0 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5d79c640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd440039b0 0x7fdd44003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5cf9b640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd44004800 0x7fdd44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5df9d640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440051c0 0x7fdd4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5d79c640 1 -- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440051c0 msgr2=0x7fdd4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5d79c640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440051c0 0x7fdd4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5d79c640 1 -- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd44004800 msgr2=0x7fdd44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5d79c640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd44004800 0x7fdd44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5d79c640 1 -- 172.21.15.113:0/4261748192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd44003520 con 0x7fdd440039b0 2024-04-25T00:05:19.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5cf9b640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd44004800 0x7fdd44004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.768+0000 7fdd5df9d640 1 -- 172.21.15.113:0/4261748192 reap_dead start 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd5d79c640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd440039b0 0x7fdd44003db0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fdd54004480 tx=0x7fdd44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59ffb1b5dc06ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54037070 con 0x7fdd440039b0 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd5402ad10 con 0x7fdd440039b0 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd4ffff640 1 -- 172.21.15.113:0/4261748192 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54004080 con 0x7fdd440039b0 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd5e79e640 1 -- 172.21.15.113:0/4261748192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd440039b0 msgr2=0x7fdd44003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd440039b0 0x7fdd44003db0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fdd54004480 tx=0x7fdd44005830 comp rx=0 tx=0).stop 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd5e79e640 1 -- 172.21.15.113:0/4261748192 shutdown_connections 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/4261748192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd440039b0 0x7fdd44003db0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.769+0000 7fdd5e79e640 1 -- 172.21.15.113:0/4261748192 >> 172.21.15.113:0/4261748192 conn(0x7fdd580b21c0 msgr2=0x7fdd580b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.770+0000 7fdd5e79e640 1 -- 172.21.15.113:0/4261748192 shutdown_connections 2024-04-25T00:05:19.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.770+0000 7fdd5e79e640 1 -- 172.21.15.113:0/4261748192 wait complete. 2024-04-25T00:05:19.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.771+0000 7fdd5e79e640 1 Processor -- start 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.771+0000 7fdd5e79e640 1 -- start start 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.771+0000 7fdd5e79e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440039b0 0x7fdd581f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5e79e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd581445b0 0x7fdd58152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5e79e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd58144a30 0x7fdd58152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5e79e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd580cec20 con 0x7fdd581445b0 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5e79e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd580ceaa0 con 0x7fdd58144a30 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5e79e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5806b4d0 con 0x7fdd440039b0 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd581445b0 0x7fdd58152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd581445b0 0x7fdd58152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53812/0 (socket says 172.21.15.113:53812) 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 -- 172.21.15.113:0/3017085873 learned_addr learned my addr 172.21.15.113:0/3017085873 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5d79c640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440039b0 0x7fdd581f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5df9d640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd58144a30 0x7fdd58152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 -- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440039b0 msgr2=0x7fdd581f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440039b0 0x7fdd581f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 -- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd58144a30 msgr2=0x7fdd58152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd58144a30 0x7fdd58152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 -- 172.21.15.113:0/3017085873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581f6420 con 0x7fdd581445b0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5df9d640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd58144a30 0x7fdd58152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5cf9b640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd581445b0 0x7fdd58152110 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800f490 tx=0x7fdd4800f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd4dffb640 1 -- 172.21.15.113:0/3017085873 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48018020 con 0x7fdd581445b0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd4dffb640 1 -- 172.21.15.113:0/3017085873 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd48014870 con 0x7fdd581445b0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5814a690 con 0x7fdd581445b0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.772+0000 7fdd4dffb640 1 -- 172.21.15.113:0/3017085873 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48014c20 con 0x7fdd581445b0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.773+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5814ac50 con 0x7fdd581445b0 2024-04-25T00:05:19.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.774+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd580691a0 con 0x7fdd581445b0 2024-04-25T00:05:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.774+0000 7fdd4dffb640 1 -- 172.21.15.113:0/3017085873 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd4801b020 con 0x7fdd581445b0 2024-04-25T00:05:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.774+0000 7fdd4dffb640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd3c06be60 0x7fdd3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.775+0000 7fdd4dffb640 1 -- 172.21.15.113:0/3017085873 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fdd4808b140 con 0x7fdd581445b0 2024-04-25T00:05:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.775+0000 7fdd5d79c640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd3c06be60 0x7fdd3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.775+0000 7fdd5d79c640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd3c06be60 0x7fdd3c06e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdd54034000 tx=0x7fdd5402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:19.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.783+0000 7fdd4dffb640 1 -- 172.21.15.113:0/3017085873 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd48022020 con 0x7fdd581445b0 2024-04-25T00:05:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.955+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd5814c200 con 0x7fdd581445b0 2024-04-25T00:05:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.955+0000 7fdd4dffb640 1 -- 172.21.15.113:0/3017085873 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7fdd48024300 con 0x7fdd581445b0 2024-04-25T00:05:19.956 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:05:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd3c06be60 msgr2=0x7fdd3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd3c06be60 0x7fdd3c06e320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdd54034000 tx=0x7fdd5402e000 comp rx=0 tx=0).stop 2024-04-25T00:05:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd581445b0 msgr2=0x7fdd58152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd581445b0 0x7fdd58152110 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800f490 tx=0x7fdd4800f950 comp rx=0 tx=0).stop 2024-04-25T00:05:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 shutdown_connections 2024-04-25T00:05:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdd3c06be60 0x7fdd3c06e320 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd58144a30 0x7fdd58152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd581445b0 0x7fdd58152110 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 --2- 172.21.15.113:0/3017085873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd440039b0 0x7fdd581f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 >> 172.21.15.113:0/3017085873 conn(0x7fdd580b21c0 msgr2=0x7fdd5814ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 shutdown_connections 2024-04-25T00:05:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:19.957+0000 7fdd5e79e640 1 -- 172.21.15.113:0/3017085873 wait complete. 2024-04-25T00:05:19.969 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:05:20.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.067+0000 7ff8614c1640 1 Processor -- start 2024-04-25T00:05:20.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.067+0000 7ff8614c1640 1 -- start start 2024-04-25T00:05:20.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff8614c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff8614c1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff85c0d28c0 0x7ff85c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff8614c1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff85c0d2e00 con 0x7ff85c0b61e0 2024-04-25T00:05:20.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff8614c1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff85c0d30e0 con 0x7ff85c0d28c0 2024-04-25T00:05:20.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff8614c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c0d33c0 con 0x7ff85c15e510 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85a7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff85c0d28c0 0x7ff85c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff85c0b61e0 0x7ff85c071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33076/0 (socket says 172.21.15.113:33076) 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85b7fe640 1 -- 172.21.15.113:0/2225249756 learned_addr learned my addr 172.21.15.113:0/2225249756 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85affd640 1 --2- 172.21.15.113:0/2225249756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85a7fc640 1 -- 172.21.15.113:0/2225249756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 msgr2=0x7ff85c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85a7fc640 1 --2- 172.21.15.113:0/2225249756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85a7fc640 1 -- 172.21.15.113:0/2225249756 >> v1:172.21.15.113:6789/0 conn(0x7ff85c0b61e0 legacy=0x7ff85c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.068+0000 7ff85a7fc640 1 -- 172.21.15.113:0/2225249756 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c0d36a0 con 0x7ff85c0d28c0 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.069+0000 7ff85a7fc640 1 --2- 172.21.15.113:0/2225249756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c0b5ca0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff8500097a0 tx=0x7ff85002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0566b09b1c495e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.069+0000 7ff859ffb640 1 -- 172.21.15.113:0/2225249756 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850037070 con 0x7ff85c0d28c0 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.069+0000 7ff859ffb640 1 -- 172.21.15.113:0/2225249756 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff85002ae50 con 0x7ff85c0d28c0 2024-04-25T00:05:20.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.069+0000 7ff859ffb640 1 -- 172.21.15.113:0/2225249756 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850032660 con 0x7ff85c0d28c0 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 -- 172.21.15.113:0/2225249756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 msgr2=0x7ff85c0b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2225249756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c0b5ca0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff8500097a0 tx=0x7ff85002a100 comp rx=0 tx=0).stop 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 -- 172.21.15.113:0/2225249756 shutdown_connections 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2225249756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c0b5ca0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2225249756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 -- 172.21.15.113:0/2225249756 >> 172.21.15.113:0/2225249756 conn(0x7ff85c06b720 msgr2=0x7ff85c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 -- 172.21.15.113:0/2225249756 shutdown_connections 2024-04-25T00:05:20.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.070+0000 7ff8614c1640 1 -- 172.21.15.113:0/2225249756 wait complete. 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.071+0000 7ff8614c1640 1 Processor -- start 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.071+0000 7ff8614c1640 1 -- start start 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff8614c1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff85c0b61e0 0x7ff85c0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff8614c1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff8614c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff8614c1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c0b9dd0 con 0x7ff85c0d28c0 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff8614c1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c0d3ea0 con 0x7ff85c0b61e0 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff8614c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c0b83d0 con 0x7ff85c15e510 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53828/0 (socket says 172.21.15.113:53828) 2024-04-25T00:05:20.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 -- 172.21.15.113:0/2849395057 learned_addr learned my addr 172.21.15.113:0/2849395057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85b7fe640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 -- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 msgr2=0x7ff85c0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85affd640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff85c0b61e0 0x7ff85c0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 -- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff85c0b61e0 msgr2=0x7ff85c0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff85c0b61e0 0x7ff85c0cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 -- 172.21.15.113:0/2849395057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c0cbd70 con 0x7ff85c0d28c0 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.072+0000 7ff85a7fc640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c160dc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff8500098c0 tx=0x7ff8500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.073+0000 7ff8437fe640 1 -- 172.21.15.113:0/2849395057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850041030 con 0x7ff85c0d28c0 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.073+0000 7ff8437fe640 1 -- 172.21.15.113:0/2849395057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff85002ac50 con 0x7ff85c0d28c0 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.073+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c15f890 con 0x7ff85c0d28c0 2024-04-25T00:05:20.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.073+0000 7ff8437fe640 1 -- 172.21.15.113:0/2849395057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850037040 con 0x7ff85c0d28c0 2024-04-25T00:05:20.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.073+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c15fe50 con 0x7ff85c0d28c0 2024-04-25T00:05:20.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.074+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c0691a0 con 0x7ff85c0d28c0 2024-04-25T00:05:20.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.074+0000 7ff8437fe640 1 -- 172.21.15.113:0/2849395057 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff850044070 con 0x7ff85c0d28c0 2024-04-25T00:05:20.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.074+0000 7ff8437fe640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff84806be30 0x7ff84806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.074+0000 7ff8437fe640 1 -- 172.21.15.113:0/2849395057 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7ff8500ab5b0 con 0x7ff85c0d28c0 2024-04-25T00:05:20.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.074+0000 7ff85affd640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff84806be30 0x7ff84806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.075+0000 7ff85affd640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff84806be30 0x7ff84806e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff84c008e50 tx=0x7ff84c0083a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.079+0000 7ff8437fe640 1 -- 172.21.15.113:0/2849395057 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff850044e40 con 0x7ff85c0d28c0 2024-04-25T00:05:20.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.251+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff85c15d1b0 con 0x7ff85c0d28c0 2024-04-25T00:05:20.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.251+0000 7ff8437fe640 1 -- 172.21.15.113:0/2849395057 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7ff85002ae20 con 0x7ff85c0d28c0 2024-04-25T00:05:20.251 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff84806be30 msgr2=0x7ff84806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff84806be30 0x7ff84806e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff84c008e50 tx=0x7ff84c0083a0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 msgr2=0x7ff85c160dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c160dc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff8500098c0 tx=0x7ff8500046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 shutdown_connections 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff84806be30 0x7ff84806e2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85c15e510 0x7ff85c0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c0d28c0 0x7ff85c160dc0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 --2- 172.21.15.113:0/2849395057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff85c0b61e0 0x7ff85c0cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 >> 172.21.15.113:0/2849395057 conn(0x7ff85c06b720 msgr2=0x7ff85c0d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 shutdown_connections 2024-04-25T00:05:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.253+0000 7ff8614c1640 1 -- 172.21.15.113:0/2849395057 wait complete. 2024-04-25T00:05:20.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:05:20.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.363+0000 7f3789a72640 1 Processor -- start 2024-04-25T00:05:20.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.363+0000 7f3789a72640 1 -- start start 2024-04-25T00:05:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f3789a72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 0x7f3784165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f3789a72640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f378415f4f0 con 0x7f378415e2d0 2024-04-25T00:05:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f3789a72640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f378415f7d0 con 0x7f37840b3080 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f3789a72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37840cde00 con 0x7f37840b4a40 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37837fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f378415e2d0 0x7f378415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33084/0 (socket says 172.21.15.113:33084) 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37837fe640 1 -- 172.21.15.113:0/3103656669 learned_addr learned my addr 172.21.15.113:0/3103656669 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f3783fff640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 0x7f3784165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867446773 0 0) 0x7f378415f4f0 con 0x7f378415e2d0 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f376c003520 con 0x7f378415e2d0 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2084837614 0 0) 0x7f376c003520 con 0x7f378415e2d0 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 msgr2=0x7f3784165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 0x7f3784165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 >> v1:172.21.15.178:6789/0 conn(0x7f37840b3080 legacy=0x7f37840b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378415fab0 con 0x7f378415e2d0 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f3774001670 con 0x7f378415e2d0 2024-04-25T00:05:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.364+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 >> v1:172.21.15.113:6789/0 conn(0x7f378415e2d0 legacy=0x7f378415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37827fc640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 0x7f376c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37827fc640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c003f40 0x7f376c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37827fc640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37840cde00 con 0x7f376c004d40 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f376c00ba70 con 0x7f376c003f40 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f376c00bc40 con 0x7f37840b4a40 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3783fff640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37837fe640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 0x7f376c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3782ffd640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c003f40 0x7f376c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3783fff640 1 -- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 msgr2=0x7f376c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3783fff640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 0x7f376c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3783fff640 1 -- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c003f40 msgr2=0x7f376c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3783fff640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c003f40 0x7f376c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3783fff640 1 -- 172.21.15.113:0/3103656669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c003520 con 0x7f376c004d40 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f3782ffd640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f376c003f40 0x7f376c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.365+0000 7f37837fe640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37840b4a40 0x7f376c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3783fff640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f377c009fc0 tx=0x7f376c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9187c7c30c7ae830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c009c10 con 0x7f376c004d40 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f377c036070 con 0x7f376c004d40 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f37827fc640 1 -- 172.21.15.113:0/3103656669 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c031840 con 0x7f376c004d40 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3789a72640 1 -- 172.21.15.113:0/3103656669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f376c004d40 msgr2=0x7f376c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3789a72640 1 --2- 172.21.15.113:0/3103656669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f377c009fc0 tx=0x7f376c005830 comp rx=0 tx=0).stop 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3783fff640 1 -- 172.21.15.113:0/3103656669 reap_dead start 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3789a72640 1 -- 172.21.15.113:0/3103656669 shutdown_connections 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3789a72640 1 -- 172.21.15.113:0/3103656669 >> 172.21.15.113:0/3103656669 conn(0x7f378406b0f0 msgr2=0x7f378406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:20.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3789a72640 1 -- 172.21.15.113:0/3103656669 shutdown_connections 2024-04-25T00:05:20.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.366+0000 7f3789a72640 1 -- 172.21.15.113:0/3103656669 wait complete. 2024-04-25T00:05:20.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.367+0000 7f3789a72640 1 Processor -- start 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3789a72640 1 -- start start 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3789a72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f378414b4e0 0x7f37840729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3789a72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3784072f30 0x7f378414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3789a72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f378414e0f0 0x7f378415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3789a72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37840cf420 con 0x7f378414e0f0 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3789a72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37840cf2a0 con 0x7f378414b4e0 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3789a72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37840cf5a0 con 0x7f3784072f30 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3783fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f378414e0f0 0x7f378415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3783fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f378414e0f0 0x7f378415b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53854/0 (socket says 172.21.15.113:53854) 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.368+0000 7f3783fff640 1 -- 172.21.15.113:0/2208533957 learned_addr learned my addr 172.21.15.113:0/2208533957 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:20.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3782ffd640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3784072f30 0x7f378414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f37837fe640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f378414b4e0 0x7f37840729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3783fff640 1 -- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3784072f30 msgr2=0x7f378414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3783fff640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3784072f30 0x7f378414dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3783fff640 1 -- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f378414b4e0 msgr2=0x7f37840729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3783fff640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f378414b4e0 0x7f37840729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3783fff640 1 -- 172.21.15.113:0/2208533957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378414e900 con 0x7f378414e0f0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f37837fe640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f378414b4e0 0x7f37840729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3782ffd640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3784072f30 0x7f378414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3783fff640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f378414e0f0 0x7f378415b390 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f377c00e230 tx=0x7f377c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f376bfff640 1 -- 172.21.15.113:0/2208533957 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c009c10 con 0x7f378414e0f0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f376bfff640 1 -- 172.21.15.113:0/2208533957 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f377c031d50 con 0x7f378414e0f0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f376bfff640 1 -- 172.21.15.113:0/2208533957 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c045650 con 0x7f378414e0f0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378414bd20 con 0x7f378414e0f0 2024-04-25T00:05:20.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.369+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378414e570 con 0x7f378414e0f0 2024-04-25T00:05:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.370+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37840691a0 con 0x7f378414e0f0 2024-04-25T00:05:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.370+0000 7f376bfff640 1 -- 172.21.15.113:0/2208533957 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f377c005d10 con 0x7f378414e0f0 2024-04-25T00:05:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.371+0000 7f376bfff640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f377006be30 0x7f377006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.371+0000 7f37837fe640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f377006be30 0x7f377006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.371+0000 7f376bfff640 1 -- 172.21.15.113:0/2208533957 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f377c0abc70 con 0x7f378414e0f0 2024-04-25T00:05:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.371+0000 7f37837fe640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f377006be30 0x7f377006e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3774005990 tx=0x7f3774001e50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.375+0000 7f376bfff640 1 -- 172.21.15.113:0/2208533957 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f377c07a510 con 0x7f378414e0f0 2024-04-25T00:05:20.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.548+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3784073310 con 0x7f378414e0f0 2024-04-25T00:05:20.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.548+0000 7f376bfff640 1 -- 172.21.15.113:0/2208533957 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7f377c079e10 con 0x7f378414e0f0 2024-04-25T00:05:20.549 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f377006be30 msgr2=0x7f377006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f377006be30 0x7f377006e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3774005990 tx=0x7f3774001e50 comp rx=0 tx=0).stop 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f378414e0f0 msgr2=0x7f378415b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f378414e0f0 0x7f378415b390 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f377c00e230 tx=0x7f377c009750 comp rx=0 tx=0).stop 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 shutdown_connections 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f377006be30 0x7f377006e2f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f378414e0f0 0x7f378415b390 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3784072f30 0x7f378414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 --2- 172.21.15.113:0/2208533957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f378414b4e0 0x7f37840729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 >> 172.21.15.113:0/2208533957 conn(0x7f378406b0f0 msgr2=0x7f37840d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:20.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 shutdown_connections 2024-04-25T00:05:20.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.550+0000 7f3789a72640 1 -- 172.21.15.113:0/2208533957 wait complete. 2024-04-25T00:05:20.561 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:05:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.658+0000 7feaf9c12640 1 Processor -- start 2024-04-25T00:05:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.658+0000 7feaf9c12640 1 -- start start 2024-04-25T00:05:20.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.658+0000 7feaf9c12640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feaf413f050 0x7feaf40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf9c12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feaf41606a0 0x7feaf40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf9c12640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf415d6b0 con 0x7feaf40d04f0 2024-04-25T00:05:20.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf9c12640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feaf40d09a0 con 0x7feaf41606a0 2024-04-25T00:05:20.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf9c12640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7feaf415c6a0 con 0x7feaf413f050 2024-04-25T00:05:20.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feaf41606a0 0x7feaf40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf2ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feaf40d04f0 0x7feaf415ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33092/0 (socket says 172.21.15.113:33092) 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf2ffd640 1 -- 172.21.15.113:0/3141286798 learned_addr learned my addr 172.21.15.113:0/3141286798 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf3fff640 1 -- 172.21.15.113:0/3141286798 >> v2:172.21.15.178:3300/0 conn(0x7feaf413f050 msgr2=0x7feaf40cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf3fff640 1 --2- 172.21.15.113:0/3141286798 >> v2:172.21.15.178:3300/0 conn(0x7feaf413f050 0x7feaf40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf3fff640 1 -- 172.21.15.113:0/3141286798 >> v1:172.21.15.113:6789/0 conn(0x7feaf40d04f0 legacy=0x7feaf415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf27fc640 1 -- 172.21.15.113:0/3141286798 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222814741 0 0) 0x7feaf415d6b0 con 0x7feaf40d04f0 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.659+0000 7feaf3fff640 1 -- 172.21.15.113:0/3141286798 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf415c8f0 con 0x7feaf41606a0 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf3fff640 1 --2- 172.21.15.113:0/3141286798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf41606a0 0x7feaf40cea60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7feaf406f550 tx=0x7feaec02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddf064c42b46bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf27fc640 1 -- 172.21.15.113:0/3141286798 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec007740 con 0x7feaf41606a0 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf27fc640 1 -- 172.21.15.113:0/3141286798 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec0078e0 con 0x7feaf41606a0 2024-04-25T00:05:20.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf27fc640 1 -- 172.21.15.113:0/3141286798 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec007cb0 con 0x7feaf41606a0 2024-04-25T00:05:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 -- 172.21.15.113:0/3141286798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf41606a0 msgr2=0x7feaf40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3141286798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf41606a0 0x7feaf40cea60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7feaf406f550 tx=0x7feaec02a5f0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 -- 172.21.15.113:0/3141286798 shutdown_connections 2024-04-25T00:05:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3141286798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf41606a0 0x7feaf40cea60 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3141286798 >> v2:172.21.15.178:3300/0 conn(0x7feaf413f050 0x7feaf40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 -- 172.21.15.113:0/3141286798 >> 172.21.15.113:0/3141286798 conn(0x7feaf406b0f0 msgr2=0x7feaf406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:20.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 -- 172.21.15.113:0/3141286798 shutdown_connections 2024-04-25T00:05:20.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.660+0000 7feaf9c12640 1 -- 172.21.15.113:0/3141286798 wait complete. 2024-04-25T00:05:20.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.661+0000 7feaf9c12640 1 Processor -- start 2024-04-25T00:05:20.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf9c12640 1 -- start start 2024-04-25T00:05:20.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf9c12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf40d04f0 0x7feaf41f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf9c12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf413f050 0x7feaf414c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf9c12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf41606a0 0x7feaf414c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf9c12640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feaf415d190 con 0x7feaf40d04f0 2024-04-25T00:05:20.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf9c12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feaf415ceb0 con 0x7feaf41606a0 2024-04-25T00:05:20.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf9c12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feaf413f430 con 0x7feaf413f050 2024-04-25T00:05:20.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.662+0000 7feaf37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf40d04f0 0x7feaf41f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf40d04f0 0x7feaf41f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53882/0 (socket says 172.21.15.113:53882) 2024-04-25T00:05:20.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 -- 172.21.15.113:0/3384782831 learned_addr learned my addr 172.21.15.113:0/3384782831 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:20.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf3fff640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf41606a0 0x7feaf414c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf2ffd640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf413f050 0x7feaf414c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 -- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf413f050 msgr2=0x7feaf414c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf413f050 0x7feaf414c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 -- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf41606a0 msgr2=0x7feaf414c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf41606a0 0x7feaf414c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 -- 172.21.15.113:0/3384782831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf4148480 con 0x7feaf40d04f0 2024-04-25T00:05:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf37fe640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf40d04f0 0x7feaf41f1d40 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7feae800b4a0 tx=0x7feae800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feadbfff640 1 -- 172.21.15.113:0/3384782831 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8014070 con 0x7feaf40d04f0 2024-04-25T00:05:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feadbfff640 1 -- 172.21.15.113:0/3384782831 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae8010610 con 0x7feaf40d04f0 2024-04-25T00:05:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf4155580 con 0x7feaf40d04f0 2024-04-25T00:05:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feadbfff640 1 -- 172.21.15.113:0/3384782831 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8010a00 con 0x7feaf40d04f0 2024-04-25T00:05:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.663+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf4155b40 con 0x7feaf40d04f0 2024-04-25T00:05:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.664+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf40691a0 con 0x7feaf40d04f0 2024-04-25T00:05:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.664+0000 7feadbfff640 1 -- 172.21.15.113:0/3384782831 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feae8017070 con 0x7feaf40d04f0 2024-04-25T00:05:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.665+0000 7feadbfff640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feae006be30 0x7feae006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.665+0000 7feadbfff640 1 -- 172.21.15.113:0/3384782831 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7feae808a880 con 0x7feaf40d04f0 2024-04-25T00:05:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.665+0000 7feaf2ffd640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feae006be30 0x7feae006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.665+0000 7feaf2ffd640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feae006be30 0x7feae006e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feaf40cf4e0 tx=0x7feae4007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.669+0000 7feadbfff640 1 -- 172.21.15.113:0/3384782831 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feae8059230 con 0x7feaf40d04f0 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.855+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feaf4149ef0 con 0x7feaf40d04f0 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.856+0000 7feadbfff640 1 -- 172.21.15.113:0/3384782831 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) ==== 80+0+0 (secure 0 0 0) 0x7feae8058b30 con 0x7feaf40d04f0 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feae006be30 msgr2=0x7feae006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feae006be30 0x7feae006e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feaf40cf4e0 tx=0x7feae4007b50 comp rx=0 tx=0).stop 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf40d04f0 msgr2=0x7feaf41f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf40d04f0 0x7feaf41f1d40 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7feae800b4a0 tx=0x7feae800b960 comp rx=0 tx=0).stop 2024-04-25T00:05:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 shutdown_connections 2024-04-25T00:05:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feae006be30 0x7feae006e2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf41606a0 0x7feaf414c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf413f050 0x7feaf414c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 --2- 172.21.15.113:0/3384782831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feaf40d04f0 0x7feaf41f1d40 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 >> 172.21.15.113:0/3384782831 conn(0x7feaf406b0f0 msgr2=0x7feaf4151a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 shutdown_connections 2024-04-25T00:05:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:20.858+0000 7feaf9c12640 1 -- 172.21.15.113:0/3384782831 wait complete. 2024-04-25T00:05:20.869 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:05:20.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:20.869 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:05:21.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:05:21.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:21.465 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:05:21.487 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:05:21.487 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:05:21.487 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:21.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:05:21.504 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:05:21.504 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:05:21.504 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:21.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:05:21.542 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:05:21.542 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:05:21.542 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:21.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:05:21.561 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:21.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.678+0000 7fc6a3f12640 1 Processor -- start 2024-04-25T00:05:21.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.678+0000 7fc6a3f12640 1 -- start start 2024-04-25T00:05:21.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a3f12640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc69c144910 0x7fc69c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:21.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a3f12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc69c0b5240 0x7fc69c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:21.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a3f12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b56b0 0x7fc69c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:21.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a3f12640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c0b4510 con 0x7fc69c0b5240 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a3f12640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c144cf0 con 0x7fc69c144910 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a3f12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c0b2f30 con 0x7fc69c0b56b0 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc69c0b5240 0x7fc69c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc69c0b5240 0x7fc69c06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53894/0 (socket says 172.21.15.113:53894) 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 -- 172.21.15.113:0/4078814951 learned_addr learned my addr 172.21.15.113:0/4078814951 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a24af640 1 --2- 172.21.15.113:0/4078814951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b56b0 0x7fc69c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a1cae640 1 --2- 172.21.15.113:0/4078814951 >> v2:172.21.15.178:3300/0 conn(0x7fc69c144910 0x7fc69c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 -- 172.21.15.113:0/4078814951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b56b0 msgr2=0x7fc69c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 --2- 172.21.15.113:0/4078814951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b56b0 0x7fc69c06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 -- 172.21.15.113:0/4078814951 >> v2:172.21.15.178:3300/0 conn(0x7fc69c144910 msgr2=0x7fc69c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 --2- 172.21.15.113:0/4078814951 >> v2:172.21.15.178:3300/0 conn(0x7fc69c144910 0x7fc69c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.679+0000 7fc6a14ad640 1 -- 172.21.15.113:0/4078814951 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c0b3210 con 0x7fc69c0b5240 2024-04-25T00:05:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a14ad640 1 --2- 172.21.15.113:0/4078814951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b5240 0x7fc69c06dfd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009870 tx=0x7fc68c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58640681bc3fc8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:21.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a0cac640 1 -- 172.21.15.113:0/4078814951 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c037070 con 0x7fc69c0b5240 2024-04-25T00:05:21.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a0cac640 1 -- 172.21.15.113:0/4078814951 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c032440 con 0x7fc69c0b5240 2024-04-25T00:05:21.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a0cac640 1 -- 172.21.15.113:0/4078814951 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c032810 con 0x7fc69c0b5240 2024-04-25T00:05:21.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a3f12640 1 -- 172.21.15.113:0/4078814951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b5240 msgr2=0x7fc69c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:21.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/4078814951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b5240 0x7fc69c06dfd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009870 tx=0x7fc68c02a210 comp rx=0 tx=0).stop 2024-04-25T00:05:21.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a3f12640 1 -- 172.21.15.113:0/4078814951 shutdown_connections 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/4078814951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b56b0 0x7fc69c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/4078814951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b5240 0x7fc69c06dfd0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/4078814951 >> v2:172.21.15.178:3300/0 conn(0x7fc69c144910 0x7fc69c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.680+0000 7fc6a3f12640 1 -- 172.21.15.113:0/4078814951 >> 172.21.15.113:0/4078814951 conn(0x7fc69c0b1a50 msgr2=0x7fc69c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.681+0000 7fc6a3f12640 1 -- 172.21.15.113:0/4078814951 shutdown_connections 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.681+0000 7fc6a3f12640 1 -- 172.21.15.113:0/4078814951 wait complete. 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 Processor -- start 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 -- start start 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b5240 0x7fc69c0aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b56b0 0x7fc69c0c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c144910 0x7fc69c0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:21.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0b4da0 con 0x7fc69c0b56b0 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0b37d0 con 0x7fc69c144910 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a3f12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c0b4f20 con 0x7fc69c0b5240 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a14ad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b56b0 0x7fc69c0c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a14ad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b56b0 0x7fc69c0c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53902/0 (socket says 172.21.15.113:53902) 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.682+0000 7fc6a14ad640 1 -- 172.21.15.113:0/337175984 learned_addr learned my addr 172.21.15.113:0/337175984 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a1cae640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b5240 0x7fc69c0aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a14ad640 1 -- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b5240 msgr2=0x7fc69c0aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a24af640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c144910 0x7fc69c0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a14ad640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b5240 0x7fc69c0aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a14ad640 1 -- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c144910 msgr2=0x7fc69c0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a14ad640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c144910 0x7fc69c0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a14ad640 1 -- 172.21.15.113:0/337175984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c1357b0 con 0x7fc69c0b56b0 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a1cae640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b5240 0x7fc69c0aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:21.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a24af640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c144910 0x7fc69c0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a14ad640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b56b0 0x7fc69c0c7840 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02a250 tx=0x7fc68c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6927fc640 1 -- 172.21.15.113:0/337175984 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c037040 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6927fc640 1 -- 172.21.15.113:0/337175984 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c042070 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6927fc640 1 -- 172.21.15.113:0/337175984 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c030860 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c0afa20 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.683+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c0affe0 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.684+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c0691a0 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.685+0000 7fc6927fc640 1 -- 172.21.15.113:0/337175984 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc68c030a00 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.685+0000 7fc6927fc640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc68006be30 0x7fc68006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.685+0000 7fc6927fc640 1 -- 172.21.15.113:0/337175984 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fc68c0ab6c0 con 0x7fc69c0b56b0 2024-04-25T00:05:21.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.685+0000 7fc6a1cae640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc68006be30 0x7fc68006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.685+0000 7fc6a1cae640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc68006be30 0x7fc68006e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc694006fd0 tx=0x7fc694008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:21.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.689+0000 7fc6927fc640 1 -- 172.21.15.113:0/337175984 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc68c045360 con 0x7fc69c0b56b0 2024-04-25T00:05:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.904+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc69c0cf1a0 con 0x7fc69c0b56b0 2024-04-25T00:05:21.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.904+0000 7fc6927fc640 1 -- 172.21.15.113:0/337175984 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7fc68c07a100 con 0x7fc69c0b56b0 2024-04-25T00:05:21.905 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:21.905 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:21.905 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:21.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc68006be30 msgr2=0x7fc68006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:21.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc68006be30 0x7fc68006e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc694006fd0 tx=0x7fc694008040 comp rx=0 tx=0).stop 2024-04-25T00:05:21.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b56b0 msgr2=0x7fc69c0c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:21.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b56b0 0x7fc69c0c7840 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc68c02a250 tx=0x7fc68c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 shutdown_connections 2024-04-25T00:05:21.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc68006be30 0x7fc68006e2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c144910 0x7fc69c0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc69c0b56b0 0x7fc69c0c7840 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 --2- 172.21.15.113:0/337175984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0b5240 0x7fc69c0aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:21.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 >> 172.21.15.113:0/337175984 conn(0x7fc69c0b1a50 msgr2=0x7fc69c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:21.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 shutdown_connections 2024-04-25T00:05:21.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:21.906+0000 7fc6a3f12640 1 -- 172.21.15.113:0/337175984 wait complete. 2024-04-25T00:05:21.918 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.014+0000 7f4ea682a640 1 Processor -- start 2024-04-25T00:05:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.014+0000 7f4ea682a640 1 -- start start 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.015+0000 7f4ea682a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ea015e240 0x7f4ea01627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.015+0000 7f4ea682a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea0162d20 con 0x7f4ea00b5620 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.015+0000 7f4ea682a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea00b3c20 con 0x7f4ea006e6e0 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.015+0000 7f4ea682a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea0072d90 con 0x7f4ea015e240 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.015+0000 7f4e9ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ea015e240 0x7f4ea01627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.015+0000 7f4e9f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4ea00b5620 0x7f4ea006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33108/0 (socket says 172.21.15.113:33108) 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.015+0000 7f4e9f7fe640 1 -- 172.21.15.113:0/4069123813 learned_addr learned my addr 172.21.15.113:0/4069123813 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.016+0000 7f4e9ffff640 1 -- 172.21.15.113:0/4069123813 >> v1:172.21.15.178:6789/0 conn(0x7f4ea006e6e0 legacy=0x7f4ea00b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.016+0000 7f4e9ffff640 1 -- 172.21.15.113:0/4069123813 >> v1:172.21.15.113:6789/0 conn(0x7f4ea00b5620 legacy=0x7f4ea006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.016+0000 7f4e9ffff640 1 -- 172.21.15.113:0/4069123813 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea015e620 con 0x7f4ea015e240 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.016+0000 7f4e9effd640 1 -- 172.21.15.113:0/4069123813 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2592318224 0 0) 0x7f4ea0162d20 con 0x7f4ea00b5620 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.016+0000 7f4e9ffff640 1 --2- 172.21.15.113:0/4069123813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea01627e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f4e900097a0 tx=0x7f4e9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eeb45b694b09655 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.016+0000 7f4e9effd640 1 -- 172.21.15.113:0/4069123813 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90037070 con 0x7f4ea015e240 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4e9effd640 1 -- 172.21.15.113:0/4069123813 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e90032440 con 0x7f4ea015e240 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4e9effd640 1 -- 172.21.15.113:0/4069123813 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90032810 con 0x7f4ea015e240 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4ea682a640 1 -- 172.21.15.113:0/4069123813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 msgr2=0x7f4ea01627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4ea682a640 1 --2- 172.21.15.113:0/4069123813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea01627e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f4e900097a0 tx=0x7f4e9002a1e0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4ea682a640 1 -- 172.21.15.113:0/4069123813 shutdown_connections 2024-04-25T00:05:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4ea682a640 1 --2- 172.21.15.113:0/4069123813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea01627e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4ea682a640 1 -- 172.21.15.113:0/4069123813 >> 172.21.15.113:0/4069123813 conn(0x7f4ea00b1a50 msgr2=0x7f4ea00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4ea682a640 1 -- 172.21.15.113:0/4069123813 shutdown_connections 2024-04-25T00:05:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.017+0000 7f4ea682a640 1 -- 172.21.15.113:0/4069123813 wait complete. 2024-04-25T00:05:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.018+0000 7f4ea682a640 1 Processor -- start 2024-04-25T00:05:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.018+0000 7f4ea682a640 1 -- start start 2024-04-25T00:05:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea682a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea006e6e0 0x7f4ea014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea682a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ea00b5620 0x7f4ea015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea682a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea006d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea682a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea00727d0 con 0x7f4ea015e240 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea682a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea00724f0 con 0x7f4ea006e6e0 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea682a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea0072ab0 con 0x7f4ea00b5620 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea006d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea006d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53920/0 (socket says 172.21.15.113:53920) 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 -- 172.21.15.113:0/3090803010 learned_addr learned my addr 172.21.15.113:0/3090803010 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4e9ffff640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea006e6e0 0x7f4ea014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 -- 172.21.15.113:0/3090803010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ea00b5620 msgr2=0x7f4ea015f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ea00b5620 0x7f4ea015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 -- 172.21.15.113:0/3090803010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea006e6e0 msgr2=0x7f4ea014ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea006e6e0 0x7f4ea014ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.019+0000 7f4ea4dc7640 1 -- 172.21.15.113:0/3090803010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea0154b00 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.020+0000 7f4ea4dc7640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea006d0e0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c780 tx=0x7f4e9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.020+0000 7f4e9cff9640 1 -- 172.21.15.113:0/3090803010 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98004b10 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.020+0000 7f4e9cff9640 1 -- 172.21.15.113:0/3090803010 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98004cb0 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.020+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea015fbc0 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.020+0000 7f4e9cff9640 1 -- 172.21.15.113:0/3090803010 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98004e60 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.020+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea0160180 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.021+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea00691a0 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.021+0000 7f4e9cff9640 1 -- 172.21.15.113:0/3090803010 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e9800aa90 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.022+0000 7f4e9cff9640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e8c06be30 0x7f4e8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.022+0000 7f4e9cff9640 1 -- 172.21.15.113:0/3090803010 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f4e9808b030 con 0x7f4ea015e240 2024-04-25T00:05:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.022+0000 7f4e9ffff640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e8c06be30 0x7f4e8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.022+0000 7f4e9ffff640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e8c06be30 0x7f4e8c06e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e90034000 tx=0x7f4e9002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.026+0000 7f4e9cff9640 1 -- 172.21.15.113:0/3090803010 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e98020070 con 0x7f4ea015e240 2024-04-25T00:05:22.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.199+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ea015efc0 con 0x7f4ea015e240 2024-04-25T00:05:22.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.201+0000 7f4e9cff9640 1 -- 172.21.15.113:0/3090803010 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16667 (secure 0 0 0) 0x7f4ea0154b00 con 0x7f4ea015e240 2024-04-25T00:05:22.201 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:22.201 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:51.690687+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:48.259110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:49.014993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:22.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.202+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e8c06be30 msgr2=0x7f4e8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.202+0000 7f4ea682a640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e8c06be30 0x7f4e8c06e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e90034000 tx=0x7f4e9002e000 comp rx=0 tx=0).stop 2024-04-25T00:05:22.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.202+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 msgr2=0x7f4ea006d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.202+0000 7f4ea682a640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea006d0e0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c780 tx=0x7f4e9800cc40 comp rx=0 tx=0).stop 2024-04-25T00:05:22.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.202+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 shutdown_connections 2024-04-25T00:05:22.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.203+0000 7f4ea682a640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e8c06be30 0x7f4e8c06e2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.203+0000 7f4ea682a640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ea015e240 0x7f4ea006d0e0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.203+0000 7f4ea682a640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ea00b5620 0x7f4ea015f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.203+0000 7f4ea682a640 1 --2- 172.21.15.113:0/3090803010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea006e6e0 0x7f4ea014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.203+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 >> 172.21.15.113:0/3090803010 conn(0x7f4ea00b1a50 msgr2=0x7f4ea014f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:22.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.203+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 shutdown_connections 2024-04-25T00:05:22.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.203+0000 7f4ea682a640 1 -- 172.21.15.113:0/3090803010 wait complete. 2024-04-25T00:05:22.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:04:48.259110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:04:49.014993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:05:22.215 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:05:22.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.314+0000 7f261e0d6640 1 Processor -- start 2024-04-25T00:05:22.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.314+0000 7f261e0d6640 1 -- start start 2024-04-25T00:05:22.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f261e0d6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f261815e2d0 0x7f261815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f261e0d6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26180b3080 0x7f26180b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f261e0d6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f261815f4f0 con 0x7f261815e6b0 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f261e0d6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f26180cde00 con 0x7f26180b3080 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f261e0d6640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f261815f7d0 con 0x7f261815e2d0 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f2616ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26180b3080 0x7f26180b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f2616ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26180b3080 0x7f26180b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53922/0 (socket says 172.21.15.113:53922) 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f2616ffd640 1 -- 172.21.15.113:0/2762669018 learned_addr learned my addr 172.21.15.113:0/2762669018 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f26177fe640 1 --2- 172.21.15.113:0/2762669018 >> v2:172.21.15.178:3300/0 conn(0x7f261815e2d0 0x7f261815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f2616ffd640 1 -- 172.21.15.113:0/2762669018 >> v2:172.21.15.178:3300/0 conn(0x7f261815e2d0 msgr2=0x7f261815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:22.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f2616ffd640 1 --2- 172.21.15.113:0/2762669018 >> v2:172.21.15.178:3300/0 conn(0x7f261815e2d0 0x7f261815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f2616ffd640 1 -- 172.21.15.113:0/2762669018 >> v1:172.21.15.178:6789/0 conn(0x7f261815e6b0 legacy=0x7f2618071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f2616ffd640 1 -- 172.21.15.113:0/2762669018 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261815fab0 con 0x7f26180b3080 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.315+0000 7f26177fe640 1 --2- 172.21.15.113:0/2762669018 >> v2:172.21.15.178:3300/0 conn(0x7f261815e2d0 0x7f261815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.316+0000 7f2616ffd640 1 --2- 172.21.15.113:0/2762669018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26180b3080 0x7f26180b4330 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f26180b2d40 tx=0x7f260c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e01894db5b55d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.316+0000 7f26167fc640 1 -- 172.21.15.113:0/2762669018 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c037070 con 0x7f26180b3080 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.316+0000 7f26167fc640 1 -- 172.21.15.113:0/2762669018 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f260c032440 con 0x7f26180b3080 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.316+0000 7f26167fc640 1 -- 172.21.15.113:0/2762669018 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c032810 con 0x7f26180b3080 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 -- 172.21.15.113:0/2762669018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26180b3080 msgr2=0x7f26180b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2762669018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26180b3080 0x7f26180b4330 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f26180b2d40 tx=0x7f260c02a270 comp rx=0 tx=0).stop 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 -- 172.21.15.113:0/2762669018 shutdown_connections 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2762669018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26180b3080 0x7f26180b4330 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2762669018 >> v2:172.21.15.178:3300/0 conn(0x7f261815e2d0 0x7f261815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 -- 172.21.15.113:0/2762669018 >> 172.21.15.113:0/2762669018 conn(0x7f261806b0f0 msgr2=0x7f261806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:22.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 -- 172.21.15.113:0/2762669018 shutdown_connections 2024-04-25T00:05:22.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.317+0000 7f261e0d6640 1 -- 172.21.15.113:0/2762669018 wait complete. 2024-04-25T00:05:22.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.318+0000 7f261e0d6640 1 Processor -- start 2024-04-25T00:05:22.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.318+0000 7f261e0d6640 1 -- start start 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f261e0d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26180b3080 0x7f26180ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f261e0d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f261815e2d0 0x7f2618150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f261e0d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f261815e6b0 0x7f261814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f261e0d6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f261815f2b0 con 0x7f261815e2d0 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f261e0d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f261815f130 con 0x7f261815e6b0 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f261e0d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f26180cf2a0 con 0x7f26180b3080 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f261815e2d0 0x7f2618150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f261815e2d0 0x7f2618150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53932/0 (socket says 172.21.15.113:53932) 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 -- 172.21.15.113:0/2523054486 learned_addr learned my addr 172.21.15.113:0/2523054486 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 -- 172.21.15.113:0/2523054486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26180b3080 msgr2=0x7f26180ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26180b3080 0x7f26180ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 -- 172.21.15.113:0/2523054486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f261815e6b0 msgr2=0x7f261814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f261815e6b0 0x7f261814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.319+0000 7f2616ffd640 1 -- 172.21.15.113:0/2523054486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2618151570 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.320+0000 7f2616ffd640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f261815e2d0 0x7f2618150ed0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f260c02a2b0 tx=0x7f260c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.320+0000 7f25fffff640 1 -- 172.21.15.113:0/2523054486 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c041020 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.320+0000 7f25fffff640 1 -- 172.21.15.113:0/2523054486 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f260c0306c0 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.320+0000 7f25fffff640 1 -- 172.21.15.113:0/2523054486 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c037040 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.320+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2618151850 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.320+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2618151e10 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.321+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26180691a0 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.321+0000 7f25fffff640 1 -- 172.21.15.113:0/2523054486 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f260c044020 con 0x7f261815e2d0 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.321+0000 7f25fffff640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f260406be30 0x7f260406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.322+0000 7f25fffff640 1 -- 172.21.15.113:0/2523054486 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f260c0ab610 con 0x7f261815e2d0 2024-04-25T00:05:22.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.322+0000 7f26177fe640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f260406be30 0x7f260406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.322+0000 7f26177fe640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f260406be30 0x7f260406e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2608006e80 tx=0x7f2608008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.326+0000 7f25fffff640 1 -- 172.21.15.113:0/2523054486 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f260c04b030 con 0x7f261815e2d0 2024-04-25T00:05:22.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.499+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26180d00d0 con 0x7f261815e2d0 2024-04-25T00:05:22.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.499+0000 7f25fffff640 1 -- 172.21.15.113:0/2523054486 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f260c07a050 con 0x7f261815e2d0 2024-04-25T00:05:22.500 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:05:22.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f260406be30 msgr2=0x7f260406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f260406be30 0x7f260406e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2608006e80 tx=0x7f2608008040 comp rx=0 tx=0).stop 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f261815e2d0 msgr2=0x7f2618150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f261815e2d0 0x7f2618150ed0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f260c02a2b0 tx=0x7f260c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 shutdown_connections 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f260406be30 0x7f260406e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f261815e6b0 0x7f261814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f261815e2d0 0x7f2618150ed0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 --2- 172.21.15.113:0/2523054486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26180b3080 0x7f26180ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 >> 172.21.15.113:0/2523054486 conn(0x7f261806b0f0 msgr2=0x7f26180afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 shutdown_connections 2024-04-25T00:05:22.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.501+0000 7f261e0d6640 1 -- 172.21.15.113:0/2523054486 wait complete. 2024-04-25T00:05:22.512 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:05:22.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.612+0000 7f18a98ae640 1 Processor -- start 2024-04-25T00:05:22.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.612+0000 7f18a98ae640 1 -- start start 2024-04-25T00:05:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.612+0000 7f18a98ae640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f18a415e2d0 0x7f18a415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.612+0000 7f18a98ae640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18a40b3080 0x7f18a40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.612+0000 7f18a98ae640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a415f4f0 con 0x7f18a415e6b0 2024-04-25T00:05:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a98ae640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f18a40cde00 con 0x7f18a40b3080 2024-04-25T00:05:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a98ae640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f18a415f7d0 con 0x7f18a415e2d0 2024-04-25T00:05:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a27fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18a40b3080 0x7f18a40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f18a415e6b0 0x7f18a4071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33120/0 (socket says 172.21.15.113:33120) 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a27fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18a40b3080 0x7f18a40b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53946/0 (socket says 172.21.15.113:53946) 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a37fe640 1 -- 172.21.15.113:0/2103054057 learned_addr learned my addr 172.21.15.113:0/2103054057 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a27fc640 1 -- 172.21.15.113:0/2103054057 >> v2:172.21.15.178:3300/0 conn(0x7f18a415e2d0 msgr2=0x7f18a415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a27fc640 1 --2- 172.21.15.113:0/2103054057 >> v2:172.21.15.178:3300/0 conn(0x7f18a415e2d0 0x7f18a415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a27fc640 1 -- 172.21.15.113:0/2103054057 >> v1:172.21.15.113:6789/0 conn(0x7f18a415e6b0 legacy=0x7f18a4071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a27fc640 1 -- 172.21.15.113:0/2103054057 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a415fab0 con 0x7f18a40b3080 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.613+0000 7f18a1ffb640 1 -- 172.21.15.113:0/2103054057 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110792892 0 0) 0x7f18a415f4f0 con 0x7f18a415e6b0 2024-04-25T00:05:22.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a27fc640 1 --2- 172.21.15.113:0/2103054057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a40b3080 0x7f18a40b4330 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f18a40b2d40 tx=0x7f189802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78361a3aef11899b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a1ffb640 1 -- 172.21.15.113:0/2103054057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898037070 con 0x7f18a40b3080 2024-04-25T00:05:22.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a1ffb640 1 -- 172.21.15.113:0/2103054057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1898032440 con 0x7f18a40b3080 2024-04-25T00:05:22.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a1ffb640 1 -- 172.21.15.113:0/2103054057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898032810 con 0x7f18a40b3080 2024-04-25T00:05:22.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 -- 172.21.15.113:0/2103054057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a40b3080 msgr2=0x7f18a40b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 --2- 172.21.15.113:0/2103054057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a40b3080 0x7f18a40b4330 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f18a40b2d40 tx=0x7f189802a1a0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 -- 172.21.15.113:0/2103054057 shutdown_connections 2024-04-25T00:05:22.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 --2- 172.21.15.113:0/2103054057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a40b3080 0x7f18a40b4330 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 --2- 172.21.15.113:0/2103054057 >> v2:172.21.15.178:3300/0 conn(0x7f18a415e2d0 0x7f18a415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 -- 172.21.15.113:0/2103054057 >> 172.21.15.113:0/2103054057 conn(0x7f18a406b0f0 msgr2=0x7f18a406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 -- 172.21.15.113:0/2103054057 shutdown_connections 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.614+0000 7f18a98ae640 1 -- 172.21.15.113:0/2103054057 wait complete. 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.615+0000 7f18a98ae640 1 Processor -- start 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a98ae640 1 -- start start 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a98ae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18a40b3080 0x7f18a415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a98ae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a415e2d0 0x7f18a41f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a98ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a415e6b0 0x7f18a41f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a98ae640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18a415f2b0 con 0x7f18a415e6b0 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a98ae640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18a415f130 con 0x7f18a415e2d0 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a98ae640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18a40cf2a0 con 0x7f18a40b3080 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a415e6b0 0x7f18a41f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a415e6b0 0x7f18a41f67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53950/0 (socket says 172.21.15.113:53950) 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.616+0000 7f18a37fe640 1 -- 172.21.15.113:0/4036713580 learned_addr learned my addr 172.21.15.113:0/4036713580 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a27fc640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a415e2d0 0x7f18a41f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a2ffd640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18a40b3080 0x7f18a415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a37fe640 1 -- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18a40b3080 msgr2=0x7f18a415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a37fe640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18a40b3080 0x7f18a415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a37fe640 1 -- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a415e2d0 msgr2=0x7f18a41f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a37fe640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a415e2d0 0x7f18a41f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a37fe640 1 -- 172.21.15.113:0/4036713580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a4147320 con 0x7f18a415e6b0 2024-04-25T00:05:22.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a37fe640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a415e6b0 0x7f18a41f67c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f189c00c700 tx=0x7f189c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f188bfff640 1 -- 172.21.15.113:0/4036713580 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c018070 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f188bfff640 1 -- 172.21.15.113:0/4036713580 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c00ce10 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a414cdd0 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f188bfff640 1 -- 172.21.15.113:0/4036713580 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c00a860 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.617+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a414d390 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.618+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a40691a0 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.619+0000 7f188bfff640 1 -- 172.21.15.113:0/4036713580 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f189c01b070 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.619+0000 7f188bfff640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f189006be60 0x7f189006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.619+0000 7f188bfff640 1 -- 172.21.15.113:0/4036713580 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f189c08a390 con 0x7f18a415e6b0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.619+0000 7f18a2ffd640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f189006be60 0x7f189006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.620+0000 7f18a2ffd640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f189006be60 0x7f189006e320 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1894009890 tx=0x7f1894008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.623+0000 7f188bfff640 1 -- 172.21.15.113:0/4036713580 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f189c058d40 con 0x7f18a415e6b0 2024-04-25T00:05:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.799+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18a4072050 con 0x7f18a415e6b0 2024-04-25T00:05:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.799+0000 7f188bfff640 1 -- 172.21.15.113:0/4036713580 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7f189c058640 con 0x7f18a415e6b0 2024-04-25T00:05:22.800 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f189006be60 msgr2=0x7f189006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f189006be60 0x7f189006e320 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1894009890 tx=0x7f1894008040 comp rx=0 tx=0).stop 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a415e6b0 msgr2=0x7f18a41f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a415e6b0 0x7f18a41f67c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f189c00c700 tx=0x7f189c00cbc0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 shutdown_connections 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f189006be60 0x7f189006e320 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18a415e2d0 0x7f18a41f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a415e6b0 0x7f18a41f67c0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 --2- 172.21.15.113:0/4036713580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18a40b3080 0x7f18a415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 >> 172.21.15.113:0/4036713580 conn(0x7f18a406b0f0 msgr2=0x7f18a41543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 shutdown_connections 2024-04-25T00:05:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.801+0000 7f18a98ae640 1 -- 172.21.15.113:0/4036713580 wait complete. 2024-04-25T00:05:22.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:05:22.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.912+0000 7fadb2d90640 1 Processor -- start 2024-04-25T00:05:22.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.912+0000 7fadb2d90640 1 -- start start 2024-04-25T00:05:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadb2d90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadac0b3080 0x7fadac0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadb2d90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e6b0 0x7fadac0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadb2d90640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadac15f4f0 con 0x7fadac15e2d0 2024-04-25T00:05:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadb2d90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fadac0cde00 con 0x7fadac0b3080 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadb2d90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadac15f7d0 con 0x7fadac15e6b0 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadabfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadac0b3080 0x7fadac0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadabfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadac0b3080 0x7fadac0b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53956/0 (socket says 172.21.15.113:53956) 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadabfff640 1 -- 172.21.15.113:0/3586591413 learned_addr learned my addr 172.21.15.113:0/3586591413 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadb132d640 1 --2- 172.21.15.113:0/3586591413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e6b0 0x7fadac0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadabfff640 1 -- 172.21.15.113:0/3586591413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e6b0 msgr2=0x7fadac0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadabfff640 1 --2- 172.21.15.113:0/3586591413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e6b0 0x7fadac0b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadabfff640 1 -- 172.21.15.113:0/3586591413 >> v1:172.21.15.178:6789/0 conn(0x7fadac15e2d0 legacy=0x7fadac15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.913+0000 7fadabfff640 1 -- 172.21.15.113:0/3586591413 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac15fab0 con 0x7fadac0b3080 2024-04-25T00:05:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.914+0000 7fadabfff640 1 --2- 172.21.15.113:0/3586591413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac0b4330 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fada0009870 tx=0x7fada002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c1698578e92ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.914+0000 7fadab7fe640 1 -- 172.21.15.113:0/3586591413 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0037070 con 0x7fadac0b3080 2024-04-25T00:05:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.914+0000 7fadab7fe640 1 -- 172.21.15.113:0/3586591413 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada0032440 con 0x7fadac0b3080 2024-04-25T00:05:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.914+0000 7fadab7fe640 1 -- 172.21.15.113:0/3586591413 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0032810 con 0x7fadac0b3080 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 -- 172.21.15.113:0/3586591413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 msgr2=0x7fadac0b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 --2- 172.21.15.113:0/3586591413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac0b4330 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fada0009870 tx=0x7fada002a210 comp rx=0 tx=0).stop 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 -- 172.21.15.113:0/3586591413 shutdown_connections 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 --2- 172.21.15.113:0/3586591413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e6b0 0x7fadac0b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 --2- 172.21.15.113:0/3586591413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac0b4330 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 -- 172.21.15.113:0/3586591413 >> 172.21.15.113:0/3586591413 conn(0x7fadac06b0f0 msgr2=0x7fadac06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 -- 172.21.15.113:0/3586591413 shutdown_connections 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.915+0000 7fadb2d90640 1 -- 172.21.15.113:0/3586591413 wait complete. 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.916+0000 7fadb2d90640 1 Processor -- start 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.916+0000 7fadb2d90640 1 -- start start 2024-04-25T00:05:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb2d90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb2d90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e2d0 0x7fadac0ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb2d90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadac15e6b0 0x7fadac0ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb2d90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0ce410 con 0x7fadac0b3080 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb2d90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0ce290 con 0x7fadac15e6b0 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb2d90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadac0ce590 con 0x7fadac15e2d0 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac147330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53970/0 (socket says 172.21.15.113:53970) 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 -- 172.21.15.113:0/4107206851 learned_addr learned my addr 172.21.15.113:0/4107206851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadabfff640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e2d0 0x7fadac0ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 -- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e2d0 msgr2=0x7fadac0ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e2d0 0x7fadac0ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb132d640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadac15e6b0 0x7fadac0ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 -- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadac15e6b0 msgr2=0x7fadac0ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadac15e6b0 0x7fadac0ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 -- 172.21.15.113:0/4107206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac151b90 con 0x7fadac0b3080 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.917+0000 7fadb0b2c640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac147330 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00c430 tx=0x7fad9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.918+0000 7fada97fa640 1 -- 172.21.15.113:0/4107206851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c011790 con 0x7fadac0b3080 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.918+0000 7fada97fa640 1 -- 172.21.15.113:0/4107206851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9c017070 con 0x7fadac0b3080 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.918+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac151e70 con 0x7fadac0b3080 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.918+0000 7fada97fa640 1 -- 172.21.15.113:0/4107206851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c011bb0 con 0x7fadac0b3080 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.918+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac0aeeb0 con 0x7fadac0b3080 2024-04-25T00:05:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.919+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac0691a0 con 0x7fadac0b3080 2024-04-25T00:05:22.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.919+0000 7fada97fa640 1 -- 172.21.15.113:0/4107206851 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad9c01c460 con 0x7fadac0b3080 2024-04-25T00:05:22.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.919+0000 7fada97fa640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:22.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.920+0000 7fada97fa640 1 -- 172.21.15.113:0/4107206851 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fad9c08add0 con 0x7fadac0b3080 2024-04-25T00:05:22.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.920+0000 7fadabfff640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:22.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.920+0000 7fadabfff640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9806be30 0x7fad9806e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:22.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:22.924+0000 7fada97fa640 1 -- 172.21.15.113:0/4107206851 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad9c059780 con 0x7fadac0b3080 2024-04-25T00:05:22.996 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:05:22.998 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:05:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.102+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fadac072050 con 0x7fadac0b3080 2024-04-25T00:05:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.103+0000 7fada97fa640 1 -- 172.21.15.113:0/4107206851 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7fad9c08c070 con 0x7fadac0b3080 2024-04-25T00:05:23.106 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:05:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.104+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9806be30 msgr2=0x7fad9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.104+0000 7fadb2d90640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9806be30 0x7fad9806e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002e000 comp rx=0 tx=0).stop 2024-04-25T00:05:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.104+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 msgr2=0x7fadac147330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.104+0000 7fadb2d90640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac147330 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00c430 tx=0x7fad9c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 shutdown_connections 2024-04-25T00:05:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9806be30 0x7fad9806e2f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadac15e6b0 0x7fadac0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadac15e2d0 0x7fadac0ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 --2- 172.21.15.113:0/4107206851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadac0b3080 0x7fadac147330 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 >> 172.21.15.113:0/4107206851 conn(0x7fadac06b0f0 msgr2=0x7fadac0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 shutdown_connections 2024-04-25T00:05:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.105+0000 7fadb2d90640 1 -- 172.21.15.113:0/4107206851 wait complete. 2024-04-25T00:05:23.116 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:23.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.213+0000 7fc88e038640 1 Processor -- start 2024-04-25T00:05:23.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.213+0000 7fc88e038640 1 -- start start 2024-04-25T00:05:23.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc88e038640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc88815e320 0x7fc8880d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc88e038640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc8880d0b00 0x7fc8880d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc88e038640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8880d3de0 con 0x7fc8880d2290 2024-04-25T00:05:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc88e038640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc8880d14b0 con 0x7fc88815e320 2024-04-25T00:05:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc88e038640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc8880d3410 con 0x7fc8880d0b00 2024-04-25T00:05:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc8877fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc88815e320 0x7fc8880d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc886ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc8880d2290 0x7fc8880b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33124/0 (socket says 172.21.15.113:33124) 2024-04-25T00:05:23.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc886ffd640 1 -- 172.21.15.113:0/790199102 learned_addr learned my addr 172.21.15.113:0/790199102 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc8877fe640 1 -- 172.21.15.113:0/790199102 >> v2:172.21.15.178:3300/0 conn(0x7fc8880d0b00 msgr2=0x7fc8880d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc8877fe640 1 --2- 172.21.15.113:0/790199102 >> v2:172.21.15.178:3300/0 conn(0x7fc8880d0b00 0x7fc8880d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc8877fe640 1 -- 172.21.15.113:0/790199102 >> v1:172.21.15.113:6789/0 conn(0x7fc8880d2290 legacy=0x7fc8880b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc8877fe640 1 -- 172.21.15.113:0/790199102 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8880d2800 con 0x7fc88815e320 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.214+0000 7fc8867fc640 1 -- 172.21.15.113:0/790199102 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291273 0 0) 0x7fc8880d3de0 con 0x7fc8880d2290 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.215+0000 7fc8877fe640 1 --2- 172.21.15.113:0/790199102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8880d38a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc8880d2030 tx=0x7fc87802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f19b9b9677b66bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.215+0000 7fc8867fc640 1 -- 172.21.15.113:0/790199102 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878037070 con 0x7fc88815e320 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.215+0000 7fc8867fc640 1 -- 172.21.15.113:0/790199102 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc878032440 con 0x7fc88815e320 2024-04-25T00:05:23.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.215+0000 7fc8867fc640 1 -- 172.21.15.113:0/790199102 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878032810 con 0x7fc88815e320 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 -- 172.21.15.113:0/790199102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 msgr2=0x7fc8880d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 --2- 172.21.15.113:0/790199102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8880d38a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc8880d2030 tx=0x7fc87802a230 comp rx=0 tx=0).stop 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 -- 172.21.15.113:0/790199102 shutdown_connections 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 --2- 172.21.15.113:0/790199102 >> v2:172.21.15.178:3300/0 conn(0x7fc8880d0b00 0x7fc8880d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 --2- 172.21.15.113:0/790199102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8880d38a0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 -- 172.21.15.113:0/790199102 >> 172.21.15.113:0/790199102 conn(0x7fc8880b5c20 msgr2=0x7fc8880b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 -- 172.21.15.113:0/790199102 shutdown_connections 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.216+0000 7fc88e038640 1 -- 172.21.15.113:0/790199102 wait complete. 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.217+0000 7fc88e038640 1 Processor -- start 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.217+0000 7fc88e038640 1 -- start start 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc88e038640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880d0b00 0x7fc8881613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc88e038640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880d2290 0x7fc88815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc88e038640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8881f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc88e038640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880d30a0 con 0x7fc88815e320 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc88e038640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880d2dc0 con 0x7fc8880d2290 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc88e038640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc88815e700 con 0x7fc8880d0b00 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8881f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8881f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53996/0 (socket says 172.21.15.113:53996) 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 -- 172.21.15.113:0/1092817284 learned_addr learned my addr 172.21.15.113:0/1092817284 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc886ffd640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880d2290 0x7fc88815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc8877fe640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880d0b00 0x7fc8881613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 -- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880d0b00 msgr2=0x7fc8881613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880d0b00 0x7fc8881613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 -- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880d2290 msgr2=0x7fc88815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880d2290 0x7fc88815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 -- 172.21.15.113:0/1092817284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8881618f0 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc886ffd640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880d2290 0x7fc88815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc8877fe640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880d0b00 0x7fc8881613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc887fff640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8881f7a20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc88000b9a0 tx=0x7fc88000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.218+0000 7fc86ffff640 1 -- 172.21.15.113:0/1092817284 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880019070 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.219+0000 7fc86ffff640 1 -- 172.21.15.113:0/1092817284 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88000cbc0 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.219+0000 7fc86ffff640 1 -- 172.21.15.113:0/1092817284 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880014440 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.219+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc888158d70 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.219+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc888158160 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.219+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8880691a0 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.220+0000 7fc86ffff640 1 -- 172.21.15.113:0/1092817284 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc88001c070 con 0x7fc88815e320 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.220+0000 7fc86ffff640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87406be30 0x7fc87406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.220+0000 7fc8877fe640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87406be30 0x7fc87406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.220+0000 7fc86ffff640 1 -- 172.21.15.113:0/1092817284 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fc88008ab20 con 0x7fc88815e320 2024-04-25T00:05:23.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.221+0000 7fc8877fe640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87406be30 0x7fc87406e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc878034000 tx=0x7fc87802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.225+0000 7fc86ffff640 1 -- 172.21.15.113:0/1092817284 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc8881618f0 con 0x7fc88815e320 2024-04-25T00:05:23.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.434+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8880b7610 con 0x7fc88815e320 2024-04-25T00:05:23.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.434+0000 7fc86ffff640 1 -- 172.21.15.113:0/1092817284 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7fc880059560 con 0x7fc88815e320 2024-04-25T00:05:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:23.435 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:23.435 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87406be30 msgr2=0x7fc87406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87406be30 0x7fc87406e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc878034000 tx=0x7fc87802e000 comp rx=0 tx=0).stop 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 msgr2=0x7fc8881f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8881f7a20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc88000b9a0 tx=0x7fc88000be60 comp rx=0 tx=0).stop 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 shutdown_connections 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc87406be30 0x7fc87406e2f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8880d2290 0x7fc88815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc88815e320 0x7fc8881f7a20 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 --2- 172.21.15.113:0/1092817284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8880d0b00 0x7fc8881613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.436+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 >> 172.21.15.113:0/1092817284 conn(0x7fc8880b5c20 msgr2=0x7fc8880d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:23.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.437+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 shutdown_connections 2024-04-25T00:05:23.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.437+0000 7fc88e038640 1 -- 172.21.15.113:0/1092817284 wait complete. 2024-04-25T00:05:23.448 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:23.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.544+0000 7f1b671b0640 1 Processor -- start 2024-04-25T00:05:23.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.544+0000 7f1b671b0640 1 -- start start 2024-04-25T00:05:23.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b671b0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b60144910 0x7f1b600b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b671b0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b60144cf0 con 0x7f1b600b5240 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b671b0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b600b2f30 con 0x7f1b600b56b0 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b671b0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1b600b4510 con 0x7f1b60144910 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b64f4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b60144910 0x7f1b600b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b5ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1b600b5240 0x7f1b6006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33128/0 (socket says 172.21.15.113:33128) 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b5ffff640 1 -- 172.21.15.113:0/3217890901 learned_addr learned my addr 172.21.15.113:0/3217890901 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b64f4c640 1 -- 172.21.15.113:0/3217890901 >> v1:172.21.15.178:6789/0 conn(0x7f1b600b56b0 legacy=0x7f1b600cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b64f4c640 1 -- 172.21.15.113:0/3217890901 >> v1:172.21.15.113:6789/0 conn(0x7f1b600b5240 legacy=0x7f1b6006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b64f4c640 1 -- 172.21.15.113:0/3217890901 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b600b3210 con 0x7f1b60144910 2024-04-25T00:05:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.545+0000 7f1b5f7fe640 1 -- 172.21.15.113:0/3217890901 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429962158 0 0) 0x7f1b60144cf0 con 0x7f1b600b5240 2024-04-25T00:05:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.546+0000 7f1b64f4c640 1 --2- 172.21.15.113:0/3217890901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b60144910 0x7f1b600b3fd0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400ba70 tx=0x7f1b5402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859b15c7fafbf867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.546+0000 7f1b5f7fe640 1 -- 172.21.15.113:0/3217890901 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5402f070 con 0x7f1b60144910 2024-04-25T00:05:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.546+0000 7f1b5f7fe640 1 -- 172.21.15.113:0/3217890901 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b54038070 con 0x7f1b60144910 2024-04-25T00:05:23.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.546+0000 7f1b5f7fe640 1 -- 172.21.15.113:0/3217890901 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54032a00 con 0x7f1b60144910 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.547+0000 7f1b671b0640 1 -- 172.21.15.113:0/3217890901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b60144910 msgr2=0x7f1b600b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.547+0000 7f1b671b0640 1 --2- 172.21.15.113:0/3217890901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b60144910 0x7f1b600b3fd0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400ba70 tx=0x7f1b5402c920 comp rx=0 tx=0).stop 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.547+0000 7f1b671b0640 1 -- 172.21.15.113:0/3217890901 shutdown_connections 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.547+0000 7f1b671b0640 1 --2- 172.21.15.113:0/3217890901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b60144910 0x7f1b600b3fd0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.547+0000 7f1b671b0640 1 -- 172.21.15.113:0/3217890901 >> 172.21.15.113:0/3217890901 conn(0x7f1b600b1a50 msgr2=0x7f1b600b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.547+0000 7f1b671b0640 1 -- 172.21.15.113:0/3217890901 shutdown_connections 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.547+0000 7f1b671b0640 1 -- 172.21.15.113:0/3217890901 wait complete. 2024-04-25T00:05:23.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.548+0000 7f1b671b0640 1 Processor -- start 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.548+0000 7f1b671b0640 1 -- start start 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b600b5240 0x7f1b6015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b600b56b0 0x7f1b6014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b60144910 0x7f1b60140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1b600b4da0 con 0x7f1b600b56b0 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b600b37d0 con 0x7f1b60144910 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b600b4f20 con 0x7f1b600b5240 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b600b56b0 0x7f1b6014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b600b56b0 0x7f1b6014ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54008/0 (socket says 172.21.15.113:54008) 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 -- 172.21.15.113:0/2018937843 learned_addr learned my addr 172.21.15.113:0/2018937843 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b64f4c640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b600b5240 0x7f1b6015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b6574d640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b60144910 0x7f1b60140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 -- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b600b5240 msgr2=0x7f1b6015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b600b5240 0x7f1b6015d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 -- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b60144910 msgr2=0x7f1b60140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b60144910 0x7f1b60140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 -- 172.21.15.113:0/2018937843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6014f440 con 0x7f1b600b56b0 2024-04-25T00:05:23.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5ffff640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b600b56b0 0x7f1b6014ef00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c4a0 tx=0x7f1b4c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5d7fa640 1 -- 172.21.15.113:0/2018937843 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c018070 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5d7fa640 1 -- 172.21.15.113:0/2018937843 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c0074e0 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b5d7fa640 1 -- 172.21.15.113:0/2018937843 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c004a20 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6013dd00 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.549+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6013e2c0 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.550+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b600691a0 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.551+0000 7f1b5d7fa640 1 -- 172.21.15.113:0/2018937843 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b4c01b070 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.551+0000 7f1b5d7fa640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b4406be30 0x7f1b4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.551+0000 7f1b5d7fa640 1 -- 172.21.15.113:0/2018937843 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f1b4c08a100 con 0x7f1b600b56b0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.551+0000 7f1b64f4c640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b4406be30 0x7f1b4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.552+0000 7f1b64f4c640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b4406be30 0x7f1b4406e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1b54037000 tx=0x7f1b54032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.555+0000 7f1b5d7fa640 1 -- 172.21.15.113:0/2018937843 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b6014f440 con 0x7f1b600b56b0 2024-04-25T00:05:23.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.731+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b600731a0 con 0x7f1b600b56b0 2024-04-25T00:05:23.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.732+0000 7f1b5d7fa640 1 -- 172.21.15.113:0/2018937843 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16667 (secure 0 0 0) 0x7f1b4c0583d0 con 0x7f1b600b56b0 2024-04-25T00:05:23.733 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:23.733 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:51.690687+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:48.259110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:49.014993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:23.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b4406be30 msgr2=0x7f1b4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b4406be30 0x7f1b4406e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1b54037000 tx=0x7f1b54032000 comp rx=0 tx=0).stop 2024-04-25T00:05:23.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b600b56b0 msgr2=0x7f1b6014ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b600b56b0 0x7f1b6014ef00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c4a0 tx=0x7f1b4c00c960 comp rx=0 tx=0).stop 2024-04-25T00:05:23.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 shutdown_connections 2024-04-25T00:05:23.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b4406be30 0x7f1b4406e2f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b60144910 0x7f1b60140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b600b56b0 0x7f1b6014ef00 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 --2- 172.21.15.113:0/2018937843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b600b5240 0x7f1b6015d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 >> 172.21.15.113:0/2018937843 conn(0x7f1b600b1a50 msgr2=0x7f1b6006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:23.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 shutdown_connections 2024-04-25T00:05:23.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.734+0000 7f1b671b0640 1 -- 172.21.15.113:0/2018937843 wait complete. 2024-04-25T00:05:23.747 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:05:23.747 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.845+0000 7f3203e64640 1 Processor -- start 2024-04-25T00:05:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.845+0000 7f3203e64640 1 -- start start 2024-04-25T00:05:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f3203e64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 0x7f31fc070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f3203e64640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f31fc0b3af0 0x7f31fc0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f3203e64640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31fc0b2d50 con 0x7f31fc072ea0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f3203e64640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc072a70 con 0x7f31fc0b3af0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f3203e64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc0b3010 con 0x7f31fc15a6b0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f32013ff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f31fc072ea0 0x7f31fc071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33144/0 (socket says 172.21.15.113:33144) 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f32013ff640 1 -- 172.21.15.113:0/709458893 learned_addr learned my addr 172.21.15.113:0/709458893 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3202401640 1 --2- 172.21.15.113:0/709458893 >> v2:172.21.15.178:3300/0 conn(0x7f31fc0b3af0 0x7f31fc0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.846+0000 7f3201c00640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 0x7f31fc070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021816822 0 0) 0x7f31fc0b2d50 con 0x7f31fc072ea0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31e8003440 con 0x7f31fc072ea0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 307446554 0 0) 0x7f31e8003440 con 0x7f31fc072ea0 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 msgr2=0x7f31fc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 0x7f31fc070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 >> v2:172.21.15.178:3300/0 conn(0x7f31fc0b3af0 msgr2=0x7f31fc0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 --2- 172.21.15.113:0/709458893 >> v2:172.21.15.178:3300/0 conn(0x7f31fc0b3af0 0x7f31fc0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc0b32f0 con 0x7f31fc072ea0 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f31ec001670 con 0x7f31fc072ea0 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 >> v1:172.21.15.113:6789/0 conn(0x7f31fc072ea0 legacy=0x7f31fc071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31e8003aa0 0x7f31e8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e8008850 0x7f31e800ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 0x7f31e800d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc072a70 con 0x7f31e8008850 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0b3010 con 0x7f31e8003aa0 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31e800db20 con 0x7f31fc15a6b0 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f32013ff640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e8008850 0x7f31e800ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3201c00640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31e8003aa0 0x7f31e8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f3202401640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 0x7f31e800d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f32013ff640 1 -- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 msgr2=0x7f31e800d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f32013ff640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31fc15a6b0 0x7f31e800d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f32013ff640 1 -- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31e8003aa0 msgr2=0x7f31e8003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f32013ff640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31e8003aa0 0x7f31e8003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.847+0000 7f32013ff640 1 -- 172.21.15.113:0/709458893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e8003440 con 0x7f31e8008850 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.848+0000 7f32013ff640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e8008850 0x7f31e800ac80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f31ec007fd0 tx=0x7f31e8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adac5a6d6634d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.848+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007d50 con 0x7f31e8008850 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.848+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec005ad0 con 0x7f31e8008850 2024-04-25T00:05:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.848+0000 7f3200bfe640 1 -- 172.21.15.113:0/709458893 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007070 con 0x7f31e8008850 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.849+0000 7f3203e64640 1 -- 172.21.15.113:0/709458893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e8008850 msgr2=0x7f31e800ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.849+0000 7f3203e64640 1 --2- 172.21.15.113:0/709458893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e8008850 0x7f31e800ac80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f31ec007fd0 tx=0x7f31e8003730 comp rx=0 tx=0).stop 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.849+0000 7f3202401640 1 -- 172.21.15.113:0/709458893 reap_dead start 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.849+0000 7f3203e64640 1 -- 172.21.15.113:0/709458893 shutdown_connections 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.849+0000 7f3203e64640 1 -- 172.21.15.113:0/709458893 >> 172.21.15.113:0/709458893 conn(0x7f31fc0b1a50 msgr2=0x7f31fc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.849+0000 7f3203e64640 1 -- 172.21.15.113:0/709458893 shutdown_connections 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.849+0000 7f3203e64640 1 -- 172.21.15.113:0/709458893 wait complete. 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.850+0000 7f3203e64640 1 Processor -- start 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.850+0000 7f3203e64640 1 -- start start 2024-04-25T00:05:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3203e64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e8008850 0x7f31fc155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3203e64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31fc157110 0x7f31fc14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3203e64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31fc158d50 0x7f31fc1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3203e64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc15aa90 con 0x7f31fc157110 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3203e64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc072720 con 0x7f31fc158d50 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3203e64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc15ac10 con 0x7f31e8008850 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31fc157110 0x7f31fc14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31fc157110 0x7f31fc14e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54034/0 (socket says 172.21.15.113:54034) 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 -- 172.21.15.113:0/2576991830 learned_addr learned my addr 172.21.15.113:0/2576991830 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3201c00640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e8008850 0x7f31fc155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3202401640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31fc158d50 0x7f31fc1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 -- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e8008850 msgr2=0x7f31fc155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e8008850 0x7f31fc155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 -- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31fc158d50 msgr2=0x7f31fc1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31fc158d50 0x7f31fc1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 -- 172.21.15.113:0/2576991830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc1578f0 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f3202401640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31fc158d50 0x7f31fc1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f32013ff640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31fc157110 0x7f31fc14e800 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c380 tx=0x7f31fc0b34c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.851+0000 7f31f27fc640 1 -- 172.21.15.113:0/2576991830 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec042030 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.852+0000 7f31f27fc640 1 -- 172.21.15.113:0/2576991830 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec0049f0 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.852+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc157bd0 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.852+0000 7f31f27fc640 1 -- 172.21.15.113:0/2576991830 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec001c80 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.852+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc158190 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.853+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc0691a0 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.853+0000 7f31f27fc640 1 -- 172.21.15.113:0/2576991830 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31ec045070 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.853+0000 7f31f27fc640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f31e006be60 0x7f31e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.854+0000 7f31f27fc640 1 -- 172.21.15.113:0/2576991830 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f31ec0aaf30 con 0x7f31fc157110 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.854+0000 7f3201c00640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f31e006be60 0x7f31e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.854+0000 7f3201c00640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f31e006be60 0x7f31e006e320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f31fc1f3830 tx=0x7f31f8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:23.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:23.858+0000 7f31f27fc640 1 -- 172.21.15.113:0/2576991830 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31ec0798e0 con 0x7f31fc157110 2024-04-25T00:05:24.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.067+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31fc0b2900 con 0x7f31fc157110 2024-04-25T00:05:24.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.067+0000 7f31f27fc640 1 -- 172.21.15.113:0/2576991830 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7f31ec0791e0 con 0x7f31fc157110 2024-04-25T00:05:24.068 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:24.068 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:24.068 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.069+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f31e006be60 msgr2=0x7f31e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.069+0000 7f3203e64640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f31e006be60 0x7f31e006e320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f31fc1f3830 tx=0x7f31f8009190 comp rx=0 tx=0).stop 2024-04-25T00:05:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31fc157110 msgr2=0x7f31fc14e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31fc157110 0x7f31fc14e800 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c380 tx=0x7f31fc0b34c0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 shutdown_connections 2024-04-25T00:05:24.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f31e006be60 0x7f31e006e320 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31fc158d50 0x7f31fc1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31fc157110 0x7f31fc14e800 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 --2- 172.21.15.113:0/2576991830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e8008850 0x7f31fc155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 >> 172.21.15.113:0/2576991830 conn(0x7f31fc0b1a50 msgr2=0x7f31fc14f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:24.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 shutdown_connections 2024-04-25T00:05:24.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.070+0000 7f3203e64640 1 -- 172.21.15.113:0/2576991830 wait complete. 2024-04-25T00:05:24.081 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.178+0000 7fdbdf196640 1 Processor -- start 2024-04-25T00:05:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.178+0000 7fdbdf196640 1 -- start start 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdf196640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbd80d2290 0x7fdbd80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdf196640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbd80d0b00 0x7fdbd80d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdf196640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd80d1480 con 0x7fdbd815e320 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdf196640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd80d3410 con 0x7fdbd80d0b00 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdf196640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd80d2800 con 0x7fdbd80d2290 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdd733640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbd80d0b00 0x7fdbd80d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdd733640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbd80d0b00 0x7fdbd80d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54046/0 (socket says 172.21.15.113:54046) 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.179+0000 7fdbdd733640 1 -- 172.21.15.113:0/343577963 learned_addr learned my addr 172.21.15.113:0/343577963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.180+0000 7fdbd7fff640 1 --2- 172.21.15.113:0/343577963 >> v2:172.21.15.178:3300/0 conn(0x7fdbd80d2290 0x7fdbd80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.180+0000 7fdbdd733640 1 -- 172.21.15.113:0/343577963 >> v2:172.21.15.178:3300/0 conn(0x7fdbd80d2290 msgr2=0x7fdbd80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.180+0000 7fdbdd733640 1 --2- 172.21.15.113:0/343577963 >> v2:172.21.15.178:3300/0 conn(0x7fdbd80d2290 0x7fdbd80b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.180+0000 7fdbdd733640 1 -- 172.21.15.113:0/343577963 >> v1:172.21.15.178:6789/0 conn(0x7fdbd815e320 legacy=0x7fdbd80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.180+0000 7fdbdd733640 1 -- 172.21.15.113:0/343577963 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd80d2ae0 con 0x7fdbd80d0b00 2024-04-25T00:05:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.180+0000 7fdbdd733640 1 --2- 172.21.15.113:0/343577963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d0b00 0x7fdbd80d0f40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8072810 tx=0x7fdbd002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3243105ed8c40f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.180+0000 7fdbd77fe640 1 -- 172.21.15.113:0/343577963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0036070 con 0x7fdbd80d0b00 2024-04-25T00:05:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbd77fe640 1 -- 172.21.15.113:0/343577963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd0031600 con 0x7fdbd80d0b00 2024-04-25T00:05:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbd77fe640 1 -- 172.21.15.113:0/343577963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00319d0 con 0x7fdbd80d0b00 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 -- 172.21.15.113:0/343577963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d0b00 msgr2=0x7fdbd80d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 --2- 172.21.15.113:0/343577963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d0b00 0x7fdbd80d0f40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8072810 tx=0x7fdbd002a780 comp rx=0 tx=0).stop 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 -- 172.21.15.113:0/343577963 shutdown_connections 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 --2- 172.21.15.113:0/343577963 >> v2:172.21.15.178:3300/0 conn(0x7fdbd80d2290 0x7fdbd80b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 --2- 172.21.15.113:0/343577963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d0b00 0x7fdbd80d0f40 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 -- 172.21.15.113:0/343577963 >> 172.21.15.113:0/343577963 conn(0x7fdbd80b5c20 msgr2=0x7fdbd80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 -- 172.21.15.113:0/343577963 shutdown_connections 2024-04-25T00:05:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.181+0000 7fdbdf196640 1 -- 172.21.15.113:0/343577963 wait complete. 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.182+0000 7fdbdf196640 1 Processor -- start 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.182+0000 7fdbdf196640 1 -- start start 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdf196640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbd80d0b00 0x7fdbd8154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdf196640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d2290 0x7fdbd8146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdf196640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbd815e320 0x7fdbd80cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdf196640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80b9fe0 con 0x7fdbd80d2290 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdf196640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80d30a0 con 0x7fdbd815e320 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdf196640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd815e700 con 0x7fdbd80d0b00 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d2290 0x7fdbd8146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d2290 0x7fdbd8146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54054/0 (socket says 172.21.15.113:54054) 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbd7fff640 1 -- 172.21.15.113:0/4266206207 learned_addr learned my addr 172.21.15.113:0/4266206207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdd733640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbd815e320 0x7fdbd80cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.183+0000 7fdbdcf32640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbd80d0b00 0x7fdbd8154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd7fff640 1 -- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbd80d0b00 msgr2=0x7fdbd8154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd7fff640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbd80d0b00 0x7fdbd8154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd7fff640 1 -- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbd815e320 msgr2=0x7fdbd80cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd7fff640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbd815e320 0x7fdbd80cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd7fff640 1 -- 172.21.15.113:0/4266206207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd81470f0 con 0x7fdbd80d2290 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbdd733640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbd815e320 0x7fdbd80cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbdcf32640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbd80d0b00 0x7fdbd8154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd7fff640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d2290 0x7fdbd8146bb0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c820 tx=0x7fdbcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd57fa640 1 -- 172.21.15.113:0/4266206207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc015070 con 0x7fdbd80d2290 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd57fa640 1 -- 172.21.15.113:0/4266206207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc004240 con 0x7fdbd80d2290 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd81f7560 con 0x7fdbd80d2290 2024-04-25T00:05:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbd57fa640 1 -- 172.21.15.113:0/4266206207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00ce10 con 0x7fdbd80d2290 2024-04-25T00:05:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.184+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd81f7ae0 con 0x7fdbd80d2290 2024-04-25T00:05:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.185+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd80691a0 con 0x7fdbd80d2290 2024-04-25T00:05:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.185+0000 7fdbd57fa640 1 -- 172.21.15.113:0/4266206207 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbcc007830 con 0x7fdbd80d2290 2024-04-25T00:05:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.186+0000 7fdbd57fa640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdbc406be30 0x7fdbc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.186+0000 7fdbd57fa640 1 -- 172.21.15.113:0/4266206207 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fdbcc08ad10 con 0x7fdbd80d2290 2024-04-25T00:05:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.186+0000 7fdbdcf32640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdbc406be30 0x7fdbc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.186+0000 7fdbdcf32640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdbc406be30 0x7fdbc406e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001750 tx=0x7fdbc8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.190+0000 7fdbd57fa640 1 -- 172.21.15.113:0/4266206207 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdbd81470f0 con 0x7fdbd80d2290 2024-04-25T00:05:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.398+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd8074f50 con 0x7fdbd80d2290 2024-04-25T00:05:24.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.398+0000 7fdbd57fa640 1 -- 172.21.15.113:0/4266206207 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5192 (secure 0 0 0) 0x7fdbcc059750 con 0x7fdbd80d2290 2024-04-25T00:05:24.399 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:24.399 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:04:50.047438+0000","modified":"2024-04-25T00:04:52.043934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/677893386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":677893386},{"type":"v1","addr":"172.21.15.113:6836","nonce":677893386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":7}]} 2024-04-25T00:05:24.400 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-04-25T00:05:24.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdbc406be30 msgr2=0x7fdbc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdbc406be30 0x7fdbc406e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001750 tx=0x7fdbc8009040 comp rx=0 tx=0).stop 2024-04-25T00:05:24.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d2290 msgr2=0x7fdbd8146bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d2290 0x7fdbd8146bb0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c820 tx=0x7fdbcc00cce0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 shutdown_connections 2024-04-25T00:05:24.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdbc406be30 0x7fdbc406e2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbd815e320 0x7fdbd80cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd80d2290 0x7fdbd8146bb0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 --2- 172.21.15.113:0/4266206207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbd80d0b00 0x7fdbd8154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 >> 172.21.15.113:0/4266206207 conn(0x7fdbd80b5c20 msgr2=0x7fdbd80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:24.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 shutdown_connections 2024-04-25T00:05:24.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.400+0000 7fdbdf196640 1 -- 172.21.15.113:0/4266206207 wait complete. 2024-04-25T00:05:24.412 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:24.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.509+0000 7f1e6f5c8640 1 Processor -- start 2024-04-25T00:05:24.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.509+0000 7f1e6f5c8640 1 -- start start 2024-04-25T00:05:24.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6f5c8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e680b3fd0 0x7f1e6806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6f5c8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e68144cf0 con 0x7f1e6806e510 2024-04-25T00:05:24.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6f5c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e680b2f30 con 0x7f1e68144910 2024-04-25T00:05:24.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6f5c8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6806e980 con 0x7f1e680b3fd0 2024-04-25T00:05:24.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6cb63640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e680b3fd0 0x7f1e6806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6db65640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1e6806e510 0x7f1e680cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33146/0 (socket says 172.21.15.113:33146) 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6db65640 1 -- 172.21.15.113:0/3308740121 learned_addr learned my addr 172.21.15.113:0/3308740121 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6cb63640 1 -- 172.21.15.113:0/3308740121 >> v1:172.21.15.178:6789/0 conn(0x7f1e68144910 legacy=0x7f1e680b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6cb63640 1 -- 172.21.15.113:0/3308740121 >> v1:172.21.15.113:6789/0 conn(0x7f1e6806e510 legacy=0x7f1e680cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.510+0000 7f1e6cb63640 1 -- 172.21.15.113:0/3308740121 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e680b3210 con 0x7f1e680b3fd0 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e6cb63640 1 --2- 172.21.15.113:0/3308740121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e680b3fd0 0x7f1e6806dfd0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097d0 tx=0x7f1e5802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9951ff6d4791be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e5ffff640 1 -- 172.21.15.113:0/3308740121 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58037070 con 0x7f1e680b3fd0 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e5ffff640 1 -- 172.21.15.113:0/3308740121 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e58032440 con 0x7f1e680b3fd0 2024-04-25T00:05:24.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e5ffff640 1 -- 172.21.15.113:0/3308740121 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58032810 con 0x7f1e680b3fd0 2024-04-25T00:05:24.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/3308740121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e680b3fd0 msgr2=0x7f1e6806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/3308740121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e680b3fd0 0x7f1e6806dfd0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097d0 tx=0x7f1e5802a210 comp rx=0 tx=0).stop 2024-04-25T00:05:24.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/3308740121 shutdown_connections 2024-04-25T00:05:24.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.511+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/3308740121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e680b3fd0 0x7f1e6806dfd0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.512+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/3308740121 >> 172.21.15.113:0/3308740121 conn(0x7f1e680b1a50 msgr2=0x7f1e680b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:24.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.512+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/3308740121 shutdown_connections 2024-04-25T00:05:24.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.512+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/3308740121 wait complete. 2024-04-25T00:05:24.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 Processor -- start 2024-04-25T00:05:24.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 -- start start 2024-04-25T00:05:24.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6806e510 0x7f1e681f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e680b3fd0 0x7f1e6814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e68144910 0x7f1e6813c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e680b4440 con 0x7f1e68144910 2024-04-25T00:05:24.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e680b37d0 con 0x7f1e680b3fd0 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6f5c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e680b4da0 con 0x7f1e6806e510 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.513+0000 7f1e6db65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e68144910 0x7f1e6813c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e68144910 0x7f1e6813c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54076/0 (socket says 172.21.15.113:54076) 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 -- 172.21.15.113:0/2108130586 learned_addr learned my addr 172.21.15.113:0/2108130586 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6cb63640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e680b3fd0 0x7f1e6814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6d364640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6806e510 0x7f1e681f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 -- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6806e510 msgr2=0x7f1e681f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6806e510 0x7f1e681f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 -- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e680b3fd0 msgr2=0x7f1e6814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e680b3fd0 0x7f1e6814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 -- 172.21.15.113:0/2108130586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6814fde0 con 0x7f1e68144910 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6cb63640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e680b3fd0 0x7f1e6814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6db65640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e68144910 0x7f1e6813c690 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000ca70 tx=0x7f1e6000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2108130586 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60017070 con 0x7f1e68144910 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2108130586 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6000abd0 con 0x7f1e68144910 2024-04-25T00:05:24.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e681500c0 con 0x7f1e68144910 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2108130586 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60012400 con 0x7f1e68144910 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.514+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6814fad0 con 0x7f1e68144910 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.515+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e681504b0 con 0x7f1e68144910 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.515+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2108130586 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e6001a070 con 0x7f1e68144910 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.516+0000 7f1e5dffb640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.516+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2108130586 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f1e6008aa80 con 0x7f1e68144910 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.516+0000 7f1e6d364640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.516+0000 7f1e6d364640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006fd0 tx=0x7f1e64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.520+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2108130586 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e6008ce20 con 0x7f1e68144910 2024-04-25T00:05:24.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.693+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e680691a0 con 0x7f1e68144910 2024-04-25T00:05:24.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.694+0000 7f1e5dffb640 1 -- 172.21.15.113:0/2108130586 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16667 (secure 0 0 0) 0x7f1e60058d40 con 0x7f1e68144910 2024-04-25T00:05:24.694 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:24.695 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:04:51.690687+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:04:48.259110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T00:04:49.014993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be30 msgr2=0x7f1e4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006fd0 tx=0x7f1e64008040 comp rx=0 tx=0).stop 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e68144910 msgr2=0x7f1e6813c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e68144910 0x7f1e6813c690 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000ca70 tx=0x7f1e6000cf30 comp rx=0 tx=0).stop 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 shutdown_connections 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1e4c06be30 0x7f1e4c06e2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e680b3fd0 0x7f1e6814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e68144910 0x7f1e6813c690 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 --2- 172.21.15.113:0/2108130586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e6806e510 0x7f1e681f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 >> 172.21.15.113:0/2108130586 conn(0x7f1e680b1a50 msgr2=0x7f1e680af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 shutdown_connections 2024-04-25T00:05:24.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.696+0000 7f1e6f5c8640 1 -- 172.21.15.113:0/2108130586 wait complete. 2024-04-25T00:05:24.708 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:05:24.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.805+0000 7f6b91cb3640 1 Processor -- start 2024-04-25T00:05:24.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.805+0000 7f6b91cb3640 1 -- start start 2024-04-25T00:05:24.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b91cb3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c15c050 0x7f6b8c15c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b91cb3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6b8c074c00 0x7f6b8c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b91cb3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b8c06f180 0x7f6b8c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b91cb3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6b8c160900 con 0x7f6b8c06f180 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b91cb3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6b8c06d870 con 0x7f6b8c074c00 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b91cb3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c06db50 con 0x7f6b8c15c050 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b8c06f180 0x7f6b8c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b8c06f180 0x7f6b8c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54090/0 (socket says 172.21.15.113:54090) 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 -- 172.21.15.113:0/3876810241 learned_addr learned my addr 172.21.15.113:0/3876810241 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8b7fe640 1 --2- 172.21.15.113:0/3876810241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c15c050 0x7f6b8c15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8affd640 1 --2- 172.21.15.113:0/3876810241 >> v2:172.21.15.178:3300/0 conn(0x7f6b8c074c00 0x7f6b8c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 -- 172.21.15.113:0/3876810241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c15c050 msgr2=0x7f6b8c15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 --2- 172.21.15.113:0/3876810241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c15c050 0x7f6b8c15c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 -- 172.21.15.113:0/3876810241 >> v2:172.21.15.178:3300/0 conn(0x7f6b8c074c00 msgr2=0x7f6b8c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 --2- 172.21.15.113:0/3876810241 >> v2:172.21.15.178:3300/0 conn(0x7f6b8c074c00 0x7f6b8c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8bfff640 1 -- 172.21.15.113:0/3876810241 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c160be0 con 0x7f6b8c06f180 2024-04-25T00:05:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.806+0000 7f6b8affd640 1 --2- 172.21.15.113:0/3876810241 >> v2:172.21.15.178:3300/0 conn(0x7f6b8c074c00 0x7f6b8c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.807+0000 7f6b8bfff640 1 --2- 172.21.15.113:0/3876810241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c06f180 0x7f6b8c0cf930 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0b5cf0 tx=0x7f6b8402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513310da1a63c6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.807+0000 7f6b8a7fc640 1 -- 172.21.15.113:0/3876810241 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84036070 con 0x7f6b8c06f180 2024-04-25T00:05:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.807+0000 7f6b8a7fc640 1 -- 172.21.15.113:0/3876810241 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b84031600 con 0x7f6b8c06f180 2024-04-25T00:05:24.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.807+0000 7f6b8a7fc640 1 -- 172.21.15.113:0/3876810241 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840319d0 con 0x7f6b8c06f180 2024-04-25T00:05:24.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 -- 172.21.15.113:0/3876810241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c06f180 msgr2=0x7f6b8c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/3876810241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c06f180 0x7f6b8c0cf930 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0b5cf0 tx=0x7f6b8402a6a0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 -- 172.21.15.113:0/3876810241 shutdown_connections 2024-04-25T00:05:24.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/3876810241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c06f180 0x7f6b8c0cf930 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/3876810241 >> v2:172.21.15.178:3300/0 conn(0x7f6b8c074c00 0x7f6b8c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/3876810241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c15c050 0x7f6b8c15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 -- 172.21.15.113:0/3876810241 >> 172.21.15.113:0/3876810241 conn(0x7f6b8c06fb60 msgr2=0x7f6b8c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 -- 172.21.15.113:0/3876810241 shutdown_connections 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.808+0000 7f6b91cb3640 1 -- 172.21.15.113:0/3876810241 wait complete. 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.809+0000 7f6b91cb3640 1 Processor -- start 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.809+0000 7f6b91cb3640 1 -- start start 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b91cb3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c06f180 0x7f6b8c1f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b91cb3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c074c00 0x7f6b8c0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b91cb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c15c050 0x7f6b8c144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b91cb3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c15caf0 con 0x7f6b8c074c00 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b91cb3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c15c970 con 0x7f6b8c15c050 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b91cb3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c06e070 con 0x7f6b8c06f180 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c074c00 0x7f6b8c0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c074c00 0x7f6b8c0b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54096/0 (socket says 172.21.15.113:54096) 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 -- 172.21.15.113:0/2518022406 learned_addr learned my addr 172.21.15.113:0/2518022406 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8b7fe640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c06f180 0x7f6b8c1f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8bfff640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c15c050 0x7f6b8c144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 -- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c06f180 msgr2=0x7f6b8c1f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c06f180 0x7f6b8c1f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 -- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c15c050 msgr2=0x7f6b8c144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c15c050 0x7f6b8c144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 -- 172.21.15.113:0/2518022406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1f2c80 con 0x7f6b8c074c00 2024-04-25T00:05:24.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8bfff640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c15c050 0x7f6b8c144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8b7fe640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c06f180 0x7f6b8c1f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.810+0000 7f6b8affd640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c074c00 0x7f6b8c0b4ba0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c7f0 tx=0x7f6b8000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.811+0000 7f6b73fff640 1 -- 172.21.15.113:0/2518022406 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80015070 con 0x7f6b8c074c00 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.811+0000 7f6b73fff640 1 -- 172.21.15.113:0/2518022406 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b80007a30 con 0x7f6b8c074c00 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.811+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c145530 con 0x7f6b8c074c00 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.811+0000 7f6b73fff640 1 -- 172.21.15.113:0/2518022406 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80007e00 con 0x7f6b8c074c00 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.811+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1f1f40 con 0x7f6b8c074c00 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.812+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c0d0090 con 0x7f6b8c074c00 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.812+0000 7f6b73fff640 1 -- 172.21.15.113:0/2518022406 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b80018070 con 0x7f6b8c074c00 2024-04-25T00:05:24.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.812+0000 7f6b73fff640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b7806be30 0x7f6b7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:24.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.812+0000 7f6b73fff640 1 -- 172.21.15.113:0/2518022406 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f6b8008ae20 con 0x7f6b8c074c00 2024-04-25T00:05:24.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.812+0000 7f6b8b7fe640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b7806be30 0x7f6b7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:24.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.813+0000 7f6b8b7fe640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b7806be30 0x7f6b7806e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c1540e0 tx=0x7f6b7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:24.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:24.817+0000 7f6b73fff640 1 -- 172.21.15.113:0/2518022406 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b8c0d0090 con 0x7f6b8c074c00 2024-04-25T00:05:25.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.025+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6b8c0691a0 con 0x7f6b8c074c00 2024-04-25T00:05:25.027 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:05:25.026+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:05:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.029+0000 7f6b73fff640 1 -- 172.21.15.113:0/2518022406 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) ==== 161+0+0 (secure 0 0 0) 0x7f6b800597d0 con 0x7f6b8c074c00 2024-04-25T00:05:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b7806be30 msgr2=0x7f6b7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b7806be30 0x7f6b7806e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c1540e0 tx=0x7f6b7c008040 comp rx=0 tx=0).stop 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c074c00 msgr2=0x7f6b8c0b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c074c00 0x7f6b8c0b4ba0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c7f0 tx=0x7f6b8000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 shutdown_connections 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6b7806be30 0x7f6b7806e2f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c15c050 0x7f6b8c144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b8c074c00 0x7f6b8c0b4ba0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 --2- 172.21.15.113:0/2518022406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b8c06f180 0x7f6b8c1f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.031+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 >> 172.21.15.113:0/2518022406 conn(0x7f6b8c06fb60 msgr2=0x7f6b8c0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.032+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 shutdown_connections 2024-04-25T00:05:25.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.032+0000 7f6b91cb3640 1 -- 172.21.15.113:0/2518022406 wait complete. 2024-04-25T00:05:25.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:05:25.071 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:05:25.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.144+0000 7fa381d42640 1 Processor -- start 2024-04-25T00:05:25.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.144+0000 7fa381d42640 1 -- start start 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa381d42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa381d42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa37c15bff0 0x7fa37c0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa381d42640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa37c0cc740 0x7fa37c06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa381d42640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa37c15c470 con 0x7fa37c0cc740 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa381d42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa37c071170 con 0x7fa37c15bff0 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa381d42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c071450 con 0x7fa37c15b670 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa37c0cc740 0x7fa37c06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa37c0cc740 0x7fa37c06cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54124/0 (socket says 172.21.15.113:54124) 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 -- 172.21.15.113:0/9603847 learned_addr learned my addr 172.21.15.113:0/9603847 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37b7fe640 1 --2- 172.21.15.113:0/9603847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37affd640 1 --2- 172.21.15.113:0/9603847 >> v2:172.21.15.178:3300/0 conn(0x7fa37c15bff0 0x7fa37c0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 -- 172.21.15.113:0/9603847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 msgr2=0x7fa37c15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 --2- 172.21.15.113:0/9603847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c15ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 -- 172.21.15.113:0/9603847 >> v2:172.21.15.178:3300/0 conn(0x7fa37c15bff0 msgr2=0x7fa37c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 --2- 172.21.15.113:0/9603847 >> v2:172.21.15.178:3300/0 conn(0x7fa37c15bff0 0x7fa37c0cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37bfff640 1 -- 172.21.15.113:0/9603847 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c071730 con 0x7fa37c0cc740 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.145+0000 7fa37affd640 1 --2- 172.21.15.113:0/9603847 >> v2:172.21.15.178:3300/0 conn(0x7fa37c15bff0 0x7fa37c0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:25.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.146+0000 7fa37bfff640 1 --2- 172.21.15.113:0/9603847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c0cc740 0x7fa37c06cea0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa37c06e230 tx=0x7fa37402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c84817dac9157bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.146+0000 7fa37a7fc640 1 -- 172.21.15.113:0/9603847 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374036070 con 0x7fa37c0cc740 2024-04-25T00:05:25.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.146+0000 7fa37a7fc640 1 -- 172.21.15.113:0/9603847 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa374031600 con 0x7fa37c0cc740 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.146+0000 7fa37a7fc640 1 -- 172.21.15.113:0/9603847 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3740319d0 con 0x7fa37c0cc740 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 -- 172.21.15.113:0/9603847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c0cc740 msgr2=0x7fa37c06cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 --2- 172.21.15.113:0/9603847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c0cc740 0x7fa37c06cea0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa37c06e230 tx=0x7fa37402a570 comp rx=0 tx=0).stop 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 -- 172.21.15.113:0/9603847 shutdown_connections 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 --2- 172.21.15.113:0/9603847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c0cc740 0x7fa37c06cea0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 --2- 172.21.15.113:0/9603847 >> v2:172.21.15.178:3300/0 conn(0x7fa37c15bff0 0x7fa37c0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 --2- 172.21.15.113:0/9603847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 -- 172.21.15.113:0/9603847 >> 172.21.15.113:0/9603847 conn(0x7fa37c06b0f0 msgr2=0x7fa37c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 -- 172.21.15.113:0/9603847 shutdown_connections 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.147+0000 7fa381d42640 1 -- 172.21.15.113:0/9603847 wait complete. 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.148+0000 7fa381d42640 1 Processor -- start 2024-04-25T00:05:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.148+0000 7fa381d42640 1 -- start start 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa381d42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa37c0cc740 0x7fa37c1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa381d42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa381d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c15bff0 0x7fa37c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa381d42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c06d560 con 0x7fa37c15bff0 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa381d42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c06d3e0 con 0x7fa37c0cc740 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa381d42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c06d6e0 con 0x7fa37c15b670 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c15bff0 0x7fa37c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c15bff0 0x7fa37c0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54132/0 (socket says 172.21.15.113:54132) 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 -- 172.21.15.113:0/753108740 learned_addr learned my addr 172.21.15.113:0/753108740 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:25.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37b7fe640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa37c0cc740 0x7fa37c1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37affd640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 -- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 msgr2=0x7fa37c1ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c1ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 -- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa37c0cc740 msgr2=0x7fa37c1506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa37c0cc740 0x7fa37c1506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 -- 172.21.15.113:0/753108740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c13c420 con 0x7fa37c15bff0 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.149+0000 7fa37bfff640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c15bff0 0x7fa37c0ce0d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa374034040 tx=0x7fa374004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.150+0000 7fa363fff640 1 -- 172.21.15.113:0/753108740 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374036050 con 0x7fa37c15bff0 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.150+0000 7fa363fff640 1 -- 172.21.15.113:0/753108740 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa374042070 con 0x7fa37c15bff0 2024-04-25T00:05:25.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.150+0000 7fa363fff640 1 -- 172.21.15.113:0/753108740 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374009510 con 0x7fa37c15bff0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.150+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c13c700 con 0x7fa37c15bff0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.150+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c13ccc0 con 0x7fa37c15bff0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.151+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c0691a0 con 0x7fa37c15bff0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.151+0000 7fa363fff640 1 -- 172.21.15.113:0/753108740 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa374007bf0 con 0x7fa37c15bff0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.151+0000 7fa363fff640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36806be30 0x7fa36806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.152+0000 7fa37b7fe640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36806be30 0x7fa36806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.152+0000 7fa363fff640 1 -- 172.21.15.113:0/753108740 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5795+0+0 (secure 0 0 0) 0x7fa3740abb60 con 0x7fa37c15bff0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.152+0000 7fa37b7fe640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36806be30 0x7fa36806e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00ab30 tx=0x7fa36c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.156+0000 7fa363fff640 1 -- 172.21.15.113:0/753108740 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa374079e60 con 0x7fa37c15bff0 2024-04-25T00:05:25.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:25.366+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa37c0b4f40 con 0x7fa37c15bff0 2024-04-25T00:05:26.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.037+0000 7fa363fff640 1 -- 172.21.15.113:0/753108740 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) ==== 104+0+0 (secure 0 0 0) 0x7fa37407e160 con 0x7fa37c15bff0 2024-04-25T00:05:26.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36806be30 msgr2=0x7fa36806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:26.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36806be30 0x7fa36806e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00ab30 tx=0x7fa36c008040 comp rx=0 tx=0).stop 2024-04-25T00:05:26.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c15bff0 msgr2=0x7fa37c0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:26.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c15bff0 0x7fa37c0ce0d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa374034040 tx=0x7fa374004a40 comp rx=0 tx=0).stop 2024-04-25T00:05:26.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 shutdown_connections 2024-04-25T00:05:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa36806be30 0x7fa36806e2f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37c15bff0 0x7fa37c0ce0d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa37c15b670 0x7fa37c1ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 --2- 172.21.15.113:0/753108740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa37c0cc740 0x7fa37c1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.040+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 >> 172.21.15.113:0/753108740 conn(0x7fa37c06b0f0 msgr2=0x7fa37c0b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.041+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 shutdown_connections 2024-04-25T00:05:26.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.041+0000 7fa381d42640 1 -- 172.21.15.113:0/753108740 wait complete. 2024-04-25T00:05:26.054 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:05:26.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.196+0000 7f5336815640 1 Processor -- start 2024-04-25T00:05:26.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.196+0000 7f5336815640 1 -- start start 2024-04-25T00:05:26.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f5336815640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5330144a80 0x7f53300b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:26.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f5336815640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f53300b3ff0 0x7f533006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:26.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f5336815640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5330072f70 con 0x7f533006e610 2024-04-25T00:05:26.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f5336815640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f533006ea60 con 0x7f5330144a80 2024-04-25T00:05:26.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f5336815640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f53300b2fd0 con 0x7f53300b3ff0 2024-04-25T00:05:26.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f532ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5330144a80 0x7f53300b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:26.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f532ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5330144a80 0x7f53300b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54148/0 (socket says 172.21.15.113:54148) 2024-04-25T00:05:26.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f532ffff640 1 -- 172.21.15.113:0/2751737770 learned_addr learned my addr 172.21.15.113:0/2751737770 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:26.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f532ffff640 1 -- 172.21.15.113:0/2751737770 >> v2:172.21.15.178:3300/0 conn(0x7f53300b3ff0 msgr2=0x7f533006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:26.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f532ffff640 1 --2- 172.21.15.113:0/2751737770 >> v2:172.21.15.178:3300/0 conn(0x7f53300b3ff0 0x7f533006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f532ffff640 1 -- 172.21.15.113:0/2751737770 >> v1:172.21.15.178:6789/0 conn(0x7f533006e610 legacy=0x7f53300cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:26.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.197+0000 7f532ffff640 1 -- 172.21.15.113:0/2751737770 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53300b32b0 con 0x7f5330144a80 2024-04-25T00:05:26.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.198+0000 7f532ffff640 1 --2- 172.21.15.113:0/2751737770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f53300b3ab0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f533006c0d0 tx=0x7f532002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd976897e85f5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:26.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.198+0000 7f532effd640 1 -- 172.21.15.113:0/2751737770 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320037070 con 0x7f5330144a80 2024-04-25T00:05:26.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.198+0000 7f532effd640 1 -- 172.21.15.113:0/2751737770 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5320032440 con 0x7f5330144a80 2024-04-25T00:05:26.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.198+0000 7f532effd640 1 -- 172.21.15.113:0/2751737770 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320032810 con 0x7f5330144a80 2024-04-25T00:05:26.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 -- 172.21.15.113:0/2751737770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 msgr2=0x7f53300b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:26.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 --2- 172.21.15.113:0/2751737770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f53300b3ab0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f533006c0d0 tx=0x7f532002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 -- 172.21.15.113:0/2751737770 shutdown_connections 2024-04-25T00:05:26.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 --2- 172.21.15.113:0/2751737770 >> v2:172.21.15.178:3300/0 conn(0x7f53300b3ff0 0x7f533006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 --2- 172.21.15.113:0/2751737770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f53300b3ab0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 -- 172.21.15.113:0/2751737770 >> 172.21.15.113:0/2751737770 conn(0x7f53300b1b30 msgr2=0x7f53300b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:26.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 -- 172.21.15.113:0/2751737770 shutdown_connections 2024-04-25T00:05:26.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.199+0000 7f5336815640 1 -- 172.21.15.113:0/2751737770 wait complete. 2024-04-25T00:05:26.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.200+0000 7f5336815640 1 Processor -- start 2024-04-25T00:05:26.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.200+0000 7f5336815640 1 -- start start 2024-04-25T00:05:26.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f533006e610 0x7f533013a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:26.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b3ff0 0x7f5330141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:26.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f5330159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:26.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f53300cdec0 con 0x7f5330144a80 2024-04-25T00:05:26.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53300cdd40 con 0x7f533006e610 2024-04-25T00:05:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53300ce040 con 0x7f53300b3ff0 2024-04-25T00:05:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f5330159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f5330159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54164/0 (socket says 172.21.15.113:54164) 2024-04-25T00:05:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 -- 172.21.15.113:0/1687667883 learned_addr learned my addr 172.21.15.113:0/1687667883 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f532f7fe640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b3ff0 0x7f5330141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:26.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f532ffff640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f533006e610 0x7f533013a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 -- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b3ff0 msgr2=0x7f5330141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b3ff0 0x7f5330141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 -- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f533006e610 msgr2=0x7f533013a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f533006e610 0x7f533013a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 -- 172.21.15.113:0/1687667883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53301428d0 con 0x7f5330144a80 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f532f7fe640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b3ff0 0x7f5330141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f532ffff640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f533006e610 0x7f533013a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:26.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5334db2640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f5330159240 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f532800c780 tx=0x7f532800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f532cff9640 1 -- 172.21.15.113:0/1687667883 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328017070 con 0x7f5330144a80 2024-04-25T00:05:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f532cff9640 1 -- 172.21.15.113:0/1687667883 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5328003da0 con 0x7f5330144a80 2024-04-25T00:05:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5330142bb0 con 0x7f5330144a80 2024-04-25T00:05:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f532cff9640 1 -- 172.21.15.113:0/1687667883 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53280052b0 con 0x7f5330144a80 2024-04-25T00:05:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.201+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5330143170 con 0x7f5330144a80 2024-04-25T00:05:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.202+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53300691a0 con 0x7f5330144a80 2024-04-25T00:05:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.203+0000 7f532cff9640 1 -- 172.21.15.113:0/1687667883 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f532801a070 con 0x7f5330144a80 2024-04-25T00:05:26.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.203+0000 7f532cff9640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f531c06be60 0x7f531c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:26.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.203+0000 7f532ffff640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f531c06be60 0x7f531c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:26.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.203+0000 7f532cff9640 1 -- 172.21.15.113:0/1687667883 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5795+0+0 (secure 0 0 0) 0x7f532808a430 con 0x7f5330144a80 2024-04-25T00:05:26.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.204+0000 7f532ffff640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f531c06be60 0x7f531c06e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5320034000 tx=0x7f532002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:26.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.207+0000 7f532cff9640 1 -- 172.21.15.113:0/1687667883 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5328058e10 con 0x7f5330144a80 2024-04-25T00:05:26.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:26.380+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f53300ced70 con 0x7f5330144a80 2024-04-25T00:05:27.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.041+0000 7f532cff9640 1 -- 172.21.15.113:0/1687667883 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) ==== 181+0+0 (secure 0 0 0) 0x7f5328058710 con 0x7f5330144a80 2024-04-25T00:05:27.041 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:05:27.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.043+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f531c06be60 msgr2=0x7f531c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:27.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.043+0000 7f5336815640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f531c06be60 0x7f531c06e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5320034000 tx=0x7f532002e000 comp rx=0 tx=0).stop 2024-04-25T00:05:27.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.043+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 msgr2=0x7f5330159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:27.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.043+0000 7f5336815640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f5330159240 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f532800c780 tx=0x7f532800cc40 comp rx=0 tx=0).stop 2024-04-25T00:05:27.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.045+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 shutdown_connections 2024-04-25T00:05:27.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.045+0000 7f5336815640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f531c06be60 0x7f531c06e320 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.045+0000 7f5336815640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5330144a80 0x7f5330159240 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.045+0000 7f5336815640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b3ff0 0x7f5330141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.045+0000 7f5336815640 1 --2- 172.21.15.113:0/1687667883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f533006e610 0x7f533013a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.045+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 >> 172.21.15.113:0/1687667883 conn(0x7f53300b1b30 msgr2=0x7f53300cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:27.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.046+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 shutdown_connections 2024-04-25T00:05:27.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.046+0000 7f5336815640 1 -- 172.21.15.113:0/1687667883 wait complete. 2024-04-25T00:05:27.062 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:05:27.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.165+0000 7f5209f70640 1 Processor -- start 2024-04-25T00:05:27.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.165+0000 7f5209f70640 1 -- start start 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5209f70640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52040ceef0 0x7f520406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5209f70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f5204071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5209f70640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52040cf3a0 con 0x7f5204145890 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5209f70640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5204145c70 con 0x7f52040ceef0 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5209f70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f520406ea60 con 0x7f520406e610 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f51fbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52040ceef0 0x7f520406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5203fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f5204071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:27.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5203fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f5204071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:56994/0 (socket says 172.21.15.113:56994) 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f51fbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52040ceef0 0x7f520406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:58712/0 (socket says 172.21.15.113:58712) 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f52037fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5204145890 0x7f52040ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:45942/0 (socket says 172.21.15.113:45942) 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5203fff640 1 -- 172.21.15.113:0/2139089889 learned_addr learned my addr 172.21.15.113:0/2139089889 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f51fbfff640 1 -- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 msgr2=0x7f5204071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f51fbfff640 1 --2- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f5204071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f51fbfff640 1 -- 172.21.15.113:0/2139089889 >> v1:172.21.15.178:6789/0 conn(0x7f5204145890 legacy=0x7f52040ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f51fbfff640 1 -- 172.21.15.113:0/2139089889 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52040cf680 con 0x7f52040ceef0 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.166+0000 7f5202ffd640 1 -- 172.21.15.113:0/2139089889 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787349015 0 0) 0x7f52040cf3a0 con 0x7f5204145890 2024-04-25T00:05:27.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.167+0000 7f5203fff640 1 --2- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f5204071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:27.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.167+0000 7f51fbfff640 1 --2- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 0x7f520406e0d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0097d0 tx=0x7f51ec02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f9e02305ecccbc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.167+0000 7f5202ffd640 1 -- 172.21.15.113:0/2139089889 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec037070 con 0x7f52040ceef0 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.167+0000 7f5202ffd640 1 -- 172.21.15.113:0/2139089889 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ec032440 con 0x7f52040ceef0 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.167+0000 7f5202ffd640 1 -- 172.21.15.113:0/2139089889 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec032810 con 0x7f52040ceef0 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 -- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 msgr2=0x7f520406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 --2- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 0x7f520406e0d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0097d0 tx=0x7f51ec02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 -- 172.21.15.113:0/2139089889 shutdown_connections 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 --2- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f5204071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 --2- 172.21.15.113:0/2139089889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 0x7f520406e0d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 -- 172.21.15.113:0/2139089889 >> 172.21.15.113:0/2139089889 conn(0x7f52040b1b30 msgr2=0x7f52040b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 -- 172.21.15.113:0/2139089889 shutdown_connections 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.168+0000 7f5209f70640 1 -- 172.21.15.113:0/2139089889 wait complete. 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.169+0000 7f5209f70640 1 Processor -- start 2024-04-25T00:05:27.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.169+0000 7f5209f70640 1 -- start start 2024-04-25T00:05:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5209f70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f520415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5209f70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 0x7f520415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5209f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5204145890 0x7f52040afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5209f70640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52040b53b0 con 0x7f5204145890 2024-04-25T00:05:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5209f70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5204070f10 con 0x7f52040ceef0 2024-04-25T00:05:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5209f70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5204071970 con 0x7f520406e610 2024-04-25T00:05:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5204145890 0x7f52040afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5204145890 0x7f52040afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54178/0 (socket says 172.21.15.113:54178) 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 -- 172.21.15.113:0/3157497439 learned_addr learned my addr 172.21.15.113:0/3157497439 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f51fbfff640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 0x7f520415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f52037fe640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f520415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 -- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 msgr2=0x7f520415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f520415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 -- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 msgr2=0x7f520415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 0x7f520415eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f5203fff640 1 -- 172.21.15.113:0/3157497439 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520415de60 con 0x7f5204145890 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.170+0000 7f52037fe640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f520415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.171+0000 7f5203fff640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5204145890 0x7f52040afd00 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00efc0 tx=0x7f51fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.171+0000 7f5200ff9640 1 -- 172.21.15.113:0/3157497439 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00cbe0 con 0x7f5204145890 2024-04-25T00:05:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.171+0000 7f5200ff9640 1 -- 172.21.15.113:0/3157497439 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc00cd80 con 0x7f5204145890 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.171+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520415e140 con 0x7f5204145890 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.171+0000 7f5200ff9640 1 -- 172.21.15.113:0/3157497439 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc003bb0 con 0x7f5204145890 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.171+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520415e700 con 0x7f5204145890 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.172+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52040691a0 con 0x7f5204145890 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.172+0000 7f5200ff9640 1 -- 172.21.15.113:0/3157497439 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51fc01b030 con 0x7f5204145890 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.173+0000 7f5200ff9640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51e406be30 0x7f51e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.173+0000 7f52037fe640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51e406be30 0x7f51e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.173+0000 7f5200ff9640 1 -- 172.21.15.113:0/3157497439 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5357+0+0 (secure 0 0 0) 0x7f51fc08b0d0 con 0x7f5204145890 2024-04-25T00:05:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.173+0000 7f52037fe640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51e406be30 0x7f51e406e2f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000f60 tx=0x7f51f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:27.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.177+0000 7f5200ff9640 1 -- 172.21.15.113:0/3157497439 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f51fc022020 con 0x7f5204145890 2024-04-25T00:05:27.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:27.351+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5204140fb0 con 0x7f5204145890 2024-04-25T00:05:28.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.047+0000 7f5200ff9640 1 -- 172.21.15.113:0/3157497439 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) ==== 169+0+0 (secure 0 0 0) 0x7f51fc010040 con 0x7f5204145890 2024-04-25T00:05:28.048 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51e406be30 msgr2=0x7f51e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51e406be30 0x7f51e406e2f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000f60 tx=0x7f51f4009040 comp rx=0 tx=0).stop 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5204145890 msgr2=0x7f52040afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5204145890 0x7f52040afd00 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00efc0 tx=0x7f51fc00c490 comp rx=0 tx=0).stop 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 shutdown_connections 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f51e406be30 0x7f51e406e2f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040ceef0 0x7f520415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5204145890 0x7f52040afd00 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 --2- 172.21.15.113:0/3157497439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f520406e610 0x7f520415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.050+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 >> 172.21.15.113:0/3157497439 conn(0x7f52040b1b30 msgr2=0x7f5204071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:28.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.051+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 shutdown_connections 2024-04-25T00:05:28.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.051+0000 7f5209f70640 1 -- 172.21.15.113:0/3157497439 wait complete. 2024-04-25T00:05:28.072 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:05:28.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.195+0000 7f3410f09640 1 Processor -- start 2024-04-25T00:05:28.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.195+0000 7f3410f09640 1 -- start start 2024-04-25T00:05:28.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f3410f09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f340c15e510 0x7f340c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f3410f09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c0d28c0 0x7f340c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f3410f09640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f340c0d2e00 con 0x7f340c0b61e0 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f3410f09640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f340c0d30e0 con 0x7f340c15e510 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f3410f09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f340c0d33c0 con 0x7f340c0d28c0 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f340a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f340c15e510 0x7f340c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f340a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f340c15e510 0x7f340c0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45418/0 (socket says 172.21.15.113:45418) 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f340a59c640 1 -- 172.21.15.113:0/3201749930 learned_addr learned my addr 172.21.15.113:0/3201749930 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f3409d9b640 1 --2- 172.21.15.113:0/3201749930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c0d28c0 0x7f340c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f340a59c640 1 -- 172.21.15.113:0/3201749930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c0d28c0 msgr2=0x7f340c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:28.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f340a59c640 1 --2- 172.21.15.113:0/3201749930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c0d28c0 0x7f340c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f340a59c640 1 -- 172.21.15.113:0/3201749930 >> v1:172.21.15.178:6789/0 conn(0x7f340c0b61e0 legacy=0x7f340c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.196+0000 7f340a59c640 1 -- 172.21.15.113:0/3201749930 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c0d36a0 con 0x7f340c15e510 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.197+0000 7f340a59c640 1 --2- 172.21.15.113:0/3201749930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c15e510 0x7f340c0d2380 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009870 tx=0x7f33fc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd0c7024032ee21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.197+0000 7f340959a640 1 -- 172.21.15.113:0/3201749930 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc037070 con 0x7f340c15e510 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.197+0000 7f340959a640 1 -- 172.21.15.113:0/3201749930 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33fc032440 con 0x7f340c15e510 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.197+0000 7f340959a640 1 -- 172.21.15.113:0/3201749930 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc032810 con 0x7f340c15e510 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.197+0000 7f3410f09640 1 -- 172.21.15.113:0/3201749930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c15e510 msgr2=0x7f340c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.198+0000 7f3410f09640 1 --2- 172.21.15.113:0/3201749930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c15e510 0x7f340c0d2380 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009870 tx=0x7f33fc02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.198+0000 7f3410f09640 1 -- 172.21.15.113:0/3201749930 shutdown_connections 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.198+0000 7f3410f09640 1 --2- 172.21.15.113:0/3201749930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c0d28c0 0x7f340c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.198+0000 7f3410f09640 1 --2- 172.21.15.113:0/3201749930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c15e510 0x7f340c0d2380 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.198+0000 7f3410f09640 1 -- 172.21.15.113:0/3201749930 >> 172.21.15.113:0/3201749930 conn(0x7f340c06b720 msgr2=0x7f340c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.198+0000 7f3410f09640 1 -- 172.21.15.113:0/3201749930 shutdown_connections 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.198+0000 7f3410f09640 1 -- 172.21.15.113:0/3201749930 wait complete. 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.199+0000 7f3410f09640 1 Processor -- start 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.199+0000 7f3410f09640 1 -- start start 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c0b61e0 0x7f340c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f340c0d28c0 0x7f340c1f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c15e510 0x7f340c1f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0b9dd0 con 0x7f340c0b61e0 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0d3ea0 con 0x7f340c0d28c0 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f340c0b83d0 con 0x7f340c15e510 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c0b61e0 0x7f340c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c0b61e0 0x7f340c153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45434/0 (socket says 172.21.15.113:45434) 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 -- 172.21.15.113:0/239434224 learned_addr learned my addr 172.21.15.113:0/239434224 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3409d9b640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f340c0d28c0 0x7f340c1f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340ad9d640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c15e510 0x7f340c1f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 -- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c15e510 msgr2=0x7f340c1f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c15e510 0x7f340c1f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 -- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f340c0d28c0 msgr2=0x7f340c1f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f340c0d28c0 0x7f340c1f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 -- 172.21.15.113:0/239434224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c148070 con 0x7f340c0b61e0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f340a59c640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c0b61e0 0x7f340c153be0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f33fc02a200 tx=0x7f33fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f33faffd640 1 -- 172.21.15.113:0/239434224 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc037040 con 0x7f340c0b61e0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f33faffd640 1 -- 172.21.15.113:0/239434224 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33fc042070 con 0x7f340c0b61e0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f33faffd640 1 -- 172.21.15.113:0/239434224 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc030880 con 0x7f340c0b61e0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c151250 con 0x7f340c0b61e0 2024-04-25T00:05:28.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.200+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c0ccc90 con 0x7f340c0b61e0 2024-04-25T00:05:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.201+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c0691a0 con 0x7f340c0b61e0 2024-04-25T00:05:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.202+0000 7f33faffd640 1 -- 172.21.15.113:0/239434224 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33fc02ad10 con 0x7f340c0b61e0 2024-04-25T00:05:28.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.202+0000 7f33faffd640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33ec06be30 0x7f33ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.202+0000 7f3409d9b640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33ec06be30 0x7f33ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.202+0000 7f33faffd640 1 -- 172.21.15.113:0/239434224 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 4943+0+0 (secure 0 0 0) 0x7f33fc0aaf30 con 0x7f340c0b61e0 2024-04-25T00:05:28.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.203+0000 7f3409d9b640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33ec06be30 0x7f33ec06e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f340c1483c0 tx=0x7f3400009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:28.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.206+0000 7f33faffd640 1 -- 172.21.15.113:0/239434224 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33fc079620 con 0x7f340c0b61e0 2024-04-25T00:05:28.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.381+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f340c0d04c0 con 0x7f340c0b61e0 2024-04-25T00:05:28.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.384+0000 7f33faffd640 1 -- 172.21.15.113:0/239434224 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f33fc07d920 con 0x7f340c0b61e0 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33ec06be30 msgr2=0x7f33ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33ec06be30 0x7f33ec06e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f340c1483c0 tx=0x7f3400009190 comp rx=0 tx=0).stop 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c0b61e0 msgr2=0x7f340c153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c0b61e0 0x7f340c153be0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f33fc02a200 tx=0x7f33fc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 shutdown_connections 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f33ec06be30 0x7f33ec06e2f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f340c0d28c0 0x7f340c1f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f340c15e510 0x7f340c1f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 --2- 172.21.15.113:0/239434224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f340c0b61e0 0x7f340c153be0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 >> 172.21.15.113:0/239434224 conn(0x7f340c06b720 msgr2=0x7f340c0d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:28.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 shutdown_connections 2024-04-25T00:05:28.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.386+0000 7f3410f09640 1 -- 172.21.15.113:0/239434224 wait complete. 2024-04-25T00:05:28.398 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T00:05:28.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.494+0000 7f63b9cbd640 1 Processor -- start 2024-04-25T00:05:28.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.494+0000 7f63b9cbd640 1 -- start start 2024-04-25T00:05:28.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b9cbd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63b4144910 0x7f63b40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b9cbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b9cbd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b4144cf0 con 0x7f63b40b5240 2024-04-25T00:05:28.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b9cbd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f63b40b4510 con 0x7f63b4144910 2024-04-25T00:05:28.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b9cbd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f63b40b2f30 con 0x7f63b40b56b0 2024-04-25T00:05:28.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63b4144910 0x7f63b40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b2ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f63b40b5240 0x7f63b406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45846/0 (socket says 172.21.15.113:45846) 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b2ffd640 1 -- 172.21.15.113:0/2624920824 learned_addr learned my addr 172.21.15.113:0/2624920824 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b37fe640 1 -- 172.21.15.113:0/2624920824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 msgr2=0x7f63b40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b3fff640 1 --2- 172.21.15.113:0/2624920824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b37fe640 1 --2- 172.21.15.113:0/2624920824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b40cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b27fc640 1 -- 172.21.15.113:0/2624920824 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4053244550 0 0) 0x7f63b4144cf0 con 0x7f63b40b5240 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b37fe640 1 -- 172.21.15.113:0/2624920824 >> v1:172.21.15.113:6789/0 conn(0x7f63b40b5240 legacy=0x7f63b406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b37fe640 1 -- 172.21.15.113:0/2624920824 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b40b3210 con 0x7f63b4144910 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.495+0000 7f63b3fff640 1 --2- 172.21.15.113:0/2624920824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b40cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.496+0000 7f63b37fe640 1 --2- 172.21.15.113:0/2624920824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b40b3fd0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097d0 tx=0x7f63a402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a33e477e1049b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.496+0000 7f63b27fc640 1 -- 172.21.15.113:0/2624920824 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4037070 con 0x7f63b4144910 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.496+0000 7f63b27fc640 1 -- 172.21.15.113:0/2624920824 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a4032440 con 0x7f63b4144910 2024-04-25T00:05:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.496+0000 7f63b27fc640 1 -- 172.21.15.113:0/2624920824 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4032810 con 0x7f63b4144910 2024-04-25T00:05:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.496+0000 7f63b9cbd640 1 -- 172.21.15.113:0/2624920824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 msgr2=0x7f63b40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.496+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/2624920824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b40b3fd0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097d0 tx=0x7f63a402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.497+0000 7f63b9cbd640 1 -- 172.21.15.113:0/2624920824 shutdown_connections 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.497+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/2624920824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b40cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.497+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/2624920824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b40b3fd0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.497+0000 7f63b9cbd640 1 -- 172.21.15.113:0/2624920824 >> 172.21.15.113:0/2624920824 conn(0x7f63b40b1a50 msgr2=0x7f63b40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.497+0000 7f63b9cbd640 1 -- 172.21.15.113:0/2624920824 shutdown_connections 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.497+0000 7f63b9cbd640 1 -- 172.21.15.113:0/2624920824 wait complete. 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.498+0000 7f63b9cbd640 1 Processor -- start 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.498+0000 7f63b9cbd640 1 -- start start 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.498+0000 7f63b9cbd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63b40b5240 0x7f63b41f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b9cbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b41406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b9cbd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b4150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b9cbd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40b4da0 con 0x7f63b4144910 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b9cbd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40b37d0 con 0x7f63b40b5240 2024-04-25T00:05:28.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b9cbd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f63b40b4f20 con 0x7f63b40b56b0 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b4150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b4150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45452/0 (socket says 172.21.15.113:45452) 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 -- 172.21.15.113:0/3528989011 learned_addr learned my addr 172.21.15.113:0/3528989011 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b2ffd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b41406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b37fe640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63b40b5240 0x7f63b41f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 -- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 msgr2=0x7f63b41406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b41406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 -- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63b40b5240 msgr2=0x7f63b41f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63b40b5240 0x7f63b41f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 -- 172.21.15.113:0/3528989011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b41f4b40 con 0x7f63b4144910 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b2ffd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b41406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b37fe640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63b40b5240 0x7f63b41f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b3fff640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b4150600 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00cdd0 tx=0x7f63ac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f639bfff640 1 -- 172.21.15.113:0/3528989011 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac01a070 con 0x7f63b4144910 2024-04-25T00:05:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f639bfff640 1 -- 172.21.15.113:0/3528989011 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac00d840 con 0x7f63b4144910 2024-04-25T00:05:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b41f7760 con 0x7f63b4144910 2024-04-25T00:05:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f639bfff640 1 -- 172.21.15.113:0/3528989011 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00dc70 con 0x7f63b4144910 2024-04-25T00:05:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.499+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b41f7d20 con 0x7f63b4144910 2024-04-25T00:05:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.500+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b40691a0 con 0x7f63b4144910 2024-04-25T00:05:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.501+0000 7f639bfff640 1 -- 172.21.15.113:0/3528989011 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63ac01d070 con 0x7f63b4144910 2024-04-25T00:05:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.501+0000 7f639bfff640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63a006be30 0x7f63a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.501+0000 7f639bfff640 1 -- 172.21.15.113:0/3528989011 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 4943+0+0 (secure 0 0 0) 0x7f63ac08b310 con 0x7f63b4144910 2024-04-25T00:05:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.501+0000 7f63b37fe640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63a006be30 0x7f63a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.501+0000 7f63b37fe640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63a006be30 0x7f63a006e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f63a4034000 tx=0x7f63a402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.505+0000 7f639bfff640 1 -- 172.21.15.113:0/3528989011 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f63ac059fc0 con 0x7f63b4144910 2024-04-25T00:05:28.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:28.746+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) -- 0x7f63b40cf920 con 0x7f63b4144910 2024-04-25T00:05:29.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.049+0000 7f639bfff640 1 -- 172.21.15.113:0/3528989011 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v353) ==== 155+0+0 (secure 0 0 0) 0x7f63ac0598c0 con 0x7f63b4144910 2024-04-25T00:05:29.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63a006be30 msgr2=0x7f63a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63a006be30 0x7f63a006e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f63a4034000 tx=0x7f63a402e000 comp rx=0 tx=0).stop 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 msgr2=0x7f63b4150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b4150600 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00cdd0 tx=0x7f63ac00a710 comp rx=0 tx=0).stop 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 shutdown_connections 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63a006be30 0x7f63a006e2f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63b4144910 0x7f63b4150600 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63b40b56b0 0x7f63b41406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 --2- 172.21.15.113:0/3528989011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63b40b5240 0x7f63b41f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 >> 172.21.15.113:0/3528989011 conn(0x7f63b40b1a50 msgr2=0x7f63b40b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 shutdown_connections 2024-04-25T00:05:29.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.051+0000 7f63b9cbd640 1 -- 172.21.15.113:0/3528989011 wait complete. 2024-04-25T00:05:29.063 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T00:05:29.063 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T00:05:29.064 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.160+0000 7f24d0038640 1 Processor -- start 2024-04-25T00:05:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.160+0000 7f24d0038640 1 -- start start 2024-04-25T00:05:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24d0038640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c80b3fd0 0x7f24c806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24d0038640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24c806e510 0x7f24c80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24d0038640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c8144cf0 con 0x7f24c8144910 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24d0038640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f24c80b2f30 con 0x7f24c806e510 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24d0038640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80b3210 con 0x7f24c80b3fd0 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24ce5d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24c806e510 0x7f24c80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24cddd4640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f24c8144910 0x7f24c80b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45852/0 (socket says 172.21.15.113:45852) 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24cddd4640 1 -- 172.21.15.113:0/3246477519 learned_addr learned my addr 172.21.15.113:0/3246477519 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24cd5d3640 1 --2- 172.21.15.113:0/3246477519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c80b3fd0 0x7f24c806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24ce5d5640 1 -- 172.21.15.113:0/3246477519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c80b3fd0 msgr2=0x7f24c806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24ce5d5640 1 --2- 172.21.15.113:0/3246477519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c80b3fd0 0x7f24c806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24ce5d5640 1 -- 172.21.15.113:0/3246477519 >> v1:172.21.15.113:6789/0 conn(0x7f24c8144910 legacy=0x7f24c80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24ce5d5640 1 -- 172.21.15.113:0/3246477519 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80b34f0 con 0x7f24c806e510 2024-04-25T00:05:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.161+0000 7f24ccdd2640 1 -- 172.21.15.113:0/3246477519 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20760300 0 0) 0x7f24c8144cf0 con 0x7f24c8144910 2024-04-25T00:05:29.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24ce5d5640 1 --2- 172.21.15.113:0/3246477519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80cd500 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009d90 tx=0x7f24c002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a233589329449b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24ccdd2640 1 -- 172.21.15.113:0/3246477519 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0007740 con 0x7f24c806e510 2024-04-25T00:05:29.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24ccdd2640 1 -- 172.21.15.113:0/3246477519 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c00078e0 con 0x7f24c806e510 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24ccdd2640 1 -- 172.21.15.113:0/3246477519 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0007cb0 con 0x7f24c806e510 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24d0038640 1 -- 172.21.15.113:0/3246477519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 msgr2=0x7f24c80cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24d0038640 1 --2- 172.21.15.113:0/3246477519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80cd500 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009d90 tx=0x7f24c002a660 comp rx=0 tx=0).stop 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24d0038640 1 -- 172.21.15.113:0/3246477519 shutdown_connections 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24d0038640 1 --2- 172.21.15.113:0/3246477519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80cd500 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24d0038640 1 --2- 172.21.15.113:0/3246477519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c80b3fd0 0x7f24c806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.162+0000 7f24d0038640 1 -- 172.21.15.113:0/3246477519 >> 172.21.15.113:0/3246477519 conn(0x7f24c80b1a50 msgr2=0x7f24c80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.163+0000 7f24d0038640 1 -- 172.21.15.113:0/3246477519 shutdown_connections 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.163+0000 7f24d0038640 1 -- 172.21.15.113:0/3246477519 wait complete. 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 Processor -- start 2024-04-25T00:05:29.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 -- start start 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c80b3fd0 0x7f24c81323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c8144910 0x7f24c80b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80b4da0 con 0x7f24c806e510 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80b4410 con 0x7f24c80b3fd0 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24d0038640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80b4f20 con 0x7f24c8144910 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24cddd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24cddd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80a9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45472/0 (socket says 172.21.15.113:45472) 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.164+0000 7f24cddd4640 1 -- 172.21.15.113:0/3264264055 learned_addr learned my addr 172.21.15.113:0/3264264055 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24cddd4640 1 -- 172.21.15.113:0/3264264055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c8144910 msgr2=0x7f24c80b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24cddd4640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c8144910 0x7f24c80b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24cddd4640 1 -- 172.21.15.113:0/3264264055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c80b3fd0 msgr2=0x7f24c81323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24cddd4640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c80b3fd0 0x7f24c81323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24cddd4640 1 -- 172.21.15.113:0/3264264055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c8132e80 con 0x7f24c806e510 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24cddd4640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80a9bd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c430 tx=0x7f24c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24be7fc640 1 -- 172.21.15.113:0/3264264055 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c40074e0 con 0x7f24c806e510 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24be7fc640 1 -- 172.21.15.113:0/3264264055 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c40047e0 con 0x7f24c806e510 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c8133160 con 0x7f24c806e510 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24be7fc640 1 -- 172.21.15.113:0/3264264055 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4004bd0 con 0x7f24c806e510 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.165+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c813f6a0 con 0x7f24c806e510 2024-04-25T00:05:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.166+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c8130350 con 0x7f24c806e510 2024-04-25T00:05:29.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.166+0000 7f24be7fc640 1 -- 172.21.15.113:0/3264264055 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24c4011070 con 0x7f24c806e510 2024-04-25T00:05:29.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.167+0000 7f24be7fc640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24ac06be60 0x7f24ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.167+0000 7f24be7fc640 1 -- 172.21.15.113:0/3264264055 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 4943+0+0 (secure 0 0 0) 0x7f24c408a700 con 0x7f24c806e510 2024-04-25T00:05:29.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.167+0000 7f24cd5d3640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24ac06be60 0x7f24ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.167+0000 7f24cd5d3640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24ac06be60 0x7f24ac06e320 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f24c8130220 tx=0x7f24b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.171+0000 7f24be7fc640 1 -- 172.21.15.113:0/3264264055 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24c400e280 con 0x7f24c806e510 2024-04-25T00:05:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.346+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24c80691a0 con 0x7f24c806e510 2024-04-25T00:05:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.347+0000 7f24be7fc640 1 -- 172.21.15.113:0/3264264055 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+13078 (secure 0 0 0) 0x7f24c4059440 con 0x7f24c806e510 2024-04-25T00:05:29.347 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:29.347 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":107,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:05:28.042295+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6836/677893386":"2024-04-26T00:05:25.026846+0000","172.21.15.113:6834/677893386":"2024-04-26T00:05:25.026846+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:29.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24ac06be60 msgr2=0x7f24ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24ac06be60 0x7f24ac06e320 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f24c8130220 tx=0x7f24b8008040 comp rx=0 tx=0).stop 2024-04-25T00:05:29.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 msgr2=0x7f24c80a9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80a9bd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c430 tx=0x7f24c400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 shutdown_connections 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f24ac06be60 0x7f24ac06e320 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c80b3fd0 0x7f24c81323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c8144910 0x7f24c80b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 --2- 172.21.15.113:0/3264264055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806e510 0x7f24c80a9bd0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 >> 172.21.15.113:0/3264264055 conn(0x7f24c80b1a50 msgr2=0x7f24c80b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 shutdown_connections 2024-04-25T00:05:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.349+0000 7f24d0038640 1 -- 172.21.15.113:0/3264264055 wait complete. 2024-04-25T00:05:29.361 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:05:29.361 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:05:29.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.460+0000 7fa97b7ee640 1 Processor -- start 2024-04-25T00:05:29.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.460+0000 7fa97b7ee640 1 -- start start 2024-04-25T00:05:29.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa97b7ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9740b4a40 0x7fa974165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa97b7ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97415f4f0 con 0x7fa9740b3080 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa97b7ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97415f7d0 con 0x7fa97415e2d0 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa97b7ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa9740cde00 con 0x7fa9740b4a40 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa979d8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9740b4a40 0x7fa974165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa979d8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9740b4a40 0x7fa974165f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45478/0 (socket says 172.21.15.113:45478) 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa979d8b640 1 -- 172.21.15.113:0/3469107030 learned_addr learned my addr 172.21.15.113:0/3469107030 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa978d89640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa9740b3080 0x7fa9740b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45866/0 (socket says 172.21.15.113:45866) 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa979d8b640 1 -- 172.21.15.113:0/3469107030 >> v1:172.21.15.178:6789/0 conn(0x7fa97415e2d0 legacy=0x7fa97415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa96bfff640 1 -- 172.21.15.113:0/3469107030 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 916236443 0 0) 0x7fa97415f4f0 con 0x7fa9740b3080 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa979d8b640 1 -- 172.21.15.113:0/3469107030 >> v1:172.21.15.113:6789/0 conn(0x7fa9740b3080 legacy=0x7fa9740b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.461+0000 7fa979d8b640 1 -- 172.21.15.113:0/3469107030 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97415fab0 con 0x7fa9740b4a40 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.462+0000 7fa979d8b640 1 --2- 172.21.15.113:0/3469107030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa974165f80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa9740cdf80 tx=0x7fa96c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf20adbef8615e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.462+0000 7fa96bfff640 1 -- 172.21.15.113:0/3469107030 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c036070 con 0x7fa9740b4a40 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.462+0000 7fa96bfff640 1 -- 172.21.15.113:0/3469107030 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c031600 con 0x7fa9740b4a40 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.462+0000 7fa96bfff640 1 -- 172.21.15.113:0/3469107030 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0319d0 con 0x7fa9740b4a40 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.463+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3469107030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 msgr2=0x7fa974165f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.463+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3469107030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa974165f80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa9740cdf80 tx=0x7fa96c02a7a0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.463+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3469107030 shutdown_connections 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.463+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3469107030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa974165f80 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.463+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3469107030 >> 172.21.15.113:0/3469107030 conn(0x7fa97406b0f0 msgr2=0x7fa97406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:29.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.463+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3469107030 shutdown_connections 2024-04-25T00:05:29.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.463+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3469107030 wait complete. 2024-04-25T00:05:29.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.464+0000 7fa97b7ee640 1 Processor -- start 2024-04-25T00:05:29.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.464+0000 7fa97b7ee640 1 -- start start 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97b7ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9740b3080 0x7fa974153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97b7ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa9741f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97b7ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa97415e2d0 0x7fa9741f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97b7ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9740cf420 con 0x7fa9740b4a40 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97b7ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9740cf2a0 con 0x7fa9740b3080 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97b7ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9740cf5a0 con 0x7fa97415e2d0 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa9741f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa9741f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45482/0 (socket says 172.21.15.113:45482) 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 -- 172.21.15.113:0/3861443972 learned_addr learned my addr 172.21.15.113:0/3861443972 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97958a640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9740b3080 0x7fa974153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa979d8b640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa97415e2d0 0x7fa9741f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 -- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa97415e2d0 msgr2=0x7fa9741f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa97415e2d0 0x7fa9741f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 -- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9740b3080 msgr2=0x7fa974153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9740b3080 0x7fa974153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 -- 172.21.15.113:0/3861443972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9741f2e50 con 0x7fa9740b4a40 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa978d89640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa9741f6420 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c4a0 tx=0x7fa96400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa969ffb640 1 -- 172.21.15.113:0/3861443972 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964018070 con 0x7fa9740b4a40 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa969ffb640 1 -- 172.21.15.113:0/3861443972 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9640074e0 con 0x7fa9740b4a40 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9741f3130 con 0x7fa9740b4a40 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.465+0000 7fa969ffb640 1 -- 172.21.15.113:0/3861443972 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964004a20 con 0x7fa9740b4a40 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.466+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741528f0 con 0x7fa9740b4a40 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.467+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9740691a0 con 0x7fa9740b4a40 2024-04-25T00:05:29.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.467+0000 7fa969ffb640 1 -- 172.21.15.113:0/3861443972 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa96401b070 con 0x7fa9740b4a40 2024-04-25T00:05:29.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.467+0000 7fa969ffb640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa95806be30 0x7fa95806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.467+0000 7fa969ffb640 1 -- 172.21.15.113:0/3861443972 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 4943+0+0 (secure 0 0 0) 0x7fa964089db0 con 0x7fa9740b4a40 2024-04-25T00:05:29.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.467+0000 7fa97958a640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa95806be30 0x7fa95806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.468+0000 7fa97958a640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa95806be30 0x7fa95806e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa970000f60 tx=0x7fa970009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.471+0000 7fa969ffb640 1 -- 172.21.15.113:0/3861443972 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa964058410 con 0x7fa9740b4a40 2024-04-25T00:05:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.647+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa974068570 con 0x7fa9740b4a40 2024-04-25T00:05:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.647+0000 7fa969ffb640 1 -- 172.21.15.113:0/3861443972 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7fa96405c710 con 0x7fa9740b4a40 2024-04-25T00:05:29.648 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:05:29.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa95806be30 msgr2=0x7fa95806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa95806be30 0x7fa95806e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa970000f60 tx=0x7fa970009040 comp rx=0 tx=0).stop 2024-04-25T00:05:29.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 msgr2=0x7fa9741f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa9741f6420 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c4a0 tx=0x7fa96400c960 comp rx=0 tx=0).stop 2024-04-25T00:05:29.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 shutdown_connections 2024-04-25T00:05:29.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa95806be30 0x7fa95806e2f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa97415e2d0 0x7fa9741f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9740b4a40 0x7fa9741f6420 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 --2- 172.21.15.113:0/3861443972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9740b3080 0x7fa974153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 >> 172.21.15.113:0/3861443972 conn(0x7fa97406b0f0 msgr2=0x7fa9740b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:29.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 shutdown_connections 2024-04-25T00:05:29.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.649+0000 7fa97b7ee640 1 -- 172.21.15.113:0/3861443972 wait complete. 2024-04-25T00:05:29.661 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T00:05:29.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.757+0000 7f80c5ae1640 1 Processor -- start 2024-04-25T00:05:29.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.758+0000 7f80c5ae1640 1 -- start start 2024-04-25T00:05:29.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.758+0000 7f80c5ae1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f80c00d0b00 0x7f80c00d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.758+0000 7f80c5ae1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c00d1480 con 0x7f80c00d2290 2024-04-25T00:05:29.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.758+0000 7f80c5ae1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c00d2800 con 0x7f80c015e320 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.758+0000 7f80c5ae1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f80c00d3410 con 0x7f80c00d0b00 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.758+0000 7f80b6ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f80c00d2290 0x7f80c00b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45882/0 (socket says 172.21.15.113:45882) 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.758+0000 7f80b6ffd640 1 -- 172.21.15.113:0/3488287501 learned_addr learned my addr 172.21.15.113:0/3488287501 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337102578 0 0) 0x7f80c00d1480 con 0x7f80c00d2290 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80ac003520 con 0x7f80c00d2290 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80bffff640 1 --2- 172.21.15.113:0/3488287501 >> v2:172.21.15.178:3300/0 conn(0x7f80c00d0b00 0x7f80c00d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3711625782 0 0) 0x7f80ac003520 con 0x7f80c00d2290 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 >> v2:172.21.15.178:3300/0 conn(0x7f80c00d0b00 msgr2=0x7f80c00d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 --2- 172.21.15.113:0/3488287501 >> v2:172.21.15.178:3300/0 conn(0x7f80c00d0b00 0x7f80c00d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 >> v1:172.21.15.178:6789/0 conn(0x7f80c015e320 legacy=0x7f80c00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c00d2ae0 con 0x7f80c00d2290 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f80a8001670 con 0x7f80c00d2290 2024-04-25T00:05:29.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 >> v1:172.21.15.113:6789/0 conn(0x7f80c00d2290 legacy=0x7f80c00b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac003a00 0x7f80ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac004800 0x7f80ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80ac0051c0 0x7f80ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f80c00d3410 con 0x7f80ac004800 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac00c0a0 con 0x7f80ac0051c0 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f80ac00c2a0 con 0x7f80ac003a00 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80b6ffd640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac004800 0x7f80ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80bf7fe640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac003a00 0x7f80ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80bffff640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80ac0051c0 0x7f80ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80b6ffd640 1 -- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac003a00 msgr2=0x7f80ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80b6ffd640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac003a00 0x7f80ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80b6ffd640 1 -- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80ac0051c0 msgr2=0x7f80ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:29.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80b6ffd640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80ac0051c0 0x7f80ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80b6ffd640 1 -- 172.21.15.113:0/3488287501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac003520 con 0x7f80ac004800 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80bf7fe640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac003a00 0x7f80ac003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80bffff640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80ac0051c0 0x7f80ac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.759+0000 7f80bffff640 1 -- 172.21.15.113:0/3488287501 reap_dead start 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.760+0000 7f80b6ffd640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac004800 0x7f80ac004c80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c190 tx=0x7f80ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8a18cd60a0280e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.760+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8007c60 con 0x7f80ac004800 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.760+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a8005af0 con 0x7f80ac004800 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.760+0000 7f80beffd640 1 -- 172.21.15.113:0/3488287501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8007070 con 0x7f80ac004800 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.761+0000 7f80c5ae1640 1 -- 172.21.15.113:0/3488287501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac004800 msgr2=0x7f80ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.761+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac004800 0x7f80ac004c80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c190 tx=0x7f80ac005830 comp rx=0 tx=0).stop 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.761+0000 7f80c5ae1640 1 -- 172.21.15.113:0/3488287501 shutdown_connections 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.761+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/3488287501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80ac004800 0x7f80ac004c80 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.761+0000 7f80c5ae1640 1 -- 172.21.15.113:0/3488287501 >> 172.21.15.113:0/3488287501 conn(0x7f80c00b5c20 msgr2=0x7f80c00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:29.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.761+0000 7f80c5ae1640 1 -- 172.21.15.113:0/3488287501 shutdown_connections 2024-04-25T00:05:29.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.761+0000 7f80c5ae1640 1 -- 172.21.15.113:0/3488287501 wait complete. 2024-04-25T00:05:29.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.762+0000 7f80c5ae1640 1 Processor -- start 2024-04-25T00:05:29.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.762+0000 7f80c5ae1640 1 -- start start 2024-04-25T00:05:29.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80c5ae1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac004800 0x7f80c014e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80c5ae1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c00cdd50 0x7f80c0160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80c5ae1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c0160f50 0x7f80c01f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80c5ae1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f80c015e700 con 0x7f80c0160f50 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80c5ae1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80c00d30a0 con 0x7f80c00cdd50 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80c5ae1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f80c015e880 con 0x7f80ac004800 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c0160f50 0x7f80c01f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c0160f50 0x7f80c01f4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45506/0 (socket says 172.21.15.113:45506) 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 -- 172.21.15.113:0/1745421107 learned_addr learned my addr 172.21.15.113:0/1745421107 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bf7fe640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac004800 0x7f80c014e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80b6ffd640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c00cdd50 0x7f80c0160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 -- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac004800 msgr2=0x7f80c014e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac004800 0x7f80c014e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 -- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c00cdd50 msgr2=0x7f80c0160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c00cdd50 0x7f80c0160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 -- 172.21.15.113:0/1745421107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c00cccb0 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80b6ffd640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c00cdd50 0x7f80c0160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bffff640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c0160f50 0x7f80c01f4170 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f80c00d51b0 tx=0x7f80b8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bcff9640 1 -- 172.21.15.113:0/1745421107 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8010e00 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bcff9640 1 -- 172.21.15.113:0/1745421107 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b8003990 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c00ccf90 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.763+0000 7f80bcff9640 1 -- 172.21.15.113:0/1745421107 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8003d80 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.764+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c00cd550 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.764+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c00691a0 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.765+0000 7f80bcff9640 1 -- 172.21.15.113:0/1745421107 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80b801e020 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.765+0000 7f80bcff9640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80a006be30 0x7f80a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.765+0000 7f80bcff9640 1 -- 172.21.15.113:0/1745421107 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 4943+0+0 (secure 0 0 0) 0x7f80b808b290 con 0x7f80c0160f50 2024-04-25T00:05:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.765+0000 7f80bf7fe640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80a006be30 0x7f80a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.766+0000 7f80bf7fe640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80a006be30 0x7f80a006e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000c00 tx=0x7f80b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:29.770+0000 7f80bcff9640 1 -- 172.21.15.113:0/1745421107 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80b8059f40 con 0x7f80c0160f50 2024-04-25T00:05:30.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.012+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) -- 0x7f80c00d1350 con 0x7f80c0160f50 2024-04-25T00:05:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.053+0000 7f80bcff9640 1 -- 172.21.15.113:0/1745421107 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v354) ==== 169+0+0 (secure 0 0 0) 0x7f80b8059840 con 0x7f80c0160f50 2024-04-25T00:05:30.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80a006be30 msgr2=0x7f80a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80a006be30 0x7f80a006e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000c00 tx=0x7f80b0009040 comp rx=0 tx=0).stop 2024-04-25T00:05:30.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c0160f50 msgr2=0x7f80c01f4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c0160f50 0x7f80c01f4170 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f80c00d51b0 tx=0x7f80b8010590 comp rx=0 tx=0).stop 2024-04-25T00:05:30.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 shutdown_connections 2024-04-25T00:05:30.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f80a006be30 0x7f80a006e2f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80c00cdd50 0x7f80c0160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80c0160f50 0x7f80c01f4170 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 --2- 172.21.15.113:0/1745421107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80ac004800 0x7f80c014e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.055+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 >> 172.21.15.113:0/1745421107 conn(0x7f80c00b5c20 msgr2=0x7f80c00ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:30.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.056+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 shutdown_connections 2024-04-25T00:05:30.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.056+0000 7f80c5ae1640 1 -- 172.21.15.113:0/1745421107 wait complete. 2024-04-25T00:05:30.069 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:05:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.169+0000 7f7223a72640 1 Processor -- start 2024-04-25T00:05:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.169+0000 7f7223a72640 1 -- start start 2024-04-25T00:05:30.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f7223a72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f7223a72640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f721c0d04f0 0x7f721c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f7223a72640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721c0d0970 con 0x7f721c1604d0 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f7223a72640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f721c15c6a0 con 0x7f721c0d04f0 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f7223a72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f721c15d6b0 con 0x7f721c13f050 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722180e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722100d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f721c0d04f0 0x7f721c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722180e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:33666/0 (socket says 172.21.15.113:33666) 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722180e640 1 -- 172.21.15.113:0/2329376335 learned_addr learned my addr 172.21.15.113:0/2329376335 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722180e640 1 -- 172.21.15.113:0/2329376335 >> v2:172.21.15.178:3300/0 conn(0x7f721c0d04f0 msgr2=0x7f721c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722180e640 1 --2- 172.21.15.113:0/2329376335 >> v2:172.21.15.178:3300/0 conn(0x7f721c0d04f0 0x7f721c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722180e640 1 -- 172.21.15.113:0/2329376335 >> v1:172.21.15.178:6789/0 conn(0x7f721c1604d0 legacy=0x7f721c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.170+0000 7f722180e640 1 -- 172.21.15.113:0/2329376335 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c15c8f0 con 0x7f721c13f050 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.171+0000 7f722180e640 1 --2- 172.21.15.113:0/2329376335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0cffb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7218009870 tx=0x7f721802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919e0026f1c2612f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.171+0000 7f722080c640 1 -- 172.21.15.113:0/2329376335 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218037070 con 0x7f721c13f050 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.171+0000 7f722080c640 1 -- 172.21.15.113:0/2329376335 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7218032440 con 0x7f721c13f050 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.171+0000 7f722080c640 1 -- 172.21.15.113:0/2329376335 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218032810 con 0x7f721c13f050 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 -- 172.21.15.113:0/2329376335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 msgr2=0x7f721c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 --2- 172.21.15.113:0/2329376335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0cffb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7218009870 tx=0x7f721802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 -- 172.21.15.113:0/2329376335 shutdown_connections 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 --2- 172.21.15.113:0/2329376335 >> v2:172.21.15.178:3300/0 conn(0x7f721c0d04f0 0x7f721c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 --2- 172.21.15.113:0/2329376335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0cffb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 -- 172.21.15.113:0/2329376335 >> 172.21.15.113:0/2329376335 conn(0x7f721c06b0f0 msgr2=0x7f721c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 -- 172.21.15.113:0/2329376335 shutdown_connections 2024-04-25T00:05:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.172+0000 7f7223a72640 1 -- 172.21.15.113:0/2329376335 wait complete. 2024-04-25T00:05:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.173+0000 7f7223a72640 1 Processor -- start 2024-04-25T00:05:30.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.173+0000 7f7223a72640 1 -- start start 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f7223a72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c0d04f0 0x7f721c0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f7223a72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f7223a72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721c1604d0 0x7f721c13a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f7223a72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f721c15d190 con 0x7f721c1604d0 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f7223a72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f721c15ceb0 con 0x7f721c0d04f0 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f7223a72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f721c13f430 con 0x7f721c13f050 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721c1604d0 0x7f721c13a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721c1604d0 0x7f721c13a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45510/0 (socket says 172.21.15.113:45510) 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 -- 172.21.15.113:0/3361042630 learned_addr learned my addr 172.21.15.113:0/3361042630 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722100d640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722180e640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c0d04f0 0x7f721c0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 -- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 msgr2=0x7f721c0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 -- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c0d04f0 msgr2=0x7f721c0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c0d04f0 0x7f721c0b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 -- 172.21.15.113:0/3361042630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c0b3f60 con 0x7f721c1604d0 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722180e640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c0d04f0 0x7f721c0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f722200f640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721c1604d0 0x7f721c13a6b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f721400c6b0 tx=0x7f721400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.174+0000 7f72127fc640 1 -- 172.21.15.113:0/3361042630 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214003b70 con 0x7f721c1604d0 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.175+0000 7f72127fc640 1 -- 172.21.15.113:0/3361042630 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7214003d30 con 0x7f721c1604d0 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.175+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c0b4240 con 0x7f721c1604d0 2024-04-25T00:05:30.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.175+0000 7f72127fc640 1 -- 172.21.15.113:0/3361042630 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214005310 con 0x7f721c1604d0 2024-04-25T00:05:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.175+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c0b4800 con 0x7f721c1604d0 2024-04-25T00:05:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.175+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721c0691a0 con 0x7f721c1604d0 2024-04-25T00:05:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.176+0000 7f72127fc640 1 -- 172.21.15.113:0/3361042630 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72140054b0 con 0x7f721c1604d0 2024-04-25T00:05:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.176+0000 7f72127fc640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f720006be30 0x7f720006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.176+0000 7f72127fc640 1 -- 172.21.15.113:0/3361042630 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 4943+0+0 (secure 0 0 0) 0x7f721408aa20 con 0x7f721c1604d0 2024-04-25T00:05:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.176+0000 7f722180e640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f720006be30 0x7f720006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.177+0000 7f722180e640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f720006be30 0x7f720006e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7218034000 tx=0x7f721802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.181+0000 7f72127fc640 1 -- 172.21.15.113:0/3361042630 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72140596d0 con 0x7f721c1604d0 2024-04-25T00:05:30.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.353+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f721c0d0e50 con 0x7f721c1604d0 2024-04-25T00:05:30.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.356+0000 7f72127fc640 1 -- 172.21.15.113:0/3361042630 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f7214058fd0 con 0x7f721c1604d0 2024-04-25T00:05:30.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f720006be30 msgr2=0x7f720006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f720006be30 0x7f720006e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7218034000 tx=0x7f721802e000 comp rx=0 tx=0).stop 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721c1604d0 msgr2=0x7f721c13a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721c1604d0 0x7f721c13a6b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f721400c6b0 tx=0x7f721400cb70 comp rx=0 tx=0).stop 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 shutdown_connections 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f720006be30 0x7f720006e2f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f721c1604d0 0x7f721c13a6b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f721c13f050 0x7f721c0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 --2- 172.21.15.113:0/3361042630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c0d04f0 0x7f721c0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 >> 172.21.15.113:0/3361042630 conn(0x7f721c06b0f0 msgr2=0x7f721c15efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 shutdown_connections 2024-04-25T00:05:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.358+0000 7f7223a72640 1 -- 172.21.15.113:0/3361042630 wait complete. 2024-04-25T00:05:30.370 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:05:30.370 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:05:30.416 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:05:30.463 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:05:30.463 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:05:30.463 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:30.479 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:05:30.479 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:05:30.479 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:30.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:05:30.499 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:05:30.499 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:30.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:05:30.534 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:05:30.534 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:30.556 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:05:30.556 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:05:30.556 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:30.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.671+0000 7f4e6e2a5640 1 Processor -- start 2024-04-25T00:05:30.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.671+0000 7f4e6e2a5640 1 -- start start 2024-04-25T00:05:30.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e6e2a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e68144910 0x7f4e680b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e6e2a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4e680b5240 0x7f4e6806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e6e2a5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e68144cf0 con 0x7f4e680b56b0 2024-04-25T00:05:30.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e6e2a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4e680b4510 con 0x7f4e68144910 2024-04-25T00:05:30.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e6e2a5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4e680b2f30 con 0x7f4e680b5240 2024-04-25T00:05:30.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e67fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e68144910 0x7f4e680b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e67fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e68144910 0x7f4e680b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45514/0 (socket says 172.21.15.113:45514) 2024-04-25T00:05:30.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e67fff640 1 -- 172.21.15.113:0/653880425 learned_addr learned my addr 172.21.15.113:0/653880425 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:30.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e67fff640 1 -- 172.21.15.113:0/653880425 >> v2:172.21.15.178:3300/0 conn(0x7f4e680b5240 msgr2=0x7f4e6806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:30.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e67fff640 1 --2- 172.21.15.113:0/653880425 >> v2:172.21.15.178:3300/0 conn(0x7f4e680b5240 0x7f4e6806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e67fff640 1 -- 172.21.15.113:0/653880425 >> v1:172.21.15.178:6789/0 conn(0x7f4e680b56b0 legacy=0x7f4e680cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:30.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.672+0000 7f4e67fff640 1 -- 172.21.15.113:0/653880425 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e680b3210 con 0x7f4e68144910 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.673+0000 7f4e67fff640 1 --2- 172.21.15.113:0/653880425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e68144910 0x7f4e680b3fd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009870 tx=0x7f4e5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8956742c7072b2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.673+0000 7f4e66ffd640 1 -- 172.21.15.113:0/653880425 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58037070 con 0x7f4e68144910 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.673+0000 7f4e66ffd640 1 -- 172.21.15.113:0/653880425 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e58032440 con 0x7f4e68144910 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.673+0000 7f4e66ffd640 1 -- 172.21.15.113:0/653880425 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58032810 con 0x7f4e68144910 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/653880425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e68144910 msgr2=0x7f4e680b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/653880425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e68144910 0x7f4e680b3fd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009870 tx=0x7f4e5802a270 comp rx=0 tx=0).stop 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/653880425 shutdown_connections 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/653880425 >> v2:172.21.15.178:3300/0 conn(0x7f4e680b5240 0x7f4e6806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/653880425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e68144910 0x7f4e680b3fd0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/653880425 >> 172.21.15.113:0/653880425 conn(0x7f4e680b1a50 msgr2=0x7f4e680b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/653880425 shutdown_connections 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.674+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/653880425 wait complete. 2024-04-25T00:05:30.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.675+0000 7f4e6e2a5640 1 Processor -- start 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.675+0000 7f4e6e2a5640 1 -- start start 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e680b5240 0x7f4e680b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e680b56b0 0x7f4e680ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e68144910 0x7f4e680aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e680b4da0 con 0x7f4e680b56b0 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e680b37d0 con 0x7f4e68144910 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4e680b4f20 con 0x7f4e680b5240 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e680b56b0 0x7f4e680ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e680b56b0 0x7f4e680ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45524/0 (socket says 172.21.15.113:45524) 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 -- 172.21.15.113:0/1498813920 learned_addr learned my addr 172.21.15.113:0/1498813920 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e67fff640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e680b5240 0x7f4e680b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6c842640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e68144910 0x7f4e680aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 -- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e680b5240 msgr2=0x7f4e680b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e680b5240 0x7f4e680b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 -- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e68144910 msgr2=0x7f4e680aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e68144910 0x7f4e680aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 -- 172.21.15.113:0/1498813920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e680b1730 con 0x7f4e680b56b0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6c842640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e68144910 0x7f4e680aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e677fe640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e680b56b0 0x7f4e680ae7b0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b4a0 tx=0x7f4e5c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e64ff9640 1 -- 172.21.15.113:0/1498813920 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c014070 con 0x7f4e680b56b0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e64ff9640 1 -- 172.21.15.113:0/1498813920 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c010610 con 0x7f4e680b56b0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e681f7ad0 con 0x7f4e680b56b0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e64ff9640 1 -- 172.21.15.113:0/1498813920 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c010a00 con 0x7f4e680b56b0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.676+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e681f8090 con 0x7f4e680b56b0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.678+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e68068580 con 0x7f4e680b56b0 2024-04-25T00:05:30.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.678+0000 7f4e64ff9640 1 -- 172.21.15.113:0/1498813920 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e5c017070 con 0x7f4e680b56b0 2024-04-25T00:05:30.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.678+0000 7f4e64ff9640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e5406be30 0x7f4e5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:30.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.678+0000 7f4e64ff9640 1 -- 172.21.15.113:0/1498813920 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 4943+0+0 (secure 0 0 0) 0x7f4e5c08a580 con 0x7f4e680b56b0 2024-04-25T00:05:30.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.678+0000 7f4e67fff640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e5406be30 0x7f4e5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:30.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.679+0000 7f4e67fff640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e5406be30 0x7f4e5406e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4e58034000 tx=0x7f4e5802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:30.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.683+0000 7f4e64ff9640 1 -- 172.21.15.113:0/1498813920 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e68068580 con 0x7f4e680b56b0 2024-04-25T00:05:30.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.890+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e680af330 con 0x7f4e680b56b0 2024-04-25T00:05:30.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.891+0000 7f4e64ff9640 1 -- 172.21.15.113:0/1498813920 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+3881 (secure 0 0 0) 0x7f4e680b1730 con 0x7f4e680b56b0 2024-04-25T00:05:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:30.891 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1437481371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1437481371},{"type":"v1","addr":"172.21.15.113:6837","nonce":1437481371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3106607585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3106607585},{"type":"v1","addr":"172.21.15.178:6834","nonce":3106607585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6098,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2416393306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2416393306},{"type":"v1","addr":"172.21.15.178:6835","nonce":2416393306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1319976211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1319976211},{"type":"v1","addr":"172.21.15.113:6836","nonce":1319976211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[]} 2024-04-25T00:05:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 58 2024-04-25T00:05:30.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e5406be30 msgr2=0x7f4e5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e5406be30 0x7f4e5406e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4e58034000 tx=0x7f4e5802e000 comp rx=0 tx=0).stop 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e680b56b0 msgr2=0x7f4e680ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e680b56b0 0x7f4e680ae7b0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b4a0 tx=0x7f4e5c00b960 comp rx=0 tx=0).stop 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 shutdown_connections 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e5406be30 0x7f4e5406e2f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e68144910 0x7f4e680aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e680b56b0 0x7f4e680ae7b0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 --2- 172.21.15.113:0/1498813920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e680b5240 0x7f4e680b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 >> 172.21.15.113:0/1498813920 conn(0x7f4e680b1a50 msgr2=0x7f4e680b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 shutdown_connections 2024-04-25T00:05:30.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:30.893+0000 7f4e6e2a5640 1 -- 172.21.15.113:0/1498813920 wait complete. 2024-04-25T00:05:30.905 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:05:30.905 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:05:30.905 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:05:30.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:05:30.905 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:05:30.905 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:05:30.906 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:05:30.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:05:30.906 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:05:30.906 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:05:30.906 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:05:30.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:05:30.906 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:05:30.906 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:05:30.907 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:05:30.907 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:05:30.914 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:05:30.914 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:05:30.916 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:05:30.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:05:30.918 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:05:30.918 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:05:30.919 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:05:30.919 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:05:30.957 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:05:30.957 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:30.957 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:05:30.958 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:30.960 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:05:30.960 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:30.961 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:05:30.961 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:30.961 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:05:31.030 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:05:31.030 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:05:31.038 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:05:31.038 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:05:31.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.065+0000 7f0974f96640 1 Processor -- start 2024-04-25T00:05:31.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.066+0000 7f0974f96640 1 -- start start 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f0974f96640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f09700b3080 0x7f09700b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f0974f96640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f097015e6b0 0x7f09700b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f0974f96640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097015f4f0 con 0x7f097015e2d0 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f0974f96640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f09700cde00 con 0x7f097015e6b0 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f0974f96640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f097015f7d0 con 0x7f09700b3080 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096ed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f097015e6b0 0x7f09700b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096ed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f097015e6b0 0x7f09700b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45568/0 (socket says 172.21.15.113:45568) 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096dd9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f09700b3080 0x7f09700b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096ed9d640 1 -- 172.21.15.113:0/1148267421 learned_addr learned my addr 172.21.15.113:0/1148267421 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096ed9d640 1 -- 172.21.15.113:0/1148267421 >> v2:172.21.15.178:3300/0 conn(0x7f09700b3080 msgr2=0x7f09700b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:31.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096ed9d640 1 --2- 172.21.15.113:0/1148267421 >> v2:172.21.15.178:3300/0 conn(0x7f09700b3080 0x7f09700b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:31.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096ed9d640 1 -- 172.21.15.113:0/1148267421 >> v1:172.21.15.178:6789/0 conn(0x7f097015e2d0 legacy=0x7f097015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:31.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.067+0000 7f096ed9d640 1 -- 172.21.15.113:0/1148267421 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097015fab0 con 0x7f097015e6b0 2024-04-25T00:05:31.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.068+0000 7f096ed9d640 1 --2- 172.21.15.113:0/1148267421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e6b0 0x7f09700b4870 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0970070ff0 tx=0x7f096802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a923b0d228ed7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:31.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.068+0000 7f096d59a640 1 -- 172.21.15.113:0/1148267421 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968036070 con 0x7f097015e6b0 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.068+0000 7f096d59a640 1 -- 172.21.15.113:0/1148267421 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0968031600 con 0x7f097015e6b0 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.068+0000 7f096d59a640 1 -- 172.21.15.113:0/1148267421 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09680319d0 con 0x7f097015e6b0 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 -- 172.21.15.113:0/1148267421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e6b0 msgr2=0x7f09700b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 --2- 172.21.15.113:0/1148267421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e6b0 0x7f09700b4870 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0970070ff0 tx=0x7f096802a730 comp rx=0 tx=0).stop 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 -- 172.21.15.113:0/1148267421 shutdown_connections 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 --2- 172.21.15.113:0/1148267421 >> v2:172.21.15.178:3300/0 conn(0x7f09700b3080 0x7f09700b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 --2- 172.21.15.113:0/1148267421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e6b0 0x7f09700b4870 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 -- 172.21.15.113:0/1148267421 >> 172.21.15.113:0/1148267421 conn(0x7f097006b0f0 msgr2=0x7f097006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 -- 172.21.15.113:0/1148267421 shutdown_connections 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.069+0000 7f0974f96640 1 -- 172.21.15.113:0/1148267421 wait complete. 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.070+0000 7f0974f96640 1 Processor -- start 2024-04-25T00:05:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.070+0000 7f0974f96640 1 -- start start 2024-04-25T00:05:31.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f0974f96640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09700b3080 0x7f09701f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:31.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f0974f96640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e2d0 0x7f0970151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:31.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f0974f96640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f097015e6b0 0x7f09700b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:31.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f0974f96640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09700ce410 con 0x7f097015e2d0 2024-04-25T00:05:31.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f0974f96640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09700ce290 con 0x7f097015e6b0 2024-04-25T00:05:31.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f0974f96640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09700ce590 con 0x7f09700b3080 2024-04-25T00:05:31.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e2d0 0x7f0970151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:31.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e2d0 0x7f0970151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45582/0 (socket says 172.21.15.113:45582) 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.071+0000 7f096dd9b640 1 -- 172.21.15.113:0/1514099899 learned_addr learned my addr 172.21.15.113:0/1514099899 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096e59c640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09700b3080 0x7f09701f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096ed9d640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f097015e6b0 0x7f09700b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096dd9b640 1 -- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09700b3080 msgr2=0x7f09701f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096dd9b640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09700b3080 0x7f09701f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096dd9b640 1 -- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f097015e6b0 msgr2=0x7f09700b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096dd9b640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f097015e6b0 0x7f09700b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096dd9b640 1 -- 172.21.15.113:0/1514099899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09701f19c0 con 0x7f097015e2d0 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096e59c640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09700b3080 0x7f09701f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f096dd9b640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e2d0 0x7f0970151f10 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0964010bf0 tx=0x7f096400e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f095effd640 1 -- 172.21.15.113:0/1514099899 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096401e070 con 0x7f097015e2d0 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f095effd640 1 -- 172.21.15.113:0/1514099899 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0964011650 con 0x7f097015e2d0 2024-04-25T00:05:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09701f1ca0 con 0x7f097015e2d0 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f095effd640 1 -- 172.21.15.113:0/1514099899 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964011a40 con 0x7f097015e2d0 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.072+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09701f2260 con 0x7f097015e2d0 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.073+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09700b54b0 con 0x7f097015e2d0 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.073+0000 7f095effd640 1 -- 172.21.15.113:0/1514099899 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0964021070 con 0x7f097015e2d0 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.074+0000 7f095effd640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.074+0000 7f095effd640 1 -- 172.21.15.113:0/1514099899 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 4943+0+0 (secure 0 0 0) 0x7f096408cd20 con 0x7f097015e2d0 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.074+0000 7f096e59c640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.074+0000 7f096e59c640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0960000f60 tx=0x7f0960009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.078+0000 7f095effd640 1 -- 172.21.15.113:0/1514099899 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09700b54b0 con 0x7f097015e2d0 2024-04-25T00:05:31.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:31.267+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f09700afec0 con 0x7f097015e2d0 2024-04-25T00:05:32.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.067+0000 7f095effd640 1 -- 172.21.15.113:0/1514099899 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) ==== 97+0+0 (secure 0 0 0) 0x7f09700afec0 con 0x7f097015e2d0 2024-04-25T00:05:32.067 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:05:32.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 msgr2=0x7f095006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0960000f60 tx=0x7f0960009040 comp rx=0 tx=0).stop 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e2d0 msgr2=0x7f0970151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e2d0 0x7f0970151f10 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0964010bf0 tx=0x7f096400e730 comp rx=0 tx=0).stop 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 shutdown_connections 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f095006be30 0x7f095006e2f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f097015e6b0 0x7f09700b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f097015e2d0 0x7f0970151f10 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 --2- 172.21.15.113:0/1514099899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09700b3080 0x7f09701f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 >> 172.21.15.113:0/1514099899 conn(0x7f097006b0f0 msgr2=0x7f09700b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 shutdown_connections 2024-04-25T00:05:32.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.070+0000 7f0974f96640 1 -- 172.21.15.113:0/1514099899 wait complete. 2024-04-25T00:05:32.082 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:05:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.178+0000 7fd6144ee640 1 Processor -- start 2024-04-25T00:05:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.178+0000 7fd6144ee640 1 -- start start 2024-04-25T00:05:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.179+0000 7fd6144ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c144d60 0x7fd60c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.179+0000 7fd6144ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd60c1453c0 0x7fd60c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.179+0000 7fd6144ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd60c0cfad0 con 0x7fd60c06e510 2024-04-25T00:05:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.179+0000 7fd6144ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd60c145c40 con 0x7fd60c1453c0 2024-04-25T00:05:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.179+0000 7fd6144ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd60c06e940 con 0x7fd60c144d60 2024-04-25T00:05:32.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd61228a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c144d60 0x7fd60c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd611a89640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd60c1453c0 0x7fd60c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd61228a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c144d60 0x7fd60c0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:33738/0 (socket says 172.21.15.113:33738) 2024-04-25T00:05:32.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd612a8b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd60c06e510 0x7fd60c15fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:40758/0 (socket says 172.21.15.113:40758) 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd611a89640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd60c1453c0 0x7fd60c06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:59080/0 (socket says 172.21.15.113:59080) 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd61228a640 1 -- 172.21.15.113:0/2763205488 learned_addr learned my addr 172.21.15.113:0/2763205488 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd61228a640 1 -- 172.21.15.113:0/2763205488 >> v2:172.21.15.178:3300/0 conn(0x7fd60c1453c0 msgr2=0x7fd60c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd61228a640 1 --2- 172.21.15.113:0/2763205488 >> v2:172.21.15.178:3300/0 conn(0x7fd60c1453c0 0x7fd60c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd61228a640 1 -- 172.21.15.113:0/2763205488 >> v1:172.21.15.178:6789/0 conn(0x7fd60c06e510 legacy=0x7fd60c15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd61228a640 1 -- 172.21.15.113:0/2763205488 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c0cf230 con 0x7fd60c144d60 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.180+0000 7fd611a89640 1 --2- 172.21.15.113:0/2763205488 >> v2:172.21.15.178:3300/0 conn(0x7fd60c1453c0 0x7fd60c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.181+0000 7fd61228a640 1 --2- 172.21.15.113:0/2763205488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c144d60 0x7fd60c0b5390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009870 tx=0x7fd5fc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96898dd5e78f806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.181+0000 7fd611288640 1 -- 172.21.15.113:0/2763205488 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc037070 con 0x7fd60c144d60 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.181+0000 7fd611288640 1 -- 172.21.15.113:0/2763205488 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5fc032440 con 0x7fd60c144d60 2024-04-25T00:05:32.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.181+0000 7fd611288640 1 -- 172.21.15.113:0/2763205488 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc032810 con 0x7fd60c144d60 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 -- 172.21.15.113:0/2763205488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c144d60 msgr2=0x7fd60c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 --2- 172.21.15.113:0/2763205488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c144d60 0x7fd60c0b5390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009870 tx=0x7fd5fc02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 -- 172.21.15.113:0/2763205488 shutdown_connections 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 --2- 172.21.15.113:0/2763205488 >> v2:172.21.15.178:3300/0 conn(0x7fd60c1453c0 0x7fd60c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 --2- 172.21.15.113:0/2763205488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c144d60 0x7fd60c0b5390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 -- 172.21.15.113:0/2763205488 >> 172.21.15.113:0/2763205488 conn(0x7fd60c0b1a50 msgr2=0x7fd60c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 -- 172.21.15.113:0/2763205488 shutdown_connections 2024-04-25T00:05:32.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.182+0000 7fd6144ee640 1 -- 172.21.15.113:0/2763205488 wait complete. 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.183+0000 7fd6144ee640 1 Processor -- start 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.183+0000 7fd6144ee640 1 -- start start 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd60c06e510 0x7fd60c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd60c144d60 0x7fd60c0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c1453c0 0x7fd60c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c070e10 con 0x7fd60c144d60 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c0cf7f0 con 0x7fd60c06e510 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd60c145140 con 0x7fd60c1453c0 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd60c144d60 0x7fd60c0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd60c144d60 0x7fd60c0b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45598/0 (socket says 172.21.15.113:45598) 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 -- 172.21.15.113:0/3687753812 learned_addr learned my addr 172.21.15.113:0/3687753812 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd61228a640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd60c06e510 0x7fd60c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd612a8b640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c1453c0 0x7fd60c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 -- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c1453c0 msgr2=0x7fd60c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c1453c0 0x7fd60c0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 -- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd60c06e510 msgr2=0x7fd60c0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd60c06e510 0x7fd60c0b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 -- 172.21.15.113:0/3687753812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c0b4de0 con 0x7fd60c144d60 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd611a89640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd60c144d60 0x7fd60c0b0d20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd60800ebe0 tx=0x7fd60800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd602ffd640 1 -- 172.21.15.113:0/3687753812 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608017070 con 0x7fd60c144d60 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd602ffd640 1 -- 172.21.15.113:0/3687753812 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd60800cc30 con 0x7fd60c144d60 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd602ffd640 1 -- 172.21.15.113:0/3687753812 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608013470 con 0x7fd60c144d60 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c0b3d40 con 0x7fd60c144d60 2024-04-25T00:05:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.184+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c0b4300 con 0x7fd60c144d60 2024-04-25T00:05:32.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.185+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c0691a0 con 0x7fd60c144d60 2024-04-25T00:05:32.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.186+0000 7fd602ffd640 1 -- 172.21.15.113:0/3687753812 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd60801a070 con 0x7fd60c144d60 2024-04-25T00:05:32.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.186+0000 7fd602ffd640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5f006be30 0x7fd5f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.186+0000 7fd61228a640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5f006be30 0x7fd5f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.186+0000 7fd602ffd640 1 -- 172.21.15.113:0/3687753812 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7fd60808a980 con 0x7fd60c144d60 2024-04-25T00:05:32.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.187+0000 7fd61228a640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5f006be30 0x7fd5f006e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc034000 tx=0x7fd5fc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.190+0000 7fd602ffd640 1 -- 172.21.15.113:0/3687753812 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd6080590c0 con 0x7fd60c144d60 2024-04-25T00:05:32.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.431+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd60c0b27e0 con 0x7fd60c144d60 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.431+0000 7fd602ffd640 1 -- 172.21.15.113:0/3687753812 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fd60805d3c0 con 0x7fd60c144d60 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:05:32.432 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.434 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:05:32.435 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.436 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:05:32.441 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:05:32.442 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:05:32.443 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:05:32.444 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:05:32.445 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5f006be30 msgr2=0x7fd5f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5f006be30 0x7fd5f006e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc034000 tx=0x7fd5fc02e000 comp rx=0 tx=0).stop 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd60c144d60 msgr2=0x7fd60c0b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd60c144d60 0x7fd60c0b0d20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd60800ebe0 tx=0x7fd60800c520 comp rx=0 tx=0).stop 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 shutdown_connections 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5f006be30 0x7fd5f006e2f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd60c1453c0 0x7fd60c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd60c144d60 0x7fd60c0b0d20 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 --2- 172.21.15.113:0/3687753812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd60c06e510 0x7fd60c0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 >> 172.21.15.113:0/3687753812 conn(0x7fd60c0b1a50 msgr2=0x7fd60c0723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:32.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 shutdown_connections 2024-04-25T00:05:32.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.434+0000 7fd6144ee640 1 -- 172.21.15.113:0/3687753812 wait complete. 2024-04-25T00:05:32.448 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.544+0000 7fec465c1640 1 Processor -- start 2024-04-25T00:05:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.544+0000 7fec465c1640 1 -- start start 2024-04-25T00:05:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec465c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec400b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec465c1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fec401453c0 0x7fec4006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec465c1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec400cfad0 con 0x7fec4006e510 2024-04-25T00:05:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec465c1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fec40145c40 con 0x7fec401453c0 2024-04-25T00:05:32.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec465c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec4006e940 con 0x7fec40144d60 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec400b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec3f7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fec401453c0 0x7fec4006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec3f7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fec401453c0 0x7fec4006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:59094/0 (socket says 172.21.15.113:59094) 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.545+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec400b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:33764/0 (socket says 172.21.15.113:33764) 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.546+0000 7fec3f7fe640 1 -- 172.21.15.113:0/766887791 learned_addr learned my addr 172.21.15.113:0/766887791 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.546+0000 7fec44b5e640 1 --1- 172.21.15.113:0/766887791 >> v1:172.21.15.178:6789/0 conn(0x7fec4006e510 0x7fec4015fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:40760/0 (socket says 172.21.15.113:40760) 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.546+0000 7fec3ffff640 1 -- 172.21.15.113:0/766887791 >> v2:172.21.15.178:3300/0 conn(0x7fec401453c0 msgr2=0x7fec4006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:32.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.546+0000 7fec3ffff640 1 --2- 172.21.15.113:0/766887791 >> v2:172.21.15.178:3300/0 conn(0x7fec401453c0 0x7fec4006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.546+0000 7fec3ffff640 1 -- 172.21.15.113:0/766887791 >> v1:172.21.15.178:6789/0 conn(0x7fec4006e510 legacy=0x7fec4015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.546+0000 7fec3ffff640 1 -- 172.21.15.113:0/766887791 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec400cf230 con 0x7fec40144d60 2024-04-25T00:05:32.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.546+0000 7fec3f7fe640 1 --2- 172.21.15.113:0/766887791 >> v2:172.21.15.178:3300/0 conn(0x7fec401453c0 0x7fec4006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:32.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec3ffff640 1 --2- 172.21.15.113:0/766887791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec400b5390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec300097a0 tx=0x7fec3002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4bcc112d56055ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec3effd640 1 -- 172.21.15.113:0/766887791 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30037070 con 0x7fec40144d60 2024-04-25T00:05:32.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec3effd640 1 -- 172.21.15.113:0/766887791 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3002ae50 con 0x7fec40144d60 2024-04-25T00:05:32.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec3effd640 1 -- 172.21.15.113:0/766887791 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30032660 con 0x7fec40144d60 2024-04-25T00:05:32.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 -- 172.21.15.113:0/766887791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 msgr2=0x7fec400b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 --2- 172.21.15.113:0/766887791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec400b5390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec300097a0 tx=0x7fec3002a100 comp rx=0 tx=0).stop 2024-04-25T00:05:32.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 -- 172.21.15.113:0/766887791 shutdown_connections 2024-04-25T00:05:32.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 --2- 172.21.15.113:0/766887791 >> v2:172.21.15.178:3300/0 conn(0x7fec401453c0 0x7fec4006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 --2- 172.21.15.113:0/766887791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec400b5390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 -- 172.21.15.113:0/766887791 >> 172.21.15.113:0/766887791 conn(0x7fec400b1a50 msgr2=0x7fec400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:32.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 -- 172.21.15.113:0/766887791 shutdown_connections 2024-04-25T00:05:32.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.547+0000 7fec465c1640 1 -- 172.21.15.113:0/766887791 wait complete. 2024-04-25T00:05:32.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.548+0000 7fec465c1640 1 Processor -- start 2024-04-25T00:05:32.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec465c1640 1 -- start start 2024-04-25T00:05:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec465c1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec4006e510 0x7fec40160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec465c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec401e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec465c1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec401453c0 0x7fec4013c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec465c1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fec40070e10 con 0x7fec4006e510 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec465c1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec400cf7f0 con 0x7fec401453c0 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec465c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec40145140 con 0x7fec40144d60 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec4006e510 0x7fec40160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec4006e510 0x7fec40160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45614/0 (socket says 172.21.15.113:45614) 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.549+0000 7fec3ffff640 1 -- 172.21.15.113:0/4020863290 learned_addr learned my addr 172.21.15.113:0/4020863290 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3f7fe640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec401e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec44b5e640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec401453c0 0x7fec4013c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3ffff640 1 -- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 msgr2=0x7fec401e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3ffff640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec401e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3ffff640 1 -- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec401453c0 msgr2=0x7fec4013c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3ffff640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec401453c0 0x7fec4013c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3ffff640 1 -- 172.21.15.113:0/4020863290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4014c1f0 con 0x7fec4006e510 2024-04-25T00:05:32.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3f7fe640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec401e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec44b5e640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec401453c0 0x7fec4013c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3ffff640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec4006e510 0x7fec40160270 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fec30035040 tx=0x7fec300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3cff9640 1 -- 172.21.15.113:0/4020863290 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30041070 con 0x7fec4006e510 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3cff9640 1 -- 172.21.15.113:0/4020863290 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3003c460 con 0x7fec4006e510 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec401e6bb0 con 0x7fec4006e510 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec3cff9640 1 -- 172.21.15.113:0/4020863290 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30037040 con 0x7fec4006e510 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.550+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec401e7170 con 0x7fec4006e510 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.551+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec400691a0 con 0x7fec4006e510 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.552+0000 7fec3cff9640 1 -- 172.21.15.113:0/4020863290 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec3003a070 con 0x7fec4006e510 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.552+0000 7fec3cff9640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec2c06be30 0x7fec2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.552+0000 7fec3f7fe640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec2c06be30 0x7fec2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.552+0000 7fec3cff9640 1 -- 172.21.15.113:0/4020863290 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7fec300aae00 con 0x7fec4006e510 2024-04-25T00:05:32.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.553+0000 7fec3f7fe640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec2c06be30 0x7fec2c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fec34009630 tx=0x7fec34008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.556+0000 7fec3cff9640 1 -- 172.21.15.113:0/4020863290 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec30079a70 con 0x7fec4006e510 2024-04-25T00:05:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.729+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec400b27e0 con 0x7fec4006e510 2024-04-25T00:05:32.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.730+0000 7fec3cff9640 1 -- 172.21.15.113:0/4020863290 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+12949 (secure 0 0 0) 0x7fec30079370 con 0x7fec4006e510 2024-04-25T00:05:32.730 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:32.730 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":112,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:05:32.064661+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:32.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec2c06be30 msgr2=0x7fec2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec2c06be30 0x7fec2c06e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fec34009630 tx=0x7fec34008040 comp rx=0 tx=0).stop 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec4006e510 msgr2=0x7fec40160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec4006e510 0x7fec40160270 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fec30035040 tx=0x7fec300046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 shutdown_connections 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec2c06be30 0x7fec2c06e2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec401453c0 0x7fec4013c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec40144d60 0x7fec401e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 --2- 172.21.15.113:0/4020863290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec4006e510 0x7fec40160270 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 >> 172.21.15.113:0/4020863290 conn(0x7fec400b1a50 msgr2=0x7fec400723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 shutdown_connections 2024-04-25T00:05:32.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.732+0000 7fec465c1640 1 -- 172.21.15.113:0/4020863290 wait complete. 2024-04-25T00:05:32.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:05:32.744 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:05:32.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.844+0000 7f401a742640 1 Processor -- start 2024-04-25T00:05:32.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.844+0000 7f401a742640 1 -- start start 2024-04-25T00:05:32.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f401a742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e2d0 0x7f401415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f401a742640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40140b3080 0x7f40140b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f401a742640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f401415e6b0 0x7f40140b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f401a742640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f40140cde00 con 0x7f40140b3080 2024-04-25T00:05:32.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f401a742640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f401415f4f0 con 0x7f401415e6b0 2024-04-25T00:05:32.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f401a742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f401415f7d0 con 0x7f401415e2d0 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40140b3080 0x7f40140b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40140b3080 0x7f40140b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45620/0 (socket says 172.21.15.113:45620) 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 -- 172.21.15.113:0/642415942 learned_addr learned my addr 172.21.15.113:0/642415942 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f4018cdf640 1 --2- 172.21.15.113:0/642415942 >> v2:172.21.15.178:3300/0 conn(0x7f401415e6b0 0x7f40140b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f4013fff640 1 --2- 172.21.15.113:0/642415942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e2d0 0x7f401415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 -- 172.21.15.113:0/642415942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e2d0 msgr2=0x7f401415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 --2- 172.21.15.113:0/642415942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e2d0 0x7f401415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 -- 172.21.15.113:0/642415942 >> v2:172.21.15.178:3300/0 conn(0x7f401415e6b0 msgr2=0x7f40140b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 --2- 172.21.15.113:0/642415942 >> v2:172.21.15.178:3300/0 conn(0x7f401415e6b0 0x7f40140b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f40137fe640 1 -- 172.21.15.113:0/642415942 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401415fab0 con 0x7f40140b3080 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f4013fff640 1 --2- 172.21.15.113:0/642415942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e2d0 0x7f401415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.845+0000 7f4018cdf640 1 --2- 172.21.15.113:0/642415942 >> v2:172.21.15.178:3300/0 conn(0x7f401415e6b0 0x7f40140b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:32.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.846+0000 7f40137fe640 1 --2- 172.21.15.113:0/642415942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40140b4330 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f401406f500 tx=0x7f400802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0458ab92a473c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.846+0000 7f4012ffd640 1 -- 172.21.15.113:0/642415942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008037070 con 0x7f40140b3080 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.846+0000 7f4012ffd640 1 -- 172.21.15.113:0/642415942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4008032440 con 0x7f40140b3080 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.846+0000 7f4012ffd640 1 -- 172.21.15.113:0/642415942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008032810 con 0x7f40140b3080 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 -- 172.21.15.113:0/642415942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 msgr2=0x7f40140b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 --2- 172.21.15.113:0/642415942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40140b4330 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f401406f500 tx=0x7f400802a140 comp rx=0 tx=0).stop 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 -- 172.21.15.113:0/642415942 shutdown_connections 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 --2- 172.21.15.113:0/642415942 >> v2:172.21.15.178:3300/0 conn(0x7f401415e6b0 0x7f40140b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 --2- 172.21.15.113:0/642415942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40140b4330 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 --2- 172.21.15.113:0/642415942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e2d0 0x7f401415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 -- 172.21.15.113:0/642415942 >> 172.21.15.113:0/642415942 conn(0x7f401406b0f0 msgr2=0x7f401406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 -- 172.21.15.113:0/642415942 shutdown_connections 2024-04-25T00:05:32.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.847+0000 7f401a742640 1 -- 172.21.15.113:0/642415942 wait complete. 2024-04-25T00:05:32.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.848+0000 7f401a742640 1 Processor -- start 2024-04-25T00:05:32.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.848+0000 7f401a742640 1 -- start start 2024-04-25T00:05:32.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40141f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401415e2d0 0x7f40141f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e6b0 0x7f40141586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f401415f2b0 con 0x7f40140b3080 2024-04-25T00:05:32.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f401415f130 con 0x7f401415e2d0 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4014071040 con 0x7f401415e6b0 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40141f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40141f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45624/0 (socket says 172.21.15.113:45624) 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 -- 172.21.15.113:0/2849550041 learned_addr learned my addr 172.21.15.113:0/2849550041 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f40137fe640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401415e2d0 0x7f40141f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4018cdf640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e6b0 0x7f40141586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 -- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e6b0 msgr2=0x7f40141586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e6b0 0x7f40141586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 -- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401415e2d0 msgr2=0x7f40141f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401415e2d0 0x7f40141f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 -- 172.21.15.113:0/2849550041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141f54e0 con 0x7f40140b3080 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4013fff640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40141f2130 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f400400c7f0 tx=0x7f400400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4010ff9640 1 -- 172.21.15.113:0/2849550041 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004015070 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4010ff9640 1 -- 172.21.15.113:0/2849550041 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4004007a30 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401415a320 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f4010ff9640 1 -- 172.21.15.113:0/2849550041 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004007e00 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.849+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401415a8e0 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.850+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40140691a0 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.851+0000 7f4010ff9640 1 -- 172.21.15.113:0/2849550041 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4004018070 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.851+0000 7f4010ff9640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.851+0000 7f4010ff9640 1 -- 172.21.15.113:0/2849550041 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7f400408a9c0 con 0x7f40140b3080 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.851+0000 7f40137fe640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.852+0000 7f40137fe640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4008034000 tx=0x7f400802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:32.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:32.855+0000 7f4010ff9640 1 -- 172.21.15.113:0/2849550041 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f400408ce20 con 0x7f40140b3080 2024-04-25T00:05:33.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.028+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f40140afe50 con 0x7f40140b3080 2024-04-25T00:05:33.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.029+0000 7f4010ff9640 1 -- 172.21.15.113:0/2849550041 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f4004059100 con 0x7f40140b3080 2024-04-25T00:05:33.029 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:05:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 msgr2=0x7f400006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4008034000 tx=0x7f400802e000 comp rx=0 tx=0).stop 2024-04-25T00:05:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 msgr2=0x7f40141f2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40141f2130 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f400400c7f0 tx=0x7f400400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 shutdown_connections 2024-04-25T00:05:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f400006be30 0x7f400006e2f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f401415e2d0 0x7f40141f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f401415e6b0 0x7f40141586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 --2- 172.21.15.113:0/2849550041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140b3080 0x7f40141f2130 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.030+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 >> 172.21.15.113:0/2849550041 conn(0x7f401406b0f0 msgr2=0x7f40140714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:33.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.031+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 shutdown_connections 2024-04-25T00:05:33.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.031+0000 7f401a742640 1 -- 172.21.15.113:0/2849550041 wait complete. 2024-04-25T00:05:33.042 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:05:33.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.138+0000 7f689327d640 1 Processor -- start 2024-04-25T00:05:33.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.139+0000 7f689327d640 1 -- start start 2024-04-25T00:05:33.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.139+0000 7f689327d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c0b3fd0 0x7f688c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.139+0000 7f689327d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f688c06e510 0x7f688c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.139+0000 7f689327d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688c144cf0 con 0x7f688c144910 2024-04-25T00:05:33.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689327d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f688c0b2f30 con 0x7f688c06e510 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689327d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f688c0b3210 con 0x7f688c0b3fd0 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689181a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f688c06e510 0x7f688c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689181a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f688c06e510 0x7f688c0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45632/0 (socket says 172.21.15.113:45632) 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689181a640 1 -- 172.21.15.113:0/2866774383 learned_addr learned my addr 172.21.15.113:0/2866774383 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f6890818640 1 --2- 172.21.15.113:0/2866774383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c0b3fd0 0x7f688c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689181a640 1 -- 172.21.15.113:0/2866774383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c0b3fd0 msgr2=0x7f688c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689181a640 1 --2- 172.21.15.113:0/2866774383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c0b3fd0 0x7f688c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689181a640 1 -- 172.21.15.113:0/2866774383 >> v1:172.21.15.178:6789/0 conn(0x7f688c144910 legacy=0x7f688c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f689181a640 1 -- 172.21.15.113:0/2866774383 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c0b34f0 con 0x7f688c06e510 2024-04-25T00:05:33.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.140+0000 7f6890818640 1 --2- 172.21.15.113:0/2866774383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c0b3fd0 0x7f688c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689181a640 1 --2- 172.21.15.113:0/2866774383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c0cd500 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6884009e50 tx=0x7f688402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff387c7dae0fa3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f6883fff640 1 -- 172.21.15.113:0/2866774383 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884036070 con 0x7f688c06e510 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f6883fff640 1 -- 172.21.15.113:0/2866774383 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884031600 con 0x7f688c06e510 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f6883fff640 1 -- 172.21.15.113:0/2866774383 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840319d0 con 0x7f688c06e510 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 -- 172.21.15.113:0/2866774383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 msgr2=0x7f688c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 --2- 172.21.15.113:0/2866774383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c0cd500 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6884009e50 tx=0x7f688402a6c0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 -- 172.21.15.113:0/2866774383 shutdown_connections 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 --2- 172.21.15.113:0/2866774383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c0b3fd0 0x7f688c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 --2- 172.21.15.113:0/2866774383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c0cd500 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 -- 172.21.15.113:0/2866774383 >> 172.21.15.113:0/2866774383 conn(0x7f688c0b1a50 msgr2=0x7f688c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:33.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 -- 172.21.15.113:0/2866774383 shutdown_connections 2024-04-25T00:05:33.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.141+0000 7f689327d640 1 -- 172.21.15.113:0/2866774383 wait complete. 2024-04-25T00:05:33.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 Processor -- start 2024-04-25T00:05:33.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 -- start start 2024-04-25T00:05:33.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f688c0b3fd0 0x7f688c0a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c144910 0x7f688c13fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f688c0b4da0 con 0x7f688c06e510 2024-04-25T00:05:33.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f688c0b4410 con 0x7f688c0b3fd0 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f689327d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f688c0b4f20 con 0x7f688c144910 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f6891019640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f6891019640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c14e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45638/0 (socket says 172.21.15.113:45638) 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.143+0000 7f6891019640 1 -- 172.21.15.113:0/2228970638 learned_addr learned my addr 172.21.15.113:0/2228970638 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6890818640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f688c0b3fd0 0x7f688c0a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f689181a640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c144910 0x7f688c13fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6891019640 1 -- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c144910 msgr2=0x7f688c13fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6891019640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c144910 0x7f688c13fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6891019640 1 -- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f688c0b3fd0 msgr2=0x7f688c0a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6891019640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f688c0b3fd0 0x7f688c0a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6891019640 1 -- 172.21.15.113:0/2228970638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c0aa2b0 con 0x7f688c06e510 2024-04-25T00:05:33.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6890818640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f688c0b3fd0 0x7f688c0a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f689181a640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c144910 0x7f688c13fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6891019640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c14e4d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f688800b3f0 tx=0x7f688800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6881ffb640 1 -- 172.21.15.113:0/2228970638 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888015020 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6881ffb640 1 -- 172.21.15.113:0/2228970638 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6888011830 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c14ec20 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f6881ffb640 1 -- 172.21.15.113:0/2228970638 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888011c20 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.144+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c14f1e0 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.145+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f688c0691a0 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.146+0000 7f6881ffb640 1 -- 172.21.15.113:0/2228970638 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6888018020 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.146+0000 7f6881ffb640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f687006be30 0x7f687006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.146+0000 7f6881ffb640 1 -- 172.21.15.113:0/2228970638 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7f688808a750 con 0x7f688c06e510 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.146+0000 7f6890818640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f687006be30 0x7f687006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.147+0000 7f6890818640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f687006be30 0x7f687006e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f687c006fd0 tx=0x7f687c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:33.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.150+0000 7f6881ffb640 1 -- 172.21.15.113:0/2228970638 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f688801f030 con 0x7f688c06e510 2024-04-25T00:05:33.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.326+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f688c15f6b0 con 0x7f688c06e510 2024-04-25T00:05:33.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.327+0000 7f6881ffb640 1 -- 172.21.15.113:0/2228970638 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) ==== 84+0+65 (secure 0 0 0) 0x7f688c0aa2b0 con 0x7f688c06e510 2024-04-25T00:05:33.327 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:33.327 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:05:33.327 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:05:33.328 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:05:33.328 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:05:33.328 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:33.328 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:05:33.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.328+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f687006be30 msgr2=0x7f687006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.328+0000 7f689327d640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f687006be30 0x7f687006e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f687c006fd0 tx=0x7f687c008040 comp rx=0 tx=0).stop 2024-04-25T00:05:33.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 msgr2=0x7f688c14e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c14e4d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f688800b3f0 tx=0x7f688800b8b0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 shutdown_connections 2024-04-25T00:05:33.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f687006be30 0x7f687006e2f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f688c0b3fd0 0x7f688c0a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f688c144910 0x7f688c13fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 --2- 172.21.15.113:0/2228970638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f688c06e510 0x7f688c14e4d0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 >> 172.21.15.113:0/2228970638 conn(0x7f688c0b1a50 msgr2=0x7f688c15eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:33.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 shutdown_connections 2024-04-25T00:05:33.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.329+0000 7f689327d640 1 -- 172.21.15.113:0/2228970638 wait complete. 2024-04-25T00:05:33.341 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:05:33.341 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:05:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.441+0000 7f80a91ee640 1 Processor -- start 2024-04-25T00:05:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.441+0000 7f80a91ee640 1 -- start start 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a91ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80a406e0d0 0x7f80a406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a91ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a4072f70 con 0x7f80a40cd630 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a91ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a40b2fd0 con 0x7f80a4144a80 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a91ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f80a406ea70 con 0x7f80a406e0d0 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80a406e0d0 0x7f80a406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80a406e0d0 0x7f80a406e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45648/0 (socket says 172.21.15.113:45648) 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a259c640 1 -- 172.21.15.113:0/1266095734 learned_addr learned my addr 172.21.15.113:0/1266095734 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a359e640 1 --1- 172.21.15.113:0/1266095734 >> v1:172.21.15.113:6789/0 conn(0x7f80a40cd630 0x7f80a40cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45918/0 (socket says 172.21.15.113:45918) 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a259c640 1 -- 172.21.15.113:0/1266095734 >> v1:172.21.15.178:6789/0 conn(0x7f80a4144a80 legacy=0x7f80a40b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a259c640 1 -- 172.21.15.113:0/1266095734 >> v1:172.21.15.113:6789/0 conn(0x7f80a40cd630 legacy=0x7f80a40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a259c640 1 -- 172.21.15.113:0/1266095734 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a40b32b0 con 0x7f80a406e0d0 2024-04-25T00:05:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.442+0000 7f80a1d9b640 1 -- 172.21.15.113:0/1266095734 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593999831 0 0) 0x7f80a4072f70 con 0x7f80a40cd630 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.443+0000 7f80a259c640 1 --2- 172.21.15.113:0/1266095734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a406e0d0 0x7f80a406e530 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f8098009870 tx=0x7f809802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc11624042513cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.443+0000 7f80a1d9b640 1 -- 172.21.15.113:0/1266095734 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098037070 con 0x7f80a406e0d0 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.443+0000 7f80a1d9b640 1 -- 172.21.15.113:0/1266095734 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8098032440 con 0x7f80a406e0d0 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.443+0000 7f80a1d9b640 1 -- 172.21.15.113:0/1266095734 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098032810 con 0x7f80a406e0d0 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.444+0000 7f80a91ee640 1 -- 172.21.15.113:0/1266095734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a406e0d0 msgr2=0x7f80a406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.444+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1266095734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a406e0d0 0x7f80a406e530 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f8098009870 tx=0x7f809802a210 comp rx=0 tx=0).stop 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.444+0000 7f80a91ee640 1 -- 172.21.15.113:0/1266095734 shutdown_connections 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.444+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1266095734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a406e0d0 0x7f80a406e530 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.444+0000 7f80a91ee640 1 -- 172.21.15.113:0/1266095734 >> 172.21.15.113:0/1266095734 conn(0x7f80a40b1b30 msgr2=0x7f80a40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.444+0000 7f80a91ee640 1 -- 172.21.15.113:0/1266095734 shutdown_connections 2024-04-25T00:05:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.444+0000 7f80a91ee640 1 -- 172.21.15.113:0/1266095734 wait complete. 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.445+0000 7f80a91ee640 1 Processor -- start 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.445+0000 7f80a91ee640 1 -- start start 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a91ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80a406e0d0 0x7f80a413a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a91ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a40cd630 0x7f80a41392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a91ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80a4144a80 0x7f80a414ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a91ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f80a40b4340 con 0x7f80a40cd630 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a91ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80a40b41c0 con 0x7f80a406e0d0 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a91ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f80a4072200 con 0x7f80a4144a80 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a40cd630 0x7f80a41392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a40cd630 0x7f80a41392b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45664/0 (socket says 172.21.15.113:45664) 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a259c640 1 -- 172.21.15.113:0/1566782243 learned_addr learned my addr 172.21.15.113:0/1566782243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a359e640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80a4144a80 0x7f80a414ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.446+0000 7f80a2d9d640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80a406e0d0 0x7f80a413a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a259c640 1 -- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80a4144a80 msgr2=0x7f80a414ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a259c640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80a4144a80 0x7f80a414ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a259c640 1 -- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80a406e0d0 msgr2=0x7f80a413a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a259c640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80a406e0d0 0x7f80a413a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a259c640 1 -- 172.21.15.113:0/1566782243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a415e1f0 con 0x7f80a40cd630 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a2d9d640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80a406e0d0 0x7f80a413a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a359e640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80a4144a80 0x7f80a414ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a259c640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a40cd630 0x7f80a41392b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8098009870 tx=0x7f80980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f808b7fe640 1 -- 172.21.15.113:0/1566782243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098037040 con 0x7f80a40cd630 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f808b7fe640 1 -- 172.21.15.113:0/1566782243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8098004250 con 0x7f80a40cd630 2024-04-25T00:05:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a413ef80 con 0x7f80a40cd630 2024-04-25T00:05:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f808b7fe640 1 -- 172.21.15.113:0/1566782243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809803c670 con 0x7f80a40cd630 2024-04-25T00:05:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.447+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a4139f10 con 0x7f80a40cd630 2024-04-25T00:05:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.448+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a40691a0 con 0x7f80a40cd630 2024-04-25T00:05:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.448+0000 7f808b7fe640 1 -- 172.21.15.113:0/1566782243 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f809803a070 con 0x7f80a40cd630 2024-04-25T00:05:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.449+0000 7f808b7fe640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f809006be60 0x7f809006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.449+0000 7f808b7fe640 1 -- 172.21.15.113:0/1566782243 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7f80980ab450 con 0x7f80a40cd630 2024-04-25T00:05:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.449+0000 7f80a2d9d640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f809006be60 0x7f809006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.449+0000 7f80a2d9d640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f809006be60 0x7f809006e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8094007e90 tx=0x7f80940073a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.453+0000 7f808b7fe640 1 -- 172.21.15.113:0/1566782243 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8098041020 con 0x7f80a40cd630 2024-04-25T00:05:33.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:33.624+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f80a40cc2d0 con 0x7f80a40cd630 2024-04-25T00:05:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.073+0000 7f808b7fe640 1 -- 172.21.15.113:0/1566782243 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) ==== 138+0+0 (secure 0 0 0) 0x7f809807a150 con 0x7f80a40cd630 2024-04-25T00:05:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.079+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f809006be60 msgr2=0x7f809006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.079+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f809006be60 0x7f809006e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8094007e90 tx=0x7f80940073a0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a40cd630 msgr2=0x7f80a41392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a40cd630 0x7f80a41392b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8098009870 tx=0x7f80980046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 shutdown_connections 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f809006be60 0x7f809006e320 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80a4144a80 0x7f80a414ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a40cd630 0x7f80a41392b0 secure :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8098009870 tx=0x7f80980046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 --2- 172.21.15.113:0/1566782243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80a406e0d0 0x7f80a413a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.080+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 >> 172.21.15.113:0/1566782243 conn(0x7f80a40b1b30 msgr2=0x7f80a4142070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.082+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 shutdown_connections 2024-04-25T00:05:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.083+0000 7f80a91ee640 1 -- 172.21.15.113:0/1566782243 wait complete. 2024-04-25T00:05:34.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:05:34.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.205+0000 7feec751b640 1 Processor -- start 2024-04-25T00:05:34.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.205+0000 7feec751b640 1 -- start start 2024-04-25T00:05:34.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec751b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 0x7feec015c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec751b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feec0074c00 0x7feec00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec751b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec0160900 con 0x7feec006f180 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec751b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7feec006d870 con 0x7feec0074c00 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec751b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feec006db50 con 0x7feec015c050 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec5ab8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feec006f180 0x7feec006e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45920/0 (socket says 172.21.15.113:45920) 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec5ab8640 1 -- 172.21.15.113:0/534661035 learned_addr learned my addr 172.21.15.113:0/534661035 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751076675 0 0) 0x7feec0160900 con 0x7feec006f180 2024-04-25T00:05:34.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeac003520 con 0x7feec006f180 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec52b7640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 0x7feec015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.206+0000 7feec4ab6640 1 --2- 172.21.15.113:0/534661035 >> v2:172.21.15.178:3300/0 conn(0x7feec0074c00 0x7feec00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2685658473 0 0) 0x7feeac003520 con 0x7feec006f180 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 msgr2=0x7feec015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 0x7feec015c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 >> v2:172.21.15.178:3300/0 conn(0x7feec0074c00 msgr2=0x7feec00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 --2- 172.21.15.113:0/534661035 >> v2:172.21.15.178:3300/0 conn(0x7feec0074c00 0x7feec00cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec0160be0 con 0x7feec006f180 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7feeb80019f0 con 0x7feec006f180 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 >> v1:172.21.15.113:6789/0 conn(0x7feec006f180 legacy=0x7feec006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 0x7feeac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeac003f30 0x7feeac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeac004d40 0x7feeac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feec006d870 con 0x7feeac004d40 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feec006db50 con 0x7feeac003f30 2024-04-25T00:05:34.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feeac00ba10 con 0x7feec015c050 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec5ab8640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeac004d40 0x7feeac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec4ab6640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeac003f30 0x7feeac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec52b7640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 0x7feeac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec5ab8640 1 -- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 msgr2=0x7feeac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec5ab8640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 0x7feeac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec5ab8640 1 -- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeac003f30 msgr2=0x7feeac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec5ab8640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeac003f30 0x7feeac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec5ab8640 1 -- 172.21.15.113:0/534661035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac003520 con 0x7feeac004d40 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.207+0000 7feec52b7640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec015c050 0x7feeac0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:34.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.208+0000 7feec5ab8640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeac004d40 0x7feeac0051c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7feeb80013d0 tx=0x7feeac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bb593170ce82f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.208+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb800a4e0 con 0x7feeac004d40 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.208+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb802f070 con 0x7feeac004d40 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.208+0000 7feeb7fff640 1 -- 172.21.15.113:0/534661035 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb802bce0 con 0x7feeac004d40 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.209+0000 7feec751b640 1 -- 172.21.15.113:0/534661035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeac004d40 msgr2=0x7feeac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.209+0000 7feec751b640 1 --2- 172.21.15.113:0/534661035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feeac004d40 0x7feeac0051c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7feeb80013d0 tx=0x7feeac005830 comp rx=0 tx=0).stop 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.209+0000 7feec5ab8640 1 -- 172.21.15.113:0/534661035 reap_dead start 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.209+0000 7feec751b640 1 -- 172.21.15.113:0/534661035 shutdown_connections 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.209+0000 7feec751b640 1 -- 172.21.15.113:0/534661035 >> 172.21.15.113:0/534661035 conn(0x7feec006fb60 msgr2=0x7feec006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.209+0000 7feec751b640 1 -- 172.21.15.113:0/534661035 shutdown_connections 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.209+0000 7feec751b640 1 -- 172.21.15.113:0/534661035 wait complete. 2024-04-25T00:05:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 Processor -- start 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 -- start start 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeac004d40 0x7feec0151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec0155260 0x7feec0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feec0158880 0x7feec0145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feec00cfab0 con 0x7feec0158880 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feec00cf930 con 0x7feec0155260 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.210+0000 7feec751b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feec00cfc30 con 0x7feeac004d40 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feec0158880 0x7feec0145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feec0158880 0x7feec0145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45682/0 (socket says 172.21.15.113:45682) 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 -- 172.21.15.113:0/2276873226 learned_addr learned my addr 172.21.15.113:0/2276873226 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec52b7640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeac004d40 0x7feec0151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec4ab6640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec0155260 0x7feec0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 -- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeac004d40 msgr2=0x7feec0151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeac004d40 0x7feec0151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 -- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec0155260 msgr2=0x7feec0156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec0155260 0x7feec0156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 -- 172.21.15.113:0/2276873226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec0156d40 con 0x7feec0158880 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec4ab6640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec0155260 0x7feec0156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec5ab8640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feec0158880 0x7feec0145c10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7feeb800a0b0 tx=0x7feeb8001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feeb5ffb640 1 -- 172.21.15.113:0/2276873226 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0009690 con 0x7feec0158880 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feeb5ffb640 1 -- 172.21.15.113:0/2276873226 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb800a4c0 con 0x7feec0158880 2024-04-25T00:05:34.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feeb5ffb640 1 -- 172.21.15.113:0/2276873226 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8002040 con 0x7feec0158880 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec0155760 con 0x7feec0158880 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.211+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec0155d20 con 0x7feec0158880 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.213+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec00691a0 con 0x7feec0158880 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.213+0000 7feeb5ffb640 1 -- 172.21.15.113:0/2276873226 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feeb8034470 con 0x7feec0158880 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.213+0000 7feeb5ffb640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.213+0000 7feeb5ffb640 1 -- 172.21.15.113:0/2276873226 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5227+0+0 (secure 0 0 0) 0x7feeb80aa460 con 0x7feec0158880 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.213+0000 7feec52b7640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:34.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.214+0000 7feec52b7640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009d60 tx=0x7feeb0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:34.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.218+0000 7feeb5ffb640 1 -- 172.21.15.113:0/2276873226 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feeb8047070 con 0x7feec0158880 2024-04-25T00:05:34.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:34.393+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7feec00b2e60 con 0x7feec0158880 2024-04-25T00:05:35.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.075+0000 7feeb5ffb640 1 -- 172.21.15.113:0/2276873226 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) ==== 170+0+0 (secure 0 0 0) 0x7feeb8079080 con 0x7feec0158880 2024-04-25T00:05:35.076 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:05:35.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.081+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 msgr2=0x7feea406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:35.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.081+0000 7feec751b640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009d60 tx=0x7feeb0009190 comp rx=0 tx=0).stop 2024-04-25T00:05:35.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.081+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feec0158880 msgr2=0x7feec0145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:35.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.081+0000 7feec751b640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feec0158880 0x7feec0145c10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7feeb800a0b0 tx=0x7feeb8001dc0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.084+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 shutdown_connections 2024-04-25T00:05:35.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.084+0000 7feec751b640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feea406be30 0x7feea406e2f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.084+0000 7feec751b640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec0155260 0x7feec0156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.084+0000 7feec751b640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feec0158880 0x7feec0145c10 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.084+0000 7feec751b640 1 --2- 172.21.15.113:0/2276873226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeac004d40 0x7feec0151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.084+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 >> 172.21.15.113:0/2276873226 conn(0x7feec006fb60 msgr2=0x7feec006e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:35.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.086+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 shutdown_connections 2024-04-25T00:05:35.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.086+0000 7feec751b640 1 -- 172.21.15.113:0/2276873226 wait complete. 2024-04-25T00:05:35.106 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:05:35.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.206+0000 7fa44e110640 1 Processor -- start 2024-04-25T00:05:35.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.206+0000 7fa44e110640 1 -- start start 2024-04-25T00:05:35.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa44e110640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:35.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa44e110640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4480b63b0 0x7fa4480b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:35.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa44e110640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4480d2e00 con 0x7fa4480d28c0 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa44e110640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa4480d30e0 con 0x7fa4480b63b0 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa44e110640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480d33c0 con 0x7fa44815e510 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa447fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4480b63b0 0x7fa4480b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa447fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4480b63b0 0x7fa4480b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45698/0 (socket says 172.21.15.113:45698) 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa447fff640 1 -- 172.21.15.113:0/2310202046 learned_addr learned my addr 172.21.15.113:0/2310202046 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.207+0000 7fa43effd640 1 --1- 172.21.15.113:0/2310202046 >> v1:172.21.15.113:6789/0 conn(0x7fa4480d28c0 0x7fa4480b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45934/0 (socket says 172.21.15.113:45934) 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa4477fe640 1 --2- 172.21.15.113:0/2310202046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa447fff640 1 -- 172.21.15.113:0/2310202046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 msgr2=0x7fa4480d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa447fff640 1 --2- 172.21.15.113:0/2310202046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa447fff640 1 -- 172.21.15.113:0/2310202046 >> v1:172.21.15.113:6789/0 conn(0x7fa4480d28c0 legacy=0x7fa4480b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa447fff640 1 -- 172.21.15.113:0/2310202046 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4480d36a0 con 0x7fa4480b63b0 2024-04-25T00:05:35.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa446ffd640 1 -- 172.21.15.113:0/2310202046 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1397374197 0 0) 0x7fa4480d2e00 con 0x7fa4480d28c0 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa4477fe640 1 --2- 172.21.15.113:0/2310202046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa447fff640 1 --2- 172.21.15.113:0/2310202046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa4480b73c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa440009e50 tx=0x7fa44002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78303682e9d68e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.208+0000 7fa446ffd640 1 -- 172.21.15.113:0/2310202046 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440036070 con 0x7fa4480b63b0 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa446ffd640 1 -- 172.21.15.113:0/2310202046 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa440031600 con 0x7fa4480b63b0 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa446ffd640 1 -- 172.21.15.113:0/2310202046 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400319d0 con 0x7fa4480b63b0 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 -- 172.21.15.113:0/2310202046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 msgr2=0x7fa4480b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 --2- 172.21.15.113:0/2310202046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa4480b73c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa440009e50 tx=0x7fa44002a680 comp rx=0 tx=0).stop 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 -- 172.21.15.113:0/2310202046 shutdown_connections 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 --2- 172.21.15.113:0/2310202046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa4480b73c0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 --2- 172.21.15.113:0/2310202046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 -- 172.21.15.113:0/2310202046 >> 172.21.15.113:0/2310202046 conn(0x7fa44806b720 msgr2=0x7fa4480d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 -- 172.21.15.113:0/2310202046 shutdown_connections 2024-04-25T00:05:35.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.209+0000 7fa44e110640 1 -- 172.21.15.113:0/2310202046 wait complete. 2024-04-25T00:05:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.210+0000 7fa44e110640 1 Processor -- start 2024-04-25T00:05:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.210+0000 7fa44e110640 1 -- start start 2024-04-25T00:05:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa44e110640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa44815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa44e110640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4480d28c0 0x7fa448071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa44e110640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa44e110640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480b9dd0 con 0x7fa4480b63b0 2024-04-25T00:05:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa44e110640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480d3ea0 con 0x7fa4480d28c0 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa44e110640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480b7900 con 0x7fa44815e510 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa44815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa44815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45714/0 (socket says 172.21.15.113:45714) 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 -- 172.21.15.113:0/3800666042 learned_addr learned my addr 172.21.15.113:0/3800666042 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 -- 172.21.15.113:0/3800666042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 msgr2=0x7fa4480b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 -- 172.21.15.113:0/3800666042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4480d28c0 msgr2=0x7fa448071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4480d28c0 0x7fa448071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.211+0000 7fa4477fe640 1 -- 172.21.15.113:0/3800666042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4480b4760 con 0x7fa4480b63b0 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.212+0000 7fa4477fe640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa44815fc60 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa43800d7d0 tx=0x7fa43800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.212+0000 7fa444ff9640 1 -- 172.21.15.113:0/3800666042 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438004030 con 0x7fa4480b63b0 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.212+0000 7fa444ff9640 1 -- 172.21.15.113:0/3800666042 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4380041d0 con 0x7fa4480b63b0 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.212+0000 7fa444ff9640 1 -- 172.21.15.113:0/3800666042 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380096a0 con 0x7fa4480b63b0 2024-04-25T00:05:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.212+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4480b4a40 con 0x7fa4480b63b0 2024-04-25T00:05:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.212+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4480cb7b0 con 0x7fa4480b63b0 2024-04-25T00:05:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.213+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4480691a0 con 0x7fa4480b63b0 2024-04-25T00:05:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.213+0000 7fa444ff9640 1 -- 172.21.15.113:0/3800666042 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa438009840 con 0x7fa4480b63b0 2024-04-25T00:05:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.214+0000 7fa444ff9640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa42806be30 0x7fa42806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.214+0000 7fa444ff9640 1 -- 172.21.15.113:0/3800666042 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5609+0+0 (secure 0 0 0) 0x7fa43808b0c0 con 0x7fa4480b63b0 2024-04-25T00:05:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.214+0000 7fa43effd640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa42806be30 0x7fa42806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.214+0000 7fa43effd640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa42806be30 0x7fa42806e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa430007fd0 tx=0x7fa430007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:35.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.218+0000 7fa444ff9640 1 -- 172.21.15.113:0/3800666042 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa438059ad0 con 0x7fa4480b63b0 2024-04-25T00:05:35.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:35.430+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa4480b4c10 con 0x7fa4480b63b0 2024-04-25T00:05:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.090+0000 7fa444ff9640 1 -- 172.21.15.113:0/3800666042 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v60) ==== 174+0+0 (secure 0 0 0) 0x7fa4380593d0 con 0x7fa4480b63b0 2024-04-25T00:05:36.091 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T00:05:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.093+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa42806be30 msgr2=0x7fa42806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.093+0000 7fa44e110640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa42806be30 0x7fa42806e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa430007fd0 tx=0x7fa430007900 comp rx=0 tx=0).stop 2024-04-25T00:05:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.093+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 msgr2=0x7fa44815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.093+0000 7fa44e110640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa44815fc60 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa43800d7d0 tx=0x7fa43800dc90 comp rx=0 tx=0).stop 2024-04-25T00:05:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.095+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 shutdown_connections 2024-04-25T00:05:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.095+0000 7fa44e110640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa42806be30 0x7fa42806e2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.095+0000 7fa44e110640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4480d28c0 0x7fa448071980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.095+0000 7fa44e110640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa44815e510 0x7fa4480b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.095+0000 7fa44e110640 1 --2- 172.21.15.113:0/3800666042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4480b63b0 0x7fa44815fc60 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.095+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 >> 172.21.15.113:0/3800666042 conn(0x7fa44806b720 msgr2=0x7fa4480b8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:36.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.101+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 shutdown_connections 2024-04-25T00:05:36.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.102+0000 7fa44e110640 1 -- 172.21.15.113:0/3800666042 wait complete. 2024-04-25T00:05:36.127 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:36.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.224+0000 7f0f487b6640 1 Processor -- start 2024-04-25T00:05:36.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.224+0000 7f0f487b6640 1 -- start start 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f487b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f4006e6e0 0x7f0f400b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f487b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f400cfad0 con 0x7f0f401453c0 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f487b6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f400cf230 con 0x7f0f40144d60 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f487b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0f40145c40 con 0x7f0f4006e6e0 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f45d51640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f4006e6e0 0x7f0f400b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f45d51640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f4006e6e0 0x7f0f400b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45728/0 (socket says 172.21.15.113:45728) 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f45d51640 1 -- 172.21.15.113:0/1924748062 learned_addr learned my addr 172.21.15.113:0/1924748062 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:36.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f45d51640 1 -- 172.21.15.113:0/1924748062 >> v1:172.21.15.178:6789/0 conn(0x7f0f40144d60 legacy=0x7f0f4006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f45d51640 1 -- 172.21.15.113:0/1924748062 >> v1:172.21.15.113:6789/0 conn(0x7f0f401453c0 legacy=0x7f0f40071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f45550640 1 -- 172.21.15.113:0/1924748062 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3754018219 0 0) 0x7f0f400cfad0 con 0x7f0f401453c0 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.225+0000 7f0f45d51640 1 -- 172.21.15.113:0/1924748062 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f400cf510 con 0x7f0f4006e6e0 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.226+0000 7f0f45d51640 1 --2- 172.21.15.113:0/1924748062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f400b5390 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009870 tx=0x7f0f3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96e673738be5a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.226+0000 7f0f45550640 1 -- 172.21.15.113:0/1924748062 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30037070 con 0x7f0f4006e6e0 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.226+0000 7f0f45550640 1 -- 172.21.15.113:0/1924748062 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30032440 con 0x7f0f4006e6e0 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.226+0000 7f0f45550640 1 -- 172.21.15.113:0/1924748062 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30032810 con 0x7f0f4006e6e0 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.227+0000 7f0f487b6640 1 -- 172.21.15.113:0/1924748062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 msgr2=0x7f0f400b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.227+0000 7f0f487b6640 1 --2- 172.21.15.113:0/1924748062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f400b5390 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009870 tx=0x7f0f3002a180 comp rx=0 tx=0).stop 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.227+0000 7f0f487b6640 1 -- 172.21.15.113:0/1924748062 shutdown_connections 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.227+0000 7f0f487b6640 1 --2- 172.21.15.113:0/1924748062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f400b5390 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.227+0000 7f0f487b6640 1 -- 172.21.15.113:0/1924748062 >> 172.21.15.113:0/1924748062 conn(0x7f0f400b1a50 msgr2=0x7f0f400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:36.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.227+0000 7f0f487b6640 1 -- 172.21.15.113:0/1924748062 shutdown_connections 2024-04-25T00:05:36.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.227+0000 7f0f487b6640 1 -- 172.21.15.113:0/1924748062 wait complete. 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 Processor -- start 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 -- start start 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f401f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f40144d60 0x7f0f400ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f401453c0 0x7f0f400ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f40145140 con 0x7f0f4006e6e0 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f40070e10 con 0x7f0f40144d60 2024-04-25T00:05:36.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.228+0000 7f0f487b6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f40071a40 con 0x7f0f401453c0 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f401f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f401f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45732/0 (socket says 172.21.15.113:45732) 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 -- 172.21.15.113:0/4290981265 learned_addr learned my addr 172.21.15.113:0/4290981265 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f45d51640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f40144d60 0x7f0f400ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 -- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f401453c0 msgr2=0x7f0f400ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46d53640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f401453c0 0x7f0f400ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f401453c0 0x7f0f400ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 -- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f40144d60 msgr2=0x7f0f400ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f40144d60 0x7f0f400ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 -- 172.21.15.113:0/4290981265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f401e92b0 con 0x7f0f4006e6e0 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f46552640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f401f6ec0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c460 tx=0x7f0f3c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f36ffd640 1 -- 172.21.15.113:0/4290981265 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c016020 con 0x7f0f4006e6e0 2024-04-25T00:05:36.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f36ffd640 1 -- 172.21.15.113:0/4290981265 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c012870 con 0x7f0f4006e6e0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f401e9590 con 0x7f0f4006e6e0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f36ffd640 1 -- 172.21.15.113:0/4290981265 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c012c60 con 0x7f0f4006e6e0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.229+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f401e9b50 con 0x7f0f4006e6e0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.230+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f400691a0 con 0x7f0f4006e6e0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.231+0000 7f0f36ffd640 1 -- 172.21.15.113:0/4290981265 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f3c019020 con 0x7f0f4006e6e0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.231+0000 7f0f36ffd640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f2406be30 0x7f0f2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.231+0000 7f0f36ffd640 1 -- 172.21.15.113:0/4290981265 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3c08ae60 con 0x7f0f4006e6e0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.231+0000 7f0f45d51640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f2406be30 0x7f0f2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.231+0000 7f0f45d51640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f2406be30 0x7f0f2406e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0f30034000 tx=0x7f0f3002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.235+0000 7f0f36ffd640 1 -- 172.21.15.113:0/4290981265 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f3c020020 con 0x7f0f4006e6e0 2024-04-25T00:05:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.410+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f40071700 con 0x7f0f4006e6e0 2024-04-25T00:05:36.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.411+0000 7f0f36ffd640 1 -- 172.21.15.113:0/4290981265 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16688 (secure 0 0 0) 0x7f0f3c0182c0 con 0x7f0f4006e6e0 2024-04-25T00:05:36.412 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":115,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:05:35.432158+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:05:33.644000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:05:34.412894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f2406be30 msgr2=0x7f0f2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f2406be30 0x7f0f2406e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0f30034000 tx=0x7f0f3002e000 comp rx=0 tx=0).stop 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 msgr2=0x7f0f401f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f401f6ec0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c460 tx=0x7f0f3c00c920 comp rx=0 tx=0).stop 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 shutdown_connections 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f2406be30 0x7f0f2406e2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f40144d60 0x7f0f400ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f401453c0 0x7f0f400ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 --2- 172.21.15.113:0/4290981265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f4006e6e0 0x7f0f401f6ec0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 >> 172.21.15.113:0/4290981265 conn(0x7f0f400b1a50 msgr2=0x7f0f400b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:36.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 shutdown_connections 2024-04-25T00:05:36.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.413+0000 7f0f487b6640 1 -- 172.21.15.113:0/4290981265 wait complete. 2024-04-25T00:05:36.425 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:36.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.522+0000 7fe1dd212640 1 Processor -- start 2024-04-25T00:05:36.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.522+0000 7fe1dd212640 1 -- start start 2024-04-25T00:05:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1dd212640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1d815e320 0x7fe1d80d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1dd212640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1d80d2290 0x7fe1d80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1dd212640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1dd212640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d80d14d0 con 0x7fe1d815e320 2024-04-25T00:05:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1dd212640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d80d3410 con 0x7fe1d80d2290 2024-04-25T00:05:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1dd212640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d80d3de0 con 0x7fe1d80d0b00 2024-04-25T00:05:36.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1d6d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1d815e320 0x7fe1d80d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1d6d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1d815e320 0x7fe1d80d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45736/0 (socket says 172.21.15.113:45736) 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1d6d9d640 1 -- 172.21.15.113:0/1815460179 learned_addr learned my addr 172.21.15.113:0/1815460179 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1d759e640 1 --2- 172.21.15.113:0/1815460179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.523+0000 7fe1d659c640 1 --2- 172.21.15.113:0/1815460179 >> v2:172.21.15.178:3300/0 conn(0x7fe1d80d2290 0x7fe1d80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d6d9d640 1 -- 172.21.15.113:0/1815460179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 msgr2=0x7fe1d80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d6d9d640 1 --2- 172.21.15.113:0/1815460179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d6d9d640 1 -- 172.21.15.113:0/1815460179 >> v2:172.21.15.178:3300/0 conn(0x7fe1d80d2290 msgr2=0x7fe1d80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d6d9d640 1 --2- 172.21.15.113:0/1815460179 >> v2:172.21.15.178:3300/0 conn(0x7fe1d80d2290 0x7fe1d80b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d6d9d640 1 -- 172.21.15.113:0/1815460179 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d80d2800 con 0x7fe1d815e320 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d759e640 1 --2- 172.21.15.113:0/1815460179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d659c640 1 --2- 172.21.15.113:0/1815460179 >> v2:172.21.15.178:3300/0 conn(0x7fe1d80d2290 0x7fe1d80b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d6d9d640 1 --2- 172.21.15.113:0/1815460179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d815e320 0x7fe1d80d38a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8072810 tx=0x7fe1c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633d69389c9a01df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d5d9b640 1 -- 172.21.15.113:0/1815460179 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8036070 con 0x7fe1d815e320 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d5d9b640 1 -- 172.21.15.113:0/1815460179 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c8031600 con 0x7fe1d815e320 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.524+0000 7fe1d5d9b640 1 -- 172.21.15.113:0/1815460179 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80319d0 con 0x7fe1d815e320 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 -- 172.21.15.113:0/1815460179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d815e320 msgr2=0x7fe1d80d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 --2- 172.21.15.113:0/1815460179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d815e320 0x7fe1d80d38a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8072810 tx=0x7fe1c802a730 comp rx=0 tx=0).stop 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 -- 172.21.15.113:0/1815460179 shutdown_connections 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 --2- 172.21.15.113:0/1815460179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 --2- 172.21.15.113:0/1815460179 >> v2:172.21.15.178:3300/0 conn(0x7fe1d80d2290 0x7fe1d80b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 --2- 172.21.15.113:0/1815460179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d815e320 0x7fe1d80d38a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 -- 172.21.15.113:0/1815460179 >> 172.21.15.113:0/1815460179 conn(0x7fe1d80b5c20 msgr2=0x7fe1d80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 -- 172.21.15.113:0/1815460179 shutdown_connections 2024-04-25T00:05:36.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.525+0000 7fe1dd212640 1 -- 172.21.15.113:0/1815460179 wait complete. 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.526+0000 7fe1dd212640 1 Processor -- start 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.526+0000 7fe1dd212640 1 -- start start 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1dd212640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d8155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1dd212640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d80d2290 0x7fe1d815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1dd212640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d815e320 0x7fe1d81463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1dd212640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d80d30a0 con 0x7fe1d80d2290 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1dd212640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d80d2dc0 con 0x7fe1d815e320 2024-04-25T00:05:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1dd212640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d80b9fe0 con 0x7fe1d80d0b00 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d80d2290 0x7fe1d815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d80d2290 0x7fe1d815eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45738/0 (socket says 172.21.15.113:45738) 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d659c640 1 -- 172.21.15.113:0/4239856541 learned_addr learned my addr 172.21.15.113:0/4239856541 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d759e640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d815e320 0x7fe1d81463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d6d9d640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d8155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d659c640 1 -- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 msgr2=0x7fe1d8155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d659c640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d8155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d659c640 1 -- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d815e320 msgr2=0x7fe1d81463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.527+0000 7fe1d659c640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d815e320 0x7fe1d81463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.528+0000 7fe1d659c640 1 -- 172.21.15.113:0/4239856541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d81566d0 con 0x7fe1d80d2290 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.528+0000 7fe1d659c640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d80d2290 0x7fe1d815eb60 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c7f0 tx=0x7fe1cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.528+0000 7fe1bf7fe640 1 -- 172.21.15.113:0/4239856541 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc015070 con 0x7fe1d80d2290 2024-04-25T00:05:36.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.528+0000 7fe1bf7fe640 1 -- 172.21.15.113:0/4239856541 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc007a30 con 0x7fe1d80d2290 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.528+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d81569b0 con 0x7fe1d80d2290 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.528+0000 7fe1bf7fe640 1 -- 172.21.15.113:0/4239856541 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc007e00 con 0x7fe1d80d2290 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.528+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d8156f70 con 0x7fe1d80d2290 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.529+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d80691a0 con 0x7fe1d80d2290 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.529+0000 7fe1bf7fe640 1 -- 172.21.15.113:0/4239856541 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1cc018070 con 0x7fe1d80d2290 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.529+0000 7fe1bf7fe640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.530+0000 7fe1d6d9d640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.530+0000 7fe1bf7fe640 1 -- 172.21.15.113:0/4239856541 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7fe1cc08b990 con 0x7fe1d80d2290 2024-04-25T00:05:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.530+0000 7fe1d6d9d640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe1c406be30 0x7fe1c406e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8033000 tx=0x7fe1c8009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.534+0000 7fe1bf7fe640 1 -- 172.21.15.113:0/4239856541 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1cc059800 con 0x7fe1d80d2290 2024-04-25T00:05:36.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.707+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1d80b4740 con 0x7fe1d80d2290 2024-04-25T00:05:36.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.708+0000 7fe1bf7fe640 1 -- 172.21.15.113:0/4239856541 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16688 (secure 0 0 0) 0x7fe1cc059100 con 0x7fe1d80d2290 2024-04-25T00:05:36.708 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":115,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:05:35.432158+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:05:33.644000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:05:34.412894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:36.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe1c406be30 msgr2=0x7fe1c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe1c406be30 0x7fe1c406e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8033000 tx=0x7fe1c8009000 comp rx=0 tx=0).stop 2024-04-25T00:05:36.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d80d2290 msgr2=0x7fe1d815eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d80d2290 0x7fe1d815eb60 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c7f0 tx=0x7fe1cc00cbf0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 shutdown_connections 2024-04-25T00:05:36.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d815e320 0x7fe1d81463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1d80d2290 0x7fe1d815eb60 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 --2- 172.21.15.113:0/4239856541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1d80d0b00 0x7fe1d8155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 >> 172.21.15.113:0/4239856541 conn(0x7fe1d80b5c20 msgr2=0x7fe1d80b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:36.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 shutdown_connections 2024-04-25T00:05:36.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.710+0000 7fe1dd212640 1 -- 172.21.15.113:0/4239856541 wait complete. 2024-04-25T00:05:36.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:05:36.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.821+0000 7f568a97e640 1 Processor -- start 2024-04-25T00:05:36.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.821+0000 7f568a97e640 1 -- start start 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f568a97e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 0x7f568415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f568a97e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5684071170 con 0x7f568415b670 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f568a97e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5684071450 con 0x7f568406cea0 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f568a97e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5684071730 con 0x7f568415ba50 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5683fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f568415b670 0x7f56840cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45952/0 (socket says 172.21.15.113:45952) 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5683fff640 1 -- 172.21.15.113:0/707326743 learned_addr learned my addr 172.21.15.113:0/707326743 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3211532629 0 0) 0x7f5684071170 con 0x7f568415b670 2024-04-25T00:05:36.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f566c003520 con 0x7f568415b670 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f56837fe640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 0x7f568415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2627035200 0 0) 0x7f566c003520 con 0x7f568415b670 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 msgr2=0x7f568415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 0x7f568415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 >> v1:172.21.15.178:6789/0 conn(0x7f568406cea0 legacy=0x7f5684072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5684071a10 con 0x7f568415b670 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f5674001670 con 0x7f568415b670 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f56837fe640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 0x7f568415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 >> v1:172.21.15.113:6789/0 conn(0x7f568415b670 legacy=0x7f56840cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c0039e0 0x7f566c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 0x7f566c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f566c004d40 0x7f566c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5684071730 con 0x7f566c004d40 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f566c00bc20 con 0x7f566c0039e0 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.822+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f566c00cf30 con 0x7f568415ba50 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5688f1b640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f566c004d40 0x7f566c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5683fff640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c0039e0 0x7f566c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f56837fe640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 0x7f566c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5688f1b640 1 -- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 msgr2=0x7f566c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5688f1b640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568415ba50 0x7f566c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5688f1b640 1 -- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c0039e0 msgr2=0x7f566c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5688f1b640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f566c0039e0 0x7f566c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5688f1b640 1 -- 172.21.15.113:0/707326743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c003520 con 0x7f566c004d40 2024-04-25T00:05:36.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.823+0000 7f5688f1b640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f566c004d40 0x7f566c0051c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f567c001e50 tx=0x7f566c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb59caacfad38c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c001a70 con 0x7f566c004d40 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c003970 con 0x7f566c004d40 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f5682ffd640 1 -- 172.21.15.113:0/707326743 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c003d60 con 0x7f566c004d40 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f568a97e640 1 -- 172.21.15.113:0/707326743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f566c004d40 msgr2=0x7f566c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f568a97e640 1 --2- 172.21.15.113:0/707326743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f566c004d40 0x7f566c0051c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f567c001e50 tx=0x7f566c005830 comp rx=0 tx=0).stop 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f5688f1b640 1 -- 172.21.15.113:0/707326743 reap_dead start 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f568a97e640 1 -- 172.21.15.113:0/707326743 shutdown_connections 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f568a97e640 1 -- 172.21.15.113:0/707326743 >> 172.21.15.113:0/707326743 conn(0x7f568406b0f0 msgr2=0x7f568406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f568a97e640 1 -- 172.21.15.113:0/707326743 shutdown_connections 2024-04-25T00:05:36.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.824+0000 7f568a97e640 1 -- 172.21.15.113:0/707326743 wait complete. 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.825+0000 7f568a97e640 1 Processor -- start 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f568a97e640 1 -- start start 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f568a97e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568413cbe0 0x7f568413a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f568a97e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568413be70 0x7f568413cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f568a97e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f568414ec40 0x7f568414f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f568a97e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f568415c170 con 0x7f568413cbe0 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f568a97e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f568415bff0 con 0x7f568414ec40 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f568a97e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f568415c2f0 con 0x7f568413be70 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f5683fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568413cbe0 0x7f568413a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f5683fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568413cbe0 0x7f568413a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45760/0 (socket says 172.21.15.113:45760) 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.826+0000 7f5683fff640 1 -- 172.21.15.113:0/1591780516 learned_addr learned my addr 172.21.15.113:0/1591780516 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5683fff640 1 -- 172.21.15.113:0/1591780516 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568413be70 msgr2=0x7f568413cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5683fff640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568413be70 0x7f568413cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5683fff640 1 -- 172.21.15.113:0/1591780516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f568414ec40 msgr2=0x7f568414f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5683fff640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f568414ec40 0x7f568414f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5683fff640 1 -- 172.21.15.113:0/1591780516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568414f6a0 con 0x7f568413cbe0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5683fff640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568413cbe0 0x7f568413a660 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f56740072c0 tx=0x7f5674007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5680ff9640 1 -- 172.21.15.113:0/1591780516 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567401a070 con 0x7f568413cbe0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5680ff9640 1 -- 172.21.15.113:0/1591780516 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567400f860 con 0x7f568413cbe0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568413add0 con 0x7f568413cbe0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f5680ff9640 1 -- 172.21.15.113:0/1591780516 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567400fc50 con 0x7f568413cbe0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.827+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568413b390 con 0x7f568413cbe0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.828+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56840691a0 con 0x7f568413cbe0 2024-04-25T00:05:36.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.828+0000 7f5680ff9640 1 -- 172.21.15.113:0/1591780516 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f567401d070 con 0x7f568413cbe0 2024-04-25T00:05:36.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.829+0000 7f5680ff9640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567006be60 0x7f567006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:36.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.829+0000 7f5680ff9640 1 -- 172.21.15.113:0/1591780516 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7f567408a7d0 con 0x7f568413cbe0 2024-04-25T00:05:36.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.829+0000 7f56837fe640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567006be60 0x7f567006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:36.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.829+0000 7f56837fe640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567006be60 0x7f567006e320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5678006a90 tx=0x7f5678006a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:36.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:36.833+0000 7f5680ff9640 1 -- 172.21.15.113:0/1591780516 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5674059210 con 0x7f568413cbe0 2024-04-25T00:05:37.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.041+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f56840b4380 con 0x7f568413cbe0 2024-04-25T00:05:37.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.111+0000 7f5680ff9640 1 -- 172.21.15.113:0/1591780516 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) ==== 118+0+0 (secure 0 0 0) 0x7f5674058b10 con 0x7f568413cbe0 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567006be60 msgr2=0x7f567006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567006be60 0x7f567006e320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5678006a90 tx=0x7f5678006a20 comp rx=0 tx=0).stop 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568413cbe0 msgr2=0x7f568413a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568413cbe0 0x7f568413a660 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f56740072c0 tx=0x7f5674007780 comp rx=0 tx=0).stop 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 shutdown_connections 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567006be60 0x7f567006e320 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f568414ec40 0x7f568414f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568413be70 0x7f568413cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 --2- 172.21.15.113:0/1591780516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568413cbe0 0x7f568413a660 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 >> 172.21.15.113:0/1591780516 conn(0x7f568406b0f0 msgr2=0x7f56840ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:37.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 shutdown_connections 2024-04-25T00:05:37.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.113+0000 7f568a97e640 1 -- 172.21.15.113:0/1591780516 wait complete. 2024-04-25T00:05:37.125 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:05:37.125 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:05:37.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.225+0000 7f1dbd4d4640 1 Processor -- start 2024-04-25T00:05:37.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.225+0000 7f1dbd4d4640 1 -- start start 2024-04-25T00:05:37.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.225+0000 7f1dbd4d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b5ca0 0x7f1db80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:37.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.225+0000 7f1dbd4d4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1db80b73c0 0x7f1db80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.225+0000 7f1dbd4d4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db80d2e00 con 0x7f1db815e510 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.225+0000 7f1dbd4d4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1db80b6640 con 0x7f1db80b73c0 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.225+0000 7f1dbd4d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80d30e0 con 0x7f1db80b5ca0 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db67fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b5ca0 0x7f1db80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db77fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1db80b73c0 0x7f1db80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db6ffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f1db815e510 0x7f1db80d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:40808/0 (socket says 172.21.15.113:40808) 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db6ffd640 1 -- 172.21.15.113:0/2343045740 learned_addr learned my addr 172.21.15.113:0/2343045740 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db67fc640 1 -- 172.21.15.113:0/2343045740 >> v2:172.21.15.178:3300/0 conn(0x7f1db80b73c0 msgr2=0x7f1db80b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db67fc640 1 --2- 172.21.15.113:0/2343045740 >> v2:172.21.15.178:3300/0 conn(0x7f1db80b73c0 0x7f1db80b7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db67fc640 1 -- 172.21.15.113:0/2343045740 >> v1:172.21.15.178:6789/0 conn(0x7f1db815e510 legacy=0x7f1db80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db67fc640 1 -- 172.21.15.113:0/2343045740 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80d33c0 con 0x7f1db80b5ca0 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db5ffb640 1 -- 172.21.15.113:0/2343045740 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280813887 0 0) 0x7f1db80d2e00 con 0x7f1db815e510 2024-04-25T00:05:37.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.226+0000 7f1db77fe640 1 --2- 172.21.15.113:0/2343045740 >> v2:172.21.15.178:3300/0 conn(0x7f1db80b73c0 0x7f1db80b7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.227+0000 7f1db67fc640 1 --2- 172.21.15.113:0/2343045740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b5ca0 0x7f1db80b6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0097a0 tx=0x7f1dac02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a8497a5a3ef3bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.227+0000 7f1db5ffb640 1 -- 172.21.15.113:0/2343045740 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac037070 con 0x7f1db80b5ca0 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.227+0000 7f1db5ffb640 1 -- 172.21.15.113:0/2343045740 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dac02ae50 con 0x7f1db80b5ca0 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.227+0000 7f1db5ffb640 1 -- 172.21.15.113:0/2343045740 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac032660 con 0x7f1db80b5ca0 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2343045740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b5ca0 msgr2=0x7f1db80b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2343045740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b5ca0 0x7f1db80b6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0097a0 tx=0x7f1dac02a100 comp rx=0 tx=0).stop 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2343045740 shutdown_connections 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2343045740 >> v2:172.21.15.178:3300/0 conn(0x7f1db80b73c0 0x7f1db80b7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2343045740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b5ca0 0x7f1db80b6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2343045740 >> 172.21.15.113:0/2343045740 conn(0x7f1db806b720 msgr2=0x7f1db80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2343045740 shutdown_connections 2024-04-25T00:05:37.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.228+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2343045740 wait complete. 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.229+0000 7f1dbd4d4640 1 Processor -- start 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.229+0000 7f1dbd4d4640 1 -- start start 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1dbd4d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1db80b5ca0 0x7f1db814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1dbd4d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b73c0 0x7f1db80ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1dbd4d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db815e510 0x7f1db8154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1dbd4d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80d3ea0 con 0x7f1db815e510 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1dbd4d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80d3980 con 0x7f1db80b5ca0 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1dbd4d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80d2a90 con 0x7f1db80b73c0 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db815e510 0x7f1db8154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db815e510 0x7f1db8154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45776/0 (socket says 172.21.15.113:45776) 2024-04-25T00:05:37.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 -- 172.21.15.113:0/2626664972 learned_addr learned my addr 172.21.15.113:0/2626664972 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db6ffd640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1db80b5ca0 0x7f1db814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db67fc640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b73c0 0x7f1db80ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 -- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b73c0 msgr2=0x7f1db80ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b73c0 0x7f1db80ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 -- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1db80b5ca0 msgr2=0x7f1db814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1db80b5ca0 0x7f1db814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 -- 172.21.15.113:0/2626664972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db815ac20 con 0x7f1db815e510 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1db77fe640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db815e510 0x7f1db8154080 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1db000efc0 tx=0x7f1db000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.230+0000 7f1d9f7fe640 1 -- 172.21.15.113:0/2626664972 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0018030 con 0x7f1db815e510 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.231+0000 7f1d9f7fe640 1 -- 172.21.15.113:0/2626664972 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db000cdf0 con 0x7f1db815e510 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.231+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db815af00 con 0x7f1db815e510 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.231+0000 7f1d9f7fe640 1 -- 172.21.15.113:0/2626664972 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0003bb0 con 0x7f1db815e510 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.231+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db815b4c0 con 0x7f1db815e510 2024-04-25T00:05:37.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.232+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db80c7590 con 0x7f1db815e510 2024-04-25T00:05:37.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.232+0000 7f1d9f7fe640 1 -- 172.21.15.113:0/2626664972 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1db001b070 con 0x7f1db815e510 2024-04-25T00:05:37.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.232+0000 7f1d9f7fe640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1da406be30 0x7f1da406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:37.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.233+0000 7f1db6ffd640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1da406be30 0x7f1da406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:37.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.233+0000 7f1d9f7fe640 1 -- 172.21.15.113:0/2626664972 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f1db008b7a0 con 0x7f1db815e510 2024-04-25T00:05:37.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.233+0000 7f1db6ffd640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1da406be30 0x7f1da406e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000f60 tx=0x7f1da8007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:37.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.237+0000 7f1d9f7fe640 1 -- 172.21.15.113:0/2626664972 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1db80c7590 con 0x7f1db815e510 2024-04-25T00:05:37.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:37.446+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1db80cfbc0 con 0x7f1db815e510 2024-04-25T00:05:38.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.128+0000 7f1d9f7fe640 1 -- 172.21.15.113:0/2626664972 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) ==== 122+0+0 (secure 0 0 0) 0x7f1db0010040 con 0x7f1db815e510 2024-04-25T00:05:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1da406be30 msgr2=0x7f1da406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1da406be30 0x7f1da406e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000f60 tx=0x7f1da8007d50 comp rx=0 tx=0).stop 2024-04-25T00:05:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db815e510 msgr2=0x7f1db8154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db815e510 0x7f1db8154080 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1db000efc0 tx=0x7f1db000c490 comp rx=0 tx=0).stop 2024-04-25T00:05:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 shutdown_connections 2024-04-25T00:05:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1da406be30 0x7f1da406e2f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db815e510 0x7f1db8154080 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1db80b73c0 0x7f1db80ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 --2- 172.21.15.113:0/2626664972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1db80b5ca0 0x7f1db814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 >> 172.21.15.113:0/2626664972 conn(0x7f1db806b720 msgr2=0x7f1db80b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 shutdown_connections 2024-04-25T00:05:38.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.130+0000 7f1dbd4d4640 1 -- 172.21.15.113:0/2626664972 wait complete. 2024-04-25T00:05:38.142 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:05:38.143 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:38.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.241+0000 7f047c74f640 1 Processor -- start 2024-04-25T00:05:38.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.241+0000 7f047c74f640 1 -- start start 2024-04-25T00:05:38.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.242+0000 7f047c74f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0474144d60 0x7f04740b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.242+0000 7f047c74f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04741453c0 0x7f047406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.242+0000 7f047c74f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04740cfad0 con 0x7f047406e510 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.242+0000 7f047c74f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0474145c40 con 0x7f0474144d60 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.242+0000 7f047c74f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f047406e940 con 0x7f04741453c0 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.242+0000 7f0479cea640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04741453c0 0x7f047406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.242+0000 7f047a4eb640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0474144d60 0x7f04740b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f047acec640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f047406e510 0x7f047415fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:38968/0 (socket says 172.21.15.113:38968) 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f047acec640 1 -- 172.21.15.113:0/3810631592 learned_addr learned my addr 172.21.15.113:0/3810631592 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f0479cea640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04741453c0 0x7f047406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:47252/0 (socket says 172.21.15.113:47252) 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f0479cea640 1 -- 172.21.15.113:0/3810631592 >> v2:172.21.15.178:3300/0 conn(0x7f0474144d60 msgr2=0x7f04740b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f0479cea640 1 --2- 172.21.15.113:0/3810631592 >> v2:172.21.15.178:3300/0 conn(0x7f0474144d60 0x7f04740b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f0479cea640 1 -- 172.21.15.113:0/3810631592 >> v1:172.21.15.178:6789/0 conn(0x7f047406e510 legacy=0x7f047415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f0479cea640 1 -- 172.21.15.113:0/3810631592 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04740cf230 con 0x7f04741453c0 2024-04-25T00:05:38.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.243+0000 7f047a4eb640 1 --2- 172.21.15.113:0/3810631592 >> v2:172.21.15.178:3300/0 conn(0x7f0474144d60 0x7f04740b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:38.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f0479cea640 1 --2- 172.21.15.113:0/3810631592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04741453c0 0x7f047406dfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f04640097a0 tx=0x7f046402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c21c1bea818bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f04794e9640 1 -- 172.21.15.113:0/3810631592 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464037070 con 0x7f04741453c0 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f04794e9640 1 -- 172.21.15.113:0/3810631592 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046402ae50 con 0x7f04741453c0 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f04794e9640 1 -- 172.21.15.113:0/3810631592 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464032660 con 0x7f04741453c0 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 -- 172.21.15.113:0/3810631592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04741453c0 msgr2=0x7f047406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 --2- 172.21.15.113:0/3810631592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04741453c0 0x7f047406dfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f04640097a0 tx=0x7f046402a100 comp rx=0 tx=0).stop 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 -- 172.21.15.113:0/3810631592 shutdown_connections 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 --2- 172.21.15.113:0/3810631592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04741453c0 0x7f047406dfd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 --2- 172.21.15.113:0/3810631592 >> v2:172.21.15.178:3300/0 conn(0x7f0474144d60 0x7f04740b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 -- 172.21.15.113:0/3810631592 >> 172.21.15.113:0/3810631592 conn(0x7f04740b1a50 msgr2=0x7f04740b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 -- 172.21.15.113:0/3810631592 shutdown_connections 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.244+0000 7f047c74f640 1 -- 172.21.15.113:0/3810631592 wait complete. 2024-04-25T00:05:38.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 Processor -- start 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 -- start start 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047406e510 0x7f04741f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0474144d60 0x7f04741586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04741453c0 0x7f04741f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0474070e10 con 0x7f047406e510 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04740cf7f0 con 0x7f04741453c0 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047c74f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0474145140 con 0x7f0474144d60 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047a4eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047406e510 0x7f04741f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047a4eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047406e510 0x7f04741f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47882/0 (socket says 172.21.15.113:47882) 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.246+0000 7f047a4eb640 1 -- 172.21.15.113:0/3233364283 learned_addr learned my addr 172.21.15.113:0/3233364283 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f0479cea640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0474144d60 0x7f04741586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047acec640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04741453c0 0x7f04741f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047a4eb640 1 -- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0474144d60 msgr2=0x7f04741586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047a4eb640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0474144d60 0x7f04741586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047a4eb640 1 -- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04741453c0 msgr2=0x7f04741f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047a4eb640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04741453c0 0x7f04741f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047a4eb640 1 -- 172.21.15.113:0/3233364283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04741f20e0 con 0x7f047406e510 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047acec640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04741453c0 0x7f04741f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047a4eb640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047406e510 0x7f04741f2d60 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f047000e960 tx=0x7f047000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f046affd640 1 -- 172.21.15.113:0/3233364283 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470017070 con 0x7f047406e510 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f046affd640 1 -- 172.21.15.113:0/3233364283 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047000ca00 con 0x7f047406e510 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741f2390 con 0x7f047406e510 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f046affd640 1 -- 172.21.15.113:0/3233364283 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000cdd0 con 0x7f047406e510 2024-04-25T00:05:38.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.247+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741f2950 con 0x7f047406e510 2024-04-25T00:05:38.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.248+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04740691a0 con 0x7f047406e510 2024-04-25T00:05:38.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.248+0000 7f046affd640 1 -- 172.21.15.113:0/3233364283 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f047001a070 con 0x7f047406e510 2024-04-25T00:05:38.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.249+0000 7f046affd640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f045806be30 0x7f045806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.249+0000 7f0479cea640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f045806be30 0x7f045806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.249+0000 7f046affd640 1 -- 172.21.15.113:0/3233364283 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f047008acc0 con 0x7f047406e510 2024-04-25T00:05:38.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.249+0000 7f0479cea640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f045806be30 0x7f045806e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0464034000 tx=0x7f046402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.253+0000 7f046affd640 1 -- 172.21.15.113:0/3233364283 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0470059700 con 0x7f047406e510 2024-04-25T00:05:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.459+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04740b27e0 con 0x7f047406e510 2024-04-25T00:05:38.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.460+0000 7f046affd640 1 -- 172.21.15.113:0/3233364283 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f0470059000 con 0x7f047406e510 2024-04-25T00:05:38.460 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:05:38.461 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:05:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f045806be30 msgr2=0x7f045806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f045806be30 0x7f045806e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0464034000 tx=0x7f046402e000 comp rx=0 tx=0).stop 2024-04-25T00:05:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047406e510 msgr2=0x7f04741f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047406e510 0x7f04741f2d60 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f047000e960 tx=0x7f047000ee20 comp rx=0 tx=0).stop 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 shutdown_connections 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f045806be30 0x7f045806e2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04741453c0 0x7f04741f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0474144d60 0x7f04741586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 --2- 172.21.15.113:0/3233364283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047406e510 0x7f04741f2d60 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 >> 172.21.15.113:0/3233364283 conn(0x7f04740b1a50 msgr2=0x7f047413f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 shutdown_connections 2024-04-25T00:05:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.462+0000 7f047c74f640 1 -- 172.21.15.113:0/3233364283 wait complete. 2024-04-25T00:05:38.473 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:05:38.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.569+0000 7ff0a33ed640 1 Processor -- start 2024-04-25T00:05:38.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.570+0000 7ff0a33ed640 1 -- start start 2024-04-25T00:05:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.570+0000 7ff0a33ed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 0x7ff09c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.570+0000 7ff0a33ed640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff09c06e510 0x7ff09c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.570+0000 7ff0a33ed640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c144cf0 con 0x7ff09c144910 2024-04-25T00:05:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.570+0000 7ff0a33ed640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c0b2f30 con 0x7ff09c06e510 2024-04-25T00:05:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.570+0000 7ff0a33ed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0b3210 con 0x7ff09c0b3fd0 2024-04-25T00:05:38.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a1189640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff09c144910 0x7ff09c0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54956/0 (socket says 172.21.15.113:54956) 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a1189640 1 -- 172.21.15.113:0/2942554698 learned_addr learned my addr 172.21.15.113:0/2942554698 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a198a640 1 --2- 172.21.15.113:0/2942554698 >> v2:172.21.15.178:3300/0 conn(0x7ff09c06e510 0x7ff09c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 0x7ff09c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133376414 0 0) 0x7ff09c144cf0 con 0x7ff09c144910 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff088005730 con 0x7ff09c144910 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2612042984 0 0) 0x7ff088005730 con 0x7ff09c144910 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 msgr2=0x7ff09c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 0x7ff09c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 >> v2:172.21.15.178:3300/0 conn(0x7ff09c06e510 msgr2=0x7ff09c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 --2- 172.21.15.113:0/2942554698 >> v2:172.21.15.178:3300/0 conn(0x7ff09c06e510 0x7ff09c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0b34f0 con 0x7ff09c144910 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff08c001670 con 0x7ff09c144910 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 0x7ff09c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:38.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a198a640 1 --2- 172.21.15.113:0/2942554698 >> v2:172.21.15.178:3300/0 conn(0x7ff09c06e510 0x7ff09c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 >> v1:172.21.15.113:6789/0 conn(0x7ff09c144910 legacy=0x7ff09c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 0x7ff088005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff088006100 0x7ff088006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff088006f50 0x7ff0880073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0b2f30 con 0x7ff088006100 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0b3210 con 0x7ff088006f50 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff08800dbd0 con 0x7ff09c0b3fd0 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff088006100 0x7ff088006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a1189640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 0x7ff088005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a198a640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff088006f50 0x7ff0880073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 -- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 msgr2=0x7ff088005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0b3fd0 0x7ff088005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 -- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff088006f50 msgr2=0x7ff0880073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff088006f50 0x7ff0880073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.571+0000 7ff0a0988640 1 -- 172.21.15.113:0/2942554698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088005730 con 0x7ff088006100 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.572+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff088006100 0x7ff088006a10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff098009c90 tx=0x7ff088007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd92cd1f8b69ccf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.572+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098036070 con 0x7ff088006100 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.572+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff098031400 con 0x7ff088006100 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.572+0000 7ff093fff640 1 -- 172.21.15.113:0/2942554698 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098009860 con 0x7ff088006100 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.573+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2942554698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff088006100 msgr2=0x7ff088006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.573+0000 7ff0a33ed640 1 --2- 172.21.15.113:0/2942554698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff088006100 0x7ff088006a10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff098009c90 tx=0x7ff088007a40 comp rx=0 tx=0).stop 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.573+0000 7ff0a198a640 1 -- 172.21.15.113:0/2942554698 reap_dead start 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.573+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2942554698 shutdown_connections 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.573+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2942554698 >> 172.21.15.113:0/2942554698 conn(0x7ff09c0b1a50 msgr2=0x7ff09c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.573+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2942554698 shutdown_connections 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.573+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2942554698 wait complete. 2024-04-25T00:05:38.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.574+0000 7ff0a33ed640 1 Processor -- start 2024-04-25T00:05:38.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.574+0000 7ff0a33ed640 1 -- start start 2024-04-25T00:05:38.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a33ed640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff088006100 0x7ff09c137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a33ed640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09c138f00 0x7ff09c13bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a33ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff09c13c150 0x7ff09c0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a33ed640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0b4da0 con 0x7ff09c138f00 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a33ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0b4410 con 0x7ff09c13c150 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a33ed640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0b4f20 con 0x7ff088006100 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09c138f00 0x7ff09c13bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09c138f00 0x7ff09c13bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47892/0 (socket says 172.21.15.113:47892) 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 -- 172.21.15.113:0/2844812096 learned_addr learned my addr 172.21.15.113:0/2844812096 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a1189640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff088006100 0x7ff09c137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a198a640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff09c13c150 0x7ff09c0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 -- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff088006100 msgr2=0x7ff09c137fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff088006100 0x7ff09c137fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 -- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff09c13c150 msgr2=0x7ff09c0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff09c13c150 0x7ff09c0cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 -- 172.21.15.113:0/2844812096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c13ea60 con 0x7ff09c138f00 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a1189640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff088006100 0x7ff09c137fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a198a640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff09c13c150 0x7ff09c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a0988640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09c138f00 0x7ff09c13bc10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ff098009db0 tx=0x7ff098004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff091ffb640 1 -- 172.21.15.113:0/2844812096 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098040020 con 0x7ff09c138f00 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff091ffb640 1 -- 172.21.15.113:0/2844812096 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0980093e0 con 0x7ff09c138f00 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c13ed40 con 0x7ff09c138f00 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.575+0000 7ff091ffb640 1 -- 172.21.15.113:0/2844812096 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098036040 con 0x7ff09c138f00 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.576+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c139380 con 0x7ff09c138f00 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.577+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c13c560 con 0x7ff09c138f00 2024-04-25T00:05:38.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.577+0000 7ff091ffb640 1 -- 172.21.15.113:0/2844812096 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff098043020 con 0x7ff09c138f00 2024-04-25T00:05:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.577+0000 7ff091ffb640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff08006be60 0x7ff08006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.578+0000 7ff091ffb640 1 -- 172.21.15.113:0/2844812096 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7ff0980ab420 con 0x7ff09c138f00 2024-04-25T00:05:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.578+0000 7ff0a1189640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff08006be60 0x7ff08006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.578+0000 7ff0a1189640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff08006be60 0x7ff08006e320 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff08c001050 tx=0x7ff08c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.582+0000 7ff091ffb640 1 -- 172.21.15.113:0/2844812096 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff09c13c560 con 0x7ff09c138f00 2024-04-25T00:05:38.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.755+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff09c0691a0 con 0x7ff09c138f00 2024-04-25T00:05:38.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.756+0000 7ff091ffb640 1 -- 172.21.15.113:0/2844812096 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16675 (secure 0 0 0) 0x7ff09804a030 con 0x7ff09c138f00 2024-04-25T00:05:38.757 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":116,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:05:37.091278+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:05:33.644000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:05:34.412894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:05:38.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff08006be60 msgr2=0x7ff08006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff08006be60 0x7ff08006e320 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff08c001050 tx=0x7ff08c009040 comp rx=0 tx=0).stop 2024-04-25T00:05:38.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09c138f00 msgr2=0x7ff09c13bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09c138f00 0x7ff09c13bc10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ff098009db0 tx=0x7ff098004730 comp rx=0 tx=0).stop 2024-04-25T00:05:38.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 shutdown_connections 2024-04-25T00:05:38.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff08006be60 0x7ff08006e320 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff09c13c150 0x7ff09c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff09c138f00 0x7ff09c13bc10 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 --2- 172.21.15.113:0/2844812096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff088006100 0x7ff09c137fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 >> 172.21.15.113:0/2844812096 conn(0x7ff09c0b1a50 msgr2=0x7ff09c142710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.758+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 shutdown_connections 2024-04-25T00:05:38.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.759+0000 7ff0a33ed640 1 -- 172.21.15.113:0/2844812096 wait complete. 2024-04-25T00:05:38.770 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:05:38.770 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:05:38.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.867+0000 7ff9c1fa9640 1 Processor -- start 2024-04-25T00:05:38.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.867+0000 7ff9c1fa9640 1 -- start start 2024-04-25T00:05:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9c1fa9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9c1fa9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9bc0b3af0 0x7ff9bc0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9c1fa9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9bc0b2d50 con 0x7ff9bc072ea0 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9c1fa9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff9bc072a70 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9c1fa9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc0b3010 con 0x7ff9bc15a6b0 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9bbfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9bc0b3af0 0x7ff9bc0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9b3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff9bc072ea0 0x7ff9bc071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54966/0 (socket says 172.21.15.113:54966) 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9bbfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9bc0b3af0 0x7ff9bc0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47906/0 (socket says 172.21.15.113:47906) 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9b3fff640 1 -- 172.21.15.113:0/2739311862 learned_addr learned my addr 172.21.15.113:0/2739311862 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9bb7fe640 1 --2- 172.21.15.113:0/2739311862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9bbfff640 1 -- 172.21.15.113:0/2739311862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 msgr2=0x7ff9bc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9bbfff640 1 --2- 172.21.15.113:0/2739311862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9bbfff640 1 -- 172.21.15.113:0/2739311862 >> v1:172.21.15.113:6789/0 conn(0x7ff9bc072ea0 legacy=0x7ff9bc071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.868+0000 7ff9bbfff640 1 -- 172.21.15.113:0/2739311862 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc0b32f0 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.869+0000 7ff9bbfff640 1 --2- 172.21.15.113:0/2739311862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc0b3f80 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009d90 tx=0x7ff9b402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a765a79969c44e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.869+0000 7ff9baffd640 1 -- 172.21.15.113:0/2739311862 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4007740 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.869+0000 7ff9baffd640 1 -- 172.21.15.113:0/2739311862 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b40078e0 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.869+0000 7ff9baffd640 1 -- 172.21.15.113:0/2739311862 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4007cb0 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/2739311862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 msgr2=0x7ff9bc0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/2739311862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc0b3f80 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009d90 tx=0x7ff9b402a660 comp rx=0 tx=0).stop 2024-04-25T00:05:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/2739311862 shutdown_connections 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/2739311862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc0b3f80 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/2739311862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/2739311862 >> 172.21.15.113:0/2739311862 conn(0x7ff9bc0b1a50 msgr2=0x7ff9bc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/2739311862 shutdown_connections 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.870+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/2739311862 wait complete. 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.871+0000 7ff9c1fa9640 1 Processor -- start 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.871+0000 7ff9c1fa9640 1 -- start start 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9bc072ea0 0x7ff9bc0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc15aa90 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc072720 con 0x7ff9bc072ea0 2024-04-25T00:05:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc15ac10 con 0x7ff9bc15a6b0 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc1458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47918/0 (socket says 172.21.15.113:47918) 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 -- 172.21.15.113:0/1075270593 learned_addr learned my addr 172.21.15.113:0/1075270593 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9bbfff640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 -- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 msgr2=0x7ff9bc145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9bb7fe640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9bc072ea0 0x7ff9bc0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 -- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9bc072ea0 msgr2=0x7ff9bc0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9bc072ea0 0x7ff9bc0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 -- 172.21.15.113:0/1075270593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc165870 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9bbfff640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b3fff640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc1458d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c440 tx=0x7ff9a4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b8ff9640 1 -- 172.21.15.113:0/1075270593 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4017070 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b8ff9640 1 -- 172.21.15.113:0/1075270593 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a4005ce0 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9b8ff9640 1 -- 172.21.15.113:0/1075270593 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40072f0 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc165b10 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.872+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc1660d0 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.873+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9bc1662d0 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.874+0000 7ff9b8ff9640 1 -- 172.21.15.113:0/1075270593 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9a401a070 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.874+0000 7ff9b8ff9640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.874+0000 7ff9bb7fe640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.874+0000 7ff9b8ff9640 1 -- 172.21.15.113:0/1075270593 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7ff9a408a210 con 0x7ff9bc0b3af0 2024-04-25T00:05:38.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.874+0000 7ff9bb7fe640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b7c0 tx=0x7ff9ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:38.878+0000 7ff9b8ff9640 1 -- 172.21.15.113:0/1075270593 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9bc1662d0 con 0x7ff9bc0b3af0 2024-04-25T00:05:39.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.049+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9bc0691a0 con 0x7ff99c06be30 2024-04-25T00:05:39.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.055+0000 7ff9b8ff9640 1 -- 172.21.15.113:0/1075270593 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252412 (secure 0 0 0) 0x7ff9bc0691a0 con 0x7ff99c06be30 2024-04-25T00:05:39.058 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:39.060 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:05:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 msgr2=0x7ff99c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b7c0 tx=0x7ff9ac008040 comp rx=0 tx=0).stop 2024-04-25T00:05:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 msgr2=0x7ff9bc1458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc1458d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c440 tx=0x7ff9a4001e70 comp rx=0 tx=0).stop 2024-04-25T00:05:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 shutdown_connections 2024-04-25T00:05:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff99c06be30 0x7ff99c06e2f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9bc15a6b0 0x7ff9bc145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9bc0b3af0 0x7ff9bc1458d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 --2- 172.21.15.113:0/1075270593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9bc072ea0 0x7ff9bc0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 >> 172.21.15.113:0/1075270593 conn(0x7ff9bc0b1a50 msgr2=0x7ff9bc0b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 shutdown_connections 2024-04-25T00:05:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.058+0000 7ff9c1fa9640 1 -- 172.21.15.113:0/1075270593 wait complete. 2024-04-25T00:05:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-04-25T00:05:37.196723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394952,"kb_used_data":12232,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754579768,"statfs":{"total":773094113280,"available":772689682432,"internally_reserved":0,"allocated":12525568,"data_stored":6078896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000586"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119482+0000","last_change":"2024-04-25T00:05:36.113111+0000","last_active":"2024-04-25T00:05:36.119482+0000","last_peered":"2024-04-25T00:05:36.119482+0000","last_clean":"2024-04-25T00:05:36.119482+0000","last_became_active":"2024-04-25T00:05:36.112593+0000","last_became_peered":"2024-04-25T00:05:36.112593+0000","last_unstale":"2024-04-25T00:05:36.119482+0000","last_undegraded":"2024-04-25T00:05:36.119482+0000","last_fullsized":"2024-04-25T00:05:36.119482+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:47.302545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.102869+0000","last_change":"2024-04-25T00:05:36.100726+0000","last_active":"2024-04-25T00:05:36.102869+0000","last_peered":"2024-04-25T00:05:36.102869+0000","last_clean":"2024-04-25T00:05:36.102869+0000","last_became_active":"2024-04-25T00:05:36.100473+0000","last_became_peered":"2024-04-25T00:05:36.100473+0000","last_unstale":"2024-04-25T00:05:36.102869+0000","last_undegraded":"2024-04-25T00:05:36.102869+0000","last_fullsized":"2024-04-25T00:05:36.102869+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:13.049564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110820+0000","last_change":"2024-04-25T00:05:36.109014+0000","last_active":"2024-04-25T00:05:36.110820+0000","last_peered":"2024-04-25T00:05:36.110820+0000","last_clean":"2024-04-25T00:05:36.110820+0000","last_became_active":"2024-04-25T00:05:36.108706+0000","last_became_peered":"2024-04-25T00:05:36.108706+0000","last_unstale":"2024-04-25T00:05:36.110820+0000","last_undegraded":"2024-04-25T00:05:36.110820+0000","last_fullsized":"2024-04-25T00:05:36.110820+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:00:43.877373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116600+0000","last_change":"2024-04-25T00:05:36.109895+0000","last_active":"2024-04-25T00:05:36.116600+0000","last_peered":"2024-04-25T00:05:36.116600+0000","last_clean":"2024-04-25T00:05:36.116600+0000","last_became_active":"2024-04-25T00:05:36.108862+0000","last_became_peered":"2024-04-25T00:05:36.108862+0000","last_unstale":"2024-04-25T00:05:36.116600+0000","last_undegraded":"2024-04-25T00:05:36.116600+0000","last_fullsized":"2024-04-25T00:05:36.116600+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:22.725103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.103274+0000","last_change":"2024-04-25T00:05:36.101549+0000","last_active":"2024-04-25T00:05:36.103274+0000","last_peered":"2024-04-25T00:05:36.103274+0000","last_clean":"2024-04-25T00:05:36.103274+0000","last_became_active":"2024-04-25T00:05:36.101262+0000","last_became_peered":"2024-04-25T00:05:36.101262+0000","last_unstale":"2024-04-25T00:05:36.103274+0000","last_undegraded":"2024-04-25T00:05:36.103274+0000","last_fullsized":"2024-04-25T00:05:36.103274+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:36.448728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107516+0000","last_change":"2024-04-25T00:05:36.104604+0000","last_active":"2024-04-25T00:05:36.107516+0000","last_peered":"2024-04-25T00:05:36.107516+0000","last_clean":"2024-04-25T00:05:36.107516+0000","last_became_active":"2024-04-25T00:05:36.104363+0000","last_became_peered":"2024-04-25T00:05:36.104363+0000","last_unstale":"2024-04-25T00:05:36.107516+0000","last_undegraded":"2024-04-25T00:05:36.107516+0000","last_fullsized":"2024-04-25T00:05:36.107516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:03.916769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.115328+0000","last_change":"2024-04-25T00:05:36.114133+0000","last_active":"2024-04-25T00:05:36.115328+0000","last_peered":"2024-04-25T00:05:36.115328+0000","last_clean":"2024-04-25T00:05:36.115328+0000","last_became_active":"2024-04-25T00:05:36.112488+0000","last_became_peered":"2024-04-25T00:05:36.112488+0000","last_unstale":"2024-04-25T00:05:36.115328+0000","last_undegraded":"2024-04-25T00:05:36.115328+0000","last_fullsized":"2024-04-25T00:05:36.115328+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:10.046278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110148+0000","last_change":"2024-04-25T00:05:36.107676+0000","last_active":"2024-04-25T00:05:36.110148+0000","last_peered":"2024-04-25T00:05:36.110148+0000","last_clean":"2024-04-25T00:05:36.110148+0000","last_became_active":"2024-04-25T00:05:36.104599+0000","last_became_peered":"2024-04-25T00:05:36.104599+0000","last_unstale":"2024-04-25T00:05:36.110148+0000","last_undegraded":"2024-04-25T00:05:36.110148+0000","last_fullsized":"2024-04-25T00:05:36.110148+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:58.123203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108992+0000","last_change":"2024-04-25T00:05:36.106067+0000","last_active":"2024-04-25T00:05:36.108992+0000","last_peered":"2024-04-25T00:05:36.108992+0000","last_clean":"2024-04-25T00:05:36.108992+0000","last_became_active":"2024-04-25T00:05:36.103418+0000","last_became_peered":"2024-04-25T00:05:36.103418+0000","last_unstale":"2024-04-25T00:05:36.108992+0000","last_undegraded":"2024-04-25T00:05:36.108992+0000","last_fullsized":"2024-04-25T00:05:36.108992+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:12.701990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116657+0000","last_change":"2024-04-25T00:05:36.109374+0000","last_active":"2024-04-25T00:05:36.116657+0000","last_peered":"2024-04-25T00:05:36.116657+0000","last_clean":"2024-04-25T00:05:36.116657+0000","last_became_active":"2024-04-25T00:05:36.108702+0000","last_became_peered":"2024-04-25T00:05:36.108702+0000","last_unstale":"2024-04-25T00:05:36.116657+0000","last_undegraded":"2024-04-25T00:05:36.116657+0000","last_fullsized":"2024-04-25T00:05:36.116657+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:50.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114612+0000","last_change":"2024-04-25T00:05:36.114031+0000","last_active":"2024-04-25T00:05:36.114612+0000","last_peered":"2024-04-25T00:05:36.114612+0000","last_clean":"2024-04-25T00:05:36.114612+0000","last_became_active":"2024-04-25T00:05:36.113458+0000","last_became_peered":"2024-04-25T00:05:36.113458+0000","last_unstale":"2024-04-25T00:05:36.114612+0000","last_undegraded":"2024-04-25T00:05:36.114612+0000","last_fullsized":"2024-04-25T00:05:36.114612+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:03.775306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118407+0000","last_change":"2024-04-25T00:05:36.111974+0000","last_active":"2024-04-25T00:05:36.118407+0000","last_peered":"2024-04-25T00:05:36.118407+0000","last_clean":"2024-04-25T00:05:36.118407+0000","last_became_active":"2024-04-25T00:05:36.111115+0000","last_became_peered":"2024-04-25T00:05:36.111115+0000","last_unstale":"2024-04-25T00:05:36.118407+0000","last_undegraded":"2024-04-25T00:05:36.118407+0000","last_fullsized":"2024-04-25T00:05:36.118407+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:28.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105916+0000","last_change":"2024-04-25T00:05:36.105240+0000","last_active":"2024-04-25T00:05:36.105916+0000","last_peered":"2024-04-25T00:05:36.105916+0000","last_clean":"2024-04-25T00:05:36.105916+0000","last_became_active":"2024-04-25T00:05:36.104970+0000","last_became_peered":"2024-04-25T00:05:36.104970+0000","last_unstale":"2024-04-25T00:05:36.105916+0000","last_undegraded":"2024-04-25T00:05:36.105916+0000","last_fullsized":"2024-04-25T00:05:36.105916+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:22.146161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110348+0000","last_change":"2024-04-25T00:05:36.109498+0000","last_active":"2024-04-25T00:05:36.110348+0000","last_peered":"2024-04-25T00:05:36.110348+0000","last_clean":"2024-04-25T00:05:36.110348+0000","last_became_active":"2024-04-25T00:05:36.108766+0000","last_became_peered":"2024-04-25T00:05:36.108766+0000","last_unstale":"2024-04-25T00:05:36.110348+0000","last_undegraded":"2024-04-25T00:05:36.110348+0000","last_fullsized":"2024-04-25T00:05:36.110348+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:59.169658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107595+0000","last_change":"2024-04-25T00:05:36.106014+0000","last_active":"2024-04-25T00:05:36.107595+0000","last_peered":"2024-04-25T00:05:36.107595+0000","last_clean":"2024-04-25T00:05:36.107595+0000","last_became_active":"2024-04-25T00:05:36.105731+0000","last_became_peered":"2024-04-25T00:05:36.105731+0000","last_unstale":"2024-04-25T00:05:36.107595+0000","last_undegraded":"2024-04-25T00:05:36.107595+0000","last_fullsized":"2024-04-25T00:05:36.107595+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:02.220173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.109179+0000","last_change":"2024-04-25T00:05:36.105776+0000","last_active":"2024-04-25T00:05:36.109179+0000","last_peered":"2024-04-25T00:05:36.109179+0000","last_clean":"2024-04-25T00:05:36.109179+0000","last_became_active":"2024-04-25T00:05:36.104260+0000","last_became_peered":"2024-04-25T00:05:36.104260+0000","last_unstale":"2024-04-25T00:05:36.109179+0000","last_undegraded":"2024-04-25T00:05:36.109179+0000","last_fullsized":"2024-04-25T00:05:36.109179+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:48.108953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114257+0000","last_change":"2024-04-25T00:05:36.112275+0000","last_active":"2024-04-25T00:05:36.114257+0000","last_peered":"2024-04-25T00:05:36.114257+0000","last_clean":"2024-04-25T00:05:36.114257+0000","last_became_active":"2024-04-25T00:05:36.111983+0000","last_became_peered":"2024-04-25T00:05:36.111983+0000","last_unstale":"2024-04-25T00:05:36.114257+0000","last_undegraded":"2024-04-25T00:05:36.114257+0000","last_fullsized":"2024-04-25T00:05:36.114257+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:18.334300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116340+0000","last_change":"2024-04-25T00:05:36.114779+0000","last_active":"2024-04-25T00:05:36.116340+0000","last_peered":"2024-04-25T00:05:36.116340+0000","last_clean":"2024-04-25T00:05:36.116340+0000","last_became_active":"2024-04-25T00:05:36.114492+0000","last_became_peered":"2024-04-25T00:05:36.114492+0000","last_unstale":"2024-04-25T00:05:36.116340+0000","last_undegraded":"2024-04-25T00:05:36.116340+0000","last_fullsized":"2024-04-25T00:05:36.116340+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:40.871259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111910+0000","last_change":"2024-04-25T00:05:36.110693+0000","last_active":"2024-04-25T00:05:36.111910+0000","last_peered":"2024-04-25T00:05:36.111910+0000","last_clean":"2024-04-25T00:05:36.111910+0000","last_became_active":"2024-04-25T00:05:36.110412+0000","last_became_peered":"2024-04-25T00:05:36.110412+0000","last_unstale":"2024-04-25T00:05:36.111910+0000","last_undegraded":"2024-04-25T00:05:36.111910+0000","last_fullsized":"2024-04-25T00:05:36.111910+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:48.108953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108755+0000","last_change":"2024-04-25T00:05:36.104566+0000","last_active":"2024-04-25T00:05:36.108755+0000","last_peered":"2024-04-25T00:05:36.108755+0000","last_clean":"2024-04-25T00:05:36.108755+0000","last_became_active":"2024-04-25T00:05:36.104295+0000","last_became_peered":"2024-04-25T00:05:36.104295+0000","last_unstale":"2024-04-25T00:05:36.108755+0000","last_undegraded":"2024-04-25T00:05:36.108755+0000","last_fullsized":"2024-04-25T00:05:36.108755+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:16.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107787+0000","last_change":"2024-04-25T00:05:36.104229+0000","last_active":"2024-04-25T00:05:36.107787+0000","last_peered":"2024-04-25T00:05:36.107787+0000","last_clean":"2024-04-25T00:05:36.107787+0000","last_became_active":"2024-04-25T00:05:36.103314+0000","last_became_peered":"2024-04-25T00:05:36.103314+0000","last_unstale":"2024-04-25T00:05:36.107787+0000","last_undegraded":"2024-04-25T00:05:36.107787+0000","last_fullsized":"2024-04-25T00:05:36.107787+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:37.688391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111422+0000","last_change":"2024-04-25T00:05:36.109879+0000","last_active":"2024-04-25T00:05:36.111422+0000","last_peered":"2024-04-25T00:05:36.111422+0000","last_clean":"2024-04-25T00:05:36.111422+0000","last_became_active":"2024-04-25T00:05:36.109633+0000","last_became_peered":"2024-04-25T00:05:36.109633+0000","last_unstale":"2024-04-25T00:05:36.111422+0000","last_undegraded":"2024-04-25T00:05:36.111422+0000","last_fullsized":"2024-04-25T00:05:36.111422+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:22.146161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107965+0000","last_change":"2024-04-25T00:05:36.105878+0000","last_active":"2024-04-25T00:05:36.107965+0000","last_peered":"2024-04-25T00:05:36.107965+0000","last_clean":"2024-04-25T00:05:36.107965+0000","last_became_active":"2024-04-25T00:05:36.104761+0000","last_became_peered":"2024-04-25T00:05:36.104761+0000","last_unstale":"2024-04-25T00:05:36.107965+0000","last_undegraded":"2024-04-25T00:05:36.107965+0000","last_fullsized":"2024-04-25T00:05:36.107965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:16.802256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.104792+0000","last_change":"2024-04-25T00:05:36.103524+0000","last_active":"2024-04-25T00:05:36.104792+0000","last_peered":"2024-04-25T00:05:36.104792+0000","last_clean":"2024-04-25T00:05:36.104792+0000","last_became_active":"2024-04-25T00:05:36.103238+0000","last_became_peered":"2024-04-25T00:05:36.103238+0000","last_unstale":"2024-04-25T00:05:36.104792+0000","last_undegraded":"2024-04-25T00:05:36.104792+0000","last_fullsized":"2024-04-25T00:05:36.104792+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.499735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114235+0000","last_change":"2024-04-25T00:05:36.112089+0000","last_active":"2024-04-25T00:05:36.114235+0000","last_peered":"2024-04-25T00:05:36.114235+0000","last_clean":"2024-04-25T00:05:36.114235+0000","last_became_active":"2024-04-25T00:05:36.111613+0000","last_became_peered":"2024-04-25T00:05:36.111613+0000","last_unstale":"2024-04-25T00:05:36.114235+0000","last_undegraded":"2024-04-25T00:05:36.114235+0000","last_fullsized":"2024-04-25T00:05:36.114235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:18.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116691+0000","last_change":"2024-04-25T00:05:36.109577+0000","last_active":"2024-04-25T00:05:36.116691+0000","last_peered":"2024-04-25T00:05:36.116691+0000","last_clean":"2024-04-25T00:05:36.116691+0000","last_became_active":"2024-04-25T00:05:36.109125+0000","last_became_peered":"2024-04-25T00:05:36.109125+0000","last_unstale":"2024-04-25T00:05:36.116691+0000","last_undegraded":"2024-04-25T00:05:36.116691+0000","last_fullsized":"2024-04-25T00:05:36.116691+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:00.477017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114031+0000","last_change":"2024-04-25T00:05:36.109601+0000","last_active":"2024-04-25T00:05:36.114031+0000","last_peered":"2024-04-25T00:05:36.114031+0000","last_clean":"2024-04-25T00:05:36.114031+0000","last_became_active":"2024-04-25T00:05:36.109363+0000","last_became_peered":"2024-04-25T00:05:36.109363+0000","last_unstale":"2024-04-25T00:05:36.114031+0000","last_undegraded":"2024-04-25T00:05:36.114031+0000","last_fullsized":"2024-04-25T00:05:36.114031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:36.448728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108026+0000","last_change":"2024-04-25T00:05:36.105233+0000","last_active":"2024-04-25T00:05:36.108026+0000","last_peered":"2024-04-25T00:05:36.108026+0000","last_clean":"2024-04-25T00:05:36.108026+0000","last_became_active":"2024-04-25T00:05:36.104654+0000","last_became_peered":"2024-04-25T00:05:36.104654+0000","last_unstale":"2024-04-25T00:05:36.108026+0000","last_undegraded":"2024-04-25T00:05:36.108026+0000","last_fullsized":"2024-04-25T00:05:36.108026+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:26.761672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107807+0000","last_change":"2024-04-25T00:05:36.104487+0000","last_active":"2024-04-25T00:05:36.107807+0000","last_peered":"2024-04-25T00:05:36.107807+0000","last_clean":"2024-04-25T00:05:36.107807+0000","last_became_active":"2024-04-25T00:05:36.103697+0000","last_became_peered":"2024-04-25T00:05:36.103697+0000","last_unstale":"2024-04-25T00:05:36.107807+0000","last_undegraded":"2024-04-25T00:05:36.107807+0000","last_fullsized":"2024-04-25T00:05:36.107807+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:40.871259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.106468+0000","last_change":"2024-04-25T00:05:36.105450+0000","last_active":"2024-04-25T00:05:36.106468+0000","last_peered":"2024-04-25T00:05:36.106468+0000","last_clean":"2024-04-25T00:05:36.106468+0000","last_became_active":"2024-04-25T00:05:36.105107+0000","last_became_peered":"2024-04-25T00:05:36.105107+0000","last_unstale":"2024-04-25T00:05:36.106468+0000","last_undegraded":"2024-04-25T00:05:36.106468+0000","last_fullsized":"2024-04-25T00:05:36.106468+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:36.121040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117602+0000","last_change":"2024-04-25T00:05:36.110465+0000","last_active":"2024-04-25T00:05:36.117602+0000","last_peered":"2024-04-25T00:05:36.117602+0000","last_clean":"2024-04-25T00:05:36.117602+0000","last_became_active":"2024-04-25T00:05:36.110207+0000","last_became_peered":"2024-04-25T00:05:36.110207+0000","last_unstale":"2024-04-25T00:05:36.117602+0000","last_undegraded":"2024-04-25T00:05:36.117602+0000","last_fullsized":"2024-04-25T00:05:36.117602+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:05.416974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107581+0000","last_change":"2024-04-25T00:05:36.106530+0000","last_active":"2024-04-25T00:05:36.107581+0000","last_peered":"2024-04-25T00:05:36.107581+0000","last_clean":"2024-04-25T00:05:36.107581+0000","last_became_active":"2024-04-25T00:05:36.106283+0000","last_became_peered":"2024-04-25T00:05:36.106283+0000","last_unstale":"2024-04-25T00:05:36.107581+0000","last_undegraded":"2024-04-25T00:05:36.107581+0000","last_fullsized":"2024-04-25T00:05:36.107581+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:14.188450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107656+0000","last_change":"2024-04-25T00:05:36.104046+0000","last_active":"2024-04-25T00:05:36.107656+0000","last_peered":"2024-04-25T00:05:36.107656+0000","last_clean":"2024-04-25T00:05:36.107656+0000","last_became_active":"2024-04-25T00:05:36.103796+0000","last_became_peered":"2024-04-25T00:05:36.103796+0000","last_unstale":"2024-04-25T00:05:36.107656+0000","last_undegraded":"2024-04-25T00:05:36.107656+0000","last_fullsized":"2024-04-25T00:05:36.107656+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:11.839589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119427+0000","last_change":"2024-04-25T00:05:35.091859+0000","last_active":"2024-04-25T00:05:36.119427+0000","last_peered":"2024-04-25T00:05:36.119427+0000","last_clean":"2024-04-25T00:05:36.119427+0000","last_became_active":"2024-04-25T00:05:35.091165+0000","last_became_peered":"2024-04-25T00:05:35.091165+0000","last_unstale":"2024-04-25T00:05:36.119427+0000","last_undegraded":"2024-04-25T00:05:36.119427+0000","last_fullsized":"2024-04-25T00:05:36.119427+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:52.326992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.113944+0000","last_change":"2024-04-25T00:05:36.109492+0000","last_active":"2024-04-25T00:05:36.113944+0000","last_peered":"2024-04-25T00:05:36.113944+0000","last_clean":"2024-04-25T00:05:36.113944+0000","last_became_active":"2024-04-25T00:05:36.109000+0000","last_became_peered":"2024-04-25T00:05:36.109000+0000","last_unstale":"2024-04-25T00:05:36.113944+0000","last_undegraded":"2024-04-25T00:05:36.113944+0000","last_fullsized":"2024-04-25T00:05:36.113944+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:35.094475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119632+0000","last_change":"2024-04-25T00:05:35.094752+0000","last_active":"2024-04-25T00:05:36.119632+0000","last_peered":"2024-04-25T00:05:36.119632+0000","last_clean":"2024-04-25T00:05:36.119632+0000","last_became_active":"2024-04-25T00:05:35.094403+0000","last_became_peered":"2024-04-25T00:05:35.094403+0000","last_unstale":"2024-04-25T00:05:36.119632+0000","last_undegraded":"2024-04-25T00:05:36.119632+0000","last_fullsized":"2024-04-25T00:05:36.119632+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:43.335618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107548+0000","last_change":"2024-04-25T00:05:36.106019+0000","last_active":"2024-04-25T00:05:36.107548+0000","last_peered":"2024-04-25T00:05:36.107548+0000","last_clean":"2024-04-25T00:05:36.107548+0000","last_became_active":"2024-04-25T00:05:36.105749+0000","last_became_peered":"2024-04-25T00:05:36.105749+0000","last_unstale":"2024-04-25T00:05:36.107548+0000","last_undegraded":"2024-04-25T00:05:36.107548+0000","last_fullsized":"2024-04-25T00:05:36.107548+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:40.851368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.120449+0000","last_change":"2024-04-25T00:05:35.095226+0000","last_active":"2024-04-25T00:05:36.120449+0000","last_peered":"2024-04-25T00:05:36.120449+0000","last_clean":"2024-04-25T00:05:36.120449+0000","last_became_active":"2024-04-25T00:05:35.094921+0000","last_became_peered":"2024-04-25T00:05:35.094921+0000","last_unstale":"2024-04-25T00:05:36.120449+0000","last_undegraded":"2024-04-25T00:05:36.120449+0000","last_fullsized":"2024-04-25T00:05:36.120449+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:13.183434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111895+0000","last_change":"2024-04-25T00:05:36.110663+0000","last_active":"2024-04-25T00:05:36.111895+0000","last_peered":"2024-04-25T00:05:36.111895+0000","last_clean":"2024-04-25T00:05:36.111895+0000","last_became_active":"2024-04-25T00:05:36.110385+0000","last_became_peered":"2024-04-25T00:05:36.110385+0000","last_unstale":"2024-04-25T00:05:36.111895+0000","last_undegraded":"2024-04-25T00:05:36.111895+0000","last_fullsized":"2024-04-25T00:05:36.111895+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:29.921695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.120897+0000","last_change":"2024-04-25T00:05:35.090579+0000","last_active":"2024-04-25T00:05:36.120897+0000","last_peered":"2024-04-25T00:05:36.120897+0000","last_clean":"2024-04-25T00:05:36.120897+0000","last_became_active":"2024-04-25T00:05:35.089876+0000","last_became_peered":"2024-04-25T00:05:35.089876+0000","last_unstale":"2024-04-25T00:05:36.120897+0000","last_undegraded":"2024-04-25T00:05:36.120897+0000","last_fullsized":"2024-04-25T00:05:36.120897+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:10.834573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108383+0000","last_change":"2024-04-25T00:05:36.102861+0000","last_active":"2024-04-25T00:05:36.108383+0000","last_peered":"2024-04-25T00:05:36.108383+0000","last_clean":"2024-04-25T00:05:36.108383+0000","last_became_active":"2024-04-25T00:05:36.101697+0000","last_became_peered":"2024-04-25T00:05:36.101697+0000","last_unstale":"2024-04-25T00:05:36.108383+0000","last_undegraded":"2024-04-25T00:05:36.108383+0000","last_fullsized":"2024-04-25T00:05:36.108383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.857589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118149+0000","last_change":"2024-04-25T00:05:35.092139+0000","last_active":"2024-04-25T00:05:36.118149+0000","last_peered":"2024-04-25T00:05:36.118149+0000","last_clean":"2024-04-25T00:05:36.118149+0000","last_became_active":"2024-04-25T00:05:35.091849+0000","last_became_peered":"2024-04-25T00:05:35.091849+0000","last_unstale":"2024-04-25T00:05:36.118149+0000","last_undegraded":"2024-04-25T00:05:36.118149+0000","last_fullsized":"2024-04-25T00:05:36.118149+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:29.454077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105466+0000","last_change":"2024-04-25T00:05:36.100905+0000","last_active":"2024-04-25T00:05:36.105466+0000","last_peered":"2024-04-25T00:05:36.105466+0000","last_clean":"2024-04-25T00:05:36.105466+0000","last_became_active":"2024-04-25T00:05:36.100379+0000","last_became_peered":"2024-04-25T00:05:36.100379+0000","last_unstale":"2024-04-25T00:05:36.105466+0000","last_undegraded":"2024-04-25T00:05:36.105466+0000","last_fullsized":"2024-04-25T00:05:36.105466+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:08.846207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.122617+0000","last_change":"2024-04-25T00:05:35.090638+0000","last_active":"2024-04-25T00:05:36.122617+0000","last_peered":"2024-04-25T00:05:36.122617+0000","last_clean":"2024-04-25T00:05:36.122617+0000","last_became_active":"2024-04-25T00:05:35.090137+0000","last_became_peered":"2024-04-25T00:05:35.090137+0000","last_unstale":"2024-04-25T00:05:36.122617+0000","last_undegraded":"2024-04-25T00:05:36.122617+0000","last_fullsized":"2024-04-25T00:05:36.122617+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:17.329284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116614+0000","last_change":"2024-04-25T00:05:36.109055+0000","last_active":"2024-04-25T00:05:36.116614+0000","last_peered":"2024-04-25T00:05:36.116614+0000","last_clean":"2024-04-25T00:05:36.116614+0000","last_became_active":"2024-04-25T00:05:36.108633+0000","last_became_peered":"2024-04-25T00:05:36.108633+0000","last_unstale":"2024-04-25T00:05:36.116614+0000","last_undegraded":"2024-04-25T00:05:36.116614+0000","last_fullsized":"2024-04-25T00:05:36.116614+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:26.705375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.094550+0000","last_change":"2024-04-25T00:05:35.088080+0000","last_active":"2024-04-25T00:05:36.094550+0000","last_peered":"2024-04-25T00:05:36.094550+0000","last_clean":"2024-04-25T00:05:36.094550+0000","last_became_active":"2024-04-25T00:05:35.087730+0000","last_became_peered":"2024-04-25T00:05:35.087730+0000","last_unstale":"2024-04-25T00:05:36.094550+0000","last_undegraded":"2024-04-25T00:05:36.094550+0000","last_fullsized":"2024-04-25T00:05:36.094550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:08.665848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.115348+0000","last_change":"2024-04-25T00:05:36.114352+0000","last_active":"2024-04-25T00:05:36.115348+0000","last_peered":"2024-04-25T00:05:36.115348+0000","last_clean":"2024-04-25T00:05:36.115348+0000","last_became_active":"2024-04-25T00:05:36.113856+0000","last_became_peered":"2024-04-25T00:05:36.113856+0000","last_unstale":"2024-04-25T00:05:36.115348+0000","last_undegraded":"2024-04-25T00:05:36.115348+0000","last_fullsized":"2024-04-25T00:05:36.115348+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:21.118927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098289+0000","last_change":"2024-04-25T00:05:35.089059+0000","last_active":"2024-04-25T00:05:36.098289+0000","last_peered":"2024-04-25T00:05:36.098289+0000","last_clean":"2024-04-25T00:05:36.098289+0000","last_became_active":"2024-04-25T00:05:35.088785+0000","last_became_peered":"2024-04-25T00:05:35.088785+0000","last_unstale":"2024-04-25T00:05:36.098289+0000","last_undegraded":"2024-04-25T00:05:36.098289+0000","last_fullsized":"2024-04-25T00:05:36.098289+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:57.118187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.104425+0000","last_change":"2024-04-25T00:05:36.103796+0000","last_active":"2024-04-25T00:05:36.104425+0000","last_peered":"2024-04-25T00:05:36.104425+0000","last_clean":"2024-04-25T00:05:36.104425+0000","last_became_active":"2024-04-25T00:05:36.103525+0000","last_became_peered":"2024-04-25T00:05:36.103525+0000","last_unstale":"2024-04-25T00:05:36.104425+0000","last_undegraded":"2024-04-25T00:05:36.104425+0000","last_fullsized":"2024-04-25T00:05:36.104425+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:12.814340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.093571+0000","last_change":"2024-04-25T00:05:35.088863+0000","last_active":"2024-04-25T00:05:36.093571+0000","last_peered":"2024-04-25T00:05:36.093571+0000","last_clean":"2024-04-25T00:05:36.093571+0000","last_became_active":"2024-04-25T00:05:35.088165+0000","last_became_peered":"2024-04-25T00:05:35.088165+0000","last_unstale":"2024-04-25T00:05:36.093571+0000","last_undegraded":"2024-04-25T00:05:36.093571+0000","last_fullsized":"2024-04-25T00:05:36.093571+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:01.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.113965+0000","last_change":"2024-04-25T00:05:36.110806+0000","last_active":"2024-04-25T00:05:36.113965+0000","last_peered":"2024-04-25T00:05:36.113965+0000","last_clean":"2024-04-25T00:05:36.113965+0000","last_became_active":"2024-04-25T00:05:36.110545+0000","last_became_peered":"2024-04-25T00:05:36.110545+0000","last_unstale":"2024-04-25T00:05:36.113965+0000","last_undegraded":"2024-04-25T00:05:36.113965+0000","last_fullsized":"2024-04-25T00:05:36.113965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:37.688391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.096590+0000","last_change":"2024-04-25T00:05:35.090833+0000","last_active":"2024-04-25T00:05:36.096590+0000","last_peered":"2024-04-25T00:05:36.096590+0000","last_clean":"2024-04-25T00:05:36.096590+0000","last_became_active":"2024-04-25T00:05:35.090468+0000","last_became_peered":"2024-04-25T00:05:35.090468+0000","last_unstale":"2024-04-25T00:05:36.096590+0000","last_undegraded":"2024-04-25T00:05:36.096590+0000","last_fullsized":"2024-04-25T00:05:36.096590+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:15.797240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111768+0000","last_change":"2024-04-25T00:05:36.107654+0000","last_active":"2024-04-25T00:05:36.111768+0000","last_peered":"2024-04-25T00:05:36.111768+0000","last_clean":"2024-04-25T00:05:36.111768+0000","last_became_active":"2024-04-25T00:05:36.107066+0000","last_became_peered":"2024-04-25T00:05:36.107066+0000","last_unstale":"2024-04-25T00:05:36.111768+0000","last_undegraded":"2024-04-25T00:05:36.111768+0000","last_fullsized":"2024-04-25T00:05:36.111768+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:38.927057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"115'3","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118968+0000","last_change":"2024-04-25T00:05:35.094234+0000","last_active":"2024-04-25T00:05:36.118968+0000","last_peered":"2024-04-25T00:05:36.118968+0000","last_clean":"2024-04-25T00:05:36.118968+0000","last_became_active":"2024-04-25T00:05:35.093973+0000","last_became_peered":"2024-04-25T00:05:35.093973+0000","last_unstale":"2024-04-25T00:05:36.118968+0000","last_undegraded":"2024-04-25T00:05:36.118968+0000","last_fullsized":"2024-04-25T00:05:36.118968+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:43.335618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.106311+0000","last_change":"2024-04-25T00:05:36.105354+0000","last_active":"2024-04-25T00:05:36.106311+0000","last_peered":"2024-04-25T00:05:36.106311+0000","last_clean":"2024-04-25T00:05:36.106311+0000","last_became_active":"2024-04-25T00:05:36.104956+0000","last_became_peered":"2024-04-25T00:05:36.104956+0000","last_unstale":"2024-04-25T00:05:36.106311+0000","last_undegraded":"2024-04-25T00:05:36.106311+0000","last_fullsized":"2024-04-25T00:05:36.106311+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.059079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"115'6","reported_seq":18,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.123007+0000","last_change":"2024-04-25T00:05:35.088052+0000","last_active":"2024-04-25T00:05:36.123007+0000","last_peered":"2024-04-25T00:05:36.123007+0000","last_clean":"2024-04-25T00:05:36.123007+0000","last_became_active":"2024-04-25T00:05:35.087751+0000","last_became_peered":"2024-04-25T00:05:35.087751+0000","last_unstale":"2024-04-25T00:05:36.123007+0000","last_undegraded":"2024-04-25T00:05:36.123007+0000","last_fullsized":"2024-04-25T00:05:36.123007+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:16.887883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107196+0000","last_change":"2024-04-25T00:05:36.105386+0000","last_active":"2024-04-25T00:05:36.107196+0000","last_peered":"2024-04-25T00:05:36.107196+0000","last_clean":"2024-04-25T00:05:36.107196+0000","last_became_active":"2024-04-25T00:05:36.104979+0000","last_became_peered":"2024-04-25T00:05:36.104979+0000","last_unstale":"2024-04-25T00:05:36.107196+0000","last_undegraded":"2024-04-25T00:05:36.107196+0000","last_fullsized":"2024-04-25T00:05:36.107196+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:44.340634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117161+0000","last_change":"2024-04-25T00:05:35.090419+0000","last_active":"2024-04-25T00:05:36.117161+0000","last_peered":"2024-04-25T00:05:36.117161+0000","last_clean":"2024-04-25T00:05:36.117161+0000","last_became_active":"2024-04-25T00:05:35.090187+0000","last_became_peered":"2024-04-25T00:05:35.090187+0000","last_unstale":"2024-04-25T00:05:36.117161+0000","last_undegraded":"2024-04-25T00:05:36.117161+0000","last_fullsized":"2024-04-25T00:05:36.117161+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:34.200146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105219+0000","last_change":"2024-04-25T00:05:36.103744+0000","last_active":"2024-04-25T00:05:36.105219+0000","last_peered":"2024-04-25T00:05:36.105219+0000","last_clean":"2024-04-25T00:05:36.105219+0000","last_became_active":"2024-04-25T00:05:36.103494+0000","last_became_peered":"2024-04-25T00:05:36.103494+0000","last_unstale":"2024-04-25T00:05:36.105219+0000","last_undegraded":"2024-04-25T00:05:36.105219+0000","last_fullsized":"2024-04-25T00:05:36.105219+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.276236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.097819+0000","last_change":"2024-04-25T00:05:35.095367+0000","last_active":"2024-04-25T00:05:36.097819+0000","last_peered":"2024-04-25T00:05:36.097819+0000","last_clean":"2024-04-25T00:05:36.097819+0000","last_became_active":"2024-04-25T00:05:35.095077+0000","last_became_peered":"2024-04-25T00:05:35.095077+0000","last_unstale":"2024-04-25T00:05:36.097819+0000","last_undegraded":"2024-04-25T00:05:36.097819+0000","last_fullsized":"2024-04-25T00:05:36.097819+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:08.665848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105758+0000","last_change":"2024-04-25T00:05:36.104474+0000","last_active":"2024-04-25T00:05:36.105758+0000","last_peered":"2024-04-25T00:05:36.105758+0000","last_clean":"2024-04-25T00:05:36.105758+0000","last_became_active":"2024-04-25T00:05:36.103894+0000","last_became_peered":"2024-04-25T00:05:36.103894+0000","last_unstale":"2024-04-25T00:05:36.105758+0000","last_undegraded":"2024-04-25T00:05:36.105758+0000","last_fullsized":"2024-04-25T00:05:36.105758+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:53.422990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":20,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.123612+0000","last_change":"2024-04-25T00:05:35.093102+0000","last_active":"2024-04-25T00:05:36.123612+0000","last_peered":"2024-04-25T00:05:36.123612+0000","last_clean":"2024-04-25T00:05:36.123612+0000","last_became_active":"2024-04-25T00:05:35.092841+0000","last_became_peered":"2024-04-25T00:05:35.092841+0000","last_unstale":"2024-04-25T00:05:36.123612+0000","last_undegraded":"2024-04-25T00:05:36.123612+0000","last_fullsized":"2024-04-25T00:05:36.123612+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:39.846352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114144+0000","last_change":"2024-04-25T00:05:36.110597+0000","last_active":"2024-04-25T00:05:36.114144+0000","last_peered":"2024-04-25T00:05:36.114144+0000","last_clean":"2024-04-25T00:05:36.114144+0000","last_became_active":"2024-04-25T00:05:36.110345+0000","last_became_peered":"2024-04-25T00:05:36.110345+0000","last_unstale":"2024-04-25T00:05:36.114144+0000","last_undegraded":"2024-04-25T00:05:36.114144+0000","last_fullsized":"2024-04-25T00:05:36.114144+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:22.047635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118530+0000","last_change":"2024-04-25T00:05:35.092034+0000","last_active":"2024-04-25T00:05:36.118530+0000","last_peered":"2024-04-25T00:05:36.118530+0000","last_clean":"2024-04-25T00:05:36.118530+0000","last_became_active":"2024-04-25T00:05:35.091777+0000","last_became_peered":"2024-04-25T00:05:35.091777+0000","last_unstale":"2024-04-25T00:05:36.118530+0000","last_undegraded":"2024-04-25T00:05:36.118530+0000","last_fullsized":"2024-04-25T00:05:36.118530+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:21.141145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.112217+0000","last_change":"2024-04-25T00:05:36.108682+0000","last_active":"2024-04-25T00:05:36.112217+0000","last_peered":"2024-04-25T00:05:36.112217+0000","last_clean":"2024-04-25T00:05:36.112217+0000","last_became_active":"2024-04-25T00:05:36.108439+0000","last_became_peered":"2024-04-25T00:05:36.108439+0000","last_unstale":"2024-04-25T00:05:36.112217+0000","last_undegraded":"2024-04-25T00:05:36.112217+0000","last_fullsized":"2024-04-25T00:05:36.112217+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:13.118263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119494+0000","last_change":"2024-04-25T00:05:35.085470+0000","last_active":"2024-04-25T00:05:36.119494+0000","last_peered":"2024-04-25T00:05:36.119494+0000","last_clean":"2024-04-25T00:05:36.119494+0000","last_became_active":"2024-04-25T00:05:35.085220+0000","last_became_peered":"2024-04-25T00:05:35.085220+0000","last_unstale":"2024-04-25T00:05:36.119494+0000","last_undegraded":"2024-04-25T00:05:36.119494+0000","last_fullsized":"2024-04-25T00:05:36.119494+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:12.113247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116530+0000","last_change":"2024-04-25T00:05:36.115342+0000","last_active":"2024-04-25T00:05:36.116530+0000","last_peered":"2024-04-25T00:05:36.116530+0000","last_clean":"2024-04-25T00:05:36.116530+0000","last_became_active":"2024-04-25T00:05:36.114994+0000","last_became_peered":"2024-04-25T00:05:36.114994+0000","last_unstale":"2024-04-25T00:05:36.116530+0000","last_undegraded":"2024-04-25T00:05:36.116530+0000","last_fullsized":"2024-04-25T00:05:36.116530+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:12.701990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.122862+0000","last_change":"2024-04-25T00:05:35.089292+0000","last_active":"2024-04-25T00:05:36.122862+0000","last_peered":"2024-04-25T00:05:36.122862+0000","last_clean":"2024-04-25T00:05:36.122862+0000","last_became_active":"2024-04-25T00:05:35.088951+0000","last_became_peered":"2024-04-25T00:05:35.088951+0000","last_unstale":"2024-04-25T00:05:36.122862+0000","last_undegraded":"2024-04-25T00:05:36.122862+0000","last_fullsized":"2024-04-25T00:05:36.122862+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:07.841191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110575+0000","last_change":"2024-04-25T00:05:36.106877+0000","last_active":"2024-04-25T00:05:36.110575+0000","last_peered":"2024-04-25T00:05:36.110575+0000","last_clean":"2024-04-25T00:05:36.110575+0000","last_became_active":"2024-04-25T00:05:36.106519+0000","last_became_peered":"2024-04-25T00:05:36.106519+0000","last_unstale":"2024-04-25T00:05:36.110575+0000","last_undegraded":"2024-04-25T00:05:36.110575+0000","last_fullsized":"2024-04-25T00:05:36.110575+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:17.892899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.097834+0000","last_change":"2024-04-25T00:05:35.093434+0000","last_active":"2024-04-25T00:05:36.097834+0000","last_peered":"2024-04-25T00:05:36.097834+0000","last_clean":"2024-04-25T00:05:36.097834+0000","last_became_active":"2024-04-25T00:05:35.093126+0000","last_became_peered":"2024-04-25T00:05:35.093126+0000","last_unstale":"2024-04-25T00:05:36.097834+0000","last_undegraded":"2024-04-25T00:05:36.097834+0000","last_fullsized":"2024-04-25T00:05:36.097834+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:24.987213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111869+0000","last_change":"2024-04-25T00:05:36.109528+0000","last_active":"2024-04-25T00:05:36.111869+0000","last_peered":"2024-04-25T00:05:36.111869+0000","last_clean":"2024-04-25T00:05:36.111869+0000","last_became_active":"2024-04-25T00:05:36.108450+0000","last_became_peered":"2024-04-25T00:05:36.108450+0000","last_unstale":"2024-04-25T00:05:36.111869+0000","last_undegraded":"2024-04-25T00:05:36.111869+0000","last_fullsized":"2024-04-25T00:05:36.111869+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.857589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117407+0000","last_change":"2024-04-25T00:05:35.089429+0000","last_active":"2024-04-25T00:05:36.117407+0000","last_peered":"2024-04-25T00:05:36.117407+0000","last_clean":"2024-04-25T00:05:36.117407+0000","last_became_active":"2024-04-25T00:05:35.089148+0000","last_became_peered":"2024-04-25T00:05:35.089148+0000","last_unstale":"2024-04-25T00:05:36.117407+0000","last_undegraded":"2024-04-25T00:05:36.117407+0000","last_fullsized":"2024-04-25T00:05:36.117407+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:11.057511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108485+0000","last_change":"2024-04-25T00:05:36.103675+0000","last_active":"2024-04-25T00:05:36.108485+0000","last_peered":"2024-04-25T00:05:36.108485+0000","last_clean":"2024-04-25T00:05:36.108485+0000","last_became_active":"2024-04-25T00:05:36.103419+0000","last_became_peered":"2024-04-25T00:05:36.103419+0000","last_unstale":"2024-04-25T00:05:36.108485+0000","last_undegraded":"2024-04-25T00:05:36.108485+0000","last_fullsized":"2024-04-25T00:05:36.108485+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:29.921695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.093550+0000","last_change":"2024-04-25T00:05:35.088725+0000","last_active":"2024-04-25T00:05:36.093550+0000","last_peered":"2024-04-25T00:05:36.093550+0000","last_clean":"2024-04-25T00:05:36.093550+0000","last_became_active":"2024-04-25T00:05:35.088485+0000","last_became_peered":"2024-04-25T00:05:35.088485+0000","last_unstale":"2024-04-25T00:05:36.093550+0000","last_undegraded":"2024-04-25T00:05:36.093550+0000","last_fullsized":"2024-04-25T00:05:36.093550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:02.911753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107893+0000","last_change":"2024-04-25T00:05:36.104271+0000","last_active":"2024-04-25T00:05:36.107893+0000","last_peered":"2024-04-25T00:05:36.107893+0000","last_clean":"2024-04-25T00:05:36.107893+0000","last_became_active":"2024-04-25T00:05:36.103983+0000","last_became_peered":"2024-04-25T00:05:36.103983+0000","last_unstale":"2024-04-25T00:05:36.107893+0000","last_undegraded":"2024-04-25T00:05:36.107893+0000","last_fullsized":"2024-04-25T00:05:36.107893+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:38.927057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.099227+0000","last_change":"2024-04-25T00:05:35.089785+0000","last_active":"2024-04-25T00:05:36.099227+0000","last_peered":"2024-04-25T00:05:36.099227+0000","last_clean":"2024-04-25T00:05:36.099227+0000","last_became_active":"2024-04-25T00:05:35.089537+0000","last_became_peered":"2024-04-25T00:05:35.089537+0000","last_unstale":"2024-04-25T00:05:36.099227+0000","last_undegraded":"2024-04-25T00:05:36.099227+0000","last_fullsized":"2024-04-25T00:05:36.099227+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:59.493199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105777+0000","last_change":"2024-04-25T00:05:36.104803+0000","last_active":"2024-04-25T00:05:36.105777+0000","last_peered":"2024-04-25T00:05:36.105777+0000","last_clean":"2024-04-25T00:05:36.105777+0000","last_became_active":"2024-04-25T00:05:36.104448+0000","last_became_peered":"2024-04-25T00:05:36.104448+0000","last_unstale":"2024-04-25T00:05:36.105777+0000","last_undegraded":"2024-04-25T00:05:36.105777+0000","last_fullsized":"2024-04-25T00:05:36.105777+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.669634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098425+0000","last_change":"2024-04-25T00:05:35.092040+0000","last_active":"2024-04-25T00:05:36.098425+0000","last_peered":"2024-04-25T00:05:36.098425+0000","last_clean":"2024-04-25T00:05:36.098425+0000","last_became_active":"2024-04-25T00:05:35.091734+0000","last_became_peered":"2024-04-25T00:05:35.091734+0000","last_unstale":"2024-04-25T00:05:36.098425+0000","last_undegraded":"2024-04-25T00:05:36.098425+0000","last_fullsized":"2024-04-25T00:05:36.098425+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:58.880900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119117+0000","last_change":"2024-04-25T00:05:36.118073+0000","last_active":"2024-04-25T00:05:36.119117+0000","last_peered":"2024-04-25T00:05:36.119117+0000","last_clean":"2024-04-25T00:05:36.119117+0000","last_became_active":"2024-04-25T00:05:36.117744+0000","last_became_peered":"2024-04-25T00:05:36.117744+0000","last_unstale":"2024-04-25T00:05:36.119117+0000","last_undegraded":"2024-04-25T00:05:36.119117+0000","last_fullsized":"2024-04-25T00:05:36.119117+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:16.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.122704+0000","last_change":"2024-04-25T00:05:35.093246+0000","last_active":"2024-04-25T00:05:36.122704+0000","last_peered":"2024-04-25T00:05:36.122704+0000","last_clean":"2024-04-25T00:05:36.122704+0000","last_became_active":"2024-04-25T00:05:35.092946+0000","last_became_peered":"2024-04-25T00:05:35.092946+0000","last_unstale":"2024-04-25T00:05:36.122704+0000","last_undegraded":"2024-04-25T00:05:36.122704+0000","last_fullsized":"2024-04-25T00:05:36.122704+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:17.909900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.106293+0000","last_change":"2024-04-25T00:05:36.100054+0000","last_active":"2024-04-25T00:05:36.106293+0000","last_peered":"2024-04-25T00:05:36.106293+0000","last_clean":"2024-04-25T00:05:36.106293+0000","last_became_active":"2024-04-25T00:05:36.099801+0000","last_became_peered":"2024-04-25T00:05:36.099801+0000","last_unstale":"2024-04-25T00:05:36.106293+0000","last_undegraded":"2024-04-25T00:05:36.106293+0000","last_fullsized":"2024-04-25T00:05:36.106293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:59.169658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098541+0000","last_change":"2024-04-25T00:05:35.093353+0000","last_active":"2024-04-25T00:05:36.098541+0000","last_peered":"2024-04-25T00:05:36.098541+0000","last_clean":"2024-04-25T00:05:36.098541+0000","last_became_active":"2024-04-25T00:05:35.092936+0000","last_became_peered":"2024-04-25T00:05:35.092936+0000","last_unstale":"2024-04-25T00:05:36.098541+0000","last_undegraded":"2024-04-25T00:05:36.098541+0000","last_fullsized":"2024-04-25T00:05:36.098541+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:12.044548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116574+0000","last_change":"2024-04-25T00:05:36.109132+0000","last_active":"2024-04-25T00:05:36.116574+0000","last_peered":"2024-04-25T00:05:36.116574+0000","last_clean":"2024-04-25T00:05:36.116574+0000","last_became_active":"2024-04-25T00:05:36.108323+0000","last_became_peered":"2024-04-25T00:05:36.108323+0000","last_unstale":"2024-04-25T00:05:36.116574+0000","last_undegraded":"2024-04-25T00:05:36.116574+0000","last_fullsized":"2024-04-25T00:05:36.116574+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:22.705213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.094149+0000","last_change":"2024-04-25T00:05:35.091713+0000","last_active":"2024-04-25T00:05:36.094149+0000","last_peered":"2024-04-25T00:05:36.094149+0000","last_clean":"2024-04-25T00:05:36.094149+0000","last_became_active":"2024-04-25T00:05:35.091271+0000","last_became_peered":"2024-04-25T00:05:35.091271+0000","last_unstale":"2024-04-25T00:05:36.094149+0000","last_undegraded":"2024-04-25T00:05:36.094149+0000","last_fullsized":"2024-04-25T00:05:36.094149+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:37.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.112512+0000","last_change":"2024-04-25T00:05:36.110156+0000","last_active":"2024-04-25T00:05:36.112512+0000","last_peered":"2024-04-25T00:05:36.112512+0000","last_clean":"2024-04-25T00:05:36.112512+0000","last_became_active":"2024-04-25T00:05:36.109897+0000","last_became_peered":"2024-04-25T00:05:36.109897+0000","last_unstale":"2024-04-25T00:05:36.112512+0000","last_undegraded":"2024-04-25T00:05:36.112512+0000","last_fullsized":"2024-04-25T00:05:36.112512+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:49.814589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.097997+0000","last_change":"2024-04-25T00:05:35.093134+0000","last_active":"2024-04-25T00:05:36.097997+0000","last_peered":"2024-04-25T00:05:36.097997+0000","last_clean":"2024-04-25T00:05:36.097997+0000","last_became_active":"2024-04-25T00:05:35.092413+0000","last_became_peered":"2024-04-25T00:05:35.092413+0000","last_unstale":"2024-04-25T00:05:36.097997+0000","last_undegraded":"2024-04-25T00:05:36.097997+0000","last_fullsized":"2024-04-25T00:05:36.097997+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:02.770290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107838+0000","last_change":"2024-04-25T00:05:36.102745+0000","last_active":"2024-04-25T00:05:36.107838+0000","last_peered":"2024-04-25T00:05:36.107838+0000","last_clean":"2024-04-25T00:05:36.107838+0000","last_became_active":"2024-04-25T00:05:36.102374+0000","last_became_peered":"2024-04-25T00:05:36.102374+0000","last_unstale":"2024-04-25T00:05:36.107838+0000","last_undegraded":"2024-04-25T00:05:36.107838+0000","last_fullsized":"2024-04-25T00:05:36.107838+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:49.814589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.099299+0000","last_change":"2024-04-25T00:05:35.092899+0000","last_active":"2024-04-25T00:05:36.099299+0000","last_peered":"2024-04-25T00:05:36.099299+0000","last_clean":"2024-04-25T00:05:36.099299+0000","last_became_active":"2024-04-25T00:05:35.092639+0000","last_became_peered":"2024-04-25T00:05:35.092639+0000","last_unstale":"2024-04-25T00:05:36.099299+0000","last_undegraded":"2024-04-25T00:05:36.099299+0000","last_fullsized":"2024-04-25T00:05:36.099299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:07.621813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105488+0000","last_change":"2024-04-25T00:05:36.101463+0000","last_active":"2024-04-25T00:05:36.105488+0000","last_peered":"2024-04-25T00:05:36.105488+0000","last_clean":"2024-04-25T00:05:36.105488+0000","last_became_active":"2024-04-25T00:05:36.101198+0000","last_became_peered":"2024-04-25T00:05:36.101198+0000","last_unstale":"2024-04-25T00:05:36.105488+0000","last_undegraded":"2024-04-25T00:05:36.105488+0000","last_fullsized":"2024-04-25T00:05:36.105488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:35.094475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.094517+0000","last_change":"2024-04-25T00:05:35.088045+0000","last_active":"2024-04-25T00:05:36.094517+0000","last_peered":"2024-04-25T00:05:36.094517+0000","last_clean":"2024-04-25T00:05:36.094517+0000","last_became_active":"2024-04-25T00:05:35.087063+0000","last_became_peered":"2024-04-25T00:05:35.087063+0000","last_unstale":"2024-04-25T00:05:36.094517+0000","last_undegraded":"2024-04-25T00:05:36.094517+0000","last_fullsized":"2024-04-25T00:05:36.094517+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:30.853248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110167+0000","last_change":"2024-04-25T00:05:36.108367+0000","last_active":"2024-04-25T00:05:36.110167+0000","last_peered":"2024-04-25T00:05:36.110167+0000","last_clean":"2024-04-25T00:05:36.110167+0000","last_became_active":"2024-04-25T00:05:36.108091+0000","last_became_peered":"2024-04-25T00:05:36.108091+0000","last_unstale":"2024-04-25T00:05:36.110167+0000","last_undegraded":"2024-04-25T00:05:36.110167+0000","last_fullsized":"2024-04-25T00:05:36.110167+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:59.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.093484+0000","last_change":"2024-04-25T00:05:35.089435+0000","last_active":"2024-04-25T00:05:36.093484+0000","last_peered":"2024-04-25T00:05:36.093484+0000","last_clean":"2024-04-25T00:05:36.093484+0000","last_became_active":"2024-04-25T00:05:35.089190+0000","last_became_peered":"2024-04-25T00:05:35.089190+0000","last_unstale":"2024-04-25T00:05:36.093484+0000","last_undegraded":"2024-04-25T00:05:36.093484+0000","last_fullsized":"2024-04-25T00:05:36.093484+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:59.493199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117577+0000","last_change":"2024-04-25T00:05:36.111080+0000","last_active":"2024-04-25T00:05:36.117577+0000","last_peered":"2024-04-25T00:05:36.117577+0000","last_clean":"2024-04-25T00:05:36.117577+0000","last_became_active":"2024-04-25T00:05:36.110807+0000","last_became_peered":"2024-04-25T00:05:36.110807+0000","last_unstale":"2024-04-25T00:05:36.117577+0000","last_undegraded":"2024-04-25T00:05:36.117577+0000","last_fullsized":"2024-04-25T00:05:36.117577+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:03.090823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098561+0000","last_change":"2024-04-25T00:05:35.090943+0000","last_active":"2024-04-25T00:05:36.098561+0000","last_peered":"2024-04-25T00:05:36.098561+0000","last_clean":"2024-04-25T00:05:36.098561+0000","last_became_active":"2024-04-25T00:05:35.090691+0000","last_became_peered":"2024-04-25T00:05:35.090691+0000","last_unstale":"2024-04-25T00:05:36.098561+0000","last_undegraded":"2024-04-25T00:05:36.098561+0000","last_fullsized":"2024-04-25T00:05:36.098561+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:00:42.872357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.104765+0000","last_change":"2024-04-25T00:05:36.102794+0000","last_active":"2024-04-25T00:05:36.104765+0000","last_peered":"2024-04-25T00:05:36.104765+0000","last_clean":"2024-04-25T00:05:36.104765+0000","last_became_active":"2024-04-25T00:05:36.101472+0000","last_became_peered":"2024-04-25T00:05:36.101472+0000","last_unstale":"2024-04-25T00:05:36.104765+0000","last_undegraded":"2024-04-25T00:05:36.104765+0000","last_fullsized":"2024-04-25T00:05:36.104765+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:51.260575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.099955+0000","last_change":"2024-04-25T00:05:35.093364+0000","last_active":"2024-04-25T00:05:36.099955+0000","last_peered":"2024-04-25T00:05:36.099955+0000","last_clean":"2024-04-25T00:05:36.099955+0000","last_became_active":"2024-04-25T00:05:35.093088+0000","last_became_peered":"2024-04-25T00:05:35.093088+0000","last_unstale":"2024-04-25T00:05:36.099955+0000","last_undegraded":"2024-04-25T00:05:36.099955+0000","last_fullsized":"2024-04-25T00:05:36.099955+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:53.265715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":340,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.100553+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:05:36.100553+0000","last_peered":"2024-04-25T00:05:36.100553+0000","last_clean":"2024-04-25T00:05:36.100553+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:05:36.100553+0000","last_undegraded":"2024-04-25T00:05:36.100553+0000","last_fullsized":"2024-04-25T00:05:36.100553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2011136,"data_stored":1202638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46456,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325384,"statfs":{"total":96636764160,"available":96589193216,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46460,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325380,"statfs":{"total":96636764160,"available":96589189120,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54780,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317060,"statfs":{"total":96636764160,"available":96580669440,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41464,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330376,"statfs":{"total":96636764160,"available":96594305024,"internally_reserved":0,"allocated":2011136,"data_stored":1202638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:05:39.074 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:05:39.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.206+0000 7f150dcae640 1 Processor -- start 2024-04-25T00:05:39.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.206+0000 7f150dcae640 1 -- start start 2024-04-25T00:05:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.206+0000 7f150dcae640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15080d2290 0x7f15080b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.206+0000 7f150dcae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15080d0b00 0x7f15080d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.206+0000 7f150dcae640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15080d2800 con 0x7f150815e320 2024-04-25T00:05:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f150dcae640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f15080d14d0 con 0x7f15080d2290 2024-04-25T00:05:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f150dcae640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f15080d3410 con 0x7f15080d0b00 2024-04-25T00:05:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f1506ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15080d2290 0x7f15080b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f1506ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15080d2290 0x7f15080b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47926/0 (socket says 172.21.15.113:47926) 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f1506ffd640 1 -- 172.21.15.113:0/4278041885 learned_addr learned my addr 172.21.15.113:0/4278041885 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f15077fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f150815e320 0x7f15080d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54982/0 (socket says 172.21.15.113:54982) 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f1506ffd640 1 -- 172.21.15.113:0/4278041885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15080d0b00 msgr2=0x7f15080d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f1506ffd640 1 --2- 172.21.15.113:0/4278041885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15080d0b00 0x7f15080d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f1506ffd640 1 -- 172.21.15.113:0/4278041885 >> v1:172.21.15.113:6789/0 conn(0x7f150815e320 legacy=0x7f15080d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f1506ffd640 1 -- 172.21.15.113:0/4278041885 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15080d2ae0 con 0x7f15080d2290 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.207+0000 7f15067fc640 1 -- 172.21.15.113:0/4278041885 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775903380 0 0) 0x7f15080d2800 con 0x7f150815e320 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f1506ffd640 1 --2- 172.21.15.113:0/4278041885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d2290 0x7f15080b9aa0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009870 tx=0x7f14fc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f71c1896cf01ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f15067fc640 1 -- 172.21.15.113:0/4278041885 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc037070 con 0x7f15080d2290 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f15067fc640 1 -- 172.21.15.113:0/4278041885 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14fc02ae50 con 0x7f15080d2290 2024-04-25T00:05:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f15067fc640 1 -- 172.21.15.113:0/4278041885 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc032860 con 0x7f15080d2290 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 -- 172.21.15.113:0/4278041885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d2290 msgr2=0x7f15080b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 --2- 172.21.15.113:0/4278041885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d2290 0x7f15080b9aa0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009870 tx=0x7f14fc02a300 comp rx=0 tx=0).stop 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 -- 172.21.15.113:0/4278041885 shutdown_connections 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 --2- 172.21.15.113:0/4278041885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15080d0b00 0x7f15080d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 --2- 172.21.15.113:0/4278041885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d2290 0x7f15080b9aa0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 -- 172.21.15.113:0/4278041885 >> 172.21.15.113:0/4278041885 conn(0x7f15080b5c20 msgr2=0x7f15080b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 -- 172.21.15.113:0/4278041885 shutdown_connections 2024-04-25T00:05:39.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.208+0000 7f150dcae640 1 -- 172.21.15.113:0/4278041885 wait complete. 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.209+0000 7f150dcae640 1 Processor -- start 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f150dcae640 1 -- start start 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f150dcae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d0b00 0x7f15080d46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f150dcae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15080d2290 0x7f15080ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f150dcae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f150815e320 0x7f15080cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f150dcae640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f15080b9fe0 con 0x7f15080d0b00 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f150dcae640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15080d30a0 con 0x7f15080d2290 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f150dcae640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f150815e700 con 0x7f150815e320 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f15077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d0b00 0x7f15080d46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f15077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d0b00 0x7f15080d46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47942/0 (socket says 172.21.15.113:47942) 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.210+0000 7f15077fe640 1 -- 172.21.15.113:0/3462723582 learned_addr learned my addr 172.21.15.113:0/3462723582 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f1506ffd640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15080d2290 0x7f15080ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f1507fff640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f150815e320 0x7f15080cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f15077fe640 1 -- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f150815e320 msgr2=0x7f15080cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f15077fe640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f150815e320 0x7f15080cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f15077fe640 1 -- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15080d2290 msgr2=0x7f15080ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f15077fe640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15080d2290 0x7f15080ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f15077fe640 1 -- 172.21.15.113:0/3462723582 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150815ebf0 con 0x7f15080d0b00 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f1506ffd640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15080d2290 0x7f15080ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f15077fe640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d0b00 0x7f15080d46d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c440 tx=0x7f14f800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f14effff640 1 -- 172.21.15.113:0/3462723582 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8018070 con 0x7f15080d0b00 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f14effff640 1 -- 172.21.15.113:0/3462723582 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f80074a0 con 0x7f15080d0b00 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f14effff640 1 -- 172.21.15.113:0/3462723582 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8004a20 con 0x7f15080d0b00 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15080cd7f0 con 0x7f15080d0b00 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.211+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15080cddb0 con 0x7f15080d0b00 2024-04-25T00:05:39.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.212+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150815edc0 con 0x7f15080d0b00 2024-04-25T00:05:39.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.212+0000 7f14effff640 1 -- 172.21.15.113:0/3462723582 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14f801b070 con 0x7f15080d0b00 2024-04-25T00:05:39.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.213+0000 7f14effff640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f14f406be30 0x7f14f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.213+0000 7f14effff640 1 -- 172.21.15.113:0/3462723582 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f14f808a020 con 0x7f15080d0b00 2024-04-25T00:05:39.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.213+0000 7f1506ffd640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f14f406be30 0x7f14f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.213+0000 7f1506ffd640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f14f406be30 0x7f14f406e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f14fc034000 tx=0x7f14fc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.217+0000 7f14effff640 1 -- 172.21.15.113:0/3462723582 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f14f8007650 con 0x7f15080d0b00 2024-04-25T00:05:39.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.388+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15080691a0 con 0x7f14f406be30 2024-04-25T00:05:39.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.393+0000 7f14effff640 1 -- 172.21.15.113:0/3462723582 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252412 (secure 0 0 0) 0x7f15080691a0 con 0x7f14f406be30 2024-04-25T00:05:39.396 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:39.397 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:05:39.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f14f406be30 msgr2=0x7f14f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f14f406be30 0x7f14f406e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f14fc034000 tx=0x7f14fc02e000 comp rx=0 tx=0).stop 2024-04-25T00:05:39.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d0b00 msgr2=0x7f15080d46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d0b00 0x7f15080d46d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c440 tx=0x7f14f800c900 comp rx=0 tx=0).stop 2024-04-25T00:05:39.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 shutdown_connections 2024-04-25T00:05:39.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f14f406be30 0x7f14f406e2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15080d2290 0x7f15080ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f150815e320 0x7f15080cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 --2- 172.21.15.113:0/3462723582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15080d0b00 0x7f15080d46d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.396+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 >> 172.21.15.113:0/3462723582 conn(0x7f15080b5c20 msgr2=0x7f15080b7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:39.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.397+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 shutdown_connections 2024-04-25T00:05:39.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.397+0000 7f150dcae640 1 -- 172.21.15.113:0/3462723582 wait complete. 2024-04-25T00:05:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-25T00:05:39.197012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394952,"kb_used_data":12232,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754579768,"statfs":{"total":773094113280,"available":772689682432,"internally_reserved":0,"allocated":12525568,"data_stored":6078896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.121858"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119482+0000","last_change":"2024-04-25T00:05:36.113111+0000","last_active":"2024-04-25T00:05:36.119482+0000","last_peered":"2024-04-25T00:05:36.119482+0000","last_clean":"2024-04-25T00:05:36.119482+0000","last_became_active":"2024-04-25T00:05:36.112593+0000","last_became_peered":"2024-04-25T00:05:36.112593+0000","last_unstale":"2024-04-25T00:05:36.119482+0000","last_undegraded":"2024-04-25T00:05:36.119482+0000","last_fullsized":"2024-04-25T00:05:36.119482+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:43:47.302545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.102869+0000","last_change":"2024-04-25T00:05:36.100726+0000","last_active":"2024-04-25T00:05:36.102869+0000","last_peered":"2024-04-25T00:05:36.102869+0000","last_clean":"2024-04-25T00:05:36.102869+0000","last_became_active":"2024-04-25T00:05:36.100473+0000","last_became_peered":"2024-04-25T00:05:36.100473+0000","last_unstale":"2024-04-25T00:05:36.102869+0000","last_undegraded":"2024-04-25T00:05:36.102869+0000","last_fullsized":"2024-04-25T00:05:36.102869+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:13.049564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110820+0000","last_change":"2024-04-25T00:05:36.109014+0000","last_active":"2024-04-25T00:05:36.110820+0000","last_peered":"2024-04-25T00:05:36.110820+0000","last_clean":"2024-04-25T00:05:36.110820+0000","last_became_active":"2024-04-25T00:05:36.108706+0000","last_became_peered":"2024-04-25T00:05:36.108706+0000","last_unstale":"2024-04-25T00:05:36.110820+0000","last_undegraded":"2024-04-25T00:05:36.110820+0000","last_fullsized":"2024-04-25T00:05:36.110820+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:00:43.877373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116600+0000","last_change":"2024-04-25T00:05:36.109895+0000","last_active":"2024-04-25T00:05:36.116600+0000","last_peered":"2024-04-25T00:05:36.116600+0000","last_clean":"2024-04-25T00:05:36.116600+0000","last_became_active":"2024-04-25T00:05:36.108862+0000","last_became_peered":"2024-04-25T00:05:36.108862+0000","last_unstale":"2024-04-25T00:05:36.116600+0000","last_undegraded":"2024-04-25T00:05:36.116600+0000","last_fullsized":"2024-04-25T00:05:36.116600+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:22.725103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.103274+0000","last_change":"2024-04-25T00:05:36.101549+0000","last_active":"2024-04-25T00:05:36.103274+0000","last_peered":"2024-04-25T00:05:36.103274+0000","last_clean":"2024-04-25T00:05:36.103274+0000","last_became_active":"2024-04-25T00:05:36.101262+0000","last_became_peered":"2024-04-25T00:05:36.101262+0000","last_unstale":"2024-04-25T00:05:36.103274+0000","last_undegraded":"2024-04-25T00:05:36.103274+0000","last_fullsized":"2024-04-25T00:05:36.103274+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:36.448728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107516+0000","last_change":"2024-04-25T00:05:36.104604+0000","last_active":"2024-04-25T00:05:36.107516+0000","last_peered":"2024-04-25T00:05:36.107516+0000","last_clean":"2024-04-25T00:05:36.107516+0000","last_became_active":"2024-04-25T00:05:36.104363+0000","last_became_peered":"2024-04-25T00:05:36.104363+0000","last_unstale":"2024-04-25T00:05:36.107516+0000","last_undegraded":"2024-04-25T00:05:36.107516+0000","last_fullsized":"2024-04-25T00:05:36.107516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:03.916769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.115328+0000","last_change":"2024-04-25T00:05:36.114133+0000","last_active":"2024-04-25T00:05:36.115328+0000","last_peered":"2024-04-25T00:05:36.115328+0000","last_clean":"2024-04-25T00:05:36.115328+0000","last_became_active":"2024-04-25T00:05:36.112488+0000","last_became_peered":"2024-04-25T00:05:36.112488+0000","last_unstale":"2024-04-25T00:05:36.115328+0000","last_undegraded":"2024-04-25T00:05:36.115328+0000","last_fullsized":"2024-04-25T00:05:36.115328+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:10.046278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110148+0000","last_change":"2024-04-25T00:05:36.107676+0000","last_active":"2024-04-25T00:05:36.110148+0000","last_peered":"2024-04-25T00:05:36.110148+0000","last_clean":"2024-04-25T00:05:36.110148+0000","last_became_active":"2024-04-25T00:05:36.104599+0000","last_became_peered":"2024-04-25T00:05:36.104599+0000","last_unstale":"2024-04-25T00:05:36.110148+0000","last_undegraded":"2024-04-25T00:05:36.110148+0000","last_fullsized":"2024-04-25T00:05:36.110148+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:58.123203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108992+0000","last_change":"2024-04-25T00:05:36.106067+0000","last_active":"2024-04-25T00:05:36.108992+0000","last_peered":"2024-04-25T00:05:36.108992+0000","last_clean":"2024-04-25T00:05:36.108992+0000","last_became_active":"2024-04-25T00:05:36.103418+0000","last_became_peered":"2024-04-25T00:05:36.103418+0000","last_unstale":"2024-04-25T00:05:36.108992+0000","last_undegraded":"2024-04-25T00:05:36.108992+0000","last_fullsized":"2024-04-25T00:05:36.108992+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:12.701990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116657+0000","last_change":"2024-04-25T00:05:36.109374+0000","last_active":"2024-04-25T00:05:36.116657+0000","last_peered":"2024-04-25T00:05:36.116657+0000","last_clean":"2024-04-25T00:05:36.116657+0000","last_became_active":"2024-04-25T00:05:36.108702+0000","last_became_peered":"2024-04-25T00:05:36.108702+0000","last_unstale":"2024-04-25T00:05:36.116657+0000","last_undegraded":"2024-04-25T00:05:36.116657+0000","last_fullsized":"2024-04-25T00:05:36.116657+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:50.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114612+0000","last_change":"2024-04-25T00:05:36.114031+0000","last_active":"2024-04-25T00:05:36.114612+0000","last_peered":"2024-04-25T00:05:36.114612+0000","last_clean":"2024-04-25T00:05:36.114612+0000","last_became_active":"2024-04-25T00:05:36.113458+0000","last_became_peered":"2024-04-25T00:05:36.113458+0000","last_unstale":"2024-04-25T00:05:36.114612+0000","last_undegraded":"2024-04-25T00:05:36.114612+0000","last_fullsized":"2024-04-25T00:05:36.114612+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:03.775306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118407+0000","last_change":"2024-04-25T00:05:36.111974+0000","last_active":"2024-04-25T00:05:36.118407+0000","last_peered":"2024-04-25T00:05:36.118407+0000","last_clean":"2024-04-25T00:05:36.118407+0000","last_became_active":"2024-04-25T00:05:36.111115+0000","last_became_peered":"2024-04-25T00:05:36.111115+0000","last_unstale":"2024-04-25T00:05:36.118407+0000","last_undegraded":"2024-04-25T00:05:36.118407+0000","last_fullsized":"2024-04-25T00:05:36.118407+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:28.521777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105916+0000","last_change":"2024-04-25T00:05:36.105240+0000","last_active":"2024-04-25T00:05:36.105916+0000","last_peered":"2024-04-25T00:05:36.105916+0000","last_clean":"2024-04-25T00:05:36.105916+0000","last_became_active":"2024-04-25T00:05:36.104970+0000","last_became_peered":"2024-04-25T00:05:36.104970+0000","last_unstale":"2024-04-25T00:05:36.105916+0000","last_undegraded":"2024-04-25T00:05:36.105916+0000","last_fullsized":"2024-04-25T00:05:36.105916+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:22.146161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110348+0000","last_change":"2024-04-25T00:05:36.109498+0000","last_active":"2024-04-25T00:05:36.110348+0000","last_peered":"2024-04-25T00:05:36.110348+0000","last_clean":"2024-04-25T00:05:36.110348+0000","last_became_active":"2024-04-25T00:05:36.108766+0000","last_became_peered":"2024-04-25T00:05:36.108766+0000","last_unstale":"2024-04-25T00:05:36.110348+0000","last_undegraded":"2024-04-25T00:05:36.110348+0000","last_fullsized":"2024-04-25T00:05:36.110348+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:59.169658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107595+0000","last_change":"2024-04-25T00:05:36.106014+0000","last_active":"2024-04-25T00:05:36.107595+0000","last_peered":"2024-04-25T00:05:36.107595+0000","last_clean":"2024-04-25T00:05:36.107595+0000","last_became_active":"2024-04-25T00:05:36.105731+0000","last_became_peered":"2024-04-25T00:05:36.105731+0000","last_unstale":"2024-04-25T00:05:36.107595+0000","last_undegraded":"2024-04-25T00:05:36.107595+0000","last_fullsized":"2024-04-25T00:05:36.107595+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:02.220173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.109179+0000","last_change":"2024-04-25T00:05:36.105776+0000","last_active":"2024-04-25T00:05:36.109179+0000","last_peered":"2024-04-25T00:05:36.109179+0000","last_clean":"2024-04-25T00:05:36.109179+0000","last_became_active":"2024-04-25T00:05:36.104260+0000","last_became_peered":"2024-04-25T00:05:36.104260+0000","last_unstale":"2024-04-25T00:05:36.109179+0000","last_undegraded":"2024-04-25T00:05:36.109179+0000","last_fullsized":"2024-04-25T00:05:36.109179+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:48.108953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114257+0000","last_change":"2024-04-25T00:05:36.112275+0000","last_active":"2024-04-25T00:05:36.114257+0000","last_peered":"2024-04-25T00:05:36.114257+0000","last_clean":"2024-04-25T00:05:36.114257+0000","last_became_active":"2024-04-25T00:05:36.111983+0000","last_became_peered":"2024-04-25T00:05:36.111983+0000","last_unstale":"2024-04-25T00:05:36.114257+0000","last_undegraded":"2024-04-25T00:05:36.114257+0000","last_fullsized":"2024-04-25T00:05:36.114257+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:18.334300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116340+0000","last_change":"2024-04-25T00:05:36.114779+0000","last_active":"2024-04-25T00:05:36.116340+0000","last_peered":"2024-04-25T00:05:36.116340+0000","last_clean":"2024-04-25T00:05:36.116340+0000","last_became_active":"2024-04-25T00:05:36.114492+0000","last_became_peered":"2024-04-25T00:05:36.114492+0000","last_unstale":"2024-04-25T00:05:36.116340+0000","last_undegraded":"2024-04-25T00:05:36.116340+0000","last_fullsized":"2024-04-25T00:05:36.116340+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:40.871259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111910+0000","last_change":"2024-04-25T00:05:36.110693+0000","last_active":"2024-04-25T00:05:36.111910+0000","last_peered":"2024-04-25T00:05:36.111910+0000","last_clean":"2024-04-25T00:05:36.111910+0000","last_became_active":"2024-04-25T00:05:36.110412+0000","last_became_peered":"2024-04-25T00:05:36.110412+0000","last_unstale":"2024-04-25T00:05:36.111910+0000","last_undegraded":"2024-04-25T00:05:36.111910+0000","last_fullsized":"2024-04-25T00:05:36.111910+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:48.108953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108755+0000","last_change":"2024-04-25T00:05:36.104566+0000","last_active":"2024-04-25T00:05:36.108755+0000","last_peered":"2024-04-25T00:05:36.108755+0000","last_clean":"2024-04-25T00:05:36.108755+0000","last_became_active":"2024-04-25T00:05:36.104295+0000","last_became_peered":"2024-04-25T00:05:36.104295+0000","last_unstale":"2024-04-25T00:05:36.108755+0000","last_undegraded":"2024-04-25T00:05:36.108755+0000","last_fullsized":"2024-04-25T00:05:36.108755+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:16.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107787+0000","last_change":"2024-04-25T00:05:36.104229+0000","last_active":"2024-04-25T00:05:36.107787+0000","last_peered":"2024-04-25T00:05:36.107787+0000","last_clean":"2024-04-25T00:05:36.107787+0000","last_became_active":"2024-04-25T00:05:36.103314+0000","last_became_peered":"2024-04-25T00:05:36.103314+0000","last_unstale":"2024-04-25T00:05:36.107787+0000","last_undegraded":"2024-04-25T00:05:36.107787+0000","last_fullsized":"2024-04-25T00:05:36.107787+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:37.688391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111422+0000","last_change":"2024-04-25T00:05:36.109879+0000","last_active":"2024-04-25T00:05:36.111422+0000","last_peered":"2024-04-25T00:05:36.111422+0000","last_clean":"2024-04-25T00:05:36.111422+0000","last_became_active":"2024-04-25T00:05:36.109633+0000","last_became_peered":"2024-04-25T00:05:36.109633+0000","last_unstale":"2024-04-25T00:05:36.111422+0000","last_undegraded":"2024-04-25T00:05:36.111422+0000","last_fullsized":"2024-04-25T00:05:36.111422+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:22.146161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107965+0000","last_change":"2024-04-25T00:05:36.105878+0000","last_active":"2024-04-25T00:05:36.107965+0000","last_peered":"2024-04-25T00:05:36.107965+0000","last_clean":"2024-04-25T00:05:36.107965+0000","last_became_active":"2024-04-25T00:05:36.104761+0000","last_became_peered":"2024-04-25T00:05:36.104761+0000","last_unstale":"2024-04-25T00:05:36.107965+0000","last_undegraded":"2024-04-25T00:05:36.107965+0000","last_fullsized":"2024-04-25T00:05:36.107965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:16.802256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.104792+0000","last_change":"2024-04-25T00:05:36.103524+0000","last_active":"2024-04-25T00:05:36.104792+0000","last_peered":"2024-04-25T00:05:36.104792+0000","last_clean":"2024-04-25T00:05:36.104792+0000","last_became_active":"2024-04-25T00:05:36.103238+0000","last_became_peered":"2024-04-25T00:05:36.103238+0000","last_unstale":"2024-04-25T00:05:36.104792+0000","last_undegraded":"2024-04-25T00:05:36.104792+0000","last_fullsized":"2024-04-25T00:05:36.104792+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.499735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114235+0000","last_change":"2024-04-25T00:05:36.112089+0000","last_active":"2024-04-25T00:05:36.114235+0000","last_peered":"2024-04-25T00:05:36.114235+0000","last_clean":"2024-04-25T00:05:36.114235+0000","last_became_active":"2024-04-25T00:05:36.111613+0000","last_became_peered":"2024-04-25T00:05:36.111613+0000","last_unstale":"2024-04-25T00:05:36.114235+0000","last_undegraded":"2024-04-25T00:05:36.114235+0000","last_fullsized":"2024-04-25T00:05:36.114235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:18.914917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116691+0000","last_change":"2024-04-25T00:05:36.109577+0000","last_active":"2024-04-25T00:05:36.116691+0000","last_peered":"2024-04-25T00:05:36.116691+0000","last_clean":"2024-04-25T00:05:36.116691+0000","last_became_active":"2024-04-25T00:05:36.109125+0000","last_became_peered":"2024-04-25T00:05:36.109125+0000","last_unstale":"2024-04-25T00:05:36.116691+0000","last_undegraded":"2024-04-25T00:05:36.116691+0000","last_fullsized":"2024-04-25T00:05:36.116691+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:00.477017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114031+0000","last_change":"2024-04-25T00:05:36.109601+0000","last_active":"2024-04-25T00:05:36.114031+0000","last_peered":"2024-04-25T00:05:36.114031+0000","last_clean":"2024-04-25T00:05:36.114031+0000","last_became_active":"2024-04-25T00:05:36.109363+0000","last_became_peered":"2024-04-25T00:05:36.109363+0000","last_unstale":"2024-04-25T00:05:36.114031+0000","last_undegraded":"2024-04-25T00:05:36.114031+0000","last_fullsized":"2024-04-25T00:05:36.114031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:36.448728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108026+0000","last_change":"2024-04-25T00:05:36.105233+0000","last_active":"2024-04-25T00:05:36.108026+0000","last_peered":"2024-04-25T00:05:36.108026+0000","last_clean":"2024-04-25T00:05:36.108026+0000","last_became_active":"2024-04-25T00:05:36.104654+0000","last_became_peered":"2024-04-25T00:05:36.104654+0000","last_unstale":"2024-04-25T00:05:36.108026+0000","last_undegraded":"2024-04-25T00:05:36.108026+0000","last_fullsized":"2024-04-25T00:05:36.108026+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:26.761672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107807+0000","last_change":"2024-04-25T00:05:36.104487+0000","last_active":"2024-04-25T00:05:36.107807+0000","last_peered":"2024-04-25T00:05:36.107807+0000","last_clean":"2024-04-25T00:05:36.107807+0000","last_became_active":"2024-04-25T00:05:36.103697+0000","last_became_peered":"2024-04-25T00:05:36.103697+0000","last_unstale":"2024-04-25T00:05:36.107807+0000","last_undegraded":"2024-04-25T00:05:36.107807+0000","last_fullsized":"2024-04-25T00:05:36.107807+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:40.871259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.106468+0000","last_change":"2024-04-25T00:05:36.105450+0000","last_active":"2024-04-25T00:05:36.106468+0000","last_peered":"2024-04-25T00:05:36.106468+0000","last_clean":"2024-04-25T00:05:36.106468+0000","last_became_active":"2024-04-25T00:05:36.105107+0000","last_became_peered":"2024-04-25T00:05:36.105107+0000","last_unstale":"2024-04-25T00:05:36.106468+0000","last_undegraded":"2024-04-25T00:05:36.106468+0000","last_fullsized":"2024-04-25T00:05:36.106468+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:36.121040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117602+0000","last_change":"2024-04-25T00:05:36.110465+0000","last_active":"2024-04-25T00:05:36.117602+0000","last_peered":"2024-04-25T00:05:36.117602+0000","last_clean":"2024-04-25T00:05:36.117602+0000","last_became_active":"2024-04-25T00:05:36.110207+0000","last_became_peered":"2024-04-25T00:05:36.110207+0000","last_unstale":"2024-04-25T00:05:36.117602+0000","last_undegraded":"2024-04-25T00:05:36.117602+0000","last_fullsized":"2024-04-25T00:05:36.117602+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:05.416974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107581+0000","last_change":"2024-04-25T00:05:36.106530+0000","last_active":"2024-04-25T00:05:36.107581+0000","last_peered":"2024-04-25T00:05:36.107581+0000","last_clean":"2024-04-25T00:05:36.107581+0000","last_became_active":"2024-04-25T00:05:36.106283+0000","last_became_peered":"2024-04-25T00:05:36.106283+0000","last_unstale":"2024-04-25T00:05:36.107581+0000","last_undegraded":"2024-04-25T00:05:36.107581+0000","last_fullsized":"2024-04-25T00:05:36.107581+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:14.188450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107656+0000","last_change":"2024-04-25T00:05:36.104046+0000","last_active":"2024-04-25T00:05:36.107656+0000","last_peered":"2024-04-25T00:05:36.107656+0000","last_clean":"2024-04-25T00:05:36.107656+0000","last_became_active":"2024-04-25T00:05:36.103796+0000","last_became_peered":"2024-04-25T00:05:36.103796+0000","last_unstale":"2024-04-25T00:05:36.107656+0000","last_undegraded":"2024-04-25T00:05:36.107656+0000","last_fullsized":"2024-04-25T00:05:36.107656+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:11.839589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119427+0000","last_change":"2024-04-25T00:05:35.091859+0000","last_active":"2024-04-25T00:05:36.119427+0000","last_peered":"2024-04-25T00:05:36.119427+0000","last_clean":"2024-04-25T00:05:36.119427+0000","last_became_active":"2024-04-25T00:05:35.091165+0000","last_became_peered":"2024-04-25T00:05:35.091165+0000","last_unstale":"2024-04-25T00:05:36.119427+0000","last_undegraded":"2024-04-25T00:05:36.119427+0000","last_fullsized":"2024-04-25T00:05:36.119427+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:52.326992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.113944+0000","last_change":"2024-04-25T00:05:36.109492+0000","last_active":"2024-04-25T00:05:36.113944+0000","last_peered":"2024-04-25T00:05:36.113944+0000","last_clean":"2024-04-25T00:05:36.113944+0000","last_became_active":"2024-04-25T00:05:36.109000+0000","last_became_peered":"2024-04-25T00:05:36.109000+0000","last_unstale":"2024-04-25T00:05:36.113944+0000","last_undegraded":"2024-04-25T00:05:36.113944+0000","last_fullsized":"2024-04-25T00:05:36.113944+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:35.094475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119632+0000","last_change":"2024-04-25T00:05:35.094752+0000","last_active":"2024-04-25T00:05:36.119632+0000","last_peered":"2024-04-25T00:05:36.119632+0000","last_clean":"2024-04-25T00:05:36.119632+0000","last_became_active":"2024-04-25T00:05:35.094403+0000","last_became_peered":"2024-04-25T00:05:35.094403+0000","last_unstale":"2024-04-25T00:05:36.119632+0000","last_undegraded":"2024-04-25T00:05:36.119632+0000","last_fullsized":"2024-04-25T00:05:36.119632+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:43.335618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107548+0000","last_change":"2024-04-25T00:05:36.106019+0000","last_active":"2024-04-25T00:05:36.107548+0000","last_peered":"2024-04-25T00:05:36.107548+0000","last_clean":"2024-04-25T00:05:36.107548+0000","last_became_active":"2024-04-25T00:05:36.105749+0000","last_became_peered":"2024-04-25T00:05:36.105749+0000","last_unstale":"2024-04-25T00:05:36.107548+0000","last_undegraded":"2024-04-25T00:05:36.107548+0000","last_fullsized":"2024-04-25T00:05:36.107548+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:40.851368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.120449+0000","last_change":"2024-04-25T00:05:35.095226+0000","last_active":"2024-04-25T00:05:36.120449+0000","last_peered":"2024-04-25T00:05:36.120449+0000","last_clean":"2024-04-25T00:05:36.120449+0000","last_became_active":"2024-04-25T00:05:35.094921+0000","last_became_peered":"2024-04-25T00:05:35.094921+0000","last_unstale":"2024-04-25T00:05:36.120449+0000","last_undegraded":"2024-04-25T00:05:36.120449+0000","last_fullsized":"2024-04-25T00:05:36.120449+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:13.183434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111895+0000","last_change":"2024-04-25T00:05:36.110663+0000","last_active":"2024-04-25T00:05:36.111895+0000","last_peered":"2024-04-25T00:05:36.111895+0000","last_clean":"2024-04-25T00:05:36.111895+0000","last_became_active":"2024-04-25T00:05:36.110385+0000","last_became_peered":"2024-04-25T00:05:36.110385+0000","last_unstale":"2024-04-25T00:05:36.111895+0000","last_undegraded":"2024-04-25T00:05:36.111895+0000","last_fullsized":"2024-04-25T00:05:36.111895+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:29.921695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.120897+0000","last_change":"2024-04-25T00:05:35.090579+0000","last_active":"2024-04-25T00:05:36.120897+0000","last_peered":"2024-04-25T00:05:36.120897+0000","last_clean":"2024-04-25T00:05:36.120897+0000","last_became_active":"2024-04-25T00:05:35.089876+0000","last_became_peered":"2024-04-25T00:05:35.089876+0000","last_unstale":"2024-04-25T00:05:36.120897+0000","last_undegraded":"2024-04-25T00:05:36.120897+0000","last_fullsized":"2024-04-25T00:05:36.120897+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:10.834573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108383+0000","last_change":"2024-04-25T00:05:36.102861+0000","last_active":"2024-04-25T00:05:36.108383+0000","last_peered":"2024-04-25T00:05:36.108383+0000","last_clean":"2024-04-25T00:05:36.108383+0000","last_became_active":"2024-04-25T00:05:36.101697+0000","last_became_peered":"2024-04-25T00:05:36.101697+0000","last_unstale":"2024-04-25T00:05:36.108383+0000","last_undegraded":"2024-04-25T00:05:36.108383+0000","last_fullsized":"2024-04-25T00:05:36.108383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.857589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118149+0000","last_change":"2024-04-25T00:05:35.092139+0000","last_active":"2024-04-25T00:05:36.118149+0000","last_peered":"2024-04-25T00:05:36.118149+0000","last_clean":"2024-04-25T00:05:36.118149+0000","last_became_active":"2024-04-25T00:05:35.091849+0000","last_became_peered":"2024-04-25T00:05:35.091849+0000","last_unstale":"2024-04-25T00:05:36.118149+0000","last_undegraded":"2024-04-25T00:05:36.118149+0000","last_fullsized":"2024-04-25T00:05:36.118149+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:29.454077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105466+0000","last_change":"2024-04-25T00:05:36.100905+0000","last_active":"2024-04-25T00:05:36.105466+0000","last_peered":"2024-04-25T00:05:36.105466+0000","last_clean":"2024-04-25T00:05:36.105466+0000","last_became_active":"2024-04-25T00:05:36.100379+0000","last_became_peered":"2024-04-25T00:05:36.100379+0000","last_unstale":"2024-04-25T00:05:36.105466+0000","last_undegraded":"2024-04-25T00:05:36.105466+0000","last_fullsized":"2024-04-25T00:05:36.105466+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:08.846207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.122617+0000","last_change":"2024-04-25T00:05:35.090638+0000","last_active":"2024-04-25T00:05:36.122617+0000","last_peered":"2024-04-25T00:05:36.122617+0000","last_clean":"2024-04-25T00:05:36.122617+0000","last_became_active":"2024-04-25T00:05:35.090137+0000","last_became_peered":"2024-04-25T00:05:35.090137+0000","last_unstale":"2024-04-25T00:05:36.122617+0000","last_undegraded":"2024-04-25T00:05:36.122617+0000","last_fullsized":"2024-04-25T00:05:36.122617+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:17.329284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116614+0000","last_change":"2024-04-25T00:05:36.109055+0000","last_active":"2024-04-25T00:05:36.116614+0000","last_peered":"2024-04-25T00:05:36.116614+0000","last_clean":"2024-04-25T00:05:36.116614+0000","last_became_active":"2024-04-25T00:05:36.108633+0000","last_became_peered":"2024-04-25T00:05:36.108633+0000","last_unstale":"2024-04-25T00:05:36.116614+0000","last_undegraded":"2024-04-25T00:05:36.116614+0000","last_fullsized":"2024-04-25T00:05:36.116614+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:26.705375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.094550+0000","last_change":"2024-04-25T00:05:35.088080+0000","last_active":"2024-04-25T00:05:36.094550+0000","last_peered":"2024-04-25T00:05:36.094550+0000","last_clean":"2024-04-25T00:05:36.094550+0000","last_became_active":"2024-04-25T00:05:35.087730+0000","last_became_peered":"2024-04-25T00:05:35.087730+0000","last_unstale":"2024-04-25T00:05:36.094550+0000","last_undegraded":"2024-04-25T00:05:36.094550+0000","last_fullsized":"2024-04-25T00:05:36.094550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:08.665848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.115348+0000","last_change":"2024-04-25T00:05:36.114352+0000","last_active":"2024-04-25T00:05:36.115348+0000","last_peered":"2024-04-25T00:05:36.115348+0000","last_clean":"2024-04-25T00:05:36.115348+0000","last_became_active":"2024-04-25T00:05:36.113856+0000","last_became_peered":"2024-04-25T00:05:36.113856+0000","last_unstale":"2024-04-25T00:05:36.115348+0000","last_undegraded":"2024-04-25T00:05:36.115348+0000","last_fullsized":"2024-04-25T00:05:36.115348+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:21.118927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098289+0000","last_change":"2024-04-25T00:05:35.089059+0000","last_active":"2024-04-25T00:05:36.098289+0000","last_peered":"2024-04-25T00:05:36.098289+0000","last_clean":"2024-04-25T00:05:36.098289+0000","last_became_active":"2024-04-25T00:05:35.088785+0000","last_became_peered":"2024-04-25T00:05:35.088785+0000","last_unstale":"2024-04-25T00:05:36.098289+0000","last_undegraded":"2024-04-25T00:05:36.098289+0000","last_fullsized":"2024-04-25T00:05:36.098289+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:57.118187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.104425+0000","last_change":"2024-04-25T00:05:36.103796+0000","last_active":"2024-04-25T00:05:36.104425+0000","last_peered":"2024-04-25T00:05:36.104425+0000","last_clean":"2024-04-25T00:05:36.104425+0000","last_became_active":"2024-04-25T00:05:36.103525+0000","last_became_peered":"2024-04-25T00:05:36.103525+0000","last_unstale":"2024-04-25T00:05:36.104425+0000","last_undegraded":"2024-04-25T00:05:36.104425+0000","last_fullsized":"2024-04-25T00:05:36.104425+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:12.814340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.093571+0000","last_change":"2024-04-25T00:05:35.088863+0000","last_active":"2024-04-25T00:05:36.093571+0000","last_peered":"2024-04-25T00:05:36.093571+0000","last_clean":"2024-04-25T00:05:36.093571+0000","last_became_active":"2024-04-25T00:05:35.088165+0000","last_became_peered":"2024-04-25T00:05:35.088165+0000","last_unstale":"2024-04-25T00:05:36.093571+0000","last_undegraded":"2024-04-25T00:05:36.093571+0000","last_fullsized":"2024-04-25T00:05:36.093571+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:01.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.113965+0000","last_change":"2024-04-25T00:05:36.110806+0000","last_active":"2024-04-25T00:05:36.113965+0000","last_peered":"2024-04-25T00:05:36.113965+0000","last_clean":"2024-04-25T00:05:36.113965+0000","last_became_active":"2024-04-25T00:05:36.110545+0000","last_became_peered":"2024-04-25T00:05:36.110545+0000","last_unstale":"2024-04-25T00:05:36.113965+0000","last_undegraded":"2024-04-25T00:05:36.113965+0000","last_fullsized":"2024-04-25T00:05:36.113965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:37.688391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.096590+0000","last_change":"2024-04-25T00:05:35.090833+0000","last_active":"2024-04-25T00:05:36.096590+0000","last_peered":"2024-04-25T00:05:36.096590+0000","last_clean":"2024-04-25T00:05:36.096590+0000","last_became_active":"2024-04-25T00:05:35.090468+0000","last_became_peered":"2024-04-25T00:05:35.090468+0000","last_unstale":"2024-04-25T00:05:36.096590+0000","last_undegraded":"2024-04-25T00:05:36.096590+0000","last_fullsized":"2024-04-25T00:05:36.096590+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:15.797240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111768+0000","last_change":"2024-04-25T00:05:36.107654+0000","last_active":"2024-04-25T00:05:36.111768+0000","last_peered":"2024-04-25T00:05:36.111768+0000","last_clean":"2024-04-25T00:05:36.111768+0000","last_became_active":"2024-04-25T00:05:36.107066+0000","last_became_peered":"2024-04-25T00:05:36.107066+0000","last_unstale":"2024-04-25T00:05:36.111768+0000","last_undegraded":"2024-04-25T00:05:36.111768+0000","last_fullsized":"2024-04-25T00:05:36.111768+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:38.927057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"115'3","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118968+0000","last_change":"2024-04-25T00:05:35.094234+0000","last_active":"2024-04-25T00:05:36.118968+0000","last_peered":"2024-04-25T00:05:36.118968+0000","last_clean":"2024-04-25T00:05:36.118968+0000","last_became_active":"2024-04-25T00:05:35.093973+0000","last_became_peered":"2024-04-25T00:05:35.093973+0000","last_unstale":"2024-04-25T00:05:36.118968+0000","last_undegraded":"2024-04-25T00:05:36.118968+0000","last_fullsized":"2024-04-25T00:05:36.118968+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:43.335618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.106311+0000","last_change":"2024-04-25T00:05:36.105354+0000","last_active":"2024-04-25T00:05:36.106311+0000","last_peered":"2024-04-25T00:05:36.106311+0000","last_clean":"2024-04-25T00:05:36.106311+0000","last_became_active":"2024-04-25T00:05:36.104956+0000","last_became_peered":"2024-04-25T00:05:36.104956+0000","last_unstale":"2024-04-25T00:05:36.106311+0000","last_undegraded":"2024-04-25T00:05:36.106311+0000","last_fullsized":"2024-04-25T00:05:36.106311+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:01.059079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"115'6","reported_seq":18,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.123007+0000","last_change":"2024-04-25T00:05:35.088052+0000","last_active":"2024-04-25T00:05:36.123007+0000","last_peered":"2024-04-25T00:05:36.123007+0000","last_clean":"2024-04-25T00:05:36.123007+0000","last_became_active":"2024-04-25T00:05:35.087751+0000","last_became_peered":"2024-04-25T00:05:35.087751+0000","last_unstale":"2024-04-25T00:05:36.123007+0000","last_undegraded":"2024-04-25T00:05:36.123007+0000","last_fullsized":"2024-04-25T00:05:36.123007+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:16.887883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107196+0000","last_change":"2024-04-25T00:05:36.105386+0000","last_active":"2024-04-25T00:05:36.107196+0000","last_peered":"2024-04-25T00:05:36.107196+0000","last_clean":"2024-04-25T00:05:36.107196+0000","last_became_active":"2024-04-25T00:05:36.104979+0000","last_became_peered":"2024-04-25T00:05:36.104979+0000","last_unstale":"2024-04-25T00:05:36.107196+0000","last_undegraded":"2024-04-25T00:05:36.107196+0000","last_fullsized":"2024-04-25T00:05:36.107196+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:44.340634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117161+0000","last_change":"2024-04-25T00:05:35.090419+0000","last_active":"2024-04-25T00:05:36.117161+0000","last_peered":"2024-04-25T00:05:36.117161+0000","last_clean":"2024-04-25T00:05:36.117161+0000","last_became_active":"2024-04-25T00:05:35.090187+0000","last_became_peered":"2024-04-25T00:05:35.090187+0000","last_unstale":"2024-04-25T00:05:36.117161+0000","last_undegraded":"2024-04-25T00:05:36.117161+0000","last_fullsized":"2024-04-25T00:05:36.117161+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:34.200146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105219+0000","last_change":"2024-04-25T00:05:36.103744+0000","last_active":"2024-04-25T00:05:36.105219+0000","last_peered":"2024-04-25T00:05:36.105219+0000","last_clean":"2024-04-25T00:05:36.105219+0000","last_became_active":"2024-04-25T00:05:36.103494+0000","last_became_peered":"2024-04-25T00:05:36.103494+0000","last_unstale":"2024-04-25T00:05:36.105219+0000","last_undegraded":"2024-04-25T00:05:36.105219+0000","last_fullsized":"2024-04-25T00:05:36.105219+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:36.276236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.097819+0000","last_change":"2024-04-25T00:05:35.095367+0000","last_active":"2024-04-25T00:05:36.097819+0000","last_peered":"2024-04-25T00:05:36.097819+0000","last_clean":"2024-04-25T00:05:36.097819+0000","last_became_active":"2024-04-25T00:05:35.095077+0000","last_became_peered":"2024-04-25T00:05:35.095077+0000","last_unstale":"2024-04-25T00:05:36.097819+0000","last_undegraded":"2024-04-25T00:05:36.097819+0000","last_fullsized":"2024-04-25T00:05:36.097819+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:08.665848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105758+0000","last_change":"2024-04-25T00:05:36.104474+0000","last_active":"2024-04-25T00:05:36.105758+0000","last_peered":"2024-04-25T00:05:36.105758+0000","last_clean":"2024-04-25T00:05:36.105758+0000","last_became_active":"2024-04-25T00:05:36.103894+0000","last_became_peered":"2024-04-25T00:05:36.103894+0000","last_unstale":"2024-04-25T00:05:36.105758+0000","last_undegraded":"2024-04-25T00:05:36.105758+0000","last_fullsized":"2024-04-25T00:05:36.105758+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:53.422990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":20,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.123612+0000","last_change":"2024-04-25T00:05:35.093102+0000","last_active":"2024-04-25T00:05:36.123612+0000","last_peered":"2024-04-25T00:05:36.123612+0000","last_clean":"2024-04-25T00:05:36.123612+0000","last_became_active":"2024-04-25T00:05:35.092841+0000","last_became_peered":"2024-04-25T00:05:35.092841+0000","last_unstale":"2024-04-25T00:05:36.123612+0000","last_undegraded":"2024-04-25T00:05:36.123612+0000","last_fullsized":"2024-04-25T00:05:36.123612+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:39.846352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.114144+0000","last_change":"2024-04-25T00:05:36.110597+0000","last_active":"2024-04-25T00:05:36.114144+0000","last_peered":"2024-04-25T00:05:36.114144+0000","last_clean":"2024-04-25T00:05:36.114144+0000","last_became_active":"2024-04-25T00:05:36.110345+0000","last_became_peered":"2024-04-25T00:05:36.110345+0000","last_unstale":"2024-04-25T00:05:36.114144+0000","last_undegraded":"2024-04-25T00:05:36.114144+0000","last_fullsized":"2024-04-25T00:05:36.114144+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:22.047635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.118530+0000","last_change":"2024-04-25T00:05:35.092034+0000","last_active":"2024-04-25T00:05:36.118530+0000","last_peered":"2024-04-25T00:05:36.118530+0000","last_clean":"2024-04-25T00:05:36.118530+0000","last_became_active":"2024-04-25T00:05:35.091777+0000","last_became_peered":"2024-04-25T00:05:35.091777+0000","last_unstale":"2024-04-25T00:05:36.118530+0000","last_undegraded":"2024-04-25T00:05:36.118530+0000","last_fullsized":"2024-04-25T00:05:36.118530+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:21.141145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.112217+0000","last_change":"2024-04-25T00:05:36.108682+0000","last_active":"2024-04-25T00:05:36.112217+0000","last_peered":"2024-04-25T00:05:36.112217+0000","last_clean":"2024-04-25T00:05:36.112217+0000","last_became_active":"2024-04-25T00:05:36.108439+0000","last_became_peered":"2024-04-25T00:05:36.108439+0000","last_unstale":"2024-04-25T00:05:36.112217+0000","last_undegraded":"2024-04-25T00:05:36.112217+0000","last_fullsized":"2024-04-25T00:05:36.112217+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:13.118263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119494+0000","last_change":"2024-04-25T00:05:35.085470+0000","last_active":"2024-04-25T00:05:36.119494+0000","last_peered":"2024-04-25T00:05:36.119494+0000","last_clean":"2024-04-25T00:05:36.119494+0000","last_became_active":"2024-04-25T00:05:35.085220+0000","last_became_peered":"2024-04-25T00:05:35.085220+0000","last_unstale":"2024-04-25T00:05:36.119494+0000","last_undegraded":"2024-04-25T00:05:36.119494+0000","last_fullsized":"2024-04-25T00:05:36.119494+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:12.113247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116530+0000","last_change":"2024-04-25T00:05:36.115342+0000","last_active":"2024-04-25T00:05:36.116530+0000","last_peered":"2024-04-25T00:05:36.116530+0000","last_clean":"2024-04-25T00:05:36.116530+0000","last_became_active":"2024-04-25T00:05:36.114994+0000","last_became_peered":"2024-04-25T00:05:36.114994+0000","last_unstale":"2024-04-25T00:05:36.116530+0000","last_undegraded":"2024-04-25T00:05:36.116530+0000","last_fullsized":"2024-04-25T00:05:36.116530+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:12.701990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.122862+0000","last_change":"2024-04-25T00:05:35.089292+0000","last_active":"2024-04-25T00:05:36.122862+0000","last_peered":"2024-04-25T00:05:36.122862+0000","last_clean":"2024-04-25T00:05:36.122862+0000","last_became_active":"2024-04-25T00:05:35.088951+0000","last_became_peered":"2024-04-25T00:05:35.088951+0000","last_unstale":"2024-04-25T00:05:36.122862+0000","last_undegraded":"2024-04-25T00:05:36.122862+0000","last_fullsized":"2024-04-25T00:05:36.122862+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:07.841191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110575+0000","last_change":"2024-04-25T00:05:36.106877+0000","last_active":"2024-04-25T00:05:36.110575+0000","last_peered":"2024-04-25T00:05:36.110575+0000","last_clean":"2024-04-25T00:05:36.110575+0000","last_became_active":"2024-04-25T00:05:36.106519+0000","last_became_peered":"2024-04-25T00:05:36.106519+0000","last_unstale":"2024-04-25T00:05:36.110575+0000","last_undegraded":"2024-04-25T00:05:36.110575+0000","last_fullsized":"2024-04-25T00:05:36.110575+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:17.892899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.097834+0000","last_change":"2024-04-25T00:05:35.093434+0000","last_active":"2024-04-25T00:05:36.097834+0000","last_peered":"2024-04-25T00:05:36.097834+0000","last_clean":"2024-04-25T00:05:36.097834+0000","last_became_active":"2024-04-25T00:05:35.093126+0000","last_became_peered":"2024-04-25T00:05:35.093126+0000","last_unstale":"2024-04-25T00:05:36.097834+0000","last_undegraded":"2024-04-25T00:05:36.097834+0000","last_fullsized":"2024-04-25T00:05:36.097834+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:24.987213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.111869+0000","last_change":"2024-04-25T00:05:36.109528+0000","last_active":"2024-04-25T00:05:36.111869+0000","last_peered":"2024-04-25T00:05:36.111869+0000","last_clean":"2024-04-25T00:05:36.111869+0000","last_became_active":"2024-04-25T00:05:36.108450+0000","last_became_peered":"2024-04-25T00:05:36.108450+0000","last_unstale":"2024-04-25T00:05:36.111869+0000","last_undegraded":"2024-04-25T00:05:36.111869+0000","last_fullsized":"2024-04-25T00:05:36.111869+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.857589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117407+0000","last_change":"2024-04-25T00:05:35.089429+0000","last_active":"2024-04-25T00:05:36.117407+0000","last_peered":"2024-04-25T00:05:36.117407+0000","last_clean":"2024-04-25T00:05:36.117407+0000","last_became_active":"2024-04-25T00:05:35.089148+0000","last_became_peered":"2024-04-25T00:05:35.089148+0000","last_unstale":"2024-04-25T00:05:36.117407+0000","last_undegraded":"2024-04-25T00:05:36.117407+0000","last_fullsized":"2024-04-25T00:05:36.117407+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:11.057511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.108485+0000","last_change":"2024-04-25T00:05:36.103675+0000","last_active":"2024-04-25T00:05:36.108485+0000","last_peered":"2024-04-25T00:05:36.108485+0000","last_clean":"2024-04-25T00:05:36.108485+0000","last_became_active":"2024-04-25T00:05:36.103419+0000","last_became_peered":"2024-04-25T00:05:36.103419+0000","last_unstale":"2024-04-25T00:05:36.108485+0000","last_undegraded":"2024-04-25T00:05:36.108485+0000","last_fullsized":"2024-04-25T00:05:36.108485+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:08:29.921695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.093550+0000","last_change":"2024-04-25T00:05:35.088725+0000","last_active":"2024-04-25T00:05:36.093550+0000","last_peered":"2024-04-25T00:05:36.093550+0000","last_clean":"2024-04-25T00:05:36.093550+0000","last_became_active":"2024-04-25T00:05:35.088485+0000","last_became_peered":"2024-04-25T00:05:35.088485+0000","last_unstale":"2024-04-25T00:05:36.093550+0000","last_undegraded":"2024-04-25T00:05:36.093550+0000","last_fullsized":"2024-04-25T00:05:36.093550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:02.911753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107893+0000","last_change":"2024-04-25T00:05:36.104271+0000","last_active":"2024-04-25T00:05:36.107893+0000","last_peered":"2024-04-25T00:05:36.107893+0000","last_clean":"2024-04-25T00:05:36.107893+0000","last_became_active":"2024-04-25T00:05:36.103983+0000","last_became_peered":"2024-04-25T00:05:36.103983+0000","last_unstale":"2024-04-25T00:05:36.107893+0000","last_undegraded":"2024-04-25T00:05:36.107893+0000","last_fullsized":"2024-04-25T00:05:36.107893+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:38.927057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.099227+0000","last_change":"2024-04-25T00:05:35.089785+0000","last_active":"2024-04-25T00:05:36.099227+0000","last_peered":"2024-04-25T00:05:36.099227+0000","last_clean":"2024-04-25T00:05:36.099227+0000","last_became_active":"2024-04-25T00:05:35.089537+0000","last_became_peered":"2024-04-25T00:05:35.089537+0000","last_unstale":"2024-04-25T00:05:36.099227+0000","last_undegraded":"2024-04-25T00:05:36.099227+0000","last_fullsized":"2024-04-25T00:05:36.099227+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:59.493199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105777+0000","last_change":"2024-04-25T00:05:36.104803+0000","last_active":"2024-04-25T00:05:36.105777+0000","last_peered":"2024-04-25T00:05:36.105777+0000","last_clean":"2024-04-25T00:05:36.105777+0000","last_became_active":"2024-04-25T00:05:36.104448+0000","last_became_peered":"2024-04-25T00:05:36.104448+0000","last_unstale":"2024-04-25T00:05:36.105777+0000","last_undegraded":"2024-04-25T00:05:36.105777+0000","last_fullsized":"2024-04-25T00:05:36.105777+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.669634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098425+0000","last_change":"2024-04-25T00:05:35.092040+0000","last_active":"2024-04-25T00:05:36.098425+0000","last_peered":"2024-04-25T00:05:36.098425+0000","last_clean":"2024-04-25T00:05:36.098425+0000","last_became_active":"2024-04-25T00:05:35.091734+0000","last_became_peered":"2024-04-25T00:05:35.091734+0000","last_unstale":"2024-04-25T00:05:36.098425+0000","last_undegraded":"2024-04-25T00:05:36.098425+0000","last_fullsized":"2024-04-25T00:05:36.098425+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:58.880900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.119117+0000","last_change":"2024-04-25T00:05:36.118073+0000","last_active":"2024-04-25T00:05:36.119117+0000","last_peered":"2024-04-25T00:05:36.119117+0000","last_clean":"2024-04-25T00:05:36.119117+0000","last_became_active":"2024-04-25T00:05:36.117744+0000","last_became_peered":"2024-04-25T00:05:36.117744+0000","last_unstale":"2024-04-25T00:05:36.119117+0000","last_undegraded":"2024-04-25T00:05:36.119117+0000","last_fullsized":"2024-04-25T00:05:36.119117+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:16.960156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.122704+0000","last_change":"2024-04-25T00:05:35.093246+0000","last_active":"2024-04-25T00:05:36.122704+0000","last_peered":"2024-04-25T00:05:36.122704+0000","last_clean":"2024-04-25T00:05:36.122704+0000","last_became_active":"2024-04-25T00:05:35.092946+0000","last_became_peered":"2024-04-25T00:05:35.092946+0000","last_unstale":"2024-04-25T00:05:36.122704+0000","last_undegraded":"2024-04-25T00:05:36.122704+0000","last_fullsized":"2024-04-25T00:05:36.122704+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:17.909900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.106293+0000","last_change":"2024-04-25T00:05:36.100054+0000","last_active":"2024-04-25T00:05:36.106293+0000","last_peered":"2024-04-25T00:05:36.106293+0000","last_clean":"2024-04-25T00:05:36.106293+0000","last_became_active":"2024-04-25T00:05:36.099801+0000","last_became_peered":"2024-04-25T00:05:36.099801+0000","last_unstale":"2024-04-25T00:05:36.106293+0000","last_undegraded":"2024-04-25T00:05:36.106293+0000","last_fullsized":"2024-04-25T00:05:36.106293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:12:59.169658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098541+0000","last_change":"2024-04-25T00:05:35.093353+0000","last_active":"2024-04-25T00:05:36.098541+0000","last_peered":"2024-04-25T00:05:36.098541+0000","last_clean":"2024-04-25T00:05:36.098541+0000","last_became_active":"2024-04-25T00:05:35.092936+0000","last_became_peered":"2024-04-25T00:05:35.092936+0000","last_unstale":"2024-04-25T00:05:36.098541+0000","last_undegraded":"2024-04-25T00:05:36.098541+0000","last_fullsized":"2024-04-25T00:05:36.098541+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:12.044548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.116574+0000","last_change":"2024-04-25T00:05:36.109132+0000","last_active":"2024-04-25T00:05:36.116574+0000","last_peered":"2024-04-25T00:05:36.116574+0000","last_clean":"2024-04-25T00:05:36.116574+0000","last_became_active":"2024-04-25T00:05:36.108323+0000","last_became_peered":"2024-04-25T00:05:36.108323+0000","last_unstale":"2024-04-25T00:05:36.116574+0000","last_undegraded":"2024-04-25T00:05:36.116574+0000","last_fullsized":"2024-04-25T00:05:36.116574+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:22.705213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.094149+0000","last_change":"2024-04-25T00:05:35.091713+0000","last_active":"2024-04-25T00:05:36.094149+0000","last_peered":"2024-04-25T00:05:36.094149+0000","last_clean":"2024-04-25T00:05:36.094149+0000","last_became_active":"2024-04-25T00:05:35.091271+0000","last_became_peered":"2024-04-25T00:05:35.091271+0000","last_unstale":"2024-04-25T00:05:36.094149+0000","last_undegraded":"2024-04-25T00:05:36.094149+0000","last_fullsized":"2024-04-25T00:05:36.094149+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:37.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.112512+0000","last_change":"2024-04-25T00:05:36.110156+0000","last_active":"2024-04-25T00:05:36.112512+0000","last_peered":"2024-04-25T00:05:36.112512+0000","last_clean":"2024-04-25T00:05:36.112512+0000","last_became_active":"2024-04-25T00:05:36.109897+0000","last_became_peered":"2024-04-25T00:05:36.109897+0000","last_unstale":"2024-04-25T00:05:36.112512+0000","last_undegraded":"2024-04-25T00:05:36.112512+0000","last_fullsized":"2024-04-25T00:05:36.112512+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:49.814589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.097997+0000","last_change":"2024-04-25T00:05:35.093134+0000","last_active":"2024-04-25T00:05:36.097997+0000","last_peered":"2024-04-25T00:05:36.097997+0000","last_clean":"2024-04-25T00:05:36.097997+0000","last_became_active":"2024-04-25T00:05:35.092413+0000","last_became_peered":"2024-04-25T00:05:35.092413+0000","last_unstale":"2024-04-25T00:05:36.097997+0000","last_undegraded":"2024-04-25T00:05:36.097997+0000","last_fullsized":"2024-04-25T00:05:36.097997+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:02.770290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.107838+0000","last_change":"2024-04-25T00:05:36.102745+0000","last_active":"2024-04-25T00:05:36.107838+0000","last_peered":"2024-04-25T00:05:36.107838+0000","last_clean":"2024-04-25T00:05:36.107838+0000","last_became_active":"2024-04-25T00:05:36.102374+0000","last_became_peered":"2024-04-25T00:05:36.102374+0000","last_unstale":"2024-04-25T00:05:36.107838+0000","last_undegraded":"2024-04-25T00:05:36.107838+0000","last_fullsized":"2024-04-25T00:05:36.107838+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:49.814589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.099299+0000","last_change":"2024-04-25T00:05:35.092899+0000","last_active":"2024-04-25T00:05:36.099299+0000","last_peered":"2024-04-25T00:05:36.099299+0000","last_clean":"2024-04-25T00:05:36.099299+0000","last_became_active":"2024-04-25T00:05:35.092639+0000","last_became_peered":"2024-04-25T00:05:35.092639+0000","last_unstale":"2024-04-25T00:05:36.099299+0000","last_undegraded":"2024-04-25T00:05:36.099299+0000","last_fullsized":"2024-04-25T00:05:36.099299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:07.621813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.105488+0000","last_change":"2024-04-25T00:05:36.101463+0000","last_active":"2024-04-25T00:05:36.105488+0000","last_peered":"2024-04-25T00:05:36.105488+0000","last_clean":"2024-04-25T00:05:36.105488+0000","last_became_active":"2024-04-25T00:05:36.101198+0000","last_became_peered":"2024-04-25T00:05:36.101198+0000","last_unstale":"2024-04-25T00:05:36.105488+0000","last_undegraded":"2024-04-25T00:05:36.105488+0000","last_fullsized":"2024-04-25T00:05:36.105488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:35.094475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.094517+0000","last_change":"2024-04-25T00:05:35.088045+0000","last_active":"2024-04-25T00:05:36.094517+0000","last_peered":"2024-04-25T00:05:36.094517+0000","last_clean":"2024-04-25T00:05:36.094517+0000","last_became_active":"2024-04-25T00:05:35.087063+0000","last_became_peered":"2024-04-25T00:05:35.087063+0000","last_unstale":"2024-04-25T00:05:36.094517+0000","last_undegraded":"2024-04-25T00:05:36.094517+0000","last_fullsized":"2024-04-25T00:05:36.094517+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:30.853248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.110167+0000","last_change":"2024-04-25T00:05:36.108367+0000","last_active":"2024-04-25T00:05:36.110167+0000","last_peered":"2024-04-25T00:05:36.110167+0000","last_clean":"2024-04-25T00:05:36.110167+0000","last_became_active":"2024-04-25T00:05:36.108091+0000","last_became_peered":"2024-04-25T00:05:36.108091+0000","last_unstale":"2024-04-25T00:05:36.110167+0000","last_undegraded":"2024-04-25T00:05:36.110167+0000","last_fullsized":"2024-04-25T00:05:36.110167+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:59.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.093484+0000","last_change":"2024-04-25T00:05:35.089435+0000","last_active":"2024-04-25T00:05:36.093484+0000","last_peered":"2024-04-25T00:05:36.093484+0000","last_clean":"2024-04-25T00:05:36.093484+0000","last_became_active":"2024-04-25T00:05:35.089190+0000","last_became_peered":"2024-04-25T00:05:35.089190+0000","last_unstale":"2024-04-25T00:05:36.093484+0000","last_undegraded":"2024-04-25T00:05:36.093484+0000","last_fullsized":"2024-04-25T00:05:36.093484+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:59.493199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.117577+0000","last_change":"2024-04-25T00:05:36.111080+0000","last_active":"2024-04-25T00:05:36.117577+0000","last_peered":"2024-04-25T00:05:36.117577+0000","last_clean":"2024-04-25T00:05:36.117577+0000","last_became_active":"2024-04-25T00:05:36.110807+0000","last_became_peered":"2024-04-25T00:05:36.110807+0000","last_unstale":"2024-04-25T00:05:36.117577+0000","last_undegraded":"2024-04-25T00:05:36.117577+0000","last_fullsized":"2024-04-25T00:05:36.117577+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:03.090823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.098561+0000","last_change":"2024-04-25T00:05:35.090943+0000","last_active":"2024-04-25T00:05:36.098561+0000","last_peered":"2024-04-25T00:05:36.098561+0000","last_clean":"2024-04-25T00:05:36.098561+0000","last_became_active":"2024-04-25T00:05:35.090691+0000","last_became_peered":"2024-04-25T00:05:35.090691+0000","last_unstale":"2024-04-25T00:05:36.098561+0000","last_undegraded":"2024-04-25T00:05:36.098561+0000","last_fullsized":"2024-04-25T00:05:36.098561+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:00:42.872357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.104765+0000","last_change":"2024-04-25T00:05:36.102794+0000","last_active":"2024-04-25T00:05:36.104765+0000","last_peered":"2024-04-25T00:05:36.104765+0000","last_clean":"2024-04-25T00:05:36.104765+0000","last_became_active":"2024-04-25T00:05:36.101472+0000","last_became_peered":"2024-04-25T00:05:36.101472+0000","last_unstale":"2024-04-25T00:05:36.104765+0000","last_undegraded":"2024-04-25T00:05:36.104765+0000","last_fullsized":"2024-04-25T00:05:36.104765+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:35.074585+0000","last_clean_scrub_stamp":"2024-04-25T00:05:35.074585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:51.260575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.099955+0000","last_change":"2024-04-25T00:05:35.093364+0000","last_active":"2024-04-25T00:05:36.099955+0000","last_peered":"2024-04-25T00:05:36.099955+0000","last_clean":"2024-04-25T00:05:36.099955+0000","last_became_active":"2024-04-25T00:05:35.093088+0000","last_became_peered":"2024-04-25T00:05:35.093088+0000","last_unstale":"2024-04-25T00:05:36.099955+0000","last_undegraded":"2024-04-25T00:05:36.099955+0000","last_fullsized":"2024-04-25T00:05:36.099955+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:05:34.069569+0000","last_clean_scrub_stamp":"2024-04-25T00:05:34.069569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:53.265715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":340,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T00:05:36.100553+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:05:36.100553+0000","last_peered":"2024-04-25T00:05:36.100553+0000","last_clean":"2024-04-25T00:05:36.100553+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:05:36.100553+0000","last_undegraded":"2024-04-25T00:05:36.100553+0000","last_fullsized":"2024-04-25T00:05:36.100553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2011136,"data_stored":1202638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46456,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325384,"statfs":{"total":96636764160,"available":96589193216,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46460,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325380,"statfs":{"total":96636764160,"available":96589189120,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54780,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317060,"statfs":{"total":96636764160,"available":96580669440,"internally_reserved":0,"allocated":1417216,"data_stored":612270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41464,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330376,"statfs":{"total":96636764160,"available":96594305024,"internally_reserved":0,"allocated":2011136,"data_stored":1202638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:05:39.412 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:05:39.412 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:05:39.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.552+0000 7fe42f26d640 1 Processor -- start 2024-04-25T00:05:39.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.552+0000 7fe42f26d640 1 -- start start 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42f26d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe42815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42f26d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4281604d0 0x7fe4280cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42f26d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42815d6b0 con 0x7fe42813f050 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42f26d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe42815c6a0 con 0x7fe4281604d0 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42f26d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe42815c8f0 con 0x7fe4280d06c0 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42d80a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4281604d0 0x7fe4280cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42d80a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4281604d0 0x7fe4280cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47954/0 (socket says 172.21.15.113:47954) 2024-04-25T00:05:39.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42d80a640 1 -- 172.21.15.113:0/3359215111 learned_addr learned my addr 172.21.15.113:0/3359215111 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.553+0000 7fe42d80a640 1 -- 172.21.15.113:0/3359215111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 msgr2=0x7fe42815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe42c808640 1 --2- 172.21.15.113:0/3359215111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe42815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe42d80a640 1 --2- 172.21.15.113:0/3359215111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe42815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe42d80a640 1 -- 172.21.15.113:0/3359215111 >> v1:172.21.15.178:6789/0 conn(0x7fe42813f050 legacy=0x7fe4280cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe42d80a640 1 -- 172.21.15.113:0/3359215111 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42815cbd0 con 0x7fe4281604d0 2024-04-25T00:05:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe42c808640 1 --2- 172.21.15.113:0/3359215111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe42815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe42d80a640 1 --2- 172.21.15.113:0/3359215111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe4280cea60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe420009870 tx=0x7fe42002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cb81bf043aa575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe41ffff640 1 -- 172.21.15.113:0/3359215111 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420037070 con 0x7fe4281604d0 2024-04-25T00:05:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe41ffff640 1 -- 172.21.15.113:0/3359215111 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe420032440 con 0x7fe4281604d0 2024-04-25T00:05:39.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.554+0000 7fe41ffff640 1 -- 172.21.15.113:0/3359215111 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420032810 con 0x7fe4281604d0 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 -- 172.21.15.113:0/3359215111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 msgr2=0x7fe4280cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 --2- 172.21.15.113:0/3359215111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe4280cea60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe420009870 tx=0x7fe42002a180 comp rx=0 tx=0).stop 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 -- 172.21.15.113:0/3359215111 shutdown_connections 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 --2- 172.21.15.113:0/3359215111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe42815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 --2- 172.21.15.113:0/3359215111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe4280cea60 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 -- 172.21.15.113:0/3359215111 >> 172.21.15.113:0/3359215111 conn(0x7fe42806b0f0 msgr2=0x7fe42806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 -- 172.21.15.113:0/3359215111 shutdown_connections 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.555+0000 7fe42f26d640 1 -- 172.21.15.113:0/3359215111 wait complete. 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.556+0000 7fe42f26d640 1 Processor -- start 2024-04-25T00:05:39.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.556+0000 7fe42f26d640 1 -- start start 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42f26d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe428156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42f26d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe42813f050 0x7fe4281f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42f26d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe42815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42f26d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe42813f430 con 0x7fe4281604d0 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42f26d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe42815d190 con 0x7fe42813f050 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42f26d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe4281608d0 con 0x7fe4280d06c0 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe42815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe42815b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47970/0 (socket says 172.21.15.113:47970) 2024-04-25T00:05:39.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 -- 172.21.15.113:0/434469112 learned_addr learned my addr 172.21.15.113:0/434469112 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 -- 172.21.15.113:0/434469112 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 msgr2=0x7fe428156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe428156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 -- 172.21.15.113:0/434469112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe42813f050 msgr2=0x7fe4281f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe42813f050 0x7fe4281f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 -- 172.21.15.113:0/434469112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42814ad20 con 0x7fe4281604d0 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.557+0000 7fe42d80a640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe42815b190 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe42002a1c0 tx=0x7fe4200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.558+0000 7fe41dffb640 1 -- 172.21.15.113:0/434469112 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420041020 con 0x7fe4281604d0 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.558+0000 7fe41dffb640 1 -- 172.21.15.113:0/434469112 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe420032e50 con 0x7fe4281604d0 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.558+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42814baa0 con 0x7fe4281604d0 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.558+0000 7fe41dffb640 1 -- 172.21.15.113:0/434469112 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420037040 con 0x7fe4281604d0 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.558+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42814b2a0 con 0x7fe4281604d0 2024-04-25T00:05:39.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.559+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4280691a0 con 0x7fe4281604d0 2024-04-25T00:05:39.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.559+0000 7fe41dffb640 1 -- 172.21.15.113:0/434469112 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe420044020 con 0x7fe4281604d0 2024-04-25T00:05:39.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.560+0000 7fe41dffb640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe40c06be30 0x7fe40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.560+0000 7fe41dffb640 1 -- 172.21.15.113:0/434469112 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fe4200ab880 con 0x7fe4281604d0 2024-04-25T00:05:39.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.560+0000 7fe42d009640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe40c06be30 0x7fe40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.560+0000 7fe42d009640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe40c06be30 0x7fe40c06e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4240043c0 tx=0x7fe424007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.564+0000 7fe41dffb640 1 -- 172.21.15.113:0/434469112 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe42004b030 con 0x7fe4281604d0 2024-04-25T00:05:39.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.802+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe4280d0e50 con 0x7fe4281604d0 2024-04-25T00:05:39.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.805+0000 7fe41dffb640 1 -- 172.21.15.113:0/434469112 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fe42007a230 con 0x7fe4281604d0 2024-04-25T00:05:39.805 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:05:39.805 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:05:39.805 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:05:39.805 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:05:39.805 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:05:39.806 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:05:39.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe40c06be30 msgr2=0x7fe40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe40c06be30 0x7fe40c06e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4240043c0 tx=0x7fe424007d30 comp rx=0 tx=0).stop 2024-04-25T00:05:39.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 msgr2=0x7fe42815b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe42815b190 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe42002a1c0 tx=0x7fe4200046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 shutdown_connections 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe40c06be30 0x7fe40c06e2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe42813f050 0x7fe4281f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281604d0 0x7fe42815b190 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 --2- 172.21.15.113:0/434469112 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4280d06c0 0x7fe428156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 >> 172.21.15.113:0/434469112 conn(0x7fe42806b0f0 msgr2=0x7fe428151740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 shutdown_connections 2024-04-25T00:05:39.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.807+0000 7fe42f26d640 1 -- 172.21.15.113:0/434469112 wait complete. 2024-04-25T00:05:39.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:39.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.915+0000 7ff8c7837640 1 Processor -- start 2024-04-25T00:05:39.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.915+0000 7ff8c7837640 1 -- start start 2024-04-25T00:05:39.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c7837640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0072ea0 0x7ff8c0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c7837640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8c00b3af0 0x7ff8c00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c7837640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c00b2d50 con 0x7ff8c015a6b0 2024-04-25T00:05:39.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c7837640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c0072a70 con 0x7ff8c00b3af0 2024-04-25T00:05:39.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c7837640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c00b3010 con 0x7ff8c0072ea0 2024-04-25T00:05:39.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c55d3640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff8c015a6b0 0x7ff8c0070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54984/0 (socket says 172.21.15.113:54984) 2024-04-25T00:05:39.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c5dd4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8c00b3af0 0x7ff8c00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.916+0000 7ff8c55d3640 1 -- 172.21.15.113:0/3053360849 learned_addr learned my addr 172.21.15.113:0/3053360849 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:39.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8c4dd2640 1 --2- 172.21.15.113:0/3053360849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0072ea0 0x7ff8c0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8c5dd4640 1 -- 172.21.15.113:0/3053360849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0072ea0 msgr2=0x7ff8c0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:39.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8c5dd4640 1 --2- 172.21.15.113:0/3053360849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0072ea0 0x7ff8c0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8c5dd4640 1 -- 172.21.15.113:0/3053360849 >> v1:172.21.15.113:6789/0 conn(0x7ff8c015a6b0 legacy=0x7ff8c0070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8b7fff640 1 -- 172.21.15.113:0/3053360849 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4129963257 0 0) 0x7ff8c00b2d50 con 0x7ff8c015a6b0 2024-04-25T00:05:39.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8c5dd4640 1 -- 172.21.15.113:0/3053360849 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c00b32f0 con 0x7ff8c00b3af0 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8c5dd4640 1 --2- 172.21.15.113:0/3053360849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c00b3af0 0x7ff8c00b3f80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009d90 tx=0x7ff8b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ff5ca02639b2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.917+0000 7ff8b7fff640 1 -- 172.21.15.113:0/3053360849 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8007740 con 0x7ff8c00b3af0 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8b7fff640 1 -- 172.21.15.113:0/3053360849 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b80078e0 con 0x7ff8c00b3af0 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8b7fff640 1 -- 172.21.15.113:0/3053360849 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8007cb0 con 0x7ff8c00b3af0 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 -- 172.21.15.113:0/3053360849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c00b3af0 msgr2=0x7ff8c00b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 --2- 172.21.15.113:0/3053360849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c00b3af0 0x7ff8c00b3f80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009d90 tx=0x7ff8b802a660 comp rx=0 tx=0).stop 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 -- 172.21.15.113:0/3053360849 shutdown_connections 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 --2- 172.21.15.113:0/3053360849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c00b3af0 0x7ff8c00b3f80 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 --2- 172.21.15.113:0/3053360849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0072ea0 0x7ff8c0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 -- 172.21.15.113:0/3053360849 >> 172.21.15.113:0/3053360849 conn(0x7ff8c00b1a50 msgr2=0x7ff8c00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 -- 172.21.15.113:0/3053360849 shutdown_connections 2024-04-25T00:05:39.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.918+0000 7ff8c7837640 1 -- 172.21.15.113:0/3053360849 wait complete. 2024-04-25T00:05:39.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.919+0000 7ff8c7837640 1 Processor -- start 2024-04-25T00:05:39.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.919+0000 7ff8c7837640 1 -- start start 2024-04-25T00:05:39.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c7837640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c0072ea0 0x7ff8c0147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c7837640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00b3af0 0x7ff8c0152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c7837640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c015a6b0 0x7ff8c01f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c7837640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c015aa90 con 0x7ff8c015a6b0 2024-04-25T00:05:39.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c7837640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c0072720 con 0x7ff8c0072ea0 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c7837640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c015ac10 con 0x7ff8c00b3af0 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c015a6b0 0x7ff8c01f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c015a6b0 0x7ff8c01f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47988/0 (socket says 172.21.15.113:47988) 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 -- 172.21.15.113:0/40976854 learned_addr learned my addr 172.21.15.113:0/40976854 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 -- 172.21.15.113:0/40976854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00b3af0 msgr2=0x7ff8c0152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00b3af0 0x7ff8c0152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 -- 172.21.15.113:0/40976854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c0072ea0 msgr2=0x7ff8c0147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c0072ea0 0x7ff8c0147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 -- 172.21.15.113:0/40976854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c01535e0 con 0x7ff8c015a6b0 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.920+0000 7ff8c5dd4640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c015a6b0 0x7ff8c01f5be0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009d90 tx=0x7ff8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.921+0000 7ff8b5ffb640 1 -- 172.21.15.113:0/40976854 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8040070 con 0x7ff8c015a6b0 2024-04-25T00:05:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.921+0000 7ff8b5ffb640 1 -- 172.21.15.113:0/40976854 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b8031bc0 con 0x7ff8c015a6b0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.921+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c01538c0 con 0x7ff8c015a6b0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.921+0000 7ff8b5ffb640 1 -- 172.21.15.113:0/40976854 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8009450 con 0x7ff8c015a6b0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.921+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c01459b0 con 0x7ff8c015a6b0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.922+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c00691a0 con 0x7ff8c015a6b0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.922+0000 7ff8b5ffb640 1 -- 172.21.15.113:0/40976854 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8b8039040 con 0x7ff8c015a6b0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.923+0000 7ff8b5ffb640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8a406be30 0x7ff8a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.923+0000 7ff8b5ffb640 1 -- 172.21.15.113:0/40976854 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7ff8b80ac290 con 0x7ff8c015a6b0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.923+0000 7ff8c55d3640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8a406be30 0x7ff8a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.923+0000 7ff8c55d3640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8a406be30 0x7ff8a406e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:39.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:39.927+0000 7ff8b5ffb640 1 -- 172.21.15.113:0/40976854 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8b807acb0 con 0x7ff8c015a6b0 2024-04-25T00:05:40.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.135+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8c01541d0 con 0x7ff8c015a6b0 2024-04-25T00:05:40.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.135+0000 7ff8b5ffb640 1 -- 172.21.15.113:0/40976854 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7ff8b807a5b0 con 0x7ff8c015a6b0 2024-04-25T00:05:40.136 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:05:40.136 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8a406be30 msgr2=0x7ff8a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8a406be30 0x7ff8a406e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc007960 comp rx=0 tx=0).stop 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c015a6b0 msgr2=0x7ff8c01f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c015a6b0 0x7ff8c01f5be0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009d90 tx=0x7ff8b8004a40 comp rx=0 tx=0).stop 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 shutdown_connections 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8a406be30 0x7ff8a406e2f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c015a6b0 0x7ff8c01f5be0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00b3af0 0x7ff8c0152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 --2- 172.21.15.113:0/40976854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c0072ea0 0x7ff8c0147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 >> 172.21.15.113:0/40976854 conn(0x7ff8c00b1a50 msgr2=0x7ff8c0070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 shutdown_connections 2024-04-25T00:05:40.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.137+0000 7ff8c7837640 1 -- 172.21.15.113:0/40976854 wait complete. 2024-04-25T00:05:40.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:05:35.432457+0000', 'modified': '2024-04-25T00:05:37.447344+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8400': {'gid': 8400, 'name': 'b', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6835/840156279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6833', 'nonce': 840156279}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 840156279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8400, 'qdb_cluster': [8400]} 2024-04-25T00:05:40.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:05:40.552 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:05:40.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.650+0000 7f58b0076640 1 Processor -- start 2024-04-25T00:05:40.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.650+0000 7f58b0076640 1 -- start start 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58b0076640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 0x7f58a815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58b0076640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a8071170 con 0x7f58a815b670 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58b0076640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a8071450 con 0x7f58a806cea0 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58b0076640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58a8071730 con 0x7f58a815ba50 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ade12640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f58a815b670 0x7f58a80cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54992/0 (socket says 172.21.15.113:54992) 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ade12640 1 -- 172.21.15.113:0/3473807089 learned_addr learned my addr 172.21.15.113:0/3473807089 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ad611640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 0x7f58a815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2938786297 0 0) 0x7f58a8071170 con 0x7f58a815b670 2024-04-25T00:05:40.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5894003520 con 0x7f58a815b670 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1744270108 0 0) 0x7f5894003520 con 0x7f58a815b670 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 msgr2=0x7f58a815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 0x7f58a815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 >> v1:172.21.15.178:6789/0 conn(0x7f58a806cea0 legacy=0x7f58a8072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a8071a10 con 0x7f58a815b670 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f58a4001670 con 0x7f58a815b670 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 >> v1:172.21.15.113:6789/0 conn(0x7f58a815b670 legacy=0x7f58a80cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f5894003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 0x7f5894004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.651+0000 7f58ace10640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5894004d40 0x7f58940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8071730 con 0x7f58940039e0 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f589400bbb0 con 0x7f5894004d40 2024-04-25T00:05:40.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f589400bdb0 con 0x7f58a815ba50 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ade12640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f5894003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ad611640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 0x7f5894004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ae613640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5894004d40 0x7f58940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ade12640 1 -- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 msgr2=0x7f5894004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ade12640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a815ba50 0x7f5894004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ade12640 1 -- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5894004d40 msgr2=0x7f58940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ade12640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5894004d40 0x7f58940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.652+0000 7f58ade12640 1 -- 172.21.15.113:0/3473807089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5894003520 con 0x7f58940039e0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58ade12640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f5894003e00 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c210 tx=0x7f5894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c98125d15892bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4007d10 con 0x7f58940039e0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a40058d0 con 0x7f58940039e0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58ace10640 1 -- 172.21.15.113:0/3473807089 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4005ca0 con 0x7f58940039e0 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58b0076640 1 -- 172.21.15.113:0/3473807089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 msgr2=0x7f5894003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58b0076640 1 --2- 172.21.15.113:0/3473807089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f5894003e00 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c210 tx=0x7f5894005830 comp rx=0 tx=0).stop 2024-04-25T00:05:40.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58ae613640 1 -- 172.21.15.113:0/3473807089 reap_dead start 2024-04-25T00:05:40.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58b0076640 1 -- 172.21.15.113:0/3473807089 shutdown_connections 2024-04-25T00:05:40.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58b0076640 1 -- 172.21.15.113:0/3473807089 >> 172.21.15.113:0/3473807089 conn(0x7f58a806b0f0 msgr2=0x7f58a806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58b0076640 1 -- 172.21.15.113:0/3473807089 shutdown_connections 2024-04-25T00:05:40.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.653+0000 7f58b0076640 1 -- 172.21.15.113:0/3473807089 wait complete. 2024-04-25T00:05:40.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.654+0000 7f58b0076640 1 Processor -- start 2024-04-25T00:05:40.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.654+0000 7f58b0076640 1 -- start start 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58b0076640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f58a8144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58b0076640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a8145540 0x7f58a81459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58b0076640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a80c5b40 0x7f58a80c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58b0076640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58a815c170 con 0x7f58940039e0 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58b0076640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a815bff0 con 0x7f58a80c5b40 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58b0076640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58a815c2f0 con 0x7f58a8145540 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58ade12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f58a8144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58ade12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f58a8144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48006/0 (socket says 172.21.15.113:48006) 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58ade12640 1 -- 172.21.15.113:0/367611070 learned_addr learned my addr 172.21.15.113:0/367611070 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.655+0000 7f58ad611640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a8145540 0x7f58a81459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58ade12640 1 -- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a8145540 msgr2=0x7f58a81459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58ade12640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a8145540 0x7f58a81459c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58ae613640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a80c5b40 0x7f58a80c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58ade12640 1 -- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a80c5b40 msgr2=0x7f58a80c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58ade12640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a80c5b40 0x7f58a80c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58ade12640 1 -- 172.21.15.113:0/367611070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a8146020 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58ade12640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f58a8144bd0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c300 tx=0x7f58a8071340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f589e7fc640 1 -- 172.21.15.113:0/367611070 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4001c70 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f589e7fc640 1 -- 172.21.15.113:0/367611070 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a4001080 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a8146300 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f589e7fc640 1 -- 172.21.15.113:0/367611070 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4007d10 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.656+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a81468c0 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.657+0000 7f589e7fc640 1 -- 172.21.15.113:0/367611070 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58a4043030 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.658+0000 7f589e7fc640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f588c06be30 0x7f588c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.658+0000 7f589e7fc640 1 -- 172.21.15.113:0/367611070 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f58a402f070 con 0x7f58940039e0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.658+0000 7f58ad611640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f588c06be30 0x7f588c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.658+0000 7f58ad611640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f588c06be30 0x7f588c06e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f58a80cc8a0 tx=0x7f5898008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.659+0000 7f58b2d88740 1 Processor -- start 2024-04-25T00:05:40.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.659+0000 7f58b2d88740 1 -- start start 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdc9b60 0x55779bd7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdacc80 0x55779bdac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdac540 0x55779bd81350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58b2d88740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55779bce4f00 con 0x55779bdac540 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55779bc8d750 con 0x55779bdc9b60 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55779bce0ec0 con 0x55779bdacc80 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdac540 0x55779bd81350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdac540 0x55779bd81350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48016/0 (socket says 172.21.15.113:48016) 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 -- 172.21.15.113:0/3219760148 learned_addr learned my addr 172.21.15.113:0/3219760148 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ade12640 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdc9b60 0x55779bd7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ad611640 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdacc80 0x55779bdac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 -- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdacc80 msgr2=0x55779bdac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdacc80 0x55779bdac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 -- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdc9b60 msgr2=0x55779bd7c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdc9b60 0x55779bd7c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 -- 172.21.15.113:0/3219760148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779b9f92d0 con 0x55779bdac540 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f58ae613640 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdac540 0x55779bd81350 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f58a8158a20 tx=0x7f58a0001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d781b16bad222b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f588b7fe640 1 -- 172.21.15.113:0/3219760148 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a001a070 con 0x55779bdac540 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f588b7fe640 1 -- 172.21.15.113:0/3219760148 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a0005070 con 0x55779bdac540 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.660+0000 7f588b7fe640 1 -- 172.21.15.113:0/3219760148 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0003e60 con 0x55779bdac540 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 -- 172.21.15.113:0/3219760148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdac540 msgr2=0x55779bd81350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdac540 0x55779bd81350 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f58a8158a20 tx=0x7f58a0001610 comp rx=0 tx=0).stop 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 -- 172.21.15.113:0/3219760148 shutdown_connections 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdac540 0x55779bd81350 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdacc80 0x55779bdac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 --2- 172.21.15.113:0/3219760148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdc9b60 0x55779bd7c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 -- 172.21.15.113:0/3219760148 >> 172.21.15.113:0/3219760148 conn(0x55779bd7e760 msgr2=0x55779bdc7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 -- 172.21.15.113:0/3219760148 shutdown_connections 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.661+0000 7f58b2d88740 1 -- 172.21.15.113:0/3219760148 wait complete. 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.662+0000 7f58b2d88740 1 -- start start 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.662+0000 7f58b2d88740 1 Processor -- start 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdaf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdd7ee0 0x55779bdc6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55779bd80910 con 0x55779bdc9b60 2024-04-25T00:05:40.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55779bca0cf0 con 0x55779bdacc80 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55779bd7fd00 con 0x55779bdd7ee0 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 10 client.0 _ll_register_callbacks cb 0x55779bca06e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdaf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdaf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48018/0 (socket says 172.21.15.113:48018) 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 -- 172.21.15.113:0/1998411168 learned_addr learned my addr 172.21.15.113:0/1998411168 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ae613640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdd7ee0 0x55779bdc6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ade12640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 -- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdd7ee0 msgr2=0x55779bdc6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdd7ee0 0x55779bdc6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 -- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 msgr2=0x55779bda7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 -- 172.21.15.113:0/1998411168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779bac36b0 con 0x55779bdc9b60 2024-04-25T00:05:40.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ade12640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58ad611640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdaf040 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5898013510 tx=0x7f5898004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c36e2835e00e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f5888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f5888ff9640 1 -- 172.21.15.113:0/1998411168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898019070 con 0x55779bdc9b60 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f5888ff9640 1 -- 172.21.15.113:0/1998411168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f5898013dd0 con 0x55779bdc9b60 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a0005070 con 0x55779bdc9b60 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f5888ff9640 1 -- 172.21.15.113:0/1998411168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f5898032050 con 0x55779bdc9b60 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 10 client.8526 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.663+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55779bdaf580 con 0x55779bdc9b60 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f5888ff9640 1 -- 172.21.15.113:0/1998411168 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a0005070 con 0x55779bdc9b60 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f5888ff9640 1 -- 172.21.15.113:0/1998411168 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f5898035020 con 0x55779bdc9b60 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58b2d88740 10 client.8526 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58b2d88740 10 client.8526 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58b2d88740 20 client.8526 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:40.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdaa040 0x55779bddac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58b2d88740 4 client.8526 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55779bcf2440 con 0x55779bdaa040 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58ade12640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdaa040 0x55779bddac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f5888ff9640 10 client.8526 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.664+0000 7f58ade12640 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdaa040 0x55779bddac20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efeb8d3be3604f94 server_cookie=49e0f4ca784bb46a in_seq=0 out_seq=0 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f5888ff9640 1 -- 172.21.15.113:0/1998411168 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55779bcf2440 con 0x55779bdaa040 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f5888ff9640 10 client.8526 handle_command_reply: tid=0 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 client.8526 shutdown 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 msgr2=0x55779bdaf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdaf040 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5898013510 tx=0x7f5898004730 comp rx=0 tx=0).stop 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 shutdown_connections 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdaa040 0x55779bddac20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdd7ee0 0x55779bdc6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f5888ff9640 0 client.8526 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:40.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdaf040 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1998411168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 >> 172.21.15.113:0/1998411168 conn(0x55779bdac540 msgr2=0x55779bda91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 shutdown_connections 2024-04-25T00:05:40.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.665+0000 7f58b2d88740 1 -- 172.21.15.113:0/1998411168 wait complete. 2024-04-25T00:05:40.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.666+0000 7f58b2d88740 20 client.8526 trim_cache size 0 max 16384 2024-04-25T00:05:40.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 Processor -- start 2024-04-25T00:05:40.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 -- start start 2024-04-25T00:05:40.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 0x55779bda9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bdab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdabf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81fdee0 con 0x55779bdc9b60 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8148410 con 0x55779bdacc80 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.684+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58a81fd890 con 0x55779bdaa040 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdabf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdabf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48032/0 (socket says 172.21.15.113:48032) 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 -- 172.21.15.113:0/1448144383 learned_addr learned my addr 172.21.15.113:0/1448144383 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 -- 172.21.15.113:0/1448144383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 msgr2=0x55779bda9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 0x55779bda9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ad611640 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bdab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 -- 172.21.15.113:0/1448144383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 msgr2=0x55779bdab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bdab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 -- 172.21.15.113:0/1448144383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779bdaf3b0 con 0x55779bdc9b60 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f58ae613640 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdabf20 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f58a000a000 tx=0x7f58a0005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14835ab3258c777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f588affd640 1 -- 172.21.15.113:0/1448144383 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a001a040 con 0x55779bdc9b60 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f588affd640 1 -- 172.21.15.113:0/1448144383 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a0006d90 con 0x55779bdc9b60 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.685+0000 7f588affd640 1 -- 172.21.15.113:0/1448144383 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0003e60 con 0x55779bdc9b60 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 -- 172.21.15.113:0/1448144383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 msgr2=0x55779bdabf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdabf20 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f58a000a000 tx=0x7f58a0005000 comp rx=0 tx=0).stop 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 -- 172.21.15.113:0/1448144383 shutdown_connections 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bdab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdabf20 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 --2- 172.21.15.113:0/1448144383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 0x55779bda9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 -- 172.21.15.113:0/1448144383 >> 172.21.15.113:0/1448144383 conn(0x55779bdac540 msgr2=0x55779bda7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 -- 172.21.15.113:0/1448144383 shutdown_connections 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 -- 172.21.15.113:0/1448144383 wait complete. 2024-04-25T00:05:40.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 -- start start 2024-04-25T00:05:40.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.686+0000 7f58b2d88740 1 Processor -- start 2024-04-25T00:05:40.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 0x55779bdbc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58b2d88740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58b2d88740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55779bda9ec0 con 0x55779bdc9b60 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55779bdc3cb0 con 0x55779bdacc80 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58b2d88740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55779bdaa4e0 con 0x55779bdaa040 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58b2d88740 10 client.0 _ll_register_callbacks cb 0x7f5898012c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58ae613640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58ae613640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48040/0 (socket says 172.21.15.113:48040) 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58ae613640 1 -- 172.21.15.113:0/2842443703 learned_addr learned my addr 172.21.15.113:0/2842443703 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58ade12640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 0x55779bdbc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58ad611640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58ae613640 1 -- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 msgr2=0x55779bdbc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.687+0000 7f58ae613640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 0x55779bdbc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58ae613640 1 -- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 msgr2=0x55779bda6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58ae613640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58ae613640 1 -- 172.21.15.113:0/2842443703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779bdb01a0 con 0x55779bdc9b60 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58ad611640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58ae613640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdab7b0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000f80 tx=0x7f58a0001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c40dc76bbe239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f5888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f5888ff9640 1 -- 172.21.15.113:0/2842443703 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0030060 con 0x55779bdc9b60 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f5888ff9640 1 -- 172.21.15.113:0/2842443703 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f58a00055c0 con 0x55779bdc9b60 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a0006d90 con 0x55779bdc9b60 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f5888ff9640 1 -- 172.21.15.113:0/2842443703 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f58a0036050 con 0x55779bdc9b60 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 10 client.8532 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55779bd7c400 con 0x55779bdc9b60 2024-04-25T00:05:40.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f5888ff9640 1 -- 172.21.15.113:0/2842443703 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a0006d90 con 0x55779bdc9b60 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f5888ff9640 1 -- 172.21.15.113:0/2842443703 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f58a0039020 con 0x55779bdc9b60 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 10 client.8532 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 10 client.8532 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 20 client.8532 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdd7ee0 0x55779bdabcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 4 client.8532 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "status"}) -- 0x55779bcec790 con 0x55779bdd7ee0 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.688+0000 7f58ade12640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdd7ee0 0x55779bdabcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f5888ff9640 10 client.8532 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58ade12640 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdd7ee0 0x55779bdabcf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f461822a09ceeaf3 server_cookie=f4103c39db86e340 in_seq=0 out_seq=0 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f5888ff9640 1 -- 172.21.15.113:0/2842443703 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55779bcec790 con 0x55779bdd7ee0 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f5888ff9640 10 client.8532 handle_command_reply: tid=0 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 client.8532 shutdown 2024-04-25T00:05:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 msgr2=0x55779bdab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdab7b0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000f80 tx=0x7f58a0001510 comp rx=0 tx=0).stop 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 shutdown_connections 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55779bdd7ee0 0x55779bdabcf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55779bdacc80 0x55779bda6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55779bdc9b60 0x55779bdab7b0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 --2- 172.21.15.113:0/2842443703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55779bdaa040 0x55779bdbc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f5888ff9640 0 client.8532 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.689+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 >> 172.21.15.113:0/2842443703 conn(0x55779bdac540 msgr2=0x55779bde0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 shutdown_connections 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b2d88740 1 -- 172.21.15.113:0/2842443703 wait complete. 2024-04-25T00:05:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b2d88740 20 client.8532 trim_cache size 0 max 16384 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 8400, 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.5982912210000002, 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 62, 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 116, 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 116, 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 9.6490005950000004 2024-04-25T00:05:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f588c06be30 msgr2=0x7f588c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f588c06be30 0x7f588c06e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f58a80cc8a0 tx=0x7f5898008040 comp rx=0 tx=0).stop 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 msgr2=0x7f58a8144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f58a8144bd0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f58a400c300 tx=0x7f58a8071340 comp rx=0 tx=0).stop 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 shutdown_connections 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f588c06be30 0x7f588c06e2f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a80c5b40 0x7f58a80c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58a8145540 0x7f58a81459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 --2- 172.21.15.113:0/367611070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940039e0 0x7f58a8144bd0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 >> 172.21.15.113:0/367611070 conn(0x7f58a806b0f0 msgr2=0x7f58a815a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 shutdown_connections 2024-04-25T00:05:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.690+0000 7f58b0076640 1 -- 172.21.15.113:0/367611070 wait complete. 2024-04-25T00:05:40.702 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:05:40.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.798+0000 7f9c71c5d640 1 Processor -- start 2024-04-25T00:05:40.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.798+0000 7f9c71c5d640 1 -- start start 2024-04-25T00:05:40.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c71c5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c15e320 0x7f9c6c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c71c5d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c6c0d2290 0x7f9c6c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c71c5d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c0d3de0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c71c5d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6c0d3410 con 0x7f9c6c0d2290 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c71c5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c0d2800 con 0x7f9c6c15e320 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c6c0d2290 0x7f9c6c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9c6c0d0b00 0x7f9c6c0b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55000/0 (socket says 172.21.15.113:55000) 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6bfff640 1 -- 172.21.15.113:0/3521588009 learned_addr learned my addr 172.21.15.113:0/3521588009 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6b7fe640 1 --2- 172.21.15.113:0/3521588009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c15e320 0x7f9c6c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6affd640 1 -- 172.21.15.113:0/3521588009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c15e320 msgr2=0x7f9c6c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6affd640 1 --2- 172.21.15.113:0/3521588009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c15e320 0x7f9c6c0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6affd640 1 -- 172.21.15.113:0/3521588009 >> v1:172.21.15.113:6789/0 conn(0x7f9c6c0d0b00 legacy=0x7f9c6c0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6affd640 1 -- 172.21.15.113:0/3521588009 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c0d2ae0 con 0x7f9c6c0d2290 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6a7fc640 1 -- 172.21.15.113:0/3521588009 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745406920 0 0) 0x7f9c6c0d3de0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.799+0000 7f9c6b7fe640 1 --2- 172.21.15.113:0/3521588009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c15e320 0x7f9c6c0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.800+0000 7f9c6affd640 1 --2- 172.21.15.113:0/3521588009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d2290 0x7f9c6c0b9aa0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f9c600097a0 tx=0x7f9c6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d84ec7ecd98102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.800+0000 7f9c6a7fc640 1 -- 172.21.15.113:0/3521588009 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60037070 con 0x7f9c6c0d2290 2024-04-25T00:05:40.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.800+0000 7f9c6a7fc640 1 -- 172.21.15.113:0/3521588009 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c60032440 con 0x7f9c6c0d2290 2024-04-25T00:05:40.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.800+0000 7f9c6a7fc640 1 -- 172.21.15.113:0/3521588009 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60032810 con 0x7f9c6c0d2290 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3521588009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d2290 msgr2=0x7f9c6c0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3521588009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d2290 0x7f9c6c0b9aa0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f9c600097a0 tx=0x7f9c6002a210 comp rx=0 tx=0).stop 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3521588009 shutdown_connections 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3521588009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d2290 0x7f9c6c0b9aa0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3521588009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c15e320 0x7f9c6c0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3521588009 >> 172.21.15.113:0/3521588009 conn(0x7f9c6c0b5c20 msgr2=0x7f9c6c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3521588009 shutdown_connections 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.801+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3521588009 wait complete. 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.802+0000 7f9c71c5d640 1 Processor -- start 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.802+0000 7f9c71c5d640 1 -- start start 2024-04-25T00:05:40.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d0b00 0x7f9c6c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c0d2290 0x7f9c6c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c6c15e320 0x7f9c6c160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c0b9fe0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c0d30a0 con 0x7f9c6c15e320 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c15e700 con 0x7f9c6c0d2290 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d0b00 0x7f9c6c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d0b00 0x7f9c6c153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48060/0 (socket says 172.21.15.113:48060) 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 -- 172.21.15.113:0/3287967002 learned_addr learned my addr 172.21.15.113:0/3287967002 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6affd640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c0d2290 0x7f9c6c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6bfff640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c6c15e320 0x7f9c6c160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 -- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c0d2290 msgr2=0x7f9c6c15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c0d2290 0x7f9c6c15f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 -- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c6c15e320 msgr2=0x7f9c6c160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c6c15e320 0x7f9c6c160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 -- 172.21.15.113:0/3287967002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c15fdb0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c6b7fe640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d0b00 0x7f9c6c153150 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00d800 tx=0x7f9c5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c53fff640 1 -- 172.21.15.113:0/3287967002 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c004030 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c53fff640 1 -- 172.21.15.113:0/3287967002 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c5c0041d0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c15eab0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c53fff640 1 -- 172.21.15.113:0/3287967002 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c009640 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.803+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c15f070 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.804+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6c0691a0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.805+0000 7f9c53fff640 1 -- 172.21.15.113:0/3287967002 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c5c0097e0 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.805+0000 7f9c53fff640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c5806be30 0x7f9c5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.805+0000 7f9c53fff640 1 -- 172.21.15.113:0/3287967002 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f9c5c08b020 con 0x7f9c6c0d0b00 2024-04-25T00:05:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.805+0000 7f9c6affd640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c5806be30 0x7f9c5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:40.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.806+0000 7f9c6affd640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c5806be30 0x7f9c5806e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9c60034000 tx=0x7f9c6002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:40.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:40.810+0000 7f9c53fff640 1 -- 172.21.15.113:0/3287967002 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c5c059a60 con 0x7f9c6c0d0b00 2024-04-25T00:05:41.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.018+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9c6c15daa0 con 0x7f9c6c0d0b00 2024-04-25T00:05:41.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.018+0000 7f9c53fff640 1 -- 172.21.15.113:0/3287967002 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f9c5c059360 con 0x7f9c6c0d0b00 2024-04-25T00:05:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c5806be30 msgr2=0x7f9c5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c5806be30 0x7f9c5806e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9c60034000 tx=0x7f9c6002e000 comp rx=0 tx=0).stop 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d0b00 msgr2=0x7f9c6c153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d0b00 0x7f9c6c153150 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00d800 tx=0x7f9c5c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 shutdown_connections 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c5806be30 0x7f9c5806e2f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c6c15e320 0x7f9c6c160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c6c0d2290 0x7f9c6c15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 --2- 172.21.15.113:0/3287967002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6c0d0b00 0x7f9c6c153150 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 >> 172.21.15.113:0/3287967002 conn(0x7f9c6c0b5c20 msgr2=0x7f9c6c0b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 shutdown_connections 2024-04-25T00:05:41.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41.020+0000 7f9c71c5d640 1 -- 172.21.15.113:0/3287967002 wait complete. 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:05:41.032 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:05:41.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:41.033 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:05:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:05:41.048 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:05:41.048 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:05:41.048 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:05:41.048 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:05:41.049 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42339sec preferred_lft 42339sec 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:05:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:05:41.051 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:05:41.051 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:05:41.051 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:05:41.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:41.051 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:05:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:05:41.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:41.112 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:05:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:05:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:05:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:05:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:05:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:05:41.169 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:05:41.169 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:41.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:05:41.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:05:41.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:05:41.169 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:41.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:05:41.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:41.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:41.291 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:05:41.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:05:41.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:05:41.306 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:05:41.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:41.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:05:41.364 INFO:tasks.ceph.mds.b.smithi178.stderr:2024-04-25T00:05:41.363+0000 7fc1be10f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:05:41.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:05:41.370 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:05:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:05:41.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:05:41.425 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:05:41.479 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:05:41.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:41.479 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:05:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:05:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:05:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:05:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:05:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:05:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:05:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:05:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:05:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:05:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:05:41.574 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:05:41.574 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:05:41.574 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:05:41.574 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:05:41.576 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:05:41.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:41.576 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:05:41.641 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:05:41.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:client8544 2024-04-25T00:05:41.743 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:05:41.743 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:05:41.758 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:41.759 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:41.759 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T00:05:41.759 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:41.759 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:41.759 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:41.760 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:41.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.868 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097442 s, 108 MB/s 2024-04-25T00:05:41.876 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.876 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.876 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644644 s, 163 MB/s 2024-04-25T00:05:41.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.886 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855615 s, 123 MB/s 2024-04-25T00:05:41.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.894 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.894 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577378 s, 182 MB/s 2024-04-25T00:05:41.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.902 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679736 s, 154 MB/s 2024-04-25T00:05:41.911 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.911 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.911 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726692 s, 144 MB/s 2024-04-25T00:05:41.921 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.921 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.921 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849302 s, 123 MB/s 2024-04-25T00:05:41.930 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.930 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.930 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685605 s, 153 MB/s 2024-04-25T00:05:41.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.938 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062935 s, 167 MB/s 2024-04-25T00:05:41.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.947 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714387 s, 147 MB/s 2024-04-25T00:05:41.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.956 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717753 s, 146 MB/s 2024-04-25T00:05:41.964 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.965 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695698 s, 151 MB/s 2024-04-25T00:05:41.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.973 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066626 s, 157 MB/s 2024-04-25T00:05:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803998 s, 130 MB/s 2024-04-25T00:05:41.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:41.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:41.993 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816446 s, 128 MB/s 2024-04-25T00:05:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726845 s, 144 MB/s 2024-04-25T00:05:42.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.011 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711219 s, 147 MB/s 2024-04-25T00:05:42.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736725 s, 142 MB/s 2024-04-25T00:05:42.028 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.028 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.028 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692203 s, 151 MB/s 2024-04-25T00:05:42.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.037 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711471 s, 147 MB/s 2024-04-25T00:05:42.045 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.045 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.045 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673823 s, 156 MB/s 2024-04-25T00:05:42.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663198 s, 158 MB/s 2024-04-25T00:05:42.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.061 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005495 s, 191 MB/s 2024-04-25T00:05:42.069 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.069 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.069 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681489 s, 154 MB/s 2024-04-25T00:05:42.078 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.078 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.078 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662668 s, 158 MB/s 2024-04-25T00:05:42.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.087 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.087 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712019 s, 147 MB/s 2024-04-25T00:05:42.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.095 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689958 s, 152 MB/s 2024-04-25T00:05:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636042 s, 165 MB/s 2024-04-25T00:05:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.111 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656135 s, 160 MB/s 2024-04-25T00:05:42.119 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.120 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654467 s, 160 MB/s 2024-04-25T00:05:42.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665191 s, 158 MB/s 2024-04-25T00:05:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747991 s, 140 MB/s 2024-04-25T00:05:42.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:42.145 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:42.146 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:42.146 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T00:05:42.146 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:42.146 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:42.146 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:42.146 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:42.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:42.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.204 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645097 s, 163 MB/s 2024-04-25T00:05:42.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.213 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751209 s, 140 MB/s 2024-04-25T00:05:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068234 s, 154 MB/s 2024-04-25T00:05:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657712 s, 159 MB/s 2024-04-25T00:05:42.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.238 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653699 s, 160 MB/s 2024-04-25T00:05:42.246 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.246 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.246 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672759 s, 156 MB/s 2024-04-25T00:05:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648496 s, 162 MB/s 2024-04-25T00:05:42.263 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.263 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.263 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724414 s, 145 MB/s 2024-04-25T00:05:42.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.271 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655333 s, 160 MB/s 2024-04-25T00:05:42.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568154 s, 185 MB/s 2024-04-25T00:05:42.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.287 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.287 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659614 s, 159 MB/s 2024-04-25T00:05:42.295 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.296 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696423 s, 151 MB/s 2024-04-25T00:05:42.304 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.305 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688374 s, 152 MB/s 2024-04-25T00:05:42.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.312 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593145 s, 177 MB/s 2024-04-25T00:05:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063306 s, 166 MB/s 2024-04-25T00:05:42.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.327 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572729 s, 183 MB/s 2024-04-25T00:05:42.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.335 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062468 s, 168 MB/s 2024-04-25T00:05:42.344 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.344 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.344 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722243 s, 145 MB/s 2024-04-25T00:05:42.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.353 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770414 s, 136 MB/s 2024-04-25T00:05:42.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.362 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689772 s, 152 MB/s 2024-04-25T00:05:42.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615515 s, 170 MB/s 2024-04-25T00:05:42.377 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.377 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.377 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572561 s, 183 MB/s 2024-04-25T00:05:42.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.388 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847641 s, 124 MB/s 2024-04-25T00:05:42.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576236 s, 182 MB/s 2024-04-25T00:05:42.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.403 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591048 s, 177 MB/s 2024-04-25T00:05:42.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.411 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641491 s, 163 MB/s 2024-04-25T00:05:42.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.419 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570191 s, 184 MB/s 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.428 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705915 s, 149 MB/s 2024-04-25T00:05:42.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.436 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681708 s, 154 MB/s 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559581 s, 187 MB/s 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744871 s, 141 MB/s 2024-04-25T00:05:42.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.460 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653665 s, 160 MB/s 2024-04-25T00:05:42.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:42.467 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:42.467 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:42.467 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T00:05:42.467 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:42.467 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:42.467 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:42.467 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:42.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:42.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.526 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694506 s, 151 MB/s 2024-04-25T00:05:42.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.534 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065311 s, 161 MB/s 2024-04-25T00:05:42.541 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594157 s, 176 MB/s 2024-04-25T00:05:42.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.551 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658376 s, 159 MB/s 2024-04-25T00:05:42.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.558 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638399 s, 164 MB/s 2024-04-25T00:05:42.567 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.567 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.567 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744753 s, 141 MB/s 2024-04-25T00:05:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651199 s, 161 MB/s 2024-04-25T00:05:42.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.584 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067613 s, 155 MB/s 2024-04-25T00:05:42.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.592 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676541 s, 155 MB/s 2024-04-25T00:05:42.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.600 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649375 s, 161 MB/s 2024-04-25T00:05:42.608 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.609 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.609 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675466 s, 155 MB/s 2024-04-25T00:05:42.616 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.616 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.616 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585258 s, 179 MB/s 2024-04-25T00:05:42.625 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.625 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.625 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666535 s, 157 MB/s 2024-04-25T00:05:42.633 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.633 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.633 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064014 s, 164 MB/s 2024-04-25T00:05:42.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.642 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075725 s, 138 MB/s 2024-04-25T00:05:42.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.650 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666944 s, 157 MB/s 2024-04-25T00:05:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703426 s, 149 MB/s 2024-04-25T00:05:42.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.666 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577746 s, 181 MB/s 2024-04-25T00:05:42.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665231 s, 158 MB/s 2024-04-25T00:05:42.682 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.682 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.682 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561668 s, 187 MB/s 2024-04-25T00:05:42.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.692 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086976 s, 121 MB/s 2024-04-25T00:05:42.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.699 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560375 s, 187 MB/s 2024-04-25T00:05:42.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.707 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594038 s, 177 MB/s 2024-04-25T00:05:42.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.716 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668675 s, 157 MB/s 2024-04-25T00:05:42.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.724 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.724 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066153 s, 159 MB/s 2024-04-25T00:05:42.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553212 s, 190 MB/s 2024-04-25T00:05:42.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.741 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705104 s, 149 MB/s 2024-04-25T00:05:42.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.749 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.749 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728697 s, 144 MB/s 2024-04-25T00:05:42.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700485 s, 150 MB/s 2024-04-25T00:05:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560207 s, 187 MB/s 2024-04-25T00:05:42.773 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.773 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.773 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681221 s, 154 MB/s 2024-04-25T00:05:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626971 s, 167 MB/s 2024-04-25T00:05:42.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:42.788 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:42.788 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:42.788 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T00:05:42.788 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:42.788 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:42.788 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:42.789 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:42.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:42.847 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.847 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.847 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645576 s, 162 MB/s 2024-04-25T00:05:42.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.855 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645767 s, 162 MB/s 2024-04-25T00:05:42.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.863 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666086 s, 157 MB/s 2024-04-25T00:05:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685482 s, 153 MB/s 2024-04-25T00:05:42.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584053 s, 180 MB/s 2024-04-25T00:05:42.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.887 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643162 s, 163 MB/s 2024-04-25T00:05:42.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.896 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730658 s, 144 MB/s 2024-04-25T00:05:42.903 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.903 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.904 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576363 s, 182 MB/s 2024-04-25T00:05:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568971 s, 184 MB/s 2024-04-25T00:05:42.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.919 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603288 s, 174 MB/s 2024-04-25T00:05:42.928 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.928 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.928 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073619 s, 142 MB/s 2024-04-25T00:05:42.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.937 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736866 s, 142 MB/s 2024-04-25T00:05:42.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.945 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658054 s, 159 MB/s 2024-04-25T00:05:42.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.952 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.952 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577909 s, 181 MB/s 2024-04-25T00:05:42.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.961 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006843 s, 153 MB/s 2024-04-25T00:05:42.968 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.968 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.968 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595889 s, 176 MB/s 2024-04-25T00:05:42.977 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.977 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.977 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065675 s, 160 MB/s 2024-04-25T00:05:42.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.986 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748364 s, 140 MB/s 2024-04-25T00:05:42.994 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:42.995 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:42.995 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689568 s, 152 MB/s 2024-04-25T00:05:43.003 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.003 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.003 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652894 s, 161 MB/s 2024-04-25T00:05:43.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.011 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.011 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630512 s, 166 MB/s 2024-04-25T00:05:43.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.019 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650198 s, 161 MB/s 2024-04-25T00:05:43.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.027 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640664 s, 164 MB/s 2024-04-25T00:05:43.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.034 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584316 s, 179 MB/s 2024-04-25T00:05:43.043 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.043 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.043 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738689 s, 142 MB/s 2024-04-25T00:05:43.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.051 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571475 s, 183 MB/s 2024-04-25T00:05:43.058 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.058 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.058 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562433 s, 186 MB/s 2024-04-25T00:05:43.066 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.067 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694199 s, 151 MB/s 2024-04-25T00:05:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675609 s, 155 MB/s 2024-04-25T00:05:43.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.083 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068925 s, 152 MB/s 2024-04-25T00:05:43.093 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.093 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.093 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777642 s, 135 MB/s 2024-04-25T00:05:43.101 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.101 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.101 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680594 s, 154 MB/s 2024-04-25T00:05:43.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:43.109 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:43.109 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:43.109 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T00:05:43.109 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:43.109 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:43.109 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:43.109 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:43.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:43.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.168 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677343 s, 155 MB/s 2024-04-25T00:05:43.175 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.175 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.175 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582954 s, 180 MB/s 2024-04-25T00:05:43.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.184 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746137 s, 141 MB/s 2024-04-25T00:05:43.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.193 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.193 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659923 s, 159 MB/s 2024-04-25T00:05:43.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.201 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.201 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652888 s, 161 MB/s 2024-04-25T00:05:43.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.208 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.208 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580046 s, 181 MB/s 2024-04-25T00:05:43.216 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.216 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.216 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557972 s, 188 MB/s 2024-04-25T00:05:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677809 s, 155 MB/s 2024-04-25T00:05:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574895 s, 182 MB/s 2024-04-25T00:05:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544629 s, 193 MB/s 2024-04-25T00:05:43.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.248 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764775 s, 137 MB/s 2024-04-25T00:05:43.257 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.257 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.257 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696429 s, 151 MB/s 2024-04-25T00:05:43.265 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.265 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.265 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650114 s, 161 MB/s 2024-04-25T00:05:43.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.272 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569854 s, 184 MB/s 2024-04-25T00:05:43.280 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.280 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.281 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652322 s, 161 MB/s 2024-04-25T00:05:43.288 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.288 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.288 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579019 s, 181 MB/s 2024-04-25T00:05:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769147 s, 136 MB/s 2024-04-25T00:05:43.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.306 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066048 s, 159 MB/s 2024-04-25T00:05:43.314 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.314 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.314 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719782 s, 146 MB/s 2024-04-25T00:05:43.322 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.322 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.322 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641774 s, 163 MB/s 2024-04-25T00:05:43.331 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.332 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753404 s, 139 MB/s 2024-04-25T00:05:43.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.340 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671855 s, 156 MB/s 2024-04-25T00:05:43.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.348 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674767 s, 155 MB/s 2024-04-25T00:05:43.355 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.357 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600019 s, 175 MB/s 2024-04-25T00:05:43.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.364 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674505 s, 155 MB/s 2024-04-25T00:05:43.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.372 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664456 s, 158 MB/s 2024-04-25T00:05:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554416 s, 189 MB/s 2024-04-25T00:05:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546969 s, 192 MB/s 2024-04-25T00:05:43.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640046 s, 164 MB/s 2024-04-25T00:05:43.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.404 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655397 s, 160 MB/s 2024-04-25T00:05:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747306 s, 140 MB/s 2024-04-25T00:05:43.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.421 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644364 s, 163 MB/s 2024-04-25T00:05:43.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:43.428 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:43.428 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:43.428 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T00:05:43.428 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:43.428 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:43.428 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:43.428 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:43.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:43.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.488 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722888 s, 145 MB/s 2024-04-25T00:05:43.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.496 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.496 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719299 s, 146 MB/s 2024-04-25T00:05:43.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.505 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739772 s, 142 MB/s 2024-04-25T00:05:43.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.513 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654368 s, 160 MB/s 2024-04-25T00:05:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728285 s, 144 MB/s 2024-04-25T00:05:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708124 s, 148 MB/s 2024-04-25T00:05:43.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.539 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686282 s, 153 MB/s 2024-04-25T00:05:43.546 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.546 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.546 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006024 s, 174 MB/s 2024-04-25T00:05:43.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.555 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687486 s, 153 MB/s 2024-04-25T00:05:43.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.563 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656354 s, 160 MB/s 2024-04-25T00:05:43.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728124 s, 144 MB/s 2024-04-25T00:05:43.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.579 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567057 s, 185 MB/s 2024-04-25T00:05:43.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.587 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.587 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592058 s, 177 MB/s 2024-04-25T00:05:43.595 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.595 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.595 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067339 s, 156 MB/s 2024-04-25T00:05:43.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.603 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.603 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589364 s, 178 MB/s 2024-04-25T00:05:43.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.612 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703427 s, 149 MB/s 2024-04-25T00:05:43.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.620 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648902 s, 162 MB/s 2024-04-25T00:05:43.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682363 s, 154 MB/s 2024-04-25T00:05:43.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555768 s, 189 MB/s 2024-04-25T00:05:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746892 s, 140 MB/s 2024-04-25T00:05:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638867 s, 164 MB/s 2024-04-25T00:05:43.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.661 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606062 s, 173 MB/s 2024-04-25T00:05:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698493 s, 150 MB/s 2024-04-25T00:05:43.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742545 s, 141 MB/s 2024-04-25T00:05:43.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.688 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732314 s, 143 MB/s 2024-04-25T00:05:43.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765601 s, 137 MB/s 2024-04-25T00:05:43.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.704 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565201 s, 186 MB/s 2024-04-25T00:05:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.713 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690096 s, 152 MB/s 2024-04-25T00:05:43.721 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.721 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.721 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066836 s, 157 MB/s 2024-04-25T00:05:43.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077001 s, 136 MB/s 2024-04-25T00:05:43.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.740 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729126 s, 144 MB/s 2024-04-25T00:05:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560013 s, 187 MB/s 2024-04-25T00:05:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:43.754 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:43.754 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:43.754 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T00:05:43.754 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:43.754 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:43.754 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:43.754 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651075 s, 161 MB/s 2024-04-25T00:05:43.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.819 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557308 s, 188 MB/s 2024-04-25T00:05:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.826 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00538159 s, 195 MB/s 2024-04-25T00:05:43.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652577 s, 161 MB/s 2024-04-25T00:05:43.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.844 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757978 s, 138 MB/s 2024-04-25T00:05:43.852 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.852 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.852 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677218 s, 155 MB/s 2024-04-25T00:05:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.862 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747461 s, 140 MB/s 2024-04-25T00:05:43.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722359 s, 145 MB/s 2024-04-25T00:05:43.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752166 s, 139 MB/s 2024-04-25T00:05:43.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.887 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.887 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651464 s, 161 MB/s 2024-04-25T00:05:43.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.895 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599611 s, 175 MB/s 2024-04-25T00:05:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.904 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713747 s, 147 MB/s 2024-04-25T00:05:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.912 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676773 s, 155 MB/s 2024-04-25T00:05:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056447 s, 186 MB/s 2024-04-25T00:05:43.927 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.927 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.927 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598213 s, 175 MB/s 2024-04-25T00:05:43.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.937 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738319 s, 142 MB/s 2024-04-25T00:05:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671149 s, 156 MB/s 2024-04-25T00:05:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679137 s, 154 MB/s 2024-04-25T00:05:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.962 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704248 s, 149 MB/s 2024-04-25T00:05:43.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.969 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556564 s, 188 MB/s 2024-04-25T00:05:43.976 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.976 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.977 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597045 s, 176 MB/s 2024-04-25T00:05:43.985 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.985 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.985 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660319 s, 159 MB/s 2024-04-25T00:05:43.994 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:43.994 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:43.994 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752409 s, 139 MB/s 2024-04-25T00:05:44.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.002 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580625 s, 181 MB/s 2024-04-25T00:05:44.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.009 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596599 s, 176 MB/s 2024-04-25T00:05:44.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.018 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561757 s, 187 MB/s 2024-04-25T00:05:44.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.025 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645018 s, 163 MB/s 2024-04-25T00:05:44.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.034 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708656 s, 148 MB/s 2024-04-25T00:05:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657234 s, 160 MB/s 2024-04-25T00:05:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657101 s, 160 MB/s 2024-04-25T00:05:44.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728376 s, 144 MB/s 2024-04-25T00:05:44.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.067 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688003 s, 152 MB/s 2024-04-25T00:05:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:44.075 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:44.075 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:44.075 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T00:05:44.075 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:44.075 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:44.075 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:44.075 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:44.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.136 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662402 s, 158 MB/s 2024-04-25T00:05:44.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.143 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653299 s, 161 MB/s 2024-04-25T00:05:44.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.152 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073142 s, 143 MB/s 2024-04-25T00:05:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.161 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.161 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718596 s, 146 MB/s 2024-04-25T00:05:44.169 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.169 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.170 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711309 s, 147 MB/s 2024-04-25T00:05:44.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.179 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723655 s, 145 MB/s 2024-04-25T00:05:44.187 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.187 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.187 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657432 s, 159 MB/s 2024-04-25T00:05:44.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.196 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.196 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707759 s, 148 MB/s 2024-04-25T00:05:44.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555501 s, 189 MB/s 2024-04-25T00:05:44.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.211 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656952 s, 160 MB/s 2024-04-25T00:05:44.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.219 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644309 s, 163 MB/s 2024-04-25T00:05:44.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.228 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065579 s, 160 MB/s 2024-04-25T00:05:44.234 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564481 s, 186 MB/s 2024-04-25T00:05:44.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.244 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070628 s, 148 MB/s 2024-04-25T00:05:44.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.252 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697593 s, 150 MB/s 2024-04-25T00:05:44.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.260 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649106 s, 162 MB/s 2024-04-25T00:05:44.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.269 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692026 s, 152 MB/s 2024-04-25T00:05:44.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.276 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561384 s, 187 MB/s 2024-04-25T00:05:44.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.283 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556836 s, 188 MB/s 2024-04-25T00:05:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585749 s, 179 MB/s 2024-04-25T00:05:44.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.300 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737005 s, 142 MB/s 2024-04-25T00:05:44.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.309 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738195 s, 142 MB/s 2024-04-25T00:05:44.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.318 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720506 s, 146 MB/s 2024-04-25T00:05:44.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.326 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699675 s, 150 MB/s 2024-04-25T00:05:44.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.335 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070074 s, 150 MB/s 2024-04-25T00:05:44.343 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.343 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.343 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630486 s, 166 MB/s 2024-04-25T00:05:44.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.351 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654944 s, 160 MB/s 2024-04-25T00:05:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669753 s, 157 MB/s 2024-04-25T00:05:44.368 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.368 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709482 s, 148 MB/s 2024-04-25T00:05:44.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731056 s, 143 MB/s 2024-04-25T00:05:44.385 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.385 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.385 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580893 s, 181 MB/s 2024-04-25T00:05:44.393 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.393 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.393 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648914 s, 162 MB/s 2024-04-25T00:05:44.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:44.400 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:44.400 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:44.400 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T00:05:44.400 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:44.400 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:44.400 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:44.400 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:44.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:44.458 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.458 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.458 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676023 s, 155 MB/s 2024-04-25T00:05:44.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.466 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597053 s, 176 MB/s 2024-04-25T00:05:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670232 s, 156 MB/s 2024-04-25T00:05:44.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.483 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718369 s, 146 MB/s 2024-04-25T00:05:44.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.492 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736564 s, 142 MB/s 2024-04-25T00:05:44.500 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.500 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.500 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642453 s, 163 MB/s 2024-04-25T00:05:44.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.508 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558083 s, 188 MB/s 2024-04-25T00:05:44.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.516 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637888 s, 164 MB/s 2024-04-25T00:05:44.525 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.525 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.525 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738871 s, 142 MB/s 2024-04-25T00:05:44.533 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.533 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.533 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673324 s, 156 MB/s 2024-04-25T00:05:44.541 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671738 s, 156 MB/s 2024-04-25T00:05:44.550 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.550 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.550 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707435 s, 148 MB/s 2024-04-25T00:05:44.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.558 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057993 s, 181 MB/s 2024-04-25T00:05:44.567 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.567 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.567 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724464 s, 145 MB/s 2024-04-25T00:05:44.576 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.576 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.576 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764237 s, 137 MB/s 2024-04-25T00:05:44.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.583 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589376 s, 178 MB/s 2024-04-25T00:05:44.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.592 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651481 s, 161 MB/s 2024-04-25T00:05:44.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.600 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663653 s, 158 MB/s 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726524 s, 144 MB/s 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068273 s, 154 MB/s 2024-04-25T00:05:44.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.627 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757808 s, 138 MB/s 2024-04-25T00:05:44.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065725 s, 160 MB/s 2024-04-25T00:05:44.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.642 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573762 s, 183 MB/s 2024-04-25T00:05:44.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.650 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.650 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577776 s, 181 MB/s 2024-04-25T00:05:44.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.658 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.658 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564705 s, 186 MB/s 2024-04-25T00:05:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698568 s, 150 MB/s 2024-04-25T00:05:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.675 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579802 s, 181 MB/s 2024-04-25T00:05:44.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.684 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.684 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687048 s, 153 MB/s 2024-04-25T00:05:44.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.691 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632801 s, 166 MB/s 2024-04-25T00:05:44.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.699 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582202 s, 180 MB/s 2024-04-25T00:05:44.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.707 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627499 s, 167 MB/s 2024-04-25T00:05:44.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.716 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746559 s, 140 MB/s 2024-04-25T00:05:44.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:44.723 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:44.723 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:44.723 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T00:05:44.723 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:44.723 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:44.723 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:44.723 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:44.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751584 s, 140 MB/s 2024-04-25T00:05:44.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.790 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.790 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652788 s, 161 MB/s 2024-04-25T00:05:44.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.797 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.798 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582474 s, 180 MB/s 2024-04-25T00:05:44.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.805 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582383 s, 180 MB/s 2024-04-25T00:05:44.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.814 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642641 s, 163 MB/s 2024-04-25T00:05:44.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.822 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068279 s, 154 MB/s 2024-04-25T00:05:44.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.830 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676764 s, 155 MB/s 2024-04-25T00:05:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.838 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.839 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680026 s, 154 MB/s 2024-04-25T00:05:44.847 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.847 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.847 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726817 s, 144 MB/s 2024-04-25T00:05:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737255 s, 142 MB/s 2024-04-25T00:05:44.865 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.865 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.865 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689151 s, 152 MB/s 2024-04-25T00:05:44.872 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.872 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.872 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056297 s, 186 MB/s 2024-04-25T00:05:44.881 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.881 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.881 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665339 s, 158 MB/s 2024-04-25T00:05:44.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.889 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726646 s, 144 MB/s 2024-04-25T00:05:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723278 s, 145 MB/s 2024-04-25T00:05:44.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.906 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640844 s, 164 MB/s 2024-04-25T00:05:44.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.915 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.915 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706635 s, 148 MB/s 2024-04-25T00:05:44.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.924 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674065 s, 156 MB/s 2024-04-25T00:05:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657152 s, 160 MB/s 2024-04-25T00:05:44.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.940 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679766 s, 154 MB/s 2024-04-25T00:05:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747112 s, 140 MB/s 2024-04-25T00:05:44.956 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.957 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.957 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571361 s, 184 MB/s 2024-04-25T00:05:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.965 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065151 s, 161 MB/s 2024-04-25T00:05:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.972 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561661 s, 187 MB/s 2024-04-25T00:05:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581153 s, 180 MB/s 2024-04-25T00:05:44.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720047 s, 146 MB/s 2024-04-25T00:05:44.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:44.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:44.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698586 s, 150 MB/s 2024-04-25T00:05:45.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006819 s, 154 MB/s 2024-04-25T00:05:45.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.014 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677342 s, 155 MB/s 2024-04-25T00:05:45.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.023 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680599 s, 154 MB/s 2024-04-25T00:05:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742649 s, 141 MB/s 2024-04-25T00:05:45.040 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.040 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.040 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671322 s, 156 MB/s 2024-04-25T00:05:45.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:45.047 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:45.047 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:45.047 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T00:05:45.047 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:45.047 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:45.047 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:45.047 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:45.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:45.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.105 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654742 s, 160 MB/s 2024-04-25T00:05:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571498 s, 183 MB/s 2024-04-25T00:05:45.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.121 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704233 s, 149 MB/s 2024-04-25T00:05:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590227 s, 178 MB/s 2024-04-25T00:05:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667114 s, 157 MB/s 2024-04-25T00:05:45.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.145 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591346 s, 177 MB/s 2024-04-25T00:05:45.153 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.153 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.153 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709834 s, 148 MB/s 2024-04-25T00:05:45.161 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.162 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659118 s, 159 MB/s 2024-04-25T00:05:45.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.170 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069299 s, 151 MB/s 2024-04-25T00:05:45.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.179 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669165 s, 157 MB/s 2024-04-25T00:05:45.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.187 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.187 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585876 s, 179 MB/s 2024-04-25T00:05:45.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.195 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657682 s, 159 MB/s 2024-04-25T00:05:45.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.204 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740537 s, 142 MB/s 2024-04-25T00:05:45.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.213 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708683 s, 148 MB/s 2024-04-25T00:05:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588266 s, 178 MB/s 2024-04-25T00:05:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566527 s, 185 MB/s 2024-04-25T00:05:45.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055066 s, 190 MB/s 2024-04-25T00:05:45.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.243 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568296 s, 185 MB/s 2024-04-25T00:05:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851906 s, 123 MB/s 2024-04-25T00:05:45.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.261 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.261 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609569 s, 172 MB/s 2024-04-25T00:05:45.269 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709167 s, 148 MB/s 2024-04-25T00:05:45.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656837 s, 160 MB/s 2024-04-25T00:05:45.286 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.286 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.286 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648276 s, 162 MB/s 2024-04-25T00:05:45.293 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.293 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.293 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583911 s, 180 MB/s 2024-04-25T00:05:45.301 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649017 s, 162 MB/s 2024-04-25T00:05:45.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.310 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706049 s, 149 MB/s 2024-04-25T00:05:45.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.318 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.318 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628555 s, 167 MB/s 2024-04-25T00:05:45.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.326 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568211 s, 185 MB/s 2024-04-25T00:05:45.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.335 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727906 s, 144 MB/s 2024-04-25T00:05:45.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577964 s, 181 MB/s 2024-04-25T00:05:45.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.351 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709589 s, 148 MB/s 2024-04-25T00:05:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074829 s, 140 MB/s 2024-04-25T00:05:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:45.367 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:45.367 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:45.367 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T00:05:45.367 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:45.367 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:45.367 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:45.367 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:45.425 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.425 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.425 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675609 s, 155 MB/s 2024-04-25T00:05:45.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.434 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731895 s, 143 MB/s 2024-04-25T00:05:45.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.442 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066662 s, 157 MB/s 2024-04-25T00:05:45.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743479 s, 141 MB/s 2024-04-25T00:05:45.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.459 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645198 s, 163 MB/s 2024-04-25T00:05:45.467 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.468 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.468 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670029 s, 156 MB/s 2024-04-25T00:05:45.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.475 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571539 s, 183 MB/s 2024-04-25T00:05:45.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.483 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651815 s, 161 MB/s 2024-04-25T00:05:45.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.492 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662104 s, 158 MB/s 2024-04-25T00:05:45.500 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.500 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.500 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662696 s, 158 MB/s 2024-04-25T00:05:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.508 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653082 s, 161 MB/s 2024-04-25T00:05:45.515 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.515 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.515 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588485 s, 178 MB/s 2024-04-25T00:05:45.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.523 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565949 s, 185 MB/s 2024-04-25T00:05:45.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.530 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057209 s, 183 MB/s 2024-04-25T00:05:45.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.538 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.538 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676588 s, 155 MB/s 2024-04-25T00:05:45.546 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.546 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.547 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662072 s, 158 MB/s 2024-04-25T00:05:45.554 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.555 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.555 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642689 s, 163 MB/s 2024-04-25T00:05:45.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.563 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706988 s, 148 MB/s 2024-04-25T00:05:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722981 s, 145 MB/s 2024-04-25T00:05:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708926 s, 148 MB/s 2024-04-25T00:05:45.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.589 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656257 s, 160 MB/s 2024-04-25T00:05:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638196 s, 164 MB/s 2024-04-25T00:05:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580302 s, 181 MB/s 2024-04-25T00:05:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672262 s, 156 MB/s 2024-04-25T00:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664232 s, 158 MB/s 2024-04-25T00:05:45.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603316 s, 174 MB/s 2024-04-25T00:05:45.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.638 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672397 s, 156 MB/s 2024-04-25T00:05:45.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.646 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670077 s, 156 MB/s 2024-04-25T00:05:45.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643927 s, 163 MB/s 2024-04-25T00:05:45.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.662 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070645 s, 148 MB/s 2024-04-25T00:05:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065518 s, 160 MB/s 2024-04-25T00:05:45.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.679 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074491 s, 141 MB/s 2024-04-25T00:05:45.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:45.686 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:45.686 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:45.686 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T00:05:45.686 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:45.686 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:45.686 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:45.686 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:45.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:45.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.745 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074289 s, 141 MB/s 2024-04-25T00:05:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681065 s, 154 MB/s 2024-04-25T00:05:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684604 s, 153 MB/s 2024-04-25T00:05:45.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.770 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065743 s, 159 MB/s 2024-04-25T00:05:45.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.779 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746795 s, 140 MB/s 2024-04-25T00:05:45.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.788 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668513 s, 157 MB/s 2024-04-25T00:05:45.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.795 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566146 s, 185 MB/s 2024-04-25T00:05:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721765 s, 145 MB/s 2024-04-25T00:05:45.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.813 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651869 s, 161 MB/s 2024-04-25T00:05:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.819 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572858 s, 183 MB/s 2024-04-25T00:05:45.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.828 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.828 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674062 s, 156 MB/s 2024-04-25T00:05:45.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.836 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725819 s, 144 MB/s 2024-04-25T00:05:45.846 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.846 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.846 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789557 s, 133 MB/s 2024-04-25T00:05:45.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.853 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583242 s, 180 MB/s 2024-04-25T00:05:45.862 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.862 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.862 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664433 s, 158 MB/s 2024-04-25T00:05:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678379 s, 155 MB/s 2024-04-25T00:05:45.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.877 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545474 s, 192 MB/s 2024-04-25T00:05:45.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.886 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668383 s, 157 MB/s 2024-04-25T00:05:45.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.934 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739231 s, 142 MB/s 2024-04-25T00:05:45.935 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.935 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064558 s, 162 MB/s 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068509 s, 153 MB/s 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583294 s, 180 MB/s 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640535 s, 164 MB/s 2024-04-25T00:05:45.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.937 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719673 s, 146 MB/s 2024-04-25T00:05:45.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.945 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734529 s, 143 MB/s 2024-04-25T00:05:45.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.954 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736378 s, 142 MB/s 2024-04-25T00:05:45.962 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.962 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.962 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652047 s, 161 MB/s 2024-04-25T00:05:45.970 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.970 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.970 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662386 s, 158 MB/s 2024-04-25T00:05:45.979 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.979 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.979 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728408 s, 144 MB/s 2024-04-25T00:05:45.987 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.987 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.987 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064988 s, 161 MB/s 2024-04-25T00:05:45.995 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:45.995 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:45.995 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665258 s, 158 MB/s 2024-04-25T00:05:46.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.004 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675676 s, 155 MB/s 2024-04-25T00:05:46.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:46.010 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:46.011 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:46.011 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T00:05:46.011 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:46.011 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:46.011 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:46.011 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:46.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:46.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.073 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606627 s, 173 MB/s 2024-04-25T00:05:46.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.081 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698725 s, 150 MB/s 2024-04-25T00:05:46.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.088 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567589 s, 185 MB/s 2024-04-25T00:05:46.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.098 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764776 s, 137 MB/s 2024-04-25T00:05:46.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.107 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707865 s, 148 MB/s 2024-04-25T00:05:46.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.115 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.115 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677341 s, 155 MB/s 2024-04-25T00:05:46.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613194 s, 171 MB/s 2024-04-25T00:05:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589754 s, 178 MB/s 2024-04-25T00:05:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707038 s, 148 MB/s 2024-04-25T00:05:46.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607109 s, 173 MB/s 2024-04-25T00:05:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571718 s, 183 MB/s 2024-04-25T00:05:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.164 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684367 s, 153 MB/s 2024-04-25T00:05:46.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.172 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617573 s, 170 MB/s 2024-04-25T00:05:46.181 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.181 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.181 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706343 s, 148 MB/s 2024-04-25T00:05:46.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.190 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670411 s, 156 MB/s 2024-04-25T00:05:46.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.197 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629805 s, 166 MB/s 2024-04-25T00:05:46.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.206 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736529 s, 142 MB/s 2024-04-25T00:05:46.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.217 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727627 s, 144 MB/s 2024-04-25T00:05:46.223 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.224 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667471 s, 157 MB/s 2024-04-25T00:05:46.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.232 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651727 s, 161 MB/s 2024-04-25T00:05:46.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.240 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702633 s, 149 MB/s 2024-04-25T00:05:46.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.249 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.249 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654596 s, 160 MB/s 2024-04-25T00:05:46.257 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.257 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.257 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691659 s, 152 MB/s 2024-04-25T00:05:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.265 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674627 s, 155 MB/s 2024-04-25T00:05:46.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.274 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661723 s, 158 MB/s 2024-04-25T00:05:46.282 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.282 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.282 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667034 s, 157 MB/s 2024-04-25T00:05:46.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.289 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557859 s, 188 MB/s 2024-04-25T00:05:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569313 s, 184 MB/s 2024-04-25T00:05:46.304 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.305 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656057 s, 160 MB/s 2024-04-25T00:05:46.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.314 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.314 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069737 s, 150 MB/s 2024-04-25T00:05:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591837 s, 177 MB/s 2024-04-25T00:05:46.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.329 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643428 s, 163 MB/s 2024-04-25T00:05:46.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:46.336 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:46.336 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:46.336 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T00:05:46.336 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:46.336 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:46.336 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:46.337 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:46.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:46.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631656 s, 166 MB/s 2024-04-25T00:05:46.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.403 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645866 s, 162 MB/s 2024-04-25T00:05:46.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.412 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073846 s, 142 MB/s 2024-04-25T00:05:46.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.421 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665516 s, 158 MB/s 2024-04-25T00:05:46.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.428 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644875 s, 163 MB/s 2024-04-25T00:05:46.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.436 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.436 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064292 s, 163 MB/s 2024-04-25T00:05:46.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.444 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.444 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582392 s, 180 MB/s 2024-04-25T00:05:46.452 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.452 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.452 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633551 s, 166 MB/s 2024-04-25T00:05:46.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.461 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733574 s, 143 MB/s 2024-04-25T00:05:46.470 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.470 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.470 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664869 s, 158 MB/s 2024-04-25T00:05:46.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.478 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645717 s, 162 MB/s 2024-04-25T00:05:46.485 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.485 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.485 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576335 s, 182 MB/s 2024-04-25T00:05:46.494 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.494 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.494 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675778 s, 155 MB/s 2024-04-25T00:05:46.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.503 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738464 s, 142 MB/s 2024-04-25T00:05:46.512 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.512 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.512 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741967 s, 141 MB/s 2024-04-25T00:05:46.519 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.519 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.519 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588825 s, 178 MB/s 2024-04-25T00:05:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686369 s, 153 MB/s 2024-04-25T00:05:46.536 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.536 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.536 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658168 s, 159 MB/s 2024-04-25T00:05:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072188 s, 145 MB/s 2024-04-25T00:05:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758275 s, 138 MB/s 2024-04-25T00:05:46.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.563 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.563 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718677 s, 146 MB/s 2024-04-25T00:05:46.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671286 s, 156 MB/s 2024-04-25T00:05:46.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.580 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661274 s, 159 MB/s 2024-04-25T00:05:46.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.589 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688493 s, 152 MB/s 2024-04-25T00:05:46.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675199 s, 155 MB/s 2024-04-25T00:05:46.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679972 s, 154 MB/s 2024-04-25T00:05:46.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.614 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733097 s, 143 MB/s 2024-04-25T00:05:46.623 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.623 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.623 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706198 s, 148 MB/s 2024-04-25T00:05:46.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.632 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716902 s, 146 MB/s 2024-04-25T00:05:46.640 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.641 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.641 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713991 s, 147 MB/s 2024-04-25T00:05:46.648 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.649 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064633 s, 162 MB/s 2024-04-25T00:05:46.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.657 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747601 s, 140 MB/s 2024-04-25T00:05:46.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:46.664 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:46.664 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:46.664 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T00:05:46.664 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:46.664 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:46.664 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:46.664 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:46.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:46.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637132 s, 165 MB/s 2024-04-25T00:05:46.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.730 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693245 s, 151 MB/s 2024-04-25T00:05:46.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.738 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654299 s, 160 MB/s 2024-04-25T00:05:46.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.748 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673272 s, 156 MB/s 2024-04-25T00:05:46.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.755 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606939 s, 173 MB/s 2024-04-25T00:05:46.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.762 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584036 s, 180 MB/s 2024-04-25T00:05:46.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.770 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566882 s, 185 MB/s 2024-04-25T00:05:46.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.777 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057194 s, 183 MB/s 2024-04-25T00:05:46.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.785 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646916 s, 162 MB/s 2024-04-25T00:05:46.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717269 s, 146 MB/s 2024-04-25T00:05:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577149 s, 182 MB/s 2024-04-25T00:05:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.809 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675082 s, 155 MB/s 2024-04-25T00:05:46.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668247 s, 157 MB/s 2024-04-25T00:05:46.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.826 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064888 s, 162 MB/s 2024-04-25T00:05:46.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.834 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625513 s, 168 MB/s 2024-04-25T00:05:46.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.842 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656638 s, 160 MB/s 2024-04-25T00:05:46.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.851 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735871 s, 142 MB/s 2024-04-25T00:05:46.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.859 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704854 s, 149 MB/s 2024-04-25T00:05:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650587 s, 161 MB/s 2024-04-25T00:05:46.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.875 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.875 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059182 s, 177 MB/s 2024-04-25T00:05:46.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.883 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572478 s, 183 MB/s 2024-04-25T00:05:46.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.891 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578764 s, 181 MB/s 2024-04-25T00:05:46.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.900 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735697 s, 143 MB/s 2024-04-25T00:05:46.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.907 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550939 s, 190 MB/s 2024-04-25T00:05:46.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.914 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581542 s, 180 MB/s 2024-04-25T00:05:46.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.923 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667646 s, 157 MB/s 2024-04-25T00:05:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.931 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064285 s, 163 MB/s 2024-04-25T00:05:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.939 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.940 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698549 s, 150 MB/s 2024-04-25T00:05:46.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.948 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640414 s, 164 MB/s 2024-04-25T00:05:46.955 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.955 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.955 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587427 s, 179 MB/s 2024-04-25T00:05:46.962 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581333 s, 180 MB/s 2024-04-25T00:05:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674098 s, 156 MB/s 2024-04-25T00:05:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:46.978 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:46.978 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:46.978 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T00:05:46.978 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:46.978 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:46.978 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:46.978 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:47.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:47.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727751 s, 144 MB/s 2024-04-25T00:05:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652588 s, 161 MB/s 2024-04-25T00:05:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556798 s, 188 MB/s 2024-04-25T00:05:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666333 s, 157 MB/s 2024-04-25T00:05:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.070 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700772 s, 150 MB/s 2024-04-25T00:05:47.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.078 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.078 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065692 s, 160 MB/s 2024-04-25T00:05:47.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.086 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657433 s, 159 MB/s 2024-04-25T00:05:47.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.095 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686328 s, 153 MB/s 2024-04-25T00:05:47.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.103 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719439 s, 146 MB/s 2024-04-25T00:05:47.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.110 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570326 s, 184 MB/s 2024-04-25T00:05:47.119 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.119 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.120 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729615 s, 144 MB/s 2024-04-25T00:05:47.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.128 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705686 s, 149 MB/s 2024-04-25T00:05:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582795 s, 180 MB/s 2024-04-25T00:05:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553256 s, 190 MB/s 2024-04-25T00:05:47.151 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.152 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659721 s, 159 MB/s 2024-04-25T00:05:47.160 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.160 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.160 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652085 s, 161 MB/s 2024-04-25T00:05:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.167 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544313 s, 193 MB/s 2024-04-25T00:05:47.176 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.176 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.177 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074888 s, 140 MB/s 2024-04-25T00:05:47.185 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.185 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.185 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713849 s, 147 MB/s 2024-04-25T00:05:47.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.194 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639528 s, 164 MB/s 2024-04-25T00:05:47.202 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.202 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732186 s, 143 MB/s 2024-04-25T00:05:47.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.211 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647194 s, 162 MB/s 2024-04-25T00:05:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.218 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563251 s, 186 MB/s 2024-04-25T00:05:47.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.227 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071882 s, 146 MB/s 2024-04-25T00:05:47.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639268 s, 164 MB/s 2024-04-25T00:05:47.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.243 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567654 s, 185 MB/s 2024-04-25T00:05:47.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.252 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747674 s, 140 MB/s 2024-04-25T00:05:47.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.262 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692736 s, 151 MB/s 2024-04-25T00:05:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666704 s, 157 MB/s 2024-04-25T00:05:47.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582948 s, 180 MB/s 2024-04-25T00:05:47.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.285 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554811 s, 189 MB/s 2024-04-25T00:05:47.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.294 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071425 s, 147 MB/s 2024-04-25T00:05:47.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:47.301 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:47.301 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:47.301 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T00:05:47.301 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:47.301 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:47.301 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:47.301 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:47.359 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.359 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.359 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588854 s, 178 MB/s 2024-04-25T00:05:47.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.367 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563002 s, 186 MB/s 2024-04-25T00:05:47.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.376 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.376 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726117 s, 144 MB/s 2024-04-25T00:05:47.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.384 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068016 s, 154 MB/s 2024-04-25T00:05:47.393 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.393 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.393 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007313 s, 143 MB/s 2024-04-25T00:05:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.401 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639766 s, 164 MB/s 2024-04-25T00:05:47.408 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.409 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.409 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061266 s, 171 MB/s 2024-04-25T00:05:47.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.416 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591846 s, 177 MB/s 2024-04-25T00:05:47.425 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.425 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.425 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006849 s, 153 MB/s 2024-04-25T00:05:47.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.434 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722125 s, 145 MB/s 2024-04-25T00:05:47.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.442 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664397 s, 158 MB/s 2024-04-25T00:05:47.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.449 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554223 s, 189 MB/s 2024-04-25T00:05:47.458 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.458 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.458 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640869 s, 164 MB/s 2024-04-25T00:05:47.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.466 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671981 s, 156 MB/s 2024-04-25T00:05:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729251 s, 144 MB/s 2024-04-25T00:05:47.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.484 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708339 s, 148 MB/s 2024-04-25T00:05:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589108 s, 178 MB/s 2024-04-25T00:05:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639096 s, 164 MB/s 2024-04-25T00:05:47.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.507 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571439 s, 183 MB/s 2024-04-25T00:05:47.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.514 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552378 s, 190 MB/s 2024-04-25T00:05:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.522 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633351 s, 166 MB/s 2024-04-25T00:05:47.531 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.531 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.531 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647057 s, 162 MB/s 2024-04-25T00:05:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.540 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774046 s, 135 MB/s 2024-04-25T00:05:47.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.549 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714728 s, 147 MB/s 2024-04-25T00:05:47.557 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.557 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.557 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065058 s, 161 MB/s 2024-04-25T00:05:47.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.564 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568901 s, 184 MB/s 2024-04-25T00:05:47.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557161 s, 188 MB/s 2024-04-25T00:05:47.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.581 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701898 s, 149 MB/s 2024-04-25T00:05:47.590 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.590 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.590 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770487 s, 136 MB/s 2024-04-25T00:05:47.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.598 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577259 s, 182 MB/s 2024-04-25T00:05:47.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568463 s, 184 MB/s 2024-04-25T00:05:47.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756171 s, 139 MB/s 2024-04-25T00:05:47.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:47.621 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:47.621 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:47.621 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T00:05:47.621 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:47.621 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:47.622 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:47.622 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:47.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.680 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645475 s, 162 MB/s 2024-04-25T00:05:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.689 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744741 s, 141 MB/s 2024-04-25T00:05:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656391 s, 160 MB/s 2024-04-25T00:05:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644244 s, 163 MB/s 2024-04-25T00:05:47.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.713 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638337 s, 164 MB/s 2024-04-25T00:05:47.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644977 s, 163 MB/s 2024-04-25T00:05:47.729 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.730 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624269 s, 168 MB/s 2024-04-25T00:05:47.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.738 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617996 s, 170 MB/s 2024-04-25T00:05:47.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.745 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560561 s, 187 MB/s 2024-04-25T00:05:47.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.754 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707958 s, 148 MB/s 2024-04-25T00:05:47.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.763 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691891 s, 152 MB/s 2024-04-25T00:05:47.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.770 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588717 s, 178 MB/s 2024-04-25T00:05:47.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.778 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654737 s, 160 MB/s 2024-04-25T00:05:47.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.787 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071031 s, 148 MB/s 2024-04-25T00:05:47.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.795 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575814 s, 182 MB/s 2024-04-25T00:05:47.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.803 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633096 s, 166 MB/s 2024-04-25T00:05:47.811 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.811 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.811 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639046 s, 164 MB/s 2024-04-25T00:05:47.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.819 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680312 s, 154 MB/s 2024-04-25T00:05:47.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.826 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550582 s, 190 MB/s 2024-04-25T00:05:47.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.834 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.834 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588082 s, 178 MB/s 2024-04-25T00:05:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663161 s, 158 MB/s 2024-04-25T00:05:47.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.851 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649906 s, 161 MB/s 2024-04-25T00:05:47.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.859 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568367 s, 184 MB/s 2024-04-25T00:05:47.867 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.867 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.867 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663696 s, 158 MB/s 2024-04-25T00:05:47.874 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.874 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.874 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556359 s, 188 MB/s 2024-04-25T00:05:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693096 s, 151 MB/s 2024-04-25T00:05:47.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.890 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576822 s, 182 MB/s 2024-04-25T00:05:47.897 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.897 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.898 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561742 s, 187 MB/s 2024-04-25T00:05:47.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.907 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740077 s, 142 MB/s 2024-04-25T00:05:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551286 s, 190 MB/s 2024-04-25T00:05:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692664 s, 151 MB/s 2024-04-25T00:05:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642647 s, 163 MB/s 2024-04-25T00:05:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:47.938 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:47.938 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:47.938 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T00:05:47.938 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:47.938 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:47.938 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:47.938 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726731 s, 144 MB/s 2024-04-25T00:05:48.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.004 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591948 s, 177 MB/s 2024-04-25T00:05:48.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.014 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764078 s, 137 MB/s 2024-04-25T00:05:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.021 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584878 s, 179 MB/s 2024-04-25T00:05:48.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.029 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641429 s, 163 MB/s 2024-04-25T00:05:48.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673819 s, 156 MB/s 2024-04-25T00:05:48.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.046 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663497 s, 158 MB/s 2024-04-25T00:05:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635056 s, 165 MB/s 2024-04-25T00:05:48.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.062 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659929 s, 159 MB/s 2024-04-25T00:05:48.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.071 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687867 s, 152 MB/s 2024-04-25T00:05:48.078 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.078 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.079 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584623 s, 179 MB/s 2024-04-25T00:05:48.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.086 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063669 s, 165 MB/s 2024-04-25T00:05:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569099 s, 184 MB/s 2024-04-25T00:05:48.101 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.101 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.101 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563224 s, 186 MB/s 2024-04-25T00:05:48.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.110 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711355 s, 147 MB/s 2024-04-25T00:05:48.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.117 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575146 s, 182 MB/s 2024-04-25T00:05:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715593 s, 147 MB/s 2024-04-25T00:05:48.135 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.135 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.135 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661974 s, 158 MB/s 2024-04-25T00:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726286 s, 144 MB/s 2024-04-25T00:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067373 s, 156 MB/s 2024-04-25T00:05:48.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.159 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553553 s, 189 MB/s 2024-04-25T00:05:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639817 s, 164 MB/s 2024-04-25T00:05:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636713 s, 165 MB/s 2024-04-25T00:05:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650584 s, 161 MB/s 2024-04-25T00:05:48.191 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.192 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633718 s, 165 MB/s 2024-04-25T00:05:48.199 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.199 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.199 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581231 s, 180 MB/s 2024-04-25T00:05:48.207 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.207 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.208 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677953 s, 155 MB/s 2024-04-25T00:05:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597539 s, 175 MB/s 2024-04-25T00:05:48.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.224 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664978 s, 158 MB/s 2024-04-25T00:05:48.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.232 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640917 s, 164 MB/s 2024-04-25T00:05:48.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.239 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567494 s, 185 MB/s 2024-04-25T00:05:48.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.248 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650467 s, 161 MB/s 2024-04-25T00:05:48.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:48.254 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:48.255 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:48.255 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T00:05:48.255 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:48.255 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:48.255 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:48.255 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:48.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:48.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668761 s, 157 MB/s 2024-04-25T00:05:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661536 s, 159 MB/s 2024-04-25T00:05:48.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.330 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713039 s, 147 MB/s 2024-04-25T00:05:48.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.338 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611738 s, 171 MB/s 2024-04-25T00:05:48.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.346 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732984 s, 143 MB/s 2024-04-25T00:05:48.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.356 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720965 s, 145 MB/s 2024-04-25T00:05:48.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.363 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561817 s, 187 MB/s 2024-04-25T00:05:48.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.372 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717621 s, 146 MB/s 2024-04-25T00:05:48.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.381 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776597 s, 135 MB/s 2024-04-25T00:05:48.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.390 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701764 s, 149 MB/s 2024-04-25T00:05:48.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.398 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598273 s, 175 MB/s 2024-04-25T00:05:48.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.406 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646206 s, 162 MB/s 2024-04-25T00:05:48.415 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.415 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.415 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760615 s, 138 MB/s 2024-04-25T00:05:48.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.424 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669859 s, 157 MB/s 2024-04-25T00:05:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744018 s, 141 MB/s 2024-04-25T00:05:48.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.442 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671645 s, 156 MB/s 2024-04-25T00:05:48.450 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.450 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.450 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672923 s, 156 MB/s 2024-04-25T00:05:48.457 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.457 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.457 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546567 s, 192 MB/s 2024-04-25T00:05:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.465 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055686 s, 188 MB/s 2024-04-25T00:05:48.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559773 s, 187 MB/s 2024-04-25T00:05:48.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.479 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567909 s, 185 MB/s 2024-04-25T00:05:48.486 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.486 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.486 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562417 s, 186 MB/s 2024-04-25T00:05:48.493 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.493 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.494 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551735 s, 190 MB/s 2024-04-25T00:05:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595968 s, 176 MB/s 2024-04-25T00:05:48.510 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.510 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.510 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679522 s, 154 MB/s 2024-04-25T00:05:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660573 s, 159 MB/s 2024-04-25T00:05:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599111 s, 175 MB/s 2024-04-25T00:05:48.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.535 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708209 s, 148 MB/s 2024-04-25T00:05:48.544 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.544 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.544 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738416 s, 142 MB/s 2024-04-25T00:05:48.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.552 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654936 s, 160 MB/s 2024-04-25T00:05:48.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.560 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681346 s, 154 MB/s 2024-04-25T00:05:48.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.568 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633047 s, 166 MB/s 2024-04-25T00:05:48.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:48.575 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:48.575 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:48.575 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T00:05:48.575 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:48.575 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:48.575 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:48.575 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:48.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:48.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.632 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593489 s, 177 MB/s 2024-04-25T00:05:48.641 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.641 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.641 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690623 s, 152 MB/s 2024-04-25T00:05:48.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.649 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650821 s, 161 MB/s 2024-04-25T00:05:48.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.657 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668131 s, 157 MB/s 2024-04-25T00:05:48.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.666 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743163 s, 141 MB/s 2024-04-25T00:05:48.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.675 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.675 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648244 s, 162 MB/s 2024-04-25T00:05:48.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.683 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685289 s, 153 MB/s 2024-04-25T00:05:48.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.691 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653267 s, 161 MB/s 2024-04-25T00:05:48.699 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691582 s, 152 MB/s 2024-04-25T00:05:48.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.708 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707165 s, 148 MB/s 2024-04-25T00:05:48.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.716 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657694 s, 159 MB/s 2024-04-25T00:05:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704955 s, 149 MB/s 2024-04-25T00:05:48.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.734 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712468 s, 147 MB/s 2024-04-25T00:05:48.742 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.742 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.742 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673743 s, 156 MB/s 2024-04-25T00:05:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669114 s, 157 MB/s 2024-04-25T00:05:48.759 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.759 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.759 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684161 s, 153 MB/s 2024-04-25T00:05:48.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.767 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654592 s, 160 MB/s 2024-04-25T00:05:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656067 s, 160 MB/s 2024-04-25T00:05:48.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.784 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719874 s, 146 MB/s 2024-04-25T00:05:48.792 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.792 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.793 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695489 s, 151 MB/s 2024-04-25T00:05:48.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.801 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737611 s, 142 MB/s 2024-04-25T00:05:48.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669473 s, 157 MB/s 2024-04-25T00:05:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671945 s, 156 MB/s 2024-04-25T00:05:48.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.827 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655805 s, 160 MB/s 2024-04-25T00:05:48.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.836 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773027 s, 136 MB/s 2024-04-25T00:05:48.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.844 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680528 s, 154 MB/s 2024-04-25T00:05:48.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.853 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073535 s, 143 MB/s 2024-04-25T00:05:48.862 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.862 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.862 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704438 s, 149 MB/s 2024-04-25T00:05:48.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598127 s, 175 MB/s 2024-04-25T00:05:48.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.879 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711043 s, 147 MB/s 2024-04-25T00:05:48.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.886 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562905 s, 186 MB/s 2024-04-25T00:05:48.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.896 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.896 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682627 s, 154 MB/s 2024-04-25T00:05:48.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:48.902 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:48.902 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:48.902 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T00:05:48.902 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:48.902 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:48.902 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:48.902 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:48.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:48.960 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.960 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.960 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645846 s, 162 MB/s 2024-04-25T00:05:48.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.969 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715276 s, 147 MB/s 2024-04-25T00:05:48.977 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.977 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.977 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691423 s, 152 MB/s 2024-04-25T00:05:48.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.986 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649506 s, 161 MB/s 2024-04-25T00:05:48.994 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:48.994 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:48.994 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689277 s, 152 MB/s 2024-04-25T00:05:49.002 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.002 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.002 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576748 s, 182 MB/s 2024-04-25T00:05:49.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.010 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703078 s, 149 MB/s 2024-04-25T00:05:49.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.020 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737194 s, 142 MB/s 2024-04-25T00:05:49.028 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.028 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.028 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646786 s, 162 MB/s 2024-04-25T00:05:49.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.036 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646522 s, 162 MB/s 2024-04-25T00:05:49.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.044 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657762 s, 159 MB/s 2024-04-25T00:05:49.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.053 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730668 s, 144 MB/s 2024-04-25T00:05:49.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.062 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740123 s, 142 MB/s 2024-04-25T00:05:49.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.070 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668012 s, 157 MB/s 2024-04-25T00:05:49.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.079 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071282 s, 147 MB/s 2024-04-25T00:05:49.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.087 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057504 s, 182 MB/s 2024-04-25T00:05:49.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.095 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663301 s, 158 MB/s 2024-04-25T00:05:49.104 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.104 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.104 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718424 s, 146 MB/s 2024-04-25T00:05:49.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.112 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641203 s, 164 MB/s 2024-04-25T00:05:49.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.121 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734306 s, 143 MB/s 2024-04-25T00:05:49.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.128 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576399 s, 182 MB/s 2024-04-25T00:05:49.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.137 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.137 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670215 s, 156 MB/s 2024-04-25T00:05:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.146 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749352 s, 140 MB/s 2024-04-25T00:05:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655971 s, 160 MB/s 2024-04-25T00:05:49.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.163 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697658 s, 150 MB/s 2024-04-25T00:05:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724469 s, 145 MB/s 2024-04-25T00:05:49.181 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.182 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785807 s, 133 MB/s 2024-04-25T00:05:49.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.189 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.190 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634799 s, 165 MB/s 2024-04-25T00:05:49.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.197 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.197 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584619 s, 179 MB/s 2024-04-25T00:05:49.205 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.206 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.206 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639032 s, 164 MB/s 2024-04-25T00:05:49.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.214 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699245 s, 150 MB/s 2024-04-25T00:05:49.222 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.222 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.222 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678693 s, 154 MB/s 2024-04-25T00:05:49.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:49.229 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:49.229 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:49.229 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T00:05:49.229 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:49.229 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:49.230 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:49.230 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.289 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720103 s, 146 MB/s 2024-04-25T00:05:49.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.297 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632653 s, 166 MB/s 2024-04-25T00:05:49.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.306 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704561 s, 149 MB/s 2024-04-25T00:05:49.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581633 s, 180 MB/s 2024-04-25T00:05:49.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645334 s, 162 MB/s 2024-04-25T00:05:49.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.329 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634509 s, 165 MB/s 2024-04-25T00:05:49.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.337 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657636 s, 159 MB/s 2024-04-25T00:05:49.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.345 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604544 s, 173 MB/s 2024-04-25T00:05:49.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.353 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645676 s, 162 MB/s 2024-04-25T00:05:49.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.361 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693208 s, 151 MB/s 2024-04-25T00:05:49.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068013 s, 154 MB/s 2024-04-25T00:05:49.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.379 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.379 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705992 s, 149 MB/s 2024-04-25T00:05:49.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.387 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664329 s, 158 MB/s 2024-04-25T00:05:49.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.395 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.395 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658949 s, 159 MB/s 2024-04-25T00:05:49.405 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.405 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.405 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076937 s, 136 MB/s 2024-04-25T00:05:49.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.413 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065494 s, 160 MB/s 2024-04-25T00:05:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676909 s, 155 MB/s 2024-04-25T00:05:49.430 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.430 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.430 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713635 s, 147 MB/s 2024-04-25T00:05:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706188 s, 148 MB/s 2024-04-25T00:05:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755273 s, 139 MB/s 2024-04-25T00:05:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676688 s, 155 MB/s 2024-04-25T00:05:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657128 s, 160 MB/s 2024-04-25T00:05:49.473 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.473 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.473 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721666 s, 145 MB/s 2024-04-25T00:05:49.482 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.482 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.482 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715113 s, 147 MB/s 2024-04-25T00:05:49.490 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.491 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617944 s, 170 MB/s 2024-04-25T00:05:49.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.498 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674796 s, 155 MB/s 2024-04-25T00:05:49.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.507 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693931 s, 151 MB/s 2024-04-25T00:05:49.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.516 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739153 s, 142 MB/s 2024-04-25T00:05:49.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.524 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596899 s, 176 MB/s 2024-04-25T00:05:49.531 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.531 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.532 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593928 s, 177 MB/s 2024-04-25T00:05:49.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.540 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703884 s, 149 MB/s 2024-04-25T00:05:49.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.548 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568778 s, 184 MB/s 2024-04-25T00:05:49.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:49.554 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:49.554 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:49.554 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T00:05:49.554 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:49.554 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:49.554 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:49.555 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:49.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677494 s, 155 MB/s 2024-04-25T00:05:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585315 s, 179 MB/s 2024-04-25T00:05:49.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.628 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569108 s, 184 MB/s 2024-04-25T00:05:49.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579291 s, 181 MB/s 2024-04-25T00:05:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733858 s, 143 MB/s 2024-04-25T00:05:49.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693852 s, 151 MB/s 2024-04-25T00:05:49.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682062 s, 154 MB/s 2024-04-25T00:05:49.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.669 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562738 s, 186 MB/s 2024-04-25T00:05:49.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719618 s, 146 MB/s 2024-04-25T00:05:49.686 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.686 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.687 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654056 s, 160 MB/s 2024-04-25T00:05:49.695 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.695 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.695 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074629 s, 141 MB/s 2024-04-25T00:05:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.704 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674524 s, 155 MB/s 2024-04-25T00:05:49.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.712 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670187 s, 156 MB/s 2024-04-25T00:05:49.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.719 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.719 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570454 s, 184 MB/s 2024-04-25T00:05:49.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.727 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569739 s, 184 MB/s 2024-04-25T00:05:49.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.734 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.734 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583284 s, 180 MB/s 2024-04-25T00:05:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748583 s, 140 MB/s 2024-04-25T00:05:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.753 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692257 s, 151 MB/s 2024-04-25T00:05:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569802 s, 184 MB/s 2024-04-25T00:05:49.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.769 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672624 s, 156 MB/s 2024-04-25T00:05:49.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.778 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743758 s, 141 MB/s 2024-04-25T00:05:49.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.786 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592341 s, 177 MB/s 2024-04-25T00:05:49.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655175 s, 160 MB/s 2024-04-25T00:05:49.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.801 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.801 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578056 s, 181 MB/s 2024-04-25T00:05:49.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657747 s, 159 MB/s 2024-04-25T00:05:49.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.819 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673005 s, 156 MB/s 2024-04-25T00:05:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.826 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595949 s, 176 MB/s 2024-04-25T00:05:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.836 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737931 s, 142 MB/s 2024-04-25T00:05:49.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.844 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660698 s, 159 MB/s 2024-04-25T00:05:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587878 s, 178 MB/s 2024-04-25T00:05:49.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.859 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644957 s, 163 MB/s 2024-04-25T00:05:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.867 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.867 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599856 s, 175 MB/s 2024-04-25T00:05:49.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:49.873 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:49.873 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:49.873 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T00:05:49.873 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:49.874 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:49.874 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:49.874 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:49.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.931 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583972 s, 180 MB/s 2024-04-25T00:05:49.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.940 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697268 s, 150 MB/s 2024-04-25T00:05:49.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.949 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751305 s, 140 MB/s 2024-04-25T00:05:49.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.958 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770491 s, 136 MB/s 2024-04-25T00:05:49.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.966 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059769 s, 175 MB/s 2024-04-25T00:05:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.975 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647391 s, 162 MB/s 2024-04-25T00:05:49.984 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.984 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.984 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766135 s, 137 MB/s 2024-04-25T00:05:49.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:49.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:49.993 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725874 s, 144 MB/s 2024-04-25T00:05:50.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.001 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596887 s, 176 MB/s 2024-04-25T00:05:50.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.010 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659258 s, 159 MB/s 2024-04-25T00:05:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.018 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647614 s, 162 MB/s 2024-04-25T00:05:50.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.027 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075083 s, 140 MB/s 2024-04-25T00:05:50.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.036 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.036 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731112 s, 143 MB/s 2024-04-25T00:05:50.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.045 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.045 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683215 s, 153 MB/s 2024-04-25T00:05:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.051 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564676 s, 186 MB/s 2024-04-25T00:05:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.060 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693433 s, 151 MB/s 2024-04-25T00:05:50.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.069 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064551 s, 162 MB/s 2024-04-25T00:05:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.077 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731232 s, 143 MB/s 2024-04-25T00:05:50.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.086 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663711 s, 158 MB/s 2024-04-25T00:05:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682802 s, 154 MB/s 2024-04-25T00:05:50.102 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.103 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648026 s, 162 MB/s 2024-04-25T00:05:50.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.110 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575213 s, 182 MB/s 2024-04-25T00:05:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587663 s, 178 MB/s 2024-04-25T00:05:50.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.126 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631447 s, 166 MB/s 2024-04-25T00:05:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608624 s, 172 MB/s 2024-04-25T00:05:50.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.142 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.142 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071915 s, 146 MB/s 2024-04-25T00:05:50.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006673 s, 157 MB/s 2024-04-25T00:05:50.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.158 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644653 s, 163 MB/s 2024-04-25T00:05:50.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.166 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637324 s, 165 MB/s 2024-04-25T00:05:50.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.174 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567879 s, 185 MB/s 2024-04-25T00:05:50.181 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.181 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.181 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598061 s, 175 MB/s 2024-04-25T00:05:50.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.190 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708383 s, 148 MB/s 2024-04-25T00:05:50.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:50.197 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:50.197 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:50.197 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T00:05:50.197 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:50.197 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:50.197 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:50.197 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:50.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:50.255 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.255 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.255 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668057 s, 157 MB/s 2024-04-25T00:05:50.264 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.264 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.264 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748095 s, 140 MB/s 2024-04-25T00:05:50.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.272 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586496 s, 179 MB/s 2024-04-25T00:05:50.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.279 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.279 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565679 s, 185 MB/s 2024-04-25T00:05:50.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.289 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.289 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076458 s, 137 MB/s 2024-04-25T00:05:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676463 s, 155 MB/s 2024-04-25T00:05:50.305 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676573 s, 155 MB/s 2024-04-25T00:05:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057816 s, 181 MB/s 2024-04-25T00:05:50.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.321 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.321 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637927 s, 164 MB/s 2024-04-25T00:05:50.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.329 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.329 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671515 s, 156 MB/s 2024-04-25T00:05:50.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.337 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.337 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653653 s, 160 MB/s 2024-04-25T00:05:50.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.345 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662116 s, 158 MB/s 2024-04-25T00:05:50.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.354 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.354 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663752 s, 158 MB/s 2024-04-25T00:05:50.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.362 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.362 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650301 s, 161 MB/s 2024-04-25T00:05:50.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636262 s, 165 MB/s 2024-04-25T00:05:50.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657006 s, 160 MB/s 2024-04-25T00:05:50.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.386 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.386 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612583 s, 171 MB/s 2024-04-25T00:05:50.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.394 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666702 s, 157 MB/s 2024-04-25T00:05:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686639 s, 153 MB/s 2024-04-25T00:05:50.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.410 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554273 s, 189 MB/s 2024-04-25T00:05:50.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.418 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589316 s, 178 MB/s 2024-04-25T00:05:50.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.426 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644104 s, 163 MB/s 2024-04-25T00:05:50.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.435 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558554 s, 188 MB/s 2024-04-25T00:05:50.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.442 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656141 s, 160 MB/s 2024-04-25T00:05:50.450 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.450 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.450 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649523 s, 161 MB/s 2024-04-25T00:05:50.457 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.457 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.457 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585339 s, 179 MB/s 2024-04-25T00:05:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657324 s, 160 MB/s 2024-04-25T00:05:50.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.474 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645504 s, 162 MB/s 2024-04-25T00:05:50.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.483 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741272 s, 141 MB/s 2024-04-25T00:05:50.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.491 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.491 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593084 s, 177 MB/s 2024-04-25T00:05:50.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.499 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563988 s, 186 MB/s 2024-04-25T00:05:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608699 s, 172 MB/s 2024-04-25T00:05:50.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:50.513 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:50.513 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:50.513 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T00:05:50.514 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:50.514 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:50.514 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:50.514 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:50.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710924 s, 147 MB/s 2024-04-25T00:05:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582356 s, 180 MB/s 2024-04-25T00:05:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727011 s, 144 MB/s 2024-04-25T00:05:50.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.598 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682365 s, 154 MB/s 2024-04-25T00:05:50.606 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.606 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.607 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681588 s, 154 MB/s 2024-04-25T00:05:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685206 s, 153 MB/s 2024-04-25T00:05:50.624 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.624 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.624 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701718 s, 149 MB/s 2024-04-25T00:05:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.632 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587924 s, 178 MB/s 2024-04-25T00:05:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005729 s, 183 MB/s 2024-04-25T00:05:50.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.648 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659521 s, 159 MB/s 2024-04-25T00:05:50.655 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.655 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.655 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632468 s, 166 MB/s 2024-04-25T00:05:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006792 s, 154 MB/s 2024-04-25T00:05:50.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.672 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645512 s, 162 MB/s 2024-04-25T00:05:50.680 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.680 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.680 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659645 s, 159 MB/s 2024-04-25T00:05:50.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.689 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700193 s, 150 MB/s 2024-04-25T00:05:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639119 s, 164 MB/s 2024-04-25T00:05:50.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.705 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670194 s, 156 MB/s 2024-04-25T00:05:50.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.715 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734798 s, 143 MB/s 2024-04-25T00:05:50.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632115 s, 166 MB/s 2024-04-25T00:05:50.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.730 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.730 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658378 s, 159 MB/s 2024-04-25T00:05:50.737 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.738 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549658 s, 191 MB/s 2024-04-25T00:05:50.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.747 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720808 s, 145 MB/s 2024-04-25T00:05:50.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.755 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.755 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662999 s, 158 MB/s 2024-04-25T00:05:50.766 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.766 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.766 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558906 s, 188 MB/s 2024-04-25T00:05:50.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.771 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671784 s, 156 MB/s 2024-04-25T00:05:50.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.779 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639031 s, 164 MB/s 2024-04-25T00:05:50.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.788 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717619 s, 146 MB/s 2024-04-25T00:05:50.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.796 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.796 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064138 s, 163 MB/s 2024-04-25T00:05:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720792 s, 145 MB/s 2024-04-25T00:05:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715431 s, 147 MB/s 2024-04-25T00:05:50.821 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.822 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669749 s, 157 MB/s 2024-04-25T00:05:50.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.830 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653869 s, 160 MB/s 2024-04-25T00:05:50.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:50.836 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:50.837 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:50.837 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T00:05:50.837 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:50.837 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:50.837 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:50.837 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:50.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.895 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660333 s, 159 MB/s 2024-04-25T00:05:50.903 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.904 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.904 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703961 s, 149 MB/s 2024-04-25T00:05:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653816 s, 160 MB/s 2024-04-25T00:05:50.920 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.921 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.921 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725255 s, 145 MB/s 2024-04-25T00:05:50.929 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.929 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.929 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675769 s, 155 MB/s 2024-04-25T00:05:50.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.937 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.937 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674684 s, 155 MB/s 2024-04-25T00:05:50.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.946 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.946 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720832 s, 145 MB/s 2024-04-25T00:05:50.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.954 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.954 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062799 s, 167 MB/s 2024-04-25T00:05:50.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.963 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.963 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719556 s, 146 MB/s 2024-04-25T00:05:50.971 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.971 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.971 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636673 s, 165 MB/s 2024-04-25T00:05:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584603 s, 179 MB/s 2024-04-25T00:05:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699814 s, 150 MB/s 2024-04-25T00:05:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664126 s, 158 MB/s 2024-04-25T00:05:51.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.005 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732748 s, 143 MB/s 2024-04-25T00:05:51.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.014 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665688 s, 158 MB/s 2024-04-25T00:05:51.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.021 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591772 s, 177 MB/s 2024-04-25T00:05:51.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.030 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.030 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659769 s, 159 MB/s 2024-04-25T00:05:51.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.037 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591547 s, 177 MB/s 2024-04-25T00:05:51.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.047 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832991 s, 126 MB/s 2024-04-25T00:05:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645822 s, 162 MB/s 2024-04-25T00:05:51.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.064 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661096 s, 159 MB/s 2024-04-25T00:05:51.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.072 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669183 s, 157 MB/s 2024-04-25T00:05:51.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.082 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727826 s, 144 MB/s 2024-04-25T00:05:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687758 s, 152 MB/s 2024-04-25T00:05:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747665 s, 140 MB/s 2024-04-25T00:05:51.108 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.109 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.109 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764344 s, 137 MB/s 2024-04-25T00:05:51.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712547 s, 147 MB/s 2024-04-25T00:05:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597151 s, 176 MB/s 2024-04-25T00:05:51.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.134 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.134 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750907 s, 140 MB/s 2024-04-25T00:05:51.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.143 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.143 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681084 s, 154 MB/s 2024-04-25T00:05:51.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583086 s, 180 MB/s 2024-04-25T00:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744095 s, 141 MB/s 2024-04-25T00:05:51.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:51.166 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:51.166 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:51.166 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T00:05:51.166 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:51.166 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:51.166 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:51.166 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:51.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:51.230 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.230 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.230 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652206 s, 161 MB/s 2024-04-25T00:05:51.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.239 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.239 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072205 s, 145 MB/s 2024-04-25T00:05:51.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.248 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069736 s, 150 MB/s 2024-04-25T00:05:51.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.258 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749389 s, 140 MB/s 2024-04-25T00:05:51.265 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.265 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.266 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545668 s, 192 MB/s 2024-04-25T00:05:51.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.274 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672465 s, 156 MB/s 2024-04-25T00:05:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659514 s, 159 MB/s 2024-04-25T00:05:51.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.291 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723394 s, 145 MB/s 2024-04-25T00:05:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.299 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659553 s, 159 MB/s 2024-04-25T00:05:51.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.308 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694663 s, 151 MB/s 2024-04-25T00:05:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640086 s, 164 MB/s 2024-04-25T00:05:51.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.324 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655069 s, 160 MB/s 2024-04-25T00:05:51.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.332 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613088 s, 171 MB/s 2024-04-25T00:05:51.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.340 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558354 s, 188 MB/s 2024-04-25T00:05:51.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.348 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652523 s, 161 MB/s 2024-04-25T00:05:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565141 s, 186 MB/s 2024-04-25T00:05:51.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.364 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720321 s, 146 MB/s 2024-04-25T00:05:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575596 s, 182 MB/s 2024-04-25T00:05:51.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.380 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703513 s, 149 MB/s 2024-04-25T00:05:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682262 s, 154 MB/s 2024-04-25T00:05:51.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.397 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.397 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596698 s, 176 MB/s 2024-04-25T00:05:51.405 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.405 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.405 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698242 s, 150 MB/s 2024-04-25T00:05:51.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.413 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605114 s, 173 MB/s 2024-04-25T00:05:51.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.421 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.421 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058175 s, 180 MB/s 2024-04-25T00:05:51.429 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658243 s, 159 MB/s 2024-04-25T00:05:51.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.438 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652083 s, 161 MB/s 2024-04-25T00:05:51.445 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.445 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.445 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584287 s, 179 MB/s 2024-04-25T00:05:51.453 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.453 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.453 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618716 s, 169 MB/s 2024-04-25T00:05:51.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.461 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.461 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553515 s, 189 MB/s 2024-04-25T00:05:51.470 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.470 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.470 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715005 s, 147 MB/s 2024-04-25T00:05:51.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.478 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.478 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704139 s, 149 MB/s 2024-04-25T00:05:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582916 s, 180 MB/s 2024-04-25T00:05:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:51.493 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:51.493 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:51.493 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T00:05:51.493 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:51.493 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:51.493 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:51.493 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:51.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:51.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.551 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640949 s, 164 MB/s 2024-04-25T00:05:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654818 s, 160 MB/s 2024-04-25T00:05:51.567 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.567 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.567 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656635 s, 160 MB/s 2024-04-25T00:05:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561754 s, 187 MB/s 2024-04-25T00:05:51.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.583 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730201 s, 144 MB/s 2024-04-25T00:05:51.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.592 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688822 s, 152 MB/s 2024-04-25T00:05:51.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.600 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694386 s, 151 MB/s 2024-04-25T00:05:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691797 s, 152 MB/s 2024-04-25T00:05:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650756 s, 161 MB/s 2024-04-25T00:05:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671426 s, 156 MB/s 2024-04-25T00:05:51.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.632 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.632 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568888 s, 184 MB/s 2024-04-25T00:05:51.640 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.640 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.640 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056784 s, 185 MB/s 2024-04-25T00:05:51.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.649 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.649 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740539 s, 142 MB/s 2024-04-25T00:05:51.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.656 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565324 s, 185 MB/s 2024-04-25T00:05:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693511 s, 151 MB/s 2024-04-25T00:05:51.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.673 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.673 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621753 s, 169 MB/s 2024-04-25T00:05:51.680 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.680 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.680 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559482 s, 187 MB/s 2024-04-25T00:05:51.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.689 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067586 s, 155 MB/s 2024-04-25T00:05:51.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.698 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718271 s, 146 MB/s 2024-04-25T00:05:51.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.705 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.705 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572565 s, 183 MB/s 2024-04-25T00:05:51.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.714 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710942 s, 147 MB/s 2024-04-25T00:05:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653627 s, 160 MB/s 2024-04-25T00:05:51.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.734 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704992 s, 149 MB/s 2024-04-25T00:05:51.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.739 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584529 s, 179 MB/s 2024-04-25T00:05:51.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.746 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568817 s, 184 MB/s 2024-04-25T00:05:51.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.754 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585184 s, 179 MB/s 2024-04-25T00:05:51.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.762 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659509 s, 159 MB/s 2024-04-25T00:05:51.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.771 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645657 s, 162 MB/s 2024-04-25T00:05:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584375 s, 179 MB/s 2024-04-25T00:05:51.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.786 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590028 s, 178 MB/s 2024-04-25T00:05:51.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673597 s, 156 MB/s 2024-04-25T00:05:51.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.803 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720024 s, 146 MB/s 2024-04-25T00:05:51.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:51.809 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:05:51.809 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:05:51.809 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T00:05:51.809 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:05:51.809 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:05:51.810 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:05:51.810 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:05:51.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:05:51.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.868 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749527 s, 140 MB/s 2024-04-25T00:05:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.877 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704437 s, 149 MB/s 2024-04-25T00:05:51.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.886 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741278 s, 141 MB/s 2024-04-25T00:05:51.894 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.895 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710262 s, 148 MB/s 2024-04-25T00:05:51.903 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.903 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.903 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670359 s, 156 MB/s 2024-04-25T00:05:51.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.910 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589225 s, 178 MB/s 2024-04-25T00:05:51.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.919 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.919 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699202 s, 150 MB/s 2024-04-25T00:05:51.927 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.927 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.927 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678515 s, 155 MB/s 2024-04-25T00:05:51.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.938 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.938 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786463 s, 133 MB/s 2024-04-25T00:05:51.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.945 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.945 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659104 s, 159 MB/s 2024-04-25T00:05:51.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.953 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642626 s, 163 MB/s 2024-04-25T00:05:51.960 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.960 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.960 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573388 s, 183 MB/s 2024-04-25T00:05:51.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.969 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706221 s, 148 MB/s 2024-04-25T00:05:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.978 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064851 s, 162 MB/s 2024-04-25T00:05:51.985 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.985 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.985 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056977 s, 184 MB/s 2024-04-25T00:05:51.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:51.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:51.994 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709237 s, 148 MB/s 2024-04-25T00:05:52.003 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.003 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.003 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077334 s, 136 MB/s 2024-04-25T00:05:52.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.010 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.010 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562738 s, 186 MB/s 2024-04-25T00:05:52.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.019 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.019 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698501 s, 150 MB/s 2024-04-25T00:05:52.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.027 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.027 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646378 s, 162 MB/s 2024-04-25T00:05:52.034 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.035 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.035 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058009 s, 181 MB/s 2024-04-25T00:05:52.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.044 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.044 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728127 s, 144 MB/s 2024-04-25T00:05:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660821 s, 159 MB/s 2024-04-25T00:05:52.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595332 s, 176 MB/s 2024-04-25T00:05:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712688 s, 147 MB/s 2024-04-25T00:05:52.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.075 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563859 s, 186 MB/s 2024-04-25T00:05:52.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.083 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581822 s, 180 MB/s 2024-04-25T00:05:52.091 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.091 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.091 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584052 s, 180 MB/s 2024-04-25T00:05:52.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.100 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720282 s, 146 MB/s 2024-04-25T00:05:52.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.107 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.107 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616884 s, 170 MB/s 2024-04-25T00:05:52.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.117 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.117 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710073 s, 148 MB/s 2024-04-25T00:05:52.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:05:52.125 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:05:52.126 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672161 s, 156 MB/s 2024-04-25T00:05:52.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:05:52.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:52.132 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T00:05:52.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.232+0000 7f43424da640 1 Processor -- start 2024-04-25T00:05:52.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.232+0000 7f43424da640 1 -- start start 2024-04-25T00:05:52.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f43424da640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f43424da640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433c0b2f40 con 0x7f433c0b4b80 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f43424da640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433c0b31e0 con 0x7f433c0b47a0 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f43424da640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f433c0b34c0 con 0x7f433c0cc0b0 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f4340a77640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f433c0b4b80 0x7f433c15bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34684/0 (socket says 172.21.15.113:34684) 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f4340a77640 1 -- 172.21.15.113:0/1281439743 learned_addr learned my addr 172.21.15.113:0/1281439743 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433b7fe640 1 --2- 172.21.15.113:0/1281439743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2456090586 0 0) 0x7f433c0b2f40 con 0x7f433c0b4b80 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4324003520 con 0x7f433c0b4b80 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512299003 0 0) 0x7f433c0b31e0 con 0x7f433c0b47a0 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f433c0b2f40 con 0x7f433c0b47a0 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1854421946 0 0) 0x7f4324003520 con 0x7f433c0b4b80 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433b7fe640 1 -- 172.21.15.113:0/1281439743 >> v1:172.21.15.178:6789/0 conn(0x7f433c0b47a0 legacy=0x7f433c15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433b7fe640 1 -- 172.21.15.113:0/1281439743 >> v1:172.21.15.113:6789/0 conn(0x7f433c0b4b80 legacy=0x7f433c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.233+0000 7f433b7fe640 1 -- 172.21.15.113:0/1281439743 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c0b37a0 con 0x7f433c0cc0b0 2024-04-25T00:05:52.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.234+0000 7f433b7fe640 1 --2- 172.21.15.113:0/1281439743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c154830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43300097d0 tx=0x7f433002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2678e876fe0fbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.234+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330037070 con 0x7f433c0cc0b0 2024-04-25T00:05:52.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.234+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433002ac30 con 0x7f433c0cc0b0 2024-04-25T00:05:52.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.234+0000 7f433affd640 1 -- 172.21.15.113:0/1281439743 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340019f0 con 0x7f433c0cc0b0 2024-04-25T00:05:52.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.235+0000 7f43424da640 1 -- 172.21.15.113:0/1281439743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 msgr2=0x7f433c154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.235+0000 7f43424da640 1 --2- 172.21.15.113:0/1281439743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c154830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43300097d0 tx=0x7f433002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.235+0000 7f43424da640 1 -- 172.21.15.113:0/1281439743 shutdown_connections 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.235+0000 7f43424da640 1 --2- 172.21.15.113:0/1281439743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c154830 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.235+0000 7f43424da640 1 -- 172.21.15.113:0/1281439743 >> 172.21.15.113:0/1281439743 conn(0x7f433c0b1b30 msgr2=0x7f433c0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.235+0000 7f43424da640 1 -- 172.21.15.113:0/1281439743 shutdown_connections 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.235+0000 7f43424da640 1 -- 172.21.15.113:0/1281439743 wait complete. 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.236+0000 7f43424da640 1 Processor -- start 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.236+0000 7f43424da640 1 -- start start 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f43424da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f433c0b47a0 0x7f433c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f43424da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f433c0b4b80 0x7f433c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f43424da640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c1f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f43424da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f433c0b4100 con 0x7f433c0b4b80 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f43424da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f433c0b3f80 con 0x7f433c0b47a0 2024-04-25T00:05:52.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f43424da640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f433c0b4280 con 0x7f433c0cc0b0 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f433c0b4b80 0x7f433c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f433c0b4b80 0x7f433c0727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46366/0 (socket says 172.21.15.113:46366) 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 -- 172.21.15.113:0/654039919 learned_addr learned my addr 172.21.15.113:0/654039919 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f4340a77640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c1f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433bfff640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f433c0b47a0 0x7f433c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 -- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 msgr2=0x7f433c1f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c1f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 -- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f433c0b47a0 msgr2=0x7f433c15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f433c0b47a0 0x7f433c15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.237+0000 7f433b7fe640 1 -- 172.21.15.113:0/654039919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c15e8d0 con 0x7f433c0b4b80 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.238+0000 7f433b7fe640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f433c0b4b80 0x7f433c0727b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f43300097d0 tx=0x7f43300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.238+0000 7f4338ff9640 1 -- 172.21.15.113:0/654039919 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330037050 con 0x7f433c0b4b80 2024-04-25T00:05:52.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.238+0000 7f4338ff9640 1 -- 172.21.15.113:0/654039919 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330038040 con 0x7f433c0b4b80 2024-04-25T00:05:52.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.238+0000 7f4338ff9640 1 -- 172.21.15.113:0/654039919 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330006e10 con 0x7f433c0b4b80 2024-04-25T00:05:52.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.238+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c1f3c40 con 0x7f433c0b4b80 2024-04-25T00:05:52.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.238+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c15db90 con 0x7f433c0b4b80 2024-04-25T00:05:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.239+0000 7f4338ff9640 1 -- 172.21.15.113:0/654039919 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4330045020 con 0x7f433c0b4b80 2024-04-25T00:05:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.239+0000 7f4338ff9640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432806be60 0x7f432806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.240+0000 7f4338ff9640 1 -- 172.21.15.113:0/654039919 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f43300aacf0 con 0x7f433c0b4b80 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.240+0000 7f433bfff640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432806be60 0x7f432806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.240+0000 7f433bfff640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432806be60 0x7f432806e320 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f432c0011e0 tx=0x7f432c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.240+0000 7f43451ed740 1 Processor -- start 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.240+0000 7f43451ed740 1 -- start start 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde8f320 0x55bbcde41cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f43451ed740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde71d00 0x55bbcde46b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f43451ed740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bbcdda9930 con 0x55bbcde71d00 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbcdd51cc0 con 0x55bbcde72440 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bbcdda58f0 con 0x55bbcde8f320 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde71d00 0x55bbcde46b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde71d00 0x55bbcde46b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46376/0 (socket says 172.21.15.113:46376) 2024-04-25T00:05:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 -- 172.21.15.113:0/1511787148 learned_addr learned my addr 172.21.15.113:0/1511787148 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f433b7fe640 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f433bfff640 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde8f320 0x55bbcde41cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 -- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde8f320 msgr2=0x55bbcde41cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde8f320 0x55bbcde41cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 -- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 msgr2=0x55bbcde717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.241+0000 7f4340a77640 1 -- 172.21.15.113:0/1511787148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcdb89ba0 con 0x55bbcde71d00 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f4340a77640 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde71d00 0x55bbcde46b10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f433400f470 tx=0x7f4334001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338cc4daef6db07b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f4321ffb640 1 -- 172.21.15.113:0/1511787148 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340072e0 con 0x55bbcde71d00 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f4321ffb640 1 -- 172.21.15.113:0/1511787148 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4334018070 con 0x55bbcde71d00 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f4321ffb640 1 -- 172.21.15.113:0/1511787148 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334007680 con 0x55bbcde71d00 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 -- 172.21.15.113:0/1511787148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde71d00 msgr2=0x55bbcde46b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde71d00 0x55bbcde46b10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f433400f470 tx=0x7f4334001970 comp rx=0 tx=0).stop 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 -- 172.21.15.113:0/1511787148 shutdown_connections 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde71d00 0x55bbcde46b10 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 --2- 172.21.15.113:0/1511787148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde8f320 0x55bbcde41cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 -- 172.21.15.113:0/1511787148 >> 172.21.15.113:0/1511787148 conn(0x55bbcde43f20 msgr2=0x55bbcde8c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 -- 172.21.15.113:0/1511787148 shutdown_connections 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.242+0000 7f43451ed740 1 -- 172.21.15.113:0/1511787148 wait complete. 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.243+0000 7f43451ed740 1 -- start start 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.243+0000 7f43451ed740 1 Processor -- start 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde8da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde8f320 0x55bbcde84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde6ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bbcde460d0 con 0x55bbcde8f320 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbcdd65330 con 0x55bbcde72440 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bbcde454c0 con 0x55bbcde847d0 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 10 client.0 _ll_register_callbacks cb 0x55bbcdda1950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde8f320 0x55bbcde84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde8f320 0x55bbcde84290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46388/0 (socket says 172.21.15.113:46388) 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 -- 172.21.15.113:0/3266791750 learned_addr learned my addr 172.21.15.113:0/3266791750 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f4340a77640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde6ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433bfff640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde8da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 -- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 msgr2=0x55bbcde6ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde6ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 -- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 msgr2=0x55bbcde8da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde8da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 -- 172.21.15.113:0/3266791750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcda7d490 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433bfff640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde8da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f4340a77640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde6ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f433b7fe640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde8f320 0x55bbcde84290 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4330079070 tx=0x7f43300790e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2779ee84ce4d6b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f431affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f431affd640 1 -- 172.21.15.113:0/3266791750 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330081980 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f431affd640 1 -- 172.21.15.113:0/3266791750 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f4330078530 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.244+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4334018070 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f431affd640 1 -- 172.21.15.113:0/3266791750 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f4330081c10 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 10 client.8556 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55bbcdbea330 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f431affd640 1 -- 172.21.15.113:0/3266791750 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4334018070 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f431affd640 1 -- 172.21.15.113:0/3266791750 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f4330074e30 con 0x55bbcde8f320 2024-04-25T00:05:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 10 client.8556 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 10 client.8556 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 20 client.8556 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcdea0490 0x55bbcdea08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 4 client.8556 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbcddb6e70 con 0x55bbcdea0490 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f433bfff640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcdea0490 0x55bbcdea08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f431affd640 10 client.8556 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.245+0000 7f433bfff640 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcdea0490 0x55bbcdea08a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3765427cd473ec7 server_cookie=82fc5b40ffa6f5b1 in_seq=0 out_seq=0 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.246+0000 7f431affd640 1 -- 172.21.15.113:0/3266791750 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbcddb6e70 con 0x55bbcdea0490 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.246+0000 7f431affd640 10 client.8556 handle_command_reply: tid=0 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.246+0000 7f43451ed740 1 client.8556 shutdown 2024-04-25T00:05:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.246+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde8f320 msgr2=0x55bbcde84290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.246+0000 7f43451ed740 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde8f320 0x55bbcde84290 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4330079070 tx=0x7f43300790e0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.246+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 shutdown_connections 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.246+0000 7f43451ed740 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcdea0490 0x55bbcdea08a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f43451ed740 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde6ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f43451ed740 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde8f320 0x55bbcde84290 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f431affd640 0 client.8556 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f43451ed740 1 --2- 172.21.15.113:0/3266791750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde72440 0x55bbcde8da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 >> 172.21.15.113:0/3266791750 conn(0x55bbcde71d00 msgr2=0x55bbcde8fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 shutdown_connections 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f43451ed740 1 -- 172.21.15.113:0/3266791750 wait complete. 2024-04-25T00:05:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.247+0000 7f43451ed740 20 client.8556 trim_cache size 0 max 16384 2024-04-25T00:05:52.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.264+0000 7f43451ed740 1 Processor -- start 2024-04-25T00:05:52.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.264+0000 7f43451ed740 1 -- start start 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde72440 0x55bbcde419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f43451ed740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f43451ed740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f433c1fc230 con 0x55bbcde847d0 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f433c15d260 con 0x55bbcde8f320 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f433c15c5c0 con 0x55bbcde72440 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46404/0 (socket says 172.21.15.113:46404) 2024-04-25T00:05:52.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 -- 172.21.15.113:0/255540036 learned_addr learned my addr 172.21.15.113:0/255540036 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433bfff640 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde72440 0x55bbcde419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f4340a77640 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 -- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde72440 msgr2=0x55bbcde419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde72440 0x55bbcde419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 -- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 msgr2=0x55bbcde9f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 -- 172.21.15.113:0/255540036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcdea0c10 con 0x55bbcde847d0 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.265+0000 7f433b7fe640 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde847d0 0x55bbcde9eee0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4330077390 tx=0x7f4330090000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e625d0a2e0b534ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43217fa640 1 -- 172.21.15.113:0/255540036 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330032e40 con 0x55bbcde847d0 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43217fa640 1 -- 172.21.15.113:0/255540036 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433009d070 con 0x55bbcde847d0 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43217fa640 1 -- 172.21.15.113:0/255540036 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330081980 con 0x55bbcde847d0 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 -- 172.21.15.113:0/255540036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde847d0 msgr2=0x55bbcde9eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde847d0 0x55bbcde9eee0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4330077390 tx=0x7f4330090000 comp rx=0 tx=0).stop 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 -- 172.21.15.113:0/255540036 shutdown_connections 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 --2- 172.21.15.113:0/255540036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde72440 0x55bbcde419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 -- 172.21.15.113:0/255540036 >> 172.21.15.113:0/255540036 conn(0x55bbcde71d00 msgr2=0x55bbcde90420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 -- 172.21.15.113:0/255540036 shutdown_connections 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.266+0000 7f43451ed740 1 -- 172.21.15.113:0/255540036 wait complete. 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.267+0000 7f43451ed740 1 -- start start 2024-04-25T00:05:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.267+0000 7f43451ed740 1 Processor -- start 2024-04-25T00:05:52.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde72440 0x55bbcdf0acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f433c15cc10 con 0x55bbcde72440 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbcde8b8a0 con 0x55bbcde8f320 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bbcde8aa50 con 0x55bbcde847d0 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 10 client.0 _ll_register_callbacks cb 0x7f4330009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde72440 0x55bbcdf0acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde72440 0x55bbcdf0acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46420/0 (socket says 172.21.15.113:46420) 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 -- 172.21.15.113:0/3606606816 learned_addr learned my addr 172.21.15.113:0/3606606816 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433b7fe640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f4340a77640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 -- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 msgr2=0x55bbcde9eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 -- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 msgr2=0x55bbcde9f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 -- 172.21.15.113:0/3606606816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcde42d60 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433b7fe640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f4340a77640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f433bfff640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde72440 0x55bbcdf0acd0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x55bbcde75140 tx=0x7f432c001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ee7051a468d411 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f431affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f431affd640 1 -- 172.21.15.113:0/3606606816 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c028130 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f431affd640 1 -- 172.21.15.113:0/3606606816 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f432c010040 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433009d070 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f431affd640 1 -- 172.21.15.113:0/3606606816 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f432c038050 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 10 client.8562 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.268+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55bbcda56610 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f431affd640 1 -- 172.21.15.113:0/3606606816 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433009d070 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f431affd640 1 -- 172.21.15.113:0/3606606816 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f432c03b020 con 0x55bbcde72440 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f43451ed740 10 client.8562 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f43451ed740 10 client.8562 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f43451ed740 20 client.8562 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f43451ed740 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcde45ea0 0x55bbcde462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f43451ed740 4 client.8562 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x55bbcde85f50 con 0x55bbcde45ea0 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f433b7fe640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcde45ea0 0x55bbcde462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f433b7fe640 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcde45ea0 0x55bbcde462b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143d5bbfbfb01bea server_cookie=a62c1d2b1bb02712 in_seq=0 out_seq=0 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.269+0000 7f431affd640 10 client.8562 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.271+0000 7f431affd640 1 -- 172.21.15.113:0/3606606816 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bbcde85f50 con 0x55bbcde45ea0 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.271+0000 7f431affd640 10 client.8562 handle_command_reply: tid=0 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 client.8562 shutdown 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde72440 msgr2=0x55bbcdf0acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde72440 0x55bbcdf0acd0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x55bbcde75140 tx=0x7f432c001150 comp rx=0 tx=0).stop 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 shutdown_connections 2024-04-25T00:05:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55bbcde45ea0 0x55bbcde462b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbcde8f320 0x55bbcde9f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbcde847d0 0x55bbcde9eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 --2- 172.21.15.113:0/3606606816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bbcde72440 0x55bbcdf0acd0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f431affd640 0 client.8562 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 >> 172.21.15.113:0/3606606816 conn(0x55bbcde71d00 msgr2=0x55bbcde419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 shutdown_connections 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 1 -- 172.21.15.113:0/3606606816 wait complete. 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43451ed740 20 client.8562 trim_cache size 0 max 16384 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:52.277 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:05:52.278 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "7795de00-10b6-4f64-ac2c-0cf5031c95cf", 2024-04-25T00:05:52.278 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:05:52.278 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432806be60 msgr2=0x7f432806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432806be60 0x7f432806e320 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f432c0011e0 tx=0x7f432c009040 comp rx=0 tx=0).stop 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f433c0b4b80 msgr2=0x7f433c0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f433c0b4b80 0x7f433c0727b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f43300097d0 tx=0x7f43300046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 shutdown_connections 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f432806be60 0x7f432806e320 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f433c0cc0b0 0x7f433c1f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f433c0b4b80 0x7f433c0727b0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 --2- 172.21.15.113:0/654039919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f433c0b47a0 0x7f433c15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.272+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 >> 172.21.15.113:0/654039919 conn(0x7f433c0b1b30 msgr2=0x7f433c1515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.273+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 shutdown_connections 2024-04-25T00:05:52.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.273+0000 7f43424da640 1 -- 172.21.15.113:0/654039919 wait complete. 2024-04-25T00:05:52.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:52.284 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T00:05:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.395+0000 7f6529c1c640 1 Processor -- start 2024-04-25T00:05:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.395+0000 7f6529c1c640 1 -- start start 2024-04-25T00:05:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.395+0000 7f6529c1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65240b3080 0x7f65240b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.395+0000 7f6529c1c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f652415f4f0 con 0x7f652415e2d0 2024-04-25T00:05:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.395+0000 7f6529c1c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f652415f7d0 con 0x7f652415e6b0 2024-04-25T00:05:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.395+0000 7f6529c1c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65240cde00 con 0x7f65240b3080 2024-04-25T00:05:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f65237fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f652415e2d0 0x7f652415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34692/0 (socket says 172.21.15.113:34692) 2024-04-25T00:05:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f65237fe640 1 -- 172.21.15.113:0/753642014 learned_addr learned my addr 172.21.15.113:0/753642014 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f6522ffd640 1 --2- 172.21.15.113:0/753642014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65240b3080 0x7f65240b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2327145924 0 0) 0x7f652415f4f0 con 0x7f652415e2d0 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f650c003520 con 0x7f652415e2d0 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467554845 0 0) 0x7f652415f7d0 con 0x7f652415e6b0 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f652415f4f0 con 0x7f652415e6b0 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f6522ffd640 1 -- 172.21.15.113:0/753642014 >> v1:172.21.15.178:6789/0 conn(0x7f652415e6b0 legacy=0x7f65240ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f6522ffd640 1 -- 172.21.15.113:0/753642014 >> v1:172.21.15.113:6789/0 conn(0x7f652415e2d0 legacy=0x7f652415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3374860780 0 0) 0x7f650c003520 con 0x7f652415e2d0 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.396+0000 7f6522ffd640 1 -- 172.21.15.113:0/753642014 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652415fab0 con 0x7f65240b3080 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.397+0000 7f6522ffd640 1 --2- 172.21.15.113:0/753642014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65240b3080 0x7f65240b4330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65180097d0 tx=0x7f651802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370ece047db5dee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.397+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518037070 con 0x7f65240b3080 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.397+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651802ab90 con 0x7f65240b3080 2024-04-25T00:05:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.397+0000 7f65227fc640 1 -- 172.21.15.113:0/753642014 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514001670 con 0x7f65240b3080 2024-04-25T00:05:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.397+0000 7f6529c1c640 1 -- 172.21.15.113:0/753642014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65240b3080 msgr2=0x7f65240b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.397+0000 7f6529c1c640 1 --2- 172.21.15.113:0/753642014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65240b3080 0x7f65240b4330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65180097d0 tx=0x7f651802a100 comp rx=0 tx=0).stop 2024-04-25T00:05:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.398+0000 7f6529c1c640 1 -- 172.21.15.113:0/753642014 shutdown_connections 2024-04-25T00:05:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.398+0000 7f6529c1c640 1 --2- 172.21.15.113:0/753642014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65240b3080 0x7f65240b4330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.398+0000 7f6529c1c640 1 -- 172.21.15.113:0/753642014 >> 172.21.15.113:0/753642014 conn(0x7f652406b0f0 msgr2=0x7f652406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.398+0000 7f6529c1c640 1 -- 172.21.15.113:0/753642014 shutdown_connections 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.398+0000 7f6529c1c640 1 -- 172.21.15.113:0/753642014 wait complete. 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.399+0000 7f6529c1c640 1 Processor -- start 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.399+0000 7f6529c1c640 1 -- start start 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65240b3080 0x7f6524153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f652415e2d0 0x7f6524145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652415e6b0 0x7f6524155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65240b49f0 con 0x7f652415e6b0 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65240b4870 con 0x7f65240b3080 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65240b4b70 con 0x7f652415e2d0 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652415e6b0 0x7f6524155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f65237fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65240b3080 0x7f6524153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f652415e2d0 0x7f6524145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652415e6b0 0x7f6524155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46434/0 (socket says 172.21.15.113:46434) 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 -- 172.21.15.113:0/3435801908 learned_addr learned my addr 172.21.15.113:0/3435801908 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 -- 172.21.15.113:0/3435801908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f652415e2d0 msgr2=0x7f6524145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f652415e2d0 0x7f6524145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 -- 172.21.15.113:0/3435801908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65240b3080 msgr2=0x7f6524153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65240b3080 0x7f6524153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 -- 172.21.15.113:0/3435801908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652415ba80 con 0x7f652415e6b0 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6523fff640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652415e6b0 0x7f6524155db0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c790 tx=0x7f651c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f650bfff640 1 -- 172.21.15.113:0/3435801908 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c003b70 con 0x7f652415e6b0 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f650bfff640 1 -- 172.21.15.113:0/3435801908 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c003d10 con 0x7f652415e6b0 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f650bfff640 1 -- 172.21.15.113:0/3435801908 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c00aa50 con 0x7f652415e6b0 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652415bd20 con 0x7f652415e6b0 2024-04-25T00:05:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.400+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652415c2e0 con 0x7f652415e6b0 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.402+0000 7f650bfff640 1 -- 172.21.15.113:0/3435801908 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f651c00abf0 con 0x7f652415e6b0 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.402+0000 7f650bfff640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f651006be30 0x7f651006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.402+0000 7f65237fe640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f651006be30 0x7f651006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.402+0000 7f650bfff640 1 -- 172.21.15.113:0/3435801908 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f651c08ace0 con 0x7f652415e6b0 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.403+0000 7f65237fe640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f651006be30 0x7f651006e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f65140017e0 tx=0x7f6514001610 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.403+0000 7f652ca0a740 1 Processor -- start 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.403+0000 7f652ca0a740 1 -- start start 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f60e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f63e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f63e560 0x55951f613370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f652ca0a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55951f577330 con 0x55951f65bb80 2024-04-25T00:05:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55951f51fab0 con 0x55951f63e560 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55951f5732f0 con 0x55951f63eca0 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f60e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f60e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46438/0 (socket says 172.21.15.113:46438) 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 -- 172.21.15.113:0/1403087188 learned_addr learned my addr 172.21.15.113:0/1403087188 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f6523fff640 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f63e560 0x55951f613370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f6522ffd640 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f63e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 -- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 msgr2=0x55951f63e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f63e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 -- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f63e560 msgr2=0x55951f613370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f63e560 0x55951f613370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.404+0000 7f65237fe640 1 -- 172.21.15.113:0/1403087188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951f3557d0 con 0x55951f65bb80 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.405+0000 7f65237fe640 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f60e510 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f651400fbd0 tx=0x7f6514012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba50170cd6f51d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.405+0000 7f65097fa640 1 -- 172.21.15.113:0/1403087188 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514025070 con 0x55951f65bb80 2024-04-25T00:05:52.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.405+0000 7f65097fa640 1 -- 172.21.15.113:0/1403087188 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651401b050 con 0x55951f65bb80 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.405+0000 7f65097fa640 1 -- 172.21.15.113:0/1403087188 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514020410 con 0x55951f65bb80 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.405+0000 7f652ca0a740 1 -- 172.21.15.113:0/1403087188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 msgr2=0x55951f60e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.405+0000 7f652ca0a740 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f60e510 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f651400fbd0 tx=0x7f6514012000 comp rx=0 tx=0).stop 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 -- 172.21.15.113:0/1403087188 shutdown_connections 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f63e560 0x55951f613370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f63e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 --2- 172.21.15.113:0/1403087188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f60e510 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 -- 172.21.15.113:0/1403087188 >> 172.21.15.113:0/1403087188 conn(0x55951f610780 msgr2=0x55951f659020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 -- 172.21.15.113:0/1403087188 shutdown_connections 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 -- 172.21.15.113:0/1403087188 wait complete. 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 -- start start 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.406+0000 7f652ca0a740 1 Processor -- start 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f639ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f650b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f6510b0 0x55951f641020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f652ca0a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55951f612930 con 0x55951f65bb80 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55951f532f40 con 0x55951f6510b0 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55951f611d20 con 0x55951f63eca0 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f652ca0a740 10 client.0 _ll_register_callbacks cb 0x55951f28abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f650b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f65237fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f639ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f650b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46452/0 (socket says 172.21.15.113:46452) 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 -- 172.21.15.113:0/2674159420 learned_addr learned my addr 172.21.15.113:0/2674159420 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6523fff640 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f6510b0 0x55951f641020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 -- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 msgr2=0x55951f639ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f639ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 -- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f6510b0 msgr2=0x55951f641020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f6510b0 0x55951f641020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 -- 172.21.15.113:0/2674159420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951f24b0f0 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f65237fe640 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f639ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f6522ffd640 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f650b70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f6518009720 tx=0x7f6518004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5529a0466b0dc5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.407+0000 7f65027fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f65027fc640 1 -- 172.21.15.113:0/2674159420 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518037040 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f65027fc640 1 -- 172.21.15.113:0/2674159420 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f6518032b80 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f651401b050 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f65027fc640 1 -- 172.21.15.113:0/2674159420 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f65180093c0 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 10 client.8574 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55951f3b5f60 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f65027fc640 1 -- 172.21.15.113:0/2674159420 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651401b050 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f65027fc640 1 -- 172.21.15.113:0/2674159420 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f651803f440 con 0x55951f65bb80 2024-04-25T00:05:52.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 10 client.8574 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 10 client.8574 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 20 client.8574 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f63d700 0x55951f63db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 4 client.8574 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55951f584870 con 0x55951f63d700 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.408+0000 7f65237fe640 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f63d700 0x55951f63db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f65027fc640 10 client.8574 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f65237fe640 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f63d700 0x55951f63db80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6ad3295ef2cca server_cookie=5051c5600fa486bc in_seq=0 out_seq=0 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f65027fc640 1 -- 172.21.15.113:0/2674159420 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55951f584870 con 0x55951f63d700 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f65027fc640 10 client.8574 handle_command_reply: tid=0 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 client.8574 shutdown 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 msgr2=0x55951f650b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f650b70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f6518009720 tx=0x7f6518004730 comp rx=0 tx=0).stop 2024-04-25T00:05:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 shutdown_connections 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f63d700 0x55951f63db80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f6510b0 0x55951f641020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f65bb80 0x55951f650b70 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f65027fc640 0 client.8574 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2674159420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f639ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 >> 172.21.15.113:0/2674159420 conn(0x55951f63e560 msgr2=0x55951f60dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 shutdown_connections 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 1 -- 172.21.15.113:0/2674159420 wait complete. 2024-04-25T00:05:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.409+0000 7f652ca0a740 20 client.8574 trim_cache size 0 max 16384 2024-04-25T00:05:52.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.426+0000 7f652ca0a740 1 Processor -- start 2024-04-25T00:05:52.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.426+0000 7f652ca0a740 1 -- start start 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f67cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f614200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f614740 0x55951f65b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f652ca0a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65241fe560 con 0x55951f614740 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65241fd730 con 0x55951f65bb80 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f652406d0e0 con 0x55951f63eca0 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f614740 0x55951f65b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f614740 0x55951f65b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46466/0 (socket says 172.21.15.113:46466) 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 -- 172.21.15.113:0/2717546875 learned_addr learned my addr 172.21.15.113:0/2717546875 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6522ffd640 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f614200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f65237fe640 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f67cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 -- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 msgr2=0x55951f67cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f67cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 -- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 msgr2=0x55951f614200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f614200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.427+0000 7f6523fff640 1 -- 172.21.15.113:0/2717546875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951f2241b0 con 0x55951f614740 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f6523fff640 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f614740 0x55951f65b000 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f651c0554a0 tx=0x7f651c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2147bb49d393652e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f6508ff9640 1 -- 172.21.15.113:0/2717546875 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c061b70 con 0x55951f614740 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f6508ff9640 1 -- 172.21.15.113:0/2717546875 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c057340 con 0x55951f614740 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f6508ff9640 1 -- 172.21.15.113:0/2717546875 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c082b70 con 0x55951f614740 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 -- 172.21.15.113:0/2717546875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f614740 msgr2=0x55951f65b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f614740 0x55951f65b000 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f651c0554a0 tx=0x7f651c005040 comp rx=0 tx=0).stop 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 -- 172.21.15.113:0/2717546875 shutdown_connections 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f614200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f614740 0x55951f65b000 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 --2- 172.21.15.113:0/2717546875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f63eca0 0x55951f67cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 -- 172.21.15.113:0/2717546875 >> 172.21.15.113:0/2717546875 conn(0x55951f63e560 msgr2=0x55951f6399b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 -- 172.21.15.113:0/2717546875 shutdown_connections 2024-04-25T00:05:52.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.428+0000 7f652ca0a740 1 -- 172.21.15.113:0/2717546875 wait complete. 2024-04-25T00:05:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.429+0000 7f652ca0a740 1 -- start start 2024-04-25T00:05:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.429+0000 7f652ca0a740 1 Processor -- start 2024-04-25T00:05:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f614740 0x55951f6d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f63eca0 0x55951f65add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f65b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55951f51a6c0 con 0x55951f63eca0 2024-04-25T00:05:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f652406d730 con 0x55951f65bb80 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55951f5839b0 con 0x55951f614740 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 10 client.0 _ll_register_callbacks cb 0x7f650c003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f63eca0 0x55951f65add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f63eca0 0x55951f65add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46482/0 (socket says 172.21.15.113:46482) 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 -- 172.21.15.113:0/3716007544 learned_addr learned my addr 172.21.15.113:0/3716007544 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f65237fe640 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f614740 0x55951f6d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6523fff640 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f65b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 -- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f614740 msgr2=0x55951f6d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f614740 0x55951f6d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 -- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 msgr2=0x55951f65b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f65b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 -- 172.21.15.113:0/3716007544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951f63f7b0 con 0x55951f63eca0 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f6522ffd640 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f63eca0 0x55951f65add0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f651802a140 tx=0x7f6518009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fd869bcb2e3b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f65027fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f65027fc640 1 -- 172.21.15.113:0/3716007544 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651805a060 con 0x55951f63eca0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f65027fc640 1 -- 172.21.15.113:0/3716007544 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f6518032b80 con 0x55951f63eca0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.430+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f651c057340 con 0x55951f63eca0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65027fc640 1 -- 172.21.15.113:0/3716007544 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f6518060050 con 0x55951f63eca0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 10 client.8580 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55951f60e900 con 0x55951f63eca0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65027fc640 1 -- 172.21.15.113:0/3716007544 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c057340 con 0x55951f63eca0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65027fc640 1 -- 172.21.15.113:0/3716007544 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f6518063020 con 0x55951f63eca0 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 10 client.8580 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 10 client.8580 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 20 client.8580 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f614200 0x55951f67bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 4 client.8580 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55951f57ebc0 con 0x55951f614200 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65237fe640 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f614200 0x55951f67bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65027fc640 10 client.8580 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65237fe640 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f614200 0x55951f67bca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc80f15678fcea5 server_cookie=5bacbd7764e44fad in_seq=0 out_seq=0 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65027fc640 1 -- 172.21.15.113:0/3716007544 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55951f57ebc0 con 0x55951f614200 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f65027fc640 10 client.8580 handle_command_reply: tid=0 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.431+0000 7f652ca0a740 1 client.8580 shutdown 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f63eca0 msgr2=0x55951f65add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f63eca0 0x55951f65add0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f651802a140 tx=0x7f6518009190 comp rx=0 tx=0).stop 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 shutdown_connections 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55951f614200 0x55951f67bca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55951f65bb80 0x55951f65b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55951f63eca0 0x55951f65add0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f65027fc640 0 client.8580 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 --2- 172.21.15.113:0/3716007544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55951f614740 0x55951f6d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 >> 172.21.15.113:0/3716007544 conn(0x55951f63e560 msgr2=0x55951f650370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 shutdown_connections 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 1 -- 172.21.15.113:0/3716007544 wait complete. 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f652ca0a740 20 client.8580 trim_cache size 0 max 16384 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:05:52.439 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:52.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f651006be30 msgr2=0x7f651006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f651006be30 0x7f651006e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f65140017e0 tx=0x7f6514001610 comp rx=0 tx=0).stop 2024-04-25T00:05:52.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652415e6b0 msgr2=0x7f6524155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652415e6b0 0x7f6524155db0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c790 tx=0x7f651c00cc50 comp rx=0 tx=0).stop 2024-04-25T00:05:52.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 shutdown_connections 2024-04-25T00:05:52.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f651006be30 0x7f651006e2f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652415e6b0 0x7f6524155db0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f652415e2d0 0x7f6524145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 --2- 172.21.15.113:0/3435801908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65240b3080 0x7f6524153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 >> 172.21.15.113:0/3435801908 conn(0x7f652406b0f0 msgr2=0x7f65240729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 shutdown_connections 2024-04-25T00:05:52.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.432+0000 7f6529c1c640 1 -- 172.21.15.113:0/3435801908 wait complete. 2024-04-25T00:05:52.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:52.443 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T00:05:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.541+0000 7f7cf4977640 1 Processor -- start 2024-04-25T00:05:52.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.541+0000 7f7cf4977640 1 -- start start 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cf4977640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7cf015e2d0 0x7f7cf015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cf4977640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf00b3080 0x7f7cf00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cf4977640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7cf015e6b0 0x7f7cf00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cf4977640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf00cde00 con 0x7f7cf015e2d0 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cf4977640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf015f4f0 con 0x7f7cf015e6b0 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cf4977640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf015f7d0 con 0x7f7cf00b3080 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cee59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7cf015e2d0 0x7f7cf015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cee59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7cf015e2d0 0x7f7cf015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46498/0 (socket says 172.21.15.113:46498) 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cee59c640 1 -- 172.21.15.113:0/472745133 learned_addr learned my addr 172.21.15.113:0/472745133 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/472745133 >> v2:172.21.15.178:3300/0 conn(0x7f7cf015e6b0 0x7f7cf00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.542+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/472745133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf00b3080 0x7f7cf00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7cee59c640 1 -- 172.21.15.113:0/472745133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf00b3080 msgr2=0x7f7cf00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7cee59c640 1 --2- 172.21.15.113:0/472745133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf00b3080 0x7f7cf00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7cee59c640 1 -- 172.21.15.113:0/472745133 >> v2:172.21.15.178:3300/0 conn(0x7f7cf015e6b0 msgr2=0x7f7cf00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7cee59c640 1 --2- 172.21.15.113:0/472745133 >> v2:172.21.15.178:3300/0 conn(0x7f7cf015e6b0 0x7f7cf00b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7cee59c640 1 -- 172.21.15.113:0/472745133 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf015fab0 con 0x7f7cf015e2d0 2024-04-25T00:05:52.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7cee59c640 1 --2- 172.21.15.113:0/472745133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e2d0 0x7f7cf015ebf0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7cf006f500 tx=0x7f7ce002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c541e36bbc73f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7ced59a640 1 -- 172.21.15.113:0/472745133 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0036070 con 0x7f7cf015e2d0 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7ced59a640 1 -- 172.21.15.113:0/472745133 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce0031600 con 0x7f7cf015e2d0 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.543+0000 7f7ced59a640 1 -- 172.21.15.113:0/472745133 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00319d0 con 0x7f7cf015e2d0 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 -- 172.21.15.113:0/472745133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e2d0 msgr2=0x7f7cf015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 --2- 172.21.15.113:0/472745133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e2d0 0x7f7cf015ebf0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7cf006f500 tx=0x7f7ce002a610 comp rx=0 tx=0).stop 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 -- 172.21.15.113:0/472745133 shutdown_connections 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 --2- 172.21.15.113:0/472745133 >> v2:172.21.15.178:3300/0 conn(0x7f7cf015e6b0 0x7f7cf00b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 --2- 172.21.15.113:0/472745133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf00b3080 0x7f7cf00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 --2- 172.21.15.113:0/472745133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e2d0 0x7f7cf015ebf0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 -- 172.21.15.113:0/472745133 >> 172.21.15.113:0/472745133 conn(0x7f7cf006b0f0 msgr2=0x7f7cf006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 -- 172.21.15.113:0/472745133 shutdown_connections 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.544+0000 7f7cf4977640 1 -- 172.21.15.113:0/472745133 wait complete. 2024-04-25T00:05:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.545+0000 7f7cf4977640 1 Processor -- start 2024-04-25T00:05:52.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.545+0000 7f7cf4977640 1 -- start start 2024-04-25T00:05:52.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cf4977640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7cf00b3080 0x7f7cf0141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cf4977640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf015e2d0 0x7f7cf01f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cf4977640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e6b0 0x7f7cf006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cf4977640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf015f2b0 con 0x7f7cf015e6b0 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cf4977640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf015f130 con 0x7f7cf00b3080 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cf4977640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf0071040 con 0x7f7cf015e2d0 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e6b0 0x7f7cf006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e6b0 0x7f7cf006cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46514/0 (socket says 172.21.15.113:46514) 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 -- 172.21.15.113:0/3728355744 learned_addr learned my addr 172.21.15.113:0/3728355744 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf015e2d0 0x7f7cf01f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7cee59c640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7cf00b3080 0x7f7cf0141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 -- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf015e2d0 msgr2=0x7f7cf01f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf015e2d0 0x7f7cf01f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 -- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7cf00b3080 msgr2=0x7f7cf0141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7cf00b3080 0x7f7cf0141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.546+0000 7f7ceed9d640 1 -- 172.21.15.113:0/3728355744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf01f2840 con 0x7f7cf015e6b0 2024-04-25T00:05:52.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.547+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e6b0 0x7f7cf006cea0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e7f0 tx=0x7f7ce800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.547+0000 7f7cdeffd640 1 -- 172.21.15.113:0/3728355744 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8017070 con 0x7f7cf015e6b0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.547+0000 7f7cdeffd640 1 -- 172.21.15.113:0/3728355744 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce800ee40 con 0x7f7cf015e6b0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.547+0000 7f7cdeffd640 1 -- 172.21.15.113:0/3728355744 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800cb20 con 0x7f7cf015e6b0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.547+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf01f2b20 con 0x7f7cf015e6b0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.547+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf0151280 con 0x7f7cf015e6b0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.548+0000 7f7cdeffd640 1 -- 172.21.15.113:0/3728355744 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ce801a070 con 0x7f7cf015e6b0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.549+0000 7f7cdeffd640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cd006be60 0x7f7cd006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.549+0000 7f7cdeffd640 1 -- 172.21.15.113:0/3728355744 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce808aed0 con 0x7f7cf015e6b0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.549+0000 7f7cee59c640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cd006be60 0x7f7cd006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.549+0000 7f7cee59c640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cd006be60 0x7f7cd006e320 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0033000 tx=0x7f7ce0009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 Processor -- start 2024-04-25T00:05:52.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 -- start start 2024-04-25T00:05:52.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef30b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef328600 0x55cfef328b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef2e0a40 0x55cfef326cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cfef2446b0 con 0x55cfef328600 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cfef1ec880 con 0x55cfef30ad60 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.550+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cfef240670 con 0x55cfef2e0a40 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef328600 0x55cfef328b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef328600 0x55cfef328b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46528/0 (socket says 172.21.15.113:46528) 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 -- 172.21.15.113:0/237238611 learned_addr learned my addr 172.21.15.113:0/237238611 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cee59c640 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef30b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef2e0a40 0x55cfef326cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 -- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef2e0a40 msgr2=0x55cfef326cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef2e0a40 0x55cfef326cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 -- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 msgr2=0x55cfef30b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef30b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 -- 172.21.15.113:0/237238611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfef0227d0 con 0x55cfef328600 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef328600 0x55cfef328b50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009b90 tx=0x7f7ce4011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26244f0a9af1e2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7ccffff640 1 -- 172.21.15.113:0/237238611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4019070 con 0x55cfef328600 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7ccffff640 1 -- 172.21.15.113:0/237238611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce4004030 con 0x55cfef328600 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.551+0000 7f7ccffff640 1 -- 172.21.15.113:0/237238611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4004400 con 0x55cfef328600 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 -- 172.21.15.113:0/237238611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef328600 msgr2=0x55cfef328b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef328600 0x55cfef328b50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009b90 tx=0x7f7ce4011690 comp rx=0 tx=0).stop 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 -- 172.21.15.113:0/237238611 shutdown_connections 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef2e0a40 0x55cfef326cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef328600 0x55cfef328b50 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 --2- 172.21.15.113:0/237238611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef30b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 -- 172.21.15.113:0/237238611 >> 172.21.15.113:0/237238611 conn(0x55cfef2dd600 msgr2=0x55cfef325b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 -- 172.21.15.113:0/237238611 shutdown_connections 2024-04-25T00:05:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 -- 172.21.15.113:0/237238611 wait complete. 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.552+0000 7f7cf7765740 1 -- start start 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 1 Processor -- start 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef3075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef3362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef31e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cfef2e17e0 con 0x55cfef30e960 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cfef1fff50 con 0x55cfef30ad60 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cfef2db530 con 0x55cfef328600 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.553+0000 7f7cf7765740 10 client.0 _ll_register_callbacks cb 0x55cfef23b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef31e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef31e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46532/0 (socket says 172.21.15.113:46532) 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 -- 172.21.15.113:0/743355404 learned_addr learned my addr 172.21.15.113:0/743355404 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7cee59c640 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef3075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef3362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 -- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 msgr2=0x55cfef3362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef3362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 -- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 msgr2=0x55cfef3075b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef3075b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 -- 172.21.15.113:0/743355404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfeef583f0 con 0x55cfef30e960 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef31e150 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8061810 tx=0x7f7ce8059350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4493930f583445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/743355404 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8021040 con 0x55cfef30e960 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/743355404 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce8058660 con 0x55cfef30e960 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce4004030 con 0x55cfef30e960 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/743355404 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7ce80711c0 con 0x55cfef30e960 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7cf7765740 10 client.8592 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55cfeef180f0 con 0x55cfef30e960 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/743355404 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce4004030 con 0x55cfef30e960 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.554+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/743355404 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f7ce8054f60 con 0x55cfef30e960 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cf7765740 10 client.8592 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cf7765740 10 client.8592 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cf7765740 20 client.8592 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cf7765740 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef31e690 0x55cfef2da730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cf7765740 4 client.8592 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfef251bf0 con 0x55cfef31e690 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cee59c640 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef31e690 0x55cfef2da730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7cee59c640 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef31e690 0x55cfef2da730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d42f9359b101d0 server_cookie=442f269fd6696aa in_seq=0 out_seq=0 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.555+0000 7f7ccd7fa640 10 client.8592 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/743355404 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfef251bf0 con 0x55cfef31e690 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7ccd7fa640 10 client.8592 handle_command_reply: tid=0 2024-04-25T00:05:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 client.8592 shutdown 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 msgr2=0x55cfef31e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef31e150 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8061810 tx=0x7f7ce8059350 comp rx=0 tx=0).stop 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 shutdown_connections 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef31e690 0x55cfef2da730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef31e150 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef3362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7ccd7fa640 0 client.8592 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 --2- 172.21.15.113:0/743355404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef3075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 >> 172.21.15.113:0/743355404 conn(0x55cfef2e0a40 msgr2=0x55cfef3290f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 shutdown_connections 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 1 -- 172.21.15.113:0/743355404 wait complete. 2024-04-25T00:05:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.556+0000 7f7cf7765740 20 client.8592 trim_cache size 0 max 16384 2024-04-25T00:05:52.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.573+0000 7f7cf7765740 1 Processor -- start 2024-04-25T00:05:52.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.573+0000 7f7cf7765740 1 -- start start 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef2da920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef326880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef326dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cf7765740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf01fe640 con 0x55cfef30e960 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf01fd640 con 0x55cfef30ad60 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf01520b0 con 0x55cfef328600 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef326880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef326880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46544/0 (socket says 172.21.15.113:46544) 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 -- 172.21.15.113:0/1734916696 learned_addr learned my addr 172.21.15.113:0/1734916696 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef326dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cee59c640 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef2da920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 -- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 msgr2=0x55cfef326dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef326dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 -- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 msgr2=0x55cfef2da920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef2da920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 -- 172.21.15.113:0/1734916696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfef2dfbe0 con 0x55cfef30e960 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef326dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.574+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef326880 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40116d0 tx=0x7f7ce4008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d728a0bffc88fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7ccf7fe640 1 -- 172.21.15.113:0/1734916696 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4019040 con 0x55cfef30e960 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7ccf7fe640 1 -- 172.21.15.113:0/1734916696 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce4004100 con 0x55cfef30e960 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7ccf7fe640 1 -- 172.21.15.113:0/1734916696 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40044d0 con 0x55cfef30e960 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 -- 172.21.15.113:0/1734916696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 msgr2=0x55cfef326880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef326880 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40116d0 tx=0x7f7ce4008070 comp rx=0 tx=0).stop 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 -- 172.21.15.113:0/1734916696 shutdown_connections 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef328600 0x55cfef326dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30e960 0x55cfef326880 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1734916696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef30ad60 0x55cfef2da920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 -- 172.21.15.113:0/1734916696 >> 172.21.15.113:0/1734916696 conn(0x55cfef2e0a40 msgr2=0x55cfef2e0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 -- 172.21.15.113:0/1734916696 shutdown_connections 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.575+0000 7f7cf7765740 1 -- 172.21.15.113:0/1734916696 wait complete. 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.576+0000 7f7cf7765740 1 -- start start 2024-04-25T00:05:52.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.576+0000 7f7cf7765740 1 Processor -- start 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30ad60 0x55cfef31f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef30e960 0x55cfef39f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef328600 0x55cfef324ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0152700 con 0x55cfef30ad60 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cfef2d72a0 con 0x55cfef328600 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cfef323e20 con 0x55cfef30e960 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 10 client.0 _ll_register_callbacks cb 0x7f7ce4009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30ad60 0x55cfef31f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30ad60 0x55cfef31f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46558/0 (socket says 172.21.15.113:46558) 2024-04-25T00:05:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 -- 172.21.15.113:0/1708981557 learned_addr learned my addr 172.21.15.113:0/1708981557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef30e960 0x55cfef39f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7ceed9d640 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef328600 0x55cfef324ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 -- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef30e960 msgr2=0x55cfef39f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef30e960 0x55cfef39f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 -- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef328600 msgr2=0x55cfef324ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef328600 0x55cfef324ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 -- 172.21.15.113:0/1708981557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfef30a030 con 0x55cfef30ad60 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cee59c640 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30ad60 0x55cfef31f060 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0034040 tx=0x7f7ce0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361c5c3627630469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/1708981557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0053070 con 0x55cfef30ad60 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/1708981557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce003c9d0 con 0x55cfef30ad60 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce4004100 con 0x55cfef30ad60 2024-04-25T00:05:52.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/1708981557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7ce005d050 con 0x55cfef30ad60 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 10 client.8598 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.577+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55cfef326880 con 0x55cfef30ad60 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/1708981557 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce4004100 con 0x55cfef30ad60 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/1708981557 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f7ce0060020 con 0x55cfef30ad60 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cf7765740 10 client.8598 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cf7765740 10 client.8598 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cf7765740 20 client.8598 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef324ff0 0x55cfef30dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cf7765740 4 client.8598 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfef24bf40 con 0x55cfef324ff0 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef324ff0 0x55cfef30dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7ccd7fa640 10 client.8598 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cedd9b640 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef324ff0 0x55cfef30dc40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268b4f1bedc0dcee server_cookie=b7f0f414f89016f0 in_seq=0 out_seq=0 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7ccd7fa640 1 -- 172.21.15.113:0/1708981557 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+276 (crc 0 0 0) 0x55cfef24bf40 con 0x55cfef324ff0 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7ccd7fa640 10 client.8598 handle_command_reply: tid=0 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.578+0000 7f7cf7765740 1 client.8598 shutdown 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30ad60 msgr2=0x55cfef31f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30ad60 0x55cfef31f060 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0034040 tx=0x7f7ce0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 shutdown_connections 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55cfef324ff0 0x55cfef30dc40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cfef328600 0x55cfef324ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cfef30e960 0x55cfef39f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 --2- 172.21.15.113:0/1708981557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfef30ad60 0x55cfef31f060 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7ccd7fa640 0 client.8598 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 >> 172.21.15.113:0/1708981557 conn(0x55cfef2e0a40 msgr2=0x55cfef31de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 shutdown_connections 2024-04-25T00:05:52.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 1 -- 172.21.15.113:0/1708981557 wait complete. 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf7765740 20 client.8598 trim_cache size 0 max 16384 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: "7795de00-10b6-4f64-ac2c-0cf5031c95cf": { 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_control_test_path", 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "7795de00-10b6-4f64-ac2c-0cf5031c95cf", 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive" 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:52.586 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:05:52.587 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:52.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cd006be60 msgr2=0x7f7cd006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cd006be60 0x7f7cd006e320 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0033000 tx=0x7f7ce0009000 comp rx=0 tx=0).stop 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e6b0 msgr2=0x7f7cf006cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e6b0 0x7f7cf006cea0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e7f0 tx=0x7f7ce800ebf0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 shutdown_connections 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cd006be60 0x7f7cd006e320 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf015e6b0 0x7f7cf006cea0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf015e2d0 0x7f7cf01f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 --2- 172.21.15.113:0/3728355744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7cf00b3080 0x7f7cf0141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 >> 172.21.15.113:0/3728355744 conn(0x7f7cf006b0f0 msgr2=0x7f7cf00b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 shutdown_connections 2024-04-25T00:05:52.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.579+0000 7f7cf4977640 1 -- 172.21.15.113:0/3728355744 wait complete. 2024-04-25T00:05:52.590 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:52.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.686+0000 7f2ed1acd640 1 Processor -- start 2024-04-25T00:05:52.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.686+0000 7f2ed1acd640 1 -- start start 2024-04-25T00:05:52.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ed1acd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ecc162710 0x7f2ecc06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ed1acd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2ecc06b630 0x7f2ecc1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ed1acd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ecc162bc0 con 0x7f2ecc158910 2024-04-25T00:05:52.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ed1acd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2ecc06ba80 con 0x7f2ecc162710 2024-04-25T00:05:52.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ed1acd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2ecc15f1d0 con 0x7f2ecc06b630 2024-04-25T00:05:52.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ec3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ecc162710 0x7f2ecc06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ecb7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2ecc158910 0x7f2ecc15e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34702/0 (socket says 172.21.15.113:34702) 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ecb7fe640 1 -- 172.21.15.113:0/1395865142 learned_addr learned my addr 172.21.15.113:0/1395865142 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ecbfff640 1 --2- 172.21.15.113:0/1395865142 >> v2:172.21.15.178:3300/0 conn(0x7f2ecc06b630 0x7f2ecc1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ec3fff640 1 -- 172.21.15.113:0/1395865142 >> v2:172.21.15.178:3300/0 conn(0x7f2ecc06b630 msgr2=0x7f2ecc1438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ec3fff640 1 --2- 172.21.15.113:0/1395865142 >> v2:172.21.15.178:3300/0 conn(0x7f2ecc06b630 0x7f2ecc1438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ec3fff640 1 -- 172.21.15.113:0/1395865142 >> v1:172.21.15.113:6789/0 conn(0x7f2ecc158910 legacy=0x7f2ecc15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.687+0000 7f2ec3fff640 1 -- 172.21.15.113:0/1395865142 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc15f4b0 con 0x7f2ecc162710 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ec3fff640 1 --2- 172.21.15.113:0/1395865142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc162710 0x7f2ecc06b0f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0b01e0 tx=0x7f2eb402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5863c8189d309b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ecaffd640 1 -- 172.21.15.113:0/1395865142 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4037070 con 0x7f2ecc162710 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ecaffd640 1 -- 172.21.15.113:0/1395865142 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb4032440 con 0x7f2ecc162710 2024-04-25T00:05:52.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ecaffd640 1 -- 172.21.15.113:0/1395865142 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4032810 con 0x7f2ecc162710 2024-04-25T00:05:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 -- 172.21.15.113:0/1395865142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc162710 msgr2=0x7f2ecc06b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/1395865142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc162710 0x7f2ecc06b0f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0b01e0 tx=0x7f2eb402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 -- 172.21.15.113:0/1395865142 shutdown_connections 2024-04-25T00:05:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/1395865142 >> v2:172.21.15.178:3300/0 conn(0x7f2ecc06b630 0x7f2ecc1438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/1395865142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc162710 0x7f2ecc06b0f0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 -- 172.21.15.113:0/1395865142 >> 172.21.15.113:0/1395865142 conn(0x7f2ecc0b22c0 msgr2=0x7f2ecc0b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 -- 172.21.15.113:0/1395865142 shutdown_connections 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.688+0000 7f2ed1acd640 1 -- 172.21.15.113:0/1395865142 wait complete. 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 Processor -- start 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 -- start start 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ecc06b630 0x7f2ecc1f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc158910 0x7f2ecc1f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ecc162710 0x7f2ecc0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc15eeb0 con 0x7f2ecc158910 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1621f0 con 0x7f2ecc162710 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ed1acd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2ecc0b3710 con 0x7f2ecc06b630 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ec3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc158910 0x7f2ecc1f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ec3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc158910 0x7f2ecc1f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46576/0 (socket says 172.21.15.113:46576) 2024-04-25T00:05:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.690+0000 7f2ec3fff640 1 -- 172.21.15.113:0/658594317 learned_addr learned my addr 172.21.15.113:0/658594317 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ecbfff640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ecc162710 0x7f2ecc0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec3fff640 1 -- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ecc06b630 msgr2=0x7f2ecc1f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ecb7fe640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ecc06b630 0x7f2ecc1f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec3fff640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ecc06b630 0x7f2ecc1f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec3fff640 1 -- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ecc162710 msgr2=0x7f2ecc0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec3fff640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ecc162710 0x7f2ecc0d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec3fff640 1 -- 172.21.15.113:0/658594317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc1f4e20 con 0x7f2ecc158910 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ecbfff640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ecc162710 0x7f2ecc0d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ecb7fe640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ecc06b630 0x7f2ecc1f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec3fff640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc158910 0x7f2ecc1f48e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009890 tx=0x7f2eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec8ff9640 1 -- 172.21.15.113:0/658594317 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4037040 con 0x7f2ecc158910 2024-04-25T00:05:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec8ff9640 1 -- 172.21.15.113:0/658594317 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb4042070 con 0x7f2ecc158910 2024-04-25T00:05:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ecc0ade30 con 0x7f2ecc158910 2024-04-25T00:05:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ec8ff9640 1 -- 172.21.15.113:0/658594317 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4030bd0 con 0x7f2ecc158910 2024-04-25T00:05:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.691+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ecc0ae3f0 con 0x7f2ecc158910 2024-04-25T00:05:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.692+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ecc0691a0 con 0x7f2ecc158910 2024-04-25T00:05:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.692+0000 7f2ec8ff9640 1 -- 172.21.15.113:0/658594317 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2eb4047460 con 0x7f2ecc158910 2024-04-25T00:05:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.693+0000 7f2ec8ff9640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2eac06be60 0x7f2eac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.693+0000 7f2ec8ff9640 1 -- 172.21.15.113:0/658594317 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f2eb40ab9e0 con 0x7f2ecc158910 2024-04-25T00:05:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.693+0000 7f2ecb7fe640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2eac06be60 0x7f2eac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.693+0000 7f2ecb7fe640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2eac06be60 0x7f2eac06e320 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0afd20 tx=0x7f2ebc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.697+0000 7f2ec8ff9640 1 -- 172.21.15.113:0/658594317 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ecc1f4e20 con 0x7f2ecc158910 2024-04-25T00:05:52.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.907+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ecc074c50 con 0x7f2ecc158910 2024-04-25T00:05:52.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.907+0000 7f2ec8ff9640 1 -- 172.21.15.113:0/658594317 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f2eb4032e20 con 0x7f2ecc158910 2024-04-25T00:05:52.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:52.908 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:05:52.908 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2eac06be60 msgr2=0x7f2eac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2eac06be60 0x7f2eac06e320 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0afd20 tx=0x7f2ebc009040 comp rx=0 tx=0).stop 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc158910 msgr2=0x7f2ecc1f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc158910 0x7f2ecc1f48e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009890 tx=0x7f2eb40046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 shutdown_connections 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2eac06be60 0x7f2eac06e320 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ecc162710 0x7f2ecc0d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ecc158910 0x7f2ecc1f48e0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 --2- 172.21.15.113:0/658594317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ecc06b630 0x7f2ecc1f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:52.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 >> 172.21.15.113:0/658594317 conn(0x7f2ecc0b22c0 msgr2=0x7f2ecc141d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:52.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 shutdown_connections 2024-04-25T00:05:52.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:52.909+0000 7f2ed1acd640 1 -- 172.21.15.113:0/658594317 wait complete. 2024-04-25T00:05:52.921 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:05:52.921 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:53.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.015+0000 7f058334f640 1 Processor -- start 2024-04-25T00:05:53.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.015+0000 7f058334f640 1 -- start start 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f058334f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f057c144910 0x7f057c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f058334f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c0b56b0 0x7f057c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f058334f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057c144cf0 con 0x7f057c0b5240 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f058334f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f057c0b4510 con 0x7f057c144910 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f058334f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f057c0b2f30 con 0x7f057c0b56b0 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05810eb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f057c144910 0x7f057c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05810eb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f057c144910 0x7f057c0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46590/0 (socket says 172.21.15.113:46590) 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05810eb640 1 -- 172.21.15.113:0/3942423816 learned_addr learned my addr 172.21.15.113:0/3942423816 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05818ec640 1 --2- 172.21.15.113:0/3942423816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c0b56b0 0x7f057c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:53.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05810eb640 1 -- 172.21.15.113:0/3942423816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c0b56b0 msgr2=0x7f057c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:53.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05810eb640 1 --2- 172.21.15.113:0/3942423816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c0b56b0 0x7f057c0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05810eb640 1 -- 172.21.15.113:0/3942423816 >> v1:172.21.15.178:6789/0 conn(0x7f057c0b5240 legacy=0x7f057c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:53.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.016+0000 7f05810eb640 1 -- 172.21.15.113:0/3942423816 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c0b3210 con 0x7f057c144910 2024-04-25T00:05:53.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.017+0000 7f05810eb640 1 --2- 172.21.15.113:0/3942423816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c144910 0x7f057c0b3fd0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f057800d180 tx=0x7f057802c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e536d57bd11ba573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:53.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.017+0000 7f0573fff640 1 -- 172.21.15.113:0/3942423816 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578009600 con 0x7f057c144910 2024-04-25T00:05:53.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.017+0000 7f0573fff640 1 -- 172.21.15.113:0/3942423816 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05780097a0 con 0x7f057c144910 2024-04-25T00:05:53.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.017+0000 7f0573fff640 1 -- 172.21.15.113:0/3942423816 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578009b70 con 0x7f057c144910 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 -- 172.21.15.113:0/3942423816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c144910 msgr2=0x7f057c0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 --2- 172.21.15.113:0/3942423816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c144910 0x7f057c0b3fd0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f057800d180 tx=0x7f057802c900 comp rx=0 tx=0).stop 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 -- 172.21.15.113:0/3942423816 shutdown_connections 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 --2- 172.21.15.113:0/3942423816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c0b56b0 0x7f057c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 --2- 172.21.15.113:0/3942423816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c144910 0x7f057c0b3fd0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 -- 172.21.15.113:0/3942423816 >> 172.21.15.113:0/3942423816 conn(0x7f057c0b1a50 msgr2=0x7f057c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 -- 172.21.15.113:0/3942423816 shutdown_connections 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.018+0000 7f058334f640 1 -- 172.21.15.113:0/3942423816 wait complete. 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.019+0000 7f058334f640 1 Processor -- start 2024-04-25T00:05:53.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.019+0000 7f058334f640 1 -- start start 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f058334f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c0b5240 0x7f057c15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f058334f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c0b56b0 0x7f057c0ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f058334f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c144910 0x7f057c0ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f058334f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0b4da0 con 0x7f057c0b56b0 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f058334f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0b37d0 con 0x7f057c0b5240 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f058334f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f057c0b4f20 con 0x7f057c144910 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c0b56b0 0x7f057c0ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c0b56b0 0x7f057c0ac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46604/0 (socket says 172.21.15.113:46604) 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 -- 172.21.15.113:0/3116506494 learned_addr learned my addr 172.21.15.113:0/3116506494 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05818ec640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c144910 0x7f057c0ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:53.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05810eb640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c0b5240 0x7f057c15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 -- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c144910 msgr2=0x7f057c0ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c144910 0x7f057c0ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 -- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c0b5240 msgr2=0x7f057c15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c0b5240 0x7f057c15d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 -- 172.21.15.113:0/3116506494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c15dce0 con 0x7f057c0b56b0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05818ec640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c144910 0x7f057c0ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.020+0000 7f05808ea640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c0b56b0 0x7f057c0ac280 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f056c00c460 tx=0x7f056c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.021+0000 7f0571ffb640 1 -- 172.21.15.113:0/3116506494 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c011790 con 0x7f057c0b56b0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.021+0000 7f0571ffb640 1 -- 172.21.15.113:0/3116506494 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c017070 con 0x7f057c0b56b0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.021+0000 7f0571ffb640 1 -- 172.21.15.113:0/3116506494 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c011bb0 con 0x7f057c0b56b0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.021+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c0acdf0 con 0x7f057c0b56b0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.021+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c0ad3b0 con 0x7f057c0b56b0 2024-04-25T00:05:53.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.022+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057c0691a0 con 0x7f057c0b56b0 2024-04-25T00:05:53.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.022+0000 7f0571ffb640 1 -- 172.21.15.113:0/3116506494 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f056c01c460 con 0x7f057c0b56b0 2024-04-25T00:05:53.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.023+0000 7f0571ffb640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f056006be30 0x7f056006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:53.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.023+0000 7f0571ffb640 1 -- 172.21.15.113:0/3116506494 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f056c08ac00 con 0x7f057c0b56b0 2024-04-25T00:05:53.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.023+0000 7f05810eb640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f056006be30 0x7f056006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:53.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.023+0000 7f05810eb640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f056006be30 0x7f056006e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0578036000 tx=0x7f057800b000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:53.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.027+0000 7f0571ffb640 1 -- 172.21.15.113:0/3116506494 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f056c091080 con 0x7f057c0b56b0 2024-04-25T00:05:53.150 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:05:53.152 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:05:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.270+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f057c06c100 con 0x7f057c0b56b0 2024-04-25T00:05:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.270+0000 7f0571ffb640 1 -- 172.21.15.113:0/3116506494 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1444 (secure 0 0 0) 0x7f056c0595b0 con 0x7f057c0b56b0 2024-04-25T00:05:53.271 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:53.271 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":478,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":952,"data_bytes":975657382,"bytes_used":2264375296,"bytes_avail":770829737984,"bytes_total":773094113280,"write_bytes_sec":81246422,"read_op_per_sec":0,"write_op_per_sec":79},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T00:05:25.194977+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f056006be30 msgr2=0x7f056006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f056006be30 0x7f056006e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0578036000 tx=0x7f057800b000 comp rx=0 tx=0).stop 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c0b56b0 msgr2=0x7f057c0ac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c0b56b0 0x7f057c0ac280 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f056c00c460 tx=0x7f056c00c920 comp rx=0 tx=0).stop 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 shutdown_connections 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f056006be30 0x7f056006e2f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f057c144910 0x7f057c0ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f057c0b56b0 0x7f057c0ac280 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 --2- 172.21.15.113:0/3116506494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c0b5240 0x7f057c15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 >> 172.21.15.113:0/3116506494 conn(0x7f057c0b1a50 msgr2=0x7f057c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 shutdown_connections 2024-04-25T00:05:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:53.272+0000 7f058334f640 1 -- 172.21.15.113:0/3116506494 wait complete. 2024-04-25T00:05:53.283 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:05:54.284 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:54.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.394+0000 7f976d316640 1 Processor -- start 2024-04-25T00:05:54.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.394+0000 7f976d316640 1 -- start start 2024-04-25T00:05:54.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f976d316640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9768161fe0 0x7f976806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f976d316640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f976806b800 con 0x7f976806c030 2024-04-25T00:05:54.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f976d316640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97681624e0 con 0x7f976806bc50 2024-04-25T00:05:54.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f976d316640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f97681627c0 con 0x7f9768161fe0 2024-04-25T00:05:54.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f975ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9768161fe0 0x7f976806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f97677fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f976806c030 0x7f97680709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34710/0 (socket says 172.21.15.113:34710) 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f97677fe640 1 -- 172.21.15.113:0/3856091447 learned_addr learned my addr 172.21.15.113:0/3856091447 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f975ffff640 1 -- 172.21.15.113:0/3856091447 >> v1:172.21.15.178:6789/0 conn(0x7f976806bc50 legacy=0x7f976806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f975ffff640 1 -- 172.21.15.113:0/3856091447 >> v1:172.21.15.113:6789/0 conn(0x7f976806c030 legacy=0x7f97680709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f975ffff640 1 -- 172.21.15.113:0/3856091447 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9768162aa0 con 0x7f9768161fe0 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.395+0000 7f97667fc640 1 -- 172.21.15.113:0/3856091447 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104976294 0 0) 0x7f976806b800 con 0x7f976806c030 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.396+0000 7f975ffff640 1 --2- 172.21.15.113:0/3856091447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768161fe0 0x7f976806cc70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f97500097a0 tx=0x7f975002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4eac9b8713d1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.396+0000 7f97667fc640 1 -- 172.21.15.113:0/3856091447 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750037070 con 0x7f9768161fe0 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.396+0000 7f97667fc640 1 -- 172.21.15.113:0/3856091447 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9750032440 con 0x7f9768161fe0 2024-04-25T00:05:54.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.396+0000 7f97667fc640 1 -- 172.21.15.113:0/3856091447 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750032810 con 0x7f9768161fe0 2024-04-25T00:05:54.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.396+0000 7f976d316640 1 -- 172.21.15.113:0/3856091447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768161fe0 msgr2=0x7f976806cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.396+0000 7f976d316640 1 --2- 172.21.15.113:0/3856091447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768161fe0 0x7f976806cc70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f97500097a0 tx=0x7f975002a230 comp rx=0 tx=0).stop 2024-04-25T00:05:54.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.397+0000 7f976d316640 1 -- 172.21.15.113:0/3856091447 shutdown_connections 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.397+0000 7f976d316640 1 --2- 172.21.15.113:0/3856091447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768161fe0 0x7f976806cc70 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.397+0000 7f976d316640 1 -- 172.21.15.113:0/3856091447 >> 172.21.15.113:0/3856091447 conn(0x7f97680b3a70 msgr2=0x7f97680b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.397+0000 7f976d316640 1 -- 172.21.15.113:0/3856091447 shutdown_connections 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.397+0000 7f976d316640 1 -- 172.21.15.113:0/3856091447 wait complete. 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 Processor -- start 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 -- start start 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f976806bc50 0x7f97681543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976806c030 0x7f9768161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9768161fe0 0x7f9768161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9768071940 con 0x7f976806c030 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f976815ea30 con 0x7f976806bc50 2024-04-25T00:05:54.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.398+0000 7f976d316640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97680710b0 con 0x7f9768161fe0 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976806c030 0x7f9768161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976806c030 0x7f9768161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46624/0 (socket says 172.21.15.113:46624) 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 -- 172.21.15.113:0/4058458622 learned_addr learned my addr 172.21.15.113:0/4058458622 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f9766ffd640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f976806bc50 0x7f97681543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f97677fe640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9768161fe0 0x7f9768161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 -- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9768161fe0 msgr2=0x7f9768161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9768161fe0 0x7f9768161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 -- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f976806bc50 msgr2=0x7f97681543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f976806bc50 0x7f97681543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 -- 172.21.15.113:0/4058458622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97680c5bc0 con 0x7f976806c030 2024-04-25T00:05:54.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975ffff640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976806c030 0x7f9768161240 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f9750000c00 tx=0x7f97500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975f7fe640 1 -- 172.21.15.113:0/4058458622 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750037040 con 0x7f976806c030 2024-04-25T00:05:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975f7fe640 1 -- 172.21.15.113:0/4058458622 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9750042070 con 0x7f976806c030 2024-04-25T00:05:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97680c5e60 con 0x7f976806c030 2024-04-25T00:05:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f975f7fe640 1 -- 172.21.15.113:0/4058458622 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750030a70 con 0x7f976806c030 2024-04-25T00:05:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.399+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97680c6420 con 0x7f976806c030 2024-04-25T00:05:54.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.400+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97680691a0 con 0x7f976806c030 2024-04-25T00:05:54.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.401+0000 7f975f7fe640 1 -- 172.21.15.113:0/4058458622 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9750030c10 con 0x7f976806c030 2024-04-25T00:05:54.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.401+0000 7f975f7fe640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f974806be30 0x7f974806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.401+0000 7f975f7fe640 1 -- 172.21.15.113:0/4058458622 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f97500ab720 con 0x7f976806c030 2024-04-25T00:05:54.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.401+0000 7f9766ffd640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f974806be30 0x7f974806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.402+0000 7f9766ffd640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f974806be30 0x7f974806e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9758000f60 tx=0x7f9758009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:54.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.405+0000 7f975f7fe640 1 -- 172.21.15.113:0/4058458622 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9750079b10 con 0x7f976806c030 2024-04-25T00:05:54.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.613+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97680714d0 con 0x7f976806c030 2024-04-25T00:05:54.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.613+0000 7f975f7fe640 1 -- 172.21.15.113:0/4058458622 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f975007de10 con 0x7f976806c030 2024-04-25T00:05:54.614 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:54.614 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:05:54.614 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f974806be30 msgr2=0x7f974806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f974806be30 0x7f974806e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9758000f60 tx=0x7f9758009040 comp rx=0 tx=0).stop 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976806c030 msgr2=0x7f9768161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976806c030 0x7f9768161240 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f9750000c00 tx=0x7f97500046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 shutdown_connections 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f974806be30 0x7f974806e2f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9768161fe0 0x7f9768161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976806c030 0x7f9768161240 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 --2- 172.21.15.113:0/4058458622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f976806bc50 0x7f97681543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 >> 172.21.15.113:0/4058458622 conn(0x7f97680b3a70 msgr2=0x7f97680cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:54.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 shutdown_connections 2024-04-25T00:05:54.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.615+0000 7f976d316640 1 -- 172.21.15.113:0/4058458622 wait complete. 2024-04-25T00:05:54.627 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:05:54.627 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.722+0000 7fdab9384640 1 Processor -- start 2024-04-25T00:05:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.722+0000 7fdab9384640 1 -- start start 2024-04-25T00:05:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.722+0000 7fdab9384640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab4161770 0x7fdab406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.722+0000 7fdab9384640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab4071870 con 0x7fdab41623e0 2024-04-25T00:05:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.722+0000 7fdab9384640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab40cfbc0 con 0x7fdab40713f0 2024-04-25T00:05:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.722+0000 7fdab9384640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdab406d0f0 con 0x7fdab4161770 2024-04-25T00:05:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdab41623e0 0x7fdab406b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34726/0 (socket says 172.21.15.113:34726) 2024-04-25T00:05:54.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab37fe640 1 -- 172.21.15.113:0/2237647419 learned_addr learned my addr 172.21.15.113:0/2237647419 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/2237647419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab4161770 0x7fdab406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab1ffb640 1 -- 172.21.15.113:0/2237647419 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2561204913 0 0) 0x7fdab4071870 con 0x7fdab41623e0 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab1ffb640 1 -- 172.21.15.113:0/2237647419 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda9c003520 con 0x7fdab41623e0 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab1ffb640 1 -- 172.21.15.113:0/2237647419 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1926287546 0 0) 0x7fdab40cfbc0 con 0x7fdab40713f0 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab1ffb640 1 -- 172.21.15.113:0/2237647419 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdab4071870 con 0x7fdab40713f0 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab2ffd640 1 -- 172.21.15.113:0/2237647419 >> v1:172.21.15.178:6789/0 conn(0x7fdab40713f0 legacy=0x7fdab40cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab2ffd640 1 -- 172.21.15.113:0/2237647419 >> v1:172.21.15.113:6789/0 conn(0x7fdab41623e0 legacy=0x7fdab406b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.723+0000 7fdab2ffd640 1 -- 172.21.15.113:0/2237647419 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab40cfea0 con 0x7fdab4161770 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/2237647419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab4161770 0x7fdab406c9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009dc0 tx=0x7fdaac02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35006cbf89c62785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab1ffb640 1 -- 172.21.15.113:0/2237647419 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac02ae60 con 0x7fdab4161770 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab1ffb640 1 -- 172.21.15.113:0/2237647419 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac007740 con 0x7fdab4161770 2024-04-25T00:05:54.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab1ffb640 1 -- 172.21.15.113:0/2237647419 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac007af0 con 0x7fdab4161770 2024-04-25T00:05:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab9384640 1 -- 172.21.15.113:0/2237647419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab4161770 msgr2=0x7fdab406c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab9384640 1 --2- 172.21.15.113:0/2237647419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab4161770 0x7fdab406c9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009dc0 tx=0x7fdaac02a640 comp rx=0 tx=0).stop 2024-04-25T00:05:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab9384640 1 -- 172.21.15.113:0/2237647419 shutdown_connections 2024-04-25T00:05:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab9384640 1 --2- 172.21.15.113:0/2237647419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab4161770 0x7fdab406c9f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab9384640 1 -- 172.21.15.113:0/2237647419 >> 172.21.15.113:0/2237647419 conn(0x7fdab40d1ec0 msgr2=0x7fdab40d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab9384640 1 -- 172.21.15.113:0/2237647419 shutdown_connections 2024-04-25T00:05:54.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.724+0000 7fdab9384640 1 -- 172.21.15.113:0/2237647419 wait complete. 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 Processor -- start 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 -- start start 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab40713f0 0x7fdab414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab4161770 0x7fdab415e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab41623e0 0x7fdab4160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4161b50 con 0x7fdab4161770 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4161f50 con 0x7fdab41623e0 2024-04-25T00:05:54.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab9384640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdab40d0180 con 0x7fdab40713f0 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab4161770 0x7fdab415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab4161770 0x7fdab415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46640/0 (socket says 172.21.15.113:46640) 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.726+0000 7fdab27fc640 1 -- 172.21.15.113:0/3609320812 learned_addr learned my addr 172.21.15.113:0/3609320812 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab40713f0 0x7fdab414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab37fe640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab41623e0 0x7fdab4160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab27fc640 1 -- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab40713f0 msgr2=0x7fdab414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab27fc640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab40713f0 0x7fdab414d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab27fc640 1 -- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab41623e0 msgr2=0x7fdab4160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab27fc640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab41623e0 0x7fdab4160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab27fc640 1 -- 172.21.15.113:0/3609320812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab40c69c0 con 0x7fdab4161770 2024-04-25T00:05:54.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab27fc640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab4161770 0x7fdab415e540 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c490 tx=0x7fdaa800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fda9b7fe640 1 -- 172.21.15.113:0/3609320812 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80052d0 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fda9b7fe640 1 -- 172.21.15.113:0/3609320812 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa8005470 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab40c6ca0 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fda9b7fe640 1 -- 172.21.15.113:0/3609320812 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8012850 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.727+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab40c7260 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.728+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab40691a0 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.729+0000 7fda9b7fe640 1 -- 172.21.15.113:0/3609320812 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdaa80129f0 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.729+0000 7fda9b7fe640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.729+0000 7fda9b7fe640 1 -- 172.21.15.113:0/3609320812 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fdaa808aa20 con 0x7fdab4161770 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.729+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.729+0000 7fdab2ffd640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdaac034000 tx=0x7fdaac02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:54.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.733+0000 7fda9b7fe640 1 -- 172.21.15.113:0/3609320812 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdaa8058d00 con 0x7fdab4161770 2024-04-25T00:05:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.974+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fdab406fd90 con 0x7fdab4161770 2024-04-25T00:05:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.974+0000 7fda9b7fe640 1 -- 172.21.15.113:0/3609320812 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1444 (secure 0 0 0) 0x7fdaa805d000 con 0x7fdab4161770 2024-04-25T00:05:54.975 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:54.975 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":479,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":952,"data_bytes":975657382,"bytes_used":2264375296,"bytes_avail":770829737984,"bytes_total":773094113280,"write_bytes_sec":81246422,"read_op_per_sec":0,"write_op_per_sec":79},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T00:05:25.194977+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 msgr2=0x7fdaa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdaac034000 tx=0x7fdaac02f000 comp rx=0 tx=0).stop 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab4161770 msgr2=0x7fdab415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab4161770 0x7fdab415e540 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c490 tx=0x7fdaa800c950 comp rx=0 tx=0).stop 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 shutdown_connections 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa006be30 0x7fdaa006e2f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab41623e0 0x7fdab4160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab4161770 0x7fdab415e540 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 --2- 172.21.15.113:0/3609320812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdab40713f0 0x7fdab414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 >> 172.21.15.113:0/3609320812 conn(0x7fdab40d1ec0 msgr2=0x7fdab40c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 shutdown_connections 2024-04-25T00:05:54.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:54.976+0000 7fdab9384640 1 -- 172.21.15.113:0/3609320812 wait complete. 2024-04-25T00:05:54.987 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:05:55.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:56.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.102+0000 7fb8da4d6640 1 Processor -- start 2024-04-25T00:05:56.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.102+0000 7fb8da4d6640 1 -- start start 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.102+0000 7fb8da4d6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8d406e6e0 0x7fb8d40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.102+0000 7fb8da4d6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d4144cf0 con 0x7fb8d4144910 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.102+0000 7fb8da4d6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d40b2f30 con 0x7fb8d40b3fd0 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.102+0000 7fb8da4d6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d40b3210 con 0x7fb8d406e6e0 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.103+0000 7fb8d8a73640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8d406e6e0 0x7fb8d40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.103+0000 7fb8d3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb8d4144910 0x7fb8d40b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34734/0 (socket says 172.21.15.113:34734) 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.103+0000 7fb8d3fff640 1 -- 172.21.15.113:0/2329745146 learned_addr learned my addr 172.21.15.113:0/2329745146 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.103+0000 7fb8d8a73640 1 -- 172.21.15.113:0/2329745146 >> v1:172.21.15.178:6789/0 conn(0x7fb8d40b3fd0 legacy=0x7fb8d406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.103+0000 7fb8d8a73640 1 -- 172.21.15.113:0/2329745146 >> v1:172.21.15.113:6789/0 conn(0x7fb8d4144910 legacy=0x7fb8d40b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.103+0000 7fb8d8a73640 1 -- 172.21.15.113:0/2329745146 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40b34f0 con 0x7fb8d406e6e0 2024-04-25T00:05:56.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.103+0000 7fb8d2ffd640 1 -- 172.21.15.113:0/2329745146 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502984599 0 0) 0x7fb8d4144cf0 con 0x7fb8d4144910 2024-04-25T00:05:56.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8d8a73640 1 --2- 172.21.15.113:0/2329745146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d406e6e0 0x7fb8d40cd500 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009dc0 tx=0x7fb8cc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc7bcd51d5ee8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:56.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8d2ffd640 1 -- 172.21.15.113:0/2329745146 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc007740 con 0x7fb8d406e6e0 2024-04-25T00:05:56.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8d2ffd640 1 -- 172.21.15.113:0/2329745146 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8cc0078e0 con 0x7fb8d406e6e0 2024-04-25T00:05:56.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8d2ffd640 1 -- 172.21.15.113:0/2329745146 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc007cb0 con 0x7fb8d406e6e0 2024-04-25T00:05:56.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2329745146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d406e6e0 msgr2=0x7fb8d40cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2329745146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d406e6e0 0x7fb8d40cd500 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009dc0 tx=0x7fb8cc02a730 comp rx=0 tx=0).stop 2024-04-25T00:05:56.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2329745146 shutdown_connections 2024-04-25T00:05:56.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2329745146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d406e6e0 0x7fb8d40cd500 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2329745146 >> 172.21.15.113:0/2329745146 conn(0x7fb8d40b1a50 msgr2=0x7fb8d40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:56.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2329745146 shutdown_connections 2024-04-25T00:05:56.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.104+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2329745146 wait complete. 2024-04-25T00:05:56.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.106+0000 7fb8da4d6640 1 Processor -- start 2024-04-25T00:05:56.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.106+0000 7fb8da4d6640 1 -- start start 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8da4d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8d406e6e0 0x7fb8d40adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8da4d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8d40b3fd0 0x7fb8d40c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8da4d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d4144910 0x7fb8d415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8da4d6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d40b4da0 con 0x7fb8d4144910 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8da4d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d40b4440 con 0x7fb8d406e6e0 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8da4d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d40b4f20 con 0x7fb8d40b3fd0 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d4144910 0x7fb8d415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d4144910 0x7fb8d415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46672/0 (socket says 172.21.15.113:46672) 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 -- 172.21.15.113:0/2500603156 learned_addr learned my addr 172.21.15.113:0/2500603156 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d3fff640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8d406e6e0 0x7fb8d40adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d37fe640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8d40b3fd0 0x7fb8d40c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 -- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8d40b3fd0 msgr2=0x7fb8d40c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8d40b3fd0 0x7fb8d40c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 -- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8d406e6e0 msgr2=0x7fb8d40adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8d406e6e0 0x7fb8d40adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 -- 172.21.15.113:0/2500603156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40c5c10 con 0x7fb8d4144910 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.107+0000 7fb8d8a73640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d4144910 0x7fb8d415db10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009ee0 tx=0x7fb8cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.108+0000 7fb8d0ff9640 1 -- 172.21.15.113:0/2500603156 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc040070 con 0x7fb8d4144910 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.108+0000 7fb8d0ff9640 1 -- 172.21.15.113:0/2500603156 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8cc031bc0 con 0x7fb8d4144910 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.108+0000 7fb8d0ff9640 1 -- 172.21.15.113:0/2500603156 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc045460 con 0x7fb8d4144910 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.108+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d40c5e90 con 0x7fb8d4144910 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.108+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d40c6450 con 0x7fb8d4144910 2024-04-25T00:05:56.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.109+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d40691a0 con 0x7fb8d4144910 2024-04-25T00:05:56.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.109+0000 7fb8d0ff9640 1 -- 172.21.15.113:0/2500603156 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8cc039040 con 0x7fb8d4144910 2024-04-25T00:05:56.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.109+0000 7fb8d0ff9640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb8c006be30 0x7fb8c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.109+0000 7fb8d0ff9640 1 -- 172.21.15.113:0/2500603156 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fb8cc0ac680 con 0x7fb8d4144910 2024-04-25T00:05:56.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.110+0000 7fb8d3fff640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb8c006be30 0x7fb8c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.110+0000 7fb8d3fff640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb8c006be30 0x7fb8c006e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000f60 tx=0x7fb8c4007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:56.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.114+0000 7fb8d0ff9640 1 -- 172.21.15.113:0/2500603156 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb8d40c5c10 con 0x7fb8d4144910 2024-04-25T00:05:56.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.320+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8d40731a0 con 0x7fb8d4144910 2024-04-25T00:05:56.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.321+0000 7fb8d0ff9640 1 -- 172.21.15.113:0/2500603156 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7fb8cc07aaa0 con 0x7fb8d4144910 2024-04-25T00:05:56.321 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:56.321 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:05:56.322 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:05:56.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb8c006be30 msgr2=0x7fb8c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb8c006be30 0x7fb8c006e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000f60 tx=0x7fb8c4007e40 comp rx=0 tx=0).stop 2024-04-25T00:05:56.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d4144910 msgr2=0x7fb8d415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d4144910 0x7fb8d415db10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009ee0 tx=0x7fb8cc004a40 comp rx=0 tx=0).stop 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 shutdown_connections 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb8c006be30 0x7fb8c006e2f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8d4144910 0x7fb8d415db10 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8d40b3fd0 0x7fb8d40c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 --2- 172.21.15.113:0/2500603156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8d406e6e0 0x7fb8d40adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 >> 172.21.15.113:0/2500603156 conn(0x7fb8d40b1a50 msgr2=0x7fb8d415eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 shutdown_connections 2024-04-25T00:05:56.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.323+0000 7fb8da4d6640 1 -- 172.21.15.113:0/2500603156 wait complete. 2024-04-25T00:05:56.335 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:05:56.335 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.430+0000 7febe0d70640 1 Processor -- start 2024-04-25T00:05:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.430+0000 7febe0d70640 1 -- start start 2024-04-25T00:05:56.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febe0d70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febe0d70640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7febdc0b3af0 0x7febdc0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febe0d70640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febdc0b2d50 con 0x7febdc15a6b0 2024-04-25T00:05:56.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febe0d70640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7febdc072a70 con 0x7febdc0b3af0 2024-04-25T00:05:56.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febe0d70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febdc0b3010 con 0x7febdc072ea0 2024-04-25T00:05:56.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febdad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7febdc0b3af0 0x7febdc0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febdad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7febdc0b3af0 0x7febdc0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46680/0 (socket says 172.21.15.113:46680) 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febdad9d640 1 -- 172.21.15.113:0/2624292768 learned_addr learned my addr 172.21.15.113:0/2624292768 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febdad9d640 1 -- 172.21.15.113:0/2624292768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 msgr2=0x7febdc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febd3fff640 1 --2- 172.21.15.113:0/2624292768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febdad9d640 1 --2- 172.21.15.113:0/2624292768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febdad9d640 1 -- 172.21.15.113:0/2624292768 >> v1:172.21.15.178:6789/0 conn(0x7febdc15a6b0 legacy=0x7febdc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.431+0000 7febdad9d640 1 -- 172.21.15.113:0/2624292768 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc0b32f0 con 0x7febdc0b3af0 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.432+0000 7febdad9d640 1 --2- 172.21.15.113:0/2624292768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc0b3f80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e50 tx=0x7febd402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8104d4e2c458c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:56.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.432+0000 7febd9d9b640 1 -- 172.21.15.113:0/2624292768 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4036070 con 0x7febdc0b3af0 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.432+0000 7febd9d9b640 1 -- 172.21.15.113:0/2624292768 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd4031600 con 0x7febdc0b3af0 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.432+0000 7febd9d9b640 1 -- 172.21.15.113:0/2624292768 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40319d0 con 0x7febdc0b3af0 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 -- 172.21.15.113:0/2624292768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 msgr2=0x7febdc0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 --2- 172.21.15.113:0/2624292768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc0b3f80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e50 tx=0x7febd402a6c0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 -- 172.21.15.113:0/2624292768 shutdown_connections 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 --2- 172.21.15.113:0/2624292768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 --2- 172.21.15.113:0/2624292768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc0b3f80 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 -- 172.21.15.113:0/2624292768 >> 172.21.15.113:0/2624292768 conn(0x7febdc0b1a50 msgr2=0x7febdc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 -- 172.21.15.113:0/2624292768 shutdown_connections 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.433+0000 7febe0d70640 1 -- 172.21.15.113:0/2624292768 wait complete. 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.434+0000 7febe0d70640 1 Processor -- start 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.434+0000 7febe0d70640 1 -- start start 2024-04-25T00:05:56.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febe0d70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febe0d70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febe0d70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc15a6b0 0x7febdc156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febe0d70640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febdc15aa90 con 0x7febdc0b3af0 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febe0d70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7febdc072720 con 0x7febdc15a6b0 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febe0d70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febdc15ac10 con 0x7febdc072ea0 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46690/0 (socket says 172.21.15.113:46690) 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 -- 172.21.15.113:0/4268177313 learned_addr learned my addr 172.21.15.113:0/4268177313 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febdad9d640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc15a6b0 0x7febdc156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 -- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 msgr2=0x7febdc14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febda59c640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc14c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 -- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc15a6b0 msgr2=0x7febdc156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc15a6b0 0x7febdc156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 -- 172.21.15.113:0/4268177313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc155c00 con 0x7febdc0b3af0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febdad9d640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc15a6b0 0x7febdc156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd3fff640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc152cf0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7febc400c7f0 tx=0x7febc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd2ffd640 1 -- 172.21.15.113:0/4268177313 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4015070 con 0x7febdc0b3af0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd2ffd640 1 -- 172.21.15.113:0/4268177313 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc4004220 con 0x7febdc0b3af0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.435+0000 7febd2ffd640 1 -- 172.21.15.113:0/4268177313 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4007620 con 0x7febdc0b3af0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.436+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc1561c0 con 0x7febdc0b3af0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.436+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc156780 con 0x7febdc0b3af0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.436+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febdc156980 con 0x7febdc0b3af0 2024-04-25T00:05:56.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.437+0000 7febd2ffd640 1 -- 172.21.15.113:0/4268177313 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febc40079d0 con 0x7febdc0b3af0 2024-04-25T00:05:56.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.437+0000 7febd2ffd640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7febbc06be30 0x7febbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:56.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.437+0000 7febda59c640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7febbc06be30 0x7febbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:56.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.437+0000 7febd2ffd640 1 -- 172.21.15.113:0/4268177313 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7febc408b290 con 0x7febdc0b3af0 2024-04-25T00:05:56.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.438+0000 7febda59c640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7febbc06be30 0x7febbc06e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7febcc00a750 tx=0x7febcc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:56.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.442+0000 7febd2ffd640 1 -- 172.21.15.113:0/4268177313 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7febdc156980 con 0x7febdc0b3af0 2024-04-25T00:05:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.680+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7febdc0691a0 con 0x7febdc0b3af0 2024-04-25T00:05:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.680+0000 7febd2ffd640 1 -- 172.21.15.113:0/4268177313 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1444 (secure 0 0 0) 0x7febc4059c40 con 0x7febdc0b3af0 2024-04-25T00:05:56.681 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:56.681 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":481,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":952,"data_bytes":975657382,"bytes_used":2264375296,"bytes_avail":770829737984,"bytes_total":773094113280,"write_bytes_sec":81248516,"read_op_per_sec":0,"write_op_per_sec":79},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T00:05:25.194977+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:56.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7febbc06be30 msgr2=0x7febbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7febbc06be30 0x7febbc06e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7febcc00a750 tx=0x7febcc008040 comp rx=0 tx=0).stop 2024-04-25T00:05:56.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 msgr2=0x7febdc152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:56.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc152cf0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7febc400c7f0 tx=0x7febc400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 shutdown_connections 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7febbc06be30 0x7febbc06e2f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc15a6b0 0x7febdc156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febdc0b3af0 0x7febdc152cf0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 --2- 172.21.15.113:0/4268177313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febdc072ea0 0x7febdc14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 >> 172.21.15.113:0/4268177313 conn(0x7febdc0b1a50 msgr2=0x7febdc157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 shutdown_connections 2024-04-25T00:05:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:56.682+0000 7febe0d70640 1 -- 172.21.15.113:0/4268177313 wait complete. 2024-04-25T00:05:56.693 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:05:57.694 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.796+0000 7f5690922640 1 Processor -- start 2024-04-25T00:05:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.796+0000 7f5690922640 1 -- start start 2024-04-25T00:05:57.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5690922640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f568c06e6e0 0x7f568c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5690922640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568c144cf0 con 0x7f568c144910 2024-04-25T00:05:57.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5690922640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568c0b2f30 con 0x7f568c0b3fd0 2024-04-25T00:05:57.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5690922640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f568c0b3210 con 0x7f568c06e6e0 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f568a59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f568c144910 0x7f568c0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56584/0 (socket says 172.21.15.113:56584) 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f568a59c640 1 -- 172.21.15.113:0/2213717237 learned_addr learned my addr 172.21.15.113:0/2213717237 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2937931035 0 0) 0x7f568c144cf0 con 0x7f568c144910 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5678003520 con 0x7f568c144910 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f568ad9d640 1 --2- 172.21.15.113:0/2213717237 >> v2:172.21.15.178:3300/0 conn(0x7f568c06e6e0 0x7f568c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 146285226 0 0) 0x7f5678003520 con 0x7f568c144910 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 >> v2:172.21.15.178:3300/0 conn(0x7f568c06e6e0 msgr2=0x7f568c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 --2- 172.21.15.113:0/2213717237 >> v2:172.21.15.178:3300/0 conn(0x7f568c06e6e0 0x7f568c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 >> v1:172.21.15.178:6789/0 conn(0x7f568c0b3fd0 legacy=0x7f568c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:57.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c0b34f0 con 0x7f568c144910 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f567c001670 con 0x7f568c144910 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 >> v1:172.21.15.113:6789/0 conn(0x7f568c144910 legacy=0x7f568c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780039b0 0x7f5678003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5678004800 0x7f5678004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.797+0000 7f5689d9b640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678005210 0x7f567800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0b3210 con 0x7f5678004800 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f567800c140 con 0x7f56780039b0 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f567800e550 con 0x7f5678005210 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5683fff640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5678004800 0x7f5678004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f568a59c640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780039b0 0x7f5678003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f568ad9d640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678005210 0x7f567800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5683fff640 1 -- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678005210 msgr2=0x7f567800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5683fff640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678005210 0x7f567800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5683fff640 1 -- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780039b0 msgr2=0x7f5678003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5683fff640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780039b0 0x7f5678003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f5683fff640 1 -- 172.21.15.113:0/2213717237 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5678003520 con 0x7f5678004800 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f568a59c640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780039b0 0x7f5678003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f568ad9d640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678005210 0x7f567800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.798+0000 7f568ad9d640 1 -- 172.21.15.113:0/2213717237 reap_dead start 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5683fff640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5678004800 0x7f5678004c80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f5674003db0 tx=0x7f5678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764c83d1599410a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674037070 con 0x7f5678004800 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567402a9f0 con 0x7f5678004800 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5689d9b640 1 -- 172.21.15.113:0/2213717237 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674003980 con 0x7f5678004800 2024-04-25T00:05:57.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5690922640 1 -- 172.21.15.113:0/2213717237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5678004800 msgr2=0x7f5678004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:57.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5690922640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5678004800 0x7f5678004c80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f5674003db0 tx=0x7f5678005830 comp rx=0 tx=0).stop 2024-04-25T00:05:57.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5690922640 1 -- 172.21.15.113:0/2213717237 shutdown_connections 2024-04-25T00:05:57.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5690922640 1 --2- 172.21.15.113:0/2213717237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5678004800 0x7f5678004c80 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:57.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5690922640 1 -- 172.21.15.113:0/2213717237 >> 172.21.15.113:0/2213717237 conn(0x7f568c0b1a50 msgr2=0x7f568c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:57.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5690922640 1 -- 172.21.15.113:0/2213717237 shutdown_connections 2024-04-25T00:05:57.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.799+0000 7f5690922640 1 -- 172.21.15.113:0/2213717237 wait complete. 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 Processor -- start 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 -- start start 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678004800 0x7f568c13e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568c0b0fc0 0x7f568c0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568c0b1440 0x7f568c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0b4da0 con 0x7f568c0b1440 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0b4440 con 0x7f5678004800 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f5690922640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f568c0b4f20 con 0x7f568c0b0fc0 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f568ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568c0b1440 0x7f568c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f568ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568c0b1440 0x7f568c0cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47848/0 (socket says 172.21.15.113:47848) 2024-04-25T00:05:57.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.801+0000 7f568ad9d640 1 -- 172.21.15.113:0/3374957081 learned_addr learned my addr 172.21.15.113:0/3374957081 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f5683fff640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568c0b0fc0 0x7f568c0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f568a59c640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678004800 0x7f568c13e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f568ad9d640 1 -- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568c0b0fc0 msgr2=0x7f568c0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f568ad9d640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568c0b0fc0 0x7f568c0cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f568ad9d640 1 -- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678004800 msgr2=0x7f568c13e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f568ad9d640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678004800 0x7f568c13e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f568ad9d640 1 -- 172.21.15.113:0/3374957081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c1efdb0 con 0x7f568c0b1440 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f5683fff640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568c0b0fc0 0x7f568c0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f568ad9d640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568c0b1440 0x7f568c0cda00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f568c0752f0 tx=0x7f5684009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f5682ffd640 1 -- 172.21.15.113:0/3374957081 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684019070 con 0x7f568c0b1440 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f5682ffd640 1 -- 172.21.15.113:0/3374957081 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5684007c10 con 0x7f568c0b1440 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f5682ffd640 1 -- 172.21.15.113:0/3374957081 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684014400 con 0x7f568c0b1440 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c13ed80 con 0x7f568c0b1440 2024-04-25T00:05:57.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.802+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c0b0060 con 0x7f568c0b1440 2024-04-25T00:05:57.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.803+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c0691a0 con 0x7f568c0b1440 2024-04-25T00:05:57.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.803+0000 7f5682ffd640 1 -- 172.21.15.113:0/3374957081 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5684005d10 con 0x7f568c0b1440 2024-04-25T00:05:57.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.804+0000 7f5682ffd640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f566c06be60 0x7f566c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:57.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.804+0000 7f5682ffd640 1 -- 172.21.15.113:0/3374957081 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f568408a910 con 0x7f568c0b1440 2024-04-25T00:05:57.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.804+0000 7f568a59c640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f566c06be60 0x7f566c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:57.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.804+0000 7f568a59c640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f566c06be60 0x7f566c06e320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f567c001ce0 tx=0x7f567c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:57.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:57.808+0000 7f5682ffd640 1 -- 172.21.15.113:0/3374957081 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f568c13ed80 con 0x7f568c0b1440 2024-04-25T00:05:58.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.019+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f568c0731a0 con 0x7f568c0b1440 2024-04-25T00:05:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.020+0000 7f5682ffd640 1 -- 172.21.15.113:0/3374957081 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f56840592a0 con 0x7f568c0b1440 2024-04-25T00:05:58.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:58.020 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:05:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:05:58.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f566c06be60 msgr2=0x7f566c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f566c06be60 0x7f566c06e320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f567c001ce0 tx=0x7f567c009040 comp rx=0 tx=0).stop 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568c0b1440 msgr2=0x7f568c0cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568c0b1440 0x7f568c0cda00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f568c0752f0 tx=0x7f5684009cd0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 shutdown_connections 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f566c06be60 0x7f566c06e320 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568c0b1440 0x7f568c0cda00 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568c0b0fc0 0x7f568c0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 --2- 172.21.15.113:0/3374957081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678004800 0x7f568c13e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 >> 172.21.15.113:0/3374957081 conn(0x7f568c0b1a50 msgr2=0x7f568c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 shutdown_connections 2024-04-25T00:05:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.022+0000 7f5690922640 1 -- 172.21.15.113:0/3374957081 wait complete. 2024-04-25T00:05:58.033 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:05:58.034 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:58.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.128+0000 7f06972d4640 1 Processor -- start 2024-04-25T00:05:58.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.128+0000 7f06972d4640 1 -- start start 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.129+0000 7f06972d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06900d2290 0x7f06900b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.129+0000 7f06972d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06900d2800 con 0x7f069015e320 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.129+0000 7f06972d4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06900d2ae0 con 0x7f06900d0b00 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.129+0000 7f06972d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f06900d3410 con 0x7f06900d2290 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.129+0000 7f069486f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06900d2290 0x7f06900b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.129+0000 7f0695070640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f069015e320 0x7f06900d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56600/0 (socket says 172.21.15.113:56600) 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.129+0000 7f0695070640 1 -- 172.21.15.113:0/2547393036 learned_addr learned my addr 172.21.15.113:0/2547393036 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.130+0000 7f069486f640 1 -- 172.21.15.113:0/2547393036 >> v1:172.21.15.178:6789/0 conn(0x7f06900d0b00 legacy=0x7f06900d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.130+0000 7f069486f640 1 -- 172.21.15.113:0/2547393036 >> v1:172.21.15.113:6789/0 conn(0x7f069015e320 legacy=0x7f06900d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.130+0000 7f069486f640 1 -- 172.21.15.113:0/2547393036 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06900d2dc0 con 0x7f06900d2290 2024-04-25T00:05:58.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.130+0000 7f0687fff640 1 -- 172.21.15.113:0/2547393036 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792183009 0 0) 0x7f06900d2800 con 0x7f069015e320 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.130+0000 7f069486f640 1 --2- 172.21.15.113:0/2547393036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f06900b9aa0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f06800097a0 tx=0x7f068002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5211d71d5a3da1cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.130+0000 7f0687fff640 1 -- 172.21.15.113:0/2547393036 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680037070 con 0x7f06900d2290 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f0687fff640 1 -- 172.21.15.113:0/2547393036 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0680032440 con 0x7f06900d2290 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f0687fff640 1 -- 172.21.15.113:0/2547393036 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680032810 con 0x7f06900d2290 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f06972d4640 1 -- 172.21.15.113:0/2547393036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 msgr2=0x7f06900b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f06972d4640 1 --2- 172.21.15.113:0/2547393036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f06900b9aa0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f06800097a0 tx=0x7f068002a270 comp rx=0 tx=0).stop 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f06972d4640 1 -- 172.21.15.113:0/2547393036 shutdown_connections 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f06972d4640 1 --2- 172.21.15.113:0/2547393036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f06900b9aa0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f06972d4640 1 -- 172.21.15.113:0/2547393036 >> 172.21.15.113:0/2547393036 conn(0x7f06900b5c20 msgr2=0x7f06900b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f06972d4640 1 -- 172.21.15.113:0/2547393036 shutdown_connections 2024-04-25T00:05:58.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.131+0000 7f06972d4640 1 -- 172.21.15.113:0/2547393036 wait complete. 2024-04-25T00:05:58.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.132+0000 7f06972d4640 1 Processor -- start 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.132+0000 7f06972d4640 1 -- start start 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f06972d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900d0b00 0x7f0690153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f06972d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f0690157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f06972d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069015e320 0x7f06900b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f06972d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f069015e700 con 0x7f06900d2290 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f06972d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06900b9fe0 con 0x7f069015e320 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f06972d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f069015e880 con 0x7f06900d0b00 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f0690157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f0690157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47860/0 (socket says 172.21.15.113:47860) 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 -- 172.21.15.113:0/144847809 learned_addr learned my addr 172.21.15.113:0/144847809 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f0695070640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900d0b00 0x7f0690153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f0695871640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069015e320 0x7f06900b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 -- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900d0b00 msgr2=0x7f0690153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900d0b00 0x7f0690153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 -- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069015e320 msgr2=0x7f06900b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069015e320 0x7f06900b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.133+0000 7f069486f640 1 -- 172.21.15.113:0/144847809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0690158930 con 0x7f06900d2290 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.134+0000 7f069486f640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f0690157f70 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f06800098c0 tx=0x7f06800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.134+0000 7f0685ffb640 1 -- 172.21.15.113:0/144847809 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680041070 con 0x7f06900d2290 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.134+0000 7f0685ffb640 1 -- 172.21.15.113:0/144847809 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068003c870 con 0x7f06900d2290 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.134+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0690158c10 con 0x7f06900d2290 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.134+0000 7f0685ffb640 1 -- 172.21.15.113:0/144847809 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680037040 con 0x7f06900d2290 2024-04-25T00:05:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.134+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901591d0 con 0x7f06900d2290 2024-04-25T00:05:58.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.135+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06900691a0 con 0x7f06900d2290 2024-04-25T00:05:58.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.135+0000 7f0685ffb640 1 -- 172.21.15.113:0/144847809 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0680044070 con 0x7f06900d2290 2024-04-25T00:05:58.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.136+0000 7f0685ffb640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be60 0x7f067406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.136+0000 7f0685ffb640 1 -- 172.21.15.113:0/144847809 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f06800aba60 con 0x7f06900d2290 2024-04-25T00:05:58.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.136+0000 7f0695070640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be60 0x7f067406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.136+0000 7f0695070640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be60 0x7f067406e320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f068c007ed0 tx=0x7f068c007e60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.140+0000 7f0685ffb640 1 -- 172.21.15.113:0/144847809 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f068007a4a0 con 0x7f06900d2290 2024-04-25T00:05:58.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.380+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f06901547b0 con 0x7f06900d2290 2024-04-25T00:05:58.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.381+0000 7f0685ffb640 1 -- 172.21.15.113:0/144847809 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7f0680079da0 con 0x7f06900d2290 2024-04-25T00:05:58.381 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:58.381 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":483,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561799,"bytes_used":2547630080,"bytes_avail":770546483200,"bytes_total":773094113280,"write_bytes_sec":89658723,"read_op_per_sec":0,"write_op_per_sec":88},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T00:05:57.200778+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:58.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be60 msgr2=0x7f067406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be60 0x7f067406e320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f068c007ed0 tx=0x7f068c007e60 comp rx=0 tx=0).stop 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 msgr2=0x7f0690157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f0690157f70 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f06800098c0 tx=0x7f06800046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 shutdown_connections 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be60 0x7f067406e320 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069015e320 0x7f06900b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900d2290 0x7f0690157f70 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 --2- 172.21.15.113:0/144847809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900d0b00 0x7f0690153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 >> 172.21.15.113:0/144847809 conn(0x7f06900b5c20 msgr2=0x7f06900d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 shutdown_connections 2024-04-25T00:05:58.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.383+0000 7f06972d4640 1 -- 172.21.15.113:0/144847809 wait complete. 2024-04-25T00:05:58.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T00:05:58.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:58.395 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.492+0000 7fca48858640 1 Processor -- start 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.492+0000 7fca48858640 1 -- start start 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca48858640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca48858640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca440d28c0 0x7fca440b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca48858640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca440b61e0 0x7fca44071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca48858640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fca440d2e00 con 0x7fca440b61e0 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca48858640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fca440d30e0 con 0x7fca440d28c0 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca48858640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca440d33c0 con 0x7fca4415e510 2024-04-25T00:05:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca42d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca440b61e0 0x7fca44071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca42d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca440b61e0 0x7fca44071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47876/0 (socket says 172.21.15.113:47876) 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.493+0000 7fca42d9d640 1 -- 172.21.15.113:0/1298622937 learned_addr learned my addr 172.21.15.113:0/1298622937 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca4259c640 1 --2- 172.21.15.113:0/1298622937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca41d9b640 1 --2- 172.21.15.113:0/1298622937 >> v2:172.21.15.178:3300/0 conn(0x7fca440d28c0 0x7fca440b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca42d9d640 1 -- 172.21.15.113:0/1298622937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 msgr2=0x7fca440d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca42d9d640 1 --2- 172.21.15.113:0/1298622937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca42d9d640 1 -- 172.21.15.113:0/1298622937 >> v2:172.21.15.178:3300/0 conn(0x7fca440d28c0 msgr2=0x7fca440b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca42d9d640 1 --2- 172.21.15.113:0/1298622937 >> v2:172.21.15.178:3300/0 conn(0x7fca440d28c0 0x7fca440b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca42d9d640 1 -- 172.21.15.113:0/1298622937 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca440d36a0 con 0x7fca440b61e0 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca4259c640 1 --2- 172.21.15.113:0/1298622937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca41d9b640 1 --2- 172.21.15.113:0/1298622937 >> v2:172.21.15.178:3300/0 conn(0x7fca440d28c0 0x7fca440b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.494+0000 7fca42d9d640 1 --2- 172.21.15.113:0/1298622937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca44071710 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fca440d5320 tx=0x7fca3402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967d6fbdb076425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca4159a640 1 -- 172.21.15.113:0/1298622937 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34036070 con 0x7fca440b61e0 2024-04-25T00:05:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca4159a640 1 -- 172.21.15.113:0/1298622937 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca34031600 con 0x7fca440b61e0 2024-04-25T00:05:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca4159a640 1 -- 172.21.15.113:0/1298622937 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca340319d0 con 0x7fca440b61e0 2024-04-25T00:05:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 -- 172.21.15.113:0/1298622937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 msgr2=0x7fca44071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 --2- 172.21.15.113:0/1298622937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca44071710 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fca440d5320 tx=0x7fca3402a610 comp rx=0 tx=0).stop 2024-04-25T00:05:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 -- 172.21.15.113:0/1298622937 shutdown_connections 2024-04-25T00:05:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 --2- 172.21.15.113:0/1298622937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca44071710 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 --2- 172.21.15.113:0/1298622937 >> v2:172.21.15.178:3300/0 conn(0x7fca440d28c0 0x7fca440b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 --2- 172.21.15.113:0/1298622937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 -- 172.21.15.113:0/1298622937 >> 172.21.15.113:0/1298622937 conn(0x7fca4406b720 msgr2=0x7fca440d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 -- 172.21.15.113:0/1298622937 shutdown_connections 2024-04-25T00:05:58.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.495+0000 7fca48858640 1 -- 172.21.15.113:0/1298622937 wait complete. 2024-04-25T00:05:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.496+0000 7fca48858640 1 Processor -- start 2024-04-25T00:05:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.496+0000 7fca48858640 1 -- start start 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca48858640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca440b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca48858640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca440d28c0 0x7fca440b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca48858640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca48858640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fca440b9dd0 con 0x7fca440b61e0 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca48858640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca440d3ea0 con 0x7fca440d28c0 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca48858640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca44071c50 con 0x7fca4415e510 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca440b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca440b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47878/0 (socket says 172.21.15.113:47878) 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 -- 172.21.15.113:0/3076287693 learned_addr learned my addr 172.21.15.113:0/3076287693 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 -- 172.21.15.113:0/3076287693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 msgr2=0x7fca440cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 -- 172.21.15.113:0/3076287693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca440d28c0 msgr2=0x7fca440b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca440d28c0 0x7fca440b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.497+0000 7fca4259c640 1 -- 172.21.15.113:0/3076287693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca441f42f0 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.498+0000 7fca4259c640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca440b7380 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00d800 tx=0x7fca3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.498+0000 7fca32ffd640 1 -- 172.21.15.113:0/3076287693 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c004030 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.498+0000 7fca32ffd640 1 -- 172.21.15.113:0/3076287693 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c0041d0 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.498+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca441f7b40 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.498+0000 7fca32ffd640 1 -- 172.21.15.113:0/3076287693 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0096a0 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.498+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca441f80c0 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.499+0000 7fca32ffd640 1 -- 172.21.15.113:0/3076287693 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca3c009840 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.500+0000 7fca32ffd640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca2406be30 0x7fca2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.500+0000 7fca32ffd640 1 -- 172.21.15.113:0/3076287693 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fca3c08b150 con 0x7fca440b61e0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.500+0000 7fca41d9b640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca2406be30 0x7fca2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.500+0000 7fca41d9b640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca2406be30 0x7fca2406e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fca4415b0e0 tx=0x7fca38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 Processor -- start 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 -- start start 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091e6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6560 0x559a091bb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559a0911f330 con 0x559a09203b80 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a090c7ab0 con 0x559a091e6560 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.501+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559a0911b2f0 con 0x559a091e6ca0 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47894/0 (socket says 172.21.15.113:47894) 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 -- 172.21.15.113:0/1437709535 learned_addr learned my addr 172.21.15.113:0/1437709535 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca41d9b640 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca42d9d640 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6560 0x559a091bb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 -- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 msgr2=0x559a091e6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091e6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 -- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6560 msgr2=0x559a091bb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6560 0x559a091bb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 -- 172.21.15.113:0/1437709535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a08efd7d0 con 0x559a09203b80 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca4259c640 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6510 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fca3c089680 tx=0x7fca3c058ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acfa11eff78d3b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca23fff640 1 -- 172.21.15.113:0/1437709535 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c071020 con 0x559a09203b80 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca23fff640 1 -- 172.21.15.113:0/1437709535 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c05d7c0 con 0x559a09203b80 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.502+0000 7fca23fff640 1 -- 172.21.15.113:0/1437709535 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c061ff0 con 0x559a09203b80 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 -- 172.21.15.113:0/1437709535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 msgr2=0x559a091b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6510 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fca3c089680 tx=0x7fca3c058ea0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 -- 172.21.15.113:0/1437709535 shutdown_connections 2024-04-25T00:05:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6560 0x559a091bb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 --2- 172.21.15.113:0/1437709535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6510 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 -- 172.21.15.113:0/1437709535 >> 172.21.15.113:0/1437709535 conn(0x559a091b8780 msgr2=0x559a09201020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 -- 172.21.15.113:0/1437709535 shutdown_connections 2024-04-25T00:05:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 -- 172.21.15.113:0/1437709535 wait complete. 2024-04-25T00:05:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 -- start start 2024-04-25T00:05:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.503+0000 7fca4b567740 1 Processor -- start 2024-04-25T00:05:58.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6ca0 0x559a091f9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a09203b80 0x559a091f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4b567740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a09210c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4b567740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559a091ba930 con 0x559a091d6c10 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a090daf40 con 0x559a091e6ca0 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559a091b9d20 con 0x559a09203b80 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4b567740 10 client.0 _ll_register_callbacks cb 0x559a08e32be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca42d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a09210c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca42d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a09210c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47908/0 (socket says 172.21.15.113:47908) 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca42d9d640 1 -- 172.21.15.113:0/4043528038 learned_addr learned my addr 172.21.15.113:0/4043528038 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca41d9b640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a09203b80 0x559a091f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.504+0000 7fca4259c640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6ca0 0x559a091f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca42d9d640 1 -- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a09203b80 msgr2=0x559a091f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca42d9d640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a09203b80 0x559a091f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca42d9d640 1 -- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6ca0 msgr2=0x559a091f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca42d9d640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6ca0 0x559a091f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca42d9d640 1 -- 172.21.15.113:0/4043528038 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a08df30f0 con 0x559a091d6c10 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4259c640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6ca0 0x559a091f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca42d9d640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a09210c40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fca3402ac00 tx=0x7fca34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d747f450106566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca217fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca217fa640 1 -- 172.21.15.113:0/4043528038 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34042070 con 0x559a091d6c10 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca217fa640 1 -- 172.21.15.113:0/4043528038 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fca34031dc0 con 0x559a091d6c10 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca3c05d7c0 con 0x559a091d6c10 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca217fa640 1 -- 172.21.15.113:0/4043528038 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fca3404d050 con 0x559a091d6c10 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 10 client.8670 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:58.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x559a08f5df60 con 0x559a091d6c10 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca217fa640 1 -- 172.21.15.113:0/4043528038 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c05d7c0 con 0x559a091d6c10 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca217fa640 1 -- 172.21.15.113:0/4043528038 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7fca34050020 con 0x559a091d6c10 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 10 client.8670 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 10 client.8670 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 20 client.8670 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091bbe10 0x559a091e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 4 client.8670 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.505+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a0912c870 con 0x559a091bbe10 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.506+0000 7fca4259c640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091bbe10 0x559a091e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.506+0000 7fca4259c640 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091bbe10 0x559a091e7190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d76c9492d57e028 server_cookie=65a2474f6b67fbfa in_seq=0 out_seq=0 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.506+0000 7fca217fa640 10 client.8670 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca217fa640 1 -- 172.21.15.113:0/4043528038 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a0912c870 con 0x559a091bbe10 2024-04-25T00:05:58.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca217fa640 10 client.8670 handle_command_reply: tid=0 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 client.8670 shutdown 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 msgr2=0x559a09210c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a09210c40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fca3402ac00 tx=0x7fca34004a40 comp rx=0 tx=0).stop 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 shutdown_connections 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091bbe10 0x559a091e7190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a09210c40 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a09203b80 0x559a091f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca217fa640 0 client.8670 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 --2- 172.21.15.113:0/4043528038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091e6ca0 0x559a091f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 >> 172.21.15.113:0/4043528038 conn(0x559a091e6560 msgr2=0x559a091b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 shutdown_connections 2024-04-25T00:05:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 1 -- 172.21.15.113:0/4043528038 wait complete. 2024-04-25T00:05:58.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.507+0000 7fca4b567740 20 client.8670 trim_cache size 0 max 16384 2024-04-25T00:05:58.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.524+0000 7fca4b567740 1 Processor -- start 2024-04-25T00:05:58.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.524+0000 7fca4b567740 1 -- start start 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4b567740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a092012b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a09203b80 0x559a091bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4b567740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fca44200900 con 0x559a091d6c10 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca440ce320 con 0x559a09203b80 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca44149560 con 0x559a091e6ca0 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a092012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a092012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47916/0 (socket says 172.21.15.113:47916) 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 -- 172.21.15.113:0/1387235818 learned_addr learned my addr 172.21.15.113:0/1387235818 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca42d9d640 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a09203b80 0x559a091bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca41d9b640 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 -- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 msgr2=0x559a091bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 -- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a09203b80 msgr2=0x559a091bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a09203b80 0x559a091bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 -- 172.21.15.113:0/1387235818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a08dcc1b0 con 0x559a091d6c10 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca4259c640 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a092012b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00d7d0 tx=0x7fca3c05d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ef4294443db541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca237fe640 1 -- 172.21.15.113:0/1387235818 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c07d070 con 0x559a091d6c10 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca237fe640 1 -- 172.21.15.113:0/1387235818 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c05d7c0 con 0x559a091d6c10 2024-04-25T00:05:58.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.525+0000 7fca237fe640 1 -- 172.21.15.113:0/1387235818 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c071040 con 0x559a091d6c10 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 -- 172.21.15.113:0/1387235818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 msgr2=0x559a092012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a092012b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00d7d0 tx=0x7fca3c05d000 comp rx=0 tx=0).stop 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 -- 172.21.15.113:0/1387235818 shutdown_connections 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a09203b80 0x559a091bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 --2- 172.21.15.113:0/1387235818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a091d6c10 0x559a092012b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 -- 172.21.15.113:0/1387235818 >> 172.21.15.113:0/1387235818 conn(0x559a091e6560 msgr2=0x559a091d5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 -- 172.21.15.113:0/1387235818 shutdown_connections 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 -- 172.21.15.113:0/1387235818 wait complete. 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 -- start start 2024-04-25T00:05:58.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.526+0000 7fca4b567740 1 Processor -- start 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.527+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091d6c10 0x559a09279d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.527+0000 7fca4b567740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.527+0000 7fca4b567740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.527+0000 7fca4b567740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559a09210540 con 0x559a09203b80 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.527+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca440cdcd0 con 0x559a091d6c10 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.527+0000 7fca4b567740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559a0927a280 con 0x559a091e6ca0 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.527+0000 7fca4b567740 10 client.0 _ll_register_callbacks cb 0x7fca34034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47922/0 (socket says 172.21.15.113:47922) 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 -- 172.21.15.113:0/2951782907 learned_addr learned my addr 172.21.15.113:0/2951782907 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca41d9b640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca4259c640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091d6c10 0x559a09279d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 -- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 msgr2=0x559a091b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 -- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091d6c10 msgr2=0x559a09279d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091d6c10 0x559a09279d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 -- 172.21.15.113:0/2951782907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a091e1180 con 0x559a09203b80 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca41d9b640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca42d9d640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6600 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x559a091e9e20 tx=0x7fca3402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876f0ae1f8ad21cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca217fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca217fa640 1 -- 172.21.15.113:0/2951782907 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34042070 con 0x559a09203b80 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca3c05d7c0 con 0x559a09203b80 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca217fa640 1 -- 172.21.15.113:0/2951782907 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fca3403c860 con 0x559a09203b80 2024-04-25T00:05:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca217fa640 1 -- 172.21.15.113:0/2951782907 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fca3405d050 con 0x559a09203b80 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca4b567740 10 client.8676 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x559a092247e0 con 0x559a09203b80 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca217fa640 1 -- 172.21.15.113:0/2951782907 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c05d7c0 con 0x559a09203b80 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.528+0000 7fca217fa640 1 -- 172.21.15.113:0/2951782907 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7fca34060020 con 0x559a09203b80 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 10 client.8676 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 10 client.8676 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 20 client.8676 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091f70f0 0x559a091f7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 4 client.8676 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x559a091faec0 con 0x559a091f70f0 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4259c640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091f70f0 0x559a091f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca217fa640 10 client.8676 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4259c640 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091f70f0 0x559a091f7500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6274c74c3a4319 server_cookie=59ddae476a4459f2 in_seq=0 out_seq=0 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca217fa640 1 -- 172.21.15.113:0/2951782907 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559a091faec0 con 0x559a091f70f0 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca217fa640 10 client.8676 handle_command_reply: tid=0 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 client.8676 shutdown 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 msgr2=0x559a091b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6600 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x559a091e9e20 tx=0x7fca3402f000 comp rx=0 tx=0).stop 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 shutdown_connections 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x559a091f70f0 0x559a091f7500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559a09203b80 0x559a091b6600 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559a091e6ca0 0x559a091b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca217fa640 0 client.8676 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 --2- 172.21.15.113:0/2951782907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a091d6c10 0x559a09279d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.529+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 >> 172.21.15.113:0/2951782907 conn(0x559a091e6560 msgr2=0x559a091f9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 shutdown_connections 2024-04-25T00:05:58.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca4b567740 1 -- 172.21.15.113:0/2951782907 wait complete. 2024-04-25T00:05:58.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca4b567740 20 client.8676 trim_cache size 0 max 16384 2024-04-25T00:05:58.538 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:58.538 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:05:58.538 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:58.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca2406be30 msgr2=0x7fca2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca2406be30 0x7fca2406e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fca4415b0e0 tx=0x7fca38008040 comp rx=0 tx=0).stop 2024-04-25T00:05:58.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 msgr2=0x7fca440b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca440b7380 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00d800 tx=0x7fca3c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 shutdown_connections 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca2406be30 0x7fca2406e2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca440d28c0 0x7fca440b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca4415e510 0x7fca440cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 --2- 172.21.15.113:0/3076287693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca440b61e0 0x7fca440b7380 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 >> 172.21.15.113:0/3076287693 conn(0x7fca4406b720 msgr2=0x7fca440b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 shutdown_connections 2024-04-25T00:05:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.530+0000 7fca48858640 1 -- 172.21.15.113:0/3076287693 wait complete. 2024-04-25T00:05:58.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:58.542 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T00:05:58.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.639+0000 7f8754c2a640 1 Processor -- start 2024-04-25T00:05:58.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.640+0000 7f8754c2a640 1 -- start start 2024-04-25T00:05:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.640+0000 7f8754c2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87500b5ca0 0x7f87500b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.640+0000 7f8754c2a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87500d2e00 con 0x7f875015e510 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.640+0000 7f8754c2a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87500d30e0 con 0x7f87500b73c0 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f8754c2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87500b6640 con 0x7f87500b5ca0 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874e59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f875015e510 0x7f87500d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56608/0 (socket says 172.21.15.113:56608) 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874e59c640 1 -- 172.21.15.113:0/2275655391 learned_addr learned my addr 172.21.15.113:0/2275655391 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874dd9b640 1 --2- 172.21.15.113:0/2275655391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87500b5ca0 0x7f87500b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82681566 0 0) 0x7f87500d2e00 con 0x7f875015e510 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8738003520 con 0x7f875015e510 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929201227 0 0) 0x7f87500d30e0 con 0x7f87500b73c0 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87500d2e00 con 0x7f87500b73c0 2024-04-25T00:05:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874dd9b640 1 -- 172.21.15.113:0/2275655391 >> v1:172.21.15.178:6789/0 conn(0x7f87500b73c0 legacy=0x7f87500b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874dd9b640 1 -- 172.21.15.113:0/2275655391 >> v1:172.21.15.113:6789/0 conn(0x7f875015e510 legacy=0x7f87500d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1677561827 0 0) 0x7f8738003520 con 0x7f875015e510 2024-04-25T00:05:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.641+0000 7f874dd9b640 1 -- 172.21.15.113:0/2275655391 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87500d33c0 con 0x7f87500b5ca0 2024-04-25T00:05:58.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.642+0000 7f874dd9b640 1 --2- 172.21.15.113:0/2275655391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87500b5ca0 0x7f87500b6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f87400097d0 tx=0x7f874002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b109d9e539bfcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.642+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740037070 con 0x7f87500b5ca0 2024-04-25T00:05:58.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.642+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874002ab90 con 0x7f87500b5ca0 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.642+0000 7f874d59a640 1 -- 172.21.15.113:0/2275655391 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744001670 con 0x7f87500b5ca0 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.643+0000 7f8754c2a640 1 -- 172.21.15.113:0/2275655391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87500b5ca0 msgr2=0x7f87500b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.643+0000 7f8754c2a640 1 --2- 172.21.15.113:0/2275655391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87500b5ca0 0x7f87500b6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f87400097d0 tx=0x7f874002a100 comp rx=0 tx=0).stop 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.643+0000 7f8754c2a640 1 -- 172.21.15.113:0/2275655391 shutdown_connections 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.643+0000 7f8754c2a640 1 --2- 172.21.15.113:0/2275655391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87500b5ca0 0x7f87500b6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.643+0000 7f8754c2a640 1 -- 172.21.15.113:0/2275655391 >> 172.21.15.113:0/2275655391 conn(0x7f875006b720 msgr2=0x7f87500d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.643+0000 7f8754c2a640 1 -- 172.21.15.113:0/2275655391 shutdown_connections 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.643+0000 7f8754c2a640 1 -- 172.21.15.113:0/2275655391 wait complete. 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.644+0000 7f8754c2a640 1 Processor -- start 2024-04-25T00:05:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.644+0000 7f8754c2a640 1 -- start start 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f8754c2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87500b5ca0 0x7f8750150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f8754c2a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87500b73c0 0x7f87500c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f8754c2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875015e510 0x7f875015f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f8754c2a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87500d3ea0 con 0x7f87500b73c0 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f8754c2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87500d3980 con 0x7f87500b5ca0 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f8754c2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87500d2a90 con 0x7f875015e510 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87500b73c0 0x7f87500c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87500b73c0 0x7f87500c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47924/0 (socket says 172.21.15.113:47924) 2024-04-25T00:05:58.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 -- 172.21.15.113:0/3456934951 learned_addr learned my addr 172.21.15.113:0/3456934951 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874e59c640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87500b5ca0 0x7f8750150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874ed9d640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875015e510 0x7f875015f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 -- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875015e510 msgr2=0x7f875015f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875015e510 0x7f875015f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 -- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87500b5ca0 msgr2=0x7f8750150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87500b5ca0 0x7f8750150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 -- 172.21.15.113:0/3456934951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8750160540 con 0x7f87500b73c0 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874e59c640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87500b5ca0 0x7f8750150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f874dd9b640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87500b73c0 0x7f87500c7d50 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f8740035040 tx=0x7f87400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f873effd640 1 -- 172.21.15.113:0/3456934951 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740041020 con 0x7f87500b73c0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f873effd640 1 -- 172.21.15.113:0/3456934951 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874003d640 con 0x7f87500b73c0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f873effd640 1 -- 172.21.15.113:0/3456934951 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740037040 con 0x7f87500b73c0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.645+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8750160820 con 0x7f87500b73c0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.646+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8750161260 con 0x7f87500b73c0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.647+0000 7f873effd640 1 -- 172.21.15.113:0/3456934951 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8740044020 con 0x7f87500b73c0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.647+0000 7f873effd640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873006be30 0x7f873006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.647+0000 7f873effd640 1 -- 172.21.15.113:0/3456934951 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f87400ab3f0 con 0x7f87500b73c0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.647+0000 7f874e59c640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873006be30 0x7f873006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.648+0000 7f874e59c640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873006be30 0x7f873006e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f87440011e0 tx=0x7f8744009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.648+0000 7f8757a14740 1 Processor -- start 2024-04-25T00:05:58.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.648+0000 7f8757a14740 1 -- start start 2024-04-25T00:05:58.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f8757a14740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x5574821bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482177a40 0x5574821bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f8757a14740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5574820db6b0 con 0x5574821a1d60 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557482083880 con 0x557482177a40 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5574820d7670 con 0x5574821bf600 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47932/0 (socket says 172.21.15.113:47932) 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 -- 172.21.15.113:0/2132211776 learned_addr learned my addr 172.21.15.113:0/2132211776 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 -- 172.21.15.113:0/2132211776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 msgr2=0x5574821bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 --2- 172.21.15.113:0/2132211776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x5574821bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 -- 172.21.15.113:0/2132211776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482177a40 msgr2=0x5574821bdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 --2- 172.21.15.113:0/2132211776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482177a40 0x5574821bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.649+0000 7f874e59c640 1 -- 172.21.15.113:0/2132211776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557481eb97d0 con 0x5574821a1d60 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f874e59c640 1 --2- 172.21.15.113:0/2132211776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821a2140 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f8744005a70 tx=0x7f8744011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63df36af97834beb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f872ffff640 1 -- 172.21.15.113:0/2132211776 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744007710 con 0x5574821a1d60 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f872ffff640 1 -- 172.21.15.113:0/2132211776 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874400e9c0 con 0x5574821a1d60 2024-04-25T00:05:58.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f872ffff640 1 -- 172.21.15.113:0/2132211776 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874400ed70 con 0x5574821a1d60 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 -- 172.21.15.113:0/2132211776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 msgr2=0x5574821a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 --2- 172.21.15.113:0/2132211776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821a2140 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f8744005a70 tx=0x7f8744011000 comp rx=0 tx=0).stop 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 -- 172.21.15.113:0/2132211776 shutdown_connections 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 --2- 172.21.15.113:0/2132211776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482177a40 0x5574821bdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 --2- 172.21.15.113:0/2132211776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x5574821bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 --2- 172.21.15.113:0/2132211776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821a2140 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 -- 172.21.15.113:0/2132211776 >> 172.21.15.113:0/2132211776 conn(0x557482174600 msgr2=0x5574821bcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 -- 172.21.15.113:0/2132211776 shutdown_connections 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.650+0000 7f8757a14740 1 -- 172.21.15.113:0/2132211776 wait complete. 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.651+0000 7f8757a14740 1 -- start start 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.651+0000 7f8757a14740 1 Processor -- start 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f8757a14740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x5574821cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482191730 0x5574821a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f8757a14740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5574821787e0 con 0x5574821a1d60 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557482096f50 con 0x557482191730 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557482172530 con 0x5574821bf600 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f8757a14740 10 client.0 _ll_register_callbacks cb 0x5574820d2000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47938/0 (socket says 172.21.15.113:47938) 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 -- 172.21.15.113:0/889164555 learned_addr learned my addr 172.21.15.113:0/889164555 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 -- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 msgr2=0x5574821cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x5574821cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 -- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482191730 msgr2=0x5574821a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482191730 0x5574821a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 -- 172.21.15.113:0/889164555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557481def3f0 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f874e59c640 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821b3c20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f87440074a0 tx=0x7f8744013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6b347c7cda99f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.652+0000 7f872d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f872d7fa640 1 -- 172.21.15.113:0/889164555 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744021860 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f872d7fa640 1 -- 172.21.15.113:0/889164555 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f8744021a00 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f874400e9c0 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f872d7fa640 1 -- 172.21.15.113:0/889164555 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8744032050 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 10 client.8688 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x557481daf0f0 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f872d7fa640 1 -- 172.21.15.113:0/889164555 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874400e9c0 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f872d7fa640 1 -- 172.21.15.113:0/889164555 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f8744035020 con 0x5574821a1d60 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 10 client.8688 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 10 client.8688 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 20 client.8688 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55748219f990 0x5574821a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 4 client.8688 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574820e8bf0 con 0x55748219f990 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.653+0000 7f874dd9b640 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55748219f990 0x5574821a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.654+0000 7f872d7fa640 10 client.8688 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.654+0000 7f874dd9b640 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55748219f990 0x5574821a05d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309d67609966f88f server_cookie=c873d54837a2dcea in_seq=0 out_seq=0 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.654+0000 7f872d7fa640 1 -- 172.21.15.113:0/889164555 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574820e8bf0 con 0x55748219f990 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.654+0000 7f872d7fa640 10 client.8688 handle_command_reply: tid=0 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.654+0000 7f8757a14740 1 client.8688 shutdown 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.654+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 msgr2=0x5574821b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.654+0000 7f8757a14740 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821b3c20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f87440074a0 tx=0x7f8744013000 comp rx=0 tx=0).stop 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 shutdown_connections 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55748219f990 0x5574821a05d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557482191730 0x5574821a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x5574821cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f872d7fa640 0 client.8688 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 --2- 172.21.15.113:0/889164555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x5574821b3c20 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 >> 172.21.15.113:0/889164555 conn(0x557482177a40 msgr2=0x5574821c00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 shutdown_connections 2024-04-25T00:05:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 1 -- 172.21.15.113:0/889164555 wait complete. 2024-04-25T00:05:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.655+0000 7f8757a14740 20 client.8688 trim_cache size 0 max 16384 2024-04-25T00:05:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.672+0000 7f8757a14740 1 Processor -- start 2024-04-25T00:05:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.672+0000 7f8757a14740 1 -- start start 2024-04-25T00:05:58.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557482191730 0x557482175910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f8757a14740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x557482175e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821bf600 0x5574821ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f8757a14740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87502007d0 con 0x5574821a1d60 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87501ff860 con 0x5574821bf600 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f87501582d0 con 0x557482191730 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x557482175e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x557482175e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47956/0 (socket says 172.21.15.113:47956) 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 -- 172.21.15.113:0/3399197578 learned_addr learned my addr 172.21.15.113:0/3399197578 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874ed9d640 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821bf600 0x5574821ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874e59c640 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557482191730 0x557482175910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 -- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557482191730 msgr2=0x557482175910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557482191730 0x557482175910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 -- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821bf600 msgr2=0x5574821ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821bf600 0x5574821ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 -- 172.21.15.113:0/3399197578 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574821bbf70 con 0x5574821a1d60 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874e59c640 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557482191730 0x557482175910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f874dd9b640 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x557482175e50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f87400793a0 tx=0x7f8740048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1073f1441622a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f872f7fe640 1 -- 172.21.15.113:0/3399197578 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740079da0 con 0x5574821a1d60 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f872f7fe640 1 -- 172.21.15.113:0/3399197578 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87400796a0 con 0x5574821a1d60 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.673+0000 7f872f7fe640 1 -- 172.21.15.113:0/3399197578 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874007da50 con 0x5574821a1d60 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 -- 172.21.15.113:0/3399197578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 msgr2=0x557482175e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x557482175e50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f87400793a0 tx=0x7f8740048040 comp rx=0 tx=0).stop 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 -- 172.21.15.113:0/3399197578 shutdown_connections 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821bf600 0x5574821ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5574821a1d60 0x557482175e50 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 --2- 172.21.15.113:0/3399197578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557482191730 0x557482175910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 -- 172.21.15.113:0/3399197578 >> 172.21.15.113:0/3399197578 conn(0x557482177a40 msgr2=0x557482190c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 -- 172.21.15.113:0/3399197578 shutdown_connections 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 -- 172.21.15.113:0/3399197578 wait complete. 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 -- start start 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.674+0000 7f8757a14740 1 Processor -- start 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f8757a14740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557482191730 0x5574821d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821a1d60 0x55748219e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f8757a14740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x55748219e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f8757a14740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5574821a3a30 con 0x557482191730 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87500ca6e0 con 0x5574821a1d60 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f8757a14740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5574821a0010 con 0x5574821bf600 2024-04-25T00:05:58.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f8757a14740 10 client.0 _ll_register_callbacks cb 0x7f8744005970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f874e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557482191730 0x5574821d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f874e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557482191730 0x5574821d6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47962/0 (socket says 172.21.15.113:47962) 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.675+0000 7f874e59c640 1 -- 172.21.15.113:0/1707650562 learned_addr learned my addr 172.21.15.113:0/1707650562 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874dd9b640 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821a1d60 0x55748219e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874ed9d640 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x55748219e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874e59c640 1 -- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 msgr2=0x55748219e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874e59c640 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x55748219e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874e59c640 1 -- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821a1d60 msgr2=0x55748219e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874e59c640 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821a1d60 0x55748219e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874e59c640 1 -- 172.21.15.113:0/1707650562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557482179070 con 0x557482191730 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874e59c640 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557482191730 0x5574821d6e00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f874402bb00 tx=0x7f8744000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f195e9ab8c0746c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f872d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f872d7fa640 1 -- 172.21.15.113:0/1707650562 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874400ebc0 con 0x557482191730 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f872d7fa640 1 -- 172.21.15.113:0/1707650562 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f874400ed60 con 0x557482191730 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87400796a0 con 0x557482191730 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f872d7fa640 1 -- 172.21.15.113:0/1707650562 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f874400fd90 con 0x557482191730 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 10 client.8694 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x557482190420 con 0x557482191730 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f872d7fa640 1 -- 172.21.15.113:0/1707650562 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87400796a0 con 0x557482191730 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f872d7fa640 1 -- 172.21.15.113:0/1707650562 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f874403e050 con 0x557482191730 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 10 client.8694 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 10 client.8694 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 20 client.8694 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x557482236140 0x5574821e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 4 client.8694 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574820e2f40 con 0x557482236140 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.676+0000 7f874dd9b640 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x557482236140 0x5574821e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f872d7fa640 10 client.8694 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f874dd9b640 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x557482236140 0x5574821e0410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=715d1c766a7c9a48 server_cookie=ef010e8320bf0229 in_seq=0 out_seq=0 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f872d7fa640 1 -- 172.21.15.113:0/1707650562 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+69 (crc 0 0 0) 0x5574820e2f40 con 0x557482236140 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f872d7fa640 10 client.8694 handle_command_reply: tid=0 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 client.8694 shutdown 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557482191730 msgr2=0x5574821d6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557482191730 0x5574821d6e00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f874402bb00 tx=0x7f8744000bc0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 shutdown_connections 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x557482236140 0x5574821e0410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5574821a1d60 0x55748219e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5574821bf600 0x55748219e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f872d7fa640 0 client.8694 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 --2- 172.21.15.113:0/1707650562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557482191730 0x5574821d6e00 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 >> 172.21.15.113:0/1707650562 conn(0x557482177a40 msgr2=0x5574821a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 shutdown_connections 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 1 -- 172.21.15.113:0/1707650562 wait complete. 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.677+0000 7f8757a14740 20 client.8694 trim_cache size 0 max 16384 2024-04-25T00:05:58.686 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:58.687 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T00:05:58.687 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:05:58.687 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:58.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873006be30 msgr2=0x7f873006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873006be30 0x7f873006e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f87440011e0 tx=0x7f8744009040 comp rx=0 tx=0).stop 2024-04-25T00:05:58.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87500b73c0 msgr2=0x7f87500c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87500b73c0 0x7f87500c7d50 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f8740035040 tx=0x7f87400046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 shutdown_connections 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f873006be30 0x7f873006e2f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f875015e510 0x7f875015f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87500b73c0 0x7f87500c7d50 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 --2- 172.21.15.113:0/3456934951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87500b5ca0 0x7f8750150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 >> 172.21.15.113:0/3456934951 conn(0x7f875006b720 msgr2=0x7f875015d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 shutdown_connections 2024-04-25T00:05:58.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.678+0000 7f8754c2a640 1 -- 172.21.15.113:0/3456934951 wait complete. 2024-04-25T00:05:58.692 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:58.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.788+0000 7f8d3811d640 1 Processor -- start 2024-04-25T00:05:58.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.788+0000 7f8d3811d640 1 -- start start 2024-04-25T00:05:58.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d3811d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8d3015e320 0x7f8d300d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d3811d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d300d0b00 0x7f8d300d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d3811d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d300d3de0 con 0x7f8d300d2290 2024-04-25T00:05:58.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d3811d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8d300d14b0 con 0x7f8d300d0b00 2024-04-25T00:05:58.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d3811d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8d300d3410 con 0x7f8d3015e320 2024-04-25T00:05:58.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d366ba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d300d0b00 0x7f8d300d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d356b8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8d300d2290 0x7f8d300b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56622/0 (socket says 172.21.15.113:56622) 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d366ba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d300d0b00 0x7f8d300d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47974/0 (socket says 172.21.15.113:47974) 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d356b8640 1 -- 172.21.15.113:0/3495598758 learned_addr learned my addr 172.21.15.113:0/3495598758 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.789+0000 7f8d35eb9640 1 --2- 172.21.15.113:0/3495598758 >> v2:172.21.15.178:3300/0 conn(0x7f8d3015e320 0x7f8d300d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d366ba640 1 -- 172.21.15.113:0/3495598758 >> v2:172.21.15.178:3300/0 conn(0x7f8d3015e320 msgr2=0x7f8d300d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d366ba640 1 --2- 172.21.15.113:0/3495598758 >> v2:172.21.15.178:3300/0 conn(0x7f8d3015e320 0x7f8d300d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d366ba640 1 -- 172.21.15.113:0/3495598758 >> v1:172.21.15.113:6789/0 conn(0x7f8d300d2290 legacy=0x7f8d300b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d366ba640 1 -- 172.21.15.113:0/3495598758 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d300d2800 con 0x7f8d300d0b00 2024-04-25T00:05:58.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d366ba640 1 --2- 172.21.15.113:0/3495598758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300d0b00 0x7f8d300d0f70 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f8d300b6d00 tx=0x7f8d2802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711ac23a6321d0ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d34eb7640 1 -- 172.21.15.113:0/3495598758 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28037070 con 0x7f8d300d0b00 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d34eb7640 1 -- 172.21.15.113:0/3495598758 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2802adc0 con 0x7f8d300d0b00 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.790+0000 7f8d34eb7640 1 -- 172.21.15.113:0/3495598758 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28032660 con 0x7f8d300d0b00 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 -- 172.21.15.113:0/3495598758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300d0b00 msgr2=0x7f8d300d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3495598758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300d0b00 0x7f8d300d0f70 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f8d300b6d00 tx=0x7f8d2802a130 comp rx=0 tx=0).stop 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 -- 172.21.15.113:0/3495598758 shutdown_connections 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3495598758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300d0b00 0x7f8d300d0f70 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3495598758 >> v2:172.21.15.178:3300/0 conn(0x7f8d3015e320 0x7f8d300d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 -- 172.21.15.113:0/3495598758 >> 172.21.15.113:0/3495598758 conn(0x7f8d300b5c20 msgr2=0x7f8d300b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 -- 172.21.15.113:0/3495598758 shutdown_connections 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.791+0000 7f8d3811d640 1 -- 172.21.15.113:0/3495598758 wait complete. 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.792+0000 7f8d3811d640 1 Processor -- start 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.792+0000 7f8d3811d640 1 -- start start 2024-04-25T00:05:58.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d3811d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d300d0b00 0x7f8d300c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d3811d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d300d2290 0x7f8d3015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d3811d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3015e320 0x7f8d3015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d3811d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d300d30a0 con 0x7f8d3015e320 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d3811d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d300d2dc0 con 0x7f8d300d0b00 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d3811d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3015e700 con 0x7f8d300d2290 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3015e320 0x7f8d3015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3015e320 0x7f8d3015ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47990/0 (socket says 172.21.15.113:47990) 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 -- 172.21.15.113:0/3202949190 learned_addr learned my addr 172.21.15.113:0/3202949190 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d35eb9640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d300d0b00 0x7f8d300c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d356b8640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d300d2290 0x7f8d3015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 -- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d300d2290 msgr2=0x7f8d3015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d300d2290 0x7f8d3015ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 -- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d300d0b00 msgr2=0x7f8d300c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d300d0b00 0x7f8d300c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 -- 172.21.15.113:0/3202949190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d300ca1d0 con 0x7f8d3015e320 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.793+0000 7f8d366ba640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3015e320 0x7f8d3015ef80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8d28035040 tx=0x7f8d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.794+0000 7f8d267fc640 1 -- 172.21.15.113:0/3202949190 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28037040 con 0x7f8d3015e320 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.794+0000 7f8d267fc640 1 -- 172.21.15.113:0/3202949190 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d28004030 con 0x7f8d3015e320 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.794+0000 7f8d267fc640 1 -- 172.21.15.113:0/3202949190 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28004420 con 0x7f8d3015e320 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.794+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d30154b60 con 0x7f8d3015e320 2024-04-25T00:05:58.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.794+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d30155120 con 0x7f8d3015e320 2024-04-25T00:05:58.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.795+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d300691a0 con 0x7f8d3015e320 2024-04-25T00:05:58.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.795+0000 7f8d267fc640 1 -- 172.21.15.113:0/3202949190 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d28003610 con 0x7f8d3015e320 2024-04-25T00:05:58.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.795+0000 7f8d267fc640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d1406be30 0x7f8d1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:58.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.795+0000 7f8d267fc640 1 -- 172.21.15.113:0/3202949190 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f8d280ab820 con 0x7f8d3015e320 2024-04-25T00:05:58.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.795+0000 7f8d35eb9640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d1406be30 0x7f8d1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:58.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.796+0000 7f8d35eb9640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d1406be30 0x7f8d1406e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0099c0 tx=0x7f8d2c003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:58.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:58.800+0000 7f8d267fc640 1 -- 172.21.15.113:0/3202949190 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d300ca1d0 con 0x7f8d3015e320 2024-04-25T00:05:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.008+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d300ca5c0 con 0x7f8d3015e320 2024-04-25T00:05:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.008+0000 7f8d267fc640 1 -- 172.21.15.113:0/3202949190 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f8d2807a2f0 con 0x7f8d3015e320 2024-04-25T00:05:59.009 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:59.010 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:05:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:05:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.010+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d1406be30 msgr2=0x7f8d1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.010+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d1406be30 0x7f8d1406e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0099c0 tx=0x7f8d2c003990 comp rx=0 tx=0).stop 2024-04-25T00:05:59.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.010+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3015e320 msgr2=0x7f8d3015ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.010+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3015e320 0x7f8d3015ef80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8d28035040 tx=0x7f8d280046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 shutdown_connections 2024-04-25T00:05:59.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d1406be30 0x7f8d1406e2f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d3015e320 0x7f8d3015ef80 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d300d2290 0x7f8d3015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 --2- 172.21.15.113:0/3202949190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d300d0b00 0x7f8d300c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 >> 172.21.15.113:0/3202949190 conn(0x7f8d300b5c20 msgr2=0x7f8d300b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 shutdown_connections 2024-04-25T00:05:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.011+0000 7f8d3811d640 1 -- 172.21.15.113:0/3202949190 wait complete. 2024-04-25T00:05:59.022 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:05:59.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:05:59.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.119+0000 7fbf4d2bd640 1 Processor -- start 2024-04-25T00:05:59.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.119+0000 7fbf4d2bd640 1 -- start start 2024-04-25T00:05:59.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf4d2bd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbf48144910 0x7fbf480b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf4d2bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 0x7fbf480cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf4d2bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf48144cf0 con 0x7fbf480b5240 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf4d2bd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbf480b4510 con 0x7fbf48144910 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf4d2bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf480b2f30 con 0x7fbf480b56b0 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf3ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbf480b5240 0x7fbf4806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56636/0 (socket says 172.21.15.113:56636) 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf3ffff640 1 -- 172.21.15.113:0/780123586 learned_addr learned my addr 172.21.15.113:0/780123586 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/780123586 >> v2:172.21.15.178:3300/0 conn(0x7fbf48144910 0x7fbf480b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 40975113 0 0) 0x7fbf48144cf0 con 0x7fbf480b5240 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf477fe640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 0x7fbf480cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf34005730 con 0x7fbf480b5240 2024-04-25T00:05:59.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1528758790 0 0) 0x7fbf34005730 con 0x7fbf480b5240 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 msgr2=0x7fbf480cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 0x7fbf480cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 >> v2:172.21.15.178:3300/0 conn(0x7fbf48144910 msgr2=0x7fbf480b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 --2- 172.21.15.113:0/780123586 >> v2:172.21.15.178:3300/0 conn(0x7fbf48144910 0x7fbf480b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf480b3210 con 0x7fbf480b5240 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fbf30001670 con 0x7fbf480b5240 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 >> v1:172.21.15.113:6789/0 conn(0x7fbf480b5240 legacy=0x7fbf4806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf34005bc0 0x7fbf34005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 0x7fbf34006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34006f50 0x7fbf340073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf480b4510 con 0x7fbf34006f50 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf480b2f30 con 0x7fbf34005bc0 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3400dbd0 con 0x7fbf480b56b0 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/780123586 >> v2:172.21.15.178:3300/0 conn(0x7fbf48144910 0x7fbf480b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.120+0000 7fbf477fe640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34006f50 0x7fbf340073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf3ffff640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 0x7fbf34006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf34005bc0 0x7fbf34005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf477fe640 1 -- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 msgr2=0x7fbf34006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf477fe640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf480b56b0 0x7fbf34006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf477fe640 1 -- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf34005bc0 msgr2=0x7fbf34005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf477fe640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf34005bc0 0x7fbf34005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf477fe640 1 -- 172.21.15.113:0/780123586 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf34005730 con 0x7fbf34006f50 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf34005bc0 0x7fbf34005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.121+0000 7fbf477fe640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34006f50 0x7fbf340073d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009f90 tx=0x7fbf34007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8519c54ea96c8843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40009c10 con 0x7fbf34006f50 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf40036070 con 0x7fbf34006f50 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf467fc640 1 -- 172.21.15.113:0/780123586 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40031850 con 0x7fbf34006f50 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/780123586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34006f50 msgr2=0x7fbf340073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf4d2bd640 1 --2- 172.21.15.113:0/780123586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34006f50 0x7fbf340073d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009f90 tx=0x7fbf34007a40 comp rx=0 tx=0).stop 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf477fe640 1 -- 172.21.15.113:0/780123586 reap_dead start 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/780123586 shutdown_connections 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/780123586 >> 172.21.15.113:0/780123586 conn(0x7fbf480b1a50 msgr2=0x7fbf480b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/780123586 shutdown_connections 2024-04-25T00:05:59.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.122+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/780123586 wait complete. 2024-04-25T00:05:59.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.123+0000 7fbf4d2bd640 1 Processor -- start 2024-04-25T00:05:59.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.123+0000 7fbf4d2bd640 1 -- start start 2024-04-25T00:05:59.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf4d2bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf481430a0 0x7fbf480adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf4d2bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf481f5910 0x7fbf481f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf4d2bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf481f5470 0x7fbf481f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf4d2bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf480b4da0 con 0x7fbf481f5910 2024-04-25T00:05:59.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf4d2bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf480b37d0 con 0x7fbf481f5470 2024-04-25T00:05:59.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf4d2bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf480b4f20 con 0x7fbf481430a0 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf481f5910 0x7fbf481f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf481f5910 0x7fbf481f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48014/0 (socket says 172.21.15.113:48014) 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 -- 172.21.15.113:0/2089502029 learned_addr learned my addr 172.21.15.113:0/2089502029 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf477fe640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf481f5470 0x7fbf481f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf481430a0 0x7fbf480adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 -- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf481430a0 msgr2=0x7fbf480adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf481430a0 0x7fbf480adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 -- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf481f5470 msgr2=0x7fbf481f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf481f5470 0x7fbf481f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf3ffff640 1 -- 172.21.15.113:0/2089502029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf481385b0 con 0x7fbf481f5910 2024-04-25T00:05:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.124+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf481430a0 0x7fbf480adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.125+0000 7fbf3ffff640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf481f5910 0x7fbf481f7180 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbf300072c0 tx=0x7fbf300076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.125+0000 7fbf3f7fe640 1 -- 172.21.15.113:0/2089502029 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000f9a0 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.125+0000 7fbf3f7fe640 1 -- 172.21.15.113:0/2089502029 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf30018070 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.125+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf48138850 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.125+0000 7fbf3f7fe640 1 -- 172.21.15.113:0/2089502029 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000fd80 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.125+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf481390f0 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.126+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf48138c40 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.126+0000 7fbf3f7fe640 1 -- 172.21.15.113:0/2089502029 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf3001d460 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.126+0000 7fbf3f7fe640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf2806be30 0x7fbf2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.126+0000 7fbf3f7fe640 1 -- 172.21.15.113:0/2089502029 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fbf3008b030 con 0x7fbf481f5910 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.126+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf2806be30 0x7fbf2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.127+0000 7fbf46ffd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf2806be30 0x7fbf2806e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf480b13f0 tx=0x7fbf38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.131+0000 7fbf3f7fe640 1 -- 172.21.15.113:0/2089502029 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf300599e0 con 0x7fbf481f5910 2024-04-25T00:05:59.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.373+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbf480b2750 con 0x7fbf481f5910 2024-04-25T00:05:59.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.373+0000 7fbf3f7fe640 1 -- 172.21.15.113:0/2089502029 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7fbf480b2750 con 0x7fbf481f5910 2024-04-25T00:05:59.374 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:05:59.374 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":484,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561799,"bytes_used":2547630080,"bytes_avail":770546483200,"bytes_total":773094113280,"write_bytes_sec":52187754,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T00:05:57.200778+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf2806be30 msgr2=0x7fbf2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf2806be30 0x7fbf2806e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf480b13f0 tx=0x7fbf38008040 comp rx=0 tx=0).stop 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf481f5910 msgr2=0x7fbf481f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf481f5910 0x7fbf481f7180 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbf300072c0 tx=0x7fbf300076c0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 shutdown_connections 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf2806be30 0x7fbf2806e2f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf481f5470 0x7fbf481f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf481f5910 0x7fbf481f7180 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 --2- 172.21.15.113:0/2089502029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf481430a0 0x7fbf480adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 >> 172.21.15.113:0/2089502029 conn(0x7fbf480b1a50 msgr2=0x7fbf480b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 shutdown_connections 2024-04-25T00:05:59.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.375+0000 7fbf4d2bd640 1 -- 172.21.15.113:0/2089502029 wait complete. 2024-04-25T00:05:59.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T00:05:59.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:05:59.387 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T00:05:59.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.485+0000 7f5b32dab640 1 Processor -- start 2024-04-25T00:05:59.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.485+0000 7f5b32dab640 1 -- start start 2024-04-25T00:05:59.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b32dab640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5b2c13f050 0x7f5b2c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b32dab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c0d04f0 0x7f5b2c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b32dab640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b2c15d6b0 con 0x7f5b2c1604d0 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b32dab640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5b2c0d09a0 con 0x7f5b2c13f050 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b32dab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c15c6a0 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b30b47640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5b2c13f050 0x7f5b2c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c0d04f0 0x7f5b2c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b31348640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5b2c1604d0 0x7f5b2c06cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:46212/0 (socket says 172.21.15.113:46212) 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b31348640 1 -- 172.21.15.113:0/287333049 learned_addr learned my addr 172.21.15.113:0/287333049 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b30b47640 1 -- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c0d04f0 msgr2=0x7f5b2c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b30b47640 1 --2- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c0d04f0 0x7f5b2c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b30b47640 1 -- 172.21.15.113:0/287333049 >> v1:172.21.15.178:6789/0 conn(0x7f5b2c1604d0 legacy=0x7f5b2c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.486+0000 7f5b30b47640 1 -- 172.21.15.113:0/287333049 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c15c8f0 con 0x7f5b2c13f050 2024-04-25T00:05:59.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.487+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c0d04f0 0x7f5b2c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:59.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.487+0000 7f5b30b47640 1 --2- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c13f050 0x7f5b2c0cffb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b200097a0 tx=0x7f5b2002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11028709b5d53858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.487+0000 7f5b2b7fe640 1 -- 172.21.15.113:0/287333049 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20037070 con 0x7f5b2c13f050 2024-04-25T00:05:59.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.487+0000 7f5b2b7fe640 1 -- 172.21.15.113:0/287333049 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2002ae50 con 0x7f5b2c13f050 2024-04-25T00:05:59.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.487+0000 7f5b2b7fe640 1 -- 172.21.15.113:0/287333049 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20032660 con 0x7f5b2c13f050 2024-04-25T00:05:59.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 -- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c13f050 msgr2=0x7f5b2c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 --2- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c13f050 0x7f5b2c0cffb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b200097a0 tx=0x7f5b2002a100 comp rx=0 tx=0).stop 2024-04-25T00:05:59.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 -- 172.21.15.113:0/287333049 shutdown_connections 2024-04-25T00:05:59.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 --2- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c0d04f0 0x7f5b2c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 --2- 172.21.15.113:0/287333049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c13f050 0x7f5b2c0cffb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 -- 172.21.15.113:0/287333049 >> 172.21.15.113:0/287333049 conn(0x7f5b2c06b0f0 msgr2=0x7f5b2c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 -- 172.21.15.113:0/287333049 shutdown_connections 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.488+0000 7f5b32dab640 1 -- 172.21.15.113:0/287333049 wait complete. 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.489+0000 7f5b32dab640 1 Processor -- start 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.489+0000 7f5b32dab640 1 -- start start 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b32dab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c0d04f0 0x7f5b2c1f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b32dab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c13f050 0x7f5b2c0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b32dab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c1604d0 0x7f5b2c15efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b32dab640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c15d190 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b32dab640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c15ceb0 con 0x7f5b2c1604d0 2024-04-25T00:05:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b32dab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c13f430 con 0x7f5b2c13f050 2024-04-25T00:05:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c0d04f0 0x7f5b2c1f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c0d04f0 0x7f5b2c1f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48028/0 (socket says 172.21.15.113:48028) 2024-04-25T00:05:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 -- 172.21.15.113:0/620407304 learned_addr learned my addr 172.21.15.113:0/620407304 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 -- 172.21.15.113:0/620407304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c13f050 msgr2=0x7f5b2c0b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c13f050 0x7f5b2c0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 -- 172.21.15.113:0/620407304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c1604d0 msgr2=0x7f5b2c15efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c1604d0 0x7f5b2c15efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.490+0000 7f5b30b47640 1 -- 172.21.15.113:0/620407304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c12c420 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.491+0000 7f5b30b47640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c0d04f0 0x7f5b2c1f25a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5b20035040 tx=0x7f5b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.491+0000 7f5b297fa640 1 -- 172.21.15.113:0/620407304 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20041070 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.491+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c0b1500 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.491+0000 7f5b297fa640 1 -- 172.21.15.113:0/620407304 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2003c460 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.491+0000 7f5b297fa640 1 -- 172.21.15.113:0/620407304 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20037040 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.491+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c12c6e0 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.492+0000 7f5b297fa640 1 -- 172.21.15.113:0/620407304 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b2003a070 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.493+0000 7f5b297fa640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b1006be30 0x7f5b1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.493+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b1006be30 0x7f5b1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.493+0000 7f5b297fa640 1 -- 172.21.15.113:0/620407304 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f5b200ab4b0 con 0x7f5b2c0d04f0 2024-04-25T00:05:59.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.493+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b1006be30 0x7f5b1006e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5b18006e80 tx=0x7f5b18006dd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 Processor -- start 2024-04-25T00:05:59.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 -- start start 2024-04-25T00:05:59.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f6c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f89b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f41a40 0x55ef03f87cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ef03ea56b0 con 0x55ef03f89600 2024-04-25T00:05:59.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef03e4d880 con 0x55ef03f41a40 2024-04-25T00:05:59.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ef03ea1670 con 0x55ef03f6bd60 2024-04-25T00:05:59.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f89b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f89b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48044/0 (socket says 172.21.15.113:48044) 2024-04-25T00:05:59.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.494+0000 7f5b2bfff640 1 -- 172.21.15.113:0/4009415017 learned_addr learned my addr 172.21.15.113:0/4009415017 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b30b47640 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f6c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b31348640 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f41a40 0x55ef03f87cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b2bfff640 1 -- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 msgr2=0x55ef03f6c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f6c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b2bfff640 1 -- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f41a40 msgr2=0x55ef03f87cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f41a40 0x55ef03f87cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b2bfff640 1 -- 172.21.15.113:0/4009415017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef03c837d0 con 0x55ef03f89600 2024-04-25T00:05:59.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f89b50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009fd0 tx=0x7f5b18011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d5dc9333b3525e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b167fc640 1 -- 172.21.15.113:0/4009415017 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1801b040 con 0x55ef03f89600 2024-04-25T00:05:59.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b167fc640 1 -- 172.21.15.113:0/4009415017 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b18024070 con 0x55ef03f89600 2024-04-25T00:05:59.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.495+0000 7f5b167fc640 1 -- 172.21.15.113:0/4009415017 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18015d00 con 0x55ef03f89600 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 -- 172.21.15.113:0/4009415017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 msgr2=0x55ef03f89b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f89b50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009fd0 tx=0x7f5b18011000 comp rx=0 tx=0).stop 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 -- 172.21.15.113:0/4009415017 shutdown_connections 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f41a40 0x55ef03f87cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f89b50 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/4009415017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f6c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 -- 172.21.15.113:0/4009415017 >> 172.21.15.113:0/4009415017 conn(0x55ef03f3e600 msgr2=0x55ef03f86b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 -- 172.21.15.113:0/4009415017 shutdown_connections 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 -- 172.21.15.113:0/4009415017 wait complete. 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 -- start start 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.496+0000 7f5b35b8a740 1 Processor -- start 2024-04-25T00:05:59.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f68690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f6f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f3b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ef03f427e0 con 0x55ef03f89600 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef03e60f50 con 0x55ef03f3b260 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ef03f3c530 con 0x55ef03f6bd60 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b35b8a740 10 client.0 _ll_register_callbacks cb 0x55ef03e9c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f6f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f6f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48062/0 (socket says 172.21.15.113:48062) 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1527866622 learned_addr learned my addr 172.21.15.113:0/1527866622 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f68690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b31348640 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f3b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 msgr2=0x55ef03f68690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.497+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f68690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 msgr2=0x55ef03f3b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f3b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1527866622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef03bb93f0 con 0x55ef03f89600 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f6f9d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5b18020000 tx=0x7f5b18009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c9673d9a0da189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1527866622 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18029810 con 0x55ef03f89600 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1527866622 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f5b180299b0 con 0x55ef03f89600 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b18024070 con 0x55ef03f89600 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1527866622 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f5b18029bf0 con 0x55ef03f89600 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 10 client.8718 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55ef03b790f0 con 0x55ef03f89600 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1527866622 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b18024070 con 0x55ef03f89600 2024-04-25T00:05:59.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1527866622 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f5b1801e030 con 0x55ef03f89600 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 10 client.8718 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 10 client.8718 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 20 client.8718 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f7e540 0x55ef03f7e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 4 client.8718 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef03eb2bf0 con 0x55ef03f7e540 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.498+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f7e540 0x55ef03f7e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.499+0000 7f5b0b7fe640 10 client.8718 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.499+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f7e540 0x55ef03f7e950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5101652f215a37f server_cookie=15d448dcf0a67eed in_seq=0 out_seq=0 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.499+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1527866622 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef03eb2bf0 con 0x55ef03f7e540 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.499+0000 7f5b0b7fe640 10 client.8718 handle_command_reply: tid=0 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.499+0000 7f5b35b8a740 1 client.8718 shutdown 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 msgr2=0x55ef03f6f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f6f9d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5b18020000 tx=0x7f5b18009190 comp rx=0 tx=0).stop 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 shutdown_connections 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f7e540 0x55ef03f7e950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f3b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f89600 0x55ef03f6f9d0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b0b7fe640 0 client.8718 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1527866622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f68690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 >> 172.21.15.113:0/1527866622 conn(0x55ef03f41a40 msgr2=0x55ef03f8a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 shutdown_connections 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1527866622 wait complete. 2024-04-25T00:05:59.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.500+0000 7f5b35b8a740 20 client.8718 trim_cache size 0 max 16384 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.517+0000 7f5b35b8a740 1 Processor -- start 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.517+0000 7f5b35b8a740 1 -- start start 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f88e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f6bd60 0x55ef03f7e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f7e540 0x55ef03f7ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c1fe6d0 con 0x55ef03f6bd60 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c1fd6b0 con 0x55ef03f3b260 2024-04-25T00:05:59.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c0b50d0 con 0x55ef03f7e540 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f6bd60 0x55ef03f7e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f6bd60 0x55ef03f7e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48068/0 (socket says 172.21.15.113:48068) 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1928676082 learned_addr learned my addr 172.21.15.113:0/1928676082 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f88e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b31348640 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f7e540 0x55ef03f7ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f7e540 msgr2=0x55ef03f7ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f7e540 0x55ef03f7ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 msgr2=0x55ef03f88e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f88e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 -- 172.21.15.113:0/1928676082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef03fab260 con 0x55ef03f6bd60 2024-04-25T00:05:59.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f88e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f6bd60 0x55ef03f7e920 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5b180334b0 tx=0x7f5b1802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeacbbdbe212a1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b15ffb640 1 -- 172.21.15.113:0/1928676082 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18029810 con 0x55ef03f6bd60 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b15ffb640 1 -- 172.21.15.113:0/1928676082 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b18024040 con 0x55ef03f6bd60 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.518+0000 7f5b15ffb640 1 -- 172.21.15.113:0/1928676082 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18029e30 con 0x55ef03f6bd60 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1928676082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f6bd60 msgr2=0x55ef03f7e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f6bd60 0x55ef03f7e920 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5b180334b0 tx=0x7f5b1802d000 comp rx=0 tx=0).stop 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1928676082 shutdown_connections 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f7e540 0x55ef03f7ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f6bd60 0x55ef03f7e920 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1928676082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f3b260 0x55ef03f88e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1928676082 >> 172.21.15.113:0/1928676082 conn(0x55ef03f41a40 msgr2=0x55ef03f85850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1928676082 shutdown_connections 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1928676082 wait complete. 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.519+0000 7f5b35b8a740 1 -- start start 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 1 Processor -- start 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f3b260 0x55ef03f7d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f7e540 0x55ef03f99000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ef03f808d0 con 0x55ef03f3b260 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c1f5cd0 con 0x55ef03f7e540 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ef03f88ae0 con 0x55ef03f6bd60 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b35b8a740 10 client.0 _ll_register_callbacks cb 0x7f5b1800eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.520+0000 7f5b30b47640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f3b260 0x55ef03f7d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f3b260 0x55ef03f7d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48078/0 (socket says 172.21.15.113:48078) 2024-04-25T00:05:59.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 -- 172.21.15.113:0/1800136076 learned_addr learned my addr 172.21.15.113:0/1800136076 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b31348640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f7e540 0x55ef03f99000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 -- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 msgr2=0x55ef03f7ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f7ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 -- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f7e540 msgr2=0x55ef03f99000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f7e540 0x55ef03f99000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 -- 172.21.15.113:0/1800136076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef03f7f260 con 0x55ef03f3b260 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b31348640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f7e540 0x55ef03f99000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b30b47640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f3b260 0x55ef03f7d080 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5b20079790 tx=0x7f5b20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1348b193c5e431d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1800136076 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2004a040 con 0x55ef03f3b260 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1800136076 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f5b200800b0 con 0x55ef03f3b260 2024-04-25T00:05:59.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b18024040 con 0x55ef03f3b260 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1800136076 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f5b20077b00 con 0x55ef03f3b260 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 10 client.8724 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55ef03f87fb0 con 0x55ef03f3b260 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1800136076 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b18024040 con 0x55ef03f3b260 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1800136076 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f5b20075e70 con 0x55ef03f3b260 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 10 client.8724 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 10 client.8724 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 20 client.8724 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f89600 0x55ef03fffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 4 client.8724 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.521+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x55ef03eacf40 con 0x55ef03f89600 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f89600 0x55ef03fffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b0b7fe640 10 client.8724 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b2bfff640 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f89600 0x55ef03fffb60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f42415b50cc520 server_cookie=d8bb9f5b28cb51c1 in_seq=0 out_seq=0 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b0b7fe640 1 -- 172.21.15.113:0/1800136076 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef03eacf40 con 0x55ef03f89600 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b0b7fe640 10 client.8724 handle_command_reply: tid=0 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 client.8724 shutdown 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f3b260 msgr2=0x55ef03f7d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f3b260 0x55ef03f7d080 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5b20079790 tx=0x7f5b20008040 comp rx=0 tx=0).stop 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 shutdown_connections 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x55ef03f89600 0x55ef03fffb60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef03f7e540 0x55ef03f99000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef03f6bd60 0x55ef03f7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b0b7fe640 0 client.8724 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 --2- 172.21.15.113:0/1800136076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef03f3b260 0x55ef03f7d080 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 >> 172.21.15.113:0/1800136076 conn(0x55ef03f41a40 msgr2=0x55ef03f6de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 shutdown_connections 2024-04-25T00:05:59.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.522+0000 7f5b35b8a740 1 -- 172.21.15.113:0/1800136076 wait complete. 2024-04-25T00:05:59.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b35b8a740 20 client.8724 trim_cache size 0 max 16384 2024-04-25T00:05:59.532 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:59.532 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:05:59.532 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:59.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b1006be30 msgr2=0x7f5b1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b1006be30 0x7f5b1006e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5b18006e80 tx=0x7f5b18006dd0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c0d04f0 msgr2=0x7f5b2c1f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c0d04f0 0x7f5b2c1f25a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5b20035040 tx=0x7f5b200046c0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 shutdown_connections 2024-04-25T00:05:59.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b1006be30 0x7f5b1006e2f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b2c1604d0 0x7f5b2c15efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c13f050 0x7f5b2c0b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 --2- 172.21.15.113:0/620407304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c0d04f0 0x7f5b2c1f25a0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 >> 172.21.15.113:0/620407304 conn(0x7f5b2c06b0f0 msgr2=0x7f5b2c15d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 shutdown_connections 2024-04-25T00:05:59.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.523+0000 7f5b32dab640 1 -- 172.21.15.113:0/620407304 wait complete. 2024-04-25T00:05:59.535 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T00:05:59.535 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T00:05:59.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.632+0000 7f7c660f5640 1 Processor -- start 2024-04-25T00:05:59.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.632+0000 7f7c660f5640 1 -- start start 2024-04-25T00:05:59.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.633+0000 7f7c660f5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c600b5ca0 0x7f7c600b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.633+0000 7f7c660f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c600d2e00 con 0x7f7c6015e510 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.633+0000 7f7c660f5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c600d30e0 con 0x7f7c600b73c0 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.633+0000 7f7c660f5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7c600b6640 con 0x7f7c600b5ca0 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.633+0000 7f7c5f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7c6015e510 0x7f7c600d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56638/0 (socket says 172.21.15.113:56638) 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.633+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/1601743845 learned_addr learned my addr 172.21.15.113:0/1601743845 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5effd640 1 --2- 172.21.15.113:0/1601743845 >> v2:172.21.15.178:3300/0 conn(0x7f7c600b5ca0 0x7f7c600b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615672080 0 0) 0x7f7c600d2e00 con 0x7f7c6015e510 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c48003520 con 0x7f7c6015e510 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3636431653 0 0) 0x7f7c48003520 con 0x7f7c6015e510 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 >> v2:172.21.15.178:3300/0 conn(0x7f7c600b5ca0 msgr2=0x7f7c600b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/1601743845 >> v2:172.21.15.178:3300/0 conn(0x7f7c600b5ca0 0x7f7c600b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 >> v1:172.21.15.178:6789/0 conn(0x7f7c600b73c0 legacy=0x7f7c600b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c600d33c0 con 0x7f7c6015e510 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f7c50001670 con 0x7f7c6015e510 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 >> v1:172.21.15.113:6789/0 conn(0x7f7c6015e510 legacy=0x7f7c600d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c48004800 0x7f7c48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c480051c0 0x7f7c4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4800c130 con 0x7f7c480039b0 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4800c2e0 con 0x7f7c480051c0 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4800c4e0 con 0x7f7c48004800 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5effd640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c48004800 0x7f7c48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c480051c0 0x7f7c4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c48004800 msgr2=0x7f7c48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c48004800 0x7f7c48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c480051c0 msgr2=0x7f7c4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c480051c0 0x7f7c4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/1601743845 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48003520 con 0x7f7c480039b0 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5effd640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c48004800 0x7f7c48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c480051c0 0x7f7c4800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.634+0000 7f7c5ffff640 1 -- 172.21.15.113:0/1601743845 reap_dead start 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.635+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c240 tx=0x7f7c48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff55e028c097ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.635+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50007d10 con 0x7f7c480039b0 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.635+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c50005ad0 con 0x7f7c480039b0 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.635+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/1601743845 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50007070 con 0x7f7c480039b0 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.636+0000 7f7c660f5640 1 -- 172.21.15.113:0/1601743845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 msgr2=0x7f7c48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.636+0000 7f7c660f5640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c240 tx=0x7f7c48005830 comp rx=0 tx=0).stop 2024-04-25T00:05:59.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.636+0000 7f7c660f5640 1 -- 172.21.15.113:0/1601743845 shutdown_connections 2024-04-25T00:05:59.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.636+0000 7f7c660f5640 1 --2- 172.21.15.113:0/1601743845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c48003dd0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.636+0000 7f7c660f5640 1 -- 172.21.15.113:0/1601743845 >> 172.21.15.113:0/1601743845 conn(0x7f7c6006b720 msgr2=0x7f7c600d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.636+0000 7f7c660f5640 1 -- 172.21.15.113:0/1601743845 shutdown_connections 2024-04-25T00:05:59.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.636+0000 7f7c660f5640 1 -- 172.21.15.113:0/1601743845 wait complete. 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.637+0000 7f7c660f5640 1 Processor -- start 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.637+0000 7f7c660f5640 1 -- start start 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c660f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c6015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c660f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c600c9240 0x7f7c600cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c660f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c600c96c0 0x7f7c600cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c660f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600d3ea0 con 0x7f7c480039b0 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c660f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600d3980 con 0x7f7c600c9240 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c660f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c600b9dd0 con 0x7f7c600c96c0 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c6015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c6015d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48088/0 (socket says 172.21.15.113:48088) 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/2689223289 learned_addr learned my addr 172.21.15.113:0/2689223289 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/2689223289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c600c96c0 msgr2=0x7f7c600cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c600c96c0 0x7f7c600cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/2689223289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c600c9240 msgr2=0x7f7c600cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5effd640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c600c9240 0x7f7c600cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c600c9240 0x7f7c600cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 -- 172.21.15.113:0/2689223289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c600cbee0 con 0x7f7c480039b0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c6015d330 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034040 tx=0x7f7c600d2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c47fff640 1 -- 172.21.15.113:0/2689223289 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50041070 con 0x7f7c480039b0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.638+0000 7f7c47fff640 1 -- 172.21.15.113:0/2689223289 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c50004e40 con 0x7f7c480039b0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.639+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c600c81a0 con 0x7f7c480039b0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.639+0000 7f7c47fff640 1 -- 172.21.15.113:0/2689223289 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50007d10 con 0x7f7c480039b0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.639+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c600c8760 con 0x7f7c480039b0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.640+0000 7f7c47fff640 1 -- 172.21.15.113:0/2689223289 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c50044070 con 0x7f7c480039b0 2024-04-25T00:05:59.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.640+0000 7f7c47fff640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c4c06be30 0x7f7c4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.640+0000 7f7c47fff640 1 -- 172.21.15.113:0/2689223289 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f7c500ab150 con 0x7f7c480039b0 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.640+0000 7f7c5effd640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c4c06be30 0x7f7c4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.641+0000 7f7c5effd640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c4c06be30 0x7f7c4c06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7c600ce800 tx=0x7f7c54008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.641+0000 7f7c68e02740 1 Processor -- start 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.641+0000 7f7c68e02740 1 -- start start 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d87140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3da4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d5ca40 0x5629e3da2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c68e02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5629e3cc06b0 con 0x5629e3da4600 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629e3c68880 con 0x5629e3d5ca40 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5629e3cbc670 con 0x5629e3d86d60 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3da4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3da4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48090/0 (socket says 172.21.15.113:48090) 2024-04-25T00:05:59.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 -- 172.21.15.113:0/1205830916 learned_addr learned my addr 172.21.15.113:0/1205830916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d87140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d5ca40 0x5629e3da2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 -- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 msgr2=0x5629e3d87140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d87140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 -- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d5ca40 msgr2=0x5629e3da2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d5ca40 0x5629e3da2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5effd640 1 -- 172.21.15.113:0/1205830916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e3a9e7d0 con 0x5629e3da4600 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d87140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.642+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d5ca40 0x5629e3da2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c5effd640 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3da4b50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7c54012ca0 tx=0x7f7c54015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaffc496cfbfea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c457fa640 1 -- 172.21.15.113:0/1205830916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54022440 con 0x5629e3da4600 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c457fa640 1 -- 172.21.15.113:0/1205830916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54027070 con 0x5629e3da4600 2024-04-25T00:05:59.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c457fa640 1 -- 172.21.15.113:0/1205830916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c540227e0 con 0x5629e3da4600 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c68e02740 1 -- 172.21.15.113:0/1205830916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 msgr2=0x5629e3da4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3da4b50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7c54012ca0 tx=0x7f7c54015000 comp rx=0 tx=0).stop 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c68e02740 1 -- 172.21.15.113:0/1205830916 shutdown_connections 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d5ca40 0x5629e3da2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3da4b50 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1205830916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d87140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.643+0000 7f7c68e02740 1 -- 172.21.15.113:0/1205830916 >> 172.21.15.113:0/1205830916 conn(0x5629e3d59600 msgr2=0x5629e3da1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.644+0000 7f7c68e02740 1 -- 172.21.15.113:0/1205830916 shutdown_connections 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.644+0000 7f7c68e02740 1 -- 172.21.15.113:0/1205830916 wait complete. 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.644+0000 7f7c68e02740 1 -- start start 2024-04-25T00:05:59.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.644+0000 7f7c68e02740 1 Processor -- start 2024-04-25T00:05:59.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d83700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3d98c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d991a0 0x5629e3db2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c68e02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5629e3d5d7e0 con 0x5629e3da4600 2024-04-25T00:05:59.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629e3c7bf50 con 0x5629e3d991a0 2024-04-25T00:05:59.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5629e3d57530 con 0x5629e3d86d60 2024-04-25T00:05:59.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c68e02740 10 client.0 _ll_register_callbacks cb 0x5629e3cb7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3d98c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3d98c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48106/0 (socket says 172.21.15.113:48106) 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 -- 172.21.15.113:0/469091040 learned_addr learned my addr 172.21.15.113:0/469091040 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d83700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d991a0 0x5629e3db2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 -- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 msgr2=0x5629e3d83700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d83700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 -- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d991a0 msgr2=0x5629e3db2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d991a0 0x5629e3db2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 -- 172.21.15.113:0/469091040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e39d43f0 con 0x5629e3da4600 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c5effd640 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3d98c60 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7c54024040 tx=0x7f7c54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae04e21ecd34f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:59.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.645+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/469091040 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54022bc0 con 0x5629e3da4600 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/469091040 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f7c54022d60 con 0x5629e3da4600 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c54027070 con 0x5629e3da4600 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/469091040 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7c5402b3f0 con 0x5629e3da4600 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 10 client.8736 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x5629e39940f0 con 0x5629e3da4600 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/469091040 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54027070 con 0x5629e3da4600 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/469091040 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f7c5402b590 con 0x5629e3da4600 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 10 client.8736 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 10 client.8736 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 20 client.8736 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3d891c0 0x5629e3d895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 4 client.8736 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629e3ccdbf0 con 0x5629e3d891c0 2024-04-25T00:05:59.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.646+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3d891c0 0x5629e3d895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c3e7fc640 10 client.8736 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3d891c0 0x5629e3d895a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd9f480151fd376 server_cookie=c7de2738f44cdefb in_seq=0 out_seq=0 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/469091040 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629e3ccdbf0 con 0x5629e3d891c0 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c3e7fc640 10 client.8736 handle_command_reply: tid=0 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 client.8736 shutdown 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 msgr2=0x5629e3d98c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3d98c60 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7c54024040 tx=0x7f7c54004730 comp rx=0 tx=0).stop 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 shutdown_connections 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3d891c0 0x5629e3d895a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d991a0 0x5629e3db2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3da4600 0x5629e3d98c60 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c3e7fc640 0 client.8736 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 --2- 172.21.15.113:0/469091040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d83700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 >> 172.21.15.113:0/469091040 conn(0x5629e3d5ca40 msgr2=0x5629e3d85de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 shutdown_connections 2024-04-25T00:05:59.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 1 -- 172.21.15.113:0/469091040 wait complete. 2024-04-25T00:05:59.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.647+0000 7f7c68e02740 20 client.8736 trim_cache size 0 max 16384 2024-04-25T00:05:59.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.665+0000 7f7c68e02740 1 Processor -- start 2024-04-25T00:05:59.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.665+0000 7f7c68e02740 1 -- start start 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d59c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d891c0 0x5629e3da1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c68e02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5629e3d80dc0 con 0x5629e3d991a0 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60200f10 con 0x5629e3d891c0 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c601fff70 con 0x5629e3d86d60 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48118/0 (socket says 172.21.15.113:48118) 2024-04-25T00:05:59.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 -- 172.21.15.113:0/324672162 learned_addr learned my addr 172.21.15.113:0/324672162 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 -- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 msgr2=0x5629e3d59c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5effd640 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d891c0 0x5629e3da1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d59c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d59c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 -- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d891c0 msgr2=0x5629e3da1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d891c0 0x5629e3da1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 -- 172.21.15.113:0/324672162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e3db2c40 con 0x5629e3d991a0 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5effd640 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d891c0 0x5629e3da1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d59c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d861d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7c600d1b90 tx=0x7f7c580054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615019c49e58ac5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c44ff9640 1 -- 172.21.15.113:0/324672162 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5801a070 con 0x5629e3d991a0 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.666+0000 7f7c44ff9640 1 -- 172.21.15.113:0/324672162 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c58001b70 con 0x5629e3d991a0 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c44ff9640 1 -- 172.21.15.113:0/324672162 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58001f60 con 0x5629e3d991a0 2024-04-25T00:05:59.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 -- 172.21.15.113:0/324672162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 msgr2=0x5629e3d861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d861d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7c600d1b90 tx=0x7f7c580054e0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 -- 172.21.15.113:0/324672162 shutdown_connections 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d891c0 0x5629e3da1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d861d0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 --2- 172.21.15.113:0/324672162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d86d60 0x5629e3d59c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 -- 172.21.15.113:0/324672162 >> 172.21.15.113:0/324672162 conn(0x5629e3d5ca40 msgr2=0x5629e3d9a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 -- 172.21.15.113:0/324672162 shutdown_connections 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.667+0000 7f7c68e02740 1 -- 172.21.15.113:0/324672162 wait complete. 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 -- start start 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 Processor -- start 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d86d60 0x5629e3d5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d891c0 0x5629e3d87a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d87f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601ff920 con 0x5629e3d991a0 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6015f6e0 con 0x5629e3d86d60 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5629e3d8a640 con 0x5629e3d891c0 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c68e02740 10 client.0 _ll_register_callbacks cb 0x7f7c54012b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.668+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d87f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d87f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48132/0 (socket says 172.21.15.113:48132) 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 -- 172.21.15.113:0/1664987293 learned_addr learned my addr 172.21.15.113:0/1664987293 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5effd640 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d891c0 0x5629e3d87a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d86d60 0x5629e3d5b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 -- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d891c0 msgr2=0x5629e3d87a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d891c0 0x5629e3d87a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 -- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d86d60 msgr2=0x5629e3d5b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:59.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d86d60 0x5629e3d5b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 -- 172.21.15.113:0/1664987293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e3d89a70 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c5ffff640 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d87f60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f7c580019e0 tx=0x7f7c58007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f080552a30b0e4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/1664987293 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58026060 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/1664987293 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f7c58015850 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c58001b70 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/1664987293 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7c5802c050 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 10 client.8742 fetch_fsmap learned FSMap version 62 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x5629e3e1eac0 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/1664987293 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c58001b70 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/1664987293 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f7c5802f020 con 0x5629e3d991a0 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 10 client.8742 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T00:05:59.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 10 client.8742 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 20 client.8742 populate_metadata read hostname 'smithi113' 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3da4600 0x5629e3da2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 4 client.8742 mds_command: new command op to 8400 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.669+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 --> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629e3cc7f40 con 0x5629e3da4600 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3da4600 0x5629e3da2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c5f7fe640 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3da4600 0x5629e3da2a00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab8b59e9e57fbbf server_cookie=1204567e6d841033 in_seq=0 out_seq=0 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c3e7fc640 10 client.8742 ms_handle_connect on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c3e7fc640 1 -- 172.21.15.113:0/1664987293 <== mds.0 v2:172.21.15.178:6833/840156279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629e3cc7f40 con 0x5629e3da4600 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c3e7fc640 10 client.8742 handle_command_reply: tid=0 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 client.8742 shutdown 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 msgr2=0x5629e3d87f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d87f60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f7c580019e0 tx=0x7f7c58007f50 comp rx=0 tx=0).stop 2024-04-25T00:05:59.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 shutdown_connections 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:6833/840156279,v1:172.21.15.178:6835/840156279] conn(0x5629e3da4600 0x5629e3da2a00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e3d991a0 0x5629e3d87f60 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5629e3d891c0 0x5629e3d87a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c3e7fc640 0 client.8742 ms_handle_reset on v2:172.21.15.178:6833/840156279 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 --2- 172.21.15.113:0/1664987293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629e3d86d60 0x5629e3d5b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 >> 172.21.15.113:0/1664987293 conn(0x5629e3d5ca40 msgr2=0x5629e3d9a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 shutdown_connections 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 1 -- 172.21.15.113:0/1664987293 wait complete. 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.670+0000 7f7c68e02740 20 client.8742 trim_cache size 0 max 16384 2024-04-25T00:05:59.678 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:05:59.679 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:05:59.679 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:05:59.679 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c4c06be30 msgr2=0x7f7c4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c4c06be30 0x7f7c4c06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7c600ce800 tx=0x7f7c54008040 comp rx=0 tx=0).stop 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 msgr2=0x7f7c6015d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c6015d330 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034040 tx=0x7f7c600d2fd0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 shutdown_connections 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c4c06be30 0x7f7c4c06e2f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c600c9240 0x7f7c600cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c600c96c0 0x7f7c600cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 --2- 172.21.15.113:0/2689223289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480039b0 0x7f7c6015d330 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 >> 172.21.15.113:0/2689223289 conn(0x7f7c6006b720 msgr2=0x7f7c600b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 shutdown_connections 2024-04-25T00:05:59.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.671+0000 7f7c660f5640 1 -- 172.21.15.113:0/2689223289 wait complete. 2024-04-25T00:05:59.682 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:05:59.682 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:05:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.778+0000 7fe593c4d640 1 Processor -- start 2024-04-25T00:05:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.779+0000 7fe593c4d640 1 -- start start 2024-04-25T00:05:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.779+0000 7fe593c4d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe58c0d0b00 0x7fe58c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.779+0000 7fe593c4d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c0d1480 con 0x7fe58c15e320 2024-04-25T00:05:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.779+0000 7fe593c4d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c0d2800 con 0x7fe58c0d2290 2024-04-25T00:05:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.779+0000 7fe593c4d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c0d3410 con 0x7fe58c0d0b00 2024-04-25T00:05:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.780+0000 7fe5921ea640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe58c0d0b00 0x7fe58c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.780+0000 7fe5919e9640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe58c15e320 0x7fe58c0d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56648/0 (socket says 172.21.15.113:56648) 2024-04-25T00:05:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.780+0000 7fe5919e9640 1 -- 172.21.15.113:0/1106585633 learned_addr learned my addr 172.21.15.113:0/1106585633 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:05:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.780+0000 7fe5921ea640 1 -- 172.21.15.113:0/1106585633 >> v1:172.21.15.178:6789/0 conn(0x7fe58c0d2290 legacy=0x7fe58c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.780+0000 7fe5921ea640 1 -- 172.21.15.113:0/1106585633 >> v1:172.21.15.113:6789/0 conn(0x7fe58c15e320 legacy=0x7fe58c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.780+0000 7fe5909e7640 1 -- 172.21.15.113:0/1106585633 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2040557713 0 0) 0x7fe58c0d1480 con 0x7fe58c15e320 2024-04-25T00:05:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.780+0000 7fe5921ea640 1 -- 172.21.15.113:0/1106585633 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c0d2ae0 con 0x7fe58c0d0b00 2024-04-25T00:05:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe5921ea640 1 --2- 172.21.15.113:0/1106585633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d0b00 0x7fe58c0d0f40 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe584009d90 tx=0x7fe58402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e6f7bc33897564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe5909e7640 1 -- 172.21.15.113:0/1106585633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584037070 con 0x7fe58c0d0b00 2024-04-25T00:05:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe5909e7640 1 -- 172.21.15.113:0/1106585633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe584007740 con 0x7fe58c0d0b00 2024-04-25T00:05:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe5909e7640 1 -- 172.21.15.113:0/1106585633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584007b10 con 0x7fe58c0d0b00 2024-04-25T00:05:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe593c4d640 1 -- 172.21.15.113:0/1106585633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d0b00 msgr2=0x7fe58c0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1106585633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d0b00 0x7fe58c0d0f40 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe584009d90 tx=0x7fe58402a780 comp rx=0 tx=0).stop 2024-04-25T00:05:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe593c4d640 1 -- 172.21.15.113:0/1106585633 shutdown_connections 2024-04-25T00:05:59.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1106585633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d0b00 0x7fe58c0d0f40 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe593c4d640 1 -- 172.21.15.113:0/1106585633 >> 172.21.15.113:0/1106585633 conn(0x7fe58c0b5c20 msgr2=0x7fe58c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:59.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe593c4d640 1 -- 172.21.15.113:0/1106585633 shutdown_connections 2024-04-25T00:05:59.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.781+0000 7fe593c4d640 1 -- 172.21.15.113:0/1106585633 wait complete. 2024-04-25T00:05:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.782+0000 7fe593c4d640 1 Processor -- start 2024-04-25T00:05:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe593c4d640 1 -- start start 2024-04-25T00:05:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe593c4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c0d0b00 0x7fe58c14d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe593c4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d2290 0x7fe58c1f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe593c4d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58c15e320 0x7fe58c0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe593c4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c15e700 con 0x7fe58c0d2290 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe593c4d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0d30a0 con 0x7fe58c0d0b00 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe593c4d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c15e880 con 0x7fe58c15e320 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe5911e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d2290 0x7fe58c1f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe5911e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d2290 0x7fe58c1f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48152/0 (socket says 172.21.15.113:48152) 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe5911e8640 1 -- 172.21.15.113:0/1009580475 learned_addr learned my addr 172.21.15.113:0/1009580475 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.783+0000 7fe5919e9640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c0d0b00 0x7fe58c14d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5921ea640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58c15e320 0x7fe58c0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5911e8640 1 -- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58c15e320 msgr2=0x7fe58c0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5911e8640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58c15e320 0x7fe58c0b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5911e8640 1 -- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c0d0b00 msgr2=0x7fe58c14d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5911e8640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c0d0b00 0x7fe58c14d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5911e8640 1 -- 172.21.15.113:0/1009580475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c1f83a0 con 0x7fe58c0d2290 2024-04-25T00:05:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5911e8640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d2290 0x7fe58c1f7e60 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c450 tx=0x7fe58800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5827fc640 1 -- 172.21.15.113:0/1009580475 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5880052d0 con 0x7fe58c0d2290 2024-04-25T00:05:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5827fc640 1 -- 172.21.15.113:0/1009580475 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe588005470 con 0x7fe58c0d2290 2024-04-25T00:05:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c14da70 con 0x7fe58c0d2290 2024-04-25T00:05:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe5827fc640 1 -- 172.21.15.113:0/1009580475 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588012870 con 0x7fe58c0d2290 2024-04-25T00:05:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.784+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c14e310 con 0x7fe58c0d2290 2024-04-25T00:05:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.785+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c14dc40 con 0x7fe58c0d2290 2024-04-25T00:05:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.785+0000 7fe5827fc640 1 -- 172.21.15.113:0/1009580475 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe588012a10 con 0x7fe58c0d2290 2024-04-25T00:05:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.786+0000 7fe5827fc640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.786+0000 7fe5827fc640 1 -- 172.21.15.113:0/1009580475 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fe58808ab30 con 0x7fe58c0d2290 2024-04-25T00:05:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.786+0000 7fe5919e9640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.786+0000 7fe5919e9640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe57006be30 0x7fe57006e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000f60 tx=0x7fe57c007b90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:59.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.790+0000 7fe5827fc640 1 -- 172.21.15.113:0/1009580475 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe58c1f83a0 con 0x7fe58c0d2290 2024-04-25T00:05:59.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.997+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe58c0691a0 con 0x7fe58c0d2290 2024-04-25T00:05:59.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:05:59.998+0000 7fe5827fc640 1 -- 172.21.15.113:0/1009580475 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7fe588059440 con 0x7fe58c0d2290 2024-04-25T00:05:59.999 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:00.000 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:06:00.000 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:06:00.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe57006be30 msgr2=0x7fe57006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe57006be30 0x7fe57006e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000f60 tx=0x7fe57c007b90 comp rx=0 tx=0).stop 2024-04-25T00:06:00.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d2290 msgr2=0x7fe58c1f7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d2290 0x7fe58c1f7e60 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c450 tx=0x7fe58800c910 comp rx=0 tx=0).stop 2024-04-25T00:06:00.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 shutdown_connections 2024-04-25T00:06:00.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe57006be30 0x7fe57006e2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58c15e320 0x7fe58c0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe58c0d2290 0x7fe58c1f7e60 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 --2- 172.21.15.113:0/1009580475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c0d0b00 0x7fe58c14d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 >> 172.21.15.113:0/1009580475 conn(0x7fe58c0b5c20 msgr2=0x7fe58c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 shutdown_connections 2024-04-25T00:06:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.000+0000 7fe593c4d640 1 -- 172.21.15.113:0/1009580475 wait complete. 2024-04-25T00:06:00.012 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:06:00.012 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:06:00.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.107+0000 7f9297441640 1 Processor -- start 2024-04-25T00:06:00.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.107+0000 7f9297441640 1 -- start start 2024-04-25T00:06:00.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.108+0000 7f9297441640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f92900cea50 0x7f929006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:00.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.108+0000 7f9297441640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929006ee00 con 0x7f929006e950 2024-04-25T00:06:00.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.108+0000 7f9297441640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929006f8e0 con 0x7f92900cb4d0 2024-04-25T00:06:00.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.108+0000 7f9297441640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f929006b0f0 con 0x7f92900cea50 2024-04-25T00:06:00.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.108+0000 7f92959de640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f929006e950 0x7f929006bb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56660/0 (socket says 172.21.15.113:56660) 2024-04-25T00:06:00.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.108+0000 7f92959de640 1 -- 172.21.15.113:0/2934429791 learned_addr learned my addr 172.21.15.113:0/2934429791 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:00.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.108+0000 7f92951dd640 1 --2- 172.21.15.113:0/2934429791 >> v2:172.21.15.178:3300/0 conn(0x7f92900cea50 0x7f929006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:00.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.109+0000 7f9287fff640 1 -- 172.21.15.113:0/2934429791 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1960740939 0 0) 0x7f929006ee00 con 0x7f929006e950 2024-04-25T00:06:00.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.109+0000 7f9287fff640 1 -- 172.21.15.113:0/2934429791 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f927c003520 con 0x7f929006e950 2024-04-25T00:06:00.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.109+0000 7f92951dd640 1 -- 172.21.15.113:0/2934429791 >> v1:172.21.15.178:6789/0 conn(0x7f92900cb4d0 legacy=0x7f92900cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.109+0000 7f92951dd640 1 -- 172.21.15.113:0/2934429791 >> v1:172.21.15.113:6789/0 conn(0x7f929006e950 legacy=0x7f929006bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.109+0000 7f92951dd640 1 -- 172.21.15.113:0/2934429791 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929006fbc0 con 0x7f92900cea50 2024-04-25T00:06:00.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f92951dd640 1 --2- 172.21.15.113:0/2934429791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cea50 0x7f929006f2c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f928c0097a0 tx=0x7f928c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a1bef31d784173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:00.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9287fff640 1 -- 172.21.15.113:0/2934429791 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c037070 con 0x7f92900cea50 2024-04-25T00:06:00.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9287fff640 1 -- 172.21.15.113:0/2934429791 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c032440 con 0x7f92900cea50 2024-04-25T00:06:00.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9287fff640 1 -- 172.21.15.113:0/2934429791 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c0327f0 con 0x7f92900cea50 2024-04-25T00:06:00.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9297441640 1 -- 172.21.15.113:0/2934429791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cea50 msgr2=0x7f929006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9297441640 1 --2- 172.21.15.113:0/2934429791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cea50 0x7f929006f2c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f928c0097a0 tx=0x7f928c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9297441640 1 -- 172.21.15.113:0/2934429791 shutdown_connections 2024-04-25T00:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9297441640 1 --2- 172.21.15.113:0/2934429791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cea50 0x7f929006f2c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.110+0000 7f9297441640 1 -- 172.21.15.113:0/2934429791 >> 172.21.15.113:0/2934429791 conn(0x7f92900cd900 msgr2=0x7f92900cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.111+0000 7f9297441640 1 -- 172.21.15.113:0/2934429791 shutdown_connections 2024-04-25T00:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.111+0000 7f9297441640 1 -- 172.21.15.113:0/2934429791 wait complete. 2024-04-25T00:06:00.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 Processor -- start 2024-04-25T00:06:00.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 -- start start 2024-04-25T00:06:00.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929006e950 0x7f92900b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cb4d0 0x7f92900a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92900cea50 0x7f92900c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f929015e6e0 con 0x7f929006e950 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f929006b620 con 0x7f92900cb4d0 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.112+0000 7f9297441640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f929015e860 con 0x7f92900cea50 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929006e950 0x7f92900b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929006e950 0x7f92900b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48154/0 (socket says 172.21.15.113:48154) 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 -- 172.21.15.113:0/281762299 learned_addr learned my addr 172.21.15.113:0/281762299 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92949dc640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cb4d0 0x7f92900a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92959de640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92900cea50 0x7f92900c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 -- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92900cea50 msgr2=0x7f92900c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92900cea50 0x7f92900c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 -- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cb4d0 msgr2=0x7f92900a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cb4d0 0x7f92900a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 -- 172.21.15.113:0/281762299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92900a1cd0 con 0x7f929006e950 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92959de640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92900cea50 0x7f92900c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f92951dd640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929006e950 0x7f92900b4c20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f928c0097a0 tx=0x7f928c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f9285ffb640 1 -- 172.21.15.113:0/281762299 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c037050 con 0x7f929006e950 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f9285ffb640 1 -- 172.21.15.113:0/281762299 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c038040 con 0x7f929006e950 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92900a0b50 con 0x7f929006e950 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f9285ffb640 1 -- 172.21.15.113:0/281762299 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c004470 con 0x7f929006e950 2024-04-25T00:06:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.113+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92900a0e30 con 0x7f929006e950 2024-04-25T00:06:00.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.114+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9290074e20 con 0x7f929006e950 2024-04-25T00:06:00.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.115+0000 7f9285ffb640 1 -- 172.21.15.113:0/281762299 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f928c03c6d0 con 0x7f929006e950 2024-04-25T00:06:00.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.115+0000 7f9285ffb640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:00.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.115+0000 7f9285ffb640 1 -- 172.21.15.113:0/281762299 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f928c0ab7a0 con 0x7f929006e950 2024-04-25T00:06:00.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.115+0000 7f92949dc640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:00.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.116+0000 7f92949dc640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f92900c3e10 tx=0x7f9280009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:00.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.120+0000 7f9285ffb640 1 -- 172.21.15.113:0/281762299 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f928c07a1e0 con 0x7f929006e950 2024-04-25T00:06:00.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.360+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9290068220 con 0x7f929006e950 2024-04-25T00:06:00.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.361+0000 7f9285ffb640 1 -- 172.21.15.113:0/281762299 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7f928c079ae0 con 0x7f929006e950 2024-04-25T00:06:00.363 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:00.363 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":485,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561799,"bytes_used":2547630080,"bytes_avail":770546483200,"bytes_total":773094113280,"write_bytes_sec":52187754,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T00:05:57.200778+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:06:00.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 msgr2=0x7f927406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f92900c3e10 tx=0x7f9280009040 comp rx=0 tx=0).stop 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929006e950 msgr2=0x7f92900b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929006e950 0x7f92900b4c20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f928c0097a0 tx=0x7f928c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 shutdown_connections 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92900cb4d0 0x7f92900a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92900cea50 0x7f92900c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 --2- 172.21.15.113:0/281762299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929006e950 0x7f92900b4c20 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 >> 172.21.15.113:0/281762299 conn(0x7f92900cd900 msgr2=0x7f92900b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:00.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 shutdown_connections 2024-04-25T00:06:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:00.363+0000 7f9297441640 1 -- 172.21.15.113:0/281762299 wait complete. 2024-04-25T00:06:00.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T00:06:01.376 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:01.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.482+0000 7f7569c76640 1 Processor -- start 2024-04-25T00:06:01.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.482+0000 7f7569c76640 1 -- start start 2024-04-25T00:06:01.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f7569c76640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f756406e6e0 0x7f75640b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f7569c76640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75641453c0 0x7f7564071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f7569c76640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75640cfad0 con 0x7f7564144d60 2024-04-25T00:06:01.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f7569c76640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7564145c40 con 0x7f75641453c0 2024-04-25T00:06:01.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f7569c76640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75640cf230 con 0x7f756406e6e0 2024-04-25T00:06:01.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f7563fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75641453c0 0x7f7564071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f75637fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7564144d60 0x7f756406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56670/0 (socket says 172.21.15.113:56670) 2024-04-25T00:06:01.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.483+0000 7f75637fe640 1 -- 172.21.15.113:0/667353711 learned_addr learned my addr 172.21.15.113:0/667353711 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:01.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f7563fff640 1 -- 172.21.15.113:0/667353711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f756406e6e0 msgr2=0x7f75640b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:01.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f7563fff640 1 --2- 172.21.15.113:0/667353711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f756406e6e0 0x7f75640b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f7563fff640 1 -- 172.21.15.113:0/667353711 >> v1:172.21.15.113:6789/0 conn(0x7f7564144d60 legacy=0x7f756406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f7563fff640 1 -- 172.21.15.113:0/667353711 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75640cf510 con 0x7f75641453c0 2024-04-25T00:06:01.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f7563fff640 1 --2- 172.21.15.113:0/667353711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75641453c0 0x7f7564071330 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdc019c8a6c9330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:01.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f75627fc640 1 -- 172.21.15.113:0/667353711 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c031600 con 0x7f75641453c0 2024-04-25T00:06:01.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f75627fc640 1 -- 172.21.15.113:0/667353711 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f755c0317a0 con 0x7f75641453c0 2024-04-25T00:06:01.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.484+0000 7f75627fc640 1 -- 172.21.15.113:0/667353711 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c031b70 con 0x7f75641453c0 2024-04-25T00:06:01.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 -- 172.21.15.113:0/667353711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75641453c0 msgr2=0x7f7564071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 --2- 172.21.15.113:0/667353711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75641453c0 0x7f7564071330 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c02a5d0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 -- 172.21.15.113:0/667353711 shutdown_connections 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 --2- 172.21.15.113:0/667353711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75641453c0 0x7f7564071330 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 --2- 172.21.15.113:0/667353711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f756406e6e0 0x7f75640b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 -- 172.21.15.113:0/667353711 >> 172.21.15.113:0/667353711 conn(0x7f75640b1a50 msgr2=0x7f75640b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 -- 172.21.15.113:0/667353711 shutdown_connections 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.485+0000 7f7569c76640 1 -- 172.21.15.113:0/667353711 wait complete. 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.486+0000 7f7569c76640 1 Processor -- start 2024-04-25T00:06:01.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.486+0000 7f7569c76640 1 -- start start 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f7569c76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f756406e6e0 0x7f75641f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f7569c76640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7564144d60 0x7f75641e7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f7569c76640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75641453c0 0x7f75641e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f7569c76640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7564145140 con 0x7f756406e6e0 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f7569c76640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7564070e10 con 0x7f75641453c0 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f7569c76640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7564071870 con 0x7f7564144d60 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f75637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f756406e6e0 0x7f75641f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f75637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f756406e6e0 0x7f75641f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48182/0 (socket says 172.21.15.113:48182) 2024-04-25T00:06:01.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.487+0000 7f75637fe640 1 -- 172.21.15.113:0/2710452655 learned_addr learned my addr 172.21.15.113:0/2710452655 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f7562ffd640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7564144d60 0x7f75641e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f75637fe640 1 -- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7564144d60 msgr2=0x7f75641e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f75637fe640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7564144d60 0x7f75641e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f7563fff640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75641453c0 0x7f75641e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f75637fe640 1 -- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75641453c0 msgr2=0x7f75641e75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f75637fe640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75641453c0 0x7f75641e75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f75637fe640 1 -- 172.21.15.113:0/2710452655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641f3b30 con 0x7f756406e6e0 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f7562ffd640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7564144d60 0x7f75641e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f75637fe640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f756406e6e0 0x7f75641f4bd0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f755400c430 tx=0x7f755400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f754bfff640 1 -- 172.21.15.113:0/2710452655 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554017070 con 0x7f756406e6e0 2024-04-25T00:06:01.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f754bfff640 1 -- 172.21.15.113:0/2710452655 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7554005ce0 con 0x7f756406e6e0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641f3e10 con 0x7f756406e6e0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f754bfff640 1 -- 172.21.15.113:0/2710452655 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540072f0 con 0x7f756406e6e0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.488+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641f43d0 con 0x7f756406e6e0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.489+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75640691a0 con 0x7f756406e6e0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.489+0000 7f754bfff640 1 -- 172.21.15.113:0/2710452655 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f755401a070 con 0x7f756406e6e0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.490+0000 7f754bfff640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f755006be30 0x7f755006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.490+0000 7f754bfff640 1 -- 172.21.15.113:0/2710452655 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5767+0+0 (secure 0 0 0) 0x7f755408a330 con 0x7f756406e6e0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.490+0000 7f7562ffd640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f755006be30 0x7f755006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.490+0000 7f7562ffd640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f755006be30 0x7f755006e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7558006fd0 tx=0x7f7558008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:01.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.494+0000 7f754bfff640 1 -- 172.21.15.113:0/2710452655 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7554058c10 con 0x7f756406e6e0 2024-04-25T00:06:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.702+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75640b27e0 con 0x7f756406e6e0 2024-04-25T00:06:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.703+0000 7f754bfff640 1 -- 172.21.15.113:0/2710452655 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f7554058510 con 0x7f756406e6e0 2024-04-25T00:06:01.704 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:01.705 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:06:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:06:01.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f755006be30 msgr2=0x7f755006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f755006be30 0x7f755006e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7558006fd0 tx=0x7f7558008040 comp rx=0 tx=0).stop 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f756406e6e0 msgr2=0x7f75641f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f756406e6e0 0x7f75641f4bd0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f755400c430 tx=0x7f755400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 shutdown_connections 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f755006be30 0x7f755006e2f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75641453c0 0x7f75641e75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7564144d60 0x7f75641e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 --2- 172.21.15.113:0/2710452655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f756406e6e0 0x7f75641f4bd0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 >> 172.21.15.113:0/2710452655 conn(0x7f75640b1a50 msgr2=0x7f75640b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 shutdown_connections 2024-04-25T00:06:01.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.705+0000 7f7569c76640 1 -- 172.21.15.113:0/2710452655 wait complete. 2024-04-25T00:06:01.717 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:06:01.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:06:01.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbec2429640 1 Processor -- start 2024-04-25T00:06:01.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbec2429640 1 -- start start 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbec2429640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbebc0d2290 0x7fbebc0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbec2429640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d0b00 0x7fbebc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbec2429640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbebc0d2800 con 0x7fbebc15e320 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbec2429640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbebc0d14d0 con 0x7fbebc0d2290 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbec2429640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbebc0d3410 con 0x7fbebc0d0b00 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbebb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbebc0d2290 0x7fbebc0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbebb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbebc0d2290 0x7fbebc0b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48196/0 (socket says 172.21.15.113:48196) 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.812+0000 7fbebb7fe640 1 -- 172.21.15.113:0/2729280964 learned_addr learned my addr 172.21.15.113:0/2729280964 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.813+0000 7fbec09c6640 1 --2- 172.21.15.113:0/2729280964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d0b00 0x7fbebc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.813+0000 7fbebb7fe640 1 -- 172.21.15.113:0/2729280964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d0b00 msgr2=0x7fbebc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:01.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.813+0000 7fbebb7fe640 1 --2- 172.21.15.113:0/2729280964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d0b00 0x7fbebc0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.813+0000 7fbebb7fe640 1 -- 172.21.15.113:0/2729280964 >> v1:172.21.15.178:6789/0 conn(0x7fbebc15e320 legacy=0x7fbebc0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.813+0000 7fbebb7fe640 1 -- 172.21.15.113:0/2729280964 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc0d2ae0 con 0x7fbebc0d2290 2024-04-25T00:06:01.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.813+0000 7fbebb7fe640 1 --2- 172.21.15.113:0/2729280964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d2290 0x7fbebc0b9aa0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009870 tx=0x7fbeac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bed48f6feeee14a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:01.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbebaffd640 1 -- 172.21.15.113:0/2729280964 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac037070 con 0x7fbebc0d2290 2024-04-25T00:06:01.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbebaffd640 1 -- 172.21.15.113:0/2729280964 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeac02ae50 con 0x7fbebc0d2290 2024-04-25T00:06:01.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbebaffd640 1 -- 172.21.15.113:0/2729280964 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac032860 con 0x7fbebc0d2290 2024-04-25T00:06:01.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 -- 172.21.15.113:0/2729280964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d2290 msgr2=0x7fbebc0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 --2- 172.21.15.113:0/2729280964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d2290 0x7fbebc0b9aa0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009870 tx=0x7fbeac02a300 comp rx=0 tx=0).stop 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 -- 172.21.15.113:0/2729280964 shutdown_connections 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 --2- 172.21.15.113:0/2729280964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d0b00 0x7fbebc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 --2- 172.21.15.113:0/2729280964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d2290 0x7fbebc0b9aa0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 -- 172.21.15.113:0/2729280964 >> 172.21.15.113:0/2729280964 conn(0x7fbebc0b5c20 msgr2=0x7fbebc0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 -- 172.21.15.113:0/2729280964 shutdown_connections 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.814+0000 7fbec2429640 1 -- 172.21.15.113:0/2729280964 wait complete. 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.815+0000 7fbec2429640 1 Processor -- start 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.815+0000 7fbec2429640 1 -- start start 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbec2429640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d0b00 0x7fbebc1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbec2429640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d2290 0x7fbebc0ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbec2429640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbebc15e320 0x7fbebc0ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbec2429640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc0b9fe0 con 0x7fbebc0d0b00 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbec2429640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc0d30a0 con 0x7fbebc15e320 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbec2429640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbebc15e700 con 0x7fbebc0d2290 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d0b00 0x7fbebc1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d0b00 0x7fbebc1f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48210/0 (socket says 172.21.15.113:48210) 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbebbfff640 1 -- 172.21.15.113:0/3679928765 learned_addr learned my addr 172.21.15.113:0/3679928765 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbebb7fe640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d2290 0x7fbebc0ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.816+0000 7fbec09c6640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbebc15e320 0x7fbebc0ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbebbfff640 1 -- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d2290 msgr2=0x7fbebc0ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbebbfff640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d2290 0x7fbebc0ac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbebbfff640 1 -- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbebc15e320 msgr2=0x7fbebc0ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbebbfff640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbebc15e320 0x7fbebc0ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbebbfff640 1 -- 172.21.15.113:0/3679928765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc0ac0e0 con 0x7fbebc0d0b00 2024-04-25T00:06:01.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbebbfff640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d0b00 0x7fbebc1f37b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000c370 tx=0x7fbeb000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:01.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbeb8ff9640 1 -- 172.21.15.113:0/3679928765 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0017070 con 0x7fbebc0d0b00 2024-04-25T00:06:01.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbeb8ff9640 1 -- 172.21.15.113:0/3679928765 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb00052d0 con 0x7fbebc0d0b00 2024-04-25T00:06:01.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbeb8ff9640 1 -- 172.21.15.113:0/3679928765 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003850 con 0x7fbebc0d0b00 2024-04-25T00:06:01.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc15eaf0 con 0x7fbebc0d0b00 2024-04-25T00:06:01.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.817+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc15f0b0 con 0x7fbebc0d0b00 2024-04-25T00:06:01.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.818+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbebc0691a0 con 0x7fbebc0d0b00 2024-04-25T00:06:01.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.818+0000 7fbeb8ff9640 1 -- 172.21.15.113:0/3679928765 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbeb001a070 con 0x7fbebc0d0b00 2024-04-25T00:06:01.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.819+0000 7fbeb8ff9640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbea806be30 0x7fbea806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:01.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.819+0000 7fbeb8ff9640 1 -- 172.21.15.113:0/3679928765 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5767+0+0 (secure 0 0 0) 0x7fbeb008a120 con 0x7fbebc0d0b00 2024-04-25T00:06:01.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.819+0000 7fbebb7fe640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbea806be30 0x7fbea806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:01.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.819+0000 7fbebb7fe640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbea806be30 0x7fbea806e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbeac034000 tx=0x7fbeac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:01.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:01.823+0000 7fbeb8ff9640 1 -- 172.21.15.113:0/3679928765 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbeb0058440 con 0x7fbebc0d0b00 2024-04-25T00:06:02.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.061+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbebc074f50 con 0x7fbebc0d0b00 2024-04-25T00:06:02.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.062+0000 7fbeb8ff9640 1 -- 172.21.15.113:0/3679928765 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7fbeb005c740 con 0x7fbebc0d0b00 2024-04-25T00:06:02.063 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:02.063 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":487,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566146,"bytes_used":2548039680,"bytes_avail":770546073600,"bytes_total":773094113280,"write_bytes_sec":52188670,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T00:05:57.200778+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:06:02.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbea806be30 msgr2=0x7fbea806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:02.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbea806be30 0x7fbea806e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbeac034000 tx=0x7fbeac02e000 comp rx=0 tx=0).stop 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d0b00 msgr2=0x7fbebc1f37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d0b00 0x7fbebc1f37b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000c370 tx=0x7fbeb000c830 comp rx=0 tx=0).stop 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 shutdown_connections 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbea806be30 0x7fbea806e2f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbebc15e320 0x7fbebc0ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbebc0d2290 0x7fbebc0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 --2- 172.21.15.113:0/3679928765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbebc0d0b00 0x7fbebc1f37b0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 >> 172.21.15.113:0/3679928765 conn(0x7fbebc0b5c20 msgr2=0x7fbebc0d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 shutdown_connections 2024-04-25T00:06:02.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:02.064+0000 7fbec2429640 1 -- 172.21.15.113:0/3679928765 wait complete. 2024-04-25T00:06:02.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T00:06:03.077 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.174+0000 7f97fbb9d640 1 Processor -- start 2024-04-25T00:06:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.175+0000 7f97fbb9d640 1 -- start start 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.175+0000 7f97fbb9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 0x7f97f40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.175+0000 7f97fbb9d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f40d3de0 con 0x7f97f40d2290 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.175+0000 7f97fbb9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f40d2800 con 0x7f97f40d0b00 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.175+0000 7f97fbb9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97f40d3410 con 0x7f97f415e320 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f9138640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f97f40d2290 0x7f97f40b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56674/0 (socket says 172.21.15.113:56674) 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f9138640 1 -- 172.21.15.113:0/2935129729 learned_addr learned my addr 172.21.15.113:0/2935129729 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f9939640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 0x7f97f40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835766331 0 0) 0x7f97f40d3de0 con 0x7f97f40d2290 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97e0003520 con 0x7f97f40d2290 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 27226099 0 0) 0x7f97e0003520 con 0x7f97f40d2290 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 msgr2=0x7f97f40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 0x7f97f40d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 >> v1:172.21.15.178:6789/0 conn(0x7f97f40d0b00 legacy=0x7f97f40b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f40d2ae0 con 0x7f97f40d2290 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f97e4001670 con 0x7f97f40d2290 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 >> v1:172.21.15.113:6789/0 conn(0x7f97f40d2290 legacy=0x7f97f40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 0x7f97e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97e0003f60 0x7f97e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97e0004d40 0x7f97e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97f40d3410 con 0x7f97e0004d40 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97e000bb30 con 0x7f97e0003f60 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97e000bd30 con 0x7f97f415e320 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97fa13a640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97e0004d40 0x7f97e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f9939640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 0x7f97e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.176+0000 7f97f9138640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97e0003f60 0x7f97e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97fa13a640 1 -- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 msgr2=0x7f97e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97fa13a640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f415e320 0x7f97e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97fa13a640 1 -- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97e0003f60 msgr2=0x7f97e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97fa13a640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97e0003f60 0x7f97e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97fa13a640 1 -- 172.21.15.113:0/2935129729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e0003520 con 0x7f97e0004d40 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97fa13a640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97e0004d40 0x7f97e00051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f97ec001750 tx=0x7f97e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82463d1599cdfba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec001370 con 0x7f97e0004d40 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ec0052d0 con 0x7f97e0004d40 2024-04-25T00:06:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.177+0000 7f97f8937640 1 -- 172.21.15.113:0/2935129729 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec02abe0 con 0x7f97e0004d40 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.178+0000 7f97fbb9d640 1 -- 172.21.15.113:0/2935129729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97e0004d40 msgr2=0x7f97e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.178+0000 7f97fbb9d640 1 --2- 172.21.15.113:0/2935129729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97e0004d40 0x7f97e00051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f97ec001750 tx=0x7f97e0005830 comp rx=0 tx=0).stop 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.178+0000 7f97fa13a640 1 -- 172.21.15.113:0/2935129729 reap_dead start 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.178+0000 7f97fbb9d640 1 -- 172.21.15.113:0/2935129729 shutdown_connections 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.178+0000 7f97fbb9d640 1 -- 172.21.15.113:0/2935129729 >> 172.21.15.113:0/2935129729 conn(0x7f97f40b5c20 msgr2=0x7f97f40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.178+0000 7f97fbb9d640 1 -- 172.21.15.113:0/2935129729 shutdown_connections 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.178+0000 7f97fbb9d640 1 -- 172.21.15.113:0/2935129729 wait complete. 2024-04-25T00:06:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.179+0000 7f97fbb9d640 1 Processor -- start 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.179+0000 7f97fbb9d640 1 -- start start 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97fbb9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f4160f50 0x7f97f41541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97fbb9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f4154730 0x7f97f4156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97fbb9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97f4156760 0x7f97f415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97fbb9d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97f415e700 con 0x7f97f4160f50 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97fbb9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97f40d30a0 con 0x7f97f4156760 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97fbb9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97f415e880 con 0x7f97f4154730 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f4160f50 0x7f97f41541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f4160f50 0x7f97f41541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48236/0 (socket says 172.21.15.113:48236) 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 -- 172.21.15.113:0/3947046024 learned_addr learned my addr 172.21.15.113:0/3947046024 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 -- 172.21.15.113:0/3947046024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f4154730 msgr2=0x7f97f4156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f4154730 0x7f97f4156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 -- 172.21.15.113:0/3947046024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97f4156760 msgr2=0x7f97f415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97f4156760 0x7f97f415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 -- 172.21.15.113:0/3947046024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f40cbb10 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97f9939640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f4160f50 0x7f97f41541f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f97f0011e30 tx=0x7f97f000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.180+0000 7f97ea7fc640 1 -- 172.21.15.113:0/3947046024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0015630 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.181+0000 7f97ea7fc640 1 -- 172.21.15.113:0/3947046024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f00157d0 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.181+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f40cbdb0 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.181+0000 7f97ea7fc640 1 -- 172.21.15.113:0/3947046024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0015ba0 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.181+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f40cc370 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.182+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f40691a0 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.182+0000 7f97ea7fc640 1 -- 172.21.15.113:0/3947046024 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97f001d400 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.182+0000 7f97ea7fc640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97d806be30 0x7f97d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.182+0000 7f97f9138640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97d806be30 0x7f97d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.182+0000 7f97ea7fc640 1 -- 172.21.15.113:0/3947046024 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f97f008c070 con 0x7f97f4160f50 2024-04-25T00:06:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.183+0000 7f97f9138640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97d806be30 0x7f97d806e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f97e4001050 tx=0x7f97e4001b60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:03.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.188+0000 7f97ea7fc640 1 -- 172.21.15.113:0/3947046024 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97f0019030 con 0x7f97f4160f50 2024-04-25T00:06:03.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.395+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97f40b99b0 con 0x7f97f4160f50 2024-04-25T00:06:03.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.396+0000 7f97ea7fc640 1 -- 172.21.15.113:0/3947046024 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f97f0010040 con 0x7f97f4160f50 2024-04-25T00:06:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:03.398 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:06:03.398 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:06:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97d806be30 msgr2=0x7f97d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97d806be30 0x7f97d806e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f97e4001050 tx=0x7f97e4001b60 comp rx=0 tx=0).stop 2024-04-25T00:06:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f4160f50 msgr2=0x7f97f41541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f4160f50 0x7f97f41541f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f97f0011e30 tx=0x7f97f000f580 comp rx=0 tx=0).stop 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 shutdown_connections 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97d806be30 0x7f97d806e2f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97f4156760 0x7f97f415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f4154730 0x7f97f4156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 --2- 172.21.15.113:0/3947046024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f4160f50 0x7f97f41541f0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 >> 172.21.15.113:0/3947046024 conn(0x7f97f40b5c20 msgr2=0x7f97f415d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 shutdown_connections 2024-04-25T00:06:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.399+0000 7f97fbb9d640 1 -- 172.21.15.113:0/3947046024 wait complete. 2024-04-25T00:06:03.410 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T00:06:03.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:06:03.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.504+0000 7fadd3390640 1 Processor -- start 2024-04-25T00:06:03.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd3390640 1 -- start start 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd3390640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fadcc15e320 0x7fadcc0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd3390640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadcc0d2290 0x7fadcc0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd3390640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadcc0d3de0 con 0x7fadcc0d0b00 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd3390640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fadcc0d3410 con 0x7fadcc0d2290 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd3390640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fadcc0d2800 con 0x7fadcc15e320 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd092b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadcc0d2290 0x7fadcc0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd092b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadcc0d2290 0x7fadcc0b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48242/0 (socket says 172.21.15.113:48242) 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.505+0000 7fadd092b640 1 -- 172.21.15.113:0/340228853 learned_addr learned my addr 172.21.15.113:0/340228853 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:03.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.506+0000 7fadd092b640 1 -- 172.21.15.113:0/340228853 >> v2:172.21.15.178:3300/0 conn(0x7fadcc15e320 msgr2=0x7fadcc0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:03.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.506+0000 7fadd092b640 1 --2- 172.21.15.113:0/340228853 >> v2:172.21.15.178:3300/0 conn(0x7fadcc15e320 0x7fadcc0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.506+0000 7fadd092b640 1 -- 172.21.15.113:0/340228853 >> v1:172.21.15.178:6789/0 conn(0x7fadcc0d0b00 legacy=0x7fadcc0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:03.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.506+0000 7fadd092b640 1 -- 172.21.15.113:0/340228853 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc0d2ae0 con 0x7fadcc0d2290 2024-04-25T00:06:03.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.506+0000 7fadd092b640 1 --2- 172.21.15.113:0/340228853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d2290 0x7fadcc0b9aa0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0b6d00 tx=0x7fadbc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d33d5a9377d14e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:03.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.506+0000 7fadc3fff640 1 -- 172.21.15.113:0/340228853 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc037070 con 0x7fadcc0d2290 2024-04-25T00:06:03.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadc3fff640 1 -- 172.21.15.113:0/340228853 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc032440 con 0x7fadcc0d2290 2024-04-25T00:06:03.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadc3fff640 1 -- 172.21.15.113:0/340228853 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc032810 con 0x7fadcc0d2290 2024-04-25T00:06:03.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 -- 172.21.15.113:0/340228853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d2290 msgr2=0x7fadcc0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 --2- 172.21.15.113:0/340228853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d2290 0x7fadcc0b9aa0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0b6d00 tx=0x7fadbc02a2c0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 -- 172.21.15.113:0/340228853 shutdown_connections 2024-04-25T00:06:03.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 --2- 172.21.15.113:0/340228853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d2290 0x7fadcc0b9aa0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 --2- 172.21.15.113:0/340228853 >> v2:172.21.15.178:3300/0 conn(0x7fadcc15e320 0x7fadcc0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 -- 172.21.15.113:0/340228853 >> 172.21.15.113:0/340228853 conn(0x7fadcc0b5c20 msgr2=0x7fadcc0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 -- 172.21.15.113:0/340228853 shutdown_connections 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.507+0000 7fadd3390640 1 -- 172.21.15.113:0/340228853 wait complete. 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.508+0000 7fadd3390640 1 Processor -- start 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.508+0000 7fadd3390640 1 -- start start 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd3390640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d0b00 0x7fadcc15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd3390640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadcc0d2290 0x7fadcc154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd3390640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadcc15e320 0x7fadcc0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd3390640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc0b9fe0 con 0x7fadcc0d0b00 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd3390640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc0d30a0 con 0x7fadcc0d2290 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd3390640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadcc15e700 con 0x7fadcc15e320 2024-04-25T00:06:03.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d0b00 0x7fadcc15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d0b00 0x7fadcc15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48258/0 (socket says 172.21.15.113:48258) 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 -- 172.21.15.113:0/1245327816 learned_addr learned my addr 172.21.15.113:0/1245327816 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 -- 172.21.15.113:0/1245327816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadcc15e320 msgr2=0x7fadcc0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadcc15e320 0x7fadcc0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 -- 172.21.15.113:0/1245327816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadcc0d2290 msgr2=0x7fadcc154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadcc0d2290 0x7fadcc154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 -- 172.21.15.113:0/1245327816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc1588a0 con 0x7fadcc0d0b00 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.509+0000 7fadd112c640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d0b00 0x7fadcc15b090 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fadc800b440 tx=0x7fadc800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.510+0000 7fadc1ffb640 1 -- 172.21.15.113:0/1245327816 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8015020 con 0x7fadcc0d0b00 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.510+0000 7fadc1ffb640 1 -- 172.21.15.113:0/1245327816 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc8011890 con 0x7fadcc0d0b00 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.510+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc1f40d0 con 0x7fadcc0d0b00 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.510+0000 7fadc1ffb640 1 -- 172.21.15.113:0/1245327816 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8011c80 con 0x7fadcc0d0b00 2024-04-25T00:06:03.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.510+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadcc1f4690 con 0x7fadcc0d0b00 2024-04-25T00:06:03.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.511+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadcc0691a0 con 0x7fadcc0d0b00 2024-04-25T00:06:03.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.511+0000 7fadc1ffb640 1 -- 172.21.15.113:0/1245327816 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadc8018020 con 0x7fadcc0d0b00 2024-04-25T00:06:03.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.511+0000 7fadc1ffb640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fadb006be30 0x7fadb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:03.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.512+0000 7fadc1ffb640 1 -- 172.21.15.113:0/1245327816 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7fadc808b000 con 0x7fadcc0d0b00 2024-04-25T00:06:03.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.512+0000 7fadd092b640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fadb006be30 0x7fadb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:03.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.512+0000 7fadd092b640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fadb006be30 0x7fadb006e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fadbc034000 tx=0x7fadbc02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:03.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.516+0000 7fadc1ffb640 1 -- 172.21.15.113:0/1245327816 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fadc801f020 con 0x7fadcc0d0b00 2024-04-25T00:06:03.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.754+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fadcc15db00 con 0x7fadcc0d0b00 2024-04-25T00:06:03.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.755+0000 7fadc1ffb640 1 -- 172.21.15.113:0/1245327816 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1575 (secure 0 0 0) 0x7fadc800d040 con 0x7fadcc0d0b00 2024-04-25T00:06:03.756 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:03.756 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":488,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":118,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566146,"bytes_used":2548039680,"bytes_avail":770546073600,"bytes_total":773094113280,"write_bytes_sec":12616206,"read_op_per_sec":0,"write_op_per_sec":13},"fsmap":{"epoch":62,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T00:06:03.202017+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fadb006be30 msgr2=0x7fadb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fadb006be30 0x7fadb006e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fadbc034000 tx=0x7fadbc02c000 comp rx=0 tx=0).stop 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d0b00 msgr2=0x7fadcc15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d0b00 0x7fadcc15b090 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fadc800b440 tx=0x7fadc800b900 comp rx=0 tx=0).stop 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 shutdown_connections 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fadb006be30 0x7fadb006e2f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadcc0d2290 0x7fadcc154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadcc15e320 0x7fadcc0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 --2- 172.21.15.113:0/1245327816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc0d0b00 0x7fadcc15b090 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 >> 172.21.15.113:0/1245327816 conn(0x7fadcc0b5c20 msgr2=0x7fadcc0b7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:03.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 shutdown_connections 2024-04-25T00:06:03.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:03.757+0000 7fadd3390640 1 -- 172.21.15.113:0/1245327816 wait complete. 2024-04-25T00:06:03.769 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:06:03.769 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:06:03.769 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:06:03.806 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:06:03.869 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:06:03.869 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:06:03.869 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:06:03.870 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:03.886 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:06:03.886 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:03.941 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:04.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.077+0000 7f83ca212640 1 Processor -- start 2024-04-25T00:06:04.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.077+0000 7f83ca212640 1 -- start start 2024-04-25T00:06:04.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83ca212640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f83c4144910 0x7f83c40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83ca212640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83c40b56b0 0x7f83c40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83ca212640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c4144cf0 con 0x7f83c40b5240 2024-04-25T00:06:04.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83ca212640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f83c40b4510 con 0x7f83c40b56b0 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83ca212640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f83c40b2f30 con 0x7f83c4144910 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83c40b56b0 0x7f83c40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83c40b56b0 0x7f83c40cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48274/0 (socket says 172.21.15.113:48274) 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c3fff640 1 -- 172.21.15.113:0/1449325660 learned_addr learned my addr 172.21.15.113:0/1449325660 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c37fe640 1 --2- 172.21.15.113:0/1449325660 >> v2:172.21.15.178:3300/0 conn(0x7f83c4144910 0x7f83c40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c3fff640 1 -- 172.21.15.113:0/1449325660 >> v2:172.21.15.178:3300/0 conn(0x7f83c4144910 msgr2=0x7f83c40b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c3fff640 1 --2- 172.21.15.113:0/1449325660 >> v2:172.21.15.178:3300/0 conn(0x7f83c4144910 0x7f83c40b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c3fff640 1 -- 172.21.15.113:0/1449325660 >> v1:172.21.15.178:6789/0 conn(0x7f83c40b5240 legacy=0x7f83c406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c3fff640 1 -- 172.21.15.113:0/1449325660 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c40b3210 con 0x7f83c40b56b0 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.078+0000 7f83c37fe640 1 --2- 172.21.15.113:0/1449325660 >> v2:172.21.15.178:3300/0 conn(0x7f83c4144910 0x7f83c40b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.079+0000 7f83c3fff640 1 --2- 172.21.15.113:0/1449325660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c40b56b0 0x7f83c40cd500 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009e50 tx=0x7f83bc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa122fab9895c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.079+0000 7f83c27fc640 1 -- 172.21.15.113:0/1449325660 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc036070 con 0x7f83c40b56b0 2024-04-25T00:06:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.079+0000 7f83c27fc640 1 -- 172.21.15.113:0/1449325660 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83bc031600 con 0x7f83c40b56b0 2024-04-25T00:06:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.079+0000 7f83c27fc640 1 -- 172.21.15.113:0/1449325660 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0319d0 con 0x7f83c40b56b0 2024-04-25T00:06:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 -- 172.21.15.113:0/1449325660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c40b56b0 msgr2=0x7f83c40cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 --2- 172.21.15.113:0/1449325660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c40b56b0 0x7f83c40cd500 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009e50 tx=0x7f83bc02a710 comp rx=0 tx=0).stop 2024-04-25T00:06:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 -- 172.21.15.113:0/1449325660 shutdown_connections 2024-04-25T00:06:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 --2- 172.21.15.113:0/1449325660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c40b56b0 0x7f83c40cd500 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 --2- 172.21.15.113:0/1449325660 >> v2:172.21.15.178:3300/0 conn(0x7f83c4144910 0x7f83c40b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 -- 172.21.15.113:0/1449325660 >> 172.21.15.113:0/1449325660 conn(0x7f83c40b1a50 msgr2=0x7f83c40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 -- 172.21.15.113:0/1449325660 shutdown_connections 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.080+0000 7f83ca212640 1 -- 172.21.15.113:0/1449325660 wait complete. 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.081+0000 7f83ca212640 1 Processor -- start 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.081+0000 7f83ca212640 1 -- start start 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83c40b5240 0x7f83c413d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83c40b56b0 0x7f83c415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c4144910 0x7f83c415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b4da0 con 0x7f83c4144910 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b37d0 con 0x7f83c40b56b0 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83c40b4f20 con 0x7f83c40b5240 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c4144910 0x7f83c415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c4144910 0x7f83c415d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48280/0 (socket says 172.21.15.113:48280) 2024-04-25T00:06:04.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 -- 172.21.15.113:0/1623921224 learned_addr learned my addr 172.21.15.113:0/1623921224 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c2ffd640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83c40b56b0 0x7f83c415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c37fe640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83c40b5240 0x7f83c413d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 -- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83c40b5240 msgr2=0x7f83c413d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83c40b5240 0x7f83c413d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 -- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83c40b56b0 msgr2=0x7f83c415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83c40b56b0 0x7f83c415c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 -- 172.21.15.113:0/1623921224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c4140290 con 0x7f83c4144910 2024-04-25T00:06:04.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83c3fff640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c4144910 0x7f83c415d710 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009f70 tx=0x7f83bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83abfff640 1 -- 172.21.15.113:0/1623921224 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc036050 con 0x7f83c4144910 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83abfff640 1 -- 172.21.15.113:0/1623921224 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83bc042070 con 0x7f83c4144910 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c413e500 con 0x7f83c4144910 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83abfff640 1 -- 172.21.15.113:0/1623921224 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc009510 con 0x7f83c4144910 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.082+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c413e220 con 0x7f83c4144910 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.083+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c40691a0 con 0x7f83c4144910 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.084+0000 7f83abfff640 1 -- 172.21.15.113:0/1623921224 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83bc007bf0 con 0x7f83c4144910 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.084+0000 7f83abfff640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b006be30 0x7f83b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.084+0000 7f83abfff640 1 -- 172.21.15.113:0/1623921224 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f83bc0abb70 con 0x7f83c4144910 2024-04-25T00:06:04.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.084+0000 7f83c37fe640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b006be30 0x7f83b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.085+0000 7f83c37fe640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b006be30 0x7f83b006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f83b400aac0 tx=0x7f83b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.088+0000 7f83abfff640 1 -- 172.21.15.113:0/1623921224 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83bc07a040 con 0x7f83c4144910 2024-04-25T00:06:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.260+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83c40b04a0 con 0x7f83c4144910 2024-04-25T00:06:04.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.261+0000 7f83abfff640 1 -- 172.21.15.113:0/1623921224 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16829 (secure 0 0 0) 0x7f83bc07e340 con 0x7f83c4144910 2024-04-25T00:06:04.262 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:04.262 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":118,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:02.284662+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:05:33.644000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:05:34.412894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.21","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:04.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.263+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b006be30 msgr2=0x7f83b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.263+0000 7f83ca212640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b006be30 0x7f83b006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f83b400aac0 tx=0x7f83b4008040 comp rx=0 tx=0).stop 2024-04-25T00:06:04.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.263+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c4144910 msgr2=0x7f83c415d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.263+0000 7f83ca212640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c4144910 0x7f83c415d710 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009f70 tx=0x7f83bc004a40 comp rx=0 tx=0).stop 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.263+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 shutdown_connections 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.263+0000 7f83ca212640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b006be30 0x7f83b006e2f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.264+0000 7f83ca212640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83c40b56b0 0x7f83c415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.264+0000 7f83ca212640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c4144910 0x7f83c415d710 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.264+0000 7f83ca212640 1 --2- 172.21.15.113:0/1623921224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83c40b5240 0x7f83c413d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.264+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 >> 172.21.15.113:0/1623921224 conn(0x7f83c40b1a50 msgr2=0x7f83c4071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.264+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 shutdown_connections 2024-04-25T00:06:04.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.264+0000 7f83ca212640 1 -- 172.21.15.113:0/1623921224 wait complete. 2024-04-25T00:06:04.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:05:33.644000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:05:34.412894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:06:04.276 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:06:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.376+0000 7efd9442c640 1 Processor -- start 2024-04-25T00:06:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.376+0000 7efd9442c640 1 -- start start 2024-04-25T00:06:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd9442c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd8c13f050 0x7efd8c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd9442c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c1606a0 0x7efd8c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd9442c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8c15d6b0 con 0x7efd8c0d04f0 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd9442c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efd8c0d09a0 con 0x7efd8c13f050 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd9442c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c15c6a0 con 0x7efd8c1606a0 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd929c9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c1606a0 0x7efd8c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd921c8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd8c13f050 0x7efd8c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd929c9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c1606a0 0x7efd8c0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:45676/0 (socket says 172.21.15.113:45676) 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd929c9640 1 -- 172.21.15.113:0/2337037194 learned_addr learned my addr 172.21.15.113:0/2337037194 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd929c9640 1 -- 172.21.15.113:0/2337037194 >> v2:172.21.15.178:3300/0 conn(0x7efd8c13f050 msgr2=0x7efd8c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd929c9640 1 --2- 172.21.15.113:0/2337037194 >> v2:172.21.15.178:3300/0 conn(0x7efd8c13f050 0x7efd8c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd929c9640 1 -- 172.21.15.113:0/2337037194 >> v1:172.21.15.178:6789/0 conn(0x7efd8c0d04f0 legacy=0x7efd8c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd929c9640 1 -- 172.21.15.113:0/2337037194 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c15c8f0 con 0x7efd8c1606a0 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd911c6640 1 -- 172.21.15.113:0/2337037194 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820003412 0 0) 0x7efd8c15d6b0 con 0x7efd8c0d04f0 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.377+0000 7efd921c8640 1 --2- 172.21.15.113:0/2337037194 >> v2:172.21.15.178:3300/0 conn(0x7efd8c13f050 0x7efd8c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.378+0000 7efd929c9640 1 --2- 172.21.15.113:0/2337037194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c1606a0 0x7efd8c0cea60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd84009e50 tx=0x7efd8402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62c23f39f2bbd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.378+0000 7efd911c6640 1 -- 172.21.15.113:0/2337037194 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84036070 con 0x7efd8c1606a0 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.378+0000 7efd911c6640 1 -- 172.21.15.113:0/2337037194 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd84031600 con 0x7efd8c1606a0 2024-04-25T00:06:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.378+0000 7efd911c6640 1 -- 172.21.15.113:0/2337037194 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840319d0 con 0x7efd8c1606a0 2024-04-25T00:06:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 -- 172.21.15.113:0/2337037194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c1606a0 msgr2=0x7efd8c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 --2- 172.21.15.113:0/2337037194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c1606a0 0x7efd8c0cea60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd84009e50 tx=0x7efd8402a680 comp rx=0 tx=0).stop 2024-04-25T00:06:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 -- 172.21.15.113:0/2337037194 shutdown_connections 2024-04-25T00:06:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 --2- 172.21.15.113:0/2337037194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c1606a0 0x7efd8c0cea60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 --2- 172.21.15.113:0/2337037194 >> v2:172.21.15.178:3300/0 conn(0x7efd8c13f050 0x7efd8c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 -- 172.21.15.113:0/2337037194 >> 172.21.15.113:0/2337037194 conn(0x7efd8c06b0f0 msgr2=0x7efd8c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 -- 172.21.15.113:0/2337037194 shutdown_connections 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.379+0000 7efd9442c640 1 -- 172.21.15.113:0/2337037194 wait complete. 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.380+0000 7efd9442c640 1 Processor -- start 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.380+0000 7efd9442c640 1 -- start start 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd9442c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c0d04f0 0x7efd8c159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd9442c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c13f050 0x7efd8c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd9442c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd8c1606a0 0x7efd8c1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd9442c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c15d190 con 0x7efd8c13f050 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd9442c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c15ceb0 con 0x7efd8c1606a0 2024-04-25T00:06:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd9442c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c13f430 con 0x7efd8c0d04f0 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c13f050 0x7efd8c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c13f050 0x7efd8c1521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48294/0 (socket says 172.21.15.113:48294) 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 -- 172.21.15.113:0/4052049345 learned_addr learned my addr 172.21.15.113:0/4052049345 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd921c8640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c0d04f0 0x7efd8c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd929c9640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd8c1606a0 0x7efd8c1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 -- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c0d04f0 msgr2=0x7efd8c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c0d04f0 0x7efd8c159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 -- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd8c1606a0 msgr2=0x7efd8c1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd8c1606a0 0x7efd8c1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 -- 172.21.15.113:0/4052049345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c157bf0 con 0x7efd8c13f050 2024-04-25T00:06:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd919c7640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c13f050 0x7efd8c1521b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c4e0 tx=0x7efd7c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.381+0000 7efd82ffd640 1 -- 172.21.15.113:0/4052049345 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c017070 con 0x7efd8c13f050 2024-04-25T00:06:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.382+0000 7efd82ffd640 1 -- 172.21.15.113:0/4052049345 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c0035f0 con 0x7efd8c13f050 2024-04-25T00:06:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.382+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1f3800 con 0x7efd8c13f050 2024-04-25T00:06:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.382+0000 7efd82ffd640 1 -- 172.21.15.113:0/4052049345 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0039e0 con 0x7efd8c13f050 2024-04-25T00:06:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.382+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c157ed0 con 0x7efd8c13f050 2024-04-25T00:06:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.382+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c0691a0 con 0x7efd8c13f050 2024-04-25T00:06:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.383+0000 7efd82ffd640 1 -- 172.21.15.113:0/4052049345 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd7c01a070 con 0x7efd8c13f050 2024-04-25T00:06:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.383+0000 7efd82ffd640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd7006be60 0x7efd7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.383+0000 7efd82ffd640 1 -- 172.21.15.113:0/4052049345 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7efd7c08a0d0 con 0x7efd8c13f050 2024-04-25T00:06:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.383+0000 7efd921c8640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd7006be60 0x7efd7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.384+0000 7efd921c8640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd7006be60 0x7efd7006e320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd88009560 tx=0x7efd88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.388+0000 7efd82ffd640 1 -- 172.21.15.113:0/4052049345 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd7c058480 con 0x7efd8c13f050 2024-04-25T00:06:04.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.560+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd8c0d0e50 con 0x7efd8c13f050 2024-04-25T00:06:04.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.561+0000 7efd82ffd640 1 -- 172.21.15.113:0/4052049345 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7efd7c05c780 con 0x7efd8c13f050 2024-04-25T00:06:04.561 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:06:04.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd7006be60 msgr2=0x7efd7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd7006be60 0x7efd7006e320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd88009560 tx=0x7efd88008040 comp rx=0 tx=0).stop 2024-04-25T00:06:04.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c13f050 msgr2=0x7efd8c1521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c13f050 0x7efd8c1521b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c4e0 tx=0x7efd7c00c9a0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 shutdown_connections 2024-04-25T00:06:04.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd7006be60 0x7efd7006e320 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd8c1606a0 0x7efd8c1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8c13f050 0x7efd8c1521b0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 --2- 172.21.15.113:0/4052049345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd8c0d04f0 0x7efd8c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.562+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 >> 172.21.15.113:0/4052049345 conn(0x7efd8c06b0f0 msgr2=0x7efd8c15ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.563+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 shutdown_connections 2024-04-25T00:06:04.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.563+0000 7efd9442c640 1 -- 172.21.15.113:0/4052049345 wait complete. 2024-04-25T00:06:04.573 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:06:04.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.672+0000 7fabbdcee640 1 Processor -- start 2024-04-25T00:06:04.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.672+0000 7fabbdcee640 1 -- start start 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabbdcee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb80d06c0 0x7fabb815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabbdcee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fabb81604d0 0x7fabb80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabbdcee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb815d6b0 con 0x7fabb813f050 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabbdcee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fabb815c6a0 con 0x7fabb81604d0 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabbdcee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fabb815c8f0 con 0x7fabb80d06c0 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabb7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fabb81604d0 0x7fabb80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabb6ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb80d06c0 0x7fabb815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabb7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fabb81604d0 0x7fabb80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:36086/0 (socket says 172.21.15.113:36086) 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabb6ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb80d06c0 0x7fabb815ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:45682/0 (socket says 172.21.15.113:45682) 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabb7fff640 1 -- 172.21.15.113:0/3333182381 learned_addr learned my addr 172.21.15.113:0/3333182381 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabb77fe640 1 --1- 172.21.15.113:0/3333182381 >> v1:172.21.15.178:6789/0 conn(0x7fabb813f050 0x7fabb80cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:46298/0 (socket says 172.21.15.113:46298) 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.673+0000 7fabb67fc640 1 -- 172.21.15.113:0/3333182381 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4087181462 0 0) 0x7fabb815d6b0 con 0x7fabb813f050 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb67fc640 1 -- 172.21.15.113:0/3333182381 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faba0003520 con 0x7fabb813f050 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb6ffd640 1 -- 172.21.15.113:0/3333182381 >> v2:172.21.15.178:3300/0 conn(0x7fabb81604d0 msgr2=0x7fabb80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb6ffd640 1 --2- 172.21.15.113:0/3333182381 >> v2:172.21.15.178:3300/0 conn(0x7fabb81604d0 0x7fabb80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb6ffd640 1 -- 172.21.15.113:0/3333182381 >> v1:172.21.15.178:6789/0 conn(0x7fabb813f050 legacy=0x7fabb80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb6ffd640 1 -- 172.21.15.113:0/3333182381 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb815cbd0 con 0x7fabb80d06c0 2024-04-25T00:06:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb7fff640 1 --2- 172.21.15.113:0/3333182381 >> v2:172.21.15.178:3300/0 conn(0x7fabb81604d0 0x7fabb80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb6ffd640 1 --2- 172.21.15.113:0/3333182381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb80d06c0 0x7fabb815ff90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabac0097a0 tx=0x7fabac02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89197695a938b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.674+0000 7fabb67fc640 1 -- 172.21.15.113:0/3333182381 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac037070 con 0x7fabb80d06c0 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabb67fc640 1 -- 172.21.15.113:0/3333182381 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabac02ae50 con 0x7fabb80d06c0 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabb67fc640 1 -- 172.21.15.113:0/3333182381 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac032640 con 0x7fabb80d06c0 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 -- 172.21.15.113:0/3333182381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb80d06c0 msgr2=0x7fabb815ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 --2- 172.21.15.113:0/3333182381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb80d06c0 0x7fabb815ff90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabac0097a0 tx=0x7fabac02a100 comp rx=0 tx=0).stop 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 -- 172.21.15.113:0/3333182381 shutdown_connections 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 --2- 172.21.15.113:0/3333182381 >> v2:172.21.15.178:3300/0 conn(0x7fabb81604d0 0x7fabb80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 --2- 172.21.15.113:0/3333182381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb80d06c0 0x7fabb815ff90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 -- 172.21.15.113:0/3333182381 >> 172.21.15.113:0/3333182381 conn(0x7fabb806b0f0 msgr2=0x7fabb806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 -- 172.21.15.113:0/3333182381 shutdown_connections 2024-04-25T00:06:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.675+0000 7fabbdcee640 1 -- 172.21.15.113:0/3333182381 wait complete. 2024-04-25T00:06:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.676+0000 7fabbdcee640 1 Processor -- start 2024-04-25T00:06:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.676+0000 7fabbdcee640 1 -- start start 2024-04-25T00:06:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabbdcee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabb80d06c0 0x7fabb814d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabbdcee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb813f050 0x7fabb813b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabbdcee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb81604d0 0x7fabb813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabbdcee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fabb813f430 con 0x7fabb81604d0 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabbdcee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fabb815d190 con 0x7fabb80d06c0 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabbdcee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fabb81608d0 con 0x7fabb813f050 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb81604d0 0x7fabb813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb81604d0 0x7fabb813b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48300/0 (socket says 172.21.15.113:48300) 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabb7fff640 1 -- 172.21.15.113:0/2830350744 learned_addr learned my addr 172.21.15.113:0/2830350744 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabb77fe640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabb80d06c0 0x7fabb814d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.677+0000 7fabb6ffd640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb813f050 0x7fabb813b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabb7fff640 1 -- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb813f050 msgr2=0x7fabb813b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabb7fff640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb813f050 0x7fabb813b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabb7fff640 1 -- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabb80d06c0 msgr2=0x7fabb814d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabb7fff640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabb80d06c0 0x7fabb814d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabb7fff640 1 -- 172.21.15.113:0/2830350744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb80b4a00 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabb7fff640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb81604d0 0x7fabb813b6b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fabb000efc0 tx=0x7fabb000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fab9ffff640 1 -- 172.21.15.113:0/2830350744 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb000cbe0 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fab9ffff640 1 -- 172.21.15.113:0/2830350744 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb000cd80 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fab9ffff640 1 -- 172.21.15.113:0/2830350744 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0003bb0 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb814bbc0 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.678+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb814c180 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.679+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb80691a0 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.679+0000 7fab9ffff640 1 -- 172.21.15.113:0/2830350744 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fabb001b030 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.680+0000 7fab9ffff640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faba406be30 0x7faba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.680+0000 7fabb77fe640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faba406be30 0x7faba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.680+0000 7fab9ffff640 1 -- 172.21.15.113:0/2830350744 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7fabb008b2c0 con 0x7fabb81604d0 2024-04-25T00:06:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.680+0000 7fabb77fe640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faba406be30 0x7faba406e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7faba8000f60 tx=0x7faba8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.684+0000 7fab9ffff640 1 -- 172.21.15.113:0/2830350744 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fabb0022020 con 0x7fabb81604d0 2024-04-25T00:06:04.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.854+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fabb80d0e50 con 0x7fabb81604d0 2024-04-25T00:06:04.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.854+0000 7fab9ffff640 1 -- 172.21.15.113:0/2830350744 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7fabb0010040 con 0x7fabb81604d0 2024-04-25T00:06:04.855 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:06:04.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faba406be30 msgr2=0x7faba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faba406be30 0x7faba406e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7faba8000f60 tx=0x7faba8009040 comp rx=0 tx=0).stop 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb81604d0 msgr2=0x7fabb813b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb81604d0 0x7fabb813b6b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fabb000efc0 tx=0x7fabb000c490 comp rx=0 tx=0).stop 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 shutdown_connections 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faba406be30 0x7faba406e2f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb81604d0 0x7fabb813b6b0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabb813f050 0x7fabb813b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 --2- 172.21.15.113:0/2830350744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabb80d06c0 0x7fabb814d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 >> 172.21.15.113:0/2830350744 conn(0x7fabb806b0f0 msgr2=0x7fabb8151590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 shutdown_connections 2024-04-25T00:06:04.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.856+0000 7fabbdcee640 1 -- 172.21.15.113:0/2830350744 wait complete. 2024-04-25T00:06:04.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:06:04.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.966+0000 7fa763027640 1 Processor -- start 2024-04-25T00:06:04.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.966+0000 7fa763027640 1 -- start start 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa763027640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c15e2d0 0x7fa75c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa763027640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa75c0b4a40 0x7fa75c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa763027640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa75c15f4f0 con 0x7fa75c0b3080 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa763027640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa75c0cde00 con 0x7fa75c0b4a40 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa763027640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa75c15f7d0 con 0x7fa75c15e2d0 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa7615c4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa75c0b4a40 0x7fa75c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa760dc3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c15e2d0 0x7fa75c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa753fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa75c0b3080 0x7fa75c0b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:46302/0 (socket says 172.21.15.113:46302) 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.967+0000 7fa753fff640 1 -- 172.21.15.113:0/2987414196 learned_addr learned my addr 172.21.15.113:0/2987414196 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:04.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.968+0000 7fa7615c4640 1 -- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c15e2d0 msgr2=0x7fa75c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.968+0000 7fa7615c4640 1 --2- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c15e2d0 0x7fa75c15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.968+0000 7fa7615c4640 1 -- 172.21.15.113:0/2987414196 >> v1:172.21.15.178:6789/0 conn(0x7fa75c0b3080 legacy=0x7fa75c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.968+0000 7fa7615c4640 1 -- 172.21.15.113:0/2987414196 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c15fab0 con 0x7fa75c0b4a40 2024-04-25T00:06:04.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.968+0000 7fa7537fe640 1 -- 172.21.15.113:0/2987414196 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756518724 0 0) 0x7fa75c15f4f0 con 0x7fa75c0b3080 2024-04-25T00:06:04.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.968+0000 7fa760dc3640 1 --2- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c15e2d0 0x7fa75c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:04.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.968+0000 7fa7615c4640 1 --2- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 0x7fa75c0ce290 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa754009d90 tx=0x7fa75402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a88cda36cdf796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa7537fe640 1 -- 172.21.15.113:0/2987414196 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754007740 con 0x7fa75c0b4a40 2024-04-25T00:06:04.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa7537fe640 1 -- 172.21.15.113:0/2987414196 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7540078e0 con 0x7fa75c0b4a40 2024-04-25T00:06:04.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa7537fe640 1 -- 172.21.15.113:0/2987414196 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754007cb0 con 0x7fa75c0b4a40 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 -- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 msgr2=0x7fa75c0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 --2- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 0x7fa75c0ce290 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa754009d90 tx=0x7fa75402a660 comp rx=0 tx=0).stop 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 -- 172.21.15.113:0/2987414196 shutdown_connections 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 --2- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 0x7fa75c0ce290 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 --2- 172.21.15.113:0/2987414196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c15e2d0 0x7fa75c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 -- 172.21.15.113:0/2987414196 >> 172.21.15.113:0/2987414196 conn(0x7fa75c06b0f0 msgr2=0x7fa75c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 -- 172.21.15.113:0/2987414196 shutdown_connections 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.969+0000 7fa763027640 1 -- 172.21.15.113:0/2987414196 wait complete. 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.970+0000 7fa763027640 1 Processor -- start 2024-04-25T00:06:04.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.970+0000 7fa763027640 1 -- start start 2024-04-25T00:06:04.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa763027640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c0b3080 0x7fa75c0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa763027640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 0x7fa75c1f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa763027640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa75c15e2d0 0x7fa75c0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa763027640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c15f2b0 con 0x7fa75c15e2d0 2024-04-25T00:06:04.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa763027640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c15f130 con 0x7fa75c0b4a40 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa763027640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa75c15e6b0 con 0x7fa75c0b3080 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa7615c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa75c15e2d0 0x7fa75c0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa7615c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa75c15e2d0 0x7fa75c0aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48316/0 (socket says 172.21.15.113:48316) 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa7615c4640 1 -- 172.21.15.113:0/3885568640 learned_addr learned my addr 172.21.15.113:0/3885568640 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa753fff640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 0x7fa75c1f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.971+0000 7fa760dc3640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c0b3080 0x7fa75c0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7615c4640 1 -- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c0b3080 msgr2=0x7fa75c0a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7615c4640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c0b3080 0x7fa75c0a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7615c4640 1 -- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 msgr2=0x7fa75c1f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7615c4640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 0x7fa75c1f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7615c4640 1 -- 172.21.15.113:0/3885568640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c1f2990 con 0x7fa75c15e2d0 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7615c4640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa75c15e2d0 0x7fa75c0aecf0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa754009d90 tx=0x7fa754004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7517fa640 1 -- 172.21.15.113:0/3885568640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754040070 con 0x7fa75c15e2d0 2024-04-25T00:06:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7517fa640 1 -- 172.21.15.113:0/3885568640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa754031c40 con 0x7fa75c15e2d0 2024-04-25T00:06:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c0a9a60 con 0x7fa75c15e2d0 2024-04-25T00:06:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa7517fa640 1 -- 172.21.15.113:0/3885568640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754009450 con 0x7fa75c15e2d0 2024-04-25T00:06:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.972+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c0aa020 con 0x7fa75c15e2d0 2024-04-25T00:06:04.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.973+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75c0691a0 con 0x7fa75c15e2d0 2024-04-25T00:06:04.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.974+0000 7fa7517fa640 1 -- 172.21.15.113:0/3885568640 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa754039040 con 0x7fa75c15e2d0 2024-04-25T00:06:04.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.974+0000 7fa7517fa640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa74806be30 0x7fa74806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.974+0000 7fa7517fa640 1 -- 172.21.15.113:0/3885568640 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7fa7540abfa0 con 0x7fa75c15e2d0 2024-04-25T00:06:04.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.974+0000 7fa760dc3640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa74806be30 0x7fa74806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.974+0000 7fa760dc3640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa74806be30 0x7fa74806e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa758009560 tx=0x7fa758009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:04.978+0000 7fa7517fa640 1 -- 172.21.15.113:0/3885568640 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa75407a2d0 con 0x7fa75c15e2d0 2024-04-25T00:06:05.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.153+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa75c072050 con 0x7fa75c15e2d0 2024-04-25T00:06:05.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.153+0000 7fa7517fa640 1 -- 172.21.15.113:0/3885568640 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7fa75407e5d0 con 0x7fa75c15e2d0 2024-04-25T00:06:05.154 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa74806be30 msgr2=0x7fa74806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa74806be30 0x7fa74806e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa758009560 tx=0x7fa758009190 comp rx=0 tx=0).stop 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa75c15e2d0 msgr2=0x7fa75c0aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa75c15e2d0 0x7fa75c0aecf0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa754009d90 tx=0x7fa754004a40 comp rx=0 tx=0).stop 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 shutdown_connections 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa74806be30 0x7fa74806e2f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c0b4a40 0x7fa75c1f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa75c15e2d0 0x7fa75c0aecf0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 --2- 172.21.15.113:0/3885568640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa75c0b3080 0x7fa75c0a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 >> 172.21.15.113:0/3885568640 conn(0x7fa75c06b0f0 msgr2=0x7fa75c0b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:05.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 shutdown_connections 2024-04-25T00:06:05.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.155+0000 7fa763027640 1 -- 172.21.15.113:0/3885568640 wait complete. 2024-04-25T00:06:05.166 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:06:05.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.264+0000 7f0a798bd640 1 Processor -- start 2024-04-25T00:06:05.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a798bd640 1 -- start start 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a798bd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a7415e510 0x7f0a740d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a798bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a740d2e00 con 0x7f0a740d28c0 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a798bd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a740d30e0 con 0x7f0a740b63b0 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a798bd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0a740d33c0 con 0x7f0a7415e510 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a72ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a7415e510 0x7f0a740d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a72ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a7415e510 0x7f0a740d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48322/0 (socket says 172.21.15.113:48322) 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a72ffd640 1 -- 172.21.15.113:0/1437984879 learned_addr learned my addr 172.21.15.113:0/1437984879 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.265+0000 7f0a727fc640 1 --1- 172.21.15.113:0/1437984879 >> v1:172.21.15.113:6789/0 conn(0x7f0a740d28c0 0x7f0a740b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56688/0 (socket says 172.21.15.113:56688) 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.266+0000 7f0a72ffd640 1 -- 172.21.15.113:0/1437984879 >> v1:172.21.15.178:6789/0 conn(0x7f0a740b63b0 legacy=0x7f0a740b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.266+0000 7f0a72ffd640 1 -- 172.21.15.113:0/1437984879 >> v1:172.21.15.113:6789/0 conn(0x7f0a740d28c0 legacy=0x7f0a740b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.266+0000 7f0a72ffd640 1 -- 172.21.15.113:0/1437984879 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a740d36a0 con 0x7f0a7415e510 2024-04-25T00:06:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.266+0000 7f0a71ffb640 1 -- 172.21.15.113:0/1437984879 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264787361 0 0) 0x7f0a740d2e00 con 0x7f0a740d28c0 2024-04-25T00:06:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.266+0000 7f0a72ffd640 1 --2- 172.21.15.113:0/1437984879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a7415e510 0x7f0a740d2380 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009e50 tx=0x7f0a6402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad767a7d44f01e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.266+0000 7f0a71ffb640 1 -- 172.21.15.113:0/1437984879 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64036070 con 0x7f0a7415e510 2024-04-25T00:06:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a71ffb640 1 -- 172.21.15.113:0/1437984879 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a64031600 con 0x7f0a7415e510 2024-04-25T00:06:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a71ffb640 1 -- 172.21.15.113:0/1437984879 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a640319d0 con 0x7f0a7415e510 2024-04-25T00:06:05.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a798bd640 1 -- 172.21.15.113:0/1437984879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a7415e510 msgr2=0x7f0a740d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a798bd640 1 --2- 172.21.15.113:0/1437984879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a7415e510 0x7f0a740d2380 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009e50 tx=0x7f0a6402a6d0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a798bd640 1 -- 172.21.15.113:0/1437984879 shutdown_connections 2024-04-25T00:06:05.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a798bd640 1 --2- 172.21.15.113:0/1437984879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a7415e510 0x7f0a740d2380 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a798bd640 1 -- 172.21.15.113:0/1437984879 >> 172.21.15.113:0/1437984879 conn(0x7f0a7406b720 msgr2=0x7f0a740d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a798bd640 1 -- 172.21.15.113:0/1437984879 shutdown_connections 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.267+0000 7f0a798bd640 1 -- 172.21.15.113:0/1437984879 wait complete. 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.268+0000 7f0a798bd640 1 Processor -- start 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.268+0000 7f0a798bd640 1 -- start start 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a798bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a740b63b0 0x7f0a740cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a798bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a740d28c0 0x7f0a741f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a798bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a7415e510 0x7f0a740cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a798bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a740b9dd0 con 0x7f0a740b63b0 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a798bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a740d3ea0 con 0x7f0a740d28c0 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a798bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a740b7ad0 con 0x7f0a7415e510 2024-04-25T00:06:05.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a740b63b0 0x7f0a740cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a740b63b0 0x7f0a740cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48334/0 (socket says 172.21.15.113:48334) 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 -- 172.21.15.113:0/3214186732 learned_addr learned my addr 172.21.15.113:0/3214186732 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 -- 172.21.15.113:0/3214186732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a7415e510 msgr2=0x7f0a740cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a7415e510 0x7f0a740cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 -- 172.21.15.113:0/3214186732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a740d28c0 msgr2=0x7f0a741f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a740d28c0 0x7f0a741f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.269+0000 7f0a72ffd640 1 -- 172.21.15.113:0/3214186732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a740ccd20 con 0x7f0a740b63b0 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.270+0000 7f0a72ffd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a740b63b0 0x7f0a740cddc0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009e50 tx=0x7f0a64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.270+0000 7f0a5bfff640 1 -- 172.21.15.113:0/3214186732 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64041030 con 0x7f0a740b63b0 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.270+0000 7f0a5bfff640 1 -- 172.21.15.113:0/3214186732 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a64031400 con 0x7f0a740b63b0 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.270+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a740cd000 con 0x7f0a740b63b0 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.270+0000 7f0a5bfff640 1 -- 172.21.15.113:0/3214186732 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64036040 con 0x7f0a740b63b0 2024-04-25T00:06:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.270+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a740cd5c0 con 0x7f0a740b63b0 2024-04-25T00:06:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.271+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a74068ae0 con 0x7f0a740b63b0 2024-04-25T00:06:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.271+0000 7f0a5bfff640 1 -- 172.21.15.113:0/3214186732 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a64044070 con 0x7f0a740b63b0 2024-04-25T00:06:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.272+0000 7f0a5bfff640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a6006be30 0x7f0a6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.272+0000 7f0a5bfff640 1 -- 172.21.15.113:0/3214186732 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f0a640abc90 con 0x7f0a740b63b0 2024-04-25T00:06:05.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.272+0000 7f0a727fc640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a6006be30 0x7f0a6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:05.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.272+0000 7f0a727fc640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a6006be30 0x7f0a6006e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0a68007fd0 tx=0x7f0a68007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:05.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.276+0000 7f0a5bfff640 1 -- 172.21.15.113:0/3214186732 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a74068ae0 con 0x7f0a740b63b0 2024-04-25T00:06:05.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.463+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a740b80c0 con 0x7f0a740b63b0 2024-04-25T00:06:05.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.463+0000 7f0a5bfff640 1 -- 172.21.15.113:0/3214186732 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7f0a6407a600 con 0x7f0a740b63b0 2024-04-25T00:06:05.464 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a6006be30 msgr2=0x7f0a6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a6006be30 0x7f0a6006e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0a68007fd0 tx=0x7f0a68007900 comp rx=0 tx=0).stop 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a740b63b0 msgr2=0x7f0a740cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a740b63b0 0x7f0a740cddc0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009e50 tx=0x7f0a64004a40 comp rx=0 tx=0).stop 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 shutdown_connections 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0a6006be30 0x7f0a6006e2f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a740d28c0 0x7f0a741f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a7415e510 0x7f0a740cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 --2- 172.21.15.113:0/3214186732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a740b63b0 0x7f0a740cddc0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:05.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 >> 172.21.15.113:0/3214186732 conn(0x7f0a7406b720 msgr2=0x7f0a7415c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:05.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 shutdown_connections 2024-04-25T00:06:05.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:05.465+0000 7f0a798bd640 1 -- 172.21.15.113:0/3214186732 wait complete. 2024-04-25T00:06:05.476 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:06:05.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:05.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:06:06.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:06:06.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:06.085 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:06:06.104 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:06:06.104 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:06:06.105 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:06.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:06:06.120 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:06:06.120 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:06:06.120 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:06.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:06:06.161 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:06:06.161 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:06:06.162 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:06.177 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:06:06.177 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:06.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.297+0000 7f3ddcbee640 1 Processor -- start 2024-04-25T00:06:06.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.297+0000 7f3ddcbee640 1 -- start start 2024-04-25T00:06:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3ddcbee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3dd80d2290 0x7f3dd80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3ddcbee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3dd80d0b00 0x7f3dd80d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3ddcbee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd80d1480 con 0x7f3dd815e320 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3ddcbee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd80d3410 con 0x7f3dd80d2290 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3ddcbee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd80d2800 con 0x7f3dd80d0b00 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3dd5d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3dd80d2290 0x7f3dd80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3dd659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3dd815e320 0x7f3dd80d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56690/0 (socket says 172.21.15.113:56690) 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.298+0000 7f3dd659c640 1 -- 172.21.15.113:0/376213719 learned_addr learned my addr 172.21.15.113:0/376213719 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd5d9b640 1 -- 172.21.15.113:0/376213719 >> v2:172.21.15.178:3300/0 conn(0x7f3dd80d0b00 msgr2=0x7f3dd80d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd6d9d640 1 --2- 172.21.15.113:0/376213719 >> v2:172.21.15.178:3300/0 conn(0x7f3dd80d0b00 0x7f3dd80d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd559a640 1 -- 172.21.15.113:0/376213719 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441696659 0 0) 0x7f3dd80d1480 con 0x7f3dd815e320 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd5d9b640 1 --2- 172.21.15.113:0/376213719 >> v2:172.21.15.178:3300/0 conn(0x7f3dd80d0b00 0x7f3dd80d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd5d9b640 1 -- 172.21.15.113:0/376213719 >> v1:172.21.15.113:6789/0 conn(0x7f3dd815e320 legacy=0x7f3dd80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd5d9b640 1 -- 172.21.15.113:0/376213719 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd80d2ae0 con 0x7f3dd80d2290 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd6d9d640 1 --2- 172.21.15.113:0/376213719 >> v2:172.21.15.178:3300/0 conn(0x7f3dd80d0b00 0x7f3dd80d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.299+0000 7f3dd5d9b640 1 --2- 172.21.15.113:0/376213719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd80d2290 0x7f3dd80b9aa0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80b3970 tx=0x7f3dcc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f13bff7d71cfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3dd559a640 1 -- 172.21.15.113:0/376213719 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007740 con 0x7f3dd80d2290 2024-04-25T00:06:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3dd559a640 1 -- 172.21.15.113:0/376213719 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc0078e0 con 0x7f3dd80d2290 2024-04-25T00:06:06.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3dd559a640 1 -- 172.21.15.113:0/376213719 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007cb0 con 0x7f3dd80d2290 2024-04-25T00:06:06.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 -- 172.21.15.113:0/376213719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd80d2290 msgr2=0x7f3dd80b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/376213719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd80d2290 0x7f3dd80b9aa0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80b3970 tx=0x7f3dcc02a710 comp rx=0 tx=0).stop 2024-04-25T00:06:06.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 -- 172.21.15.113:0/376213719 shutdown_connections 2024-04-25T00:06:06.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/376213719 >> v2:172.21.15.178:3300/0 conn(0x7f3dd80d0b00 0x7f3dd80d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/376213719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd80d2290 0x7f3dd80b9aa0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 -- 172.21.15.113:0/376213719 >> 172.21.15.113:0/376213719 conn(0x7f3dd80b5c20 msgr2=0x7f3dd80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:06.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 -- 172.21.15.113:0/376213719 shutdown_connections 2024-04-25T00:06:06.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.300+0000 7f3ddcbee640 1 -- 172.21.15.113:0/376213719 wait complete. 2024-04-25T00:06:06.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 Processor -- start 2024-04-25T00:06:06.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 -- start start 2024-04-25T00:06:06.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd80d0b00 0x7f3dd80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3dd80d2290 0x7f3dd81f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd815e320 0x7f3dd81f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80b9fe0 con 0x7f3dd815e320 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80d30a0 con 0x7f3dd80d0b00 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3ddcbee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd815e700 con 0x7f3dd80d2290 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3dd6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd815e320 0x7f3dd81f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3dd6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd815e320 0x7f3dd81f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48344/0 (socket says 172.21.15.113:48344) 2024-04-25T00:06:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.302+0000 7f3dd6d9d640 1 -- 172.21.15.113:0/3347366080 learned_addr learned my addr 172.21.15.113:0/3347366080 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd659c640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd80d0b00 0x7f3dd80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd5d9b640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3dd80d2290 0x7f3dd81f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd6d9d640 1 -- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3dd80d2290 msgr2=0x7f3dd81f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd6d9d640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3dd80d2290 0x7f3dd81f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd6d9d640 1 -- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd80d0b00 msgr2=0x7f3dd80cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd6d9d640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd80d0b00 0x7f3dd80cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd6d9d640 1 -- 172.21.15.113:0/3347366080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd80cc4d0 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dd6d9d640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd815e320 0x7f3dd81f7860 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c7c0 tx=0x7f3dc800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dc6ffd640 1 -- 172.21.15.113:0/3347366080 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8015020 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dc6ffd640 1 -- 172.21.15.113:0/3347366080 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc8004230 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3dc6ffd640 1 -- 172.21.15.113:0/3347366080 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8007620 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd80cb430 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.303+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd80cb9f0 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.304+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd80691a0 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.304+0000 7f3dc6ffd640 1 -- 172.21.15.113:0/3347366080 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3dc8018020 con 0x7f3dd815e320 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.305+0000 7f3dc6ffd640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3db806be30 0x7f3db806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.305+0000 7f3dc6ffd640 1 -- 172.21.15.113:0/3347366080 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f3dc808ad90 con 0x7f3dd815e320 2024-04-25T00:06:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.305+0000 7f3dd659c640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3db806be30 0x7f3db806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.305+0000 7f3dd659c640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3db806be30 0x7f3db806e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000f60 tx=0x7f3dd0007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.309+0000 7f3dc6ffd640 1 -- 172.21.15.113:0/3347366080 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3dc801f020 con 0x7f3dd815e320 2024-04-25T00:06:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.520+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dd815d140 con 0x7f3dd815e320 2024-04-25T00:06:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.521+0000 7f3dc6ffd640 1 -- 172.21.15.113:0/3347366080 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f3dc8059200 con 0x7f3dd815e320 2024-04-25T00:06:06.521 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:06.522 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:06:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3db806be30 msgr2=0x7f3db806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3db806be30 0x7f3db806e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000f60 tx=0x7f3dd0007d50 comp rx=0 tx=0).stop 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd815e320 msgr2=0x7f3dd81f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd815e320 0x7f3dd81f7860 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c7c0 tx=0x7f3dc800cc80 comp rx=0 tx=0).stop 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 shutdown_connections 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3db806be30 0x7f3db806e2f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dd815e320 0x7f3dd81f7860 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3dd80d2290 0x7f3dd81f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 --2- 172.21.15.113:0/3347366080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd80d0b00 0x7f3dd80cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 >> 172.21.15.113:0/3347366080 conn(0x7f3dd80b5c20 msgr2=0x7f3dd815d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 shutdown_connections 2024-04-25T00:06:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.523+0000 7f3ddcbee640 1 -- 172.21.15.113:0/3347366080 wait complete. 2024-04-25T00:06:06.535 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:06.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.630+0000 7f7a13fb7640 1 Processor -- start 2024-04-25T00:06:06.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.631+0000 7f7a13fb7640 1 -- start start 2024-04-25T00:06:06.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.631+0000 7f7a13fb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c144910 0x7f7a0c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.631+0000 7f7a13fb7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a0c06e6e0 0x7f7a0c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.631+0000 7f7a13fb7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0c144cf0 con 0x7f7a0c0b5240 2024-04-25T00:06:06.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.631+0000 7f7a13fb7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7a0c0b4510 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.631+0000 7f7a13fb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0c0b2f30 con 0x7f7a0c144910 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a11552640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7a0c0b5240 0x7f7a0c06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56692/0 (socket says 172.21.15.113:56692) 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a11552640 1 -- 172.21.15.113:0/149606133 learned_addr learned my addr 172.21.15.113:0/149606133 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a12554640 1 --2- 172.21.15.113:0/149606133 >> v2:172.21.15.178:3300/0 conn(0x7f7a0c06e6e0 0x7f7a0c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a11d53640 1 --2- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c144910 0x7f7a0c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a10d51640 1 -- 172.21.15.113:0/149606133 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912491457 0 0) 0x7f7a0c144cf0 con 0x7f7a0c0b5240 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a10d51640 1 -- 172.21.15.113:0/149606133 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f8003520 con 0x7f7a0c0b5240 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a10d51640 1 -- 172.21.15.113:0/149606133 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1790915026 0 0) 0x7f79f8003520 con 0x7f7a0c0b5240 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a12554640 1 -- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c144910 msgr2=0x7f7a0c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a12554640 1 --2- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c144910 0x7f7a0c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a12554640 1 -- 172.21.15.113:0/149606133 >> v1:172.21.15.113:6789/0 conn(0x7f7a0c0b5240 legacy=0x7f7a0c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a12554640 1 -- 172.21.15.113:0/149606133 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c0b3210 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.632+0000 7f7a11d53640 1 --2- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c144910 0x7f7a0c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:06.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.633+0000 7f7a12554640 1 --2- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c0cd500 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009d90 tx=0x7f7a0402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fd0bae661fff05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.633+0000 7f7a10d51640 1 -- 172.21.15.113:0/149606133 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00001670 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.633+0000 7f7a10d51640 1 -- 172.21.15.113:0/149606133 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a04007540 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.633+0000 7f7a10d51640 1 -- 172.21.15.113:0/149606133 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040078f0 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.633+0000 7f7a13fb7640 1 -- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c06e6e0 msgr2=0x7f7a0c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.633+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c0cd500 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009d90 tx=0x7f7a0402a5f0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.634+0000 7f7a13fb7640 1 -- 172.21.15.113:0/149606133 shutdown_connections 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.634+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c0cd500 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.634+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/149606133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c144910 0x7f7a0c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.634+0000 7f7a13fb7640 1 -- 172.21.15.113:0/149606133 >> 172.21.15.113:0/149606133 conn(0x7f7a0c0b1a50 msgr2=0x7f7a0c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.634+0000 7f7a13fb7640 1 -- 172.21.15.113:0/149606133 shutdown_connections 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.634+0000 7f7a13fb7640 1 -- 172.21.15.113:0/149606133 wait complete. 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.635+0000 7f7a13fb7640 1 Processor -- start 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.635+0000 7f7a13fb7640 1 -- start start 2024-04-25T00:06:06.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.635+0000 7f7a13fb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c13f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.635+0000 7f7a13fb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c0b5240 0x7f7a0c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.635+0000 7f7a13fb7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c144910 0x7f7a0c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.635+0000 7f7a13fb7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c0b4da0 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.635+0000 7f7a13fb7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c0b37d0 con 0x7f7a0c144910 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a13fb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0c0b4f20 con 0x7f7a0c0b5240 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c13f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c13f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48358/0 (socket says 172.21.15.113:48358) 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 -- 172.21.15.113:0/1609202875 learned_addr learned my addr 172.21.15.113:0/1609202875 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11552640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c0b5240 0x7f7a0c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a12554640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c144910 0x7f7a0c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 -- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c0b5240 msgr2=0x7f7a0c0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c0b5240 0x7f7a0c0b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 -- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c144910 msgr2=0x7f7a0c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c144910 0x7f7a0c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 -- 172.21.15.113:0/1609202875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c0c52b0 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a11d53640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c13f850 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800ebe0 tx=0x7f7a0800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f79fe7fc640 1 -- 172.21.15.113:0/1609202875 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08017070 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f79fe7fc640 1 -- 172.21.15.113:0/1609202875 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a0800cc30 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0c1f5c90 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f79fe7fc640 1 -- 172.21.15.113:0/1609202875 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08013470 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.636+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0c0ad460 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.637+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a0c0691a0 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.638+0000 7f79fe7fc640 1 -- 172.21.15.113:0/1609202875 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a0801a070 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.638+0000 7f79fe7fc640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f79f006be60 0x7f79f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.638+0000 7f79fe7fc640 1 -- 172.21.15.113:0/1609202875 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f7a0808ad50 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.638+0000 7f7a11552640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f79f006be60 0x7f79f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.638+0000 7f7a11552640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f79f006be60 0x7f79f006e320 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c144170 tx=0x7f7a000057f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.642+0000 7f79fe7fc640 1 -- 172.21.15.113:0/1609202875 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a0808e030 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.815+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a0c06c100 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.816+0000 7f79fe7fc640 1 -- 172.21.15.113:0/1609202875 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16829 (secure 0 0 0) 0x7f7a0c0c52b0 con 0x7f7a0c06e6e0 2024-04-25T00:06:06.817 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:06.817 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":118,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:02.284662+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:05:33.644000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:05:34.412894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.21","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f79f006be60 msgr2=0x7f79f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f79f006be60 0x7f79f006e320 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c144170 tx=0x7f7a000057f0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a0c06e6e0 msgr2=0x7f7a0c13f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c13f850 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800ebe0 tx=0x7f7a0800c520 comp rx=0 tx=0).stop 2024-04-25T00:06:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 shutdown_connections 2024-04-25T00:06:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f79f006be60 0x7f79f006e320 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c144910 0x7f7a0c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a0c0b5240 0x7f7a0c0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 --2- 172.21.15.113:0/1609202875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a0c06e6e0 0x7f7a0c13f850 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 >> 172.21.15.113:0/1609202875 conn(0x7f7a0c0b1a50 msgr2=0x7f7a0c072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 shutdown_connections 2024-04-25T00:06:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.818+0000 7f7a13fb7640 1 -- 172.21.15.113:0/1609202875 wait complete. 2024-04-25T00:06:06.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:05:33.644000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:05:34.412894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:06:06.831 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:06:06.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.930+0000 7f7d2d6ba640 1 Processor -- start 2024-04-25T00:06:06.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.930+0000 7f7d2d6ba640 1 -- start start 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d2d6ba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d2815e510 0x7f7d280d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d2d6ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280b63b0 0x7f7d280b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d2d6ba640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d280d2e00 con 0x7f7d280d28c0 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d2d6ba640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7d280d30e0 con 0x7f7d2815e510 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d2d6ba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d280d33c0 con 0x7f7d280b63b0 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d26ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d2815e510 0x7f7d280d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d26ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d2815e510 0x7f7d280d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48360/0 (socket says 172.21.15.113:48360) 2024-04-25T00:06:06.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d26ffd640 1 -- 172.21.15.113:0/1886292408 learned_addr learned my addr 172.21.15.113:0/1886292408 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d277fe640 1 --2- 172.21.15.113:0/1886292408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280b63b0 0x7f7d280b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d26ffd640 1 -- 172.21.15.113:0/1886292408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280b63b0 msgr2=0x7f7d280b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d26ffd640 1 --2- 172.21.15.113:0/1886292408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280b63b0 0x7f7d280b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d26ffd640 1 -- 172.21.15.113:0/1886292408 >> v1:172.21.15.178:6789/0 conn(0x7f7d280d28c0 legacy=0x7f7d280b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.931+0000 7f7d26ffd640 1 -- 172.21.15.113:0/1886292408 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d280d36a0 con 0x7f7d2815e510 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.932+0000 7f7d26ffd640 1 --2- 172.21.15.113:0/1886292408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d280d2380 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009870 tx=0x7f7d1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e520fd361d96d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.932+0000 7f7d267fc640 1 -- 172.21.15.113:0/1886292408 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18037070 con 0x7f7d2815e510 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.932+0000 7f7d267fc640 1 -- 172.21.15.113:0/1886292408 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d18032440 con 0x7f7d2815e510 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.932+0000 7f7d267fc640 1 -- 172.21.15.113:0/1886292408 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18032810 con 0x7f7d2815e510 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/1886292408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 msgr2=0x7f7d280d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/1886292408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d280d2380 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009870 tx=0x7f7d1802a180 comp rx=0 tx=0).stop 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/1886292408 shutdown_connections 2024-04-25T00:06:06.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/1886292408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280b63b0 0x7f7d280b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/1886292408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d280d2380 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/1886292408 >> 172.21.15.113:0/1886292408 conn(0x7f7d2806b720 msgr2=0x7f7d280d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:06.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/1886292408 shutdown_connections 2024-04-25T00:06:06.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.933+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/1886292408 wait complete. 2024-04-25T00:06:06.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.934+0000 7f7d2d6ba640 1 Processor -- start 2024-04-25T00:06:06.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.934+0000 7f7d2d6ba640 1 -- start start 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.934+0000 7f7d2d6ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d280b63b0 0x7f7d281611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.934+0000 7f7d2d6ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280d28c0 0x7f7d280ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.934+0000 7f7d2d6ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d2814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d2d6ba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280b9dd0 con 0x7f7d2815e510 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d2d6ba640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280d3ea0 con 0x7f7d280b63b0 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d2d6ba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d280b7900 con 0x7f7d280d28c0 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d2814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d2814a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48362/0 (socket says 172.21.15.113:48362) 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 -- 172.21.15.113:0/2983341257 learned_addr learned my addr 172.21.15.113:0/2983341257 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d26ffd640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d280b63b0 0x7f7d281611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d1ffff640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280d28c0 0x7f7d280ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 -- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280d28c0 msgr2=0x7f7d280ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280d28c0 0x7f7d280ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 -- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d280b63b0 msgr2=0x7f7d281611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d280b63b0 0x7f7d281611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 -- 172.21.15.113:0/2983341257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2815fac0 con 0x7f7d2815e510 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d26ffd640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d280b63b0 0x7f7d281611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d277fe640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d2814a320 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000cd90 tx=0x7f7d2000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d1f7fe640 1 -- 172.21.15.113:0/2983341257 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20003b90 con 0x7f7d2815e510 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d1f7fe640 1 -- 172.21.15.113:0/2983341257 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d20003d50 con 0x7f7d2815e510 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d280c82a0 con 0x7f7d2815e510 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d1f7fe640 1 -- 172.21.15.113:0/2983341257 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20004260 con 0x7f7d2815e510 2024-04-25T00:06:06.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.935+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d280c8b40 con 0x7f7d2815e510 2024-04-25T00:06:06.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.936+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d280c8690 con 0x7f7d2815e510 2024-04-25T00:06:06.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.937+0000 7f7d1f7fe640 1 -- 172.21.15.113:0/2983341257 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d20004400 con 0x7f7d2815e510 2024-04-25T00:06:06.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.937+0000 7f7d1f7fe640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7d0806be30 0x7f7d0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.937+0000 7f7d26ffd640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7d0806be30 0x7f7d0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.937+0000 7f7d1f7fe640 1 -- 172.21.15.113:0/2983341257 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f7d2008b820 con 0x7f7d2815e510 2024-04-25T00:06:06.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.937+0000 7f7d26ffd640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7d0806be30 0x7f7d0806e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7d18034000 tx=0x7f7d1802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:06.941+0000 7f7d1f7fe640 1 -- 172.21.15.113:0/2983341257 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d2005a220 con 0x7f7d2815e510 2024-04-25T00:06:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.116+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d280691a0 con 0x7f7d2815e510 2024-04-25T00:06:07.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.116+0000 7f7d1f7fe640 1 -- 172.21.15.113:0/2983341257 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f7d20059b20 con 0x7f7d2815e510 2024-04-25T00:06:07.117 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7d0806be30 msgr2=0x7f7d0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7d0806be30 0x7f7d0806e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7d18034000 tx=0x7f7d1802e000 comp rx=0 tx=0).stop 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 msgr2=0x7f7d2814a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d2814a320 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000cd90 tx=0x7f7d2000a660 comp rx=0 tx=0).stop 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 shutdown_connections 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7d0806be30 0x7f7d0806e2f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d2815e510 0x7f7d2814a320 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d280d28c0 0x7f7d280ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 --2- 172.21.15.113:0/2983341257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d280b63b0 0x7f7d281611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 >> 172.21.15.113:0/2983341257 conn(0x7f7d2806b720 msgr2=0x7f7d2815c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 shutdown_connections 2024-04-25T00:06:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.118+0000 7f7d2d6ba640 1 -- 172.21.15.113:0/2983341257 wait complete. 2024-04-25T00:06:07.130 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:06:07.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.229+0000 7f6e511fb640 1 Processor -- start 2024-04-25T00:06:07.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.229+0000 7f6e511fb640 1 -- start start 2024-04-25T00:06:07.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e511fb640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6e4c0b3080 0x7f6e4c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e511fb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4c15f4f0 con 0x7f6e4c15e2d0 2024-04-25T00:06:07.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e511fb640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4c15f7d0 con 0x7f6e4c15e6b0 2024-04-25T00:06:07.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e511fb640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6e4c0cde00 con 0x7f6e4c0b3080 2024-04-25T00:06:07.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e4ad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6e4c15e2d0 0x7f6e4c15ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56706/0 (socket says 172.21.15.113:56706) 2024-04-25T00:06:07.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e4ad9d640 1 -- 172.21.15.113:0/1475332940 learned_addr learned my addr 172.21.15.113:0/1475332940 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342912071 0 0) 0x7f6e4c15f4f0 con 0x7f6e4c15e2d0 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e34003520 con 0x7f6e4c15e2d0 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/1475332940 >> v2:172.21.15.178:3300/0 conn(0x7f6e4c0b3080 0x7f6e4c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2959199651 0 0) 0x7f6e34003520 con 0x7f6e4c15e2d0 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 >> v2:172.21.15.178:3300/0 conn(0x7f6e4c0b3080 msgr2=0x7f6e4c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e49d9b640 1 --2- 172.21.15.113:0/1475332940 >> v2:172.21.15.178:3300/0 conn(0x7f6e4c0b3080 0x7f6e4c0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.230+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 >> v1:172.21.15.178:6789/0 conn(0x7f6e4c15e6b0 legacy=0x7f6e4c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c15fab0 con 0x7f6e4c15e2d0 2024-04-25T00:06:07.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6e3c0019f0 con 0x7f6e4c15e2d0 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 >> v1:172.21.15.113:6789/0 conn(0x7f6e4c15e2d0 legacy=0x7f6e4c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e340039b0 0x7f6e34003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e340051c0 0x7f6e3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c0cde00 con 0x7f6e34004800 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3400c0f0 con 0x7f6e340051c0 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3400c2f0 con 0x7f6e340039b0 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4ad9d640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e340039b0 0x7f6e34003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4b59e640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e340051c0 0x7f6e3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4a59c640 1 -- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e340039b0 msgr2=0x7f6e34003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e340039b0 0x7f6e34003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4a59c640 1 -- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e340051c0 msgr2=0x7f6e3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e340051c0 0x7f6e3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4a59c640 1 -- 172.21.15.113:0/1475332940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e34003520 con 0x7f6e34004800 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4ad9d640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e340039b0 0x7f6e34003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:07.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4b59e640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e340051c0 0x7f6e3400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.231+0000 7f6e4b59e640 1 -- 172.21.15.113:0/1475332940 reap_dead start 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000ba00 tx=0x7f6e34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1005585b5debb6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40030070 con 0x7f6e34004800 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4002adb0 con 0x7f6e34004800 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e49d9b640 1 -- 172.21.15.113:0/1475332940 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000b600 con 0x7f6e34004800 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e511fb640 1 -- 172.21.15.113:0/1475332940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e34004800 msgr2=0x7f6e34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e511fb640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000ba00 tx=0x7f6e34005830 comp rx=0 tx=0).stop 2024-04-25T00:06:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e511fb640 1 -- 172.21.15.113:0/1475332940 shutdown_connections 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e511fb640 1 --2- 172.21.15.113:0/1475332940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e511fb640 1 -- 172.21.15.113:0/1475332940 >> 172.21.15.113:0/1475332940 conn(0x7f6e4c06b0f0 msgr2=0x7f6e4c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e511fb640 1 -- 172.21.15.113:0/1475332940 shutdown_connections 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.232+0000 7f6e511fb640 1 -- 172.21.15.113:0/1475332940 wait complete. 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.234+0000 7f6e511fb640 1 Processor -- start 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.234+0000 7f6e511fb640 1 -- start start 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.234+0000 7f6e511fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e34004800 0x7f6e4c1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.234+0000 7f6e511fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4c154d10 0x7f6e4c0aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e511fb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4c0aa760 0x7f6e4c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e511fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c15ed70 con 0x7f6e4c154d10 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e511fb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c15ebf0 con 0x7f6e34004800 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e511fb640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4c15eef0 con 0x7f6e4c0aa760 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4c154d10 0x7f6e4c0aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4c154d10 0x7f6e4c0aa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48378/0 (socket says 172.21.15.113:48378) 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 -- 172.21.15.113:0/444163317 learned_addr learned my addr 172.21.15.113:0/444163317 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4b59e640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4c0aa760 0x7f6e4c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4ad9d640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e34004800 0x7f6e4c1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 -- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4c0aa760 msgr2=0x7f6e4c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4c0aa760 0x7f6e4c15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 -- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e34004800 msgr2=0x7f6e4c1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e34004800 0x7f6e4c1547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 -- 172.21.15.113:0/444163317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c14c240 con 0x7f6e4c154d10 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4ad9d640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e34004800 0x7f6e4c1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e4a59c640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4c154d10 0x7f6e4c0aa220 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002a8a0 tx=0x7f6e40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e337fe640 1 -- 172.21.15.113:0/444163317 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40043030 con 0x7f6e4c154d10 2024-04-25T00:06:07.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e337fe640 1 -- 172.21.15.113:0/444163317 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4002abb0 con 0x7f6e4c154d10 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c153450 con 0x7f6e4c154d10 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e337fe640 1 -- 172.21.15.113:0/444163317 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000b600 con 0x7f6e4c154d10 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.235+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c153a10 con 0x7f6e4c154d10 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.236+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4c0691a0 con 0x7f6e4c154d10 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.237+0000 7f6e337fe640 1 -- 172.21.15.113:0/444163317 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e40046070 con 0x7f6e4c154d10 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.237+0000 7f6e337fe640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e3806be60 0x7f6e3806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.237+0000 7f6e4ad9d640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e3806be60 0x7f6e3806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.237+0000 7f6e337fe640 1 -- 172.21.15.113:0/444163317 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f6e400ab870 con 0x7f6e4c154d10 2024-04-25T00:06:07.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.238+0000 7f6e4ad9d640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e3806be60 0x7f6e3806e320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c002230 tx=0x7f6e3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.241+0000 7f6e337fe640 1 -- 172.21.15.113:0/444163317 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e4007a270 con 0x7f6e4c154d10 2024-04-25T00:06:07.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.417+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6e4c072050 con 0x7f6e4c154d10 2024-04-25T00:06:07.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.417+0000 7f6e337fe640 1 -- 172.21.15.113:0/444163317 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f6e40079b70 con 0x7f6e4c154d10 2024-04-25T00:06:07.417 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e3806be60 msgr2=0x7f6e3806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e3806be60 0x7f6e3806e320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c002230 tx=0x7f6e3c009040 comp rx=0 tx=0).stop 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4c154d10 msgr2=0x7f6e4c0aa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4c154d10 0x7f6e4c0aa220 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002a8a0 tx=0x7f6e40004730 comp rx=0 tx=0).stop 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 shutdown_connections 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e3806be60 0x7f6e3806e320 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4c0aa760 0x7f6e4c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e4c154d10 0x7f6e4c0aa220 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 --2- 172.21.15.113:0/444163317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e34004800 0x7f6e4c1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 >> 172.21.15.113:0/444163317 conn(0x7f6e4c06b0f0 msgr2=0x7f6e4c0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:07.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 shutdown_connections 2024-04-25T00:06:07.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.419+0000 7f6e511fb640 1 -- 172.21.15.113:0/444163317 wait complete. 2024-04-25T00:06:07.431 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:06:07.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.530+0000 7f5b58db7640 1 Processor -- start 2024-04-25T00:06:07.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b58db7640 1 -- start start 2024-04-25T00:06:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b58db7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b5415e510 0x7f5b540d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b58db7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5b540b63b0 0x7f5b540b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b58db7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b540d2e00 con 0x7f5b540d28c0 2024-04-25T00:06:07.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b58db7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5b540d30e0 con 0x7f5b5415e510 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b58db7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5b540d33c0 con 0x7f5b540b63b0 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b5259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b5415e510 0x7f5b540d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b5259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b5415e510 0x7f5b540d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45686/0 (socket says 172.21.15.113:45686) 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.531+0000 7f5b5259c640 1 -- 172.21.15.113:0/2692899449 learned_addr learned my addr 172.21.15.113:0/2692899449 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.532+0000 7f5b52d9d640 1 --2- 172.21.15.113:0/2692899449 >> v2:172.21.15.178:3300/0 conn(0x7f5b540b63b0 0x7f5b540b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.532+0000 7f5b5259c640 1 -- 172.21.15.113:0/2692899449 >> v2:172.21.15.178:3300/0 conn(0x7f5b540b63b0 msgr2=0x7f5b540b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.532+0000 7f5b5259c640 1 --2- 172.21.15.113:0/2692899449 >> v2:172.21.15.178:3300/0 conn(0x7f5b540b63b0 0x7f5b540b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.532+0000 7f5b5259c640 1 -- 172.21.15.113:0/2692899449 >> v1:172.21.15.178:6789/0 conn(0x7f5b540d28c0 legacy=0x7f5b540b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.532+0000 7f5b5259c640 1 -- 172.21.15.113:0/2692899449 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b540d36a0 con 0x7f5b5415e510 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.532+0000 7f5b52d9d640 1 --2- 172.21.15.113:0/2692899449 >> v2:172.21.15.178:3300/0 conn(0x7f5b540b63b0 0x7f5b540b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.532+0000 7f5b5259c640 1 --2- 172.21.15.113:0/2692899449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b5415e510 0x7f5b540d2380 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5b540b7190 tx=0x7f5b4402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49423ad511415250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b51d9b640 1 -- 172.21.15.113:0/2692899449 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44037070 con 0x7f5b5415e510 2024-04-25T00:06:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b51d9b640 1 -- 172.21.15.113:0/2692899449 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b44032440 con 0x7f5b5415e510 2024-04-25T00:06:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b51d9b640 1 -- 172.21.15.113:0/2692899449 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44032810 con 0x7f5b5415e510 2024-04-25T00:06:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 -- 172.21.15.113:0/2692899449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b5415e510 msgr2=0x7f5b540d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 --2- 172.21.15.113:0/2692899449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b5415e510 0x7f5b540d2380 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5b540b7190 tx=0x7f5b4402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 -- 172.21.15.113:0/2692899449 shutdown_connections 2024-04-25T00:06:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 --2- 172.21.15.113:0/2692899449 >> v2:172.21.15.178:3300/0 conn(0x7f5b540b63b0 0x7f5b540b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 --2- 172.21.15.113:0/2692899449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b5415e510 0x7f5b540d2380 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 -- 172.21.15.113:0/2692899449 >> 172.21.15.113:0/2692899449 conn(0x7f5b5406b720 msgr2=0x7f5b540d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:07.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 -- 172.21.15.113:0/2692899449 shutdown_connections 2024-04-25T00:06:07.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.533+0000 7f5b58db7640 1 -- 172.21.15.113:0/2692899449 wait complete. 2024-04-25T00:06:07.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.534+0000 7f5b58db7640 1 Processor -- start 2024-04-25T00:06:07.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b58db7640 1 -- start start 2024-04-25T00:06:07.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b58db7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b540b63b0 0x7f5b540b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b58db7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b540d28c0 0x7f5b54160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b58db7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b5415e510 0x7f5b54151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b58db7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540b9dd0 con 0x7f5b540b63b0 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b58db7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540d3ea0 con 0x7f5b540d28c0 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b58db7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5b540b7900 con 0x7f5b5415e510 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b5259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b540b63b0 0x7f5b540b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b5259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b540b63b0 0x7f5b540b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45694/0 (socket says 172.21.15.113:45694) 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.535+0000 7f5b5259c640 1 -- 172.21.15.113:0/3245352878 learned_addr learned my addr 172.21.15.113:0/3245352878 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b4bfff640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b540d28c0 0x7f5b54160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b52d9d640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b5415e510 0x7f5b54151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b5259c640 1 -- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b5415e510 msgr2=0x7f5b54151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b5259c640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b5415e510 0x7f5b54151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b5259c640 1 -- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b540d28c0 msgr2=0x7f5b54160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b5259c640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b540d28c0 0x7f5b54160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b5259c640 1 -- 172.21.15.113:0/3245352878 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b540cb7f0 con 0x7f5b540b63b0 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b4bfff640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b540d28c0 0x7f5b54160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b52d9d640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b5415e510 0x7f5b54151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b5259c640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b540b63b0 0x7f5b540b4f40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5b4402a1e0 tx=0x7f5b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b4affd640 1 -- 172.21.15.113:0/3245352878 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44037040 con 0x7f5b540b63b0 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b4affd640 1 -- 172.21.15.113:0/3245352878 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b44042070 con 0x7f5b540b63b0 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b540cbad0 con 0x7f5b540b63b0 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b4affd640 1 -- 172.21.15.113:0/3245352878 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44030bb0 con 0x7f5b540b63b0 2024-04-25T00:06:07.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.536+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b540cc090 con 0x7f5b540b63b0 2024-04-25T00:06:07.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.537+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b540691a0 con 0x7f5b540b63b0 2024-04-25T00:06:07.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.537+0000 7f5b4affd640 1 -- 172.21.15.113:0/3245352878 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b44047460 con 0x7f5b540b63b0 2024-04-25T00:06:07.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.538+0000 7f5b4affd640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b3406be30 0x7f5b3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.538+0000 7f5b4affd640 1 -- 172.21.15.113:0/3245352878 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f5b440abbf0 con 0x7f5b540b63b0 2024-04-25T00:06:07.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.538+0000 7f5b4bfff640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b3406be30 0x7f5b3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.538+0000 7f5b4bfff640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b3406be30 0x7f5b3406e2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5b540ce3f0 tx=0x7f5b3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.542+0000 7f5b4affd640 1 -- 172.21.15.113:0/3245352878 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b44045360 con 0x7f5b540b63b0 2024-04-25T00:06:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.714+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5b540b5d10 con 0x7f5b540b63b0 2024-04-25T00:06:07.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.714+0000 7f5b4affd640 1 -- 172.21.15.113:0/3245352878 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f5b4407a4e0 con 0x7f5b540b63b0 2024-04-25T00:06:07.715 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:06:07.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b3406be30 msgr2=0x7f5b3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b3406be30 0x7f5b3406e2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5b540ce3f0 tx=0x7f5b3c009040 comp rx=0 tx=0).stop 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b540b63b0 msgr2=0x7f5b540b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b540b63b0 0x7f5b540b4f40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5b4402a1e0 tx=0x7f5b440046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 shutdown_connections 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5b3406be30 0x7f5b3406e2f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b540d28c0 0x7f5b54160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b5415e510 0x7f5b54151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 --2- 172.21.15.113:0/3245352878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b540b63b0 0x7f5b540b4f40 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 >> 172.21.15.113:0/3245352878 conn(0x7f5b5406b720 msgr2=0x7f5b54071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 shutdown_connections 2024-04-25T00:06:07.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.716+0000 7f5b58db7640 1 -- 172.21.15.113:0/3245352878 wait complete. 2024-04-25T00:06:07.728 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:07.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.824+0000 7f3ed78b3640 1 Processor -- start 2024-04-25T00:06:07.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.824+0000 7f3ed78b3640 1 -- start start 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed78b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ed0144910 0x7f3ed00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed78b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 0x7f3ed006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed78b3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed0144cf0 con 0x7f3ed00b5680 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed78b3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed00b4510 con 0x7f3ed0144910 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed78b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed00b2f30 con 0x7f3ed00b5240 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed564f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ed0144910 0x7f3ed00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed564f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ed0144910 0x7f3ed00b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45700/0 (socket says 172.21.15.113:45700) 2024-04-25T00:06:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed564f640 1 -- 172.21.15.113:0/3593304084 learned_addr learned my addr 172.21.15.113:0/3593304084 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:07.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed564f640 1 -- 172.21.15.113:0/3593304084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 msgr2=0x7f3ed006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed564f640 1 --2- 172.21.15.113:0/3593304084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 0x7f3ed006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed564f640 1 -- 172.21.15.113:0/3593304084 >> v1:172.21.15.178:6789/0 conn(0x7f3ed00b5680 legacy=0x7f3ed00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:07.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.825+0000 7f3ed564f640 1 -- 172.21.15.113:0/3593304084 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed00b3210 con 0x7f3ed0144910 2024-04-25T00:06:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.826+0000 7f3ed564f640 1 --2- 172.21.15.113:0/3593304084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b3fd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00d150 tx=0x7f3ecc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6ecc1e7e7e10f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.826+0000 7f3ec7fff640 1 -- 172.21.15.113:0/3593304084 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00b2e0 con 0x7f3ed0144910 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.826+0000 7f3ec7fff640 1 -- 172.21.15.113:0/3593304084 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc009600 con 0x7f3ed0144910 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.826+0000 7f3ec7fff640 1 -- 172.21.15.113:0/3593304084 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00b480 con 0x7f3ed0144910 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3593304084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 msgr2=0x7f3ed00b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3593304084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b3fd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00d150 tx=0x7f3ecc02c8e0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3593304084 shutdown_connections 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3593304084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 0x7f3ed006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3593304084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b3fd0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3593304084 >> 172.21.15.113:0/3593304084 conn(0x7f3ed00b1a50 msgr2=0x7f3ed00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3593304084 shutdown_connections 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.827+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3593304084 wait complete. 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.828+0000 7f3ed78b3640 1 Processor -- start 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.828+0000 7f3ed78b3640 1 -- start start 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed78b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 0x7f3ed013ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed78b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ed00b5680 0x7f3ed00b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed78b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed78b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed00b4da0 con 0x7f3ed0144910 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed78b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed00b37d0 con 0x7f3ed00b5680 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed78b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed00b4f20 con 0x7f3ed00b5240 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45712/0 (socket says 172.21.15.113:45712) 2024-04-25T00:06:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 -- 172.21.15.113:0/3343791447 learned_addr learned my addr 172.21.15.113:0/3343791447 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed564f640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 0x7f3ed013ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed4e4e640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ed00b5680 0x7f3ed00b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 -- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 msgr2=0x7f3ed013ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 0x7f3ed013ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 -- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ed00b5680 msgr2=0x7f3ed00b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ed00b5680 0x7f3ed00b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 -- 172.21.15.113:0/3343791447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed00af360 con 0x7f3ed0144910 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed5e50640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b13c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c410 tx=0x7f3ec800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ec5ffb640 1 -- 172.21.15.113:0/3343791447 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8017070 con 0x7f3ed0144910 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ec5ffb640 1 -- 172.21.15.113:0/3343791447 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec80035f0 con 0x7f3ed0144910 2024-04-25T00:06:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ec5ffb640 1 -- 172.21.15.113:0/3343791447 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80039e0 con 0x7f3ed0144910 2024-04-25T00:06:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.829+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed00ae5a0 con 0x7f3ed0144910 2024-04-25T00:06:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.830+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed00aeb60 con 0x7f3ed0144910 2024-04-25T00:06:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.831+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed00691a0 con 0x7f3ed0144910 2024-04-25T00:06:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.831+0000 7f3ec5ffb640 1 -- 172.21.15.113:0/3343791447 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ec801a070 con 0x7f3ed0144910 2024-04-25T00:06:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.831+0000 7f3ec5ffb640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb406be30 0x7f3eb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.831+0000 7f3ec5ffb640 1 -- 172.21.15.113:0/3343791447 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f3ec808a1b0 con 0x7f3ed0144910 2024-04-25T00:06:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.831+0000 7f3ed564f640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb406be30 0x7f3eb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.832+0000 7f3ed564f640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb406be30 0x7f3eb406e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc036000 tx=0x7f3ecc00b000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:07.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:07.836+0000 7f3ec5ffb640 1 -- 172.21.15.113:0/3343791447 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ec8058b20 con 0x7f3ed0144910 2024-04-25T00:06:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.044+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed013f150 con 0x7f3ed0144910 2024-04-25T00:06:08.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.045+0000 7f3ec5ffb640 1 -- 172.21.15.113:0/3343791447 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f3ec8058420 con 0x7f3ed0144910 2024-04-25T00:06:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:08.046 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:06:08.046 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:06:08.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb406be30 msgr2=0x7f3eb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb406be30 0x7f3eb406e2f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc036000 tx=0x7f3ecc00b000 comp rx=0 tx=0).stop 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 msgr2=0x7f3ed00b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b13c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c410 tx=0x7f3ec800c8d0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 shutdown_connections 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb406be30 0x7f3eb406e2f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ed00b5680 0x7f3ed00b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ed0144910 0x7f3ed00b13c0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 --2- 172.21.15.113:0/3343791447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ed00b5240 0x7f3ed013ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 >> 172.21.15.113:0/3343791447 conn(0x7f3ed00b1a50 msgr2=0x7f3ed0143420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 shutdown_connections 2024-04-25T00:06:08.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.047+0000 7f3ed78b3640 1 -- 172.21.15.113:0/3343791447 wait complete. 2024-04-25T00:06:08.059 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:08.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.157+0000 7f2e4a353640 1 Processor -- start 2024-04-25T00:06:08.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.157+0000 7f2e4a353640 1 -- start start 2024-04-25T00:06:08.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e4a353640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e44144d60 0x7f2e440b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e4a353640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e44071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e4a353640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e440cfad0 con 0x7f2e441453c0 2024-04-25T00:06:08.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e4a353640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2e44145c40 con 0x7f2e44144d60 2024-04-25T00:06:08.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e4a353640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e440cf230 con 0x7f2e4406e6e0 2024-04-25T00:06:08.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e43fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e44144d60 0x7f2e440b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e43fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e44144d60 0x7f2e440b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45716/0 (socket says 172.21.15.113:45716) 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e43fff640 1 -- 172.21.15.113:0/1206389557 learned_addr learned my addr 172.21.15.113:0/1206389557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e488f0640 1 --2- 172.21.15.113:0/1206389557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e44071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e43fff640 1 -- 172.21.15.113:0/1206389557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 msgr2=0x7f2e44071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e43fff640 1 --2- 172.21.15.113:0/1206389557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e44071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e43fff640 1 -- 172.21.15.113:0/1206389557 >> v1:172.21.15.178:6789/0 conn(0x7f2e441453c0 legacy=0x7f2e4406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e43fff640 1 -- 172.21.15.113:0/1206389557 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e440cf510 con 0x7f2e44144d60 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.158+0000 7f2e488f0640 1 --2- 172.21.15.113:0/1206389557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e44071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.159+0000 7f2e43fff640 1 --2- 172.21.15.113:0/1206389557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e440b5390 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97385adf8ccd09c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.159+0000 7f2e42ffd640 1 -- 172.21.15.113:0/1206389557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34037070 con 0x7f2e44144d60 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.159+0000 7f2e42ffd640 1 -- 172.21.15.113:0/1206389557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34032440 con 0x7f2e44144d60 2024-04-25T00:06:08.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.159+0000 7f2e42ffd640 1 -- 172.21.15.113:0/1206389557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34032810 con 0x7f2e44144d60 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.159+0000 7f2e4a353640 1 -- 172.21.15.113:0/1206389557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 msgr2=0x7f2e440b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.159+0000 7f2e4a353640 1 --2- 172.21.15.113:0/1206389557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e440b5390 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402a180 comp rx=0 tx=0).stop 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.160+0000 7f2e4a353640 1 -- 172.21.15.113:0/1206389557 shutdown_connections 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.160+0000 7f2e4a353640 1 --2- 172.21.15.113:0/1206389557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e44071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.160+0000 7f2e4a353640 1 --2- 172.21.15.113:0/1206389557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e440b5390 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.160+0000 7f2e4a353640 1 -- 172.21.15.113:0/1206389557 >> 172.21.15.113:0/1206389557 conn(0x7f2e440b1a50 msgr2=0x7f2e440b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.160+0000 7f2e4a353640 1 -- 172.21.15.113:0/1206389557 shutdown_connections 2024-04-25T00:06:08.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.160+0000 7f2e4a353640 1 -- 172.21.15.113:0/1206389557 wait complete. 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.161+0000 7f2e4a353640 1 Processor -- start 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.161+0000 7f2e4a353640 1 -- start start 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e4413c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e44158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e441453c0 0x7f2e44143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44145140 con 0x7f2e44144d60 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44070e10 con 0x7f2e441453c0 2024-04-25T00:06:08.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e44071870 con 0x7f2e4406e6e0 2024-04-25T00:06:08.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e44158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e44158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45730/0 (socket says 172.21.15.113:45730) 2024-04-25T00:06:08.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 -- 172.21.15.113:0/759468727 learned_addr learned my addr 172.21.15.113:0/759468727 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:08.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e488f0640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e441453c0 0x7f2e44143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e43fff640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e4413c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 -- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 msgr2=0x7f2e4413c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e4413c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 -- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e441453c0 msgr2=0x7f2e44143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e441453c0 0x7f2e44143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 -- 172.21.15.113:0/759468727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e44158dc0 con 0x7f2e44144d60 2024-04-25T00:06:08.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e488f0640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e441453c0 0x7f2e44143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:08.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e43fff640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e4413c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:08.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e437fe640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e44158880 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800b3f0 tx=0x7f2e3800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e40ff9640 1 -- 172.21.15.113:0/759468727 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38015020 con 0x7f2e44144d60 2024-04-25T00:06:08.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e40ff9640 1 -- 172.21.15.113:0/759468727 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e38011830 con 0x7f2e44144d60 2024-04-25T00:06:08.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441567d0 con 0x7f2e44144d60 2024-04-25T00:06:08.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e40ff9640 1 -- 172.21.15.113:0/759468727 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38011c20 con 0x7f2e44144d60 2024-04-25T00:06:08.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.162+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e44156d90 con 0x7f2e44144d60 2024-04-25T00:06:08.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.163+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e440691a0 con 0x7f2e44144d60 2024-04-25T00:06:08.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.164+0000 7f2e40ff9640 1 -- 172.21.15.113:0/759468727 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e38018020 con 0x7f2e44144d60 2024-04-25T00:06:08.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.164+0000 7f2e40ff9640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e3006be30 0x7f2e3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.164+0000 7f2e40ff9640 1 -- 172.21.15.113:0/759468727 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f2e3808aa30 con 0x7f2e44144d60 2024-04-25T00:06:08.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.164+0000 7f2e43fff640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e3006be30 0x7f2e3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.164+0000 7f2e43fff640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e3006be30 0x7f2e3006e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2e34034000 tx=0x7f2e3402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.168+0000 7f2e40ff9640 1 -- 172.21.15.113:0/759468727 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e3801f030 con 0x7f2e44144d60 2024-04-25T00:06:08.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.342+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e440b27e0 con 0x7f2e44144d60 2024-04-25T00:06:08.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.343+0000 7f2e40ff9640 1 -- 172.21.15.113:0/759468727 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16829 (secure 0 0 0) 0x7f2e380172c0 con 0x7f2e44144d60 2024-04-25T00:06:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:08.344 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":118,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:02.284662+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:05:33.644000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:05:34.412894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.21","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:08.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e3006be30 msgr2=0x7f2e3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e3006be30 0x7f2e3006e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2e34034000 tx=0x7f2e3402e000 comp rx=0 tx=0).stop 2024-04-25T00:06:08.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 msgr2=0x7f2e44158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e44158880 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800b3f0 tx=0x7f2e3800b8b0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 shutdown_connections 2024-04-25T00:06:08.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2e3006be30 0x7f2e3006e2f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e441453c0 0x7f2e44143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e44144d60 0x7f2e44158880 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 --2- 172.21.15.113:0/759468727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e4406e6e0 0x7f2e4413c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 >> 172.21.15.113:0/759468727 conn(0x7f2e440b1a50 msgr2=0x7f2e440cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 shutdown_connections 2024-04-25T00:06:08.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.345+0000 7f2e4a353640 1 -- 172.21.15.113:0/759468727 wait complete. 2024-04-25T00:06:08.357 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:06:08.358 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.454+0000 7f55667f9640 1 Processor -- start 2024-04-25T00:06:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.454+0000 7f55667f9640 1 -- start start 2024-04-25T00:06:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.454+0000 7f55667f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d2290 0x7f55600b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.454+0000 7f55667f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55600d0b00 0x7f55600d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.454+0000 7f55667f9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55600d2800 con 0x7f556015e320 2024-04-25T00:06:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.454+0000 7f55667f9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f55600d14d0 con 0x7f55600d0b00 2024-04-25T00:06:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.454+0000 7f55667f9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f55600d3410 con 0x7f55600d2290 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f5564d96640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55600d0b00 0x7f55600d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f5564d96640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55600d0b00 0x7f55600d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45734/0 (socket says 172.21.15.113:45734) 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f555ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f556015e320 0x7f55600d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51452/0 (socket says 172.21.15.113:51452) 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f5564d96640 1 -- 172.21.15.113:0/560075043 learned_addr learned my addr 172.21.15.113:0/560075043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f555f7fe640 1 --2- 172.21.15.113:0/560075043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d2290 0x7f55600b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f5564d96640 1 -- 172.21.15.113:0/560075043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d2290 msgr2=0x7f55600b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f5564d96640 1 --2- 172.21.15.113:0/560075043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d2290 0x7f55600b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f5564d96640 1 -- 172.21.15.113:0/560075043 >> v1:172.21.15.113:6789/0 conn(0x7f556015e320 legacy=0x7f55600d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f5564d96640 1 -- 172.21.15.113:0/560075043 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55600d2ae0 con 0x7f55600d0b00 2024-04-25T00:06:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.455+0000 7f555effd640 1 -- 172.21.15.113:0/560075043 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792003819 0 0) 0x7f55600d2800 con 0x7f556015e320 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f5564d96640 1 --2- 172.21.15.113:0/560075043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d0b00 0x7f55600d0f90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f5558009e50 tx=0x7f555802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6375727dd98aff0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f555effd640 1 -- 172.21.15.113:0/560075043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558036070 con 0x7f55600d0b00 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f555effd640 1 -- 172.21.15.113:0/560075043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5558031600 con 0x7f55600d0b00 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f555effd640 1 -- 172.21.15.113:0/560075043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580319d0 con 0x7f55600d0b00 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 -- 172.21.15.113:0/560075043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d0b00 msgr2=0x7f55600d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 --2- 172.21.15.113:0/560075043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d0b00 0x7f55600d0f90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f5558009e50 tx=0x7f555802a710 comp rx=0 tx=0).stop 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 -- 172.21.15.113:0/560075043 shutdown_connections 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 --2- 172.21.15.113:0/560075043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d0b00 0x7f55600d0f90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 --2- 172.21.15.113:0/560075043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d2290 0x7f55600b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 -- 172.21.15.113:0/560075043 >> 172.21.15.113:0/560075043 conn(0x7f55600b5c20 msgr2=0x7f55600b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 -- 172.21.15.113:0/560075043 shutdown_connections 2024-04-25T00:06:08.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.456+0000 7f55667f9640 1 -- 172.21.15.113:0/560075043 wait complete. 2024-04-25T00:06:08.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.457+0000 7f55667f9640 1 Processor -- start 2024-04-25T00:06:08.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f55667f9640 1 -- start start 2024-04-25T00:06:08.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f55667f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d0b00 0x7f55601f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f55667f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d2290 0x7f5560155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f55667f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556015e320 0x7f5560156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f55667f9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f55600b9fe0 con 0x7f55600d2290 2024-04-25T00:06:08.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f55667f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55600d30a0 con 0x7f556015e320 2024-04-25T00:06:08.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f55667f9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f556015e700 con 0x7f55600d0b00 2024-04-25T00:06:08.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d2290 0x7f5560155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d2290 0x7f5560155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45746/0 (socket says 172.21.15.113:45746) 2024-04-25T00:06:08.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.458+0000 7f555f7fe640 1 -- 172.21.15.113:0/1735599397 learned_addr learned my addr 172.21.15.113:0/1735599397 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:08.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f5564d96640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556015e320 0x7f5560156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555ffff640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d0b00 0x7f55601f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555f7fe640 1 -- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d0b00 msgr2=0x7f55601f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555f7fe640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d0b00 0x7f55601f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555f7fe640 1 -- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556015e320 msgr2=0x7f5560156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555f7fe640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556015e320 0x7f5560156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555f7fe640 1 -- 172.21.15.113:0/1735599397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556015ed90 con 0x7f55600d2290 2024-04-25T00:06:08.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555f7fe640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d2290 0x7f5560155d10 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f555400d830 tx=0x7f555400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555cff9640 1 -- 172.21.15.113:0/1735599397 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400b6c0 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555cff9640 1 -- 172.21.15.113:0/1735599397 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5554019070 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556015f070 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f555cff9640 1 -- 172.21.15.113:0/1735599397 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400ba80 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.459+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556015f630 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.460+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5560154290 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.460+0000 7f555cff9640 1 -- 172.21.15.113:0/1735599397 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f555401e460 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.461+0000 7f555cff9640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f554c06be30 0x7f554c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.461+0000 7f555ffff640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f554c06be30 0x7f554c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.461+0000 7f555cff9640 1 -- 172.21.15.113:0/1735599397 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f555408b5f0 con 0x7f55600d2290 2024-04-25T00:06:08.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.461+0000 7f555ffff640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f554c06be30 0x7f554c06e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5550000f60 tx=0x7f5550007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.465+0000 7f555cff9640 1 -- 172.21.15.113:0/1735599397 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5554059ee0 con 0x7f55600d2290 2024-04-25T00:06:08.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.673+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55600691a0 con 0x7f55600d2290 2024-04-25T00:06:08.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.673+0000 7f555cff9640 1 -- 172.21.15.113:0/1735599397 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f55540597e0 con 0x7f55600d2290 2024-04-25T00:06:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:08.674 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:06:08.674 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f554c06be30 msgr2=0x7f554c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f554c06be30 0x7f554c06e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5550000f60 tx=0x7f5550007c00 comp rx=0 tx=0).stop 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d2290 msgr2=0x7f5560155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d2290 0x7f5560155d10 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f555400d830 tx=0x7f555400dcf0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 shutdown_connections 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f554c06be30 0x7f554c06e2f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556015e320 0x7f5560156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55600d2290 0x7f5560155d10 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 --2- 172.21.15.113:0/1735599397 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55600d0b00 0x7f55601f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 >> 172.21.15.113:0/1735599397 conn(0x7f55600b5c20 msgr2=0x7f55600b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 shutdown_connections 2024-04-25T00:06:08.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.675+0000 7f55667f9640 1 -- 172.21.15.113:0/1735599397 wait complete. 2024-04-25T00:06:08.687 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:08.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.783+0000 7f985b215640 1 Processor -- start 2024-04-25T00:06:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.783+0000 7f985b215640 1 -- start start 2024-04-25T00:06:08.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f985b215640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9854144910 0x7f98540b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f985b215640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f985406e6e0 0x7f98540cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f985b215640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9854144cf0 con 0x7f98540b5240 2024-04-25T00:06:08.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f985b215640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f98540b4510 con 0x7f985406e6e0 2024-04-25T00:06:08.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f985b215640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f98540b2f30 con 0x7f9854144910 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f98597b2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f985406e6e0 0x7f98540cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f98597b2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f985406e6e0 0x7f98540cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45750/0 (socket says 172.21.15.113:45750) 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f98597b2640 1 -- 172.21.15.113:0/2197240204 learned_addr learned my addr 172.21.15.113:0/2197240204 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f98597b2640 1 -- 172.21.15.113:0/2197240204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9854144910 msgr2=0x7f98540b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f9858fb1640 1 --2- 172.21.15.113:0/2197240204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9854144910 0x7f98540b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f98597b2640 1 --2- 172.21.15.113:0/2197240204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9854144910 0x7f98540b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f98597b2640 1 -- 172.21.15.113:0/2197240204 >> v1:172.21.15.178:6789/0 conn(0x7f98540b5240 legacy=0x7f985406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.784+0000 7f98597b2640 1 -- 172.21.15.113:0/2197240204 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98540b3210 con 0x7f985406e6e0 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.785+0000 7f98597b2640 1 --2- 172.21.15.113:0/2197240204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f98540cd500 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f984c00d150 tx=0x7f984c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5833bc18f8368b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.785+0000 7f98537fe640 1 -- 172.21.15.113:0/2197240204 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c00b2e0 con 0x7f985406e6e0 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.785+0000 7f98537fe640 1 -- 172.21.15.113:0/2197240204 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c009600 con 0x7f985406e6e0 2024-04-25T00:06:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.785+0000 7f98537fe640 1 -- 172.21.15.113:0/2197240204 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c00b480 con 0x7f985406e6e0 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.785+0000 7f985b215640 1 -- 172.21.15.113:0/2197240204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 msgr2=0x7f98540cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.785+0000 7f985b215640 1 --2- 172.21.15.113:0/2197240204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f98540cd500 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f984c00d150 tx=0x7f984c02c8e0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.786+0000 7f985b215640 1 -- 172.21.15.113:0/2197240204 shutdown_connections 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.786+0000 7f985b215640 1 --2- 172.21.15.113:0/2197240204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f98540cd500 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.786+0000 7f985b215640 1 --2- 172.21.15.113:0/2197240204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9854144910 0x7f98540b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.786+0000 7f985b215640 1 -- 172.21.15.113:0/2197240204 >> 172.21.15.113:0/2197240204 conn(0x7f98540b1a50 msgr2=0x7f98540b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.786+0000 7f985b215640 1 -- 172.21.15.113:0/2197240204 shutdown_connections 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.786+0000 7f985b215640 1 -- 172.21.15.113:0/2197240204 wait complete. 2024-04-25T00:06:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.787+0000 7f985b215640 1 Processor -- start 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.787+0000 7f985b215640 1 -- start start 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f985413b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98540b5240 0x7f98540aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9854144910 0x7f98541540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98540b4da0 con 0x7f985406e6e0 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98540b37d0 con 0x7f9854144910 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f98540b4f20 con 0x7f98540b5240 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f985413b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f985413b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45758/0 (socket says 172.21.15.113:45758) 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 -- 172.21.15.113:0/3805534078 learned_addr learned my addr 172.21.15.113:0/3805534078 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f98597b2640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9854144910 0x7f98541540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9853fff640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98540b5240 0x7f98540aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 -- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98540b5240 msgr2=0x7f98540aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98540b5240 0x7f98540aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 -- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9854144910 msgr2=0x7f98541540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9854144910 0x7f98541540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 -- 172.21.15.113:0/3805534078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98540af400 con 0x7f985406e6e0 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f9858fb1640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f985413b710 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f984800c7f0 tx=0x7f984800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f98517fa640 1 -- 172.21.15.113:0/3805534078 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848015070 con 0x7f985406e6e0 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f98517fa640 1 -- 172.21.15.113:0/3805534078 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9848004240 con 0x7f985406e6e0 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f98517fa640 1 -- 172.21.15.113:0/3805534078 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848007620 con 0x7f985406e6e0 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985413c0d0 con 0x7f985406e6e0 2024-04-25T00:06:08.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.788+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985413c690 con 0x7f985406e6e0 2024-04-25T00:06:08.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.789+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985415e750 con 0x7f985406e6e0 2024-04-25T00:06:08.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.790+0000 7f98517fa640 1 -- 172.21.15.113:0/3805534078 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98480079d0 con 0x7f985406e6e0 2024-04-25T00:06:08.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.790+0000 7f98517fa640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f983806be30 0x7f983806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.790+0000 7f98517fa640 1 -- 172.21.15.113:0/3805534078 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f984808b1f0 con 0x7f985406e6e0 2024-04-25T00:06:08.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.790+0000 7f9853fff640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f983806be30 0x7f983806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.791+0000 7f9853fff640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f983806be30 0x7f983806e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f985415dbb0 tx=0x7f9840009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:08.794+0000 7f98517fa640 1 -- 172.21.15.113:0/3805534078 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f985415e750 con 0x7f985406e6e0 2024-04-25T00:06:09.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.001+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98540691a0 con 0x7f985406e6e0 2024-04-25T00:06:09.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.002+0000 7f98517fa640 1 -- 172.21.15.113:0/3805534078 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5192 (secure 0 0 0) 0x7f98540af400 con 0x7f985406e6e0 2024-04-25T00:06:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:09.002 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:05:35.432457+0000","modified":"2024-04-25T00:05:37.447344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8400":{"gid":8400,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/840156279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":840156279},{"type":"v1","addr":"172.21.15.178:6835","nonce":840156279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8400,"qdb_cluster":[8400]},"id":8}]} 2024-04-25T00:06:09.003 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-04-25T00:06:09.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f983806be30 msgr2=0x7f983806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f983806be30 0x7f983806e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f985415dbb0 tx=0x7f9840009040 comp rx=0 tx=0).stop 2024-04-25T00:06:09.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 msgr2=0x7f985413b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f985413b710 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f984800c7f0 tx=0x7f984800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 shutdown_connections 2024-04-25T00:06:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f983806be30 0x7f983806e2f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9854144910 0x7f98541540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98540b5240 0x7f98540aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 --2- 172.21.15.113:0/3805534078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985406e6e0 0x7f985413b710 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 >> 172.21.15.113:0/3805534078 conn(0x7f98540b1a50 msgr2=0x7f98540b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 shutdown_connections 2024-04-25T00:06:09.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.004+0000 7f985b215640 1 -- 172.21.15.113:0/3805534078 wait complete. 2024-04-25T00:06:09.015 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:09.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.111+0000 7f4b3c0f5640 1 Processor -- start 2024-04-25T00:06:09.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.112+0000 7f4b3c0f5640 1 -- start start 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b3c0f5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4b3406e6e0 0x7f4b340b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b3c0f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b341453c0 0x7f4b34071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b3c0f5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b340cfad0 con 0x7f4b34144d60 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b3c0f5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4b34145c40 con 0x7f4b3406e6e0 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b3c0f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4b340cf230 con 0x7f4b341453c0 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b39690640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4b3406e6e0 0x7f4b340b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b39690640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4b3406e6e0 0x7f4b340b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45762/0 (socket says 172.21.15.113:45762) 2024-04-25T00:06:09.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b39690640 1 -- 172.21.15.113:0/3959331159 learned_addr learned my addr 172.21.15.113:0/3959331159 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b3a692640 1 --2- 172.21.15.113:0/3959331159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b341453c0 0x7f4b34071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b39690640 1 -- 172.21.15.113:0/3959331159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b341453c0 msgr2=0x7f4b34071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b39690640 1 --2- 172.21.15.113:0/3959331159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b341453c0 0x7f4b34071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b39690640 1 -- 172.21.15.113:0/3959331159 >> v1:172.21.15.178:6789/0 conn(0x7f4b34144d60 legacy=0x7f4b3406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.113+0000 7f4b39690640 1 -- 172.21.15.113:0/3959331159 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b340cf510 con 0x7f4b3406e6e0 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b39690640 1 --2- 172.21.15.113:0/3959331159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b3406e6e0 0x7f4b340b5390 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009870 tx=0x7f4b3002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fb6f7c90beab7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b38e8f640 1 -- 172.21.15.113:0/3959331159 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30037070 con 0x7f4b3406e6e0 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b38e8f640 1 -- 172.21.15.113:0/3959331159 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b30032440 con 0x7f4b3406e6e0 2024-04-25T00:06:09.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b38e8f640 1 -- 172.21.15.113:0/3959331159 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30032810 con 0x7f4b3406e6e0 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/3959331159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b3406e6e0 msgr2=0x7f4b340b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/3959331159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b3406e6e0 0x7f4b340b5390 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009870 tx=0x7f4b3002a130 comp rx=0 tx=0).stop 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/3959331159 shutdown_connections 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/3959331159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b341453c0 0x7f4b34071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/3959331159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b3406e6e0 0x7f4b340b5390 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/3959331159 >> 172.21.15.113:0/3959331159 conn(0x7f4b340b1a50 msgr2=0x7f4b340b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/3959331159 shutdown_connections 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.114+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/3959331159 wait complete. 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.115+0000 7f4b3c0f5640 1 Processor -- start 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3c0f5640 1 -- start start 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3c0f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b3406e6e0 0x7f4b341f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3c0f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b34144d60 0x7f4b341f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3c0f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341453c0 0x7f4b3413e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3c0f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4b34145140 con 0x7f4b341453c0 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3c0f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4b34070e10 con 0x7f4b34144d60 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3c0f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4b34071870 con 0x7f4b3406e6e0 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3a692640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341453c0 0x7f4b3413e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3a692640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341453c0 0x7f4b3413e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45774/0 (socket says 172.21.15.113:45774) 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.116+0000 7f4b3a692640 1 -- 172.21.15.113:0/4259199193 learned_addr learned my addr 172.21.15.113:0/4259199193 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3a692640 1 -- 172.21.15.113:0/4259199193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b3406e6e0 msgr2=0x7f4b341f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3a692640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b3406e6e0 0x7f4b341f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3a692640 1 -- 172.21.15.113:0/4259199193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b34144d60 msgr2=0x7f4b341f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3a692640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b34144d60 0x7f4b341f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3a692640 1 -- 172.21.15.113:0/4259199193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b34139bf0 con 0x7f4b341453c0 2024-04-25T00:06:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3a692640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341453c0 0x7f4b3413e460 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00ec10 tx=0x7f4b2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b267fc640 1 -- 172.21.15.113:0/4259199193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c018020 con 0x7f4b341453c0 2024-04-25T00:06:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b267fc640 1 -- 172.21.15.113:0/4259199193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c00cd20 con 0x7f4b341453c0 2024-04-25T00:06:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b267fc640 1 -- 172.21.15.113:0/4259199193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0146c0 con 0x7f4b341453c0 2024-04-25T00:06:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b34139eb0 con 0x7f4b341453c0 2024-04-25T00:06:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.117+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3413a470 con 0x7f4b341453c0 2024-04-25T00:06:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.118+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b340691a0 con 0x7f4b341453c0 2024-04-25T00:06:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.118+0000 7f4b267fc640 1 -- 172.21.15.113:0/4259199193 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b2c005d10 con 0x7f4b341453c0 2024-04-25T00:06:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.119+0000 7f4b267fc640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4b1806be30 0x7f4b1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.119+0000 7f4b267fc640 1 -- 172.21.15.113:0/4259199193 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7f4b2c08ba80 con 0x7f4b341453c0 2024-04-25T00:06:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.119+0000 7f4b39e91640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4b1806be30 0x7f4b1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.119+0000 7f4b39e91640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4b1806be30 0x7f4b1806e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4b28000f60 tx=0x7f4b280079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.123+0000 7f4b267fc640 1 -- 172.21.15.113:0/4259199193 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b2c022070 con 0x7f4b341453c0 2024-04-25T00:06:09.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.299+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b340b27e0 con 0x7f4b341453c0 2024-04-25T00:06:09.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.301+0000 7f4b267fc640 1 -- 172.21.15.113:0/4259199193 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16829 (secure 0 0 0) 0x7f4b2c010040 con 0x7f4b341453c0 2024-04-25T00:06:09.301 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:09.301 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":118,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:02.284662+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:05:33.644000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T00:05:34.412894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.21","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:09.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4b1806be30 msgr2=0x7f4b1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4b1806be30 0x7f4b1806e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4b28000f60 tx=0x7f4b280079d0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341453c0 msgr2=0x7f4b3413e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341453c0 0x7f4b3413e460 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00ec10 tx=0x7f4b2c00c490 comp rx=0 tx=0).stop 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 shutdown_connections 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4b1806be30 0x7f4b1806e2f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b34144d60 0x7f4b341f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341453c0 0x7f4b3413e460 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 --2- 172.21.15.113:0/4259199193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b3406e6e0 0x7f4b341f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 >> 172.21.15.113:0/4259199193 conn(0x7f4b340b1a50 msgr2=0x7f4b340b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 shutdown_connections 2024-04-25T00:06:09.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.303+0000 7f4b3c0f5640 1 -- 172.21.15.113:0/4259199193 wait complete. 2024-04-25T00:06:09.315 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:06:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.411+0000 7fc49359e640 1 Processor -- start 2024-04-25T00:06:09.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.411+0000 7fc49359e640 1 -- start start 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc49359e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc4940d09e0 0x7fc4940cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc49359e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4940cd990 0x7fc4940af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc49359e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4940aeaf0 con 0x7fc49415c040 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc49359e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc4940af120 con 0x7fc4940cd990 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc49359e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc4940ce140 con 0x7fc4940d09e0 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc492d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4940cd990 0x7fc4940af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc49259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc49415c040 0x7fc4940d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51462/0 (socket says 172.21.15.113:51462) 2024-04-25T00:06:09.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.412+0000 7fc49259c640 1 -- 172.21.15.113:0/2422537632 learned_addr learned my addr 172.21.15.113:0/2422537632 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc491d9b640 1 --2- 172.21.15.113:0/2422537632 >> v2:172.21.15.178:3300/0 conn(0x7fc4940d09e0 0x7fc4940cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc492d9d640 1 -- 172.21.15.113:0/2422537632 >> v2:172.21.15.178:3300/0 conn(0x7fc4940d09e0 msgr2=0x7fc4940cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc492d9d640 1 --2- 172.21.15.113:0/2422537632 >> v2:172.21.15.178:3300/0 conn(0x7fc4940d09e0 0x7fc4940cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc492d9d640 1 -- 172.21.15.113:0/2422537632 >> v1:172.21.15.113:6789/0 conn(0x7fc49415c040 legacy=0x7fc4940d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc49159a640 1 -- 172.21.15.113:0/2422537632 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733307287 0 0) 0x7fc4940aeaf0 con 0x7fc49415c040 2024-04-25T00:06:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc492d9d640 1 -- 172.21.15.113:0/2422537632 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4940ce420 con 0x7fc4940cd990 2024-04-25T00:06:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc491d9b640 1 --2- 172.21.15.113:0/2422537632 >> v2:172.21.15.178:3300/0 conn(0x7fc4940d09e0 0x7fc4940cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:09.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.413+0000 7fc492d9d640 1 --2- 172.21.15.113:0/2422537632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940cd990 0x7fc4940af5b0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc4940af2a0 tx=0x7fc48402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cc6b1eefe92642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49159a640 1 -- 172.21.15.113:0/2422537632 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484007740 con 0x7fc4940cd990 2024-04-25T00:06:09.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49159a640 1 -- 172.21.15.113:0/2422537632 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4840078e0 con 0x7fc4940cd990 2024-04-25T00:06:09.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49159a640 1 -- 172.21.15.113:0/2422537632 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484007cb0 con 0x7fc4940cd990 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 -- 172.21.15.113:0/2422537632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940cd990 msgr2=0x7fc4940af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 --2- 172.21.15.113:0/2422537632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940cd990 0x7fc4940af5b0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fc4940af2a0 tx=0x7fc48402a6f0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 -- 172.21.15.113:0/2422537632 shutdown_connections 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 --2- 172.21.15.113:0/2422537632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940cd990 0x7fc4940af5b0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 --2- 172.21.15.113:0/2422537632 >> v2:172.21.15.178:3300/0 conn(0x7fc4940d09e0 0x7fc4940cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 -- 172.21.15.113:0/2422537632 >> 172.21.15.113:0/2422537632 conn(0x7fc49409aa20 msgr2=0x7fc49409ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 -- 172.21.15.113:0/2422537632 shutdown_connections 2024-04-25T00:06:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.414+0000 7fc49359e640 1 -- 172.21.15.113:0/2422537632 wait complete. 2024-04-25T00:06:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.415+0000 7fc49359e640 1 Processor -- start 2024-04-25T00:06:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc49359e640 1 -- start start 2024-04-25T00:06:09.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc49359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4940cd990 0x7fc4940b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc49359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940d09e0 0x7fc4940c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc49359e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc49415c040 0x7fc4941f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc49359e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc49415c420 con 0x7fc4940d09e0 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc49359e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940cdde0 con 0x7fc4940cd990 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc49359e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc4940afaf0 con 0x7fc49415c040 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc491d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940d09e0 0x7fc4940c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc491d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940d09e0 0x7fc4940c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45790/0 (socket says 172.21.15.113:45790) 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.416+0000 7fc491d9b640 1 -- 172.21.15.113:0/3104707731 learned_addr learned my addr 172.21.15.113:0/3104707731 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc49259c640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4940cd990 0x7fc4940b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc492d9d640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc49415c040 0x7fc4941f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc491d9b640 1 -- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc49415c040 msgr2=0x7fc4941f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc491d9b640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc49415c040 0x7fc4941f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc491d9b640 1 -- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4940cd990 msgr2=0x7fc4940b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc491d9b640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4940cd990 0x7fc4940b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc491d9b640 1 -- 172.21.15.113:0/3104707731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49406b190 con 0x7fc4940d09e0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc492d9d640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc49415c040 0x7fc4941f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc49259c640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4940cd990 0x7fc4940b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc491d9b640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940d09e0 0x7fc4940c5960 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c7f0 tx=0x7fc48800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc482ffd640 1 -- 172.21.15.113:0/3104707731 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488015070 con 0x7fc4940d09e0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc482ffd640 1 -- 172.21.15.113:0/3104707731 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4880079d0 con 0x7fc4940d09e0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc482ffd640 1 -- 172.21.15.113:0/3104707731 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488007da0 con 0x7fc4940d09e0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4940c8200 con 0x7fc4940d09e0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.417+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4940c87c0 con 0x7fc4940d09e0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.418+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4940691a0 con 0x7fc4940d09e0 2024-04-25T00:06:09.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.418+0000 7fc482ffd640 1 -- 172.21.15.113:0/3104707731 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc488018070 con 0x7fc4940d09e0 2024-04-25T00:06:09.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.419+0000 7fc482ffd640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc47406be30 0x7fc47406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.419+0000 7fc482ffd640 1 -- 172.21.15.113:0/3104707731 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5767+0+0 (secure 0 0 0) 0x7fc48808ad90 con 0x7fc4940d09e0 2024-04-25T00:06:09.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.419+0000 7fc49259c640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc47406be30 0x7fc47406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.419+0000 7fc49259c640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc47406be30 0x7fc47406e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000f60 tx=0x7fc48c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.423+0000 7fc482ffd640 1 -- 172.21.15.113:0/3104707731 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc488059790 con 0x7fc4940d09e0 2024-04-25T00:06:09.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.631+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc4940cb530 con 0x7fc4940d09e0 2024-04-25T00:06:09.632 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:06:09.632+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:06:09.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.633+0000 7fc482ffd640 1 -- 172.21.15.113:0/3104707731 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7fc488059090 con 0x7fc4940d09e0 2024-04-25T00:06:09.634 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:06:09.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.636+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc47406be30 msgr2=0x7fc47406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.636+0000 7fc49359e640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc47406be30 0x7fc47406e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000f60 tx=0x7fc48c009040 comp rx=0 tx=0).stop 2024-04-25T00:06:09.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.636+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940d09e0 msgr2=0x7fc4940c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.636+0000 7fc49359e640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940d09e0 0x7fc4940c5960 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c7f0 tx=0x7fc48800cbf0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 shutdown_connections 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc47406be30 0x7fc47406e2f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc49415c040 0x7fc4941f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4940d09e0 0x7fc4940c5960 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 --2- 172.21.15.113:0/3104707731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4940cd990 0x7fc4940b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 >> 172.21.15.113:0/3104707731 conn(0x7fc49409aa20 msgr2=0x7fc4940d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 shutdown_connections 2024-04-25T00:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.637+0000 7fc49359e640 1 -- 172.21.15.113:0/3104707731 wait complete. 2024-04-25T00:06:09.649 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:06:09.675 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:06:09.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.748+0000 7fba16c64640 1 Processor -- start 2024-04-25T00:06:09.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.748+0000 7fba16c64640 1 -- start start 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.748+0000 7fba16c64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.748+0000 7fba16c64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba100d28c0 0x7fba100b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.748+0000 7fba16c64640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba100d2e00 con 0x7fba100b61e0 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.748+0000 7fba16c64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fba100d30e0 con 0x7fba100d28c0 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.748+0000 7fba16c64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fba100d33c0 con 0x7fba1015e510 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba100d28c0 0x7fba100b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba100d28c0 0x7fba100b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45794/0 (socket says 172.21.15.113:45794) 2024-04-25T00:06:09.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0ffff640 1 -- 172.21.15.113:0/1168133212 learned_addr learned my addr 172.21.15.113:0/1168133212 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba14a00640 1 --2- 172.21.15.113:0/1168133212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0ffff640 1 -- 172.21.15.113:0/1168133212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 msgr2=0x7fba100d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0ffff640 1 --2- 172.21.15.113:0/1168133212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0ffff640 1 -- 172.21.15.113:0/1168133212 >> v1:172.21.15.113:6789/0 conn(0x7fba100b61e0 legacy=0x7fba10071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0f7fe640 1 -- 172.21.15.113:0/1168133212 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546126132 0 0) 0x7fba100d2e00 con 0x7fba100b61e0 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.749+0000 7fba0ffff640 1 -- 172.21.15.113:0/1168133212 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba100d36a0 con 0x7fba100d28c0 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba0ffff640 1 --2- 172.21.15.113:0/1168133212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba100b5ca0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fba04009870 tx=0x7fba0402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16701c30dad591d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba0f7fe640 1 -- 172.21.15.113:0/1168133212 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04037070 con 0x7fba100d28c0 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba0f7fe640 1 -- 172.21.15.113:0/1168133212 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba04032440 con 0x7fba100d28c0 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba0f7fe640 1 -- 172.21.15.113:0/1168133212 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04032810 con 0x7fba100d28c0 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 -- 172.21.15.113:0/1168133212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 msgr2=0x7fba100b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 --2- 172.21.15.113:0/1168133212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba100b5ca0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fba04009870 tx=0x7fba0402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 -- 172.21.15.113:0/1168133212 shutdown_connections 2024-04-25T00:06:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 --2- 172.21.15.113:0/1168133212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba100b5ca0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 --2- 172.21.15.113:0/1168133212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 -- 172.21.15.113:0/1168133212 >> 172.21.15.113:0/1168133212 conn(0x7fba1006b720 msgr2=0x7fba100d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 -- 172.21.15.113:0/1168133212 shutdown_connections 2024-04-25T00:06:09.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.750+0000 7fba16c64640 1 -- 172.21.15.113:0/1168133212 wait complete. 2024-04-25T00:06:09.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 Processor -- start 2024-04-25T00:06:09.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 -- start start 2024-04-25T00:06:09.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba100b61e0 0x7fba101613e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba1015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba100b9dd0 con 0x7fba100d28c0 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba100d3ea0 con 0x7fba100b61e0 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.752+0000 7fba16c64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fba100b83d0 con 0x7fba1015e510 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba1015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba1015f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45810/0 (socket says 172.21.15.113:45810) 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 -- 172.21.15.113:0/2340843887 learned_addr learned my addr 172.21.15.113:0/2340843887 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba14a00640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba100b61e0 0x7fba101613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba15201640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 -- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 msgr2=0x7fba100b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 -- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba100b61e0 msgr2=0x7fba101613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba100b61e0 0x7fba101613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 -- 172.21.15.113:0/2340843887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba100c9030 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba14a00640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba100b61e0 0x7fba101613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0ffff640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba1015f340 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fba04035040 tx=0x7fba040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0d7fa640 1 -- 172.21.15.113:0/2340843887 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04037040 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0d7fa640 1 -- 172.21.15.113:0/2340843887 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba04004230 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba10160340 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba0d7fa640 1 -- 172.21.15.113:0/2340843887 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04003610 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.753+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba10160900 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.754+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba100691a0 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.755+0000 7fba0d7fa640 1 -- 172.21.15.113:0/2340843887 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba0402ad10 con 0x7fba100d28c0 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.755+0000 7fba0d7fa640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9fc06be60 0x7fb9fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.755+0000 7fba0d7fa640 1 -- 172.21.15.113:0/2340843887 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5853+0+0 (secure 0 0 0) 0x7fba040ab8c0 con 0x7fba100d28c0 2024-04-25T00:06:09.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.755+0000 7fba14a00640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9fc06be60 0x7fb9fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.756+0000 7fba14a00640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9fc06be60 0x7fb9fc06e320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fba00006fd0 tx=0x7fba00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.759+0000 7fba0d7fa640 1 -- 172.21.15.113:0/2340843887 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba0407a390 con 0x7fba100d28c0 2024-04-25T00:06:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:09.966+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fba100d0ee0 con 0x7fba100d28c0 2024-04-25T00:06:10.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.635+0000 7fba0d7fa640 1 -- 172.21.15.113:0/2340843887 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7fba04079c90 con 0x7fba100d28c0 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9fc06be60 msgr2=0x7fb9fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9fc06be60 0x7fb9fc06e320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fba00006fd0 tx=0x7fba00008040 comp rx=0 tx=0).stop 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 msgr2=0x7fba1015f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba1015f340 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fba04035040 tx=0x7fba040046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 shutdown_connections 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9fc06be60 0x7fb9fc06e320 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba1015e510 0x7fba100b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba100d28c0 0x7fba1015f340 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 --2- 172.21.15.113:0/2340843887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba100b61e0 0x7fba101613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.637+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 >> 172.21.15.113:0/2340843887 conn(0x7fba1006b720 msgr2=0x7fba100b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:10.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.638+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 shutdown_connections 2024-04-25T00:06:10.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.638+0000 7fba16c64640 1 -- 172.21.15.113:0/2340843887 wait complete. 2024-04-25T00:06:10.651 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:06:10.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.755+0000 7f45c3709640 1 Processor -- start 2024-04-25T00:06:10.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.755+0000 7f45c3709640 1 -- start start 2024-04-25T00:06:10.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45c3709640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 0x7f45bc0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45c3709640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc072f70 con 0x7f45bc06e7e0 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45c3709640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc0b2fd0 con 0x7f45bc0b3ff0 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45c3709640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0b32b0 con 0x7f45bc144a80 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45c1ca6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f45bc06e7e0 0x7f45bc0cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51476/0 (socket says 172.21.15.113:51476) 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45c1ca6640 1 -- 172.21.15.113:0/3079615864 learned_addr learned my addr 172.21.15.113:0/3079615864 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 35602790 0 0) 0x7f45bc072f70 con 0x7f45bc06e7e0 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a8003520 con 0x7f45bc06e7e0 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45c14a5640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 0x7f45bc0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3809753077 0 0) 0x7f45a8003520 con 0x7f45bc06e7e0 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 msgr2=0x7f45bc0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 0x7f45bc0b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 >> v1:172.21.15.178:6789/0 conn(0x7f45bc0b3ff0 legacy=0x7f45bc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc0b3590 con 0x7f45bc06e7e0 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f45b40019f0 con 0x7f45bc06e7e0 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 >> v1:172.21.15.113:6789/0 conn(0x7f45bc06e7e0 legacy=0x7f45bc0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a80039e0 0x7f45a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 0x7f45a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0b32b0 con 0x7f45a8004d40 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800bba0 con 0x7f45a80039e0 2024-04-25T00:06:10.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.756+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45a800dfb0 con 0x7f45bc144a80 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c1ca6640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c14a5640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a80039e0 0x7f45a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c0ca4640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 0x7f45a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c1ca6640 1 -- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 msgr2=0x7f45a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c1ca6640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc144a80 0x7f45a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c1ca6640 1 -- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a80039e0 msgr2=0x7f45a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c1ca6640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a80039e0 0x7f45a8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c1ca6640 1 -- 172.21.15.113:0/3079615864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a8003520 con 0x7f45a8004d40 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.757+0000 7f45c1ca6640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f45bc06c3b0 tx=0x7f45a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6581354e75d7183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400a4a0 con 0x7f45a8004d40 2024-04-25T00:06:10.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b402f070 con 0x7f45a8004d40 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45b3fff640 1 -- 172.21.15.113:0/3079615864 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b402bcb0 con 0x7f45a8004d40 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45c3709640 1 -- 172.21.15.113:0/3079615864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 msgr2=0x7f45a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45c3709640 1 --2- 172.21.15.113:0/3079615864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f45bc06c3b0 tx=0x7f45a8005830 comp rx=0 tx=0).stop 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45c1ca6640 1 -- 172.21.15.113:0/3079615864 reap_dead start 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45c3709640 1 -- 172.21.15.113:0/3079615864 shutdown_connections 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45c3709640 1 -- 172.21.15.113:0/3079615864 >> 172.21.15.113:0/3079615864 conn(0x7f45bc0b1b30 msgr2=0x7f45bc0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45c3709640 1 -- 172.21.15.113:0/3079615864 shutdown_connections 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.758+0000 7f45c3709640 1 -- 172.21.15.113:0/3079615864 wait complete. 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.759+0000 7f45c3709640 1 Processor -- start 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.759+0000 7f45c3709640 1 -- start start 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c3709640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45bc142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c3709640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc1f7770 0x7f45bc143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c3709640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45bc1f7bd0 0x7f45bc144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c3709640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0b3ab0 con 0x7f45a8004d40 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c3709640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0cdf90 con 0x7f45bc1f7bd0 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c3709640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0b3c30 con 0x7f45bc1f7770 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45bc142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45bc142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45836/0 (socket says 172.21.15.113:45836) 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 -- 172.21.15.113:0/630353090 learned_addr learned my addr 172.21.15.113:0/630353090 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 -- 172.21.15.113:0/630353090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc1f7770 msgr2=0x7f45bc143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc1f7770 0x7f45bc143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 -- 172.21.15.113:0/630353090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45bc1f7bd0 msgr2=0x7f45bc144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45bc1f7bd0 0x7f45bc144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.760+0000 7f45c14a5640 1 -- 172.21.15.113:0/630353090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc142890 con 0x7f45a8004d40 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.761+0000 7f45c14a5640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45bc142350 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f45b800b850 tx=0x7f45b800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:10.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.761+0000 7f45b1ffb640 1 -- 172.21.15.113:0/630353090 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8018070 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.761+0000 7f45b1ffb640 1 -- 172.21.15.113:0/630353090 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b800be40 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.761+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc1412b0 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.761+0000 7f45b1ffb640 1 -- 172.21.15.113:0/630353090 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8010be0 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.761+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc141870 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.762+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc0691a0 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.762+0000 7f45b1ffb640 1 -- 172.21.15.113:0/630353090 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45b801b070 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.763+0000 7f45b1ffb640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45a006be30 0x7f45a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.763+0000 7f45b1ffb640 1 -- 172.21.15.113:0/630353090 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5853+0+0 (secure 0 0 0) 0x7f45b808b270 con 0x7f45a8004d40 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.763+0000 7f45c0ca4640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45a006be30 0x7f45a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:10.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.763+0000 7f45c0ca4640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45a006be30 0x7f45a006e2f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f45bc13ed20 tx=0x7f45ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:10.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.767+0000 7f45b1ffb640 1 -- 172.21.15.113:0/630353090 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f45b8059b90 con 0x7f45a8004d40 2024-04-25T00:06:10.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:10.939+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f45bc0b29a0 con 0x7f45a8004d40 2024-04-25T00:06:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.639+0000 7f45b1ffb640 1 -- 172.21.15.113:0/630353090 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f45b8059490 con 0x7f45a8004d40 2024-04-25T00:06:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:06:11.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.644+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45a006be30 msgr2=0x7f45a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:11.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.644+0000 7f45c3709640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45a006be30 0x7f45a006e2f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f45bc13ed20 tx=0x7f45ac009040 comp rx=0 tx=0).stop 2024-04-25T00:06:11.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.644+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 msgr2=0x7f45bc142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:11.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.644+0000 7f45c3709640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45bc142350 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f45b800b850 tx=0x7f45b800bd10 comp rx=0 tx=0).stop 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.645+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 shutdown_connections 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.645+0000 7f45c3709640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45a006be30 0x7f45a006e2f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.645+0000 7f45c3709640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45bc1f7bd0 0x7f45bc144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.645+0000 7f45c3709640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45bc1f7770 0x7f45bc143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.645+0000 7f45c3709640 1 --2- 172.21.15.113:0/630353090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45a8004d40 0x7f45bc142350 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.645+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 >> 172.21.15.113:0/630353090 conn(0x7f45bc0b1b30 msgr2=0x7f45bc0ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.645+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 shutdown_connections 2024-04-25T00:06:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.646+0000 7f45c3709640 1 -- 172.21.15.113:0/630353090 wait complete. 2024-04-25T00:06:11.660 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:06:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.762+0000 7fa8f0ad2640 1 Processor -- start 2024-04-25T00:06:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.762+0000 7fa8f0ad2640 1 -- start start 2024-04-25T00:06:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8f0ad2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8f0ad2640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa8ec0b3ff0 0x7fa8ec06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8f0ad2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8ec06e610 0x7fa8ec0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8f0ad2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa8ec072f70 con 0x7fa8ec06e610 2024-04-25T00:06:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8f0ad2640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa8ec0b2fd0 con 0x7fa8ec0b3ff0 2024-04-25T00:06:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8f0ad2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec0b32b0 con 0x7fa8ec144a80 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8ec06e610 0x7fa8ec0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8ec06e610 0x7fa8ec0cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45846/0 (socket says 172.21.15.113:45846) 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 -- 172.21.15.113:0/712650711 learned_addr learned my addr 172.21.15.113:0/712650711 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ea59c640 1 --2- 172.21.15.113:0/712650711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8e3fff640 1 --2- 172.21.15.113:0/712650711 >> v2:172.21.15.178:3300/0 conn(0x7fa8ec0b3ff0 0x7fa8ec06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 -- 172.21.15.113:0/712650711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 msgr2=0x7fa8ec0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 --2- 172.21.15.113:0/712650711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec0b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 -- 172.21.15.113:0/712650711 >> v2:172.21.15.178:3300/0 conn(0x7fa8ec0b3ff0 msgr2=0x7fa8ec06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 --2- 172.21.15.113:0/712650711 >> v2:172.21.15.178:3300/0 conn(0x7fa8ec0b3ff0 0x7fa8ec06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.763+0000 7fa8ead9d640 1 -- 172.21.15.113:0/712650711 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec0b3590 con 0x7fa8ec06e610 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8ead9d640 1 --2- 172.21.15.113:0/712650711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec0cd630 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0d0d80 tx=0x7fa8e402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb63ec36acc5eed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8e9d9b640 1 -- 172.21.15.113:0/712650711 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4036070 con 0x7fa8ec06e610 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8e9d9b640 1 -- 172.21.15.113:0/712650711 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e4031600 con 0x7fa8ec06e610 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8e9d9b640 1 -- 172.21.15.113:0/712650711 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40319d0 con 0x7fa8ec06e610 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/712650711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 msgr2=0x7fa8ec0cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/712650711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec0cd630 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0d0d80 tx=0x7fa8e402a610 comp rx=0 tx=0).stop 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/712650711 shutdown_connections 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/712650711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec0cd630 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/712650711 >> v2:172.21.15.178:3300/0 conn(0x7fa8ec0b3ff0 0x7fa8ec06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/712650711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec0b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.764+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/712650711 >> 172.21.15.113:0/712650711 conn(0x7fa8ec0b1b30 msgr2=0x7fa8ec0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.765+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/712650711 shutdown_connections 2024-04-25T00:06:11.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.765+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/712650711 wait complete. 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.766+0000 7fa8f0ad2640 1 Processor -- start 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.766+0000 7fa8f0ad2640 1 -- start start 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.766+0000 7fa8f0ad2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.766+0000 7fa8f0ad2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8ec0b3ff0 0x7fa8ec0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8f0ad2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec1e6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8f0ad2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec0cdf00 con 0x7fa8ec06e610 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8f0ad2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec0cdd80 con 0x7fa8ec0b3ff0 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8f0ad2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec072200 con 0x7fa8ec144a80 2024-04-25T00:06:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45862/0 (socket says 172.21.15.113:45862) 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 -- 172.21.15.113:0/3262457158 learned_addr learned my addr 172.21.15.113:0/3262457158 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8e3fff640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8ec0b3ff0 0x7fa8ec0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ead9d640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec1e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 -- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 msgr2=0x7fa8ec1e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec1e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 -- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8ec0b3ff0 msgr2=0x7fa8ec0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8ec0b3ff0 0x7fa8ec0aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 -- 172.21.15.113:0/3262457158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec144140 con 0x7fa8ec06e610 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8ea59c640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec159200 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d620 tx=0x7fa8dc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8e2ffd640 1 -- 172.21.15.113:0/3262457158 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00b620 con 0x7fa8ec06e610 2024-04-25T00:06:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8e2ffd640 1 -- 172.21.15.113:0/3262457158 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8dc019070 con 0x7fa8ec06e610 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec159800 con 0x7fa8ec06e610 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8e2ffd640 1 -- 172.21.15.113:0/3262457158 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00ba00 con 0x7fa8ec06e610 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.767+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec159dc0 con 0x7fa8ec06e610 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.768+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec0691a0 con 0x7fa8ec06e610 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.769+0000 7fa8e2ffd640 1 -- 172.21.15.113:0/3262457158 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8dc00bba0 con 0x7fa8ec06e610 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.769+0000 7fa8e2ffd640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa8cc06be30 0x7fa8cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.769+0000 7fa8e2ffd640 1 -- 172.21.15.113:0/3262457158 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5415+0+0 (secure 0 0 0) 0x7fa8dc08aff0 con 0x7fa8ec06e610 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.769+0000 7fa8e3fff640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa8cc06be30 0x7fa8cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.770+0000 7fa8e3fff640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa8cc06be30 0x7fa8cc06e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec1e53d0 tx=0x7fa8d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.773+0000 7fa8e2ffd640 1 -- 172.21.15.113:0/3262457158 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8ec144140 con 0x7fa8ec06e610 2024-04-25T00:06:11.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:11.951+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8ec0cd950 con 0x7fa8ec06e610 2024-04-25T00:06:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.641+0000 7fa8e2ffd640 1 -- 172.21.15.113:0/3262457158 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7fa8dc059470 con 0x7fa8ec06e610 2024-04-25T00:06:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.645+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa8cc06be30 msgr2=0x7fa8cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.645+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa8cc06be30 0x7fa8cc06e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec1e53d0 tx=0x7fa8d4008040 comp rx=0 tx=0).stop 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.645+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 msgr2=0x7fa8ec159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.645+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec159200 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d620 tx=0x7fa8dc00dae0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.646+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 shutdown_connections 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.646+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa8cc06be30 0x7fa8cc06e2f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.646+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8ec0b3ff0 0x7fa8ec0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.646+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8ec144a80 0x7fa8ec1e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.646+0000 7fa8f0ad2640 1 --2- 172.21.15.113:0/3262457158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8ec06e610 0x7fa8ec159200 secure :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d620 tx=0x7fa8dc00dae0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.646+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 >> 172.21.15.113:0/3262457158 conn(0x7fa8ec0b1b30 msgr2=0x7fa8ec0af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.648+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 shutdown_connections 2024-04-25T00:06:12.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.648+0000 7fa8f0ad2640 1 -- 172.21.15.113:0/3262457158 wait complete. 2024-04-25T00:06:12.670 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:06:12.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.774+0000 7f579d302640 1 Processor -- start 2024-04-25T00:06:12.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.774+0000 7f579d302640 1 -- start start 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f579d302640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f57980b5ca0 0x7f57980b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f579d302640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57980d2e00 con 0x7f579815e510 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f579d302640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57980d30e0 con 0x7f57980b73c0 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f579d302640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f57980b6640 con 0x7f57980b5ca0 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f57967fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f57980b5ca0 0x7f57980b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f5796ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f579815e510 0x7f57980d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51484/0 (socket says 172.21.15.113:51484) 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f5796ffd640 1 -- 172.21.15.113:0/3630486837 learned_addr learned my addr 172.21.15.113:0/3630486837 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f57967fc640 1 -- 172.21.15.113:0/3630486837 >> v1:172.21.15.178:6789/0 conn(0x7f57980b73c0 legacy=0x7f57980b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f57967fc640 1 -- 172.21.15.113:0/3630486837 >> v1:172.21.15.113:6789/0 conn(0x7f579815e510 legacy=0x7f57980d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f5795ffb640 1 -- 172.21.15.113:0/3630486837 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212518033 0 0) 0x7f57980d2e00 con 0x7f579815e510 2024-04-25T00:06:12.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.775+0000 7f57967fc640 1 -- 172.21.15.113:0/3630486837 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57980d33c0 con 0x7f57980b5ca0 2024-04-25T00:06:12.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.776+0000 7f57967fc640 1 --2- 172.21.15.113:0/3630486837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b5ca0 0x7f57980b6100 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f578c0097a0 tx=0x7f578c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af420c0c7544a213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.776+0000 7f5795ffb640 1 -- 172.21.15.113:0/3630486837 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c037070 con 0x7f57980b5ca0 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.776+0000 7f5795ffb640 1 -- 172.21.15.113:0/3630486837 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f578c02ae50 con 0x7f57980b5ca0 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.776+0000 7f5795ffb640 1 -- 172.21.15.113:0/3630486837 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c032660 con 0x7f57980b5ca0 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.777+0000 7f579d302640 1 -- 172.21.15.113:0/3630486837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b5ca0 msgr2=0x7f57980b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.777+0000 7f579d302640 1 --2- 172.21.15.113:0/3630486837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b5ca0 0x7f57980b6100 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f578c0097a0 tx=0x7f578c02a140 comp rx=0 tx=0).stop 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.777+0000 7f579d302640 1 -- 172.21.15.113:0/3630486837 shutdown_connections 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.777+0000 7f579d302640 1 --2- 172.21.15.113:0/3630486837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b5ca0 0x7f57980b6100 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.777+0000 7f579d302640 1 -- 172.21.15.113:0/3630486837 >> 172.21.15.113:0/3630486837 conn(0x7f579806b720 msgr2=0x7f57980d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.777+0000 7f579d302640 1 -- 172.21.15.113:0/3630486837 shutdown_connections 2024-04-25T00:06:12.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.777+0000 7f579d302640 1 -- 172.21.15.113:0/3630486837 wait complete. 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.778+0000 7f579d302640 1 Processor -- start 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.778+0000 7f579d302640 1 -- start start 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f579d302640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57980b5ca0 0x7f57981f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f579d302640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b73c0 0x7f57980cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f579d302640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f579815e510 0x7f57980cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f579d302640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f57980d3ea0 con 0x7f57980b73c0 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f579d302640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57980d3980 con 0x7f57980b5ca0 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f579d302640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57980d2a90 con 0x7f579815e510 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b73c0 0x7f57980cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b73c0 0x7f57980cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45878/0 (socket says 172.21.15.113:45878) 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 -- 172.21.15.113:0/2612472808 learned_addr learned my addr 172.21.15.113:0/2612472808 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f5796ffd640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57980b5ca0 0x7f57981f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:12.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57977fe640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f579815e510 0x7f57980cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 -- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f579815e510 msgr2=0x7f57980cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f579815e510 0x7f57980cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 -- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57980b5ca0 msgr2=0x7f57981f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57980b5ca0 0x7f57981f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 -- 172.21.15.113:0/2612472808 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57980b8890 con 0x7f57980b73c0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.779+0000 7f57967fc640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b73c0 0x7f57980cb2f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f578c0098c0 tx=0x7f578c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.780+0000 7f577f7fe640 1 -- 172.21.15.113:0/2612472808 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c042070 con 0x7f57980b73c0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.780+0000 7f577f7fe640 1 -- 172.21.15.113:0/2612472808 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f578c03e670 con 0x7f57980b73c0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.780+0000 7f577f7fe640 1 -- 172.21.15.113:0/2612472808 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c037040 con 0x7f57980b73c0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.780+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57980b8b70 con 0x7f57980b73c0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.780+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57980b3f90 con 0x7f57980b73c0 2024-04-25T00:06:12.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.781+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57980691a0 con 0x7f57980b73c0 2024-04-25T00:06:12.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.781+0000 7f577f7fe640 1 -- 172.21.15.113:0/2612472808 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f578c045070 con 0x7f57980b73c0 2024-04-25T00:06:12.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.781+0000 7f577f7fe640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578406be30 0x7f578406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:12.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.781+0000 7f577f7fe640 1 -- 172.21.15.113:0/2612472808 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f578c0ab4c0 con 0x7f57980b73c0 2024-04-25T00:06:12.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.781+0000 7f5796ffd640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578406be30 0x7f578406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:12.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.782+0000 7f5796ffd640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578406be30 0x7f578406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f57880012e0 tx=0x7f5788001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:12.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.786+0000 7f577f7fe640 1 -- 172.21.15.113:0/2612472808 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f578c079c40 con 0x7f57980b73c0 2024-04-25T00:06:12.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.956+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f579815b750 con 0x7f57980b73c0 2024-04-25T00:06:12.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.959+0000 7f577f7fe640 1 -- 172.21.15.113:0/2612472808 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f578c07df40 con 0x7f57980b73c0 2024-04-25T00:06:12.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578406be30 msgr2=0x7f578406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578406be30 0x7f578406e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f57880012e0 tx=0x7f5788001700 comp rx=0 tx=0).stop 2024-04-25T00:06:12.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b73c0 msgr2=0x7f57980cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b73c0 0x7f57980cb2f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f578c0098c0 tx=0x7f578c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 shutdown_connections 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578406be30 0x7f578406e2f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f579815e510 0x7f57980cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57980b73c0 0x7f57980cb2f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 --2- 172.21.15.113:0/2612472808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57980b5ca0 0x7f57981f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 >> 172.21.15.113:0/2612472808 conn(0x7f579806b720 msgr2=0x7f579815b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 shutdown_connections 2024-04-25T00:06:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:12.961+0000 7f579d302640 1 -- 172.21.15.113:0/2612472808 wait complete. 2024-04-25T00:06:12.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T00:06:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.070+0000 7fe4515b7640 1 Processor -- start 2024-04-25T00:06:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.070+0000 7fe4515b7640 1 -- start start 2024-04-25T00:06:13.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe4515b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe4515b7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe44c0b5b70 0x7fe44c0b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe4515b7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe44c0cf480 con 0x7fe44c15e630 2024-04-25T00:06:13.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe4515b7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe44c0b6520 con 0x7fe44c0b5b70 2024-04-25T00:06:13.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe4515b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c0cf760 con 0x7fe44c15e250 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe44c0b5b70 0x7fe44c0b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe44c0b5b70 0x7fe44c0b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45884/0 (socket says 172.21.15.113:45884) 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2150066262 learned_addr learned my addr 172.21.15.113:0/2150066262 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44affd640 1 --2- 172.21.15.113:0/2150066262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2150066262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 msgr2=0x7fe44c0b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44b7fe640 1 --2- 172.21.15.113:0/2150066262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2150066262 >> v1:172.21.15.178:6789/0 conn(0x7fe44c15e630 legacy=0x7fe44c0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2150066262 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c0d2370 con 0x7fe44c0b5b70 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.071+0000 7fe44affd640 1 --2- 172.21.15.113:0/2150066262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe44b7fe640 1 --2- 172.21.15.113:0/2150066262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c0b5b70 0x7fe44c0b5fe0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe444009e50 tx=0x7fe44402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903b50a18b6a52ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe44a7fc640 1 -- 172.21.15.113:0/2150066262 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444036070 con 0x7fe44c0b5b70 2024-04-25T00:06:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe44a7fc640 1 -- 172.21.15.113:0/2150066262 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe444031600 con 0x7fe44c0b5b70 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe44a7fc640 1 -- 172.21.15.113:0/2150066262 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4440319d0 con 0x7fe44c0b5b70 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe4515b7640 1 -- 172.21.15.113:0/2150066262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c0b5b70 msgr2=0x7fe44c0b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2150066262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c0b5b70 0x7fe44c0b5fe0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe444009e50 tx=0x7fe44402a6c0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe4515b7640 1 -- 172.21.15.113:0/2150066262 shutdown_connections 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2150066262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c0b5b70 0x7fe44c0b5fe0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2150066262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe4515b7640 1 -- 172.21.15.113:0/2150066262 >> 172.21.15.113:0/2150066262 conn(0x7fe44c071df0 msgr2=0x7fe44c0d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.072+0000 7fe4515b7640 1 -- 172.21.15.113:0/2150066262 shutdown_connections 2024-04-25T00:06:13.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.073+0000 7fe4515b7640 1 -- 172.21.15.113:0/2150066262 wait complete. 2024-04-25T00:06:13.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.074+0000 7fe4515b7640 1 Processor -- start 2024-04-25T00:06:13.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.074+0000 7fe4515b7640 1 -- start start 2024-04-25T00:06:13.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.074+0000 7fe4515b7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe44c0b5b70 0x7fe44c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.074+0000 7fe4515b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.074+0000 7fe4515b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c15e630 0x7fe44c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe4515b7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c0cfa40 con 0x7fe44c15e630 2024-04-25T00:06:13.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe4515b7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c0b3770 con 0x7fe44c0b5b70 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe4515b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c0cf0f0 con 0x7fe44c15e250 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c15e630 0x7fe44c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c15e630 0x7fe44c15f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45888/0 (socket says 172.21.15.113:45888) 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2850832191 learned_addr learned my addr 172.21.15.113:0/2850832191 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44affd640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe44c0b5b70 0x7fe44c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe4427fc640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 msgr2=0x7fe44c0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe44c0b5b70 msgr2=0x7fe44c154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe44c0b5b70 0x7fe44c154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 -- 172.21.15.113:0/2850832191 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c0cba80 con 0x7fe44c15e630 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe44b7fe640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c15e630 0x7fe44c15f3a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe444009f70 tx=0x7fe444004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:13.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe443fff640 1 -- 172.21.15.113:0/2850832191 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444036050 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe443fff640 1 -- 172.21.15.113:0/2850832191 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe444042070 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe443fff640 1 -- 172.21.15.113:0/2850832191 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444009510 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c0cbd60 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.075+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c0cc320 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.076+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c068220 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.077+0000 7fe443fff640 1 -- 172.21.15.113:0/2850832191 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe444007bd0 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.077+0000 7fe443fff640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe42c06be30 0x7fe42c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.077+0000 7fe443fff640 1 -- 172.21.15.113:0/2850832191 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7fe4440ab900 con 0x7fe44c15e630 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.077+0000 7fe44affd640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe42c06be30 0x7fe42c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.078+0000 7fe44affd640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe42c06be30 0x7fe42c06e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ab60 tx=0x7fe43c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:13.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.082+0000 7fe443fff640 1 -- 172.21.15.113:0/2850832191 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe444079ff0 con 0x7fe44c15e630 2024-04-25T00:06:13.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.345+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) -- 0x7fe44c0d3440 con 0x7fe44c15e630 2024-04-25T00:06:13.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.649+0000 7fe443fff640 1 -- 172.21.15.113:0/2850832191 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v392) ==== 166+0+0 (secure 0 0 0) 0x7fe44407e2f0 con 0x7fe44c15e630 2024-04-25T00:06:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe42c06be30 msgr2=0x7fe42c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe42c06be30 0x7fe42c06e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ab60 tx=0x7fe43c008040 comp rx=0 tx=0).stop 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c15e630 msgr2=0x7fe44c15f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c15e630 0x7fe44c15f3a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe444009f70 tx=0x7fe444004a40 comp rx=0 tx=0).stop 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 shutdown_connections 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe42c06be30 0x7fe42c06e2f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe44c15e630 0x7fe44c15f3a0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe44c15e250 0x7fe44c0cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 --2- 172.21.15.113:0/2850832191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe44c0b5b70 0x7fe44c154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 >> 172.21.15.113:0/2850832191 conn(0x7fe44c071df0 msgr2=0x7fe44c0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 shutdown_connections 2024-04-25T00:06:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.651+0000 7fe4515b7640 1 -- 172.21.15.113:0/2850832191 wait complete. 2024-04-25T00:06:13.663 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T00:06:13.663 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T00:06:13.664 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:13.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.760+0000 7f7f8755a640 1 Processor -- start 2024-04-25T00:06:13.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.760+0000 7f7f8755a640 1 -- start start 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f8755a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7f8006e6e0 0x7f7f800b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f8755a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 0x7f7f80071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f8755a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f800cfad0 con 0x7f7f80144d60 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f8755a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7f80145c40 con 0x7f7f8006e6e0 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f8755a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f800cf230 con 0x7f7f801453c0 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f852f6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7f80144d60 0x7f7f8006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51494/0 (socket says 172.21.15.113:51494) 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f852f6640 1 -- 172.21.15.113:0/2719144115 learned_addr learned my addr 172.21.15.113:0/2719144115 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f84af5640 1 --2- 172.21.15.113:0/2719144115 >> v2:172.21.15.178:3300/0 conn(0x7f7f8006e6e0 0x7f7f800b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f85af7640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 0x7f7f80071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239286251 0 0) 0x7f7f800cfad0 con 0x7f7f80144d60 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f6c003520 con 0x7f7f80144d60 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.761+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 984618882 0 0) 0x7f7f6c003520 con 0x7f7f80144d60 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 msgr2=0x7f7f80071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 0x7f7f80071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 >> v2:172.21.15.178:3300/0 conn(0x7f7f8006e6e0 msgr2=0x7f7f800b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 --2- 172.21.15.113:0/2719144115 >> v2:172.21.15.178:3300/0 conn(0x7f7f8006e6e0 0x7f7f800b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f800cf510 con 0x7f7f80144d60 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f7f7c0019f0 con 0x7f7f80144d60 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 >> v1:172.21.15.113:6789/0 conn(0x7f7f80144d60 legacy=0x7f7f8006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c0039b0 0x7f7f6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 0x7f7f6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80145c40 con 0x7f7f6c004800 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f800cf230 con 0x7f7f6c0039b0 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c00ba70 con 0x7f7f801453c0 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f84af5640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f852f6640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c0039b0 0x7f7f6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f85af7640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 0x7f7f6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f84af5640 1 -- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 msgr2=0x7f7f6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f84af5640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801453c0 0x7f7f6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f84af5640 1 -- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c0039b0 msgr2=0x7f7f6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f84af5640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c0039b0 0x7f7f6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.762+0000 7f7f84af5640 1 -- 172.21.15.113:0/2719144115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c003520 con 0x7f7f6c004800 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f84af5640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f6c004c80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009a90 tx=0x7f7f6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7589bbd374ef4672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70036070 con 0x7f7f6c004800 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f70031400 con 0x7f7f6c004800 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f77fff640 1 -- 172.21.15.113:0/2719144115 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70009690 con 0x7f7f6c004800 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f8755a640 1 -- 172.21.15.113:0/2719144115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 msgr2=0x7f7f6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f8755a640 1 --2- 172.21.15.113:0/2719144115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f6c004c80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009a90 tx=0x7f7f6c005830 comp rx=0 tx=0).stop 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f85af7640 1 -- 172.21.15.113:0/2719144115 reap_dead start 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f8755a640 1 -- 172.21.15.113:0/2719144115 shutdown_connections 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f8755a640 1 -- 172.21.15.113:0/2719144115 >> 172.21.15.113:0/2719144115 conn(0x7f7f800b1a50 msgr2=0x7f7f800b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f8755a640 1 -- 172.21.15.113:0/2719144115 shutdown_connections 2024-04-25T00:06:13.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.763+0000 7f7f8755a640 1 -- 172.21.15.113:0/2719144115 wait complete. 2024-04-25T00:06:13.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 Processor -- start 2024-04-25T00:06:13.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 -- start start 2024-04-25T00:06:13.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f8015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8015b660 0x7f7f80138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801433a0 0x7f7f80143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80145140 con 0x7f7f6c004800 2024-04-25T00:06:13.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80070e10 con 0x7f7f8015b660 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f8755a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f80071330 con 0x7f7f801433a0 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f852f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f8015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f852f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f8015ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45902/0 (socket says 172.21.15.113:45902) 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.765+0000 7f7f852f6640 1 -- 172.21.15.113:0/1664780114 learned_addr learned my addr 172.21.15.113:0/1664780114 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f852f6640 1 -- 172.21.15.113:0/1664780114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801433a0 msgr2=0x7f7f80143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f852f6640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801433a0 0x7f7f80143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f852f6640 1 -- 172.21.15.113:0/1664780114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8015b660 msgr2=0x7f7f80138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f852f6640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8015b660 0x7f7f80138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f852f6640 1 -- 172.21.15.113:0/1664780114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f80139950 con 0x7f7f6c004800 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f852f6640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f8015ad40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0076f0 tx=0x7f7f7c007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f75ffb640 1 -- 172.21.15.113:0/1664780114 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c017020 con 0x7f7f6c004800 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f75ffb640 1 -- 172.21.15.113:0/1664780114 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7c00abf0 con 0x7f7f6c004800 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f80139670 con 0x7f7f6c004800 2024-04-25T00:06:13.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f75ffb640 1 -- 172.21.15.113:0/1664780114 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0133f0 con 0x7f7f6c004800 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.766+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f80139f10 con 0x7f7f6c004800 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.767+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f800691a0 con 0x7f7f6c004800 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.767+0000 7f7f75ffb640 1 -- 172.21.15.113:0/1664780114 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f7c01a020 con 0x7f7f6c004800 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.768+0000 7f7f75ffb640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6406be30 0x7f7f6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.768+0000 7f7f75ffb640 1 -- 172.21.15.113:0/1664780114 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f7f7c08b460 con 0x7f7f6c004800 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.768+0000 7f7f84af5640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6406be30 0x7f7f6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.768+0000 7f7f84af5640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6406be30 0x7f7f6406e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7f70033000 tx=0x7f7f7002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.772+0000 7f7f75ffb640 1 -- 172.21.15.113:0/1664780114 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f7c021020 con 0x7f7f6c004800 2024-04-25T00:06:13.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.945+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f800b27e0 con 0x7f7f6c004800 2024-04-25T00:06:13.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.946+0000 7f7f75ffb640 1 -- 172.21.15.113:0/1664780114 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+13078 (secure 0 0 0) 0x7f7f7c05a110 con 0x7f7f6c004800 2024-04-25T00:06:13.946 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:13.946 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":121,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:12.640516+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/840156279":"2024-04-26T00:06:09.633072+0000","172.21.15.178:6833/840156279":"2024-04-26T00:06:09.633072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:13.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6406be30 msgr2=0x7f7f6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6406be30 0x7f7f6406e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7f70033000 tx=0x7f7f7002e000 comp rx=0 tx=0).stop 2024-04-25T00:06:13.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 msgr2=0x7f7f8015ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:13.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f8015ad40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0076f0 tx=0x7f7f7c007bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 shutdown_connections 2024-04-25T00:06:13.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7f6406be30 0x7f7f6406e2f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8015b660 0x7f7f80138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f801433a0 0x7f7f80143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 --2- 172.21.15.113:0/1664780114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c004800 0x7f7f8015ad40 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:13.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 >> 172.21.15.113:0/1664780114 conn(0x7f7f800b1a50 msgr2=0x7f7f800af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:13.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 shutdown_connections 2024-04-25T00:06:13.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:13.948+0000 7f7f8755a640 1 -- 172.21.15.113:0/1664780114 wait complete. 2024-04-25T00:06:13.960 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:06:13.960 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:06:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.059+0000 7f43f922a640 1 Processor -- start 2024-04-25T00:06:14.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.059+0000 7f43f922a640 1 -- start start 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f922a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 0x7f43f40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f922a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f43f40d28c0 0x7f43f40b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f922a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43f40d2e00 con 0x7f43f40b61e0 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f922a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f43f40d30e0 con 0x7f43f40d28c0 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f922a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43f40d33c0 con 0x7f43f415e510 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f359e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f43f40b61e0 0x7f43f4071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51502/0 (socket says 172.21.15.113:51502) 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f359e640 1 -- 172.21.15.113:0/2736750086 learned_addr learned my addr 172.21.15.113:0/2736750086 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.060+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1467907385 0 0) 0x7f43f40d2e00 con 0x7f43f40b61e0 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43dc003520 con 0x7f43f40b61e0 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f259c640 1 --2- 172.21.15.113:0/2736750086 >> v2:172.21.15.178:3300/0 conn(0x7f43f40d28c0 0x7f43f40b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f2d9d640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 0x7f43f40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2560111404 0 0) 0x7f43dc003520 con 0x7f43f40b61e0 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 msgr2=0x7f43f40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 0x7f43f40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 >> v2:172.21.15.178:3300/0 conn(0x7f43f40d28c0 msgr2=0x7f43f40b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 --2- 172.21.15.113:0/2736750086 >> v2:172.21.15.178:3300/0 conn(0x7f43f40d28c0 0x7f43f40b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f40d36a0 con 0x7f43f40b61e0 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f43ec001670 con 0x7f43f40b61e0 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f2d9d640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 0x7f43f40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 >> v1:172.21.15.113:6789/0 conn(0x7f43f40b61e0 legacy=0x7f43f4071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 0x7f43dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43dc003f40 0x7f43dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc004d40 0x7f43dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40d30e0 con 0x7f43dc003f40 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40d33c0 con 0x7f43dc004d40 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc00bac0 con 0x7f43f415e510 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f259c640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43dc003f40 0x7f43dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f2d9d640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 0x7f43dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f359e640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc004d40 0x7f43dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f259c640 1 -- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 msgr2=0x7f43dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f259c640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415e510 0x7f43dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f259c640 1 -- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc004d40 msgr2=0x7f43dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f259c640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc004d40 0x7f43dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f259c640 1 -- 172.21.15.113:0/2736750086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc003520 con 0x7f43dc003f40 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.061+0000 7f43f359e640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc004d40 0x7f43dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.062+0000 7f43f259c640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43dc003f40 0x7f43dc004800 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009ac0 tx=0x7f43dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c778b6bc6ee4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.062+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8036070 con 0x7f43dc003f40 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.062+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e8031400 con 0x7f43dc003f40 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.062+0000 7f43f1d9b640 1 -- 172.21.15.113:0/2736750086 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8009690 con 0x7f43dc003f40 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.062+0000 7f43f922a640 1 -- 172.21.15.113:0/2736750086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43dc003f40 msgr2=0x7f43dc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.062+0000 7f43f922a640 1 --2- 172.21.15.113:0/2736750086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43dc003f40 0x7f43dc004800 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009ac0 tx=0x7f43dc005830 comp rx=0 tx=0).stop 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.063+0000 7f43f359e640 1 -- 172.21.15.113:0/2736750086 reap_dead start 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.063+0000 7f43f922a640 1 -- 172.21.15.113:0/2736750086 shutdown_connections 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.063+0000 7f43f922a640 1 -- 172.21.15.113:0/2736750086 >> 172.21.15.113:0/2736750086 conn(0x7f43f406b720 msgr2=0x7f43f40d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.063+0000 7f43f922a640 1 -- 172.21.15.113:0/2736750086 shutdown_connections 2024-04-25T00:06:14.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.063+0000 7f43f922a640 1 -- 172.21.15.113:0/2736750086 wait complete. 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.064+0000 7f43f922a640 1 Processor -- start 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.064+0000 7f43f922a640 1 -- start start 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f922a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc003f40 0x7f43f4159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f922a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415a3b0 0x7f43f41f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f922a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43f40cce30 0x7f43f40cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f922a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40b9dd0 con 0x7f43f40cce30 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f922a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40d3ea0 con 0x7f43dc003f40 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f922a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43f40b83d0 con 0x7f43f415a3b0 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43f40cce30 0x7f43f40cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43f40cce30 0x7f43f40cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45922/0 (socket says 172.21.15.113:45922) 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 -- 172.21.15.113:0/354839603 learned_addr learned my addr 172.21.15.113:0/354839603 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f259c640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415a3b0 0x7f43f41f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 -- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415a3b0 msgr2=0x7f43f41f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f2d9d640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc003f40 0x7f43f4159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415a3b0 0x7f43f41f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 -- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc003f40 msgr2=0x7f43f4159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc003f40 0x7f43f4159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 -- 172.21.15.113:0/354839603 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f415a130 con 0x7f43f40cce30 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43f359e640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43f40cce30 0x7f43f40cd2b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f43f40d2140 tx=0x7f43ec00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.065+0000 7f43db7fe640 1 -- 172.21.15.113:0/354839603 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec019070 con 0x7f43f40cce30 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.066+0000 7f43db7fe640 1 -- 172.21.15.113:0/354839603 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec00c530 con 0x7f43f40cce30 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.066+0000 7f43db7fe640 1 -- 172.21.15.113:0/354839603 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00c900 con 0x7f43f40cce30 2024-04-25T00:06:14.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.066+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f41f8180 con 0x7f43f40cce30 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.066+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f40c7cd0 con 0x7f43f40cce30 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.066+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f40b9f50 con 0x7f43f40cce30 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.067+0000 7f43db7fe640 1 -- 172.21.15.113:0/354839603 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43ec01c070 con 0x7f43f40cce30 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.067+0000 7f43db7fe640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f43e006be30 0x7f43e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.067+0000 7f43db7fe640 1 -- 172.21.15.113:0/354839603 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f43ec089ef0 con 0x7f43f40cce30 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.067+0000 7f43f2d9d640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f43e006be30 0x7f43e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.068+0000 7f43f2d9d640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f43e006be30 0x7f43e006e2f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43f41f8910 tx=0x7f43e400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.072+0000 7f43db7fe640 1 -- 172.21.15.113:0/354839603 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f43f40b9f50 con 0x7f43f40cce30 2024-04-25T00:06:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.244+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f43f40691a0 con 0x7f43f40cce30 2024-04-25T00:06:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.244+0000 7f43db7fe640 1 -- 172.21.15.113:0/354839603 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f43ec058c30 con 0x7f43f40cce30 2024-04-25T00:06:14.245 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f43e006be30 msgr2=0x7f43e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f43e006be30 0x7f43e006e2f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43f41f8910 tx=0x7f43e400b040 comp rx=0 tx=0).stop 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43f40cce30 msgr2=0x7f43f40cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43f40cce30 0x7f43f40cd2b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f43f40d2140 tx=0x7f43ec00eb90 comp rx=0 tx=0).stop 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 shutdown_connections 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f43e006be30 0x7f43e006e2f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43f40cce30 0x7f43f40cd2b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43f415a3b0 0x7f43f41f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 --2- 172.21.15.113:0/354839603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc003f40 0x7f43f4159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 >> 172.21.15.113:0/354839603 conn(0x7f43f406b720 msgr2=0x7f43f4071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:14.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 shutdown_connections 2024-04-25T00:06:14.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.246+0000 7f43f922a640 1 -- 172.21.15.113:0/354839603 wait complete. 2024-04-25T00:06:14.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T00:06:14.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.355+0000 7f86108b3640 1 Processor -- start 2024-04-25T00:06:14.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.355+0000 7f86108b3640 1 -- start start 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f86108b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f860c0b3fd0 0x7f860c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f86108b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f860c06e510 0x7f860c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f86108b3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f860c144cf0 con 0x7f860c144910 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f86108b3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f860c06e980 con 0x7f860c0b3fd0 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f86108b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f860c0b2f30 con 0x7f860c06e510 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f8609d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f860c0b3fd0 0x7f860c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f8609d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f860c0b3fd0 0x7f860c06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45926/0 (socket says 172.21.15.113:45926) 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f8609d9b640 1 -- 172.21.15.113:0/3133399151 learned_addr learned my addr 172.21.15.113:0/3133399151 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:14.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.356+0000 7f860ad9d640 1 --2- 172.21.15.113:0/3133399151 >> v2:172.21.15.178:3300/0 conn(0x7f860c06e510 0x7f860c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f8609d9b640 1 -- 172.21.15.113:0/3133399151 >> v2:172.21.15.178:3300/0 conn(0x7f860c06e510 msgr2=0x7f860c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f8609d9b640 1 --2- 172.21.15.113:0/3133399151 >> v2:172.21.15.178:3300/0 conn(0x7f860c06e510 0x7f860c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f8609d9b640 1 -- 172.21.15.113:0/3133399151 >> v1:172.21.15.178:6789/0 conn(0x7f860c144910 legacy=0x7f860c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f8609d9b640 1 -- 172.21.15.113:0/3133399151 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c0b3210 con 0x7f860c0b3fd0 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f8609d9b640 1 --2- 172.21.15.113:0/3133399151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c06dfd0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f860000bb10 tx=0x7f860002c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654743d6be520fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f860959a640 1 -- 172.21.15.113:0/3133399151 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600036020 con 0x7f860c0b3fd0 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f860959a640 1 -- 172.21.15.113:0/3133399151 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8600032820 con 0x7f860c0b3fd0 2024-04-25T00:06:14.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.357+0000 7f860959a640 1 -- 172.21.15.113:0/3133399151 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600032bf0 con 0x7f860c0b3fd0 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 -- 172.21.15.113:0/3133399151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 msgr2=0x7f860c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 --2- 172.21.15.113:0/3133399151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c06dfd0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f860000bb10 tx=0x7f860002c960 comp rx=0 tx=0).stop 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 -- 172.21.15.113:0/3133399151 shutdown_connections 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 --2- 172.21.15.113:0/3133399151 >> v2:172.21.15.178:3300/0 conn(0x7f860c06e510 0x7f860c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 --2- 172.21.15.113:0/3133399151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c06dfd0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 -- 172.21.15.113:0/3133399151 >> 172.21.15.113:0/3133399151 conn(0x7f860c0b1a50 msgr2=0x7f860c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 -- 172.21.15.113:0/3133399151 shutdown_connections 2024-04-25T00:06:14.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.358+0000 7f86108b3640 1 -- 172.21.15.113:0/3133399151 wait complete. 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.359+0000 7f86108b3640 1 Processor -- start 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.359+0000 7f86108b3640 1 -- start start 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f86108b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f860c06e510 0x7f860c138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f86108b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f86108b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860c144910 0x7f860c14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f86108b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f860c0b4440 con 0x7f860c0b3fd0 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f86108b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f860c0b37d0 con 0x7f860c06e510 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f86108b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f860c0b4da0 con 0x7f860c144910 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c0ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45932/0 (socket says 172.21.15.113:45932) 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 -- 172.21.15.113:0/1132726470 learned_addr learned my addr 172.21.15.113:0/1132726470 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f860ad9d640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860c144910 0x7f860c14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f860a59c640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f860c06e510 0x7f860c138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 -- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860c144910 msgr2=0x7f860c14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860c144910 0x7f860c14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 -- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f860c06e510 msgr2=0x7f860c138200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f860c06e510 0x7f860c138200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f8609d9b640 1 -- 172.21.15.113:0/1132726470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c0cd5f0 con 0x7f860c0b3fd0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.360+0000 7f860ad9d640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860c144910 0x7f860c14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.361+0000 7f8609d9b640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c0ce690 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f860000b2c0 tx=0x7f86000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.361+0000 7f85faffd640 1 -- 172.21.15.113:0/1132726470 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600036040 con 0x7f860c0b3fd0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.361+0000 7f85faffd640 1 -- 172.21.15.113:0/1132726470 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860003cd70 con 0x7f860c0b3fd0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.361+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860c0cd8d0 con 0x7f860c0b3fd0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.361+0000 7f85faffd640 1 -- 172.21.15.113:0/1132726470 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600046630 con 0x7f860c0b3fd0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.361+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860c0cde90 con 0x7f860c0b3fd0 2024-04-25T00:06:14.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.362+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860c0691a0 con 0x7f860c0b3fd0 2024-04-25T00:06:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.362+0000 7f85faffd640 1 -- 172.21.15.113:0/1132726470 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f860004d060 con 0x7f860c0b3fd0 2024-04-25T00:06:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.363+0000 7f85faffd640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85ec06be30 0x7f85ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.363+0000 7f85faffd640 1 -- 172.21.15.113:0/1132726470 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f86000adcf0 con 0x7f860c0b3fd0 2024-04-25T00:06:14.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.363+0000 7f860a59c640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85ec06be30 0x7f85ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.363+0000 7f860a59c640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85ec06be30 0x7f85ec06e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f85fc007fd0 tx=0x7f85fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.367+0000 7f85faffd640 1 -- 172.21.15.113:0/1132726470 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86000b2070 con 0x7f860c0b3fd0 2024-04-25T00:06:14.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.607+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) -- 0x7f860c1429b0 con 0x7f860c0b3fd0 2024-04-25T00:06:14.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.650+0000 7f85faffd640 1 -- 172.21.15.113:0/1132726470 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v393) ==== 169+0+0 (secure 0 0 0) 0x7f8600041070 con 0x7f860c0b3fd0 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85ec06be30 msgr2=0x7f85ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85ec06be30 0x7f85ec06e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f85fc007fd0 tx=0x7f85fc009040 comp rx=0 tx=0).stop 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 msgr2=0x7f860c0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c0ce690 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f860000b2c0 tx=0x7f86000046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 shutdown_connections 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85ec06be30 0x7f85ec06e2f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860c144910 0x7f860c14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860c0b3fd0 0x7f860c0ce690 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 --2- 172.21.15.113:0/1132726470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f860c06e510 0x7f860c138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 >> 172.21.15.113:0/1132726470 conn(0x7f860c0b1a50 msgr2=0x7f860c071fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:14.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 shutdown_connections 2024-04-25T00:06:14.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.652+0000 7f86108b3640 1 -- 172.21.15.113:0/1132726470 wait complete. 2024-04-25T00:06:14.664 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:06:14.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.762+0000 7f54893e2640 1 Processor -- start 2024-04-25T00:06:14.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.762+0000 7f54893e2640 1 -- start start 2024-04-25T00:06:14.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.762+0000 7f54893e2640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f54840cd4b0 0x7f548406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.762+0000 7f54893e2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f548415ea00 con 0x7f5484061b80 2024-04-25T00:06:14.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.762+0000 7f54893e2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5484162490 con 0x7f5484161fd0 2024-04-25T00:06:14.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.762+0000 7f54893e2640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f54840cfc30 con 0x7f54840cd4b0 2024-04-25T00:06:14.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f54837fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5484061b80 0x7f548415eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51506/0 (socket says 172.21.15.113:51506) 2024-04-25T00:06:14.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f54837fe640 1 -- 172.21.15.113:0/2380191935 learned_addr learned my addr 172.21.15.113:0/2380191935 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44227794 0 0) 0x7f548415ea00 con 0x7f5484061b80 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f546c003520 con 0x7f5484061b80 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5482ffd640 1 --2- 172.21.15.113:0/2380191935 >> v2:172.21.15.178:3300/0 conn(0x7f54840cd4b0 0x7f548406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4040698401 0 0) 0x7f546c003520 con 0x7f5484061b80 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 >> v2:172.21.15.178:3300/0 conn(0x7f54840cd4b0 msgr2=0x7f548406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 --2- 172.21.15.113:0/2380191935 >> v2:172.21.15.178:3300/0 conn(0x7f54840cd4b0 0x7f548406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 >> v1:172.21.15.178:6789/0 conn(0x7f5484161fd0 legacy=0x7f54840cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5484162770 con 0x7f5484061b80 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f547c0019f0 con 0x7f5484061b80 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 >> v1:172.21.15.113:6789/0 conn(0x7f5484061b80 legacy=0x7f548415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f546c0039b0 0x7f546c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c004800 0x7f546c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f546c0051c0 0x7f546c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54840cfc30 con 0x7f546c0039b0 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f546c00c110 con 0x7f546c0051c0 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f546c00c310 con 0x7f546c004800 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5482ffd640 1 --2- 172.21.15.113:0/2380191935 >> v2:172.21.15.178:3300/0 conn(0x7f54840cd4b0 0x7f548406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f5482ffd640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f546c0039b0 0x7f546c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f54827fc640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c004800 0x7f546c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.763+0000 7f54837fe640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f546c0051c0 0x7f546c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f5482ffd640 1 -- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c004800 msgr2=0x7f546c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f5482ffd640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c004800 0x7f546c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f5482ffd640 1 -- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f546c0051c0 msgr2=0x7f546c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f5482ffd640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f546c0051c0 0x7f546c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f5482ffd640 1 -- 172.21.15.113:0/2380191935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c003520 con 0x7f546c0039b0 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f54837fe640 1 -- 172.21.15.113:0/2380191935 reap_dead start 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f5482ffd640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f546c0039b0 0x7f546c003dd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f54740098a0 tx=0x7f546c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c92a329ebec17d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.764+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474037070 con 0x7f546c0039b0 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547402ac30 con 0x7f546c0039b0 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f5481ffb640 1 -- 172.21.15.113:0/2380191935 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547400f040 con 0x7f546c0039b0 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f54893e2640 1 -- 172.21.15.113:0/2380191935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f546c0039b0 msgr2=0x7f546c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f54893e2640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f546c0039b0 0x7f546c003dd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f54740098a0 tx=0x7f546c005830 comp rx=0 tx=0).stop 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f54893e2640 1 -- 172.21.15.113:0/2380191935 shutdown_connections 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f54893e2640 1 --2- 172.21.15.113:0/2380191935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f546c0039b0 0x7f546c003dd0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f54893e2640 1 -- 172.21.15.113:0/2380191935 >> 172.21.15.113:0/2380191935 conn(0x7f54840d0e30 msgr2=0x7f54840d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f54893e2640 1 -- 172.21.15.113:0/2380191935 shutdown_connections 2024-04-25T00:06:14.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.765+0000 7f54893e2640 1 -- 172.21.15.113:0/2380191935 wait complete. 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.766+0000 7f54893e2640 1 Processor -- start 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.766+0000 7f54893e2640 1 -- start start 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54893e2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c0039b0 0x7f54841f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54893e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54841f4f80 0x7f54840ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54893e2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54840aedf0 0x7f54840c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54893e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54840cf1d0 con 0x7f54841f4f80 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54893e2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5484162d30 con 0x7f54840aedf0 2024-04-25T00:06:14.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54893e2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f548406c210 con 0x7f546c0039b0 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54827fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54841f4f80 0x7f54840ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54827fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54841f4f80 0x7f54840ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45954/0 (socket says 172.21.15.113:45954) 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54827fc640 1 -- 172.21.15.113:0/4276294480 learned_addr learned my addr 172.21.15.113:0/4276294480 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54837fe640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54840aedf0 0x7f54840c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54827fc640 1 -- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c0039b0 msgr2=0x7f54841f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f5482ffd640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c0039b0 0x7f54841f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.767+0000 7f54827fc640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c0039b0 0x7f54841f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f54827fc640 1 -- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54840aedf0 msgr2=0x7f54840c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f54827fc640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54840aedf0 0x7f54840c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f54827fc640 1 -- 172.21.15.113:0/4276294480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54840b4000 con 0x7f54841f4f80 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f54827fc640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54841f4f80 0x7f54840ae8b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5478001f00 tx=0x7f547800fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f546b7fe640 1 -- 172.21.15.113:0/4276294480 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547801a070 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f546b7fe640 1 -- 172.21.15.113:0/4276294480 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5478005320 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f546b7fe640 1 -- 172.21.15.113:0/4276294480 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478006a40 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54840b42e0 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.768+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54840c6940 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.769+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54840691a0 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.769+0000 7f546b7fe640 1 -- 172.21.15.113:0/4276294480 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f547801d070 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.770+0000 7f546b7fe640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547006be30 0x7f547006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.770+0000 7f546b7fe640 1 -- 172.21.15.113:0/4276294480 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f547808a2c0 con 0x7f54841f4f80 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.770+0000 7f5482ffd640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547006be30 0x7f547006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:14.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.770+0000 7f5482ffd640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547006be30 0x7f547006e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5474034000 tx=0x7f547402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:14.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.774+0000 7f546b7fe640 1 -- 172.21.15.113:0/4276294480 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5478059000 con 0x7f54841f4f80 2024-04-25T00:06:14.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.947+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f548406c740 con 0x7f54841f4f80 2024-04-25T00:06:14.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.951+0000 7f546b7fe640 1 -- 172.21.15.113:0/4276294480 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f5478058900 con 0x7f54841f4f80 2024-04-25T00:06:14.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547006be30 msgr2=0x7f547006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547006be30 0x7f547006e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5474034000 tx=0x7f547402e000 comp rx=0 tx=0).stop 2024-04-25T00:06:14.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54841f4f80 msgr2=0x7f54840ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:14.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54841f4f80 0x7f54840ae8b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5478001f00 tx=0x7f547800fa50 comp rx=0 tx=0).stop 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 shutdown_connections 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f547006be30 0x7f547006e2f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54840aedf0 0x7f54840c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54841f4f80 0x7f54840ae8b0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 --2- 172.21.15.113:0/4276294480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f546c0039b0 0x7f54841f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 >> 172.21.15.113:0/4276294480 conn(0x7f54840d0e30 msgr2=0x7f54840cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 shutdown_connections 2024-04-25T00:06:14.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:14.953+0000 7f54893e2640 1 -- 172.21.15.113:0/4276294480 wait complete. 2024-04-25T00:06:14.965 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:06:14.965 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:06:15.008 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:06:15.046 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:06:15.046 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:06:15.046 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:15.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:06:15.063 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:06:15.063 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:15.103 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:06:15.103 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:06:15.104 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:15.119 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:06:15.119 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:06:15.119 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:15.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:06:15.160 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:06:15.160 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:15.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.256+0000 7fcea3bb7640 1 Processor -- start 2024-04-25T00:06:15.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.257+0000 7fcea3bb7640 1 -- start start 2024-04-25T00:06:15.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.257+0000 7fcea3bb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c0b3fd0 0x7fce9c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.257+0000 7fcea3bb7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce9c06e510 0x7fce9c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.257+0000 7fcea3bb7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce9c144cf0 con 0x7fce9c144910 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.257+0000 7fcea3bb7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fce9c0b2f30 con 0x7fce9c06e510 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.257+0000 7fcea3bb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c0b3210 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea2154640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce9c06e510 0x7fce9c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea2154640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce9c06e510 0x7fce9c0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45964/0 (socket says 172.21.15.113:45964) 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea2154640 1 -- 172.21.15.113:0/2125519685 learned_addr learned my addr 172.21.15.113:0/2125519685 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea1152640 1 --2- 172.21.15.113:0/2125519685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c0b3fd0 0x7fce9c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea2154640 1 -- 172.21.15.113:0/2125519685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c0b3fd0 msgr2=0x7fce9c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea2154640 1 --2- 172.21.15.113:0/2125519685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c0b3fd0 0x7fce9c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea2154640 1 -- 172.21.15.113:0/2125519685 >> v1:172.21.15.178:6789/0 conn(0x7fce9c144910 legacy=0x7fce9c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.258+0000 7fcea2154640 1 -- 172.21.15.113:0/2125519685 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c0b34f0 con 0x7fce9c06e510 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea2154640 1 --2- 172.21.15.113:0/2125519685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c06e510 0x7fce9c0cd500 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fce98009e50 tx=0x7fce9802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7169a3b9d786467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea0951640 1 -- 172.21.15.113:0/2125519685 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98036070 con 0x7fce9c06e510 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea0951640 1 -- 172.21.15.113:0/2125519685 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce98031600 con 0x7fce9c06e510 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea0951640 1 -- 172.21.15.113:0/2125519685 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce980319d0 con 0x7fce9c06e510 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 -- 172.21.15.113:0/2125519685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c06e510 msgr2=0x7fce9c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/2125519685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c06e510 0x7fce9c0cd500 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fce98009e50 tx=0x7fce9802a710 comp rx=0 tx=0).stop 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 -- 172.21.15.113:0/2125519685 shutdown_connections 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/2125519685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c0b3fd0 0x7fce9c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/2125519685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c06e510 0x7fce9c0cd500 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 -- 172.21.15.113:0/2125519685 >> 172.21.15.113:0/2125519685 conn(0x7fce9c0b1a50 msgr2=0x7fce9c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 -- 172.21.15.113:0/2125519685 shutdown_connections 2024-04-25T00:06:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.259+0000 7fcea3bb7640 1 -- 172.21.15.113:0/2125519685 wait complete. 2024-04-25T00:06:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.260+0000 7fcea3bb7640 1 Processor -- start 2024-04-25T00:06:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea3bb7640 1 -- start start 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea3bb7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce9c06e510 0x7fce9c13f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea3bb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c0b3fd0 0x7fce9c142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea3bb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c144910 0x7fce9c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea3bb7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0b4da0 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea3bb7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0b4410 con 0x7fce9c06e510 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea3bb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c0b4f20 con 0x7fce9c144910 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea1152640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c0b3fd0 0x7fce9c142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea1152640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c0b3fd0 0x7fce9c142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45978/0 (socket says 172.21.15.113:45978) 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea1152640 1 -- 172.21.15.113:0/676375245 learned_addr learned my addr 172.21.15.113:0/676375245 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea2154640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c144910 0x7fce9c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.261+0000 7fcea1953640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce9c06e510 0x7fce9c13f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea1152640 1 -- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c144910 msgr2=0x7fce9c1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea1152640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c144910 0x7fce9c1431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea1152640 1 -- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce9c06e510 msgr2=0x7fce9c13f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea1152640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce9c06e510 0x7fce9c13f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea1152640 1 -- 172.21.15.113:0/676375245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c14ff60 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea1953640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce9c06e510 0x7fce9c13f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea1152640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c0b3fd0 0x7fce9c142cb0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007e90 tx=0x7fce8c00dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fce927fc640 1 -- 172.21.15.113:0/676375245 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00ba50 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fce927fc640 1 -- 172.21.15.113:0/676375245 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8c00bbf0 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c0ab730 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fce927fc640 1 -- 172.21.15.113:0/676375245 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00de20 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.262+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c0abcf0 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.263+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c0691a0 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.264+0000 7fce927fc640 1 -- 172.21.15.113:0/676375245 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce8c019020 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.264+0000 7fce927fc640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.264+0000 7fce927fc640 1 -- 172.21.15.113:0/676375245 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7fce8c08c9a0 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.264+0000 7fcea1953640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.264+0000 7fcea1953640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce8006be30 0x7fce8006e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fce94001660 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.268+0000 7fce927fc640 1 -- 172.21.15.113:0/676375245 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce8c05b000 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.478+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce9c0b59c0 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.479+0000 7fce927fc640 1 -- 172.21.15.113:0/676375245 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+3881 (secure 0 0 0) 0x7fce8c05f300 con 0x7fce9c0b3fd0 2024-04-25T00:06:15.480 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:15.480 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/2306642748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2306642748},{"type":"v1","addr":"172.21.15.178:6834","nonce":2306642748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6659,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3611794981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3611794981},{"type":"v1","addr":"172.21.15.178:6835","nonce":3611794981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":8385,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3353991823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3353991823},{"type":"v1","addr":"172.21.15.113:6835","nonce":3353991823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8388,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4028840746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":4028840746},{"type":"v1","addr":"172.21.15.113:6837","nonce":4028840746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-04-25T00:06:15.480 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 64 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce8006be30 msgr2=0x7fce8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce8006be30 0x7fce8006e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fce94001660 tx=0x7fce94009040 comp rx=0 tx=0).stop 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c0b3fd0 msgr2=0x7fce9c142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c0b3fd0 0x7fce9c142cb0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007e90 tx=0x7fce8c00dbd0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 shutdown_connections 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce9c144910 0x7fce9c1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9c0b3fd0 0x7fce9c142cb0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 --2- 172.21.15.113:0/676375245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce9c06e510 0x7fce9c13f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 >> 172.21.15.113:0/676375245 conn(0x7fce9c0b1a50 msgr2=0x7fce9c0cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:15.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 shutdown_connections 2024-04-25T00:06:15.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.481+0000 7fcea3bb7640 1 -- 172.21.15.113:0/676375245 wait complete. 2024-04-25T00:06:15.493 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:06:15.493 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:06:15.493 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:06:15.493 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:06:15.493 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:06:15.493 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:06:15.494 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:06:15.494 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:06:15.494 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:06:15.494 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:06:15.494 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:06:15.494 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:06:15.495 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:06:15.495 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:06:15.495 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:06:15.495 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:06:15.502 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:06:15.503 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:06:15.503 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:06:15.503 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:06:15.506 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:06:15.506 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:06:15.507 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:06:15.507 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:06:15.546 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:06:15.546 DEBUG:teuthology.parallel:result is None 2024-04-25T00:06:15.546 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:06:15.547 DEBUG:teuthology.parallel:result is None 2024-04-25T00:06:15.549 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:06:15.549 DEBUG:teuthology.parallel:result is None 2024-04-25T00:06:15.549 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:06:15.550 DEBUG:teuthology.parallel:result is None 2024-04-25T00:06:15.550 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:06:15.618 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:06:15.618 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:06:15.619 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:06:15.618+0000 7f41ec65f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-25T00:06:15.629 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:06:15.629 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:06:15.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.652+0000 7f9c94cd4640 1 Processor -- start 2024-04-25T00:06:15.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.653+0000 7f9c94cd4640 1 -- start start 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.653+0000 7f9c94cd4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c900c6b80 0x7f9c9006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.653+0000 7f9c94cd4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c900c6f60 con 0x7f9c901602f0 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.653+0000 7f9c94cd4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9015caf0 con 0x7f9c9015c640 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.653+0000 7f9c94cd4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9015cdb0 con 0x7f9c900c6b80 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.653+0000 7f9c8e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c900c6b80 0x7f9c9006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.654+0000 7f9c8ed9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9c901602f0 0x7f9c900cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51520/0 (socket says 172.21.15.113:51520) 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.654+0000 7f9c8ed9d640 1 -- 172.21.15.113:0/3352718160 learned_addr learned my addr 172.21.15.113:0/3352718160 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:15.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.654+0000 7f9c8e59c640 1 -- 172.21.15.113:0/3352718160 >> v1:172.21.15.178:6789/0 conn(0x7f9c9015c640 legacy=0x7f9c900c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.654+0000 7f9c8e59c640 1 -- 172.21.15.113:0/3352718160 >> v1:172.21.15.113:6789/0 conn(0x7f9c901602f0 legacy=0x7f9c900cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.654+0000 7f9c8e59c640 1 -- 172.21.15.113:0/3352718160 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9015d090 con 0x7f9c900c6b80 2024-04-25T00:06:15.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.655+0000 7f9c8e59c640 1 --2- 172.21.15.113:0/3352718160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c900c6b80 0x7f9c9006e5e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9c800097a0 tx=0x7f9c8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d62326580a629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.655+0000 7f9c8dd9b640 1 -- 172.21.15.113:0/3352718160 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80037070 con 0x7f9c900c6b80 2024-04-25T00:06:15.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.655+0000 7f9c8dd9b640 1 -- 172.21.15.113:0/3352718160 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c80032440 con 0x7f9c900c6b80 2024-04-25T00:06:15.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.655+0000 7f9c8dd9b640 1 -- 172.21.15.113:0/3352718160 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80032810 con 0x7f9c900c6b80 2024-04-25T00:06:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.656+0000 7f9c94cd4640 1 -- 172.21.15.113:0/3352718160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c900c6b80 msgr2=0x7f9c9006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.656+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/3352718160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c900c6b80 0x7f9c9006e5e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9c800097a0 tx=0x7f9c8002a230 comp rx=0 tx=0).stop 2024-04-25T00:06:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.656+0000 7f9c94cd4640 1 -- 172.21.15.113:0/3352718160 shutdown_connections 2024-04-25T00:06:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.656+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/3352718160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c900c6b80 0x7f9c9006e5e0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.656+0000 7f9c94cd4640 1 -- 172.21.15.113:0/3352718160 >> 172.21.15.113:0/3352718160 conn(0x7f9c9006b0f0 msgr2=0x7f9c9006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:15.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.656+0000 7f9c94cd4640 1 -- 172.21.15.113:0/3352718160 shutdown_connections 2024-04-25T00:06:15.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.656+0000 7f9c94cd4640 1 -- 172.21.15.113:0/3352718160 wait complete. 2024-04-25T00:06:15.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.658+0000 7f9c94cd4640 1 Processor -- start 2024-04-25T00:06:15.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.658+0000 7f9c94cd4640 1 -- start start 2024-04-25T00:06:15.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c94cd4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c900c6b80 0x7f9c90143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c94cd4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9015c640 0x7f9c901428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c94cd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c901602f0 0x7f9c9013f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c94cd4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c900c5da0 con 0x7f9c901602f0 2024-04-25T00:06:15.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c94cd4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9015fdd0 con 0x7f9c900c6b80 2024-04-25T00:06:15.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c94cd4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c900c5f20 con 0x7f9c9015c640 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c901602f0 0x7f9c9013f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c901602f0 0x7f9c9013f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46018/0 (socket says 172.21.15.113:46018) 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 -- 172.21.15.113:0/915373092 learned_addr learned my addr 172.21.15.113:0/915373092 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c87fff640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9015c640 0x7f9c901428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8e59c640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c900c6b80 0x7f9c90143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 -- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9015c640 msgr2=0x7f9c901428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9015c640 0x7f9c901428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 -- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c900c6b80 msgr2=0x7f9c90143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c900c6b80 0x7f9c90143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.659+0000 7f9c8ed9d640 1 -- 172.21.15.113:0/915373092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c90140680 con 0x7f9c901602f0 2024-04-25T00:06:15.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.660+0000 7f9c8ed9d640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c901602f0 0x7f9c9013f920 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800ca80 tx=0x7f9c8800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.662+0000 7f9c86ffd640 1 -- 172.21.15.113:0/915373092 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017070 con 0x7f9c901602f0 2024-04-25T00:06:15.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.662+0000 7f9c86ffd640 1 -- 172.21.15.113:0/915373092 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8800ac30 con 0x7f9c901602f0 2024-04-25T00:06:15.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.662+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c90140960 con 0x7f9c901602f0 2024-04-25T00:06:15.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.662+0000 7f9c86ffd640 1 -- 172.21.15.113:0/915373092 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88012600 con 0x7f9c901602f0 2024-04-25T00:06:15.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.662+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c90140f20 con 0x7f9c901602f0 2024-04-25T00:06:15.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.663+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c90068220 con 0x7f9c901602f0 2024-04-25T00:06:15.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.664+0000 7f9c86ffd640 1 -- 172.21.15.113:0/915373092 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c8801a070 con 0x7f9c901602f0 2024-04-25T00:06:15.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.664+0000 7f9c86ffd640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7006be30 0x7f9c7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.664+0000 7f9c86ffd640 1 -- 172.21.15.113:0/915373092 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4943+0+0 (secure 0 0 0) 0x7f9c8808b4e0 con 0x7f9c901602f0 2024-04-25T00:06:15.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.664+0000 7f9c8e59c640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7006be30 0x7f9c7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.665+0000 7f9c8e59c640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7006be30 0x7f9c7006e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c80034000 tx=0x7f9c8002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.669+0000 7f9c86ffd640 1 -- 172.21.15.113:0/915373092 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c8800ae20 con 0x7f9c901602f0 2024-04-25T00:06:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:15.855+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9c900c5b80 con 0x7f9c901602f0 2024-04-25T00:06:16.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.664+0000 7f9c86ffd640 1 -- 172.21.15.113:0/915373092 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) ==== 97+0+0 (secure 0 0 0) 0x7f9c88058e10 con 0x7f9c901602f0 2024-04-25T00:06:16.665 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:06:16.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.667+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7006be30 msgr2=0x7f9c7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:16.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.667+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7006be30 0x7f9c7006e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c80034000 tx=0x7f9c8002e000 comp rx=0 tx=0).stop 2024-04-25T00:06:16.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.667+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c901602f0 msgr2=0x7f9c9013f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:16.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.667+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c901602f0 0x7f9c9013f920 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800ca80 tx=0x7f9c8800cf40 comp rx=0 tx=0).stop 2024-04-25T00:06:16.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 shutdown_connections 2024-04-25T00:06:16.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9c7006be30 0x7f9c7006e2f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:16.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c901602f0 0x7f9c9013f920 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:16.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c9015c640 0x7f9c901428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:16.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 --2- 172.21.15.113:0/915373092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c900c6b80 0x7f9c90143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:16.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 >> 172.21.15.113:0/915373092 conn(0x7f9c9006b0f0 msgr2=0x7f9c900ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:16.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 shutdown_connections 2024-04-25T00:06:16.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.668+0000 7f9c94cd4640 1 -- 172.21.15.113:0/915373092 wait complete. 2024-04-25T00:06:16.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:06:16.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.790+0000 7fdc84d70640 1 Processor -- start 2024-04-25T00:06:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.790+0000 7fdc84d70640 1 -- start start 2024-04-25T00:06:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc84d70640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc8006e6e0 0x7fdc800b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc84d70640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc800cfad0 con 0x7fdc801453c0 2024-04-25T00:06:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc84d70640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc800cf230 con 0x7fdc80144d60 2024-04-25T00:06:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc84d70640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdc80145c40 con 0x7fdc8006e6e0 2024-04-25T00:06:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc7ed9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdc801453c0 0x7fdc80071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51534/0 (socket says 172.21.15.113:51534) 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc7dd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc8006e6e0 0x7fdc800b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc7ed9d640 1 -- 172.21.15.113:0/261466709 learned_addr learned my addr 172.21.15.113:0/261466709 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc7dd9b640 1 -- 172.21.15.113:0/261466709 >> v1:172.21.15.178:6789/0 conn(0x7fdc80144d60 legacy=0x7fdc8006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc7dd9b640 1 -- 172.21.15.113:0/261466709 >> v1:172.21.15.113:6789/0 conn(0x7fdc801453c0 legacy=0x7fdc80071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc7d59a640 1 -- 172.21.15.113:0/261466709 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266355063 0 0) 0x7fdc800cfad0 con 0x7fdc801453c0 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.791+0000 7fdc7dd9b640 1 -- 172.21.15.113:0/261466709 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc800cf510 con 0x7fdc8006e6e0 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.792+0000 7fdc7dd9b640 1 --2- 172.21.15.113:0/261466709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc8006e6e0 0x7fdc800b5390 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fdc740097a0 tx=0x7fdc7402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1385d53320fc9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.792+0000 7fdc7d59a640 1 -- 172.21.15.113:0/261466709 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74037070 con 0x7fdc8006e6e0 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.792+0000 7fdc7d59a640 1 -- 172.21.15.113:0/261466709 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc7402adc0 con 0x7fdc8006e6e0 2024-04-25T00:06:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.792+0000 7fdc7d59a640 1 -- 172.21.15.113:0/261466709 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74032660 con 0x7fdc8006e6e0 2024-04-25T00:06:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.793+0000 7fdc84d70640 1 -- 172.21.15.113:0/261466709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc8006e6e0 msgr2=0x7fdc800b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.793+0000 7fdc84d70640 1 --2- 172.21.15.113:0/261466709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc8006e6e0 0x7fdc800b5390 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fdc740097a0 tx=0x7fdc7402a0b0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.793+0000 7fdc84d70640 1 -- 172.21.15.113:0/261466709 shutdown_connections 2024-04-25T00:06:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.793+0000 7fdc84d70640 1 --2- 172.21.15.113:0/261466709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc8006e6e0 0x7fdc800b5390 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.793+0000 7fdc84d70640 1 -- 172.21.15.113:0/261466709 >> 172.21.15.113:0/261466709 conn(0x7fdc800b1a50 msgr2=0x7fdc800b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.793+0000 7fdc84d70640 1 -- 172.21.15.113:0/261466709 shutdown_connections 2024-04-25T00:06:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.793+0000 7fdc84d70640 1 -- 172.21.15.113:0/261466709 wait complete. 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.794+0000 7fdc84d70640 1 Processor -- start 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.794+0000 7fdc84d70640 1 -- start start 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc84d70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc8006e6e0 0x7fdc8015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc84d70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc80144d60 0x7fdc801433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc84d70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc801453c0 0x7fdc801438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc84d70640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80145140 con 0x7fdc801453c0 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc84d70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80070e10 con 0x7fdc8006e6e0 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc84d70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdc80071a40 con 0x7fdc80144d60 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc801453c0 0x7fdc801438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc801453c0 0x7fdc801438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46036/0 (socket says 172.21.15.113:46036) 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 -- 172.21.15.113:0/684500726 learned_addr learned my addr 172.21.15.113:0/684500726 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7e59c640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc8006e6e0 0x7fdc8015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 -- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc80144d60 msgr2=0x7fdc801433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7dd9b640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc80144d60 0x7fdc801433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc80144d60 0x7fdc801433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 -- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc8006e6e0 msgr2=0x7fdc8015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc8006e6e0 0x7fdc8015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7ed9d640 1 -- 172.21.15.113:0/684500726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc801f8a10 con 0x7fdc801453c0 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7e59c640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc8006e6e0 0x7fdc8015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.795+0000 7fdc7dd9b640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc80144d60 0x7fdc801433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.796+0000 7fdc7ed9d640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc801453c0 0x7fdc801438e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800c410 tx=0x7fdc7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.796+0000 7fdc6effd640 1 -- 172.21.15.113:0/684500726 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780074e0 con 0x7fdc801453c0 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.796+0000 7fdc6effd640 1 -- 172.21.15.113:0/684500726 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc780047e0 con 0x7fdc801453c0 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.796+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8015d4f0 con 0x7fdc801453c0 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.796+0000 7fdc6effd640 1 -- 172.21.15.113:0/684500726 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78004bd0 con 0x7fdc801453c0 2024-04-25T00:06:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.796+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8015dab0 con 0x7fdc801453c0 2024-04-25T00:06:17.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.797+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc800691a0 con 0x7fdc801453c0 2024-04-25T00:06:17.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.797+0000 7fdc6effd640 1 -- 172.21.15.113:0/684500726 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc78011070 con 0x7fdc801453c0 2024-04-25T00:06:17.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.798+0000 7fdc6effd640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdc6006be30 0x7fdc6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.798+0000 7fdc6effd640 1 -- 172.21.15.113:0/684500726 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4857+0+0 (secure 0 0 0) 0x7fdc7808a580 con 0x7fdc801453c0 2024-04-25T00:06:17.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.798+0000 7fdc7e59c640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdc6006be30 0x7fdc6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.798+0000 7fdc7e59c640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdc6006be30 0x7fdc6006e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdc70004520 tx=0x7fdc70009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:16.802+0000 7fdc6effd640 1 -- 172.21.15.113:0/684500726 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc78059310 con 0x7fdc801453c0 2024-04-25T00:06:17.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.043+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdc800b27e0 con 0x7fdc801453c0 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.044+0000 7fdc6effd640 1 -- 172.21.15.113:0/684500726 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fdc78058c10 con 0x7fdc801453c0 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:06:17.044 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.045 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.049 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.050 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.051 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.052 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.053 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:06:17.054 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.055 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:06:17.056 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:06:17.057 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:06:17.057 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:06:17.057 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:06:17.057 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.057 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.057 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:06:17.057 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:06:17.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.045+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdc6006be30 msgr2=0x7fdc6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.045+0000 7fdc84d70640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdc6006be30 0x7fdc6006e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdc70004520 tx=0x7fdc70009040 comp rx=0 tx=0).stop 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc801453c0 msgr2=0x7fdc801438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc801453c0 0x7fdc801438e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800c410 tx=0x7fdc7800c8d0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 shutdown_connections 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdc6006be30 0x7fdc6006e2f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc801453c0 0x7fdc801438e0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc80144d60 0x7fdc801433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 --2- 172.21.15.113:0/684500726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc8006e6e0 0x7fdc8015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 >> 172.21.15.113:0/684500726 conn(0x7fdc800b1a50 msgr2=0x7fdc800ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 shutdown_connections 2024-04-25T00:06:17.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.046+0000 7fdc84d70640 1 -- 172.21.15.113:0/684500726 wait complete. 2024-04-25T00:06:17.060 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:17.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.156+0000 7fea505a2640 1 Processor -- start 2024-04-25T00:06:17.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.156+0000 7fea505a2640 1 -- start start 2024-04-25T00:06:17.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea505a2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea48144910 0x7fea480b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea505a2640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea480b5240 0x7fea4806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea505a2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea48144cf0 con 0x7fea480b5680 2024-04-25T00:06:17.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea505a2640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fea480b4510 con 0x7fea480b5240 2024-04-25T00:06:17.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea505a2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea480b2f30 con 0x7fea48144910 2024-04-25T00:06:17.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4db3d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea480b5240 0x7fea4806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4e33e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea48144910 0x7fea480b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4db3d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea480b5240 0x7fea4806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:56720/0 (socket says 172.21.15.113:56720) 2024-04-25T00:06:17.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4db3d640 1 -- 172.21.15.113:0/3789260712 learned_addr learned my addr 172.21.15.113:0/3789260712 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:17.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4eb3f640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fea480b5680 0x7fea480cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:48908/0 (socket says 172.21.15.113:48908) 2024-04-25T00:06:17.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4db3d640 1 -- 172.21.15.113:0/3789260712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea48144910 msgr2=0x7fea480b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4db3d640 1 --2- 172.21.15.113:0/3789260712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea48144910 0x7fea480b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4db3d640 1 -- 172.21.15.113:0/3789260712 >> v1:172.21.15.178:6789/0 conn(0x7fea480b5680 legacy=0x7fea480cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.157+0000 7fea4db3d640 1 -- 172.21.15.113:0/3789260712 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea480b3210 con 0x7fea480b5240 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.158+0000 7fea4db3d640 1 --2- 172.21.15.113:0/3789260712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea480b5240 0x7fea4806dfd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea3802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7ab445fbbdcf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.158+0000 7fea4d33c640 1 -- 172.21.15.113:0/3789260712 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38037070 con 0x7fea480b5240 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.158+0000 7fea4d33c640 1 -- 172.21.15.113:0/3789260712 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38032440 con 0x7fea480b5240 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.158+0000 7fea4d33c640 1 -- 172.21.15.113:0/3789260712 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38032810 con 0x7fea480b5240 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 -- 172.21.15.113:0/3789260712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea480b5240 msgr2=0x7fea4806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 --2- 172.21.15.113:0/3789260712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea480b5240 0x7fea4806dfd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea3802a270 comp rx=0 tx=0).stop 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 -- 172.21.15.113:0/3789260712 shutdown_connections 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 --2- 172.21.15.113:0/3789260712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea480b5240 0x7fea4806dfd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 --2- 172.21.15.113:0/3789260712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea48144910 0x7fea480b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 -- 172.21.15.113:0/3789260712 >> 172.21.15.113:0/3789260712 conn(0x7fea480b1a50 msgr2=0x7fea480b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 -- 172.21.15.113:0/3789260712 shutdown_connections 2024-04-25T00:06:17.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.159+0000 7fea505a2640 1 -- 172.21.15.113:0/3789260712 wait complete. 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.160+0000 7fea505a2640 1 Processor -- start 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.160+0000 7fea505a2640 1 -- start start 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea505a2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea480b5240 0x7fea48142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea505a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea480b5680 0x7fea481e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea505a2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea48144910 0x7fea481e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea505a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea480b4da0 con 0x7fea480b5680 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea505a2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea480b37d0 con 0x7fea48144910 2024-04-25T00:06:17.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea505a2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea480b4f20 con 0x7fea480b5240 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea480b5680 0x7fea481e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea480b5680 0x7fea481e84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46040/0 (socket says 172.21.15.113:46040) 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 -- 172.21.15.113:0/1798863680 learned_addr learned my addr 172.21.15.113:0/1798863680 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4eb3f640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea48144910 0x7fea481e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4e33e640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea480b5240 0x7fea48142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 -- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea480b5240 msgr2=0x7fea48142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea480b5240 0x7fea48142ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 -- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea48144910 msgr2=0x7fea481e8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea48144910 0x7fea481e8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 -- 172.21.15.113:0/1798863680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea481f8820 con 0x7fea480b5680 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4eb3f640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea48144910 0x7fea481e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4e33e640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea480b5240 0x7fea48142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:17.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea4db3d640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea480b5680 0x7fea481e84f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.161+0000 7fea3effd640 1 -- 172.21.15.113:0/1798863680 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38037040 con 0x7fea480b5680 2024-04-25T00:06:17.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.162+0000 7fea3effd640 1 -- 172.21.15.113:0/1798863680 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38042070 con 0x7fea480b5680 2024-04-25T00:06:17.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.162+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea481e5c30 con 0x7fea480b5680 2024-04-25T00:06:17.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.162+0000 7fea3effd640 1 -- 172.21.15.113:0/1798863680 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38030880 con 0x7fea480b5680 2024-04-25T00:06:17.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.162+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea481e61f0 con 0x7fea480b5680 2024-04-25T00:06:17.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.163+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea480691a0 con 0x7fea480b5680 2024-04-25T00:06:17.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.163+0000 7fea3effd640 1 -- 172.21.15.113:0/1798863680 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea38030a20 con 0x7fea480b5680 2024-04-25T00:06:17.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.163+0000 7fea3effd640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea2c06be30 0x7fea2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.164+0000 7fea3effd640 1 -- 172.21.15.113:0/1798863680 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4857+0+0 (secure 0 0 0) 0x7fea380ab2c0 con 0x7fea480b5680 2024-04-25T00:06:17.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.164+0000 7fea4e33e640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea2c06be30 0x7fea2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.164+0000 7fea4e33e640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea2c06be30 0x7fea2c06e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fea44009ac0 tx=0x7fea44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.168+0000 7fea3effd640 1 -- 172.21.15.113:0/1798863680 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fea3807a050 con 0x7fea480b5680 2024-04-25T00:06:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.342+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea480b2080 con 0x7fea480b5680 2024-04-25T00:06:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.343+0000 7fea3effd640 1 -- 172.21.15.113:0/1798863680 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+12949 (secure 0 0 0) 0x7fea38079950 con 0x7fea480b5680 2024-04-25T00:06:17.344 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:17.344 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":126,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:16.663629+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea2c06be30 msgr2=0x7fea2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea2c06be30 0x7fea2c06e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fea44009ac0 tx=0x7fea44008040 comp rx=0 tx=0).stop 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea480b5680 msgr2=0x7fea481e84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea480b5680 0x7fea481e84f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea380046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 shutdown_connections 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea2c06be30 0x7fea2c06e2f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea48144910 0x7fea481e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea480b5680 0x7fea481e84f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 --2- 172.21.15.113:0/1798863680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea480b5240 0x7fea48142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 >> 172.21.15.113:0/1798863680 conn(0x7fea480b1a50 msgr2=0x7fea480cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 shutdown_connections 2024-04-25T00:06:17.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.345+0000 7fea505a2640 1 -- 172.21.15.113:0/1798863680 wait complete. 2024-04-25T00:06:17.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:06:17.357 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:06:17.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.456+0000 7f57a67ee640 1 Processor -- start 2024-04-25T00:06:17.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.456+0000 7f57a67ee640 1 -- start start 2024-04-25T00:06:17.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f57a67ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 0x7f57a00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f57a67ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a015f4f0 con 0x7f57a015e2d0 2024-04-25T00:06:17.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f57a67ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a015f7d0 con 0x7f57a015e6b0 2024-04-25T00:06:17.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f57a67ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57a00cde00 con 0x7f57a00b3080 2024-04-25T00:06:17.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f579ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f57a015e2d0 0x7f57a015ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51546/0 (socket says 172.21.15.113:51546) 2024-04-25T00:06:17.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f579ffff640 1 -- 172.21.15.113:0/1367902496 learned_addr learned my addr 172.21.15.113:0/1367902496 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3369579226 0 0) 0x7f57a015f4f0 con 0x7f57a015e2d0 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.457+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5788003520 con 0x7f57a015e2d0 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 0x7f57a00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4096598009 0 0) 0x7f5788003520 con 0x7f57a015e2d0 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 msgr2=0x7f57a00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 0x7f57a00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 >> v1:172.21.15.178:6789/0 conn(0x7f57a015e6b0 legacy=0x7f57a00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a015fab0 con 0x7f57a015e2d0 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f5790001670 con 0x7f57a015e2d0 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 0x7f57a00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 >> v1:172.21.15.113:6789/0 conn(0x7f57a015e2d0 legacy=0x7f57a015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57880039e0 0x7f5788003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5788004800 0x7f5788004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 0x7f578800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f57a00cde00 con 0x7f5788004800 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57880054e0 con 0x7f57880039e0 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f578800e110 con 0x7f57a00b3080 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5788004800 0x7f5788004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579ffff640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57880039e0 0x7f5788003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f57a4d8b640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 0x7f578800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 -- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 msgr2=0x7f578800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57a00b3080 0x7f578800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 -- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57880039e0 msgr2=0x7f5788003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57880039e0 0x7f5788003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.458+0000 7f579f7fe640 1 -- 172.21.15.113:0/1367902496 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5788003520 con 0x7f5788004800 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.459+0000 7f579f7fe640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5788004800 0x7f5788004c80 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b34c0 tx=0x7f5788005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55245b4fb73db30f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.459+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400f040 con 0x7f5788004800 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.459+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579402ac90 con 0x7f5788004800 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.459+0000 7f579effd640 1 -- 172.21.15.113:0/1367902496 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794004250 con 0x7f5788004800 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.459+0000 7f57a67ee640 1 -- 172.21.15.113:0/1367902496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5788004800 msgr2=0x7f5788004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.460+0000 7f57a67ee640 1 --2- 172.21.15.113:0/1367902496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5788004800 0x7f5788004c80 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b34c0 tx=0x7f5788005830 comp rx=0 tx=0).stop 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.460+0000 7f57a4d8b640 1 -- 172.21.15.113:0/1367902496 reap_dead start 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.460+0000 7f57a67ee640 1 -- 172.21.15.113:0/1367902496 shutdown_connections 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.460+0000 7f57a67ee640 1 -- 172.21.15.113:0/1367902496 >> 172.21.15.113:0/1367902496 conn(0x7f57a006b0f0 msgr2=0x7f57a006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.460+0000 7f57a67ee640 1 -- 172.21.15.113:0/1367902496 shutdown_connections 2024-04-25T00:06:17.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.460+0000 7f57a67ee640 1 -- 172.21.15.113:0/1367902496 wait complete. 2024-04-25T00:06:17.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.461+0000 7f57a67ee640 1 Processor -- start 2024-04-25T00:06:17.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.461+0000 7f57a67ee640 1 -- start start 2024-04-25T00:06:17.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5788004800 0x7f57a014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01512f0 0x7f57a014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0151770 0x7f57a014f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f57a00b44b0 con 0x7f57a01512f0 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57a00b4330 con 0x7f57a0151770 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57a00b4630 con 0x7f5788004800 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01512f0 0x7f57a014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01512f0 0x7f57a014f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46048/0 (socket says 172.21.15.113:46048) 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 -- 172.21.15.113:0/3426246378 learned_addr learned my addr 172.21.15.113:0/3426246378 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579ffff640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5788004800 0x7f57a014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a4d8b640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0151770 0x7f57a014f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 -- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5788004800 msgr2=0x7f57a014daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5788004800 0x7f57a014daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 -- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0151770 msgr2=0x7f57a014f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0151770 0x7f57a014f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 -- 172.21.15.113:0/3426246378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a014edf0 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579f7fe640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01512f0 0x7f57a014f420 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5794009960 tx=0x7f5794004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579cff9640 1 -- 172.21.15.113:0/3426246378 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794041070 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579cff9640 1 -- 172.21.15.113:0/3426246378 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579400f000 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a0153120 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f579cff9640 1 -- 172.21.15.113:0/3426246378 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794003c80 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.462+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a01536e0 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.464+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a00691a0 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.464+0000 7f579cff9640 1 -- 172.21.15.113:0/3426246378 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5794044070 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.464+0000 7f579cff9640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.464+0000 7f579cff9640 1 -- 172.21.15.113:0/3426246378 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4857+0+0 (secure 0 0 0) 0x7f57940ab1f0 con 0x7f57a01512f0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.464+0000 7f579ffff640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.465+0000 7f579ffff640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578c06be30 0x7f578c06e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5790001050 tx=0x7f5790000f80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.468+0000 7f579cff9640 1 -- 172.21.15.113:0/3426246378 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f57940798a0 con 0x7f57a01512f0 2024-04-25T00:06:17.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.641+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57a0072050 con 0x7f57a01512f0 2024-04-25T00:06:17.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.642+0000 7f579cff9640 1 -- 172.21.15.113:0/3426246378 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f579407dba0 con 0x7f57a01512f0 2024-04-25T00:06:17.642 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:06:17.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578c06be30 msgr2=0x7f578c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578c06be30 0x7f578c06e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5790001050 tx=0x7f5790000f80 comp rx=0 tx=0).stop 2024-04-25T00:06:17.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01512f0 msgr2=0x7f57a014f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01512f0 0x7f57a014f420 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5794009960 tx=0x7f5794004730 comp rx=0 tx=0).stop 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 shutdown_connections 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0151770 0x7f57a014f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01512f0 0x7f57a014f420 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 --2- 172.21.15.113:0/3426246378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5788004800 0x7f57a014daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 >> 172.21.15.113:0/3426246378 conn(0x7f57a006b0f0 msgr2=0x7f57a00afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 shutdown_connections 2024-04-25T00:06:17.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.644+0000 7f57a67ee640 1 -- 172.21.15.113:0/3426246378 wait complete. 2024-04-25T00:06:17.656 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:06:17.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.752+0000 7fcaf6627640 1 Processor -- start 2024-04-25T00:06:17.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.752+0000 7fcaf6627640 1 -- start start 2024-04-25T00:06:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaf6627640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcaf0144910 0x7fcaf00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaf6627640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcaf00b5240 0x7fcaf006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaf6627640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b56b0 0x7fcaf006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaf6627640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf00b4510 con 0x7fcaf0144910 2024-04-25T00:06:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaf6627640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf0144cf0 con 0x7fcaf00b5240 2024-04-25T00:06:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaf6627640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00b2f30 con 0x7fcaf00b56b0 2024-04-25T00:06:17.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcaf0144910 0x7fcaf00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcaf0144910 0x7fcaf00b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55826/0 (socket says 172.21.15.113:55826) 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 -- 172.21.15.113:0/4191222554 learned_addr learned my addr 172.21.15.113:0/4191222554 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 -- 172.21.15.113:0/4191222554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b56b0 msgr2=0x7fcaf006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 --2- 172.21.15.113:0/4191222554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b56b0 0x7fcaf006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 -- 172.21.15.113:0/4191222554 >> v2:172.21.15.178:3300/0 conn(0x7fcaf00b5240 msgr2=0x7fcaf006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 --2- 172.21.15.113:0/4191222554 >> v2:172.21.15.178:3300/0 conn(0x7fcaf00b5240 0x7fcaf006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.753+0000 7fcaeffff640 1 -- 172.21.15.113:0/4191222554 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf00b3210 con 0x7fcaf0144910 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaeffff640 1 --2- 172.21.15.113:0/4191222554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf0144910 0x7fcaf00b3fd0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009870 tx=0x7fcae002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30f928e3a831426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaeeffd640 1 -- 172.21.15.113:0/4191222554 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0037070 con 0x7fcaf0144910 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaeeffd640 1 -- 172.21.15.113:0/4191222554 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae0032440 con 0x7fcaf0144910 2024-04-25T00:06:17.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaeeffd640 1 -- 172.21.15.113:0/4191222554 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0032810 con 0x7fcaf0144910 2024-04-25T00:06:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 -- 172.21.15.113:0/4191222554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf0144910 msgr2=0x7fcaf00b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 --2- 172.21.15.113:0/4191222554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf0144910 0x7fcaf00b3fd0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009870 tx=0x7fcae002a210 comp rx=0 tx=0).stop 2024-04-25T00:06:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 -- 172.21.15.113:0/4191222554 shutdown_connections 2024-04-25T00:06:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 --2- 172.21.15.113:0/4191222554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b56b0 0x7fcaf006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 --2- 172.21.15.113:0/4191222554 >> v2:172.21.15.178:3300/0 conn(0x7fcaf00b5240 0x7fcaf006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 --2- 172.21.15.113:0/4191222554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf0144910 0x7fcaf00b3fd0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 -- 172.21.15.113:0/4191222554 >> 172.21.15.113:0/4191222554 conn(0x7fcaf00b1a50 msgr2=0x7fcaf00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 -- 172.21.15.113:0/4191222554 shutdown_connections 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.754+0000 7fcaf6627640 1 -- 172.21.15.113:0/4191222554 wait complete. 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.755+0000 7fcaf6627640 1 Processor -- start 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaf6627640 1 -- start start 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaf6627640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b5240 0x7fcaf01e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaf6627640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf00b56b0 0x7fcaf015d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaf6627640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0144910 0x7fcaf00aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaf6627640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf00b4da0 con 0x7fcaf00b56b0 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaf6627640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf00b37d0 con 0x7fcaf0144910 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaf6627640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00b4f20 con 0x7fcaf00b5240 2024-04-25T00:06:17.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf00b56b0 0x7fcaf015d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf00b56b0 0x7fcaf015d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55834/0 (socket says 172.21.15.113:55834) 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.756+0000 7fcaef7fe640 1 -- 172.21.15.113:0/2087741125 learned_addr learned my addr 172.21.15.113:0/2087741125 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaeffff640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b5240 0x7fcaf01e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaef7fe640 1 -- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b5240 msgr2=0x7fcaf01e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaef7fe640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b5240 0x7fcaf01e6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaf4bc4640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0144910 0x7fcaf00aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaef7fe640 1 -- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0144910 msgr2=0x7fcaf00aa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaef7fe640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0144910 0x7fcaf00aa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaef7fe640 1 -- 172.21.15.113:0/2087741125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf01e59c0 con 0x7fcaf00b56b0 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaeffff640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b5240 0x7fcaf01e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaf4bc4640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0144910 0x7fcaf00aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:17.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaef7fe640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf00b56b0 0x7fcaf015d500 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b440 tx=0x7fcae400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaecff9640 1 -- 172.21.15.113:0/2087741125 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4015020 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaecff9640 1 -- 172.21.15.113:0/2087741125 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae4011890 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf01e5ca0 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaecff9640 1 -- 172.21.15.113:0/2087741125 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4011c80 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.757+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf01e6260 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.758+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf00691a0 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.759+0000 7fcaecff9640 1 -- 172.21.15.113:0/2087741125 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcae4018020 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.759+0000 7fcaecff9640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcadc06be30 0x7fcadc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.759+0000 7fcaecff9640 1 -- 172.21.15.113:0/2087741125 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4857+0+0 (secure 0 0 0) 0x7fcae408ac70 con 0x7fcaf00b56b0 2024-04-25T00:06:17.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.759+0000 7fcaeffff640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcadc06be30 0x7fcadc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.759+0000 7fcaeffff640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcadc06be30 0x7fcadc06e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcae0034000 tx=0x7fcae002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.763+0000 7fcaecff9640 1 -- 172.21.15.113:0/2087741125 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcae401f020 con 0x7fcaf00b56b0 2024-04-25T00:06:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.937+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcaf00731a0 con 0x7fcaf00b56b0 2024-04-25T00:06:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.937+0000 7fcaecff9640 1 -- 172.21.15.113:0/2087741125 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v126) ==== 84+0+65 (secure 0 0 0) 0x7fcae400d040 con 0x7fcaf00b56b0 2024-04-25T00:06:17.939 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:17.939 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:06:17.939 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:06:17.939 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:06:17.939 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:06:17.939 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:17.939 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:06:17.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcadc06be30 msgr2=0x7fcadc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcadc06be30 0x7fcadc06e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcae0034000 tx=0x7fcae002e000 comp rx=0 tx=0).stop 2024-04-25T00:06:17.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf00b56b0 msgr2=0x7fcaf015d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf00b56b0 0x7fcaf015d500 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b440 tx=0x7fcae400b900 comp rx=0 tx=0).stop 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 shutdown_connections 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcadc06be30 0x7fcadc06e2f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0144910 0x7fcaf00aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf00b56b0 0x7fcaf015d500 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 --2- 172.21.15.113:0/2087741125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf00b5240 0x7fcaf01e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 >> 172.21.15.113:0/2087741125 conn(0x7fcaf00b1a50 msgr2=0x7fcaf00af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 shutdown_connections 2024-04-25T00:06:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:17.939+0000 7fcaf6627640 1 -- 172.21.15.113:0/2087741125 wait complete. 2024-04-25T00:06:17.951 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:06:17.951 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:06:18.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.051+0000 7f58ba6bd640 1 Processor -- start 2024-04-25T00:06:18.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.052+0000 7f58ba6bd640 1 -- start start 2024-04-25T00:06:18.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.052+0000 7f58ba6bd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58b4144a80 0x7f58b40b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.052+0000 7f58ba6bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b40b3ff0 0x7f58b406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.052+0000 7f58ba6bd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f58b406e610 0x7f58b40cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.052+0000 7f58ba6bd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f58b4072f70 con 0x7f58b4144a80 2024-04-25T00:06:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.052+0000 7f58ba6bd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f58b40b2fd0 con 0x7f58b406e610 2024-04-25T00:06:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.052+0000 7f58ba6bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58b40b32b0 con 0x7f58b40b3ff0 2024-04-25T00:06:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58b4144a80 0x7f58b40b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58b4144a80 0x7f58b40b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55844/0 (socket says 172.21.15.113:55844) 2024-04-25T00:06:18.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 -- 172.21.15.113:0/2810615550 learned_addr learned my addr 172.21.15.113:0/2810615550 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:18.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b8c5a640 1 --2- 172.21.15.113:0/2810615550 >> v2:172.21.15.178:3300/0 conn(0x7f58b406e610 0x7f58b40cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b37fe640 1 --2- 172.21.15.113:0/2810615550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b40b3ff0 0x7f58b406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 -- 172.21.15.113:0/2810615550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b40b3ff0 msgr2=0x7f58b406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:18.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 --2- 172.21.15.113:0/2810615550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b40b3ff0 0x7f58b406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 -- 172.21.15.113:0/2810615550 >> v2:172.21.15.178:3300/0 conn(0x7f58b406e610 msgr2=0x7f58b40cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:18.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 --2- 172.21.15.113:0/2810615550 >> v2:172.21.15.178:3300/0 conn(0x7f58b406e610 0x7f58b40cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.053+0000 7f58b3fff640 1 -- 172.21.15.113:0/2810615550 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40b3590 con 0x7f58b4144a80 2024-04-25T00:06:18.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58b3fff640 1 --2- 172.21.15.113:0/2810615550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b4144a80 0x7f58b40b3ab0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f58b4070e00 tx=0x7f58a402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e29254786349eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58b2ffd640 1 -- 172.21.15.113:0/2810615550 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4032700 con 0x7f58b4144a80 2024-04-25T00:06:18.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58b2ffd640 1 -- 172.21.15.113:0/2810615550 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a40328a0 con 0x7f58b4144a80 2024-04-25T00:06:18.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58b2ffd640 1 -- 172.21.15.113:0/2810615550 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4032c70 con 0x7f58b4144a80 2024-04-25T00:06:18.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 -- 172.21.15.113:0/2810615550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b4144a80 msgr2=0x7f58b40b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/2810615550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b4144a80 0x7f58b40b3ab0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f58b4070e00 tx=0x7f58a402c800 comp rx=0 tx=0).stop 2024-04-25T00:06:18.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 -- 172.21.15.113:0/2810615550 shutdown_connections 2024-04-25T00:06:18.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/2810615550 >> v2:172.21.15.178:3300/0 conn(0x7f58b406e610 0x7f58b40cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/2810615550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b40b3ff0 0x7f58b406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/2810615550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b4144a80 0x7f58b40b3ab0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 -- 172.21.15.113:0/2810615550 >> 172.21.15.113:0/2810615550 conn(0x7f58b40b1b30 msgr2=0x7f58b40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:18.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 -- 172.21.15.113:0/2810615550 shutdown_connections 2024-04-25T00:06:18.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.054+0000 7f58ba6bd640 1 -- 172.21.15.113:0/2810615550 wait complete. 2024-04-25T00:06:18.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.055+0000 7f58ba6bd640 1 Processor -- start 2024-04-25T00:06:18.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58ba6bd640 1 -- start start 2024-04-25T00:06:18.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58ba6bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b406e610 0x7f58b413a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58ba6bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b40b3ff0 0x7f58b41e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58ba6bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b4144a80 0x7f58b41472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58ba6bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40cdf00 con 0x7f58b40b3ff0 2024-04-25T00:06:18.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58ba6bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40cdd80 con 0x7f58b406e610 2024-04-25T00:06:18.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58ba6bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58b4072200 con 0x7f58b4144a80 2024-04-25T00:06:18.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58b37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b40b3ff0 0x7f58b41e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58b37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b40b3ff0 0x7f58b41e5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55860/0 (socket says 172.21.15.113:55860) 2024-04-25T00:06:18.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.056+0000 7f58b37fe640 1 -- 172.21.15.113:0/1911347826 learned_addr learned my addr 172.21.15.113:0/1911347826 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:18.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b8c5a640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b4144a80 0x7f58b41472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b3fff640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b406e610 0x7f58b413a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b37fe640 1 -- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b4144a80 msgr2=0x7f58b41472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b37fe640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b4144a80 0x7f58b41472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b37fe640 1 -- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b406e610 msgr2=0x7f58b413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b37fe640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b406e610 0x7f58b413a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b37fe640 1 -- 172.21.15.113:0/1911347826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b41e4d00 con 0x7f58b40b3ff0 2024-04-25T00:06:18.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b37fe640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b40b3ff0 0x7f58b41e5da0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c7f0 tx=0x7f58a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b0ff9640 1 -- 172.21.15.113:0/1911347826 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8015070 con 0x7f58b40b3ff0 2024-04-25T00:06:18.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b0ff9640 1 -- 172.21.15.113:0/1911347826 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a8004240 con 0x7f58b40b3ff0 2024-04-25T00:06:18.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b41e4fe0 con 0x7f58b40b3ff0 2024-04-25T00:06:18.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58b0ff9640 1 -- 172.21.15.113:0/1911347826 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8007620 con 0x7f58b40b3ff0 2024-04-25T00:06:18.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.057+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b41e55a0 con 0x7f58b40b3ff0 2024-04-25T00:06:18.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.058+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b41e62e0 con 0x7f58b40b3ff0 2024-04-25T00:06:18.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.058+0000 7f58b0ff9640 1 -- 172.21.15.113:0/1911347826 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58a80079d0 con 0x7f58b40b3ff0 2024-04-25T00:06:18.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.059+0000 7f58b0ff9640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a006be30 0x7f58a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.059+0000 7f58b0ff9640 1 -- 172.21.15.113:0/1911347826 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4857+0+0 (secure 0 0 0) 0x7f58a808ac10 con 0x7f58b40b3ff0 2024-04-25T00:06:18.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.059+0000 7f58b3fff640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a006be30 0x7f58a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.059+0000 7f58b3fff640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a006be30 0x7f58a006e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f58a4037000 tx=0x7f58a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.063+0000 7f58b0ff9640 1 -- 172.21.15.113:0/1911347826 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58a8059910 con 0x7f58b40b3ff0 2024-04-25T00:06:18.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.236+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f58b40b29a0 con 0x7f58b40b3ff0 2024-04-25T00:06:18.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.675+0000 7f58b0ff9640 1 -- 172.21.15.113:0/1911347826 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v127) ==== 138+0+0 (secure 0 0 0) 0x7f58a8059210 con 0x7f58b40b3ff0 2024-04-25T00:06:18.675 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a006be30 msgr2=0x7f58a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a006be30 0x7f58a006e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f58a4037000 tx=0x7f58a4008040 comp rx=0 tx=0).stop 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b40b3ff0 msgr2=0x7f58b41e5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b40b3ff0 0x7f58b41e5da0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c7f0 tx=0x7f58a800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 shutdown_connections 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f58a006be30 0x7f58a006e2f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b4144a80 0x7f58b41472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58b40b3ff0 0x7f58b41e5da0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.677+0000 7f58ba6bd640 1 --2- 172.21.15.113:0/1911347826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b406e610 0x7f58b413a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.678+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 >> 172.21.15.113:0/1911347826 conn(0x7f58b40b1b30 msgr2=0x7f58b40af810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.678+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 shutdown_connections 2024-04-25T00:06:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.678+0000 7f58ba6bd640 1 -- 172.21.15.113:0/1911347826 wait complete. 2024-04-25T00:06:18.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:06:18.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.796+0000 7f8b19041640 1 Processor -- start 2024-04-25T00:06:18.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.796+0000 7f8b19041640 1 -- start start 2024-04-25T00:06:18.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b19041640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b1415e2d0 0x7f8b1415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b19041640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b140b3080 0x7f8b140b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b19041640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b1415e6b0 0x7f8b140b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b19041640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8b140cde00 con 0x7f8b1415e6b0 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b19041640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1415f4f0 con 0x7f8b1415e2d0 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b19041640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1415f7d0 con 0x7f8b140b3080 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b1415e6b0 0x7f8b140b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b1415e6b0 0x7f8b140b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55864/0 (socket says 172.21.15.113:55864) 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 -- 172.21.15.113:0/1767632733 learned_addr learned my addr 172.21.15.113:0/1767632733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b12d9d640 1 --2- 172.21.15.113:0/1767632733 >> v2:172.21.15.178:3300/0 conn(0x7f8b1415e2d0 0x7f8b1415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1259c640 1 --2- 172.21.15.113:0/1767632733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b140b3080 0x7f8b140b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 -- 172.21.15.113:0/1767632733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b140b3080 msgr2=0x7f8b140b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 --2- 172.21.15.113:0/1767632733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b140b3080 0x7f8b140b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 -- 172.21.15.113:0/1767632733 >> v2:172.21.15.178:3300/0 conn(0x7f8b1415e2d0 msgr2=0x7f8b1415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:18.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 --2- 172.21.15.113:0/1767632733 >> v2:172.21.15.178:3300/0 conn(0x7f8b1415e2d0 0x7f8b1415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.797+0000 7f8b1359e640 1 -- 172.21.15.113:0/1767632733 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1415fab0 con 0x7f8b1415e6b0 2024-04-25T00:06:18.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.798+0000 7f8b1359e640 1 --2- 172.21.15.113:0/1767632733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e6b0 0x7f8b140b4870 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f8b140b2d40 tx=0x7f8b0c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48ee373cfb523a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.798+0000 7f8b11d9b640 1 -- 172.21.15.113:0/1767632733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c036070 con 0x7f8b1415e6b0 2024-04-25T00:06:18.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.798+0000 7f8b11d9b640 1 -- 172.21.15.113:0/1767632733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0c031600 con 0x7f8b1415e6b0 2024-04-25T00:06:18.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.798+0000 7f8b11d9b640 1 -- 172.21.15.113:0/1767632733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0319d0 con 0x7f8b1415e6b0 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 -- 172.21.15.113:0/1767632733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e6b0 msgr2=0x7f8b140b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 --2- 172.21.15.113:0/1767632733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e6b0 0x7f8b140b4870 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f8b140b2d40 tx=0x7f8b0c02a610 comp rx=0 tx=0).stop 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 -- 172.21.15.113:0/1767632733 shutdown_connections 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 --2- 172.21.15.113:0/1767632733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e6b0 0x7f8b140b4870 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 --2- 172.21.15.113:0/1767632733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b140b3080 0x7f8b140b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 --2- 172.21.15.113:0/1767632733 >> v2:172.21.15.178:3300/0 conn(0x7f8b1415e2d0 0x7f8b1415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 -- 172.21.15.113:0/1767632733 >> 172.21.15.113:0/1767632733 conn(0x7f8b1406b0f0 msgr2=0x7f8b1406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 -- 172.21.15.113:0/1767632733 shutdown_connections 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.799+0000 7f8b19041640 1 -- 172.21.15.113:0/1767632733 wait complete. 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.800+0000 7f8b19041640 1 Processor -- start 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.800+0000 7f8b19041640 1 -- start start 2024-04-25T00:06:18.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b19041640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b140b3080 0x7f8b14155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b19041640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e2d0 0x7f8b14151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b19041640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b1415e6b0 0x7f8b141f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b19041640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1415f2b0 con 0x7f8b1415e2d0 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b19041640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1415f130 con 0x7f8b140b3080 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b19041640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b14071040 con 0x7f8b1415e6b0 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b1259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e2d0 0x7f8b14151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b1259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e2d0 0x7f8b14151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55866/0 (socket says 172.21.15.113:55866) 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b1259c640 1 -- 172.21.15.113:0/4281563651 learned_addr learned my addr 172.21.15.113:0/4281563651 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b1359e640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b1415e6b0 0x7f8b141f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.801+0000 7f8b12d9d640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b140b3080 0x7f8b14155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b1259c640 1 -- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b1415e6b0 msgr2=0x7f8b141f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b1259c640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b1415e6b0 0x7f8b141f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b1259c640 1 -- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b140b3080 msgr2=0x7f8b14155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b1259c640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b140b3080 0x7f8b14155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b1259c640 1 -- 172.21.15.113:0/4281563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b141f2520 con 0x7f8b1415e2d0 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b1259c640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e2d0 0x7f8b14151ee0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800d7d0 tx=0x7f8b0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8afb7fe640 1 -- 172.21.15.113:0/4281563651 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800b6c0 con 0x7f8b1415e2d0 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8afb7fe640 1 -- 172.21.15.113:0/4281563651 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b08019070 con 0x7f8b1415e2d0 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b14153f50 con 0x7f8b1415e2d0 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8afb7fe640 1 -- 172.21.15.113:0/4281563651 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800bac0 con 0x7f8b1415e2d0 2024-04-25T00:06:18.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.802+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b140a9760 con 0x7f8b1415e2d0 2024-04-25T00:06:18.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.803+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b140691a0 con 0x7f8b1415e2d0 2024-04-25T00:06:18.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.803+0000 7f8afb7fe640 1 -- 172.21.15.113:0/4281563651 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b0800bc60 con 0x7f8b1415e2d0 2024-04-25T00:06:18.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.804+0000 7f8afb7fe640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b0006be30 0x7f8b0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.804+0000 7f8afb7fe640 1 -- 172.21.15.113:0/4281563651 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5227+0+0 (secure 0 0 0) 0x7f8b0808b0b0 con 0x7f8b1415e2d0 2024-04-25T00:06:18.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.804+0000 7f8b12d9d640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b0006be30 0x7f8b0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.804+0000 7f8b12d9d640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b0006be30 0x7f8b0006e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006fd0 tx=0x7f8b04008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.808+0000 7f8afb7fe640 1 -- 172.21.15.113:0/4281563651 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8b141f2520 con 0x7f8b1415e2d0 2024-04-25T00:06:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:18.984+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8b14072050 con 0x7f8b1415e2d0 2024-04-25T00:06:19.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.680+0000 7f8afb7fe640 1 -- 172.21.15.113:0/4281563651 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v128) ==== 170+0+0 (secure 0 0 0) 0x7f8b0801c360 con 0x7f8b1415e2d0 2024-04-25T00:06:19.681 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:06:19.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b0006be30 msgr2=0x7f8b0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b0006be30 0x7f8b0006e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006fd0 tx=0x7f8b04008040 comp rx=0 tx=0).stop 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e2d0 msgr2=0x7f8b14151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e2d0 0x7f8b14151ee0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800d7d0 tx=0x7f8b0800dc90 comp rx=0 tx=0).stop 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 shutdown_connections 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8b0006be30 0x7f8b0006e2f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b1415e6b0 0x7f8b141f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1415e2d0 0x7f8b14151ee0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 --2- 172.21.15.113:0/4281563651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b140b3080 0x7f8b14155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.685+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 >> 172.21.15.113:0/4281563651 conn(0x7f8b1406b0f0 msgr2=0x7f8b140b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:19.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.688+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 shutdown_connections 2024-04-25T00:06:19.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.688+0000 7f8b19041640 1 -- 172.21.15.113:0/4281563651 wait complete. 2024-04-25T00:06:19.714 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:06:19.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.817+0000 7f183e853640 1 Processor -- start 2024-04-25T00:06:19.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.817+0000 7f183e853640 1 -- start start 2024-04-25T00:06:19.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.817+0000 7f183e853640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f183815e2d0 0x7f183815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:19.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.817+0000 7f183e853640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 0x7f18380b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:19.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.817+0000 7f183e853640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183815f4f0 con 0x7f183815e6b0 2024-04-25T00:06:19.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.817+0000 7f183e853640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f18380cde00 con 0x7f183815e2d0 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.817+0000 7f183e853640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f183815f7d0 con 0x7f18380b3080 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f1837fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f183815e2d0 0x7f183815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f183cdf0640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f183815e6b0 0x7f1838071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47612/0 (socket says 172.21.15.113:47612) 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f183cdf0640 1 -- 172.21.15.113:0/2421212261 learned_addr learned my addr 172.21.15.113:0/2421212261 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f1837fff640 1 -- 172.21.15.113:0/2421212261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 msgr2=0x7f18380b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f1837fff640 1 --2- 172.21.15.113:0/2421212261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 0x7f18380b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f1837fff640 1 -- 172.21.15.113:0/2421212261 >> v1:172.21.15.113:6789/0 conn(0x7f183815e6b0 legacy=0x7f1838071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f1836ffd640 1 -- 172.21.15.113:0/2421212261 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145922639 0 0) 0x7f183815f4f0 con 0x7f183815e6b0 2024-04-25T00:06:19.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.818+0000 7f1837fff640 1 -- 172.21.15.113:0/2421212261 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183815fab0 con 0x7f183815e2d0 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f1837fff640 1 --2- 172.21.15.113:0/2421212261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f183815ebf0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f18280097a0 tx=0x7f182802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42af0e4f5aeddad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f1836ffd640 1 -- 172.21.15.113:0/2421212261 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828037070 con 0x7f183815e2d0 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f1836ffd640 1 -- 172.21.15.113:0/2421212261 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182802ae50 con 0x7f183815e2d0 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f1836ffd640 1 -- 172.21.15.113:0/2421212261 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828032660 con 0x7f183815e2d0 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 -- 172.21.15.113:0/2421212261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 msgr2=0x7f183815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 --2- 172.21.15.113:0/2421212261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f183815ebf0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f18280097a0 tx=0x7f182802a100 comp rx=0 tx=0).stop 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 -- 172.21.15.113:0/2421212261 shutdown_connections 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 --2- 172.21.15.113:0/2421212261 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 0x7f18380b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 --2- 172.21.15.113:0/2421212261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f183815ebf0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 -- 172.21.15.113:0/2421212261 >> 172.21.15.113:0/2421212261 conn(0x7f183806b0f0 msgr2=0x7f183806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 -- 172.21.15.113:0/2421212261 shutdown_connections 2024-04-25T00:06:19.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.819+0000 7f183e853640 1 -- 172.21.15.113:0/2421212261 wait complete. 2024-04-25T00:06:19.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.820+0000 7f183e853640 1 Processor -- start 2024-04-25T00:06:19.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f183e853640 1 -- start start 2024-04-25T00:06:19.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f183e853640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 0x7f183814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f183e853640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f18381f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f183e853640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f183815e6b0 0x7f18380ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f183e853640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f183815f2b0 con 0x7f183815e2d0 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f183e853640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f183815f130 con 0x7f183815e6b0 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f183e853640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18380cf2a0 con 0x7f18380b3080 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f18377fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f18381f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f18377fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f18381f4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55892/0 (socket says 172.21.15.113:55892) 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.821+0000 7f18377fe640 1 -- 172.21.15.113:0/2141031272 learned_addr learned my addr 172.21.15.113:0/2141031272 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f18377fe640 1 -- 172.21.15.113:0/2141031272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 msgr2=0x7f183814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f18377fe640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 0x7f183814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f18377fe640 1 -- 172.21.15.113:0/2141031272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f183815e6b0 msgr2=0x7f18380ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f18377fe640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f183815e6b0 0x7f18380ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:19.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f18377fe640 1 -- 172.21.15.113:0/2141031272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18380cc3a0 con 0x7f183815e2d0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f18377fe640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f18381f4170 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b480 tx=0x7f182c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f1834ff9640 1 -- 172.21.15.113:0/2141031272 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c014070 con 0x7f183815e2d0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f1834ff9640 1 -- 172.21.15.113:0/2141031272 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182c010870 con 0x7f183815e2d0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18380cc680 con 0x7f183815e2d0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f1834ff9640 1 -- 172.21.15.113:0/2141031272 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c010c60 con 0x7f183815e2d0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.822+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183815ca30 con 0x7f183815e2d0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.823+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18380691a0 con 0x7f183815e2d0 2024-04-25T00:06:19.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.823+0000 7f1834ff9640 1 -- 172.21.15.113:0/2141031272 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f182c017070 con 0x7f183815e2d0 2024-04-25T00:06:19.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.824+0000 7f1834ff9640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f182406be30 0x7f182406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:19.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.824+0000 7f1834ff9640 1 -- 172.21.15.113:0/2141031272 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5609+0+0 (secure 0 0 0) 0x7f182c08ac40 con 0x7f183815e2d0 2024-04-25T00:06:19.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.824+0000 7f1837fff640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f182406be30 0x7f182406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:19.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.824+0000 7f1837fff640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f182406be30 0x7f182406e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1828034000 tx=0x7f182802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:19.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:19.828+0000 7f1834ff9640 1 -- 172.21.15.113:0/2141031272 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f182c0596e0 con 0x7f183815e2d0 2024-04-25T00:06:20.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.037+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1838072050 con 0x7f183815e2d0 2024-04-25T00:06:20.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.682+0000 7f1834ff9640 1 -- 172.21.15.113:0/2141031272 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v66) ==== 174+0+0 (secure 0 0 0) 0x7f182c058fe0 con 0x7f183815e2d0 2024-04-25T00:06:20.683 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 18 and data pool 19 2024-04-25T00:06:20.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.689+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f182406be30 msgr2=0x7f182406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:20.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.689+0000 7f183e853640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f182406be30 0x7f182406e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1828034000 tx=0x7f182802c000 comp rx=0 tx=0).stop 2024-04-25T00:06:20.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.689+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 msgr2=0x7f18381f4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:20.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.689+0000 7f183e853640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f18381f4170 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b480 tx=0x7f182c00b940 comp rx=0 tx=0).stop 2024-04-25T00:06:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.692+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 shutdown_connections 2024-04-25T00:06:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.692+0000 7f183e853640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f182406be30 0x7f182406e2f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.692+0000 7f183e853640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f183815e6b0 0x7f18380ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.692+0000 7f183e853640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f183815e2d0 0x7f18381f4170 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.692+0000 7f183e853640 1 --2- 172.21.15.113:0/2141031272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18380b3080 0x7f183814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.692+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 >> 172.21.15.113:0/2141031272 conn(0x7f183806b0f0 msgr2=0x7f183806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:20.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.698+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 shutdown_connections 2024-04-25T00:06:20.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.698+0000 7f183e853640 1 -- 172.21.15.113:0/2141031272 wait complete. 2024-04-25T00:06:20.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:20.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.823+0000 7f955e31d640 1 Processor -- start 2024-04-25T00:06:20.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.823+0000 7f955e31d640 1 -- start start 2024-04-25T00:06:20.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955e31d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f955806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:20.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955e31d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f955806e510 0x7f95580cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:20.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955e31d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9558144cf0 con 0x7f9558144910 2024-04-25T00:06:20.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955e31d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f95580b2f30 con 0x7f955806e510 2024-04-25T00:06:20.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955e31d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f95580b3210 con 0x7f95580b3fd0 2024-04-25T00:06:20.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f95577fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f955806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955c8ba640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f955806e510 0x7f95580cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f95577fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f955806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:54280/0 (socket says 172.21.15.113:54280) 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955c8ba640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f955806e510 0x7f95580cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55474/0 (socket says 172.21.15.113:55474) 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f95577fe640 1 -- 172.21.15.113:0/3234911070 learned_addr learned my addr 172.21.15.113:0/3234911070 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f9557fff640 1 --1- 172.21.15.113:0/3234911070 >> v1:172.21.15.178:6789/0 conn(0x7f9558144910 0x7f95580b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:40936/0 (socket says 172.21.15.113:40936) 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f95577fe640 1 -- 172.21.15.113:0/3234911070 >> v2:172.21.15.178:3300/0 conn(0x7f955806e510 msgr2=0x7f95580cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f95577fe640 1 --2- 172.21.15.113:0/3234911070 >> v2:172.21.15.178:3300/0 conn(0x7f955806e510 0x7f95580cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f95577fe640 1 -- 172.21.15.113:0/3234911070 >> v1:172.21.15.178:6789/0 conn(0x7f9558144910 legacy=0x7f95580b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f95577fe640 1 -- 172.21.15.113:0/3234911070 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95580b34f0 con 0x7f95580b3fd0 2024-04-25T00:06:20.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.824+0000 7f955c8ba640 1 --2- 172.21.15.113:0/3234911070 >> v2:172.21.15.178:3300/0 conn(0x7f955806e510 0x7f95580cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:20.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.825+0000 7f95577fe640 1 --2- 172.21.15.113:0/3234911070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f955806dfd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f954c009870 tx=0x7f954c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e254412789b5c636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:20.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.825+0000 7f9556ffd640 1 -- 172.21.15.113:0/3234911070 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c037070 con 0x7f95580b3fd0 2024-04-25T00:06:20.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.825+0000 7f9556ffd640 1 -- 172.21.15.113:0/3234911070 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c032440 con 0x7f95580b3fd0 2024-04-25T00:06:20.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.825+0000 7f9556ffd640 1 -- 172.21.15.113:0/3234911070 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c032810 con 0x7f95580b3fd0 2024-04-25T00:06:20.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 -- 172.21.15.113:0/3234911070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 msgr2=0x7f955806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 --2- 172.21.15.113:0/3234911070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f955806dfd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f954c009870 tx=0x7f954c02a270 comp rx=0 tx=0).stop 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 -- 172.21.15.113:0/3234911070 shutdown_connections 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 --2- 172.21.15.113:0/3234911070 >> v2:172.21.15.178:3300/0 conn(0x7f955806e510 0x7f95580cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 --2- 172.21.15.113:0/3234911070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f955806dfd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 -- 172.21.15.113:0/3234911070 >> 172.21.15.113:0/3234911070 conn(0x7f95580b1a50 msgr2=0x7f95580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 -- 172.21.15.113:0/3234911070 shutdown_connections 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.826+0000 7f955e31d640 1 -- 172.21.15.113:0/3234911070 wait complete. 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.827+0000 7f955e31d640 1 Processor -- start 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.827+0000 7f955e31d640 1 -- start start 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f955e31d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f955806e510 0x7f9558138750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f955e31d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f95580cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f955e31d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9558144910 0x7f95580ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f955e31d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f95580b4da0 con 0x7f955806e510 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f955e31d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95580b4410 con 0x7f9558144910 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f955e31d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f95580b4f20 con 0x7f95580b3fd0 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f955806e510 0x7f9558138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f955806e510 0x7f9558138750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55896/0 (socket says 172.21.15.113:55896) 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 -- 172.21.15.113:0/17117905 learned_addr learned my addr 172.21.15.113:0/17117905 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f95577fe640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f95580cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f955c8ba640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9558144910 0x7f95580ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 -- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 msgr2=0x7f95580cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f95580cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 -- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9558144910 msgr2=0x7f95580ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9558144910 0x7f95580ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.828+0000 7f9557fff640 1 -- 172.21.15.113:0/17117905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955813bdf0 con 0x7f955806e510 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.829+0000 7f9557fff640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f955806e510 0x7f9558138750 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f954800e730 tx=0x7f954800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.829+0000 7f9554ff9640 1 -- 172.21.15.113:0/17117905 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954801a070 con 0x7f955806e510 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.829+0000 7f9554ff9640 1 -- 172.21.15.113:0/17117905 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9548006bf0 con 0x7f955806e510 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.829+0000 7f9554ff9640 1 -- 172.21.15.113:0/17117905 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954800c640 con 0x7f955806e510 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.829+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95580cea40 con 0x7f955806e510 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.829+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955813d1d0 con 0x7f955806e510 2024-04-25T00:06:20.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.830+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95580691a0 con 0x7f955806e510 2024-04-25T00:06:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.830+0000 7f9554ff9640 1 -- 172.21.15.113:0/17117905 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f954800c7e0 con 0x7f955806e510 2024-04-25T00:06:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.830+0000 7f9554ff9640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f954406be60 0x7f954406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.831+0000 7f9554ff9640 1 -- 172.21.15.113:0/17117905 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f954808c070 con 0x7f955806e510 2024-04-25T00:06:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.831+0000 7f95577fe640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f954406be60 0x7f954406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.831+0000 7f95577fe640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f954406be60 0x7f954406e320 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f954c034000 tx=0x7f954c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:20.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:20.835+0000 7f9554ff9640 1 -- 172.21.15.113:0/17117905 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f954805a3e0 con 0x7f955806e510 2024-04-25T00:06:21.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.009+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95580b10f0 con 0x7f955806e510 2024-04-25T00:06:21.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.010+0000 7f9554ff9640 1 -- 172.21.15.113:0/17117905 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16740 (secure 0 0 0) 0x7f954805e6e0 con 0x7f955806e510 2024-04-25T00:06:21.011 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:21.011 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":129,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:20.038794+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:06:18.255623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:06:19.003747+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:21.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f954406be60 msgr2=0x7f954406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f954406be60 0x7f954406e320 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f954c034000 tx=0x7f954c02e000 comp rx=0 tx=0).stop 2024-04-25T00:06:21.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f955806e510 msgr2=0x7f9558138750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f955806e510 0x7f9558138750 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f954800e730 tx=0x7f954800ebf0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 shutdown_connections 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f954406be60 0x7f954406e320 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9558144910 0x7f95580ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95580b3fd0 0x7f95580cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 --2- 172.21.15.113:0/17117905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f955806e510 0x7f9558138750 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.012+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 >> 172.21.15.113:0/17117905 conn(0x7f95580b1a50 msgr2=0x7f95580b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.013+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 shutdown_connections 2024-04-25T00:06:21.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.013+0000 7f955e31d640 1 -- 172.21.15.113:0/17117905 wait complete. 2024-04-25T00:06:21.025 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:21.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.120+0000 7f4590b3b640 1 Processor -- start 2024-04-25T00:06:21.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.120+0000 7f4590b3b640 1 -- start start 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.121+0000 7f4590b3b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f458c144910 0x7f458c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.121+0000 7f4590b3b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f458c0b5240 0x7f458c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.121+0000 7f4590b3b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c144cf0 con 0x7f458c0b56b0 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.121+0000 7f4590b3b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f458c0b4510 con 0x7f458c144910 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.121+0000 7f4590b3b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f458c0b2f30 con 0x7f458c0b5240 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f458c144910 0x7f458c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458ad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f458c0b56b0 0x7f458c0cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47618/0 (socket says 172.21.15.113:47618) 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458ad9d640 1 -- 172.21.15.113:0/2601072089 learned_addr learned my addr 172.21.15.113:0/2601072089 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f458c144910 0x7f458c0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55904/0 (socket says 172.21.15.113:55904) 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f4589d9b640 1 --2- 172.21.15.113:0/2601072089 >> v2:172.21.15.178:3300/0 conn(0x7f458c0b5240 0x7f458c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458a59c640 1 -- 172.21.15.113:0/2601072089 >> v2:172.21.15.178:3300/0 conn(0x7f458c0b5240 msgr2=0x7f458c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458a59c640 1 --2- 172.21.15.113:0/2601072089 >> v2:172.21.15.178:3300/0 conn(0x7f458c0b5240 0x7f458c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458a59c640 1 -- 172.21.15.113:0/2601072089 >> v1:172.21.15.113:6789/0 conn(0x7f458c0b56b0 legacy=0x7f458c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458959a640 1 -- 172.21.15.113:0/2601072089 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726259301 0 0) 0x7f458c144cf0 con 0x7f458c0b56b0 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f458a59c640 1 -- 172.21.15.113:0/2601072089 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0b3210 con 0x7f458c144910 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.122+0000 7f4589d9b640 1 --2- 172.21.15.113:0/2601072089 >> v2:172.21.15.178:3300/0 conn(0x7f458c0b5240 0x7f458c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f458a59c640 1 --2- 172.21.15.113:0/2601072089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c144910 0x7f458c0b3fd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f45800097a0 tx=0x7f458002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e802b2e3f2562207 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f458959a640 1 -- 172.21.15.113:0/2601072089 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580037070 con 0x7f458c144910 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f458959a640 1 -- 172.21.15.113:0/2601072089 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4580032440 con 0x7f458c144910 2024-04-25T00:06:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f458959a640 1 -- 172.21.15.113:0/2601072089 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580032810 con 0x7f458c144910 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 -- 172.21.15.113:0/2601072089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c144910 msgr2=0x7f458c0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 --2- 172.21.15.113:0/2601072089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c144910 0x7f458c0b3fd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f45800097a0 tx=0x7f458002a1e0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 -- 172.21.15.113:0/2601072089 shutdown_connections 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 --2- 172.21.15.113:0/2601072089 >> v2:172.21.15.178:3300/0 conn(0x7f458c0b5240 0x7f458c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 --2- 172.21.15.113:0/2601072089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c144910 0x7f458c0b3fd0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 -- 172.21.15.113:0/2601072089 >> 172.21.15.113:0/2601072089 conn(0x7f458c0b1a50 msgr2=0x7f458c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 -- 172.21.15.113:0/2601072089 shutdown_connections 2024-04-25T00:06:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.123+0000 7f4590b3b640 1 -- 172.21.15.113:0/2601072089 wait complete. 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 Processor -- start 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 -- start start 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c0b5240 0x7f458c0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f458c0b56b0 0x7f458c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f458c144910 0x7f458c134340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0b4da0 con 0x7f458c0b5240 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0b37d0 con 0x7f458c144910 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f4590b3b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f458c0b4f20 con 0x7f458c0b56b0 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f458a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c0b5240 0x7f458c0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f458a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c0b5240 0x7f458c0af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55916/0 (socket says 172.21.15.113:55916) 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.125+0000 7f458a59c640 1 -- 172.21.15.113:0/1519749751 learned_addr learned my addr 172.21.15.113:0/1519749751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f458a59c640 1 -- 172.21.15.113:0/1519749751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f458c0b56b0 msgr2=0x7f458c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f458a59c640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f458c0b56b0 0x7f458c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f458a59c640 1 -- 172.21.15.113:0/1519749751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f458c144910 msgr2=0x7f458c134340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f458a59c640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f458c144910 0x7f458c134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f458a59c640 1 -- 172.21.15.113:0/1519749751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c134c20 con 0x7f458c0b5240 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f458a59c640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c0b5240 0x7f458c0af090 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f45800097a0 tx=0x7f45800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f457affd640 1 -- 172.21.15.113:0/1519749751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580037040 con 0x7f458c0b5240 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f457affd640 1 -- 172.21.15.113:0/1519749751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4580004030 con 0x7f458c0b5240 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f457affd640 1 -- 172.21.15.113:0/1519749751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580006e10 con 0x7f458c0b5240 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c134f00 con 0x7f458c0b5240 2024-04-25T00:06:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.126+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1354c0 con 0x7f458c0b5240 2024-04-25T00:06:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.127+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c0691a0 con 0x7f458c0b5240 2024-04-25T00:06:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.128+0000 7f457affd640 1 -- 172.21.15.113:0/1519749751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4580030770 con 0x7f458c0b5240 2024-04-25T00:06:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.128+0000 7f457affd640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f456c06be60 0x7f456c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.128+0000 7f4589d9b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f456c06be60 0x7f456c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.128+0000 7f457affd640 1 -- 172.21.15.113:0/1519749751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f45800ac050 con 0x7f458c0b5240 2024-04-25T00:06:21.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.128+0000 7f4589d9b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f456c06be60 0x7f456c06e320 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f457c006fd0 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.132+0000 7f457affd640 1 -- 172.21.15.113:0/1519749751 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f458007aaa0 con 0x7f458c0b5240 2024-04-25T00:06:21.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.307+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f458c0af9b0 con 0x7f458c0b5240 2024-04-25T00:06:21.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.308+0000 7f457affd640 1 -- 172.21.15.113:0/1519749751 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16740 (secure 0 0 0) 0x7f458007a3a0 con 0x7f458c0b5240 2024-04-25T00:06:21.309 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:21.309 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":129,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:20.038794+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:06:18.255623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:06:19.003747+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f456c06be60 msgr2=0x7f456c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f456c06be60 0x7f456c06e320 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f457c006fd0 tx=0x7f457c008040 comp rx=0 tx=0).stop 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c0b5240 msgr2=0x7f458c0af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c0b5240 0x7f458c0af090 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f45800097a0 tx=0x7f45800046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 shutdown_connections 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f456c06be60 0x7f456c06e320 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f458c144910 0x7f458c134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f458c0b56b0 0x7f458c15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 --2- 172.21.15.113:0/1519749751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458c0b5240 0x7f458c0af090 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 >> 172.21.15.113:0/1519749751 conn(0x7f458c0b1a50 msgr2=0x7f458c13b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 shutdown_connections 2024-04-25T00:06:21.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.310+0000 7f4590b3b640 1 -- 172.21.15.113:0/1519749751 wait complete. 2024-04-25T00:06:21.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:06:21.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.421+0000 7fc39e3a9640 1 Processor -- start 2024-04-25T00:06:21.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.421+0000 7fc39e3a9640 1 -- start start 2024-04-25T00:06:21.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39e3a9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39813f050 0x7fc3980cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39e3a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3981606a0 0x7fc3980cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39e3a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3980d0970 con 0x7fc3980d04f0 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39e3a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc39815c6a0 con 0x7fc3981606a0 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39e3a9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc398160aa0 con 0x7fc39813f050 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39c946640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3981606a0 0x7fc3980cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc3977fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc3980d04f0 0x7fc39815ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47626/0 (socket says 172.21.15.113:47626) 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc3977fe640 1 -- 172.21.15.113:0/3478147121 learned_addr learned my addr 172.21.15.113:0/3478147121 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39c946640 1 -- 172.21.15.113:0/3478147121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39813f050 msgr2=0x7fc3980cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39c946640 1 --2- 172.21.15.113:0/3478147121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39813f050 0x7fc3980cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39c946640 1 -- 172.21.15.113:0/3478147121 >> v1:172.21.15.113:6789/0 conn(0x7fc3980d04f0 legacy=0x7fc39815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc39c946640 1 -- 172.21.15.113:0/3478147121 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39815d6b0 con 0x7fc3981606a0 2024-04-25T00:06:21.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.422+0000 7fc396ffd640 1 -- 172.21.15.113:0/3478147121 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3483381334 0 0) 0x7fc3980d0970 con 0x7fc3980d04f0 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.423+0000 7fc39c946640 1 --2- 172.21.15.113:0/3478147121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3980cea60 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc390009d90 tx=0x7fc39002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281707afdf97b3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.423+0000 7fc396ffd640 1 -- 172.21.15.113:0/3478147121 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390031600 con 0x7fc3981606a0 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.423+0000 7fc396ffd640 1 -- 172.21.15.113:0/3478147121 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3900317a0 con 0x7fc3981606a0 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.423+0000 7fc396ffd640 1 -- 172.21.15.113:0/3478147121 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390031b70 con 0x7fc3981606a0 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 -- 172.21.15.113:0/3478147121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 msgr2=0x7fc3980cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/3478147121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3980cea60 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc390009d90 tx=0x7fc39002a5d0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 -- 172.21.15.113:0/3478147121 shutdown_connections 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/3478147121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3980cea60 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/3478147121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39813f050 0x7fc3980cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 -- 172.21.15.113:0/3478147121 >> 172.21.15.113:0/3478147121 conn(0x7fc39806b0f0 msgr2=0x7fc39806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 -- 172.21.15.113:0/3478147121 shutdown_connections 2024-04-25T00:06:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.424+0000 7fc39e3a9640 1 -- 172.21.15.113:0/3478147121 wait complete. 2024-04-25T00:06:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.425+0000 7fc39e3a9640 1 Processor -- start 2024-04-25T00:06:21.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.425+0000 7fc39e3a9640 1 -- start start 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3980d04f0 0x7fc3981f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39813f050 0x7fc39813b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3981f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc39815ceb0 con 0x7fc3981606a0 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc39815cbd0 con 0x7fc39813f050 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc39815d190 con 0x7fc3980d04f0 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3981f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3981f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55940/0 (socket says 172.21.15.113:55940) 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 -- 172.21.15.113:0/2061776813 learned_addr learned my addr 172.21.15.113:0/2061776813 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc3977fe640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39813f050 0x7fc39813b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc397fff640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3980d04f0 0x7fc3981f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 -- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3980d04f0 msgr2=0x7fc3981f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3980d04f0 0x7fc3981f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 -- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39813f050 msgr2=0x7fc39813b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39813f050 0x7fc39813b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 -- 172.21.15.113:0/2061776813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981f2ce0 con 0x7fc3981606a0 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc397fff640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3980d04f0 0x7fc3981f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39c946640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3981f2ee0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc390009eb0 tx=0x7fc390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc394ff9640 1 -- 172.21.15.113:0/2061776813 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390040070 con 0x7fc3981606a0 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc394ff9640 1 -- 172.21.15.113:0/2061776813 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc390030be0 con 0x7fc3981606a0 2024-04-25T00:06:21.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc394ff9640 1 -- 172.21.15.113:0/2061776813 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390009450 con 0x7fc3981606a0 2024-04-25T00:06:21.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3980b0520 con 0x7fc3981606a0 2024-04-25T00:06:21.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.426+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39815a5b0 con 0x7fc3981606a0 2024-04-25T00:06:21.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.427+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3980691a0 con 0x7fc3981606a0 2024-04-25T00:06:21.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.428+0000 7fc394ff9640 1 -- 172.21.15.113:0/2061776813 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc390039040 con 0x7fc3981606a0 2024-04-25T00:06:21.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.428+0000 7fc394ff9640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc38406be30 0x7fc38406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.428+0000 7fc394ff9640 1 -- 172.21.15.113:0/2061776813 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fc3900abd40 con 0x7fc3981606a0 2024-04-25T00:06:21.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.428+0000 7fc397fff640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc38406be30 0x7fc38406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.429+0000 7fc397fff640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc38406be30 0x7fc38406e2f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc388006fd0 tx=0x7fc388008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.433+0000 7fc394ff9640 1 -- 172.21.15.113:0/2061776813 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc3981f2ce0 con 0x7fc3981606a0 2024-04-25T00:06:21.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.643+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc3980d0e50 con 0x7fc3981606a0 2024-04-25T00:06:21.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.691+0000 7fc394ff9640 1 -- 172.21.15.113:0/2061776813 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) ==== 118+0+0 (secure 0 0 0) 0x7fc39007a780 con 0x7fc3981606a0 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc38406be30 msgr2=0x7fc38406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc38406be30 0x7fc38406e2f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc388006fd0 tx=0x7fc388008040 comp rx=0 tx=0).stop 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 msgr2=0x7fc3981f2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3981f2ee0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc390009eb0 tx=0x7fc390004a40 comp rx=0 tx=0).stop 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 shutdown_connections 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc38406be30 0x7fc38406e2f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39813f050 0x7fc39813b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3981606a0 0x7fc3981f2ee0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 --2- 172.21.15.113:0/2061776813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3980d04f0 0x7fc3981f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 >> 172.21.15.113:0/2061776813 conn(0x7fc39806b0f0 msgr2=0x7fc3980b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 shutdown_connections 2024-04-25T00:06:21.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.693+0000 7fc39e3a9640 1 -- 172.21.15.113:0/2061776813 wait complete. 2024-04-25T00:06:21.706 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:06:21.706 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:06:21.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.805+0000 7f18d70e1640 1 Processor -- start 2024-04-25T00:06:21.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.805+0000 7f18d70e1640 1 -- start start 2024-04-25T00:06:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18d70e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 0x7f18d015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18d70e1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d0071170 con 0x7f18d015b670 2024-04-25T00:06:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18d70e1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d0071450 con 0x7f18d006cea0 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18d70e1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18d0071730 con 0x7f18d015ba50 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18d4e7d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f18d015b670 0x7f18d00cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47638/0 (socket says 172.21.15.113:47638) 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18d4e7d640 1 -- 172.21.15.113:0/928820867 learned_addr learned my addr 172.21.15.113:0/928820867 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 202019881 0 0) 0x7f18d0071170 con 0x7f18d015b670 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18c0004630 con 0x7f18d015b670 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cffff640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 0x7f18d015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 449716863 0 0) 0x7f18c0004630 con 0x7f18d015b670 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 msgr2=0x7f18d015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 0x7f18d015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 >> v1:172.21.15.178:6789/0 conn(0x7f18d006cea0 legacy=0x7f18d0072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d0071a10 con 0x7f18d015b670 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f18c4001670 con 0x7f18d015b670 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 >> v1:172.21.15.113:6789/0 conn(0x7f18d015b670 legacy=0x7f18d00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c0004af0 0x7f18c0004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0005910 0x7f18c0005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 0x7f18c00063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0071730 con 0x7f18c0004af0 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18c000ccc0 con 0x7f18c0005910 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18c00035a0 con 0x7f18d015ba50 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.806+0000 7f18d4e7d640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c0004af0 0x7f18c0004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18cffff640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0005910 0x7f18c0005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18d567e640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 0x7f18c00063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18d4e7d640 1 -- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 msgr2=0x7f18c00063a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18d4e7d640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d015ba50 0x7f18c00063a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18d4e7d640 1 -- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0005910 msgr2=0x7f18c0005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18d4e7d640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0005910 0x7f18c0005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18d4e7d640 1 -- 172.21.15.113:0/928820867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c0004630 con 0x7f18c0004af0 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18cffff640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0005910 0x7f18c0005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18d4e7d640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c0004af0 0x7f18c0004f10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f18c400c240 tx=0x7f18c0006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115c6ba6cfcde6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4007d10 con 0x7f18c0004af0 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c4005ad0 con 0x7f18c0004af0 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.807+0000 7f18cf7fe640 1 -- 172.21.15.113:0/928820867 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4007070 con 0x7f18c0004af0 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.808+0000 7f18d70e1640 1 -- 172.21.15.113:0/928820867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c0004af0 msgr2=0x7f18c0004f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.808+0000 7f18d70e1640 1 --2- 172.21.15.113:0/928820867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18c0004af0 0x7f18c0004f10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f18c400c240 tx=0x7f18c0006940 comp rx=0 tx=0).stop 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.808+0000 7f18d567e640 1 -- 172.21.15.113:0/928820867 reap_dead start 2024-04-25T00:06:21.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.808+0000 7f18d70e1640 1 -- 172.21.15.113:0/928820867 shutdown_connections 2024-04-25T00:06:21.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.808+0000 7f18d70e1640 1 -- 172.21.15.113:0/928820867 >> 172.21.15.113:0/928820867 conn(0x7f18d006b0f0 msgr2=0x7f18d006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.808+0000 7f18d70e1640 1 -- 172.21.15.113:0/928820867 shutdown_connections 2024-04-25T00:06:21.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.808+0000 7f18d70e1640 1 -- 172.21.15.113:0/928820867 wait complete. 2024-04-25T00:06:21.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.809+0000 7f18d70e1640 1 Processor -- start 2024-04-25T00:06:21.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.809+0000 7f18d70e1640 1 -- start start 2024-04-25T00:06:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18d70e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0004af0 0x7f18d00cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18d70e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d00cf540 0x7f18d00cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18d70e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d01f7710 0x7f18d01f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18d70e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18d015c170 con 0x7f18d00cf540 2024-04-25T00:06:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18d70e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18d015bff0 con 0x7f18c0004af0 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18d70e1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18d015c2f0 con 0x7f18d01f7710 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d00cf540 0x7f18d00cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d00cf540 0x7f18d00cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55962/0 (socket says 172.21.15.113:55962) 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 -- 172.21.15.113:0/2012993573 learned_addr learned my addr 172.21.15.113:0/2012993573 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 -- 172.21.15.113:0/2012993573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d01f7710 msgr2=0x7f18d01f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d01f7710 0x7f18d01f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 -- 172.21.15.113:0/2012993573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0004af0 msgr2=0x7f18d00cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0004af0 0x7f18d00cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.810+0000 7f18cffff640 1 -- 172.21.15.113:0/2012993573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d00cb040 con 0x7f18d00cf540 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.811+0000 7f18cffff640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d00cf540 0x7f18d00cf9c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0116d0 tx=0x7f18bc011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.811+0000 7f18cd7fa640 1 -- 172.21.15.113:0/2012993573 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00f9c0 con 0x7f18d00cf540 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.811+0000 7f18cd7fa640 1 -- 172.21.15.113:0/2012993573 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc00fb60 con 0x7f18d00cf540 2024-04-25T00:06:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.811+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01f4ae0 con 0x7f18d00cf540 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.811+0000 7f18cd7fa640 1 -- 172.21.15.113:0/2012993573 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc015460 con 0x7f18d00cf540 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.811+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01f4000 con 0x7f18d00cf540 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.812+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d00691a0 con 0x7f18d00cf540 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.812+0000 7f18cd7fa640 1 -- 172.21.15.113:0/2012993573 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18bc015600 con 0x7f18d00cf540 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.812+0000 7f18cd7fa640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18b406be60 0x7f18b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.812+0000 7f18cd7fa640 1 -- 172.21.15.113:0/2012993573 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f18bc08b7a0 con 0x7f18d00cf540 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.812+0000 7f18d4e7d640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18b406be60 0x7f18b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.813+0000 7f18d4e7d640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18b406be60 0x7f18b406e320 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f18c4033000 tx=0x7f18c4009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:21.817+0000 7f18cd7fa640 1 -- 172.21.15.113:0/2012993573 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18bc019030 con 0x7f18d00cf540 2024-04-25T00:06:22.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.028+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f18d00cc180 con 0x7f18d00cf540 2024-04-25T00:06:22.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.699+0000 7f18cd7fa640 1 -- 172.21.15.113:0/2012993573 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) ==== 122+0+0 (secure 0 0 0) 0x7f18bc05a0c0 con 0x7f18d00cf540 2024-04-25T00:06:22.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18b406be60 msgr2=0x7f18b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:22.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18b406be60 0x7f18b406e320 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f18c4033000 tx=0x7f18c4009030 comp rx=0 tx=0).stop 2024-04-25T00:06:22.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d00cf540 msgr2=0x7f18d00cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d00cf540 0x7f18d00cf9c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0116d0 tx=0x7f18bc011b90 comp rx=0 tx=0).stop 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 shutdown_connections 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18b406be60 0x7f18b406e320 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18d01f7710 0x7f18d01f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d00cf540 0x7f18d00cf9c0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 --2- 172.21.15.113:0/2012993573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18c0004af0 0x7f18d00cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 >> 172.21.15.113:0/2012993573 conn(0x7f18d006b0f0 msgr2=0x7f18d0153be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 shutdown_connections 2024-04-25T00:06:22.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.703+0000 7f18d70e1640 1 -- 172.21.15.113:0/2012993573 wait complete. 2024-04-25T00:06:22.716 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:06:22.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.820+0000 7f99dfe76640 1 Processor -- start 2024-04-25T00:06:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.820+0000 7f99dfe76640 1 -- start start 2024-04-25T00:06:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99dfe76640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99d815e320 0x7f99d80d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99dfe76640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99d80d2290 0x7f99d80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99dfe76640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d80d0b00 0x7f99d80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99dfe76640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f99d80d14d0 con 0x7f99d815e320 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99dfe76640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f99d80d3410 con 0x7f99d80d2290 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99dfe76640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99d80d3de0 con 0x7f99d80d0b00 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99d815e320 0x7f99d80d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99d815e320 0x7f99d80d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55966/0 (socket says 172.21.15.113:55966) 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 -- 172.21.15.113:0/2349928635 learned_addr learned my addr 172.21.15.113:0/2349928635 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 -- 172.21.15.113:0/2349928635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d80d0b00 msgr2=0x7f99d80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 --2- 172.21.15.113:0/2349928635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d80d0b00 0x7f99d80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 -- 172.21.15.113:0/2349928635 >> v2:172.21.15.178:3300/0 conn(0x7f99d80d2290 msgr2=0x7f99d80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 --2- 172.21.15.113:0/2349928635 >> v2:172.21.15.178:3300/0 conn(0x7f99d80d2290 0x7f99d80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.821+0000 7f99ddc12640 1 -- 172.21.15.113:0/2349928635 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d80d2800 con 0x7f99d815e320 2024-04-25T00:06:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99ddc12640 1 --2- 172.21.15.113:0/2349928635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d815e320 0x7f99d80d38a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f99d8072810 tx=0x7f99d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226eff77c0d0a5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dcc10640 1 -- 172.21.15.113:0/2349928635 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4037070 con 0x7f99d815e320 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dcc10640 1 -- 172.21.15.113:0/2349928635 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d4032440 con 0x7f99d815e320 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dcc10640 1 -- 172.21.15.113:0/2349928635 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4032810 con 0x7f99d815e320 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dfe76640 1 -- 172.21.15.113:0/2349928635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d815e320 msgr2=0x7f99d80d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2349928635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d815e320 0x7f99d80d38a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f99d8072810 tx=0x7f99d402a270 comp rx=0 tx=0).stop 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dfe76640 1 -- 172.21.15.113:0/2349928635 shutdown_connections 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2349928635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d80d0b00 0x7f99d80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2349928635 >> v2:172.21.15.178:3300/0 conn(0x7f99d80d2290 0x7f99d80b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2349928635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d815e320 0x7f99d80d38a0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.822+0000 7f99dfe76640 1 -- 172.21.15.113:0/2349928635 >> 172.21.15.113:0/2349928635 conn(0x7f99d80b5c20 msgr2=0x7f99d80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.823+0000 7f99dfe76640 1 -- 172.21.15.113:0/2349928635 shutdown_connections 2024-04-25T00:06:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.823+0000 7f99dfe76640 1 -- 172.21.15.113:0/2349928635 wait complete. 2024-04-25T00:06:22.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 Processor -- start 2024-04-25T00:06:22.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 -- start start 2024-04-25T00:06:22.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d80d0b00 0x7f99d8154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:22.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d80d2290 0x7f99d81536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:22.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d815e320 0x7f99d80cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:22.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f99d80d30a0 con 0x7f99d80d2290 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99d80d2dc0 con 0x7f99d80d0b00 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dfe76640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99d80b9fe0 con 0x7f99d815e320 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.824+0000 7f99dd411640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d80d2290 0x7f99d81536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d80d2290 0x7f99d81536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55976/0 (socket says 172.21.15.113:55976) 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 -- 172.21.15.113:0/2679019079 learned_addr learned my addr 172.21.15.113:0/2679019079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99ddc12640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d80d0b00 0x7f99d8154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99de413640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d815e320 0x7f99d80cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 -- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d815e320 msgr2=0x7f99d80cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d815e320 0x7f99d80cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 -- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d80d0b00 msgr2=0x7f99d8154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d80d0b00 0x7f99d8154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 -- 172.21.15.113:0/2679019079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d815ffa0 con 0x7f99d80d2290 2024-04-25T00:06:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99ddc12640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d80d0b00 0x7f99d8154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dd411640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d80d2290 0x7f99d81536a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f99c800b4a0 tx=0x7f99c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99ce7fc640 1 -- 172.21.15.113:0/2679019079 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8014070 con 0x7f99d80d2290 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99ce7fc640 1 -- 172.21.15.113:0/2679019079 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c8010610 con 0x7f99d80d2290 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d8160280 con 0x7f99d80d2290 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99ce7fc640 1 -- 172.21.15.113:0/2679019079 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8010a00 con 0x7f99d80d2290 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.825+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d8155320 con 0x7f99d80d2290 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.826+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d80cab10 con 0x7f99d80d2290 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.827+0000 7f99ce7fc640 1 -- 172.21.15.113:0/2679019079 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99c8017070 con 0x7f99d80d2290 2024-04-25T00:06:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.827+0000 7f99ce7fc640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f99bc06be30 0x7f99bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.827+0000 7f99ce7fc640 1 -- 172.21.15.113:0/2679019079 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f99c808a960 con 0x7f99d80d2290 2024-04-25T00:06:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.827+0000 7f99ddc12640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f99bc06be30 0x7f99bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.827+0000 7f99ddc12640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f99bc06be30 0x7f99bc06e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f99d4034000 tx=0x7f99d402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:22.831+0000 7f99ce7fc640 1 -- 172.21.15.113:0/2679019079 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f99d80cab10 con 0x7f99d80d2290 2024-04-25T00:06:23.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.049+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99d80691a0 con 0x7f99d80d2290 2024-04-25T00:06:23.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.049+0000 7f99ce7fc640 1 -- 172.21.15.113:0/2679019079 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7f99c8059290 con 0x7f99d80d2290 2024-04-25T00:06:23.050 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:23.050 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:23.051 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f99bc06be30 msgr2=0x7f99bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f99bc06be30 0x7f99bc06e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f99d4034000 tx=0x7f99d402e000 comp rx=0 tx=0).stop 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d80d2290 msgr2=0x7f99d81536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d80d2290 0x7f99d81536a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f99c800b4a0 tx=0x7f99c800b960 comp rx=0 tx=0).stop 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 shutdown_connections 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f99bc06be30 0x7f99bc06e2f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99d815e320 0x7f99d80cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99d80d2290 0x7f99d81536a0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 --2- 172.21.15.113:0/2679019079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d80d0b00 0x7f99d8154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 >> 172.21.15.113:0/2679019079 conn(0x7f99d80b5c20 msgr2=0x7f99d8071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 shutdown_connections 2024-04-25T00:06:23.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.052+0000 7f99dfe76640 1 -- 172.21.15.113:0/2679019079 wait complete. 2024-04-25T00:06:23.063 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:06:23.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.162+0000 7fad0f21c640 1 Processor -- start 2024-04-25T00:06:23.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.162+0000 7fad0f21c640 1 -- start start 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0f21c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad08144d60 0x7fad080b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0f21c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad080cfad0 con 0x7fad0806e6e0 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0f21c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad080cf230 con 0x7fad081453c0 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0f21c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fad08145c40 con 0x7fad08144d60 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0cfb8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad08144d60 0x7fad080b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0d7b9640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fad0806e6e0 0x7fad08071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47654/0 (socket says 172.21.15.113:47654) 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0d7b9640 1 -- 172.21.15.113:0/2150600827 learned_addr learned my addr 172.21.15.113:0/2150600827 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0cfb8640 1 -- 172.21.15.113:0/2150600827 >> v1:172.21.15.178:6789/0 conn(0x7fad081453c0 legacy=0x7fad0806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0cfb8640 1 -- 172.21.15.113:0/2150600827 >> v1:172.21.15.113:6789/0 conn(0x7fad0806e6e0 legacy=0x7fad08071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.163+0000 7fad0cfb8640 1 -- 172.21.15.113:0/2150600827 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad080cf510 con 0x7fad08144d60 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.164+0000 7fad0cfb8640 1 --2- 172.21.15.113:0/2150600827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad080b5390 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7facfc0097a0 tx=0x7facfc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f61739bd09322c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.164+0000 7fad077fe640 1 -- 172.21.15.113:0/2150600827 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc037070 con 0x7fad08144d60 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.164+0000 7fad077fe640 1 -- 172.21.15.113:0/2150600827 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc02ae50 con 0x7fad08144d60 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.164+0000 7fad077fe640 1 -- 172.21.15.113:0/2150600827 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc032660 con 0x7fad08144d60 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.165+0000 7fad0f21c640 1 -- 172.21.15.113:0/2150600827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 msgr2=0x7fad080b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.165+0000 7fad0f21c640 1 --2- 172.21.15.113:0/2150600827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad080b5390 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7facfc0097a0 tx=0x7facfc02a140 comp rx=0 tx=0).stop 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.165+0000 7fad0f21c640 1 -- 172.21.15.113:0/2150600827 shutdown_connections 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.165+0000 7fad0f21c640 1 --2- 172.21.15.113:0/2150600827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad080b5390 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.165+0000 7fad0f21c640 1 -- 172.21.15.113:0/2150600827 >> 172.21.15.113:0/2150600827 conn(0x7fad080b1a50 msgr2=0x7fad080b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.165+0000 7fad0f21c640 1 -- 172.21.15.113:0/2150600827 shutdown_connections 2024-04-25T00:06:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.165+0000 7fad0f21c640 1 -- 172.21.15.113:0/2150600827 wait complete. 2024-04-25T00:06:23.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.166+0000 7fad0f21c640 1 Processor -- start 2024-04-25T00:06:23.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.166+0000 7fad0f21c640 1 -- start start 2024-04-25T00:06:23.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0f21c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad0806e6e0 0x7fad0813e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0f21c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad081e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0f21c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad081453c0 0x7fad081e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0f21c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad08145140 con 0x7fad08144d60 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0f21c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad08070e10 con 0x7fad081453c0 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0f21c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad08071a40 con 0x7fad0806e6e0 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad081e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad081e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55992/0 (socket says 172.21.15.113:55992) 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 -- 172.21.15.113:0/106180387 learned_addr learned my addr 172.21.15.113:0/106180387 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0cfb8640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad0806e6e0 0x7fad0813e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad0d7b9640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad081453c0 0x7fad081e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 -- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad0806e6e0 msgr2=0x7fad0813e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad0806e6e0 0x7fad0813e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 -- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad081453c0 msgr2=0x7fad081e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad081453c0 0x7fad081e6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.167+0000 7fad07fff640 1 -- 172.21.15.113:0/106180387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0815b8a0 con 0x7fad08144d60 2024-04-25T00:06:23.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.168+0000 7fad07fff640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad081e5f40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7facf400b3f0 tx=0x7facf400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.168+0000 7fad057fa640 1 -- 172.21.15.113:0/106180387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4015020 con 0x7fad08144d60 2024-04-25T00:06:23.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.168+0000 7fad057fa640 1 -- 172.21.15.113:0/106180387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf4011830 con 0x7fad08144d60 2024-04-25T00:06:23.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.168+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad080b44f0 con 0x7fad08144d60 2024-04-25T00:06:23.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.168+0000 7fad057fa640 1 -- 172.21.15.113:0/106180387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4011c20 con 0x7fad08144d60 2024-04-25T00:06:23.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.168+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081f41c0 con 0x7fad08144d60 2024-04-25T00:06:23.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.169+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad081f3fb0 con 0x7fad08144d60 2024-04-25T00:06:23.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.170+0000 7fad057fa640 1 -- 172.21.15.113:0/106180387 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7facf4018020 con 0x7fad08144d60 2024-04-25T00:06:23.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.170+0000 7fad057fa640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facec06be30 0x7facec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.170+0000 7fad057fa640 1 -- 172.21.15.113:0/106180387 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7facf408aaf0 con 0x7fad08144d60 2024-04-25T00:06:23.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.170+0000 7fad0cfb8640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facec06be30 0x7facec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.171+0000 7fad0cfb8640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facec06be30 0x7facec06e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7facfc034000 tx=0x7facfc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.175+0000 7fad057fa640 1 -- 172.21.15.113:0/106180387 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7facf401f030 con 0x7fad08144d60 2024-04-25T00:06:23.239 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:06:23.241 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:06:23.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.359+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad080691a0 con 0x7fad08144d60 2024-04-25T00:06:23.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.360+0000 7fad057fa640 1 -- 172.21.15.113:0/106180387 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16727 (secure 0 0 0) 0x7fad0815b8a0 con 0x7fad08144d60 2024-04-25T00:06:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:23.362 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":130,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:06:21.691072+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:06:18.255623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:06:19.003747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facec06be30 msgr2=0x7facec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facec06be30 0x7facec06e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7facfc034000 tx=0x7facfc02e000 comp rx=0 tx=0).stop 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 msgr2=0x7fad081e5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad081e5f40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7facf400b3f0 tx=0x7facf400b8b0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 shutdown_connections 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7facec06be30 0x7facec06e2f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad081453c0 0x7fad081e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad08144d60 0x7fad081e5f40 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 --2- 172.21.15.113:0/106180387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad0806e6e0 0x7fad0813e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 >> 172.21.15.113:0/106180387 conn(0x7fad080b1a50 msgr2=0x7fad08143e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:23.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 shutdown_connections 2024-04-25T00:06:23.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.363+0000 7fad0f21c640 1 -- 172.21.15.113:0/106180387 wait complete. 2024-04-25T00:06:23.374 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:06:23.375 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:06:23.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.472+0000 7f7bccf63640 1 Processor -- start 2024-04-25T00:06:23.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.472+0000 7f7bccf63640 1 -- start start 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bccf63640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7bc8144910 0x7f7bc80b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bccf63640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bc80b56b0 0x7f7bc80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bccf63640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc8144cf0 con 0x7f7bc80b5240 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bccf63640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc80b4510 con 0x7f7bc80b56b0 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bccf63640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc80b2f30 con 0x7f7bc8144910 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bc6d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bc80b56b0 0x7f7bc80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bbffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7bc80b5240 0x7f7bc806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47656/0 (socket says 172.21.15.113:47656) 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bbffff640 1 -- 172.21.15.113:0/1213549442 learned_addr learned my addr 172.21.15.113:0/1213549442 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bc659c640 1 --2- 172.21.15.113:0/1213549442 >> v2:172.21.15.178:3300/0 conn(0x7f7bc8144910 0x7f7bc80b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bc6d9d640 1 -- 172.21.15.113:0/1213549442 >> v2:172.21.15.178:3300/0 conn(0x7f7bc8144910 msgr2=0x7f7bc80b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:23.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bc6d9d640 1 --2- 172.21.15.113:0/1213549442 >> v2:172.21.15.178:3300/0 conn(0x7f7bc8144910 0x7f7bc80b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bc6d9d640 1 -- 172.21.15.113:0/1213549442 >> v1:172.21.15.113:6789/0 conn(0x7f7bc80b5240 legacy=0x7f7bc806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bc5d9b640 1 -- 172.21.15.113:0/1213549442 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2828800321 0 0) 0x7f7bc8144cf0 con 0x7f7bc80b5240 2024-04-25T00:06:23.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.473+0000 7f7bc6d9d640 1 -- 172.21.15.113:0/1213549442 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc80b3210 con 0x7f7bc80b56b0 2024-04-25T00:06:23.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.474+0000 7f7bc6d9d640 1 --2- 172.21.15.113:0/1213549442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b56b0 0x7f7bc80cd500 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009d90 tx=0x7f7bc002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5bd4b93c69eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.474+0000 7f7bc5d9b640 1 -- 172.21.15.113:0/1213549442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0007740 con 0x7f7bc80b56b0 2024-04-25T00:06:23.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.474+0000 7f7bc5d9b640 1 -- 172.21.15.113:0/1213549442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc00078e0 con 0x7f7bc80b56b0 2024-04-25T00:06:23.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.474+0000 7f7bc5d9b640 1 -- 172.21.15.113:0/1213549442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0007cb0 con 0x7f7bc80b56b0 2024-04-25T00:06:23.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 -- 172.21.15.113:0/1213549442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b56b0 msgr2=0x7f7bc80cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1213549442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b56b0 0x7f7bc80cd500 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009d90 tx=0x7f7bc002a6a0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 -- 172.21.15.113:0/1213549442 shutdown_connections 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1213549442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b56b0 0x7f7bc80cd500 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1213549442 >> v2:172.21.15.178:3300/0 conn(0x7f7bc8144910 0x7f7bc80b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 -- 172.21.15.113:0/1213549442 >> 172.21.15.113:0/1213549442 conn(0x7f7bc80b1a50 msgr2=0x7f7bc80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 -- 172.21.15.113:0/1213549442 shutdown_connections 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.475+0000 7f7bccf63640 1 -- 172.21.15.113:0/1213549442 wait complete. 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.476+0000 7f7bccf63640 1 Processor -- start 2024-04-25T00:06:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.476+0000 7f7bccf63640 1 -- start start 2024-04-25T00:06:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bccf63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b5240 0x7f7bc813ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bccf63640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bc80b56b0 0x7f7bc815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bccf63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bc8144910 0x7f7bc815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bccf63640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80b4da0 con 0x7f7bc80b5240 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bccf63640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80b37d0 con 0x7f7bc80b56b0 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bccf63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc80b4f20 con 0x7f7bc8144910 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b5240 0x7f7bc813ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b5240 0x7f7bc813ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56010/0 (socket says 172.21.15.113:56010) 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 -- 172.21.15.113:0/1311687622 learned_addr learned my addr 172.21.15.113:0/1311687622 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bbffff640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bc80b56b0 0x7f7bc815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc6d9d640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bc8144910 0x7f7bc815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 -- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bc8144910 msgr2=0x7f7bc815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bc8144910 0x7f7bc815dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 -- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bc80b56b0 msgr2=0x7f7bc815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bc80b56b0 0x7f7bc815d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.477+0000 7f7bc659c640 1 -- 172.21.15.113:0/1311687622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc8139000 con 0x7f7bc80b5240 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.478+0000 7f7bc659c640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b5240 0x7f7bc813ada0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c7f0 tx=0x7f7bb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.478+0000 7f7bbeffd640 1 -- 172.21.15.113:0/1311687622 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8015070 con 0x7f7bc80b5240 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.478+0000 7f7bbeffd640 1 -- 172.21.15.113:0/1311687622 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb8004220 con 0x7f7bc80b5240 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.478+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc81392a0 con 0x7f7bc80b5240 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.478+0000 7f7bbeffd640 1 -- 172.21.15.113:0/1311687622 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8007620 con 0x7f7bc80b5240 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.478+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc8139860 con 0x7f7bc80b5240 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.479+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc8144470 con 0x7f7bc80b5240 2024-04-25T00:06:23.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.479+0000 7f7bbeffd640 1 -- 172.21.15.113:0/1311687622 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bb80079d0 con 0x7f7bc80b5240 2024-04-25T00:06:23.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.480+0000 7f7bbeffd640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ba806be30 0x7f7ba806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.480+0000 7f7bbeffd640 1 -- 172.21.15.113:0/1311687622 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f7bb808b0f0 con 0x7f7bc80b5240 2024-04-25T00:06:23.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.480+0000 7f7bbffff640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ba806be30 0x7f7ba806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.480+0000 7f7bbffff640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ba806be30 0x7f7ba806e2f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8141e70 tx=0x7f7bb0007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.484+0000 7f7bbeffd640 1 -- 172.21.15.113:0/1311687622 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bc8144470 con 0x7f7bc80b5240 2024-04-25T00:06:23.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.662+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bc80691a0 con 0x7f7ba806be30 2024-04-25T00:06:23.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.668+0000 7f7bbeffd640 1 -- 172.21.15.113:0/1311687622 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252415 (secure 0 0 0) 0x7f7bc80691a0 con 0x7f7ba806be30 2024-04-25T00:06:23.671 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:23.672 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:06:23.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ba806be30 msgr2=0x7f7ba806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ba806be30 0x7f7ba806e2f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8141e70 tx=0x7f7bb0007b50 comp rx=0 tx=0).stop 2024-04-25T00:06:23.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b5240 msgr2=0x7f7bc813ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b5240 0x7f7bc813ada0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c7f0 tx=0x7f7bb800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 shutdown_connections 2024-04-25T00:06:23.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ba806be30 0x7f7ba806e2f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bc80b56b0 0x7f7bc815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bc8144910 0x7f7bc815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 --2- 172.21.15.113:0/1311687622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b5240 0x7f7bc813ada0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 >> 172.21.15.113:0/1311687622 conn(0x7f7bc80b1a50 msgr2=0x7f7bc8142e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:23.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 shutdown_connections 2024-04-25T00:06:23.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.671+0000 7f7bccf63640 1 -- 172.21.15.113:0/1311687622 wait complete. 2024-04-25T00:06:23.682 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-25T00:06:23.205475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406164,"kb_used_data":13736,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754568556,"statfs":{"total":773094113280,"available":772678201344,"internally_reserved":0,"allocated":14065664,"data_stored":6762017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.528520"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713189+0000","last_change":"2024-04-25T00:06:20.709882+0000","last_active":"2024-04-25T00:06:20.713189+0000","last_peered":"2024-04-25T00:06:20.713189+0000","last_clean":"2024-04-25T00:06:20.713189+0000","last_became_active":"2024-04-25T00:06:20.708972+0000","last_became_peered":"2024-04-25T00:06:20.708972+0000","last_unstale":"2024-04-25T00:06:20.713189+0000","last_undegraded":"2024-04-25T00:06:20.713189+0000","last_fullsized":"2024-04-25T00:06:20.713189+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:35.862394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.692069+0000","last_change":"2024-04-25T00:06:20.690953+0000","last_active":"2024-04-25T00:06:20.692069+0000","last_peered":"2024-04-25T00:06:20.692069+0000","last_clean":"2024-04-25T00:06:20.692069+0000","last_became_active":"2024-04-25T00:06:20.690681+0000","last_became_peered":"2024-04-25T00:06:20.690681+0000","last_unstale":"2024-04-25T00:06:20.692069+0000","last_undegraded":"2024-04-25T00:06:20.692069+0000","last_fullsized":"2024-04-25T00:06:20.692069+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:05.271453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703412+0000","last_change":"2024-04-25T00:06:20.701691+0000","last_active":"2024-04-25T00:06:20.703412+0000","last_peered":"2024-04-25T00:06:20.703412+0000","last_clean":"2024-04-25T00:06:20.703412+0000","last_became_active":"2024-04-25T00:06:20.701307+0000","last_became_peered":"2024-04-25T00:06:20.701307+0000","last_unstale":"2024-04-25T00:06:20.703412+0000","last_undegraded":"2024-04-25T00:06:20.703412+0000","last_fullsized":"2024-04-25T00:06:20.703412+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.078836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712923+0000","last_change":"2024-04-25T00:06:20.710601+0000","last_active":"2024-04-25T00:06:20.712923+0000","last_peered":"2024-04-25T00:06:20.712923+0000","last_clean":"2024-04-25T00:06:20.712923+0000","last_became_active":"2024-04-25T00:06:20.710186+0000","last_became_peered":"2024-04-25T00:06:20.710186+0000","last_unstale":"2024-04-25T00:06:20.712923+0000","last_undegraded":"2024-04-25T00:06:20.712923+0000","last_fullsized":"2024-04-25T00:06:20.712923+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:03.516736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703574+0000","last_change":"2024-04-25T00:06:20.698390+0000","last_active":"2024-04-25T00:06:20.703574+0000","last_peered":"2024-04-25T00:06:20.703574+0000","last_clean":"2024-04-25T00:06:20.703574+0000","last_became_active":"2024-04-25T00:06:20.697867+0000","last_became_peered":"2024-04-25T00:06:20.697867+0000","last_unstale":"2024-04-25T00:06:20.703574+0000","last_undegraded":"2024-04-25T00:06:20.703574+0000","last_fullsized":"2024-04-25T00:06:20.703574+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:18.863376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707052+0000","last_change":"2024-04-25T00:06:20.705758+0000","last_active":"2024-04-25T00:06:20.707052+0000","last_peered":"2024-04-25T00:06:20.707052+0000","last_clean":"2024-04-25T00:06:20.707052+0000","last_became_active":"2024-04-25T00:06:20.705459+0000","last_became_peered":"2024-04-25T00:06:20.705459+0000","last_unstale":"2024-04-25T00:06:20.707052+0000","last_undegraded":"2024-04-25T00:06:20.707052+0000","last_fullsized":"2024-04-25T00:06:20.707052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:26.589209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702623+0000","last_change":"2024-04-25T00:06:20.695108+0000","last_active":"2024-04-25T00:06:20.702623+0000","last_peered":"2024-04-25T00:06:20.702623+0000","last_clean":"2024-04-25T00:06:20.702623+0000","last_became_active":"2024-04-25T00:06:20.694826+0000","last_became_peered":"2024-04-25T00:06:20.694826+0000","last_unstale":"2024-04-25T00:06:20.702623+0000","last_undegraded":"2024-04-25T00:06:20.702623+0000","last_fullsized":"2024-04-25T00:06:20.702623+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:56.963948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705923+0000","last_change":"2024-04-25T00:06:20.704055+0000","last_active":"2024-04-25T00:06:20.705923+0000","last_peered":"2024-04-25T00:06:20.705923+0000","last_clean":"2024-04-25T00:06:20.705923+0000","last_became_active":"2024-04-25T00:06:20.703761+0000","last_became_peered":"2024-04-25T00:06:20.703761+0000","last_unstale":"2024-04-25T00:06:20.705923+0000","last_undegraded":"2024-04-25T00:06:20.705923+0000","last_fullsized":"2024-04-25T00:06:20.705923+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:50.018793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703442+0000","last_change":"2024-04-25T00:06:20.700557+0000","last_active":"2024-04-25T00:06:20.703442+0000","last_peered":"2024-04-25T00:06:20.703442+0000","last_clean":"2024-04-25T00:06:20.703442+0000","last_became_active":"2024-04-25T00:06:20.700171+0000","last_became_peered":"2024-04-25T00:06:20.700171+0000","last_unstale":"2024-04-25T00:06:20.703442+0000","last_undegraded":"2024-04-25T00:06:20.703442+0000","last_fullsized":"2024-04-25T00:06:20.703442+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:11.307194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.692089+0000","last_change":"2024-04-25T00:06:20.691472+0000","last_active":"2024-04-25T00:06:20.692089+0000","last_peered":"2024-04-25T00:06:20.692089+0000","last_clean":"2024-04-25T00:06:20.692089+0000","last_became_active":"2024-04-25T00:06:20.691202+0000","last_became_peered":"2024-04-25T00:06:20.691202+0000","last_unstale":"2024-04-25T00:06:20.692089+0000","last_undegraded":"2024-04-25T00:06:20.692089+0000","last_fullsized":"2024-04-25T00:06:20.692089+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:11.363491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706411+0000","last_change":"2024-04-25T00:06:20.705753+0000","last_active":"2024-04-25T00:06:20.706411+0000","last_peered":"2024-04-25T00:06:20.706411+0000","last_clean":"2024-04-25T00:06:20.706411+0000","last_became_active":"2024-04-25T00:06:20.705491+0000","last_became_peered":"2024-04-25T00:06:20.705491+0000","last_unstale":"2024-04-25T00:06:20.706411+0000","last_undegraded":"2024-04-25T00:06:20.706411+0000","last_fullsized":"2024-04-25T00:06:20.706411+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:48.614037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703291+0000","last_change":"2024-04-25T00:06:20.700675+0000","last_active":"2024-04-25T00:06:20.703291+0000","last_peered":"2024-04-25T00:06:20.703291+0000","last_clean":"2024-04-25T00:06:20.703291+0000","last_became_active":"2024-04-25T00:06:20.699442+0000","last_became_peered":"2024-04-25T00:06:20.699442+0000","last_unstale":"2024-04-25T00:06:20.703291+0000","last_undegraded":"2024-04-25T00:06:20.703291+0000","last_fullsized":"2024-04-25T00:06:20.703291+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:07.307032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713388+0000","last_change":"2024-04-25T00:06:20.711646+0000","last_active":"2024-04-25T00:06:20.713388+0000","last_peered":"2024-04-25T00:06:20.713388+0000","last_clean":"2024-04-25T00:06:20.713388+0000","last_became_active":"2024-04-25T00:06:20.710449+0000","last_became_peered":"2024-04-25T00:06:20.710449+0000","last_unstale":"2024-04-25T00:06:20.713388+0000","last_undegraded":"2024-04-25T00:06:20.713388+0000","last_fullsized":"2024-04-25T00:06:20.713388+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:20.878055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.708098+0000","last_change":"2024-04-25T00:06:20.707224+0000","last_active":"2024-04-25T00:06:20.708098+0000","last_peered":"2024-04-25T00:06:20.708098+0000","last_clean":"2024-04-25T00:06:20.708098+0000","last_became_active":"2024-04-25T00:06:20.706783+0000","last_became_peered":"2024-04-25T00:06:20.706783+0000","last_unstale":"2024-04-25T00:06:20.708098+0000","last_undegraded":"2024-04-25T00:06:20.708098+0000","last_fullsized":"2024-04-25T00:06:20.708098+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:43.771477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711090+0000","last_change":"2024-04-25T00:06:20.708544+0000","last_active":"2024-04-25T00:06:20.711090+0000","last_peered":"2024-04-25T00:06:20.711090+0000","last_clean":"2024-04-25T00:06:20.711090+0000","last_became_active":"2024-04-25T00:06:20.708282+0000","last_became_peered":"2024-04-25T00:06:20.708282+0000","last_unstale":"2024-04-25T00:06:20.711090+0000","last_undegraded":"2024-04-25T00:06:20.711090+0000","last_fullsized":"2024-04-25T00:06:20.711090+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:25.473078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709515+0000","last_change":"2024-04-25T00:06:20.708499+0000","last_active":"2024-04-25T00:06:20.709515+0000","last_peered":"2024-04-25T00:06:20.709515+0000","last_clean":"2024-04-25T00:06:20.709515+0000","last_became_active":"2024-04-25T00:06:20.708237+0000","last_became_peered":"2024-04-25T00:06:20.708237+0000","last_unstale":"2024-04-25T00:06:20.709515+0000","last_undegraded":"2024-04-25T00:06:20.709515+0000","last_fullsized":"2024-04-25T00:06:20.709515+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:40.303050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.699871+0000","last_change":"2024-04-25T00:06:20.695498+0000","last_active":"2024-04-25T00:06:20.699871+0000","last_peered":"2024-04-25T00:06:20.699871+0000","last_clean":"2024-04-25T00:06:20.699871+0000","last_became_active":"2024-04-25T00:06:20.694332+0000","last_became_peered":"2024-04-25T00:06:20.694332+0000","last_unstale":"2024-04-25T00:06:20.699871+0000","last_undegraded":"2024-04-25T00:06:20.699871+0000","last_fullsized":"2024-04-25T00:06:20.699871+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:58.121035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707336+0000","last_change":"2024-04-25T00:06:20.704253+0000","last_active":"2024-04-25T00:06:20.707336+0000","last_peered":"2024-04-25T00:06:20.707336+0000","last_clean":"2024-04-25T00:06:20.707336+0000","last_became_active":"2024-04-25T00:06:20.703825+0000","last_became_peered":"2024-04-25T00:06:20.703825+0000","last_unstale":"2024-04-25T00:06:20.707336+0000","last_undegraded":"2024-04-25T00:06:20.707336+0000","last_fullsized":"2024-04-25T00:06:20.707336+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:02.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.698398+0000","last_change":"2024-04-25T00:06:20.693498+0000","last_active":"2024-04-25T00:06:20.698398+0000","last_peered":"2024-04-25T00:06:20.698398+0000","last_clean":"2024-04-25T00:06:20.698398+0000","last_became_active":"2024-04-25T00:06:20.693205+0000","last_became_peered":"2024-04-25T00:06:20.693205+0000","last_unstale":"2024-04-25T00:06:20.698398+0000","last_undegraded":"2024-04-25T00:06:20.698398+0000","last_fullsized":"2024-04-25T00:06:20.698398+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:33.384120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707176+0000","last_change":"2024-04-25T00:06:20.703532+0000","last_active":"2024-04-25T00:06:20.707176+0000","last_peered":"2024-04-25T00:06:20.707176+0000","last_clean":"2024-04-25T00:06:20.707176+0000","last_became_active":"2024-04-25T00:06:20.703157+0000","last_became_peered":"2024-04-25T00:06:20.703157+0000","last_unstale":"2024-04-25T00:06:20.707176+0000","last_undegraded":"2024-04-25T00:06:20.707176+0000","last_fullsized":"2024-04-25T00:06:20.707176+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:03.180423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703552+0000","last_change":"2024-04-25T00:06:20.698164+0000","last_active":"2024-04-25T00:06:20.703552+0000","last_peered":"2024-04-25T00:06:20.703552+0000","last_clean":"2024-04-25T00:06:20.703552+0000","last_became_active":"2024-04-25T00:06:20.697501+0000","last_became_peered":"2024-04-25T00:06:20.697501+0000","last_unstale":"2024-04-25T00:06:20.703552+0000","last_undegraded":"2024-04-25T00:06:20.703552+0000","last_fullsized":"2024-04-25T00:06:20.703552+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:40.303050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.698366+0000","last_change":"2024-04-25T00:06:20.694285+0000","last_active":"2024-04-25T00:06:20.698366+0000","last_peered":"2024-04-25T00:06:20.698366+0000","last_clean":"2024-04-25T00:06:20.698366+0000","last_became_active":"2024-04-25T00:06:20.694013+0000","last_became_peered":"2024-04-25T00:06:20.694013+0000","last_unstale":"2024-04-25T00:06:20.698366+0000","last_undegraded":"2024-04-25T00:06:20.698366+0000","last_fullsized":"2024-04-25T00:06:20.698366+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:11.450117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703325+0000","last_change":"2024-04-25T00:06:20.700518+0000","last_active":"2024-04-25T00:06:20.703325+0000","last_peered":"2024-04-25T00:06:20.703325+0000","last_clean":"2024-04-25T00:06:20.703325+0000","last_became_active":"2024-04-25T00:06:20.700240+0000","last_became_peered":"2024-04-25T00:06:20.700240+0000","last_unstale":"2024-04-25T00:06:20.703325+0000","last_undegraded":"2024-04-25T00:06:20.703325+0000","last_fullsized":"2024-04-25T00:06:20.703325+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:57.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695191+0000","last_change":"2024-04-25T00:06:20.692640+0000","last_active":"2024-04-25T00:06:20.695191+0000","last_peered":"2024-04-25T00:06:20.695191+0000","last_clean":"2024-04-25T00:06:20.695191+0000","last_became_active":"2024-04-25T00:06:20.692370+0000","last_became_peered":"2024-04-25T00:06:20.692370+0000","last_unstale":"2024-04-25T00:06:20.695191+0000","last_undegraded":"2024-04-25T00:06:20.695191+0000","last_fullsized":"2024-04-25T00:06:20.695191+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:27.526068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.700801+0000","last_change":"2024-04-25T00:06:20.696510+0000","last_active":"2024-04-25T00:06:20.700801+0000","last_peered":"2024-04-25T00:06:20.700801+0000","last_clean":"2024-04-25T00:06:20.700801+0000","last_became_active":"2024-04-25T00:06:20.695803+0000","last_became_peered":"2024-04-25T00:06:20.695803+0000","last_unstale":"2024-04-25T00:06:20.700801+0000","last_undegraded":"2024-04-25T00:06:20.700801+0000","last_fullsized":"2024-04-25T00:06:20.700801+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:02.494718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712092+0000","last_change":"2024-04-25T00:06:20.711069+0000","last_active":"2024-04-25T00:06:20.712092+0000","last_peered":"2024-04-25T00:06:20.712092+0000","last_clean":"2024-04-25T00:06:20.712092+0000","last_became_active":"2024-04-25T00:06:20.710761+0000","last_became_peered":"2024-04-25T00:06:20.710761+0000","last_unstale":"2024-04-25T00:06:20.712092+0000","last_undegraded":"2024-04-25T00:06:20.712092+0000","last_fullsized":"2024-04-25T00:06:20.712092+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:18.863376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712947+0000","last_change":"2024-04-25T00:06:20.710612+0000","last_active":"2024-04-25T00:06:20.712947+0000","last_peered":"2024-04-25T00:06:20.712947+0000","last_clean":"2024-04-25T00:06:20.712947+0000","last_became_active":"2024-04-25T00:06:20.710333+0000","last_became_peered":"2024-04-25T00:06:20.710333+0000","last_unstale":"2024-04-25T00:06:20.712947+0000","last_undegraded":"2024-04-25T00:06:20.712947+0000","last_fullsized":"2024-04-25T00:06:20.712947+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:02.936119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706829+0000","last_change":"2024-04-25T00:06:20.700894+0000","last_active":"2024-04-25T00:06:20.706829+0000","last_peered":"2024-04-25T00:06:20.706829+0000","last_clean":"2024-04-25T00:06:20.706829+0000","last_became_active":"2024-04-25T00:06:20.700609+0000","last_became_peered":"2024-04-25T00:06:20.700609+0000","last_unstale":"2024-04-25T00:06:20.706829+0000","last_undegraded":"2024-04-25T00:06:20.706829+0000","last_fullsized":"2024-04-25T00:06:20.706829+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:35.278809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712140+0000","last_change":"2024-04-25T00:06:20.709320+0000","last_active":"2024-04-25T00:06:20.712140+0000","last_peered":"2024-04-25T00:06:20.712140+0000","last_clean":"2024-04-25T00:06:20.712140+0000","last_became_active":"2024-04-25T00:06:20.709058+0000","last_became_peered":"2024-04-25T00:06:20.709058+0000","last_unstale":"2024-04-25T00:06:20.712140+0000","last_undegraded":"2024-04-25T00:06:20.712140+0000","last_fullsized":"2024-04-25T00:06:20.712140+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:01.404075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713206+0000","last_change":"2024-04-25T00:06:20.711081+0000","last_active":"2024-04-25T00:06:20.713206+0000","last_peered":"2024-04-25T00:06:20.713206+0000","last_clean":"2024-04-25T00:06:20.713206+0000","last_became_active":"2024-04-25T00:06:20.710280+0000","last_became_peered":"2024-04-25T00:06:20.710280+0000","last_unstale":"2024-04-25T00:06:20.713206+0000","last_undegraded":"2024-04-25T00:06:20.713206+0000","last_fullsized":"2024-04-25T00:06:20.713206+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:48.377125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709656+0000","last_change":"2024-04-25T00:06:20.709185+0000","last_active":"2024-04-25T00:06:20.709656+0000","last_peered":"2024-04-25T00:06:20.709656+0000","last_clean":"2024-04-25T00:06:20.709656+0000","last_became_active":"2024-04-25T00:06:20.708758+0000","last_became_peered":"2024-04-25T00:06:20.708758+0000","last_unstale":"2024-04-25T00:06:20.709656+0000","last_undegraded":"2024-04-25T00:06:20.709656+0000","last_fullsized":"2024-04-25T00:06:20.709656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:38.475784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702446+0000","last_change":"2024-04-25T00:06:20.700855+0000","last_active":"2024-04-25T00:06:20.702446+0000","last_peered":"2024-04-25T00:06:20.702446+0000","last_clean":"2024-04-25T00:06:20.702446+0000","last_became_active":"2024-04-25T00:06:20.700561+0000","last_became_peered":"2024-04-25T00:06:20.700561+0000","last_unstale":"2024-04-25T00:06:20.702446+0000","last_undegraded":"2024-04-25T00:06:20.702446+0000","last_fullsized":"2024-04-25T00:06:20.702446+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:13.149918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705585+0000","last_change":"2024-04-25T00:06:20.700547+0000","last_active":"2024-04-25T00:06:20.705585+0000","last_peered":"2024-04-25T00:06:20.705585+0000","last_clean":"2024-04-25T00:06:20.705585+0000","last_became_active":"2024-04-25T00:06:20.700252+0000","last_became_peered":"2024-04-25T00:06:20.700252+0000","last_unstale":"2024-04-25T00:06:20.705585+0000","last_undegraded":"2024-04-25T00:06:20.705585+0000","last_fullsized":"2024-04-25T00:06:20.705585+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:02.892640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"129'4","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713894+0000","last_change":"2024-04-25T00:06:19.701625+0000","last_active":"2024-04-25T00:06:20.713894+0000","last_peered":"2024-04-25T00:06:20.713894+0000","last_clean":"2024-04-25T00:06:20.713894+0000","last_became_active":"2024-04-25T00:06:19.701325+0000","last_became_peered":"2024-04-25T00:06:19.701325+0000","last_unstale":"2024-04-25T00:06:20.713894+0000","last_undegraded":"2024-04-25T00:06:20.713894+0000","last_fullsized":"2024-04-25T00:06:20.713894+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:15.208488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707939+0000","last_change":"2024-04-25T00:06:20.706028+0000","last_active":"2024-04-25T00:06:20.707939+0000","last_peered":"2024-04-25T00:06:20.707939+0000","last_clean":"2024-04-25T00:06:20.707939+0000","last_became_active":"2024-04-25T00:06:20.705747+0000","last_became_peered":"2024-04-25T00:06:20.705747+0000","last_unstale":"2024-04-25T00:06:20.707939+0000","last_undegraded":"2024-04-25T00:06:20.707939+0000","last_fullsized":"2024-04-25T00:06:20.707939+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:47.692642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709166+0000","last_change":"2024-04-25T00:06:19.693228+0000","last_active":"2024-04-25T00:06:20.709166+0000","last_peered":"2024-04-25T00:06:20.709166+0000","last_clean":"2024-04-25T00:06:20.709166+0000","last_became_active":"2024-04-25T00:06:19.692959+0000","last_became_peered":"2024-04-25T00:06:19.692959+0000","last_unstale":"2024-04-25T00:06:20.709166+0000","last_undegraded":"2024-04-25T00:06:20.709166+0000","last_fullsized":"2024-04-25T00:06:20.709166+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:31.704449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703772+0000","last_change":"2024-04-25T00:06:20.701610+0000","last_active":"2024-04-25T00:06:20.703772+0000","last_peered":"2024-04-25T00:06:20.703772+0000","last_clean":"2024-04-25T00:06:20.703772+0000","last_became_active":"2024-04-25T00:06:20.701101+0000","last_became_peered":"2024-04-25T00:06:20.701101+0000","last_unstale":"2024-04-25T00:06:20.703772+0000","last_undegraded":"2024-04-25T00:06:20.703772+0000","last_fullsized":"2024-04-25T00:06:20.703772+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:55.905435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709050+0000","last_change":"2024-04-25T00:06:19.691146+0000","last_active":"2024-04-25T00:06:20.709050+0000","last_peered":"2024-04-25T00:06:20.709050+0000","last_clean":"2024-04-25T00:06:20.709050+0000","last_became_active":"2024-04-25T00:06:19.690870+0000","last_became_peered":"2024-04-25T00:06:19.690870+0000","last_unstale":"2024-04-25T00:06:20.709050+0000","last_undegraded":"2024-04-25T00:06:20.709050+0000","last_fullsized":"2024-04-25T00:06:20.709050+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:14.697845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.708596+0000","last_change":"2024-04-25T00:06:20.706752+0000","last_active":"2024-04-25T00:06:20.708596+0000","last_peered":"2024-04-25T00:06:20.708596+0000","last_clean":"2024-04-25T00:06:20.708596+0000","last_became_active":"2024-04-25T00:06:20.706343+0000","last_became_peered":"2024-04-25T00:06:20.706343+0000","last_unstale":"2024-04-25T00:06:20.708596+0000","last_undegraded":"2024-04-25T00:06:20.708596+0000","last_fullsized":"2024-04-25T00:06:20.708596+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:26.247824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.700724+0000","last_change":"2024-04-25T00:06:19.690623+0000","last_active":"2024-04-25T00:06:20.700724+0000","last_peered":"2024-04-25T00:06:20.700724+0000","last_clean":"2024-04-25T00:06:20.700724+0000","last_became_active":"2024-04-25T00:06:19.690258+0000","last_became_peered":"2024-04-25T00:06:19.690258+0000","last_unstale":"2024-04-25T00:06:20.700724+0000","last_undegraded":"2024-04-25T00:06:20.700724+0000","last_fullsized":"2024-04-25T00:06:20.700724+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:37.018486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705774+0000","last_change":"2024-04-25T00:06:20.701448+0000","last_active":"2024-04-25T00:06:20.705774+0000","last_peered":"2024-04-25T00:06:20.705774+0000","last_clean":"2024-04-25T00:06:20.705774+0000","last_became_active":"2024-04-25T00:06:20.701147+0000","last_became_peered":"2024-04-25T00:06:20.701147+0000","last_unstale":"2024-04-25T00:06:20.705774+0000","last_undegraded":"2024-04-25T00:06:20.705774+0000","last_fullsized":"2024-04-25T00:06:20.705774+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:32.385455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702156+0000","last_change":"2024-04-25T00:06:19.691740+0000","last_active":"2024-04-25T00:06:20.702156+0000","last_peered":"2024-04-25T00:06:20.702156+0000","last_clean":"2024-04-25T00:06:20.702156+0000","last_became_active":"2024-04-25T00:06:19.691455+0000","last_became_peered":"2024-04-25T00:06:19.691455+0000","last_unstale":"2024-04-25T00:06:20.702156+0000","last_undegraded":"2024-04-25T00:06:20.702156+0000","last_fullsized":"2024-04-25T00:06:20.702156+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:01.929796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.701595+0000","last_change":"2024-04-25T00:06:20.694485+0000","last_active":"2024-04-25T00:06:20.701595+0000","last_peered":"2024-04-25T00:06:20.701595+0000","last_clean":"2024-04-25T00:06:20.701595+0000","last_became_active":"2024-04-25T00:06:20.694131+0000","last_became_peered":"2024-04-25T00:06:20.694131+0000","last_unstale":"2024-04-25T00:06:20.701595+0000","last_undegraded":"2024-04-25T00:06:20.701595+0000","last_fullsized":"2024-04-25T00:06:20.701595+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:29.420293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702745+0000","last_change":"2024-04-25T00:06:19.692230+0000","last_active":"2024-04-25T00:06:20.702745+0000","last_peered":"2024-04-25T00:06:20.702745+0000","last_clean":"2024-04-25T00:06:20.702745+0000","last_became_active":"2024-04-25T00:06:19.691359+0000","last_became_peered":"2024-04-25T00:06:19.691359+0000","last_unstale":"2024-04-25T00:06:20.702745+0000","last_undegraded":"2024-04-25T00:06:20.702745+0000","last_fullsized":"2024-04-25T00:06:20.702745+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:37.075006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705899+0000","last_change":"2024-04-25T00:06:20.703126+0000","last_active":"2024-04-25T00:06:20.705899+0000","last_peered":"2024-04-25T00:06:20.705899+0000","last_clean":"2024-04-25T00:06:20.705899+0000","last_became_active":"2024-04-25T00:06:20.702851+0000","last_became_peered":"2024-04-25T00:06:20.702851+0000","last_unstale":"2024-04-25T00:06:20.705899+0000","last_undegraded":"2024-04-25T00:06:20.705899+0000","last_fullsized":"2024-04-25T00:06:20.705899+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:07.326922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686651+0000","last_change":"2024-04-25T00:06:19.698522+0000","last_active":"2024-04-25T00:06:20.686651+0000","last_peered":"2024-04-25T00:06:20.686651+0000","last_clean":"2024-04-25T00:06:20.686651+0000","last_became_active":"2024-04-25T00:06:19.698106+0000","last_became_peered":"2024-04-25T00:06:19.698106+0000","last_unstale":"2024-04-25T00:06:20.686651+0000","last_undegraded":"2024-04-25T00:06:20.686651+0000","last_fullsized":"2024-04-25T00:06:20.686651+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:12.117273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703336+0000","last_change":"2024-04-25T00:06:20.697604+0000","last_active":"2024-04-25T00:06:20.703336+0000","last_peered":"2024-04-25T00:06:20.703336+0000","last_clean":"2024-04-25T00:06:20.703336+0000","last_became_active":"2024-04-25T00:06:20.697357+0000","last_became_peered":"2024-04-25T00:06:20.697357+0000","last_unstale":"2024-04-25T00:06:20.703336+0000","last_undegraded":"2024-04-25T00:06:20.703336+0000","last_fullsized":"2024-04-25T00:06:20.703336+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:42.725022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686243+0000","last_change":"2024-04-25T00:06:19.693617+0000","last_active":"2024-04-25T00:06:20.686243+0000","last_peered":"2024-04-25T00:06:20.686243+0000","last_clean":"2024-04-25T00:06:20.686243+0000","last_became_active":"2024-04-25T00:06:19.693199+0000","last_became_peered":"2024-04-25T00:06:19.693199+0000","last_unstale":"2024-04-25T00:06:20.686243+0000","last_undegraded":"2024-04-25T00:06:20.686243+0000","last_fullsized":"2024-04-25T00:06:20.686243+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:47.370802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711129+0000","last_change":"2024-04-25T00:06:20.708428+0000","last_active":"2024-04-25T00:06:20.711129+0000","last_peered":"2024-04-25T00:06:20.711129+0000","last_clean":"2024-04-25T00:06:20.711129+0000","last_became_active":"2024-04-25T00:06:20.708138+0000","last_became_peered":"2024-04-25T00:06:20.708138+0000","last_unstale":"2024-04-25T00:06:20.711129+0000","last_undegraded":"2024-04-25T00:06:20.711129+0000","last_fullsized":"2024-04-25T00:06:20.711129+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:22.290210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.696335+0000","last_change":"2024-04-25T00:06:19.691769+0000","last_active":"2024-04-25T00:06:20.696335+0000","last_peered":"2024-04-25T00:06:20.696335+0000","last_clean":"2024-04-25T00:06:20.696335+0000","last_became_active":"2024-04-25T00:06:19.691303+0000","last_became_peered":"2024-04-25T00:06:19.691303+0000","last_unstale":"2024-04-25T00:06:20.696335+0000","last_undegraded":"2024-04-25T00:06:20.696335+0000","last_fullsized":"2024-04-25T00:06:20.696335+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:10.300871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703519+0000","last_change":"2024-04-25T00:06:20.700961+0000","last_active":"2024-04-25T00:06:20.703519+0000","last_peered":"2024-04-25T00:06:20.703519+0000","last_clean":"2024-04-25T00:06:20.703519+0000","last_became_active":"2024-04-25T00:06:20.700521+0000","last_became_peered":"2024-04-25T00:06:20.700521+0000","last_unstale":"2024-04-25T00:06:20.703519+0000","last_undegraded":"2024-04-25T00:06:20.703519+0000","last_fullsized":"2024-04-25T00:06:20.703519+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.078836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.696426+0000","last_change":"2024-04-25T00:06:19.699765+0000","last_active":"2024-04-25T00:06:20.696426+0000","last_peered":"2024-04-25T00:06:20.696426+0000","last_clean":"2024-04-25T00:06:20.696426+0000","last_became_active":"2024-04-25T00:06:19.699073+0000","last_became_peered":"2024-04-25T00:06:19.699073+0000","last_unstale":"2024-04-25T00:06:20.696426+0000","last_undegraded":"2024-04-25T00:06:20.696426+0000","last_fullsized":"2024-04-25T00:06:20.696426+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:22.576364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703752+0000","last_change":"2024-04-25T00:06:20.701386+0000","last_active":"2024-04-25T00:06:20.703752+0000","last_peered":"2024-04-25T00:06:20.703752+0000","last_clean":"2024-04-25T00:06:20.703752+0000","last_became_active":"2024-04-25T00:06:20.700725+0000","last_became_peered":"2024-04-25T00:06:20.700725+0000","last_unstale":"2024-04-25T00:06:20.703752+0000","last_undegraded":"2024-04-25T00:06:20.703752+0000","last_fullsized":"2024-04-25T00:06:20.703752+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:03.845419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"129'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.710399+0000","last_change":"2024-04-25T00:06:19.694114+0000","last_active":"2024-04-25T00:06:20.710399+0000","last_peered":"2024-04-25T00:06:20.710399+0000","last_clean":"2024-04-25T00:06:20.710399+0000","last_became_active":"2024-04-25T00:06:19.693839+0000","last_became_peered":"2024-04-25T00:06:19.693839+0000","last_unstale":"2024-04-25T00:06:20.710399+0000","last_undegraded":"2024-04-25T00:06:20.710399+0000","last_fullsized":"2024-04-25T00:06:20.710399+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:42.765154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703688+0000","last_change":"2024-04-25T00:06:20.701871+0000","last_active":"2024-04-25T00:06:20.703688+0000","last_peered":"2024-04-25T00:06:20.703688+0000","last_clean":"2024-04-25T00:06:20.703688+0000","last_became_active":"2024-04-25T00:06:20.701563+0000","last_became_peered":"2024-04-25T00:06:20.701563+0000","last_unstale":"2024-04-25T00:06:20.703688+0000","last_undegraded":"2024-04-25T00:06:20.703688+0000","last_fullsized":"2024-04-25T00:06:20.703688+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:44.265788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.715678+0000","last_change":"2024-04-25T00:06:19.698406+0000","last_active":"2024-04-25T00:06:20.715678+0000","last_peered":"2024-04-25T00:06:20.715678+0000","last_clean":"2024-04-25T00:06:20.715678+0000","last_became_active":"2024-04-25T00:06:19.698143+0000","last_became_peered":"2024-04-25T00:06:19.698143+0000","last_unstale":"2024-04-25T00:06:20.715678+0000","last_undegraded":"2024-04-25T00:06:20.715678+0000","last_fullsized":"2024-04-25T00:06:20.715678+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:13.517191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711785+0000","last_change":"2024-04-25T00:06:20.708777+0000","last_active":"2024-04-25T00:06:20.711785+0000","last_peered":"2024-04-25T00:06:20.711785+0000","last_clean":"2024-04-25T00:06:20.711785+0000","last_became_active":"2024-04-25T00:06:20.707550+0000","last_became_peered":"2024-04-25T00:06:20.707550+0000","last_unstale":"2024-04-25T00:06:20.711785+0000","last_undegraded":"2024-04-25T00:06:20.711785+0000","last_fullsized":"2024-04-25T00:06:20.711785+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:01.561976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695364+0000","last_change":"2024-04-25T00:06:19.692457+0000","last_active":"2024-04-25T00:06:20.695364+0000","last_peered":"2024-04-25T00:06:20.695364+0000","last_clean":"2024-04-25T00:06:20.695364+0000","last_became_active":"2024-04-25T00:06:19.692182+0000","last_became_peered":"2024-04-25T00:06:19.692182+0000","last_unstale":"2024-04-25T00:06:20.695364+0000","last_undegraded":"2024-04-25T00:06:20.695364+0000","last_fullsized":"2024-04-25T00:06:20.695364+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:52.441703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.704241+0000","last_change":"2024-04-25T00:06:20.703255+0000","last_active":"2024-04-25T00:06:20.704241+0000","last_peered":"2024-04-25T00:06:20.704241+0000","last_clean":"2024-04-25T00:06:20.704241+0000","last_became_active":"2024-04-25T00:06:20.698029+0000","last_became_peered":"2024-04-25T00:06:20.698029+0000","last_unstale":"2024-04-25T00:06:20.704241+0000","last_undegraded":"2024-04-25T00:06:20.704241+0000","last_fullsized":"2024-04-25T00:06:20.704241+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:28.479192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686634+0000","last_change":"2024-04-25T00:06:19.702000+0000","last_active":"2024-04-25T00:06:20.686634+0000","last_peered":"2024-04-25T00:06:20.686634+0000","last_clean":"2024-04-25T00:06:20.686634+0000","last_became_active":"2024-04-25T00:06:19.701423+0000","last_became_peered":"2024-04-25T00:06:19.701423+0000","last_unstale":"2024-04-25T00:06:20.686634+0000","last_undegraded":"2024-04-25T00:06:20.686634+0000","last_fullsized":"2024-04-25T00:06:20.686634+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:19.871732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703343+0000","last_change":"2024-04-25T00:06:20.701068+0000","last_active":"2024-04-25T00:06:20.703343+0000","last_peered":"2024-04-25T00:06:20.703343+0000","last_clean":"2024-04-25T00:06:20.703343+0000","last_became_active":"2024-04-25T00:06:20.700797+0000","last_became_peered":"2024-04-25T00:06:20.700797+0000","last_unstale":"2024-04-25T00:06:20.703343+0000","last_undegraded":"2024-04-25T00:06:20.703343+0000","last_fullsized":"2024-04-25T00:06:20.703343+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:35.278809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"129'8","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.715266+0000","last_change":"2024-04-25T00:06:19.696059+0000","last_active":"2024-04-25T00:06:20.715266+0000","last_peered":"2024-04-25T00:06:20.715266+0000","last_clean":"2024-04-25T00:06:20.715266+0000","last_became_active":"2024-04-25T00:06:19.695569+0000","last_became_peered":"2024-04-25T00:06:19.695569+0000","last_unstale":"2024-04-25T00:06:20.715266+0000","last_undegraded":"2024-04-25T00:06:20.715266+0000","last_fullsized":"2024-04-25T00:06:20.715266+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:31.704449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709157+0000","last_change":"2024-04-25T00:06:20.708688+0000","last_active":"2024-04-25T00:06:20.709157+0000","last_peered":"2024-04-25T00:06:20.709157+0000","last_clean":"2024-04-25T00:06:20.709157+0000","last_became_active":"2024-04-25T00:06:20.708415+0000","last_became_peered":"2024-04-25T00:06:20.708415+0000","last_unstale":"2024-04-25T00:06:20.709157+0000","last_undegraded":"2024-04-25T00:06:20.709157+0000","last_fullsized":"2024-04-25T00:06:20.709157+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:13.123596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709574+0000","last_change":"2024-04-25T00:06:19.698440+0000","last_active":"2024-04-25T00:06:20.709574+0000","last_peered":"2024-04-25T00:06:20.709574+0000","last_clean":"2024-04-25T00:06:20.709574+0000","last_became_active":"2024-04-25T00:06:19.698150+0000","last_became_peered":"2024-04-25T00:06:19.698150+0000","last_unstale":"2024-04-25T00:06:20.709574+0000","last_undegraded":"2024-04-25T00:06:20.709574+0000","last_fullsized":"2024-04-25T00:06:20.709574+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:56.297486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703498+0000","last_change":"2024-04-25T00:06:20.700160+0000","last_active":"2024-04-25T00:06:20.703498+0000","last_peered":"2024-04-25T00:06:20.703498+0000","last_clean":"2024-04-25T00:06:20.703498+0000","last_became_active":"2024-04-25T00:06:20.699859+0000","last_became_peered":"2024-04-25T00:06:20.699859+0000","last_unstale":"2024-04-25T00:06:20.703498+0000","last_undegraded":"2024-04-25T00:06:20.703498+0000","last_fullsized":"2024-04-25T00:06:20.703498+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:07.307032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706213+0000","last_change":"2024-04-25T00:06:19.694018+0000","last_active":"2024-04-25T00:06:20.706213+0000","last_peered":"2024-04-25T00:06:20.706213+0000","last_clean":"2024-04-25T00:06:20.706213+0000","last_became_active":"2024-04-25T00:06:19.693621+0000","last_became_peered":"2024-04-25T00:06:19.693621+0000","last_unstale":"2024-04-25T00:06:20.706213+0000","last_undegraded":"2024-04-25T00:06:20.706213+0000","last_fullsized":"2024-04-25T00:06:20.706213+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.869942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709959+0000","last_change":"2024-04-25T00:06:20.706848+0000","last_active":"2024-04-25T00:06:20.709959+0000","last_peered":"2024-04-25T00:06:20.709959+0000","last_clean":"2024-04-25T00:06:20.709959+0000","last_became_active":"2024-04-25T00:06:20.706517+0000","last_became_peered":"2024-04-25T00:06:20.706517+0000","last_unstale":"2024-04-25T00:06:20.709959+0000","last_undegraded":"2024-04-25T00:06:20.709959+0000","last_fullsized":"2024-04-25T00:06:20.709959+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:16.214811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.710208+0000","last_change":"2024-04-25T00:06:19.692349+0000","last_active":"2024-04-25T00:06:20.710208+0000","last_peered":"2024-04-25T00:06:20.710208+0000","last_clean":"2024-04-25T00:06:20.710208+0000","last_became_active":"2024-04-25T00:06:19.692069+0000","last_became_peered":"2024-04-25T00:06:19.692069+0000","last_unstale":"2024-04-25T00:06:20.710208+0000","last_undegraded":"2024-04-25T00:06:20.710208+0000","last_fullsized":"2024-04-25T00:06:20.710208+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:46.686319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703193+0000","last_change":"2024-04-25T00:06:20.697691+0000","last_active":"2024-04-25T00:06:20.703193+0000","last_peered":"2024-04-25T00:06:20.703193+0000","last_clean":"2024-04-25T00:06:20.703193+0000","last_became_active":"2024-04-25T00:06:20.697456+0000","last_became_peered":"2024-04-25T00:06:20.697456+0000","last_unstale":"2024-04-25T00:06:20.703193+0000","last_undegraded":"2024-04-25T00:06:20.703193+0000","last_fullsized":"2024-04-25T00:06:20.703193+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:44.487735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.691922+0000","last_change":"2024-04-25T00:06:19.692518+0000","last_active":"2024-04-25T00:06:20.691922+0000","last_peered":"2024-04-25T00:06:20.691922+0000","last_clean":"2024-04-25T00:06:20.691922+0000","last_became_active":"2024-04-25T00:06:19.692247+0000","last_became_peered":"2024-04-25T00:06:19.692247+0000","last_unstale":"2024-04-25T00:06:20.691922+0000","last_undegraded":"2024-04-25T00:06:20.691922+0000","last_fullsized":"2024-04-25T00:06:20.691922+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:13.517191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711359+0000","last_change":"2024-04-25T00:06:20.705820+0000","last_active":"2024-04-25T00:06:20.711359+0000","last_peered":"2024-04-25T00:06:20.711359+0000","last_clean":"2024-04-25T00:06:20.711359+0000","last_became_active":"2024-04-25T00:06:20.705130+0000","last_became_peered":"2024-04-25T00:06:20.705130+0000","last_unstale":"2024-04-25T00:06:20.711359+0000","last_undegraded":"2024-04-25T00:06:20.711359+0000","last_fullsized":"2024-04-25T00:06:20.711359+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:34.416408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713544+0000","last_change":"2024-04-25T00:06:19.690058+0000","last_active":"2024-04-25T00:06:20.713544+0000","last_peered":"2024-04-25T00:06:20.713544+0000","last_clean":"2024-04-25T00:06:20.713544+0000","last_became_active":"2024-04-25T00:06:19.689695+0000","last_became_peered":"2024-04-25T00:06:19.689695+0000","last_unstale":"2024-04-25T00:06:20.713544+0000","last_undegraded":"2024-04-25T00:06:20.713544+0000","last_fullsized":"2024-04-25T00:06:20.713544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:18.453085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.701615+0000","last_change":"2024-04-25T00:06:20.695256+0000","last_active":"2024-04-25T00:06:20.701615+0000","last_peered":"2024-04-25T00:06:20.701615+0000","last_clean":"2024-04-25T00:06:20.701615+0000","last_became_active":"2024-04-25T00:06:20.694836+0000","last_became_peered":"2024-04-25T00:06:20.694836+0000","last_unstale":"2024-04-25T00:06:20.701615+0000","last_undegraded":"2024-04-25T00:06:20.701615+0000","last_fullsized":"2024-04-25T00:06:20.701615+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:15.164079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.697191+0000","last_change":"2024-04-25T00:06:19.699252+0000","last_active":"2024-04-25T00:06:20.697191+0000","last_peered":"2024-04-25T00:06:20.697191+0000","last_clean":"2024-04-25T00:06:20.697191+0000","last_became_active":"2024-04-25T00:06:19.698811+0000","last_became_peered":"2024-04-25T00:06:19.698811+0000","last_unstale":"2024-04-25T00:06:20.697191+0000","last_undegraded":"2024-04-25T00:06:20.697191+0000","last_fullsized":"2024-04-25T00:06:20.697191+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:38.079026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703474+0000","last_change":"2024-04-25T00:06:20.694392+0000","last_active":"2024-04-25T00:06:20.703474+0000","last_peered":"2024-04-25T00:06:20.703474+0000","last_clean":"2024-04-25T00:06:20.703474+0000","last_became_active":"2024-04-25T00:06:20.694114+0000","last_became_peered":"2024-04-25T00:06:20.694114+0000","last_unstale":"2024-04-25T00:06:20.703474+0000","last_undegraded":"2024-04-25T00:06:20.703474+0000","last_fullsized":"2024-04-25T00:06:20.703474+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:28.942453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695092+0000","last_change":"2024-04-25T00:06:19.695574+0000","last_active":"2024-04-25T00:06:20.695092+0000","last_peered":"2024-04-25T00:06:20.695092+0000","last_clean":"2024-04-25T00:06:20.695092+0000","last_became_active":"2024-04-25T00:06:19.695299+0000","last_became_peered":"2024-04-25T00:06:19.695299+0000","last_unstale":"2024-04-25T00:06:20.695092+0000","last_undegraded":"2024-04-25T00:06:20.695092+0000","last_fullsized":"2024-04-25T00:06:20.695092+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:39.758070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709636+0000","last_change":"2024-04-25T00:06:20.709029+0000","last_active":"2024-04-25T00:06:20.709636+0000","last_peered":"2024-04-25T00:06:20.709636+0000","last_clean":"2024-04-25T00:06:20.709636+0000","last_became_active":"2024-04-25T00:06:20.708416+0000","last_became_peered":"2024-04-25T00:06:20.708416+0000","last_unstale":"2024-04-25T00:06:20.709636+0000","last_undegraded":"2024-04-25T00:06:20.709636+0000","last_fullsized":"2024-04-25T00:06:20.709636+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:44.943130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686672+0000","last_change":"2024-04-25T00:06:19.693790+0000","last_active":"2024-04-25T00:06:20.686672+0000","last_peered":"2024-04-25T00:06:20.686672+0000","last_clean":"2024-04-25T00:06:20.686672+0000","last_became_active":"2024-04-25T00:06:19.693496+0000","last_became_peered":"2024-04-25T00:06:19.693496+0000","last_unstale":"2024-04-25T00:06:20.686672+0000","last_undegraded":"2024-04-25T00:06:20.686672+0000","last_fullsized":"2024-04-25T00:06:20.686672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:32.957137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.701485+0000","last_change":"2024-04-25T00:06:20.694136+0000","last_active":"2024-04-25T00:06:20.701485+0000","last_peered":"2024-04-25T00:06:20.701485+0000","last_clean":"2024-04-25T00:06:20.701485+0000","last_became_active":"2024-04-25T00:06:20.693829+0000","last_became_peered":"2024-04-25T00:06:20.693829+0000","last_unstale":"2024-04-25T00:06:20.701485+0000","last_undegraded":"2024-04-25T00:06:20.701485+0000","last_fullsized":"2024-04-25T00:06:20.701485+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:31.904364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.704939+0000","last_change":"2024-04-25T00:06:19.693661+0000","last_active":"2024-04-25T00:06:20.704939+0000","last_peered":"2024-04-25T00:06:20.704939+0000","last_clean":"2024-04-25T00:06:20.704939+0000","last_became_active":"2024-04-25T00:06:19.693378+0000","last_became_peered":"2024-04-25T00:06:19.693378+0000","last_unstale":"2024-04-25T00:06:20.704939+0000","last_undegraded":"2024-04-25T00:06:20.704939+0000","last_fullsized":"2024-04-25T00:06:20.704939+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:18.689971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713257+0000","last_change":"2024-04-25T00:06:20.711317+0000","last_active":"2024-04-25T00:06:20.713257+0000","last_peered":"2024-04-25T00:06:20.713257+0000","last_clean":"2024-04-25T00:06:20.713257+0000","last_became_active":"2024-04-25T00:06:20.709611+0000","last_became_peered":"2024-04-25T00:06:20.709611+0000","last_unstale":"2024-04-25T00:06:20.713257+0000","last_undegraded":"2024-04-25T00:06:20.713257+0000","last_fullsized":"2024-04-25T00:06:20.713257+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:31.101554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.697003+0000","last_change":"2024-04-25T00:06:19.698904+0000","last_active":"2024-04-25T00:06:20.697003+0000","last_peered":"2024-04-25T00:06:20.697003+0000","last_clean":"2024-04-25T00:06:20.697003+0000","last_became_active":"2024-04-25T00:06:19.698584+0000","last_became_peered":"2024-04-25T00:06:19.698584+0000","last_unstale":"2024-04-25T00:06:20.697003+0000","last_undegraded":"2024-04-25T00:06:20.697003+0000","last_fullsized":"2024-04-25T00:06:20.697003+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:26.226242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.700847+0000","last_change":"2024-04-25T00:06:20.695880+0000","last_active":"2024-04-25T00:06:20.700847+0000","last_peered":"2024-04-25T00:06:20.700847+0000","last_clean":"2024-04-25T00:06:20.700847+0000","last_became_active":"2024-04-25T00:06:20.695607+0000","last_became_peered":"2024-04-25T00:06:20.695607+0000","last_unstale":"2024-04-25T00:06:20.700847+0000","last_undegraded":"2024-04-25T00:06:20.700847+0000","last_fullsized":"2024-04-25T00:06:20.700847+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:06.747980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686319+0000","last_change":"2024-04-25T00:06:19.693205+0000","last_active":"2024-04-25T00:06:20.686319+0000","last_peered":"2024-04-25T00:06:20.686319+0000","last_clean":"2024-04-25T00:06:20.686319+0000","last_became_active":"2024-04-25T00:06:19.692924+0000","last_became_peered":"2024-04-25T00:06:19.692924+0000","last_unstale":"2024-04-25T00:06:20.686319+0000","last_undegraded":"2024-04-25T00:06:20.686319+0000","last_fullsized":"2024-04-25T00:06:20.686319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:30.095231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.708156+0000","last_change":"2024-04-25T00:06:20.707624+0000","last_active":"2024-04-25T00:06:20.708156+0000","last_peered":"2024-04-25T00:06:20.708156+0000","last_clean":"2024-04-25T00:06:20.708156+0000","last_became_active":"2024-04-25T00:06:20.707351+0000","last_became_peered":"2024-04-25T00:06:20.707351+0000","last_unstale":"2024-04-25T00:06:20.708156+0000","last_undegraded":"2024-04-25T00:06:20.708156+0000","last_fullsized":"2024-04-25T00:06:20.708156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:59.322666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.690219+0000","last_change":"2024-04-25T00:06:19.691478+0000","last_active":"2024-04-25T00:06:20.690219+0000","last_peered":"2024-04-25T00:06:20.690219+0000","last_clean":"2024-04-25T00:06:20.690219+0000","last_became_active":"2024-04-25T00:06:19.690432+0000","last_became_peered":"2024-04-25T00:06:19.690432+0000","last_unstale":"2024-04-25T00:06:20.690219+0000","last_undegraded":"2024-04-25T00:06:20.690219+0000","last_fullsized":"2024-04-25T00:06:20.690219+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:39.110465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706944+0000","last_change":"2024-04-25T00:06:20.702775+0000","last_active":"2024-04-25T00:06:20.706944+0000","last_peered":"2024-04-25T00:06:20.706944+0000","last_clean":"2024-04-25T00:06:20.706944+0000","last_became_active":"2024-04-25T00:06:20.702503+0000","last_became_peered":"2024-04-25T00:06:20.702503+0000","last_unstale":"2024-04-25T00:06:20.706944+0000","last_undegraded":"2024-04-25T00:06:20.706944+0000","last_fullsized":"2024-04-25T00:06:20.706944+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:51.495938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686277+0000","last_change":"2024-04-25T00:06:19.701737+0000","last_active":"2024-04-25T00:06:20.686277+0000","last_peered":"2024-04-25T00:06:20.686277+0000","last_clean":"2024-04-25T00:06:20.686277+0000","last_became_active":"2024-04-25T00:06:19.701467+0000","last_became_peered":"2024-04-25T00:06:19.701467+0000","last_unstale":"2024-04-25T00:06:20.686277+0000","last_undegraded":"2024-04-25T00:06:20.686277+0000","last_fullsized":"2024-04-25T00:06:20.686277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:34.856071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711486+0000","last_change":"2024-04-25T00:06:20.710960+0000","last_active":"2024-04-25T00:06:20.711486+0000","last_peered":"2024-04-25T00:06:20.711486+0000","last_clean":"2024-04-25T00:06:20.711486+0000","last_became_active":"2024-04-25T00:06:20.710670+0000","last_became_peered":"2024-04-25T00:06:20.710670+0000","last_unstale":"2024-04-25T00:06:20.711486+0000","last_undegraded":"2024-04-25T00:06:20.711486+0000","last_fullsized":"2024-04-25T00:06:20.711486+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:07.326922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.690788+0000","last_change":"2024-04-25T00:06:19.700988+0000","last_active":"2024-04-25T00:06:20.690788+0000","last_peered":"2024-04-25T00:06:20.690788+0000","last_clean":"2024-04-25T00:06:20.690788+0000","last_became_active":"2024-04-25T00:06:19.700681+0000","last_became_peered":"2024-04-25T00:06:19.700681+0000","last_unstale":"2024-04-25T00:06:20.690788+0000","last_undegraded":"2024-04-25T00:06:20.690788+0000","last_fullsized":"2024-04-25T00:06:20.690788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:02.174100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709494+0000","last_change":"2024-04-25T00:06:20.705602+0000","last_active":"2024-04-25T00:06:20.709494+0000","last_peered":"2024-04-25T00:06:20.709494+0000","last_clean":"2024-04-25T00:06:20.709494+0000","last_became_active":"2024-04-25T00:06:20.704784+0000","last_became_peered":"2024-04-25T00:06:20.704784+0000","last_unstale":"2024-04-25T00:06:20.709494+0000","last_undegraded":"2024-04-25T00:06:20.709494+0000","last_fullsized":"2024-04-25T00:06:20.709494+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:27.232565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695264+0000","last_change":"2024-04-25T00:06:19.700725+0000","last_active":"2024-04-25T00:06:20.695264+0000","last_peered":"2024-04-25T00:06:20.695264+0000","last_clean":"2024-04-25T00:06:20.695264+0000","last_became_active":"2024-04-25T00:06:19.700392+0000","last_became_peered":"2024-04-25T00:06:19.700392+0000","last_unstale":"2024-04-25T00:06:20.695264+0000","last_undegraded":"2024-04-25T00:06:20.695264+0000","last_fullsized":"2024-04-25T00:06:20.695264+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:15.030428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703315+0000","last_change":"2024-04-25T00:06:20.695073+0000","last_active":"2024-04-25T00:06:20.703315+0000","last_peered":"2024-04-25T00:06:20.703315+0000","last_clean":"2024-04-25T00:06:20.703315+0000","last_became_active":"2024-04-25T00:06:20.694794+0000","last_became_peered":"2024-04-25T00:06:20.694794+0000","last_unstale":"2024-04-25T00:06:20.703315+0000","last_undegraded":"2024-04-25T00:06:20.703315+0000","last_fullsized":"2024-04-25T00:06:20.703315+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:23.528876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686831+0000","last_change":"2024-04-25T00:06:19.690208+0000","last_active":"2024-04-25T00:06:20.686831+0000","last_peered":"2024-04-25T00:06:20.686831+0000","last_clean":"2024-04-25T00:06:20.686831+0000","last_became_active":"2024-04-25T00:06:19.689873+0000","last_became_peered":"2024-04-25T00:06:19.689873+0000","last_unstale":"2024-04-25T00:06:20.686831+0000","last_undegraded":"2024-04-25T00:06:20.686831+0000","last_fullsized":"2024-04-25T00:06:20.686831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:59.464203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709455+0000","last_change":"2024-04-25T00:06:20.708808+0000","last_active":"2024-04-25T00:06:20.709455+0000","last_peered":"2024-04-25T00:06:20.709455+0000","last_clean":"2024-04-25T00:06:20.709455+0000","last_became_active":"2024-04-25T00:06:20.708526+0000","last_became_peered":"2024-04-25T00:06:20.708526+0000","last_unstale":"2024-04-25T00:06:20.709455+0000","last_undegraded":"2024-04-25T00:06:20.709455+0000","last_fullsized":"2024-04-25T00:06:20.709455+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:12.282006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713983+0000","last_change":"2024-04-25T00:06:19.701735+0000","last_active":"2024-04-25T00:06:20.713983+0000","last_peered":"2024-04-25T00:06:20.713983+0000","last_clean":"2024-04-25T00:06:20.713983+0000","last_became_active":"2024-04-25T00:06:19.701418+0000","last_became_peered":"2024-04-25T00:06:19.701418+0000","last_unstale":"2024-04-25T00:06:20.713983+0000","last_undegraded":"2024-04-25T00:06:20.713983+0000","last_fullsized":"2024-04-25T00:06:20.713983+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:01.669322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.691194+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:06:20.691194+0000","last_peered":"2024-04-25T00:06:20.691194+0000","last_clean":"2024-04-25T00:06:20.691194+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:06:20.691194+0000","last_undegraded":"2024-04-25T00:06:20.691194+0000","last_fullsized":"2024-04-25T00:06:20.691194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59724,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312116,"statfs":{"total":96636764160,"available":96575606784,"internally_reserved":0,"allocated":2228224,"data_stored":1292336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":1622016,"data_stored":701330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50788,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321052,"statfs":{"total":96636764160,"available":96584757248,"internally_reserved":0,"allocated":1597440,"data_stored":692943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59120,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312720,"statfs":{"total":96636764160,"available":96576225280,"internally_reserved":0,"allocated":1609728,"data_stored":699718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46632,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325208,"statfs":{"total":96636764160,"available":96589012992,"internally_reserved":0,"allocated":1597440,"data_stored":692943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46632,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325208,"statfs":{"total":96636764160,"available":96589012992,"internally_reserved":0,"allocated":1597440,"data_stored":692943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54960,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316880,"statfs":{"total":96636764160,"available":96580485120,"internally_reserved":0,"allocated":1609728,"data_stored":699718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29176,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342664,"statfs":{"total":96636764160,"available":96606887936,"internally_reserved":0,"allocated":2203648,"data_stored":1290086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:06:23.687 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:06:23.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.821+0000 7feb8d858640 1 Processor -- start 2024-04-25T00:06:23.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.821+0000 7feb8d858640 1 -- start start 2024-04-25T00:06:23.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.822+0000 7feb8d858640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feb8815e320 0x7feb880d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.822+0000 7feb8d858640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.822+0000 7feb8d858640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb880d3de0 con 0x7feb880d2290 2024-04-25T00:06:23.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.822+0000 7feb8d858640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7feb880d14d0 con 0x7feb8815e320 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.822+0000 7feb8d858640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feb880d3410 con 0x7feb880d0b00 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb877fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb86ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feb8815e320 0x7feb880d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb877fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:54412/0 (socket says 172.21.15.113:54412) 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb877fe640 1 -- 172.21.15.113:0/1084856674 learned_addr learned my addr 172.21.15.113:0/1084856674 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb867fc640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7feb880d2290 0x7feb880b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:40950/0 (socket says 172.21.15.113:40950) 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb86ffd640 1 -- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 msgr2=0x7feb880d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb86ffd640 1 --2- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb86ffd640 1 -- 172.21.15.113:0/1084856674 >> v1:172.21.15.178:6789/0 conn(0x7feb880d2290 legacy=0x7feb880b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb86ffd640 1 -- 172.21.15.113:0/1084856674 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb880d2800 con 0x7feb8815e320 2024-04-25T00:06:23.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb85ffb640 1 -- 172.21.15.113:0/1084856674 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2490172653 0 0) 0x7feb880d3de0 con 0x7feb880d2290 2024-04-25T00:06:23.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.823+0000 7feb877fe640 1 --2- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:23.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.824+0000 7feb86ffd640 1 --2- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb8815e320 0x7feb880d38a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0097a0 tx=0x7feb7c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698c1384307be4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.824+0000 7feb85ffb640 1 -- 172.21.15.113:0/1084856674 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c037070 con 0x7feb8815e320 2024-04-25T00:06:23.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.824+0000 7feb85ffb640 1 -- 172.21.15.113:0/1084856674 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c032440 con 0x7feb8815e320 2024-04-25T00:06:23.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.824+0000 7feb85ffb640 1 -- 172.21.15.113:0/1084856674 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c032810 con 0x7feb8815e320 2024-04-25T00:06:23.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 -- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb8815e320 msgr2=0x7feb880d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 --2- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb8815e320 0x7feb880d38a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0097a0 tx=0x7feb7c02a210 comp rx=0 tx=0).stop 2024-04-25T00:06:23.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 -- 172.21.15.113:0/1084856674 shutdown_connections 2024-04-25T00:06:23.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 --2- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 --2- 172.21.15.113:0/1084856674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb8815e320 0x7feb880d38a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 -- 172.21.15.113:0/1084856674 >> 172.21.15.113:0/1084856674 conn(0x7feb880b5c20 msgr2=0x7feb880b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:23.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 -- 172.21.15.113:0/1084856674 shutdown_connections 2024-04-25T00:06:23.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.825+0000 7feb8d858640 1 -- 172.21.15.113:0/1084856674 wait complete. 2024-04-25T00:06:23.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.826+0000 7feb8d858640 1 Processor -- start 2024-04-25T00:06:23.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.826+0000 7feb8d858640 1 -- start start 2024-04-25T00:06:23.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb880d2290 0x7feb881f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb8815e320 0x7feb880cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feb880d30a0 con 0x7feb8815e320 2024-04-25T00:06:23.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feb880d2dc0 con 0x7feb880d2290 2024-04-25T00:06:23.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feb8815e700 con 0x7feb880d0b00 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb8815e320 0x7feb880cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb8815e320 0x7feb880cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56014/0 (socket says 172.21.15.113:56014) 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 -- 172.21.15.113:0/2144328659 learned_addr learned my addr 172.21.15.113:0/2144328659 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb86ffd640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb867fc640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb880d2290 0x7feb881f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 -- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 msgr2=0x7feb880c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 -- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb880d2290 msgr2=0x7feb881f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb880d2290 0x7feb881f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 -- 172.21.15.113:0/2144328659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb881410b0 con 0x7feb8815e320 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb877fe640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb8815e320 0x7feb880cd520 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7feb80009f10 tx=0x7feb8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb6ffff640 1 -- 172.21.15.113:0/2144328659 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8000cc40 con 0x7feb8815e320 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb6ffff640 1 -- 172.21.15.113:0/2144328659 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8000cde0 con 0x7feb8815e320 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb88141390 con 0x7feb8815e320 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb6ffff640 1 -- 172.21.15.113:0/2144328659 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80003bb0 con 0x7feb8815e320 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.827+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb88141950 con 0x7feb8815e320 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.829+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb880691a0 con 0x7feb8815e320 2024-04-25T00:06:23.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.829+0000 7feb6ffff640 1 -- 172.21.15.113:0/2144328659 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb8001b030 con 0x7feb8815e320 2024-04-25T00:06:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.829+0000 7feb6ffff640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb7406be30 0x7feb7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.829+0000 7feb6ffff640 1 -- 172.21.15.113:0/2144328659 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7feb8008b0e0 con 0x7feb8815e320 2024-04-25T00:06:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.829+0000 7feb86ffd640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb7406be30 0x7feb7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.830+0000 7feb86ffd640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb7406be30 0x7feb7406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feb7c034000 tx=0x7feb7c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:23.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:23.834+0000 7feb6ffff640 1 -- 172.21.15.113:0/2144328659 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb80022020 con 0x7feb8815e320 2024-04-25T00:06:24.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.005+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb880b3c30 con 0x7feb7406be30 2024-04-25T00:06:24.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.011+0000 7feb6ffff640 1 -- 172.21.15.113:0/2144328659 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252415 (secure 0 0 0) 0x7feb880b3c30 con 0x7feb7406be30 2024-04-25T00:06:24.013 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:24.015 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:06:24.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb7406be30 msgr2=0x7feb7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb7406be30 0x7feb7406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feb7c034000 tx=0x7feb7c02e000 comp rx=0 tx=0).stop 2024-04-25T00:06:24.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb8815e320 msgr2=0x7feb880cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb8815e320 0x7feb880cd520 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7feb80009f10 tx=0x7feb8000c490 comp rx=0 tx=0).stop 2024-04-25T00:06:24.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 shutdown_connections 2024-04-25T00:06:24.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb7406be30 0x7feb7406e2f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb880d2290 0x7feb881f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb8815e320 0x7feb880cd520 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 --2- 172.21.15.113:0/2144328659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb880d0b00 0x7feb880c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 >> 172.21.15.113:0/2144328659 conn(0x7feb880b5c20 msgr2=0x7feb880b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:24.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 shutdown_connections 2024-04-25T00:06:24.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.014+0000 7feb8d858640 1 -- 172.21.15.113:0/2144328659 wait complete. 2024-04-25T00:06:24.026 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-25T00:06:23.205475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406164,"kb_used_data":13736,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754568556,"statfs":{"total":773094113280,"available":772678201344,"internally_reserved":0,"allocated":14065664,"data_stored":6762017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.528520"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713189+0000","last_change":"2024-04-25T00:06:20.709882+0000","last_active":"2024-04-25T00:06:20.713189+0000","last_peered":"2024-04-25T00:06:20.713189+0000","last_clean":"2024-04-25T00:06:20.713189+0000","last_became_active":"2024-04-25T00:06:20.708972+0000","last_became_peered":"2024-04-25T00:06:20.708972+0000","last_unstale":"2024-04-25T00:06:20.713189+0000","last_undegraded":"2024-04-25T00:06:20.713189+0000","last_fullsized":"2024-04-25T00:06:20.713189+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:35.862394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.692069+0000","last_change":"2024-04-25T00:06:20.690953+0000","last_active":"2024-04-25T00:06:20.692069+0000","last_peered":"2024-04-25T00:06:20.692069+0000","last_clean":"2024-04-25T00:06:20.692069+0000","last_became_active":"2024-04-25T00:06:20.690681+0000","last_became_peered":"2024-04-25T00:06:20.690681+0000","last_unstale":"2024-04-25T00:06:20.692069+0000","last_undegraded":"2024-04-25T00:06:20.692069+0000","last_fullsized":"2024-04-25T00:06:20.692069+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:05.271453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703412+0000","last_change":"2024-04-25T00:06:20.701691+0000","last_active":"2024-04-25T00:06:20.703412+0000","last_peered":"2024-04-25T00:06:20.703412+0000","last_clean":"2024-04-25T00:06:20.703412+0000","last_became_active":"2024-04-25T00:06:20.701307+0000","last_became_peered":"2024-04-25T00:06:20.701307+0000","last_unstale":"2024-04-25T00:06:20.703412+0000","last_undegraded":"2024-04-25T00:06:20.703412+0000","last_fullsized":"2024-04-25T00:06:20.703412+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.078836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712923+0000","last_change":"2024-04-25T00:06:20.710601+0000","last_active":"2024-04-25T00:06:20.712923+0000","last_peered":"2024-04-25T00:06:20.712923+0000","last_clean":"2024-04-25T00:06:20.712923+0000","last_became_active":"2024-04-25T00:06:20.710186+0000","last_became_peered":"2024-04-25T00:06:20.710186+0000","last_unstale":"2024-04-25T00:06:20.712923+0000","last_undegraded":"2024-04-25T00:06:20.712923+0000","last_fullsized":"2024-04-25T00:06:20.712923+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:03.516736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703574+0000","last_change":"2024-04-25T00:06:20.698390+0000","last_active":"2024-04-25T00:06:20.703574+0000","last_peered":"2024-04-25T00:06:20.703574+0000","last_clean":"2024-04-25T00:06:20.703574+0000","last_became_active":"2024-04-25T00:06:20.697867+0000","last_became_peered":"2024-04-25T00:06:20.697867+0000","last_unstale":"2024-04-25T00:06:20.703574+0000","last_undegraded":"2024-04-25T00:06:20.703574+0000","last_fullsized":"2024-04-25T00:06:20.703574+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:18.863376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707052+0000","last_change":"2024-04-25T00:06:20.705758+0000","last_active":"2024-04-25T00:06:20.707052+0000","last_peered":"2024-04-25T00:06:20.707052+0000","last_clean":"2024-04-25T00:06:20.707052+0000","last_became_active":"2024-04-25T00:06:20.705459+0000","last_became_peered":"2024-04-25T00:06:20.705459+0000","last_unstale":"2024-04-25T00:06:20.707052+0000","last_undegraded":"2024-04-25T00:06:20.707052+0000","last_fullsized":"2024-04-25T00:06:20.707052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:26.589209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702623+0000","last_change":"2024-04-25T00:06:20.695108+0000","last_active":"2024-04-25T00:06:20.702623+0000","last_peered":"2024-04-25T00:06:20.702623+0000","last_clean":"2024-04-25T00:06:20.702623+0000","last_became_active":"2024-04-25T00:06:20.694826+0000","last_became_peered":"2024-04-25T00:06:20.694826+0000","last_unstale":"2024-04-25T00:06:20.702623+0000","last_undegraded":"2024-04-25T00:06:20.702623+0000","last_fullsized":"2024-04-25T00:06:20.702623+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:56.963948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705923+0000","last_change":"2024-04-25T00:06:20.704055+0000","last_active":"2024-04-25T00:06:20.705923+0000","last_peered":"2024-04-25T00:06:20.705923+0000","last_clean":"2024-04-25T00:06:20.705923+0000","last_became_active":"2024-04-25T00:06:20.703761+0000","last_became_peered":"2024-04-25T00:06:20.703761+0000","last_unstale":"2024-04-25T00:06:20.705923+0000","last_undegraded":"2024-04-25T00:06:20.705923+0000","last_fullsized":"2024-04-25T00:06:20.705923+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:50.018793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703442+0000","last_change":"2024-04-25T00:06:20.700557+0000","last_active":"2024-04-25T00:06:20.703442+0000","last_peered":"2024-04-25T00:06:20.703442+0000","last_clean":"2024-04-25T00:06:20.703442+0000","last_became_active":"2024-04-25T00:06:20.700171+0000","last_became_peered":"2024-04-25T00:06:20.700171+0000","last_unstale":"2024-04-25T00:06:20.703442+0000","last_undegraded":"2024-04-25T00:06:20.703442+0000","last_fullsized":"2024-04-25T00:06:20.703442+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:11.307194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.692089+0000","last_change":"2024-04-25T00:06:20.691472+0000","last_active":"2024-04-25T00:06:20.692089+0000","last_peered":"2024-04-25T00:06:20.692089+0000","last_clean":"2024-04-25T00:06:20.692089+0000","last_became_active":"2024-04-25T00:06:20.691202+0000","last_became_peered":"2024-04-25T00:06:20.691202+0000","last_unstale":"2024-04-25T00:06:20.692089+0000","last_undegraded":"2024-04-25T00:06:20.692089+0000","last_fullsized":"2024-04-25T00:06:20.692089+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:11.363491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706411+0000","last_change":"2024-04-25T00:06:20.705753+0000","last_active":"2024-04-25T00:06:20.706411+0000","last_peered":"2024-04-25T00:06:20.706411+0000","last_clean":"2024-04-25T00:06:20.706411+0000","last_became_active":"2024-04-25T00:06:20.705491+0000","last_became_peered":"2024-04-25T00:06:20.705491+0000","last_unstale":"2024-04-25T00:06:20.706411+0000","last_undegraded":"2024-04-25T00:06:20.706411+0000","last_fullsized":"2024-04-25T00:06:20.706411+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:48.614037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703291+0000","last_change":"2024-04-25T00:06:20.700675+0000","last_active":"2024-04-25T00:06:20.703291+0000","last_peered":"2024-04-25T00:06:20.703291+0000","last_clean":"2024-04-25T00:06:20.703291+0000","last_became_active":"2024-04-25T00:06:20.699442+0000","last_became_peered":"2024-04-25T00:06:20.699442+0000","last_unstale":"2024-04-25T00:06:20.703291+0000","last_undegraded":"2024-04-25T00:06:20.703291+0000","last_fullsized":"2024-04-25T00:06:20.703291+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:07.307032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713388+0000","last_change":"2024-04-25T00:06:20.711646+0000","last_active":"2024-04-25T00:06:20.713388+0000","last_peered":"2024-04-25T00:06:20.713388+0000","last_clean":"2024-04-25T00:06:20.713388+0000","last_became_active":"2024-04-25T00:06:20.710449+0000","last_became_peered":"2024-04-25T00:06:20.710449+0000","last_unstale":"2024-04-25T00:06:20.713388+0000","last_undegraded":"2024-04-25T00:06:20.713388+0000","last_fullsized":"2024-04-25T00:06:20.713388+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:20.878055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.708098+0000","last_change":"2024-04-25T00:06:20.707224+0000","last_active":"2024-04-25T00:06:20.708098+0000","last_peered":"2024-04-25T00:06:20.708098+0000","last_clean":"2024-04-25T00:06:20.708098+0000","last_became_active":"2024-04-25T00:06:20.706783+0000","last_became_peered":"2024-04-25T00:06:20.706783+0000","last_unstale":"2024-04-25T00:06:20.708098+0000","last_undegraded":"2024-04-25T00:06:20.708098+0000","last_fullsized":"2024-04-25T00:06:20.708098+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:43.771477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711090+0000","last_change":"2024-04-25T00:06:20.708544+0000","last_active":"2024-04-25T00:06:20.711090+0000","last_peered":"2024-04-25T00:06:20.711090+0000","last_clean":"2024-04-25T00:06:20.711090+0000","last_became_active":"2024-04-25T00:06:20.708282+0000","last_became_peered":"2024-04-25T00:06:20.708282+0000","last_unstale":"2024-04-25T00:06:20.711090+0000","last_undegraded":"2024-04-25T00:06:20.711090+0000","last_fullsized":"2024-04-25T00:06:20.711090+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:25.473078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709515+0000","last_change":"2024-04-25T00:06:20.708499+0000","last_active":"2024-04-25T00:06:20.709515+0000","last_peered":"2024-04-25T00:06:20.709515+0000","last_clean":"2024-04-25T00:06:20.709515+0000","last_became_active":"2024-04-25T00:06:20.708237+0000","last_became_peered":"2024-04-25T00:06:20.708237+0000","last_unstale":"2024-04-25T00:06:20.709515+0000","last_undegraded":"2024-04-25T00:06:20.709515+0000","last_fullsized":"2024-04-25T00:06:20.709515+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:40.303050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.699871+0000","last_change":"2024-04-25T00:06:20.695498+0000","last_active":"2024-04-25T00:06:20.699871+0000","last_peered":"2024-04-25T00:06:20.699871+0000","last_clean":"2024-04-25T00:06:20.699871+0000","last_became_active":"2024-04-25T00:06:20.694332+0000","last_became_peered":"2024-04-25T00:06:20.694332+0000","last_unstale":"2024-04-25T00:06:20.699871+0000","last_undegraded":"2024-04-25T00:06:20.699871+0000","last_fullsized":"2024-04-25T00:06:20.699871+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:58.121035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707336+0000","last_change":"2024-04-25T00:06:20.704253+0000","last_active":"2024-04-25T00:06:20.707336+0000","last_peered":"2024-04-25T00:06:20.707336+0000","last_clean":"2024-04-25T00:06:20.707336+0000","last_became_active":"2024-04-25T00:06:20.703825+0000","last_became_peered":"2024-04-25T00:06:20.703825+0000","last_unstale":"2024-04-25T00:06:20.707336+0000","last_undegraded":"2024-04-25T00:06:20.707336+0000","last_fullsized":"2024-04-25T00:06:20.707336+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:02.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.698398+0000","last_change":"2024-04-25T00:06:20.693498+0000","last_active":"2024-04-25T00:06:20.698398+0000","last_peered":"2024-04-25T00:06:20.698398+0000","last_clean":"2024-04-25T00:06:20.698398+0000","last_became_active":"2024-04-25T00:06:20.693205+0000","last_became_peered":"2024-04-25T00:06:20.693205+0000","last_unstale":"2024-04-25T00:06:20.698398+0000","last_undegraded":"2024-04-25T00:06:20.698398+0000","last_fullsized":"2024-04-25T00:06:20.698398+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:33.384120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707176+0000","last_change":"2024-04-25T00:06:20.703532+0000","last_active":"2024-04-25T00:06:20.707176+0000","last_peered":"2024-04-25T00:06:20.707176+0000","last_clean":"2024-04-25T00:06:20.707176+0000","last_became_active":"2024-04-25T00:06:20.703157+0000","last_became_peered":"2024-04-25T00:06:20.703157+0000","last_unstale":"2024-04-25T00:06:20.707176+0000","last_undegraded":"2024-04-25T00:06:20.707176+0000","last_fullsized":"2024-04-25T00:06:20.707176+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:03.180423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703552+0000","last_change":"2024-04-25T00:06:20.698164+0000","last_active":"2024-04-25T00:06:20.703552+0000","last_peered":"2024-04-25T00:06:20.703552+0000","last_clean":"2024-04-25T00:06:20.703552+0000","last_became_active":"2024-04-25T00:06:20.697501+0000","last_became_peered":"2024-04-25T00:06:20.697501+0000","last_unstale":"2024-04-25T00:06:20.703552+0000","last_undegraded":"2024-04-25T00:06:20.703552+0000","last_fullsized":"2024-04-25T00:06:20.703552+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:08:40.303050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.698366+0000","last_change":"2024-04-25T00:06:20.694285+0000","last_active":"2024-04-25T00:06:20.698366+0000","last_peered":"2024-04-25T00:06:20.698366+0000","last_clean":"2024-04-25T00:06:20.698366+0000","last_became_active":"2024-04-25T00:06:20.694013+0000","last_became_peered":"2024-04-25T00:06:20.694013+0000","last_unstale":"2024-04-25T00:06:20.698366+0000","last_undegraded":"2024-04-25T00:06:20.698366+0000","last_fullsized":"2024-04-25T00:06:20.698366+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:11.450117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703325+0000","last_change":"2024-04-25T00:06:20.700518+0000","last_active":"2024-04-25T00:06:20.703325+0000","last_peered":"2024-04-25T00:06:20.703325+0000","last_clean":"2024-04-25T00:06:20.703325+0000","last_became_active":"2024-04-25T00:06:20.700240+0000","last_became_peered":"2024-04-25T00:06:20.700240+0000","last_unstale":"2024-04-25T00:06:20.703325+0000","last_undegraded":"2024-04-25T00:06:20.703325+0000","last_fullsized":"2024-04-25T00:06:20.703325+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:57.303809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695191+0000","last_change":"2024-04-25T00:06:20.692640+0000","last_active":"2024-04-25T00:06:20.695191+0000","last_peered":"2024-04-25T00:06:20.695191+0000","last_clean":"2024-04-25T00:06:20.695191+0000","last_became_active":"2024-04-25T00:06:20.692370+0000","last_became_peered":"2024-04-25T00:06:20.692370+0000","last_unstale":"2024-04-25T00:06:20.695191+0000","last_undegraded":"2024-04-25T00:06:20.695191+0000","last_fullsized":"2024-04-25T00:06:20.695191+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:27.526068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.700801+0000","last_change":"2024-04-25T00:06:20.696510+0000","last_active":"2024-04-25T00:06:20.700801+0000","last_peered":"2024-04-25T00:06:20.700801+0000","last_clean":"2024-04-25T00:06:20.700801+0000","last_became_active":"2024-04-25T00:06:20.695803+0000","last_became_peered":"2024-04-25T00:06:20.695803+0000","last_unstale":"2024-04-25T00:06:20.700801+0000","last_undegraded":"2024-04-25T00:06:20.700801+0000","last_fullsized":"2024-04-25T00:06:20.700801+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:02.494718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712092+0000","last_change":"2024-04-25T00:06:20.711069+0000","last_active":"2024-04-25T00:06:20.712092+0000","last_peered":"2024-04-25T00:06:20.712092+0000","last_clean":"2024-04-25T00:06:20.712092+0000","last_became_active":"2024-04-25T00:06:20.710761+0000","last_became_peered":"2024-04-25T00:06:20.710761+0000","last_unstale":"2024-04-25T00:06:20.712092+0000","last_undegraded":"2024-04-25T00:06:20.712092+0000","last_fullsized":"2024-04-25T00:06:20.712092+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:18.863376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712947+0000","last_change":"2024-04-25T00:06:20.710612+0000","last_active":"2024-04-25T00:06:20.712947+0000","last_peered":"2024-04-25T00:06:20.712947+0000","last_clean":"2024-04-25T00:06:20.712947+0000","last_became_active":"2024-04-25T00:06:20.710333+0000","last_became_peered":"2024-04-25T00:06:20.710333+0000","last_unstale":"2024-04-25T00:06:20.712947+0000","last_undegraded":"2024-04-25T00:06:20.712947+0000","last_fullsized":"2024-04-25T00:06:20.712947+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:02.936119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706829+0000","last_change":"2024-04-25T00:06:20.700894+0000","last_active":"2024-04-25T00:06:20.706829+0000","last_peered":"2024-04-25T00:06:20.706829+0000","last_clean":"2024-04-25T00:06:20.706829+0000","last_became_active":"2024-04-25T00:06:20.700609+0000","last_became_peered":"2024-04-25T00:06:20.700609+0000","last_unstale":"2024-04-25T00:06:20.706829+0000","last_undegraded":"2024-04-25T00:06:20.706829+0000","last_fullsized":"2024-04-25T00:06:20.706829+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:35.278809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.712140+0000","last_change":"2024-04-25T00:06:20.709320+0000","last_active":"2024-04-25T00:06:20.712140+0000","last_peered":"2024-04-25T00:06:20.712140+0000","last_clean":"2024-04-25T00:06:20.712140+0000","last_became_active":"2024-04-25T00:06:20.709058+0000","last_became_peered":"2024-04-25T00:06:20.709058+0000","last_unstale":"2024-04-25T00:06:20.712140+0000","last_undegraded":"2024-04-25T00:06:20.712140+0000","last_fullsized":"2024-04-25T00:06:20.712140+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:01.404075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713206+0000","last_change":"2024-04-25T00:06:20.711081+0000","last_active":"2024-04-25T00:06:20.713206+0000","last_peered":"2024-04-25T00:06:20.713206+0000","last_clean":"2024-04-25T00:06:20.713206+0000","last_became_active":"2024-04-25T00:06:20.710280+0000","last_became_peered":"2024-04-25T00:06:20.710280+0000","last_unstale":"2024-04-25T00:06:20.713206+0000","last_undegraded":"2024-04-25T00:06:20.713206+0000","last_fullsized":"2024-04-25T00:06:20.713206+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:48.377125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709656+0000","last_change":"2024-04-25T00:06:20.709185+0000","last_active":"2024-04-25T00:06:20.709656+0000","last_peered":"2024-04-25T00:06:20.709656+0000","last_clean":"2024-04-25T00:06:20.709656+0000","last_became_active":"2024-04-25T00:06:20.708758+0000","last_became_peered":"2024-04-25T00:06:20.708758+0000","last_unstale":"2024-04-25T00:06:20.709656+0000","last_undegraded":"2024-04-25T00:06:20.709656+0000","last_fullsized":"2024-04-25T00:06:20.709656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:38.475784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702446+0000","last_change":"2024-04-25T00:06:20.700855+0000","last_active":"2024-04-25T00:06:20.702446+0000","last_peered":"2024-04-25T00:06:20.702446+0000","last_clean":"2024-04-25T00:06:20.702446+0000","last_became_active":"2024-04-25T00:06:20.700561+0000","last_became_peered":"2024-04-25T00:06:20.700561+0000","last_unstale":"2024-04-25T00:06:20.702446+0000","last_undegraded":"2024-04-25T00:06:20.702446+0000","last_fullsized":"2024-04-25T00:06:20.702446+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:13.149918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705585+0000","last_change":"2024-04-25T00:06:20.700547+0000","last_active":"2024-04-25T00:06:20.705585+0000","last_peered":"2024-04-25T00:06:20.705585+0000","last_clean":"2024-04-25T00:06:20.705585+0000","last_became_active":"2024-04-25T00:06:20.700252+0000","last_became_peered":"2024-04-25T00:06:20.700252+0000","last_unstale":"2024-04-25T00:06:20.705585+0000","last_undegraded":"2024-04-25T00:06:20.705585+0000","last_fullsized":"2024-04-25T00:06:20.705585+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:02.892640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"129'4","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713894+0000","last_change":"2024-04-25T00:06:19.701625+0000","last_active":"2024-04-25T00:06:20.713894+0000","last_peered":"2024-04-25T00:06:20.713894+0000","last_clean":"2024-04-25T00:06:20.713894+0000","last_became_active":"2024-04-25T00:06:19.701325+0000","last_became_peered":"2024-04-25T00:06:19.701325+0000","last_unstale":"2024-04-25T00:06:20.713894+0000","last_undegraded":"2024-04-25T00:06:20.713894+0000","last_fullsized":"2024-04-25T00:06:20.713894+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:15.208488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.707939+0000","last_change":"2024-04-25T00:06:20.706028+0000","last_active":"2024-04-25T00:06:20.707939+0000","last_peered":"2024-04-25T00:06:20.707939+0000","last_clean":"2024-04-25T00:06:20.707939+0000","last_became_active":"2024-04-25T00:06:20.705747+0000","last_became_peered":"2024-04-25T00:06:20.705747+0000","last_unstale":"2024-04-25T00:06:20.707939+0000","last_undegraded":"2024-04-25T00:06:20.707939+0000","last_fullsized":"2024-04-25T00:06:20.707939+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:47.692642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709166+0000","last_change":"2024-04-25T00:06:19.693228+0000","last_active":"2024-04-25T00:06:20.709166+0000","last_peered":"2024-04-25T00:06:20.709166+0000","last_clean":"2024-04-25T00:06:20.709166+0000","last_became_active":"2024-04-25T00:06:19.692959+0000","last_became_peered":"2024-04-25T00:06:19.692959+0000","last_unstale":"2024-04-25T00:06:20.709166+0000","last_undegraded":"2024-04-25T00:06:20.709166+0000","last_fullsized":"2024-04-25T00:06:20.709166+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:31.704449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703772+0000","last_change":"2024-04-25T00:06:20.701610+0000","last_active":"2024-04-25T00:06:20.703772+0000","last_peered":"2024-04-25T00:06:20.703772+0000","last_clean":"2024-04-25T00:06:20.703772+0000","last_became_active":"2024-04-25T00:06:20.701101+0000","last_became_peered":"2024-04-25T00:06:20.701101+0000","last_unstale":"2024-04-25T00:06:20.703772+0000","last_undegraded":"2024-04-25T00:06:20.703772+0000","last_fullsized":"2024-04-25T00:06:20.703772+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:55.905435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709050+0000","last_change":"2024-04-25T00:06:19.691146+0000","last_active":"2024-04-25T00:06:20.709050+0000","last_peered":"2024-04-25T00:06:20.709050+0000","last_clean":"2024-04-25T00:06:20.709050+0000","last_became_active":"2024-04-25T00:06:19.690870+0000","last_became_peered":"2024-04-25T00:06:19.690870+0000","last_unstale":"2024-04-25T00:06:20.709050+0000","last_undegraded":"2024-04-25T00:06:20.709050+0000","last_fullsized":"2024-04-25T00:06:20.709050+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:14.697845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.708596+0000","last_change":"2024-04-25T00:06:20.706752+0000","last_active":"2024-04-25T00:06:20.708596+0000","last_peered":"2024-04-25T00:06:20.708596+0000","last_clean":"2024-04-25T00:06:20.708596+0000","last_became_active":"2024-04-25T00:06:20.706343+0000","last_became_peered":"2024-04-25T00:06:20.706343+0000","last_unstale":"2024-04-25T00:06:20.708596+0000","last_undegraded":"2024-04-25T00:06:20.708596+0000","last_fullsized":"2024-04-25T00:06:20.708596+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:26.247824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.700724+0000","last_change":"2024-04-25T00:06:19.690623+0000","last_active":"2024-04-25T00:06:20.700724+0000","last_peered":"2024-04-25T00:06:20.700724+0000","last_clean":"2024-04-25T00:06:20.700724+0000","last_became_active":"2024-04-25T00:06:19.690258+0000","last_became_peered":"2024-04-25T00:06:19.690258+0000","last_unstale":"2024-04-25T00:06:20.700724+0000","last_undegraded":"2024-04-25T00:06:20.700724+0000","last_fullsized":"2024-04-25T00:06:20.700724+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:37.018486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705774+0000","last_change":"2024-04-25T00:06:20.701448+0000","last_active":"2024-04-25T00:06:20.705774+0000","last_peered":"2024-04-25T00:06:20.705774+0000","last_clean":"2024-04-25T00:06:20.705774+0000","last_became_active":"2024-04-25T00:06:20.701147+0000","last_became_peered":"2024-04-25T00:06:20.701147+0000","last_unstale":"2024-04-25T00:06:20.705774+0000","last_undegraded":"2024-04-25T00:06:20.705774+0000","last_fullsized":"2024-04-25T00:06:20.705774+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:32.385455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702156+0000","last_change":"2024-04-25T00:06:19.691740+0000","last_active":"2024-04-25T00:06:20.702156+0000","last_peered":"2024-04-25T00:06:20.702156+0000","last_clean":"2024-04-25T00:06:20.702156+0000","last_became_active":"2024-04-25T00:06:19.691455+0000","last_became_peered":"2024-04-25T00:06:19.691455+0000","last_unstale":"2024-04-25T00:06:20.702156+0000","last_undegraded":"2024-04-25T00:06:20.702156+0000","last_fullsized":"2024-04-25T00:06:20.702156+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:01.929796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.701595+0000","last_change":"2024-04-25T00:06:20.694485+0000","last_active":"2024-04-25T00:06:20.701595+0000","last_peered":"2024-04-25T00:06:20.701595+0000","last_clean":"2024-04-25T00:06:20.701595+0000","last_became_active":"2024-04-25T00:06:20.694131+0000","last_became_peered":"2024-04-25T00:06:20.694131+0000","last_unstale":"2024-04-25T00:06:20.701595+0000","last_undegraded":"2024-04-25T00:06:20.701595+0000","last_fullsized":"2024-04-25T00:06:20.701595+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:29.420293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.702745+0000","last_change":"2024-04-25T00:06:19.692230+0000","last_active":"2024-04-25T00:06:20.702745+0000","last_peered":"2024-04-25T00:06:20.702745+0000","last_clean":"2024-04-25T00:06:20.702745+0000","last_became_active":"2024-04-25T00:06:19.691359+0000","last_became_peered":"2024-04-25T00:06:19.691359+0000","last_unstale":"2024-04-25T00:06:20.702745+0000","last_undegraded":"2024-04-25T00:06:20.702745+0000","last_fullsized":"2024-04-25T00:06:20.702745+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:37.075006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.705899+0000","last_change":"2024-04-25T00:06:20.703126+0000","last_active":"2024-04-25T00:06:20.705899+0000","last_peered":"2024-04-25T00:06:20.705899+0000","last_clean":"2024-04-25T00:06:20.705899+0000","last_became_active":"2024-04-25T00:06:20.702851+0000","last_became_peered":"2024-04-25T00:06:20.702851+0000","last_unstale":"2024-04-25T00:06:20.705899+0000","last_undegraded":"2024-04-25T00:06:20.705899+0000","last_fullsized":"2024-04-25T00:06:20.705899+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:07.326922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686651+0000","last_change":"2024-04-25T00:06:19.698522+0000","last_active":"2024-04-25T00:06:20.686651+0000","last_peered":"2024-04-25T00:06:20.686651+0000","last_clean":"2024-04-25T00:06:20.686651+0000","last_became_active":"2024-04-25T00:06:19.698106+0000","last_became_peered":"2024-04-25T00:06:19.698106+0000","last_unstale":"2024-04-25T00:06:20.686651+0000","last_undegraded":"2024-04-25T00:06:20.686651+0000","last_fullsized":"2024-04-25T00:06:20.686651+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:12.117273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703336+0000","last_change":"2024-04-25T00:06:20.697604+0000","last_active":"2024-04-25T00:06:20.703336+0000","last_peered":"2024-04-25T00:06:20.703336+0000","last_clean":"2024-04-25T00:06:20.703336+0000","last_became_active":"2024-04-25T00:06:20.697357+0000","last_became_peered":"2024-04-25T00:06:20.697357+0000","last_unstale":"2024-04-25T00:06:20.703336+0000","last_undegraded":"2024-04-25T00:06:20.703336+0000","last_fullsized":"2024-04-25T00:06:20.703336+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:42.725022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686243+0000","last_change":"2024-04-25T00:06:19.693617+0000","last_active":"2024-04-25T00:06:20.686243+0000","last_peered":"2024-04-25T00:06:20.686243+0000","last_clean":"2024-04-25T00:06:20.686243+0000","last_became_active":"2024-04-25T00:06:19.693199+0000","last_became_peered":"2024-04-25T00:06:19.693199+0000","last_unstale":"2024-04-25T00:06:20.686243+0000","last_undegraded":"2024-04-25T00:06:20.686243+0000","last_fullsized":"2024-04-25T00:06:20.686243+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:47.370802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711129+0000","last_change":"2024-04-25T00:06:20.708428+0000","last_active":"2024-04-25T00:06:20.711129+0000","last_peered":"2024-04-25T00:06:20.711129+0000","last_clean":"2024-04-25T00:06:20.711129+0000","last_became_active":"2024-04-25T00:06:20.708138+0000","last_became_peered":"2024-04-25T00:06:20.708138+0000","last_unstale":"2024-04-25T00:06:20.711129+0000","last_undegraded":"2024-04-25T00:06:20.711129+0000","last_fullsized":"2024-04-25T00:06:20.711129+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:22.290210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.696335+0000","last_change":"2024-04-25T00:06:19.691769+0000","last_active":"2024-04-25T00:06:20.696335+0000","last_peered":"2024-04-25T00:06:20.696335+0000","last_clean":"2024-04-25T00:06:20.696335+0000","last_became_active":"2024-04-25T00:06:19.691303+0000","last_became_peered":"2024-04-25T00:06:19.691303+0000","last_unstale":"2024-04-25T00:06:20.696335+0000","last_undegraded":"2024-04-25T00:06:20.696335+0000","last_fullsized":"2024-04-25T00:06:20.696335+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:10.300871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703519+0000","last_change":"2024-04-25T00:06:20.700961+0000","last_active":"2024-04-25T00:06:20.703519+0000","last_peered":"2024-04-25T00:06:20.703519+0000","last_clean":"2024-04-25T00:06:20.703519+0000","last_became_active":"2024-04-25T00:06:20.700521+0000","last_became_peered":"2024-04-25T00:06:20.700521+0000","last_unstale":"2024-04-25T00:06:20.703519+0000","last_undegraded":"2024-04-25T00:06:20.703519+0000","last_fullsized":"2024-04-25T00:06:20.703519+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.078836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.696426+0000","last_change":"2024-04-25T00:06:19.699765+0000","last_active":"2024-04-25T00:06:20.696426+0000","last_peered":"2024-04-25T00:06:20.696426+0000","last_clean":"2024-04-25T00:06:20.696426+0000","last_became_active":"2024-04-25T00:06:19.699073+0000","last_became_peered":"2024-04-25T00:06:19.699073+0000","last_unstale":"2024-04-25T00:06:20.696426+0000","last_undegraded":"2024-04-25T00:06:20.696426+0000","last_fullsized":"2024-04-25T00:06:20.696426+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:34:22.576364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703752+0000","last_change":"2024-04-25T00:06:20.701386+0000","last_active":"2024-04-25T00:06:20.703752+0000","last_peered":"2024-04-25T00:06:20.703752+0000","last_clean":"2024-04-25T00:06:20.703752+0000","last_became_active":"2024-04-25T00:06:20.700725+0000","last_became_peered":"2024-04-25T00:06:20.700725+0000","last_unstale":"2024-04-25T00:06:20.703752+0000","last_undegraded":"2024-04-25T00:06:20.703752+0000","last_fullsized":"2024-04-25T00:06:20.703752+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:42:03.845419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"129'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.710399+0000","last_change":"2024-04-25T00:06:19.694114+0000","last_active":"2024-04-25T00:06:20.710399+0000","last_peered":"2024-04-25T00:06:20.710399+0000","last_clean":"2024-04-25T00:06:20.710399+0000","last_became_active":"2024-04-25T00:06:19.693839+0000","last_became_peered":"2024-04-25T00:06:19.693839+0000","last_unstale":"2024-04-25T00:06:20.710399+0000","last_undegraded":"2024-04-25T00:06:20.710399+0000","last_fullsized":"2024-04-25T00:06:20.710399+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:42.765154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703688+0000","last_change":"2024-04-25T00:06:20.701871+0000","last_active":"2024-04-25T00:06:20.703688+0000","last_peered":"2024-04-25T00:06:20.703688+0000","last_clean":"2024-04-25T00:06:20.703688+0000","last_became_active":"2024-04-25T00:06:20.701563+0000","last_became_peered":"2024-04-25T00:06:20.701563+0000","last_unstale":"2024-04-25T00:06:20.703688+0000","last_undegraded":"2024-04-25T00:06:20.703688+0000","last_fullsized":"2024-04-25T00:06:20.703688+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:44.265788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.715678+0000","last_change":"2024-04-25T00:06:19.698406+0000","last_active":"2024-04-25T00:06:20.715678+0000","last_peered":"2024-04-25T00:06:20.715678+0000","last_clean":"2024-04-25T00:06:20.715678+0000","last_became_active":"2024-04-25T00:06:19.698143+0000","last_became_peered":"2024-04-25T00:06:19.698143+0000","last_unstale":"2024-04-25T00:06:20.715678+0000","last_undegraded":"2024-04-25T00:06:20.715678+0000","last_fullsized":"2024-04-25T00:06:20.715678+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:13.517191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711785+0000","last_change":"2024-04-25T00:06:20.708777+0000","last_active":"2024-04-25T00:06:20.711785+0000","last_peered":"2024-04-25T00:06:20.711785+0000","last_clean":"2024-04-25T00:06:20.711785+0000","last_became_active":"2024-04-25T00:06:20.707550+0000","last_became_peered":"2024-04-25T00:06:20.707550+0000","last_unstale":"2024-04-25T00:06:20.711785+0000","last_undegraded":"2024-04-25T00:06:20.711785+0000","last_fullsized":"2024-04-25T00:06:20.711785+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:01.561976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695364+0000","last_change":"2024-04-25T00:06:19.692457+0000","last_active":"2024-04-25T00:06:20.695364+0000","last_peered":"2024-04-25T00:06:20.695364+0000","last_clean":"2024-04-25T00:06:20.695364+0000","last_became_active":"2024-04-25T00:06:19.692182+0000","last_became_peered":"2024-04-25T00:06:19.692182+0000","last_unstale":"2024-04-25T00:06:20.695364+0000","last_undegraded":"2024-04-25T00:06:20.695364+0000","last_fullsized":"2024-04-25T00:06:20.695364+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:52.441703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.704241+0000","last_change":"2024-04-25T00:06:20.703255+0000","last_active":"2024-04-25T00:06:20.704241+0000","last_peered":"2024-04-25T00:06:20.704241+0000","last_clean":"2024-04-25T00:06:20.704241+0000","last_became_active":"2024-04-25T00:06:20.698029+0000","last_became_peered":"2024-04-25T00:06:20.698029+0000","last_unstale":"2024-04-25T00:06:20.704241+0000","last_undegraded":"2024-04-25T00:06:20.704241+0000","last_fullsized":"2024-04-25T00:06:20.704241+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:28.479192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686634+0000","last_change":"2024-04-25T00:06:19.702000+0000","last_active":"2024-04-25T00:06:20.686634+0000","last_peered":"2024-04-25T00:06:20.686634+0000","last_clean":"2024-04-25T00:06:20.686634+0000","last_became_active":"2024-04-25T00:06:19.701423+0000","last_became_peered":"2024-04-25T00:06:19.701423+0000","last_unstale":"2024-04-25T00:06:20.686634+0000","last_undegraded":"2024-04-25T00:06:20.686634+0000","last_fullsized":"2024-04-25T00:06:20.686634+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:19.871732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703343+0000","last_change":"2024-04-25T00:06:20.701068+0000","last_active":"2024-04-25T00:06:20.703343+0000","last_peered":"2024-04-25T00:06:20.703343+0000","last_clean":"2024-04-25T00:06:20.703343+0000","last_became_active":"2024-04-25T00:06:20.700797+0000","last_became_peered":"2024-04-25T00:06:20.700797+0000","last_unstale":"2024-04-25T00:06:20.703343+0000","last_undegraded":"2024-04-25T00:06:20.703343+0000","last_fullsized":"2024-04-25T00:06:20.703343+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:35.278809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"129'8","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.715266+0000","last_change":"2024-04-25T00:06:19.696059+0000","last_active":"2024-04-25T00:06:20.715266+0000","last_peered":"2024-04-25T00:06:20.715266+0000","last_clean":"2024-04-25T00:06:20.715266+0000","last_became_active":"2024-04-25T00:06:19.695569+0000","last_became_peered":"2024-04-25T00:06:19.695569+0000","last_unstale":"2024-04-25T00:06:20.715266+0000","last_undegraded":"2024-04-25T00:06:20.715266+0000","last_fullsized":"2024-04-25T00:06:20.715266+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:31.704449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709157+0000","last_change":"2024-04-25T00:06:20.708688+0000","last_active":"2024-04-25T00:06:20.709157+0000","last_peered":"2024-04-25T00:06:20.709157+0000","last_clean":"2024-04-25T00:06:20.709157+0000","last_became_active":"2024-04-25T00:06:20.708415+0000","last_became_peered":"2024-04-25T00:06:20.708415+0000","last_unstale":"2024-04-25T00:06:20.709157+0000","last_undegraded":"2024-04-25T00:06:20.709157+0000","last_fullsized":"2024-04-25T00:06:20.709157+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:13.123596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709574+0000","last_change":"2024-04-25T00:06:19.698440+0000","last_active":"2024-04-25T00:06:20.709574+0000","last_peered":"2024-04-25T00:06:20.709574+0000","last_clean":"2024-04-25T00:06:20.709574+0000","last_became_active":"2024-04-25T00:06:19.698150+0000","last_became_peered":"2024-04-25T00:06:19.698150+0000","last_unstale":"2024-04-25T00:06:20.709574+0000","last_undegraded":"2024-04-25T00:06:20.709574+0000","last_fullsized":"2024-04-25T00:06:20.709574+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:56.297486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703498+0000","last_change":"2024-04-25T00:06:20.700160+0000","last_active":"2024-04-25T00:06:20.703498+0000","last_peered":"2024-04-25T00:06:20.703498+0000","last_clean":"2024-04-25T00:06:20.703498+0000","last_became_active":"2024-04-25T00:06:20.699859+0000","last_became_peered":"2024-04-25T00:06:20.699859+0000","last_unstale":"2024-04-25T00:06:20.703498+0000","last_undegraded":"2024-04-25T00:06:20.703498+0000","last_fullsized":"2024-04-25T00:06:20.703498+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:07.307032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706213+0000","last_change":"2024-04-25T00:06:19.694018+0000","last_active":"2024-04-25T00:06:20.706213+0000","last_peered":"2024-04-25T00:06:20.706213+0000","last_clean":"2024-04-25T00:06:20.706213+0000","last_became_active":"2024-04-25T00:06:19.693621+0000","last_became_peered":"2024-04-25T00:06:19.693621+0000","last_unstale":"2024-04-25T00:06:20.706213+0000","last_undegraded":"2024-04-25T00:06:20.706213+0000","last_fullsized":"2024-04-25T00:06:20.706213+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.869942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709959+0000","last_change":"2024-04-25T00:06:20.706848+0000","last_active":"2024-04-25T00:06:20.709959+0000","last_peered":"2024-04-25T00:06:20.709959+0000","last_clean":"2024-04-25T00:06:20.709959+0000","last_became_active":"2024-04-25T00:06:20.706517+0000","last_became_peered":"2024-04-25T00:06:20.706517+0000","last_unstale":"2024-04-25T00:06:20.709959+0000","last_undegraded":"2024-04-25T00:06:20.709959+0000","last_fullsized":"2024-04-25T00:06:20.709959+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:16.214811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.710208+0000","last_change":"2024-04-25T00:06:19.692349+0000","last_active":"2024-04-25T00:06:20.710208+0000","last_peered":"2024-04-25T00:06:20.710208+0000","last_clean":"2024-04-25T00:06:20.710208+0000","last_became_active":"2024-04-25T00:06:19.692069+0000","last_became_peered":"2024-04-25T00:06:19.692069+0000","last_unstale":"2024-04-25T00:06:20.710208+0000","last_undegraded":"2024-04-25T00:06:20.710208+0000","last_fullsized":"2024-04-25T00:06:20.710208+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:46.686319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703193+0000","last_change":"2024-04-25T00:06:20.697691+0000","last_active":"2024-04-25T00:06:20.703193+0000","last_peered":"2024-04-25T00:06:20.703193+0000","last_clean":"2024-04-25T00:06:20.703193+0000","last_became_active":"2024-04-25T00:06:20.697456+0000","last_became_peered":"2024-04-25T00:06:20.697456+0000","last_unstale":"2024-04-25T00:06:20.703193+0000","last_undegraded":"2024-04-25T00:06:20.703193+0000","last_fullsized":"2024-04-25T00:06:20.703193+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:44.487735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.691922+0000","last_change":"2024-04-25T00:06:19.692518+0000","last_active":"2024-04-25T00:06:20.691922+0000","last_peered":"2024-04-25T00:06:20.691922+0000","last_clean":"2024-04-25T00:06:20.691922+0000","last_became_active":"2024-04-25T00:06:19.692247+0000","last_became_peered":"2024-04-25T00:06:19.692247+0000","last_unstale":"2024-04-25T00:06:20.691922+0000","last_undegraded":"2024-04-25T00:06:20.691922+0000","last_fullsized":"2024-04-25T00:06:20.691922+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:13.517191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711359+0000","last_change":"2024-04-25T00:06:20.705820+0000","last_active":"2024-04-25T00:06:20.711359+0000","last_peered":"2024-04-25T00:06:20.711359+0000","last_clean":"2024-04-25T00:06:20.711359+0000","last_became_active":"2024-04-25T00:06:20.705130+0000","last_became_peered":"2024-04-25T00:06:20.705130+0000","last_unstale":"2024-04-25T00:06:20.711359+0000","last_undegraded":"2024-04-25T00:06:20.711359+0000","last_fullsized":"2024-04-25T00:06:20.711359+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:34.416408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713544+0000","last_change":"2024-04-25T00:06:19.690058+0000","last_active":"2024-04-25T00:06:20.713544+0000","last_peered":"2024-04-25T00:06:20.713544+0000","last_clean":"2024-04-25T00:06:20.713544+0000","last_became_active":"2024-04-25T00:06:19.689695+0000","last_became_peered":"2024-04-25T00:06:19.689695+0000","last_unstale":"2024-04-25T00:06:20.713544+0000","last_undegraded":"2024-04-25T00:06:20.713544+0000","last_fullsized":"2024-04-25T00:06:20.713544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:18.453085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.701615+0000","last_change":"2024-04-25T00:06:20.695256+0000","last_active":"2024-04-25T00:06:20.701615+0000","last_peered":"2024-04-25T00:06:20.701615+0000","last_clean":"2024-04-25T00:06:20.701615+0000","last_became_active":"2024-04-25T00:06:20.694836+0000","last_became_peered":"2024-04-25T00:06:20.694836+0000","last_unstale":"2024-04-25T00:06:20.701615+0000","last_undegraded":"2024-04-25T00:06:20.701615+0000","last_fullsized":"2024-04-25T00:06:20.701615+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:15.164079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.697191+0000","last_change":"2024-04-25T00:06:19.699252+0000","last_active":"2024-04-25T00:06:20.697191+0000","last_peered":"2024-04-25T00:06:20.697191+0000","last_clean":"2024-04-25T00:06:20.697191+0000","last_became_active":"2024-04-25T00:06:19.698811+0000","last_became_peered":"2024-04-25T00:06:19.698811+0000","last_unstale":"2024-04-25T00:06:20.697191+0000","last_undegraded":"2024-04-25T00:06:20.697191+0000","last_fullsized":"2024-04-25T00:06:20.697191+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:38.079026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703474+0000","last_change":"2024-04-25T00:06:20.694392+0000","last_active":"2024-04-25T00:06:20.703474+0000","last_peered":"2024-04-25T00:06:20.703474+0000","last_clean":"2024-04-25T00:06:20.703474+0000","last_became_active":"2024-04-25T00:06:20.694114+0000","last_became_peered":"2024-04-25T00:06:20.694114+0000","last_unstale":"2024-04-25T00:06:20.703474+0000","last_undegraded":"2024-04-25T00:06:20.703474+0000","last_fullsized":"2024-04-25T00:06:20.703474+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:28.942453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695092+0000","last_change":"2024-04-25T00:06:19.695574+0000","last_active":"2024-04-25T00:06:20.695092+0000","last_peered":"2024-04-25T00:06:20.695092+0000","last_clean":"2024-04-25T00:06:20.695092+0000","last_became_active":"2024-04-25T00:06:19.695299+0000","last_became_peered":"2024-04-25T00:06:19.695299+0000","last_unstale":"2024-04-25T00:06:20.695092+0000","last_undegraded":"2024-04-25T00:06:20.695092+0000","last_fullsized":"2024-04-25T00:06:20.695092+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:39.758070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709636+0000","last_change":"2024-04-25T00:06:20.709029+0000","last_active":"2024-04-25T00:06:20.709636+0000","last_peered":"2024-04-25T00:06:20.709636+0000","last_clean":"2024-04-25T00:06:20.709636+0000","last_became_active":"2024-04-25T00:06:20.708416+0000","last_became_peered":"2024-04-25T00:06:20.708416+0000","last_unstale":"2024-04-25T00:06:20.709636+0000","last_undegraded":"2024-04-25T00:06:20.709636+0000","last_fullsized":"2024-04-25T00:06:20.709636+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:44.943130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686672+0000","last_change":"2024-04-25T00:06:19.693790+0000","last_active":"2024-04-25T00:06:20.686672+0000","last_peered":"2024-04-25T00:06:20.686672+0000","last_clean":"2024-04-25T00:06:20.686672+0000","last_became_active":"2024-04-25T00:06:19.693496+0000","last_became_peered":"2024-04-25T00:06:19.693496+0000","last_unstale":"2024-04-25T00:06:20.686672+0000","last_undegraded":"2024-04-25T00:06:20.686672+0000","last_fullsized":"2024-04-25T00:06:20.686672+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:32.957137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.701485+0000","last_change":"2024-04-25T00:06:20.694136+0000","last_active":"2024-04-25T00:06:20.701485+0000","last_peered":"2024-04-25T00:06:20.701485+0000","last_clean":"2024-04-25T00:06:20.701485+0000","last_became_active":"2024-04-25T00:06:20.693829+0000","last_became_peered":"2024-04-25T00:06:20.693829+0000","last_unstale":"2024-04-25T00:06:20.701485+0000","last_undegraded":"2024-04-25T00:06:20.701485+0000","last_fullsized":"2024-04-25T00:06:20.701485+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:31.904364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.704939+0000","last_change":"2024-04-25T00:06:19.693661+0000","last_active":"2024-04-25T00:06:20.704939+0000","last_peered":"2024-04-25T00:06:20.704939+0000","last_clean":"2024-04-25T00:06:20.704939+0000","last_became_active":"2024-04-25T00:06:19.693378+0000","last_became_peered":"2024-04-25T00:06:19.693378+0000","last_unstale":"2024-04-25T00:06:20.704939+0000","last_undegraded":"2024-04-25T00:06:20.704939+0000","last_fullsized":"2024-04-25T00:06:20.704939+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:18.689971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713257+0000","last_change":"2024-04-25T00:06:20.711317+0000","last_active":"2024-04-25T00:06:20.713257+0000","last_peered":"2024-04-25T00:06:20.713257+0000","last_clean":"2024-04-25T00:06:20.713257+0000","last_became_active":"2024-04-25T00:06:20.709611+0000","last_became_peered":"2024-04-25T00:06:20.709611+0000","last_unstale":"2024-04-25T00:06:20.713257+0000","last_undegraded":"2024-04-25T00:06:20.713257+0000","last_fullsized":"2024-04-25T00:06:20.713257+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:31.101554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.697003+0000","last_change":"2024-04-25T00:06:19.698904+0000","last_active":"2024-04-25T00:06:20.697003+0000","last_peered":"2024-04-25T00:06:20.697003+0000","last_clean":"2024-04-25T00:06:20.697003+0000","last_became_active":"2024-04-25T00:06:19.698584+0000","last_became_peered":"2024-04-25T00:06:19.698584+0000","last_unstale":"2024-04-25T00:06:20.697003+0000","last_undegraded":"2024-04-25T00:06:20.697003+0000","last_fullsized":"2024-04-25T00:06:20.697003+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:26.226242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.700847+0000","last_change":"2024-04-25T00:06:20.695880+0000","last_active":"2024-04-25T00:06:20.700847+0000","last_peered":"2024-04-25T00:06:20.700847+0000","last_clean":"2024-04-25T00:06:20.700847+0000","last_became_active":"2024-04-25T00:06:20.695607+0000","last_became_peered":"2024-04-25T00:06:20.695607+0000","last_unstale":"2024-04-25T00:06:20.700847+0000","last_undegraded":"2024-04-25T00:06:20.700847+0000","last_fullsized":"2024-04-25T00:06:20.700847+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:06.747980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686319+0000","last_change":"2024-04-25T00:06:19.693205+0000","last_active":"2024-04-25T00:06:20.686319+0000","last_peered":"2024-04-25T00:06:20.686319+0000","last_clean":"2024-04-25T00:06:20.686319+0000","last_became_active":"2024-04-25T00:06:19.692924+0000","last_became_peered":"2024-04-25T00:06:19.692924+0000","last_unstale":"2024-04-25T00:06:20.686319+0000","last_undegraded":"2024-04-25T00:06:20.686319+0000","last_fullsized":"2024-04-25T00:06:20.686319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:30.095231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.708156+0000","last_change":"2024-04-25T00:06:20.707624+0000","last_active":"2024-04-25T00:06:20.708156+0000","last_peered":"2024-04-25T00:06:20.708156+0000","last_clean":"2024-04-25T00:06:20.708156+0000","last_became_active":"2024-04-25T00:06:20.707351+0000","last_became_peered":"2024-04-25T00:06:20.707351+0000","last_unstale":"2024-04-25T00:06:20.708156+0000","last_undegraded":"2024-04-25T00:06:20.708156+0000","last_fullsized":"2024-04-25T00:06:20.708156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:59.322666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.690219+0000","last_change":"2024-04-25T00:06:19.691478+0000","last_active":"2024-04-25T00:06:20.690219+0000","last_peered":"2024-04-25T00:06:20.690219+0000","last_clean":"2024-04-25T00:06:20.690219+0000","last_became_active":"2024-04-25T00:06:19.690432+0000","last_became_peered":"2024-04-25T00:06:19.690432+0000","last_unstale":"2024-04-25T00:06:20.690219+0000","last_undegraded":"2024-04-25T00:06:20.690219+0000","last_fullsized":"2024-04-25T00:06:20.690219+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:39.110465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.706944+0000","last_change":"2024-04-25T00:06:20.702775+0000","last_active":"2024-04-25T00:06:20.706944+0000","last_peered":"2024-04-25T00:06:20.706944+0000","last_clean":"2024-04-25T00:06:20.706944+0000","last_became_active":"2024-04-25T00:06:20.702503+0000","last_became_peered":"2024-04-25T00:06:20.702503+0000","last_unstale":"2024-04-25T00:06:20.706944+0000","last_undegraded":"2024-04-25T00:06:20.706944+0000","last_fullsized":"2024-04-25T00:06:20.706944+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:51.495938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686277+0000","last_change":"2024-04-25T00:06:19.701737+0000","last_active":"2024-04-25T00:06:20.686277+0000","last_peered":"2024-04-25T00:06:20.686277+0000","last_clean":"2024-04-25T00:06:20.686277+0000","last_became_active":"2024-04-25T00:06:19.701467+0000","last_became_peered":"2024-04-25T00:06:19.701467+0000","last_unstale":"2024-04-25T00:06:20.686277+0000","last_undegraded":"2024-04-25T00:06:20.686277+0000","last_fullsized":"2024-04-25T00:06:20.686277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:34.856071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.711486+0000","last_change":"2024-04-25T00:06:20.710960+0000","last_active":"2024-04-25T00:06:20.711486+0000","last_peered":"2024-04-25T00:06:20.711486+0000","last_clean":"2024-04-25T00:06:20.711486+0000","last_became_active":"2024-04-25T00:06:20.710670+0000","last_became_peered":"2024-04-25T00:06:20.710670+0000","last_unstale":"2024-04-25T00:06:20.711486+0000","last_undegraded":"2024-04-25T00:06:20.711486+0000","last_fullsized":"2024-04-25T00:06:20.711486+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:07.326922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.690788+0000","last_change":"2024-04-25T00:06:19.700988+0000","last_active":"2024-04-25T00:06:20.690788+0000","last_peered":"2024-04-25T00:06:20.690788+0000","last_clean":"2024-04-25T00:06:20.690788+0000","last_became_active":"2024-04-25T00:06:19.700681+0000","last_became_peered":"2024-04-25T00:06:19.700681+0000","last_unstale":"2024-04-25T00:06:20.690788+0000","last_undegraded":"2024-04-25T00:06:20.690788+0000","last_fullsized":"2024-04-25T00:06:20.690788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:02.174100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709494+0000","last_change":"2024-04-25T00:06:20.705602+0000","last_active":"2024-04-25T00:06:20.709494+0000","last_peered":"2024-04-25T00:06:20.709494+0000","last_clean":"2024-04-25T00:06:20.709494+0000","last_became_active":"2024-04-25T00:06:20.704784+0000","last_became_peered":"2024-04-25T00:06:20.704784+0000","last_unstale":"2024-04-25T00:06:20.709494+0000","last_undegraded":"2024-04-25T00:06:20.709494+0000","last_fullsized":"2024-04-25T00:06:20.709494+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:32:27.232565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.695264+0000","last_change":"2024-04-25T00:06:19.700725+0000","last_active":"2024-04-25T00:06:20.695264+0000","last_peered":"2024-04-25T00:06:20.695264+0000","last_clean":"2024-04-25T00:06:20.695264+0000","last_became_active":"2024-04-25T00:06:19.700392+0000","last_became_peered":"2024-04-25T00:06:19.700392+0000","last_unstale":"2024-04-25T00:06:20.695264+0000","last_undegraded":"2024-04-25T00:06:20.695264+0000","last_fullsized":"2024-04-25T00:06:20.695264+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:15.030428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.703315+0000","last_change":"2024-04-25T00:06:20.695073+0000","last_active":"2024-04-25T00:06:20.703315+0000","last_peered":"2024-04-25T00:06:20.703315+0000","last_clean":"2024-04-25T00:06:20.703315+0000","last_became_active":"2024-04-25T00:06:20.694794+0000","last_became_peered":"2024-04-25T00:06:20.694794+0000","last_unstale":"2024-04-25T00:06:20.703315+0000","last_undegraded":"2024-04-25T00:06:20.703315+0000","last_fullsized":"2024-04-25T00:06:20.703315+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:12:23.528876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.686831+0000","last_change":"2024-04-25T00:06:19.690208+0000","last_active":"2024-04-25T00:06:20.686831+0000","last_peered":"2024-04-25T00:06:20.686831+0000","last_clean":"2024-04-25T00:06:20.686831+0000","last_became_active":"2024-04-25T00:06:19.689873+0000","last_became_peered":"2024-04-25T00:06:19.689873+0000","last_unstale":"2024-04-25T00:06:20.686831+0000","last_undegraded":"2024-04-25T00:06:20.686831+0000","last_fullsized":"2024-04-25T00:06:20.686831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:59.464203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.709455+0000","last_change":"2024-04-25T00:06:20.708808+0000","last_active":"2024-04-25T00:06:20.709455+0000","last_peered":"2024-04-25T00:06:20.709455+0000","last_clean":"2024-04-25T00:06:20.709455+0000","last_became_active":"2024-04-25T00:06:20.708526+0000","last_became_peered":"2024-04-25T00:06:20.708526+0000","last_unstale":"2024-04-25T00:06:20.709455+0000","last_undegraded":"2024-04-25T00:06:20.709455+0000","last_fullsized":"2024-04-25T00:06:20.709455+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:19.676404+0000","last_clean_scrub_stamp":"2024-04-25T00:06:19.676404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:12.282006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.713983+0000","last_change":"2024-04-25T00:06:19.701735+0000","last_active":"2024-04-25T00:06:20.713983+0000","last_peered":"2024-04-25T00:06:20.713983+0000","last_clean":"2024-04-25T00:06:20.713983+0000","last_became_active":"2024-04-25T00:06:19.701418+0000","last_became_peered":"2024-04-25T00:06:19.701418+0000","last_unstale":"2024-04-25T00:06:20.713983+0000","last_undegraded":"2024-04-25T00:06:20.713983+0000","last_fullsized":"2024-04-25T00:06:20.713983+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:06:18.670081+0000","last_clean_scrub_stamp":"2024-04-25T00:06:18.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:01.669322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T00:06:20.691194+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:06:20.691194+0000","last_peered":"2024-04-25T00:06:20.691194+0000","last_clean":"2024-04-25T00:06:20.691194+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:06:20.691194+0000","last_undegraded":"2024-04-25T00:06:20.691194+0000","last_fullsized":"2024-04-25T00:06:20.691194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59724,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312116,"statfs":{"total":96636764160,"available":96575606784,"internally_reserved":0,"allocated":2228224,"data_stored":1292336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":1622016,"data_stored":701330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50788,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321052,"statfs":{"total":96636764160,"available":96584757248,"internally_reserved":0,"allocated":1597440,"data_stored":692943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59120,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312720,"statfs":{"total":96636764160,"available":96576225280,"internally_reserved":0,"allocated":1609728,"data_stored":699718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46632,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325208,"statfs":{"total":96636764160,"available":96589012992,"internally_reserved":0,"allocated":1597440,"data_stored":692943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46632,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325208,"statfs":{"total":96636764160,"available":96589012992,"internally_reserved":0,"allocated":1597440,"data_stored":692943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54960,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316880,"statfs":{"total":96636764160,"available":96580485120,"internally_reserved":0,"allocated":1609728,"data_stored":699718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29176,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342664,"statfs":{"total":96636764160,"available":96606887936,"internally_reserved":0,"allocated":2203648,"data_stored":1290086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:06:24.030 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:06:24.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:06:24.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.171+0000 7f9db79c8640 1 Processor -- start 2024-04-25T00:06:24.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.171+0000 7f9db79c8640 1 -- start start 2024-04-25T00:06:24.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db79c8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9db013f050 0x7f9db00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db79c8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db015d6b0 con 0x7f9db00d04f0 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db79c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db015c8f0 con 0x7f9db01606a0 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db79c8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9db00d09a0 con 0x7f9db013f050 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db4f63640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9db00d04f0 0x7f9db015ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47660/0 (socket says 172.21.15.113:47660) 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db4f63640 1 -- 172.21.15.113:0/2823208271 learned_addr learned my addr 172.21.15.113:0/2823208271 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db5764640 1 --2- 172.21.15.113:0/2823208271 >> v2:172.21.15.178:3300/0 conn(0x7f9db013f050 0x7f9db00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9da7fff640 1 -- 172.21.15.113:0/2823208271 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591323833 0 0) 0x7f9db015d6b0 con 0x7f9db00d04f0 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9da7fff640 1 -- 172.21.15.113:0/2823208271 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d9c003520 con 0x7f9db00d04f0 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db5764640 1 -- 172.21.15.113:0/2823208271 >> v1:172.21.15.178:6789/0 conn(0x7f9db01606a0 legacy=0x7f9db00cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db5764640 1 -- 172.21.15.113:0/2823208271 >> v1:172.21.15.113:6789/0 conn(0x7f9db00d04f0 legacy=0x7f9db015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.172+0000 7f9db5764640 1 -- 172.21.15.113:0/2823208271 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db015cbd0 con 0x7f9db013f050 2024-04-25T00:06:24.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.173+0000 7f9db5764640 1 --2- 172.21.15.113:0/2823208271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db013f050 0x7f9db00cffb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9da00097a0 tx=0x7f9da002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abfec3da95e02a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:24.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.173+0000 7f9da7fff640 1 -- 172.21.15.113:0/2823208271 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0037070 con 0x7f9db013f050 2024-04-25T00:06:24.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.173+0000 7f9da7fff640 1 -- 172.21.15.113:0/2823208271 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0032440 con 0x7f9db013f050 2024-04-25T00:06:24.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.173+0000 7f9da7fff640 1 -- 172.21.15.113:0/2823208271 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da00327f0 con 0x7f9db013f050 2024-04-25T00:06:24.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.173+0000 7f9db79c8640 1 -- 172.21.15.113:0/2823208271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db013f050 msgr2=0x7f9db00cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.174+0000 7f9db79c8640 1 --2- 172.21.15.113:0/2823208271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db013f050 0x7f9db00cffb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9da00097a0 tx=0x7f9da002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.174+0000 7f9db79c8640 1 -- 172.21.15.113:0/2823208271 shutdown_connections 2024-04-25T00:06:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.174+0000 7f9db79c8640 1 --2- 172.21.15.113:0/2823208271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db013f050 0x7f9db00cffb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.174+0000 7f9db79c8640 1 -- 172.21.15.113:0/2823208271 >> 172.21.15.113:0/2823208271 conn(0x7f9db006b0f0 msgr2=0x7f9db006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.174+0000 7f9db79c8640 1 -- 172.21.15.113:0/2823208271 shutdown_connections 2024-04-25T00:06:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.174+0000 7f9db79c8640 1 -- 172.21.15.113:0/2823208271 wait complete. 2024-04-25T00:06:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 Processor -- start 2024-04-25T00:06:24.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 -- start start 2024-04-25T00:06:24.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00d04f0 0x7f9db013ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db013f050 0x7f9db0158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db01606a0 0x7f9db0072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9db015d190 con 0x7f9db00d04f0 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9db015c6a0 con 0x7f9db01606a0 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.175+0000 7f9db79c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9db013f430 con 0x7f9db013f050 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00d04f0 0x7f9db013ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00d04f0 0x7f9db013ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56016/0 (socket says 172.21.15.113:56016) 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 -- 172.21.15.113:0/104771073 learned_addr learned my addr 172.21.15.113:0/104771073 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db4f63640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db013f050 0x7f9db0158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5f65640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db01606a0 0x7f9db0072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 -- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db013f050 msgr2=0x7f9db0158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db013f050 0x7f9db0158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 -- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db01606a0 msgr2=0x7f9db0072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db01606a0 0x7f9db0072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 -- 172.21.15.113:0/104771073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db0157330 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5f65640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db01606a0 0x7f9db0072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db5764640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00d04f0 0x7f9db013ab50 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f9da00097a0 tx=0x7f9da00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9da5ffb640 1 -- 172.21.15.113:0/104771073 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0037050 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9da5ffb640 1 -- 172.21.15.113:0/104771073 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0038040 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9da5ffb640 1 -- 172.21.15.113:0/104771073 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0004470 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db0157610 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.176+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db0157bd0 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.177+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db00691a0 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.178+0000 7f9da5ffb640 1 -- 172.21.15.113:0/104771073 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9da003c6d0 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.178+0000 7f9da5ffb640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9d9406be30 0x7f9d9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.178+0000 7f9da5ffb640 1 -- 172.21.15.113:0/104771073 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f9da00ab870 con 0x7f9db00d04f0 2024-04-25T00:06:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.178+0000 7f9db4f63640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9d9406be30 0x7f9d9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.179+0000 7f9db4f63640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9d9406be30 0x7f9d9406e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9db0073220 tx=0x7f9dac007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.183+0000 7f9da5ffb640 1 -- 172.21.15.113:0/104771073 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9da007a180 con 0x7f9db00d04f0 2024-04-25T00:06:24.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.420+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9db0152900 con 0x7f9db00d04f0 2024-04-25T00:06:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.422+0000 7f9da5ffb640 1 -- 172.21.15.113:0/104771073 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f9da0079a80 con 0x7f9db00d04f0 2024-04-25T00:06:24.422 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:06:24.422 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:06:24.422 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:06:24.422 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:06:24.422 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:06:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:06:24.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9d9406be30 msgr2=0x7f9d9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9d9406be30 0x7f9d9406e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9db0073220 tx=0x7f9dac007d30 comp rx=0 tx=0).stop 2024-04-25T00:06:24.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00d04f0 msgr2=0x7f9db013ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00d04f0 0x7f9db013ab50 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f9da00097a0 tx=0x7f9da00046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 shutdown_connections 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9d9406be30 0x7f9d9406e2f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db01606a0 0x7f9db0072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9db013f050 0x7f9db0158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 --2- 172.21.15.113:0/104771073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db00d04f0 0x7f9db013ab50 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 >> 172.21.15.113:0/104771073 conn(0x7f9db006b0f0 msgr2=0x7f9db015f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 shutdown_connections 2024-04-25T00:06:24.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.424+0000 7f9db79c8640 1 -- 172.21.15.113:0/104771073 wait complete. 2024-04-25T00:06:24.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:24.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.532+0000 7fd12bca7640 1 Processor -- start 2024-04-25T00:06:24.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.532+0000 7fd12bca7640 1 -- start start 2024-04-25T00:06:24.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd12bca7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1240b3fd0 0x7fd12406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd12bca7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12406e510 0x7fd1240cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd12bca7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd124144cf0 con 0x7fd124144910 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd12bca7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd1240b2f30 con 0x7fd1240b3fd0 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd12bca7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240b3210 con 0x7fd12406e510 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd129242640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1240b3fd0 0x7fd12406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd129242640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1240b3fd0 0x7fd12406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56022/0 (socket says 172.21.15.113:56022) 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd129242640 1 -- 172.21.15.113:0/3627478376 learned_addr learned my addr 172.21.15.113:0/3627478376 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd129242640 1 -- 172.21.15.113:0/3627478376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12406e510 msgr2=0x7fd1240cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd129242640 1 --2- 172.21.15.113:0/3627478376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12406e510 0x7fd1240cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd129242640 1 -- 172.21.15.113:0/3627478376 >> v1:172.21.15.178:6789/0 conn(0x7fd124144910 legacy=0x7fd1240b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.533+0000 7fd129242640 1 -- 172.21.15.113:0/3627478376 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1240b34f0 con 0x7fd1240b3fd0 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd129242640 1 --2- 172.21.15.113:0/3627478376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1240b3fd0 0x7fd12406dfd0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd11400bb40 tx=0x7fd11402c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb3a740da854cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd128a41640 1 -- 172.21.15.113:0/3627478376 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140327c0 con 0x7fd1240b3fd0 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd128a41640 1 -- 172.21.15.113:0/3627478376 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd114038070 con 0x7fd1240b3fd0 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd128a41640 1 -- 172.21.15.113:0/3627478376 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114032b80 con 0x7fd1240b3fd0 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd12bca7640 1 -- 172.21.15.113:0/3627478376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1240b3fd0 msgr2=0x7fd12406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3627478376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1240b3fd0 0x7fd12406dfd0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd11400bb40 tx=0x7fd11402c900 comp rx=0 tx=0).stop 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd12bca7640 1 -- 172.21.15.113:0/3627478376 shutdown_connections 2024-04-25T00:06:24.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3627478376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12406e510 0x7fd1240cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3627478376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1240b3fd0 0x7fd12406dfd0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.534+0000 7fd12bca7640 1 -- 172.21.15.113:0/3627478376 >> 172.21.15.113:0/3627478376 conn(0x7fd1240b1a50 msgr2=0x7fd1240b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:24.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.535+0000 7fd12bca7640 1 -- 172.21.15.113:0/3627478376 shutdown_connections 2024-04-25T00:06:24.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.535+0000 7fd12bca7640 1 -- 172.21.15.113:0/3627478376 wait complete. 2024-04-25T00:06:24.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 Processor -- start 2024-04-25T00:06:24.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 -- start start 2024-04-25T00:06:24.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12406e510 0x7fd12413d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1240b3fd0 0x7fd1240cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd124144910 0x7fd1240cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240b4da0 con 0x7fd124144910 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240b4440 con 0x7fd12406e510 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12bca7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240b4f20 con 0x7fd1240b3fd0 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12a244640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd124144910 0x7fd1240cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12a244640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd124144910 0x7fd1240cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56030/0 (socket says 172.21.15.113:56030) 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.536+0000 7fd12a244640 1 -- 172.21.15.113:0/3787642825 learned_addr learned my addr 172.21.15.113:0/3787642825 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd129242640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1240b3fd0 0x7fd1240cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12a244640 1 -- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1240b3fd0 msgr2=0x7fd1240cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd129a43640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12406e510 0x7fd12413d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12a244640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1240b3fd0 0x7fd1240cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12a244640 1 -- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12406e510 msgr2=0x7fd12413d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12a244640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12406e510 0x7fd12413d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12a244640 1 -- 172.21.15.113:0/3787642825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12413f300 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd129242640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1240b3fd0 0x7fd1240cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd129a43640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12406e510 0x7fd12413d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12a244640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd124144910 0x7fd1240cda00 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00b9e0 tx=0x7fd11c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd11a7fc640 1 -- 172.21.15.113:0/3787642825 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c019070 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd11a7fc640 1 -- 172.21.15.113:0/3787642825 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c00cac0 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12413e2c0 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd11a7fc640 1 -- 172.21.15.113:0/3787642825 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c014440 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.537+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12413eb00 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.538+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1240691a0 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.539+0000 7fd11a7fc640 1 -- 172.21.15.113:0/3787642825 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd11c01c070 con 0x7fd124144910 2024-04-25T00:06:24.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.539+0000 7fd11a7fc640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd10806be30 0x7fd10806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:24.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.539+0000 7fd11a7fc640 1 -- 172.21.15.113:0/3787642825 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fd11c08ab30 con 0x7fd124144910 2024-04-25T00:06:24.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.539+0000 7fd129a43640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd10806be30 0x7fd10806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:24.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.539+0000 7fd129a43640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd10806be30 0x7fd10806e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd120003eb0 tx=0x7fd120009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:24.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.543+0000 7fd11a7fc640 1 -- 172.21.15.113:0/3787642825 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd12413e2c0 con 0x7fd124144910 2024-04-25T00:06:24.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.751+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1240731a0 con 0x7fd124144910 2024-04-25T00:06:24.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.752+0000 7fd11a7fc640 1 -- 172.21.15.113:0/3787642825 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7fd11c0594e0 con 0x7fd124144910 2024-04-25T00:06:24.753 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:24.753 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:24.753 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd10806be30 msgr2=0x7fd10806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd10806be30 0x7fd10806e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd120003eb0 tx=0x7fd120009040 comp rx=0 tx=0).stop 2024-04-25T00:06:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd124144910 msgr2=0x7fd1240cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd124144910 0x7fd1240cda00 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00b9e0 tx=0x7fd11c00bea0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 shutdown_connections 2024-04-25T00:06:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd10806be30 0x7fd10806e2f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd124144910 0x7fd1240cda00 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1240b3fd0 0x7fd1240cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 --2- 172.21.15.113:0/3787642825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd12406e510 0x7fd12413d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 >> 172.21.15.113:0/3787642825 conn(0x7fd1240b1a50 msgr2=0x7fd124142aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 shutdown_connections 2024-04-25T00:06:24.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:24.754+0000 7fd12bca7640 1 -- 172.21.15.113:0/3787642825 wait complete. 2024-04-25T00:06:25.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:06:20.039117+0000', 'modified': '2024-04-25T00:06:22.029624+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8973': {'gid': 8973, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6836/2664195523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 2664195523}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 2664195523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8973, 'qdb_cluster': [8973]} 2024-04-25T00:06:25.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:06:25.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.269+0000 7f073ffb0640 1 Processor -- start 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.269+0000 7f073ffb0640 1 -- start start 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073ffb0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f073815e2d0 0x7f073815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073ffb0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07380b4a40 0x7f07380ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073ffb0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073815f4f0 con 0x7f07380b3080 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073ffb0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f07380cde00 con 0x7f073815e2d0 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073ffb0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f073815f7d0 con 0x7f07380b4a40 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073dd4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f073815e2d0 0x7f073815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073d54b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f07380b3080 0x7f07380b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47664/0 (socket says 172.21.15.113:47664) 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073d54b640 1 -- 172.21.15.113:0/2248330932 learned_addr learned my addr 172.21.15.113:0/2248330932 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073e54d640 1 --2- 172.21.15.113:0/2248330932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07380b4a40 0x7f07380ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073cd4a640 1 -- 172.21.15.113:0/2248330932 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3550197129 0 0) 0x7f073815f4f0 con 0x7f07380b3080 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073cd4a640 1 -- 172.21.15.113:0/2248330932 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0724003520 con 0x7f07380b3080 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073dd4c640 1 -- 172.21.15.113:0/2248330932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07380b4a40 msgr2=0x7f07380ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073dd4c640 1 --2- 172.21.15.113:0/2248330932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07380b4a40 0x7f07380ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073dd4c640 1 -- 172.21.15.113:0/2248330932 >> v1:172.21.15.113:6789/0 conn(0x7f07380b3080 legacy=0x7f07380b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.270+0000 7f073dd4c640 1 -- 172.21.15.113:0/2248330932 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073815fab0 con 0x7f073815e2d0 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.271+0000 7f073dd4c640 1 --2- 172.21.15.113:0/2248330932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f073815e2d0 0x7f073815ebf0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f07340097a0 tx=0x7f073402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ff83925a321645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.271+0000 7f073cd4a640 1 -- 172.21.15.113:0/2248330932 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734037070 con 0x7f073815e2d0 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.271+0000 7f073cd4a640 1 -- 172.21.15.113:0/2248330932 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0734032440 con 0x7f073815e2d0 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.271+0000 7f073cd4a640 1 -- 172.21.15.113:0/2248330932 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734032810 con 0x7f073815e2d0 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 -- 172.21.15.113:0/2248330932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f073815e2d0 msgr2=0x7f073815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 --2- 172.21.15.113:0/2248330932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f073815e2d0 0x7f073815ebf0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f07340097a0 tx=0x7f073402a180 comp rx=0 tx=0).stop 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 -- 172.21.15.113:0/2248330932 shutdown_connections 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 --2- 172.21.15.113:0/2248330932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07380b4a40 0x7f07380ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 --2- 172.21.15.113:0/2248330932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f073815e2d0 0x7f073815ebf0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 -- 172.21.15.113:0/2248330932 >> 172.21.15.113:0/2248330932 conn(0x7f073806b0f0 msgr2=0x7f073806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 -- 172.21.15.113:0/2248330932 shutdown_connections 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.272+0000 7f073ffb0640 1 -- 172.21.15.113:0/2248330932 wait complete. 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.273+0000 7f073ffb0640 1 Processor -- start 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.273+0000 7f073ffb0640 1 -- start start 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.273+0000 7f073ffb0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07380b3080 0x7f073814e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.273+0000 7f073ffb0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07380b4a40 0x7f0738165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073ffb0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f073815e2d0 0x7f07380b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073ffb0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f073815f2b0 con 0x7f07380b4a40 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073ffb0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f073815f130 con 0x7f07380b3080 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073ffb0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f073815e6b0 con 0x7f073815e2d0 2024-04-25T00:06:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07380b4a40 0x7f0738165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07380b4a40 0x7f0738165f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56050/0 (socket says 172.21.15.113:56050) 2024-04-25T00:06:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 -- 172.21.15.113:0/4233176405 learned_addr learned my addr 172.21.15.113:0/4233176405 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073dd4c640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07380b3080 0x7f073814e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073e54d640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f073815e2d0 0x7f07380b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 -- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f073815e2d0 msgr2=0x7f07380b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f073815e2d0 0x7f07380b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 -- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07380b3080 msgr2=0x7f073814e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07380b3080 0x7f073814e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 -- 172.21.15.113:0/4233176405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07381667b0 con 0x7f07380b4a40 2024-04-25T00:06:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073e54d640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f073815e2d0 0x7f07380b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073dd4c640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07380b3080 0x7f073814e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:25.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073d54b640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07380b4a40 0x7f0738165f80 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f072800c430 tx=0x7f072800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f072e7fc640 1 -- 172.21.15.113:0/4233176405 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728017070 con 0x7f07380b4a40 2024-04-25T00:06:25.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f072e7fc640 1 -- 172.21.15.113:0/4233176405 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0728005ce0 con 0x7f07380b4a40 2024-04-25T00:06:25.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0738166a90 con 0x7f07380b4a40 2024-04-25T00:06:25.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f072e7fc640 1 -- 172.21.15.113:0/4233176405 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07280072f0 con 0x7f07380b4a40 2024-04-25T00:06:25.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.274+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0738167050 con 0x7f07380b4a40 2024-04-25T00:06:25.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.276+0000 7f072e7fc640 1 -- 172.21.15.113:0/4233176405 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f072801a070 con 0x7f07380b4a40 2024-04-25T00:06:25.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.276+0000 7f072e7fc640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f071c06be30 0x7f071c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.276+0000 7f072e7fc640 1 -- 172.21.15.113:0/4233176405 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f072808a360 con 0x7f07380b4a40 2024-04-25T00:06:25.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.276+0000 7f073dd4c640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f071c06be30 0x7f071c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.276+0000 7f073dd4c640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f071c06be30 0x7f071c06e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0734034000 tx=0x7f073402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.277+0000 7f0742da0740 1 Processor -- start 2024-04-25T00:06:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.277+0000 7f0742da0740 1 -- start start 2024-04-25T00:06:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b2c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f0742da0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b5cc80 0x55ede0b5c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5c540 0x55ede0b31350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f0742da0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ede0a94f00 con 0x55ede0b5cc80 2024-04-25T00:06:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ede0a3d750 con 0x55ede0b5c540 2024-04-25T00:06:25.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ede0a90ec0 con 0x55ede0b79b60 2024-04-25T00:06:25.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b5cc80 0x55ede0b5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b5cc80 0x55ede0b5c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56052/0 (socket says 172.21.15.113:56052) 2024-04-25T00:06:25.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 -- 172.21.15.113:0/4147042373 learned_addr learned my addr 172.21.15.113:0/4147042373 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:25.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073e54d640 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5c540 0x55ede0b31350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073dd4c640 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b2c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 -- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 msgr2=0x55ede0b2c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b2c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 -- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5c540 msgr2=0x55ede0b31350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5c540 0x55ede0b31350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 -- 172.21.15.113:0/4147042373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede07a92d0 con 0x55ede0b5cc80 2024-04-25T00:06:25.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073e54d640 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5c540 0x55ede0b31350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:25.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073dd4c640 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b2c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:25.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f073d54b640 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b5cc80 0x55ede0b5c000 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f07280888b0 tx=0x7f07280586d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb74fa30ebec575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.278+0000 7f071b7fe640 1 -- 172.21.15.113:0/4147042373 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728056910 con 0x55ede0b5cc80 2024-04-25T00:06:25.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f071b7fe640 1 -- 172.21.15.113:0/4147042373 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072808a0c0 con 0x55ede0b5cc80 2024-04-25T00:06:25.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f071b7fe640 1 -- 172.21.15.113:0/4147042373 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728007490 con 0x55ede0b5cc80 2024-04-25T00:06:25.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 -- 172.21.15.113:0/4147042373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b5cc80 msgr2=0x55ede0b5c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b5cc80 0x55ede0b5c000 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f07280888b0 tx=0x7f07280586d0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 -- 172.21.15.113:0/4147042373 shutdown_connections 2024-04-25T00:06:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5c540 0x55ede0b31350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b5cc80 0x55ede0b5c000 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 --2- 172.21.15.113:0/4147042373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b2c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 -- 172.21.15.113:0/4147042373 >> 172.21.15.113:0/4147042373 conn(0x55ede0b2e760 msgr2=0x55ede0b77000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 -- 172.21.15.113:0/4147042373 shutdown_connections 2024-04-25T00:06:25.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.279+0000 7f0742da0740 1 -- 172.21.15.113:0/4147042373 wait complete. 2024-04-25T00:06:25.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.280+0000 7f0742da0740 1 -- start start 2024-04-25T00:06:25.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.280+0000 7f0742da0740 1 Processor -- start 2024-04-25T00:06:25.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b79b60 0x55ede0b5b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b5ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ede0b30910 con 0x55ede0b87820 2024-04-25T00:06:25.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ede0a50cf0 con 0x55ede0b79b60 2024-04-25T00:06:25.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ede0b2fd00 con 0x55ede0b5cc80 2024-04-25T00:06:25.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 10 client.0 _ll_register_callbacks cb 0x55ede0a506e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:25.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b5ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b5ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56054/0 (socket says 172.21.15.113:56054) 2024-04-25T00:06:25.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 -- 172.21.15.113:0/2448784626 learned_addr learned my addr 172.21.15.113:0/2448784626 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:25.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073dd4c640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073d54b640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b79b60 0x55ede0b5b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 -- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 msgr2=0x55ede0b872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 -- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b79b60 msgr2=0x55ede0b5b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b79b60 0x55ede0b5b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 -- 172.21.15.113:0/2448784626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede08736b0 con 0x55ede0b87820 2024-04-25T00:06:25.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073dd4c640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:25.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073d54b640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b79b60 0x55ede0b5b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:25.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f073e54d640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b5ba70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f0730007dd0 tx=0x7f0730012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a264b36078e8ef29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0718ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:25.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0718ff9640 1 -- 172.21.15.113:0/2448784626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073001a070 con 0x55ede0b87820 2024-04-25T00:06:25.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f072808a0c0 con 0x55ede0b87820 2024-04-25T00:06:25.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0718ff9640 1 -- 172.21.15.113:0/2448784626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f0730016440 con 0x55ede0b87820 2024-04-25T00:06:25.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.281+0000 7f0718ff9640 1 -- 172.21.15.113:0/2448784626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f0730025050 con 0x55ede0b87820 2024-04-25T00:06:25.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 10 client.9117 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:25.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55ede0b5d210 con 0x55ede0b87820 2024-04-25T00:06:25.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0718ff9640 1 -- 172.21.15.113:0/2448784626 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072808a0c0 con 0x55ede0b87820 2024-04-25T00:06:25.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0718ff9640 1 -- 172.21.15.113:0/2448784626 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f0730028020 con 0x55ede0b87820 2024-04-25T00:06:25.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 10 client.9117 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:25.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 10 client.9117 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:06:25.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 20 client.9117 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b77390 0x55ede0b5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 4 client.9117 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ede0aa2440 con 0x55ede0b77390 2024-04-25T00:06:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f073dd4c640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b77390 0x55ede0b5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f0718ff9640 10 client.9117 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:25.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.282+0000 7f073dd4c640 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b77390 0x55ede0b5e350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1461b4b763b87a1a server_cookie=ab45f943d0ca54e2 in_seq=0 out_seq=0 2024-04-25T00:06:25.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0718ff9640 1 -- 172.21.15.113:0/2448784626 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ede0aa2440 con 0x55ede0b77390 2024-04-25T00:06:25.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0718ff9640 10 client.9117 handle_command_reply: tid=0 2024-04-25T00:06:25.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 client.9117 shutdown 2024-04-25T00:06:25.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 msgr2=0x55ede0b5ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b5ba70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f0730007dd0 tx=0x7f0730012610 comp rx=0 tx=0).stop 2024-04-25T00:06:25.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 shutdown_connections 2024-04-25T00:06:25.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b77390 0x55ede0b5e350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b79b60 0x55ede0b5b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b5ba70 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0718ff9640 0 client.9117 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:25.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 --2- 172.21.15.113:0/2448784626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 >> 172.21.15.113:0/2448784626 conn(0x55ede0b5c540 msgr2=0x55ede0b2bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 shutdown_connections 2024-04-25T00:06:25.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 1 -- 172.21.15.113:0/2448784626 wait complete. 2024-04-25T00:06:25.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.283+0000 7f0742da0740 20 client.9117 trim_cache size 0 max 16384 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.301+0000 7f0742da0740 1 Processor -- start 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.301+0000 7f0742da0740 1 -- start start 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5cc80 0x55ede0b78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b83f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f0742da0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f0742da0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f07381493d0 con 0x55ede0b87820 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0738148730 con 0x55ede0b5cc80 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0738148d80 con 0x55ede0b79b60 2024-04-25T00:06:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56064/0 (socket says 172.21.15.113:56064) 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 -- 172.21.15.113:0/2154603420 learned_addr learned my addr 172.21.15.113:0/2154603420 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 -- 172.21.15.113:0/2154603420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 msgr2=0x55ede0b83f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 --2- 172.21.15.113:0/2154603420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b83f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 -- 172.21.15.113:0/2154603420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5cc80 msgr2=0x55ede0b78810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 --2- 172.21.15.113:0/2154603420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5cc80 0x55ede0b78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 -- 172.21.15.113:0/2154603420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede0b2f8a0 con 0x55ede0b87820 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f073e54d640 1 --2- 172.21.15.113:0/2154603420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b844b0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0730002230 tx=0x7f0730017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7559750832c0ee93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f071affd640 1 -- 172.21.15.113:0/2154603420 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073002d070 con 0x55ede0b87820 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f071affd640 1 -- 172.21.15.113:0/2154603420 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0730012ca0 con 0x55ede0b87820 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.302+0000 7f071affd640 1 -- 172.21.15.113:0/2154603420 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073001a040 con 0x55ede0b87820 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 -- 172.21.15.113:0/2154603420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 msgr2=0x55ede0b844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 --2- 172.21.15.113:0/2154603420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b844b0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0730002230 tx=0x7f0730017000 comp rx=0 tx=0).stop 2024-04-25T00:06:25.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 -- 172.21.15.113:0/2154603420 shutdown_connections 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 --2- 172.21.15.113:0/2154603420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b87820 0x55ede0b844b0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 --2- 172.21.15.113:0/2154603420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b79b60 0x55ede0b83f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 --2- 172.21.15.113:0/2154603420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b5cc80 0x55ede0b78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 -- 172.21.15.113:0/2154603420 >> 172.21.15.113:0/2154603420 conn(0x55ede0b5c540 msgr2=0x55ede0b27690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 -- 172.21.15.113:0/2154603420 shutdown_connections 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 -- 172.21.15.113:0/2154603420 wait complete. 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.303+0000 7f0742da0740 1 -- start start 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 1 Processor -- start 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b9afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b79b60 0x55ede0b57fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b87820 0x55ede0b584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ede0b77210 con 0x55ede0b79b60 2024-04-25T00:06:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ede0b5d4f0 con 0x55ede0b87820 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ede0b77860 con 0x55ede0b5cc80 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f0742da0740 10 client.0 _ll_register_callbacks cb 0x7f07300079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.304+0000 7f073d54b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b79b60 0x55ede0b57fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b79b60 0x55ede0b57fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56074/0 (socket says 172.21.15.113:56074) 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 -- 172.21.15.113:0/1703268053 learned_addr learned my addr 172.21.15.113:0/1703268053 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073dd4c640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b9afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073e54d640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b87820 0x55ede0b584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 -- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 msgr2=0x55ede0b9afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b9afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 -- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b87820 msgr2=0x55ede0b584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b87820 0x55ede0b584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 -- 172.21.15.113:0/1703268053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede0b5f190 con 0x55ede0b79b60 2024-04-25T00:06:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073e54d640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b87820 0x55ede0b584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f073d54b640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b79b60 0x55ede0b57fa0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0728000f30 tx=0x7f0728088760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db6d651c6865a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0718ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0718ff9640 1 -- 172.21.15.113:0/1703268053 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728007690 con 0x55ede0b79b60 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0718ff9640 1 -- 172.21.15.113:0/1703268053 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f072805db50 con 0x55ede0b79b60 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0730012ca0 con 0x55ede0b79b60 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0718ff9640 1 -- 172.21.15.113:0/1703268053 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f0728056910 con 0x55ede0b79b60 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 10 client.9123 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55ede0b2b250 con 0x55ede0b79b60 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0718ff9640 1 -- 172.21.15.113:0/1703268053 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0730012ca0 con 0x55ede0b79b60 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0718ff9640 1 -- 172.21.15.113:0/1703268053 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f0728057a60 con 0x55ede0b79b60 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 10 client.9123 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 10 client.9123 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 20 client.9123 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b91010 0x55ede0b83a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 4 client.9123 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.305+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "status"}) -- 0x55ede0b2dbb0 con 0x55ede0b91010 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f073dd4c640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b91010 0x55ede0b83a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0718ff9640 10 client.9123 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f073dd4c640 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b91010 0x55ede0b83a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9232f33a5565e31 server_cookie=355317e36f3f0c42 in_seq=0 out_seq=0 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0718ff9640 1 -- 172.21.15.113:0/1703268053 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55ede0b2dbb0 con 0x55ede0b91010 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0718ff9640 10 client.9123 handle_command_reply: tid=0 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 client.9123 shutdown 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b79b60 msgr2=0x55ede0b57fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b79b60 0x55ede0b57fa0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0728000f30 tx=0x7f0728088760 comp rx=0 tx=0).stop 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 shutdown_connections 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55ede0b91010 0x55ede0b83a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ede0b87820 0x55ede0b584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0718ff9640 0 client.9123 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ede0b79b60 0x55ede0b57fa0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 --2- 172.21.15.113:0/1703268053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ede0b5cc80 0x55ede0b9afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 >> 172.21.15.113:0/1703268053 conn(0x55ede0b5c540 msgr2=0x55ede0b789f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 shutdown_connections 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 1 -- 172.21.15.113:0/1703268053 wait complete. 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.306+0000 7f0742da0740 20 client.9123 trim_cache size 0 max 16384 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:06:25.338 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 8973, 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.6232832269999999, 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 68, 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 130, 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 130, 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 9.6868137500000007 2024-04-25T00:06:25.339 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:06:25.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f071c06be30 msgr2=0x7f071c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f071c06be30 0x7f071c06e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0734034000 tx=0x7f073402e000 comp rx=0 tx=0).stop 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07380b4a40 msgr2=0x7f0738165f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07380b4a40 0x7f0738165f80 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f072800c430 tx=0x7f072800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 shutdown_connections 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f071c06be30 0x7f071c06e2f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f073815e2d0 0x7f07380b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07380b4a40 0x7f0738165f80 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 --2- 172.21.15.113:0/4233176405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07380b3080 0x7f073814e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 >> 172.21.15.113:0/4233176405 conn(0x7f073806b0f0 msgr2=0x7f073806cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 shutdown_connections 2024-04-25T00:06:25.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.307+0000 7f073ffb0640 1 -- 172.21.15.113:0/4233176405 wait complete. 2024-04-25T00:06:25.342 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:06:25.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.438+0000 7fb4eed12640 1 Processor -- start 2024-04-25T00:06:25.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.438+0000 7fb4eed12640 1 -- start start 2024-04-25T00:06:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4eed12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4e815a6b0 0x7fb4e8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4eed12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4eed12640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4e80b2d50 con 0x7fb4e8072ea0 2024-04-25T00:06:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4eed12640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e8072a70 con 0x7fb4e815a6b0 2024-04-25T00:06:25.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4eed12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e80b3010 con 0x7fb4e80b3af0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4e7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb4e8072ea0 0x7fb4e8071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47666/0 (socket says 172.21.15.113:47666) 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4ecaae640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4e815a6b0 0x7fb4e8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4e7fff640 1 -- 172.21.15.113:0/4276553851 learned_addr learned my addr 172.21.15.113:0/4276553851 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4ed2af640 1 --2- 172.21.15.113:0/4276553851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4ecaae640 1 -- 172.21.15.113:0/4276553851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 msgr2=0x7fb4e80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4ecaae640 1 --2- 172.21.15.113:0/4276553851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e80b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4ecaae640 1 -- 172.21.15.113:0/4276553851 >> v1:172.21.15.113:6789/0 conn(0x7fb4e8072ea0 legacy=0x7fb4e8071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4ecaae640 1 -- 172.21.15.113:0/4276553851 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e80b32f0 con 0x7fb4e815a6b0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4e77fe640 1 -- 172.21.15.113:0/4276553851 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011411915 0 0) 0x7fb4e80b2d50 con 0x7fb4e8072ea0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.439+0000 7fb4ed2af640 1 --2- 172.21.15.113:0/4276553851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4ecaae640 1 --2- 172.21.15.113:0/4276553851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e8070d00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0097a0 tx=0x7fb4dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3365beceec0dccdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4e77fe640 1 -- 172.21.15.113:0/4276553851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc037070 con 0x7fb4e815a6b0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4e77fe640 1 -- 172.21.15.113:0/4276553851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4dc032440 con 0x7fb4e815a6b0 2024-04-25T00:06:25.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4e77fe640 1 -- 172.21.15.113:0/4276553851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc032810 con 0x7fb4e815a6b0 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4eed12640 1 -- 172.21.15.113:0/4276553851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 msgr2=0x7fb4e8070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4eed12640 1 --2- 172.21.15.113:0/4276553851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e8070d00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0097a0 tx=0x7fb4dc02a180 comp rx=0 tx=0).stop 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4eed12640 1 -- 172.21.15.113:0/4276553851 shutdown_connections 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4eed12640 1 --2- 172.21.15.113:0/4276553851 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4eed12640 1 --2- 172.21.15.113:0/4276553851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e8070d00 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.440+0000 7fb4eed12640 1 -- 172.21.15.113:0/4276553851 >> 172.21.15.113:0/4276553851 conn(0x7fb4e80b1a50 msgr2=0x7fb4e80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.441+0000 7fb4eed12640 1 -- 172.21.15.113:0/4276553851 shutdown_connections 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.441+0000 7fb4eed12640 1 -- 172.21.15.113:0/4276553851 wait complete. 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.442+0000 7fb4eed12640 1 Processor -- start 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.442+0000 7fb4eed12640 1 -- start start 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.442+0000 7fb4eed12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e8072ea0 0x7fb4e8155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4eed12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e81575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4eed12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e81f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4eed12640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e815aa90 con 0x7fb4e815a6b0 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4eed12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8072720 con 0x7fb4e8072ea0 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4eed12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e815ac10 con 0x7fb4e80b3af0 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e81f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e81f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56102/0 (socket says 172.21.15.113:56102) 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 -- 172.21.15.113:0/3010774779 learned_addr learned my addr 172.21.15.113:0/3010774779 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4e7fff640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e81575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ecaae640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e8072ea0 0x7fb4e8155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 -- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 msgr2=0x7fb4e81575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e81575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 -- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e8072ea0 msgr2=0x7fb4e8155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e8072ea0 0x7fb4e8155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 -- 172.21.15.113:0/3010774779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e81567f0 con 0x7fb4e815a6b0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4e7fff640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e81575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4ed2af640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e81f53e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0007de0 tx=0x7fb4e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4e57fa640 1 -- 172.21.15.113:0/3010774779 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e001a070 con 0x7fb4e815a6b0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4e57fa640 1 -- 172.21.15.113:0/3010774779 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e0003990 con 0x7fb4e815a6b0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e8156ad0 con 0x7fb4e815a6b0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4e57fa640 1 -- 172.21.15.113:0/3010774779 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0003d80 con 0x7fb4e815a6b0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.443+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e8155d60 con 0x7fb4e815a6b0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.444+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e80691a0 con 0x7fb4e815a6b0 2024-04-25T00:06:25.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.445+0000 7fb4e57fa640 1 -- 172.21.15.113:0/3010774779 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4e001d070 con 0x7fb4e815a6b0 2024-04-25T00:06:25.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.445+0000 7fb4e57fa640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4cc06be30 0x7fb4cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.445+0000 7fb4e57fa640 1 -- 172.21.15.113:0/3010774779 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fb4e008b150 con 0x7fb4e815a6b0 2024-04-25T00:06:25.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.445+0000 7fb4ecaae640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4cc06be30 0x7fb4cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.445+0000 7fb4ecaae640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4cc06be30 0x7fb4cc06e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc034000 tx=0x7fb4dc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.449+0000 7fb4e57fa640 1 -- 172.21.15.113:0/3010774779 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb4e0010070 con 0x7fb4e815a6b0 2024-04-25T00:06:25.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.659+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb4e80b2900 con 0x7fb4e815a6b0 2024-04-25T00:06:25.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.659+0000 7fb4e57fa640 1 -- 172.21.15.113:0/3010774779 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7fb4e0059b90 con 0x7fb4e815a6b0 2024-04-25T00:06:25.660 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4cc06be30 msgr2=0x7fb4cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4cc06be30 0x7fb4cc06e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc034000 tx=0x7fb4dc02e000 comp rx=0 tx=0).stop 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 msgr2=0x7fb4e81f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e81f53e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0007de0 tx=0x7fb4e000d4a0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 shutdown_connections 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4cc06be30 0x7fb4cc06e2f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4e815a6b0 0x7fb4e81f53e0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4e80b3af0 0x7fb4e81575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 --2- 172.21.15.113:0/3010774779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4e8072ea0 0x7fb4e8155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 >> 172.21.15.113:0/3010774779 conn(0x7fb4e80b1a50 msgr2=0x7fb4e80cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 shutdown_connections 2024-04-25T00:06:25.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25.661+0000 7fb4eed12640 1 -- 172.21.15.113:0/3010774779 wait complete. 2024-04-25T00:06:25.672 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:06:25.672 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:06:25.673 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:06:25.673 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:06:25.673 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:06:25.673 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:06:25.673 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:06:25.673 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:06:25.673 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:06:25.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:25.673 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:06:25.690 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:06:25.690 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:06:25.690 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:06:25.690 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:06:25.690 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:06:25.690 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42295sec preferred_lft 42295sec 2024-04-25T00:06:25.691 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:06:25.692 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:06:25.693 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:06:25.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:25.693 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:06:25.752 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:06:25.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:25.754 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:06:25.809 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:06:25.809 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:06:25.809 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:06:25.809 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:06:25.809 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:06:25.810 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:06:25.810 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:25.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:06:25.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:06:25.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:06:25.810 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:25.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:06:25.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:25.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:25.933 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:06:25.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:06:25.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:06:25.947 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:06:25.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:25.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:06:25.959 INFO:tasks.ceph.mds.c.smithi113.stderr:2024-04-25T00:06:25.959+0000 7f827e7ad640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:06:25.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:06:25.982 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:06:26.037 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:06:26.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:06:26.037 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:06:26.091 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:06:26.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:26.092 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:06:26.171 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:06:26.172 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:06:26.172 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:06:26.180 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:06:26.180 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:06:26.180 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:06:26.180 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:06:26.180 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:06:26.187 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:06:26.187 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:06:26.187 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:06:26.187 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:06:26.187 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:06:26.188 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:06:26.188 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:06:26.188 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:06:26.188 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:06:26.188 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:06:26.188 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:06:26.188 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:06:26.189 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:06:26.190 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:06:26.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:26.191 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:06:26.255 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:06:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:26.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:26.356 INFO:teuthology.orchestra.run.smithi113.stdout:client7471 2024-04-25T00:06:26.356 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:06:26.356 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:06:26.371 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.508+0000 7f980f6ae640 1 Processor -- start 2024-04-25T00:06:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.508+0000 7f980f6ae640 1 -- start start 2024-04-25T00:06:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980f6ae640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f980815e320 0x7f98080d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980f6ae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98080d2290 0x7f98080b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980f6ae640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98080d3de0 con 0x7f98080d0b00 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980f6ae640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f98080d3410 con 0x7f980815e320 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980f6ae640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f98080d2800 con 0x7f98080d2290 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980d44a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f980815e320 0x7f98080d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980cc49640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98080d2290 0x7f98080b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980dc4b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f98080d0b00 0x7f98080b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:40980/0 (socket says 172.21.15.113:40980) 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980d44a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f980815e320 0x7f98080d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55672/0 (socket says 172.21.15.113:55672) 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980dc4b640 1 -- 172.21.15.113:0/4103663170 learned_addr learned my addr 172.21.15.113:0/4103663170 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980d44a640 1 -- 172.21.15.113:0/4103663170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98080d2290 msgr2=0x7f98080b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980d44a640 1 --2- 172.21.15.113:0/4103663170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98080d2290 0x7f98080b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980d44a640 1 -- 172.21.15.113:0/4103663170 >> v1:172.21.15.178:6789/0 conn(0x7f98080d0b00 legacy=0x7f98080b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f97fffff640 1 -- 172.21.15.113:0/4103663170 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 100913338 0 0) 0x7f98080d3de0 con 0x7f98080d0b00 2024-04-25T00:06:26.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.509+0000 7f980d44a640 1 -- 172.21.15.113:0/4103663170 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98080d2ae0 con 0x7f980815e320 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.510+0000 7f980d44a640 1 --2- 172.21.15.113:0/4103663170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980815e320 0x7f98080d38a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f98040097a0 tx=0x7f980402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30170d61328335aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.510+0000 7f97fffff640 1 -- 172.21.15.113:0/4103663170 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804037070 con 0x7f980815e320 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.510+0000 7f97fffff640 1 -- 172.21.15.113:0/4103663170 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9804032440 con 0x7f980815e320 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.510+0000 7f97fffff640 1 -- 172.21.15.113:0/4103663170 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804032810 con 0x7f980815e320 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 -- 172.21.15.113:0/4103663170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980815e320 msgr2=0x7f98080d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 --2- 172.21.15.113:0/4103663170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980815e320 0x7f98080d38a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f98040097a0 tx=0x7f980402a210 comp rx=0 tx=0).stop 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 -- 172.21.15.113:0/4103663170 shutdown_connections 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 --2- 172.21.15.113:0/4103663170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98080d2290 0x7f98080b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 --2- 172.21.15.113:0/4103663170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980815e320 0x7f98080d38a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 -- 172.21.15.113:0/4103663170 >> 172.21.15.113:0/4103663170 conn(0x7f98080b5c20 msgr2=0x7f98080b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 -- 172.21.15.113:0/4103663170 shutdown_connections 2024-04-25T00:06:26.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.511+0000 7f980f6ae640 1 -- 172.21.15.113:0/4103663170 wait complete. 2024-04-25T00:06:26.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.512+0000 7f980f6ae640 1 Processor -- start 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.512+0000 7f980f6ae640 1 -- start start 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980f6ae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98080d0b00 0x7f980814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980f6ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98080d2290 0x7f980815ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980f6ae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f980815e320 0x7f98080cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980f6ae640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98080b9fe0 con 0x7f98080d2290 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980f6ae640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98080d30a0 con 0x7f98080d0b00 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980f6ae640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f980815e700 con 0x7f980815e320 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98080d2290 0x7f980815ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98080d2290 0x7f980815ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56108/0 (socket says 172.21.15.113:56108) 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 -- 172.21.15.113:0/3608798050 learned_addr learned my addr 172.21.15.113:0/3608798050 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980dc4b640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f980815e320 0x7f98080cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980d44a640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98080d0b00 0x7f980814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 -- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f980815e320 msgr2=0x7f98080cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f980815e320 0x7f98080cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 -- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98080d0b00 msgr2=0x7f980814ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98080d0b00 0x7f980814ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 -- 172.21.15.113:0/3608798050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980815fc80 con 0x7f98080d2290 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980cc49640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98080d2290 0x7f980815ea00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e9b0 tx=0x7f97f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f97fdffb640 1 -- 172.21.15.113:0/3608798050 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8017070 con 0x7f98080d2290 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f97fdffb640 1 -- 172.21.15.113:0/3608798050 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f800c9a0 con 0x7f98080d2290 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980815ff60 con 0x7f98080d2290 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.513+0000 7f97fdffb640 1 -- 172.21.15.113:0/3608798050 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f800cd90 con 0x7f98080d2290 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.514+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980814e830 con 0x7f98080d2290 2024-04-25T00:06:26.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.515+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98080691a0 con 0x7f98080d2290 2024-04-25T00:06:26.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.515+0000 7f97fdffb640 1 -- 172.21.15.113:0/3608798050 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97f801a070 con 0x7f98080d2290 2024-04-25T00:06:26.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.515+0000 7f97fdffb640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97ec06be60 0x7f97ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.515+0000 7f97fdffb640 1 -- 172.21.15.113:0/3608798050 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f97f808b1d0 con 0x7f98080d2290 2024-04-25T00:06:26.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.515+0000 7f980d44a640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97ec06be60 0x7f97ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.516+0000 7f980d44a640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97ec06be60 0x7f97ec06e320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f980402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.520+0000 7f97fdffb640 1 -- 172.21.15.113:0/3608798050 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97f80594b0 con 0x7f98080d2290 2024-04-25T00:06:26.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.728+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98080d1990 con 0x7f98080d2290 2024-04-25T00:06:26.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.728+0000 7f97fdffb640 1 -- 172.21.15.113:0/3608798050 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7f97f805d7b0 con 0x7f98080d2290 2024-04-25T00:06:26.729 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:26.729 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:26.729 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:26.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97ec06be60 msgr2=0x7f97ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97ec06be60 0x7f97ec06e320 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f980402e000 comp rx=0 tx=0).stop 2024-04-25T00:06:26.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98080d2290 msgr2=0x7f980815ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98080d2290 0x7f980815ea00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e9b0 tx=0x7f97f800ee70 comp rx=0 tx=0).stop 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 shutdown_connections 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97ec06be60 0x7f97ec06e320 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f980815e320 0x7f98080cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98080d2290 0x7f980815ea00 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 --2- 172.21.15.113:0/3608798050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98080d0b00 0x7f980814ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 >> 172.21.15.113:0/3608798050 conn(0x7f98080b5c20 msgr2=0x7f98080b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 shutdown_connections 2024-04-25T00:06:26.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.731+0000 7f980f6ae640 1 -- 172.21.15.113:0/3608798050 wait complete. 2024-04-25T00:06:26.742 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:26.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.837+0000 7f098359e640 1 Processor -- start 2024-04-25T00:06:26.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.837+0000 7f098359e640 1 -- start start 2024-04-25T00:06:26.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.838+0000 7f098359e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f098406d510 0x7f09840cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.838+0000 7f098359e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09840d1080 con 0x7f098406e730 2024-04-25T00:06:26.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.838+0000 7f098359e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09840cf8d0 con 0x7f0984161750 2024-04-25T00:06:26.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.838+0000 7f098359e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f09840cfbb0 con 0x7f098406d510 2024-04-25T00:06:26.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.839+0000 7f0981d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f098406d510 0x7f09840cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.839+0000 7f0982d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f098406e730 0x7f09840ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47670/0 (socket says 172.21.15.113:47670) 2024-04-25T00:06:26.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.839+0000 7f0982d9d640 1 -- 172.21.15.113:0/1084632951 learned_addr learned my addr 172.21.15.113:0/1084632951 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:26.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.839+0000 7f0981d9b640 1 -- 172.21.15.113:0/1084632951 >> v1:172.21.15.178:6789/0 conn(0x7f0984161750 legacy=0x7f09840d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:26.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.839+0000 7f0981d9b640 1 -- 172.21.15.113:0/1084632951 >> v1:172.21.15.113:6789/0 conn(0x7f098406e730 legacy=0x7f09840ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.839+0000 7f0981d9b640 1 -- 172.21.15.113:0/1084632951 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09840cfe90 con 0x7f098406d510 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f0981d9b640 1 --2- 172.21.15.113:0/1084632951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406d510 0x7f09840cf2a0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f09740097a0 tx=0x7f097402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa46e4537f57b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098159a640 1 -- 172.21.15.113:0/1084632951 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974037070 con 0x7f098406d510 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098159a640 1 -- 172.21.15.113:0/1084632951 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0974032440 con 0x7f098406d510 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098159a640 1 -- 172.21.15.113:0/1084632951 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974032810 con 0x7f098406d510 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098359e640 1 -- 172.21.15.113:0/1084632951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406d510 msgr2=0x7f09840cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098359e640 1 --2- 172.21.15.113:0/1084632951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406d510 0x7f09840cf2a0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f09740097a0 tx=0x7f097402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098359e640 1 -- 172.21.15.113:0/1084632951 shutdown_connections 2024-04-25T00:06:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098359e640 1 --2- 172.21.15.113:0/1084632951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406d510 0x7f09840cf2a0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098359e640 1 -- 172.21.15.113:0/1084632951 >> 172.21.15.113:0/1084632951 conn(0x7f09840b2070 msgr2=0x7f09840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098359e640 1 -- 172.21.15.113:0/1084632951 shutdown_connections 2024-04-25T00:06:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.840+0000 7f098359e640 1 -- 172.21.15.113:0/1084632951 wait complete. 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 Processor -- start 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 -- start start 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f098406d510 0x7f09841e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406e730 0x7f0984146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0984161750 0x7f09841496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09840cd580 con 0x7f098406e730 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09840d0450 con 0x7f098406d510 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f098359e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09840cd700 con 0x7f0984161750 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f0981d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406e730 0x7f0984146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f0981d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406e730 0x7f0984146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56126/0 (socket says 172.21.15.113:56126) 2024-04-25T00:06:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.842+0000 7f0981d9b640 1 -- 172.21.15.113:0/1759414678 learned_addr learned my addr 172.21.15.113:0/1759414678 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0981d9b640 1 -- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0984161750 msgr2=0x7f09841496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0981d9b640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0984161750 0x7f09841496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f098259c640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f098406d510 0x7f09841e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0981d9b640 1 -- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f098406d510 msgr2=0x7f09841e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0981d9b640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f098406d510 0x7f09841e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0981d9b640 1 -- 172.21.15.113:0/1759414678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0984146950 con 0x7f098406e730 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f098259c640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f098406d510 0x7f09841e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0981d9b640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406e730 0x7f0984146410 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f0974000c00 tx=0x7f09740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0972ffd640 1 -- 172.21.15.113:0/1759414678 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974037040 con 0x7f098406e730 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0972ffd640 1 -- 172.21.15.113:0/1759414678 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0974042070 con 0x7f098406e730 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0984149c00 con 0x7f098406e730 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f0972ffd640 1 -- 172.21.15.113:0/1759414678 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974030a70 con 0x7f098406e730 2024-04-25T00:06:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.843+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09841e6d00 con 0x7f098406e730 2024-04-25T00:06:26.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.844+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09840691a0 con 0x7f098406e730 2024-04-25T00:06:26.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.844+0000 7f0972ffd640 1 -- 172.21.15.113:0/1759414678 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0974030c10 con 0x7f098406e730 2024-04-25T00:06:26.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.845+0000 7f0972ffd640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f096406be60 0x7f096406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.845+0000 7f0972ffd640 1 -- 172.21.15.113:0/1759414678 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f09740ab890 con 0x7f098406e730 2024-04-25T00:06:26.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.845+0000 7f098259c640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f096406be60 0x7f096406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.845+0000 7f098259c640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f096406be60 0x7f096406e320 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f097800a830 tx=0x7f0978008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:26.849+0000 7f0972ffd640 1 -- 172.21.15.113:0/1759414678 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0974045360 con 0x7f098406e730 2024-04-25T00:06:27.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.062+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f098406c490 con 0x7f098406e730 2024-04-25T00:06:27.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.062+0000 7f0972ffd640 1 -- 172.21.15.113:0/1759414678 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7f097407a240 con 0x7f098406e730 2024-04-25T00:06:27.064 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:27.064 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:27.064 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:27.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.064+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f096406be60 msgr2=0x7f096406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:27.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.064+0000 7f098359e640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f096406be60 0x7f096406e320 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f097800a830 tx=0x7f0978008040 comp rx=0 tx=0).stop 2024-04-25T00:06:27.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.064+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406e730 msgr2=0x7f0984146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:27.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.064+0000 7f098359e640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406e730 0x7f0984146410 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f0974000c00 tx=0x7f09740046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.064+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 shutdown_connections 2024-04-25T00:06:27.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.064+0000 7f098359e640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f096406be60 0x7f096406e320 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.065+0000 7f098359e640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0984161750 0x7f09841496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.065+0000 7f098359e640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f098406e730 0x7f0984146410 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.065+0000 7f098359e640 1 --2- 172.21.15.113:0/1759414678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f098406d510 0x7f09841e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.065+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 >> 172.21.15.113:0/1759414678 conn(0x7f09840b2070 msgr2=0x7f09840d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:27.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.065+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 shutdown_connections 2024-04-25T00:06:27.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27.065+0000 7f098359e640 1 -- 172.21.15.113:0/1759414678 wait complete. 2024-04-25T00:06:27.077 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:27.077 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:27.077 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T00:06:27.077 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:27.077 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:27.077 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:27.077 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:27.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:27.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975424 s, 107 MB/s 2024-04-25T00:06:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775282 s, 135 MB/s 2024-04-25T00:06:27.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.164 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583237 s, 180 MB/s 2024-04-25T00:06:27.172 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666939 s, 157 MB/s 2024-04-25T00:06:27.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.182 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072577 s, 144 MB/s 2024-04-25T00:06:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611562 s, 171 MB/s 2024-04-25T00:06:27.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.198 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631396 s, 166 MB/s 2024-04-25T00:06:27.207 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.207 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.207 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723398 s, 145 MB/s 2024-04-25T00:06:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.217 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760921 s, 138 MB/s 2024-04-25T00:06:27.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.224 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.224 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566467 s, 185 MB/s 2024-04-25T00:06:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576273 s, 182 MB/s 2024-04-25T00:06:27.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.243 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862048 s, 122 MB/s 2024-04-25T00:06:27.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.251 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688369 s, 152 MB/s 2024-04-25T00:06:27.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.260 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701545 s, 149 MB/s 2024-04-25T00:06:27.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.271 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.271 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845715 s, 124 MB/s 2024-04-25T00:06:27.281 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.281 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.281 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828089 s, 127 MB/s 2024-04-25T00:06:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698549 s, 150 MB/s 2024-04-25T00:06:27.298 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.298 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689858 s, 152 MB/s 2024-04-25T00:06:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706685 s, 148 MB/s 2024-04-25T00:06:27.315 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.315 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.315 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575478 s, 182 MB/s 2024-04-25T00:06:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058544 s, 179 MB/s 2024-04-25T00:06:27.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.333 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768549 s, 136 MB/s 2024-04-25T00:06:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069288 s, 151 MB/s 2024-04-25T00:06:27.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.351 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.351 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073846 s, 142 MB/s 2024-04-25T00:06:27.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.360 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758906 s, 138 MB/s 2024-04-25T00:06:27.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709012 s, 148 MB/s 2024-04-25T00:06:27.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069 s, 152 MB/s 2024-04-25T00:06:27.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.388 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781741 s, 134 MB/s 2024-04-25T00:06:27.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.396 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.396 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592475 s, 177 MB/s 2024-04-25T00:06:27.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.404 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.404 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571268 s, 184 MB/s 2024-04-25T00:06:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748136 s, 140 MB/s 2024-04-25T00:06:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822169 s, 128 MB/s 2024-04-25T00:06:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:27.430 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:27.430 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:27.430 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T00:06:27.430 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:27.431 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:27.431 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:27.431 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:27.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728098 s, 144 MB/s 2024-04-25T00:06:27.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.499 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.499 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690431 s, 152 MB/s 2024-04-25T00:06:27.508 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.509 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.509 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743962 s, 141 MB/s 2024-04-25T00:06:27.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.517 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689329 s, 152 MB/s 2024-04-25T00:06:27.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.526 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071121 s, 147 MB/s 2024-04-25T00:06:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.535 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715936 s, 146 MB/s 2024-04-25T00:06:27.542 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.543 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574623 s, 182 MB/s 2024-04-25T00:06:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066285 s, 158 MB/s 2024-04-25T00:06:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718381 s, 146 MB/s 2024-04-25T00:06:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675686 s, 155 MB/s 2024-04-25T00:06:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748588 s, 140 MB/s 2024-04-25T00:06:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748151 s, 140 MB/s 2024-04-25T00:06:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570844 s, 184 MB/s 2024-04-25T00:06:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.602 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674162 s, 156 MB/s 2024-04-25T00:06:27.610 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.611 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.611 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071125 s, 147 MB/s 2024-04-25T00:06:27.618 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.619 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.619 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679433 s, 154 MB/s 2024-04-25T00:06:27.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.626 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590882 s, 177 MB/s 2024-04-25T00:06:27.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.635 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739203 s, 142 MB/s 2024-04-25T00:06:27.643 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.643 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.644 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670182 s, 156 MB/s 2024-04-25T00:06:27.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.652 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684173 s, 153 MB/s 2024-04-25T00:06:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686033 s, 153 MB/s 2024-04-25T00:06:27.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.669 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006798 s, 154 MB/s 2024-04-25T00:06:27.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781002 s, 134 MB/s 2024-04-25T00:06:27.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.687 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765508 s, 137 MB/s 2024-04-25T00:06:27.696 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.696 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748676 s, 140 MB/s 2024-04-25T00:06:27.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.707 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.707 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828939 s, 126 MB/s 2024-04-25T00:06:27.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.715 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.715 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717973 s, 146 MB/s 2024-04-25T00:06:27.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.723 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.723 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067796 s, 155 MB/s 2024-04-25T00:06:27.732 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.732 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.732 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688821 s, 152 MB/s 2024-04-25T00:06:27.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.741 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.741 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072427 s, 145 MB/s 2024-04-25T00:06:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658161 s, 159 MB/s 2024-04-25T00:06:27.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.757 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642394 s, 163 MB/s 2024-04-25T00:06:27.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:27.764 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:27.764 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:27.764 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T00:06:27.765 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:27.765 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:27.765 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:27.765 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:27.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:27.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.823 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.823 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717228 s, 146 MB/s 2024-04-25T00:06:27.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.830 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576547 s, 182 MB/s 2024-04-25T00:06:27.838 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.838 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.838 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655113 s, 160 MB/s 2024-04-25T00:06:27.846 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.846 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.847 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637935 s, 164 MB/s 2024-04-25T00:06:27.854 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.854 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.854 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580465 s, 181 MB/s 2024-04-25T00:06:27.862 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.862 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.862 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064643 s, 162 MB/s 2024-04-25T00:06:27.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651052 s, 161 MB/s 2024-04-25T00:06:27.878 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.878 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.879 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653935 s, 160 MB/s 2024-04-25T00:06:27.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.886 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565203 s, 186 MB/s 2024-04-25T00:06:27.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.893 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556068 s, 189 MB/s 2024-04-25T00:06:27.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.901 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656567 s, 160 MB/s 2024-04-25T00:06:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659726 s, 159 MB/s 2024-04-25T00:06:27.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.917 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.917 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058606 s, 179 MB/s 2024-04-25T00:06:27.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.924 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058172 s, 180 MB/s 2024-04-25T00:06:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.933 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727443 s, 144 MB/s 2024-04-25T00:06:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676549 s, 155 MB/s 2024-04-25T00:06:27.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.950 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707005 s, 148 MB/s 2024-04-25T00:06:27.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.958 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698415 s, 150 MB/s 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718572 s, 146 MB/s 2024-04-25T00:06:27.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.975 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685054 s, 153 MB/s 2024-04-25T00:06:27.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.983 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682826 s, 154 MB/s 2024-04-25T00:06:27.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:27.992 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:27.992 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705879 s, 149 MB/s 2024-04-25T00:06:28.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.000 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655427 s, 160 MB/s 2024-04-25T00:06:28.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.009 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069763 s, 150 MB/s 2024-04-25T00:06:28.016 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.016 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.016 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057425 s, 183 MB/s 2024-04-25T00:06:28.024 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.024 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.024 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669419 s, 157 MB/s 2024-04-25T00:06:28.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.032 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564089 s, 186 MB/s 2024-04-25T00:06:28.041 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.041 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.041 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737802 s, 142 MB/s 2024-04-25T00:06:28.049 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.050 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726194 s, 144 MB/s 2024-04-25T00:06:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653124 s, 161 MB/s 2024-04-25T00:06:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681124 s, 154 MB/s 2024-04-25T00:06:28.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.074 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678849 s, 154 MB/s 2024-04-25T00:06:28.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:28.081 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:28.081 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:28.081 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T00:06:28.081 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:28.081 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:28.081 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:28.081 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:28.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.139 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660164 s, 159 MB/s 2024-04-25T00:06:28.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.145 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578412 s, 181 MB/s 2024-04-25T00:06:28.153 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.153 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.153 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642939 s, 163 MB/s 2024-04-25T00:06:28.161 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.161 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.162 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661382 s, 159 MB/s 2024-04-25T00:06:28.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.171 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758747 s, 138 MB/s 2024-04-25T00:06:28.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.178 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576497 s, 182 MB/s 2024-04-25T00:06:28.185 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.186 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561115 s, 187 MB/s 2024-04-25T00:06:28.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.194 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711989 s, 147 MB/s 2024-04-25T00:06:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.202 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646806 s, 162 MB/s 2024-04-25T00:06:28.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.211 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700623 s, 150 MB/s 2024-04-25T00:06:28.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.218 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559835 s, 187 MB/s 2024-04-25T00:06:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637467 s, 164 MB/s 2024-04-25T00:06:28.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.233 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.233 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578256 s, 181 MB/s 2024-04-25T00:06:28.241 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.241 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.242 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654236 s, 160 MB/s 2024-04-25T00:06:28.249 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.249 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.249 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565417 s, 185 MB/s 2024-04-25T00:06:28.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.258 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729825 s, 144 MB/s 2024-04-25T00:06:28.266 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.266 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.266 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676871 s, 155 MB/s 2024-04-25T00:06:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730672 s, 144 MB/s 2024-04-25T00:06:28.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.283 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649526 s, 161 MB/s 2024-04-25T00:06:28.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.291 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636744 s, 165 MB/s 2024-04-25T00:06:28.299 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.299 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.299 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663434 s, 158 MB/s 2024-04-25T00:06:28.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.306 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.307 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584783 s, 179 MB/s 2024-04-25T00:06:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645816 s, 162 MB/s 2024-04-25T00:06:28.322 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655667 s, 160 MB/s 2024-04-25T00:06:28.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.330 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.331 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635273 s, 165 MB/s 2024-04-25T00:06:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.339 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646943 s, 162 MB/s 2024-04-25T00:06:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066065 s, 159 MB/s 2024-04-25T00:06:28.355 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.355 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.355 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674851 s, 155 MB/s 2024-04-25T00:06:28.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.363 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066694 s, 157 MB/s 2024-04-25T00:06:28.371 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.371 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.371 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593959 s, 177 MB/s 2024-04-25T00:06:28.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.380 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751127 s, 140 MB/s 2024-04-25T00:06:28.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.388 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.388 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658846 s, 159 MB/s 2024-04-25T00:06:28.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:28.395 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:28.395 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:28.395 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T00:06:28.396 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:28.396 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:28.396 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:28.396 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:28.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:28.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.455 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.455 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758392 s, 138 MB/s 2024-04-25T00:06:28.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.463 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709762 s, 148 MB/s 2024-04-25T00:06:28.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680377 s, 154 MB/s 2024-04-25T00:06:28.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.479 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616404 s, 170 MB/s 2024-04-25T00:06:28.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.488 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007385 s, 142 MB/s 2024-04-25T00:06:28.496 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066043 s, 159 MB/s 2024-04-25T00:06:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671645 s, 156 MB/s 2024-04-25T00:06:28.512 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.513 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.513 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635277 s, 165 MB/s 2024-04-25T00:06:28.521 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.521 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.522 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726792 s, 144 MB/s 2024-04-25T00:06:28.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.530 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729038 s, 144 MB/s 2024-04-25T00:06:28.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.539 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745929 s, 141 MB/s 2024-04-25T00:06:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.548 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680823 s, 154 MB/s 2024-04-25T00:06:28.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.557 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.557 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725293 s, 145 MB/s 2024-04-25T00:06:28.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.564 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.565 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635072 s, 165 MB/s 2024-04-25T00:06:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.573 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.573 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648054 s, 162 MB/s 2024-04-25T00:06:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744425 s, 141 MB/s 2024-04-25T00:06:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.589 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595863 s, 176 MB/s 2024-04-25T00:06:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620255 s, 169 MB/s 2024-04-25T00:06:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701827 s, 149 MB/s 2024-04-25T00:06:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584452 s, 179 MB/s 2024-04-25T00:06:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655529 s, 160 MB/s 2024-04-25T00:06:28.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633057 s, 166 MB/s 2024-04-25T00:06:28.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.637 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.637 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641009 s, 164 MB/s 2024-04-25T00:06:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657382 s, 160 MB/s 2024-04-25T00:06:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064174 s, 163 MB/s 2024-04-25T00:06:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750116 s, 140 MB/s 2024-04-25T00:06:28.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.670 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661803 s, 158 MB/s 2024-04-25T00:06:28.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.679 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.679 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723126 s, 145 MB/s 2024-04-25T00:06:28.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.688 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076524 s, 137 MB/s 2024-04-25T00:06:28.695 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.696 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.696 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587733 s, 178 MB/s 2024-04-25T00:06:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688143 s, 152 MB/s 2024-04-25T00:06:28.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.712 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061402 s, 171 MB/s 2024-04-25T00:06:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:28.719 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:28.719 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:28.719 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T00:06:28.719 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:28.719 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:28.719 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:28.719 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:28.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:28.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.778 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715093 s, 147 MB/s 2024-04-25T00:06:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673329 s, 156 MB/s 2024-04-25T00:06:28.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067712 s, 155 MB/s 2024-04-25T00:06:28.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.802 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672098 s, 156 MB/s 2024-04-25T00:06:28.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589796 s, 178 MB/s 2024-04-25T00:06:28.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657697 s, 159 MB/s 2024-04-25T00:06:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722543 s, 145 MB/s 2024-04-25T00:06:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640605 s, 164 MB/s 2024-04-25T00:06:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.844 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.844 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713185 s, 147 MB/s 2024-04-25T00:06:28.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.852 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.852 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656691 s, 160 MB/s 2024-04-25T00:06:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075389 s, 139 MB/s 2024-04-25T00:06:28.869 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.870 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.870 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736161 s, 142 MB/s 2024-04-25T00:06:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690475 s, 152 MB/s 2024-04-25T00:06:28.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.886 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649615 s, 161 MB/s 2024-04-25T00:06:28.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.895 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.895 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725544 s, 145 MB/s 2024-04-25T00:06:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583497 s, 180 MB/s 2024-04-25T00:06:28.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.911 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660909 s, 159 MB/s 2024-04-25T00:06:28.917 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.917 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.918 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550167 s, 191 MB/s 2024-04-25T00:06:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725145 s, 145 MB/s 2024-04-25T00:06:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.934 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581349 s, 180 MB/s 2024-04-25T00:06:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.942 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662157 s, 158 MB/s 2024-04-25T00:06:28.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.950 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.951 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703006 s, 149 MB/s 2024-04-25T00:06:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.959 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708792 s, 148 MB/s 2024-04-25T00:06:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.967 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646316 s, 162 MB/s 2024-04-25T00:06:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.976 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684903 s, 153 MB/s 2024-04-25T00:06:28.984 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.984 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.985 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733291 s, 143 MB/s 2024-04-25T00:06:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:28.993 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676405 s, 155 MB/s 2024-04-25T00:06:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.000 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558495 s, 188 MB/s 2024-04-25T00:06:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694763 s, 151 MB/s 2024-04-25T00:06:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747441 s, 140 MB/s 2024-04-25T00:06:29.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.025 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.025 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561275 s, 187 MB/s 2024-04-25T00:06:29.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.033 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667559 s, 157 MB/s 2024-04-25T00:06:29.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:29.040 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:29.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:29.096 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.096 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.096 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556032 s, 189 MB/s 2024-04-25T00:06:29.104 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.105 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674293 s, 156 MB/s 2024-04-25T00:06:29.113 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.113 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.113 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663581 s, 158 MB/s 2024-04-25T00:06:29.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.122 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731848 s, 143 MB/s 2024-04-25T00:06:29.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.130 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704064 s, 149 MB/s 2024-04-25T00:06:29.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.138 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067091 s, 156 MB/s 2024-04-25T00:06:29.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065741 s, 160 MB/s 2024-04-25T00:06:29.155 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.155 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.155 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647528 s, 162 MB/s 2024-04-25T00:06:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579392 s, 181 MB/s 2024-04-25T00:06:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.170 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661598 s, 158 MB/s 2024-04-25T00:06:29.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.179 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657666 s, 159 MB/s 2024-04-25T00:06:29.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.186 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559424 s, 187 MB/s 2024-04-25T00:06:29.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.195 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743444 s, 141 MB/s 2024-04-25T00:06:29.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065359 s, 160 MB/s 2024-04-25T00:06:29.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.211 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684993 s, 153 MB/s 2024-04-25T00:06:29.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.219 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677139 s, 155 MB/s 2024-04-25T00:06:29.226 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.227 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557946 s, 188 MB/s 2024-04-25T00:06:29.236 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.236 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.236 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738372 s, 142 MB/s 2024-04-25T00:06:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558473 s, 188 MB/s 2024-04-25T00:06:29.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.252 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.252 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707219 s, 148 MB/s 2024-04-25T00:06:29.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.260 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.260 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688501 s, 152 MB/s 2024-04-25T00:06:29.268 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.268 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.268 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653248 s, 161 MB/s 2024-04-25T00:06:29.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.275 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578595 s, 181 MB/s 2024-04-25T00:06:29.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.284 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709037 s, 148 MB/s 2024-04-25T00:06:29.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.292 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665779 s, 157 MB/s 2024-04-25T00:06:29.301 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.301 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.301 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684661 s, 153 MB/s 2024-04-25T00:06:29.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.309 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641569 s, 163 MB/s 2024-04-25T00:06:29.317 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.317 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732224 s, 143 MB/s 2024-04-25T00:06:29.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.326 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.326 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679787 s, 154 MB/s 2024-04-25T00:06:29.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.335 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784779 s, 134 MB/s 2024-04-25T00:06:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566957 s, 185 MB/s 2024-04-25T00:06:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641727 s, 163 MB/s 2024-04-25T00:06:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:29.357 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:29.357 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:29.357 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T00:06:29.358 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:29.358 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:29.358 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:29.358 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:29.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:29.415 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.416 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708768 s, 148 MB/s 2024-04-25T00:06:29.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.423 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596299 s, 176 MB/s 2024-04-25T00:06:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664634 s, 158 MB/s 2024-04-25T00:06:29.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.439 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575521 s, 182 MB/s 2024-04-25T00:06:29.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.447 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646632 s, 162 MB/s 2024-04-25T00:06:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.456 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741291 s, 141 MB/s 2024-04-25T00:06:29.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.464 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612394 s, 171 MB/s 2024-04-25T00:06:29.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712962 s, 147 MB/s 2024-04-25T00:06:29.480 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.480 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642701 s, 163 MB/s 2024-04-25T00:06:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649923 s, 161 MB/s 2024-04-25T00:06:29.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.497 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.497 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707568 s, 148 MB/s 2024-04-25T00:06:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071835 s, 146 MB/s 2024-04-25T00:06:29.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.514 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007044 s, 149 MB/s 2024-04-25T00:06:29.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.523 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.523 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732551 s, 143 MB/s 2024-04-25T00:06:29.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.530 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.531 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595869 s, 176 MB/s 2024-04-25T00:06:29.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.539 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.539 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649656 s, 161 MB/s 2024-04-25T00:06:29.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.548 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742745 s, 141 MB/s 2024-04-25T00:06:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.556 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675587 s, 155 MB/s 2024-04-25T00:06:29.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.565 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717082 s, 146 MB/s 2024-04-25T00:06:29.573 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.574 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721421 s, 145 MB/s 2024-04-25T00:06:29.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.582 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725322 s, 145 MB/s 2024-04-25T00:06:29.590 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.590 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.590 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579901 s, 181 MB/s 2024-04-25T00:06:29.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.599 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651283 s, 161 MB/s 2024-04-25T00:06:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.607 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.607 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721306 s, 145 MB/s 2024-04-25T00:06:29.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656023 s, 160 MB/s 2024-04-25T00:06:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679679 s, 154 MB/s 2024-04-25T00:06:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665163 s, 158 MB/s 2024-04-25T00:06:29.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.639 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555668 s, 189 MB/s 2024-04-25T00:06:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596809 s, 176 MB/s 2024-04-25T00:06:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711307 s, 147 MB/s 2024-04-25T00:06:29.663 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.663 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.663 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646765 s, 162 MB/s 2024-04-25T00:06:29.671 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.672 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715743 s, 147 MB/s 2024-04-25T00:06:29.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:29.678 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:29.679 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:29.679 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T00:06:29.679 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:29.679 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:29.679 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:29.679 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:29.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:29.737 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.737 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.737 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722282 s, 145 MB/s 2024-04-25T00:06:29.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.746 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.746 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769288 s, 136 MB/s 2024-04-25T00:06:29.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.753 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587639 s, 178 MB/s 2024-04-25T00:06:29.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.762 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641024 s, 164 MB/s 2024-04-25T00:06:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.770 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697145 s, 150 MB/s 2024-04-25T00:06:29.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.777 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580259 s, 181 MB/s 2024-04-25T00:06:29.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.785 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646154 s, 162 MB/s 2024-04-25T00:06:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713197 s, 147 MB/s 2024-04-25T00:06:29.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.803 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704783 s, 149 MB/s 2024-04-25T00:06:29.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554469 s, 189 MB/s 2024-04-25T00:06:29.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695118 s, 151 MB/s 2024-04-25T00:06:29.825 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.826 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005699 s, 184 MB/s 2024-04-25T00:06:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671514 s, 156 MB/s 2024-04-25T00:06:29.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.842 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.842 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676982 s, 155 MB/s 2024-04-25T00:06:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643094 s, 163 MB/s 2024-04-25T00:06:29.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.859 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693605 s, 151 MB/s 2024-04-25T00:06:29.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.866 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576191 s, 182 MB/s 2024-04-25T00:06:29.873 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.873 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.873 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562383 s, 186 MB/s 2024-04-25T00:06:29.881 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.881 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.881 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064547 s, 162 MB/s 2024-04-25T00:06:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706915 s, 148 MB/s 2024-04-25T00:06:29.899 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.899 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.899 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734316 s, 143 MB/s 2024-04-25T00:06:29.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.907 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662119 s, 158 MB/s 2024-04-25T00:06:29.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.916 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686148 s, 153 MB/s 2024-04-25T00:06:29.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.924 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717447 s, 146 MB/s 2024-04-25T00:06:29.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.933 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.933 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679536 s, 154 MB/s 2024-04-25T00:06:29.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.941 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720289 s, 146 MB/s 2024-04-25T00:06:29.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.949 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633221 s, 166 MB/s 2024-04-25T00:06:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638229 s, 164 MB/s 2024-04-25T00:06:29.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.966 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743969 s, 141 MB/s 2024-04-25T00:06:29.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.975 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699569 s, 150 MB/s 2024-04-25T00:06:29.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.983 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.983 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696201 s, 151 MB/s 2024-04-25T00:06:29.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:29.991 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:29.991 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646018 s, 162 MB/s 2024-04-25T00:06:29.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:29.998 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:29.998 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:29.998 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T00:06:29.998 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:29.998 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:29.998 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:29.998 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:30.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:30.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.055 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058813 s, 178 MB/s 2024-04-25T00:06:30.063 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677169 s, 155 MB/s 2024-04-25T00:06:30.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.072 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723986 s, 145 MB/s 2024-04-25T00:06:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.080 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637505 s, 164 MB/s 2024-04-25T00:06:30.089 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.089 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.089 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711301 s, 147 MB/s 2024-04-25T00:06:30.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.097 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663632 s, 158 MB/s 2024-04-25T00:06:30.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.106 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667789 s, 157 MB/s 2024-04-25T00:06:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706789 s, 148 MB/s 2024-04-25T00:06:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720043 s, 146 MB/s 2024-04-25T00:06:30.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629615 s, 167 MB/s 2024-04-25T00:06:30.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.140 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.140 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737405 s, 142 MB/s 2024-04-25T00:06:30.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633144 s, 166 MB/s 2024-04-25T00:06:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690099 s, 152 MB/s 2024-04-25T00:06:30.164 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.165 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.165 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751414 s, 140 MB/s 2024-04-25T00:06:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.174 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700949 s, 150 MB/s 2024-04-25T00:06:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.180 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569326 s, 184 MB/s 2024-04-25T00:06:30.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.188 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.188 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582694 s, 180 MB/s 2024-04-25T00:06:30.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.195 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.195 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580858 s, 181 MB/s 2024-04-25T00:06:30.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590479 s, 178 MB/s 2024-04-25T00:06:30.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.211 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668696 s, 157 MB/s 2024-04-25T00:06:30.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.220 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693982 s, 151 MB/s 2024-04-25T00:06:30.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.228 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707343 s, 148 MB/s 2024-04-25T00:06:30.236 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665434 s, 158 MB/s 2024-04-25T00:06:30.245 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.245 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.245 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728885 s, 144 MB/s 2024-04-25T00:06:30.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.254 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719708 s, 146 MB/s 2024-04-25T00:06:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643641 s, 163 MB/s 2024-04-25T00:06:30.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643943 s, 163 MB/s 2024-04-25T00:06:30.277 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.277 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.277 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057343 s, 183 MB/s 2024-04-25T00:06:30.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.284 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560432 s, 187 MB/s 2024-04-25T00:06:30.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.292 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.293 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656646 s, 160 MB/s 2024-04-25T00:06:30.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.300 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628572 s, 167 MB/s 2024-04-25T00:06:30.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.308 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583912 s, 180 MB/s 2024-04-25T00:06:30.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:30.315 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:30.315 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:30.315 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T00:06:30.315 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:30.315 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:30.315 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:30.315 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:30.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:30.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.373 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691268 s, 152 MB/s 2024-04-25T00:06:30.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.381 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.381 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675371 s, 155 MB/s 2024-04-25T00:06:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066279 s, 158 MB/s 2024-04-25T00:06:30.397 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.398 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698142 s, 150 MB/s 2024-04-25T00:06:30.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.406 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723467 s, 145 MB/s 2024-04-25T00:06:30.414 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.414 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.414 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651256 s, 161 MB/s 2024-04-25T00:06:30.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.423 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.423 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730156 s, 144 MB/s 2024-04-25T00:06:30.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.431 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594217 s, 176 MB/s 2024-04-25T00:06:30.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.439 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067132 s, 156 MB/s 2024-04-25T00:06:30.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.446 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.446 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548434 s, 191 MB/s 2024-04-25T00:06:30.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.454 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.454 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636221 s, 165 MB/s 2024-04-25T00:06:30.462 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.462 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.462 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626799 s, 167 MB/s 2024-04-25T00:06:30.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.471 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730226 s, 144 MB/s 2024-04-25T00:06:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646308 s, 162 MB/s 2024-04-25T00:06:30.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.487 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660065 s, 159 MB/s 2024-04-25T00:06:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648583 s, 162 MB/s 2024-04-25T00:06:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637186 s, 165 MB/s 2024-04-25T00:06:30.510 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.511 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639612 s, 164 MB/s 2024-04-25T00:06:30.518 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.518 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.518 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586516 s, 179 MB/s 2024-04-25T00:06:30.527 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.527 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.527 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723371 s, 145 MB/s 2024-04-25T00:06:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.536 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723945 s, 145 MB/s 2024-04-25T00:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643764 s, 163 MB/s 2024-04-25T00:06:30.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.552 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653277 s, 161 MB/s 2024-04-25T00:06:30.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.561 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713175 s, 147 MB/s 2024-04-25T00:06:30.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.568 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574211 s, 183 MB/s 2024-04-25T00:06:30.576 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.576 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.576 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638819 s, 164 MB/s 2024-04-25T00:06:30.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.584 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627762 s, 167 MB/s 2024-04-25T00:06:30.593 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.594 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.594 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653187 s, 161 MB/s 2024-04-25T00:06:30.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.600 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644115 s, 163 MB/s 2024-04-25T00:06:30.608 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.608 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.608 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064442 s, 163 MB/s 2024-04-25T00:06:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570701 s, 184 MB/s 2024-04-25T00:06:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585524 s, 179 MB/s 2024-04-25T00:06:30.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:30.630 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:30.630 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:30.630 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T00:06:30.631 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:30.631 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:30.631 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:30.631 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:30.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:30.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612132 s, 171 MB/s 2024-04-25T00:06:30.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.698 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.698 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757346 s, 138 MB/s 2024-04-25T00:06:30.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.706 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.706 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682711 s, 154 MB/s 2024-04-25T00:06:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006021 s, 174 MB/s 2024-04-25T00:06:30.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618165 s, 170 MB/s 2024-04-25T00:06:30.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682425 s, 154 MB/s 2024-04-25T00:06:30.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.739 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645304 s, 162 MB/s 2024-04-25T00:06:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635519 s, 165 MB/s 2024-04-25T00:06:30.756 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.756 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.756 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738954 s, 142 MB/s 2024-04-25T00:06:30.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.764 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580155 s, 181 MB/s 2024-04-25T00:06:30.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.771 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581796 s, 180 MB/s 2024-04-25T00:06:30.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.779 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675275 s, 155 MB/s 2024-04-25T00:06:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.788 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066552 s, 158 MB/s 2024-04-25T00:06:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736219 s, 142 MB/s 2024-04-25T00:06:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649617 s, 161 MB/s 2024-04-25T00:06:30.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.813 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702088 s, 149 MB/s 2024-04-25T00:06:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703779 s, 149 MB/s 2024-04-25T00:06:30.830 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.831 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.831 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759737 s, 138 MB/s 2024-04-25T00:06:30.838 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.839 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.839 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641996 s, 163 MB/s 2024-04-25T00:06:30.846 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.847 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.847 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644856 s, 163 MB/s 2024-04-25T00:06:30.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.855 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.855 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712681 s, 147 MB/s 2024-04-25T00:06:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.863 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065478 s, 160 MB/s 2024-04-25T00:06:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632074 s, 166 MB/s 2024-04-25T00:06:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681534 s, 154 MB/s 2024-04-25T00:06:30.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.889 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.889 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726824 s, 144 MB/s 2024-04-25T00:06:30.897 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.897 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.897 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644443 s, 163 MB/s 2024-04-25T00:06:30.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.906 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649365 s, 161 MB/s 2024-04-25T00:06:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667329 s, 157 MB/s 2024-04-25T00:06:30.922 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.922 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.922 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569302 s, 184 MB/s 2024-04-25T00:06:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738609 s, 142 MB/s 2024-04-25T00:06:30.939 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633177 s, 166 MB/s 2024-04-25T00:06:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578234 s, 181 MB/s 2024-04-25T00:06:30.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:30.954 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:30.954 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:30.954 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T00:06:30.954 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:30.954 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:30.954 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:30.954 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:31.012 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.012 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.012 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582808 s, 180 MB/s 2024-04-25T00:06:31.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.021 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.021 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706508 s, 148 MB/s 2024-04-25T00:06:31.029 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.030 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.030 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661405 s, 159 MB/s 2024-04-25T00:06:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.037 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567149 s, 185 MB/s 2024-04-25T00:06:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.046 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655263 s, 160 MB/s 2024-04-25T00:06:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.053 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566961 s, 185 MB/s 2024-04-25T00:06:31.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.062 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006966 s, 151 MB/s 2024-04-25T00:06:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.071 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698447 s, 150 MB/s 2024-04-25T00:06:31.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.079 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563186 s, 186 MB/s 2024-04-25T00:06:31.087 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.087 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.087 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653798 s, 160 MB/s 2024-04-25T00:06:31.096 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.096 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.096 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687226 s, 153 MB/s 2024-04-25T00:06:31.104 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.104 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.104 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630669 s, 166 MB/s 2024-04-25T00:06:31.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.114 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818236 s, 128 MB/s 2024-04-25T00:06:31.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.124 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.124 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737114 s, 142 MB/s 2024-04-25T00:06:31.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.132 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.132 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657739 s, 159 MB/s 2024-04-25T00:06:31.140 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.141 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.141 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643226 s, 163 MB/s 2024-04-25T00:06:31.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.150 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.150 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722949 s, 145 MB/s 2024-04-25T00:06:31.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.158 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.158 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660891 s, 159 MB/s 2024-04-25T00:06:31.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.166 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.166 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552884 s, 190 MB/s 2024-04-25T00:06:31.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.173 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.173 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582881 s, 180 MB/s 2024-04-25T00:06:31.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.182 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.182 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671395 s, 156 MB/s 2024-04-25T00:06:31.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.190 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.190 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673601 s, 156 MB/s 2024-04-25T00:06:31.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.198 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.198 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662898 s, 158 MB/s 2024-04-25T00:06:31.205 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.205 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.205 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567667 s, 185 MB/s 2024-04-25T00:06:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658235 s, 159 MB/s 2024-04-25T00:06:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702368 s, 149 MB/s 2024-04-25T00:06:31.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.231 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.232 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663408 s, 158 MB/s 2024-04-25T00:06:31.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.240 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.240 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643255 s, 163 MB/s 2024-04-25T00:06:31.249 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.249 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.249 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696682 s, 151 MB/s 2024-04-25T00:06:31.257 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.257 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.257 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650399 s, 161 MB/s 2024-04-25T00:06:31.265 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.266 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.266 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673693 s, 156 MB/s 2024-04-25T00:06:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635234 s, 165 MB/s 2024-04-25T00:06:31.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:31.281 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:31.281 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:31.282 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T00:06:31.282 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:31.282 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:31.282 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:31.282 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:31.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:31.339 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664563 s, 158 MB/s 2024-04-25T00:06:31.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.348 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.348 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661772 s, 158 MB/s 2024-04-25T00:06:31.357 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.357 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.357 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734331 s, 143 MB/s 2024-04-25T00:06:31.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.366 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684522 s, 153 MB/s 2024-04-25T00:06:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069162 s, 152 MB/s 2024-04-25T00:06:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556605 s, 188 MB/s 2024-04-25T00:06:31.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.391 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641926 s, 163 MB/s 2024-04-25T00:06:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.400 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.400 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733885 s, 143 MB/s 2024-04-25T00:06:31.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.410 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652647 s, 161 MB/s 2024-04-25T00:06:31.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.416 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.417 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656539 s, 160 MB/s 2024-04-25T00:06:31.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.424 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.425 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610518 s, 172 MB/s 2024-04-25T00:06:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666871 s, 157 MB/s 2024-04-25T00:06:31.441 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.442 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651805 s, 161 MB/s 2024-04-25T00:06:31.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.449 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.449 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592305 s, 177 MB/s 2024-04-25T00:06:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731656 s, 143 MB/s 2024-04-25T00:06:31.467 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.467 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.467 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067721 s, 155 MB/s 2024-04-25T00:06:31.477 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.477 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.477 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764672 s, 137 MB/s 2024-04-25T00:06:31.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.484 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.484 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557621 s, 188 MB/s 2024-04-25T00:06:31.493 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.493 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.493 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670446 s, 156 MB/s 2024-04-25T00:06:31.500 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.500 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.500 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559541 s, 187 MB/s 2024-04-25T00:06:31.509 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.509 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.509 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634322 s, 165 MB/s 2024-04-25T00:06:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590242 s, 178 MB/s 2024-04-25T00:06:31.525 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.525 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.525 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636351 s, 165 MB/s 2024-04-25T00:06:31.533 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.533 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.533 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592379 s, 177 MB/s 2024-04-25T00:06:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572392 s, 183 MB/s 2024-04-25T00:06:31.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.549 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.549 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647414 s, 162 MB/s 2024-04-25T00:06:31.557 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.558 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629128 s, 167 MB/s 2024-04-25T00:06:31.566 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.566 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.566 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656811 s, 160 MB/s 2024-04-25T00:06:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644008 s, 163 MB/s 2024-04-25T00:06:31.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641548 s, 163 MB/s 2024-04-25T00:06:31.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.592 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.592 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713717 s, 147 MB/s 2024-04-25T00:06:31.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.599 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563541 s, 186 MB/s 2024-04-25T00:06:31.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:31.606 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:31.606 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:31.606 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T00:06:31.606 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:31.606 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:31.606 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:31.606 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:31.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:31.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.664 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.664 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667003 s, 157 MB/s 2024-04-25T00:06:31.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.672 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.672 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646129 s, 162 MB/s 2024-04-25T00:06:31.680 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.680 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.681 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702436 s, 149 MB/s 2024-04-25T00:06:31.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.688 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.688 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581124 s, 180 MB/s 2024-04-25T00:06:31.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.697 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.697 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722779 s, 145 MB/s 2024-04-25T00:06:31.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.704 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057053 s, 184 MB/s 2024-04-25T00:06:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663612 s, 158 MB/s 2024-04-25T00:06:31.721 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.721 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.721 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727573 s, 144 MB/s 2024-04-25T00:06:31.729 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.729 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.729 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656576 s, 160 MB/s 2024-04-25T00:06:31.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.739 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729036 s, 144 MB/s 2024-04-25T00:06:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.747 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703291 s, 149 MB/s 2024-04-25T00:06:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623923 s, 168 MB/s 2024-04-25T00:06:31.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.763 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.763 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693298 s, 151 MB/s 2024-04-25T00:06:31.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.771 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.771 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645567 s, 162 MB/s 2024-04-25T00:06:31.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.779 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.779 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664061 s, 158 MB/s 2024-04-25T00:06:31.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.788 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.788 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704056 s, 149 MB/s 2024-04-25T00:06:31.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.795 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577082 s, 182 MB/s 2024-04-25T00:06:31.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.803 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573379 s, 183 MB/s 2024-04-25T00:06:31.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592918 s, 177 MB/s 2024-04-25T00:06:31.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.821 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.821 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681898 s, 154 MB/s 2024-04-25T00:06:31.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.827 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675824 s, 155 MB/s 2024-04-25T00:06:31.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687939 s, 152 MB/s 2024-04-25T00:06:31.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.843 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656151 s, 160 MB/s 2024-04-25T00:06:31.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.851 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.851 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656663 s, 160 MB/s 2024-04-25T00:06:31.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.859 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572889 s, 183 MB/s 2024-04-25T00:06:31.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.866 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646278 s, 162 MB/s 2024-04-25T00:06:31.874 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.874 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.874 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642809 s, 163 MB/s 2024-04-25T00:06:31.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.883 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706824 s, 148 MB/s 2024-04-25T00:06:31.892 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.892 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.892 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694022 s, 151 MB/s 2024-04-25T00:06:31.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.900 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633173 s, 166 MB/s 2024-04-25T00:06:31.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.907 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.907 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583768 s, 180 MB/s 2024-04-25T00:06:31.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.916 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.916 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706207 s, 148 MB/s 2024-04-25T00:06:31.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:31.923 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:31.923 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:31.923 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T00:06:31.923 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:31.923 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:31.923 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:31.923 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635187 s, 165 MB/s 2024-04-25T00:06:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.990 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644838 s, 163 MB/s 2024-04-25T00:06:31.998 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:31.998 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:31.998 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730295 s, 144 MB/s 2024-04-25T00:06:32.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.005 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556816 s, 188 MB/s 2024-04-25T00:06:32.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.014 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736247 s, 142 MB/s 2024-04-25T00:06:32.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.022 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669852 s, 157 MB/s 2024-04-25T00:06:32.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.032 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775522 s, 135 MB/s 2024-04-25T00:06:32.041 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.041 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.041 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756048 s, 139 MB/s 2024-04-25T00:06:32.049 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.049 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.049 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662123 s, 158 MB/s 2024-04-25T00:06:32.057 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.058 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.058 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685062 s, 153 MB/s 2024-04-25T00:06:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650104 s, 161 MB/s 2024-04-25T00:06:32.073 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.074 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.074 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669708 s, 157 MB/s 2024-04-25T00:06:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.081 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571665 s, 183 MB/s 2024-04-25T00:06:32.089 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.089 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.089 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648061 s, 162 MB/s 2024-04-25T00:06:32.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.097 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655438 s, 160 MB/s 2024-04-25T00:06:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642728 s, 163 MB/s 2024-04-25T00:06:32.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.114 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713134 s, 147 MB/s 2024-04-25T00:06:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691344 s, 152 MB/s 2024-04-25T00:06:32.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.130 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645757 s, 162 MB/s 2024-04-25T00:06:32.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.138 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631196 s, 166 MB/s 2024-04-25T00:06:32.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.146 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664748 s, 158 MB/s 2024-04-25T00:06:32.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.154 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.154 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696484 s, 151 MB/s 2024-04-25T00:06:32.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.162 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648299 s, 162 MB/s 2024-04-25T00:06:32.170 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066953 s, 157 MB/s 2024-04-25T00:06:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737377 s, 142 MB/s 2024-04-25T00:06:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.187 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584984 s, 179 MB/s 2024-04-25T00:06:32.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.194 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.194 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581958 s, 180 MB/s 2024-04-25T00:06:32.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.203 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.203 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665699 s, 158 MB/s 2024-04-25T00:06:32.210 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.210 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.210 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555245 s, 189 MB/s 2024-04-25T00:06:32.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.218 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.218 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677947 s, 155 MB/s 2024-04-25T00:06:32.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.227 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730808 s, 143 MB/s 2024-04-25T00:06:32.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.235 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632448 s, 166 MB/s 2024-04-25T00:06:32.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:32.242 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:32.242 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:32.242 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T00:06:32.242 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:32.242 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:32.242 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:32.242 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:32.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067903 s, 154 MB/s 2024-04-25T00:06:32.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.308 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659385 s, 159 MB/s 2024-04-25T00:06:32.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651047 s, 161 MB/s 2024-04-25T00:06:32.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.324 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689127 s, 152 MB/s 2024-04-25T00:06:32.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.332 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601351 s, 174 MB/s 2024-04-25T00:06:32.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.341 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.341 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700881 s, 150 MB/s 2024-04-25T00:06:32.349 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667056 s, 157 MB/s 2024-04-25T00:06:32.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.357 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628165 s, 167 MB/s 2024-04-25T00:06:32.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.364 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.364 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636819 s, 165 MB/s 2024-04-25T00:06:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703767 s, 149 MB/s 2024-04-25T00:06:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721215 s, 145 MB/s 2024-04-25T00:06:32.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.390 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.390 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708402 s, 148 MB/s 2024-04-25T00:06:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651119 s, 161 MB/s 2024-04-25T00:06:32.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.406 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.406 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656035 s, 160 MB/s 2024-04-25T00:06:32.414 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.414 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.414 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660761 s, 159 MB/s 2024-04-25T00:06:32.422 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.422 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.422 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588943 s, 178 MB/s 2024-04-25T00:06:32.430 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.430 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.430 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070242 s, 149 MB/s 2024-04-25T00:06:32.438 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.439 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.439 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655982 s, 160 MB/s 2024-04-25T00:06:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.447 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.448 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721203 s, 145 MB/s 2024-04-25T00:06:32.455 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.455 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.455 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642317 s, 163 MB/s 2024-04-25T00:06:32.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.464 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.464 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678523 s, 155 MB/s 2024-04-25T00:06:32.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.471 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.471 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005834 s, 180 MB/s 2024-04-25T00:06:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.480 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672663 s, 156 MB/s 2024-04-25T00:06:32.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.487 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.487 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593213 s, 177 MB/s 2024-04-25T00:06:32.494 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.495 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.495 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577947 s, 181 MB/s 2024-04-25T00:06:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066183 s, 158 MB/s 2024-04-25T00:06:32.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.511 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.511 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657374 s, 160 MB/s 2024-04-25T00:06:32.518 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.519 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.519 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623629 s, 168 MB/s 2024-04-25T00:06:32.526 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.527 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.527 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642173 s, 163 MB/s 2024-04-25T00:06:32.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.534 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.535 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657986 s, 159 MB/s 2024-04-25T00:06:32.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732838 s, 143 MB/s 2024-04-25T00:06:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670559 s, 156 MB/s 2024-04-25T00:06:32.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:32.559 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:32.559 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:32.559 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T00:06:32.559 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:32.559 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:32.559 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:32.559 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:32.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:32.617 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.617 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.617 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746781 s, 140 MB/s 2024-04-25T00:06:32.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.626 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727905 s, 144 MB/s 2024-04-25T00:06:32.634 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.635 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704935 s, 149 MB/s 2024-04-25T00:06:32.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.642 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590527 s, 178 MB/s 2024-04-25T00:06:32.651 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.651 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.651 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715208 s, 147 MB/s 2024-04-25T00:06:32.659 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.659 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.659 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645309 s, 162 MB/s 2024-04-25T00:06:32.668 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.668 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.668 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727377 s, 144 MB/s 2024-04-25T00:06:32.676 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.677 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728066 s, 144 MB/s 2024-04-25T00:06:32.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.685 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717927 s, 146 MB/s 2024-04-25T00:06:32.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.692 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.693 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563414 s, 186 MB/s 2024-04-25T00:06:32.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.702 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757566 s, 138 MB/s 2024-04-25T00:06:32.711 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.711 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.711 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755779 s, 139 MB/s 2024-04-25T00:06:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700161 s, 150 MB/s 2024-04-25T00:06:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666205 s, 157 MB/s 2024-04-25T00:06:32.737 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.737 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.737 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726773 s, 144 MB/s 2024-04-25T00:06:32.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.745 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.745 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642997 s, 163 MB/s 2024-04-25T00:06:32.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.754 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.754 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748438 s, 140 MB/s 2024-04-25T00:06:32.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.761 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579343 s, 181 MB/s 2024-04-25T00:06:32.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.769 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066662 s, 157 MB/s 2024-04-25T00:06:32.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.778 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643885 s, 163 MB/s 2024-04-25T00:06:32.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.786 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646337 s, 162 MB/s 2024-04-25T00:06:32.793 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.793 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.793 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566204 s, 185 MB/s 2024-04-25T00:06:32.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.802 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723331 s, 145 MB/s 2024-04-25T00:06:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.811 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724128 s, 145 MB/s 2024-04-25T00:06:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650473 s, 161 MB/s 2024-04-25T00:06:32.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.826 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.827 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064182 s, 163 MB/s 2024-04-25T00:06:32.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.836 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.836 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746764 s, 140 MB/s 2024-04-25T00:06:32.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.843 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.844 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634542 s, 165 MB/s 2024-04-25T00:06:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668101 s, 157 MB/s 2024-04-25T00:06:32.860 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.860 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.860 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661472 s, 159 MB/s 2024-04-25T00:06:32.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.868 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.868 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669559 s, 157 MB/s 2024-04-25T00:06:32.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.877 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.877 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706032 s, 149 MB/s 2024-04-25T00:06:32.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:32.883 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:32.883 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:32.883 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T00:06:32.884 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:32.884 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:32.884 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:32.884 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:32.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:32.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.940 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.940 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595154 s, 176 MB/s 2024-04-25T00:06:32.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.949 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646989 s, 162 MB/s 2024-04-25T00:06:32.957 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.957 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.957 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707819 s, 148 MB/s 2024-04-25T00:06:32.964 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.964 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.964 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560791 s, 187 MB/s 2024-04-25T00:06:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.972 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581581 s, 180 MB/s 2024-04-25T00:06:32.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.980 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.980 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696464 s, 151 MB/s 2024-04-25T00:06:32.988 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664361 s, 158 MB/s 2024-04-25T00:06:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657824 s, 159 MB/s 2024-04-25T00:06:33.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.005 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701655 s, 149 MB/s 2024-04-25T00:06:33.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.014 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695674 s, 151 MB/s 2024-04-25T00:06:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639202 s, 164 MB/s 2024-04-25T00:06:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718578 s, 146 MB/s 2024-04-25T00:06:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.038 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560504 s, 187 MB/s 2024-04-25T00:06:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.047 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709675 s, 148 MB/s 2024-04-25T00:06:33.054 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.055 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645791 s, 162 MB/s 2024-04-25T00:06:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.062 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585022 s, 179 MB/s 2024-04-25T00:06:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670162 s, 156 MB/s 2024-04-25T00:06:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562197 s, 187 MB/s 2024-04-25T00:06:33.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.087 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720974 s, 145 MB/s 2024-04-25T00:06:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.095 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736834 s, 142 MB/s 2024-04-25T00:06:33.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.103 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.104 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064037 s, 164 MB/s 2024-04-25T00:06:33.111 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.112 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.112 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673284 s, 156 MB/s 2024-04-25T00:06:33.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.121 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074174 s, 141 MB/s 2024-04-25T00:06:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588259 s, 178 MB/s 2024-04-25T00:06:33.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.136 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672277 s, 156 MB/s 2024-04-25T00:06:33.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.145 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.145 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072291 s, 145 MB/s 2024-04-25T00:06:33.153 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.153 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.153 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638695 s, 164 MB/s 2024-04-25T00:06:33.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.162 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.162 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716047 s, 146 MB/s 2024-04-25T00:06:33.169 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.169 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.169 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560834 s, 187 MB/s 2024-04-25T00:06:33.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.177 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.177 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671771 s, 156 MB/s 2024-04-25T00:06:33.185 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.185 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.185 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064834 s, 162 MB/s 2024-04-25T00:06:33.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.192 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547909 s, 191 MB/s 2024-04-25T00:06:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:33.199 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:33.199 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:33.199 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T00:06:33.200 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:33.200 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:33.200 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:33.200 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:33.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:33.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.258 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.259 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693847 s, 151 MB/s 2024-04-25T00:06:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716638 s, 146 MB/s 2024-04-25T00:06:33.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.275 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662897 s, 158 MB/s 2024-04-25T00:06:33.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.283 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.283 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646934 s, 162 MB/s 2024-04-25T00:06:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655953 s, 160 MB/s 2024-04-25T00:06:33.299 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.299 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.300 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681877 s, 154 MB/s 2024-04-25T00:06:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725258 s, 145 MB/s 2024-04-25T00:06:33.317 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.317 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.317 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713626 s, 147 MB/s 2024-04-25T00:06:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649145 s, 162 MB/s 2024-04-25T00:06:33.334 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.334 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.334 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756347 s, 139 MB/s 2024-04-25T00:06:33.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649261 s, 162 MB/s 2024-04-25T00:06:33.349 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.349 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586807 s, 179 MB/s 2024-04-25T00:06:33.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.358 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730398 s, 144 MB/s 2024-04-25T00:06:33.367 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.367 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.367 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708849 s, 148 MB/s 2024-04-25T00:06:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724064 s, 145 MB/s 2024-04-25T00:06:33.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.384 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653901 s, 160 MB/s 2024-04-25T00:06:33.392 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.392 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.392 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664934 s, 158 MB/s 2024-04-25T00:06:33.400 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.401 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.401 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722144 s, 145 MB/s 2024-04-25T00:06:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729039 s, 144 MB/s 2024-04-25T00:06:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572116 s, 183 MB/s 2024-04-25T00:06:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640777 s, 164 MB/s 2024-04-25T00:06:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651728 s, 161 MB/s 2024-04-25T00:06:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677208 s, 155 MB/s 2024-04-25T00:06:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659062 s, 159 MB/s 2024-04-25T00:06:33.457 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643032 s, 163 MB/s 2024-04-25T00:06:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067478 s, 155 MB/s 2024-04-25T00:06:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666065 s, 157 MB/s 2024-04-25T00:06:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714768 s, 147 MB/s 2024-04-25T00:06:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573503 s, 183 MB/s 2024-04-25T00:06:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652024 s, 161 MB/s 2024-04-25T00:06:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732109 s, 143 MB/s 2024-04-25T00:06:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574873 s, 182 MB/s 2024-04-25T00:06:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:33.521 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:33.521 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:33.521 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T00:06:33.521 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:33.521 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:33.521 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:33.521 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:33.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701016 s, 150 MB/s 2024-04-25T00:06:33.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.588 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655266 s, 160 MB/s 2024-04-25T00:06:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681713 s, 154 MB/s 2024-04-25T00:06:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654544 s, 160 MB/s 2024-04-25T00:06:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599259 s, 175 MB/s 2024-04-25T00:06:33.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.620 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.620 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563125 s, 186 MB/s 2024-04-25T00:06:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.629 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.629 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072799 s, 144 MB/s 2024-04-25T00:06:33.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.636 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.636 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561575 s, 187 MB/s 2024-04-25T00:06:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646641 s, 162 MB/s 2024-04-25T00:06:33.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.652 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673703 s, 156 MB/s 2024-04-25T00:06:33.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.660 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.660 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652755 s, 161 MB/s 2024-04-25T00:06:33.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.670 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669641 s, 157 MB/s 2024-04-25T00:06:33.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.677 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.677 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680249 s, 154 MB/s 2024-04-25T00:06:33.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.685 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.685 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065805 s, 159 MB/s 2024-04-25T00:06:33.693 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.693 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.693 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627312 s, 167 MB/s 2024-04-25T00:06:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.702 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740488 s, 142 MB/s 2024-04-25T00:06:33.710 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.710 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.710 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661522 s, 159 MB/s 2024-04-25T00:06:33.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.719 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681574 s, 154 MB/s 2024-04-25T00:06:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685198 s, 153 MB/s 2024-04-25T00:06:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787305 s, 133 MB/s 2024-04-25T00:06:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066854 s, 157 MB/s 2024-04-25T00:06:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668939 s, 157 MB/s 2024-04-25T00:06:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675824 s, 155 MB/s 2024-04-25T00:06:33.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.770 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719111 s, 146 MB/s 2024-04-25T00:06:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662093 s, 158 MB/s 2024-04-25T00:06:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633795 s, 165 MB/s 2024-04-25T00:06:33.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678231 s, 155 MB/s 2024-04-25T00:06:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639571 s, 164 MB/s 2024-04-25T00:06:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639095 s, 164 MB/s 2024-04-25T00:06:33.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.818 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.818 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702497 s, 149 MB/s 2024-04-25T00:06:33.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.827 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736289 s, 142 MB/s 2024-04-25T00:06:33.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.835 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.835 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064632 s, 162 MB/s 2024-04-25T00:06:33.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:33.842 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:33.842 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:33.842 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T00:06:33.842 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:33.842 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:33.842 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:33.843 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:33.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.900 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.900 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649267 s, 162 MB/s 2024-04-25T00:06:33.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.909 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.909 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705592 s, 149 MB/s 2024-04-25T00:06:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678387 s, 155 MB/s 2024-04-25T00:06:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649581 s, 161 MB/s 2024-04-25T00:06:33.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.934 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724332 s, 145 MB/s 2024-04-25T00:06:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687404 s, 153 MB/s 2024-04-25T00:06:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702124 s, 149 MB/s 2024-04-25T00:06:33.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.959 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.959 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643258 s, 163 MB/s 2024-04-25T00:06:33.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.966 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555034 s, 189 MB/s 2024-04-25T00:06:33.974 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.975 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.975 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653419 s, 160 MB/s 2024-04-25T00:06:33.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.982 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675151 s, 155 MB/s 2024-04-25T00:06:33.990 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.990 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.990 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643214 s, 163 MB/s 2024-04-25T00:06:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064086 s, 164 MB/s 2024-04-25T00:06:34.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640493 s, 164 MB/s 2024-04-25T00:06:34.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.015 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647111 s, 162 MB/s 2024-04-25T00:06:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676556 s, 155 MB/s 2024-04-25T00:06:34.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.031 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.031 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732342 s, 143 MB/s 2024-04-25T00:06:34.038 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.039 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056608 s, 185 MB/s 2024-04-25T00:06:34.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.047 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.047 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650632 s, 161 MB/s 2024-04-25T00:06:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642775 s, 163 MB/s 2024-04-25T00:06:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712032 s, 147 MB/s 2024-04-25T00:06:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575417 s, 182 MB/s 2024-04-25T00:06:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646853 s, 162 MB/s 2024-04-25T00:06:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714978 s, 147 MB/s 2024-04-25T00:06:34.096 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.096 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.096 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678281 s, 155 MB/s 2024-04-25T00:06:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720935 s, 145 MB/s 2024-04-25T00:06:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.114 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716873 s, 146 MB/s 2024-04-25T00:06:34.121 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.122 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.122 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665729 s, 158 MB/s 2024-04-25T00:06:34.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.130 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652854 s, 161 MB/s 2024-04-25T00:06:34.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.138 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624813 s, 168 MB/s 2024-04-25T00:06:34.146 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066675 s, 157 MB/s 2024-04-25T00:06:34.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.156 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.156 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763656 s, 137 MB/s 2024-04-25T00:06:34.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:34.163 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:34.163 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:34.163 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T00:06:34.163 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:34.163 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:34.164 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:34.164 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:34.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.221 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576284 s, 182 MB/s 2024-04-25T00:06:34.227 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562582 s, 186 MB/s 2024-04-25T00:06:34.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.237 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729329 s, 144 MB/s 2024-04-25T00:06:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727041 s, 144 MB/s 2024-04-25T00:06:34.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.254 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069502 s, 151 MB/s 2024-04-25T00:06:34.263 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.263 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.263 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726014 s, 144 MB/s 2024-04-25T00:06:34.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568408 s, 184 MB/s 2024-04-25T00:06:34.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.278 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639876 s, 164 MB/s 2024-04-25T00:06:34.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.285 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580297 s, 181 MB/s 2024-04-25T00:06:34.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.294 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.294 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752535 s, 139 MB/s 2024-04-25T00:06:34.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.302 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.302 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570973 s, 184 MB/s 2024-04-25T00:06:34.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.310 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720921 s, 145 MB/s 2024-04-25T00:06:34.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.320 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.320 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758194 s, 138 MB/s 2024-04-25T00:06:34.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.327 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587346 s, 179 MB/s 2024-04-25T00:06:34.336 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.336 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.336 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724433 s, 145 MB/s 2024-04-25T00:06:34.344 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.344 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.345 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674069 s, 156 MB/s 2024-04-25T00:06:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649971 s, 161 MB/s 2024-04-25T00:06:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639478 s, 164 MB/s 2024-04-25T00:06:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737369 s, 142 MB/s 2024-04-25T00:06:34.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.378 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.378 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706045 s, 149 MB/s 2024-04-25T00:06:34.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.387 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.387 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726541 s, 144 MB/s 2024-04-25T00:06:34.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.394 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568687 s, 184 MB/s 2024-04-25T00:06:34.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.403 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.403 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664359 s, 158 MB/s 2024-04-25T00:06:34.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.411 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675334 s, 155 MB/s 2024-04-25T00:06:34.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.418 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.418 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561422 s, 187 MB/s 2024-04-25T00:06:34.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.426 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612664 s, 171 MB/s 2024-04-25T00:06:34.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.435 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.435 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707487 s, 148 MB/s 2024-04-25T00:06:34.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.443 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.443 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660333 s, 159 MB/s 2024-04-25T00:06:34.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065433 s, 160 MB/s 2024-04-25T00:06:34.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.460 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704359 s, 149 MB/s 2024-04-25T00:06:34.468 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.468 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.468 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668974 s, 157 MB/s 2024-04-25T00:06:34.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.475 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.475 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573361 s, 183 MB/s 2024-04-25T00:06:34.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:34.485 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:34.485 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:34.485 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T00:06:34.485 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:34.485 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:34.485 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:34.486 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:34.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.544 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.544 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071841 s, 146 MB/s 2024-04-25T00:06:34.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.552 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.552 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693279 s, 151 MB/s 2024-04-25T00:06:34.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.560 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.560 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658869 s, 159 MB/s 2024-04-25T00:06:34.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.568 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.568 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711081 s, 147 MB/s 2024-04-25T00:06:34.577 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.577 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.577 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667184 s, 157 MB/s 2024-04-25T00:06:34.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.584 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.584 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551466 s, 190 MB/s 2024-04-25T00:06:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605384 s, 173 MB/s 2024-04-25T00:06:34.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.600 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.601 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074494 s, 141 MB/s 2024-04-25T00:06:34.608 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.608 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.608 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563371 s, 186 MB/s 2024-04-25T00:06:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666153 s, 157 MB/s 2024-04-25T00:06:34.625 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.625 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.625 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727003 s, 144 MB/s 2024-04-25T00:06:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.633 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709804 s, 148 MB/s 2024-04-25T00:06:34.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.642 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.643 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761554 s, 138 MB/s 2024-04-25T00:06:34.651 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.651 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.651 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658723 s, 159 MB/s 2024-04-25T00:06:34.659 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.659 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.659 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673247 s, 156 MB/s 2024-04-25T00:06:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.666 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577196 s, 182 MB/s 2024-04-25T00:06:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708206 s, 148 MB/s 2024-04-25T00:06:34.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.684 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.684 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725244 s, 145 MB/s 2024-04-25T00:06:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.692 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613779 s, 171 MB/s 2024-04-25T00:06:34.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712061 s, 147 MB/s 2024-04-25T00:06:34.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.708 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.708 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608435 s, 172 MB/s 2024-04-25T00:06:34.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.716 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.716 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058087 s, 181 MB/s 2024-04-25T00:06:34.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.725 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.725 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706334 s, 148 MB/s 2024-04-25T00:06:34.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.733 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.733 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588137 s, 178 MB/s 2024-04-25T00:06:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780198 s, 134 MB/s 2024-04-25T00:06:34.752 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.752 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.752 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725511 s, 145 MB/s 2024-04-25T00:06:34.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.760 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.760 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671941 s, 156 MB/s 2024-04-25T00:06:34.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.769 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.769 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665634 s, 158 MB/s 2024-04-25T00:06:34.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.777 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661977 s, 158 MB/s 2024-04-25T00:06:34.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.786 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724622 s, 145 MB/s 2024-04-25T00:06:34.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.795 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.795 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707786 s, 148 MB/s 2024-04-25T00:06:34.804 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.805 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.805 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733465 s, 143 MB/s 2024-04-25T00:06:34.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:34.811 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:34.812 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:34.812 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T00:06:34.812 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:34.812 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:34.812 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:34.812 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:34.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:34.873 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.874 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.874 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735407 s, 143 MB/s 2024-04-25T00:06:34.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.882 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.882 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666377 s, 157 MB/s 2024-04-25T00:06:34.891 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.891 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.891 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710921 s, 147 MB/s 2024-04-25T00:06:34.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.901 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.902 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075499 s, 139 MB/s 2024-04-25T00:06:34.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.910 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.910 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756436 s, 139 MB/s 2024-04-25T00:06:34.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.918 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.918 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574293 s, 183 MB/s 2024-04-25T00:06:34.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.926 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.926 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577138 s, 182 MB/s 2024-04-25T00:06:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.935 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650639 s, 161 MB/s 2024-04-25T00:06:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641771 s, 163 MB/s 2024-04-25T00:06:34.951 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.951 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.951 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641257 s, 164 MB/s 2024-04-25T00:06:34.960 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.961 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072571 s, 144 MB/s 2024-04-25T00:06:34.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.970 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.970 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713786 s, 147 MB/s 2024-04-25T00:06:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.978 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651644 s, 161 MB/s 2024-04-25T00:06:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643104 s, 163 MB/s 2024-04-25T00:06:34.995 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:34.996 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:34.996 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704237 s, 149 MB/s 2024-04-25T00:06:35.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.004 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.004 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687282 s, 153 MB/s 2024-04-25T00:06:35.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.013 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.013 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725023 s, 145 MB/s 2024-04-25T00:06:35.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.023 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761342 s, 138 MB/s 2024-04-25T00:06:35.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.033 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776222 s, 135 MB/s 2024-04-25T00:06:35.041 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703824 s, 149 MB/s 2024-04-25T00:06:35.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.050 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640105 s, 164 MB/s 2024-04-25T00:06:35.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698741 s, 150 MB/s 2024-04-25T00:06:35.067 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.068 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.068 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656995 s, 160 MB/s 2024-04-25T00:06:35.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.076 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.076 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716373 s, 146 MB/s 2024-04-25T00:06:35.085 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.086 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.086 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736024 s, 142 MB/s 2024-04-25T00:06:35.093 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.093 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.093 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568421 s, 184 MB/s 2024-04-25T00:06:35.102 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.102 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.102 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661466 s, 159 MB/s 2024-04-25T00:06:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650145 s, 161 MB/s 2024-04-25T00:06:35.119 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.120 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.120 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745057 s, 141 MB/s 2024-04-25T00:06:35.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.128 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068447 s, 153 MB/s 2024-04-25T00:06:35.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.138 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.138 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725787 s, 144 MB/s 2024-04-25T00:06:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754995 s, 139 MB/s 2024-04-25T00:06:35.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682022 s, 154 MB/s 2024-04-25T00:06:35.222 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.222 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.222 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671828 s, 156 MB/s 2024-04-25T00:06:35.230 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.230 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.230 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646358 s, 162 MB/s 2024-04-25T00:06:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057904 s, 181 MB/s 2024-04-25T00:06:35.247 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.247 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.247 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719083 s, 146 MB/s 2024-04-25T00:06:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.254 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547543 s, 192 MB/s 2024-04-25T00:06:35.263 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.264 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.264 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712612 s, 147 MB/s 2024-04-25T00:06:35.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.272 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.272 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665226 s, 158 MB/s 2024-04-25T00:06:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.281 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674786 s, 155 MB/s 2024-04-25T00:06:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.290 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724013 s, 145 MB/s 2024-04-25T00:06:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.297 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557184 s, 188 MB/s 2024-04-25T00:06:35.307 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.307 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.307 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720764 s, 145 MB/s 2024-04-25T00:06:35.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705442 s, 149 MB/s 2024-04-25T00:06:35.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.324 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679454 s, 154 MB/s 2024-04-25T00:06:35.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.333 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641297 s, 164 MB/s 2024-04-25T00:06:35.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.342 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.342 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696699 s, 151 MB/s 2024-04-25T00:06:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665403 s, 158 MB/s 2024-04-25T00:06:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652718 s, 161 MB/s 2024-04-25T00:06:35.368 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.368 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.368 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727107 s, 144 MB/s 2024-04-25T00:06:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576846 s, 182 MB/s 2024-04-25T00:06:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652887 s, 161 MB/s 2024-04-25T00:06:35.393 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.394 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.394 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718027 s, 146 MB/s 2024-04-25T00:06:35.402 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.402 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.402 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663795 s, 158 MB/s 2024-04-25T00:06:35.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.411 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.411 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066491 s, 158 MB/s 2024-04-25T00:06:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.420 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.420 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702162 s, 149 MB/s 2024-04-25T00:06:35.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.428 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.428 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678328 s, 155 MB/s 2024-04-25T00:06:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662265 s, 158 MB/s 2024-04-25T00:06:35.445 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.445 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.445 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610732 s, 172 MB/s 2024-04-25T00:06:35.457 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.457 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.457 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721631 s, 145 MB/s 2024-04-25T00:06:35.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.463 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.463 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707167 s, 148 MB/s 2024-04-25T00:06:35.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.472 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.472 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740989 s, 142 MB/s 2024-04-25T00:06:35.481 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.481 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.481 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696551 s, 151 MB/s 2024-04-25T00:06:35.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:35.488 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:35.488 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:35.488 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T00:06:35.488 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:35.488 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:35.488 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:35.488 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:35.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:35.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.547 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.548 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700843 s, 150 MB/s 2024-04-25T00:06:35.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.556 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.556 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653905 s, 160 MB/s 2024-04-25T00:06:35.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.566 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736005 s, 142 MB/s 2024-04-25T00:06:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660269 s, 159 MB/s 2024-04-25T00:06:35.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.582 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.583 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654364 s, 160 MB/s 2024-04-25T00:06:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.591 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066761 s, 157 MB/s 2024-04-25T00:06:35.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.599 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.599 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633847 s, 165 MB/s 2024-04-25T00:06:35.608 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.608 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.608 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737233 s, 142 MB/s 2024-04-25T00:06:35.618 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.618 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.618 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772518 s, 136 MB/s 2024-04-25T00:06:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.627 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.627 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685291 s, 153 MB/s 2024-04-25T00:06:35.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.635 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.635 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651023 s, 161 MB/s 2024-04-25T00:06:35.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.644 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.644 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764968 s, 137 MB/s 2024-04-25T00:06:35.652 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.653 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.653 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065849 s, 159 MB/s 2024-04-25T00:06:35.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.661 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.661 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678806 s, 154 MB/s 2024-04-25T00:06:35.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.669 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.669 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624102 s, 168 MB/s 2024-04-25T00:06:35.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.678 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.678 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076926 s, 136 MB/s 2024-04-25T00:06:35.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.687 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.687 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071437 s, 147 MB/s 2024-04-25T00:06:35.696 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.696 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.696 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752636 s, 139 MB/s 2024-04-25T00:06:35.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.704 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.704 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669616 s, 157 MB/s 2024-04-25T00:06:35.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.713 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.713 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664684 s, 158 MB/s 2024-04-25T00:06:35.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.722 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.722 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728039 s, 144 MB/s 2024-04-25T00:06:35.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.731 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.731 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736337 s, 142 MB/s 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668645 s, 157 MB/s 2024-04-25T00:06:35.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.748 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.748 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635167 s, 165 MB/s 2024-04-25T00:06:35.757 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074519 s, 141 MB/s 2024-04-25T00:06:35.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.767 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759065 s, 138 MB/s 2024-04-25T00:06:35.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.776 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.776 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734852 s, 143 MB/s 2024-04-25T00:06:35.784 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.785 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.785 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067902 s, 154 MB/s 2024-04-25T00:06:35.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.794 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.794 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780774 s, 134 MB/s 2024-04-25T00:06:35.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.803 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.803 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673478 s, 156 MB/s 2024-04-25T00:06:35.811 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.811 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.811 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640076 s, 164 MB/s 2024-04-25T00:06:35.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.819 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.819 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627458 s, 167 MB/s 2024-04-25T00:06:35.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:35.825 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:35.825 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:35.826 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T00:06:35.826 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:35.826 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:35.826 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:35.826 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:35.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:35.885 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.886 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.886 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736457 s, 142 MB/s 2024-04-25T00:06:35.893 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.894 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.894 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656862 s, 160 MB/s 2024-04-25T00:06:35.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.902 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.902 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715187 s, 147 MB/s 2024-04-25T00:06:35.911 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.911 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.912 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772218 s, 136 MB/s 2024-04-25T00:06:35.920 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.920 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.921 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714654 s, 147 MB/s 2024-04-25T00:06:35.928 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.928 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.928 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622364 s, 168 MB/s 2024-04-25T00:06:35.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.936 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.936 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643777 s, 163 MB/s 2024-04-25T00:06:35.944 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.944 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.944 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653045 s, 161 MB/s 2024-04-25T00:06:35.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.953 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.953 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072271 s, 145 MB/s 2024-04-25T00:06:35.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.961 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.961 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065965 s, 159 MB/s 2024-04-25T00:06:35.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.969 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.970 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665628 s, 158 MB/s 2024-04-25T00:06:35.978 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.978 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.978 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662365 s, 158 MB/s 2024-04-25T00:06:35.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.986 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.986 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650834 s, 161 MB/s 2024-04-25T00:06:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058501 s, 179 MB/s 2024-04-25T00:06:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582184 s, 180 MB/s 2024-04-25T00:06:36.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.009 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.009 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653633 s, 160 MB/s 2024-04-25T00:06:36.016 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.016 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.016 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549828 s, 191 MB/s 2024-04-25T00:06:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648383 s, 162 MB/s 2024-04-25T00:06:36.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.033 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.033 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071415 s, 147 MB/s 2024-04-25T00:06:36.041 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.042 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.042 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724836 s, 145 MB/s 2024-04-25T00:06:36.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.050 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.050 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670342 s, 156 MB/s 2024-04-25T00:06:36.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.059 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.059 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715424 s, 147 MB/s 2024-04-25T00:06:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584084 s, 180 MB/s 2024-04-25T00:06:36.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.075 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.075 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708997 s, 148 MB/s 2024-04-25T00:06:36.082 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.082 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.082 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057606 s, 182 MB/s 2024-04-25T00:06:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.089 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554394 s, 189 MB/s 2024-04-25T00:06:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724806 s, 145 MB/s 2024-04-25T00:06:36.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.106 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624847 s, 168 MB/s 2024-04-25T00:06:36.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.114 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.114 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620629 s, 169 MB/s 2024-04-25T00:06:36.122 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.122 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.122 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663299 s, 158 MB/s 2024-04-25T00:06:36.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.131 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.131 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072066 s, 146 MB/s 2024-04-25T00:06:36.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.139 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.139 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661724 s, 158 MB/s 2024-04-25T00:06:36.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:36.146 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:36.146 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:36.146 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T00:06:36.146 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:36.146 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:36.147 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:36.147 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:36.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.204 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.204 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647166 s, 162 MB/s 2024-04-25T00:06:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.212 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.212 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573238 s, 183 MB/s 2024-04-25T00:06:36.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.220 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688118 s, 152 MB/s 2024-04-25T00:06:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633677 s, 165 MB/s 2024-04-25T00:06:36.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.237 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.237 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722158 s, 145 MB/s 2024-04-25T00:06:36.246 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.246 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.246 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721139 s, 145 MB/s 2024-04-25T00:06:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645268 s, 163 MB/s 2024-04-25T00:06:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639609 s, 164 MB/s 2024-04-25T00:06:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622038 s, 169 MB/s 2024-04-25T00:06:36.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.278 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.279 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744476 s, 141 MB/s 2024-04-25T00:06:36.288 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.288 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.288 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766903 s, 137 MB/s 2024-04-25T00:06:36.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.296 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.296 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643648 s, 163 MB/s 2024-04-25T00:06:36.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.303 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058096 s, 180 MB/s 2024-04-25T00:06:36.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.312 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.312 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705534 s, 149 MB/s 2024-04-25T00:06:36.319 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.319 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.319 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567373 s, 185 MB/s 2024-04-25T00:06:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067057 s, 156 MB/s 2024-04-25T00:06:36.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.335 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567141 s, 185 MB/s 2024-04-25T00:06:36.343 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.343 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.343 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663878 s, 158 MB/s 2024-04-25T00:06:36.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.350 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.350 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584516 s, 179 MB/s 2024-04-25T00:06:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.358 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.358 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571229 s, 184 MB/s 2024-04-25T00:06:36.366 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738933 s, 142 MB/s 2024-04-25T00:06:36.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.375 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.375 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676494 s, 155 MB/s 2024-04-25T00:06:36.383 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.383 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.383 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699655 s, 150 MB/s 2024-04-25T00:06:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.391 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651555 s, 161 MB/s 2024-04-25T00:06:36.399 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.399 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.399 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664827 s, 158 MB/s 2024-04-25T00:06:36.408 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.408 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.408 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734119 s, 143 MB/s 2024-04-25T00:06:36.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.417 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.417 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699517 s, 150 MB/s 2024-04-25T00:06:36.425 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.426 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.426 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731082 s, 143 MB/s 2024-04-25T00:06:36.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.434 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.434 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645521 s, 162 MB/s 2024-04-25T00:06:36.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.442 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.443 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735251 s, 143 MB/s 2024-04-25T00:06:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698129 s, 150 MB/s 2024-04-25T00:06:36.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.460 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.460 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722004 s, 145 MB/s 2024-04-25T00:06:36.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:36.467 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:36.467 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:36.467 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T00:06:36.467 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:36.467 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:36.467 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:36.467 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:36.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:36.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.524 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.524 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632882 s, 166 MB/s 2024-04-25T00:06:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.532 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639353 s, 164 MB/s 2024-04-25T00:06:36.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.540 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620592 s, 169 MB/s 2024-04-25T00:06:36.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.548 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.548 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708272 s, 148 MB/s 2024-04-25T00:06:36.557 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.557 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.557 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717953 s, 146 MB/s 2024-04-25T00:06:36.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.565 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.565 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612784 s, 171 MB/s 2024-04-25T00:06:36.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.572 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.572 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559801 s, 187 MB/s 2024-04-25T00:06:36.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.579 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.580 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588606 s, 178 MB/s 2024-04-25T00:06:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.588 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668462 s, 157 MB/s 2024-04-25T00:06:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.598 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.598 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788904 s, 133 MB/s 2024-04-25T00:06:36.605 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.606 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.606 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676846 s, 155 MB/s 2024-04-25T00:06:36.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.614 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.614 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655357 s, 160 MB/s 2024-04-25T00:06:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741242 s, 141 MB/s 2024-04-25T00:06:36.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.631 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.631 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705438 s, 149 MB/s 2024-04-25T00:06:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.639 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567589 s, 185 MB/s 2024-04-25T00:06:36.647 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.648 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.648 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728618 s, 144 MB/s 2024-04-25T00:06:36.656 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.657 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.657 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748527 s, 140 MB/s 2024-04-25T00:06:36.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.666 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.666 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765301 s, 137 MB/s 2024-04-25T00:06:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630045 s, 166 MB/s 2024-04-25T00:06:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.683 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746621 s, 140 MB/s 2024-04-25T00:06:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.691 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649731 s, 161 MB/s 2024-04-25T00:06:36.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.700 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.700 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776373 s, 135 MB/s 2024-04-25T00:06:36.709 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.709 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.709 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728877 s, 144 MB/s 2024-04-25T00:06:36.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.718 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.718 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730882 s, 143 MB/s 2024-04-25T00:06:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680926 s, 154 MB/s 2024-04-25T00:06:36.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.735 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.735 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723469 s, 145 MB/s 2024-04-25T00:06:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720175 s, 146 MB/s 2024-04-25T00:06:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744957 s, 141 MB/s 2024-04-25T00:06:36.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.762 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.762 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698552 s, 150 MB/s 2024-04-25T00:06:36.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.770 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.770 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691847 s, 152 MB/s 2024-04-25T00:06:36.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.777 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.777 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566215 s, 185 MB/s 2024-04-25T00:06:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696457 s, 151 MB/s 2024-04-25T00:06:36.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:36.793 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:36.793 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:36.793 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T00:06:36.793 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:36.793 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:36.793 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:36.793 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:36.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:36.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.850 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.851 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577243 s, 182 MB/s 2024-04-25T00:06:36.858 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.859 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.859 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006733 s, 156 MB/s 2024-04-25T00:06:36.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.866 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.866 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059755 s, 175 MB/s 2024-04-25T00:06:36.876 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.876 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.876 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752411 s, 139 MB/s 2024-04-25T00:06:36.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.883 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.883 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584833 s, 179 MB/s 2024-04-25T00:06:36.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.890 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.890 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595693 s, 176 MB/s 2024-04-25T00:06:36.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.898 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.898 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650617 s, 161 MB/s 2024-04-25T00:06:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566367 s, 185 MB/s 2024-04-25T00:06:36.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.914 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722292 s, 145 MB/s 2024-04-25T00:06:36.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.924 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738453 s, 142 MB/s 2024-04-25T00:06:36.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.932 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.933 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739879 s, 142 MB/s 2024-04-25T00:06:36.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.941 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.941 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709625 s, 148 MB/s 2024-04-25T00:06:36.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.949 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627764 s, 167 MB/s 2024-04-25T00:06:36.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.958 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.958 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713455 s, 147 MB/s 2024-04-25T00:06:36.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.966 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.967 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704569 s, 149 MB/s 2024-04-25T00:06:36.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.973 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.973 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557763 s, 188 MB/s 2024-04-25T00:06:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.982 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674054 s, 156 MB/s 2024-04-25T00:06:36.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.989 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.989 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576299 s, 182 MB/s 2024-04-25T00:06:36.997 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669542 s, 157 MB/s 2024-04-25T00:06:37.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.006 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.006 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722251 s, 145 MB/s 2024-04-25T00:06:37.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.014 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.015 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652596 s, 161 MB/s 2024-04-25T00:06:37.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.023 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.023 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716994 s, 146 MB/s 2024-04-25T00:06:37.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.032 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.032 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688992 s, 152 MB/s 2024-04-25T00:06:37.039 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.040 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.040 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630672 s, 166 MB/s 2024-04-25T00:06:37.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.048 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.048 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656861 s, 160 MB/s 2024-04-25T00:06:37.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.056 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707985 s, 148 MB/s 2024-04-25T00:06:37.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.064 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.064 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639641 s, 164 MB/s 2024-04-25T00:06:37.071 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.072 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.072 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590229 s, 178 MB/s 2024-04-25T00:06:37.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.080 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.080 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651945 s, 161 MB/s 2024-04-25T00:06:37.088 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.089 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.089 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711318 s, 147 MB/s 2024-04-25T00:06:37.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.097 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.097 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668457 s, 157 MB/s 2024-04-25T00:06:37.105 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.106 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.106 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720724 s, 145 MB/s 2024-04-25T00:06:37.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:37.112 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-04-25T00:06:37.112 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:06:37.113 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T00:06:37.113 DEBUG:teuthology.orchestra.run.smithi113:> for ((i = 0; i < 32; i++)); do 2024-04-25T00:06:37.113 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T00:06:37.113 DEBUG:teuthology.orchestra.run.smithi113:> done 2024-04-25T00:06:37.113 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:06:37.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:06:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.171 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744965 s, 141 MB/s 2024-04-25T00:06:37.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.178 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.178 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579034 s, 181 MB/s 2024-04-25T00:06:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.186 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582985 s, 180 MB/s 2024-04-25T00:06:37.193 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.193 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.193 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568959 s, 184 MB/s 2024-04-25T00:06:37.201 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.201 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066181 s, 158 MB/s 2024-04-25T00:06:37.210 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.210 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.210 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741989 s, 141 MB/s 2024-04-25T00:06:37.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.219 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.219 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068753 s, 153 MB/s 2024-04-25T00:06:37.226 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.226 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.226 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577408 s, 182 MB/s 2024-04-25T00:06:37.234 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.234 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.235 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686078 s, 153 MB/s 2024-04-25T00:06:37.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.242 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.242 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639221 s, 164 MB/s 2024-04-25T00:06:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.251 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683843 s, 153 MB/s 2024-04-25T00:06:37.259 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.259 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.259 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638108 s, 164 MB/s 2024-04-25T00:06:37.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.267 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.267 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659768 s, 159 MB/s 2024-04-25T00:06:37.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.275 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.275 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620223 s, 169 MB/s 2024-04-25T00:06:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.284 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712087 s, 147 MB/s 2024-04-25T00:06:37.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.291 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.292 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638305 s, 164 MB/s 2024-04-25T00:06:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758708 s, 138 MB/s 2024-04-25T00:06:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069087 s, 152 MB/s 2024-04-25T00:06:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552395 s, 190 MB/s 2024-04-25T00:06:37.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.324 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.325 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678405 s, 155 MB/s 2024-04-25T00:06:37.332 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.333 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.333 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063718 s, 165 MB/s 2024-04-25T00:06:37.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.340 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.340 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585853 s, 179 MB/s 2024-04-25T00:06:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.348 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548595 s, 191 MB/s 2024-04-25T00:06:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715234 s, 147 MB/s 2024-04-25T00:06:37.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.363 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.363 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550501 s, 190 MB/s 2024-04-25T00:06:37.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.372 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.372 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701828 s, 149 MB/s 2024-04-25T00:06:37.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.380 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070211 s, 149 MB/s 2024-04-25T00:06:37.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.389 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.389 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669808 s, 157 MB/s 2024-04-25T00:06:37.397 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.397 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.397 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735141 s, 143 MB/s 2024-04-25T00:06:37.405 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.405 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.406 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642677 s, 163 MB/s 2024-04-25T00:06:37.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.414 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.414 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664369 s, 158 MB/s 2024-04-25T00:06:37.422 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-25T00:06:37.422 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-25T00:06:37.422 INFO:teuthology.orchestra.run.smithi113.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650778 s, 161 MB/s 2024-04-25T00:06:37.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:06:37.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:37.429 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T00:06:37.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.530+0000 7fe8fc27b640 1 Processor -- start 2024-04-25T00:06:37.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.530+0000 7fe8fc27b640 1 -- start start 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8fc27b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 0x7fe8f40b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8fc27b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f4072f70 con 0x7fe8f406e7e0 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8fc27b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f40b2fd0 con 0x7fe8f40b3ff0 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8fc27b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f40b32b0 con 0x7fe8f4144a80 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8fa818640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe8f406e7e0 0x7fe8f40cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57934/0 (socket says 172.21.15.113:57934) 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8fa818640 1 -- 172.21.15.113:0/913207794 learned_addr learned my addr 172.21.15.113:0/913207794 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485096553 0 0) 0x7fe8f4072f70 con 0x7fe8f406e7e0 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8fa017640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 0x7fe8f40b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8e0003520 con 0x7fe8f406e7e0 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3658082032 0 0) 0x7fe8e0003520 con 0x7fe8f406e7e0 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 msgr2=0x7fe8f40b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 0x7fe8f40b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 >> v1:172.21.15.178:6789/0 conn(0x7fe8f40b3ff0 legacy=0x7fe8f406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f40b3590 con 0x7fe8f406e7e0 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe8ec0019f0 con 0x7fe8f406e7e0 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 >> v1:172.21.15.113:6789/0 conn(0x7fe8f406e7e0 legacy=0x7fe8f40cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 0x7fe8e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e0003f60 0x7fe8e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e0004d40 0x7fe8e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f40b32b0 con 0x7fe8e0004d40 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e000bb50 con 0x7fe8e0003f60 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.531+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e000bd50 con 0x7fe8f4144a80 2024-04-25T00:06:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8fa818640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e0004d40 0x7fe8e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8f9816640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e0003f60 0x7fe8e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8fa017640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 0x7fe8e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8fa818640 1 -- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 msgr2=0x7fe8e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8fa818640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8f4144a80 0x7fe8e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8fa818640 1 -- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e0003f60 msgr2=0x7fe8e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8fa818640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e0003f60 0x7fe8e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.532+0000 7fe8fa818640 1 -- 172.21.15.113:0/913207794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e0003520 con 0x7fe8e0004d40 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fa818640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e0004d40 0x7fe8e00051c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe8f406c3b0 tx=0x7fe8e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578308fc597b2eda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00a4a0 con 0x7fe8e0004d40 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec02f070 con 0x7fe8e0004d40 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8f9015640 1 -- 172.21.15.113:0/913207794 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec02bcb0 con 0x7fe8e0004d40 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fc27b640 1 -- 172.21.15.113:0/913207794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e0004d40 msgr2=0x7fe8e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fc27b640 1 --2- 172.21.15.113:0/913207794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e0004d40 0x7fe8e00051c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe8f406c3b0 tx=0x7fe8e0005830 comp rx=0 tx=0).stop 2024-04-25T00:06:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fa818640 1 -- 172.21.15.113:0/913207794 reap_dead start 2024-04-25T00:06:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fc27b640 1 -- 172.21.15.113:0/913207794 shutdown_connections 2024-04-25T00:06:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fc27b640 1 -- 172.21.15.113:0/913207794 >> 172.21.15.113:0/913207794 conn(0x7fe8f40b1b30 msgr2=0x7fe8f40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fc27b640 1 -- 172.21.15.113:0/913207794 shutdown_connections 2024-04-25T00:06:37.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.533+0000 7fe8fc27b640 1 -- 172.21.15.113:0/913207794 wait complete. 2024-04-25T00:06:37.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.534+0000 7fe8fc27b640 1 Processor -- start 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fc27b640 1 -- start start 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fc27b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e0004d40 0x7fe8f4142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fc27b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8f41f6b50 0x7fe8f41f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fc27b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f40ada20 0x7fe8f40adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fc27b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f40b3ab0 con 0x7fe8f41f6b50 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fc27b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f40cdf50 con 0x7fe8f40ada20 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fc27b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f40b3c30 con 0x7fe8e0004d40 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8f9816640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8f41f6b50 0x7fe8f41f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8f9816640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8f41f6b50 0x7fe8f41f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54262/0 (socket says 172.21.15.113:54262) 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8f9816640 1 -- 172.21.15.113:0/1131953351 learned_addr learned my addr 172.21.15.113:0/1131953351 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fa017640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e0004d40 0x7fe8f4142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.535+0000 7fe8fa818640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f40ada20 0x7fe8f40adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8f9816640 1 -- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e0004d40 msgr2=0x7fe8f4142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8f9816640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e0004d40 0x7fe8f4142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8f9816640 1 -- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f40ada20 msgr2=0x7fe8f40adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8f9816640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f40ada20 0x7fe8f40adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8f9816640 1 -- 172.21.15.113:0/1131953351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f40ae710 con 0x7fe8f41f6b50 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8f9816640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8f41f6b50 0x7fe8f41f6fd0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400f820 tx=0x7fe8e400fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8eaffd640 1 -- 172.21.15.113:0/1131953351 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40050d0 con 0x7fe8f41f6b50 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8eaffd640 1 -- 172.21.15.113:0/1131953351 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e4018070 con 0x7fe8f41f6b50 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f40ae9f0 con 0x7fe8f41f6b50 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8eaffd640 1 -- 172.21.15.113:0/1131953351 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400d730 con 0x7fe8f41f6b50 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.536+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f40aefb0 con 0x7fe8f41f6b50 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.538+0000 7fe8eaffd640 1 -- 172.21.15.113:0/1131953351 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8e400d8d0 con 0x7fe8f41f6b50 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.538+0000 7fe8eaffd640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d806be60 0x7fe8d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.538+0000 7fe8eaffd640 1 -- 172.21.15.113:0/1131953351 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fe8e408abf0 con 0x7fe8f41f6b50 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.538+0000 7fe8fa017640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d806be60 0x7fe8d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.539+0000 7fe8fa017640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d806be60 0x7fe8d806e320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009bc0 tx=0x7fe8f0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.539+0000 7fe8fef82740 1 Processor -- start 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.539+0000 7fe8fef82740 1 -- start start 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c0695f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0696237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069623d00 0x55c0695f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fef82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c06955b930 con 0x55c069623d00 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c069503cc0 con 0x55c069641320 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c0695578f0 con 0x55c069624440 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069623d00 0x55c0695f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069623d00 0x55c0695f8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54264/0 (socket says 172.21.15.113:54264) 2024-04-25T00:06:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 -- 172.21.15.113:0/3070427961 learned_addr learned my addr 172.21.15.113:0/3070427961 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8f9816640 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0696237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c0695f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 -- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 msgr2=0x55c0696237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0696237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 -- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 msgr2=0x55c0695f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c0695f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa818640 1 -- 172.21.15.113:0/3070427961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c06933bba0 con 0x55c069623d00 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.540+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c0695f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fa818640 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069623d00 0x55c0695f8b10 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec02b500 tx=0x7fe8ec007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21130f65d0640910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8d7fff640 1 -- 172.21.15.113:0/3070427961 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0019f0 con 0x55c069623d00 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8d7fff640 1 -- 172.21.15.113:0/3070427961 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec02f070 con 0x55c069623d00 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8d7fff640 1 -- 172.21.15.113:0/3070427961 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec02ba90 con 0x55c069623d00 2024-04-25T00:06:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fef82740 1 -- 172.21.15.113:0/3070427961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069623d00 msgr2=0x55c0695f8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069623d00 0x55c0695f8b10 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec02b500 tx=0x7fe8ec007000 comp rx=0 tx=0).stop 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fef82740 1 -- 172.21.15.113:0/3070427961 shutdown_connections 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069623d00 0x55c0695f8b10 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0696237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3070427961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c0695f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.541+0000 7fe8fef82740 1 -- 172.21.15.113:0/3070427961 >> 172.21.15.113:0/3070427961 conn(0x55c0695f5f20 msgr2=0x55c06963e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.542+0000 7fe8fef82740 1 -- 172.21.15.113:0/3070427961 shutdown_connections 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.542+0000 7fe8fef82740 1 -- 172.21.15.113:0/3070427961 wait complete. 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.542+0000 7fe8fef82740 1 -- start start 2024-04-25T00:06:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.542+0000 7fe8fef82740 1 Processor -- start 2024-04-25T00:06:37.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c069611060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069641320 0x55c069622c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0696231c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fef82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c0695f80d0 con 0x55c069641320 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c069517330 con 0x55c0696260b0 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c0695f74c0 con 0x55c069624440 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fef82740 10 client.0 _ll_register_callbacks cb 0x55c069553950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069641320 0x55c069622c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069641320 0x55c069622c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54266/0 (socket says 172.21.15.113:54266) 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 -- 172.21.15.113:0/1558262737 learned_addr learned my addr 172.21.15.113:0/1558262737 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fa818640 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0696231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8fa017640 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c069611060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 -- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 msgr2=0x55c069611060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c069611060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 -- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 msgr2=0x55c0696231c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0696231c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.543+0000 7fe8f9816640 1 -- 172.21.15.113:0/1558262737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06922f490 con 0x55c069641320 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8f9816640 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069641320 0x55c069622c80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d520 tx=0x7fe8e40588c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e831fb82b245ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8d57fa640 1 -- 172.21.15.113:0/1558262737 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4086070 con 0x55c069641320 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8d57fa640 1 -- 172.21.15.113:0/1558262737 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fe8e400dbc0 con 0x55c069641320 2024-04-25T00:06:37.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ec02f070 con 0x55c069641320 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8d57fa640 1 -- 172.21.15.113:0/1558262737 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fe8e405d250 con 0x55c069641320 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 10 client.9156 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c06939c330 con 0x55c069641320 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8d57fa640 1 -- 172.21.15.113:0/1558262737 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec02f070 con 0x55c069641320 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8d57fa640 1 -- 172.21.15.113:0/1558262737 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fe8e40582f0 con 0x55c069641320 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 10 client.9156 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 10 client.9156 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 20 client.9156 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069611a20 0x55c069611e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 4 client.9156 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c069568e70 con 0x55c069611a20 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.544+0000 7fe8fa017640 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069611a20 0x55c069611e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8d57fa640 10 client.9156 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fa017640 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069611a20 0x55c069611e40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76d80e86ea1fc3b server_cookie=f6e1b404dc6067d2 in_seq=0 out_seq=0 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8d57fa640 1 -- 172.21.15.113:0/1558262737 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c069568e70 con 0x55c069611a20 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8d57fa640 10 client.9156 handle_command_reply: tid=0 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 client.9156 shutdown 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069641320 msgr2=0x55c069622c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069641320 0x55c069622c80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d520 tx=0x7fe8e40588c0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 shutdown_connections 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069611a20 0x55c069611e40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0696231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069641320 0x55c069622c80 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 --2- 172.21.15.113:0/1558262737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c069611060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8d57fa640 0 client.9156 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 >> 172.21.15.113:0/1558262737 conn(0x55c069623d00 msgr2=0x55c0695f45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 shutdown_connections 2024-04-25T00:06:37.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.545+0000 7fe8fef82740 1 -- 172.21.15.113:0/1558262737 wait complete. 2024-04-25T00:06:37.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.546+0000 7fe8fef82740 20 client.9156 trim_cache size 0 max 16384 2024-04-25T00:06:37.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.563+0000 7fe8fef82740 1 Processor -- start 2024-04-25T00:06:37.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.563+0000 7fe8fef82740 1 -- start start 2024-04-25T00:06:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069624440 0x55c0695f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0695ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069641320 0x55c069611060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fef82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4200f40 con 0x55c069624440 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41fffc0 con 0x55c0696260b0 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f41f5280 con 0x55c069641320 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069624440 0x55c0695f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069624440 0x55c0695f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54280/0 (socket says 172.21.15.113:54280) 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 -- 172.21.15.113:0/3636497265 learned_addr learned my addr 172.21.15.113:0/3636497265 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa818640 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069641320 0x55c069611060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8f9816640 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0695ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 -- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069641320 msgr2=0x55c069611060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069641320 0x55c069611060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 -- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 msgr2=0x55c0695ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0695ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 -- 172.21.15.113:0/3636497265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c069208610 con 0x55c069624440 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa818640 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069641320 0x55c069611060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069624440 0x55c0695f96e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0014ea0 tx=0x7fe8f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd97bed01484fb80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8d77fe640 1 -- 172.21.15.113:0/3636497265 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f001b070 con 0x55c069624440 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8d77fe640 1 -- 172.21.15.113:0/3636497265 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f0034070 con 0x55c069624440 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.564+0000 7fe8d77fe640 1 -- 172.21.15.113:0/3636497265 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f002e050 con 0x55c069624440 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 -- 172.21.15.113:0/3636497265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069624440 msgr2=0x55c0695f96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069624440 0x55c0695f96e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0014ea0 tx=0x7fe8f0017000 comp rx=0 tx=0).stop 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 -- 172.21.15.113:0/3636497265 shutdown_connections 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c0696260b0 0x55c0695ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069641320 0x55c069611060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3636497265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c069624440 0x55c0695f96e0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 -- 172.21.15.113:0/3636497265 >> 172.21.15.113:0/3636497265 conn(0x55c069623d00 msgr2=0x55c06961f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 -- 172.21.15.113:0/3636497265 shutdown_connections 2024-04-25T00:06:37.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 -- 172.21.15.113:0/3636497265 wait complete. 2024-04-25T00:06:37.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 -- start start 2024-04-25T00:06:37.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.565+0000 7fe8fef82740 1 Processor -- start 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0695f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c0696260b0 0x55c069634150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fef82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c069634690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fef82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c0695f4a20 con 0x55c0696260b0 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41ff9a0 con 0x55c069641320 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fef82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c0694ff6a0 con 0x55c069624440 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fef82740 10 client.0 _ll_register_callbacks cb 0x7fe8e40589e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8f9816640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c0696260b0 0x55c069634150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8f9816640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c0696260b0 0x55c069634150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54294/0 (socket says 172.21.15.113:54294) 2024-04-25T00:06:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8f9816640 1 -- 172.21.15.113:0/3411035 learned_addr learned my addr 172.21.15.113:0/3411035 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.566+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0695f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fa818640 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c069634690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8f9816640 1 -- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 msgr2=0x55c0695f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8f9816640 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0695f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8f9816640 1 -- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 msgr2=0x55c069634690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8f9816640 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c069634690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8f9816640 1 -- 172.21.15.113:0/3411035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c069658860 con 0x55c0696260b0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8f9816640 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c0696260b0 0x55c069634150 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000fb0 tx=0x7fe8e4070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c60885239aa0e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8d57fa640 1 -- 172.21.15.113:0/3411035 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4082020 con 0x55c0696260b0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8d57fa640 1 -- 172.21.15.113:0/3411035 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fe8e405f660 con 0x55c0696260b0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f0034070 con 0x55c0696260b0 2024-04-25T00:06:37.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8d57fa640 1 -- 172.21.15.113:0/3411035 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fe8e4086040 con 0x55c0696260b0 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 10 client.9162 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c069658860 con 0x55c0696260b0 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8d57fa640 1 -- 172.21.15.113:0/3411035 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f0034070 con 0x55c0696260b0 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8d57fa640 1 -- 172.21.15.113:0/3411035 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fe8e4054bf0 con 0x55c0696260b0 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 10 client.9162 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 10 client.9162 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 20 client.9162 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069620e10 0x55c06965f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 4 client.9162 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) -- 0x55c0695631c0 con 0x55c069620e10 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.567+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069620e10 0x55c06965f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.568+0000 7fe8d57fa640 10 client.9162 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.568+0000 7fe8fa017640 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069620e10 0x55c06965f8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c910aca5272db56d server_cookie=4855b5fec4e927dc in_seq=0 out_seq=0 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8d57fa640 1 -- 172.21.15.113:0/3411035 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c0695631c0 con 0x55c069620e10 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8d57fa640 10 client.9162 handle_command_reply: tid=0 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 client.9162 shutdown 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c0696260b0 msgr2=0x55c069634150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c0696260b0 0x55c069634150 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000fb0 tx=0x7fe8e4070000 comp rx=0 tx=0).stop 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 shutdown_connections 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c069620e10 0x55c06965f8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c069641320 0x55c069634690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c0696260b0 0x55c069634150 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 --2- 172.21.15.113:0/3411035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c069624440 0x55c0695f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 >> 172.21.15.113:0/3411035 conn(0x55c069623d00 msgr2=0x55c06963b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8d57fa640 0 client.9162 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 shutdown_connections 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.570+0000 7fe8fef82740 1 -- 172.21.15.113:0/3411035 wait complete. 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fef82740 20 client.9162 trim_cache size 0 max 16384 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "8ea40a5e-ffc7-4dbe-9ff2-e035d0184e94", 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:06:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:06:37.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d806be60 msgr2=0x7fe8d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d806be60 0x7fe8d806e320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009bc0 tx=0x7fe8f0009190 comp rx=0 tx=0).stop 2024-04-25T00:06:37.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8f41f6b50 msgr2=0x7fe8f41f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8f41f6b50 0x7fe8f41f6fd0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400f820 tx=0x7fe8e400fc20 comp rx=0 tx=0).stop 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 shutdown_connections 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d806be60 0x7fe8d806e320 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8f40ada20 0x7fe8f40adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8f41f6b50 0x7fe8f41f6fd0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 --2- 172.21.15.113:0/1131953351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e0004d40 0x7fe8f4142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 >> 172.21.15.113:0/1131953351 conn(0x7fe8f40b1b30 msgr2=0x7fe8f40cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 shutdown_connections 2024-04-25T00:06:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.571+0000 7fe8fc27b640 1 -- 172.21.15.113:0/1131953351 wait complete. 2024-04-25T00:06:37.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:37.583 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T00:06:37.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.705+0000 7efe739d5640 1 Processor -- start 2024-04-25T00:06:37.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.705+0000 7efe739d5640 1 -- start start 2024-04-25T00:06:37.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe739d5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efe6c15e430 0x7efe6c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe739d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c0c9b10 0x7efe6c15e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe739d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe6c0716e0 0x7efe6c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe739d5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c072090 con 0x7efe6c0716e0 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe739d5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c15faf0 con 0x7efe6c15e430 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe739d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c15fdd0 con 0x7efe6c0c9b10 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe71f72640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe6c0716e0 0x7efe6c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe71f72640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe6c0716e0 0x7efe6c071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54300/0 (socket says 172.21.15.113:54300) 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe71f72640 1 -- 172.21.15.113:0/3706231315 learned_addr learned my addr 172.21.15.113:0/3706231315 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe70f70640 1 --2- 172.21.15.113:0/3706231315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c0c9b10 0x7efe6c15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.706+0000 7efe71771640 1 --2- 172.21.15.113:0/3706231315 >> v2:172.21.15.178:3300/0 conn(0x7efe6c15e430 0x7efe6c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe71f72640 1 -- 172.21.15.113:0/3706231315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c0c9b10 msgr2=0x7efe6c15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe71f72640 1 --2- 172.21.15.113:0/3706231315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c0c9b10 0x7efe6c15e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe71f72640 1 -- 172.21.15.113:0/3706231315 >> v2:172.21.15.178:3300/0 conn(0x7efe6c15e430 msgr2=0x7efe6c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe71f72640 1 --2- 172.21.15.113:0/3706231315 >> v2:172.21.15.178:3300/0 conn(0x7efe6c15e430 0x7efe6c0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe71f72640 1 -- 172.21.15.113:0/3706231315 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1600b0 con 0x7efe6c0716e0 2024-04-25T00:06:37.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe71f72640 1 --2- 172.21.15.113:0/3706231315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0716e0 0x7efe6c071b50 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0b3550 tx=0x7efe6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d77b60a26c4198b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe5ffff640 1 -- 172.21.15.113:0/3706231315 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64036070 con 0x7efe6c0716e0 2024-04-25T00:06:37.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe5ffff640 1 -- 172.21.15.113:0/3706231315 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe64031600 con 0x7efe6c0716e0 2024-04-25T00:06:37.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.707+0000 7efe5ffff640 1 -- 172.21.15.113:0/3706231315 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640319d0 con 0x7efe6c0716e0 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 -- 172.21.15.113:0/3706231315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0716e0 msgr2=0x7efe6c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 --2- 172.21.15.113:0/3706231315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0716e0 0x7efe6c071b50 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0b3550 tx=0x7efe6402a6a0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 -- 172.21.15.113:0/3706231315 shutdown_connections 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 --2- 172.21.15.113:0/3706231315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0716e0 0x7efe6c071b50 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 --2- 172.21.15.113:0/3706231315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c0c9b10 0x7efe6c15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 --2- 172.21.15.113:0/3706231315 >> v2:172.21.15.178:3300/0 conn(0x7efe6c15e430 0x7efe6c0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 -- 172.21.15.113:0/3706231315 >> 172.21.15.113:0/3706231315 conn(0x7efe6c06fb60 msgr2=0x7efe6c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 -- 172.21.15.113:0/3706231315 shutdown_connections 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.708+0000 7efe739d5640 1 -- 172.21.15.113:0/3706231315 wait complete. 2024-04-25T00:06:37.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.709+0000 7efe739d5640 1 Processor -- start 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.709+0000 7efe739d5640 1 -- start start 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe739d5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe6c0716e0 0x7efe6c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe739d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0c9b10 0x7efe6c1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe739d5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c15e430 0x7efe6c144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe739d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c160950 con 0x7efe6c0c9b10 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe739d5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c160670 con 0x7efe6c0716e0 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe739d5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c06d7d0 con 0x7efe6c15e430 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0c9b10 0x7efe6c1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0c9b10 0x7efe6c1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54312/0 (socket says 172.21.15.113:54312) 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 -- 172.21.15.113:0/2801863623 learned_addr learned my addr 172.21.15.113:0/2801863623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe71f72640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c15e430 0x7efe6c144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 -- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c15e430 msgr2=0x7efe6c144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c15e430 0x7efe6c144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 -- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe6c0716e0 msgr2=0x7efe6c144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe71771640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe6c0716e0 0x7efe6c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe6c0716e0 0x7efe6c144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 -- 172.21.15.113:0/2801863623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c157ae0 con 0x7efe6c0c9b10 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe70f70640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0c9b10 0x7efe6c1575a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7efe6000d800 tx=0x7efe6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.710+0000 7efe5dffb640 1 -- 172.21.15.113:0/2801863623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6000b6c0 con 0x7efe6c0c9b10 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.711+0000 7efe5dffb640 1 -- 172.21.15.113:0/2801863623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe60019070 con 0x7efe6c0c9b10 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.711+0000 7efe5dffb640 1 -- 172.21.15.113:0/2801863623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6000ba80 con 0x7efe6c0c9b10 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.711+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1447d0 con 0x7efe6c0c9b10 2024-04-25T00:06:37.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.711+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1453c0 con 0x7efe6c0c9b10 2024-04-25T00:06:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.712+0000 7efe5dffb640 1 -- 172.21.15.113:0/2801863623 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe6000bc20 con 0x7efe6c0c9b10 2024-04-25T00:06:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.712+0000 7efe5dffb640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe5006be30 0x7efe5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.712+0000 7efe5dffb640 1 -- 172.21.15.113:0/2801863623 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7efe6008b3c0 con 0x7efe6c0c9b10 2024-04-25T00:06:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.713+0000 7efe71771640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe5006be30 0x7efe5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.713+0000 7efe71771640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe5006be30 0x7efe5006e2f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.713+0000 7efe767c4740 1 Processor -- start 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe767c4740 1 -- start start 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe767c4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3f560 0x55c6a5a14370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe767c4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5978330 con 0x55c6a5a3f560 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5920ab0 con 0x55c6a5a5cb80 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c6a59742f0 con 0x55c6a5a3fca0 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe71f72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3f560 0x55c6a5a14370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe71f72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3f560 0x55c6a5a14370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54314/0 (socket says 172.21.15.113:54314) 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe71f72640 1 -- 172.21.15.113:0/1630798151 learned_addr learned my addr 172.21.15.113:0/1630798151 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe70f70640 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.714+0000 7efe71771640 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe71f72640 1 -- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 msgr2=0x55c6a5a3f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe71f72640 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe71f72640 1 -- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 msgr2=0x55c6a5a0f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe71f72640 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe71f72640 1 -- 172.21.15.113:0/1630798151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a57567d0 con 0x55c6a5a3f560 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe71f72640 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3f560 0x55c6a5a14370 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7efe6402ac90 tx=0x7efe6402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d87cc4046ebfda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe4effd640 1 -- 172.21.15.113:0/1630798151 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64036040 con 0x55c6a5a3f560 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe4effd640 1 -- 172.21.15.113:0/1630798151 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6403c470 con 0x55c6a5a3f560 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe4effd640 1 -- 172.21.15.113:0/1630798151 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6403c840 con 0x55c6a5a3f560 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe767c4740 1 -- 172.21.15.113:0/1630798151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3f560 msgr2=0x55c6a5a14370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.715+0000 7efe767c4740 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3f560 0x55c6a5a14370 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7efe6402ac90 tx=0x7efe6402e000 comp rx=0 tx=0).stop 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 -- 172.21.15.113:0/1630798151 shutdown_connections 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3f560 0x55c6a5a14370 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 --2- 172.21.15.113:0/1630798151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 -- 172.21.15.113:0/1630798151 >> 172.21.15.113:0/1630798151 conn(0x55c6a5a11780 msgr2=0x55c6a5a5a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 -- 172.21.15.113:0/1630798151 shutdown_connections 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 -- 172.21.15.113:0/1630798151 wait complete. 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 -- start start 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.716+0000 7efe767c4740 1 Processor -- start 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe767c4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a2d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a2c890 0x55c6a5a417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe767c4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5a13930 con 0x55c6a5a5cb80 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5933f40 con 0x55c6a5a2c890 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c6a5a12d20 con 0x55c6a5a3fca0 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe767c4740 10 client.0 _ll_register_callbacks cb 0x55c6a568bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a2d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a2d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54316/0 (socket says 172.21.15.113:54316) 2024-04-25T00:06:37.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 -- 172.21.15.113:0/1595462115 learned_addr learned my addr 172.21.15.113:0/1595462115 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe71f72640 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a2c890 0x55c6a5a417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe71771640 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 -- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 msgr2=0x55c6a5a3ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 -- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a2c890 msgr2=0x55c6a5a417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a2c890 0x55c6a5a417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.717+0000 7efe70f70640 1 -- 172.21.15.113:0/1595462115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a564c0f0 con 0x55c6a5a5cb80 2024-04-25T00:06:37.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe70f70640 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a2d280 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7efe600559f0 tx=0x7efe60059e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232c8642f393bfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe47fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe47fff640 1 -- 172.21.15.113:0/1595462115 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6008b070 con 0x55c6a5a5cb80 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe47fff640 1 -- 172.21.15.113:0/1595462115 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7efe60058a30 con 0x55c6a5a5cb80 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6403c470 con 0x55c6a5a5cb80 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe47fff640 1 -- 172.21.15.113:0/1595462115 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7efe6008b210 con 0x55c6a5a5cb80 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 10 client.9174 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c6a57b6f60 con 0x55c6a5a5cb80 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe47fff640 1 -- 172.21.15.113:0/1595462115 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6403c470 con 0x55c6a5a5cb80 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe47fff640 1 -- 172.21.15.113:0/1595462115 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7efe60085020 con 0x55c6a5a5cb80 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 10 client.9174 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 10 client.9174 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 20 client.9174 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:37.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5a5b670 0x55c6a5a77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 4 client.9174 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6a5985870 con 0x55c6a5a5b670 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.718+0000 7efe71771640 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5a5b670 0x55c6a5a77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe47fff640 10 client.9174 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe71771640 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5a5b670 0x55c6a5a77730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc0396142ef93ee server_cookie=5f42e8f2468cd5e9 in_seq=0 out_seq=0 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe47fff640 1 -- 172.21.15.113:0/1595462115 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6a5985870 con 0x55c6a5a5b670 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe47fff640 10 client.9174 handle_command_reply: tid=0 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 client.9174 shutdown 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a5cb80 msgr2=0x55c6a5a2d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a2d280 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7efe600559f0 tx=0x7efe60059e20 comp rx=0 tx=0).stop 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 shutdown_connections 2024-04-25T00:06:37.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5a5b670 0x55c6a5a77730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a2c890 0x55c6a5a417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe47fff640 0 client.9174 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a2d280 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 --2- 172.21.15.113:0/1595462115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a3ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 >> 172.21.15.113:0/1595462115 conn(0x55c6a5a3f560 msgr2=0x55c6a5a0eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 shutdown_connections 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.719+0000 7efe767c4740 1 -- 172.21.15.113:0/1595462115 wait complete. 2024-04-25T00:06:37.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.720+0000 7efe767c4740 20 client.9174 trim_cache size 0 max 16384 2024-04-25T00:06:37.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 Processor -- start 2024-04-25T00:06:37.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 -- start start 2024-04-25T00:06:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a2c890 0x55c6a5a5d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a5d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a7c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c2007b0 con 0x55c6a5a2c890 2024-04-25T00:06:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1ff900 con 0x55c6a5a5cb80 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c146e20 con 0x55c6a5a3fca0 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe71771640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a2c890 0x55c6a5a5d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe71771640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a2c890 0x55c6a5a5d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54328/0 (socket says 172.21.15.113:54328) 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe71771640 1 -- 172.21.15.113:0/306844518 learned_addr learned my addr 172.21.15.113:0/306844518 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe71f72640 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a7c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.737+0000 7efe70f70640 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a5d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe71771640 1 -- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 msgr2=0x55c6a5a5d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe71771640 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a5d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe71771640 1 -- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 msgr2=0x55c6a5a7c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe71771640 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a7c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe71771640 1 -- 172.21.15.113:0/306844518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a56251b0 con 0x55c6a5a2c890 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe71f72640 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a7c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe71771640 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a2c890 0x55c6a5a5d440 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7efe680155e0 tx=0x7efe68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5400f3efc51bb024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe4e7fc640 1 -- 172.21.15.113:0/306844518 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6800f040 con 0x55c6a5a2c890 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe4e7fc640 1 -- 172.21.15.113:0/306844518 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6802f070 con 0x55c6a5a2c890 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe4e7fc640 1 -- 172.21.15.113:0/306844518 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68015ce0 con 0x55c6a5a2c890 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe767c4740 1 -- 172.21.15.113:0/306844518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a2c890 msgr2=0x55c6a5a5d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe767c4740 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a2c890 0x55c6a5a5d440 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7efe680155e0 tx=0x7efe68014000 comp rx=0 tx=0).stop 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe767c4740 1 -- 172.21.15.113:0/306844518 shutdown_connections 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe767c4740 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a7c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe767c4740 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a3fca0 0x55c6a5a5d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe767c4740 1 --2- 172.21.15.113:0/306844518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a2c890 0x55c6a5a5d440 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.738+0000 7efe767c4740 1 -- 172.21.15.113:0/306844518 >> 172.21.15.113:0/306844518 conn(0x55c6a5a3f560 msgr2=0x55c6a5a16690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.739+0000 7efe767c4740 1 -- 172.21.15.113:0/306844518 shutdown_connections 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.739+0000 7efe767c4740 1 -- 172.21.15.113:0/306844518 wait complete. 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.739+0000 7efe767c4740 1 -- start start 2024-04-25T00:06:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.739+0000 7efe767c4740 1 Processor -- start 2024-04-25T00:06:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a2c890 0x55c6a5adc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe767c4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3fca0 0x55c6a5a0e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe767c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe767c4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c147470 con 0x55c6a5a3fca0 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5a51680 con 0x55c6a5a5cb80 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe767c4740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c6a5a0f040 con 0x55c6a5a2c890 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe767c4740 10 client.0 _ll_register_callbacks cb 0x7efe60006d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3fca0 0x55c6a5a0e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3fca0 0x55c6a5a0e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54342/0 (socket says 172.21.15.113:54342) 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 -- 172.21.15.113:0/3234308911 learned_addr learned my addr 172.21.15.113:0/3234308911 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe71771640 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a2c890 0x55c6a5adc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe71f72640 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 -- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a2c890 msgr2=0x55c6a5adc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a2c890 0x55c6a5adc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 -- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 msgr2=0x55c6a5a0e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.740+0000 7efe70f70640 1 -- 172.21.15.113:0/3234308911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a5adbaf0 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe70f70640 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3fca0 0x55c6a5a0e2b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7efe6000b4b0 tx=0x7efe600897c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419a76128112e2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe47fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe47fff640 1 -- 172.21.15.113:0/3234308911 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6008b210 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe47fff640 1 -- 172.21.15.113:0/3234308911 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7efe6006db00 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6802f070 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe47fff640 1 -- 172.21.15.113:0/3234308911 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7efe60058c40 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 10 client.9180 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c6a5a69750 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe47fff640 1 -- 172.21.15.113:0/3234308911 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6802f070 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe47fff640 1 -- 172.21.15.113:0/3234308911 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7efe60085030 con 0x55c6a5a3fca0 2024-04-25T00:06:37.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 10 client.9180 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 10 client.9180 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 20 client.9180 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5ac19d0 0x55c6a5a5d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 4 client.9180 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55c6a597fbc0 con 0x55c6a5ac19d0 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe71771640 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5ac19d0 0x55c6a5a5d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe71771640 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5ac19d0 0x55c6a5a5d440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e06804181850512 server_cookie=f2fe1e3d46f201aa in_seq=0 out_seq=0 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.741+0000 7efe47fff640 10 client.9180 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe47fff640 1 -- 172.21.15.113:0/3234308911 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c6a597fbc0 con 0x55c6a5ac19d0 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe47fff640 10 client.9180 handle_command_reply: tid=0 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 client.9180 shutdown 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3fca0 msgr2=0x55c6a5a0e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3fca0 0x55c6a5a0e2b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7efe6000b4b0 tx=0x7efe600897c0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 shutdown_connections 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55c6a5ac19d0 0x55c6a5a5d440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6a5a5cb80 0x55c6a5a0e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6a5a3fca0 0x55c6a5a0e2b0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe47fff640 0 client.9180 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 --2- 172.21.15.113:0/3234308911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6a5a2c890 0x55c6a5adc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 >> 172.21.15.113:0/3234308911 conn(0x55c6a5a3f560 msgr2=0x55c6a5a16540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 shutdown_connections 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 1 -- 172.21.15.113:0/3234308911 wait complete. 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe767c4740 20 client.9180 trim_cache size 0 max 16384 2024-04-25T00:06:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:06:37.750 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0 2024-04-25T00:06:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:06:37.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe5006be30 msgr2=0x7efe5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe739d5640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe5006be30 0x7efe5006e2f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68008040 comp rx=0 tx=0).stop 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0c9b10 msgr2=0x7efe6c1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.742+0000 7efe739d5640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0c9b10 0x7efe6c1575a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7efe6000d800 tx=0x7efe6000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 shutdown_connections 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efe5006be30 0x7efe5006e2f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe6c15e430 0x7efe6c144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe6c0c9b10 0x7efe6c1575a0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 --2- 172.21.15.113:0/2801863623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe6c0716e0 0x7efe6c144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 >> 172.21.15.113:0/2801863623 conn(0x7efe6c06fb60 msgr2=0x7efe6c0b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 shutdown_connections 2024-04-25T00:06:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.743+0000 7efe739d5640 1 -- 172.21.15.113:0/2801863623 wait complete. 2024-04-25T00:06:37.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:06:37.754 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T00:06:37.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.852+0000 7f95a03a2640 1 Processor -- start 2024-04-25T00:06:37.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.852+0000 7f95a03a2640 1 -- start start 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f95a03a2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95980d06c0 0x7f959815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f95a03a2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95981604d0 0x7f95980cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f95a03a2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959815d6b0 con 0x7f959813f050 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f95a03a2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f959815c6a0 con 0x7f95981604d0 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f95a03a2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f959815c8f0 con 0x7f95980d06c0 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959e93f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95981604d0 0x7f95980cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959e13e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f959813f050 0x7f95980cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57946/0 (socket says 172.21.15.113:57946) 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959e13e640 1 -- 172.21.15.113:0/4123967816 learned_addr learned my addr 172.21.15.113:0/4123967816 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:37.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959d93d640 1 --2- 172.21.15.113:0/4123967816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95980d06c0 0x7f959815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959e93f640 1 -- 172.21.15.113:0/4123967816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95980d06c0 msgr2=0x7f959815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959e93f640 1 --2- 172.21.15.113:0/4123967816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95980d06c0 0x7f959815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959e93f640 1 -- 172.21.15.113:0/4123967816 >> v1:172.21.15.113:6789/0 conn(0x7f959813f050 legacy=0x7f95980cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959d13c640 1 -- 172.21.15.113:0/4123967816 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3816693001 0 0) 0x7f959815d6b0 con 0x7f959813f050 2024-04-25T00:06:37.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.853+0000 7f959e93f640 1 -- 172.21.15.113:0/4123967816 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959815cbd0 con 0x7f95981604d0 2024-04-25T00:06:37.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.854+0000 7f959e93f640 1 --2- 172.21.15.113:0/4123967816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95981604d0 0x7f95980cea60 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f95900097a0 tx=0x7f959002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1826fb26eeb617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.854+0000 7f959d13c640 1 -- 172.21.15.113:0/4123967816 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590037070 con 0x7f95981604d0 2024-04-25T00:06:37.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.854+0000 7f959d13c640 1 -- 172.21.15.113:0/4123967816 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959002ae50 con 0x7f95981604d0 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.854+0000 7f959d13c640 1 -- 172.21.15.113:0/4123967816 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590032660 con 0x7f95981604d0 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 -- 172.21.15.113:0/4123967816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95981604d0 msgr2=0x7f95980cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4123967816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95981604d0 0x7f95980cea60 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f95900097a0 tx=0x7f959002a100 comp rx=0 tx=0).stop 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 -- 172.21.15.113:0/4123967816 shutdown_connections 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4123967816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95981604d0 0x7f95980cea60 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4123967816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95980d06c0 0x7f959815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 -- 172.21.15.113:0/4123967816 >> 172.21.15.113:0/4123967816 conn(0x7f959806b0f0 msgr2=0x7f959806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 -- 172.21.15.113:0/4123967816 shutdown_connections 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.855+0000 7f95a03a2640 1 -- 172.21.15.113:0/4123967816 wait complete. 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.856+0000 7f95a03a2640 1 Processor -- start 2024-04-25T00:06:37.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.856+0000 7f95a03a2640 1 -- start start 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f95a03a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95980d06c0 0x7f959814a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f95a03a2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f959813f050 0x7f9598133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f95a03a2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981604d0 0x7f959815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f95a03a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f959813f430 con 0x7f95980d06c0 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f95a03a2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f959815d190 con 0x7f95981604d0 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f95a03a2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f95981608d0 con 0x7f959813f050 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95980d06c0 0x7f959814a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95980d06c0 0x7f959814a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54354/0 (socket says 172.21.15.113:54354) 2024-04-25T00:06:37.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 -- 172.21.15.113:0/4098934867 learned_addr learned my addr 172.21.15.113:0/4098934867 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959d93d640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f959813f050 0x7f9598133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e93f640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981604d0 0x7f959815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 -- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f959813f050 msgr2=0x7f9598133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f959813f050 0x7f9598133540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 -- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981604d0 msgr2=0x7f959815b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981604d0 0x7f959815b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 -- 172.21.15.113:0/4098934867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959814b010 con 0x7f95980d06c0 2024-04-25T00:06:37.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.857+0000 7f959e13e640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95980d06c0 0x7f959814a5a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f959400c890 tx=0x7f959400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.859+0000 7f958effd640 1 -- 172.21.15.113:0/4098934867 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594018070 con 0x7f95980d06c0 2024-04-25T00:06:37.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.859+0000 7f958effd640 1 -- 172.21.15.113:0/4098934867 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959400a640 con 0x7f95980d06c0 2024-04-25T00:06:37.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.859+0000 7f958effd640 1 -- 172.21.15.113:0/4098934867 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959400aa30 con 0x7f95980d06c0 2024-04-25T00:06:37.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.859+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959814b2f0 con 0x7f95980d06c0 2024-04-25T00:06:37.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.859+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959814b8b0 con 0x7f95980d06c0 2024-04-25T00:06:37.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.861+0000 7f958effd640 1 -- 172.21.15.113:0/4098934867 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f959401b070 con 0x7f95980d06c0 2024-04-25T00:06:37.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.861+0000 7f958effd640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f957c06be30 0x7f957c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.861+0000 7f958effd640 1 -- 172.21.15.113:0/4098934867 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f959408a580 con 0x7f95980d06c0 2024-04-25T00:06:37.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.861+0000 7f959d93d640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f957c06be30 0x7f957c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.862+0000 7f959d93d640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f957c06be30 0x7f957c06e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f959815f8b0 tx=0x7f9588008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.862+0000 7f95a3194740 1 Processor -- start 2024-04-25T00:06:37.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.863+0000 7f95a3194740 1 -- start start 2024-04-25T00:06:37.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.863+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e985140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.863+0000 7f95a3194740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.863+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e95aa40 0x55785e9a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.863+0000 7f95a3194740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55785e8be6b0 con 0x55785e9a2600 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.863+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55785e866880 con 0x55785e984d60 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.863+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55785e8ba670 con 0x55785e95aa40 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54368/0 (socket says 172.21.15.113:54368) 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 -- 172.21.15.113:0/2755057941 learned_addr learned my addr 172.21.15.113:0/2755057941 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959e93f640 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e95aa40 0x55785e9a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959e13e640 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e985140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 -- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e95aa40 msgr2=0x55785e9a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e95aa40 0x55785e9a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 -- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 msgr2=0x55785e985140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e985140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 -- 172.21.15.113:0/2755057941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785e69c7d0 con 0x55785e9a2600 2024-04-25T00:06:37.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f959d93d640 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9a2b50 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9588010aa0 tx=0x7f9588013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a9d44232b8806c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f957bfff640 1 -- 172.21.15.113:0/2755057941 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958801c050 con 0x55785e9a2600 2024-04-25T00:06:37.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f957bfff640 1 -- 172.21.15.113:0/2755057941 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9588026070 con 0x55785e9a2600 2024-04-25T00:06:37.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.864+0000 7f957bfff640 1 -- 172.21.15.113:0/2755057941 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588016e30 con 0x55785e9a2600 2024-04-25T00:06:37.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 -- 172.21.15.113:0/2755057941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 msgr2=0x55785e9a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9a2b50 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9588010aa0 tx=0x7f9588013000 comp rx=0 tx=0).stop 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 -- 172.21.15.113:0/2755057941 shutdown_connections 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e95aa40 0x55785e9a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9a2b50 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 --2- 172.21.15.113:0/2755057941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e985140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 -- 172.21.15.113:0/2755057941 >> 172.21.15.113:0/2755057941 conn(0x55785e957600 msgr2=0x55785e99fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 -- 172.21.15.113:0/2755057941 shutdown_connections 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 -- 172.21.15.113:0/2755057941 wait complete. 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.865+0000 7f95a3194740 1 -- start start 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.866+0000 7f95a3194740 1 Processor -- start 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.866+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e984d60 0x55785e99eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.866+0000 7f95a3194740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.866+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e9b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.866+0000 7f95a3194740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55785e95b7e0 con 0x55785e9a2600 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.866+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55785e879f50 con 0x55785e988c80 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.866+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55785e955530 con 0x55785e984d60 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95a3194740 10 client.0 _ll_register_callbacks cb 0x55785e8b5000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959e13e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e984d60 0x55785e99eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54382/0 (socket says 172.21.15.113:54382) 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 -- 172.21.15.113:0/4002117743 learned_addr learned my addr 172.21.15.113:0/4002117743 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959e93f640 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e9b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 -- 172.21.15.113:0/4002117743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e984d60 msgr2=0x55785e99eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e984d60 0x55785e99eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 -- 172.21.15.113:0/4002117743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 msgr2=0x55785e9b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e9b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 -- 172.21.15.113:0/4002117743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785e5d23f0 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f959d93d640 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9b42f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f9588023040 tx=0x7f9588004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a604112c12099bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95797fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95797fa640 1 -- 172.21.15.113:0/4002117743 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588006040 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95797fa640 1 -- 172.21.15.113:0/4002117743 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f95880061e0 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9588026070 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95797fa640 1 -- 172.21.15.113:0/4002117743 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f958801c020 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95a3194740 10 client.9195 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55785e5920f0 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.867+0000 7f95797fa640 1 -- 172.21.15.113:0/4002117743 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9588026070 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95797fa640 1 -- 172.21.15.113:0/4002117743 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f9588006420 con 0x55785e9a2600 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95a3194740 10 client.9195 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95a3194740 10 client.9195 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95a3194740 20 client.9195 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:37.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95a3194740 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9a11b0 0x55785e986cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95a3194740 4 client.9195 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55785e8cbbf0 con 0x55785e9a11b0 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f959e13e640 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9a11b0 0x55785e986cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f95797fa640 10 client.9195 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.868+0000 7f959e13e640 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9a11b0 0x55785e986cf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167cda1a63b2f3e1 server_cookie=ef3cf0e7d6e82299 in_seq=0 out_seq=0 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95797fa640 1 -- 172.21.15.113:0/4002117743 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55785e8cbbf0 con 0x55785e9a11b0 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95797fa640 10 client.9195 handle_command_reply: tid=0 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 client.9195 shutdown 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 msgr2=0x55785e9b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9b42f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f9588023040 tx=0x7f9588004730 comp rx=0 tx=0).stop 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 shutdown_connections 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9a11b0 0x55785e986cf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e9b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9b42f0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 --2- 172.21.15.113:0/4002117743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e984d60 0x55785e99eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95797fa640 0 client.9195 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 >> 172.21.15.113:0/4002117743 conn(0x55785e95aa40 msgr2=0x55785e9a30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 shutdown_connections 2024-04-25T00:06:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 1 -- 172.21.15.113:0/4002117743 wait complete. 2024-04-25T00:06:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.869+0000 7f95a3194740 20 client.9195 trim_cache size 0 max 16384 2024-04-25T00:06:37.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.886+0000 7f95a3194740 1 Processor -- start 2024-04-25T00:06:37.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.886+0000 7f95a3194740 1 -- start start 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e99a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e988c80 0x55785e995ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f95a3194740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f95a3194740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9598200680 con 0x55785e9a2600 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9598155e40 con 0x55785e984d60 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9598071c00 con 0x55785e988c80 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54396/0 (socket says 172.21.15.113:54396) 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 -- 172.21.15.113:0/1373429388 learned_addr learned my addr 172.21.15.113:0/1373429388 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 -- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e988c80 msgr2=0x55785e995ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e13e640 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e99a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959d93d640 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e988c80 0x55785e995ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e988c80 0x55785e995ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 -- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 msgr2=0x55785e99a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e99a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 -- 172.21.15.113:0/1373429388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785e985630 con 0x55785e9a2600 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f959e93f640 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9c3fe0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f95900098c0 tx=0x7f959002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b56f379c5bae17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f957b7fe640 1 -- 172.21.15.113:0/1373429388 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590037040 con 0x55785e9a2600 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f957b7fe640 1 -- 172.21.15.113:0/1373429388 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95900328a0 con 0x55785e9a2600 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.887+0000 7f957b7fe640 1 -- 172.21.15.113:0/1373429388 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590032c50 con 0x55785e9a2600 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 -- 172.21.15.113:0/1373429388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 msgr2=0x55785e9c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9c3fe0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f95900098c0 tx=0x7f959002c000 comp rx=0 tx=0).stop 2024-04-25T00:06:37.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 -- 172.21.15.113:0/1373429388 shutdown_connections 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e9a2600 0x55785e9c3fe0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e988c80 0x55785e995ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 --2- 172.21.15.113:0/1373429388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e984d60 0x55785e99a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 -- 172.21.15.113:0/1373429388 >> 172.21.15.113:0/1373429388 conn(0x55785e95aa40 msgr2=0x55785e99e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 -- 172.21.15.113:0/1373429388 shutdown_connections 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.888+0000 7f95a3194740 1 -- 172.21.15.113:0/1373429388 wait complete. 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.889+0000 7f95a3194740 1 -- start start 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.889+0000 7f95a3194740 1 Processor -- start 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e984d60 0x55785e997a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e997f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e9a2600 0x55785e9fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f95981557f0 con 0x55785e984d60 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55785e9b1030 con 0x55785e988c80 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55785e99f5f0 con 0x55785e9a2600 2024-04-25T00:06:37.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 10 client.0 _ll_register_callbacks cb 0x7f95880071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e984d60 0x55785e997a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e984d60 0x55785e997a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54408/0 (socket says 172.21.15.113:54408) 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 -- 172.21.15.113:0/1547598390 learned_addr learned my addr 172.21.15.113:0/1547598390 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959d93d640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e997f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e93f640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e9a2600 0x55785e9fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 -- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e9a2600 msgr2=0x55785e9fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e9a2600 0x55785e9fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 -- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 msgr2=0x55785e997f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e997f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 -- 172.21.15.113:0/1547598390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785ea18f90 con 0x55785e984d60 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e93f640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e9a2600 0x55785e9fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959d93d640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e997f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:37.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f959e13e640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e984d60 0x55785e997a30 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f959400a430 tx=0x7f9594009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef6796eb3656619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95797fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95797fa640 1 -- 172.21.15.113:0/1547598390 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594088060 con 0x55785e984d60 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95797fa640 1 -- 172.21.15.113:0/1547598390 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f9594060f80 con 0x55785e984d60 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95900328a0 con 0x55785e984d60 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95797fa640 1 -- 172.21.15.113:0/1547598390 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f9594061180 con 0x55785e984d60 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 10 client.9201 fetch_fsmap learned FSMap version 68 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55785e9fba20 con 0x55785e984d60 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.890+0000 7f95797fa640 1 -- 172.21.15.113:0/1547598390 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95900328a0 con 0x55785e984d60 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95797fa640 1 -- 172.21.15.113:0/1547598390 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f9594077bd0 con 0x55785e984d60 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 10 client.9201 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 10 client.9201 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 20 client.9201 populate_metadata read hostname 'smithi113' 2024-04-25T00:06:37.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9feac0 0x55785e99cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 4 client.9201 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 --> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55785e997320 con 0x55785e9feac0 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f959d93d640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9feac0 0x55785e99cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95797fa640 10 client.9201 ms_handle_connect on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f959d93d640 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9feac0 0x55785e99cec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252b5b3b6e387b20 server_cookie=bb134c2f73932150 in_seq=0 out_seq=0 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95797fa640 1 -- 172.21.15.113:0/1547598390 <== mds.0 v2:172.21.15.113:6834/2664195523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+276 (crc 0 0 0) 0x55785e997320 con 0x55785e9feac0 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95797fa640 10 client.9201 handle_command_reply: tid=0 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 client.9201 shutdown 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e984d60 msgr2=0x55785e997a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e984d60 0x55785e997a30 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f959400a430 tx=0x7f9594009040 comp rx=0 tx=0).stop 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 shutdown_connections 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:6834/2664195523,v1:172.21.15.113:6836/2664195523] conn(0x55785e9feac0 0x55785e99cec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55785e988c80 0x55785e997f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55785e9a2600 0x55785e9fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95797fa640 0 client.9201 ms_handle_reset on v2:172.21.15.113:6834/2664195523 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 --2- 172.21.15.113:0/1547598390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55785e984d60 0x55785e997a30 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.891+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 >> 172.21.15.113:0/1547598390 conn(0x55785e95aa40 msgr2=0x55785e95ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 shutdown_connections 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a3194740 1 -- 172.21.15.113:0/1547598390 wait complete. 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a3194740 20 client.9201 trim_cache size 0 max 16384 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T00:06:37.903 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:06:37.904 INFO:teuthology.orchestra.run.smithi113.stdout: "8ea40a5e-ffc7-4dbe-9ff2-e035d0184e94": { 2024-04-25T00:06:37.904 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/scrub_control_test_path", 2024-04-25T00:06:37.904 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "8ea40a5e-ffc7-4dbe-9ff2-e035d0184e94", 2024-04-25T00:06:37.904 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive" 2024-04-25T00:06:37.904 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:37.904 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:06:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:06:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f957c06be30 msgr2=0x7f957c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f957c06be30 0x7f957c06e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f959815f8b0 tx=0x7f9588008040 comp rx=0 tx=0).stop 2024-04-25T00:06:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95980d06c0 msgr2=0x7f959814a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:37.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95980d06c0 0x7f959814a5a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f959400c890 tx=0x7f959400cd50 comp rx=0 tx=0).stop 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 shutdown_connections 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f957c06be30 0x7f957c06e2f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95981604d0 0x7f959815b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f959813f050 0x7f9598133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 --2- 172.21.15.113:0/4098934867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95980d06c0 0x7f959814a5a0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 >> 172.21.15.113:0/4098934867 conn(0x7f959806b0f0 msgr2=0x7f959813c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 shutdown_connections 2024-04-25T00:06:37.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:37.892+0000 7f95a03a2640 1 -- 172.21.15.113:0/4098934867 wait complete. 2024-04-25T00:06:37.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:38.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.002+0000 7f568e8f5640 1 Processor -- start 2024-04-25T00:06:38.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.002+0000 7f568e8f5640 1 -- start start 2024-04-25T00:06:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f568e8f5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5688144910 0x7f56880b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f568e8f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 0x7f568806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f568e8f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5688144cf0 con 0x7f56880b5680 2024-04-25T00:06:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f568e8f5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f56880b4510 con 0x7f5688144910 2024-04-25T00:06:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f568e8f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56880b2f30 con 0x7f56880b5240 2024-04-25T00:06:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f568ce92640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f56880b5680 0x7f56880cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57958/0 (socket says 172.21.15.113:57958) 2024-04-25T00:06:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f568ce92640 1 -- 172.21.15.113:0/3953035250 learned_addr learned my addr 172.21.15.113:0/3953035250 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5687fff640 1 --2- 172.21.15.113:0/3953035250 >> v2:172.21.15.178:3300/0 conn(0x7f5688144910 0x7f56880b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f56877fe640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 0x7f568806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751211371 0 0) 0x7f5688144cf0 con 0x7f56880b5680 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5670003520 con 0x7f56880b5680 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3032647647 0 0) 0x7f5670003520 con 0x7f56880b5680 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 msgr2=0x7f568806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 0x7f568806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 >> v2:172.21.15.178:3300/0 conn(0x7f5688144910 msgr2=0x7f56880b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 --2- 172.21.15.113:0/3953035250 >> v2:172.21.15.178:3300/0 conn(0x7f5688144910 0x7f56880b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56880b3210 con 0x7f56880b5680 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.003+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f56800019f0 con 0x7f56880b5680 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 >> v1:172.21.15.113:6789/0 conn(0x7f56880b5680 legacy=0x7f56880cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5686ffd640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 0x7f56700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5686ffd640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5670003f30 0x7f5670004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5686ffd640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670004d40 0x7f56700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56880b4510 con 0x7f5670003f30 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56880b2f30 con 0x7f5670004d40 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f567000b9c0 con 0x7f56880b5240 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f56877fe640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5670003f30 0x7f5670004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f5687fff640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 0x7f56700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f568ce92640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670004d40 0x7f56700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f56877fe640 1 -- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 msgr2=0x7f56700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f56877fe640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56880b5240 0x7f56700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f56877fe640 1 -- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670004d40 msgr2=0x7f56700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f56877fe640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670004d40 0x7f56700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.004+0000 7f56877fe640 1 -- 172.21.15.113:0/3953035250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5670003520 con 0x7f5670003f30 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f56877fe640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5670003f30 0x7f5670004800 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f567c009ac0 tx=0x7f5670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e93acf3a85a228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c036070 con 0x7f5670003f30 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c031400 con 0x7f5670003f30 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f5686ffd640 1 -- 172.21.15.113:0/3953035250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c009690 con 0x7f5670003f30 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f568e8f5640 1 -- 172.21.15.113:0/3953035250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5670003f30 msgr2=0x7f5670004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f568e8f5640 1 --2- 172.21.15.113:0/3953035250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5670003f30 0x7f5670004800 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f567c009ac0 tx=0x7f5670005830 comp rx=0 tx=0).stop 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f568ce92640 1 -- 172.21.15.113:0/3953035250 reap_dead start 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f568e8f5640 1 -- 172.21.15.113:0/3953035250 shutdown_connections 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f568e8f5640 1 -- 172.21.15.113:0/3953035250 >> 172.21.15.113:0/3953035250 conn(0x7f56880b1a50 msgr2=0x7f56880b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f568e8f5640 1 -- 172.21.15.113:0/3953035250 shutdown_connections 2024-04-25T00:06:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.005+0000 7f568e8f5640 1 -- 172.21.15.113:0/3953035250 wait complete. 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.006+0000 7f568e8f5640 1 Processor -- start 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568e8f5640 1 -- start start 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568e8f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670003f30 0x7f568813b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568e8f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568813bc10 0x7f568813c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568e8f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568814e670 0x7f568814eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568e8f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56880b4da0 con 0x7f568813bc10 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568e8f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56880b37d0 con 0x7f5670003f30 2024-04-25T00:06:38.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568e8f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56880b4f20 con 0x7f568814e670 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f56877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568813bc10 0x7f568813c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f56877fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568813bc10 0x7f568813c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54424/0 (socket says 172.21.15.113:54424) 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f56877fe640 1 -- 172.21.15.113:0/3034212243 learned_addr learned my addr 172.21.15.113:0/3034212243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.007+0000 7f568ce92640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568814e670 0x7f568814eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f5687fff640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670003f30 0x7f568813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f56877fe640 1 -- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568814e670 msgr2=0x7f568814eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f56877fe640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568814e670 0x7f568814eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f56877fe640 1 -- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670003f30 msgr2=0x7f568813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f56877fe640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670003f30 0x7f568813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f56877fe640 1 -- 172.21.15.113:0/3034212243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568814f030 con 0x7f568813bc10 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f568ce92640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568814e670 0x7f568814eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f5687fff640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670003f30 0x7f568813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f56877fe640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568813bc10 0x7f568813c070 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f567c000c00 tx=0x7f567c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f5684ff9640 1 -- 172.21.15.113:0/3034212243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c040020 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f5684ff9640 1 -- 172.21.15.113:0/3034212243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c003c60 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56880ce2b0 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f5684ff9640 1 -- 172.21.15.113:0/3034212243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c036040 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.008+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56880ce870 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.009+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568806eb90 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.009+0000 7f5684ff9640 1 -- 172.21.15.113:0/3034212243 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f567c043020 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.010+0000 7f5684ff9640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567406be00 0x7f567406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.010+0000 7f5684ff9640 1 -- 172.21.15.113:0/3034212243 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f567c0ab010 con 0x7f568813bc10 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.010+0000 7f5687fff640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567406be00 0x7f567406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.010+0000 7f5687fff640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567406be00 0x7f567406e2c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5678009c40 tx=0x7f5678008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.014+0000 7f5684ff9640 1 -- 172.21.15.113:0/3034212243 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f568806eb90 con 0x7f568813bc10 2024-04-25T00:06:38.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.223+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56880691a0 con 0x7f568813bc10 2024-04-25T00:06:38.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.223+0000 7f5684ff9640 1 -- 172.21.15.113:0/3034212243 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7f567c04a070 con 0x7f568813bc10 2024-04-25T00:06:38.224 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:38.224 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:38.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567406be00 msgr2=0x7f567406e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567406be00 0x7f567406e2c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5678009c40 tx=0x7f5678008040 comp rx=0 tx=0).stop 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568813bc10 msgr2=0x7f568813c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568813bc10 0x7f568813c070 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f567c000c00 tx=0x7f567c004730 comp rx=0 tx=0).stop 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 shutdown_connections 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f567406be00 0x7f567406e2c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568814e670 0x7f568814eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f568813bc10 0x7f568813c070 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 --2- 172.21.15.113:0/3034212243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5670003f30 0x7f568813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 >> 172.21.15.113:0/3034212243 conn(0x7f56880b1a50 msgr2=0x7f56880716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 shutdown_connections 2024-04-25T00:06:38.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.225+0000 7f568e8f5640 1 -- 172.21.15.113:0/3034212243 wait complete. 2024-04-25T00:06:38.237 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T00:06:38.237 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:06:38.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.331+0000 7f0388f89640 1 Processor -- start 2024-04-25T00:06:38.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.331+0000 7f0388f89640 1 -- start start 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.332+0000 7f0388f89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f038415e320 0x7f03840d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.332+0000 7f0388f89640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f03840d0b00 0x7f03840d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.332+0000 7f0388f89640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03840d3de0 con 0x7f03840d2290 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.332+0000 7f0388f89640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f03840d14b0 con 0x7f038415e320 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.332+0000 7f0388f89640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f03840d3410 con 0x7f03840d0b00 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f038259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f038415e320 0x7f03840d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f0381d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f03840d2290 0x7f03840b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57966/0 (socket says 172.21.15.113:57966) 2024-04-25T00:06:38.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f0381d9b640 1 -- 172.21.15.113:0/2190320944 learned_addr learned my addr 172.21.15.113:0/2190320944 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f0382d9d640 1 --2- 172.21.15.113:0/2190320944 >> v2:172.21.15.178:3300/0 conn(0x7f03840d0b00 0x7f03840d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f038259c640 1 -- 172.21.15.113:0/2190320944 >> v2:172.21.15.178:3300/0 conn(0x7f03840d0b00 msgr2=0x7f03840d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f038259c640 1 --2- 172.21.15.113:0/2190320944 >> v2:172.21.15.178:3300/0 conn(0x7f03840d0b00 0x7f03840d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f038259c640 1 -- 172.21.15.113:0/2190320944 >> v1:172.21.15.113:6789/0 conn(0x7f03840d2290 legacy=0x7f03840b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f038259c640 1 -- 172.21.15.113:0/2190320944 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03840d2800 con 0x7f038415e320 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.333+0000 7f038159a640 1 -- 172.21.15.113:0/2190320944 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4050216621 0 0) 0x7f03840d3de0 con 0x7f03840d2290 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f038259c640 1 --2- 172.21.15.113:0/2190320944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f038415e320 0x7f03840d38a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f03840d2030 tx=0x7f037c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f805d2e0acb85f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f038159a640 1 -- 172.21.15.113:0/2190320944 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c037070 con 0x7f038415e320 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f038159a640 1 -- 172.21.15.113:0/2190320944 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037c032440 con 0x7f038415e320 2024-04-25T00:06:38.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f038159a640 1 -- 172.21.15.113:0/2190320944 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c032810 con 0x7f038415e320 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 -- 172.21.15.113:0/2190320944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f038415e320 msgr2=0x7f03840d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 --2- 172.21.15.113:0/2190320944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f038415e320 0x7f03840d38a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f03840d2030 tx=0x7f037c02a230 comp rx=0 tx=0).stop 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 -- 172.21.15.113:0/2190320944 shutdown_connections 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 --2- 172.21.15.113:0/2190320944 >> v2:172.21.15.178:3300/0 conn(0x7f03840d0b00 0x7f03840d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 --2- 172.21.15.113:0/2190320944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f038415e320 0x7f03840d38a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 -- 172.21.15.113:0/2190320944 >> 172.21.15.113:0/2190320944 conn(0x7f03840b5c20 msgr2=0x7f03840b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 -- 172.21.15.113:0/2190320944 shutdown_connections 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.334+0000 7f0388f89640 1 -- 172.21.15.113:0/2190320944 wait complete. 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 Processor -- start 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 -- start start 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03840d0b00 0x7f0384161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03840d2290 0x7f03840ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f038415e320 0x7f03841f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f03840d30a0 con 0x7f03840d0b00 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03840d2dc0 con 0x7f03840d2290 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f0388f89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f038415e700 con 0x7f038415e320 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.336+0000 7f038259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03840d0b00 0x7f0384161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03840d0b00 0x7f0384161600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54444/0 (socket says 172.21.15.113:54444) 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 -- 172.21.15.113:0/146066258 learned_addr learned my addr 172.21.15.113:0/146066258 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0381d9b640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03840d2290 0x7f03840ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0382d9d640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f038415e320 0x7f03841f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 -- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f038415e320 msgr2=0x7f03841f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f038415e320 0x7f03841f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 -- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03840d2290 msgr2=0x7f03840ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03840d2290 0x7f03840ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 -- 172.21.15.113:0/146066258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03841f4cb0 con 0x7f03840d0b00 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0382d9d640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f038415e320 0x7f03841f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0381d9b640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03840d2290 0x7f03840ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f038259c640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03840d0b00 0x7f0384161600 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f037c000c00 tx=0x7f037c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0372ffd640 1 -- 172.21.15.113:0/146066258 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c037040 con 0x7f03840d0b00 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0372ffd640 1 -- 172.21.15.113:0/146066258 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037c042070 con 0x7f03840d0b00 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0384160b20 con 0x7f03840d0b00 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0372ffd640 1 -- 172.21.15.113:0/146066258 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c030960 con 0x7f03840d0b00 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.337+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03841607d0 con 0x7f03840d0b00 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.338+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03840691a0 con 0x7f03840d0b00 2024-04-25T00:06:38.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.339+0000 7f0372ffd640 1 -- 172.21.15.113:0/146066258 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f037c030b00 con 0x7f03840d0b00 2024-04-25T00:06:38.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.339+0000 7f0372ffd640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.339+0000 7f0381d9b640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.339+0000 7f0372ffd640 1 -- 172.21.15.113:0/146066258 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f037c0ab770 con 0x7f03840d0b00 2024-04-25T00:06:38.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.339+0000 7f0381d9b640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0384147280 tx=0x7f0378007f90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.343+0000 7f0372ffd640 1 -- 172.21.15.113:0/146066258 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f037c045360 con 0x7f03840d0b00 2024-04-25T00:06:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.582+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f03840b4010 con 0x7f03840d0b00 2024-04-25T00:06:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.583+0000 7f0372ffd640 1 -- 172.21.15.113:0/146066258 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1573 (secure 0 0 0) 0x7f037c07a1b0 con 0x7f03840d0b00 2024-04-25T00:06:38.583 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":523,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":920,"data_bytes":941996034,"bytes_used":2196013056,"bytes_avail":770898100224,"bytes_total":773094113280,"write_bytes_sec":78444249,"read_op_per_sec":0,"write_op_per_sec":76},"fsmap":{"epoch":68,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8973}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T00:06:03.202017+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:06:38.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 msgr2=0x7f036406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0384147280 tx=0x7f0378007f90 comp rx=0 tx=0).stop 2024-04-25T00:06:38.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03840d0b00 msgr2=0x7f0384161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03840d0b00 0x7f0384161600 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f037c000c00 tx=0x7f037c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 shutdown_connections 2024-04-25T00:06:38.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03840d2290 0x7f03840ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f038415e320 0x7f03841f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 --2- 172.21.15.113:0/146066258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03840d0b00 0x7f0384161600 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.584+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 >> 172.21.15.113:0/146066258 conn(0x7f03840b5c20 msgr2=0x7f03840b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.585+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 shutdown_connections 2024-04-25T00:06:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:38.585+0000 7f0388f89640 1 -- 172.21.15.113:0/146066258 wait complete. 2024-04-25T00:06:38.595 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:06:39.597 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:39.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.706+0000 7f88aedb0640 1 Processor -- start 2024-04-25T00:06:39.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.706+0000 7f88aedb0640 1 -- start start 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.706+0000 7f88aedb0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f88a815e320 0x7f88a80d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.706+0000 7f88aedb0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88a80d0b00 0x7f88a80d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.706+0000 7f88aedb0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a80d3de0 con 0x7f88a80d2290 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.706+0000 7f88aedb0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f88a80d14b0 con 0x7f88a80d0b00 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.706+0000 7f88aedb0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f88a80d3410 con 0x7f88a815e320 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88ad34d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88a80d0b00 0x7f88a80d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88ad34d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88a80d0b00 0x7f88a80d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54456/0 (socket says 172.21.15.113:54456) 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88ad34d640 1 -- 172.21.15.113:0/4265890689 learned_addr learned my addr 172.21.15.113:0/4265890689 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88acb4c640 1 --2- 172.21.15.113:0/4265890689 >> v2:172.21.15.178:3300/0 conn(0x7f88a815e320 0x7f88a80d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88ad34d640 1 -- 172.21.15.113:0/4265890689 >> v2:172.21.15.178:3300/0 conn(0x7f88a815e320 msgr2=0x7f88a80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88ad34d640 1 --2- 172.21.15.113:0/4265890689 >> v2:172.21.15.178:3300/0 conn(0x7f88a815e320 0x7f88a80d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88ad34d640 1 -- 172.21.15.113:0/4265890689 >> v1:172.21.15.178:6789/0 conn(0x7f88a80d2290 legacy=0x7f88a80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.707+0000 7f88ad34d640 1 -- 172.21.15.113:0/4265890689 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a80d2800 con 0x7f88a80d0b00 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88ad34d640 1 --2- 172.21.15.113:0/4265890689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d0b00 0x7f88a80d0f70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f88a80b6d00 tx=0x7f889802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973ebbbd5236a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88a77fe640 1 -- 172.21.15.113:0/4265890689 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898037070 con 0x7f88a80d0b00 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88a77fe640 1 -- 172.21.15.113:0/4265890689 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8898032440 con 0x7f88a80d0b00 2024-04-25T00:06:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88a77fe640 1 -- 172.21.15.113:0/4265890689 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898032810 con 0x7f88a80d0b00 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 -- 172.21.15.113:0/4265890689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d0b00 msgr2=0x7f88a80d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 --2- 172.21.15.113:0/4265890689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d0b00 0x7f88a80d0f70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f88a80b6d00 tx=0x7f889802a2c0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 -- 172.21.15.113:0/4265890689 shutdown_connections 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 --2- 172.21.15.113:0/4265890689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d0b00 0x7f88a80d0f70 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 --2- 172.21.15.113:0/4265890689 >> v2:172.21.15.178:3300/0 conn(0x7f88a815e320 0x7f88a80d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 -- 172.21.15.113:0/4265890689 >> 172.21.15.113:0/4265890689 conn(0x7f88a80b5c20 msgr2=0x7f88a80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 -- 172.21.15.113:0/4265890689 shutdown_connections 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.708+0000 7f88aedb0640 1 -- 172.21.15.113:0/4265890689 wait complete. 2024-04-25T00:06:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.709+0000 7f88aedb0640 1 Processor -- start 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.709+0000 7f88aedb0640 1 -- start start 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.710+0000 7f88aedb0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a80d0b00 0x7f88a80c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.710+0000 7f88aedb0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d2290 0x7f88a815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.710+0000 7f88aedb0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88a815e320 0x7f88a815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.710+0000 7f88aedb0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88a80d30a0 con 0x7f88a80d2290 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.710+0000 7f88aedb0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88a80d2dc0 con 0x7f88a80d0b00 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.710+0000 7f88aedb0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f88a815e700 con 0x7f88a815e320 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.710+0000 7f88a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d2290 0x7f88a815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d2290 0x7f88a815ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54470/0 (socket says 172.21.15.113:54470) 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 -- 172.21.15.113:0/3147770169 learned_addr learned my addr 172.21.15.113:0/3147770169 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88acb4c640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a80d0b00 0x7f88a80c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88ad34d640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88a815e320 0x7f88a815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 -- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88a815e320 msgr2=0x7f88a815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88a815e320 0x7f88a815ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 -- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a80d0b00 msgr2=0x7f88a80c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a80d0b00 0x7f88a80c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 -- 172.21.15.113:0/3147770169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a80ca1d0 con 0x7f88a80d2290 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88ad34d640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88a815e320 0x7f88a815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a7fff640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d2290 0x7f88a815ea40 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f889c00c430 tx=0x7f889c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a57fa640 1 -- 172.21.15.113:0/3147770169 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c011790 con 0x7f88a80d2290 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a57fa640 1 -- 172.21.15.113:0/3147770169 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c017070 con 0x7f88a80d2290 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a8154b60 con 0x7f88a80d2290 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88a57fa640 1 -- 172.21.15.113:0/3147770169 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c011bb0 con 0x7f88a80d2290 2024-04-25T00:06:39.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.711+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a8155120 con 0x7f88a80d2290 2024-04-25T00:06:39.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.712+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a80691a0 con 0x7f88a80d2290 2024-04-25T00:06:39.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.713+0000 7f88a57fa640 1 -- 172.21.15.113:0/3147770169 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f889c01c460 con 0x7f88a80d2290 2024-04-25T00:06:39.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.713+0000 7f88a57fa640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f889406be30 0x7f889406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.713+0000 7f88acb4c640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f889406be30 0x7f889406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.713+0000 7f88a57fa640 1 -- 172.21.15.113:0/3147770169 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f889c08ac90 con 0x7f88a80d2290 2024-04-25T00:06:39.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.714+0000 7f88acb4c640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f889406be30 0x7f889406e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f88a0003aa0 tx=0x7f88a0009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:39.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.718+0000 7f88a57fa640 1 -- 172.21.15.113:0/3147770169 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f88a80ca1d0 con 0x7f88a80d2290 2024-04-25T00:06:39.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.926+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88a80d44b0 con 0x7f88a80d2290 2024-04-25T00:06:39.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.927+0000 7f88a57fa640 1 -- 172.21.15.113:0/3147770169 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7f889c059640 con 0x7f88a80d2290 2024-04-25T00:06:39.927 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:39.928 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:39.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f889406be30 msgr2=0x7f889406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f889406be30 0x7f889406e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f88a0003aa0 tx=0x7f88a0009830 comp rx=0 tx=0).stop 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d2290 msgr2=0x7f88a815ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d2290 0x7f88a815ea40 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f889c00c430 tx=0x7f889c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 shutdown_connections 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f889406be30 0x7f889406e2f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88a815e320 0x7f88a815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a80d2290 0x7f88a815ea40 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 --2- 172.21.15.113:0/3147770169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88a80d0b00 0x7f88a80c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 >> 172.21.15.113:0/3147770169 conn(0x7f88a80b5c20 msgr2=0x7f88a80b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 shutdown_connections 2024-04-25T00:06:39.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:39.929+0000 7f88aedb0640 1 -- 172.21.15.113:0/3147770169 wait complete. 2024-04-25T00:06:39.940 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T00:06:39.940 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:06:40.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.035+0000 7faa47fd5640 1 Processor -- start 2024-04-25T00:06:40.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.035+0000 7faa47fd5640 1 -- start start 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa47fd5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faa40072ea0 0x7faa40071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa47fd5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa400b3af0 0x7faa400b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa47fd5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa400b2d50 con 0x7faa4015a6b0 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa47fd5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7faa40072a70 con 0x7faa40072ea0 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa47fd5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faa400b3010 con 0x7faa400b3af0 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa45570640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faa40072ea0 0x7faa40071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa45d71640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7faa4015a6b0 0x7faa40070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57970/0 (socket says 172.21.15.113:57970) 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa45d71640 1 -- 172.21.15.113:0/240528192 learned_addr learned my addr 172.21.15.113:0/240528192 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa45570640 1 -- 172.21.15.113:0/240528192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa400b3af0 msgr2=0x7faa400b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa45570640 1 --2- 172.21.15.113:0/240528192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa400b3af0 0x7faa400b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa45570640 1 -- 172.21.15.113:0/240528192 >> v1:172.21.15.113:6789/0 conn(0x7faa4015a6b0 legacy=0x7faa40070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa45570640 1 -- 172.21.15.113:0/240528192 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa400b32f0 con 0x7faa40072ea0 2024-04-25T00:06:40.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.036+0000 7faa44d6f640 1 -- 172.21.15.113:0/240528192 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3217951107 0 0) 0x7faa400b2d50 con 0x7faa4015a6b0 2024-04-25T00:06:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.037+0000 7faa45570640 1 --2- 172.21.15.113:0/240528192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40072ea0 0x7faa40071410 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7faa300097a0 tx=0x7faa3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9dc3a4ccba14c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.037+0000 7faa44d6f640 1 -- 172.21.15.113:0/240528192 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30037070 con 0x7faa40072ea0 2024-04-25T00:06:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.037+0000 7faa44d6f640 1 -- 172.21.15.113:0/240528192 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa30032440 con 0x7faa40072ea0 2024-04-25T00:06:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.037+0000 7faa44d6f640 1 -- 172.21.15.113:0/240528192 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30032810 con 0x7faa40072ea0 2024-04-25T00:06:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 -- 172.21.15.113:0/240528192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40072ea0 msgr2=0x7faa40071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 --2- 172.21.15.113:0/240528192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40072ea0 0x7faa40071410 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7faa300097a0 tx=0x7faa3002a180 comp rx=0 tx=0).stop 2024-04-25T00:06:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 -- 172.21.15.113:0/240528192 shutdown_connections 2024-04-25T00:06:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 --2- 172.21.15.113:0/240528192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa400b3af0 0x7faa400b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 --2- 172.21.15.113:0/240528192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40072ea0 0x7faa40071410 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 -- 172.21.15.113:0/240528192 >> 172.21.15.113:0/240528192 conn(0x7faa400b1a50 msgr2=0x7faa400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 -- 172.21.15.113:0/240528192 shutdown_connections 2024-04-25T00:06:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.038+0000 7faa47fd5640 1 -- 172.21.15.113:0/240528192 wait complete. 2024-04-25T00:06:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.039+0000 7faa47fd5640 1 Processor -- start 2024-04-25T00:06:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.039+0000 7faa47fd5640 1 -- start start 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa40072ea0 0x7faa400adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa400b3af0 0x7faa400ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4015a6b0 0x7faa401f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faa4015aa90 con 0x7faa400b3af0 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa40072720 con 0x7faa4015a6b0 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faa4015ac10 con 0x7faa40072ea0 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa400b3af0 0x7faa400ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa400b3af0 0x7faa400ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54494/0 (socket says 172.21.15.113:54494) 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 -- 172.21.15.113:0/456771736 learned_addr learned my addr 172.21.15.113:0/456771736 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45d71640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa40072ea0 0x7faa400adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa46572640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4015a6b0 0x7faa401f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 -- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa40072ea0 msgr2=0x7faa400adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa40072ea0 0x7faa400adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 -- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4015a6b0 msgr2=0x7faa401f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4015a6b0 0x7faa401f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 -- 172.21.15.113:0/456771736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa400aebc0 con 0x7faa400b3af0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa46572640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4015a6b0 0x7faa401f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa45570640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa400b3af0 0x7faa400ae4f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7faa300097a0 tx=0x7faa300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa367fc640 1 -- 172.21.15.113:0/456771736 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30041070 con 0x7faa400b3af0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa367fc640 1 -- 172.21.15.113:0/456771736 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3003c870 con 0x7faa400b3af0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa400aeea0 con 0x7faa400b3af0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa367fc640 1 -- 172.21.15.113:0/456771736 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30037040 con 0x7faa400b3af0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.040+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa400af460 con 0x7faa400b3af0 2024-04-25T00:06:40.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.041+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa400691a0 con 0x7faa400b3af0 2024-04-25T00:06:40.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.042+0000 7faa367fc640 1 -- 172.21.15.113:0/456771736 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa30044070 con 0x7faa400b3af0 2024-04-25T00:06:40.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.042+0000 7faa367fc640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faa2406be30 0x7faa2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.042+0000 7faa45d71640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faa2406be30 0x7faa2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.042+0000 7faa367fc640 1 -- 172.21.15.113:0/456771736 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7faa300abb20 con 0x7faa400b3af0 2024-04-25T00:06:40.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.043+0000 7faa45d71640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faa2406be30 0x7faa2406e2f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7faa380012e0 tx=0x7faa38009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.047+0000 7faa367fc640 1 -- 172.21.15.113:0/456771736 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa3007a680 con 0x7faa400b3af0 2024-04-25T00:06:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.286+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7faa400b2900 con 0x7faa400b3af0 2024-04-25T00:06:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.287+0000 7faa367fc640 1 -- 172.21.15.113:0/456771736 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1573 (secure 0 0 0) 0x7faa30079f80 con 0x7faa400b3af0 2024-04-25T00:06:40.287 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:40.287 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":525,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":920,"data_bytes":941996034,"bytes_used":2196013056,"bytes_avail":770898100224,"bytes_total":773094113280,"write_bytes_sec":78442985,"read_op_per_sec":0,"write_op_per_sec":76},"fsmap":{"epoch":68,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8973}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T00:06:03.202017+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:06:40.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.288+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faa2406be30 msgr2=0x7faa2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.288+0000 7faa47fd5640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faa2406be30 0x7faa2406e2f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7faa380012e0 tx=0x7faa38009040 comp rx=0 tx=0).stop 2024-04-25T00:06:40.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa400b3af0 msgr2=0x7faa400ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa400b3af0 0x7faa400ae4f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7faa300097a0 tx=0x7faa300046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 shutdown_connections 2024-04-25T00:06:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faa2406be30 0x7faa2406e2f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4015a6b0 0x7faa401f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa400b3af0 0x7faa400ae4f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 --2- 172.21.15.113:0/456771736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa40072ea0 0x7faa400adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 >> 172.21.15.113:0/456771736 conn(0x7faa400b1a50 msgr2=0x7faa400b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:40.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 shutdown_connections 2024-04-25T00:06:40.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:40.289+0000 7faa47fd5640 1 -- 172.21.15.113:0/456771736 wait complete. 2024-04-25T00:06:40.300 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:06:41.301 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:41.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.403+0000 7fbbb509a640 1 Processor -- start 2024-04-25T00:06:41.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.403+0000 7fbbb509a640 1 -- start start 2024-04-25T00:06:41.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.403+0000 7fbbb509a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbb0144d60 0x7fbbb00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.403+0000 7fbbb509a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb006e6e0 0x7fbbb0071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.403+0000 7fbbb509a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb00cfad0 con 0x7fbbb01453c0 2024-04-25T00:06:41.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.403+0000 7fbbb509a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb0145c40 con 0x7fbbb0144d60 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.403+0000 7fbbb509a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb00cf230 con 0x7fbbb006e6e0 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbaed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbb0144d60 0x7fbbb00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbae59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbbb01453c0 0x7fbbb006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57986/0 (socket says 172.21.15.113:57986) 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbae59c640 1 -- 172.21.15.113:0/3116160808 learned_addr learned my addr 172.21.15.113:0/3116160808 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbaf59e640 1 --2- 172.21.15.113:0/3116160808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb006e6e0 0x7fbbb0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbaed9d640 1 -- 172.21.15.113:0/3116160808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb006e6e0 msgr2=0x7fbbb0071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbaed9d640 1 --2- 172.21.15.113:0/3116160808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb006e6e0 0x7fbbb0071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbadd9b640 1 -- 172.21.15.113:0/3116160808 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955609868 0 0) 0x7fbbb00cfad0 con 0x7fbbb01453c0 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbaed9d640 1 -- 172.21.15.113:0/3116160808 >> v1:172.21.15.113:6789/0 conn(0x7fbbb01453c0 legacy=0x7fbbb006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbaed9d640 1 -- 172.21.15.113:0/3116160808 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb00cf510 con 0x7fbbb0144d60 2024-04-25T00:06:41.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.404+0000 7fbbaf59e640 1 --2- 172.21.15.113:0/3116160808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb006e6e0 0x7fbbb0071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbaed9d640 1 --2- 172.21.15.113:0/3116160808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b5390 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fbba00097a0 tx=0x7fbba002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81b479fcd8cb05a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbadd9b640 1 -- 172.21.15.113:0/3116160808 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0037070 con 0x7fbbb0144d60 2024-04-25T00:06:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbadd9b640 1 -- 172.21.15.113:0/3116160808 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba002ae50 con 0x7fbbb0144d60 2024-04-25T00:06:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbadd9b640 1 -- 172.21.15.113:0/3116160808 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0032660 con 0x7fbbb0144d60 2024-04-25T00:06:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 -- 172.21.15.113:0/3116160808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb0144d60 msgr2=0x7fbbb00b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 --2- 172.21.15.113:0/3116160808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b5390 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fbba00097a0 tx=0x7fbba002a100 comp rx=0 tx=0).stop 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 -- 172.21.15.113:0/3116160808 shutdown_connections 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 --2- 172.21.15.113:0/3116160808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb006e6e0 0x7fbbb0071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 --2- 172.21.15.113:0/3116160808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b5390 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 -- 172.21.15.113:0/3116160808 >> 172.21.15.113:0/3116160808 conn(0x7fbbb00b1a50 msgr2=0x7fbbb00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 -- 172.21.15.113:0/3116160808 shutdown_connections 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.405+0000 7fbbb509a640 1 -- 172.21.15.113:0/3116160808 wait complete. 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 Processor -- start 2024-04-25T00:06:41.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 -- start start 2024-04-25T00:06:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb006e6e0 0x7fbbb013c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb01453c0 0x7fbbb013e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0145140 con 0x7fbbb006e6e0 2024-04-25T00:06:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0070e10 con 0x7fbbb0144d60 2024-04-25T00:06:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.407+0000 7fbbb509a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb0071870 con 0x7fbbb01453c0 2024-04-25T00:06:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb006e6e0 0x7fbbb013c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb006e6e0 0x7fbbb013c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54510/0 (socket says 172.21.15.113:54510) 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 -- 172.21.15.113:0/2161002590 learned_addr learned my addr 172.21.15.113:0/2161002590 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaf59e640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb01453c0 0x7fbbb013e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbae59c640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 -- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb01453c0 msgr2=0x7fbbb013e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb01453c0 0x7fbbb013e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 -- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbb0144d60 msgr2=0x7fbbb00b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 -- 172.21.15.113:0/2161002590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb0159110 con 0x7fbbb006e6e0 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaf59e640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb01453c0 0x7fbbb013e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbae59c640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbaed9d640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb006e6e0 0x7fbbb013c910 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fbba00098c0 tx=0x7fbba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbb977fe640 1 -- 172.21.15.113:0/2161002590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0041030 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbb977fe640 1 -- 172.21.15.113:0/2161002590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba002ac50 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbb977fe640 1 -- 172.21.15.113:0/2161002590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0037040 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb01593b0 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.408+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb0159970 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.409+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb00691a0 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.410+0000 7fbb977fe640 1 -- 172.21.15.113:0/2161002590 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbba0044070 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.410+0000 7fbb977fe640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb9c06be30 0x7fbb9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.410+0000 7fbb977fe640 1 -- 172.21.15.113:0/2161002590 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fbba00ab570 con 0x7fbbb006e6e0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.410+0000 7fbbae59c640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb9c06be30 0x7fbb9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.411+0000 7fbbae59c640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb9c06be30 0x7fbb9c06e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00af4b0 tx=0x7fbba4007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:41.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.414+0000 7fbb977fe640 1 -- 172.21.15.113:0/2161002590 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbba0044e40 con 0x7fbbb006e6e0 2024-04-25T00:06:41.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.621+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbb00b27e0 con 0x7fbbb006e6e0 2024-04-25T00:06:41.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.622+0000 7fbb977fe640 1 -- 172.21.15.113:0/2161002590 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7fbba0079fc0 con 0x7fbbb006e6e0 2024-04-25T00:06:41.623 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:41.623 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:41.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb9c06be30 msgr2=0x7fbb9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb9c06be30 0x7fbb9c06e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00af4b0 tx=0x7fbba4007ca0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb006e6e0 msgr2=0x7fbbb013c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb006e6e0 0x7fbbb013c910 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fbba00098c0 tx=0x7fbba00046c0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 shutdown_connections 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb9c06be30 0x7fbb9c06e2f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbb0144d60 0x7fbbb00b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbb01453c0 0x7fbbb013e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 --2- 172.21.15.113:0/2161002590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb006e6e0 0x7fbbb013c910 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 >> 172.21.15.113:0/2161002590 conn(0x7fbbb00b1a50 msgr2=0x7fbbb00ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 shutdown_connections 2024-04-25T00:06:41.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.624+0000 7fbbb509a640 1 -- 172.21.15.113:0/2161002590 wait complete. 2024-04-25T00:06:41.635 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T00:06:41.635 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:06:41.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.731+0000 7fd5caa9a640 1 Processor -- start 2024-04-25T00:06:41.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.731+0000 7fd5caa9a640 1 -- start start 2024-04-25T00:06:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5caa9a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5c4144910 0x7fd5c40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5caa9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c40b5240 0x7fd5c406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5caa9a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c4144cf0 con 0x7fd5c40b5680 2024-04-25T00:06:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5caa9a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c40b4510 con 0x7fd5c4144910 2024-04-25T00:06:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5caa9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c40b2f30 con 0x7fd5c40b5240 2024-04-25T00:06:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c8836640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5c4144910 0x7fd5c40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c8836640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5c4144910 0x7fd5c40b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54518/0 (socket says 172.21.15.113:54518) 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c8836640 1 -- 172.21.15.113:0/2439280807 learned_addr learned my addr 172.21.15.113:0/2439280807 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c3fff640 1 --2- 172.21.15.113:0/2439280807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c40b5240 0x7fd5c406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c8836640 1 -- 172.21.15.113:0/2439280807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c40b5240 msgr2=0x7fd5c406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c8836640 1 --2- 172.21.15.113:0/2439280807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c40b5240 0x7fd5c406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c8836640 1 -- 172.21.15.113:0/2439280807 >> v1:172.21.15.178:6789/0 conn(0x7fd5c40b5680 legacy=0x7fd5c40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.732+0000 7fd5c8836640 1 -- 172.21.15.113:0/2439280807 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c40b3210 con 0x7fd5c4144910 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5c8836640 1 --2- 172.21.15.113:0/2439280807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c4144910 0x7fd5c40b3fd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009870 tx=0x7fd5b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce8015e53c68fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5c37fe640 1 -- 172.21.15.113:0/2439280807 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8037070 con 0x7fd5c4144910 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5c37fe640 1 -- 172.21.15.113:0/2439280807 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b8032440 con 0x7fd5c4144910 2024-04-25T00:06:41.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5c37fe640 1 -- 172.21.15.113:0/2439280807 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8032810 con 0x7fd5c4144910 2024-04-25T00:06:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 -- 172.21.15.113:0/2439280807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c4144910 msgr2=0x7fd5c40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/2439280807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c4144910 0x7fd5c40b3fd0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009870 tx=0x7fd5b802a210 comp rx=0 tx=0).stop 2024-04-25T00:06:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 -- 172.21.15.113:0/2439280807 shutdown_connections 2024-04-25T00:06:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/2439280807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c40b5240 0x7fd5c406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/2439280807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c4144910 0x7fd5c40b3fd0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 -- 172.21.15.113:0/2439280807 >> 172.21.15.113:0/2439280807 conn(0x7fd5c40b1a50 msgr2=0x7fd5c40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 -- 172.21.15.113:0/2439280807 shutdown_connections 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.733+0000 7fd5caa9a640 1 -- 172.21.15.113:0/2439280807 wait complete. 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 Processor -- start 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 -- start start 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5c40b5240 0x7fd5c415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c40b5680 0x7fd5c4133230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c4144910 0x7fd5c4131620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c40b4da0 con 0x7fd5c40b5680 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c40b37d0 con 0x7fd5c40b5240 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5caa9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c40b4f20 con 0x7fd5c4144910 2024-04-25T00:06:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.735+0000 7fd5c3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c40b5680 0x7fd5c4133230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c40b5680 0x7fd5c4133230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54524/0 (socket says 172.21.15.113:54524) 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 -- 172.21.15.113:0/3059541178 learned_addr learned my addr 172.21.15.113:0/3059541178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c8836640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5c40b5240 0x7fd5c415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c9037640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c4144910 0x7fd5c4131620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 -- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c4144910 msgr2=0x7fd5c4131620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c4144910 0x7fd5c4131620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 -- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5c40b5240 msgr2=0x7fd5c415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5c40b5240 0x7fd5c415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 -- 172.21.15.113:0/3059541178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c40ce8d0 con 0x7fd5c40b5680 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c8836640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5c40b5240 0x7fd5c415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c3fff640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c40b5680 0x7fd5c4133230 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000c820 tx=0x7fd5b000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c17fa640 1 -- 172.21.15.113:0/3059541178 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0015070 con 0x7fd5c40b5680 2024-04-25T00:06:41.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c17fa640 1 -- 172.21.15.113:0/3059541178 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b0004020 con 0x7fd5c40b5680 2024-04-25T00:06:41.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c4132190 con 0x7fd5c40b5680 2024-04-25T00:06:41.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5c17fa640 1 -- 172.21.15.113:0/3059541178 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00043f0 con 0x7fd5c40b5680 2024-04-25T00:06:41.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.736+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c4132750 con 0x7fd5c40b5680 2024-04-25T00:06:41.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.737+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c40691a0 con 0x7fd5c40b5680 2024-04-25T00:06:41.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.738+0000 7fd5c17fa640 1 -- 172.21.15.113:0/3059541178 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5b0007830 con 0x7fd5c40b5680 2024-04-25T00:06:41.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.738+0000 7fd5c17fa640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5a806be60 0x7fd5a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:41.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.738+0000 7fd5c17fa640 1 -- 172.21.15.113:0/3059541178 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fd5b008ac80 con 0x7fd5c40b5680 2024-04-25T00:06:41.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.738+0000 7fd5c8836640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5a806be60 0x7fd5a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:41.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.738+0000 7fd5c8836640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5a806be60 0x7fd5a806e320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8034000 tx=0x7fd5b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:41.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.742+0000 7fd5c17fa640 1 -- 172.21.15.113:0/3059541178 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd5b00595a0 con 0x7fd5c40b5680 2024-04-25T00:06:41.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.979+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd5c415f5b0 con 0x7fd5c40b5680 2024-04-25T00:06:41.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.980+0000 7fd5c17fa640 1 -- 172.21.15.113:0/3059541178 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1598 (secure 0 0 0) 0x7fd5b0058ea0 con 0x7fd5c40b5680 2024-04-25T00:06:41.980 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":526,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076562958,"bytes_used":2546671616,"bytes_avail":770547441664,"bytes_total":773094113280,"write_bytes_sec":89658506,"read_op_per_sec":0,"write_op_per_sec":88},"fsmap":{"epoch":68,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8973}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T00:06:41.209198+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:06:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5a806be60 msgr2=0x7fd5a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5a806be60 0x7fd5a806e320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8034000 tx=0x7fd5b802e000 comp rx=0 tx=0).stop 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c40b5680 msgr2=0x7fd5c4133230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c40b5680 0x7fd5c4133230 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000c820 tx=0x7fd5b000cce0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 shutdown_connections 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd5a806be60 0x7fd5a806e320 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5c4144910 0x7fd5c4131620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c40b5680 0x7fd5c4133230 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 --2- 172.21.15.113:0/3059541178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5c40b5240 0x7fd5c415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 >> 172.21.15.113:0/3059541178 conn(0x7fd5c40b1a50 msgr2=0x7fd5c406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 shutdown_connections 2024-04-25T00:06:41.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:41.982+0000 7fd5caa9a640 1 -- 172.21.15.113:0/3059541178 wait complete. 2024-04-25T00:06:41.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T00:06:41.994 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:06:41.994 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:06:42.004 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:06:42.004 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'b', 'a'} 2024-04-25T00:06:42.004 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T00:06:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T00:06:42.148 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T00:06:42.149 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.245+0000 7f2c31abd640 1 Processor -- start 2024-04-25T00:06:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.245+0000 7f2c31abd640 1 -- start start 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c31abd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c2c15a6b0 0x7f2c2c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c31abd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 0x7f2c2c071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c31abd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c2c0b2d50 con 0x7f2c2c071780 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c31abd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2c2c072a70 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c31abd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b3010 con 0x7f2c2c072ea0 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c2c15a6b0 0x7f2c2c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2c2c071780 0x7f2c2c15e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57988/0 (socket says 172.21.15.113:57988) 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2bfff640 1 -- 172.21.15.113:0/544926510 learned_addr learned my addr 172.21.15.113:0/544926510 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2b7fe640 1 -- 172.21.15.113:0/544926510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 msgr2=0x7f2c2c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2b7fe640 1 --2- 172.21.15.113:0/544926510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 0x7f2c2c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2b7fe640 1 -- 172.21.15.113:0/544926510 >> v1:172.21.15.113:6789/0 conn(0x7f2c2c071780 legacy=0x7f2c2c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:42.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2b7fe640 1 -- 172.21.15.113:0/544926510 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c0b32f0 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.246+0000 7f2c2a7fc640 1 -- 172.21.15.113:0/544926510 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1568822979 0 0) 0x7f2c2c0b2d50 con 0x7f2c2c071780 2024-04-25T00:06:42.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.247+0000 7f2c2b7fe640 1 --2- 172.21.15.113:0/544926510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c070d00 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0097a0 tx=0x7f2c1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b7c1099ae7fbe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:42.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.247+0000 7f2c2a7fc640 1 -- 172.21.15.113:0/544926510 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c037070 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.247+0000 7f2c2a7fc640 1 -- 172.21.15.113:0/544926510 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c032440 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.247+0000 7f2c2a7fc640 1 -- 172.21.15.113:0/544926510 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c032810 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.247+0000 7f2c31abd640 1 -- 172.21.15.113:0/544926510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 msgr2=0x7f2c2c070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.247+0000 7f2c31abd640 1 --2- 172.21.15.113:0/544926510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c070d00 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0097a0 tx=0x7f2c1c02a180 comp rx=0 tx=0).stop 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.248+0000 7f2c31abd640 1 -- 172.21.15.113:0/544926510 shutdown_connections 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.248+0000 7f2c31abd640 1 --2- 172.21.15.113:0/544926510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 0x7f2c2c071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.248+0000 7f2c31abd640 1 --2- 172.21.15.113:0/544926510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c070d00 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.248+0000 7f2c31abd640 1 -- 172.21.15.113:0/544926510 >> 172.21.15.113:0/544926510 conn(0x7f2c2c0b1a50 msgr2=0x7f2c2c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.248+0000 7f2c31abd640 1 -- 172.21.15.113:0/544926510 shutdown_connections 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.248+0000 7f2c31abd640 1 -- 172.21.15.113:0/544926510 wait complete. 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.249+0000 7f2c31abd640 1 Processor -- start 2024-04-25T00:06:42.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.249+0000 7f2c31abd640 1 -- start start 2024-04-25T00:06:42.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c071780 0x7f2c2c14f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:42.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 0x7f2c2c1f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:42.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c1f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:42.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c15aa90 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c072720 con 0x7f2c2c071780 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2c15ac10 con 0x7f2c2c072ea0 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c1f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c1f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54546/0 (socket says 172.21.15.113:54546) 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 -- 172.21.15.113:0/3495100644 learned_addr learned my addr 172.21.15.113:0/3495100644 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2b7fe640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c071780 0x7f2c2c14f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2affd640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 0x7f2c2c1f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 -- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 msgr2=0x7f2c2c1f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 0x7f2c2c1f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 -- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c071780 msgr2=0x7f2c2c14f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c071780 0x7f2c2c14f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 -- 172.21.15.113:0/3495100644 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1f5fe0 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c2bfff640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c1f5aa0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400c800 tx=0x7f2c2400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:42.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c13fff640 1 -- 172.21.15.113:0/3495100644 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24018070 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c13fff640 1 -- 172.21.15.113:0/3495100644 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c24004d80 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c1f62f0 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c13fff640 1 -- 172.21.15.113:0/3495100644 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2400a880 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.250+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c1e0130 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.251+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c0691a0 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.252+0000 7f2c13fff640 1 -- 172.21.15.113:0/3495100644 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c2401b070 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.252+0000 7f2c13fff640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c1806be60 0x7f2c1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.252+0000 7f2c2b7fe640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c1806be60 0x7f2c1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.252+0000 7f2c13fff640 1 -- 172.21.15.113:0/3495100644 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f2c2408a160 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.253+0000 7f2c2b7fe640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c1806be60 0x7f2c1806e320 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c034000 tx=0x7f2c1c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:42.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.257+0000 7f2c13fff640 1 -- 172.21.15.113:0/3495100644 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c2c1f62f0 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.465+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c2c070d00 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.465+0000 7f2c13fff640 1 -- 172.21.15.113:0/3495100644 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7f2c24058610 con 0x7f2c2c15a6b0 2024-04-25T00:06:42.466 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:42.466 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:42.466 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:42.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c1806be60 msgr2=0x7f2c1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:42.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c1806be60 0x7f2c1806e320 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c034000 tx=0x7f2c1c02e000 comp rx=0 tx=0).stop 2024-04-25T00:06:42.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 msgr2=0x7f2c2c1f5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:42.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c1f5aa0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400c800 tx=0x7f2c2400cc00 comp rx=0 tx=0).stop 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 shutdown_connections 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c1806be60 0x7f2c1806e320 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c15a6b0 0x7f2c2c1f5aa0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c072ea0 0x7f2c2c1f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 --2- 172.21.15.113:0/3495100644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c071780 0x7f2c2c14f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 >> 172.21.15.113:0/3495100644 conn(0x7f2c2c0b1a50 msgr2=0x7f2c2c150a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 shutdown_connections 2024-04-25T00:06:42.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:42.468+0000 7f2c31abd640 1 -- 172.21.15.113:0/3495100644 wait complete. 2024-04-25T00:06:42.480 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T00:06:47.482 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:47.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.588+0000 7fe4c695a640 1 Processor -- start 2024-04-25T00:06:47.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.588+0000 7fe4c695a640 1 -- start start 2024-04-25T00:06:47.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c695a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe4c006e6e0 0x7fe4c00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c695a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4c01453c0 0x7fe4c0071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c695a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c00cfad0 con 0x7fe4c0144d60 2024-04-25T00:06:47.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c695a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c0145c40 con 0x7fe4c01453c0 2024-04-25T00:06:47.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c695a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c00cf230 con 0x7fe4c006e6e0 2024-04-25T00:06:47.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c4ef7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4c01453c0 0x7fe4c0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4bffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe4c0144d60 0x7fe4c006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53184/0 (socket says 172.21.15.113:53184) 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4bffff640 1 -- 172.21.15.113:0/2075354694 learned_addr learned my addr 172.21.15.113:0/2075354694 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4bf7fe640 1 --2- 172.21.15.113:0/2075354694 >> v2:172.21.15.178:3300/0 conn(0x7fe4c006e6e0 0x7fe4c00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c4ef7640 1 -- 172.21.15.113:0/2075354694 >> v2:172.21.15.178:3300/0 conn(0x7fe4c006e6e0 msgr2=0x7fe4c00b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c4ef7640 1 --2- 172.21.15.113:0/2075354694 >> v2:172.21.15.178:3300/0 conn(0x7fe4c006e6e0 0x7fe4c00b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c4ef7640 1 -- 172.21.15.113:0/2075354694 >> v1:172.21.15.113:6789/0 conn(0x7fe4c0144d60 legacy=0x7fe4c006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4beffd640 1 -- 172.21.15.113:0/2075354694 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1265507568 0 0) 0x7fe4c00cfad0 con 0x7fe4c0144d60 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.589+0000 7fe4c4ef7640 1 -- 172.21.15.113:0/2075354694 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c00cf510 con 0x7fe4c01453c0 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4c4ef7640 1 --2- 172.21.15.113:0/2075354694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c0071330 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0075250 tx=0x7fe4b802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac34541b0e2cdea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4beffd640 1 -- 172.21.15.113:0/2075354694 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8007740 con 0x7fe4c01453c0 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4beffd640 1 -- 172.21.15.113:0/2075354694 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b80078e0 con 0x7fe4c01453c0 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4beffd640 1 -- 172.21.15.113:0/2075354694 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8007cb0 con 0x7fe4c01453c0 2024-04-25T00:06:47.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4c695a640 1 -- 172.21.15.113:0/2075354694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 msgr2=0x7fe4c0071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4c695a640 1 --2- 172.21.15.113:0/2075354694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c0071330 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0075250 tx=0x7fe4b802a560 comp rx=0 tx=0).stop 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4c695a640 1 -- 172.21.15.113:0/2075354694 shutdown_connections 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4c695a640 1 --2- 172.21.15.113:0/2075354694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c0071330 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4c695a640 1 --2- 172.21.15.113:0/2075354694 >> v2:172.21.15.178:3300/0 conn(0x7fe4c006e6e0 0x7fe4c00b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.590+0000 7fe4c695a640 1 -- 172.21.15.113:0/2075354694 >> 172.21.15.113:0/2075354694 conn(0x7fe4c00b1a50 msgr2=0x7fe4c00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.591+0000 7fe4c695a640 1 -- 172.21.15.113:0/2075354694 shutdown_connections 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.591+0000 7fe4c695a640 1 -- 172.21.15.113:0/2075354694 wait complete. 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.592+0000 7fe4c695a640 1 Processor -- start 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.592+0000 7fe4c695a640 1 -- start start 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c006e6e0 0x7fe4c013c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c0144d60 0x7fe4c0160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0145140 con 0x7fe4c01453c0 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0070e10 con 0x7fe4c0144d60 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c0071870 con 0x7fe4c006e6e0 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c013f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43136/0 (socket says 172.21.15.113:43136) 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 -- 172.21.15.113:0/3592292118 learned_addr learned my addr 172.21.15.113:0/3592292118 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 -- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c006e6e0 msgr2=0x7fe4c013c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4bffff640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c006e6e0 0x7fe4c013c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4bf7fe640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c0144d60 0x7fe4c0160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c006e6e0 0x7fe4c013c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 -- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c0144d60 msgr2=0x7fe4c0160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c0144d60 0x7fe4c0160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 -- 172.21.15.113:0/3592292118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c013d120 con 0x7fe4c01453c0 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4bf7fe640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c0144d60 0x7fe4c0160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:47.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4bffff640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c006e6e0 0x7fe4c013c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c4ef7640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c013f5c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fe4b802ab50 tx=0x7fe4b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4bcff9640 1 -- 172.21.15.113:0/3592292118 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b803f070 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4bcff9640 1 -- 172.21.15.113:0/3592292118 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8031c40 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c013d400 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4bcff9640 1 -- 172.21.15.113:0/3592292118 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8009450 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.593+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c013d9c0 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.594+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c013dbc0 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.595+0000 7fe4bcff9640 1 -- 172.21.15.113:0/3592292118 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4b8003990 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.595+0000 7fe4bcff9640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.595+0000 7fe4bcff9640 1 -- 172.21.15.113:0/3592292118 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7fe4b80abbb0 con 0x7fe4c01453c0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.595+0000 7fe4bffff640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.596+0000 7fe4bffff640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0000f60 tx=0x7fe4b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:47.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.599+0000 7fe4bcff9640 1 -- 172.21.15.113:0/3592292118 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe4c013dbc0 con 0x7fe4c01453c0 2024-04-25T00:06:47.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.808+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4c00691a0 con 0x7fe4c01453c0 2024-04-25T00:06:47.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.809+0000 7fe4bcff9640 1 -- 172.21.15.113:0/3592292118 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7fe4b807a5f0 con 0x7fe4c01453c0 2024-04-25T00:06:47.810 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:47.810 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:47.811 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:47.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 msgr2=0x7fe4ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0000f60 tx=0x7fe4b0009040 comp rx=0 tx=0).stop 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 msgr2=0x7fe4c013f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c013f5c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fe4b802ab50 tx=0x7fe4b8004a40 comp rx=0 tx=0).stop 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 shutdown_connections 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe4ac06be30 0x7fe4ac06e2f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4c0144d60 0x7fe4c0160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4c01453c0 0x7fe4c013f5c0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 --2- 172.21.15.113:0/3592292118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4c006e6e0 0x7fe4c013c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 >> 172.21.15.113:0/3592292118 conn(0x7fe4c00b1a50 msgr2=0x7fe4c00b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 shutdown_connections 2024-04-25T00:06:47.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:47.811+0000 7fe4c695a640 1 -- 172.21.15.113:0/3592292118 wait complete. 2024-04-25T00:06:47.824 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T00:06:52.824 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:52.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.934+0000 7ff819c93640 1 Processor -- start 2024-04-25T00:06:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.934+0000 7ff819c93640 1 -- start start 2024-04-25T00:06:52.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff819c93640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff814144d60 0x7ff8140b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff819c93640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8140cfad0 con 0x7ff81406e6e0 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff819c93640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8140cf230 con 0x7ff8141453c0 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff819c93640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff814145c40 con 0x7ff814144d60 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff813fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff81406e6e0 0x7ff814071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53194/0 (socket says 172.21.15.113:53194) 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff813fff640 1 -- 172.21.15.113:0/3329387663 learned_addr learned my addr 172.21.15.113:0/3329387663 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138531114 0 0) 0x7ff8140cfad0 con 0x7ff81406e6e0 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3329387663 >> v2:172.21.15.178:3300/0 conn(0x7ff814144d60 0x7ff8140b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7fc003520 con 0x7ff81406e6e0 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3974098689 0 0) 0x7ff7fc003520 con 0x7ff81406e6e0 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 >> v2:172.21.15.178:3300/0 conn(0x7ff814144d60 msgr2=0x7ff8140b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff8127fc640 1 --2- 172.21.15.113:0/3329387663 >> v2:172.21.15.178:3300/0 conn(0x7ff814144d60 0x7ff8140b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.935+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 >> v1:172.21.15.178:6789/0 conn(0x7ff8141453c0 legacy=0x7ff81406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8140cf510 con 0x7ff81406e6e0 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff80c0019f0 con 0x7ff81406e6e0 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 >> v1:172.21.15.113:6789/0 conn(0x7ff81406e6e0 legacy=0x7ff814071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7fc0039b0 0x7ff7fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0043b0 0x7ff7fc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7fc005120 0x7ff7fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff814145c40 con 0x7ff7fc0039b0 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc00c060 con 0x7ff7fc0043b0 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff7fc00c260 con 0x7ff7fc005120 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7fc0039b0 0x7ff7fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff813fff640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7fc005120 0x7ff7fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff812ffd640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0043b0 0x7ff7fc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8137fe640 1 -- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7fc005120 msgr2=0x7ff7fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7fc005120 0x7ff7fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8137fe640 1 -- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0043b0 msgr2=0x7ff7fc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0043b0 0x7ff7fc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff8137fe640 1 -- 172.21.15.113:0/3329387663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc003520 con 0x7ff7fc0039b0 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.936+0000 7ff813fff640 1 -- 172.21.15.113:0/3329387663 reap_dead start 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7fc0039b0 0x7ff7fc003d90 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7ff804009ac0 tx=0x7ff7fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d581f01e35726fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804036070 con 0x7ff7fc0039b0 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff804031400 con 0x7ff7fc0039b0 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff8127fc640 1 -- 172.21.15.113:0/3329387663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804009690 con 0x7ff7fc0039b0 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff819c93640 1 -- 172.21.15.113:0/3329387663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7fc0039b0 msgr2=0x7ff7fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff819c93640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7fc0039b0 0x7ff7fc003d90 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7ff804009ac0 tx=0x7ff7fc005830 comp rx=0 tx=0).stop 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff819c93640 1 -- 172.21.15.113:0/3329387663 shutdown_connections 2024-04-25T00:06:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff819c93640 1 --2- 172.21.15.113:0/3329387663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7fc0039b0 0x7ff7fc003d90 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff819c93640 1 -- 172.21.15.113:0/3329387663 >> 172.21.15.113:0/3329387663 conn(0x7ff8140b1a50 msgr2=0x7ff8140b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff819c93640 1 -- 172.21.15.113:0/3329387663 shutdown_connections 2024-04-25T00:06:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.937+0000 7ff819c93640 1 -- 172.21.15.113:0/3329387663 wait complete. 2024-04-25T00:06:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.939+0000 7ff819c93640 1 Processor -- start 2024-04-25T00:06:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.939+0000 7ff819c93640 1 -- start start 2024-04-25T00:06:52.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0039b0 0x7ff81415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff81415eb20 0x7ff81415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff81413f420 0x7ff81413f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff814145140 con 0x7ff81415eb20 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff814070e10 con 0x7ff7fc0039b0 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff8140b5390 con 0x7ff81413f420 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff81415eb20 0x7ff81415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff81415eb20 0x7ff81415efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43164/0 (socket says 172.21.15.113:43164) 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 -- 172.21.15.113:0/3321014193 learned_addr learned my addr 172.21.15.113:0/3321014193 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0039b0 0x7ff81415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff813fff640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff81413f420 0x7ff81413f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 -- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff81413f420 msgr2=0x7ff81413f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff81413f420 0x7ff81413f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 -- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0039b0 msgr2=0x7ff81415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0039b0 0x7ff81415b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 -- 172.21.15.113:0/3321014193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81413ea20 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0039b0 0x7ff81415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff812ffd640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff81415eb20 0x7ff81415efa0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ff80800e830 tx=0x7ff80800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff7fbfff640 1 -- 172.21.15.113:0/3321014193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8080050d0 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff7fbfff640 1 -- 172.21.15.113:0/3321014193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff808018070 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8140ae470 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff7fbfff640 1 -- 172.21.15.113:0/3321014193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80800c660 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.940+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8140b2ff0 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.941+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8140691a0 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.942+0000 7ff7fbfff640 1 -- 172.21.15.113:0/3321014193 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff80800ca10 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.942+0000 7ff7fbfff640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff80006be60 0x7ff80006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.942+0000 7ff7fbfff640 1 -- 172.21.15.113:0/3321014193 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7ff80808ad70 con 0x7ff81415eb20 2024-04-25T00:06:52.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.942+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff80006be60 0x7ff80006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.943+0000 7ff8137fe640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff80006be60 0x7ff80006e320 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff804033000 tx=0x7ff804008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:52.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:52.946+0000 7ff7fbfff640 1 -- 172.21.15.113:0/3321014193 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff81413ea20 con 0x7ff81415eb20 2024-04-25T00:06:53.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.154+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8140ae990 con 0x7ff81415eb20 2024-04-25T00:06:53.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.154+0000 7ff7fbfff640 1 -- 172.21.15.113:0/3321014193 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7ff80801a310 con 0x7ff81415eb20 2024-04-25T00:06:53.155 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:53.155 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:53.155 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff80006be60 msgr2=0x7ff80006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff80006be60 0x7ff80006e320 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff804033000 tx=0x7ff804008030 comp rx=0 tx=0).stop 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff81415eb20 msgr2=0x7ff81415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff81415eb20 0x7ff81415efa0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ff80800e830 tx=0x7ff80800ecf0 comp rx=0 tx=0).stop 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 shutdown_connections 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff80006be60 0x7ff80006e320 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff81413f420 0x7ff81413f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff81415eb20 0x7ff81415efa0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 --2- 172.21.15.113:0/3321014193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7fc0039b0 0x7ff81415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 >> 172.21.15.113:0/3321014193 conn(0x7ff8140b1a50 msgr2=0x7ff8140af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 shutdown_connections 2024-04-25T00:06:53.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:53.156+0000 7ff819c93640 1 -- 172.21.15.113:0/3321014193 wait complete. 2024-04-25T00:06:53.168 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T00:06:53.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:06:53.305 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:06:58.170 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:06:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.278+0000 7f4e24196640 1 Processor -- start 2024-04-25T00:06:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.278+0000 7f4e24196640 1 -- start start 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e24196640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e1c144910 0x7f4e1c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e24196640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b56b0 0x7f4e1c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e24196640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e1c144cf0 con 0x7f4e1c0b5240 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e24196640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4e1c0b4510 con 0x7f4e1c144910 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e24196640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b2f30 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e21f32640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e1c144910 0x7f4e1c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e21731640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4e1c0b5240 0x7f4e1c06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48442/0 (socket says 172.21.15.113:48442) 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e21731640 1 -- 172.21.15.113:0/36383675 learned_addr learned my addr 172.21.15.113:0/36383675 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:06:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e22733640 1 --2- 172.21.15.113:0/36383675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b56b0 0x7f4e1c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e21f32640 1 -- 172.21.15.113:0/36383675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b56b0 msgr2=0x7f4e1c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.279+0000 7f4e21f32640 1 --2- 172.21.15.113:0/36383675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b56b0 0x7f4e1c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.280+0000 7f4e21f32640 1 -- 172.21.15.113:0/36383675 >> v1:172.21.15.113:6789/0 conn(0x7f4e1c0b5240 legacy=0x7f4e1c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.280+0000 7f4e21f32640 1 -- 172.21.15.113:0/36383675 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c0b3210 con 0x7f4e1c144910 2024-04-25T00:06:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.280+0000 7f4e20f30640 1 -- 172.21.15.113:0/36383675 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308117359 0 0) 0x7f4e1c144cf0 con 0x7f4e1c0b5240 2024-04-25T00:06:58.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.280+0000 7f4e21f32640 1 --2- 172.21.15.113:0/36383675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c144910 0x7f4e1c0b3fd0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f4e140097d0 tx=0x7f4e1402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1ccb795a72b13f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:58.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.280+0000 7f4e20f30640 1 -- 172.21.15.113:0/36383675 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14037070 con 0x7f4e1c144910 2024-04-25T00:06:58.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.280+0000 7f4e20f30640 1 -- 172.21.15.113:0/36383675 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e14032440 con 0x7f4e1c144910 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.280+0000 7f4e20f30640 1 -- 172.21.15.113:0/36383675 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14032810 con 0x7f4e1c144910 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 -- 172.21.15.113:0/36383675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c144910 msgr2=0x7f4e1c0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 --2- 172.21.15.113:0/36383675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c144910 0x7f4e1c0b3fd0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f4e140097d0 tx=0x7f4e1402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 -- 172.21.15.113:0/36383675 shutdown_connections 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 --2- 172.21.15.113:0/36383675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b56b0 0x7f4e1c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 --2- 172.21.15.113:0/36383675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c144910 0x7f4e1c0b3fd0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 -- 172.21.15.113:0/36383675 >> 172.21.15.113:0/36383675 conn(0x7f4e1c0b1a50 msgr2=0x7f4e1c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 -- 172.21.15.113:0/36383675 shutdown_connections 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.281+0000 7f4e24196640 1 -- 172.21.15.113:0/36383675 wait complete. 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.282+0000 7f4e24196640 1 Processor -- start 2024-04-25T00:06:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.282+0000 7f4e24196640 1 -- start start 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e24196640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b5240 0x7f4e1c0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e24196640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c0b56b0 0x7f4e1c15c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e24196640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e1c144910 0x7f4e1c139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e24196640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b4da0 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e24196640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b37d0 con 0x7f4e1c144910 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e24196640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b4f20 con 0x7f4e1c0b5240 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c0b56b0 0x7f4e1c15c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c0b56b0 0x7f4e1c15c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55158/0 (socket says 172.21.15.113:55158) 2024-04-25T00:06:58.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 -- 172.21.15.113:0/2104110877 learned_addr learned my addr 172.21.15.113:0/2104110877 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 -- 172.21.15.113:0/2104110877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b5240 msgr2=0x7f4e1c0cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b5240 0x7f4e1c0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 -- 172.21.15.113:0/2104110877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e1c144910 msgr2=0x7f4e1c139240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e1c144910 0x7f4e1c139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 -- 172.21.15.113:0/2104110877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c15cee0 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.283+0000 7f4e21731640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c0b56b0 0x7f4e1c15c4f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c4d0 tx=0x7f4e0c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.284+0000 7f4e127fc640 1 -- 172.21.15.113:0/2104110877 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c018070 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.284+0000 7f4e127fc640 1 -- 172.21.15.113:0/2104110877 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e0c007500 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.284+0000 7f4e127fc640 1 -- 172.21.15.113:0/2104110877 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c004a20 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.284+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c15d1c0 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.284+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c15d780 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.285+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1c0691a0 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.285+0000 7f4e127fc640 1 -- 172.21.15.113:0/2104110877 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e0c01b070 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.285+0000 7f4e127fc640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e0006be30 0x7f4e0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:58.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.286+0000 7f4e127fc640 1 -- 172.21.15.113:0/2104110877 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5709+0+0 (secure 0 0 0) 0x7f4e0c08a150 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.286+0000 7f4e21f32640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e0006be30 0x7f4e0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:58.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.286+0000 7f4e21f32640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e0006be30 0x7f4e0006e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4e14034000 tx=0x7f4e1402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.290+0000 7f4e127fc640 1 -- 172.21.15.113:0/2104110877 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e0c058b90 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.499+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e1c072d00 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.500+0000 7f4e127fc640 1 -- 172.21.15.113:0/2104110877 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5192 (secure 0 0 0) 0x7f4e0c058490 con 0x7f4e1c0b56b0 2024-04-25T00:06:58.500 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:06:58.500 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:06:22.029624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/2664195523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2664195523},{"type":"v1","addr":"172.21.15.113:6836","nonce":2664195523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8973,"qdb_cluster":[8973]},"id":9}]} 2024-04-25T00:06:58.501 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-04-25T00:06:58.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e0006be30 msgr2=0x7f4e0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:58.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e0006be30 0x7f4e0006e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4e14034000 tx=0x7f4e1402c000 comp rx=0 tx=0).stop 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c0b56b0 msgr2=0x7f4e1c15c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c0b56b0 0x7f4e1c15c4f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c4d0 tx=0x7f4e0c00c990 comp rx=0 tx=0).stop 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 shutdown_connections 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4e0006be30 0x7f4e0006e2f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e1c144910 0x7f4e1c139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e1c0b56b0 0x7f4e1c15c4f0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 --2- 172.21.15.113:0/2104110877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e1c0b5240 0x7f4e1c0cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 >> 172.21.15.113:0/2104110877 conn(0x7f4e1c0b1a50 msgr2=0x7f4e1c0af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 shutdown_connections 2024-04-25T00:06:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:06:58.502+0000 7f4e24196640 1 -- 172.21.15.113:0/2104110877 wait complete. 2024-04-25T00:06:58.513 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T00:07:03.515 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.662+0000 7fb849114640 1 Processor -- start 2024-04-25T00:07:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.662+0000 7fb849114640 1 -- start start 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb849114640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb84415e320 0x7fb8440d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb849114640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8440d3de0 con 0x7fb8440d2290 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb849114640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8440d2800 con 0x7fb8440d0b00 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb849114640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb8440d3410 con 0x7fb84415e320 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb842d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb84415e320 0x7fb8440d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb84259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb8440d2290 0x7fb8440b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48452/0 (socket says 172.21.15.113:48452) 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb84259c640 1 -- 172.21.15.113:0/2106427947 learned_addr learned my addr 172.21.15.113:0/2106427947 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb842d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb84415e320 0x7fb8440d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55160/0 (socket says 172.21.15.113:55160) 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb842d9d640 1 -- 172.21.15.113:0/2106427947 >> v1:172.21.15.178:6789/0 conn(0x7fb8440d0b00 legacy=0x7fb8440b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb842d9d640 1 -- 172.21.15.113:0/2106427947 >> v1:172.21.15.113:6789/0 conn(0x7fb8440d2290 legacy=0x7fb8440b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb842d9d640 1 -- 172.21.15.113:0/2106427947 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8440d2ae0 con 0x7fb84415e320 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.663+0000 7fb841d9b640 1 -- 172.21.15.113:0/2106427947 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377471462 0 0) 0x7fb8440d3de0 con 0x7fb8440d2290 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb842d9d640 1 --2- 172.21.15.113:0/2106427947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb84415e320 0x7fb8440d38a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb8340097a0 tx=0x7fb83402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf1565cbcf7aef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb841d9b640 1 -- 172.21.15.113:0/2106427947 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834037070 con 0x7fb84415e320 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb841d9b640 1 -- 172.21.15.113:0/2106427947 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb834032640 con 0x7fb84415e320 2024-04-25T00:07:03.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb841d9b640 1 -- 172.21.15.113:0/2106427947 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834032a10 con 0x7fb84415e320 2024-04-25T00:07:03.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb849114640 1 -- 172.21.15.113:0/2106427947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb84415e320 msgr2=0x7fb8440d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb849114640 1 --2- 172.21.15.113:0/2106427947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb84415e320 0x7fb8440d38a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb8340097a0 tx=0x7fb83402a2c0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb849114640 1 -- 172.21.15.113:0/2106427947 shutdown_connections 2024-04-25T00:07:03.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb849114640 1 --2- 172.21.15.113:0/2106427947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb84415e320 0x7fb8440d38a0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb849114640 1 -- 172.21.15.113:0/2106427947 >> 172.21.15.113:0/2106427947 conn(0x7fb8440b5c20 msgr2=0x7fb8440b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:03.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb849114640 1 -- 172.21.15.113:0/2106427947 shutdown_connections 2024-04-25T00:07:03.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.664+0000 7fb849114640 1 -- 172.21.15.113:0/2106427947 wait complete. 2024-04-25T00:07:03.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.666+0000 7fb849114640 1 Processor -- start 2024-04-25T00:07:03.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.666+0000 7fb849114640 1 -- start start 2024-04-25T00:07:03.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8440d0b00 0x7fb84415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440d2290 0x7fb8440cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb84415e320 0x7fb84415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb84415e700 con 0x7fb8440d0b00 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8440d30a0 con 0x7fb84415e320 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb84415e880 con 0x7fb8440d2290 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8440d0b00 0x7fb84415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8440d0b00 0x7fb84415a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55172/0 (socket says 172.21.15.113:55172) 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 -- 172.21.15.113:0/2661757762 learned_addr learned my addr 172.21.15.113:0/2661757762 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb84359e640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb84415e320 0x7fb84415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb84259c640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440d2290 0x7fb8440cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 -- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440d2290 msgr2=0x7fb8440cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440d2290 0x7fb8440cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 -- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb84415e320 msgr2=0x7fb84415f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb84415e320 0x7fb84415f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 -- 172.21.15.113:0/2661757762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb844154ab0 con 0x7fb8440d0b00 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb842d9d640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8440d0b00 0x7fb84415a300 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb8340098c0 tx=0x7fb8340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb82b7fe640 1 -- 172.21.15.113:0/2661757762 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834041020 con 0x7fb8440d0b00 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb82b7fe640 1 -- 172.21.15.113:0/2661757762 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb834030970 con 0x7fb8440d0b00 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb82b7fe640 1 -- 172.21.15.113:0/2661757762 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834037040 con 0x7fb8440d0b00 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb844154d50 con 0x7fb8440d0b00 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.667+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8440cce50 con 0x7fb8440d0b00 2024-04-25T00:07:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.668+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8440d1000 con 0x7fb8440d0b00 2024-04-25T00:07:03.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.669+0000 7fb82b7fe640 1 -- 172.21.15.113:0/2661757762 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb834044020 con 0x7fb8440d0b00 2024-04-25T00:07:03.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.669+0000 7fb82b7fe640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb83006be30 0x7fb83006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:03.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.669+0000 7fb82b7fe640 1 -- 172.21.15.113:0/2661757762 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7fb8340ab720 con 0x7fb8440d0b00 2024-04-25T00:07:03.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.669+0000 7fb84259c640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb83006be30 0x7fb83006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:03.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.670+0000 7fb84259c640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb83006be30 0x7fb83006e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb838007f70 tx=0x7fb838007f00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:03.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.674+0000 7fb82b7fe640 1 -- 172.21.15.113:0/2661757762 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb8440d1000 con 0x7fb8440d0b00 2024-04-25T00:07:03.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.882+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8440691a0 con 0x7fb8440d0b00 2024-04-25T00:07:03.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.883+0000 7fb82b7fe640 1 -- 172.21.15.113:0/2661757762 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+4367 (secure 0 0 0) 0x7fb83404b030 con 0x7fb8440d0b00 2024-04-25T00:07:03.884 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:03.884 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:07:02.919908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":15,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":9}]} 2024-04-25T00:07:03.884 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb83006be30 msgr2=0x7fb83006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb83006be30 0x7fb83006e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb838007f70 tx=0x7fb838007f00 comp rx=0 tx=0).stop 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8440d0b00 msgr2=0x7fb84415a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8440d0b00 0x7fb84415a300 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb8340098c0 tx=0x7fb8340046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 shutdown_connections 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb83006be30 0x7fb83006e2f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb84415e320 0x7fb84415f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440d2290 0x7fb8440cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 --2- 172.21.15.113:0/2661757762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8440d0b00 0x7fb84415a300 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 >> 172.21.15.113:0/2661757762 conn(0x7fb8440b5c20 msgr2=0x7fb8440b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:03.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 shutdown_connections 2024-04-25T00:07:03.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.885+0000 7fb849114640 1 -- 172.21.15.113:0/2661757762 wait complete. 2024-04-25T00:07:03.896 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T00:07:03.897 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.993+0000 7f557ed5c640 1 Processor -- start 2024-04-25T00:07:03.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.993+0000 7f557ed5c640 1 -- start start 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.993+0000 7f557ed5c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f557815e320 0x7f55780d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.993+0000 7f557ed5c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 0x7f55780b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.993+0000 7f557ed5c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55780d3de0 con 0x7f55780d0b00 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.993+0000 7f557ed5c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f55780d3410 con 0x7f557815e320 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.993+0000 7f557ed5c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f55780d2800 con 0x7f55780d2290 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557caf8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f557815e320 0x7f55780d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557d2f9640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f55780d0b00 0x7f55780b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48454/0 (socket says 172.21.15.113:48454) 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557d2f9640 1 -- 172.21.15.113:0/2211790597 learned_addr learned my addr 172.21.15.113:0/2211790597 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f5577fff640 1 --2- 172.21.15.113:0/2211790597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 0x7f55780b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557caf8640 1 -- 172.21.15.113:0/2211790597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 msgr2=0x7f55780b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:03.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557caf8640 1 --2- 172.21.15.113:0/2211790597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 0x7f55780b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557caf8640 1 -- 172.21.15.113:0/2211790597 >> v1:172.21.15.113:6789/0 conn(0x7f55780d0b00 legacy=0x7f55780b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557caf8640 1 -- 172.21.15.113:0/2211790597 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55780d2ae0 con 0x7f557815e320 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f55777fe640 1 -- 172.21.15.113:0/2211790597 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024218833 0 0) 0x7f55780d3de0 con 0x7f55780d0b00 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.994+0000 7f557caf8640 1 --2- 172.21.15.113:0/2211790597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f557815e320 0x7f55780d38a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f556c0097a0 tx=0x7f556c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b83eadf55549dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f55777fe640 1 -- 172.21.15.113:0/2211790597 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c037070 con 0x7f557815e320 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f55777fe640 1 -- 172.21.15.113:0/2211790597 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556c032440 con 0x7f557815e320 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f55777fe640 1 -- 172.21.15.113:0/2211790597 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c032810 con 0x7f557815e320 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 -- 172.21.15.113:0/2211790597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f557815e320 msgr2=0x7f55780d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:03.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 --2- 172.21.15.113:0/2211790597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f557815e320 0x7f55780d38a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f556c0097a0 tx=0x7f556c02a210 comp rx=0 tx=0).stop 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 -- 172.21.15.113:0/2211790597 shutdown_connections 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 --2- 172.21.15.113:0/2211790597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 0x7f55780b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 --2- 172.21.15.113:0/2211790597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f557815e320 0x7f55780d38a0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 -- 172.21.15.113:0/2211790597 >> 172.21.15.113:0/2211790597 conn(0x7f55780b5c20 msgr2=0x7f55780b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 -- 172.21.15.113:0/2211790597 shutdown_connections 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.995+0000 7f557ed5c640 1 -- 172.21.15.113:0/2211790597 wait complete. 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.996+0000 7f557ed5c640 1 Processor -- start 2024-04-25T00:07:03.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.996+0000 7f557ed5c640 1 -- start start 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557ed5c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55780d0b00 0x7f557814e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557ed5c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 0x7f55780cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557ed5c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f557815e320 0x7f55780b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557ed5c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f55780b9fe0 con 0x7f55780d0b00 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557ed5c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55780d30a0 con 0x7f557815e320 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557ed5c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f557815e700 con 0x7f55780d2290 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557caf8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55780d0b00 0x7f557814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557caf8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55780d0b00 0x7f557814e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55194/0 (socket says 172.21.15.113:55194) 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.997+0000 7f557caf8640 1 -- 172.21.15.113:0/509947666 learned_addr learned my addr 172.21.15.113:0/509947666 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557caf8640 1 -- 172.21.15.113:0/509947666 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 msgr2=0x7f55780cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557caf8640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 0x7f55780cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557caf8640 1 -- 172.21.15.113:0/509947666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f557815e320 msgr2=0x7f55780b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557d2f9640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f557815e320 0x7f55780b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557caf8640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f557815e320 0x7f55780b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557caf8640 1 -- 172.21.15.113:0/509947666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55780cdec0 con 0x7f55780d0b00 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557caf8640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55780d0b00 0x7f557814e800 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f556c0098c0 tx=0x7f556c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f55757fa640 1 -- 172.21.15.113:0/509947666 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c037040 con 0x7f55780d0b00 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f55757fa640 1 -- 172.21.15.113:0/509947666 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556c030770 con 0x7f55780d0b00 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55781544f0 con 0x7f55780d0b00 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f55757fa640 1 -- 172.21.15.113:0/509947666 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c030b40 con 0x7f55780d0b00 2024-04-25T00:07:04.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.998+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5578154fd0 con 0x7f55780d0b00 2024-04-25T00:07:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.999+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55780691a0 con 0x7f55780d0b00 2024-04-25T00:07:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:03.999+0000 7f55757fa640 1 -- 172.21.15.113:0/509947666 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f556c044020 con 0x7f55780d0b00 2024-04-25T00:07:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.000+0000 7f55757fa640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f555c06be60 0x7f555c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.000+0000 7f55757fa640 1 -- 172.21.15.113:0/509947666 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f556c0ab3b0 con 0x7f55780d0b00 2024-04-25T00:07:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.000+0000 7f5577fff640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f555c06be60 0x7f555c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.000+0000 7f5577fff640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f555c06be60 0x7f555c06e320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f55780cd1a0 tx=0x7f5564008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.004+0000 7f55757fa640 1 -- 172.21.15.113:0/509947666 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f556c032e20 con 0x7f55780d0b00 2024-04-25T00:07:04.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.212+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55780b3260 con 0x7f55780d0b00 2024-04-25T00:07:04.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.212+0000 7f55757fa640 1 -- 172.21.15.113:0/509947666 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+4367 (secure 0 0 0) 0x7f556c079cd0 con 0x7f55780d0b00 2024-04-25T00:07:04.213 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:04.213 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:07:02.919908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":15,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":9}]} 2024-04-25T00:07:04.213 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-04-25T00:07:04.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f555c06be60 msgr2=0x7f555c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f555c06be60 0x7f555c06e320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f55780cd1a0 tx=0x7f5564008040 comp rx=0 tx=0).stop 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55780d0b00 msgr2=0x7f557814e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55780d0b00 0x7f557814e800 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f556c0098c0 tx=0x7f556c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 shutdown_connections 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f555c06be60 0x7f555c06e320 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f557815e320 0x7f55780b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55780d2290 0x7f55780cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 --2- 172.21.15.113:0/509947666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55780d0b00 0x7f557814e800 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 >> 172.21.15.113:0/509947666 conn(0x7f55780b5c20 msgr2=0x7f55780b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 shutdown_connections 2024-04-25T00:07:04.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.214+0000 7f557ed5c640 1 -- 172.21.15.113:0/509947666 wait complete. 2024-04-25T00:07:04.226 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T00:07:04.226 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:07:04.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.322+0000 7f040e753640 1 Processor -- start 2024-04-25T00:07:04.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.322+0000 7f040e753640 1 -- start start 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.322+0000 7f040e753640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0408144d60 0x7f04080b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.322+0000 7f040e753640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f040806e6e0 0x7f0408071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.322+0000 7f040e753640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04080cfad0 con 0x7f04081453c0 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.322+0000 7f040e753640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0408145c40 con 0x7f0408144d60 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.322+0000 7f040e753640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f04080cf230 con 0x7f040806e6e0 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f0407fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0408144d60 0x7f04080b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f0407fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0408144d60 0x7f04080b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55198/0 (socket says 172.21.15.113:55198) 2024-04-25T00:07:04.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f0407fff640 1 -- 172.21.15.113:0/2734991616 learned_addr learned my addr 172.21.15.113:0/2734991616 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f040ccf0640 1 --2- 172.21.15.113:0/2734991616 >> v2:172.21.15.178:3300/0 conn(0x7f040806e6e0 0x7f0408071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f0407fff640 1 -- 172.21.15.113:0/2734991616 >> v2:172.21.15.178:3300/0 conn(0x7f040806e6e0 msgr2=0x7f0408071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f0407fff640 1 --2- 172.21.15.113:0/2734991616 >> v2:172.21.15.178:3300/0 conn(0x7f040806e6e0 0x7f0408071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f0407fff640 1 -- 172.21.15.113:0/2734991616 >> v1:172.21.15.178:6789/0 conn(0x7f04081453c0 legacy=0x7f040806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.323+0000 7f0407fff640 1 -- 172.21.15.113:0/2734991616 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04080cf510 con 0x7f0408144d60 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f0407fff640 1 --2- 172.21.15.113:0/2734991616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f04080b5390 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0408145850 tx=0x7f03f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7287a6e805a0b4c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f0406ffd640 1 -- 172.21.15.113:0/2734991616 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8037070 con 0x7f0408144d60 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f0406ffd640 1 -- 172.21.15.113:0/2734991616 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f8032440 con 0x7f0408144d60 2024-04-25T00:07:04.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f0406ffd640 1 -- 172.21.15.113:0/2734991616 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8032810 con 0x7f0408144d60 2024-04-25T00:07:04.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 -- 172.21.15.113:0/2734991616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 msgr2=0x7f04080b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 --2- 172.21.15.113:0/2734991616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f04080b5390 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0408145850 tx=0x7f03f802a1a0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 -- 172.21.15.113:0/2734991616 shutdown_connections 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 --2- 172.21.15.113:0/2734991616 >> v2:172.21.15.178:3300/0 conn(0x7f040806e6e0 0x7f0408071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 --2- 172.21.15.113:0/2734991616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f04080b5390 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 -- 172.21.15.113:0/2734991616 >> 172.21.15.113:0/2734991616 conn(0x7f04080b1a50 msgr2=0x7f04080b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 -- 172.21.15.113:0/2734991616 shutdown_connections 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.324+0000 7f040e753640 1 -- 172.21.15.113:0/2734991616 wait complete. 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 Processor -- start 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 -- start start 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f040806e6e0 0x7f040813c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f040815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04081453c0 0x7f04080a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0408145140 con 0x7f0408144d60 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0408070e10 con 0x7f04081453c0 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.326+0000 7f040e753640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0408071870 con 0x7f040806e6e0 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f040815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f040815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55210/0 (socket says 172.21.15.113:55210) 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 -- 172.21.15.113:0/1400892442 learned_addr learned my addr 172.21.15.113:0/1400892442 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f0407fff640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f040806e6e0 0x7f040813c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f040ccf0640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04081453c0 0x7f04080a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 -- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f040806e6e0 msgr2=0x7f040813c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f040806e6e0 0x7f040813c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 -- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04081453c0 msgr2=0x7f04080a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04081453c0 0x7f04080a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 -- 172.21.15.113:0/1400892442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040815d180 con 0x7f0408144d60 2024-04-25T00:07:04.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f040ccf0640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04081453c0 0x7f04080a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:04.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f04077fe640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f040815ece0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f03f000b3f0 tx=0x7f03f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f0404ff9640 1 -- 172.21.15.113:0/1400892442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0015020 con 0x7f0408144d60 2024-04-25T00:07:04.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f0404ff9640 1 -- 172.21.15.113:0/1400892442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f0011830 con 0x7f0408144d60 2024-04-25T00:07:04.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040815d460 con 0x7f0408144d60 2024-04-25T00:07:04.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f0404ff9640 1 -- 172.21.15.113:0/1400892442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0011c20 con 0x7f0408144d60 2024-04-25T00:07:04.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.327+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040815df40 con 0x7f0408144d60 2024-04-25T00:07:04.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.328+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04080691a0 con 0x7f0408144d60 2024-04-25T00:07:04.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.329+0000 7f0404ff9640 1 -- 172.21.15.113:0/1400892442 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03f0018020 con 0x7f0408144d60 2024-04-25T00:07:04.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.329+0000 7f0404ff9640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f03e806be30 0x7f03e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.329+0000 7f0404ff9640 1 -- 172.21.15.113:0/1400892442 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f03f008ab10 con 0x7f0408144d60 2024-04-25T00:07:04.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.329+0000 7f0407fff640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f03e806be30 0x7f03e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.330+0000 7f0407fff640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f03e806be30 0x7f03e806e2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f03f8034000 tx=0x7f03f802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.334+0000 7f0404ff9640 1 -- 172.21.15.113:0/1400892442 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03f001f030 con 0x7f0408144d60 2024-04-25T00:07:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.572+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f04081407f0 con 0x7f0408144d60 2024-04-25T00:07:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.572+0000 7f0404ff9640 1 -- 172.21.15.113:0/1400892442 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1442 (secure 0 0 0) 0x7f03f0059430 con 0x7f0408144d60 2024-04-25T00:07:04.573 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:04.573 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":549,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076562958,"bytes_used":2546802688,"bytes_avail":770547310592,"bytes_total":773094113280,"read_bytes_sec":384913,"read_op_per_sec":2,"write_op_per_sec":0},"fsmap":{"epoch":72,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":8970}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T00:07:01.214371+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f03e806be30 msgr2=0x7f03e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f03e806be30 0x7f03e806e2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f03f8034000 tx=0x7f03f802e000 comp rx=0 tx=0).stop 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 msgr2=0x7f040815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f040815ece0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f03f000b3f0 tx=0x7f03f000b8b0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 shutdown_connections 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f03e806be30 0x7f03e806e2f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04081453c0 0x7f04080a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0408144d60 0x7f040815ece0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 --2- 172.21.15.113:0/1400892442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f040806e6e0 0x7f040813c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 >> 172.21.15.113:0/1400892442 conn(0x7f04080b1a50 msgr2=0x7f0408140d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:04.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 shutdown_connections 2024-04-25T00:07:04.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.574+0000 7f040e753640 1 -- 172.21.15.113:0/1400892442 wait complete. 2024-04-25T00:07:04.585 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T00:07:04.586 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:07:04.586 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:07:04.620 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:07:04.666 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:07:04.666 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:04.666 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:04.666 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:04.682 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:04.682 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:04.737 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:04.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.873+0000 7f4a9a8ee640 1 Processor -- start 2024-04-25T00:07:04.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.873+0000 7f4a9a8ee640 1 -- start start 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a9a8ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a94144910 0x7f4a940b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a9a8ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a940b5240 0x7f4a9406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a9a8ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a94144cf0 con 0x7f4a940b5680 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a9a8ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4a940b4510 con 0x7f4a940b5240 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a9a8ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4a940b2f30 con 0x7f4a94144910 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a937fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a940b5240 0x7f4a9406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a937fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a940b5240 0x7f4a9406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55218/0 (socket says 172.21.15.113:55218) 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.874+0000 7f4a937fe640 1 -- 172.21.15.113:0/2683754321 learned_addr learned my addr 172.21.15.113:0/2683754321 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.875+0000 7f4a937fe640 1 -- 172.21.15.113:0/2683754321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a94144910 msgr2=0x7f4a940b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.875+0000 7f4a93fff640 1 --2- 172.21.15.113:0/2683754321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a94144910 0x7f4a940b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.875+0000 7f4a937fe640 1 --2- 172.21.15.113:0/2683754321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a94144910 0x7f4a940b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.875+0000 7f4a937fe640 1 -- 172.21.15.113:0/2683754321 >> v1:172.21.15.178:6789/0 conn(0x7f4a940b5680 legacy=0x7f4a940cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:04.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.875+0000 7f4a937fe640 1 -- 172.21.15.113:0/2683754321 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a940b3210 con 0x7f4a940b5240 2024-04-25T00:07:04.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.875+0000 7f4a937fe640 1 --2- 172.21.15.113:0/2683754321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a9406dfd0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009870 tx=0x7f4a8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb7acc3ca82a8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.875+0000 7f4a92ffd640 1 -- 172.21.15.113:0/2683754321 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88037070 con 0x7f4a940b5240 2024-04-25T00:07:04.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a92ffd640 1 -- 172.21.15.113:0/2683754321 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a88032440 con 0x7f4a940b5240 2024-04-25T00:07:04.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a92ffd640 1 -- 172.21.15.113:0/2683754321 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88032810 con 0x7f4a940b5240 2024-04-25T00:07:04.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/2683754321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 msgr2=0x7f4a9406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/2683754321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a9406dfd0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009870 tx=0x7f4a8802a210 comp rx=0 tx=0).stop 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/2683754321 shutdown_connections 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/2683754321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a9406dfd0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/2683754321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a94144910 0x7f4a940b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/2683754321 >> 172.21.15.113:0/2683754321 conn(0x7f4a940b1a50 msgr2=0x7f4a940b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/2683754321 shutdown_connections 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.876+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/2683754321 wait complete. 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.877+0000 7f4a9a8ee640 1 Processor -- start 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.877+0000 7f4a9a8ee640 1 -- start start 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a9a8ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a940aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a9a8ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a940b5680 0x7f4a940abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a9a8ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a94144910 0x7f4a9415d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a9a8ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4a940b4da0 con 0x7f4a940b5240 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a9a8ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a940b37d0 con 0x7f4a94144910 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a9a8ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4a940b4f20 con 0x7f4a940b5680 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a940aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a940aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55232/0 (socket says 172.21.15.113:55232) 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a93fff640 1 -- 172.21.15.113:0/4032728824 learned_addr learned my addr 172.21.15.113:0/4032728824 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a937fe640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a940b5680 0x7f4a940abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a98e8b640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a94144910 0x7f4a9415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.878+0000 7f4a93fff640 1 -- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a940b5680 msgr2=0x7f4a940abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a93fff640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a940b5680 0x7f4a940abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a93fff640 1 -- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a94144910 msgr2=0x7f4a9415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a93fff640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a94144910 0x7f4a9415d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a93fff640 1 -- 172.21.15.113:0/4032728824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a940ac510 con 0x7f4a940b5240 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a93fff640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a940aeff0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c7f0 tx=0x7f4a8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a90ff9640 1 -- 172.21.15.113:0/4032728824 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84015070 con 0x7f4a940b5240 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a90ff9640 1 -- 172.21.15.113:0/4032728824 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a84004240 con 0x7f4a940b5240 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a940ac790 con 0x7f4a940b5240 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a90ff9640 1 -- 172.21.15.113:0/4032728824 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84007620 con 0x7f4a940b5240 2024-04-25T00:07:04.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.879+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a940a9700 con 0x7f4a940b5240 2024-04-25T00:07:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.880+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a940691a0 con 0x7f4a940b5240 2024-04-25T00:07:04.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.880+0000 7f4a90ff9640 1 -- 172.21.15.113:0/4032728824 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a840079d0 con 0x7f4a940b5240 2024-04-25T00:07:04.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.881+0000 7f4a90ff9640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a8006be30 0x7f4a8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.881+0000 7f4a90ff9640 1 -- 172.21.15.113:0/4032728824 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f4a8408afd0 con 0x7f4a940b5240 2024-04-25T00:07:04.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.881+0000 7f4a937fe640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a8006be30 0x7f4a8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.881+0000 7f4a937fe640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a8006be30 0x7f4a8006e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4a88034000 tx=0x7f4a8802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:04.885+0000 7f4a90ff9640 1 -- 172.21.15.113:0/4032728824 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a940ac510 con 0x7f4a940b5240 2024-04-25T00:07:05.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.062+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a941438e0 con 0x7f4a940b5240 2024-04-25T00:07:05.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.063+0000 7f4a90ff9640 1 -- 172.21.15.113:0/4032728824 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16900 (secure 0 0 0) 0x7f4a84059860 con 0x7f4a940b5240 2024-04-25T00:07:05.066 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:05.067 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":133,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:02.920311+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:06:18.255623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:06:19.003747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.12","mappings":[{"from":2,"to":0}]},{"pgid":"19.23","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6836/2664195523":"2024-04-26T00:06:59.857692+0000","172.21.15.113:6834/2664195523":"2024-04-26T00:06:59.857692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:05.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a8006be30 msgr2=0x7f4a8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a8006be30 0x7f4a8006e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4a88034000 tx=0x7f4a8802e000 comp rx=0 tx=0).stop 2024-04-25T00:07:05.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 msgr2=0x7f4a940aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a940aeff0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c7f0 tx=0x7f4a8400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 shutdown_connections 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a8006be30 0x7f4a8006e2f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a94144910 0x7f4a9415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a940b5680 0x7f4a940abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 --2- 172.21.15.113:0/4032728824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a940b5240 0x7f4a940aeff0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 >> 172.21.15.113:0/4032728824 conn(0x7f4a940b1a50 msgr2=0x7f4a940b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 shutdown_connections 2024-04-25T00:07:05.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.066+0000 7f4a9a8ee640 1 -- 172.21.15.113:0/4032728824 wait complete. 2024-04-25T00:07:05.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:06:18.255623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:06:19.003747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:07:05.078 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:07:05.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.178+0000 7f81e5a38640 1 Processor -- start 2024-04-25T00:07:05.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.178+0000 7f81e5a38640 1 -- start start 2024-04-25T00:07:05.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.178+0000 7f81e5a38640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81e0161fc0 0x7f81e00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.178+0000 7f81e5a38640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81e0162560 con 0x7f81e00cf3c0 2024-04-25T00:07:05.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.178+0000 7f81e5a38640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81e0162840 con 0x7f81e00cf7a0 2024-04-25T00:07:05.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.178+0000 7f81e5a38640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f81e0162b20 con 0x7f81e0161fc0 2024-04-25T00:07:05.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.179+0000 7f81deffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f81e00cf3c0 0x7f81e00d3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48470/0 (socket says 172.21.15.113:48470) 2024-04-25T00:07:05.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.179+0000 7f81df7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81e0161fc0 0x7f81e00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.179+0000 7f81deffd640 1 -- 172.21.15.113:0/866223980 learned_addr learned my addr 172.21.15.113:0/866223980 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:05.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.179+0000 7f81df7fe640 1 -- 172.21.15.113:0/866223980 >> v1:172.21.15.178:6789/0 conn(0x7f81e00cf7a0 legacy=0x7f81e00d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.179+0000 7f81df7fe640 1 -- 172.21.15.113:0/866223980 >> v1:172.21.15.113:6789/0 conn(0x7f81e00cf3c0 legacy=0x7f81e00d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.179+0000 7f81ddffb640 1 -- 172.21.15.113:0/866223980 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1135828843 0 0) 0x7f81e0162560 con 0x7f81e00cf3c0 2024-04-25T00:07:05.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.179+0000 7f81df7fe640 1 -- 172.21.15.113:0/866223980 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e0162e00 con 0x7f81e0161fc0 2024-04-25T00:07:05.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81df7fe640 1 --2- 172.21.15.113:0/866223980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e00d1210 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009d90 tx=0x7f81d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9073cded03dbeba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81ddffb640 1 -- 172.21.15.113:0/866223980 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8007740 con 0x7f81e0161fc0 2024-04-25T00:07:05.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81ddffb640 1 -- 172.21.15.113:0/866223980 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d80078e0 con 0x7f81e0161fc0 2024-04-25T00:07:05.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81ddffb640 1 -- 172.21.15.113:0/866223980 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8007cb0 con 0x7f81e0161fc0 2024-04-25T00:07:05.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81e5a38640 1 -- 172.21.15.113:0/866223980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 msgr2=0x7f81e00d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81e5a38640 1 --2- 172.21.15.113:0/866223980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e00d1210 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009d90 tx=0x7f81d802a660 comp rx=0 tx=0).stop 2024-04-25T00:07:05.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81e5a38640 1 -- 172.21.15.113:0/866223980 shutdown_connections 2024-04-25T00:07:05.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81e5a38640 1 --2- 172.21.15.113:0/866223980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e00d1210 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81e5a38640 1 -- 172.21.15.113:0/866223980 >> 172.21.15.113:0/866223980 conn(0x7f81e00718f0 msgr2=0x7f81e0071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81e5a38640 1 -- 172.21.15.113:0/866223980 shutdown_connections 2024-04-25T00:07:05.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.180+0000 7f81e5a38640 1 -- 172.21.15.113:0/866223980 wait complete. 2024-04-25T00:07:05.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.181+0000 7f81e5a38640 1 Processor -- start 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81e5a38640 1 -- start start 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81e5a38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81e00cf3c0 0x7f81e0151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81e5a38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e00cf7a0 0x7f81e00b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81e5a38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81e5a38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f81e00d18d0 con 0x7f81e0161fc0 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81e5a38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81e00d1750 con 0x7f81e00cf7a0 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81e5a38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81e00d1a50 con 0x7f81e00cf3c0 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e015a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55258/0 (socket says 172.21.15.113:55258) 2024-04-25T00:07:05.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.182+0000 7f81df7fe640 1 -- 172.21.15.113:0/4224813493 learned_addr learned my addr 172.21.15.113:0/4224813493 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81de7fc640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e00cf7a0 0x7f81e00b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81df7fe640 1 -- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81e00cf3c0 msgr2=0x7f81e0151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81deffd640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81e00cf3c0 0x7f81e0151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81df7fe640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81e00cf3c0 0x7f81e0151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81df7fe640 1 -- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e00cf7a0 msgr2=0x7f81e00b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81df7fe640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e00cf7a0 0x7f81e00b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81df7fe640 1 -- 172.21.15.113:0/4224813493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e00b8800 con 0x7f81e0161fc0 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81de7fc640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e00cf7a0 0x7f81e00b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81df7fe640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e015a830 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009eb0 tx=0x7f81d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81c7fff640 1 -- 172.21.15.113:0/4224813493 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8040070 con 0x7f81e0161fc0 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81c7fff640 1 -- 172.21.15.113:0/4224813493 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d8031c40 con 0x7f81e0161fc0 2024-04-25T00:07:05.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e00b8ae0 con 0x7f81e0161fc0 2024-04-25T00:07:05.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81c7fff640 1 -- 172.21.15.113:0/4224813493 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8003990 con 0x7f81e0161fc0 2024-04-25T00:07:05.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.183+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e00b90a0 con 0x7f81e0161fc0 2024-04-25T00:07:05.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.184+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e00691a0 con 0x7f81e0161fc0 2024-04-25T00:07:05.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.184+0000 7f81c7fff640 1 -- 172.21.15.113:0/4224813493 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81d8039040 con 0x7f81e0161fc0 2024-04-25T00:07:05.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.185+0000 7f81c7fff640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81cc06be30 0x7f81cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.185+0000 7f81c7fff640 1 -- 172.21.15.113:0/4224813493 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f81d80ac2e0 con 0x7f81e0161fc0 2024-04-25T00:07:05.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.185+0000 7f81deffd640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81cc06be30 0x7f81cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.185+0000 7f81deffd640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81cc06be30 0x7f81cc06e2f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f81d0007fd0 tx=0x7f81d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.189+0000 7f81c7fff640 1 -- 172.21.15.113:0/4224813493 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81d807ab80 con 0x7f81e0161fc0 2024-04-25T00:07:05.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.362+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81e00b6920 con 0x7f81e0161fc0 2024-04-25T00:07:05.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.362+0000 7f81c7fff640 1 -- 172.21.15.113:0/4224813493 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) ==== 93+0+10 (secure 0 0 0) 0x7f81d807a480 con 0x7f81e0161fc0 2024-04-25T00:07:05.363 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:07:05.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81cc06be30 msgr2=0x7f81cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81cc06be30 0x7f81cc06e2f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f81d0007fd0 tx=0x7f81d0009040 comp rx=0 tx=0).stop 2024-04-25T00:07:05.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 msgr2=0x7f81e015a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e015a830 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009eb0 tx=0x7f81d8004a40 comp rx=0 tx=0).stop 2024-04-25T00:07:05.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 shutdown_connections 2024-04-25T00:07:05.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f81cc06be30 0x7f81cc06e2f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e00cf7a0 0x7f81e00b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81e0161fc0 0x7f81e015a830 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 --2- 172.21.15.113:0/4224813493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81e00cf3c0 0x7f81e0151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 >> 172.21.15.113:0/4224813493 conn(0x7f81e00718f0 msgr2=0x7f81e015c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 shutdown_connections 2024-04-25T00:07:05.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.364+0000 7f81e5a38640 1 -- 172.21.15.113:0/4224813493 wait complete. 2024-04-25T00:07:05.376 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:07:05.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.475+0000 7f7efd1ee640 1 Processor -- start 2024-04-25T00:07:05.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.475+0000 7f7efd1ee640 1 -- start start 2024-04-25T00:07:05.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7efd1ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ef815b670 0x7f7ef815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7efd1ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7ef806cea0 0x7f7ef806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7efd1ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef8071170 con 0x7f7ef815bff0 2024-04-25T00:07:05.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7efd1ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef815c4a0 con 0x7f7ef815b670 2024-04-25T00:07:05.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7efd1ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef806d810 con 0x7f7ef806cea0 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef6d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ef815b670 0x7f7ef815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef6d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ef815b670 0x7f7ef815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55264/0 (socket says 172.21.15.113:55264) 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/1772302861 learned_addr learned my addr 172.21.15.113:0/1772302861 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef759e640 1 --2- 172.21.15.113:0/1772302861 >> v2:172.21.15.178:3300/0 conn(0x7f7ef806cea0 0x7f7ef806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/1772302861 >> v2:172.21.15.178:3300/0 conn(0x7f7ef806cea0 msgr2=0x7f7ef806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef6d9d640 1 --2- 172.21.15.113:0/1772302861 >> v2:172.21.15.178:3300/0 conn(0x7f7ef806cea0 0x7f7ef806d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/1772302861 >> v1:172.21.15.178:6789/0 conn(0x7f7ef815bff0 legacy=0x7f7ef80cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/1772302861 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef8071450 con 0x7f7ef815b670 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.476+0000 7f7ef759e640 1 --2- 172.21.15.113:0/1772302861 >> v2:172.21.15.178:3300/0 conn(0x7f7ef806cea0 0x7f7ef806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.477+0000 7f7ef6d9d640 1 --2- 172.21.15.113:0/1772302861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef815b670 0x7f7ef815ba50 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80cceb0 tx=0x7f7ee802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ed770bb891f5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.477+0000 7f7ef5d9b640 1 -- 172.21.15.113:0/1772302861 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8036070 con 0x7f7ef815b670 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.477+0000 7f7ef5d9b640 1 -- 172.21.15.113:0/1772302861 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8031600 con 0x7f7ef815b670 2024-04-25T00:07:05.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.477+0000 7f7ef5d9b640 1 -- 172.21.15.113:0/1772302861 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80319d0 con 0x7f7ef815b670 2024-04-25T00:07:05.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.477+0000 7f7efd1ee640 1 -- 172.21.15.113:0/1772302861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef815b670 msgr2=0x7f7ef815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.477+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/1772302861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef815b670 0x7f7ef815ba50 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80cceb0 tx=0x7f7ee802a5d0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.478+0000 7f7efd1ee640 1 -- 172.21.15.113:0/1772302861 shutdown_connections 2024-04-25T00:07:05.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.478+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/1772302861 >> v2:172.21.15.178:3300/0 conn(0x7f7ef806cea0 0x7f7ef806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.478+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/1772302861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef815b670 0x7f7ef815ba50 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.478+0000 7f7efd1ee640 1 -- 172.21.15.113:0/1772302861 >> 172.21.15.113:0/1772302861 conn(0x7f7ef806b0f0 msgr2=0x7f7ef806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.478+0000 7f7efd1ee640 1 -- 172.21.15.113:0/1772302861 shutdown_connections 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.478+0000 7f7efd1ee640 1 -- 172.21.15.113:0/1772302861 wait complete. 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.479+0000 7f7efd1ee640 1 Processor -- start 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.479+0000 7f7efd1ee640 1 -- start start 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7efd1ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef806cea0 0x7f7ef80cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7efd1ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ef815b670 0x7f7ef8151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7efd1ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ef815bff0 0x7f7ef80a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7efd1ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef80cc910 con 0x7f7ef806cea0 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7efd1ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8071a10 con 0x7f7ef815b670 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7efd1ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef80cca90 con 0x7f7ef815bff0 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef806cea0 0x7f7ef80cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef806cea0 0x7f7ef80cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55280/0 (socket says 172.21.15.113:55280) 2024-04-25T00:07:05.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/2150545209 learned_addr learned my addr 172.21.15.113:0/2150545209 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef659c640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ef815b670 0x7f7ef8151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef759e640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ef815bff0 0x7f7ef80a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ef815bff0 msgr2=0x7f7ef80a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ef815bff0 0x7f7ef80a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ef815b670 msgr2=0x7f7ef8151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ef815b670 0x7f7ef8151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 -- 172.21.15.113:0/2150545209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef80ca8a0 con 0x7f7ef806cea0 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef759e640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ef815bff0 0x7f7ef80a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7ef6d9d640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef806cea0 0x7f7ef80cb940 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8034040 tx=0x7f7ee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7edf7fe640 1 -- 172.21.15.113:0/2150545209 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8036050 con 0x7f7ef806cea0 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7edf7fe640 1 -- 172.21.15.113:0/2150545209 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8042070 con 0x7f7ef806cea0 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7edf7fe640 1 -- 172.21.15.113:0/2150545209 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8009510 con 0x7f7ef806cea0 2024-04-25T00:07:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.480+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef80cab80 con 0x7f7ef806cea0 2024-04-25T00:07:05.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.481+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef80cb140 con 0x7f7ef806cea0 2024-04-25T00:07:05.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.482+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef80691a0 con 0x7f7ef806cea0 2024-04-25T00:07:05.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.482+0000 7f7edf7fe640 1 -- 172.21.15.113:0/2150545209 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ee8007c50 con 0x7f7ef806cea0 2024-04-25T00:07:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.482+0000 7f7edf7fe640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ee406be60 0x7f7ee406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.482+0000 7f7edf7fe640 1 -- 172.21.15.113:0/2150545209 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f7ee80abea0 con 0x7f7ef806cea0 2024-04-25T00:07:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.482+0000 7f7ef659c640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ee406be60 0x7f7ee406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.483+0000 7f7ef659c640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ee406be60 0x7f7ee406e320 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006fd0 tx=0x7f7eec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.487+0000 7f7edf7fe640 1 -- 172.21.15.113:0/2150545209 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ee807a7c0 con 0x7f7ef806cea0 2024-04-25T00:07:05.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.660+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ef80cb560 con 0x7f7ef806cea0 2024-04-25T00:07:05.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.661+0000 7f7edf7fe640 1 -- 172.21.15.113:0/2150545209 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) ==== 104+0+11 (secure 0 0 0) 0x7f7ee807a0c0 con 0x7f7ef806cea0 2024-04-25T00:07:05.661 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ee406be60 msgr2=0x7f7ee406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ee406be60 0x7f7ee406e320 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006fd0 tx=0x7f7eec008040 comp rx=0 tx=0).stop 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef806cea0 msgr2=0x7f7ef80cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef806cea0 0x7f7ef80cb940 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8034040 tx=0x7f7ee8004a40 comp rx=0 tx=0).stop 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 shutdown_connections 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ee406be60 0x7f7ee406e320 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ef815b670 0x7f7ef8151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ef815bff0 0x7f7ef80a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 --2- 172.21.15.113:0/2150545209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ef806cea0 0x7f7ef80cb940 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 >> 172.21.15.113:0/2150545209 conn(0x7f7ef806b0f0 msgr2=0x7f7ef80cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 shutdown_connections 2024-04-25T00:07:05.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.663+0000 7f7efd1ee640 1 -- 172.21.15.113:0/2150545209 wait complete. 2024-04-25T00:07:05.675 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:07:05.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.773+0000 7f8e908cd640 1 Processor -- start 2024-04-25T00:07:05.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.773+0000 7f8e908cd640 1 -- start start 2024-04-25T00:07:05.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e908cd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8e8c15e2d0 0x7f8e8c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e908cd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b4a40 0x7f8e8c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e908cd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e8c15f4f0 con 0x7f8e8c0b3080 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e908cd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8c0cde00 con 0x7f8e8c15e2d0 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e908cd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8c15f7d0 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e8a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8e8c15e2d0 0x7f8e8c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e89d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8e8c0b3080 0x7f8e8c0b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48482/0 (socket says 172.21.15.113:48482) 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e89d9b640 1 -- 172.21.15.113:0/37262573 learned_addr learned my addr 172.21.15.113:0/37262573 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e8ad9d640 1 --2- 172.21.15.113:0/37262573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b4a40 0x7f8e8c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e8a59c640 1 -- 172.21.15.113:0/37262573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b4a40 msgr2=0x7f8e8c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e8a59c640 1 --2- 172.21.15.113:0/37262573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b4a40 0x7f8e8c0ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e8959a640 1 -- 172.21.15.113:0/37262573 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3473397217 0 0) 0x7f8e8c15f4f0 con 0x7f8e8c0b3080 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e8a59c640 1 -- 172.21.15.113:0/37262573 >> v1:172.21.15.113:6789/0 conn(0x7f8e8c0b3080 legacy=0x7f8e8c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.774+0000 7f8e8a59c640 1 -- 172.21.15.113:0/37262573 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c15fab0 con 0x7f8e8c15e2d0 2024-04-25T00:07:05.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.775+0000 7f8e8a59c640 1 --2- 172.21.15.113:0/37262573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c15e2d0 0x7f8e8c15ebf0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009d90 tx=0x7f8e7c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f5c9efd5665f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.775+0000 7f8e8959a640 1 -- 172.21.15.113:0/37262573 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c007740 con 0x7f8e8c15e2d0 2024-04-25T00:07:05.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.775+0000 7f8e8959a640 1 -- 172.21.15.113:0/37262573 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0078e0 con 0x7f8e8c15e2d0 2024-04-25T00:07:05.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.775+0000 7f8e8959a640 1 -- 172.21.15.113:0/37262573 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c007cb0 con 0x7f8e8c15e2d0 2024-04-25T00:07:05.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 -- 172.21.15.113:0/37262573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c15e2d0 msgr2=0x7f8e8c15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 --2- 172.21.15.113:0/37262573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c15e2d0 0x7f8e8c15ebf0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009d90 tx=0x7f8e7c02a660 comp rx=0 tx=0).stop 2024-04-25T00:07:05.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 -- 172.21.15.113:0/37262573 shutdown_connections 2024-04-25T00:07:05.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 --2- 172.21.15.113:0/37262573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b4a40 0x7f8e8c0ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 --2- 172.21.15.113:0/37262573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c15e2d0 0x7f8e8c15ebf0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 -- 172.21.15.113:0/37262573 >> 172.21.15.113:0/37262573 conn(0x7f8e8c06b0f0 msgr2=0x7f8e8c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 -- 172.21.15.113:0/37262573 shutdown_connections 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.776+0000 7f8e908cd640 1 -- 172.21.15.113:0/37262573 wait complete. 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.777+0000 7f8e908cd640 1 Processor -- start 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.777+0000 7f8e908cd640 1 -- start start 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e908cd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b3080 0x7f8e8c144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e908cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0b4a40 0x7f8e8c1462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e908cd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c15e2d0 0x7f8e8c0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e908cd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c15f2b0 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e908cd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c15f130 con 0x7f8e8c15e2d0 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e908cd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8c15e6b0 con 0x7f8e8c0b3080 2024-04-25T00:07:05.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0b4a40 0x7f8e8c1462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0b4a40 0x7f8e8c1462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55300/0 (socket says 172.21.15.113:55300) 2024-04-25T00:07:05.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 -- 172.21.15.113:0/4222533321 learned_addr learned my addr 172.21.15.113:0/4222533321 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:05.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e8ad9d640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c15e2d0 0x7f8e8c0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e8a59c640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b3080 0x7f8e8c144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 -- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b3080 msgr2=0x7f8e8c144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b3080 0x7f8e8c144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 -- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c15e2d0 msgr2=0x7f8e8c0b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c15e2d0 0x7f8e8c0b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.778+0000 7f8e89d9b640 1 -- 172.21.15.113:0/4222533321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c0b1b70 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.779+0000 7f8e89d9b640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0b4a40 0x7f8e8c1462c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000c4d0 tx=0x7f8e8000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.779+0000 7f8e7affd640 1 -- 172.21.15.113:0/4222533321 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80018070 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.779+0000 7f8e7affd640 1 -- 172.21.15.113:0/4222533321 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80007500 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.779+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c0b1e50 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.779+0000 7f8e7affd640 1 -- 172.21.15.113:0/4222533321 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80004a20 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.779+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1684f0 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.780+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8c0ccc30 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.780+0000 7f8e7affd640 1 -- 172.21.15.113:0/4222533321 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e8001b070 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.780+0000 7f8e7affd640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.781+0000 7f8e8a59c640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.781+0000 7f8e7affd640 1 -- 172.21.15.113:0/4222533321 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f8e8008a100 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.781+0000 7f8e8a59c640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c034000 tx=0x7f8e7c02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.785+0000 7f8e7affd640 1 -- 172.21.15.113:0/4222533321 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e8c0ccc30 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.957+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e8c0691a0 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.957+0000 7f8e7affd640 1 -- 172.21.15.113:0/4222533321 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) ==== 100+0+11 (secure 0 0 0) 0x7f8e80058460 con 0x7f8e8c0b4a40 2024-04-25T00:07:05.958 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6c06be30 msgr2=0x7f8e6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c034000 tx=0x7f8e7c02f000 comp rx=0 tx=0).stop 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0b4a40 msgr2=0x7f8e8c1462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0b4a40 0x7f8e8c1462c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000c4d0 tx=0x7f8e8000c990 comp rx=0 tx=0).stop 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 shutdown_connections 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c15e2d0 0x7f8e8c0b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0b4a40 0x7f8e8c1462c0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 --2- 172.21.15.113:0/4222533321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e8c0b3080 0x7f8e8c144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 >> 172.21.15.113:0/4222533321 conn(0x7f8e8c06b0f0 msgr2=0x7f8e8c1542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 shutdown_connections 2024-04-25T00:07:05.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:05.959+0000 7f8e908cd640 1 -- 172.21.15.113:0/4222533321 wait complete. 2024-04-25T00:07:05.961 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:07:05.960+0000 7f41e6653640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:07:05.971 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:07:06.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.067+0000 7fd3900a5640 1 Processor -- start 2024-04-25T00:07:06.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.067+0000 7fd3900a5640 1 -- start start 2024-04-25T00:07:06.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd3900a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd38815e2d0 0x7fd38815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd3900a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3880b3080 0x7fd3880b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd3900a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd38815f4f0 con 0x7fd38815e6b0 2024-04-25T00:07:06.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd3900a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd3880cde00 con 0x7fd3880b3080 2024-04-25T00:07:06.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd3900a5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd38815f7d0 con 0x7fd38815e2d0 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd38d640640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3880b3080 0x7fd3880b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd38e642640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd38815e6b0 0x7fd388071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48492/0 (socket says 172.21.15.113:48492) 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.068+0000 7fd38e642640 1 -- 172.21.15.113:0/3466952842 learned_addr learned my addr 172.21.15.113:0/3466952842 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38d640640 1 -- 172.21.15.113:0/3466952842 >> v2:172.21.15.178:3300/0 conn(0x7fd38815e2d0 msgr2=0x7fd38815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38de41640 1 --2- 172.21.15.113:0/3466952842 >> v2:172.21.15.178:3300/0 conn(0x7fd38815e2d0 0x7fd38815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38d640640 1 --2- 172.21.15.113:0/3466952842 >> v2:172.21.15.178:3300/0 conn(0x7fd38815e2d0 0x7fd38815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38ce3f640 1 -- 172.21.15.113:0/3466952842 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614142574 0 0) 0x7fd38815f4f0 con 0x7fd38815e6b0 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38d640640 1 -- 172.21.15.113:0/3466952842 >> v1:172.21.15.113:6789/0 conn(0x7fd38815e6b0 legacy=0x7fd388071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38d640640 1 -- 172.21.15.113:0/3466952842 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38815fab0 con 0x7fd3880b3080 2024-04-25T00:07:06.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38d640640 1 --2- 172.21.15.113:0/3466952842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd3880b4330 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fd3880b2d40 tx=0x7fd37802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3685f7e45257e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38ce3f640 1 -- 172.21.15.113:0/3466952842 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378037070 con 0x7fd3880b3080 2024-04-25T00:07:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38ce3f640 1 -- 172.21.15.113:0/3466952842 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd378032440 con 0x7fd3880b3080 2024-04-25T00:07:06.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.069+0000 7fd38ce3f640 1 -- 172.21.15.113:0/3466952842 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378032810 con 0x7fd3880b3080 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 -- 172.21.15.113:0/3466952842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 msgr2=0x7fd3880b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 --2- 172.21.15.113:0/3466952842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd3880b4330 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fd3880b2d40 tx=0x7fd37802a1a0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 -- 172.21.15.113:0/3466952842 shutdown_connections 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 --2- 172.21.15.113:0/3466952842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd3880b4330 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 --2- 172.21.15.113:0/3466952842 >> v2:172.21.15.178:3300/0 conn(0x7fd38815e2d0 0x7fd38815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 -- 172.21.15.113:0/3466952842 >> 172.21.15.113:0/3466952842 conn(0x7fd38806b0f0 msgr2=0x7fd38806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 -- 172.21.15.113:0/3466952842 shutdown_connections 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.070+0000 7fd3900a5640 1 -- 172.21.15.113:0/3466952842 wait complete. 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.071+0000 7fd3900a5640 1 Processor -- start 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.071+0000 7fd3900a5640 1 -- start start 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd3900a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd38814ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd3900a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd38815e2d0 0x7fd388159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd3900a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd38815e6b0 0x7fd38815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd3900a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd38815f2b0 con 0x7fd3880b3080 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd3900a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd38815f130 con 0x7fd38815e6b0 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd3900a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3880cf2a0 con 0x7fd38815e2d0 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd38814ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd38814ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55320/0 (socket says 172.21.15.113:55320) 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 -- 172.21.15.113:0/2415298431 learned_addr learned my addr 172.21.15.113:0/2415298431 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 -- 172.21.15.113:0/2415298431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd38815e2d0 msgr2=0x7fd388159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd38815e2d0 0x7fd388159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 -- 172.21.15.113:0/2415298431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd38815e6b0 msgr2=0x7fd38815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd38815e6b0 0x7fd38815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 -- 172.21.15.113:0/2415298431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3880ada30 con 0x7fd3880b3080 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd38de41640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd38814ba30 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fd38400ce10 tx=0x7fd38400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd37e7fc640 1 -- 172.21.15.113:0/2415298431 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384003b90 con 0x7fd3880b3080 2024-04-25T00:07:06.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd37e7fc640 1 -- 172.21.15.113:0/2415298431 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd384003d30 con 0x7fd3880b3080 2024-04-25T00:07:06.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3880add10 con 0x7fd3880b3080 2024-04-25T00:07:06.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.072+0000 7fd37e7fc640 1 -- 172.21.15.113:0/2415298431 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384004240 con 0x7fd3880b3080 2024-04-25T00:07:06.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.073+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3880ae2d0 con 0x7fd3880b3080 2024-04-25T00:07:06.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.073+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3880691a0 con 0x7fd3880b3080 2024-04-25T00:07:06.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.074+0000 7fd37e7fc640 1 -- 172.21.15.113:0/2415298431 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3840043e0 con 0x7fd3880b3080 2024-04-25T00:07:06.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.074+0000 7fd37e7fc640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd36c06be30 0x7fd36c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.074+0000 7fd37e7fc640 1 -- 172.21.15.113:0/2415298431 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7fd38408ba60 con 0x7fd3880b3080 2024-04-25T00:07:06.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.074+0000 7fd38d640640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd36c06be30 0x7fd36c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.075+0000 7fd38d640640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd36c06be30 0x7fd36c06e2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd378034000 tx=0x7fd37802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.078+0000 7fd37e7fc640 1 -- 172.21.15.113:0/2415298431 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd38401d340 con 0x7fd3880b3080 2024-04-25T00:07:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.263+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd388072050 con 0x7fd3880b3080 2024-04-25T00:07:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.263+0000 7fd37e7fc640 1 -- 172.21.15.113:0/2415298431 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v133) ==== 80+0+124 (secure 0 0 0) 0x7fd38405a380 con 0x7fd3880b3080 2024-04-25T00:07:06.264 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/2664195523 2024-04-26T00:06:59.857692+0000 2024-04-25T00:07:06.264 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/2664195523 2024-04-26T00:06:59.857692+0000 2024-04-25T00:07:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:listed 2 entries 2024-04-25T00:07:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd36c06be30 msgr2=0x7fd36c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd36c06be30 0x7fd36c06e2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd378034000 tx=0x7fd37802c000 comp rx=0 tx=0).stop 2024-04-25T00:07:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 msgr2=0x7fd38814ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd38814ba30 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fd38400ce10 tx=0x7fd38400a660 comp rx=0 tx=0).stop 2024-04-25T00:07:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 shutdown_connections 2024-04-25T00:07:06.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd36c06be30 0x7fd36c06e2f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd38815e6b0 0x7fd38815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd38815e2d0 0x7fd388159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.265+0000 7fd3900a5640 1 --2- 172.21.15.113:0/2415298431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3880b3080 0x7fd38814ba30 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.266+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 >> 172.21.15.113:0/2415298431 conn(0x7fd38806b0f0 msgr2=0x7fd388158890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:06.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.266+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 shutdown_connections 2024-04-25T00:07:06.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.266+0000 7fd3900a5640 1 -- 172.21.15.113:0/2415298431 wait complete. 2024-04-25T00:07:06.277 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:07:06.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:06.277 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:07:06.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:07:06.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:06.304 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:07:06.359 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:06.359 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:07:06.359 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:06.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:07:06.379 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:06.379 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:06.379 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:06.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:07:06.417 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:06.417 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:07:06.417 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:06.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:07:06.437 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:06.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.554+0000 7f129fc38640 1 Processor -- start 2024-04-25T00:07:06.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.554+0000 7f129fc38640 1 -- start start 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129fc38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f12980b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129fc38640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f129806e6e0 0x7f12980cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129fc38640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1298144cf0 con 0x7f12980b5240 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129fc38640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f12980b4510 con 0x7f129806e6e0 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129fc38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12980b2f30 con 0x7f1298144910 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129e1d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f129806e6e0 0x7f12980cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129e1d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f129806e6e0 0x7f12980cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55334/0 (socket says 172.21.15.113:55334) 2024-04-25T00:07:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129e1d5640 1 -- 172.21.15.113:0/2751061778 learned_addr learned my addr 172.21.15.113:0/2751061778 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129d9d4640 1 --2- 172.21.15.113:0/2751061778 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f12980b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129e1d5640 1 -- 172.21.15.113:0/2751061778 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 msgr2=0x7f12980b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129e1d5640 1 --2- 172.21.15.113:0/2751061778 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f12980b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129e1d5640 1 -- 172.21.15.113:0/2751061778 >> v1:172.21.15.178:6789/0 conn(0x7f12980b5240 legacy=0x7f129806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129e1d5640 1 -- 172.21.15.113:0/2751061778 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12980b3210 con 0x7f129806e6e0 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.555+0000 7f129d9d4640 1 --2- 172.21.15.113:0/2751061778 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f12980b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129e1d5640 1 --2- 172.21.15.113:0/2751061778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f129806e6e0 0x7f12980cd500 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f1290009870 tx=0x7f129002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeb66725d79ea84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129c9d2640 1 -- 172.21.15.113:0/2751061778 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290037070 con 0x7f129806e6e0 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129c9d2640 1 -- 172.21.15.113:0/2751061778 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1290032440 con 0x7f129806e6e0 2024-04-25T00:07:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129c9d2640 1 -- 172.21.15.113:0/2751061778 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290032810 con 0x7f129806e6e0 2024-04-25T00:07:06.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 -- 172.21.15.113:0/2751061778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f129806e6e0 msgr2=0x7f12980cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 --2- 172.21.15.113:0/2751061778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f129806e6e0 0x7f12980cd500 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f1290009870 tx=0x7f129002a210 comp rx=0 tx=0).stop 2024-04-25T00:07:06.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 -- 172.21.15.113:0/2751061778 shutdown_connections 2024-04-25T00:07:06.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 --2- 172.21.15.113:0/2751061778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f129806e6e0 0x7f12980cd500 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 --2- 172.21.15.113:0/2751061778 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f12980b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 -- 172.21.15.113:0/2751061778 >> 172.21.15.113:0/2751061778 conn(0x7f12980b1a50 msgr2=0x7f12980b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 -- 172.21.15.113:0/2751061778 shutdown_connections 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.556+0000 7f129fc38640 1 -- 172.21.15.113:0/2751061778 wait complete. 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 Processor -- start 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 -- start start 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f129806e6e0 0x7f129813be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980b5240 0x7f1298142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f1298142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12980b4da0 con 0x7f12980b5240 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12980b37d0 con 0x7f129806e6e0 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129fc38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12980b4f20 con 0x7f1298144910 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129d1d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980b5240 0x7f1298142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129d1d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980b5240 0x7f1298142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55340/0 (socket says 172.21.15.113:55340) 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.558+0000 7f129d1d3640 1 -- 172.21.15.113:0/3887055234 learned_addr learned my addr 172.21.15.113:0/3887055234 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129d9d4640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f129806e6e0 0x7f129813be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129e1d5640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f1298142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129d1d3640 1 -- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 msgr2=0x7f1298142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129d1d3640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f1298142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129d1d3640 1 -- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f129806e6e0 msgr2=0x7f129813be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129d1d3640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f129806e6e0 0x7f129813be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129d1d3640 1 -- 172.21.15.113:0/3887055234 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12981f1f40 con 0x7f12980b5240 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129d1d3640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980b5240 0x7f1298142780 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f128800c460 tx=0x7f128800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f128e7fc640 1 -- 172.21.15.113:0/3887055234 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288016020 con 0x7f12980b5240 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f128e7fc640 1 -- 172.21.15.113:0/3887055234 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1288012870 con 0x7f12980b5240 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12981f21e0 con 0x7f12980b5240 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f128e7fc640 1 -- 172.21.15.113:0/3887055234 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288012c60 con 0x7f12980b5240 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.559+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981f27a0 con 0x7f12980b5240 2024-04-25T00:07:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.560+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12980691a0 con 0x7f12980b5240 2024-04-25T00:07:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.561+0000 7f128e7fc640 1 -- 172.21.15.113:0/3887055234 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1288019020 con 0x7f12980b5240 2024-04-25T00:07:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.561+0000 7f128e7fc640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f127c06be30 0x7f127c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.561+0000 7f128e7fc640 1 -- 172.21.15.113:0/3887055234 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f128808a960 con 0x7f12980b5240 2024-04-25T00:07:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.561+0000 7f129d9d4640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f127c06be30 0x7f127c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.562+0000 7f129d9d4640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f127c06be30 0x7f127c06e2f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f12940075b0 tx=0x7f1294007540 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.565+0000 7f128e7fc640 1 -- 172.21.15.113:0/3887055234 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1288020070 con 0x7f12980b5240 2024-04-25T00:07:06.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.772+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f129806e300 con 0x7f12980b5240 2024-04-25T00:07:06.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.773+0000 7f128e7fc640 1 -- 172.21.15.113:0/3887055234 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+4367 (secure 0 0 0) 0x7f1288059280 con 0x7f12980b5240 2024-04-25T00:07:06.773 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:06.773 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:07:02.919908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":15,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":9}]} 2024-04-25T00:07:06.774 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f127c06be30 msgr2=0x7f127c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f127c06be30 0x7f127c06e2f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f12940075b0 tx=0x7f1294007540 comp rx=0 tx=0).stop 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980b5240 msgr2=0x7f1298142780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980b5240 0x7f1298142780 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f128800c460 tx=0x7f128800c920 comp rx=0 tx=0).stop 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 shutdown_connections 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f127c06be30 0x7f127c06e2f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1298144910 0x7f1298142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12980b5240 0x7f1298142780 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 --2- 172.21.15.113:0/3887055234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f129806e6e0 0x7f129813be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 >> 172.21.15.113:0/3887055234 conn(0x7f12980b1a50 msgr2=0x7f12980b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 shutdown_connections 2024-04-25T00:07:06.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.775+0000 7f129fc38640 1 -- 172.21.15.113:0/3887055234 wait complete. 2024-04-25T00:07:06.786 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.882+0000 7fc1f3f1d640 1 Processor -- start 2024-04-25T00:07:06.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.882+0000 7fc1f3f1d640 1 -- start start 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f3f1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc1ec0b3fd0 0x7fc1ec06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f3f1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 0x7fc1ec0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f3f1d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1ec144cf0 con 0x7fc1ec144910 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f3f1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ec0b2f30 con 0x7fc1ec0b3fd0 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f3f1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0b3210 con 0x7fc1ec06e510 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f1cb9640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc1ec144910 0x7fc1ec0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48494/0 (socket says 172.21.15.113:48494) 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f1cb9640 1 -- 172.21.15.113:0/3964991332 learned_addr learned my addr 172.21.15.113:0/3964991332 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1758155144 0 0) 0x7fc1ec144cf0 con 0x7fc1ec144910 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1d8003520 con 0x7fc1ec144910 2024-04-25T00:07:06.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f14b8640 1 --2- 172.21.15.113:0/3964991332 >> v2:172.21.15.178:3300/0 conn(0x7fc1ec0b3fd0 0x7fc1ec06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f24ba640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 0x7fc1ec0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1062636098 0 0) 0x7fc1d8003520 con 0x7fc1ec144910 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 msgr2=0x7fc1ec0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 0x7fc1ec0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 >> v2:172.21.15.178:3300/0 conn(0x7fc1ec0b3fd0 msgr2=0x7fc1ec06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 --2- 172.21.15.113:0/3964991332 >> v2:172.21.15.178:3300/0 conn(0x7fc1ec0b3fd0 0x7fc1ec06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec0b34f0 con 0x7fc1ec144910 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fc1e8001670 con 0x7fc1ec144910 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f14b8640 1 --2- 172.21.15.113:0/3964991332 >> v2:172.21.15.178:3300/0 conn(0x7fc1ec0b3fd0 0x7fc1ec06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 >> v1:172.21.15.113:6789/0 conn(0x7fc1ec144910 legacy=0x7fc1ec0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d80039b0 0x7fc1d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d8004800 0x7fc1d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 0x7fc1d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0b2f30 con 0x7fc1d80039b0 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0b3210 con 0x7fc1d8004800 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d800ba70 con 0x7fc1ec06e510 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.883+0000 7fc1f1cb9640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d80039b0 0x7fc1d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f14b8640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d8004800 0x7fc1d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f24ba640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 0x7fc1d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f1cb9640 1 -- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 msgr2=0x7fc1d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f1cb9640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 0x7fc1d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f1cb9640 1 -- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d8004800 msgr2=0x7fc1d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f1cb9640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d8004800 0x7fc1d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f1cb9640 1 -- 172.21.15.113:0/3964991332 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d8003520 con 0x7fc1d80039b0 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f24ba640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec06e510 0x7fc1d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.884+0000 7fc1f1cb9640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d80039b0 0x7fc1d8003dd0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c210 tx=0x7fc1d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adc65daebd6dc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8007d10 con 0x7fc1d80039b0 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e8005ad0 con 0x7fc1d80039b0 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f0cb7640 1 -- 172.21.15.113:0/3964991332 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8007070 con 0x7fc1d80039b0 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/3964991332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d80039b0 msgr2=0x7fc1d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f3f1d640 1 --2- 172.21.15.113:0/3964991332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d80039b0 0x7fc1d8003dd0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c210 tx=0x7fc1d8005830 comp rx=0 tx=0).stop 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f24ba640 1 -- 172.21.15.113:0/3964991332 reap_dead start 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/3964991332 shutdown_connections 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/3964991332 >> 172.21.15.113:0/3964991332 conn(0x7fc1ec0b1a50 msgr2=0x7fc1ec0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/3964991332 shutdown_connections 2024-04-25T00:07:06.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.885+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/3964991332 wait complete. 2024-04-25T00:07:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.886+0000 7fc1f3f1d640 1 Processor -- start 2024-04-25T00:07:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.886+0000 7fc1f3f1d640 1 -- start start 2024-04-25T00:07:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f3f1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d80039b0 0x7fc1ec13aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f3f1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec13d070 0x7fc1ec13d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f3f1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ec140d00 0x7fc1ec141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f3f1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0b4da0 con 0x7fc1ec140d00 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f3f1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0b4440 con 0x7fc1d80039b0 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f3f1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0b4f20 con 0x7fc1ec13d070 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ec140d00 0x7fc1ec141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ec140d00 0x7fc1ec141180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55348/0 (socket says 172.21.15.113:55348) 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 -- 172.21.15.113:0/1919748751 learned_addr learned my addr 172.21.15.113:0/1919748751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f14b8640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec13d070 0x7fc1ec13d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f1cb9640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d80039b0 0x7fc1ec13aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 -- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec13d070 msgr2=0x7fc1ec13d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec13d070 0x7fc1ec13d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 -- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d80039b0 msgr2=0x7fc1ec13aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d80039b0 0x7fc1ec13aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 -- 172.21.15.113:0/1919748751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec15dec0 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.887+0000 7fc1f24ba640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ec140d00 0x7fc1ec141180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0cfc70 tx=0x7fc1e4004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.888+0000 7fc1e27fc640 1 -- 172.21.15.113:0/1919748751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4007ca0 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.888+0000 7fc1e27fc640 1 -- 172.21.15.113:0/1919748751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e4019070 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.888+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec13dbe0 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.888+0000 7fc1e27fc640 1 -- 172.21.15.113:0/1919748751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4007e60 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.888+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec13e1a0 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.889+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ec0691a0 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.889+0000 7fc1e27fc640 1 -- 172.21.15.113:0/1919748751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1e4014650 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.889+0000 7fc1e27fc640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1d006be30 0x7fc1d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.889+0000 7fc1e27fc640 1 -- 172.21.15.113:0/1919748751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7fc1e408b460 con 0x7fc1ec140d00 2024-04-25T00:07:06.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.889+0000 7fc1f1cb9640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1d006be30 0x7fc1d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.890+0000 7fc1f1cb9640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1d006be30 0x7fc1d006e2f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8032000 tx=0x7fc1e802d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:06.894+0000 7fc1e27fc640 1 -- 172.21.15.113:0/1919748751 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc1e4059cf0 con 0x7fc1ec140d00 2024-04-25T00:07:07.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.069+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1ec0731a0 con 0x7fc1ec140d00 2024-04-25T00:07:07.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.071+0000 7fc1e27fc640 1 -- 172.21.15.113:0/1919748751 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16900 (secure 0 0 0) 0x7fc1e40595f0 con 0x7fc1ec140d00 2024-04-25T00:07:07.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:07.071 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":133,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:02.920311+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:06:18.255623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:06:19.003747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.12","mappings":[{"from":2,"to":0}]},{"pgid":"19.23","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6836/2664195523":"2024-04-26T00:06:59.857692+0000","172.21.15.113:6834/2664195523":"2024-04-26T00:06:59.857692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1d006be30 msgr2=0x7fc1d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1d006be30 0x7fc1d006e2f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8032000 tx=0x7fc1e802d000 comp rx=0 tx=0).stop 2024-04-25T00:07:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ec140d00 msgr2=0x7fc1ec141180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ec140d00 0x7fc1ec141180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0cfc70 tx=0x7fc1e4004700 comp rx=0 tx=0).stop 2024-04-25T00:07:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 shutdown_connections 2024-04-25T00:07:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1d006be30 0x7fc1d006e2f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ec140d00 0x7fc1ec141180 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1ec13d070 0x7fc1ec13d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 --2- 172.21.15.113:0/1919748751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d80039b0 0x7fc1ec13aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 >> 172.21.15.113:0/1919748751 conn(0x7fc1ec0b1a50 msgr2=0x7fc1ec0aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 shutdown_connections 2024-04-25T00:07:07.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.073+0000 7fc1f3f1d640 1 -- 172.21.15.113:0/1919748751 wait complete. 2024-04-25T00:07:07.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:06:18.255623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:06:19.003747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:07:07.085 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:07:07.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.184+0000 7f94ad89e640 1 Processor -- start 2024-04-25T00:07:07.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.184+0000 7f94ad89e640 1 -- start start 2024-04-25T00:07:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94ad89e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a815f230 0x7f94a815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94ad89e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a8158f60 con 0x7f94a80afa40 2024-04-25T00:07:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94ad89e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a815f770 con 0x7f94a80cd720 2024-04-25T00:07:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94ad89e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94a8071ac0 con 0x7f94a815f230 2024-04-25T00:07:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a6ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f94a80afa40 0x7f94a80afe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48496/0 (socket says 172.21.15.113:48496) 2024-04-25T00:07:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a6ffd640 1 -- 172.21.15.113:0/976296806 learned_addr learned my addr 172.21.15.113:0/976296806 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:07.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a77fe640 1 --2- 172.21.15.113:0/976296806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a815f230 0x7f94a815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a5ffb640 1 -- 172.21.15.113:0/976296806 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481464312 0 0) 0x7f94a8158f60 con 0x7f94a80afa40 2024-04-25T00:07:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a5ffb640 1 -- 172.21.15.113:0/976296806 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9490003520 con 0x7f94a80afa40 2024-04-25T00:07:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a5ffb640 1 -- 172.21.15.113:0/976296806 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579326541 0 0) 0x7f94a815f770 con 0x7f94a80cd720 2024-04-25T00:07:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a77fe640 1 -- 172.21.15.113:0/976296806 >> v1:172.21.15.178:6789/0 conn(0x7f94a80cd720 legacy=0x7f94a815eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a77fe640 1 -- 172.21.15.113:0/976296806 >> v1:172.21.15.113:6789/0 conn(0x7f94a80afa40 legacy=0x7f94a80afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.185+0000 7f94a77fe640 1 -- 172.21.15.113:0/976296806 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a80ccc40 con 0x7f94a815f230 2024-04-25T00:07:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.186+0000 7f94a77fe640 1 --2- 172.21.15.113:0/976296806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a815f230 0x7f94a815fc30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f94a00097d0 tx=0x7f94a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f5b4048a261704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.186+0000 7f94a5ffb640 1 -- 172.21.15.113:0/976296806 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0037070 con 0x7f94a815f230 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.186+0000 7f94a5ffb640 1 -- 172.21.15.113:0/976296806 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a0032440 con 0x7f94a815f230 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.186+0000 7f94a5ffb640 1 -- 172.21.15.113:0/976296806 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0032810 con 0x7f94a815f230 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.187+0000 7f94ad89e640 1 -- 172.21.15.113:0/976296806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a815f230 msgr2=0x7f94a815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.187+0000 7f94ad89e640 1 --2- 172.21.15.113:0/976296806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a815f230 0x7f94a815fc30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f94a00097d0 tx=0x7f94a002a180 comp rx=0 tx=0).stop 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.187+0000 7f94ad89e640 1 -- 172.21.15.113:0/976296806 shutdown_connections 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.187+0000 7f94ad89e640 1 --2- 172.21.15.113:0/976296806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a815f230 0x7f94a815fc30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.187+0000 7f94ad89e640 1 -- 172.21.15.113:0/976296806 >> 172.21.15.113:0/976296806 conn(0x7f94a80b57e0 msgr2=0x7f94a8072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.187+0000 7f94ad89e640 1 -- 172.21.15.113:0/976296806 shutdown_connections 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.187+0000 7f94ad89e640 1 -- 172.21.15.113:0/976296806 wait complete. 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.188+0000 7f94ad89e640 1 Processor -- start 2024-04-25T00:07:07.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.188+0000 7f94ad89e640 1 -- start start 2024-04-25T00:07:07.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94ad89e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a80afa40 0x7f94a80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94ad89e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94a80cd720 0x7f94a80ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94ad89e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a815f230 0x7f94a80aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94ad89e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80cd290 con 0x7f94a80cd720 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94ad89e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94a815fa50 con 0x7f94a815f230 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94ad89e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94a80b1380 con 0x7f94a80afa40 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94a80cd720 0x7f94a80ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94a80cd720 0x7f94a80ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55358/0 (socket says 172.21.15.113:55358) 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 -- 172.21.15.113:0/2361284184 learned_addr learned my addr 172.21.15.113:0/2361284184 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 -- 172.21.15.113:0/2361284184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a80afa40 msgr2=0x7f94a80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a80afa40 0x7f94a80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 -- 172.21.15.113:0/2361284184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a815f230 msgr2=0x7f94a80aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a815f230 0x7f94a80aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.189+0000 7f94a67fc640 1 -- 172.21.15.113:0/2361284184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81f4430 con 0x7f94a80cd720 2024-04-25T00:07:07.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.190+0000 7f94a67fc640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94a80cd720 0x7f94a80ae4f0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c4b0 tx=0x7f949c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.190+0000 7f948ffff640 1 -- 172.21.15.113:0/2361284184 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c017070 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.190+0000 7f948ffff640 1 -- 172.21.15.113:0/2361284184 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949c005480 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.190+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81f4710 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.190+0000 7f948ffff640 1 -- 172.21.15.113:0/2361284184 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c003830 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.190+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81f4cd0 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.191+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a80691a0 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.191+0000 7f948ffff640 1 -- 172.21.15.113:0/2361284184 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f949c01a070 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.192+0000 7f948ffff640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949406be30 0x7f949406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.192+0000 7f948ffff640 1 -- 172.21.15.113:0/2361284184 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f949c08a260 con 0x7f94a80cd720 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.192+0000 7f94a6ffd640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949406be30 0x7f949406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.192+0000 7f94a6ffd640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949406be30 0x7f949406e2f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f94980012e0 tx=0x7f9498001650 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.196+0000 7f948ffff640 1 -- 172.21.15.113:0/2361284184 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f949c058c10 con 0x7f94a80cd720 2024-04-25T00:07:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.368+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f94a80ae2d0 con 0x7f94a80cd720 2024-04-25T00:07:07.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.369+0000 7f948ffff640 1 -- 172.21.15.113:0/2361284184 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) ==== 93+0+10 (secure 0 0 0) 0x7f949c058510 con 0x7f94a80cd720 2024-04-25T00:07:07.369 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:07:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949406be30 msgr2=0x7f949406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949406be30 0x7f949406e2f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f94980012e0 tx=0x7f9498001650 comp rx=0 tx=0).stop 2024-04-25T00:07:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94a80cd720 msgr2=0x7f94a80ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94a80cd720 0x7f94a80ae4f0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c4b0 tx=0x7f949c00c970 comp rx=0 tx=0).stop 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 shutdown_connections 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949406be30 0x7f949406e2f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a815f230 0x7f94a80aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94a80cd720 0x7f94a80ae4f0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 --2- 172.21.15.113:0/2361284184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94a80afa40 0x7f94a80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 >> 172.21.15.113:0/2361284184 conn(0x7f94a80b57e0 msgr2=0x7f94a8157120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 shutdown_connections 2024-04-25T00:07:07.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.371+0000 7f94ad89e640 1 -- 172.21.15.113:0/2361284184 wait complete. 2024-04-25T00:07:07.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:07:07.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.481+0000 7fbe1cc38640 1 Processor -- start 2024-04-25T00:07:07.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.481+0000 7fbe1cc38640 1 -- start start 2024-04-25T00:07:07.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe1cc38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b5ca0 0x7fbe180b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe1cc38640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe180b73c0 0x7fbe180b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe1cc38640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe180d2e00 con 0x7fbe1815e510 2024-04-25T00:07:07.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe1cc38640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbe180b6640 con 0x7fbe180b73c0 2024-04-25T00:07:07.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe1cc38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbe180d30e0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe16d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe180b73c0 0x7fbe180b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe1659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbe1815e510 0x7fbe180d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36180/0 (socket says 172.21.15.113:36180) 2024-04-25T00:07:07.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.482+0000 7fbe1659c640 1 -- 172.21.15.113:0/190349299 learned_addr learned my addr 172.21.15.113:0/190349299 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:07.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.483+0000 7fbe16d9d640 1 -- 172.21.15.113:0/190349299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b5ca0 msgr2=0x7fbe180b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:07.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.483+0000 7fbe16d9d640 1 --2- 172.21.15.113:0/190349299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b5ca0 0x7fbe180b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.483+0000 7fbe16d9d640 1 -- 172.21.15.113:0/190349299 >> v1:172.21.15.113:6789/0 conn(0x7fbe1815e510 legacy=0x7fbe180d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.483+0000 7fbe16d9d640 1 -- 172.21.15.113:0/190349299 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe180d33c0 con 0x7fbe180b73c0 2024-04-25T00:07:07.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.483+0000 7fbe1559a640 1 -- 172.21.15.113:0/190349299 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 501675482 0 0) 0x7fbe180d2e00 con 0x7fbe1815e510 2024-04-25T00:07:07.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.483+0000 7fbe16d9d640 1 --2- 172.21.15.113:0/190349299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b73c0 0x7fbe180b7830 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbe100097a0 tx=0x7fbe1002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5503f53f99d1174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1559a640 1 -- 172.21.15.113:0/190349299 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10037070 con 0x7fbe180b73c0 2024-04-25T00:07:07.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1559a640 1 -- 172.21.15.113:0/190349299 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1002ae50 con 0x7fbe180b73c0 2024-04-25T00:07:07.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1559a640 1 -- 172.21.15.113:0/190349299 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10032660 con 0x7fbe180b73c0 2024-04-25T00:07:07.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 -- 172.21.15.113:0/190349299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b73c0 msgr2=0x7fbe180b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/190349299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b73c0 0x7fbe180b7830 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbe100097a0 tx=0x7fbe1002a100 comp rx=0 tx=0).stop 2024-04-25T00:07:07.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 -- 172.21.15.113:0/190349299 shutdown_connections 2024-04-25T00:07:07.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/190349299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b73c0 0x7fbe180b7830 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/190349299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b5ca0 0x7fbe180b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 -- 172.21.15.113:0/190349299 >> 172.21.15.113:0/190349299 conn(0x7fbe1806b720 msgr2=0x7fbe180d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:07.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 -- 172.21.15.113:0/190349299 shutdown_connections 2024-04-25T00:07:07.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.484+0000 7fbe1cc38640 1 -- 172.21.15.113:0/190349299 wait complete. 2024-04-25T00:07:07.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.485+0000 7fbe1cc38640 1 Processor -- start 2024-04-25T00:07:07.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1cc38640 1 -- start start 2024-04-25T00:07:07.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1cc38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b5ca0 0x7fbe180c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1cc38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b73c0 0x7fbe18160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1cc38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815e510 0x7fbe18161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1cc38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe180d3ea0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1cc38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe180d3980 con 0x7fbe1815e510 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1cc38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbe180d2a90 con 0x7fbe180b73c0 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b5ca0 0x7fbe180c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b5ca0 0x7fbe180c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49658/0 (socket says 172.21.15.113:49658) 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.486+0000 7fbe1659c640 1 -- 172.21.15.113:0/1384887290 learned_addr learned my addr 172.21.15.113:0/1384887290 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe16d9d640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815e510 0x7fbe18161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe15d9b640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b73c0 0x7fbe18160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1659c640 1 -- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b73c0 msgr2=0x7fbe18160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1659c640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b73c0 0x7fbe18160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1659c640 1 -- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815e510 msgr2=0x7fbe18161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1659c640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815e510 0x7fbe18161260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1659c640 1 -- 172.21.15.113:0/1384887290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe180c7f60 con 0x7fbe180b5ca0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1659c640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b5ca0 0x7fbe180c9fb0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c8c0 tx=0x7fbe0800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe06ffd640 1 -- 172.21.15.113:0/1384887290 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08018070 con 0x7fbe180b5ca0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe06ffd640 1 -- 172.21.15.113:0/1384887290 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0800a5e0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe06ffd640 1 -- 172.21.15.113:0/1384887290 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800a9d0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe180c8240 con 0x7fbe180b5ca0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.487+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe180c8d20 con 0x7fbe180b5ca0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.488+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe180691a0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.488+0000 7fbe06ffd640 1 -- 172.21.15.113:0/1384887290 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe0801b070 con 0x7fbe180b5ca0 2024-04-25T00:07:07.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.489+0000 7fbe06ffd640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbdf806be30 0x7fbdf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.489+0000 7fbe06ffd640 1 -- 172.21.15.113:0/1384887290 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7fbe0808a5a0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.489+0000 7fbe15d9b640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbdf806be30 0x7fbdf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.489+0000 7fbe15d9b640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbdf806be30 0x7fbdf806e2f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbe180ceb70 tx=0x7fbe0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.493+0000 7fbe06ffd640 1 -- 172.21.15.113:0/1384887290 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe08058ec0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.666+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe1815c180 con 0x7fbe180b5ca0 2024-04-25T00:07:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.667+0000 7fbe06ffd640 1 -- 172.21.15.113:0/1384887290 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) ==== 104+0+11 (secure 0 0 0) 0x7fbe080587c0 con 0x7fbe180b5ca0 2024-04-25T00:07:07.667 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:07:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbdf806be30 msgr2=0x7fbdf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbdf806be30 0x7fbdf806e2f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbe180ceb70 tx=0x7fbe0c008040 comp rx=0 tx=0).stop 2024-04-25T00:07:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b5ca0 msgr2=0x7fbe180c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b5ca0 0x7fbe180c9fb0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c8c0 tx=0x7fbe0800cd80 comp rx=0 tx=0).stop 2024-04-25T00:07:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 shutdown_connections 2024-04-25T00:07:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbdf806be30 0x7fbdf806e2f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815e510 0x7fbe18161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180b73c0 0x7fbe18160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 --2- 172.21.15.113:0/1384887290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe180b5ca0 0x7fbe180c9fb0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 >> 172.21.15.113:0/1384887290 conn(0x7fbe1806b720 msgr2=0x7fbe18071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 shutdown_connections 2024-04-25T00:07:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.669+0000 7fbe1cc38640 1 -- 172.21.15.113:0/1384887290 wait complete. 2024-04-25T00:07:07.680 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:07:07.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.780+0000 7f5e22312640 1 Processor -- start 2024-04-25T00:07:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.780+0000 7f5e22312640 1 -- start start 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e22312640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e1c15ba50 0x7f5e1c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e22312640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5e1c06cea0 0x7f5e1c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e22312640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e1c06d7e0 con 0x7f5e1c15b670 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e22312640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1c071170 con 0x7f5e1c15ba50 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e22312640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1c071450 con 0x7f5e1c06cea0 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e1c15ba50 0x7f5e1c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e1c15ba50 0x7f5e1c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49662/0 (socket says 172.21.15.113:49662) 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1b7fe640 1 -- 172.21.15.113:0/1052145137 learned_addr learned my addr 172.21.15.113:0/1052145137 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1bfff640 1 --1- 172.21.15.113:0/1052145137 >> v1:172.21.15.113:6789/0 conn(0x7f5e1c15b670 0x7f5e1c0cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36196/0 (socket says 172.21.15.113:36196) 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1b7fe640 1 -- 172.21.15.113:0/1052145137 >> v2:172.21.15.178:3300/0 conn(0x7f5e1c06cea0 msgr2=0x7f5e1c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1b7fe640 1 --2- 172.21.15.113:0/1052145137 >> v2:172.21.15.178:3300/0 conn(0x7f5e1c06cea0 0x7f5e1c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1b7fe640 1 -- 172.21.15.113:0/1052145137 >> v1:172.21.15.113:6789/0 conn(0x7f5e1c15b670 legacy=0x7f5e1c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1b7fe640 1 -- 172.21.15.113:0/1052145137 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c071730 con 0x7f5e1c15ba50 2024-04-25T00:07:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.781+0000 7f5e1affd640 1 -- 172.21.15.113:0/1052145137 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546239285 0 0) 0x7f5e1c06d7e0 con 0x7f5e1c15b670 2024-04-25T00:07:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.782+0000 7f5e1b7fe640 1 --2- 172.21.15.113:0/1052145137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c15ba50 0x7f5e1c15bff0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c1589f0 tx=0x7f5e1002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd6b66ab569db2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.782+0000 7f5e1affd640 1 -- 172.21.15.113:0/1052145137 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10037070 con 0x7f5e1c15ba50 2024-04-25T00:07:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.782+0000 7f5e1affd640 1 -- 172.21.15.113:0/1052145137 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e10032440 con 0x7f5e1c15ba50 2024-04-25T00:07:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.782+0000 7f5e1affd640 1 -- 172.21.15.113:0/1052145137 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10032810 con 0x7f5e1c15ba50 2024-04-25T00:07:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 -- 172.21.15.113:0/1052145137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c15ba50 msgr2=0x7f5e1c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 --2- 172.21.15.113:0/1052145137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c15ba50 0x7f5e1c15bff0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c1589f0 tx=0x7f5e1002a150 comp rx=0 tx=0).stop 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 -- 172.21.15.113:0/1052145137 shutdown_connections 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 --2- 172.21.15.113:0/1052145137 >> v2:172.21.15.178:3300/0 conn(0x7f5e1c06cea0 0x7f5e1c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 --2- 172.21.15.113:0/1052145137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c15ba50 0x7f5e1c15bff0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 -- 172.21.15.113:0/1052145137 >> 172.21.15.113:0/1052145137 conn(0x7f5e1c06b0f0 msgr2=0x7f5e1c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 -- 172.21.15.113:0/1052145137 shutdown_connections 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.783+0000 7f5e22312640 1 -- 172.21.15.113:0/1052145137 wait complete. 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.784+0000 7f5e22312640 1 Processor -- start 2024-04-25T00:07:07.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.784+0000 7f5e22312640 1 -- start start 2024-04-25T00:07:07.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e22312640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c06cea0 0x7f5e1c150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e22312640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e1c15b670 0x7f5e1c142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e22312640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e1c15ba50 0x7f5e1c144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e22312640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c0cca90 con 0x7f5e1c06cea0 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e22312640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c0cc910 con 0x7f5e1c15b670 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e22312640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c0d0080 con 0x7f5e1c15ba50 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c06cea0 0x7f5e1c150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c06cea0 0x7f5e1c150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49676/0 (socket says 172.21.15.113:49676) 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 -- 172.21.15.113:0/1562534305 learned_addr learned my addr 172.21.15.113:0/1562534305 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 -- 172.21.15.113:0/1562534305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e1c15ba50 msgr2=0x7f5e1c144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e1c15ba50 0x7f5e1c144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1b7fe640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e1c15b670 0x7f5e1c142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 -- 172.21.15.113:0/1562534305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e1c15b670 msgr2=0x7f5e1c142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e1c15b670 0x7f5e1c142080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 -- 172.21.15.113:0/1562534305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c156e20 con 0x7f5e1c06cea0 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.785+0000 7f5e1bfff640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c06cea0 0x7f5e1c150bb0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00c4d0 tx=0x7f5e0c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.786+0000 7f5e18ff9640 1 -- 172.21.15.113:0/1562534305 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c018070 con 0x7f5e1c06cea0 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.786+0000 7f5e18ff9640 1 -- 172.21.15.113:0/1562534305 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e0c007500 con 0x7f5e1c06cea0 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.786+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c1425c0 con 0x7f5e1c06cea0 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.786+0000 7f5e18ff9640 1 -- 172.21.15.113:0/1562534305 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c004a20 con 0x7f5e1c06cea0 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.786+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c156b40 con 0x7f5e1c06cea0 2024-04-25T00:07:07.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.787+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1c0691a0 con 0x7f5e1c06cea0 2024-04-25T00:07:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.787+0000 7f5e18ff9640 1 -- 172.21.15.113:0/1562534305 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e0c01b070 con 0x7f5e1c06cea0 2024-04-25T00:07:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.787+0000 7f5e18ff9640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e0806be60 0x7f5e0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.787+0000 7f5e18ff9640 1 -- 172.21.15.113:0/1562534305 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f5e0c08a190 con 0x7f5e1c06cea0 2024-04-25T00:07:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.787+0000 7f5e1b7fe640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e0806be60 0x7f5e0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.788+0000 7f5e1b7fe640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e0806be60 0x7f5e0806e320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5e10034000 tx=0x7f5e1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:07.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.792+0000 7f5e18ff9640 1 -- 172.21.15.113:0/1562534305 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e1c1425c0 con 0x7f5e1c06cea0 2024-04-25T00:07:07.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.966+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5e1c159830 con 0x7f5e1c06cea0 2024-04-25T00:07:07.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.966+0000 7f5e18ff9640 1 -- 172.21.15.113:0/1562534305 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) ==== 100+0+11 (secure 0 0 0) 0x7f5e0c058ab0 con 0x7f5e1c06cea0 2024-04-25T00:07:07.968 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:07:07.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e0806be60 msgr2=0x7f5e0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e0806be60 0x7f5e0806e320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5e10034000 tx=0x7f5e1002e000 comp rx=0 tx=0).stop 2024-04-25T00:07:07.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c06cea0 msgr2=0x7f5e1c150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:07.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c06cea0 0x7f5e1c150bb0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00c4d0 tx=0x7f5e0c00c990 comp rx=0 tx=0).stop 2024-04-25T00:07:07.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 shutdown_connections 2024-04-25T00:07:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e0806be60 0x7f5e0806e320 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e1c15b670 0x7f5e1c142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e1c15ba50 0x7f5e1c144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 --2- 172.21.15.113:0/1562534305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e1c06cea0 0x7f5e1c150bb0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 >> 172.21.15.113:0/1562534305 conn(0x7f5e1c06b0f0 msgr2=0x7f5e1c158730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 shutdown_connections 2024-04-25T00:07:07.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:07.969+0000 7f5e22312640 1 -- 172.21.15.113:0/1562534305 wait complete. 2024-04-25T00:07:07.980 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.076+0000 7efe13a15640 1 Processor -- start 2024-04-25T00:07:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.077+0000 7efe13a15640 1 -- start start 2024-04-25T00:07:08.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.077+0000 7efe13a15640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 0x7efe0c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.077+0000 7efe13a15640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c0d2800 con 0x7efe0c0d2290 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.077+0000 7efe13a15640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c0d2ae0 con 0x7efe0c15e320 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.077+0000 7efe13a15640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe0c0d14d0 con 0x7efe0c0d0b00 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe10fb0640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efe0c0d2290 0x7efe0c0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36198/0 (socket says 172.21.15.113:36198) 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe10fb0640 1 -- 172.21.15.113:0/3412149710 learned_addr learned my addr 172.21.15.113:0/3412149710 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe11fb2640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 0x7efe0c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2151734335 0 0) 0x7efe0c0d2800 con 0x7efe0c0d2290 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdf8003520 con 0x7efe0c0d2290 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470826602 0 0) 0x7efe0c0d2ae0 con 0x7efe0c15e320 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe0c0d2800 con 0x7efe0c15e320 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2020848346 0 0) 0x7efdf8003520 con 0x7efe0c0d2290 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 msgr2=0x7efe0c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 0x7efe0c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 >> v1:172.21.15.178:6789/0 conn(0x7efe0c15e320 legacy=0x7efe0c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c0d2dc0 con 0x7efe0c0d2290 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7efdfc001690 con 0x7efe0c0d2290 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 >> v1:172.21.15.113:6789/0 conn(0x7efe0c0d2290 legacy=0x7efe0c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 0x7efdf8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80040d0 0x7efdf8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf8004d40 0x7efdf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c0d14d0 con 0x7efdf80040d0 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800baa0 con 0x7efdf8004d40 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efdf800bca0 con 0x7efe0c0d0b00 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe10fb0640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80040d0 0x7efdf8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe117b1640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 0x7efdf8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.078+0000 7efe11fb2640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf8004d40 0x7efdf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe10fb0640 1 -- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 msgr2=0x7efdf8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe10fb0640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c0d0b00 0x7efdf8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe10fb0640 1 -- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf8004d40 msgr2=0x7efdf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe10fb0640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf8004d40 0x7efdf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe10fb0640 1 -- 172.21.15.113:0/3412149710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf8003520 con 0x7efdf80040d0 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe11fb2640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf8004d40 0x7efdf80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe10fb0640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80040d0 0x7efdf8004800 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c160 tx=0x7efdf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611cbde7c77e0746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc007cd0 con 0x7efdf80040d0 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc005af0 con 0x7efdf80040d0 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.079+0000 7efe03fff640 1 -- 172.21.15.113:0/3412149710 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc007070 con 0x7efdf80040d0 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.080+0000 7efe13a15640 1 -- 172.21.15.113:0/3412149710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80040d0 msgr2=0x7efdf8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.080+0000 7efe13a15640 1 --2- 172.21.15.113:0/3412149710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdf80040d0 0x7efdf8004800 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c160 tx=0x7efdf8005830 comp rx=0 tx=0).stop 2024-04-25T00:07:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.080+0000 7efe11fb2640 1 -- 172.21.15.113:0/3412149710 reap_dead start 2024-04-25T00:07:08.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.080+0000 7efe13a15640 1 -- 172.21.15.113:0/3412149710 shutdown_connections 2024-04-25T00:07:08.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.080+0000 7efe13a15640 1 -- 172.21.15.113:0/3412149710 >> 172.21.15.113:0/3412149710 conn(0x7efe0c0b5c20 msgr2=0x7efe0c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.080+0000 7efe13a15640 1 -- 172.21.15.113:0/3412149710 shutdown_connections 2024-04-25T00:07:08.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.080+0000 7efe13a15640 1 -- 172.21.15.113:0/3412149710 wait complete. 2024-04-25T00:07:08.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.081+0000 7efe13a15640 1 Processor -- start 2024-04-25T00:07:08.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.081+0000 7efe13a15640 1 -- start start 2024-04-25T00:07:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe13a15640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf80040d0 0x7efe0c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe13a15640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c161380 0x7efe0c0ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe13a15640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe0c071710 0x7efe0c071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe13a15640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c15e700 con 0x7efe0c071710 2024-04-25T00:07:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe13a15640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c0d3410 con 0x7efdf80040d0 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe13a15640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe0c15e880 con 0x7efe0c161380 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe0c071710 0x7efe0c071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe0c071710 0x7efe0c071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49704/0 (socket says 172.21.15.113:49704) 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 -- 172.21.15.113:0/4111777290 learned_addr learned my addr 172.21.15.113:0/4111777290 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe10fb0640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c161380 0x7efe0c0ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe117b1640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf80040d0 0x7efe0c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 -- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c161380 msgr2=0x7efe0c0ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c161380 0x7efe0c0ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 -- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf80040d0 msgr2=0x7efe0c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf80040d0 0x7efe0c151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 -- 172.21.15.113:0/4111777290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c1602e0 con 0x7efe0c071710 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe11fb2640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe0c071710 0x7efe0c071b90 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0d51b0 tx=0x7efe04010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.082+0000 7efe01ffb640 1 -- 172.21.15.113:0/4111777290 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0401a030 con 0x7efe0c071710 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.083+0000 7efe01ffb640 1 -- 172.21.15.113:0/4111777290 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe04010e00 con 0x7efe0c071710 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.083+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c1605c0 con 0x7efe0c071710 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.083+0000 7efe01ffb640 1 -- 172.21.15.113:0/4111777290 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04003bd0 con 0x7efe0c071710 2024-04-25T00:07:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.083+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c160b80 con 0x7efe0c071710 2024-04-25T00:07:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.084+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0c0691a0 con 0x7efe0c071710 2024-04-25T00:07:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.084+0000 7efe01ffb640 1 -- 172.21.15.113:0/4111777290 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe0401d070 con 0x7efe0c071710 2024-04-25T00:07:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.084+0000 7efe01ffb640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdf006be60 0x7efdf006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.085+0000 7efe01ffb640 1 -- 172.21.15.113:0/4111777290 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7efe0408b2b0 con 0x7efe0c071710 2024-04-25T00:07:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.085+0000 7efe117b1640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdf006be60 0x7efdf006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.085+0000 7efe117b1640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdf006be60 0x7efdf006e320 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7efe080018d0 tx=0x7efe08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.089+0000 7efe01ffb640 1 -- 172.21.15.113:0/4111777290 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe04059c60 con 0x7efe0c071710 2024-04-25T00:07:08.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.297+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe0c0b2d80 con 0x7efe0c071710 2024-04-25T00:07:08.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.298+0000 7efe01ffb640 1 -- 172.21.15.113:0/4111777290 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+4367 (secure 0 0 0) 0x7efe04059560 con 0x7efe0c071710 2024-04-25T00:07:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:08.298 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:07:02.919908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":15,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":9}]} 2024-04-25T00:07:08.299 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-04-25T00:07:08.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdf006be60 msgr2=0x7efdf006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdf006be60 0x7efdf006e320 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7efe080018d0 tx=0x7efe08009040 comp rx=0 tx=0).stop 2024-04-25T00:07:08.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe0c071710 msgr2=0x7efe0c071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe0c071710 0x7efe0c071b90 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0d51b0 tx=0x7efe04010590 comp rx=0 tx=0).stop 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 shutdown_connections 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdf006be60 0x7efdf006e320 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe0c071710 0x7efe0c071b90 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe0c161380 0x7efe0c0ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 --2- 172.21.15.113:0/4111777290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf80040d0 0x7efe0c151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 >> 172.21.15.113:0/4111777290 conn(0x7efe0c0b5c20 msgr2=0x7efe0c0b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 shutdown_connections 2024-04-25T00:07:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.300+0000 7efe13a15640 1 -- 172.21.15.113:0/4111777290 wait complete. 2024-04-25T00:07:08.311 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:08.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.408+0000 7f8e45542640 1 Processor -- start 2024-04-25T00:07:08.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.408+0000 7f8e45542640 1 -- start start 2024-04-25T00:07:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e45542640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8e4015e4f0 0x7f8e4015f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e45542640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4015f8b0 con 0x7f8e400cf2b0 2024-04-25T00:07:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e45542640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e40161ff0 con 0x7f8e400b6470 2024-04-25T00:07:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e45542640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8e401622d0 con 0x7f8e4015e4f0 2024-04-25T00:07:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e37fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8e400cf2b0 0x7f8e400b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36206/0 (socket says 172.21.15.113:36206) 2024-04-25T00:07:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e37fff640 1 -- 172.21.15.113:0/209128526 learned_addr learned my addr 172.21.15.113:0/209128526 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3effd640 1 --2- 172.21.15.113:0/209128526 >> v2:172.21.15.178:3300/0 conn(0x7f8e4015e4f0 0x7f8e4015f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483519124 0 0) 0x7f8e28001110 con 0x7f8e400cf2b0 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e2c003520 con 0x7f8e400cf2b0 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3896708601 0 0) 0x7f8e2c003520 con 0x7f8e400cf2b0 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 >> v2:172.21.15.178:3300/0 conn(0x7f8e4015e4f0 msgr2=0x7f8e4015f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 --2- 172.21.15.113:0/209128526 >> v2:172.21.15.178:3300/0 conn(0x7f8e4015e4f0 0x7f8e4015f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 >> v1:172.21.15.178:6789/0 conn(0x7f8e400b6470 legacy=0x7f8e400d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e401625b0 con 0x7f8e400cf2b0 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.409+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8e28001840 con 0x7f8e400cf2b0 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 >> v1:172.21.15.113:6789/0 conn(0x7f8e400cf2b0 legacy=0x7f8e400b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3e7fc640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2c0039b0 0x7f8e2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3e7fc640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2c004800 0x7f8e2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3e7fc640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0052e0 0x7f8e2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401622d0 con 0x7f8e2c0039b0 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c00c1b0 con 0x7f8e2c004800 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2c00c3b0 con 0x7f8e2c0052e0 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3effd640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2c0039b0 0x7f8e2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e37fff640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2c004800 0x7f8e2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3f7fe640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0052e0 0x7f8e2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3effd640 1 -- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0052e0 msgr2=0x7f8e2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3effd640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0052e0 0x7f8e2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3effd640 1 -- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2c004800 msgr2=0x7f8e2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3effd640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2c004800 0x7f8e2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3effd640 1 -- 172.21.15.113:0/209128526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c003520 con 0x7f8e2c0039b0 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.410+0000 7f8e3f7fe640 1 -- 172.21.15.113:0/209128526 reap_dead start 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e3effd640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2c0039b0 0x7f8e2c003dd0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009a90 tx=0x7f8e2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44840a87df8fae8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30036070 con 0x7f8e2c0039b0 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e30031400 con 0x7f8e2c0039b0 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e3e7fc640 1 -- 172.21.15.113:0/209128526 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30009690 con 0x7f8e2c0039b0 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e45542640 1 -- 172.21.15.113:0/209128526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2c0039b0 msgr2=0x7f8e2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e45542640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2c0039b0 0x7f8e2c003dd0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009a90 tx=0x7f8e2c005830 comp rx=0 tx=0).stop 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e45542640 1 -- 172.21.15.113:0/209128526 shutdown_connections 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e45542640 1 --2- 172.21.15.113:0/209128526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2c0039b0 0x7f8e2c003dd0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e45542640 1 -- 172.21.15.113:0/209128526 >> 172.21.15.113:0/209128526 conn(0x7f8e4006b720 msgr2=0x7f8e400d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e45542640 1 -- 172.21.15.113:0/209128526 shutdown_connections 2024-04-25T00:07:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.411+0000 7f8e45542640 1 -- 172.21.15.113:0/209128526 wait complete. 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 Processor -- start 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 -- start start 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0039b0 0x7f8e4014d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4014d690 0x7f8e4014aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e4014b3f0 0x7f8e400cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4015f4f0 con 0x7f8e4014b3f0 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4015f370 con 0x7f8e4014d690 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e45542640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4015f670 con 0x7f8e2c0039b0 2024-04-25T00:07:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e4014b3f0 0x7f8e400cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e4014b3f0 0x7f8e400cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49722/0 (socket says 172.21.15.113:49722) 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.413+0000 7f8e3f7fe640 1 -- 172.21.15.113:0/1053073961 learned_addr learned my addr 172.21.15.113:0/1053073961 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e3effd640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0039b0 0x7f8e4014d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e37fff640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4014d690 0x7f8e4014aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e3f7fe640 1 -- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0039b0 msgr2=0x7f8e4014d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e3f7fe640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0039b0 0x7f8e4014d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e3f7fe640 1 -- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4014d690 msgr2=0x7f8e4014aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e3f7fe640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4014d690 0x7f8e4014aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e3f7fe640 1 -- 172.21.15.113:0/1053073961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4014bac0 con 0x7f8e4014b3f0 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e37fff640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4014d690 0x7f8e4014aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e3f7fe640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e4014b3f0 0x7f8e400cd7f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f8e400b7270 tx=0x7f8e3800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e377fe640 1 -- 172.21.15.113:0/1053073961 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003b80 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e377fe640 1 -- 172.21.15.113:0/1053073961 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e38018070 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e377fe640 1 -- 172.21.15.113:0/1053073961 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380050d0 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4014bda0 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.414+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4014c360 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.415+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e400691a0 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.415+0000 7f8e377fe640 1 -- 172.21.15.113:0/1053073961 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e38005270 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.416+0000 7f8e377fe640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e2006be30 0x7f8e2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.416+0000 7f8e3effd640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e2006be30 0x7f8e2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.416+0000 7f8e377fe640 1 -- 172.21.15.113:0/1053073961 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f8e3808ade0 con 0x7f8e4014b3f0 2024-04-25T00:07:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.416+0000 7f8e3effd640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e2006be30 0x7f8e2006e2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8e30033000 tx=0x7f8e30008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.420+0000 7f8e377fe640 1 -- 172.21.15.113:0/1053073961 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e38059700 con 0x7f8e4014b3f0 2024-04-25T00:07:08.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.592+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e400d1ff0 con 0x7f8e4014b3f0 2024-04-25T00:07:08.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.593+0000 7f8e377fe640 1 -- 172.21.15.113:0/1053073961 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16900 (secure 0 0 0) 0x7f8e38059000 con 0x7f8e4014b3f0 2024-04-25T00:07:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:08.594 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":133,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:02.920311+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:06:18.255623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:06:19.003747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.12","mappings":[{"from":2,"to":0}]},{"pgid":"19.23","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6836/2664195523":"2024-04-26T00:06:59.857692+0000","172.21.15.113:6834/2664195523":"2024-04-26T00:06:59.857692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e2006be30 msgr2=0x7f8e2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e2006be30 0x7f8e2006e2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8e30033000 tx=0x7f8e30008030 comp rx=0 tx=0).stop 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e4014b3f0 msgr2=0x7f8e400cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e4014b3f0 0x7f8e400cd7f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f8e400b7270 tx=0x7f8e3800fd40 comp rx=0 tx=0).stop 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 shutdown_connections 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8e2006be30 0x7f8e2006e2f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4014d690 0x7f8e4014aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e4014b3f0 0x7f8e400cd7f0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 --2- 172.21.15.113:0/1053073961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e2c0039b0 0x7f8e4014d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 >> 172.21.15.113:0/1053073961 conn(0x7f8e4006b720 msgr2=0x7f8e400d4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 shutdown_connections 2024-04-25T00:07:08.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.595+0000 7f8e45542640 1 -- 172.21.15.113:0/1053073961 wait complete. 2024-04-25T00:07:08.607 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:07:08.607 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:08.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.704+0000 7f1e00b09640 1 Processor -- start 2024-04-25T00:07:08.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.704+0000 7f1e00b09640 1 -- start start 2024-04-25T00:07:08.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.705+0000 7f1e00b09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1dfc0d2290 0x7f1dfc0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.705+0000 7f1e00b09640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1dfc0d0b00 0x7f1dfc0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.705+0000 7f1e00b09640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dfc0d1480 con 0x7f1dfc15e320 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.705+0000 7f1e00b09640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1dfc0d3410 con 0x7f1dfc0d2290 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.705+0000 7f1e00b09640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1dfc0d2800 con 0x7f1dfc0d0b00 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df9d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1dfc0d2290 0x7f1dfc0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df9d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1dfc0d2290 0x7f1dfc0b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49738/0 (socket says 172.21.15.113:49738) 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1dfa59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1dfc15e320 0x7f1dfc0d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36210/0 (socket says 172.21.15.113:36210) 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df9d9b640 1 -- 172.21.15.113:0/1996885434 learned_addr learned my addr 172.21.15.113:0/1996885434 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1dfad9d640 1 --2- 172.21.15.113:0/1996885434 >> v2:172.21.15.178:3300/0 conn(0x7f1dfc0d0b00 0x7f1dfc0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df9d9b640 1 -- 172.21.15.113:0/1996885434 >> v2:172.21.15.178:3300/0 conn(0x7f1dfc0d0b00 msgr2=0x7f1dfc0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df9d9b640 1 --2- 172.21.15.113:0/1996885434 >> v2:172.21.15.178:3300/0 conn(0x7f1dfc0d0b00 0x7f1dfc0d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df9d9b640 1 -- 172.21.15.113:0/1996885434 >> v1:172.21.15.113:6789/0 conn(0x7f1dfc15e320 legacy=0x7f1dfc0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df9d9b640 1 -- 172.21.15.113:0/1996885434 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc0d2ae0 con 0x7f1dfc0d2290 2024-04-25T00:07:08.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.706+0000 7f1df959a640 1 -- 172.21.15.113:0/1996885434 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1774732904 0 0) 0x7f1dfc0d1480 con 0x7f1dfc15e320 2024-04-25T00:07:08.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1df9d9b640 1 --2- 172.21.15.113:0/1996885434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0b9aa0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0b3970 tx=0x7f1df002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd38533fa4e4f1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1df959a640 1 -- 172.21.15.113:0/1996885434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0037070 con 0x7f1dfc0d2290 2024-04-25T00:07:08.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1df959a640 1 -- 172.21.15.113:0/1996885434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df002ae50 con 0x7f1dfc0d2290 2024-04-25T00:07:08.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1df959a640 1 -- 172.21.15.113:0/1996885434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0032860 con 0x7f1dfc0d2290 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1e00b09640 1 -- 172.21.15.113:0/1996885434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 msgr2=0x7f1dfc0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1e00b09640 1 --2- 172.21.15.113:0/1996885434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0b9aa0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0b3970 tx=0x7f1df002a300 comp rx=0 tx=0).stop 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1e00b09640 1 -- 172.21.15.113:0/1996885434 shutdown_connections 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1e00b09640 1 --2- 172.21.15.113:0/1996885434 >> v2:172.21.15.178:3300/0 conn(0x7f1dfc0d0b00 0x7f1dfc0d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1e00b09640 1 --2- 172.21.15.113:0/1996885434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0b9aa0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.707+0000 7f1e00b09640 1 -- 172.21.15.113:0/1996885434 >> 172.21.15.113:0/1996885434 conn(0x7f1dfc0b5c20 msgr2=0x7f1dfc0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.708+0000 7f1e00b09640 1 -- 172.21.15.113:0/1996885434 shutdown_connections 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.708+0000 7f1e00b09640 1 -- 172.21.15.113:0/1996885434 wait complete. 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.709+0000 7f1e00b09640 1 Processor -- start 2024-04-25T00:07:08.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.709+0000 7f1e00b09640 1 -- start start 2024-04-25T00:07:08.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1e00b09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dfc0d0b00 0x7f1dfc161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1e00b09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1e00b09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1dfc15e320 0x7f1dfc0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1e00b09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc0b9fe0 con 0x7f1dfc0d2290 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1e00b09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc0d30a0 con 0x7f1dfc0d0b00 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1e00b09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc15e700 con 0x7f1dfc15e320 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1dfa59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dfc0d0b00 0x7f1dfc161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49752/0 (socket says 172.21.15.113:49752) 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 -- 172.21.15.113:0/355761277 learned_addr learned my addr 172.21.15.113:0/355761277 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1dfad9d640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1dfc15e320 0x7f1dfc0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 -- 172.21.15.113:0/355761277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1dfc15e320 msgr2=0x7f1dfc0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1dfc15e320 0x7f1dfc0cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 -- 172.21.15.113:0/355761277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dfc0d0b00 msgr2=0x7f1dfc161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dfc0d0b00 0x7f1dfc161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.710+0000 7f1df9d9b640 1 -- 172.21.15.113:0/355761277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc160670 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.711+0000 7f1df9d9b640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0ab4e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1df002a340 tx=0x7f1df00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.711+0000 7f1deaffd640 1 -- 172.21.15.113:0/355761277 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0041070 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.711+0000 7f1deaffd640 1 -- 172.21.15.113:0/355761277 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df003c680 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.711+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc160950 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.711+0000 7f1deaffd640 1 -- 172.21.15.113:0/355761277 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0037040 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.711+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc160f10 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.712+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc0691a0 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.712+0000 7f1deaffd640 1 -- 172.21.15.113:0/355761277 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1df0044070 con 0x7f1dfc0d2290 2024-04-25T00:07:08.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.712+0000 7f1deaffd640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.713+0000 7f1deaffd640 1 -- 172.21.15.113:0/355761277 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f1df00ab830 con 0x7f1dfc0d2290 2024-04-25T00:07:08.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.713+0000 7f1dfa59c640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.713+0000 7f1dfa59c640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000f60 tx=0x7f1dec007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.717+0000 7f1deaffd640 1 -- 172.21.15.113:0/355761277 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1df0079c20 con 0x7f1dfc0d2290 2024-04-25T00:07:08.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.926+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dfc15cc30 con 0x7f1dfc0d2290 2024-04-25T00:07:08.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.927+0000 7f1deaffd640 1 -- 172.21.15.113:0/355761277 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+4367 (secure 0 0 0) 0x7f1df007df20 con 0x7f1dfc0d2290 2024-04-25T00:07:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:08.928 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:07:02.919908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":15,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":9}]} 2024-04-25T00:07:08.928 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ddc06be30 msgr2=0x7f1ddc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000f60 tx=0x7f1dec007d50 comp rx=0 tx=0).stop 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 msgr2=0x7f1dfc0ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0ab4e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1df002a340 tx=0x7f1df00046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 shutdown_connections 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1dfc15e320 0x7f1dfc0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dfc0d2290 0x7f1dfc0ab4e0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 --2- 172.21.15.113:0/355761277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dfc0d0b00 0x7f1dfc161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 >> 172.21.15.113:0/355761277 conn(0x7f1dfc0b5c20 msgr2=0x7f1dfc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 shutdown_connections 2024-04-25T00:07:08.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:08.929+0000 7f1e00b09640 1 -- 172.21.15.113:0/355761277 wait complete. 2024-04-25T00:07:08.941 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:09.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.037+0000 7f09e1990640 1 Processor -- start 2024-04-25T00:07:09.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.037+0000 7f09e1990640 1 -- start start 2024-04-25T00:07:09.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09e1990640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f09dc15e310 0x7f09dc0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09e1990640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09dc0c46d0 con 0x7f09dc0ce9a0 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09e1990640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09dc06b190 con 0x7f09dc0ce370 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09e1990640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f09dc15e6f0 con 0x7f09dc15e310 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09db7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f09dc0ce9a0 0x7f09dc061b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36212/0 (socket says 172.21.15.113:36212) 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09db7fe640 1 -- 172.21.15.113:0/2128152592 learned_addr learned my addr 172.21.15.113:0/2128152592 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09daffd640 1 --2- 172.21.15.113:0/2128152592 >> v2:172.21.15.178:3300/0 conn(0x7f09dc15e310 0x7f09dc0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228445862 0 0) 0x7f09dc0c46d0 con 0x7f09dc0ce9a0 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09c8003520 con 0x7f09dc0ce9a0 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4101690860 0 0) 0x7f09c8003520 con 0x7f09dc0ce9a0 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 >> v2:172.21.15.178:3300/0 conn(0x7f09dc15e310 msgr2=0x7f09dc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 --2- 172.21.15.113:0/2128152592 >> v2:172.21.15.178:3300/0 conn(0x7f09dc15e310 0x7f09dc0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 >> v1:172.21.15.178:6789/0 conn(0x7f09dc0ce370 legacy=0x7f09dc06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc06b470 con 0x7f09dc0ce9a0 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f09d40019f0 con 0x7f09dc0ce9a0 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 >> v1:172.21.15.113:6789/0 conn(0x7f09dc0ce9a0 legacy=0x7f09dc061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c80039b0 0x7f09c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c80051c0 0x7f09c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09c800c030 con 0x7f09c8004800 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09c800c1e0 con 0x7f09c80051c0 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.038+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09c800c3e0 con 0x7f09c80039b0 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09d3fff640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09db7fe640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c80051c0 0x7f09c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09daffd640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c80039b0 0x7f09c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09d3fff640 1 -- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c80039b0 msgr2=0x7f09c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09d3fff640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c80039b0 0x7f09c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09d3fff640 1 -- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c80051c0 msgr2=0x7f09c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09d3fff640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c80051c0 0x7f09c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09d3fff640 1 -- 172.21.15.113:0/2128152592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c8003520 con 0x7f09c8004800 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09db7fe640 1 -- 172.21.15.113:0/2128152592 reap_dead start 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.039+0000 7f09d3fff640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f09c4001750 tx=0x7f09c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489367016852dbfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4001370 con 0x7f09c8004800 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c40052d0 con 0x7f09c8004800 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09da7fc640 1 -- 172.21.15.113:0/2128152592 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c402abc0 con 0x7f09c8004800 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09e1990640 1 -- 172.21.15.113:0/2128152592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 msgr2=0x7f09c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09e1990640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f09c4001750 tx=0x7f09c8005830 comp rx=0 tx=0).stop 2024-04-25T00:07:09.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09e1990640 1 -- 172.21.15.113:0/2128152592 shutdown_connections 2024-04-25T00:07:09.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09e1990640 1 --2- 172.21.15.113:0/2128152592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09e1990640 1 -- 172.21.15.113:0/2128152592 >> 172.21.15.113:0/2128152592 conn(0x7f09dc0cd850 msgr2=0x7f09dc0cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:09.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09e1990640 1 -- 172.21.15.113:0/2128152592 shutdown_connections 2024-04-25T00:07:09.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.040+0000 7f09e1990640 1 -- 172.21.15.113:0/2128152592 wait complete. 2024-04-25T00:07:09.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.041+0000 7f09e1990640 1 Processor -- start 2024-04-25T00:07:09.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.041+0000 7f09e1990640 1 -- start start 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09e1990640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09dc1f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09e1990640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09dc0a0ab0 0x7f09dc0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09e1990640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09dc0a0f10 0x7f09dc0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09e1990640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc0cd600 con 0x7f09c8004800 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09e1990640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc06bac0 con 0x7f09dc0a0ab0 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09e1990640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09dc0ccf70 con 0x7f09dc0a0f10 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09dc1f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09dc1f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49770/0 (socket says 172.21.15.113:49770) 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 -- 172.21.15.113:0/1340620916 learned_addr learned my addr 172.21.15.113:0/1340620916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09db7fe640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09dc0a0f10 0x7f09dc0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09d3fff640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09dc0a0ab0 0x7f09dc0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 -- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09dc0a0f10 msgr2=0x7f09dc0b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09dc0a0f10 0x7f09dc0b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 -- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09dc0a0ab0 msgr2=0x7f09dc0b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09dc0a0ab0 0x7f09dc0b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.042+0000 7f09daffd640 1 -- 172.21.15.113:0/1340620916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc15fa40 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.043+0000 7f09daffd640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09dc1f4560 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f09cc011a70 tx=0x7f09cc011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.043+0000 7f09d8ff9640 1 -- 172.21.15.113:0/1340620916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00fb50 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.043+0000 7f09d8ff9640 1 -- 172.21.15.113:0/1340620916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc00fcf0 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.043+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc15fce0 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.043+0000 7f09d8ff9640 1 -- 172.21.15.113:0/1340620916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00fea0 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.043+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc1602a0 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.044+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc0691a0 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.044+0000 7f09d8ff9640 1 -- 172.21.15.113:0/1340620916 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09cc012040 con 0x7f09c8004800 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.045+0000 7f09d8ff9640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09bc06be30 0x7f09bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.045+0000 7f09d3fff640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09bc06be30 0x7f09bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.045+0000 7f09d8ff9640 1 -- 172.21.15.113:0/1340620916 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f09cc08b620 con 0x7f09c8004800 2024-04-25T00:07:09.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.045+0000 7f09d3fff640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09bc06be30 0x7f09bc06e2f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f09c4032000 tx=0x7f09c402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.049+0000 7f09d8ff9640 1 -- 172.21.15.113:0/1340620916 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09cc019030 con 0x7f09c8004800 2024-04-25T00:07:09.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.257+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09dc0628f0 con 0x7f09c8004800 2024-04-25T00:07:09.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.257+0000 7f09d8ff9640 1 -- 172.21.15.113:0/1340620916 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+4367 (secure 0 0 0) 0x7f09cc0598f0 con 0x7f09c8004800 2024-04-25T00:07:09.258 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:09.258 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:06:20.039117+0000","modified":"2024-04-25T00:07:02.919908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":15,"addr":"172.21.15.113:6837/409819700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":409819700},{"type":"v1","addr":"172.21.15.113:6837","nonce":409819700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":9}]} 2024-04-25T00:07:09.258 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-04-25T00:07:09.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.259+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09bc06be30 msgr2=0x7f09bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.259+0000 7f09e1990640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09bc06be30 0x7f09bc06e2f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f09c4032000 tx=0x7f09c402e000 comp rx=0 tx=0).stop 2024-04-25T00:07:09.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 msgr2=0x7f09dc1f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09dc1f4560 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f09cc011a70 tx=0x7f09cc011f30 comp rx=0 tx=0).stop 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 shutdown_connections 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f09bc06be30 0x7f09bc06e2f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09dc0a0ab0 0x7f09dc0b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09dc0a0f10 0x7f09dc0b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 --2- 172.21.15.113:0/1340620916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09c8004800 0x7f09dc1f4560 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 >> 172.21.15.113:0/1340620916 conn(0x7f09dc0cd850 msgr2=0x7f09dc072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 shutdown_connections 2024-04-25T00:07:09.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.260+0000 7f09e1990640 1 -- 172.21.15.113:0/1340620916 wait complete. 2024-04-25T00:07:09.271 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:09.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.367+0000 7f1014f5a640 1 Processor -- start 2024-04-25T00:07:09.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.367+0000 7f1014f5a640 1 -- start start 2024-04-25T00:07:09.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1014f5a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10100b3fd0 0x7f101006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1014f5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f101006e510 0x7f10100cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1014f5a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1010144cf0 con 0x7f1010144910 2024-04-25T00:07:09.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1014f5a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f101006e980 con 0x7f10100b3fd0 2024-04-25T00:07:09.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1014f5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f10100b2f30 con 0x7f101006e510 2024-04-25T00:07:09.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1007fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10100b3fd0 0x7f101006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1007fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10100b3fd0 0x7f101006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49784/0 (socket says 172.21.15.113:49784) 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1007fff640 1 -- 172.21.15.113:0/2077903863 learned_addr learned my addr 172.21.15.113:0/2077903863 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f100ed9d640 1 --2- 172.21.15.113:0/2077903863 >> v2:172.21.15.178:3300/0 conn(0x7f101006e510 0x7f10100cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1007fff640 1 -- 172.21.15.113:0/2077903863 >> v2:172.21.15.178:3300/0 conn(0x7f101006e510 msgr2=0x7f10100cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1007fff640 1 --2- 172.21.15.113:0/2077903863 >> v2:172.21.15.178:3300/0 conn(0x7f101006e510 0x7f10100cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1007fff640 1 -- 172.21.15.113:0/2077903863 >> v1:172.21.15.178:6789/0 conn(0x7f1010144910 legacy=0x7f10100b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.368+0000 7f1007fff640 1 -- 172.21.15.113:0/2077903863 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10100b3210 con 0x7f10100b3fd0 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.369+0000 7f1007fff640 1 --2- 172.21.15.113:0/2077903863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10100b3fd0 0x7f101006dfd0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009870 tx=0x7f0ff802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5307cd7efea099b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.369+0000 7f100dd9b640 1 -- 172.21.15.113:0/2077903863 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8037070 con 0x7f10100b3fd0 2024-04-25T00:07:09.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.369+0000 7f100dd9b640 1 -- 172.21.15.113:0/2077903863 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff8032440 con 0x7f10100b3fd0 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.369+0000 7f100dd9b640 1 -- 172.21.15.113:0/2077903863 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8032810 con 0x7f10100b3fd0 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 -- 172.21.15.113:0/2077903863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10100b3fd0 msgr2=0x7f101006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2077903863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10100b3fd0 0x7f101006dfd0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009870 tx=0x7f0ff802a270 comp rx=0 tx=0).stop 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 -- 172.21.15.113:0/2077903863 shutdown_connections 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2077903863 >> v2:172.21.15.178:3300/0 conn(0x7f101006e510 0x7f10100cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2077903863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10100b3fd0 0x7f101006dfd0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 -- 172.21.15.113:0/2077903863 >> 172.21.15.113:0/2077903863 conn(0x7f10100b1a50 msgr2=0x7f10100b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 -- 172.21.15.113:0/2077903863 shutdown_connections 2024-04-25T00:07:09.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.370+0000 7f1014f5a640 1 -- 172.21.15.113:0/2077903863 wait complete. 2024-04-25T00:07:09.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.371+0000 7f1014f5a640 1 Processor -- start 2024-04-25T00:07:09.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.371+0000 7f1014f5a640 1 -- start start 2024-04-25T00:07:09.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1014f5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101006e510 0x7f101013d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1014f5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10100b3fd0 0x7f10101f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1014f5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010144910 0x7f1010070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1014f5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f10100b4440 con 0x7f101006e510 2024-04-25T00:07:09.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1014f5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10100b37d0 con 0x7f10100b3fd0 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1014f5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10100b4da0 con 0x7f1010144910 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101006e510 0x7f101013d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101006e510 0x7f101013d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49800/0 (socket says 172.21.15.113:49800) 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 -- 172.21.15.113:0/2821674705 learned_addr learned my addr 172.21.15.113:0/2821674705 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1007fff640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10100b3fd0 0x7f10101f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100ed9d640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010144910 0x7f1010070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 -- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010144910 msgr2=0x7f1010070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010144910 0x7f1010070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 -- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10100b3fd0 msgr2=0x7f10101f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10100b3fd0 0x7f10101f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 -- 172.21.15.113:0/2821674705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101015b860 con 0x7f101006e510 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f100e59c640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101006e510 0x7f101013d8e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f100000b3e0 tx=0x7f100000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.372+0000 7f1006ffd640 1 -- 172.21.15.113:0/2821674705 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000015020 con 0x7f101006e510 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.373+0000 7f1006ffd640 1 -- 172.21.15.113:0/2821674705 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1000011850 con 0x7f101006e510 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.373+0000 7f1006ffd640 1 -- 172.21.15.113:0/2821674705 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000011c40 con 0x7f101006e510 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.373+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10100ae660 con 0x7f101006e510 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.373+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101013d470 con 0x7f101006e510 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.373+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101013cfc0 con 0x7f101006e510 2024-04-25T00:07:09.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.374+0000 7f1006ffd640 1 -- 172.21.15.113:0/2821674705 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1000018020 con 0x7f101006e510 2024-04-25T00:07:09.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.374+0000 7f1006ffd640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ff006be30 0x7f0ff006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.374+0000 7f1006ffd640 1 -- 172.21.15.113:0/2821674705 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f100008ad50 con 0x7f101006e510 2024-04-25T00:07:09.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.374+0000 7f1007fff640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ff006be30 0x7f0ff006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.375+0000 7f1007fff640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ff006be30 0x7f0ff006e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.379+0000 7f1006ffd640 1 -- 172.21.15.113:0/2821674705 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f100008e030 con 0x7f101006e510 2024-04-25T00:07:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.552+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10100691a0 con 0x7f101006e510 2024-04-25T00:07:09.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.553+0000 7f1006ffd640 1 -- 172.21.15.113:0/2821674705 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16900 (secure 0 0 0) 0x7f100001f020 con 0x7f101006e510 2024-04-25T00:07:09.554 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:09.554 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":133,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:02.920311+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:06:18.255623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T00:06:19.003747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.12","mappings":[{"from":2,"to":0}]},{"pgid":"19.23","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6836/2664195523":"2024-04-26T00:06:59.857692+0000","172.21.15.113:6834/2664195523":"2024-04-26T00:06:59.857692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ff006be30 msgr2=0x7f0ff006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ff006be30 0x7f0ff006e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).stop 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101006e510 msgr2=0x7f101013d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101006e510 0x7f101013d8e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f100000b3e0 tx=0x7f100000b8a0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 shutdown_connections 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0ff006be30 0x7f0ff006e2f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10100b3fd0 0x7f10101f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010144910 0x7f1010070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 --2- 172.21.15.113:0/2821674705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101006e510 0x7f101013d8e0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.555+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 >> 172.21.15.113:0/2821674705 conn(0x7f10100b1a50 msgr2=0x7f10100b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:09.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.556+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 shutdown_connections 2024-04-25T00:07:09.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.556+0000 7f1014f5a640 1 -- 172.21.15.113:0/2821674705 wait complete. 2024-04-25T00:07:09.568 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:07:09.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.664+0000 7f46de464640 1 Processor -- start 2024-04-25T00:07:09.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.664+0000 7f46de464640 1 -- start start 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46de464640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d815e2d0 0x7f46d815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46de464640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46d80b3080 0x7f46d80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46de464640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d815f4f0 con 0x7f46d815e6b0 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46de464640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f46d80cde00 con 0x7f46d80b3080 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46de464640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46d815f7d0 con 0x7f46d815e2d0 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d815e2d0 0x7f46d815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d77fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46d80b3080 0x7f46d80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d77fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46d80b3080 0x7f46d80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:41216/0 (socket says 172.21.15.113:41216) 2024-04-25T00:07:09.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d77fe640 1 -- 172.21.15.113:0/1174195705 learned_addr learned my addr 172.21.15.113:0/1174195705 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d77fe640 1 -- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d815e2d0 msgr2=0x7f46d815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d77fe640 1 --2- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d815e2d0 0x7f46d815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d77fe640 1 -- 172.21.15.113:0/1174195705 >> v1:172.21.15.178:6789/0 conn(0x7f46d815e6b0 legacy=0x7f46d8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d6ffd640 1 -- 172.21.15.113:0/1174195705 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681715040 0 0) 0x7f46d815f4f0 con 0x7f46d815e6b0 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d77fe640 1 -- 172.21.15.113:0/1174195705 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d815fab0 con 0x7f46d80b3080 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.665+0000 7f46d7fff640 1 --2- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d815e2d0 0x7f46d815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.666+0000 7f46d77fe640 1 --2- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d80b3080 0x7f46d80b4330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009870 tx=0x7f46c802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8fea866e9ccc16f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.666+0000 7f46d6ffd640 1 -- 172.21.15.113:0/1174195705 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8037070 con 0x7f46d80b3080 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.666+0000 7f46d6ffd640 1 -- 172.21.15.113:0/1174195705 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8032440 con 0x7f46d80b3080 2024-04-25T00:07:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.666+0000 7f46d6ffd640 1 -- 172.21.15.113:0/1174195705 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8032810 con 0x7f46d80b3080 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 -- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d80b3080 msgr2=0x7f46d80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 --2- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d80b3080 0x7f46d80b4330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009870 tx=0x7f46c802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 -- 172.21.15.113:0/1174195705 shutdown_connections 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 --2- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d80b3080 0x7f46d80b4330 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 --2- 172.21.15.113:0/1174195705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d815e2d0 0x7f46d815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 -- 172.21.15.113:0/1174195705 >> 172.21.15.113:0/1174195705 conn(0x7f46d806b0f0 msgr2=0x7f46d806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 -- 172.21.15.113:0/1174195705 shutdown_connections 2024-04-25T00:07:09.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.667+0000 7f46de464640 1 -- 172.21.15.113:0/1174195705 wait complete. 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.668+0000 7f46de464640 1 Processor -- start 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.668+0000 7f46de464640 1 -- start start 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46de464640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d80b3080 0x7f46d8146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46de464640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46d815e2d0 0x7f46d8168130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46de464640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d815e6b0 0x7f46d8152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46de464640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f46d815f2b0 con 0x7f46d815e2d0 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46de464640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46d815f130 con 0x7f46d815e6b0 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46de464640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46d80cf2a0 con 0x7f46d80b3080 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46d815e2d0 0x7f46d8168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46d815e2d0 0x7f46d8168130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49810/0 (socket says 172.21.15.113:49810) 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 -- 172.21.15.113:0/4035167791 learned_addr learned my addr 172.21.15.113:0/4035167791 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:09.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46dca01640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d815e6b0 0x7f46d8152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d7fff640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d80b3080 0x7f46d8146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 -- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d80b3080 msgr2=0x7f46d8146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d80b3080 0x7f46d8146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 -- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d815e6b0 msgr2=0x7f46d8152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d815e6b0 0x7f46d8152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.669+0000 7f46d77fe640 1 -- 172.21.15.113:0/4035167791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d8152680 con 0x7f46d815e2d0 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.670+0000 7f46d77fe640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46d815e2d0 0x7f46d8168130 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f46c802a200 tx=0x7f46c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.670+0000 7f46d4ff9640 1 -- 172.21.15.113:0/4035167791 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8037040 con 0x7f46d815e2d0 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.670+0000 7f46d4ff9640 1 -- 172.21.15.113:0/4035167791 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8042070 con 0x7f46d815e2d0 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.670+0000 7f46d4ff9640 1 -- 172.21.15.113:0/4035167791 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8030880 con 0x7f46d815e2d0 2024-04-25T00:07:09.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.670+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d8152960 con 0x7f46d815e2d0 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.670+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d8168670 con 0x7f46d815e2d0 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.671+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d80691a0 con 0x7f46d815e2d0 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.671+0000 7f46d4ff9640 1 -- 172.21.15.113:0/4035167791 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46c802ad10 con 0x7f46d815e2d0 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.671+0000 7f46d4ff9640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f46c406be30 0x7f46c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.672+0000 7f46d4ff9640 1 -- 172.21.15.113:0/4035167791 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5853+0+0 (secure 0 0 0) 0x7f46c80ab2c0 con 0x7f46d815e2d0 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.672+0000 7f46d7fff640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f46c406be30 0x7f46c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.672+0000 7f46d7fff640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f46c406be30 0x7f46c406e2f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009560 tx=0x7f46cc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:09.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.676+0000 7f46d4ff9640 1 -- 172.21.15.113:0/4035167791 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46c8079620 con 0x7f46d815e2d0 2024-04-25T00:07:09.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.887+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f46d8152b30 con 0x7f46d815e2d0 2024-04-25T00:07:09.888 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:07:09.888+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:07:09.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.889+0000 7f46d4ff9640 1 -- 172.21.15.113:0/4035167791 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) ==== 161+0+0 (secure 0 0 0) 0x7f46c807d920 con 0x7f46d815e2d0 2024-04-25T00:07:09.889 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:07:09.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f46c406be30 msgr2=0x7f46c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f46c406be30 0x7f46c406e2f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009560 tx=0x7f46cc009190 comp rx=0 tx=0).stop 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46d815e2d0 msgr2=0x7f46d8168130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46d815e2d0 0x7f46d8168130 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f46c802a200 tx=0x7f46c80046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 shutdown_connections 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f46c406be30 0x7f46c406e2f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d815e6b0 0x7f46d8152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46d815e2d0 0x7f46d8168130 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 --2- 172.21.15.113:0/4035167791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d80b3080 0x7f46d8146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 >> 172.21.15.113:0/4035167791 conn(0x7f46d806b0f0 msgr2=0x7f46d80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 shutdown_connections 2024-04-25T00:07:09.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:09.893+0000 7f46de464640 1 -- 172.21.15.113:0/4035167791 wait complete. 2024-04-25T00:07:09.905 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:07:09.906 INFO:tasks.ceph.mds.a.smithi113.stderr: -163> 2024-04-25T00:07:05.960+0000 7f41e6653640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:07:09.930 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:07:10.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.005+0000 7fb5cef70640 1 Processor -- start 2024-04-25T00:07:10.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.005+0000 7fb5cef70640 1 -- start start 2024-04-25T00:07:10.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cef70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cef70640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5c81606a0 0x7fb5c80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cef70640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c80d0970 con 0x7fb5c80d04f0 2024-04-25T00:07:10.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cef70640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c815c6a0 con 0x7fb5c81606a0 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cef70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c8160aa0 con 0x7fb5c813f050 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cd50d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5c81606a0 0x7fb5c80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5c7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb5c80d04f0 0x7fb5c815ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36222/0 (socket says 172.21.15.113:36222) 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5c7fff640 1 -- 172.21.15.113:0/859976859 learned_addr learned my addr 172.21.15.113:0/859976859 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cd50d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5c81606a0 0x7fb5c80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49840/0 (socket says 172.21.15.113:49840) 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5ccd0c640 1 --2- 172.21.15.113:0/859976859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cd50d640 1 -- 172.21.15.113:0/859976859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 msgr2=0x7fb5c80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cd50d640 1 --2- 172.21.15.113:0/859976859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c80cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cd50d640 1 -- 172.21.15.113:0/859976859 >> v1:172.21.15.113:6789/0 conn(0x7fb5c80d04f0 legacy=0x7fb5c815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5cd50d640 1 -- 172.21.15.113:0/859976859 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c815d6b0 con 0x7fb5c81606a0 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.006+0000 7fb5c77fe640 1 -- 172.21.15.113:0/859976859 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4167834020 0 0) 0x7fb5c80d0970 con 0x7fb5c80d04f0 2024-04-25T00:07:10.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.007+0000 7fb5cd50d640 1 --2- 172.21.15.113:0/859976859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c80cea60 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009d90 tx=0x7fb5c002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdf0cb71bff1701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.007+0000 7fb5c77fe640 1 -- 172.21.15.113:0/859976859 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0031600 con 0x7fb5c81606a0 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.007+0000 7fb5c77fe640 1 -- 172.21.15.113:0/859976859 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c00317a0 con 0x7fb5c81606a0 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.007+0000 7fb5c77fe640 1 -- 172.21.15.113:0/859976859 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0031b70 con 0x7fb5c81606a0 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.007+0000 7fb5cef70640 1 -- 172.21.15.113:0/859976859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 msgr2=0x7fb5c80cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.007+0000 7fb5cef70640 1 --2- 172.21.15.113:0/859976859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c80cea60 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009d90 tx=0x7fb5c002a5d0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.008+0000 7fb5cef70640 1 -- 172.21.15.113:0/859976859 shutdown_connections 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.008+0000 7fb5cef70640 1 --2- 172.21.15.113:0/859976859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c80cea60 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.008+0000 7fb5cef70640 1 --2- 172.21.15.113:0/859976859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c80cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.008+0000 7fb5cef70640 1 -- 172.21.15.113:0/859976859 >> 172.21.15.113:0/859976859 conn(0x7fb5c806b0f0 msgr2=0x7fb5c806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.008+0000 7fb5cef70640 1 -- 172.21.15.113:0/859976859 shutdown_connections 2024-04-25T00:07:10.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.008+0000 7fb5cef70640 1 -- 172.21.15.113:0/859976859 wait complete. 2024-04-25T00:07:10.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.009+0000 7fb5cef70640 1 Processor -- start 2024-04-25T00:07:10.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.009+0000 7fb5cef70640 1 -- start start 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c80d04f0 0x7fb5c81f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c81f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c8071a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c815ceb0 con 0x7fb5c81606a0 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c815cbd0 con 0x7fb5c80d04f0 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c815d190 con 0x7fb5c813f050 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c8071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c8071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49844/0 (socket says 172.21.15.113:49844) 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 -- 172.21.15.113:0/196141993 learned_addr learned my addr 172.21.15.113:0/196141993 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5ccd0c640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c80d04f0 0x7fb5c81f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5c7fff640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c81f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 -- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 msgr2=0x7fb5c81f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c81f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 -- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c80d04f0 msgr2=0x7fb5c81f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c80d04f0 0x7fb5c81f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 -- 172.21.15.113:0/196141993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c81f4870 con 0x7fb5c81606a0 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cd50d640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c8071a50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb5c002ab40 tx=0x7fb5c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5c57fa640 1 -- 172.21.15.113:0/196141993 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c003f070 con 0x7fb5c81606a0 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5c57fa640 1 -- 172.21.15.113:0/196141993 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c0030b60 con 0x7fb5c81606a0 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c81f4b50 con 0x7fb5c81606a0 2024-04-25T00:07:10.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5c57fa640 1 -- 172.21.15.113:0/196141993 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0009450 con 0x7fb5c81606a0 2024-04-25T00:07:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.010+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c81f5110 con 0x7fb5c81606a0 2024-04-25T00:07:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.011+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c80691a0 con 0x7fb5c81606a0 2024-04-25T00:07:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.012+0000 7fb5c57fa640 1 -- 172.21.15.113:0/196141993 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5c003b410 con 0x7fb5c81606a0 2024-04-25T00:07:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.012+0000 7fb5c57fa640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb5ac06be30 0x7fb5ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.012+0000 7fb5c57fa640 1 -- 172.21.15.113:0/196141993 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5939+0+0 (secure 0 0 0) 0x7fb5c00abf40 con 0x7fb5c81606a0 2024-04-25T00:07:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.012+0000 7fb5ccd0c640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb5ac06be30 0x7fb5ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.013+0000 7fb5ccd0c640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb5ac06be30 0x7fb5ac06e2f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc006fd0 tx=0x7fb5bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.017+0000 7fb5c57fa640 1 -- 172.21.15.113:0/196141993 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb5c007a130 con 0x7fb5c81606a0 2024-04-25T00:07:10.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.228+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb5c813e7d0 con 0x7fb5c81606a0 2024-04-25T00:07:10.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.891+0000 7fb5c57fa640 1 -- 172.21.15.113:0/196141993 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7fb5c007e430 con 0x7fb5c81606a0 2024-04-25T00:07:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb5ac06be30 msgr2=0x7fb5ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb5ac06be30 0x7fb5ac06e2f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc006fd0 tx=0x7fb5bc008040 comp rx=0 tx=0).stop 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 msgr2=0x7fb5c8071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c8071a50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb5c002ab40 tx=0x7fb5c0004a40 comp rx=0 tx=0).stop 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 shutdown_connections 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb5ac06be30 0x7fb5ac06e2f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5c81606a0 0x7fb5c8071a50 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5c813f050 0x7fb5c81f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 --2- 172.21.15.113:0/196141993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c80d04f0 0x7fb5c81f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 >> 172.21.15.113:0/196141993 conn(0x7fb5c806b0f0 msgr2=0x7fb5c813ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 shutdown_connections 2024-04-25T00:07:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:10.893+0000 7fb5cef70640 1 -- 172.21.15.113:0/196141993 wait complete. 2024-04-25T00:07:10.907 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:07:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb68ac58640 1 Processor -- start 2024-04-25T00:07:11.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb68ac58640 1 -- start start 2024-04-25T00:07:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb68ac58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6840b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb68ac58640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6841626f0 con 0x7fb68415e510 2024-04-25T00:07:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb68ac58640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6841629d0 con 0x7fb6840b61e0 2024-04-25T00:07:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb68ac58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb684162cb0 con 0x7fb6840b9bb0 2024-04-25T00:07:11.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb6889f4640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb68415e510 0x7fb684161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36238/0 (socket says 172.21.15.113:36238) 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.050+0000 7fb6889f4640 1 -- 172.21.15.113:0/3370055050 learned_addr learned my addr 172.21.15.113:0/3370055050 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb683fff640 1 --2- 172.21.15.113:0/3370055050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6840b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2791244673 0 0) 0x7fb6841626f0 con 0x7fb68415e510 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb66c003520 con 0x7fb68415e510 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752789757 0 0) 0x7fb6841629d0 con 0x7fb6840b61e0 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6841626f0 con 0x7fb6840b61e0 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb683fff640 1 -- 172.21.15.113:0/3370055050 >> v1:172.21.15.178:6789/0 conn(0x7fb6840b61e0 legacy=0x7fb6840d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb683fff640 1 -- 172.21.15.113:0/3370055050 >> v1:172.21.15.113:6789/0 conn(0x7fb68415e510 legacy=0x7fb684161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1156271571 0 0) 0x7fb66c003520 con 0x7fb68415e510 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.051+0000 7fb683fff640 1 -- 172.21.15.113:0/3370055050 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6840d2660 con 0x7fb6840b9bb0 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb683fff640 1 --2- 172.21.15.113:0/3370055050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6840b5ca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb6780097d0 tx=0x7fb67802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20901ccca8659d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678037070 con 0x7fb6840b9bb0 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67802ab90 con 0x7fb6840b9bb0 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb6837fe640 1 -- 172.21.15.113:0/3370055050 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674001670 con 0x7fb6840b9bb0 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb68ac58640 1 -- 172.21.15.113:0/3370055050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 msgr2=0x7fb6840b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3370055050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6840b5ca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb6780097d0 tx=0x7fb67802a100 comp rx=0 tx=0).stop 2024-04-25T00:07:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb68ac58640 1 -- 172.21.15.113:0/3370055050 shutdown_connections 2024-04-25T00:07:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3370055050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6840b5ca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.052+0000 7fb68ac58640 1 -- 172.21.15.113:0/3370055050 >> 172.21.15.113:0/3370055050 conn(0x7fb68406b720 msgr2=0x7fb6840d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.053+0000 7fb68ac58640 1 -- 172.21.15.113:0/3370055050 shutdown_connections 2024-04-25T00:07:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.053+0000 7fb68ac58640 1 -- 172.21.15.113:0/3370055050 wait complete. 2024-04-25T00:07:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.054+0000 7fb68ac58640 1 Processor -- start 2024-04-25T00:07:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.054+0000 7fb68ac58640 1 -- start start 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.054+0000 7fb68ac58640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6840b61e0 0x7fb684149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.054+0000 7fb68ac58640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6841609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb68ac58640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb68415e510 0x7fb6840c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb68ac58640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6840b7540 con 0x7fb68415e510 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb68ac58640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6840b73c0 con 0x7fb6840b61e0 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb68ac58640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6840d3570 con 0x7fb6840b9bb0 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb68415e510 0x7fb6840c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb68415e510 0x7fb6840c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49856/0 (socket says 172.21.15.113:49856) 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 -- 172.21.15.113:0/3968179403 learned_addr learned my addr 172.21.15.113:0/3968179403 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb683fff640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6841609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6889f4640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6840b61e0 0x7fb684149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 -- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 msgr2=0x7fb6841609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6841609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 -- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6840b61e0 msgr2=0x7fb684149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6840b61e0 0x7fb684149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 -- 172.21.15.113:0/3968179403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68415f390 con 0x7fb68415e510 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6891f5640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb68415e510 0x7fb6840c9500 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00c790 tx=0x7fb67c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6817fa640 1 -- 172.21.15.113:0/3968179403 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c003b70 con 0x7fb68415e510 2024-04-25T00:07:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6817fa640 1 -- 172.21.15.113:0/3968179403 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c003d10 con 0x7fb68415e510 2024-04-25T00:07:11.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68415f670 con 0x7fb68415e510 2024-04-25T00:07:11.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb6817fa640 1 -- 172.21.15.113:0/3968179403 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00aa50 con 0x7fb68415e510 2024-04-25T00:07:11.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.055+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68415fc30 con 0x7fb68415e510 2024-04-25T00:07:11.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.056+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6840691a0 con 0x7fb68415e510 2024-04-25T00:07:11.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.057+0000 7fb6817fa640 1 -- 172.21.15.113:0/3968179403 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb67c00abf0 con 0x7fb68415e510 2024-04-25T00:07:11.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.057+0000 7fb6817fa640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb67006be30 0x7fb67006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:11.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.057+0000 7fb6889f4640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb67006be30 0x7fb67006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:11.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.057+0000 7fb6817fa640 1 -- 172.21.15.113:0/3968179403 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5939+0+0 (secure 0 0 0) 0x7fb67c08ae10 con 0x7fb68415e510 2024-04-25T00:07:11.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.057+0000 7fb6889f4640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb67006be30 0x7fb67006e2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb6740017e0 tx=0x7fb674001610 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:11.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.061+0000 7fb6817fa640 1 -- 172.21.15.113:0/3968179403 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb67c059080 con 0x7fb68415e510 2024-04-25T00:07:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.236+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb6840b5c70 con 0x7fb68415e510 2024-04-25T00:07:11.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.953+0000 7fb6817fa640 1 -- 172.21.15.113:0/3968179403 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) ==== 181+0+0 (secure 0 0 0) 0x7fb67c05d380 con 0x7fb68415e510 2024-04-25T00:07:11.954 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:07:11.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb67006be30 msgr2=0x7fb67006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb67006be30 0x7fb67006e2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb6740017e0 tx=0x7fb674001610 comp rx=0 tx=0).stop 2024-04-25T00:07:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb68415e510 msgr2=0x7fb6840c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb68415e510 0x7fb6840c9500 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00c790 tx=0x7fb67c00cc50 comp rx=0 tx=0).stop 2024-04-25T00:07:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 shutdown_connections 2024-04-25T00:07:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb67006be30 0x7fb67006e2f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb68415e510 0x7fb6840c9500 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6840b9bb0 0x7fb6841609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 --2- 172.21.15.113:0/3968179403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6840b61e0 0x7fb684149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 >> 172.21.15.113:0/3968179403 conn(0x7fb68406b720 msgr2=0x7fb6840d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.959+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 shutdown_connections 2024-04-25T00:07:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:11.960+0000 7fb68ac58640 1 -- 172.21.15.113:0/3968179403 wait complete. 2024-04-25T00:07:11.975 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:07:12.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.080+0000 7f0047129640 1 Processor -- start 2024-04-25T00:07:12.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.080+0000 7f0047129640 1 -- start start 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f0047129640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f004015e510 0x7f0040161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f0047129640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00401626f0 con 0x7f00400b63b0 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f0047129640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00401629d0 con 0x7f00400b9bb0 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f0047129640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0040162cb0 con 0x7f004015e510 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f0044ec5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f004015e510 0x7f0040161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f00456c6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f00400b63b0 0x7f00400d2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36250/0 (socket says 172.21.15.113:36250) 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f0044ec5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f004015e510 0x7f0040161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49872/0 (socket says 172.21.15.113:49872) 2024-04-25T00:07:12.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.081+0000 7f00456c6640 1 -- 172.21.15.113:0/997846071 learned_addr learned my addr 172.21.15.113:0/997846071 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:12.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.082+0000 7f0044ec5640 1 -- 172.21.15.113:0/997846071 >> v1:172.21.15.178:6789/0 conn(0x7f00400b9bb0 legacy=0x7f00400b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.082+0000 7f0044ec5640 1 -- 172.21.15.113:0/997846071 >> v1:172.21.15.113:6789/0 conn(0x7f00400b63b0 legacy=0x7f00400d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.082+0000 7f0044ec5640 1 -- 172.21.15.113:0/997846071 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00400d2660 con 0x7f004015e510 2024-04-25T00:07:12.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.082+0000 7f003f7fe640 1 -- 172.21.15.113:0/997846071 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3204519608 0 0) 0x7f00401626f0 con 0x7f00400b63b0 2024-04-25T00:07:12.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.082+0000 7f0044ec5640 1 --2- 172.21.15.113:0/997846071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004015e510 0x7f0040161ea0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f00340097a0 tx=0x7f003402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec1980d34c6a6ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:12.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f003f7fe640 1 -- 172.21.15.113:0/997846071 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034037070 con 0x7f004015e510 2024-04-25T00:07:12.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f003f7fe640 1 -- 172.21.15.113:0/997846071 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0034032440 con 0x7f004015e510 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f003f7fe640 1 -- 172.21.15.113:0/997846071 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034032810 con 0x7f004015e510 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f0047129640 1 -- 172.21.15.113:0/997846071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004015e510 msgr2=0x7f0040161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f0047129640 1 --2- 172.21.15.113:0/997846071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004015e510 0x7f0040161ea0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f00340097a0 tx=0x7f003402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f0047129640 1 -- 172.21.15.113:0/997846071 shutdown_connections 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f0047129640 1 --2- 172.21.15.113:0/997846071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f004015e510 0x7f0040161ea0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f0047129640 1 -- 172.21.15.113:0/997846071 >> 172.21.15.113:0/997846071 conn(0x7f004006b720 msgr2=0x7f00400d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f0047129640 1 -- 172.21.15.113:0/997846071 shutdown_connections 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.083+0000 7f0047129640 1 -- 172.21.15.113:0/997846071 wait complete. 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.084+0000 7f0047129640 1 Processor -- start 2024-04-25T00:07:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.084+0000 7f0047129640 1 -- start start 2024-04-25T00:07:12.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f0047129640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00400b63b0 0x7f004014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:12.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f0047129640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00400b9bb0 0x7f0040153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:12.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f0047129640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f004015e510 0x7f00401538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:12.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f0047129640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00400b73c0 con 0x7f00400b9bb0 2024-04-25T00:07:12.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f0047129640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00401623e0 con 0x7f004015e510 2024-04-25T00:07:12.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f0047129640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f00400b7540 con 0x7f00400b63b0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f003ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00400b9bb0 0x7f0040153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f003ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00400b9bb0 0x7f0040153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49888/0 (socket says 172.21.15.113:49888) 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.085+0000 7f003ffff640 1 -- 172.21.15.113:0/3532445615 learned_addr learned my addr 172.21.15.113:0/3532445615 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f0044ec5640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00400b63b0 0x7f004014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f00456c6640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f004015e510 0x7f00401538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003ffff640 1 -- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00400b63b0 msgr2=0x7f004014ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003ffff640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00400b63b0 0x7f004014ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003ffff640 1 -- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f004015e510 msgr2=0x7f00401538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003ffff640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f004015e510 0x7f00401538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003ffff640 1 -- 172.21.15.113:0/3532445615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00401f3f10 con 0x7f00400b9bb0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003ffff640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00400b9bb0 0x7f0040153380 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c400 tx=0x7f002c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003d7fa640 1 -- 172.21.15.113:0/3532445615 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c017070 con 0x7f00400b9bb0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003d7fa640 1 -- 172.21.15.113:0/3532445615 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c0052d0 con 0x7f00400b9bb0 2024-04-25T00:07:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f003d7fa640 1 -- 172.21.15.113:0/3532445615 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c003830 con 0x7f00400b9bb0 2024-04-25T00:07:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00401f41f0 con 0x7f00400b9bb0 2024-04-25T00:07:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.086+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00401f47b0 con 0x7f00400b9bb0 2024-04-25T00:07:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.087+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00400691a0 con 0x7f00400b9bb0 2024-04-25T00:07:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.087+0000 7f003d7fa640 1 -- 172.21.15.113:0/3532445615 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f002c01a070 con 0x7f00400b9bb0 2024-04-25T00:07:12.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.088+0000 7f003d7fa640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:12.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.088+0000 7f003d7fa640 1 -- 172.21.15.113:0/3532445615 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5501+0+0 (secure 0 0 0) 0x7f002c08a230 con 0x7f00400b9bb0 2024-04-25T00:07:12.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.088+0000 7f0044ec5640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:12.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.088+0000 7f0044ec5640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0034034000 tx=0x7f003402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:12.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.092+0000 7f003d7fa640 1 -- 172.21.15.113:0/3532445615 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f002c058c10 con 0x7f00400b9bb0 2024-04-25T00:07:12.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.266+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f00400d1f30 con 0x7f00400b9bb0 2024-04-25T00:07:12.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.955+0000 7f003d7fa640 1 -- 172.21.15.113:0/3532445615 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) ==== 169+0+0 (secure 0 0 0) 0x7f002c058510 con 0x7f00400b9bb0 2024-04-25T00:07:12.956 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:07:12.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.961+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f002406be30 msgr2=0x7f002406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.961+0000 7f0047129640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0034034000 tx=0x7f003402e000 comp rx=0 tx=0).stop 2024-04-25T00:07:12.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.961+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00400b9bb0 msgr2=0x7f0040153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.961+0000 7f0047129640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00400b9bb0 0x7f0040153380 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c400 tx=0x7f002c00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 shutdown_connections 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f004015e510 0x7f00401538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00400b9bb0 0x7f0040153380 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 --2- 172.21.15.113:0/3532445615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00400b63b0 0x7f004014ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 >> 172.21.15.113:0/3532445615 conn(0x7f004006b720 msgr2=0x7f00400d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 shutdown_connections 2024-04-25T00:07:12.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:12.962+0000 7f0047129640 1 -- 172.21.15.113:0/3532445615 wait complete. 2024-04-25T00:07:12.981 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:07:13.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.081+0000 7ff68ae64640 1 Processor -- start 2024-04-25T00:07:13.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.081+0000 7ff68ae64640 1 -- start start 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.082+0000 7ff68ae64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff6840b5ca0 0x7ff6840b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.082+0000 7ff68ae64640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6840d2e00 con 0x7ff68415e510 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.082+0000 7ff68ae64640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6840d30e0 con 0x7ff6840b73c0 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.082+0000 7ff68ae64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff6840b6640 con 0x7ff6840b5ca0 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.082+0000 7ff688c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff68415e510 0x7ff6840d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36260/0 (socket says 172.21.15.113:36260) 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.082+0000 7ff683fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff6840b5ca0 0x7ff6840b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.082+0000 7ff688c00640 1 -- 172.21.15.113:0/3032334387 learned_addr learned my addr 172.21.15.113:0/3032334387 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff683fff640 1 -- 172.21.15.113:0/3032334387 >> v1:172.21.15.178:6789/0 conn(0x7ff6840b73c0 legacy=0x7ff6840b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff683fff640 1 -- 172.21.15.113:0/3032334387 >> v1:172.21.15.113:6789/0 conn(0x7ff68415e510 legacy=0x7ff6840d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff683fff640 1 -- 172.21.15.113:0/3032334387 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6840d33c0 con 0x7ff6840b5ca0 2024-04-25T00:07:13.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff6837fe640 1 -- 172.21.15.113:0/3032334387 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086875982 0 0) 0x7ff6840d2e00 con 0x7ff68415e510 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff683fff640 1 --2- 172.21.15.113:0/3032334387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6840b5ca0 0x7ff6840b6100 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0097a0 tx=0x7ff66c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0fbb19dd4c1ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff6837fe640 1 -- 172.21.15.113:0/3032334387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c037070 con 0x7ff6840b5ca0 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff6837fe640 1 -- 172.21.15.113:0/3032334387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66c02ae50 con 0x7ff6840b5ca0 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.083+0000 7ff6837fe640 1 -- 172.21.15.113:0/3032334387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c032660 con 0x7ff6840b5ca0 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.084+0000 7ff68ae64640 1 -- 172.21.15.113:0/3032334387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6840b5ca0 msgr2=0x7ff6840b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.084+0000 7ff68ae64640 1 --2- 172.21.15.113:0/3032334387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6840b5ca0 0x7ff6840b6100 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0097a0 tx=0x7ff66c02a140 comp rx=0 tx=0).stop 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.084+0000 7ff68ae64640 1 -- 172.21.15.113:0/3032334387 shutdown_connections 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.084+0000 7ff68ae64640 1 --2- 172.21.15.113:0/3032334387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6840b5ca0 0x7ff6840b6100 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.084+0000 7ff68ae64640 1 -- 172.21.15.113:0/3032334387 >> 172.21.15.113:0/3032334387 conn(0x7ff68406b720 msgr2=0x7ff6840d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:13.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.084+0000 7ff68ae64640 1 -- 172.21.15.113:0/3032334387 shutdown_connections 2024-04-25T00:07:13.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.084+0000 7ff68ae64640 1 -- 172.21.15.113:0/3032334387 wait complete. 2024-04-25T00:07:13.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.085+0000 7ff68ae64640 1 Processor -- start 2024-04-25T00:07:13.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.085+0000 7ff68ae64640 1 -- start start 2024-04-25T00:07:13.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff68ae64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6840b5ca0 0x7ff6840c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff68ae64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6840b73c0 0x7ff6841f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff68ae64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68415e510 0x7ff6840ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff68ae64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6840d3ea0 con 0x7ff68415e510 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff68ae64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6840d3980 con 0x7ff6840b73c0 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff68ae64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff6840d2a90 con 0x7ff6840b5ca0 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68415e510 0x7ff6840ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68415e510 0x7ff6840ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49906/0 (socket says 172.21.15.113:49906) 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 -- 172.21.15.113:0/484037682 learned_addr learned my addr 172.21.15.113:0/484037682 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 -- 172.21.15.113:0/484037682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6840b5ca0 msgr2=0x7ff6840c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6840b5ca0 0x7ff6840c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 -- 172.21.15.113:0/484037682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6840b73c0 msgr2=0x7ff6841f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff683fff640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6840b73c0 0x7ff6841f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6840b73c0 0x7ff6841f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.086+0000 7ff689401640 1 -- 172.21.15.113:0/484037682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6841f4620 con 0x7ff68415e510 2024-04-25T00:07:13.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.087+0000 7ff689401640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68415e510 0x7ff6840ce970 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c780 tx=0x7ff67c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:13.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.087+0000 7ff6817fa640 1 -- 172.21.15.113:0/484037682 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c017070 con 0x7ff68415e510 2024-04-25T00:07:13.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.087+0000 7ff6817fa640 1 -- 172.21.15.113:0/484037682 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff67c003da0 con 0x7ff68415e510 2024-04-25T00:07:13.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.087+0000 7ff6817fa640 1 -- 172.21.15.113:0/484037682 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0052b0 con 0x7ff68415e510 2024-04-25T00:07:13.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.087+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6841f4900 con 0x7ff68415e510 2024-04-25T00:07:13.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.087+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6840c7200 con 0x7ff68415e510 2024-04-25T00:07:13.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.088+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6840691a0 con 0x7ff68415e510 2024-04-25T00:07:13.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.088+0000 7ff6817fa640 1 -- 172.21.15.113:0/484037682 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff67c01a070 con 0x7ff68415e510 2024-04-25T00:07:13.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.089+0000 7ff6817fa640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff66406be30 0x7ff66406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.089+0000 7ff6817fa640 1 -- 172.21.15.113:0/484037682 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5029+0+0 (secure 0 0 0) 0x7ff67c08a280 con 0x7ff68415e510 2024-04-25T00:07:13.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.089+0000 7ff688c00640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff66406be30 0x7ff66406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.089+0000 7ff688c00640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff66406be30 0x7ff66406e2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff674007fd0 tx=0x7ff674007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:13.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.093+0000 7ff6817fa640 1 -- 172.21.15.113:0/484037682 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff67c058ed0 con 0x7ff68415e510 2024-04-25T00:07:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.269+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff68406c100 con 0x7ff68415e510 2024-04-25T00:07:13.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.273+0000 7ff6817fa640 1 -- 172.21.15.113:0/484037682 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7ff67c0587d0 con 0x7ff68415e510 2024-04-25T00:07:13.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff66406be30 msgr2=0x7ff66406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff66406be30 0x7ff66406e2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff674007fd0 tx=0x7ff674007920 comp rx=0 tx=0).stop 2024-04-25T00:07:13.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68415e510 msgr2=0x7ff6840ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68415e510 0x7ff6840ce970 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c780 tx=0x7ff67c00cc40 comp rx=0 tx=0).stop 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 shutdown_connections 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff66406be30 0x7ff66406e2f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6840b73c0 0x7ff6841f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68415e510 0x7ff6840ce970 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 --2- 172.21.15.113:0/484037682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6840b5ca0 0x7ff6840c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 >> 172.21.15.113:0/484037682 conn(0x7ff68406b720 msgr2=0x7ff6840b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 shutdown_connections 2024-04-25T00:07:13.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.275+0000 7ff68ae64640 1 -- 172.21.15.113:0/484037682 wait complete. 2024-04-25T00:07:13.286 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T00:07:13.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.383+0000 7f2c44342640 1 Processor -- start 2024-04-25T00:07:13.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.383+0000 7f2c44342640 1 -- start start 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c44342640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2c3c0b4b00 0x7f2c3c1656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c44342640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3c071b20 con 0x7f2c3c15a6b0 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c44342640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3c0b2d50 con 0x7f2c3c072ea0 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c44342640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3c072a70 con 0x7f2c3c0b4b00 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c420de640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2c3c15a6b0 0x7f2c3c070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36274/0 (socket says 172.21.15.113:36274) 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c420de640 1 -- 172.21.15.113:0/2556466127 learned_addr learned my addr 172.21.15.113:0/2556466127 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416885085 0 0) 0x7f2c3c071b20 con 0x7f2c3c15a6b0 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c28003520 con 0x7f2c3c15a6b0 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c428df640 1 --2- 172.21.15.113:0/2556466127 >> v2:172.21.15.178:3300/0 conn(0x7f2c3c0b4b00 0x7f2c3c1656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3902333797 0 0) 0x7f2c28003520 con 0x7f2c3c15a6b0 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 >> v2:172.21.15.178:3300/0 conn(0x7f2c3c0b4b00 msgr2=0x7f2c3c1656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 --2- 172.21.15.113:0/2556466127 >> v2:172.21.15.178:3300/0 conn(0x7f2c3c0b4b00 0x7f2c3c1656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 >> v1:172.21.15.178:6789/0 conn(0x7f2c3c072ea0 legacy=0x7f2c3c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c0b3010 con 0x7f2c3c15a6b0 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.384+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f2c38001670 con 0x7f2c3c15a6b0 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 >> v1:172.21.15.113:6789/0 conn(0x7f2c3c15a6b0 legacy=0x7f2c3c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c410dc640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280039b0 0x7f2c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c410dc640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c28004800 0x7f2c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c410dc640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c280051c0 0x7f2c2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c072a70 con 0x7f2c280051c0 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2800c060 con 0x7f2c28004800 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2800c260 con 0x7f2c280039b0 2024-04-25T00:07:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c428df640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c280051c0 0x7f2c2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c420de640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280039b0 0x7f2c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c418dd640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c28004800 0x7f2c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c428df640 1 -- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280039b0 msgr2=0x7f2c28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c428df640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280039b0 0x7f2c28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c428df640 1 -- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c28004800 msgr2=0x7f2c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c428df640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c28004800 0x7f2c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c428df640 1 -- 172.21.15.113:0/2556466127 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c28003520 con 0x7f2c280051c0 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.385+0000 7f2c420de640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280039b0 0x7f2c28003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c428df640 1 -- 172.21.15.113:0/2556466127 reap_dead start 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c428df640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c280051c0 0x7f2c2800b9c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009fc0 tx=0x7f2c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e7e004d2aabe8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035020 con 0x7f2c280051c0 2024-04-25T00:07:13.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c34003bd0 con 0x7f2c280051c0 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c410dc640 1 -- 172.21.15.113:0/2556466127 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34009c10 con 0x7f2c280051c0 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c44342640 1 -- 172.21.15.113:0/2556466127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c280051c0 msgr2=0x7f2c2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c44342640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c280051c0 0x7f2c2800b9c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009fc0 tx=0x7f2c28005830 comp rx=0 tx=0).stop 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c44342640 1 -- 172.21.15.113:0/2556466127 shutdown_connections 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c44342640 1 --2- 172.21.15.113:0/2556466127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c280051c0 0x7f2c2800b9c0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c44342640 1 -- 172.21.15.113:0/2556466127 >> 172.21.15.113:0/2556466127 conn(0x7f2c3c0b1a50 msgr2=0x7f2c3c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c44342640 1 -- 172.21.15.113:0/2556466127 shutdown_connections 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.386+0000 7f2c44342640 1 -- 172.21.15.113:0/2556466127 wait complete. 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.387+0000 7f2c44342640 1 Processor -- start 2024-04-25T00:07:13.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c44342640 1 -- start start 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c44342640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280051c0 0x7f2c3c1f0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c44342640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c3c14eb70 0x7f2c3c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c44342640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c3c14eff0 0x7f2c3c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c44342640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c072720 con 0x7f2c3c14eff0 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c44342640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0b35d0 con 0x7f2c3c14eb70 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c44342640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c15aa90 con 0x7f2c280051c0 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c428df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c3c14eff0 0x7f2c3c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c428df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c3c14eff0 0x7f2c3c1589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49922/0 (socket says 172.21.15.113:49922) 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.388+0000 7f2c428df640 1 -- 172.21.15.113:0/2803986079 learned_addr learned my addr 172.21.15.113:0/2803986079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c418dd640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c3c14eb70 0x7f2c3c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c420de640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280051c0 0x7f2c3c1f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c428df640 1 -- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280051c0 msgr2=0x7f2c3c1f0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c428df640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280051c0 0x7f2c3c1f0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c428df640 1 -- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c3c14eb70 msgr2=0x7f2c3c157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c428df640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c3c14eb70 0x7f2c3c157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c428df640 1 -- 172.21.15.113:0/2803986079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c156750 con 0x7f2c3c14eff0 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c428df640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c3c14eff0 0x7f2c3c1589a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400e230 tx=0x7f2c34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c32ffd640 1 -- 172.21.15.113:0/2803986079 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34040070 con 0x7f2c3c14eff0 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c32ffd640 1 -- 172.21.15.113:0/2803986079 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c3403c680 con 0x7f2c3c14eff0 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c156a30 con 0x7f2c3c14eff0 2024-04-25T00:07:13.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c32ffd640 1 -- 172.21.15.113:0/2803986079 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035030 con 0x7f2c3c14eff0 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.389+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c156ff0 con 0x7f2c3c14eff0 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.390+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c0691a0 con 0x7f2c3c14eff0 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.390+0000 7f2c32ffd640 1 -- 172.21.15.113:0/2803986079 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c34043070 con 0x7f2c3c14eff0 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.391+0000 7f2c32ffd640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c2006be30 0x7f2c2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.391+0000 7f2c32ffd640 1 -- 172.21.15.113:0/2803986079 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5029+0+0 (secure 0 0 0) 0x7f2c340abc80 con 0x7f2c3c14eff0 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.391+0000 7f2c420de640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c2006be30 0x7f2c2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.391+0000 7f2c420de640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c2006be30 0x7f2c2006e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c155c40 tx=0x7f2c38009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:13.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.395+0000 7f2c32ffd640 1 -- 172.21.15.113:0/2803986079 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c3407a310 con 0x7f2c3c14eff0 2024-04-25T00:07:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.635+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) -- 0x7f2c3c0b2900 con 0x7f2c3c14eff0 2024-04-25T00:07:13.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.967+0000 7f2c32ffd640 1 -- 172.21.15.113:0/2803986079 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v441) ==== 166+0+0 (secure 0 0 0) 0x7f2c3407e610 con 0x7f2c3c14eff0 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c2006be30 msgr2=0x7f2c2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c2006be30 0x7f2c2006e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c155c40 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c3c14eff0 msgr2=0x7f2c3c1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c3c14eff0 0x7f2c3c1589a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400e230 tx=0x7f2c34009750 comp rx=0 tx=0).stop 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 shutdown_connections 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c2006be30 0x7f2c2006e2f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c3c14eb70 0x7f2c3c157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c3c14eff0 0x7f2c3c1589a0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 --2- 172.21.15.113:0/2803986079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c280051c0 0x7f2c3c1f0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 >> 172.21.15.113:0/2803986079 conn(0x7f2c3c0b1a50 msgr2=0x7f2c3c0cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 shutdown_connections 2024-04-25T00:07:13.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:13.970+0000 7f2c44342640 1 -- 172.21.15.113:0/2803986079 wait complete. 2024-04-25T00:07:13.982 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T00:07:13.982 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T00:07:13.982 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:14.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.078+0000 7f01b974f640 1 Processor -- start 2024-04-25T00:07:14.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.078+0000 7f01b974f640 1 -- start start 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.078+0000 7f01b974f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 0x7f01b415f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.078+0000 7f01b974f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b415f8b0 con 0x7f01b40cf2b0 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.078+0000 7f01b974f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b4161ff0 con 0x7f01b40b6470 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.078+0000 7f01b974f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f01b41622d0 con 0x7f01b415e4f0 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b27fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f01b40cf2b0 0x7f01b40b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36286/0 (socket says 172.21.15.113:36286) 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b27fc640 1 -- 172.21.15.113:0/2070314619 learned_addr learned my addr 172.21.15.113:0/2070314619 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b2ffd640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 0x7f01b415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436002921 0 0) 0x7f01a8001110 con 0x7f01b40cf2b0 2024-04-25T00:07:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f019c003520 con 0x7f01b40cf2b0 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10307560 0 0) 0x7f01b4161ff0 con 0x7f01b40b6470 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01a8001110 con 0x7f01b40b6470 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3749608960 0 0) 0x7f019c003520 con 0x7f01b40cf2b0 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 msgr2=0x7f01b415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 0x7f01b415f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 >> v1:172.21.15.178:6789/0 conn(0x7f01b40b6470 legacy=0x7f01b40d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b41625b0 con 0x7f01b40cf2b0 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f01a8001840 con 0x7f01b40cf2b0 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b2ffd640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 0x7f01b415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 >> v1:172.21.15.113:6789/0 conn(0x7f01b40cf2b0 legacy=0x7f01b40b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 0x7f019c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f019c0040b0 0x7f019c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c004d40 0x7f019c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f01b41622d0 con 0x7f019c0040b0 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f019c00bbb0 con 0x7f019c004d40 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f019c00bdb0 con 0x7f01b415e4f0 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.079+0000 7f01b27fc640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f019c0040b0 0x7f019c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.080+0000 7f01b2ffd640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 0x7f019c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.080+0000 7f01b37fe640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c004d40 0x7f019c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.080+0000 7f01b27fc640 1 -- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 msgr2=0x7f019c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.080+0000 7f01b27fc640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01b415e4f0 0x7f019c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.080+0000 7f01b27fc640 1 -- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c004d40 msgr2=0x7f019c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.080+0000 7f01b27fc640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c004d40 0x7f019c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.080+0000 7f01b27fc640 1 -- 172.21.15.113:0/2070314619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c003520 con 0x7f019c0040b0 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b27fc640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f019c0040b0 0x7f019c004800 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c490 tx=0x7f019c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cad18c01dd999c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a800c0e0 con 0x7f019c0040b0 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8034070 con 0x7f019c0040b0 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b1ffb640 1 -- 172.21.15.113:0/2070314619 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8005d70 con 0x7f019c0040b0 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b974f640 1 -- 172.21.15.113:0/2070314619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f019c0040b0 msgr2=0x7f019c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b974f640 1 --2- 172.21.15.113:0/2070314619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f019c0040b0 0x7f019c004800 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c490 tx=0x7f019c005830 comp rx=0 tx=0).stop 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b37fe640 1 -- 172.21.15.113:0/2070314619 reap_dead start 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b974f640 1 -- 172.21.15.113:0/2070314619 shutdown_connections 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b974f640 1 -- 172.21.15.113:0/2070314619 >> 172.21.15.113:0/2070314619 conn(0x7f01b406b720 msgr2=0x7f01b40d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b974f640 1 -- 172.21.15.113:0/2070314619 shutdown_connections 2024-04-25T00:07:14.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.081+0000 7f01b974f640 1 -- 172.21.15.113:0/2070314619 wait complete. 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.082+0000 7f01b974f640 1 Processor -- start 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b974f640 1 -- start start 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b974f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f019c0040b0 0x7f01b4152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b974f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b4156050 0x7f01b4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b974f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01b4157450 0x7f01b4157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b974f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f01b415f4f0 con 0x7f01b4157450 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b974f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01b415f370 con 0x7f01b4156050 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b974f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f01b415f670 con 0x7f019c0040b0 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01b4157450 0x7f01b4157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01b4157450 0x7f01b4157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49942/0 (socket says 172.21.15.113:49942) 2024-04-25T00:07:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b37fe640 1 -- 172.21.15.113:0/431537975 learned_addr learned my addr 172.21.15.113:0/431537975 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.083+0000 7f01b27fc640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b4156050 0x7f01b4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b2ffd640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f019c0040b0 0x7f01b4152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b37fe640 1 -- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f019c0040b0 msgr2=0x7f01b4152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b37fe640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f019c0040b0 0x7f01b4152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b37fe640 1 -- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b4156050 msgr2=0x7f01b4156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b37fe640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b4156050 0x7f01b4156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b37fe640 1 -- 172.21.15.113:0/431537975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b4158a80 con 0x7f01b4157450 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b27fc640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b4156050 0x7f01b4156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b37fe640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01b4157450 0x7f01b4157900 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f01b40cf050 tx=0x7f01ac00fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f019b7fe640 1 -- 172.21.15.113:0/431537975 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac019070 con 0x7f01b4157450 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f019b7fe640 1 -- 172.21.15.113:0/431537975 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ac006d60 con 0x7f01b4157450 2024-04-25T00:07:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b4158d30 con 0x7f01b4157450 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f019b7fe640 1 -- 172.21.15.113:0/431537975 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac003bb0 con 0x7f01b4157450 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.084+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b41592f0 con 0x7f01b4157450 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.085+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b4156430 con 0x7f01b4157450 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.085+0000 7f019b7fe640 1 -- 172.21.15.113:0/431537975 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01ac01c070 con 0x7f01b4157450 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.086+0000 7f019b7fe640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f01a006be60 0x7f01a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.086+0000 7f01b2ffd640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f01a006be60 0x7f01a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.086+0000 7f019b7fe640 1 -- 172.21.15.113:0/431537975 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5029+0+0 (secure 0 0 0) 0x7f01ac089d50 con 0x7f01b4157450 2024-04-25T00:07:14.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.086+0000 7f01b2ffd640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f01a006be60 0x7f01a006e320 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f01b4161560 tx=0x7f01a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.090+0000 7f019b7fe640 1 -- 172.21.15.113:0/431537975 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01ac08d030 con 0x7f01b4157450 2024-04-25T00:07:14.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.282+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01b40691a0 con 0x7f01b4157450 2024-04-25T00:07:14.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.283+0000 7f019b7fe640 1 -- 172.21.15.113:0/431537975 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+13210 (secure 0 0 0) 0x7f01ac0589a0 con 0x7f01b4157450 2024-04-25T00:07:14.283 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":136,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:12.954664+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6837/409819700":"2024-04-26T00:07:09.888799+0000","172.21.15.113:6835/409819700":"2024-04-26T00:07:09.888799+0000","172.21.15.113:6836/2664195523":"2024-04-26T00:06:59.857692+0000","172.21.15.113:6834/2664195523":"2024-04-26T00:06:59.857692+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:14.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.284+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f01a006be60 msgr2=0x7f01a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.284+0000 7f01b974f640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f01a006be60 0x7f01a006e320 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f01b4161560 tx=0x7f01a4008040 comp rx=0 tx=0).stop 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.284+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01b4157450 msgr2=0x7f01b4157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.284+0000 7f01b974f640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01b4157450 0x7f01b4157900 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f01b40cf050 tx=0x7f01ac00fc50 comp rx=0 tx=0).stop 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 shutdown_connections 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f01a006be60 0x7f01a006e320 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b4156050 0x7f01b4156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01b4157450 0x7f01b4157900 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 --2- 172.21.15.113:0/431537975 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f019c0040b0 0x7f01b4152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 >> 172.21.15.113:0/431537975 conn(0x7f01b406b720 msgr2=0x7f01b4161710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 shutdown_connections 2024-04-25T00:07:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.285+0000 7f01b974f640 1 -- 172.21.15.113:0/431537975 wait complete. 2024-04-25T00:07:14.296 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:07:14.297 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:07:14.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.396+0000 7f6504f1d640 1 Processor -- start 2024-04-25T00:07:14.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.396+0000 7f6504f1d640 1 -- start start 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f6504f1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65000b5ca0 0x7f65000b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f6504f1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65000b73c0 0x7f65000b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f6504f1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65000d2e00 con 0x7f650015e510 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f6504f1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f65000b6640 con 0x7f65000b5ca0 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f6504f1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65000d30e0 con 0x7f65000b73c0 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f64fdd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65000b5ca0 0x7f65000b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f64fdd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65000b5ca0 0x7f65000b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49946/0 (socket says 172.21.15.113:49946) 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.397+0000 7f64fdd9b640 1 -- 172.21.15.113:0/174254848 learned_addr learned my addr 172.21.15.113:0/174254848 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fed9d640 1 --2- 172.21.15.113:0/174254848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65000b73c0 0x7f65000b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fdd9b640 1 -- 172.21.15.113:0/174254848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65000b73c0 msgr2=0x7f65000b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fdd9b640 1 --2- 172.21.15.113:0/174254848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65000b73c0 0x7f65000b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fdd9b640 1 -- 172.21.15.113:0/174254848 >> v1:172.21.15.178:6789/0 conn(0x7f650015e510 legacy=0x7f65000d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fdd9b640 1 -- 172.21.15.113:0/174254848 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65000d33c0 con 0x7f65000b5ca0 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fed9d640 1 --2- 172.21.15.113:0/174254848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65000b73c0 0x7f65000b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fdd9b640 1 --2- 172.21.15.113:0/174254848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b5ca0 0x7f65000b6100 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009870 tx=0x7f64f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1734cf8abc20f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fd59a640 1 -- 172.21.15.113:0/174254848 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4037070 con 0x7f65000b5ca0 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.398+0000 7f64fd59a640 1 -- 172.21.15.113:0/174254848 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f4032440 con 0x7f65000b5ca0 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f64fd59a640 1 -- 172.21.15.113:0/174254848 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4032810 con 0x7f65000b5ca0 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 -- 172.21.15.113:0/174254848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b5ca0 msgr2=0x7f65000b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 --2- 172.21.15.113:0/174254848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b5ca0 0x7f65000b6100 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009870 tx=0x7f64f402a180 comp rx=0 tx=0).stop 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 -- 172.21.15.113:0/174254848 shutdown_connections 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 --2- 172.21.15.113:0/174254848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65000b73c0 0x7f65000b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 --2- 172.21.15.113:0/174254848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b5ca0 0x7f65000b6100 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 -- 172.21.15.113:0/174254848 >> 172.21.15.113:0/174254848 conn(0x7f650006b720 msgr2=0x7f65000d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:14.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 -- 172.21.15.113:0/174254848 shutdown_connections 2024-04-25T00:07:14.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.399+0000 7f6504f1d640 1 -- 172.21.15.113:0/174254848 wait complete. 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.400+0000 7f6504f1d640 1 Processor -- start 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.400+0000 7f6504f1d640 1 -- start start 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f6504f1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65000b5ca0 0x7f65000c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f6504f1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b73c0 0x7f6500147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f6504f1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650015e510 0x7f65000cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f6504f1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65000d3ea0 con 0x7f65000b73c0 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f6504f1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65000d3980 con 0x7f65000b5ca0 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f6504f1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65000d2a90 con 0x7f650015e510 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b73c0 0x7f6500147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b73c0 0x7f6500147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49958/0 (socket says 172.21.15.113:49958) 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 -- 172.21.15.113:0/3536830602 learned_addr learned my addr 172.21.15.113:0/3536830602 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:14.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fe59c640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65000b5ca0 0x7f65000c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fed9d640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650015e510 0x7f65000cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 -- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650015e510 msgr2=0x7f65000cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650015e510 0x7f65000cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 -- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65000b5ca0 msgr2=0x7f65000c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65000b5ca0 0x7f65000c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.401+0000 7f64fdd9b640 1 -- 172.21.15.113:0/3536830602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650015fa30 con 0x7f65000b73c0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.402+0000 7f64fdd9b640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b73c0 0x7f6500147430 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009870 tx=0x7f64f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.402+0000 7f64eeffd640 1 -- 172.21.15.113:0/3536830602 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4041020 con 0x7f65000b73c0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.402+0000 7f64eeffd640 1 -- 172.21.15.113:0/3536830602 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f4030970 con 0x7f65000b73c0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.402+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650015fd10 con 0x7f65000b73c0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.402+0000 7f64eeffd640 1 -- 172.21.15.113:0/3536830602 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4037040 con 0x7f65000b73c0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.402+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65001602d0 con 0x7f65000b73c0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.403+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65000691a0 con 0x7f65000b73c0 2024-04-25T00:07:14.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.403+0000 7f64eeffd640 1 -- 172.21.15.113:0/3536830602 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64f4044020 con 0x7f65000b73c0 2024-04-25T00:07:14.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.403+0000 7f64eeffd640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f64e006be60 0x7f64e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.404+0000 7f64eeffd640 1 -- 172.21.15.113:0/3536830602 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5029+0+0 (secure 0 0 0) 0x7f64f40ab540 con 0x7f65000b73c0 2024-04-25T00:07:14.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.404+0000 7f64fe59c640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f64e006be60 0x7f64e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.404+0000 7f64fe59c640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f64e006be60 0x7f64e006e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005520 tx=0x7f64f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.408+0000 7f64eeffd640 1 -- 172.21.15.113:0/3536830602 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f64f404b030 con 0x7f65000b73c0 2024-04-25T00:07:14.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.579+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f650015d5c0 con 0x7f65000b73c0 2024-04-25T00:07:14.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.580+0000 7f64eeffd640 1 -- 172.21.15.113:0/3536830602 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) ==== 93+0+10 (secure 0 0 0) 0x7f64f407a190 con 0x7f65000b73c0 2024-04-25T00:07:14.580 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:07:14.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f64e006be60 msgr2=0x7f64e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f64e006be60 0x7f64e006e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005520 tx=0x7f64f0009040 comp rx=0 tx=0).stop 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b73c0 msgr2=0x7f6500147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b73c0 0x7f6500147430 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009870 tx=0x7f64f40046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 shutdown_connections 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f64e006be60 0x7f64e006e320 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650015e510 0x7f65000cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65000b73c0 0x7f6500147430 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 --2- 172.21.15.113:0/3536830602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65000b5ca0 0x7f65000c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 >> 172.21.15.113:0/3536830602 conn(0x7f650006b720 msgr2=0x7f650015cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 shutdown_connections 2024-04-25T00:07:14.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.582+0000 7f6504f1d640 1 -- 172.21.15.113:0/3536830602 wait complete. 2024-04-25T00:07:14.593 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T00:07:14.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.690+0000 7f2ce08bd640 1 Processor -- start 2024-04-25T00:07:14.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.690+0000 7f2ce08bd640 1 -- start start 2024-04-25T00:07:14.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2ce08bd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2cdc0b3fd0 0x7f2cdc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2ce08bd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cdc144cf0 con 0x7f2cdc144910 2024-04-25T00:07:14.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2ce08bd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cdc0b2f30 con 0x7f2cdc06e510 2024-04-25T00:07:14.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2ce08bd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2cdc06e980 con 0x7f2cdc0b3fd0 2024-04-25T00:07:14.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cda59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2cdc144910 0x7f2cdc0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36302/0 (socket says 172.21.15.113:36302) 2024-04-25T00:07:14.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cda59c640 1 -- 172.21.15.113:0/1527266684 learned_addr learned my addr 172.21.15.113:0/1527266684 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:14.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd3fff640 1 --2- 172.21.15.113:0/1527266684 >> v2:172.21.15.178:3300/0 conn(0x7f2cdc0b3fd0 0x7f2cdc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346270470 0 0) 0x7f2cdc144cf0 con 0x7f2cdc144910 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cc8003520 con 0x7f2cdc144910 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601225249 0 0) 0x7f2cc8003520 con 0x7f2cdc144910 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 >> v2:172.21.15.178:3300/0 conn(0x7f2cdc0b3fd0 msgr2=0x7f2cdc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 --2- 172.21.15.113:0/1527266684 >> v2:172.21.15.178:3300/0 conn(0x7f2cdc0b3fd0 0x7f2cdc06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 >> v1:172.21.15.178:6789/0 conn(0x7f2cdc06e510 legacy=0x7f2cdc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc0b3210 con 0x7f2cdc144910 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f2ccc001670 con 0x7f2cdc144910 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 >> v1:172.21.15.113:6789/0 conn(0x7f2cdc144910 legacy=0x7f2cdc0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cc8003a30 0x7f2cc8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cc8004800 0x7f2cc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc06e980 con 0x7f2cc80051c0 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc800c0f0 con 0x7f2cc8003a30 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.691+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc800c2f0 con 0x7f2cc8004800 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cda59c640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cc8003a30 0x7f2cc8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cd3fff640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cc8004800 0x7f2cc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 -- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cc8004800 msgr2=0x7f2cc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cc8004800 0x7f2cc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 -- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cc8003a30 msgr2=0x7f2cc8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cc8003a30 0x7f2cc8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 -- 172.21.15.113:0/1527266684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc8003520 con 0x7f2cc80051c0 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 -- 172.21.15.113:0/1527266684 reap_dead start 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.692+0000 7f2cdad9d640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cc800b9c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4001e50 tx=0x7f2cc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c193a61f82b37aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4001a70 con 0x7f2cc80051c0 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd4003970 con 0x7f2cc80051c0 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2cd9d9b640 1 -- 172.21.15.113:0/1527266684 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4003d60 con 0x7f2cc80051c0 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2ce08bd640 1 -- 172.21.15.113:0/1527266684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 msgr2=0x7f2cc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cc800b9c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4001e50 tx=0x7f2cc8005830 comp rx=0 tx=0).stop 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2ce08bd640 1 -- 172.21.15.113:0/1527266684 shutdown_connections 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/1527266684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cc800b9c0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2ce08bd640 1 -- 172.21.15.113:0/1527266684 >> 172.21.15.113:0/1527266684 conn(0x7f2cdc0b1a50 msgr2=0x7f2cdc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2ce08bd640 1 -- 172.21.15.113:0/1527266684 shutdown_connections 2024-04-25T00:07:14.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.693+0000 7f2ce08bd640 1 -- 172.21.15.113:0/1527266684 wait complete. 2024-04-25T00:07:14.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.694+0000 7f2ce08bd640 1 Processor -- start 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.694+0000 7f2ce08bd640 1 -- start start 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2ce08bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cdc15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2ce08bd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cdc070de0 0x7f2cdc071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2ce08bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cdc14ed70 0x7f2cdc14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2ce08bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc0b4440 con 0x7f2cc80051c0 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2ce08bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc0b37d0 con 0x7f2cdc14ed70 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2ce08bd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2cdc0b4da0 con 0x7f2cdc070de0 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cda59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cdc15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cda59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cdc15d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49978/0 (socket says 172.21.15.113:49978) 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cda59c640 1 -- 172.21.15.113:0/2222250156 learned_addr learned my addr 172.21.15.113:0/2222250156 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cdad9d640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cdc14ed70 0x7f2cdc14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cda59c640 1 -- 172.21.15.113:0/2222250156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cdc070de0 msgr2=0x7f2cdc071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cda59c640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cdc070de0 0x7f2cdc071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cda59c640 1 -- 172.21.15.113:0/2222250156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cdc14ed70 msgr2=0x7f2cdc14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.695+0000 7f2cda59c640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cdc14ed70 0x7f2cdc14f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.696+0000 7f2cda59c640 1 -- 172.21.15.113:0/2222250156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc14f980 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.696+0000 7f2cda59c640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cdc15d1d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004f40 tx=0x7f2ccc0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.696+0000 7f2cd2ffd640 1 -- 172.21.15.113:0/2222250156 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00f750 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.696+0000 7f2cd2ffd640 1 -- 172.21.15.113:0/2222250156 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc018070 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.696+0000 7f2cd2ffd640 1 -- 172.21.15.113:0/2222250156 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00fb50 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.696+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc14fc60 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.696+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc15d710 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.697+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc0691a0 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.697+0000 7f2cd2ffd640 1 -- 172.21.15.113:0/2222250156 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ccc00fcf0 con 0x7f2cc80051c0 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.698+0000 7f2cd2ffd640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cbc06be30 0x7f2cbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:14.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.698+0000 7f2cd2ffd640 1 -- 172.21.15.113:0/2222250156 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5029+0+0 (secure 0 0 0) 0x7f2ccc08aaa0 con 0x7f2cc80051c0 2024-04-25T00:07:14.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.698+0000 7f2cd3fff640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cbc06be30 0x7f2cbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:14.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.698+0000 7f2cd3fff640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cbc06be30 0x7f2cbc06e2f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009a40 tx=0x7f2cc4009320 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:14.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.702+0000 7f2cd2ffd640 1 -- 172.21.15.113:0/2222250156 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ccc0596f0 con 0x7f2cc80051c0 2024-04-25T00:07:14.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.941+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) -- 0x7f2cdc151850 con 0x7f2cc80051c0 2024-04-25T00:07:14.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.969+0000 7f2cd2ffd640 1 -- 172.21.15.113:0/2222250156 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v442) ==== 143+0+0 (secure 0 0 0) 0x7f2ccc058ff0 con 0x7f2cc80051c0 2024-04-25T00:07:14.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cbc06be30 msgr2=0x7f2cbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cbc06be30 0x7f2cbc06e2f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009a40 tx=0x7f2cc4009320 comp rx=0 tx=0).stop 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 msgr2=0x7f2cdc15d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cdc15d1d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004f40 tx=0x7f2ccc0074f0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 shutdown_connections 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2cbc06be30 0x7f2cbc06e2f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cdc14ed70 0x7f2cdc14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2cdc070de0 0x7f2cdc071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 --2- 172.21.15.113:0/2222250156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc80051c0 0x7f2cdc15d1d0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 >> 172.21.15.113:0/2222250156 conn(0x7f2cdc0b1a50 msgr2=0x7f2cdc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 shutdown_connections 2024-04-25T00:07:14.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:14.972+0000 7f2ce08bd640 1 -- 172.21.15.113:0/2222250156 wait complete. 2024-04-25T00:07:14.983 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:07:15.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.082+0000 7f0c06b42640 1 Processor -- start 2024-04-25T00:07:15.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.082+0000 7f0c06b42640 1 -- start start 2024-04-25T00:07:15.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.083+0000 7f0c06b42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 0x7f0c000d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.083+0000 7f0c06b42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0c000b63b0 0x7f0c000b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.083+0000 7f0c06b42640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c000d2e00 con 0x7f0c000d28c0 2024-04-25T00:07:15.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.083+0000 7f0c06b42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0c000d30e0 con 0x7f0c000b63b0 2024-04-25T00:07:15.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.083+0000 7f0c06b42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000d33c0 con 0x7f0c0015e510 2024-04-25T00:07:15.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.083+0000 7f0bfffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0c000d28c0 0x7f0c000b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36318/0 (socket says 172.21.15.113:36318) 2024-04-25T00:07:15.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.083+0000 7f0bfffff640 1 -- 172.21.15.113:0/4257095860 learned_addr learned my addr 172.21.15.113:0/4257095860 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2272811245 0 0) 0x7f0c000d2e00 con 0x7f0c000d28c0 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bec003520 con 0x7f0c000d28c0 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c048de640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 0x7f0c000d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c050df640 1 --2- 172.21.15.113:0/4257095860 >> v2:172.21.15.178:3300/0 conn(0x7f0c000b63b0 0x7f0c000b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 720857387 0 0) 0x7f0bec003520 con 0x7f0c000d28c0 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 msgr2=0x7f0c000d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 0x7f0c000d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 >> v2:172.21.15.178:3300/0 conn(0x7f0c000b63b0 msgr2=0x7f0c000b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 --2- 172.21.15.113:0/4257095860 >> v2:172.21.15.178:3300/0 conn(0x7f0c000b63b0 0x7f0c000b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c000d36a0 con 0x7f0c000d28c0 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f0bf00019f0 con 0x7f0c000d28c0 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 >> v1:172.21.15.113:6789/0 conn(0x7f0c000d28c0 legacy=0x7f0c000b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec0039b0 0x7f0bec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 0x7f0bec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bec004d40 0x7f0bec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000d30e0 con 0x7f0bec004d40 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000d33c0 con 0x7f0bec0039b0 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0bec00b9c0 con 0x7f0c0015e510 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c050df640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bec004d40 0x7f0bec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c048de640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec0039b0 0x7f0bec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0bfffff640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 0x7f0bec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c050df640 1 -- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 msgr2=0x7f0bec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c050df640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015e510 0x7f0bec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c050df640 1 -- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec0039b0 msgr2=0x7f0bec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c050df640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec0039b0 0x7f0bec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c050df640 1 -- 172.21.15.113:0/4257095860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec003520 con 0x7f0bec004d40 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.084+0000 7f0c048de640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec0039b0 0x7f0bec003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.085+0000 7f0c050df640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bec004d40 0x7f0bec0051c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0c000b6d40 tx=0x7f0bec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed3036c03c0fb24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.085+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8036070 con 0x7f0bec004d40 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.085+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be8031400 con 0x7f0bec004d40 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.085+0000 7f0bff7fe640 1 -- 172.21.15.113:0/4257095860 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8009690 con 0x7f0bec004d40 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.086+0000 7f0c06b42640 1 -- 172.21.15.113:0/4257095860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bec004d40 msgr2=0x7f0bec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.086+0000 7f0c06b42640 1 --2- 172.21.15.113:0/4257095860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bec004d40 0x7f0bec0051c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0c000b6d40 tx=0x7f0bec005830 comp rx=0 tx=0).stop 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.086+0000 7f0c050df640 1 -- 172.21.15.113:0/4257095860 reap_dead start 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.086+0000 7f0c06b42640 1 -- 172.21.15.113:0/4257095860 shutdown_connections 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.086+0000 7f0c06b42640 1 -- 172.21.15.113:0/4257095860 >> 172.21.15.113:0/4257095860 conn(0x7f0c0006b720 msgr2=0x7f0c000d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.086+0000 7f0c06b42640 1 -- 172.21.15.113:0/4257095860 shutdown_connections 2024-04-25T00:07:15.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.086+0000 7f0c06b42640 1 -- 172.21.15.113:0/4257095860 wait complete. 2024-04-25T00:07:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.087+0000 7f0c06b42640 1 Processor -- start 2024-04-25T00:07:15.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.087+0000 7f0c06b42640 1 -- start start 2024-04-25T00:07:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c00161210 0x7f0c0015f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015ff10 0x7f0c00160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00152d30 0x7f0c001531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000b9dd0 con 0x7f0c00161210 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000d3ea0 con 0x7f0c00152d30 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000d2380 con 0x7f0c0015ff10 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c00161210 0x7f0c0015f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c00161210 0x7f0c0015f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49998/0 (socket says 172.21.15.113:49998) 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 -- 172.21.15.113:0/4054493747 learned_addr learned my addr 172.21.15.113:0/4054493747 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0bfffff640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015ff10 0x7f0c00160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c050df640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00152d30 0x7f0c001531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 -- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015ff10 msgr2=0x7f0c00160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015ff10 0x7f0c00160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 -- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00152d30 msgr2=0x7f0c001531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00152d30 0x7f0c001531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 -- 172.21.15.113:0/4054493747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001f4080 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c048de640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c00161210 0x7f0c0015f9d0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8011570 tx=0x7f0bf8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0bfd7fa640 1 -- 172.21.15.113:0/4054493747 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8015870 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0bfd7fa640 1 -- 172.21.15.113:0/4054493747 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf8015a10 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c000ca4b0 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0bfd7fa640 1 -- 172.21.15.113:0/4054493747 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8015dc0 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.088+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c00152250 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.090+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c000691a0 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.090+0000 7f0bfd7fa640 1 -- 172.21.15.113:0/4054493747 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bf801d400 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.090+0000 7f0bfd7fa640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0be006be60 0x7f0be006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.090+0000 7f0bfd7fa640 1 -- 172.21.15.113:0/4054493747 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5029+0+0 (secure 0 0 0) 0x7f0bf808b420 con 0x7f0c00161210 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.090+0000 7f0bfffff640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0be006be60 0x7f0be006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.091+0000 7f0bfffff640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0be006be60 0x7f0be006e320 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0004bc0 tx=0x7f0bf0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.095+0000 7f0bfd7fa640 1 -- 172.21.15.113:0/4054493747 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0bf8019030 con 0x7f0c00161210 2024-04-25T00:07:15.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.267+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0c00159480 con 0x7f0c00161210 2024-04-25T00:07:15.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.271+0000 7f0bfd7fa640 1 -- 172.21.15.113:0/4054493747 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f0bf805a070 con 0x7f0c00161210 2024-04-25T00:07:15.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0be006be60 msgr2=0x7f0be006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0be006be60 0x7f0be006e320 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0004bc0 tx=0x7f0bf0009040 comp rx=0 tx=0).stop 2024-04-25T00:07:15.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c00161210 msgr2=0x7f0c0015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c00161210 0x7f0c0015f9d0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8011570 tx=0x7f0bf8011a30 comp rx=0 tx=0).stop 2024-04-25T00:07:15.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 shutdown_connections 2024-04-25T00:07:15.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0be006be60 0x7f0be006e320 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00152d30 0x7f0c001531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c0015ff10 0x7f0c00160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 --2- 172.21.15.113:0/4054493747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c00161210 0x7f0c0015f9d0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 >> 172.21.15.113:0/4054493747 conn(0x7f0c0006b720 msgr2=0x7f0c001589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:15.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 shutdown_connections 2024-04-25T00:07:15.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.273+0000 7f0c06b42640 1 -- 172.21.15.113:0/4054493747 wait complete. 2024-04-25T00:07:15.286 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:07:15.286 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:07:15.323 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:07:15.374 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:07:15.374 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:15.374 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:15.389 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:07:15.389 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:07:15.389 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:15.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:07:15.407 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:15.407 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:15.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:07:15.444 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:07:15.444 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:15.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:07:15.464 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:07:15.464 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:15.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.579+0000 7f6cb027d640 1 Processor -- start 2024-04-25T00:07:15.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.579+0000 7f6cb027d640 1 -- start start 2024-04-25T00:07:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.580+0000 7f6cb027d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6ca80d2290 0x7f6ca80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.580+0000 7f6cb027d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca80d2800 con 0x7f6ca80d0b00 2024-04-25T00:07:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.580+0000 7f6cb027d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca80d2ae0 con 0x7f6ca815e320 2024-04-25T00:07:15.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.580+0000 7f6cb027d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca80d3410 con 0x7f6ca80d2290 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.580+0000 7f6cae81a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6ca80d0b00 0x7f6ca80d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36332/0 (socket says 172.21.15.113:36332) 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae81a640 1 -- 172.21.15.113:0/2871491284 learned_addr learned my addr 172.21.15.113:0/2871491284 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2008580469 0 0) 0x7f6ca80d2800 con 0x7f6ca80d0b00 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c94003520 con 0x7f6ca80d0b00 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad818640 1 --2- 172.21.15.113:0/2871491284 >> v2:172.21.15.178:3300/0 conn(0x7f6ca80d2290 0x7f6ca80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2557617760 0 0) 0x7f6c94003520 con 0x7f6ca80d0b00 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 >> v2:172.21.15.178:3300/0 conn(0x7f6ca80d2290 msgr2=0x7f6ca80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 --2- 172.21.15.113:0/2871491284 >> v2:172.21.15.178:3300/0 conn(0x7f6ca80d2290 0x7f6ca80b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 >> v1:172.21.15.178:6789/0 conn(0x7f6ca815e320 legacy=0x7f6ca80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca80d2dc0 con 0x7f6ca80d0b00 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6ca0001670 con 0x7f6ca80d0b00 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 >> v1:172.21.15.113:6789/0 conn(0x7f6ca80d0b00 legacy=0x7f6ca80d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940039b0 0x7f6c94003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c94004800 0x7f6c94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940051c0 0x7f6c9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca80d3410 con 0x7f6c940039b0 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9400c0d0 con 0x7f6c940051c0 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9400c2d0 con 0x7f6c94004800 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae019640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940039b0 0x7f6c94003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae81a640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940051c0 0x7f6c9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cad818640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c94004800 0x7f6c94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae019640 1 -- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c94004800 msgr2=0x7f6c94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae019640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c94004800 0x7f6c94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae019640 1 -- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940051c0 msgr2=0x7f6c9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae019640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940051c0 0x7f6c9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae81a640 1 -- 172.21.15.113:0/2871491284 reap_dead start 2024-04-25T00:07:15.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.581+0000 7f6cae019640 1 -- 172.21.15.113:0/2871491284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c94003520 con 0x7f6c940039b0 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.582+0000 7f6cae019640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940039b0 0x7f6c94003db0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40044b0 tx=0x7f6c94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6477bf7651c9645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.582+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4037070 con 0x7f6c940039b0 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.582+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca402ad60 con 0x7f6c940039b0 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.582+0000 7f6cad017640 1 -- 172.21.15.113:0/2871491284 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4004080 con 0x7f6c940039b0 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.583+0000 7f6cb027d640 1 -- 172.21.15.113:0/2871491284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940039b0 msgr2=0x7f6c94003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.583+0000 7f6cb027d640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940039b0 0x7f6c94003db0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40044b0 tx=0x7f6c94005830 comp rx=0 tx=0).stop 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.583+0000 7f6cb027d640 1 -- 172.21.15.113:0/2871491284 shutdown_connections 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.583+0000 7f6cb027d640 1 --2- 172.21.15.113:0/2871491284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940039b0 0x7f6c94003db0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.583+0000 7f6cb027d640 1 -- 172.21.15.113:0/2871491284 >> 172.21.15.113:0/2871491284 conn(0x7f6ca80b5c20 msgr2=0x7f6ca80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.583+0000 7f6cb027d640 1 -- 172.21.15.113:0/2871491284 shutdown_connections 2024-04-25T00:07:15.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.583+0000 7f6cb027d640 1 -- 172.21.15.113:0/2871491284 wait complete. 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.584+0000 7f6cb027d640 1 Processor -- start 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.584+0000 7f6cb027d640 1 -- start start 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940039b0 0x7f6ca81f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ca80cd460 0x7f6ca80ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca80ca5a0 0x7f6ca814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca815e880 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca815e700 con 0x7f6c940039b0 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca80d0f40 con 0x7f6ca80cd460 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca80ca5a0 0x7f6ca814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca80ca5a0 0x7f6ca814e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50020/0 (socket says 172.21.15.113:50020) 2024-04-25T00:07:15.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 -- 172.21.15.113:0/1203092696 learned_addr learned my addr 172.21.15.113:0/1203092696 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cad818640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ca80cd460 0x7f6ca80ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae019640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940039b0 0x7f6ca81f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 -- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ca80cd460 msgr2=0x7f6ca80ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ca80cd460 0x7f6ca80ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 -- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940039b0 msgr2=0x7f6ca81f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940039b0 0x7f6ca81f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 -- 172.21.15.113:0/1203092696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca815fcd0 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cae81a640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca80ca5a0 0x7f6ca814e250 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000fb0 tx=0x7f6ca0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6c9effd640 1 -- 172.21.15.113:0/1203092696 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000f680 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6c9effd640 1 -- 172.21.15.113:0/1203092696 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca0018070 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca815ffb0 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6c9effd640 1 -- 172.21.15.113:0/1203092696 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000fa80 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.585+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca8160570 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.586+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca80691a0 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.587+0000 7f6c9effd640 1 -- 172.21.15.113:0/1203092696 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ca000fc20 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.587+0000 7f6c9effd640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6c8c06be30 0x7f6c8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.587+0000 7f6c9effd640 1 -- 172.21.15.113:0/1203092696 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5029+0+0 (secure 0 0 0) 0x7f6ca008a810 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.587+0000 7f6cae019640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6c8c06be30 0x7f6c8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.588+0000 7f6cae019640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6c8c06be30 0x7f6c8c06e2f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4034000 tx=0x7f6ca402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.591+0000 7f6c9effd640 1 -- 172.21.15.113:0/1203092696 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ca0058ea0 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.800+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ca80b50e0 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.800+0000 7f6c9effd640 1 -- 172.21.15.113:0/1203092696 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+3050 (secure 0 0 0) 0x7f6ca005d1a0 con 0x7f6ca80ca5a0 2024-04-25T00:07:15.801 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:15.801 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6707,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3952441640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3952441640},{"type":"v1","addr":"172.21.15.178:6834","nonce":3952441640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3591181801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3591181801},{"type":"v1","addr":"172.21.15.178:6835","nonce":3591181801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":9420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/348452771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":348452771},{"type":"v1","addr":"172.21.15.113:6835","nonce":348452771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[]} 2024-04-25T00:07:15.801 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 74 2024-04-25T00:07:15.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6c8c06be30 msgr2=0x7f6c8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6c8c06be30 0x7f6c8c06e2f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4034000 tx=0x7f6ca402e000 comp rx=0 tx=0).stop 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca80ca5a0 msgr2=0x7f6ca814e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca80ca5a0 0x7f6ca814e250 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000fb0 tx=0x7f6ca0007430 comp rx=0 tx=0).stop 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 shutdown_connections 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6c8c06be30 0x7f6c8c06e2f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca80ca5a0 0x7f6ca814e250 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ca80cd460 0x7f6ca80ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 --2- 172.21.15.113:0/1203092696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c940039b0 0x7f6ca81f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 >> 172.21.15.113:0/1203092696 conn(0x7f6ca80b5c20 msgr2=0x7f6ca80b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 shutdown_connections 2024-04-25T00:07:15.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.802+0000 7f6cb027d640 1 -- 172.21.15.113:0/1203092696 wait complete. 2024-04-25T00:07:15.814 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:07:15.814 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:07:15.814 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:07:15.814 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:07:15.814 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:07:15.814 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:07:15.815 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:07:15.815 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:07:15.815 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:07:15.815 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:07:15.815 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:07:15.816 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:07:15.816 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:07:15.816 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:07:15.818 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:07:15.818 DEBUG:teuthology.parallel:result is None 2024-04-25T00:07:15.823 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:07:15.823 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:07:15.826 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:07:15.827 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:07:15.828 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:07:15.828 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:07:15.865 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:07:15.865 DEBUG:teuthology.parallel:result is None 2024-04-25T00:07:15.869 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:07:15.869 DEBUG:teuthology.parallel:result is None 2024-04-25T00:07:15.870 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:07:15.870 DEBUG:teuthology.parallel:result is None 2024-04-25T00:07:15.870 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:07:15.889 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:07:15.939 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:07:15.949 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:07:15.949 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:07:15.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.972+0000 7ff15056a640 1 Processor -- start 2024-04-25T00:07:15.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.972+0000 7ff15056a640 1 -- start start 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.973+0000 7ff15056a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff1480cf3c0 0x7ff1480d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.973+0000 7ff15056a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff148161fc0 0x7ff1480d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.973+0000 7ff15056a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff148162560 con 0x7ff1480cf7a0 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.973+0000 7ff15056a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff148162840 con 0x7ff1480cf3c0 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.973+0000 7ff15056a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff148162b20 con 0x7ff148161fc0 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14e306640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff1480cf3c0 0x7ff1480d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14db05640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff1480cf7a0 0x7ff1480d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36372/0 (socket says 172.21.15.113:36372) 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14db05640 1 -- 172.21.15.113:0/975676202 learned_addr learned my addr 172.21.15.113:0/975676202 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14eb07640 1 --2- 172.21.15.113:0/975676202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff148161fc0 0x7ff1480d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14e306640 1 -- 172.21.15.113:0/975676202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff148161fc0 msgr2=0x7ff1480d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:15.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14e306640 1 --2- 172.21.15.113:0/975676202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff148161fc0 0x7ff1480d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14e306640 1 -- 172.21.15.113:0/975676202 >> v1:172.21.15.113:6789/0 conn(0x7ff1480cf7a0 legacy=0x7ff1480d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.974+0000 7ff14e306640 1 -- 172.21.15.113:0/975676202 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1480d40f0 con 0x7ff1480cf3c0 2024-04-25T00:07:15.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.975+0000 7ff14e306640 1 --2- 172.21.15.113:0/975676202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1480cf3c0 0x7ff1480d3bb0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff1440097a0 tx=0x7ff14402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29e26b8ef8a4ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.976+0000 7ff14d304640 1 -- 172.21.15.113:0/975676202 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144037070 con 0x7ff1480cf3c0 2024-04-25T00:07:15.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.976+0000 7ff14d304640 1 -- 172.21.15.113:0/975676202 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff144032440 con 0x7ff1480cf3c0 2024-04-25T00:07:15.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.976+0000 7ff14d304640 1 -- 172.21.15.113:0/975676202 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144032810 con 0x7ff1480cf3c0 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 -- 172.21.15.113:0/975676202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1480cf3c0 msgr2=0x7ff1480d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 --2- 172.21.15.113:0/975676202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1480cf3c0 0x7ff1480d3bb0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff1440097a0 tx=0x7ff14402a180 comp rx=0 tx=0).stop 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 -- 172.21.15.113:0/975676202 shutdown_connections 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 --2- 172.21.15.113:0/975676202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff148161fc0 0x7ff1480d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 --2- 172.21.15.113:0/975676202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1480cf3c0 0x7ff1480d3bb0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 -- 172.21.15.113:0/975676202 >> 172.21.15.113:0/975676202 conn(0x7ff1480718f0 msgr2=0x7ff148071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 -- 172.21.15.113:0/975676202 shutdown_connections 2024-04-25T00:07:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.977+0000 7ff15056a640 1 -- 172.21.15.113:0/975676202 wait complete. 2024-04-25T00:07:15.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 Processor -- start 2024-04-25T00:07:15.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 -- start start 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1480cf3c0 0x7ff148150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1480cf7a0 0x7ff148153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff148161fc0 0x7ff148154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1480d1750 con 0x7ff148161fc0 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1480d2bf0 con 0x7ff1480cf7a0 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff15056a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff1480d18d0 con 0x7ff1480cf3c0 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.979+0000 7ff14eb07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff148161fc0 0x7ff148154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff148161fc0 0x7ff148154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50068/0 (socket says 172.21.15.113:50068) 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 -- 172.21.15.113:0/1462434370 learned_addr learned my addr 172.21.15.113:0/1462434370 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14e306640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1480cf3c0 0x7ff148150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14db05640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1480cf7a0 0x7ff148153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 -- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1480cf3c0 msgr2=0x7ff148150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1480cf3c0 0x7ff148150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 -- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1480cf7a0 msgr2=0x7ff148153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1480cf7a0 0x7ff148153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 -- 172.21.15.113:0/1462434370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff148152d30 con 0x7ff148161fc0 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14db05640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1480cf7a0 0x7ff148153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff14eb07640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff148161fc0 0x7ff148154020 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff140009d30 tx=0x7ff14000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff13effd640 1 -- 172.21.15.113:0/1462434370 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140018020 con 0x7ff148161fc0 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff13effd640 1 -- 172.21.15.113:0/1462434370 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140014480 con 0x7ff148161fc0 2024-04-25T00:07:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff148153010 con 0x7ff148161fc0 2024-04-25T00:07:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff13effd640 1 -- 172.21.15.113:0/1462434370 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140014870 con 0x7ff148161fc0 2024-04-25T00:07:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.980+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff148150a60 con 0x7ff148161fc0 2024-04-25T00:07:15.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.981+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1480691a0 con 0x7ff148161fc0 2024-04-25T00:07:15.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.982+0000 7ff13effd640 1 -- 172.21.15.113:0/1462434370 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff14001b020 con 0x7ff148161fc0 2024-04-25T00:07:15.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.982+0000 7ff13effd640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff12c06be60 0x7ff12c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.982+0000 7ff13effd640 1 -- 172.21.15.113:0/1462434370 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5029+0+0 (secure 0 0 0) 0x7ff14008b4b0 con 0x7ff148161fc0 2024-04-25T00:07:15.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.982+0000 7ff14e306640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff12c06be60 0x7ff12c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.983+0000 7ff14e306640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff12c06be60 0x7ff12c06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff144034000 tx=0x7ff14402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:15.986+0000 7ff13effd640 1 -- 172.21.15.113:0/1462434370 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff140022030 con 0x7ff148161fc0 2024-04-25T00:07:16.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.171+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff14815ac20 con 0x7ff148161fc0 2024-04-25T00:07:16.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.980+0000 7ff13effd640 1 -- 172.21.15.113:0/1462434370 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v140) ==== 97+0+0 (secure 0 0 0) 0x7ff140010040 con 0x7ff148161fc0 2024-04-25T00:07:16.980 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff12c06be60 msgr2=0x7ff12c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff12c06be60 0x7ff12c06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff144034000 tx=0x7ff14402e000 comp rx=0 tx=0).stop 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff148161fc0 msgr2=0x7ff148154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff148161fc0 0x7ff148154020 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff140009d30 tx=0x7ff14000c490 comp rx=0 tx=0).stop 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 shutdown_connections 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff12c06be60 0x7ff12c06e320 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1480cf7a0 0x7ff148153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff148161fc0 0x7ff148154020 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:16.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 --2- 172.21.15.113:0/1462434370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1480cf3c0 0x7ff148150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:16.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 >> 172.21.15.113:0/1462434370 conn(0x7ff1480718f0 msgr2=0x7ff14815dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:16.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 shutdown_connections 2024-04-25T00:07:16.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:16.982+0000 7ff15056a640 1 -- 172.21.15.113:0/1462434370 wait complete. 2024-04-25T00:07:16.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:07:17.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.133+0000 7f0d0561c640 1 Processor -- start 2024-04-25T00:07:17.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.133+0000 7f0d0561c640 1 -- start start 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.133+0000 7f0d0561c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d000b3fd0 0x7f0d0006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.133+0000 7f0d0561c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d00144cf0 con 0x7f0d0006e510 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.133+0000 7f0d0561c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d000b2f30 con 0x7f0d00144910 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.133+0000 7f0d0561c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0006e980 con 0x7f0d000b3fd0 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cfe7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d000b3fd0 0x7f0d0006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cfe7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d000b3fd0 0x7f0d0006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50084/0 (socket says 172.21.15.113:50084) 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cff7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0d0006e510 0x7f0d000cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36380/0 (socket says 172.21.15.113:36380) 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cfe7fc640 1 -- 172.21.15.113:0/1860954478 learned_addr learned my addr 172.21.15.113:0/1860954478 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:17.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cfe7fc640 1 -- 172.21.15.113:0/1860954478 >> v1:172.21.15.178:6789/0 conn(0x7f0d00144910 legacy=0x7f0d000b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:17.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cfe7fc640 1 -- 172.21.15.113:0/1860954478 >> v1:172.21.15.113:6789/0 conn(0x7f0d0006e510 legacy=0x7f0d000cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cfdffb640 1 -- 172.21.15.113:0/1860954478 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2200809586 0 0) 0x7f0d00144cf0 con 0x7f0d0006e510 2024-04-25T00:07:17.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.134+0000 7f0cfe7fc640 1 -- 172.21.15.113:0/1860954478 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d000b3210 con 0x7f0d000b3fd0 2024-04-25T00:07:17.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0cfe7fc640 1 --2- 172.21.15.113:0/1860954478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000b3fd0 0x7f0d0006dfd0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400bb40 tx=0x7f0cf402c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647f1036120800f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0cfdffb640 1 -- 172.21.15.113:0/1860954478 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4036020 con 0x7f0d000b3fd0 2024-04-25T00:07:17.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0cfdffb640 1 -- 172.21.15.113:0/1860954478 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf4032640 con 0x7f0d000b3fd0 2024-04-25T00:07:17.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0cfdffb640 1 -- 172.21.15.113:0/1860954478 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf402e040 con 0x7f0d000b3fd0 2024-04-25T00:07:17.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0d0561c640 1 -- 172.21.15.113:0/1860954478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000b3fd0 msgr2=0x7f0d0006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0d0561c640 1 --2- 172.21.15.113:0/1860954478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000b3fd0 0x7f0d0006dfd0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400bb40 tx=0x7f0cf402c990 comp rx=0 tx=0).stop 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0d0561c640 1 -- 172.21.15.113:0/1860954478 shutdown_connections 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0d0561c640 1 --2- 172.21.15.113:0/1860954478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d000b3fd0 0x7f0d0006dfd0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0d0561c640 1 -- 172.21.15.113:0/1860954478 >> 172.21.15.113:0/1860954478 conn(0x7f0d000b1a50 msgr2=0x7f0d000b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0d0561c640 1 -- 172.21.15.113:0/1860954478 shutdown_connections 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.135+0000 7f0d0561c640 1 -- 172.21.15.113:0/1860954478 wait complete. 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 Processor -- start 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 -- start start 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d0006e510 0x7f0d00138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d000b3fd0 0x7f0d0013af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d00144910 0x7f0d000b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000b4440 con 0x7f0d0006e510 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d000b37d0 con 0x7f0d00144910 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0d0561c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d000b4da0 con 0x7f0d000b3fd0 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d0006e510 0x7f0d00138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0cfeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d0006e510 0x7f0d00138200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50088/0 (socket says 172.21.15.113:50088) 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.137+0000 7f0cfeffd640 1 -- 172.21.15.113:0/3144978596 learned_addr learned my addr 172.21.15.113:0/3144978596 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0cff7fe640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d00144910 0x7f0d000b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0cfeffd640 1 -- 172.21.15.113:0/3144978596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d000b3fd0 msgr2=0x7f0d0013af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0cfeffd640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d000b3fd0 0x7f0d0013af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0cfeffd640 1 -- 172.21.15.113:0/3144978596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d00144910 msgr2=0x7f0d000b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0cfeffd640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d00144910 0x7f0d000b0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0cfeffd640 1 -- 172.21.15.113:0/3144978596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d00138fa0 con 0x7f0d0006e510 2024-04-25T00:07:17.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0cfeffd640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d0006e510 0x7f0d00138200 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b3f0 tx=0x7f0cf000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0ce77fe640 1 -- 172.21.15.113:0/3144978596 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0015020 con 0x7f0d0006e510 2024-04-25T00:07:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0ce77fe640 1 -- 172.21.15.113:0/3144978596 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf0011830 con 0x7f0d0006e510 2024-04-25T00:07:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d001389e0 con 0x7f0d0006e510 2024-04-25T00:07:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0ce77fe640 1 -- 172.21.15.113:0/3144978596 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0011c20 con 0x7f0d0006e510 2024-04-25T00:07:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.138+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0013c580 con 0x7f0d0006e510 2024-04-25T00:07:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.139+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d000691a0 con 0x7f0d0006e510 2024-04-25T00:07:17.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.139+0000 7f0ce77fe640 1 -- 172.21.15.113:0/3144978596 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cf0018020 con 0x7f0d0006e510 2024-04-25T00:07:17.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.140+0000 7f0ce77fe640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cec06be30 0x7f0cec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.140+0000 7f0ce77fe640 1 -- 172.21.15.113:0/3144978596 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4857+0+0 (secure 0 0 0) 0x7f0cf008a720 con 0x7f0d0006e510 2024-04-25T00:07:17.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.140+0000 7f0cfe7fc640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cec06be30 0x7f0cec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.140+0000 7f0cfe7fc640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cec06be30 0x7f0cec06e2f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4037000 tx=0x7f0cf403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.144+0000 7f0ce77fe640 1 -- 172.21.15.113:0/3144978596 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0cf001f030 con 0x7f0d0006e510 2024-04-25T00:07:17.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.386+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0d000731a0 con 0x7f0d0006e510 2024-04-25T00:07:17.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.386+0000 7f0ce77fe640 1 -- 172.21.15.113:0/3144978596 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f0d00138fa0 con 0x7f0d0006e510 2024-04-25T00:07:17.387 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:17.388 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:07:17.388 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:07:17.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.388 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:07:17.390 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.391 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.391 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:07:17.391 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:07:17.391 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.391 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:07:17.392 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.393 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:07:17.394 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:07:17.394 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.394 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.394 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.394 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.394 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.395 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.395 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:07:17.395 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:07:17.395 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.395 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.395 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.395 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.396 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.397 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.397 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.397 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:07:17.397 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:07:17.398 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.398 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.398 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.398 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.398 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.399 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:07:17.399 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:07:17.399 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.399 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.399 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.399 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.400 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.400 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.400 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:07:17.400 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:07:17.400 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.400 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:07:17.401 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.402 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.402 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.402 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.402 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.402 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.402 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.403 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.404 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:07:17.406 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:07:17.407 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.408 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:07:17.409 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:07:17.410 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.410 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:07:17.410 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:07:17.410 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.410 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:07:17.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.411 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:17.412 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:07:17.412 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:07:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cec06be30 msgr2=0x7f0cec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cec06be30 0x7f0cec06e2f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4037000 tx=0x7f0cf403d040 comp rx=0 tx=0).stop 2024-04-25T00:07:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d0006e510 msgr2=0x7f0d00138200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d0006e510 0x7f0d00138200 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b3f0 tx=0x7f0cf000b8b0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 shutdown_connections 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cec06be30 0x7f0cec06e2f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d00144910 0x7f0d000b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d000b3fd0 0x7f0d0013af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 --2- 172.21.15.113:0/3144978596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d0006e510 0x7f0d00138200 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 >> 172.21.15.113:0/3144978596 conn(0x7f0d000b1a50 msgr2=0x7f0d000715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 shutdown_connections 2024-04-25T00:07:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.388+0000 7f0d0561c640 1 -- 172.21.15.113:0/3144978596 wait complete. 2024-04-25T00:07:17.416 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.512+0000 7f8eff4df640 1 Processor -- start 2024-04-25T00:07:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.513+0000 7f8eff4df640 1 -- start start 2024-04-25T00:07:17.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.513+0000 7f8eff4df640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 0x7f8ef80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.513+0000 7f8eff4df640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef80d2800 con 0x7f8ef815e320 2024-04-25T00:07:17.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.513+0000 7f8eff4df640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef80d2ae0 con 0x7f8ef80d2290 2024-04-25T00:07:17.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.513+0000 7f8eff4df640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef80d14d0 con 0x7f8ef80d0b00 2024-04-25T00:07:17.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.513+0000 7f8efd27b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8ef815e320 0x7f8ef80d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56512/0 (socket says 172.21.15.113:56512) 2024-04-25T00:07:17.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.513+0000 7f8efd27b640 1 -- 172.21.15.113:0/4014715518 learned_addr learned my addr 172.21.15.113:0/4014715518 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:17.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efda7c640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 0x7f8ef80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3807742021 0 0) 0x7f8ef80d2800 con 0x7f8ef815e320 2024-04-25T00:07:17.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ee4003520 con 0x7f8ef815e320 2024-04-25T00:07:17.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3459936471 0 0) 0x7f8ee4003520 con 0x7f8ef815e320 2024-04-25T00:07:17.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 msgr2=0x7f8ef80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:17.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 0x7f8ef80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 >> v1:172.21.15.178:6789/0 conn(0x7f8ef80d2290 legacy=0x7f8ef80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef80d2dc0 con 0x7f8ef815e320 2024-04-25T00:07:17.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8ee8001690 con 0x7f8ef815e320 2024-04-25T00:07:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 >> v1:172.21.15.113:6789/0 conn(0x7f8ef815e320 legacy=0x7f8ef80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 0x7f8ee40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ee4003f20 0x7f8ee4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee4004d40 0x7f8ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef80d14d0 con 0x7f8ee4004d40 2024-04-25T00:07:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee400ba20 con 0x7f8ee4003f20 2024-04-25T00:07:17.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee400bc20 con 0x7f8ef80d0b00 2024-04-25T00:07:17.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efda7c640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee4004d40 0x7f8ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efca7a640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ee4003f20 0x7f8ee4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efd27b640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 0x7f8ee40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efda7c640 1 -- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 msgr2=0x7f8ee40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:17.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efda7c640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 0x7f8ee40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efda7c640 1 -- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ee4003f20 msgr2=0x7f8ee4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efda7c640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ee4003f20 0x7f8ee4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efda7c640 1 -- 172.21.15.113:0/4014715518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee4003520 con 0x7f8ee4004d40 2024-04-25T00:07:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.514+0000 7f8efd27b640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef80d0b00 0x7f8ee40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.515+0000 7f8efda7c640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee4004d40 0x7f8ee40051c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009fc0 tx=0x7f8ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc4ff65d5a570cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.515+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0009c10 con 0x7f8ee4004d40 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.515+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef0036070 con 0x7f8ee4004d40 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.515+0000 7f8eeffff640 1 -- 172.21.15.113:0/4014715518 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0031880 con 0x7f8ee4004d40 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.516+0000 7f8eff4df640 1 -- 172.21.15.113:0/4014715518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee4004d40 msgr2=0x7f8ee40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.516+0000 7f8eff4df640 1 --2- 172.21.15.113:0/4014715518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee4004d40 0x7f8ee40051c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009fc0 tx=0x7f8ee4005830 comp rx=0 tx=0).stop 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.516+0000 7f8efda7c640 1 -- 172.21.15.113:0/4014715518 reap_dead start 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.516+0000 7f8eff4df640 1 -- 172.21.15.113:0/4014715518 shutdown_connections 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.516+0000 7f8eff4df640 1 -- 172.21.15.113:0/4014715518 >> 172.21.15.113:0/4014715518 conn(0x7f8ef80b5c20 msgr2=0x7f8ef80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.516+0000 7f8eff4df640 1 -- 172.21.15.113:0/4014715518 shutdown_connections 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.516+0000 7f8eff4df640 1 -- 172.21.15.113:0/4014715518 wait complete. 2024-04-25T00:07:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.517+0000 7f8eff4df640 1 Processor -- start 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.517+0000 7f8eff4df640 1 -- start start 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eff4df640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ee4004d40 0x7f8ef80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eff4df640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef80cb1e0 0x7f8ef8156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eff4df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ef80b8920 0x7f8ef80b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eff4df640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef815e700 con 0x7f8ef80b8920 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eff4df640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef80d3410 con 0x7f8ef80cb1e0 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eff4df640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef815e880 con 0x7f8ee4004d40 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ef80b8920 0x7f8ef80b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ef80b8920 0x7f8ef80b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37194/0 (socket says 172.21.15.113:37194) 2024-04-25T00:07:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 -- 172.21.15.113:0/1980676190 learned_addr learned my addr 172.21.15.113:0/1980676190 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efd27b640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ee4004d40 0x7f8ef80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efca7a640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef80cb1e0 0x7f8ef8156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 -- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ee4004d40 msgr2=0x7f8ef80cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ee4004d40 0x7f8ef80cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 -- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef80cb1e0 msgr2=0x7f8ef8156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef80cb1e0 0x7f8ef8156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 -- 172.21.15.113:0/1980676190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef80ce030 con 0x7f8ef80b8920 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8efda7c640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ef80b8920 0x7f8ef80b8dd0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0033040 tx=0x7f8ef0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eedffb640 1 -- 172.21.15.113:0/1980676190 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0009c10 con 0x7f8ef80b8920 2024-04-25T00:07:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.518+0000 7f8eedffb640 1 -- 172.21.15.113:0/1980676190 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef0031dd0 con 0x7f8ef80b8920 2024-04-25T00:07:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.519+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef81e8ff0 con 0x7f8ef80b8920 2024-04-25T00:07:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.519+0000 7f8eedffb640 1 -- 172.21.15.113:0/1980676190 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0003bd0 con 0x7f8ef80b8920 2024-04-25T00:07:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.519+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef80cd240 con 0x7f8ef80b8920 2024-04-25T00:07:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.520+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef80691a0 con 0x7f8ef80b8920 2024-04-25T00:07:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.520+0000 7f8eedffb640 1 -- 172.21.15.113:0/1980676190 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ef0005d10 con 0x7f8ef80b8920 2024-04-25T00:07:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.520+0000 7f8eedffb640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8edc06be60 0x7f8edc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.520+0000 7f8eedffb640 1 -- 172.21.15.113:0/1980676190 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4857+0+0 (secure 0 0 0) 0x7f8ef00ac0d0 con 0x7f8ef80b8920 2024-04-25T00:07:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.520+0000 7f8efd27b640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8edc06be60 0x7f8edc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.521+0000 7f8efd27b640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8edc06be60 0x7f8edc06e320 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8001800 tx=0x7f8ee8007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.525+0000 7f8eedffb640 1 -- 172.21.15.113:0/1980676190 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ef004a060 con 0x7f8ef80b8920 2024-04-25T00:07:17.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.699+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ef81e91f0 con 0x7f8ef80b8920 2024-04-25T00:07:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.700+0000 7f8eedffb640 1 -- 172.21.15.113:0/1980676190 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+12949 (secure 0 0 0) 0x7f8ef80ce030 con 0x7f8ef80b8920 2024-04-25T00:07:17.701 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:17.701 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":140,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:16.978384+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8edc06be60 msgr2=0x7f8edc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8edc06be60 0x7f8edc06e320 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8001800 tx=0x7f8ee8007040 comp rx=0 tx=0).stop 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ef80b8920 msgr2=0x7f8ef80b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ef80b8920 0x7f8ef80b8dd0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0033040 tx=0x7f8ef0009750 comp rx=0 tx=0).stop 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 shutdown_connections 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8edc06be60 0x7f8edc06e320 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef80cb1e0 0x7f8ef8156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ef80b8920 0x7f8ef80b8dd0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 --2- 172.21.15.113:0/1980676190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ee4004d40 0x7f8ef80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 >> 172.21.15.113:0/1980676190 conn(0x7f8ef80b5c20 msgr2=0x7f8ef80b98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 shutdown_connections 2024-04-25T00:07:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.702+0000 7f8eff4df640 1 -- 172.21.15.113:0/1980676190 wait complete. 2024-04-25T00:07:17.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:07:17.715 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:07:17.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.814+0000 7f54756e7640 1 Processor -- start 2024-04-25T00:07:17.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.814+0000 7f54756e7640 1 -- start start 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f54756e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f547015e2d0 0x7f547015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f54756e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f54700b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f54756e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547015f4f0 con 0x7f547015e6b0 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f54756e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f54700cde00 con 0x7f547015e2d0 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f54756e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f547015f7d0 con 0x7f54700b3080 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f547015e2d0 0x7f547015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f547015e2d0 0x7f547015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37210/0 (socket says 172.21.15.113:37210) 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f547015e6b0 0x7f5470071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56520/0 (socket says 172.21.15.113:56520) 2024-04-25T00:07:17.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546effd640 1 -- 172.21.15.113:0/2485379862 learned_addr learned my addr 172.21.15.113:0/2485379862 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546e7fc640 1 --2- 172.21.15.113:0/2485379862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f54700b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546effd640 1 -- 172.21.15.113:0/2485379862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 msgr2=0x7f54700b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546effd640 1 --2- 172.21.15.113:0/2485379862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f54700b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546effd640 1 -- 172.21.15.113:0/2485379862 >> v1:172.21.15.113:6789/0 conn(0x7f547015e6b0 legacy=0x7f5470071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546effd640 1 -- 172.21.15.113:0/2485379862 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547015fab0 con 0x7f547015e2d0 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.815+0000 7f546dffb640 1 -- 172.21.15.113:0/2485379862 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029836174 0 0) 0x7f547015f4f0 con 0x7f547015e6b0 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.816+0000 7f546effd640 1 --2- 172.21.15.113:0/2485379862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e2d0 0x7f547015ebf0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72172a97557f9391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.816+0000 7f546dffb640 1 -- 172.21.15.113:0/2485379862 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460037070 con 0x7f547015e2d0 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.816+0000 7f546dffb640 1 -- 172.21.15.113:0/2485379862 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5460032440 con 0x7f547015e2d0 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.816+0000 7f546dffb640 1 -- 172.21.15.113:0/2485379862 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460032810 con 0x7f547015e2d0 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 -- 172.21.15.113:0/2485379862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e2d0 msgr2=0x7f547015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 --2- 172.21.15.113:0/2485379862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e2d0 0x7f547015ebf0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546002a1c0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 -- 172.21.15.113:0/2485379862 shutdown_connections 2024-04-25T00:07:17.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 --2- 172.21.15.113:0/2485379862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f54700b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 --2- 172.21.15.113:0/2485379862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e2d0 0x7f547015ebf0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 -- 172.21.15.113:0/2485379862 >> 172.21.15.113:0/2485379862 conn(0x7f547006b0f0 msgr2=0x7f547006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:17.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 -- 172.21.15.113:0/2485379862 shutdown_connections 2024-04-25T00:07:17.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.817+0000 7f54756e7640 1 -- 172.21.15.113:0/2485379862 wait complete. 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.818+0000 7f54756e7640 1 Processor -- start 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.818+0000 7f54756e7640 1 -- start start 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54756e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f547015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54756e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547015e2d0 0x7f54700cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54756e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e6b0 0x7f547015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54756e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f547015f2b0 con 0x7f547015e6b0 2024-04-25T00:07:17.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54756e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f547015f130 con 0x7f547015e2d0 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54756e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f54700cf2a0 con 0x7f54700b3080 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e6b0 0x7f547015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e6b0 0x7f547015c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37224/0 (socket says 172.21.15.113:37224) 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 -- 172.21.15.113:0/294503260 learned_addr learned my addr 172.21.15.113:0/294503260 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546effd640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f547015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546e7fc640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547015e2d0 0x7f54700cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 -- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 msgr2=0x7f547015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f547015b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 -- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547015e2d0 msgr2=0x7f54700cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547015e2d0 0x7f54700cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 -- 172.21.15.113:0/294503260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547015d390 con 0x7f547015e6b0 2024-04-25T00:07:17.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546e7fc640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547015e2d0 0x7f54700cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f546f7fe640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e6b0 0x7f547015c560 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f546800c700 tx=0x7f546800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54577fe640 1 -- 172.21.15.113:0/294503260 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468018070 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54577fe640 1 -- 172.21.15.113:0/294503260 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546800ce10 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.819+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5470150160 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.820+0000 7f54577fe640 1 -- 172.21.15.113:0/294503260 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546800a860 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.820+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54701509a0 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.821+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54700691a0 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.821+0000 7f54577fe640 1 -- 172.21.15.113:0/294503260 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f546801b070 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.821+0000 7f54577fe640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f545c06be30 0x7f545c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.821+0000 7f546effd640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f545c06be30 0x7f545c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.821+0000 7f54577fe640 1 -- 172.21.15.113:0/294503260 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4857+0+0 (secure 0 0 0) 0x7f546808a090 con 0x7f547015e6b0 2024-04-25T00:07:17.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.822+0000 7f546effd640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f545c06be30 0x7f545c06e2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5460034000 tx=0x7f546002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:17.826+0000 7f54577fe640 1 -- 172.21.15.113:0/294503260 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5470150160 con 0x7f547015e6b0 2024-04-25T00:07:18.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.000+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5470072050 con 0x7f547015e6b0 2024-04-25T00:07:18.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.001+0000 7f54577fe640 1 -- 172.21.15.113:0/294503260 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) ==== 93+0+10 (secure 0 0 0) 0x7f5468058d90 con 0x7f547015e6b0 2024-04-25T00:07:18.001 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:07:18.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f545c06be30 msgr2=0x7f545c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f545c06be30 0x7f545c06e2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5460034000 tx=0x7f546002e000 comp rx=0 tx=0).stop 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e6b0 msgr2=0x7f547015c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e6b0 0x7f547015c560 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f546800c700 tx=0x7f546800cbc0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 shutdown_connections 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f545c06be30 0x7f545c06e2f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547015e2d0 0x7f54700cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547015e6b0 0x7f547015c560 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 --2- 172.21.15.113:0/294503260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54700b3080 0x7f547015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 >> 172.21.15.113:0/294503260 conn(0x7f547006b0f0 msgr2=0x7f54700d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 shutdown_connections 2024-04-25T00:07:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.003+0000 7f54756e7640 1 -- 172.21.15.113:0/294503260 wait complete. 2024-04-25T00:07:18.014 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:07:18.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.110+0000 7f25da872640 1 Processor -- start 2024-04-25T00:07:18.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.110+0000 7f25da872640 1 -- start start 2024-04-25T00:07:18.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25da872640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f25d4144910 0x7f25d40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25da872640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d40b5240 0x7f25d406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25da872640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25d40b5680 0x7f25d406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25da872640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f25d40b4510 con 0x7f25d40b5680 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25da872640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f25d4144cf0 con 0x7f25d4144910 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25da872640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f25d40b2f30 con 0x7f25d40b5240 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25d40b5680 0x7f25d406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25d40b5680 0x7f25d406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37240/0 (socket says 172.21.15.113:37240) 2024-04-25T00:07:18.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 -- 172.21.15.113:0/570409654 learned_addr learned my addr 172.21.15.113:0/570409654 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:18.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d3fff640 1 --2- 172.21.15.113:0/570409654 >> v2:172.21.15.178:3300/0 conn(0x7f25d4144910 0x7f25d40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 -- 172.21.15.113:0/570409654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d40b5240 msgr2=0x7f25d406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:18.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 --2- 172.21.15.113:0/570409654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d40b5240 0x7f25d406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 -- 172.21.15.113:0/570409654 >> v2:172.21.15.178:3300/0 conn(0x7f25d4144910 msgr2=0x7f25d40b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 --2- 172.21.15.113:0/570409654 >> v2:172.21.15.178:3300/0 conn(0x7f25d4144910 0x7f25d40b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.111+0000 7f25d8e0f640 1 -- 172.21.15.113:0/570409654 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d40b3210 con 0x7f25d40b5680 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.112+0000 7f25d8e0f640 1 --2- 172.21.15.113:0/570409654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5680 0x7f25d406e510 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f25d4070af0 tx=0x7f25cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649dce62d63bbf6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.112+0000 7f25d2ffd640 1 -- 172.21.15.113:0/570409654 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc036070 con 0x7f25d40b5680 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.112+0000 7f25d2ffd640 1 -- 172.21.15.113:0/570409654 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc031600 con 0x7f25d40b5680 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.112+0000 7f25d2ffd640 1 -- 172.21.15.113:0/570409654 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0319d0 con 0x7f25d40b5680 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 -- 172.21.15.113:0/570409654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5680 msgr2=0x7f25d406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 --2- 172.21.15.113:0/570409654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5680 0x7f25d406e510 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f25d4070af0 tx=0x7f25cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 -- 172.21.15.113:0/570409654 shutdown_connections 2024-04-25T00:07:18.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 --2- 172.21.15.113:0/570409654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5680 0x7f25d406e510 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 --2- 172.21.15.113:0/570409654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d40b5240 0x7f25d406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 --2- 172.21.15.113:0/570409654 >> v2:172.21.15.178:3300/0 conn(0x7f25d4144910 0x7f25d40b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 -- 172.21.15.113:0/570409654 >> 172.21.15.113:0/570409654 conn(0x7f25d40b1a50 msgr2=0x7f25d40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:18.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 -- 172.21.15.113:0/570409654 shutdown_connections 2024-04-25T00:07:18.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.113+0000 7f25da872640 1 -- 172.21.15.113:0/570409654 wait complete. 2024-04-25T00:07:18.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.114+0000 7f25da872640 1 Processor -- start 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.114+0000 7f25da872640 1 -- start start 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25da872640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5240 0x7f25d4135210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25da872640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d40b5680 0x7f25d40cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25da872640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d4144910 0x7f25d415d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25da872640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f25d40b4da0 con 0x7f25d40b5240 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25da872640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25d40b37d0 con 0x7f25d40b5680 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25da872640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f25d40b4f20 con 0x7f25d4144910 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5240 0x7f25d4135210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5240 0x7f25d4135210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37252/0 (socket says 172.21.15.113:37252) 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 -- 172.21.15.113:0/3898440702 learned_addr learned my addr 172.21.15.113:0/3898440702 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d8e0f640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d4144910 0x7f25d415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d37fe640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d40b5680 0x7f25d40cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 -- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d4144910 msgr2=0x7f25d415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d4144910 0x7f25d415d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 -- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d40b5680 msgr2=0x7f25d40cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d40b5680 0x7f25d40cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 -- 172.21.15.113:0/3898440702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d415d9a0 con 0x7f25d40b5240 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.115+0000 7f25d3fff640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5240 0x7f25d4135210 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c7f0 tx=0x7f25c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.116+0000 7f25d0ff9640 1 -- 172.21.15.113:0/3898440702 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4015070 con 0x7f25d40b5240 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.116+0000 7f25d0ff9640 1 -- 172.21.15.113:0/3898440702 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c4004240 con 0x7f25d40b5240 2024-04-25T00:07:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.116+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d40ce300 con 0x7f25d40b5240 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.116+0000 7f25d0ff9640 1 -- 172.21.15.113:0/3898440702 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4007620 con 0x7f25d40b5240 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.116+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d40ce8c0 con 0x7f25d40b5240 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.117+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d40691a0 con 0x7f25d40b5240 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.117+0000 7f25d0ff9640 1 -- 172.21.15.113:0/3898440702 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25c40079d0 con 0x7f25d40b5240 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.117+0000 7f25d0ff9640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25c006be30 0x7f25c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.118+0000 7f25d0ff9640 1 -- 172.21.15.113:0/3898440702 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4857+0+0 (secure 0 0 0) 0x7f25c408af10 con 0x7f25d40b5240 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.118+0000 7f25d37fe640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25c006be30 0x7f25c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.118+0000 7f25d37fe640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25c006be30 0x7f25c006e2f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f25d413e860 tx=0x7f25c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.122+0000 7f25d0ff9640 1 -- 172.21.15.113:0/3898440702 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25d415d9a0 con 0x7f25d40b5240 2024-04-25T00:07:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.299+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f25d415e090 con 0x7f25d40b5240 2024-04-25T00:07:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.299+0000 7f25d0ff9640 1 -- 172.21.15.113:0/3898440702 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v140) ==== 84+0+65 (secure 0 0 0) 0x7f25c4059c10 con 0x7f25d40b5240 2024-04-25T00:07:18.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:18.300 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:07:18.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:07:18.300 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:07:18.300 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:07:18.300 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:18.300 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25c006be30 msgr2=0x7f25c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25c006be30 0x7f25c006e2f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f25d413e860 tx=0x7f25c8008040 comp rx=0 tx=0).stop 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5240 msgr2=0x7f25d4135210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5240 0x7f25d4135210 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c7f0 tx=0x7f25c400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 shutdown_connections 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f25c006be30 0x7f25c006e2f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d40b5680 0x7f25d40cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25d4144910 0x7f25d415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 --2- 172.21.15.113:0/3898440702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40b5240 0x7f25d4135210 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 >> 172.21.15.113:0/3898440702 conn(0x7f25d40b1a50 msgr2=0x7f25d406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 shutdown_connections 2024-04-25T00:07:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.301+0000 7f25da872640 1 -- 172.21.15.113:0/3898440702 wait complete. 2024-04-25T00:07:18.313 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:07:18.313 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:07:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.412+0000 7f1c62a1b640 1 Processor -- start 2024-04-25T00:07:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.412+0000 7f1c62a1b640 1 -- start start 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c62a1b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 0x7f1c5c0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c62a1b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c5c0b2740 0x7f1c5c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c62a1b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c5c072a80 con 0x7f1c5c15e2f0 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c62a1b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1c5c15e6d0 con 0x7f1c5c0b2740 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c62a1b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c072d20 con 0x7f1c5c06e7e0 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c5bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1c5c15e2f0 0x7f1c5c06e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56526/0 (socket says 172.21.15.113:56526) 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c5bfff640 1 -- 172.21.15.113:0/599154429 learned_addr learned my addr 172.21.15.113:0/599154429 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 925555358 0 0) 0x7f1c5c072a80 con 0x7f1c5c15e2f0 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c44003520 con 0x7f1c5c15e2f0 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c60fb8640 1 --2- 172.21.15.113:0/599154429 >> v2:172.21.15.178:3300/0 conn(0x7f1c5c0b2740 0x7f1c5c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.413+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 0x7f1c5c0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2008788778 0 0) 0x7f1c44003520 con 0x7f1c5c15e2f0 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 msgr2=0x7f1c5c0ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 0x7f1c5c0ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 >> v2:172.21.15.178:3300/0 conn(0x7f1c5c0b2740 msgr2=0x7f1c5c0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 --2- 172.21.15.113:0/599154429 >> v2:172.21.15.178:3300/0 conn(0x7f1c5c0b2740 0x7f1c5c0b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c0ad890 con 0x7f1c5c15e2f0 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1c4c001670 con 0x7f1c5c15e2f0 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 >> v1:172.21.15.113:6789/0 conn(0x7f1c5c15e2f0 legacy=0x7f1c5c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 0x7f1c440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c44003f30 0x7f1c44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c44004d40 0x7f1c440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c15e6d0 con 0x7f1c44003f30 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c072d20 con 0x7f1c44004d40 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4400ba10 con 0x7f1c5c06e7e0 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c44003f30 0x7f1c44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5bfff640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 0x7f1c440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c60fb8640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c44004d40 0x7f1c440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5b7fe640 1 -- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 msgr2=0x7f1c440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c5c06e7e0 0x7f1c440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5b7fe640 1 -- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c44004d40 msgr2=0x7f1c440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c44004d40 0x7f1c440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.414+0000 7f1c5b7fe640 1 -- 172.21.15.113:0/599154429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c44003520 con 0x7f1c44003f30 2024-04-25T00:07:18.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c44003f30 0x7f1c44004800 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009af0 tx=0x7f1c44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a950513f0acd7413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50036070 con 0x7f1c44003f30 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c50031400 con 0x7f1c44003f30 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c5affd640 1 -- 172.21.15.113:0/599154429 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50009690 con 0x7f1c44003f30 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c62a1b640 1 -- 172.21.15.113:0/599154429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c44003f30 msgr2=0x7f1c44004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c62a1b640 1 --2- 172.21.15.113:0/599154429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c44003f30 0x7f1c44004800 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009af0 tx=0x7f1c44005830 comp rx=0 tx=0).stop 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c60fb8640 1 -- 172.21.15.113:0/599154429 reap_dead start 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c62a1b640 1 -- 172.21.15.113:0/599154429 shutdown_connections 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.415+0000 7f1c62a1b640 1 -- 172.21.15.113:0/599154429 >> 172.21.15.113:0/599154429 conn(0x7f1c5c0b1b30 msgr2=0x7f1c5c0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.416+0000 7f1c62a1b640 1 -- 172.21.15.113:0/599154429 shutdown_connections 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.416+0000 7f1c62a1b640 1 -- 172.21.15.113:0/599154429 wait complete. 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.417+0000 7f1c62a1b640 1 Processor -- start 2024-04-25T00:07:18.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.417+0000 7f1c62a1b640 1 -- start start 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c44003f30 0x7f1c5c14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c5c157b80 0x7f1c5c156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c158000 0x7f1c5c1572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0ae130 con 0x7f1c5c157b80 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0ade50 con 0x7f1c5c158000 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c0aed70 con 0x7f1c44003f30 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c5c157b80 0x7f1c5c156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c5c157b80 0x7f1c5c156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37264/0 (socket says 172.21.15.113:37264) 2024-04-25T00:07:18.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 -- 172.21.15.113:0/3320561564 learned_addr learned my addr 172.21.15.113:0/3320561564 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5bfff640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c44003f30 0x7f1c5c14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c60fb8640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c158000 0x7f1c5c1572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 -- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c44003f30 msgr2=0x7f1c5c14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c44003f30 0x7f1c5c14b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 -- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c158000 msgr2=0x7f1c5c1572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c158000 0x7f1c5c1572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 -- 172.21.15.113:0/3320561564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c15fab0 con 0x7f1c5c157b80 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c5b7fe640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c5c157b80 0x7f1c5c156d60 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f1c50034040 tx=0x7f1c50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c58ff9640 1 -- 172.21.15.113:0/3320561564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50040020 con 0x7f1c5c157b80 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c58ff9640 1 -- 172.21.15.113:0/3320561564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c50003810 con 0x7f1c5c157b80 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c15fd90 con 0x7f1c5c157b80 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c58ff9640 1 -- 172.21.15.113:0/3320561564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50036040 con 0x7f1c5c157b80 2024-04-25T00:07:18.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.418+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c160350 con 0x7f1c5c157b80 2024-04-25T00:07:18.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.419+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c0691a0 con 0x7f1c5c157b80 2024-04-25T00:07:18.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.420+0000 7f1c58ff9640 1 -- 172.21.15.113:0/3320561564 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c50043020 con 0x7f1c5c157b80 2024-04-25T00:07:18.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.420+0000 7f1c58ff9640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.420+0000 7f1c58ff9640 1 -- 172.21.15.113:0/3320561564 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4857+0+0 (secure 0 0 0) 0x7f1c500aa980 con 0x7f1c5c157b80 2024-04-25T00:07:18.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.420+0000 7f1c5bfff640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.421+0000 7f1c5bfff640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c4806be30 0x7f1c4806e2f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c001050 tx=0x7f1c4c007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.424+0000 7f1c58ff9640 1 -- 172.21.15.113:0/3320561564 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c5004a070 con 0x7f1c5c157b80 2024-04-25T00:07:18.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.595+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1c5c0719e0 con 0x7f1c5c157b80 2024-04-25T00:07:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.989+0000 7f1c58ff9640 1 -- 172.21.15.113:0/3320561564 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v141) ==== 138+0+0 (secure 0 0 0) 0x7f1c500790c0 con 0x7f1c5c157b80 2024-04-25T00:07:18.990 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:07:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.993+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c4806be30 msgr2=0x7f1c4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.993+0000 7f1c62a1b640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c4806be30 0x7f1c4806e2f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c001050 tx=0x7f1c4c007970 comp rx=0 tx=0).stop 2024-04-25T00:07:18.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.993+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c5c157b80 msgr2=0x7f1c5c156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.993+0000 7f1c62a1b640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c5c157b80 0x7f1c5c156d60 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f1c50034040 tx=0x7f1c50004730 comp rx=0 tx=0).stop 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 shutdown_connections 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c4806be30 0x7f1c4806e2f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c158000 0x7f1c5c1572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c5c157b80 0x7f1c5c156d60 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 --2- 172.21.15.113:0/3320561564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c44003f30 0x7f1c5c14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 >> 172.21.15.113:0/3320561564 conn(0x7f1c5c0b1b30 msgr2=0x7f1c5c070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 shutdown_connections 2024-04-25T00:07:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:18.994+0000 7f1c62a1b640 1 -- 172.21.15.113:0/3320561564 wait complete. 2024-04-25T00:07:19.008 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:07:19.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.113+0000 7fd0528c3640 1 Processor -- start 2024-04-25T00:07:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.113+0000 7fd0528c3640 1 -- start start 2024-04-25T00:07:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd0528c3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd04c13f050 0x7fd04c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd0528c3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c0d04f0 0x7fd04c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd0528c3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd04c15d6b0 con 0x7fd04c1604d0 2024-04-25T00:07:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd0528c3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd04c0d09a0 con 0x7fd04c13f050 2024-04-25T00:07:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd0528c3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c15c6a0 con 0x7fd04c0d04f0 2024-04-25T00:07:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd04c13f050 0x7fd04c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd04c13f050 0x7fd04c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37276/0 (socket says 172.21.15.113:37276) 2024-04-25T00:07:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04bfff640 1 -- 172.21.15.113:0/1172419039 learned_addr learned my addr 172.21.15.113:0/1172419039 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04b7fe640 1 --2- 172.21.15.113:0/1172419039 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c0d04f0 0x7fd04c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04bfff640 1 -- 172.21.15.113:0/1172419039 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c0d04f0 msgr2=0x7fd04c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04bfff640 1 --2- 172.21.15.113:0/1172419039 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c0d04f0 0x7fd04c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04bfff640 1 -- 172.21.15.113:0/1172419039 >> v1:172.21.15.178:6789/0 conn(0x7fd04c1604d0 legacy=0x7fd04c06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.114+0000 7fd04bfff640 1 -- 172.21.15.113:0/1172419039 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c15c8f0 con 0x7fd04c13f050 2024-04-25T00:07:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.115+0000 7fd04bfff640 1 --2- 172.21.15.113:0/1172419039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0cffb0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009870 tx=0x7fd03c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9824cc3ed1c532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:19.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.115+0000 7fd04affd640 1 -- 172.21.15.113:0/1172419039 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c037070 con 0x7fd04c13f050 2024-04-25T00:07:19.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.115+0000 7fd04affd640 1 -- 172.21.15.113:0/1172419039 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03c032440 con 0x7fd04c13f050 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.115+0000 7fd04affd640 1 -- 172.21.15.113:0/1172419039 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c032810 con 0x7fd04c13f050 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 -- 172.21.15.113:0/1172419039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 msgr2=0x7fd04c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 --2- 172.21.15.113:0/1172419039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0cffb0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009870 tx=0x7fd03c02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 -- 172.21.15.113:0/1172419039 shutdown_connections 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 --2- 172.21.15.113:0/1172419039 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c0d04f0 0x7fd04c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 --2- 172.21.15.113:0/1172419039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0cffb0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 -- 172.21.15.113:0/1172419039 >> 172.21.15.113:0/1172419039 conn(0x7fd04c06b0f0 msgr2=0x7fd04c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 -- 172.21.15.113:0/1172419039 shutdown_connections 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.116+0000 7fd0528c3640 1 -- 172.21.15.113:0/1172419039 wait complete. 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.117+0000 7fd0528c3640 1 Processor -- start 2024-04-25T00:07:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.117+0000 7fd0528c3640 1 -- start start 2024-04-25T00:07:19.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd0528c3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd04c0d04f0 0x7fd04c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd0528c3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd0528c3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c1604d0 0x7fd04c0ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd0528c3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c15d190 con 0x7fd04c13f050 2024-04-25T00:07:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd0528c3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c15ceb0 con 0x7fd04c0d04f0 2024-04-25T00:07:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd0528c3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c13f430 con 0x7fd04c1604d0 2024-04-25T00:07:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37288/0 (socket says 172.21.15.113:37288) 2024-04-25T00:07:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 -- 172.21.15.113:0/4055129966 learned_addr learned my addr 172.21.15.113:0/4055129966 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04bfff640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd04c0d04f0 0x7fd04c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd050e60640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c1604d0 0x7fd04c0ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 -- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c1604d0 msgr2=0x7fd04c0ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:19.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c1604d0 0x7fd04c0ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 -- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd04c0d04f0 msgr2=0x7fd04c151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:19.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd04c0d04f0 0x7fd04c151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 -- 172.21.15.113:0/4055129966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c0ac7c0 con 0x7fd04c13f050 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.118+0000 7fd04b7fe640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0abe00 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd04000d800 tx=0x7fd04000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.119+0000 7fd048ff9640 1 -- 172.21.15.113:0/4055129966 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04000b6c0 con 0x7fd04c13f050 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.119+0000 7fd048ff9640 1 -- 172.21.15.113:0/4055129966 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd040019070 con 0x7fd04c13f050 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.119+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c0acaa0 con 0x7fd04c13f050 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.119+0000 7fd048ff9640 1 -- 172.21.15.113:0/4055129966 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04000ba80 con 0x7fd04c13f050 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.119+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c0a9a30 con 0x7fd04c13f050 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.120+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c0a9e90 con 0x7fd04c13f050 2024-04-25T00:07:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.120+0000 7fd048ff9640 1 -- 172.21.15.113:0/4055129966 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd04000bc20 con 0x7fd04c13f050 2024-04-25T00:07:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.121+0000 7fd048ff9640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03806be30 0x7fd03806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.121+0000 7fd048ff9640 1 -- 172.21.15.113:0/4055129966 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5227+0+0 (secure 0 0 0) 0x7fd04008af70 con 0x7fd04c13f050 2024-04-25T00:07:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.121+0000 7fd04bfff640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03806be30 0x7fd03806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.121+0000 7fd04bfff640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03806be30 0x7fd03806e2f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd03c034000 tx=0x7fd03c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.125+0000 7fd048ff9640 1 -- 172.21.15.113:0/4055129966 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd040059540 con 0x7fd04c13f050 2024-04-25T00:07:19.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.300+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd04c160900 con 0x7fd04c13f050 2024-04-25T00:07:19.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:19.996+0000 7fd048ff9640 1 -- 172.21.15.113:0/4055129966 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v142) ==== 170+0+0 (secure 0 0 0) 0x7fd04c160900 con 0x7fd04c13f050 2024-04-25T00:07:19.997 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:07:20.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03806be30 msgr2=0x7fd03806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03806be30 0x7fd03806e2f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd03c034000 tx=0x7fd03c02e000 comp rx=0 tx=0).stop 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 msgr2=0x7fd04c0abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0abe00 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd04000d800 tx=0x7fd04000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 shutdown_connections 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03806be30 0x7fd03806e2f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd04c1604d0 0x7fd04c0ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04c13f050 0x7fd04c0abe00 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 --2- 172.21.15.113:0/4055129966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd04c0d04f0 0x7fd04c151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.000+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 >> 172.21.15.113:0/4055129966 conn(0x7fd04c06b0f0 msgr2=0x7fd04c13cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:20.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.003+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 shutdown_connections 2024-04-25T00:07:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.005+0000 7fd0528c3640 1 -- 172.21.15.113:0/4055129966 wait complete. 2024-04-25T00:07:20.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:07:20.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.133+0000 7f9953ec3640 1 Processor -- start 2024-04-25T00:07:20.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9953ec3640 1 -- start start 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9953ec3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b5ca0 0x7f994c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9953ec3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f994c0b73c0 0x7f994c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9953ec3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994c0d2e00 con 0x7f994c15e510 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9953ec3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f994c0b6640 con 0x7f994c0b73c0 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9953ec3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0d30e0 con 0x7f994c0b5ca0 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9951c5f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f994c15e510 0x7f994c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56530/0 (socket says 172.21.15.113:56530) 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9951c5f640 1 -- 172.21.15.113:0/2768985509 learned_addr learned my addr 172.21.15.113:0/2768985509 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.134+0000 7f9952460640 1 --2- 172.21.15.113:0/2768985509 >> v2:172.21.15.113:3300/0 conn(0x7f994c0b73c0 0x7f994c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f9950c5d640 1 -- 172.21.15.113:0/2768985509 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2529009577 0 0) 0x7f994c0d2e00 con 0x7f994c15e510 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f9950c5d640 1 -- 172.21.15.113:0/2768985509 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9934003520 con 0x7f994c15e510 2024-04-25T00:07:20.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f995145e640 1 --2- 172.21.15.113:0/2768985509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b5ca0 0x7f994c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f9952460640 1 -- 172.21.15.113:0/2768985509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b5ca0 msgr2=0x7f994c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f9952460640 1 --2- 172.21.15.113:0/2768985509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b5ca0 0x7f994c0b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f9952460640 1 -- 172.21.15.113:0/2768985509 >> v1:172.21.15.113:6789/0 conn(0x7f994c15e510 legacy=0x7f994c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f9952460640 1 -- 172.21.15.113:0/2768985509 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c0d33c0 con 0x7f994c0b73c0 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.135+0000 7f9952460640 1 --2- 172.21.15.113:0/2768985509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c0b73c0 0x7f994c0b7830 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f99440097a0 tx=0x7f994402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60bdc5e71357add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9950c5d640 1 -- 172.21.15.113:0/2768985509 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944037070 con 0x7f994c0b73c0 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9950c5d640 1 -- 172.21.15.113:0/2768985509 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994402ae50 con 0x7f994c0b73c0 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9950c5d640 1 -- 172.21.15.113:0/2768985509 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944032660 con 0x7f994c0b73c0 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 -- 172.21.15.113:0/2768985509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c0b73c0 msgr2=0x7f994c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2768985509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c0b73c0 0x7f994c0b7830 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f99440097a0 tx=0x7f994402a100 comp rx=0 tx=0).stop 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 -- 172.21.15.113:0/2768985509 shutdown_connections 2024-04-25T00:07:20.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2768985509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c0b73c0 0x7f994c0b7830 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2768985509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b5ca0 0x7f994c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 -- 172.21.15.113:0/2768985509 >> 172.21.15.113:0/2768985509 conn(0x7f994c06b720 msgr2=0x7f994c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:20.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 -- 172.21.15.113:0/2768985509 shutdown_connections 2024-04-25T00:07:20.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.136+0000 7f9953ec3640 1 -- 172.21.15.113:0/2768985509 wait complete. 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.137+0000 7f9953ec3640 1 Processor -- start 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.137+0000 7f9953ec3640 1 -- start start 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9953ec3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f994c0b5ca0 0x7f994c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9953ec3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b73c0 0x7f994c14b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9953ec3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c15e510 0x7f994c0d4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9953ec3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0d3ea0 con 0x7f994c15e510 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9953ec3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0d3980 con 0x7f994c0b5ca0 2024-04-25T00:07:20.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9953ec3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0d2a90 con 0x7f994c0b73c0 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9952460640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c15e510 0x7f994c0d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9952460640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c15e510 0x7f994c0d4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37298/0 (socket says 172.21.15.113:37298) 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9952460640 1 -- 172.21.15.113:0/2893182840 learned_addr learned my addr 172.21.15.113:0/2893182840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9951c5f640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f994c0b5ca0 0x7f994c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f995145e640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b73c0 0x7f994c14b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9952460640 1 -- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b73c0 msgr2=0x7f994c14b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9952460640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b73c0 0x7f994c14b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9952460640 1 -- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f994c0b5ca0 msgr2=0x7f994c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.138+0000 7f9952460640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f994c0b5ca0 0x7f994c0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.139+0000 7f9952460640 1 -- 172.21.15.113:0/2893182840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c14bf00 con 0x7f994c15e510 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.139+0000 7f9952460640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c15e510 0x7f994c0d4200 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f99440098c0 tx=0x7f99440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:20.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.139+0000 7f994a7fc640 1 -- 172.21.15.113:0/2893182840 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944041020 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.139+0000 7f994a7fc640 1 -- 172.21.15.113:0/2893182840 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994403c670 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.139+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1f8250 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.139+0000 7f994a7fc640 1 -- 172.21.15.113:0/2893182840 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944037040 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.139+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1f79b0 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.140+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994c0691a0 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.140+0000 7f994a7fc640 1 -- 172.21.15.113:0/2893182840 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9944044020 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.141+0000 7f994a7fc640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f993806be30 0x7f993806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.141+0000 7f9951c5f640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f993806be30 0x7f993806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.141+0000 7f994a7fc640 1 -- 172.21.15.113:0/2893182840 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5609+0+0 (secure 0 0 0) 0x7f99440ab510 con 0x7f994c15e510 2024-04-25T00:07:20.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.141+0000 7f9951c5f640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f993806be30 0x7f993806e2f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f993c005c00 tx=0x7f993c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:20.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.145+0000 7f994a7fc640 1 -- 172.21.15.113:0/2893182840 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f994404b030 con 0x7f994c15e510 2024-04-25T00:07:20.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.359+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f994c0b55f0 con 0x7f994c15e510 2024-04-25T00:07:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:20.998+0000 7f994a7fc640 1 -- 172.21.15.113:0/2893182840 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v76) ==== 174+0+0 (secure 0 0 0) 0x7f9944079fb0 con 0x7f994c15e510 2024-04-25T00:07:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 20 and data pool 21 2024-04-25T00:07:21.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.003+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f993806be30 msgr2=0x7f993806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.003+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f993806be30 0x7f993806e2f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f993c005c00 tx=0x7f993c009040 comp rx=0 tx=0).stop 2024-04-25T00:07:21.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.003+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c15e510 msgr2=0x7f994c0d4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.003+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c15e510 0x7f994c0d4200 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f99440098c0 tx=0x7f99440046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.005+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 shutdown_connections 2024-04-25T00:07:21.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.005+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f993806be30 0x7f993806e2f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.005+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f994c15e510 0x7f994c0d4200 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.005+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994c0b73c0 0x7f994c14b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.005+0000 7f9953ec3640 1 --2- 172.21.15.113:0/2893182840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f994c0b5ca0 0x7f994c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.005+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 >> 172.21.15.113:0/2893182840 conn(0x7f994c06b720 msgr2=0x7f994c0b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:21.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.006+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 shutdown_connections 2024-04-25T00:07:21.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.007+0000 7f9953ec3640 1 -- 172.21.15.113:0/2893182840 wait complete. 2024-04-25T00:07:21.035 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.134+0000 7f8a7a742640 1 Processor -- start 2024-04-25T00:07:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.134+0000 7f8a7a742640 1 -- start start 2024-04-25T00:07:21.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a7a742640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8a74144910 0x7f8a740b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a7a742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b56b0 0x7f8a740cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a7a742640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a74144cf0 con 0x7f8a740b5240 2024-04-25T00:07:21.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a7a742640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8a740b4510 con 0x7f8a74144910 2024-04-25T00:07:21.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a7a742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8a740b2f30 con 0x7f8a740b56b0 2024-04-25T00:07:21.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a78cdf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b56b0 0x7f8a740cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a73fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8a74144910 0x7f8a740b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a737fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8a740b5240 0x7f8a7406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:49318/0 (socket says 172.21.15.113:49318) 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a73fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8a74144910 0x7f8a740b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55666/0 (socket says 172.21.15.113:55666) 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a737fe640 1 -- 172.21.15.113:0/4258485985 learned_addr learned my addr 172.21.15.113:0/4258485985 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.135+0000 7f8a78cdf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b56b0 0x7f8a740cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:44198/0 (socket says 172.21.15.113:44198) 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.136+0000 7f8a73fff640 1 -- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b56b0 msgr2=0x7f8a740cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.136+0000 7f8a73fff640 1 --2- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b56b0 0x7f8a740cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.136+0000 7f8a73fff640 1 -- 172.21.15.113:0/4258485985 >> v1:172.21.15.178:6789/0 conn(0x7f8a740b5240 legacy=0x7f8a7406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.136+0000 7f8a73fff640 1 -- 172.21.15.113:0/4258485985 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a740b3210 con 0x7f8a74144910 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.136+0000 7f8a78cdf640 1 --2- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b56b0 0x7f8a740cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.136+0000 7f8a73fff640 1 --2- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a74144910 0x7f8a740b3fd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8a640097d0 tx=0x7f8a6402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ea1d06ae896cb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a72ffd640 1 -- 172.21.15.113:0/4258485985 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64037070 con 0x7f8a74144910 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a72ffd640 1 -- 172.21.15.113:0/4258485985 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a64032440 con 0x7f8a74144910 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a72ffd640 1 -- 172.21.15.113:0/4258485985 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64032810 con 0x7f8a74144910 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 -- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a74144910 msgr2=0x7f8a740b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 --2- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a74144910 0x7f8a740b3fd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8a640097d0 tx=0x7f8a6402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 -- 172.21.15.113:0/4258485985 shutdown_connections 2024-04-25T00:07:21.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 --2- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b56b0 0x7f8a740cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 --2- 172.21.15.113:0/4258485985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a74144910 0x7f8a740b3fd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 -- 172.21.15.113:0/4258485985 >> 172.21.15.113:0/4258485985 conn(0x7f8a740b1a50 msgr2=0x7f8a740b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:21.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 -- 172.21.15.113:0/4258485985 shutdown_connections 2024-04-25T00:07:21.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.137+0000 7f8a7a742640 1 -- 172.21.15.113:0/4258485985 wait complete. 2024-04-25T00:07:21.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.138+0000 7f8a7a742640 1 Processor -- start 2024-04-25T00:07:21.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.138+0000 7f8a7a742640 1 -- start start 2024-04-25T00:07:21.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a7a742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b5240 0x7f8a74140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a7a742640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a740b56b0 0x7f8a741f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a7a742640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74144910 0x7f8a74142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a7a742640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8a740b4da0 con 0x7f8a74144910 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a7a742640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a740b37d0 con 0x7f8a740b56b0 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a7a742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8a740b4f20 con 0x7f8a740b5240 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a78cdf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74144910 0x7f8a74142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a78cdf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74144910 0x7f8a74142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37304/0 (socket says 172.21.15.113:37304) 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a78cdf640 1 -- 172.21.15.113:0/404990906 learned_addr learned my addr 172.21.15.113:0/404990906 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a737fe640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a740b56b0 0x7f8a741f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.139+0000 7f8a73fff640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b5240 0x7f8a74140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a78cdf640 1 -- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b5240 msgr2=0x7f8a74140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a78cdf640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b5240 0x7f8a74140280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a78cdf640 1 -- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a740b56b0 msgr2=0x7f8a741f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a78cdf640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a740b56b0 0x7f8a741f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a78cdf640 1 -- 172.21.15.113:0/404990906 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a741407c0 con 0x7f8a74144910 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a737fe640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a740b56b0 0x7f8a741f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a78cdf640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74144910 0x7f8a74142040 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00efb0 tx=0x7f8a6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a70ff9640 1 -- 172.21.15.113:0/404990906 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c018020 con 0x7f8a74144910 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a70ff9640 1 -- 172.21.15.113:0/404990906 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a6c014480 con 0x7f8a74144910 2024-04-25T00:07:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a70ff9640 1 -- 172.21.15.113:0/404990906 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c014830 con 0x7f8a74144910 2024-04-25T00:07:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a741e6b10 con 0x7f8a74144910 2024-04-25T00:07:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.140+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741e70d0 con 0x7f8a74144910 2024-04-25T00:07:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.141+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a741e7890 con 0x7f8a74144910 2024-04-25T00:07:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.141+0000 7f8a70ff9640 1 -- 172.21.15.113:0/404990906 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a6c005d10 con 0x7f8a74144910 2024-04-25T00:07:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.142+0000 7f8a70ff9640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a6006be30 0x7f8a6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.142+0000 7f8a70ff9640 1 -- 172.21.15.113:0/404990906 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f8a6c08bba0 con 0x7f8a74144910 2024-04-25T00:07:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.142+0000 7f8a73fff640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a6006be30 0x7f8a6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.142+0000 7f8a73fff640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a6006be30 0x7f8a6006e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8a64034000 tx=0x7f8a6402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.146+0000 7f8a70ff9640 1 -- 172.21.15.113:0/404990906 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a741e7890 con 0x7f8a74144910 2024-04-25T00:07:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.322+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a740cc580 con 0x7f8a74144910 2024-04-25T00:07:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.323+0000 7f8a70ff9640 1 -- 172.21.15.113:0/404990906 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16740 (secure 0 0 0) 0x7f8a6c022030 con 0x7f8a74144910 2024-04-25T00:07:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:21.324 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":143,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:20.360984+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:07:18.614948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:07:19.319166+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:21.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a6006be30 msgr2=0x7f8a6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a6006be30 0x7f8a6006e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8a64034000 tx=0x7f8a6402e000 comp rx=0 tx=0).stop 2024-04-25T00:07:21.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74144910 msgr2=0x7f8a74142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74144910 0x7f8a74142040 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00efb0 tx=0x7f8a6c00c490 comp rx=0 tx=0).stop 2024-04-25T00:07:21.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 shutdown_connections 2024-04-25T00:07:21.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8a6006be30 0x7f8a6006e2f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a740b56b0 0x7f8a741f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a74144910 0x7f8a74142040 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 --2- 172.21.15.113:0/404990906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a740b5240 0x7f8a74140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 >> 172.21.15.113:0/404990906 conn(0x7f8a740b1a50 msgr2=0x7f8a74072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:21.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 shutdown_connections 2024-04-25T00:07:21.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.325+0000 7f8a7a742640 1 -- 172.21.15.113:0/404990906 wait complete. 2024-04-25T00:07:21.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:21.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.434+0000 7f7250464640 1 Processor -- start 2024-04-25T00:07:21.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.434+0000 7f7250464640 1 -- start start 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.434+0000 7f7250464640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f72480d2290 0x7f72480b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.434+0000 7f7250464640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f72480d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.434+0000 7f7250464640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72480d2800 con 0x7f724815e320 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f7250464640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f72480d14d0 con 0x7f72480d2290 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f7250464640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f72480d3410 con 0x7f72480d0b00 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724e200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f724815e320 0x7f72480d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56540/0 (socket says 172.21.15.113:56540) 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724e200640 1 -- 172.21.15.113:0/1435748838 learned_addr learned my addr 172.21.15.113:0/1435748838 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724d9ff640 1 --2- 172.21.15.113:0/1435748838 >> v2:172.21.15.178:3300/0 conn(0x7f72480d2290 0x7f72480b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724ea01640 1 --2- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f72480d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724d1fe640 1 -- 172.21.15.113:0/1435748838 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003305178 0 0) 0x7f72480d2800 con 0x7f724815e320 2024-04-25T00:07:21.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724d1fe640 1 -- 172.21.15.113:0/1435748838 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7234003520 con 0x7f724815e320 2024-04-25T00:07:21.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724d9ff640 1 -- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 msgr2=0x7f72480d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:21.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724d9ff640 1 --2- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f72480d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724d9ff640 1 -- 172.21.15.113:0/1435748838 >> v1:172.21.15.113:6789/0 conn(0x7f724815e320 legacy=0x7f72480d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.435+0000 7f724d9ff640 1 -- 172.21.15.113:0/1435748838 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72480d2ae0 con 0x7f72480d2290 2024-04-25T00:07:21.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.436+0000 7f724d9ff640 1 --2- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480d2290 0x7f72480b9aa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72380097a0 tx=0x7f723802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=117b9a3d4da861dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.436+0000 7f724d1fe640 1 -- 172.21.15.113:0/1435748838 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238037070 con 0x7f72480d2290 2024-04-25T00:07:21.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.436+0000 7f724d1fe640 1 -- 172.21.15.113:0/1435748838 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7238032440 con 0x7f72480d2290 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.436+0000 7f724d1fe640 1 -- 172.21.15.113:0/1435748838 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72380327f0 con 0x7f72480d2290 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 -- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480d2290 msgr2=0x7f72480b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 --2- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480d2290 0x7f72480b9aa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72380097a0 tx=0x7f723802a210 comp rx=0 tx=0).stop 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 -- 172.21.15.113:0/1435748838 shutdown_connections 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 --2- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f72480d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 --2- 172.21.15.113:0/1435748838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72480d2290 0x7f72480b9aa0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 -- 172.21.15.113:0/1435748838 >> 172.21.15.113:0/1435748838 conn(0x7f72480b5c20 msgr2=0x7f72480b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 -- 172.21.15.113:0/1435748838 shutdown_connections 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.437+0000 7f7250464640 1 -- 172.21.15.113:0/1435748838 wait complete. 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.438+0000 7f7250464640 1 Processor -- start 2024-04-25T00:07:21.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.438+0000 7f7250464640 1 -- start start 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f7250464640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f724814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f7250464640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72480d2290 0x7f72480ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f7250464640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f724815e320 0x7f72480b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f7250464640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f72480b9fe0 con 0x7f72480d2290 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f7250464640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72480d30a0 con 0x7f724815e320 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f7250464640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f724815e700 con 0x7f72480d0b00 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72480d2290 0x7f72480ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72480d2290 0x7f72480ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37312/0 (socket says 172.21.15.113:37312) 2024-04-25T00:07:21.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 -- 172.21.15.113:0/131680689 learned_addr learned my addr 172.21.15.113:0/131680689 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724e200640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f724814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724ea01640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f724815e320 0x7f72480b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 -- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 msgr2=0x7f724814ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f724814ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 -- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f724815e320 msgr2=0x7f72480b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f724815e320 0x7f72480b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 -- 172.21.15.113:0/131680689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724815eaf0 con 0x7f72480d2290 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f724d9ff640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72480d2290 0x7f72480ccf40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f7238035040 tx=0x7f72380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.439+0000 7f723effd640 1 -- 172.21.15.113:0/131680689 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238037040 con 0x7f72480d2290 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.440+0000 7f723effd640 1 -- 172.21.15.113:0/131680689 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7238042070 con 0x7f72480d2290 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.440+0000 7f723effd640 1 -- 172.21.15.113:0/131680689 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723803b930 con 0x7f72480d2290 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.440+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724815fb50 con 0x7f72480d2290 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.440+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724815f070 con 0x7f72480d2290 2024-04-25T00:07:21.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.441+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72480691a0 con 0x7f72480d2290 2024-04-25T00:07:21.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.441+0000 7f723effd640 1 -- 172.21.15.113:0/131680689 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f723803bad0 con 0x7f72480d2290 2024-04-25T00:07:21.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.441+0000 7f723effd640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f722c06be60 0x7f722c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.441+0000 7f723effd640 1 -- 172.21.15.113:0/131680689 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f72380ab730 con 0x7f72480d2290 2024-04-25T00:07:21.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.441+0000 7f724e200640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f722c06be60 0x7f722c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.442+0000 7f724e200640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f722c06be60 0x7f722c06e320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7244001580 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.446+0000 7f723effd640 1 -- 172.21.15.113:0/131680689 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f723807a050 con 0x7f72480d2290 2024-04-25T00:07:21.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.618+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f724815dac0 con 0x7f72480d2290 2024-04-25T00:07:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.619+0000 7f723effd640 1 -- 172.21.15.113:0/131680689 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16740 (secure 0 0 0) 0x7f7238079950 con 0x7f72480d2290 2024-04-25T00:07:21.620 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:21.620 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":143,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:20.360984+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:07:18.614948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:07:19.319166+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f722c06be60 msgr2=0x7f722c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f722c06be60 0x7f722c06e320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7244001580 tx=0x7f7244009040 comp rx=0 tx=0).stop 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72480d2290 msgr2=0x7f72480ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72480d2290 0x7f72480ccf40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f7238035040 tx=0x7f72380046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 shutdown_connections 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f722c06be60 0x7f722c06e320 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f724815e320 0x7f72480b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72480d2290 0x7f72480ccf40 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 --2- 172.21.15.113:0/131680689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72480d0b00 0x7f724814ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 >> 172.21.15.113:0/131680689 conn(0x7f72480b5c20 msgr2=0x7f72480b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:21.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 shutdown_connections 2024-04-25T00:07:21.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.621+0000 7f7250464640 1 -- 172.21.15.113:0/131680689 wait complete. 2024-04-25T00:07:21.633 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:07:21.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.732+0000 7f1290b96640 1 Processor -- start 2024-04-25T00:07:21.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.732+0000 7f1290b96640 1 -- start start 2024-04-25T00:07:21.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.733+0000 7f1290b96640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f128c0b5ca0 0x7f128c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.733+0000 7f1290b96640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0b73c0 0x7f128c0b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.733+0000 7f1290b96640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128c0d2e00 con 0x7f128c15e510 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.733+0000 7f1290b96640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f128c0b6640 con 0x7f128c0b5ca0 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.733+0000 7f1290b96640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f128c0d30e0 con 0x7f128c0b73c0 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.733+0000 7f128a59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f128c15e510 0x7f128c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56546/0 (socket says 172.21.15.113:56546) 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.733+0000 7f128a59c640 1 -- 172.21.15.113:0/811844034 learned_addr learned my addr 172.21.15.113:0/811844034 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f1289d9b640 1 --2- 172.21.15.113:0/811844034 >> v2:172.21.15.113:3300/0 conn(0x7f128c0b5ca0 0x7f128c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f128ad9d640 1 --2- 172.21.15.113:0/811844034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0b73c0 0x7f128c0b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f128959a640 1 -- 172.21.15.113:0/811844034 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784441378 0 0) 0x7f128c0d2e00 con 0x7f128c15e510 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f128959a640 1 -- 172.21.15.113:0/811844034 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1274003520 con 0x7f128c15e510 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f1289d9b640 1 -- 172.21.15.113:0/811844034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0b73c0 msgr2=0x7f128c0b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f1289d9b640 1 --2- 172.21.15.113:0/811844034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0b73c0 0x7f128c0b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f1289d9b640 1 -- 172.21.15.113:0/811844034 >> v1:172.21.15.113:6789/0 conn(0x7f128c15e510 legacy=0x7f128c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f1289d9b640 1 -- 172.21.15.113:0/811844034 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c0d33c0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.734+0000 7f1289d9b640 1 --2- 172.21.15.113:0/811844034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0b6100 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f1280009d90 tx=0x7f128002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aef9ac28575f9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f128959a640 1 -- 172.21.15.113:0/811844034 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280031600 con 0x7f128c0b5ca0 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f128959a640 1 -- 172.21.15.113:0/811844034 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12800317a0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f128959a640 1 -- 172.21.15.113:0/811844034 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280031b70 con 0x7f128c0b5ca0 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 -- 172.21.15.113:0/811844034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 msgr2=0x7f128c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 --2- 172.21.15.113:0/811844034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0b6100 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f1280009d90 tx=0x7f128002a5d0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 -- 172.21.15.113:0/811844034 shutdown_connections 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 --2- 172.21.15.113:0/811844034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c0b73c0 0x7f128c0b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 --2- 172.21.15.113:0/811844034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0b6100 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 -- 172.21.15.113:0/811844034 >> 172.21.15.113:0/811844034 conn(0x7f128c06b720 msgr2=0x7f128c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 -- 172.21.15.113:0/811844034 shutdown_connections 2024-04-25T00:07:21.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.735+0000 7f1290b96640 1 -- 172.21.15.113:0/811844034 wait complete. 2024-04-25T00:07:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 Processor -- start 2024-04-25T00:07:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 -- start start 2024-04-25T00:07:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0b73c0 0x7f128c160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c15e510 0x7f128c161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0d3ea0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0d3980 con 0x7f128c0b73c0 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f1290b96640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f128c0d2a90 con 0x7f128c15e510 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f128a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f128a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37330/0 (socket says 172.21.15.113:37330) 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.737+0000 7f128a59c640 1 -- 172.21.15.113:0/2444258828 learned_addr learned my addr 172.21.15.113:0/2444258828 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f128ad9d640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c15e510 0x7f128c161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f1289d9b640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0b73c0 0x7f128c160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f128a59c640 1 -- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c15e510 msgr2=0x7f128c161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f128a59c640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c15e510 0x7f128c161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f128a59c640 1 -- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0b73c0 msgr2=0x7f128c160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f128a59c640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0b73c0 0x7f128c160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f128a59c640 1 -- 172.21.15.113:0/2444258828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c0cb940 con 0x7f128c0b5ca0 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f128a59c640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0c8110 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f128400c430 tx=0x7f128400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f127affd640 1 -- 172.21.15.113:0/2444258828 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284017070 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f127affd640 1 -- 172.21.15.113:0/2444258828 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1284005ce0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c0cbbe0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f127affd640 1 -- 172.21.15.113:0/2444258828 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12840072f0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.738+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c0cc1a0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.739+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c0691a0 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.740+0000 7f127affd640 1 -- 172.21.15.113:0/2444258828 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f128401a070 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.740+0000 7f127affd640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.740+0000 7f127affd640 1 -- 172.21.15.113:0/2444258828 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f128408a250 con 0x7f128c0b5ca0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.740+0000 7f1289d9b640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.740+0000 7f1289d9b640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f126c06be30 0x7f126c06e2f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1280034000 tx=0x7f1280033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.744+0000 7f127affd640 1 -- 172.21.15.113:0/2444258828 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1284058c90 con 0x7f128c0b5ca0 2024-04-25T00:07:21.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:21.953+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f128c0ce740 con 0x7f128c0b5ca0 2024-04-25T00:07:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.015+0000 7f127affd640 1 -- 172.21.15.113:0/2444258828 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) ==== 118+0+0 (secure 0 0 0) 0x7f1284058590 con 0x7f128c0b5ca0 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f126c06be30 msgr2=0x7f126c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f126c06be30 0x7f126c06e2f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1280034000 tx=0x7f1280033040 comp rx=0 tx=0).stop 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 msgr2=0x7f128c0c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0c8110 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f128400c430 tx=0x7f128400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 shutdown_connections 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f126c06be30 0x7f126c06e2f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0b73c0 0x7f128c160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c15e510 0x7f128c161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 --2- 172.21.15.113:0/2444258828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f128c0b5ca0 0x7f128c0c8110 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 >> 172.21.15.113:0/2444258828 conn(0x7f128c06b720 msgr2=0x7f128c15d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 shutdown_connections 2024-04-25T00:07:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.017+0000 7f1290b96640 1 -- 172.21.15.113:0/2444258828 wait complete. 2024-04-25T00:07:22.028 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:07:22.029 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:07:22.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.129+0000 7f60008a5640 1 Processor -- start 2024-04-25T00:07:22.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.129+0000 7f60008a5640 1 -- start start 2024-04-25T00:07:22.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f60008a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ffc15e2d0 0x7f5ffc15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:22.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f60008a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ffc15f4f0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f60008a5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ffc15f7d0 con 0x7f5ffc0b3080 2024-04-25T00:07:22.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f60008a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5ffc0cde00 con 0x7f5ffc15e2d0 2024-04-25T00:07:22.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f5ffa59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ffc15e2d0 0x7f5ffc15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:22.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f5ffad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5ffc0b4a40 0x7f5ffc0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56554/0 (socket says 172.21.15.113:56554) 2024-04-25T00:07:22.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f5ffad9d640 1 -- 172.21.15.113:0/2685109434 learned_addr learned my addr 172.21.15.113:0/2685109434 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:22.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f5ffa59c640 1 -- 172.21.15.113:0/2685109434 >> v1:172.21.15.178:6789/0 conn(0x7f5ffc0b3080 legacy=0x7f5ffc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:22.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f5ffa59c640 1 -- 172.21.15.113:0/2685109434 >> v1:172.21.15.113:6789/0 conn(0x7f5ffc0b4a40 legacy=0x7f5ffc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f5ffa59c640 1 -- 172.21.15.113:0/2685109434 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc15fab0 con 0x7f5ffc15e2d0 2024-04-25T00:07:22.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.130+0000 7f5ff9d9b640 1 -- 172.21.15.113:0/2685109434 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870166924 0 0) 0x7f5ffc15f4f0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f5ffa59c640 1 --2- 172.21.15.113:0/2685109434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc15e2d0 0x7f5ffc15ebf0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0cdf80 tx=0x7f5fec02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aecd8f8fb82dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:22.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f5ff9d9b640 1 -- 172.21.15.113:0/2685109434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec037070 con 0x7f5ffc15e2d0 2024-04-25T00:07:22.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f5ff9d9b640 1 -- 172.21.15.113:0/2685109434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fec032440 con 0x7f5ffc15e2d0 2024-04-25T00:07:22.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f5ff9d9b640 1 -- 172.21.15.113:0/2685109434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec032810 con 0x7f5ffc15e2d0 2024-04-25T00:07:22.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f60008a5640 1 -- 172.21.15.113:0/2685109434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc15e2d0 msgr2=0x7f5ffc15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f60008a5640 1 --2- 172.21.15.113:0/2685109434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc15e2d0 0x7f5ffc15ebf0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0cdf80 tx=0x7f5fec02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f60008a5640 1 -- 172.21.15.113:0/2685109434 shutdown_connections 2024-04-25T00:07:22.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f60008a5640 1 --2- 172.21.15.113:0/2685109434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc15e2d0 0x7f5ffc15ebf0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f60008a5640 1 -- 172.21.15.113:0/2685109434 >> 172.21.15.113:0/2685109434 conn(0x7f5ffc06b0f0 msgr2=0x7f5ffc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:22.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.131+0000 7f60008a5640 1 -- 172.21.15.113:0/2685109434 shutdown_connections 2024-04-25T00:07:22.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.132+0000 7f60008a5640 1 -- 172.21.15.113:0/2685109434 wait complete. 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 Processor -- start 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 -- start start 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc0b3080 0x7f5ffc15b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc0b4a40 0x7f5ffc0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ffc15e2d0 0x7f5ffc0a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc15f2b0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc15f130 con 0x7f5ffc0b3080 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.133+0000 7f60008a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc15e6b0 con 0x7f5ffc15e2d0 2024-04-25T00:07:22.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc0b4a40 0x7f5ffc0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:22.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc0b4a40 0x7f5ffc0a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37352/0 (socket says 172.21.15.113:37352) 2024-04-25T00:07:22.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 -- 172.21.15.113:0/3193552477 learned_addr learned my addr 172.21.15.113:0/3193552477 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:22.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ffa59c640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc0b3080 0x7f5ffc15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ffad9d640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ffc15e2d0 0x7f5ffc0a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 -- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ffc15e2d0 msgr2=0x7f5ffc0a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ffc15e2d0 0x7f5ffc0a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 -- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc0b3080 msgr2=0x7f5ffc15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc0b3080 0x7f5ffc15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 -- 172.21.15.113:0/3193552477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc15b800 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff3fff640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc0b4a40 0x7f5ffc0a9480 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c410 tx=0x7f5fe400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff2ffd640 1 -- 172.21.15.113:0/3193552477 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4017070 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff2ffd640 1 -- 172.21.15.113:0/3193552477 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe40035f0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f5ff2ffd640 1 -- 172.21.15.113:0/3193552477 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4003a20 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc15bae0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.134+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc15c0a0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.135+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc0691a0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.136+0000 7f5ff2ffd640 1 -- 172.21.15.113:0/3193552477 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5fe4003bc0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.136+0000 7f5ff2ffd640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5fdc06be60 0x7f5fdc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:22.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.136+0000 7f5ff2ffd640 1 -- 172.21.15.113:0/3193552477 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f5fe408a0a0 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.136+0000 7f5ffa59c640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5fdc06be60 0x7f5fdc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:22.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.136+0000 7f5ffa59c640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5fdc06be60 0x7f5fdc06e320 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:22.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.140+0000 7f5ff2ffd640 1 -- 172.21.15.113:0/3193552477 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5fe4058400 con 0x7f5ffc0b4a40 2024-04-25T00:07:22.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:22.347+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5ffc072050 con 0x7f5ffc0b4a40 2024-04-25T00:07:23.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.020+0000 7f5ff2ffd640 1 -- 172.21.15.113:0/3193552477 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) ==== 122+0+0 (secure 0 0 0) 0x7f5fe405c700 con 0x7f5ffc0b4a40 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5fdc06be60 msgr2=0x7f5fdc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5fdc06be60 0x7f5fdc06e320 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec02e000 comp rx=0 tx=0).stop 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc0b4a40 msgr2=0x7f5ffc0a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc0b4a40 0x7f5ffc0a9480 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c410 tx=0x7f5fe400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 shutdown_connections 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5fdc06be60 0x7f5fdc06e320 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ffc15e2d0 0x7f5ffc0a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ffc0b4a40 0x7f5ffc0a9480 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 --2- 172.21.15.113:0/3193552477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc0b3080 0x7f5ffc15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 >> 172.21.15.113:0/3193552477 conn(0x7f5ffc06b0f0 msgr2=0x7f5ffc0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 shutdown_connections 2024-04-25T00:07:23.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.022+0000 7f60008a5640 1 -- 172.21.15.113:0/3193552477 wait complete. 2024-04-25T00:07:23.035 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:07:23.036 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.134+0000 7fc21ca9e640 1 Processor -- start 2024-04-25T00:07:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.134+0000 7fc21ca9e640 1 -- start start 2024-04-25T00:07:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.135+0000 7fc21ca9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc21815e240 0x7fc2181627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.135+0000 7fc21ca9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc218162d20 con 0x7fc2180b5620 2024-04-25T00:07:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.135+0000 7fc21ca9e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2180b3c20 con 0x7fc21806e6e0 2024-04-25T00:07:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.135+0000 7fc21ca9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc218072d90 con 0x7fc21815e240 2024-04-25T00:07:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.136+0000 7fc21659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc21815e240 0x7fc2181627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.136+0000 7fc215d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc2180b5620 0x7fc21806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56562/0 (socket says 172.21.15.113:56562) 2024-04-25T00:07:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.136+0000 7fc215d9b640 1 -- 172.21.15.113:0/2226986645 learned_addr learned my addr 172.21.15.113:0/2226986645 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.136+0000 7fc21659c640 1 -- 172.21.15.113:0/2226986645 >> v1:172.21.15.178:6789/0 conn(0x7fc21806e6e0 legacy=0x7fc2180b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.136+0000 7fc21659c640 1 -- 172.21.15.113:0/2226986645 >> v1:172.21.15.113:6789/0 conn(0x7fc2180b5620 legacy=0x7fc21806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.136+0000 7fc21659c640 1 -- 172.21.15.113:0/2226986645 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21815e620 con 0x7fc21815e240 2024-04-25T00:07:23.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.136+0000 7fc21659c640 1 --2- 172.21.15.113:0/2226986645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21815e240 0x7fc2181627e0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0097a0 tx=0x7fc20c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58be8c40a8db2f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21559a640 1 -- 172.21.15.113:0/2226986645 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c037070 con 0x7fc21815e240 2024-04-25T00:07:23.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21559a640 1 -- 172.21.15.113:0/2226986645 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc20c032440 con 0x7fc21815e240 2024-04-25T00:07:23.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21559a640 1 -- 172.21.15.113:0/2226986645 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c032810 con 0x7fc21815e240 2024-04-25T00:07:23.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21ca9e640 1 -- 172.21.15.113:0/2226986645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21815e240 msgr2=0x7fc2181627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/2226986645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21815e240 0x7fc2181627e0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0097a0 tx=0x7fc20c02a1e0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21ca9e640 1 -- 172.21.15.113:0/2226986645 shutdown_connections 2024-04-25T00:07:23.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/2226986645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21815e240 0x7fc2181627e0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21ca9e640 1 -- 172.21.15.113:0/2226986645 >> 172.21.15.113:0/2226986645 conn(0x7fc2180b1a50 msgr2=0x7fc2180b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21ca9e640 1 -- 172.21.15.113:0/2226986645 shutdown_connections 2024-04-25T00:07:23.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.137+0000 7fc21ca9e640 1 -- 172.21.15.113:0/2226986645 wait complete. 2024-04-25T00:07:23.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.138+0000 7fc21ca9e640 1 Processor -- start 2024-04-25T00:07:23.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc21ca9e640 1 -- start start 2024-04-25T00:07:23.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc21ca9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21806e6e0 0x7fc218150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc21ca9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2180b5620 0x7fc218155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc21ca9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21815e240 0x7fc218160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc21ca9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2180727d0 con 0x7fc2180b5620 2024-04-25T00:07:23.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc21ca9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2180724f0 con 0x7fc21815e240 2024-04-25T00:07:23.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc21ca9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc218072ab0 con 0x7fc21806e6e0 2024-04-25T00:07:23.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc215d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2180b5620 0x7fc218155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc215d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2180b5620 0x7fc218155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37378/0 (socket says 172.21.15.113:37378) 2024-04-25T00:07:23.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.139+0000 7fc215d9b640 1 -- 172.21.15.113:0/231074697 learned_addr learned my addr 172.21.15.113:0/231074697 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:23.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc21659c640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21806e6e0 0x7fc218150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc216d9d640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21815e240 0x7fc218160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc215d9b640 1 -- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21806e6e0 msgr2=0x7fc218150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc215d9b640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21806e6e0 0x7fc218150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc215d9b640 1 -- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21815e240 msgr2=0x7fc218160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc215d9b640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21815e240 0x7fc218160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc215d9b640 1 -- 172.21.15.113:0/231074697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc218151430 con 0x7fc2180b5620 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc21659c640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21806e6e0 0x7fc218150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc216d9d640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21815e240 0x7fc218160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc215d9b640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2180b5620 0x7fc218155f10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c470 tx=0x7fc208001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc206ffd640 1 -- 172.21.15.113:0/231074697 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208017070 con 0x7fc2180b5620 2024-04-25T00:07:23.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc206ffd640 1 -- 172.21.15.113:0/231074697 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc208005ce0 con 0x7fc2180b5620 2024-04-25T00:07:23.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc218151710 con 0x7fc2180b5620 2024-04-25T00:07:23.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc206ffd640 1 -- 172.21.15.113:0/231074697 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080072f0 con 0x7fc2180b5620 2024-04-25T00:07:23.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.140+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc218151cd0 con 0x7fc2180b5620 2024-04-25T00:07:23.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.141+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2180691a0 con 0x7fc2180b5620 2024-04-25T00:07:23.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.141+0000 7fc206ffd640 1 -- 172.21.15.113:0/231074697 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc20801a070 con 0x7fc2180b5620 2024-04-25T00:07:23.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.142+0000 7fc206ffd640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1f806be30 0x7fc1f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.142+0000 7fc206ffd640 1 -- 172.21.15.113:0/231074697 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fc20808a1d0 con 0x7fc2180b5620 2024-04-25T00:07:23.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.142+0000 7fc21659c640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1f806be30 0x7fc1f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.142+0000 7fc21659c640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1f806be30 0x7fc1f806e2f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc20c034000 tx=0x7fc20c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.146+0000 7fc206ffd640 1 -- 172.21.15.113:0/231074697 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc208058c10 con 0x7fc2180b5620 2024-04-25T00:07:23.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.354+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc218161b40 con 0x7fc2180b5620 2024-04-25T00:07:23.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.354+0000 7fc206ffd640 1 -- 172.21.15.113:0/231074697 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5197 (secure 0 0 0) 0x7fc208058510 con 0x7fc2180b5620 2024-04-25T00:07:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:23.356 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1199283489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1199283489},{"type":"v1","addr":"172.21.15.178:6834","nonce":1199283489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4259795043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4259795043},{"type":"v1","addr":"172.21.15.178:6835","nonce":4259795043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4146002756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4146002756},{"type":"v1","addr":"172.21.15.113:6835","nonce":4146002756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:07:20.361298+0000","modified":"2024-04-25T00:07:22.348971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2111641286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2111641286},{"type":"v1","addr":"172.21.15.113:6837","nonce":2111641286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":10}]} 2024-04-25T00:07:23.356 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-04-25T00:07:23.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1f806be30 msgr2=0x7fc1f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1f806be30 0x7fc1f806e2f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc20c034000 tx=0x7fc20c02e000 comp rx=0 tx=0).stop 2024-04-25T00:07:23.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2180b5620 msgr2=0x7fc218155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2180b5620 0x7fc218155f10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c470 tx=0x7fc208001e70 comp rx=0 tx=0).stop 2024-04-25T00:07:23.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 shutdown_connections 2024-04-25T00:07:23.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc1f806be30 0x7fc1f806e2f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21815e240 0x7fc218160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2180b5620 0x7fc218155f10 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 --2- 172.21.15.113:0/231074697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21806e6e0 0x7fc218150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 >> 172.21.15.113:0/231074697 conn(0x7fc2180b1a50 msgr2=0x7fc2180afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 shutdown_connections 2024-04-25T00:07:23.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.356+0000 7fc21ca9e640 1 -- 172.21.15.113:0/231074697 wait complete. 2024-04-25T00:07:23.364 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:07:23.367 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:07:23.369 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:23.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.504+0000 7f68d71e7640 1 Processor -- start 2024-04-25T00:07:23.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.504+0000 7f68d71e7640 1 -- start start 2024-04-25T00:07:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68d71e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d015e320 0x7f68d00d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68d71e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68d00d2290 0x7f68d00b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68d71e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d00d3de0 con 0x7f68d00d0b00 2024-04-25T00:07:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68d71e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f68d00d3410 con 0x7f68d00d2290 2024-04-25T00:07:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68d71e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f68d00d2800 con 0x7f68d015e320 2024-04-25T00:07:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68cffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68d00d2290 0x7f68d00b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68d5784640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f68d00d0b00 0x7f68d00b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56574/0 (socket says 172.21.15.113:56574) 2024-04-25T00:07:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.505+0000 7f68d5784640 1 -- 172.21.15.113:0/2589710599 learned_addr learned my addr 172.21.15.113:0/2589710599 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.506+0000 7f68d4f83640 1 --2- 172.21.15.113:0/2589710599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d015e320 0x7f68d00d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.506+0000 7f68cffff640 1 -- 172.21.15.113:0/2589710599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d015e320 msgr2=0x7f68d00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.506+0000 7f68cffff640 1 --2- 172.21.15.113:0/2589710599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d015e320 0x7f68d00d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.506+0000 7f68cffff640 1 -- 172.21.15.113:0/2589710599 >> v1:172.21.15.113:6789/0 conn(0x7f68d00d0b00 legacy=0x7f68d00b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.506+0000 7f68cffff640 1 -- 172.21.15.113:0/2589710599 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d00d2ae0 con 0x7f68d00d2290 2024-04-25T00:07:23.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.506+0000 7f68cffff640 1 --2- 172.21.15.113:0/2589710599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00d2290 0x7f68d00b9aa0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097a0 tx=0x7f68bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455582cf4ca90a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.506+0000 7f68cf7fe640 1 -- 172.21.15.113:0/2589710599 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc037070 con 0x7f68d00d2290 2024-04-25T00:07:23.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68cf7fe640 1 -- 172.21.15.113:0/2589710599 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc032440 con 0x7f68d00d2290 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68cf7fe640 1 -- 172.21.15.113:0/2589710599 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc032810 con 0x7f68d00d2290 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 -- 172.21.15.113:0/2589710599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00d2290 msgr2=0x7f68d00b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2589710599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00d2290 0x7f68d00b9aa0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097a0 tx=0x7f68bc02a210 comp rx=0 tx=0).stop 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 -- 172.21.15.113:0/2589710599 shutdown_connections 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2589710599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00d2290 0x7f68d00b9aa0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2589710599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d015e320 0x7f68d00d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 -- 172.21.15.113:0/2589710599 >> 172.21.15.113:0/2589710599 conn(0x7f68d00b5c20 msgr2=0x7f68d00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 -- 172.21.15.113:0/2589710599 shutdown_connections 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.507+0000 7f68d71e7640 1 -- 172.21.15.113:0/2589710599 wait complete. 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.508+0000 7f68d71e7640 1 Processor -- start 2024-04-25T00:07:23.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.508+0000 7f68d71e7640 1 -- start start 2024-04-25T00:07:23.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d71e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d00d0b00 0x7f68d00c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d71e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68d00d2290 0x7f68d00ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d71e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d015e320 0x7f68d00cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d71e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f68d00b9fe0 con 0x7f68d015e320 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d71e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68d00d30a0 con 0x7f68d00d2290 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d71e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f68d015e700 con 0x7f68d00d0b00 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d015e320 0x7f68d00cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d015e320 0x7f68d00cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37400/0 (socket says 172.21.15.113:37400) 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 -- 172.21.15.113:0/2705937288 learned_addr learned my addr 172.21.15.113:0/2705937288 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d4f83640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d00d0b00 0x7f68d00c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68cffff640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68d00d2290 0x7f68d00ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 -- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d00d0b00 msgr2=0x7f68d00c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d00d0b00 0x7f68d00c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 -- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68d00d2290 msgr2=0x7f68d00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68d00d2290 0x7f68d00ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d5784640 1 -- 172.21.15.113:0/2705937288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d00cd8e0 con 0x7f68d015e320 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.509+0000 7f68d4f83640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d00d0b00 0x7f68d00c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.510+0000 7f68d5784640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d015e320 0x7f68d00cd180 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f68c800caa0 tx=0x7f68c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.510+0000 7f68cd7fa640 1 -- 172.21.15.113:0/2705937288 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8017070 con 0x7f68d015e320 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.510+0000 7f68cd7fa640 1 -- 172.21.15.113:0/2705937288 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c800ac00 con 0x7f68d015e320 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.510+0000 7f68cd7fa640 1 -- 172.21.15.113:0/2705937288 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8012620 con 0x7f68d015e320 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.510+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d015f160 con 0x7f68d015e320 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.510+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d015f720 con 0x7f68d015e320 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.511+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d00691a0 con 0x7f68d015e320 2024-04-25T00:07:23.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.511+0000 7f68cd7fa640 1 -- 172.21.15.113:0/2705937288 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68c801a070 con 0x7f68d015e320 2024-04-25T00:07:23.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.512+0000 7f68cd7fa640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68b406be60 0x7f68b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.512+0000 7f68cd7fa640 1 -- 172.21.15.113:0/2705937288 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f68c808ab50 con 0x7f68d015e320 2024-04-25T00:07:23.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.512+0000 7f68d4f83640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68b406be60 0x7f68b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.512+0000 7f68d4f83640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68b406be60 0x7f68b406e320 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f68c4006fd0 tx=0x7f68c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.516+0000 7f68cd7fa640 1 -- 172.21.15.113:0/2705937288 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f68c8059500 con 0x7f68d015e320 2024-04-25T00:07:23.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.691+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68d00cdbc0 con 0x7f68d015e320 2024-04-25T00:07:23.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.692+0000 7f68cd7fa640 1 -- 172.21.15.113:0/2705937288 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16727 (secure 0 0 0) 0x7f68c8058e00 con 0x7f68d015e320 2024-04-25T00:07:23.692 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:23.692 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:21.999089+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:07:18.614948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:07:19.319166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:23.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68b406be60 msgr2=0x7f68b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68b406be60 0x7f68b406e320 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f68c4006fd0 tx=0x7f68c4008040 comp rx=0 tx=0).stop 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d015e320 msgr2=0x7f68d00cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d015e320 0x7f68d00cd180 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f68c800caa0 tx=0x7f68c800cf60 comp rx=0 tx=0).stop 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 shutdown_connections 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f68b406be60 0x7f68b406e320 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68d00d2290 0x7f68d00ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d015e320 0x7f68d00cd180 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 --2- 172.21.15.113:0/2705937288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d00d0b00 0x7f68d00c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 >> 172.21.15.113:0/2705937288 conn(0x7f68d00b5c20 msgr2=0x7f68d00b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 shutdown_connections 2024-04-25T00:07:23.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.694+0000 7f68d71e7640 1 -- 172.21.15.113:0/2705937288 wait complete. 2024-04-25T00:07:23.705 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:07:23.706 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:07:23.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.801+0000 7ff807e9e640 1 Processor -- start 2024-04-25T00:07:23.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.801+0000 7ff807e9e640 1 -- start start 2024-04-25T00:07:23.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff807e9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff800144d60 0x7ff8000b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff807e9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 0x7ff80006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff807e9e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8000cfad0 con 0x7ff80006e510 2024-04-25T00:07:23.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff807e9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff800145c40 con 0x7ff800144d60 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff807e9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff80006e940 con 0x7ff8001453c0 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff805c3a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff800144d60 0x7ff8000b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff805c3a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff800144d60 0x7ff8000b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37408/0 (socket says 172.21.15.113:37408) 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.802+0000 7ff805c3a640 1 -- 172.21.15.113:0/1088047349 learned_addr learned my addr 172.21.15.113:0/1088047349 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff805c3a640 1 -- 172.21.15.113:0/1088047349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 msgr2=0x7ff80006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff805c3a640 1 --2- 172.21.15.113:0/1088047349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 0x7ff80006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff805c3a640 1 -- 172.21.15.113:0/1088047349 >> v1:172.21.15.178:6789/0 conn(0x7ff80006e510 legacy=0x7ff80015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff805c3a640 1 -- 172.21.15.113:0/1088047349 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8000cf230 con 0x7ff800144d60 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff805c3a640 1 --2- 172.21.15.113:0/1088047349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff800144d60 0x7ff8000b5390 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009e50 tx=0x7ff7fc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0b148e5cbc3d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff804c38640 1 -- 172.21.15.113:0/1088047349 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc036070 con 0x7ff800144d60 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff804c38640 1 -- 172.21.15.113:0/1088047349 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7fc031600 con 0x7ff800144d60 2024-04-25T00:07:23.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.803+0000 7ff804c38640 1 -- 172.21.15.113:0/1088047349 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0319d0 con 0x7ff800144d60 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 -- 172.21.15.113:0/1088047349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff800144d60 msgr2=0x7ff8000b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 --2- 172.21.15.113:0/1088047349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff800144d60 0x7ff8000b5390 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009e50 tx=0x7ff7fc02a630 comp rx=0 tx=0).stop 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 -- 172.21.15.113:0/1088047349 shutdown_connections 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 --2- 172.21.15.113:0/1088047349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 0x7ff80006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 --2- 172.21.15.113:0/1088047349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff800144d60 0x7ff8000b5390 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 -- 172.21.15.113:0/1088047349 >> 172.21.15.113:0/1088047349 conn(0x7ff8000b1a50 msgr2=0x7ff8000b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 -- 172.21.15.113:0/1088047349 shutdown_connections 2024-04-25T00:07:23.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.804+0000 7ff807e9e640 1 -- 172.21.15.113:0/1088047349 wait complete. 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.805+0000 7ff807e9e640 1 Processor -- start 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.805+0000 7ff807e9e640 1 -- start start 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff807e9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff80006e510 0x7ff80015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff807e9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800144d60 0x7ff8001535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff807e9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 0x7ff8000ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff807e9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff800070e10 con 0x7ff80006e510 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff807e9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8000cf7f0 con 0x7ff800144d60 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff807e9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff800145140 con 0x7ff8001453c0 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff80006e510 0x7ff80015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff80006e510 0x7ff80015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37420/0 (socket says 172.21.15.113:37420) 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 -- 172.21.15.113:0/2514627827 learned_addr learned my addr 172.21.15.113:0/2514627827 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:23.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805439640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800144d60 0x7ff8001535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 -- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 msgr2=0x7ff8000ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff80643b640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 0x7ff8000ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 0x7ff8000ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 -- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800144d60 msgr2=0x7ff8001535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800144d60 0x7ff8001535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 -- 172.21.15.113:0/2514627827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8000b2e50 con 0x7ff80006e510 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805439640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800144d60 0x7ff8001535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff805c3a640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff80006e510 0x7ff80015d6d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009e50 tx=0x7ff7fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.806+0000 7ff7f67fc640 1 -- 172.21.15.113:0/2514627827 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc041020 con 0x7ff80006e510 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.807+0000 7ff7f67fc640 1 -- 172.21.15.113:0/2514627827 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7fc007800 con 0x7ff80006e510 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.807+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8000b30f0 con 0x7ff80006e510 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.807+0000 7ff7f67fc640 1 -- 172.21.15.113:0/2514627827 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc036040 con 0x7ff80006e510 2024-04-25T00:07:23.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.807+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8000b3990 con 0x7ff80006e510 2024-04-25T00:07:23.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.807+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8000691a0 con 0x7ff80006e510 2024-04-25T00:07:23.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.808+0000 7ff7f67fc640 1 -- 172.21.15.113:0/2514627827 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7fc044020 con 0x7ff80006e510 2024-04-25T00:07:23.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.808+0000 7ff7f67fc640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff7e406be30 0x7ff7e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.808+0000 7ff7f67fc640 1 -- 172.21.15.113:0/2514627827 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7ff7fc0abd10 con 0x7ff80006e510 2024-04-25T00:07:23.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.808+0000 7ff805439640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff7e406be30 0x7ff7e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.809+0000 7ff805439640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff7e406be30 0x7ff7e406e2f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0006fd0 tx=0x7ff7f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.812+0000 7ff7f67fc640 1 -- 172.21.15.113:0/2514627827 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff7fc04b030 con 0x7ff80006e510 2024-04-25T00:07:23.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.985+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8000b27e0 con 0x7ff7e406be30 2024-04-25T00:07:23.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.992+0000 7ff7f67fc640 1 -- 172.21.15.113:0/2514627827 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252416 (secure 0 0 0) 0x7ff8000b27e0 con 0x7ff7e406be30 2024-04-25T00:07:23.996 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:23.997 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:07:23.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.994+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff7e406be30 msgr2=0x7ff7e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.994+0000 7ff807e9e640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff7e406be30 0x7ff7e406e2f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0006fd0 tx=0x7ff7f0008040 comp rx=0 tx=0).stop 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.994+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff80006e510 msgr2=0x7ff80015d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.994+0000 7ff807e9e640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff80006e510 0x7ff80015d6d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009e50 tx=0x7ff7fc004a40 comp rx=0 tx=0).stop 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 shutdown_connections 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff7e406be30 0x7ff7e406e2f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800144d60 0x7ff8001535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8001453c0 0x7ff8000ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 --2- 172.21.15.113:0/2514627827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff80006e510 0x7ff80015d6d0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 >> 172.21.15.113:0/2514627827 conn(0x7ff8000b1a50 msgr2=0x7ff800143ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 shutdown_connections 2024-04-25T00:07:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:23.995+0000 7ff807e9e640 1 -- 172.21.15.113:0/2514627827 wait complete. 2024-04-25T00:07:24.006 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-04-25T00:07:23.218010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":400616,"kb_used_data":15084,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754574104,"statfs":{"total":773094113280,"available":772683882496,"internally_reserved":0,"allocated":15446016,"data_stored":7398016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.227155"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018749+0000","last_change":"2024-04-25T00:07:21.013908+0000","last_active":"2024-04-25T00:07:21.018749+0000","last_peered":"2024-04-25T00:07:21.018749+0000","last_clean":"2024-04-25T00:07:21.018749+0000","last_became_active":"2024-04-25T00:07:21.013662+0000","last_became_peered":"2024-04-25T00:07:21.013662+0000","last_unstale":"2024-04-25T00:07:21.018749+0000","last_undegraded":"2024-04-25T00:07:21.018749+0000","last_fullsized":"2024-04-25T00:07:21.018749+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:04.159990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019861+0000","last_change":"2024-04-25T00:07:21.018411+0000","last_active":"2024-04-25T00:07:21.019861+0000","last_peered":"2024-04-25T00:07:21.019861+0000","last_clean":"2024-04-25T00:07:21.019861+0000","last_became_active":"2024-04-25T00:07:21.018145+0000","last_became_peered":"2024-04-25T00:07:21.018145+0000","last_unstale":"2024-04-25T00:07:21.019861+0000","last_undegraded":"2024-04-25T00:07:21.019861+0000","last_fullsized":"2024-04-25T00:07:21.019861+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:13.319914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023014+0000","last_change":"2024-04-25T00:07:21.021747+0000","last_active":"2024-04-25T00:07:21.023014+0000","last_peered":"2024-04-25T00:07:21.023014+0000","last_clean":"2024-04-25T00:07:21.023014+0000","last_became_active":"2024-04-25T00:07:21.021477+0000","last_became_peered":"2024-04-25T00:07:21.021477+0000","last_unstale":"2024-04-25T00:07:21.023014+0000","last_undegraded":"2024-04-25T00:07:21.023014+0000","last_fullsized":"2024-04-25T00:07:21.023014+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:48.928608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022318+0000","last_change":"2024-04-25T00:07:21.016871+0000","last_active":"2024-04-25T00:07:21.022318+0000","last_peered":"2024-04-25T00:07:21.022318+0000","last_clean":"2024-04-25T00:07:21.022318+0000","last_became_active":"2024-04-25T00:07:21.016589+0000","last_became_peered":"2024-04-25T00:07:21.016589+0000","last_unstale":"2024-04-25T00:07:21.022318+0000","last_undegraded":"2024-04-25T00:07:21.022318+0000","last_fullsized":"2024-04-25T00:07:21.022318+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:56.385071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017043+0000","last_change":"2024-04-25T00:07:21.016026+0000","last_active":"2024-04-25T00:07:21.017043+0000","last_peered":"2024-04-25T00:07:21.017043+0000","last_clean":"2024-04-25T00:07:21.017043+0000","last_became_active":"2024-04-25T00:07:21.015637+0000","last_became_peered":"2024-04-25T00:07:21.015637+0000","last_unstale":"2024-04-25T00:07:21.017043+0000","last_undegraded":"2024-04-25T00:07:21.017043+0000","last_fullsized":"2024-04-25T00:07:21.017043+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.580359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018984+0000","last_change":"2024-04-25T00:07:21.016168+0000","last_active":"2024-04-25T00:07:21.018984+0000","last_peered":"2024-04-25T00:07:21.018984+0000","last_clean":"2024-04-25T00:07:21.018984+0000","last_became_active":"2024-04-25T00:07:21.015903+0000","last_became_peered":"2024-04-25T00:07:21.015903+0000","last_unstale":"2024-04-25T00:07:21.018984+0000","last_undegraded":"2024-04-25T00:07:21.018984+0000","last_fullsized":"2024-04-25T00:07:21.018984+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:51.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009603+0000","last_change":"2024-04-25T00:07:21.007316+0000","last_active":"2024-04-25T00:07:21.009603+0000","last_peered":"2024-04-25T00:07:21.009603+0000","last_clean":"2024-04-25T00:07:21.009603+0000","last_became_active":"2024-04-25T00:07:21.006957+0000","last_became_peered":"2024-04-25T00:07:21.006957+0000","last_unstale":"2024-04-25T00:07:21.009603+0000","last_undegraded":"2024-04-25T00:07:21.009603+0000","last_fullsized":"2024-04-25T00:07:21.009603+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:47.351172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018964+0000","last_change":"2024-04-25T00:07:21.010995+0000","last_active":"2024-04-25T00:07:21.018964+0000","last_peered":"2024-04-25T00:07:21.018964+0000","last_clean":"2024-04-25T00:07:21.018964+0000","last_became_active":"2024-04-25T00:07:21.010671+0000","last_became_peered":"2024-04-25T00:07:21.010671+0000","last_unstale":"2024-04-25T00:07:21.018964+0000","last_undegraded":"2024-04-25T00:07:21.018964+0000","last_fullsized":"2024-04-25T00:07:21.018964+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:19.177947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017320+0000","last_change":"2024-04-25T00:07:21.011906+0000","last_active":"2024-04-25T00:07:21.017320+0000","last_peered":"2024-04-25T00:07:21.017320+0000","last_clean":"2024-04-25T00:07:21.017320+0000","last_became_active":"2024-04-25T00:07:21.011640+0000","last_became_peered":"2024-04-25T00:07:21.011640+0000","last_unstale":"2024-04-25T00:07:21.017320+0000","last_undegraded":"2024-04-25T00:07:21.017320+0000","last_fullsized":"2024-04-25T00:07:21.017320+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:59.812148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023496+0000","last_change":"2024-04-25T00:07:21.021161+0000","last_active":"2024-04-25T00:07:21.023496+0000","last_peered":"2024-04-25T00:07:21.023496+0000","last_clean":"2024-04-25T00:07:21.023496+0000","last_became_active":"2024-04-25T00:07:21.020808+0000","last_became_peered":"2024-04-25T00:07:21.020808+0000","last_unstale":"2024-04-25T00:07:21.023496+0000","last_undegraded":"2024-04-25T00:07:21.023496+0000","last_fullsized":"2024-04-25T00:07:21.023496+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:28.413225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019226+0000","last_change":"2024-04-25T00:07:21.014643+0000","last_active":"2024-04-25T00:07:21.019226+0000","last_peered":"2024-04-25T00:07:21.019226+0000","last_clean":"2024-04-25T00:07:21.019226+0000","last_became_active":"2024-04-25T00:07:21.014354+0000","last_became_peered":"2024-04-25T00:07:21.014354+0000","last_unstale":"2024-04-25T00:07:21.019226+0000","last_undegraded":"2024-04-25T00:07:21.019226+0000","last_fullsized":"2024-04-25T00:07:21.019226+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:56.220006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.021510+0000","last_change":"2024-04-25T00:07:21.018265+0000","last_active":"2024-04-25T00:07:21.021510+0000","last_peered":"2024-04-25T00:07:21.021510+0000","last_clean":"2024-04-25T00:07:21.021510+0000","last_became_active":"2024-04-25T00:07:21.017988+0000","last_became_peered":"2024-04-25T00:07:21.017988+0000","last_unstale":"2024-04-25T00:07:21.021510+0000","last_undegraded":"2024-04-25T00:07:21.021510+0000","last_fullsized":"2024-04-25T00:07:21.021510+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:46.134975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.016921+0000","last_change":"2024-04-25T00:07:21.014417+0000","last_active":"2024-04-25T00:07:21.016921+0000","last_peered":"2024-04-25T00:07:21.016921+0000","last_clean":"2024-04-25T00:07:21.016921+0000","last_became_active":"2024-04-25T00:07:21.014133+0000","last_became_peered":"2024-04-25T00:07:21.014133+0000","last_unstale":"2024-04-25T00:07:21.016921+0000","last_undegraded":"2024-04-25T00:07:21.016921+0000","last_fullsized":"2024-04-25T00:07:21.016921+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:19.177947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018162+0000","last_change":"2024-04-25T00:07:21.015743+0000","last_active":"2024-04-25T00:07:21.018162+0000","last_peered":"2024-04-25T00:07:21.018162+0000","last_clean":"2024-04-25T00:07:21.018162+0000","last_became_active":"2024-04-25T00:07:21.015213+0000","last_became_peered":"2024-04-25T00:07:21.015213+0000","last_unstale":"2024-04-25T00:07:21.018162+0000","last_undegraded":"2024-04-25T00:07:21.018162+0000","last_fullsized":"2024-04-25T00:07:21.018162+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:32.972002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027693+0000","last_change":"2024-04-25T00:07:21.026844+0000","last_active":"2024-04-25T00:07:21.027693+0000","last_peered":"2024-04-25T00:07:21.027693+0000","last_clean":"2024-04-25T00:07:21.027693+0000","last_became_active":"2024-04-25T00:07:21.026586+0000","last_became_peered":"2024-04-25T00:07:21.026586+0000","last_unstale":"2024-04-25T00:07:21.027693+0000","last_undegraded":"2024-04-25T00:07:21.027693+0000","last_fullsized":"2024-04-25T00:07:21.027693+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:57.640988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025753+0000","last_change":"2024-04-25T00:07:21.021966+0000","last_active":"2024-04-25T00:07:21.025753+0000","last_peered":"2024-04-25T00:07:21.025753+0000","last_clean":"2024-04-25T00:07:21.025753+0000","last_became_active":"2024-04-25T00:07:21.021707+0000","last_became_peered":"2024-04-25T00:07:21.021707+0000","last_unstale":"2024-04-25T00:07:21.025753+0000","last_undegraded":"2024-04-25T00:07:21.025753+0000","last_fullsized":"2024-04-25T00:07:21.025753+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:03.494994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.024951+0000","last_change":"2024-04-25T00:07:21.023245+0000","last_active":"2024-04-25T00:07:21.024951+0000","last_peered":"2024-04-25T00:07:21.024951+0000","last_clean":"2024-04-25T00:07:21.024951+0000","last_became_active":"2024-04-25T00:07:21.022943+0000","last_became_peered":"2024-04-25T00:07:21.022943+0000","last_unstale":"2024-04-25T00:07:21.024951+0000","last_undegraded":"2024-04-25T00:07:21.024951+0000","last_fullsized":"2024-04-25T00:07:21.024951+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:04.531859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025136+0000","last_change":"2024-04-25T00:07:21.020564+0000","last_active":"2024-04-25T00:07:21.025136+0000","last_peered":"2024-04-25T00:07:21.025136+0000","last_clean":"2024-04-25T00:07:21.025136+0000","last_became_active":"2024-04-25T00:07:21.020251+0000","last_became_peered":"2024-04-25T00:07:21.020251+0000","last_unstale":"2024-04-25T00:07:21.025136+0000","last_undegraded":"2024-04-25T00:07:21.025136+0000","last_fullsized":"2024-04-25T00:07:21.025136+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:33.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022184+0000","last_change":"2024-04-25T00:07:21.019012+0000","last_active":"2024-04-25T00:07:21.022184+0000","last_peered":"2024-04-25T00:07:21.022184+0000","last_clean":"2024-04-25T00:07:21.022184+0000","last_became_active":"2024-04-25T00:07:21.017011+0000","last_became_peered":"2024-04-25T00:07:21.017011+0000","last_unstale":"2024-04-25T00:07:21.022184+0000","last_undegraded":"2024-04-25T00:07:21.022184+0000","last_fullsized":"2024-04-25T00:07:21.022184+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.257701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025320+0000","last_change":"2024-04-25T00:07:21.014844+0000","last_active":"2024-04-25T00:07:21.025320+0000","last_peered":"2024-04-25T00:07:21.025320+0000","last_clean":"2024-04-25T00:07:21.025320+0000","last_became_active":"2024-04-25T00:07:21.014543+0000","last_became_peered":"2024-04-25T00:07:21.014543+0000","last_unstale":"2024-04-25T00:07:21.025320+0000","last_undegraded":"2024-04-25T00:07:21.025320+0000","last_fullsized":"2024-04-25T00:07:21.025320+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:38.395901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023078+0000","last_change":"2024-04-25T00:07:21.019329+0000","last_active":"2024-04-25T00:07:21.023078+0000","last_peered":"2024-04-25T00:07:21.023078+0000","last_clean":"2024-04-25T00:07:21.023078+0000","last_became_active":"2024-04-25T00:07:21.018779+0000","last_became_peered":"2024-04-25T00:07:21.018779+0000","last_unstale":"2024-04-25T00:07:21.023078+0000","last_undegraded":"2024-04-25T00:07:21.023078+0000","last_fullsized":"2024-04-25T00:07:21.023078+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:35.018370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018145+0000","last_change":"2024-04-25T00:07:21.015490+0000","last_active":"2024-04-25T00:07:21.018145+0000","last_peered":"2024-04-25T00:07:21.018145+0000","last_clean":"2024-04-25T00:07:21.018145+0000","last_became_active":"2024-04-25T00:07:21.013184+0000","last_became_peered":"2024-04-25T00:07:21.013184+0000","last_unstale":"2024-04-25T00:07:21.018145+0000","last_undegraded":"2024-04-25T00:07:21.018145+0000","last_fullsized":"2024-04-25T00:07:21.018145+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:05.543942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017094+0000","last_change":"2024-04-25T00:07:21.010658+0000","last_active":"2024-04-25T00:07:21.017094+0000","last_peered":"2024-04-25T00:07:21.017094+0000","last_clean":"2024-04-25T00:07:21.017094+0000","last_became_active":"2024-04-25T00:07:21.009589+0000","last_became_peered":"2024-04-25T00:07:21.009589+0000","last_unstale":"2024-04-25T00:07:21.017094+0000","last_undegraded":"2024-04-25T00:07:21.017094+0000","last_fullsized":"2024-04-25T00:07:21.017094+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:49.467963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023431+0000","last_change":"2024-04-25T00:07:21.020443+0000","last_active":"2024-04-25T00:07:21.023431+0000","last_peered":"2024-04-25T00:07:21.023431+0000","last_clean":"2024-04-25T00:07:21.023431+0000","last_became_active":"2024-04-25T00:07:21.020158+0000","last_became_peered":"2024-04-25T00:07:21.020158+0000","last_unstale":"2024-04-25T00:07:21.023431+0000","last_undegraded":"2024-04-25T00:07:21.023431+0000","last_fullsized":"2024-04-25T00:07:21.023431+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:47.351172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028534+0000","last_change":"2024-04-25T00:07:21.026566+0000","last_active":"2024-04-25T00:07:21.028534+0000","last_peered":"2024-04-25T00:07:21.028534+0000","last_clean":"2024-04-25T00:07:21.028534+0000","last_became_active":"2024-04-25T00:07:21.024864+0000","last_became_peered":"2024-04-25T00:07:21.024864+0000","last_unstale":"2024-04-25T00:07:21.028534+0000","last_undegraded":"2024-04-25T00:07:21.028534+0000","last_fullsized":"2024-04-25T00:07:21.028534+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:02.990216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029407+0000","last_change":"2024-04-25T00:07:21.026030+0000","last_active":"2024-04-25T00:07:21.029407+0000","last_peered":"2024-04-25T00:07:21.029407+0000","last_clean":"2024-04-25T00:07:21.029407+0000","last_became_active":"2024-04-25T00:07:21.025691+0000","last_became_peered":"2024-04-25T00:07:21.025691+0000","last_unstale":"2024-04-25T00:07:21.029407+0000","last_undegraded":"2024-04-25T00:07:21.029407+0000","last_fullsized":"2024-04-25T00:07:21.029407+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:38.790355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025177+0000","last_change":"2024-04-25T00:07:21.016811+0000","last_active":"2024-04-25T00:07:21.025177+0000","last_peered":"2024-04-25T00:07:21.025177+0000","last_clean":"2024-04-25T00:07:21.025177+0000","last_became_active":"2024-04-25T00:07:21.016564+0000","last_became_peered":"2024-04-25T00:07:21.016564+0000","last_unstale":"2024-04-25T00:07:21.025177+0000","last_undegraded":"2024-04-25T00:07:21.025177+0000","last_fullsized":"2024-04-25T00:07:21.025177+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:03.207212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.013451+0000","last_change":"2024-04-25T00:07:21.012011+0000","last_active":"2024-04-25T00:07:21.013451+0000","last_peered":"2024-04-25T00:07:21.013451+0000","last_clean":"2024-04-25T00:07:21.013451+0000","last_became_active":"2024-04-25T00:07:21.011740+0000","last_became_peered":"2024-04-25T00:07:21.011740+0000","last_unstale":"2024-04-25T00:07:21.013451+0000","last_undegraded":"2024-04-25T00:07:21.013451+0000","last_fullsized":"2024-04-25T00:07:21.013451+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:00.332178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018860+0000","last_change":"2024-04-25T00:07:21.016312+0000","last_active":"2024-04-25T00:07:21.018860+0000","last_peered":"2024-04-25T00:07:21.018860+0000","last_clean":"2024-04-25T00:07:21.018860+0000","last_became_active":"2024-04-25T00:07:21.016050+0000","last_became_peered":"2024-04-25T00:07:21.016050+0000","last_unstale":"2024-04-25T00:07:21.018860+0000","last_undegraded":"2024-04-25T00:07:21.018860+0000","last_fullsized":"2024-04-25T00:07:21.018860+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:26.562395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017249+0000","last_change":"2024-04-25T00:07:21.009397+0000","last_active":"2024-04-25T00:07:21.017249+0000","last_peered":"2024-04-25T00:07:21.017249+0000","last_clean":"2024-04-25T00:07:21.017249+0000","last_became_active":"2024-04-25T00:07:21.008146+0000","last_became_peered":"2024-04-25T00:07:21.008146+0000","last_unstale":"2024-04-25T00:07:21.017249+0000","last_undegraded":"2024-04-25T00:07:21.017249+0000","last_fullsized":"2024-04-25T00:07:21.017249+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.169614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018579+0000","last_change":"2024-04-25T00:07:21.016953+0000","last_active":"2024-04-25T00:07:21.018579+0000","last_peered":"2024-04-25T00:07:21.018579+0000","last_clean":"2024-04-25T00:07:21.018579+0000","last_became_active":"2024-04-25T00:07:21.016681+0000","last_became_peered":"2024-04-25T00:07:21.016681+0000","last_unstale":"2024-04-25T00:07:21.018579+0000","last_undegraded":"2024-04-25T00:07:21.018579+0000","last_fullsized":"2024-04-25T00:07:21.018579+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:31.831532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025235+0000","last_change":"2024-04-25T00:07:21.015129+0000","last_active":"2024-04-25T00:07:21.025235+0000","last_peered":"2024-04-25T00:07:21.025235+0000","last_clean":"2024-04-25T00:07:21.025235+0000","last_became_active":"2024-04-25T00:07:21.014451+0000","last_became_peered":"2024-04-25T00:07:21.014451+0000","last_unstale":"2024-04-25T00:07:21.025235+0000","last_undegraded":"2024-04-25T00:07:21.025235+0000","last_fullsized":"2024-04-25T00:07:21.025235+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:23.897258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022084+0000","last_change":"2024-04-25T00:07:21.018586+0000","last_active":"2024-04-25T00:07:21.022084+0000","last_peered":"2024-04-25T00:07:21.022084+0000","last_clean":"2024-04-25T00:07:21.022084+0000","last_became_active":"2024-04-25T00:07:21.017731+0000","last_became_peered":"2024-04-25T00:07:21.017731+0000","last_unstale":"2024-04-25T00:07:21.022084+0000","last_undegraded":"2024-04-25T00:07:21.022084+0000","last_fullsized":"2024-04-25T00:07:21.022084+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:17.874184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"143'4","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029913+0000","last_change":"2024-04-25T00:07:20.006506+0000","last_active":"2024-04-25T00:07:21.029913+0000","last_peered":"2024-04-25T00:07:21.029913+0000","last_clean":"2024-04-25T00:07:21.029913+0000","last_became_active":"2024-04-25T00:07:20.006227+0000","last_became_peered":"2024-04-25T00:07:20.006227+0000","last_unstale":"2024-04-25T00:07:21.029913+0000","last_undegraded":"2024-04-25T00:07:21.029913+0000","last_fullsized":"2024-04-25T00:07:21.029913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:08.342565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022312+0000","last_change":"2024-04-25T00:07:21.021356+0000","last_active":"2024-04-25T00:07:21.022312+0000","last_peered":"2024-04-25T00:07:21.022312+0000","last_clean":"2024-04-25T00:07:21.022312+0000","last_became_active":"2024-04-25T00:07:21.020933+0000","last_became_peered":"2024-04-25T00:07:21.020933+0000","last_unstale":"2024-04-25T00:07:21.022312+0000","last_undegraded":"2024-04-25T00:07:21.022312+0000","last_fullsized":"2024-04-25T00:07:21.022312+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:32.898564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025975+0000","last_change":"2024-04-25T00:07:20.010470+0000","last_active":"2024-04-25T00:07:21.025975+0000","last_peered":"2024-04-25T00:07:21.025975+0000","last_clean":"2024-04-25T00:07:21.025975+0000","last_became_active":"2024-04-25T00:07:20.010210+0000","last_became_peered":"2024-04-25T00:07:20.010210+0000","last_unstale":"2024-04-25T00:07:21.025975+0000","last_undegraded":"2024-04-25T00:07:21.025975+0000","last_fullsized":"2024-04-25T00:07:21.025975+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:11.281961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006943+0000","last_change":"2024-04-25T00:07:21.005772+0000","last_active":"2024-04-25T00:07:21.006943+0000","last_peered":"2024-04-25T00:07:21.006943+0000","last_clean":"2024-04-25T00:07:21.006943+0000","last_became_active":"2024-04-25T00:07:21.005471+0000","last_became_peered":"2024-04-25T00:07:21.005471+0000","last_unstale":"2024-04-25T00:07:21.006943+0000","last_undegraded":"2024-04-25T00:07:21.006943+0000","last_fullsized":"2024-04-25T00:07:21.006943+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:40.617621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027983+0000","last_change":"2024-04-25T00:07:20.009610+0000","last_active":"2024-04-25T00:07:21.027983+0000","last_peered":"2024-04-25T00:07:21.027983+0000","last_clean":"2024-04-25T00:07:21.027983+0000","last_became_active":"2024-04-25T00:07:20.009272+0000","last_became_peered":"2024-04-25T00:07:20.009272+0000","last_unstale":"2024-04-25T00:07:21.027983+0000","last_undegraded":"2024-04-25T00:07:21.027983+0000","last_fullsized":"2024-04-25T00:07:21.027983+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:10.759574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022308+0000","last_change":"2024-04-25T00:07:21.018114+0000","last_active":"2024-04-25T00:07:21.022308+0000","last_peered":"2024-04-25T00:07:21.022308+0000","last_clean":"2024-04-25T00:07:21.022308+0000","last_became_active":"2024-04-25T00:07:21.017581+0000","last_became_peered":"2024-04-25T00:07:21.017581+0000","last_unstale":"2024-04-25T00:07:21.022308+0000","last_undegraded":"2024-04-25T00:07:21.022308+0000","last_fullsized":"2024-04-25T00:07:21.022308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:24.839751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029898+0000","last_change":"2024-04-25T00:07:20.008716+0000","last_active":"2024-04-25T00:07:21.029898+0000","last_peered":"2024-04-25T00:07:21.029898+0000","last_clean":"2024-04-25T00:07:21.029898+0000","last_became_active":"2024-04-25T00:07:20.008437+0000","last_became_peered":"2024-04-25T00:07:20.008437+0000","last_unstale":"2024-04-25T00:07:21.029898+0000","last_undegraded":"2024-04-25T00:07:21.029898+0000","last_fullsized":"2024-04-25T00:07:21.029898+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:29.053924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018575+0000","last_change":"2024-04-25T00:07:21.016184+0000","last_active":"2024-04-25T00:07:21.018575+0000","last_peered":"2024-04-25T00:07:21.018575+0000","last_clean":"2024-04-25T00:07:21.018575+0000","last_became_active":"2024-04-25T00:07:21.015873+0000","last_became_peered":"2024-04-25T00:07:21.015873+0000","last_unstale":"2024-04-25T00:07:21.018575+0000","last_undegraded":"2024-04-25T00:07:21.018575+0000","last_fullsized":"2024-04-25T00:07:21.018575+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:18.611528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025169+0000","last_change":"2024-04-25T00:07:20.013802+0000","last_active":"2024-04-25T00:07:21.025169+0000","last_peered":"2024-04-25T00:07:21.025169+0000","last_clean":"2024-04-25T00:07:21.025169+0000","last_became_active":"2024-04-25T00:07:20.013446+0000","last_became_peered":"2024-04-25T00:07:20.013446+0000","last_unstale":"2024-04-25T00:07:21.025169+0000","last_undegraded":"2024-04-25T00:07:21.025169+0000","last_fullsized":"2024-04-25T00:07:21.025169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:50.319190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018380+0000","last_change":"2024-04-25T00:07:21.016939+0000","last_active":"2024-04-25T00:07:21.018380+0000","last_peered":"2024-04-25T00:07:21.018380+0000","last_clean":"2024-04-25T00:07:21.018380+0000","last_became_active":"2024-04-25T00:07:21.016629+0000","last_became_peered":"2024-04-25T00:07:21.016629+0000","last_unstale":"2024-04-25T00:07:21.018380+0000","last_undegraded":"2024-04-25T00:07:21.018380+0000","last_fullsized":"2024-04-25T00:07:21.018380+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:13.100124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027754+0000","last_change":"2024-04-25T00:07:20.015639+0000","last_active":"2024-04-25T00:07:21.027754+0000","last_peered":"2024-04-25T00:07:21.027754+0000","last_clean":"2024-04-25T00:07:21.027754+0000","last_became_active":"2024-04-25T00:07:20.014682+0000","last_became_peered":"2024-04-25T00:07:20.014682+0000","last_unstale":"2024-04-25T00:07:21.027754+0000","last_undegraded":"2024-04-25T00:07:21.027754+0000","last_fullsized":"2024-04-25T00:07:21.027754+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:15.524268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019112+0000","last_change":"2024-04-25T00:07:21.015450+0000","last_active":"2024-04-25T00:07:21.019112+0000","last_peered":"2024-04-25T00:07:21.019112+0000","last_clean":"2024-04-25T00:07:21.019112+0000","last_became_active":"2024-04-25T00:07:21.014424+0000","last_became_peered":"2024-04-25T00:07:21.014424+0000","last_unstale":"2024-04-25T00:07:21.019112+0000","last_undegraded":"2024-04-25T00:07:21.019112+0000","last_fullsized":"2024-04-25T00:07:21.019112+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:10.765664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.010839+0000","last_change":"2024-04-25T00:07:20.011465+0000","last_active":"2024-04-25T00:07:21.010839+0000","last_peered":"2024-04-25T00:07:21.010839+0000","last_clean":"2024-04-25T00:07:21.010839+0000","last_became_active":"2024-04-25T00:07:20.011143+0000","last_became_peered":"2024-04-25T00:07:20.011143+0000","last_unstale":"2024-04-25T00:07:21.010839+0000","last_undegraded":"2024-04-25T00:07:21.010839+0000","last_fullsized":"2024-04-25T00:07:21.010839+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:40.073850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018016+0000","last_change":"2024-04-25T00:07:21.016724+0000","last_active":"2024-04-25T00:07:21.018016+0000","last_peered":"2024-04-25T00:07:21.018016+0000","last_clean":"2024-04-25T00:07:21.018016+0000","last_became_active":"2024-04-25T00:07:21.016445+0000","last_became_peered":"2024-04-25T00:07:21.016445+0000","last_unstale":"2024-04-25T00:07:21.018016+0000","last_undegraded":"2024-04-25T00:07:21.018016+0000","last_fullsized":"2024-04-25T00:07:21.018016+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:26.562395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002641+0000","last_change":"2024-04-25T00:07:20.010465+0000","last_active":"2024-04-25T00:07:21.002641+0000","last_peered":"2024-04-25T00:07:21.002641+0000","last_clean":"2024-04-25T00:07:21.002641+0000","last_became_active":"2024-04-25T00:07:20.010199+0000","last_became_peered":"2024-04-25T00:07:20.010199+0000","last_unstale":"2024-04-25T00:07:21.002641+0000","last_undegraded":"2024-04-25T00:07:21.002641+0000","last_fullsized":"2024-04-25T00:07:21.002641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:47.923494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027886+0000","last_change":"2024-04-25T00:07:21.026575+0000","last_active":"2024-04-25T00:07:21.027886+0000","last_peered":"2024-04-25T00:07:21.027886+0000","last_clean":"2024-04-25T00:07:21.027886+0000","last_became_active":"2024-04-25T00:07:21.026304+0000","last_became_peered":"2024-04-25T00:07:21.026304+0000","last_unstale":"2024-04-25T00:07:21.027886+0000","last_undegraded":"2024-04-25T00:07:21.027886+0000","last_fullsized":"2024-04-25T00:07:21.027886+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:12.596577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006941+0000","last_change":"2024-04-25T00:07:20.006837+0000","last_active":"2024-04-25T00:07:21.006941+0000","last_peered":"2024-04-25T00:07:21.006941+0000","last_clean":"2024-04-25T00:07:21.006941+0000","last_became_active":"2024-04-25T00:07:20.006445+0000","last_became_peered":"2024-04-25T00:07:20.006445+0000","last_unstale":"2024-04-25T00:07:21.006941+0000","last_undegraded":"2024-04-25T00:07:21.006941+0000","last_fullsized":"2024-04-25T00:07:21.006941+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:35.871345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.026484+0000","last_change":"2024-04-25T00:07:21.024044+0000","last_active":"2024-04-25T00:07:21.026484+0000","last_peered":"2024-04-25T00:07:21.026484+0000","last_clean":"2024-04-25T00:07:21.026484+0000","last_became_active":"2024-04-25T00:07:21.023739+0000","last_became_peered":"2024-04-25T00:07:21.023739+0000","last_unstale":"2024-04-25T00:07:21.026484+0000","last_undegraded":"2024-04-25T00:07:21.026484+0000","last_fullsized":"2024-04-25T00:07:21.026484+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:00.332178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008004+0000","last_change":"2024-04-25T00:07:20.008802+0000","last_active":"2024-04-25T00:07:21.008004+0000","last_peered":"2024-04-25T00:07:21.008004+0000","last_clean":"2024-04-25T00:07:21.008004+0000","last_became_active":"2024-04-25T00:07:20.008515+0000","last_became_peered":"2024-04-25T00:07:20.008515+0000","last_unstale":"2024-04-25T00:07:21.008004+0000","last_undegraded":"2024-04-25T00:07:21.008004+0000","last_fullsized":"2024-04-25T00:07:21.008004+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:11.620560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025200+0000","last_change":"2024-04-25T00:07:21.017611+0000","last_active":"2024-04-25T00:07:21.025200+0000","last_peered":"2024-04-25T00:07:21.025200+0000","last_clean":"2024-04-25T00:07:21.025200+0000","last_became_active":"2024-04-25T00:07:21.017294+0000","last_became_peered":"2024-04-25T00:07:21.017294+0000","last_unstale":"2024-04-25T00:07:21.025200+0000","last_undegraded":"2024-04-25T00:07:21.025200+0000","last_fullsized":"2024-04-25T00:07:21.025200+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:32.700026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"143'6","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.031158+0000","last_change":"2024-04-25T00:07:20.008498+0000","last_active":"2024-04-25T00:07:21.031158+0000","last_peered":"2024-04-25T00:07:21.031158+0000","last_clean":"2024-04-25T00:07:21.031158+0000","last_became_active":"2024-04-25T00:07:20.008233+0000","last_became_peered":"2024-04-25T00:07:20.008233+0000","last_unstale":"2024-04-25T00:07:21.031158+0000","last_undegraded":"2024-04-25T00:07:21.031158+0000","last_fullsized":"2024-04-25T00:07:21.031158+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:16.869070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022957+0000","last_change":"2024-04-25T00:07:21.020790+0000","last_active":"2024-04-25T00:07:21.022957+0000","last_peered":"2024-04-25T00:07:21.022957+0000","last_clean":"2024-04-25T00:07:21.022957+0000","last_became_active":"2024-04-25T00:07:21.020230+0000","last_became_peered":"2024-04-25T00:07:21.020230+0000","last_unstale":"2024-04-25T00:07:21.022957+0000","last_undegraded":"2024-04-25T00:07:21.022957+0000","last_fullsized":"2024-04-25T00:07:21.022957+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:11.764688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.030276+0000","last_change":"2024-04-25T00:07:20.016657+0000","last_active":"2024-04-25T00:07:21.030276+0000","last_peered":"2024-04-25T00:07:21.030276+0000","last_clean":"2024-04-25T00:07:21.030276+0000","last_became_active":"2024-04-25T00:07:20.016344+0000","last_became_peered":"2024-04-25T00:07:20.016344+0000","last_unstale":"2024-04-25T00:07:21.030276+0000","last_undegraded":"2024-04-25T00:07:21.030276+0000","last_fullsized":"2024-04-25T00:07:21.030276+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:39.426245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017926+0000","last_change":"2024-04-25T00:07:21.015381+0000","last_active":"2024-04-25T00:07:21.017926+0000","last_peered":"2024-04-25T00:07:21.017926+0000","last_clean":"2024-04-25T00:07:21.017926+0000","last_became_active":"2024-04-25T00:07:21.015075+0000","last_became_peered":"2024-04-25T00:07:21.015075+0000","last_unstale":"2024-04-25T00:07:21.017926+0000","last_undegraded":"2024-04-25T00:07:21.017926+0000","last_fullsized":"2024-04-25T00:07:21.017926+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:29.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008163+0000","last_change":"2024-04-25T00:07:20.016875+0000","last_active":"2024-04-25T00:07:21.008163+0000","last_peered":"2024-04-25T00:07:21.008163+0000","last_clean":"2024-04-25T00:07:21.008163+0000","last_became_active":"2024-04-25T00:07:20.016517+0000","last_became_peered":"2024-04-25T00:07:20.016517+0000","last_unstale":"2024-04-25T00:07:21.008163+0000","last_undegraded":"2024-04-25T00:07:21.008163+0000","last_fullsized":"2024-04-25T00:07:21.008163+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:15.013625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017019+0000","last_change":"2024-04-25T00:07:21.015169+0000","last_active":"2024-04-25T00:07:21.017019+0000","last_peered":"2024-04-25T00:07:21.017019+0000","last_clean":"2024-04-25T00:07:21.017019+0000","last_became_active":"2024-04-25T00:07:21.014899+0000","last_became_peered":"2024-04-25T00:07:21.014899+0000","last_unstale":"2024-04-25T00:07:21.017019+0000","last_undegraded":"2024-04-25T00:07:21.017019+0000","last_fullsized":"2024-04-25T00:07:21.017019+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:56.220006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002935+0000","last_change":"2024-04-25T00:07:20.007066+0000","last_active":"2024-04-25T00:07:21.002935+0000","last_peered":"2024-04-25T00:07:21.002935+0000","last_clean":"2024-04-25T00:07:21.002935+0000","last_became_active":"2024-04-25T00:07:20.006786+0000","last_became_peered":"2024-04-25T00:07:20.006786+0000","last_unstale":"2024-04-25T00:07:21.002935+0000","last_undegraded":"2024-04-25T00:07:21.002935+0000","last_fullsized":"2024-04-25T00:07:21.002935+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:26.542022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028474+0000","last_change":"2024-04-25T00:07:21.027428+0000","last_active":"2024-04-25T00:07:21.028474+0000","last_peered":"2024-04-25T00:07:21.028474+0000","last_clean":"2024-04-25T00:07:21.028474+0000","last_became_active":"2024-04-25T00:07:21.027135+0000","last_became_peered":"2024-04-25T00:07:21.027135+0000","last_unstale":"2024-04-25T00:07:21.028474+0000","last_undegraded":"2024-04-25T00:07:21.028474+0000","last_fullsized":"2024-04-25T00:07:21.028474+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.834891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"143'8","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028296+0000","last_change":"2024-04-25T00:07:20.010840+0000","last_active":"2024-04-25T00:07:21.028296+0000","last_peered":"2024-04-25T00:07:21.028296+0000","last_clean":"2024-04-25T00:07:21.028296+0000","last_became_active":"2024-04-25T00:07:20.009809+0000","last_became_peered":"2024-04-25T00:07:20.009809+0000","last_unstale":"2024-04-25T00:07:21.028296+0000","last_undegraded":"2024-04-25T00:07:21.028296+0000","last_fullsized":"2024-04-25T00:07:21.028296+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:11.493715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022790+0000","last_change":"2024-04-25T00:07:21.019741+0000","last_active":"2024-04-25T00:07:21.022790+0000","last_peered":"2024-04-25T00:07:21.022790+0000","last_clean":"2024-04-25T00:07:21.022790+0000","last_became_active":"2024-04-25T00:07:21.019462+0000","last_became_peered":"2024-04-25T00:07:21.019462+0000","last_unstale":"2024-04-25T00:07:21.022790+0000","last_undegraded":"2024-04-25T00:07:21.022790+0000","last_fullsized":"2024-04-25T00:07:21.022790+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:18.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028940+0000","last_change":"2024-04-25T00:07:20.017466+0000","last_active":"2024-04-25T00:07:21.028940+0000","last_peered":"2024-04-25T00:07:21.028940+0000","last_clean":"2024-04-25T00:07:21.028940+0000","last_became_active":"2024-04-25T00:07:20.016941+0000","last_became_peered":"2024-04-25T00:07:20.016941+0000","last_unstale":"2024-04-25T00:07:21.028940+0000","last_undegraded":"2024-04-25T00:07:21.028940+0000","last_fullsized":"2024-04-25T00:07:21.028940+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:39.612507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.026086+0000","last_change":"2024-04-25T00:07:21.016055+0000","last_active":"2024-04-25T00:07:21.026086+0000","last_peered":"2024-04-25T00:07:21.026086+0000","last_clean":"2024-04-25T00:07:21.026086+0000","last_became_active":"2024-04-25T00:07:21.015792+0000","last_became_peered":"2024-04-25T00:07:21.015792+0000","last_unstale":"2024-04-25T00:07:21.026086+0000","last_undegraded":"2024-04-25T00:07:21.026086+0000","last_fullsized":"2024-04-25T00:07:21.026086+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:39.399920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028857+0000","last_change":"2024-04-25T00:07:20.014876+0000","last_active":"2024-04-25T00:07:21.028857+0000","last_peered":"2024-04-25T00:07:21.028857+0000","last_clean":"2024-04-25T00:07:21.028857+0000","last_became_active":"2024-04-25T00:07:20.014594+0000","last_became_peered":"2024-04-25T00:07:20.014594+0000","last_unstale":"2024-04-25T00:07:21.028857+0000","last_undegraded":"2024-04-25T00:07:21.028857+0000","last_fullsized":"2024-04-25T00:07:21.028857+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:12.674290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025815+0000","last_change":"2024-04-25T00:07:21.024281+0000","last_active":"2024-04-25T00:07:21.025815+0000","last_peered":"2024-04-25T00:07:21.025815+0000","last_clean":"2024-04-25T00:07:21.025815+0000","last_became_active":"2024-04-25T00:07:21.023991+0000","last_became_peered":"2024-04-25T00:07:21.023991+0000","last_unstale":"2024-04-25T00:07:21.025815+0000","last_undegraded":"2024-04-25T00:07:21.025815+0000","last_fullsized":"2024-04-25T00:07:21.025815+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.169614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025258+0000","last_change":"2024-04-25T00:07:20.013426+0000","last_active":"2024-04-25T00:07:21.025258+0000","last_peered":"2024-04-25T00:07:21.025258+0000","last_clean":"2024-04-25T00:07:21.025258+0000","last_became_active":"2024-04-25T00:07:20.012844+0000","last_became_peered":"2024-04-25T00:07:20.012844+0000","last_unstale":"2024-04-25T00:07:21.025258+0000","last_undegraded":"2024-04-25T00:07:21.025258+0000","last_fullsized":"2024-04-25T00:07:21.025258+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:15.346208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019263+0000","last_change":"2024-04-25T00:07:21.017328+0000","last_active":"2024-04-25T00:07:21.019263+0000","last_peered":"2024-04-25T00:07:21.019263+0000","last_clean":"2024-04-25T00:07:21.019263+0000","last_became_active":"2024-04-25T00:07:21.016679+0000","last_became_peered":"2024-04-25T00:07:21.016679+0000","last_unstale":"2024-04-25T00:07:21.019263+0000","last_undegraded":"2024-04-25T00:07:21.019263+0000","last_fullsized":"2024-04-25T00:07:21.019263+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:12.596577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006903+0000","last_change":"2024-04-25T00:07:20.011616+0000","last_active":"2024-04-25T00:07:21.006903+0000","last_peered":"2024-04-25T00:07:21.006903+0000","last_clean":"2024-04-25T00:07:21.006903+0000","last_became_active":"2024-04-25T00:07:20.011298+0000","last_became_peered":"2024-04-25T00:07:20.011298+0000","last_unstale":"2024-04-25T00:07:21.006903+0000","last_undegraded":"2024-04-25T00:07:21.006903+0000","last_fullsized":"2024-04-25T00:07:21.006903+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:52.235634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018110+0000","last_change":"2024-04-25T00:07:21.011387+0000","last_active":"2024-04-25T00:07:21.018110+0000","last_peered":"2024-04-25T00:07:21.018110+0000","last_clean":"2024-04-25T00:07:21.018110+0000","last_became_active":"2024-04-25T00:07:21.008657+0000","last_became_peered":"2024-04-25T00:07:21.008657+0000","last_unstale":"2024-04-25T00:07:21.018110+0000","last_undegraded":"2024-04-25T00:07:21.018110+0000","last_fullsized":"2024-04-25T00:07:21.018110+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.385850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029379+0000","last_change":"2024-04-25T00:07:20.015444+0000","last_active":"2024-04-25T00:07:21.029379+0000","last_peered":"2024-04-25T00:07:21.029379+0000","last_clean":"2024-04-25T00:07:21.029379+0000","last_became_active":"2024-04-25T00:07:20.015093+0000","last_became_peered":"2024-04-25T00:07:20.015093+0000","last_unstale":"2024-04-25T00:07:21.029379+0000","last_undegraded":"2024-04-25T00:07:21.029379+0000","last_fullsized":"2024-04-25T00:07:21.029379+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:45.129861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008618+0000","last_change":"2024-04-25T00:07:21.006955+0000","last_active":"2024-04-25T00:07:21.008618+0000","last_peered":"2024-04-25T00:07:21.008618+0000","last_clean":"2024-04-25T00:07:21.008618+0000","last_became_active":"2024-04-25T00:07:21.006632+0000","last_became_peered":"2024-04-25T00:07:21.006632+0000","last_unstale":"2024-04-25T00:07:21.008618+0000","last_undegraded":"2024-04-25T00:07:21.008618+0000","last_fullsized":"2024-04-25T00:07:21.008618+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:18.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009365+0000","last_change":"2024-04-25T00:07:20.005829+0000","last_active":"2024-04-25T00:07:21.009365+0000","last_peered":"2024-04-25T00:07:21.009365+0000","last_clean":"2024-04-25T00:07:21.009365+0000","last_became_active":"2024-04-25T00:07:20.005523+0000","last_became_peered":"2024-04-25T00:07:20.005523+0000","last_unstale":"2024-04-25T00:07:21.009365+0000","last_undegraded":"2024-04-25T00:07:21.009365+0000","last_fullsized":"2024-04-25T00:07:21.009365+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:32.060924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.011166+0000","last_change":"2024-04-25T00:07:21.009919+0000","last_active":"2024-04-25T00:07:21.011166+0000","last_peered":"2024-04-25T00:07:21.011166+0000","last_clean":"2024-04-25T00:07:21.011166+0000","last_became_active":"2024-04-25T00:07:21.009631+0000","last_became_peered":"2024-04-25T00:07:21.009631+0000","last_unstale":"2024-04-25T00:07:21.011166+0000","last_undegraded":"2024-04-25T00:07:21.011166+0000","last_fullsized":"2024-04-25T00:07:21.011166+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.834891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.010175+0000","last_change":"2024-04-25T00:07:20.010722+0000","last_active":"2024-04-25T00:07:21.010175+0000","last_peered":"2024-04-25T00:07:21.010175+0000","last_clean":"2024-04-25T00:07:21.010175+0000","last_became_active":"2024-04-25T00:07:20.010433+0000","last_became_peered":"2024-04-25T00:07:20.010433+0000","last_unstale":"2024-04-25T00:07:21.010175+0000","last_undegraded":"2024-04-25T00:07:21.010175+0000","last_fullsized":"2024-04-25T00:07:21.010175+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:06.801027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009626+0000","last_change":"2024-04-25T00:07:21.007738+0000","last_active":"2024-04-25T00:07:21.009626+0000","last_peered":"2024-04-25T00:07:21.009626+0000","last_clean":"2024-04-25T00:07:21.009626+0000","last_became_active":"2024-04-25T00:07:21.007469+0000","last_became_peered":"2024-04-25T00:07:21.007469+0000","last_unstale":"2024-04-25T00:07:21.009626+0000","last_undegraded":"2024-04-25T00:07:21.009626+0000","last_fullsized":"2024-04-25T00:07:21.009626+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:28.413225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008569+0000","last_change":"2024-04-25T00:07:20.018219+0000","last_active":"2024-04-25T00:07:21.008569+0000","last_peered":"2024-04-25T00:07:21.008569+0000","last_clean":"2024-04-25T00:07:21.008569+0000","last_became_active":"2024-04-25T00:07:20.017909+0000","last_became_peered":"2024-04-25T00:07:20.017909+0000","last_unstale":"2024-04-25T00:07:21.008569+0000","last_undegraded":"2024-04-25T00:07:21.008569+0000","last_fullsized":"2024-04-25T00:07:21.008569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:40.185722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.016289+0000","last_change":"2024-04-25T00:07:21.014996+0000","last_active":"2024-04-25T00:07:21.016289+0000","last_peered":"2024-04-25T00:07:21.016289+0000","last_clean":"2024-04-25T00:07:21.016289+0000","last_became_active":"2024-04-25T00:07:21.014717+0000","last_became_peered":"2024-04-25T00:07:21.014717+0000","last_unstale":"2024-04-25T00:07:21.016289+0000","last_undegraded":"2024-04-25T00:07:21.016289+0000","last_fullsized":"2024-04-25T00:07:21.016289+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:04.159990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028773+0000","last_change":"2024-04-25T00:07:20.016031+0000","last_active":"2024-04-25T00:07:21.028773+0000","last_peered":"2024-04-25T00:07:21.028773+0000","last_clean":"2024-04-25T00:07:21.028773+0000","last_became_active":"2024-04-25T00:07:20.015619+0000","last_became_peered":"2024-04-25T00:07:20.015619+0000","last_unstale":"2024-04-25T00:07:21.028773+0000","last_undegraded":"2024-04-25T00:07:21.028773+0000","last_fullsized":"2024-04-25T00:07:21.028773+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:53.093311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023831+0000","last_change":"2024-04-25T00:07:21.022531+0000","last_active":"2024-04-25T00:07:21.023831+0000","last_peered":"2024-04-25T00:07:21.023831+0000","last_clean":"2024-04-25T00:07:21.023831+0000","last_became_active":"2024-04-25T00:07:21.022208+0000","last_became_peered":"2024-04-25T00:07:21.022208+0000","last_unstale":"2024-04-25T00:07:21.023831+0000","last_undegraded":"2024-04-25T00:07:21.023831+0000","last_fullsized":"2024-04-25T00:07:21.023831+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:30.059038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002877+0000","last_change":"2024-04-25T00:07:20.006003+0000","last_active":"2024-04-25T00:07:21.002877+0000","last_peered":"2024-04-25T00:07:21.002877+0000","last_clean":"2024-04-25T00:07:21.002877+0000","last_became_active":"2024-04-25T00:07:20.005694+0000","last_became_peered":"2024-04-25T00:07:20.005694+0000","last_unstale":"2024-04-25T00:07:21.002877+0000","last_undegraded":"2024-04-25T00:07:21.002877+0000","last_fullsized":"2024-04-25T00:07:21.002877+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:01.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.013748+0000","last_change":"2024-04-25T00:07:21.011207+0000","last_active":"2024-04-25T00:07:21.013748+0000","last_peered":"2024-04-25T00:07:21.013748+0000","last_clean":"2024-04-25T00:07:21.013748+0000","last_became_active":"2024-04-25T00:07:21.010239+0000","last_became_peered":"2024-04-25T00:07:21.010239+0000","last_unstale":"2024-04-25T00:07:21.013748+0000","last_undegraded":"2024-04-25T00:07:21.013748+0000","last_fullsized":"2024-04-25T00:07:21.013748+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:45.498938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.004215+0000","last_change":"2024-04-25T00:07:20.008469+0000","last_active":"2024-04-25T00:07:21.004215+0000","last_peered":"2024-04-25T00:07:21.004215+0000","last_clean":"2024-04-25T00:07:21.004215+0000","last_became_active":"2024-04-25T00:07:20.008184+0000","last_became_peered":"2024-04-25T00:07:20.008184+0000","last_unstale":"2024-04-25T00:07:21.004215+0000","last_undegraded":"2024-04-25T00:07:21.004215+0000","last_fullsized":"2024-04-25T00:07:21.004215+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:15.524268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019248+0000","last_change":"2024-04-25T00:07:21.014927+0000","last_active":"2024-04-25T00:07:21.019248+0000","last_peered":"2024-04-25T00:07:21.019248+0000","last_clean":"2024-04-25T00:07:21.019248+0000","last_became_active":"2024-04-25T00:07:21.011048+0000","last_became_peered":"2024-04-25T00:07:21.011048+0000","last_unstale":"2024-04-25T00:07:21.019248+0000","last_undegraded":"2024-04-25T00:07:21.019248+0000","last_fullsized":"2024-04-25T00:07:21.019248+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.580359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002316+0000","last_change":"2024-04-25T00:07:20.014660+0000","last_active":"2024-04-25T00:07:21.002316+0000","last_peered":"2024-04-25T00:07:21.002316+0000","last_clean":"2024-04-25T00:07:21.002316+0000","last_became_active":"2024-04-25T00:07:20.014394+0000","last_became_peered":"2024-04-25T00:07:20.014394+0000","last_unstale":"2024-04-25T00:07:21.002316+0000","last_undegraded":"2024-04-25T00:07:21.002316+0000","last_fullsized":"2024-04-25T00:07:21.002316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028562+0000","last_change":"2024-04-25T00:07:21.027013+0000","last_active":"2024-04-25T00:07:21.028562+0000","last_peered":"2024-04-25T00:07:21.028562+0000","last_clean":"2024-04-25T00:07:21.028562+0000","last_became_active":"2024-04-25T00:07:21.025129+0000","last_became_peered":"2024-04-25T00:07:21.025129+0000","last_unstale":"2024-04-25T00:07:21.028562+0000","last_undegraded":"2024-04-25T00:07:21.028562+0000","last_fullsized":"2024-04-25T00:07:21.028562+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:27.547137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002572+0000","last_change":"2024-04-25T00:07:20.009590+0000","last_active":"2024-04-25T00:07:21.002572+0000","last_peered":"2024-04-25T00:07:21.002572+0000","last_clean":"2024-04-25T00:07:21.002572+0000","last_became_active":"2024-04-25T00:07:20.009137+0000","last_became_peered":"2024-04-25T00:07:20.009137+0000","last_unstale":"2024-04-25T00:07:21.002572+0000","last_undegraded":"2024-04-25T00:07:21.002572+0000","last_fullsized":"2024-04-25T00:07:21.002572+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:34.013256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018072+0000","last_change":"2024-04-25T00:07:21.016255+0000","last_active":"2024-04-25T00:07:21.018072+0000","last_peered":"2024-04-25T00:07:21.018072+0000","last_clean":"2024-04-25T00:07:21.018072+0000","last_became_active":"2024-04-25T00:07:21.015966+0000","last_became_peered":"2024-04-25T00:07:21.015966+0000","last_unstale":"2024-04-25T00:07:21.018072+0000","last_undegraded":"2024-04-25T00:07:21.018072+0000","last_fullsized":"2024-04-25T00:07:21.018072+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:51.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009514+0000","last_change":"2024-04-25T00:07:20.014387+0000","last_active":"2024-04-25T00:07:21.009514+0000","last_peered":"2024-04-25T00:07:21.009514+0000","last_clean":"2024-04-25T00:07:21.009514+0000","last_became_active":"2024-04-25T00:07:20.014109+0000","last_became_peered":"2024-04-25T00:07:20.014109+0000","last_unstale":"2024-04-25T00:07:21.009514+0000","last_undegraded":"2024-04-25T00:07:21.009514+0000","last_fullsized":"2024-04-25T00:07:21.009514+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:16.192208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027352+0000","last_change":"2024-04-25T00:07:21.026453+0000","last_active":"2024-04-25T00:07:21.027352+0000","last_peered":"2024-04-25T00:07:21.027352+0000","last_clean":"2024-04-25T00:07:21.027352+0000","last_became_active":"2024-04-25T00:07:21.026152+0000","last_became_peered":"2024-04-25T00:07:21.026152+0000","last_unstale":"2024-04-25T00:07:21.027352+0000","last_undegraded":"2024-04-25T00:07:21.027352+0000","last_fullsized":"2024-04-25T00:07:21.027352+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:04.807321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.005959+0000","last_change":"2024-04-25T00:07:20.012330+0000","last_active":"2024-04-25T00:07:21.005959+0000","last_peered":"2024-04-25T00:07:21.005959+0000","last_clean":"2024-04-25T00:07:21.005959+0000","last_became_active":"2024-04-25T00:07:20.011847+0000","last_became_peered":"2024-04-25T00:07:20.011847+0000","last_unstale":"2024-04-25T00:07:21.005959+0000","last_undegraded":"2024-04-25T00:07:21.005959+0000","last_fullsized":"2024-04-25T00:07:21.005959+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.779983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028111+0000","last_change":"2024-04-25T00:07:21.027389+0000","last_active":"2024-04-25T00:07:21.028111+0000","last_peered":"2024-04-25T00:07:21.028111+0000","last_clean":"2024-04-25T00:07:21.028111+0000","last_became_active":"2024-04-25T00:07:21.027049+0000","last_became_peered":"2024-04-25T00:07:21.027049+0000","last_unstale":"2024-04-25T00:07:21.028111+0000","last_undegraded":"2024-04-25T00:07:21.028111+0000","last_fullsized":"2024-04-25T00:07:21.028111+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.257701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006064+0000","last_change":"2024-04-25T00:07:20.012877+0000","last_active":"2024-04-25T00:07:21.006064+0000","last_peered":"2024-04-25T00:07:21.006064+0000","last_clean":"2024-04-25T00:07:21.006064+0000","last_became_active":"2024-04-25T00:07:20.011114+0000","last_became_peered":"2024-04-25T00:07:20.011114+0000","last_unstale":"2024-04-25T00:07:21.006064+0000","last_undegraded":"2024-04-25T00:07:21.006064+0000","last_fullsized":"2024-04-25T00:07:21.006064+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:16.006603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009634+0000","last_change":"2024-04-25T00:07:21.007775+0000","last_active":"2024-04-25T00:07:21.009634+0000","last_peered":"2024-04-25T00:07:21.009634+0000","last_clean":"2024-04-25T00:07:21.009634+0000","last_became_active":"2024-04-25T00:07:21.007478+0000","last_became_peered":"2024-04-25T00:07:21.007478+0000","last_unstale":"2024-04-25T00:07:21.009634+0000","last_undegraded":"2024-04-25T00:07:21.009634+0000","last_fullsized":"2024-04-25T00:07:21.009634+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:04.531859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"143'3","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029480+0000","last_change":"2024-04-25T00:07:20.018022+0000","last_active":"2024-04-25T00:07:21.029480+0000","last_peered":"2024-04-25T00:07:21.029480+0000","last_clean":"2024-04-25T00:07:21.029480+0000","last_became_active":"2024-04-25T00:07:20.017748+0000","last_became_peered":"2024-04-25T00:07:20.017748+0000","last_unstale":"2024-04-25T00:07:21.029480+0000","last_undegraded":"2024-04-25T00:07:21.029480+0000","last_fullsized":"2024-04-25T00:07:21.029480+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:26.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":392,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006696+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:07:21.006696+0000","last_peered":"2024-04-25T00:07:21.006696+0000","last_clean":"2024-04-25T00:07:21.006696+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:07:21.006696+0000","last_undegraded":"2024-04-25T00:07:21.006696+0000","last_fullsized":"2024-04-25T00:07:21.006696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47412,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324428,"statfs":{"total":96636764160,"available":96588214272,"internally_reserved":0,"allocated":2392064,"data_stored":1373107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50964,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320876,"statfs":{"total":96636764160,"available":96584577024,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55144,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316696,"statfs":{"total":96636764160,"available":96580296704,"internally_reserved":0,"allocated":1794048,"data_stored":782669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47384,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324456,"statfs":{"total":96636764160,"available":96588242944,"internally_reserved":0,"allocated":2371584,"data_stored":1365680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:07:24.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:07:24.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.142+0000 7faf5b596640 1 Processor -- start 2024-04-25T00:07:24.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.142+0000 7faf5b596640 1 -- start start 2024-04-25T00:07:24.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf5b596640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faf540b9c90 0x7faf540b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf5b596640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf540b6640 0x7faf540b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf5b596640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf5415e6a0 con 0x7faf5415e2c0 2024-04-25T00:07:24.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf5b596640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7faf540b6a90 con 0x7faf540b6640 2024-04-25T00:07:24.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf5b596640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faf540d2c80 con 0x7faf540b9c90 2024-04-25T00:07:24.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf59b33640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf540b6640 0x7faf540b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf59b33640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf540b6640 0x7faf540b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37430/0 (socket says 172.21.15.113:37430) 2024-04-25T00:07:24.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.143+0000 7faf59b33640 1 -- 172.21.15.113:0/2562587455 learned_addr learned my addr 172.21.15.113:0/2562587455 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:24.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.144+0000 7faf59b33640 1 -- 172.21.15.113:0/2562587455 >> v2:172.21.15.178:3300/0 conn(0x7faf540b9c90 msgr2=0x7faf540b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.144+0000 7faf59b33640 1 --2- 172.21.15.113:0/2562587455 >> v2:172.21.15.178:3300/0 conn(0x7faf540b9c90 0x7faf540b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.144+0000 7faf59b33640 1 -- 172.21.15.113:0/2562587455 >> v1:172.21.15.178:6789/0 conn(0x7faf5415e2c0 legacy=0x7faf540d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.144+0000 7faf59b33640 1 -- 172.21.15.113:0/2562587455 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf540d2f60 con 0x7faf540b6640 2024-04-25T00:07:24.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.144+0000 7faf59b33640 1 --2- 172.21.15.113:0/2562587455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b6640 0x7faf540b7200 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7faf540c7a20 tx=0x7faf4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528d55282f6f70d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.144+0000 7faf4bfff640 1 -- 172.21.15.113:0/2562587455 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c037070 con 0x7faf540b6640 2024-04-25T00:07:24.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf4bfff640 1 -- 172.21.15.113:0/2562587455 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4c032440 con 0x7faf540b6640 2024-04-25T00:07:24.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf4bfff640 1 -- 172.21.15.113:0/2562587455 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c032810 con 0x7faf540b6640 2024-04-25T00:07:24.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 -- 172.21.15.113:0/2562587455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b6640 msgr2=0x7faf540b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 --2- 172.21.15.113:0/2562587455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b6640 0x7faf540b7200 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7faf540c7a20 tx=0x7faf4c02a180 comp rx=0 tx=0).stop 2024-04-25T00:07:24.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 -- 172.21.15.113:0/2562587455 shutdown_connections 2024-04-25T00:07:24.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 --2- 172.21.15.113:0/2562587455 >> v2:172.21.15.178:3300/0 conn(0x7faf540b9c90 0x7faf540b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 --2- 172.21.15.113:0/2562587455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b6640 0x7faf540b7200 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 -- 172.21.15.113:0/2562587455 >> 172.21.15.113:0/2562587455 conn(0x7faf5406b510 msgr2=0x7faf5406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 -- 172.21.15.113:0/2562587455 shutdown_connections 2024-04-25T00:07:24.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.145+0000 7faf5b596640 1 -- 172.21.15.113:0/2562587455 wait complete. 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.146+0000 7faf5b596640 1 Processor -- start 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.146+0000 7faf5b596640 1 -- start start 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf5b596640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf540b6640 0x7faf540c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf5b596640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b9c90 0x7faf54157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf5b596640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf5415e2c0 0x7faf54158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf5b596640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faf540d3d20 con 0x7faf540b9c90 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf5b596640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf540d3520 con 0x7faf5415e2c0 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf5b596640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faf540b7740 con 0x7faf540b6640 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b9c90 0x7faf54157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b9c90 0x7faf54157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37444/0 (socket says 172.21.15.113:37444) 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 -- 172.21.15.113:0/2741406057 learned_addr learned my addr 172.21.15.113:0/2741406057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 -- 172.21.15.113:0/2741406057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf540b6640 msgr2=0x7faf540c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf540b6640 0x7faf540c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 -- 172.21.15.113:0/2741406057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf5415e2c0 msgr2=0x7faf54158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf5415e2c0 0x7faf54158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.147+0000 7faf58b31640 1 -- 172.21.15.113:0/2741406057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf54159f70 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.148+0000 7faf58b31640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b9c90 0x7faf54157c30 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7faf4400b4a0 tx=0x7faf4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.148+0000 7faf49ffb640 1 -- 172.21.15.113:0/2741406057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44014070 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.148+0000 7faf49ffb640 1 -- 172.21.15.113:0/2741406057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf44010610 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.148+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf540cc140 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.148+0000 7faf49ffb640 1 -- 172.21.15.113:0/2741406057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44010a00 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.148+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf540cc700 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.149+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf540691a0 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.149+0000 7faf49ffb640 1 -- 172.21.15.113:0/2741406057 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf44017070 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.149+0000 7faf49ffb640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf3806be30 0x7faf3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.149+0000 7faf49ffb640 1 -- 172.21.15.113:0/2741406057 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7faf4408a7f0 con 0x7faf540b9c90 2024-04-25T00:07:24.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.150+0000 7faf59332640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf3806be30 0x7faf3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.150+0000 7faf59332640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf3806be30 0x7faf3806e2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7faf5000abb0 tx=0x7faf50009250 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.154+0000 7faf49ffb640 1 -- 172.21.15.113:0/2741406057 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faf44059230 con 0x7faf540b9c90 2024-04-25T00:07:24.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.323+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf541590b0 con 0x7faf3806be30 2024-04-25T00:07:24.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.329+0000 7faf49ffb640 1 -- 172.21.15.113:0/2741406057 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252416 (secure 0 0 0) 0x7faf541590b0 con 0x7faf3806be30 2024-04-25T00:07:24.331 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:24.333 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:07:24.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf3806be30 msgr2=0x7faf3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf3806be30 0x7faf3806e2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7faf5000abb0 tx=0x7faf50009250 comp rx=0 tx=0).stop 2024-04-25T00:07:24.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b9c90 msgr2=0x7faf54157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b9c90 0x7faf54157c30 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7faf4400b4a0 tx=0x7faf4400b960 comp rx=0 tx=0).stop 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 shutdown_connections 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf3806be30 0x7faf3806e2f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf5415e2c0 0x7faf54158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf540b9c90 0x7faf54157c30 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 --2- 172.21.15.113:0/2741406057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf540b6640 0x7faf540c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 >> 172.21.15.113:0/2741406057 conn(0x7faf5406b510 msgr2=0x7faf540c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 shutdown_connections 2024-04-25T00:07:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.332+0000 7faf5b596640 1 -- 172.21.15.113:0/2741406057 wait complete. 2024-04-25T00:07:24.343 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-04-25T00:07:23.218010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":400616,"kb_used_data":15084,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754574104,"statfs":{"total":773094113280,"available":772683882496,"internally_reserved":0,"allocated":15446016,"data_stored":7398016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.227155"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018749+0000","last_change":"2024-04-25T00:07:21.013908+0000","last_active":"2024-04-25T00:07:21.018749+0000","last_peered":"2024-04-25T00:07:21.018749+0000","last_clean":"2024-04-25T00:07:21.018749+0000","last_became_active":"2024-04-25T00:07:21.013662+0000","last_became_peered":"2024-04-25T00:07:21.013662+0000","last_unstale":"2024-04-25T00:07:21.018749+0000","last_undegraded":"2024-04-25T00:07:21.018749+0000","last_fullsized":"2024-04-25T00:07:21.018749+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:04.159990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019861+0000","last_change":"2024-04-25T00:07:21.018411+0000","last_active":"2024-04-25T00:07:21.019861+0000","last_peered":"2024-04-25T00:07:21.019861+0000","last_clean":"2024-04-25T00:07:21.019861+0000","last_became_active":"2024-04-25T00:07:21.018145+0000","last_became_peered":"2024-04-25T00:07:21.018145+0000","last_unstale":"2024-04-25T00:07:21.019861+0000","last_undegraded":"2024-04-25T00:07:21.019861+0000","last_fullsized":"2024-04-25T00:07:21.019861+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:13.319914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023014+0000","last_change":"2024-04-25T00:07:21.021747+0000","last_active":"2024-04-25T00:07:21.023014+0000","last_peered":"2024-04-25T00:07:21.023014+0000","last_clean":"2024-04-25T00:07:21.023014+0000","last_became_active":"2024-04-25T00:07:21.021477+0000","last_became_peered":"2024-04-25T00:07:21.021477+0000","last_unstale":"2024-04-25T00:07:21.023014+0000","last_undegraded":"2024-04-25T00:07:21.023014+0000","last_fullsized":"2024-04-25T00:07:21.023014+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:48.928608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022318+0000","last_change":"2024-04-25T00:07:21.016871+0000","last_active":"2024-04-25T00:07:21.022318+0000","last_peered":"2024-04-25T00:07:21.022318+0000","last_clean":"2024-04-25T00:07:21.022318+0000","last_became_active":"2024-04-25T00:07:21.016589+0000","last_became_peered":"2024-04-25T00:07:21.016589+0000","last_unstale":"2024-04-25T00:07:21.022318+0000","last_undegraded":"2024-04-25T00:07:21.022318+0000","last_fullsized":"2024-04-25T00:07:21.022318+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:56.385071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017043+0000","last_change":"2024-04-25T00:07:21.016026+0000","last_active":"2024-04-25T00:07:21.017043+0000","last_peered":"2024-04-25T00:07:21.017043+0000","last_clean":"2024-04-25T00:07:21.017043+0000","last_became_active":"2024-04-25T00:07:21.015637+0000","last_became_peered":"2024-04-25T00:07:21.015637+0000","last_unstale":"2024-04-25T00:07:21.017043+0000","last_undegraded":"2024-04-25T00:07:21.017043+0000","last_fullsized":"2024-04-25T00:07:21.017043+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.580359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018984+0000","last_change":"2024-04-25T00:07:21.016168+0000","last_active":"2024-04-25T00:07:21.018984+0000","last_peered":"2024-04-25T00:07:21.018984+0000","last_clean":"2024-04-25T00:07:21.018984+0000","last_became_active":"2024-04-25T00:07:21.015903+0000","last_became_peered":"2024-04-25T00:07:21.015903+0000","last_unstale":"2024-04-25T00:07:21.018984+0000","last_undegraded":"2024-04-25T00:07:21.018984+0000","last_fullsized":"2024-04-25T00:07:21.018984+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:51.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009603+0000","last_change":"2024-04-25T00:07:21.007316+0000","last_active":"2024-04-25T00:07:21.009603+0000","last_peered":"2024-04-25T00:07:21.009603+0000","last_clean":"2024-04-25T00:07:21.009603+0000","last_became_active":"2024-04-25T00:07:21.006957+0000","last_became_peered":"2024-04-25T00:07:21.006957+0000","last_unstale":"2024-04-25T00:07:21.009603+0000","last_undegraded":"2024-04-25T00:07:21.009603+0000","last_fullsized":"2024-04-25T00:07:21.009603+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:47.351172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018964+0000","last_change":"2024-04-25T00:07:21.010995+0000","last_active":"2024-04-25T00:07:21.018964+0000","last_peered":"2024-04-25T00:07:21.018964+0000","last_clean":"2024-04-25T00:07:21.018964+0000","last_became_active":"2024-04-25T00:07:21.010671+0000","last_became_peered":"2024-04-25T00:07:21.010671+0000","last_unstale":"2024-04-25T00:07:21.018964+0000","last_undegraded":"2024-04-25T00:07:21.018964+0000","last_fullsized":"2024-04-25T00:07:21.018964+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:19.177947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017320+0000","last_change":"2024-04-25T00:07:21.011906+0000","last_active":"2024-04-25T00:07:21.017320+0000","last_peered":"2024-04-25T00:07:21.017320+0000","last_clean":"2024-04-25T00:07:21.017320+0000","last_became_active":"2024-04-25T00:07:21.011640+0000","last_became_peered":"2024-04-25T00:07:21.011640+0000","last_unstale":"2024-04-25T00:07:21.017320+0000","last_undegraded":"2024-04-25T00:07:21.017320+0000","last_fullsized":"2024-04-25T00:07:21.017320+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:59.812148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023496+0000","last_change":"2024-04-25T00:07:21.021161+0000","last_active":"2024-04-25T00:07:21.023496+0000","last_peered":"2024-04-25T00:07:21.023496+0000","last_clean":"2024-04-25T00:07:21.023496+0000","last_became_active":"2024-04-25T00:07:21.020808+0000","last_became_peered":"2024-04-25T00:07:21.020808+0000","last_unstale":"2024-04-25T00:07:21.023496+0000","last_undegraded":"2024-04-25T00:07:21.023496+0000","last_fullsized":"2024-04-25T00:07:21.023496+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:28.413225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019226+0000","last_change":"2024-04-25T00:07:21.014643+0000","last_active":"2024-04-25T00:07:21.019226+0000","last_peered":"2024-04-25T00:07:21.019226+0000","last_clean":"2024-04-25T00:07:21.019226+0000","last_became_active":"2024-04-25T00:07:21.014354+0000","last_became_peered":"2024-04-25T00:07:21.014354+0000","last_unstale":"2024-04-25T00:07:21.019226+0000","last_undegraded":"2024-04-25T00:07:21.019226+0000","last_fullsized":"2024-04-25T00:07:21.019226+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:56.220006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.021510+0000","last_change":"2024-04-25T00:07:21.018265+0000","last_active":"2024-04-25T00:07:21.021510+0000","last_peered":"2024-04-25T00:07:21.021510+0000","last_clean":"2024-04-25T00:07:21.021510+0000","last_became_active":"2024-04-25T00:07:21.017988+0000","last_became_peered":"2024-04-25T00:07:21.017988+0000","last_unstale":"2024-04-25T00:07:21.021510+0000","last_undegraded":"2024-04-25T00:07:21.021510+0000","last_fullsized":"2024-04-25T00:07:21.021510+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:46.134975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.016921+0000","last_change":"2024-04-25T00:07:21.014417+0000","last_active":"2024-04-25T00:07:21.016921+0000","last_peered":"2024-04-25T00:07:21.016921+0000","last_clean":"2024-04-25T00:07:21.016921+0000","last_became_active":"2024-04-25T00:07:21.014133+0000","last_became_peered":"2024-04-25T00:07:21.014133+0000","last_unstale":"2024-04-25T00:07:21.016921+0000","last_undegraded":"2024-04-25T00:07:21.016921+0000","last_fullsized":"2024-04-25T00:07:21.016921+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:19.177947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018162+0000","last_change":"2024-04-25T00:07:21.015743+0000","last_active":"2024-04-25T00:07:21.018162+0000","last_peered":"2024-04-25T00:07:21.018162+0000","last_clean":"2024-04-25T00:07:21.018162+0000","last_became_active":"2024-04-25T00:07:21.015213+0000","last_became_peered":"2024-04-25T00:07:21.015213+0000","last_unstale":"2024-04-25T00:07:21.018162+0000","last_undegraded":"2024-04-25T00:07:21.018162+0000","last_fullsized":"2024-04-25T00:07:21.018162+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:32.972002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027693+0000","last_change":"2024-04-25T00:07:21.026844+0000","last_active":"2024-04-25T00:07:21.027693+0000","last_peered":"2024-04-25T00:07:21.027693+0000","last_clean":"2024-04-25T00:07:21.027693+0000","last_became_active":"2024-04-25T00:07:21.026586+0000","last_became_peered":"2024-04-25T00:07:21.026586+0000","last_unstale":"2024-04-25T00:07:21.027693+0000","last_undegraded":"2024-04-25T00:07:21.027693+0000","last_fullsized":"2024-04-25T00:07:21.027693+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:57.640988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025753+0000","last_change":"2024-04-25T00:07:21.021966+0000","last_active":"2024-04-25T00:07:21.025753+0000","last_peered":"2024-04-25T00:07:21.025753+0000","last_clean":"2024-04-25T00:07:21.025753+0000","last_became_active":"2024-04-25T00:07:21.021707+0000","last_became_peered":"2024-04-25T00:07:21.021707+0000","last_unstale":"2024-04-25T00:07:21.025753+0000","last_undegraded":"2024-04-25T00:07:21.025753+0000","last_fullsized":"2024-04-25T00:07:21.025753+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:03.494994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.024951+0000","last_change":"2024-04-25T00:07:21.023245+0000","last_active":"2024-04-25T00:07:21.024951+0000","last_peered":"2024-04-25T00:07:21.024951+0000","last_clean":"2024-04-25T00:07:21.024951+0000","last_became_active":"2024-04-25T00:07:21.022943+0000","last_became_peered":"2024-04-25T00:07:21.022943+0000","last_unstale":"2024-04-25T00:07:21.024951+0000","last_undegraded":"2024-04-25T00:07:21.024951+0000","last_fullsized":"2024-04-25T00:07:21.024951+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:04.531859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025136+0000","last_change":"2024-04-25T00:07:21.020564+0000","last_active":"2024-04-25T00:07:21.025136+0000","last_peered":"2024-04-25T00:07:21.025136+0000","last_clean":"2024-04-25T00:07:21.025136+0000","last_became_active":"2024-04-25T00:07:21.020251+0000","last_became_peered":"2024-04-25T00:07:21.020251+0000","last_unstale":"2024-04-25T00:07:21.025136+0000","last_undegraded":"2024-04-25T00:07:21.025136+0000","last_fullsized":"2024-04-25T00:07:21.025136+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:33.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022184+0000","last_change":"2024-04-25T00:07:21.019012+0000","last_active":"2024-04-25T00:07:21.022184+0000","last_peered":"2024-04-25T00:07:21.022184+0000","last_clean":"2024-04-25T00:07:21.022184+0000","last_became_active":"2024-04-25T00:07:21.017011+0000","last_became_peered":"2024-04-25T00:07:21.017011+0000","last_unstale":"2024-04-25T00:07:21.022184+0000","last_undegraded":"2024-04-25T00:07:21.022184+0000","last_fullsized":"2024-04-25T00:07:21.022184+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.257701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025320+0000","last_change":"2024-04-25T00:07:21.014844+0000","last_active":"2024-04-25T00:07:21.025320+0000","last_peered":"2024-04-25T00:07:21.025320+0000","last_clean":"2024-04-25T00:07:21.025320+0000","last_became_active":"2024-04-25T00:07:21.014543+0000","last_became_peered":"2024-04-25T00:07:21.014543+0000","last_unstale":"2024-04-25T00:07:21.025320+0000","last_undegraded":"2024-04-25T00:07:21.025320+0000","last_fullsized":"2024-04-25T00:07:21.025320+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:38.395901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023078+0000","last_change":"2024-04-25T00:07:21.019329+0000","last_active":"2024-04-25T00:07:21.023078+0000","last_peered":"2024-04-25T00:07:21.023078+0000","last_clean":"2024-04-25T00:07:21.023078+0000","last_became_active":"2024-04-25T00:07:21.018779+0000","last_became_peered":"2024-04-25T00:07:21.018779+0000","last_unstale":"2024-04-25T00:07:21.023078+0000","last_undegraded":"2024-04-25T00:07:21.023078+0000","last_fullsized":"2024-04-25T00:07:21.023078+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:35.018370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018145+0000","last_change":"2024-04-25T00:07:21.015490+0000","last_active":"2024-04-25T00:07:21.018145+0000","last_peered":"2024-04-25T00:07:21.018145+0000","last_clean":"2024-04-25T00:07:21.018145+0000","last_became_active":"2024-04-25T00:07:21.013184+0000","last_became_peered":"2024-04-25T00:07:21.013184+0000","last_unstale":"2024-04-25T00:07:21.018145+0000","last_undegraded":"2024-04-25T00:07:21.018145+0000","last_fullsized":"2024-04-25T00:07:21.018145+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:05.543942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017094+0000","last_change":"2024-04-25T00:07:21.010658+0000","last_active":"2024-04-25T00:07:21.017094+0000","last_peered":"2024-04-25T00:07:21.017094+0000","last_clean":"2024-04-25T00:07:21.017094+0000","last_became_active":"2024-04-25T00:07:21.009589+0000","last_became_peered":"2024-04-25T00:07:21.009589+0000","last_unstale":"2024-04-25T00:07:21.017094+0000","last_undegraded":"2024-04-25T00:07:21.017094+0000","last_fullsized":"2024-04-25T00:07:21.017094+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:49.467963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023431+0000","last_change":"2024-04-25T00:07:21.020443+0000","last_active":"2024-04-25T00:07:21.023431+0000","last_peered":"2024-04-25T00:07:21.023431+0000","last_clean":"2024-04-25T00:07:21.023431+0000","last_became_active":"2024-04-25T00:07:21.020158+0000","last_became_peered":"2024-04-25T00:07:21.020158+0000","last_unstale":"2024-04-25T00:07:21.023431+0000","last_undegraded":"2024-04-25T00:07:21.023431+0000","last_fullsized":"2024-04-25T00:07:21.023431+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:47.351172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028534+0000","last_change":"2024-04-25T00:07:21.026566+0000","last_active":"2024-04-25T00:07:21.028534+0000","last_peered":"2024-04-25T00:07:21.028534+0000","last_clean":"2024-04-25T00:07:21.028534+0000","last_became_active":"2024-04-25T00:07:21.024864+0000","last_became_peered":"2024-04-25T00:07:21.024864+0000","last_unstale":"2024-04-25T00:07:21.028534+0000","last_undegraded":"2024-04-25T00:07:21.028534+0000","last_fullsized":"2024-04-25T00:07:21.028534+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:02.990216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029407+0000","last_change":"2024-04-25T00:07:21.026030+0000","last_active":"2024-04-25T00:07:21.029407+0000","last_peered":"2024-04-25T00:07:21.029407+0000","last_clean":"2024-04-25T00:07:21.029407+0000","last_became_active":"2024-04-25T00:07:21.025691+0000","last_became_peered":"2024-04-25T00:07:21.025691+0000","last_unstale":"2024-04-25T00:07:21.029407+0000","last_undegraded":"2024-04-25T00:07:21.029407+0000","last_fullsized":"2024-04-25T00:07:21.029407+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:38.790355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025177+0000","last_change":"2024-04-25T00:07:21.016811+0000","last_active":"2024-04-25T00:07:21.025177+0000","last_peered":"2024-04-25T00:07:21.025177+0000","last_clean":"2024-04-25T00:07:21.025177+0000","last_became_active":"2024-04-25T00:07:21.016564+0000","last_became_peered":"2024-04-25T00:07:21.016564+0000","last_unstale":"2024-04-25T00:07:21.025177+0000","last_undegraded":"2024-04-25T00:07:21.025177+0000","last_fullsized":"2024-04-25T00:07:21.025177+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:03.207212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.013451+0000","last_change":"2024-04-25T00:07:21.012011+0000","last_active":"2024-04-25T00:07:21.013451+0000","last_peered":"2024-04-25T00:07:21.013451+0000","last_clean":"2024-04-25T00:07:21.013451+0000","last_became_active":"2024-04-25T00:07:21.011740+0000","last_became_peered":"2024-04-25T00:07:21.011740+0000","last_unstale":"2024-04-25T00:07:21.013451+0000","last_undegraded":"2024-04-25T00:07:21.013451+0000","last_fullsized":"2024-04-25T00:07:21.013451+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:00.332178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018860+0000","last_change":"2024-04-25T00:07:21.016312+0000","last_active":"2024-04-25T00:07:21.018860+0000","last_peered":"2024-04-25T00:07:21.018860+0000","last_clean":"2024-04-25T00:07:21.018860+0000","last_became_active":"2024-04-25T00:07:21.016050+0000","last_became_peered":"2024-04-25T00:07:21.016050+0000","last_unstale":"2024-04-25T00:07:21.018860+0000","last_undegraded":"2024-04-25T00:07:21.018860+0000","last_fullsized":"2024-04-25T00:07:21.018860+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:26.562395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017249+0000","last_change":"2024-04-25T00:07:21.009397+0000","last_active":"2024-04-25T00:07:21.017249+0000","last_peered":"2024-04-25T00:07:21.017249+0000","last_clean":"2024-04-25T00:07:21.017249+0000","last_became_active":"2024-04-25T00:07:21.008146+0000","last_became_peered":"2024-04-25T00:07:21.008146+0000","last_unstale":"2024-04-25T00:07:21.017249+0000","last_undegraded":"2024-04-25T00:07:21.017249+0000","last_fullsized":"2024-04-25T00:07:21.017249+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.169614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018579+0000","last_change":"2024-04-25T00:07:21.016953+0000","last_active":"2024-04-25T00:07:21.018579+0000","last_peered":"2024-04-25T00:07:21.018579+0000","last_clean":"2024-04-25T00:07:21.018579+0000","last_became_active":"2024-04-25T00:07:21.016681+0000","last_became_peered":"2024-04-25T00:07:21.016681+0000","last_unstale":"2024-04-25T00:07:21.018579+0000","last_undegraded":"2024-04-25T00:07:21.018579+0000","last_fullsized":"2024-04-25T00:07:21.018579+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:45:31.831532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025235+0000","last_change":"2024-04-25T00:07:21.015129+0000","last_active":"2024-04-25T00:07:21.025235+0000","last_peered":"2024-04-25T00:07:21.025235+0000","last_clean":"2024-04-25T00:07:21.025235+0000","last_became_active":"2024-04-25T00:07:21.014451+0000","last_became_peered":"2024-04-25T00:07:21.014451+0000","last_unstale":"2024-04-25T00:07:21.025235+0000","last_undegraded":"2024-04-25T00:07:21.025235+0000","last_fullsized":"2024-04-25T00:07:21.025235+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:23.897258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022084+0000","last_change":"2024-04-25T00:07:21.018586+0000","last_active":"2024-04-25T00:07:21.022084+0000","last_peered":"2024-04-25T00:07:21.022084+0000","last_clean":"2024-04-25T00:07:21.022084+0000","last_became_active":"2024-04-25T00:07:21.017731+0000","last_became_peered":"2024-04-25T00:07:21.017731+0000","last_unstale":"2024-04-25T00:07:21.022084+0000","last_undegraded":"2024-04-25T00:07:21.022084+0000","last_fullsized":"2024-04-25T00:07:21.022084+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:17.874184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"143'4","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029913+0000","last_change":"2024-04-25T00:07:20.006506+0000","last_active":"2024-04-25T00:07:21.029913+0000","last_peered":"2024-04-25T00:07:21.029913+0000","last_clean":"2024-04-25T00:07:21.029913+0000","last_became_active":"2024-04-25T00:07:20.006227+0000","last_became_peered":"2024-04-25T00:07:20.006227+0000","last_unstale":"2024-04-25T00:07:21.029913+0000","last_undegraded":"2024-04-25T00:07:21.029913+0000","last_fullsized":"2024-04-25T00:07:21.029913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:08.342565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022312+0000","last_change":"2024-04-25T00:07:21.021356+0000","last_active":"2024-04-25T00:07:21.022312+0000","last_peered":"2024-04-25T00:07:21.022312+0000","last_clean":"2024-04-25T00:07:21.022312+0000","last_became_active":"2024-04-25T00:07:21.020933+0000","last_became_peered":"2024-04-25T00:07:21.020933+0000","last_unstale":"2024-04-25T00:07:21.022312+0000","last_undegraded":"2024-04-25T00:07:21.022312+0000","last_fullsized":"2024-04-25T00:07:21.022312+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:32.898564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025975+0000","last_change":"2024-04-25T00:07:20.010470+0000","last_active":"2024-04-25T00:07:21.025975+0000","last_peered":"2024-04-25T00:07:21.025975+0000","last_clean":"2024-04-25T00:07:21.025975+0000","last_became_active":"2024-04-25T00:07:20.010210+0000","last_became_peered":"2024-04-25T00:07:20.010210+0000","last_unstale":"2024-04-25T00:07:21.025975+0000","last_undegraded":"2024-04-25T00:07:21.025975+0000","last_fullsized":"2024-04-25T00:07:21.025975+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:11.281961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006943+0000","last_change":"2024-04-25T00:07:21.005772+0000","last_active":"2024-04-25T00:07:21.006943+0000","last_peered":"2024-04-25T00:07:21.006943+0000","last_clean":"2024-04-25T00:07:21.006943+0000","last_became_active":"2024-04-25T00:07:21.005471+0000","last_became_peered":"2024-04-25T00:07:21.005471+0000","last_unstale":"2024-04-25T00:07:21.006943+0000","last_undegraded":"2024-04-25T00:07:21.006943+0000","last_fullsized":"2024-04-25T00:07:21.006943+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:40.617621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027983+0000","last_change":"2024-04-25T00:07:20.009610+0000","last_active":"2024-04-25T00:07:21.027983+0000","last_peered":"2024-04-25T00:07:21.027983+0000","last_clean":"2024-04-25T00:07:21.027983+0000","last_became_active":"2024-04-25T00:07:20.009272+0000","last_became_peered":"2024-04-25T00:07:20.009272+0000","last_unstale":"2024-04-25T00:07:21.027983+0000","last_undegraded":"2024-04-25T00:07:21.027983+0000","last_fullsized":"2024-04-25T00:07:21.027983+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:10.759574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022308+0000","last_change":"2024-04-25T00:07:21.018114+0000","last_active":"2024-04-25T00:07:21.022308+0000","last_peered":"2024-04-25T00:07:21.022308+0000","last_clean":"2024-04-25T00:07:21.022308+0000","last_became_active":"2024-04-25T00:07:21.017581+0000","last_became_peered":"2024-04-25T00:07:21.017581+0000","last_unstale":"2024-04-25T00:07:21.022308+0000","last_undegraded":"2024-04-25T00:07:21.022308+0000","last_fullsized":"2024-04-25T00:07:21.022308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:24.839751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029898+0000","last_change":"2024-04-25T00:07:20.008716+0000","last_active":"2024-04-25T00:07:21.029898+0000","last_peered":"2024-04-25T00:07:21.029898+0000","last_clean":"2024-04-25T00:07:21.029898+0000","last_became_active":"2024-04-25T00:07:20.008437+0000","last_became_peered":"2024-04-25T00:07:20.008437+0000","last_unstale":"2024-04-25T00:07:21.029898+0000","last_undegraded":"2024-04-25T00:07:21.029898+0000","last_fullsized":"2024-04-25T00:07:21.029898+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:29.053924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018575+0000","last_change":"2024-04-25T00:07:21.016184+0000","last_active":"2024-04-25T00:07:21.018575+0000","last_peered":"2024-04-25T00:07:21.018575+0000","last_clean":"2024-04-25T00:07:21.018575+0000","last_became_active":"2024-04-25T00:07:21.015873+0000","last_became_peered":"2024-04-25T00:07:21.015873+0000","last_unstale":"2024-04-25T00:07:21.018575+0000","last_undegraded":"2024-04-25T00:07:21.018575+0000","last_fullsized":"2024-04-25T00:07:21.018575+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:18.611528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025169+0000","last_change":"2024-04-25T00:07:20.013802+0000","last_active":"2024-04-25T00:07:21.025169+0000","last_peered":"2024-04-25T00:07:21.025169+0000","last_clean":"2024-04-25T00:07:21.025169+0000","last_became_active":"2024-04-25T00:07:20.013446+0000","last_became_peered":"2024-04-25T00:07:20.013446+0000","last_unstale":"2024-04-25T00:07:21.025169+0000","last_undegraded":"2024-04-25T00:07:21.025169+0000","last_fullsized":"2024-04-25T00:07:21.025169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:50.319190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018380+0000","last_change":"2024-04-25T00:07:21.016939+0000","last_active":"2024-04-25T00:07:21.018380+0000","last_peered":"2024-04-25T00:07:21.018380+0000","last_clean":"2024-04-25T00:07:21.018380+0000","last_became_active":"2024-04-25T00:07:21.016629+0000","last_became_peered":"2024-04-25T00:07:21.016629+0000","last_unstale":"2024-04-25T00:07:21.018380+0000","last_undegraded":"2024-04-25T00:07:21.018380+0000","last_fullsized":"2024-04-25T00:07:21.018380+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:13.100124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027754+0000","last_change":"2024-04-25T00:07:20.015639+0000","last_active":"2024-04-25T00:07:21.027754+0000","last_peered":"2024-04-25T00:07:21.027754+0000","last_clean":"2024-04-25T00:07:21.027754+0000","last_became_active":"2024-04-25T00:07:20.014682+0000","last_became_peered":"2024-04-25T00:07:20.014682+0000","last_unstale":"2024-04-25T00:07:21.027754+0000","last_undegraded":"2024-04-25T00:07:21.027754+0000","last_fullsized":"2024-04-25T00:07:21.027754+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:15.524268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019112+0000","last_change":"2024-04-25T00:07:21.015450+0000","last_active":"2024-04-25T00:07:21.019112+0000","last_peered":"2024-04-25T00:07:21.019112+0000","last_clean":"2024-04-25T00:07:21.019112+0000","last_became_active":"2024-04-25T00:07:21.014424+0000","last_became_peered":"2024-04-25T00:07:21.014424+0000","last_unstale":"2024-04-25T00:07:21.019112+0000","last_undegraded":"2024-04-25T00:07:21.019112+0000","last_fullsized":"2024-04-25T00:07:21.019112+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:27:10.765664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.010839+0000","last_change":"2024-04-25T00:07:20.011465+0000","last_active":"2024-04-25T00:07:21.010839+0000","last_peered":"2024-04-25T00:07:21.010839+0000","last_clean":"2024-04-25T00:07:21.010839+0000","last_became_active":"2024-04-25T00:07:20.011143+0000","last_became_peered":"2024-04-25T00:07:20.011143+0000","last_unstale":"2024-04-25T00:07:21.010839+0000","last_undegraded":"2024-04-25T00:07:21.010839+0000","last_fullsized":"2024-04-25T00:07:21.010839+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:40.073850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018016+0000","last_change":"2024-04-25T00:07:21.016724+0000","last_active":"2024-04-25T00:07:21.018016+0000","last_peered":"2024-04-25T00:07:21.018016+0000","last_clean":"2024-04-25T00:07:21.018016+0000","last_became_active":"2024-04-25T00:07:21.016445+0000","last_became_peered":"2024-04-25T00:07:21.016445+0000","last_unstale":"2024-04-25T00:07:21.018016+0000","last_undegraded":"2024-04-25T00:07:21.018016+0000","last_fullsized":"2024-04-25T00:07:21.018016+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:26.562395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002641+0000","last_change":"2024-04-25T00:07:20.010465+0000","last_active":"2024-04-25T00:07:21.002641+0000","last_peered":"2024-04-25T00:07:21.002641+0000","last_clean":"2024-04-25T00:07:21.002641+0000","last_became_active":"2024-04-25T00:07:20.010199+0000","last_became_peered":"2024-04-25T00:07:20.010199+0000","last_unstale":"2024-04-25T00:07:21.002641+0000","last_undegraded":"2024-04-25T00:07:21.002641+0000","last_fullsized":"2024-04-25T00:07:21.002641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:47.923494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027886+0000","last_change":"2024-04-25T00:07:21.026575+0000","last_active":"2024-04-25T00:07:21.027886+0000","last_peered":"2024-04-25T00:07:21.027886+0000","last_clean":"2024-04-25T00:07:21.027886+0000","last_became_active":"2024-04-25T00:07:21.026304+0000","last_became_peered":"2024-04-25T00:07:21.026304+0000","last_unstale":"2024-04-25T00:07:21.027886+0000","last_undegraded":"2024-04-25T00:07:21.027886+0000","last_fullsized":"2024-04-25T00:07:21.027886+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:12.596577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006941+0000","last_change":"2024-04-25T00:07:20.006837+0000","last_active":"2024-04-25T00:07:21.006941+0000","last_peered":"2024-04-25T00:07:21.006941+0000","last_clean":"2024-04-25T00:07:21.006941+0000","last_became_active":"2024-04-25T00:07:20.006445+0000","last_became_peered":"2024-04-25T00:07:20.006445+0000","last_unstale":"2024-04-25T00:07:21.006941+0000","last_undegraded":"2024-04-25T00:07:21.006941+0000","last_fullsized":"2024-04-25T00:07:21.006941+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:35.871345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.026484+0000","last_change":"2024-04-25T00:07:21.024044+0000","last_active":"2024-04-25T00:07:21.026484+0000","last_peered":"2024-04-25T00:07:21.026484+0000","last_clean":"2024-04-25T00:07:21.026484+0000","last_became_active":"2024-04-25T00:07:21.023739+0000","last_became_peered":"2024-04-25T00:07:21.023739+0000","last_unstale":"2024-04-25T00:07:21.026484+0000","last_undegraded":"2024-04-25T00:07:21.026484+0000","last_fullsized":"2024-04-25T00:07:21.026484+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:00.332178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008004+0000","last_change":"2024-04-25T00:07:20.008802+0000","last_active":"2024-04-25T00:07:21.008004+0000","last_peered":"2024-04-25T00:07:21.008004+0000","last_clean":"2024-04-25T00:07:21.008004+0000","last_became_active":"2024-04-25T00:07:20.008515+0000","last_became_peered":"2024-04-25T00:07:20.008515+0000","last_unstale":"2024-04-25T00:07:21.008004+0000","last_undegraded":"2024-04-25T00:07:21.008004+0000","last_fullsized":"2024-04-25T00:07:21.008004+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:11.620560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025200+0000","last_change":"2024-04-25T00:07:21.017611+0000","last_active":"2024-04-25T00:07:21.025200+0000","last_peered":"2024-04-25T00:07:21.025200+0000","last_clean":"2024-04-25T00:07:21.025200+0000","last_became_active":"2024-04-25T00:07:21.017294+0000","last_became_peered":"2024-04-25T00:07:21.017294+0000","last_unstale":"2024-04-25T00:07:21.025200+0000","last_undegraded":"2024-04-25T00:07:21.025200+0000","last_fullsized":"2024-04-25T00:07:21.025200+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:32.700026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"143'6","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.031158+0000","last_change":"2024-04-25T00:07:20.008498+0000","last_active":"2024-04-25T00:07:21.031158+0000","last_peered":"2024-04-25T00:07:21.031158+0000","last_clean":"2024-04-25T00:07:21.031158+0000","last_became_active":"2024-04-25T00:07:20.008233+0000","last_became_peered":"2024-04-25T00:07:20.008233+0000","last_unstale":"2024-04-25T00:07:21.031158+0000","last_undegraded":"2024-04-25T00:07:21.031158+0000","last_fullsized":"2024-04-25T00:07:21.031158+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:51:16.869070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022957+0000","last_change":"2024-04-25T00:07:21.020790+0000","last_active":"2024-04-25T00:07:21.022957+0000","last_peered":"2024-04-25T00:07:21.022957+0000","last_clean":"2024-04-25T00:07:21.022957+0000","last_became_active":"2024-04-25T00:07:21.020230+0000","last_became_peered":"2024-04-25T00:07:21.020230+0000","last_unstale":"2024-04-25T00:07:21.022957+0000","last_undegraded":"2024-04-25T00:07:21.022957+0000","last_fullsized":"2024-04-25T00:07:21.022957+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:18:11.764688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.030276+0000","last_change":"2024-04-25T00:07:20.016657+0000","last_active":"2024-04-25T00:07:21.030276+0000","last_peered":"2024-04-25T00:07:21.030276+0000","last_clean":"2024-04-25T00:07:21.030276+0000","last_became_active":"2024-04-25T00:07:20.016344+0000","last_became_peered":"2024-04-25T00:07:20.016344+0000","last_unstale":"2024-04-25T00:07:21.030276+0000","last_undegraded":"2024-04-25T00:07:21.030276+0000","last_fullsized":"2024-04-25T00:07:21.030276+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:39.426245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017926+0000","last_change":"2024-04-25T00:07:21.015381+0000","last_active":"2024-04-25T00:07:21.017926+0000","last_peered":"2024-04-25T00:07:21.017926+0000","last_clean":"2024-04-25T00:07:21.017926+0000","last_became_active":"2024-04-25T00:07:21.015075+0000","last_became_peered":"2024-04-25T00:07:21.015075+0000","last_unstale":"2024-04-25T00:07:21.017926+0000","last_undegraded":"2024-04-25T00:07:21.017926+0000","last_fullsized":"2024-04-25T00:07:21.017926+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:29.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008163+0000","last_change":"2024-04-25T00:07:20.016875+0000","last_active":"2024-04-25T00:07:21.008163+0000","last_peered":"2024-04-25T00:07:21.008163+0000","last_clean":"2024-04-25T00:07:21.008163+0000","last_became_active":"2024-04-25T00:07:20.016517+0000","last_became_peered":"2024-04-25T00:07:20.016517+0000","last_unstale":"2024-04-25T00:07:21.008163+0000","last_undegraded":"2024-04-25T00:07:21.008163+0000","last_fullsized":"2024-04-25T00:07:21.008163+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:15.013625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.017019+0000","last_change":"2024-04-25T00:07:21.015169+0000","last_active":"2024-04-25T00:07:21.017019+0000","last_peered":"2024-04-25T00:07:21.017019+0000","last_clean":"2024-04-25T00:07:21.017019+0000","last_became_active":"2024-04-25T00:07:21.014899+0000","last_became_peered":"2024-04-25T00:07:21.014899+0000","last_unstale":"2024-04-25T00:07:21.017019+0000","last_undegraded":"2024-04-25T00:07:21.017019+0000","last_fullsized":"2024-04-25T00:07:21.017019+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:56.220006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002935+0000","last_change":"2024-04-25T00:07:20.007066+0000","last_active":"2024-04-25T00:07:21.002935+0000","last_peered":"2024-04-25T00:07:21.002935+0000","last_clean":"2024-04-25T00:07:21.002935+0000","last_became_active":"2024-04-25T00:07:20.006786+0000","last_became_peered":"2024-04-25T00:07:20.006786+0000","last_unstale":"2024-04-25T00:07:21.002935+0000","last_undegraded":"2024-04-25T00:07:21.002935+0000","last_fullsized":"2024-04-25T00:07:21.002935+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:26.542022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028474+0000","last_change":"2024-04-25T00:07:21.027428+0000","last_active":"2024-04-25T00:07:21.028474+0000","last_peered":"2024-04-25T00:07:21.028474+0000","last_clean":"2024-04-25T00:07:21.028474+0000","last_became_active":"2024-04-25T00:07:21.027135+0000","last_became_peered":"2024-04-25T00:07:21.027135+0000","last_unstale":"2024-04-25T00:07:21.028474+0000","last_undegraded":"2024-04-25T00:07:21.028474+0000","last_fullsized":"2024-04-25T00:07:21.028474+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.834891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"143'8","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028296+0000","last_change":"2024-04-25T00:07:20.010840+0000","last_active":"2024-04-25T00:07:21.028296+0000","last_peered":"2024-04-25T00:07:21.028296+0000","last_clean":"2024-04-25T00:07:21.028296+0000","last_became_active":"2024-04-25T00:07:20.009809+0000","last_became_peered":"2024-04-25T00:07:20.009809+0000","last_unstale":"2024-04-25T00:07:21.028296+0000","last_undegraded":"2024-04-25T00:07:21.028296+0000","last_fullsized":"2024-04-25T00:07:21.028296+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:11.493715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.022790+0000","last_change":"2024-04-25T00:07:21.019741+0000","last_active":"2024-04-25T00:07:21.022790+0000","last_peered":"2024-04-25T00:07:21.022790+0000","last_clean":"2024-04-25T00:07:21.022790+0000","last_became_active":"2024-04-25T00:07:21.019462+0000","last_became_peered":"2024-04-25T00:07:21.019462+0000","last_unstale":"2024-04-25T00:07:21.022790+0000","last_undegraded":"2024-04-25T00:07:21.022790+0000","last_fullsized":"2024-04-25T00:07:21.022790+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:18.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028940+0000","last_change":"2024-04-25T00:07:20.017466+0000","last_active":"2024-04-25T00:07:21.028940+0000","last_peered":"2024-04-25T00:07:21.028940+0000","last_clean":"2024-04-25T00:07:21.028940+0000","last_became_active":"2024-04-25T00:07:20.016941+0000","last_became_peered":"2024-04-25T00:07:20.016941+0000","last_unstale":"2024-04-25T00:07:21.028940+0000","last_undegraded":"2024-04-25T00:07:21.028940+0000","last_fullsized":"2024-04-25T00:07:21.028940+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:39.612507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.026086+0000","last_change":"2024-04-25T00:07:21.016055+0000","last_active":"2024-04-25T00:07:21.026086+0000","last_peered":"2024-04-25T00:07:21.026086+0000","last_clean":"2024-04-25T00:07:21.026086+0000","last_became_active":"2024-04-25T00:07:21.015792+0000","last_became_peered":"2024-04-25T00:07:21.015792+0000","last_unstale":"2024-04-25T00:07:21.026086+0000","last_undegraded":"2024-04-25T00:07:21.026086+0000","last_fullsized":"2024-04-25T00:07:21.026086+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:39.399920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028857+0000","last_change":"2024-04-25T00:07:20.014876+0000","last_active":"2024-04-25T00:07:21.028857+0000","last_peered":"2024-04-25T00:07:21.028857+0000","last_clean":"2024-04-25T00:07:21.028857+0000","last_became_active":"2024-04-25T00:07:20.014594+0000","last_became_peered":"2024-04-25T00:07:20.014594+0000","last_unstale":"2024-04-25T00:07:21.028857+0000","last_undegraded":"2024-04-25T00:07:21.028857+0000","last_fullsized":"2024-04-25T00:07:21.028857+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:12.674290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025815+0000","last_change":"2024-04-25T00:07:21.024281+0000","last_active":"2024-04-25T00:07:21.025815+0000","last_peered":"2024-04-25T00:07:21.025815+0000","last_clean":"2024-04-25T00:07:21.025815+0000","last_became_active":"2024-04-25T00:07:21.023991+0000","last_became_peered":"2024-04-25T00:07:21.023991+0000","last_unstale":"2024-04-25T00:07:21.025815+0000","last_undegraded":"2024-04-25T00:07:21.025815+0000","last_fullsized":"2024-04-25T00:07:21.025815+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.169614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.025258+0000","last_change":"2024-04-25T00:07:20.013426+0000","last_active":"2024-04-25T00:07:21.025258+0000","last_peered":"2024-04-25T00:07:21.025258+0000","last_clean":"2024-04-25T00:07:21.025258+0000","last_became_active":"2024-04-25T00:07:20.012844+0000","last_became_peered":"2024-04-25T00:07:20.012844+0000","last_unstale":"2024-04-25T00:07:21.025258+0000","last_undegraded":"2024-04-25T00:07:21.025258+0000","last_fullsized":"2024-04-25T00:07:21.025258+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:15.346208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019263+0000","last_change":"2024-04-25T00:07:21.017328+0000","last_active":"2024-04-25T00:07:21.019263+0000","last_peered":"2024-04-25T00:07:21.019263+0000","last_clean":"2024-04-25T00:07:21.019263+0000","last_became_active":"2024-04-25T00:07:21.016679+0000","last_became_peered":"2024-04-25T00:07:21.016679+0000","last_unstale":"2024-04-25T00:07:21.019263+0000","last_undegraded":"2024-04-25T00:07:21.019263+0000","last_fullsized":"2024-04-25T00:07:21.019263+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:12.596577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006903+0000","last_change":"2024-04-25T00:07:20.011616+0000","last_active":"2024-04-25T00:07:21.006903+0000","last_peered":"2024-04-25T00:07:21.006903+0000","last_clean":"2024-04-25T00:07:21.006903+0000","last_became_active":"2024-04-25T00:07:20.011298+0000","last_became_peered":"2024-04-25T00:07:20.011298+0000","last_unstale":"2024-04-25T00:07:21.006903+0000","last_undegraded":"2024-04-25T00:07:21.006903+0000","last_fullsized":"2024-04-25T00:07:21.006903+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:52.235634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018110+0000","last_change":"2024-04-25T00:07:21.011387+0000","last_active":"2024-04-25T00:07:21.018110+0000","last_peered":"2024-04-25T00:07:21.018110+0000","last_clean":"2024-04-25T00:07:21.018110+0000","last_became_active":"2024-04-25T00:07:21.008657+0000","last_became_peered":"2024-04-25T00:07:21.008657+0000","last_unstale":"2024-04-25T00:07:21.018110+0000","last_undegraded":"2024-04-25T00:07:21.018110+0000","last_fullsized":"2024-04-25T00:07:21.018110+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.385850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029379+0000","last_change":"2024-04-25T00:07:20.015444+0000","last_active":"2024-04-25T00:07:21.029379+0000","last_peered":"2024-04-25T00:07:21.029379+0000","last_clean":"2024-04-25T00:07:21.029379+0000","last_became_active":"2024-04-25T00:07:20.015093+0000","last_became_peered":"2024-04-25T00:07:20.015093+0000","last_unstale":"2024-04-25T00:07:21.029379+0000","last_undegraded":"2024-04-25T00:07:21.029379+0000","last_fullsized":"2024-04-25T00:07:21.029379+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:45.129861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008618+0000","last_change":"2024-04-25T00:07:21.006955+0000","last_active":"2024-04-25T00:07:21.008618+0000","last_peered":"2024-04-25T00:07:21.008618+0000","last_clean":"2024-04-25T00:07:21.008618+0000","last_became_active":"2024-04-25T00:07:21.006632+0000","last_became_peered":"2024-04-25T00:07:21.006632+0000","last_unstale":"2024-04-25T00:07:21.008618+0000","last_undegraded":"2024-04-25T00:07:21.008618+0000","last_fullsized":"2024-04-25T00:07:21.008618+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:18.978891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009365+0000","last_change":"2024-04-25T00:07:20.005829+0000","last_active":"2024-04-25T00:07:21.009365+0000","last_peered":"2024-04-25T00:07:21.009365+0000","last_clean":"2024-04-25T00:07:21.009365+0000","last_became_active":"2024-04-25T00:07:20.005523+0000","last_became_peered":"2024-04-25T00:07:20.005523+0000","last_unstale":"2024-04-25T00:07:21.009365+0000","last_undegraded":"2024-04-25T00:07:21.009365+0000","last_fullsized":"2024-04-25T00:07:21.009365+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:32.060924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.011166+0000","last_change":"2024-04-25T00:07:21.009919+0000","last_active":"2024-04-25T00:07:21.011166+0000","last_peered":"2024-04-25T00:07:21.011166+0000","last_clean":"2024-04-25T00:07:21.011166+0000","last_became_active":"2024-04-25T00:07:21.009631+0000","last_became_peered":"2024-04-25T00:07:21.009631+0000","last_unstale":"2024-04-25T00:07:21.011166+0000","last_undegraded":"2024-04-25T00:07:21.011166+0000","last_fullsized":"2024-04-25T00:07:21.011166+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.834891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.010175+0000","last_change":"2024-04-25T00:07:20.010722+0000","last_active":"2024-04-25T00:07:21.010175+0000","last_peered":"2024-04-25T00:07:21.010175+0000","last_clean":"2024-04-25T00:07:21.010175+0000","last_became_active":"2024-04-25T00:07:20.010433+0000","last_became_peered":"2024-04-25T00:07:20.010433+0000","last_unstale":"2024-04-25T00:07:21.010175+0000","last_undegraded":"2024-04-25T00:07:21.010175+0000","last_fullsized":"2024-04-25T00:07:21.010175+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:06.801027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009626+0000","last_change":"2024-04-25T00:07:21.007738+0000","last_active":"2024-04-25T00:07:21.009626+0000","last_peered":"2024-04-25T00:07:21.009626+0000","last_clean":"2024-04-25T00:07:21.009626+0000","last_became_active":"2024-04-25T00:07:21.007469+0000","last_became_peered":"2024-04-25T00:07:21.007469+0000","last_unstale":"2024-04-25T00:07:21.009626+0000","last_undegraded":"2024-04-25T00:07:21.009626+0000","last_fullsized":"2024-04-25T00:07:21.009626+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:28.413225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.008569+0000","last_change":"2024-04-25T00:07:20.018219+0000","last_active":"2024-04-25T00:07:21.008569+0000","last_peered":"2024-04-25T00:07:21.008569+0000","last_clean":"2024-04-25T00:07:21.008569+0000","last_became_active":"2024-04-25T00:07:20.017909+0000","last_became_peered":"2024-04-25T00:07:20.017909+0000","last_unstale":"2024-04-25T00:07:21.008569+0000","last_undegraded":"2024-04-25T00:07:21.008569+0000","last_fullsized":"2024-04-25T00:07:21.008569+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:40.185722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.016289+0000","last_change":"2024-04-25T00:07:21.014996+0000","last_active":"2024-04-25T00:07:21.016289+0000","last_peered":"2024-04-25T00:07:21.016289+0000","last_clean":"2024-04-25T00:07:21.016289+0000","last_became_active":"2024-04-25T00:07:21.014717+0000","last_became_peered":"2024-04-25T00:07:21.014717+0000","last_unstale":"2024-04-25T00:07:21.016289+0000","last_undegraded":"2024-04-25T00:07:21.016289+0000","last_fullsized":"2024-04-25T00:07:21.016289+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:43:04.159990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028773+0000","last_change":"2024-04-25T00:07:20.016031+0000","last_active":"2024-04-25T00:07:21.028773+0000","last_peered":"2024-04-25T00:07:21.028773+0000","last_clean":"2024-04-25T00:07:21.028773+0000","last_became_active":"2024-04-25T00:07:20.015619+0000","last_became_peered":"2024-04-25T00:07:20.015619+0000","last_unstale":"2024-04-25T00:07:21.028773+0000","last_undegraded":"2024-04-25T00:07:21.028773+0000","last_fullsized":"2024-04-25T00:07:21.028773+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:53.093311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.023831+0000","last_change":"2024-04-25T00:07:21.022531+0000","last_active":"2024-04-25T00:07:21.023831+0000","last_peered":"2024-04-25T00:07:21.023831+0000","last_clean":"2024-04-25T00:07:21.023831+0000","last_became_active":"2024-04-25T00:07:21.022208+0000","last_became_peered":"2024-04-25T00:07:21.022208+0000","last_unstale":"2024-04-25T00:07:21.023831+0000","last_undegraded":"2024-04-25T00:07:21.023831+0000","last_fullsized":"2024-04-25T00:07:21.023831+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:30.059038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002877+0000","last_change":"2024-04-25T00:07:20.006003+0000","last_active":"2024-04-25T00:07:21.002877+0000","last_peered":"2024-04-25T00:07:21.002877+0000","last_clean":"2024-04-25T00:07:21.002877+0000","last_became_active":"2024-04-25T00:07:20.005694+0000","last_became_peered":"2024-04-25T00:07:20.005694+0000","last_unstale":"2024-04-25T00:07:21.002877+0000","last_undegraded":"2024-04-25T00:07:21.002877+0000","last_fullsized":"2024-04-25T00:07:21.002877+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:01.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.013748+0000","last_change":"2024-04-25T00:07:21.011207+0000","last_active":"2024-04-25T00:07:21.013748+0000","last_peered":"2024-04-25T00:07:21.013748+0000","last_clean":"2024-04-25T00:07:21.013748+0000","last_became_active":"2024-04-25T00:07:21.010239+0000","last_became_peered":"2024-04-25T00:07:21.010239+0000","last_unstale":"2024-04-25T00:07:21.013748+0000","last_undegraded":"2024-04-25T00:07:21.013748+0000","last_fullsized":"2024-04-25T00:07:21.013748+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:45.498938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.004215+0000","last_change":"2024-04-25T00:07:20.008469+0000","last_active":"2024-04-25T00:07:21.004215+0000","last_peered":"2024-04-25T00:07:21.004215+0000","last_clean":"2024-04-25T00:07:21.004215+0000","last_became_active":"2024-04-25T00:07:20.008184+0000","last_became_peered":"2024-04-25T00:07:20.008184+0000","last_unstale":"2024-04-25T00:07:21.004215+0000","last_undegraded":"2024-04-25T00:07:21.004215+0000","last_fullsized":"2024-04-25T00:07:21.004215+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:15.524268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.019248+0000","last_change":"2024-04-25T00:07:21.014927+0000","last_active":"2024-04-25T00:07:21.019248+0000","last_peered":"2024-04-25T00:07:21.019248+0000","last_clean":"2024-04-25T00:07:21.019248+0000","last_became_active":"2024-04-25T00:07:21.011048+0000","last_became_peered":"2024-04-25T00:07:21.011048+0000","last_unstale":"2024-04-25T00:07:21.019248+0000","last_undegraded":"2024-04-25T00:07:21.019248+0000","last_fullsized":"2024-04-25T00:07:21.019248+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:44.580359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002316+0000","last_change":"2024-04-25T00:07:20.014660+0000","last_active":"2024-04-25T00:07:21.002316+0000","last_peered":"2024-04-25T00:07:21.002316+0000","last_clean":"2024-04-25T00:07:21.002316+0000","last_became_active":"2024-04-25T00:07:20.014394+0000","last_became_peered":"2024-04-25T00:07:20.014394+0000","last_unstale":"2024-04-25T00:07:21.002316+0000","last_undegraded":"2024-04-25T00:07:21.002316+0000","last_fullsized":"2024-04-25T00:07:21.002316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:02.489880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028562+0000","last_change":"2024-04-25T00:07:21.027013+0000","last_active":"2024-04-25T00:07:21.028562+0000","last_peered":"2024-04-25T00:07:21.028562+0000","last_clean":"2024-04-25T00:07:21.028562+0000","last_became_active":"2024-04-25T00:07:21.025129+0000","last_became_peered":"2024-04-25T00:07:21.025129+0000","last_unstale":"2024-04-25T00:07:21.028562+0000","last_undegraded":"2024-04-25T00:07:21.028562+0000","last_fullsized":"2024-04-25T00:07:21.028562+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:27.547137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.002572+0000","last_change":"2024-04-25T00:07:20.009590+0000","last_active":"2024-04-25T00:07:21.002572+0000","last_peered":"2024-04-25T00:07:21.002572+0000","last_clean":"2024-04-25T00:07:21.002572+0000","last_became_active":"2024-04-25T00:07:20.009137+0000","last_became_peered":"2024-04-25T00:07:20.009137+0000","last_unstale":"2024-04-25T00:07:21.002572+0000","last_undegraded":"2024-04-25T00:07:21.002572+0000","last_fullsized":"2024-04-25T00:07:21.002572+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:34.013256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.018072+0000","last_change":"2024-04-25T00:07:21.016255+0000","last_active":"2024-04-25T00:07:21.018072+0000","last_peered":"2024-04-25T00:07:21.018072+0000","last_clean":"2024-04-25T00:07:21.018072+0000","last_became_active":"2024-04-25T00:07:21.015966+0000","last_became_peered":"2024-04-25T00:07:21.015966+0000","last_unstale":"2024-04-25T00:07:21.018072+0000","last_undegraded":"2024-04-25T00:07:21.018072+0000","last_fullsized":"2024-04-25T00:07:21.018072+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:05:51.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009514+0000","last_change":"2024-04-25T00:07:20.014387+0000","last_active":"2024-04-25T00:07:21.009514+0000","last_peered":"2024-04-25T00:07:21.009514+0000","last_clean":"2024-04-25T00:07:21.009514+0000","last_became_active":"2024-04-25T00:07:20.014109+0000","last_became_peered":"2024-04-25T00:07:20.014109+0000","last_unstale":"2024-04-25T00:07:21.009514+0000","last_undegraded":"2024-04-25T00:07:21.009514+0000","last_fullsized":"2024-04-25T00:07:21.009514+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:16.192208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.027352+0000","last_change":"2024-04-25T00:07:21.026453+0000","last_active":"2024-04-25T00:07:21.027352+0000","last_peered":"2024-04-25T00:07:21.027352+0000","last_clean":"2024-04-25T00:07:21.027352+0000","last_became_active":"2024-04-25T00:07:21.026152+0000","last_became_peered":"2024-04-25T00:07:21.026152+0000","last_unstale":"2024-04-25T00:07:21.027352+0000","last_undegraded":"2024-04-25T00:07:21.027352+0000","last_fullsized":"2024-04-25T00:07:21.027352+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:04.807321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.005959+0000","last_change":"2024-04-25T00:07:20.012330+0000","last_active":"2024-04-25T00:07:21.005959+0000","last_peered":"2024-04-25T00:07:21.005959+0000","last_clean":"2024-04-25T00:07:21.005959+0000","last_became_active":"2024-04-25T00:07:20.011847+0000","last_became_peered":"2024-04-25T00:07:20.011847+0000","last_unstale":"2024-04-25T00:07:21.005959+0000","last_undegraded":"2024-04-25T00:07:21.005959+0000","last_fullsized":"2024-04-25T00:07:21.005959+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:59.779983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.028111+0000","last_change":"2024-04-25T00:07:21.027389+0000","last_active":"2024-04-25T00:07:21.028111+0000","last_peered":"2024-04-25T00:07:21.028111+0000","last_clean":"2024-04-25T00:07:21.028111+0000","last_became_active":"2024-04-25T00:07:21.027049+0000","last_became_peered":"2024-04-25T00:07:21.027049+0000","last_unstale":"2024-04-25T00:07:21.028111+0000","last_undegraded":"2024-04-25T00:07:21.028111+0000","last_fullsized":"2024-04-25T00:07:21.028111+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:45.257701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006064+0000","last_change":"2024-04-25T00:07:20.012877+0000","last_active":"2024-04-25T00:07:21.006064+0000","last_peered":"2024-04-25T00:07:21.006064+0000","last_clean":"2024-04-25T00:07:21.006064+0000","last_became_active":"2024-04-25T00:07:20.011114+0000","last_became_peered":"2024-04-25T00:07:20.011114+0000","last_unstale":"2024-04-25T00:07:21.006064+0000","last_undegraded":"2024-04-25T00:07:21.006064+0000","last_fullsized":"2024-04-25T00:07:21.006064+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:16.006603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.009634+0000","last_change":"2024-04-25T00:07:21.007775+0000","last_active":"2024-04-25T00:07:21.009634+0000","last_peered":"2024-04-25T00:07:21.009634+0000","last_clean":"2024-04-25T00:07:21.009634+0000","last_became_active":"2024-04-25T00:07:21.007478+0000","last_became_peered":"2024-04-25T00:07:21.007478+0000","last_unstale":"2024-04-25T00:07:21.009634+0000","last_undegraded":"2024-04-25T00:07:21.009634+0000","last_fullsized":"2024-04-25T00:07:21.009634+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:19.990975+0000","last_clean_scrub_stamp":"2024-04-25T00:07:19.990975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:04.531859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"143'3","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.029480+0000","last_change":"2024-04-25T00:07:20.018022+0000","last_active":"2024-04-25T00:07:21.029480+0000","last_peered":"2024-04-25T00:07:21.029480+0000","last_clean":"2024-04-25T00:07:21.029480+0000","last_became_active":"2024-04-25T00:07:20.017748+0000","last_became_peered":"2024-04-25T00:07:20.017748+0000","last_unstale":"2024-04-25T00:07:21.029480+0000","last_undegraded":"2024-04-25T00:07:21.029480+0000","last_fullsized":"2024-04-25T00:07:21.029480+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:07:18.985861+0000","last_clean_scrub_stamp":"2024-04-25T00:07:18.985861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:26.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":392,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T00:07:21.006696+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:07:21.006696+0000","last_peered":"2024-04-25T00:07:21.006696+0000","last_clean":"2024-04-25T00:07:21.006696+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:07:21.006696+0000","last_undegraded":"2024-04-25T00:07:21.006696+0000","last_fullsized":"2024-04-25T00:07:21.006696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47412,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324428,"statfs":{"total":96636764160,"available":96588214272,"internally_reserved":0,"allocated":2392064,"data_stored":1373107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50964,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320876,"statfs":{"total":96636764160,"available":96584577024,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1777664,"data_stored":775312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55144,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316696,"statfs":{"total":96636764160,"available":96580296704,"internally_reserved":0,"allocated":1794048,"data_stored":782669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47384,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324456,"statfs":{"total":96636764160,"available":96588242944,"internally_reserved":0,"allocated":2371584,"data_stored":1365680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:07:24.347 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:07:24.347 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:07:24.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.487+0000 7f9ab2342640 1 Processor -- start 2024-04-25T00:07:24.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.487+0000 7f9ab2342640 1 -- start start 2024-04-25T00:07:24.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9ab2342640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9aac1606a0 0x7f9aac167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9ab2342640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aac15d6b0 con 0x7f9aac13f050 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9ab2342640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aac15c8f0 con 0x7f9aac0d06c0 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9ab2342640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9aac15c6a0 con 0x7f9aac1606a0 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9aabfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9aac13f050 0x7f9aac0cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56576/0 (socket says 172.21.15.113:56576) 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9aabfff640 1 -- 172.21.15.113:0/1090645599 learned_addr learned my addr 172.21.15.113:0/1090645599 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9ab08df640 1 --2- 172.21.15.113:0/1090645599 >> v2:172.21.15.178:3300/0 conn(0x7f9aac1606a0 0x7f9aac167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.488+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132863143 0 0) 0x7f9aac15d6b0 con 0x7f9aac13f050 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a94003520 con 0x7f9aac13f050 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 762346485 0 0) 0x7f9a94003520 con 0x7f9aac13f050 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 >> v2:172.21.15.178:3300/0 conn(0x7f9aac1606a0 msgr2=0x7f9aac167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 --2- 172.21.15.113:0/1090645599 >> v2:172.21.15.178:3300/0 conn(0x7f9aac1606a0 0x7f9aac167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 >> v1:172.21.15.178:6789/0 conn(0x7f9aac0d06c0 legacy=0x7f9aac15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac15cbd0 con 0x7f9aac13f050 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f9a9c001670 con 0x7f9aac13f050 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 >> v1:172.21.15.113:6789/0 conn(0x7f9aac13f050 legacy=0x7f9aac0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a940039b0 0x7f9a94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a940051c0 0x7f9a9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac15c6a0 con 0x7f9a94004800 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9400c160 con 0x7f9a940039b0 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9400c360 con 0x7f9a940051c0 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aab7fe640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aabfff640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a940039b0 0x7f9a94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9ab08df640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a940051c0 0x7f9a9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aab7fe640 1 -- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a940051c0 msgr2=0x7f9a9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aab7fe640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a940051c0 0x7f9a9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aab7fe640 1 -- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a940039b0 msgr2=0x7f9a94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aab7fe640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a940039b0 0x7f9a94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9ab08df640 1 -- 172.21.15.113:0/1090645599 reap_dead start 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.489+0000 7f9aab7fe640 1 -- 172.21.15.113:0/1090645599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a94003520 con 0x7f9a94004800 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.490+0000 7f9aab7fe640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0d0b10 tx=0x7f9a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2549ff59a50f45eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.490+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0001370 con 0x7f9a94004800 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.490+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa00052d0 con 0x7f9a94004800 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.490+0000 7f9aaaffd640 1 -- 172.21.15.113:0/1090645599 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa002ab70 con 0x7f9a94004800 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.491+0000 7f9ab2342640 1 -- 172.21.15.113:0/1090645599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a94004800 msgr2=0x7f9a94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.491+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0d0b10 tx=0x7f9a94005830 comp rx=0 tx=0).stop 2024-04-25T00:07:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.491+0000 7f9ab2342640 1 -- 172.21.15.113:0/1090645599 shutdown_connections 2024-04-25T00:07:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.491+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1090645599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.491+0000 7f9ab2342640 1 -- 172.21.15.113:0/1090645599 >> 172.21.15.113:0/1090645599 conn(0x7f9aac06b0f0 msgr2=0x7f9aac06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.491+0000 7f9ab2342640 1 -- 172.21.15.113:0/1090645599 shutdown_connections 2024-04-25T00:07:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.491+0000 7f9ab2342640 1 -- 172.21.15.113:0/1090645599 wait complete. 2024-04-25T00:07:24.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.492+0000 7f9ab2342640 1 Processor -- start 2024-04-25T00:07:24.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.492+0000 7f9ab2342640 1 -- start start 2024-04-25T00:07:24.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a94004800 0x7f9aac0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aac071cd0 0x7f9aac13b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aac072480 0x7f9aac15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac13f430 con 0x7f9aac072480 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac15d190 con 0x7f9a94004800 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac15ff90 con 0x7f9aac071cd0 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aac072480 0x7f9aac15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aac072480 0x7f9aac15b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37458/0 (socket says 172.21.15.113:37458) 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 -- 172.21.15.113:0/1405561336 learned_addr learned my addr 172.21.15.113:0/1405561336 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9aab7fe640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aac071cd0 0x7f9aac13b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9aabfff640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a94004800 0x7f9aac0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 -- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aac071cd0 msgr2=0x7f9aac13b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aac071cd0 0x7f9aac13b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 -- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a94004800 msgr2=0x7f9aac0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a94004800 0x7f9aac0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 -- 172.21.15.113:0/1405561336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac072c90 con 0x7f9aac072480 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9aab7fe640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aac071cd0 0x7f9aac13b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9aabfff640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a94004800 0x7f9aac0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab08df640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aac072480 0x7f9aac15b1d0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0b5680 tx=0x7f9aa400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9aa8ff9640 1 -- 172.21.15.113:0/1405561336 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400fd80 con 0x7f9aac072480 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9aa8ff9640 1 -- 172.21.15.113:0/1405561336 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4015430 con 0x7f9aac072480 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9aa8ff9640 1 -- 172.21.15.113:0/1405561336 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4015820 con 0x7f9aac072480 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac072f70 con 0x7f9aac072480 2024-04-25T00:07:24.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.493+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac0b42b0 con 0x7f9aac072480 2024-04-25T00:07:24.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.494+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac0691a0 con 0x7f9aac072480 2024-04-25T00:07:24.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.495+0000 7f9aa8ff9640 1 -- 172.21.15.113:0/1405561336 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9aa40159c0 con 0x7f9aac072480 2024-04-25T00:07:24.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.495+0000 7f9aa8ff9640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a9806be60 0x7f9a9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.495+0000 7f9aa8ff9640 1 -- 172.21.15.113:0/1405561336 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f9aa408bdb0 con 0x7f9aac072480 2024-04-25T00:07:24.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.495+0000 7f9aabfff640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a9806be60 0x7f9a9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.496+0000 7f9aabfff640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a9806be60 0x7f9a9806e320 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0b2740 tx=0x7f9a9c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.499+0000 7f9aa8ff9640 1 -- 172.21.15.113:0/1405561336 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9aa4019030 con 0x7f9aac072480 2024-04-25T00:07:24.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.736+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9aac0d0e50 con 0x7f9aac072480 2024-04-25T00:07:24.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.738+0000 7f9aa8ff9640 1 -- 172.21.15.113:0/1405561336 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f9aa4010040 con 0x7f9aac072480 2024-04-25T00:07:24.739 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:07:24.739 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:07:24.739 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:07:24.739 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:07:24.739 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:07:24.739 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a9806be60 msgr2=0x7f9a9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a9806be60 0x7f9a9806e320 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0b2740 tx=0x7f9a9c009040 comp rx=0 tx=0).stop 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aac072480 msgr2=0x7f9aac15b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aac072480 0x7f9aac15b1d0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0b5680 tx=0x7f9aa400f580 comp rx=0 tx=0).stop 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 shutdown_connections 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9a9806be60 0x7f9a9806e320 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9aac072480 0x7f9aac15b1d0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aac071cd0 0x7f9aac13b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 --2- 172.21.15.113:0/1405561336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a94004800 0x7f9aac0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 >> 172.21.15.113:0/1405561336 conn(0x7f9aac06b0f0 msgr2=0x7f9aac13c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 shutdown_connections 2024-04-25T00:07:24.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.740+0000 7f9ab2342640 1 -- 172.21.15.113:0/1405561336 wait complete. 2024-04-25T00:07:24.752 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:07:24.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.847+0000 7fadbedc8640 1 Processor -- start 2024-04-25T00:07:24.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.847+0000 7fadbedc8640 1 -- start start 2024-04-25T00:07:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadbedc8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fadb8144910 0x7fadb80b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadbedc8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb8144cf0 con 0x7fadb80b5240 2024-04-25T00:07:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadbedc8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb80b2f30 con 0x7fadb80b56b0 2024-04-25T00:07:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadbedc8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fadb80b4510 con 0x7fadb8144910 2024-04-25T00:07:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fadb80b5240 0x7fadb806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56580/0 (socket says 172.21.15.113:56580) 2024-04-25T00:07:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb7fff640 1 -- 172.21.15.113:0/1414820979 learned_addr learned my addr 172.21.15.113:0/1414820979 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1100420686 0 0) 0x7fadb8144cf0 con 0x7fadb80b5240 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadbcb64640 1 --2- 172.21.15.113:0/1414820979 >> v2:172.21.15.178:3300/0 conn(0x7fadb8144910 0x7fadb80b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fada8003520 con 0x7fadb80b5240 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 694054854 0 0) 0x7fada8003520 con 0x7fadb80b5240 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 >> v2:172.21.15.178:3300/0 conn(0x7fadb8144910 msgr2=0x7fadb80b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 --2- 172.21.15.113:0/1414820979 >> v2:172.21.15.178:3300/0 conn(0x7fadb8144910 0x7fadb80b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 >> v1:172.21.15.178:6789/0 conn(0x7fadb80b56b0 legacy=0x7fadb80cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb80b3210 con 0x7fadb80b5240 2024-04-25T00:07:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fada4001670 con 0x7fadb80b5240 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 >> v1:172.21.15.113:6789/0 conn(0x7fadb80b5240 legacy=0x7fadb806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.848+0000 7fadb77fe640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fada80039b0 0x7fada8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb77fe640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fada8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb77fe640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada80051c0 0x7fada800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80b4510 con 0x7fada8004800 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fada800c090 con 0x7fada80051c0 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fada800c290 con 0x7fada80039b0 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb7fff640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fada8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadbcb64640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fada80039b0 0x7fada8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadbd365640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada80051c0 0x7fada800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb7fff640 1 -- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fada80039b0 msgr2=0x7fada8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb7fff640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fada80039b0 0x7fada8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb7fff640 1 -- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada80051c0 msgr2=0x7fada800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb7fff640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada80051c0 0x7fada800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadb7fff640 1 -- 172.21.15.113:0/1414820979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada8003520 con 0x7fada8004800 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadbcb64640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fada80039b0 0x7fada8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:24.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadbd365640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada80051c0 0x7fada800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.849+0000 7fadbd365640 1 -- 172.21.15.113:0/1414820979 reap_dead start 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadb7fff640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fada8004c80 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fada400c210 tx=0x7fada8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80492f058839b739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4007d10 con 0x7fada8004800 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada40058d0 con 0x7fada8004800 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadb77fe640 1 -- 172.21.15.113:0/1414820979 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4005ce0 con 0x7fada8004800 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadbedc8640 1 -- 172.21.15.113:0/1414820979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 msgr2=0x7fada8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadbedc8640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fada8004c80 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fada400c210 tx=0x7fada8005830 comp rx=0 tx=0).stop 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadbedc8640 1 -- 172.21.15.113:0/1414820979 shutdown_connections 2024-04-25T00:07:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadbedc8640 1 --2- 172.21.15.113:0/1414820979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fada8004c80 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadbedc8640 1 -- 172.21.15.113:0/1414820979 >> 172.21.15.113:0/1414820979 conn(0x7fadb80b1a50 msgr2=0x7fadb80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadbedc8640 1 -- 172.21.15.113:0/1414820979 shutdown_connections 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.850+0000 7fadbedc8640 1 -- 172.21.15.113:0/1414820979 wait complete. 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.851+0000 7fadbedc8640 1 Processor -- start 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.851+0000 7fadbedc8640 1 -- start start 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbedc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fadb813e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbedc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb81e4c10 0x7fadb81e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbedc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb80b0d60 0x7fadb80b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbedc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80b4da0 con 0x7fada8004800 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbedc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80b37d0 con 0x7fadb80b0d60 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbedc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadb80b4f20 con 0x7fadb81e4c10 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbcb64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fadb813e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbcb64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fadb813e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37482/0 (socket says 172.21.15.113:37482) 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.852+0000 7fadbcb64640 1 -- 172.21.15.113:0/2473805412 learned_addr learned my addr 172.21.15.113:0/2473805412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbd365640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb80b0d60 0x7fadb80b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbcb64640 1 -- 172.21.15.113:0/2473805412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb81e4c10 msgr2=0x7fadb81e5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbcb64640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb81e4c10 0x7fadb81e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbcb64640 1 -- 172.21.15.113:0/2473805412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb80b0d60 msgr2=0x7fadb80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbcb64640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb80b0d60 0x7fadb80b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbcb64640 1 -- 172.21.15.113:0/2473805412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb81e6360 con 0x7fada8004800 2024-04-25T00:07:24.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbcb64640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fadb813e3f0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fadac0114e0 tx=0x7fadac0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadb57fa640 1 -- 172.21.15.113:0/2473805412 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac015870 con 0x7fada8004800 2024-04-25T00:07:24.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadb57fa640 1 -- 172.21.15.113:0/2473805412 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac015a10 con 0x7fada8004800 2024-04-25T00:07:24.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb8138830 con 0x7fada8004800 2024-04-25T00:07:24.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadb57fa640 1 -- 172.21.15.113:0/2473805412 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac015e00 con 0x7fada8004800 2024-04-25T00:07:24.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.853+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb8138df0 con 0x7fada8004800 2024-04-25T00:07:24.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.854+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb80b3fd0 con 0x7fada8004800 2024-04-25T00:07:24.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.854+0000 7fadb57fa640 1 -- 172.21.15.113:0/2473805412 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadac01d400 con 0x7fada8004800 2024-04-25T00:07:24.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.855+0000 7fadb57fa640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9c06be30 0x7fad9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.855+0000 7fadb57fa640 1 -- 172.21.15.113:0/2473805412 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fadac08b770 con 0x7fada8004800 2024-04-25T00:07:24.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.855+0000 7fadb7fff640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9c06be30 0x7fad9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.855+0000 7fadb7fff640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9c06be30 0x7fad9c06e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fada4033000 tx=0x7fada4007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:24.859+0000 7fadb57fa640 1 -- 172.21.15.113:0/2473805412 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fadb80b3fd0 con 0x7fada8004800 2024-04-25T00:07:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.072+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadb80691a0 con 0x7fada8004800 2024-04-25T00:07:25.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.072+0000 7fadb57fa640 1 -- 172.21.15.113:0/2473805412 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5197 (secure 0 0 0) 0x7fadac019030 con 0x7fada8004800 2024-04-25T00:07:25.073 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:25.073 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1199283489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1199283489},{"type":"v1","addr":"172.21.15.178:6834","nonce":1199283489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4259795043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4259795043},{"type":"v1","addr":"172.21.15.178:6835","nonce":4259795043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4146002756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4146002756},{"type":"v1","addr":"172.21.15.113:6835","nonce":4146002756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:07:20.361298+0000","modified":"2024-04-25T00:07:22.348971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2111641286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2111641286},{"type":"v1","addr":"172.21.15.113:6837","nonce":2111641286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":10}]} 2024-04-25T00:07:25.073 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-04-25T00:07:25.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.074+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9c06be30 msgr2=0x7fad9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.074+0000 7fadbedc8640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9c06be30 0x7fad9c06e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fada4033000 tx=0x7fada4007000 comp rx=0 tx=0).stop 2024-04-25T00:07:25.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.074+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 msgr2=0x7fadb813e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.074+0000 7fadbedc8640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fadb813e3f0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fadac0114e0 tx=0x7fadac0119a0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.074+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 shutdown_connections 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.074+0000 7fadbedc8640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9c06be30 0x7fad9c06e2f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.074+0000 7fadbedc8640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb80b0d60 0x7fadb80b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.075+0000 7fadbedc8640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb81e4c10 0x7fadb81e5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.075+0000 7fadbedc8640 1 --2- 172.21.15.113:0/2473805412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada8004800 0x7fadb813e3f0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.075+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 >> 172.21.15.113:0/2473805412 conn(0x7fadb80b1a50 msgr2=0x7fadb815f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.075+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 shutdown_connections 2024-04-25T00:07:25.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.075+0000 7fadbedc8640 1 -- 172.21.15.113:0/2473805412 wait complete. 2024-04-25T00:07:25.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:07:20.361298+0000', 'modified': '2024-04-25T00:07:22.348971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9501}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9501': {'gid': 9501, 'name': 'a', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6837/2111641286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 2111641286}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 2111641286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9501, 'qdb_cluster': [9501]} 2024-04-25T00:07:25.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:07:25.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:07:25.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.588+0000 7f204c390640 1 Processor -- start 2024-04-25T00:07:25.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.588+0000 7f204c390640 1 -- start start 2024-04-25T00:07:25.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.589+0000 7f204c390640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 0x7f20440b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.589+0000 7f204c390640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f20440b73c0 0x7f20440b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.589+0000 7f204c390640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20440d2e00 con 0x7f204415e510 2024-04-25T00:07:25.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.589+0000 7f204c390640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f20440b6640 con 0x7f20440b73c0 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.589+0000 7f204c390640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f20440d30e0 con 0x7f20440b5ca0 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.589+0000 7f204a12c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f204415e510 0x7f20440d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56592/0 (socket says 172.21.15.113:56592) 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.589+0000 7f204a12c640 1 -- 172.21.15.113:0/3374665557 learned_addr learned my addr 172.21.15.113:0/3374665557 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204992b640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 0x7f20440b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a92d640 1 --2- 172.21.15.113:0/3374665557 >> v2:172.21.15.178:3300/0 conn(0x7f20440b73c0 0x7f20440b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2050518534 0 0) 0x7f20440d2e00 con 0x7f204415e510 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2030003520 con 0x7f204415e510 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1543539984 0 0) 0x7f2030003520 con 0x7f204415e510 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 msgr2=0x7f20440b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 0x7f20440b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 >> v2:172.21.15.178:3300/0 conn(0x7f20440b73c0 msgr2=0x7f20440b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 --2- 172.21.15.113:0/3374665557 >> v2:172.21.15.178:3300/0 conn(0x7f20440b73c0 0x7f20440b7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20440d33c0 con 0x7f204415e510 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f20400019f0 con 0x7f204415e510 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204992b640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 0x7f20440b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 >> v1:172.21.15.113:6789/0 conn(0x7f204415e510 legacy=0x7f20440d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f2030003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 0x7f2030004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2030004d40 0x7f20300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20440d30e0 con 0x7f20300039b0 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f203000b9c0 con 0x7f2030004d40 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f203000bb40 con 0x7f20440b5ca0 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a12c640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f2030003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204992b640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 0x7f2030004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a92d640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2030004d40 0x7f20300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a12c640 1 -- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 msgr2=0x7f2030004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a12c640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440b5ca0 0x7f2030004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a12c640 1 -- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2030004d40 msgr2=0x7f20300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a12c640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2030004d40 0x7f20300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.590+0000 7f204a12c640 1 -- 172.21.15.113:0/3374665557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2030003520 con 0x7f20300039b0 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.591+0000 7f204a12c640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f2030003dd0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f204000a920 tx=0x7f2030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e765090116d3dcc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.591+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204000a4c0 con 0x7f20300039b0 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.591+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204002f070 con 0x7f20300039b0 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.591+0000 7f204912a640 1 -- 172.21.15.113:0/3374665557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204002bd20 con 0x7f20300039b0 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.592+0000 7f204c390640 1 -- 172.21.15.113:0/3374665557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 msgr2=0x7f2030003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.592+0000 7f204c390640 1 --2- 172.21.15.113:0/3374665557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f2030003dd0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f204000a920 tx=0x7f2030005830 comp rx=0 tx=0).stop 2024-04-25T00:07:25.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.592+0000 7f204a92d640 1 -- 172.21.15.113:0/3374665557 reap_dead start 2024-04-25T00:07:25.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.592+0000 7f204c390640 1 -- 172.21.15.113:0/3374665557 shutdown_connections 2024-04-25T00:07:25.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.592+0000 7f204c390640 1 -- 172.21.15.113:0/3374665557 >> 172.21.15.113:0/3374665557 conn(0x7f204406b720 msgr2=0x7f20440d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.592+0000 7f204c390640 1 -- 172.21.15.113:0/3374665557 shutdown_connections 2024-04-25T00:07:25.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.592+0000 7f204c390640 1 -- 172.21.15.113:0/3374665557 wait complete. 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.593+0000 7f204c390640 1 Processor -- start 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.593+0000 7f204c390640 1 -- start start 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f20440c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204414fe20 0x7f2044151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440d4200 0x7f20440d4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20440d3ea0 con 0x7f20300039b0 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20440d3980 con 0x7f204414fe20 2024-04-25T00:07:25.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f20440d2a90 con 0x7f20440d4200 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f20440c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f20440c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37506/0 (socket says 172.21.15.113:37506) 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 -- 172.21.15.113:0/2181057491 learned_addr learned my addr 172.21.15.113:0/2181057491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204992b640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204414fe20 0x7f2044151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a92d640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440d4200 0x7f20440d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 -- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440d4200 msgr2=0x7f20440d4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440d4200 0x7f20440d4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 -- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204414fe20 msgr2=0x7f2044151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204414fe20 0x7f2044151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 -- 172.21.15.113:0/2181057491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2044150390 con 0x7f20300039b0 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a92d640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440d4200 0x7f20440d4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:25.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204a12c640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f20440c9d30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f204000aa40 tx=0x7f20440d2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f203affd640 1 -- 172.21.15.113:0/2181057491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204002b8a0 con 0x7f20300039b0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f203affd640 1 -- 172.21.15.113:0/2181057491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204002ba40 con 0x7f20300039b0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2044150670 con 0x7f20300039b0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f203affd640 1 -- 172.21.15.113:0/2181057491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204002be10 con 0x7f20300039b0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.594+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2044150c30 con 0x7f20300039b0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.596+0000 7f203affd640 1 -- 172.21.15.113:0/2181057491 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2040048020 con 0x7f20300039b0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.596+0000 7f203affd640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f202806be60 0x7f202806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.596+0000 7f203affd640 1 -- 172.21.15.113:0/2181057491 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f20400ab960 con 0x7f20300039b0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.596+0000 7f204992b640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f202806be60 0x7f202806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.597+0000 7f204992b640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f202806be60 0x7f202806e320 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f20440720d0 tx=0x7f2034009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.597+0000 7f204f17d740 1 Processor -- start 2024-04-25T00:07:25.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204f17d740 1 -- start start 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204f17d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4e1b60 0x56357b4944f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4c4c80 0x56357b4c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4540 0x56357b499350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204f17d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56357b3fcf00 con 0x56357b4e1b60 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56357b3a5750 con 0x56357b4c4540 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56357b3f8ec0 con 0x56357b4c4c80 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204a12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4e1b60 0x56357b4944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204a12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4e1b60 0x56357b4944f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37508/0 (socket says 172.21.15.113:37508) 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.598+0000 7f204a12c640 1 -- 172.21.15.113:0/2609456306 learned_addr learned my addr 172.21.15.113:0/2609456306 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f204a12c640 1 -- 172.21.15.113:0/2609456306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4c4c80 msgr2=0x56357b4c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f204a12c640 1 --2- 172.21.15.113:0/2609456306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4c4c80 0x56357b4c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f204a12c640 1 -- 172.21.15.113:0/2609456306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4540 msgr2=0x56357b499350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f204a12c640 1 --2- 172.21.15.113:0/2609456306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4540 0x56357b499350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f204a12c640 1 -- 172.21.15.113:0/2609456306 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357b1112d0 con 0x56357b4e1b60 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f204a12c640 1 --2- 172.21.15.113:0/2609456306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4e1b60 0x56357b4944f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f20400a9e00 tx=0x7f2040079ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39748f82c122431a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f2027fff640 1 -- 172.21.15.113:0/2609456306 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040082910 con 0x56357b4e1b60 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f2027fff640 1 -- 172.21.15.113:0/2609456306 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204002f070 con 0x56357b4e1b60 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.599+0000 7f2027fff640 1 -- 172.21.15.113:0/2609456306 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040076310 con 0x56357b4e1b60 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 -- 172.21.15.113:0/2609456306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4e1b60 msgr2=0x56357b4944f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 --2- 172.21.15.113:0/2609456306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4e1b60 0x56357b4944f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f20400a9e00 tx=0x7f2040079ca0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 -- 172.21.15.113:0/2609456306 shutdown_connections 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 --2- 172.21.15.113:0/2609456306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4540 0x56357b499350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 --2- 172.21.15.113:0/2609456306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4c4c80 0x56357b4c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 --2- 172.21.15.113:0/2609456306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4e1b60 0x56357b4944f0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 -- 172.21.15.113:0/2609456306 >> 172.21.15.113:0/2609456306 conn(0x56357b496760 msgr2=0x56357b4df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 -- 172.21.15.113:0/2609456306 shutdown_connections 2024-04-25T00:07:25.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 -- 172.21.15.113:0/2609456306 wait complete. 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 -- start start 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.600+0000 7f204f17d740 1 Processor -- start 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204f17d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c4c80 0x56357b4b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b4c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c6890 0x56357b4e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204f17d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56357b3a5750 con 0x56357b4c4c80 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56357b3b8cf0 con 0x56357b4c6890 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56357b3fcf00 con 0x56357b4e1b60 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204f17d740 10 client.0 _ll_register_callbacks cb 0x56357b3b86e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204a12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c4c80 0x56357b4b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204a12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c4c80 0x56357b4b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37512/0 (socket says 172.21.15.113:37512) 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204a12c640 1 -- 172.21.15.113:0/580930203 learned_addr learned my addr 172.21.15.113:0/580930203 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:25.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204a12c640 1 -- 172.21.15.113:0/580930203 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 msgr2=0x56357b4c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204a12c640 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b4c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204a12c640 1 -- 172.21.15.113:0/580930203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c6890 msgr2=0x56357b4e0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.601+0000 7f204a12c640 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c6890 0x56357b4e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204a12c640 1 -- 172.21.15.113:0/580930203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357b1db6b0 con 0x56357b4c4c80 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204a12c640 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c4c80 0x56357b4b49d0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f20400a9ee0 tx=0x7f2040043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac72868673fe74c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f20257fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f20257fa640 1 -- 172.21.15.113:0/580930203 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040087030 con 0x56357b4c4c80 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f20257fa640 1 -- 172.21.15.113:0/580930203 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f2040082690 con 0x56357b4c4c80 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f204002f070 con 0x56357b4c4c80 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f20257fa640 1 -- 172.21.15.113:0/580930203 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2040078060 con 0x56357b4c4c80 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 10 client.9651 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:25.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x56357b1db6b0 con 0x56357b4c4c80 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f20257fa640 1 -- 172.21.15.113:0/580930203 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204002f070 con 0x56357b4c4c80 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f20257fa640 1 -- 172.21.15.113:0/580930203 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f2040082b60 con 0x56357b4c4c80 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 10 client.9651 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 10 client.9651 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 20 client.9651 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4d5cc0 0x56357b4d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 4 client.9651 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357b40a440 con 0x56357b4d5cc0 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.602+0000 7f204992b640 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4d5cc0 0x56357b4d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.603+0000 7f204992b640 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4d5cc0 0x56357b4d60a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a0715d6e64fe02 server_cookie=c1b0be936e9fa6a8 in_seq=0 out_seq=0 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.603+0000 7f20257fa640 10 client.9651 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.603+0000 7f20257fa640 1 -- 172.21.15.113:0/580930203 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56357b40a440 con 0x56357b4d5cc0 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.603+0000 7f20257fa640 10 client.9651 handle_command_reply: tid=0 2024-04-25T00:07:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.603+0000 7f204f17d740 1 client.9651 shutdown 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.603+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c4c80 msgr2=0x56357b4b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.603+0000 7f204f17d740 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c4c80 0x56357b4b49d0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f20400a9ee0 tx=0x7f2040043040 comp rx=0 tx=0).stop 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 shutdown_connections 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4d5cc0 0x56357b4d60a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c6890 0x56357b4e0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b4c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f20257fa640 0 client.9651 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 --2- 172.21.15.113:0/580930203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c4c80 0x56357b4b49d0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 >> 172.21.15.113:0/580930203 conn(0x56357b4c4540 msgr2=0x56357b497f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 shutdown_connections 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 1 -- 172.21.15.113:0/580930203 wait complete. 2024-04-25T00:07:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.604+0000 7f204f17d740 20 client.9651 trim_cache size 0 max 16384 2024-04-25T00:07:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.621+0000 7f204f17d740 1 Processor -- start 2024-04-25T00:07:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204f17d740 1 -- start start 2024-04-25T00:07:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204f17d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4d7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b502540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204f17d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20441fff70 con 0x56357b4c6890 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20440ccea0 con 0x56357b4c4c80 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f20441ff920 con 0x56357b4e1b60 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204992b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4d7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204992b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4d7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37520/0 (socket says 172.21.15.113:37520) 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.622+0000 7f204992b640 1 -- 172.21.15.113:0/977181815 learned_addr learned my addr 172.21.15.113:0/977181815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204a12c640 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204a92d640 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b502540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204992b640 1 -- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 msgr2=0x56357b502540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204992b640 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b502540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204992b640 1 -- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 msgr2=0x56357b4f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204992b640 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204992b640 1 -- 172.21.15.113:0/977181815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357b1db6b0 con 0x56357b4c6890 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204992b640 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4d7b30 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x56357b4e0b90 tx=0x7f2034017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c350afae3f3c2b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f20277fe640 1 -- 172.21.15.113:0/977181815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203401b070 con 0x56357b4c6890 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f20277fe640 1 -- 172.21.15.113:0/977181815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2034019070 con 0x56357b4c6890 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f20277fe640 1 -- 172.21.15.113:0/977181815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034029d50 con 0x56357b4c6890 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204f17d740 1 -- 172.21.15.113:0/977181815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 msgr2=0x56357b4d7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.623+0000 7f204f17d740 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4d7b30 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x56357b4e0b90 tx=0x7f2034017070 comp rx=0 tx=0).stop 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 -- 172.21.15.113:0/977181815 shutdown_connections 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b502540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4d7b30 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 --2- 172.21.15.113:0/977181815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 -- 172.21.15.113:0/977181815 >> 172.21.15.113:0/977181815 conn(0x56357b4c4540 msgr2=0x56357b4c5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 -- 172.21.15.113:0/977181815 shutdown_connections 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 -- 172.21.15.113:0/977181815 wait complete. 2024-04-25T00:07:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 -- start start 2024-04-25T00:07:25.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.624+0000 7f204f17d740 1 Processor -- start 2024-04-25T00:07:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204f17d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204f17d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b4dff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204f17d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56357b3a5750 con 0x56357b4c6890 2024-04-25T00:07:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56357b3fcf00 con 0x56357b4c4c80 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204f17d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56357b3b8cf0 con 0x56357b4e1b60 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204f17d740 10 client.0 _ll_register_callbacks cb 0x7f2040082a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4e13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37522/0 (socket says 172.21.15.113:37522) 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 -- 172.21.15.113:0/737302367 learned_addr learned my addr 172.21.15.113:0/737302367 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204a12c640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 -- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 msgr2=0x56357b4dff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204a92d640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b4dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b4dff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 -- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 msgr2=0x56357b4e0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4e0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204992b640 1 -- 172.21.15.113:0/737302367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357b4c1b20 con 0x56357b4c6890 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.625+0000 7f204a12c640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4e0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204992b640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4e13d0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f203402a000 tx=0x7f20340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99dcddf47fa613a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f20257fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f20257fa640 1 -- 172.21.15.113:0/737302367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203401b070 con 0x56357b4c6890 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f20257fa640 1 -- 172.21.15.113:0/737302367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f203403baf0 con 0x56357b4c6890 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2034019070 con 0x56357b4c6890 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f20257fa640 1 -- 172.21.15.113:0/737302367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2034042050 con 0x56357b4c6890 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 10 client.9657 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x56357b5023f0 con 0x56357b4c6890 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f20257fa640 1 -- 172.21.15.113:0/737302367 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2034019070 con 0x56357b4c6890 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f20257fa640 1 -- 172.21.15.113:0/737302367 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f2034045020 con 0x56357b4c6890 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 10 client.9657 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 10 client.9657 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 20 client.9657 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4ed700 0x56357b4edae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 4 client.9657 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "status"}) -- 0x56357b404790 con 0x56357b4ed700 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f204a12c640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4ed700 0x56357b4edae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.626+0000 7f20257fa640 10 client.9657 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204a12c640 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4ed700 0x56357b4edae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cc59d3970b8252 server_cookie=7fe22e05b4f02b28 in_seq=0 out_seq=0 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f20257fa640 1 -- 172.21.15.113:0/737302367 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56357b404790 con 0x56357b4ed700 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f20257fa640 10 client.9657 handle_command_reply: tid=0 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 client.9657 shutdown 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 msgr2=0x56357b4e13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4e13d0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f203402a000 tx=0x7f20340046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 shutdown_connections 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56357b4ed700 0x56357b4edae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56357b4e1b60 0x56357b4dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56357b4c6890 0x56357b4e13d0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 --2- 172.21.15.113:0/737302367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56357b4c4c80 0x56357b4e0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f20257fa640 0 client.9657 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 >> 172.21.15.113:0/737302367 conn(0x56357b4c4540 msgr2=0x56357b495230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 shutdown_connections 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 1 -- 172.21.15.113:0/737302367 wait complete. 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.627+0000 7f204f17d740 20 client.9657 trim_cache size 0 max 16384 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:07:25.636 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 9501, 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.6284056060000003, 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 78, 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 144, 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 144, 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 9.686405895 2024-04-25T00:07:25.637 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f202806be60 msgr2=0x7f202806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f202806be60 0x7f202806e320 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f20440720d0 tx=0x7f2034009190 comp rx=0 tx=0).stop 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 msgr2=0x7f20440c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f20440c9d30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f204000aa40 tx=0x7f20440d2fd0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 shutdown_connections 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f202806be60 0x7f202806e320 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f204414fe20 0x7f2044151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20440d4200 0x7f20440d4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 --2- 172.21.15.113:0/2181057491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20300039b0 0x7f20440c9d30 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 >> 172.21.15.113:0/2181057491 conn(0x7f204406b720 msgr2=0x7f20440d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 shutdown_connections 2024-04-25T00:07:25.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.628+0000 7f204c390640 1 -- 172.21.15.113:0/2181057491 wait complete. 2024-04-25T00:07:25.640 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:07:25.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.736+0000 7f086da9e640 1 Processor -- start 2024-04-25T00:07:25.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.736+0000 7f086da9e640 1 -- start start 2024-04-25T00:07:25.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f086da9e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0868144910 0x7f08680b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f086da9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0868144cf0 con 0x7f08680b56b0 2024-04-25T00:07:25.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f086da9e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08680b2f30 con 0x7f08680b5240 2024-04-25T00:07:25.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f086da9e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f08680b4510 con 0x7f0868144910 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f0867fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f08680b56b0 0x7f08680cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56606/0 (socket says 172.21.15.113:56606) 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f0867fff640 1 -- 172.21.15.113:0/3183082977 learned_addr learned my addr 172.21.15.113:0/3183082977 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08677fe640 1 --2- 172.21.15.113:0/3183082977 >> v2:172.21.15.178:3300/0 conn(0x7f0868144910 0x7f08680b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 570590784 0 0) 0x7f0868144cf0 con 0x7f08680b56b0 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0850003520 con 0x7f08680b56b0 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3496295525 0 0) 0x7f0850003520 con 0x7f08680b56b0 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 >> v2:172.21.15.178:3300/0 conn(0x7f0868144910 msgr2=0x7f08680b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 --2- 172.21.15.113:0/3183082977 >> v2:172.21.15.178:3300/0 conn(0x7f0868144910 0x7f08680b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 >> v1:172.21.15.178:6789/0 conn(0x7f08680b5240 legacy=0x7f086806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08680b3210 con 0x7f08680b56b0 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f08600019f0 con 0x7f08680b56b0 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 >> v1:172.21.15.113:6789/0 conn(0x7f08680b56b0 legacy=0x7f08680cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.737+0000 7f08667fc640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0850004800 0x7f0850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08667fc640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08500051c0 0x7f085000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f08680b4510 con 0x7f08500039b0 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f085000c090 con 0x7f0850004800 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f085000c290 con 0x7f08500051c0 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08677fe640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f0866ffd640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0850004800 0x7f0850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f0867fff640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08500051c0 0x7f085000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08677fe640 1 -- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08500051c0 msgr2=0x7f085000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08677fe640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08500051c0 0x7f085000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08677fe640 1 -- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0850004800 msgr2=0x7f0850004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08677fe640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0850004800 0x7f0850004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f0867fff640 1 -- 172.21.15.113:0/3183082977 reap_dead start 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08677fe640 1 -- 172.21.15.113:0/3183082977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0850003520 con 0x7f08500039b0 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f0866ffd640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0850004800 0x7f0850004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.738+0000 7f08677fe640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f0858009ac0 tx=0x7f0850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329dcd5661b155b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858036070 con 0x7f08500039b0 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858031400 con 0x7f08500039b0 2024-04-25T00:07:25.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f08667fc640 1 -- 172.21.15.113:0/3183082977 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858009690 con 0x7f08500039b0 2024-04-25T00:07:25.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f086da9e640 1 -- 172.21.15.113:0/3183082977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08500039b0 msgr2=0x7f0850003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f086da9e640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f0858009ac0 tx=0x7f0850005830 comp rx=0 tx=0).stop 2024-04-25T00:07:25.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f086da9e640 1 -- 172.21.15.113:0/3183082977 shutdown_connections 2024-04-25T00:07:25.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f086da9e640 1 --2- 172.21.15.113:0/3183082977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08500039b0 0x7f0850003dd0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f086da9e640 1 -- 172.21.15.113:0/3183082977 >> 172.21.15.113:0/3183082977 conn(0x7f08680b1a50 msgr2=0x7f08680b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f086da9e640 1 -- 172.21.15.113:0/3183082977 shutdown_connections 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.739+0000 7f086da9e640 1 -- 172.21.15.113:0/3183082977 wait complete. 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.740+0000 7f086da9e640 1 Processor -- start 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f086da9e640 1 -- start start 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f086da9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08500039b0 0x7f0868139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f086da9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086815d1d0 0x7f086816a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f086da9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086816ad30 0x7f0868138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f086da9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f08680b4da0 con 0x7f086815d1d0 2024-04-25T00:07:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f086da9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08680b37d0 con 0x7f08500039b0 2024-04-25T00:07:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f086da9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08680b4f20 con 0x7f086816ad30 2024-04-25T00:07:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f0866ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086815d1d0 0x7f086816a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f0866ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086815d1d0 0x7f086816a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37542/0 (socket says 172.21.15.113:37542) 2024-04-25T00:07:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f0866ffd640 1 -- 172.21.15.113:0/2405983316 learned_addr learned my addr 172.21.15.113:0/2405983316 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f0866ffd640 1 -- 172.21.15.113:0/2405983316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086816ad30 msgr2=0x7f0868138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f0866ffd640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086816ad30 0x7f0868138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f0866ffd640 1 -- 172.21.15.113:0/2405983316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08500039b0 msgr2=0x7f0868139680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.741+0000 7f0866ffd640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08500039b0 0x7f0868139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.742+0000 7f0866ffd640 1 -- 172.21.15.113:0/2405983316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086813b3a0 con 0x7f086815d1d0 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.742+0000 7f0866ffd640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086815d1d0 0x7f086816a7f0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e480 tx=0x7f085c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.742+0000 7f084ffff640 1 -- 172.21.15.113:0/2405983316 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c0052e0 con 0x7f086815d1d0 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.742+0000 7f084ffff640 1 -- 172.21.15.113:0/2405983316 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c007020 con 0x7f086815d1d0 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.742+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086813b640 con 0x7f086815d1d0 2024-04-25T00:07:25.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.742+0000 7f084ffff640 1 -- 172.21.15.113:0/2405983316 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c007210 con 0x7f086815d1d0 2024-04-25T00:07:25.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.742+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0868139c60 con 0x7f086815d1d0 2024-04-25T00:07:25.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.743+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08680cce00 con 0x7f086815d1d0 2024-04-25T00:07:25.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.743+0000 7f084ffff640 1 -- 172.21.15.113:0/2405983316 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f085c01a020 con 0x7f086815d1d0 2024-04-25T00:07:25.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.744+0000 7f084ffff640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f085406be30 0x7f085406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:25.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.744+0000 7f084ffff640 1 -- 172.21.15.113:0/2405983316 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f085c08a3a0 con 0x7f086815d1d0 2024-04-25T00:07:25.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.744+0000 7f08677fe640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f085406be30 0x7f085406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:25.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.744+0000 7f08677fe640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f085406be30 0x7f085406e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0858033000 tx=0x7f085802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:25.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.748+0000 7f084ffff640 1 -- 172.21.15.113:0/2405983316 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f08680cce00 con 0x7f086815d1d0 2024-04-25T00:07:25.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.958+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f08680691a0 con 0x7f086815d1d0 2024-04-25T00:07:25.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.959+0000 7f084ffff640 1 -- 172.21.15.113:0/2405983316 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) ==== 53+0+73 (secure 0 0 0) 0x7f085c058de0 con 0x7f086815d1d0 2024-04-25T00:07:25.959 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:07:25.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f085406be30 msgr2=0x7f085406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f085406be30 0x7f085406e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0858033000 tx=0x7f085802e000 comp rx=0 tx=0).stop 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086815d1d0 msgr2=0x7f086816a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086815d1d0 0x7f086816a7f0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e480 tx=0x7f085c00e940 comp rx=0 tx=0).stop 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 shutdown_connections 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f085406be30 0x7f085406e2f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086816ad30 0x7f0868138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086815d1d0 0x7f086816a7f0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 --2- 172.21.15.113:0/2405983316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08500039b0 0x7f0868139680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 >> 172.21.15.113:0/2405983316 conn(0x7f08680b1a50 msgr2=0x7f08680b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 shutdown_connections 2024-04-25T00:07:25.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:25.961+0000 7f086da9e640 1 -- 172.21.15.113:0/2405983316 wait complete. 2024-04-25T00:07:25.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:07:25.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:07:25.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:07:25.974 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:07:25.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:07:25.974 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:07:25.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:07:25.975 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:07:25.975 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:07:25.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:25.976 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:07:25.991 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42234sec preferred_lft 42234sec 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:07:25.992 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:07:25.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:25.993 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:07:26.053 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:07:26.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:26.055 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:07:26.111 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:07:26.111 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:07:26.111 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:07:26.111 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:07:26.111 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:07:26.112 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:07:26.112 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:07:26.112 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:07:26.112 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:07:26.112 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:07:26.112 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:07:26.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:07:26.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:07:26.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:26.235 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:07:26.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:07:26.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:07:26.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:07:26.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:26.250 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:07:26.276 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:07:26.276+0000 7fcbd5bfe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:07:26.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:07:26.299 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:07:26.354 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:07:26.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:07:26.354 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:07:26.409 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:07:26.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:26.409 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:07:26.490 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:07:26.490 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:07:26.490 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:07:26.498 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:07:26.498 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:07:26.498 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:07:26.498 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:07:26.498 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:07:26.502 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:07:26.502 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:07:26.503 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:07:26.504 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:07:26.505 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:07:26.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:26.505 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:07:26.569 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:07:26.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:07:26.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:07:26.669 INFO:teuthology.orchestra.run.smithi113.stdout:client9666 2024-04-25T00:07:26.669 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:26.670 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:26.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.10:0 damage ls 2024-04-25T00:07:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.823+0000 7f1b402a7640 1 Processor -- start 2024-04-25T00:07:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.823+0000 7f1b402a7640 1 -- start start 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b402a7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b380d06c0 0x7f1b3815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b402a7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b381604d0 0x7f1b380cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b402a7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3815d6b0 con 0x7f1b3813f050 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b402a7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3815c6a0 con 0x7f1b381604d0 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b402a7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3815c8f0 con 0x7f1b380d06c0 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3e844640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b381604d0 0x7f1b380cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3e844640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b381604d0 0x7f1b380cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37554/0 (socket says 172.21.15.113:37554) 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3e844640 1 -- 172.21.15.113:0/3612108766 learned_addr learned my addr 172.21.15.113:0/3612108766 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3612108766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b380d06c0 0x7f1b3815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3e844640 1 -- 172.21.15.113:0/3612108766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b380d06c0 msgr2=0x7f1b3815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3e844640 1 --2- 172.21.15.113:0/3612108766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b380d06c0 0x7f1b3815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3e844640 1 -- 172.21.15.113:0/3612108766 >> v1:172.21.15.178:6789/0 conn(0x7f1b3813f050 legacy=0x7f1b380cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.824+0000 7f1b3e844640 1 -- 172.21.15.113:0/3612108766 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3815cbd0 con 0x7f1b381604d0 2024-04-25T00:07:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.825+0000 7f1b3e844640 1 --2- 172.21.15.113:0/3612108766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b381604d0 0x7f1b380cea60 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009870 tx=0x7f1b3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d5b4432b070b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.825+0000 7f1b3d041640 1 -- 172.21.15.113:0/3612108766 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30037070 con 0x7f1b381604d0 2024-04-25T00:07:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.825+0000 7f1b3d041640 1 -- 172.21.15.113:0/3612108766 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b30032440 con 0x7f1b381604d0 2024-04-25T00:07:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.825+0000 7f1b3d041640 1 -- 172.21.15.113:0/3612108766 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30032810 con 0x7f1b381604d0 2024-04-25T00:07:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 -- 172.21.15.113:0/3612108766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b381604d0 msgr2=0x7f1b380cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3612108766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b381604d0 0x7f1b380cea60 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009870 tx=0x7f1b3002a180 comp rx=0 tx=0).stop 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 -- 172.21.15.113:0/3612108766 shutdown_connections 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3612108766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b380d06c0 0x7f1b3815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3612108766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b381604d0 0x7f1b380cea60 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 -- 172.21.15.113:0/3612108766 >> 172.21.15.113:0/3612108766 conn(0x7f1b3806b0f0 msgr2=0x7f1b3806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 -- 172.21.15.113:0/3612108766 shutdown_connections 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.826+0000 7f1b402a7640 1 -- 172.21.15.113:0/3612108766 wait complete. 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.827+0000 7f1b402a7640 1 Processor -- start 2024-04-25T00:07:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.827+0000 7f1b402a7640 1 -- start start 2024-04-25T00:07:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b402a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b380d06c0 0x7f1b381e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b402a7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b3813f050 0x7f1b38072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b402a7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b381604d0 0x7f1b3815b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b402a7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3813f430 con 0x7f1b3813f050 2024-04-25T00:07:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b402a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3815d190 con 0x7f1b380d06c0 2024-04-25T00:07:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b402a7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b381608d0 con 0x7f1b381604d0 2024-04-25T00:07:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b3813f050 0x7f1b38072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b3813f050 0x7f1b38072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37558/0 (socket says 172.21.15.113:37558) 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 -- 172.21.15.113:0/3490252219 learned_addr learned my addr 172.21.15.113:0/3490252219 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 -- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b381604d0 msgr2=0x7f1b3815b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3e844640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b381604d0 0x7f1b3815b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b381604d0 0x7f1b3815b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b380d06c0 0x7f1b381e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 -- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b380d06c0 msgr2=0x7f1b381e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b380d06c0 0x7f1b381e63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.828+0000 7f1b3d842640 1 -- 172.21.15.113:0/3490252219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b38071040 con 0x7f1b3813f050 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.829+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b3813f050 0x7f1b38072b80 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d7d0 tx=0x7f1b2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.829+0000 7f1b2effd640 1 -- 172.21.15.113:0/3490252219 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800b6c0 con 0x7f1b3813f050 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.829+0000 7f1b2effd640 1 -- 172.21.15.113:0/3490252219 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b28019070 con 0x7f1b3813f050 2024-04-25T00:07:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.829+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b380712e0 con 0x7f1b3813f050 2024-04-25T00:07:26.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.829+0000 7f1b2effd640 1 -- 172.21.15.113:0/3490252219 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800bac0 con 0x7f1b3813f050 2024-04-25T00:07:26.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.829+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b380718a0 con 0x7f1b3813f050 2024-04-25T00:07:26.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.830+0000 7f1b2effd640 1 -- 172.21.15.113:0/3490252219 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b2800bc60 con 0x7f1b3813f050 2024-04-25T00:07:26.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.830+0000 7f1b2effd640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.831+0000 7f1b2effd640 1 -- 172.21.15.113:0/3490252219 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f1b2808b2e0 con 0x7f1b3813f050 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.831+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.831+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007fd0 tx=0x7f1b34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.832+0000 7f1b42fb0740 1 Processor -- start 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.832+0000 7f1b42fb0740 1 -- start start 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405990950 0x55d4059432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d405972df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405973330 0x55d405948140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d4058ac670 con 0x55d405990950 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d405854970 con 0x55d405973330 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d4058a8630 con 0x55d405973a70 2024-04-25T00:07:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405990950 0x55d4059432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405990950 0x55d4059432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37566/0 (socket says 172.21.15.113:37566) 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 -- 172.21.15.113:0/923603250 learned_addr learned my addr 172.21.15.113:0/923603250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3d842640 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d405972df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e844640 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405973330 0x55d405948140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 -- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 msgr2=0x55d405972df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d405972df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 -- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405973330 msgr2=0x55d405948140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405973330 0x55d405948140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 -- 172.21.15.113:0/923603250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d405878400 con 0x55d405990950 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b3e043640 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405990950 0x55d4059432e0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400dc80 tx=0x7f1b34010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e055334c75728b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b1bfff640 1 -- 172.21.15.113:0/923603250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34024070 con 0x55d405990950 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b1bfff640 1 -- 172.21.15.113:0/923603250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34004b10 con 0x55d405990950 2024-04-25T00:07:26.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.833+0000 7f1b1bfff640 1 -- 172.21.15.113:0/923603250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34005050 con 0x55d405990950 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 -- 172.21.15.113:0/923603250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405990950 msgr2=0x55d4059432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405990950 0x55d4059432e0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400dc80 tx=0x7f1b34010000 comp rx=0 tx=0).stop 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 -- 172.21.15.113:0/923603250 shutdown_connections 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405973330 0x55d405948140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d405972df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/923603250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405990950 0x55d4059432e0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 -- 172.21.15.113:0/923603250 >> 172.21.15.113:0/923603250 conn(0x55d405945550 msgr2=0x55d40598ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 -- 172.21.15.113:0/923603250 shutdown_connections 2024-04-25T00:07:26.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.834+0000 7f1b42fb0740 1 -- 172.21.15.113:0/923603250 wait complete. 2024-04-25T00:07:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.835+0000 7f1b42fb0740 1 -- start start 2024-04-25T00:07:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.835+0000 7f1b42fb0740 1 Processor -- start 2024-04-25T00:07:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d40598f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405990950 0x55d40599d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d40596e8a0 0x55d4059767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d405947700 con 0x55d40596e8a0 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d405868050 con 0x55d405990950 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d405946af0 con 0x55d405973a70 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 10 client.0 _ll_register_callbacks cb 0x55d4055bfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d40596e8a0 0x55d4059767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d40596e8a0 0x55d4059767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37572/0 (socket says 172.21.15.113:37572) 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 -- 172.21.15.113:0/3689710449 learned_addr learned my addr 172.21.15.113:0/3689710449 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d40598f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405990950 0x55d40599d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 -- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 msgr2=0x55d40598f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d40598f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 -- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405990950 msgr2=0x55d40599d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405990950 0x55d40599d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 -- 172.21.15.113:0/3689710449 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d405839dc0 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405990950 0x55d40599d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b3e844640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d40596e8a0 0x55d4059767f0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009870 tx=0x7f1b30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f2130535b5bbbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b197fa640 1 -- 172.21.15.113:0/3689710449 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30037040 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b197fa640 1 -- 172.21.15.113:0/3689710449 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f1b3003c640 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.836+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b34004b10 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b197fa640 1 -- 172.21.15.113:0/3689710449 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f1b3004d050 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 10 client.9678 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55d4055c03f0 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b197fa640 1 -- 172.21.15.113:0/3689710449 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34004b10 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b197fa640 1 -- 172.21.15.113:0/3689710449 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f1b30050020 con 0x55d40596e8a0 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 10 client.9678 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:26.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 10 client.9678 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 20 client.9678 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d405963660 0x55d4059ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 4 client.9678 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4058b9bb0 con 0x55d405963660 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d405963660 0x55d4059ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d405963660 0x55d4059ab110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5664cea78ee4d222 server_cookie=ac11f7ed6f12f0d8 in_seq=0 out_seq=0 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.837+0000 7f1b197fa640 10 client.9678 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b197fa640 1 -- 172.21.15.113:0/3689710449 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4058b9bb0 con 0x55d405963660 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b197fa640 10 client.9678 handle_command_reply: tid=0 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 client.9678 shutdown 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d40596e8a0 msgr2=0x55d4059767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d40596e8a0 0x55d4059767f0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009870 tx=0x7f1b30004730 comp rx=0 tx=0).stop 2024-04-25T00:07:26.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 shutdown_connections 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d405963660 0x55d4059ab110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d405990950 0x55d40599d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d40596e8a0 0x55d4059767f0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3689710449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405973a70 0x55d40598f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b197fa640 0 client.9678 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 >> 172.21.15.113:0/3689710449 conn(0x55d405973330 msgr2=0x55d405947b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 shutdown_connections 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3689710449 wait complete. 2024-04-25T00:07:26.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.838+0000 7f1b42fb0740 20 client.9678 trim_cache size 0 max 16384 2024-04-25T00:07:26.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.851+0000 7f1b42fb0740 1 Processor -- start 2024-04-25T00:07:26.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.851+0000 7f1b42fb0740 1 -- start start 2024-04-25T00:07:26.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d40598feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405991210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 0x55d405991750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38200800 con 0x55d405973a70 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381f5570 con 0x55d40596e8a0 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b38155f90 con 0x55d405990950 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405991210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405991210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37574/0 (socket says 172.21.15.113:37574) 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 -- 172.21.15.113:0/95672996 learned_addr learned my addr 172.21.15.113:0/95672996 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 -- 172.21.15.113:0/95672996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 msgr2=0x55d405991750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 --2- 172.21.15.113:0/95672996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 0x55d405991750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 -- 172.21.15.113:0/95672996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 msgr2=0x55d40598feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 --2- 172.21.15.113:0/95672996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d40598feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 -- 172.21.15.113:0/95672996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40568a7d0 con 0x55d405973a70 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b3d842640 1 --2- 172.21.15.113:0/95672996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405991210 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800b4b0 tx=0x7f1b28059d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc68d7975cce179 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.852+0000 7f1b1b7fe640 1 -- 172.21.15.113:0/95672996 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28084020 con 0x55d405973a70 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b1b7fe640 1 -- 172.21.15.113:0/95672996 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b28062170 con 0x55d405973a70 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b1b7fe640 1 -- 172.21.15.113:0/95672996 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28071070 con 0x55d405973a70 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 -- 172.21.15.113:0/95672996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 msgr2=0x55d405991210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/95672996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405991210 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800b4b0 tx=0x7f1b28059d50 comp rx=0 tx=0).stop 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 -- 172.21.15.113:0/95672996 shutdown_connections 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/95672996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 0x55d405991750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/95672996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405991210 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/95672996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d40598feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 -- 172.21.15.113:0/95672996 >> 172.21.15.113:0/95672996 conn(0x55d405973330 msgr2=0x55d405944020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 -- 172.21.15.113:0/95672996 shutdown_connections 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.853+0000 7f1b42fb0740 1 -- 172.21.15.113:0/95672996 wait complete. 2024-04-25T00:07:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.854+0000 7f1b42fb0740 1 -- start start 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.854+0000 7f1b42fb0740 1 Processor -- start 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d405a068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405a0aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 0x55d405a0af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d405976b40 con 0x55d405973a70 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381f4f20 con 0x55d40596e8a0 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d405863ad0 con 0x55d405990950 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 10 client.0 _ll_register_callbacks cb 0x7f1b30035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405a0aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405a0aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37578/0 (socket says 172.21.15.113:37578) 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 -- 172.21.15.113:0/3827582430 learned_addr learned my addr 172.21.15.113:0/3827582430 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d405a068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3e844640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 0x55d405a0af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 -- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 msgr2=0x55d405a0af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 0x55d405a0af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 -- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 msgr2=0x55d405a068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d405a068c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 -- 172.21.15.113:0/3827582430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4059a7150 con 0x55d405973a70 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d405a068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b3d842640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405a0aa30 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f1b28059290 tx=0x7f1b28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1a01a8ce600f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b197fa640 1 -- 172.21.15.113:0/3827582430 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28084030 con 0x55d405973a70 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b197fa640 1 -- 172.21.15.113:0/3827582430 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f1b28089420 con 0x55d405973a70 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b28062170 con 0x55d405973a70 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b197fa640 1 -- 172.21.15.113:0/3827582430 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f1b28071040 con 0x55d405973a70 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 10 client.9684 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:26.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.855+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55d40598fad0 con 0x55d405973a70 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b197fa640 1 -- 172.21.15.113:0/3827582430 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b28062170 con 0x55d405973a70 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b197fa640 1 -- 172.21.15.113:0/3827582430 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f1b28013070 con 0x55d405973a70 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 10 client.9684 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 10 client.9684 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 20 client.9684 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d40598ff40 0x55d40598b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 4 client.9684 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4058b3f00 con 0x55d40598ff40 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d40598ff40 0x55d40598b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b197fa640 10 client.9684 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b3e043640 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d40598ff40 0x55d40598b5d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42bf23741a65e698 server_cookie=3d7851b65698c49b in_seq=0 out_seq=0 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b197fa640 1 -- 172.21.15.113:0/3827582430 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4058b3f00 con 0x55d40598ff40 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b197fa640 10 client.9684 handle_command_reply: tid=0 2024-04-25T00:07:26.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 client.9684 shutdown 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 msgr2=0x55d405a0aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405a0aa30 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f1b28059290 tx=0x7f1b28000bc0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 shutdown_connections 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55d40598ff40 0x55d40598b5d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d405990950 0x55d405a0af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d405973a70 0x55d405a0aa30 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b197fa640 0 client.9684 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 --2- 172.21.15.113:0/3827582430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d40596e8a0 0x55d405a068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 >> 172.21.15.113:0/3827582430 conn(0x55d405973330 msgr2=0x55d405991210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.856+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 shutdown_connections 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b42fb0740 1 -- 172.21.15.113:0/3827582430 wait complete. 2024-04-25T00:07:26.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b42fb0740 20 client.9684 trim_cache size 0 max 16384 2024-04-25T00:07:26.866 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1c06be30 msgr2=0x7f1b1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007fd0 tx=0x7f1b34009040 comp rx=0 tx=0).stop 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b3813f050 msgr2=0x7f1b38072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b3813f050 0x7f1b38072b80 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d7d0 tx=0x7f1b2800dc90 comp rx=0 tx=0).stop 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 shutdown_connections 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1b1c06be30 0x7f1b1c06e2f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b381604d0 0x7f1b3815b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b3813f050 0x7f1b38072b80 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 --2- 172.21.15.113:0/3490252219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b380d06c0 0x7f1b381e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 >> 172.21.15.113:0/3490252219 conn(0x7f1b3806b0f0 msgr2=0x7f1b380cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 shutdown_connections 2024-04-25T00:07:26.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.857+0000 7f1b402a7640 1 -- 172.21.15.113:0/3490252219 wait complete. 2024-04-25T00:07:26.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T00:07:26.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.963+0000 7f849f62a640 1 Processor -- start 2024-04-25T00:07:26.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.963+0000 7f849f62a640 1 -- start start 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.964+0000 7f849f62a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84980d2290 0x7f84980b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.964+0000 7f849f62a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 0x7f84980d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.964+0000 7f849f62a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84980d2800 con 0x7f849815e320 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.964+0000 7f849f62a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f84980d14d0 con 0x7f84980d2290 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.964+0000 7f849f62a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84980d3410 con 0x7f84980d0b00 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.964+0000 7f849d3c6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f849815e320 0x7f84980d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56618/0 (socket says 172.21.15.113:56618) 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.964+0000 7f849d3c6640 1 -- 172.21.15.113:0/3946589860 learned_addr learned my addr 172.21.15.113:0/3946589860 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:26.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849cbc5640 1 --2- 172.21.15.113:0/3946589860 >> v2:172.21.15.178:3300/0 conn(0x7f84980d2290 0x7f84980b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 0x7f84980d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3339699238 0 0) 0x7f84980d2800 con 0x7f849815e320 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8484003520 con 0x7f849815e320 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1555544824 0 0) 0x7f8484003520 con 0x7f849815e320 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 msgr2=0x7f84980d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 0x7f84980d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 >> v2:172.21.15.178:3300/0 conn(0x7f84980d2290 msgr2=0x7f84980b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 --2- 172.21.15.113:0/3946589860 >> v2:172.21.15.178:3300/0 conn(0x7f84980d2290 0x7f84980b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84980d2ae0 con 0x7f849815e320 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8494001a10 con 0x7f849815e320 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 >> v1:172.21.15.113:6789/0 conn(0x7f849815e320 legacy=0x7f84980d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 0x7f84840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8484003f30 0x7f8484004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8484004d40 0x7f84840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f84980d14d0 con 0x7f8484004d40 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84980d3410 con 0x7f8484003f30 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f848400b9f0 con 0x7f84980d0b00 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8484004d40 0x7f84840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849d3c6640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 0x7f84840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849cbc5640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8484003f30 0x7f8484004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849dbc7640 1 -- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 msgr2=0x7f84840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980d0b00 0x7f84840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849dbc7640 1 -- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8484003f30 msgr2=0x7f8484004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8484003f30 0x7f8484004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.965+0000 7f849dbc7640 1 -- 172.21.15.113:0/3946589860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8484003520 con 0x7f8484004d40 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.966+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8484004d40 0x7f84840051c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8490009ac0 tx=0x7f8484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359e171f658887e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.966+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490036070 con 0x7f8484004d40 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.966+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490031400 con 0x7f8484004d40 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.966+0000 7f848ffff640 1 -- 172.21.15.113:0/3946589860 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490009690 con 0x7f8484004d40 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.967+0000 7f849f62a640 1 -- 172.21.15.113:0/3946589860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8484004d40 msgr2=0x7f84840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.967+0000 7f849f62a640 1 --2- 172.21.15.113:0/3946589860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8484004d40 0x7f84840051c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8490009ac0 tx=0x7f8484005830 comp rx=0 tx=0).stop 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.967+0000 7f849dbc7640 1 -- 172.21.15.113:0/3946589860 reap_dead start 2024-04-25T00:07:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.967+0000 7f849f62a640 1 -- 172.21.15.113:0/3946589860 shutdown_connections 2024-04-25T00:07:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.967+0000 7f849f62a640 1 -- 172.21.15.113:0/3946589860 >> 172.21.15.113:0/3946589860 conn(0x7f84980b5c20 msgr2=0x7f84980b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.967+0000 7f849f62a640 1 -- 172.21.15.113:0/3946589860 shutdown_connections 2024-04-25T00:07:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.967+0000 7f849f62a640 1 -- 172.21.15.113:0/3946589860 wait complete. 2024-04-25T00:07:26.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.968+0000 7f849f62a640 1 Processor -- start 2024-04-25T00:07:26.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.968+0000 7f849f62a640 1 -- start start 2024-04-25T00:07:26.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849f62a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84980c9c20 0x7f84981f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849f62a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980b7f30 0x7f84980b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849f62a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849815a300 0x7f849815a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849f62a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f84980b9fe0 con 0x7f849815a300 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849f62a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84980d30a0 con 0x7f84980c9c20 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849f62a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f849815e700 con 0x7f84980b7f30 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849815a300 0x7f849815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849815a300 0x7f849815a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37600/0 (socket says 172.21.15.113:37600) 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 -- 172.21.15.113:0/3060792417 learned_addr learned my addr 172.21.15.113:0/3060792417 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849cbc5640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980b7f30 0x7f84980b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849d3c6640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84980c9c20 0x7f84981f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 -- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980b7f30 msgr2=0x7f84980b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980b7f30 0x7f84980b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 -- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84980c9c20 msgr2=0x7f84981f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84980c9c20 0x7f84981f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 -- 172.21.15.113:0/3060792417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84981f4e70 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849dbc7640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849815a300 0x7f849815a780 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8490034040 tx=0x7f8490004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f848dffb640 1 -- 172.21.15.113:0/3060792417 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490041070 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f848dffb640 1 -- 172.21.15.113:0/3060792417 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490037040 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f848dffb640 1 -- 172.21.15.113:0/3060792417 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490036070 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.969+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8498158c10 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.970+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8498158410 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.971+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84980691a0 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.971+0000 7f848dffb640 1 -- 172.21.15.113:0/3060792417 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8490044070 con 0x7f849815a300 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.971+0000 7f848dffb640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f847c06be30 0x7f847c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:26.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.971+0000 7f848dffb640 1 -- 172.21.15.113:0/3060792417 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f84900aa300 con 0x7f849815a300 2024-04-25T00:07:26.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.972+0000 7f849d3c6640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f847c06be30 0x7f847c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:26.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.972+0000 7f849d3c6640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f847c06be30 0x7f847c06e2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8494004bc0 tx=0x7f8494006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:26.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:26.976+0000 7f848dffb640 1 -- 172.21.15.113:0/3060792417 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8490078690 con 0x7f849815a300 2024-04-25T00:07:27.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.187+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f84980d1290 con 0x7f849815a300 2024-04-25T00:07:27.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.187+0000 7f848dffb640 1 -- 172.21.15.113:0/3060792417 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f849007c990 con 0x7f849815a300 2024-04-25T00:07:27.187 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:27.188 INFO:teuthology.orchestra.run.smithi113.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:07:27.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f847c06be30 msgr2=0x7f847c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f847c06be30 0x7f847c06e2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8494004bc0 tx=0x7f8494006040 comp rx=0 tx=0).stop 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849815a300 msgr2=0x7f849815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849815a300 0x7f849815a780 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f8490034040 tx=0x7f8490004730 comp rx=0 tx=0).stop 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 shutdown_connections 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f847c06be30 0x7f847c06e2f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849815a300 0x7f849815a780 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84980b7f30 0x7f84980b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 --2- 172.21.15.113:0/3060792417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84980c9c20 0x7f84981f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 >> 172.21.15.113:0/3060792417 conn(0x7f84980b5c20 msgr2=0x7f849815d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 shutdown_connections 2024-04-25T00:07:27.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.189+0000 7f849f62a640 1 -- 172.21.15.113:0/3060792417 wait complete. 2024-04-25T00:07:27.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:27.200 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub start '~mdsdir' recursive 2024-04-25T00:07:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.301+0000 7fa31b891640 1 Processor -- start 2024-04-25T00:07:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.301+0000 7fa31b891640 1 -- start start 2024-04-25T00:07:27.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.302+0000 7fa31b891640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa3140cdd40 0x7fa31406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.302+0000 7fa31b891640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa314072f70 con 0x7fa31406e0d0 2024-04-25T00:07:27.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.302+0000 7fa31b891640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3140b2fd0 con 0x7fa314144a80 2024-04-25T00:07:27.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.302+0000 7fa31b891640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa31406ea70 con 0x7fa3140cdd40 2024-04-25T00:07:27.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.302+0000 7fa318e2c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa31406e0d0 0x7fa3140cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56634/0 (socket says 172.21.15.113:56634) 2024-04-25T00:07:27.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.302+0000 7fa318e2c640 1 -- 172.21.15.113:0/2490374367 learned_addr learned my addr 172.21.15.113:0/2490374367 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.302+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434398082 0 0) 0x7fa314072f70 con 0x7fa31406e0d0 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa300005730 con 0x7fa31406e0d0 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2490374367 >> v2:172.21.15.178:3300/0 conn(0x7fa3140cdd40 0x7fa31406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 685801763 0 0) 0x7fa300005730 con 0x7fa31406e0d0 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 >> v2:172.21.15.178:3300/0 conn(0x7fa3140cdd40 msgr2=0x7fa31406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 --2- 172.21.15.113:0/2490374367 >> v2:172.21.15.178:3300/0 conn(0x7fa3140cdd40 0x7fa31406e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 >> v1:172.21.15.178:6789/0 conn(0x7fa314144a80 legacy=0x7fa3140b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3140b32b0 con 0x7fa31406e0d0 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fa304001670 con 0x7fa31406e0d0 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 >> v1:172.21.15.113:6789/0 conn(0x7fa31406e0d0 legacy=0x7fa3140cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa300005bc0 0x7fa300005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa300006a10 0x7fa300006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa30000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa30000e340 con 0x7fa3000073d0 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa30000e4f0 con 0x7fa300005bc0 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa30000e6f0 con 0x7fa300006a10 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa30000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa318e2c640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa300006a10 0x7fa300006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa31962d640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa300005bc0 0x7fa300005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa319e2e640 1 -- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa300006a10 msgr2=0x7fa300006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa300006a10 0x7fa300006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa319e2e640 1 -- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa300005bc0 msgr2=0x7fa300005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa300005bc0 0x7fa300005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.303+0000 7fa319e2e640 1 -- 172.21.15.113:0/2490374367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa300005730 con 0x7fa3000073d0 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.304+0000 7fa319e2e640 1 -- 172.21.15.113:0/2490374367 reap_dead start 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.304+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa30000dbd0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009a90 tx=0x7fa300007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9475ba5b2866d321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.304+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c036070 con 0x7fa3000073d0 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.304+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30c031400 con 0x7fa3000073d0 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.304+0000 7fa30bfff640 1 -- 172.21.15.113:0/2490374367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c009690 con 0x7fa3000073d0 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.305+0000 7fa31b891640 1 -- 172.21.15.113:0/2490374367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 msgr2=0x7fa30000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.305+0000 7fa31b891640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa30000dbd0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009a90 tx=0x7fa300007a40 comp rx=0 tx=0).stop 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.305+0000 7fa31b891640 1 -- 172.21.15.113:0/2490374367 shutdown_connections 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.305+0000 7fa31b891640 1 --2- 172.21.15.113:0/2490374367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa30000dbd0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.305+0000 7fa31b891640 1 -- 172.21.15.113:0/2490374367 >> 172.21.15.113:0/2490374367 conn(0x7fa3140b1b30 msgr2=0x7fa3140b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.305+0000 7fa31b891640 1 -- 172.21.15.113:0/2490374367 shutdown_connections 2024-04-25T00:07:27.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.305+0000 7fa31b891640 1 -- 172.21.15.113:0/2490374367 wait complete. 2024-04-25T00:07:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.306+0000 7fa31b891640 1 Processor -- start 2024-04-25T00:07:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.306+0000 7fa31b891640 1 -- start start 2024-04-25T00:07:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31b891640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa3140b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31b891640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3141429e0 0x7fa3140b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31b891640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31415d6e0 0x7fa31415db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31b891640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa31406e6b0 con 0x7fa3000073d0 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31b891640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa31406e530 con 0x7fa31415d6e0 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31b891640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa31406e830 con 0x7fa3141429e0 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa3140b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa3140b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37608/0 (socket says 172.21.15.113:37608) 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 -- 172.21.15.113:0/970826422 learned_addr learned my addr 172.21.15.113:0/970826422 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 -- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3141429e0 msgr2=0x7fa3140b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3141429e0 0x7fa3140b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 -- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31415d6e0 msgr2=0x7fa31415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa319e2e640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31415d6e0 0x7fa31415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31415d6e0 0x7fa31415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 -- 172.21.15.113:0/970826422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa314142f80 con 0x7fa3000073d0 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa319e2e640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31415d6e0 0x7fa31415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:27.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31962d640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa3140b39b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa31000f960 tx=0x7fa31000fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa309ffb640 1 -- 172.21.15.113:0/970826422 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310003b80 con 0x7fa3000073d0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa309ffb640 1 -- 172.21.15.113:0/970826422 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310018070 con 0x7fa3000073d0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa314143260 con 0x7fa3000073d0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.307+0000 7fa309ffb640 1 -- 172.21.15.113:0/970826422 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31000d790 con 0x7fa3000073d0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.308+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa314143820 con 0x7fa3000073d0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.309+0000 7fa309ffb640 1 -- 172.21.15.113:0/970826422 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa31000d930 con 0x7fa3000073d0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.309+0000 7fa309ffb640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2f806be60 0x7fa2f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.309+0000 7fa309ffb640 1 -- 172.21.15.113:0/970826422 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fa31008af10 con 0x7fa3000073d0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.309+0000 7fa318e2c640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2f806be60 0x7fa2f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.310+0000 7fa318e2c640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2f806be60 0x7fa2f806e320 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa304001cd0 tx=0x7fa304001c60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.310+0000 7fa31e595740 1 Processor -- start 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.310+0000 7fa31e595740 1 -- start start 2024-04-25T00:07:27.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 0x5591297357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa31e595740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591296ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591296ec640 0x559129717dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa31e595740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559129650940 con 0x559129713ee0 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5591295f8920 con 0x5591297179f0 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55912964c900 con 0x5591296ec640 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591296ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591296ec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37610/0 (socket says 172.21.15.113:37610) 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 -- 172.21.15.113:0/3502710344 learned_addr learned my addr 172.21.15.113:0/3502710344 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 -- 172.21.15.113:0/3502710344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591296ec640 msgr2=0x559129717dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 --2- 172.21.15.113:0/3502710344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591296ec640 0x559129717dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 -- 172.21.15.113:0/3502710344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 msgr2=0x5591297357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 --2- 172.21.15.113:0/3502710344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 0x5591297357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 -- 172.21.15.113:0/3502710344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559129430a20 con 0x559129713ee0 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.311+0000 7fa318e2c640 1 --2- 172.21.15.113:0/3502710344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591296ec100 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa304004e90 tx=0x7fa304013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852fa19d5f7a628a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa2feffd640 1 -- 172.21.15.113:0/3502710344 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304026070 con 0x559129713ee0 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa2feffd640 1 -- 172.21.15.113:0/3502710344 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa304007530 con 0x559129713ee0 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa2feffd640 1 -- 172.21.15.113:0/3502710344 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304012350 con 0x559129713ee0 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 -- 172.21.15.113:0/3502710344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 msgr2=0x5591296ec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 --2- 172.21.15.113:0/3502710344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591296ec100 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa304004e90 tx=0x7fa304013000 comp rx=0 tx=0).stop 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 -- 172.21.15.113:0/3502710344 shutdown_connections 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 --2- 172.21.15.113:0/3502710344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591296ec640 0x559129717dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 --2- 172.21.15.113:0/3502710344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591296ec100 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 --2- 172.21.15.113:0/3502710344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 0x5591297357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 -- 172.21.15.113:0/3502710344 >> 172.21.15.113:0/3502710344 conn(0x5591296ea150 msgr2=0x559129735f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 -- 172.21.15.113:0/3502710344 shutdown_connections 2024-04-25T00:07:27.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.312+0000 7fa31e595740 1 -- 172.21.15.113:0/3502710344 wait complete. 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.313+0000 7fa31e595740 1 -- start start 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.313+0000 7fa31e595740 1 Processor -- start 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713ee0 0x559129712bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 0x559129716ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713290 0x5591297170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559129736e40 con 0x559129713290 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55912960c030 con 0x5591297179f0 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559129736680 con 0x559129713ee0 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 10 client.0 _ll_register_callbacks cb 0x559129606e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713290 0x5591297170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713290 0x5591297170e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37616/0 (socket says 172.21.15.113:37616) 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 -- 172.21.15.113:0/1647505871 learned_addr learned my addr 172.21.15.113:0/1647505871 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31962d640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713ee0 0x559129712bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa318e2c640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 0x559129716ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 -- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713ee0 msgr2=0x559129712bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713ee0 0x559129712bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 -- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 msgr2=0x559129716ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 0x559129716ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 -- 172.21.15.113:0/1647505871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559129324310 con 0x559129713290 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31962d640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713ee0 0x559129712bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa319e2e640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713290 0x5591297170e0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009bb0 tx=0x7fa30c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caad7baadc71a0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa2f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa2f3fff640 1 -- 172.21.15.113:0/1647505871 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c036040 con 0x559129713290 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa2f3fff640 1 -- 172.21.15.113:0/1647505871 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fa30c031aa0 con 0x559129713290 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa304007530 con 0x559129713290 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa2f3fff640 1 -- 172.21.15.113:0/1647505871 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fa30c04b050 con 0x559129713290 2024-04-25T00:07:27.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.314+0000 7fa31e595740 10 client.9702 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x5591294911b0 con 0x559129713290 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa2f3fff640 1 -- 172.21.15.113:0/1647505871 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa304007530 con 0x559129713290 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa2f3fff640 1 -- 172.21.15.113:0/1647505871 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7fa30c04e020 con 0x559129713290 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31e595740 10 client.9702 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31e595740 10 client.9702 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31e595740 20 client.9702 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31e595740 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591297311f0 0x559129731600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31e595740 4 client.9702 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55912965de80 con 0x5591297311f0 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31962d640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591297311f0 0x559129731600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa2f3fff640 10 client.9702 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.315+0000 7fa31962d640 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591297311f0 0x559129731600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606b5f7f50e942db server_cookie=da8380d574f3eed8 in_seq=0 out_seq=0 2024-04-25T00:07:27.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa2f3fff640 1 -- 172.21.15.113:0/1647505871 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55912965de80 con 0x5591297311f0 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa2f3fff640 10 client.9702 handle_command_reply: tid=0 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 client.9702 shutdown 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713290 msgr2=0x5591297170e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713290 0x5591297170e0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009bb0 tx=0x7fa30c004730 comp rx=0 tx=0).stop 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 shutdown_connections 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591297311f0 0x559129731600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591297179f0 0x559129716ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa2f3fff640 0 client.9702 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713290 0x5591297170e0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 --2- 172.21.15.113:0/1647505871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713ee0 0x559129712bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 >> 172.21.15.113:0/1647505871 conn(0x5591296ec640 msgr2=0x5591297142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 shutdown_connections 2024-04-25T00:07:27.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 1 -- 172.21.15.113:0/1647505871 wait complete. 2024-04-25T00:07:27.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.316+0000 7fa31e595740 20 client.9702 trim_cache size 0 max 16384 2024-04-25T00:07:27.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.333+0000 7fa31e595740 1 Processor -- start 2024-04-25T00:07:27.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.333+0000 7fa31e595740 1 -- start start 2024-04-25T00:07:27.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713290 0x559129731b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa31e595740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591297564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591297179f0 0x559129716d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa31e595740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3141fc890 con 0x559129713ee0 2024-04-25T00:07:27.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3141fb940 con 0x559129713290 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa31415c330 con 0x5591297179f0 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa318e2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591297564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa318e2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591297564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37620/0 (socket says 172.21.15.113:37620) 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa318e2c640 1 -- 172.21.15.113:0/447423683 learned_addr learned my addr 172.21.15.113:0/447423683 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.334+0000 7fa318e2c640 1 -- 172.21.15.113:0/447423683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591297179f0 msgr2=0x559129716d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa318e2c640 1 --2- 172.21.15.113:0/447423683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591297179f0 0x559129716d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa318e2c640 1 -- 172.21.15.113:0/447423683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713290 msgr2=0x559129731b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa318e2c640 1 --2- 172.21.15.113:0/447423683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713290 0x559129731b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa318e2c640 1 -- 172.21.15.113:0/447423683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591292fd490 con 0x559129713ee0 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa318e2c640 1 --2- 172.21.15.113:0/447423683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591297564a0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fa304024040 tx=0x7fa304014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c57f9fb365d67c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa2fe7fc640 1 -- 172.21.15.113:0/447423683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304031070 con 0x559129713ee0 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa2fe7fc640 1 -- 172.21.15.113:0/447423683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa304027040 con 0x559129713ee0 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa2fe7fc640 1 -- 172.21.15.113:0/447423683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304026070 con 0x559129713ee0 2024-04-25T00:07:27.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 -- 172.21.15.113:0/447423683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 msgr2=0x5591297564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 --2- 172.21.15.113:0/447423683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591297564a0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fa304024040 tx=0x7fa304014000 comp rx=0 tx=0).stop 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 -- 172.21.15.113:0/447423683 shutdown_connections 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 --2- 172.21.15.113:0/447423683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591297179f0 0x559129716d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 --2- 172.21.15.113:0/447423683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559129713ee0 0x5591297564a0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 --2- 172.21.15.113:0/447423683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713290 0x559129731b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 -- 172.21.15.113:0/447423683 >> 172.21.15.113:0/447423683 conn(0x5591296ec640 msgr2=0x5591297129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 -- 172.21.15.113:0/447423683 shutdown_connections 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.335+0000 7fa31e595740 1 -- 172.21.15.113:0/447423683 wait complete. 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.336+0000 7fa31e595740 1 -- start start 2024-04-25T00:07:27.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.336+0000 7fa31e595740 1 Processor -- start 2024-04-25T00:07:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713290 0x5591297315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713ee0 0x559129731b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5591297179f0 0x559129715ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559129736d70 con 0x5591297179f0 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa31415c980 con 0x559129713ee0 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559129746160 con 0x559129713290 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 10 client.0 _ll_register_callbacks cb 0x7fa30c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5591297179f0 0x559129715ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5591297179f0 0x559129715ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37622/0 (socket says 172.21.15.113:37622) 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 -- 172.21.15.113:0/2851931787 learned_addr learned my addr 172.21.15.113:0/2851931787 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31962d640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713290 0x5591297315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa318e2c640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713ee0 0x559129731b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 -- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713290 msgr2=0x5591297315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713290 0x5591297315d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 -- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713ee0 msgr2=0x559129731b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713ee0 0x559129731b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 -- 172.21.15.113:0/2851931787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912971b8f0 con 0x5591297179f0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa318e2c640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713ee0 0x559129731b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa319e2e640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5591297179f0 0x559129715ed0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009a90 tx=0x7fa30c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255d4f1b8c749a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa2f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa2f3fff640 1 -- 172.21.15.113:0/2851931787 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c053020 con 0x5591297179f0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa2f3fff640 1 -- 172.21.15.113:0/2851931787 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fa30c005b90 con 0x5591297179f0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.337+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa304027040 con 0x5591297179f0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa2f3fff640 1 -- 172.21.15.113:0/2851931787 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fa30c05a050 con 0x5591297179f0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 10 client.9708 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x5591297b0af0 con 0x5591297179f0 2024-04-25T00:07:27.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa2f3fff640 1 -- 172.21.15.113:0/2851931787 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa304027040 con 0x5591297179f0 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa2f3fff640 1 -- 172.21.15.113:0/2851931787 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7fa30c05d020 con 0x5591297179f0 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 10 client.9708 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 10 client.9708 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 20 client.9708 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591296e87b0 0x559129716410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 4 client.9708 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) -- 0x5591296581d0 con 0x5591296e87b0 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31962d640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591296e87b0 0x559129716410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa2f3fff640 10 client.9708 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.338+0000 7fa31962d640 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591296e87b0 0x559129716410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d9c5a19d14f229b server_cookie=2020dc861a58f4fd in_seq=0 out_seq=0 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa2f3fff640 1 -- 172.21.15.113:0/2851931787 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591296581d0 con 0x5591296e87b0 2024-04-25T00:07:27.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa2f3fff640 10 client.9708 handle_command_reply: tid=0 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 client.9708 shutdown 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5591297179f0 msgr2=0x559129715ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5591297179f0 0x559129715ed0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009a90 tx=0x7fa30c009190 comp rx=0 tx=0).stop 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 shutdown_connections 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x5591296e87b0 0x559129716410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559129713ee0 0x559129731b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5591297179f0 0x559129715ed0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa2f3fff640 0 client.9708 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 --2- 172.21.15.113:0/2851931787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559129713290 0x5591297315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 >> 172.21.15.113:0/2851931787 conn(0x5591296ec640 msgr2=0x5591297129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 shutdown_connections 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 1 -- 172.21.15.113:0/2851931787 wait complete. 2024-04-25T00:07:27.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.339+0000 7fa31e595740 20 client.9708 trim_cache size 0 max 16384 2024-04-25T00:07:27.347 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:07:27.347 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:07:27.347 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "24cee8b1-3586-4928-b3e5-06eb8aae44cb", 2024-04-25T00:07:27.347 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:07:27.347 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:07:27.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2f806be60 msgr2=0x7fa2f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2f806be60 0x7fa2f806e320 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa304001cd0 tx=0x7fa304001c60 comp rx=0 tx=0).stop 2024-04-25T00:07:27.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 msgr2=0x7fa3140b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa3140b39b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa31000f960 tx=0x7fa31000fe20 comp rx=0 tx=0).stop 2024-04-25T00:07:27.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 shutdown_connections 2024-04-25T00:07:27.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2f806be60 0x7fa2f806e320 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31415d6e0 0x7fa31415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3141429e0 0x7fa3140b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 --2- 172.21.15.113:0/970826422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3000073d0 0x7fa3140b39b0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 >> 172.21.15.113:0/970826422 conn(0x7fa3140b1b30 msgr2=0x7fa3140afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 shutdown_connections 2024-04-25T00:07:27.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.340+0000 7fa31b891640 1 -- 172.21.15.113:0/970826422 wait complete. 2024-04-25T00:07:27.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:27.351 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub status 2024-04-25T00:07:27.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.448+0000 7fc332e88640 1 Processor -- start 2024-04-25T00:07:27.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.448+0000 7fc332e88640 1 -- start start 2024-04-25T00:07:27.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc332e88640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc32c0b3080 0x7fc32c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc332e88640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc32c15e6b0 0x7fc32c0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc332e88640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32c15f4f0 con 0x7fc32c15e2d0 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc332e88640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc32c0cde00 con 0x7fc32c15e6b0 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc332e88640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc32c15f7d0 con 0x7fc32c0b3080 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc331425640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc32c15e6b0 0x7fc32c0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc331425640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc32c15e6b0 0x7fc32c0b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37636/0 (socket says 172.21.15.113:37636) 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc331425640 1 -- 172.21.15.113:0/2316575952 learned_addr learned my addr 172.21.15.113:0/2316575952 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc32bfff640 1 --2- 172.21.15.113:0/2316575952 >> v2:172.21.15.178:3300/0 conn(0x7fc32c0b3080 0x7fc32c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc331425640 1 -- 172.21.15.113:0/2316575952 >> v2:172.21.15.178:3300/0 conn(0x7fc32c0b3080 msgr2=0x7fc32c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc331425640 1 --2- 172.21.15.113:0/2316575952 >> v2:172.21.15.178:3300/0 conn(0x7fc32c0b3080 0x7fc32c0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.449+0000 7fc331425640 1 -- 172.21.15.113:0/2316575952 >> v1:172.21.15.178:6789/0 conn(0x7fc32c15e2d0 legacy=0x7fc32c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.450+0000 7fc331425640 1 -- 172.21.15.113:0/2316575952 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c15fab0 con 0x7fc32c15e6b0 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.450+0000 7fc331425640 1 --2- 172.21.15.113:0/2316575952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c0b4870 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fc32c070ff0 tx=0x7fc32402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61be1d948a1936ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.450+0000 7fc32b7fe640 1 -- 172.21.15.113:0/2316575952 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324036070 con 0x7fc32c15e6b0 2024-04-25T00:07:27.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.450+0000 7fc32b7fe640 1 -- 172.21.15.113:0/2316575952 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc324031600 con 0x7fc32c15e6b0 2024-04-25T00:07:27.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.450+0000 7fc32b7fe640 1 -- 172.21.15.113:0/2316575952 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3240319d0 con 0x7fc32c15e6b0 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 -- 172.21.15.113:0/2316575952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 msgr2=0x7fc32c0b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 --2- 172.21.15.113:0/2316575952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c0b4870 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fc32c070ff0 tx=0x7fc32402a6f0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 -- 172.21.15.113:0/2316575952 shutdown_connections 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 --2- 172.21.15.113:0/2316575952 >> v2:172.21.15.178:3300/0 conn(0x7fc32c0b3080 0x7fc32c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 --2- 172.21.15.113:0/2316575952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c0b4870 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 -- 172.21.15.113:0/2316575952 >> 172.21.15.113:0/2316575952 conn(0x7fc32c06b0f0 msgr2=0x7fc32c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 -- 172.21.15.113:0/2316575952 shutdown_connections 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.451+0000 7fc332e88640 1 -- 172.21.15.113:0/2316575952 wait complete. 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.452+0000 7fc332e88640 1 Processor -- start 2024-04-25T00:07:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.452+0000 7fc332e88640 1 -- start start 2024-04-25T00:07:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc332e88640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc32c0b3080 0x7fc32c147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc332e88640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc32c15e2d0 0x7fc32c167950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc332e88640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c166cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc332e88640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c06d020 con 0x7fc32c15e6b0 2024-04-25T00:07:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc332e88640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c06cea0 con 0x7fc32c15e2d0 2024-04-25T00:07:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc332e88640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c06d1a0 con 0x7fc32c0b3080 2024-04-25T00:07:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c166cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c166cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37650/0 (socket says 172.21.15.113:37650) 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 -- 172.21.15.113:0/143009306 learned_addr learned my addr 172.21.15.113:0/143009306 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc330c24640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc32c0b3080 0x7fc32c147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc32bfff640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc32c15e2d0 0x7fc32c167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 -- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc32c0b3080 msgr2=0x7fc32c147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc32c0b3080 0x7fc32c147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 -- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc32c15e2d0 msgr2=0x7fc32c167950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc32c15e2d0 0x7fc32c167950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.453+0000 7fc331425640 1 -- 172.21.15.113:0/143009306 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c0b0000 con 0x7fc32c15e6b0 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.454+0000 7fc331425640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c166cd0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc324009f40 tx=0x7fc324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.454+0000 7fc3297fa640 1 -- 172.21.15.113:0/143009306 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324036050 con 0x7fc32c15e6b0 2024-04-25T00:07:27.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.454+0000 7fc3297fa640 1 -- 172.21.15.113:0/143009306 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc324042070 con 0x7fc32c15e6b0 2024-04-25T00:07:27.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.454+0000 7fc3297fa640 1 -- 172.21.15.113:0/143009306 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324009510 con 0x7fc32c15e6b0 2024-04-25T00:07:27.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.454+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c0aedd0 con 0x7fc32c15e6b0 2024-04-25T00:07:27.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.454+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c0b02a0 con 0x7fc32c15e6b0 2024-04-25T00:07:27.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.455+0000 7fc3297fa640 1 -- 172.21.15.113:0/143009306 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc324007c30 con 0x7fc32c15e6b0 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.456+0000 7fc3297fa640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc31806be30 0x7fc31806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.456+0000 7fc3297fa640 1 -- 172.21.15.113:0/143009306 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fc3240abde0 con 0x7fc32c15e6b0 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.456+0000 7fc330c24640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc31806be30 0x7fc31806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.456+0000 7fc330c24640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc31806be30 0x7fc31806e2f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc32c14bbd0 tx=0x7fc31c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 Processor -- start 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 -- start start 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bc5d60 0x559157bc6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157be3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157b9ba40 0x559157be1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559157aff6b0 con 0x559157b9ba40 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559157aa7880 con 0x559157be3600 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.457+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559157afb670 con 0x559157bc5d60 2024-04-25T00:07:27.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157b9ba40 0x559157be1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157b9ba40 0x559157be1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37664/0 (socket says 172.21.15.113:37664) 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 -- 172.21.15.113:0/329065759 learned_addr learned my addr 172.21.15.113:0/329065759 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc330c24640 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bc5d60 0x559157bc6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc32bfff640 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157be3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 -- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bc5d60 msgr2=0x559157bc6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bc5d60 0x559157bc6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 -- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 msgr2=0x559157be3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157be3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 -- 172.21.15.113:0/329065759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591578dd7d0 con 0x559157b9ba40 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc330c24640 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bc5d60 0x559157bc6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc32bfff640 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157be3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:27.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc331425640 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157b9ba40 0x559157be1cc0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc324082750 tx=0x7fc324079be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eeae66583cee7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc3127fc640 1 -- 172.21.15.113:0/329065759 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324045360 con 0x559157b9ba40 2024-04-25T00:07:27.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc3127fc640 1 -- 172.21.15.113:0/329065759 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc324037040 con 0x559157b9ba40 2024-04-25T00:07:27.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.458+0000 7fc3127fc640 1 -- 172.21.15.113:0/329065759 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32407a7a0 con 0x559157b9ba40 2024-04-25T00:07:27.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 -- 172.21.15.113:0/329065759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157b9ba40 msgr2=0x559157be1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157b9ba40 0x559157be1cc0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc324082750 tx=0x7fc324079be0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 -- 172.21.15.113:0/329065759 shutdown_connections 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157be3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157b9ba40 0x559157be1cc0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 --2- 172.21.15.113:0/329065759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bc5d60 0x559157bc6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 -- 172.21.15.113:0/329065759 >> 172.21.15.113:0/329065759 conn(0x559157b98600 msgr2=0x559157be0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 -- 172.21.15.113:0/329065759 shutdown_connections 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 -- 172.21.15.113:0/329065759 wait complete. 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.459+0000 7fc335b94740 1 -- start start 2024-04-25T00:07:27.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 1 Processor -- start 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157bc9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bb1680 0x559157bc9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559157aff6b0 con 0x559157bc5d60 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559157abaf50 con 0x559157be3600 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559157b9c7e0 con 0x559157bb1680 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc335b94740 10 client.0 _ll_register_callbacks cb 0x559157af6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc330c24640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc330c24640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37668/0 (socket says 172.21.15.113:37668) 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc330c24640 1 -- 172.21.15.113:0/684867376 learned_addr learned my addr 172.21.15.113:0/684867376 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc32bfff640 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157bc9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.460+0000 7fc331425640 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bb1680 0x559157bc9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc330c24640 1 -- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bb1680 msgr2=0x559157bc9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc330c24640 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bb1680 0x559157bc9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc330c24640 1 -- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 msgr2=0x559157bc9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc330c24640 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157bc9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc330c24640 1 -- 172.21.15.113:0/684867376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591578133f0 con 0x559157bc5d60 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc330c24640 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd8600 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00ed60 tx=0x7fc31c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2705f75cd54e8190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc30b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc30b7fe640 1 -- 172.21.15.113:0/684867376 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c026060 con 0x559157bc5d60 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc30b7fe640 1 -- 172.21.15.113:0/684867376 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fc31c021470 con 0x559157bc5d60 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc324037040 con 0x559157bc5d60 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc30b7fe640 1 -- 172.21.15.113:0/684867376 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fc31c02c050 con 0x559157bc5d60 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 10 client.9720 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:27.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x5591577d30f0 con 0x559157bc5d60 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc30b7fe640 1 -- 172.21.15.113:0/684867376 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc324037040 con 0x559157bc5d60 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc30b7fe640 1 -- 172.21.15.113:0/684867376 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7fc31c02f020 con 0x559157bc5d60 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 10 client.9720 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 10 client.9720 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 20 client.9720 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157bb3340 0x559157bb3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 4 client.9720 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559157b0cbf0 con 0x559157bb3340 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.461+0000 7fc32bfff640 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157bb3340 0x559157bb3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc30b7fe640 10 client.9720 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc32bfff640 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157bb3340 0x559157bb3c10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4af034b6949aeea server_cookie=b1496d54522e9af8 in_seq=0 out_seq=0 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc30b7fe640 1 -- 172.21.15.113:0/684867376 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559157b0cbf0 con 0x559157bb3340 2024-04-25T00:07:27.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc30b7fe640 10 client.9720 handle_command_reply: tid=0 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 client.9720 shutdown 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 msgr2=0x559157bd8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd8600 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00ed60 tx=0x7fc31c0036f0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 shutdown_connections 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157bb3340 0x559157bb3c10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157be3600 0x559157bc9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157bb1680 0x559157bc9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc30b7fe640 0 client.9720 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.462+0000 7fc335b94740 1 --2- 172.21.15.113:0/684867376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd8600 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.463+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 >> 172.21.15.113:0/684867376 conn(0x559157b9ba40 msgr2=0x559157be4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.463+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 shutdown_connections 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.463+0000 7fc335b94740 1 -- 172.21.15.113:0/684867376 wait complete. 2024-04-25T00:07:27.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.463+0000 7fc335b94740 20 client.9720 trim_cache size 0 max 16384 2024-04-25T00:07:27.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.480+0000 7fc335b94740 1 Processor -- start 2024-04-25T00:07:27.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.480+0000 7fc335b94740 1 -- start start 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.480+0000 7fc335b94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bb1680 0x559157bb7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.480+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bc5d60 0x559157bb7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.480+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157be2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.480+0000 7fc335b94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c2008a0 con 0x559157bb1680 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1ff8a0 con 0x559157bc5d60 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c1fec30 con 0x559157be3600 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bb1680 0x559157bb7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bb1680 0x559157bb7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56836/0 (socket says 172.21.15.113:56836) 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 -- 172.21.15.113:0/3275600522 learned_addr learned my addr 172.21.15.113:0/3275600522 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc32bfff640 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bc5d60 0x559157bb7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc331425640 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157be2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 -- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 msgr2=0x559157be2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157be2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 -- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bc5d60 msgr2=0x559157bb7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bc5d60 0x559157bb7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 -- 172.21.15.113:0/3275600522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559157b97ac0 con 0x559157bb1680 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc331425640 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157be2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc32bfff640 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bc5d60 0x559157bb7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc330c24640 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bb1680 0x559157bb7740 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fc31c003cd0 tx=0x7fc31c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bf0e2ee17ce311 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc311ffb640 1 -- 172.21.15.113:0/3275600522 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c035070 con 0x559157bb1680 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc311ffb640 1 -- 172.21.15.113:0/3275600522 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c021470 con 0x559157bb1680 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.481+0000 7fc311ffb640 1 -- 172.21.15.113:0/3275600522 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0219c0 con 0x559157bb1680 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 -- 172.21.15.113:0/3275600522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bb1680 msgr2=0x559157bb7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bb1680 0x559157bb7740 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fc31c003cd0 tx=0x7fc31c01e000 comp rx=0 tx=0).stop 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 -- 172.21.15.113:0/3275600522 shutdown_connections 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bc5d60 0x559157bb7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157be2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 --2- 172.21.15.113:0/3275600522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bb1680 0x559157bb7740 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 -- 172.21.15.113:0/3275600522 >> 172.21.15.113:0/3275600522 conn(0x559157b9ba40 msgr2=0x559157bdd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 -- 172.21.15.113:0/3275600522 shutdown_connections 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 -- 172.21.15.113:0/3275600522 wait complete. 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.482+0000 7fc335b94740 1 -- start start 2024-04-25T00:07:27.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 1 Processor -- start 2024-04-25T00:07:27.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bb1680 0x559157c4e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157bd6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559157b992c0 con 0x559157bc5d60 2024-04-25T00:07:27.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1ff250 con 0x559157bb1680 2024-04-25T00:07:27.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559157bb1490 con 0x559157be3600 2024-04-25T00:07:27.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc335b94740 10 client.0 _ll_register_callbacks cb 0x7fc31c007dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc32bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc32bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56840/0 (socket says 172.21.15.113:56840) 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.483+0000 7fc32bfff640 1 -- 172.21.15.113:0/1946189178 learned_addr learned my addr 172.21.15.113:0/1946189178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc330c24640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bb1680 0x559157c4e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc331425640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157bd6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc32bfff640 1 -- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 msgr2=0x559157bd6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc32bfff640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157bd6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc32bfff640 1 -- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bb1680 msgr2=0x559157c4e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc32bfff640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bb1680 0x559157c4e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc32bfff640 1 -- 172.21.15.113:0/1946189178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559157c04ec0 con 0x559157bc5d60 2024-04-25T00:07:27.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc331425640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157bd6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc32bfff640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd6500 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc32c166c80 tx=0x7fc320018810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307d8799c0adb58e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc30b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc30b7fe640 1 -- 172.21.15.113:0/1946189178 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32000f040 con 0x559157bc5d60 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc30b7fe640 1 -- 172.21.15.113:0/1946189178 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fc320004220 con 0x559157bc5d60 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc31c021470 con 0x559157bc5d60 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc30b7fe640 1 -- 172.21.15.113:0/1946189178 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fc32002d050 con 0x559157bc5d60 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc335b94740 10 client.9726 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x559157bf5e30 con 0x559157bc5d60 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc30b7fe640 1 -- 172.21.15.113:0/1946189178 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c021470 con 0x559157bc5d60 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc30b7fe640 1 -- 172.21.15.113:0/1946189178 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7fc320030020 con 0x559157bc5d60 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc335b94740 10 client.9726 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc335b94740 10 client.9726 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:27.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.484+0000 7fc335b94740 20 client.9726 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157b956c0 0x559157bf4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 4 client.9726 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559157bc9fb0 con 0x559157b956c0 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc330c24640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157b956c0 0x559157bf4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc30b7fe640 10 client.9726 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc330c24640 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157b956c0 0x559157bf4cf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd7e14a58b4158d server_cookie=7cee1e65f664c3e0 in_seq=0 out_seq=0 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc30b7fe640 1 -- 172.21.15.113:0/1946189178 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+257 (crc 0 0 0) 0x559157bc9fb0 con 0x559157b956c0 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc30b7fe640 10 client.9726 handle_command_reply: tid=0 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 client.9726 shutdown 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 msgr2=0x559157bd6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd6500 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc32c166c80 tx=0x7fc320018810 comp rx=0 tx=0).stop 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 shutdown_connections 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x559157b956c0 0x559157bf4cf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559157be3600 0x559157bd6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559157bc5d60 0x559157bd6500 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 --2- 172.21.15.113:0/1946189178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559157bb1680 0x559157c4e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc30b7fe640 0 client.9726 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 >> 172.21.15.113:0/1946189178 conn(0x559157b9ba40 msgr2=0x559157b9c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 shutdown_connections 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.485+0000 7fc335b94740 1 -- 172.21.15.113:0/1946189178 wait complete. 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc335b94740 20 client.9726 trim_cache size 0 max 16384 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:07:27.493 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:07:27.494 INFO:teuthology.orchestra.run.smithi113.stdout: "24cee8b1-3586-4928-b3e5-06eb8aae44cb": { 2024-04-25T00:07:27.494 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "~mds0", 2024-04-25T00:07:27.494 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "24cee8b1-3586-4928-b3e5-06eb8aae44cb", 2024-04-25T00:07:27.494 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive" 2024-04-25T00:07:27.494 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:27.494 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:07:27.494 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc31806be30 msgr2=0x7fc31806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc31806be30 0x7fc31806e2f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc32c14bbd0 tx=0x7fc31c009040 comp rx=0 tx=0).stop 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 msgr2=0x7fc32c166cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c166cd0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc324009f40 tx=0x7fc324004a40 comp rx=0 tx=0).stop 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 shutdown_connections 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc31806be30 0x7fc31806e2f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc32c15e2d0 0x7fc32c167950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32c15e6b0 0x7fc32c166cd0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 --2- 172.21.15.113:0/143009306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc32c0b3080 0x7fc32c147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 >> 172.21.15.113:0/143009306 conn(0x7fc32c06b0f0 msgr2=0x7fc32c0cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 shutdown_connections 2024-04-25T00:07:27.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:27.486+0000 7fc332e88640 1 -- 172.21.15.113:0/143009306 wait complete. 2024-04-25T00:07:27.497 INFO:tasks.cephfs.filesystem:scrub status for tag:24cee8b1-3586-4928-b3e5-06eb8aae44cb - {'path': '~mds0', 'tag': '24cee8b1-3586-4928-b3e5-06eb8aae44cb', 'options': 'recursive'} 2024-04-25T00:07:53.528 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:07:53.532 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:07:57.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:57.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.10:0 scrub status 2024-04-25T00:07:57.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.608+0000 7f4296812640 1 Processor -- start 2024-04-25T00:07:57.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.608+0000 7f4296812640 1 -- start start 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.609+0000 7f4296812640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f42900b3080 0x7f42900b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.609+0000 7f4296812640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429015f4f0 con 0x7f429015e6b0 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.609+0000 7f4296812640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429015f7d0 con 0x7f429015e2d0 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.609+0000 7f4296812640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f42900cde00 con 0x7f42900b3080 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.609+0000 7f428f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f42900b3080 0x7f42900b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.609+0000 7f4294daf640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f429015e6b0 0x7f42900ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60680/0 (socket says 172.21.15.113:60680) 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.609+0000 7f4294daf640 1 -- 172.21.15.113:0/2258208389 learned_addr learned my addr 172.21.15.113:0/2258208389 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:57.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.610+0000 7f428f7fe640 1 -- 172.21.15.113:0/2258208389 >> v1:172.21.15.178:6789/0 conn(0x7f429015e2d0 legacy=0x7f429015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.610+0000 7f428f7fe640 1 -- 172.21.15.113:0/2258208389 >> v1:172.21.15.113:6789/0 conn(0x7f429015e6b0 legacy=0x7f42900ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.610+0000 7f428f7fe640 1 -- 172.21.15.113:0/2258208389 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429015fab0 con 0x7f42900b3080 2024-04-25T00:07:57.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.610+0000 7f428effd640 1 -- 172.21.15.113:0/2258208389 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307551640 0 0) 0x7f429015f4f0 con 0x7f429015e6b0 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.610+0000 7f428f7fe640 1 --2- 172.21.15.113:0/2258208389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42900b3080 0x7f42900b4330 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f42840097a0 tx=0x7f428402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dcba45a3221294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f428effd640 1 -- 172.21.15.113:0/2258208389 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284037070 con 0x7f42900b3080 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f428effd640 1 -- 172.21.15.113:0/2258208389 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4284032440 con 0x7f42900b3080 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f428effd640 1 -- 172.21.15.113:0/2258208389 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284032810 con 0x7f42900b3080 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f4296812640 1 -- 172.21.15.113:0/2258208389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42900b3080 msgr2=0x7f42900b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f4296812640 1 --2- 172.21.15.113:0/2258208389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42900b3080 0x7f42900b4330 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f42840097a0 tx=0x7f428402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f4296812640 1 -- 172.21.15.113:0/2258208389 shutdown_connections 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f4296812640 1 --2- 172.21.15.113:0/2258208389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42900b3080 0x7f42900b4330 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f4296812640 1 -- 172.21.15.113:0/2258208389 >> 172.21.15.113:0/2258208389 conn(0x7f429006b0f0 msgr2=0x7f429006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f4296812640 1 -- 172.21.15.113:0/2258208389 shutdown_connections 2024-04-25T00:07:57.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.611+0000 7f4296812640 1 -- 172.21.15.113:0/2258208389 wait complete. 2024-04-25T00:07:57.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.612+0000 7f4296812640 1 Processor -- start 2024-04-25T00:07:57.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f4296812640 1 -- start start 2024-04-25T00:07:57.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f4296812640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f429015e2d0 0x7f4290154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f4296812640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f429015e6b0 0x7f42901f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f4296812640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42901558a0 0x7f4290155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f4296812640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f42900b49f0 con 0x7f429015e6b0 2024-04-25T00:07:57.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f4296812640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42900b4870 con 0x7f42901558a0 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f4296812640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42900b4b70 con 0x7f429015e2d0 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f429015e6b0 0x7f42901f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f429015e6b0 0x7f42901f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34404/0 (socket says 172.21.15.113:34404) 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.613+0000 7f428f7fe640 1 -- 172.21.15.113:0/3860786278 learned_addr learned my addr 172.21.15.113:0/3860786278 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f4294daf640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42901558a0 0x7f4290155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428f7fe640 1 -- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f429015e2d0 msgr2=0x7f4290154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428ffff640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f429015e2d0 0x7f4290154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428f7fe640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f429015e2d0 0x7f4290154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428f7fe640 1 -- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42901558a0 msgr2=0x7f4290155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428f7fe640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42901558a0 0x7f4290155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428f7fe640 1 -- 172.21.15.113:0/3860786278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42901531b0 con 0x7f429015e6b0 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f4294daf640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42901558a0 0x7f4290155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:57.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428f7fe640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f429015e6b0 0x7f42901f7bc0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f42840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428cff9640 1 -- 172.21.15.113:0/3860786278 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284037040 con 0x7f429015e6b0 2024-04-25T00:07:57.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428cff9640 1 -- 172.21.15.113:0/3860786278 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4284004230 con 0x7f429015e6b0 2024-04-25T00:07:57.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4290153490 con 0x7f429015e6b0 2024-04-25T00:07:57.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f428cff9640 1 -- 172.21.15.113:0/3860786278 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428403c670 con 0x7f429015e6b0 2024-04-25T00:07:57.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.614+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4290153a50 con 0x7f429015e6b0 2024-04-25T00:07:57.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.615+0000 7f428cff9640 1 -- 172.21.15.113:0/3860786278 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f428403a070 con 0x7f429015e6b0 2024-04-25T00:07:57.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.616+0000 7f428cff9640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f427c06be30 0x7f427c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.616+0000 7f428cff9640 1 -- 172.21.15.113:0/3860786278 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f42840ab790 con 0x7f429015e6b0 2024-04-25T00:07:57.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.616+0000 7f428ffff640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f427c06be30 0x7f427c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.616+0000 7f428ffff640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f427c06be30 0x7f427c06e2f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4290159880 tx=0x7f4280009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.617+0000 7f42995f8740 1 Processor -- start 2024-04-25T00:07:57.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.617+0000 7f42995f8740 1 -- start start 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f648d60 0x56216f649140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f666600 0x56216f666b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f42995f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f61ea40 0x56216f664cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f42995f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56216f5826b0 con 0x56216f61ea40 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56216f52a880 con 0x56216f648d60 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56216f57e670 con 0x56216f666600 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f61ea40 0x56216f664cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f61ea40 0x56216f664cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34414/0 (socket says 172.21.15.113:34414) 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 -- 172.21.15.113:0/3407313028 learned_addr learned my addr 172.21.15.113:0/3407313028 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f428ffff640 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f648d60 0x56216f649140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f428f7fe640 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f666600 0x56216f666b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 -- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f666600 msgr2=0x56216f666b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f666600 0x56216f666b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 -- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f648d60 msgr2=0x56216f649140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f648d60 0x56216f649140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 -- 172.21.15.113:0/3407313028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216f3607d0 con 0x56216f61ea40 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4294daf640 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f61ea40 0x56216f664cc0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f4288001e90 tx=0x7f428800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adee3954ae358f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4275ffb640 1 -- 172.21.15.113:0/3407313028 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288004d60 con 0x56216f61ea40 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4275ffb640 1 -- 172.21.15.113:0/3407313028 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42880020a0 con 0x56216f61ea40 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.618+0000 7f4275ffb640 1 -- 172.21.15.113:0/3407313028 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880072d0 con 0x56216f61ea40 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 -- 172.21.15.113:0/3407313028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f61ea40 msgr2=0x56216f664cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f61ea40 0x56216f664cc0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f4288001e90 tx=0x7f428800d520 comp rx=0 tx=0).stop 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 -- 172.21.15.113:0/3407313028 shutdown_connections 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f61ea40 0x56216f664cc0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f666600 0x56216f666b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 --2- 172.21.15.113:0/3407313028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f648d60 0x56216f649140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 -- 172.21.15.113:0/3407313028 >> 172.21.15.113:0/3407313028 conn(0x56216f61b600 msgr2=0x56216f663b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 -- 172.21.15.113:0/3407313028 shutdown_connections 2024-04-25T00:07:57.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 -- 172.21.15.113:0/3407313028 wait complete. 2024-04-25T00:07:57.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.619+0000 7f42995f8740 1 -- start start 2024-04-25T00:07:57.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.620+0000 7f42995f8740 1 Processor -- start 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.620+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f645420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.620+0000 7f42995f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f64b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.620+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f664f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f42995f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56216f61f7e0 con 0x56216f666600 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56216f53df50 con 0x56216f664af0 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56216f619530 con 0x56216f648d60 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f42995f8740 10 client.0 _ll_register_callbacks cb 0x56216f579000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f64b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f64b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34426/0 (socket says 172.21.15.113:34426) 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 -- 172.21.15.113:0/703840943 learned_addr learned my addr 172.21.15.113:0/703840943 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f4294daf640 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f664f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428ffff640 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f645420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 -- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 msgr2=0x56216f645420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f645420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 -- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 msgr2=0x56216f664f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f664f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 -- 172.21.15.113:0/703840943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216f2963f0 con 0x56216f666600 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f428f7fe640 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f64b980 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f4284075e50 tx=0x7f428407a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf342520543eea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f426effd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f426effd640 1 -- 172.21.15.113:0/703840943 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284091020 con 0x56216f666600 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f426effd640 1 -- 172.21.15.113:0/703840943 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f42840a4cd0 con 0x56216f666600 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56216f58fbf0 con 0x56216f666600 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f426effd640 1 -- 172.21.15.113:0/703840943 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f428404a040 con 0x56216f666600 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f42995f8740 10 client.9741 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.621+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x56216f2560f0 con 0x56216f666600 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f426effd640 1 -- 172.21.15.113:0/703840943 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56216f58fbf0 con 0x56216f666600 2024-04-25T00:07:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f426effd640 1 -- 172.21.15.113:0/703840943 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f4284082190 con 0x56216f666600 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f42995f8740 10 client.9741 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f42995f8740 10 client.9741 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f42995f8740 20 client.9741 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f42995f8740 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f6737c0 0x56216f673ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f42995f8740 4 client.9741 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56216f589f40 con 0x56216f6737c0 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f428ffff640 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f6737c0 0x56216f673ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f426effd640 10 client.9741 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.622+0000 7f428ffff640 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f6737c0 0x56216f673ba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc671e41ba7fd85 server_cookie=6d5f588367dc98f2 in_seq=0 out_seq=0 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f426effd640 1 -- 172.21.15.113:0/703840943 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56216f589f40 con 0x56216f6737c0 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f426effd640 10 client.9741 handle_command_reply: tid=0 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 client.9741 shutdown 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 msgr2=0x56216f64b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f64b980 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f4284075e50 tx=0x7f428407a280 comp rx=0 tx=0).stop 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 shutdown_connections 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f6737c0 0x56216f673ba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f664f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f64b980 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 --2- 172.21.15.113:0/703840943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f645420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f426effd640 0 client.9741 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 >> 172.21.15.113:0/703840943 conn(0x56216f61ea40 msgr2=0x56216f6670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 shutdown_connections 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 1 -- 172.21.15.113:0/703840943 wait complete. 2024-04-25T00:07:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.623+0000 7f42995f8740 20 client.9741 trim_cache size 0 max 16384 2024-04-25T00:07:57.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.640+0000 7f42995f8740 1 Processor -- start 2024-04-25T00:07:57.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.640+0000 7f42995f8740 1 -- start start 2024-04-25T00:07:57.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f42995f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f664af0 0x56216f689160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f666600 0x56216f663000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f42995f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4290200d10 con 0x56216f664af0 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42901ffd70 con 0x56216f666600 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42901f51c0 con 0x56216f648d60 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f664af0 0x56216f689160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f664af0 0x56216f689160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34428/0 (socket says 172.21.15.113:34428) 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f428f7fe640 1 -- 172.21.15.113:0/2143962579 learned_addr learned my addr 172.21.15.113:0/2143962579 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f4294daf640 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f666600 0x56216f663000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.641+0000 7f428ffff640 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f428f7fe640 1 -- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 msgr2=0x56216f688c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f428f7fe640 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f428f7fe640 1 -- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f666600 msgr2=0x56216f663000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f428f7fe640 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f666600 0x56216f663000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f428f7fe640 1 -- 172.21.15.113:0/2143962579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216f613f00 con 0x56216f664af0 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f4294daf640 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f666600 0x56216f663000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f428ffff640 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f428f7fe640 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f664af0 0x56216f689160 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f4284075ea0 tx=0x7f4284047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137c48a3bb19410c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42757fa640 1 -- 172.21.15.113:0/2143962579 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284091070 con 0x56216f664af0 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42757fa640 1 -- 172.21.15.113:0/2143962579 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4284082190 con 0x56216f664af0 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42757fa640 1 -- 172.21.15.113:0/2143962579 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840793a0 con 0x56216f664af0 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42995f8740 1 -- 172.21.15.113:0/2143962579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f664af0 msgr2=0x56216f689160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42995f8740 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f664af0 0x56216f689160 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f4284075ea0 tx=0x7f4284047000 comp rx=0 tx=0).stop 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42995f8740 1 -- 172.21.15.113:0/2143962579 shutdown_connections 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42995f8740 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f666600 0x56216f663000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42995f8740 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f664af0 0x56216f689160 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42995f8740 1 --2- 172.21.15.113:0/2143962579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.642+0000 7f42995f8740 1 -- 172.21.15.113:0/2143962579 >> 172.21.15.113:0/2143962579 conn(0x56216f61ea40 msgr2=0x56216f645230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.643+0000 7f42995f8740 1 -- 172.21.15.113:0/2143962579 shutdown_connections 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.643+0000 7f42995f8740 1 -- 172.21.15.113:0/2143962579 wait complete. 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.643+0000 7f42995f8740 1 -- start start 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.643+0000 7f42995f8740 1 Processor -- start 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f42995f8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f61d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f42995f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f663b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f42995f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56216f65c8e0 con 0x56216f666600 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42900b14b0 con 0x56216f664af0 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f42995f8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56216f64bb70 con 0x56216f648d60 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f42995f8740 10 client.0 _ll_register_callbacks cb 0x7f4284079580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:57.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f663b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f663b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34442/0 (socket says 172.21.15.113:34442) 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 -- 172.21.15.113:0/1601501929 learned_addr learned my addr 172.21.15.113:0/1601501929 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f428f7fe640 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f61d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f428ffff640 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 -- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 msgr2=0x56216f688e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 -- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 msgr2=0x56216f61d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f61d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.644+0000 7f4294daf640 1 -- 172.21.15.113:0/1601501929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216f661ab0 con 0x56216f666600 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f4294daf640 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f663b70 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f4288011000 tx=0x7f4288001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184d7ecaae48e243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f426effd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f426effd640 1 -- 172.21.15.113:0/1601501929 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428800de40 con 0x56216f666600 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f426effd640 1 -- 172.21.15.113:0/1601501929 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f428802aab0 con 0x56216f666600 2024-04-25T00:07:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4284082190 con 0x56216f666600 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f426effd640 1 -- 172.21.15.113:0/1601501929 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f428802acb0 con 0x56216f666600 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 10 client.9747 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x56216f67d8d0 con 0x56216f666600 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f426effd640 1 -- 172.21.15.113:0/1601501929 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4284082190 con 0x56216f666600 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f426effd640 1 -- 172.21.15.113:0/1601501929 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f4288007250 con 0x56216f666600 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 10 client.9747 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 10 client.9747 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 20 client.9747 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f647aa0 0x56216f6640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 4 client.9747 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56216f5076f0 con 0x56216f647aa0 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f428ffff640 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f647aa0 0x56216f6640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.645+0000 7f426effd640 10 client.9747 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f428ffff640 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f647aa0 0x56216f6640b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7e87ab5001c412 server_cookie=769de05d93747571 in_seq=0 out_seq=0 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f426effd640 1 -- 172.21.15.113:0/1601501929 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56216f5076f0 con 0x56216f647aa0 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f426effd640 10 client.9747 handle_command_reply: tid=0 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 client.9747 shutdown 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 msgr2=0x56216f663b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f663b70 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f4288011000 tx=0x7f4288001630 comp rx=0 tx=0).stop 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 shutdown_connections 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x56216f647aa0 0x56216f6640b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56216f664af0 0x56216f61d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56216f666600 0x56216f663b70 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f426effd640 0 client.9747 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 --2- 172.21.15.113:0/1601501929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56216f648d60 0x56216f688e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 >> 172.21.15.113:0/1601501929 conn(0x56216f61ea40 msgr2=0x56216f665e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 shutdown_connections 2024-04-25T00:07:57.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 1 -- 172.21.15.113:0/1601501929 wait complete. 2024-04-25T00:07:57.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.646+0000 7f42995f8740 20 client.9747 trim_cache size 0 max 16384 2024-04-25T00:07:57.655 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:07:57.655 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:07:57.655 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:07:57.655 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f427c06be30 msgr2=0x7f427c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f427c06be30 0x7f427c06e2f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4290159880 tx=0x7f4280009040 comp rx=0 tx=0).stop 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f429015e6b0 msgr2=0x7f42901f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f429015e6b0 0x7f42901f7bc0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f42840046c0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 shutdown_connections 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f427c06be30 0x7f427c06e2f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42901558a0 0x7f4290155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f429015e6b0 0x7f42901f7bc0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 --2- 172.21.15.113:0/3860786278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f429015e2d0 0x7f4290154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 >> 172.21.15.113:0/3860786278 conn(0x7f429006b0f0 msgr2=0x7f429015c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 shutdown_connections 2024-04-25T00:07:57.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.647+0000 7f4296812640 1 -- 172.21.15.113:0/3860786278 wait complete. 2024-04-25T00:07:57.658 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:07:57.658 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.10:0 damage ls 2024-04-25T00:07:57.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.756+0000 7f099955a640 1 Processor -- start 2024-04-25T00:07:57.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.756+0000 7f099955a640 1 -- start start 2024-04-25T00:07:57.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f099955a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f099415e510 0x7f09940d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f099955a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940b63b0 0x7f09940b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f099955a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09940d2e00 con 0x7f09940d28c0 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f099955a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f09940d30e0 con 0x7f099415e510 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f099955a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09940d33c0 con 0x7f09940b63b0 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f0992ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f099415e510 0x7f09940d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f09927fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f09940d28c0 0x7f09940b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60694/0 (socket says 172.21.15.113:60694) 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f0992ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f099415e510 0x7f09940d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34446/0 (socket says 172.21.15.113:34446) 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.757+0000 7f09927fc640 1 -- 172.21.15.113:0/2478939376 learned_addr learned my addr 172.21.15.113:0/2478939376 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f09937fe640 1 --2- 172.21.15.113:0/2478939376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940b63b0 0x7f09940b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0992ffd640 1 -- 172.21.15.113:0/2478939376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940b63b0 msgr2=0x7f09940b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0992ffd640 1 --2- 172.21.15.113:0/2478939376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940b63b0 0x7f09940b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0992ffd640 1 -- 172.21.15.113:0/2478939376 >> v1:172.21.15.113:6789/0 conn(0x7f09940d28c0 legacy=0x7f09940b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0992ffd640 1 -- 172.21.15.113:0/2478939376 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09940d36a0 con 0x7f099415e510 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0992ffd640 1 --2- 172.21.15.113:0/2478939376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f099415e510 0x7f09940d2380 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f098c009d90 tx=0x7f098c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4892d2c89c4ed87f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0991ffb640 1 -- 172.21.15.113:0/2478939376 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c031600 con 0x7f099415e510 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0991ffb640 1 -- 172.21.15.113:0/2478939376 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c0317a0 con 0x7f099415e510 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.758+0000 7f0991ffb640 1 -- 172.21.15.113:0/2478939376 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c031b70 con 0x7f099415e510 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 -- 172.21.15.113:0/2478939376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f099415e510 msgr2=0x7f09940d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 --2- 172.21.15.113:0/2478939376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f099415e510 0x7f09940d2380 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f098c009d90 tx=0x7f098c02a5d0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 -- 172.21.15.113:0/2478939376 shutdown_connections 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 --2- 172.21.15.113:0/2478939376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940b63b0 0x7f09940b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 --2- 172.21.15.113:0/2478939376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f099415e510 0x7f09940d2380 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 -- 172.21.15.113:0/2478939376 >> 172.21.15.113:0/2478939376 conn(0x7f099406b720 msgr2=0x7f09940d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 -- 172.21.15.113:0/2478939376 shutdown_connections 2024-04-25T00:07:57.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.759+0000 7f099955a640 1 -- 172.21.15.113:0/2478939376 wait complete. 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.760+0000 7f099955a640 1 Processor -- start 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.760+0000 7f099955a640 1 -- start start 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f099955a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09940b63b0 0x7f0994149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f099955a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940d28c0 0x7f09940b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f099955a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f099415e510 0x7f09940b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f099955a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09940b9dd0 con 0x7f09940b63b0 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f099955a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09940d3ea0 con 0x7f099415e510 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f099955a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09940b7900 con 0x7f09940d28c0 2024-04-25T00:07:57.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09940b63b0 0x7f0994149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09940b63b0 0x7f0994149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34448/0 (socket says 172.21.15.113:34448) 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 -- 172.21.15.113:0/2991925712 learned_addr learned my addr 172.21.15.113:0/2991925712 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 -- 172.21.15.113:0/2991925712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940d28c0 msgr2=0x7f09940b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940d28c0 0x7f09940b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 -- 172.21.15.113:0/2991925712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f099415e510 msgr2=0x7f09940b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f099415e510 0x7f09940b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.761+0000 7f0992ffd640 1 -- 172.21.15.113:0/2991925712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09941611d0 con 0x7f09940b63b0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.762+0000 7f0992ffd640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09940b63b0 0x7f0994149580 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f098c009eb0 tx=0x7f098c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.762+0000 7f098b7fe640 1 -- 172.21.15.113:0/2991925712 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c03f070 con 0x7f09940b63b0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.762+0000 7f098b7fe640 1 -- 172.21.15.113:0/2991925712 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c030be0 con 0x7f09940b63b0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.762+0000 7f098b7fe640 1 -- 172.21.15.113:0/2991925712 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c044460 con 0x7f09940b63b0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.762+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09940c7d50 con 0x7f09940b63b0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.762+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09940c8310 con 0x7f09940b63b0 2024-04-25T00:07:57.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.763+0000 7f098b7fe640 1 -- 172.21.15.113:0/2991925712 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f098c044600 con 0x7f09940b63b0 2024-04-25T00:07:57.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.763+0000 7f098b7fe640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f097406be30 0x7f097406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.764+0000 7f098b7fe640 1 -- 172.21.15.113:0/2991925712 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f098c0ac560 con 0x7f09940b63b0 2024-04-25T00:07:57.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.764+0000 7f09927fc640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f097406be30 0x7f097406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.764+0000 7f09927fc640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f097406be30 0x7f097406e2f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f097c000f60 tx=0x7f097c007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.764+0000 7f099c266740 1 Processor -- start 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f099c266740 1 -- start start 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f099c266740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d00950 0x55dd26cb32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26ce2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26ce3330 0x55dd26cb8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f099c266740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dd26c1c670 con 0x55dd26d00950 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd26bc4970 con 0x55dd26ce3a70 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd26c18630 con 0x55dd26ce3330 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d00950 0x55dd26cb32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d00950 0x55dd26cb32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34458/0 (socket says 172.21.15.113:34458) 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.765+0000 7f0992ffd640 1 -- 172.21.15.113:0/604429123 learned_addr learned my addr 172.21.15.113:0/604429123 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f09927fc640 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26ce2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0992ffd640 1 -- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26ce3330 msgr2=0x55dd26cb8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0992ffd640 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26ce3330 0x55dd26cb8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0992ffd640 1 -- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 msgr2=0x55dd26ce2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0992ffd640 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26ce2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0992ffd640 1 -- 172.21.15.113:0/604429123 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd26be8400 con 0x55dd26d00950 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f09927fc640 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26ce2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0992ffd640 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d00950 0x55dd26cb32e0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f098c076d30 tx=0x7f098c07a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401168ac6d998e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0988ff9640 1 -- 172.21.15.113:0/604429123 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c031700 con 0x55dd26d00950 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0988ff9640 1 -- 172.21.15.113:0/604429123 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c078bd0 con 0x55dd26d00950 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f0988ff9640 1 -- 172.21.15.113:0/604429123 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c081030 con 0x55dd26d00950 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f099c266740 1 -- 172.21.15.113:0/604429123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d00950 msgr2=0x55dd26cb32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.766+0000 7f099c266740 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d00950 0x55dd26cb32e0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f098c076d30 tx=0x7f098c07a990 comp rx=0 tx=0).stop 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 -- 172.21.15.113:0/604429123 shutdown_connections 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26ce2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26ce3330 0x55dd26cb8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 --2- 172.21.15.113:0/604429123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d00950 0x55dd26cb32e0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 -- 172.21.15.113:0/604429123 >> 172.21.15.113:0/604429123 conn(0x55dd26cb5550 msgr2=0x55dd26cfddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 -- 172.21.15.113:0/604429123 shutdown_connections 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 -- 172.21.15.113:0/604429123 wait complete. 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 -- start start 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.767+0000 7f099c266740 1 Processor -- start 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26ce6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f099c266740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f099c266740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dd26c1c670 con 0x55dd26d116d0 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd26bd8050 con 0x55dd26ce3a70 2024-04-25T00:07:57.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd26cb7700 con 0x55dd26d00950 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f099c266740 10 client.0 _ll_register_callbacks cb 0x55dd2692fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cf4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34470/0 (socket says 172.21.15.113:34470) 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 -- 172.21.15.113:0/14250192 learned_addr learned my addr 172.21.15.113:0/14250192 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09927fc640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26ce6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f0992ffd640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 -- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 msgr2=0x55dd26ce6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26ce6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 -- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 msgr2=0x55dd26cde800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cde800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 -- 172.21.15.113:0/14250192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd26ba9dc0 con 0x55dd26d116d0 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f0992ffd640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cde800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09927fc640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26ce6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:57.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f09937fe640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cf4ab0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f098400af30 tx=0x7f098400c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86ebc85fa7ee1b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.768+0000 7f0971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0971ffb640 1 -- 172.21.15.113:0/14250192 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984015070 con 0x55dd26d116d0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c078bd0 con 0x55dd26d116d0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0971ffb640 1 -- 172.21.15.113:0/14250192 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f09840115f0 con 0x55dd26d116d0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0971ffb640 1 -- 172.21.15.113:0/14250192 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f0984020050 con 0x55dd26d116d0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 10 client.9762 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55dd269303f0 con 0x55dd26d116d0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0971ffb640 1 -- 172.21.15.113:0/14250192 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c078bd0 con 0x55dd26d116d0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0971ffb640 1 -- 172.21.15.113:0/14250192 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f0984023020 con 0x55dd26d116d0 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 10 client.9762 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 10 client.9762 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 20 client.9762 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:57.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26cffb90 0x55dd26ce3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 4 client.9762 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd26c29bb0 con 0x55dd26cffb90 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0992ffd640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26cffb90 0x55dd26ce3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0971ffb640 10 client.9762 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.769+0000 7f0992ffd640 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26cffb90 0x55dd26ce3f60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5e1a16228d127cb server_cookie=5cd44f89fa1401b5 in_seq=0 out_seq=0 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f0971ffb640 1 -- 172.21.15.113:0/14250192 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd26c29bb0 con 0x55dd26cffb90 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f0971ffb640 10 client.9762 handle_command_reply: tid=0 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 client.9762 shutdown 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 msgr2=0x55dd26cf4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cf4ab0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f098400af30 tx=0x7f098400c5e0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 shutdown_connections 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26cffb90 0x55dd26ce3f60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cf4ab0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26ce6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 --2- 172.21.15.113:0/14250192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cde800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 >> 172.21.15.113:0/14250192 conn(0x55dd26ce3330 msgr2=0x55dd26cb6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f0971ffb640 0 client.9762 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 shutdown_connections 2024-04-25T00:07:57.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 1 -- 172.21.15.113:0/14250192 wait complete. 2024-04-25T00:07:57.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.770+0000 7f099c266740 20 client.9762 trim_cache size 0 max 16384 2024-04-25T00:07:57.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.783+0000 7f099c266740 1 Processor -- start 2024-04-25T00:07:57.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.783+0000 7f099c266740 1 -- start start 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cf7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26cf7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f099c266740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cb2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f099c266740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dd26ce5dd0 con 0x55dd26d116d0 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0994200a00 con 0x55dd26ce3a70 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09941ffa80 con 0x55dd26d00950 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cb2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cb2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34480/0 (socket says 172.21.15.113:34480) 2024-04-25T00:07:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 -- 172.21.15.113:0/2350704099 learned_addr learned my addr 172.21.15.113:0/2350704099 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 -- 172.21.15.113:0/2350704099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 msgr2=0x55dd26cf7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 --2- 172.21.15.113:0/2350704099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26cf7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 -- 172.21.15.113:0/2350704099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 msgr2=0x55dd26cf7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 --2- 172.21.15.113:0/2350704099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cf7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 -- 172.21.15.113:0/2350704099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd269fa7d0 con 0x55dd26d116d0 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.784+0000 7f09937fe640 1 --2- 172.21.15.113:0/2350704099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cb2fd0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f0984006b10 tx=0x7f0984012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f06a196bca0ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f0973fff640 1 -- 172.21.15.113:0/2350704099 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984028070 con 0x55dd26d116d0 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f0973fff640 1 -- 172.21.15.113:0/2350704099 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09840113f0 con 0x55dd26d116d0 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f0973fff640 1 -- 172.21.15.113:0/2350704099 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984015070 con 0x55dd26d116d0 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 -- 172.21.15.113:0/2350704099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 msgr2=0x55dd26cb2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 --2- 172.21.15.113:0/2350704099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cb2fd0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f0984006b10 tx=0x7f0984012000 comp rx=0 tx=0).stop 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 -- 172.21.15.113:0/2350704099 shutdown_connections 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 --2- 172.21.15.113:0/2350704099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26d116d0 0x55dd26cb2fd0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 --2- 172.21.15.113:0/2350704099 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d00950 0x55dd26cf7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 --2- 172.21.15.113:0/2350704099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26ce3a70 0x55dd26cf7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 -- 172.21.15.113:0/2350704099 >> 172.21.15.113:0/2350704099 conn(0x55dd26ce3330 msgr2=0x55dd26ce64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 -- 172.21.15.113:0/2350704099 shutdown_connections 2024-04-25T00:07:57.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.785+0000 7f099c266740 1 -- 172.21.15.113:0/2350704099 wait complete. 2024-04-25T00:07:57.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 -- start start 2024-04-25T00:07:57.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 Processor -- start 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26ce3a70 0x55dd26cf3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26d00950 0x55dd26cf6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d116d0 0x55dd26d0abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09940ccfb0 con 0x55dd26ce3a70 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09940cc960 con 0x55dd26d00950 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.786+0000 7f099c266740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd26cf4190 con 0x55dd26d116d0 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 10 client.0 _ll_register_callbacks cb 0x7f098400a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26ce3a70 0x55dd26cf3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26ce3a70 0x55dd26cf3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34490/0 (socket says 172.21.15.113:34490) 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 -- 172.21.15.113:0/3433546420 learned_addr learned my addr 172.21.15.113:0/3433546420 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f09927fc640 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26d00950 0x55dd26cf6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f09937fe640 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d116d0 0x55dd26d0abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 -- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d116d0 msgr2=0x55dd26d0abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d116d0 0x55dd26d0abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 -- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26d00950 msgr2=0x55dd26cf6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26d00950 0x55dd26cf6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 -- 172.21.15.113:0/3433546420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd26cf3cc0 con 0x55dd26ce3a70 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0992ffd640 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26ce3a70 0x55dd26cf3780 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f098c07a5d0 tx=0x7f098c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f375c28ca29bedfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0971ffb640 1 -- 172.21.15.113:0/3433546420 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c0810b0 con 0x55dd26ce3a70 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0971ffb640 1 -- 172.21.15.113:0/3433546420 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f098c076f40 con 0x55dd26ce3a70 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09840113f0 con 0x55dd26ce3a70 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0971ffb640 1 -- 172.21.15.113:0/3433546420 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f098c0a0050 con 0x55dd26ce3a70 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 10 client.9768 fetch_fsmap learned FSMap version 78 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55dd26cf3ed0 con 0x55dd26ce3a70 2024-04-25T00:07:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0971ffb640 1 -- 172.21.15.113:0/3433546420 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09840113f0 con 0x55dd26ce3a70 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f0971ffb640 1 -- 172.21.15.113:0/3433546420 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f098c0a3020 con 0x55dd26ce3a70 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 10 client.9768 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 10 client.9768 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 20 client.9768 populate_metadata read hostname 'smithi113' 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26d11ca0 0x55dd26d0b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 4 client.9768 mds_command: new command op to 9501 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.787+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 --> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd26c23f00 con 0x55dd26d11ca0 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f09927fc640 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26d11ca0 0x55dd26d0b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f0971ffb640 10 client.9768 ms_handle_connect on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f09927fc640 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26d11ca0 0x55dd26d0b130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba4c48bae358703 server_cookie=38a1ee383bb68339 in_seq=0 out_seq=0 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f0971ffb640 1 -- 172.21.15.113:0/3433546420 <== mds.0 v2:172.21.15.113:6836/2111641286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd26c23f00 con 0x55dd26d11ca0 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f0971ffb640 10 client.9768 handle_command_reply: tid=0 2024-04-25T00:07:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 client.9768 shutdown 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26ce3a70 msgr2=0x55dd26cf3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26ce3a70 0x55dd26cf3780 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f098c07a5d0 tx=0x7f098c009750 comp rx=0 tx=0).stop 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 shutdown_connections 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:6836/2111641286,v1:172.21.15.113:6837/2111641286] conn(0x55dd26d11ca0 0x55dd26d0b130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd26d00950 0x55dd26cf6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd26d116d0 0x55dd26d0abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f0971ffb640 0 client.9768 ms_handle_reset on v2:172.21.15.113:6836/2111641286 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 --2- 172.21.15.113:0/3433546420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd26ce3a70 0x55dd26cf3780 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 >> 172.21.15.113:0/3433546420 conn(0x55dd26ce3330 msgr2=0x55dd26cfc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 shutdown_connections 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 1 -- 172.21.15.113:0/3433546420 wait complete. 2024-04-25T00:07:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.788+0000 7f099c266740 20 client.9768 trim_cache size 0 max 16384 2024-04-25T00:07:57.796 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-25T00:07:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f097406be30 msgr2=0x7f097406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f097406be30 0x7f097406e2f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f097c000f60 tx=0x7f097c007a80 comp rx=0 tx=0).stop 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09940b63b0 msgr2=0x7f0994149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09940b63b0 0x7f0994149580 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f098c009eb0 tx=0x7f098c004a40 comp rx=0 tx=0).stop 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 shutdown_connections 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f097406be30 0x7f097406e2f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f099415e510 0x7f09940b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09940d28c0 0x7f09940b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 --2- 172.21.15.113:0/2991925712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09940b63b0 0x7f0994149580 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 >> 172.21.15.113:0/2991925712 conn(0x7f099406b720 msgr2=0x7f099415bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 shutdown_connections 2024-04-25T00:07:57.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.789+0000 7f099955a640 1 -- 172.21.15.113:0/2991925712 wait complete. 2024-04-25T00:07:57.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T00:07:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.894+0000 7f3da11ce640 1 Processor -- start 2024-04-25T00:07:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.894+0000 7f3da11ce640 1 -- start start 2024-04-25T00:07:57.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3da11ce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3d9c144910 0x7f3d9c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3da11ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c144cf0 con 0x7f3d9c0b5240 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3da11ce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c0b2f30 con 0x7f3d9c0b56b0 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3da11ce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3d9c0b4510 con 0x7f3d9c144910 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d9a59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3d9c0b5240 0x7f3d9c06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60700/0 (socket says 172.21.15.113:60700) 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d9a59c640 1 -- 172.21.15.113:0/25272135 learned_addr learned my addr 172.21.15.113:0/25272135 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1191399940 0 0) 0x7f3d9c144cf0 con 0x7f3d9c0b5240 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d84003520 con 0x7f3d9c0b5240 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d9ad9d640 1 --2- 172.21.15.113:0/25272135 >> v2:172.21.15.178:3300/0 conn(0x7f3d9c144910 0x7f3d9c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3654563323 0 0) 0x7f3d84003520 con 0x7f3d9c0b5240 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 >> v2:172.21.15.178:3300/0 conn(0x7f3d9c144910 msgr2=0x7f3d9c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 --2- 172.21.15.113:0/25272135 >> v2:172.21.15.178:3300/0 conn(0x7f3d9c144910 0x7f3d9c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 >> v1:172.21.15.178:6789/0 conn(0x7f3d9c0b56b0 legacy=0x7f3d9c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c0b3210 con 0x7f3d9c0b5240 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f3d90001670 con 0x7f3d9c0b5240 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d9ad9d640 1 --2- 172.21.15.113:0/25272135 >> v2:172.21.15.178:3300/0 conn(0x7f3d9c144910 0x7f3d9c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.895+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 >> v1:172.21.15.113:6789/0 conn(0x7f3d9c0b5240 legacy=0x7f3d9c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d99d9b640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840039b0 0x7f3d84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d99d9b640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d84004800 0x7f3d84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d99d9b640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84005210 0x7f3d8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0b4510 con 0x7f3d84004800 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8400c140 con 0x7f3d84005210 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8400c340 con 0x7f3d840039b0 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9a59c640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d84004800 0x7f3d84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9ad9d640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840039b0 0x7f3d84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9b59e640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84005210 0x7f3d8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9a59c640 1 -- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840039b0 msgr2=0x7f3d84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9a59c640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840039b0 0x7f3d84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9a59c640 1 -- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84005210 msgr2=0x7f3d8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9a59c640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84005210 0x7f3d8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9b59e640 1 -- 172.21.15.113:0/25272135 reap_dead start 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.896+0000 7f3d9a59c640 1 -- 172.21.15.113:0/25272135 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84003520 con 0x7f3d84004800 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3d9a59c640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d84004800 0x7f3d84004c80 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007fd0 tx=0x7f3d84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed133fdb61418d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90007ce0 con 0x7f3d84004800 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d90005af0 con 0x7f3d84004800 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3d99d9b640 1 -- 172.21.15.113:0/25272135 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90007070 con 0x7f3d84004800 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3da11ce640 1 -- 172.21.15.113:0/25272135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d84004800 msgr2=0x7f3d84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3da11ce640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d84004800 0x7f3d84004c80 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007fd0 tx=0x7f3d84005830 comp rx=0 tx=0).stop 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3da11ce640 1 -- 172.21.15.113:0/25272135 shutdown_connections 2024-04-25T00:07:57.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3da11ce640 1 --2- 172.21.15.113:0/25272135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d84004800 0x7f3d84004c80 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3da11ce640 1 -- 172.21.15.113:0/25272135 >> 172.21.15.113:0/25272135 conn(0x7f3d9c0b1a50 msgr2=0x7f3d9c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3da11ce640 1 -- 172.21.15.113:0/25272135 shutdown_connections 2024-04-25T00:07:57.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.897+0000 7f3da11ce640 1 -- 172.21.15.113:0/25272135 wait complete. 2024-04-25T00:07:57.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 Processor -- start 2024-04-25T00:07:57.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 -- start start 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84004800 0x7f3d9c13ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d9c15d2d0 0x7f3d9c139cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0b04f0 0x7f3d9c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0b4da0 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0b37d0 con 0x7f3d84004800 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.899+0000 7f3da11ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9c0b4f20 con 0x7f3d9c15d2d0 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0b04f0 0x7f3d9c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0b04f0 0x7f3d9c0b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34504/0 (socket says 172.21.15.113:34504) 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 -- 172.21.15.113:0/1713491066 learned_addr learned my addr 172.21.15.113:0/1713491066 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9ad9d640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84004800 0x7f3d9c13ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9a59c640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d9c15d2d0 0x7f3d9c139cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 -- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d9c15d2d0 msgr2=0x7f3d9c139cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d9c15d2d0 0x7f3d9c139cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 -- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84004800 msgr2=0x7f3d9c13ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84004800 0x7f3d9c13ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 -- 172.21.15.113:0/1713491066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c13bf30 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9ad9d640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84004800 0x7f3d9c13ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d9b59e640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0b04f0 0x7f3d9c0b0970 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0752f0 tx=0x7f3d9400f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d837fe640 1 -- 172.21.15.113:0/1713491066 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9401a070 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d837fe640 1 -- 172.21.15.113:0/1713491066 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d94005280 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c13b970 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3d837fe640 1 -- 172.21.15.113:0/1713491066 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94007030 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.900+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c15deb0 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.901+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c0691a0 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.902+0000 7f3d837fe640 1 -- 172.21.15.113:0/1713491066 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d9401d070 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.902+0000 7f3d837fe640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d8806be30 0x7f3d8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.902+0000 7f3d837fe640 1 -- 172.21.15.113:0/1713491066 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f3d94089ea0 con 0x7f3d9c0b04f0 2024-04-25T00:07:57.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.902+0000 7f3d9ad9d640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d8806be30 0x7f3d8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.902+0000 7f3d9ad9d640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d8806be30 0x7f3d8806e2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c007520 tx=0x7f3d8c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:57.906+0000 7f3d837fe640 1 -- 172.21.15.113:0/1713491066 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d94058850 con 0x7f3d9c0b04f0 2024-04-25T00:07:58.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.118+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3d9c0731a0 con 0x7f3d9c0b04f0 2024-04-25T00:07:58.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.118+0000 7f3d837fe640 1 -- 172.21.15.113:0/1713491066 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3d94058150 con 0x7f3d9c0b04f0 2024-04-25T00:07:58.119 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:58.119 INFO:teuthology.orchestra.run.smithi113.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d8806be30 msgr2=0x7f3d8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d8806be30 0x7f3d8806e2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c007520 tx=0x7f3d8c00b040 comp rx=0 tx=0).stop 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0b04f0 msgr2=0x7f3d9c0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0b04f0 0x7f3d9c0b0970 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0752f0 tx=0x7f3d9400f890 comp rx=0 tx=0).stop 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 shutdown_connections 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3d8806be30 0x7f3d8806e2f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0b04f0 0x7f3d9c0b0970 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d9c15d2d0 0x7f3d9c139cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 --2- 172.21.15.113:0/1713491066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d84004800 0x7f3d9c13ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 >> 172.21.15.113:0/1713491066 conn(0x7f3d9c0b1a50 msgr2=0x7f3d9c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:58.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 shutdown_connections 2024-04-25T00:07:58.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.120+0000 7f3da11ce640 1 -- 172.21.15.113:0/1713491066 wait complete. 2024-04-25T00:07:58.132 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:07:58.132 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:07:58.169 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:07:58.207 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:07:58.207 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:58.207 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:58.207 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:58.224 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:58.224 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:58.279 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:07:58.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.414+0000 7f04cbc9a640 1 Processor -- start 2024-04-25T00:07:58.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.415+0000 7f04cbc9a640 1 -- start start 2024-04-25T00:07:58.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.415+0000 7f04cbc9a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f04c4144910 0x7f04c40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.415+0000 7f04cbc9a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04c40b5240 0x7f04c406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.415+0000 7f04cbc9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.415+0000 7f04cbc9a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f04c40b4510 con 0x7f04c40b5240 2024-04-25T00:07:58.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.415+0000 7f04cbc9a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f04c4144cf0 con 0x7f04c4144910 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.415+0000 7f04cbc9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40b2f30 con 0x7f04c40b56b0 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04c40b5240 0x7f04c406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04c40b5240 0x7f04c406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34512/0 (socket says 172.21.15.113:34512) 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 -- 172.21.15.113:0/2272415679 learned_addr learned my addr 172.21.15.113:0/2272415679 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9a36640 1 --2- 172.21.15.113:0/2272415679 >> v2:172.21.15.178:3300/0 conn(0x7f04c4144910 0x7f04c40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04ca237640 1 --2- 172.21.15.113:0/2272415679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 -- 172.21.15.113:0/2272415679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 msgr2=0x7f04c406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 --2- 172.21.15.113:0/2272415679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c406e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 -- 172.21.15.113:0/2272415679 >> v2:172.21.15.178:3300/0 conn(0x7f04c4144910 msgr2=0x7f04c40b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 --2- 172.21.15.113:0/2272415679 >> v2:172.21.15.178:3300/0 conn(0x7f04c4144910 0x7f04c40b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.416+0000 7f04c9235640 1 -- 172.21.15.113:0/2272415679 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c40b3210 con 0x7f04c40b5240 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04c9235640 1 --2- 172.21.15.113:0/2272415679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c406dfd0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009870 tx=0x7f04c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631aad084c42ca36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04c8a34640 1 -- 172.21.15.113:0/2272415679 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0037070 con 0x7f04c40b5240 2024-04-25T00:07:58.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04c8a34640 1 -- 172.21.15.113:0/2272415679 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c0032440 con 0x7f04c40b5240 2024-04-25T00:07:58.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04c8a34640 1 -- 172.21.15.113:0/2272415679 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0032810 con 0x7f04c40b5240 2024-04-25T00:07:58.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 -- 172.21.15.113:0/2272415679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 msgr2=0x7f04c406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/2272415679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c406dfd0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009870 tx=0x7f04c002a210 comp rx=0 tx=0).stop 2024-04-25T00:07:58.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 -- 172.21.15.113:0/2272415679 shutdown_connections 2024-04-25T00:07:58.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/2272415679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/2272415679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c406dfd0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/2272415679 >> v2:172.21.15.178:3300/0 conn(0x7f04c4144910 0x7f04c40b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 -- 172.21.15.113:0/2272415679 >> 172.21.15.113:0/2272415679 conn(0x7f04c40b1a50 msgr2=0x7f04c40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 -- 172.21.15.113:0/2272415679 shutdown_connections 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.417+0000 7f04cbc9a640 1 -- 172.21.15.113:0/2272415679 wait complete. 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.418+0000 7f04cbc9a640 1 Processor -- start 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04cbc9a640 1 -- start start 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04cbc9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c415d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04cbc9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c40ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04cbc9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c4144910 0x7f04c40ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04cbc9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40b4da0 con 0x7f04c40b5240 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04cbc9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40b37d0 con 0x7f04c4144910 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04cbc9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40b4f20 con 0x7f04c40b56b0 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04c9a36640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c415d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04c9a36640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c415d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34524/0 (socket says 172.21.15.113:34524) 2024-04-25T00:07:58.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.419+0000 7f04c9a36640 1 -- 172.21.15.113:0/4204258384 learned_addr learned my addr 172.21.15.113:0/4204258384 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04ca237640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c4144910 0x7f04c40ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04c9235640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c40ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04c9a36640 1 -- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 msgr2=0x7f04c40ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04c9a36640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c40ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04c9a36640 1 -- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c4144910 msgr2=0x7f04c40ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04c9a36640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c4144910 0x7f04c40ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04c9a36640 1 -- 172.21.15.113:0/4204258384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c4135420 con 0x7f04c40b5240 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04ca237640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c4144910 0x7f04c40ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04c9a36640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c415d970 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f04b800d800 tx=0x7f04b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04b67fc640 1 -- 172.21.15.113:0/4204258384 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b800b6c0 con 0x7f04c40b5240 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04b67fc640 1 -- 172.21.15.113:0/4204258384 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b8019070 con 0x7f04c40b5240 2024-04-25T00:07:58.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04b67fc640 1 -- 172.21.15.113:0/4204258384 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b800ba80 con 0x7f04c40b5240 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c4134c10 con 0x7f04c40b5240 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.420+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c40c55a0 con 0x7f04c40b5240 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.421+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c40691a0 con 0x7f04c40b5240 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.421+0000 7f04b67fc640 1 -- 172.21.15.113:0/4204258384 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04b800bc20 con 0x7f04c40b5240 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.422+0000 7f04b67fc640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a806be30 0x7f04a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.422+0000 7f04b67fc640 1 -- 172.21.15.113:0/4204258384 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f04b808b2a0 con 0x7f04c40b5240 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.422+0000 7f04c9235640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a806be30 0x7f04a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.422+0000 7f04c9235640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a806be30 0x7f04a806e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f04c0034000 tx=0x7f04c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.426+0000 7f04b67fc640 1 -- 172.21.15.113:0/4204258384 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f04b8059ce0 con 0x7f04c40b5240 2024-04-25T00:07:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.598+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04c40c4f70 con 0x7f04c40b5240 2024-04-25T00:07:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.599+0000 7f04b67fc640 1 -- 172.21.15.113:0/4204258384 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16727 (secure 0 0 0) 0x7f04b80595e0 con 0x7f04c40b5240 2024-04-25T00:07:58.599 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:07:58.600 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:21.999089+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:07:18.614948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:07:19.319166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:07:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a806be30 msgr2=0x7f04a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a806be30 0x7f04a806e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f04c0034000 tx=0x7f04c002e000 comp rx=0 tx=0).stop 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 msgr2=0x7f04c415d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c415d970 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f04b800d800 tx=0x7f04b800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 shutdown_connections 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f04a806be30 0x7f04a806e2f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04c4144910 0x7f04c40ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04c40b56b0 0x7f04c40ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 --2- 172.21.15.113:0/4204258384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04c40b5240 0x7f04c415d970 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 >> 172.21.15.113:0/4204258384 conn(0x7f04c40b1a50 msgr2=0x7f04c40af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 shutdown_connections 2024-04-25T00:07:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.601+0000 7f04cbc9a640 1 -- 172.21.15.113:0/4204258384 wait complete. 2024-04-25T00:07:58.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:07:18.614948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:07:19.319166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:07:58.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:07:58.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.713+0000 7f3eca396640 1 Processor -- start 2024-04-25T00:07:58.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.713+0000 7f3eca396640 1 -- start start 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.713+0000 7f3eca396640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3ec415e510 0x7f3ec40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.713+0000 7f3eca396640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ec40b63b0 0x7f3ec40b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.713+0000 7f3eca396640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec40d2e00 con 0x7f3ec40d28c0 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.713+0000 7f3eca396640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec40d30e0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.713+0000 7f3eca396640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec40d33c0 con 0x7f3ec415e510 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec8933640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ec40b63b0 0x7f3ec40b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec8933640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ec40b63b0 0x7f3ec40b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34534/0 (socket says 172.21.15.113:34534) 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3ec40d28c0 0x7f3ec40b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60706/0 (socket says 172.21.15.113:60706) 2024-04-25T00:07:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec8933640 1 -- 172.21.15.113:0/378083996 learned_addr learned my addr 172.21.15.113:0/378083996 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec3fff640 1 --2- 172.21.15.113:0/378083996 >> v2:172.21.15.178:3300/0 conn(0x7f3ec415e510 0x7f3ec40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec8933640 1 -- 172.21.15.113:0/378083996 >> v2:172.21.15.178:3300/0 conn(0x7f3ec415e510 msgr2=0x7f3ec40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec2ffd640 1 -- 172.21.15.113:0/378083996 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837042436 0 0) 0x7f3ec40d2e00 con 0x7f3ec40d28c0 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec8933640 1 --2- 172.21.15.113:0/378083996 >> v2:172.21.15.178:3300/0 conn(0x7f3ec415e510 0x7f3ec40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec8933640 1 -- 172.21.15.113:0/378083996 >> v1:172.21.15.113:6789/0 conn(0x7f3ec40d28c0 legacy=0x7f3ec40b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec8933640 1 -- 172.21.15.113:0/378083996 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec40d36a0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.714+0000 7f3ec3fff640 1 --2- 172.21.15.113:0/378083996 >> v2:172.21.15.178:3300/0 conn(0x7f3ec415e510 0x7f3ec40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3ec8933640 1 --2- 172.21.15.113:0/378083996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec40b73c0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40d52a0 tx=0x7f3ebc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f99cc23f5fe57a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3ec2ffd640 1 -- 172.21.15.113:0/378083996 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc036070 con 0x7f3ec40b63b0 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3ec2ffd640 1 -- 172.21.15.113:0/378083996 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc031600 con 0x7f3ec40b63b0 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3ec2ffd640 1 -- 172.21.15.113:0/378083996 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0319d0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 -- 172.21.15.113:0/378083996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 msgr2=0x7f3ec40b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 --2- 172.21.15.113:0/378083996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec40b73c0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40d52a0 tx=0x7f3ebc02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 -- 172.21.15.113:0/378083996 shutdown_connections 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 --2- 172.21.15.113:0/378083996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec40b73c0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 --2- 172.21.15.113:0/378083996 >> v2:172.21.15.178:3300/0 conn(0x7f3ec415e510 0x7f3ec40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 -- 172.21.15.113:0/378083996 >> 172.21.15.113:0/378083996 conn(0x7f3ec406b720 msgr2=0x7f3ec40d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 -- 172.21.15.113:0/378083996 shutdown_connections 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.715+0000 7f3eca396640 1 -- 172.21.15.113:0/378083996 wait complete. 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.716+0000 7f3eca396640 1 Processor -- start 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3eca396640 1 -- start start 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3eca396640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec41f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3eca396640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ec40d28c0 0x7f3ec40cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3eca396640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ec415e510 0x7f3ec40b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3eca396640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec40b9dd0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3eca396640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec40d3ea0 con 0x7f3ec415e510 2024-04-25T00:07:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3eca396640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec40b7900 con 0x7f3ec40d28c0 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3ec3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec41f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3ec3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec41f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34544/0 (socket says 172.21.15.113:34544) 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.717+0000 7f3ec3fff640 1 -- 172.21.15.113:0/77075645 learned_addr learned my addr 172.21.15.113:0/77075645 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec8933640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ec415e510 0x7f3ec40b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec37fe640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ec40d28c0 0x7f3ec40cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec3fff640 1 -- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ec40d28c0 msgr2=0x7f3ec40cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec3fff640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ec40d28c0 0x7f3ec40cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec3fff640 1 -- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ec415e510 msgr2=0x7f3ec40b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec3fff640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ec415e510 0x7f3ec40b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec3fff640 1 -- 172.21.15.113:0/77075645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec41f4b60 con 0x7f3ec40b63b0 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec8933640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ec415e510 0x7f3ec40b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec3fff640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec41f6a00 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400c820 tx=0x7f3eb400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec0ff9640 1 -- 172.21.15.113:0/77075645 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4015070 con 0x7f3ec40b63b0 2024-04-25T00:07:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec0ff9640 1 -- 172.21.15.113:0/77075645 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb4007a30 con 0x7f3ec40b63b0 2024-04-25T00:07:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec41f6f40 con 0x7f3ec40b63b0 2024-04-25T00:07:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3ec0ff9640 1 -- 172.21.15.113:0/77075645 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4007de0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.718+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec41f42c0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.719+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec40691a0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.720+0000 7f3ec0ff9640 1 -- 172.21.15.113:0/77075645 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3eb4018070 con 0x7f3ec40b63b0 2024-04-25T00:07:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.720+0000 7f3ec0ff9640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb006be30 0x7f3eb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.720+0000 7f3ec0ff9640 1 -- 172.21.15.113:0/77075645 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f3eb408acf0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.720+0000 7f3ec37fe640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb006be30 0x7f3eb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.720+0000 7f3ec37fe640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb006be30 0x7f3eb006e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40b8840 tx=0x7f3eb8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.724+0000 7f3ec0ff9640 1 -- 172.21.15.113:0/77075645 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3eb408f070 con 0x7f3ec40b63b0 2024-04-25T00:07:58.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.895+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ec406c100 con 0x7f3ec40b63b0 2024-04-25T00:07:58.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.896+0000 7f3ec0ff9640 1 -- 172.21.15.113:0/77075645 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7f3eb40596a0 con 0x7f3ec40b63b0 2024-04-25T00:07:58.896 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:07:58.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb006be30 msgr2=0x7f3eb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb006be30 0x7f3eb006e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40b8840 tx=0x7f3eb8009040 comp rx=0 tx=0).stop 2024-04-25T00:07:58.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 msgr2=0x7f3ec41f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec41f6a00 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400c820 tx=0x7f3eb400cce0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 shutdown_connections 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3eb006be30 0x7f3eb006e2f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ec415e510 0x7f3ec40b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ec40d28c0 0x7f3ec40cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 --2- 172.21.15.113:0/77075645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ec40b63b0 0x7f3ec41f6a00 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 >> 172.21.15.113:0/77075645 conn(0x7f3ec406b720 msgr2=0x7f3ec415cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 shutdown_connections 2024-04-25T00:07:58.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:58.898+0000 7f3eca396640 1 -- 172.21.15.113:0/77075645 wait complete. 2024-04-25T00:07:58.909 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:07:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.008+0000 7f1c33283640 1 Processor -- start 2024-04-25T00:07:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.008+0000 7f1c33283640 1 -- start start 2024-04-25T00:07:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c33283640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c2c162870 0x7f1c2c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c33283640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c072320 con 0x7f1c2c071cc0 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c33283640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c072600 con 0x7f1c2c0d0510 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c33283640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c162fe0 con 0x7f1c2c162870 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c3101f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c2c162870 0x7f1c2c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c31820640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1c2c071cc0 0x7f1c2c1637e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60716/0 (socket says 172.21.15.113:60716) 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c31820640 1 -- 172.21.15.113:0/3297092179 learned_addr learned my addr 172.21.15.113:0/3297092179 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c3101f640 1 -- 172.21.15.113:0/3297092179 >> v1:172.21.15.178:6789/0 conn(0x7f1c2c0d0510 legacy=0x7f1c2c0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c3101f640 1 -- 172.21.15.113:0/3297092179 >> v1:172.21.15.113:6789/0 conn(0x7f1c2c071cc0 legacy=0x7f1c2c1637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c23fff640 1 -- 172.21.15.113:0/3297092179 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644579695 0 0) 0x7f1c2c072320 con 0x7f1c2c071cc0 2024-04-25T00:07:59.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.009+0000 7f1c3101f640 1 -- 172.21.15.113:0/3297092179 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c1632c0 con 0x7f1c2c162870 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c3101f640 1 --2- 172.21.15.113:0/3297092179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c162870 0x7f1c2c0cffd0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009d90 tx=0x7f1c2402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e759d69dd8ce6178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c23fff640 1 -- 172.21.15.113:0/3297092179 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24031600 con 0x7f1c2c162870 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c23fff640 1 -- 172.21.15.113:0/3297092179 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c240317a0 con 0x7f1c2c162870 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c23fff640 1 -- 172.21.15.113:0/3297092179 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24031b70 con 0x7f1c2c162870 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c33283640 1 -- 172.21.15.113:0/3297092179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c162870 msgr2=0x7f1c2c0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c33283640 1 --2- 172.21.15.113:0/3297092179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c162870 0x7f1c2c0cffd0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009d90 tx=0x7f1c2402a610 comp rx=0 tx=0).stop 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c33283640 1 -- 172.21.15.113:0/3297092179 shutdown_connections 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.010+0000 7f1c33283640 1 --2- 172.21.15.113:0/3297092179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c162870 0x7f1c2c0cffd0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.011+0000 7f1c33283640 1 -- 172.21.15.113:0/3297092179 >> 172.21.15.113:0/3297092179 conn(0x7f1c2c06b0f0 msgr2=0x7f1c2c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.011+0000 7f1c33283640 1 -- 172.21.15.113:0/3297092179 shutdown_connections 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.011+0000 7f1c33283640 1 -- 172.21.15.113:0/3297092179 wait complete. 2024-04-25T00:07:59.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 Processor -- start 2024-04-25T00:07:59.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 -- start start 2024-04-25T00:07:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2c071cc0 0x7f1c2c148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c0d0510 0x7f1c2c0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c2c162870 0x7f1c2c0ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c164070 con 0x7f1c2c0d0510 2024-04-25T00:07:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c163ef0 con 0x7f1c2c162870 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c33283640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c1641f0 con 0x7f1c2c071cc0 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.012+0000 7f1c3081e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c0d0510 0x7f1c2c0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c0d0510 0x7f1c2c0abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34556/0 (socket says 172.21.15.113:34556) 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 -- 172.21.15.113:0/4194471938 learned_addr learned my addr 172.21.15.113:0/4194471938 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c31820640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c2c162870 0x7f1c2c0ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3101f640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2c071cc0 0x7f1c2c148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 -- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2c071cc0 msgr2=0x7f1c2c148c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2c071cc0 0x7f1c2c148c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 -- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c2c162870 msgr2=0x7f1c2c0ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c2c162870 0x7f1c2c0ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 -- 172.21.15.113:0/4194471938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c149180 con 0x7f1c2c0d0510 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3101f640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2c071cc0 0x7f1c2c148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c3081e640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c0d0510 0x7f1c2c0abc30 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c410 tx=0x7f1c1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c21ffb640 1 -- 172.21.15.113:0/4194471938 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c017070 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c21ffb640 1 -- 172.21.15.113:0/4194471938 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0035f0 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c1edbc0 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c21ffb640 1 -- 172.21.15.113:0/4194471938 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c003a20 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.013+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c149930 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.014+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c0691a0 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.015+0000 7f1c21ffb640 1 -- 172.21.15.113:0/4194471938 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c1c01a070 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.015+0000 7f1c21ffb640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c1006be30 0x7f1c1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.015+0000 7f1c21ffb640 1 -- 172.21.15.113:0/4194471938 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f1c1c08a300 con 0x7f1c2c0d0510 2024-04-25T00:07:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.015+0000 7f1c3101f640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c1006be30 0x7f1c1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.015+0000 7f1c3101f640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c1006be30 0x7f1c1006e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c2402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.019+0000 7f1c21ffb640 1 -- 172.21.15.113:0/4194471938 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c1c058cb0 con 0x7f1c2c0d0510 2024-04-25T00:07:59.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.193+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c2c1654f0 con 0x7f1c2c0d0510 2024-04-25T00:07:59.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.194+0000 7f1c21ffb640 1 -- 172.21.15.113:0/4194471938 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7f1c1c0585b0 con 0x7f1c2c0d0510 2024-04-25T00:07:59.195 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:07:59.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c1006be30 msgr2=0x7f1c1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c1006be30 0x7f1c1006e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c2402f000 comp rx=0 tx=0).stop 2024-04-25T00:07:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c0d0510 msgr2=0x7f1c2c0abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c0d0510 0x7f1c2c0abc30 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c410 tx=0x7f1c1c00c8d0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 shutdown_connections 2024-04-25T00:07:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1c1006be30 0x7f1c1006e2f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c2c162870 0x7f1c2c0ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c2c0d0510 0x7f1c2c0abc30 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 --2- 172.21.15.113:0/4194471938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c2c071cc0 0x7f1c2c148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 >> 172.21.15.113:0/4194471938 conn(0x7f1c2c06b0f0 msgr2=0x7f1c2c1645d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 shutdown_connections 2024-04-25T00:07:59.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.196+0000 7f1c33283640 1 -- 172.21.15.113:0/4194471938 wait complete. 2024-04-25T00:07:59.207 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:07:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.306+0000 7f8284c8c640 1 Processor -- start 2024-04-25T00:07:59.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f8284c8c640 1 -- start start 2024-04-25T00:07:59.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f8284c8c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f828015e2d0 0x7f828015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f8284c8c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82800b4a40 0x7f82800ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f8284c8c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f828015f4f0 con 0x7f82800b3080 2024-04-25T00:07:59.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f8284c8c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f82800cde00 con 0x7f828015e2d0 2024-04-25T00:07:59.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f8284c8c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f828015f7d0 con 0x7f82800b4a40 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f827e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f828015e2d0 0x7f828015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f827e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f828015e2d0 0x7f828015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34570/0 (socket says 172.21.15.113:34570) 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f827e59c640 1 -- 172.21.15.113:0/286812677 learned_addr learned my addr 172.21.15.113:0/286812677 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.307+0000 7f8277fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f82800b3080 0x7f82800b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60728/0 (socket says 172.21.15.113:60728) 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.308+0000 7f827e59c640 1 -- 172.21.15.113:0/286812677 >> v2:172.21.15.178:3300/0 conn(0x7f82800b4a40 msgr2=0x7f82800ce290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.308+0000 7f827e59c640 1 --2- 172.21.15.113:0/286812677 >> v2:172.21.15.178:3300/0 conn(0x7f82800b4a40 0x7f82800ce290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.308+0000 7f827e59c640 1 -- 172.21.15.113:0/286812677 >> v1:172.21.15.113:6789/0 conn(0x7f82800b3080 legacy=0x7f82800b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.308+0000 7f827e59c640 1 -- 172.21.15.113:0/286812677 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828015fab0 con 0x7f828015e2d0 2024-04-25T00:07:59.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.308+0000 7f827dd9b640 1 -- 172.21.15.113:0/286812677 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051889782 0 0) 0x7f828015f4f0 con 0x7f82800b3080 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.308+0000 7f827e59c640 1 --2- 172.21.15.113:0/286812677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f828015ebf0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f82800d0f20 tx=0x7f827002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfe1f0df55aed87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.308+0000 7f827dd9b640 1 -- 172.21.15.113:0/286812677 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270037070 con 0x7f828015e2d0 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f827dd9b640 1 -- 172.21.15.113:0/286812677 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270032440 con 0x7f828015e2d0 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f827dd9b640 1 -- 172.21.15.113:0/286812677 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82700327f0 con 0x7f828015e2d0 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 -- 172.21.15.113:0/286812677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 msgr2=0x7f828015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 --2- 172.21.15.113:0/286812677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f828015ebf0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f82800d0f20 tx=0x7f827002a290 comp rx=0 tx=0).stop 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 -- 172.21.15.113:0/286812677 shutdown_connections 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 --2- 172.21.15.113:0/286812677 >> v2:172.21.15.178:3300/0 conn(0x7f82800b4a40 0x7f82800ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 --2- 172.21.15.113:0/286812677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f828015ebf0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 -- 172.21.15.113:0/286812677 >> 172.21.15.113:0/286812677 conn(0x7f828006b0f0 msgr2=0x7f828006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 -- 172.21.15.113:0/286812677 shutdown_connections 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.309+0000 7f8284c8c640 1 -- 172.21.15.113:0/286812677 wait complete. 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.310+0000 7f8284c8c640 1 Processor -- start 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.310+0000 7f8284c8c640 1 -- start start 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f8284c8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82800b3080 0x7f82800ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f8284c8c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82800b4a40 0x7f82801567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f8284c8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f82801487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f8284c8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f828015f2b0 con 0x7f828015e2d0 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f8284c8c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f828015f130 con 0x7f82800b4a40 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f8284c8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f828015e6b0 con 0x7f82800b3080 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f82801487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f82801487a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34578/0 (socket says 172.21.15.113:34578) 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827ed9d640 1 -- 172.21.15.113:0/3084169607 learned_addr learned my addr 172.21.15.113:0/3084169607 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f8277fff640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82800b4a40 0x7f82801567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827e59c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82800b3080 0x7f82800ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827ed9d640 1 -- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82800b3080 msgr2=0x7f82800ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827ed9d640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82800b3080 0x7f82800ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827ed9d640 1 -- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82800b4a40 msgr2=0x7f82801567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.311+0000 7f827ed9d640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82800b4a40 0x7f82801567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f827ed9d640 1 -- 172.21.15.113:0/3084169607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8280157100 con 0x7f828015e2d0 2024-04-25T00:07:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f827e59c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82800b3080 0x7f82800ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f8277fff640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82800b4a40 0x7f82801567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f827ed9d640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f82801487a0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f827800ba10 tx=0x7f827800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f8276ffd640 1 -- 172.21.15.113:0/3084169607 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278019070 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f8276ffd640 1 -- 172.21.15.113:0/3084169607 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827800cba0 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f8276ffd640 1 -- 172.21.15.113:0/3084169607 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278014440 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82800ad930 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.312+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82801573e0 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.313+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82800cc420 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.313+0000 7f8276ffd640 1 -- 172.21.15.113:0/3084169607 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8278003a10 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.314+0000 7f8276ffd640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f826006be60 0x7f826006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.314+0000 7f8276ffd640 1 -- 172.21.15.113:0/3084169607 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f827808aa10 con 0x7f828015e2d0 2024-04-25T00:07:59.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.314+0000 7f827e59c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f826006be60 0x7f826006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.314+0000 7f827e59c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f826006be60 0x7f826006e320 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8270034000 tx=0x7f827002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.318+0000 7f8276ffd640 1 -- 172.21.15.113:0/3084169607 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82800cc420 con 0x7f828015e2d0 2024-04-25T00:07:59.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.492+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82800cc970 con 0x7f828015e2d0 2024-04-25T00:07:59.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.492+0000 7f8276ffd640 1 -- 172.21.15.113:0/3084169607 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7f82780593c0 con 0x7f828015e2d0 2024-04-25T00:07:59.493 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:07:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f826006be60 msgr2=0x7f826006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f826006be60 0x7f826006e320 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8270034000 tx=0x7f827002e000 comp rx=0 tx=0).stop 2024-04-25T00:07:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 msgr2=0x7f82801487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f82801487a0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f827800ba10 tx=0x7f827800bed0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 shutdown_connections 2024-04-25T00:07:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f826006be60 0x7f826006e320 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82800b4a40 0x7f82801567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f828015e2d0 0x7f82801487a0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 --2- 172.21.15.113:0/3084169607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82800b3080 0x7f82800ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 >> 172.21.15.113:0/3084169607 conn(0x7f828006b0f0 msgr2=0x7f828006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 shutdown_connections 2024-04-25T00:07:59.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.494+0000 7f8284c8c640 1 -- 172.21.15.113:0/3084169607 wait complete. 2024-04-25T00:07:59.506 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:07:59.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.603+0000 7fbf299a2640 1 Processor -- start 2024-04-25T00:07:59.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.603+0000 7fbf299a2640 1 -- start start 2024-04-25T00:07:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf299a2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf240cf3c0 0x7fbf240d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf299a2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf7a0 0x7fbf240d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf299a2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf24162560 con 0x7fbf240d2a20 2024-04-25T00:07:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf299a2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbf24162840 con 0x7fbf240cf3c0 2024-04-25T00:07:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf299a2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf24162b20 con 0x7fbf240cf7a0 2024-04-25T00:07:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf22ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf240cf3c0 0x7fbf240d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf237fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbf240d2a20 0x7fbf240d1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60740/0 (socket says 172.21.15.113:60740) 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf237fe640 1 -- 172.21.15.113:0/335822817 learned_addr learned my addr 172.21.15.113:0/335822817 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/335822817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf7a0 0x7fbf240d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf22ffd640 1 -- 172.21.15.113:0/335822817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf7a0 msgr2=0x7fbf240d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf22ffd640 1 --2- 172.21.15.113:0/335822817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf7a0 0x7fbf240d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf22ffd640 1 -- 172.21.15.113:0/335822817 >> v1:172.21.15.113:6789/0 conn(0x7fbf240d2a20 legacy=0x7fbf240d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf22ffd640 1 -- 172.21.15.113:0/335822817 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf240d40f0 con 0x7fbf240cf3c0 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf227fc640 1 -- 172.21.15.113:0/335822817 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 720937868 0 0) 0x7fbf24162560 con 0x7fbf240d2a20 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.604+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/335822817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf7a0 0x7fbf240d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.605+0000 7fbf22ffd640 1 --2- 172.21.15.113:0/335822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240cf3c0 0x7fbf240d3bb0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097a0 tx=0x7fbf1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dfec1340888c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.605+0000 7fbf227fc640 1 -- 172.21.15.113:0/335822817 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14037070 con 0x7fbf240cf3c0 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.605+0000 7fbf227fc640 1 -- 172.21.15.113:0/335822817 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf14032440 con 0x7fbf240cf3c0 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.605+0000 7fbf227fc640 1 -- 172.21.15.113:0/335822817 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14032810 con 0x7fbf240cf3c0 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 -- 172.21.15.113:0/335822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240cf3c0 msgr2=0x7fbf240d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 --2- 172.21.15.113:0/335822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240cf3c0 0x7fbf240d3bb0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097a0 tx=0x7fbf1402a180 comp rx=0 tx=0).stop 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 -- 172.21.15.113:0/335822817 shutdown_connections 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 --2- 172.21.15.113:0/335822817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf7a0 0x7fbf240d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 --2- 172.21.15.113:0/335822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240cf3c0 0x7fbf240d3bb0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 -- 172.21.15.113:0/335822817 >> 172.21.15.113:0/335822817 conn(0x7fbf240718f0 msgr2=0x7fbf24071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 -- 172.21.15.113:0/335822817 shutdown_connections 2024-04-25T00:07:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.606+0000 7fbf299a2640 1 -- 172.21.15.113:0/335822817 wait complete. 2024-04-25T00:07:59.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.607+0000 7fbf299a2640 1 Processor -- start 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.607+0000 7fbf299a2640 1 -- start start 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.607+0000 7fbf299a2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf3c0 0x7fbf24156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.607+0000 7fbf299a2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf240cf7a0 0x7fbf241f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.607+0000 7fbf299a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240d2a20 0x7fbf240ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf299a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240d1aa0 con 0x7fbf240d2a20 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf299a2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240d1920 con 0x7fbf240cf7a0 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf299a2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf24161fc0 con 0x7fbf240cf3c0 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240d2a20 0x7fbf240ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240d2a20 0x7fbf240ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34596/0 (socket says 172.21.15.113:34596) 2024-04-25T00:07:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 -- 172.21.15.113:0/609188420 learned_addr learned my addr 172.21.15.113:0/609188420 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf22ffd640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf3c0 0x7fbf24156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf240cf7a0 0x7fbf241f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 -- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf3c0 msgr2=0x7fbf24156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf3c0 0x7fbf24156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 -- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf240cf7a0 msgr2=0x7fbf241f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf240cf7a0 0x7fbf241f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 -- 172.21.15.113:0/609188420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf24157230 con 0x7fbf240d2a20 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf237fe640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240d2a20 0x7fbf240ce680 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00c700 tx=0x7fbf1c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf20ff9640 1 -- 172.21.15.113:0/609188420 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c018070 con 0x7fbf240d2a20 2024-04-25T00:07:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf20ff9640 1 -- 172.21.15.113:0/609188420 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c00ce10 con 0x7fbf240d2a20 2024-04-25T00:07:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf240cebc0 con 0x7fbf240d2a20 2024-04-25T00:07:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf20ff9640 1 -- 172.21.15.113:0/609188420 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00a880 con 0x7fbf240d2a20 2024-04-25T00:07:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.608+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf240cb940 con 0x7fbf240d2a20 2024-04-25T00:07:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.609+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf240691a0 con 0x7fbf240d2a20 2024-04-25T00:07:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.610+0000 7fbf20ff9640 1 -- 172.21.15.113:0/609188420 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf1c01b070 con 0x7fbf240d2a20 2024-04-25T00:07:59.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.610+0000 7fbf20ff9640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0406be30 0x7fbf0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.610+0000 7fbf22ffd640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0406be30 0x7fbf0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.610+0000 7fbf20ff9640 1 -- 172.21.15.113:0/609188420 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7fbf1c08a2d0 con 0x7fbf240d2a20 2024-04-25T00:07:59.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.610+0000 7fbf22ffd640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0406be30 0x7fbf0406e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbf14034000 tx=0x7fbf1402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.614+0000 7fbf20ff9640 1 -- 172.21.15.113:0/609188420 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf1c058d10 con 0x7fbf240d2a20 2024-04-25T00:07:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.798+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf2406c100 con 0x7fbf240d2a20 2024-04-25T00:07:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.799+0000 7fbf20ff9640 1 -- 172.21.15.113:0/609188420 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v144) ==== 80+0+0 (secure 0 0 0) 0x7fbf1c058610 con 0x7fbf240d2a20 2024-04-25T00:07:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:07:59.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0406be30 msgr2=0x7fbf0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0406be30 0x7fbf0406e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbf14034000 tx=0x7fbf1402e000 comp rx=0 tx=0).stop 2024-04-25T00:07:59.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240d2a20 msgr2=0x7fbf240ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240d2a20 0x7fbf240ce680 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00c700 tx=0x7fbf1c00cbc0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 shutdown_connections 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0406be30 0x7fbf0406e2f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf240cf7a0 0x7fbf241f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf240d2a20 0x7fbf240ce680 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 --2- 172.21.15.113:0/609188420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf240cf3c0 0x7fbf24156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 >> 172.21.15.113:0/609188420 conn(0x7fbf240718f0 msgr2=0x7fbf2415fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 shutdown_connections 2024-04-25T00:07:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:07:59.801+0000 7fbf299a2640 1 -- 172.21.15.113:0/609188420 wait complete. 2024-04-25T00:07:59.812 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:07:59.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:59.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:07:59.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:07:59.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:07:59.839 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:07:59.894 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:59.894 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:07:59.894 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:59.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:07:59.913 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:59.914 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:07:59.914 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:59.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:07:59.950 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:07:59.950 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:07:59.950 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:07:59.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:07:59.971 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:00.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.087+0000 7f121ab89640 1 Processor -- start 2024-04-25T00:08:00.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.087+0000 7f121ab89640 1 -- start start 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f121ab89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f121ab89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12140b3af0 0x7f12140b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f121ab89640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12140b2d50 con 0x7f1214072ea0 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f121ab89640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1214072a70 con 0x7f12140b3af0 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f121ab89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12140b3010 con 0x7f121415a6b0 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f1219126640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12140b3af0 0x7f12140b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f120bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1214072ea0 0x7f1214071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60752/0 (socket says 172.21.15.113:60752) 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f120bfff640 1 -- 172.21.15.113:0/2024161534 learned_addr learned my addr 172.21.15.113:0/2024161534 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f1218925640 1 --2- 172.21.15.113:0/2024161534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f1219126640 1 -- 172.21.15.113:0/2024161534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 msgr2=0x7f1214070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f1219126640 1 --2- 172.21.15.113:0/2024161534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f1219126640 1 -- 172.21.15.113:0/2024161534 >> v1:172.21.15.113:6789/0 conn(0x7f1214072ea0 legacy=0x7f1214071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f1219126640 1 -- 172.21.15.113:0/2024161534 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12140b32f0 con 0x7f12140b3af0 2024-04-25T00:08:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.088+0000 7f120b7fe640 1 -- 172.21.15.113:0/2024161534 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497059887 0 0) 0x7f12140b2d50 con 0x7f1214072ea0 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.089+0000 7f1219126640 1 --2- 172.21.15.113:0/2024161534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12140b3af0 0x7f12140b3f80 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f120c009d90 tx=0x7f120c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f6b6bb04d5ecf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.089+0000 7f120b7fe640 1 -- 172.21.15.113:0/2024161534 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c007740 con 0x7f12140b3af0 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.089+0000 7f120b7fe640 1 -- 172.21.15.113:0/2024161534 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120c0078e0 con 0x7f12140b3af0 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.089+0000 7f120b7fe640 1 -- 172.21.15.113:0/2024161534 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c007cb0 con 0x7f12140b3af0 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 -- 172.21.15.113:0/2024161534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12140b3af0 msgr2=0x7f12140b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 --2- 172.21.15.113:0/2024161534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12140b3af0 0x7f12140b3f80 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f120c009d90 tx=0x7f120c02a660 comp rx=0 tx=0).stop 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 -- 172.21.15.113:0/2024161534 shutdown_connections 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 --2- 172.21.15.113:0/2024161534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12140b3af0 0x7f12140b3f80 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 --2- 172.21.15.113:0/2024161534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 -- 172.21.15.113:0/2024161534 >> 172.21.15.113:0/2024161534 conn(0x7f12140b1a50 msgr2=0x7f12140b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 -- 172.21.15.113:0/2024161534 shutdown_connections 2024-04-25T00:08:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.090+0000 7f121ab89640 1 -- 172.21.15.113:0/2024161534 wait complete. 2024-04-25T00:08:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.091+0000 7f121ab89640 1 Processor -- start 2024-04-25T00:08:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.091+0000 7f121ab89640 1 -- start start 2024-04-25T00:08:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f121ab89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1214072ea0 0x7f121414a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f121ab89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12140b3af0 0x7f1214070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f121ab89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f121ab89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f121415aa90 con 0x7f1214072ea0 2024-04-25T00:08:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f121ab89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1214072720 con 0x7f12140b3af0 2024-04-25T00:08:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f121ab89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f121415ac10 con 0x7f121415a6b0 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1214072ea0 0x7f121414a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1214072ea0 0x7f121414a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34606/0 (socket says 172.21.15.113:34606) 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 -- 172.21.15.113:0/3734007794 learned_addr learned my addr 172.21.15.113:0/3734007794 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1219126640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f120bfff640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12140b3af0 0x7f1214070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 -- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 msgr2=0x7f1214071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 -- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12140b3af0 msgr2=0x7f1214070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12140b3af0 0x7f1214070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 -- 172.21.15.113:0/3734007794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12141560c0 con 0x7f1214072ea0 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.092+0000 7f1218925640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1214072ea0 0x7f121414a870 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f121000d5f0 tx=0x7f121000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.094+0000 7f12097fa640 1 -- 172.21.15.113:0/3734007794 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12100125f0 con 0x7f1214072ea0 2024-04-25T00:08:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.094+0000 7f12097fa640 1 -- 172.21.15.113:0/3734007794 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1210019070 con 0x7f1214072ea0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.094+0000 7f12097fa640 1 -- 172.21.15.113:0/3734007794 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12100129d0 con 0x7f1214072ea0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.094+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141563a0 con 0x7f1214072ea0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.094+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141f52f0 con 0x7f1214072ea0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.095+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12140691a0 con 0x7f1214072ea0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.096+0000 7f12097fa640 1 -- 172.21.15.113:0/3734007794 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1210012b70 con 0x7f1214072ea0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.096+0000 7f12097fa640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f120006be30 0x7f120006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.096+0000 7f12097fa640 1 -- 172.21.15.113:0/3734007794 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f121008b0b0 con 0x7f1214072ea0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.096+0000 7f120bfff640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f120006be30 0x7f120006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.096+0000 7f120bfff640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f120006be30 0x7f120006e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f12140cc070 tx=0x7f1204007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.100+0000 7f12097fa640 1 -- 172.21.15.113:0/3734007794 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1210059410 con 0x7f1214072ea0 2024-04-25T00:08:00.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.310+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12140b2900 con 0x7f1214072ea0 2024-04-25T00:08:00.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.311+0000 7f12097fa640 1 -- 172.21.15.113:0/3734007794 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5197 (secure 0 0 0) 0x7f121005d710 con 0x7f1214072ea0 2024-04-25T00:08:00.312 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:00.312 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1199283489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1199283489},{"type":"v1","addr":"172.21.15.178:6834","nonce":1199283489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4259795043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4259795043},{"type":"v1","addr":"172.21.15.178:6835","nonce":4259795043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4146002756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4146002756},{"type":"v1","addr":"172.21.15.113:6835","nonce":4146002756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:07:20.361298+0000","modified":"2024-04-25T00:07:22.348971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2111641286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2111641286},{"type":"v1","addr":"172.21.15.113:6837","nonce":2111641286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":10}]} 2024-04-25T00:08:00.312 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-04-25T00:08:00.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f120006be30 msgr2=0x7f120006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f120006be30 0x7f120006e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f12140cc070 tx=0x7f1204007d50 comp rx=0 tx=0).stop 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1214072ea0 msgr2=0x7f121414a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1214072ea0 0x7f121414a870 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f121000d5f0 tx=0x7f121000dab0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 shutdown_connections 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f120006be30 0x7f120006e2f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12140b3af0 0x7f1214070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f121415a6b0 0x7f1214071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 --2- 172.21.15.113:0/3734007794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1214072ea0 0x7f121414a870 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 >> 172.21.15.113:0/3734007794 conn(0x7f12140b1a50 msgr2=0x7f1214153b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 shutdown_connections 2024-04-25T00:08:00.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.313+0000 7f121ab89640 1 -- 172.21.15.113:0/3734007794 wait complete. 2024-04-25T00:08:00.325 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:00.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.421+0000 7f29fc264640 1 Processor -- start 2024-04-25T00:08:00.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.422+0000 7f29fc264640 1 -- start start 2024-04-25T00:08:00.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.422+0000 7f29fc264640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.422+0000 7f29fc264640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29f40d0b00 0x7f29f40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.422+0000 7f29fc264640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f40d3de0 con 0x7f29f40d2290 2024-04-25T00:08:00.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.422+0000 7f29fc264640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f29f40d14d0 con 0x7f29f40d0b00 2024-04-25T00:08:00.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.422+0000 7f29fc264640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f29f40d3410 con 0x7f29f415e320 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29f97ff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f29f40d2290 0x7f29f40b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60758/0 (socket says 172.21.15.113:60758) 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29f97ff640 1 -- 172.21.15.113:0/2637860624 learned_addr learned my addr 172.21.15.113:0/2637860624 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29fa801640 1 --2- 172.21.15.113:0/2637860624 >> v2:172.21.15.113:3300/0 conn(0x7f29f40d0b00 0x7f29f40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29fa000640 1 --2- 172.21.15.113:0/2637860624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29f8ffe640 1 -- 172.21.15.113:0/2637860624 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147983273 0 0) 0x7f29f40d3de0 con 0x7f29f40d2290 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29f8ffe640 1 -- 172.21.15.113:0/2637860624 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29dc003520 con 0x7f29f40d2290 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29fa801640 1 -- 172.21.15.113:0/2637860624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 msgr2=0x7f29f40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29fa801640 1 --2- 172.21.15.113:0/2637860624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29fa801640 1 -- 172.21.15.113:0/2637860624 >> v1:172.21.15.113:6789/0 conn(0x7f29f40d2290 legacy=0x7f29f40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.423+0000 7f29fa801640 1 -- 172.21.15.113:0/2637860624 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f40d2800 con 0x7f29f40d0b00 2024-04-25T00:08:00.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29fa801640 1 --2- 172.21.15.113:0/2637860624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d0b00 0x7f29f40d0f90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0097a0 tx=0x7f29ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2772371ed970dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29f8ffe640 1 -- 172.21.15.113:0/2637860624 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec037070 con 0x7f29f40d0b00 2024-04-25T00:08:00.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29f8ffe640 1 -- 172.21.15.113:0/2637860624 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ec032440 con 0x7f29f40d0b00 2024-04-25T00:08:00.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29f8ffe640 1 -- 172.21.15.113:0/2637860624 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec032810 con 0x7f29f40d0b00 2024-04-25T00:08:00.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29fc264640 1 -- 172.21.15.113:0/2637860624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d0b00 msgr2=0x7f29f40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29fc264640 1 --2- 172.21.15.113:0/2637860624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d0b00 0x7f29f40d0f90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0097a0 tx=0x7f29ec02a210 comp rx=0 tx=0).stop 2024-04-25T00:08:00.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29fc264640 1 -- 172.21.15.113:0/2637860624 shutdown_connections 2024-04-25T00:08:00.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29fc264640 1 --2- 172.21.15.113:0/2637860624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d0b00 0x7f29f40d0f90 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29fc264640 1 --2- 172.21.15.113:0/2637860624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.424+0000 7f29fc264640 1 -- 172.21.15.113:0/2637860624 >> 172.21.15.113:0/2637860624 conn(0x7f29f40b5c20 msgr2=0x7f29f40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.425+0000 7f29fc264640 1 -- 172.21.15.113:0/2637860624 shutdown_connections 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.425+0000 7f29fc264640 1 -- 172.21.15.113:0/2637860624 wait complete. 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 Processor -- start 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 -- start start 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29f40d0b00 0x7f29f40c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d2290 0x7f29f40ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40b9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f29f40d30a0 con 0x7f29f40d2290 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29f40d2dc0 con 0x7f29f40d0b00 2024-04-25T00:08:00.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29fc264640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f29f415e700 con 0x7f29f415e320 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.426+0000 7f29f97ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d2290 0x7f29f40ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d2290 0x7f29f40ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34624/0 (socket says 172.21.15.113:34624) 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 -- 172.21.15.113:0/2463288691 learned_addr learned my addr 172.21.15.113:0/2463288691 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29fa000640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29f40d0b00 0x7f29f40c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29fa801640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 -- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 msgr2=0x7f29f40b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 -- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29f40d0b00 msgr2=0x7f29f40c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29f40d0b00 0x7f29f40c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 -- 172.21.15.113:0/2463288691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f415ed90 con 0x7f29f40d2290 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29fa000640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29f40d0b00 0x7f29f40c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29fa801640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:00.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f97ff640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d2290 0x7f29f40ca5e0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c460 tx=0x7f29e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f27fc640 1 -- 172.21.15.113:0/2463288691 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8017070 con 0x7f29f40d2290 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f27fc640 1 -- 172.21.15.113:0/2463288691 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e8005ce0 con 0x7f29f40d2290 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f415f4c0 con 0x7f29f40d2290 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29f27fc640 1 -- 172.21.15.113:0/2463288691 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8007290 con 0x7f29f40d2290 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.427+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f40b9a80 con 0x7f29f40d2290 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.428+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f40691a0 con 0x7f29f40d2290 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.429+0000 7f29f27fc640 1 -- 172.21.15.113:0/2463288691 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29e801a070 con 0x7f29f40d2290 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.429+0000 7f29f27fc640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29e006be30 0x7f29e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.429+0000 7f29fa000640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29e006be30 0x7f29e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.429+0000 7f29f27fc640 1 -- 172.21.15.113:0/2463288691 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f29e808a180 con 0x7f29f40d2290 2024-04-25T00:08:00.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.429+0000 7f29fa000640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29e006be30 0x7f29e006e2f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009fd0 tx=0x7f29e400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.433+0000 7f29f27fc640 1 -- 172.21.15.113:0/2463288691 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f29e8058bc0 con 0x7f29f40d2290 2024-04-25T00:08:00.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.607+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29f40cd830 con 0x7f29f40d2290 2024-04-25T00:08:00.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.608+0000 7f29f27fc640 1 -- 172.21.15.113:0/2463288691 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16727 (secure 0 0 0) 0x7f29e80584c0 con 0x7f29f40d2290 2024-04-25T00:08:00.608 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:00.608 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:07:21.999089+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:07:18.614948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:07:19.319166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.609+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29e006be30 msgr2=0x7f29e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.609+0000 7f29fc264640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29e006be30 0x7f29e006e2f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009fd0 tx=0x7f29e400b040 comp rx=0 tx=0).stop 2024-04-25T00:08:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d2290 msgr2=0x7f29f40ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d2290 0x7f29f40ca5e0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f29e800c460 tx=0x7f29e800c920 comp rx=0 tx=0).stop 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 shutdown_connections 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f29e006be30 0x7f29e006e2f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29f415e320 0x7f29f40b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29f40d2290 0x7f29f40ca5e0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 --2- 172.21.15.113:0/2463288691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29f40d0b00 0x7f29f40c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 >> 172.21.15.113:0/2463288691 conn(0x7f29f40b5c20 msgr2=0x7f29f40cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 shutdown_connections 2024-04-25T00:08:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.610+0000 7f29fc264640 1 -- 172.21.15.113:0/2463288691 wait complete. 2024-04-25T00:08:00.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:07:18.614948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:07:19.319166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:08:00.622 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:08:00.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.720+0000 7f6fc72ee640 1 Processor -- start 2024-04-25T00:08:00.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.720+0000 7f6fc72ee640 1 -- start start 2024-04-25T00:08:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.721+0000 7f6fc72ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc015e510 0x7f6fc00d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.721+0000 7f6fc72ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6fc00b63b0 0x7f6fc00b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.721+0000 7f6fc72ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc00d2e00 con 0x7f6fc00d28c0 2024-04-25T00:08:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.721+0000 7f6fc72ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc00d30e0 con 0x7f6fc00b63b0 2024-04-25T00:08:00.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.721+0000 7f6fc72ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc00d33c0 con 0x7f6fc015e510 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc508a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc015e510 0x7f6fc00d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc508a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc015e510 0x7f6fc00d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:51306/0 (socket says 172.21.15.113:51306) 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc588b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6fc00b63b0 0x7f6fc00b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc4889640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6fc00d28c0 0x7f6fc00b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:49870/0 (socket says 172.21.15.113:49870) 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc508a640 1 -- 172.21.15.113:0/3301909376 learned_addr learned my addr 172.21.15.113:0/3301909376 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc588b640 1 -- 172.21.15.113:0/3301909376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc015e510 msgr2=0x7f6fc00d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc588b640 1 --2- 172.21.15.113:0/3301909376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc015e510 0x7f6fc00d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc588b640 1 -- 172.21.15.113:0/3301909376 >> v1:172.21.15.178:6789/0 conn(0x7f6fc00d28c0 legacy=0x7f6fc00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.722+0000 7f6fc588b640 1 -- 172.21.15.113:0/3301909376 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc00d36a0 con 0x7f6fc00b63b0 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fc588b640 1 --2- 172.21.15.113:0/3301909376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00b63b0 0x7f6fc00b73c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80097a0 tx=0x7f6fb802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b2342784335f25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fbffff640 1 -- 172.21.15.113:0/3301909376 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8037070 con 0x7f6fc00b63b0 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fbffff640 1 -- 172.21.15.113:0/3301909376 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb802ae50 con 0x7f6fc00b63b0 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fbffff640 1 -- 172.21.15.113:0/3301909376 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8032660 con 0x7f6fc00b63b0 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3301909376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00b63b0 msgr2=0x7f6fc00b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3301909376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00b63b0 0x7f6fc00b73c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80097a0 tx=0x7f6fb802a100 comp rx=0 tx=0).stop 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3301909376 shutdown_connections 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3301909376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00b63b0 0x7f6fc00b73c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3301909376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc015e510 0x7f6fc00d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.723+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3301909376 >> 172.21.15.113:0/3301909376 conn(0x7f6fc006b720 msgr2=0x7f6fc00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.724+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3301909376 shutdown_connections 2024-04-25T00:08:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.724+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3301909376 wait complete. 2024-04-25T00:08:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 Processor -- start 2024-04-25T00:08:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 -- start start 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc00b63b0 0x7f6fc014b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00d28c0 0x7f6fc00c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc015e510 0x7f6fc00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc00b9dd0 con 0x7f6fc015e510 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc00d3ea0 con 0x7f6fc00d28c0 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.725+0000 7f6fc72ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc00b7900 con 0x7f6fc00b63b0 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc015e510 0x7f6fc00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc015e510 0x7f6fc00c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34638/0 (socket says 172.21.15.113:34638) 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 -- 172.21.15.113:0/3483312672 learned_addr learned my addr 172.21.15.113:0/3483312672 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc508a640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc00b63b0 0x7f6fc014b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc4889640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00d28c0 0x7f6fc00c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 -- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc00b63b0 msgr2=0x7f6fc014b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc00b63b0 0x7f6fc014b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 -- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00d28c0 msgr2=0x7f6fc00c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00d28c0 0x7f6fc00c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 -- 172.21.15.113:0/3483312672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc014e900 con 0x7f6fc015e510 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc588b640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc015e510 0x7f6fc00c8290 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8035040 tx=0x7f6fb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fbdffb640 1 -- 172.21.15.113:0/3483312672 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8041070 con 0x7f6fc015e510 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fbdffb640 1 -- 172.21.15.113:0/3483312672 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb803c460 con 0x7f6fc015e510 2024-04-25T00:08:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc00c88c0 con 0x7f6fc015e510 2024-04-25T00:08:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fbdffb640 1 -- 172.21.15.113:0/3483312672 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8037040 con 0x7f6fc015e510 2024-04-25T00:08:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.726+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc00c8e80 con 0x7f6fc015e510 2024-04-25T00:08:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.727+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc00691a0 con 0x7f6fc015e510 2024-04-25T00:08:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.728+0000 7f6fbdffb640 1 -- 172.21.15.113:0/3483312672 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fb8044070 con 0x7f6fc015e510 2024-04-25T00:08:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.728+0000 7f6fbdffb640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6fac06be30 0x7f6fac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.728+0000 7f6fbdffb640 1 -- 172.21.15.113:0/3483312672 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f6fb80ab410 con 0x7f6fc015e510 2024-04-25T00:08:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.728+0000 7f6fc508a640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6fac06be30 0x7f6fac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.729+0000 7f6fc508a640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6fac06be30 0x7f6fac06e2f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0011340 tx=0x7f6fb000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:00.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.732+0000 7f6fbdffb640 1 -- 172.21.15.113:0/3483312672 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fb80796d0 con 0x7f6fc015e510 2024-04-25T00:08:00.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.903+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6fc006c100 con 0x7f6fc015e510 2024-04-25T00:08:00.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.904+0000 7f6fbdffb640 1 -- 172.21.15.113:0/3483312672 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7f6fb807d9d0 con 0x7f6fc015e510 2024-04-25T00:08:00.904 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6fac06be30 msgr2=0x7f6fac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6fac06be30 0x7f6fac06e2f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0011340 tx=0x7f6fb000b040 comp rx=0 tx=0).stop 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc015e510 msgr2=0x7f6fc00c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc015e510 0x7f6fc00c8290 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8035040 tx=0x7f6fb80046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 shutdown_connections 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6fac06be30 0x7f6fac06e2f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc00d28c0 0x7f6fc00c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc015e510 0x7f6fc00c8290 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 --2- 172.21.15.113:0/3483312672 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6fc00b63b0 0x7f6fc014b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 >> 172.21.15.113:0/3483312672 conn(0x7f6fc006b720 msgr2=0x7f6fc00b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 shutdown_connections 2024-04-25T00:08:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:00.906+0000 7f6fc72ee640 1 -- 172.21.15.113:0/3483312672 wait complete. 2024-04-25T00:08:00.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:08:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.017+0000 7febbb3ce640 1 Processor -- start 2024-04-25T00:08:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.017+0000 7febbb3ce640 1 -- start start 2024-04-25T00:08:01.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febbb3ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 0x7febb40b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febbb3ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb40d2e00 con 0x7febb40b5ca0 2024-04-25T00:08:01.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febbb3ce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb40d30e0 con 0x7febb415e510 2024-04-25T00:08:01.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febbb3ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febb40d33c0 con 0x7febb40b6100 2024-04-25T00:08:01.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febb8969640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7febb40b5ca0 0x7febb40b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60766/0 (socket says 172.21.15.113:60766) 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febb8969640 1 -- 172.21.15.113:0/2127098798 learned_addr learned my addr 172.21.15.113:0/2127098798 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febb996b640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 0x7febb40b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709356843 0 0) 0x7febb40d2e00 con 0x7febb40b5ca0 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feba0003520 con 0x7febb40b5ca0 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1942552906 0 0) 0x7feba0003520 con 0x7febb40b5ca0 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 msgr2=0x7febb40b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febabfff640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 0x7febb40b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 >> v1:172.21.15.178:6789/0 conn(0x7febb415e510 legacy=0x7febb40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.018+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb40d36a0 con 0x7febb40b5ca0 2024-04-25T00:08:01.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7feba4001670 con 0x7febb40b5ca0 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 >> v1:172.21.15.113:6789/0 conn(0x7febb40b5ca0 legacy=0x7febb40b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba00039e0 0x7feba0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 0x7feba0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0004d40 0x7feba00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febb40d33c0 con 0x7feba00039e0 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba000ba70 con 0x7feba0004d40 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feba000bc70 con 0x7febb40b6100 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb916a640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba00039e0 0x7feba0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb996b640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0004d40 0x7feba00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb8969640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 0x7feba0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb916a640 1 -- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 msgr2=0x7feba0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb916a640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 0x7feba0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb916a640 1 -- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0004d40 msgr2=0x7feba00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb916a640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0004d40 0x7feba00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb916a640 1 -- 172.21.15.113:0/2127098798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba0003520 con 0x7feba00039e0 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb8969640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb40b6100 0x7feba0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.019+0000 7febb996b640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0004d40 0x7feba00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febb916a640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba00039e0 0x7feba0003dc0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7febb0001860 tx=0x7feba0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e109df9c1a4b3010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0037070 con 0x7feba00039e0 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb002ac30 con 0x7feba00039e0 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febabfff640 1 -- 172.21.15.113:0/2127098798 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0001430 con 0x7feba00039e0 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febbb3ce640 1 -- 172.21.15.113:0/2127098798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba00039e0 msgr2=0x7feba0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febbb3ce640 1 --2- 172.21.15.113:0/2127098798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba00039e0 0x7feba0003dc0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7febb0001860 tx=0x7feba0005830 comp rx=0 tx=0).stop 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febb996b640 1 -- 172.21.15.113:0/2127098798 reap_dead start 2024-04-25T00:08:01.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febbb3ce640 1 -- 172.21.15.113:0/2127098798 shutdown_connections 2024-04-25T00:08:01.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.020+0000 7febbb3ce640 1 -- 172.21.15.113:0/2127098798 >> 172.21.15.113:0/2127098798 conn(0x7febb406b720 msgr2=0x7febb40d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:01.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.021+0000 7febbb3ce640 1 -- 172.21.15.113:0/2127098798 shutdown_connections 2024-04-25T00:08:01.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.021+0000 7febbb3ce640 1 -- 172.21.15.113:0/2127098798 wait complete. 2024-04-25T00:08:01.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 Processor -- start 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 -- start start 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039e0 0x7febb415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb4154500 0x7febb41f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb415b1e0 0x7febb4157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febb40d3ea0 con 0x7febb415b1e0 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7febb40b6550 con 0x7feba00039e0 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.022+0000 7febbb3ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febb40d2a90 con 0x7febb4154500 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb415b1e0 0x7febb4157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb415b1e0 0x7febb4157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34654/0 (socket says 172.21.15.113:34654) 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 -- 172.21.15.113:0/674783695 learned_addr learned my addr 172.21.15.113:0/674783695 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb916a640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039e0 0x7febb415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb8969640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb4154500 0x7febb41f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 -- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb4154500 msgr2=0x7febb41f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb4154500 0x7febb41f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 -- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039e0 msgr2=0x7febb415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039e0 0x7febb415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 -- 172.21.15.113:0/674783695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb4154980 con 0x7febb415b1e0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febb996b640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb415b1e0 0x7febb4157640 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7febb40b6c20 tx=0x7febac010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7feba9ffb640 1 -- 172.21.15.113:0/674783695 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac01a030 con 0x7febb415b1e0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7feba9ffb640 1 -- 172.21.15.113:0/674783695 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febac010d80 con 0x7febb415b1e0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb41551a0 con 0x7febb415b1e0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7feba9ffb640 1 -- 172.21.15.113:0/674783695 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac003bd0 con 0x7febb415b1e0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.023+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb4155760 con 0x7febb415b1e0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.025+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb40691a0 con 0x7febb415b1e0 2024-04-25T00:08:01.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.025+0000 7feba9ffb640 1 -- 172.21.15.113:0/674783695 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febac01d070 con 0x7febb415b1e0 2024-04-25T00:08:01.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.025+0000 7feba9ffb640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.025+0000 7feba9ffb640 1 -- 172.21.15.113:0/674783695 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7febac08b220 con 0x7febb415b1e0 2024-04-25T00:08:01.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.025+0000 7febb916a640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.026+0000 7febb916a640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7febb0034000 tx=0x7febb0009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.030+0000 7feba9ffb640 1 -- 172.21.15.113:0/674783695 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7febac059610 con 0x7febb415b1e0 2024-04-25T00:08:01.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.205+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7febb415d910 con 0x7febb415b1e0 2024-04-25T00:08:01.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.206+0000 7feba9ffb640 1 -- 172.21.15.113:0/674783695 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7febac05d910 con 0x7febb415b1e0 2024-04-25T00:08:01.206 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:08:01.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 msgr2=0x7feb9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7febb0034000 tx=0x7febb0009030 comp rx=0 tx=0).stop 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb415b1e0 msgr2=0x7febb4157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb415b1e0 0x7febb4157640 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7febb40b6c20 tx=0x7febac010590 comp rx=0 tx=0).stop 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 shutdown_connections 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb415b1e0 0x7febb4157640 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb4154500 0x7febb41f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 --2- 172.21.15.113:0/674783695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039e0 0x7febb415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 >> 172.21.15.113:0/674783695 conn(0x7febb406b720 msgr2=0x7febb4158720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 shutdown_connections 2024-04-25T00:08:01.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.208+0000 7febbb3ce640 1 -- 172.21.15.113:0/674783695 wait complete. 2024-04-25T00:08:01.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:08:01.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.319+0000 7f9444f53640 1 Processor -- start 2024-04-25T00:08:01.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.319+0000 7f9444f53640 1 -- start start 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f9444f53640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f944013f050 0x7f94400cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f9444f53640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 0x7f94400cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f9444f53640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944015d6b0 con 0x7f94400d04f0 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f9444f53640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f94400d09a0 con 0x7f944013f050 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f9444f53640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f944015c6a0 con 0x7f94401606a0 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f9437fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f94400d04f0 0x7f944015ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60774/0 (socket says 172.21.15.113:60774) 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f9437fff640 1 -- 172.21.15.113:0/3450072853 learned_addr learned my addr 172.21.15.113:0/3450072853 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3688721665 0 0) 0x7f944015d6b0 con 0x7f94400d04f0 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f942c003520 con 0x7f94400d04f0 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943ed9d640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 0x7f94400cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943e59c640 1 --2- 172.21.15.113:0/3450072853 >> v2:172.21.15.178:3300/0 conn(0x7f944013f050 0x7f94400cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4103970273 0 0) 0x7f942c003520 con 0x7f94400d04f0 2024-04-25T00:08:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 msgr2=0x7f94400cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 0x7f94400cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 >> v2:172.21.15.178:3300/0 conn(0x7f944013f050 msgr2=0x7f94400cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 --2- 172.21.15.113:0/3450072853 >> v2:172.21.15.178:3300/0 conn(0x7f944013f050 0x7f94400cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944015c8f0 con 0x7f94400d04f0 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f9428001670 con 0x7f94400d04f0 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943e59c640 1 --2- 172.21.15.113:0/3450072853 >> v2:172.21.15.178:3300/0 conn(0x7f944013f050 0x7f94400cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943ed9d640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 0x7f94400cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 >> v1:172.21.15.113:6789/0 conn(0x7f94400d04f0 legacy=0x7f944015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 0x7f942c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c003fa0 0x7f942c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.320+0000 7f943dd9b640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c004d40 0x7f942c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94400d09a0 con 0x7f942c003fa0 2024-04-25T00:08:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f944015c6a0 con 0x7f942c004d40 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f942c00bb20 con 0x7f94401606a0 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f9437fff640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c003fa0 0x7f942c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f943e59c640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 0x7f942c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f943ed9d640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c004d40 0x7f942c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f9437fff640 1 -- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 msgr2=0x7f942c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f9437fff640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94401606a0 0x7f942c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f9437fff640 1 -- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c004d40 msgr2=0x7f942c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f9437fff640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c004d40 0x7f942c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f9437fff640 1 -- 172.21.15.113:0/3450072853 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c003520 con 0x7f942c003fa0 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f943ed9d640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c004d40 0x7f942c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.321+0000 7f9437fff640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c003fa0 0x7f942c004800 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f942800c290 tx=0x7f942c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9e57024a4d28f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428007d50 con 0x7f942c003fa0 2024-04-25T00:08:01.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9428005af0 con 0x7f942c003fa0 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f943dd9b640 1 -- 172.21.15.113:0/3450072853 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428007070 con 0x7f942c003fa0 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f9444f53640 1 -- 172.21.15.113:0/3450072853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c003fa0 msgr2=0x7f942c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f9444f53640 1 --2- 172.21.15.113:0/3450072853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c003fa0 0x7f942c004800 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f942800c290 tx=0x7f942c005830 comp rx=0 tx=0).stop 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f943ed9d640 1 -- 172.21.15.113:0/3450072853 reap_dead start 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f9444f53640 1 -- 172.21.15.113:0/3450072853 shutdown_connections 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f9444f53640 1 -- 172.21.15.113:0/3450072853 >> 172.21.15.113:0/3450072853 conn(0x7f944006b0f0 msgr2=0x7f944006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f9444f53640 1 -- 172.21.15.113:0/3450072853 shutdown_connections 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.322+0000 7f9444f53640 1 -- 172.21.15.113:0/3450072853 wait complete. 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 Processor -- start 2024-04-25T00:08:01.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 -- start start 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942c003fa0 0x7f94401537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94401f4ac0 0x7f94400afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94400b0190 0x7f94400b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f944015d190 con 0x7f94401f4ac0 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f944015ceb0 con 0x7f94400b0190 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9444f53640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f944013f430 con 0x7f942c003fa0 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9437fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94401f4ac0 0x7f94400afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9437fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94401f4ac0 0x7f94400afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34670/0 (socket says 172.21.15.113:34670) 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.324+0000 7f9437fff640 1 -- 172.21.15.113:0/1283588243 learned_addr learned my addr 172.21.15.113:0/1283588243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f943ed9d640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94400b0190 0x7f94400b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f943e59c640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942c003fa0 0x7f94401537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9437fff640 1 -- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942c003fa0 msgr2=0x7f94401537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9437fff640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942c003fa0 0x7f94401537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9437fff640 1 -- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94400b0190 msgr2=0x7f94400b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9437fff640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94400b0190 0x7f94400b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9437fff640 1 -- 172.21.15.113:0/1283588243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94401389d0 con 0x7f94401f4ac0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9437fff640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94401f4ac0 0x7f94400afc50 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f9428001970 tx=0x7f944015cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9436ffd640 1 -- 172.21.15.113:0/1283588243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428042070 con 0x7f94401f4ac0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9436ffd640 1 -- 172.21.15.113:0/1283588243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9428007480 con 0x7f94401f4ac0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9440156470 con 0x7f94401f4ac0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9436ffd640 1 -- 172.21.15.113:0/1283588243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428001c80 con 0x7f94401f4ac0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.325+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9440156a30 con 0x7f94401f4ac0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.326+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94400691a0 con 0x7f94401f4ac0 2024-04-25T00:08:01.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.327+0000 7f9436ffd640 1 -- 172.21.15.113:0/1283588243 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9428045070 con 0x7f94401f4ac0 2024-04-25T00:08:01.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.327+0000 7f9436ffd640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f942006be60 0x7f942006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.327+0000 7f943e59c640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f942006be60 0x7f942006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.327+0000 7f9436ffd640 1 -- 172.21.15.113:0/1283588243 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f94280abb30 con 0x7f94401f4ac0 2024-04-25T00:08:01.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.327+0000 7f943e59c640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f942006be60 0x7f942006e320 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9430006fd0 tx=0x7f9430008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.331+0000 7f9436ffd640 1 -- 172.21.15.113:0/1283588243 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9428005ac0 con 0x7f94401f4ac0 2024-04-25T00:08:01.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.503+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94400d0e50 con 0x7f94401f4ac0 2024-04-25T00:08:01.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.503+0000 7f9436ffd640 1 -- 172.21.15.113:0/1283588243 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7f9428079e00 con 0x7f94401f4ac0 2024-04-25T00:08:01.504 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f942006be60 msgr2=0x7f942006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f942006be60 0x7f942006e320 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9430006fd0 tx=0x7f9430008040 comp rx=0 tx=0).stop 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94401f4ac0 msgr2=0x7f94400afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94401f4ac0 0x7f94400afc50 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f9428001970 tx=0x7f944015cac0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 shutdown_connections 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f942006be60 0x7f942006e320 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94400b0190 0x7f94400b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94401f4ac0 0x7f94400afc50 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 --2- 172.21.15.113:0/1283588243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942c003fa0 0x7f94401537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 >> 172.21.15.113:0/1283588243 conn(0x7f944006b0f0 msgr2=0x7f94401524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:01.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 shutdown_connections 2024-04-25T00:08:01.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.505+0000 7f9444f53640 1 -- 172.21.15.113:0/1283588243 wait complete. 2024-04-25T00:08:01.517 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.612+0000 7f91f9d5a640 1 Processor -- start 2024-04-25T00:08:01.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.612+0000 7f91f9d5a640 1 -- start start 2024-04-25T00:08:01.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.613+0000 7f91f9d5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 0x7f91f40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.613+0000 7f91f9d5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f91f40b5240 0x7f91f406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.613+0000 7f91f9d5a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f4144cf0 con 0x7f91f40b5680 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.613+0000 7f91f9d5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f91f40b4510 con 0x7f91f40b5240 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.613+0000 7f91f9d5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91f40b2f30 con 0x7f91f4144910 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.613+0000 7f91f3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f91f40b5680 0x7f91f40cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60790/0 (socket says 172.21.15.113:60790) 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.613+0000 7f91f3fff640 1 -- 172.21.15.113:0/2268489434 learned_addr learned my addr 172.21.15.113:0/2268489434 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2246484148 0 0) 0x7f91f4144cf0 con 0x7f91f40b5680 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f37fe640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 0x7f91f40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f2ffd640 1 --2- 172.21.15.113:0/2268489434 >> v2:172.21.15.178:3300/0 conn(0x7f91f40b5240 0x7f91f406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91dc005730 con 0x7f91f40b5680 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2280942699 0 0) 0x7f91dc005730 con 0x7f91f40b5680 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 msgr2=0x7f91f40b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 0x7f91f40b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 >> v2:172.21.15.178:3300/0 conn(0x7f91f40b5240 msgr2=0x7f91f406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 --2- 172.21.15.113:0/2268489434 >> v2:172.21.15.178:3300/0 conn(0x7f91f40b5240 0x7f91f406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f40b3210 con 0x7f91f40b5680 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f91ec0019f0 con 0x7f91f40b5680 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 >> v1:172.21.15.113:6789/0 conn(0x7f91f40b5680 legacy=0x7f91f40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91dc005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 0x7f91dc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91dc006f50 0x7f91dc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91f40b4510 con 0x7f91dc005bc0 2024-04-25T00:08:01.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91f40b2f30 con 0x7f91dc006f50 2024-04-25T00:08:01.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc00dbd0 con 0x7f91f4144910 2024-04-25T00:08:01.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f37fe640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91dc005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f2ffd640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 0x7f91dc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f3fff640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91dc006f50 0x7f91dc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f37fe640 1 -- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 msgr2=0x7f91dc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f37fe640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f4144910 0x7f91dc006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f37fe640 1 -- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91dc006f50 msgr2=0x7f91dc0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f37fe640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91dc006f50 0x7f91dc0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.614+0000 7f91f37fe640 1 -- 172.21.15.113:0/2268489434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc005730 con 0x7f91dc005bc0 2024-04-25T00:08:01.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.615+0000 7f91f37fe640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91dc005fe0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009ac0 tx=0x7f91dc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1f771f270dea0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.615+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4036070 con 0x7f91dc005bc0 2024-04-25T00:08:01.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.615+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e4031400 con 0x7f91dc005bc0 2024-04-25T00:08:01.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.615+0000 7f91f27fc640 1 -- 172.21.15.113:0/2268489434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4009690 con 0x7f91dc005bc0 2024-04-25T00:08:01.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.616+0000 7f91f9d5a640 1 -- 172.21.15.113:0/2268489434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 msgr2=0x7f91dc005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.616+0000 7f91f9d5a640 1 --2- 172.21.15.113:0/2268489434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91dc005fe0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009ac0 tx=0x7f91dc007a40 comp rx=0 tx=0).stop 2024-04-25T00:08:01.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.616+0000 7f91f3fff640 1 -- 172.21.15.113:0/2268489434 reap_dead start 2024-04-25T00:08:01.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.616+0000 7f91f9d5a640 1 -- 172.21.15.113:0/2268489434 shutdown_connections 2024-04-25T00:08:01.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.616+0000 7f91f9d5a640 1 -- 172.21.15.113:0/2268489434 >> 172.21.15.113:0/2268489434 conn(0x7f91f40b1a50 msgr2=0x7f91f40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:01.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.616+0000 7f91f9d5a640 1 -- 172.21.15.113:0/2268489434 shutdown_connections 2024-04-25T00:08:01.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.616+0000 7f91f9d5a640 1 -- 172.21.15.113:0/2268489434 wait complete. 2024-04-25T00:08:01.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.617+0000 7f91f9d5a640 1 Processor -- start 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.617+0000 7f91f9d5a640 1 -- start start 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f9d5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91f415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f9d5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f40cec30 0x7f91f40cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f9d5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91f413e5f0 0x7f91f413eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f9d5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91f40b4da0 con 0x7f91dc005bc0 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f9d5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91f40b37d0 con 0x7f91f413e5f0 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f9d5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91f40b4f20 con 0x7f91f40cec30 2024-04-25T00:08:01.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91f415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91f415da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34682/0 (socket says 172.21.15.113:34682) 2024-04-25T00:08:01.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 -- 172.21.15.113:0/204277352 learned_addr learned my addr 172.21.15.113:0/204277352 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:01.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f3fff640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91f413e5f0 0x7f91f413eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f2ffd640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f40cec30 0x7f91f40cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 -- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f40cec30 msgr2=0x7f91f40cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f40cec30 0x7f91f40cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 -- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91f413e5f0 msgr2=0x7f91f413eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91f413e5f0 0x7f91f413eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 -- 172.21.15.113:0/204277352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f413f390 con 0x7f91dc005bc0 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f3fff640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91f413e5f0 0x7f91f413eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.618+0000 7f91f37fe640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91f415da40 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009ac0 tx=0x7f91e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.619+0000 7f91dbfff640 1 -- 172.21.15.113:0/204277352 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4040020 con 0x7f91dc005bc0 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.619+0000 7f91dbfff640 1 -- 172.21.15.113:0/204277352 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e4003bc0 con 0x7f91dc005bc0 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.619+0000 7f91dbfff640 1 -- 172.21.15.113:0/204277352 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4036040 con 0x7f91dc005bc0 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.619+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f413f670 con 0x7f91dc005bc0 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.619+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f413f950 con 0x7f91dc005bc0 2024-04-25T00:08:01.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.620+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f40691a0 con 0x7f91dc005bc0 2024-04-25T00:08:01.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.620+0000 7f91dbfff640 1 -- 172.21.15.113:0/204277352 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91e4043020 con 0x7f91dc005bc0 2024-04-25T00:08:01.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.620+0000 7f91dbfff640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.620+0000 7f91dbfff640 1 -- 172.21.15.113:0/204277352 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7f91e40aa9b0 con 0x7f91dc005bc0 2024-04-25T00:08:01.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.620+0000 7f91f2ffd640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.621+0000 7f91f2ffd640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f91f40cc3c0 tx=0x7f91e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.625+0000 7f91dbfff640 1 -- 172.21.15.113:0/204277352 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91e40044e0 con 0x7f91dc005bc0 2024-04-25T00:08:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.832+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91f415f330 con 0x7f91dc005bc0 2024-04-25T00:08:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.833+0000 7f91dbfff640 1 -- 172.21.15.113:0/204277352 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5197 (secure 0 0 0) 0x7f91e4079360 con 0x7f91dc005bc0 2024-04-25T00:08:01.834 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:01.834 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1199283489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1199283489},{"type":"v1","addr":"172.21.15.178:6834","nonce":1199283489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4259795043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4259795043},{"type":"v1","addr":"172.21.15.178:6835","nonce":4259795043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4146002756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4146002756},{"type":"v1","addr":"172.21.15.113:6835","nonce":4146002756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:07:20.361298+0000","modified":"2024-04-25T00:07:22.348971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2111641286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2111641286},{"type":"v1","addr":"172.21.15.113:6837","nonce":2111641286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":10}]} 2024-04-25T00:08:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-04-25T00:08:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 msgr2=0x7f91e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f91f40cc3c0 tx=0x7f91e8008040 comp rx=0 tx=0).stop 2024-04-25T00:08:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 msgr2=0x7f91f415da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91f415da40 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009ac0 tx=0x7f91e4004730 comp rx=0 tx=0).stop 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 shutdown_connections 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91e006be60 0x7f91e006e320 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91f413e5f0 0x7f91f413eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91f40cec30 0x7f91f40cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 --2- 172.21.15.113:0/204277352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91dc005bc0 0x7f91f415da40 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 >> 172.21.15.113:0/204277352 conn(0x7f91f40b1a50 msgr2=0x7f91f406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 shutdown_connections 2024-04-25T00:08:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.835+0000 7f91f9d5a640 1 -- 172.21.15.113:0/204277352 wait complete. 2024-04-25T00:08:01.847 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:01.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.943+0000 7ff8df9fb640 1 Processor -- start 2024-04-25T00:08:01.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.943+0000 7ff8df9fb640 1 -- start start 2024-04-25T00:08:01.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8df9fb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8d80b44c0 0x7ff8d8162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8df9fb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8d814c360 con 0x7ff8d814bf80 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8df9fb640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8d8071600 con 0x7ff8d806dfd0 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8df9fb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff8d80718e0 con 0x7ff8d80b44c0 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8dd797640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff8d814bf80 0x7ff8d8070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60804/0 (socket says 172.21.15.113:60804) 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8dd797640 1 -- 172.21.15.113:0/3469138721 learned_addr learned my addr 172.21.15.113:0/3469138721 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8dcf96640 1 --2- 172.21.15.113:0/3469138721 >> v2:172.21.15.113:3300/0 conn(0x7ff8d80b44c0 0x7ff8d8162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8cffff640 1 -- 172.21.15.113:0/3469138721 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3255311136 0 0) 0x7ff8d814c360 con 0x7ff8d814bf80 2024-04-25T00:08:01.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.944+0000 7ff8cffff640 1 -- 172.21.15.113:0/3469138721 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8c4003520 con 0x7ff8d814bf80 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.945+0000 7ff8dcf96640 1 -- 172.21.15.113:0/3469138721 >> v1:172.21.15.178:6789/0 conn(0x7ff8d806dfd0 legacy=0x7ff8d80726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.945+0000 7ff8dcf96640 1 -- 172.21.15.113:0/3469138721 >> v1:172.21.15.113:6789/0 conn(0x7ff8d814bf80 legacy=0x7ff8d8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.945+0000 7ff8dcf96640 1 -- 172.21.15.113:0/3469138721 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d8071bc0 con 0x7ff8d80b44c0 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.945+0000 7ff8dcf96640 1 --2- 172.21.15.113:0/3469138721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b44c0 0x7ff8d8162cf0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80097a0 tx=0x7ff8c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce1eba474e4b0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.945+0000 7ff8cffff640 1 -- 172.21.15.113:0/3469138721 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8037070 con 0x7ff8d80b44c0 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.945+0000 7ff8cffff640 1 -- 172.21.15.113:0/3469138721 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c8032440 con 0x7ff8d80b44c0 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.945+0000 7ff8cffff640 1 -- 172.21.15.113:0/3469138721 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8032810 con 0x7ff8d80b44c0 2024-04-25T00:08:01.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.946+0000 7ff8df9fb640 1 -- 172.21.15.113:0/3469138721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b44c0 msgr2=0x7ff8d8162cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.946+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/3469138721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b44c0 0x7ff8d8162cf0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80097a0 tx=0x7ff8c802a270 comp rx=0 tx=0).stop 2024-04-25T00:08:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.946+0000 7ff8df9fb640 1 -- 172.21.15.113:0/3469138721 shutdown_connections 2024-04-25T00:08:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.946+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/3469138721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d80b44c0 0x7ff8d8162cf0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.946+0000 7ff8df9fb640 1 -- 172.21.15.113:0/3469138721 >> 172.21.15.113:0/3469138721 conn(0x7ff8d80b1a50 msgr2=0x7ff8d80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.946+0000 7ff8df9fb640 1 -- 172.21.15.113:0/3469138721 shutdown_connections 2024-04-25T00:08:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.946+0000 7ff8df9fb640 1 -- 172.21.15.113:0/3469138721 wait complete. 2024-04-25T00:08:01.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.947+0000 7ff8df9fb640 1 Processor -- start 2024-04-25T00:08:01.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.947+0000 7ff8df9fb640 1 -- start start 2024-04-25T00:08:01.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8df9fb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8d806dfd0 0x7ff8d815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8df9fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8d80b44c0 0x7ff8d81649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8df9fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d814bf80 0x7ff8d815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8df9fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8163230 con 0x7ff8d814bf80 2024-04-25T00:08:01.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8df9fb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8072180 con 0x7ff8d80b44c0 2024-04-25T00:08:01.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8df9fb640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d81633b0 con 0x7ff8d806dfd0 2024-04-25T00:08:01.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d814bf80 0x7ff8d815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d814bf80 0x7ff8d815fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34702/0 (socket says 172.21.15.113:34702) 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 -- 172.21.15.113:0/1618460761 learned_addr learned my addr 172.21.15.113:0/1618460761 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8dd797640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8d806dfd0 0x7ff8d815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8dcf96640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8d80b44c0 0x7ff8d81649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 -- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8d806dfd0 msgr2=0x7ff8d815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8d806dfd0 0x7ff8d815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 -- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8d80b44c0 msgr2=0x7ff8d81649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:01.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8d80b44c0 0x7ff8d81649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 -- 172.21.15.113:0/1618460761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d815e910 con 0x7ff8d814bf80 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8ddf98640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d814bf80 0x7ff8d815fef0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c410 tx=0x7ff8d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8cdffb640 1 -- 172.21.15.113:0/1618460761 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0017070 con 0x7ff8d814bf80 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8cdffb640 1 -- 172.21.15.113:0/1618460761 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d0005230 con 0x7ff8d814bf80 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8cdffb640 1 -- 172.21.15.113:0/1618460761 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0011590 con 0x7ff8d814bf80 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.948+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d815ebf0 con 0x7ff8d814bf80 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.949+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d815f490 con 0x7ff8d814bf80 2024-04-25T00:08:01.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.950+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d815edc0 con 0x7ff8d814bf80 2024-04-25T00:08:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.950+0000 7ff8cdffb640 1 -- 172.21.15.113:0/1618460761 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8d001a070 con 0x7ff8d814bf80 2024-04-25T00:08:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.950+0000 7ff8cdffb640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8bc06be30 0x7ff8bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.950+0000 7ff8dd797640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8bc06be30 0x7ff8bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.950+0000 7ff8cdffb640 1 -- 172.21.15.113:0/1618460761 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5709+0+0 (secure 0 0 0) 0x7ff8d008a150 con 0x7ff8d814bf80 2024-04-25T00:08:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.951+0000 7ff8dd797640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8bc06be30 0x7ff8bc06e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4005c00 tx=0x7ff8d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:01.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:01.955+0000 7ff8cdffb640 1 -- 172.21.15.113:0/1618460761 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8d0058b00 con 0x7ff8d814bf80 2024-04-25T00:08:02.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.130+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8d80691a0 con 0x7ff8d814bf80 2024-04-25T00:08:02.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.131+0000 7ff8cdffb640 1 -- 172.21.15.113:0/1618460761 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16777 (secure 0 0 0) 0x7ff8d0058400 con 0x7ff8d814bf80 2024-04-25T00:08:02.132 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:02.132 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":145,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:02.094572+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:07:18.614948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:07:19.319166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:02.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8bc06be30 msgr2=0x7ff8bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8bc06be30 0x7ff8bc06e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4005c00 tx=0x7ff8d4009040 comp rx=0 tx=0).stop 2024-04-25T00:08:02.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d814bf80 msgr2=0x7ff8d815fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d814bf80 0x7ff8d815fef0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c410 tx=0x7ff8d000c8d0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 shutdown_connections 2024-04-25T00:08:02.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8bc06be30 0x7ff8bc06e2f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8d80b44c0 0x7ff8d81649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8d814bf80 0x7ff8d815fef0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 --2- 172.21.15.113:0/1618460761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8d806dfd0 0x7ff8d815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 >> 172.21.15.113:0/1618460761 conn(0x7ff8d80b1a50 msgr2=0x7ff8d8070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 shutdown_connections 2024-04-25T00:08:02.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.133+0000 7ff8df9fb640 1 -- 172.21.15.113:0/1618460761 wait complete. 2024-04-25T00:08:02.145 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:08:02.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:02.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.242+0000 7f3585fab640 1 Processor -- start 2024-04-25T00:08:02.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.242+0000 7f3585fab640 1 -- start start 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.242+0000 7f3585fab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3580072ea0 0x7f358015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.242+0000 7f3585fab640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f358006dfd0 0x7f3580070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.242+0000 7f3585fab640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f358015abf0 con 0x7f358006e410 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.242+0000 7f3585fab640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f35800b2d50 con 0x7f358006dfd0 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.242+0000 7f3585fab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3580072a70 con 0x7f3580072ea0 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3580072ea0 0x7f358015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3580072ea0 0x7f358015a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:51402/0 (socket says 172.21.15.113:51402) 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357f7fe640 1 -- 172.21.15.113:0/36303260 learned_addr learned my addr 172.21.15.113:0/36303260 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357f7fe640 1 -- 172.21.15.113:0/36303260 >> v2:172.21.15.178:3300/0 conn(0x7f358006dfd0 msgr2=0x7f3580070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:02.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357f7fe640 1 --2- 172.21.15.113:0/36303260 >> v2:172.21.15.178:3300/0 conn(0x7f358006dfd0 0x7f3580070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357e7fc640 1 -- 172.21.15.113:0/36303260 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3495398746 0 0) 0x7f358015abf0 con 0x7f358006e410 2024-04-25T00:08:02.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357f7fe640 1 -- 172.21.15.113:0/36303260 >> v1:172.21.15.178:6789/0 conn(0x7f358006e410 legacy=0x7f3580071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.243+0000 7f357f7fe640 1 -- 172.21.15.113:0/36303260 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35800b3010 con 0x7f3580072ea0 2024-04-25T00:08:02.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.244+0000 7f357f7fe640 1 --2- 172.21.15.113:0/36303260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3580072ea0 0x7f358015a6b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3570009870 tx=0x7f357002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88a8882dbe687a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.244+0000 7f357e7fc640 1 -- 172.21.15.113:0/36303260 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570037070 con 0x7f3580072ea0 2024-04-25T00:08:02.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.244+0000 7f357e7fc640 1 -- 172.21.15.113:0/36303260 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3570032440 con 0x7f3580072ea0 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.244+0000 7f357e7fc640 1 -- 172.21.15.113:0/36303260 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570032810 con 0x7f3580072ea0 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 -- 172.21.15.113:0/36303260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3580072ea0 msgr2=0x7f358015a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 --2- 172.21.15.113:0/36303260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3580072ea0 0x7f358015a6b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3570009870 tx=0x7f357002a270 comp rx=0 tx=0).stop 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 -- 172.21.15.113:0/36303260 shutdown_connections 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 --2- 172.21.15.113:0/36303260 >> v2:172.21.15.178:3300/0 conn(0x7f358006dfd0 0x7f3580070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 --2- 172.21.15.113:0/36303260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3580072ea0 0x7f358015a6b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 -- 172.21.15.113:0/36303260 >> 172.21.15.113:0/36303260 conn(0x7f35800b1a50 msgr2=0x7f35800b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 -- 172.21.15.113:0/36303260 shutdown_connections 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.245+0000 7f3585fab640 1 -- 172.21.15.113:0/36303260 wait complete. 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.246+0000 7f3585fab640 1 Processor -- start 2024-04-25T00:08:02.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.246+0000 7f3585fab640 1 -- start start 2024-04-25T00:08:02.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f3585fab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f358006dfd0 0x7f35801472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f3585fab640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f358006e410 0x7f3580148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f3585fab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3580072ea0 0x7f35801555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f3585fab640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3580072720 con 0x7f358006dfd0 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f3585fab640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35800b35d0 con 0x7f3580072ea0 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f3585fab640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3580071950 con 0x7f358006e410 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f358006dfd0 0x7f35801472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f358006dfd0 0x7f35801472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34714/0 (socket says 172.21.15.113:34714) 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 -- 172.21.15.113:0/1532067972 learned_addr learned my addr 172.21.15.113:0/1532067972 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357ffff640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3580072ea0 0x7f35801555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357effd640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f358006e410 0x7f3580148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 -- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f358006e410 msgr2=0x7f3580148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f358006e410 0x7f3580148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 -- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3580072ea0 msgr2=0x7f35801555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3580072ea0 0x7f35801555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 -- 172.21.15.113:0/1532067972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3580148c20 con 0x7f358006dfd0 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.247+0000 7f357f7fe640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f358006dfd0 0x7f35801472f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f35700090b0 tx=0x7f35700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.248+0000 7f3567fff640 1 -- 172.21.15.113:0/1532067972 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570041070 con 0x7f358006dfd0 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.248+0000 7f3567fff640 1 -- 172.21.15.113:0/1532067972 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f357003c890 con 0x7f358006dfd0 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.248+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3580148f00 con 0x7f358006dfd0 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.248+0000 7f3567fff640 1 -- 172.21.15.113:0/1532067972 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570037040 con 0x7f358006dfd0 2024-04-25T00:08:02.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.248+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801494c0 con 0x7f358006dfd0 2024-04-25T00:08:02.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.249+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35800691a0 con 0x7f358006dfd0 2024-04-25T00:08:02.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.249+0000 7f3567fff640 1 -- 172.21.15.113:0/1532067972 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3570044070 con 0x7f358006dfd0 2024-04-25T00:08:02.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.249+0000 7f3567fff640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f356c06be30 0x7f356c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.249+0000 7f3567fff640 1 -- 172.21.15.113:0/1532067972 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5738+0+0 (secure 0 0 0) 0x7f35700aba60 con 0x7f358006dfd0 2024-04-25T00:08:02.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.250+0000 7f357effd640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f356c06be30 0x7f356c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.250+0000 7f357effd640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f356c06be30 0x7f356c06e2f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f35800b1950 tx=0x7f3574008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.254+0000 7f3567fff640 1 -- 172.21.15.113:0/1532067972 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f357007a400 con 0x7f358006dfd0 2024-04-25T00:08:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.462+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35800b13c0 con 0x7f358006dfd0 2024-04-25T00:08:02.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.463+0000 7f3567fff640 1 -- 172.21.15.113:0/1532067972 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5197 (secure 0 0 0) 0x7f3570079d00 con 0x7f358006dfd0 2024-04-25T00:08:02.463 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:02.463 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1199283489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1199283489},{"type":"v1","addr":"172.21.15.178:6834","nonce":1199283489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4259795043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4259795043},{"type":"v1","addr":"172.21.15.178:6835","nonce":4259795043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4146002756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4146002756},{"type":"v1","addr":"172.21.15.113:6835","nonce":4146002756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:07:20.361298+0000","modified":"2024-04-25T00:07:22.348971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2111641286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2111641286},{"type":"v1","addr":"172.21.15.113:6837","nonce":2111641286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":10}]} 2024-04-25T00:08:02.464 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-04-25T00:08:02.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f356c06be30 msgr2=0x7f356c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f356c06be30 0x7f356c06e2f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f35800b1950 tx=0x7f3574008040 comp rx=0 tx=0).stop 2024-04-25T00:08:02.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f358006dfd0 msgr2=0x7f35801472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f358006dfd0 0x7f35801472f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f35700090b0 tx=0x7f35700046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 shutdown_connections 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f356c06be30 0x7f356c06e2f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3580072ea0 0x7f35801555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f358006e410 0x7f3580148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 --2- 172.21.15.113:0/1532067972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f358006dfd0 0x7f35801472f0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 >> 172.21.15.113:0/1532067972 conn(0x7f35800b1a50 msgr2=0x7f358014e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 shutdown_connections 2024-04-25T00:08:02.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.465+0000 7f3585fab640 1 -- 172.21.15.113:0/1532067972 wait complete. 2024-04-25T00:08:02.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:02.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.572+0000 7f607fa8c640 1 Processor -- start 2024-04-25T00:08:02.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.572+0000 7f607fa8c640 1 -- start start 2024-04-25T00:08:02.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607fa8c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60780b3fd0 0x7f607806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607fa8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f607806e510 0x7f60780cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607fa8c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6078144cf0 con 0x7f6078144910 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607fa8c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f60780b2f30 con 0x7f60780b3fd0 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607fa8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60780b3210 con 0x7f607806e510 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607d027640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60780b3fd0 0x7f607806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607d027640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60780b3fd0 0x7f607806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34716/0 (socket says 172.21.15.113:34716) 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607d027640 1 -- 172.21.15.113:0/1648144074 learned_addr learned my addr 172.21.15.113:0/1648144074 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607e029640 1 --2- 172.21.15.113:0/1648144074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f607806e510 0x7f60780cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607d027640 1 -- 172.21.15.113:0/1648144074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f607806e510 msgr2=0x7f60780cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607d027640 1 --2- 172.21.15.113:0/1648144074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f607806e510 0x7f60780cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607d027640 1 -- 172.21.15.113:0/1648144074 >> v1:172.21.15.178:6789/0 conn(0x7f6078144910 legacy=0x7f60780b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.573+0000 7f607d027640 1 -- 172.21.15.113:0/1648144074 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60780b34f0 con 0x7f60780b3fd0 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607d027640 1 --2- 172.21.15.113:0/1648144074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f607806dfd0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f60680098a0 tx=0x7f606802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54791398e0085279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607c826640 1 -- 172.21.15.113:0/1648144074 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068037070 con 0x7f60780b3fd0 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607c826640 1 -- 172.21.15.113:0/1648144074 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6068032440 con 0x7f60780b3fd0 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607c826640 1 -- 172.21.15.113:0/1648144074 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068032810 con 0x7f60780b3fd0 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607fa8c640 1 -- 172.21.15.113:0/1648144074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 msgr2=0x7f607806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607fa8c640 1 --2- 172.21.15.113:0/1648144074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f607806dfd0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f60680098a0 tx=0x7f606802a290 comp rx=0 tx=0).stop 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607fa8c640 1 -- 172.21.15.113:0/1648144074 shutdown_connections 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607fa8c640 1 --2- 172.21.15.113:0/1648144074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f607806e510 0x7f60780cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607fa8c640 1 --2- 172.21.15.113:0/1648144074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f607806dfd0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.574+0000 7f607fa8c640 1 -- 172.21.15.113:0/1648144074 >> 172.21.15.113:0/1648144074 conn(0x7f60780b1a50 msgr2=0x7f60780b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.575+0000 7f607fa8c640 1 -- 172.21.15.113:0/1648144074 shutdown_connections 2024-04-25T00:08:02.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.575+0000 7f607fa8c640 1 -- 172.21.15.113:0/1648144074 wait complete. 2024-04-25T00:08:02.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 Processor -- start 2024-04-25T00:08:02.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 -- start start 2024-04-25T00:08:02.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f607806e510 0x7f607813a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f60780ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6078144910 0x7f607815d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60780b4440 con 0x7f60780b3fd0 2024-04-25T00:08:02.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f607806e920 con 0x7f607806e510 2024-04-25T00:08:02.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607fa8c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60780b4da0 con 0x7f6078144910 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607d027640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f60780ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607d027640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f60780ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34732/0 (socket says 172.21.15.113:34732) 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.576+0000 7f607d027640 1 -- 172.21.15.113:0/3178555357 learned_addr learned my addr 172.21.15.113:0/3178555357 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607e029640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6078144910 0x7f607815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607d828640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f607806e510 0x7f607813a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607d027640 1 -- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6078144910 msgr2=0x7f607815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607d027640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6078144910 0x7f607815d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607d027640 1 -- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f607806e510 msgr2=0x7f607813a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607d027640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f607806e510 0x7f607813a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607d027640 1 -- 172.21.15.113:0/3178555357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60781f5e50 con 0x7f60780b3fd0 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607e029640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6078144910 0x7f607815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607d027640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f60780ce970 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6068000c00 tx=0x7f60680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f606e7fc640 1 -- 172.21.15.113:0/3178555357 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068041020 con 0x7f60780b3fd0 2024-04-25T00:08:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f606e7fc640 1 -- 172.21.15.113:0/3178555357 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60680306c0 con 0x7f60780b3fd0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60781f6150 con 0x7f60780b3fd0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f606e7fc640 1 -- 172.21.15.113:0/3178555357 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068037040 con 0x7f60780b3fd0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.577+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60781f6990 con 0x7f60780b3fd0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.578+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60780691a0 con 0x7f60780b3fd0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.578+0000 7f606e7fc640 1 -- 172.21.15.113:0/3178555357 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6068044020 con 0x7f60780b3fd0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.579+0000 7f606e7fc640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f605c06be30 0x7f605c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.579+0000 7f606e7fc640 1 -- 172.21.15.113:0/3178555357 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5738+0+0 (secure 0 0 0) 0x7f60680ab5e0 con 0x7f60780b3fd0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.579+0000 7f607d828640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f605c06be30 0x7f605c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.579+0000 7f607d828640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f605c06be30 0x7f605c06e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6074007fd0 tx=0x7f6074009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.583+0000 7f606e7fc640 1 -- 172.21.15.113:0/3178555357 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f606804b030 con 0x7f60780b3fd0 2024-04-25T00:08:02.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.793+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60780726b0 con 0x7f60780b3fd0 2024-04-25T00:08:02.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.793+0000 7f606e7fc640 1 -- 172.21.15.113:0/3178555357 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5197 (secure 0 0 0) 0x7f60781f6150 con 0x7f60780b3fd0 2024-04-25T00:08:02.794 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:02.794 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1199283489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1199283489},{"type":"v1","addr":"172.21.15.178:6834","nonce":1199283489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4259795043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4259795043},{"type":"v1","addr":"172.21.15.178:6835","nonce":4259795043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4146002756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4146002756},{"type":"v1","addr":"172.21.15.113:6835","nonce":4146002756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:07:20.361298+0000","modified":"2024-04-25T00:07:22.348971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9501":{"gid":9501,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/2111641286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2111641286},{"type":"v1","addr":"172.21.15.113:6837","nonce":2111641286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9501,"qdb_cluster":[9501]},"id":10}]} 2024-04-25T00:08:02.794 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.795+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f605c06be30 msgr2=0x7f605c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.795+0000 7f607fa8c640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f605c06be30 0x7f605c06e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6074007fd0 tx=0x7f6074009040 comp rx=0 tx=0).stop 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.795+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 msgr2=0x7f60780ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.795+0000 7f607fa8c640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f60780ce970 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6068000c00 tx=0x7f60680046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 shutdown_connections 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f605c06be30 0x7f605c06e2f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6078144910 0x7f607815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60780b3fd0 0x7f60780ce970 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 --2- 172.21.15.113:0/3178555357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f607806e510 0x7f607813a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 >> 172.21.15.113:0/3178555357 conn(0x7f60780b1a50 msgr2=0x7f6078071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 shutdown_connections 2024-04-25T00:08:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.796+0000 7f607fa8c640 1 -- 172.21.15.113:0/3178555357 wait complete. 2024-04-25T00:08:02.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:02.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.903+0000 7fd1c149a640 1 Processor -- start 2024-04-25T00:08:02.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.903+0000 7fd1c149a640 1 -- start start 2024-04-25T00:08:02.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.903+0000 7fd1c149a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1bc0b3fd0 0x7fd1bc06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.903+0000 7fd1c149a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc144cf0 con 0x7fd1bc144910 2024-04-25T00:08:02.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.903+0000 7fd1c149a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc0b2f30 con 0x7fd1bc06e510 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.903+0000 7fd1c149a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd1bc06e980 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.904+0000 7fd1ba7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1bc0b3fd0 0x7fd1bc06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.904+0000 7fd1ba7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1bc0b3fd0 0x7fd1bc06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34742/0 (socket says 172.21.15.113:34742) 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.904+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/4076468562 learned_addr learned my addr 172.21.15.113:0/4076468562 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.904+0000 7fd1baffd640 1 --1- 172.21.15.113:0/4076468562 >> v1:172.21.15.113:6789/0 conn(0x7fd1bc144910 0x7fd1bc0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60812/0 (socket says 172.21.15.113:60812) 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.904+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/4076468562 >> v1:172.21.15.178:6789/0 conn(0x7fd1bc06e510 legacy=0x7fd1bc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.904+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/4076468562 >> v1:172.21.15.113:6789/0 conn(0x7fd1bc144910 legacy=0x7fd1bc0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.904+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/4076468562 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc0b3210 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1ba7fc640 1 --2- 172.21.15.113:0/4076468562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc06dfd0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000bb40 tx=0x7fd1b002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75038f69256bef9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1b9ffb640 1 -- 172.21.15.113:0/4076468562 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0035070 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1b9ffb640 1 -- 172.21.15.113:0/4076468562 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0031610 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1b9ffb640 1 -- 172.21.15.113:0/4076468562 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00319e0 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1c149a640 1 -- 172.21.15.113:0/4076468562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 msgr2=0x7fd1bc06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1c149a640 1 --2- 172.21.15.113:0/4076468562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc06dfd0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000bb40 tx=0x7fd1b002c9f0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1c149a640 1 -- 172.21.15.113:0/4076468562 shutdown_connections 2024-04-25T00:08:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1c149a640 1 --2- 172.21.15.113:0/4076468562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc06dfd0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1c149a640 1 -- 172.21.15.113:0/4076468562 >> 172.21.15.113:0/4076468562 conn(0x7fd1bc0b1a50 msgr2=0x7fd1bc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1c149a640 1 -- 172.21.15.113:0/4076468562 shutdown_connections 2024-04-25T00:08:02.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.905+0000 7fd1c149a640 1 -- 172.21.15.113:0/4076468562 wait complete. 2024-04-25T00:08:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.906+0000 7fd1c149a640 1 Processor -- start 2024-04-25T00:08:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1c149a640 1 -- start start 2024-04-25T00:08:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1c149a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1bc06e510 0x7fd1bc13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1c149a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1c149a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1bc144910 0x7fd1bc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1c149a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0b4440 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1c149a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0b37d0 con 0x7fd1bc06e510 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1c149a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc0b4da0 con 0x7fd1bc144910 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc0aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34746/0 (socket says 172.21.15.113:34746) 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/302086348 learned_addr learned my addr 172.21.15.113:0/302086348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1baffd640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1bc06e510 0x7fd1bc13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.907+0000 7fd1bb7fe640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1bc144910 0x7fd1bc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1bc144910 msgr2=0x7fd1bc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1ba7fc640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1bc144910 0x7fd1bc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1bc06e510 msgr2=0x7fd1bc13ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1ba7fc640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1bc06e510 0x7fd1bc13ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1ba7fc640 1 -- 172.21.15.113:0/302086348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc0ad9c0 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1ba7fc640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc0aec70 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fd1b002ca30 tx=0x7fd1b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1a37fe640 1 -- 172.21.15.113:0/302086348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0035040 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1a37fe640 1 -- 172.21.15.113:0/302086348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0031e30 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc0adca0 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1a37fe640 1 -- 172.21.15.113:0/302086348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b003a670 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.908+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc0ae260 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.909+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc068a60 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.909+0000 7fd1a37fe640 1 -- 172.21.15.113:0/302086348 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1b0038070 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.910+0000 7fd1a37fe640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1a806be30 0x7fd1a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.910+0000 7fd1baffd640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1a806be30 0x7fd1a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.910+0000 7fd1a37fe640 1 -- 172.21.15.113:0/302086348 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5738+0+0 (secure 0 0 0) 0x7fd1b00abe50 con 0x7fd1bc0b3fd0 2024-04-25T00:08:02.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.910+0000 7fd1baffd640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1a806be30 0x7fd1a806e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac000f60 tx=0x7fd1ac000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:02.914+0000 7fd1a37fe640 1 -- 172.21.15.113:0/302086348 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd1bc068a60 con 0x7fd1bc0b3fd0 2024-04-25T00:08:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.087+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1bc0691a0 con 0x7fd1bc0b3fd0 2024-04-25T00:08:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.088+0000 7fd1a37fe640 1 -- 172.21.15.113:0/302086348 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16777 (secure 0 0 0) 0x7fd1b007a220 con 0x7fd1bc0b3fd0 2024-04-25T00:08:03.089 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:03.089 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":145,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:02.094572+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:07:18.614948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T00:07:19.319166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:03.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1a806be30 msgr2=0x7fd1a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1a806be30 0x7fd1a806e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac000f60 tx=0x7fd1ac000ed0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 msgr2=0x7fd1bc0aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc0aec70 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fd1b002ca30 tx=0x7fd1b0004730 comp rx=0 tx=0).stop 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 shutdown_connections 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd1a806be30 0x7fd1a806e2f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1bc144910 0x7fd1bc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1bc0b3fd0 0x7fd1bc0aec70 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.090+0000 7fd1c149a640 1 --2- 172.21.15.113:0/302086348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1bc06e510 0x7fd1bc13ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.091+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 >> 172.21.15.113:0/302086348 conn(0x7fd1bc0b1a50 msgr2=0x7fd1bc071fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.091+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 shutdown_connections 2024-04-25T00:08:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.091+0000 7fd1c149a640 1 -- 172.21.15.113:0/302086348 wait complete. 2024-04-25T00:08:03.102 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:08:03.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.200+0000 7f860cdc8640 1 Processor -- start 2024-04-25T00:08:03.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.200+0000 7f860cdc8640 1 -- start start 2024-04-25T00:08:03.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f860cdc8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86080b5ca0 0x7f86080b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f860cdc8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f86080b73c0 0x7f86080b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f860cdc8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86080d2e00 con 0x7f860815e510 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f860cdc8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f86080b6640 con 0x7f86080b5ca0 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f860cdc8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f86080d30e0 con 0x7f86080b73c0 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f85fffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86080b5ca0 0x7f86080b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f860659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f860815e510 0x7f86080d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60824/0 (socket says 172.21.15.113:60824) 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f860659c640 1 -- 172.21.15.113:0/3740804476 learned_addr learned my addr 172.21.15.113:0/3740804476 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f8606d9d640 1 --2- 172.21.15.113:0/3740804476 >> v2:172.21.15.178:3300/0 conn(0x7f86080b73c0 0x7f86080b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f85fffff640 1 -- 172.21.15.113:0/3740804476 >> v2:172.21.15.178:3300/0 conn(0x7f86080b73c0 msgr2=0x7f86080b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f85fffff640 1 --2- 172.21.15.113:0/3740804476 >> v2:172.21.15.178:3300/0 conn(0x7f86080b73c0 0x7f86080b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f8605d9b640 1 -- 172.21.15.113:0/3740804476 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 380776325 0 0) 0x7f86080d2e00 con 0x7f860815e510 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f85fffff640 1 -- 172.21.15.113:0/3740804476 >> v1:172.21.15.113:6789/0 conn(0x7f860815e510 legacy=0x7f86080d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.201+0000 7f85fffff640 1 -- 172.21.15.113:0/3740804476 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86080d33c0 con 0x7f86080b5ca0 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.202+0000 7f85fffff640 1 --2- 172.21.15.113:0/3740804476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b5ca0 0x7f86080b6100 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f86080d1d80 tx=0x7f85f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1469c05303fc6795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.202+0000 7f8605d9b640 1 -- 172.21.15.113:0/3740804476 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0037070 con 0x7f86080b5ca0 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.202+0000 7f8605d9b640 1 -- 172.21.15.113:0/3740804476 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f0032440 con 0x7f86080b5ca0 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.202+0000 7f8605d9b640 1 -- 172.21.15.113:0/3740804476 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f002ae60 con 0x7f86080b5ca0 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.202+0000 7f860cdc8640 1 -- 172.21.15.113:0/3740804476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b5ca0 msgr2=0x7f86080b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.202+0000 7f860cdc8640 1 --2- 172.21.15.113:0/3740804476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b5ca0 0x7f86080b6100 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f86080d1d80 tx=0x7f85f002a110 comp rx=0 tx=0).stop 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.203+0000 7f860cdc8640 1 -- 172.21.15.113:0/3740804476 shutdown_connections 2024-04-25T00:08:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.203+0000 7f860cdc8640 1 --2- 172.21.15.113:0/3740804476 >> v2:172.21.15.178:3300/0 conn(0x7f86080b73c0 0x7f86080b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.203+0000 7f860cdc8640 1 --2- 172.21.15.113:0/3740804476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b5ca0 0x7f86080b6100 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.203+0000 7f860cdc8640 1 -- 172.21.15.113:0/3740804476 >> 172.21.15.113:0/3740804476 conn(0x7f860806b720 msgr2=0x7f86080d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.203+0000 7f860cdc8640 1 -- 172.21.15.113:0/3740804476 shutdown_connections 2024-04-25T00:08:03.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.203+0000 7f860cdc8640 1 -- 172.21.15.113:0/3740804476 wait complete. 2024-04-25T00:08:03.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.204+0000 7f860cdc8640 1 Processor -- start 2024-04-25T00:08:03.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.204+0000 7f860cdc8640 1 -- start start 2024-04-25T00:08:03.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86080b5ca0 0x7f860814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b73c0 0x7f8608160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860815e510 0x7f86081f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86080d3ea0 con 0x7f86080b73c0 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86080d3980 con 0x7f86080b5ca0 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f86080d2a90 con 0x7f860815e510 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b73c0 0x7f8608160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b73c0 0x7f8608160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34772/0 (socket says 172.21.15.113:34772) 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 -- 172.21.15.113:0/4261446001 learned_addr learned my addr 172.21.15.113:0/4261446001 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860659c640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86080b5ca0 0x7f860814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f8606d9d640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860815e510 0x7f86081f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 -- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860815e510 msgr2=0x7f86081f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860815e510 0x7f86081f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 -- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86080b5ca0 msgr2=0x7f860814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86080b5ca0 0x7f860814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 -- 172.21.15.113:0/4261446001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860814cd10 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85fffff640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b73c0 0x7f8608160b10 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f85f0000c00 tx=0x7f85f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85feffd640 1 -- 172.21.15.113:0/4261446001 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0037040 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85feffd640 1 -- 172.21.15.113:0/4261446001 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f0042070 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860815fa70 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f85feffd640 1 -- 172.21.15.113:0/4261446001 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0030a50 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.205+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8608160030 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.206+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86080691a0 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.207+0000 7f85feffd640 1 -- 172.21.15.113:0/4261446001 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85f0030bf0 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.207+0000 7f85feffd640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85e806be60 0x7f85e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.207+0000 7f860659c640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85e806be60 0x7f85e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.207+0000 7f85feffd640 1 -- 172.21.15.113:0/4261446001 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5738+0+0 (secure 0 0 0) 0x7f85f00ab660 con 0x7f86080b73c0 2024-04-25T00:08:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.207+0000 7f860659c640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85e806be60 0x7f85e806e320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000f60 tx=0x7f85f8007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.211+0000 7f85feffd640 1 -- 172.21.15.113:0/4261446001 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f860814cd10 con 0x7f86080b73c0 2024-04-25T00:08:03.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.420+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f86080c76a0 con 0x7f86080b73c0 2024-04-25T00:08:03.421 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:08:03.421+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:08:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.424+0000 7f85feffd640 1 -- 172.21.15.113:0/4261446001 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) ==== 161+0+0 (secure 0 0 0) 0x7f85f0079a30 con 0x7f86080b73c0 2024-04-25T00:08:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:08:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85e806be60 msgr2=0x7f85e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85e806be60 0x7f85e806e320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000f60 tx=0x7f85f8007d50 comp rx=0 tx=0).stop 2024-04-25T00:08:03.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b73c0 msgr2=0x7f8608160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b73c0 0x7f8608160b10 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f85f0000c00 tx=0x7f85f00046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 shutdown_connections 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f85e806be60 0x7f85e806e320 secure :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000f60 tx=0x7f85f8007d50 comp rx=0 tx=0).stop 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f860815e510 0x7f86081f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86080b73c0 0x7f8608160b10 secure :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f85f0000c00 tx=0x7f85f00046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 --2- 172.21.15.113:0/4261446001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86080b5ca0 0x7f860814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 >> 172.21.15.113:0/4261446001 conn(0x7f860806b720 msgr2=0x7f860815cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.426+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 shutdown_connections 2024-04-25T00:08:03.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.427+0000 7f860cdc8640 1 -- 172.21.15.113:0/4261446001 wait complete. 2024-04-25T00:08:03.429 INFO:tasks.ceph.mds.a.smithi113.stderr: -3363> 2024-04-25T00:07:26.276+0000 7fcbd5bfe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:08:03.439 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:08:03.458 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:08:03.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.538+0000 7fe3e2d63640 1 Processor -- start 2024-04-25T00:08:03.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.538+0000 7fe3e2d63640 1 -- start start 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e2d63640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe3dc15b670 0x7fe3dc15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e2d63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e2d63640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3dc0cc740 0x7fe3dc06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e2d63640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe3dc15c470 con 0x7fe3dc0cc740 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e2d63640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe3dc071170 con 0x7fe3dc15b670 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e2d63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3dc071450 con 0x7fe3dc15bff0 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3dc0cc740 0x7fe3dc06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3dc0cc740 0x7fe3dc06cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34800/0 (socket says 172.21.15.113:34800) 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 -- 172.21.15.113:0/2638259159 learned_addr learned my addr 172.21.15.113:0/2638259159 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3dbfff640 1 --2- 172.21.15.113:0/2638259159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e0aff640 1 --2- 172.21.15.113:0/2638259159 >> v2:172.21.15.178:3300/0 conn(0x7fe3dc15b670 0x7fe3dc15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 -- 172.21.15.113:0/2638259159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 msgr2=0x7fe3dc0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 --2- 172.21.15.113:0/2638259159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc0cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 -- 172.21.15.113:0/2638259159 >> v2:172.21.15.178:3300/0 conn(0x7fe3dc15b670 msgr2=0x7fe3dc15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 --2- 172.21.15.113:0/2638259159 >> v2:172.21.15.178:3300/0 conn(0x7fe3dc15b670 0x7fe3dc15ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e1300640 1 -- 172.21.15.113:0/2638259159 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc071730 con 0x7fe3dc0cc740 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.539+0000 7fe3e0aff640 1 --2- 172.21.15.113:0/2638259159 >> v2:172.21.15.178:3300/0 conn(0x7fe3dc15b670 0x7fe3dc15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.540+0000 7fe3e1300640 1 --2- 172.21.15.113:0/2638259159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc0cc740 0x7fe3dc06cea0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06f1a0 tx=0x7fe3d402c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d343d460c81269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.540+0000 7fe3db7fe640 1 -- 172.21.15.113:0/2638259159 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4009600 con 0x7fe3dc0cc740 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.540+0000 7fe3db7fe640 1 -- 172.21.15.113:0/2638259159 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d40097a0 con 0x7fe3dc0cc740 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.540+0000 7fe3db7fe640 1 -- 172.21.15.113:0/2638259159 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4009b70 con 0x7fe3dc0cc740 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 -- 172.21.15.113:0/2638259159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc0cc740 msgr2=0x7fe3dc06cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/2638259159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc0cc740 0x7fe3dc06cea0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06f1a0 tx=0x7fe3d402c7c0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 -- 172.21.15.113:0/2638259159 shutdown_connections 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/2638259159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc0cc740 0x7fe3dc06cea0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/2638259159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/2638259159 >> v2:172.21.15.178:3300/0 conn(0x7fe3dc15b670 0x7fe3dc15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 -- 172.21.15.113:0/2638259159 >> 172.21.15.113:0/2638259159 conn(0x7fe3dc06b0f0 msgr2=0x7fe3dc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 -- 172.21.15.113:0/2638259159 shutdown_connections 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.541+0000 7fe3e2d63640 1 -- 172.21.15.113:0/2638259159 wait complete. 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.542+0000 7fe3e2d63640 1 Processor -- start 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.542+0000 7fe3e2d63640 1 -- start start 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e2d63640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3dc0cc740 0x7fe3dc1e4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e2d63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc15b670 0x7fe3dc1f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e2d63640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e2d63640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc06d560 con 0x7fe3dc15b670 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e2d63640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc06d3e0 con 0x7fe3dc0cc740 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e2d63640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3dc06d6e0 con 0x7fe3dc15bff0 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc15b670 0x7fe3dc1f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc15b670 0x7fe3dc1f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34806/0 (socket says 172.21.15.113:34806) 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 -- 172.21.15.113:0/1199442327 learned_addr learned my addr 172.21.15.113:0/1199442327 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e0aff640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3dc0cc740 0x7fe3dc1e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3e1300640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 -- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 msgr2=0x7fe3dc1f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc1f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 -- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3dc0cc740 msgr2=0x7fe3dc1e4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3dc0cc740 0x7fe3dc1e4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 -- 172.21.15.113:0/1199442327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc1f5410 con 0x7fe3dc15b670 2024-04-25T00:08:03.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.543+0000 7fe3dbfff640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc15b670 0x7fe3dc1f43c0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800d5f0 tx=0x7fe3c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.544+0000 7fe3d97fa640 1 -- 172.21.15.113:0/1199442327 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80125f0 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.544+0000 7fe3d97fa640 1 -- 172.21.15.113:0/1199442327 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c8019070 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.544+0000 7fe3d97fa640 1 -- 172.21.15.113:0/1199442327 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80129d0 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.544+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc1f5130 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.544+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc155920 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.545+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3dc0691a0 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.545+0000 7fe3d97fa640 1 -- 172.21.15.113:0/1199442327 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3c8012b70 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.545+0000 7fe3d97fa640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3c006be30 0x7fe3c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.545+0000 7fe3d97fa640 1 -- 172.21.15.113:0/1199442327 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5824+0+0 (secure 0 0 0) 0x7fe3c808b1e0 con 0x7fe3dc15b670 2024-04-25T00:08:03.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.546+0000 7fe3e0aff640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3c006be30 0x7fe3c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.546+0000 7fe3e0aff640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3c006be30 0x7fe3c006e2f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0006fd0 tx=0x7fe3d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.550+0000 7fe3d97fa640 1 -- 172.21.15.113:0/1199442327 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3c8059440 con 0x7fe3dc15b670 2024-04-25T00:08:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:03.758+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe3dc15b1e0 con 0x7fe3dc15b670 2024-04-25T00:08:04.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.427+0000 7fe3d97fa640 1 -- 172.21.15.113:0/1199442327 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) ==== 104+0+0 (secure 0 0 0) 0x7fe3c805d740 con 0x7fe3dc15b670 2024-04-25T00:08:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3c006be30 msgr2=0x7fe3c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3c006be30 0x7fe3c006e2f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0006fd0 tx=0x7fe3d0008040 comp rx=0 tx=0).stop 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc15b670 msgr2=0x7fe3dc1f43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc15b670 0x7fe3dc1f43c0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800d5f0 tx=0x7fe3c800dab0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 shutdown_connections 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe3c006be30 0x7fe3c006e2f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3dc15bff0 0x7fe3dc1f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3dc15b670 0x7fe3dc1f43c0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 --2- 172.21.15.113:0/1199442327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3dc0cc740 0x7fe3dc1e4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 >> 172.21.15.113:0/1199442327 conn(0x7fe3dc06b0f0 msgr2=0x7fe3dc1576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 shutdown_connections 2024-04-25T00:08:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.430+0000 7fe3e2d63640 1 -- 172.21.15.113:0/1199442327 wait complete. 2024-04-25T00:08:04.442 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:08:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.583+0000 7f5c5ce93640 1 Processor -- start 2024-04-25T00:08:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.583+0000 7f5c5ce93640 1 -- start start 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.584+0000 7f5c5ce93640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c580b9bb0 0x7f5c580b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.584+0000 7f5c5ce93640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5c580b61e0 0x7f5c580d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.584+0000 7f5c5ce93640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c581626f0 con 0x7f5c5815e510 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.584+0000 7f5c5ce93640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5c581629d0 con 0x7f5c580b9bb0 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.584+0000 7f5c5ce93640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5c58162cb0 con 0x7f5c580b61e0 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.584+0000 7f5c55d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c580b9bb0 0x7f5c580b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.584+0000 7f5c55d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c580b9bb0 0x7f5c580b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34812/0 (socket says 172.21.15.113:34812) 2024-04-25T00:08:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.585+0000 7f5c55d9b640 1 -- 172.21.15.113:0/504235892 learned_addr learned my addr 172.21.15.113:0/504235892 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.585+0000 7f5c56d9d640 1 --2- 172.21.15.113:0/504235892 >> v2:172.21.15.178:3300/0 conn(0x7f5c580b61e0 0x7f5c580d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.585+0000 7f5c55d9b640 1 -- 172.21.15.113:0/504235892 >> v2:172.21.15.178:3300/0 conn(0x7f5c580b61e0 msgr2=0x7f5c580d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.585+0000 7f5c55d9b640 1 --2- 172.21.15.113:0/504235892 >> v2:172.21.15.178:3300/0 conn(0x7f5c580b61e0 0x7f5c580d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.585+0000 7f5c55d9b640 1 -- 172.21.15.113:0/504235892 >> v1:172.21.15.178:6789/0 conn(0x7f5c5815e510 legacy=0x7f5c58161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.585+0000 7f5c55d9b640 1 -- 172.21.15.113:0/504235892 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c580d2660 con 0x7f5c580b9bb0 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.585+0000 7f5c55d9b640 1 --2- 172.21.15.113:0/504235892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c580b5ca0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5c580b6de0 tx=0x7f5c4c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2819c86f49056984 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5559a640 1 -- 172.21.15.113:0/504235892 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c037070 con 0x7f5c580b9bb0 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5559a640 1 -- 172.21.15.113:0/504235892 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c032440 con 0x7f5c580b9bb0 2024-04-25T00:08:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5559a640 1 -- 172.21.15.113:0/504235892 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c032810 con 0x7f5c580b9bb0 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 -- 172.21.15.113:0/504235892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 msgr2=0x7f5c580b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/504235892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c580b5ca0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5c580b6de0 tx=0x7f5c4c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 -- 172.21.15.113:0/504235892 shutdown_connections 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/504235892 >> v2:172.21.15.178:3300/0 conn(0x7f5c580b61e0 0x7f5c580d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/504235892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c580b5ca0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 -- 172.21.15.113:0/504235892 >> 172.21.15.113:0/504235892 conn(0x7f5c5806b720 msgr2=0x7f5c580d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 -- 172.21.15.113:0/504235892 shutdown_connections 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.586+0000 7f5c5ce93640 1 -- 172.21.15.113:0/504235892 wait complete. 2024-04-25T00:08:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 Processor -- start 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 -- start start 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c580b61e0 0x7f5c580c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c58159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5815e510 0x7f5c58152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580b7540 con 0x7f5c580b9bb0 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580b73c0 con 0x7f5c5815e510 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5ce93640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5c580d33a0 con 0x7f5c580b61e0 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c55d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c58159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c55d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c58159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34820/0 (socket says 172.21.15.113:34820) 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c55d9b640 1 -- 172.21.15.113:0/3430191809 learned_addr learned my addr 172.21.15.113:0/3430191809 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.588+0000 7f5c5659c640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c580b61e0 0x7f5c580c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c55d9b640 1 -- 172.21.15.113:0/3430191809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c580b61e0 msgr2=0x7f5c580c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c55d9b640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c580b61e0 0x7f5c580c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c55d9b640 1 -- 172.21.15.113:0/3430191809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5815e510 msgr2=0x7f5c58152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c55d9b640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5815e510 0x7f5c58152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c55d9b640 1 -- 172.21.15.113:0/3430191809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c581f80f0 con 0x7f5c580b9bb0 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c55d9b640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c58159840 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02a1e0 tx=0x7f5c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c46ffd640 1 -- 172.21.15.113:0/3430191809 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c041020 con 0x7f5c580b9bb0 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c46ffd640 1 -- 172.21.15.113:0/3430191809 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c0306c0 con 0x7f5c580b9bb0 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c581f83d0 con 0x7f5c580b9bb0 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c46ffd640 1 -- 172.21.15.113:0/3430191809 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c037040 con 0x7f5c580b9bb0 2024-04-25T00:08:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.589+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581f8990 con 0x7f5c580b9bb0 2024-04-25T00:08:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.590+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c580691a0 con 0x7f5c580b9bb0 2024-04-25T00:08:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.590+0000 7f5c46ffd640 1 -- 172.21.15.113:0/3430191809 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c4c044020 con 0x7f5c580b9bb0 2024-04-25T00:08:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.591+0000 7f5c46ffd640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c3806be30 0x7f5c3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.591+0000 7f5c46ffd640 1 -- 172.21.15.113:0/3430191809 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5824+0+0 (secure 0 0 0) 0x7f5c4c0ab610 con 0x7f5c580b9bb0 2024-04-25T00:08:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.591+0000 7f5c5659c640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c3806be30 0x7f5c3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.591+0000 7f5c5659c640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c3806be30 0x7f5c3806e2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b450 tx=0x7f5c4800d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:04.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.595+0000 7f5c46ffd640 1 -- 172.21.15.113:0/3430191809 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c4c04b030 con 0x7f5c580b9bb0 2024-04-25T00:08:04.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:04.767+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5c580ce520 con 0x7f5c580b9bb0 2024-04-25T00:08:05.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.428+0000 7f5c46ffd640 1 -- 172.21.15.113:0/3430191809 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) ==== 181+0+0 (secure 0 0 0) 0x7f5c4c07ee20 con 0x7f5c580b9bb0 2024-04-25T00:08:05.429 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:08:05.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.432+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c3806be30 msgr2=0x7f5c3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:05.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.432+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c3806be30 0x7f5c3806e2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b450 tx=0x7f5c4800d040 comp rx=0 tx=0).stop 2024-04-25T00:08:05.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.432+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 msgr2=0x7f5c58159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:05.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.432+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c58159840 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02a1e0 tx=0x7f5c4c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 shutdown_connections 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5c3806be30 0x7f5c3806e2f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5815e510 0x7f5c58152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c580b9bb0 0x7f5c58159840 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 --2- 172.21.15.113:0/3430191809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c580b61e0 0x7f5c580c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 >> 172.21.15.113:0/3430191809 conn(0x7f5c5806b720 msgr2=0x7f5c5815d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 shutdown_connections 2024-04-25T00:08:05.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.433+0000 7f5c5ce93640 1 -- 172.21.15.113:0/3430191809 wait complete. 2024-04-25T00:08:05.447 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:08:05.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.549+0000 7febbc4cd640 1 Processor -- start 2024-04-25T00:08:05.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.549+0000 7febbc4cd640 1 -- start start 2024-04-25T00:08:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.550+0000 7febbc4cd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7febb40cdd40 0x7febb406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.550+0000 7febbc4cd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb4072f70 con 0x7febb406e0d0 2024-04-25T00:08:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.550+0000 7febbc4cd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb40b2fd0 con 0x7febb4144a80 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.550+0000 7febbc4cd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7febb406ea70 con 0x7febb40cdd40 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.550+0000 7febb9a68640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7febb406e0d0 0x7febb40cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60842/0 (socket says 172.21.15.113:60842) 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.550+0000 7febb9a68640 1 -- 172.21.15.113:0/1835406426 learned_addr learned my addr 172.21.15.113:0/1835406426 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734232549 0 0) 0x7febb4072f70 con 0x7febb406e0d0 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feba0003520 con 0x7febb406e0d0 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febbaa6a640 1 --2- 172.21.15.113:0/1835406426 >> v2:172.21.15.178:3300/0 conn(0x7febb40cdd40 0x7febb406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 365810760 0 0) 0x7feba0003520 con 0x7febb406e0d0 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 >> v2:172.21.15.178:3300/0 conn(0x7febb40cdd40 msgr2=0x7febb406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 --2- 172.21.15.113:0/1835406426 >> v2:172.21.15.178:3300/0 conn(0x7febb40cdd40 0x7febb406e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 >> v1:172.21.15.178:6789/0 conn(0x7febb4144a80 legacy=0x7febb40b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb40b32b0 con 0x7febb406e0d0 2024-04-25T00:08:05.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7feba4001670 con 0x7febb406e0d0 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 >> v1:172.21.15.113:6789/0 conn(0x7febb406e0d0 legacy=0x7febb40cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039b0 0x7feba0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0004800 0x7feba0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba00051c0 0x7feba000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feba000c130 con 0x7feba0004800 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba000c2e0 con 0x7feba00039b0 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feba000c4e0 con 0x7feba00051c0 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9a68640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0004800 0x7feba0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febba269640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039b0 0x7feba0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febbaa6a640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba00051c0 0x7feba000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9a68640 1 -- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba00051c0 msgr2=0x7feba000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9a68640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba00051c0 0x7feba000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9a68640 1 -- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039b0 msgr2=0x7feba0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9a68640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039b0 0x7feba0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febb9a68640 1 -- 172.21.15.113:0/1835406426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba0003520 con 0x7feba0004800 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febba269640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba00039b0 0x7feba0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.551+0000 7febbaa6a640 1 -- 172.21.15.113:0/1835406426 reap_dead start 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.552+0000 7febb9a68640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0004800 0x7feba0004c80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7feba400c210 tx=0x7feba0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202d36ad700cf94f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.552+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4007d30 con 0x7feba0004800 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.552+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4005af0 con 0x7feba0004800 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.552+0000 7febb9267640 1 -- 172.21.15.113:0/1835406426 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4007070 con 0x7feba0004800 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.553+0000 7febbc4cd640 1 -- 172.21.15.113:0/1835406426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0004800 msgr2=0x7feba0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.553+0000 7febbc4cd640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0004800 0x7feba0004c80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7feba400c210 tx=0x7feba0005830 comp rx=0 tx=0).stop 2024-04-25T00:08:05.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.553+0000 7febbc4cd640 1 -- 172.21.15.113:0/1835406426 shutdown_connections 2024-04-25T00:08:05.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.553+0000 7febbc4cd640 1 --2- 172.21.15.113:0/1835406426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0004800 0x7feba0004c80 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.553+0000 7febbc4cd640 1 -- 172.21.15.113:0/1835406426 >> 172.21.15.113:0/1835406426 conn(0x7febb40b1b30 msgr2=0x7febb40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:05.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.553+0000 7febbc4cd640 1 -- 172.21.15.113:0/1835406426 shutdown_connections 2024-04-25T00:08:05.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.553+0000 7febbc4cd640 1 -- 172.21.15.113:0/1835406426 wait complete. 2024-04-25T00:08:05.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.554+0000 7febbc4cd640 1 Processor -- start 2024-04-25T00:08:05.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.554+0000 7febbc4cd640 1 -- start start 2024-04-25T00:08:05.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.554+0000 7febbc4cd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba0004800 0x7febb41342a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.554+0000 7febbc4cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb40aa610 0x7febb40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febbc4cd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb40ce700 0x7febb415d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febbc4cd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febb406e6b0 con 0x7febb40aa610 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febbc4cd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7febb406e530 con 0x7febb40ce700 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febbc4cd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febb406e830 con 0x7feba0004800 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb40aa610 0x7febb40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb40aa610 0x7febb40ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34828/0 (socket says 172.21.15.113:34828) 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 -- 172.21.15.113:0/3312460413 learned_addr learned my addr 172.21.15.113:0/3312460413 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febbaa6a640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb40ce700 0x7febb415d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febba269640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba0004800 0x7febb41342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 -- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba0004800 msgr2=0x7febb41342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba0004800 0x7febb41342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 -- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb40ce700 msgr2=0x7febb415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb40ce700 0x7febb415d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:05.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 -- 172.21.15.113:0/3312460413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb40a95b0 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febb9a68640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb40aa610 0x7febb40ce1c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7feba400c210 tx=0x7feba4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febaaffd640 1 -- 172.21.15.113:0/3312460413 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4040030 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febaaffd640 1 -- 172.21.15.113:0/3312460413 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4007d10 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb40a9850 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febaaffd640 1 -- 172.21.15.113:0/3312460413 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4037040 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.555+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb40a9e10 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.556+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb40691a0 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.556+0000 7febaaffd640 1 -- 172.21.15.113:0/3312460413 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feba4043070 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.557+0000 7febaaffd640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.557+0000 7febba269640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.557+0000 7febaaffd640 1 -- 172.21.15.113:0/3312460413 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5386+0+0 (secure 0 0 0) 0x7feba40aa6d0 con 0x7febb40aa610 2024-04-25T00:08:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.557+0000 7febba269640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f80 tx=0x7febb0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:05.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.561+0000 7febaaffd640 1 -- 172.21.15.113:0/3312460413 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feba4079250 con 0x7febb40aa610 2024-04-25T00:08:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:05.734+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7febb40b4240 con 0x7febb40aa610 2024-04-25T00:08:06.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.430+0000 7febaaffd640 1 -- 172.21.15.113:0/3312460413 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7feba4078b50 con 0x7febb40aa610 2024-04-25T00:08:06.430 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:08:06.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.433+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 msgr2=0x7feb9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.433+0000 7febbc4cd640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f80 tx=0x7febb0009040 comp rx=0 tx=0).stop 2024-04-25T00:08:06.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.433+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb40aa610 msgr2=0x7febb40ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.433+0000 7febbc4cd640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb40aa610 0x7febb40ce1c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7feba400c210 tx=0x7feba4001a40 comp rx=0 tx=0).stop 2024-04-25T00:08:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.434+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 shutdown_connections 2024-04-25T00:08:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.434+0000 7febbc4cd640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7feb9806be30 0x7feb9806e2f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.434+0000 7febbc4cd640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb40ce700 0x7febb415d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.434+0000 7febbc4cd640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febb40aa610 0x7febb40ce1c0 secure :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7feba400c210 tx=0x7feba4001a40 comp rx=0 tx=0).stop 2024-04-25T00:08:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.434+0000 7febbc4cd640 1 --2- 172.21.15.113:0/3312460413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feba0004800 0x7febb41342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.434+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 >> 172.21.15.113:0/3312460413 conn(0x7febb40b1b30 msgr2=0x7febb40b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.437+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 shutdown_connections 2024-04-25T00:08:06.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.437+0000 7febbc4cd640 1 -- 172.21.15.113:0/3312460413 wait complete. 2024-04-25T00:08:06.451 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:08:06.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.551+0000 7fbfa5790640 1 Processor -- start 2024-04-25T00:08:06.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbfa5790640 1 -- start start 2024-04-25T00:08:06.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbfa5790640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 0x7fbfa015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbfa5790640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbfa00b4a40 0x7fbfa00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbfa5790640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa015f4f0 con 0x7fbfa00b3080 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbfa5790640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa00cde00 con 0x7fbfa00b4a40 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbfa5790640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa015f7d0 con 0x7fbfa015e2d0 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbf9e7fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbfa00b3080 0x7fbfa00b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60850/0 (socket says 172.21.15.113:60850) 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.552+0000 7fbf9e7fc640 1 -- 172.21.15.113:0/4079921272 learned_addr learned my addr 172.21.15.113:0/4079921272 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819291221 0 0) 0x7fbfa015f4f0 con 0x7fbfa00b3080 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf88003520 con 0x7fbfa00b3080 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9f7fe640 1 --2- 172.21.15.113:0/4079921272 >> v2:172.21.15.178:3300/0 conn(0x7fbfa00b4a40 0x7fbfa00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9effd640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 0x7fbfa015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3452606214 0 0) 0x7fbf88003520 con 0x7fbfa00b3080 2024-04-25T00:08:06.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 msgr2=0x7fbfa015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 0x7fbfa015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 >> v2:172.21.15.178:3300/0 conn(0x7fbfa00b4a40 msgr2=0x7fbfa00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 --2- 172.21.15.113:0/4079921272 >> v2:172.21.15.178:3300/0 conn(0x7fbfa00b4a40 0x7fbfa00ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa015fab0 con 0x7fbfa00b3080 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fbf90001670 con 0x7fbfa00b3080 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9effd640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 0x7fbfa015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 >> v1:172.21.15.113:6789/0 conn(0x7fbfa00b3080 legacy=0x7fbfa00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 0x7fbf880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88003ef0 0x7fbf88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf88004d40 0x7fbf880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa00cde00 con 0x7fbf88004d40 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa015f7d0 con 0x7fbf88003ef0 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8800dc00 con 0x7fbfa015e2d0 2024-04-25T00:08:06.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9effd640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 0x7fbf880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9e7fc640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88003ef0 0x7fbf88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.553+0000 7fbf9f7fe640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf88004d40 0x7fbf880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9f7fe640 1 -- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 msgr2=0x7fbf880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9f7fe640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa015e2d0 0x7fbf880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9f7fe640 1 -- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88003ef0 msgr2=0x7fbf88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9f7fe640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88003ef0 0x7fbf88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9f7fe640 1 -- 172.21.15.113:0/4079921272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf88003520 con 0x7fbf88004d40 2024-04-25T00:08:06.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9f7fe640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf88004d40 0x7fbf880051c0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fc0 tx=0x7fbf88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a934cadced2fa46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98009c10 con 0x7fbf88004d40 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf98036070 con 0x7fbf88004d40 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.554+0000 7fbf9dffb640 1 -- 172.21.15.113:0/4079921272 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9802d070 con 0x7fbf88004d40 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.555+0000 7fbfa5790640 1 -- 172.21.15.113:0/4079921272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf88004d40 msgr2=0x7fbf880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.555+0000 7fbfa5790640 1 --2- 172.21.15.113:0/4079921272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf88004d40 0x7fbf880051c0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fc0 tx=0x7fbf88005830 comp rx=0 tx=0).stop 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.555+0000 7fbf9f7fe640 1 -- 172.21.15.113:0/4079921272 reap_dead start 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.555+0000 7fbfa5790640 1 -- 172.21.15.113:0/4079921272 shutdown_connections 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.555+0000 7fbfa5790640 1 -- 172.21.15.113:0/4079921272 >> 172.21.15.113:0/4079921272 conn(0x7fbfa006b0f0 msgr2=0x7fbfa006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.555+0000 7fbfa5790640 1 -- 172.21.15.113:0/4079921272 shutdown_connections 2024-04-25T00:08:06.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.555+0000 7fbfa5790640 1 -- 172.21.15.113:0/4079921272 wait complete. 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.556+0000 7fbfa5790640 1 Processor -- start 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.556+0000 7fbfa5790640 1 -- start start 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbfa5790640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88004d40 0x7fbfa014dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbfa5790640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa00a9480 0x7fbfa00aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbfa5790640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa00cf1e0 0x7fbfa00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbfa5790640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa015e830 con 0x7fbfa00a9480 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbfa5790640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa015e6b0 con 0x7fbf88004d40 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbfa5790640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa015e9b0 con 0x7fbfa00cf1e0 2024-04-25T00:08:06.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa00a9480 0x7fbfa00aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa00a9480 0x7fbfa00aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34846/0 (socket says 172.21.15.113:34846) 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 -- 172.21.15.113:0/1988247663 learned_addr learned my addr 172.21.15.113:0/1988247663 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9effd640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88004d40 0x7fbfa014dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9f7fe640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa00cf1e0 0x7fbfa00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 -- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa00cf1e0 msgr2=0x7fbfa00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa00cf1e0 0x7fbfa00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 -- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88004d40 msgr2=0x7fbfa014dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88004d40 0x7fbfa014dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.557+0000 7fbf9e7fc640 1 -- 172.21.15.113:0/1988247663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa00aa4e0 con 0x7fbfa00a9480 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.558+0000 7fbf9e7fc640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa00a9480 0x7fbfa00aae40 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000e780 tx=0x7fbf9000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.558+0000 7fbf877fe640 1 -- 172.21.15.113:0/1988247663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000c8d0 con 0x7fbfa00a9480 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.558+0000 7fbf877fe640 1 -- 172.21.15.113:0/1988247663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf90018070 con 0x7fbfa00a9480 2024-04-25T00:08:06.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.558+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa00ab380 con 0x7fbfa00a9480 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.558+0000 7fbf877fe640 1 -- 172.21.15.113:0/1988247663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000ccb0 con 0x7fbfa00a9480 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.558+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa00a9c40 con 0x7fbfa00a9480 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.559+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa00691a0 con 0x7fbfa00a9480 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.559+0000 7fbf877fe640 1 -- 172.21.15.113:0/1988247663 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf90013420 con 0x7fbfa00a9480 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.559+0000 7fbf877fe640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf8c06be60 0x7fbf8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.560+0000 7fbf877fe640 1 -- 172.21.15.113:0/1988247663 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4943+0+0 (secure 0 0 0) 0x7fbf9008a730 con 0x7fbfa00a9480 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.560+0000 7fbf9effd640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf8c06be60 0x7fbf8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.560+0000 7fbf9effd640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf8c06be60 0x7fbf8c06e320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006fd0 tx=0x7fbf94008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.564+0000 7fbf877fe640 1 -- 172.21.15.113:0/1988247663 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf900593e0 con 0x7fbfa00a9480 2024-04-25T00:08:06.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.741+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fbfa00cfbd0 con 0x7fbfa00a9480 2024-04-25T00:08:06.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.744+0000 7fbf877fe640 1 -- 172.21.15.113:0/1988247663 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fbf90058ce0 con 0x7fbfa00a9480 2024-04-25T00:08:06.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf8c06be60 msgr2=0x7fbf8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf8c06be60 0x7fbf8c06e320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbf94006fd0 tx=0x7fbf94008040 comp rx=0 tx=0).stop 2024-04-25T00:08:06.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa00a9480 msgr2=0x7fbfa00aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa00a9480 0x7fbfa00aae40 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000e780 tx=0x7fbf9000ec40 comp rx=0 tx=0).stop 2024-04-25T00:08:06.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 shutdown_connections 2024-04-25T00:08:06.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf8c06be60 0x7fbf8c06e320 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfa00cf1e0 0x7fbfa00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa00a9480 0x7fbfa00aae40 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 --2- 172.21.15.113:0/1988247663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88004d40 0x7fbfa014dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 >> 172.21.15.113:0/1988247663 conn(0x7fbfa006b0f0 msgr2=0x7fbfa00cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.746+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 shutdown_connections 2024-04-25T00:08:06.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.747+0000 7fbfa5790640 1 -- 172.21.15.113:0/1988247663 wait complete. 2024-04-25T00:08:06.758 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T00:08:06.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.854+0000 7f91d14d4640 1 Processor -- start 2024-04-25T00:08:06.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.854+0000 7f91d14d4640 1 -- start start 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91d14d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc15e320 0x7f91cc0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91d14d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f91cc0d0b00 0x7f91cc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91d14d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc0d3de0 con 0x7f91cc0d2290 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91d14d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f91cc0d14d0 con 0x7f91cc0d0b00 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91d14d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc0d3410 con 0x7f91cc15e320 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91cb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f91cc0d0b00 0x7f91cc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91cb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f91cc0d0b00 0x7f91cc0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34856/0 (socket says 172.21.15.113:34856) 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.855+0000 7f91cb7fe640 1 -- 172.21.15.113:0/3422044155 learned_addr learned my addr 172.21.15.113:0/3422044155 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91caffd640 1 --2- 172.21.15.113:0/3422044155 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc15e320 0x7f91cc0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91cb7fe640 1 -- 172.21.15.113:0/3422044155 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc15e320 msgr2=0x7f91cc0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91cb7fe640 1 --2- 172.21.15.113:0/3422044155 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc15e320 0x7f91cc0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91cb7fe640 1 -- 172.21.15.113:0/3422044155 >> v1:172.21.15.113:6789/0 conn(0x7f91cc0d2290 legacy=0x7f91cc0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91cb7fe640 1 -- 172.21.15.113:0/3422044155 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc0d2800 con 0x7f91cc0d0b00 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91cb7fe640 1 --2- 172.21.15.113:0/3422044155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc0d0b00 0x7f91cc0d0f90 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009e50 tx=0x7f91c402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a037acb0cd78ee0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91ca7fc640 1 -- 172.21.15.113:0/3422044155 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4036070 con 0x7f91cc0d0b00 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.856+0000 7f91ca7fc640 1 -- 172.21.15.113:0/3422044155 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c4031600 con 0x7f91cc0d0b00 2024-04-25T00:08:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91ca7fc640 1 -- 172.21.15.113:0/3422044155 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40319d0 con 0x7f91cc0d0b00 2024-04-25T00:08:06.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 -- 172.21.15.113:0/3422044155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc0d0b00 msgr2=0x7f91cc0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 --2- 172.21.15.113:0/3422044155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc0d0b00 0x7f91cc0d0f90 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009e50 tx=0x7f91c402a7a0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 -- 172.21.15.113:0/3422044155 shutdown_connections 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 --2- 172.21.15.113:0/3422044155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc0d0b00 0x7f91cc0d0f90 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 --2- 172.21.15.113:0/3422044155 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc15e320 0x7f91cc0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 -- 172.21.15.113:0/3422044155 >> 172.21.15.113:0/3422044155 conn(0x7f91cc0b5c20 msgr2=0x7f91cc0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 -- 172.21.15.113:0/3422044155 shutdown_connections 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.857+0000 7f91d14d4640 1 -- 172.21.15.113:0/3422044155 wait complete. 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.858+0000 7f91d14d4640 1 Processor -- start 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.858+0000 7f91d14d4640 1 -- start start 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91d14d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc0d0b00 0x7f91cc0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91d14d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91cc0d2290 0x7f91cc15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91d14d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc15e320 0x7f91cc159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91d14d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0d30a0 con 0x7f91cc15e320 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91d14d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0d2dc0 con 0x7f91cc0d2290 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91d14d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc15e700 con 0x7f91cc0d0b00 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91cb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc15e320 0x7f91cc159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91cb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc15e320 0x7f91cc159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34868/0 (socket says 172.21.15.113:34868) 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91cb7fe640 1 -- 172.21.15.113:0/1703372394 learned_addr learned my addr 172.21.15.113:0/1703372394 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91caffd640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc0d0b00 0x7f91cc0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91c3fff640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91cc0d2290 0x7f91cc15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91cb7fe640 1 -- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc0d0b00 msgr2=0x7f91cc0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91cb7fe640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc0d0b00 0x7f91cc0b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91cb7fe640 1 -- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91cc0d2290 msgr2=0x7f91cc15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.859+0000 7f91cb7fe640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91cc0d2290 0x7f91cc15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91cb7fe640 1 -- 172.21.15.113:0/1703372394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc15a330 con 0x7f91cc15e320 2024-04-25T00:08:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91c3fff640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91cc0d2290 0x7f91cc15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91caffd640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc0d0b00 0x7f91cc0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91cb7fe640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc15e320 0x7f91cc159df0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009e50 tx=0x7f91c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91c37fe640 1 -- 172.21.15.113:0/1703372394 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4036050 con 0x7f91cc15e320 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91c37fe640 1 -- 172.21.15.113:0/1703372394 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c4042070 con 0x7f91cc15e320 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc0b8b50 con 0x7f91cc15e320 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91c37fe640 1 -- 172.21.15.113:0/1703372394 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4009510 con 0x7f91cc15e320 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.860+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc0b9110 con 0x7f91cc15e320 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.861+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91cc0b9720 con 0x7f91cc15e320 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.861+0000 7f91c37fe640 1 -- 172.21.15.113:0/1703372394 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91c4007b30 con 0x7f91cc15e320 2024-04-25T00:08:06.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.862+0000 7f91c37fe640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.862+0000 7f91c37fe640 1 -- 172.21.15.113:0/1703372394 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4943+0+0 (secure 0 0 0) 0x7f91c40ab960 con 0x7f91cc15e320 2024-04-25T00:08:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.862+0000 7f91caffd640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.862+0000 7f91caffd640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ba150 tx=0x7f91bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:06.866+0000 7f91c37fe640 1 -- 172.21.15.113:0/1703372394 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91c4007e20 con 0x7f91cc15e320 2024-04-25T00:08:07.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.104+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) -- 0x7f91cc0691a0 con 0x7f91cc15e320 2024-04-25T00:08:07.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.436+0000 7f91c37fe640 1 -- 172.21.15.113:0/1703372394 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v480) ==== 140+0+0 (secure 0 0 0) 0x7f91c407a7c0 con 0x7f91cc15e320 2024-04-25T00:08:07.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91ac06be30 msgr2=0x7f91ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ba150 tx=0x7f91bc008040 comp rx=0 tx=0).stop 2024-04-25T00:08:07.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc15e320 msgr2=0x7f91cc159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc15e320 0x7f91cc159df0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009e50 tx=0x7f91c4004a40 comp rx=0 tx=0).stop 2024-04-25T00:08:07.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 shutdown_connections 2024-04-25T00:08:07.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91cc0d2290 0x7f91cc15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91cc15e320 0x7f91cc159df0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 --2- 172.21.15.113:0/1703372394 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91cc0d0b00 0x7f91cc0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 >> 172.21.15.113:0/1703372394 conn(0x7f91cc0b5c20 msgr2=0x7f91cc0d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:07.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 shutdown_connections 2024-04-25T00:08:07.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.439+0000 7f91d14d4640 1 -- 172.21.15.113:0/1703372394 wait complete. 2024-04-25T00:08:07.451 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T00:08:07.451 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T00:08:07.452 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:07.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.550+0000 7fef69dc8640 1 Processor -- start 2024-04-25T00:08:07.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.550+0000 7fef69dc8640 1 -- start start 2024-04-25T00:08:07.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.550+0000 7fef69dc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef6415e320 0x7fef640d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.550+0000 7fef69dc8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fef640d0b00 0x7fef640d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.550+0000 7fef69dc8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef640d3de0 con 0x7fef640d2290 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.550+0000 7fef69dc8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fef640d14d0 con 0x7fef640d0b00 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.550+0000 7fef69dc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef640d3410 con 0x7fef6415e320 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef637fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef6415e320 0x7fef640d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef63fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fef640d0b00 0x7fef640d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef637fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef6415e320 0x7fef640d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:58774/0 (socket says 172.21.15.113:58774) 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef637fe640 1 -- 172.21.15.113:0/1694992998 learned_addr learned my addr 172.21.15.113:0/1694992998 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef637fe640 1 -- 172.21.15.113:0/1694992998 >> v2:172.21.15.178:3300/0 conn(0x7fef640d0b00 msgr2=0x7fef640d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef637fe640 1 --2- 172.21.15.113:0/1694992998 >> v2:172.21.15.178:3300/0 conn(0x7fef640d0b00 0x7fef640d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef637fe640 1 -- 172.21.15.113:0/1694992998 >> v1:172.21.15.178:6789/0 conn(0x7fef640d2290 legacy=0x7fef640b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.551+0000 7fef637fe640 1 -- 172.21.15.113:0/1694992998 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef640d2800 con 0x7fef6415e320 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.552+0000 7fef637fe640 1 --2- 172.21.15.113:0/1694992998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef6415e320 0x7fef640d38a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef54009870 tx=0x7fef5402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6122924e44dde06d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.552+0000 7fef627fc640 1 -- 172.21.15.113:0/1694992998 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54037070 con 0x7fef6415e320 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.552+0000 7fef627fc640 1 -- 172.21.15.113:0/1694992998 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5402ae50 con 0x7fef6415e320 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.552+0000 7fef627fc640 1 -- 172.21.15.113:0/1694992998 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54032860 con 0x7fef6415e320 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 -- 172.21.15.113:0/1694992998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef6415e320 msgr2=0x7fef640d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1694992998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef6415e320 0x7fef640d38a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef54009870 tx=0x7fef5402a300 comp rx=0 tx=0).stop 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 -- 172.21.15.113:0/1694992998 shutdown_connections 2024-04-25T00:08:07.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1694992998 >> v2:172.21.15.178:3300/0 conn(0x7fef640d0b00 0x7fef640d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1694992998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef6415e320 0x7fef640d38a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 -- 172.21.15.113:0/1694992998 >> 172.21.15.113:0/1694992998 conn(0x7fef640b5c20 msgr2=0x7fef640b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:07.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 -- 172.21.15.113:0/1694992998 shutdown_connections 2024-04-25T00:08:07.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.553+0000 7fef69dc8640 1 -- 172.21.15.113:0/1694992998 wait complete. 2024-04-25T00:08:07.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.554+0000 7fef69dc8640 1 Processor -- start 2024-04-25T00:08:07.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.554+0000 7fef69dc8640 1 -- start start 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef640d0b00 0x7fef641f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef640d2290 0x7fef64156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef6415e320 0x7fef64156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fef640d30a0 con 0x7fef640d2290 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef640d2dc0 con 0x7fef6415e320 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef6415e700 con 0x7fef640d0b00 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef640d2290 0x7fef64156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef637fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef640d0b00 0x7fef641f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef637fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef640d0b00 0x7fef641f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:58784/0 (socket says 172.21.15.113:58784) 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef640d2290 0x7fef64156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48614/0 (socket says 172.21.15.113:48614) 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef637fe640 1 -- 172.21.15.113:0/1752479995 learned_addr learned my addr 172.21.15.113:0/1752479995 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef63fff640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef6415e320 0x7fef64156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 -- 172.21.15.113:0/1752479995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef640d0b00 msgr2=0x7fef641f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef640d0b00 0x7fef641f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 -- 172.21.15.113:0/1752479995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef6415e320 msgr2=0x7fef64156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef6415e320 0x7fef64156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 -- 172.21.15.113:0/1752479995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef640b9390 con 0x7fef640d2290 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef62ffd640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef640d2290 0x7fef64156a00 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c370 tx=0x7fef5800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef4bfff640 1 -- 172.21.15.113:0/1752479995 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580052d0 con 0x7fef640d2290 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef4bfff640 1 -- 172.21.15.113:0/1752479995 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef58005490 con 0x7fef640d2290 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef640b9670 con 0x7fef640d2290 2024-04-25T00:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef4bfff640 1 -- 172.21.15.113:0/1752479995 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58007260 con 0x7fef640d2290 2024-04-25T00:08:07.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.555+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef640b9c30 con 0x7fef640d2290 2024-04-25T00:08:07.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.556+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef640691a0 con 0x7fef640d2290 2024-04-25T00:08:07.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.557+0000 7fef4bfff640 1 -- 172.21.15.113:0/1752479995 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef58007400 con 0x7fef640d2290 2024-04-25T00:08:07.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.557+0000 7fef4bfff640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fef5006be30 0x7fef5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.557+0000 7fef4bfff640 1 -- 172.21.15.113:0/1752479995 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4943+0+0 (secure 0 0 0) 0x7fef58089ff0 con 0x7fef640d2290 2024-04-25T00:08:07.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.557+0000 7fef637fe640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fef5006be30 0x7fef5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.558+0000 7fef637fe640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fef5006be30 0x7fef5006e2f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef54035040 tx=0x7fef5402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.561+0000 7fef4bfff640 1 -- 172.21.15.113:0/1752479995 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fef580585d0 con 0x7fef640d2290 2024-04-25T00:08:07.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.737+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef6415f9c0 con 0x7fef640d2290 2024-04-25T00:08:07.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.738+0000 7fef4bfff640 1 -- 172.21.15.113:0/1752479995 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+13080 (secure 0 0 0) 0x7fef5805c8d0 con 0x7fef640d2290 2024-04-25T00:08:07.738 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:07.738 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":149,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:06.429336+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6837/2111641286":"2024-04-26T00:08:03.422014+0000","172.21.15.113:6836/2111641286":"2024-04-26T00:08:03.422014+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:07.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.739+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fef5006be30 msgr2=0x7fef5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.739+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fef5006be30 0x7fef5006e2f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef54035040 tx=0x7fef5402e000 comp rx=0 tx=0).stop 2024-04-25T00:08:07.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.739+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef640d2290 msgr2=0x7fef64156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.739+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef640d2290 0x7fef64156a00 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c370 tx=0x7fef5800c830 comp rx=0 tx=0).stop 2024-04-25T00:08:07.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 shutdown_connections 2024-04-25T00:08:07.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fef5006be30 0x7fef5006e2f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef6415e320 0x7fef64156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef640d2290 0x7fef64156a00 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 --2- 172.21.15.113:0/1752479995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef640d0b00 0x7fef641f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 >> 172.21.15.113:0/1752479995 conn(0x7fef640b5c20 msgr2=0x7fef64071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:07.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 shutdown_connections 2024-04-25T00:08:07.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.740+0000 7fef69dc8640 1 -- 172.21.15.113:0/1752479995 wait complete. 2024-04-25T00:08:07.751 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:08:07.752 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:08:07.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.851+0000 7fd4035c8640 1 Processor -- start 2024-04-25T00:08:07.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.851+0000 7fd4035c8640 1 -- start start 2024-04-25T00:08:07.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd4035c8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3fc15e2d0 0x7fd3fc15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd4035c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd4035c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3fc15f4f0 con 0x7fd3fc15e6b0 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd4035c8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd3fc0cde00 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd4035c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc15f7d0 con 0x7fd3fc0b3080 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd401364640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3fc15e2d0 0x7fd3fc15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd401364640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3fc15e2d0 0x7fd3fc15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48620/0 (socket says 172.21.15.113:48620) 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd401364640 1 -- 172.21.15.113:0/3086397696 learned_addr learned my addr 172.21.15.113:0/3086397696 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd400b63640 1 --2- 172.21.15.113:0/3086397696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd401364640 1 -- 172.21.15.113:0/3086397696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 msgr2=0x7fd3fc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:07.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd401364640 1 --2- 172.21.15.113:0/3086397696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc0b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd401364640 1 -- 172.21.15.113:0/3086397696 >> v1:172.21.15.178:6789/0 conn(0x7fd3fc15e6b0 legacy=0x7fd3fc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:07.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.852+0000 7fd401364640 1 -- 172.21.15.113:0/3086397696 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc15fab0 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.853+0000 7fd401364640 1 --2- 172.21.15.113:0/3086397696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc15ebf0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009870 tx=0x7fd3f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97daf3fd7027a716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:07.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.853+0000 7fd3f3fff640 1 -- 172.21.15.113:0/3086397696 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8037070 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.853+0000 7fd3f3fff640 1 -- 172.21.15.113:0/3086397696 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8032440 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.853+0000 7fd3f3fff640 1 -- 172.21.15.113:0/3086397696 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8032810 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.853+0000 7fd4035c8640 1 -- 172.21.15.113:0/3086397696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 msgr2=0x7fd3fc15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.853+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3086397696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc15ebf0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009870 tx=0x7fd3f802a180 comp rx=0 tx=0).stop 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.854+0000 7fd4035c8640 1 -- 172.21.15.113:0/3086397696 shutdown_connections 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.854+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3086397696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.854+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3086397696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc15ebf0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.854+0000 7fd4035c8640 1 -- 172.21.15.113:0/3086397696 >> 172.21.15.113:0/3086397696 conn(0x7fd3fc06b0f0 msgr2=0x7fd3fc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.854+0000 7fd4035c8640 1 -- 172.21.15.113:0/3086397696 shutdown_connections 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.854+0000 7fd4035c8640 1 -- 172.21.15.113:0/3086397696 wait complete. 2024-04-25T00:08:07.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 Processor -- start 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 -- start start 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc1f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc15e6b0 0x7fd3fc0afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc15f2b0 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc15f130 con 0x7fd3fc15e6b0 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.855+0000 7fd4035c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc0cf2a0 con 0x7fd3fc0b3080 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc14ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48632/0 (socket says 172.21.15.113:48632) 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 -- 172.21.15.113:0/3142296081 learned_addr learned my addr 172.21.15.113:0/3142296081 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd401364640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc1f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd401b65640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc15e6b0 0x7fd3fc0afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 -- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 msgr2=0x7fd3fc1f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc1f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 -- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc15e6b0 msgr2=0x7fd3fc0afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc15e6b0 0x7fd3fc0afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 -- 172.21.15.113:0/3142296081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc14f1d0 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd401b65640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc15e6b0 0x7fd3fc0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd400b63640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc14ec90 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00c7f0 tx=0x7fd3ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd3f1ffb640 1 -- 172.21.15.113:0/3142296081 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec015070 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd3f1ffb640 1 -- 172.21.15.113:0/3142296081 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ec004220 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd3f1ffb640 1 -- 172.21.15.113:0/3142296081 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec007620 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc1542c0 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.856+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc154880 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.857+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc0691a0 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.858+0000 7fd3f1ffb640 1 -- 172.21.15.113:0/3142296081 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3ec0079d0 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.858+0000 7fd3f1ffb640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd3e006be30 0x7fd3e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:07.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.858+0000 7fd3f1ffb640 1 -- 172.21.15.113:0/3142296081 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4943+0+0 (secure 0 0 0) 0x7fd3ec08af30 con 0x7fd3fc15e2d0 2024-04-25T00:08:07.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.858+0000 7fd401364640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd3e006be30 0x7fd3e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:07.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.858+0000 7fd401364640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd3e006be30 0x7fd3e006e2f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8034000 tx=0x7fd3f802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:07.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:07.862+0000 7fd3f1ffb640 1 -- 172.21.15.113:0/3142296081 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3fc14f1d0 con 0x7fd3fc15e2d0 2024-04-25T00:08:08.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.034+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3fc072050 con 0x7fd3fc15e2d0 2024-04-25T00:08:08.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.034+0000 7fd3f1ffb640 1 -- 172.21.15.113:0/3142296081 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) ==== 93+0+10 (secure 0 0 0) 0x7fd3ec059be0 con 0x7fd3fc15e2d0 2024-04-25T00:08:08.035 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd3e006be30 msgr2=0x7fd3e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd3e006be30 0x7fd3e006e2f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8034000 tx=0x7fd3f802e000 comp rx=0 tx=0).stop 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 msgr2=0x7fd3fc14ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc14ec90 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00c7f0 tx=0x7fd3ec00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 shutdown_connections 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd3e006be30 0x7fd3e006e2f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc15e6b0 0x7fd3fc0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc15e2d0 0x7fd3fc14ec90 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 --2- 172.21.15.113:0/3142296081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3fc0b3080 0x7fd3fc1f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 >> 172.21.15.113:0/3142296081 conn(0x7fd3fc06b0f0 msgr2=0x7fd3fc0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 shutdown_connections 2024-04-25T00:08:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.036+0000 7fd4035c8640 1 -- 172.21.15.113:0/3142296081 wait complete. 2024-04-25T00:08:08.048 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T00:08:08.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.144+0000 7ffb3c789640 1 Processor -- start 2024-04-25T00:08:08.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.144+0000 7ffb3c789640 1 -- start start 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3c789640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb3415a6b0 0x7ffb34070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3c789640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb34072ea0 0x7ffb34071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3c789640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb340b2d50 con 0x7ffb34071780 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3c789640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ffb34072a70 con 0x7ffb3415a6b0 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3c789640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffb340b3010 con 0x7ffb34072ea0 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3a525640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb3415a6b0 0x7ffb34070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3ad26640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffb34071780 0x7ffb3415e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58450/0 (socket says 172.21.15.113:58450) 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3ad26640 1 -- 172.21.15.113:0/3126990700 learned_addr learned my addr 172.21.15.113:0/3126990700 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:08.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb39d24640 1 --2- 172.21.15.113:0/3126990700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb34072ea0 0x7ffb34071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3a525640 1 -- 172.21.15.113:0/3126990700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb34072ea0 msgr2=0x7ffb34071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:08.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3a525640 1 --2- 172.21.15.113:0/3126990700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb34072ea0 0x7ffb34071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.145+0000 7ffb3a525640 1 -- 172.21.15.113:0/3126990700 >> v1:172.21.15.113:6789/0 conn(0x7ffb34071780 legacy=0x7ffb3415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.146+0000 7ffb39523640 1 -- 172.21.15.113:0/3126990700 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1235119423 0 0) 0x7ffb340b2d50 con 0x7ffb34071780 2024-04-25T00:08:08.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.146+0000 7ffb3a525640 1 -- 172.21.15.113:0/3126990700 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb340b32f0 con 0x7ffb3415a6b0 2024-04-25T00:08:08.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.146+0000 7ffb3a525640 1 --2- 172.21.15.113:0/3126990700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb3415a6b0 0x7ffb34070d00 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ffb240097a0 tx=0x7ffb2402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f982a345f976f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.146+0000 7ffb39523640 1 -- 172.21.15.113:0/3126990700 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24037070 con 0x7ffb3415a6b0 2024-04-25T00:08:08.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.146+0000 7ffb39523640 1 -- 172.21.15.113:0/3126990700 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb24032440 con 0x7ffb3415a6b0 2024-04-25T00:08:08.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.146+0000 7ffb39523640 1 -- 172.21.15.113:0/3126990700 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24032810 con 0x7ffb3415a6b0 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 -- 172.21.15.113:0/3126990700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb3415a6b0 msgr2=0x7ffb34070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 --2- 172.21.15.113:0/3126990700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb3415a6b0 0x7ffb34070d00 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ffb240097a0 tx=0x7ffb2402a180 comp rx=0 tx=0).stop 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 -- 172.21.15.113:0/3126990700 shutdown_connections 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 --2- 172.21.15.113:0/3126990700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb34072ea0 0x7ffb34071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 --2- 172.21.15.113:0/3126990700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb3415a6b0 0x7ffb34070d00 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 -- 172.21.15.113:0/3126990700 >> 172.21.15.113:0/3126990700 conn(0x7ffb340b1a50 msgr2=0x7ffb340b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 -- 172.21.15.113:0/3126990700 shutdown_connections 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.147+0000 7ffb3c789640 1 -- 172.21.15.113:0/3126990700 wait complete. 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.148+0000 7ffb3c789640 1 Processor -- start 2024-04-25T00:08:08.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.148+0000 7ffb3c789640 1 -- start start 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3c789640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34071780 0x7ffb3413e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3c789640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb34072ea0 0x7ffb340b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3c789640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb3415a6b0 0x7ffb340b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3c789640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3415aa90 con 0x7ffb34071780 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3c789640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffb34072720 con 0x7ffb34072ea0 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3c789640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3415ac10 con 0x7ffb3415a6b0 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34071780 0x7ffb3413e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34071780 0x7ffb3413e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48652/0 (socket says 172.21.15.113:48652) 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 -- 172.21.15.113:0/182826528 learned_addr learned my addr 172.21.15.113:0/182826528 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3ad26640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb3415a6b0 0x7ffb340b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 -- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb3415a6b0 msgr2=0x7ffb340b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb39d24640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb34072ea0 0x7ffb340b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb3415a6b0 0x7ffb340b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 -- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb34072ea0 msgr2=0x7ffb340b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb34072ea0 0x7ffb340b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 -- 172.21.15.113:0/182826528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb34141950 con 0x7ffb34071780 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3ad26640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb3415a6b0 0x7ffb340b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb39d24640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb34072ea0 0x7ffb340b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3a525640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34071780 0x7ffb3413e720 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098c0 tx=0x7ffb240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb32ffd640 1 -- 172.21.15.113:0/182826528 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24037040 con 0x7ffb34071780 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb32ffd640 1 -- 172.21.15.113:0/182826528 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb24042070 con 0x7ffb34071780 2024-04-25T00:08:08.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb34141c30 con 0x7ffb34071780 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.149+0000 7ffb32ffd640 1 -- 172.21.15.113:0/182826528 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24030bb0 con 0x7ffb34071780 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.150+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb341421f0 con 0x7ffb34071780 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.150+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb340691a0 con 0x7ffb34071780 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.151+0000 7ffb32ffd640 1 -- 172.21.15.113:0/182826528 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb24047460 con 0x7ffb34071780 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.151+0000 7ffb32ffd640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb2006be30 0x7ffb2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.151+0000 7ffb32ffd640 1 -- 172.21.15.113:0/182826528 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4943+0+0 (secure 0 0 0) 0x7ffb240ab750 con 0x7ffb34071780 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.151+0000 7ffb39d24640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb2006be30 0x7ffb2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.152+0000 7ffb39d24640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb2006be30 0x7ffb2006e2f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffb34147250 tx=0x7ffb28008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.155+0000 7ffb32ffd640 1 -- 172.21.15.113:0/182826528 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb24032e20 con 0x7ffb34071780 2024-04-25T00:08:08.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.400+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) -- 0x7ffb3413ec60 con 0x7ffb34071780 2024-04-25T00:08:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.437+0000 7ffb32ffd640 1 -- 172.21.15.113:0/182826528 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v481) ==== 136+0+0 (secure 0 0 0) 0x7ffb2407a400 con 0x7ffb34071780 2024-04-25T00:08:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.439+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb2006be30 msgr2=0x7ffb2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.439+0000 7ffb3c789640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb2006be30 0x7ffb2006e2f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffb34147250 tx=0x7ffb28008040 comp rx=0 tx=0).stop 2024-04-25T00:08:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.439+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34071780 msgr2=0x7ffb3413e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.439+0000 7ffb3c789640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34071780 0x7ffb3413e720 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098c0 tx=0x7ffb240046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.439+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 shutdown_connections 2024-04-25T00:08:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.439+0000 7ffb3c789640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ffb2006be30 0x7ffb2006e2f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.440+0000 7ffb3c789640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb34072ea0 0x7ffb340b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.440+0000 7ffb3c789640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb3415a6b0 0x7ffb340b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.440+0000 7ffb3c789640 1 --2- 172.21.15.113:0/182826528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34071780 0x7ffb3413e720 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.440+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 >> 172.21.15.113:0/182826528 conn(0x7ffb340b1a50 msgr2=0x7ffb340ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.440+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 shutdown_connections 2024-04-25T00:08:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.440+0000 7ffb3c789640 1 -- 172.21.15.113:0/182826528 wait complete. 2024-04-25T00:08:08.451 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:08:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.550+0000 7ff8b78ee640 1 Processor -- start 2024-04-25T00:08:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.551+0000 7ff8b78ee640 1 -- start start 2024-04-25T00:08:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.551+0000 7ff8b78ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8b00cf3c0 0x7ff8b00d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.551+0000 7ff8b78ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8b0161fc0 0x7ff8b00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.551+0000 7ff8b78ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b0162560 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.551+0000 7ff8b78ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b0162840 con 0x7ff8b0161fc0 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.551+0000 7ff8b78ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b0162b20 con 0x7ff8b00cf3c0 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.552+0000 7ff8b5e8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8b0161fc0 0x7ff8b00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.552+0000 7ff8b5e8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8b0161fc0 0x7ff8b00d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48660/0 (socket says 172.21.15.113:48660) 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.552+0000 7ff8b5e8b640 1 -- 172.21.15.113:0/2266776048 learned_addr learned my addr 172.21.15.113:0/2266776048 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.552+0000 7ff8b5e8b640 1 -- 172.21.15.113:0/2266776048 >> v2:172.21.15.178:3300/0 conn(0x7ff8b00cf3c0 msgr2=0x7ff8b00d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.552+0000 7ff8b5e8b640 1 --2- 172.21.15.113:0/2266776048 >> v2:172.21.15.178:3300/0 conn(0x7ff8b00cf3c0 0x7ff8b00d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.552+0000 7ff8b5e8b640 1 -- 172.21.15.113:0/2266776048 >> v1:172.21.15.178:6789/0 conn(0x7ff8b00cf7a0 legacy=0x7ff8b00d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.552+0000 7ff8b5e8b640 1 -- 172.21.15.113:0/2266776048 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b00d40f0 con 0x7ff8b0161fc0 2024-04-25T00:08:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b5e8b640 1 --2- 172.21.15.113:0/2266776048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b0161fc0 0x7ff8b00d1210 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0161f00 tx=0x7ff8a802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a666bb20ab89ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8a7fff640 1 -- 172.21.15.113:0/2266776048 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8036070 con 0x7ff8b0161fc0 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8a7fff640 1 -- 172.21.15.113:0/2266776048 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a8031600 con 0x7ff8b0161fc0 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8a7fff640 1 -- 172.21.15.113:0/2266776048 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80319d0 con 0x7ff8b0161fc0 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 -- 172.21.15.113:0/2266776048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b0161fc0 msgr2=0x7ff8b00d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/2266776048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b0161fc0 0x7ff8b00d1210 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0161f00 tx=0x7ff8a802a640 comp rx=0 tx=0).stop 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 -- 172.21.15.113:0/2266776048 shutdown_connections 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/2266776048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b0161fc0 0x7ff8b00d1210 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/2266776048 >> v2:172.21.15.178:3300/0 conn(0x7ff8b00cf3c0 0x7ff8b00d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 -- 172.21.15.113:0/2266776048 >> 172.21.15.113:0/2266776048 conn(0x7ff8b00718f0 msgr2=0x7ff8b0071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 -- 172.21.15.113:0/2266776048 shutdown_connections 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.553+0000 7ff8b78ee640 1 -- 172.21.15.113:0/2266776048 wait complete. 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.554+0000 7ff8b78ee640 1 Processor -- start 2024-04-25T00:08:08.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.554+0000 7ff8b78ee640 1 -- start start 2024-04-25T00:08:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b78ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b00cf3c0 0x7ff8b00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b78ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b00cf7a0 0x7ff8b00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b78ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0161fc0 0x7ff8b015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b78ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b00d1750 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b78ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b00d2bf0 con 0x7ff8b00cf3c0 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b78ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b00d18d0 con 0x7ff8b0161fc0 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b4e89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b00cf7a0 0x7ff8b00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b4e89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b00cf7a0 0x7ff8b00ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48668/0 (socket says 172.21.15.113:48668) 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b4e89640 1 -- 172.21.15.113:0/1629820416 learned_addr learned my addr 172.21.15.113:0/1629820416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b4e89640 1 -- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0161fc0 msgr2=0x7ff8b015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b4e89640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0161fc0 0x7ff8b015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b4e89640 1 -- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b00cf3c0 msgr2=0x7ff8b00cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.555+0000 7ff8b568a640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b00cf3c0 0x7ff8b00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8b4e89640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b00cf3c0 0x7ff8b00cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8b4e89640 1 -- 172.21.15.113:0/1629820416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b00b97f0 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8b568a640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b00cf3c0 0x7ff8b00cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8b4e89640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b00cf7a0 0x7ff8b00ca040 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00c410 tx=0x7ff8ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8a5ffb640 1 -- 172.21.15.113:0/1629820416 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac017070 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8a5ffb640 1 -- 172.21.15.113:0/1629820416 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0035f0 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b00cb570 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8a5ffb640 1 -- 172.21.15.113:0/1629820416 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac003a20 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.556+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b00cbb30 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.557+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b00cb390 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.557+0000 7ff8a5ffb640 1 -- 172.21.15.113:0/1629820416 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8ac01a070 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.558+0000 7ff8a5ffb640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff89406be60 0x7ff89406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.558+0000 7ff8a5ffb640 1 -- 172.21.15.113:0/1629820416 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4943+0+0 (secure 0 0 0) 0x7ff8ac089fb0 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.558+0000 7ff8b568a640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff89406be60 0x7ff89406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.558+0000 7ff8b568a640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff89406be60 0x7ff89406e320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0006fd0 tx=0x7ff8a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.562+0000 7ff8a5ffb640 1 -- 172.21.15.113:0/1629820416 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8b00cb390 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.734+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff8b0071cd0 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.737+0000 7ff8a5ffb640 1 -- 172.21.15.113:0/1629820416 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7ff8ac058cf0 con 0x7ff8b00cf7a0 2024-04-25T00:08:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff89406be60 msgr2=0x7ff89406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff89406be60 0x7ff89406e320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0006fd0 tx=0x7ff8a0008040 comp rx=0 tx=0).stop 2024-04-25T00:08:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b00cf7a0 msgr2=0x7ff8b00ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b00cf7a0 0x7ff8b00ca040 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00c410 tx=0x7ff8ac00c8d0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 shutdown_connections 2024-04-25T00:08:08.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff89406be60 0x7ff89406e320 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0161fc0 0x7ff8b015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8b00cf7a0 0x7ff8b00ca040 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 --2- 172.21.15.113:0/1629820416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b00cf3c0 0x7ff8b00cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 >> 172.21.15.113:0/1629820416 conn(0x7ff8b00718f0 msgr2=0x7ff8b015c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 shutdown_connections 2024-04-25T00:08:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:08.739+0000 7ff8b78ee640 1 -- 172.21.15.113:0/1629820416 wait complete. 2024-04-25T00:08:08.750 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:08:08.751 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:08:08.799 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:08:08.846 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:08:08.847 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:08:08.847 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:08.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:08:08.863 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:08:08.863 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:08.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:08:08.881 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:08:08.882 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:08.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:08:08.918 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:08:08.918 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:08.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:08:08.939 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:08:08.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:09.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.054+0000 7f8860438640 1 Processor -- start 2024-04-25T00:08:09.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.054+0000 7f8860438640 1 -- start start 2024-04-25T00:08:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f8860438640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f885815e240 0x7f88581627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f8860438640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f885806e6e0 0x7f88580b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f8860438640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8858162d20 con 0x7f88580b5620 2024-04-25T00:08:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f8860438640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8858072d90 con 0x7f885806e6e0 2024-04-25T00:08:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f8860438640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f88580b3c20 con 0x7f885815e240 2024-04-25T00:08:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f885e9d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f885806e6e0 0x7f88580b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f885d9d3640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f88580b5620 0x7f885806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58456/0 (socket says 172.21.15.113:58456) 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.055+0000 7f885d9d3640 1 -- 172.21.15.113:0/4145602213 learned_addr learned my addr 172.21.15.113:0/4145602213 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885e9d5640 1 -- 172.21.15.113:0/4145602213 >> v2:172.21.15.178:3300/0 conn(0x7f885815e240 msgr2=0x7f88581627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885e1d4640 1 --2- 172.21.15.113:0/4145602213 >> v2:172.21.15.178:3300/0 conn(0x7f885815e240 0x7f88581627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885e9d5640 1 --2- 172.21.15.113:0/4145602213 >> v2:172.21.15.178:3300/0 conn(0x7f885815e240 0x7f88581627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885d1d2640 1 -- 172.21.15.113:0/4145602213 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2732217380 0 0) 0x7f8858162d20 con 0x7f88580b5620 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885e9d5640 1 -- 172.21.15.113:0/4145602213 >> v1:172.21.15.113:6789/0 conn(0x7f88580b5620 legacy=0x7f885806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885e9d5640 1 -- 172.21.15.113:0/4145602213 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885815e620 con 0x7f885806e6e0 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885e1d4640 1 --2- 172.21.15.113:0/4145602213 >> v2:172.21.15.178:3300/0 conn(0x7f885815e240 0x7f88581627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885e9d5640 1 --2- 172.21.15.113:0/4145602213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885806e6e0 0x7f88580b4120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f88580628a0 tx=0x7f885002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756fc334a3340b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885d1d2640 1 -- 172.21.15.113:0/4145602213 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850037070 con 0x7f885806e6e0 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885d1d2640 1 -- 172.21.15.113:0/4145602213 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8850032440 con 0x7f885806e6e0 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.056+0000 7f885d1d2640 1 -- 172.21.15.113:0/4145602213 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850032810 con 0x7f885806e6e0 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 -- 172.21.15.113:0/4145602213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885806e6e0 msgr2=0x7f88580b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 --2- 172.21.15.113:0/4145602213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885806e6e0 0x7f88580b4120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f88580628a0 tx=0x7f885002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 -- 172.21.15.113:0/4145602213 shutdown_connections 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 --2- 172.21.15.113:0/4145602213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885806e6e0 0x7f88580b4120 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 --2- 172.21.15.113:0/4145602213 >> v2:172.21.15.178:3300/0 conn(0x7f885815e240 0x7f88581627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 -- 172.21.15.113:0/4145602213 >> 172.21.15.113:0/4145602213 conn(0x7f88580b1a50 msgr2=0x7f88580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 -- 172.21.15.113:0/4145602213 shutdown_connections 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.057+0000 7f8860438640 1 -- 172.21.15.113:0/4145602213 wait complete. 2024-04-25T00:08:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.058+0000 7f8860438640 1 Processor -- start 2024-04-25T00:08:09.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.058+0000 7f8860438640 1 -- start start 2024-04-25T00:08:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f8860438640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f885806e6e0 0x7f88581f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f8860438640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88580b5620 0x7f8858156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f8860438640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885815e240 0x7f885814a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f8860438640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88580727d0 con 0x7f885815e240 2024-04-25T00:08:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f8860438640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88580724f0 con 0x7f88580b5620 2024-04-25T00:08:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f8860438640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8858072ab0 con 0x7f885806e6e0 2024-04-25T00:08:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885815e240 0x7f885814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885815e240 0x7f885814a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48690/0 (socket says 172.21.15.113:48690) 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 -- 172.21.15.113:0/3916766392 learned_addr learned my addr 172.21.15.113:0/3916766392 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885d9d3640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88580b5620 0x7f8858156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 -- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f885806e6e0 msgr2=0x7f88581f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e1d4640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f885806e6e0 0x7f88581f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f885806e6e0 0x7f88581f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 -- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88580b5620 msgr2=0x7f8858156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88580b5620 0x7f8858156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.059+0000 7f885e9d5640 1 -- 172.21.15.113:0/3916766392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8858156e60 con 0x7f885815e240 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.060+0000 7f885e9d5640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885815e240 0x7f885814a770 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f885002a1e0 tx=0x7f88500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.060+0000 7f884effd640 1 -- 172.21.15.113:0/3916766392 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850041020 con 0x7f885815e240 2024-04-25T00:08:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.060+0000 7f884effd640 1 -- 172.21.15.113:0/3916766392 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88500306e0 con 0x7f885815e240 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.060+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885815eaa0 con 0x7f885815e240 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.060+0000 7f884effd640 1 -- 172.21.15.113:0/3916766392 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850037040 con 0x7f885815e240 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.060+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885815f060 con 0x7f885815e240 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.061+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88580691a0 con 0x7f885815e240 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.061+0000 7f884effd640 1 -- 172.21.15.113:0/3916766392 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8850044020 con 0x7f885815e240 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.061+0000 7f884effd640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f883c06be30 0x7f883c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.062+0000 7f884effd640 1 -- 172.21.15.113:0/3916766392 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4943+0+0 (secure 0 0 0) 0x7f88500ab440 con 0x7f885815e240 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.062+0000 7f885e1d4640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f883c06be30 0x7f883c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.062+0000 7f885e1d4640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f883c06be30 0x7f883c06e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8854008ee0 tx=0x7f8854008430 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:09.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.066+0000 7f884effd640 1 -- 172.21.15.113:0/3916766392 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f885004b030 con 0x7f885815e240 2024-04-25T00:08:09.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.277+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8858161b40 con 0x7f885815e240 2024-04-25T00:08:09.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.277+0000 7f884effd640 1 -- 172.21.15.113:0/3916766392 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+3878 (secure 0 0 0) 0x7f885007a180 con 0x7f885815e240 2024-04-25T00:08:09.278 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:09.278 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1199283489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1199283489},{"type":"v1","addr":"172.21.15.178:6834","nonce":1199283489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4259795043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4259795043},{"type":"v1","addr":"172.21.15.178:6835","nonce":4259795043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9495,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4146002756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4146002756},{"type":"v1","addr":"172.21.15.113:6835","nonce":4146002756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":9897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/230934856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":230934856},{"type":"v1","addr":"172.21.15.113:6837","nonce":230934856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[]} 2024-04-25T00:08:09.279 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 80 2024-04-25T00:08:09.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.279+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f883c06be30 msgr2=0x7f883c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.279+0000 7f8860438640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f883c06be30 0x7f883c06e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8854008ee0 tx=0x7f8854008430 comp rx=0 tx=0).stop 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.279+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885815e240 msgr2=0x7f885814a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.279+0000 7f8860438640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885815e240 0x7f885814a770 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f885002a1e0 tx=0x7f88500046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 shutdown_connections 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f883c06be30 0x7f883c06e2f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88580b5620 0x7f8858156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f885815e240 0x7f885814a770 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 --2- 172.21.15.113:0/3916766392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f885806e6e0 0x7f88581f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 >> 172.21.15.113:0/3916766392 conn(0x7f88580b1a50 msgr2=0x7f88580ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 shutdown_connections 2024-04-25T00:08:09.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.280+0000 7f8860438640 1 -- 172.21.15.113:0/3916766392 wait complete. 2024-04-25T00:08:09.291 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:08:09.291 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:08:09.292 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:08:09.292 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:08:09.292 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:08:09.292 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:08:09.292 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:08:09.292 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:08:09.293 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:08:09.293 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:08:09.293 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:08:09.293 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:08:09.293 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:08:09.293 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:08:09.293 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:08:09.293 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:08:09.301 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:08:09.301 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:08:09.302 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:08:09.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:08:09.308 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:08:09.309 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:08:09.310 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:08:09.310 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:08:09.343 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:08:09.344 DEBUG:teuthology.parallel:result is None 2024-04-25T00:08:09.344 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:08:09.344 DEBUG:teuthology.parallel:result is None 2024-04-25T00:08:09.352 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:08:09.352 DEBUG:teuthology.parallel:result is None 2024-04-25T00:08:09.352 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:08:09.352 DEBUG:teuthology.parallel:result is None 2024-04-25T00:08:09.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:08:09.417 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:08:09.417 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:08:09.427 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:08:09.428 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:08:09.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.459+0000 7f8161cee640 1 Processor -- start 2024-04-25T00:08:09.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f8161cee640 1 -- start start 2024-04-25T00:08:09.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f8161cee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f815c15e510 0x7f815c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f8161cee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f815c0d28c0 0x7f815c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f8161cee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815c0d2e00 con 0x7f815c0b61e0 2024-04-25T00:08:09.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f8161cee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f815c0b6610 con 0x7f815c0d28c0 2024-04-25T00:08:09.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f8161cee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f815c0d30e0 con 0x7f815c15e510 2024-04-25T00:08:09.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f815bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f815c0b61e0 0x7f815c071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58472/0 (socket says 172.21.15.113:58472) 2024-04-25T00:08:09.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.460+0000 7f815bfff640 1 -- 172.21.15.113:0/1630586871 learned_addr learned my addr 172.21.15.113:0/1630586871 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815affd640 1 --2- 172.21.15.113:0/1630586871 >> v2:172.21.15.113:3300/0 conn(0x7f815c0d28c0 0x7f815c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815b7fe640 1 --2- 172.21.15.113:0/1630586871 >> v2:172.21.15.178:3300/0 conn(0x7f815c15e510 0x7f815c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815a7fc640 1 -- 172.21.15.113:0/1630586871 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829249896 0 0) 0x7f815c0d2e00 con 0x7f815c0b61e0 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815a7fc640 1 -- 172.21.15.113:0/1630586871 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8144003520 con 0x7f815c0b61e0 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815affd640 1 -- 172.21.15.113:0/1630586871 >> v2:172.21.15.178:3300/0 conn(0x7f815c15e510 msgr2=0x7f815c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815affd640 1 --2- 172.21.15.113:0/1630586871 >> v2:172.21.15.178:3300/0 conn(0x7f815c15e510 0x7f815c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815affd640 1 -- 172.21.15.113:0/1630586871 >> v1:172.21.15.113:6789/0 conn(0x7f815c0b61e0 legacy=0x7f815c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815a7fc640 1 -- 172.21.15.113:0/1630586871 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1801693389 0 0) 0x7f8144003520 con 0x7f815c0b61e0 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.461+0000 7f815affd640 1 -- 172.21.15.113:0/1630586871 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c0d33c0 con 0x7f815c0d28c0 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.462+0000 7f815affd640 1 --2- 172.21.15.113:0/1630586871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0b5ca0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f815c0d52a0 tx=0x7f815002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed70b04fcccfacd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:09.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.462+0000 7f815a7fc640 1 -- 172.21.15.113:0/1630586871 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150037070 con 0x7f815c0d28c0 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.462+0000 7f815a7fc640 1 -- 172.21.15.113:0/1630586871 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815002ab30 con 0x7f815c0d28c0 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.462+0000 7f815a7fc640 1 -- 172.21.15.113:0/1630586871 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540019f0 con 0x7f815c0d28c0 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 -- 172.21.15.113:0/1630586871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 msgr2=0x7f815c0b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 --2- 172.21.15.113:0/1630586871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0b5ca0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f815c0d52a0 tx=0x7f815002a0a0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 -- 172.21.15.113:0/1630586871 shutdown_connections 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 --2- 172.21.15.113:0/1630586871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0b5ca0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 --2- 172.21.15.113:0/1630586871 >> v2:172.21.15.178:3300/0 conn(0x7f815c15e510 0x7f815c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 -- 172.21.15.113:0/1630586871 >> 172.21.15.113:0/1630586871 conn(0x7f815c06b720 msgr2=0x7f815c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 -- 172.21.15.113:0/1630586871 shutdown_connections 2024-04-25T00:08:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.463+0000 7f8161cee640 1 -- 172.21.15.113:0/1630586871 wait complete. 2024-04-25T00:08:09.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.464+0000 7f8161cee640 1 Processor -- start 2024-04-25T00:08:09.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.464+0000 7f8161cee640 1 -- start start 2024-04-25T00:08:09.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f8161cee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0b61e0 0x7f815c14f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f8161cee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f8161cee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c15e510 0x7f815c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f8161cee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0d3ea0 con 0x7f815c0d28c0 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f8161cee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0d3980 con 0x7f815c15e510 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f8161cee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f815c0b9dd0 con 0x7f815c0b61e0 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48738/0 (socket says 172.21.15.113:48738) 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 -- 172.21.15.113:0/3784145621 learned_addr learned my addr 172.21.15.113:0/3784145621 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815bfff640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c15e510 0x7f815c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815b7fe640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0b61e0 0x7f815c14f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 -- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0b61e0 msgr2=0x7f815c14f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0b61e0 0x7f815c14f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 -- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c15e510 msgr2=0x7f815c0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c15e510 0x7f815c0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:09.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815affd640 1 -- 172.21.15.113:0/3784145621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c1614d0 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815bfff640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c15e510 0x7f815c0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.465+0000 7f815b7fe640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0b61e0 0x7f815c14f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.466+0000 7f815affd640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0c8760 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f8150009890 tx=0x7f81500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.466+0000 7f8143fff640 1 -- 172.21.15.113:0/3784145621 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150037050 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.466+0000 7f8143fff640 1 -- 172.21.15.113:0/3784145621 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150038040 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.466+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c0c7e80 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.466+0000 7f8143fff640 1 -- 172.21.15.113:0/3784145621 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150004470 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.466+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c0c8ca0 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.467+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c0691a0 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.467+0000 7f8143fff640 1 -- 172.21.15.113:0/3784145621 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8150003610 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.468+0000 7f8143fff640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f814806be60 0x7f814806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.468+0000 7f8143fff640 1 -- 172.21.15.113:0/3784145621 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 4943+0+0 (secure 0 0 0) 0x7f81500ab1f0 con 0x7f815c0d28c0 2024-04-25T00:08:09.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.468+0000 7f815b7fe640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f814806be60 0x7f814806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:09.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.468+0000 7f815b7fe640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f814806be60 0x7f814806e320 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f814c009af0 tx=0x7f814c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:09.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.472+0000 7f8143fff640 1 -- 172.21.15.113:0/3784145621 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8150079f30 con 0x7f815c0d28c0 2024-04-25T00:08:09.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:09.668+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f815c15b060 con 0x7f815c0d28c0 2024-04-25T00:08:10.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.667+0000 7f8143fff640 1 -- 172.21.15.113:0/3784145621 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v154) ==== 97+0+0 (secure 0 0 0) 0x7f8150079830 con 0x7f815c0d28c0 2024-04-25T00:08:10.668 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:08:10.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f814806be60 msgr2=0x7f814806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f814806be60 0x7f814806e320 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f814c009af0 tx=0x7f814c008040 comp rx=0 tx=0).stop 2024-04-25T00:08:10.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 msgr2=0x7f815c0c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0c8760 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f8150009890 tx=0x7f81500046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 shutdown_connections 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f814806be60 0x7f814806e320 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c15e510 0x7f815c0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815c0d28c0 0x7f815c0c8760 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 --2- 172.21.15.113:0/3784145621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0b61e0 0x7f815c14f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 >> 172.21.15.113:0/3784145621 conn(0x7f815c06b720 msgr2=0x7f815c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 shutdown_connections 2024-04-25T00:08:10.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.670+0000 7f8161cee640 1 -- 172.21.15.113:0/3784145621 wait complete. 2024-04-25T00:08:10.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:08:10.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea84d7e640 1 Processor -- start 2024-04-25T00:08:10.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea84d7e640 1 -- start start 2024-04-25T00:08:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea84d7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 0x7fea800b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea84d7e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea800d2800 con 0x7fea8015e320 2024-04-25T00:08:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea84d7e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea800d2ae0 con 0x7fea800d0b00 2024-04-25T00:08:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea84d7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea800d3410 con 0x7fea800d2290 2024-04-25T00:08:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea7e59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fea8015e320 0x7fea800d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58484/0 (socket says 172.21.15.113:58484) 2024-04-25T00:08:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.782+0000 7fea7e59c640 1 -- 172.21.15.113:0/775582150 learned_addr learned my addr 172.21.15.113:0/775582150 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701203609 0 0) 0x7fea800d2800 con 0x7fea8015e320 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea6c003520 con 0x7fea8015e320 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea77fff640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 0x7fea800b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 397615088 0 0) 0x7fea6c003520 con 0x7fea8015e320 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 msgr2=0x7fea800b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 0x7fea800b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 >> v1:172.21.15.178:6789/0 conn(0x7fea800d0b00 legacy=0x7fea800b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea800d2dc0 con 0x7fea8015e320 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fea70001670 con 0x7fea8015e320 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 >> v1:172.21.15.113:6789/0 conn(0x7fea8015e320 legacy=0x7fea800d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea6c0039e0 0x7fea6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 0x7fea6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea800d3410 con 0x7fea6c004800 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c00ba70 con 0x7fea6c0039e0 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c00bc70 con 0x7fea800d2290 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea77fff640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7e59c640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea6c0039e0 0x7fea6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7ed9d640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 0x7fea6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea77fff640 1 -- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 msgr2=0x7fea6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea77fff640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 0x7fea6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea77fff640 1 -- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea6c0039e0 msgr2=0x7fea6c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea77fff640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea6c0039e0 0x7fea6c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea77fff640 1 -- 172.21.15.113:0/775582150 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c003520 con 0x7fea6c004800 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.783+0000 7fea7ed9d640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800d2290 0x7fea6c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.784+0000 7fea77fff640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea6c004c80 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fea68009ac0 tx=0x7fea6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192b41e9693be577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.784+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68036070 con 0x7fea6c004800 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.784+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea68031400 con 0x7fea6c004800 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.784+0000 7fea7dd9b640 1 -- 172.21.15.113:0/775582150 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68009690 con 0x7fea6c004800 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.785+0000 7fea84d7e640 1 -- 172.21.15.113:0/775582150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 msgr2=0x7fea6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.785+0000 7fea84d7e640 1 --2- 172.21.15.113:0/775582150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea6c004c80 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fea68009ac0 tx=0x7fea6c005830 comp rx=0 tx=0).stop 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.785+0000 7fea7ed9d640 1 -- 172.21.15.113:0/775582150 reap_dead start 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.785+0000 7fea84d7e640 1 -- 172.21.15.113:0/775582150 shutdown_connections 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.785+0000 7fea84d7e640 1 -- 172.21.15.113:0/775582150 >> 172.21.15.113:0/775582150 conn(0x7fea800b5c20 msgr2=0x7fea800b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.785+0000 7fea84d7e640 1 -- 172.21.15.113:0/775582150 shutdown_connections 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.785+0000 7fea84d7e640 1 -- 172.21.15.113:0/775582150 wait complete. 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.786+0000 7fea84d7e640 1 Processor -- start 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.786+0000 7fea84d7e640 1 -- start start 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea8015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea800c8f20 0x7fea800b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800b8860 0x7fea80160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea8015e880 con 0x7fea6c004800 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea8015e700 con 0x7fea800c8f20 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea800d0f90 con 0x7fea800b8860 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea8015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea8015c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48772/0 (socket says 172.21.15.113:48772) 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 -- 172.21.15.113:0/3781701699 learned_addr learned my addr 172.21.15.113:0/3781701699 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea77fff640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea800c8f20 0x7fea800b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7ed9d640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800b8860 0x7fea80160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 -- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800b8860 msgr2=0x7fea80160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800b8860 0x7fea80160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 -- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea800c8f20 msgr2=0x7fea800b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea800c8f20 0x7fea800b8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 -- 172.21.15.113:0/3781701699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea800c7be0 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea77fff640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea800c8f20 0x7fea800b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea7e59c640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea8015c680 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fea70007320 tx=0x7fea700077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea76ffd640 1 -- 172.21.15.113:0/3781701699 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7001a070 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea76ffd640 1 -- 172.21.15.113:0/3781701699 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7000f9c0 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea800c7e80 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea76ffd640 1 -- 172.21.15.113:0/3781701699 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7000fd90 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.787+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea800c8440 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.788+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea800691a0 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.789+0000 7fea76ffd640 1 -- 172.21.15.113:0/3781701699 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea7001d070 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.789+0000 7fea76ffd640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea6006be30 0x7fea6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.789+0000 7fea76ffd640 1 -- 172.21.15.113:0/3781701699 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4857+0+0 (secure 0 0 0) 0x7fea7008a560 con 0x7fea6c004800 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.789+0000 7fea77fff640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea6006be30 0x7fea6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.790+0000 7fea77fff640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea6006be30 0x7fea6006e2f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fea68033000 tx=0x7fea68008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:10.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:10.794+0000 7fea76ffd640 1 -- 172.21.15.113:0/3781701699 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fea70059260 con 0x7fea6c004800 2024-04-25T00:08:11.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.033+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fea800b4620 con 0x7fea6c004800 2024-04-25T00:08:11.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.033+0000 7fea76ffd640 1 -- 172.21.15.113:0/3781701699 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7fea70058b60 con 0x7fea6c004800 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.034 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.036 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.037 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:08:11.038 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:08:11.039 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:08:11.040 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.040 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.040 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.040 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.041 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.042 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.043 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.044 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.046 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:08:11.047 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:08:11.048 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:08:11.050 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea6006be30 msgr2=0x7fea6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea6006be30 0x7fea6006e2f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fea68033000 tx=0x7fea68008030 comp rx=0 tx=0).stop 2024-04-25T00:08:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 msgr2=0x7fea8015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea8015c680 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fea70007320 tx=0x7fea700077e0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 shutdown_connections 2024-04-25T00:08:11.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fea6006be30 0x7fea6006e2f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea800c8f20 0x7fea800b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea800b8860 0x7fea80160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 --2- 172.21.15.113:0/3781701699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea6c004800 0x7fea8015c680 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 >> 172.21.15.113:0/3781701699 conn(0x7fea800b5c20 msgr2=0x7fea800d4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 shutdown_connections 2024-04-25T00:08:11.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.035+0000 7fea84d7e640 1 -- 172.21.15.113:0/3781701699 wait complete. 2024-04-25T00:08:11.053 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.150+0000 7f735784c640 1 Processor -- start 2024-04-25T00:08:11.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.150+0000 7f735784c640 1 -- start start 2024-04-25T00:08:11.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.150+0000 7f735784c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f73501453c0 0x7f73500b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.150+0000 7f735784c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73500cfad0 con 0x7f735006e6e0 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f735784c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73500cf230 con 0x7f7350144d60 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f735784c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7350145c40 con 0x7f73501453c0 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f7354de7640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f735006e6e0 0x7f7350071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58488/0 (socket says 172.21.15.113:58488) 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f7354de7640 1 -- 172.21.15.113:0/456803090 learned_addr learned my addr 172.21.15.113:0/456803090 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086030704 0 0) 0x7f73500cfad0 con 0x7f735006e6e0 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f7355de9640 1 --2- 172.21.15.113:0/456803090 >> v2:172.21.15.178:3300/0 conn(0x7f73501453c0 0x7f73500b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7338003520 con 0x7f735006e6e0 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1331140255 0 0) 0x7f7338003520 con 0x7f735006e6e0 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 >> v2:172.21.15.178:3300/0 conn(0x7f73501453c0 msgr2=0x7f73500b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 --2- 172.21.15.113:0/456803090 >> v2:172.21.15.178:3300/0 conn(0x7f73501453c0 0x7f73500b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 >> v1:172.21.15.178:6789/0 conn(0x7f7350144d60 legacy=0x7f735006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500cf510 con 0x7f735006e6e0 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f7344001670 con 0x7f735006e6e0 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 >> v1:172.21.15.113:6789/0 conn(0x7f735006e6e0 legacy=0x7f7350071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73380039b0 0x7f7338003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7338004800 0x7f7338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73380052e0 0x7f733800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7350145c40 con 0x7f7338004800 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f733800c1b0 con 0x7f73380052e0 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f733800c3b0 con 0x7f73380039b0 2024-04-25T00:08:11.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f7354de7640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7338004800 0x7f7338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f7355de9640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73380052e0 0x7f733800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.151+0000 7f73555e8640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73380039b0 0x7f7338003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7354de7640 1 -- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73380039b0 msgr2=0x7f7338003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7354de7640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73380039b0 0x7f7338003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7354de7640 1 -- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73380052e0 msgr2=0x7f733800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7354de7640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73380052e0 0x7f733800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7354de7640 1 -- 172.21.15.113:0/456803090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7338003520 con 0x7f7338004800 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7355de9640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73380052e0 0x7f733800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f73555e8640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73380039b0 0x7f7338003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7355de9640 1 -- 172.21.15.113:0/456803090 reap_dead start 2024-04-25T00:08:11.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f7354de7640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7338004800 0x7f7338004c80 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f73500b1a00 tx=0x7f7338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f343c5d2ffd5f3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344007d10 con 0x7f7338004800 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7344005ad0 con 0x7f7338004800 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.152+0000 7f734ffff640 1 -- 172.21.15.113:0/456803090 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344007070 con 0x7f7338004800 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.153+0000 7f735784c640 1 -- 172.21.15.113:0/456803090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7338004800 msgr2=0x7f7338004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.153+0000 7f735784c640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7338004800 0x7f7338004c80 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f73500b1a00 tx=0x7f7338005830 comp rx=0 tx=0).stop 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.153+0000 7f735784c640 1 -- 172.21.15.113:0/456803090 shutdown_connections 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.153+0000 7f735784c640 1 --2- 172.21.15.113:0/456803090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7338004800 0x7f7338004c80 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.153+0000 7f735784c640 1 -- 172.21.15.113:0/456803090 >> 172.21.15.113:0/456803090 conn(0x7f73500b1a50 msgr2=0x7f73500b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.153+0000 7f735784c640 1 -- 172.21.15.113:0/456803090 shutdown_connections 2024-04-25T00:08:11.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.153+0000 7f735784c640 1 -- 172.21.15.113:0/456803090 wait complete. 2024-04-25T00:08:11.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.154+0000 7f735784c640 1 Processor -- start 2024-04-25T00:08:11.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.154+0000 7f735784c640 1 -- start start 2024-04-25T00:08:11.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f735784c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338004800 0x7f735015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f735784c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735015adc0 0x7f73500ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f735784c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73500a9480 0x7f73500adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f735784c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7350145140 con 0x7f73500a9480 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f735784c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7350070e10 con 0x7f7338004800 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f735784c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73500b5390 con 0x7f735015adc0 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73500a9480 0x7f73500adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73500a9480 0x7f73500adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48784/0 (socket says 172.21.15.113:48784) 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 -- 172.21.15.113:0/3147147659 learned_addr learned my addr 172.21.15.113:0/3147147659 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 -- 172.21.15.113:0/3147147659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735015adc0 msgr2=0x7f73500ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735015adc0 0x7f73500ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 -- 172.21.15.113:0/3147147659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338004800 msgr2=0x7f735015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338004800 0x7f735015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 -- 172.21.15.113:0/3147147659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500aba30 con 0x7f73500a9480 2024-04-25T00:08:11.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.155+0000 7f7355de9640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73500a9480 0x7f73500adf40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f73500ce060 tx=0x7f7340011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.156+0000 7f734dffb640 1 -- 172.21.15.113:0/3147147659 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340019020 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.156+0000 7f734dffb640 1 -- 172.21.15.113:0/3147147659 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7340003610 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.156+0000 7f734dffb640 1 -- 172.21.15.113:0/3147147659 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340003a00 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.156+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73500ad460 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.156+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73500ad110 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.157+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73500691a0 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.157+0000 7f734dffb640 1 -- 172.21.15.113:0/3147147659 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f734001c020 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.158+0000 7f734dffb640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f733c06be30 0x7f733c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.158+0000 7f73555e8640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f733c06be30 0x7f733c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.158+0000 7f734dffb640 1 -- 172.21.15.113:0/3147147659 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4857+0+0 (secure 0 0 0) 0x7f734008adb0 con 0x7f73500a9480 2024-04-25T00:08:11.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.158+0000 7f73555e8640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f733c06be30 0x7f733c06e2f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7348007fd0 tx=0x7f7348009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.162+0000 7f734dffb640 1 -- 172.21.15.113:0/3147147659 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7340023030 con 0x7f73500a9480 2024-04-25T00:08:11.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.337+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73500b27e0 con 0x7f73500a9480 2024-04-25T00:08:11.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.338+0000 7f734dffb640 1 -- 172.21.15.113:0/3147147659 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+12949 (secure 0 0 0) 0x7f7340059b40 con 0x7f73500a9480 2024-04-25T00:08:11.339 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:11.340 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":154,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:10.665130+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:11.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f733c06be30 msgr2=0x7f733c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f733c06be30 0x7f733c06e2f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7348007fd0 tx=0x7f7348009040 comp rx=0 tx=0).stop 2024-04-25T00:08:11.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73500a9480 msgr2=0x7f73500adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73500a9480 0x7f73500adf40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f73500ce060 tx=0x7f7340011920 comp rx=0 tx=0).stop 2024-04-25T00:08:11.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 shutdown_connections 2024-04-25T00:08:11.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f733c06be30 0x7f733c06e2f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73500a9480 0x7f73500adf40 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735015adc0 0x7f73500ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 --2- 172.21.15.113:0/3147147659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338004800 0x7f735015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 >> 172.21.15.113:0/3147147659 conn(0x7f73500b1a50 msgr2=0x7f735006df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 shutdown_connections 2024-04-25T00:08:11.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.340+0000 7f735784c640 1 -- 172.21.15.113:0/3147147659 wait complete. 2024-04-25T00:08:11.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:08:11.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:08:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92aeca7640 1 Processor -- start 2024-04-25T00:08:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92aeca7640 1 -- start start 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92aeca7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a815e510 0x7f92a80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92aeca7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92a80b63b0 0x7f92a80b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92aeca7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a80d2e00 con 0x7f92a80d28c0 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92aeca7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f92a80d30e0 con 0x7f92a80b63b0 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92aeca7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92a80d33c0 con 0x7f92a815e510 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.453+0000 7f92ad244640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92a80b63b0 0x7f92a80b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92ad244640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92a80b63b0 0x7f92a80b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48786/0 (socket says 172.21.15.113:48786) 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92ad244640 1 -- 172.21.15.113:0/599793888 learned_addr learned my addr 172.21.15.113:0/599793888 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:11.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92aca43640 1 --2- 172.21.15.113:0/599793888 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a815e510 0x7f92a80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92ad244640 1 -- 172.21.15.113:0/599793888 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a815e510 msgr2=0x7f92a80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92ad244640 1 --2- 172.21.15.113:0/599793888 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a815e510 0x7f92a80d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92ad244640 1 -- 172.21.15.113:0/599793888 >> v1:172.21.15.178:6789/0 conn(0x7f92a80d28c0 legacy=0x7f92a80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92ad244640 1 -- 172.21.15.113:0/599793888 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a80d36a0 con 0x7f92a80b63b0 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.454+0000 7f92ad244640 1 --2- 172.21.15.113:0/599793888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a80b63b0 0x7f92a80b73c0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009e50 tx=0x7f92a002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956ed8caeafcf081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92a77fe640 1 -- 172.21.15.113:0/599793888 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0036070 con 0x7f92a80b63b0 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92a77fe640 1 -- 172.21.15.113:0/599793888 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a0031600 con 0x7f92a80b63b0 2024-04-25T00:08:11.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92a77fe640 1 -- 172.21.15.113:0/599793888 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a00319d0 con 0x7f92a80b63b0 2024-04-25T00:08:11.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 -- 172.21.15.113:0/599793888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a80b63b0 msgr2=0x7f92a80b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 --2- 172.21.15.113:0/599793888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a80b63b0 0x7f92a80b73c0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009e50 tx=0x7f92a002a630 comp rx=0 tx=0).stop 2024-04-25T00:08:11.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 -- 172.21.15.113:0/599793888 shutdown_connections 2024-04-25T00:08:11.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 --2- 172.21.15.113:0/599793888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a80b63b0 0x7f92a80b73c0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 --2- 172.21.15.113:0/599793888 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a815e510 0x7f92a80d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 -- 172.21.15.113:0/599793888 >> 172.21.15.113:0/599793888 conn(0x7f92a806b720 msgr2=0x7f92a80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 -- 172.21.15.113:0/599793888 shutdown_connections 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.455+0000 7f92aeca7640 1 -- 172.21.15.113:0/599793888 wait complete. 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.456+0000 7f92aeca7640 1 Processor -- start 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.456+0000 7f92aeca7640 1 -- start start 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92aeca7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a80b63b0 0x7f92a80cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92aeca7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a80d28c0 0x7f92a8150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92aeca7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a815e510 0x7f92a80cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92aeca7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92a80b9dd0 con 0x7f92a815e510 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92aeca7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92a80d3ea0 con 0x7f92a80d28c0 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92aeca7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92a80b7900 con 0x7f92a80b63b0 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92ad244640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a815e510 0x7f92a80cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92ad244640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a815e510 0x7f92a80cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48796/0 (socket says 172.21.15.113:48796) 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92ad244640 1 -- 172.21.15.113:0/2118464226 learned_addr learned my addr 172.21.15.113:0/2118464226 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:11.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92aca43640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a80b63b0 0x7f92a80cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.457+0000 7f92a7fff640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a80d28c0 0x7f92a8150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92ad244640 1 -- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a80b63b0 msgr2=0x7f92a80cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92ad244640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a80b63b0 0x7f92a80cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92ad244640 1 -- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a80d28c0 msgr2=0x7f92a8150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92ad244640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a80d28c0 0x7f92a8150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92ad244640 1 -- 172.21.15.113:0/2118464226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a80cc240 con 0x7f92a815e510 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92aca43640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a80b63b0 0x7f92a80cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92ad244640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a815e510 0x7f92a80cd950 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f92a0034040 tx=0x7f92a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92a57fa640 1 -- 172.21.15.113:0/2118464226 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0036050 con 0x7f92a815e510 2024-04-25T00:08:11.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92a57fa640 1 -- 172.21.15.113:0/2118464226 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a0042070 con 0x7f92a815e510 2024-04-25T00:08:11.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a80cc5d0 con 0x7f92a815e510 2024-04-25T00:08:11.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92a57fa640 1 -- 172.21.15.113:0/2118464226 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0009510 con 0x7f92a815e510 2024-04-25T00:08:11.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.458+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a80ccb90 con 0x7f92a815e510 2024-04-25T00:08:11.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.459+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a80691a0 con 0x7f92a815e510 2024-04-25T00:08:11.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.459+0000 7f92a57fa640 1 -- 172.21.15.113:0/2118464226 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92a0007bd0 con 0x7f92a815e510 2024-04-25T00:08:11.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.460+0000 7f92a57fa640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f928c06be60 0x7f928c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.460+0000 7f92a57fa640 1 -- 172.21.15.113:0/2118464226 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4857+0+0 (secure 0 0 0) 0x7f92a00abb00 con 0x7f92a815e510 2024-04-25T00:08:11.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.460+0000 7f92aca43640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f928c06be60 0x7f928c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.460+0000 7f92aca43640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f928c06be60 0x7f928c06e320 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f929c006fd0 tx=0x7f929c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.464+0000 7f92a57fa640 1 -- 172.21.15.113:0/2118464226 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f92a80cc240 con 0x7f92a815e510 2024-04-25T00:08:11.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.636+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92a815b9d0 con 0x7f92a815e510 2024-04-25T00:08:11.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.637+0000 7f92a57fa640 1 -- 172.21.15.113:0/2118464226 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) ==== 93+0+10 (secure 0 0 0) 0x7f92a0045360 con 0x7f92a815e510 2024-04-25T00:08:11.638 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f928c06be60 msgr2=0x7f928c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f928c06be60 0x7f928c06e320 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f929c006fd0 tx=0x7f929c008040 comp rx=0 tx=0).stop 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a815e510 msgr2=0x7f92a80cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a815e510 0x7f92a80cd950 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f92a0034040 tx=0x7f92a0004a40 comp rx=0 tx=0).stop 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 shutdown_connections 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f928c06be60 0x7f928c06e320 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a80d28c0 0x7f92a8150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a815e510 0x7f92a80cd950 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 --2- 172.21.15.113:0/2118464226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92a80b63b0 0x7f92a80cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 >> 172.21.15.113:0/2118464226 conn(0x7f92a806b720 msgr2=0x7f92a80b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 shutdown_connections 2024-04-25T00:08:11.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.639+0000 7f92aeca7640 1 -- 172.21.15.113:0/2118464226 wait complete. 2024-04-25T00:08:11.651 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:08:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.747+0000 7fd049b90640 1 Processor -- start 2024-04-25T00:08:11.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.747+0000 7fd049b90640 1 -- start start 2024-04-25T00:08:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd049b90640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd04415a6b0 0x7fd044070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd049b90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0440b2d50 con 0x7fd044072ea0 2024-04-25T00:08:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd049b90640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd044072a70 con 0x7fd0440b3af0 2024-04-25T00:08:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd049b90640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd0440b3010 con 0x7fd04415a6b0 2024-04-25T00:08:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd042ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd044072ea0 0x7fd044071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58502/0 (socket says 172.21.15.113:58502) 2024-04-25T00:08:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd042ffd640 1 -- 172.21.15.113:0/1406787212 learned_addr learned my addr 172.21.15.113:0/1406787212 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd0437fe640 1 --2- 172.21.15.113:0/1406787212 >> v2:172.21.15.178:3300/0 conn(0x7fd04415a6b0 0x7fd044070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459888573 0 0) 0x7fd0440b2d50 con 0x7fd044072ea0 2024-04-25T00:08:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.748+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd02c003520 con 0x7fd044072ea0 2024-04-25T00:08:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1410229902 0 0) 0x7fd02c003520 con 0x7fd044072ea0 2024-04-25T00:08:11.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 >> v2:172.21.15.178:3300/0 conn(0x7fd04415a6b0 msgr2=0x7fd044070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 --2- 172.21.15.113:0/1406787212 >> v2:172.21.15.178:3300/0 conn(0x7fd04415a6b0 0x7fd044070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 >> v1:172.21.15.178:6789/0 conn(0x7fd0440b3af0 legacy=0x7fd0440b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0440b32f0 con 0x7fd044072ea0 2024-04-25T00:08:11.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fd038001670 con 0x7fd044072ea0 2024-04-25T00:08:11.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 >> v1:172.21.15.113:6789/0 conn(0x7fd044072ea0 legacy=0x7fd044071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd02c0039b0 0x7fd02c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02c004800 0x7fd02c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0051c0 0x7fd02c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0440b3010 con 0x7fd02c0039b0 2024-04-25T00:08:11.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c00c060 con 0x7fd02c004800 2024-04-25T00:08:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd02c00c260 con 0x7fd02c0051c0 2024-04-25T00:08:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0437fe640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd02c0039b0 0x7fd02c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd042ffd640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02c004800 0x7fd02c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd043fff640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0051c0 0x7fd02c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0437fe640 1 -- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0051c0 msgr2=0x7fd02c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0437fe640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0051c0 0x7fd02c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0437fe640 1 -- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02c004800 msgr2=0x7fd02c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0437fe640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02c004800 0x7fd02c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd0437fe640 1 -- 172.21.15.113:0/1406787212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c003520 con 0x7fd02c0039b0 2024-04-25T00:08:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd042ffd640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02c004800 0x7fd02c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.749+0000 7fd043fff640 1 -- 172.21.15.113:0/1406787212 reap_dead start 2024-04-25T00:08:11.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd0437fe640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd02c0039b0 0x7fd02c003dd0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fd034009650 tx=0x7fd02c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23990c9f1bb3aac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034037070 con 0x7fd02c0039b0 2024-04-25T00:08:11.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd034032400 con 0x7fd02c0039b0 2024-04-25T00:08:11.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd0427fc640 1 -- 172.21.15.113:0/1406787212 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034009690 con 0x7fd02c0039b0 2024-04-25T00:08:11.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd049b90640 1 -- 172.21.15.113:0/1406787212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd02c0039b0 msgr2=0x7fd02c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd049b90640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd02c0039b0 0x7fd02c003dd0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fd034009650 tx=0x7fd02c005830 comp rx=0 tx=0).stop 2024-04-25T00:08:11.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd049b90640 1 -- 172.21.15.113:0/1406787212 shutdown_connections 2024-04-25T00:08:11.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd049b90640 1 --2- 172.21.15.113:0/1406787212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd02c0039b0 0x7fd02c003dd0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd049b90640 1 -- 172.21.15.113:0/1406787212 >> 172.21.15.113:0/1406787212 conn(0x7fd0440b1a50 msgr2=0x7fd0440b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd049b90640 1 -- 172.21.15.113:0/1406787212 shutdown_connections 2024-04-25T00:08:11.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.750+0000 7fd049b90640 1 -- 172.21.15.113:0/1406787212 wait complete. 2024-04-25T00:08:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 Processor -- start 2024-04-25T00:08:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 -- start start 2024-04-25T00:08:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0039b0 0x7fd044147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd044154da0 0x7fd0441534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044157d10 0x7fd0441559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd04415aa90 con 0x7fd044157d10 2024-04-25T00:08:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd044072720 con 0x7fd044154da0 2024-04-25T00:08:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.752+0000 7fd049b90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd04415ac10 con 0x7fd02c0039b0 2024-04-25T00:08:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044157d10 0x7fd0441559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044157d10 0x7fd0441559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48820/0 (socket says 172.21.15.113:48820) 2024-04-25T00:08:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 -- 172.21.15.113:0/2205593044 learned_addr learned my addr 172.21.15.113:0/2205593044 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd0437fe640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0039b0 0x7fd044147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd042ffd640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd044154da0 0x7fd0441534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 -- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0039b0 msgr2=0x7fd044147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0039b0 0x7fd044147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 -- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd044154da0 msgr2=0x7fd0441534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd044154da0 0x7fd0441534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 -- 172.21.15.113:0/2205593044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd044153d00 con 0x7fd044157d10 2024-04-25T00:08:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd042ffd640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd044154da0 0x7fd0441534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd043fff640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044157d10 0x7fd0441559a0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fd0440b5d40 tx=0x7fd03c00e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd02bfff640 1 -- 172.21.15.113:0/2205593044 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c003b80 con 0x7fd044157d10 2024-04-25T00:08:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd02bfff640 1 -- 172.21.15.113:0/2205593044 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03c018070 con 0x7fd044157d10 2024-04-25T00:08:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd044153fe0 con 0x7fd044157d10 2024-04-25T00:08:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd02bfff640 1 -- 172.21.15.113:0/2205593044 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0050d0 con 0x7fd044157d10 2024-04-25T00:08:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.753+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0441545a0 con 0x7fd044157d10 2024-04-25T00:08:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.754+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0440691a0 con 0x7fd044157d10 2024-04-25T00:08:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.755+0000 7fd02bfff640 1 -- 172.21.15.113:0/2205593044 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd03c005270 con 0x7fd044157d10 2024-04-25T00:08:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.755+0000 7fd02bfff640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03006be30 0x7fd03006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.755+0000 7fd02bfff640 1 -- 172.21.15.113:0/2205593044 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4857+0+0 (secure 0 0 0) 0x7fd03c08a630 con 0x7fd044157d10 2024-04-25T00:08:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.755+0000 7fd0437fe640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03006be30 0x7fd03006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.756+0000 7fd0437fe640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03006be30 0x7fd03006e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd034034000 tx=0x7fd034010040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.759+0000 7fd02bfff640 1 -- 172.21.15.113:0/2205593044 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd03c059330 con 0x7fd044157d10 2024-04-25T00:08:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.933+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd0440b2d50 con 0x7fd044157d10 2024-04-25T00:08:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.933+0000 7fd02bfff640 1 -- 172.21.15.113:0/2205593044 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v154) ==== 84+0+65 (secure 0 0 0) 0x7fd03c058c30 con 0x7fd044157d10 2024-04-25T00:08:11.934 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:11.934 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:08:11.934 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:11.934 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:08:11.934 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:08:11.934 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:11.934 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03006be30 msgr2=0x7fd03006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03006be30 0x7fd03006e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd034034000 tx=0x7fd034010040 comp rx=0 tx=0).stop 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044157d10 msgr2=0x7fd0441559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044157d10 0x7fd0441559a0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fd0440b5d40 tx=0x7fd03c00e9e0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 shutdown_connections 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd03006be30 0x7fd03006e2f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd044154da0 0x7fd0441534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044157d10 0x7fd0441559a0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 --2- 172.21.15.113:0/2205593044 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd02c0039b0 0x7fd044147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 >> 172.21.15.113:0/2205593044 conn(0x7fd0440b1a50 msgr2=0x7fd04414f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 shutdown_connections 2024-04-25T00:08:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:11.935+0000 7fd049b90640 1 -- 172.21.15.113:0/2205593044 wait complete. 2024-04-25T00:08:11.946 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:08:11.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:08:12.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.047+0000 7f54a8512640 1 Processor -- start 2024-04-25T00:08:12.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.047+0000 7f54a8512640 1 -- start start 2024-04-25T00:08:12.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a8512640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54a0145890 0x7f54a00ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a8512640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a00ced20 0x7f54a006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a8512640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a00cf3a0 con 0x7f54a006e610 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a8512640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f54a00cf180 con 0x7f54a0145890 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a8512640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f54a0145c70 con 0x7f54a00ced20 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a62ae640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54a0145890 0x7f54a00ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a62ae640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54a0145890 0x7f54a00ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48824/0 (socket says 172.21.15.113:48824) 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a62ae640 1 -- 172.21.15.113:0/307177217 learned_addr learned my addr 172.21.15.113:0/307177217 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a5aad640 1 --2- 172.21.15.113:0/307177217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a00ced20 0x7f54a006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a62ae640 1 -- 172.21.15.113:0/307177217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a00ced20 msgr2=0x7f54a006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a62ae640 1 --2- 172.21.15.113:0/307177217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a00ced20 0x7f54a006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a62ae640 1 -- 172.21.15.113:0/307177217 >> v1:172.21.15.178:6789/0 conn(0x7f54a006e610 legacy=0x7f54a015fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a62ae640 1 -- 172.21.15.113:0/307177217 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a00cf680 con 0x7f54a0145890 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.048+0000 7f54a5aad640 1 --2- 172.21.15.113:0/307177217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a00ced20 0x7f54a006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.049+0000 7f54a62ae640 1 --2- 172.21.15.113:0/307177217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0145890 0x7f54a00ce7e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b5c1e860c7d0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.049+0000 7f54a52ac640 1 -- 172.21.15.113:0/307177217 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c037070 con 0x7f54a0145890 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.049+0000 7f54a52ac640 1 -- 172.21.15.113:0/307177217 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549c032440 con 0x7f54a0145890 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.049+0000 7f54a52ac640 1 -- 172.21.15.113:0/307177217 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c032810 con 0x7f54a0145890 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 -- 172.21.15.113:0/307177217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0145890 msgr2=0x7f54a00ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 --2- 172.21.15.113:0/307177217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0145890 0x7f54a00ce7e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c02a210 comp rx=0 tx=0).stop 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 -- 172.21.15.113:0/307177217 shutdown_connections 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 --2- 172.21.15.113:0/307177217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a00ced20 0x7f54a006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 --2- 172.21.15.113:0/307177217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0145890 0x7f54a00ce7e0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 -- 172.21.15.113:0/307177217 >> 172.21.15.113:0/307177217 conn(0x7f54a00b1b30 msgr2=0x7f54a00b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:12.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 -- 172.21.15.113:0/307177217 shutdown_connections 2024-04-25T00:08:12.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.050+0000 7f54a8512640 1 -- 172.21.15.113:0/307177217 wait complete. 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.051+0000 7f54a8512640 1 Processor -- start 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.051+0000 7f54a8512640 1 -- start start 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a006e610 0x7f54a01f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54a00ced20 0x7f54a00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a0145890 0x7f54a00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54a0070f10 con 0x7f54a006e610 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54a006ea60 con 0x7f54a00ced20 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f54a00b53b0 con 0x7f54a0145890 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a006e610 0x7f54a01f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a006e610 0x7f54a01f8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48832/0 (socket says 172.21.15.113:48832) 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 -- 172.21.15.113:0/2644656080 learned_addr learned my addr 172.21.15.113:0/2644656080 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a5aad640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54a00ced20 0x7f54a00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a6aaf640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a0145890 0x7f54a00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 -- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a0145890 msgr2=0x7f54a00afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a0145890 0x7f54a00afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 -- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54a00ced20 msgr2=0x7f54a00b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54a00ced20 0x7f54a00b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 -- 172.21.15.113:0/2644656080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a01f85b0 con 0x7f54a006e610 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a6aaf640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a0145890 0x7f54a00afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a5aad640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54a00ced20 0x7f54a00b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a62ae640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a006e610 0x7f54a01f8070 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f5496ffd640 1 -- 172.21.15.113:0/2644656080 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c041020 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f5496ffd640 1 -- 172.21.15.113:0/2644656080 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549c030970 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f5496ffd640 1 -- 172.21.15.113:0/2644656080 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c037040 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a015ef70 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.052+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a015e490 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.053+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a00b58c0 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.054+0000 7f5496ffd640 1 -- 172.21.15.113:0/2644656080 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f549c044020 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.054+0000 7f5496ffd640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f548406be30 0x7f548406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.054+0000 7f5496ffd640 1 -- 172.21.15.113:0/2644656080 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4857+0+0 (secure 0 0 0) 0x7f549c0ab510 con 0x7f54a006e610 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.054+0000 7f54a5aad640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f548406be30 0x7f548406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.054+0000 7f54a5aad640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f548406be30 0x7f548406e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5490006fd0 tx=0x7f5490008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.058+0000 7f5496ffd640 1 -- 172.21.15.113:0/2644656080 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54a00b58c0 con 0x7f54a006e610 2024-04-25T00:08:12.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.233+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f54a0144f60 con 0x7f54a006e610 2024-04-25T00:08:12.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.681+0000 7f5496ffd640 1 -- 172.21.15.113:0/2644656080 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v155) ==== 138+0+0 (secure 0 0 0) 0x7f549c0ab030 con 0x7f54a006e610 2024-04-25T00:08:12.682 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:08:12.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f548406be30 msgr2=0x7f548406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f548406be30 0x7f548406e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5490006fd0 tx=0x7f5490008040 comp rx=0 tx=0).stop 2024-04-25T00:08:12.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a006e610 msgr2=0x7f54a01f8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a006e610 0x7f54a01f8070 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 shutdown_connections 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f548406be30 0x7f548406e2f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54a00ced20 0x7f54a00b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54a0145890 0x7f54a00afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 --2- 172.21.15.113:0/2644656080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a006e610 0x7f54a01f8070 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 >> 172.21.15.113:0/2644656080 conn(0x7f54a00b1b30 msgr2=0x7f54a00b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 shutdown_connections 2024-04-25T00:08:12.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.684+0000 7f54a8512640 1 -- 172.21.15.113:0/2644656080 wait complete. 2024-04-25T00:08:12.698 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:08:12.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.803+0000 7f2633437640 1 Processor -- start 2024-04-25T00:08:12.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.803+0000 7f2633437640 1 -- start start 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f2633437640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f262c0d06c0 0x7f262c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f2633437640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f262c1604d0 0x7f262c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f2633437640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f262c15d6b0 con 0x7f262c13f050 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f2633437640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f262c15c6a0 con 0x7f262c0d06c0 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f2633437640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f262c15c8f0 con 0x7f262c1604d0 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26311d3640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f262c13f050 0x7f262c0cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58508/0 (socket says 172.21.15.113:58508) 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26311d3640 1 -- 172.21.15.113:0/1679465502 learned_addr learned my addr 172.21.15.113:0/1679465502 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26309d2640 1 --2- 172.21.15.113:0/1679465502 >> v2:172.21.15.113:3300/0 conn(0x7f262c0d06c0 0x7f262c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26319d4640 1 --2- 172.21.15.113:0/1679465502 >> v2:172.21.15.178:3300/0 conn(0x7f262c1604d0 0x7f262c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f2623fff640 1 -- 172.21.15.113:0/1679465502 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3866199577 0 0) 0x7f262c15d6b0 con 0x7f262c13f050 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f2623fff640 1 -- 172.21.15.113:0/1679465502 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2618003520 con 0x7f262c13f050 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26309d2640 1 -- 172.21.15.113:0/1679465502 >> v2:172.21.15.178:3300/0 conn(0x7f262c1604d0 msgr2=0x7f262c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26309d2640 1 --2- 172.21.15.113:0/1679465502 >> v2:172.21.15.178:3300/0 conn(0x7f262c1604d0 0x7f262c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26309d2640 1 -- 172.21.15.113:0/1679465502 >> v1:172.21.15.113:6789/0 conn(0x7f262c13f050 legacy=0x7f262c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26309d2640 1 -- 172.21.15.113:0/1679465502 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c15cbd0 con 0x7f262c0d06c0 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.804+0000 7f26319d4640 1 --2- 172.21.15.113:0/1679465502 >> v2:172.21.15.178:3300/0 conn(0x7f262c1604d0 0x7f262c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.805+0000 7f26309d2640 1 --2- 172.21.15.113:0/1679465502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c0d06c0 0x7f262c15ff90 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f262c156d50 tx=0x7f261c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d160a368fa533d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.805+0000 7f2623fff640 1 -- 172.21.15.113:0/1679465502 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c037070 con 0x7f262c0d06c0 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.805+0000 7f2623fff640 1 -- 172.21.15.113:0/1679465502 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261c032440 con 0x7f262c0d06c0 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.805+0000 7f2623fff640 1 -- 172.21.15.113:0/1679465502 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c02ae60 con 0x7f262c0d06c0 2024-04-25T00:08:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.805+0000 7f2633437640 1 -- 172.21.15.113:0/1679465502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c0d06c0 msgr2=0x7f262c15ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.805+0000 7f2633437640 1 --2- 172.21.15.113:0/1679465502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c0d06c0 0x7f262c15ff90 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f262c156d50 tx=0x7f261c02a110 comp rx=0 tx=0).stop 2024-04-25T00:08:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.806+0000 7f2633437640 1 -- 172.21.15.113:0/1679465502 shutdown_connections 2024-04-25T00:08:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.806+0000 7f2633437640 1 --2- 172.21.15.113:0/1679465502 >> v2:172.21.15.178:3300/0 conn(0x7f262c1604d0 0x7f262c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.806+0000 7f2633437640 1 --2- 172.21.15.113:0/1679465502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c0d06c0 0x7f262c15ff90 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.806+0000 7f2633437640 1 -- 172.21.15.113:0/1679465502 >> 172.21.15.113:0/1679465502 conn(0x7f262c06b0f0 msgr2=0x7f262c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.806+0000 7f2633437640 1 -- 172.21.15.113:0/1679465502 shutdown_connections 2024-04-25T00:08:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.806+0000 7f2633437640 1 -- 172.21.15.113:0/1679465502 wait complete. 2024-04-25T00:08:12.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 Processor -- start 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 -- start start 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f262c0d06c0 0x7f262c1f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f262c13f050 0x7f262c1f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c1604d0 0x7f262c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f262c13f430 con 0x7f262c1604d0 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f262c15d190 con 0x7f262c13f050 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f2633437640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f262c160960 con 0x7f262c0d06c0 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.807+0000 7f26319d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c1604d0 0x7f262c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c1604d0 0x7f262c0cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48854/0 (socket says 172.21.15.113:48854) 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 -- 172.21.15.113:0/2503419988 learned_addr learned my addr 172.21.15.113:0/2503419988 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26309d2640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f262c13f050 0x7f262c1f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 -- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f262c0d06c0 msgr2=0x7f262c1f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f262c0d06c0 0x7f262c1f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 -- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f262c13f050 msgr2=0x7f262c1f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f262c13f050 0x7f262c1f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 -- 172.21.15.113:0/2503419988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c1f21c0 con 0x7f262c1604d0 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26309d2640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f262c13f050 0x7f262c1f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f26319d4640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c1604d0 0x7f262c0cf570 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f2624007e70 tx=0x7f262400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f2621ffb640 1 -- 172.21.15.113:0/2503419988 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262401a070 con 0x7f262c1604d0 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f2621ffb640 1 -- 172.21.15.113:0/2503419988 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2624003bb0 con 0x7f262c1604d0 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c1f24a0 con 0x7f262c1604d0 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f2621ffb640 1 -- 172.21.15.113:0/2503419988 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624004030 con 0x7f262c1604d0 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.808+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1f1ed0 con 0x7f262c1604d0 2024-04-25T00:08:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.809+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262c0691a0 con 0x7f262c1604d0 2024-04-25T00:08:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.810+0000 7f2621ffb640 1 -- 172.21.15.113:0/2503419988 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f262401d070 con 0x7f262c1604d0 2024-04-25T00:08:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.810+0000 7f2621ffb640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f261006be60 0x7f261006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.810+0000 7f2621ffb640 1 -- 172.21.15.113:0/2503419988 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5227+0+0 (secure 0 0 0) 0x7f262408af30 con 0x7f262c1604d0 2024-04-25T00:08:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.810+0000 7f26311d3640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f261006be60 0x7f261006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.810+0000 7f26311d3640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f261006be60 0x7f261006e320 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2628000f60 tx=0x7f26280078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.814+0000 7f2621ffb640 1 -- 172.21.15.113:0/2503419988 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2624010070 con 0x7f262c1604d0 2024-04-25T00:08:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:12.988+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f262c152ec0 con 0x7f262c1604d0 2024-04-25T00:08:13.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.686+0000 7f2621ffb640 1 -- 172.21.15.113:0/2503419988 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v156) ==== 170+0+0 (secure 0 0 0) 0x7f2624059ac0 con 0x7f262c1604d0 2024-04-25T00:08:13.687 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:08:13.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.690+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f261006be60 msgr2=0x7f261006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.690+0000 7f2633437640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f261006be60 0x7f261006e320 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2628000f60 tx=0x7f26280078f0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.690+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c1604d0 msgr2=0x7f262c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.690+0000 7f2633437640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c1604d0 0x7f262c0cf570 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f2624007e70 tx=0x7f262400d4a0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.693+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 shutdown_connections 2024-04-25T00:08:13.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.693+0000 7f2633437640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f261006be60 0x7f261006e320 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.693+0000 7f2633437640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f262c13f050 0x7f262c1f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.693+0000 7f2633437640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f262c1604d0 0x7f262c0cf570 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.693+0000 7f2633437640 1 --2- 172.21.15.113:0/2503419988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f262c0d06c0 0x7f262c1f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.693+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 >> 172.21.15.113:0/2503419988 conn(0x7f262c06b0f0 msgr2=0x7f262c13c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:13.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.700+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 shutdown_connections 2024-04-25T00:08:13.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.700+0000 7f2633437640 1 -- 172.21.15.113:0/2503419988 wait complete. 2024-04-25T00:08:13.715 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:08:13.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.813+0000 7f9b08cd4640 1 Processor -- start 2024-04-25T00:08:13.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.814+0000 7f9b08cd4640 1 -- start start 2024-04-25T00:08:13.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.814+0000 7f9b08cd4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b0413f050 0x7f9b040cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.814+0000 7f9b08cd4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b041606a0 0x7f9b040cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.814+0000 7f9b08cd4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0415d6b0 con 0x7f9b040d04f0 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.814+0000 7f9b08cd4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9b040d09a0 con 0x7f9b0413f050 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.814+0000 7f9b08cd4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0415c6a0 con 0x7f9b041606a0 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b0413f050 0x7f9b040cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b0413f050 0x7f9b040cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48868/0 (socket says 172.21.15.113:48868) 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 -- 172.21.15.113:0/1699176283 learned_addr learned my addr 172.21.15.113:0/1699176283 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b02d9d640 1 --2- 172.21.15.113:0/1699176283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b041606a0 0x7f9b040cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 -- 172.21.15.113:0/1699176283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b041606a0 msgr2=0x7f9b040cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 --2- 172.21.15.113:0/1699176283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b041606a0 0x7f9b040cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 -- 172.21.15.113:0/1699176283 >> v1:172.21.15.178:6789/0 conn(0x7f9b040d04f0 legacy=0x7f9b0415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 -- 172.21.15.113:0/1699176283 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0415c8f0 con 0x7f9b0413f050 2024-04-25T00:08:13.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.815+0000 7f9b0259c640 1 --2- 172.21.15.113:0/1699176283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b040cffb0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009870 tx=0x7f9af802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8117736050fd132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b0159a640 1 -- 172.21.15.113:0/1699176283 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8037070 con 0x7f9b0413f050 2024-04-25T00:08:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b0159a640 1 -- 172.21.15.113:0/1699176283 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af8032440 con 0x7f9b0413f050 2024-04-25T00:08:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b0159a640 1 -- 172.21.15.113:0/1699176283 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8032810 con 0x7f9b0413f050 2024-04-25T00:08:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 -- 172.21.15.113:0/1699176283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 msgr2=0x7f9b040cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/1699176283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b040cffb0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009870 tx=0x7f9af802a180 comp rx=0 tx=0).stop 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 -- 172.21.15.113:0/1699176283 shutdown_connections 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/1699176283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b041606a0 0x7f9b040cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/1699176283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b040cffb0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 -- 172.21.15.113:0/1699176283 >> 172.21.15.113:0/1699176283 conn(0x7f9b0406b0f0 msgr2=0x7f9b0406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 -- 172.21.15.113:0/1699176283 shutdown_connections 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.816+0000 7f9b08cd4640 1 -- 172.21.15.113:0/1699176283 wait complete. 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 Processor -- start 2024-04-25T00:08:13.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 -- start start 2024-04-25T00:08:13.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b040d04f0 0x7f9b0415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b0415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b041606a0 0x7f9b0413aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0415d190 con 0x7f9b0413f050 2024-04-25T00:08:13.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0415ceb0 con 0x7f9b041606a0 2024-04-25T00:08:13.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b08cd4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0413f430 con 0x7f9b040d04f0 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.818+0000 7f9b01d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b0415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b0415bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48876/0 (socket says 172.21.15.113:48876) 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 -- 172.21.15.113:0/4208004461 learned_addr learned my addr 172.21.15.113:0/4208004461 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b02d9d640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b041606a0 0x7f9b0413aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b0259c640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b040d04f0 0x7f9b0415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 -- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b040d04f0 msgr2=0x7f9b0415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b040d04f0 0x7f9b0415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 -- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b041606a0 msgr2=0x7f9b0413aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b041606a0 0x7f9b0413aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 -- 172.21.15.113:0/4208004461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0415a910 con 0x7f9b0413f050 2024-04-25T00:08:13.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b01d9b640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b0415bc70 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c430 tx=0x7f9af400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9af2ffd640 1 -- 172.21.15.113:0/4208004461 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4011790 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9af2ffd640 1 -- 172.21.15.113:0/4208004461 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af4017070 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0415abd0 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9af2ffd640 1 -- 172.21.15.113:0/4208004461 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4011bb0 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.819+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0415b190 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.820+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b040691a0 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.821+0000 7f9af2ffd640 1 -- 172.21.15.113:0/4208004461 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9af401c460 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.821+0000 7f9af2ffd640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9ae406be30 0x7f9ae406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.821+0000 7f9b0259c640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9ae406be30 0x7f9ae406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.821+0000 7f9af2ffd640 1 -- 172.21.15.113:0/4208004461 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5609+0+0 (secure 0 0 0) 0x7f9af408ab80 con 0x7f9b0413f050 2024-04-25T00:08:13.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.822+0000 7f9b0259c640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9ae406be30 0x7f9ae406e2f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034000 tx=0x7f9af802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:13.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:13.826+0000 7f9af2ffd640 1 -- 172.21.15.113:0/4208004461 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9af4058f40 con 0x7f9b0413f050 2024-04-25T00:08:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.034+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9b040b2090 con 0x7f9b0413f050 2024-04-25T00:08:14.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.688+0000 7f9af2ffd640 1 -- 172.21.15.113:0/4208004461 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v82) ==== 174+0+0 (secure 0 0 0) 0x7f9af405d240 con 0x7f9b0413f050 2024-04-25T00:08:14.689 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 22 and data pool 23 2024-04-25T00:08:14.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.692+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9ae406be30 msgr2=0x7f9ae406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.692+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9ae406be30 0x7f9ae406e2f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034000 tx=0x7f9af802e000 comp rx=0 tx=0).stop 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.692+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 msgr2=0x7f9b0415bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.692+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b0415bc70 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c430 tx=0x7f9af400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.695+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 shutdown_connections 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.695+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9ae406be30 0x7f9ae406e2f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.695+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b041606a0 0x7f9b0413aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.695+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b0413f050 0x7f9b0415bc70 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.695+0000 7f9b08cd4640 1 --2- 172.21.15.113:0/4208004461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b040d04f0 0x7f9b0415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.695+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 >> 172.21.15.113:0/4208004461 conn(0x7f9b0406b0f0 msgr2=0x7f9b040b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:14.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.697+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 shutdown_connections 2024-04-25T00:08:14.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.699+0000 7f9b08cd4640 1 -- 172.21.15.113:0/4208004461 wait complete. 2024-04-25T00:08:14.723 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:14.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.818+0000 7ff72a04c640 1 Processor -- start 2024-04-25T00:08:14.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.818+0000 7ff72a04c640 1 -- start start 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.819+0000 7ff72a04c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff72415e320 0x7ff7240d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.819+0000 7ff72a04c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7240d3de0 con 0x7ff7240d0b00 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.819+0000 7ff72a04c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7240d2800 con 0x7ff7240d2290 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.819+0000 7ff72a04c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff7240d3410 con 0x7ff72415e320 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.819+0000 7ff7237fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff72415e320 0x7ff7240d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.819+0000 7ff723fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff7240d0b00 0x7ff7240b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58512/0 (socket says 172.21.15.113:58512) 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.819+0000 7ff723fff640 1 -- 172.21.15.113:0/626481623 learned_addr learned my addr 172.21.15.113:0/626481623 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:14.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.820+0000 7ff7237fe640 1 -- 172.21.15.113:0/626481623 >> v1:172.21.15.178:6789/0 conn(0x7ff7240d2290 legacy=0x7ff7240b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:14.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.820+0000 7ff7237fe640 1 -- 172.21.15.113:0/626481623 >> v1:172.21.15.113:6789/0 conn(0x7ff7240d0b00 legacy=0x7ff7240b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:14.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.820+0000 7ff722ffd640 1 -- 172.21.15.113:0/626481623 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781692992 0 0) 0x7ff7240d3de0 con 0x7ff7240d0b00 2024-04-25T00:08:14.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.820+0000 7ff7237fe640 1 -- 172.21.15.113:0/626481623 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7240d2ae0 con 0x7ff72415e320 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.820+0000 7ff7237fe640 1 --2- 172.21.15.113:0/626481623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff72415e320 0x7ff7240d38a0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff7140097a0 tx=0x7ff71402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0376bf8ac71dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff722ffd640 1 -- 172.21.15.113:0/626481623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714037070 con 0x7ff72415e320 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff722ffd640 1 -- 172.21.15.113:0/626481623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff714032640 con 0x7ff72415e320 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff722ffd640 1 -- 172.21.15.113:0/626481623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714032a10 con 0x7ff72415e320 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff72a04c640 1 -- 172.21.15.113:0/626481623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff72415e320 msgr2=0x7ff7240d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff72a04c640 1 --2- 172.21.15.113:0/626481623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff72415e320 0x7ff7240d38a0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff7140097a0 tx=0x7ff71402a2c0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff72a04c640 1 -- 172.21.15.113:0/626481623 shutdown_connections 2024-04-25T00:08:14.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff72a04c640 1 --2- 172.21.15.113:0/626481623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff72415e320 0x7ff7240d38a0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff72a04c640 1 -- 172.21.15.113:0/626481623 >> 172.21.15.113:0/626481623 conn(0x7ff7240b5c20 msgr2=0x7ff7240b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:14.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff72a04c640 1 -- 172.21.15.113:0/626481623 shutdown_connections 2024-04-25T00:08:14.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.821+0000 7ff72a04c640 1 -- 172.21.15.113:0/626481623 wait complete. 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.822+0000 7ff72a04c640 1 Processor -- start 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.822+0000 7ff72a04c640 1 -- start start 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff72a04c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7240d0b00 0x7ff72414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff72a04c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7240d2290 0x7ff7240aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff72a04c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff72415e320 0x7ff72414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff72a04c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff72415e700 con 0x7ff7240d0b00 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff72a04c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7240d30a0 con 0x7ff7240d2290 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff72a04c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff72415e880 con 0x7ff72415e320 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff7237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7240d0b00 0x7ff72414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff7237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7240d0b00 0x7ff72414f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48886/0 (socket says 172.21.15.113:48886) 2024-04-25T00:08:14.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.823+0000 7ff7237fe640 1 -- 172.21.15.113:0/250097600 learned_addr learned my addr 172.21.15.113:0/250097600 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff7237fe640 1 -- 172.21.15.113:0/250097600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff72415e320 msgr2=0x7ff72414ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff7237fe640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff72415e320 0x7ff72414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff7237fe640 1 -- 172.21.15.113:0/250097600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7240d2290 msgr2=0x7ff7240aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff7237fe640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7240d2290 0x7ff7240aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff7237fe640 1 -- 172.21.15.113:0/250097600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72414faa0 con 0x7ff7240d0b00 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff7237fe640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7240d0b00 0x7ff72414f560 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ff714000c00 tx=0x7ff7140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff720ff9640 1 -- 172.21.15.113:0/250097600 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714041020 con 0x7ff7240d0b00 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff720ff9640 1 -- 172.21.15.113:0/250097600 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff714030970 con 0x7ff7240d0b00 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff720ff9640 1 -- 172.21.15.113:0/250097600 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714037040 con 0x7ff7240d0b00 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7240ab320 con 0x7ff7240d0b00 2024-04-25T00:08:14.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.824+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7240ab8e0 con 0x7ff7240d0b00 2024-04-25T00:08:14.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.825+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7240691a0 con 0x7ff7240d0b00 2024-04-25T00:08:14.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.825+0000 7ff720ff9640 1 -- 172.21.15.113:0/250097600 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff714044020 con 0x7ff7240d0b00 2024-04-25T00:08:14.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.826+0000 7ff720ff9640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff70406be30 0x7ff70406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:14.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.826+0000 7ff720ff9640 1 -- 172.21.15.113:0/250097600 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5709+0+0 (secure 0 0 0) 0x7ff7140ab750 con 0x7ff7240d0b00 2024-04-25T00:08:14.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.826+0000 7ff71affd640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff70406be30 0x7ff70406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:14.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.826+0000 7ff71affd640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff70406be30 0x7ff70406e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff70c000f60 tx=0x7ff70c0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:14.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:14.830+0000 7ff720ff9640 1 -- 172.21.15.113:0/250097600 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff71404b030 con 0x7ff7240d0b00 2024-04-25T00:08:15.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.005+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7240b3f10 con 0x7ff7240d0b00 2024-04-25T00:08:15.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.006+0000 7ff720ff9640 1 -- 172.21.15.113:0/250097600 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+16732 (secure 0 0 0) 0x7ff72414faa0 con 0x7ff7240d0b00 2024-04-25T00:08:15.007 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:15.007 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":157,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:14.035936+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:08:12.252280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:08:13.007636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff70406be30 msgr2=0x7ff70406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff70406be30 0x7ff70406e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff70c000f60 tx=0x7ff70c0079b0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7240d0b00 msgr2=0x7ff72414f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7240d0b00 0x7ff72414f560 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ff714000c00 tx=0x7ff7140046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 shutdown_connections 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff70406be30 0x7ff70406e2f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7240d2290 0x7ff7240aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff72415e320 0x7ff72414ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 --2- 172.21.15.113:0/250097600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7240d0b00 0x7ff72414f560 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 >> 172.21.15.113:0/250097600 conn(0x7ff7240b5c20 msgr2=0x7ff7240d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 shutdown_connections 2024-04-25T00:08:15.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.008+0000 7ff72a04c640 1 -- 172.21.15.113:0/250097600 wait complete. 2024-04-25T00:08:15.020 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:15.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.117+0000 7fa9aa89e640 1 Processor -- start 2024-04-25T00:08:15.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.117+0000 7fa9aa89e640 1 -- start start 2024-04-25T00:08:15.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9aa89e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9a406e6e0 0x7fa9a40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9aa89e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a4144cf0 con 0x7fa9a40b3fd0 2024-04-25T00:08:15.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9aa89e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a40b2f30 con 0x7fa9a4144910 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9aa89e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a40b3210 con 0x7fa9a406e6e0 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9a8e3b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9a406e6e0 0x7fa9a40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9a37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa9a40b3fd0 0x7fa9a406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58514/0 (socket says 172.21.15.113:58514) 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9a37fe640 1 -- 172.21.15.113:0/3786383144 learned_addr learned my addr 172.21.15.113:0/3786383144 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9a8e3b640 1 -- 172.21.15.113:0/3786383144 >> v1:172.21.15.178:6789/0 conn(0x7fa9a4144910 legacy=0x7fa9a40b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9a8e3b640 1 -- 172.21.15.113:0/3786383144 >> v1:172.21.15.113:6789/0 conn(0x7fa9a40b3fd0 legacy=0x7fa9a406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9a8e3b640 1 -- 172.21.15.113:0/3786383144 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a40b34f0 con 0x7fa9a406e6e0 2024-04-25T00:08:15.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.118+0000 7fa9a2ffd640 1 -- 172.21.15.113:0/3786383144 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104338312 0 0) 0x7fa9a4144cf0 con 0x7fa9a40b3fd0 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9a8e3b640 1 --2- 172.21.15.113:0/3786383144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a40cd500 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00bfd0 tx=0x7fa99c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69586532af90ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9a2ffd640 1 -- 172.21.15.113:0/3786383144 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00b2e0 con 0x7fa9a406e6e0 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9a2ffd640 1 -- 172.21.15.113:0/3786383144 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99c009540 con 0x7fa9a406e6e0 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9a2ffd640 1 -- 172.21.15.113:0/3786383144 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c009910 con 0x7fa9a406e6e0 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9aa89e640 1 -- 172.21.15.113:0/3786383144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 msgr2=0x7fa9a40cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/3786383144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a40cd500 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00bfd0 tx=0x7fa99c02c8e0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9aa89e640 1 -- 172.21.15.113:0/3786383144 shutdown_connections 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/3786383144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a40cd500 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9aa89e640 1 -- 172.21.15.113:0/3786383144 >> 172.21.15.113:0/3786383144 conn(0x7fa9a40b1a50 msgr2=0x7fa9a40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9aa89e640 1 -- 172.21.15.113:0/3786383144 shutdown_connections 2024-04-25T00:08:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.119+0000 7fa9aa89e640 1 -- 172.21.15.113:0/3786383144 wait complete. 2024-04-25T00:08:15.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.120+0000 7fa9aa89e640 1 Processor -- start 2024-04-25T00:08:15.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9aa89e640 1 -- start start 2024-04-25T00:08:15.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9aa89e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a41525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9aa89e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a40b3fd0 0x7fa9a413e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9aa89e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4144910 0x7fa9a413e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9aa89e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a40b4da0 con 0x7fa9a406e6e0 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9aa89e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a40b4440 con 0x7fa9a40b3fd0 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9aa89e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a40b4f20 con 0x7fa9a4144910 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9a3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a41525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9a3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a41525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48896/0 (socket says 172.21.15.113:48896) 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.121+0000 7fa9a3fff640 1 -- 172.21.15.113:0/2500324557 learned_addr learned my addr 172.21.15.113:0/2500324557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a37fe640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a40b3fd0 0x7fa9a413e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a8e3b640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4144910 0x7fa9a413e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a3fff640 1 -- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4144910 msgr2=0x7fa9a413e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a3fff640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4144910 0x7fa9a413e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a3fff640 1 -- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a40b3fd0 msgr2=0x7fa9a413e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a3fff640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a40b3fd0 0x7fa9a413e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a3fff640 1 -- 172.21.15.113:0/2500324557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a415df20 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a3fff640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a41525c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b410 tx=0x7fa99400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a0ff9640 1 -- 172.21.15.113:0/2500324557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994015020 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a0ff9640 1 -- 172.21.15.113:0/2500324557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa994011830 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a413a940 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9a0ff9640 1 -- 172.21.15.113:0/2500324557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994011c20 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.122+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a413af00 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.123+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a40691a0 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.123+0000 7fa9a0ff9640 1 -- 172.21.15.113:0/2500324557 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa994018020 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.124+0000 7fa9a0ff9640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa99006be30 0x7fa99006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.124+0000 7fa9a0ff9640 1 -- 172.21.15.113:0/2500324557 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5709+0+0 (secure 0 0 0) 0x7fa99408adb0 con 0x7fa9a406e6e0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.124+0000 7fa9a37fe640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa99006be30 0x7fa99006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.124+0000 7fa9a37fe640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa99006be30 0x7fa99006e2f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40722a0 tx=0x7fa998007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.128+0000 7fa9a0ff9640 1 -- 172.21.15.113:0/2500324557 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa99401f020 con 0x7fa9a406e6e0 2024-04-25T00:08:15.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.305+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9a40731a0 con 0x7fa9a406e6e0 2024-04-25T00:08:15.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.306+0000 7fa9a0ff9640 1 -- 172.21.15.113:0/2500324557 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+16732 (secure 0 0 0) 0x7fa9940172c0 con 0x7fa9a406e6e0 2024-04-25T00:08:15.307 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:15.307 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":157,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:14.035936+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:08:12.252280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:08:13.007636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:15.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa99006be30 msgr2=0x7fa99006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa99006be30 0x7fa99006e2f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40722a0 tx=0x7fa998007b50 comp rx=0 tx=0).stop 2024-04-25T00:08:15.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 msgr2=0x7fa9a41525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a41525c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b410 tx=0x7fa99400b8d0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 shutdown_connections 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa99006be30 0x7fa99006e2f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a40b3fd0 0x7fa9a413e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4144910 0x7fa9a413e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 --2- 172.21.15.113:0/2500324557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9a406e6e0 0x7fa9a41525c0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 >> 172.21.15.113:0/2500324557 conn(0x7fa9a40b1a50 msgr2=0x7fa9a40719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 shutdown_connections 2024-04-25T00:08:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.309+0000 7fa9aa89e640 1 -- 172.21.15.113:0/2500324557 wait complete. 2024-04-25T00:08:15.320 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:08:15.337 INFO:tasks.ceph.mds.b.smithi178.stderr:2024-04-25T00:08:15.336+0000 7f06ec2ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:08:15.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.420+0000 7f593be9a640 1 Processor -- start 2024-04-25T00:08:15.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.420+0000 7f593be9a640 1 -- start start 2024-04-25T00:08:15.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593be9a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f593415e2d0 0x7f593415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593be9a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59340b3080 0x7f59340b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593be9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e6b0 0x7f59340b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593be9a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f59340cde00 con 0x7f59340b3080 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593be9a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f593415f4f0 con 0x7f593415e2d0 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593be9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f593415f7d0 con 0x7f593415e6b0 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59340b3080 0x7f59340b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59340b3080 0x7f59340b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48910/0 (socket says 172.21.15.113:48910) 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 -- 172.21.15.113:0/3028024165 learned_addr learned my addr 172.21.15.113:0/3028024165 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939c36640 1 --2- 172.21.15.113:0/3028024165 >> v2:172.21.15.178:3300/0 conn(0x7f593415e2d0 0x7f593415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593a437640 1 --2- 172.21.15.113:0/3028024165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e6b0 0x7f59340b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 -- 172.21.15.113:0/3028024165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e6b0 msgr2=0x7f59340b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 --2- 172.21.15.113:0/3028024165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e6b0 0x7f59340b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 -- 172.21.15.113:0/3028024165 >> v2:172.21.15.178:3300/0 conn(0x7f593415e2d0 msgr2=0x7f593415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:15.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 --2- 172.21.15.113:0/3028024165 >> v2:172.21.15.178:3300/0 conn(0x7f593415e2d0 0x7f593415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939435640 1 -- 172.21.15.113:0/3028024165 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593415fab0 con 0x7f59340b3080 2024-04-25T00:08:15.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f593a437640 1 --2- 172.21.15.113:0/3028024165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e6b0 0x7f59340b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:15.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.421+0000 7f5939c36640 1 --2- 172.21.15.113:0/3028024165 >> v2:172.21.15.178:3300/0 conn(0x7f593415e2d0 0x7f593415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:15.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.422+0000 7f5939435640 1 --2- 172.21.15.113:0/3028024165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340b3080 0x7f59340b4330 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f59340b2d40 tx=0x7f592402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb3474e1d502956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.422+0000 7f5938c34640 1 -- 172.21.15.113:0/3028024165 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924037070 con 0x7f59340b3080 2024-04-25T00:08:15.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.422+0000 7f5938c34640 1 -- 172.21.15.113:0/3028024165 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5924032440 con 0x7f59340b3080 2024-04-25T00:08:15.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.422+0000 7f5938c34640 1 -- 172.21.15.113:0/3028024165 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924032810 con 0x7f59340b3080 2024-04-25T00:08:15.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 -- 172.21.15.113:0/3028024165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340b3080 msgr2=0x7f59340b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 --2- 172.21.15.113:0/3028024165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340b3080 0x7f59340b4330 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f59340b2d40 tx=0x7f592402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 -- 172.21.15.113:0/3028024165 shutdown_connections 2024-04-25T00:08:15.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 --2- 172.21.15.113:0/3028024165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e6b0 0x7f59340b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 --2- 172.21.15.113:0/3028024165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340b3080 0x7f59340b4330 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 --2- 172.21.15.113:0/3028024165 >> v2:172.21.15.178:3300/0 conn(0x7f593415e2d0 0x7f593415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 -- 172.21.15.113:0/3028024165 >> 172.21.15.113:0/3028024165 conn(0x7f593406b0f0 msgr2=0x7f593406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 -- 172.21.15.113:0/3028024165 shutdown_connections 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.423+0000 7f593be9a640 1 -- 172.21.15.113:0/3028024165 wait complete. 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.424+0000 7f593be9a640 1 Processor -- start 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.424+0000 7f593be9a640 1 -- start start 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59340b3080 0x7f593415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e2d0 0x7f593413c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f593415e6b0 0x7f593414eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f593415f2b0 con 0x7f593415e6b0 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f593415f130 con 0x7f59340b3080 2024-04-25T00:08:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5934071040 con 0x7f593415e2d0 2024-04-25T00:08:15.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f593415e6b0 0x7f593414eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f593415e6b0 0x7f593414eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48920/0 (socket says 172.21.15.113:48920) 2024-04-25T00:08:15.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 -- 172.21.15.113:0/457053042 learned_addr learned my addr 172.21.15.113:0/457053042 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:15.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f5939c36640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59340b3080 0x7f593415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f5939435640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e2d0 0x7f593413c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 -- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e2d0 msgr2=0x7f593413c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e2d0 0x7f593413c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 -- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59340b3080 msgr2=0x7f593415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59340b3080 0x7f593415acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 -- 172.21.15.113:0/457053042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593413cbd0 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593a437640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f593415e6b0 0x7f593414eaa0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f592c00cd50 tx=0x7f592c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f592a7fc640 1 -- 172.21.15.113:0/457053042 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c00f680 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f592a7fc640 1 -- 172.21.15.113:0/457053042 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c00f840 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f592a7fc640 1 -- 172.21.15.113:0/457053042 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c00fc30 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5934150e30 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.425+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341513f0 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.426+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59340691a0 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.427+0000 7f592a7fc640 1 -- 172.21.15.113:0/457053042 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f592c0213f0 con 0x7f593415e6b0 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.427+0000 7f592a7fc640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f591806be30 0x7f591806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.427+0000 7f592a7fc640 1 -- 172.21.15.113:0/457053042 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5709+0+0 (secure 0 0 0) 0x7f592c08b9e0 con 0x7f593415e6b0 2024-04-25T00:08:15.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.427+0000 7f5939c36640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f591806be30 0x7f591806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.428+0000 7f5939c36640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f591806be30 0x7f591806e2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f593000ab60 tx=0x7f5930008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.432+0000 7f592a7fc640 1 -- 172.21.15.113:0/457053042 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f592c014030 con 0x7f593415e6b0 2024-04-25T00:08:15.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.640+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f593415ee60 con 0x7f593415e6b0 2024-04-25T00:08:15.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.873+0000 7f592a7fc640 1 -- 172.21.15.113:0/457053042 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) ==== 118+0+0 (secure 0 0 0) 0x7f592c059dd0 con 0x7f593415e6b0 2024-04-25T00:08:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f591806be30 msgr2=0x7f591806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f591806be30 0x7f591806e2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f593000ab60 tx=0x7f5930008040 comp rx=0 tx=0).stop 2024-04-25T00:08:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f593415e6b0 msgr2=0x7f593414eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f593415e6b0 0x7f593414eaa0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f592c00cd50 tx=0x7f592c00a650 comp rx=0 tx=0).stop 2024-04-25T00:08:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 shutdown_connections 2024-04-25T00:08:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f591806be30 0x7f591806e2f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f593415e6b0 0x7f593414eaa0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f593415e2d0 0x7f593413c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.875+0000 7f593be9a640 1 --2- 172.21.15.113:0/457053042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59340b3080 0x7f593415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.876+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 >> 172.21.15.113:0/457053042 conn(0x7f593406b0f0 msgr2=0x7f59340d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.876+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 shutdown_connections 2024-04-25T00:08:15.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.876+0000 7f593be9a640 1 -- 172.21.15.113:0/457053042 wait complete. 2024-04-25T00:08:15.887 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:08:15.887 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:08:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.985+0000 7f965aaee640 1 Processor -- start 2024-04-25T00:08:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.985+0000 7f965aaee640 1 -- start start 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965aaee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96540d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965aaee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f96540b63b0 0x7f96540b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965aaee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96540d2e00 con 0x7f96540d28c0 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965aaee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f96540d30e0 con 0x7f96540b63b0 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965aaee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f96540d33c0 con 0x7f965415e510 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965908b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f96540b63b0 0x7f96540b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965908b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f96540b63b0 0x7f96540b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48934/0 (socket says 172.21.15.113:48934) 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.986+0000 7f965908b640 1 -- 172.21.15.113:0/507743590 learned_addr learned my addr 172.21.15.113:0/507743590 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.987+0000 7f965888a640 1 --2- 172.21.15.113:0/507743590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96540d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.987+0000 7f965908b640 1 -- 172.21.15.113:0/507743590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 msgr2=0x7f96540d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.987+0000 7f965908b640 1 --2- 172.21.15.113:0/507743590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96540d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.987+0000 7f965908b640 1 -- 172.21.15.113:0/507743590 >> v1:172.21.15.178:6789/0 conn(0x7f96540d28c0 legacy=0x7f96540b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.987+0000 7f965908b640 1 -- 172.21.15.113:0/507743590 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96540d36a0 con 0x7f96540b63b0 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.987+0000 7f965908b640 1 --2- 172.21.15.113:0/507743590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f96540b73c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f964c009e50 tx=0x7f964c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be45a7cea5043f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.987+0000 7f96537fe640 1 -- 172.21.15.113:0/507743590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c036070 con 0x7f96540b63b0 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f96537fe640 1 -- 172.21.15.113:0/507743590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c031600 con 0x7f96540b63b0 2024-04-25T00:08:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f96537fe640 1 -- 172.21.15.113:0/507743590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c0319d0 con 0x7f96540b63b0 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 -- 172.21.15.113:0/507743590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 msgr2=0x7f96540b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 --2- 172.21.15.113:0/507743590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f96540b73c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f964c009e50 tx=0x7f964c02a680 comp rx=0 tx=0).stop 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 -- 172.21.15.113:0/507743590 shutdown_connections 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 --2- 172.21.15.113:0/507743590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f96540b73c0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 --2- 172.21.15.113:0/507743590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96540d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 -- 172.21.15.113:0/507743590 >> 172.21.15.113:0/507743590 conn(0x7f965406b720 msgr2=0x7f96540d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 -- 172.21.15.113:0/507743590 shutdown_connections 2024-04-25T00:08:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.988+0000 7f965aaee640 1 -- 172.21.15.113:0/507743590 wait complete. 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.989+0000 7f965aaee640 1 Processor -- start 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.989+0000 7f965aaee640 1 -- start start 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965aaee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f965415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965aaee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96540d28c0 0x7f9654159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965aaee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96541f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965aaee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96540b9dd0 con 0x7f96540b63b0 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965aaee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96540d3ea0 con 0x7f96540d28c0 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965aaee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f96540b7900 con 0x7f965415e510 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965888a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f965415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965888a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f965415fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48946/0 (socket says 172.21.15.113:48946) 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965888a640 1 -- 172.21.15.113:0/2868905746 learned_addr learned my addr 172.21.15.113:0/2868905746 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f9653fff640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96540d28c0 0x7f9654159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.990+0000 7f965908b640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96541f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965888a640 1 -- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 msgr2=0x7f96541f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965888a640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96541f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965888a640 1 -- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96540d28c0 msgr2=0x7f9654159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965888a640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96540d28c0 0x7f9654159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965888a640 1 -- 172.21.15.113:0/2868905746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96541f8310 con 0x7f96540b63b0 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965908b640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96541f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965888a640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f965415fa40 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f964400d5f0 tx=0x7f964400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f96517fa640 1 -- 172.21.15.113:0/2868905746 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440125f0 con 0x7f96540b63b0 2024-04-25T00:08:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f96517fa640 1 -- 172.21.15.113:0/2868905746 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9644019070 con 0x7f96540b63b0 2024-04-25T00:08:15.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96540cb390 con 0x7f96540b63b0 2024-04-25T00:08:15.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f96517fa640 1 -- 172.21.15.113:0/2868905746 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440129d0 con 0x7f96540b63b0 2024-04-25T00:08:15.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.991+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9654153b80 con 0x7f96540b63b0 2024-04-25T00:08:15.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.992+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96540691a0 con 0x7f96540b63b0 2024-04-25T00:08:15.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.992+0000 7f96517fa640 1 -- 172.21.15.113:0/2868905746 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9644012b70 con 0x7f96540b63b0 2024-04-25T00:08:15.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.993+0000 7f96517fa640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f964006be60 0x7f964006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.993+0000 7f96517fa640 1 -- 172.21.15.113:0/2868905746 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f964408b180 con 0x7f96540b63b0 2024-04-25T00:08:15.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.993+0000 7f9653fff640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f964006be60 0x7f964006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.993+0000 7f9653fff640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f964006be60 0x7f964006e320 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f965415b050 tx=0x7f9648009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:15.997+0000 7f96517fa640 1 -- 172.21.15.113:0/2868905746 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9644059bc0 con 0x7f96540b63b0 2024-04-25T00:08:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.208+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f96540cee20 con 0x7f96540b63b0 2024-04-25T00:08:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.875+0000 7f96517fa640 1 -- 172.21.15.113:0/2868905746 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) ==== 122+0+0 (secure 0 0 0) 0x7f96440594c0 con 0x7f96540b63b0 2024-04-25T00:08:16.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f964006be60 msgr2=0x7f964006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f964006be60 0x7f964006e320 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f965415b050 tx=0x7f9648009040 comp rx=0 tx=0).stop 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 msgr2=0x7f965415fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f965415fa40 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f964400d5f0 tx=0x7f964400dab0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 shutdown_connections 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f964006be60 0x7f964006e320 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96540d28c0 0x7f9654159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965415e510 0x7f96541f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 --2- 172.21.15.113:0/2868905746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96540b63b0 0x7f965415fa40 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 >> 172.21.15.113:0/2868905746 conn(0x7f965406b720 msgr2=0x7f96540d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 shutdown_connections 2024-04-25T00:08:16.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.878+0000 7f965aaee640 1 -- 172.21.15.113:0/2868905746 wait complete. 2024-04-25T00:08:16.891 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:08:16.892 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:16.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.993+0000 7f6810316640 1 Processor -- start 2024-04-25T00:08:16.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.993+0000 7f6810316640 1 -- start start 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f6810316640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 0x7f680806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f6810316640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6808144cf0 con 0x7f6808144910 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f6810316640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68080b2f30 con 0x7f680806e510 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f6810316640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f68080b3210 con 0x7f68080b3fd0 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f680e0b2640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6808144910 0x7f68080b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58516/0 (socket says 172.21.15.113:58516) 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f680e0b2640 1 -- 172.21.15.113:0/920867106 learned_addr learned my addr 172.21.15.113:0/920867106 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f680d8b1640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 0x7f680806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502436726 0 0) 0x7f6808144cf0 con 0x7f6808144910 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.994+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67f4003520 con 0x7f6808144910 2024-04-25T00:08:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1155873453 0 0) 0x7f67f4003520 con 0x7f6808144910 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 msgr2=0x7f680806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 0x7f680806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 >> v1:172.21.15.178:6789/0 conn(0x7f680806e510 legacy=0x7f68080cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68080b34f0 con 0x7f6808144910 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6804001670 con 0x7f6808144910 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 >> v1:172.21.15.113:6789/0 conn(0x7f6808144910 legacy=0x7f68080b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67f40039e0 0x7f67f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 0x7f67f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f4004d40 0x7f67f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f68080b3210 con 0x7f67f4004d40 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67f400b9f0 con 0x7f67f40039e0 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f67f400bbf0 con 0x7f68080b3fd0 2024-04-25T00:08:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680e8b3640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f4004d40 0x7f67f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680e0b2640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67f40039e0 0x7f67f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680d8b1640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 0x7f67f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680e8b3640 1 -- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 msgr2=0x7f67f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680e8b3640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68080b3fd0 0x7f67f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680e8b3640 1 -- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67f40039e0 msgr2=0x7f67f4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680e8b3640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67f40039e0 0x7f67f4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.995+0000 7f680e8b3640 1 -- 172.21.15.113:0/920867106 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f4003520 con 0x7f67f4004d40 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.996+0000 7f680e8b3640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f4004d40 0x7f67f40051c0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f6800001ff0 tx=0x7f67f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea1853481dabfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.996+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800001bc0 con 0x7f67f4004d40 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.996+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680002e650 con 0x7f67f4004d40 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.996+0000 7f680d0b0640 1 -- 172.21.15.113:0/920867106 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680002ea40 con 0x7f67f4004d40 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.996+0000 7f6810316640 1 -- 172.21.15.113:0/920867106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f4004d40 msgr2=0x7f67f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.996+0000 7f6810316640 1 --2- 172.21.15.113:0/920867106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f4004d40 0x7f67f40051c0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f6800001ff0 tx=0x7f67f4005830 comp rx=0 tx=0).stop 2024-04-25T00:08:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.996+0000 7f680e8b3640 1 -- 172.21.15.113:0/920867106 reap_dead start 2024-04-25T00:08:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.997+0000 7f6810316640 1 -- 172.21.15.113:0/920867106 shutdown_connections 2024-04-25T00:08:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.997+0000 7f6810316640 1 -- 172.21.15.113:0/920867106 >> 172.21.15.113:0/920867106 conn(0x7f68080b1a50 msgr2=0x7f68080b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.997+0000 7f6810316640 1 -- 172.21.15.113:0/920867106 shutdown_connections 2024-04-25T00:08:17.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.997+0000 7f6810316640 1 -- 172.21.15.113:0/920867106 wait complete. 2024-04-25T00:08:17.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.998+0000 7f6810316640 1 Processor -- start 2024-04-25T00:08:17.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.998+0000 7f6810316640 1 -- start start 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.998+0000 7f6810316640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67f4004d40 0x7f680815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.998+0000 7f6810316640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680813bbf0 0x7f6808138670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.998+0000 7f6810316640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6808138bb0 0x7f68080b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f6810316640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f68080b4da0 con 0x7f6808138bb0 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f6810316640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68080b4410 con 0x7f680813bbf0 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f6810316640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f68080b4f20 con 0x7f67f4004d40 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6808138bb0 0x7f68080b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6808138bb0 0x7f68080b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48966/0 (socket says 172.21.15.113:48966) 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 -- 172.21.15.113:0/3805093192 learned_addr learned my addr 172.21.15.113:0/3805093192 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680d8b1640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680813bbf0 0x7f6808138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e0b2640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67f4004d40 0x7f680815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 -- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67f4004d40 msgr2=0x7f680815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67f4004d40 0x7f680815d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 -- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680813bbf0 msgr2=0x7f6808138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680813bbf0 0x7f6808138670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 -- 172.21.15.113:0/3805093192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68080cd4c0 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680d8b1640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680813bbf0 0x7f6808138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f680e8b3640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6808138bb0 0x7f68080b0b40 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f680002d0d0 tx=0x7f68000015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f67feffd640 1 -- 172.21.15.113:0/3805093192 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680002e800 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f67feffd640 1 -- 172.21.15.113:0/3805093192 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680002e9a0 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680815d790 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f67feffd640 1 -- 172.21.15.113:0/3805093192 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680002ed70 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:16.999+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68080cd760 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.000+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68080691a0 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.001+0000 7f67feffd640 1 -- 172.21.15.113:0/3805093192 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68000050e0 con 0x7f6808138bb0 2024-04-25T00:08:17.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.001+0000 7f67feffd640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f67ec06be60 0x7f67ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.001+0000 7f680e0b2640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f67ec06be60 0x7f67ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.001+0000 7f67feffd640 1 -- 172.21.15.113:0/3805093192 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f68000ad4c0 con 0x7f6808138bb0 2024-04-25T00:08:17.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.002+0000 7f680e0b2640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f67ec06be60 0x7f67ec06e320 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6804000c00 tx=0x7f6804009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.006+0000 7f67feffd640 1 -- 172.21.15.113:0/3805093192 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6800034030 con 0x7f6808138bb0 2024-04-25T00:08:17.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.217+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f680815e7b0 con 0x7f6808138bb0 2024-04-25T00:08:17.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.217+0000 7f67feffd640 1 -- 172.21.15.113:0/3805093192 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7f680007b790 con 0x7f6808138bb0 2024-04-25T00:08:17.218 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:17.218 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:17.218 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f67ec06be60 msgr2=0x7f67ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f67ec06be60 0x7f67ec06e320 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6804000c00 tx=0x7f6804009040 comp rx=0 tx=0).stop 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6808138bb0 msgr2=0x7f68080b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6808138bb0 0x7f68080b0b40 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f680002d0d0 tx=0x7f68000015b0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 shutdown_connections 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f67ec06be60 0x7f67ec06e320 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680813bbf0 0x7f6808138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6808138bb0 0x7f68080b0b40 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 --2- 172.21.15.113:0/3805093192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67f4004d40 0x7f680815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 >> 172.21.15.113:0/3805093192 conn(0x7f68080b1a50 msgr2=0x7f68081419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 shutdown_connections 2024-04-25T00:08:17.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.219+0000 7f6810316640 1 -- 172.21.15.113:0/3805093192 wait complete. 2024-04-25T00:08:17.231 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:17.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.326+0000 7fd34e664640 1 Processor -- start 2024-04-25T00:08:17.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.326+0000 7fd34e664640 1 -- start start 2024-04-25T00:08:17.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.327+0000 7fd34e664640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3480d2290 0x7fd3480b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.327+0000 7fd34e664640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3480d2800 con 0x7fd3480d0b00 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.327+0000 7fd34e664640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3480d2ae0 con 0x7fd34815e320 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.327+0000 7fd34e664640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3480d3410 con 0x7fd3480d2290 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.327+0000 7fd34cc01640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd3480d0b00 0x7fd3480d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58524/0 (socket says 172.21.15.113:58524) 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.327+0000 7fd34cc01640 1 -- 172.21.15.113:0/2046304384 learned_addr learned my addr 172.21.15.113:0/2046304384 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.327+0000 7fd3477fe640 1 --2- 172.21.15.113:0/2046304384 >> v2:172.21.15.178:3300/0 conn(0x7fd3480d2290 0x7fd3480b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.328+0000 7fd346ffd640 1 -- 172.21.15.113:0/2046304384 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024986504 0 0) 0x7fd3480d2800 con 0x7fd3480d0b00 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.328+0000 7fd346ffd640 1 -- 172.21.15.113:0/2046304384 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd330003520 con 0x7fd3480d0b00 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.328+0000 7fd3477fe640 1 -- 172.21.15.113:0/2046304384 >> v1:172.21.15.178:6789/0 conn(0x7fd34815e320 legacy=0x7fd3480d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.328+0000 7fd3477fe640 1 -- 172.21.15.113:0/2046304384 >> v1:172.21.15.113:6789/0 conn(0x7fd3480d0b00 legacy=0x7fd3480d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.328+0000 7fd346ffd640 1 -- 172.21.15.113:0/2046304384 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2783261424 0 0) 0x7fd330003520 con 0x7fd3480d0b00 2024-04-25T00:08:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.328+0000 7fd3477fe640 1 -- 172.21.15.113:0/2046304384 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3480d2dc0 con 0x7fd3480d2290 2024-04-25T00:08:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd3477fe640 1 --2- 172.21.15.113:0/2046304384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d2290 0x7fd3480b9aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0097a0 tx=0x7fd33c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de79d3d0a6b0124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd346ffd640 1 -- 172.21.15.113:0/2046304384 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c037070 con 0x7fd3480d2290 2024-04-25T00:08:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd346ffd640 1 -- 172.21.15.113:0/2046304384 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c02ace0 con 0x7fd3480d2290 2024-04-25T00:08:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd346ffd640 1 -- 172.21.15.113:0/2046304384 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3400019f0 con 0x7fd3480d2290 2024-04-25T00:08:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd34e664640 1 -- 172.21.15.113:0/2046304384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d2290 msgr2=0x7fd3480b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd34e664640 1 --2- 172.21.15.113:0/2046304384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d2290 0x7fd3480b9aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0097a0 tx=0x7fd33c02a250 comp rx=0 tx=0).stop 2024-04-25T00:08:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd34e664640 1 -- 172.21.15.113:0/2046304384 shutdown_connections 2024-04-25T00:08:17.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd34e664640 1 --2- 172.21.15.113:0/2046304384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d2290 0x7fd3480b9aa0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd34e664640 1 -- 172.21.15.113:0/2046304384 >> 172.21.15.113:0/2046304384 conn(0x7fd3480b5c20 msgr2=0x7fd3480b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd34e664640 1 -- 172.21.15.113:0/2046304384 shutdown_connections 2024-04-25T00:08:17.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.329+0000 7fd34e664640 1 -- 172.21.15.113:0/2046304384 wait complete. 2024-04-25T00:08:17.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 Processor -- start 2024-04-25T00:08:17.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 -- start start 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d0b00 0x7fd3481574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3480d2290 0x7fd3480b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd34815e320 0x7fd3480cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd34815e700 con 0x7fd3480d2290 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3480b9fe0 con 0x7fd3480d0b00 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.331+0000 7fd34e664640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd34815e880 con 0x7fd34815e320 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3480d2290 0x7fd3480b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3480d2290 0x7fd3480b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48980/0 (socket says 172.21.15.113:48980) 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 -- 172.21.15.113:0/3919964640 learned_addr learned my addr 172.21.15.113:0/3919964640 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd347fff640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d0b00 0x7fd3481574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd34cc01640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd34815e320 0x7fd3480cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 -- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd34815e320 msgr2=0x7fd3480cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd34815e320 0x7fd3480cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 -- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d0b00 msgr2=0x7fd3481574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d0b00 0x7fd3481574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 -- 172.21.15.113:0/3919964640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34815ec00 con 0x7fd3480d2290 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd347fff640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d0b00 0x7fd3481574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd34cc01640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd34815e320 0x7fd3480cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd3477fe640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3480d2290 0x7fd3480b8110 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fd33c035040 tx=0x7fd33c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd344ff9640 1 -- 172.21.15.113:0/3919964640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c042020 con 0x7fd3480d2290 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd344ff9640 1 -- 172.21.15.113:0/3919964640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c03d670 con 0x7fd3480d2290 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd344ff9640 1 -- 172.21.15.113:0/3919964640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c037040 con 0x7fd3480d2290 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34815eee0 con 0x7fd3480d2290 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.332+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3480cdb60 con 0x7fd3480d2290 2024-04-25T00:08:17.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.333+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3480691a0 con 0x7fd3480d2290 2024-04-25T00:08:17.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.334+0000 7fd344ff9640 1 -- 172.21.15.113:0/3919964640 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd33c045020 con 0x7fd3480d2290 2024-04-25T00:08:17.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.334+0000 7fd344ff9640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd33406be30 0x7fd33406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.334+0000 7fd344ff9640 1 -- 172.21.15.113:0/3919964640 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fd33c0ab750 con 0x7fd3480d2290 2024-04-25T00:08:17.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.334+0000 7fd347fff640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd33406be30 0x7fd33406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.335+0000 7fd347fff640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd33406be30 0x7fd33406e2f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd338000f60 tx=0x7fd338009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.338+0000 7fd344ff9640 1 -- 172.21.15.113:0/3919964640 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd33c04c030 con 0x7fd3480d2290 2024-04-25T00:08:17.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.512+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3480b2ea0 con 0x7fd3480d2290 2024-04-25T00:08:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.513+0000 7fd344ff9640 1 -- 172.21.15.113:0/3919964640 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16719 (secure 0 0 0) 0x7fd33c07a190 con 0x7fd3480d2290 2024-04-25T00:08:17.514 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:17.514 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":158,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:15.696246+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:08:12.252280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:08:13.007636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd33406be30 msgr2=0x7fd33406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd33406be30 0x7fd33406e2f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd338000f60 tx=0x7fd338009040 comp rx=0 tx=0).stop 2024-04-25T00:08:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3480d2290 msgr2=0x7fd3480b8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3480d2290 0x7fd3480b8110 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fd33c035040 tx=0x7fd33c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 shutdown_connections 2024-04-25T00:08:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd33406be30 0x7fd33406e2f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd34815e320 0x7fd3480cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3480d2290 0x7fd3480b8110 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 --2- 172.21.15.113:0/3919964640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3480d0b00 0x7fd3481574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.515+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 >> 172.21.15.113:0/3919964640 conn(0x7fd3480b5c20 msgr2=0x7fd3480b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.516+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 shutdown_connections 2024-04-25T00:08:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.516+0000 7fd34e664640 1 -- 172.21.15.113:0/3919964640 wait complete. 2024-04-25T00:08:17.527 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:08:17.527 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:08:17.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.624+0000 7fb781eda640 1 Processor -- start 2024-04-25T00:08:17.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.624+0000 7fb781eda640 1 -- start start 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb781eda640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb77c06c9f0 0x7fb77c0713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb781eda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c1623e0 0x7fb77c162870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb781eda640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77c0cfbc0 con 0x7fb77c161770 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb781eda640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb77c06d0f0 con 0x7fb77c06c9f0 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb781eda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c161f50 con 0x7fb77c1623e0 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb77c06c9f0 0x7fb77c0713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77b7fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb77c161770 0x7fb77c0cc510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:45090/0 (socket says 172.21.15.113:45090) 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb77c06c9f0 0x7fb77c0713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:34374/0 (socket says 172.21.15.113:34374) 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77b7fe640 1 -- 172.21.15.113:0/3524009355 learned_addr learned my addr 172.21.15.113:0/3524009355 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77bfff640 1 --2- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c1623e0 0x7fb77c162870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77affd640 1 -- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c1623e0 msgr2=0x7fb77c162870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77affd640 1 --2- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c1623e0 0x7fb77c162870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77affd640 1 -- 172.21.15.113:0/3524009355 >> v1:172.21.15.178:6789/0 conn(0x7fb77c161770 legacy=0x7fb77c0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.625+0000 7fb77affd640 1 -- 172.21.15.113:0/3524009355 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c0cfea0 con 0x7fb77c06c9f0 2024-04-25T00:08:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.626+0000 7fb77affd640 1 --2- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0713f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb7700097d0 tx=0x7fb77002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc3417d9af47585 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.626+0000 7fb77a7fc640 1 -- 172.21.15.113:0/3524009355 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770037070 con 0x7fb77c06c9f0 2024-04-25T00:08:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.626+0000 7fb77a7fc640 1 -- 172.21.15.113:0/3524009355 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77002ae40 con 0x7fb77c06c9f0 2024-04-25T00:08:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.626+0000 7fb77a7fc640 1 -- 172.21.15.113:0/3524009355 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770032660 con 0x7fb77c06c9f0 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 -- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c06c9f0 msgr2=0x7fb77c0713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 --2- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0713f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb7700097d0 tx=0x7fb77002a130 comp rx=0 tx=0).stop 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 -- 172.21.15.113:0/3524009355 shutdown_connections 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 --2- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c1623e0 0x7fb77c162870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 --2- 172.21.15.113:0/3524009355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0713f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 -- 172.21.15.113:0/3524009355 >> 172.21.15.113:0/3524009355 conn(0x7fb77c0d1ec0 msgr2=0x7fb77c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 -- 172.21.15.113:0/3524009355 shutdown_connections 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.627+0000 7fb781eda640 1 -- 172.21.15.113:0/3524009355 wait complete. 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.628+0000 7fb781eda640 1 Processor -- start 2024-04-25T00:08:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.628+0000 7fb781eda640 1 -- start start 2024-04-25T00:08:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb781eda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb781eda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c161770 0x7fb77c0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb781eda640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c1623e0 0x7fb77c0c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb781eda640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c161b50 con 0x7fb77c06c9f0 2024-04-25T00:08:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb781eda640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c06cea0 con 0x7fb77c1623e0 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb781eda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c0d0180 con 0x7fb77c161770 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48314/0 (socket says 172.21.15.113:48314) 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 -- 172.21.15.113:0/1699578012 learned_addr learned my addr 172.21.15.113:0/1699578012 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77bfff640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c1623e0 0x7fb77c0c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77affd640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c161770 0x7fb77c0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 -- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c161770 msgr2=0x7fb77c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c161770 0x7fb77c0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 -- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c1623e0 msgr2=0x7fb77c0c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c1623e0 0x7fb77c0c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 -- 172.21.15.113:0/1699578012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c0b6a70 con 0x7fb77c06c9f0 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77affd640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c161770 0x7fb77c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.629+0000 7fb77b7fe640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0c85a0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c400 tx=0x7fb76c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.630+0000 7fb763fff640 1 -- 172.21.15.113:0/1699578012 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c017070 con 0x7fb77c06c9f0 2024-04-25T00:08:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.630+0000 7fb763fff640 1 -- 172.21.15.113:0/1699578012 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c005ce0 con 0x7fb77c06c9f0 2024-04-25T00:08:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.630+0000 7fb763fff640 1 -- 172.21.15.113:0/1699578012 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0072f0 con 0x7fb77c06c9f0 2024-04-25T00:08:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.630+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c0c5c40 con 0x7fb77c06c9f0 2024-04-25T00:08:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.630+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c0c8ae0 con 0x7fb77c06c9f0 2024-04-25T00:08:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.631+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c0691a0 con 0x7fb77c06c9f0 2024-04-25T00:08:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.631+0000 7fb763fff640 1 -- 172.21.15.113:0/1699578012 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb76c01a070 con 0x7fb77c06c9f0 2024-04-25T00:08:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.632+0000 7fb763fff640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb76806be30 0x7fb76806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.632+0000 7fb763fff640 1 -- 172.21.15.113:0/1699578012 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fb76c08a1e0 con 0x7fb77c06c9f0 2024-04-25T00:08:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.632+0000 7fb77affd640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb76806be30 0x7fb76806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.632+0000 7fb77affd640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb76806be30 0x7fb76806e2f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb770034000 tx=0x7fb77002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.636+0000 7fb763fff640 1 -- 172.21.15.113:0/1699578012 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb76c058540 con 0x7fb77c06c9f0 2024-04-25T00:08:17.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.806+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb77c0c6570 con 0x7fb76806be30 2024-04-25T00:08:17.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.812+0000 7fb763fff640 1 -- 172.21.15.113:0/1699578012 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7fb77007fa50 con 0x7fb76806be30 2024-04-25T00:08:17.815 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:17.817 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb76806be30 msgr2=0x7fb76806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb76806be30 0x7fb76806e2f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb770034000 tx=0x7fb77002e000 comp rx=0 tx=0).stop 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77c06c9f0 msgr2=0x7fb77c0c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0c85a0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c400 tx=0x7fb76c00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 shutdown_connections 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb76806be30 0x7fb76806e2f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c1623e0 0x7fb77c0c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb77c161770 0x7fb77c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 --2- 172.21.15.113:0/1699578012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77c06c9f0 0x7fb77c0c85a0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 >> 172.21.15.113:0/1699578012 conn(0x7fb77c0d1ec0 msgr2=0x7fb77c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 shutdown_connections 2024-04-25T00:08:17.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.815+0000 7fb781eda640 1 -- 172.21.15.113:0/1699578012 wait complete. 2024-04-25T00:08:17.827 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-04-25T00:08:17.229683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":254,"num_write_kb":3476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406344,"kb_used_data":16652,"kb_used_omap":12,"kb_used_meta":389555,"kb_avail":754568376,"statfs":{"total":773094113280,"available":772678017024,"internally_reserved":0,"allocated":17051648,"data_stored":8105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":398904880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.546095"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054022+0000","last_change":"2024-04-25T00:08:14.716516+0000","last_active":"2024-04-25T00:08:16.054022+0000","last_peered":"2024-04-25T00:08:16.054022+0000","last_clean":"2024-04-25T00:08:16.054022+0000","last_became_active":"2024-04-25T00:08:14.716162+0000","last_became_peered":"2024-04-25T00:08:14.716162+0000","last_unstale":"2024-04-25T00:08:16.054022+0000","last_undegraded":"2024-04-25T00:08:16.054022+0000","last_fullsized":"2024-04-25T00:08:16.054022+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:01.992891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703184+0000","last_change":"2024-04-25T00:08:14.717856+0000","last_active":"2024-04-25T00:08:15.703184+0000","last_peered":"2024-04-25T00:08:15.703184+0000","last_clean":"2024-04-25T00:08:15.703184+0000","last_became_active":"2024-04-25T00:08:14.717432+0000","last_became_peered":"2024-04-25T00:08:14.717432+0000","last_unstale":"2024-04-25T00:08:15.703184+0000","last_undegraded":"2024-04-25T00:08:15.703184+0000","last_fullsized":"2024-04-25T00:08:15.703184+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:13.514673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053410+0000","last_change":"2024-04-25T00:08:14.712341+0000","last_active":"2024-04-25T00:08:16.053410+0000","last_peered":"2024-04-25T00:08:16.053410+0000","last_clean":"2024-04-25T00:08:16.053410+0000","last_became_active":"2024-04-25T00:08:14.711770+0000","last_became_peered":"2024-04-25T00:08:14.711770+0000","last_unstale":"2024-04-25T00:08:16.053410+0000","last_undegraded":"2024-04-25T00:08:16.053410+0000","last_fullsized":"2024-04-25T00:08:16.053410+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:12.670911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053290+0000","last_change":"2024-04-25T00:08:14.706575+0000","last_active":"2024-04-25T00:08:16.053290+0000","last_peered":"2024-04-25T00:08:16.053290+0000","last_clean":"2024-04-25T00:08:16.053290+0000","last_became_active":"2024-04-25T00:08:14.705115+0000","last_became_peered":"2024-04-25T00:08:14.705115+0000","last_unstale":"2024-04-25T00:08:16.053290+0000","last_undegraded":"2024-04-25T00:08:16.053290+0000","last_fullsized":"2024-04-25T00:08:16.053290+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:54.024198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703023+0000","last_change":"2024-04-25T00:08:14.701773+0000","last_active":"2024-04-25T00:08:15.703023+0000","last_peered":"2024-04-25T00:08:15.703023+0000","last_clean":"2024-04-25T00:08:15.703023+0000","last_became_active":"2024-04-25T00:08:14.701471+0000","last_became_peered":"2024-04-25T00:08:14.701471+0000","last_unstale":"2024-04-25T00:08:15.703023+0000","last_undegraded":"2024-04-25T00:08:15.703023+0000","last_fullsized":"2024-04-25T00:08:15.703023+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:10.005638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702152+0000","last_change":"2024-04-25T00:08:14.710986+0000","last_active":"2024-04-25T00:08:15.702152+0000","last_peered":"2024-04-25T00:08:15.702152+0000","last_clean":"2024-04-25T00:08:15.702152+0000","last_became_active":"2024-04-25T00:08:14.710720+0000","last_became_peered":"2024-04-25T00:08:14.710720+0000","last_unstale":"2024-04-25T00:08:15.702152+0000","last_undegraded":"2024-04-25T00:08:15.702152+0000","last_fullsized":"2024-04-25T00:08:15.702152+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:53.504168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702524+0000","last_change":"2024-04-25T00:08:14.700518+0000","last_active":"2024-04-25T00:08:15.702524+0000","last_peered":"2024-04-25T00:08:15.702524+0000","last_clean":"2024-04-25T00:08:15.702524+0000","last_became_active":"2024-04-25T00:08:14.699700+0000","last_became_peered":"2024-04-25T00:08:14.699700+0000","last_unstale":"2024-04-25T00:08:15.702524+0000","last_undegraded":"2024-04-25T00:08:15.702524+0000","last_fullsized":"2024-04-25T00:08:15.702524+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:21.035999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702665+0000","last_change":"2024-04-25T00:08:14.709383+0000","last_active":"2024-04-25T00:08:15.702665+0000","last_peered":"2024-04-25T00:08:15.702665+0000","last_clean":"2024-04-25T00:08:15.702665+0000","last_became_active":"2024-04-25T00:08:14.709045+0000","last_became_peered":"2024-04-25T00:08:14.709045+0000","last_unstale":"2024-04-25T00:08:15.702665+0000","last_undegraded":"2024-04-25T00:08:15.702665+0000","last_fullsized":"2024-04-25T00:08:15.702665+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:28.769856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703201+0000","last_change":"2024-04-25T00:08:14.717070+0000","last_active":"2024-04-25T00:08:15.703201+0000","last_peered":"2024-04-25T00:08:15.703201+0000","last_clean":"2024-04-25T00:08:15.703201+0000","last_became_active":"2024-04-25T00:08:14.716787+0000","last_became_peered":"2024-04-25T00:08:14.716787+0000","last_unstale":"2024-04-25T00:08:15.703201+0000","last_undegraded":"2024-04-25T00:08:15.703201+0000","last_fullsized":"2024-04-25T00:08:15.703201+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:07.211582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703350+0000","last_change":"2024-04-25T00:08:14.710369+0000","last_active":"2024-04-25T00:08:15.703350+0000","last_peered":"2024-04-25T00:08:15.703350+0000","last_clean":"2024-04-25T00:08:15.703350+0000","last_became_active":"2024-04-25T00:08:14.710078+0000","last_became_peered":"2024-04-25T00:08:14.710078+0000","last_unstale":"2024-04-25T00:08:15.703350+0000","last_undegraded":"2024-04-25T00:08:15.703350+0000","last_fullsized":"2024-04-25T00:08:15.703350+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:06.843612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703203+0000","last_change":"2024-04-25T00:08:14.702387+0000","last_active":"2024-04-25T00:08:15.703203+0000","last_peered":"2024-04-25T00:08:15.703203+0000","last_clean":"2024-04-25T00:08:15.703203+0000","last_became_active":"2024-04-25T00:08:14.702102+0000","last_became_peered":"2024-04-25T00:08:14.702102+0000","last_unstale":"2024-04-25T00:08:15.703203+0000","last_undegraded":"2024-04-25T00:08:15.703203+0000","last_fullsized":"2024-04-25T00:08:15.703203+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:21.846196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702980+0000","last_change":"2024-04-25T00:08:14.718107+0000","last_active":"2024-04-25T00:08:15.702980+0000","last_peered":"2024-04-25T00:08:15.702980+0000","last_clean":"2024-04-25T00:08:15.702980+0000","last_became_active":"2024-04-25T00:08:14.717489+0000","last_became_peered":"2024-04-25T00:08:14.717489+0000","last_unstale":"2024-04-25T00:08:15.702980+0000","last_undegraded":"2024-04-25T00:08:15.702980+0000","last_fullsized":"2024-04-25T00:08:15.702980+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:22.739984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053595+0000","last_change":"2024-04-25T00:08:14.711772+0000","last_active":"2024-04-25T00:08:16.053595+0000","last_peered":"2024-04-25T00:08:16.053595+0000","last_clean":"2024-04-25T00:08:16.053595+0000","last_became_active":"2024-04-25T00:08:14.709256+0000","last_became_peered":"2024-04-25T00:08:14.709256+0000","last_unstale":"2024-04-25T00:08:16.053595+0000","last_undegraded":"2024-04-25T00:08:16.053595+0000","last_fullsized":"2024-04-25T00:08:16.053595+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:22.105245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052859+0000","last_change":"2024-04-25T00:08:14.718375+0000","last_active":"2024-04-25T00:08:16.052859+0000","last_peered":"2024-04-25T00:08:16.052859+0000","last_clean":"2024-04-25T00:08:16.052859+0000","last_became_active":"2024-04-25T00:08:14.718101+0000","last_became_peered":"2024-04-25T00:08:14.718101+0000","last_unstale":"2024-04-25T00:08:16.052859+0000","last_undegraded":"2024-04-25T00:08:16.052859+0000","last_fullsized":"2024-04-25T00:08:16.052859+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:18.531771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053116+0000","last_change":"2024-04-25T00:08:14.713124+0000","last_active":"2024-04-25T00:08:16.053116+0000","last_peered":"2024-04-25T00:08:16.053116+0000","last_clean":"2024-04-25T00:08:16.053116+0000","last_became_active":"2024-04-25T00:08:14.712809+0000","last_became_peered":"2024-04-25T00:08:14.712809+0000","last_unstale":"2024-04-25T00:08:16.053116+0000","last_undegraded":"2024-04-25T00:08:16.053116+0000","last_fullsized":"2024-04-25T00:08:16.053116+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:29.924646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702694+0000","last_change":"2024-04-25T00:08:14.717399+0000","last_active":"2024-04-25T00:08:15.702694+0000","last_peered":"2024-04-25T00:08:15.702694+0000","last_clean":"2024-04-25T00:08:15.702694+0000","last_became_active":"2024-04-25T00:08:14.717012+0000","last_became_peered":"2024-04-25T00:08:14.717012+0000","last_unstale":"2024-04-25T00:08:15.702694+0000","last_undegraded":"2024-04-25T00:08:15.702694+0000","last_fullsized":"2024-04-25T00:08:15.702694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:30.092968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702293+0000","last_change":"2024-04-25T00:08:14.709858+0000","last_active":"2024-04-25T00:08:15.702293+0000","last_peered":"2024-04-25T00:08:15.702293+0000","last_clean":"2024-04-25T00:08:15.702293+0000","last_became_active":"2024-04-25T00:08:14.709588+0000","last_became_peered":"2024-04-25T00:08:14.709588+0000","last_unstale":"2024-04-25T00:08:15.702293+0000","last_undegraded":"2024-04-25T00:08:15.702293+0000","last_fullsized":"2024-04-25T00:08:15.702293+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:12.303548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052762+0000","last_change":"2024-04-25T00:08:14.707255+0000","last_active":"2024-04-25T00:08:16.052762+0000","last_peered":"2024-04-25T00:08:16.052762+0000","last_clean":"2024-04-25T00:08:16.052762+0000","last_became_active":"2024-04-25T00:08:14.706213+0000","last_became_peered":"2024-04-25T00:08:14.706213+0000","last_unstale":"2024-04-25T00:08:16.052762+0000","last_undegraded":"2024-04-25T00:08:16.052762+0000","last_fullsized":"2024-04-25T00:08:16.052762+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:47.626356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702578+0000","last_change":"2024-04-25T00:08:14.711548+0000","last_active":"2024-04-25T00:08:15.702578+0000","last_peered":"2024-04-25T00:08:15.702578+0000","last_clean":"2024-04-25T00:08:15.702578+0000","last_became_active":"2024-04-25T00:08:14.711309+0000","last_became_peered":"2024-04-25T00:08:14.711309+0000","last_unstale":"2024-04-25T00:08:15.702578+0000","last_undegraded":"2024-04-25T00:08:15.702578+0000","last_fullsized":"2024-04-25T00:08:15.702578+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:06.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052836+0000","last_change":"2024-04-25T00:08:14.719499+0000","last_active":"2024-04-25T00:08:16.052836+0000","last_peered":"2024-04-25T00:08:16.052836+0000","last_clean":"2024-04-25T00:08:16.052836+0000","last_became_active":"2024-04-25T00:08:14.719235+0000","last_became_peered":"2024-04-25T00:08:14.719235+0000","last_unstale":"2024-04-25T00:08:16.052836+0000","last_undegraded":"2024-04-25T00:08:16.052836+0000","last_fullsized":"2024-04-25T00:08:16.052836+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:01.498161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702452+0000","last_change":"2024-04-25T00:08:14.710352+0000","last_active":"2024-04-25T00:08:15.702452+0000","last_peered":"2024-04-25T00:08:15.702452+0000","last_clean":"2024-04-25T00:08:15.702452+0000","last_became_active":"2024-04-25T00:08:14.710090+0000","last_became_peered":"2024-04-25T00:08:14.710090+0000","last_unstale":"2024-04-25T00:08:15.702452+0000","last_undegraded":"2024-04-25T00:08:15.702452+0000","last_fullsized":"2024-04-25T00:08:15.702452+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:26.590584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702920+0000","last_change":"2024-04-25T00:08:14.706576+0000","last_active":"2024-04-25T00:08:15.702920+0000","last_peered":"2024-04-25T00:08:15.702920+0000","last_clean":"2024-04-25T00:08:15.702920+0000","last_became_active":"2024-04-25T00:08:14.706295+0000","last_became_peered":"2024-04-25T00:08:14.706295+0000","last_unstale":"2024-04-25T00:08:15.702920+0000","last_undegraded":"2024-04-25T00:08:15.702920+0000","last_fullsized":"2024-04-25T00:08:15.702920+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:08.591028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702405+0000","last_change":"2024-04-25T00:08:14.713136+0000","last_active":"2024-04-25T00:08:15.702405+0000","last_peered":"2024-04-25T00:08:15.702405+0000","last_clean":"2024-04-25T00:08:15.702405+0000","last_became_active":"2024-04-25T00:08:14.712841+0000","last_became_peered":"2024-04-25T00:08:14.712841+0000","last_unstale":"2024-04-25T00:08:15.702405+0000","last_undegraded":"2024-04-25T00:08:15.702405+0000","last_fullsized":"2024-04-25T00:08:15.702405+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:07.011934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702875+0000","last_change":"2024-04-25T00:08:14.705935+0000","last_active":"2024-04-25T00:08:15.702875+0000","last_peered":"2024-04-25T00:08:15.702875+0000","last_clean":"2024-04-25T00:08:15.702875+0000","last_became_active":"2024-04-25T00:08:14.705630+0000","last_became_peered":"2024-04-25T00:08:14.705630+0000","last_unstale":"2024-04-25T00:08:15.702875+0000","last_undegraded":"2024-04-25T00:08:15.702875+0000","last_fullsized":"2024-04-25T00:08:15.702875+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:00.514572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054361+0000","last_change":"2024-04-25T00:08:14.714748+0000","last_active":"2024-04-25T00:08:16.054361+0000","last_peered":"2024-04-25T00:08:16.054361+0000","last_clean":"2024-04-25T00:08:16.054361+0000","last_became_active":"2024-04-25T00:08:14.713364+0000","last_became_peered":"2024-04-25T00:08:14.713364+0000","last_unstale":"2024-04-25T00:08:16.054361+0000","last_undegraded":"2024-04-25T00:08:16.054361+0000","last_fullsized":"2024-04-25T00:08:16.054361+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.499340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702709+0000","last_change":"2024-04-25T00:08:14.705484+0000","last_active":"2024-04-25T00:08:15.702709+0000","last_peered":"2024-04-25T00:08:15.702709+0000","last_clean":"2024-04-25T00:08:15.702709+0000","last_became_active":"2024-04-25T00:08:14.705168+0000","last_became_peered":"2024-04-25T00:08:14.705168+0000","last_unstale":"2024-04-25T00:08:15.702709+0000","last_undegraded":"2024-04-25T00:08:15.702709+0000","last_fullsized":"2024-04-25T00:08:15.702709+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:32.502667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053558+0000","last_change":"2024-04-25T00:08:14.715537+0000","last_active":"2024-04-25T00:08:16.053558+0000","last_peered":"2024-04-25T00:08:16.053558+0000","last_clean":"2024-04-25T00:08:16.053558+0000","last_became_active":"2024-04-25T00:08:14.715269+0000","last_became_peered":"2024-04-25T00:08:14.715269+0000","last_unstale":"2024-04-25T00:08:16.053558+0000","last_undegraded":"2024-04-25T00:08:16.053558+0000","last_fullsized":"2024-04-25T00:08:16.053558+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:41.043191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053436+0000","last_change":"2024-04-25T00:08:14.709225+0000","last_active":"2024-04-25T00:08:16.053436+0000","last_peered":"2024-04-25T00:08:16.053436+0000","last_clean":"2024-04-25T00:08:16.053436+0000","last_became_active":"2024-04-25T00:08:14.708983+0000","last_became_peered":"2024-04-25T00:08:14.708983+0000","last_unstale":"2024-04-25T00:08:16.053436+0000","last_undegraded":"2024-04-25T00:08:16.053436+0000","last_fullsized":"2024-04-25T00:08:16.053436+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.333007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053662+0000","last_change":"2024-04-25T00:08:14.714945+0000","last_active":"2024-04-25T00:08:16.053662+0000","last_peered":"2024-04-25T00:08:16.053662+0000","last_clean":"2024-04-25T00:08:16.053662+0000","last_became_active":"2024-04-25T00:08:14.714354+0000","last_became_peered":"2024-04-25T00:08:14.714354+0000","last_unstale":"2024-04-25T00:08:16.053662+0000","last_undegraded":"2024-04-25T00:08:16.053662+0000","last_fullsized":"2024-04-25T00:08:16.053662+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:58.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053097+0000","last_change":"2024-04-25T00:08:14.713878+0000","last_active":"2024-04-25T00:08:16.053097+0000","last_peered":"2024-04-25T00:08:16.053097+0000","last_clean":"2024-04-25T00:08:16.053097+0000","last_became_active":"2024-04-25T00:08:14.713577+0000","last_became_peered":"2024-04-25T00:08:14.713577+0000","last_unstale":"2024-04-25T00:08:16.053097+0000","last_undegraded":"2024-04-25T00:08:16.053097+0000","last_fullsized":"2024-04-25T00:08:16.053097+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:56.200296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054136+0000","last_change":"2024-04-25T00:08:14.699958+0000","last_active":"2024-04-25T00:08:16.054136+0000","last_peered":"2024-04-25T00:08:16.054136+0000","last_clean":"2024-04-25T00:08:16.054136+0000","last_became_active":"2024-04-25T00:08:14.699659+0000","last_became_peered":"2024-04-25T00:08:14.699659+0000","last_unstale":"2024-04-25T00:08:16.054136+0000","last_undegraded":"2024-04-25T00:08:16.054136+0000","last_fullsized":"2024-04-25T00:08:16.054136+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:42.041767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052911+0000","last_change":"2024-04-25T00:08:14.701530+0000","last_active":"2024-04-25T00:08:16.052911+0000","last_peered":"2024-04-25T00:08:16.052911+0000","last_clean":"2024-04-25T00:08:16.052911+0000","last_became_active":"2024-04-25T00:08:14.701271+0000","last_became_peered":"2024-04-25T00:08:14.701271+0000","last_unstale":"2024-04-25T00:08:16.052911+0000","last_undegraded":"2024-04-25T00:08:16.052911+0000","last_fullsized":"2024-04-25T00:08:16.052911+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:40.670725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702673+0000","last_change":"2024-04-25T00:08:14.706577+0000","last_active":"2024-04-25T00:08:15.702673+0000","last_peered":"2024-04-25T00:08:15.702673+0000","last_clean":"2024-04-25T00:08:15.702673+0000","last_became_active":"2024-04-25T00:08:14.704713+0000","last_became_peered":"2024-04-25T00:08:14.704713+0000","last_unstale":"2024-04-25T00:08:15.702673+0000","last_undegraded":"2024-04-25T00:08:15.702673+0000","last_fullsized":"2024-04-25T00:08:15.702673+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:57.507086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"157'4","reported_seq":18,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054134+0000","last_change":"2024-04-25T00:08:13.707841+0000","last_active":"2024-04-25T00:08:16.054134+0000","last_peered":"2024-04-25T00:08:16.054134+0000","last_clean":"2024-04-25T00:08:16.054134+0000","last_became_active":"2024-04-25T00:08:13.707571+0000","last_became_peered":"2024-04-25T00:08:13.707571+0000","last_unstale":"2024-04-25T00:08:16.054134+0000","last_undegraded":"2024-04-25T00:08:16.054134+0000","last_fullsized":"2024-04-25T00:08:16.054134+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:21.502717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053262+0000","last_change":"2024-04-25T00:08:14.701146+0000","last_active":"2024-04-25T00:08:16.053262+0000","last_peered":"2024-04-25T00:08:16.053262+0000","last_clean":"2024-04-25T00:08:16.053262+0000","last_became_active":"2024-04-25T00:08:14.700595+0000","last_became_peered":"2024-04-25T00:08:14.700595+0000","last_unstale":"2024-04-25T00:08:16.053262+0000","last_undegraded":"2024-04-25T00:08:16.053262+0000","last_fullsized":"2024-04-25T00:08:16.053262+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:54.226601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054055+0000","last_change":"2024-04-25T00:08:13.704657+0000","last_active":"2024-04-25T00:08:16.054055+0000","last_peered":"2024-04-25T00:08:16.054055+0000","last_clean":"2024-04-25T00:08:16.054055+0000","last_became_active":"2024-04-25T00:08:13.704376+0000","last_became_peered":"2024-04-25T00:08:13.704376+0000","last_unstale":"2024-04-25T00:08:16.054055+0000","last_undegraded":"2024-04-25T00:08:16.054055+0000","last_fullsized":"2024-04-25T00:08:16.054055+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:59.509497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702151+0000","last_change":"2024-04-25T00:08:14.716933+0000","last_active":"2024-04-25T00:08:15.702151+0000","last_peered":"2024-04-25T00:08:15.702151+0000","last_clean":"2024-04-25T00:08:15.702151+0000","last_became_active":"2024-04-25T00:08:14.716650+0000","last_became_peered":"2024-04-25T00:08:14.716650+0000","last_unstale":"2024-04-25T00:08:15.702151+0000","last_undegraded":"2024-04-25T00:08:15.702151+0000","last_fullsized":"2024-04-25T00:08:15.702151+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:39.190958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053311+0000","last_change":"2024-04-25T00:08:13.705148+0000","last_active":"2024-04-25T00:08:16.053311+0000","last_peered":"2024-04-25T00:08:16.053311+0000","last_clean":"2024-04-25T00:08:16.053311+0000","last_became_active":"2024-04-25T00:08:13.704884+0000","last_became_peered":"2024-04-25T00:08:13.704884+0000","last_unstale":"2024-04-25T00:08:16.053311+0000","last_undegraded":"2024-04-25T00:08:16.053311+0000","last_fullsized":"2024-04-25T00:08:16.053311+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:46.785370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053793+0000","last_change":"2024-04-25T00:08:14.714525+0000","last_active":"2024-04-25T00:08:16.053793+0000","last_peered":"2024-04-25T00:08:16.053793+0000","last_clean":"2024-04-25T00:08:16.053793+0000","last_became_active":"2024-04-25T00:08:14.713431+0000","last_became_peered":"2024-04-25T00:08:14.713431+0000","last_unstale":"2024-04-25T00:08:16.053793+0000","last_undegraded":"2024-04-25T00:08:16.053793+0000","last_fullsized":"2024-04-25T00:08:16.053793+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:54.024198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053543+0000","last_change":"2024-04-25T00:08:13.699351+0000","last_active":"2024-04-25T00:08:16.053543+0000","last_peered":"2024-04-25T00:08:16.053543+0000","last_clean":"2024-04-25T00:08:16.053543+0000","last_became_active":"2024-04-25T00:08:13.699077+0000","last_became_peered":"2024-04-25T00:08:13.699077+0000","last_unstale":"2024-04-25T00:08:16.053543+0000","last_undegraded":"2024-04-25T00:08:16.053543+0000","last_fullsized":"2024-04-25T00:08:16.053543+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:09.884267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702449+0000","last_change":"2024-04-25T00:08:14.701395+0000","last_active":"2024-04-25T00:08:15.702449+0000","last_peered":"2024-04-25T00:08:15.702449+0000","last_clean":"2024-04-25T00:08:15.702449+0000","last_became_active":"2024-04-25T00:08:14.699111+0000","last_became_peered":"2024-04-25T00:08:14.699111+0000","last_unstale":"2024-04-25T00:08:15.702449+0000","last_undegraded":"2024-04-25T00:08:15.702449+0000","last_fullsized":"2024-04-25T00:08:15.702449+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.675509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053785+0000","last_change":"2024-04-25T00:08:13.706500+0000","last_active":"2024-04-25T00:08:16.053785+0000","last_peered":"2024-04-25T00:08:16.053785+0000","last_clean":"2024-04-25T00:08:16.053785+0000","last_became_active":"2024-04-25T00:08:13.706178+0000","last_became_peered":"2024-04-25T00:08:13.706178+0000","last_unstale":"2024-04-25T00:08:16.053785+0000","last_undegraded":"2024-04-25T00:08:16.053785+0000","last_fullsized":"2024-04-25T00:08:16.053785+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:45.927693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053328+0000","last_change":"2024-04-25T00:08:14.706886+0000","last_active":"2024-04-25T00:08:16.053328+0000","last_peered":"2024-04-25T00:08:16.053328+0000","last_clean":"2024-04-25T00:08:16.053328+0000","last_became_active":"2024-04-25T00:08:14.705156+0000","last_became_peered":"2024-04-25T00:08:14.705156+0000","last_unstale":"2024-04-25T00:08:16.053328+0000","last_undegraded":"2024-04-25T00:08:16.053328+0000","last_fullsized":"2024-04-25T00:08:16.053328+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.499340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053081+0000","last_change":"2024-04-25T00:08:13.698272+0000","last_active":"2024-04-25T00:08:16.053081+0000","last_peered":"2024-04-25T00:08:16.053081+0000","last_clean":"2024-04-25T00:08:16.053081+0000","last_became_active":"2024-04-25T00:08:13.697992+0000","last_became_peered":"2024-04-25T00:08:13.697992+0000","last_unstale":"2024-04-25T00:08:16.053081+0000","last_undegraded":"2024-04-25T00:08:16.053081+0000","last_fullsized":"2024-04-25T00:08:16.053081+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:25.585509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052671+0000","last_change":"2024-04-25T00:08:14.717854+0000","last_active":"2024-04-25T00:08:16.052671+0000","last_peered":"2024-04-25T00:08:16.052671+0000","last_clean":"2024-04-25T00:08:16.052671+0000","last_became_active":"2024-04-25T00:08:14.717547+0000","last_became_peered":"2024-04-25T00:08:14.717547+0000","last_unstale":"2024-04-25T00:08:16.052671+0000","last_undegraded":"2024-04-25T00:08:16.052671+0000","last_fullsized":"2024-04-25T00:08:16.052671+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:22.105245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702682+0000","last_change":"2024-04-25T00:08:13.702223+0000","last_active":"2024-04-25T00:08:15.702682+0000","last_peered":"2024-04-25T00:08:15.702682+0000","last_clean":"2024-04-25T00:08:15.702682+0000","last_became_active":"2024-04-25T00:08:13.701905+0000","last_became_peered":"2024-04-25T00:08:13.701905+0000","last_unstale":"2024-04-25T00:08:15.702682+0000","last_undegraded":"2024-04-25T00:08:15.702682+0000","last_fullsized":"2024-04-25T00:08:15.702682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:11.665836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702177+0000","last_change":"2024-04-25T00:08:14.707467+0000","last_active":"2024-04-25T00:08:15.702177+0000","last_peered":"2024-04-25T00:08:15.702177+0000","last_clean":"2024-04-25T00:08:15.702177+0000","last_became_active":"2024-04-25T00:08:14.707211+0000","last_became_peered":"2024-04-25T00:08:14.707211+0000","last_unstale":"2024-04-25T00:08:15.702177+0000","last_undegraded":"2024-04-25T00:08:15.702177+0000","last_fullsized":"2024-04-25T00:08:15.702177+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:23.174081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702786+0000","last_change":"2024-04-25T00:08:13.702235+0000","last_active":"2024-04-25T00:08:15.702786+0000","last_peered":"2024-04-25T00:08:15.702786+0000","last_clean":"2024-04-25T00:08:15.702786+0000","last_became_active":"2024-04-25T00:08:13.701744+0000","last_became_peered":"2024-04-25T00:08:13.701744+0000","last_unstale":"2024-04-25T00:08:15.702786+0000","last_undegraded":"2024-04-25T00:08:15.702786+0000","last_fullsized":"2024-04-25T00:08:15.702786+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:18.545568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053059+0000","last_change":"2024-04-25T00:08:14.699053+0000","last_active":"2024-04-25T00:08:16.053059+0000","last_peered":"2024-04-25T00:08:16.053059+0000","last_clean":"2024-04-25T00:08:16.053059+0000","last_became_active":"2024-04-25T00:08:14.698733+0000","last_became_peered":"2024-04-25T00:08:14.698733+0000","last_unstale":"2024-04-25T00:08:16.053059+0000","last_undegraded":"2024-04-25T00:08:16.053059+0000","last_fullsized":"2024-04-25T00:08:16.053059+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:22.507792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702449+0000","last_change":"2024-04-25T00:08:13.709325+0000","last_active":"2024-04-25T00:08:15.702449+0000","last_peered":"2024-04-25T00:08:15.702449+0000","last_clean":"2024-04-25T00:08:15.702449+0000","last_became_active":"2024-04-25T00:08:13.709050+0000","last_became_peered":"2024-04-25T00:08:13.709050+0000","last_unstale":"2024-04-25T00:08:15.702449+0000","last_undegraded":"2024-04-25T00:08:15.702449+0000","last_fullsized":"2024-04-25T00:08:15.702449+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:06.072795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052592+0000","last_change":"2024-04-25T00:08:14.699287+0000","last_active":"2024-04-25T00:08:16.052592+0000","last_peered":"2024-04-25T00:08:16.052592+0000","last_clean":"2024-04-25T00:08:16.052592+0000","last_became_active":"2024-04-25T00:08:14.699022+0000","last_became_peered":"2024-04-25T00:08:14.699022+0000","last_unstale":"2024-04-25T00:08:16.052592+0000","last_undegraded":"2024-04-25T00:08:16.052592+0000","last_fullsized":"2024-04-25T00:08:16.052592+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:18.538560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"157'3","reported_seq":17,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702563+0000","last_change":"2024-04-25T00:08:13.708197+0000","last_active":"2024-04-25T00:08:15.702563+0000","last_peered":"2024-04-25T00:08:15.702563+0000","last_clean":"2024-04-25T00:08:15.702563+0000","last_became_active":"2024-04-25T00:08:13.707893+0000","last_became_peered":"2024-04-25T00:08:13.707893+0000","last_unstale":"2024-04-25T00:08:15.702563+0000","last_undegraded":"2024-04-25T00:08:15.702563+0000","last_fullsized":"2024-04-25T00:08:15.702563+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:42.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053635+0000","last_change":"2024-04-25T00:08:14.714483+0000","last_active":"2024-04-25T00:08:16.053635+0000","last_peered":"2024-04-25T00:08:16.053635+0000","last_clean":"2024-04-25T00:08:16.053635+0000","last_became_active":"2024-04-25T00:08:14.714224+0000","last_became_peered":"2024-04-25T00:08:14.714224+0000","last_unstale":"2024-04-25T00:08:16.053635+0000","last_undegraded":"2024-04-25T00:08:16.053635+0000","last_fullsized":"2024-04-25T00:08:16.053635+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.555958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"157'6","reported_seq":20,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053970+0000","last_change":"2024-04-25T00:08:13.699685+0000","last_active":"2024-04-25T00:08:16.053970+0000","last_peered":"2024-04-25T00:08:16.053970+0000","last_clean":"2024-04-25T00:08:16.053970+0000","last_became_active":"2024-04-25T00:08:13.698917+0000","last_became_peered":"2024-04-25T00:08:13.698917+0000","last_unstale":"2024-04-25T00:08:16.053970+0000","last_undegraded":"2024-04-25T00:08:16.053970+0000","last_fullsized":"2024-04-25T00:08:16.053970+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:56.226946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052597+0000","last_change":"2024-04-25T00:08:14.701982+0000","last_active":"2024-04-25T00:08:16.052597+0000","last_peered":"2024-04-25T00:08:16.052597+0000","last_clean":"2024-04-25T00:08:16.052597+0000","last_became_active":"2024-04-25T00:08:14.701511+0000","last_became_peered":"2024-04-25T00:08:14.701511+0000","last_unstale":"2024-04-25T00:08:16.052597+0000","last_undegraded":"2024-04-25T00:08:16.052597+0000","last_fullsized":"2024-04-25T00:08:16.052597+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:07.285153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053279+0000","last_change":"2024-04-25T00:08:13.701056+0000","last_active":"2024-04-25T00:08:16.053279+0000","last_peered":"2024-04-25T00:08:16.053279+0000","last_clean":"2024-04-25T00:08:16.053279+0000","last_became_active":"2024-04-25T00:08:13.700615+0000","last_became_peered":"2024-04-25T00:08:13.700615+0000","last_unstale":"2024-04-25T00:08:16.053279+0000","last_undegraded":"2024-04-25T00:08:16.053279+0000","last_fullsized":"2024-04-25T00:08:16.053279+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:02.034623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702769+0000","last_change":"2024-04-25T00:08:14.709839+0000","last_active":"2024-04-25T00:08:15.702769+0000","last_peered":"2024-04-25T00:08:15.702769+0000","last_clean":"2024-04-25T00:08:15.702769+0000","last_became_active":"2024-04-25T00:08:14.709287+0000","last_became_peered":"2024-04-25T00:08:14.709287+0000","last_unstale":"2024-04-25T00:08:15.702769+0000","last_undegraded":"2024-04-25T00:08:15.702769+0000","last_fullsized":"2024-04-25T00:08:15.702769+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:26.031254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052702+0000","last_change":"2024-04-25T00:08:13.702677+0000","last_active":"2024-04-25T00:08:16.052702+0000","last_peered":"2024-04-25T00:08:16.052702+0000","last_clean":"2024-04-25T00:08:16.052702+0000","last_became_active":"2024-04-25T00:08:13.702397+0000","last_became_peered":"2024-04-25T00:08:13.702397+0000","last_unstale":"2024-04-25T00:08:16.052702+0000","last_undegraded":"2024-04-25T00:08:16.052702+0000","last_fullsized":"2024-04-25T00:08:16.052702+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:06.206506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052839+0000","last_change":"2024-04-25T00:08:14.705759+0000","last_active":"2024-04-25T00:08:16.052839+0000","last_peered":"2024-04-25T00:08:16.052839+0000","last_clean":"2024-04-25T00:08:16.052839+0000","last_became_active":"2024-04-25T00:08:14.705512+0000","last_became_peered":"2024-04-25T00:08:14.705512+0000","last_unstale":"2024-04-25T00:08:16.052839+0000","last_undegraded":"2024-04-25T00:08:16.052839+0000","last_fullsized":"2024-04-25T00:08:16.052839+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:58.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052981+0000","last_change":"2024-04-25T00:08:13.704877+0000","last_active":"2024-04-25T00:08:16.052981+0000","last_peered":"2024-04-25T00:08:16.052981+0000","last_clean":"2024-04-25T00:08:16.052981+0000","last_became_active":"2024-04-25T00:08:13.704587+0000","last_became_peered":"2024-04-25T00:08:13.704587+0000","last_unstale":"2024-04-25T00:08:16.052981+0000","last_undegraded":"2024-04-25T00:08:16.052981+0000","last_fullsized":"2024-04-25T00:08:16.052981+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:24.518477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053114+0000","last_change":"2024-04-25T00:08:14.709756+0000","last_active":"2024-04-25T00:08:16.053114+0000","last_peered":"2024-04-25T00:08:16.053114+0000","last_clean":"2024-04-25T00:08:16.053114+0000","last_became_active":"2024-04-25T00:08:14.709467+0000","last_became_peered":"2024-04-25T00:08:14.709467+0000","last_unstale":"2024-04-25T00:08:16.053114+0000","last_undegraded":"2024-04-25T00:08:16.053114+0000","last_fullsized":"2024-04-25T00:08:16.053114+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:11.566204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"157'8","reported_seq":22,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053207+0000","last_change":"2024-04-25T00:08:13.700545+0000","last_active":"2024-04-25T00:08:16.053207+0000","last_peered":"2024-04-25T00:08:16.053207+0000","last_clean":"2024-04-25T00:08:16.053207+0000","last_became_active":"2024-04-25T00:08:13.698906+0000","last_became_peered":"2024-04-25T00:08:13.698906+0000","last_unstale":"2024-04-25T00:08:16.053207+0000","last_undegraded":"2024-04-25T00:08:16.053207+0000","last_fullsized":"2024-04-25T00:08:16.053207+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:21.734909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053580+0000","last_change":"2024-04-25T00:08:14.704282+0000","last_active":"2024-04-25T00:08:16.053580+0000","last_peered":"2024-04-25T00:08:16.053580+0000","last_clean":"2024-04-25T00:08:16.053580+0000","last_became_active":"2024-04-25T00:08:14.704009+0000","last_became_peered":"2024-04-25T00:08:14.704009+0000","last_unstale":"2024-04-25T00:08:16.053580+0000","last_undegraded":"2024-04-25T00:08:16.053580+0000","last_fullsized":"2024-04-25T00:08:16.053580+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:55.520083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054570+0000","last_change":"2024-04-25T00:08:13.707307+0000","last_active":"2024-04-25T00:08:16.054570+0000","last_peered":"2024-04-25T00:08:16.054570+0000","last_clean":"2024-04-25T00:08:16.054570+0000","last_became_active":"2024-04-25T00:08:13.706889+0000","last_became_peered":"2024-04-25T00:08:13.706889+0000","last_unstale":"2024-04-25T00:08:16.054570+0000","last_undegraded":"2024-04-25T00:08:16.054570+0000","last_fullsized":"2024-04-25T00:08:16.054570+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:52.499092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053078+0000","last_change":"2024-04-25T00:08:14.697118+0000","last_active":"2024-04-25T00:08:16.053078+0000","last_peered":"2024-04-25T00:08:16.053078+0000","last_clean":"2024-04-25T00:08:16.053078+0000","last_became_active":"2024-04-25T00:08:14.696837+0000","last_became_peered":"2024-04-25T00:08:14.696837+0000","last_unstale":"2024-04-25T00:08:16.053078+0000","last_undegraded":"2024-04-25T00:08:16.053078+0000","last_fullsized":"2024-04-25T00:08:16.053078+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.675509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053629+0000","last_change":"2024-04-25T00:08:13.702265+0000","last_active":"2024-04-25T00:08:16.053629+0000","last_peered":"2024-04-25T00:08:16.053629+0000","last_clean":"2024-04-25T00:08:16.053629+0000","last_became_active":"2024-04-25T00:08:13.701970+0000","last_became_peered":"2024-04-25T00:08:13.701970+0000","last_unstale":"2024-04-25T00:08:16.053629+0000","last_undegraded":"2024-04-25T00:08:16.053629+0000","last_fullsized":"2024-04-25T00:08:16.053629+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:29.563404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053517+0000","last_change":"2024-04-25T00:08:14.706118+0000","last_active":"2024-04-25T00:08:16.053517+0000","last_peered":"2024-04-25T00:08:16.053517+0000","last_clean":"2024-04-25T00:08:16.053517+0000","last_became_active":"2024-04-25T00:08:14.705368+0000","last_became_peered":"2024-04-25T00:08:14.705368+0000","last_unstale":"2024-04-25T00:08:16.053517+0000","last_undegraded":"2024-04-25T00:08:16.053517+0000","last_fullsized":"2024-04-25T00:08:16.053517+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:03.543592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054107+0000","last_change":"2024-04-25T00:08:13.702965+0000","last_active":"2024-04-25T00:08:16.054107+0000","last_peered":"2024-04-25T00:08:16.054107+0000","last_clean":"2024-04-25T00:08:16.054107+0000","last_became_active":"2024-04-25T00:08:13.702084+0000","last_became_peered":"2024-04-25T00:08:13.702084+0000","last_unstale":"2024-04-25T00:08:16.054107+0000","last_undegraded":"2024-04-25T00:08:16.054107+0000","last_fullsized":"2024-04-25T00:08:16.054107+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:09.670434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702421+0000","last_change":"2024-04-25T00:08:14.699402+0000","last_active":"2024-04-25T00:08:15.702421+0000","last_peered":"2024-04-25T00:08:15.702421+0000","last_clean":"2024-04-25T00:08:15.702421+0000","last_became_active":"2024-04-25T00:08:14.699149+0000","last_became_peered":"2024-04-25T00:08:14.699149+0000","last_unstale":"2024-04-25T00:08:15.702421+0000","last_undegraded":"2024-04-25T00:08:15.702421+0000","last_fullsized":"2024-04-25T00:08:15.702421+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:22.507792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053744+0000","last_change":"2024-04-25T00:08:13.695341+0000","last_active":"2024-04-25T00:08:16.053744+0000","last_peered":"2024-04-25T00:08:16.053744+0000","last_clean":"2024-04-25T00:08:16.053744+0000","last_became_active":"2024-04-25T00:08:13.695009+0000","last_became_peered":"2024-04-25T00:08:13.695009+0000","last_unstale":"2024-04-25T00:08:16.053744+0000","last_undegraded":"2024-04-25T00:08:16.053744+0000","last_fullsized":"2024-04-25T00:08:16.053744+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:20.030924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053042+0000","last_change":"2024-04-25T00:08:14.714235+0000","last_active":"2024-04-25T00:08:16.053042+0000","last_peered":"2024-04-25T00:08:16.053042+0000","last_clean":"2024-04-25T00:08:16.053042+0000","last_became_active":"2024-04-25T00:08:14.711984+0000","last_became_peered":"2024-04-25T00:08:14.711984+0000","last_unstale":"2024-04-25T00:08:16.053042+0000","last_undegraded":"2024-04-25T00:08:16.053042+0000","last_fullsized":"2024-04-25T00:08:16.053042+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:39.826995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053371+0000","last_change":"2024-04-25T00:08:13.704841+0000","last_active":"2024-04-25T00:08:16.053371+0000","last_peered":"2024-04-25T00:08:16.053371+0000","last_clean":"2024-04-25T00:08:16.053371+0000","last_became_active":"2024-04-25T00:08:13.704550+0000","last_became_peered":"2024-04-25T00:08:13.704550+0000","last_unstale":"2024-04-25T00:08:16.053371+0000","last_undegraded":"2024-04-25T00:08:16.053371+0000","last_fullsized":"2024-04-25T00:08:16.053371+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:25.752983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702756+0000","last_change":"2024-04-25T00:08:14.717098+0000","last_active":"2024-04-25T00:08:15.702756+0000","last_peered":"2024-04-25T00:08:15.702756+0000","last_clean":"2024-04-25T00:08:15.702756+0000","last_became_active":"2024-04-25T00:08:14.716806+0000","last_became_peered":"2024-04-25T00:08:14.716806+0000","last_unstale":"2024-04-25T00:08:15.702756+0000","last_undegraded":"2024-04-25T00:08:15.702756+0000","last_fullsized":"2024-04-25T00:08:15.702756+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:29.924646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053005+0000","last_change":"2024-04-25T00:08:13.704231+0000","last_active":"2024-04-25T00:08:16.053005+0000","last_peered":"2024-04-25T00:08:16.053005+0000","last_clean":"2024-04-25T00:08:16.053005+0000","last_became_active":"2024-04-25T00:08:13.702037+0000","last_became_peered":"2024-04-25T00:08:13.702037+0000","last_unstale":"2024-04-25T00:08:16.053005+0000","last_undegraded":"2024-04-25T00:08:16.053005+0000","last_fullsized":"2024-04-25T00:08:16.053005+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:36.481898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053601+0000","last_change":"2024-04-25T00:08:14.717219+0000","last_active":"2024-04-25T00:08:16.053601+0000","last_peered":"2024-04-25T00:08:16.053601+0000","last_clean":"2024-04-25T00:08:16.053601+0000","last_became_active":"2024-04-25T00:08:14.716942+0000","last_became_peered":"2024-04-25T00:08:14.716942+0000","last_unstale":"2024-04-25T00:08:16.053601+0000","last_undegraded":"2024-04-25T00:08:16.053601+0000","last_fullsized":"2024-04-25T00:08:16.053601+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:26.031254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053074+0000","last_change":"2024-04-25T00:08:13.706523+0000","last_active":"2024-04-25T00:08:16.053074+0000","last_peered":"2024-04-25T00:08:16.053074+0000","last_clean":"2024-04-25T00:08:16.053074+0000","last_became_active":"2024-04-25T00:08:13.706264+0000","last_became_peered":"2024-04-25T00:08:13.706264+0000","last_unstale":"2024-04-25T00:08:16.053074+0000","last_undegraded":"2024-04-25T00:08:16.053074+0000","last_fullsized":"2024-04-25T00:08:16.053074+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:06.006859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702329+0000","last_change":"2024-04-25T00:08:14.703062+0000","last_active":"2024-04-25T00:08:15.702329+0000","last_peered":"2024-04-25T00:08:15.702329+0000","last_clean":"2024-04-25T00:08:15.702329+0000","last_became_active":"2024-04-25T00:08:14.702785+0000","last_became_peered":"2024-04-25T00:08:14.702785+0000","last_unstale":"2024-04-25T00:08:15.702329+0000","last_undegraded":"2024-04-25T00:08:15.702329+0000","last_fullsized":"2024-04-25T00:08:15.702329+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:25.523552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053707+0000","last_change":"2024-04-25T00:08:13.704556+0000","last_active":"2024-04-25T00:08:16.053707+0000","last_peered":"2024-04-25T00:08:16.053707+0000","last_clean":"2024-04-25T00:08:16.053707+0000","last_became_active":"2024-04-25T00:08:13.704237+0000","last_became_peered":"2024-04-25T00:08:13.704237+0000","last_unstale":"2024-04-25T00:08:16.053707+0000","last_undegraded":"2024-04-25T00:08:16.053707+0000","last_fullsized":"2024-04-25T00:08:16.053707+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:36.589578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052958+0000","last_change":"2024-04-25T00:08:14.705549+0000","last_active":"2024-04-25T00:08:16.052958+0000","last_peered":"2024-04-25T00:08:16.052958+0000","last_clean":"2024-04-25T00:08:16.052958+0000","last_became_active":"2024-04-25T00:08:14.704198+0000","last_became_peered":"2024-04-25T00:08:14.704198+0000","last_unstale":"2024-04-25T00:08:16.052958+0000","last_undegraded":"2024-04-25T00:08:16.052958+0000","last_fullsized":"2024-04-25T00:08:16.052958+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:37.486973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053159+0000","last_change":"2024-04-25T00:08:13.703254+0000","last_active":"2024-04-25T00:08:16.053159+0000","last_peered":"2024-04-25T00:08:16.053159+0000","last_clean":"2024-04-25T00:08:16.053159+0000","last_became_active":"2024-04-25T00:08:13.702987+0000","last_became_peered":"2024-04-25T00:08:13.702987+0000","last_unstale":"2024-04-25T00:08:16.053159+0000","last_undegraded":"2024-04-25T00:08:16.053159+0000","last_fullsized":"2024-04-25T00:08:16.053159+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:20.841121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702577+0000","last_change":"2024-04-25T00:08:14.701092+0000","last_active":"2024-04-25T00:08:15.702577+0000","last_peered":"2024-04-25T00:08:15.702577+0000","last_clean":"2024-04-25T00:08:15.702577+0000","last_became_active":"2024-04-25T00:08:14.700825+0000","last_became_peered":"2024-04-25T00:08:14.700825+0000","last_unstale":"2024-04-25T00:08:15.702577+0000","last_undegraded":"2024-04-25T00:08:15.702577+0000","last_fullsized":"2024-04-25T00:08:15.702577+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:37.594653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053759+0000","last_change":"2024-04-25T00:08:13.705008+0000","last_active":"2024-04-25T00:08:16.053759+0000","last_peered":"2024-04-25T00:08:16.053759+0000","last_clean":"2024-04-25T00:08:16.053759+0000","last_became_active":"2024-04-25T00:08:13.704758+0000","last_became_peered":"2024-04-25T00:08:13.704758+0000","last_unstale":"2024-04-25T00:08:16.053759+0000","last_undegraded":"2024-04-25T00:08:16.053759+0000","last_fullsized":"2024-04-25T00:08:16.053759+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:38.185882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702284+0000","last_change":"2024-04-25T00:08:14.711078+0000","last_active":"2024-04-25T00:08:15.702284+0000","last_peered":"2024-04-25T00:08:15.702284+0000","last_clean":"2024-04-25T00:08:15.702284+0000","last_became_active":"2024-04-25T00:08:14.710773+0000","last_became_peered":"2024-04-25T00:08:14.710773+0000","last_unstale":"2024-04-25T00:08:15.702284+0000","last_undegraded":"2024-04-25T00:08:15.702284+0000","last_fullsized":"2024-04-25T00:08:15.702284+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:04.457684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703111+0000","last_change":"2024-04-25T00:08:13.703390+0000","last_active":"2024-04-25T00:08:15.703111+0000","last_peered":"2024-04-25T00:08:15.703111+0000","last_clean":"2024-04-25T00:08:15.703111+0000","last_became_active":"2024-04-25T00:08:13.703022+0000","last_became_peered":"2024-04-25T00:08:13.703022+0000","last_unstale":"2024-04-25T00:08:15.703111+0000","last_undegraded":"2024-04-25T00:08:15.703111+0000","last_fullsized":"2024-04-25T00:08:15.703111+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:27.705314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052720+0000","last_change":"2024-04-25T00:08:14.716945+0000","last_active":"2024-04-25T00:08:16.052720+0000","last_peered":"2024-04-25T00:08:16.052720+0000","last_clean":"2024-04-25T00:08:16.052720+0000","last_became_active":"2024-04-25T00:08:14.716660+0000","last_became_peered":"2024-04-25T00:08:14.716660+0000","last_unstale":"2024-04-25T00:08:16.052720+0000","last_undegraded":"2024-04-25T00:08:16.052720+0000","last_fullsized":"2024-04-25T00:08:16.052720+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:03.079423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052732+0000","last_change":"2024-04-25T00:08:13.706632+0000","last_active":"2024-04-25T00:08:16.052732+0000","last_peered":"2024-04-25T00:08:16.052732+0000","last_clean":"2024-04-25T00:08:16.052732+0000","last_became_active":"2024-04-25T00:08:13.706307+0000","last_became_peered":"2024-04-25T00:08:13.706307+0000","last_unstale":"2024-04-25T00:08:16.052732+0000","last_undegraded":"2024-04-25T00:08:16.052732+0000","last_fullsized":"2024-04-25T00:08:16.052732+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:05.838537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052612+0000","last_change":"2024-04-25T00:08:14.701762+0000","last_active":"2024-04-25T00:08:16.052612+0000","last_peered":"2024-04-25T00:08:16.052612+0000","last_clean":"2024-04-25T00:08:16.052612+0000","last_became_active":"2024-04-25T00:08:14.701168+0000","last_became_peered":"2024-04-25T00:08:14.701168+0000","last_unstale":"2024-04-25T00:08:16.052612+0000","last_undegraded":"2024-04-25T00:08:16.052612+0000","last_fullsized":"2024-04-25T00:08:16.052612+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:55.854379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054126+0000","last_change":"2024-04-25T00:08:13.702241+0000","last_active":"2024-04-25T00:08:16.054126+0000","last_peered":"2024-04-25T00:08:16.054126+0000","last_clean":"2024-04-25T00:08:16.054126+0000","last_became_active":"2024-04-25T00:08:13.700411+0000","last_became_peered":"2024-04-25T00:08:13.700411+0000","last_unstale":"2024-04-25T00:08:16.054126+0000","last_undegraded":"2024-04-25T00:08:16.054126+0000","last_fullsized":"2024-04-25T00:08:16.054126+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:07.585953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054107+0000","last_change":"2024-04-25T00:08:14.706523+0000","last_active":"2024-04-25T00:08:16.054107+0000","last_peered":"2024-04-25T00:08:16.054107+0000","last_clean":"2024-04-25T00:08:16.054107+0000","last_became_active":"2024-04-25T00:08:14.706185+0000","last_became_peered":"2024-04-25T00:08:14.706185+0000","last_unstale":"2024-04-25T00:08:16.054107+0000","last_undegraded":"2024-04-25T00:08:16.054107+0000","last_fullsized":"2024-04-25T00:08:16.054107+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:53.406643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053091+0000","last_change":"2024-04-25T00:08:13.703855+0000","last_active":"2024-04-25T00:08:16.053091+0000","last_peered":"2024-04-25T00:08:16.053091+0000","last_clean":"2024-04-25T00:08:16.053091+0000","last_became_active":"2024-04-25T00:08:13.703589+0000","last_became_peered":"2024-04-25T00:08:13.703589+0000","last_unstale":"2024-04-25T00:08:16.053091+0000","last_undegraded":"2024-04-25T00:08:16.053091+0000","last_fullsized":"2024-04-25T00:08:16.053091+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:05.312619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702736+0000","last_change":"2024-04-25T00:08:14.717759+0000","last_active":"2024-04-25T00:08:15.702736+0000","last_peered":"2024-04-25T00:08:15.702736+0000","last_clean":"2024-04-25T00:08:15.702736+0000","last_became_active":"2024-04-25T00:08:14.716906+0000","last_became_peered":"2024-04-25T00:08:14.716906+0000","last_unstale":"2024-04-25T00:08:15.702736+0000","last_undegraded":"2024-04-25T00:08:15.702736+0000","last_fullsized":"2024-04-25T00:08:15.702736+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:53.829710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702959+0000","last_change":"2024-04-25T00:08:13.703232+0000","last_active":"2024-04-25T00:08:15.702959+0000","last_peered":"2024-04-25T00:08:15.702959+0000","last_clean":"2024-04-25T00:08:15.702959+0000","last_became_active":"2024-04-25T00:08:13.702930+0000","last_became_peered":"2024-04-25T00:08:13.702930+0000","last_unstale":"2024-04-25T00:08:15.702959+0000","last_undegraded":"2024-04-25T00:08:15.702959+0000","last_fullsized":"2024-04-25T00:08:15.702959+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:40.038116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053268+0000","last_change":"2024-04-25T00:08:14.709152+0000","last_active":"2024-04-25T00:08:16.053268+0000","last_peered":"2024-04-25T00:08:16.053268+0000","last_clean":"2024-04-25T00:08:16.053268+0000","last_became_active":"2024-04-25T00:08:14.707342+0000","last_became_peered":"2024-04-25T00:08:14.707342+0000","last_unstale":"2024-04-25T00:08:16.053268+0000","last_undegraded":"2024-04-25T00:08:16.053268+0000","last_fullsized":"2024-04-25T00:08:16.053268+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.710390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053318+0000","last_change":"2024-04-25T00:08:13.701962+0000","last_active":"2024-04-25T00:08:16.053318+0000","last_peered":"2024-04-25T00:08:16.053318+0000","last_clean":"2024-04-25T00:08:16.053318+0000","last_became_active":"2024-04-25T00:08:13.701654+0000","last_became_peered":"2024-04-25T00:08:13.701654+0000","last_unstale":"2024-04-25T00:08:16.053318+0000","last_undegraded":"2024-04-25T00:08:16.053318+0000","last_fullsized":"2024-04-25T00:08:16.053318+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:09.000563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052958+0000","last_change":"2024-04-25T00:08:14.705872+0000","last_active":"2024-04-25T00:08:16.052958+0000","last_peered":"2024-04-25T00:08:16.052958+0000","last_clean":"2024-04-25T00:08:16.052958+0000","last_became_active":"2024-04-25T00:08:14.705302+0000","last_became_peered":"2024-04-25T00:08:14.705302+0000","last_unstale":"2024-04-25T00:08:16.052958+0000","last_undegraded":"2024-04-25T00:08:16.052958+0000","last_fullsized":"2024-04-25T00:08:16.052958+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:40.729436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053226+0000","last_change":"2024-04-25T00:08:13.700172+0000","last_active":"2024-04-25T00:08:16.053226+0000","last_peered":"2024-04-25T00:08:16.053226+0000","last_clean":"2024-04-25T00:08:16.053226+0000","last_became_active":"2024-04-25T00:08:13.699775+0000","last_became_peered":"2024-04-25T00:08:13.699775+0000","last_unstale":"2024-04-25T00:08:16.053226+0000","last_undegraded":"2024-04-25T00:08:16.053226+0000","last_fullsized":"2024-04-25T00:08:16.053226+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:27.764780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":473,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053766+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:08:16.053766+0000","last_peered":"2024-04-25T00:08:16.053766+0000","last_clean":"2024-04-25T00:08:16.053766+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:08:16.053766+0000","last_undegraded":"2024-04-25T00:08:16.053766+0000","last_fullsized":"2024-04-25T00:08:16.053766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47596,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324244,"statfs":{"total":96636764160,"available":96588025856,"internally_reserved":0,"allocated":2588672,"data_stored":1455312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51176,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320664,"statfs":{"total":96636764160,"available":96584359936,"internally_reserved":0,"allocated":1986560,"data_stored":868912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55320,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316520,"statfs":{"total":96636764160,"available":96580116480,"internally_reserved":0,"allocated":1978368,"data_stored":863194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47016,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324824,"statfs":{"total":96636764160,"available":96588619776,"internally_reserved":0,"allocated":1986560,"data_stored":864272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51160,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320680,"statfs":{"total":96636764160,"available":96584376320,"internally_reserved":0,"allocated":1974272,"data_stored":862624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51168,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320672,"statfs":{"total":96636764160,"available":96584368128,"internally_reserved":0,"allocated":1986560,"data_stored":868980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55320,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316520,"statfs":{"total":96636764160,"available":96580116480,"internally_reserved":0,"allocated":1974272,"data_stored":862624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47588,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324252,"statfs":{"total":96636764160,"available":96588034048,"internally_reserved":0,"allocated":2576384,"data_stored":1459258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:08:17.831 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:08:17.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.963+0000 7fc241d14640 1 Processor -- start 2024-04-25T00:08:17.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.963+0000 7fc241d14640 1 -- start start 2024-04-25T00:08:17.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc241d14640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc23c15e320 0x7fc23c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc241d14640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc23c0d0b00 0x7fc23c0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc241d14640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc23c0d3de0 con 0x7fc23c0d2290 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc241d14640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c0d14b0 con 0x7fc23c0d0b00 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc241d14640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c0d3410 con 0x7fc23c15e320 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc23bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc23c0d0b00 0x7fc23c0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc23bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc23c0d0b00 0x7fc23c0d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48316/0 (socket says 172.21.15.113:48316) 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.964+0000 7fc23bfff640 1 -- 172.21.15.113:0/2285068501 learned_addr learned my addr 172.21.15.113:0/2285068501 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.965+0000 7fc23b7fe640 1 --2- 172.21.15.113:0/2285068501 >> v2:172.21.15.178:3300/0 conn(0x7fc23c15e320 0x7fc23c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.965+0000 7fc23bfff640 1 -- 172.21.15.113:0/2285068501 >> v2:172.21.15.178:3300/0 conn(0x7fc23c15e320 msgr2=0x7fc23c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.965+0000 7fc23bfff640 1 --2- 172.21.15.113:0/2285068501 >> v2:172.21.15.178:3300/0 conn(0x7fc23c15e320 0x7fc23c0d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.965+0000 7fc23bfff640 1 -- 172.21.15.113:0/2285068501 >> v1:172.21.15.178:6789/0 conn(0x7fc23c0d2290 legacy=0x7fc23c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.965+0000 7fc23bfff640 1 -- 172.21.15.113:0/2285068501 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c0d2800 con 0x7fc23c0d0b00 2024-04-25T00:08:17.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.965+0000 7fc23bfff640 1 --2- 172.21.15.113:0/2285068501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c0d0b00 0x7fc23c0d0f70 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0b6d00 tx=0x7fc23402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179f5c88e06b347c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc23a7fc640 1 -- 172.21.15.113:0/2285068501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234036070 con 0x7fc23c0d0b00 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc23a7fc640 1 -- 172.21.15.113:0/2285068501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc234031600 con 0x7fc23c0d0b00 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc23a7fc640 1 -- 172.21.15.113:0/2285068501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340319d0 con 0x7fc23c0d0b00 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 -- 172.21.15.113:0/2285068501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c0d0b00 msgr2=0x7fc23c0d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 --2- 172.21.15.113:0/2285068501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c0d0b00 0x7fc23c0d0f70 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0b6d00 tx=0x7fc23402a780 comp rx=0 tx=0).stop 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 -- 172.21.15.113:0/2285068501 shutdown_connections 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 --2- 172.21.15.113:0/2285068501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c0d0b00 0x7fc23c0d0f70 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 --2- 172.21.15.113:0/2285068501 >> v2:172.21.15.178:3300/0 conn(0x7fc23c15e320 0x7fc23c0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 -- 172.21.15.113:0/2285068501 >> 172.21.15.113:0/2285068501 conn(0x7fc23c0b5c20 msgr2=0x7fc23c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 -- 172.21.15.113:0/2285068501 shutdown_connections 2024-04-25T00:08:17.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.966+0000 7fc241d14640 1 -- 172.21.15.113:0/2285068501 wait complete. 2024-04-25T00:08:17.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.967+0000 7fc241d14640 1 Processor -- start 2024-04-25T00:08:17.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.967+0000 7fc241d14640 1 -- start start 2024-04-25T00:08:17.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc241d14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc23c0d0b00 0x7fc23c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc241d14640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc23c0d2290 0x7fc23c15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc241d14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c15e320 0x7fc23c15ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc241d14640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c0d30a0 con 0x7fc23c15e320 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc241d14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c0d2dc0 con 0x7fc23c0d0b00 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc241d14640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c15e700 con 0x7fc23c0d2290 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc23bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c15e320 0x7fc23c15ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc23bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c15e320 0x7fc23c15ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48328/0 (socket says 172.21.15.113:48328) 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc23bfff640 1 -- 172.21.15.113:0/2679338619 learned_addr learned my addr 172.21.15.113:0/2679338619 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc23b7fe640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc23c0d0b00 0x7fc23c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.968+0000 7fc23affd640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc23c0d2290 0x7fc23c15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc23bfff640 1 -- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc23c0d2290 msgr2=0x7fc23c15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc23bfff640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc23c0d2290 0x7fc23c15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc23bfff640 1 -- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc23c0d0b00 msgr2=0x7fc23c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc23bfff640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc23c0d0b00 0x7fc23c0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc23bfff640 1 -- 172.21.15.113:0/2679338619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c0ca360 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc23bfff640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c15e320 0x7fc23c15ef80 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fc234009f40 tx=0x7fc234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc223fff640 1 -- 172.21.15.113:0/2679338619 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234036050 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc223fff640 1 -- 172.21.15.113:0/2679338619 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc234042070 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc223fff640 1 -- 172.21.15.113:0/2679338619 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234009510 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c15f5b0 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.969+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c15fb70 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.970+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c0691a0 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.970+0000 7fc223fff640 1 -- 172.21.15.113:0/2679338619 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc234007bf0 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.971+0000 7fc223fff640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc22806be30 0x7fc22806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.971+0000 7fc223fff640 1 -- 172.21.15.113:0/2679338619 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fc2340abe00 con 0x7fc23c15e320 2024-04-25T00:08:17.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.971+0000 7fc23b7fe640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc22806be30 0x7fc22806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.971+0000 7fc23b7fe640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc22806be30 0x7fc22806e2f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00aa20 tx=0x7fc22c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:17.975+0000 7fc223fff640 1 -- 172.21.15.113:0/2679338619 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc23407a720 con 0x7fc23c15e320 2024-04-25T00:08:18.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.144+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc23c0b47b0 con 0x7fc22806be30 2024-04-25T00:08:18.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.149+0000 7fc223fff640 1 -- 172.21.15.113:0/2679338619 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7fc23c0b47b0 con 0x7fc22806be30 2024-04-25T00:08:18.152 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:18.153 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:08:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.152+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc22806be30 msgr2=0x7fc22806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.152+0000 7fc241d14640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc22806be30 0x7fc22806e2f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00aa20 tx=0x7fc22c008040 comp rx=0 tx=0).stop 2024-04-25T00:08:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c15e320 msgr2=0x7fc23c15ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c15e320 0x7fc23c15ef80 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fc234009f40 tx=0x7fc234004a40 comp rx=0 tx=0).stop 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 shutdown_connections 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc22806be30 0x7fc22806e2f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c15e320 0x7fc23c15ef80 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc23c0d2290 0x7fc23c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 --2- 172.21.15.113:0/2679338619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc23c0d0b00 0x7fc23c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 >> 172.21.15.113:0/2679338619 conn(0x7fc23c0b5c20 msgr2=0x7fc23c0b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 shutdown_connections 2024-04-25T00:08:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.153+0000 7fc241d14640 1 -- 172.21.15.113:0/2679338619 wait complete. 2024-04-25T00:08:18.164 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-04-25T00:08:17.229683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":254,"num_write_kb":3476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406344,"kb_used_data":16652,"kb_used_omap":12,"kb_used_meta":389555,"kb_avail":754568376,"statfs":{"total":773094113280,"available":772678017024,"internally_reserved":0,"allocated":17051648,"data_stored":8105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":398904880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.546095"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054022+0000","last_change":"2024-04-25T00:08:14.716516+0000","last_active":"2024-04-25T00:08:16.054022+0000","last_peered":"2024-04-25T00:08:16.054022+0000","last_clean":"2024-04-25T00:08:16.054022+0000","last_became_active":"2024-04-25T00:08:14.716162+0000","last_became_peered":"2024-04-25T00:08:14.716162+0000","last_unstale":"2024-04-25T00:08:16.054022+0000","last_undegraded":"2024-04-25T00:08:16.054022+0000","last_fullsized":"2024-04-25T00:08:16.054022+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:01.992891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703184+0000","last_change":"2024-04-25T00:08:14.717856+0000","last_active":"2024-04-25T00:08:15.703184+0000","last_peered":"2024-04-25T00:08:15.703184+0000","last_clean":"2024-04-25T00:08:15.703184+0000","last_became_active":"2024-04-25T00:08:14.717432+0000","last_became_peered":"2024-04-25T00:08:14.717432+0000","last_unstale":"2024-04-25T00:08:15.703184+0000","last_undegraded":"2024-04-25T00:08:15.703184+0000","last_fullsized":"2024-04-25T00:08:15.703184+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:59:13.514673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053410+0000","last_change":"2024-04-25T00:08:14.712341+0000","last_active":"2024-04-25T00:08:16.053410+0000","last_peered":"2024-04-25T00:08:16.053410+0000","last_clean":"2024-04-25T00:08:16.053410+0000","last_became_active":"2024-04-25T00:08:14.711770+0000","last_became_peered":"2024-04-25T00:08:14.711770+0000","last_unstale":"2024-04-25T00:08:16.053410+0000","last_undegraded":"2024-04-25T00:08:16.053410+0000","last_fullsized":"2024-04-25T00:08:16.053410+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:12.670911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053290+0000","last_change":"2024-04-25T00:08:14.706575+0000","last_active":"2024-04-25T00:08:16.053290+0000","last_peered":"2024-04-25T00:08:16.053290+0000","last_clean":"2024-04-25T00:08:16.053290+0000","last_became_active":"2024-04-25T00:08:14.705115+0000","last_became_peered":"2024-04-25T00:08:14.705115+0000","last_unstale":"2024-04-25T00:08:16.053290+0000","last_undegraded":"2024-04-25T00:08:16.053290+0000","last_fullsized":"2024-04-25T00:08:16.053290+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:54.024198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703023+0000","last_change":"2024-04-25T00:08:14.701773+0000","last_active":"2024-04-25T00:08:15.703023+0000","last_peered":"2024-04-25T00:08:15.703023+0000","last_clean":"2024-04-25T00:08:15.703023+0000","last_became_active":"2024-04-25T00:08:14.701471+0000","last_became_peered":"2024-04-25T00:08:14.701471+0000","last_unstale":"2024-04-25T00:08:15.703023+0000","last_undegraded":"2024-04-25T00:08:15.703023+0000","last_fullsized":"2024-04-25T00:08:15.703023+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:10.005638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702152+0000","last_change":"2024-04-25T00:08:14.710986+0000","last_active":"2024-04-25T00:08:15.702152+0000","last_peered":"2024-04-25T00:08:15.702152+0000","last_clean":"2024-04-25T00:08:15.702152+0000","last_became_active":"2024-04-25T00:08:14.710720+0000","last_became_peered":"2024-04-25T00:08:14.710720+0000","last_unstale":"2024-04-25T00:08:15.702152+0000","last_undegraded":"2024-04-25T00:08:15.702152+0000","last_fullsized":"2024-04-25T00:08:15.702152+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:53.504168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702524+0000","last_change":"2024-04-25T00:08:14.700518+0000","last_active":"2024-04-25T00:08:15.702524+0000","last_peered":"2024-04-25T00:08:15.702524+0000","last_clean":"2024-04-25T00:08:15.702524+0000","last_became_active":"2024-04-25T00:08:14.699700+0000","last_became_peered":"2024-04-25T00:08:14.699700+0000","last_unstale":"2024-04-25T00:08:15.702524+0000","last_undegraded":"2024-04-25T00:08:15.702524+0000","last_fullsized":"2024-04-25T00:08:15.702524+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:21.035999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702665+0000","last_change":"2024-04-25T00:08:14.709383+0000","last_active":"2024-04-25T00:08:15.702665+0000","last_peered":"2024-04-25T00:08:15.702665+0000","last_clean":"2024-04-25T00:08:15.702665+0000","last_became_active":"2024-04-25T00:08:14.709045+0000","last_became_peered":"2024-04-25T00:08:14.709045+0000","last_unstale":"2024-04-25T00:08:15.702665+0000","last_undegraded":"2024-04-25T00:08:15.702665+0000","last_fullsized":"2024-04-25T00:08:15.702665+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:28.769856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703201+0000","last_change":"2024-04-25T00:08:14.717070+0000","last_active":"2024-04-25T00:08:15.703201+0000","last_peered":"2024-04-25T00:08:15.703201+0000","last_clean":"2024-04-25T00:08:15.703201+0000","last_became_active":"2024-04-25T00:08:14.716787+0000","last_became_peered":"2024-04-25T00:08:14.716787+0000","last_unstale":"2024-04-25T00:08:15.703201+0000","last_undegraded":"2024-04-25T00:08:15.703201+0000","last_fullsized":"2024-04-25T00:08:15.703201+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:07.211582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703350+0000","last_change":"2024-04-25T00:08:14.710369+0000","last_active":"2024-04-25T00:08:15.703350+0000","last_peered":"2024-04-25T00:08:15.703350+0000","last_clean":"2024-04-25T00:08:15.703350+0000","last_became_active":"2024-04-25T00:08:14.710078+0000","last_became_peered":"2024-04-25T00:08:14.710078+0000","last_unstale":"2024-04-25T00:08:15.703350+0000","last_undegraded":"2024-04-25T00:08:15.703350+0000","last_fullsized":"2024-04-25T00:08:15.703350+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:06.843612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703203+0000","last_change":"2024-04-25T00:08:14.702387+0000","last_active":"2024-04-25T00:08:15.703203+0000","last_peered":"2024-04-25T00:08:15.703203+0000","last_clean":"2024-04-25T00:08:15.703203+0000","last_became_active":"2024-04-25T00:08:14.702102+0000","last_became_peered":"2024-04-25T00:08:14.702102+0000","last_unstale":"2024-04-25T00:08:15.703203+0000","last_undegraded":"2024-04-25T00:08:15.703203+0000","last_fullsized":"2024-04-25T00:08:15.703203+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:21.846196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702980+0000","last_change":"2024-04-25T00:08:14.718107+0000","last_active":"2024-04-25T00:08:15.702980+0000","last_peered":"2024-04-25T00:08:15.702980+0000","last_clean":"2024-04-25T00:08:15.702980+0000","last_became_active":"2024-04-25T00:08:14.717489+0000","last_became_peered":"2024-04-25T00:08:14.717489+0000","last_unstale":"2024-04-25T00:08:15.702980+0000","last_undegraded":"2024-04-25T00:08:15.702980+0000","last_fullsized":"2024-04-25T00:08:15.702980+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:22.739984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053595+0000","last_change":"2024-04-25T00:08:14.711772+0000","last_active":"2024-04-25T00:08:16.053595+0000","last_peered":"2024-04-25T00:08:16.053595+0000","last_clean":"2024-04-25T00:08:16.053595+0000","last_became_active":"2024-04-25T00:08:14.709256+0000","last_became_peered":"2024-04-25T00:08:14.709256+0000","last_unstale":"2024-04-25T00:08:16.053595+0000","last_undegraded":"2024-04-25T00:08:16.053595+0000","last_fullsized":"2024-04-25T00:08:16.053595+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:22.105245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052859+0000","last_change":"2024-04-25T00:08:14.718375+0000","last_active":"2024-04-25T00:08:16.052859+0000","last_peered":"2024-04-25T00:08:16.052859+0000","last_clean":"2024-04-25T00:08:16.052859+0000","last_became_active":"2024-04-25T00:08:14.718101+0000","last_became_peered":"2024-04-25T00:08:14.718101+0000","last_unstale":"2024-04-25T00:08:16.052859+0000","last_undegraded":"2024-04-25T00:08:16.052859+0000","last_fullsized":"2024-04-25T00:08:16.052859+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:18.531771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053116+0000","last_change":"2024-04-25T00:08:14.713124+0000","last_active":"2024-04-25T00:08:16.053116+0000","last_peered":"2024-04-25T00:08:16.053116+0000","last_clean":"2024-04-25T00:08:16.053116+0000","last_became_active":"2024-04-25T00:08:14.712809+0000","last_became_peered":"2024-04-25T00:08:14.712809+0000","last_unstale":"2024-04-25T00:08:16.053116+0000","last_undegraded":"2024-04-25T00:08:16.053116+0000","last_fullsized":"2024-04-25T00:08:16.053116+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:29.924646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702694+0000","last_change":"2024-04-25T00:08:14.717399+0000","last_active":"2024-04-25T00:08:15.702694+0000","last_peered":"2024-04-25T00:08:15.702694+0000","last_clean":"2024-04-25T00:08:15.702694+0000","last_became_active":"2024-04-25T00:08:14.717012+0000","last_became_peered":"2024-04-25T00:08:14.717012+0000","last_unstale":"2024-04-25T00:08:15.702694+0000","last_undegraded":"2024-04-25T00:08:15.702694+0000","last_fullsized":"2024-04-25T00:08:15.702694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:30.092968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702293+0000","last_change":"2024-04-25T00:08:14.709858+0000","last_active":"2024-04-25T00:08:15.702293+0000","last_peered":"2024-04-25T00:08:15.702293+0000","last_clean":"2024-04-25T00:08:15.702293+0000","last_became_active":"2024-04-25T00:08:14.709588+0000","last_became_peered":"2024-04-25T00:08:14.709588+0000","last_unstale":"2024-04-25T00:08:15.702293+0000","last_undegraded":"2024-04-25T00:08:15.702293+0000","last_fullsized":"2024-04-25T00:08:15.702293+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:12.303548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052762+0000","last_change":"2024-04-25T00:08:14.707255+0000","last_active":"2024-04-25T00:08:16.052762+0000","last_peered":"2024-04-25T00:08:16.052762+0000","last_clean":"2024-04-25T00:08:16.052762+0000","last_became_active":"2024-04-25T00:08:14.706213+0000","last_became_peered":"2024-04-25T00:08:14.706213+0000","last_unstale":"2024-04-25T00:08:16.052762+0000","last_undegraded":"2024-04-25T00:08:16.052762+0000","last_fullsized":"2024-04-25T00:08:16.052762+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:47.626356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702578+0000","last_change":"2024-04-25T00:08:14.711548+0000","last_active":"2024-04-25T00:08:15.702578+0000","last_peered":"2024-04-25T00:08:15.702578+0000","last_clean":"2024-04-25T00:08:15.702578+0000","last_became_active":"2024-04-25T00:08:14.711309+0000","last_became_peered":"2024-04-25T00:08:14.711309+0000","last_unstale":"2024-04-25T00:08:15.702578+0000","last_undegraded":"2024-04-25T00:08:15.702578+0000","last_fullsized":"2024-04-25T00:08:15.702578+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:06.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052836+0000","last_change":"2024-04-25T00:08:14.719499+0000","last_active":"2024-04-25T00:08:16.052836+0000","last_peered":"2024-04-25T00:08:16.052836+0000","last_clean":"2024-04-25T00:08:16.052836+0000","last_became_active":"2024-04-25T00:08:14.719235+0000","last_became_peered":"2024-04-25T00:08:14.719235+0000","last_unstale":"2024-04-25T00:08:16.052836+0000","last_undegraded":"2024-04-25T00:08:16.052836+0000","last_fullsized":"2024-04-25T00:08:16.052836+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:01.498161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702452+0000","last_change":"2024-04-25T00:08:14.710352+0000","last_active":"2024-04-25T00:08:15.702452+0000","last_peered":"2024-04-25T00:08:15.702452+0000","last_clean":"2024-04-25T00:08:15.702452+0000","last_became_active":"2024-04-25T00:08:14.710090+0000","last_became_peered":"2024-04-25T00:08:14.710090+0000","last_unstale":"2024-04-25T00:08:15.702452+0000","last_undegraded":"2024-04-25T00:08:15.702452+0000","last_fullsized":"2024-04-25T00:08:15.702452+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:26.590584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702920+0000","last_change":"2024-04-25T00:08:14.706576+0000","last_active":"2024-04-25T00:08:15.702920+0000","last_peered":"2024-04-25T00:08:15.702920+0000","last_clean":"2024-04-25T00:08:15.702920+0000","last_became_active":"2024-04-25T00:08:14.706295+0000","last_became_peered":"2024-04-25T00:08:14.706295+0000","last_unstale":"2024-04-25T00:08:15.702920+0000","last_undegraded":"2024-04-25T00:08:15.702920+0000","last_fullsized":"2024-04-25T00:08:15.702920+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:08.591028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702405+0000","last_change":"2024-04-25T00:08:14.713136+0000","last_active":"2024-04-25T00:08:15.702405+0000","last_peered":"2024-04-25T00:08:15.702405+0000","last_clean":"2024-04-25T00:08:15.702405+0000","last_became_active":"2024-04-25T00:08:14.712841+0000","last_became_peered":"2024-04-25T00:08:14.712841+0000","last_unstale":"2024-04-25T00:08:15.702405+0000","last_undegraded":"2024-04-25T00:08:15.702405+0000","last_fullsized":"2024-04-25T00:08:15.702405+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:07.011934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702875+0000","last_change":"2024-04-25T00:08:14.705935+0000","last_active":"2024-04-25T00:08:15.702875+0000","last_peered":"2024-04-25T00:08:15.702875+0000","last_clean":"2024-04-25T00:08:15.702875+0000","last_became_active":"2024-04-25T00:08:14.705630+0000","last_became_peered":"2024-04-25T00:08:14.705630+0000","last_unstale":"2024-04-25T00:08:15.702875+0000","last_undegraded":"2024-04-25T00:08:15.702875+0000","last_fullsized":"2024-04-25T00:08:15.702875+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:00.514572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054361+0000","last_change":"2024-04-25T00:08:14.714748+0000","last_active":"2024-04-25T00:08:16.054361+0000","last_peered":"2024-04-25T00:08:16.054361+0000","last_clean":"2024-04-25T00:08:16.054361+0000","last_became_active":"2024-04-25T00:08:14.713364+0000","last_became_peered":"2024-04-25T00:08:14.713364+0000","last_unstale":"2024-04-25T00:08:16.054361+0000","last_undegraded":"2024-04-25T00:08:16.054361+0000","last_fullsized":"2024-04-25T00:08:16.054361+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.499340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702709+0000","last_change":"2024-04-25T00:08:14.705484+0000","last_active":"2024-04-25T00:08:15.702709+0000","last_peered":"2024-04-25T00:08:15.702709+0000","last_clean":"2024-04-25T00:08:15.702709+0000","last_became_active":"2024-04-25T00:08:14.705168+0000","last_became_peered":"2024-04-25T00:08:14.705168+0000","last_unstale":"2024-04-25T00:08:15.702709+0000","last_undegraded":"2024-04-25T00:08:15.702709+0000","last_fullsized":"2024-04-25T00:08:15.702709+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:32.502667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053558+0000","last_change":"2024-04-25T00:08:14.715537+0000","last_active":"2024-04-25T00:08:16.053558+0000","last_peered":"2024-04-25T00:08:16.053558+0000","last_clean":"2024-04-25T00:08:16.053558+0000","last_became_active":"2024-04-25T00:08:14.715269+0000","last_became_peered":"2024-04-25T00:08:14.715269+0000","last_unstale":"2024-04-25T00:08:16.053558+0000","last_undegraded":"2024-04-25T00:08:16.053558+0000","last_fullsized":"2024-04-25T00:08:16.053558+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:41.043191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053436+0000","last_change":"2024-04-25T00:08:14.709225+0000","last_active":"2024-04-25T00:08:16.053436+0000","last_peered":"2024-04-25T00:08:16.053436+0000","last_clean":"2024-04-25T00:08:16.053436+0000","last_became_active":"2024-04-25T00:08:14.708983+0000","last_became_peered":"2024-04-25T00:08:14.708983+0000","last_unstale":"2024-04-25T00:08:16.053436+0000","last_undegraded":"2024-04-25T00:08:16.053436+0000","last_fullsized":"2024-04-25T00:08:16.053436+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:51.333007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053662+0000","last_change":"2024-04-25T00:08:14.714945+0000","last_active":"2024-04-25T00:08:16.053662+0000","last_peered":"2024-04-25T00:08:16.053662+0000","last_clean":"2024-04-25T00:08:16.053662+0000","last_became_active":"2024-04-25T00:08:14.714354+0000","last_became_peered":"2024-04-25T00:08:14.714354+0000","last_unstale":"2024-04-25T00:08:16.053662+0000","last_undegraded":"2024-04-25T00:08:16.053662+0000","last_fullsized":"2024-04-25T00:08:16.053662+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:58.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053097+0000","last_change":"2024-04-25T00:08:14.713878+0000","last_active":"2024-04-25T00:08:16.053097+0000","last_peered":"2024-04-25T00:08:16.053097+0000","last_clean":"2024-04-25T00:08:16.053097+0000","last_became_active":"2024-04-25T00:08:14.713577+0000","last_became_peered":"2024-04-25T00:08:14.713577+0000","last_unstale":"2024-04-25T00:08:16.053097+0000","last_undegraded":"2024-04-25T00:08:16.053097+0000","last_fullsized":"2024-04-25T00:08:16.053097+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:17:56.200296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054136+0000","last_change":"2024-04-25T00:08:14.699958+0000","last_active":"2024-04-25T00:08:16.054136+0000","last_peered":"2024-04-25T00:08:16.054136+0000","last_clean":"2024-04-25T00:08:16.054136+0000","last_became_active":"2024-04-25T00:08:14.699659+0000","last_became_peered":"2024-04-25T00:08:14.699659+0000","last_unstale":"2024-04-25T00:08:16.054136+0000","last_undegraded":"2024-04-25T00:08:16.054136+0000","last_fullsized":"2024-04-25T00:08:16.054136+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:42.041767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052911+0000","last_change":"2024-04-25T00:08:14.701530+0000","last_active":"2024-04-25T00:08:16.052911+0000","last_peered":"2024-04-25T00:08:16.052911+0000","last_clean":"2024-04-25T00:08:16.052911+0000","last_became_active":"2024-04-25T00:08:14.701271+0000","last_became_peered":"2024-04-25T00:08:14.701271+0000","last_unstale":"2024-04-25T00:08:16.052911+0000","last_undegraded":"2024-04-25T00:08:16.052911+0000","last_fullsized":"2024-04-25T00:08:16.052911+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:40.670725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702673+0000","last_change":"2024-04-25T00:08:14.706577+0000","last_active":"2024-04-25T00:08:15.702673+0000","last_peered":"2024-04-25T00:08:15.702673+0000","last_clean":"2024-04-25T00:08:15.702673+0000","last_became_active":"2024-04-25T00:08:14.704713+0000","last_became_peered":"2024-04-25T00:08:14.704713+0000","last_unstale":"2024-04-25T00:08:15.702673+0000","last_undegraded":"2024-04-25T00:08:15.702673+0000","last_fullsized":"2024-04-25T00:08:15.702673+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:57.507086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"157'4","reported_seq":18,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054134+0000","last_change":"2024-04-25T00:08:13.707841+0000","last_active":"2024-04-25T00:08:16.054134+0000","last_peered":"2024-04-25T00:08:16.054134+0000","last_clean":"2024-04-25T00:08:16.054134+0000","last_became_active":"2024-04-25T00:08:13.707571+0000","last_became_peered":"2024-04-25T00:08:13.707571+0000","last_unstale":"2024-04-25T00:08:16.054134+0000","last_undegraded":"2024-04-25T00:08:16.054134+0000","last_fullsized":"2024-04-25T00:08:16.054134+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:21.502717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053262+0000","last_change":"2024-04-25T00:08:14.701146+0000","last_active":"2024-04-25T00:08:16.053262+0000","last_peered":"2024-04-25T00:08:16.053262+0000","last_clean":"2024-04-25T00:08:16.053262+0000","last_became_active":"2024-04-25T00:08:14.700595+0000","last_became_peered":"2024-04-25T00:08:14.700595+0000","last_unstale":"2024-04-25T00:08:16.053262+0000","last_undegraded":"2024-04-25T00:08:16.053262+0000","last_fullsized":"2024-04-25T00:08:16.053262+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:54.226601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054055+0000","last_change":"2024-04-25T00:08:13.704657+0000","last_active":"2024-04-25T00:08:16.054055+0000","last_peered":"2024-04-25T00:08:16.054055+0000","last_clean":"2024-04-25T00:08:16.054055+0000","last_became_active":"2024-04-25T00:08:13.704376+0000","last_became_peered":"2024-04-25T00:08:13.704376+0000","last_unstale":"2024-04-25T00:08:16.054055+0000","last_undegraded":"2024-04-25T00:08:16.054055+0000","last_fullsized":"2024-04-25T00:08:16.054055+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:59.509497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702151+0000","last_change":"2024-04-25T00:08:14.716933+0000","last_active":"2024-04-25T00:08:15.702151+0000","last_peered":"2024-04-25T00:08:15.702151+0000","last_clean":"2024-04-25T00:08:15.702151+0000","last_became_active":"2024-04-25T00:08:14.716650+0000","last_became_peered":"2024-04-25T00:08:14.716650+0000","last_unstale":"2024-04-25T00:08:15.702151+0000","last_undegraded":"2024-04-25T00:08:15.702151+0000","last_fullsized":"2024-04-25T00:08:15.702151+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:39.190958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053311+0000","last_change":"2024-04-25T00:08:13.705148+0000","last_active":"2024-04-25T00:08:16.053311+0000","last_peered":"2024-04-25T00:08:16.053311+0000","last_clean":"2024-04-25T00:08:16.053311+0000","last_became_active":"2024-04-25T00:08:13.704884+0000","last_became_peered":"2024-04-25T00:08:13.704884+0000","last_unstale":"2024-04-25T00:08:16.053311+0000","last_undegraded":"2024-04-25T00:08:16.053311+0000","last_fullsized":"2024-04-25T00:08:16.053311+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:46.785370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053793+0000","last_change":"2024-04-25T00:08:14.714525+0000","last_active":"2024-04-25T00:08:16.053793+0000","last_peered":"2024-04-25T00:08:16.053793+0000","last_clean":"2024-04-25T00:08:16.053793+0000","last_became_active":"2024-04-25T00:08:14.713431+0000","last_became_peered":"2024-04-25T00:08:14.713431+0000","last_unstale":"2024-04-25T00:08:16.053793+0000","last_undegraded":"2024-04-25T00:08:16.053793+0000","last_fullsized":"2024-04-25T00:08:16.053793+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:54.024198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053543+0000","last_change":"2024-04-25T00:08:13.699351+0000","last_active":"2024-04-25T00:08:16.053543+0000","last_peered":"2024-04-25T00:08:16.053543+0000","last_clean":"2024-04-25T00:08:16.053543+0000","last_became_active":"2024-04-25T00:08:13.699077+0000","last_became_peered":"2024-04-25T00:08:13.699077+0000","last_unstale":"2024-04-25T00:08:16.053543+0000","last_undegraded":"2024-04-25T00:08:16.053543+0000","last_fullsized":"2024-04-25T00:08:16.053543+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:09.884267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702449+0000","last_change":"2024-04-25T00:08:14.701395+0000","last_active":"2024-04-25T00:08:15.702449+0000","last_peered":"2024-04-25T00:08:15.702449+0000","last_clean":"2024-04-25T00:08:15.702449+0000","last_became_active":"2024-04-25T00:08:14.699111+0000","last_became_peered":"2024-04-25T00:08:14.699111+0000","last_unstale":"2024-04-25T00:08:15.702449+0000","last_undegraded":"2024-04-25T00:08:15.702449+0000","last_fullsized":"2024-04-25T00:08:15.702449+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.675509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053785+0000","last_change":"2024-04-25T00:08:13.706500+0000","last_active":"2024-04-25T00:08:16.053785+0000","last_peered":"2024-04-25T00:08:16.053785+0000","last_clean":"2024-04-25T00:08:16.053785+0000","last_became_active":"2024-04-25T00:08:13.706178+0000","last_became_peered":"2024-04-25T00:08:13.706178+0000","last_unstale":"2024-04-25T00:08:16.053785+0000","last_undegraded":"2024-04-25T00:08:16.053785+0000","last_fullsized":"2024-04-25T00:08:16.053785+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:45.927693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053328+0000","last_change":"2024-04-25T00:08:14.706886+0000","last_active":"2024-04-25T00:08:16.053328+0000","last_peered":"2024-04-25T00:08:16.053328+0000","last_clean":"2024-04-25T00:08:16.053328+0000","last_became_active":"2024-04-25T00:08:14.705156+0000","last_became_peered":"2024-04-25T00:08:14.705156+0000","last_unstale":"2024-04-25T00:08:16.053328+0000","last_undegraded":"2024-04-25T00:08:16.053328+0000","last_fullsized":"2024-04-25T00:08:16.053328+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:58.499340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053081+0000","last_change":"2024-04-25T00:08:13.698272+0000","last_active":"2024-04-25T00:08:16.053081+0000","last_peered":"2024-04-25T00:08:16.053081+0000","last_clean":"2024-04-25T00:08:16.053081+0000","last_became_active":"2024-04-25T00:08:13.697992+0000","last_became_peered":"2024-04-25T00:08:13.697992+0000","last_unstale":"2024-04-25T00:08:16.053081+0000","last_undegraded":"2024-04-25T00:08:16.053081+0000","last_fullsized":"2024-04-25T00:08:16.053081+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:25.585509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052671+0000","last_change":"2024-04-25T00:08:14.717854+0000","last_active":"2024-04-25T00:08:16.052671+0000","last_peered":"2024-04-25T00:08:16.052671+0000","last_clean":"2024-04-25T00:08:16.052671+0000","last_became_active":"2024-04-25T00:08:14.717547+0000","last_became_peered":"2024-04-25T00:08:14.717547+0000","last_unstale":"2024-04-25T00:08:16.052671+0000","last_undegraded":"2024-04-25T00:08:16.052671+0000","last_fullsized":"2024-04-25T00:08:16.052671+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:22.105245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702682+0000","last_change":"2024-04-25T00:08:13.702223+0000","last_active":"2024-04-25T00:08:15.702682+0000","last_peered":"2024-04-25T00:08:15.702682+0000","last_clean":"2024-04-25T00:08:15.702682+0000","last_became_active":"2024-04-25T00:08:13.701905+0000","last_became_peered":"2024-04-25T00:08:13.701905+0000","last_unstale":"2024-04-25T00:08:15.702682+0000","last_undegraded":"2024-04-25T00:08:15.702682+0000","last_fullsized":"2024-04-25T00:08:15.702682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:11.665836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702177+0000","last_change":"2024-04-25T00:08:14.707467+0000","last_active":"2024-04-25T00:08:15.702177+0000","last_peered":"2024-04-25T00:08:15.702177+0000","last_clean":"2024-04-25T00:08:15.702177+0000","last_became_active":"2024-04-25T00:08:14.707211+0000","last_became_peered":"2024-04-25T00:08:14.707211+0000","last_unstale":"2024-04-25T00:08:15.702177+0000","last_undegraded":"2024-04-25T00:08:15.702177+0000","last_fullsized":"2024-04-25T00:08:15.702177+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:23.174081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702786+0000","last_change":"2024-04-25T00:08:13.702235+0000","last_active":"2024-04-25T00:08:15.702786+0000","last_peered":"2024-04-25T00:08:15.702786+0000","last_clean":"2024-04-25T00:08:15.702786+0000","last_became_active":"2024-04-25T00:08:13.701744+0000","last_became_peered":"2024-04-25T00:08:13.701744+0000","last_unstale":"2024-04-25T00:08:15.702786+0000","last_undegraded":"2024-04-25T00:08:15.702786+0000","last_fullsized":"2024-04-25T00:08:15.702786+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:18.545568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053059+0000","last_change":"2024-04-25T00:08:14.699053+0000","last_active":"2024-04-25T00:08:16.053059+0000","last_peered":"2024-04-25T00:08:16.053059+0000","last_clean":"2024-04-25T00:08:16.053059+0000","last_became_active":"2024-04-25T00:08:14.698733+0000","last_became_peered":"2024-04-25T00:08:14.698733+0000","last_unstale":"2024-04-25T00:08:16.053059+0000","last_undegraded":"2024-04-25T00:08:16.053059+0000","last_fullsized":"2024-04-25T00:08:16.053059+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:22.507792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702449+0000","last_change":"2024-04-25T00:08:13.709325+0000","last_active":"2024-04-25T00:08:15.702449+0000","last_peered":"2024-04-25T00:08:15.702449+0000","last_clean":"2024-04-25T00:08:15.702449+0000","last_became_active":"2024-04-25T00:08:13.709050+0000","last_became_peered":"2024-04-25T00:08:13.709050+0000","last_unstale":"2024-04-25T00:08:15.702449+0000","last_undegraded":"2024-04-25T00:08:15.702449+0000","last_fullsized":"2024-04-25T00:08:15.702449+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:06.072795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052592+0000","last_change":"2024-04-25T00:08:14.699287+0000","last_active":"2024-04-25T00:08:16.052592+0000","last_peered":"2024-04-25T00:08:16.052592+0000","last_clean":"2024-04-25T00:08:16.052592+0000","last_became_active":"2024-04-25T00:08:14.699022+0000","last_became_peered":"2024-04-25T00:08:14.699022+0000","last_unstale":"2024-04-25T00:08:16.052592+0000","last_undegraded":"2024-04-25T00:08:16.052592+0000","last_fullsized":"2024-04-25T00:08:16.052592+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:18.538560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"157'3","reported_seq":17,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702563+0000","last_change":"2024-04-25T00:08:13.708197+0000","last_active":"2024-04-25T00:08:15.702563+0000","last_peered":"2024-04-25T00:08:15.702563+0000","last_clean":"2024-04-25T00:08:15.702563+0000","last_became_active":"2024-04-25T00:08:13.707893+0000","last_became_peered":"2024-04-25T00:08:13.707893+0000","last_unstale":"2024-04-25T00:08:15.702563+0000","last_undegraded":"2024-04-25T00:08:15.702563+0000","last_fullsized":"2024-04-25T00:08:15.702563+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:42.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053635+0000","last_change":"2024-04-25T00:08:14.714483+0000","last_active":"2024-04-25T00:08:16.053635+0000","last_peered":"2024-04-25T00:08:16.053635+0000","last_clean":"2024-04-25T00:08:16.053635+0000","last_became_active":"2024-04-25T00:08:14.714224+0000","last_became_peered":"2024-04-25T00:08:14.714224+0000","last_unstale":"2024-04-25T00:08:16.053635+0000","last_undegraded":"2024-04-25T00:08:16.053635+0000","last_fullsized":"2024-04-25T00:08:16.053635+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.555958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"157'6","reported_seq":20,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053970+0000","last_change":"2024-04-25T00:08:13.699685+0000","last_active":"2024-04-25T00:08:16.053970+0000","last_peered":"2024-04-25T00:08:16.053970+0000","last_clean":"2024-04-25T00:08:16.053970+0000","last_became_active":"2024-04-25T00:08:13.698917+0000","last_became_peered":"2024-04-25T00:08:13.698917+0000","last_unstale":"2024-04-25T00:08:16.053970+0000","last_undegraded":"2024-04-25T00:08:16.053970+0000","last_fullsized":"2024-04-25T00:08:16.053970+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:00:56.226946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052597+0000","last_change":"2024-04-25T00:08:14.701982+0000","last_active":"2024-04-25T00:08:16.052597+0000","last_peered":"2024-04-25T00:08:16.052597+0000","last_clean":"2024-04-25T00:08:16.052597+0000","last_became_active":"2024-04-25T00:08:14.701511+0000","last_became_peered":"2024-04-25T00:08:14.701511+0000","last_unstale":"2024-04-25T00:08:16.052597+0000","last_undegraded":"2024-04-25T00:08:16.052597+0000","last_fullsized":"2024-04-25T00:08:16.052597+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:07.285153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053279+0000","last_change":"2024-04-25T00:08:13.701056+0000","last_active":"2024-04-25T00:08:16.053279+0000","last_peered":"2024-04-25T00:08:16.053279+0000","last_clean":"2024-04-25T00:08:16.053279+0000","last_became_active":"2024-04-25T00:08:13.700615+0000","last_became_peered":"2024-04-25T00:08:13.700615+0000","last_unstale":"2024-04-25T00:08:16.053279+0000","last_undegraded":"2024-04-25T00:08:16.053279+0000","last_fullsized":"2024-04-25T00:08:16.053279+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:02.034623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702769+0000","last_change":"2024-04-25T00:08:14.709839+0000","last_active":"2024-04-25T00:08:15.702769+0000","last_peered":"2024-04-25T00:08:15.702769+0000","last_clean":"2024-04-25T00:08:15.702769+0000","last_became_active":"2024-04-25T00:08:14.709287+0000","last_became_peered":"2024-04-25T00:08:14.709287+0000","last_unstale":"2024-04-25T00:08:15.702769+0000","last_undegraded":"2024-04-25T00:08:15.702769+0000","last_fullsized":"2024-04-25T00:08:15.702769+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:26.031254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052702+0000","last_change":"2024-04-25T00:08:13.702677+0000","last_active":"2024-04-25T00:08:16.052702+0000","last_peered":"2024-04-25T00:08:16.052702+0000","last_clean":"2024-04-25T00:08:16.052702+0000","last_became_active":"2024-04-25T00:08:13.702397+0000","last_became_peered":"2024-04-25T00:08:13.702397+0000","last_unstale":"2024-04-25T00:08:16.052702+0000","last_undegraded":"2024-04-25T00:08:16.052702+0000","last_fullsized":"2024-04-25T00:08:16.052702+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:06.206506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052839+0000","last_change":"2024-04-25T00:08:14.705759+0000","last_active":"2024-04-25T00:08:16.052839+0000","last_peered":"2024-04-25T00:08:16.052839+0000","last_clean":"2024-04-25T00:08:16.052839+0000","last_became_active":"2024-04-25T00:08:14.705512+0000","last_became_peered":"2024-04-25T00:08:14.705512+0000","last_unstale":"2024-04-25T00:08:16.052839+0000","last_undegraded":"2024-04-25T00:08:16.052839+0000","last_fullsized":"2024-04-25T00:08:16.052839+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:58.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052981+0000","last_change":"2024-04-25T00:08:13.704877+0000","last_active":"2024-04-25T00:08:16.052981+0000","last_peered":"2024-04-25T00:08:16.052981+0000","last_clean":"2024-04-25T00:08:16.052981+0000","last_became_active":"2024-04-25T00:08:13.704587+0000","last_became_peered":"2024-04-25T00:08:13.704587+0000","last_unstale":"2024-04-25T00:08:16.052981+0000","last_undegraded":"2024-04-25T00:08:16.052981+0000","last_fullsized":"2024-04-25T00:08:16.052981+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:24.518477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053114+0000","last_change":"2024-04-25T00:08:14.709756+0000","last_active":"2024-04-25T00:08:16.053114+0000","last_peered":"2024-04-25T00:08:16.053114+0000","last_clean":"2024-04-25T00:08:16.053114+0000","last_became_active":"2024-04-25T00:08:14.709467+0000","last_became_peered":"2024-04-25T00:08:14.709467+0000","last_unstale":"2024-04-25T00:08:16.053114+0000","last_undegraded":"2024-04-25T00:08:16.053114+0000","last_fullsized":"2024-04-25T00:08:16.053114+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:52:11.566204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"157'8","reported_seq":22,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053207+0000","last_change":"2024-04-25T00:08:13.700545+0000","last_active":"2024-04-25T00:08:16.053207+0000","last_peered":"2024-04-25T00:08:16.053207+0000","last_clean":"2024-04-25T00:08:16.053207+0000","last_became_active":"2024-04-25T00:08:13.698906+0000","last_became_peered":"2024-04-25T00:08:13.698906+0000","last_unstale":"2024-04-25T00:08:16.053207+0000","last_undegraded":"2024-04-25T00:08:16.053207+0000","last_fullsized":"2024-04-25T00:08:16.053207+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:39:21.734909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053580+0000","last_change":"2024-04-25T00:08:14.704282+0000","last_active":"2024-04-25T00:08:16.053580+0000","last_peered":"2024-04-25T00:08:16.053580+0000","last_clean":"2024-04-25T00:08:16.053580+0000","last_became_active":"2024-04-25T00:08:14.704009+0000","last_became_peered":"2024-04-25T00:08:14.704009+0000","last_unstale":"2024-04-25T00:08:16.053580+0000","last_undegraded":"2024-04-25T00:08:16.053580+0000","last_fullsized":"2024-04-25T00:08:16.053580+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:55.520083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054570+0000","last_change":"2024-04-25T00:08:13.707307+0000","last_active":"2024-04-25T00:08:16.054570+0000","last_peered":"2024-04-25T00:08:16.054570+0000","last_clean":"2024-04-25T00:08:16.054570+0000","last_became_active":"2024-04-25T00:08:13.706889+0000","last_became_peered":"2024-04-25T00:08:13.706889+0000","last_unstale":"2024-04-25T00:08:16.054570+0000","last_undegraded":"2024-04-25T00:08:16.054570+0000","last_fullsized":"2024-04-25T00:08:16.054570+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:52.499092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053078+0000","last_change":"2024-04-25T00:08:14.697118+0000","last_active":"2024-04-25T00:08:16.053078+0000","last_peered":"2024-04-25T00:08:16.053078+0000","last_clean":"2024-04-25T00:08:16.053078+0000","last_became_active":"2024-04-25T00:08:14.696837+0000","last_became_peered":"2024-04-25T00:08:14.696837+0000","last_unstale":"2024-04-25T00:08:16.053078+0000","last_undegraded":"2024-04-25T00:08:16.053078+0000","last_fullsized":"2024-04-25T00:08:16.053078+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:10.675509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053629+0000","last_change":"2024-04-25T00:08:13.702265+0000","last_active":"2024-04-25T00:08:16.053629+0000","last_peered":"2024-04-25T00:08:16.053629+0000","last_clean":"2024-04-25T00:08:16.053629+0000","last_became_active":"2024-04-25T00:08:13.701970+0000","last_became_peered":"2024-04-25T00:08:13.701970+0000","last_unstale":"2024-04-25T00:08:16.053629+0000","last_undegraded":"2024-04-25T00:08:16.053629+0000","last_fullsized":"2024-04-25T00:08:16.053629+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:29.563404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053517+0000","last_change":"2024-04-25T00:08:14.706118+0000","last_active":"2024-04-25T00:08:16.053517+0000","last_peered":"2024-04-25T00:08:16.053517+0000","last_clean":"2024-04-25T00:08:16.053517+0000","last_became_active":"2024-04-25T00:08:14.705368+0000","last_became_peered":"2024-04-25T00:08:14.705368+0000","last_unstale":"2024-04-25T00:08:16.053517+0000","last_undegraded":"2024-04-25T00:08:16.053517+0000","last_fullsized":"2024-04-25T00:08:16.053517+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:03.543592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054107+0000","last_change":"2024-04-25T00:08:13.702965+0000","last_active":"2024-04-25T00:08:16.054107+0000","last_peered":"2024-04-25T00:08:16.054107+0000","last_clean":"2024-04-25T00:08:16.054107+0000","last_became_active":"2024-04-25T00:08:13.702084+0000","last_became_peered":"2024-04-25T00:08:13.702084+0000","last_unstale":"2024-04-25T00:08:16.054107+0000","last_undegraded":"2024-04-25T00:08:16.054107+0000","last_fullsized":"2024-04-25T00:08:16.054107+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:09.670434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702421+0000","last_change":"2024-04-25T00:08:14.699402+0000","last_active":"2024-04-25T00:08:15.702421+0000","last_peered":"2024-04-25T00:08:15.702421+0000","last_clean":"2024-04-25T00:08:15.702421+0000","last_became_active":"2024-04-25T00:08:14.699149+0000","last_became_peered":"2024-04-25T00:08:14.699149+0000","last_unstale":"2024-04-25T00:08:15.702421+0000","last_undegraded":"2024-04-25T00:08:15.702421+0000","last_fullsized":"2024-04-25T00:08:15.702421+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:22.507792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053744+0000","last_change":"2024-04-25T00:08:13.695341+0000","last_active":"2024-04-25T00:08:16.053744+0000","last_peered":"2024-04-25T00:08:16.053744+0000","last_clean":"2024-04-25T00:08:16.053744+0000","last_became_active":"2024-04-25T00:08:13.695009+0000","last_became_peered":"2024-04-25T00:08:13.695009+0000","last_unstale":"2024-04-25T00:08:16.053744+0000","last_undegraded":"2024-04-25T00:08:16.053744+0000","last_fullsized":"2024-04-25T00:08:16.053744+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:20.030924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053042+0000","last_change":"2024-04-25T00:08:14.714235+0000","last_active":"2024-04-25T00:08:16.053042+0000","last_peered":"2024-04-25T00:08:16.053042+0000","last_clean":"2024-04-25T00:08:16.053042+0000","last_became_active":"2024-04-25T00:08:14.711984+0000","last_became_peered":"2024-04-25T00:08:14.711984+0000","last_unstale":"2024-04-25T00:08:16.053042+0000","last_undegraded":"2024-04-25T00:08:16.053042+0000","last_fullsized":"2024-04-25T00:08:16.053042+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:39.826995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053371+0000","last_change":"2024-04-25T00:08:13.704841+0000","last_active":"2024-04-25T00:08:16.053371+0000","last_peered":"2024-04-25T00:08:16.053371+0000","last_clean":"2024-04-25T00:08:16.053371+0000","last_became_active":"2024-04-25T00:08:13.704550+0000","last_became_peered":"2024-04-25T00:08:13.704550+0000","last_unstale":"2024-04-25T00:08:16.053371+0000","last_undegraded":"2024-04-25T00:08:16.053371+0000","last_fullsized":"2024-04-25T00:08:16.053371+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:25.752983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702756+0000","last_change":"2024-04-25T00:08:14.717098+0000","last_active":"2024-04-25T00:08:15.702756+0000","last_peered":"2024-04-25T00:08:15.702756+0000","last_clean":"2024-04-25T00:08:15.702756+0000","last_became_active":"2024-04-25T00:08:14.716806+0000","last_became_peered":"2024-04-25T00:08:14.716806+0000","last_unstale":"2024-04-25T00:08:15.702756+0000","last_undegraded":"2024-04-25T00:08:15.702756+0000","last_fullsized":"2024-04-25T00:08:15.702756+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:29.924646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053005+0000","last_change":"2024-04-25T00:08:13.704231+0000","last_active":"2024-04-25T00:08:16.053005+0000","last_peered":"2024-04-25T00:08:16.053005+0000","last_clean":"2024-04-25T00:08:16.053005+0000","last_became_active":"2024-04-25T00:08:13.702037+0000","last_became_peered":"2024-04-25T00:08:13.702037+0000","last_unstale":"2024-04-25T00:08:16.053005+0000","last_undegraded":"2024-04-25T00:08:16.053005+0000","last_fullsized":"2024-04-25T00:08:16.053005+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:36.481898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053601+0000","last_change":"2024-04-25T00:08:14.717219+0000","last_active":"2024-04-25T00:08:16.053601+0000","last_peered":"2024-04-25T00:08:16.053601+0000","last_clean":"2024-04-25T00:08:16.053601+0000","last_became_active":"2024-04-25T00:08:14.716942+0000","last_became_peered":"2024-04-25T00:08:14.716942+0000","last_unstale":"2024-04-25T00:08:16.053601+0000","last_undegraded":"2024-04-25T00:08:16.053601+0000","last_fullsized":"2024-04-25T00:08:16.053601+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:26.031254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053074+0000","last_change":"2024-04-25T00:08:13.706523+0000","last_active":"2024-04-25T00:08:16.053074+0000","last_peered":"2024-04-25T00:08:16.053074+0000","last_clean":"2024-04-25T00:08:16.053074+0000","last_became_active":"2024-04-25T00:08:13.706264+0000","last_became_peered":"2024-04-25T00:08:13.706264+0000","last_unstale":"2024-04-25T00:08:16.053074+0000","last_undegraded":"2024-04-25T00:08:16.053074+0000","last_fullsized":"2024-04-25T00:08:16.053074+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:31:06.006859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702329+0000","last_change":"2024-04-25T00:08:14.703062+0000","last_active":"2024-04-25T00:08:15.702329+0000","last_peered":"2024-04-25T00:08:15.702329+0000","last_clean":"2024-04-25T00:08:15.702329+0000","last_became_active":"2024-04-25T00:08:14.702785+0000","last_became_peered":"2024-04-25T00:08:14.702785+0000","last_unstale":"2024-04-25T00:08:15.702329+0000","last_undegraded":"2024-04-25T00:08:15.702329+0000","last_fullsized":"2024-04-25T00:08:15.702329+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:25.523552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053707+0000","last_change":"2024-04-25T00:08:13.704556+0000","last_active":"2024-04-25T00:08:16.053707+0000","last_peered":"2024-04-25T00:08:16.053707+0000","last_clean":"2024-04-25T00:08:16.053707+0000","last_became_active":"2024-04-25T00:08:13.704237+0000","last_became_peered":"2024-04-25T00:08:13.704237+0000","last_unstale":"2024-04-25T00:08:16.053707+0000","last_undegraded":"2024-04-25T00:08:16.053707+0000","last_fullsized":"2024-04-25T00:08:16.053707+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:36.589578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052958+0000","last_change":"2024-04-25T00:08:14.705549+0000","last_active":"2024-04-25T00:08:16.052958+0000","last_peered":"2024-04-25T00:08:16.052958+0000","last_clean":"2024-04-25T00:08:16.052958+0000","last_became_active":"2024-04-25T00:08:14.704198+0000","last_became_peered":"2024-04-25T00:08:14.704198+0000","last_unstale":"2024-04-25T00:08:16.052958+0000","last_undegraded":"2024-04-25T00:08:16.052958+0000","last_fullsized":"2024-04-25T00:08:16.052958+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:37.486973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053159+0000","last_change":"2024-04-25T00:08:13.703254+0000","last_active":"2024-04-25T00:08:16.053159+0000","last_peered":"2024-04-25T00:08:16.053159+0000","last_clean":"2024-04-25T00:08:16.053159+0000","last_became_active":"2024-04-25T00:08:13.702987+0000","last_became_peered":"2024-04-25T00:08:13.702987+0000","last_unstale":"2024-04-25T00:08:16.053159+0000","last_undegraded":"2024-04-25T00:08:16.053159+0000","last_fullsized":"2024-04-25T00:08:16.053159+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:20.841121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702577+0000","last_change":"2024-04-25T00:08:14.701092+0000","last_active":"2024-04-25T00:08:15.702577+0000","last_peered":"2024-04-25T00:08:15.702577+0000","last_clean":"2024-04-25T00:08:15.702577+0000","last_became_active":"2024-04-25T00:08:14.700825+0000","last_became_peered":"2024-04-25T00:08:14.700825+0000","last_unstale":"2024-04-25T00:08:15.702577+0000","last_undegraded":"2024-04-25T00:08:15.702577+0000","last_fullsized":"2024-04-25T00:08:15.702577+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:37.594653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053759+0000","last_change":"2024-04-25T00:08:13.705008+0000","last_active":"2024-04-25T00:08:16.053759+0000","last_peered":"2024-04-25T00:08:16.053759+0000","last_clean":"2024-04-25T00:08:16.053759+0000","last_became_active":"2024-04-25T00:08:13.704758+0000","last_became_peered":"2024-04-25T00:08:13.704758+0000","last_unstale":"2024-04-25T00:08:16.053759+0000","last_undegraded":"2024-04-25T00:08:16.053759+0000","last_fullsized":"2024-04-25T00:08:16.053759+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:38.185882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702284+0000","last_change":"2024-04-25T00:08:14.711078+0000","last_active":"2024-04-25T00:08:15.702284+0000","last_peered":"2024-04-25T00:08:15.702284+0000","last_clean":"2024-04-25T00:08:15.702284+0000","last_became_active":"2024-04-25T00:08:14.710773+0000","last_became_peered":"2024-04-25T00:08:14.710773+0000","last_unstale":"2024-04-25T00:08:15.702284+0000","last_undegraded":"2024-04-25T00:08:15.702284+0000","last_fullsized":"2024-04-25T00:08:15.702284+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:28:04.457684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.703111+0000","last_change":"2024-04-25T00:08:13.703390+0000","last_active":"2024-04-25T00:08:15.703111+0000","last_peered":"2024-04-25T00:08:15.703111+0000","last_clean":"2024-04-25T00:08:15.703111+0000","last_became_active":"2024-04-25T00:08:13.703022+0000","last_became_peered":"2024-04-25T00:08:13.703022+0000","last_unstale":"2024-04-25T00:08:15.703111+0000","last_undegraded":"2024-04-25T00:08:15.703111+0000","last_fullsized":"2024-04-25T00:08:15.703111+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:27.705314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052720+0000","last_change":"2024-04-25T00:08:14.716945+0000","last_active":"2024-04-25T00:08:16.052720+0000","last_peered":"2024-04-25T00:08:16.052720+0000","last_clean":"2024-04-25T00:08:16.052720+0000","last_became_active":"2024-04-25T00:08:14.716660+0000","last_became_peered":"2024-04-25T00:08:14.716660+0000","last_unstale":"2024-04-25T00:08:16.052720+0000","last_undegraded":"2024-04-25T00:08:16.052720+0000","last_fullsized":"2024-04-25T00:08:16.052720+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:03.079423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052732+0000","last_change":"2024-04-25T00:08:13.706632+0000","last_active":"2024-04-25T00:08:16.052732+0000","last_peered":"2024-04-25T00:08:16.052732+0000","last_clean":"2024-04-25T00:08:16.052732+0000","last_became_active":"2024-04-25T00:08:13.706307+0000","last_became_peered":"2024-04-25T00:08:13.706307+0000","last_unstale":"2024-04-25T00:08:16.052732+0000","last_undegraded":"2024-04-25T00:08:16.052732+0000","last_fullsized":"2024-04-25T00:08:16.052732+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:05.838537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052612+0000","last_change":"2024-04-25T00:08:14.701762+0000","last_active":"2024-04-25T00:08:16.052612+0000","last_peered":"2024-04-25T00:08:16.052612+0000","last_clean":"2024-04-25T00:08:16.052612+0000","last_became_active":"2024-04-25T00:08:14.701168+0000","last_became_peered":"2024-04-25T00:08:14.701168+0000","last_unstale":"2024-04-25T00:08:16.052612+0000","last_undegraded":"2024-04-25T00:08:16.052612+0000","last_fullsized":"2024-04-25T00:08:16.052612+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:55.854379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054126+0000","last_change":"2024-04-25T00:08:13.702241+0000","last_active":"2024-04-25T00:08:16.054126+0000","last_peered":"2024-04-25T00:08:16.054126+0000","last_clean":"2024-04-25T00:08:16.054126+0000","last_became_active":"2024-04-25T00:08:13.700411+0000","last_became_peered":"2024-04-25T00:08:13.700411+0000","last_unstale":"2024-04-25T00:08:16.054126+0000","last_undegraded":"2024-04-25T00:08:16.054126+0000","last_fullsized":"2024-04-25T00:08:16.054126+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:07.585953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.054107+0000","last_change":"2024-04-25T00:08:14.706523+0000","last_active":"2024-04-25T00:08:16.054107+0000","last_peered":"2024-04-25T00:08:16.054107+0000","last_clean":"2024-04-25T00:08:16.054107+0000","last_became_active":"2024-04-25T00:08:14.706185+0000","last_became_peered":"2024-04-25T00:08:14.706185+0000","last_unstale":"2024-04-25T00:08:16.054107+0000","last_undegraded":"2024-04-25T00:08:16.054107+0000","last_fullsized":"2024-04-25T00:08:16.054107+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:53.406643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053091+0000","last_change":"2024-04-25T00:08:13.703855+0000","last_active":"2024-04-25T00:08:16.053091+0000","last_peered":"2024-04-25T00:08:16.053091+0000","last_clean":"2024-04-25T00:08:16.053091+0000","last_became_active":"2024-04-25T00:08:13.703589+0000","last_became_peered":"2024-04-25T00:08:13.703589+0000","last_unstale":"2024-04-25T00:08:16.053091+0000","last_undegraded":"2024-04-25T00:08:16.053091+0000","last_fullsized":"2024-04-25T00:08:16.053091+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:05.312619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702736+0000","last_change":"2024-04-25T00:08:14.717759+0000","last_active":"2024-04-25T00:08:15.702736+0000","last_peered":"2024-04-25T00:08:15.702736+0000","last_clean":"2024-04-25T00:08:15.702736+0000","last_became_active":"2024-04-25T00:08:14.716906+0000","last_became_peered":"2024-04-25T00:08:14.716906+0000","last_unstale":"2024-04-25T00:08:15.702736+0000","last_undegraded":"2024-04-25T00:08:15.702736+0000","last_fullsized":"2024-04-25T00:08:15.702736+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:53.829710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:15.702959+0000","last_change":"2024-04-25T00:08:13.703232+0000","last_active":"2024-04-25T00:08:15.702959+0000","last_peered":"2024-04-25T00:08:15.702959+0000","last_clean":"2024-04-25T00:08:15.702959+0000","last_became_active":"2024-04-25T00:08:13.702930+0000","last_became_peered":"2024-04-25T00:08:13.702930+0000","last_unstale":"2024-04-25T00:08:15.702959+0000","last_undegraded":"2024-04-25T00:08:15.702959+0000","last_fullsized":"2024-04-25T00:08:15.702959+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:40.038116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053268+0000","last_change":"2024-04-25T00:08:14.709152+0000","last_active":"2024-04-25T00:08:16.053268+0000","last_peered":"2024-04-25T00:08:16.053268+0000","last_clean":"2024-04-25T00:08:16.053268+0000","last_became_active":"2024-04-25T00:08:14.707342+0000","last_became_peered":"2024-04-25T00:08:14.707342+0000","last_unstale":"2024-04-25T00:08:16.053268+0000","last_undegraded":"2024-04-25T00:08:16.053268+0000","last_fullsized":"2024-04-25T00:08:16.053268+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:28.710390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053318+0000","last_change":"2024-04-25T00:08:13.701962+0000","last_active":"2024-04-25T00:08:16.053318+0000","last_peered":"2024-04-25T00:08:16.053318+0000","last_clean":"2024-04-25T00:08:16.053318+0000","last_became_active":"2024-04-25T00:08:13.701654+0000","last_became_peered":"2024-04-25T00:08:13.701654+0000","last_unstale":"2024-04-25T00:08:16.053318+0000","last_undegraded":"2024-04-25T00:08:16.053318+0000","last_fullsized":"2024-04-25T00:08:16.053318+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:38:09.000563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.052958+0000","last_change":"2024-04-25T00:08:14.705872+0000","last_active":"2024-04-25T00:08:16.052958+0000","last_peered":"2024-04-25T00:08:16.052958+0000","last_clean":"2024-04-25T00:08:16.052958+0000","last_became_active":"2024-04-25T00:08:14.705302+0000","last_became_peered":"2024-04-25T00:08:14.705302+0000","last_unstale":"2024-04-25T00:08:16.052958+0000","last_undegraded":"2024-04-25T00:08:16.052958+0000","last_fullsized":"2024-04-25T00:08:16.052958+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:13.682995+0000","last_clean_scrub_stamp":"2024-04-25T00:08:13.682995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:40.729436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053226+0000","last_change":"2024-04-25T00:08:13.700172+0000","last_active":"2024-04-25T00:08:16.053226+0000","last_peered":"2024-04-25T00:08:16.053226+0000","last_clean":"2024-04-25T00:08:16.053226+0000","last_became_active":"2024-04-25T00:08:13.699775+0000","last_became_peered":"2024-04-25T00:08:13.699775+0000","last_unstale":"2024-04-25T00:08:16.053226+0000","last_undegraded":"2024-04-25T00:08:16.053226+0000","last_fullsized":"2024-04-25T00:08:16.053226+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:08:12.677920+0000","last_clean_scrub_stamp":"2024-04-25T00:08:12.677920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:27.764780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":473,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T00:08:16.053766+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:08:16.053766+0000","last_peered":"2024-04-25T00:08:16.053766+0000","last_clean":"2024-04-25T00:08:16.053766+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:08:16.053766+0000","last_undegraded":"2024-04-25T00:08:16.053766+0000","last_fullsized":"2024-04-25T00:08:16.053766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47596,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324244,"statfs":{"total":96636764160,"available":96588025856,"internally_reserved":0,"allocated":2588672,"data_stored":1455312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51176,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320664,"statfs":{"total":96636764160,"available":96584359936,"internally_reserved":0,"allocated":1986560,"data_stored":868912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55320,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316520,"statfs":{"total":96636764160,"available":96580116480,"internally_reserved":0,"allocated":1978368,"data_stored":863194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47016,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324824,"statfs":{"total":96636764160,"available":96588619776,"internally_reserved":0,"allocated":1986560,"data_stored":864272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51160,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320680,"statfs":{"total":96636764160,"available":96584376320,"internally_reserved":0,"allocated":1974272,"data_stored":862624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51168,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320672,"statfs":{"total":96636764160,"available":96584368128,"internally_reserved":0,"allocated":1986560,"data_stored":868980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55320,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316520,"statfs":{"total":96636764160,"available":96580116480,"internally_reserved":0,"allocated":1974272,"data_stored":862624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47588,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324252,"statfs":{"total":96636764160,"available":96588034048,"internally_reserved":0,"allocated":2576384,"data_stored":1459258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:08:18.168 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:08:18.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:08:18.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.308+0000 7f3343fff640 1 Processor -- start 2024-04-25T00:08:18.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.308+0000 7f3343fff640 1 -- start start 2024-04-25T00:08:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3343fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3344144520 0x7f33441627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3343fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce730 0x7f33440c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3343fff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33440cbf40 con 0x7f33440ce2d0 2024-04-25T00:08:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3343fff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3344162cf0 con 0x7f3344144520 2024-04-25T00:08:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3343fff640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f33440cbab0 con 0x7f33440ce730 2024-04-25T00:08:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3342ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3344144520 0x7f33441627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f333bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f33440ce2d0 0x7f33440c8e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46798/0 (socket says 172.21.15.113:46798) 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f333bfff640 1 -- 172.21.15.113:0/2544210821 learned_addr learned my addr 172.21.15.113:0/2544210821 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f33437fe640 1 --2- 172.21.15.113:0/2544210821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce730 0x7f33440c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3342ffd640 1 -- 172.21.15.113:0/2544210821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce730 msgr2=0x7f33440c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3342ffd640 1 --2- 172.21.15.113:0/2544210821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce730 0x7f33440c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3342ffd640 1 -- 172.21.15.113:0/2544210821 >> v1:172.21.15.113:6789/0 conn(0x7f33440ce2d0 legacy=0x7f33440c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f33427fc640 1 -- 172.21.15.113:0/2544210821 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4169710554 0 0) 0x7f332c001110 con 0x7f33440ce2d0 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.309+0000 7f3342ffd640 1 -- 172.21.15.113:0/2544210821 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33440caea0 con 0x7f3344144520 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.310+0000 7f3342ffd640 1 --2- 172.21.15.113:0/2544210821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441627b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f33340097a0 tx=0x7f333402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f649c6bef283c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.310+0000 7f33427fc640 1 -- 172.21.15.113:0/2544210821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334037070 con 0x7f3344144520 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.310+0000 7f33427fc640 1 -- 172.21.15.113:0/2544210821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3334032440 con 0x7f3344144520 2024-04-25T00:08:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.310+0000 7f33427fc640 1 -- 172.21.15.113:0/2544210821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334032810 con 0x7f3344144520 2024-04-25T00:08:18.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 -- 172.21.15.113:0/2544210821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 msgr2=0x7f33441627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 --2- 172.21.15.113:0/2544210821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441627b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f33340097a0 tx=0x7f333402a180 comp rx=0 tx=0).stop 2024-04-25T00:08:18.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 -- 172.21.15.113:0/2544210821 shutdown_connections 2024-04-25T00:08:18.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 --2- 172.21.15.113:0/2544210821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce730 0x7f33440c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 --2- 172.21.15.113:0/2544210821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441627b0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 -- 172.21.15.113:0/2544210821 >> 172.21.15.113:0/2544210821 conn(0x7f33440cfb20 msgr2=0x7f33440cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 -- 172.21.15.113:0/2544210821 shutdown_connections 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.311+0000 7f3343fff640 1 -- 172.21.15.113:0/2544210821 wait complete. 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.312+0000 7f3343fff640 1 Processor -- start 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.312+0000 7f3343fff640 1 -- start start 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce2d0 0x7f33440b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33440ce730 0x7f33441624e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f33440cb740 con 0x7f3344144520 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33440cb460 con 0x7f33440ce730 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f33440cebc0 con 0x7f33440ce2d0 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48348/0 (socket says 172.21.15.113:48348) 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 -- 172.21.15.113:0/94725195 learned_addr learned my addr 172.21.15.113:0/94725195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:18.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3342ffd640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce2d0 0x7f33440b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 -- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce2d0 msgr2=0x7f33440b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce2d0 0x7f33440b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 -- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33440ce730 msgr2=0x7f33441624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f333bfff640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33440ce730 0x7f33441624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33440ce730 0x7f33441624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 -- 172.21.15.113:0/94725195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33441598c0 con 0x7f3344144520 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3342ffd640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce2d0 0x7f33440b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f33437fe640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441f3e20 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f333c007e40 tx=0x7f333c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3340ff9640 1 -- 172.21.15.113:0/94725195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c014620 con 0x7f3344144520 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3340ff9640 1 -- 172.21.15.113:0/94725195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c019070 con 0x7f3344144520 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3340ff9640 1 -- 172.21.15.113:0/94725195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c014a20 con 0x7f3344144520 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33441f4dc0 con 0x7f3344144520 2024-04-25T00:08:18.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.313+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33441f4800 con 0x7f3344144520 2024-04-25T00:08:18.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.314+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33440691a0 con 0x7f3344144520 2024-04-25T00:08:18.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.315+0000 7f3340ff9640 1 -- 172.21.15.113:0/94725195 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f333c014bc0 con 0x7f3344144520 2024-04-25T00:08:18.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.315+0000 7f3340ff9640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f332406be30 0x7f332406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.315+0000 7f3342ffd640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f332406be30 0x7f332406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.315+0000 7f3340ff9640 1 -- 172.21.15.113:0/94725195 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f333c08b6e0 con 0x7f3344144520 2024-04-25T00:08:18.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.316+0000 7f3342ffd640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f332406be30 0x7f332406e2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3334034000 tx=0x7f333402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:18.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.320+0000 7f3340ff9640 1 -- 172.21.15.113:0/94725195 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33441598c0 con 0x7f3344144520 2024-04-25T00:08:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.556+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3344159370 con 0x7f3344144520 2024-04-25T00:08:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.558+0000 7f3340ff9640 1 -- 172.21.15.113:0/94725195 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f333c05a120 con 0x7f3344144520 2024-04-25T00:08:18.559 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:08:18.559 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:08:18.559 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:08:18.559 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:08:18.559 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:08:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f332406be30 msgr2=0x7f332406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f332406be30 0x7f332406e2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3334034000 tx=0x7f333402e000 comp rx=0 tx=0).stop 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 msgr2=0x7f33441f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441f3e20 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f333c007e40 tx=0x7f333c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 shutdown_connections 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f332406be30 0x7f332406e2f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33440ce730 0x7f33441624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3344144520 0x7f33441f3e20 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 --2- 172.21.15.113:0/94725195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33440ce2d0 0x7f33440b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 >> 172.21.15.113:0/94725195 conn(0x7f33440cfb20 msgr2=0x7f3344142ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 shutdown_connections 2024-04-25T00:08:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.560+0000 7f3343fff640 1 -- 172.21.15.113:0/94725195 wait complete. 2024-04-25T00:08:18.572 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:18.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.668+0000 7f673beae640 1 Processor -- start 2024-04-25T00:08:18.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.668+0000 7f673beae640 1 -- start start 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673beae640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f67340b3fd0 0x7f673406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673beae640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f673406e510 0x7f67340cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673beae640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6734144cf0 con 0x7f6734144910 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673beae640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f673406e980 con 0x7f673406e510 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673beae640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f67340b2f30 con 0x7f67340b3fd0 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673a44b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f673406e510 0x7f67340cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f6739c4a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6734144910 0x7f67340b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46814/0 (socket says 172.21.15.113:46814) 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f6739c4a640 1 -- 172.21.15.113:0/2026386784 learned_addr learned my addr 172.21.15.113:0/2026386784 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f6739449640 1 --2- 172.21.15.113:0/2026386784 >> v2:172.21.15.178:3300/0 conn(0x7f67340b3fd0 0x7f673406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673a44b640 1 -- 172.21.15.113:0/2026386784 >> v2:172.21.15.178:3300/0 conn(0x7f67340b3fd0 msgr2=0x7f673406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:18.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673a44b640 1 --2- 172.21.15.113:0/2026386784 >> v2:172.21.15.178:3300/0 conn(0x7f67340b3fd0 0x7f673406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673a44b640 1 -- 172.21.15.113:0/2026386784 >> v1:172.21.15.113:6789/0 conn(0x7f6734144910 legacy=0x7f67340b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f6738c48640 1 -- 172.21.15.113:0/2026386784 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1224261607 0 0) 0x7f6734144cf0 con 0x7f6734144910 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f673a44b640 1 -- 172.21.15.113:0/2026386784 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67340b3210 con 0x7f673406e510 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.669+0000 7f6739449640 1 --2- 172.21.15.113:0/2026386784 >> v2:172.21.15.178:3300/0 conn(0x7f67340b3fd0 0x7f673406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f673a44b640 1 --2- 172.21.15.113:0/2026386784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f673406e510 0x7f67340cd500 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f672c009d90 tx=0x7f672c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c0861e82490c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f6738c48640 1 -- 172.21.15.113:0/2026386784 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c007740 con 0x7f673406e510 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f6738c48640 1 -- 172.21.15.113:0/2026386784 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c0078e0 con 0x7f673406e510 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f6738c48640 1 -- 172.21.15.113:0/2026386784 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c007cb0 con 0x7f673406e510 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f673beae640 1 -- 172.21.15.113:0/2026386784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f673406e510 msgr2=0x7f67340cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f673beae640 1 --2- 172.21.15.113:0/2026386784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f673406e510 0x7f67340cd500 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f672c009d90 tx=0x7f672c02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f673beae640 1 -- 172.21.15.113:0/2026386784 shutdown_connections 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f673beae640 1 --2- 172.21.15.113:0/2026386784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f673406e510 0x7f67340cd500 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f673beae640 1 --2- 172.21.15.113:0/2026386784 >> v2:172.21.15.178:3300/0 conn(0x7f67340b3fd0 0x7f673406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.670+0000 7f673beae640 1 -- 172.21.15.113:0/2026386784 >> 172.21.15.113:0/2026386784 conn(0x7f67340b1a50 msgr2=0x7f67340b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:18.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.671+0000 7f673beae640 1 -- 172.21.15.113:0/2026386784 shutdown_connections 2024-04-25T00:08:18.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.671+0000 7f673beae640 1 -- 172.21.15.113:0/2026386784 wait complete. 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 Processor -- start 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 -- start start 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f673406e510 0x7f67340cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67340b3fd0 0x7f673415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6734144910 0x7f673415f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67340b4440 con 0x7f6734144910 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67340b37d0 con 0x7f67340b3fd0 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673beae640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f67340b4da0 con 0x7f673406e510 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673a44b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6734144910 0x7f673415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673a44b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6734144910 0x7f673415f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48362/0 (socket says 172.21.15.113:48362) 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.672+0000 7f673a44b640 1 -- 172.21.15.113:0/1131380176 learned_addr learned my addr 172.21.15.113:0/1131380176 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f6739449640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67340b3fd0 0x7f673415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673a44b640 1 -- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f673406e510 msgr2=0x7f67340cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f6739c4a640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f673406e510 0x7f67340cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673a44b640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f673406e510 0x7f67340cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673a44b640 1 -- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67340b3fd0 msgr2=0x7f673415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673a44b640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67340b3fd0 0x7f673415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673a44b640 1 -- 172.21.15.113:0/1131380176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67340ae320 con 0x7f6734144910 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f6739449640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67340b3fd0 0x7f673415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673a44b640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6734144910 0x7f673415f050 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f672c009d90 tx=0x7f672c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:18.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f672a7fc640 1 -- 172.21.15.113:0/1131380176 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c040070 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f672a7fc640 1 -- 172.21.15.113:0/1131380176 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c031c40 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f672a7fc640 1 -- 172.21.15.113:0/1131380176 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c003990 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67340ae600 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.673+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673414eb70 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.674+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67340ae800 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.675+0000 7f672a7fc640 1 -- 172.21.15.113:0/1131380176 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f672c039040 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.675+0000 7f672a7fc640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f671806be30 0x7f671806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.675+0000 7f672a7fc640 1 -- 172.21.15.113:0/1131380176 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f672c0abd60 con 0x7f6734144910 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.675+0000 7f6739c4a640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f671806be30 0x7f671806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.675+0000 7f6739c4a640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f671806be30 0x7f671806e2f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6730000f60 tx=0x7f6730009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.679+0000 7f672a7fc640 1 -- 172.21.15.113:0/1131380176 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f672c07a7a0 con 0x7f6734144910 2024-04-25T00:08:18.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.892+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f673414ed70 con 0x7f6734144910 2024-04-25T00:08:18.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.892+0000 7f672a7fc640 1 -- 172.21.15.113:0/1131380176 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7f672c07a0a0 con 0x7f6734144910 2024-04-25T00:08:18.893 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:18.893 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:18.893 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:18.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f671806be30 msgr2=0x7f671806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f671806be30 0x7f671806e2f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6730000f60 tx=0x7f6730009040 comp rx=0 tx=0).stop 2024-04-25T00:08:18.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6734144910 msgr2=0x7f673415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6734144910 0x7f673415f050 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f672c009d90 tx=0x7f672c004a40 comp rx=0 tx=0).stop 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 shutdown_connections 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f671806be30 0x7f671806e2f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67340b3fd0 0x7f673415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6734144910 0x7f673415f050 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 --2- 172.21.15.113:0/1131380176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f673406e510 0x7f67340cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 >> 172.21.15.113:0/1131380176 conn(0x7f67340b1a50 msgr2=0x7f67340b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 shutdown_connections 2024-04-25T00:08:18.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:18.895+0000 7f673beae640 1 -- 172.21.15.113:0/1131380176 wait complete. 2024-04-25T00:08:19.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:08:14.036250+0000', 'modified': '2024-04-25T00:08:16.209381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9987': {'gid': 9987, 'name': 'b', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6834/340854702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 340854702}, {'type': 'v1', 'addr': '172.21.15.178:6834', 'nonce': 340854702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9987, 'qdb_cluster': [9987]} 2024-04-25T00:08:19.310 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:08:19.310 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:08:19.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.411+0000 7f8d01d9d640 1 Processor -- start 2024-04-25T00:08:19.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.411+0000 7f8d01d9d640 1 -- start start 2024-04-25T00:08:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.411+0000 7f8d01d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8cfc15e510 0x7f8cfc0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.411+0000 7f8d01d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 0x7f8cfc0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.411+0000 7f8d01d9d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cfc0d2e00 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.411+0000 7f8d01d9d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8cfc0d30e0 con 0x7f8cfc15e510 2024-04-25T00:08:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.411+0000 7f8d01d9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc0d33c0 con 0x7f8cfc0d28c0 2024-04-25T00:08:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8cfc15e510 0x7f8cfc0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfbfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8cfc0b61e0 0x7f8cfc071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46826/0 (socket says 172.21.15.113:46826) 2024-04-25T00:08:19.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfbfff640 1 -- 172.21.15.113:0/1246832160 learned_addr learned my addr 172.21.15.113:0/1246832160 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:19.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/1246832160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 msgr2=0x7f8cfc0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/1246832160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 0x7f8cfc0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/1246832160 >> v1:172.21.15.113:6789/0 conn(0x7f8cfc0b61e0 legacy=0x7f8cfc071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/1246832160 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc0d36a0 con 0x7f8cfc15e510 2024-04-25T00:08:19.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfaffd640 1 -- 172.21.15.113:0/1246832160 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888710247 0 0) 0x7f8cfc0d2e00 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.412+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/1246832160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc15e510 0x7f8cfc0d2380 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0097a0 tx=0x7f8cec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0a0c554b38c495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8cfaffd640 1 -- 172.21.15.113:0/1246832160 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec037070 con 0x7f8cfc15e510 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8cfaffd640 1 -- 172.21.15.113:0/1246832160 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec02ae50 con 0x7f8cfc15e510 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8cfaffd640 1 -- 172.21.15.113:0/1246832160 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec032660 con 0x7f8cfc15e510 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 -- 172.21.15.113:0/1246832160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc15e510 msgr2=0x7f8cfc0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/1246832160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc15e510 0x7f8cfc0d2380 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0097a0 tx=0x7f8cec02a100 comp rx=0 tx=0).stop 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 -- 172.21.15.113:0/1246832160 shutdown_connections 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/1246832160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 0x7f8cfc0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/1246832160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc15e510 0x7f8cfc0d2380 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 -- 172.21.15.113:0/1246832160 >> 172.21.15.113:0/1246832160 conn(0x7f8cfc06b720 msgr2=0x7f8cfc0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 -- 172.21.15.113:0/1246832160 shutdown_connections 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.413+0000 7f8d01d9d640 1 -- 172.21.15.113:0/1246832160 wait complete. 2024-04-25T00:08:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.414+0000 7f8d01d9d640 1 Processor -- start 2024-04-25T00:08:19.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.414+0000 7f8d01d9d640 1 -- start start 2024-04-25T00:08:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8d01d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0b61e0 0x7f8cfc1f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8d01d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 0x7f8cfc0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8d01d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc15e510 0x7f8cfc0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8d01d9d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b9dd0 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8d01d9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0d3ea0 con 0x7f8cfc15e510 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8d01d9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b83d0 con 0x7f8cfc0d28c0 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0b61e0 0x7f8cfc1f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0b61e0 0x7f8cfc1f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48388/0 (socket says 172.21.15.113:48388) 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/337283859 learned_addr learned my addr 172.21.15.113:0/337283859 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/337283859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 msgr2=0x7f8cfc0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 0x7f8cfc0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/337283859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc15e510 msgr2=0x7f8cfc0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc15e510 0x7f8cfc0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.415+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/337283859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc0c9840 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.416+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0b61e0 0x7f8cfc1f37c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098c0 tx=0x7f8cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.416+0000 7f8cf8ff9640 1 -- 172.21.15.113:0/337283859 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec041020 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.416+0000 7f8cf8ff9640 1 -- 172.21.15.113:0/337283859 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec03c870 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.416+0000 7f8cf8ff9640 1 -- 172.21.15.113:0/337283859 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec037040 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.416+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc0c87a0 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.416+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc0c9040 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.418+0000 7f8cf8ff9640 1 -- 172.21.15.113:0/337283859 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cec044020 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.418+0000 7f8cf8ff9640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cdc06be30 0x7f8cdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.418+0000 7f8cf8ff9640 1 -- 172.21.15.113:0/337283859 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f8cec0ab5d0 con 0x7f8cfc0b61e0 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.418+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cdc06be30 0x7f8cdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.418+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cdc06be30 0x7f8cdc06e2f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc071c30 tx=0x7f8ce4006c90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.419+0000 7f8d04b82740 1 Processor -- start 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.419+0000 7f8d04b82740 1 -- start start 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7f8b60 0x562aba7ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7dbc80 0x562aba7db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7db540 0x562aba7b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8d04b82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562aba713f00 con 0x562aba7f8b60 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562aba6bc750 con 0x562aba7dbc80 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562aba70fec0 con 0x562aba7db540 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7f8b60 0x562aba7ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7f8b60 0x562aba7ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48390/0 (socket says 172.21.15.113:48390) 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/1604148287 learned_addr learned my addr 172.21.15.113:0/1604148287 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7dbc80 0x562aba7db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfbfff640 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7db540 0x562aba7b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7db540 msgr2=0x562aba7b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7db540 0x562aba7b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7dbc80 msgr2=0x562aba7db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7dbc80 0x562aba7db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/1604148287 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aba4282d0 con 0x562aba7f8b60 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7f8b60 0x562aba7ab4f0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8cec075d70 tx=0x7f8cec075bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfc01ea95d490c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cf17fa640 1 -- 172.21.15.113:0/1604148287 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec07dcc0 con 0x562aba7f8b60 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cf17fa640 1 -- 172.21.15.113:0/1604148287 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec07ee50 con 0x562aba7f8b60 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.420+0000 7f8cf17fa640 1 -- 172.21.15.113:0/1604148287 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0824f0 con 0x562aba7f8b60 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 -- 172.21.15.113:0/1604148287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7f8b60 msgr2=0x562aba7ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7f8b60 0x562aba7ab4f0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8cec075d70 tx=0x7f8cec075bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 -- 172.21.15.113:0/1604148287 shutdown_connections 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7dbc80 0x562aba7db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7db540 0x562aba7b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 --2- 172.21.15.113:0/1604148287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7f8b60 0x562aba7ab4f0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 -- 172.21.15.113:0/1604148287 >> 172.21.15.113:0/1604148287 conn(0x562aba7ad760 msgr2=0x562aba7f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 -- 172.21.15.113:0/1604148287 shutdown_connections 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.421+0000 7f8d04b82740 1 -- 172.21.15.113:0/1604148287 wait complete. 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 -- start start 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 Processor -- start 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba7ed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 0x562aba7c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562aba713f00 con 0x562aba7dbc80 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562aba6cfcf0 con 0x562aba7c82e0 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.422+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562aba7af910 con 0x562aba7f8b60 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8d04b82740 10 client.0 _ll_register_callbacks cb 0x562aba6cf6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:19.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba7ed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba7ed660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48394/0 (socket says 172.21.15.113:48394) 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3631808059 learned_addr learned my addr 172.21.15.113:0/3631808059 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 msgr2=0x562aba7f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfbfff640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 0x562aba7c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 msgr2=0x562aba7c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 0x562aba7c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3631808059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aba4f26b0 con 0x562aba7dbc80 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba7ed660 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0798a0 tx=0x7f8cec0ab070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2766dcd706be86dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cda7fc640 1 -- 172.21.15.113:0/3631808059 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec095070 con 0x562aba7dbc80 2024-04-25T00:08:19.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cda7fc640 1 -- 172.21.15.113:0/3631808059 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f8cec079b70 con 0x562aba7dbc80 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cec07ee50 con 0x562aba7dbc80 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cda7fc640 1 -- 172.21.15.113:0/3631808059 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8cec09e050 con 0x562aba7dbc80 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8d04b82740 10 client.10131 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x562aba7dd0d0 con 0x562aba7dbc80 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cda7fc640 1 -- 172.21.15.113:0/3631808059 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec07ee50 con 0x562aba7dbc80 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.423+0000 7f8cda7fc640 1 -- 172.21.15.113:0/3631808059 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7f8cec0a1020 con 0x562aba7dbc80 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8d04b82740 10 client.10131 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8d04b82740 10 client.10131 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8d04b82740 20 client.10131 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba8119e0 0x562aba813dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8d04b82740 4 client.10131 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aba721440 con 0x562aba8119e0 2024-04-25T00:08:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba8119e0 0x562aba813dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba8119e0 0x562aba813dd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a8bf899acaec287 server_cookie=99f6a601b8416c0d in_seq=0 out_seq=0 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.424+0000 7f8cda7fc640 10 client.10131 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8cda7fc640 1 -- 172.21.15.113:0/3631808059 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562aba721440 con 0x562aba8119e0 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8cda7fc640 10 client.10131 handle_command_reply: tid=0 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 client.10131 shutdown 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 msgr2=0x562aba7ed660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba7ed660 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0798a0 tx=0x7f8cec0ab070 comp rx=0 tx=0).stop 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 shutdown_connections 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 0x562aba7c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba8119e0 0x562aba813dd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8cda7fc640 0 client.10131 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3631808059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba7ed660 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 >> 172.21.15.113:0/3631808059 conn(0x562aba7db540 msgr2=0x562aba7f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 shutdown_connections 2024-04-25T00:08:19.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.425+0000 7f8d04b82740 1 -- 172.21.15.113:0/3631808059 wait complete. 2024-04-25T00:08:19.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.426+0000 7f8d04b82740 20 client.10131 trim_cache size 0 max 16384 2024-04-25T00:08:19.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.443+0000 7f8d04b82740 1 Processor -- start 2024-04-25T00:08:19.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.443+0000 7f8d04b82740 1 -- start start 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7c82e0 0x562aba809150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7dbc80 0x562aba7ee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7f8b60 0x562aba7aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8d04b82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1ff850 con 0x562aba7c82e0 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1522f0 con 0x562aba7f8b60 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc152940 con 0x562aba7dbc80 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7c82e0 0x562aba809150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8cfb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7c82e0 0x562aba809150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48406/0 (socket says 172.21.15.113:48406) 2024-04-25T00:08:19.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.444+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3181557346 learned_addr learned my addr 172.21.15.113:0/3181557346 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7dbc80 msgr2=0x562aba7ee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7dbc80 0x562aba7ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7f8b60 msgr2=0x562aba7aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfbfff640 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7f8b60 0x562aba7aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7f8b60 0x562aba7aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfb7fe640 1 -- 172.21.15.113:0/3181557346 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aba7abf60 con 0x562aba7c82e0 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfbfff640 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7f8b60 0x562aba7aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7c82e0 0x562aba809150 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f8cec075d70 tx=0x7f8cec008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361d153906450848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cf0ff9640 1 -- 172.21.15.113:0/3181557346 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec095070 con 0x562aba7c82e0 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cf0ff9640 1 -- 172.21.15.113:0/3181557346 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec07ee50 con 0x562aba7c82e0 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.445+0000 7f8cf0ff9640 1 -- 172.21.15.113:0/3181557346 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec079dc0 con 0x562aba7c82e0 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 -- 172.21.15.113:0/3181557346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7c82e0 msgr2=0x562aba809150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7c82e0 0x562aba809150 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f8cec075d70 tx=0x7f8cec008000 comp rx=0 tx=0).stop 2024-04-25T00:08:19.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 -- 172.21.15.113:0/3181557346 shutdown_connections 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7f8b60 0x562aba7aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7dbc80 0x562aba7ee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 --2- 172.21.15.113:0/3181557346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7c82e0 0x562aba809150 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 -- 172.21.15.113:0/3181557346 >> 172.21.15.113:0/3181557346 conn(0x562aba7db540 msgr2=0x562aba7a6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 -- 172.21.15.113:0/3181557346 shutdown_connections 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 -- 172.21.15.113:0/3181557346 wait complete. 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 -- start start 2024-04-25T00:08:19.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.446+0000 7f8d04b82740 1 Processor -- start 2024-04-25T00:08:19.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 0x562aba7dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba818900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8d04b82740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8d04b82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562aba713f00 con 0x562aba7dbc80 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562aba6cfcf0 con 0x562aba7c82e0 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8d04b82740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562aba7dc310 con 0x562aba7f8b60 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8d04b82740 10 client.0 _ll_register_callbacks cb 0x7f8cec006c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba818900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba818900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48412/0 (socket says 172.21.15.113:48412) 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 -- 172.21.15.113:0/4284432104 learned_addr learned my addr 172.21.15.113:0/4284432104 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 -- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 msgr2=0x562aba7f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 -- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 msgr2=0x562aba7dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 0x562aba7dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.447+0000 7f8cf3fff640 1 -- 172.21.15.113:0/4284432104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aba7c7b90 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cf3fff640 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba818900 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x562aba7f7100 tx=0x7f8ce40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631d25089061bbde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cda7fc640 1 -- 172.21.15.113:0/4284432104 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400bcd0 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cda7fc640 1 -- 172.21.15.113:0/4284432104 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f8ce40240f0 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cec07ee50 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cda7fc640 1 -- 172.21.15.113:0/4284432104 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8ce4033050 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 10 client.10137 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x562aba7c7740 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cda7fc640 1 -- 172.21.15.113:0/4284432104 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec07ee50 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cda7fc640 1 -- 172.21.15.113:0/4284432104 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7f8ce4036020 con 0x562aba7dbc80 2024-04-25T00:08:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 10 client.10137 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 10 client.10137 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 20 client.10137 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba879730 0x562aba7f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 4 client.10137 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "status"}) -- 0x562aba71b790 con 0x562aba879730 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.448+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba879730 0x562aba7f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8cfb7fe640 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba879730 0x562aba7f4a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe436cb5802223e9 server_cookie=2be2d7b743f5b036 in_seq=0 out_seq=0 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8cda7fc640 10 client.10137 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8cda7fc640 1 -- 172.21.15.113:0/4284432104 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x562aba71b790 con 0x562aba879730 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8cda7fc640 10 client.10137 handle_command_reply: tid=0 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 client.10137 shutdown 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 msgr2=0x562aba818900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba818900 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x562aba7f7100 tx=0x7f8ce40047d0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 shutdown_connections 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x562aba879730 0x562aba7f4a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562aba7f8b60 0x562aba7f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8cda7fc640 0 client.10137 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562aba7dbc80 0x562aba818900 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 --2- 172.21.15.113:0/4284432104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562aba7c82e0 0x562aba7dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 >> 172.21.15.113:0/4284432104 conn(0x562aba7db540 msgr2=0x562aba7aad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 shutdown_connections 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 1 -- 172.21.15.113:0/4284432104 wait complete. 2024-04-25T00:08:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.449+0000 7f8d04b82740 20 client.10137 trim_cache size 0 max 16384 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 9987, 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.7603059180000002, 2024-04-25T00:08:19.457 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 85, 2024-04-25T00:08:19.458 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 158, 2024-04-25T00:08:19.458 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 158, 2024-04-25T00:08:19.458 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 10.020646405999999 2024-04-25T00:08:19.458 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cdc06be30 msgr2=0x7f8cdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cdc06be30 0x7f8cdc06e2f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc071c30 tx=0x7f8ce4006c90 comp rx=0 tx=0).stop 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0b61e0 msgr2=0x7f8cfc1f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0b61e0 0x7f8cfc1f37c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098c0 tx=0x7f8cec0046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 shutdown_connections 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8cdc06be30 0x7f8cdc06e2f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cfc15e510 0x7f8cfc0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc0d28c0 0x7f8cfc0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 --2- 172.21.15.113:0/337283859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cfc0b61e0 0x7f8cfc1f37c0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 >> 172.21.15.113:0/337283859 conn(0x7f8cfc06b720 msgr2=0x7f8cfc15d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 shutdown_connections 2024-04-25T00:08:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.450+0000 7f8d01d9d640 1 -- 172.21.15.113:0/337283859 wait complete. 2024-04-25T00:08:19.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:08:19.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.556+0000 7f2d92509640 1 Processor -- start 2024-04-25T00:08:19.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.557+0000 7f2d92509640 1 -- start start 2024-04-25T00:08:19.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.557+0000 7f2d92509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 0x7f2d8c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.557+0000 7f2d92509640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d8c0d2800 con 0x7f2d8c0d0b00 2024-04-25T00:08:19.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.557+0000 7f2d92509640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d8c0d2ae0 con 0x7f2d8c15e320 2024-04-25T00:08:19.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.557+0000 7f2d92509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8c0d3410 con 0x7f2d8c0d2290 2024-04-25T00:08:19.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.557+0000 7f2d90aa6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2d8c0d0b00 0x7f2d8c0b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46842/0 (socket says 172.21.15.113:46842) 2024-04-25T00:08:19.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.557+0000 7f2d90aa6640 1 -- 172.21.15.113:0/1596771850 learned_addr learned my addr 172.21.15.113:0/1596771850 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:19.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8b7fe640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 0x7f2d8c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2567787075 0 0) 0x7f2d8c0d2800 con 0x7f2d8c0d0b00 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d74003520 con 0x7f2d8c0d0b00 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3676830364 0 0) 0x7f2d74003520 con 0x7f2d8c0d0b00 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 msgr2=0x7f2d8c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 0x7f2d8c0b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 >> v1:172.21.15.178:6789/0 conn(0x7f2d8c15e320 legacy=0x7f2d8c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c0d2dc0 con 0x7f2d8c0d0b00 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f2d840019f0 con 0x7f2d8c0d0b00 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 >> v1:172.21.15.113:6789/0 conn(0x7f2d8c0d0b00 legacy=0x7f2d8c0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d740039e0 0x7f2d74003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 0x7f2d74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d74004d40 0x7f2d740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c0d3410 con 0x7f2d740039e0 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7400b9f0 con 0x7f2d74004d40 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7400bbf0 con 0x7f2d8c0d2290 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8bfff640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d740039e0 0x7f2d74003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8b7fe640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 0x7f2d74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d90aa6640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d74004d40 0x7f2d740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8bfff640 1 -- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 msgr2=0x7f2d74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8bfff640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 0x7f2d74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8bfff640 1 -- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d74004d40 msgr2=0x7f2d740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8bfff640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d74004d40 0x7f2d740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8bfff640 1 -- 172.21.15.113:0/1596771850 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d74003520 con 0x7f2d740039e0 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d8b7fe640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d8c0d2290 0x7f2d74004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.558+0000 7f2d90aa6640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d74004d40 0x7f2d740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.559+0000 7f2d8bfff640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d740039e0 0x7f2d74003dc0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c001910 tx=0x7f2d74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd03ff384ee93c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.559+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c037070 con 0x7f2d740039e0 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.559+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7c02aba0 con 0x7f2d740039e0 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.559+0000 7f2d8affd640 1 -- 172.21.15.113:0/1596771850 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0014e0 con 0x7f2d740039e0 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.560+0000 7f2d92509640 1 -- 172.21.15.113:0/1596771850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d740039e0 msgr2=0x7f2d74003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.560+0000 7f2d92509640 1 --2- 172.21.15.113:0/1596771850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d740039e0 0x7f2d74003dc0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c001910 tx=0x7f2d74005830 comp rx=0 tx=0).stop 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.560+0000 7f2d90aa6640 1 -- 172.21.15.113:0/1596771850 reap_dead start 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.560+0000 7f2d92509640 1 -- 172.21.15.113:0/1596771850 shutdown_connections 2024-04-25T00:08:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.560+0000 7f2d92509640 1 -- 172.21.15.113:0/1596771850 >> 172.21.15.113:0/1596771850 conn(0x7f2d8c0b5c20 msgr2=0x7f2d8c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.560+0000 7f2d92509640 1 -- 172.21.15.113:0/1596771850 shutdown_connections 2024-04-25T00:08:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.560+0000 7f2d92509640 1 -- 172.21.15.113:0/1596771850 wait complete. 2024-04-25T00:08:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.561+0000 7f2d92509640 1 Processor -- start 2024-04-25T00:08:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.561+0000 7f2d92509640 1 -- start start 2024-04-25T00:08:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d740039e0 0x7f2d8c1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d8c0cdb10 0x7f2d8c0c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d8c0c8120 0x7f2d8c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c15e880 con 0x7f2d8c0c8120 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c15e700 con 0x7f2d8c0cdb10 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8c0d0f90 con 0x7f2d740039e0 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d8c0c8120 0x7f2d8c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d8c0c8120 0x7f2d8c1524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48426/0 (socket says 172.21.15.113:48426) 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 -- 172.21.15.113:0/3988155101 learned_addr learned my addr 172.21.15.113:0/3988155101 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d8bfff640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d740039e0 0x7f2d8c1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d8b7fe640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d8c0cdb10 0x7f2d8c0c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 -- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d740039e0 msgr2=0x7f2d8c1519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d740039e0 0x7f2d8c1519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 -- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d8c0cdb10 msgr2=0x7f2d8c0c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d8c0cdb10 0x7f2d8c0c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 -- 172.21.15.113:0/3988155101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c0ccd90 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d90aa6640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d8c0c8120 0x7f2d8c1524e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0b97f0 tx=0x7f2d8400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d88ff9640 1 -- 172.21.15.113:0/3988155101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400eb90 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d88ff9640 1 -- 172.21.15.113:0/3988155101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d84018070 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8c0c86a0 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d88ff9640 1 -- 172.21.15.113:0/3988155101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84013420 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.562+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8c0cd310 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.563+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8c0691a0 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.564+0000 7f2d88ff9640 1 -- 172.21.15.113:0/3988155101 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d840135c0 con 0x7f2d8c0c8120 2024-04-25T00:08:19.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.564+0000 7f2d88ff9640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7806be60 0x7f2d7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.564+0000 7f2d88ff9640 1 -- 172.21.15.113:0/3988155101 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f2d8408b140 con 0x7f2d8c0c8120 2024-04-25T00:08:19.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.564+0000 7f2d8bfff640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7806be60 0x7f2d7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.565+0000 7f2d8bfff640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7806be60 0x7f2d7806e320 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c034000 tx=0x7f2d7c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.568+0000 7f2d88ff9640 1 -- 172.21.15.113:0/3988155101 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d84059370 con 0x7f2d8c0c8120 2024-04-25T00:08:19.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.775+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2d8c0c89c0 con 0x7f2d8c0c8120 2024-04-25T00:08:19.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.776+0000 7f2d88ff9640 1 -- 172.21.15.113:0/3988155101 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7f2d8405d670 con 0x7f2d8c0c8120 2024-04-25T00:08:19.777 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:08:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7806be60 msgr2=0x7f2d7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7806be60 0x7f2d7806e320 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c034000 tx=0x7f2d7c02e000 comp rx=0 tx=0).stop 2024-04-25T00:08:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d8c0c8120 msgr2=0x7f2d8c1524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d8c0c8120 0x7f2d8c1524e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0b97f0 tx=0x7f2d8400e530 comp rx=0 tx=0).stop 2024-04-25T00:08:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 shutdown_connections 2024-04-25T00:08:19.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7806be60 0x7f2d7806e320 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d8c0cdb10 0x7f2d8c0c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d8c0c8120 0x7f2d8c1524e0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 --2- 172.21.15.113:0/3988155101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d740039e0 0x7f2d8c1519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 >> 172.21.15.113:0/3988155101 conn(0x7f2d8c0b5c20 msgr2=0x7f2d8c1535e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 shutdown_connections 2024-04-25T00:08:19.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:19.778+0000 7f2d92509640 1 -- 172.21.15.113:0/3988155101 wait complete. 2024-04-25T00:08:19.789 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:08:19.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:08:19.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:08:19.789 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:08:19.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:08:19.790 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:08:19.790 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:08:19.790 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:08:19.790 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:08:19.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:19.790 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:08:19.806 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:08:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:08:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42180sec preferred_lft 42180sec 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:08:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:08:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:08:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:08:19.810 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:08:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:08:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:08:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:08:19.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:19.811 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:08:19.873 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:08:19.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:19.875 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:08:19.936 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:08:19.936 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:08:19.937 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:08:19.937 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:08:19.937 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:08:19.937 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:08:19.938 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:08:19.938 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:08:19.938 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:08:19.938 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:08:19.938 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:08:20.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:08:20.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:08:20.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:20.062 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:08:20.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:08:20.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:08:20.079 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:08:20.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:20.080 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:08:20.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:08:20.143 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:08:20.202 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:08:20.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:08:20.202 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:08:20.258 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:08:20.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:20.258 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:08:20.339 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:08:20.340 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:08:20.340 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:08:20.348 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:08:20.348 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:08:20.348 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:08:20.348 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:08:20.348 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:08:20.356 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:08:20.356 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:08:20.356 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:08:20.357 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:08:20.357 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:08:20.357 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:08:20.357 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:08:20.357 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:08:20.357 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:08:20.358 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:08:20.358 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:08:20.358 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:08:20.358 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:08:20.359 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:08:20.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:20.359 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:08:20.423 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:08:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:08:20.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:08:20.527 INFO:teuthology.orchestra.run.smithi113.stdout:client7268 2024-04-25T00:08:20.527 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:08:20.527 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:20.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:20.544 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T00:08:20.605 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:08:20.605 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:20.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:20.660 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T00:08:20.751 INFO:teuthology.orchestra.run.smithi113.stderr:6+0 records in 2024-04-25T00:08:20.751 INFO:teuthology.orchestra.run.smithi113.stderr:6+0 records out 2024-04-25T00:08:20.751 INFO:teuthology.orchestra.run.smithi113.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0343258 s, 183 MB/s 2024-04-25T00:08:20.752 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:08:20.755 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:08:20.792 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-25T00:08:20.792 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-25T00:08:20.792 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:08:20.792 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-25T00:08:20.792 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.792 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-25T00:08:20.792 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:08:20.793 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:08:20.794 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-25T00:08:20.795 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-25T00:08:20.796 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-25T00:08:20.797 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-25T00:08:20.798 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-04-25T00:08:20.801 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-04-25T00:08:20.801 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.801 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-25T00:08:20.801 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-04-25T00:08:20.801 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:08:20.801 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-25T00:08:20.801 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-25T00:08:20.802 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.803 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:08:20.804 INFO:teuthology.orchestra.run.smithi113.stdout: mon down mkfs grace = 300 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-04-25T00:08:20.805 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.806 INFO:teuthology.orchestra.run.smithi113.stdout: mds cache max size = 1000 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: mds log max segments = 1 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = False 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.807 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.808 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-25T00:08:20.808 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-25T00:08:20.808 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-25T00:08:20.809 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-25T00:08:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:08:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-25T00:08:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-25T00:08:20.813 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-25T00:08:20.814 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:08:20.815 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.816 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:08:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-25T00:08:20.818 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-25T00:08:20.819 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: mds cache max size = 1000 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = True 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: mds log max segments = 1 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = False 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:08:20.820 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.821 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.823 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:08:20.864 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:08:20.903 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-25T00:08:20.903 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-25T00:08:20.903 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:08:20.903 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-25T00:08:20.903 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.903 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-25T00:08:20.903 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-25T00:08:20.904 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-25T00:08:20.905 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-25T00:08:20.906 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-25T00:08:20.907 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.908 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.909 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.910 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.911 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.912 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:08:20.913 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: mon down mkfs grace = 300 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-25T00:08:20.914 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-25T00:08:20.915 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: mds cache max size = 1000 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = False 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: mds log max segments = 1 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = False 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.916 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.917 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-25T00:08:20.917 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-25T00:08:20.917 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:08:20.917 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-25T00:08:20.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.917 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-25T00:08:20.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:08:20.918 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.919 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-25T00:08:20.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-25T00:08:20.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-25T00:08:20.922 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:08:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.924 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:08:20.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-25T00:08:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-25T00:08:20.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: mds cache max size = 1000 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = False 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: mds log max segments = 1 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = False 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:08:20.928 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> import stat 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> import json 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> import sys 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> try: 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T00:08:20.930 DEBUG:teuthology.orchestra.run.smithi113:> except OSError as e: 2024-04-25T00:08:20.931 DEBUG:teuthology.orchestra.run.smithi113:> sys.exit(e.errno) 2024-04-25T00:08:20.931 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:08:20.931 DEBUG:teuthology.orchestra.run.smithi113:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T00:08:20.931 DEBUG:teuthology.orchestra.run.smithi113:> print(json.dumps( 2024-04-25T00:08:20.931 DEBUG:teuthology.orchestra.run.smithi113:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T00:08:20.931 DEBUG:teuthology.orchestra.run.smithi113:> indent=2)) 2024-04-25T00:08:20.931 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:08:21.035 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:21.035 INFO:teuthology.orchestra.run.smithi113.stdout: "st_mode": 33188, 2024-04-25T00:08:21.035 INFO:teuthology.orchestra.run.smithi113.stdout: "st_ino": 1099511627777, 2024-04-25T00:08:21.035 INFO:teuthology.orchestra.run.smithi113.stdout: "st_dev": 43, 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout: "st_nlink": 1, 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout: "st_uid": 1000, 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout: "st_gid": 1316, 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout: "st_size": 6291456, 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout: "st_atime": 1714003700.7150195, 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout: "st_mtime": 1714003700.734019, 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout: "st_ctime": 1714003700.734019 2024-04-25T00:08:21.036 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:21.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:08:21.044 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.180+0000 7f587035a640 1 Processor -- start 2024-04-25T00:08:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.180+0000 7f587035a640 1 -- start start 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.181+0000 7f587035a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f587035a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58680d0b00 0x7f58680d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f587035a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58680d2800 con 0x7f586815e320 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f587035a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f58680d14d0 con 0x7f58680d0b00 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f587035a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58680d3410 con 0x7f58680d2290 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586e0f6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f586815e320 0x7f58680d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46854/0 (socket says 172.21.15.113:46854) 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586e0f6640 1 -- 172.21.15.113:0/4045976740 learned_addr learned my addr 172.21.15.113:0/4045976740 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586e8f7640 1 --2- 172.21.15.113:0/4045976740 >> v2:172.21.15.113:3300/0 conn(0x7f58680d0b00 0x7f58680d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586d8f5640 1 --2- 172.21.15.113:0/4045976740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586d0f4640 1 -- 172.21.15.113:0/4045976740 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1342637831 0 0) 0x7f58680d2800 con 0x7f586815e320 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586d0f4640 1 -- 172.21.15.113:0/4045976740 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5850003520 con 0x7f586815e320 2024-04-25T00:08:21.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586e8f7640 1 -- 172.21.15.113:0/4045976740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 msgr2=0x7f58680b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586e8f7640 1 --2- 172.21.15.113:0/4045976740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586e8f7640 1 -- 172.21.15.113:0/4045976740 >> v1:172.21.15.113:6789/0 conn(0x7f586815e320 legacy=0x7f58680d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.182+0000 7f586e8f7640 1 -- 172.21.15.113:0/4045976740 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680d2ae0 con 0x7f58680d0b00 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.183+0000 7f586e8f7640 1 --2- 172.21.15.113:0/4045976740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58680d0b00 0x7f58680d0f90 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f58580097a0 tx=0x7f585802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa4d0c55c7ac1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.183+0000 7f586d0f4640 1 -- 172.21.15.113:0/4045976740 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858037070 con 0x7f58680d0b00 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.183+0000 7f586d0f4640 1 -- 172.21.15.113:0/4045976740 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5858032440 con 0x7f58680d0b00 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.183+0000 7f586d0f4640 1 -- 172.21.15.113:0/4045976740 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858032810 con 0x7f58680d0b00 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 -- 172.21.15.113:0/4045976740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58680d0b00 msgr2=0x7f58680d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 --2- 172.21.15.113:0/4045976740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58680d0b00 0x7f58680d0f90 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f58580097a0 tx=0x7f585802a180 comp rx=0 tx=0).stop 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 -- 172.21.15.113:0/4045976740 shutdown_connections 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 --2- 172.21.15.113:0/4045976740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58680d0b00 0x7f58680d0f90 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 --2- 172.21.15.113:0/4045976740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 -- 172.21.15.113:0/4045976740 >> 172.21.15.113:0/4045976740 conn(0x7f58680b5c20 msgr2=0x7f58680b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 -- 172.21.15.113:0/4045976740 shutdown_connections 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.184+0000 7f587035a640 1 -- 172.21.15.113:0/4045976740 wait complete. 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.185+0000 7f587035a640 1 Processor -- start 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.185+0000 7f587035a640 1 -- start start 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58680d0b00 0x7f58680b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f586815e320 0x7f58680cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58680b9fe0 con 0x7f586815e320 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58680d30a0 con 0x7f58680d0b00 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f586815e700 con 0x7f58680d2290 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f586815e320 0x7f58680cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f586815e320 0x7f58680cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48446/0 (socket says 172.21.15.113:48446) 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 -- 172.21.15.113:0/2526444954 learned_addr learned my addr 172.21.15.113:0/2526444954 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e0f6640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58680d0b00 0x7f58680b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586d8f5640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 -- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 msgr2=0x7f58680ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 -- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58680d0b00 msgr2=0x7f58680b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58680d0b00 0x7f58680b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 -- 172.21.15.113:0/2526444954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5868160e20 con 0x7f586815e320 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586d8f5640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f586e8f7640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f586815e320 0x7f58680cd3b0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f5858035040 tx=0x7f58580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f5866ffd640 1 -- 172.21.15.113:0/2526444954 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858041020 con 0x7f586815e320 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f5866ffd640 1 -- 172.21.15.113:0/2526444954 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585803d640 con 0x7f586815e320 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5868161100 con 0x7f586815e320 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f5866ffd640 1 -- 172.21.15.113:0/2526444954 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858037040 con 0x7f586815e320 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.186+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681616c0 con 0x7f586815e320 2024-04-25T00:08:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.187+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58680691a0 con 0x7f586815e320 2024-04-25T00:08:21.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.188+0000 7f5866ffd640 1 -- 172.21.15.113:0/2526444954 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5858044020 con 0x7f586815e320 2024-04-25T00:08:21.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.188+0000 7f5866ffd640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f585406be30 0x7f585406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.188+0000 7f586e0f6640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f585406be30 0x7f585406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.188+0000 7f5866ffd640 1 -- 172.21.15.113:0/2526444954 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f58580ab4b0 con 0x7f586815e320 2024-04-25T00:08:21.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.189+0000 7f586e0f6640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f585406be30 0x7f585406e2f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5860001560 tx=0x7f58600019f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.193+0000 7f5866ffd640 1 -- 172.21.15.113:0/2526444954 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f585804b030 con 0x7f586815e320 2024-04-25T00:08:21.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.402+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58681441a0 con 0x7f586815e320 2024-04-25T00:08:21.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.402+0000 7f5866ffd640 1 -- 172.21.15.113:0/2526444954 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7f5858079e70 con 0x7f586815e320 2024-04-25T00:08:21.403 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:21.403 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:21.404 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f585406be30 msgr2=0x7f585406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f585406be30 0x7f585406e2f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5860001560 tx=0x7f58600019f0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f586815e320 msgr2=0x7f58680cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f586815e320 0x7f58680cd3b0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f5858035040 tx=0x7f58580046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 shutdown_connections 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f585406be30 0x7f585406e2f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f586815e320 0x7f58680cd3b0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58680d2290 0x7f58680ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 --2- 172.21.15.113:0/2526444954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58680d0b00 0x7f58680b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 >> 172.21.15.113:0/2526444954 conn(0x7f58680b5c20 msgr2=0x7f586815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 shutdown_connections 2024-04-25T00:08:21.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.404+0000 7f587035a640 1 -- 172.21.15.113:0/2526444954 wait complete. 2024-04-25T00:08:21.416 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T00:08:21.539 INFO:teuthology.orchestra.run.smithi178.stdout:{"message":"","return_code":0} 2024-04-25T00:08:21.540 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T00:08:21.540 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-25T00:08:21.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.582+0000 7fb63e3c9f40 1 Processor -- start 2024-04-25T00:08:21.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.582+0000 7fb63e3c9f40 1 -- start start 2024-04-25T00:08:21.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63e3c9f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 0x55c6bd5e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63e3c9f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63e3c9f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 0x55c6bd5e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63e3c9f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c6bd4bd640 con 0x55c6bd5454f0 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63e3c9f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6bd4c1680 con 0x55c6bd540560 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63e3c9f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c6bd560900 con 0x55c6bd5458d0 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48450/0 (socket says 172.21.15.113:48450) 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3180326416 learned_addr learned my addr 172.21.15.113:0/3180326416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63beca640 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 0x55c6bd5e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63c6cb640 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 0x55c6bd5e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 msgr2=0x55c6bd5e3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 0x55c6bd5e3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 msgr2=0x55c6bd5e0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 0x55c6bd5e0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3180326416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6bd5eec80 con 0x55c6bd5454f0 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.583+0000 7fb63b6c9640 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ef0c0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fb630001e00 tx=0x7fb63000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad24958ec51de57b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63a6c7640 1 -- 172.21.15.113:0/3180326416 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63000bdc0 con 0x55c6bd5454f0 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63a6c7640 1 -- 172.21.15.113:0/3180326416 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63000f400 con 0x55c6bd5454f0 2024-04-25T00:08:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63a6c7640 1 -- 172.21.15.113:0/3180326416 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63000f7f0 con 0x55c6bd5454f0 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3180326416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 msgr2=0x55c6bd5ef0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ef0c0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fb630001e00 tx=0x7fb63000b670 comp rx=0 tx=0).stop 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3180326416 shutdown_connections 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 0x55c6bd5e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ef0c0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3180326416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 0x55c6bd5e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3180326416 >> 172.21.15.113:0/3180326416 conn(0x55c6bd5610e0 msgr2=0x55c6bd5ebbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3180326416 shutdown_connections 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.584+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3180326416 wait complete. 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 Processor -- start 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 -- start start 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 0x55c6bd67f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 0x55c6bd67d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c6bd5ed260 con 0x55c6bd5454f0 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6bd5eea30 con 0x55c6bd540560 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.585+0000 7fb63e3c9f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c6bd542610 con 0x55c6bd5458d0 2024-04-25T00:08:21.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48454/0 (socket says 172.21.15.113:48454) 2024-04-25T00:08:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63beca640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 0x55c6bd67f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3631399266 learned_addr learned my addr 172.21.15.113:0/3631399266 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63c6cb640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 0x55c6bd67d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3631399266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 msgr2=0x55c6bd67d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c6bd5458d0 0x55c6bd67d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3631399266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 msgr2=0x55c6bd67f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6bd540560 0x55c6bd67f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 -- 172.21.15.113:0/3631399266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6bd5e8f60 con 0x55c6bd5454f0 2024-04-25T00:08:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63b6c9640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ec4d0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fb63000b6b0 tx=0x7fb630000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb6396c5640 1 -- 172.21.15.113:0/3631399266 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630020070 con 0x55c6bd5454f0 2024-04-25T00:08:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb6396c5640 1 -- 172.21.15.113:0/3631399266 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63000bdc0 con 0x55c6bd5454f0 2024-04-25T00:08:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb6396c5640 1 -- 172.21.15.113:0/3631399266 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63001c660 con 0x55c6bd5454f0 2024-04-25T00:08:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c6bd5eb920 con 0x55c6bd5454f0 2024-04-25T00:08:21.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.586+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6bd67f8a0 con 0x55c6bd5454f0 2024-04-25T00:08:21.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.587+0000 7fb6396c5640 1 -- 172.21.15.113:0/3631399266 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb630023070 con 0x55c6bd5454f0 2024-04-25T00:08:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb6396c5640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62806a840 0x7fb62806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb6396c5640 1 -- 172.21.15.113:0/3631399266 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fb63008b100 con 0x55c6bd5454f0 2024-04-25T00:08:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb63beca640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62806a840 0x7fb62806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55c6bd5eca10 0x55c6bd6880d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 --> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] -- osd_op(unknown.0.0:1 23.7 23:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) -- 0x55c6bd5e3770 con 0x55c6bd5eca10 2024-04-25T00:08:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb63c6cb640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55c6bd5eca10 0x55c6bd6880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb63beca640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62806a840 0x7fb62806cd00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb62c056000 tx=0x7fb62c050000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.588+0000 7fb63c6cb640 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55c6bd5eca10 0x55c6bd6880d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63c6cb640 1 -- 172.21.15.113:0/3631399266 <== osd.0 v2:172.21.15.113:6800/1851994484 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v158'4 uv4 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb6340057f0 con 0x55c6bd5eca10 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55c6bd5eca10 msgr2=0x55c6bd6880d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55c6bd5eca10 0x55c6bd6880d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62806a840 msgr2=0x7fb62806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb62806a840 0x7fb62806cd00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb62c056000 tx=0x7fb62c050000 comp rx=0 tx=0).stop 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 msgr2=0x55c6bd5ec4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 --2- 172.21.15.113:0/3631399266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bd5454f0 0x55c6bd5ec4d0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fb63000b6b0 tx=0x7fb630000ed0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63c6cb640 1 -- 172.21.15.113:0/3631399266 reap_dead start 2024-04-25T00:08:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 shutdown_connections 2024-04-25T00:08:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 >> 172.21.15.113:0/3631399266 conn(0x55c6bd5610e0 msgr2=0x55c6bd5f13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 shutdown_connections 2024-04-25T00:08:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.590+0000 7fb63e3c9f40 1 -- 172.21.15.113:0/3631399266 wait complete. 2024-04-25T00:08:21.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:21.601 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start / recursive,repair 2024-04-25T00:08:21.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.738+0000 7f31637a5640 1 Processor -- start 2024-04-25T00:08:21.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.738+0000 7f31637a5640 1 -- start start 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f31637a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f315c0b47a0 0x7f315c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f31637a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f315c15bcf0 con 0x7f315c154f40 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f31637a5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f315c0b2f40 con 0x7f315c0cc0b0 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f31637a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f315c0b31e0 con 0x7f315c0b47a0 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f3161541640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f315c0b47a0 0x7f315c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f3161d42640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f315c154f40 0x7f315c15bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46868/0 (socket says 172.21.15.113:46868) 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f3161d42640 1 -- 172.21.15.113:0/3906517776 learned_addr learned my addr 172.21.15.113:0/3906517776 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f3161541640 1 -- 172.21.15.113:0/3906517776 >> v1:172.21.15.178:6789/0 conn(0x7f315c0cc0b0 legacy=0x7f315c154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f3153fff640 1 -- 172.21.15.113:0/3906517776 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861891625 0 0) 0x7f3154001110 con 0x7f315c154f40 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f3161541640 1 -- 172.21.15.113:0/3906517776 >> v1:172.21.15.113:6789/0 conn(0x7f315c154f40 legacy=0x7f315c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.739+0000 7f3161541640 1 -- 172.21.15.113:0/3906517776 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c0b34c0 con 0x7f315c0b47a0 2024-04-25T00:08:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.740+0000 7f3161541640 1 --2- 172.21.15.113:0/3906517776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c0b47a0 0x7f315c15b7b0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f3158009d90 tx=0x7f315802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb4caf85020346e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.740+0000 7f3153fff640 1 -- 172.21.15.113:0/3906517776 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158007740 con 0x7f315c0b47a0 2024-04-25T00:08:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.740+0000 7f3153fff640 1 -- 172.21.15.113:0/3906517776 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31580078e0 con 0x7f315c0b47a0 2024-04-25T00:08:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.740+0000 7f3153fff640 1 -- 172.21.15.113:0/3906517776 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158007cb0 con 0x7f315c0b47a0 2024-04-25T00:08:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.741+0000 7f31637a5640 1 -- 172.21.15.113:0/3906517776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c0b47a0 msgr2=0x7f315c15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.741+0000 7f31637a5640 1 --2- 172.21.15.113:0/3906517776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c0b47a0 0x7f315c15b7b0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f3158009d90 tx=0x7f315802a6a0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.741+0000 7f31637a5640 1 -- 172.21.15.113:0/3906517776 shutdown_connections 2024-04-25T00:08:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.741+0000 7f31637a5640 1 --2- 172.21.15.113:0/3906517776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c0b47a0 0x7f315c15b7b0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.741+0000 7f31637a5640 1 -- 172.21.15.113:0/3906517776 >> 172.21.15.113:0/3906517776 conn(0x7f315c0b1b30 msgr2=0x7f315c0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.741+0000 7f31637a5640 1 -- 172.21.15.113:0/3906517776 shutdown_connections 2024-04-25T00:08:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.741+0000 7f31637a5640 1 -- 172.21.15.113:0/3906517776 wait complete. 2024-04-25T00:08:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.742+0000 7f31637a5640 1 Processor -- start 2024-04-25T00:08:21.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.742+0000 7f31637a5640 1 -- start start 2024-04-25T00:08:21.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f31637a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f315c0b47a0 0x7f315c169220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f31637a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f315c0cc0b0 0x7f315c1f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f31637a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c154f40 0x7f315c1f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f31637a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f315c0b4b80 con 0x7f315c154f40 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f31637a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f315c0b3a80 con 0x7f315c0cc0b0 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f31637a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f315c0b4d00 con 0x7f315c0b47a0 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c154f40 0x7f315c1f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c154f40 0x7f315c1f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48474/0 (socket says 172.21.15.113:48474) 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 -- 172.21.15.113:0/3914324945 learned_addr learned my addr 172.21.15.113:0/3914324945 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161541640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f315c0b47a0 0x7f315c169220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3160d40640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f315c0cc0b0 0x7f315c1f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 -- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f315c0b47a0 msgr2=0x7f315c169220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f315c0b47a0 0x7f315c169220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 -- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f315c0cc0b0 msgr2=0x7f315c1f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f315c0cc0b0 0x7f315c1f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 -- 172.21.15.113:0/3914324945 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c142d60 con 0x7f315c154f40 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3161d42640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c154f40 0x7f315c1f6060 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f315400c410 tx=0x7f315400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3151ffb640 1 -- 172.21.15.113:0/3914324945 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31540074e0 con 0x7f315c154f40 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3151ffb640 1 -- 172.21.15.113:0/3914324945 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31540047e0 con 0x7f315c154f40 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c149a60 con 0x7f315c154f40 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.743+0000 7f3151ffb640 1 -- 172.21.15.113:0/3914324945 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154004bd0 con 0x7f315c154f40 2024-04-25T00:08:21.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.744+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c147950 con 0x7f315c154f40 2024-04-25T00:08:21.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.745+0000 7f3151ffb640 1 -- 172.21.15.113:0/3914324945 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3154011070 con 0x7f315c154f40 2024-04-25T00:08:21.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.745+0000 7f3151ffb640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f314006be30 0x7f314006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.745+0000 7f3151ffb640 1 -- 172.21.15.113:0/3914324945 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f315408a930 con 0x7f315c154f40 2024-04-25T00:08:21.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.745+0000 7f3161541640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f314006be30 0x7f314006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.746+0000 7f3161541640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f314006be30 0x7f314006e2f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f315802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.746+0000 7f316658d740 1 Processor -- start 2024-04-25T00:08:21.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.746+0000 7f316658d740 1 -- start start 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f316658d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f32b280 0x55855f32d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f367950 0x55855f32b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f3561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f316658d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55855f28e2c0 con 0x55855f32b280 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55855f236910 con 0x55855f355d40 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55855f28a280 con 0x55855f367950 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f3161541640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f32b280 0x55855f32d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f3161541640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f32b280 0x55855f32d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48476/0 (socket says 172.21.15.113:48476) 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f3161541640 1 -- 172.21.15.113:0/794328041 learned_addr learned my addr 172.21.15.113:0/794328041 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f3160d40640 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f367950 0x55855f32b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.747+0000 7f3161d42640 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f3561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3161541640 1 -- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f367950 msgr2=0x55855f32b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3161541640 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f367950 0x55855f32b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3161541640 1 -- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 msgr2=0x55855f3561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3161541640 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f3561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3161541640 1 -- 172.21.15.113:0/794328041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855f247fb0 con 0x55855f32b280 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3161d42640 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f3561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3161541640 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f32b280 0x55855f32d3e0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f3158009e80 tx=0x7f315802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bdfa1b2de7c848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3146ffd640 1 -- 172.21.15.113:0/794328041 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315804d070 con 0x55855f32b280 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3146ffd640 1 -- 172.21.15.113:0/794328041 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315803d9b0 con 0x55855f32b280 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f3146ffd640 1 -- 172.21.15.113:0/794328041 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315803dd60 con 0x55855f32b280 2024-04-25T00:08:21.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f316658d740 1 -- 172.21.15.113:0/794328041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f32b280 msgr2=0x55855f32d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f316658d740 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f32b280 0x55855f32d3e0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f3158009e80 tx=0x7f315802e000 comp rx=0 tx=0).stop 2024-04-25T00:08:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f316658d740 1 -- 172.21.15.113:0/794328041 shutdown_connections 2024-04-25T00:08:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f316658d740 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f3561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f316658d740 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f367950 0x55855f32b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f316658d740 1 --2- 172.21.15.113:0/794328041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f32b280 0x55855f32d3e0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.748+0000 7f316658d740 1 -- 172.21.15.113:0/794328041 >> 172.21.15.113:0/794328041 conn(0x55855f2f2950 msgr2=0x55855f370d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.749+0000 7f316658d740 1 -- 172.21.15.113:0/794328041 shutdown_connections 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.749+0000 7f316658d740 1 -- 172.21.15.113:0/794328041 wait complete. 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.749+0000 7f316658d740 1 -- start start 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.749+0000 7f316658d740 1 Processor -- start 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f32b280 0x55855f353160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f355d40 0x55855f36d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f316658d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f36d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f316658d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55855f249e90 con 0x55855f367950 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55855f353890 con 0x55855f32b280 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55855f3823c0 con 0x55855f355d40 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f316658d740 10 client.0 _ll_register_callbacks cb 0x55855f285970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f36d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f36d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48482/0 (socket says 172.21.15.113:48482) 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161541640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f32b280 0x55855f353160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 -- 172.21.15.113:0/3596858835 learned_addr learned my addr 172.21.15.113:0/3596858835 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3160d40640 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f355d40 0x55855f36d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 -- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f355d40 msgr2=0x55855f36d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f355d40 0x55855f36d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 -- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f32b280 msgr2=0x55855f353160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f32b280 0x55855f353160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.750+0000 7f3161d42640 1 -- 172.21.15.113:0/3596858835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855f06a7e0 con 0x55855f367950 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f3161d42640 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f36d8f0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f315400c3e0 tx=0x7f31540077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34135facf7c646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f313bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f313bfff640 1 -- 172.21.15.113:0/3596858835 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154061850 con 0x55855f367950 2024-04-25T00:08:21.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f313bfff640 1 -- 172.21.15.113:0/3596858835 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f3154010d60 con 0x55855f367950 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f315803d9b0 con 0x55855f367950 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f313bfff640 1 -- 172.21.15.113:0/3596858835 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f31540580c0 con 0x55855f367950 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 10 client.10176 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55855f0caf70 con 0x55855f367950 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f313bfff640 1 -- 172.21.15.113:0/3596858835 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315803d9b0 con 0x55855f367950 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f313bfff640 1 -- 172.21.15.113:0/3596858835 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7f3154058260 con 0x55855f367950 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 10 client.10176 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 10 client.10176 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 20 client.10176 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f36c560 0x55855f381550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 4 client.10176 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55855f2625a0 con 0x55855f36c560 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.751+0000 7f3161541640 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f36c560 0x55855f381550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.752+0000 7f313bfff640 10 client.10176 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.752+0000 7f3161541640 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f36c560 0x55855f381550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47484302952ef5f0 server_cookie=7f15e776f1c89800 in_seq=0 out_seq=0 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.752+0000 7f313bfff640 1 -- 172.21.15.113:0/3596858835 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55855f2625a0 con 0x55855f36c560 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.752+0000 7f313bfff640 10 client.10176 handle_command_reply: tid=0 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.752+0000 7f316658d740 1 client.10176 shutdown 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.752+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 msgr2=0x55855f36d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.752+0000 7f316658d740 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f36d8f0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f315400c3e0 tx=0x7f31540077c0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 shutdown_connections 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f36c560 0x55855f381550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f36d8f0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f355d40 0x55855f36d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f313bfff640 0 client.10176 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 --2- 172.21.15.113:0/3596858835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f32b280 0x55855f353160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 >> 172.21.15.113:0/3596858835 conn(0x55855f2f2950 msgr2=0x55855f353c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 shutdown_connections 2024-04-25T00:08:21.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 1 -- 172.21.15.113:0/3596858835 wait complete. 2024-04-25T00:08:21.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.753+0000 7f316658d740 20 client.10176 trim_cache size 0 max 16384 2024-04-25T00:08:21.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.770+0000 7f316658d740 1 Processor -- start 2024-04-25T00:08:21.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f316658d740 1 -- start start 2024-04-25T00:08:21.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f358b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f373940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f316658d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f3252b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f316658d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f315c16b5b0 con 0x55855f367950 2024-04-25T00:08:21.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f315c16a5d0 con 0x55855f355d40 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f315c169910 con 0x55855f32b280 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f3252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f3252b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48496/0 (socket says 172.21.15.113:48496) 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f3161d42640 1 -- 172.21.15.113:0/4005882250 learned_addr learned my addr 172.21.15.113:0/4005882250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f3160d40640 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f373940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.771+0000 7f3161541640 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f358b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f3161d42640 1 -- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 msgr2=0x55855f358b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f3161d42640 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f358b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f3161d42640 1 -- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 msgr2=0x55855f373940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f3161d42640 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f373940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f3161d42640 1 -- 172.21.15.113:0/4005882250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855f3818c0 con 0x55855f367950 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f3161d42640 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f3252b0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f315400cdd0 tx=0x7f315406f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d826c60a52a0b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f31467fc640 1 -- 172.21.15.113:0/4005882250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315407d070 con 0x55855f367950 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f31467fc640 1 -- 172.21.15.113:0/4005882250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3154087070 con 0x55855f367950 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f31467fc640 1 -- 172.21.15.113:0/4005882250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154061850 con 0x55855f367950 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f316658d740 1 -- 172.21.15.113:0/4005882250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 msgr2=0x55855f3252b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f316658d740 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f3252b0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f315400cdd0 tx=0x7f315406f000 comp rx=0 tx=0).stop 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f316658d740 1 -- 172.21.15.113:0/4005882250 shutdown_connections 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f316658d740 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f373940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f316658d740 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f3252b0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f316658d740 1 --2- 172.21.15.113:0/4005882250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f358b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.772+0000 7f316658d740 1 -- 172.21.15.113:0/4005882250 >> 172.21.15.113:0/4005882250 conn(0x55855f2f2950 msgr2=0x55855f393630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.773+0000 7f316658d740 1 -- 172.21.15.113:0/4005882250 shutdown_connections 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.773+0000 7f316658d740 1 -- 172.21.15.113:0/4005882250 wait complete. 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.773+0000 7f316658d740 1 -- start start 2024-04-25T00:08:21.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.773+0000 7f316658d740 1 Processor -- start 2024-04-25T00:08:21.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f3dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f316658d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f32a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f316658d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f382370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f316658d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55855f36f150 con 0x55855f367950 2024-04-25T00:08:21.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f315c169f60 con 0x55855f355d40 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f316658d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55855f372900 con 0x55855f32b280 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f316658d740 10 client.0 _ll_register_callbacks cb 0x7f3154058a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f382370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161541640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f3dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f382370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48512/0 (socket says 172.21.15.113:48512) 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 -- 172.21.15.113:0/208154495 learned_addr learned my addr 172.21.15.113:0/208154495 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3160d40640 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f32a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 -- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 msgr2=0x55855f3dd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f3dd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 -- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 msgr2=0x55855f32a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f32a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161d42640 1 -- 172.21.15.113:0/208154495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855f3295e0 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.774+0000 7f3161541640 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f3dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f3161d42640 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f382370 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3154014000 tx=0x7f315406e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff783c3a5f973098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f313bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f313bfff640 1 -- 172.21.15.113:0/208154495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154065060 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f313bfff640 1 -- 172.21.15.113:0/208154495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f3154082590 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3154087070 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f313bfff640 1 -- 172.21.15.113:0/208154495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f315406b050 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 10 client.10182 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55855ef62170 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f313bfff640 1 -- 172.21.15.113:0/208154495 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3154087070 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f313bfff640 1 -- 172.21.15.113:0/208154495 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7f3154043bb0 con 0x55855f367950 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 10 client.10182 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:21.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 10 client.10182 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 20 client.10182 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f3828b0 0x55855f392400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 4 client.10182 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) -- 0x55855f29b800 con 0x55855f3828b0 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.775+0000 7f3161541640 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f3828b0 0x55855f392400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f313bfff640 10 client.10182 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f3161541640 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f3828b0 0x55855f392400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce8d82e41f21e80 server_cookie=18f82286316529a5 in_seq=0 out_seq=0 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f313bfff640 1 -- 172.21.15.113:0/208154495 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55855f29b800 con 0x55855f3828b0 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f313bfff640 10 client.10182 handle_command_reply: tid=0 2024-04-25T00:08:21.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 client.10182 shutdown 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 msgr2=0x55855f382370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f382370 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3154014000 tx=0x7f315406e000 comp rx=0 tx=0).stop 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 shutdown_connections 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55855f355d40 0x55855f32a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x55855f3828b0 0x55855f392400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55855f367950 0x55855f382370 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 --2- 172.21.15.113:0/208154495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55855f32b280 0x55855f3dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 >> 172.21.15.113:0/208154495 conn(0x55855f2f2950 msgr2=0x55855f358b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.776+0000 7f313bfff640 0 client.10182 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 shutdown_connections 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f316658d740 1 -- 172.21.15.113:0/208154495 wait complete. 2024-04-25T00:08:21.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f316658d740 20 client.10182 trim_cache size 0 max 16384 2024-04-25T00:08:21.786 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:21.786 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:08:21.786 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "85ef7ff3-f8ae-43e6-b95c-83fc78e0494a", 2024-04-25T00:08:21.786 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:08:21.786 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:21.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f314006be30 msgr2=0x7f314006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f314006be30 0x7f314006e2f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f315802f000 comp rx=0 tx=0).stop 2024-04-25T00:08:21.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c154f40 msgr2=0x7f315c1f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c154f40 0x7f315c1f6060 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f315400c410 tx=0x7f315400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 shutdown_connections 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f314006be30 0x7f314006e2f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f315c0cc0b0 0x7f315c1f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f315c154f40 0x7f315c1f6060 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 --2- 172.21.15.113:0/3914324945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f315c0b47a0 0x7f315c169220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 >> 172.21.15.113:0/3914324945 conn(0x7f315c0b1b30 msgr2=0x7f315c06df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 shutdown_connections 2024-04-25T00:08:21.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.777+0000 7f31637a5640 1 -- 172.21.15.113:0/3914324945 wait complete. 2024-04-25T00:08:21.789 INFO:tasks.ceph.mds.b.smithi178.stderr:2024-04-25T00:08:21.778+0000 7f06e82f7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T00:08:20.731019+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={7268=0-12582912@1} caps={7268=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=7268 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x556320ddec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(23)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T00:08:21.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:21.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T00:08:21.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.888+0000 7faf2fc4c640 1 Processor -- start 2024-04-25T00:08:21.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.888+0000 7faf2fc4c640 1 -- start start 2024-04-25T00:08:21.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2fc4c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faf280b6100 0x7faf280b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2fc4c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf280d2e00 con 0x7faf2815e510 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2fc4c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf280d30e0 con 0x7faf280b5ca0 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2fc4c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faf280d33c0 con 0x7faf280b6100 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2d9e8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7faf2815e510 0x7faf280d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46880/0 (socket says 172.21.15.113:46880) 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2d9e8640 1 -- 172.21.15.113:0/1012451746 learned_addr learned my addr 172.21.15.113:0/1012451746 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/1012451746 >> v2:172.21.15.178:3300/0 conn(0x7faf280b6100 0x7faf280b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2e1e9640 1 -- 172.21.15.113:0/1012451746 >> v1:172.21.15.178:6789/0 conn(0x7faf280b5ca0 legacy=0x7faf280b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2e1e9640 1 -- 172.21.15.113:0/1012451746 >> v1:172.21.15.113:6789/0 conn(0x7faf2815e510 legacy=0x7faf280d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.889+0000 7faf2e1e9640 1 -- 172.21.15.113:0/1012451746 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf280d36a0 con 0x7faf280b6100 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.890+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/1012451746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 0x7faf280b7ad0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faf20009d90 tx=0x7faf2002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1887f74caada3e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.890+0000 7faf2c9e6640 1 -- 172.21.15.113:0/1012451746 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20007740 con 0x7faf280b6100 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.890+0000 7faf2c9e6640 1 -- 172.21.15.113:0/1012451746 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf200078e0 con 0x7faf280b6100 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2c9e6640 1 -- 172.21.15.113:0/1012451746 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20007cb0 con 0x7faf280b6100 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2fc4c640 1 -- 172.21.15.113:0/1012451746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 msgr2=0x7faf280b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/1012451746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 0x7faf280b7ad0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faf20009d90 tx=0x7faf2002a660 comp rx=0 tx=0).stop 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2fc4c640 1 -- 172.21.15.113:0/1012451746 shutdown_connections 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/1012451746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 0x7faf280b7ad0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2fc4c640 1 -- 172.21.15.113:0/1012451746 >> 172.21.15.113:0/1012451746 conn(0x7faf2806b720 msgr2=0x7faf280d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2fc4c640 1 -- 172.21.15.113:0/1012451746 shutdown_connections 2024-04-25T00:08:21.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.891+0000 7faf2fc4c640 1 -- 172.21.15.113:0/1012451746 wait complete. 2024-04-25T00:08:21.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.892+0000 7faf2fc4c640 1 Processor -- start 2024-04-25T00:08:21.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.892+0000 7faf2fc4c640 1 -- start start 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2fc4c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf280b5ca0 0x7faf28149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2fc4c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 0x7faf280b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2fc4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf2815e510 0x7faf280c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2fc4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faf280b6510 con 0x7faf2815e510 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2fc4c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf280d3ea0 con 0x7faf280b6100 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2fc4c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faf280d2a90 con 0x7faf280b5ca0 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2e1e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf2815e510 0x7faf280c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2e1e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf2815e510 0x7faf280c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48514/0 (socket says 172.21.15.113:48514) 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2e1e9640 1 -- 172.21.15.113:0/3074327716 learned_addr learned my addr 172.21.15.113:0/3074327716 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 0x7faf280b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.893+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf280b5ca0 0x7faf28149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2e1e9640 1 -- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf280b5ca0 msgr2=0x7faf28149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf280b5ca0 0x7faf28149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2e1e9640 1 -- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 msgr2=0x7faf280b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 0x7faf280b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2e1e9640 1 -- 172.21.15.113:0/3074327716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2815f370 con 0x7faf2815e510 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf2815e510 0x7faf280c7d50 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7faf20009eb0 tx=0x7faf20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf1e7fc640 1 -- 172.21.15.113:0/3074327716 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20040070 con 0x7faf2815e510 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf1e7fc640 1 -- 172.21.15.113:0/3074327716 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf20031c90 con 0x7faf2815e510 2024-04-25T00:08:21.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2815f650 con 0x7faf2815e510 2024-04-25T00:08:21.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf1e7fc640 1 -- 172.21.15.113:0/3074327716 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20045680 con 0x7faf2815e510 2024-04-25T00:08:21.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.894+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2815fc10 con 0x7faf2815e510 2024-04-25T00:08:21.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.895+0000 7faf1e7fc640 1 -- 172.21.15.113:0/3074327716 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf20039040 con 0x7faf2815e510 2024-04-25T00:08:21.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.896+0000 7faf1e7fc640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf0c06be30 0x7faf0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.896+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf0c06be30 0x7faf0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.896+0000 7faf1e7fc640 1 -- 172.21.15.113:0/3074327716 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7faf200ac820 con 0x7faf2815e510 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.896+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf0c06be30 0x7faf0c06e2f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faf24005880 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.897+0000 7faf32a30740 1 Processor -- start 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.897+0000 7faf32a30740 1 -- start start 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf32a30740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a26a40 0x564d73a6ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf32a30740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564d7398a6b0 con 0x564d73a50d60 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564d73932880 con 0x564d73a6e600 2024-04-25T00:08:21.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564d73986670 con 0x564d73a26a40 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a51140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48524/0 (socket says 172.21.15.113:48524) 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 -- 172.21.15.113:0/4037247575 learned_addr learned my addr 172.21.15.113:0/4037247575 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a26a40 0x564d73a6ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 -- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a26a40 msgr2=0x564d73a6ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a26a40 0x564d73a6ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 -- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 msgr2=0x564d73a6eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 -- 172.21.15.113:0/4037247575 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d737687d0 con 0x564d73a50d60 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a51140 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7faf24001e70 tx=0x7faf24011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6025bd099aa62c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf0b7fe640 1 -- 172.21.15.113:0/4037247575 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24007710 con 0x564d73a50d60 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf0b7fe640 1 -- 172.21.15.113:0/4037247575 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2400e7b0 con 0x564d73a50d60 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.898+0000 7faf0b7fe640 1 -- 172.21.15.113:0/4037247575 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2400eb60 con 0x564d73a50d60 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 -- 172.21.15.113:0/4037247575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 msgr2=0x564d73a51140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a51140 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7faf24001e70 tx=0x7faf24011000 comp rx=0 tx=0).stop 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 -- 172.21.15.113:0/4037247575 shutdown_connections 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a26a40 0x564d73a6ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 --2- 172.21.15.113:0/4037247575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a51140 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 -- 172.21.15.113:0/4037247575 >> 172.21.15.113:0/4037247575 conn(0x564d73a23600 msgr2=0x564d73a6bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 -- 172.21.15.113:0/4037247575 shutdown_connections 2024-04-25T00:08:21.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.899+0000 7faf32a30740 1 -- 172.21.15.113:0/4037247575 wait complete. 2024-04-25T00:08:21.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.900+0000 7faf32a30740 1 -- start start 2024-04-25T00:08:21.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.900+0000 7faf32a30740 1 Processor -- start 2024-04-25T00:08:21.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a42630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a6e600 0x564d73a53f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a544a0 0x564d73a24910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564d73a277e0 con 0x564d73a6e600 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564d73945f50 con 0x564d73a544a0 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564d73a21530 con 0x564d73a50d60 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 10 client.0 _ll_register_callbacks cb 0x564d73981000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a6e600 0x564d73a53f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a6e600 0x564d73a53f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48532/0 (socket says 172.21.15.113:48532) 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2e1e9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a544a0 0x564d73a24910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 -- 172.21.15.113:0/762167168 learned_addr learned my addr 172.21.15.113:0/762167168 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a42630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 -- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 msgr2=0x564d73a42630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a42630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 -- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a544a0 msgr2=0x564d73a24910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a544a0 0x564d73a24910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 -- 172.21.15.113:0/762167168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7369e3f0 con 0x564d73a6e600 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a6e600 0x564d73a53f60 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7faf18003f50 tx=0x7faf180105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a261b8b2f1dcc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf08ff9640 1 -- 172.21.15.113:0/762167168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18007230 con 0x564d73a6e600 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf08ff9640 1 -- 172.21.15.113:0/762167168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7faf180073d0 con 0x564d73a6e600 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf2400e7b0 con 0x564d73a6e600 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf08ff9640 1 -- 172.21.15.113:0/762167168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7faf18021050 con 0x564d73a6e600 2024-04-25T00:08:21.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 10 client.10191 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.901+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x564d7365e0f0 con 0x564d73a6e600 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf08ff9640 1 -- 172.21.15.113:0/762167168 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2400e7b0 con 0x564d73a6e600 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf08ff9640 1 -- 172.21.15.113:0/762167168 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7faf18024020 con 0x564d73a6e600 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf32a30740 10 client.10191 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf32a30740 10 client.10191 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf32a30740 20 client.10191 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf32a30740 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a7fea0 0x564d73a80280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf32a30740 4 client.10191 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d73997bf0 con 0x564d73a7fea0 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a7fea0 0x564d73a80280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf08ff9640 10 client.10191 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.902+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a7fea0 0x564d73a80280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93cd50d149b6f436 server_cookie=2e6e6ccfed9e2e3 in_seq=0 out_seq=0 2024-04-25T00:08:21.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf08ff9640 1 -- 172.21.15.113:0/762167168 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d73997bf0 con 0x564d73a7fea0 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf08ff9640 10 client.10191 handle_command_reply: tid=0 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 client.10191 shutdown 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a6e600 msgr2=0x564d73a53f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a6e600 0x564d73a53f60 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7faf18003f50 tx=0x7faf180105b0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 shutdown_connections 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a544a0 0x564d73a24910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a7fea0 0x564d73a80280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a6e600 0x564d73a53f60 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf08ff9640 0 client.10191 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 --2- 172.21.15.113:0/762167168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a42630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 >> 172.21.15.113:0/762167168 conn(0x564d73a26a40 msgr2=0x564d73a6f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 shutdown_connections 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 1 -- 172.21.15.113:0/762167168 wait complete. 2024-04-25T00:08:21.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.903+0000 7faf32a30740 20 client.10191 trim_cache size 0 max 16384 2024-04-25T00:08:21.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.920+0000 7faf32a30740 1 Processor -- start 2024-04-25T00:08:21.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf32a30740 1 -- start start 2024-04-25T00:08:21.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf32a30740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a53e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a544a0 0x564d73a640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a64620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf32a30740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faf28200a10 con 0x564d73a50d60 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf281ffb40 con 0x564d73a6e600 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faf280cabe0 con 0x564d73a544a0 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf2d9e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a53e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf2d9e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a53e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48546/0 (socket says 172.21.15.113:48546) 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf2d9e8640 1 -- 172.21.15.113:0/3751034451 learned_addr learned my addr 172.21.15.113:0/3751034451 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a544a0 0x564d73a640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.921+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a64620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf2d9e8640 1 -- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a544a0 msgr2=0x564d73a640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a544a0 0x564d73a640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf2d9e8640 1 -- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 msgr2=0x564d73a64620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a64620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf2d9e8640 1 -- 172.21.15.113:0/3751034451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d73a24c80 con 0x564d73a50d60 2024-04-25T00:08:21.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a53e90 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0975c50600157a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf0affd640 1 -- 172.21.15.113:0/3751034451 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2400ec60 con 0x564d73a50d60 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf0affd640 1 -- 172.21.15.113:0/3751034451 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2400ee00 con 0x564d73a50d60 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf0affd640 1 -- 172.21.15.113:0/3751034451 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2402d230 con 0x564d73a50d60 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 -- 172.21.15.113:0/3751034451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 msgr2=0x564d73a53e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a53e90 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24017070 comp rx=0 tx=0).stop 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 -- 172.21.15.113:0/3751034451 shutdown_connections 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a64620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a544a0 0x564d73a640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 --2- 172.21.15.113:0/3751034451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a50d60 0x564d73a53e90 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 -- 172.21.15.113:0/3751034451 >> 172.21.15.113:0/3751034451 conn(0x564d73a26a40 msgr2=0x564d73a41aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 -- 172.21.15.113:0/3751034451 shutdown_connections 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.922+0000 7faf32a30740 1 -- 172.21.15.113:0/3751034451 wait complete. 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.923+0000 7faf32a30740 1 -- start start 2024-04-25T00:08:21.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.923+0000 7faf32a30740 1 Processor -- start 2024-04-25T00:08:21.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf32a30740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a544a0 0x564d73a6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf32a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf32a30740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564d73a51140 con 0x564d73a544a0 2024-04-25T00:08:21.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf281611f0 con 0x564d73a6e600 2024-04-25T00:08:21.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf32a30740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564d73a6d9f0 con 0x564d73a50d60 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf32a30740 10 client.0 _ll_register_callbacks cb 0x7faf18005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a544a0 0x564d73a6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a544a0 0x564d73a6caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48560/0 (socket says 172.21.15.113:48560) 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 -- 172.21.15.113:0/3437833628 learned_addr learned my addr 172.21.15.113:0/3437833628 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2e1e9640 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 -- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 msgr2=0x564d73a62c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a62c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 -- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 msgr2=0x564d73a6cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 -- 172.21.15.113:0/3437833628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d73a42060 con 0x564d73a544a0 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf2d1e7640 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a544a0 0x564d73a6caa0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7faf18003e00 tx=0x7faf18000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6a5adc813e81b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:21.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.924+0000 7faf08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf08ff9640 1 -- 172.21.15.113:0/3437833628 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18014810 con 0x564d73a544a0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf08ff9640 1 -- 172.21.15.113:0/3437833628 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7faf180149b0 con 0x564d73a544a0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf2400ee00 con 0x564d73a544a0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf08ff9640 1 -- 172.21.15.113:0/3437833628 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7faf18014bb0 con 0x564d73a544a0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 10 client.10197 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x564d73ae6930 con 0x564d73a544a0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf08ff9640 1 -- 172.21.15.113:0/3437833628 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2400ee00 con 0x564d73a544a0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf08ff9640 1 -- 172.21.15.113:0/3437833628 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7faf18024040 con 0x564d73a544a0 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 10 client.10197 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 10 client.10197 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T00:08:21.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 20 client.10197 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a527d0 0x564d73ae5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 4 client.10197 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d73991f40 con 0x564d73a527d0 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.925+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a527d0 0x564d73ae5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf08ff9640 10 client.10197 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf2d9e8640 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a527d0 0x564d73ae5c60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900f14f35c170833 server_cookie=67e38d51791940de in_seq=0 out_seq=0 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf08ff9640 1 -- 172.21.15.113:0/3437833628 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x564d73991f40 con 0x564d73a527d0 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf08ff9640 10 client.10197 handle_command_reply: tid=0 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 client.10197 shutdown 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a544a0 msgr2=0x564d73a6caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a544a0 0x564d73a6caa0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7faf18003e00 tx=0x7faf18000ed0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 shutdown_connections 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d73a6e600 0x564d73a6cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x564d73a527d0 0x564d73ae5c60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564d73a544a0 0x564d73a6caa0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 --2- 172.21.15.113:0/3437833628 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564d73a50d60 0x564d73a62c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf08ff9640 0 client.10197 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 >> 172.21.15.113:0/3437833628 conn(0x564d73a26a40 msgr2=0x564d73a64440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 shutdown_connections 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 1 -- 172.21.15.113:0/3437833628 wait complete. 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.926+0000 7faf32a30740 20 client.10197 trim_cache size 0 max 16384 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:21.934 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout: "85ef7ff3-f8ae-43e6-b95c-83fc78e0494a": { 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "85ef7ff3-f8ae-43e6-b95c-83fc78e0494a", 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,repair" 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:21.935 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf0c06be30 msgr2=0x7faf0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf0c06be30 0x7faf0c06e2f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faf24005880 tx=0x7faf24009040 comp rx=0 tx=0).stop 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf2815e510 msgr2=0x7faf280c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf2815e510 0x7faf280c7d50 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7faf20009eb0 tx=0x7faf20004a40 comp rx=0 tx=0).stop 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 shutdown_connections 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faf0c06be30 0x7faf0c06e2f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf280b6100 0x7faf280b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf2815e510 0x7faf280c7d50 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 --2- 172.21.15.113:0/3074327716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf280b5ca0 0x7faf28149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:21.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 >> 172.21.15.113:0/3074327716 conn(0x7faf2806b720 msgr2=0x7faf2815d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:21.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 shutdown_connections 2024-04-25T00:08:21.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:21.927+0000 7faf2fc4c640 1 -- 172.21.15.113:0/3074327716 wait complete. 2024-04-25T00:08:21.938 INFO:tasks.cephfs.filesystem:scrub status for tag:85ef7ff3-f8ae-43e6-b95c-83fc78e0494a - {'path': '/', 'tag': '85ef7ff3-f8ae-43e6-b95c-83fc78e0494a', 'options': 'recursive,repair'} 2024-04-25T00:08:23.872 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:08:23.875 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:08:51.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:51.940 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T00:08:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.050+0000 7f8644568640 1 Processor -- start 2024-04-25T00:08:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.050+0000 7f8644568640 1 -- start start 2024-04-25T00:08:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.051+0000 7f8644568640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f863c0b3080 0x7f863c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.051+0000 7f8644568640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f863c15f4f0 con 0x7f863c15e2d0 2024-04-25T00:08:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.051+0000 7f8644568640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f863c15f7d0 con 0x7f863c15e6b0 2024-04-25T00:08:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.051+0000 7f8644568640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f863c0cde00 con 0x7f863c0b3080 2024-04-25T00:08:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.052+0000 7f8641b03640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f863c0b3080 0x7f863c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.052+0000 7f8642304640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f863c15e2d0 0x7f863c15ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37194/0 (socket says 172.21.15.113:37194) 2024-04-25T00:08:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.052+0000 7f8642304640 1 -- 172.21.15.113:0/3444175882 learned_addr learned my addr 172.21.15.113:0/3444175882 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.052+0000 7f8641b03640 1 -- 172.21.15.113:0/3444175882 >> v1:172.21.15.178:6789/0 conn(0x7f863c15e6b0 legacy=0x7f863c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.052+0000 7f8641302640 1 -- 172.21.15.113:0/3444175882 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701220342 0 0) 0x7f863c15f4f0 con 0x7f863c15e2d0 2024-04-25T00:08:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.052+0000 7f8641b03640 1 -- 172.21.15.113:0/3444175882 >> v1:172.21.15.113:6789/0 conn(0x7f863c15e2d0 legacy=0x7f863c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.052+0000 7f8641b03640 1 -- 172.21.15.113:0/3444175882 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c15fab0 con 0x7f863c0b3080 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8641b03640 1 --2- 172.21.15.113:0/3444175882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c0b3080 0x7f863c0b4330 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f862c0097a0 tx=0x7f862c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740cb72c0d6afb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8641302640 1 -- 172.21.15.113:0/3444175882 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c037070 con 0x7f863c0b3080 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8641302640 1 -- 172.21.15.113:0/3444175882 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862c032440 con 0x7f863c0b3080 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8641302640 1 -- 172.21.15.113:0/3444175882 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c032810 con 0x7f863c0b3080 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8644568640 1 -- 172.21.15.113:0/3444175882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c0b3080 msgr2=0x7f863c0b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8644568640 1 --2- 172.21.15.113:0/3444175882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c0b3080 0x7f863c0b4330 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f862c0097a0 tx=0x7f862c02a1e0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8644568640 1 -- 172.21.15.113:0/3444175882 shutdown_connections 2024-04-25T00:08:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8644568640 1 --2- 172.21.15.113:0/3444175882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c0b3080 0x7f863c0b4330 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8644568640 1 -- 172.21.15.113:0/3444175882 >> 172.21.15.113:0/3444175882 conn(0x7f863c06b0f0 msgr2=0x7f863c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8644568640 1 -- 172.21.15.113:0/3444175882 shutdown_connections 2024-04-25T00:08:52.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.053+0000 7f8644568640 1 -- 172.21.15.113:0/3444175882 wait complete. 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 Processor -- start 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 -- start start 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f863c0b3080 0x7f863c13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f863c15e2d0 0x7f863c153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c15e6b0 0x7f863c1e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f863c0b49f0 con 0x7f863c15e6b0 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f863c0b4870 con 0x7f863c0b3080 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8644568640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f863c0b4b70 con 0x7f863c15e2d0 2024-04-25T00:08:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8642b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c15e6b0 0x7f863c1e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8642b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c15e6b0 0x7f863c1e6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51354/0 (socket says 172.21.15.113:51354) 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.055+0000 7f8642b05640 1 -- 172.21.15.113:0/1558505094 learned_addr learned my addr 172.21.15.113:0/1558505094 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8642304640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f863c0b3080 0x7f863c13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8641b03640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f863c15e2d0 0x7f863c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8642b05640 1 -- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f863c15e2d0 msgr2=0x7f863c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8642b05640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f863c15e2d0 0x7f863c153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8642b05640 1 -- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f863c0b3080 msgr2=0x7f863c13bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8642b05640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f863c0b3080 0x7f863c13bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8642b05640 1 -- 172.21.15.113:0/1558505094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c154da0 con 0x7f863c15e6b0 2024-04-25T00:08:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8642b05640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c15e6b0 0x7f863c1e6ba0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f863400c780 tx=0x7f863400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8632ffd640 1 -- 172.21.15.113:0/1558505094 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634004b60 con 0x7f863c15e6b0 2024-04-25T00:08:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8632ffd640 1 -- 172.21.15.113:0/1558505094 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8634004d00 con 0x7f863c15e6b0 2024-04-25T00:08:52.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c155020 con 0x7f863c15e6b0 2024-04-25T00:08:52.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8632ffd640 1 -- 172.21.15.113:0/1558505094 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863400aa90 con 0x7f863c15e6b0 2024-04-25T00:08:52.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.056+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c1555e0 con 0x7f863c15e6b0 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.058+0000 7f8632ffd640 1 -- 172.21.15.113:0/1558505094 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f863400ac30 con 0x7f863c15e6b0 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.058+0000 7f8632ffd640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f862006be30 0x7f862006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.058+0000 7f8632ffd640 1 -- 172.21.15.113:0/1558505094 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f863408b390 con 0x7f863c15e6b0 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.058+0000 7f8642304640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f862006be30 0x7f862006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.059+0000 7f8642304640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f862006be30 0x7f862006e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8638000f60 tx=0x7f8638007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.059+0000 7f8647272740 1 Processor -- start 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.059+0000 7f8647272740 1 -- start start 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911dd7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 0x563911df4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8647272740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911dace50 0x563911df30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8647272740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563911d0c6b0 con 0x563911dace50 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563911cb4880 con 0x563911dd7170 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563911d08670 con 0x563911df4a10 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8642b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911dace50 0x563911df30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8642b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911dace50 0x563911df30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51370/0 (socket says 172.21.15.113:51370) 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8642b05640 1 -- 172.21.15.113:0/2018978484 learned_addr learned my addr 172.21.15.113:0/2018978484 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8642304640 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911dd7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.060+0000 7f8641b03640 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 0x563911df4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f8642b05640 1 -- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 msgr2=0x563911df4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f8642b05640 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 0x563911df4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f8642b05640 1 -- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 msgr2=0x563911dd7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f8642b05640 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911dd7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f8642b05640 1 -- 172.21.15.113:0/2018978484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563911aea7d0 con 0x563911dace50 2024-04-25T00:08:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f8642304640 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911dd7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f8642b05640 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911dace50 0x563911df30d0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f86340898e0 tx=0x7f8634059d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfda39fe60c5752c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f861ffff640 1 -- 172.21.15.113:0/2018978484 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634062220 con 0x563911dace50 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f861ffff640 1 -- 172.21.15.113:0/2018978484 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8634057940 con 0x563911dace50 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.061+0000 7f861ffff640 1 -- 172.21.15.113:0/2018978484 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634022da0 con 0x563911dace50 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 -- 172.21.15.113:0/2018978484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911dace50 msgr2=0x563911df30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911dace50 0x563911df30d0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f86340898e0 tx=0x7f8634059d40 comp rx=0 tx=0).stop 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 -- 172.21.15.113:0/2018978484 shutdown_connections 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911dace50 0x563911df30d0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 0x563911df4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 --2- 172.21.15.113:0/2018978484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911dd7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 -- 172.21.15.113:0/2018978484 >> 172.21.15.113:0/2018978484 conn(0x563911da9a10 msgr2=0x563911df1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 -- 172.21.15.113:0/2018978484 shutdown_connections 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 -- 172.21.15.113:0/2018978484 wait complete. 2024-04-25T00:08:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 -- start start 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.062+0000 7f8647272740 1 Processor -- start 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911da2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8647272740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911da27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911dd27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8647272740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563911d0c6b0 con 0x563911df4a10 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563911cc7f50 con 0x563911dd7170 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563911dadbf0 con 0x563911df1480 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8647272740 10 client.0 _ll_register_callbacks cb 0x563911d03000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8641b03640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911da27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8641b03640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911da27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51376/0 (socket says 172.21.15.113:51376) 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8641b03640 1 -- 172.21.15.113:0/2902552896 learned_addr learned my addr 172.21.15.113:0/2902552896 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.063+0000 7f8642304640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911da2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8642b05640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911dd27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8641b03640 1 -- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 msgr2=0x563911dd27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8641b03640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911dd27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8641b03640 1 -- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 msgr2=0x563911da2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8641b03640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911da2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8641b03640 1 -- 172.21.15.113:0/2902552896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563911a203f0 con 0x563911df4a10 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8642b05640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911dd27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8641b03640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911da27a0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f862c0098c0 tx=0x7f862c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32276fc72d99db44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f861d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f861d7fa640 1 -- 172.21.15.113:0/2902552896 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c042070 con 0x563911df4a10 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f861d7fa640 1 -- 172.21.15.113:0/2902552896 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f862c032b80 con 0x563911df4a10 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8634057940 con 0x563911df4a10 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f861d7fa640 1 -- 172.21.15.113:0/2902552896 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f862c050050 con 0x563911df4a10 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8647272740 10 client.10212 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5639119e00f0 con 0x563911df4a10 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.064+0000 7f861d7fa640 1 -- 172.21.15.113:0/2902552896 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8634057940 con 0x563911df4a10 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f861d7fa640 1 -- 172.21.15.113:0/2902552896 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7f862c053020 con 0x563911df4a10 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8647272740 10 client.10212 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8647272740 10 client.10212 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8647272740 20 client.10212 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8647272740 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd2d80 0x563911dd3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8647272740 4 client.10212 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563911d19bf0 con 0x563911dd2d80 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8642304640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd2d80 0x563911dd3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f861d7fa640 10 client.10212 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.065+0000 7f8642304640 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd2d80 0x563911dd3160 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad13262ad4144b8 server_cookie=4862f9cf127273af in_seq=0 out_seq=0 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f861d7fa640 1 -- 172.21.15.113:0/2902552896 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563911d19bf0 con 0x563911dd2d80 2024-04-25T00:08:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f861d7fa640 10 client.10212 handle_command_reply: tid=0 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 client.10212 shutdown 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 msgr2=0x563911da27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911da27a0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f862c0098c0 tx=0x7f862c004730 comp rx=0 tx=0).stop 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 shutdown_connections 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd2d80 0x563911dd3160 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911dd27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911da27a0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 --2- 172.21.15.113:0/2902552896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911da2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f861d7fa640 0 client.10212 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 >> 172.21.15.113:0/2902552896 conn(0x563911dace50 msgr2=0x563911daad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 shutdown_connections 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.066+0000 7f8647272740 1 -- 172.21.15.113:0/2902552896 wait complete. 2024-04-25T00:08:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.067+0000 7f8647272740 20 client.10212 trim_cache size 0 max 16384 2024-04-25T00:08:52.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.084+0000 7f8647272740 1 Processor -- start 2024-04-25T00:08:52.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.084+0000 7f8647272740 1 -- start start 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911df5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8647272740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df1480 0x563911e15d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 0x563911df1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8647272740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1fc340 con 0x563911df1480 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1e5cc0 con 0x563911dd7170 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f863c06dc60 con 0x563911df4a10 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df1480 0x563911e15d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df1480 0x563911e15d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51380/0 (socket says 172.21.15.113:51380) 2024-04-25T00:08:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 -- 172.21.15.113:0/4039351289 learned_addr learned my addr 172.21.15.113:0/4039351289 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 -- 172.21.15.113:0/4039351289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 msgr2=0x563911df1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 --2- 172.21.15.113:0/4039351289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 0x563911df1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 -- 172.21.15.113:0/4039351289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 msgr2=0x563911df5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 --2- 172.21.15.113:0/4039351289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911df5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 -- 172.21.15.113:0/4039351289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563911dd34d0 con 0x563911df1480 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f8641b03640 1 --2- 172.21.15.113:0/4039351289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df1480 0x563911e15d20 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f862c000c00 tx=0x7f862c03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaf445ee6abde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f861f7fe640 1 -- 172.21.15.113:0/4039351289 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c042040 con 0x563911df1480 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f861f7fe640 1 -- 172.21.15.113:0/4039351289 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862c032980 con 0x563911df1480 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.085+0000 7f861f7fe640 1 -- 172.21.15.113:0/4039351289 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c037040 con 0x563911df1480 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 -- 172.21.15.113:0/4039351289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df1480 msgr2=0x563911e15d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 --2- 172.21.15.113:0/4039351289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df1480 0x563911e15d20 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f862c000c00 tx=0x7f862c03e070 comp rx=0 tx=0).stop 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 -- 172.21.15.113:0/4039351289 shutdown_connections 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 --2- 172.21.15.113:0/4039351289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df4a10 0x563911df1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 --2- 172.21.15.113:0/4039351289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df1480 0x563911e15d20 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 --2- 172.21.15.113:0/4039351289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911df5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 -- 172.21.15.113:0/4039351289 >> 172.21.15.113:0/4039351289 conn(0x563911dace50 msgr2=0x563911dac490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 -- 172.21.15.113:0/4039351289 shutdown_connections 2024-04-25T00:08:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 -- 172.21.15.113:0/4039351289 wait complete. 2024-04-25T00:08:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 -- start start 2024-04-25T00:08:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.086+0000 7f8647272740 1 Processor -- start 2024-04-25T00:08:52.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911e153b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8647272740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911e040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8647272740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911dd9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8647272740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563911dda960 con 0x563911df4a10 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f863c13d250 con 0x563911dd7170 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8647272740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563911e04cd0 con 0x563911df1480 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8647272740 10 client.0 _ll_register_callbacks cb 0x7f862c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8642b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911dd9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8642b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911dd9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51396/0 (socket says 172.21.15.113:51396) 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.087+0000 7f8642b05640 1 -- 172.21.15.113:0/3972174617 learned_addr learned my addr 172.21.15.113:0/3972174617 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8642b05640 1 -- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 msgr2=0x563911e040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8642b05640 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911e040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8642b05640 1 -- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 msgr2=0x563911e153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8642b05640 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911e153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8642b05640 1 -- 172.21.15.113:0/3972174617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563911e158f0 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8642b05640 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911dd9910 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f8634061cf0 tx=0x7f8634089790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fd073e3fdc90fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f861d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f861d7fa640 1 -- 172.21.15.113:0/3972174617 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634062220 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f861d7fa640 1 -- 172.21.15.113:0/3972174617 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f8634022da0 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f862c032980 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f861d7fa640 1 -- 172.21.15.113:0/3972174617 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8634055cb0 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 10 client.10218 fetch_fsmap learned FSMap version 85 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x563911e04a10 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f861d7fa640 1 -- 172.21.15.113:0/3972174617 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862c032980 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f861d7fa640 1 -- 172.21.15.113:0/3972174617 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7f8634089ce0 con 0x563911df4a10 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 10 client.10218 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 10 client.10218 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T00:08:52.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 20 client.10218 populate_metadata read hostname 'smithi113' 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd82e0 0x563911dd9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 4 client.10218 mds_command: new command op to 9987 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.088+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 --> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563911d13f40 con 0x563911dd82e0 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8642304640 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd82e0 0x563911dd9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f861d7fa640 10 client.10218 ms_handle_connect on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8642304640 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd82e0 0x563911dd9e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61fa561b6c96a46 server_cookie=6d748c6a478e58c in_seq=0 out_seq=0 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f861d7fa640 1 -- 172.21.15.113:0/3972174617 <== mds.0 v2:172.21.15.178:6832/340854702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563911d13f40 con 0x563911dd82e0 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f861d7fa640 10 client.10218 handle_command_reply: tid=0 2024-04-25T00:08:52.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 client.10218 shutdown 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 msgr2=0x563911dd9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911dd9910 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f8634061cf0 tx=0x7f8634089790 comp rx=0 tx=0).stop 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 shutdown_connections 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:6832/340854702,v1:172.21.15.178:6834/340854702] conn(0x563911dd82e0 0x563911dd9e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563911df4a10 0x563911dd9910 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f861d7fa640 0 client.10218 ms_handle_reset on v2:172.21.15.178:6832/340854702 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563911df1480 0x563911e040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 --2- 172.21.15.113:0/3972174617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563911dd7170 0x563911e153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 >> 172.21.15.113:0/3972174617 conn(0x563911dace50 msgr2=0x563911dad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 shutdown_connections 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.089+0000 7f8647272740 1 -- 172.21.15.113:0/3972174617 wait complete. 2024-04-25T00:08:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8647272740 20 client.10218 trim_cache size 0 max 16384 2024-04-25T00:08:52.098 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:52.098 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:08:52.098 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:08:52.098 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f862006be30 msgr2=0x7f862006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f862006be30 0x7f862006e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8638000f60 tx=0x7f8638007b50 comp rx=0 tx=0).stop 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c15e6b0 msgr2=0x7f863c1e6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c15e6b0 0x7f863c1e6ba0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f863400c780 tx=0x7f863400cc40 comp rx=0 tx=0).stop 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 shutdown_connections 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f862006be30 0x7f862006e2f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863c15e6b0 0x7f863c1e6ba0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f863c15e2d0 0x7f863c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 --2- 172.21.15.113:0/1558505094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f863c0b3080 0x7f863c13bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 >> 172.21.15.113:0/1558505094 conn(0x7f863c06b0f0 msgr2=0x7f863c0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 shutdown_connections 2024-04-25T00:08:52.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.090+0000 7f8644568640 1 -- 172.21.15.113:0/1558505094 wait complete. 2024-04-25T00:08:52.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:08:52.101 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> import stat 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> import json 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> import sys 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> try: 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> except OSError as e: 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> sys.exit(e.errno) 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> print(json.dumps( 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T00:08:52.102 DEBUG:teuthology.orchestra.run.smithi113:> indent=2)) 2024-04-25T00:08:52.103 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:08:52.169 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:52.169 INFO:teuthology.orchestra.run.smithi113.stdout: "st_mode": 33188, 2024-04-25T00:08:52.169 INFO:teuthology.orchestra.run.smithi113.stdout: "st_ino": 1099511627777, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_dev": 43, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_nlink": 1, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_uid": 1000, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_gid": 1316, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_size": 6291456, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_atime": 1714003700.7150195, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_mtime": 1714003700.734019, 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout: "st_ctime": 1714003700.734019 2024-04-25T00:08:52.170 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:52.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:08:52.179 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.315+0000 7fcee0e2c640 1 Processor -- start 2024-04-25T00:08:52.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.315+0000 7fcee0e2c640 1 -- start start 2024-04-25T00:08:52.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fcee0e2c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 0x7fcedc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fcee0e2c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcedc0b3af0 0x7fcedc0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fcee0e2c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcedc0b2d50 con 0x7fcedc15a6b0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fcee0e2c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcedc072a70 con 0x7fcedc0b3af0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fcee0e2c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc0b3010 con 0x7fcedc072ea0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fceda59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcedc15a6b0 0x7fcedc070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37200/0 (socket says 172.21.15.113:37200) 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fceda59c640 1 -- 172.21.15.113:0/2520509271 learned_addr learned my addr 172.21.15.113:0/2520509271 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389740798 0 0) 0x7fcedc0b2d50 con 0x7fcedc15a6b0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fced9d9b640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 0x7fcedc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcec8003520 con 0x7fcedc15a6b0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.316+0000 7fcedad9d640 1 --2- 172.21.15.113:0/2520509271 >> v2:172.21.15.178:3300/0 conn(0x7fcedc0b3af0 0x7fcedc0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1876244976 0 0) 0x7fcec8003520 con 0x7fcedc15a6b0 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 msgr2=0x7fcedc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 0x7fcedc071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 >> v2:172.21.15.178:3300/0 conn(0x7fcedc0b3af0 msgr2=0x7fcedc0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 --2- 172.21.15.113:0/2520509271 >> v2:172.21.15.178:3300/0 conn(0x7fcedc0b3af0 0x7fcedc0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc0b32f0 con 0x7fcedc15a6b0 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fcecc0019f0 con 0x7fcedc15a6b0 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fcedad9d640 1 --2- 172.21.15.113:0/2520509271 >> v2:172.21.15.178:3300/0 conn(0x7fcedc0b3af0 0x7fcedc0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 >> v1:172.21.15.113:6789/0 conn(0x7fcedc15a6b0 legacy=0x7fcedc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 0x7fcec80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcec8003f30 0x7fcec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcec8004d40 0x7fcec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc072a70 con 0x7fcec8003f30 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0b3010 con 0x7fcec8004d40 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcec800ba10 con 0x7fcedc072ea0 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced9d9b640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcec8003f30 0x7fcec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fceda59c640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 0x7fcec80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fcedad9d640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcec8004d40 0x7fcec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced9d9b640 1 -- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 msgr2=0x7fcec80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced9d9b640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcedc072ea0 0x7fcec80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced9d9b640 1 -- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcec8004d40 msgr2=0x7fcec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced9d9b640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcec8004d40 0x7fcec80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fced9d9b640 1 -- 172.21.15.113:0/2520509271 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec8003520 con 0x7fcec8003f30 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.317+0000 7fcedad9d640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcec8004d40 0x7fcec80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fced9d9b640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcec8003f30 0x7fcec8004800 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009a90 tx=0x7fcec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c32b4c27931514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4036070 con 0x7fcec8003f30 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec4031400 con 0x7fcec8003f30 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fced959a640 1 -- 172.21.15.113:0/2520509271 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4009690 con 0x7fcec8003f30 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fcee0e2c640 1 -- 172.21.15.113:0/2520509271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcec8003f30 msgr2=0x7fcec8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fcee0e2c640 1 --2- 172.21.15.113:0/2520509271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcec8003f30 0x7fcec8004800 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009a90 tx=0x7fcec8005830 comp rx=0 tx=0).stop 2024-04-25T00:08:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fcedad9d640 1 -- 172.21.15.113:0/2520509271 reap_dead start 2024-04-25T00:08:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fcee0e2c640 1 -- 172.21.15.113:0/2520509271 shutdown_connections 2024-04-25T00:08:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fcee0e2c640 1 -- 172.21.15.113:0/2520509271 >> 172.21.15.113:0/2520509271 conn(0x7fcedc0b1a50 msgr2=0x7fcedc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fcee0e2c640 1 -- 172.21.15.113:0/2520509271 shutdown_connections 2024-04-25T00:08:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.318+0000 7fcee0e2c640 1 -- 172.21.15.113:0/2520509271 wait complete. 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 Processor -- start 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 -- start start 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcec8003f30 0x7fcedc140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc15d480 0x7fcedc15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcedc15afc0 0x7fcedc15b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc15aa90 con 0x7fcedc15d480 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc072720 con 0x7fcedc15afc0 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.320+0000 7fcee0e2c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc15ac10 con 0x7fcec8003f30 2024-04-25T00:08:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc15d480 0x7fcedc15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc15d480 0x7fcedc15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51400/0 (socket says 172.21.15.113:51400) 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 -- 172.21.15.113:0/972938743 learned_addr learned my addr 172.21.15.113:0/972938743 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 -- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcec8003f30 msgr2=0x7fcedc140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fceda59c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcec8003f30 0x7fcedc140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fcedad9d640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcedc15afc0 0x7fcedc15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcec8003f30 0x7fcedc140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 -- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcedc15afc0 msgr2=0x7fcedc15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcedc15afc0 0x7fcedc15b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 -- 172.21.15.113:0/972938743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc15bcf0 con 0x7fcedc15d480 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fcedad9d640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcedc15afc0 0x7fcedc15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced9d9b640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc15d480 0x7fcedc15d900 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fcec4000c00 tx=0x7fcec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced2ffd640 1 -- 172.21.15.113:0/972938743 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4040020 con 0x7fcedc15d480 2024-04-25T00:08:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced2ffd640 1 -- 172.21.15.113:0/972938743 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec4003c60 con 0x7fcedc15d480 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fced2ffd640 1 -- 172.21.15.113:0/972938743 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4036040 con 0x7fcedc15d480 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc15bfd0 con 0x7fcedc15d480 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.321+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc15c590 con 0x7fcedc15d480 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.322+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc0691a0 con 0x7fcedc15d480 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.323+0000 7fced2ffd640 1 -- 172.21.15.113:0/972938743 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcec4043020 con 0x7fcedc15d480 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.323+0000 7fced2ffd640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcebc06be30 0x7fcebc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.323+0000 7fceda59c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcebc06be30 0x7fcebc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.323+0000 7fced2ffd640 1 -- 172.21.15.113:0/972938743 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fcec40aae30 con 0x7fcedc15d480 2024-04-25T00:08:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.324+0000 7fceda59c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcebc06be30 0x7fcebc06e2f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcecc004c20 tx=0x7fcecc006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.328+0000 7fced2ffd640 1 -- 172.21.15.113:0/972938743 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcec404a070 con 0x7fcedc15d480 2024-04-25T00:08:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.536+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcedc0b2900 con 0x7fcedc15d480 2024-04-25T00:08:52.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.537+0000 7fced2ffd640 1 -- 172.21.15.113:0/972938743 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7fcec40044e0 con 0x7fcedc15d480 2024-04-25T00:08:52.537 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:52.537 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:52.538 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:52.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcebc06be30 msgr2=0x7fcebc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcebc06be30 0x7fcebc06e2f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcecc004c20 tx=0x7fcecc006040 comp rx=0 tx=0).stop 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc15d480 msgr2=0x7fcedc15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc15d480 0x7fcedc15d900 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fcec4000c00 tx=0x7fcec4004730 comp rx=0 tx=0).stop 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 shutdown_connections 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcebc06be30 0x7fcebc06e2f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcedc15afc0 0x7fcedc15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc15d480 0x7fcedc15d900 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 --2- 172.21.15.113:0/972938743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcec8003f30 0x7fcedc140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 >> 172.21.15.113:0/972938743 conn(0x7fcedc0b1a50 msgr2=0x7fcedc0cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 shutdown_connections 2024-04-25T00:08:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.539+0000 7fcee0e2c640 1 -- 172.21.15.113:0/972938743 wait complete. 2024-04-25T00:08:52.551 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T00:08:52.680 INFO:teuthology.orchestra.run.smithi178.stdout:{"message":"","return_code":0} 2024-04-25T00:08:52.681 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T00:08:52.681 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T00:08:52.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.723+0000 7f70ee9cbf40 1 Processor -- start 2024-04-25T00:08:52.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.723+0000 7f70ee9cbf40 1 -- start start 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ee9cbf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 0x55950e56e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ee9cbf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 0x55950e57a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ee9cbf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e570fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ee9cbf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55950e4a56a0 con 0x55950e576410 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ee9cbf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55950e44a640 con 0x55950e4d2270 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ee9cbf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55950e4a32e0 con 0x55950e4a22c0 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e570fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e570fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51422/0 (socket says 172.21.15.113:51422) 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 -- 172.21.15.113:0/379602243 learned_addr learned my addr 172.21.15.113:0/379602243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ec4cc640 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 0x55950e56e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70e3fff640 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 0x55950e57a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 -- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 msgr2=0x55950e56e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 0x55950e56e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 -- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 msgr2=0x55950e57a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 0x55950e57a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 -- 172.21.15.113:0/379602243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950e4eaa80 con 0x55950e576410 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70ecccd640 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e570fe0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f70e4001620 tx=0x7f70e4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1ee47165d4dd40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.724+0000 7f70eb4ca640 1 -- 172.21.15.113:0/379602243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4017070 con 0x55950e576410 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70eb4ca640 1 -- 172.21.15.113:0/379602243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e400f710 con 0x55950e576410 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70eb4ca640 1 -- 172.21.15.113:0/379602243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e400fb40 con 0x55950e576410 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/379602243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 msgr2=0x55950e570fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e570fe0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f70e4001620 tx=0x7f70e4001ae0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/379602243 shutdown_connections 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 0x55950e57a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e570fe0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/379602243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 0x55950e56e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/379602243 >> 172.21.15.113:0/379602243 conn(0x55950e4ee0e0 msgr2=0x55950e576e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/379602243 shutdown_connections 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.725+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/379602243 wait complete. 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 Processor -- start 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 -- start start 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 0x55950e56dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 0x55950e57b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e57b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55950e4eb350 con 0x55950e576410 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55950e579810 con 0x55950e4d2270 2024-04-25T00:08:52.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ee9cbf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55950e579a60 con 0x55950e4a22c0 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ecccd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e57b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ecccd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e57b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51432/0 (socket says 172.21.15.113:51432) 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.726+0000 7f70ecccd640 1 -- 172.21.15.113:0/1228186870 learned_addr learned my addr 172.21.15.113:0/1228186870 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ec4cc640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 0x55950e56dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70e3fff640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 0x55950e57b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ecccd640 1 -- 172.21.15.113:0/1228186870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 msgr2=0x55950e56dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ecccd640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55950e4a22c0 0x55950e56dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ecccd640 1 -- 172.21.15.113:0/1228186870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 msgr2=0x55950e57b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ecccd640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55950e4d2270 0x55950e57b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ecccd640 1 -- 172.21.15.113:0/1228186870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950e56e260 con 0x55950e576410 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ecccd640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e57b590 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x55950e4a2d50 tx=0x7f70e4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ea4c8640 1 -- 172.21.15.113:0/1228186870 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4017040 con 0x55950e576410 2024-04-25T00:08:52.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ea4c8640 1 -- 172.21.15.113:0/1228186870 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e40126d0 con 0x55950e576410 2024-04-25T00:08:52.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55950e4e4320 con 0x55950e576410 2024-04-25T00:08:52.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ea4c8640 1 -- 172.21.15.113:0/1228186870 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4012b20 con 0x55950e576410 2024-04-25T00:08:52.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.727+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950e4e4040 con 0x55950e576410 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.728+0000 7f70ea4c8640 1 -- 172.21.15.113:0/1228186870 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70e4012cc0 con 0x55950e576410 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70ea4c8640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f70cc06a810 0x7f70cc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70ea4c8640 1 -- 172.21.15.113:0/1228186870 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f70e408b3f0 con 0x55950e576410 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70ec4cc640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f70cc06a810 0x7f70cc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55950e5767f0 0x55950e611ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 --> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] -- osd_op(unknown.0.0:1 23.7 23:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e158) -- 0x55950e571310 con 0x55950e5767f0 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70e3fff640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55950e5767f0 0x55950e611ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70ec4cc640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f70cc06a810 0x7f70cc06ccd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x55950e57ae80 tx=0x7f70dc00c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.729+0000 7f70e3fff640 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55950e5767f0 0x55950e611ad0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70e3fff640 1 -- 172.21.15.113:0/1228186870 <== osd.0 v2:172.21.15.113:6800/1851994484 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f70d40536f0 con 0x55950e5767f0 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55950e5767f0 msgr2=0x55950e611ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x55950e5767f0 0x55950e611ad0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f70cc06a810 msgr2=0x7f70cc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f70cc06a810 0x7f70cc06ccd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x55950e57ae80 tx=0x7f70dc00c040 comp rx=0 tx=0).stop 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 msgr2=0x55950e57b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 --2- 172.21.15.113:0/1228186870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55950e576410 0x55950e57b590 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x55950e4a2d50 tx=0x7f70e4001250 comp rx=0 tx=0).stop 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ecccd640 1 -- 172.21.15.113:0/1228186870 reap_dead start 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 shutdown_connections 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 >> 172.21.15.113:0/1228186870 conn(0x55950e4ee0e0 msgr2=0x55950e4cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 shutdown_connections 2024-04-25T00:08:52.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:52.730+0000 7f70ee9cbf40 1 -- 172.21.15.113:0/1228186870 wait complete. 2024-04-25T00:08:52.736 INFO:teuthology.orchestra.run.smithi113.stdout:Ysixmegssubdir 2024-04-25T00:08:52.736 DEBUG:teuthology.orchestra.run.smithi113:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T00:08:52.852 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:08:52.852 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 1099511627777, 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: "ancestors": [ 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: "dirino": 1099511627776, 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: "dname": "sixmegs", 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: "version": 6 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: "dirino": 1, 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: "dname": "subdir", 2024-04-25T00:08:52.853 INFO:teuthology.orchestra.run.smithi113.stdout: "version": 8 2024-04-25T00:08:52.854 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:08:52.854 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-25T00:08:52.854 INFO:teuthology.orchestra.run.smithi113.stdout: "pool": 23, 2024-04-25T00:08:52.854 INFO:teuthology.orchestra.run.smithi113.stdout: "old_pools": [] 2024-04-25T00:08:52.854 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:08:52.854 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:52.869 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:08:52.870 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:08:52.945 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:08:52.982 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:08:52.983 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:08:52.983 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:08:52.983 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:53.000 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:08:53.000 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:53.055 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:53.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.191+0000 7f317d95a640 1 Processor -- start 2024-04-25T00:08:53.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.191+0000 7f317d95a640 1 -- start start 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.191+0000 7f317d95a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3178161770 0x7f317806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.191+0000 7f317d95a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31780713f0 0x7f31780cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f317d95a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780cca50 0x7f31781623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f317d95a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f317806d0f0 con 0x7f31780713f0 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f317d95a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3178161f50 con 0x7f3178161770 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f317d95a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31780718a0 con 0x7f31780cca50 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31780713f0 0x7f31780cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31780713f0 0x7f31780cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51438/0 (socket says 172.21.15.113:51438) 2024-04-25T00:08:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 -- 172.21.15.113:0/3348078945 learned_addr learned my addr 172.21.15.113:0/3348078945 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f3176ffd640 1 --2- 172.21.15.113:0/3348078945 >> v2:172.21.15.178:3300/0 conn(0x7f3178161770 0x7f317806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f31777fe640 1 --2- 172.21.15.113:0/3348078945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780cca50 0x7f31781623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 -- 172.21.15.113:0/3348078945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780cca50 msgr2=0x7f31781623e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 --2- 172.21.15.113:0/3348078945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780cca50 0x7f31781623e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 -- 172.21.15.113:0/3348078945 >> v2:172.21.15.178:3300/0 conn(0x7f3178161770 msgr2=0x7f317806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 --2- 172.21.15.113:0/3348078945 >> v2:172.21.15.178:3300/0 conn(0x7f3178161770 0x7f317806c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f316ffff640 1 -- 172.21.15.113:0/3348078945 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31780cfbc0 con 0x7f31780713f0 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f3176ffd640 1 --2- 172.21.15.113:0/3348078945 >> v2:172.21.15.178:3300/0 conn(0x7f3178161770 0x7f317806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.192+0000 7f31777fe640 1 --2- 172.21.15.113:0/3348078945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780cca50 0x7f31781623e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f316ffff640 1 --2- 172.21.15.113:0/3348078945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780713f0 0x7f31780cc510 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f3160009870 tx=0x7f316002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c755d0010842003f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f31767fc640 1 -- 172.21.15.113:0/3348078945 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160037070 con 0x7f31780713f0 2024-04-25T00:08:53.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f31767fc640 1 -- 172.21.15.113:0/3348078945 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3160032440 con 0x7f31780713f0 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f31767fc640 1 -- 172.21.15.113:0/3348078945 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160032810 con 0x7f31780713f0 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 -- 172.21.15.113:0/3348078945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780713f0 msgr2=0x7f31780cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 --2- 172.21.15.113:0/3348078945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780713f0 0x7f31780cc510 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f3160009870 tx=0x7f316002a180 comp rx=0 tx=0).stop 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 -- 172.21.15.113:0/3348078945 shutdown_connections 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 --2- 172.21.15.113:0/3348078945 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780cca50 0x7f31781623e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 --2- 172.21.15.113:0/3348078945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780713f0 0x7f31780cc510 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 --2- 172.21.15.113:0/3348078945 >> v2:172.21.15.178:3300/0 conn(0x7f3178161770 0x7f317806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 -- 172.21.15.113:0/3348078945 >> 172.21.15.113:0/3348078945 conn(0x7f31780d1ec0 msgr2=0x7f31780d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 -- 172.21.15.113:0/3348078945 shutdown_connections 2024-04-25T00:08:53.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.193+0000 7f317d95a640 1 -- 172.21.15.113:0/3348078945 wait complete. 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 Processor -- start 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 -- start start 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780713f0 0x7f317814f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780cca50 0x7f31780c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3178161770 0x7f31780b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f31780d0180 con 0x7f31780cca50 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3178161b50 con 0x7f3178161770 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f317d95a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31780d0300 con 0x7f31780713f0 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f316ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780cca50 0x7f31780c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f316ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780cca50 0x7f31780c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51454/0 (socket says 172.21.15.113:51454) 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.195+0000 7f316ffff640 1 -- 172.21.15.113:0/936580753 learned_addr learned my addr 172.21.15.113:0/936580753 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f31777fe640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3178161770 0x7f31780b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f3176ffd640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780713f0 0x7f317814f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f316ffff640 1 -- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780713f0 msgr2=0x7f317814f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f316ffff640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780713f0 0x7f317814f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f316ffff640 1 -- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3178161770 msgr2=0x7f31780b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f316ffff640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3178161770 0x7f31780b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f316ffff640 1 -- 172.21.15.113:0/936580753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31780b60e0 con 0x7f31780cca50 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f316ffff640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780cca50 0x7f31780c8730 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f31600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f3174ff9640 1 -- 172.21.15.113:0/936580753 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160041020 con 0x7f31780cca50 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f3174ff9640 1 -- 172.21.15.113:0/936580753 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31600306c0 con 0x7f31780cca50 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f3174ff9640 1 -- 172.21.15.113:0/936580753 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160037040 con 0x7f31780cca50 2024-04-25T00:08:53.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31780b5840 con 0x7f31780cca50 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.196+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31780b5e00 con 0x7f31780cca50 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.197+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31780c5440 con 0x7f31780cca50 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.197+0000 7f3174ff9640 1 -- 172.21.15.113:0/936580753 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3160044020 con 0x7f31780cca50 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.198+0000 7f3174ff9640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315806be30 0x7f315806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.198+0000 7f3174ff9640 1 -- 172.21.15.113:0/936580753 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f31600ab6e0 con 0x7f31780cca50 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.198+0000 7f3176ffd640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315806be30 0x7f315806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.198+0000 7f3176ffd640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315806be30 0x7f315806e2f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3168006fd0 tx=0x7f3168008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.202+0000 7f3174ff9640 1 -- 172.21.15.113:0/936580753 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31780c5440 con 0x7f31780cca50 2024-04-25T00:08:53.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.376+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31780d2d30 con 0x7f31780cca50 2024-04-25T00:08:53.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.377+0000 7f3174ff9640 1 -- 172.21.15.113:0/936580753 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16719 (secure 0 0 0) 0x7f316004b030 con 0x7f31780cca50 2024-04-25T00:08:53.377 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:53.378 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":158,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:15.696246+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:08:12.252280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:08:13.007636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315806be30 msgr2=0x7f315806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315806be30 0x7f315806e2f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3168006fd0 tx=0x7f3168008040 comp rx=0 tx=0).stop 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780cca50 msgr2=0x7f31780c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780cca50 0x7f31780c8730 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f31600046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 shutdown_connections 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f315806be30 0x7f315806e2f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3178161770 0x7f31780b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31780cca50 0x7f31780c8730 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 --2- 172.21.15.113:0/936580753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31780713f0 0x7f317814f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 >> 172.21.15.113:0/936580753 conn(0x7f31780d1ec0 msgr2=0x7f317806fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 shutdown_connections 2024-04-25T00:08:53.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.379+0000 7f317d95a640 1 -- 172.21.15.113:0/936580753 wait complete. 2024-04-25T00:08:53.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:08:12.252280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:08:13.007636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:08:53.391 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:08:53.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.489+0000 7fadb72b3640 1 Processor -- start 2024-04-25T00:08:53.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.490+0000 7fadb72b3640 1 -- start start 2024-04-25T00:08:53.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.490+0000 7fadb72b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 0x7fadb015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.490+0000 7fadb72b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fadb00b3080 0x7fadb00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.490+0000 7fadb72b3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb015f4f0 con 0x7fadb015e6b0 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.490+0000 7fadb72b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fadb00cde00 con 0x7fadb00b3080 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.490+0000 7fadb72b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadb015f7d0 con 0x7fadb015e2d0 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb5850640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fadb015e6b0 0x7fadb0071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37204/0 (socket says 172.21.15.113:37204) 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb5850640 1 -- 172.21.15.113:0/3231403750 learned_addr learned my addr 172.21.15.113:0/3231403750 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727348994 0 0) 0x7fadb015f4f0 con 0x7fadb015e6b0 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb484e640 1 --2- 172.21.15.113:0/3231403750 >> v2:172.21.15.178:3300/0 conn(0x7fadb00b3080 0x7fadb00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad9c003520 con 0x7fadb015e6b0 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb504f640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 0x7fadb015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3888727900 0 0) 0x7fad9c003520 con 0x7fadb015e6b0 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 msgr2=0x7fadb015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 0x7fadb015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 >> v2:172.21.15.178:3300/0 conn(0x7fadb00b3080 msgr2=0x7fadb00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 --2- 172.21.15.113:0/3231403750 >> v2:172.21.15.178:3300/0 conn(0x7fadb00b3080 0x7fadb00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb015fab0 con 0x7fadb015e6b0 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fada80019f0 con 0x7fadb015e6b0 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 >> v1:172.21.15.113:6789/0 conn(0x7fadb015e6b0 legacy=0x7fadb0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 0x7fad9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c003f30 0x7fad9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004d40 0x7fad9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadb00cde00 con 0x7fad9c003f30 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadb015f7d0 con 0x7fad9c004d40 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad9c00ba10 con 0x7fadb015e2d0 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb484e640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c003f30 0x7fad9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb504f640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 0x7fad9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb5850640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004d40 0x7fad9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb484e640 1 -- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 msgr2=0x7fad9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb484e640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015e2d0 0x7fad9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb484e640 1 -- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004d40 msgr2=0x7fad9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.491+0000 7fadb484e640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004d40 0x7fad9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.492+0000 7fadb484e640 1 -- 172.21.15.113:0/3231403750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c003520 con 0x7fad9c003f30 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.492+0000 7fadb484e640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c003f30 0x7fad9c004800 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fada0009ac0 tx=0x7fad9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aea33333fe6950f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.492+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0036070 con 0x7fad9c003f30 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.492+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada0031400 con 0x7fad9c003f30 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.492+0000 7fada7fff640 1 -- 172.21.15.113:0/3231403750 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0009690 con 0x7fad9c003f30 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.493+0000 7fadb72b3640 1 -- 172.21.15.113:0/3231403750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c003f30 msgr2=0x7fad9c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.493+0000 7fadb72b3640 1 --2- 172.21.15.113:0/3231403750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c003f30 0x7fad9c004800 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fada0009ac0 tx=0x7fad9c005830 comp rx=0 tx=0).stop 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.493+0000 7fadb5850640 1 -- 172.21.15.113:0/3231403750 reap_dead start 2024-04-25T00:08:53.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.493+0000 7fadb72b3640 1 -- 172.21.15.113:0/3231403750 shutdown_connections 2024-04-25T00:08:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.493+0000 7fadb72b3640 1 -- 172.21.15.113:0/3231403750 >> 172.21.15.113:0/3231403750 conn(0x7fadb006b0f0 msgr2=0x7fadb006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.493+0000 7fadb72b3640 1 -- 172.21.15.113:0/3231403750 shutdown_connections 2024-04-25T00:08:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.493+0000 7fadb72b3640 1 -- 172.21.15.113:0/3231403750 wait complete. 2024-04-25T00:08:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.494+0000 7fadb72b3640 1 Processor -- start 2024-04-25T00:08:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.494+0000 7fadb72b3640 1 -- start start 2024-04-25T00:08:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb72b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c003f30 0x7fadb0142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb72b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015cac0 0x7fadb0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb72b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb0151fe0 0x7fadb006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb72b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadb00cf420 con 0x7fadb0151fe0 2024-04-25T00:08:53.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb72b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadb00cf2a0 con 0x7fad9c003f30 2024-04-25T00:08:53.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb72b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadb00cf5a0 con 0x7fadb015cac0 2024-04-25T00:08:53.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb0151fe0 0x7fadb006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb0151fe0 0x7fadb006cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51476/0 (socket says 172.21.15.113:51476) 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 -- 172.21.15.113:0/321645717 learned_addr learned my addr 172.21.15.113:0/321645717 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 -- 172.21.15.113:0/321645717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015cac0 msgr2=0x7fadb0151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015cac0 0x7fadb0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 -- 172.21.15.113:0/321645717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c003f30 msgr2=0x7fadb0142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c003f30 0x7fadb0142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 -- 172.21.15.113:0/321645717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb0155470 con 0x7fadb0151fe0 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb5850640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb0151fe0 0x7fadb006cea0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fada80013d0 tx=0x7fada800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fada5ffb640 1 -- 172.21.15.113:0/321645717 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada800ec40 con 0x7fadb0151fe0 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fada5ffb640 1 -- 172.21.15.113:0/321645717 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada800ee00 con 0x7fadb0151fe0 2024-04-25T00:08:53.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fada5ffb640 1 -- 172.21.15.113:0/321645717 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8014670 con 0x7fadb0151fe0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.495+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb0155750 con 0x7fadb0151fe0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.496+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb0155d10 con 0x7fadb0151fe0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.497+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb0156040 con 0x7fadb0151fe0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.497+0000 7fada5ffb640 1 -- 172.21.15.113:0/321645717 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fada8014810 con 0x7fadb0151fe0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.497+0000 7fada5ffb640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9406be30 0x7fad9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.497+0000 7fada5ffb640 1 -- 172.21.15.113:0/321645717 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fada808ac00 con 0x7fadb0151fe0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.497+0000 7fadb504f640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9406be30 0x7fad9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.498+0000 7fadb504f640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9406be30 0x7fad9406e2f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fadac0099e0 tx=0x7fadac006c30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.502+0000 7fada5ffb640 1 -- 172.21.15.113:0/321645717 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fada8091080 con 0x7fadb0151fe0 2024-04-25T00:08:53.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.671+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fadb00691a0 con 0x7fadb0151fe0 2024-04-25T00:08:53.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.672+0000 7fada5ffb640 1 -- 172.21.15.113:0/321645717 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7fada8059640 con 0x7fadb0151fe0 2024-04-25T00:08:53.673 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9406be30 msgr2=0x7fad9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9406be30 0x7fad9406e2f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fadac0099e0 tx=0x7fadac006c30 comp rx=0 tx=0).stop 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb0151fe0 msgr2=0x7fadb006cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb0151fe0 0x7fadb006cea0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fada80013d0 tx=0x7fada800e530 comp rx=0 tx=0).stop 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 shutdown_connections 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad9406be30 0x7fad9406e2f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb0151fe0 0x7fadb006cea0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb015cac0 0x7fadb0151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 --2- 172.21.15.113:0/321645717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c003f30 0x7fadb0142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 >> 172.21.15.113:0/321645717 conn(0x7fadb006b0f0 msgr2=0x7fadb01587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:53.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 shutdown_connections 2024-04-25T00:08:53.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.674+0000 7fadb72b3640 1 -- 172.21.15.113:0/321645717 wait complete. 2024-04-25T00:08:53.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:08:53.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.785+0000 7fc16af90640 1 Processor -- start 2024-04-25T00:08:53.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.785+0000 7fc16af90640 1 -- start start 2024-04-25T00:08:53.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16af90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 0x7fc1640cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16af90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1641606a0 0x7fc1640cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16af90640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1640d0970 con 0x7fc1640d04f0 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16af90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc16415c6a0 con 0x7fc1641606a0 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16af90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc164160aa0 con 0x7fc16413f050 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16952d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1641606a0 0x7fc1640cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16952d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1641606a0 0x7fc1640cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51482/0 (socket says 172.21.15.113:51482) 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16952d640 1 -- 172.21.15.113:0/3548405348 learned_addr learned my addr 172.21.15.113:0/3548405348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16952d640 1 -- 172.21.15.113:0/3548405348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 msgr2=0x7fc1640cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16952d640 1 --2- 172.21.15.113:0/3548405348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 0x7fc1640cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16952d640 1 -- 172.21.15.113:0/3548405348 >> v1:172.21.15.178:6789/0 conn(0x7fc1640d04f0 legacy=0x7fc16415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.786+0000 7fc16952d640 1 -- 172.21.15.113:0/3548405348 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16415d6b0 con 0x7fc1641606a0 2024-04-25T00:08:53.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.787+0000 7fc16952d640 1 --2- 172.21.15.113:0/3548405348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1641606a0 0x7fc1640cea60 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009e50 tx=0x7fc15c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b29199921fbb2db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.787+0000 7fc1637fe640 1 -- 172.21.15.113:0/3548405348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c036070 con 0x7fc1641606a0 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.787+0000 7fc1637fe640 1 -- 172.21.15.113:0/3548405348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c031600 con 0x7fc1641606a0 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.787+0000 7fc1637fe640 1 -- 172.21.15.113:0/3548405348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0319d0 con 0x7fc1641606a0 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.787+0000 7fc16af90640 1 -- 172.21.15.113:0/3548405348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1641606a0 msgr2=0x7fc1640cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.787+0000 7fc16af90640 1 --2- 172.21.15.113:0/3548405348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1641606a0 0x7fc1640cea60 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009e50 tx=0x7fc15c02a630 comp rx=0 tx=0).stop 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.788+0000 7fc16af90640 1 -- 172.21.15.113:0/3548405348 shutdown_connections 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.788+0000 7fc16af90640 1 --2- 172.21.15.113:0/3548405348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1641606a0 0x7fc1640cea60 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.788+0000 7fc16af90640 1 --2- 172.21.15.113:0/3548405348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 0x7fc1640cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.788+0000 7fc16af90640 1 -- 172.21.15.113:0/3548405348 >> 172.21.15.113:0/3548405348 conn(0x7fc16406b0f0 msgr2=0x7fc16406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:53.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.788+0000 7fc16af90640 1 -- 172.21.15.113:0/3548405348 shutdown_connections 2024-04-25T00:08:53.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.788+0000 7fc16af90640 1 -- 172.21.15.113:0/3548405348 wait complete. 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.789+0000 7fc16af90640 1 Processor -- start 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.789+0000 7fc16af90640 1 -- start start 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640d04f0 0x7fc16413bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 0x7fc1641f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1641606a0 0x7fc164071a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415ceb0 con 0x7fc1640d04f0 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415cbd0 con 0x7fc1641606a0 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc16415d190 con 0x7fc16413f050 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640d04f0 0x7fc16413bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640d04f0 0x7fc16413bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51494/0 (socket says 172.21.15.113:51494) 2024-04-25T00:08:53.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 -- 172.21.15.113:0/2072067780 learned_addr learned my addr 172.21.15.113:0/2072067780 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16952d640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1641606a0 0x7fc164071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc163fff640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 0x7fc1641f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 -- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 msgr2=0x7fc1641f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 0x7fc1641f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 -- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1641606a0 msgr2=0x7fc164071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1641606a0 0x7fc164071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 -- 172.21.15.113:0/2072067780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc164159130 con 0x7fc1640d04f0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16952d640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1641606a0 0x7fc164071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc168d2c640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640d04f0 0x7fc16413bc70 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fc15800b4a0 tx=0x7fc15800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc1617fa640 1 -- 172.21.15.113:0/2072067780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158014070 con 0x7fc1640d04f0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc1617fa640 1 -- 172.21.15.113:0/2072067780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc158010610 con 0x7fc1640d04f0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc164159410 con 0x7fc1640d04f0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc1617fa640 1 -- 172.21.15.113:0/2072067780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158010a00 con 0x7fc1640d04f0 2024-04-25T00:08:53.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.790+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1641599d0 con 0x7fc1640d04f0 2024-04-25T00:08:53.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.791+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1640691a0 con 0x7fc1640d04f0 2024-04-25T00:08:53.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.792+0000 7fc1617fa640 1 -- 172.21.15.113:0/2072067780 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc158017070 con 0x7fc1640d04f0 2024-04-25T00:08:53.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.792+0000 7fc1617fa640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc14806be30 0x7fc14806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:53.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.792+0000 7fc1617fa640 1 -- 172.21.15.113:0/2072067780 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fc15808a980 con 0x7fc1640d04f0 2024-04-25T00:08:53.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.792+0000 7fc163fff640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc14806be30 0x7fc14806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:53.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.793+0000 7fc163fff640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc14806be30 0x7fc14806e2f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc164072a60 tx=0x7fc150009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:53.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.797+0000 7fc1617fa640 1 -- 172.21.15.113:0/2072067780 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc1580592a0 con 0x7fc1640d04f0 2024-04-25T00:08:53.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.973+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1640d0e50 con 0x7fc1640d04f0 2024-04-25T00:08:53.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.974+0000 7fc1617fa640 1 -- 172.21.15.113:0/2072067780 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7fc158058ba0 con 0x7fc1640d04f0 2024-04-25T00:08:53.975 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc14806be30 msgr2=0x7fc14806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc14806be30 0x7fc14806e2f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc164072a60 tx=0x7fc150009040 comp rx=0 tx=0).stop 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640d04f0 msgr2=0x7fc16413bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640d04f0 0x7fc16413bc70 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fc15800b4a0 tx=0x7fc15800b960 comp rx=0 tx=0).stop 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 shutdown_connections 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc14806be30 0x7fc14806e2f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1641606a0 0x7fc164071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16413f050 0x7fc1641f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 --2- 172.21.15.113:0/2072067780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640d04f0 0x7fc16413bc70 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 >> 172.21.15.113:0/2072067780 conn(0x7fc16406b0f0 msgr2=0x7fc1641557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 shutdown_connections 2024-04-25T00:08:53.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:53.976+0000 7fc16af90640 1 -- 172.21.15.113:0/2072067780 wait complete. 2024-04-25T00:08:53.987 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:08:54.006 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:08:54.033 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:08:54.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.088+0000 7fc5b0d90640 1 Processor -- start 2024-04-25T00:08:54.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.088+0000 7fc5b0d90640 1 -- start start 2024-04-25T00:08:54.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5b0d90640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc5ac147640 0x7fc5ac15e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5b0d90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 0x7fc5ac06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5b0d90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac06b420 con 0x7fc5ac06fe70 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5b0d90640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc5ac15ee60 con 0x7fc5ac147640 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5b0d90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ac15f140 con 0x7fc5ac0d2590 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc5ac06fe70 0x7fc5ac0d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37208/0 (socket says 172.21.15.113:37208) 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a3fff640 1 -- 172.21.15.113:0/1242257876 learned_addr learned my addr 172.21.15.113:0/1242257876 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5aad9d640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 0x7fc5ac06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/1242257876 >> v2:172.21.15.178:3300/0 conn(0x7fc5ac147640 0x7fc5ac15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344336641 0 0) 0x7fc5ac06b420 con 0x7fc5ac06fe70 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc598003520 con 0x7fc5ac06fe70 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2070206064 0 0) 0x7fc598003520 con 0x7fc5ac06fe70 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 msgr2=0x7fc5ac06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 0x7fc5ac06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 >> v2:172.21.15.178:3300/0 conn(0x7fc5ac147640 msgr2=0x7fc5ac15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 --2- 172.21.15.113:0/1242257876 >> v2:172.21.15.178:3300/0 conn(0x7fc5ac147640 0x7fc5ac15e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac15f420 con 0x7fc5ac06fe70 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fc594001670 con 0x7fc5ac06fe70 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 >> v1:172.21.15.113:6789/0 conn(0x7fc5ac06fe70 legacy=0x7fc5ac0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc598003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc598004800 0x7fc598004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 0x7fc5980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15ee60 con 0x7fc5980039b0 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15f140 con 0x7fc598004800 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc59800b9c0 con 0x7fc5ac0d2590 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.089+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc598003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5a3fff640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc598004800 0x7fc598004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5aad9d640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 0x7fc5980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5aa59c640 1 -- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 msgr2=0x7fc5980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac0d2590 0x7fc5980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5aa59c640 1 -- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc598004800 msgr2=0x7fc598004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc598004800 0x7fc598004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5aa59c640 1 -- 172.21.15.113:0/1242257876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc598003520 con 0x7fc5980039b0 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5a3fff640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc598004800 0x7fc598004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:54.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.090+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc598003dd0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009ac0 tx=0x7fc598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a366209a7e433e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c036070 con 0x7fc5980039b0 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c031400 con 0x7fc5980039b0 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5a9d9b640 1 -- 172.21.15.113:0/1242257876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c009690 con 0x7fc5980039b0 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5b0d90640 1 -- 172.21.15.113:0/1242257876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 msgr2=0x7fc598003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5b0d90640 1 --2- 172.21.15.113:0/1242257876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc598003dd0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009ac0 tx=0x7fc598005830 comp rx=0 tx=0).stop 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5aad9d640 1 -- 172.21.15.113:0/1242257876 reap_dead start 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5b0d90640 1 -- 172.21.15.113:0/1242257876 shutdown_connections 2024-04-25T00:08:54.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5b0d90640 1 -- 172.21.15.113:0/1242257876 >> 172.21.15.113:0/1242257876 conn(0x7fc5ac0b7370 msgr2=0x7fc5ac0b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5b0d90640 1 -- 172.21.15.113:0/1242257876 shutdown_connections 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.091+0000 7fc5b0d90640 1 -- 172.21.15.113:0/1242257876 wait complete. 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.092+0000 7fc5b0d90640 1 Processor -- start 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.092+0000 7fc5b0d90640 1 -- start start 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5b0d90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc5ac0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5b0d90640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac1434c0 0x7fc5ac15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5b0d90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5ac0cbbe0 0x7fc5ac0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5b0d90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15fbc0 con 0x7fc5980039b0 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5b0d90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac15fa40 con 0x7fc5ac0cbbe0 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5b0d90640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ac070300 con 0x7fc5ac1434c0 2024-04-25T00:08:54.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5aa59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc5ac0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5aa59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc5ac0cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51518/0 (socket says 172.21.15.113:51518) 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5aa59c640 1 -- 172.21.15.113:0/466566240 learned_addr learned my addr 172.21.15.113:0/466566240 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5aad9d640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5ac0cbbe0 0x7fc5ac0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.093+0000 7fc5a3fff640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac1434c0 0x7fc5ac15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5aa59c640 1 -- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac1434c0 msgr2=0x7fc5ac15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac1434c0 0x7fc5ac15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5aa59c640 1 -- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5ac0cbbe0 msgr2=0x7fc5ac0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5ac0cbbe0 0x7fc5ac0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5aa59c640 1 -- 172.21.15.113:0/466566240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac0cc5c0 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5aad9d640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5ac0cbbe0 0x7fc5ac0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5aa59c640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc5ac0cf450 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009ac0 tx=0x7fc59c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5a2ffd640 1 -- 172.21.15.113:0/466566240 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c041020 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5a2ffd640 1 -- 172.21.15.113:0/466566240 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c03c470 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5a2ffd640 1 -- 172.21.15.113:0/466566240 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c036040 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac141820 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.094+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac141de0 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.095+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac0691a0 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.095+0000 7fc5a2ffd640 1 -- 172.21.15.113:0/466566240 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc59c044020 con 0x7fc5980039b0 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.096+0000 7fc5a2ffd640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc58c06be30 0x7fc58c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.096+0000 7fc5a2ffd640 1 -- 172.21.15.113:0/466566240 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fc59c0ab610 con 0x7fc5980039b0 2024-04-25T00:08:54.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.096+0000 7fc5a3fff640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc58c06be30 0x7fc58c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.096+0000 7fc5a3fff640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc58c06be30 0x7fc58c06e2f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0b78c0 tx=0x7fc594009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.100+0000 7fc5a2ffd640 1 -- 172.21.15.113:0/466566240 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc59c04b030 con 0x7fc5980039b0 2024-04-25T00:08:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.273+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5ac06c8b0 con 0x7fc5980039b0 2024-04-25T00:08:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.273+0000 7fc5a2ffd640 1 -- 172.21.15.113:0/466566240 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7fc59c079fc0 con 0x7fc5980039b0 2024-04-25T00:08:54.274 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc58c06be30 msgr2=0x7fc58c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc58c06be30 0x7fc58c06e2f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0b78c0 tx=0x7fc594009040 comp rx=0 tx=0).stop 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 msgr2=0x7fc5ac0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc5ac0cf450 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009ac0 tx=0x7fc59c004730 comp rx=0 tx=0).stop 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 shutdown_connections 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc58c06be30 0x7fc58c06e2f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5ac0cbbe0 0x7fc5ac0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5ac1434c0 0x7fc5ac15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 --2- 172.21.15.113:0/466566240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5980039b0 0x7fc5ac0cf450 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 >> 172.21.15.113:0/466566240 conn(0x7fc5ac0b7370 msgr2=0x7fc5ac1445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 shutdown_connections 2024-04-25T00:08:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.275+0000 7fc5b0d90640 1 -- 172.21.15.113:0/466566240 wait complete. 2024-04-25T00:08:54.287 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:08:54.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.424+0000 7fde32bc1640 1 Processor -- start 2024-04-25T00:08:54.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.424+0000 7fde32bc1640 1 -- start start 2024-04-25T00:08:54.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.425+0000 7fde32bc1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde2c0b5ca0 0x7fde2c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.425+0000 7fde32bc1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde2c0d2e00 con 0x7fde2c0b73c0 2024-04-25T00:08:54.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.425+0000 7fde32bc1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde2c0d30e0 con 0x7fde2c15e510 2024-04-25T00:08:54.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.425+0000 7fde32bc1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fde2c0b6640 con 0x7fde2c0b5ca0 2024-04-25T00:08:54.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.425+0000 7fde2bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde2c0b5ca0 0x7fde2c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.425+0000 7fde3115e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fde2c0b73c0 0x7fde2c0b77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37210/0 (socket says 172.21.15.113:37210) 2024-04-25T00:08:54.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.425+0000 7fde3115e640 1 -- 172.21.15.113:0/3200013568 learned_addr learned my addr 172.21.15.113:0/3200013568 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:54.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2bfff640 1 -- 172.21.15.113:0/3200013568 >> v1:172.21.15.178:6789/0 conn(0x7fde2c15e510 legacy=0x7fde2c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2bfff640 1 -- 172.21.15.113:0/3200013568 >> v1:172.21.15.113:6789/0 conn(0x7fde2c0b73c0 legacy=0x7fde2c0b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2bfff640 1 -- 172.21.15.113:0/3200013568 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c0d33c0 con 0x7fde2c0b5ca0 2024-04-25T00:08:54.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2b7fe640 1 -- 172.21.15.113:0/3200013568 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1338082540 0 0) 0x7fde2c0d2e00 con 0x7fde2c0b73c0 2024-04-25T00:08:54.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2bfff640 1 --2- 172.21.15.113:0/3200013568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b5ca0 0x7fde2c0b6100 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fde180097a0 tx=0x7fde1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6334ff02cd9792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2b7fe640 1 -- 172.21.15.113:0/3200013568 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18037070 con 0x7fde2c0b5ca0 2024-04-25T00:08:54.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2b7fe640 1 -- 172.21.15.113:0/3200013568 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18032440 con 0x7fde2c0b5ca0 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.426+0000 7fde2b7fe640 1 -- 172.21.15.113:0/3200013568 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18032810 con 0x7fde2c0b5ca0 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.427+0000 7fde32bc1640 1 -- 172.21.15.113:0/3200013568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b5ca0 msgr2=0x7fde2c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.427+0000 7fde32bc1640 1 --2- 172.21.15.113:0/3200013568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b5ca0 0x7fde2c0b6100 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fde180097a0 tx=0x7fde1802a1a0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.427+0000 7fde32bc1640 1 -- 172.21.15.113:0/3200013568 shutdown_connections 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.427+0000 7fde32bc1640 1 --2- 172.21.15.113:0/3200013568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b5ca0 0x7fde2c0b6100 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.427+0000 7fde32bc1640 1 -- 172.21.15.113:0/3200013568 >> 172.21.15.113:0/3200013568 conn(0x7fde2c06b720 msgr2=0x7fde2c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.427+0000 7fde32bc1640 1 -- 172.21.15.113:0/3200013568 shutdown_connections 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.427+0000 7fde32bc1640 1 -- 172.21.15.113:0/3200013568 wait complete. 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.428+0000 7fde32bc1640 1 Processor -- start 2024-04-25T00:08:54.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.428+0000 7fde32bc1640 1 -- start start 2024-04-25T00:08:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde32bc1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde2c0b5ca0 0x7fde2c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde32bc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b73c0 0x7fde2c160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde32bc1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde2c15e510 0x7fde2c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde32bc1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c0d3ea0 con 0x7fde2c0b73c0 2024-04-25T00:08:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde32bc1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c0d3980 con 0x7fde2c0b5ca0 2024-04-25T00:08:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde32bc1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde2c0d2a90 con 0x7fde2c15e510 2024-04-25T00:08:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b73c0 0x7fde2c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b73c0 0x7fde2c160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51538/0 (socket says 172.21.15.113:51538) 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 -- 172.21.15.113:0/2239140002 learned_addr learned my addr 172.21.15.113:0/2239140002 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 -- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde2c15e510 msgr2=0x7fde2c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde3115e640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde2c15e510 0x7fde2c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde3095d640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde2c0b5ca0 0x7fde2c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde2c15e510 0x7fde2c0c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 -- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde2c0b5ca0 msgr2=0x7fde2c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde2c0b5ca0 0x7fde2c0cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.429+0000 7fde2bfff640 1 -- 172.21.15.113:0/2239140002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c0b9460 con 0x7fde2c0b73c0 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.430+0000 7fde2bfff640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b73c0 0x7fde2c160270 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fde18000c00 tx=0x7fde180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.430+0000 7fde297fa640 1 -- 172.21.15.113:0/2239140002 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18037040 con 0x7fde2c0b73c0 2024-04-25T00:08:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.430+0000 7fde297fa640 1 -- 172.21.15.113:0/2239140002 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18042070 con 0x7fde2c0b73c0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.430+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2c0cce80 con 0x7fde2c0b73c0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.430+0000 7fde297fa640 1 -- 172.21.15.113:0/2239140002 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18030a70 con 0x7fde2c0b73c0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.430+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2c0cd440 con 0x7fde2c0b73c0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.431+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2c0691a0 con 0x7fde2c0b73c0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.431+0000 7fde297fa640 1 -- 172.21.15.113:0/2239140002 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde18030c10 con 0x7fde2c0b73c0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.431+0000 7fde297fa640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde1006be30 0x7fde1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.432+0000 7fde297fa640 1 -- 172.21.15.113:0/2239140002 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fde180ab930 con 0x7fde2c0b73c0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.432+0000 7fde3095d640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde1006be30 0x7fde1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.432+0000 7fde3095d640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde1006be30 0x7fde1006e2f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fde20000f60 tx=0x7fde20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.436+0000 7fde297fa640 1 -- 172.21.15.113:0/2239140002 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fde18045360 con 0x7fde2c0b73c0 2024-04-25T00:08:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.621+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde2c0cfbc0 con 0x7fde2c0b73c0 2024-04-25T00:08:54.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.621+0000 7fde297fa640 1 -- 172.21.15.113:0/2239140002 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) ==== 80+0+0 (secure 0 0 0) 0x7fde1807a2e0 con 0x7fde2c0b73c0 2024-04-25T00:08:54.622 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde1006be30 msgr2=0x7fde1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde1006be30 0x7fde1006e2f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fde20000f60 tx=0x7fde20009040 comp rx=0 tx=0).stop 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b73c0 msgr2=0x7fde2c160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b73c0 0x7fde2c160270 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fde18000c00 tx=0x7fde180046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 shutdown_connections 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fde1006be30 0x7fde1006e2f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde2c15e510 0x7fde2c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0b73c0 0x7fde2c160270 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 --2- 172.21.15.113:0/2239140002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde2c0b5ca0 0x7fde2c0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 >> 172.21.15.113:0/2239140002 conn(0x7fde2c06b720 msgr2=0x7fde2c0b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 shutdown_connections 2024-04-25T00:08:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.623+0000 7fde32bc1640 1 -- 172.21.15.113:0/2239140002 wait complete. 2024-04-25T00:08:54.634 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:08:54.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:54.635 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:08:54.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:08:54.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:08:54.683 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:08:54.707 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:08:54.707 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:08:54.707 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:54.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:08:54.724 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:08:54.725 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:08:54.725 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:54.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:08:54.763 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:08:54.763 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:08:54.763 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:08:54.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:08:54.780 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:54.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.899+0000 7fe9a685d640 1 Processor -- start 2024-04-25T00:08:54.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.899+0000 7fe9a685d640 1 -- start start 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.900+0000 7fe9a685d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a00d2290 0x7fe9a00b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.900+0000 7fe9a685d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a00d2800 con 0x7fe9a00d0b00 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.900+0000 7fe9a685d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a00d2ae0 con 0x7fe9a015e320 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.900+0000 7fe9a685d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a00d3410 con 0x7fe9a00d2290 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.900+0000 7fe9a4dfa640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe9a00d0b00 0x7fe9a00b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37218/0 (socket says 172.21.15.113:37218) 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.900+0000 7fe9a4dfa640 1 -- 172.21.15.113:0/3430178860 learned_addr learned my addr 172.21.15.113:0/3430178860 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.900+0000 7fe99f7fe640 1 --2- 172.21.15.113:0/3430178860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a00d2290 0x7fe9a00b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435320308 0 0) 0x7fe9a00d2800 con 0x7fe9a00d0b00 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe988003520 con 0x7fe9a00d0b00 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757853045 0 0) 0x7fe9a00d2ae0 con 0x7fe9a015e320 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9a00d2800 con 0x7fe9a015e320 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1303972068 0 0) 0x7fe988003520 con 0x7fe9a00d0b00 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99f7fe640 1 -- 172.21.15.113:0/3430178860 >> v1:172.21.15.178:6789/0 conn(0x7fe9a015e320 legacy=0x7fe9a00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99f7fe640 1 -- 172.21.15.113:0/3430178860 >> v1:172.21.15.113:6789/0 conn(0x7fe9a00d0b00 legacy=0x7fe9a00b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.901+0000 7fe99f7fe640 1 -- 172.21.15.113:0/3430178860 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a00d2dc0 con 0x7fe9a00d2290 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe99f7fe640 1 --2- 172.21.15.113:0/3430178860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a00d2290 0x7fe9a00b9aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9940097d0 tx=0x7fe99402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd0d93f698a6f983 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994037070 con 0x7fe9a00d2290 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99402ac30 con 0x7fe9a00d2290 2024-04-25T00:08:54.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe99effd640 1 -- 172.21.15.113:0/3430178860 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980019f0 con 0x7fe9a00d2290 2024-04-25T00:08:54.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe9a685d640 1 -- 172.21.15.113:0/3430178860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a00d2290 msgr2=0x7fe9a00b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe9a685d640 1 --2- 172.21.15.113:0/3430178860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a00d2290 0x7fe9a00b9aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9940097d0 tx=0x7fe99402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe9a685d640 1 -- 172.21.15.113:0/3430178860 shutdown_connections 2024-04-25T00:08:54.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe9a685d640 1 --2- 172.21.15.113:0/3430178860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a00d2290 0x7fe9a00b9aa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe9a685d640 1 -- 172.21.15.113:0/3430178860 >> 172.21.15.113:0/3430178860 conn(0x7fe9a00b5c20 msgr2=0x7fe9a00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:54.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe9a685d640 1 -- 172.21.15.113:0/3430178860 shutdown_connections 2024-04-25T00:08:54.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.902+0000 7fe9a685d640 1 -- 172.21.15.113:0/3430178860 wait complete. 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.903+0000 7fe9a685d640 1 Processor -- start 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe9a685d640 1 -- start start 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe9a685d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9a00d0b00 0x7fe9a015f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe9a685d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00d2290 0x7fe9a0153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe9a685d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a015e320 0x7fe9a00b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe9a685d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a015e700 con 0x7fe9a00d2290 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe9a685d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00b9fe0 con 0x7fe9a00d0b00 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe9a685d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a015e880 con 0x7fe9a015e320 2024-04-25T00:08:54.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.904+0000 7fe99f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00d2290 0x7fe9a0153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00d2290 0x7fe9a0153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51548/0 (socket says 172.21.15.113:51548) 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 -- 172.21.15.113:0/2926616358 learned_addr learned my addr 172.21.15.113:0/2926616358 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe9a4dfa640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a015e320 0x7fe9a00b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99ffff640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9a00d0b00 0x7fe9a015f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 -- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a015e320 msgr2=0x7fe9a00b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a015e320 0x7fe9a00b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 -- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9a00d0b00 msgr2=0x7fe9a015f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9a00d0b00 0x7fe9a015f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 -- 172.21.15.113:0/2926616358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a015af30 con 0x7fe9a00d2290 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99f7fe640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00d2290 0x7fe9a0153ef0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fe9940097d0 tx=0x7fe9940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99cff9640 1 -- 172.21.15.113:0/2926616358 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994037050 con 0x7fe9a00d2290 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99cff9640 1 -- 172.21.15.113:0/2926616358 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe994038040 con 0x7fe9a00d2290 2024-04-25T00:08:54.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a01f3f50 con 0x7fe9a00d2290 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe99cff9640 1 -- 172.21.15.113:0/2926616358 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994006e10 con 0x7fe9a00d2290 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.905+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a01f4510 con 0x7fe9a00d2290 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.906+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a00691a0 con 0x7fe9a00d2290 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.907+0000 7fe99cff9640 1 -- 172.21.15.113:0/2926616358 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe994045020 con 0x7fe9a00d2290 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.907+0000 7fe99cff9640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe98c06be60 0x7fe98c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.907+0000 7fe99ffff640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe98c06be60 0x7fe98c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.907+0000 7fe99cff9640 1 -- 172.21.15.113:0/2926616358 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fe9940aac80 con 0x7fe9a00d2290 2024-04-25T00:08:54.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.908+0000 7fe99ffff640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe98c06be60 0x7fe98c06e320 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe9900011e0 tx=0x7fe990009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:54.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:54.912+0000 7fe99cff9640 1 -- 172.21.15.113:0/2926616358 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe99404c020 con 0x7fe9a00d2290 2024-04-25T00:08:55.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.123+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9a00d47b0 con 0x7fe9a00d2290 2024-04-25T00:08:55.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.124+0000 7fe99cff9640 1 -- 172.21.15.113:0/2926616358 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7fe9a015af30 con 0x7fe9a00d2290 2024-04-25T00:08:55.124 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:55.124 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:55.125 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe98c06be60 msgr2=0x7fe98c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe98c06be60 0x7fe98c06e320 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe9900011e0 tx=0x7fe990009040 comp rx=0 tx=0).stop 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00d2290 msgr2=0x7fe9a0153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00d2290 0x7fe9a0153ef0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fe9940097d0 tx=0x7fe9940046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 shutdown_connections 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe98c06be60 0x7fe98c06e320 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9a015e320 0x7fe9a00b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a00d2290 0x7fe9a0153ef0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 --2- 172.21.15.113:0/2926616358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9a00d0b00 0x7fe9a015f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 >> 172.21.15.113:0/2926616358 conn(0x7fe9a00b5c20 msgr2=0x7fe9a00b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:55.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 shutdown_connections 2024-04-25T00:08:55.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.126+0000 7fe9a685d640 1 -- 172.21.15.113:0/2926616358 wait complete. 2024-04-25T00:08:55.138 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:55.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.234+0000 7fd321770640 1 Processor -- start 2024-04-25T00:08:55.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.234+0000 7fd321770640 1 -- start start 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.234+0000 7fd321770640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 0x7fd31c0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd321770640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c0c46d0 con 0x7fd31c15e310 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd321770640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c06b190 con 0x7fd31c0ce9a0 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd321770640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c15e6f0 con 0x7fd31c0ce370 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31affd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd31c15e310 0x7fd31c06bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37222/0 (socket says 172.21.15.113:37222) 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31affd640 1 -- 172.21.15.113:0/2129619280 learned_addr learned my addr 172.21.15.113:0/2129619280 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2218735897 0 0) 0x7fd31c0c46d0 con 0x7fd31c15e310 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd308003520 con 0x7fd31c15e310 2024-04-25T00:08:55.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd313fff640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 0x7fd31c0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1654570011 0 0) 0x7fd308003520 con 0x7fd31c15e310 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 msgr2=0x7fd31c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 0x7fd31c0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 >> v1:172.21.15.178:6789/0 conn(0x7fd31c0ce9a0 legacy=0x7fd31c061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c06b470 con 0x7fd31c15e310 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fd30c001670 con 0x7fd31c15e310 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 >> v1:172.21.15.113:6789/0 conn(0x7fd31c15e310 legacy=0x7fd31c06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3080039e0 0x7fd308003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd308004800 0x7fd308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 0x7fd308005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd30800bac0 con 0x7fd308004800 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd30800bc70 con 0x7fd3080039e0 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd30800be70 con 0x7fd31c0ce370 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd313fff640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd308004800 0x7fd308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31affd640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3080039e0 0x7fd308003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.235+0000 7fd31b7fe640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 0x7fd308005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd313fff640 1 -- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 msgr2=0x7fd308005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd313fff640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0ce370 0x7fd308005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd313fff640 1 -- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3080039e0 msgr2=0x7fd308003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd313fff640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3080039e0 0x7fd308003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd313fff640 1 -- 172.21.15.113:0/2129619280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd308003520 con 0x7fd308004800 2024-04-25T00:08:55.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd31affd640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3080039e0 0x7fd308003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd313fff640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd308004800 0x7fd308004c80 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fd30400ba30 tx=0x7fd308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a53c8eb7c16cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304030070 con 0x7fd308004800 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30402ade0 con 0x7fd308004800 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.236+0000 7fd31a7fc640 1 -- 172.21.15.113:0/2129619280 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30400b600 con 0x7fd308004800 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.237+0000 7fd321770640 1 -- 172.21.15.113:0/2129619280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd308004800 msgr2=0x7fd308004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.237+0000 7fd321770640 1 --2- 172.21.15.113:0/2129619280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd308004800 0x7fd308004c80 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fd30400ba30 tx=0x7fd308005830 comp rx=0 tx=0).stop 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.237+0000 7fd31b7fe640 1 -- 172.21.15.113:0/2129619280 reap_dead start 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.237+0000 7fd321770640 1 -- 172.21.15.113:0/2129619280 shutdown_connections 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.237+0000 7fd321770640 1 -- 172.21.15.113:0/2129619280 >> 172.21.15.113:0/2129619280 conn(0x7fd31c0cd850 msgr2=0x7fd31c0cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.237+0000 7fd321770640 1 -- 172.21.15.113:0/2129619280 shutdown_connections 2024-04-25T00:08:55.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.237+0000 7fd321770640 1 -- 172.21.15.113:0/2129619280 wait complete. 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.238+0000 7fd321770640 1 Processor -- start 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.238+0000 7fd321770640 1 -- start start 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd321770640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd308004800 0x7fd31c0c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd321770640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31c15d560 0x7fd31c0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd321770640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0b1f90 0x7fd31c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd321770640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0cd600 con 0x7fd31c15d560 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd321770640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c06bac0 con 0x7fd308004800 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd321770640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c06c660 con 0x7fd31c0b1f90 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31c15d560 0x7fd31c0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31c15d560 0x7fd31c0b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51570/0 (socket says 172.21.15.113:51570) 2024-04-25T00:08:55.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 -- 172.21.15.113:0/4188320506 learned_addr learned my addr 172.21.15.113:0/4188320506 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:55.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 -- 172.21.15.113:0/4188320506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0b1f90 msgr2=0x7fd31c06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0b1f90 0x7fd31c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 -- 172.21.15.113:0/4188320506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd308004800 msgr2=0x7fd31c0c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd308004800 0x7fd31c0c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 -- 172.21.15.113:0/4188320506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c15f4e0 con 0x7fd31c15d560 2024-04-25T00:08:55.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd313fff640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31c15d560 0x7fd31c0b1a50 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fd30402a8d0 tx=0x7fd3040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.239+0000 7fd3137fe640 1 -- 172.21.15.113:0/4188320506 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304043020 con 0x7fd31c15d560 2024-04-25T00:08:55.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.240+0000 7fd3137fe640 1 -- 172.21.15.113:0/4188320506 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30403f680 con 0x7fd31c15d560 2024-04-25T00:08:55.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.240+0000 7fd3137fe640 1 -- 172.21.15.113:0/4188320506 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30400b600 con 0x7fd31c15d560 2024-04-25T00:08:55.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.240+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c15f7c0 con 0x7fd31c15d560 2024-04-25T00:08:55.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.240+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c15fd80 con 0x7fd31c15d560 2024-04-25T00:08:55.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.240+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c0b23b0 con 0x7fd31c15d560 2024-04-25T00:08:55.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.241+0000 7fd3137fe640 1 -- 172.21.15.113:0/4188320506 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd304046020 con 0x7fd31c15d560 2024-04-25T00:08:55.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.241+0000 7fd3137fe640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd2fc06be60 0x7fd2fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.241+0000 7fd3137fe640 1 -- 172.21.15.113:0/4188320506 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fd3040abd30 con 0x7fd31c15d560 2024-04-25T00:08:55.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.242+0000 7fd31affd640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd2fc06be60 0x7fd2fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.242+0000 7fd31affd640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd2fc06be60 0x7fd2fc06e320 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd30c001c10 tx=0x7fd30c001b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.246+0000 7fd3137fe640 1 -- 172.21.15.113:0/4188320506 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd30404d020 con 0x7fd31c15d560 2024-04-25T00:08:55.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.419+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd31c0628f0 con 0x7fd31c15d560 2024-04-25T00:08:55.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.420+0000 7fd3137fe640 1 -- 172.21.15.113:0/4188320506 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16719 (secure 0 0 0) 0x7fd30407a6e0 con 0x7fd31c15d560 2024-04-25T00:08:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:55.421 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":158,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:15.696246+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:08:12.252280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:08:13.007636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd2fc06be60 msgr2=0x7fd2fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd2fc06be60 0x7fd2fc06e320 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd30c001c10 tx=0x7fd30c001b80 comp rx=0 tx=0).stop 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31c15d560 msgr2=0x7fd31c0b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31c15d560 0x7fd31c0b1a50 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fd30402a8d0 tx=0x7fd3040046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 shutdown_connections 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd2fc06be60 0x7fd2fc06e320 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd31c0b1f90 0x7fd31c06ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31c15d560 0x7fd31c0b1a50 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 --2- 172.21.15.113:0/4188320506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd308004800 0x7fd31c0c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 >> 172.21.15.113:0/4188320506 conn(0x7fd31c0cd850 msgr2=0x7fd31c158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:55.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 shutdown_connections 2024-04-25T00:08:55.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.422+0000 7fd321770640 1 -- 172.21.15.113:0/4188320506 wait complete. 2024-04-25T00:08:55.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:08:12.252280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:08:13.007636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:08:55.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.533+0000 7f17c19dc640 1 Processor -- start 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.533+0000 7f17c19dc640 1 -- start start 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17c19dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17bc15e510 0x7f17bc0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17c19dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17c19dc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17bc0d2e00 con 0x7f17bc0d28c0 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17c19dc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f17bc0d30e0 con 0x7f17bc15e510 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17c19dc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17bc0d33c0 con 0x7f17bc0b63b0 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17baffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17bc15e510 0x7f17bc0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17baffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17bc15e510 0x7f17bc0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51576/0 (socket says 172.21.15.113:51576) 2024-04-25T00:08:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17baffd640 1 -- 172.21.15.113:0/2832938629 learned_addr learned my addr 172.21.15.113:0/2832938629 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17baffd640 1 -- 172.21.15.113:0/2832938629 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 msgr2=0x7f17bc0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17baffd640 1 --2- 172.21.15.113:0/2832938629 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17baffd640 1 -- 172.21.15.113:0/2832938629 >> v1:172.21.15.178:6789/0 conn(0x7f17bc0d28c0 legacy=0x7f17bc0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.534+0000 7f17baffd640 1 -- 172.21.15.113:0/2832938629 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc0d36a0 con 0x7f17bc15e510 2024-04-25T00:08:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.535+0000 7f17baffd640 1 --2- 172.21.15.113:0/2832938629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0d2380 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009e50 tx=0x7f17ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ffc1c2e8179131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.535+0000 7f17b9ffb640 1 -- 172.21.15.113:0/2832938629 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac036070 con 0x7f17bc15e510 2024-04-25T00:08:55.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.535+0000 7f17b9ffb640 1 -- 172.21.15.113:0/2832938629 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ac031600 con 0x7f17bc15e510 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.535+0000 7f17b9ffb640 1 -- 172.21.15.113:0/2832938629 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0319d0 con 0x7f17bc15e510 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 -- 172.21.15.113:0/2832938629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 msgr2=0x7f17bc0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 --2- 172.21.15.113:0/2832938629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0d2380 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009e50 tx=0x7f17ac02a660 comp rx=0 tx=0).stop 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 -- 172.21.15.113:0/2832938629 shutdown_connections 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 --2- 172.21.15.113:0/2832938629 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc0b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 --2- 172.21.15.113:0/2832938629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0d2380 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 -- 172.21.15.113:0/2832938629 >> 172.21.15.113:0/2832938629 conn(0x7f17bc06b720 msgr2=0x7f17bc0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 -- 172.21.15.113:0/2832938629 shutdown_connections 2024-04-25T00:08:55.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.536+0000 7f17c19dc640 1 -- 172.21.15.113:0/2832938629 wait complete. 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.537+0000 7f17c19dc640 1 Processor -- start 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.537+0000 7f17c19dc640 1 -- start start 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17c19dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc1f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17c19dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17bc0d28c0 0x7f17bc0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17c19dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17c19dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc0b9dd0 con 0x7f17bc15e510 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17c19dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc0d3ea0 con 0x7f17bc0d28c0 2024-04-25T00:08:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17c19dc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17bc0b7900 con 0x7f17bc0b63b0 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51592/0 (socket says 172.21.15.113:51592) 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 -- 172.21.15.113:0/456645565 learned_addr learned my addr 172.21.15.113:0/456645565 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17baffd640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc1f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17ba7fc640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17bc0d28c0 0x7f17bc0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 -- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 msgr2=0x7f17bc1f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc1f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 -- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17bc0d28c0 msgr2=0x7f17bc0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17bc0d28c0 0x7f17bc0b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17bb7fe640 1 -- 172.21.15.113:0/456645565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc0b4ec0 con 0x7f17bc15e510 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.538+0000 7f17baffd640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc1f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.539+0000 7f17bb7fe640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0cb600 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b3f0 tx=0x7f17b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.539+0000 7f17a3fff640 1 -- 172.21.15.113:0/456645565 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4015020 con 0x7f17bc15e510 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.539+0000 7f17a3fff640 1 -- 172.21.15.113:0/456645565 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b4011850 con 0x7f17bc15e510 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.539+0000 7f17a3fff640 1 -- 172.21.15.113:0/456645565 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4011c40 con 0x7f17bc15e510 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.539+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc0b51a0 con 0x7f17bc15e510 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.539+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc0c88a0 con 0x7f17bc15e510 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.540+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17bc0c8aa0 con 0x7f17bc15e510 2024-04-25T00:08:55.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.540+0000 7f17a3fff640 1 -- 172.21.15.113:0/456645565 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17b4018020 con 0x7f17bc15e510 2024-04-25T00:08:55.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.541+0000 7f17a3fff640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17a806be30 0x7f17a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.541+0000 7f17baffd640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17a806be30 0x7f17a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.541+0000 7f17a3fff640 1 -- 172.21.15.113:0/456645565 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f17b408add0 con 0x7f17bc15e510 2024-04-25T00:08:55.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.541+0000 7f17baffd640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17a806be30 0x7f17a806e2f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f17ac033000 tx=0x7f17ac009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.545+0000 7f17a3fff640 1 -- 172.21.15.113:0/456645565 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f17bc0c8aa0 con 0x7f17bc15e510 2024-04-25T00:08:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.717+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17bc0c8690 con 0x7f17bc15e510 2024-04-25T00:08:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.718+0000 7f17a3fff640 1 -- 172.21.15.113:0/456645565 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7f17b408c020 con 0x7f17bc15e510 2024-04-25T00:08:55.718 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:08:55.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17a806be30 msgr2=0x7f17a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17a806be30 0x7f17a806e2f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f17ac033000 tx=0x7f17ac009000 comp rx=0 tx=0).stop 2024-04-25T00:08:55.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 msgr2=0x7f17bc0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0cb600 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b3f0 tx=0x7f17b400b8b0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 shutdown_connections 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f17a806be30 0x7f17a806e2f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17bc0d28c0 0x7f17bc0b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17bc15e510 0x7f17bc0cb600 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 --2- 172.21.15.113:0/456645565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17bc0b63b0 0x7f17bc1f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 >> 172.21.15.113:0/456645565 conn(0x7f17bc06b720 msgr2=0x7f17bc071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 shutdown_connections 2024-04-25T00:08:55.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.720+0000 7f17c19dc640 1 -- 172.21.15.113:0/456645565 wait complete. 2024-04-25T00:08:55.731 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:08:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.830+0000 7fb70df8b640 1 Processor -- start 2024-04-25T00:08:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.830+0000 7fb70df8b640 1 -- start start 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb70df8b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb70815e2d0 0x7fb70815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb70df8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7080b3080 0x7fb7080b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb70df8b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb70815e6b0 0x7fb7080b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb70df8b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb7080cde00 con 0x7fb7080b3080 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb70df8b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb70815f4f0 con 0x7fb70815e6b0 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb70df8b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb70815f7d0 con 0x7fb70815e2d0 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7080b3080 0x7fb7080b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7080b3080 0x7fb7080b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51608/0 (socket says 172.21.15.113:51608) 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 -- 172.21.15.113:0/1355042768 learned_addr learned my addr 172.21.15.113:0/1355042768 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb707fff640 1 --2- 172.21.15.113:0/1355042768 >> v2:172.21.15.178:3300/0 conn(0x7fb70815e6b0 0x7fb7080b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 -- 172.21.15.113:0/1355042768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb70815e2d0 msgr2=0x7fb70815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb7077fe640 1 --2- 172.21.15.113:0/1355042768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb70815e2d0 0x7fb70815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 --2- 172.21.15.113:0/1355042768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb70815e2d0 0x7fb70815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 -- 172.21.15.113:0/1355042768 >> v2:172.21.15.178:3300/0 conn(0x7fb70815e6b0 msgr2=0x7fb7080b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 --2- 172.21.15.113:0/1355042768 >> v2:172.21.15.178:3300/0 conn(0x7fb70815e6b0 0x7fb7080b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb706ffd640 1 -- 172.21.15.113:0/1355042768 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70815fab0 con 0x7fb7080b3080 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb707fff640 1 --2- 172.21.15.113:0/1355042768 >> v2:172.21.15.178:3300/0 conn(0x7fb70815e6b0 0x7fb7080b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.831+0000 7fb7077fe640 1 --2- 172.21.15.113:0/1355042768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb70815e2d0 0x7fb70815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.832+0000 7fb706ffd640 1 --2- 172.21.15.113:0/1355042768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7080b3080 0x7fb7080b4330 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fb70806f500 tx=0x7fb6fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2106062320e2544a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.832+0000 7fb7067fc640 1 -- 172.21.15.113:0/1355042768 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc037070 con 0x7fb7080b3080 2024-04-25T00:08:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.832+0000 7fb7067fc640 1 -- 172.21.15.113:0/1355042768 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6fc032440 con 0x7fb7080b3080 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.832+0000 7fb7067fc640 1 -- 172.21.15.113:0/1355042768 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc032810 con 0x7fb7080b3080 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 -- 172.21.15.113:0/1355042768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7080b3080 msgr2=0x7fb7080b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 --2- 172.21.15.113:0/1355042768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7080b3080 0x7fb7080b4330 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fb70806f500 tx=0x7fb6fc02a140 comp rx=0 tx=0).stop 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 -- 172.21.15.113:0/1355042768 shutdown_connections 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 --2- 172.21.15.113:0/1355042768 >> v2:172.21.15.178:3300/0 conn(0x7fb70815e6b0 0x7fb7080b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 --2- 172.21.15.113:0/1355042768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7080b3080 0x7fb7080b4330 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 --2- 172.21.15.113:0/1355042768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb70815e2d0 0x7fb70815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 -- 172.21.15.113:0/1355042768 >> 172.21.15.113:0/1355042768 conn(0x7fb70806b0f0 msgr2=0x7fb70806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 -- 172.21.15.113:0/1355042768 shutdown_connections 2024-04-25T00:08:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.833+0000 7fb70df8b640 1 -- 172.21.15.113:0/1355042768 wait complete. 2024-04-25T00:08:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.834+0000 7fb70df8b640 1 Processor -- start 2024-04-25T00:08:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.834+0000 7fb70df8b640 1 -- start start 2024-04-25T00:08:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb70df8b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7080b3080 0x7fb7081f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb70df8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb70815e2d0 0x7fb708156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb70df8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb70815e6b0 0x7fb7080aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb70df8b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb70815f2b0 con 0x7fb70815e6b0 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb70df8b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb70815f130 con 0x7fb70815e2d0 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb70df8b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb708071040 con 0x7fb7080b3080 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb70815e6b0 0x7fb7080aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb70815e6b0 0x7fb7080aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51614/0 (socket says 172.21.15.113:51614) 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 -- 172.21.15.113:0/4139769405 learned_addr learned my addr 172.21.15.113:0/4139769405 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb7077fe640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7080b3080 0x7fb7081f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb706ffd640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb70815e2d0 0x7fb708156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 -- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7080b3080 msgr2=0x7fb7081f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7080b3080 0x7fb7081f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 -- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb70815e2d0 msgr2=0x7fb708156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb70815e2d0 0x7fb708156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 -- 172.21.15.113:0/4139769405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb708153c40 con 0x7fb70815e6b0 2024-04-25T00:08:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.835+0000 7fb707fff640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb70815e6b0 0x7fb7080aef80 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fb70000cd50 tx=0x7fb70000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.836+0000 7fb6effff640 1 -- 172.21.15.113:0/4139769405 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000f680 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.836+0000 7fb6effff640 1 -- 172.21.15.113:0/4139769405 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70000f840 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.836+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081f2350 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.836+0000 7fb6effff640 1 -- 172.21.15.113:0/4139769405 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000fc30 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.836+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081f2910 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.837+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7080691a0 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.837+0000 7fb6effff640 1 -- 172.21.15.113:0/4139769405 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7000213f0 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.837+0000 7fb6effff640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb6f406be30 0x7fb6f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.838+0000 7fb6effff640 1 -- 172.21.15.113:0/4139769405 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fb70008bb70 con 0x7fb70815e6b0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.838+0000 7fb7077fe640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb6f406be30 0x7fb6f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.838+0000 7fb7077fe640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb6f406be30 0x7fb6f406e2f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb7080d0ec0 tx=0x7fb6f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:55.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:55.842+0000 7fb6effff640 1 -- 172.21.15.113:0/4139769405 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb700014030 con 0x7fb70815e6b0 2024-04-25T00:08:56.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.014+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb708072050 con 0x7fb70815e6b0 2024-04-25T00:08:56.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.015+0000 7fb6effff640 1 -- 172.21.15.113:0/4139769405 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7fb70005a5b0 con 0x7fb70815e6b0 2024-04-25T00:08:56.015 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:08:56.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb6f406be30 msgr2=0x7fb6f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb6f406be30 0x7fb6f406e2f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb7080d0ec0 tx=0x7fb6f8008040 comp rx=0 tx=0).stop 2024-04-25T00:08:56.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb70815e6b0 msgr2=0x7fb7080aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb70815e6b0 0x7fb7080aef80 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fb70000cd50 tx=0x7fb70000a650 comp rx=0 tx=0).stop 2024-04-25T00:08:56.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 shutdown_connections 2024-04-25T00:08:56.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb6f406be30 0x7fb6f406e2f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb70815e2d0 0x7fb708156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb70815e6b0 0x7fb7080aef80 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 --2- 172.21.15.113:0/4139769405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7080b3080 0x7fb7081f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 >> 172.21.15.113:0/4139769405 conn(0x7fb70806b0f0 msgr2=0x7fb7080d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 shutdown_connections 2024-04-25T00:08:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.017+0000 7fb70df8b640 1 -- 172.21.15.113:0/4139769405 wait complete. 2024-04-25T00:08:56.029 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:08:56.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.127+0000 7f2b5b9dc640 1 Processor -- start 2024-04-25T00:08:56.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.128+0000 7f2b5b9dc640 1 -- start start 2024-04-25T00:08:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.128+0000 7f2b5b9dc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2b540b3080 0x7f2b540b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.128+0000 7f2b5b9dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b5415e6b0 0x7f2b540b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.128+0000 7f2b5b9dc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b5415f4f0 con 0x7f2b5415e2d0 2024-04-25T00:08:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.128+0000 7f2b5b9dc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2b540cde00 con 0x7f2b5415e6b0 2024-04-25T00:08:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.128+0000 7f2b5b9dc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2b5415f7d0 con 0x7f2b540b3080 2024-04-25T00:08:56.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b59f79640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b5415e6b0 0x7f2b540b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b59f79640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b5415e6b0 0x7f2b540b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51626/0 (socket says 172.21.15.113:51626) 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b59f79640 1 -- 172.21.15.113:0/1193236398 learned_addr learned my addr 172.21.15.113:0/1193236398 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b58f77640 1 --2- 172.21.15.113:0/1193236398 >> v2:172.21.15.178:3300/0 conn(0x7f2b540b3080 0x7f2b540b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b59f79640 1 -- 172.21.15.113:0/1193236398 >> v2:172.21.15.178:3300/0 conn(0x7f2b540b3080 msgr2=0x7f2b540b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b59f79640 1 --2- 172.21.15.113:0/1193236398 >> v2:172.21.15.178:3300/0 conn(0x7f2b540b3080 0x7f2b540b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b59f79640 1 -- 172.21.15.113:0/1193236398 >> v1:172.21.15.178:6789/0 conn(0x7f2b5415e2d0 legacy=0x7f2b5415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.129+0000 7f2b59f79640 1 -- 172.21.15.113:0/1193236398 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5415fab0 con 0x7f2b5415e6b0 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b59f79640 1 --2- 172.21.15.113:0/1193236398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e6b0 0x7f2b540b4870 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f2b54070ff0 tx=0x7f2b4c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c715c2542ebe84fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b4bfff640 1 -- 172.21.15.113:0/1193236398 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c036070 con 0x7f2b5415e6b0 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b4bfff640 1 -- 172.21.15.113:0/1193236398 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4c031600 con 0x7f2b5415e6b0 2024-04-25T00:08:56.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b4bfff640 1 -- 172.21.15.113:0/1193236398 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0319d0 con 0x7f2b5415e6b0 2024-04-25T00:08:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/1193236398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e6b0 msgr2=0x7f2b540b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/1193236398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e6b0 0x7f2b540b4870 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f2b54070ff0 tx=0x7f2b4c02a730 comp rx=0 tx=0).stop 2024-04-25T00:08:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/1193236398 shutdown_connections 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/1193236398 >> v2:172.21.15.178:3300/0 conn(0x7f2b540b3080 0x7f2b540b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/1193236398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e6b0 0x7f2b540b4870 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/1193236398 >> 172.21.15.113:0/1193236398 conn(0x7f2b5406b0f0 msgr2=0x7f2b5406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/1193236398 shutdown_connections 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.130+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/1193236398 wait complete. 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.131+0000 7f2b5b9dc640 1 Processor -- start 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b5b9dc640 1 -- start start 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b5b9dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b540b3080 0x7f2b54155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b5b9dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e2d0 0x7f2b540ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b5b9dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b5415e6b0 0x7f2b540aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b5b9dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540ce410 con 0x7f2b5415e2d0 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b5b9dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540ce290 con 0x7f2b540b3080 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b5b9dc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2b540ce590 con 0x7f2b5415e6b0 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b58f77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e2d0 0x7f2b540ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b58f77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e2d0 0x7f2b540ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51638/0 (socket says 172.21.15.113:51638) 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b58f77640 1 -- 172.21.15.113:0/3123343733 learned_addr learned my addr 172.21.15.113:0/3123343733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.132+0000 7f2b59778640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b540b3080 0x7f2b54155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b59f79640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b5415e6b0 0x7f2b540aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b58f77640 1 -- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b5415e6b0 msgr2=0x7f2b540aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b58f77640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b5415e6b0 0x7f2b540aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b58f77640 1 -- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b540b3080 msgr2=0x7f2b54155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b58f77640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b540b3080 0x7f2b54155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b58f77640 1 -- 172.21.15.113:0/3123343733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b540af050 con 0x7f2b5415e2d0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b59f79640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b5415e6b0 0x7f2b540aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b58f77640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e2d0 0x7f2b540ae4f0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000d800 tx=0x7f2b5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b49ffb640 1 -- 172.21.15.113:0/3123343733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000b6c0 con 0x7f2b5415e2d0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b49ffb640 1 -- 172.21.15.113:0/3123343733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b50019070 con 0x7f2b5415e2d0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b541f5260 con 0x7f2b5415e2d0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b49ffb640 1 -- 172.21.15.113:0/3123343733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000ba80 con 0x7f2b5415e2d0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.133+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b540a9d70 con 0x7f2b5415e2d0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.134+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b540691a0 con 0x7f2b5415e2d0 2024-04-25T00:08:56.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.134+0000 7f2b49ffb640 1 -- 172.21.15.113:0/3123343733 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b5000bc20 con 0x7f2b5415e2d0 2024-04-25T00:08:56.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.135+0000 7f2b49ffb640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b3806be30 0x7f2b3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.135+0000 7f2b49ffb640 1 -- 172.21.15.113:0/3123343733 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f2b5008b2a0 con 0x7f2b5415e2d0 2024-04-25T00:08:56.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.135+0000 7f2b59778640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b3806be30 0x7f2b3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.135+0000 7f2b59778640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b3806be30 0x7f2b3806e2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2b44001640 tx=0x7f2b44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.139+0000 7f2b49ffb640 1 -- 172.21.15.113:0/3123343733 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b50059ce0 con 0x7f2b5415e2d0 2024-04-25T00:08:56.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.314+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b540aa2c0 con 0x7f2b5415e2d0 2024-04-25T00:08:56.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.314+0000 7f2b49ffb640 1 -- 172.21.15.113:0/3123343733 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7f2b500595e0 con 0x7f2b5415e2d0 2024-04-25T00:08:56.315 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b3806be30 msgr2=0x7f2b3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b3806be30 0x7f2b3806e2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2b44001640 tx=0x7f2b44009040 comp rx=0 tx=0).stop 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e2d0 msgr2=0x7f2b540ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e2d0 0x7f2b540ae4f0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000d800 tx=0x7f2b5000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 shutdown_connections 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b3806be30 0x7f2b3806e2f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b5415e6b0 0x7f2b540aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b5415e2d0 0x7f2b540ae4f0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 --2- 172.21.15.113:0/3123343733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b540b3080 0x7f2b54155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 >> 172.21.15.113:0/3123343733 conn(0x7f2b5406b0f0 msgr2=0x7f2b540b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 shutdown_connections 2024-04-25T00:08:56.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.316+0000 7f2b5b9dc640 1 -- 172.21.15.113:0/3123343733 wait complete. 2024-04-25T00:08:56.327 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.423+0000 7ff127f7e640 1 Processor -- start 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.423+0000 7ff127f7e640 1 -- start start 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff127f7e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff120144910 0x7ff1200b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff127f7e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff120144cf0 con 0x7ff1200b5240 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff127f7e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1200b2f30 con 0x7ff1200b56b0 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff127f7e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff1200b4510 con 0x7ff120144910 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff125d1a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff120144910 0x7ff1200b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff125519640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff1200b5240 0x7ff12006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37232/0 (socket says 172.21.15.113:37232) 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff125519640 1 -- 172.21.15.113:0/2580590898 learned_addr learned my addr 172.21.15.113:0/2580590898 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff125d1a640 1 -- 172.21.15.113:0/2580590898 >> v1:172.21.15.178:6789/0 conn(0x7ff1200b56b0 legacy=0x7ff1200cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff125d1a640 1 -- 172.21.15.113:0/2580590898 >> v1:172.21.15.113:6789/0 conn(0x7ff1200b5240 legacy=0x7ff12006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff125d1a640 1 -- 172.21.15.113:0/2580590898 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1200b3210 con 0x7ff120144910 2024-04-25T00:08:56.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.424+0000 7ff124d18640 1 -- 172.21.15.113:0/2580590898 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 966081200 0 0) 0x7ff120144cf0 con 0x7ff1200b5240 2024-04-25T00:08:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.425+0000 7ff125d1a640 1 --2- 172.21.15.113:0/2580590898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff120144910 0x7ff1200b3fd0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097d0 tx=0x7ff11c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6964aac77a7f2885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.425+0000 7ff124d18640 1 -- 172.21.15.113:0/2580590898 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c037070 con 0x7ff120144910 2024-04-25T00:08:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.425+0000 7ff124d18640 1 -- 172.21.15.113:0/2580590898 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c032440 con 0x7ff120144910 2024-04-25T00:08:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.425+0000 7ff124d18640 1 -- 172.21.15.113:0/2580590898 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c032810 con 0x7ff120144910 2024-04-25T00:08:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.426+0000 7ff127f7e640 1 -- 172.21.15.113:0/2580590898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff120144910 msgr2=0x7ff1200b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.426+0000 7ff127f7e640 1 --2- 172.21.15.113:0/2580590898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff120144910 0x7ff1200b3fd0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097d0 tx=0x7ff11c02a210 comp rx=0 tx=0).stop 2024-04-25T00:08:56.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.426+0000 7ff127f7e640 1 -- 172.21.15.113:0/2580590898 shutdown_connections 2024-04-25T00:08:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.426+0000 7ff127f7e640 1 --2- 172.21.15.113:0/2580590898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff120144910 0x7ff1200b3fd0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.426+0000 7ff127f7e640 1 -- 172.21.15.113:0/2580590898 >> 172.21.15.113:0/2580590898 conn(0x7ff1200b1a50 msgr2=0x7ff1200b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.426+0000 7ff127f7e640 1 -- 172.21.15.113:0/2580590898 shutdown_connections 2024-04-25T00:08:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.426+0000 7ff127f7e640 1 -- 172.21.15.113:0/2580590898 wait complete. 2024-04-25T00:08:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.427+0000 7ff127f7e640 1 Processor -- start 2024-04-25T00:08:56.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.427+0000 7ff127f7e640 1 -- start start 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1200b5240 0x7ff1200cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1200b56b0 0x7ff1200cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff120144910 0x7ff1200ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1200b4da0 con 0x7ff1200b5240 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1200b37d0 con 0x7ff1200b56b0 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff1200b4f20 con 0x7ff120144910 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1200b5240 0x7ff1200cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1200b5240 0x7ff1200cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51642/0 (socket says 172.21.15.113:51642) 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 -- 172.21.15.113:0/3586392527 learned_addr learned my addr 172.21.15.113:0/3586392527 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:56.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 -- 172.21.15.113:0/3586392527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff120144910 msgr2=0x7ff1200ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff120144910 0x7ff1200ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 -- 172.21.15.113:0/3586392527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1200b56b0 msgr2=0x7ff1200cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1200b56b0 0x7ff1200cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 -- 172.21.15.113:0/3586392527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1200cda40 con 0x7ff1200b5240 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff125d1a640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1200b5240 0x7ff1200cd500 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097d0 tx=0x7ff11c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff1167fc640 1 -- 172.21.15.113:0/3586392527 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c037040 con 0x7ff1200b5240 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff1167fc640 1 -- 172.21.15.113:0/3586392527 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c004230 con 0x7ff1200b5240 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1200cdce0 con 0x7ff1200b5240 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff1167fc640 1 -- 172.21.15.113:0/3586392527 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c03c730 con 0x7ff1200b5240 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.428+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1200ce2a0 con 0x7ff1200b5240 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.429+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1200ce5e0 con 0x7ff1200b5240 2024-04-25T00:08:56.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.430+0000 7ff1167fc640 1 -- 172.21.15.113:0/3586392527 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff11c03a070 con 0x7ff1200b5240 2024-04-25T00:08:56.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.430+0000 7ff1167fc640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff10406be30 0x7ff10406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.430+0000 7ff125519640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff10406be30 0x7ff10406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.430+0000 7ff1167fc640 1 -- 172.21.15.113:0/3586392527 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7ff11c0ab740 con 0x7ff1200b5240 2024-04-25T00:08:56.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.431+0000 7ff125519640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff10406be30 0x7ff10406e2f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff110007fd0 tx=0x7ff110007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.435+0000 7ff1167fc640 1 -- 172.21.15.113:0/3586392527 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1200ce5e0 con 0x7ff1200b5240 2024-04-25T00:08:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.643+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1200731a0 con 0x7ff1200b5240 2024-04-25T00:08:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.644+0000 7ff1167fc640 1 -- 172.21.15.113:0/3586392527 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7ff11c041020 con 0x7ff1200b5240 2024-04-25T00:08:56.644 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:56.645 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff10406be30 msgr2=0x7ff10406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff10406be30 0x7ff10406e2f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff110007fd0 tx=0x7ff110007920 comp rx=0 tx=0).stop 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1200b5240 msgr2=0x7ff1200cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1200b5240 0x7ff1200cd500 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097d0 tx=0x7ff11c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 shutdown_connections 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff10406be30 0x7ff10406e2f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1200b56b0 0x7ff1200cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff120144910 0x7ff1200ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 --2- 172.21.15.113:0/3586392527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1200b5240 0x7ff1200cd500 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 >> 172.21.15.113:0/3586392527 conn(0x7ff1200b1a50 msgr2=0x7ff1200b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 shutdown_connections 2024-04-25T00:08:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.646+0000 7ff127f7e640 1 -- 172.21.15.113:0/3586392527 wait complete. 2024-04-25T00:08:56.657 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:56.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.753+0000 7fa1d706b640 1 Processor -- start 2024-04-25T00:08:56.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.753+0000 7fa1d706b640 1 -- start start 2024-04-25T00:08:56.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d706b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa1d0144910 0x7fa1d00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d706b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1d00b56b0 0x7fa1d00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d706b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d0144cf0 con 0x7fa1d00b5240 2024-04-25T00:08:56.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d706b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d00b4510 con 0x7fa1d00b56b0 2024-04-25T00:08:56.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d706b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d00b2f30 con 0x7fa1d0144910 2024-04-25T00:08:56.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d5608640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1d00b56b0 0x7fa1d00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d5608640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1d00b56b0 0x7fa1d00cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51644/0 (socket says 172.21.15.113:51644) 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d5608640 1 -- 172.21.15.113:0/2196609148 learned_addr learned my addr 172.21.15.113:0/2196609148 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d4e07640 1 --2- 172.21.15.113:0/2196609148 >> v2:172.21.15.178:3300/0 conn(0x7fa1d0144910 0x7fa1d00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d5608640 1 -- 172.21.15.113:0/2196609148 >> v2:172.21.15.178:3300/0 conn(0x7fa1d0144910 msgr2=0x7fa1d00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d5608640 1 --2- 172.21.15.113:0/2196609148 >> v2:172.21.15.178:3300/0 conn(0x7fa1d0144910 0x7fa1d00b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d5608640 1 -- 172.21.15.113:0/2196609148 >> v1:172.21.15.178:6789/0 conn(0x7fa1d00b5240 legacy=0x7fa1d006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d5608640 1 -- 172.21.15.113:0/2196609148 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d00b3210 con 0x7fa1d00b56b0 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.754+0000 7fa1d4e07640 1 --2- 172.21.15.113:0/2196609148 >> v2:172.21.15.178:3300/0 conn(0x7fa1d0144910 0x7fa1d00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:56.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.755+0000 7fa1d5608640 1 --2- 172.21.15.113:0/2196609148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b56b0 0x7fa1d00cd500 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009e50 tx=0x7fa1c802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda5ae6efa2afefd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.755+0000 7fa1cf7fe640 1 -- 172.21.15.113:0/2196609148 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8036070 con 0x7fa1d00b56b0 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.755+0000 7fa1cf7fe640 1 -- 172.21.15.113:0/2196609148 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c8031600 con 0x7fa1d00b56b0 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.755+0000 7fa1cf7fe640 1 -- 172.21.15.113:0/2196609148 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80319d0 con 0x7fa1d00b56b0 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 -- 172.21.15.113:0/2196609148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b56b0 msgr2=0x7fa1d00cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 --2- 172.21.15.113:0/2196609148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b56b0 0x7fa1d00cd500 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009e50 tx=0x7fa1c802a710 comp rx=0 tx=0).stop 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 -- 172.21.15.113:0/2196609148 shutdown_connections 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 --2- 172.21.15.113:0/2196609148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b56b0 0x7fa1d00cd500 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 --2- 172.21.15.113:0/2196609148 >> v2:172.21.15.178:3300/0 conn(0x7fa1d0144910 0x7fa1d00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 -- 172.21.15.113:0/2196609148 >> 172.21.15.113:0/2196609148 conn(0x7fa1d00b1a50 msgr2=0x7fa1d00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 -- 172.21.15.113:0/2196609148 shutdown_connections 2024-04-25T00:08:56.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.756+0000 7fa1d706b640 1 -- 172.21.15.113:0/2196609148 wait complete. 2024-04-25T00:08:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.757+0000 7fa1d706b640 1 Processor -- start 2024-04-25T00:08:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.757+0000 7fa1d706b640 1 -- start start 2024-04-25T00:08:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d706b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b5240 0x7fa1d01e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d706b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1d00b56b0 0x7fa1d00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d706b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1d0144910 0x7fa1d00ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d706b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d00b4da0 con 0x7fa1d00b5240 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d706b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d00b37d0 con 0x7fa1d0144910 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d706b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d00b4f20 con 0x7fa1d00b56b0 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b5240 0x7fa1d01e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b5240 0x7fa1d01e6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51656/0 (socket says 172.21.15.113:51656) 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 -- 172.21.15.113:0/3745815928 learned_addr learned my addr 172.21.15.113:0/3745815928 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1cffff640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1d00b56b0 0x7fa1d00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d5608640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1d0144910 0x7fa1d00ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 -- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1d00b56b0 msgr2=0x7fa1d00abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1d00b56b0 0x7fa1d00abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 -- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1d0144910 msgr2=0x7fa1d00ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1d0144910 0x7fa1d00ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 -- 172.21.15.113:0/3745815928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01e72f0 con 0x7fa1d00b5240 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d5608640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1d0144910 0x7fa1d00ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1cffff640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1d00b56b0 0x7fa1d00abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1d4e07640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b5240 0x7fa1d01e6db0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c7f0 tx=0x7fa1c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.758+0000 7fa1cd7fa640 1 -- 172.21.15.113:0/3745815928 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0015070 con 0x7fa1d00b5240 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.759+0000 7fa1cd7fa640 1 -- 172.21.15.113:0/3745815928 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c0007820 con 0x7fa1d00b5240 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.759+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01e5d10 con 0x7fa1d00b5240 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.759+0000 7fa1cd7fa640 1 -- 172.21.15.113:0/3745815928 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0007bd0 con 0x7fa1d00b5240 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.759+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01e62d0 con 0x7fa1d00b5240 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.760+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d00691a0 con 0x7fa1d00b5240 2024-04-25T00:08:56.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.760+0000 7fa1cd7fa640 1 -- 172.21.15.113:0/3745815928 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1c0018070 con 0x7fa1d00b5240 2024-04-25T00:08:56.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.761+0000 7fa1cd7fa640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1bc06be30 0x7fa1bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.761+0000 7fa1cffff640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1bc06be30 0x7fa1bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.761+0000 7fa1cd7fa640 1 -- 172.21.15.113:0/3745815928 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fa1c008aed0 con 0x7fa1d00b5240 2024-04-25T00:08:56.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.761+0000 7fa1cffff640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1bc06be30 0x7fa1bc06e2f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00af5b0 tx=0x7fa1c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.765+0000 7fa1cd7fa640 1 -- 172.21.15.113:0/3745815928 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1c0059910 con 0x7fa1d00b5240 2024-04-25T00:08:56.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.939+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1d00cef90 con 0x7fa1d00b5240 2024-04-25T00:08:56.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.940+0000 7fa1cd7fa640 1 -- 172.21.15.113:0/3745815928 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16719 (secure 0 0 0) 0x7fa1c0059210 con 0x7fa1d00b5240 2024-04-25T00:08:56.941 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:56.941 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":158,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:15.696246+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:08:12.252280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:08:13.007636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1bc06be30 msgr2=0x7fa1bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1bc06be30 0x7fa1bc06e2f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00af5b0 tx=0x7fa1c4009040 comp rx=0 tx=0).stop 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b5240 msgr2=0x7fa1d01e6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b5240 0x7fa1d01e6db0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c7f0 tx=0x7fa1c000ccb0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 shutdown_connections 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa1bc06be30 0x7fa1bc06e2f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1d0144910 0x7fa1d00ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1d00b56b0 0x7fa1d00abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 --2- 172.21.15.113:0/3745815928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d00b5240 0x7fa1d01e6db0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 >> 172.21.15.113:0/3745815928 conn(0x7fa1d00b1a50 msgr2=0x7fa1d00b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 shutdown_connections 2024-04-25T00:08:56.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:56.942+0000 7fa1d706b640 1 -- 172.21.15.113:0/3745815928 wait complete. 2024-04-25T00:08:56.954 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:08:56.954 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.050+0000 7f1158f5a640 1 Processor -- start 2024-04-25T00:08:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.050+0000 7f1158f5a640 1 -- start start 2024-04-25T00:08:57.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1158f5a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1154144910 0x7f11540b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1158f5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5240 0x7f115406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1158f5a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f11540b5680 0x7f115406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1158f5a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f11540b4510 con 0x7f1154144910 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1158f5a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1154144cf0 con 0x7f11540b5680 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1158f5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11540b2f30 con 0x7f11540b5240 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1154144910 0x7f11540b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1154144910 0x7f11540b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51664/0 (socket says 172.21.15.113:51664) 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 -- 172.21.15.113:0/2038430214 learned_addr learned my addr 172.21.15.113:0/2038430214 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f114bfff640 1 --2- 172.21.15.113:0/2038430214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5240 0x7f115406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1152d9d640 1 --2- 172.21.15.113:0/2038430214 >> v2:172.21.15.178:3300/0 conn(0x7f11540b5680 0x7f115406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 -- 172.21.15.113:0/2038430214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5240 msgr2=0x7f115406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 --2- 172.21.15.113:0/2038430214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5240 0x7f115406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 -- 172.21.15.113:0/2038430214 >> v2:172.21.15.178:3300/0 conn(0x7f11540b5680 msgr2=0x7f115406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 --2- 172.21.15.113:0/2038430214 >> v2:172.21.15.178:3300/0 conn(0x7f11540b5680 0x7f115406e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f115259c640 1 -- 172.21.15.113:0/2038430214 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11540b3210 con 0x7f1154144910 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f114bfff640 1 --2- 172.21.15.113:0/2038430214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5240 0x7f115406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:57.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.051+0000 7f1152d9d640 1 --2- 172.21.15.113:0/2038430214 >> v2:172.21.15.178:3300/0 conn(0x7f11540b5680 0x7f115406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:57.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f115259c640 1 --2- 172.21.15.113:0/2038430214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f11540b3fd0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f1154062820 tx=0x7f114402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2f059bd57a377f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1151d9b640 1 -- 172.21.15.113:0/2038430214 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144037070 con 0x7f1154144910 2024-04-25T00:08:57.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1151d9b640 1 -- 172.21.15.113:0/2038430214 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1144032440 con 0x7f1154144910 2024-04-25T00:08:57.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1151d9b640 1 -- 172.21.15.113:0/2038430214 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144032810 con 0x7f1154144910 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1158f5a640 1 -- 172.21.15.113:0/2038430214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 msgr2=0x7f11540b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1158f5a640 1 --2- 172.21.15.113:0/2038430214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f11540b3fd0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f1154062820 tx=0x7f114402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1158f5a640 1 -- 172.21.15.113:0/2038430214 shutdown_connections 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1158f5a640 1 --2- 172.21.15.113:0/2038430214 >> v2:172.21.15.178:3300/0 conn(0x7f11540b5680 0x7f115406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1158f5a640 1 --2- 172.21.15.113:0/2038430214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5240 0x7f115406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1158f5a640 1 --2- 172.21.15.113:0/2038430214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f11540b3fd0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.052+0000 7f1158f5a640 1 -- 172.21.15.113:0/2038430214 >> 172.21.15.113:0/2038430214 conn(0x7f11540b1a50 msgr2=0x7f11540b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.053+0000 7f1158f5a640 1 -- 172.21.15.113:0/2038430214 shutdown_connections 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.053+0000 7f1158f5a640 1 -- 172.21.15.113:0/2038430214 wait complete. 2024-04-25T00:08:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.054+0000 7f1158f5a640 1 Processor -- start 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.054+0000 7f1158f5a640 1 -- start start 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.054+0000 7f1158f5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11540b5240 0x7f11540ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1158f5a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5680 0x7f115415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1158f5a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f115415da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1158f5a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f11540b4da0 con 0x7f1154144910 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1158f5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11540b37d0 con 0x7f11540b5240 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1158f5a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11540b4f20 con 0x7f11540b5680 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f115415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f115415da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51666/0 (socket says 172.21.15.113:51666) 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 -- 172.21.15.113:0/617360103 learned_addr learned my addr 172.21.15.113:0/617360103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f115259c640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11540b5240 0x7f11540ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f114bfff640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5680 0x7f115415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 -- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5680 msgr2=0x7f115415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5680 0x7f115415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 -- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11540b5240 msgr2=0x7f11540ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11540b5240 0x7f11540ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 -- 172.21.15.113:0/617360103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11540c4cc0 con 0x7f1154144910 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1152d9d640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f115415da80 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f114c00cdd0 tx=0x7f114c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f114affd640 1 -- 172.21.15.113:0/617360103 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c01a070 con 0x7f1154144910 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f114affd640 1 -- 172.21.15.113:0/617360103 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114c00d840 con 0x7f1154144910 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f114affd640 1 -- 172.21.15.113:0/617360103 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c00dc70 con 0x7f1154144910 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11540acf80 con 0x7f1154144910 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.055+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11540c5240 con 0x7f1154144910 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.056+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11540691a0 con 0x7f1154144910 2024-04-25T00:08:57.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.057+0000 7f114affd640 1 -- 172.21.15.113:0/617360103 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f114c01d070 con 0x7f1154144910 2024-04-25T00:08:57.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.057+0000 7f114affd640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f113406be30 0x7f113406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.057+0000 7f115259c640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f113406be30 0x7f113406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.057+0000 7f114affd640 1 -- 172.21.15.113:0/617360103 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f114c08b5b0 con 0x7f1154144910 2024-04-25T00:08:57.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.058+0000 7f115259c640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f113406be30 0x7f113406e2f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1144034000 tx=0x7f114402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.062+0000 7f114affd640 1 -- 172.21.15.113:0/617360103 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f114c059910 con 0x7f1154144910 2024-04-25T00:08:57.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.271+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1154143500 con 0x7f1154144910 2024-04-25T00:08:57.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.272+0000 7f114affd640 1 -- 172.21.15.113:0/617360103 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7f114c05dc10 con 0x7f1154144910 2024-04-25T00:08:57.272 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:57.273 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:57.273 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:57.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f113406be30 msgr2=0x7f113406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f113406be30 0x7f113406e2f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1144034000 tx=0x7f114402e000 comp rx=0 tx=0).stop 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 msgr2=0x7f115415da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f115415da80 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f114c00cdd0 tx=0x7f114c00a710 comp rx=0 tx=0).stop 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 shutdown_connections 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f113406be30 0x7f113406e2f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1154144910 0x7f115415da80 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11540b5680 0x7f115415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 --2- 172.21.15.113:0/617360103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11540b5240 0x7f11540ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 >> 172.21.15.113:0/617360103 conn(0x7f11540b1a50 msgr2=0x7f11540cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 shutdown_connections 2024-04-25T00:08:57.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.274+0000 7f1158f5a640 1 -- 172.21.15.113:0/617360103 wait complete. 2024-04-25T00:08:57.285 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:08:57.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.382+0000 7f4659c37640 1 Processor -- start 2024-04-25T00:08:57.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.382+0000 7f4659c37640 1 -- start start 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.382+0000 7f4659c37640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46540d2290 0x7f46540b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.382+0000 7f4659c37640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46540d0b00 0x7f46540d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.382+0000 7f4659c37640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46540d1480 con 0x7f465415e320 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.382+0000 7f4659c37640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f46540d3410 con 0x7f46540d2290 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.382+0000 7f4659c37640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f46540d2800 con 0x7f46540d0b00 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f46537fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f465415e320 0x7f46540d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37244/0 (socket says 172.21.15.113:37244) 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f46537fe640 1 -- 172.21.15.113:0/3087634421 learned_addr learned my addr 172.21.15.113:0/3087634421 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f4652ffd640 1 --2- 172.21.15.113:0/3087634421 >> v2:172.21.15.113:3300/0 conn(0x7f46540d2290 0x7f46540b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f4653fff640 1 --2- 172.21.15.113:0/3087634421 >> v2:172.21.15.178:3300/0 conn(0x7f46540d0b00 0x7f46540d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f46527fc640 1 -- 172.21.15.113:0/3087634421 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3550863310 0 0) 0x7f46540d1480 con 0x7f465415e320 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f46527fc640 1 -- 172.21.15.113:0/3087634421 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f463c003520 con 0x7f465415e320 2024-04-25T00:08:57.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f4652ffd640 1 -- 172.21.15.113:0/3087634421 >> v2:172.21.15.178:3300/0 conn(0x7f46540d0b00 msgr2=0x7f46540d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f4652ffd640 1 --2- 172.21.15.113:0/3087634421 >> v2:172.21.15.178:3300/0 conn(0x7f46540d0b00 0x7f46540d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f4652ffd640 1 -- 172.21.15.113:0/3087634421 >> v1:172.21.15.113:6789/0 conn(0x7f465415e320 legacy=0x7f46540d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f4652ffd640 1 -- 172.21.15.113:0/3087634421 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46540d2ae0 con 0x7f46540d2290 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.383+0000 7f4653fff640 1 --2- 172.21.15.113:0/3087634421 >> v2:172.21.15.178:3300/0 conn(0x7f46540d0b00 0x7f46540d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4652ffd640 1 --2- 172.21.15.113:0/3087634421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d2290 0x7f46540b9aa0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f46540b3970 tx=0x7f464802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b8fb02bbec2249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f46527fc640 1 -- 172.21.15.113:0/3087634421 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648037070 con 0x7f46540d2290 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f46527fc640 1 -- 172.21.15.113:0/3087634421 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4648032440 con 0x7f46540d2290 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f46527fc640 1 -- 172.21.15.113:0/3087634421 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648032810 con 0x7f46540d2290 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 -- 172.21.15.113:0/3087634421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d2290 msgr2=0x7f46540b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 --2- 172.21.15.113:0/3087634421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d2290 0x7f46540b9aa0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f46540b3970 tx=0x7f464802a230 comp rx=0 tx=0).stop 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 -- 172.21.15.113:0/3087634421 shutdown_connections 2024-04-25T00:08:57.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 --2- 172.21.15.113:0/3087634421 >> v2:172.21.15.178:3300/0 conn(0x7f46540d0b00 0x7f46540d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 --2- 172.21.15.113:0/3087634421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d2290 0x7f46540b9aa0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 -- 172.21.15.113:0/3087634421 >> 172.21.15.113:0/3087634421 conn(0x7f46540b5c20 msgr2=0x7f46540b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:57.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 -- 172.21.15.113:0/3087634421 shutdown_connections 2024-04-25T00:08:57.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.384+0000 7f4659c37640 1 -- 172.21.15.113:0/3087634421 wait complete. 2024-04-25T00:08:57.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 Processor -- start 2024-04-25T00:08:57.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 -- start start 2024-04-25T00:08:57.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d0b00 0x7f46540d4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46540d2290 0x7f4654160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465415e320 0x7f4654151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f46540b9fe0 con 0x7f46540d0b00 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46540d30a0 con 0x7f46540d2290 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.386+0000 7f4659c37640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f465415e700 con 0x7f465415e320 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d0b00 0x7f46540d4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d0b00 0x7f46540d4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51684/0 (socket says 172.21.15.113:51684) 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 -- 172.21.15.113:0/2027496534 learned_addr learned my addr 172.21.15.113:0/2027496534 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 -- 172.21.15.113:0/2027496534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465415e320 msgr2=0x7f4654151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465415e320 0x7f4654151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 -- 172.21.15.113:0/2027496534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46540d2290 msgr2=0x7f4654160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46540d2290 0x7f4654160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 -- 172.21.15.113:0/2027496534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465415fbb0 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f46537fe640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d0b00 0x7f46540d4ac0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f464400c430 tx=0x7f464400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f463bfff640 1 -- 172.21.15.113:0/2027496534 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644017070 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f463bfff640 1 -- 172.21.15.113:0/2027496534 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4644005ce0 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465415fe90 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f463bfff640 1 -- 172.21.15.113:0/2027496534 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46440072f0 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.387+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4654160450 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.388+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46540691a0 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.388+0000 7f463bfff640 1 -- 172.21.15.113:0/2027496534 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f464401a070 con 0x7f46540d0b00 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.389+0000 7f463bfff640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f464006be60 0x7f464006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.389+0000 7f4652ffd640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f464006be60 0x7f464006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.389+0000 7f463bfff640 1 -- 172.21.15.113:0/2027496534 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7f464408a2f0 con 0x7f46540d0b00 2024-04-25T00:08:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.389+0000 7f4652ffd640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f464006be60 0x7f464006e320 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4648034000 tx=0x7f464802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.393+0000 7f463bfff640 1 -- 172.21.15.113:0/2027496534 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4644058ca0 con 0x7f46540d0b00 2024-04-25T00:08:57.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.604+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46540d14e0 con 0x7f46540d0b00 2024-04-25T00:08:57.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.605+0000 7f463bfff640 1 -- 172.21.15.113:0/2027496534 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5188 (secure 0 0 0) 0x7f46440585a0 con 0x7f46540d0b00 2024-04-25T00:08:57.605 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:57.606 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:08:14.036250+0000","modified":"2024-04-25T00:08:16.209381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9987":{"gid":9987,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6834/340854702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":340854702},{"type":"v1","addr":"172.21.15.178:6834","nonce":340854702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9987,"qdb_cluster":[9987]},"id":11}]} 2024-04-25T00:08:57.606 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f464006be60 msgr2=0x7f464006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f464006be60 0x7f464006e320 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4648034000 tx=0x7f464802c000 comp rx=0 tx=0).stop 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d0b00 msgr2=0x7f46540d4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d0b00 0x7f46540d4ac0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f464400c430 tx=0x7f464400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 shutdown_connections 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f464006be60 0x7f464006e320 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46540d2290 0x7f4654160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465415e320 0x7f4654151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 --2- 172.21.15.113:0/2027496534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46540d0b00 0x7f46540d4ac0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 >> 172.21.15.113:0/2027496534 conn(0x7f46540b5c20 msgr2=0x7f46540ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:57.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 shutdown_connections 2024-04-25T00:08:57.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.607+0000 7f4659c37640 1 -- 172.21.15.113:0/2027496534 wait complete. 2024-04-25T00:08:57.619 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:08:57.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.714+0000 7fd3abcb3640 1 Processor -- start 2024-04-25T00:08:57.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.714+0000 7fd3abcb3640 1 -- start start 2024-04-25T00:08:57.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3abcb3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3a4144910 0x7fd3a40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3abcb3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5240 0x7fd3a406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3abcb3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a4144cf0 con 0x7fd3a40b5680 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3abcb3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3a40b4510 con 0x7fd3a4144910 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3abcb3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a40b2f30 con 0x7fd3a40b5240 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a9a4f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3a4144910 0x7fd3a40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3aa250640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd3a40b5680 0x7fd3a40cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:35060/0 (socket says 172.21.15.113:35060) 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3aa250640 1 -- 172.21.15.113:0/595328062 learned_addr learned my addr 172.21.15.113:0/595328062 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a924e640 1 --2- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5240 0x7fd3a406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a8a4d640 1 -- 172.21.15.113:0/595328062 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142591088 0 0) 0x7fd3a4144cf0 con 0x7fd3a40b5680 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a8a4d640 1 -- 172.21.15.113:0/595328062 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd390003520 con 0x7fd3a40b5680 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a9a4f640 1 -- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5240 msgr2=0x7fd3a406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a9a4f640 1 --2- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5240 0x7fd3a406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a9a4f640 1 -- 172.21.15.113:0/595328062 >> v1:172.21.15.178:6789/0 conn(0x7fd3a40b5680 legacy=0x7fd3a40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a9a4f640 1 -- 172.21.15.113:0/595328062 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a40b3210 con 0x7fd3a4144910 2024-04-25T00:08:57.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.715+0000 7fd3a924e640 1 --2- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5240 0x7fd3a406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:08:57.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.716+0000 7fd3a9a4f640 1 --2- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 0x7fd3a40b3fd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00097a0 tx=0x7fd3a002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc310b397f1bdd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.716+0000 7fd3a8a4d640 1 -- 172.21.15.113:0/595328062 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0037070 con 0x7fd3a4144910 2024-04-25T00:08:57.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.716+0000 7fd3a8a4d640 1 -- 172.21.15.113:0/595328062 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a0032440 con 0x7fd3a4144910 2024-04-25T00:08:57.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.716+0000 7fd3a8a4d640 1 -- 172.21.15.113:0/595328062 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a00327f0 con 0x7fd3a4144910 2024-04-25T00:08:57.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 -- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 msgr2=0x7fd3a40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 0x7fd3a40b3fd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00097a0 tx=0x7fd3a002a180 comp rx=0 tx=0).stop 2024-04-25T00:08:57.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 -- 172.21.15.113:0/595328062 shutdown_connections 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5240 0x7fd3a406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/595328062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 0x7fd3a40b3fd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 -- 172.21.15.113:0/595328062 >> 172.21.15.113:0/595328062 conn(0x7fd3a40b1a50 msgr2=0x7fd3a40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 -- 172.21.15.113:0/595328062 shutdown_connections 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.717+0000 7fd3abcb3640 1 -- 172.21.15.113:0/595328062 wait complete. 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.718+0000 7fd3abcb3640 1 Processor -- start 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.718+0000 7fd3abcb3640 1 -- start start 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3abcb3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a40b5240 0x7fd3a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3abcb3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5680 0x7fd3a40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3abcb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 0x7fd3a413bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3abcb3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a40b4da0 con 0x7fd3a40b5240 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3abcb3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a40b37d0 con 0x7fd3a4144910 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3abcb3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a40b4f20 con 0x7fd3a40b5680 2024-04-25T00:08:57.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a40b5240 0x7fd3a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a40b5240 0x7fd3a415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36638/0 (socket says 172.21.15.113:36638) 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 -- 172.21.15.113:0/2789166416 learned_addr learned my addr 172.21.15.113:0/2789166416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a924e640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5680 0x7fd3a40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3aa250640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 0x7fd3a413bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 -- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5680 msgr2=0x7fd3a40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5680 0x7fd3a40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 -- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 msgr2=0x7fd3a413bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 0x7fd3a413bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 -- 172.21.15.113:0/2789166416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a413c0c0 con 0x7fd3a40b5240 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a924e640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5680 0x7fd3a40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.719+0000 7fd3a9a4f640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a40b5240 0x7fd3a415d5d0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fd3a002a1c0 tx=0x7fd3a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.720+0000 7fd39a7fc640 1 -- 172.21.15.113:0/2789166416 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0037040 con 0x7fd3a40b5240 2024-04-25T00:08:57.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.720+0000 7fd39a7fc640 1 -- 172.21.15.113:0/2789166416 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a0042070 con 0x7fd3a40b5240 2024-04-25T00:08:57.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.720+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a40cd8a0 con 0x7fd3a40b5240 2024-04-25T00:08:57.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.720+0000 7fd39a7fc640 1 -- 172.21.15.113:0/2789166416 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a003b930 con 0x7fd3a40b5240 2024-04-25T00:08:57.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.720+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a40cde60 con 0x7fd3a40b5240 2024-04-25T00:08:57.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.721+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a40cce00 con 0x7fd3a40b5240 2024-04-25T00:08:57.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.721+0000 7fd39a7fc640 1 -- 172.21.15.113:0/2789166416 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3a003bad0 con 0x7fd3a40b5240 2024-04-25T00:08:57.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.722+0000 7fd39a7fc640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd38806be30 0x7fd38806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:57.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.722+0000 7fd39a7fc640 1 -- 172.21.15.113:0/2789166416 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fd3a00ab560 con 0x7fd3a40b5240 2024-04-25T00:08:57.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.722+0000 7fd3a924e640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd38806be30 0x7fd38806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:57.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.722+0000 7fd3a924e640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd38806be30 0x7fd38806e2f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd394009af0 tx=0x7fd394008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:57.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.726+0000 7fd39a7fc640 1 -- 172.21.15.113:0/2789166416 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3a40cce00 con 0x7fd3a40b5240 2024-04-25T00:08:57.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.901+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3a40691a0 con 0x7fd3a40b5240 2024-04-25T00:08:57.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.903+0000 7fd39a7fc640 1 -- 172.21.15.113:0/2789166416 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16719 (secure 0 0 0) 0x7fd3a0079fa0 con 0x7fd3a40b5240 2024-04-25T00:08:57.903 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:08:57.903 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":158,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:08:15.696246+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:08:12.252280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T00:08:13.007636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:08:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.904+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd38806be30 msgr2=0x7fd38806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.904+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd38806be30 0x7fd38806e2f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd394009af0 tx=0x7fd394008040 comp rx=0 tx=0).stop 2024-04-25T00:08:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.904+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a40b5240 msgr2=0x7fd3a415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.904+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a40b5240 0x7fd3a415d5d0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fd3a002a1c0 tx=0x7fd3a00046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 shutdown_connections 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd38806be30 0x7fd38806e2f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a4144910 0x7fd3a413bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a40b5680 0x7fd3a40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 --2- 172.21.15.113:0/2789166416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a40b5240 0x7fd3a415d5d0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 >> 172.21.15.113:0/2789166416 conn(0x7fd3a40b1a50 msgr2=0x7fd3a4143be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 shutdown_connections 2024-04-25T00:08:57.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:57.905+0000 7fd3abcb3640 1 -- 172.21.15.113:0/2789166416 wait complete. 2024-04-25T00:08:57.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:08:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.013+0000 7fe8ea693640 1 Processor -- start 2024-04-25T00:08:58.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.013+0000 7fe8ea693640 1 -- start start 2024-04-25T00:08:58.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8ea693640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 0x7fe8e415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8ea693640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe8e40b3080 0x7fe8e40b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8ea693640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e415f4f0 con 0x7fe8e415e6b0 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8ea693640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e40cde00 con 0x7fe8e40b3080 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8ea693640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e415f7d0 con 0x7fe8e415e2d0 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8e37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe8e40b3080 0x7fe8e40b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8e8c30640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe8e415e6b0 0x7fe8e4071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33774/0 (socket says 172.21.15.113:33774) 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8e8c30640 1 -- 172.21.15.113:0/3926860017 learned_addr learned my addr 172.21.15.113:0/3926860017 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.014+0000 7fe8e3fff640 1 --2- 172.21.15.113:0/3926860017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 0x7fe8e415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e37fe640 1 -- 172.21.15.113:0/3926860017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 msgr2=0x7fe8e415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e37fe640 1 --2- 172.21.15.113:0/3926860017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 0x7fe8e415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e37fe640 1 -- 172.21.15.113:0/3926860017 >> v1:172.21.15.113:6789/0 conn(0x7fe8e415e6b0 legacy=0x7fe8e4071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e37fe640 1 -- 172.21.15.113:0/3926860017 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e415fab0 con 0x7fe8e40b3080 2024-04-25T00:08:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e2ffd640 1 -- 172.21.15.113:0/3926860017 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172313046 0 0) 0x7fe8e415f4f0 con 0x7fe8e415e6b0 2024-04-25T00:08:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e37fe640 1 --2- 172.21.15.113:0/3926860017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e40b4330 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097a0 tx=0x7fe8d802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3550d08d44156b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e2ffd640 1 -- 172.21.15.113:0/3926860017 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8037070 con 0x7fe8e40b3080 2024-04-25T00:08:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e2ffd640 1 -- 172.21.15.113:0/3926860017 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d802ae50 con 0x7fe8e40b3080 2024-04-25T00:08:58.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.015+0000 7fe8e2ffd640 1 -- 172.21.15.113:0/3926860017 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8032660 con 0x7fe8e40b3080 2024-04-25T00:08:58.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 -- 172.21.15.113:0/3926860017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 msgr2=0x7fe8e40b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 --2- 172.21.15.113:0/3926860017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e40b4330 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097a0 tx=0x7fe8d802a100 comp rx=0 tx=0).stop 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 -- 172.21.15.113:0/3926860017 shutdown_connections 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 --2- 172.21.15.113:0/3926860017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e40b4330 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 --2- 172.21.15.113:0/3926860017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 0x7fe8e415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 -- 172.21.15.113:0/3926860017 >> 172.21.15.113:0/3926860017 conn(0x7fe8e406b0f0 msgr2=0x7fe8e406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 -- 172.21.15.113:0/3926860017 shutdown_connections 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.016+0000 7fe8ea693640 1 -- 172.21.15.113:0/3926860017 wait complete. 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.017+0000 7fe8ea693640 1 Processor -- start 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.017+0000 7fe8ea693640 1 -- start start 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8ea693640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e4148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8ea693640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 0x7fe8e415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8ea693640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e415e6b0 0x7fe8e4152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8ea693640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e415f2b0 con 0x7fe8e40b3080 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8ea693640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e415f130 con 0x7fe8e415e6b0 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8ea693640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40cf2a0 con 0x7fe8e415e2d0 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e4148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e4148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36654/0 (socket says 172.21.15.113:36654) 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 -- 172.21.15.113:0/580665631 learned_addr learned my addr 172.21.15.113:0/580665631 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 -- 172.21.15.113:0/580665631 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 msgr2=0x7fe8e415a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 0x7fe8e415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 -- 172.21.15.113:0/580665631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e415e6b0 msgr2=0x7fe8e4152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e415e6b0 0x7fe8e4152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 -- 172.21.15.113:0/580665631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e4149170 con 0x7fe8e40b3080 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.018+0000 7fe8e3fff640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e4148c30 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400d830 tx=0x7fe8d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.019+0000 7fe8e0ff9640 1 -- 172.21.15.113:0/580665631 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400b6c0 con 0x7fe8e40b3080 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.019+0000 7fe8e0ff9640 1 -- 172.21.15.113:0/580665631 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4019070 con 0x7fe8e40b3080 2024-04-25T00:08:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.019+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e40afdc0 con 0x7fe8e40b3080 2024-04-25T00:08:58.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.019+0000 7fe8e0ff9640 1 -- 172.21.15.113:0/580665631 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400ba80 con 0x7fe8e40b3080 2024-04-25T00:08:58.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.019+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e40b0380 con 0x7fe8e40b3080 2024-04-25T00:08:58.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.020+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e40691a0 con 0x7fe8e40b3080 2024-04-25T00:08:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.020+0000 7fe8e0ff9640 1 -- 172.21.15.113:0/580665631 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8d401e460 con 0x7fe8e40b3080 2024-04-25T00:08:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.020+0000 7fe8e0ff9640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d006be30 0x7fe8d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.021+0000 7fe8e0ff9640 1 -- 172.21.15.113:0/580665631 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5709+0+0 (secure 0 0 0) 0x7fe8d408b540 con 0x7fe8e40b3080 2024-04-25T00:08:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.021+0000 7fe8e37fe640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d006be30 0x7fe8d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.021+0000 7fe8e37fe640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d006be30 0x7fe8d006e2f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8034000 tx=0x7fe8d802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.025+0000 7fe8e0ff9640 1 -- 172.21.15.113:0/580665631 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe8d4059ef0 con 0x7fe8e40b3080 2024-04-25T00:08:58.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.234+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe8e415daf0 con 0x7fe8e40b3080 2024-04-25T00:08:58.235 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:08:58.235+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:08:58.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.236+0000 7fe8e0ff9640 1 -- 172.21.15.113:0/580665631 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7fe8d40600c0 con 0x7fe8e40b3080 2024-04-25T00:08:58.237 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:08:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.239+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d006be30 msgr2=0x7fe8d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.239+0000 7fe8ea693640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d006be30 0x7fe8d006e2f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8034000 tx=0x7fe8d802c000 comp rx=0 tx=0).stop 2024-04-25T00:08:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.239+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 msgr2=0x7fe8e4148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.239+0000 7fe8ea693640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e4148c30 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400d830 tx=0x7fe8d400dcf0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 shutdown_connections 2024-04-25T00:08:58.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe8d006be30 0x7fe8d006e2f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8e415e6b0 0x7fe8e4152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8e415e2d0 0x7fe8e415a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 --2- 172.21.15.113:0/580665631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8e40b3080 0x7fe8e4148c30 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 >> 172.21.15.113:0/580665631 conn(0x7fe8e406b0f0 msgr2=0x7fe8e40cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:58.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 shutdown_connections 2024-04-25T00:08:58.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.240+0000 7fe8ea693640 1 -- 172.21.15.113:0/580665631 wait complete. 2024-04-25T00:08:58.243 INFO:tasks.ceph.mds.b.smithi178.stderr: -6437> 2024-04-25T00:08:15.336+0000 7f06ec2ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:08:58.246 INFO:tasks.ceph.mds.b.smithi178.stderr: -3209> 2024-04-25T00:08:21.778+0000 7f06e82f7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T00:08:20.731019+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={7268=0-12582912@1} caps={7268=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=7268 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x556320ddec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(23)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T00:08:58.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:08:58.275 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:08:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.350+0000 7ff2e3a5d640 1 Processor -- start 2024-04-25T00:08:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.350+0000 7ff2e3a5d640 1 -- start start 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e3a5d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2dc15e2d0 0x7ff2dc15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e3a5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc0b3080 0x7ff2dc0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e3a5d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2dc15f4f0 con 0x7ff2dc15e6b0 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e3a5d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff2dc0cde00 con 0x7ff2dc15e2d0 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e3a5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc15f7d0 con 0x7ff2dc0b3080 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e17f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2dc15e2d0 0x7ff2dc15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e17f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2dc15e2d0 0x7ff2dc15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36660/0 (socket says 172.21.15.113:36660) 2024-04-25T00:08:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1143223535 learned_addr learned my addr 172.21.15.113:0/1143223535 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e1ffa640 1 --1- 172.21.15.113:0/1143223535 >> v1:172.21.15.113:6789/0 conn(0x7ff2dc15e6b0 0x7ff2dc071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33776/0 (socket says 172.21.15.113:33776) 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e0ff8640 1 --2- 172.21.15.113:0/1143223535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc0b3080 0x7ff2dc0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1143223535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc0b3080 msgr2=0x7ff2dc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e17f9640 1 --2- 172.21.15.113:0/1143223535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc0b3080 0x7ff2dc0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1143223535 >> v1:172.21.15.113:6789/0 conn(0x7ff2dc15e6b0 legacy=0x7ff2dc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1143223535 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc15fab0 con 0x7ff2dc15e2d0 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.351+0000 7ff2d3fff640 1 -- 172.21.15.113:0/1143223535 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178961190 0 0) 0x7ff2dc15f4f0 con 0x7ff2dc15e6b0 2024-04-25T00:08:58.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.352+0000 7ff2e17f9640 1 --2- 172.21.15.113:0/1143223535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc15e2d0 0x7ff2dc15ebf0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009870 tx=0x7ff2d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d38bb117a216dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:58.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.352+0000 7ff2d3fff640 1 -- 172.21.15.113:0/1143223535 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8037070 con 0x7ff2dc15e2d0 2024-04-25T00:08:58.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.352+0000 7ff2d3fff640 1 -- 172.21.15.113:0/1143223535 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d8032440 con 0x7ff2dc15e2d0 2024-04-25T00:08:58.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.352+0000 7ff2d3fff640 1 -- 172.21.15.113:0/1143223535 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8032810 con 0x7ff2dc15e2d0 2024-04-25T00:08:58.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1143223535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc15e2d0 msgr2=0x7ff2dc15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1143223535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc15e2d0 0x7ff2dc15ebf0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009870 tx=0x7ff2d802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1143223535 shutdown_connections 2024-04-25T00:08:58.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1143223535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc0b3080 0x7ff2dc0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1143223535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc15e2d0 0x7ff2dc15ebf0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1143223535 >> 172.21.15.113:0/1143223535 conn(0x7ff2dc06b0f0 msgr2=0x7ff2dc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1143223535 shutdown_connections 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.353+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1143223535 wait complete. 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.354+0000 7ff2e3a5d640 1 Processor -- start 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.354+0000 7ff2e3a5d640 1 -- start start 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.354+0000 7ff2e3a5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc0b3080 0x7ff2dc14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e3a5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc15e2d0 0x7ff2dc137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e3a5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2dc15e6b0 0x7ff2dc152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e3a5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc15f2b0 con 0x7ff2dc0b3080 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e3a5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc15f130 con 0x7ff2dc15e6b0 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e3a5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc0cf2a0 con 0x7ff2dc15e2d0 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc0b3080 0x7ff2dc14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc0b3080 0x7ff2dc14ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36676/0 (socket says 172.21.15.113:36676) 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1608496441 learned_addr learned my addr 172.21.15.113:0/1608496441 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e1ffa640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2dc15e6b0 0x7ff2dc152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e0ff8640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc15e2d0 0x7ff2dc137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc15e2d0 msgr2=0x7ff2dc137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc15e2d0 0x7ff2dc137130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2dc15e6b0 msgr2=0x7ff2dc152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2dc15e6b0 0x7ff2dc152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 -- 172.21.15.113:0/1608496441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc13a9e0 con 0x7ff2dc0b3080 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e17f9640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc0b3080 0x7ff2dc14ff50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009870 tx=0x7ff2d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2d1ffb640 1 -- 172.21.15.113:0/1608496441 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8037040 con 0x7ff2dc0b3080 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2d1ffb640 1 -- 172.21.15.113:0/1608496441 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d8004230 con 0x7ff2dc0b3080 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2d1ffb640 1 -- 172.21.15.113:0/1608496441 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8003610 con 0x7ff2dc0b3080 2024-04-25T00:08:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.355+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc14f190 con 0x7ff2dc0b3080 2024-04-25T00:08:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.356+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc14fa30 con 0x7ff2dc0b3080 2024-04-25T00:08:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.357+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc14f360 con 0x7ff2dc0b3080 2024-04-25T00:08:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.357+0000 7ff2d1ffb640 1 -- 172.21.15.113:0/1608496441 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2d802ad10 con 0x7ff2dc0b3080 2024-04-25T00:08:58.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.357+0000 7ff2d1ffb640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2c006be30 0x7ff2c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:58.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.357+0000 7ff2d1ffb640 1 -- 172.21.15.113:0/1608496441 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5795+0+0 (secure 0 0 0) 0x7ff2d80ab7b0 con 0x7ff2dc0b3080 2024-04-25T00:08:58.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.358+0000 7ff2e0ff8640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2c006be30 0x7ff2c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:58.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.358+0000 7ff2e0ff8640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2c006be30 0x7ff2c006e2f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0a9cb0 tx=0x7ff2cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:58.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.362+0000 7ff2d1ffb640 1 -- 172.21.15.113:0/1608496441 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff2d8079c60 con 0x7ff2dc0b3080 2024-04-25T00:08:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:58.572+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff2dc0691a0 con 0x7ff2dc0b3080 2024-04-25T00:08:59.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.238+0000 7ff2d1ffb640 1 -- 172.21.15.113:0/1608496441 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) ==== 104+0+0 (secure 0 0 0) 0x7ff2d807df60 con 0x7ff2dc0b3080 2024-04-25T00:08:59.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2c006be30 msgr2=0x7ff2c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:59.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2c006be30 0x7ff2c006e2f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0a9cb0 tx=0x7ff2cc008040 comp rx=0 tx=0).stop 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc0b3080 msgr2=0x7ff2dc14ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc0b3080 0x7ff2dc14ff50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009870 tx=0x7ff2d80046c0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 shutdown_connections 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff2c006be30 0x7ff2c006e2f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2dc15e6b0 0x7ff2dc152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2dc15e2d0 0x7ff2dc137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 --2- 172.21.15.113:0/1608496441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2dc0b3080 0x7ff2dc14ff50 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 >> 172.21.15.113:0/1608496441 conn(0x7ff2dc06b0f0 msgr2=0x7ff2dc0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 shutdown_connections 2024-04-25T00:08:59.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.242+0000 7ff2e3a5d640 1 -- 172.21.15.113:0/1608496441 wait complete. 2024-04-25T00:08:59.255 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:08:59.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.358+0000 7f06976b3640 1 Processor -- start 2024-04-25T00:08:59.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.358+0000 7f06976b3640 1 -- start start 2024-04-25T00:08:59.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f06976b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0690145890 0x7f06900ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:59.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f06976b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06900ced20 0x7f069006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:59.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f06976b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f069006e610 0x7f069015fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:59.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f06976b3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0690145c70 con 0x7f06900ced20 2024-04-25T00:08:59.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f06976b3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f069006ea40 con 0x7f0690145890 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f06976b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06900cf3a0 con 0x7f069006e610 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06900ced20 0x7f069006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06900ced20 0x7f069006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36678/0 (socket says 172.21.15.113:36678) 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 -- 172.21.15.113:0/2094942962 learned_addr learned my addr 172.21.15.113:0/2094942962 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 -- 172.21.15.113:0/2094942962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f069006e610 msgr2=0x7f069015fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 --2- 172.21.15.113:0/2094942962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f069006e610 0x7f069015fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 -- 172.21.15.113:0/2094942962 >> v2:172.21.15.178:3300/0 conn(0x7f0690145890 msgr2=0x7f06900ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 --2- 172.21.15.113:0/2094942962 >> v2:172.21.15.178:3300/0 conn(0x7f0690145890 0x7f06900ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.359+0000 7f0694c4e640 1 -- 172.21.15.113:0/2094942962 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06900cf680 con 0x7f06900ced20 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.360+0000 7f0694c4e640 1 --2- 172.21.15.113:0/2094942962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900ced20 0x7f069006e0d0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f069006c0d0 tx=0x7f068002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327163211a814194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.360+0000 7f0687fff640 1 -- 172.21.15.113:0/2094942962 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680037070 con 0x7f06900ced20 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.360+0000 7f0687fff640 1 -- 172.21.15.113:0/2094942962 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0680032440 con 0x7f06900ced20 2024-04-25T00:08:59.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.360+0000 7f0687fff640 1 -- 172.21.15.113:0/2094942962 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680032810 con 0x7f06900ced20 2024-04-25T00:08:59.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 -- 172.21.15.113:0/2094942962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900ced20 msgr2=0x7f069006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:59.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 --2- 172.21.15.113:0/2094942962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900ced20 0x7f069006e0d0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f069006c0d0 tx=0x7f068002a210 comp rx=0 tx=0).stop 2024-04-25T00:08:59.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 -- 172.21.15.113:0/2094942962 shutdown_connections 2024-04-25T00:08:59.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 --2- 172.21.15.113:0/2094942962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f069006e610 0x7f069015fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 --2- 172.21.15.113:0/2094942962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06900ced20 0x7f069006e0d0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 --2- 172.21.15.113:0/2094942962 >> v2:172.21.15.178:3300/0 conn(0x7f0690145890 0x7f06900ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 -- 172.21.15.113:0/2094942962 >> 172.21.15.113:0/2094942962 conn(0x7f06900b1b30 msgr2=0x7f06900b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 -- 172.21.15.113:0/2094942962 shutdown_connections 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.361+0000 7f06976b3640 1 -- 172.21.15.113:0/2094942962 wait complete. 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.362+0000 7f06976b3640 1 Processor -- start 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.362+0000 7f06976b3640 1 -- start start 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069006e610 0x7f0690133750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900ced20 0x7f06900aa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0690145890 0x7f0690142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06900b53b0 con 0x7f0690145890 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0690070f10 con 0x7f069006e610 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0690160130 con 0x7f06900ced20 2024-04-25T00:08:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0690145890 0x7f0690142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0690145890 0x7f0690142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36684/0 (socket says 172.21.15.113:36684) 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 -- 172.21.15.113:0/2729487384 learned_addr learned my addr 172.21.15.113:0/2729487384 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 -- 172.21.15.113:0/2729487384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900ced20 msgr2=0x7f06900aa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900ced20 0x7f06900aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 -- 172.21.15.113:0/2729487384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069006e610 msgr2=0x7f0690133750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069006e610 0x7f0690133750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 -- 172.21.15.113:0/2729487384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069015d6e0 con 0x7f0690145890 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0695c50640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0690145890 0x7f0690142550 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f068800b9a0 tx=0x7f068800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0685ffb640 1 -- 172.21.15.113:0/2729487384 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688019070 con 0x7f0690145890 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0685ffb640 1 -- 172.21.15.113:0/2729487384 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068800cbc0 con 0x7f0690145890 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069015d9c0 con 0x7f0690145890 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f0685ffb640 1 -- 172.21.15.113:0/2729487384 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688014440 con 0x7f0690145890 2024-04-25T00:08:59.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.363+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069015df80 con 0x7f0690145890 2024-04-25T00:08:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.365+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06900691a0 con 0x7f0690145890 2024-04-25T00:08:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.365+0000 7f0685ffb640 1 -- 172.21.15.113:0/2729487384 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f068801c070 con 0x7f0690145890 2024-04-25T00:08:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.365+0000 7f0685ffb640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be30 0x7f067406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.365+0000 7f0685ffb640 1 -- 172.21.15.113:0/2729487384 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5795+0+0 (secure 0 0 0) 0x7f068808ab80 con 0x7f0690145890 2024-04-25T00:08:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.365+0000 7f069544f640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be30 0x7f067406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.365+0000 7f069544f640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be30 0x7f067406e2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f068c009670 tx=0x7f068c006bf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:59.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.369+0000 7f0685ffb640 1 -- 172.21.15.113:0/2729487384 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0688059560 con 0x7f0690145890 2024-04-25T00:08:59.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:08:59.540+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0690142cc0 con 0x7f0690145890 2024-04-25T00:09:00.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.243+0000 7f0685ffb640 1 -- 172.21.15.113:0/2729487384 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v160) ==== 181+0+0 (secure 0 0 0) 0x7f0688058e60 con 0x7f0690145890 2024-04-25T00:09:00.243 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be30 msgr2=0x7f067406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be30 0x7f067406e2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f068c009670 tx=0x7f068c006bf0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0690145890 msgr2=0x7f0690142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0690145890 0x7f0690142550 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f068800b9a0 tx=0x7f068800be60 comp rx=0 tx=0).stop 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 shutdown_connections 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f067406be30 0x7f067406e2f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0690145890 0x7f0690142550 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06900ced20 0x7f06900aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 --2- 172.21.15.113:0/2729487384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f069006e610 0x7f0690133750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.247+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 >> 172.21.15.113:0/2729487384 conn(0x7f06900b1b30 msgr2=0x7f06900741e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:00.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.248+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 shutdown_connections 2024-04-25T00:09:00.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.248+0000 7f06976b3640 1 -- 172.21.15.113:0/2729487384 wait complete. 2024-04-25T00:09:00.263 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:09:00.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.368+0000 7ff1b23c1640 1 Processor -- start 2024-04-25T00:09:00.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.368+0000 7ff1b23c1640 1 -- start start 2024-04-25T00:09:00.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.369+0000 7ff1b23c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.369+0000 7ff1b23c1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1ac0b61e0 0x7ff1ac0d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.369+0000 7ff1b23c1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1ac1626f0 con 0x7ff1ac15e510 2024-04-25T00:09:00.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.369+0000 7ff1b23c1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff1ac1629d0 con 0x7ff1ac0b61e0 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.369+0000 7ff1b23c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac162cb0 con 0x7ff1ac0b9bb0 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1ab7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1b095e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1ac0b61e0 0x7ff1ac0d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1ab7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:39642/0 (socket says 172.21.15.113:39642) 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1ab7fe640 1 -- 172.21.15.113:0/455451338 learned_addr learned my addr 172.21.15.113:0/455451338 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1ab7fe640 1 -- 172.21.15.113:0/455451338 >> v2:172.21.15.178:3300/0 conn(0x7ff1ac0b61e0 msgr2=0x7ff1ac0d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1ab7fe640 1 --2- 172.21.15.113:0/455451338 >> v2:172.21.15.178:3300/0 conn(0x7ff1ac0b61e0 0x7ff1ac0d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1ab7fe640 1 -- 172.21.15.113:0/455451338 >> v1:172.21.15.178:6789/0 conn(0x7ff1ac15e510 legacy=0x7ff1ac161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1ab7fe640 1 -- 172.21.15.113:0/455451338 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac0d2660 con 0x7ff1ac0b9bb0 2024-04-25T00:09:00.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.370+0000 7ff1b095e640 1 --2- 172.21.15.113:0/455451338 >> v2:172.21.15.178:3300/0 conn(0x7ff1ac0b61e0 0x7ff1ac0d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1ab7fe640 1 --2- 172.21.15.113:0/455451338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0b5ca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009870 tx=0x7ff1a002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a388d4734a46f72d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1aaffd640 1 -- 172.21.15.113:0/455451338 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0037070 con 0x7ff1ac0b9bb0 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1aaffd640 1 -- 172.21.15.113:0/455451338 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a0032440 con 0x7ff1ac0b9bb0 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1aaffd640 1 -- 172.21.15.113:0/455451338 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0032810 con 0x7ff1ac0b9bb0 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1b23c1640 1 -- 172.21.15.113:0/455451338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 msgr2=0x7ff1ac0b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/455451338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0b5ca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009870 tx=0x7ff1a002a1c0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1b23c1640 1 -- 172.21.15.113:0/455451338 shutdown_connections 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/455451338 >> v2:172.21.15.178:3300/0 conn(0x7ff1ac0b61e0 0x7ff1ac0d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/455451338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0b5ca0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.371+0000 7ff1b23c1640 1 -- 172.21.15.113:0/455451338 >> 172.21.15.113:0/455451338 conn(0x7ff1ac06b720 msgr2=0x7ff1ac0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.372+0000 7ff1b23c1640 1 -- 172.21.15.113:0/455451338 shutdown_connections 2024-04-25T00:09:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.372+0000 7ff1b23c1640 1 -- 172.21.15.113:0/455451338 wait complete. 2024-04-25T00:09:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 Processor -- start 2024-04-25T00:09:00.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 -- start start 2024-04-25T00:09:00.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1ac0b61e0 0x7ff1ac161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1ac15e510 0x7ff1ac0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac0b7540 con 0x7ff1ac15e510 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac0b73c0 con 0x7ff1ac0b61e0 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.373+0000 7ff1b23c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac0d33a0 con 0x7ff1ac0b9bb0 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1ac15e510 0x7ff1ac0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1ac15e510 0x7ff1ac0c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36694/0 (socket says 172.21.15.113:36694) 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 -- 172.21.15.113:0/2610529161 learned_addr learned my addr 172.21.15.113:0/2610529161 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1ab7fe640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1abfff640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1ac0b61e0 0x7ff1ac161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 -- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 msgr2=0x7ff1ac0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:00.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 -- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1ac0b61e0 msgr2=0x7ff1ac161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1ac0b61e0 0x7ff1ac161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 -- 172.21.15.113:0/2610529161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac0c9aa0 con 0x7ff1ac15e510 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1ab7fe640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b095e640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1ac15e510 0x7ff1ac0c9560 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009f00 tx=0x7ff1a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1a8ff9640 1 -- 172.21.15.113:0/2610529161 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4018030 con 0x7ff1ac15e510 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1a8ff9640 1 -- 172.21.15.113:0/2610529161 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a400ce50 con 0x7ff1ac15e510 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac0c84c0 con 0x7ff1ac15e510 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1a8ff9640 1 -- 172.21.15.113:0/2610529161 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4003bb0 con 0x7ff1ac15e510 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.374+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac0c8a80 con 0x7ff1ac15e510 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.375+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ac0691a0 con 0x7ff1ac15e510 2024-04-25T00:09:00.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.376+0000 7ff1a8ff9640 1 -- 172.21.15.113:0/2610529161 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1a4005d10 con 0x7ff1ac15e510 2024-04-25T00:09:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.376+0000 7ff1a8ff9640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff19806be30 0x7ff19806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.376+0000 7ff1a8ff9640 1 -- 172.21.15.113:0/2610529161 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5357+0+0 (secure 0 0 0) 0x7ff1a408b690 con 0x7ff1ac15e510 2024-04-25T00:09:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.376+0000 7ff1abfff640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff19806be30 0x7ff19806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.377+0000 7ff1abfff640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff19806be30 0x7ff19806e2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0016e0 tx=0x7ff19c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:00.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.381+0000 7ff1a8ff9640 1 -- 172.21.15.113:0/2610529161 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1a4022070 con 0x7ff1ac15e510 2024-04-25T00:09:00.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:00.555+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff1ac0b76c0 con 0x7ff1ac15e510 2024-04-25T00:09:01.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.244+0000 7ff1a8ff9640 1 -- 172.21.15.113:0/2610529161 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v161) ==== 169+0+0 (secure 0 0 0) 0x7ff1a4010040 con 0x7ff1ac15e510 2024-04-25T00:09:01.246 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:09:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.248+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff19806be30 msgr2=0x7ff19806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.248+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff19806be30 0x7ff19806e2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0016e0 tx=0x7ff19c009040 comp rx=0 tx=0).stop 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.248+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1ac15e510 msgr2=0x7ff1ac0c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.248+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1ac15e510 0x7ff1ac0c9560 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009f00 tx=0x7ff1a400c490 comp rx=0 tx=0).stop 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.250+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 shutdown_connections 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.250+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff19806be30 0x7ff19806e2f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.250+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1ac15e510 0x7ff1ac0c9560 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.250+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1ac0b9bb0 0x7ff1ac0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.250+0000 7ff1b23c1640 1 --2- 172.21.15.113:0/2610529161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1ac0b61e0 0x7ff1ac161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.250+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 >> 172.21.15.113:0/2610529161 conn(0x7ff1ac06b720 msgr2=0x7ff1ac0742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:01.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.252+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 shutdown_connections 2024-04-25T00:09:01.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.252+0000 7ff1b23c1640 1 -- 172.21.15.113:0/2610529161 wait complete. 2024-04-25T00:09:01.267 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:09:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.366+0000 7fee6806d640 1 Processor -- start 2024-04-25T00:09:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.366+0000 7fee6806d640 1 -- start start 2024-04-25T00:09:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee6806d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee6015e510 0x7fee600d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee6806d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee600d28c0 0x7fee600b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee6806d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee600d2e00 con 0x7fee600b61e0 2024-04-25T00:09:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee6806d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fee600d30e0 con 0x7fee6015e510 2024-04-25T00:09:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee6806d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fee600d33c0 con 0x7fee600d28c0 2024-04-25T00:09:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee65e09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee6015e510 0x7fee600d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee6660a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fee600b61e0 0x7fee60071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33790/0 (socket says 172.21.15.113:33790) 2024-04-25T00:09:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.367+0000 7fee6660a640 1 -- 172.21.15.113:0/906833805 learned_addr learned my addr 172.21.15.113:0/906833805 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee65e09640 1 -- 172.21.15.113:0/906833805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee600d28c0 msgr2=0x7fee600b5ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:09:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee65e09640 1 --2- 172.21.15.113:0/906833805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee600d28c0 0x7fee600b5ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee65e09640 1 -- 172.21.15.113:0/906833805 >> v1:172.21.15.113:6789/0 conn(0x7fee600b61e0 legacy=0x7fee60071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee65e09640 1 -- 172.21.15.113:0/906833805 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee600d36a0 con 0x7fee6015e510 2024-04-25T00:09:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee64e07640 1 -- 172.21.15.113:0/906833805 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888656108 0 0) 0x7fee600d2e00 con 0x7fee600b61e0 2024-04-25T00:09:01.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee65e09640 1 --2- 172.21.15.113:0/906833805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6015e510 0x7fee600d2380 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9793394426df0253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee64e07640 1 -- 172.21.15.113:0/906833805 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007740 con 0x7fee6015e510 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.368+0000 7fee64e07640 1 -- 172.21.15.113:0/906833805 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee5c0078e0 con 0x7fee6015e510 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee64e07640 1 -- 172.21.15.113:0/906833805 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007cb0 con 0x7fee6015e510 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 -- 172.21.15.113:0/906833805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6015e510 msgr2=0x7fee600d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 --2- 172.21.15.113:0/906833805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6015e510 0x7fee600d2380 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c02a5f0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 -- 172.21.15.113:0/906833805 shutdown_connections 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 --2- 172.21.15.113:0/906833805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee600d28c0 0x7fee600b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 --2- 172.21.15.113:0/906833805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee6015e510 0x7fee600d2380 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 -- 172.21.15.113:0/906833805 >> 172.21.15.113:0/906833805 conn(0x7fee6006b720 msgr2=0x7fee600d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 -- 172.21.15.113:0/906833805 shutdown_connections 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.369+0000 7fee6806d640 1 -- 172.21.15.113:0/906833805 wait complete. 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.370+0000 7fee6806d640 1 Processor -- start 2024-04-25T00:09:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.370+0000 7fee6806d640 1 -- start start 2024-04-25T00:09:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee6806d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee600b61e0 0x7fee60146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee6806d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee600d28c0 0x7fee600c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee6806d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee6015e510 0x7fee600b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee6806d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fee600b9dd0 con 0x7fee600d28c0 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee6806d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee600d3ea0 con 0x7fee600b61e0 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee6806d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fee600b83d0 con 0x7fee6015e510 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee600d28c0 0x7fee600c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee600d28c0 0x7fee600c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36706/0 (socket says 172.21.15.113:36706) 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 -- 172.21.15.113:0/4065535073 learned_addr learned my addr 172.21.15.113:0/4065535073 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 -- 172.21.15.113:0/4065535073 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee6015e510 msgr2=0x7fee600b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee6015e510 0x7fee600b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 -- 172.21.15.113:0/4065535073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee600b61e0 msgr2=0x7fee60146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee600b61e0 0x7fee60146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 -- 172.21.15.113:0/4065535073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee600c8a10 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.371+0000 7fee65608640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee600d28c0 0x7fee600c84d0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b450 tx=0x7fee5000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.372+0000 7fee567fc640 1 -- 172.21.15.113:0/4065535073 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50015020 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.372+0000 7fee567fc640 1 -- 172.21.15.113:0/4065535073 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee500118b0 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.372+0000 7fee567fc640 1 -- 172.21.15.113:0/4065535073 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50011ca0 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.372+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee600c8c90 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.372+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee600c9250 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.373+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee600691a0 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.373+0000 7fee567fc640 1 -- 172.21.15.113:0/4065535073 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee50018020 con 0x7fee600d28c0 2024-04-25T00:09:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.373+0000 7fee567fc640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee4406be30 0x7fee4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.374+0000 7fee65e09640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee4406be30 0x7fee4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.374+0000 7fee567fc640 1 -- 172.21.15.113:0/4065535073 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7fee5008add0 con 0x7fee600d28c0 2024-04-25T00:09:01.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.374+0000 7fee65e09640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee4406be30 0x7fee4406e2f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fee5c034000 tx=0x7fee5c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.378+0000 7fee567fc640 1 -- 172.21.15.113:0/4065535073 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fee5001f020 con 0x7fee600d28c0 2024-04-25T00:09:01.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.559+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fee600b9560 con 0x7fee600d28c0 2024-04-25T00:09:01.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.562+0000 7fee567fc640 1 -- 172.21.15.113:0/4065535073 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fee5008c020 con 0x7fee600d28c0 2024-04-25T00:09:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee4406be30 msgr2=0x7fee4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee4406be30 0x7fee4406e2f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fee5c034000 tx=0x7fee5c02e000 comp rx=0 tx=0).stop 2024-04-25T00:09:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee600d28c0 msgr2=0x7fee600c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee600d28c0 0x7fee600c84d0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b450 tx=0x7fee5000b910 comp rx=0 tx=0).stop 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 shutdown_connections 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fee4406be30 0x7fee4406e2f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee6015e510 0x7fee600b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee600d28c0 0x7fee600c84d0 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 --2- 172.21.15.113:0/4065535073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee600b61e0 0x7fee60146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 >> 172.21.15.113:0/4065535073 conn(0x7fee6006b720 msgr2=0x7fee600d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 shutdown_connections 2024-04-25T00:09:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.565+0000 7fee6806d640 1 -- 172.21.15.113:0/4065535073 wait complete. 2024-04-25T00:09:01.576 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T00:09:01.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.681+0000 7f7632415640 1 Processor -- start 2024-04-25T00:09:01.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.681+0000 7f7632415640 1 -- start start 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.681+0000 7f7632415640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f762c144910 0x7f762c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.681+0000 7f7632415640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b56b0 0x7f762c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.681+0000 7f7632415640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f762c144cf0 con 0x7f762c0b5240 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.681+0000 7f7632415640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f762c0b4510 con 0x7f762c144910 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.681+0000 7f7632415640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f762c0b2f30 con 0x7f762c0b56b0 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f762c144910 0x7f762c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762b7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f762c0b5240 0x7f762c06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33796/0 (socket says 172.21.15.113:33796) 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762b7fe640 1 -- 172.21.15.113:0/2982459232 learned_addr learned my addr 172.21.15.113:0/2982459232 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762bfff640 1 -- 172.21.15.113:0/2982459232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b56b0 msgr2=0x7f762c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762bfff640 1 --2- 172.21.15.113:0/2982459232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b56b0 0x7f762c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762bfff640 1 -- 172.21.15.113:0/2982459232 >> v1:172.21.15.113:6789/0 conn(0x7f762c0b5240 legacy=0x7f762c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762bfff640 1 -- 172.21.15.113:0/2982459232 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c0b3210 con 0x7f762c144910 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.682+0000 7f762affd640 1 -- 172.21.15.113:0/2982459232 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172704364 0 0) 0x7f762c144cf0 con 0x7f762c0b5240 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f762bfff640 1 --2- 172.21.15.113:0/2982459232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0b3fd0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f761c0097d0 tx=0x7f761c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1cd69ece2e612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f762affd640 1 -- 172.21.15.113:0/2982459232 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c037070 con 0x7f762c144910 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f762affd640 1 -- 172.21.15.113:0/2982459232 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761c032440 con 0x7f762c144910 2024-04-25T00:09:01.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f762affd640 1 -- 172.21.15.113:0/2982459232 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c032810 con 0x7f762c144910 2024-04-25T00:09:01.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 -- 172.21.15.113:0/2982459232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 msgr2=0x7f762c0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 --2- 172.21.15.113:0/2982459232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0b3fd0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f761c0097d0 tx=0x7f761c02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 -- 172.21.15.113:0/2982459232 shutdown_connections 2024-04-25T00:09:01.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 --2- 172.21.15.113:0/2982459232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b56b0 0x7f762c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 --2- 172.21.15.113:0/2982459232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0b3fd0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 -- 172.21.15.113:0/2982459232 >> 172.21.15.113:0/2982459232 conn(0x7f762c0b1a50 msgr2=0x7f762c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:01.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 -- 172.21.15.113:0/2982459232 shutdown_connections 2024-04-25T00:09:01.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.683+0000 7f7632415640 1 -- 172.21.15.113:0/2982459232 wait complete. 2024-04-25T00:09:01.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.684+0000 7f7632415640 1 Processor -- start 2024-04-25T00:09:01.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.685+0000 7f7632415640 1 -- start start 2024-04-25T00:09:01.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.685+0000 7f7632415640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b5240 0x7f762c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.685+0000 7f7632415640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f762c0b56b0 0x7f762c13eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.685+0000 7f7632415640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.685+0000 7f7632415640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f762c0b4da0 con 0x7f762c144910 2024-04-25T00:09:01.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.685+0000 7f7632415640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f762c0b37d0 con 0x7f762c0b56b0 2024-04-25T00:09:01.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.685+0000 7f7632415640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f762c0b4f20 con 0x7f762c0b5240 2024-04-25T00:09:01.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36734/0 (socket says 172.21.15.113:36734) 2024-04-25T00:09:01.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 -- 172.21.15.113:0/3569322665 learned_addr learned my addr 172.21.15.113:0/3569322665 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:01.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f762bfff640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b5240 0x7f762c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 -- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b5240 msgr2=0x7f762c15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b5240 0x7f762c15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 -- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f762c0b56b0 msgr2=0x7f762c13eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f762b7fe640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f762c0b56b0 0x7f762c13eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f762c0b56b0 0x7f762c13eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 -- 172.21.15.113:0/3569322665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c15c7f0 con 0x7f762c144910 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f76309b2640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0ac190 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f762400db30 tx=0x7f762400e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f7628ff9640 1 -- 172.21.15.113:0/3569322665 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400e8c0 con 0x7f762c144910 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f7628ff9640 1 -- 172.21.15.113:0/3569322665 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f762400ea80 con 0x7f762c144910 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762c15cad0 con 0x7f762c144910 2024-04-25T00:09:01.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f7628ff9640 1 -- 172.21.15.113:0/3569322665 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624009550 con 0x7f762c144910 2024-04-25T00:09:01.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.686+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762c0a9980 con 0x7f762c144910 2024-04-25T00:09:01.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.687+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762c0691a0 con 0x7f762c144910 2024-04-25T00:09:01.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.688+0000 7f7628ff9640 1 -- 172.21.15.113:0/3569322665 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76240096f0 con 0x7f762c144910 2024-04-25T00:09:01.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.688+0000 7f7628ff9640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f761806be30 0x7f761806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.688+0000 7f7628ff9640 1 -- 172.21.15.113:0/3569322665 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f762408c7d0 con 0x7f762c144910 2024-04-25T00:09:01.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.688+0000 7f762bfff640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f761806be30 0x7f761806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.689+0000 7f762bfff640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f761806be30 0x7f761806e2f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f761c034000 tx=0x7f761c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.692+0000 7f7628ff9640 1 -- 172.21.15.113:0/3569322665 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f762405b480 con 0x7f762c144910 2024-04-25T00:09:01.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:01.937+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) -- 0x7f762c0af9b0 con 0x7f762c144910 2024-04-25T00:09:02.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.245+0000 7f7628ff9640 1 -- 172.21.15.113:0/3569322665 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v520) ==== 133+0+0 (secure 0 0 0) 0x7f762405ad80 con 0x7f762c144910 2024-04-25T00:09:02.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f761806be30 msgr2=0x7f761806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f761806be30 0x7f761806e2f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f761c034000 tx=0x7f761c02e000 comp rx=0 tx=0).stop 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 msgr2=0x7f762c0ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0ac190 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f762400db30 tx=0x7f762400e0c0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 shutdown_connections 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f761806be30 0x7f761806e2f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f762c0b56b0 0x7f762c13eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f762c144910 0x7f762c0ac190 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 --2- 172.21.15.113:0/3569322665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f762c0b5240 0x7f762c15bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 >> 172.21.15.113:0/3569322665 conn(0x7f762c0b1a50 msgr2=0x7f762c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 shutdown_connections 2024-04-25T00:09:02.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.248+0000 7f7632415640 1 -- 172.21.15.113:0/3569322665 wait complete. 2024-04-25T00:09:02.259 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T00:09:02.260 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T00:09:02.260 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:02.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.357+0000 7f319bdb7640 1 Processor -- start 2024-04-25T00:09:02.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.357+0000 7f319bdb7640 1 -- start start 2024-04-25T00:09:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319bdb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3194072ea0 0x7f3194071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319bdb7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31940b3af0 0x7f31940b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319bdb7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31940b2d50 con 0x7f319415a6b0 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319bdb7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3194072a70 con 0x7f31940b3af0 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319bdb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31940b3010 con 0x7f3194072ea0 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f3199b53640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f319415a6b0 0x7f3194070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33806/0 (socket says 172.21.15.113:33806) 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319a354640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31940b3af0 0x7f31940b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f3199b53640 1 -- 172.21.15.113:0/3832398611 learned_addr learned my addr 172.21.15.113:0/3832398611 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319a354640 1 -- 172.21.15.113:0/3832398611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3194072ea0 msgr2=0x7f3194071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f3199352640 1 --2- 172.21.15.113:0/3832398611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3194072ea0 0x7f3194071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319a354640 1 --2- 172.21.15.113:0/3832398611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3194072ea0 0x7f3194071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319a354640 1 -- 172.21.15.113:0/3832398611 >> v1:172.21.15.113:6789/0 conn(0x7f319415a6b0 legacy=0x7f3194070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f3198b51640 1 -- 172.21.15.113:0/3832398611 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1148911594 0 0) 0x7f31940b2d50 con 0x7f319415a6b0 2024-04-25T00:09:02.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f319a354640 1 -- 172.21.15.113:0/3832398611 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31940b32f0 con 0x7f31940b3af0 2024-04-25T00:09:02.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.358+0000 7f3199352640 1 --2- 172.21.15.113:0/3832398611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3194072ea0 0x7f3194071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f319a354640 1 --2- 172.21.15.113:0/3832398611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31940b3f80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f318c00aea0 tx=0x7f318c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ef975cb3b6bd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f3198b51640 1 -- 172.21.15.113:0/3832398611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c037070 con 0x7f31940b3af0 2024-04-25T00:09:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f3198b51640 1 -- 172.21.15.113:0/3832398611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c008640 con 0x7f31940b3af0 2024-04-25T00:09:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f3198b51640 1 -- 172.21.15.113:0/3832398611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c008a10 con 0x7f31940b3af0 2024-04-25T00:09:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f319bdb7640 1 -- 172.21.15.113:0/3832398611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 msgr2=0x7f31940b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f319bdb7640 1 --2- 172.21.15.113:0/3832398611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31940b3f80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f318c00aea0 tx=0x7f318c02b7b0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f319bdb7640 1 -- 172.21.15.113:0/3832398611 shutdown_connections 2024-04-25T00:09:02.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f319bdb7640 1 --2- 172.21.15.113:0/3832398611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31940b3f80 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f319bdb7640 1 --2- 172.21.15.113:0/3832398611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3194072ea0 0x7f3194071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.359+0000 7f319bdb7640 1 -- 172.21.15.113:0/3832398611 >> 172.21.15.113:0/3832398611 conn(0x7f31940b1a50 msgr2=0x7f31940b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:02.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.360+0000 7f319bdb7640 1 -- 172.21.15.113:0/3832398611 shutdown_connections 2024-04-25T00:09:02.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.360+0000 7f319bdb7640 1 -- 172.21.15.113:0/3832398611 wait complete. 2024-04-25T00:09:02.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 Processor -- start 2024-04-25T00:09:02.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 -- start start 2024-04-25T00:09:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3194072ea0 0x7f3194141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31941dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f319415a6b0 0x7f31941e0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f319415aa90 con 0x7f31940b3af0 2024-04-25T00:09:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3194072720 con 0x7f3194072ea0 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f319bdb7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f319415ac10 con 0x7f319415a6b0 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f3199352640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31941dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f3199352640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31941dfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36746/0 (socket says 172.21.15.113:36746) 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.361+0000 7f3199352640 1 -- 172.21.15.113:0/2406102443 learned_addr learned my addr 172.21.15.113:0/2406102443 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f3199352640 1 -- 172.21.15.113:0/2406102443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f319415a6b0 msgr2=0x7f31941e0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f3199352640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f319415a6b0 0x7f31941e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f3199352640 1 -- 172.21.15.113:0/2406102443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3194072ea0 msgr2=0x7f3194141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f3199352640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3194072ea0 0x7f3194141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f3199352640 1 -- 172.21.15.113:0/2406102443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3194145680 con 0x7f31940b3af0 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f3199352640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31941dfee0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f318400c7f0 tx=0x7f318400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f318a7fc640 1 -- 172.21.15.113:0/2406102443 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184015070 con 0x7f31940b3af0 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f318a7fc640 1 -- 172.21.15.113:0/2406102443 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3184007a30 con 0x7f31940b3af0 2024-04-25T00:09:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31941427a0 con 0x7f31940b3af0 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f318a7fc640 1 -- 172.21.15.113:0/2406102443 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184007e00 con 0x7f31940b3af0 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.362+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3194142d60 con 0x7f31940b3af0 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.363+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31940691a0 con 0x7f31940b3af0 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.363+0000 7f318a7fc640 1 -- 172.21.15.113:0/2406102443 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3184018070 con 0x7f31940b3af0 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.364+0000 7f318a7fc640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f317806be60 0x7f317806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.364+0000 7f318a7fc640 1 -- 172.21.15.113:0/2406102443 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f318408aa70 con 0x7f31940b3af0 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.364+0000 7f3199b53640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f317806be60 0x7f317806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.364+0000 7f3199b53640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f317806be60 0x7f317806e320 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3190007fd0 tx=0x7f3190007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.368+0000 7f318a7fc640 1 -- 172.21.15.113:0/2406102443 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3184059720 con 0x7f31940b3af0 2024-04-25T00:09:02.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.545+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3194071d70 con 0x7f31940b3af0 2024-04-25T00:09:02.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.546+0000 7f318a7fc640 1 -- 172.21.15.113:0/2406102443 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+13078 (secure 0 0 0) 0x7f3184059020 con 0x7f31940b3af0 2024-04-25T00:09:02.547 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:02.547 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":161,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:01.244004+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6834/340854702":"2024-04-26T00:08:58.236161+0000","172.21.15.178:6832/340854702":"2024-04-26T00:08:58.236161+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f317806be60 msgr2=0x7f317806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f317806be60 0x7f317806e320 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3190007fd0 tx=0x7f3190007960 comp rx=0 tx=0).stop 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 msgr2=0x7f31941dfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31941dfee0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f318400c7f0 tx=0x7f318400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 shutdown_connections 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f317806be60 0x7f317806e320 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f319415a6b0 0x7f31941e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31940b3af0 0x7f31941dfee0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 --2- 172.21.15.113:0/2406102443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3194072ea0 0x7f3194141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 >> 172.21.15.113:0/2406102443 conn(0x7f31940b1a50 msgr2=0x7f31940b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:02.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 shutdown_connections 2024-04-25T00:09:02.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.548+0000 7f319bdb7640 1 -- 172.21.15.113:0/2406102443 wait complete. 2024-04-25T00:09:02.560 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:09:02.560 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:09:02.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.659+0000 7fcf56d1d640 1 Processor -- start 2024-04-25T00:09:02.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.659+0000 7fcf56d1d640 1 -- start start 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf56d1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf5013f050 0x7fcf500cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf56d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf501606a0 0x7fcf500cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf56d1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf5015d6b0 con 0x7fcf500d04f0 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf56d1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcf500d09a0 con 0x7fcf5013f050 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf56d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5015c6a0 con 0x7fcf501606a0 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf54ab9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf5013f050 0x7fcf500cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf552ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf501606a0 0x7fcf500cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf4ffff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fcf500d04f0 0x7fcf5015ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:35078/0 (socket says 172.21.15.113:35078) 2024-04-25T00:09:02.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf4ffff640 1 -- 172.21.15.113:0/4224139807 learned_addr learned my addr 172.21.15.113:0/4224139807 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf4f7fe640 1 -- 172.21.15.113:0/4224139807 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2952020999 0 0) 0x7fcf5015d6b0 con 0x7fcf500d04f0 2024-04-25T00:09:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf4f7fe640 1 -- 172.21.15.113:0/4224139807 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf40003520 con 0x7fcf500d04f0 2024-04-25T00:09:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf552ba640 1 -- 172.21.15.113:0/4224139807 >> v2:172.21.15.178:3300/0 conn(0x7fcf5013f050 msgr2=0x7fcf500cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf552ba640 1 --2- 172.21.15.113:0/4224139807 >> v2:172.21.15.178:3300/0 conn(0x7fcf5013f050 0x7fcf500cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf552ba640 1 -- 172.21.15.113:0/4224139807 >> v1:172.21.15.178:6789/0 conn(0x7fcf500d04f0 legacy=0x7fcf5015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf552ba640 1 -- 172.21.15.113:0/4224139807 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5015c8f0 con 0x7fcf501606a0 2024-04-25T00:09:02.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.660+0000 7fcf54ab9640 1 --2- 172.21.15.113:0/4224139807 >> v2:172.21.15.178:3300/0 conn(0x7fcf5013f050 0x7fcf500cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:02.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.661+0000 7fcf552ba640 1 --2- 172.21.15.113:0/4224139807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf501606a0 0x7fcf500cea60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009d90 tx=0x7fcf4802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd38d435672284c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.661+0000 7fcf4f7fe640 1 -- 172.21.15.113:0/4224139807 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48031600 con 0x7fcf501606a0 2024-04-25T00:09:02.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.661+0000 7fcf4f7fe640 1 -- 172.21.15.113:0/4224139807 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf480317a0 con 0x7fcf501606a0 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.661+0000 7fcf4f7fe640 1 -- 172.21.15.113:0/4224139807 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48031b50 con 0x7fcf501606a0 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 -- 172.21.15.113:0/4224139807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf501606a0 msgr2=0x7fcf500cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/4224139807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf501606a0 0x7fcf500cea60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009d90 tx=0x7fcf4802a5d0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 -- 172.21.15.113:0/4224139807 shutdown_connections 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/4224139807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf501606a0 0x7fcf500cea60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/4224139807 >> v2:172.21.15.178:3300/0 conn(0x7fcf5013f050 0x7fcf500cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 -- 172.21.15.113:0/4224139807 >> 172.21.15.113:0/4224139807 conn(0x7fcf5006b0f0 msgr2=0x7fcf5006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 -- 172.21.15.113:0/4224139807 shutdown_connections 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.662+0000 7fcf56d1d640 1 -- 172.21.15.113:0/4224139807 wait complete. 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.663+0000 7fcf56d1d640 1 Processor -- start 2024-04-25T00:09:02.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.663+0000 7fcf56d1d640 1 -- start start 2024-04-25T00:09:02.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf56d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf500d04f0 0x7fcf5014c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf56d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf5013f050 0x7fcf500b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf56d1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf501606a0 0x7fcf50149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf56d1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5015d190 con 0x7fcf501606a0 2024-04-25T00:09:02.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf56d1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5015ceb0 con 0x7fcf500d04f0 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf56d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5013f430 con 0x7fcf5013f050 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf501606a0 0x7fcf50149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf501606a0 0x7fcf50149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36760/0 (socket says 172.21.15.113:36760) 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 -- 172.21.15.113:0/3812675369 learned_addr learned my addr 172.21.15.113:0/3812675369 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf54ab9640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf500d04f0 0x7fcf5014c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf4ffff640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf5013f050 0x7fcf500b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 -- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf5013f050 msgr2=0x7fcf500b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf5013f050 0x7fcf500b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 -- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf500d04f0 msgr2=0x7fcf5014c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf500d04f0 0x7fcf5014c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 -- 172.21.15.113:0/3812675369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf500b0d40 con 0x7fcf501606a0 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf552ba640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf501606a0 0x7fcf50149840 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009eb0 tx=0x7fcf48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.664+0000 7fcf4d7fa640 1 -- 172.21.15.113:0/3812675369 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4803f070 con 0x7fcf501606a0 2024-04-25T00:09:02.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.665+0000 7fcf4d7fa640 1 -- 172.21.15.113:0/3812675369 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf48030850 con 0x7fcf501606a0 2024-04-25T00:09:02.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.665+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf500b1020 con 0x7fcf501606a0 2024-04-25T00:09:02.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.665+0000 7fcf4d7fa640 1 -- 172.21.15.113:0/3812675369 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48030c40 con 0x7fcf501606a0 2024-04-25T00:09:02.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.665+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf500b15e0 con 0x7fcf501606a0 2024-04-25T00:09:02.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.666+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf500691a0 con 0x7fcf501606a0 2024-04-25T00:09:02.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.666+0000 7fcf4d7fa640 1 -- 172.21.15.113:0/3812675369 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf4803b410 con 0x7fcf501606a0 2024-04-25T00:09:02.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.667+0000 7fcf4d7fa640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf3406be30 0x7fcf3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.667+0000 7fcf4d7fa640 1 -- 172.21.15.113:0/3812675369 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7fcf480abad0 con 0x7fcf501606a0 2024-04-25T00:09:02.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.667+0000 7fcf54ab9640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf3406be30 0x7fcf3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.667+0000 7fcf54ab9640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf3406be30 0x7fcf3406e2f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcf5015c550 tx=0x7fcf44009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.671+0000 7fcf4d7fa640 1 -- 172.21.15.113:0/3812675369 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf4807a140 con 0x7fcf501606a0 2024-04-25T00:09:02.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.843+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf500d0e50 con 0x7fcf501606a0 2024-04-25T00:09:02.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.844+0000 7fcf4d7fa640 1 -- 172.21.15.113:0/3812675369 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) ==== 93+0+10 (secure 0 0 0) 0x7fcf4807e440 con 0x7fcf501606a0 2024-04-25T00:09:02.844 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:09:02.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf3406be30 msgr2=0x7fcf3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf3406be30 0x7fcf3406e2f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcf5015c550 tx=0x7fcf44009190 comp rx=0 tx=0).stop 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf501606a0 msgr2=0x7fcf50149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf501606a0 0x7fcf50149840 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009eb0 tx=0x7fcf48004a40 comp rx=0 tx=0).stop 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 shutdown_connections 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf3406be30 0x7fcf3406e2f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf501606a0 0x7fcf50149840 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf5013f050 0x7fcf500b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 --2- 172.21.15.113:0/3812675369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf500d04f0 0x7fcf5014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 >> 172.21.15.113:0/3812675369 conn(0x7fcf5006b0f0 msgr2=0x7fcf5015ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 shutdown_connections 2024-04-25T00:09:02.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.846+0000 7fcf56d1d640 1 -- 172.21.15.113:0/3812675369 wait complete. 2024-04-25T00:09:02.858 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T00:09:02.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.955+0000 7f848d4da640 1 Processor -- start 2024-04-25T00:09:02.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.955+0000 7f848d4da640 1 -- start start 2024-04-25T00:09:02.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f848d4da640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f848815e590 0x7f84881628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f848d4da640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8488162de0 con 0x7f84880b99e0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f848d4da640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84880b9f30 con 0x7f84880d0ad0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f848d4da640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84880cf040 con 0x7f848815e590 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f84877fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f84880b99e0 0x7f84880d1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33822/0 (socket says 172.21.15.113:33822) 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f84877fe640 1 -- 172.21.15.113:0/1947891119 learned_addr learned my addr 172.21.15.113:0/1947891119 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f847ffff640 1 --2- 172.21.15.113:0/1947891119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f848815e590 0x7f84881628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 845587096 0 0) 0x7f8488162de0 con 0x7f84880b99e0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8474003520 con 0x7f84880b99e0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221144373 0 0) 0x7f84880b9f30 con 0x7f84880d0ad0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8488162de0 con 0x7f84880d0ad0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1460111086 0 0) 0x7f8474003520 con 0x7f84880b99e0 2024-04-25T00:09:02.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f847ffff640 1 -- 172.21.15.113:0/1947891119 >> v1:172.21.15.178:6789/0 conn(0x7f84880d0ad0 legacy=0x7f84880d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f847ffff640 1 -- 172.21.15.113:0/1947891119 >> v1:172.21.15.113:6789/0 conn(0x7f84880b99e0 legacy=0x7f84880d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.956+0000 7f847ffff640 1 -- 172.21.15.113:0/1947891119 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84880ce710 con 0x7f848815e590 2024-04-25T00:09:02.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.957+0000 7f847ffff640 1 --2- 172.21.15.113:0/1947891119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f848815e590 0x7f84881628a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84700097d0 tx=0x7f847002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9750f6ff2bce8b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.957+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470037070 con 0x7f848815e590 2024-04-25T00:09:02.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.957+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847002ac30 con 0x7f848815e590 2024-04-25T00:09:02.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.957+0000 7f84867fc640 1 -- 172.21.15.113:0/1947891119 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800019f0 con 0x7f848815e590 2024-04-25T00:09:02.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.958+0000 7f848d4da640 1 -- 172.21.15.113:0/1947891119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f848815e590 msgr2=0x7f84881628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.958+0000 7f848d4da640 1 --2- 172.21.15.113:0/1947891119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f848815e590 0x7f84881628a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84700097d0 tx=0x7f847002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.958+0000 7f848d4da640 1 -- 172.21.15.113:0/1947891119 shutdown_connections 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.958+0000 7f848d4da640 1 --2- 172.21.15.113:0/1947891119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f848815e590 0x7f84881628a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.958+0000 7f848d4da640 1 -- 172.21.15.113:0/1947891119 >> 172.21.15.113:0/1947891119 conn(0x7f848806b0f0 msgr2=0x7f848806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.958+0000 7f848d4da640 1 -- 172.21.15.113:0/1947891119 shutdown_connections 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.958+0000 7f848d4da640 1 -- 172.21.15.113:0/1947891119 wait complete. 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.959+0000 7f848d4da640 1 Processor -- start 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.959+0000 7f848d4da640 1 -- start start 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84880b99e0 0x7f84880c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84880d0ad0 0x7f84881e8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f848815e590 0x7f848815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f84880cf4d0 con 0x7f84880d0ad0 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84880cecd0 con 0x7f848815e590 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84880cf650 con 0x7f84880b99e0 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84880d0ad0 0x7f84881e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84880d0ad0 0x7f84881e8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36776/0 (socket says 172.21.15.113:36776) 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 -- 172.21.15.113:0/2804662708 learned_addr learned my addr 172.21.15.113:0/2804662708 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f8486ffd640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84880b99e0 0x7f84880c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 -- 172.21.15.113:0/2804662708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84880b99e0 msgr2=0x7f84880c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84880b99e0 0x7f84880c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 -- 172.21.15.113:0/2804662708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f848815e590 msgr2=0x7f848815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f848815e590 0x7f848815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 -- 172.21.15.113:0/2804662708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84880c90f0 con 0x7f84880d0ad0 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847ffff640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84880d0ad0 0x7f84881e8730 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f8470035040 tx=0x7f84700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847f7fe640 1 -- 172.21.15.113:0/2804662708 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470037050 con 0x7f84880d0ad0 2024-04-25T00:09:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847f7fe640 1 -- 172.21.15.113:0/2804662708 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8470043070 con 0x7f84880d0ad0 2024-04-25T00:09:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f847f7fe640 1 -- 172.21.15.113:0/2804662708 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470006e10 con 0x7f84880d0ad0 2024-04-25T00:09:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84880c93d0 con 0x7f84880d0ad0 2024-04-25T00:09:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.960+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84880c9990 con 0x7f84880d0ad0 2024-04-25T00:09:02.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.961+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84880691a0 con 0x7f84880d0ad0 2024-04-25T00:09:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.962+0000 7f847f7fe640 1 -- 172.21.15.113:0/2804662708 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f847003ca30 con 0x7f84880d0ad0 2024-04-25T00:09:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.962+0000 7f847f7fe640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f846806be30 0x7f846806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.962+0000 7f8486ffd640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f846806be30 0x7f846806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.962+0000 7f847f7fe640 1 -- 172.21.15.113:0/2804662708 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f84700ab6d0 con 0x7f84880d0ad0 2024-04-25T00:09:02.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.963+0000 7f8486ffd640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f846806be30 0x7f846806e2f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f84780011e0 tx=0x7f8478009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:02.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:02.966+0000 7f847f7fe640 1 -- 172.21.15.113:0/2804662708 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f847007a530 con 0x7f84880d0ad0 2024-04-25T00:09:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.204+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) -- 0x7f84880d1d60 con 0x7f84880d0ad0 2024-04-25T00:09:03.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.250+0000 7f847f7fe640 1 -- 172.21.15.113:0/2804662708 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v521) ==== 150+0+0 (secure 0 0 0) 0x7f8470079e30 con 0x7f84880d0ad0 2024-04-25T00:09:03.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f846806be30 msgr2=0x7f846806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f846806be30 0x7f846806e2f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f84780011e0 tx=0x7f8478009040 comp rx=0 tx=0).stop 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84880d0ad0 msgr2=0x7f84881e8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84880d0ad0 0x7f84881e8730 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f8470035040 tx=0x7f84700046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 shutdown_connections 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f846806be30 0x7f846806e2f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f848815e590 0x7f848815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84880d0ad0 0x7f84881e8730 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 --2- 172.21.15.113:0/2804662708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84880b99e0 0x7f84880c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 >> 172.21.15.113:0/2804662708 conn(0x7f848806b0f0 msgr2=0x7f84880cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 shutdown_connections 2024-04-25T00:09:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.252+0000 7f848d4da640 1 -- 172.21.15.113:0/2804662708 wait complete. 2024-04-25T00:09:03.263 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:09:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.363+0000 7f4a69c4d640 1 Processor -- start 2024-04-25T00:09:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.363+0000 7f4a69c4d640 1 -- start start 2024-04-25T00:09:03.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a69c4d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 0x7f4a6415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a69c4d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a64071db0 0x7f4a641418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a69c4d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a64071580 con 0x7f4a6406cea0 2024-04-25T00:09:03.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a69c4d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4a64141620 con 0x7f4a64071db0 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a69c4d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6406d3f0 con 0x7f4a640719d0 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a62ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a64071db0 0x7f4a641418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a62ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a64071db0 0x7f4a641418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36786/0 (socket says 172.21.15.113:36786) 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a62ffd640 1 -- 172.21.15.113:0/2125001262 learned_addr learned my addr 172.21.15.113:0/2125001262 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a62ffd640 1 -- 172.21.15.113:0/2125001262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 msgr2=0x7f4a6415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a62ffd640 1 --2- 172.21.15.113:0/2125001262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 0x7f4a6415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a62ffd640 1 -- 172.21.15.113:0/2125001262 >> v1:172.21.15.178:6789/0 conn(0x7f4a6406cea0 legacy=0x7f4a640cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.364+0000 7f4a62ffd640 1 -- 172.21.15.113:0/2125001262 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6406d6d0 con 0x7f4a64071db0 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.365+0000 7f4a62ffd640 1 --2- 172.21.15.113:0/2125001262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a641418b0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009870 tx=0x7f4a5802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3793b59926a0c72e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.365+0000 7f4a627fc640 1 -- 172.21.15.113:0/2125001262 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58037070 con 0x7f4a64071db0 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.365+0000 7f4a627fc640 1 -- 172.21.15.113:0/2125001262 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a58032440 con 0x7f4a64071db0 2024-04-25T00:09:03.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.365+0000 7f4a627fc640 1 -- 172.21.15.113:0/2125001262 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58032810 con 0x7f4a64071db0 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.365+0000 7f4a69c4d640 1 -- 172.21.15.113:0/2125001262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 msgr2=0x7f4a641418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.365+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/2125001262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a641418b0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009870 tx=0x7f4a5802a2a0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.366+0000 7f4a69c4d640 1 -- 172.21.15.113:0/2125001262 shutdown_connections 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.366+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/2125001262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a641418b0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.366+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/2125001262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 0x7f4a6415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.366+0000 7f4a69c4d640 1 -- 172.21.15.113:0/2125001262 >> 172.21.15.113:0/2125001262 conn(0x7f4a6406b0f0 msgr2=0x7f4a6406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.366+0000 7f4a69c4d640 1 -- 172.21.15.113:0/2125001262 shutdown_connections 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.366+0000 7f4a69c4d640 1 -- 172.21.15.113:0/2125001262 wait complete. 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.367+0000 7f4a69c4d640 1 Processor -- start 2024-04-25T00:09:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.367+0000 7f4a69c4d640 1 -- start start 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a6406cea0 0x7f4a64155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 0x7f4a64157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a64157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6406dc90 con 0x7f4a64071db0 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641439e0 con 0x7f4a6406cea0 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4a64141df0 con 0x7f4a640719d0 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a64157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a64157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36796/0 (socket says 172.21.15.113:36796) 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 -- 172.21.15.113:0/1561546116 learned_addr learned my addr 172.21.15.113:0/1561546116 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a637fe640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a6406cea0 0x7f4a64155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 -- 172.21.15.113:0/1561546116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 msgr2=0x7f4a64157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 0x7f4a64157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 -- 172.21.15.113:0/1561546116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a6406cea0 msgr2=0x7f4a64155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a6406cea0 0x7f4a64155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 -- 172.21.15.113:0/1561546116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a64159e60 con 0x7f4a64071db0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a63fff640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a64157e50 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c780 tx=0x7f4a5c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a4bfff640 1 -- 172.21.15.113:0/1561546116 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c017070 con 0x7f4a64071db0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a4bfff640 1 -- 172.21.15.113:0/1561546116 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5c003b70 con 0x7f4a64071db0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6415a140 con 0x7f4a64071db0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a4bfff640 1 -- 172.21.15.113:0/1561546116 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c005070 con 0x7f4a64071db0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.368+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6415a700 con 0x7f4a64071db0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.369+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a640691a0 con 0x7f4a64071db0 2024-04-25T00:09:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.370+0000 7f4a4bfff640 1 -- 172.21.15.113:0/1561546116 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a5c01a070 con 0x7f4a64071db0 2024-04-25T00:09:03.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.370+0000 7f4a4bfff640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a5006be30 0x7f4a5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.370+0000 7f4a637fe640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a5006be30 0x7f4a5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.370+0000 7f4a4bfff640 1 -- 172.21.15.113:0/1561546116 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f4a5c08a100 con 0x7f4a64071db0 2024-04-25T00:09:03.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.371+0000 7f4a637fe640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a5006be30 0x7f4a5006e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009990 tx=0x7f4a54009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:03.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.374+0000 7f4a4bfff640 1 -- 172.21.15.113:0/1561546116 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a5c058e40 con 0x7f4a64071db0 2024-04-25T00:09:03.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.550+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4a64071310 con 0x7f4a64071db0 2024-04-25T00:09:03.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.553+0000 7f4a4bfff640 1 -- 172.21.15.113:0/1561546116 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f4a5c058740 con 0x7f4a64071db0 2024-04-25T00:09:03.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a5006be30 msgr2=0x7f4a5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a5006be30 0x7f4a5006e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009990 tx=0x7f4a54009110 comp rx=0 tx=0).stop 2024-04-25T00:09:03.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 msgr2=0x7f4a64157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a64157e50 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c780 tx=0x7f4a5c00cb80 comp rx=0 tx=0).stop 2024-04-25T00:09:03.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 shutdown_connections 2024-04-25T00:09:03.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4a5006be30 0x7f4a5006e2f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a64071db0 0x7f4a64157e50 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a640719d0 0x7f4a64157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 --2- 172.21.15.113:0/1561546116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a6406cea0 0x7f4a64155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 >> 172.21.15.113:0/1561546116 conn(0x7f4a6406b0f0 msgr2=0x7f4a640ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 shutdown_connections 2024-04-25T00:09:03.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.555+0000 7f4a69c4d640 1 -- 172.21.15.113:0/1561546116 wait complete. 2024-04-25T00:09:03.567 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:09:03.567 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:09:03.599 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:09:03.642 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:09:03.642 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:03.642 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:03.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:09:03.658 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:09:03.658 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:03.701 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:09:03.701 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:03.701 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:03.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:09:03.717 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:09:03.717 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:03.758 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:09:03.758 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:09:03.759 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:03.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.854+0000 7f6987fce640 1 Processor -- start 2024-04-25T00:09:03.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.854+0000 7f6987fce640 1 -- start start 2024-04-25T00:09:03.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.855+0000 7f6987fce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 0x7f69800b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.855+0000 7f6987fce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69800d2800 con 0x7f698015e320 2024-04-25T00:09:03.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.855+0000 7f6987fce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69800d2ae0 con 0x7f69800d0b00 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.855+0000 7f6987fce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f69800d3410 con 0x7f69800d2290 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.855+0000 7f6985d6a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f698015e320 0x7f69800d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33836/0 (socket says 172.21.15.113:33836) 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.855+0000 7f6985d6a640 1 -- 172.21.15.113:0/3515024508 learned_addr learned my addr 172.21.15.113:0/3515024508 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3266813332 0 0) 0x7f69800d2800 con 0x7f698015e320 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6968003520 con 0x7f698015e320 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 0x7f69800b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4221624762 0 0) 0x7f6968003520 con 0x7f698015e320 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 msgr2=0x7f69800b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 0x7f69800b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 >> v1:172.21.15.178:6789/0 conn(0x7f69800d0b00 legacy=0x7f69800b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69800d2dc0 con 0x7f698015e320 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6978001670 con 0x7f698015e320 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 0x7f69800b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 >> v1:172.21.15.113:6789/0 conn(0x7f698015e320 legacy=0x7f69800d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69680039e0 0x7f6968003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6968004800 0x7f6968004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 0x7f696800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f69800d3410 con 0x7f6968004800 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6968005530 con 0x7f69680039e0 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f696800bf00 con 0x7f69800d2290 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6968004800 0x7f6968004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f698656b640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 0x7f696800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 -- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 msgr2=0x7f696800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d2290 0x7f696800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 -- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69680039e0 msgr2=0x7f6968003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69680039e0 0x7f6968003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.856+0000 7f6985569640 1 -- 172.21.15.113:0/3515024508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6968003520 con 0x7f6968004800 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.857+0000 7f6985569640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6968004800 0x7f6968004c80 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f697400aa80 tx=0x7f6968005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4379e5cae1d6237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.857+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974037070 con 0x7f6968004800 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.857+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697402abf0 con 0x7f6968004800 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.857+0000 7f6984d68640 1 -- 172.21.15.113:0/3515024508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400a650 con 0x7f6968004800 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.858+0000 7f6987fce640 1 -- 172.21.15.113:0/3515024508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6968004800 msgr2=0x7f6968004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.858+0000 7f6987fce640 1 --2- 172.21.15.113:0/3515024508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6968004800 0x7f6968004c80 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f697400aa80 tx=0x7f6968005830 comp rx=0 tx=0).stop 2024-04-25T00:09:03.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.858+0000 7f698656b640 1 -- 172.21.15.113:0/3515024508 reap_dead start 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.858+0000 7f6987fce640 1 -- 172.21.15.113:0/3515024508 shutdown_connections 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.858+0000 7f6987fce640 1 -- 172.21.15.113:0/3515024508 >> 172.21.15.113:0/3515024508 conn(0x7f69800b5c20 msgr2=0x7f69800b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.858+0000 7f6987fce640 1 -- 172.21.15.113:0/3515024508 shutdown_connections 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.858+0000 7f6987fce640 1 -- 172.21.15.113:0/3515024508 wait complete. 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.859+0000 7f6987fce640 1 Processor -- start 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.859+0000 7f6987fce640 1 -- start start 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968004800 0x7f69800c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6980156ec0 0x7f69800b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698014a100 0x7f698014a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f698015e880 con 0x7f6980156ec0 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f698015e700 con 0x7f6968004800 2024-04-25T00:09:03.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f69800d0f90 con 0x7f698014a100 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6980156ec0 0x7f69800b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6980156ec0 0x7f69800b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36808/0 (socket says 172.21.15.113:36808) 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 -- 172.21.15.113:0/1195500533 learned_addr learned my addr 172.21.15.113:0/1195500533 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985d6a640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968004800 0x7f69800c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f698656b640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698014a100 0x7f698014a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 -- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698014a100 msgr2=0x7f698014a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698014a100 0x7f698014a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 -- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968004800 msgr2=0x7f69800c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968004800 0x7f69800c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 -- 172.21.15.113:0/1195500533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69800ca2a0 con 0x7f6980156ec0 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f698656b640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698014a100 0x7f698014a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:03.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6985569640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6980156ec0 0x7f69800b5110 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f6974035040 tx=0x7f6974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f697e7fc640 1 -- 172.21.15.113:0/1195500533 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974041020 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f697e7fc640 1 -- 172.21.15.113:0/1195500533 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697403d420 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69800ca580 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f697e7fc640 1 -- 172.21.15.113:0/1195500533 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974037040 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.860+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801544f0 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.862+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69800691a0 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.862+0000 7f697e7fc640 1 -- 172.21.15.113:0/1195500533 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6974044020 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.862+0000 7f697e7fc640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be60 0x7f696c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.862+0000 7f697e7fc640 1 -- 172.21.15.113:0/1195500533 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4943+0+0 (secure 0 0 0) 0x7f69740ab010 con 0x7f6980156ec0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.862+0000 7f6985d6a640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be60 0x7f696c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:03.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.863+0000 7f6985d6a640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be60 0x7f696c06e320 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f69780017e0 tx=0x7f6978001aa0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:03.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:03.867+0000 7f697e7fc640 1 -- 172.21.15.113:0/1195500533 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f697404b030 con 0x7f6980156ec0 2024-04-25T00:09:04.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.077+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69800b4260 con 0x7f6980156ec0 2024-04-25T00:09:04.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.078+0000 7f697e7fc640 1 -- 172.21.15.113:0/1195500533 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+3875 (secure 0 0 0) 0x7f6974079700 con 0x7f6980156ec0 2024-04-25T00:09:04.079 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:04.079 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/152654285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":152654285},{"type":"v1","addr":"172.21.15.113:6837","nonce":152654285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/868374087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":868374087},{"type":"v1","addr":"172.21.15.178:6835","nonce":868374087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1863542234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1863542234},{"type":"v1","addr":"172.21.15.178:6834","nonce":1863542234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":9972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1442672980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1442672980},{"type":"v1","addr":"172.21.15.113:6835","nonce":1442672980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[]} 2024-04-25T00:09:04.080 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 87 2024-04-25T00:09:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be60 msgr2=0x7f696c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be60 0x7f696c06e320 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f69780017e0 tx=0x7f6978001aa0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6980156ec0 msgr2=0x7f69800b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6980156ec0 0x7f69800b5110 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f6974035040 tx=0x7f6974004730 comp rx=0 tx=0).stop 2024-04-25T00:09:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 shutdown_connections 2024-04-25T00:09:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f696c06be60 0x7f696c06e320 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698014a100 0x7f698014a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6980156ec0 0x7f69800b5110 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 --2- 172.21.15.113:0/1195500533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968004800 0x7f69800c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 >> 172.21.15.113:0/1195500533 conn(0x7f69800b5c20 msgr2=0x7f69800b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:04.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 shutdown_connections 2024-04-25T00:09:04.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.080+0000 7f6987fce640 1 -- 172.21.15.113:0/1195500533 wait complete. 2024-04-25T00:09:04.092 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:09:04.092 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:09:04.092 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:09:04.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:04.093 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:09:04.093 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:09:04.093 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:09:04.093 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:04.093 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:09:04.093 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:09:04.093 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:09:04.093 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:04.094 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:09:04.094 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:09:04.094 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:09:04.094 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:04.101 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:09:04.101 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:09:04.102 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:09:04.102 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:09:04.105 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:09:04.105 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:09:04.106 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:09:04.106 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:09:04.144 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:09:04.144 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:04.144 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:09:04.145 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:04.147 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:09:04.148 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:04.148 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:09:04.148 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:04.149 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:09:04.215 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:09:04.216 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:09:04.227 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:09:04.227 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:09:04.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.250+0000 7f9e59f61640 1 Processor -- start 2024-04-25T00:09:04.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.250+0000 7f9e59f61640 1 -- start start 2024-04-25T00:09:04.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e59f61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e540b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:04.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e59f61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e540d2e00 con 0x7f9e540b5ca0 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e59f61640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e540d30e0 con 0x7f9e5415e510 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e59f61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e540d33c0 con 0x7f9e540b6100 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e52ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9e540b5ca0 0x7f9e540b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33860/0 (socket says 172.21.15.113:33860) 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e52ffd640 1 -- 172.21.15.113:0/2508230839 learned_addr learned my addr 172.21.15.113:0/2508230839 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e53fff640 1 --2- 172.21.15.113:0/2508230839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e540b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e53fff640 1 -- 172.21.15.113:0/2508230839 >> v1:172.21.15.178:6789/0 conn(0x7f9e5415e510 legacy=0x7f9e540d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e53fff640 1 -- 172.21.15.113:0/2508230839 >> v1:172.21.15.113:6789/0 conn(0x7f9e540b5ca0 legacy=0x7f9e540b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.251+0000 7f9e53fff640 1 -- 172.21.15.113:0/2508230839 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e540d36a0 con 0x7f9e540b6100 2024-04-25T00:09:04.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.252+0000 7f9e53fff640 1 --2- 172.21.15.113:0/2508230839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e540b7ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009dc0 tx=0x7f9e4c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb604101ab4c01f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:04.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.254+0000 7f9e527fc640 1 -- 172.21.15.113:0/2508230839 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c02ae60 con 0x7f9e540b6100 2024-04-25T00:09:04.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.254+0000 7f9e527fc640 1 -- 172.21.15.113:0/2508230839 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c007740 con 0x7f9e540b6100 2024-04-25T00:09:04.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.254+0000 7f9e527fc640 1 -- 172.21.15.113:0/2508230839 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c007b10 con 0x7f9e540b6100 2024-04-25T00:09:04.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.255+0000 7f9e59f61640 1 -- 172.21.15.113:0/2508230839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 msgr2=0x7f9e540b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:04.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.255+0000 7f9e59f61640 1 --2- 172.21.15.113:0/2508230839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e540b7ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009dc0 tx=0x7f9e4c02a640 comp rx=0 tx=0).stop 2024-04-25T00:09:04.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.255+0000 7f9e59f61640 1 -- 172.21.15.113:0/2508230839 shutdown_connections 2024-04-25T00:09:04.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.255+0000 7f9e59f61640 1 --2- 172.21.15.113:0/2508230839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e540b7ad0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.255+0000 7f9e59f61640 1 -- 172.21.15.113:0/2508230839 >> 172.21.15.113:0/2508230839 conn(0x7f9e5406b720 msgr2=0x7f9e540d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:04.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.255+0000 7f9e59f61640 1 -- 172.21.15.113:0/2508230839 shutdown_connections 2024-04-25T00:09:04.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.255+0000 7f9e59f61640 1 -- 172.21.15.113:0/2508230839 wait complete. 2024-04-25T00:09:04.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.256+0000 7f9e59f61640 1 Processor -- start 2024-04-25T00:09:04.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.256+0000 7f9e59f61640 1 -- start start 2024-04-25T00:09:04.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.257+0000 7f9e59f61640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e540b5ca0 0x7f9e5414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.257+0000 7f9e59f61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e5415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.257+0000 7f9e59f61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5415e510 0x7f9e5415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.257+0000 7f9e59f61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e540d3ea0 con 0x7f9e5415e510 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.257+0000 7f9e59f61640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9e540b6550 con 0x7f9e540b5ca0 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.257+0000 7f9e59f61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e540d2a90 con 0x7f9e540b6100 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.257+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5415e510 0x7f9e5415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5415e510 0x7f9e5415eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36842/0 (socket says 172.21.15.113:36842) 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 -- 172.21.15.113:0/3717085621 learned_addr learned my addr 172.21.15.113:0/3717085621 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e537fe640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e540b5ca0 0x7f9e5414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e52ffd640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e5415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 -- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 msgr2=0x7f9e5415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e5415e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 -- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e540b5ca0 msgr2=0x7f9e5414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:04.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e540b5ca0 0x7f9e5414a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:04.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 -- 172.21.15.113:0/3717085621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e540ca080 con 0x7f9e5415e510 2024-04-25T00:09:04.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.258+0000 7f9e53fff640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5415e510 0x7f9e5415eec0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c02ac30 tx=0x7f9e4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:04.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.259+0000 7f9e3bfff640 1 -- 172.21.15.113:0/3717085621 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c040070 con 0x7f9e5415e510 2024-04-25T00:09:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.259+0000 7f9e3bfff640 1 -- 172.21.15.113:0/3717085621 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c02ae60 con 0x7f9e5415e510 2024-04-25T00:09:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.259+0000 7f9e3bfff640 1 -- 172.21.15.113:0/3717085621 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c045460 con 0x7f9e5415e510 2024-04-25T00:09:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.259+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e540ca360 con 0x7f9e5415e510 2024-04-25T00:09:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.259+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e540ca920 con 0x7f9e5415e510 2024-04-25T00:09:04.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.260+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e540691a0 con 0x7f9e5415e510 2024-04-25T00:09:04.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.261+0000 7f9e3bfff640 1 -- 172.21.15.113:0/3717085621 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e4c039040 con 0x7f9e5415e510 2024-04-25T00:09:04.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.261+0000 7f9e3bfff640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4006be80 0x7f9e4006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:04.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.262+0000 7f9e3bfff640 1 -- 172.21.15.113:0/3717085621 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4943+0+0 (secure 0 0 0) 0x7f9e4c0ac0b0 con 0x7f9e5415e510 2024-04-25T00:09:04.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.262+0000 7f9e537fe640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4006be80 0x7f9e4006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:04.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.262+0000 7f9e537fe640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4006be80 0x7f9e4006e340 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9e44000f60 tx=0x7f9e44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:04.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.268+0000 7f9e3bfff640 1 -- 172.21.15.113:0/3717085621 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e4c07a9d0 con 0x7f9e5415e510 2024-04-25T00:09:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:04.453+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9e5415cdd0 con 0x7f9e5415e510 2024-04-25T00:09:05.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.269+0000 7f9e3bfff640 1 -- 172.21.15.113:0/3717085621 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v166) ==== 97+0+0 (secure 0 0 0) 0x7f9e4c07ecd0 con 0x7f9e5415e510 2024-04-25T00:09:05.270 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:09:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4006be80 msgr2=0x7f9e4006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4006be80 0x7f9e4006e340 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9e44000f60 tx=0x7f9e44009040 comp rx=0 tx=0).stop 2024-04-25T00:09:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5415e510 msgr2=0x7f9e5415eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5415e510 0x7f9e5415eec0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c02ac30 tx=0x7f9e4c004a40 comp rx=0 tx=0).stop 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 shutdown_connections 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4006be80 0x7f9e4006e340 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5415e510 0x7f9e5415eec0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e540b6100 0x7f9e5415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 --2- 172.21.15.113:0/3717085621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e540b5ca0 0x7f9e5414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 >> 172.21.15.113:0/3717085621 conn(0x7f9e5406b720 msgr2=0x7f9e540b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 shutdown_connections 2024-04-25T00:09:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.272+0000 7f9e59f61640 1 -- 172.21.15.113:0/3717085621 wait complete. 2024-04-25T00:09:05.284 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:09:05.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.381+0000 7fbf229e9640 1 Processor -- start 2024-04-25T00:09:05.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.381+0000 7fbf229e9640 1 -- start start 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf229e9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf1c144d60 0x7fbf1c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf229e9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbf1c1453c0 0x7fbf1c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf229e9640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1c0cfad0 con 0x7fbf1c06e510 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf229e9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1c145c40 con 0x7fbf1c144d60 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf229e9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1c06e920 con 0x7fbf1c1453c0 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf1bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf1c144d60 0x7fbf1c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf1bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf1c144d60 0x7fbf1c0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36848/0 (socket says 172.21.15.113:36848) 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf1bfff640 1 -- 172.21.15.113:0/3909505734 learned_addr learned my addr 172.21.15.113:0/3909505734 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:05.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf1bfff640 1 -- 172.21.15.113:0/3909505734 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c1453c0 msgr2=0x7fbf1c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/3909505734 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c1453c0 0x7fbf1c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf1bfff640 1 -- 172.21.15.113:0/3909505734 >> v1:172.21.15.178:6789/0 conn(0x7fbf1c06e510 legacy=0x7fbf1c15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.382+0000 7fbf1bfff640 1 -- 172.21.15.113:0/3909505734 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c0cf230 con 0x7fbf1c144d60 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.383+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/3909505734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c0b5390 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0ce060 tx=0x7fbf0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe502b6e0992b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.383+0000 7fbf1affd640 1 -- 172.21.15.113:0/3909505734 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c037070 con 0x7fbf1c144d60 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.383+0000 7fbf1affd640 1 -- 172.21.15.113:0/3909505734 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c032440 con 0x7fbf1c144d60 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.383+0000 7fbf1affd640 1 -- 172.21.15.113:0/3909505734 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c032810 con 0x7fbf1c144d60 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.383+0000 7fbf229e9640 1 -- 172.21.15.113:0/3909505734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 msgr2=0x7fbf1c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.383+0000 7fbf229e9640 1 --2- 172.21.15.113:0/3909505734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c0b5390 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0ce060 tx=0x7fbf0c02a1a0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.383+0000 7fbf229e9640 1 -- 172.21.15.113:0/3909505734 shutdown_connections 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.384+0000 7fbf229e9640 1 --2- 172.21.15.113:0/3909505734 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c1453c0 0x7fbf1c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.384+0000 7fbf229e9640 1 --2- 172.21.15.113:0/3909505734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c0b5390 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.384+0000 7fbf229e9640 1 -- 172.21.15.113:0/3909505734 >> 172.21.15.113:0/3909505734 conn(0x7fbf1c0b1a50 msgr2=0x7fbf1c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:05.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.384+0000 7fbf229e9640 1 -- 172.21.15.113:0/3909505734 shutdown_connections 2024-04-25T00:09:05.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.384+0000 7fbf229e9640 1 -- 172.21.15.113:0/3909505734 wait complete. 2024-04-25T00:09:05.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 Processor -- start 2024-04-25T00:09:05.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 -- start start 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c06e510 0x7fbf1c13eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c1e8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1453c0 0x7fbf1c1e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c070e10 con 0x7fbf1c144d60 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c0cf7f0 con 0x7fbf1c1453c0 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.385+0000 7fbf229e9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1c145140 con 0x7fbf1c06e510 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c1e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c1e8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36854/0 (socket says 172.21.15.113:36854) 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 -- 172.21.15.113:0/394976698 learned_addr learned my addr 172.21.15.113:0/394976698 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c06e510 0x7fbf1c13eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf20f86640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1453c0 0x7fbf1c1e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 -- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c06e510 msgr2=0x7fbf1c13eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c06e510 0x7fbf1c13eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 -- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1453c0 msgr2=0x7fbf1c1e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1453c0 0x7fbf1c1e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 -- 172.21.15.113:0/394976698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c1601d0 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf20f86640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1453c0 0x7fbf1c1e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf1b7fe640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c1e8480 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000b410 tx=0x7fbf1000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf18ff9640 1 -- 172.21.15.113:0/394976698 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10015020 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf18ff9640 1 -- 172.21.15.113:0/394976698 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf10011850 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c1604b0 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf18ff9640 1 -- 172.21.15.113:0/394976698 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10011c40 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.386+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c15d9c0 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.387+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c0691a0 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.388+0000 7fbf18ff9640 1 -- 172.21.15.113:0/394976698 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf10018020 con 0x7fbf1c144d60 2024-04-25T00:09:05.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.388+0000 7fbf18ff9640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0806be30 0x7fbf0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.388+0000 7fbf18ff9640 1 -- 172.21.15.113:0/394976698 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4857+0+0 (secure 0 0 0) 0x7fbf1008a9c0 con 0x7fbf1c144d60 2024-04-25T00:09:05.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.388+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0806be30 0x7fbf0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.388+0000 7fbf1bfff640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0806be30 0x7fbf0806e2f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c034000 tx=0x7fbf0c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.392+0000 7fbf18ff9640 1 -- 172.21.15.113:0/394976698 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf1008ce20 con 0x7fbf1c144d60 2024-04-25T00:09:05.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.635+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbf1c0b27e0 con 0x7fbf1c144d60 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.635+0000 7fbf18ff9640 1 -- 172.21.15.113:0/394976698 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fbf1001f020 con 0x7fbf1c144d60 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:09:05.636 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:05.637 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.638 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.639 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.640 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.641 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:09:05.642 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.643 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.644 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.645 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:09:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.647 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:05.648 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:09:05.649 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:09:05.649 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:05.649 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.649 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:05.649 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:09:05.649 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:05.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0806be30 msgr2=0x7fbf0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0806be30 0x7fbf0806e2f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c034000 tx=0x7fbf0c02e000 comp rx=0 tx=0).stop 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 msgr2=0x7fbf1c1e8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c1e8480 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000b410 tx=0x7fbf1000b8d0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 shutdown_connections 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbf0806be30 0x7fbf0806e2f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1453c0 0x7fbf1c1e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf1c144d60 0x7fbf1c1e8480 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 --2- 172.21.15.113:0/394976698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c06e510 0x7fbf1c13eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.637+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 >> 172.21.15.113:0/394976698 conn(0x7fbf1c0b1a50 msgr2=0x7fbf1c13f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.638+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 shutdown_connections 2024-04-25T00:09:05.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.638+0000 7fbf229e9640 1 -- 172.21.15.113:0/394976698 wait complete. 2024-04-25T00:09:05.652 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.747+0000 7f7a71cee640 1 Processor -- start 2024-04-25T00:09:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.747+0000 7f7a71cee640 1 -- start start 2024-04-25T00:09:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a71cee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c144910 0x7f7a6c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a71cee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a6c0b5240 0x7f7a6c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a71cee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a6c144cf0 con 0x7f7a6c0b5680 2024-04-25T00:09:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a71cee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7a6c0b4510 con 0x7f7a6c0b5240 2024-04-25T00:09:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a71cee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6c0b2f30 con 0x7f7a6c144910 2024-04-25T00:09:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a6affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a6c0b5240 0x7f7a6c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a6b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c144910 0x7f7a6c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a6affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a6c0b5240 0x7f7a6c06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:52700/0 (socket says 172.21.15.113:52700) 2024-04-25T00:09:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.748+0000 7f7a6affd640 1 -- 172.21.15.113:0/734107352 learned_addr learned my addr 172.21.15.113:0/734107352 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.749+0000 7f7a6affd640 1 -- 172.21.15.113:0/734107352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c144910 msgr2=0x7f7a6c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.749+0000 7f7a6affd640 1 --2- 172.21.15.113:0/734107352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c144910 0x7f7a6c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.749+0000 7f7a6affd640 1 -- 172.21.15.113:0/734107352 >> v1:172.21.15.178:6789/0 conn(0x7f7a6c0b5680 legacy=0x7f7a6c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.749+0000 7f7a6affd640 1 -- 172.21.15.113:0/734107352 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c0b3210 con 0x7f7a6c0b5240 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a6affd640 1 --2- 172.21.15.113:0/734107352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 0x7f7a6c06dfd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000bb10 tx=0x7f7a6002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba7fc0d3c89d05c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a6a7fc640 1 -- 172.21.15.113:0/734107352 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60036020 con 0x7f7a6c0b5240 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a6a7fc640 1 -- 172.21.15.113:0/734107352 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a60032820 con 0x7f7a6c0b5240 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a6a7fc640 1 -- 172.21.15.113:0/734107352 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60032bf0 con 0x7f7a6c0b5240 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 -- 172.21.15.113:0/734107352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 msgr2=0x7f7a6c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 --2- 172.21.15.113:0/734107352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 0x7f7a6c06dfd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000bb10 tx=0x7f7a6002c920 comp rx=0 tx=0).stop 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 -- 172.21.15.113:0/734107352 shutdown_connections 2024-04-25T00:09:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 --2- 172.21.15.113:0/734107352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 0x7f7a6c06dfd0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 --2- 172.21.15.113:0/734107352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c144910 0x7f7a6c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 -- 172.21.15.113:0/734107352 >> 172.21.15.113:0/734107352 conn(0x7f7a6c0b1a50 msgr2=0x7f7a6c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 -- 172.21.15.113:0/734107352 shutdown_connections 2024-04-25T00:09:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.750+0000 7f7a71cee640 1 -- 172.21.15.113:0/734107352 wait complete. 2024-04-25T00:09:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.751+0000 7f7a71cee640 1 Processor -- start 2024-04-25T00:09:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a71cee640 1 -- start start 2024-04-25T00:09:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a71cee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 0x7f7a6c137a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a71cee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c0b5680 0x7f7a6c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a71cee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c144910 0x7f7a6c0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a71cee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c0b4da0 con 0x7f7a6c144910 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a71cee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c0b37d0 con 0x7f7a6c0b5240 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a71cee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6c0b4f20 con 0x7f7a6c0b5680 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a6bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c144910 0x7f7a6c0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a6bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c144910 0x7f7a6c0719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36866/0 (socket says 172.21.15.113:36866) 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.752+0000 7f7a6bfff640 1 -- 172.21.15.113:0/1503390470 learned_addr learned my addr 172.21.15.113:0/1503390470 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6b7fe640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 0x7f7a6c137a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6affd640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c0b5680 0x7f7a6c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6bfff640 1 -- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c0b5680 msgr2=0x7f7a6c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6bfff640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c0b5680 0x7f7a6c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6bfff640 1 -- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 msgr2=0x7f7a6c137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6bfff640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 0x7f7a6c137a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6bfff640 1 -- 172.21.15.113:0/1503390470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c0addd0 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a6bfff640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c144910 0x7f7a6c0719f0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c730 tx=0x7f7a6400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a53fff640 1 -- 172.21.15.113:0/1503390470 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64016020 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a53fff640 1 -- 172.21.15.113:0/1503390470 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a64006d50 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c0ae0b0 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a53fff640 1 -- 172.21.15.113:0/1503390470 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64003bb0 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.753+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c0ae670 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.754+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6c0691a0 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.754+0000 7f7a53fff640 1 -- 172.21.15.113:0/1503390470 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a640050d0 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.755+0000 7f7a53fff640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a5806be30 0x7f7a5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.755+0000 7f7a53fff640 1 -- 172.21.15.113:0/1503390470 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4857+0+0 (secure 0 0 0) 0x7f7a6408a870 con 0x7f7a6c144910 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.755+0000 7f7a6b7fe640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a5806be30 0x7f7a5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.755+0000 7f7a6b7fe640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a5806be30 0x7f7a5806e2f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009ad0 tx=0x7f7a5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:05.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.759+0000 7f7a53fff640 1 -- 172.21.15.113:0/1503390470 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a64020020 con 0x7f7a6c144910 2024-04-25T00:09:05.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.933+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a6c072de0 con 0x7f7a6c144910 2024-04-25T00:09:05.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.934+0000 7f7a53fff640 1 -- 172.21.15.113:0/1503390470 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+12949 (secure 0 0 0) 0x7f7a64059600 con 0x7f7a6c144910 2024-04-25T00:09:05.934 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:05.934 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":166,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:05.269197+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:05.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a5806be30 msgr2=0x7f7a5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a5806be30 0x7f7a5806e2f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009ad0 tx=0x7f7a5c008040 comp rx=0 tx=0).stop 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c144910 msgr2=0x7f7a6c0719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c144910 0x7f7a6c0719f0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c730 tx=0x7f7a6400cbf0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 shutdown_connections 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a5806be30 0x7f7a5806e2f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a6c144910 0x7f7a6c0719f0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6c0b5680 0x7f7a6c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 --2- 172.21.15.113:0/1503390470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c0b5240 0x7f7a6c137a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 >> 172.21.15.113:0/1503390470 conn(0x7f7a6c0b1a50 msgr2=0x7f7a6c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 shutdown_connections 2024-04-25T00:09:05.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:05.936+0000 7f7a71cee640 1 -- 172.21.15.113:0/1503390470 wait complete. 2024-04-25T00:09:05.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:09:05.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:09:06.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.047+0000 7f6e496f5640 1 Processor -- start 2024-04-25T00:09:06.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.047+0000 7f6e496f5640 1 -- start start 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.047+0000 7f6e496f5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6e4415e2d0 0x7f6e4415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.047+0000 7f6e496f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 0x7f6e440ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.047+0000 7f6e496f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4415f4f0 con 0x7f6e440b3080 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e496f5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6e440cde00 con 0x7f6e4415e2d0 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e496f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4415f7d0 con 0x7f6e440b4a40 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e3bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6e440b3080 0x7f6e440b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33872/0 (socket says 172.21.15.113:33872) 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e3bfff640 1 -- 172.21.15.113:0/511267194 learned_addr learned my addr 172.21.15.113:0/511267194 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/511267194 >> v2:172.21.15.178:3300/0 conn(0x7f6e4415e2d0 0x7f6e4415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e437fe640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 0x7f6e440ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497680112 0 0) 0x7f6e4415f4f0 con 0x7f6e440b3080 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e30003520 con 0x7f6e440b3080 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3463457025 0 0) 0x7f6e30003520 con 0x7f6e440b3080 2024-04-25T00:09:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 msgr2=0x7f6e440ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 0x7f6e440ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 >> v2:172.21.15.178:3300/0 conn(0x7f6e4415e2d0 msgr2=0x7f6e4415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 --2- 172.21.15.113:0/511267194 >> v2:172.21.15.178:3300/0 conn(0x7f6e4415e2d0 0x7f6e4415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4415fab0 con 0x7f6e440b3080 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f6e2c001670 con 0x7f6e440b3080 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 >> v1:172.21.15.113:6789/0 conn(0x7f6e440b3080 legacy=0x7f6e440b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 0x7f6e30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e440cde00 con 0x7f6e300039b0 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4415f7d0 con 0x7f6e30004d40 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3000b9c0 con 0x7f6e440b4a40 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.048+0000 7f6e3bfff640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 0x7f6e30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e437fe640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e42ffd640 1 -- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 msgr2=0x7f6e30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440b4a40 0x7f6e30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e42ffd640 1 -- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e30004d40 msgr2=0x7f6e300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e42ffd640 1 -- 172.21.15.113:0/511267194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e30003520 con 0x7f6e300039b0 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e437fe640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e30003dd0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009ac0 tx=0x7f6e30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bc28ee89599a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34036070 con 0x7f6e300039b0 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e34031400 con 0x7f6e300039b0 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.049+0000 7f6e427fc640 1 -- 172.21.15.113:0/511267194 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34009690 con 0x7f6e300039b0 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.050+0000 7f6e496f5640 1 -- 172.21.15.113:0/511267194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 msgr2=0x7f6e30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.050+0000 7f6e496f5640 1 --2- 172.21.15.113:0/511267194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e30003dd0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009ac0 tx=0x7f6e30005830 comp rx=0 tx=0).stop 2024-04-25T00:09:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.050+0000 7f6e437fe640 1 -- 172.21.15.113:0/511267194 reap_dead start 2024-04-25T00:09:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.050+0000 7f6e496f5640 1 -- 172.21.15.113:0/511267194 shutdown_connections 2024-04-25T00:09:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.050+0000 7f6e496f5640 1 -- 172.21.15.113:0/511267194 >> 172.21.15.113:0/511267194 conn(0x7f6e4406b0f0 msgr2=0x7f6e4406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.050+0000 7f6e496f5640 1 -- 172.21.15.113:0/511267194 shutdown_connections 2024-04-25T00:09:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.050+0000 7f6e496f5640 1 -- 172.21.15.113:0/511267194 wait complete. 2024-04-25T00:09:06.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.051+0000 7f6e496f5640 1 Processor -- start 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.051+0000 7f6e496f5640 1 -- start start 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e496f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e441f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e496f5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e441589f0 0x7f6e4415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e496f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4415d850 0x7f6e4415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e496f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4415f2b0 con 0x7f6e300039b0 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e496f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4415f130 con 0x7f6e4415d850 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e496f5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4415e6b0 con 0x7f6e441589f0 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e441f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e441f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36886/0 (socket says 172.21.15.113:36886) 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 -- 172.21.15.113:0/202969927 learned_addr learned my addr 172.21.15.113:0/202969927 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e3bfff640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e441589f0 0x7f6e4415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e437fe640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4415d850 0x7f6e4415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 -- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e441589f0 msgr2=0x7f6e4415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e441589f0 0x7f6e4415d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 -- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4415d850 msgr2=0x7f6e4415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4415d850 0x7f6e4415a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 -- 172.21.15.113:0/202969927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e44157950 con 0x7f6e300039b0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e437fe640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4415d850 0x7f6e4415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.052+0000 7f6e42ffd640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e441f7410 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009be0 tx=0x7f6e34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.053+0000 7f6e3b7fe640 1 -- 172.21.15.113:0/202969927 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34040020 con 0x7f6e300039b0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.053+0000 7f6e3b7fe640 1 -- 172.21.15.113:0/202969927 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e34003950 con 0x7f6e300039b0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.053+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e44157c30 con 0x7f6e300039b0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.053+0000 7f6e3b7fe640 1 -- 172.21.15.113:0/202969927 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34036040 con 0x7f6e300039b0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.053+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e441581f0 con 0x7f6e300039b0 2024-04-25T00:09:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.053+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e440ccac0 con 0x7f6e300039b0 2024-04-25T00:09:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.054+0000 7f6e3b7fe640 1 -- 172.21.15.113:0/202969927 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e34043020 con 0x7f6e300039b0 2024-04-25T00:09:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.054+0000 7f6e3b7fe640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e2406be30 0x7f6e2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.054+0000 7f6e3b7fe640 1 -- 172.21.15.113:0/202969927 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4857+0+0 (secure 0 0 0) 0x7f6e340aac10 con 0x7f6e300039b0 2024-04-25T00:09:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.054+0000 7f6e3bfff640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e2406be30 0x7f6e2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.055+0000 7f6e3bfff640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e2406be30 0x7f6e2406e2f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e440afea0 tx=0x7f6e2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.059+0000 7f6e3b7fe640 1 -- 172.21.15.113:0/202969927 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e440ccac0 con 0x7f6e300039b0 2024-04-25T00:09:06.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.230+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e4415eef0 con 0x7f6e300039b0 2024-04-25T00:09:06.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.231+0000 7f6e3b7fe640 1 -- 172.21.15.113:0/202969927 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v166) ==== 93+0+10 (secure 0 0 0) 0x7f6e4415eef0 con 0x7f6e300039b0 2024-04-25T00:09:06.231 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:09:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e2406be30 msgr2=0x7f6e2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e2406be30 0x7f6e2406e2f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e440afea0 tx=0x7f6e2c009040 comp rx=0 tx=0).stop 2024-04-25T00:09:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 msgr2=0x7f6e441f7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e441f7410 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009be0 tx=0x7f6e34004730 comp rx=0 tx=0).stop 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 shutdown_connections 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6e2406be30 0x7f6e2406e2f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4415d850 0x7f6e4415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e441589f0 0x7f6e4415d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 --2- 172.21.15.113:0/202969927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e300039b0 0x7f6e441f7410 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 >> 172.21.15.113:0/202969927 conn(0x7f6e4406b0f0 msgr2=0x7f6e440cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 shutdown_connections 2024-04-25T00:09:06.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.233+0000 7f6e496f5640 1 -- 172.21.15.113:0/202969927 wait complete. 2024-04-25T00:09:06.244 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:09:06.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.341+0000 7f799bcd2640 1 Processor -- start 2024-04-25T00:09:06.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.341+0000 7f799bcd2640 1 -- start start 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799bcd2640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f799415a6b0 0x7f7994070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799bcd2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79940b3af0 0x7f79940b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799bcd2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79940b2d50 con 0x7f7994072ea0 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799bcd2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7994072a70 con 0x7f79940b3af0 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799bcd2640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f79940b3010 con 0x7f799415a6b0 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799a26f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79940b3af0 0x7f79940b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799a26f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79940b3af0 0x7f79940b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36900/0 (socket says 172.21.15.113:36900) 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.342+0000 7f799a26f640 1 -- 172.21.15.113:0/1710505241 learned_addr learned my addr 172.21.15.113:0/1710505241 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f799a26f640 1 -- 172.21.15.113:0/1710505241 >> v2:172.21.15.178:3300/0 conn(0x7f799415a6b0 msgr2=0x7f7994070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:06.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f799a26f640 1 --2- 172.21.15.113:0/1710505241 >> v2:172.21.15.178:3300/0 conn(0x7f799415a6b0 0x7f7994070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f799a26f640 1 -- 172.21.15.113:0/1710505241 >> v1:172.21.15.178:6789/0 conn(0x7f7994072ea0 legacy=0x7f7994071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:06.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f799a26f640 1 -- 172.21.15.113:0/1710505241 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79940b32f0 con 0x7f79940b3af0 2024-04-25T00:09:06.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f799a26f640 1 --2- 172.21.15.113:0/1710505241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79940b3f30 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f7994062850 tx=0x7f798c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990a01b45bc6444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f7998a6c640 1 -- 172.21.15.113:0/1710505241 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0325a0 con 0x7f79940b3af0 2024-04-25T00:09:06.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f7998a6c640 1 -- 172.21.15.113:0/1710505241 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798c038070 con 0x7f79940b3af0 2024-04-25T00:09:06.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.343+0000 7f7998a6c640 1 -- 172.21.15.113:0/1710505241 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c032960 con 0x7f79940b3af0 2024-04-25T00:09:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 -- 172.21.15.113:0/1710505241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 msgr2=0x7f79940b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 --2- 172.21.15.113:0/1710505241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79940b3f30 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f7994062850 tx=0x7f798c02c8e0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 -- 172.21.15.113:0/1710505241 shutdown_connections 2024-04-25T00:09:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 --2- 172.21.15.113:0/1710505241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79940b3f30 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 --2- 172.21.15.113:0/1710505241 >> v2:172.21.15.178:3300/0 conn(0x7f799415a6b0 0x7f7994070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 -- 172.21.15.113:0/1710505241 >> 172.21.15.113:0/1710505241 conn(0x7f79940b1a50 msgr2=0x7f79940b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:06.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 -- 172.21.15.113:0/1710505241 shutdown_connections 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.344+0000 7f799bcd2640 1 -- 172.21.15.113:0/1710505241 wait complete. 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.345+0000 7f799bcd2640 1 Processor -- start 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.345+0000 7f799bcd2640 1 -- start start 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799bcd2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7994072ea0 0x7f79940ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799bcd2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79941ef660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799bcd2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f799415a6b0 0x7f799413fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799bcd2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f799415aa90 con 0x7f79940b3af0 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799bcd2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7994072720 con 0x7f7994072ea0 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799bcd2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f799415ac10 con 0x7f799415a6b0 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79941ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79941ef660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36914/0 (socket says 172.21.15.113:36914) 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 -- 172.21.15.113:0/4222311735 learned_addr learned my addr 172.21.15.113:0/4222311735 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 -- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f799415a6b0 msgr2=0x7f799413fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799a26f640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f799415a6b0 0x7f799413fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f799415a6b0 0x7f799413fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f7999a6e640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7994072ea0 0x7f79940ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 -- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7994072ea0 msgr2=0x7f79940ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7994072ea0 0x7f79940ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.346+0000 7f799926d640 1 -- 172.21.15.113:0/4222311735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79940cedc0 con 0x7f79940b3af0 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.347+0000 7f799926d640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79941ef660 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f798400c410 tx=0x7f798400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.347+0000 7f798a7fc640 1 -- 172.21.15.113:0/4222311735 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984017070 con 0x7f79940b3af0 2024-04-25T00:09:06.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.347+0000 7f798a7fc640 1 -- 172.21.15.113:0/4222311735 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79840035f0 con 0x7f79940b3af0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.347+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799415afe0 con 0x7f79940b3af0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.347+0000 7f798a7fc640 1 -- 172.21.15.113:0/4222311735 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984003a20 con 0x7f79940b3af0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.347+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799415cc90 con 0x7f79940b3af0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.348+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79940691a0 con 0x7f79940b3af0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.348+0000 7f798a7fc640 1 -- 172.21.15.113:0/4222311735 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f798401a070 con 0x7f79940b3af0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.348+0000 7f798a7fc640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f797806be30 0x7f797806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.349+0000 7f798a7fc640 1 -- 172.21.15.113:0/4222311735 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4857+0+0 (secure 0 0 0) 0x7f7984089fc0 con 0x7f79940b3af0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.349+0000 7f7999a6e640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f797806be30 0x7f797806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.349+0000 7f7999a6e640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f797806be30 0x7f797806e2f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7990007dc0 tx=0x7f79900072f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.353+0000 7f798a7fc640 1 -- 172.21.15.113:0/4222311735 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7984058cc0 con 0x7f79940b3af0 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.526+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f79940b2900 con 0x7f79940b3af0 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.527+0000 7f798a7fc640 1 -- 172.21.15.113:0/4222311735 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v166) ==== 84+0+65 (secure 0 0 0) 0x7f79840010c0 con 0x7f79940b3af0 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:09:06.527 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:06.528 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:09:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f797806be30 msgr2=0x7f797806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f797806be30 0x7f797806e2f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7990007dc0 tx=0x7f79900072f0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 msgr2=0x7f79941ef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79941ef660 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f798400c410 tx=0x7f798400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 shutdown_connections 2024-04-25T00:09:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f797806be30 0x7f797806e2f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f799415a6b0 0x7f799413fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79940b3af0 0x7f79941ef660 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 --2- 172.21.15.113:0/4222311735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7994072ea0 0x7f79940ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.528+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 >> 172.21.15.113:0/4222311735 conn(0x7f79940b1a50 msgr2=0x7f79940b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.529+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 shutdown_connections 2024-04-25T00:09:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.529+0000 7f799bcd2640 1 -- 172.21.15.113:0/4222311735 wait complete. 2024-04-25T00:09:06.540 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:09:06.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:09:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.639+0000 7f8da9161640 1 Processor -- start 2024-04-25T00:09:06.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.639+0000 7f8da9161640 1 -- start start 2024-04-25T00:09:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da9161640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8da415e510 0x7f8da4161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da9161640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8da40b9bb0 0x7f8da40b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da9161640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8da41626f0 con 0x7f8da40b61e0 2024-04-25T00:09:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da9161640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8da41629d0 con 0x7f8da40b9bb0 2024-04-25T00:09:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da9161640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8da4162cb0 con 0x7f8da415e510 2024-04-25T00:09:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8da40b9bb0 0x7f8da40b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da359e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8da40b61e0 0x7f8da4071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33888/0 (socket says 172.21.15.113:33888) 2024-04-25T00:09:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.640+0000 7f8da359e640 1 -- 172.21.15.113:0/4107970554 learned_addr learned my addr 172.21.15.113:0/4107970554 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da2d9d640 1 --2- 172.21.15.113:0/4107970554 >> v2:172.21.15.178:3300/0 conn(0x7f8da415e510 0x7f8da4161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da259c640 1 -- 172.21.15.113:0/4107970554 >> v2:172.21.15.178:3300/0 conn(0x7f8da415e510 msgr2=0x7f8da4161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da259c640 1 --2- 172.21.15.113:0/4107970554 >> v2:172.21.15.178:3300/0 conn(0x7f8da415e510 0x7f8da4161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da259c640 1 -- 172.21.15.113:0/4107970554 >> v1:172.21.15.113:6789/0 conn(0x7f8da40b61e0 legacy=0x7f8da4071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da259c640 1 -- 172.21.15.113:0/4107970554 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da40d2660 con 0x7f8da40b9bb0 2024-04-25T00:09:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da259c640 1 --2- 172.21.15.113:0/4107970554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da40b5ca0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f8da40727e0 tx=0x7f8d9802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e91134212de771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da1d9b640 1 -- 172.21.15.113:0/4107970554 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98037070 con 0x7f8da40b9bb0 2024-04-25T00:09:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da1d9b640 1 -- 172.21.15.113:0/4107970554 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d98032440 con 0x7f8da40b9bb0 2024-04-25T00:09:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.641+0000 7f8da1d9b640 1 -- 172.21.15.113:0/4107970554 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9802ae60 con 0x7f8da40b9bb0 2024-04-25T00:09:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 -- 172.21.15.113:0/4107970554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 msgr2=0x7f8da40b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 --2- 172.21.15.113:0/4107970554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da40b5ca0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f8da40727e0 tx=0x7f8d9802a110 comp rx=0 tx=0).stop 2024-04-25T00:09:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 -- 172.21.15.113:0/4107970554 shutdown_connections 2024-04-25T00:09:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 --2- 172.21.15.113:0/4107970554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da40b5ca0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 --2- 172.21.15.113:0/4107970554 >> v2:172.21.15.178:3300/0 conn(0x7f8da415e510 0x7f8da4161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 -- 172.21.15.113:0/4107970554 >> 172.21.15.113:0/4107970554 conn(0x7f8da406b720 msgr2=0x7f8da40d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 -- 172.21.15.113:0/4107970554 shutdown_connections 2024-04-25T00:09:06.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.642+0000 7f8da9161640 1 -- 172.21.15.113:0/4107970554 wait complete. 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.643+0000 7f8da9161640 1 Processor -- start 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.643+0000 7f8da9161640 1 -- start start 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da9161640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8da40b61e0 0x7f8da4152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da9161640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da9161640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8da415e510 0x7f8da4160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da9161640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8da40d3e70 con 0x7f8da40b9bb0 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da9161640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8da41623e0 con 0x7f8da40b61e0 2024-04-25T00:09:06.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da9161640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8da40b73c0 con 0x7f8da415e510 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da414d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36932/0 (socket says 172.21.15.113:36932) 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 -- 172.21.15.113:0/2148466886 learned_addr learned my addr 172.21.15.113:0/2148466886 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 -- 172.21.15.113:0/2148466886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8da415e510 msgr2=0x7f8da4160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8da415e510 0x7f8da4160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 -- 172.21.15.113:0/2148466886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8da40b61e0 msgr2=0x7f8da4152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8da40b61e0 0x7f8da4152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.644+0000 7f8da259c640 1 -- 172.21.15.113:0/2148466886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da41609e0 con 0x7f8da40b9bb0 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.645+0000 7f8da259c640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da414d070 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f8d98000c00 tx=0x7f8d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.645+0000 7f8d8b7fe640 1 -- 172.21.15.113:0/2148466886 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98037040 con 0x7f8da40b9bb0 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.645+0000 7f8d8b7fe640 1 -- 172.21.15.113:0/2148466886 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d98042070 con 0x7f8da40b9bb0 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.645+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da4151ef0 con 0x7f8da40b9bb0 2024-04-25T00:09:06.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.645+0000 7f8d8b7fe640 1 -- 172.21.15.113:0/2148466886 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98030a50 con 0x7f8da40b9bb0 2024-04-25T00:09:06.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.645+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da41524b0 con 0x7f8da40b9bb0 2024-04-25T00:09:06.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.646+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da40691a0 con 0x7f8da40b9bb0 2024-04-25T00:09:06.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.646+0000 7f8d8b7fe640 1 -- 172.21.15.113:0/2148466886 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d98030bf0 con 0x7f8da40b9bb0 2024-04-25T00:09:06.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.647+0000 7f8d8b7fe640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d9006be30 0x7f8d9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:06.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.647+0000 7f8d8b7fe640 1 -- 172.21.15.113:0/2148466886 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4857+0+0 (secure 0 0 0) 0x7f8d980ab590 con 0x7f8da40b9bb0 2024-04-25T00:09:06.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.647+0000 7f8da2d9d640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d9006be30 0x7f8d9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:06.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.647+0000 7f8da2d9d640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d9006be30 0x7f8d9006e2f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8d94007e60 tx=0x7f8d94007390 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:06.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.651+0000 7f8d8b7fe640 1 -- 172.21.15.113:0/2148466886 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d98045360 con 0x7f8da40b9bb0 2024-04-25T00:09:06.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:06.825+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8da40d1f30 con 0x7f8da40b9bb0 2024-04-25T00:09:07.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.276+0000 7f8d8b7fe640 1 -- 172.21.15.113:0/2148466886 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v167) ==== 138+0+0 (secure 0 0 0) 0x7f8d9807a290 con 0x7f8da40b9bb0 2024-04-25T00:09:07.276 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d9006be30 msgr2=0x7f8d9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d9006be30 0x7f8d9006e2f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8d94007e60 tx=0x7f8d94007390 comp rx=0 tx=0).stop 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 msgr2=0x7f8da414d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da414d070 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f8d98000c00 tx=0x7f8d980046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 shutdown_connections 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f8d9006be30 0x7f8d9006e2f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8da415e510 0x7f8da4160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8da40b9bb0 0x7f8da414d070 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 --2- 172.21.15.113:0/2148466886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8da40b61e0 0x7f8da4152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 >> 172.21.15.113:0/2148466886 conn(0x7f8da406b720 msgr2=0x7f8da40d49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 shutdown_connections 2024-04-25T00:09:07.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.279+0000 7f8da9161640 1 -- 172.21.15.113:0/2148466886 wait complete. 2024-04-25T00:09:07.294 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:09:07.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.397+0000 7f84b0f16640 1 Processor -- start 2024-04-25T00:09:07.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.397+0000 7f84b0f16640 1 -- start start 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.398+0000 7f84b0f16640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84ac15e2d0 0x7f84ac15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.398+0000 7f84b0f16640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 0x7f84ac0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.398+0000 7f84b0f16640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84ac15e6b0 0x7f84ac0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.398+0000 7f84b0f16640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f84ac0cde00 con 0x7f84ac15e6b0 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.398+0000 7f84b0f16640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f84ac15f4f0 con 0x7f84ac15e2d0 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.398+0000 7f84b0f16640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac15f7d0 con 0x7f84ac0b3080 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.398+0000 7f84aad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84ac15e6b0 0x7f84ac0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84ac15e6b0 0x7f84ac0b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36944/0 (socket says 172.21.15.113:36944) 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aad9d640 1 -- 172.21.15.113:0/193653582 learned_addr learned my addr 172.21.15.113:0/193653582 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:07.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84a9d9b640 1 --2- 172.21.15.113:0/193653582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 0x7f84ac0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aa59c640 1 --2- 172.21.15.113:0/193653582 >> v2:172.21.15.178:3300/0 conn(0x7f84ac15e2d0 0x7f84ac15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aad9d640 1 -- 172.21.15.113:0/193653582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 msgr2=0x7f84ac0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aad9d640 1 --2- 172.21.15.113:0/193653582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 0x7f84ac0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aad9d640 1 -- 172.21.15.113:0/193653582 >> v2:172.21.15.178:3300/0 conn(0x7f84ac15e2d0 msgr2=0x7f84ac15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aad9d640 1 --2- 172.21.15.113:0/193653582 >> v2:172.21.15.178:3300/0 conn(0x7f84ac15e2d0 0x7f84ac15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.399+0000 7f84aad9d640 1 -- 172.21.15.113:0/193653582 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac15fab0 con 0x7f84ac15e6b0 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84aad9d640 1 --2- 172.21.15.113:0/193653582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac0b4870 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0b2d40 tx=0x7f84a402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0df0cce842de7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84a959a640 1 -- 172.21.15.113:0/193653582 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4036070 con 0x7f84ac15e6b0 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84a959a640 1 -- 172.21.15.113:0/193653582 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a4031600 con 0x7f84ac15e6b0 2024-04-25T00:09:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84a959a640 1 -- 172.21.15.113:0/193653582 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a40319d0 con 0x7f84ac15e6b0 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 -- 172.21.15.113:0/193653582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 msgr2=0x7f84ac0b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 --2- 172.21.15.113:0/193653582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac0b4870 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0b2d40 tx=0x7f84a402a610 comp rx=0 tx=0).stop 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 -- 172.21.15.113:0/193653582 shutdown_connections 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 --2- 172.21.15.113:0/193653582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac0b4870 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 --2- 172.21.15.113:0/193653582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 0x7f84ac0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 --2- 172.21.15.113:0/193653582 >> v2:172.21.15.178:3300/0 conn(0x7f84ac15e2d0 0x7f84ac15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 -- 172.21.15.113:0/193653582 >> 172.21.15.113:0/193653582 conn(0x7f84ac06b0f0 msgr2=0x7f84ac06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 -- 172.21.15.113:0/193653582 shutdown_connections 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.400+0000 7f84b0f16640 1 -- 172.21.15.113:0/193653582 wait complete. 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 Processor -- start 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 -- start start 2024-04-25T00:09:07.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 0x7f84ac0b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ac15e2d0 0x7f84ac0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac14bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac15f2b0 con 0x7f84ac15e6b0 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac15f130 con 0x7f84ac15e2d0 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84b0f16640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac071040 con 0x7f84ac0b3080 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.402+0000 7f84aad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac14bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac14bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36958/0 (socket says 172.21.15.113:36958) 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 -- 172.21.15.113:0/3147624699 learned_addr learned my addr 172.21.15.113:0/3147624699 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 -- 172.21.15.113:0/3147624699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 msgr2=0x7f84ac0b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 0x7f84ac0b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 -- 172.21.15.113:0/3147624699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ac15e2d0 msgr2=0x7f84ac0a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ac15e2d0 0x7f84ac0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 -- 172.21.15.113:0/3147624699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac0a9b80 con 0x7f84ac15e6b0 2024-04-25T00:09:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84aad9d640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac14bb60 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009f40 tx=0x7f84a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:07.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f849affd640 1 -- 172.21.15.113:0/3147624699 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4036050 con 0x7f84ac15e6b0 2024-04-25T00:09:07.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f849affd640 1 -- 172.21.15.113:0/3147624699 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a4042070 con 0x7f84ac15e6b0 2024-04-25T00:09:07.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac0a9e60 con 0x7f84ac15e6b0 2024-04-25T00:09:07.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f849affd640 1 -- 172.21.15.113:0/3147624699 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4009510 con 0x7f84ac15e6b0 2024-04-25T00:09:07.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.403+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac0aa420 con 0x7f84ac15e6b0 2024-04-25T00:09:07.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.404+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ac0691a0 con 0x7f84ac15e6b0 2024-04-25T00:09:07.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.405+0000 7f849affd640 1 -- 172.21.15.113:0/3147624699 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84a4007bd0 con 0x7f84ac15e6b0 2024-04-25T00:09:07.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.405+0000 7f849affd640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f848c06be60 0x7f848c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:07.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.405+0000 7f849affd640 1 -- 172.21.15.113:0/3147624699 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5227+0+0 (secure 0 0 0) 0x7f84a40aba50 con 0x7f84ac15e6b0 2024-04-25T00:09:07.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.405+0000 7f84aa59c640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f848c06be60 0x7f848c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:07.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.406+0000 7f84aa59c640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f848c06be60 0x7f848c06e320 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009870 tx=0x7f84a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:07.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.410+0000 7f849affd640 1 -- 172.21.15.113:0/3147624699 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f84a407a670 con 0x7f84ac15e6b0 2024-04-25T00:09:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:07.583+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f84ac072050 con 0x7f84ac15e6b0 2024-04-25T00:09:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.278+0000 7f849affd640 1 -- 172.21.15.113:0/3147624699 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v168) ==== 170+0+0 (secure 0 0 0) 0x7f84a4079f70 con 0x7f84ac15e6b0 2024-04-25T00:09:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:09:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.281+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f848c06be60 msgr2=0x7f848c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.281+0000 7f84b0f16640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f848c06be60 0x7f848c06e320 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009870 tx=0x7f84a0008040 comp rx=0 tx=0).stop 2024-04-25T00:09:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.281+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 msgr2=0x7f84ac14bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.281+0000 7f84b0f16640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac14bb60 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009f40 tx=0x7f84a4004a40 comp rx=0 tx=0).stop 2024-04-25T00:09:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.282+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 shutdown_connections 2024-04-25T00:09:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.282+0000 7f84b0f16640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f848c06be60 0x7f848c06e320 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.282+0000 7f84b0f16640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ac15e2d0 0x7f84ac0a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.282+0000 7f84b0f16640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84ac15e6b0 0x7f84ac14bb60 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.282+0000 7f84b0f16640 1 --2- 172.21.15.113:0/3147624699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ac0b3080 0x7f84ac0b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.282+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 >> 172.21.15.113:0/3147624699 conn(0x7f84ac06b0f0 msgr2=0x7f84ac06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:08.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.282+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 shutdown_connections 2024-04-25T00:09:08.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.283+0000 7f84b0f16640 1 -- 172.21.15.113:0/3147624699 wait complete. 2024-04-25T00:09:08.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:09:08.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.407+0000 7f13aab10640 1 Processor -- start 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.407+0000 7f13aab10640 1 -- start start 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.407+0000 7f13aab10640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f13a415b670 0x7f13a415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.407+0000 7f13aab10640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f13a415bff0 0x7f13a40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.407+0000 7f13aab10640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a4071170 con 0x7f13a40cc740 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.407+0000 7f13aab10640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f13a415c4a0 con 0x7f13a415bff0 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.407+0000 7f13aab10640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f13a4071450 con 0x7f13a415b670 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f13a415bff0 0x7f13a40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f13a415bff0 0x7f13a40cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45916/0 (socket says 172.21.15.113:45916) 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a3fff640 1 -- 172.21.15.113:0/1111481869 learned_addr learned my addr 172.21.15.113:0/1111481869 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a88ac640 1 --2- 172.21.15.113:0/1111481869 >> v2:172.21.15.178:3300/0 conn(0x7f13a415b670 0x7f13a415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a3fff640 1 -- 172.21.15.113:0/1111481869 >> v2:172.21.15.178:3300/0 conn(0x7f13a415b670 msgr2=0x7f13a415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a3fff640 1 --2- 172.21.15.113:0/1111481869 >> v2:172.21.15.178:3300/0 conn(0x7f13a415b670 0x7f13a415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a3fff640 1 -- 172.21.15.113:0/1111481869 >> v1:172.21.15.178:6789/0 conn(0x7f13a40cc740 legacy=0x7f13a406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a3fff640 1 -- 172.21.15.113:0/1111481869 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a4071730 con 0x7f13a415bff0 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.408+0000 7f13a88ac640 1 --2- 172.21.15.113:0/1111481869 >> v2:172.21.15.178:3300/0 conn(0x7f13a415b670 0x7f13a415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13a3fff640 1 --2- 172.21.15.113:0/1111481869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415bff0 0x7f13a40cc200 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f13a406f1a0 tx=0x7f139002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9def599f220ef427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13a37fe640 1 -- 172.21.15.113:0/1111481869 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390037070 con 0x7f13a415bff0 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13a37fe640 1 -- 172.21.15.113:0/1111481869 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1390032440 con 0x7f13a415bff0 2024-04-25T00:09:08.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13a37fe640 1 -- 172.21.15.113:0/1111481869 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390032810 con 0x7f13a415bff0 2024-04-25T00:09:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 -- 172.21.15.113:0/1111481869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415bff0 msgr2=0x7f13a40cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 --2- 172.21.15.113:0/1111481869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415bff0 0x7f13a40cc200 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f13a406f1a0 tx=0x7f139002a150 comp rx=0 tx=0).stop 2024-04-25T00:09:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 -- 172.21.15.113:0/1111481869 shutdown_connections 2024-04-25T00:09:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 --2- 172.21.15.113:0/1111481869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415bff0 0x7f13a40cc200 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 --2- 172.21.15.113:0/1111481869 >> v2:172.21.15.178:3300/0 conn(0x7f13a415b670 0x7f13a415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 -- 172.21.15.113:0/1111481869 >> 172.21.15.113:0/1111481869 conn(0x7f13a406b0f0 msgr2=0x7f13a406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 -- 172.21.15.113:0/1111481869 shutdown_connections 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.409+0000 7f13aab10640 1 -- 172.21.15.113:0/1111481869 wait complete. 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.410+0000 7f13aab10640 1 Processor -- start 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.410+0000 7f13aab10640 1 -- start start 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13aab10640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40cc740 0x7f13a41554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13aab10640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415b670 0x7f13a4144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13aab10640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a415bff0 0x7f13a4140710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13aab10640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f13a406d730 con 0x7f13a415b670 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13aab10640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13a406d5b0 con 0x7f13a40cc740 2024-04-25T00:09:08.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13aab10640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13a40d0080 con 0x7f13a415bff0 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13a3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415b670 0x7f13a4144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13a3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415b670 0x7f13a4144b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45926/0 (socket says 172.21.15.113:45926) 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.411+0000 7f13a3fff640 1 -- 172.21.15.113:0/442490696 learned_addr learned my addr 172.21.15.113:0/442490696 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a88ac640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40cc740 0x7f13a41554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a90ad640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a415bff0 0x7f13a4140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a3fff640 1 -- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a415bff0 msgr2=0x7f13a4140710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a3fff640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a415bff0 0x7f13a4140710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a3fff640 1 -- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40cc740 msgr2=0x7f13a41554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a3fff640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40cc740 0x7f13a41554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a3fff640 1 -- 172.21.15.113:0/442490696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a4149340 con 0x7f13a415b670 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a88ac640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40cc740 0x7f13a41554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a3fff640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415b670 0x7f13a4144b50 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f1390000c00 tx=0x7f13900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a17fa640 1 -- 172.21.15.113:0/442490696 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390041020 con 0x7f13a415b670 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a17fa640 1 -- 172.21.15.113:0/442490696 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13900306c0 con 0x7f13a415b670 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a4149e20 con 0x7f13a415b670 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13a17fa640 1 -- 172.21.15.113:0/442490696 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390037040 con 0x7f13a415b670 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.412+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a4145090 con 0x7f13a415b670 2024-04-25T00:09:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.413+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a40691a0 con 0x7f13a415b670 2024-04-25T00:09:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.413+0000 7f13a17fa640 1 -- 172.21.15.113:0/442490696 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1390044020 con 0x7f13a415b670 2024-04-25T00:09:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.414+0000 7f13a17fa640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138806be30 0x7f138806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.414+0000 7f13a88ac640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138806be30 0x7f138806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.414+0000 7f13a17fa640 1 -- 172.21.15.113:0/442490696 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5609+0+0 (secure 0 0 0) 0x7f13900ab590 con 0x7f13a415b670 2024-04-25T00:09:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.414+0000 7f13a88ac640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138806be30 0x7f138806e2f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1398006fd0 tx=0x7f1398008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.418+0000 7f13a17fa640 1 -- 172.21.15.113:0/442490696 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f139004b030 con 0x7f13a415b670 2024-04-25T00:09:08.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:08.627+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f13a406daf0 con 0x7f13a415b670 2024-04-25T00:09:09.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.288+0000 7f13a17fa640 1 -- 172.21.15.113:0/442490696 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v89) ==== 174+0+0 (secure 0 0 0) 0x7f139007a030 con 0x7f13a415b670 2024-04-25T00:09:09.289 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 24 and data pool 25 2024-04-25T00:09:09.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.297+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138806be30 msgr2=0x7f138806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.297+0000 7f13aab10640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138806be30 0x7f138806e2f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1398006fd0 tx=0x7f1398008040 comp rx=0 tx=0).stop 2024-04-25T00:09:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.297+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415b670 msgr2=0x7f13a4144b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.297+0000 7f13aab10640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415b670 0x7f13a4144b50 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f1390000c00 tx=0x7f13900046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.298+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 shutdown_connections 2024-04-25T00:09:09.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.298+0000 7f13aab10640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f138806be30 0x7f138806e2f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.298+0000 7f13aab10640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a415bff0 0x7f13a4140710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.298+0000 7f13aab10640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a415b670 0x7f13a4144b50 secure :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f1390000c00 tx=0x7f13900046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.298+0000 7f13aab10640 1 --2- 172.21.15.113:0/442490696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40cc740 0x7f13a41554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.298+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 >> 172.21.15.113:0/442490696 conn(0x7f13a406b0f0 msgr2=0x7f13a40cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.302+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 shutdown_connections 2024-04-25T00:09:09.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.302+0000 7f13aab10640 1 -- 172.21.15.113:0/442490696 wait complete. 2024-04-25T00:09:09.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:09.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.421+0000 7fad988a5640 1 Processor -- start 2024-04-25T00:09:09.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.421+0000 7fad988a5640 1 -- start start 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad988a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad9415e320 0x7fad940d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad988a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fad940d0b00 0x7fad940d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad988a5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad940d3de0 con 0x7fad940d2290 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad988a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fad940d14b0 con 0x7fad9415e320 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad988a5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fad940d3410 con 0x7fad940d0b00 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad9259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad9415e320 0x7fad940d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad9259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad9415e320 0x7fad940d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45928/0 (socket says 172.21.15.113:45928) 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad9259c640 1 -- 172.21.15.113:0/3578107446 learned_addr learned my addr 172.21.15.113:0/3578107446 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:09.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad92d9d640 1 --2- 172.21.15.113:0/3578107446 >> v2:172.21.15.178:3300/0 conn(0x7fad940d0b00 0x7fad940d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad9259c640 1 -- 172.21.15.113:0/3578107446 >> v2:172.21.15.178:3300/0 conn(0x7fad940d0b00 msgr2=0x7fad940d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad9259c640 1 --2- 172.21.15.113:0/3578107446 >> v2:172.21.15.178:3300/0 conn(0x7fad940d0b00 0x7fad940d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad9259c640 1 -- 172.21.15.113:0/3578107446 >> v1:172.21.15.178:6789/0 conn(0x7fad940d2290 legacy=0x7fad940b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad9259c640 1 -- 172.21.15.113:0/3578107446 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad940d2800 con 0x7fad9415e320 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.422+0000 7fad92d9d640 1 --2- 172.21.15.113:0/3578107446 >> v2:172.21.15.178:3300/0 conn(0x7fad940d0b00 0x7fad940d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad9259c640 1 --2- 172.21.15.113:0/3578107446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad940d38a0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fad940d2030 tx=0x7fad8402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ec0d8b9f2afa19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad9159a640 1 -- 172.21.15.113:0/3578107446 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84037070 con 0x7fad9415e320 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad9159a640 1 -- 172.21.15.113:0/3578107446 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8402ae50 con 0x7fad9415e320 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad9159a640 1 -- 172.21.15.113:0/3578107446 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84032860 con 0x7fad9415e320 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 -- 172.21.15.113:0/3578107446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 msgr2=0x7fad940d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 --2- 172.21.15.113:0/3578107446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad940d38a0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fad940d2030 tx=0x7fad8402a300 comp rx=0 tx=0).stop 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 -- 172.21.15.113:0/3578107446 shutdown_connections 2024-04-25T00:09:09.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 --2- 172.21.15.113:0/3578107446 >> v2:172.21.15.178:3300/0 conn(0x7fad940d0b00 0x7fad940d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 --2- 172.21.15.113:0/3578107446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad940d38a0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 -- 172.21.15.113:0/3578107446 >> 172.21.15.113:0/3578107446 conn(0x7fad940b5c20 msgr2=0x7fad940b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 -- 172.21.15.113:0/3578107446 shutdown_connections 2024-04-25T00:09:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.423+0000 7fad988a5640 1 -- 172.21.15.113:0/3578107446 wait complete. 2024-04-25T00:09:09.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 Processor -- start 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 -- start start 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940d0b00 0x7fad941542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940d2290 0x7fad941f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad94161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad940d30a0 con 0x7fad9415e320 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad940d2dc0 con 0x7fad940d2290 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.425+0000 7fad988a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad9415e700 con 0x7fad940d0b00 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad94161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad94161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45934/0 (socket says 172.21.15.113:45934) 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 -- 172.21.15.113:0/679595880 learned_addr learned my addr 172.21.15.113:0/679595880 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad9259c640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940d0b00 0x7fad941542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad91d9b640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940d2290 0x7fad941f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 -- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940d0b00 msgr2=0x7fad941542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940d0b00 0x7fad941542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 -- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940d2290 msgr2=0x7fad941f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940d2290 0x7fad941f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 -- 172.21.15.113:0/679595880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9414a8f0 con 0x7fad9415e320 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad9259c640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940d0b00 0x7fad941542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad91d9b640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940d2290 0x7fad941f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad92d9d640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad94161440 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fad8c007f20 tx=0x7fad8c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad82ffd640 1 -- 172.21.15.113:0/679595880 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c01a070 con 0x7fad9415e320 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad82ffd640 1 -- 172.21.15.113:0/679595880 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c003b90 con 0x7fad9415e320 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad94156720 con 0x7fad9415e320 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad82ffd640 1 -- 172.21.15.113:0/679595880 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c004030 con 0x7fad9415e320 2024-04-25T00:09:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.426+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad94156ca0 con 0x7fad9415e320 2024-04-25T00:09:09.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.427+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad940691a0 con 0x7fad9415e320 2024-04-25T00:09:09.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.428+0000 7fad82ffd640 1 -- 172.21.15.113:0/679595880 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad8c01d070 con 0x7fad9415e320 2024-04-25T00:09:09.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.428+0000 7fad82ffd640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad7406be30 0x7fad7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.428+0000 7fad82ffd640 1 -- 172.21.15.113:0/679595880 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7fad8c08b0b0 con 0x7fad9415e320 2024-04-25T00:09:09.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.428+0000 7fad9259c640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad7406be30 0x7fad7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.428+0000 7fad9259c640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad7406be30 0x7fad7406e2f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fad84034000 tx=0x7fad8402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.432+0000 7fad82ffd640 1 -- 172.21.15.113:0/679595880 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad8c010070 con 0x7fad9415e320 2024-04-25T00:09:09.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.606+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad94072140 con 0x7fad9415e320 2024-04-25T00:09:09.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.607+0000 7fad82ffd640 1 -- 172.21.15.113:0/679595880 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+16740 (secure 0 0 0) 0x7fad8c059af0 con 0x7fad9415e320 2024-04-25T00:09:09.609 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:09.609 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":169,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:08.628766+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:06.844691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:07.603153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:09.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad7406be30 msgr2=0x7fad7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad7406be30 0x7fad7406e2f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fad84034000 tx=0x7fad8402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:09.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 msgr2=0x7fad94161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad94161440 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fad8c007f20 tx=0x7fad8c00d560 comp rx=0 tx=0).stop 2024-04-25T00:09:09.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 shutdown_connections 2024-04-25T00:09:09.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fad7406be30 0x7fad7406e2f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940d2290 0x7fad941f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9415e320 0x7fad94161440 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 --2- 172.21.15.113:0/679595880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940d0b00 0x7fad941542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 >> 172.21.15.113:0/679595880 conn(0x7fad940b5c20 msgr2=0x7fad940b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 shutdown_connections 2024-04-25T00:09:09.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.610+0000 7fad988a5640 1 -- 172.21.15.113:0/679595880 wait complete. 2024-04-25T00:09:09.622 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:09.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.718+0000 7fe9b2fce640 1 Processor -- start 2024-04-25T00:09:09.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.718+0000 7fe9b2fce640 1 -- start start 2024-04-25T00:09:09.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.719+0000 7fe9b2fce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9ac0b4b00 0x7fe9ac1656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.719+0000 7fe9b2fce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ac071b20 con 0x7fe9ac15a6b0 2024-04-25T00:09:09.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.719+0000 7fe9b2fce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ac0b2d50 con 0x7fe9ac072ea0 2024-04-25T00:09:09.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.719+0000 7fe9b2fce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ac072a70 con 0x7fe9ac0b4b00 2024-04-25T00:09:09.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.719+0000 7fe9b156b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9ac0b4b00 0x7fe9ac1656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.719+0000 7fe9b0d6a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe9ac15a6b0 0x7fe9ac070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39902/0 (socket says 172.21.15.113:39902) 2024-04-25T00:09:09.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.719+0000 7fe9b0d6a640 1 -- 172.21.15.113:0/370243318 learned_addr learned my addr 172.21.15.113:0/370243318 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:09.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9b156b640 1 -- 172.21.15.113:0/370243318 >> v1:172.21.15.178:6789/0 conn(0x7fe9ac072ea0 legacy=0x7fe9ac071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9b156b640 1 -- 172.21.15.113:0/370243318 >> v1:172.21.15.113:6789/0 conn(0x7fe9ac15a6b0 legacy=0x7fe9ac070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9b156b640 1 -- 172.21.15.113:0/370243318 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac0b3010 con 0x7fe9ac0b4b00 2024-04-25T00:09:09.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9ab7fe640 1 -- 172.21.15.113:0/370243318 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753450012 0 0) 0x7fe9ac071b20 con 0x7fe9ac15a6b0 2024-04-25T00:09:09.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9b156b640 1 --2- 172.21.15.113:0/370243318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac0b4b00 0x7fe9ac1656d0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009d90 tx=0x7fe9a402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14445c88d71d310b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9ab7fe640 1 -- 172.21.15.113:0/370243318 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4007740 con 0x7fe9ac0b4b00 2024-04-25T00:09:09.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9ab7fe640 1 -- 172.21.15.113:0/370243318 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a40078e0 con 0x7fe9ac0b4b00 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.720+0000 7fe9ab7fe640 1 -- 172.21.15.113:0/370243318 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4007cb0 con 0x7fe9ac0b4b00 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.721+0000 7fe9b2fce640 1 -- 172.21.15.113:0/370243318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac0b4b00 msgr2=0x7fe9ac1656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.721+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/370243318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac0b4b00 0x7fe9ac1656d0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009d90 tx=0x7fe9a402a6a0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.721+0000 7fe9b2fce640 1 -- 172.21.15.113:0/370243318 shutdown_connections 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.721+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/370243318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac0b4b00 0x7fe9ac1656d0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.721+0000 7fe9b2fce640 1 -- 172.21.15.113:0/370243318 >> 172.21.15.113:0/370243318 conn(0x7fe9ac0b1a50 msgr2=0x7fe9ac0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.721+0000 7fe9b2fce640 1 -- 172.21.15.113:0/370243318 shutdown_connections 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.721+0000 7fe9b2fce640 1 -- 172.21.15.113:0/370243318 wait complete. 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.722+0000 7fe9b2fce640 1 Processor -- start 2024-04-25T00:09:09.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.722+0000 7fe9b2fce640 1 -- start start 2024-04-25T00:09:09.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b2fce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac072ea0 0x7fe9ac151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b2fce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0b4b00 0x7fe9ac158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b2fce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac15a6b0 0x7fe9ac14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b2fce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac072720 con 0x7fe9ac072ea0 2024-04-25T00:09:09.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b2fce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac0b35d0 con 0x7fe9ac15a6b0 2024-04-25T00:09:09.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b2fce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac15aa90 con 0x7fe9ac0b4b00 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac072ea0 0x7fe9ac151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac072ea0 0x7fe9ac151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45960/0 (socket says 172.21.15.113:45960) 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 -- 172.21.15.113:0/116643182 learned_addr learned my addr 172.21.15.113:0/116643182 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b156b640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac15a6b0 0x7fe9ac14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9abfff640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0b4b00 0x7fe9ac158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 -- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0b4b00 msgr2=0x7fe9ac158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0b4b00 0x7fe9ac158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 -- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac15a6b0 msgr2=0x7fe9ac14d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac15a6b0 0x7fe9ac14d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 -- 172.21.15.113:0/116643182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac152240 con 0x7fe9ac072ea0 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b156b640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac15a6b0 0x7fe9ac14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9abfff640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0b4b00 0x7fe9ac158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.723+0000 7fe9b0d6a640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac072ea0 0x7fe9ac151bb0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c430 tx=0x7fe99c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.724+0000 7fe9a97fa640 1 -- 172.21.15.113:0/116643182 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0074e0 con 0x7fe9ac072ea0 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.724+0000 7fe9a97fa640 1 -- 172.21.15.113:0/116643182 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c0047e0 con 0x7fe9ac072ea0 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.724+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac152520 con 0x7fe9ac072ea0 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.724+0000 7fe9a97fa640 1 -- 172.21.15.113:0/116643182 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c004bd0 con 0x7fe9ac072ea0 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.724+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac152ae0 con 0x7fe9ac072ea0 2024-04-25T00:09:09.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.725+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac0691a0 con 0x7fe9ac072ea0 2024-04-25T00:09:09.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.725+0000 7fe9a97fa640 1 -- 172.21.15.113:0/116643182 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe99c011070 con 0x7fe9ac072ea0 2024-04-25T00:09:09.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.725+0000 7fe9a97fa640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe99806be60 0x7fe99806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.726+0000 7fe9a97fa640 1 -- 172.21.15.113:0/116643182 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7fe99c08aad0 con 0x7fe9ac072ea0 2024-04-25T00:09:09.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.726+0000 7fe9abfff640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe99806be60 0x7fe99806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.726+0000 7fe9abfff640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe99806be60 0x7fe99806e320 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac14e140 tx=0x7fe9a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.730+0000 7fe9a97fa640 1 -- 172.21.15.113:0/116643182 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe99c059480 con 0x7fe9ac072ea0 2024-04-25T00:09:09.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.904+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9ac0b2900 con 0x7fe9ac072ea0 2024-04-25T00:09:09.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.905+0000 7fe9a97fa640 1 -- 172.21.15.113:0/116643182 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+16740 (secure 0 0 0) 0x7fe99c058d80 con 0x7fe9ac072ea0 2024-04-25T00:09:09.906 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:09.906 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":169,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:08.628766+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:06.844691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:07.603153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe99806be60 msgr2=0x7fe99806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe99806be60 0x7fe99806e320 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac14e140 tx=0x7fe9a0009040 comp rx=0 tx=0).stop 2024-04-25T00:09:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac072ea0 msgr2=0x7fe9ac151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac072ea0 0x7fe9ac151bb0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c430 tx=0x7fe99c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 shutdown_connections 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe99806be60 0x7fe99806e320 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac15a6b0 0x7fe9ac14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0b4b00 0x7fe9ac158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 --2- 172.21.15.113:0/116643182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac072ea0 0x7fe9ac151bb0 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 >> 172.21.15.113:0/116643182 conn(0x7fe9ac0b1a50 msgr2=0x7fe9ac156b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 shutdown_connections 2024-04-25T00:09:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:09.907+0000 7fe9b2fce640 1 -- 172.21.15.113:0/116643182 wait complete. 2024-04-25T00:09:09.919 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.017+0000 7fb5b4184640 1 Processor -- start 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.017+0000 7fb5b4184640 1 -- start start 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b4184640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb5ac15e2d0 0x7fb5ac15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b4184640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ac15f4f0 con 0x7fb5ac0b3080 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b4184640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ac15f7d0 con 0x7fb5ac0b4a40 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b4184640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb5ac0cde00 con 0x7fb5ac15e2d0 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b171f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb5ac0b3080 0x7fb5ac0b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39908/0 (socket says 172.21.15.113:39908) 2024-04-25T00:09:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b171f640 1 -- 172.21.15.113:0/1811360775 learned_addr learned my addr 172.21.15.113:0/1811360775 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/1811360775 >> v2:172.21.15.178:3300/0 conn(0x7fb5ac15e2d0 0x7fb5ac15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231214655 0 0) 0x7fb5ac15f4f0 con 0x7fb5ac0b3080 2024-04-25T00:09:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb598003520 con 0x7fb5ac0b3080 2024-04-25T00:09:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1651195357 0 0) 0x7fb598003520 con 0x7fb5ac0b3080 2024-04-25T00:09:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 >> v2:172.21.15.178:3300/0 conn(0x7fb5ac15e2d0 msgr2=0x7fb5ac15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 --2- 172.21.15.113:0/1811360775 >> v2:172.21.15.178:3300/0 conn(0x7fb5ac15e2d0 0x7fb5ac15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 >> v1:172.21.15.178:6789/0 conn(0x7fb5ac0b4a40 legacy=0x7fb5ac0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac15fab0 con 0x7fb5ac0b3080 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fb59c001670 con 0x7fb5ac0b3080 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 >> v1:172.21.15.113:6789/0 conn(0x7fb5ac0b3080 legacy=0x7fb5ac0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5980039b0 0x7fb598003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980043d0 0x7fb5980048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5980050a0 0x7fb59800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac0cde00 con 0x7fb5980039b0 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb59800bfe0 con 0x7fb5980050a0 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb59800c1e0 con 0x7fb5980043d0 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.018+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/1811360775 >> v2:172.21.15.178:3300/0 conn(0x7fb5ac15e2d0 0x7fb5ac15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5980039b0 0x7fb598003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b171f640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980043d0 0x7fb5980048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b2721640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5980050a0 0x7fb59800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b1f20640 1 -- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980043d0 msgr2=0x7fb5980048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980043d0 0x7fb5980048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b1f20640 1 -- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5980050a0 msgr2=0x7fb59800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5980050a0 0x7fb59800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b2721640 1 -- 172.21.15.113:0/1811360775 reap_dead start 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b1f20640 1 -- 172.21.15.113:0/1811360775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb598003520 con 0x7fb5980039b0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.019+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5980039b0 0x7fb598003db0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800e350 tx=0x7fb598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8129b34d2cd08b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8033070 con 0x7fb5980039b0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a8003b90 con 0x7fb5980039b0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b0f1e640 1 -- 172.21.15.113:0/1811360775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8009dc0 con 0x7fb5980039b0 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b4184640 1 -- 172.21.15.113:0/1811360775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5980039b0 msgr2=0x7fb598003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b4184640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5980039b0 0x7fb598003db0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800e350 tx=0x7fb598005830 comp rx=0 tx=0).stop 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b4184640 1 -- 172.21.15.113:0/1811360775 shutdown_connections 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b4184640 1 --2- 172.21.15.113:0/1811360775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5980039b0 0x7fb598003db0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b4184640 1 -- 172.21.15.113:0/1811360775 >> 172.21.15.113:0/1811360775 conn(0x7fb5ac06b0f0 msgr2=0x7fb5ac06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b4184640 1 -- 172.21.15.113:0/1811360775 shutdown_connections 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.020+0000 7fb5b4184640 1 -- 172.21.15.113:0/1811360775 wait complete. 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.021+0000 7fb5b4184640 1 Processor -- start 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.021+0000 7fb5b4184640 1 -- start start 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b4184640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980039b0 0x7fb5ac143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b4184640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5ac0ad890 0x7fb5ac0add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b4184640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5ac0b1070 0x7fb5ac0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b4184640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac15e830 con 0x7fb5ac0ad890 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b4184640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac15e6b0 con 0x7fb5ac0b1070 2024-04-25T00:09:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b4184640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ac15e9b0 con 0x7fb5980039b0 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5ac0ad890 0x7fb5ac0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5ac0ad890 0x7fb5ac0add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45970/0 (socket says 172.21.15.113:45970) 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 -- 172.21.15.113:0/536258381 learned_addr learned my addr 172.21.15.113:0/536258381 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 -- 172.21.15.113:0/536258381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980039b0 msgr2=0x7fb5ac143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980039b0 0x7fb5ac143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 -- 172.21.15.113:0/536258381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5ac0b1070 msgr2=0x7fb5ac0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5ac0b1070 0x7fb5ac0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.022+0000 7fb5b171f640 1 -- 172.21.15.113:0/536258381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac0ae400 con 0x7fb5ac0ad890 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.023+0000 7fb5b171f640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5ac0ad890 0x7fb5ac0add10 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0072f0 tx=0x7fb59c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.023+0000 7fb5a27fc640 1 -- 172.21.15.113:0/536258381 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c01a070 con 0x7fb5ac0ad890 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.023+0000 7fb5a27fc640 1 -- 172.21.15.113:0/536258381 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c00f8e0 con 0x7fb5ac0ad890 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.023+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ac0ae6e0 con 0x7fb5ac0ad890 2024-04-25T00:09:10.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.023+0000 7fb5a27fc640 1 -- 172.21.15.113:0/536258381 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00fd10 con 0x7fb5ac0ad890 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.023+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ac0aeca0 con 0x7fb5ac0ad890 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.024+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ac0691a0 con 0x7fb5ac0ad890 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.024+0000 7fb5a27fc640 1 -- 172.21.15.113:0/536258381 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb59c01d070 con 0x7fb5ac0ad890 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.025+0000 7fb5a27fc640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb59006be30 0x7fb59006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.025+0000 7fb5a27fc640 1 -- 172.21.15.113:0/536258381 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5709+0+0 (secure 0 0 0) 0x7fb59c08a9c0 con 0x7fb5ac0ad890 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.025+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb59006be30 0x7fb59006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.025+0000 7fb5b1f20640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb59006be30 0x7fb59006e2f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8034000 tx=0x7fb5a802d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.029+0000 7fb5a27fc640 1 -- 172.21.15.113:0/536258381 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb59c08ce20 con 0x7fb5ac0ad890 2024-04-25T00:09:10.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.238+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb5ac072050 con 0x7fb5ac0ad890 2024-04-25T00:09:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.877+0000 7fb5a27fc640 1 -- 172.21.15.113:0/536258381 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v91) ==== 118+0+0 (secure 0 0 0) 0x7fb59c05f020 con 0x7fb5ac0ad890 2024-04-25T00:09:10.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.879+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb59006be30 msgr2=0x7fb59006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:10.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.879+0000 7fb5b4184640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb59006be30 0x7fb59006e2f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8034000 tx=0x7fb5a802d000 comp rx=0 tx=0).stop 2024-04-25T00:09:10.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.879+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5ac0ad890 msgr2=0x7fb5ac0add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5ac0ad890 0x7fb5ac0add10 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0072f0 tx=0x7fb59c0077b0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 shutdown_connections 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb59006be30 0x7fb59006e2f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5ac0b1070 0x7fb5ac0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5ac0ad890 0x7fb5ac0add10 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 --2- 172.21.15.113:0/536258381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5980039b0 0x7fb5ac143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 >> 172.21.15.113:0/536258381 conn(0x7fb5ac06b0f0 msgr2=0x7fb5ac0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 shutdown_connections 2024-04-25T00:09:10.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.880+0000 7fb5b4184640 1 -- 172.21.15.113:0/536258381 wait complete. 2024-04-25T00:09:10.893 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:09:10.894 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:09:10.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.996+0000 7f9756ee1640 1 Processor -- start 2024-04-25T00:09:10.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.996+0000 7f9756ee1640 1 -- start start 2024-04-25T00:09:11.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9756ee1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97500719d0 0x7f975015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9756ee1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 0x7f97500cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9756ee1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9750071580 con 0x7f9750071db0 2024-04-25T00:09:11.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9756ee1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9750141620 con 0x7f97500719d0 2024-04-25T00:09:11.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9756ee1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9750141fc0 con 0x7f975006cea0 2024-04-25T00:09:11.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9747fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9750071db0 0x7f97501418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39914/0 (socket says 172.21.15.113:39914) 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9747fff640 1 -- 172.21.15.113:0/1538935275 learned_addr learned my addr 172.21.15.113:0/1538935275 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f975547e640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 0x7f97500cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f9754c7d640 1 --2- 172.21.15.113:0/1538935275 >> v2:172.21.15.178:3300/0 conn(0x7f97500719d0 0x7f975015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 310630248 0 0) 0x7f9740001110 con 0x7f9750071db0 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9738003520 con 0x7f9750071db0 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1475755607 0 0) 0x7f9738003520 con 0x7f9750071db0 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 msgr2=0x7f97500cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 0x7f97500cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 >> v2:172.21.15.178:3300/0 conn(0x7f97500719d0 msgr2=0x7f975015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 --2- 172.21.15.113:0/1538935275 >> v2:172.21.15.178:3300/0 conn(0x7f97500719d0 0x7f975015f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975006d3f0 con 0x7f9750071db0 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.997+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f9740001840 con 0x7f9750071db0 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 >> v1:172.21.15.113:6789/0 conn(0x7f9750071db0 legacy=0x7f97501418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:11.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f97477fe640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 0x7f97380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f97477fe640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9738003f30 0x7f9738004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f97477fe640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9738004d40 0x7f97380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9750141fc0 con 0x7f9738004d40 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f973800b9c0 con 0x7f9738003f30 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f973800bb70 con 0x7f975006cea0 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f975547e640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9738004d40 0x7f97380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f9754c7d640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 0x7f97380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f9747fff640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9738003f30 0x7f9738004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f975547e640 1 -- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 msgr2=0x7f97380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f975547e640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 0x7f97380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f975547e640 1 -- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9738003f30 msgr2=0x7f9738004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f975547e640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9738003f30 0x7f9738004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f975547e640 1 -- 172.21.15.113:0/1538935275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9738003520 con 0x7f9738004d40 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.998+0000 7f9754c7d640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975006cea0 0x7f97380039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f975547e640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9738004d40 0x7f97380051c0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f9748009fc0 tx=0x7f9738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adcd05e2276af27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748009c10 con 0x7f9738004d40 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748036070 con 0x7f9738004d40 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f97477fe640 1 -- 172.21.15.113:0/1538935275 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748031880 con 0x7f9738004d40 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f9756ee1640 1 -- 172.21.15.113:0/1538935275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9738004d40 msgr2=0x7f97380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f9756ee1640 1 --2- 172.21.15.113:0/1538935275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9738004d40 0x7f97380051c0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f9748009fc0 tx=0x7f9738005830 comp rx=0 tx=0).stop 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f975547e640 1 -- 172.21.15.113:0/1538935275 reap_dead start 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f9756ee1640 1 -- 172.21.15.113:0/1538935275 shutdown_connections 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f9756ee1640 1 -- 172.21.15.113:0/1538935275 >> 172.21.15.113:0/1538935275 conn(0x7f975006b0f0 msgr2=0x7f975006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f9756ee1640 1 -- 172.21.15.113:0/1538935275 shutdown_connections 2024-04-25T00:09:11.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:10.999+0000 7f9756ee1640 1 -- 172.21.15.113:0/1538935275 wait complete. 2024-04-25T00:09:11.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.000+0000 7f9756ee1640 1 Processor -- start 2024-04-25T00:09:11.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.000+0000 7f9756ee1640 1 -- start start 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9756ee1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97501579b0 0x7f9750158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9756ee1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9750157d90 0x7f97501593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9756ee1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975015ad10 0x7f975015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9756ee1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97501439e0 con 0x7f9750157d90 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9756ee1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f975006d9b0 con 0x7f97501579b0 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9756ee1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f975006dc90 con 0x7f975015ad10 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9747fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9750157d90 0x7f97501593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9747fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9750157d90 0x7f97501593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45988/0 (socket says 172.21.15.113:45988) 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.001+0000 7f9747fff640 1 -- 172.21.15.113:0/31009252 learned_addr learned my addr 172.21.15.113:0/31009252 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f975547e640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975015ad10 0x7f975015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9747fff640 1 -- 172.21.15.113:0/31009252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975015ad10 msgr2=0x7f975015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9747fff640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975015ad10 0x7f975015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9747fff640 1 -- 172.21.15.113:0/31009252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97501579b0 msgr2=0x7f9750158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9747fff640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97501579b0 0x7f9750158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9747fff640 1 -- 172.21.15.113:0/31009252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975015e1e0 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9747fff640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9750157d90 0x7f97501593d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f9740004840 tx=0x7f974000ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f97457fa640 1 -- 172.21.15.113:0/31009252 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974000c8a0 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f97457fa640 1 -- 172.21.15.113:0/31009252 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9740018070 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97501599c0 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f97457fa640 1 -- 172.21.15.113:0/31009252 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974000cca0 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.002+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9750159f80 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.003+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97500691a0 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.003+0000 7f97457fa640 1 -- 172.21.15.113:0/31009252 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f974001d460 con 0x7f9750157d90 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.004+0000 7f97457fa640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f973c06be30 0x7f973c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.004+0000 7f9754c7d640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f973c06be30 0x7f973c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:11.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.004+0000 7f97457fa640 1 -- 172.21.15.113:0/31009252 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f974008ae60 con 0x7f9750157d90 2024-04-25T00:09:11.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.004+0000 7f9754c7d640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f973c06be30 0x7f973c06e2f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f974c0099b0 tx=0x7f974c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:11.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.008+0000 7f97457fa640 1 -- 172.21.15.113:0/31009252 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9750071580 con 0x7f9750157d90 2024-04-25T00:09:11.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.217+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f97500cfdf0 con 0x7f9750157d90 2024-04-25T00:09:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.885+0000 7f97457fa640 1 -- 172.21.15.113:0/31009252 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v92) ==== 122+0+0 (secure 0 0 0) 0x7f97400598d0 con 0x7f9750157d90 2024-04-25T00:09:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f973c06be30 msgr2=0x7f973c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f973c06be30 0x7f973c06e2f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f974c0099b0 tx=0x7f974c008040 comp rx=0 tx=0).stop 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9750157d90 msgr2=0x7f97501593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9750157d90 0x7f97501593d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f9740004840 tx=0x7f974000ef80 comp rx=0 tx=0).stop 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 shutdown_connections 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f973c06be30 0x7f973c06e2f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f975015ad10 0x7f975015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9750157d90 0x7f97501593d0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 --2- 172.21.15.113:0/31009252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97501579b0 0x7f9750158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 >> 172.21.15.113:0/31009252 conn(0x7f975006b0f0 msgr2=0x7f97500b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 shutdown_connections 2024-04-25T00:09:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:11.887+0000 7f9756ee1640 1 -- 172.21.15.113:0/31009252 wait complete. 2024-04-25T00:09:11.901 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:09:11.902 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:12.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.003+0000 7ff448f53640 1 Processor -- start 2024-04-25T00:09:12.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.003+0000 7ff448f53640 1 -- start start 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.003+0000 7ff448f53640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff44415e320 0x7ff4440d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.003+0000 7ff448f53640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 0x7ff4440d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.003+0000 7ff448f53640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4440d3de0 con 0x7ff4440d2290 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.003+0000 7ff448f53640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff4440d14d0 con 0x7ff44415e320 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.003+0000 7ff448f53640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4440d3410 con 0x7ff4440d0b00 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff43bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff4440d2290 0x7ff4440b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39926/0 (socket says 172.21.15.113:39926) 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff43bfff640 1 -- 172.21.15.113:0/487769904 learned_addr learned my addr 172.21.15.113:0/487769904 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218544141 0 0) 0x7ff4440d3de0 con 0x7ff4440d2290 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff430003520 con 0x7ff4440d2290 2024-04-25T00:09:12.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff442d9d640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 0x7ff4440d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff44259c640 1 --2- 172.21.15.113:0/487769904 >> v2:172.21.15.178:3300/0 conn(0x7ff44415e320 0x7ff4440d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 714971262 0 0) 0x7ff430003520 con 0x7ff4440d2290 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 msgr2=0x7ff4440d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 0x7ff4440d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 >> v2:172.21.15.178:3300/0 conn(0x7ff44415e320 msgr2=0x7ff4440d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 --2- 172.21.15.113:0/487769904 >> v2:172.21.15.178:3300/0 conn(0x7ff44415e320 0x7ff4440d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4440d2800 con 0x7ff4440d2290 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7ff42c001670 con 0x7ff4440d2290 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 >> v1:172.21.15.113:6789/0 conn(0x7ff4440d2290 legacy=0x7ff4440b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4300039b0 0x7ff430003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 0x7ff430004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff430004d40 0x7ff4300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440d14d0 con 0x7ff430004d40 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440d3410 con 0x7ff4300039b0 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff43000ba80 con 0x7ff4440d0b00 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff442d9d640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff430004d40 0x7ff4300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.004+0000 7ff43bfff640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 0x7ff430004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff442d9d640 1 -- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 msgr2=0x7ff430004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff442d9d640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 0x7ff430004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff442d9d640 1 -- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4300039b0 msgr2=0x7ff430003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff442d9d640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4300039b0 0x7ff430003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff442d9d640 1 -- 172.21.15.113:0/487769904 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff430003520 con 0x7ff430004d40 2024-04-25T00:09:12.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff43bfff640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440d0b00 0x7ff430004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff442d9d640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff430004d40 0x7ff4300051c0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009fc0 tx=0x7ff430005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f331a520b5787d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c009c10 con 0x7ff430004d40 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43c036070 con 0x7ff430004d40 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.005+0000 7ff441d9b640 1 -- 172.21.15.113:0/487769904 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c031880 con 0x7ff430004d40 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.006+0000 7ff448f53640 1 -- 172.21.15.113:0/487769904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff430004d40 msgr2=0x7ff4300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.006+0000 7ff448f53640 1 --2- 172.21.15.113:0/487769904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff430004d40 0x7ff4300051c0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009fc0 tx=0x7ff430005830 comp rx=0 tx=0).stop 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.006+0000 7ff442d9d640 1 -- 172.21.15.113:0/487769904 reap_dead start 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.006+0000 7ff448f53640 1 -- 172.21.15.113:0/487769904 shutdown_connections 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.006+0000 7ff448f53640 1 -- 172.21.15.113:0/487769904 >> 172.21.15.113:0/487769904 conn(0x7ff4440b5c20 msgr2=0x7ff4440b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.006+0000 7ff448f53640 1 -- 172.21.15.113:0/487769904 shutdown_connections 2024-04-25T00:09:12.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.006+0000 7ff448f53640 1 -- 172.21.15.113:0/487769904 wait complete. 2024-04-25T00:09:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.007+0000 7ff448f53640 1 Processor -- start 2024-04-25T00:09:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.007+0000 7ff448f53640 1 -- start start 2024-04-25T00:09:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff448f53640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4440cda70 0x7ff444153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff448f53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4440cb1e0 0x7ff444071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff448f53640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440cc710 0x7ff444071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff448f53640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440d30a0 con 0x7ff4440cb1e0 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff448f53640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440d2dc0 con 0x7ff4440cda70 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff448f53640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff44415e700 con 0x7ff4440cc710 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4440cb1e0 0x7ff444071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4440cb1e0 0x7ff444071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46012/0 (socket says 172.21.15.113:46012) 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 -- 172.21.15.113:0/389434385 learned_addr learned my addr 172.21.15.113:0/389434385 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 -- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440cc710 msgr2=0x7ff444071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440cc710 0x7ff444071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 -- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4440cda70 msgr2=0x7ff444153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff44259c640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4440cda70 0x7ff444153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4440cda70 0x7ff444153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff43bfff640 1 -- 172.21.15.113:0/389434385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4440cb670 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.008+0000 7ff44259c640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4440cda70 0x7ff444153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.009+0000 7ff43bfff640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4440cb1e0 0x7ff444071710 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00da90 tx=0x7ff42c00de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.009+0000 7ff43affd640 1 -- 172.21.15.113:0/389434385 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c019070 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.009+0000 7ff43affd640 1 -- 172.21.15.113:0/389434385 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42c004cb0 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.009+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4440cb950 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.009+0000 7ff43affd640 1 -- 172.21.15.113:0/389434385 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004e60 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.009+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4440cbf10 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.010+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4440691a0 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.010+0000 7ff43affd640 1 -- 172.21.15.113:0/389434385 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff42c01c070 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.010+0000 7ff43affd640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff42406be30 0x7ff42406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.010+0000 7ff43affd640 1 -- 172.21.15.113:0/389434385 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7ff42c08a940 con 0x7ff4440cb1e0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.010+0000 7ff44259c640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff42406be30 0x7ff42406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.011+0000 7ff44259c640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff42406be30 0x7ff42406e2f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff434006a90 tx=0x7ff434008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.015+0000 7ff43affd640 1 -- 172.21.15.113:0/389434385 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff42c059380 con 0x7ff4440cb1e0 2024-04-25T00:09:12.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.226+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4440d1650 con 0x7ff4440cb1e0 2024-04-25T00:09:12.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.226+0000 7ff43affd640 1 -- 172.21.15.113:0/389434385 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5195 (secure 0 0 0) 0x7ff42c058c80 con 0x7ff4440cb1e0 2024-04-25T00:09:12.228 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:12.228 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/992895335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":992895335},{"type":"v1","addr":"172.21.15.178:6834","nonce":992895335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/17230728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":17230728},{"type":"v1","addr":"172.21.15.113:6836","nonce":17230728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4170863917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4170863917},{"type":"v1","addr":"172.21.15.113:6837","nonce":4170863917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:08.629096+0000","modified":"2024-04-25T00:09:11.220151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10434":{"gid":10434,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/2813561857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2813561857},{"type":"v1","addr":"172.21.15.178:6835","nonce":2813561857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10434,"qdb_cluster":[10434]},"id":12}]} 2024-04-25T00:09:12.228 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-04-25T00:09:12.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff42406be30 msgr2=0x7ff42406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff42406be30 0x7ff42406e2f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff434006a90 tx=0x7ff434008040 comp rx=0 tx=0).stop 2024-04-25T00:09:12.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4440cb1e0 msgr2=0x7ff444071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4440cb1e0 0x7ff444071710 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00da90 tx=0x7ff42c00de90 comp rx=0 tx=0).stop 2024-04-25T00:09:12.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 shutdown_connections 2024-04-25T00:09:12.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff42406be30 0x7ff42406e2f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4440cc710 0x7ff444071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4440cb1e0 0x7ff444071710 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.228+0000 7ff448f53640 1 --2- 172.21.15.113:0/389434385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4440cda70 0x7ff444153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.229+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 >> 172.21.15.113:0/389434385 conn(0x7ff4440b5c20 msgr2=0x7ff4440b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.229+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 shutdown_connections 2024-04-25T00:09:12.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.229+0000 7ff448f53640 1 -- 172.21.15.113:0/389434385 wait complete. 2024-04-25T00:09:12.241 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.338+0000 7f2f0cb89640 1 Processor -- start 2024-04-25T00:09:12.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.338+0000 7f2f0cb89640 1 -- start start 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0cb89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2f08144910 0x7f2f080b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0cb89640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2f080b5240 0x7f2f0806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0cb89640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f08144cf0 con 0x7f2f080b56b0 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0cb89640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2f080b4510 con 0x7f2f08144910 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0cb89640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2f080b2f30 con 0x7f2f080b5240 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2f08144910 0x7f2f080b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2f08144910 0x7f2f080b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46018/0 (socket says 172.21.15.113:46018) 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 -- 172.21.15.113:0/3683533417 learned_addr learned my addr 172.21.15.113:0/3683533417 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f05d9b640 1 --2- 172.21.15.113:0/3683533417 >> v2:172.21.15.178:3300/0 conn(0x7f2f080b5240 0x7f2f0806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 -- 172.21.15.113:0/3683533417 >> v2:172.21.15.178:3300/0 conn(0x7f2f080b5240 msgr2=0x7f2f0806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 --2- 172.21.15.113:0/3683533417 >> v2:172.21.15.178:3300/0 conn(0x7f2f080b5240 0x7f2f0806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 -- 172.21.15.113:0/3683533417 >> v1:172.21.15.178:6789/0 conn(0x7f2f080b56b0 legacy=0x7f2f080cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 -- 172.21.15.113:0/3683533417 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f080b3210 con 0x7f2f08144910 2024-04-25T00:09:12.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0659c640 1 --2- 172.21.15.113:0/3683533417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f080b3fd0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009e50 tx=0x7f2ef802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1953afbda51f628e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0559a640 1 -- 172.21.15.113:0/3683533417 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8036070 con 0x7f2f08144910 2024-04-25T00:09:12.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0559a640 1 -- 172.21.15.113:0/3683533417 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef8031600 con 0x7f2f08144910 2024-04-25T00:09:12.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.340+0000 7f2f0559a640 1 -- 172.21.15.113:0/3683533417 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80319d0 con 0x7f2f08144910 2024-04-25T00:09:12.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 -- 172.21.15.113:0/3683533417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 msgr2=0x7f2f080b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/3683533417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f080b3fd0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009e50 tx=0x7f2ef802a710 comp rx=0 tx=0).stop 2024-04-25T00:09:12.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 -- 172.21.15.113:0/3683533417 shutdown_connections 2024-04-25T00:09:12.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/3683533417 >> v2:172.21.15.178:3300/0 conn(0x7f2f080b5240 0x7f2f0806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/3683533417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f080b3fd0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 -- 172.21.15.113:0/3683533417 >> 172.21.15.113:0/3683533417 conn(0x7f2f080b1a50 msgr2=0x7f2f080b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 -- 172.21.15.113:0/3683533417 shutdown_connections 2024-04-25T00:09:12.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.341+0000 7f2f0cb89640 1 -- 172.21.15.113:0/3683533417 wait complete. 2024-04-25T00:09:12.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.342+0000 7f2f0cb89640 1 Processor -- start 2024-04-25T00:09:12.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.342+0000 7f2f0cb89640 1 -- start start 2024-04-25T00:09:12.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f0cb89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f080b5240 0x7f2f0815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f0cb89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f080b56b0 0x7f2f08142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f0cb89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f08139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f0cb89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080b4da0 con 0x7f2f08144910 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f0cb89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080b37d0 con 0x7f2f080b5240 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f0cb89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2f080b4f20 con 0x7f2f080b56b0 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f06d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f08139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f06d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f08139230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46028/0 (socket says 172.21.15.113:46028) 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.343+0000 7f2f06d9d640 1 -- 172.21.15.113:0/1259901544 learned_addr learned my addr 172.21.15.113:0/1259901544 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f05d9b640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f080b56b0 0x7f2f08142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f06d9d640 1 -- 172.21.15.113:0/1259901544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f080b56b0 msgr2=0x7f2f08142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f06d9d640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f080b56b0 0x7f2f08142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f06d9d640 1 -- 172.21.15.113:0/1259901544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f080b5240 msgr2=0x7f2f0815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f06d9d640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f080b5240 0x7f2f0815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f06d9d640 1 -- 172.21.15.113:0/1259901544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f08143770 con 0x7f2f08144910 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f06d9d640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f08139230 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c3e0 tx=0x7f2f0000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2ef6ffd640 1 -- 172.21.15.113:0/1259901544 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00017070 con 0x7f2f08144910 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2ef6ffd640 1 -- 172.21.15.113:0/1259901544 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f00007250 con 0x7f2f08144910 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f08143a50 con 0x7f2f08144910 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2ef6ffd640 1 -- 172.21.15.113:0/1259901544 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00007680 con 0x7f2f08144910 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.344+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f08144010 con 0x7f2f08144910 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.345+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f080691a0 con 0x7f2f08144910 2024-04-25T00:09:12.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.345+0000 7f2ef6ffd640 1 -- 172.21.15.113:0/1259901544 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f0001a070 con 0x7f2f08144910 2024-04-25T00:09:12.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.346+0000 7f2ef6ffd640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ee806be60 0x7f2ee806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.346+0000 7f2ef6ffd640 1 -- 172.21.15.113:0/1259901544 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f2f0008a110 con 0x7f2f08144910 2024-04-25T00:09:12.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.346+0000 7f2f0659c640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ee806be60 0x7f2ee806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.346+0000 7f2f0659c640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ee806be60 0x7f2ee806e320 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8033000 tx=0x7f2ef802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.351+0000 7f2ef6ffd640 1 -- 172.21.15.113:0/1259901544 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2f00058ac0 con 0x7f2f08144910 2024-04-25T00:09:12.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.522+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f080b07a0 con 0x7f2f08144910 2024-04-25T00:09:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.523+0000 7f2ef6ffd640 1 -- 172.21.15.113:0/1259901544 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16727 (secure 0 0 0) 0x7f2f000583c0 con 0x7f2f08144910 2024-04-25T00:09:12.525 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:12.525 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:10.300297+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:06.844691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:07.603153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ee806be60 msgr2=0x7f2ee806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ee806be60 0x7f2ee806e320 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8033000 tx=0x7f2ef802e000 comp rx=0 tx=0).stop 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 msgr2=0x7f2f08139230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f08139230 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c3e0 tx=0x7f2f0000c8a0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 shutdown_connections 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2ee806be60 0x7f2ee806e320 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f08144910 0x7f2f08139230 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f080b56b0 0x7f2f08142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 --2- 172.21.15.113:0/1259901544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f080b5240 0x7f2f0815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 >> 172.21.15.113:0/1259901544 conn(0x7f2f080b1a50 msgr2=0x7f2f08071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 shutdown_connections 2024-04-25T00:09:12.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.526+0000 7f2f0cb89640 1 -- 172.21.15.113:0/1259901544 wait complete. 2024-04-25T00:09:12.538 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:09:12.538 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:09:12.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdacbb09640 1 Processor -- start 2024-04-25T00:09:12.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdacbb09640 1 -- start start 2024-04-25T00:09:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdacbb09640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdac40d2290 0x7fdac40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdacbb09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdac40d0b00 0x7fdac40d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdacbb09640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdac40d1480 con 0x7fdac415e320 2024-04-25T00:09:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdacbb09640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdac40d3410 con 0x7fdac40d0b00 2024-04-25T00:09:12.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdacbb09640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdac40d2800 con 0x7fdac40d2290 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdaca0a6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdac40d0b00 0x7fdac40d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdac98a5640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdac415e320 0x7fdac40d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39932/0 (socket says 172.21.15.113:39932) 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.633+0000 7fdac98a5640 1 -- 172.21.15.113:0/3673556384 learned_addr learned my addr 172.21.15.113:0/3673556384 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.634+0000 7fdac90a4640 1 --2- 172.21.15.113:0/3673556384 >> v2:172.21.15.178:3300/0 conn(0x7fdac40d2290 0x7fdac40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.634+0000 7fdaca0a6640 1 -- 172.21.15.113:0/3673556384 >> v2:172.21.15.178:3300/0 conn(0x7fdac40d2290 msgr2=0x7fdac40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.634+0000 7fdaca0a6640 1 --2- 172.21.15.113:0/3673556384 >> v2:172.21.15.178:3300/0 conn(0x7fdac40d2290 0x7fdac40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.634+0000 7fdaca0a6640 1 -- 172.21.15.113:0/3673556384 >> v1:172.21.15.113:6789/0 conn(0x7fdac415e320 legacy=0x7fdac40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.634+0000 7fdaca0a6640 1 -- 172.21.15.113:0/3673556384 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac40d2ae0 con 0x7fdac40d0b00 2024-04-25T00:09:12.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.634+0000 7fdac90a4640 1 --2- 172.21.15.113:0/3673556384 >> v2:172.21.15.178:3300/0 conn(0x7fdac40d2290 0x7fdac40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.634+0000 7fdaca0a6640 1 --2- 172.21.15.113:0/3673556384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d0b00 0x7fdac40d0f40 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fdac4072810 tx=0x7fdabc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c773958bcd0842c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdac88a3640 1 -- 172.21.15.113:0/3673556384 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc007740 con 0x7fdac40d0b00 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdac88a3640 1 -- 172.21.15.113:0/3673556384 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc0078e0 con 0x7fdac40d0b00 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdac88a3640 1 -- 172.21.15.113:0/3673556384 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc007cb0 con 0x7fdac40d0b00 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 -- 172.21.15.113:0/3673556384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d0b00 msgr2=0x7fdac40d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 --2- 172.21.15.113:0/3673556384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d0b00 0x7fdac40d0f40 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fdac4072810 tx=0x7fdabc02a710 comp rx=0 tx=0).stop 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 -- 172.21.15.113:0/3673556384 shutdown_connections 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 --2- 172.21.15.113:0/3673556384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d0b00 0x7fdac40d0f40 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 --2- 172.21.15.113:0/3673556384 >> v2:172.21.15.178:3300/0 conn(0x7fdac40d2290 0x7fdac40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 -- 172.21.15.113:0/3673556384 >> 172.21.15.113:0/3673556384 conn(0x7fdac40b5c20 msgr2=0x7fdac40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 -- 172.21.15.113:0/3673556384 shutdown_connections 2024-04-25T00:09:12.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.635+0000 7fdacbb09640 1 -- 172.21.15.113:0/3673556384 wait complete. 2024-04-25T00:09:12.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.636+0000 7fdacbb09640 1 Processor -- start 2024-04-25T00:09:12.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.636+0000 7fdacbb09640 1 -- start start 2024-04-25T00:09:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdacbb09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdac40d0b00 0x7fdac4151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdacbb09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d2290 0x7fdac40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdacbb09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac415e320 0x7fdac40b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdacbb09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdac40b9fe0 con 0x7fdac40d2290 2024-04-25T00:09:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdacbb09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdac40d30a0 con 0x7fdac415e320 2024-04-25T00:09:12.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdacbb09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdac415e700 con 0x7fdac40d0b00 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d2290 0x7fdac40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d2290 0x7fdac40cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46042/0 (socket says 172.21.15.113:46042) 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 -- 172.21.15.113:0/4009620924 learned_addr learned my addr 172.21.15.113:0/4009620924 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 -- 172.21.15.113:0/4009620924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdac40d0b00 msgr2=0x7fdac4151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdac40d0b00 0x7fdac4151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 -- 172.21.15.113:0/4009620924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac415e320 msgr2=0x7fdac40b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac415e320 0x7fdac40b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.637+0000 7fdac90a4640 1 -- 172.21.15.113:0/4009620924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac415ef00 con 0x7fdac40d2290 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.638+0000 7fdac90a4640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d2290 0x7fdac40cd230 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c7f0 tx=0x7fdac000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.638+0000 7fdaba7fc640 1 -- 172.21.15.113:0/4009620924 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0015070 con 0x7fdac40d2290 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.638+0000 7fdaba7fc640 1 -- 172.21.15.113:0/4009620924 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac0007a30 con 0x7fdac40d2290 2024-04-25T00:09:12.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.638+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac415f1e0 con 0x7fdac40d2290 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.638+0000 7fdaba7fc640 1 -- 172.21.15.113:0/4009620924 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0007e00 con 0x7fdac40d2290 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.638+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac415f7a0 con 0x7fdac40d2290 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.639+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac40691a0 con 0x7fdac40d2290 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.639+0000 7fdaba7fc640 1 -- 172.21.15.113:0/4009620924 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdac0018070 con 0x7fdac40d2290 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.639+0000 7fdaba7fc640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa806be30 0x7fdaa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.640+0000 7fdaba7fc640 1 -- 172.21.15.113:0/4009620924 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fdac008ae50 con 0x7fdac40d2290 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.640+0000 7fdac98a5640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa806be30 0x7fdaa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.640+0000 7fdac98a5640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa806be30 0x7fdaa806e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000f60 tx=0x7fdab4007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.644+0000 7fdaba7fc640 1 -- 172.21.15.113:0/4009620924 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdac0059800 con 0x7fdac40d2290 2024-04-25T00:09:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.812+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdac40b77c0 con 0x7fdaa806be30 2024-04-25T00:09:12.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.818+0000 7fdaba7fc640 1 -- 172.21.15.113:0/4009620924 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252450 (secure 0 0 0) 0x7fdac40b77c0 con 0x7fdaa806be30 2024-04-25T00:09:12.822 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:12.823 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa806be30 msgr2=0x7fdaa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa806be30 0x7fdaa806e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000f60 tx=0x7fdab4007a80 comp rx=0 tx=0).stop 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d2290 msgr2=0x7fdac40cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d2290 0x7fdac40cd230 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c7f0 tx=0x7fdac000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 shutdown_connections 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdaa806be30 0x7fdaa806e2f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac415e320 0x7fdac40b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac40d2290 0x7fdac40cd230 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 --2- 172.21.15.113:0/4009620924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdac40d0b00 0x7fdac4151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 >> 172.21.15.113:0/4009620924 conn(0x7fdac40b5c20 msgr2=0x7fdac4071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 shutdown_connections 2024-04-25T00:09:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.822+0000 7fdacbb09640 1 -- 172.21.15.113:0/4009620924 wait complete. 2024-04-25T00:09:12.833 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-04-25T00:09:11.242151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":254,"num_write_kb":3476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411736,"kb_used_data":17868,"kb_used_omap":12,"kb_used_meta":393715,"kb_avail":754562984,"statfs":{"total":773094113280,"available":772672495616,"internally_reserved":0,"allocated":18296832,"data_stored":8653296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":403164720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000725"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318270+0000","last_change":"2024-04-25T00:09:09.316654+0000","last_active":"2024-04-25T00:09:09.318270+0000","last_peered":"2024-04-25T00:09:09.318270+0000","last_clean":"2024-04-25T00:09:09.318270+0000","last_became_active":"2024-04-25T00:09:09.316354+0000","last_became_peered":"2024-04-25T00:09:09.316354+0000","last_unstale":"2024-04-25T00:09:09.318270+0000","last_undegraded":"2024-04-25T00:09:09.318270+0000","last_fullsized":"2024-04-25T00:09:09.318270+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:34.904655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.311065+0000","last_change":"2024-04-25T00:09:09.307459+0000","last_active":"2024-04-25T00:09:09.311065+0000","last_peered":"2024-04-25T00:09:09.311065+0000","last_clean":"2024-04-25T00:09:09.311065+0000","last_became_active":"2024-04-25T00:09:09.307213+0000","last_became_peered":"2024-04-25T00:09:09.307213+0000","last_unstale":"2024-04-25T00:09:09.311065+0000","last_undegraded":"2024-04-25T00:09:09.311065+0000","last_fullsized":"2024-04-25T00:09:09.311065+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:32.188804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307695+0000","last_change":"2024-04-25T00:09:09.301074+0000","last_active":"2024-04-25T00:09:10.307695+0000","last_peered":"2024-04-25T00:09:10.307695+0000","last_clean":"2024-04-25T00:09:10.307695+0000","last_became_active":"2024-04-25T00:09:09.300775+0000","last_became_peered":"2024-04-25T00:09:09.300775+0000","last_unstale":"2024-04-25T00:09:10.307695+0000","last_undegraded":"2024-04-25T00:09:10.307695+0000","last_fullsized":"2024-04-25T00:09:10.307695+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:24.687119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.311097+0000","last_change":"2024-04-25T00:09:09.306732+0000","last_active":"2024-04-25T00:09:09.311097+0000","last_peered":"2024-04-25T00:09:09.311097+0000","last_clean":"2024-04-25T00:09:09.311097+0000","last_became_active":"2024-04-25T00:09:09.306441+0000","last_became_peered":"2024-04-25T00:09:09.306441+0000","last_unstale":"2024-04-25T00:09:09.311097+0000","last_undegraded":"2024-04-25T00:09:09.311097+0000","last_fullsized":"2024-04-25T00:09:09.311097+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:15.630150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952585+0000","last_change":"2024-04-25T00:09:09.307186+0000","last_active":"2024-04-25T00:09:10.952585+0000","last_peered":"2024-04-25T00:09:10.952585+0000","last_clean":"2024-04-25T00:09:10.952585+0000","last_became_active":"2024-04-25T00:09:09.306907+0000","last_became_peered":"2024-04-25T00:09:09.306907+0000","last_unstale":"2024-04-25T00:09:10.952585+0000","last_undegraded":"2024-04-25T00:09:10.952585+0000","last_fullsized":"2024-04-25T00:09:10.952585+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:50.448530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307829+0000","last_change":"2024-04-25T00:09:09.308940+0000","last_active":"2024-04-25T00:09:10.307829+0000","last_peered":"2024-04-25T00:09:10.307829+0000","last_clean":"2024-04-25T00:09:10.307829+0000","last_became_active":"2024-04-25T00:09:09.308666+0000","last_became_peered":"2024-04-25T00:09:09.308666+0000","last_unstale":"2024-04-25T00:09:10.307829+0000","last_undegraded":"2024-04-25T00:09:10.307829+0000","last_fullsized":"2024-04-25T00:09:10.307829+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:27.096818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.305907+0000","last_change":"2024-04-25T00:09:09.302385+0000","last_active":"2024-04-25T00:09:10.305907+0000","last_peered":"2024-04-25T00:09:10.305907+0000","last_clean":"2024-04-25T00:09:10.305907+0000","last_became_active":"2024-04-25T00:09:09.301541+0000","last_became_peered":"2024-04-25T00:09:09.301541+0000","last_unstale":"2024-04-25T00:09:10.305907+0000","last_undegraded":"2024-04-25T00:09:10.305907+0000","last_fullsized":"2024-04-25T00:09:10.305907+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:07.605187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.304808+0000","last_change":"2024-04-25T00:09:09.298602+0000","last_active":"2024-04-25T00:09:09.304808+0000","last_peered":"2024-04-25T00:09:09.304808+0000","last_clean":"2024-04-25T00:09:09.304808+0000","last_became_active":"2024-04-25T00:09:09.297806+0000","last_became_peered":"2024-04-25T00:09:09.297806+0000","last_unstale":"2024-04-25T00:09:09.304808+0000","last_undegraded":"2024-04-25T00:09:09.304808+0000","last_fullsized":"2024-04-25T00:09:09.304808+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:40.039966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309004+0000","last_change":"2024-04-25T00:09:09.304073+0000","last_active":"2024-04-25T00:09:09.309004+0000","last_peered":"2024-04-25T00:09:09.309004+0000","last_clean":"2024-04-25T00:09:09.309004+0000","last_became_active":"2024-04-25T00:09:09.303202+0000","last_became_peered":"2024-04-25T00:09:09.303202+0000","last_unstale":"2024-04-25T00:09:09.309004+0000","last_undegraded":"2024-04-25T00:09:09.309004+0000","last_fullsized":"2024-04-25T00:09:09.309004+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:42.026281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306875+0000","last_change":"2024-04-25T00:09:09.301875+0000","last_active":"2024-04-25T00:09:10.306875+0000","last_peered":"2024-04-25T00:09:10.306875+0000","last_clean":"2024-04-25T00:09:10.306875+0000","last_became_active":"2024-04-25T00:09:09.301568+0000","last_became_peered":"2024-04-25T00:09:09.301568+0000","last_unstale":"2024-04-25T00:09:10.306875+0000","last_undegraded":"2024-04-25T00:09:10.306875+0000","last_fullsized":"2024-04-25T00:09:10.306875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:56.520243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306367+0000","last_change":"2024-04-25T00:09:09.311603+0000","last_active":"2024-04-25T00:09:10.306367+0000","last_peered":"2024-04-25T00:09:10.306367+0000","last_clean":"2024-04-25T00:09:10.306367+0000","last_became_active":"2024-04-25T00:09:09.311355+0000","last_became_peered":"2024-04-25T00:09:09.311355+0000","last_unstale":"2024-04-25T00:09:10.306367+0000","last_undegraded":"2024-04-25T00:09:10.306367+0000","last_fullsized":"2024-04-25T00:09:10.306367+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:02.205920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.312693+0000","last_change":"2024-04-25T00:09:09.312025+0000","last_active":"2024-04-25T00:09:09.312693+0000","last_peered":"2024-04-25T00:09:09.312693+0000","last_clean":"2024-04-25T00:09:09.312693+0000","last_became_active":"2024-04-25T00:09:09.311749+0000","last_became_peered":"2024-04-25T00:09:09.311749+0000","last_unstale":"2024-04-25T00:09:09.312693+0000","last_undegraded":"2024-04-25T00:09:09.312693+0000","last_fullsized":"2024-04-25T00:09:09.312693+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:27.096818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.312435+0000","last_change":"2024-04-25T00:09:09.310818+0000","last_active":"2024-04-25T00:09:09.312435+0000","last_peered":"2024-04-25T00:09:09.312435+0000","last_clean":"2024-04-25T00:09:09.312435+0000","last_became_active":"2024-04-25T00:09:09.310561+0000","last_became_peered":"2024-04-25T00:09:09.310561+0000","last_unstale":"2024-04-25T00:09:09.312435+0000","last_undegraded":"2024-04-25T00:09:09.312435+0000","last_fullsized":"2024-04-25T00:09:09.312435+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:24.687119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314682+0000","last_change":"2024-04-25T00:09:09.312557+0000","last_active":"2024-04-25T00:09:09.314682+0000","last_peered":"2024-04-25T00:09:09.314682+0000","last_clean":"2024-04-25T00:09:09.314682+0000","last_became_active":"2024-04-25T00:09:09.312176+0000","last_became_peered":"2024-04-25T00:09:09.312176+0000","last_unstale":"2024-04-25T00:09:09.314682+0000","last_undegraded":"2024-04-25T00:09:09.314682+0000","last_fullsized":"2024-04-25T00:09:09.314682+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:42.220507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314202+0000","last_change":"2024-04-25T00:09:09.312759+0000","last_active":"2024-04-25T00:09:09.314202+0000","last_peered":"2024-04-25T00:09:09.314202+0000","last_clean":"2024-04-25T00:09:09.314202+0000","last_became_active":"2024-04-25T00:09:09.312451+0000","last_became_peered":"2024-04-25T00:09:09.312451+0000","last_unstale":"2024-04-25T00:09:09.314202+0000","last_undegraded":"2024-04-25T00:09:09.314202+0000","last_fullsized":"2024-04-25T00:09:09.314202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:48.423861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305443+0000","last_change":"2024-04-25T00:09:09.302357+0000","last_active":"2024-04-25T00:09:09.305443+0000","last_peered":"2024-04-25T00:09:09.305443+0000","last_clean":"2024-04-25T00:09:09.305443+0000","last_became_active":"2024-04-25T00:09:09.301943+0000","last_became_peered":"2024-04-25T00:09:09.301943+0000","last_unstale":"2024-04-25T00:09:09.305443+0000","last_undegraded":"2024-04-25T00:09:09.305443+0000","last_fullsized":"2024-04-25T00:09:09.305443+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:21.712611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952809+0000","last_change":"2024-04-25T00:09:09.314137+0000","last_active":"2024-04-25T00:09:10.952809+0000","last_peered":"2024-04-25T00:09:10.952809+0000","last_clean":"2024-04-25T00:09:10.952809+0000","last_became_active":"2024-04-25T00:09:09.313873+0000","last_became_peered":"2024-04-25T00:09:09.313873+0000","last_unstale":"2024-04-25T00:09:10.952809+0000","last_undegraded":"2024-04-25T00:09:10.952809+0000","last_fullsized":"2024-04-25T00:09:10.952809+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:32.081124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318044+0000","last_change":"2024-04-25T00:09:09.316794+0000","last_active":"2024-04-25T00:09:09.318044+0000","last_peered":"2024-04-25T00:09:09.318044+0000","last_clean":"2024-04-25T00:09:09.318044+0000","last_became_active":"2024-04-25T00:09:09.316533+0000","last_became_peered":"2024-04-25T00:09:09.316533+0000","last_unstale":"2024-04-25T00:09:09.318044+0000","last_undegraded":"2024-04-25T00:09:09.318044+0000","last_fullsized":"2024-04-25T00:09:09.318044+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:06.524960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306910+0000","last_change":"2024-04-25T00:09:09.304158+0000","last_active":"2024-04-25T00:09:10.306910+0000","last_peered":"2024-04-25T00:09:10.306910+0000","last_clean":"2024-04-25T00:09:10.306910+0000","last_became_active":"2024-04-25T00:09:09.303877+0000","last_became_peered":"2024-04-25T00:09:09.303877+0000","last_unstale":"2024-04-25T00:09:10.306910+0000","last_undegraded":"2024-04-25T00:09:10.306910+0000","last_fullsized":"2024-04-25T00:09:10.306910+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:25.602112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952669+0000","last_change":"2024-04-25T00:09:09.305665+0000","last_active":"2024-04-25T00:09:10.952669+0000","last_peered":"2024-04-25T00:09:10.952669+0000","last_clean":"2024-04-25T00:09:10.952669+0000","last_became_active":"2024-04-25T00:09:09.305356+0000","last_became_peered":"2024-04-25T00:09:09.305356+0000","last_unstale":"2024-04-25T00:09:10.952669+0000","last_undegraded":"2024-04-25T00:09:10.952669+0000","last_fullsized":"2024-04-25T00:09:10.952669+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:42.220507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952669+0000","last_change":"2024-04-25T00:09:09.310461+0000","last_active":"2024-04-25T00:09:10.952669+0000","last_peered":"2024-04-25T00:09:10.952669+0000","last_clean":"2024-04-25T00:09:10.952669+0000","last_became_active":"2024-04-25T00:09:09.310205+0000","last_became_peered":"2024-04-25T00:09:09.310205+0000","last_unstale":"2024-04-25T00:09:10.952669+0000","last_undegraded":"2024-04-25T00:09:10.952669+0000","last_fullsized":"2024-04-25T00:09:10.952669+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:57.673575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306765+0000","last_change":"2024-04-25T00:09:09.303670+0000","last_active":"2024-04-25T00:09:10.306765+0000","last_peered":"2024-04-25T00:09:10.306765+0000","last_clean":"2024-04-25T00:09:10.306765+0000","last_became_active":"2024-04-25T00:09:09.303441+0000","last_became_peered":"2024-04-25T00:09:09.303441+0000","last_unstale":"2024-04-25T00:09:10.306765+0000","last_undegraded":"2024-04-25T00:09:10.306765+0000","last_fullsized":"2024-04-25T00:09:10.306765+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:44.035786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307598+0000","last_change":"2024-04-25T00:09:09.311962+0000","last_active":"2024-04-25T00:09:10.307598+0000","last_peered":"2024-04-25T00:09:10.307598+0000","last_clean":"2024-04-25T00:09:10.307598+0000","last_became_active":"2024-04-25T00:09:09.311562+0000","last_became_peered":"2024-04-25T00:09:09.311562+0000","last_unstale":"2024-04-25T00:09:10.307598+0000","last_undegraded":"2024-04-25T00:09:10.307598+0000","last_fullsized":"2024-04-25T00:09:10.307598+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:32.188804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305389+0000","last_change":"2024-04-25T00:09:09.300845+0000","last_active":"2024-04-25T00:09:09.305389+0000","last_peered":"2024-04-25T00:09:09.305389+0000","last_clean":"2024-04-25T00:09:09.305389+0000","last_became_active":"2024-04-25T00:09:09.300586+0000","last_became_peered":"2024-04-25T00:09:09.300586+0000","last_unstale":"2024-04-25T00:09:09.305389+0000","last_undegraded":"2024-04-25T00:09:09.305389+0000","last_fullsized":"2024-04-25T00:09:09.305389+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:12.887814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.302308+0000","last_change":"2024-04-25T00:09:09.301818+0000","last_active":"2024-04-25T00:09:09.302308+0000","last_peered":"2024-04-25T00:09:09.302308+0000","last_clean":"2024-04-25T00:09:09.302308+0000","last_became_active":"2024-04-25T00:09:09.300787+0000","last_became_peered":"2024-04-25T00:09:09.300787+0000","last_unstale":"2024-04-25T00:09:09.302308+0000","last_undegraded":"2024-04-25T00:09:09.302308+0000","last_fullsized":"2024-04-25T00:09:09.302308+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:05.269660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306409+0000","last_change":"2024-04-25T00:09:09.301714+0000","last_active":"2024-04-25T00:09:10.306409+0000","last_peered":"2024-04-25T00:09:10.306409+0000","last_clean":"2024-04-25T00:09:10.306409+0000","last_became_active":"2024-04-25T00:09:09.301206+0000","last_became_peered":"2024-04-25T00:09:09.301206+0000","last_unstale":"2024-04-25T00:09:10.306409+0000","last_undegraded":"2024-04-25T00:09:10.306409+0000","last_fullsized":"2024-04-25T00:09:10.306409+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:03.728720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307836+0000","last_change":"2024-04-25T00:09:09.308944+0000","last_active":"2024-04-25T00:09:10.307836+0000","last_peered":"2024-04-25T00:09:10.307836+0000","last_clean":"2024-04-25T00:09:10.307836+0000","last_became_active":"2024-04-25T00:09:09.308639+0000","last_became_peered":"2024-04-25T00:09:09.308639+0000","last_unstale":"2024-04-25T00:09:10.307836+0000","last_undegraded":"2024-04-25T00:09:10.307836+0000","last_fullsized":"2024-04-25T00:09:10.307836+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:24.518797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952713+0000","last_change":"2024-04-25T00:09:09.305107+0000","last_active":"2024-04-25T00:09:10.952713+0000","last_peered":"2024-04-25T00:09:10.952713+0000","last_clean":"2024-04-25T00:09:10.952713+0000","last_became_active":"2024-04-25T00:09:09.304865+0000","last_became_peered":"2024-04-25T00:09:09.304865+0000","last_unstale":"2024-04-25T00:09:10.952713+0000","last_undegraded":"2024-04-25T00:09:10.952713+0000","last_fullsized":"2024-04-25T00:09:10.952713+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:35.323587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.319363+0000","last_change":"2024-04-25T00:09:09.319003+0000","last_active":"2024-04-25T00:09:09.319363+0000","last_peered":"2024-04-25T00:09:09.319363+0000","last_clean":"2024-04-25T00:09:09.319363+0000","last_became_active":"2024-04-25T00:09:09.318667+0000","last_became_peered":"2024-04-25T00:09:09.318667+0000","last_unstale":"2024-04-25T00:09:09.319363+0000","last_undegraded":"2024-04-25T00:09:09.319363+0000","last_fullsized":"2024-04-25T00:09:09.319363+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:56.587042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309266+0000","last_change":"2024-04-25T00:09:09.308018+0000","last_active":"2024-04-25T00:09:09.309266+0000","last_peered":"2024-04-25T00:09:09.309266+0000","last_clean":"2024-04-25T00:09:09.309266+0000","last_became_active":"2024-04-25T00:09:09.307699+0000","last_became_peered":"2024-04-25T00:09:09.307699+0000","last_unstale":"2024-04-25T00:09:09.309266+0000","last_undegraded":"2024-04-25T00:09:09.309266+0000","last_fullsized":"2024-04-25T00:09:09.309266+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:13.132711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307626+0000","last_change":"2024-04-25T00:09:09.305787+0000","last_active":"2024-04-25T00:09:10.307626+0000","last_peered":"2024-04-25T00:09:10.307626+0000","last_clean":"2024-04-25T00:09:10.307626+0000","last_became_active":"2024-04-25T00:09:09.305510+0000","last_became_peered":"2024-04-25T00:09:09.305510+0000","last_unstale":"2024-04-25T00:09:10.307626+0000","last_undegraded":"2024-04-25T00:09:10.307626+0000","last_fullsized":"2024-04-25T00:09:10.307626+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:52.101238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952454+0000","last_change":"2024-04-25T00:09:09.304975+0000","last_active":"2024-04-25T00:09:10.952454+0000","last_peered":"2024-04-25T00:09:10.952454+0000","last_clean":"2024-04-25T00:09:10.952454+0000","last_became_active":"2024-04-25T00:09:09.304553+0000","last_became_peered":"2024-04-25T00:09:09.304553+0000","last_unstale":"2024-04-25T00:09:10.952454+0000","last_undegraded":"2024-04-25T00:09:10.952454+0000","last_fullsized":"2024-04-25T00:09:10.952454+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:13.132711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305537+0000","last_change":"2024-04-25T00:09:09.302255+0000","last_active":"2024-04-25T00:09:09.305537+0000","last_peered":"2024-04-25T00:09:09.305537+0000","last_clean":"2024-04-25T00:09:09.305537+0000","last_became_active":"2024-04-25T00:09:09.301981+0000","last_became_peered":"2024-04-25T00:09:09.301981+0000","last_unstale":"2024-04-25T00:09:09.305537+0000","last_undegraded":"2024-04-25T00:09:09.305537+0000","last_fullsized":"2024-04-25T00:09:09.305537+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:53.555397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"169'4","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952877+0000","last_change":"2024-04-25T00:09:08.297448+0000","last_active":"2024-04-25T00:09:10.952877+0000","last_peered":"2024-04-25T00:09:10.952877+0000","last_clean":"2024-04-25T00:09:10.952877+0000","last_became_active":"2024-04-25T00:09:08.297174+0000","last_became_peered":"2024-04-25T00:09:08.297174+0000","last_unstale":"2024-04-25T00:09:10.952877+0000","last_undegraded":"2024-04-25T00:09:10.952877+0000","last_fullsized":"2024-04-25T00:09:10.952877+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:15.435259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307115+0000","last_change":"2024-04-25T00:09:09.309300+0000","last_active":"2024-04-25T00:09:10.307115+0000","last_peered":"2024-04-25T00:09:10.307115+0000","last_clean":"2024-04-25T00:09:10.307115+0000","last_became_active":"2024-04-25T00:09:09.309020+0000","last_became_peered":"2024-04-25T00:09:09.309020+0000","last_unstale":"2024-04-25T00:09:10.307115+0000","last_undegraded":"2024-04-25T00:09:10.307115+0000","last_fullsized":"2024-04-25T00:09:10.307115+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:05.525468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"169'1","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.308841+0000","last_change":"2024-04-25T00:09:08.292954+0000","last_active":"2024-04-25T00:09:09.308841+0000","last_peered":"2024-04-25T00:09:09.308841+0000","last_clean":"2024-04-25T00:09:09.308841+0000","last_became_active":"2024-04-25T00:09:08.292691+0000","last_became_peered":"2024-04-25T00:09:08.292691+0000","last_unstale":"2024-04-25T00:09:09.308841+0000","last_undegraded":"2024-04-25T00:09:09.308841+0000","last_fullsized":"2024-04-25T00:09:09.308841+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:25.993778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306181+0000","last_change":"2024-04-25T00:09:09.316123+0000","last_active":"2024-04-25T00:09:10.306181+0000","last_peered":"2024-04-25T00:09:10.306181+0000","last_clean":"2024-04-25T00:09:10.306181+0000","last_became_active":"2024-04-25T00:09:09.315857+0000","last_became_peered":"2024-04-25T00:09:09.315857+0000","last_unstale":"2024-04-25T00:09:10.306181+0000","last_undegraded":"2024-04-25T00:09:10.306181+0000","last_fullsized":"2024-04-25T00:09:10.306181+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:35.963959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307941+0000","last_change":"2024-04-25T00:09:08.296595+0000","last_active":"2024-04-25T00:09:10.307941+0000","last_peered":"2024-04-25T00:09:10.307941+0000","last_clean":"2024-04-25T00:09:10.307941+0000","last_became_active":"2024-04-25T00:09:08.296304+0000","last_became_peered":"2024-04-25T00:09:08.296304+0000","last_unstale":"2024-04-25T00:09:10.307941+0000","last_undegraded":"2024-04-25T00:09:10.307941+0000","last_fullsized":"2024-04-25T00:09:10.307941+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.443443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314161+0000","last_change":"2024-04-25T00:09:09.310600+0000","last_active":"2024-04-25T00:09:09.314161+0000","last_peered":"2024-04-25T00:09:09.314161+0000","last_clean":"2024-04-25T00:09:09.314161+0000","last_became_active":"2024-04-25T00:09:09.310274+0000","last_became_peered":"2024-04-25T00:09:09.310274+0000","last_unstale":"2024-04-25T00:09:09.314161+0000","last_undegraded":"2024-04-25T00:09:09.314161+0000","last_fullsized":"2024-04-25T00:09:09.314161+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:50.448530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318588+0000","last_change":"2024-04-25T00:09:08.291847+0000","last_active":"2024-04-25T00:09:09.318588+0000","last_peered":"2024-04-25T00:09:09.318588+0000","last_clean":"2024-04-25T00:09:09.318588+0000","last_became_active":"2024-04-25T00:09:08.291550+0000","last_became_peered":"2024-04-25T00:09:08.291550+0000","last_unstale":"2024-04-25T00:09:09.318588+0000","last_undegraded":"2024-04-25T00:09:09.318588+0000","last_fullsized":"2024-04-25T00:09:09.318588+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.076036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306931+0000","last_change":"2024-04-25T00:09:09.303097+0000","last_active":"2024-04-25T00:09:10.306931+0000","last_peered":"2024-04-25T00:09:10.306931+0000","last_clean":"2024-04-25T00:09:10.306931+0000","last_became_active":"2024-04-25T00:09:09.302834+0000","last_became_peered":"2024-04-25T00:09:09.302834+0000","last_unstale":"2024-04-25T00:09:10.306931+0000","last_undegraded":"2024-04-25T00:09:10.306931+0000","last_fullsized":"2024-04-25T00:09:10.306931+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:26.402178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952749+0000","last_change":"2024-04-25T00:09:08.286147+0000","last_active":"2024-04-25T00:09:10.952749+0000","last_peered":"2024-04-25T00:09:10.952749+0000","last_clean":"2024-04-25T00:09:10.952749+0000","last_became_active":"2024-04-25T00:09:08.285861+0000","last_became_peered":"2024-04-25T00:09:08.285861+0000","last_unstale":"2024-04-25T00:09:10.952749+0000","last_undegraded":"2024-04-25T00:09:10.952749+0000","last_fullsized":"2024-04-25T00:09:10.952749+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:00:07.103736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305324+0000","last_change":"2024-04-25T00:09:09.300559+0000","last_active":"2024-04-25T00:09:09.305324+0000","last_peered":"2024-04-25T00:09:09.305324+0000","last_clean":"2024-04-25T00:09:09.305324+0000","last_became_active":"2024-04-25T00:09:09.300297+0000","last_became_peered":"2024-04-25T00:09:09.300297+0000","last_unstale":"2024-04-25T00:09:09.305324+0000","last_undegraded":"2024-04-25T00:09:09.305324+0000","last_fullsized":"2024-04-25T00:09:09.305324+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.751439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"169'1","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.313562+0000","last_change":"2024-04-25T00:09:08.293708+0000","last_active":"2024-04-25T00:09:09.313562+0000","last_peered":"2024-04-25T00:09:09.313562+0000","last_clean":"2024-04-25T00:09:09.313562+0000","last_became_active":"2024-04-25T00:09:08.293455+0000","last_became_peered":"2024-04-25T00:09:08.293455+0000","last_unstale":"2024-04-25T00:09:09.313562+0000","last_undegraded":"2024-04-25T00:09:09.313562+0000","last_fullsized":"2024-04-25T00:09:09.313562+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:02.723632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.305980+0000","last_change":"2024-04-25T00:09:09.310794+0000","last_active":"2024-04-25T00:09:10.305980+0000","last_peered":"2024-04-25T00:09:10.305980+0000","last_clean":"2024-04-25T00:09:10.305980+0000","last_became_active":"2024-04-25T00:09:09.310521+0000","last_became_peered":"2024-04-25T00:09:09.310521+0000","last_unstale":"2024-04-25T00:09:10.305980+0000","last_undegraded":"2024-04-25T00:09:10.305980+0000","last_fullsized":"2024-04-25T00:09:10.305980+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:11.257886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309949+0000","last_change":"2024-04-25T00:09:08.296474+0000","last_active":"2024-04-25T00:09:09.309949+0000","last_peered":"2024-04-25T00:09:09.309949+0000","last_clean":"2024-04-25T00:09:09.309949+0000","last_became_active":"2024-04-25T00:09:08.295879+0000","last_became_peered":"2024-04-25T00:09:08.295879+0000","last_unstale":"2024-04-25T00:09:09.309949+0000","last_undegraded":"2024-04-25T00:09:09.309949+0000","last_fullsized":"2024-04-25T00:09:09.309949+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:34.571547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307834+0000","last_change":"2024-04-25T00:09:09.307757+0000","last_active":"2024-04-25T00:09:10.307834+0000","last_peered":"2024-04-25T00:09:10.307834+0000","last_clean":"2024-04-25T00:09:10.307834+0000","last_became_active":"2024-04-25T00:09:09.307521+0000","last_became_peered":"2024-04-25T00:09:09.307521+0000","last_unstale":"2024-04-25T00:09:10.307834+0000","last_undegraded":"2024-04-25T00:09:10.307834+0000","last_fullsized":"2024-04-25T00:09:10.307834+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:48.423861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306505+0000","last_change":"2024-04-25T00:09:08.293592+0000","last_active":"2024-04-25T00:09:10.306505+0000","last_peered":"2024-04-25T00:09:10.306505+0000","last_clean":"2024-04-25T00:09:10.306505+0000","last_became_active":"2024-04-25T00:09:08.292827+0000","last_became_peered":"2024-04-25T00:09:08.292827+0000","last_unstale":"2024-04-25T00:09:10.306505+0000","last_undegraded":"2024-04-25T00:09:10.306505+0000","last_fullsized":"2024-04-25T00:09:10.306505+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:23.682031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.304711+0000","last_change":"2024-04-25T00:09:09.298093+0000","last_active":"2024-04-25T00:09:09.304711+0000","last_peered":"2024-04-25T00:09:09.304711+0000","last_clean":"2024-04-25T00:09:09.304711+0000","last_became_active":"2024-04-25T00:09:09.297754+0000","last_became_peered":"2024-04-25T00:09:09.297754+0000","last_unstale":"2024-04-25T00:09:09.304711+0000","last_undegraded":"2024-04-25T00:09:09.304711+0000","last_fullsized":"2024-04-25T00:09:09.304711+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:45.900104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952795+0000","last_change":"2024-04-25T00:09:08.286726+0000","last_active":"2024-04-25T00:09:10.952795+0000","last_peered":"2024-04-25T00:09:10.952795+0000","last_clean":"2024-04-25T00:09:10.952795+0000","last_became_active":"2024-04-25T00:09:08.286499+0000","last_became_peered":"2024-04-25T00:09:08.286499+0000","last_unstale":"2024-04-25T00:09:10.952795+0000","last_undegraded":"2024-04-25T00:09:10.952795+0000","last_fullsized":"2024-04-25T00:09:10.952795+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.800645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306215+0000","last_change":"2024-04-25T00:09:09.303568+0000","last_active":"2024-04-25T00:09:10.306215+0000","last_peered":"2024-04-25T00:09:10.306215+0000","last_clean":"2024-04-25T00:09:10.306215+0000","last_became_active":"2024-04-25T00:09:09.301599+0000","last_became_peered":"2024-04-25T00:09:09.301599+0000","last_unstale":"2024-04-25T00:09:10.306215+0000","last_undegraded":"2024-04-25T00:09:10.306215+0000","last_fullsized":"2024-04-25T00:09:10.306215+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.979605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"169'3","reported_seq":17,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952631+0000","last_change":"2024-04-25T00:09:08.299474+0000","last_active":"2024-04-25T00:09:10.952631+0000","last_peered":"2024-04-25T00:09:10.952631+0000","last_clean":"2024-04-25T00:09:10.952631+0000","last_became_active":"2024-04-25T00:09:08.299187+0000","last_became_peered":"2024-04-25T00:09:08.299187+0000","last_unstale":"2024-04-25T00:09:10.952631+0000","last_undegraded":"2024-04-25T00:09:10.952631+0000","last_fullsized":"2024-04-25T00:09:10.952631+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:02.180092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307081+0000","last_change":"2024-04-25T00:09:09.301462+0000","last_active":"2024-04-25T00:09:10.307081+0000","last_peered":"2024-04-25T00:09:10.307081+0000","last_clean":"2024-04-25T00:09:10.307081+0000","last_became_active":"2024-04-25T00:09:09.301180+0000","last_became_peered":"2024-04-25T00:09:09.301180+0000","last_unstale":"2024-04-25T00:09:10.307081+0000","last_undegraded":"2024-04-25T00:09:10.307081+0000","last_fullsized":"2024-04-25T00:09:10.307081+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:03.225070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"169'6","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307746+0000","last_change":"2024-04-25T00:09:08.298938+0000","last_active":"2024-04-25T00:09:10.307746+0000","last_peered":"2024-04-25T00:09:10.307746+0000","last_clean":"2024-04-25T00:09:10.307746+0000","last_became_active":"2024-04-25T00:09:08.298660+0000","last_became_peered":"2024-04-25T00:09:08.298660+0000","last_unstale":"2024-04-25T00:09:10.307746+0000","last_undegraded":"2024-04-25T00:09:10.307746+0000","last_fullsized":"2024-04-25T00:09:10.307746+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:00.874216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307963+0000","last_change":"2024-04-25T00:09:09.311849+0000","last_active":"2024-04-25T00:09:10.307963+0000","last_peered":"2024-04-25T00:09:10.307963+0000","last_clean":"2024-04-25T00:09:10.307963+0000","last_became_active":"2024-04-25T00:09:09.311571+0000","last_became_peered":"2024-04-25T00:09:09.311571+0000","last_unstale":"2024-04-25T00:09:10.307963+0000","last_undegraded":"2024-04-25T00:09:10.307963+0000","last_fullsized":"2024-04-25T00:09:10.307963+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:20.625405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.319674+0000","last_change":"2024-04-25T00:09:08.295405+0000","last_active":"2024-04-25T00:09:09.319674+0000","last_peered":"2024-04-25T00:09:09.319674+0000","last_clean":"2024-04-25T00:09:09.319674+0000","last_became_active":"2024-04-25T00:09:08.295101+0000","last_became_peered":"2024-04-25T00:09:08.295101+0000","last_unstale":"2024-04-25T00:09:09.319674+0000","last_undegraded":"2024-04-25T00:09:09.319674+0000","last_fullsized":"2024-04-25T00:09:09.319674+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.554917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306133+0000","last_change":"2024-04-25T00:09:09.303190+0000","last_active":"2024-04-25T00:09:10.306133+0000","last_peered":"2024-04-25T00:09:10.306133+0000","last_clean":"2024-04-25T00:09:10.306133+0000","last_became_active":"2024-04-25T00:09:09.301182+0000","last_became_peered":"2024-04-25T00:09:09.301182+0000","last_unstale":"2024-04-25T00:09:10.306133+0000","last_undegraded":"2024-04-25T00:09:10.306133+0000","last_fullsized":"2024-04-25T00:09:10.306133+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:39.920324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307093+0000","last_change":"2024-04-25T00:09:08.289740+0000","last_active":"2024-04-25T00:09:10.307093+0000","last_peered":"2024-04-25T00:09:10.307093+0000","last_clean":"2024-04-25T00:09:10.307093+0000","last_became_active":"2024-04-25T00:09:08.289461+0000","last_became_peered":"2024-04-25T00:09:08.289461+0000","last_unstale":"2024-04-25T00:09:10.307093+0000","last_undegraded":"2024-04-25T00:09:10.307093+0000","last_fullsized":"2024-04-25T00:09:10.307093+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:46.897885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.315195+0000","last_change":"2024-04-25T00:09:09.314649+0000","last_active":"2024-04-25T00:09:09.315195+0000","last_peered":"2024-04-25T00:09:09.315195+0000","last_clean":"2024-04-25T00:09:09.315195+0000","last_became_active":"2024-04-25T00:09:09.312243+0000","last_became_peered":"2024-04-25T00:09:09.312243+0000","last_unstale":"2024-04-25T00:09:09.315195+0000","last_undegraded":"2024-04-25T00:09:09.315195+0000","last_fullsized":"2024-04-25T00:09:09.315195+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:35.264876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.292433+0000","last_change":"2024-04-25T00:09:08.288812+0000","last_active":"2024-04-25T00:09:09.292433+0000","last_peered":"2024-04-25T00:09:09.292433+0000","last_clean":"2024-04-25T00:09:09.292433+0000","last_became_active":"2024-04-25T00:09:08.288510+0000","last_became_peered":"2024-04-25T00:09:08.288510+0000","last_unstale":"2024-04-25T00:09:09.292433+0000","last_undegraded":"2024-04-25T00:09:09.292433+0000","last_fullsized":"2024-04-25T00:09:09.292433+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:01.200833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307021+0000","last_change":"2024-04-25T00:09:09.311013+0000","last_active":"2024-04-25T00:09:10.307021+0000","last_peered":"2024-04-25T00:09:10.307021+0000","last_clean":"2024-04-25T00:09:10.307021+0000","last_became_active":"2024-04-25T00:09:09.310704+0000","last_became_peered":"2024-04-25T00:09:09.310704+0000","last_unstale":"2024-04-25T00:09:10.307021+0000","last_undegraded":"2024-04-25T00:09:10.307021+0000","last_fullsized":"2024-04-25T00:09:10.307021+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:44.051175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"169'8","reported_seq":20,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.321281+0000","last_change":"2024-04-25T00:09:08.296682+0000","last_active":"2024-04-25T00:09:09.321281+0000","last_peered":"2024-04-25T00:09:09.321281+0000","last_clean":"2024-04-25T00:09:09.321281+0000","last_became_active":"2024-04-25T00:09:08.296235+0000","last_became_peered":"2024-04-25T00:09:08.296235+0000","last_unstale":"2024-04-25T00:09:09.321281+0000","last_undegraded":"2024-04-25T00:09:09.321281+0000","last_fullsized":"2024-04-25T00:09:09.321281+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:04.462272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307133+0000","last_change":"2024-04-25T00:09:09.310080+0000","last_active":"2024-04-25T00:09:10.307133+0000","last_peered":"2024-04-25T00:09:10.307133+0000","last_clean":"2024-04-25T00:09:10.307133+0000","last_became_active":"2024-04-25T00:09:09.309823+0000","last_became_peered":"2024-04-25T00:09:09.309823+0000","last_unstale":"2024-04-25T00:09:10.307133+0000","last_undegraded":"2024-04-25T00:09:10.307133+0000","last_fullsized":"2024-04-25T00:09:10.307133+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:07.835738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952940+0000","last_change":"2024-04-25T00:09:08.286698+0000","last_active":"2024-04-25T00:09:10.952940+0000","last_peered":"2024-04-25T00:09:10.952940+0000","last_clean":"2024-04-25T00:09:10.952940+0000","last_became_active":"2024-04-25T00:09:08.286455+0000","last_became_peered":"2024-04-25T00:09:08.286455+0000","last_unstale":"2024-04-25T00:09:10.952940+0000","last_undegraded":"2024-04-25T00:09:10.952940+0000","last_fullsized":"2024-04-25T00:09:10.952940+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:00.432676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314935+0000","last_change":"2024-04-25T00:09:09.311548+0000","last_active":"2024-04-25T00:09:09.314935+0000","last_peered":"2024-04-25T00:09:09.314935+0000","last_clean":"2024-04-25T00:09:09.314935+0000","last_became_active":"2024-04-25T00:09:09.311293+0000","last_became_peered":"2024-04-25T00:09:09.311293+0000","last_unstale":"2024-04-25T00:09:09.314935+0000","last_undegraded":"2024-04-25T00:09:09.314935+0000","last_fullsized":"2024-04-25T00:09:09.314935+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:57.673575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.313416+0000","last_change":"2024-04-25T00:09:08.290430+0000","last_active":"2024-04-25T00:09:09.313416+0000","last_peered":"2024-04-25T00:09:09.313416+0000","last_clean":"2024-04-25T00:09:09.313416+0000","last_became_active":"2024-04-25T00:09:08.290166+0000","last_became_peered":"2024-04-25T00:09:08.290166+0000","last_unstale":"2024-04-25T00:09:09.313416+0000","last_undegraded":"2024-04-25T00:09:09.313416+0000","last_fullsized":"2024-04-25T00:09:09.313416+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:54.802990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.310145+0000","last_change":"2024-04-25T00:09:09.308904+0000","last_active":"2024-04-25T00:09:09.310145+0000","last_peered":"2024-04-25T00:09:09.310145+0000","last_clean":"2024-04-25T00:09:09.310145+0000","last_became_active":"2024-04-25T00:09:09.308615+0000","last_became_peered":"2024-04-25T00:09:09.308615+0000","last_unstale":"2024-04-25T00:09:09.310145+0000","last_undegraded":"2024-04-25T00:09:09.310145+0000","last_fullsized":"2024-04-25T00:09:09.310145+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:01.879304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"169'1","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.319468+0000","last_change":"2024-04-25T00:09:08.294971+0000","last_active":"2024-04-25T00:09:09.319468+0000","last_peered":"2024-04-25T00:09:09.319468+0000","last_clean":"2024-04-25T00:09:09.319468+0000","last_became_active":"2024-04-25T00:09:08.294684+0000","last_became_peered":"2024-04-25T00:09:08.294684+0000","last_unstale":"2024-04-25T00:09:09.319468+0000","last_undegraded":"2024-04-25T00:09:09.319468+0000","last_fullsized":"2024-04-25T00:09:09.319468+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:41.957166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.310982+0000","last_change":"2024-04-25T00:09:09.308033+0000","last_active":"2024-04-25T00:09:09.310982+0000","last_peered":"2024-04-25T00:09:09.310982+0000","last_clean":"2024-04-25T00:09:09.310982+0000","last_became_active":"2024-04-25T00:09:09.307771+0000","last_became_peered":"2024-04-25T00:09:09.307771+0000","last_unstale":"2024-04-25T00:09:09.310982+0000","last_undegraded":"2024-04-25T00:09:09.310982+0000","last_fullsized":"2024-04-25T00:09:09.310982+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:52.101238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306403+0000","last_change":"2024-04-25T00:09:08.298434+0000","last_active":"2024-04-25T00:09:10.306403+0000","last_peered":"2024-04-25T00:09:10.306403+0000","last_clean":"2024-04-25T00:09:10.306403+0000","last_became_active":"2024-04-25T00:09:08.298098+0000","last_became_peered":"2024-04-25T00:09:08.298098+0000","last_unstale":"2024-04-25T00:09:10.306403+0000","last_undegraded":"2024-04-25T00:09:10.306403+0000","last_fullsized":"2024-04-25T00:09:10.306403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:26.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307091+0000","last_change":"2024-04-25T00:09:09.308278+0000","last_active":"2024-04-25T00:09:10.307091+0000","last_peered":"2024-04-25T00:09:10.307091+0000","last_clean":"2024-04-25T00:09:10.307091+0000","last_became_active":"2024-04-25T00:09:09.308010+0000","last_became_peered":"2024-04-25T00:09:09.308010+0000","last_unstale":"2024-04-25T00:09:10.307091+0000","last_undegraded":"2024-04-25T00:09:10.307091+0000","last_fullsized":"2024-04-25T00:09:10.307091+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:58.164977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307594+0000","last_change":"2024-04-25T00:09:08.297295+0000","last_active":"2024-04-25T00:09:10.307594+0000","last_peered":"2024-04-25T00:09:10.307594+0000","last_clean":"2024-04-25T00:09:10.307594+0000","last_became_active":"2024-04-25T00:09:08.297027+0000","last_became_peered":"2024-04-25T00:09:08.297027+0000","last_unstale":"2024-04-25T00:09:10.307594+0000","last_undegraded":"2024-04-25T00:09:10.307594+0000","last_fullsized":"2024-04-25T00:09:10.307594+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:12.127623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306290+0000","last_change":"2024-04-25T00:09:09.304335+0000","last_active":"2024-04-25T00:09:10.306290+0000","last_peered":"2024-04-25T00:09:10.306290+0000","last_clean":"2024-04-25T00:09:10.306290+0000","last_became_active":"2024-04-25T00:09:09.304082+0000","last_became_peered":"2024-04-25T00:09:09.304082+0000","last_unstale":"2024-04-25T00:09:10.306290+0000","last_undegraded":"2024-04-25T00:09:10.306290+0000","last_fullsized":"2024-04-25T00:09:10.306290+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:45.787992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952519+0000","last_change":"2024-04-25T00:09:08.287319+0000","last_active":"2024-04-25T00:09:10.952519+0000","last_peered":"2024-04-25T00:09:10.952519+0000","last_clean":"2024-04-25T00:09:10.952519+0000","last_became_active":"2024-04-25T00:09:08.287069+0000","last_became_peered":"2024-04-25T00:09:08.287069+0000","last_unstale":"2024-04-25T00:09:10.952519+0000","last_undegraded":"2024-04-25T00:09:10.952519+0000","last_fullsized":"2024-04-25T00:09:10.952519+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:03.594702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318161+0000","last_change":"2024-04-25T00:09:09.317058+0000","last_active":"2024-04-25T00:09:09.318161+0000","last_peered":"2024-04-25T00:09:09.318161+0000","last_clean":"2024-04-25T00:09:09.318161+0000","last_became_active":"2024-04-25T00:09:09.316794+0000","last_became_peered":"2024-04-25T00:09:09.316794+0000","last_unstale":"2024-04-25T00:09:09.318161+0000","last_undegraded":"2024-04-25T00:09:09.318161+0000","last_fullsized":"2024-04-25T00:09:09.318161+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:20.625405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307694+0000","last_change":"2024-04-25T00:09:08.295834+0000","last_active":"2024-04-25T00:09:10.307694+0000","last_peered":"2024-04-25T00:09:10.307694+0000","last_clean":"2024-04-25T00:09:10.307694+0000","last_became_active":"2024-04-25T00:09:08.295553+0000","last_became_peered":"2024-04-25T00:09:08.295553+0000","last_unstale":"2024-04-25T00:09:10.307694+0000","last_undegraded":"2024-04-25T00:09:10.307694+0000","last_fullsized":"2024-04-25T00:09:10.307694+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:56.668487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952420+0000","last_change":"2024-04-25T00:09:09.311578+0000","last_active":"2024-04-25T00:09:10.952420+0000","last_peered":"2024-04-25T00:09:10.952420+0000","last_clean":"2024-04-25T00:09:10.952420+0000","last_became_active":"2024-04-25T00:09:09.311332+0000","last_became_peered":"2024-04-25T00:09:09.311332+0000","last_unstale":"2024-04-25T00:09:10.952420+0000","last_undegraded":"2024-04-25T00:09:10.952420+0000","last_fullsized":"2024-04-25T00:09:10.952420+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:42.026281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298715+0000","last_change":"2024-04-25T00:09:08.292118+0000","last_active":"2024-04-25T00:09:09.298715+0000","last_peered":"2024-04-25T00:09:09.298715+0000","last_clean":"2024-04-25T00:09:09.298715+0000","last_became_active":"2024-04-25T00:09:08.291845+0000","last_became_peered":"2024-04-25T00:09:08.291845+0000","last_unstale":"2024-04-25T00:09:09.298715+0000","last_undegraded":"2024-04-25T00:09:09.298715+0000","last_fullsized":"2024-04-25T00:09:09.298715+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:13.139707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952697+0000","last_change":"2024-04-25T00:09:09.316868+0000","last_active":"2024-04-25T00:09:10.952697+0000","last_peered":"2024-04-25T00:09:10.952697+0000","last_clean":"2024-04-25T00:09:10.952697+0000","last_became_active":"2024-04-25T00:09:09.316605+0000","last_became_peered":"2024-04-25T00:09:09.316605+0000","last_unstale":"2024-04-25T00:09:10.952697+0000","last_undegraded":"2024-04-25T00:09:10.952697+0000","last_fullsized":"2024-04-25T00:09:10.952697+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:35.264876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318149+0000","last_change":"2024-04-25T00:09:08.295803+0000","last_active":"2024-04-25T00:09:09.318149+0000","last_peered":"2024-04-25T00:09:09.318149+0000","last_clean":"2024-04-25T00:09:09.318149+0000","last_became_active":"2024-04-25T00:09:08.295539+0000","last_became_peered":"2024-04-25T00:09:08.295539+0000","last_unstale":"2024-04-25T00:09:09.318149+0000","last_undegraded":"2024-04-25T00:09:09.318149+0000","last_fullsized":"2024-04-25T00:09:09.318149+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:15.787112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307196+0000","last_change":"2024-04-25T00:09:09.308317+0000","last_active":"2024-04-25T00:09:10.307196+0000","last_peered":"2024-04-25T00:09:10.307196+0000","last_clean":"2024-04-25T00:09:10.307196+0000","last_became_active":"2024-04-25T00:09:09.308028+0000","last_became_peered":"2024-04-25T00:09:09.308028+0000","last_unstale":"2024-04-25T00:09:10.307196+0000","last_undegraded":"2024-04-25T00:09:10.307196+0000","last_fullsized":"2024-04-25T00:09:10.307196+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:56.974218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307070+0000","last_change":"2024-04-25T00:09:08.293441+0000","last_active":"2024-04-25T00:09:10.307070+0000","last_peered":"2024-04-25T00:09:10.307070+0000","last_clean":"2024-04-25T00:09:10.307070+0000","last_became_active":"2024-04-25T00:09:08.291906+0000","last_became_peered":"2024-04-25T00:09:08.291906+0000","last_unstale":"2024-04-25T00:09:10.307070+0000","last_undegraded":"2024-04-25T00:09:10.307070+0000","last_fullsized":"2024-04-25T00:09:10.307070+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:16.313266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305531+0000","last_change":"2024-04-25T00:09:09.304379+0000","last_active":"2024-04-25T00:09:09.305531+0000","last_peered":"2024-04-25T00:09:09.305531+0000","last_clean":"2024-04-25T00:09:09.305531+0000","last_became_active":"2024-04-25T00:09:09.304128+0000","last_became_peered":"2024-04-25T00:09:09.304128+0000","last_unstale":"2024-04-25T00:09:09.305531+0000","last_undegraded":"2024-04-25T00:09:09.305531+0000","last_fullsized":"2024-04-25T00:09:09.305531+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:04.078901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307168+0000","last_change":"2024-04-25T00:09:08.291856+0000","last_active":"2024-04-25T00:09:10.307168+0000","last_peered":"2024-04-25T00:09:10.307168+0000","last_clean":"2024-04-25T00:09:10.307168+0000","last_became_active":"2024-04-25T00:09:08.291610+0000","last_became_peered":"2024-04-25T00:09:08.291610+0000","last_unstale":"2024-04-25T00:09:10.307168+0000","last_undegraded":"2024-04-25T00:09:10.307168+0000","last_fullsized":"2024-04-25T00:09:10.307168+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:03.438834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305411+0000","last_change":"2024-04-25T00:09:09.301509+0000","last_active":"2024-04-25T00:09:09.305411+0000","last_peered":"2024-04-25T00:09:09.305411+0000","last_clean":"2024-04-25T00:09:09.305411+0000","last_became_active":"2024-04-25T00:09:09.301123+0000","last_became_peered":"2024-04-25T00:09:09.301123+0000","last_unstale":"2024-04-25T00:09:09.305411+0000","last_undegraded":"2024-04-25T00:09:09.305411+0000","last_fullsized":"2024-04-25T00:09:09.305411+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:07.086387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.293309+0000","last_change":"2024-04-25T00:09:08.298988+0000","last_active":"2024-04-25T00:09:09.293309+0000","last_peered":"2024-04-25T00:09:09.293309+0000","last_clean":"2024-04-25T00:09:09.293309+0000","last_became_active":"2024-04-25T00:09:08.298700+0000","last_became_peered":"2024-04-25T00:09:08.298700+0000","last_unstale":"2024-04-25T00:09:09.293309+0000","last_undegraded":"2024-04-25T00:09:09.293309+0000","last_fullsized":"2024-04-25T00:09:09.293309+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:49.638450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.316940+0000","last_change":"2024-04-25T00:09:09.315615+0000","last_active":"2024-04-25T00:09:09.316940+0000","last_peered":"2024-04-25T00:09:09.316940+0000","last_clean":"2024-04-25T00:09:09.316940+0000","last_became_active":"2024-04-25T00:09:09.315347+0000","last_became_peered":"2024-04-25T00:09:09.315347+0000","last_unstale":"2024-04-25T00:09:09.316940+0000","last_undegraded":"2024-04-25T00:09:09.316940+0000","last_fullsized":"2024-04-25T00:09:09.316940+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:34.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298386+0000","last_change":"2024-04-25T00:09:08.297157+0000","last_active":"2024-04-25T00:09:09.298386+0000","last_peered":"2024-04-25T00:09:09.298386+0000","last_clean":"2024-04-25T00:09:09.298386+0000","last_became_active":"2024-04-25T00:09:08.296829+0000","last_became_peered":"2024-04-25T00:09:08.296829+0000","last_unstale":"2024-04-25T00:09:09.298386+0000","last_undegraded":"2024-04-25T00:09:09.298386+0000","last_fullsized":"2024-04-25T00:09:09.298386+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:35.630831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.317981+0000","last_change":"2024-04-25T00:09:09.316744+0000","last_active":"2024-04-25T00:09:09.317981+0000","last_peered":"2024-04-25T00:09:09.317981+0000","last_clean":"2024-04-25T00:09:09.317981+0000","last_became_active":"2024-04-25T00:09:09.316500+0000","last_became_peered":"2024-04-25T00:09:09.316500+0000","last_unstale":"2024-04-25T00:09:09.317981+0000","last_undegraded":"2024-04-25T00:09:09.317981+0000","last_fullsized":"2024-04-25T00:09:09.317981+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:08.448079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298850+0000","last_change":"2024-04-25T00:09:08.293620+0000","last_active":"2024-04-25T00:09:09.298850+0000","last_peered":"2024-04-25T00:09:09.298850+0000","last_clean":"2024-04-25T00:09:09.298850+0000","last_became_active":"2024-04-25T00:09:08.293342+0000","last_became_peered":"2024-04-25T00:09:08.293342+0000","last_unstale":"2024-04-25T00:09:09.298850+0000","last_undegraded":"2024-04-25T00:09:09.298850+0000","last_fullsized":"2024-04-25T00:09:09.298850+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:33.514019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314173+0000","last_change":"2024-04-25T00:09:09.313418+0000","last_active":"2024-04-25T00:09:09.314173+0000","last_peered":"2024-04-25T00:09:09.314173+0000","last_clean":"2024-04-25T00:09:09.314173+0000","last_became_active":"2024-04-25T00:09:09.313027+0000","last_became_peered":"2024-04-25T00:09:09.313027+0000","last_unstale":"2024-04-25T00:09:09.314173+0000","last_undegraded":"2024-04-25T00:09:09.314173+0000","last_fullsized":"2024-04-25T00:09:09.314173+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:24.518797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298585+0000","last_change":"2024-04-25T00:09:08.291402+0000","last_active":"2024-04-25T00:09:09.298585+0000","last_peered":"2024-04-25T00:09:09.298585+0000","last_clean":"2024-04-25T00:09:09.298585+0000","last_became_active":"2024-04-25T00:09:08.290490+0000","last_became_peered":"2024-04-25T00:09:08.290490+0000","last_unstale":"2024-04-25T00:09:09.298585+0000","last_undegraded":"2024-04-25T00:09:09.298585+0000","last_fullsized":"2024-04-25T00:09:09.298585+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:16.096856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952474+0000","last_change":"2024-04-25T00:09:09.305545+0000","last_active":"2024-04-25T00:09:10.952474+0000","last_peered":"2024-04-25T00:09:10.952474+0000","last_clean":"2024-04-25T00:09:10.952474+0000","last_became_active":"2024-04-25T00:09:09.305249+0000","last_became_peered":"2024-04-25T00:09:09.305249+0000","last_unstale":"2024-04-25T00:09:10.952474+0000","last_undegraded":"2024-04-25T00:09:10.952474+0000","last_fullsized":"2024-04-25T00:09:10.952474+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:01.879304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307517+0000","last_change":"2024-04-25T00:09:08.297106+0000","last_active":"2024-04-25T00:09:10.307517+0000","last_peered":"2024-04-25T00:09:10.307517+0000","last_clean":"2024-04-25T00:09:10.307517+0000","last_became_active":"2024-04-25T00:09:08.296791+0000","last_became_peered":"2024-04-25T00:09:08.296791+0000","last_unstale":"2024-04-25T00:09:10.307517+0000","last_undegraded":"2024-04-25T00:09:10.307517+0000","last_fullsized":"2024-04-25T00:09:10.307517+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:41.021193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306559+0000","last_change":"2024-04-25T00:09:09.303153+0000","last_active":"2024-04-25T00:09:10.306559+0000","last_peered":"2024-04-25T00:09:10.306559+0000","last_clean":"2024-04-25T00:09:10.306559+0000","last_became_active":"2024-04-25T00:09:09.302813+0000","last_became_peered":"2024-04-25T00:09:09.302813+0000","last_unstale":"2024-04-25T00:09:10.306559+0000","last_undegraded":"2024-04-25T00:09:10.306559+0000","last_fullsized":"2024-04-25T00:09:10.306559+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:30.150558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309289+0000","last_change":"2024-04-25T00:09:08.295549+0000","last_active":"2024-04-25T00:09:09.309289+0000","last_peered":"2024-04-25T00:09:09.309289+0000","last_clean":"2024-04-25T00:09:09.309289+0000","last_became_active":"2024-04-25T00:09:08.295227+0000","last_became_peered":"2024-04-25T00:09:08.295227+0000","last_unstale":"2024-04-25T00:09:09.309289+0000","last_undegraded":"2024-04-25T00:09:09.309289+0000","last_fullsized":"2024-04-25T00:09:09.309289+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.354475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":491,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.310137+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:09:09.310137+0000","last_peered":"2024-04-25T00:09:09.310137+0000","last_clean":"2024-04-25T00:09:09.310137+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:09:09.310137+0000","last_undegraded":"2024-04-25T00:09:09.310137+0000","last_fullsized":"2024-04-25T00:09:09.310137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574984,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2727936,"data_stored":1521577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51312,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320528,"statfs":{"total":96636764160,"available":96584220672,"internally_reserved":0,"allocated":2125824,"data_stored":931105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55468,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316372,"statfs":{"total":96636764160,"available":96579964928,"internally_reserved":0,"allocated":2125824,"data_stored":931105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":2154496,"data_stored":939599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2138112,"data_stored":931299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51316,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320524,"statfs":{"total":96636764160,"available":96584216576,"internally_reserved":0,"allocated":2138112,"data_stored":931799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2150400,"data_stored":939017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51908,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319932,"statfs":{"total":96636764160,"available":96583610368,"internally_reserved":0,"allocated":2736128,"data_stored":1527795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:09:12.838 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:09:12.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.969+0000 7f7bf8688640 1 Processor -- start 2024-04-25T00:09:12.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.969+0000 7f7bf8688640 1 -- start start 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf8688640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7bf015e240 0x7f7bf01627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf8688640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf00b5620 0x7f7bf006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf8688640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bf006e510 0x7f7bf0070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf8688640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf0072d90 con 0x7f7bf006e510 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf8688640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf006e930 con 0x7f7bf015e240 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf8688640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf0162d20 con 0x7f7bf00b5620 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bf006e510 0x7f7bf0070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bf006e510 0x7f7bf0070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46044/0 (socket says 172.21.15.113:46044) 2024-04-25T00:09:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 -- 172.21.15.113:0/1599913335 learned_addr learned my addr 172.21.15.113:0/1599913335 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf5c23640 1 --2- 172.21.15.113:0/1599913335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf00b5620 0x7f7bf006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6424640 1 --2- 172.21.15.113:0/1599913335 >> v2:172.21.15.178:3300/0 conn(0x7f7bf015e240 0x7f7bf01627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 -- 172.21.15.113:0/1599913335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf00b5620 msgr2=0x7f7bf006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 --2- 172.21.15.113:0/1599913335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf00b5620 0x7f7bf006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 -- 172.21.15.113:0/1599913335 >> v2:172.21.15.178:3300/0 conn(0x7f7bf015e240 msgr2=0x7f7bf01627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 --2- 172.21.15.113:0/1599913335 >> v2:172.21.15.178:3300/0 conn(0x7f7bf015e240 0x7f7bf01627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6c25640 1 -- 172.21.15.113:0/1599913335 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf00b3c20 con 0x7f7bf006e510 2024-04-25T00:09:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.970+0000 7f7bf6424640 1 --2- 172.21.15.113:0/1599913335 >> v2:172.21.15.178:3300/0 conn(0x7f7bf015e240 0x7f7bf01627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.971+0000 7f7bf6c25640 1 --2- 172.21.15.113:0/1599913335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0070d00 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00628a0 tx=0x7f7be802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3c7915a5720479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.971+0000 7f7bf5422640 1 -- 172.21.15.113:0/1599913335 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8037070 con 0x7f7bf006e510 2024-04-25T00:09:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.971+0000 7f7bf5422640 1 -- 172.21.15.113:0/1599913335 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be8032440 con 0x7f7bf006e510 2024-04-25T00:09:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.971+0000 7f7bf5422640 1 -- 172.21.15.113:0/1599913335 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8032810 con 0x7f7bf006e510 2024-04-25T00:09:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 -- 172.21.15.113:0/1599913335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 msgr2=0x7f7bf0070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 --2- 172.21.15.113:0/1599913335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0070d00 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00628a0 tx=0x7f7be802a210 comp rx=0 tx=0).stop 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 -- 172.21.15.113:0/1599913335 shutdown_connections 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 --2- 172.21.15.113:0/1599913335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0070d00 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 --2- 172.21.15.113:0/1599913335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf00b5620 0x7f7bf006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 --2- 172.21.15.113:0/1599913335 >> v2:172.21.15.178:3300/0 conn(0x7f7bf015e240 0x7f7bf01627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 -- 172.21.15.113:0/1599913335 >> 172.21.15.113:0/1599913335 conn(0x7f7bf00b1a50 msgr2=0x7f7bf00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 -- 172.21.15.113:0/1599913335 shutdown_connections 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.972+0000 7f7bf8688640 1 -- 172.21.15.113:0/1599913335 wait complete. 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.973+0000 7f7bf8688640 1 Processor -- start 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.973+0000 7f7bf8688640 1 -- start start 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf00b5620 0x7f7bf00b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf015e240 0x7f7bf015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf00724f0 con 0x7f7bf006e510 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0072210 con 0x7f7bf00b5620 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf00727d0 con 0x7f7bf015e240 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0142660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46052/0 (socket says 172.21.15.113:46052) 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 -- 172.21.15.113:0/2526207929 learned_addr learned my addr 172.21.15.113:0/2526207929 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6c25640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf015e240 0x7f7bf015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf5c23640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf00b5620 0x7f7bf00b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 -- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf015e240 msgr2=0x7f7bf015f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf015e240 0x7f7bf015f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 -- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf00b5620 msgr2=0x7f7bf00b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf00b5620 0x7f7bf00b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 -- 172.21.15.113:0/2526207929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf00b16a0 con 0x7f7bf006e510 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf6424640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0142660 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007ef0 tx=0x7f7bec00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7be6ffd640 1 -- 172.21.15.113:0/2526207929 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec01a070 con 0x7f7bf006e510 2024-04-25T00:09:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7be6ffd640 1 -- 172.21.15.113:0/2526207929 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bec003b90 con 0x7f7bf006e510 2024-04-25T00:09:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf00b10e0 con 0x7f7bf006e510 2024-04-25T00:09:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7be6ffd640 1 -- 172.21.15.113:0/2526207929 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec004030 con 0x7f7bf006e510 2024-04-25T00:09:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.974+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf0152260 con 0x7f7bf006e510 2024-04-25T00:09:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.975+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf00691a0 con 0x7f7bf006e510 2024-04-25T00:09:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.976+0000 7f7be6ffd640 1 -- 172.21.15.113:0/2526207929 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bec01d070 con 0x7f7bf006e510 2024-04-25T00:09:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.976+0000 7f7be6ffd640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bd406be30 0x7f7bd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.976+0000 7f7bf5c23640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bd406be30 0x7f7bd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.976+0000 7f7be6ffd640 1 -- 172.21.15.113:0/2526207929 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f7bec08b010 con 0x7f7bf006e510 2024-04-25T00:09:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.977+0000 7f7bf5c23640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bd406be30 0x7f7bd406e2f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00ceea0 tx=0x7f7be0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:12.981+0000 7f7be6ffd640 1 -- 172.21.15.113:0/2526207929 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bec010070 con 0x7f7bf006e510 2024-04-25T00:09:13.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.148+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bf0072aa0 con 0x7f7bd406be30 2024-04-25T00:09:13.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.154+0000 7f7be6ffd640 1 -- 172.21.15.113:0/2526207929 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252450 (secure 0 0 0) 0x7f7bf0072aa0 con 0x7f7bd406be30 2024-04-25T00:09:13.157 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:13.158 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bd406be30 msgr2=0x7f7bd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bd406be30 0x7f7bd406e2f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00ceea0 tx=0x7f7be0008040 comp rx=0 tx=0).stop 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 msgr2=0x7f7bf0142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0142660 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007ef0 tx=0x7f7bec00d4a0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 shutdown_connections 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bd406be30 0x7f7bd406e2f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf00b5620 0x7f7bf00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf015e240 0x7f7bf015f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 --2- 172.21.15.113:0/2526207929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf006e510 0x7f7bf0142660 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 >> 172.21.15.113:0/2526207929 conn(0x7f7bf00b1a50 msgr2=0x7f7bf00b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 shutdown_connections 2024-04-25T00:09:13.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.157+0000 7f7bf8688640 1 -- 172.21.15.113:0/2526207929 wait complete. 2024-04-25T00:09:13.168 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-04-25T00:09:11.242151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":254,"num_write_kb":3476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411736,"kb_used_data":17868,"kb_used_omap":12,"kb_used_meta":393715,"kb_avail":754562984,"statfs":{"total":773094113280,"available":772672495616,"internally_reserved":0,"allocated":18296832,"data_stored":8653296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":403164720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000725"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318270+0000","last_change":"2024-04-25T00:09:09.316654+0000","last_active":"2024-04-25T00:09:09.318270+0000","last_peered":"2024-04-25T00:09:09.318270+0000","last_clean":"2024-04-25T00:09:09.318270+0000","last_became_active":"2024-04-25T00:09:09.316354+0000","last_became_peered":"2024-04-25T00:09:09.316354+0000","last_unstale":"2024-04-25T00:09:09.318270+0000","last_undegraded":"2024-04-25T00:09:09.318270+0000","last_fullsized":"2024-04-25T00:09:09.318270+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:34.904655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.311065+0000","last_change":"2024-04-25T00:09:09.307459+0000","last_active":"2024-04-25T00:09:09.311065+0000","last_peered":"2024-04-25T00:09:09.311065+0000","last_clean":"2024-04-25T00:09:09.311065+0000","last_became_active":"2024-04-25T00:09:09.307213+0000","last_became_peered":"2024-04-25T00:09:09.307213+0000","last_unstale":"2024-04-25T00:09:09.311065+0000","last_undegraded":"2024-04-25T00:09:09.311065+0000","last_fullsized":"2024-04-25T00:09:09.311065+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:32.188804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307695+0000","last_change":"2024-04-25T00:09:09.301074+0000","last_active":"2024-04-25T00:09:10.307695+0000","last_peered":"2024-04-25T00:09:10.307695+0000","last_clean":"2024-04-25T00:09:10.307695+0000","last_became_active":"2024-04-25T00:09:09.300775+0000","last_became_peered":"2024-04-25T00:09:09.300775+0000","last_unstale":"2024-04-25T00:09:10.307695+0000","last_undegraded":"2024-04-25T00:09:10.307695+0000","last_fullsized":"2024-04-25T00:09:10.307695+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:24.687119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.311097+0000","last_change":"2024-04-25T00:09:09.306732+0000","last_active":"2024-04-25T00:09:09.311097+0000","last_peered":"2024-04-25T00:09:09.311097+0000","last_clean":"2024-04-25T00:09:09.311097+0000","last_became_active":"2024-04-25T00:09:09.306441+0000","last_became_peered":"2024-04-25T00:09:09.306441+0000","last_unstale":"2024-04-25T00:09:09.311097+0000","last_undegraded":"2024-04-25T00:09:09.311097+0000","last_fullsized":"2024-04-25T00:09:09.311097+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:15.630150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952585+0000","last_change":"2024-04-25T00:09:09.307186+0000","last_active":"2024-04-25T00:09:10.952585+0000","last_peered":"2024-04-25T00:09:10.952585+0000","last_clean":"2024-04-25T00:09:10.952585+0000","last_became_active":"2024-04-25T00:09:09.306907+0000","last_became_peered":"2024-04-25T00:09:09.306907+0000","last_unstale":"2024-04-25T00:09:10.952585+0000","last_undegraded":"2024-04-25T00:09:10.952585+0000","last_fullsized":"2024-04-25T00:09:10.952585+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:50.448530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307829+0000","last_change":"2024-04-25T00:09:09.308940+0000","last_active":"2024-04-25T00:09:10.307829+0000","last_peered":"2024-04-25T00:09:10.307829+0000","last_clean":"2024-04-25T00:09:10.307829+0000","last_became_active":"2024-04-25T00:09:09.308666+0000","last_became_peered":"2024-04-25T00:09:09.308666+0000","last_unstale":"2024-04-25T00:09:10.307829+0000","last_undegraded":"2024-04-25T00:09:10.307829+0000","last_fullsized":"2024-04-25T00:09:10.307829+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:27.096818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.305907+0000","last_change":"2024-04-25T00:09:09.302385+0000","last_active":"2024-04-25T00:09:10.305907+0000","last_peered":"2024-04-25T00:09:10.305907+0000","last_clean":"2024-04-25T00:09:10.305907+0000","last_became_active":"2024-04-25T00:09:09.301541+0000","last_became_peered":"2024-04-25T00:09:09.301541+0000","last_unstale":"2024-04-25T00:09:10.305907+0000","last_undegraded":"2024-04-25T00:09:10.305907+0000","last_fullsized":"2024-04-25T00:09:10.305907+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:07.605187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.304808+0000","last_change":"2024-04-25T00:09:09.298602+0000","last_active":"2024-04-25T00:09:09.304808+0000","last_peered":"2024-04-25T00:09:09.304808+0000","last_clean":"2024-04-25T00:09:09.304808+0000","last_became_active":"2024-04-25T00:09:09.297806+0000","last_became_peered":"2024-04-25T00:09:09.297806+0000","last_unstale":"2024-04-25T00:09:09.304808+0000","last_undegraded":"2024-04-25T00:09:09.304808+0000","last_fullsized":"2024-04-25T00:09:09.304808+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:09:40.039966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309004+0000","last_change":"2024-04-25T00:09:09.304073+0000","last_active":"2024-04-25T00:09:09.309004+0000","last_peered":"2024-04-25T00:09:09.309004+0000","last_clean":"2024-04-25T00:09:09.309004+0000","last_became_active":"2024-04-25T00:09:09.303202+0000","last_became_peered":"2024-04-25T00:09:09.303202+0000","last_unstale":"2024-04-25T00:09:09.309004+0000","last_undegraded":"2024-04-25T00:09:09.309004+0000","last_fullsized":"2024-04-25T00:09:09.309004+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:42.026281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306875+0000","last_change":"2024-04-25T00:09:09.301875+0000","last_active":"2024-04-25T00:09:10.306875+0000","last_peered":"2024-04-25T00:09:10.306875+0000","last_clean":"2024-04-25T00:09:10.306875+0000","last_became_active":"2024-04-25T00:09:09.301568+0000","last_became_peered":"2024-04-25T00:09:09.301568+0000","last_unstale":"2024-04-25T00:09:10.306875+0000","last_undegraded":"2024-04-25T00:09:10.306875+0000","last_fullsized":"2024-04-25T00:09:10.306875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:56.520243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306367+0000","last_change":"2024-04-25T00:09:09.311603+0000","last_active":"2024-04-25T00:09:10.306367+0000","last_peered":"2024-04-25T00:09:10.306367+0000","last_clean":"2024-04-25T00:09:10.306367+0000","last_became_active":"2024-04-25T00:09:09.311355+0000","last_became_peered":"2024-04-25T00:09:09.311355+0000","last_unstale":"2024-04-25T00:09:10.306367+0000","last_undegraded":"2024-04-25T00:09:10.306367+0000","last_fullsized":"2024-04-25T00:09:10.306367+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:02.205920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.312693+0000","last_change":"2024-04-25T00:09:09.312025+0000","last_active":"2024-04-25T00:09:09.312693+0000","last_peered":"2024-04-25T00:09:09.312693+0000","last_clean":"2024-04-25T00:09:09.312693+0000","last_became_active":"2024-04-25T00:09:09.311749+0000","last_became_peered":"2024-04-25T00:09:09.311749+0000","last_unstale":"2024-04-25T00:09:09.312693+0000","last_undegraded":"2024-04-25T00:09:09.312693+0000","last_fullsized":"2024-04-25T00:09:09.312693+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:27.096818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.312435+0000","last_change":"2024-04-25T00:09:09.310818+0000","last_active":"2024-04-25T00:09:09.312435+0000","last_peered":"2024-04-25T00:09:09.312435+0000","last_clean":"2024-04-25T00:09:09.312435+0000","last_became_active":"2024-04-25T00:09:09.310561+0000","last_became_peered":"2024-04-25T00:09:09.310561+0000","last_unstale":"2024-04-25T00:09:09.312435+0000","last_undegraded":"2024-04-25T00:09:09.312435+0000","last_fullsized":"2024-04-25T00:09:09.312435+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:24.687119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314682+0000","last_change":"2024-04-25T00:09:09.312557+0000","last_active":"2024-04-25T00:09:09.314682+0000","last_peered":"2024-04-25T00:09:09.314682+0000","last_clean":"2024-04-25T00:09:09.314682+0000","last_became_active":"2024-04-25T00:09:09.312176+0000","last_became_peered":"2024-04-25T00:09:09.312176+0000","last_unstale":"2024-04-25T00:09:09.314682+0000","last_undegraded":"2024-04-25T00:09:09.314682+0000","last_fullsized":"2024-04-25T00:09:09.314682+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:42.220507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314202+0000","last_change":"2024-04-25T00:09:09.312759+0000","last_active":"2024-04-25T00:09:09.314202+0000","last_peered":"2024-04-25T00:09:09.314202+0000","last_clean":"2024-04-25T00:09:09.314202+0000","last_became_active":"2024-04-25T00:09:09.312451+0000","last_became_peered":"2024-04-25T00:09:09.312451+0000","last_unstale":"2024-04-25T00:09:09.314202+0000","last_undegraded":"2024-04-25T00:09:09.314202+0000","last_fullsized":"2024-04-25T00:09:09.314202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:48.423861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305443+0000","last_change":"2024-04-25T00:09:09.302357+0000","last_active":"2024-04-25T00:09:09.305443+0000","last_peered":"2024-04-25T00:09:09.305443+0000","last_clean":"2024-04-25T00:09:09.305443+0000","last_became_active":"2024-04-25T00:09:09.301943+0000","last_became_peered":"2024-04-25T00:09:09.301943+0000","last_unstale":"2024-04-25T00:09:09.305443+0000","last_undegraded":"2024-04-25T00:09:09.305443+0000","last_fullsized":"2024-04-25T00:09:09.305443+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:21.712611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952809+0000","last_change":"2024-04-25T00:09:09.314137+0000","last_active":"2024-04-25T00:09:10.952809+0000","last_peered":"2024-04-25T00:09:10.952809+0000","last_clean":"2024-04-25T00:09:10.952809+0000","last_became_active":"2024-04-25T00:09:09.313873+0000","last_became_peered":"2024-04-25T00:09:09.313873+0000","last_unstale":"2024-04-25T00:09:10.952809+0000","last_undegraded":"2024-04-25T00:09:10.952809+0000","last_fullsized":"2024-04-25T00:09:10.952809+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:32.081124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318044+0000","last_change":"2024-04-25T00:09:09.316794+0000","last_active":"2024-04-25T00:09:09.318044+0000","last_peered":"2024-04-25T00:09:09.318044+0000","last_clean":"2024-04-25T00:09:09.318044+0000","last_became_active":"2024-04-25T00:09:09.316533+0000","last_became_peered":"2024-04-25T00:09:09.316533+0000","last_unstale":"2024-04-25T00:09:09.318044+0000","last_undegraded":"2024-04-25T00:09:09.318044+0000","last_fullsized":"2024-04-25T00:09:09.318044+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:06.524960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306910+0000","last_change":"2024-04-25T00:09:09.304158+0000","last_active":"2024-04-25T00:09:10.306910+0000","last_peered":"2024-04-25T00:09:10.306910+0000","last_clean":"2024-04-25T00:09:10.306910+0000","last_became_active":"2024-04-25T00:09:09.303877+0000","last_became_peered":"2024-04-25T00:09:09.303877+0000","last_unstale":"2024-04-25T00:09:10.306910+0000","last_undegraded":"2024-04-25T00:09:10.306910+0000","last_fullsized":"2024-04-25T00:09:10.306910+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:25.602112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952669+0000","last_change":"2024-04-25T00:09:09.305665+0000","last_active":"2024-04-25T00:09:10.952669+0000","last_peered":"2024-04-25T00:09:10.952669+0000","last_clean":"2024-04-25T00:09:10.952669+0000","last_became_active":"2024-04-25T00:09:09.305356+0000","last_became_peered":"2024-04-25T00:09:09.305356+0000","last_unstale":"2024-04-25T00:09:10.952669+0000","last_undegraded":"2024-04-25T00:09:10.952669+0000","last_fullsized":"2024-04-25T00:09:10.952669+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:42.220507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952669+0000","last_change":"2024-04-25T00:09:09.310461+0000","last_active":"2024-04-25T00:09:10.952669+0000","last_peered":"2024-04-25T00:09:10.952669+0000","last_clean":"2024-04-25T00:09:10.952669+0000","last_became_active":"2024-04-25T00:09:09.310205+0000","last_became_peered":"2024-04-25T00:09:09.310205+0000","last_unstale":"2024-04-25T00:09:10.952669+0000","last_undegraded":"2024-04-25T00:09:10.952669+0000","last_fullsized":"2024-04-25T00:09:10.952669+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:57.673575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306765+0000","last_change":"2024-04-25T00:09:09.303670+0000","last_active":"2024-04-25T00:09:10.306765+0000","last_peered":"2024-04-25T00:09:10.306765+0000","last_clean":"2024-04-25T00:09:10.306765+0000","last_became_active":"2024-04-25T00:09:09.303441+0000","last_became_peered":"2024-04-25T00:09:09.303441+0000","last_unstale":"2024-04-25T00:09:10.306765+0000","last_undegraded":"2024-04-25T00:09:10.306765+0000","last_fullsized":"2024-04-25T00:09:10.306765+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:28:44.035786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307598+0000","last_change":"2024-04-25T00:09:09.311962+0000","last_active":"2024-04-25T00:09:10.307598+0000","last_peered":"2024-04-25T00:09:10.307598+0000","last_clean":"2024-04-25T00:09:10.307598+0000","last_became_active":"2024-04-25T00:09:09.311562+0000","last_became_peered":"2024-04-25T00:09:09.311562+0000","last_unstale":"2024-04-25T00:09:10.307598+0000","last_undegraded":"2024-04-25T00:09:10.307598+0000","last_fullsized":"2024-04-25T00:09:10.307598+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:32.188804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305389+0000","last_change":"2024-04-25T00:09:09.300845+0000","last_active":"2024-04-25T00:09:09.305389+0000","last_peered":"2024-04-25T00:09:09.305389+0000","last_clean":"2024-04-25T00:09:09.305389+0000","last_became_active":"2024-04-25T00:09:09.300586+0000","last_became_peered":"2024-04-25T00:09:09.300586+0000","last_unstale":"2024-04-25T00:09:09.305389+0000","last_undegraded":"2024-04-25T00:09:09.305389+0000","last_fullsized":"2024-04-25T00:09:09.305389+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:12.887814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.302308+0000","last_change":"2024-04-25T00:09:09.301818+0000","last_active":"2024-04-25T00:09:09.302308+0000","last_peered":"2024-04-25T00:09:09.302308+0000","last_clean":"2024-04-25T00:09:09.302308+0000","last_became_active":"2024-04-25T00:09:09.300787+0000","last_became_peered":"2024-04-25T00:09:09.300787+0000","last_unstale":"2024-04-25T00:09:09.302308+0000","last_undegraded":"2024-04-25T00:09:09.302308+0000","last_fullsized":"2024-04-25T00:09:09.302308+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:05.269660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306409+0000","last_change":"2024-04-25T00:09:09.301714+0000","last_active":"2024-04-25T00:09:10.306409+0000","last_peered":"2024-04-25T00:09:10.306409+0000","last_clean":"2024-04-25T00:09:10.306409+0000","last_became_active":"2024-04-25T00:09:09.301206+0000","last_became_peered":"2024-04-25T00:09:09.301206+0000","last_unstale":"2024-04-25T00:09:10.306409+0000","last_undegraded":"2024-04-25T00:09:10.306409+0000","last_fullsized":"2024-04-25T00:09:10.306409+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:03.728720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307836+0000","last_change":"2024-04-25T00:09:09.308944+0000","last_active":"2024-04-25T00:09:10.307836+0000","last_peered":"2024-04-25T00:09:10.307836+0000","last_clean":"2024-04-25T00:09:10.307836+0000","last_became_active":"2024-04-25T00:09:09.308639+0000","last_became_peered":"2024-04-25T00:09:09.308639+0000","last_unstale":"2024-04-25T00:09:10.307836+0000","last_undegraded":"2024-04-25T00:09:10.307836+0000","last_fullsized":"2024-04-25T00:09:10.307836+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:24.518797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952713+0000","last_change":"2024-04-25T00:09:09.305107+0000","last_active":"2024-04-25T00:09:10.952713+0000","last_peered":"2024-04-25T00:09:10.952713+0000","last_clean":"2024-04-25T00:09:10.952713+0000","last_became_active":"2024-04-25T00:09:09.304865+0000","last_became_peered":"2024-04-25T00:09:09.304865+0000","last_unstale":"2024-04-25T00:09:10.952713+0000","last_undegraded":"2024-04-25T00:09:10.952713+0000","last_fullsized":"2024-04-25T00:09:10.952713+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:35.323587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.319363+0000","last_change":"2024-04-25T00:09:09.319003+0000","last_active":"2024-04-25T00:09:09.319363+0000","last_peered":"2024-04-25T00:09:09.319363+0000","last_clean":"2024-04-25T00:09:09.319363+0000","last_became_active":"2024-04-25T00:09:09.318667+0000","last_became_peered":"2024-04-25T00:09:09.318667+0000","last_unstale":"2024-04-25T00:09:09.319363+0000","last_undegraded":"2024-04-25T00:09:09.319363+0000","last_fullsized":"2024-04-25T00:09:09.319363+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:56.587042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309266+0000","last_change":"2024-04-25T00:09:09.308018+0000","last_active":"2024-04-25T00:09:09.309266+0000","last_peered":"2024-04-25T00:09:09.309266+0000","last_clean":"2024-04-25T00:09:09.309266+0000","last_became_active":"2024-04-25T00:09:09.307699+0000","last_became_peered":"2024-04-25T00:09:09.307699+0000","last_unstale":"2024-04-25T00:09:09.309266+0000","last_undegraded":"2024-04-25T00:09:09.309266+0000","last_fullsized":"2024-04-25T00:09:09.309266+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:13.132711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307626+0000","last_change":"2024-04-25T00:09:09.305787+0000","last_active":"2024-04-25T00:09:10.307626+0000","last_peered":"2024-04-25T00:09:10.307626+0000","last_clean":"2024-04-25T00:09:10.307626+0000","last_became_active":"2024-04-25T00:09:09.305510+0000","last_became_peered":"2024-04-25T00:09:09.305510+0000","last_unstale":"2024-04-25T00:09:10.307626+0000","last_undegraded":"2024-04-25T00:09:10.307626+0000","last_fullsized":"2024-04-25T00:09:10.307626+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:52.101238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952454+0000","last_change":"2024-04-25T00:09:09.304975+0000","last_active":"2024-04-25T00:09:10.952454+0000","last_peered":"2024-04-25T00:09:10.952454+0000","last_clean":"2024-04-25T00:09:10.952454+0000","last_became_active":"2024-04-25T00:09:09.304553+0000","last_became_peered":"2024-04-25T00:09:09.304553+0000","last_unstale":"2024-04-25T00:09:10.952454+0000","last_undegraded":"2024-04-25T00:09:10.952454+0000","last_fullsized":"2024-04-25T00:09:10.952454+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:13.132711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305537+0000","last_change":"2024-04-25T00:09:09.302255+0000","last_active":"2024-04-25T00:09:09.305537+0000","last_peered":"2024-04-25T00:09:09.305537+0000","last_clean":"2024-04-25T00:09:09.305537+0000","last_became_active":"2024-04-25T00:09:09.301981+0000","last_became_peered":"2024-04-25T00:09:09.301981+0000","last_unstale":"2024-04-25T00:09:09.305537+0000","last_undegraded":"2024-04-25T00:09:09.305537+0000","last_fullsized":"2024-04-25T00:09:09.305537+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:53.555397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"169'4","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952877+0000","last_change":"2024-04-25T00:09:08.297448+0000","last_active":"2024-04-25T00:09:10.952877+0000","last_peered":"2024-04-25T00:09:10.952877+0000","last_clean":"2024-04-25T00:09:10.952877+0000","last_became_active":"2024-04-25T00:09:08.297174+0000","last_became_peered":"2024-04-25T00:09:08.297174+0000","last_unstale":"2024-04-25T00:09:10.952877+0000","last_undegraded":"2024-04-25T00:09:10.952877+0000","last_fullsized":"2024-04-25T00:09:10.952877+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:15.435259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307115+0000","last_change":"2024-04-25T00:09:09.309300+0000","last_active":"2024-04-25T00:09:10.307115+0000","last_peered":"2024-04-25T00:09:10.307115+0000","last_clean":"2024-04-25T00:09:10.307115+0000","last_became_active":"2024-04-25T00:09:09.309020+0000","last_became_peered":"2024-04-25T00:09:09.309020+0000","last_unstale":"2024-04-25T00:09:10.307115+0000","last_undegraded":"2024-04-25T00:09:10.307115+0000","last_fullsized":"2024-04-25T00:09:10.307115+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:05.525468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"169'1","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.308841+0000","last_change":"2024-04-25T00:09:08.292954+0000","last_active":"2024-04-25T00:09:09.308841+0000","last_peered":"2024-04-25T00:09:09.308841+0000","last_clean":"2024-04-25T00:09:09.308841+0000","last_became_active":"2024-04-25T00:09:08.292691+0000","last_became_peered":"2024-04-25T00:09:08.292691+0000","last_unstale":"2024-04-25T00:09:09.308841+0000","last_undegraded":"2024-04-25T00:09:09.308841+0000","last_fullsized":"2024-04-25T00:09:09.308841+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:54:25.993778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306181+0000","last_change":"2024-04-25T00:09:09.316123+0000","last_active":"2024-04-25T00:09:10.306181+0000","last_peered":"2024-04-25T00:09:10.306181+0000","last_clean":"2024-04-25T00:09:10.306181+0000","last_became_active":"2024-04-25T00:09:09.315857+0000","last_became_peered":"2024-04-25T00:09:09.315857+0000","last_unstale":"2024-04-25T00:09:10.306181+0000","last_undegraded":"2024-04-25T00:09:10.306181+0000","last_fullsized":"2024-04-25T00:09:10.306181+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:35.963959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307941+0000","last_change":"2024-04-25T00:09:08.296595+0000","last_active":"2024-04-25T00:09:10.307941+0000","last_peered":"2024-04-25T00:09:10.307941+0000","last_clean":"2024-04-25T00:09:10.307941+0000","last_became_active":"2024-04-25T00:09:08.296304+0000","last_became_peered":"2024-04-25T00:09:08.296304+0000","last_unstale":"2024-04-25T00:09:10.307941+0000","last_undegraded":"2024-04-25T00:09:10.307941+0000","last_fullsized":"2024-04-25T00:09:10.307941+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.443443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314161+0000","last_change":"2024-04-25T00:09:09.310600+0000","last_active":"2024-04-25T00:09:09.314161+0000","last_peered":"2024-04-25T00:09:09.314161+0000","last_clean":"2024-04-25T00:09:09.314161+0000","last_became_active":"2024-04-25T00:09:09.310274+0000","last_became_peered":"2024-04-25T00:09:09.310274+0000","last_unstale":"2024-04-25T00:09:09.314161+0000","last_undegraded":"2024-04-25T00:09:09.314161+0000","last_fullsized":"2024-04-25T00:09:09.314161+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:50.448530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318588+0000","last_change":"2024-04-25T00:09:08.291847+0000","last_active":"2024-04-25T00:09:09.318588+0000","last_peered":"2024-04-25T00:09:09.318588+0000","last_clean":"2024-04-25T00:09:09.318588+0000","last_became_active":"2024-04-25T00:09:08.291550+0000","last_became_peered":"2024-04-25T00:09:08.291550+0000","last_unstale":"2024-04-25T00:09:09.318588+0000","last_undegraded":"2024-04-25T00:09:09.318588+0000","last_fullsized":"2024-04-25T00:09:09.318588+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:31.076036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306931+0000","last_change":"2024-04-25T00:09:09.303097+0000","last_active":"2024-04-25T00:09:10.306931+0000","last_peered":"2024-04-25T00:09:10.306931+0000","last_clean":"2024-04-25T00:09:10.306931+0000","last_became_active":"2024-04-25T00:09:09.302834+0000","last_became_peered":"2024-04-25T00:09:09.302834+0000","last_unstale":"2024-04-25T00:09:10.306931+0000","last_undegraded":"2024-04-25T00:09:10.306931+0000","last_fullsized":"2024-04-25T00:09:10.306931+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:26.402178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952749+0000","last_change":"2024-04-25T00:09:08.286147+0000","last_active":"2024-04-25T00:09:10.952749+0000","last_peered":"2024-04-25T00:09:10.952749+0000","last_clean":"2024-04-25T00:09:10.952749+0000","last_became_active":"2024-04-25T00:09:08.285861+0000","last_became_peered":"2024-04-25T00:09:08.285861+0000","last_unstale":"2024-04-25T00:09:10.952749+0000","last_undegraded":"2024-04-25T00:09:10.952749+0000","last_fullsized":"2024-04-25T00:09:10.952749+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:00:07.103736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305324+0000","last_change":"2024-04-25T00:09:09.300559+0000","last_active":"2024-04-25T00:09:09.305324+0000","last_peered":"2024-04-25T00:09:09.305324+0000","last_clean":"2024-04-25T00:09:09.305324+0000","last_became_active":"2024-04-25T00:09:09.300297+0000","last_became_peered":"2024-04-25T00:09:09.300297+0000","last_unstale":"2024-04-25T00:09:09.305324+0000","last_undegraded":"2024-04-25T00:09:09.305324+0000","last_fullsized":"2024-04-25T00:09:09.305324+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.751439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"169'1","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.313562+0000","last_change":"2024-04-25T00:09:08.293708+0000","last_active":"2024-04-25T00:09:09.313562+0000","last_peered":"2024-04-25T00:09:09.313562+0000","last_clean":"2024-04-25T00:09:09.313562+0000","last_became_active":"2024-04-25T00:09:08.293455+0000","last_became_peered":"2024-04-25T00:09:08.293455+0000","last_unstale":"2024-04-25T00:09:09.313562+0000","last_undegraded":"2024-04-25T00:09:09.313562+0000","last_fullsized":"2024-04-25T00:09:09.313562+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:02.723632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.305980+0000","last_change":"2024-04-25T00:09:09.310794+0000","last_active":"2024-04-25T00:09:10.305980+0000","last_peered":"2024-04-25T00:09:10.305980+0000","last_clean":"2024-04-25T00:09:10.305980+0000","last_became_active":"2024-04-25T00:09:09.310521+0000","last_became_peered":"2024-04-25T00:09:09.310521+0000","last_unstale":"2024-04-25T00:09:10.305980+0000","last_undegraded":"2024-04-25T00:09:10.305980+0000","last_fullsized":"2024-04-25T00:09:10.305980+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:11.257886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309949+0000","last_change":"2024-04-25T00:09:08.296474+0000","last_active":"2024-04-25T00:09:09.309949+0000","last_peered":"2024-04-25T00:09:09.309949+0000","last_clean":"2024-04-25T00:09:09.309949+0000","last_became_active":"2024-04-25T00:09:08.295879+0000","last_became_peered":"2024-04-25T00:09:08.295879+0000","last_unstale":"2024-04-25T00:09:09.309949+0000","last_undegraded":"2024-04-25T00:09:09.309949+0000","last_fullsized":"2024-04-25T00:09:09.309949+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:34.571547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307834+0000","last_change":"2024-04-25T00:09:09.307757+0000","last_active":"2024-04-25T00:09:10.307834+0000","last_peered":"2024-04-25T00:09:10.307834+0000","last_clean":"2024-04-25T00:09:10.307834+0000","last_became_active":"2024-04-25T00:09:09.307521+0000","last_became_peered":"2024-04-25T00:09:09.307521+0000","last_unstale":"2024-04-25T00:09:10.307834+0000","last_undegraded":"2024-04-25T00:09:10.307834+0000","last_fullsized":"2024-04-25T00:09:10.307834+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:48.423861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306505+0000","last_change":"2024-04-25T00:09:08.293592+0000","last_active":"2024-04-25T00:09:10.306505+0000","last_peered":"2024-04-25T00:09:10.306505+0000","last_clean":"2024-04-25T00:09:10.306505+0000","last_became_active":"2024-04-25T00:09:08.292827+0000","last_became_peered":"2024-04-25T00:09:08.292827+0000","last_unstale":"2024-04-25T00:09:10.306505+0000","last_undegraded":"2024-04-25T00:09:10.306505+0000","last_fullsized":"2024-04-25T00:09:10.306505+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:23.682031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.304711+0000","last_change":"2024-04-25T00:09:09.298093+0000","last_active":"2024-04-25T00:09:09.304711+0000","last_peered":"2024-04-25T00:09:09.304711+0000","last_clean":"2024-04-25T00:09:09.304711+0000","last_became_active":"2024-04-25T00:09:09.297754+0000","last_became_peered":"2024-04-25T00:09:09.297754+0000","last_unstale":"2024-04-25T00:09:09.304711+0000","last_undegraded":"2024-04-25T00:09:09.304711+0000","last_fullsized":"2024-04-25T00:09:09.304711+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:45.900104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952795+0000","last_change":"2024-04-25T00:09:08.286726+0000","last_active":"2024-04-25T00:09:10.952795+0000","last_peered":"2024-04-25T00:09:10.952795+0000","last_clean":"2024-04-25T00:09:10.952795+0000","last_became_active":"2024-04-25T00:09:08.286499+0000","last_became_peered":"2024-04-25T00:09:08.286499+0000","last_unstale":"2024-04-25T00:09:10.952795+0000","last_undegraded":"2024-04-25T00:09:10.952795+0000","last_fullsized":"2024-04-25T00:09:10.952795+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:00.800645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306215+0000","last_change":"2024-04-25T00:09:09.303568+0000","last_active":"2024-04-25T00:09:10.306215+0000","last_peered":"2024-04-25T00:09:10.306215+0000","last_clean":"2024-04-25T00:09:10.306215+0000","last_became_active":"2024-04-25T00:09:09.301599+0000","last_became_peered":"2024-04-25T00:09:09.301599+0000","last_unstale":"2024-04-25T00:09:10.306215+0000","last_undegraded":"2024-04-25T00:09:10.306215+0000","last_fullsized":"2024-04-25T00:09:10.306215+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:29.979605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"169'3","reported_seq":17,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952631+0000","last_change":"2024-04-25T00:09:08.299474+0000","last_active":"2024-04-25T00:09:10.952631+0000","last_peered":"2024-04-25T00:09:10.952631+0000","last_clean":"2024-04-25T00:09:10.952631+0000","last_became_active":"2024-04-25T00:09:08.299187+0000","last_became_peered":"2024-04-25T00:09:08.299187+0000","last_unstale":"2024-04-25T00:09:10.952631+0000","last_undegraded":"2024-04-25T00:09:10.952631+0000","last_fullsized":"2024-04-25T00:09:10.952631+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:02.180092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307081+0000","last_change":"2024-04-25T00:09:09.301462+0000","last_active":"2024-04-25T00:09:10.307081+0000","last_peered":"2024-04-25T00:09:10.307081+0000","last_clean":"2024-04-25T00:09:10.307081+0000","last_became_active":"2024-04-25T00:09:09.301180+0000","last_became_peered":"2024-04-25T00:09:09.301180+0000","last_unstale":"2024-04-25T00:09:10.307081+0000","last_undegraded":"2024-04-25T00:09:10.307081+0000","last_fullsized":"2024-04-25T00:09:10.307081+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:03.225070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"169'6","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307746+0000","last_change":"2024-04-25T00:09:08.298938+0000","last_active":"2024-04-25T00:09:10.307746+0000","last_peered":"2024-04-25T00:09:10.307746+0000","last_clean":"2024-04-25T00:09:10.307746+0000","last_became_active":"2024-04-25T00:09:08.298660+0000","last_became_peered":"2024-04-25T00:09:08.298660+0000","last_unstale":"2024-04-25T00:09:10.307746+0000","last_undegraded":"2024-04-25T00:09:10.307746+0000","last_fullsized":"2024-04-25T00:09:10.307746+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:00.874216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307963+0000","last_change":"2024-04-25T00:09:09.311849+0000","last_active":"2024-04-25T00:09:10.307963+0000","last_peered":"2024-04-25T00:09:10.307963+0000","last_clean":"2024-04-25T00:09:10.307963+0000","last_became_active":"2024-04-25T00:09:09.311571+0000","last_became_peered":"2024-04-25T00:09:09.311571+0000","last_unstale":"2024-04-25T00:09:10.307963+0000","last_undegraded":"2024-04-25T00:09:10.307963+0000","last_fullsized":"2024-04-25T00:09:10.307963+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:20.625405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.319674+0000","last_change":"2024-04-25T00:09:08.295405+0000","last_active":"2024-04-25T00:09:09.319674+0000","last_peered":"2024-04-25T00:09:09.319674+0000","last_clean":"2024-04-25T00:09:09.319674+0000","last_became_active":"2024-04-25T00:09:08.295101+0000","last_became_peered":"2024-04-25T00:09:08.295101+0000","last_unstale":"2024-04-25T00:09:09.319674+0000","last_undegraded":"2024-04-25T00:09:09.319674+0000","last_fullsized":"2024-04-25T00:09:09.319674+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.554917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306133+0000","last_change":"2024-04-25T00:09:09.303190+0000","last_active":"2024-04-25T00:09:10.306133+0000","last_peered":"2024-04-25T00:09:10.306133+0000","last_clean":"2024-04-25T00:09:10.306133+0000","last_became_active":"2024-04-25T00:09:09.301182+0000","last_became_peered":"2024-04-25T00:09:09.301182+0000","last_unstale":"2024-04-25T00:09:10.306133+0000","last_undegraded":"2024-04-25T00:09:10.306133+0000","last_fullsized":"2024-04-25T00:09:10.306133+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:39.920324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307093+0000","last_change":"2024-04-25T00:09:08.289740+0000","last_active":"2024-04-25T00:09:10.307093+0000","last_peered":"2024-04-25T00:09:10.307093+0000","last_clean":"2024-04-25T00:09:10.307093+0000","last_became_active":"2024-04-25T00:09:08.289461+0000","last_became_peered":"2024-04-25T00:09:08.289461+0000","last_unstale":"2024-04-25T00:09:10.307093+0000","last_undegraded":"2024-04-25T00:09:10.307093+0000","last_fullsized":"2024-04-25T00:09:10.307093+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:46.897885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.315195+0000","last_change":"2024-04-25T00:09:09.314649+0000","last_active":"2024-04-25T00:09:09.315195+0000","last_peered":"2024-04-25T00:09:09.315195+0000","last_clean":"2024-04-25T00:09:09.315195+0000","last_became_active":"2024-04-25T00:09:09.312243+0000","last_became_peered":"2024-04-25T00:09:09.312243+0000","last_unstale":"2024-04-25T00:09:09.315195+0000","last_undegraded":"2024-04-25T00:09:09.315195+0000","last_fullsized":"2024-04-25T00:09:09.315195+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:35.264876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.292433+0000","last_change":"2024-04-25T00:09:08.288812+0000","last_active":"2024-04-25T00:09:09.292433+0000","last_peered":"2024-04-25T00:09:09.292433+0000","last_clean":"2024-04-25T00:09:09.292433+0000","last_became_active":"2024-04-25T00:09:08.288510+0000","last_became_peered":"2024-04-25T00:09:08.288510+0000","last_unstale":"2024-04-25T00:09:09.292433+0000","last_undegraded":"2024-04-25T00:09:09.292433+0000","last_fullsized":"2024-04-25T00:09:09.292433+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:01.200833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307021+0000","last_change":"2024-04-25T00:09:09.311013+0000","last_active":"2024-04-25T00:09:10.307021+0000","last_peered":"2024-04-25T00:09:10.307021+0000","last_clean":"2024-04-25T00:09:10.307021+0000","last_became_active":"2024-04-25T00:09:09.310704+0000","last_became_peered":"2024-04-25T00:09:09.310704+0000","last_unstale":"2024-04-25T00:09:10.307021+0000","last_undegraded":"2024-04-25T00:09:10.307021+0000","last_fullsized":"2024-04-25T00:09:10.307021+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:56:44.051175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"169'8","reported_seq":20,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.321281+0000","last_change":"2024-04-25T00:09:08.296682+0000","last_active":"2024-04-25T00:09:09.321281+0000","last_peered":"2024-04-25T00:09:09.321281+0000","last_clean":"2024-04-25T00:09:09.321281+0000","last_became_active":"2024-04-25T00:09:08.296235+0000","last_became_peered":"2024-04-25T00:09:08.296235+0000","last_unstale":"2024-04-25T00:09:09.321281+0000","last_undegraded":"2024-04-25T00:09:09.321281+0000","last_fullsized":"2024-04-25T00:09:09.321281+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:04.462272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307133+0000","last_change":"2024-04-25T00:09:09.310080+0000","last_active":"2024-04-25T00:09:10.307133+0000","last_peered":"2024-04-25T00:09:10.307133+0000","last_clean":"2024-04-25T00:09:10.307133+0000","last_became_active":"2024-04-25T00:09:09.309823+0000","last_became_peered":"2024-04-25T00:09:09.309823+0000","last_unstale":"2024-04-25T00:09:10.307133+0000","last_undegraded":"2024-04-25T00:09:10.307133+0000","last_fullsized":"2024-04-25T00:09:10.307133+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:07.835738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952940+0000","last_change":"2024-04-25T00:09:08.286698+0000","last_active":"2024-04-25T00:09:10.952940+0000","last_peered":"2024-04-25T00:09:10.952940+0000","last_clean":"2024-04-25T00:09:10.952940+0000","last_became_active":"2024-04-25T00:09:08.286455+0000","last_became_peered":"2024-04-25T00:09:08.286455+0000","last_unstale":"2024-04-25T00:09:10.952940+0000","last_undegraded":"2024-04-25T00:09:10.952940+0000","last_fullsized":"2024-04-25T00:09:10.952940+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:00.432676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314935+0000","last_change":"2024-04-25T00:09:09.311548+0000","last_active":"2024-04-25T00:09:09.314935+0000","last_peered":"2024-04-25T00:09:09.314935+0000","last_clean":"2024-04-25T00:09:09.314935+0000","last_became_active":"2024-04-25T00:09:09.311293+0000","last_became_peered":"2024-04-25T00:09:09.311293+0000","last_unstale":"2024-04-25T00:09:09.314935+0000","last_undegraded":"2024-04-25T00:09:09.314935+0000","last_fullsized":"2024-04-25T00:09:09.314935+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:57.673575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.313416+0000","last_change":"2024-04-25T00:09:08.290430+0000","last_active":"2024-04-25T00:09:09.313416+0000","last_peered":"2024-04-25T00:09:09.313416+0000","last_clean":"2024-04-25T00:09:09.313416+0000","last_became_active":"2024-04-25T00:09:08.290166+0000","last_became_peered":"2024-04-25T00:09:08.290166+0000","last_unstale":"2024-04-25T00:09:09.313416+0000","last_undegraded":"2024-04-25T00:09:09.313416+0000","last_fullsized":"2024-04-25T00:09:09.313416+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:23:54.802990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.310145+0000","last_change":"2024-04-25T00:09:09.308904+0000","last_active":"2024-04-25T00:09:09.310145+0000","last_peered":"2024-04-25T00:09:09.310145+0000","last_clean":"2024-04-25T00:09:09.310145+0000","last_became_active":"2024-04-25T00:09:09.308615+0000","last_became_peered":"2024-04-25T00:09:09.308615+0000","last_unstale":"2024-04-25T00:09:09.310145+0000","last_undegraded":"2024-04-25T00:09:09.310145+0000","last_fullsized":"2024-04-25T00:09:09.310145+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:01.879304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"169'1","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.319468+0000","last_change":"2024-04-25T00:09:08.294971+0000","last_active":"2024-04-25T00:09:09.319468+0000","last_peered":"2024-04-25T00:09:09.319468+0000","last_clean":"2024-04-25T00:09:09.319468+0000","last_became_active":"2024-04-25T00:09:08.294684+0000","last_became_peered":"2024-04-25T00:09:08.294684+0000","last_unstale":"2024-04-25T00:09:09.319468+0000","last_undegraded":"2024-04-25T00:09:09.319468+0000","last_fullsized":"2024-04-25T00:09:09.319468+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:41.957166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.310982+0000","last_change":"2024-04-25T00:09:09.308033+0000","last_active":"2024-04-25T00:09:09.310982+0000","last_peered":"2024-04-25T00:09:09.310982+0000","last_clean":"2024-04-25T00:09:09.310982+0000","last_became_active":"2024-04-25T00:09:09.307771+0000","last_became_peered":"2024-04-25T00:09:09.307771+0000","last_unstale":"2024-04-25T00:09:09.310982+0000","last_undegraded":"2024-04-25T00:09:09.310982+0000","last_fullsized":"2024-04-25T00:09:09.310982+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:52.101238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306403+0000","last_change":"2024-04-25T00:09:08.298434+0000","last_active":"2024-04-25T00:09:10.306403+0000","last_peered":"2024-04-25T00:09:10.306403+0000","last_clean":"2024-04-25T00:09:10.306403+0000","last_became_active":"2024-04-25T00:09:08.298098+0000","last_became_peered":"2024-04-25T00:09:08.298098+0000","last_unstale":"2024-04-25T00:09:10.306403+0000","last_undegraded":"2024-04-25T00:09:10.306403+0000","last_fullsized":"2024-04-25T00:09:10.306403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:26.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307091+0000","last_change":"2024-04-25T00:09:09.308278+0000","last_active":"2024-04-25T00:09:10.307091+0000","last_peered":"2024-04-25T00:09:10.307091+0000","last_clean":"2024-04-25T00:09:10.307091+0000","last_became_active":"2024-04-25T00:09:09.308010+0000","last_became_peered":"2024-04-25T00:09:09.308010+0000","last_unstale":"2024-04-25T00:09:10.307091+0000","last_undegraded":"2024-04-25T00:09:10.307091+0000","last_fullsized":"2024-04-25T00:09:10.307091+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:58.164977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307594+0000","last_change":"2024-04-25T00:09:08.297295+0000","last_active":"2024-04-25T00:09:10.307594+0000","last_peered":"2024-04-25T00:09:10.307594+0000","last_clean":"2024-04-25T00:09:10.307594+0000","last_became_active":"2024-04-25T00:09:08.297027+0000","last_became_peered":"2024-04-25T00:09:08.297027+0000","last_unstale":"2024-04-25T00:09:10.307594+0000","last_undegraded":"2024-04-25T00:09:10.307594+0000","last_fullsized":"2024-04-25T00:09:10.307594+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:12.127623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306290+0000","last_change":"2024-04-25T00:09:09.304335+0000","last_active":"2024-04-25T00:09:10.306290+0000","last_peered":"2024-04-25T00:09:10.306290+0000","last_clean":"2024-04-25T00:09:10.306290+0000","last_became_active":"2024-04-25T00:09:09.304082+0000","last_became_peered":"2024-04-25T00:09:09.304082+0000","last_unstale":"2024-04-25T00:09:10.306290+0000","last_undegraded":"2024-04-25T00:09:10.306290+0000","last_fullsized":"2024-04-25T00:09:10.306290+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:45.787992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952519+0000","last_change":"2024-04-25T00:09:08.287319+0000","last_active":"2024-04-25T00:09:10.952519+0000","last_peered":"2024-04-25T00:09:10.952519+0000","last_clean":"2024-04-25T00:09:10.952519+0000","last_became_active":"2024-04-25T00:09:08.287069+0000","last_became_peered":"2024-04-25T00:09:08.287069+0000","last_unstale":"2024-04-25T00:09:10.952519+0000","last_undegraded":"2024-04-25T00:09:10.952519+0000","last_fullsized":"2024-04-25T00:09:10.952519+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:03.594702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318161+0000","last_change":"2024-04-25T00:09:09.317058+0000","last_active":"2024-04-25T00:09:09.318161+0000","last_peered":"2024-04-25T00:09:09.318161+0000","last_clean":"2024-04-25T00:09:09.318161+0000","last_became_active":"2024-04-25T00:09:09.316794+0000","last_became_peered":"2024-04-25T00:09:09.316794+0000","last_unstale":"2024-04-25T00:09:09.318161+0000","last_undegraded":"2024-04-25T00:09:09.318161+0000","last_fullsized":"2024-04-25T00:09:09.318161+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:07:20.625405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307694+0000","last_change":"2024-04-25T00:09:08.295834+0000","last_active":"2024-04-25T00:09:10.307694+0000","last_peered":"2024-04-25T00:09:10.307694+0000","last_clean":"2024-04-25T00:09:10.307694+0000","last_became_active":"2024-04-25T00:09:08.295553+0000","last_became_peered":"2024-04-25T00:09:08.295553+0000","last_unstale":"2024-04-25T00:09:10.307694+0000","last_undegraded":"2024-04-25T00:09:10.307694+0000","last_fullsized":"2024-04-25T00:09:10.307694+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:56.668487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952420+0000","last_change":"2024-04-25T00:09:09.311578+0000","last_active":"2024-04-25T00:09:10.952420+0000","last_peered":"2024-04-25T00:09:10.952420+0000","last_clean":"2024-04-25T00:09:10.952420+0000","last_became_active":"2024-04-25T00:09:09.311332+0000","last_became_peered":"2024-04-25T00:09:09.311332+0000","last_unstale":"2024-04-25T00:09:10.952420+0000","last_undegraded":"2024-04-25T00:09:10.952420+0000","last_fullsized":"2024-04-25T00:09:10.952420+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:42.026281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298715+0000","last_change":"2024-04-25T00:09:08.292118+0000","last_active":"2024-04-25T00:09:09.298715+0000","last_peered":"2024-04-25T00:09:09.298715+0000","last_clean":"2024-04-25T00:09:09.298715+0000","last_became_active":"2024-04-25T00:09:08.291845+0000","last_became_peered":"2024-04-25T00:09:08.291845+0000","last_unstale":"2024-04-25T00:09:09.298715+0000","last_undegraded":"2024-04-25T00:09:09.298715+0000","last_fullsized":"2024-04-25T00:09:09.298715+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:13.139707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952697+0000","last_change":"2024-04-25T00:09:09.316868+0000","last_active":"2024-04-25T00:09:10.952697+0000","last_peered":"2024-04-25T00:09:10.952697+0000","last_clean":"2024-04-25T00:09:10.952697+0000","last_became_active":"2024-04-25T00:09:09.316605+0000","last_became_peered":"2024-04-25T00:09:09.316605+0000","last_unstale":"2024-04-25T00:09:10.952697+0000","last_undegraded":"2024-04-25T00:09:10.952697+0000","last_fullsized":"2024-04-25T00:09:10.952697+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:18:35.264876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"169'2","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.318149+0000","last_change":"2024-04-25T00:09:08.295803+0000","last_active":"2024-04-25T00:09:09.318149+0000","last_peered":"2024-04-25T00:09:09.318149+0000","last_clean":"2024-04-25T00:09:09.318149+0000","last_became_active":"2024-04-25T00:09:08.295539+0000","last_became_peered":"2024-04-25T00:09:08.295539+0000","last_unstale":"2024-04-25T00:09:09.318149+0000","last_undegraded":"2024-04-25T00:09:09.318149+0000","last_fullsized":"2024-04-25T00:09:09.318149+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:15.787112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307196+0000","last_change":"2024-04-25T00:09:09.308317+0000","last_active":"2024-04-25T00:09:10.307196+0000","last_peered":"2024-04-25T00:09:10.307196+0000","last_clean":"2024-04-25T00:09:10.307196+0000","last_became_active":"2024-04-25T00:09:09.308028+0000","last_became_peered":"2024-04-25T00:09:09.308028+0000","last_unstale":"2024-04-25T00:09:10.307196+0000","last_undegraded":"2024-04-25T00:09:10.307196+0000","last_fullsized":"2024-04-25T00:09:10.307196+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:56.974218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307070+0000","last_change":"2024-04-25T00:09:08.293441+0000","last_active":"2024-04-25T00:09:10.307070+0000","last_peered":"2024-04-25T00:09:10.307070+0000","last_clean":"2024-04-25T00:09:10.307070+0000","last_became_active":"2024-04-25T00:09:08.291906+0000","last_became_peered":"2024-04-25T00:09:08.291906+0000","last_unstale":"2024-04-25T00:09:10.307070+0000","last_undegraded":"2024-04-25T00:09:10.307070+0000","last_fullsized":"2024-04-25T00:09:10.307070+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:16.313266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305531+0000","last_change":"2024-04-25T00:09:09.304379+0000","last_active":"2024-04-25T00:09:09.305531+0000","last_peered":"2024-04-25T00:09:09.305531+0000","last_clean":"2024-04-25T00:09:09.305531+0000","last_became_active":"2024-04-25T00:09:09.304128+0000","last_became_peered":"2024-04-25T00:09:09.304128+0000","last_unstale":"2024-04-25T00:09:09.305531+0000","last_undegraded":"2024-04-25T00:09:09.305531+0000","last_fullsized":"2024-04-25T00:09:09.305531+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:04.078901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307168+0000","last_change":"2024-04-25T00:09:08.291856+0000","last_active":"2024-04-25T00:09:10.307168+0000","last_peered":"2024-04-25T00:09:10.307168+0000","last_clean":"2024-04-25T00:09:10.307168+0000","last_became_active":"2024-04-25T00:09:08.291610+0000","last_became_peered":"2024-04-25T00:09:08.291610+0000","last_unstale":"2024-04-25T00:09:10.307168+0000","last_undegraded":"2024-04-25T00:09:10.307168+0000","last_fullsized":"2024-04-25T00:09:10.307168+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:03.438834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.305411+0000","last_change":"2024-04-25T00:09:09.301509+0000","last_active":"2024-04-25T00:09:09.305411+0000","last_peered":"2024-04-25T00:09:09.305411+0000","last_clean":"2024-04-25T00:09:09.305411+0000","last_became_active":"2024-04-25T00:09:09.301123+0000","last_became_peered":"2024-04-25T00:09:09.301123+0000","last_unstale":"2024-04-25T00:09:09.305411+0000","last_undegraded":"2024-04-25T00:09:09.305411+0000","last_fullsized":"2024-04-25T00:09:09.305411+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:07.086387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.293309+0000","last_change":"2024-04-25T00:09:08.298988+0000","last_active":"2024-04-25T00:09:09.293309+0000","last_peered":"2024-04-25T00:09:09.293309+0000","last_clean":"2024-04-25T00:09:09.293309+0000","last_became_active":"2024-04-25T00:09:08.298700+0000","last_became_peered":"2024-04-25T00:09:08.298700+0000","last_unstale":"2024-04-25T00:09:09.293309+0000","last_undegraded":"2024-04-25T00:09:09.293309+0000","last_fullsized":"2024-04-25T00:09:09.293309+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:49.638450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.316940+0000","last_change":"2024-04-25T00:09:09.315615+0000","last_active":"2024-04-25T00:09:09.316940+0000","last_peered":"2024-04-25T00:09:09.316940+0000","last_clean":"2024-04-25T00:09:09.316940+0000","last_became_active":"2024-04-25T00:09:09.315347+0000","last_became_peered":"2024-04-25T00:09:09.315347+0000","last_unstale":"2024-04-25T00:09:09.316940+0000","last_undegraded":"2024-04-25T00:09:09.316940+0000","last_fullsized":"2024-04-25T00:09:09.316940+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:34.881783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298386+0000","last_change":"2024-04-25T00:09:08.297157+0000","last_active":"2024-04-25T00:09:09.298386+0000","last_peered":"2024-04-25T00:09:09.298386+0000","last_clean":"2024-04-25T00:09:09.298386+0000","last_became_active":"2024-04-25T00:09:08.296829+0000","last_became_peered":"2024-04-25T00:09:08.296829+0000","last_unstale":"2024-04-25T00:09:09.298386+0000","last_undegraded":"2024-04-25T00:09:09.298386+0000","last_fullsized":"2024-04-25T00:09:09.298386+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:35.630831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.317981+0000","last_change":"2024-04-25T00:09:09.316744+0000","last_active":"2024-04-25T00:09:09.317981+0000","last_peered":"2024-04-25T00:09:09.317981+0000","last_clean":"2024-04-25T00:09:09.317981+0000","last_became_active":"2024-04-25T00:09:09.316500+0000","last_became_peered":"2024-04-25T00:09:09.316500+0000","last_unstale":"2024-04-25T00:09:09.317981+0000","last_undegraded":"2024-04-25T00:09:09.317981+0000","last_fullsized":"2024-04-25T00:09:09.317981+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:08.448079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298850+0000","last_change":"2024-04-25T00:09:08.293620+0000","last_active":"2024-04-25T00:09:09.298850+0000","last_peered":"2024-04-25T00:09:09.298850+0000","last_clean":"2024-04-25T00:09:09.298850+0000","last_became_active":"2024-04-25T00:09:08.293342+0000","last_became_peered":"2024-04-25T00:09:08.293342+0000","last_unstale":"2024-04-25T00:09:09.298850+0000","last_undegraded":"2024-04-25T00:09:09.298850+0000","last_fullsized":"2024-04-25T00:09:09.298850+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:33.514019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.314173+0000","last_change":"2024-04-25T00:09:09.313418+0000","last_active":"2024-04-25T00:09:09.314173+0000","last_peered":"2024-04-25T00:09:09.314173+0000","last_clean":"2024-04-25T00:09:09.314173+0000","last_became_active":"2024-04-25T00:09:09.313027+0000","last_became_peered":"2024-04-25T00:09:09.313027+0000","last_unstale":"2024-04-25T00:09:09.314173+0000","last_undegraded":"2024-04-25T00:09:09.314173+0000","last_fullsized":"2024-04-25T00:09:09.314173+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:24.518797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.298585+0000","last_change":"2024-04-25T00:09:08.291402+0000","last_active":"2024-04-25T00:09:09.298585+0000","last_peered":"2024-04-25T00:09:09.298585+0000","last_clean":"2024-04-25T00:09:09.298585+0000","last_became_active":"2024-04-25T00:09:08.290490+0000","last_became_peered":"2024-04-25T00:09:08.290490+0000","last_unstale":"2024-04-25T00:09:09.298585+0000","last_undegraded":"2024-04-25T00:09:09.298585+0000","last_fullsized":"2024-04-25T00:09:09.298585+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:16.096856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.952474+0000","last_change":"2024-04-25T00:09:09.305545+0000","last_active":"2024-04-25T00:09:10.952474+0000","last_peered":"2024-04-25T00:09:10.952474+0000","last_clean":"2024-04-25T00:09:10.952474+0000","last_became_active":"2024-04-25T00:09:09.305249+0000","last_became_peered":"2024-04-25T00:09:09.305249+0000","last_unstale":"2024-04-25T00:09:10.952474+0000","last_undegraded":"2024-04-25T00:09:10.952474+0000","last_fullsized":"2024-04-25T00:09:10.952474+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:01.879304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.307517+0000","last_change":"2024-04-25T00:09:08.297106+0000","last_active":"2024-04-25T00:09:10.307517+0000","last_peered":"2024-04-25T00:09:10.307517+0000","last_clean":"2024-04-25T00:09:10.307517+0000","last_became_active":"2024-04-25T00:09:08.296791+0000","last_became_peered":"2024-04-25T00:09:08.296791+0000","last_unstale":"2024-04-25T00:09:10.307517+0000","last_undegraded":"2024-04-25T00:09:10.307517+0000","last_fullsized":"2024-04-25T00:09:10.307517+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:41.021193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T00:09:10.306559+0000","last_change":"2024-04-25T00:09:09.303153+0000","last_active":"2024-04-25T00:09:10.306559+0000","last_peered":"2024-04-25T00:09:10.306559+0000","last_clean":"2024-04-25T00:09:10.306559+0000","last_became_active":"2024-04-25T00:09:09.302813+0000","last_became_peered":"2024-04-25T00:09:09.302813+0000","last_unstale":"2024-04-25T00:09:10.306559+0000","last_undegraded":"2024-04-25T00:09:10.306559+0000","last_fullsized":"2024-04-25T00:09:10.306559+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:08.277146+0000","last_clean_scrub_stamp":"2024-04-25T00:09:08.277146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:30.150558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.309289+0000","last_change":"2024-04-25T00:09:08.295549+0000","last_active":"2024-04-25T00:09:09.309289+0000","last_peered":"2024-04-25T00:09:09.309289+0000","last_clean":"2024-04-25T00:09:09.309289+0000","last_became_active":"2024-04-25T00:09:08.295227+0000","last_became_peered":"2024-04-25T00:09:08.295227+0000","last_unstale":"2024-04-25T00:09:09.309289+0000","last_undegraded":"2024-04-25T00:09:09.309289+0000","last_fullsized":"2024-04-25T00:09:09.309289+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:07.272058+0000","last_clean_scrub_stamp":"2024-04-25T00:09:07.272058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:15.354475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":491,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T00:09:09.310137+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:09:09.310137+0000","last_peered":"2024-04-25T00:09:09.310137+0000","last_clean":"2024-04-25T00:09:09.310137+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:09:09.310137+0000","last_undegraded":"2024-04-25T00:09:09.310137+0000","last_fullsized":"2024-04-25T00:09:09.310137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574984,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2727936,"data_stored":1521577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51312,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320528,"statfs":{"total":96636764160,"available":96584220672,"internally_reserved":0,"allocated":2125824,"data_stored":931105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55468,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316372,"statfs":{"total":96636764160,"available":96579964928,"internally_reserved":0,"allocated":2125824,"data_stored":931105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":2154496,"data_stored":939599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2138112,"data_stored":931299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51316,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320524,"statfs":{"total":96636764160,"available":96584216576,"internally_reserved":0,"allocated":2138112,"data_stored":931799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2150400,"data_stored":939017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51908,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319932,"statfs":{"total":96636764160,"available":96583610368,"internally_reserved":0,"allocated":2736128,"data_stored":1527795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:09:13.172 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:09:13.172 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:09:13.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.313+0000 7fc0361fb640 1 Processor -- start 2024-04-25T00:09:13.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.313+0000 7fc0361fb640 1 -- start start 2024-04-25T00:09:13.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc0361fb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e2d0 0x7fc03015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc0361fb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0300b3080 0x7fc0300b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc0361fb640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03015f4f0 con 0x7fc03015e6b0 2024-04-25T00:09:13.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc0361fb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc0300cde00 con 0x7fc0300b3080 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc0361fb640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc03015f7d0 con 0x7fc03015e2d0 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc02effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0300b3080 0x7fc0300b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc02effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0300b3080 0x7fc0300b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46058/0 (socket says 172.21.15.113:46058) 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc02effd640 1 -- 172.21.15.113:0/774363177 learned_addr learned my addr 172.21.15.113:0/774363177 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc02effd640 1 -- 172.21.15.113:0/774363177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e2d0 msgr2=0x7fc03015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc02effd640 1 --2- 172.21.15.113:0/774363177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e2d0 0x7fc03015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc02effd640 1 -- 172.21.15.113:0/774363177 >> v1:172.21.15.178:6789/0 conn(0x7fc03015e6b0 legacy=0x7fc030071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.314+0000 7fc02effd640 1 -- 172.21.15.113:0/774363177 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03015fab0 con 0x7fc0300b3080 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.315+0000 7fc02effd640 1 --2- 172.21.15.113:0/774363177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc0300b4330 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc024009870 tx=0x7fc02402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb119d411692c683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.315+0000 7fc02e7fc640 1 -- 172.21.15.113:0/774363177 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024037070 con 0x7fc0300b3080 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.315+0000 7fc02e7fc640 1 -- 172.21.15.113:0/774363177 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc024032440 con 0x7fc0300b3080 2024-04-25T00:09:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.315+0000 7fc02e7fc640 1 -- 172.21.15.113:0/774363177 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024032810 con 0x7fc0300b3080 2024-04-25T00:09:13.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.315+0000 7fc0361fb640 1 -- 172.21.15.113:0/774363177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 msgr2=0x7fc0300b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.316+0000 7fc0361fb640 1 --2- 172.21.15.113:0/774363177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc0300b4330 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc024009870 tx=0x7fc02402a180 comp rx=0 tx=0).stop 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.316+0000 7fc0361fb640 1 -- 172.21.15.113:0/774363177 shutdown_connections 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.316+0000 7fc0361fb640 1 --2- 172.21.15.113:0/774363177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc0300b4330 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.316+0000 7fc0361fb640 1 --2- 172.21.15.113:0/774363177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e2d0 0x7fc03015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.316+0000 7fc0361fb640 1 -- 172.21.15.113:0/774363177 >> 172.21.15.113:0/774363177 conn(0x7fc03006b0f0 msgr2=0x7fc03006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.316+0000 7fc0361fb640 1 -- 172.21.15.113:0/774363177 shutdown_connections 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.316+0000 7fc0361fb640 1 -- 172.21.15.113:0/774363177 wait complete. 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.317+0000 7fc0361fb640 1 Processor -- start 2024-04-25T00:09:13.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.317+0000 7fc0361fb640 1 -- start start 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc0361fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc03014b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc0361fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03015e2d0 0x7fc0300aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc0361fb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e6b0 0x7fc030144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc0361fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc03015f2b0 con 0x7fc0300b3080 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc0361fb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc03015f130 con 0x7fc03015e2d0 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc0361fb640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc0300cf2a0 con 0x7fc03015e6b0 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc03014b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc03014b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46062/0 (socket says 172.21.15.113:46062) 2024-04-25T00:09:13.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 -- 172.21.15.113:0/2373526267 learned_addr learned my addr 172.21.15.113:0/2373526267 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02effd640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03015e2d0 0x7fc0300aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02ffff640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e6b0 0x7fc030144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 -- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e6b0 msgr2=0x7fc030144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e6b0 0x7fc030144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 -- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03015e2d0 msgr2=0x7fc0300aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03015e2d0 0x7fc0300aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 -- 172.21.15.113:0/2373526267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc030144e50 con 0x7fc0300b3080 2024-04-25T00:09:13.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.318+0000 7fc02f7fe640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc03014b1d0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b4a0 tx=0x7fc02000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:13.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.322+0000 7fc017fff640 1 -- 172.21.15.113:0/2373526267 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020014070 con 0x7fc0300b3080 2024-04-25T00:09:13.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.322+0000 7fc017fff640 1 -- 172.21.15.113:0/2373526267 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc020010610 con 0x7fc0300b3080 2024-04-25T00:09:13.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.322+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0300b0ad0 con 0x7fc0300b3080 2024-04-25T00:09:13.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.322+0000 7fc017fff640 1 -- 172.21.15.113:0/2373526267 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020010a00 con 0x7fc0300b3080 2024-04-25T00:09:13.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.322+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0300b1090 con 0x7fc0300b3080 2024-04-25T00:09:13.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.323+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0300691a0 con 0x7fc0300b3080 2024-04-25T00:09:13.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.323+0000 7fc017fff640 1 -- 172.21.15.113:0/2373526267 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc020017070 con 0x7fc0300b3080 2024-04-25T00:09:13.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.324+0000 7fc017fff640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.324+0000 7fc017fff640 1 -- 172.21.15.113:0/2373526267 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fc02008a860 con 0x7fc0300b3080 2024-04-25T00:09:13.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.324+0000 7fc02effd640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.324+0000 7fc02effd640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc01c06be30 0x7fc01c06e2f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc02402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:13.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.328+0000 7fc017fff640 1 -- 172.21.15.113:0/2373526267 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc020058bc0 con 0x7fc0300b3080 2024-04-25T00:09:13.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.566+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc030072050 con 0x7fc0300b3080 2024-04-25T00:09:13.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.569+0000 7fc017fff640 1 -- 172.21.15.113:0/2373526267 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7fc02008a070 con 0x7fc0300b3080 2024-04-25T00:09:13.569 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:09:13.569 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:09:13.569 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:09:13.570 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:09:13.570 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:09:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:09:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc01c06be30 msgr2=0x7fc01c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc01c06be30 0x7fc01c06e2f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc02402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 msgr2=0x7fc03014b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc03014b1d0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b4a0 tx=0x7fc02000b960 comp rx=0 tx=0).stop 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 shutdown_connections 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03015e2d0 0x7fc0300aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc03015e6b0 0x7fc030144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 --2- 172.21.15.113:0/2373526267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0300b3080 0x7fc03014b1d0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.571+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 >> 172.21.15.113:0/2373526267 conn(0x7fc03006b0f0 msgr2=0x7fc0301546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.572+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 shutdown_connections 2024-04-25T00:09:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.572+0000 7fc0361fb640 1 -- 172.21.15.113:0/2373526267 wait complete. 2024-04-25T00:09:13.583 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:13.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.679+0000 7f49df57e640 1 Processor -- start 2024-04-25T00:09:13.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.680+0000 7f49df57e640 1 -- start start 2024-04-25T00:09:13.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49df57e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49d8072ea0 0x7f49d8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49df57e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f49d80b3af0 0x7f49d80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49df57e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80b2d50 con 0x7f49d815a6b0 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49df57e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f49d8072a70 con 0x7f49d8072ea0 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49df57e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80b3010 con 0x7f49d80b3af0 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49dcb19640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49d8072ea0 0x7f49d8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49dcb19640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49d8072ea0 0x7f49d8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46070/0 (socket says 172.21.15.113:46070) 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49dcb19640 1 -- 172.21.15.113:0/2835589345 learned_addr learned my addr 172.21.15.113:0/2835589345 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49dcb19640 1 -- 172.21.15.113:0/2835589345 >> v2:172.21.15.178:3300/0 conn(0x7f49d80b3af0 msgr2=0x7f49d80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49ddb1b640 1 --2- 172.21.15.113:0/2835589345 >> v2:172.21.15.178:3300/0 conn(0x7f49d80b3af0 0x7f49d80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49dcb19640 1 --2- 172.21.15.113:0/2835589345 >> v2:172.21.15.178:3300/0 conn(0x7f49d80b3af0 0x7f49d80b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49dcb19640 1 -- 172.21.15.113:0/2835589345 >> v1:172.21.15.178:6789/0 conn(0x7f49d815a6b0 legacy=0x7f49d8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49dcb19640 1 -- 172.21.15.113:0/2835589345 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80b32f0 con 0x7f49d8072ea0 2024-04-25T00:09:13.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.681+0000 7f49ddb1b640 1 --2- 172.21.15.113:0/2835589345 >> v2:172.21.15.178:3300/0 conn(0x7f49d80b3af0 0x7f49d80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49dcb19640 1 --2- 172.21.15.113:0/2835589345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d8071410 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f49d80cc5a0 tx=0x7f49c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaa2479efdc5aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49cffff640 1 -- 172.21.15.113:0/2835589345 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8037070 con 0x7f49d8072ea0 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49cffff640 1 -- 172.21.15.113:0/2835589345 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c8032440 con 0x7f49d8072ea0 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49cffff640 1 -- 172.21.15.113:0/2835589345 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8032810 con 0x7f49d8072ea0 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 -- 172.21.15.113:0/2835589345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 msgr2=0x7f49d8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 --2- 172.21.15.113:0/2835589345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d8071410 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f49d80cc5a0 tx=0x7f49c802a230 comp rx=0 tx=0).stop 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 -- 172.21.15.113:0/2835589345 shutdown_connections 2024-04-25T00:09:13.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 --2- 172.21.15.113:0/2835589345 >> v2:172.21.15.178:3300/0 conn(0x7f49d80b3af0 0x7f49d80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 --2- 172.21.15.113:0/2835589345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d8071410 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 -- 172.21.15.113:0/2835589345 >> 172.21.15.113:0/2835589345 conn(0x7f49d80b1a50 msgr2=0x7f49d80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:13.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 -- 172.21.15.113:0/2835589345 shutdown_connections 2024-04-25T00:09:13.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.682+0000 7f49df57e640 1 -- 172.21.15.113:0/2835589345 wait complete. 2024-04-25T00:09:13.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 Processor -- start 2024-04-25T00:09:13.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 -- start start 2024-04-25T00:09:13.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d81511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d80b3af0 0x7f49d81586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49d815a6b0 0x7f49d814cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49d815aa90 con 0x7f49d8072ea0 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8072720 con 0x7f49d80b3af0 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49df57e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49d815ac10 con 0x7f49d815a6b0 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49dd31a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d81511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49dd31a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d81511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46072/0 (socket says 172.21.15.113:46072) 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.684+0000 7f49dd31a640 1 -- 172.21.15.113:0/529905802 learned_addr learned my addr 172.21.15.113:0/529905802 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dcb19640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d80b3af0 0x7f49d81586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dd31a640 1 -- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49d815a6b0 msgr2=0x7f49d814cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dd31a640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49d815a6b0 0x7f49d814cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dd31a640 1 -- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d80b3af0 msgr2=0x7f49d81586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dd31a640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d80b3af0 0x7f49d81586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dd31a640 1 -- 172.21.15.113:0/529905802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d8151780 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dcb19640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d80b3af0 0x7f49d81586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49dd31a640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d81511a0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b3f0 tx=0x7f49d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49cdffb640 1 -- 172.21.15.113:0/529905802 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4015020 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49cdffb640 1 -- 172.21.15.113:0/529905802 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d4011830 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d80cac40 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49cdffb640 1 -- 172.21.15.113:0/529905802 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4011c20 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.685+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d80cb200 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.686+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d80691a0 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.687+0000 7f49cdffb640 1 -- 172.21.15.113:0/529905802 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49d4018020 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.687+0000 7f49cdffb640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49bc06be30 0x7f49bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.687+0000 7f49cdffb640 1 -- 172.21.15.113:0/529905802 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f49d408aa50 con 0x7f49d8072ea0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.687+0000 7f49dcb19640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49bc06be30 0x7f49bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:13.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.687+0000 7f49dcb19640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49bc06be30 0x7f49bc06e2f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034000 tx=0x7f49c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:13.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.691+0000 7f49cdffb640 1 -- 172.21.15.113:0/529905802 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49d401f030 con 0x7f49d8072ea0 2024-04-25T00:09:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.904+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49d81568e0 con 0x7f49d8072ea0 2024-04-25T00:09:13.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.904+0000 7f49cdffb640 1 -- 172.21.15.113:0/529905802 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5195 (secure 0 0 0) 0x7f49d4059490 con 0x7f49d8072ea0 2024-04-25T00:09:13.905 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:13.905 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/992895335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":992895335},{"type":"v1","addr":"172.21.15.178:6834","nonce":992895335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/17230728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":17230728},{"type":"v1","addr":"172.21.15.113:6836","nonce":17230728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4170863917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4170863917},{"type":"v1","addr":"172.21.15.113:6837","nonce":4170863917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:08.629096+0000","modified":"2024-04-25T00:09:11.220151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10434":{"gid":10434,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/2813561857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2813561857},{"type":"v1","addr":"172.21.15.178:6835","nonce":2813561857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10434,"qdb_cluster":[10434]},"id":12}]} 2024-04-25T00:09:13.905 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-04-25T00:09:13.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49bc06be30 msgr2=0x7f49bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49bc06be30 0x7f49bc06e2f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034000 tx=0x7f49c802e000 comp rx=0 tx=0).stop 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 msgr2=0x7f49d81511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d81511a0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b3f0 tx=0x7f49d400b8b0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 shutdown_connections 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f49bc06be30 0x7f49bc06e2f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d80b3af0 0x7f49d81586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49d815a6b0 0x7f49d814cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 --2- 172.21.15.113:0/529905802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49d8072ea0 0x7f49d81511a0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 >> 172.21.15.113:0/529905802 conn(0x7f49d80b1a50 msgr2=0x7f49d8070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 shutdown_connections 2024-04-25T00:09:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:13.907+0000 7f49df57e640 1 -- 172.21.15.113:0/529905802 wait complete. 2024-04-25T00:09:14.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:09:08.629096+0000', 'modified': '2024-04-25T00:09:11.220151+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10434': {'gid': 10434, 'name': 'b', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6835/2813561857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6833', 'nonce': 2813561857}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 2813561857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10434, 'qdb_cluster': [10434]} 2024-04-25T00:09:14.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:09:14.320 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:09:14.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.418+0000 7f7c0f6ee640 1 Processor -- start 2024-04-25T00:09:14.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0f6ee640 1 -- start start 2024-04-25T00:09:14.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0f6ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0f6ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c080d28c0 0x7f7c080b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0f6ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c080b61e0 0x7f7c08071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0f6ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7c080d2e00 con 0x7f7c080b61e0 2024-04-25T00:09:14.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0f6ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7c080d30e0 con 0x7f7c080d28c0 2024-04-25T00:09:14.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0f6ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c080d33c0 con 0x7f7c0815e510 2024-04-25T00:09:14.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0dc8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c080b61e0 0x7f7c08071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0dc8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c080b61e0 0x7f7c08071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46078/0 (socket says 172.21.15.113:46078) 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.419+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2885084186 learned_addr learned my addr 172.21.15.113:0/2885084186 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/2885084186 >> v2:172.21.15.178:3300/0 conn(0x7f7c080d28c0 0x7f7c080b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/2885084186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2885084186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 msgr2=0x7f7c080d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2885084186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2885084186 >> v2:172.21.15.178:3300/0 conn(0x7f7c080d28c0 msgr2=0x7f7c080b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2885084186 >> v2:172.21.15.178:3300/0 conn(0x7f7c080d28c0 0x7f7c080b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2885084186 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c080d36a0 con 0x7f7c080b61e0 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2885084186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080b61e0 0x7f7c08071710 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f7c080d5320 tx=0x7f7c0002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b721ae84ac0d4a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.420+0000 7f7bfffff640 1 -- 172.21.15.113:0/2885084186 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00037070 con 0x7f7c080b61e0 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7bfffff640 1 -- 172.21.15.113:0/2885084186 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c00032440 con 0x7f7c080b61e0 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7bfffff640 1 -- 172.21.15.113:0/2885084186 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00032810 con 0x7f7c080b61e0 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/2885084186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080b61e0 msgr2=0x7f7c08071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/2885084186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080b61e0 0x7f7c08071710 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f7c080d5320 tx=0x7f7c0002a140 comp rx=0 tx=0).stop 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/2885084186 shutdown_connections 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/2885084186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080b61e0 0x7f7c08071710 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/2885084186 >> v2:172.21.15.178:3300/0 conn(0x7f7c080d28c0 0x7f7c080b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/2885084186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/2885084186 >> 172.21.15.113:0/2885084186 conn(0x7f7c0806b720 msgr2=0x7f7c080d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/2885084186 shutdown_connections 2024-04-25T00:09:14.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.421+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/2885084186 wait complete. 2024-04-25T00:09:14.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.422+0000 7f7c0f6ee640 1 Processor -- start 2024-04-25T00:09:14.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.422+0000 7f7c0f6ee640 1 -- start start 2024-04-25T00:09:14.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0f6ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c080b61e0 0x7f7c080cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0f6ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080d28c0 0x7f7c080cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0f6ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0f6ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080b9dd0 con 0x7f7c080d28c0 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0f6ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080d3ea0 con 0x7f7c080b61e0 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0f6ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c08071c50 con 0x7f7c0815e510 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080d28c0 0x7f7c080cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080d28c0 0x7f7c080cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46092/0 (socket says 172.21.15.113:46092) 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 -- 172.21.15.113:0/1788080821 learned_addr learned my addr 172.21.15.113:0/1788080821 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c080b61e0 0x7f7c080cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 -- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 msgr2=0x7f7c080cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 -- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c080b61e0 msgr2=0x7f7c080cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c080b61e0 0x7f7c080cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0cc89640 1 -- 172.21.15.113:0/1788080821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c081f3e50 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c080b61e0 0x7f7c080cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.423+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.424+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080d28c0 0x7f7c080cb830 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d800 tx=0x7f7bf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.424+0000 7f7bfdffb640 1 -- 172.21.15.113:0/1788080821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8018020 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.424+0000 7f7bfdffb640 1 -- 172.21.15.113:0/1788080821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf80096a0 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.424+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c080c7d60 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.424+0000 7f7bfdffb640 1 -- 172.21.15.113:0/1788080821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8009a70 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.424+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c080c8320 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.425+0000 7f7bfdffb640 1 -- 172.21.15.113:0/1788080821 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bf801b020 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.426+0000 7f7bfdffb640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bec06be30 0x7f7bec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.426+0000 7f7bfdffb640 1 -- 172.21.15.113:0/1788080821 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f7bf808b330 con 0x7f7c080d28c0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.426+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bec06be30 0x7f7bec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.426+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bec06be30 0x7f7bec06e2f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009f50 tx=0x7f7c0400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.427+0000 7f7c123f6740 1 Processor -- start 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.427+0000 7f7c123f6740 1 -- start start 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45beb60 0x55b0b45714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45a1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45a1540 0x55b0b4576350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c123f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b0b44d9f00 con 0x55b0b45a1540 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b0b4482750 con 0x55b0b45a1c80 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b0b44d5ec0 con 0x55b0b45beb60 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45a1540 0x55b0b4576350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45a1540 0x55b0b4576350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46098/0 (socket says 172.21.15.113:46098) 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/3472717567 learned_addr learned my addr 172.21.15.113:0/3472717567 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45beb60 0x55b0b45714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45beb60 msgr2=0x55b0b45714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45beb60 0x55b0b45714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 msgr2=0x55b0b45a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/3472717567 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b41ee2d0 con 0x55b0b45a1540 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45beb60 0x55b0b45714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45a1540 0x55b0b4576350 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f7c0002a730 tx=0x7f7c0002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4031cc440acbb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7bf2ffd640 1 -- 172.21.15.113:0/3472717567 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00037040 con 0x55b0b45a1540 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7bf2ffd640 1 -- 172.21.15.113:0/3472717567 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c000326c0 con 0x55b0b45a1540 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.428+0000 7f7bf2ffd640 1 -- 172.21.15.113:0/3472717567 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00032a90 con 0x55b0b45a1540 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 -- 172.21.15.113:0/3472717567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45a1540 msgr2=0x55b0b4576350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45a1540 0x55b0b4576350 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f7c0002a730 tx=0x7f7c0002c000 comp rx=0 tx=0).stop 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 -- 172.21.15.113:0/3472717567 shutdown_connections 2024-04-25T00:09:14.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45a1540 0x55b0b4576350 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 --2- 172.21.15.113:0/3472717567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45beb60 0x55b0b45714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 -- 172.21.15.113:0/3472717567 >> 172.21.15.113:0/3472717567 conn(0x55b0b4573760 msgr2=0x55b0b45bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 -- 172.21.15.113:0/3472717567 shutdown_connections 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.429+0000 7f7c123f6740 1 -- 172.21.15.113:0/3472717567 wait complete. 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.430+0000 7f7c123f6740 1 -- start start 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.430+0000 7f7c123f6740 1 Processor -- start 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.430+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b459cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.430+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.430+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c123f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b0b4575910 con 0x55b0b45bbfb0 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b0b4495cf0 con 0x55b0b45beb60 2024-04-25T00:09:14.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b0b4574d00 con 0x55b0b45a1c80 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c123f6740 10 client.0 _ll_register_callbacks cb 0x55b0b44956e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45bc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46100/0 (socket says 172.21.15.113:46100) 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/178167277 learned_addr learned my addr 172.21.15.113:0/178167277 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b459cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 msgr2=0x55b0b459cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b459cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 msgr2=0x55b0b45a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/178167277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b42b86b0 con 0x55b0b45bbfb0 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45bc430 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f7c0003d000 tx=0x7f7c00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d136295b34a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7be7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7be7fff640 1 -- 172.21.15.113:0/178167277 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0004a070 con 0x55b0b45bbfb0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7be7fff640 1 -- 172.21.15.113:0/178167277 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f7c00032a80 con 0x55b0b45bbfb0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c000326c0 con 0x55b0b45bbfb0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7be7fff640 1 -- 172.21.15.113:0/178167277 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7c00055050 con 0x55b0b45bbfb0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c123f6740 10 client.10560 fetch_fsmap learned FSMap version 92 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55b0b45a30d0 con 0x55b0b45bbfb0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.431+0000 7f7be7fff640 1 -- 172.21.15.113:0/178167277 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c000326c0 con 0x55b0b45bbfb0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7be7fff640 1 -- 172.21.15.113:0/178167277 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7f7c00058020 con 0x55b0b45bbfb0 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c123f6740 10 client.10560 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c123f6740 10 client.10560 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c123f6740 20 client.10560 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c123f6740 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b45cc7d0 0x55b0b45ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c123f6740 4 client.10560 mds_command: new command op to 10434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:14.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 --> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0b44e7440 con 0x55b0b45cc7d0 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b45cc7d0 0x55b0b45ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7be7fff640 10 client.10560 ms_handle_connect on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.432+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b45cc7d0 0x55b0b45ccbb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5482a0c5c6092885 server_cookie=92b91eb8243d6b3b in_seq=0 out_seq=0 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7be7fff640 1 -- 172.21.15.113:0/178167277 <== mds.0 v2:172.21.15.178:6833/2813561857 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0b44e7440 con 0x55b0b45cc7d0 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7be7fff640 10 client.10560 handle_command_reply: tid=0 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 client.10560 shutdown 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 msgr2=0x55b0b45bc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45bc430 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f7c0003d000 tx=0x7f7c00004730 comp rx=0 tx=0).stop 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 shutdown_connections 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b45cc7d0 0x55b0b45ccbb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45bc430 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 --2- 172.21.15.113:0/178167277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b459cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7be7fff640 0 client.10560 ms_handle_reset on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:14.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.433+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 >> 172.21.15.113:0/178167277 conn(0x55b0b45a1540 msgr2=0x55b0b4570ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.434+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 shutdown_connections 2024-04-25T00:09:14.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.434+0000 7f7c123f6740 1 -- 172.21.15.113:0/178167277 wait complete. 2024-04-25T00:09:14.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.434+0000 7f7c123f6740 20 client.10560 trim_cache size 0 max 16384 2024-04-25T00:09:14.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.451+0000 7f7c123f6740 1 Processor -- start 2024-04-25T00:09:14.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.451+0000 7f7c123f6740 1 -- start start 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45bbfb0 0x55b0b45e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45beb60 0x55b0b45b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c123f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081ff7f0 con 0x55b0b45beb60 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080c9710 con 0x55b0b45a1c80 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c080c9d60 con 0x55b0b45bbfb0 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45beb60 0x55b0b45b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45beb60 0x55b0b45b85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46102/0 (socket says 172.21.15.113:46102) 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2022390221 learned_addr learned my addr 172.21.15.113:0/2022390221 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:14.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45bbfb0 0x55b0b45e00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45bbfb0 msgr2=0x55b0b45e00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45bbfb0 0x55b0b45e00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 msgr2=0x55b0b45cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.452+0000 7f7c0dc8b640 1 -- 172.21.15.113:0/2022390221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b459c640 con 0x55b0b45beb60 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45beb60 0x55b0b45b85f0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f7c000045b0 tx=0x7f7c0003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7aa95ecc23b0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7bf27fc640 1 -- 172.21.15.113:0/2022390221 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0004a040 con 0x55b0b45beb60 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7bf27fc640 1 -- 172.21.15.113:0/2022390221 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c00032a80 con 0x55b0b45beb60 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7bf27fc640 1 -- 172.21.15.113:0/2022390221 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00037040 con 0x55b0b45beb60 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 -- 172.21.15.113:0/2022390221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45beb60 msgr2=0x55b0b45b85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45beb60 0x55b0b45b85f0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f7c000045b0 tx=0x7f7c0003f000 comp rx=0 tx=0).stop 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 -- 172.21.15.113:0/2022390221 shutdown_connections 2024-04-25T00:09:14.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45beb60 0x55b0b45b85f0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45bbfb0 0x55b0b45e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2022390221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45a1c80 0x55b0b45cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 -- 172.21.15.113:0/2022390221 >> 172.21.15.113:0/2022390221 conn(0x55b0b45a1540 msgr2=0x55b0b45b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 -- 172.21.15.113:0/2022390221 shutdown_connections 2024-04-25T00:09:14.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.453+0000 7f7c123f6740 1 -- 172.21.15.113:0/2022390221 wait complete. 2024-04-25T00:09:14.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.454+0000 7f7c123f6740 1 -- start start 2024-04-25T00:09:14.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.454+0000 7f7c123f6740 1 Processor -- start 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b4635aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45a4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b0b4575910 con 0x55b0b45bbfb0 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b0b4495cf0 con 0x55b0b45beb60 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b0b45ba150 con 0x55b0b45a1c80 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 10 client.0 _ll_register_callbacks cb 0x7f7c00004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:14.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46106/0 (socket says 172.21.15.113:46106) 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 -- 172.21.15.113:0/2215298633 learned_addr learned my addr 172.21.15.113:0/2215298633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b4635aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 -- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 msgr2=0x55b0b4635aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b4635aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 -- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 msgr2=0x55b0b45decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45decb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 -- 172.21.15.113:0/2215298633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b4636370 con 0x55b0b45bbfb0 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0dc8b640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c0cc89640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45a4710 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8006de0 tx=0x7f7bf8060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d34840830f3e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7be7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:14.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7be7fff640 1 -- 172.21.15.113:0/2215298633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf808b170 con 0x55b0b45bbfb0 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7be7fff640 1 -- 172.21.15.113:0/2215298633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f7bf805ebb0 con 0x55b0b45bbfb0 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c00032a80 con 0x55b0b45bbfb0 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7be7fff640 1 -- 172.21.15.113:0/2215298633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7bf805edb0 con 0x55b0b45bbfb0 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 10 client.10566 fetch_fsmap learned FSMap version 92 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.455+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55b0b46360a0 con 0x55b0b45bbfb0 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7be7fff640 1 -- 172.21.15.113:0/2215298633 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c00032a80 con 0x55b0b45bbfb0 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7be7fff640 1 -- 172.21.15.113:0/2215298633 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7f7bf8085060 con 0x55b0b45bbfb0 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c123f6740 10 client.10566 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c123f6740 10 client.10566 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c123f6740 20 client.10566 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b459d790 0x55b0b459db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c123f6740 4 client.10566 mds_command: new command op to 10434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:09:14.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 --> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] -- command(tid 0: {"prefix": "status"}) -- 0x55b0b44e1790 con 0x55b0b459d790 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b459d790 0x55b0b459db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7c0d48a640 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b459d790 0x55b0b459db70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9593e6fa6bf558e2 server_cookie=7676e8c5177dd9b4 in_seq=0 out_seq=0 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7be7fff640 10 client.10566 ms_handle_connect on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7be7fff640 1 -- 172.21.15.113:0/2215298633 <== mds.0 v2:172.21.15.178:6833/2813561857 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55b0b44e1790 con 0x55b0b459d790 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.456+0000 7f7be7fff640 10 client.10566 handle_command_reply: tid=0 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 client.10566 shutdown 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 msgr2=0x55b0b45a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45a4710 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8006de0 tx=0x7f7bf8060040 comp rx=0 tx=0).stop 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 shutdown_connections 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55b0b459d790 0x55b0b459db70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0b45beb60 0x55b0b45decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b0b45bbfb0 0x55b0b45a4710 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 --2- 172.21.15.113:0/2215298633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b0b45a1c80 0x55b0b4635aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7be7fff640 0 client.10566 ms_handle_reset on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 >> 172.21.15.113:0/2215298633 conn(0x55b0b45a1540 msgr2=0x55b0b45b85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 shutdown_connections 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 1 -- 172.21.15.113:0/2215298633 wait complete. 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c123f6740 20 client.10566 trim_cache size 0 max 16384 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 10434, 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:09:14.464 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:09:14.465 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:09:14.465 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 5.1671074829999997, 2024-04-25T00:09:14.465 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 92, 2024-04-25T00:09:14.465 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 170, 2024-04-25T00:09:14.465 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 170, 2024-04-25T00:09:14.465 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 10.228683590999999 2024-04-25T00:09:14.465 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bec06be30 msgr2=0x7f7bec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bec06be30 0x7f7bec06e2f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009f50 tx=0x7f7c0400b040 comp rx=0 tx=0).stop 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080d28c0 msgr2=0x7f7c080cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080d28c0 0x7f7c080cb830 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d800 tx=0x7f7bf800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 shutdown_connections 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7bec06be30 0x7f7bec06e2f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c0815e510 0x7f7c080cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c080d28c0 0x7f7c080cb830 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 --2- 172.21.15.113:0/1788080821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c080b61e0 0x7f7c080cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 >> 172.21.15.113:0/1788080821 conn(0x7f7c0806b720 msgr2=0x7f7c0815bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 shutdown_connections 2024-04-25T00:09:14.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.457+0000 7f7c0f6ee640 1 -- 172.21.15.113:0/1788080821 wait complete. 2024-04-25T00:09:14.468 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:09:14.560 INFO:tasks.ceph.mds.b.smithi178.stderr:2024-04-25T00:09:14.560+0000 7fbb3bad9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:09:14.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.565+0000 7fc2da729640 1 Processor -- start 2024-04-25T00:09:14.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.565+0000 7fc2da729640 1 -- start start 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2da729640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc2d4144910 0x7fc2d40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2da729640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d40b5240 0x7fc2d406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2da729640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2d4144cf0 con 0x7fc2d40b5680 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2da729640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d40b4510 con 0x7fc2d4144910 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2da729640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d40b2f30 con 0x7fc2d40b5240 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc2d4144910 0x7fc2d40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d8cc6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc2d40b5680 0x7fc2d40cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39944/0 (socket says 172.21.15.113:39944) 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d8cc6640 1 -- 172.21.15.113:0/3340211057 learned_addr learned my addr 172.21.15.113:0/3340211057 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d37fe640 1 --2- 172.21.15.113:0/3340211057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d40b5240 0x7fc2d406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d3fff640 1 -- 172.21.15.113:0/3340211057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d40b5240 msgr2=0x7fc2d406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d3fff640 1 --2- 172.21.15.113:0/3340211057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d40b5240 0x7fc2d406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d3fff640 1 -- 172.21.15.113:0/3340211057 >> v1:172.21.15.113:6789/0 conn(0x7fc2d40b5680 legacy=0x7fc2d40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d3fff640 1 -- 172.21.15.113:0/3340211057 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d40b3210 con 0x7fc2d4144910 2024-04-25T00:09:14.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.566+0000 7fc2d2ffd640 1 -- 172.21.15.113:0/3340211057 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566946525 0 0) 0x7fc2d4144cf0 con 0x7fc2d40b5680 2024-04-25T00:09:14.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.567+0000 7fc2d3fff640 1 --2- 172.21.15.113:0/3340211057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d4144910 0x7fc2d40b3fd0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40097a0 tx=0x7fc2c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1582c00a21c3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.567+0000 7fc2d2ffd640 1 -- 172.21.15.113:0/3340211057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4037070 con 0x7fc2d4144910 2024-04-25T00:09:14.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.567+0000 7fc2d2ffd640 1 -- 172.21.15.113:0/3340211057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c4032440 con 0x7fc2d4144910 2024-04-25T00:09:14.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.567+0000 7fc2d2ffd640 1 -- 172.21.15.113:0/3340211057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4032810 con 0x7fc2d4144910 2024-04-25T00:09:14.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 -- 172.21.15.113:0/3340211057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d4144910 msgr2=0x7fc2d40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 --2- 172.21.15.113:0/3340211057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d4144910 0x7fc2d40b3fd0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40097a0 tx=0x7fc2c402a180 comp rx=0 tx=0).stop 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 -- 172.21.15.113:0/3340211057 shutdown_connections 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 --2- 172.21.15.113:0/3340211057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d40b5240 0x7fc2d406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 --2- 172.21.15.113:0/3340211057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d4144910 0x7fc2d40b3fd0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 -- 172.21.15.113:0/3340211057 >> 172.21.15.113:0/3340211057 conn(0x7fc2d40b1a50 msgr2=0x7fc2d40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 -- 172.21.15.113:0/3340211057 shutdown_connections 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.568+0000 7fc2da729640 1 -- 172.21.15.113:0/3340211057 wait complete. 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.569+0000 7fc2da729640 1 Processor -- start 2024-04-25T00:09:14.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.569+0000 7fc2da729640 1 -- start start 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d40b5240 0x7fc2d415da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d40b5680 0x7fc2d4142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d4144910 0x7fc2d415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d40b4da0 con 0x7fc2d40b5240 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d40b37d0 con 0x7fc2d40b5680 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d40b4f20 con 0x7fc2d4144910 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d40b5240 0x7fc2d415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d40b5240 0x7fc2d415da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46130/0 (socket says 172.21.15.113:46130) 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 -- 172.21.15.113:0/3558555501 learned_addr learned my addr 172.21.15.113:0/3558555501 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 -- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d4144910 msgr2=0x7fc2d415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d8cc6640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d4144910 0x7fc2d415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d37fe640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d40b5680 0x7fc2d4142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d4144910 0x7fc2d415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 -- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d40b5680 msgr2=0x7fc2d4142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d40b5680 0x7fc2d4142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 -- 172.21.15.113:0/3558555501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d4142ea0 con 0x7fc2d40b5240 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d3fff640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d40b5240 0x7fc2d415da80 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4035040 tx=0x7fc2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d0ff9640 1 -- 172.21.15.113:0/3558555501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4037040 con 0x7fc2d40b5240 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d0ff9640 1 -- 172.21.15.113:0/3558555501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c4030770 con 0x7fc2d40b5240 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2d0ff9640 1 -- 172.21.15.113:0/3558555501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4030b80 con 0x7fc2d40b5240 2024-04-25T00:09:14.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d4143180 con 0x7fc2d40b5240 2024-04-25T00:09:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.570+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d4140570 con 0x7fc2d40b5240 2024-04-25T00:09:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.571+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d40691a0 con 0x7fc2d40b5240 2024-04-25T00:09:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.572+0000 7fc2d0ff9640 1 -- 172.21.15.113:0/3558555501 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2c4044020 con 0x7fc2d40b5240 2024-04-25T00:09:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.572+0000 7fc2d0ff9640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc2c006be30 0x7fc2c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.572+0000 7fc2d0ff9640 1 -- 172.21.15.113:0/3558555501 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fc2c40ab3a0 con 0x7fc2d40b5240 2024-04-25T00:09:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.572+0000 7fc2d37fe640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc2c006be30 0x7fc2c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:14.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.572+0000 7fc2d37fe640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc2c006be30 0x7fc2c006e2f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:14.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.576+0000 7fc2d0ff9640 1 -- 172.21.15.113:0/3558555501 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2c4032e20 con 0x7fc2d40b5240 2024-04-25T00:09:14.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.786+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2d4166cd0 con 0x7fc2d40b5240 2024-04-25T00:09:14.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.786+0000 7fc2d0ff9640 1 -- 172.21.15.113:0/3558555501 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7fc2c4079d50 con 0x7fc2d40b5240 2024-04-25T00:09:14.787 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:09:14.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc2c006be30 msgr2=0x7fc2c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc2c006be30 0x7fc2c006e2f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8008040 comp rx=0 tx=0).stop 2024-04-25T00:09:14.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d40b5240 msgr2=0x7fc2d415da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d40b5240 0x7fc2d415da80 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4035040 tx=0x7fc2c40046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 shutdown_connections 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc2c006be30 0x7fc2c006e2f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d40b5680 0x7fc2d4142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d4144910 0x7fc2d415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 --2- 172.21.15.113:0/3558555501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2d40b5240 0x7fc2d415da80 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 >> 172.21.15.113:0/3558555501 conn(0x7fc2d40b1a50 msgr2=0x7fc2d4071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 shutdown_connections 2024-04-25T00:09:14.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:14.789+0000 7fc2da729640 1 -- 172.21.15.113:0/3558555501 wait complete. 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:09:14.800 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:09:14.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:14.801 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:09:14.816 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:09:14.816 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:09:14.816 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:09:14.816 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:14.816 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:09:14.816 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:14.817 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42125sec preferred_lft 42125sec 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:09:14.818 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:14.819 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:09:14.819 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:09:14.819 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:09:14.819 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:14.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:14.819 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:09:14.878 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:09:14.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:14.879 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:09:14.935 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:09:14.935 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:09:14.935 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:09:14.936 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:09:14.936 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:09:14.936 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:09:14.936 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:09:14.936 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:09:14.936 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:09:14.936 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:09:14.936 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:09:15.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:09:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:09:15.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:15.058 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:09:15.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:09:15.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:09:15.073 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:09:15.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:15.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:09:15.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:09:15.137 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:15.192 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:09:15.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:09:15.193 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:15.246 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:09:15.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:15.247 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:15.328 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:15.328 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:09:15.328 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:09:15.336 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:15.336 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:09:15.337 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:09:15.337 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:09:15.337 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:09:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:09:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:09:15.345 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:09:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:09:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:09:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:09:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:09:15.346 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:09:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:09:15.346 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:09:15.347 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:09:15.347 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:09:15.348 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:09:15.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:15.348 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:15.412 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:09:15.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:09:15.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:09:15.515 INFO:teuthology.orchestra.run.smithi113.stdout:client8326 2024-04-25T00:09:15.515 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:15.515 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:15.530 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:15.530 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:15.587 DEBUG:teuthology.orchestra.run.smithi113:> stdin-killer -- python3 -c ' 2024-04-25T00:09:15.587 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-04-25T00:09:15.587 DEBUG:teuthology.orchestra.run.smithi113:> import uuid 2024-04-25T00:09:15.587 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:09:15.587 DEBUG:teuthology.orchestra.run.smithi113:> n = 1000 2024-04-25T00:09:15.587 DEBUG:teuthology.orchestra.run.smithi113:> create_hard_links = False 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> if 0 > 0: 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> create_hard_links = True 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> dpath = os.path.dirname(path) 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> fnameprefix = os.path.basename(path) 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> os.makedirs(dpath, exist_ok=True) 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> try: 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> for i in range(n): 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T00:09:15.588 DEBUG:teuthology.orchestra.run.smithi113:> f.write(f"{i}") 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> f.flush() 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(f.fileno()) 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> os.unlink(fpath) 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(dirfd) 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> if create_hard_links: 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> for j in range(0): 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> if False: 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> os.fsync(dirfd) 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> finally: 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> os.close(dirfd) 2024-04-25T00:09:15.589 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-04-25T00:09:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:09:16.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:16.463 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T00:09:16.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.604+0000 7fea17272640 1 Processor -- start 2024-04-25T00:09:16.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.604+0000 7fea17272640 1 -- start start 2024-04-25T00:09:16.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea17272640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea1015e2d0 0x7fea1015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea17272640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea1015f4f0 con 0x7fea100b4a40 2024-04-25T00:09:16.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea17272640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea1015f7d0 con 0x7fea100b3080 2024-04-25T00:09:16.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea17272640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fea100cde00 con 0x7fea1015e2d0 2024-04-25T00:09:16.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea1580f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fea100b4a40 0x7fea100ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39950/0 (socket says 172.21.15.113:39950) 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea1580f640 1 -- 172.21.15.113:0/321534883 learned_addr learned my addr 172.21.15.113:0/321534883 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea1500e640 1 --2- 172.21.15.113:0/321534883 >> v2:172.21.15.178:3300/0 conn(0x7fea1015e2d0 0x7fea1015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2594250811 0 0) 0x7fea1015f4f0 con 0x7fea100b4a40 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9fc003520 con 0x7fea100b4a40 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3742940336 0 0) 0x7fe9fc003520 con 0x7fea100b4a40 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 >> v2:172.21.15.178:3300/0 conn(0x7fea1015e2d0 msgr2=0x7fea1015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 --2- 172.21.15.113:0/321534883 >> v2:172.21.15.178:3300/0 conn(0x7fea1015e2d0 0x7fea1015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 >> v1:172.21.15.178:6789/0 conn(0x7fea100b3080 legacy=0x7fea100b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1015fab0 con 0x7fea100b4a40 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.605+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fea080019f0 con 0x7fea100b4a40 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 >> v1:172.21.15.113:6789/0 conn(0x7fea100b4a40 legacy=0x7fea100ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea07fff640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9fc0039b0 0x7fe9fc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea07fff640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc004800 0x7fe9fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea07fff640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0051c0 0x7fe9fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea100cde00 con 0x7fe9fc0039b0 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc00c090 con 0x7fe9fc004800 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc00c290 con 0x7fe9fc0051c0 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1500e640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9fc0039b0 0x7fe9fc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1480d640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc004800 0x7fe9fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1500e640 1 -- 172.21.15.113:0/321534883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0051c0 msgr2=0x7fe9fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1500e640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0051c0 0x7fe9fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1500e640 1 -- 172.21.15.113:0/321534883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc004800 msgr2=0x7fe9fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1500e640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc004800 0x7fe9fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1500e640 1 -- 172.21.15.113:0/321534883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc003520 con 0x7fe9fc0039b0 2024-04-25T00:09:16.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.606+0000 7fea1580f640 1 -- 172.21.15.113:0/321534883 reap_dead start 2024-04-25T00:09:16.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea1500e640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9fc0039b0 0x7fe9fc003df0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009ac0 tx=0x7fe9fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d650f1d360967f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c036070 con 0x7fe9fc0039b0 2024-04-25T00:09:16.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c031400 con 0x7fe9fc0039b0 2024-04-25T00:09:16.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea07fff640 1 -- 172.21.15.113:0/321534883 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c009690 con 0x7fe9fc0039b0 2024-04-25T00:09:16.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea17272640 1 -- 172.21.15.113:0/321534883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9fc0039b0 msgr2=0x7fe9fc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea17272640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9fc0039b0 0x7fe9fc003df0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009ac0 tx=0x7fe9fc005830 comp rx=0 tx=0).stop 2024-04-25T00:09:16.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea17272640 1 -- 172.21.15.113:0/321534883 shutdown_connections 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea17272640 1 --2- 172.21.15.113:0/321534883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9fc0039b0 0x7fe9fc003df0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.607+0000 7fea17272640 1 -- 172.21.15.113:0/321534883 >> 172.21.15.113:0/321534883 conn(0x7fea1006b0f0 msgr2=0x7fea1006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.608+0000 7fea17272640 1 -- 172.21.15.113:0/321534883 shutdown_connections 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.608+0000 7fea17272640 1 -- 172.21.15.113:0/321534883 wait complete. 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 Processor -- start 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 -- start start 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0039b0 0x7fea100b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea10144a80 0x7fea10155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea101557d0 0x7fea1015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea1015e830 con 0x7fea101557d0 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea1015e6b0 con 0x7fea10144a80 2024-04-25T00:09:16.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.609+0000 7fea17272640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea1015e9b0 con 0x7fe9fc0039b0 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea101557d0 0x7fea1015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea101557d0 0x7fea1015b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46146/0 (socket says 172.21.15.113:46146) 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 -- 172.21.15.113:0/237408601 learned_addr learned my addr 172.21.15.113:0/237408601 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1500e640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0039b0 0x7fea100b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 -- 172.21.15.113:0/237408601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0039b0 msgr2=0x7fea100b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0039b0 0x7fea100b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 -- 172.21.15.113:0/237408601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea10144a80 msgr2=0x7fea10155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea10144a80 0x7fea10155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 -- 172.21.15.113:0/237408601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea10154a80 con 0x7fea101557d0 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea1580f640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea101557d0 0x7fea1015b230 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fea08001330 tx=0x7fea0800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea05ffb640 1 -- 172.21.15.113:0/237408601 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08018070 con 0x7fea101557d0 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea05ffb640 1 -- 172.21.15.113:0/237408601 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea08004da0 con 0x7fea101557d0 2024-04-25T00:09:16.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea05ffb640 1 -- 172.21.15.113:0/237408601 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0800a7f0 con 0x7fea101557d0 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea10151850 con 0x7fea101557d0 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.610+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea10151e10 con 0x7fea101557d0 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.612+0000 7fea05ffb640 1 -- 172.21.15.113:0/237408601 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea0800a990 con 0x7fea101557d0 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.612+0000 7fea05ffb640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9f406be30 0x7fe9f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.612+0000 7fea05ffb640 1 -- 172.21.15.113:0/237408601 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fea0808a130 con 0x7fea101557d0 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.612+0000 7fea1500e640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9f406be30 0x7fe9f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.613+0000 7fea1500e640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9f406be30 0x7fe9f406e2f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009ac0 tx=0x7fea0c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.613+0000 7fea19f7c740 1 Processor -- start 2024-04-25T00:09:16.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.613+0000 7fea19f7c740 1 -- start start 2024-04-25T00:09:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e126140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e143b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e0fba40 0x55dc7e141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea19f7c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e05f6b0 con 0x55dc7e143600 2024-04-25T00:09:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e007880 con 0x55dc7e0fba40 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dc7e05b670 con 0x55dc7e125d60 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46162/0 (socket says 172.21.15.113:46162) 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 -- 172.21.15.113:0/3200388491 learned_addr learned my addr 172.21.15.113:0/3200388491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 -- 172.21.15.113:0/3200388491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 msgr2=0x55dc7e126140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 --2- 172.21.15.113:0/3200388491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e126140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 -- 172.21.15.113:0/3200388491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e0fba40 msgr2=0x55dc7e141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 --2- 172.21.15.113:0/3200388491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e0fba40 0x55dc7e141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.614+0000 7fea1480d640 1 -- 172.21.15.113:0/3200388491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7de3d7d0 con 0x55dc7e143600 2024-04-25T00:09:16.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea1480d640 1 --2- 172.21.15.113:0/3200388491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e143b50 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fea00001730 tx=0x7fea00001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d6c65b5799a532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fe9f2ffd640 1 -- 172.21.15.113:0/3200388491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000052e0 con 0x55dc7e143600 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fe9f2ffd640 1 -- 172.21.15.113:0/3200388491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00018070 con 0x55dc7e143600 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fe9f2ffd640 1 -- 172.21.15.113:0/3200388491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0000f9f0 con 0x55dc7e143600 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 -- 172.21.15.113:0/3200388491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 msgr2=0x55dc7e143b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3200388491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e143b50 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fea00001730 tx=0x7fea00001bf0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 -- 172.21.15.113:0/3200388491 shutdown_connections 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3200388491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e0fba40 0x55dc7e141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3200388491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e143b50 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3200388491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e126140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 -- 172.21.15.113:0/3200388491 >> 172.21.15.113:0/3200388491 conn(0x55dc7e0f8600 msgr2=0x55dc7e140b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 -- 172.21.15.113:0/3200388491 shutdown_connections 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.615+0000 7fea19f7c740 1 -- 172.21.15.113:0/3200388491 wait complete. 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.616+0000 7fea19f7c740 1 -- start start 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.616+0000 7fea19f7c740 1 Processor -- start 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e125d60 0x55dc7e115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e11b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e1526b0 0x55dc7e152b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea19f7c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e0fc7e0 con 0x55dc7e143600 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e01af50 con 0x55dc7e125d60 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dc7e0f6530 con 0x55dc7e1526b0 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea19f7c740 10 client.0 _ll_register_callbacks cb 0x55dc7e056000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e11b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e11b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46172/0 (socket says 172.21.15.113:46172) 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 -- 172.21.15.113:0/3917154769 learned_addr learned my addr 172.21.15.113:0/3917154769 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1500e640 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e125d60 0x55dc7e115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1580f640 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e1526b0 0x55dc7e152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 -- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e1526b0 msgr2=0x55dc7e152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e1526b0 0x55dc7e152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 -- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e125d60 msgr2=0x55dc7e115cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e125d60 0x55dc7e115cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.617+0000 7fea1480d640 1 -- 172.21.15.113:0/3917154769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7dd733f0 con 0x55dc7e143600 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea1480d640 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e11b5b0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fea0000f5a0 tx=0x7fea00007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4f72b10ca91e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fe9ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fe9ebfff640 1 -- 172.21.15.113:0/3917154769 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00022070 con 0x55dc7e143600 2024-04-25T00:09:16.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fe9ebfff640 1 -- 172.21.15.113:0/3917154769 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fea000069e0 con 0x55dc7e143600 2024-04-25T00:09:16.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea00018070 con 0x55dc7e143600 2024-04-25T00:09:16.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fe9ebfff640 1 -- 172.21.15.113:0/3917154769 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fea0002d050 con 0x55dc7e143600 2024-04-25T00:09:16.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 10 client.10587 fetch_fsmap learned FSMap version 92 2024-04-25T00:09:16.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55dc7dd330f0 con 0x55dc7e143600 2024-04-25T00:09:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fe9ebfff640 1 -- 172.21.15.113:0/3917154769 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00018070 con 0x55dc7e143600 2024-04-25T00:09:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fe9ebfff640 1 -- 172.21.15.113:0/3917154769 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7fea00030020 con 0x55dc7e143600 2024-04-25T00:09:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 10 client.10587 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:09:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 10 client.10587 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 20 client.10587 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e128040 0x55dc7e11c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 4 client.10587 mds_command: new command op to 10434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 --> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7e06cbf0 con 0x55dc7e128040 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.618+0000 7fea1500e640 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e128040 0x55dc7e11c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.619+0000 7fe9ebfff640 10 client.10587 ms_handle_connect on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.619+0000 7fea1500e640 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e128040 0x55dc7e11c940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7950ba7260c04148 server_cookie=13fc1576c5da6f8d in_seq=0 out_seq=0 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.619+0000 7fe9ebfff640 1 -- 172.21.15.113:0/3917154769 <== mds.0 v2:172.21.15.178:6833/2813561857 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc7e06cbf0 con 0x55dc7e128040 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.619+0000 7fe9ebfff640 10 client.10587 handle_command_reply: tid=0 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 client.10587 shutdown 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 msgr2=0x55dc7e11b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e11b5b0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fea0000f5a0 tx=0x7fea00007c80 comp rx=0 tx=0).stop 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 shutdown_connections 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e128040 0x55dc7e11c940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e1526b0 0x55dc7e152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e11b5b0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fe9ebfff640 0 client.10587 ms_handle_reset on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 --2- 172.21.15.113:0/3917154769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e125d60 0x55dc7e115cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 >> 172.21.15.113:0/3917154769 conn(0x55dc7e0fba40 msgr2=0x55dc7e1440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 shutdown_connections 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 1 -- 172.21.15.113:0/3917154769 wait complete. 2024-04-25T00:09:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.620+0000 7fea19f7c740 20 client.10587 trim_cache size 0 max 16384 2024-04-25T00:09:16.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.637+0000 7fea19f7c740 1 Processor -- start 2024-04-25T00:09:16.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.637+0000 7fea19f7c740 1 -- start start 2024-04-25T00:09:16.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e128880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e128dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e1526b0 0x55dc7e13fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea19f7c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea101fc9a0 con 0x55dc7e143600 2024-04-25T00:09:16.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea101fba10 con 0x55dc7e1526b0 2024-04-25T00:09:16.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea1006d7b0 con 0x55dc7e125d60 2024-04-25T00:09:16.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e128dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e128dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46178/0 (socket says 172.21.15.113:46178) 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 -- 172.21.15.113:0/2782354267 learned_addr learned my addr 172.21.15.113:0/2782354267 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1580f640 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e1526b0 0x55dc7e13fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1500e640 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e128880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 -- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 msgr2=0x55dc7e128880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e128880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 -- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e1526b0 msgr2=0x55dc7e13fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e1526b0 0x55dc7e13fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 -- 172.21.15.113:0/2782354267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7e1264b0 con 0x55dc7e143600 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fea1480d640 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e128dc0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fea00001c30 tx=0x7fea0001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ee4eb5c92c47f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.638+0000 7fe9f27fc640 1 -- 172.21.15.113:0/2782354267 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00022040 con 0x55dc7e143600 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fe9f27fc640 1 -- 172.21.15.113:0/2782354267 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00018070 con 0x55dc7e143600 2024-04-25T00:09:16.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fe9f27fc640 1 -- 172.21.15.113:0/2782354267 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00003810 con 0x55dc7e143600 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 -- 172.21.15.113:0/2782354267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 msgr2=0x55dc7e128dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e128dc0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fea00001c30 tx=0x7fea0001f000 comp rx=0 tx=0).stop 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 -- 172.21.15.113:0/2782354267 shutdown_connections 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e1526b0 0x55dc7e13fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e143600 0x55dc7e128dc0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2782354267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e128880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 -- 172.21.15.113:0/2782354267 >> 172.21.15.113:0/2782354267 conn(0x55dc7e0fba40 msgr2=0x55dc7e0f8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 -- 172.21.15.113:0/2782354267 shutdown_connections 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.639+0000 7fea19f7c740 1 -- 172.21.15.113:0/2782354267 wait complete. 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.640+0000 7fea19f7c740 1 -- start start 2024-04-25T00:09:16.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.640+0000 7fea19f7c740 1 Processor -- start 2024-04-25T00:09:16.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e1bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e143600 0x55dc7e1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e1526b0 0x55dc7e11b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e11a290 con 0x55dc7e1526b0 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea1006de00 con 0x55dc7e143600 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dc7e152b30 con 0x55dc7e125d60 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 10 client.0 _ll_register_callbacks cb 0x7fea00001680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e1526b0 0x55dc7e11b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e1526b0 0x55dc7e11b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46186/0 (socket says 172.21.15.113:46186) 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1500e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e1bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 -- 172.21.15.113:0/2288362798 learned_addr learned my addr 172.21.15.113:0/2288362798 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1480d640 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e143600 0x55dc7e1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 -- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 msgr2=0x55dc7e1bf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e1bf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 -- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e143600 msgr2=0x55dc7e1401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e143600 0x55dc7e1401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 -- 172.21.15.113:0/2288362798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7e11ea80 con 0x55dc7e1526b0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1480d640 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e143600 0x55dc7e1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea1580f640 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e1526b0 0x55dc7e11b170 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fea080548d0 tx=0x7fea0805c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9042211f8a13111f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fe9ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fe9ebfff640 1 -- 172.21.15.113:0/2288362798 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0800ae80 con 0x55dc7e1526b0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fe9ebfff640 1 -- 172.21.15.113:0/2288362798 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fea080578c0 con 0x55dc7e1526b0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea00018070 con 0x55dc7e1526b0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.641+0000 7fe9ebfff640 1 -- 172.21.15.113:0/2288362798 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fea08057ac0 con 0x55dc7e1526b0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 10 client.10593 fetch_fsmap learned FSMap version 92 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55dc7e140ff0 con 0x55dc7e1526b0 2024-04-25T00:09:16.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fe9ebfff640 1 -- 172.21.15.113:0/2288362798 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00018070 con 0x55dc7e1526b0 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fe9ebfff640 1 -- 172.21.15.113:0/2288362798 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7fea080881a0 con 0x55dc7e1526b0 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 10 client.10593 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 10 client.10593 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 20 client.10593 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e0fab00 0x55dc7e11b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 4 client.10593 mds_command: new command op to 10434 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 --> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc7e066f40 con 0x55dc7e0fab00 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea1500e640 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e0fab00 0x55dc7e11b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea1500e640 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e0fab00 0x55dc7e11b6b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78a6a068247568c server_cookie=86a0b9c42987544b in_seq=0 out_seq=0 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fe9ebfff640 10 client.10593 ms_handle_connect on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fe9ebfff640 1 -- 172.21.15.113:0/2288362798 <== mds.0 v2:172.21.15.178:6833/2813561857 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc7e066f40 con 0x55dc7e0fab00 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fe9ebfff640 10 client.10593 handle_command_reply: tid=0 2024-04-25T00:09:16.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.642+0000 7fea19f7c740 1 client.10593 shutdown 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e1526b0 msgr2=0x55dc7e11b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e1526b0 0x55dc7e11b170 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fea080548d0 tx=0x7fea0805c960 comp rx=0 tx=0).stop 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 shutdown_connections 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55dc7e0fab00 0x55dc7e11b6b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc7e143600 0x55dc7e1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc7e1526b0 0x55dc7e11b170 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 --2- 172.21.15.113:0/2288362798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dc7e125d60 0x55dc7e1bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 >> 172.21.15.113:0/2288362798 conn(0x55dc7e0fba40 msgr2=0x55dc7e128b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fe9ebfff640 0 client.10593 ms_handle_reset on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 shutdown_connections 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 1 -- 172.21.15.113:0/2288362798 wait complete. 2024-04-25T00:09:16.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea19f7c740 20 client.10593 trim_cache size 0 max 16384 2024-04-25T00:09:16.651 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:16.651 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:09:16.651 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:09:16.651 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9f406be30 msgr2=0x7fe9f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9f406be30 0x7fe9f406e2f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009ac0 tx=0x7fea0c008030 comp rx=0 tx=0).stop 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea101557d0 msgr2=0x7fea1015b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea101557d0 0x7fea1015b230 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fea08001330 tx=0x7fea0800cc00 comp rx=0 tx=0).stop 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 shutdown_connections 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe9f406be30 0x7fe9f406e2f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea10144a80 0x7fea10155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea101557d0 0x7fea1015b230 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 --2- 172.21.15.113:0/237408601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc0039b0 0x7fea100b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 >> 172.21.15.113:0/237408601 conn(0x7fea1006b0f0 msgr2=0x7fea100b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.643+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 shutdown_connections 2024-04-25T00:09:16.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.644+0000 7fea17272640 1 -- 172.21.15.113:0/237408601 wait complete. 2024-04-25T00:09:16.654 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:09:16.654 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.12:0 damage ls 2024-04-25T00:09:16.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe17cfa9640 1 Processor -- start 2024-04-25T00:09:16.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe17cfa9640 1 -- start start 2024-04-25T00:09:16.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe17cfa9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe17815e510 0x7fe1780d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe17cfa9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1780d2e00 con 0x7fe1780d28c0 2024-04-25T00:09:16.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe17cfa9640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1780d30e0 con 0x7fe1780b63b0 2024-04-25T00:09:16.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe17cfa9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780d33c0 con 0x7fe17815e510 2024-04-25T00:09:16.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe16ffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe1780d28c0 0x7fe1780b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39962/0 (socket says 172.21.15.113:39962) 2024-04-25T00:09:16.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.754+0000 7fe16ffff640 1 -- 172.21.15.113:0/2817466395 learned_addr learned my addr 172.21.15.113:0/2817466395 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe17659c640 1 --2- 172.21.15.113:0/2817466395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe17815e510 0x7fe1780d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe175d9b640 1 -- 172.21.15.113:0/2817466395 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295651425 0 0) 0x7fe1780d2e00 con 0x7fe1780d28c0 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe175d9b640 1 -- 172.21.15.113:0/2817466395 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe164003520 con 0x7fe1780d28c0 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe175d9b640 1 -- 172.21.15.113:0/2817466395 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818075646 0 0) 0x7fe1780d30e0 con 0x7fe1780b63b0 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe175d9b640 1 -- 172.21.15.113:0/2817466395 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1780d2e00 con 0x7fe1780b63b0 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe17659c640 1 -- 172.21.15.113:0/2817466395 >> v1:172.21.15.178:6789/0 conn(0x7fe1780b63b0 legacy=0x7fe1780b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe17659c640 1 -- 172.21.15.113:0/2817466395 >> v1:172.21.15.113:6789/0 conn(0x7fe1780d28c0 legacy=0x7fe1780b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.755+0000 7fe17659c640 1 -- 172.21.15.113:0/2817466395 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1780d36a0 con 0x7fe17815e510 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.756+0000 7fe17659c640 1 --2- 172.21.15.113:0/2817466395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe17815e510 0x7fe1780d2380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe1680097d0 tx=0x7fe16802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660a2cd0b3e156ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.756+0000 7fe175d9b640 1 -- 172.21.15.113:0/2817466395 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168037070 con 0x7fe17815e510 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.756+0000 7fe175d9b640 1 -- 172.21.15.113:0/2817466395 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe168032440 con 0x7fe17815e510 2024-04-25T00:09:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.756+0000 7fe175d9b640 1 -- 172.21.15.113:0/2817466395 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1680327f0 con 0x7fe17815e510 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.757+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2817466395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe17815e510 msgr2=0x7fe1780d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.757+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2817466395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe17815e510 0x7fe1780d2380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe1680097d0 tx=0x7fe16802a180 comp rx=0 tx=0).stop 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.757+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2817466395 shutdown_connections 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.757+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2817466395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe17815e510 0x7fe1780d2380 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.757+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2817466395 >> 172.21.15.113:0/2817466395 conn(0x7fe17806b720 msgr2=0x7fe1780d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.757+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2817466395 shutdown_connections 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.757+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2817466395 wait complete. 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.759+0000 7fe17cfa9640 1 Processor -- start 2024-04-25T00:09:16.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.759+0000 7fe17cfa9640 1 -- start start 2024-04-25T00:09:16.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe17cfa9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1780b63b0 0x7fe178154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe17cfa9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1780d28c0 0x7fe178155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe17cfa9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe17815e510 0x7fe1781599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe17cfa9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780b9dd0 con 0x7fe17815e510 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe17cfa9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780d3ea0 con 0x7fe1780d28c0 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe17cfa9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780b7ad0 con 0x7fe1780b63b0 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe17815e510 0x7fe1781599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe17815e510 0x7fe1781599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46198/0 (socket says 172.21.15.113:46198) 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 -- 172.21.15.113:0/2411218642 learned_addr learned my addr 172.21.15.113:0/2411218642 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe16ffff640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1780d28c0 0x7fe178155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe17659c640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1780b63b0 0x7fe178154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 -- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1780b63b0 msgr2=0x7fe178154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1780b63b0 0x7fe178154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 -- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1780d28c0 msgr2=0x7fe178155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1780d28c0 0x7fe178155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 -- 172.21.15.113:0/2411218642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe178156d90 con 0x7fe17815e510 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.760+0000 7fe176d9d640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe17815e510 0x7fe1781599a0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fe17000c760 tx=0x7fe17000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.761+0000 7fe16effd640 1 -- 172.21.15.113:0/2411218642 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170003b70 con 0x7fe17815e510 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.761+0000 7fe16effd640 1 -- 172.21.15.113:0/2411218642 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe170003d10 con 0x7fe17815e510 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.761+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe178157070 con 0x7fe17815e510 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.761+0000 7fe16effd640 1 -- 172.21.15.113:0/2411218642 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000aa90 con 0x7fe17815e510 2024-04-25T00:09:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.761+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe178157630 con 0x7fe17815e510 2024-04-25T00:09:16.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.763+0000 7fe16effd640 1 -- 172.21.15.113:0/2411218642 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe17000ac30 con 0x7fe17815e510 2024-04-25T00:09:16.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.763+0000 7fe16effd640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe15806be30 0x7fe15806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.763+0000 7fe16effd640 1 -- 172.21.15.113:0/2411218642 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fe17008ac50 con 0x7fe17815e510 2024-04-25T00:09:16.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.763+0000 7fe17659c640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe15806be30 0x7fe15806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.763+0000 7fe17659c640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe15806be30 0x7fe15806e2f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe168034000 tx=0x7fe16802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.764+0000 7fe17fd98740 1 Processor -- start 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.764+0000 7fe17fd98740 1 -- start start 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725842e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725b4330 0x55a572589140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17fd98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a5724ed670 con 0x55a5725d1950 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a572495970 con 0x55a5725b4a70 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a5724e9630 con 0x55a5725b4330 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725842e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725842e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46208/0 (socket says 172.21.15.113:46208) 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 -- 172.21.15.113:0/315565199 learned_addr learned my addr 172.21.15.113:0/315565199 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe16ffff640 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe176d9d640 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725b4330 0x55a572589140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 -- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725b4330 msgr2=0x55a572589140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725b4330 0x55a572589140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 -- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 msgr2=0x55a5725b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 -- 172.21.15.113:0/315565199 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5724b9400 con 0x55a5725d1950 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe16ffff640 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe17659c640 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725842e0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098c0 tx=0x7fe16802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867e2b50f0a54dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe157fff640 1 -- 172.21.15.113:0/315565199 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168037040 con 0x55a5725d1950 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe157fff640 1 -- 172.21.15.113:0/315565199 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16804d070 con 0x55a5725d1950 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.765+0000 7fe157fff640 1 -- 172.21.15.113:0/315565199 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168032e00 con 0x55a5725d1950 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 -- 172.21.15.113:0/315565199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 msgr2=0x55a5725842e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725842e0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098c0 tx=0x7fe16802c000 comp rx=0 tx=0).stop 2024-04-25T00:09:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 -- 172.21.15.113:0/315565199 shutdown_connections 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725b4330 0x55a572589140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 --2- 172.21.15.113:0/315565199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725842e0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 -- 172.21.15.113:0/315565199 >> 172.21.15.113:0/315565199 conn(0x55a572586550 msgr2=0x55a5725cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 -- 172.21.15.113:0/315565199 shutdown_connections 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.766+0000 7fe17fd98740 1 -- 172.21.15.113:0/315565199 wait complete. 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.767+0000 7fe17fd98740 1 -- start start 2024-04-25T00:09:16.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.767+0000 7fe17fd98740 1 Processor -- start 2024-04-25T00:09:16.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a572588700 con 0x55a5725d1950 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a5724a9050 con 0x55a5725b4a70 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a572587af0 con 0x55a5725c6f20 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 10 client.0 _ll_register_callbacks cb 0x55a572200be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46222/0 (socket says 172.21.15.113:46222) 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 -- 172.21.15.113:0/184486392 learned_addr learned my addr 172.21.15.113:0/184486392 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe176d9d640 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17659c640 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 -- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 msgr2=0x55a5725cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 -- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 msgr2=0x55a5725a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 -- 172.21.15.113:0/184486392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57247adc0 con 0x55a5725d1950 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe16ffff640 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725c69e0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fe160007700 tx=0x7fe1600016a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3c0cc7cfe97a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe1557fa640 1 -- 172.21.15.113:0/184486392 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160019020 con 0x55a5725d1950 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe1557fa640 1 -- 172.21.15.113:0/184486392 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fe160015650 con 0x55a5725d1950 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.768+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16804d070 con 0x55a5725d1950 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe1557fa640 1 -- 172.21.15.113:0/184486392 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe160023050 con 0x55a5725d1950 2024-04-25T00:09:16.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 10 client.10605 fetch_fsmap learned FSMap version 92 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55a5722013f0 con 0x55a5725d1950 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe1557fa640 1 -- 172.21.15.113:0/184486392 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16804d070 con 0x55a5725d1950 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe1557fa640 1 -- 172.21.15.113:0/184486392 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7fe160026020 con 0x55a5725d1950 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 10 client.10605 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 10 client.10605 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 20 client.10605 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b5430 0x55a5725b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 4 client.10605 mds_command: new command op to 10434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 --> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5724fabb0 con 0x55a5725b5430 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17659c640 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b5430 0x55a5725b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe1557fa640 10 client.10605 ms_handle_connect on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.769+0000 7fe17659c640 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b5430 0x55a5725b5870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e59c489558ed8d server_cookie=d75521044e84462b in_seq=0 out_seq=0 2024-04-25T00:09:16.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe1557fa640 1 -- 172.21.15.113:0/184486392 <== mds.0 v2:172.21.15.178:6833/2813561857 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5724fabb0 con 0x55a5725b5430 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe1557fa640 10 client.10605 handle_command_reply: tid=0 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 client.10605 shutdown 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 msgr2=0x55a5725c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725c69e0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fe160007700 tx=0x7fe1600016a0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 shutdown_connections 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b5430 0x55a5725b5870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725d1950 0x55a5725c69e0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 --2- 172.21.15.113:0/184486392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe1557fa640 0 client.10605 ms_handle_reset on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 >> 172.21.15.113:0/184486392 conn(0x55a5725b4330 msgr2=0x55a5725d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 shutdown_connections 2024-04-25T00:09:16.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 1 -- 172.21.15.113:0/184486392 wait complete. 2024-04-25T00:09:16.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.770+0000 7fe17fd98740 20 client.10605 trim_cache size 0 max 16384 2024-04-25T00:09:16.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.783+0000 7fe17fd98740 1 Processor -- start 2024-04-25T00:09:16.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.783+0000 7fe17fd98740 1 -- start start 2024-04-25T00:09:16.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725c6f20 0x55a572589070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725d1950 0x55a5725895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe17fd98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1782007a0 con 0x55a5725c6f20 2024-04-25T00:09:16.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ff7c0 con 0x55a5725b4a70 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe1781feb50 con 0x55a5725d1950 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725c6f20 0x55a572589070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725c6f20 0x55a572589070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46230/0 (socket says 172.21.15.113:46230) 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe16ffff640 1 -- 172.21.15.113:0/239886767 learned_addr learned my addr 172.21.15.113:0/239886767 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe17659c640 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe176d9d640 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725d1950 0x55a5725895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe16ffff640 1 -- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725d1950 msgr2=0x55a5725895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.784+0000 7fe16ffff640 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725d1950 0x55a5725895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe16ffff640 1 -- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 msgr2=0x55a5725ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe16ffff640 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe16ffff640 1 -- 172.21.15.113:0/239886767 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a572587690 con 0x55a5725c6f20 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17659c640 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe16ffff640 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725c6f20 0x55a572589070 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fe160006a50 tx=0x7fe16000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6764c46a7607d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe1577fe640 1 -- 172.21.15.113:0/239886767 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160019020 con 0x55a5725c6f20 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe1577fe640 1 -- 172.21.15.113:0/239886767 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe160001ba0 con 0x55a5725c6f20 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe1577fe640 1 -- 172.21.15.113:0/239886767 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16001e610 con 0x55a5725c6f20 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17fd98740 1 -- 172.21.15.113:0/239886767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725c6f20 msgr2=0x55a572589070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17fd98740 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725c6f20 0x55a572589070 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fe160006a50 tx=0x7fe16000f000 comp rx=0 tx=0).stop 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17fd98740 1 -- 172.21.15.113:0/239886767 shutdown_connections 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17fd98740 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725d1950 0x55a5725895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17fd98740 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725c6f20 0x55a572589070 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17fd98740 1 --2- 172.21.15.113:0/239886767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.785+0000 7fe17fd98740 1 -- 172.21.15.113:0/239886767 >> 172.21.15.113:0/239886767 conn(0x55a5725b4330 msgr2=0x55a57258acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.786+0000 7fe17fd98740 1 -- 172.21.15.113:0/239886767 shutdown_connections 2024-04-25T00:09:16.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.786+0000 7fe17fd98740 1 -- 172.21.15.113:0/239886767 wait complete. 2024-04-25T00:09:16.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.786+0000 7fe17fd98740 1 -- start start 2024-04-25T00:09:16.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.786+0000 7fe17fd98740 1 Processor -- start 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17fd98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725d1950 0x55a5725de9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17fd98740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ff170 con 0x55a5725b4a70 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a5724a4ab0 con 0x55a5725d1950 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17fd98740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a572649790 con 0x55a5725c6f20 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17fd98740 10 client.0 _ll_register_callbacks cb 0x7fe160006af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46242/0 (socket says 172.21.15.113:46242) 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 -- 172.21.15.113:0/2343620330 learned_addr learned my addr 172.21.15.113:0/2343620330 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe176d9d640 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725d1950 0x55a5725de9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe16ffff640 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 -- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 msgr2=0x55a5725ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 -- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725d1950 msgr2=0x55a5725de9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725d1950 0x55a5725de9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.787+0000 7fe17659c640 1 -- 172.21.15.113:0/2343620330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5725b6a90 con 0x55a5725b4a70 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17659c640 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fe16802a1c0 tx=0x7fe168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d5c90095a9b078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe1557fa640 1 -- 172.21.15.113:0/2343620330 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168037040 con 0x55a5725b4a70 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe1557fa640 1 -- 172.21.15.113:0/2343620330 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fe168058b50 con 0x55a5725b4a70 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe160001ba0 con 0x55a5725b4a70 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe1557fa640 1 -- 172.21.15.113:0/2343620330 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe168063050 con 0x55a5725b4a70 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 10 client.10611 fetch_fsmap learned FSMap version 92 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55a5722cb7d0 con 0x55a5725b4a70 2024-04-25T00:09:16.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe1557fa640 1 -- 172.21.15.113:0/2343620330 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe160001ba0 con 0x55a5725b4a70 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe1557fa640 1 -- 172.21.15.113:0/2343620330 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 92) ==== 3494+0+0 (secure 0 0 0) 0x7fe168066020 con 0x55a5725b4a70 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 10 client.10611 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 10 client.10611 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 20 client.10611 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b6190 0x55a5725deee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 4 client.10611 mds_command: new command op to 10434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 --> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5724f4f00 con 0x55a5725b6190 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.788+0000 7fe16ffff640 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b6190 0x55a5725deee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe1557fa640 10 client.10611 ms_handle_connect on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe16ffff640 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b6190 0x55a5725deee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879713b080c15925 server_cookie=49daba5046921d7a in_seq=0 out_seq=0 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe1557fa640 1 -- 172.21.15.113:0/2343620330 <== mds.0 v2:172.21.15.178:6833/2813561857 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5724f4f00 con 0x55a5725b6190 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe1557fa640 10 client.10611 handle_command_reply: tid=0 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 client.10611 shutdown 2024-04-25T00:09:16.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725b4a70 msgr2=0x55a5725ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fe16802a1c0 tx=0x7fe168004730 comp rx=0 tx=0).stop 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 shutdown_connections 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:6833/2813561857,v1:172.21.15.178:6835/2813561857] conn(0x55a5725b6190 0x55a5725deee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5725d1950 0x55a5725de9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a5725c6f20 0x55a5725ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe1557fa640 0 client.10611 ms_handle_reset on v2:172.21.15.178:6833/2813561857 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 --2- 172.21.15.113:0/2343620330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5725b4a70 0x55a5725ce280 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 >> 172.21.15.113:0/2343620330 conn(0x55a5725b4330 msgr2=0x55a5725d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 shutdown_connections 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 1 -- 172.21.15.113:0/2343620330 wait complete. 2024-04-25T00:09:16.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.789+0000 7fe17fd98740 20 client.10611 trim_cache size 0 max 16384 2024-04-25T00:09:16.797 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-25T00:09:16.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe15806be30 msgr2=0x7fe15806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe15806be30 0x7fe15806e2f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe168034000 tx=0x7fe16802e000 comp rx=0 tx=0).stop 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe17815e510 msgr2=0x7fe1781599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe17815e510 0x7fe1781599a0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fe17000c760 tx=0x7fe17000cc20 comp rx=0 tx=0).stop 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 shutdown_connections 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe15806be30 0x7fe15806e2f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1780d28c0 0x7fe178155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe17815e510 0x7fe1781599a0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 --2- 172.21.15.113:0/2411218642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1780b63b0 0x7fe178154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 >> 172.21.15.113:0/2411218642 conn(0x7fe17806b720 msgr2=0x7fe17815c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 shutdown_connections 2024-04-25T00:09:16.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:16.790+0000 7fe17cfa9640 1 -- 172.21.15.113:0/2411218642 wait complete. 2024-04-25T00:09:16.801 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:09:16.801 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:09:16.836 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:09:16.869 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:09:16.870 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:09:16.870 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:16.870 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:16.892 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:16.892 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:16.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:17.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.096+0000 7f97c8ad6640 1 Processor -- start 2024-04-25T00:09:17.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.096+0000 7f97c8ad6640 1 -- start start 2024-04-25T00:09:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.096+0000 7f97c8ad6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.096+0000 7f97c8ad6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97c40d2290 0x7f97c40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.096+0000 7f97c8ad6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c40d3de0 con 0x7f97c40d0b00 2024-04-25T00:09:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.096+0000 7f97c8ad6640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f97c40d3410 con 0x7f97c40d2290 2024-04-25T00:09:17.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.096+0000 7f97c8ad6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97c40d2800 con 0x7f97c415e320 2024-04-25T00:09:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97c259c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97bbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97c40d2290 0x7f97c40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97bbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97c40d2290 0x7f97c40b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:59622/0 (socket says 172.21.15.113:59622) 2024-04-25T00:09:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97bbfff640 1 -- 172.21.15.113:0/2731373106 learned_addr learned my addr 172.21.15.113:0/2731373106 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97c2d9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f97c40d0b00 0x7f97c40b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:56804/0 (socket says 172.21.15.113:56804) 2024-04-25T00:09:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97c259c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c40d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:51222/0 (socket says 172.21.15.113:51222) 2024-04-25T00:09:17.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97bbfff640 1 -- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 msgr2=0x7f97c40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97bbfff640 1 --2- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c40d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97bbfff640 1 -- 172.21.15.113:0/2731373106 >> v1:172.21.15.178:6789/0 conn(0x7f97c40d0b00 legacy=0x7f97c40b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97bbfff640 1 -- 172.21.15.113:0/2731373106 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c40d2ae0 con 0x7f97c40d2290 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97c1d9b640 1 -- 172.21.15.113:0/2731373106 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557202457 0 0) 0x7f97c40d3de0 con 0x7f97c40d0b00 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.097+0000 7f97c259c640 1 --2- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c40d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97bbfff640 1 --2- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d2290 0x7f97c40b9aa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009870 tx=0x7f97ac02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d309d9ad0b64c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c1d9b640 1 -- 172.21.15.113:0/2731373106 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac037070 con 0x7f97c40d2290 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c1d9b640 1 -- 172.21.15.113:0/2731373106 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac02ae50 con 0x7f97c40d2290 2024-04-25T00:09:17.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c1d9b640 1 -- 172.21.15.113:0/2731373106 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac032860 con 0x7f97c40d2290 2024-04-25T00:09:17.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 -- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d2290 msgr2=0x7f97c40b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d2290 0x7f97c40b9aa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009870 tx=0x7f97ac02a300 comp rx=0 tx=0).stop 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 -- 172.21.15.113:0/2731373106 shutdown_connections 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d2290 0x7f97c40b9aa0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/2731373106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c40d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 -- 172.21.15.113:0/2731373106 >> 172.21.15.113:0/2731373106 conn(0x7f97c40b5c20 msgr2=0x7f97c40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 -- 172.21.15.113:0/2731373106 shutdown_connections 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.098+0000 7f97c8ad6640 1 -- 172.21.15.113:0/2731373106 wait complete. 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 Processor -- start 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 -- start start 2024-04-25T00:09:17.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d0b00 0x7f97c41f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c40d2290 0x7f97c4160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c41518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97c40b9fe0 con 0x7f97c40d2290 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97c40d30a0 con 0x7f97c40d0b00 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97c8ad6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97c415e700 con 0x7f97c415e320 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c40d2290 0x7f97c4160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c40d2290 0x7f97c4160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46252/0 (socket says 172.21.15.113:46252) 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.100+0000 7f97bbfff640 1 -- 172.21.15.113:0/1083923891 learned_addr learned my addr 172.21.15.113:0/1083923891 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97c2d9d640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c41518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97c259c640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d0b00 0x7f97c41f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97bbfff640 1 -- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 msgr2=0x7f97c41518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97bbfff640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c41518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97bbfff640 1 -- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d0b00 msgr2=0x7f97c41f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97bbfff640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d0b00 0x7f97c41f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97bbfff640 1 -- 172.21.15.113:0/1083923891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c41f43e0 con 0x7f97c40d2290 2024-04-25T00:09:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97bbfff640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c40d2290 0x7f97c4160050 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f97ac035040 tx=0x7f97ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97baffd640 1 -- 172.21.15.113:0/1083923891 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac037040 con 0x7f97c40d2290 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97baffd640 1 -- 172.21.15.113:0/1083923891 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac004230 con 0x7f97c40d2290 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c41f46c0 con 0x7f97c40d2290 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97baffd640 1 -- 172.21.15.113:0/1083923891 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac003610 con 0x7f97c40d2290 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.101+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c41f4c80 con 0x7f97c40d2290 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.102+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c40691a0 con 0x7f97c40d2290 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.102+0000 7f97baffd640 1 -- 172.21.15.113:0/1083923891 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97ac03a070 con 0x7f97c40d2290 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.103+0000 7f97baffd640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97a406be30 0x7f97a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.103+0000 7f97baffd640 1 -- 172.21.15.113:0/1083923891 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f97ac0ab340 con 0x7f97c40d2290 2024-04-25T00:09:17.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.103+0000 7f97c259c640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97a406be30 0x7f97a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.103+0000 7f97c259c640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97a406be30 0x7f97a406e2f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009560 tx=0x7f97b4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.107+0000 7f97baffd640 1 -- 172.21.15.113:0/1083923891 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97ac041020 con 0x7f97c40d2290 2024-04-25T00:09:17.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.282+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97c415d890 con 0x7f97c40d2290 2024-04-25T00:09:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.283+0000 7f97baffd640 1 -- 172.21.15.113:0/1083923891 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16727 (secure 0 0 0) 0x7f97ac079730 con 0x7f97c40d2290 2024-04-25T00:09:17.285 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:17.285 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:10.300297+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:06.844691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:07.603153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97a406be30 msgr2=0x7f97a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97a406be30 0x7f97a406e2f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009560 tx=0x7f97b4009190 comp rx=0 tx=0).stop 2024-04-25T00:09:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c40d2290 msgr2=0x7f97c4160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c40d2290 0x7f97c4160050 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f97ac035040 tx=0x7f97ac0046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 shutdown_connections 2024-04-25T00:09:17.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f97a406be30 0x7f97a406e2f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c415e320 0x7f97c41518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c40d2290 0x7f97c4160050 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 --2- 172.21.15.113:0/1083923891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c40d0b00 0x7f97c41f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 >> 172.21.15.113:0/1083923891 conn(0x7f97c40b5c20 msgr2=0x7f97c40b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:17.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 shutdown_connections 2024-04-25T00:09:17.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.285+0000 7f97c8ad6640 1 -- 172.21.15.113:0/1083923891 wait complete. 2024-04-25T00:09:17.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:09:06.844691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:09:07.603153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:09:17.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:09:17.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.407+0000 7f2b7893b640 1 Processor -- start 2024-04-25T00:09:17.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.407+0000 7f2b7893b640 1 -- start start 2024-04-25T00:09:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.407+0000 7f2b7893b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b740b4a40 0x7f2b74165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.407+0000 7f2b7893b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7415f4f0 con 0x7f2b7415e2d0 2024-04-25T00:09:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.407+0000 7f2b7893b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7415f7d0 con 0x7f2b740b3080 2024-04-25T00:09:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.407+0000 7f2b7893b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2b740cde00 con 0x7f2b740b4a40 2024-04-25T00:09:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.408+0000 7f2b72d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b740b4a40 0x7f2b74165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.408+0000 7f2b7259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2b7415e2d0 0x7f2b7415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39968/0 (socket says 172.21.15.113:39968) 2024-04-25T00:09:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.408+0000 7f2b7259c640 1 -- 172.21.15.113:0/2585461750 learned_addr learned my addr 172.21.15.113:0/2585461750 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.408+0000 7f2b72d9d640 1 -- 172.21.15.113:0/2585461750 >> v1:172.21.15.178:6789/0 conn(0x7f2b740b3080 legacy=0x7f2b740b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.408+0000 7f2b72d9d640 1 -- 172.21.15.113:0/2585461750 >> v1:172.21.15.113:6789/0 conn(0x7f2b7415e2d0 legacy=0x7f2b7415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.408+0000 7f2b72d9d640 1 -- 172.21.15.113:0/2585461750 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7415fab0 con 0x7f2b740b4a40 2024-04-25T00:09:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.408+0000 7f2b7159a640 1 -- 172.21.15.113:0/2585461750 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 868705358 0 0) 0x7f2b7415f4f0 con 0x7f2b7415e2d0 2024-04-25T00:09:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b72d9d640 1 --2- 172.21.15.113:0/2585461750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b4a40 0x7f2b74165f80 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f2b740cdf80 tx=0x7f2b6c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c86ea6b48f8385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7159a640 1 -- 172.21.15.113:0/2585461750 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c037070 con 0x7f2b740b4a40 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7159a640 1 -- 172.21.15.113:0/2585461750 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c032440 con 0x7f2b740b4a40 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7159a640 1 -- 172.21.15.113:0/2585461750 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c032810 con 0x7f2b740b4a40 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7893b640 1 -- 172.21.15.113:0/2585461750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b4a40 msgr2=0x7f2b74165f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7893b640 1 --2- 172.21.15.113:0/2585461750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b4a40 0x7f2b74165f80 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f2b740cdf80 tx=0x7f2b6c02a210 comp rx=0 tx=0).stop 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7893b640 1 -- 172.21.15.113:0/2585461750 shutdown_connections 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7893b640 1 --2- 172.21.15.113:0/2585461750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b4a40 0x7f2b74165f80 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7893b640 1 -- 172.21.15.113:0/2585461750 >> 172.21.15.113:0/2585461750 conn(0x7f2b7406b0f0 msgr2=0x7f2b7406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7893b640 1 -- 172.21.15.113:0/2585461750 shutdown_connections 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.409+0000 7f2b7893b640 1 -- 172.21.15.113:0/2585461750 wait complete. 2024-04-25T00:09:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 Processor -- start 2024-04-25T00:09:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 -- start start 2024-04-25T00:09:17.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b3080 0x7f2b741466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b740b4a40 0x7f2b7414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7415e2d0 0x7f2b7414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740cf420 con 0x7f2b740b3080 2024-04-25T00:09:17.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740cf2a0 con 0x7f2b7415e2d0 2024-04-25T00:09:17.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7893b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740cf5a0 con 0x7f2b740b4a40 2024-04-25T00:09:17.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b3080 0x7f2b741466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b3080 0x7f2b741466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46278/0 (socket says 172.21.15.113:46278) 2024-04-25T00:09:17.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.411+0000 7f2b7259c640 1 -- 172.21.15.113:0/639101060 learned_addr learned my addr 172.21.15.113:0/639101060 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:17.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b71d9b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b740b4a40 0x7f2b7414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b72d9d640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7415e2d0 0x7f2b7414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7259c640 1 -- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b740b4a40 msgr2=0x7f2b7414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7259c640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b740b4a40 0x7f2b7414dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7259c640 1 -- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7415e2d0 msgr2=0x7f2b7414e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7259c640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7415e2d0 0x7f2b7414e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7259c640 1 -- 172.21.15.113:0/639101060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b740afee0 con 0x7f2b740b3080 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b72d9d640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7415e2d0 0x7f2b7414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7259c640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b3080 0x7f2b741466c0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c780 tx=0x7f2b6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b62ffd640 1 -- 172.21.15.113:0/639101060 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64018070 con 0x7f2b740b3080 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b62ffd640 1 -- 172.21.15.113:0/639101060 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6400a640 con 0x7f2b740b3080 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b62ffd640 1 -- 172.21.15.113:0/639101060 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400aa70 con 0x7f2b740b3080 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b740b01c0 con 0x7f2b740b3080 2024-04-25T00:09:17.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.412+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b740b0780 con 0x7f2b740b3080 2024-04-25T00:09:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.413+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b740691a0 con 0x7f2b740b3080 2024-04-25T00:09:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.413+0000 7f2b62ffd640 1 -- 172.21.15.113:0/639101060 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b6401b070 con 0x7f2b740b3080 2024-04-25T00:09:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.414+0000 7f2b62ffd640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b5406be30 0x7f2b5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.414+0000 7f2b62ffd640 1 -- 172.21.15.113:0/639101060 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f2b6408a390 con 0x7f2b740b3080 2024-04-25T00:09:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.414+0000 7f2b71d9b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b5406be30 0x7f2b5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.414+0000 7f2b71d9b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b5406be30 0x7f2b5406e2f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2b74168340 tx=0x7f2b68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.418+0000 7f2b62ffd640 1 -- 172.21.15.113:0/639101060 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b64058780 con 0x7f2b740b3080 2024-04-25T00:09:17.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.589+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b740cc320 con 0x7f2b740b3080 2024-04-25T00:09:17.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.590+0000 7f2b62ffd640 1 -- 172.21.15.113:0/639101060 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) ==== 93+0+10 (secure 0 0 0) 0x7f2b6405ca80 con 0x7f2b740b3080 2024-04-25T00:09:17.591 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:09:17.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b5406be30 msgr2=0x7f2b5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b5406be30 0x7f2b5406e2f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2b74168340 tx=0x7f2b68009040 comp rx=0 tx=0).stop 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b3080 msgr2=0x7f2b741466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b3080 0x7f2b741466c0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c780 tx=0x7f2b6400cc40 comp rx=0 tx=0).stop 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 shutdown_connections 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b5406be30 0x7f2b5406e2f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b7415e2d0 0x7f2b7414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b740b4a40 0x7f2b7414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 --2- 172.21.15.113:0/639101060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b3080 0x7f2b741466c0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 >> 172.21.15.113:0/639101060 conn(0x7f2b7406b0f0 msgr2=0x7f2b740cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 shutdown_connections 2024-04-25T00:09:17.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.592+0000 7f2b7893b640 1 -- 172.21.15.113:0/639101060 wait complete. 2024-04-25T00:09:17.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:09:17.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.705+0000 7fcfa69c1640 1 Processor -- start 2024-04-25T00:09:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.705+0000 7fcfa69c1640 1 -- start start 2024-04-25T00:09:17.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcfa69c1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfa015e2d0 0x7fcfa015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcfa69c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcfa69c1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa015f4f0 con 0x7fcfa015e6b0 2024-04-25T00:09:17.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcfa69c1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa00cde00 con 0x7fcfa015e2d0 2024-04-25T00:09:17.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcfa69c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa015f7d0 con 0x7fcfa00b3080 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfa015e2d0 0x7fcfa015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfa015e2d0 0x7fcfa015ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34862/0 (socket says 172.21.15.113:34862) 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9ffff640 1 -- 172.21.15.113:0/650260347 learned_addr learned my addr 172.21.15.113:0/650260347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9f7fe640 1 --2- 172.21.15.113:0/650260347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9ffff640 1 -- 172.21.15.113:0/650260347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 msgr2=0x7fcfa00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9ffff640 1 --2- 172.21.15.113:0/650260347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9ffff640 1 -- 172.21.15.113:0/650260347 >> v1:172.21.15.178:6789/0 conn(0x7fcfa015e6b0 legacy=0x7fcfa0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.706+0000 7fcf9ffff640 1 -- 172.21.15.113:0/650260347 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa015fab0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.707+0000 7fcf9ffff640 1 --2- 172.21.15.113:0/650260347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa015ebf0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000bb10 tx=0x7fcf9002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95841a3e519ff67a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.707+0000 7fcf9effd640 1 -- 172.21.15.113:0/650260347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90032790 con 0x7fcfa015e2d0 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.707+0000 7fcf9effd640 1 -- 172.21.15.113:0/650260347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf90038070 con 0x7fcfa015e2d0 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.707+0000 7fcf9effd640 1 -- 172.21.15.113:0/650260347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90032b50 con 0x7fcfa015e2d0 2024-04-25T00:09:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 -- 172.21.15.113:0/650260347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 msgr2=0x7fcfa015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/650260347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa015ebf0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000bb10 tx=0x7fcf9002c890 comp rx=0 tx=0).stop 2024-04-25T00:09:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 -- 172.21.15.113:0/650260347 shutdown_connections 2024-04-25T00:09:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/650260347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa00b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/650260347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa015ebf0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 -- 172.21.15.113:0/650260347 >> 172.21.15.113:0/650260347 conn(0x7fcfa006b0f0 msgr2=0x7fcfa006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 -- 172.21.15.113:0/650260347 shutdown_connections 2024-04-25T00:09:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.708+0000 7fcfa69c1640 1 -- 172.21.15.113:0/650260347 wait complete. 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.709+0000 7fcfa69c1640 1 Processor -- start 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.709+0000 7fcfa69c1640 1 -- start start 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa01429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa0150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa015e6b0 0x7fcfa0150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa015f2b0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa015f130 con 0x7fcfa015e6b0 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa00cf2a0 con 0x7fcfa00b3080 2024-04-25T00:09:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa0150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa0150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34874/0 (socket says 172.21.15.113:34874) 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 -- 172.21.15.113:0/2539871701 learned_addr learned my addr 172.21.15.113:0/2539871701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa4f5e640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa015e6b0 0x7fcfa0150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9ffff640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa01429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 -- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 msgr2=0x7fcfa01429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa01429a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 -- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa015e6b0 msgr2=0x7fcfa0150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa015e6b0 0x7fcfa0150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 -- 172.21.15.113:0/2539871701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa01515c0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa4f5e640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa015e6b0 0x7fcfa0150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9f7fe640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa0150a70 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e9b0 tx=0x7fcf9400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9cff9640 1 -- 172.21.15.113:0/2539871701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94017070 con 0x7fcfa015e2d0 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9cff9640 1 -- 172.21.15.113:0/2539871701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9400c9a0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa0152720 con 0x7fcfa015e2d0 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcf9cff9640 1 -- 172.21.15.113:0/2539871701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400cd70 con 0x7fcfa015e2d0 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.710+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa0142ee0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.711+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa00691a0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.712+0000 7fcf9cff9640 1 -- 172.21.15.113:0/2539871701 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf9401a070 con 0x7fcfa015e2d0 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.712+0000 7fcf9cff9640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf8c06be30 0x7fcf8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.712+0000 7fcf9cff9640 1 -- 172.21.15.113:0/2539871701 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fcf9408b220 con 0x7fcfa015e2d0 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.712+0000 7fcf9ffff640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf8c06be30 0x7fcf8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:17.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.713+0000 7fcf9ffff640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf8c06be30 0x7fcf8c06e2f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcf90037000 tx=0x7fcf9003e040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:17.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.716+0000 7fcf9cff9640 1 -- 172.21.15.113:0/2539871701 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf94059bd0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.889+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcfa0159ad0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.889+0000 7fcf9cff9640 1 -- 172.21.15.113:0/2539871701 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) ==== 104+0+11 (secure 0 0 0) 0x7fcf940594d0 con 0x7fcfa015e2d0 2024-04-25T00:09:17.890 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:09:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf8c06be30 msgr2=0x7fcf8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf8c06be30 0x7fcf8c06e2f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcf90037000 tx=0x7fcf9003e040 comp rx=0 tx=0).stop 2024-04-25T00:09:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 msgr2=0x7fcfa0150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa0150a70 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e9b0 tx=0x7fcf9400ee70 comp rx=0 tx=0).stop 2024-04-25T00:09:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 shutdown_connections 2024-04-25T00:09:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcf8c06be30 0x7fcf8c06e2f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa015e6b0 0x7fcfa0150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfa015e2d0 0x7fcfa0150a70 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 --2- 172.21.15.113:0/2539871701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa00b3080 0x7fcfa01429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 >> 172.21.15.113:0/2539871701 conn(0x7fcfa006b0f0 msgr2=0x7fcfa00b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 shutdown_connections 2024-04-25T00:09:17.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:17.892+0000 7fcfa69c1640 1 -- 172.21.15.113:0/2539871701 wait complete. 2024-04-25T00:09:17.903 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:09:18.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.002+0000 7f6d18a38640 1 Processor -- start 2024-04-25T00:09:18.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.002+0000 7f6d18a38640 1 -- start start 2024-04-25T00:09:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.003+0000 7f6d18a38640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6d140b5ca0 0x7f6d140b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.003+0000 7f6d18a38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d140b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.003+0000 7f6d18a38640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d140d2e00 con 0x7f6d1415e510 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.003+0000 7f6d18a38640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6d140b6640 con 0x7f6d140b5ca0 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.003+0000 7f6d18a38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6d140d30e0 con 0x7f6d140b73c0 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d12d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d140b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d11d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6d140b5ca0 0x7f6d140b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d12d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d140b7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:39812/0 (socket says 172.21.15.113:39812) 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d1259c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6d1415e510 0x7f6d140d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33822/0 (socket says 172.21.15.113:33822) 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d12d9d640 1 -- 172.21.15.113:0/1076190470 learned_addr learned my addr 172.21.15.113:0/1076190470 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d11d9b640 1 -- 172.21.15.113:0/1076190470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 msgr2=0x7f6d140b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d11d9b640 1 --2- 172.21.15.113:0/1076190470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d140b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d11d9b640 1 -- 172.21.15.113:0/1076190470 >> v1:172.21.15.178:6789/0 conn(0x7f6d1415e510 legacy=0x7f6d140d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d11d9b640 1 -- 172.21.15.113:0/1076190470 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140d33c0 con 0x7f6d140b5ca0 2024-04-25T00:09:18.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.004+0000 7f6d1159a640 1 -- 172.21.15.113:0/1076190470 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206357020 0 0) 0x7f6d140d2e00 con 0x7f6d1415e510 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.005+0000 7f6d11d9b640 1 --2- 172.21.15.113:0/1076190470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d140b6100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d080097a0 tx=0x7f6d0802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620cc0915e3b89bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.005+0000 7f6d1159a640 1 -- 172.21.15.113:0/1076190470 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08037070 con 0x7f6d140b5ca0 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.005+0000 7f6d1159a640 1 -- 172.21.15.113:0/1076190470 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0802ae50 con 0x7f6d140b5ca0 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.005+0000 7f6d1159a640 1 -- 172.21.15.113:0/1076190470 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08032660 con 0x7f6d140b5ca0 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.005+0000 7f6d18a38640 1 -- 172.21.15.113:0/1076190470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 msgr2=0x7f6d140b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.005+0000 7f6d18a38640 1 --2- 172.21.15.113:0/1076190470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d140b6100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d080097a0 tx=0x7f6d0802a100 comp rx=0 tx=0).stop 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.006+0000 7f6d18a38640 1 -- 172.21.15.113:0/1076190470 shutdown_connections 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.006+0000 7f6d18a38640 1 --2- 172.21.15.113:0/1076190470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d140b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.006+0000 7f6d18a38640 1 --2- 172.21.15.113:0/1076190470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d140b6100 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.006+0000 7f6d18a38640 1 -- 172.21.15.113:0/1076190470 >> 172.21.15.113:0/1076190470 conn(0x7f6d1406b720 msgr2=0x7f6d140d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.006+0000 7f6d18a38640 1 -- 172.21.15.113:0/1076190470 shutdown_connections 2024-04-25T00:09:18.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.006+0000 7f6d18a38640 1 -- 172.21.15.113:0/1076190470 wait complete. 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.007+0000 7f6d18a38640 1 Processor -- start 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.007+0000 7f6d18a38640 1 -- start start 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d18a38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d1414ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d18a38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d141f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d18a38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d1415e510 0x7f6d140c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d18a38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d3ea0 con 0x7f6d1415e510 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d18a38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d3980 con 0x7f6d140b5ca0 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d18a38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6d140d2a90 con 0x7f6d140b73c0 2024-04-25T00:09:18.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d1415e510 0x7f6d140c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d1415e510 0x7f6d140c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34884/0 (socket says 172.21.15.113:34884) 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 -- 172.21.15.113:0/679385655 learned_addr learned my addr 172.21.15.113:0/679385655 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d1259c640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d1414ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d11d9b640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d141f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 -- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 msgr2=0x7f6d141f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d141f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 -- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 msgr2=0x7f6d1414ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d1414ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 -- 172.21.15.113:0/679385655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1414cfa0 con 0x7f6d1415e510 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d1259c640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d1414ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d12d9d640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d1415e510 0x7f6d140c9000 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00e870 tx=0x7f6d0c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d02ffd640 1 -- 172.21.15.113:0/679385655 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c017070 con 0x7f6d1415e510 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d02ffd640 1 -- 172.21.15.113:0/679385655 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c00c940 con 0x7f6d1415e510 2024-04-25T00:09:18.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d141f7c40 con 0x7f6d1415e510 2024-04-25T00:09:18.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.008+0000 7f6d02ffd640 1 -- 172.21.15.113:0/679385655 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00cd30 con 0x7f6d1415e510 2024-04-25T00:09:18.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.009+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d141f8200 con 0x7f6d1415e510 2024-04-25T00:09:18.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.009+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d140691a0 con 0x7f6d1415e510 2024-04-25T00:09:18.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.010+0000 7f6d02ffd640 1 -- 172.21.15.113:0/679385655 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d0c01a070 con 0x7f6d1415e510 2024-04-25T00:09:18.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.010+0000 7f6d02ffd640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6cf406be30 0x7f6cf406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.010+0000 7f6d02ffd640 1 -- 172.21.15.113:0/679385655 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f6d0c08ad30 con 0x7f6d1415e510 2024-04-25T00:09:18.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.010+0000 7f6d1259c640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6cf406be30 0x7f6cf406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.011+0000 7f6d1259c640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6cf406be30 0x7f6cf406e2f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6d04007fd0 tx=0x7f6d04005b70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:18.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.014+0000 7f6d02ffd640 1 -- 172.21.15.113:0/679385655 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d0c059120 con 0x7f6d1415e510 2024-04-25T00:09:18.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.187+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d140d23f0 con 0x7f6d1415e510 2024-04-25T00:09:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.187+0000 7f6d02ffd640 1 -- 172.21.15.113:0/679385655 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) ==== 100+0+11 (secure 0 0 0) 0x7f6d0c05d420 con 0x7f6d1415e510 2024-04-25T00:09:18.188 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6cf406be30 msgr2=0x7f6cf406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6cf406be30 0x7f6cf406e2f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6d04007fd0 tx=0x7f6d04005b70 comp rx=0 tx=0).stop 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d1415e510 msgr2=0x7f6d140c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d1415e510 0x7f6d140c9000 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00e870 tx=0x7f6d0c00ed30 comp rx=0 tx=0).stop 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 shutdown_connections 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6cf406be30 0x7f6cf406e2f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d1415e510 0x7f6d140c9000 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d140b73c0 0x7f6d141f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 --2- 172.21.15.113:0/679385655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d140b5ca0 0x7f6d1414ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 >> 172.21.15.113:0/679385655 conn(0x7f6d1406b720 msgr2=0x7f6d1415b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:18.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 shutdown_connections 2024-04-25T00:09:18.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.189+0000 7f6d18a38640 1 -- 172.21.15.113:0/679385655 wait complete. 2024-04-25T00:09:18.201 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:09:18.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.296+0000 7fdb227f9640 1 Processor -- start 2024-04-25T00:09:18.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.296+0000 7fdb227f9640 1 -- start start 2024-04-25T00:09:18.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.297+0000 7fdb227f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 0x7fdb1c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.297+0000 7fdb227f9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1c0d2e00 con 0x7fdb1c15e510 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.297+0000 7fdb227f9640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1c0d30e0 con 0x7fdb1c0b73c0 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.297+0000 7fdb227f9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c0b6640 con 0x7fdb1c0b5ca0 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.297+0000 7fdb1bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdb1c15e510 0x7fdb1c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36676/0 (socket says 172.21.15.113:36676) 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.297+0000 7fdb1bfff640 1 -- 172.21.15.113:0/210217968 learned_addr learned my addr 172.21.15.113:0/210217968 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1b7fe640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 0x7fdb1c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822573736 0 0) 0x7fdb1c0d2e00 con 0x7fdb1c15e510 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb04003520 con 0x7fdb1c15e510 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 194421078 0 0) 0x7fdb04003520 con 0x7fdb1c15e510 2024-04-25T00:09:18.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 msgr2=0x7fdb1c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 0x7fdb1c0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 >> v1:172.21.15.178:6789/0 conn(0x7fdb1c0b73c0 legacy=0x7fdb1c0b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c0d33c0 con 0x7fdb1c15e510 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fdb0c001670 con 0x7fdb1c15e510 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 >> v1:172.21.15.113:6789/0 conn(0x7fdb1c15e510 legacy=0x7fdb1c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 0x7fdb040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb04003f60 0x7fdb04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb04004d40 0x7fdb040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400bb50 con 0x7fdb04004d40 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400bcd0 con 0x7fdb04003f60 2024-04-25T00:09:18.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0400bed0 con 0x7fdb1c0b5ca0 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb20d96640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb04004d40 0x7fdb040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1bfff640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 0x7fdb040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1b7fe640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb04003f60 0x7fdb04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb20d96640 1 -- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 msgr2=0x7fdb040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb20d96640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0b5ca0 0x7fdb040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb20d96640 1 -- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb04003f60 msgr2=0x7fdb04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb20d96640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb04003f60 0x7fdb04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb20d96640 1 -- 172.21.15.113:0/210217968 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb04003520 con 0x7fdb04004d40 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.298+0000 7fdb1b7fe640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb04003f60 0x7fdb04004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.299+0000 7fdb20d96640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb04004d40 0x7fdb040051c0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fdb14001e50 tx=0x7fdb04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de420eee0cd38fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.299+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14001a70 con 0x7fdb04004d40 2024-04-25T00:09:18.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.299+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb14003970 con 0x7fdb04004d40 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.299+0000 7fdb1affd640 1 -- 172.21.15.113:0/210217968 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14003d60 con 0x7fdb04004d40 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.300+0000 7fdb227f9640 1 -- 172.21.15.113:0/210217968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb04004d40 msgr2=0x7fdb040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.300+0000 7fdb227f9640 1 --2- 172.21.15.113:0/210217968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb04004d40 0x7fdb040051c0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fdb14001e50 tx=0x7fdb04005830 comp rx=0 tx=0).stop 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.300+0000 7fdb20d96640 1 -- 172.21.15.113:0/210217968 reap_dead start 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.300+0000 7fdb227f9640 1 -- 172.21.15.113:0/210217968 shutdown_connections 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.300+0000 7fdb227f9640 1 -- 172.21.15.113:0/210217968 >> 172.21.15.113:0/210217968 conn(0x7fdb1c06b720 msgr2=0x7fdb1c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.300+0000 7fdb227f9640 1 -- 172.21.15.113:0/210217968 shutdown_connections 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.300+0000 7fdb227f9640 1 -- 172.21.15.113:0/210217968 wait complete. 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.301+0000 7fdb227f9640 1 Processor -- start 2024-04-25T00:09:18.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.301+0000 7fdb227f9640 1 -- start start 2024-04-25T00:09:18.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb1c0c6e00 0x7fdb1c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c0cd930 0x7fdb1c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0c8cc0 0x7fdb1c1f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0d3ea0 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0d3980 con 0x7fdb1c0cd930 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c0d2a90 con 0x7fdb1c0c8cc0 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb1c0c6e00 0x7fdb1c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb1c0c6e00 0x7fdb1c0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34900/0 (socket says 172.21.15.113:34900) 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 -- 172.21.15.113:0/1998479522 learned_addr learned my addr 172.21.15.113:0/1998479522 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb20d96640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0c8cc0 0x7fdb1c1f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1b7fe640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c0cd930 0x7fdb1c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 -- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0c8cc0 msgr2=0x7fdb1c1f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0c8cc0 0x7fdb1c1f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 -- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c0cd930 msgr2=0x7fdb1c0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c0cd930 0x7fdb1c0c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 -- 172.21.15.113:0/1998479522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c0c9540 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb20d96640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0c8cc0 0x7fdb1c1f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1b7fe640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c0cd930 0x7fdb1c0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb1bfff640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb1c0c6e00 0x7fdb1c0cd3f0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c007290 tx=0x7fdb0c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb18ff9640 1 -- 172.21.15.113:0/1998479522 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00f960 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb18ff9640 1 -- 172.21.15.113:0/1998479522 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb0c018070 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb18ff9640 1 -- 172.21.15.113:0/1998479522 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00fd60 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c0c9820 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.302+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c0cb5d0 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.303+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c0691a0 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.304+0000 7fdb18ff9640 1 -- 172.21.15.113:0/1998479522 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb0c01d460 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.304+0000 7fdb18ff9640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb0806be60 0x7fdb0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:18.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.304+0000 7fdb1b7fe640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb0806be60 0x7fdb0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:18.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.304+0000 7fdb18ff9640 1 -- 172.21.15.113:0/1998479522 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fdb0c08af90 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.305+0000 7fdb1b7fe640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb0806be60 0x7fdb0806e320 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c149590 tx=0x7fdb10008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:18.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.308+0000 7fdb18ff9640 1 -- 172.21.15.113:0/1998479522 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdb0c059940 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.492+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb1c15b750 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.492+0000 7fdb18ff9640 1 -- 172.21.15.113:0/1998479522 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v170) ==== 80+0+0 (secure 0 0 0) 0x7fdb0c059240 con 0x7fdb1c0c6e00 2024-04-25T00:09:18.492 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:09:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb0806be60 msgr2=0x7fdb0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb0806be60 0x7fdb0806e320 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c149590 tx=0x7fdb10008040 comp rx=0 tx=0).stop 2024-04-25T00:09:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb1c0c6e00 msgr2=0x7fdb1c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb1c0c6e00 0x7fdb1c0cd3f0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c007290 tx=0x7fdb0c007750 comp rx=0 tx=0).stop 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 shutdown_connections 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fdb0806be60 0x7fdb0806e320 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c0cd930 0x7fdb1c0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb1c0c8cc0 0x7fdb1c1f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 --2- 172.21.15.113:0/1998479522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb1c0c6e00 0x7fdb1c0cd3f0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 >> 172.21.15.113:0/1998479522 conn(0x7fdb1c06b720 msgr2=0x7fdb1c0b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 shutdown_connections 2024-04-25T00:09:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:18.494+0000 7fdb227f9640 1 -- 172.21.15.113:0/1998479522 wait complete. 2024-04-25T00:09:18.505 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:09:18.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:18.505 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:19.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:09:19.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:19.252 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:09:19.268 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:09:19.268 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:09:19.268 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:19.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:09:19.284 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:09:19.285 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:19.285 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:19.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:09:19.323 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:09:19.323 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:09:19.323 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:19.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:09:19.340 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.459+0000 7f65ac714640 1 Processor -- start 2024-04-25T00:09:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.459+0000 7f65ac714640 1 -- start start 2024-04-25T00:09:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.460+0000 7f65ac714640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65a40b3fd0 0x7f65a406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.460+0000 7f65ac714640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a406e510 0x7f65a40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.460+0000 7f65ac714640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a4144cf0 con 0x7f65a4144910 2024-04-25T00:09:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.460+0000 7f65ac714640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f65a40b2f30 con 0x7f65a40b3fd0 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.460+0000 7f65ac714640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65a40b3210 con 0x7f65a406e510 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.460+0000 7f65a9caf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65a40b3fd0 0x7f65a406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65a9caf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65a40b3fd0 0x7f65a406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34916/0 (socket says 172.21.15.113:34916) 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65a9caf640 1 -- 172.21.15.113:0/1587476475 learned_addr learned my addr 172.21.15.113:0/1587476475 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65aacb1640 1 --2- 172.21.15.113:0/1587476475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a406e510 0x7f65a40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65a9caf640 1 -- 172.21.15.113:0/1587476475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a406e510 msgr2=0x7f65a40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65a9caf640 1 --2- 172.21.15.113:0/1587476475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a406e510 0x7f65a40cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65a9caf640 1 -- 172.21.15.113:0/1587476475 >> v1:172.21.15.178:6789/0 conn(0x7f65a4144910 legacy=0x7f65a40b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65a9caf640 1 -- 172.21.15.113:0/1587476475 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a40b34f0 con 0x7f65a40b3fd0 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.461+0000 7f65a9caf640 1 --2- 172.21.15.113:0/1587476475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a406dfd0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f65940098a0 tx=0x7f659402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f15261caebb687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65a94ae640 1 -- 172.21.15.113:0/1587476475 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037070 con 0x7f65a40b3fd0 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65a94ae640 1 -- 172.21.15.113:0/1587476475 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594032440 con 0x7f65a40b3fd0 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65a94ae640 1 -- 172.21.15.113:0/1587476475 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594032810 con 0x7f65a40b3fd0 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 -- 172.21.15.113:0/1587476475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 msgr2=0x7f65a406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 --2- 172.21.15.113:0/1587476475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a406dfd0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f65940098a0 tx=0x7f659402a250 comp rx=0 tx=0).stop 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 -- 172.21.15.113:0/1587476475 shutdown_connections 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 --2- 172.21.15.113:0/1587476475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a406e510 0x7f65a40cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 --2- 172.21.15.113:0/1587476475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a406dfd0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 -- 172.21.15.113:0/1587476475 >> 172.21.15.113:0/1587476475 conn(0x7f65a40b1a50 msgr2=0x7f65a40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 -- 172.21.15.113:0/1587476475 shutdown_connections 2024-04-25T00:09:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.462+0000 7f65ac714640 1 -- 172.21.15.113:0/1587476475 wait complete. 2024-04-25T00:09:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.463+0000 7f65ac714640 1 Processor -- start 2024-04-25T00:09:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.463+0000 7f65ac714640 1 -- start start 2024-04-25T00:09:19.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65ac714640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a406e510 0x7f65a4155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65ac714640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65ac714640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a4144910 0x7f65a4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65ac714640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40b4da0 con 0x7f65a40b3fd0 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65ac714640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40b4440 con 0x7f65a406e510 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65ac714640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65a40b4f20 con 0x7f65a4144910 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a415c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34924/0 (socket says 172.21.15.113:34924) 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 -- 172.21.15.113:0/4245626811 learned_addr learned my addr 172.21.15.113:0/4245626811 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65aacb1640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a4144910 0x7f65a4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65aa4b0640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a406e510 0x7f65a4155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 -- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a4144910 msgr2=0x7f65a4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a4144910 0x7f65a4159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 -- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a406e510 msgr2=0x7f65a4155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a406e510 0x7f65a4155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65a9caf640 1 -- 172.21.15.113:0/4245626811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a413f450 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.464+0000 7f65aacb1640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a4144910 0x7f65a4159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.465+0000 7f65a9caf640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a415c260 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f6594000c00 tx=0x7f65940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.465+0000 7f659affd640 1 -- 172.21.15.113:0/4245626811 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037040 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.465+0000 7f659affd640 1 -- 172.21.15.113:0/4245626811 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594004230 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.465+0000 7f659affd640 1 -- 172.21.15.113:0/4245626811 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594003610 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.465+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a413f730 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.465+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a413ffd0 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.466+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a413fb20 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.466+0000 7f659affd640 1 -- 172.21.15.113:0/4245626811 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65940037b0 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.467+0000 7f659affd640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f658806be60 0x7f658806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.467+0000 7f659affd640 1 -- 172.21.15.113:0/4245626811 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f65940abc60 con 0x7f65a40b3fd0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.467+0000 7f65aa4b0640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f658806be60 0x7f658806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.467+0000 7f65aa4b0640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f658806be60 0x7f658806e320 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f659c005660 tx=0x7f659c003d90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.471+0000 7f659affd640 1 -- 172.21.15.113:0/4245626811 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f659407a6a0 con 0x7f65a40b3fd0 2024-04-25T00:09:19.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.679+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65a40691a0 con 0x7f65a40b3fd0 2024-04-25T00:09:19.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.679+0000 7f659affd640 1 -- 172.21.15.113:0/4245626811 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5195 (secure 0 0 0) 0x7f6594079fa0 con 0x7f65a40b3fd0 2024-04-25T00:09:19.680 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:19.680 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/992895335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":992895335},{"type":"v1","addr":"172.21.15.178:6834","nonce":992895335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/17230728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":17230728},{"type":"v1","addr":"172.21.15.113:6836","nonce":17230728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4170863917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4170863917},{"type":"v1","addr":"172.21.15.113:6837","nonce":4170863917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:08.629096+0000","modified":"2024-04-25T00:09:11.220151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10434":{"gid":10434,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/2813561857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2813561857},{"type":"v1","addr":"172.21.15.178:6835","nonce":2813561857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10434,"qdb_cluster":[10434]},"id":12}]} 2024-04-25T00:09:19.680 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-04-25T00:09:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f658806be60 msgr2=0x7f658806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f658806be60 0x7f658806e320 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f659c005660 tx=0x7f659c003d90 comp rx=0 tx=0).stop 2024-04-25T00:09:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 msgr2=0x7f65a415c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a415c260 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f6594000c00 tx=0x7f65940046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 shutdown_connections 2024-04-25T00:09:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f658806be60 0x7f658806e320 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a4144910 0x7f65a4159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a40b3fd0 0x7f65a415c260 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 --2- 172.21.15.113:0/4245626811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a406e510 0x7f65a4155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 >> 172.21.15.113:0/4245626811 conn(0x7f65a40b1a50 msgr2=0x7f65a4072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 shutdown_connections 2024-04-25T00:09:19.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.681+0000 7f65ac714640 1 -- 172.21.15.113:0/4245626811 wait complete. 2024-04-25T00:09:19.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:19.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.788+0000 7f0b8274f640 1 Processor -- start 2024-04-25T00:09:19.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.788+0000 7f0b8274f640 1 -- start start 2024-04-25T00:09:19.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b8274f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0b7c0d2290 0x7f0b7c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b8274f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b7c0d0b00 0x7f0b7c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b8274f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7c0d1480 con 0x7f0b7c15e320 2024-04-25T00:09:19.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b8274f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0d3410 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b8274f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0d2800 con 0x7f0b7c0d2290 2024-04-25T00:09:19.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b80cec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b7c0d0b00 0x7f0b7c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b80cec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b7c0d0b00 0x7f0b7c0d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34926/0 (socket says 172.21.15.113:34926) 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.789+0000 7f0b80cec640 1 -- 172.21.15.113:0/2863064184 learned_addr learned my addr 172.21.15.113:0/2863064184 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b7b7fe640 1 --2- 172.21.15.113:0/2863064184 >> v2:172.21.15.178:3300/0 conn(0x7f0b7c0d2290 0x7f0b7c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b80cec640 1 -- 172.21.15.113:0/2863064184 >> v2:172.21.15.178:3300/0 conn(0x7f0b7c0d2290 msgr2=0x7f0b7c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b80cec640 1 --2- 172.21.15.113:0/2863064184 >> v2:172.21.15.178:3300/0 conn(0x7f0b7c0d2290 0x7f0b7c0b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b80cec640 1 -- 172.21.15.113:0/2863064184 >> v1:172.21.15.178:6789/0 conn(0x7f0b7c15e320 legacy=0x7f0b7c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b80cec640 1 -- 172.21.15.113:0/2863064184 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0d2ae0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b7b7fe640 1 --2- 172.21.15.113:0/2863064184 >> v2:172.21.15.178:3300/0 conn(0x7f0b7c0d2290 0x7f0b7c0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b80cec640 1 --2- 172.21.15.113:0/2863064184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c0d0f40 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c072810 tx=0x7f0b7402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eeb63f3e46fb42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b7affd640 1 -- 172.21.15.113:0/2863064184 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74036070 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.790+0000 7f0b7affd640 1 -- 172.21.15.113:0/2863064184 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b74031600 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b7affd640 1 -- 172.21.15.113:0/2863064184 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b740319f0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 -- 172.21.15.113:0/2863064184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 msgr2=0x7f0b7c0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 --2- 172.21.15.113:0/2863064184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c0d0f40 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c072810 tx=0x7f0b7402a7c0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 -- 172.21.15.113:0/2863064184 shutdown_connections 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 --2- 172.21.15.113:0/2863064184 >> v2:172.21.15.178:3300/0 conn(0x7f0b7c0d2290 0x7f0b7c0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 --2- 172.21.15.113:0/2863064184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c0d0f40 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 -- 172.21.15.113:0/2863064184 >> 172.21.15.113:0/2863064184 conn(0x7f0b7c0b5c20 msgr2=0x7f0b7c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 -- 172.21.15.113:0/2863064184 shutdown_connections 2024-04-25T00:09:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.791+0000 7f0b8274f640 1 -- 172.21.15.113:0/2863064184 wait complete. 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.792+0000 7f0b8274f640 1 Processor -- start 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.792+0000 7f0b8274f640 1 -- start start 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b8274f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b8274f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7c0d2290 0x7f0b7c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b8274f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7c15e320 0x7f0b7c15ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b8274f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0b9fe0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b8274f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d30a0 con 0x7f0b7c15e320 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b8274f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c15e700 con 0x7f0b7c0d2290 2024-04-25T00:09:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c14e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34934/0 (socket says 172.21.15.113:34934) 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b7bfff640 1 -- 172.21.15.113:0/4186656663 learned_addr learned my addr 172.21.15.113:0/4186656663 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b80cec640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7c15e320 0x7f0b7c15ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.793+0000 7f0b7b7fe640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7c0d2290 0x7f0b7c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b7bfff640 1 -- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7c0d2290 msgr2=0x7f0b7c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b7bfff640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7c0d2290 0x7f0b7c1608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b7bfff640 1 -- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7c15e320 msgr2=0x7f0b7c15ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b7bfff640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7c15e320 0x7f0b7c15ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b7bfff640 1 -- 172.21.15.113:0/4186656663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0cdb80 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b7bfff640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c14e650 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c400 tx=0x7f0b6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b78ff9640 1 -- 172.21.15.113:0/4186656663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0052d0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b78ff9640 1 -- 172.21.15.113:0/4186656663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6c005490 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c0cde60 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b78ff9640 1 -- 172.21.15.113:0/4186656663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c007260 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.794+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c0cb4c0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.795+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c0691a0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.795+0000 7f0b78ff9640 1 -- 172.21.15.113:0/4186656663 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b6c007400 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.796+0000 7f0b78ff9640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b6806be30 0x7f0b6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.796+0000 7f0b78ff9640 1 -- 172.21.15.113:0/4186656663 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f0b6c08a2b0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.796+0000 7f0b7b7fe640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b6806be30 0x7f0b6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.796+0000 7f0b7b7fe640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b6806be30 0x7f0b6806e2f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c152090 tx=0x7f0b70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.800+0000 7f0b78ff9640 1 -- 172.21.15.113:0/4186656663 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b6c0586a0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.974+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b7c0caa10 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.975+0000 7f0b78ff9640 1 -- 172.21.15.113:0/4186656663 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16727 (secure 0 0 0) 0x7f0b6c05c9a0 con 0x7f0b7c0d0b00 2024-04-25T00:09:19.975 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:19.975 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:10.300297+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:06.844691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:07.603153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b6806be30 msgr2=0x7f0b6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b6806be30 0x7f0b6806e2f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c152090 tx=0x7f0b70008040 comp rx=0 tx=0).stop 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 msgr2=0x7f0b7c14e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c14e650 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c400 tx=0x7f0b6c00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 shutdown_connections 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0b6806be30 0x7f0b6806e2f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7c15e320 0x7f0b7c15ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7c0d2290 0x7f0b7c1608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 --2- 172.21.15.113:0/4186656663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b7c0d0b00 0x7f0b7c14e650 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 >> 172.21.15.113:0/4186656663 conn(0x7f0b7c0b5c20 msgr2=0x7f0b7c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 shutdown_connections 2024-04-25T00:09:19.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:19.977+0000 7f0b8274f640 1 -- 172.21.15.113:0/4186656663 wait complete. 2024-04-25T00:09:19.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:09:06.844691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:09:07.603153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:09:19.989 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:09:20.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.087+0000 7f518d03f640 1 Processor -- start 2024-04-25T00:09:20.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.088+0000 7f518d03f640 1 -- start start 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.088+0000 7f518d03f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f51880b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.088+0000 7f518d03f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51880d2e00 con 0x7f518815e510 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.088+0000 7f518d03f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51880d30e0 con 0x7f51880b73c0 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.088+0000 7f518d03f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51880b6640 con 0x7f51880b5ca0 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.088+0000 7f5186d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f518815e510 0x7f51880d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36692/0 (socket says 172.21.15.113:36692) 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.088+0000 7f5186d9d640 1 -- 172.21.15.113:0/2696597252 learned_addr learned my addr 172.21.15.113:0/2696597252 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f518659c640 1 --2- 172.21.15.113:0/2696597252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f51880b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1043039537 0 0) 0x7f51880d2e00 con 0x7f518815e510 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5170003520 con 0x7f518815e510 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357634988 0 0) 0x7f51880d30e0 con 0x7f51880b73c0 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51880d2e00 con 0x7f51880b73c0 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2966658592 0 0) 0x7f5170003520 con 0x7f518815e510 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f518659c640 1 -- 172.21.15.113:0/2696597252 >> v1:172.21.15.178:6789/0 conn(0x7f51880b73c0 legacy=0x7f51880b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f518659c640 1 -- 172.21.15.113:0/2696597252 >> v1:172.21.15.113:6789/0 conn(0x7f518815e510 legacy=0x7f51880d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.089+0000 7f518659c640 1 -- 172.21.15.113:0/2696597252 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880d33c0 con 0x7f51880b5ca0 2024-04-25T00:09:20.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518659c640 1 --2- 172.21.15.113:0/2696597252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f51880b6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f51780097d0 tx=0x7f517802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9ae5ab3e4f913b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178037070 con 0x7f51880b5ca0 2024-04-25T00:09:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517802ab90 con 0x7f51880b5ca0 2024-04-25T00:09:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f5185d9b640 1 -- 172.21.15.113:0/2696597252 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c001670 con 0x7f51880b5ca0 2024-04-25T00:09:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518d03f640 1 -- 172.21.15.113:0/2696597252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 msgr2=0x7f51880b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518d03f640 1 --2- 172.21.15.113:0/2696597252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f51880b6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f51780097d0 tx=0x7f517802a100 comp rx=0 tx=0).stop 2024-04-25T00:09:20.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518d03f640 1 -- 172.21.15.113:0/2696597252 shutdown_connections 2024-04-25T00:09:20.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518d03f640 1 --2- 172.21.15.113:0/2696597252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f51880b6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518d03f640 1 -- 172.21.15.113:0/2696597252 >> 172.21.15.113:0/2696597252 conn(0x7f518806b720 msgr2=0x7f51880d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518d03f640 1 -- 172.21.15.113:0/2696597252 shutdown_connections 2024-04-25T00:09:20.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.090+0000 7f518d03f640 1 -- 172.21.15.113:0/2696597252 wait complete. 2024-04-25T00:09:20.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.092+0000 7f518d03f640 1 Processor -- start 2024-04-25T00:09:20.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.092+0000 7f518d03f640 1 -- start start 2024-04-25T00:09:20.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.092+0000 7f518d03f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f518815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.092+0000 7f518d03f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51880b73c0 0x7f51881f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518d03f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f518815e510 0x7f518815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518d03f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51880d3ea0 con 0x7f51880b73c0 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518d03f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51880d3980 con 0x7f518815e510 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518d03f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51880d2a90 con 0x7f51880b5ca0 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51880b73c0 0x7f51881f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51880b73c0 0x7f51881f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34948/0 (socket says 172.21.15.113:34948) 2024-04-25T00:09:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 -- 172.21.15.113:0/2921516959 learned_addr learned my addr 172.21.15.113:0/2921516959 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f5186d9d640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f518815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518759e640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f518815e510 0x7f518815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 -- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 msgr2=0x7f518815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f518815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 -- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f518815e510 msgr2=0x7f518815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f518815e510 0x7f518815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 -- 172.21.15.113:0/2921516959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51881f3fa0 con 0x7f51880b73c0 2024-04-25T00:09:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518759e640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f518815e510 0x7f518815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518659c640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51880b73c0 0x7f51881f3a60 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f51780097d0 tx=0x7f51780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f516f7fe640 1 -- 172.21.15.113:0/2921516959 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178037050 con 0x7f51880b73c0 2024-04-25T00:09:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f516f7fe640 1 -- 172.21.15.113:0/2921516959 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5178043070 con 0x7f51880b73c0 2024-04-25T00:09:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5188160d30 con 0x7f51880b73c0 2024-04-25T00:09:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f516f7fe640 1 -- 172.21.15.113:0/2921516959 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178006e10 con 0x7f51880b73c0 2024-04-25T00:09:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.093+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881612f0 con 0x7f51880b73c0 2024-04-25T00:09:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.094+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880691a0 con 0x7f51880b73c0 2024-04-25T00:09:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.095+0000 7f516f7fe640 1 -- 172.21.15.113:0/2921516959 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f517803ca30 con 0x7f51880b73c0 2024-04-25T00:09:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.095+0000 7f516f7fe640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f517406be30 0x7f517406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.095+0000 7f516f7fe640 1 -- 172.21.15.113:0/2921516959 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f51780ab890 con 0x7f51880b73c0 2024-04-25T00:09:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.095+0000 7f5186d9d640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f517406be30 0x7f517406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.096+0000 7f5186d9d640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f517406be30 0x7f517406e2f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f517c0011e0 tx=0x7f517c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.100+0000 7f516f7fe640 1 -- 172.21.15.113:0/2921516959 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f517807a3f0 con 0x7f51880b73c0 2024-04-25T00:09:20.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.276+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51881556b0 con 0x7f51880b73c0 2024-04-25T00:09:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.276+0000 7f516f7fe640 1 -- 172.21.15.113:0/2921516959 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) ==== 93+0+10 (secure 0 0 0) 0x7f5178079cf0 con 0x7f51880b73c0 2024-04-25T00:09:20.277 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f517406be30 msgr2=0x7f517406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f517406be30 0x7f517406e2f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f517c0011e0 tx=0x7f517c009040 comp rx=0 tx=0).stop 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51880b73c0 msgr2=0x7f51881f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51880b73c0 0x7f51881f3a60 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f51780097d0 tx=0x7f51780046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 shutdown_connections 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f517406be30 0x7f517406e2f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f518815e510 0x7f518815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51880b73c0 0x7f51881f3a60 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 --2- 172.21.15.113:0/2921516959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51880b5ca0 0x7f518815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 >> 172.21.15.113:0/2921516959 conn(0x7f518806b720 msgr2=0x7f518815c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 shutdown_connections 2024-04-25T00:09:20.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.278+0000 7f518d03f640 1 -- 172.21.15.113:0/2921516959 wait complete. 2024-04-25T00:09:20.289 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:09:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.389+0000 7f73b5010640 1 Processor -- start 2024-04-25T00:09:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.389+0000 7f73b5010640 1 -- start start 2024-04-25T00:09:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73b5010640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 0x7f73b00b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73b5010640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b00d2e00 con 0x7f73b015e510 2024-04-25T00:09:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73b5010640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b00d30e0 con 0x7f73b00b73c0 2024-04-25T00:09:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73b5010640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73b00b6640 con 0x7f73b00b5ca0 2024-04-25T00:09:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73aed9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f73b015e510 0x7f73b00d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36704/0 (socket says 172.21.15.113:36704) 2024-04-25T00:09:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73aed9d640 1 -- 172.21.15.113:0/1809550620 learned_addr learned my addr 172.21.15.113:0/1809550620 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1485939393 0 0) 0x7f73b00d2e00 con 0x7f73b015e510 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f739c003520 con 0x7f73b015e510 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.390+0000 7f73ae59c640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 0x7f73b00b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3336168616 0 0) 0x7f739c003520 con 0x7f73b015e510 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 msgr2=0x7f73b00b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 0x7f73b00b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 >> v1:172.21.15.178:6789/0 conn(0x7f73b00b73c0 legacy=0x7f73b00b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b00d33c0 con 0x7f73b015e510 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f73a8001670 con 0x7f73b015e510 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 >> v1:172.21.15.113:6789/0 conn(0x7f73b015e510 legacy=0x7f73b00d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 0x7f739c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004010 0x7f739c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f739c004d40 0x7f739c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f739c00bbd0 con 0x7f739c004d40 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f739c00bd80 con 0x7f739c004010 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f739c00bf80 con 0x7f73b00b5ca0 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73af59e640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f739c004d40 0x7f739c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73aed9d640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 0x7f739c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73ae59c640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004010 0x7f739c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73af59e640 1 -- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 msgr2=0x7f739c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73af59e640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b00b5ca0 0x7f739c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73af59e640 1 -- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004010 msgr2=0x7f739c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73af59e640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004010 0x7f739c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.391+0000 7f73af59e640 1 -- 172.21.15.113:0/1809550620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739c003520 con 0x7f739c004d40 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.392+0000 7f73af59e640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f739c004d40 0x7f739c0051c0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f73a0001e50 tx=0x7f739c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445124c34545b60a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.392+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0001a70 con 0x7f739c004d40 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.392+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0003970 con 0x7f739c004d40 2024-04-25T00:09:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.392+0000 7f73add9b640 1 -- 172.21.15.113:0/1809550620 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0003d60 con 0x7f739c004d40 2024-04-25T00:09:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.392+0000 7f73b5010640 1 -- 172.21.15.113:0/1809550620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f739c004d40 msgr2=0x7f739c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.392+0000 7f73b5010640 1 --2- 172.21.15.113:0/1809550620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f739c004d40 0x7f739c0051c0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f73a0001e50 tx=0x7f739c005830 comp rx=0 tx=0).stop 2024-04-25T00:09:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.392+0000 7f73af59e640 1 -- 172.21.15.113:0/1809550620 reap_dead start 2024-04-25T00:09:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.393+0000 7f73b5010640 1 -- 172.21.15.113:0/1809550620 shutdown_connections 2024-04-25T00:09:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.393+0000 7f73b5010640 1 -- 172.21.15.113:0/1809550620 >> 172.21.15.113:0/1809550620 conn(0x7f73b006b720 msgr2=0x7f73b00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.393+0000 7f73b5010640 1 -- 172.21.15.113:0/1809550620 shutdown_connections 2024-04-25T00:09:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.393+0000 7f73b5010640 1 -- 172.21.15.113:0/1809550620 wait complete. 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.394+0000 7f73b5010640 1 Processor -- start 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.394+0000 7f73b5010640 1 -- start start 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73b5010640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004d40 0x7f73b00ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73b5010640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b00c8cb0 0x7f73b00cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73b5010640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b014fd50 0x7f73b0153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73b5010640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00d3ea0 con 0x7f73b00c8cb0 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73b5010640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00d3980 con 0x7f739c004d40 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73b5010640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73b00d2a90 con 0x7f73b014fd50 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b00c8cb0 0x7f73b00cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b00c8cb0 0x7f73b00cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34970/0 (socket says 172.21.15.113:34970) 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 -- 172.21.15.113:0/1020689372 learned_addr learned my addr 172.21.15.113:0/1020689372 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73aed9d640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004d40 0x7f73b00ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73af59e640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b014fd50 0x7f73b0153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 -- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b014fd50 msgr2=0x7f73b0153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b014fd50 0x7f73b0153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 -- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004d40 msgr2=0x7f73b00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004d40 0x7f73b00ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 -- 172.21.15.113:0/1020689372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b00cbc60 con 0x7f73b00c8cb0 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73af59e640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b014fd50 0x7f73b0153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73aed9d640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004d40 0x7f73b00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73ae59c640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b00c8cb0 0x7f73b00cb2f0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f739800b850 tx=0x7f739800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73a77fe640 1 -- 172.21.15.113:0/1020689372 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398018070 con 0x7f73b00c8cb0 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73a77fe640 1 -- 172.21.15.113:0/1020689372 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f739800be40 con 0x7f73b00c8cb0 2024-04-25T00:09:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73a77fe640 1 -- 172.21.15.113:0/1020689372 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398010c60 con 0x7f73b00c8cb0 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.395+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b00cbf40 con 0x7f73b00c8cb0 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.396+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b00cc500 con 0x7f73b00c8cb0 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.396+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b00691a0 con 0x7f73b00c8cb0 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.397+0000 7f73a77fe640 1 -- 172.21.15.113:0/1020689372 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f739801b070 con 0x7f73b00c8cb0 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.397+0000 7f73a77fe640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f739006be60 0x7f739006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.397+0000 7f73a77fe640 1 -- 172.21.15.113:0/1020689372 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f739808b290 con 0x7f73b00c8cb0 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.397+0000 7f73aed9d640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f739006be60 0x7f739006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.398+0000 7f73aed9d640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f739006be60 0x7f739006e320 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f73a8001050 tx=0x7f73a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.402+0000 7f73a77fe640 1 -- 172.21.15.113:0/1020689372 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7398059c40 con 0x7f73b00c8cb0 2024-04-25T00:09:20.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.574+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f73b00c7930 con 0x7f73b00c8cb0 2024-04-25T00:09:20.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.574+0000 7f73a77fe640 1 -- 172.21.15.113:0/1020689372 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) ==== 104+0+11 (secure 0 0 0) 0x7f7398059540 con 0x7f73b00c8cb0 2024-04-25T00:09:20.575 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f739006be60 msgr2=0x7f739006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f739006be60 0x7f739006e320 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f73a8001050 tx=0x7f73a8009040 comp rx=0 tx=0).stop 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b00c8cb0 msgr2=0x7f73b00cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b00c8cb0 0x7f73b00cb2f0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f739800b850 tx=0x7f739800bd10 comp rx=0 tx=0).stop 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 shutdown_connections 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f739006be60 0x7f739006e320 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73b014fd50 0x7f73b0153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b00c8cb0 0x7f73b00cb2f0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 --2- 172.21.15.113:0/1020689372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f739c004d40 0x7f73b00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 >> 172.21.15.113:0/1020689372 conn(0x7f73b006b720 msgr2=0x7f73b00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 shutdown_connections 2024-04-25T00:09:20.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.576+0000 7f73b5010640 1 -- 172.21.15.113:0/1020689372 wait complete. 2024-04-25T00:09:20.588 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:09:20.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.686+0000 7f82b13ce640 1 Processor -- start 2024-04-25T00:09:20.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.686+0000 7f82b13ce640 1 -- start start 2024-04-25T00:09:20.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82b13ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82ac062290 0x7f82ac06c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82b13ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 0x7f82ac1604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82b13ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82ac06b5f0 con 0x7f82ac15d150 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82b13ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f82ac06b890 con 0x7f82ac062290 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82b13ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac06bb70 con 0x7f82ac160050 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82a3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82ac062290 0x7f82ac06c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82aaffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f82ac15d150 0x7f82ac061b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36718/0 (socket says 172.21.15.113:36718) 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82aaffd640 1 -- 172.21.15.113:0/88547465 learned_addr learned my addr 172.21.15.113:0/88547465 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82ab7fe640 1 --2- 172.21.15.113:0/88547465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 0x7f82ac1604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82a3fff640 1 -- 172.21.15.113:0/88547465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 msgr2=0x7f82ac1604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82a3fff640 1 --2- 172.21.15.113:0/88547465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 0x7f82ac1604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82a3fff640 1 -- 172.21.15.113:0/88547465 >> v1:172.21.15.113:6789/0 conn(0x7f82ac15d150 legacy=0x7f82ac061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.687+0000 7f82a3fff640 1 -- 172.21.15.113:0/88547465 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac06be50 con 0x7f82ac062290 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82a3fff640 1 --2- 172.21.15.113:0/88547465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac06c650 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f8294009790 tx=0x7f829402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827978e0dffa23c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82aa7fc640 1 -- 172.21.15.113:0/88547465 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294037070 con 0x7f82ac062290 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82aa7fc640 1 -- 172.21.15.113:0/88547465 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294032440 con 0x7f82ac062290 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82aa7fc640 1 -- 172.21.15.113:0/88547465 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294032810 con 0x7f82ac062290 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82b13ce640 1 -- 172.21.15.113:0/88547465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 msgr2=0x7f82ac06c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82b13ce640 1 --2- 172.21.15.113:0/88547465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac06c650 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f8294009790 tx=0x7f829402a180 comp rx=0 tx=0).stop 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82b13ce640 1 -- 172.21.15.113:0/88547465 shutdown_connections 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82b13ce640 1 --2- 172.21.15.113:0/88547465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 0x7f82ac1604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82b13ce640 1 --2- 172.21.15.113:0/88547465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac06c650 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82b13ce640 1 -- 172.21.15.113:0/88547465 >> 172.21.15.113:0/88547465 conn(0x7f82ac0d0e30 msgr2=0x7f82ac0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.688+0000 7f82b13ce640 1 -- 172.21.15.113:0/88547465 shutdown_connections 2024-04-25T00:09:20.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.689+0000 7f82b13ce640 1 -- 172.21.15.113:0/88547465 wait complete. 2024-04-25T00:09:20.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 Processor -- start 2024-04-25T00:09:20.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 -- start start 2024-04-25T00:09:20.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82ac15d150 0x7f82ac1f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 0x7f82ac1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac160b80 con 0x7f82ac062290 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac160a00 con 0x7f82ac15d150 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82b13ce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac15d530 con 0x7f82ac160050 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82aaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac1459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34994/0 (socket says 172.21.15.113:34994) 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.690+0000 7f82aaffd640 1 -- 172.21.15.113:0/43874227 learned_addr learned my addr 172.21.15.113:0/43874227 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82aaffd640 1 -- 172.21.15.113:0/43874227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 msgr2=0x7f82ac1584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82a3fff640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82ac15d150 0x7f82ac1f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82aaffd640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 0x7f82ac1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82aaffd640 1 -- 172.21.15.113:0/43874227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82ac15d150 msgr2=0x7f82ac1f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82aaffd640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82ac15d150 0x7f82ac1f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82aaffd640 1 -- 172.21.15.113:0/43874227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1f5860 con 0x7f82ac062290 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82aaffd640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac1459b0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c430 tx=0x7f829c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82a37fe640 1 -- 172.21.15.113:0/43874227 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c017070 con 0x7f82ac062290 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82a37fe640 1 -- 172.21.15.113:0/43874227 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c005ce0 con 0x7f82ac062290 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1f5b40 con 0x7f82ac062290 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82a37fe640 1 -- 172.21.15.113:0/43874227 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0072f0 con 0x7f82ac062290 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.691+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1f6100 con 0x7f82ac062290 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.692+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac0b0790 con 0x7f82ac062290 2024-04-25T00:09:20.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.692+0000 7f82a37fe640 1 -- 172.21.15.113:0/43874227 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f829c01a070 con 0x7f82ac062290 2024-04-25T00:09:20.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.693+0000 7f82a37fe640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f828c06be30 0x7f828c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.693+0000 7f82a37fe640 1 -- 172.21.15.113:0/43874227 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f829c08a260 con 0x7f82ac062290 2024-04-25T00:09:20.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.693+0000 7f82a3fff640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f828c06be30 0x7f828c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.693+0000 7f82a3fff640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f828c06be30 0x7f828c06e2f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8294034000 tx=0x7f829402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.697+0000 7f82a37fe640 1 -- 172.21.15.113:0/43874227 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f829c058c10 con 0x7f82ac062290 2024-04-25T00:09:20.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.868+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82ac0691a0 con 0x7f82ac062290 2024-04-25T00:09:20.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.868+0000 7f82a37fe640 1 -- 172.21.15.113:0/43874227 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) ==== 100+0+11 (secure 0 0 0) 0x7f829c058510 con 0x7f82ac062290 2024-04-25T00:09:20.869 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:09:20.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f828c06be30 msgr2=0x7f828c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f828c06be30 0x7f828c06e2f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8294034000 tx=0x7f829402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 msgr2=0x7f82ac1459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac1459b0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c430 tx=0x7f829c00c8f0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 shutdown_connections 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f828c06be30 0x7f828c06e2f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82ac15d150 0x7f82ac1f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82ac160050 0x7f82ac1584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 --2- 172.21.15.113:0/43874227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ac062290 0x7f82ac1459b0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 >> 172.21.15.113:0/43874227 conn(0x7f82ac0d0e30 msgr2=0x7f82ac0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 shutdown_connections 2024-04-25T00:09:20.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.870+0000 7f82b13ce640 1 -- 172.21.15.113:0/43874227 wait complete. 2024-04-25T00:09:20.881 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:20.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.977+0000 7f9869e88640 1 Processor -- start 2024-04-25T00:09:20.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.977+0000 7f9869e88640 1 -- start start 2024-04-25T00:09:20.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f9869e88640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f98640b3fd0 0x7f986406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f9869e88640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f98640cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f9869e88640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9864144cf0 con 0x7f9864144910 2024-04-25T00:09:20.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f9869e88640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f98640b2f30 con 0x7f98640b3fd0 2024-04-25T00:09:20.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f9869e88640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f98640b3210 con 0x7f986406e510 2024-04-25T00:09:20.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f98637fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9864144910 0x7f98640b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36720/0 (socket says 172.21.15.113:36720) 2024-04-25T00:09:20.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f98637fe640 1 -- 172.21.15.113:0/142638024 learned_addr learned my addr 172.21.15.113:0/142638024 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f9862ffd640 1 --2- 172.21.15.113:0/142638024 >> v2:172.21.15.113:3300/0 conn(0x7f98640b3fd0 0x7f986406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f98627fc640 1 -- 172.21.15.113:0/142638024 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1233554012 0 0) 0x7f9864144cf0 con 0x7f9864144910 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f98627fc640 1 -- 172.21.15.113:0/142638024 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f984c003520 con 0x7f9864144910 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.978+0000 7f9863fff640 1 --2- 172.21.15.113:0/142638024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f98640cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f9862ffd640 1 -- 172.21.15.113:0/142638024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 msgr2=0x7f98640cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f9862ffd640 1 --2- 172.21.15.113:0/142638024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f98640cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f9862ffd640 1 -- 172.21.15.113:0/142638024 >> v1:172.21.15.113:6789/0 conn(0x7f9864144910 legacy=0x7f98640b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f9862ffd640 1 -- 172.21.15.113:0/142638024 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98640b34f0 con 0x7f98640b3fd0 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f9862ffd640 1 --2- 172.21.15.113:0/142638024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640b3fd0 0x7f986406dfd0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f98580097d0 tx=0x7f985802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348a29e0e35ef73b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f98627fc640 1 -- 172.21.15.113:0/142638024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858037070 con 0x7f98640b3fd0 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f98627fc640 1 -- 172.21.15.113:0/142638024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9858032440 con 0x7f98640b3fd0 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.979+0000 7f98627fc640 1 -- 172.21.15.113:0/142638024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858032810 con 0x7f98640b3fd0 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 -- 172.21.15.113:0/142638024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640b3fd0 msgr2=0x7f986406dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 --2- 172.21.15.113:0/142638024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640b3fd0 0x7f986406dfd0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f98580097d0 tx=0x7f985802a1c0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 -- 172.21.15.113:0/142638024 shutdown_connections 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 --2- 172.21.15.113:0/142638024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f98640cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 --2- 172.21.15.113:0/142638024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640b3fd0 0x7f986406dfd0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 -- 172.21.15.113:0/142638024 >> 172.21.15.113:0/142638024 conn(0x7f98640b1a50 msgr2=0x7f98640b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 -- 172.21.15.113:0/142638024 shutdown_connections 2024-04-25T00:09:20.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.980+0000 7f9869e88640 1 -- 172.21.15.113:0/142638024 wait complete. 2024-04-25T00:09:20.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.981+0000 7f9869e88640 1 Processor -- start 2024-04-25T00:09:20.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.981+0000 7f9869e88640 1 -- start start 2024-04-25T00:09:20.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9869e88640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f986413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9869e88640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98640b3fd0 0x7f98641e4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9869e88640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9864144910 0x7f98641e4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9869e88640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98640b4da0 con 0x7f9864144910 2024-04-25T00:09:20.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9869e88640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98640b4440 con 0x7f98640b3fd0 2024-04-25T00:09:20.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9869e88640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f98640b4f20 con 0x7f986406e510 2024-04-25T00:09:20.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9864144910 0x7f98641e4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9864144910 0x7f98641e4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35020/0 (socket says 172.21.15.113:35020) 2024-04-25T00:09:20.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 -- 172.21.15.113:0/3276479502 learned_addr learned my addr 172.21.15.113:0/3276479502 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 -- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 msgr2=0x7f986413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f98637fe640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f986413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9862ffd640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98640b3fd0 0x7f98641e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f986413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 -- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98640b3fd0 msgr2=0x7f98641e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98640b3fd0 0x7f98641e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9863fff640 1 -- 172.21.15.113:0/3276479502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986415df90 con 0x7f9864144910 2024-04-25T00:09:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f98637fe640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f986413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.982+0000 7f9862ffd640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98640b3fd0 0x7f98641e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.983+0000 7f9863fff640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9864144910 0x7f98641e4eb0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f985c007cf0 tx=0x7f985c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.983+0000 7f984bfff640 1 -- 172.21.15.113:0/3276479502 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c014420 con 0x7f9864144910 2024-04-25T00:09:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.983+0000 7f984bfff640 1 -- 172.21.15.113:0/3276479502 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985c019070 con 0x7f9864144910 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.983+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98641e54e0 con 0x7f9864144910 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.983+0000 7f984bfff640 1 -- 172.21.15.113:0/3276479502 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c014820 con 0x7f9864144910 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.983+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641e5aa0 con 0x7f9864144910 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.984+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98640cce00 con 0x7f9864144910 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.984+0000 7f984bfff640 1 -- 172.21.15.113:0/3276479502 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f985c0149c0 con 0x7f9864144910 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.984+0000 7f984bfff640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f985006be30 0x7f985006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.985+0000 7f98637fe640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f985006be30 0x7f985006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.985+0000 7f984bfff640 1 -- 172.21.15.113:0/3276479502 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f985c08b560 con 0x7f9864144910 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.985+0000 7f98637fe640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f985006be30 0x7f985006e2f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9854005880 tx=0x7f9854009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:20.989+0000 7f984bfff640 1 -- 172.21.15.113:0/3276479502 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f98640cce00 con 0x7f9864144910 2024-04-25T00:09:21.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.200+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98640b5200 con 0x7f9864144910 2024-04-25T00:09:21.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.200+0000 7f984bfff640 1 -- 172.21.15.113:0/3276479502 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5195 (secure 0 0 0) 0x7f985c059fa0 con 0x7f9864144910 2024-04-25T00:09:21.201 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:21.201 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/992895335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":992895335},{"type":"v1","addr":"172.21.15.178:6834","nonce":992895335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/17230728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":17230728},{"type":"v1","addr":"172.21.15.113:6836","nonce":17230728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4170863917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4170863917},{"type":"v1","addr":"172.21.15.113:6837","nonce":4170863917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:08.629096+0000","modified":"2024-04-25T00:09:11.220151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10434":{"gid":10434,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/2813561857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2813561857},{"type":"v1","addr":"172.21.15.178:6835","nonce":2813561857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10434,"qdb_cluster":[10434]},"id":12}]} 2024-04-25T00:09:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-04-25T00:09:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f985006be30 msgr2=0x7f985006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f985006be30 0x7f985006e2f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9854005880 tx=0x7f9854009040 comp rx=0 tx=0).stop 2024-04-25T00:09:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9864144910 msgr2=0x7f98641e4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9864144910 0x7f98641e4eb0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f985c007cf0 tx=0x7f985c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 shutdown_connections 2024-04-25T00:09:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f985006be30 0x7f985006e2f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98640b3fd0 0x7f98641e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9864144910 0x7f98641e4eb0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 --2- 172.21.15.113:0/3276479502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f986406e510 0x7f986413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 >> 172.21.15.113:0/3276479502 conn(0x7f98640b1a50 msgr2=0x7f98640716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 shutdown_connections 2024-04-25T00:09:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.202+0000 7f9869e88640 1 -- 172.21.15.113:0/3276479502 wait complete. 2024-04-25T00:09:21.214 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.310+0000 7f0cd6ce1640 1 Processor -- start 2024-04-25T00:09:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.310+0000 7f0cd6ce1640 1 -- start start 2024-04-25T00:09:21.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd6ce1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0cd0144910 0x7f0cd00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd6ce1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cd00b56b0 0x7f0cd00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd6ce1640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd0144cf0 con 0x7f0cd00b5240 2024-04-25T00:09:21.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd6ce1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd00b4510 con 0x7f0cd00b56b0 2024-04-25T00:09:21.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd6ce1640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd00b2f30 con 0x7f0cd0144910 2024-04-25T00:09:21.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd527e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cd00b56b0 0x7f0cd00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd527e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cd00b56b0 0x7f0cd00cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35034/0 (socket says 172.21.15.113:35034) 2024-04-25T00:09:21.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd527e640 1 -- 172.21.15.113:0/1308368782 learned_addr learned my addr 172.21.15.113:0/1308368782 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:21.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd4a7d640 1 --2- 172.21.15.113:0/1308368782 >> v2:172.21.15.178:3300/0 conn(0x7f0cd0144910 0x7f0cd00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd527e640 1 -- 172.21.15.113:0/1308368782 >> v2:172.21.15.178:3300/0 conn(0x7f0cd0144910 msgr2=0x7f0cd00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd527e640 1 --2- 172.21.15.113:0/1308368782 >> v2:172.21.15.178:3300/0 conn(0x7f0cd0144910 0x7f0cd00b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd527e640 1 -- 172.21.15.113:0/1308368782 >> v1:172.21.15.178:6789/0 conn(0x7f0cd00b5240 legacy=0x7f0cd006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.311+0000 7f0cd527e640 1 -- 172.21.15.113:0/1308368782 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd00b3210 con 0x7f0cd00b56b0 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.312+0000 7f0cd527e640 1 --2- 172.21.15.113:0/1308368782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd00cd500 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009870 tx=0x7f0cb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d33969ad704332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.312+0000 7f0ccf7fe640 1 -- 172.21.15.113:0/1308368782 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8037070 con 0x7f0cd00b56b0 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.312+0000 7f0ccf7fe640 1 -- 172.21.15.113:0/1308368782 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb8032440 con 0x7f0cd00b56b0 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.312+0000 7f0ccf7fe640 1 -- 172.21.15.113:0/1308368782 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8032810 con 0x7f0cd00b56b0 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/1308368782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 msgr2=0x7f0cd00cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/1308368782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd00cd500 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009870 tx=0x7f0cb802a270 comp rx=0 tx=0).stop 2024-04-25T00:09:21.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/1308368782 shutdown_connections 2024-04-25T00:09:21.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/1308368782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd00cd500 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/1308368782 >> v2:172.21.15.178:3300/0 conn(0x7f0cd0144910 0x7f0cd00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/1308368782 >> 172.21.15.113:0/1308368782 conn(0x7f0cd00b1a50 msgr2=0x7f0cd00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:21.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/1308368782 shutdown_connections 2024-04-25T00:09:21.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.313+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/1308368782 wait complete. 2024-04-25T00:09:21.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.314+0000 7f0cd6ce1640 1 Processor -- start 2024-04-25T00:09:21.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.314+0000 7f0cd6ce1640 1 -- start start 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd6ce1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cd00b5240 0x7f0cd0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd6ce1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd013b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd6ce1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd0144910 0x7f0cd013b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd6ce1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00b4da0 con 0x7f0cd00b56b0 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd6ce1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00b37d0 con 0x7f0cd00b5240 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd6ce1640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd00b4f20 con 0x7f0cd0144910 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd013b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd013b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35038/0 (socket says 172.21.15.113:35038) 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 -- 172.21.15.113:0/3830672614 learned_addr learned my addr 172.21.15.113:0/3830672614 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd4a7d640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cd00b5240 0x7f0cd0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd527e640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd0144910 0x7f0cd013b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 -- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd0144910 msgr2=0x7f0cd013b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd0144910 0x7f0cd013b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 -- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cd00b5240 msgr2=0x7f0cd0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cd00b5240 0x7f0cd0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 -- 172.21.15.113:0/3830672614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd0153c80 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd4a7d640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cd00b5240 0x7f0cd0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccffff640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd013b080 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c7f0 tx=0x7f0cc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccd7fa640 1 -- 172.21.15.113:0/3830672614 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0016020 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccd7fa640 1 -- 172.21.15.113:0/3830672614 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc000aa50 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0ccd7fa640 1 -- 172.21.15.113:0/3830672614 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc000ae40 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.315+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd013a160 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.316+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd013cb30 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.317+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd00691a0 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.317+0000 7f0ccd7fa640 1 -- 172.21.15.113:0/3830672614 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cc0019020 con 0x7f0cd00b56b0 2024-04-25T00:09:21.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.317+0000 7f0ccd7fa640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cb006be30 0x7f0cb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.317+0000 7f0ccd7fa640 1 -- 172.21.15.113:0/3830672614 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f0cc008aa20 con 0x7f0cd00b56b0 2024-04-25T00:09:21.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.317+0000 7f0cd4a7d640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cb006be30 0x7f0cb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.318+0000 7f0cd4a7d640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cb006be30 0x7f0cb006e2f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8006fd0 tx=0x7f0cc8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.322+0000 7f0ccd7fa640 1 -- 172.21.15.113:0/3830672614 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0cc0020070 con 0x7f0cd00b56b0 2024-04-25T00:09:21.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.496+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cd00cf3c0 con 0x7f0cd00b56b0 2024-04-25T00:09:21.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.497+0000 7f0ccd7fa640 1 -- 172.21.15.113:0/3830672614 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16727 (secure 0 0 0) 0x7f0cc0059460 con 0x7f0cd00b56b0 2024-04-25T00:09:21.498 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:21.498 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:10.300297+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:06.844691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:07.603153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cb006be30 msgr2=0x7f0cb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cb006be30 0x7f0cb006e2f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8006fd0 tx=0x7f0cc8008040 comp rx=0 tx=0).stop 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 msgr2=0x7f0cd013b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd013b080 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c7f0 tx=0x7f0cc000ccb0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 shutdown_connections 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0cb006be30 0x7f0cb006e2f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0cd0144910 0x7f0cd013b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00b56b0 0x7f0cd013b080 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 --2- 172.21.15.113:0/3830672614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cd00b5240 0x7f0cd0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 >> 172.21.15.113:0/3830672614 conn(0x7f0cd00b1a50 msgr2=0x7f0cd00cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 shutdown_connections 2024-04-25T00:09:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.499+0000 7f0cd6ce1640 1 -- 172.21.15.113:0/3830672614 wait complete. 2024-04-25T00:09:21.511 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:09:21.511 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:21.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.606+0000 7fb03d15a640 1 Processor -- start 2024-04-25T00:09:21.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.606+0000 7fb03d15a640 1 -- start start 2024-04-25T00:09:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb03d15a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 0x7fb0380cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb03d15a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb038144cf0 con 0x7fb0380b3fd0 2024-04-25T00:09:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb03d15a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0380b2f30 con 0x7fb038144910 2024-04-25T00:09:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb03d15a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb0380b3210 con 0x7fb03806e6e0 2024-04-25T00:09:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb03659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb0380b3fd0 0x7fb03806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36736/0 (socket says 172.21.15.113:36736) 2024-04-25T00:09:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb03659c640 1 -- 172.21.15.113:0/3209606986 learned_addr learned my addr 172.21.15.113:0/3209606986 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb03759e640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 0x7fb0380cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52298291 0 0) 0x7fb038144cf0 con 0x7fb0380b3fd0 2024-04-25T00:09:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb020003520 con 0x7fb0380b3fd0 2024-04-25T00:09:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1992745836 0 0) 0x7fb020003520 con 0x7fb0380b3fd0 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 msgr2=0x7fb0380cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 0x7fb0380cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 >> v1:172.21.15.178:6789/0 conn(0x7fb038144910 legacy=0x7fb0380b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0380b34f0 con 0x7fb0380b3fd0 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fb02c001670 con 0x7fb0380b3fd0 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 >> v1:172.21.15.113:6789/0 conn(0x7fb0380b3fd0 legacy=0x7fb03806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 0x7fb0200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb020003f60 0x7fb020004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb020004d40 0x7fb0200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380b3210 con 0x7fb020003f60 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb02000baa0 con 0x7fb020004d40 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.607+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb02000bca0 con 0x7fb03806e6e0 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03659c640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb020003f60 0x7fb020004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb036d9d640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 0x7fb0200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03759e640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb020004d40 0x7fb0200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03659c640 1 -- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 msgr2=0x7fb0200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03659c640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb03806e6e0 0x7fb0200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03659c640 1 -- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb020004d40 msgr2=0x7fb0200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03659c640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb020004d40 0x7fb0200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03659c640 1 -- 172.21.15.113:0/3209606986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb020003520 con 0x7fb020003f60 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03759e640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb020004d40 0x7fb0200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.608+0000 7fb03659c640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb020003f60 0x7fb020004800 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00c240 tx=0x7fb020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ebeb53bba1ce9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c007d10 con 0x7fb020003f60 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb02c005ad0 con 0x7fb020003f60 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb035d9b640 1 -- 172.21.15.113:0/3209606986 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c007070 con 0x7fb020003f60 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb03d15a640 1 -- 172.21.15.113:0/3209606986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb020003f60 msgr2=0x7fb020004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb03d15a640 1 --2- 172.21.15.113:0/3209606986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb020003f60 0x7fb020004800 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00c240 tx=0x7fb020005830 comp rx=0 tx=0).stop 2024-04-25T00:09:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb03759e640 1 -- 172.21.15.113:0/3209606986 reap_dead start 2024-04-25T00:09:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb03d15a640 1 -- 172.21.15.113:0/3209606986 shutdown_connections 2024-04-25T00:09:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb03d15a640 1 -- 172.21.15.113:0/3209606986 >> 172.21.15.113:0/3209606986 conn(0x7fb0380b1a50 msgr2=0x7fb0380b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb03d15a640 1 -- 172.21.15.113:0/3209606986 shutdown_connections 2024-04-25T00:09:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.609+0000 7fb03d15a640 1 -- 172.21.15.113:0/3209606986 wait complete. 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 Processor -- start 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 -- start start 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb020003f60 0x7fb038155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03812f4d0 0x7fb03815db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0380b02c0 0x7fb0380b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380b4da0 con 0x7fb0380b02c0 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb0380b4440 con 0x7fb03812f4d0 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03d15a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb0380b4f20 con 0x7fb020003f60 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0380b02c0 0x7fb0380b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0380b02c0 0x7fb0380b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35048/0 (socket says 172.21.15.113:35048) 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb03759e640 1 -- 172.21.15.113:0/4109028429 learned_addr learned my addr 172.21.15.113:0/4109028429 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03659c640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03812f4d0 0x7fb03815db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.611+0000 7fb036d9d640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb020003f60 0x7fb038155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03759e640 1 -- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb020003f60 msgr2=0x7fb038155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03759e640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb020003f60 0x7fb038155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03759e640 1 -- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03812f4d0 msgr2=0x7fb03815db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03759e640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03812f4d0 0x7fb03815db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03759e640 1 -- 172.21.15.113:0/4109028429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb038156340 con 0x7fb0380b02c0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03759e640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0380b02c0 0x7fb0380b0750 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fb0380cfc70 tx=0x7fb030011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb01f7fe640 1 -- 172.21.15.113:0/4109028429 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030019020 con 0x7fb0380b02c0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb01f7fe640 1 -- 172.21.15.113:0/4109028429 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb030004440 con 0x7fb0380b02c0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03812f990 con 0x7fb0380b02c0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb01f7fe640 1 -- 172.21.15.113:0/4109028429 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030003850 con 0x7fb0380b02c0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.612+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03812ff50 con 0x7fb0380b02c0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.613+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0380691a0 con 0x7fb0380b02c0 2024-04-25T00:09:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.613+0000 7fb01f7fe640 1 -- 172.21.15.113:0/4109028429 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb03001c020 con 0x7fb0380b02c0 2024-04-25T00:09:21.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.614+0000 7fb01f7fe640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb02406be30 0x7fb02406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.614+0000 7fb01f7fe640 1 -- 172.21.15.113:0/4109028429 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fb03008aee0 con 0x7fb0380b02c0 2024-04-25T00:09:21.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.614+0000 7fb036d9d640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb02406be30 0x7fb02406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.614+0000 7fb036d9d640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb02406be30 0x7fb02406e2f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb028001ee0 tx=0x7fb028009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.618+0000 7fb01f7fe640 1 -- 172.21.15.113:0/4109028429 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb030023030 con 0x7fb0380b02c0 2024-04-25T00:09:21.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.827+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0380b0c90 con 0x7fb0380b02c0 2024-04-25T00:09:21.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.827+0000 7fb01f7fe640 1 -- 172.21.15.113:0/4109028429 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5195 (secure 0 0 0) 0x7fb030059240 con 0x7fb0380b02c0 2024-04-25T00:09:21.828 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:21.828 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/992895335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":992895335},{"type":"v1","addr":"172.21.15.178:6834","nonce":992895335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/17230728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":17230728},{"type":"v1","addr":"172.21.15.113:6836","nonce":17230728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4170863917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4170863917},{"type":"v1","addr":"172.21.15.113:6837","nonce":4170863917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:08.629096+0000","modified":"2024-04-25T00:09:11.220151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10434":{"gid":10434,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/2813561857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2813561857},{"type":"v1","addr":"172.21.15.178:6835","nonce":2813561857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10434,"qdb_cluster":[10434]},"id":12}]} 2024-04-25T00:09:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb02406be30 msgr2=0x7fb02406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb02406be30 0x7fb02406e2f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb028001ee0 tx=0x7fb028009040 comp rx=0 tx=0).stop 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0380b02c0 msgr2=0x7fb0380b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0380b02c0 0x7fb0380b0750 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fb0380cfc70 tx=0x7fb030011920 comp rx=0 tx=0).stop 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 shutdown_connections 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb02406be30 0x7fb02406e2f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03812f4d0 0x7fb03815db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0380b02c0 0x7fb0380b0750 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 --2- 172.21.15.113:0/4109028429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb020003f60 0x7fb038155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 >> 172.21.15.113:0/4109028429 conn(0x7fb0380b1a50 msgr2=0x7fb0380ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 shutdown_connections 2024-04-25T00:09:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.829+0000 7fb03d15a640 1 -- 172.21.15.113:0/4109028429 wait complete. 2024-04-25T00:09:21.840 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:21.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.936+0000 7f313e13b640 1 Processor -- start 2024-04-25T00:09:21.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.936+0000 7f313e13b640 1 -- start start 2024-04-25T00:09:21.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f313e13b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31380b3fd0 0x7f313806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f313e13b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f313806e510 0x7f31380cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f313e13b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3138144cf0 con 0x7f3138144910 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f313e13b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f313806e980 con 0x7f31380b3fd0 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f313e13b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f31380b2f30 con 0x7f313806e510 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f31377fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3138144910 0x7f31380b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36744/0 (socket says 172.21.15.113:36744) 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f31377fe640 1 -- 172.21.15.113:0/2779670907 learned_addr learned my addr 172.21.15.113:0/2779670907 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f3136ffd640 1 --2- 172.21.15.113:0/2779670907 >> v2:172.21.15.113:3300/0 conn(0x7f31380b3fd0 0x7f313806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.937+0000 7f31367fc640 1 -- 172.21.15.113:0/2779670907 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689452850 0 0) 0x7f3138144cf0 con 0x7f3138144910 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f3137fff640 1 --2- 172.21.15.113:0/2779670907 >> v2:172.21.15.178:3300/0 conn(0x7f313806e510 0x7f31380cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f31367fc640 1 -- 172.21.15.113:0/2779670907 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3120005730 con 0x7f3138144910 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f3136ffd640 1 -- 172.21.15.113:0/2779670907 >> v2:172.21.15.178:3300/0 conn(0x7f313806e510 msgr2=0x7f31380cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:21.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f3136ffd640 1 --2- 172.21.15.113:0/2779670907 >> v2:172.21.15.178:3300/0 conn(0x7f313806e510 0x7f31380cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f3136ffd640 1 -- 172.21.15.113:0/2779670907 >> v1:172.21.15.113:6789/0 conn(0x7f3138144910 legacy=0x7f31380b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f3136ffd640 1 -- 172.21.15.113:0/2779670907 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31380b3210 con 0x7f31380b3fd0 2024-04-25T00:09:21.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f3137fff640 1 --2- 172.21.15.113:0/2779670907 >> v2:172.21.15.178:3300/0 conn(0x7f313806e510 0x7f31380cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:21.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f3136ffd640 1 --2- 172.21.15.113:0/2779670907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31380b3fd0 0x7f313806dfd0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f312c0097a0 tx=0x7f312c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20140e8d8976641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f31367fc640 1 -- 172.21.15.113:0/2779670907 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c037070 con 0x7f31380b3fd0 2024-04-25T00:09:21.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f31367fc640 1 -- 172.21.15.113:0/2779670907 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c032440 con 0x7f31380b3fd0 2024-04-25T00:09:21.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.938+0000 7f31367fc640 1 -- 172.21.15.113:0/2779670907 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c032810 con 0x7f31380b3fd0 2024-04-25T00:09:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 -- 172.21.15.113:0/2779670907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31380b3fd0 msgr2=0x7f313806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 --2- 172.21.15.113:0/2779670907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31380b3fd0 0x7f313806dfd0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f312c0097a0 tx=0x7f312c02a1e0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 -- 172.21.15.113:0/2779670907 shutdown_connections 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 --2- 172.21.15.113:0/2779670907 >> v2:172.21.15.178:3300/0 conn(0x7f313806e510 0x7f31380cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 --2- 172.21.15.113:0/2779670907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31380b3fd0 0x7f313806dfd0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 -- 172.21.15.113:0/2779670907 >> 172.21.15.113:0/2779670907 conn(0x7f31380b1a50 msgr2=0x7f31380b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 -- 172.21.15.113:0/2779670907 shutdown_connections 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.939+0000 7f313e13b640 1 -- 172.21.15.113:0/2779670907 wait complete. 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.940+0000 7f313e13b640 1 Processor -- start 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.940+0000 7f313e13b640 1 -- start start 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f313e13b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f313806e510 0x7f313813af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f313e13b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31380b3fd0 0x7f313815d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f313e13b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3138144910 0x7f31380b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f313e13b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f31380b4440 con 0x7f3138144910 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f313e13b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31380b37d0 con 0x7f313806e510 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f313e13b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f31380b4da0 con 0x7f31380b3fd0 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3138144910 0x7f31380b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3138144910 0x7f31380b0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35054/0 (socket says 172.21.15.113:35054) 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 -- 172.21.15.113:0/1650737608 learned_addr learned my addr 172.21.15.113:0/1650737608 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 -- 172.21.15.113:0/1650737608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31380b3fd0 msgr2=0x7f313815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31380b3fd0 0x7f313815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 -- 172.21.15.113:0/1650737608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f313806e510 msgr2=0x7f313813af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f313806e510 0x7f313813af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.941+0000 7f3137fff640 1 -- 172.21.15.113:0/1650737608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313813dfb0 con 0x7f3138144910 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.942+0000 7f3137fff640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3138144910 0x7f31380b0a90 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f313000ce70 tx=0x7f313000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.942+0000 7f311ffff640 1 -- 172.21.15.113:0/1650737608 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313001a070 con 0x7f3138144910 2024-04-25T00:09:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.942+0000 7f311ffff640 1 -- 172.21.15.113:0/1650737608 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313000d840 con 0x7f3138144910 2024-04-25T00:09:21.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.942+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313813e230 con 0x7f3138144910 2024-04-25T00:09:21.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.942+0000 7f311ffff640 1 -- 172.21.15.113:0/1650737608 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313000dc30 con 0x7f3138144910 2024-04-25T00:09:21.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.942+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313813e7f0 con 0x7f3138144910 2024-04-25T00:09:21.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.943+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31380691a0 con 0x7f3138144910 2024-04-25T00:09:21.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.943+0000 7f311ffff640 1 -- 172.21.15.113:0/1650737608 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f313001d070 con 0x7f3138144910 2024-04-25T00:09:21.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.944+0000 7f311ffff640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f312406be30 0x7f312406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:21.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.944+0000 7f31377fe640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f312406be30 0x7f312406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:21.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.944+0000 7f311ffff640 1 -- 172.21.15.113:0/1650737608 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f313008b5b0 con 0x7f3138144910 2024-04-25T00:09:21.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.944+0000 7f31377fe640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f312406be30 0x7f312406e2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3128000f60 tx=0x7f31280078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:21.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:21.948+0000 7f311ffff640 1 -- 172.21.15.113:0/1650737608 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f313813dfb0 con 0x7f3138144910 2024-04-25T00:09:22.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.158+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3138142900 con 0x7f3138144910 2024-04-25T00:09:22.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.158+0000 7f311ffff640 1 -- 172.21.15.113:0/1650737608 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5195 (secure 0 0 0) 0x7f3130059ff0 con 0x7f3138144910 2024-04-25T00:09:22.159 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:22.159 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/992895335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":992895335},{"type":"v1","addr":"172.21.15.178:6834","nonce":992895335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/17230728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":17230728},{"type":"v1","addr":"172.21.15.113:6836","nonce":17230728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4170863917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4170863917},{"type":"v1","addr":"172.21.15.113:6837","nonce":4170863917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:08.629096+0000","modified":"2024-04-25T00:09:11.220151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10434":{"gid":10434,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/2813561857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2813561857},{"type":"v1","addr":"172.21.15.178:6835","nonce":2813561857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10434,"qdb_cluster":[10434]},"id":12}]} 2024-04-25T00:09:22.159 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-04-25T00:09:22.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f312406be30 msgr2=0x7f312406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f312406be30 0x7f312406e2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3128000f60 tx=0x7f31280078f0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3138144910 msgr2=0x7f31380b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3138144910 0x7f31380b0a90 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f313000ce70 tx=0x7f313000a710 comp rx=0 tx=0).stop 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 shutdown_connections 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f312406be30 0x7f312406e2f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3138144910 0x7f31380b0a90 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31380b3fd0 0x7f313815d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 --2- 172.21.15.113:0/1650737608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f313806e510 0x7f313813af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 >> 172.21.15.113:0/1650737608 conn(0x7f31380b1a50 msgr2=0x7f3138142e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 shutdown_connections 2024-04-25T00:09:22.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.160+0000 7f313e13b640 1 -- 172.21.15.113:0/1650737608 wait complete. 2024-04-25T00:09:22.172 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:22.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.268+0000 7fc0c81e7640 1 Processor -- start 2024-04-25T00:09:22.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.268+0000 7fc0c81e7640 1 -- start start 2024-04-25T00:09:22.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.269+0000 7fc0c81e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c0144d60 0x7fc0c00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.269+0000 7fc0c81e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c00cfad0 con 0x7fc0c006e6e0 2024-04-25T00:09:22.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.269+0000 7fc0c81e7640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c00cf230 con 0x7fc0c01453c0 2024-04-25T00:09:22.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.269+0000 7fc0c81e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c0145c40 con 0x7fc0c0144d60 2024-04-25T00:09:22.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.269+0000 7fc0c6784640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc0c006e6e0 0x7fc0c0071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36760/0 (socket says 172.21.15.113:36760) 2024-04-25T00:09:22.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.269+0000 7fc0c6784640 1 -- 172.21.15.113:0/2207804697 learned_addr learned my addr 172.21.15.113:0/2207804697 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:22.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.269+0000 7fc0c5f83640 1 --2- 172.21.15.113:0/2207804697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c0144d60 0x7fc0c00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1992851532 0 0) 0x7fc0c00cfad0 con 0x7fc0c006e6e0 2024-04-25T00:09:22.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0ac003520 con 0x7fc0c006e6e0 2024-04-25T00:09:22.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155897539 0 0) 0x7fc0c00cf230 con 0x7fc0c01453c0 2024-04-25T00:09:22.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0c00cfad0 con 0x7fc0c01453c0 2024-04-25T00:09:22.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3499631400 0 0) 0x7fc0ac003520 con 0x7fc0c006e6e0 2024-04-25T00:09:22.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c5f83640 1 -- 172.21.15.113:0/2207804697 >> v1:172.21.15.178:6789/0 conn(0x7fc0c01453c0 legacy=0x7fc0c006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c5f83640 1 -- 172.21.15.113:0/2207804697 >> v1:172.21.15.113:6789/0 conn(0x7fc0c006e6e0 legacy=0x7fc0c0071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.270+0000 7fc0c5f83640 1 -- 172.21.15.113:0/2207804697 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c00cf510 con 0x7fc0c0144d60 2024-04-25T00:09:22.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c5f83640 1 --2- 172.21.15.113:0/2207804697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c0144d60 0x7fc0c00b5390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009dc0 tx=0x7fc0bc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8b78abab115303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80015f0 con 0x7fc0c0144d60 2024-04-25T00:09:22.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0bc007540 con 0x7fc0c0144d60 2024-04-25T00:09:22.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c4f81640 1 -- 172.21.15.113:0/2207804697 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0078f0 con 0x7fc0c0144d60 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c81e7640 1 -- 172.21.15.113:0/2207804697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c0144d60 msgr2=0x7fc0c00b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/2207804697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c0144d60 0x7fc0c00b5390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009dc0 tx=0x7fc0bc02a5d0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c81e7640 1 -- 172.21.15.113:0/2207804697 shutdown_connections 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/2207804697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c0144d60 0x7fc0c00b5390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c81e7640 1 -- 172.21.15.113:0/2207804697 >> 172.21.15.113:0/2207804697 conn(0x7fc0c00b1a50 msgr2=0x7fc0c00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c81e7640 1 -- 172.21.15.113:0/2207804697 shutdown_connections 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.271+0000 7fc0c81e7640 1 -- 172.21.15.113:0/2207804697 wait complete. 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.272+0000 7fc0c81e7640 1 Processor -- start 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c81e7640 1 -- start start 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c81e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c006e6e0 0x7fc0c013c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c81e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c0144d60 0x7fc0c00a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c81e7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c01453c0 0x7fc0c01f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c81e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0145140 con 0x7fc0c006e6e0 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c81e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0070e10 con 0x7fc0c0144d60 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c81e7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c0071a40 con 0x7fc0c01453c0 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c5f83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c006e6e0 0x7fc0c013c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c5f83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c006e6e0 0x7fc0c013c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35056/0 (socket says 172.21.15.113:35056) 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c5f83640 1 -- 172.21.15.113:0/1952453623 learned_addr learned my addr 172.21.15.113:0/1952453623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c6784640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c01453c0 0x7fc0c01f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.273+0000 7fc0c5782640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c0144d60 0x7fc0c00a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c5f83640 1 -- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c01453c0 msgr2=0x7fc0c01f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c5f83640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c01453c0 0x7fc0c01f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c5f83640 1 -- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c0144d60 msgr2=0x7fc0c00a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c5f83640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c0144d60 0x7fc0c00a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c5f83640 1 -- 172.21.15.113:0/1952453623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c015f460 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c5f83640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c006e6e0 0x7fc0c013c9b0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009ee0 tx=0x7fc0bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0b67fc640 1 -- 172.21.15.113:0/1952453623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc040070 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0b67fc640 1 -- 172.21.15.113:0/1952453623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0bc031bc0 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0b67fc640 1 -- 172.21.15.113:0/1952453623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc009450 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c015f740 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.274+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c015fd00 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.275+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c00691a0 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.275+0000 7fc0b67fc640 1 -- 172.21.15.113:0/1952453623 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0bc039040 con 0x7fc0c006e6e0 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.276+0000 7fc0b67fc640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0a406be30 0x7fc0a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.276+0000 7fc0b67fc640 1 -- 172.21.15.113:0/1952453623 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7fc0bc0ac720 con 0x7fc0c006e6e0 2024-04-25T00:09:22.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.276+0000 7fc0c5782640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0a406be30 0x7fc0a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.276+0000 7fc0c5782640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0a406be30 0x7fc0a406e2f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00730c0 tx=0x7fc0b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.280+0000 7fc0b67fc640 1 -- 172.21.15.113:0/1952453623 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0bc07b040 con 0x7fc0c006e6e0 2024-04-25T00:09:22.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.455+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0c0068a20 con 0x7fc0c006e6e0 2024-04-25T00:09:22.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.456+0000 7fc0b67fc640 1 -- 172.21.15.113:0/1952453623 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16727 (secure 0 0 0) 0x7fc0bc07a940 con 0x7fc0c006e6e0 2024-04-25T00:09:22.457 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:22.457 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:10.300297+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:06.844691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:07.603153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:22.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.458+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0a406be30 msgr2=0x7fc0a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.458+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0a406be30 0x7fc0a406e2f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00730c0 tx=0x7fc0b0009040 comp rx=0 tx=0).stop 2024-04-25T00:09:22.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.458+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c006e6e0 msgr2=0x7fc0c013c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c006e6e0 0x7fc0c013c9b0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009ee0 tx=0x7fc0bc004a40 comp rx=0 tx=0).stop 2024-04-25T00:09:22.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 shutdown_connections 2024-04-25T00:09:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc0a406be30 0x7fc0a406e2f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c0144d60 0x7fc0c00a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c01453c0 0x7fc0c01f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 --2- 172.21.15.113:0/1952453623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c006e6e0 0x7fc0c013c9b0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 >> 172.21.15.113:0/1952453623 conn(0x7fc0c00b1a50 msgr2=0x7fc0c00ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 shutdown_connections 2024-04-25T00:09:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.459+0000 7fc0c81e7640 1 -- 172.21.15.113:0/1952453623 wait complete. 2024-04-25T00:09:22.470 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:09:22.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.567+0000 7f2d95ca5640 1 Processor -- start 2024-04-25T00:09:22.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.567+0000 7f2d95ca5640 1 -- start start 2024-04-25T00:09:22.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d95ca5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2d900d06c0 0x7f2d9015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d95ca5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d901604d0 0x7f2d900cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d95ca5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9015d6b0 con 0x7f2d9013f050 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d95ca5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2d9015c6a0 con 0x7f2d901604d0 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d95ca5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2d9015c8f0 con 0x7f2d900d06c0 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2d9013f050 0x7f2d900cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36762/0 (socket says 172.21.15.113:36762) 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8f7fe640 1 -- 172.21.15.113:0/2781229072 learned_addr learned my addr 172.21.15.113:0/2781229072 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8ffff640 1 --2- 172.21.15.113:0/2781229072 >> v2:172.21.15.113:3300/0 conn(0x7f2d901604d0 0x7f2d900cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8effd640 1 --2- 172.21.15.113:0/2781229072 >> v2:172.21.15.178:3300/0 conn(0x7f2d900d06c0 0x7f2d9015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8e7fc640 1 -- 172.21.15.113:0/2781229072 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4012539090 0 0) 0x7f2d9015d6b0 con 0x7f2d9013f050 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8e7fc640 1 -- 172.21.15.113:0/2781229072 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d78003520 con 0x7f2d9013f050 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8ffff640 1 -- 172.21.15.113:0/2781229072 >> v2:172.21.15.178:3300/0 conn(0x7f2d900d06c0 msgr2=0x7f2d9015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8ffff640 1 --2- 172.21.15.113:0/2781229072 >> v2:172.21.15.178:3300/0 conn(0x7f2d900d06c0 0x7f2d9015ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8ffff640 1 -- 172.21.15.113:0/2781229072 >> v1:172.21.15.113:6789/0 conn(0x7f2d9013f050 legacy=0x7f2d900cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.568+0000 7f2d8ffff640 1 -- 172.21.15.113:0/2781229072 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9015cbd0 con 0x7f2d901604d0 2024-04-25T00:09:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.569+0000 7f2d8ffff640 1 --2- 172.21.15.113:0/2781229072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d901604d0 0x7f2d900cea60 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006f490 tx=0x7f2d8802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b670220e21397b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.569+0000 7f2d8e7fc640 1 -- 172.21.15.113:0/2781229072 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88007740 con 0x7f2d901604d0 2024-04-25T00:09:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.569+0000 7f2d8e7fc640 1 -- 172.21.15.113:0/2781229072 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d880078e0 con 0x7f2d901604d0 2024-04-25T00:09:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.569+0000 7f2d8e7fc640 1 -- 172.21.15.113:0/2781229072 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88007cb0 con 0x7f2d901604d0 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 -- 172.21.15.113:0/2781229072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d901604d0 msgr2=0x7f2d900cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/2781229072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d901604d0 0x7f2d900cea60 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006f490 tx=0x7f2d8802a5f0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 -- 172.21.15.113:0/2781229072 shutdown_connections 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/2781229072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d901604d0 0x7f2d900cea60 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/2781229072 >> v2:172.21.15.178:3300/0 conn(0x7f2d900d06c0 0x7f2d9015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 -- 172.21.15.113:0/2781229072 >> 172.21.15.113:0/2781229072 conn(0x7f2d9006b0f0 msgr2=0x7f2d9006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 -- 172.21.15.113:0/2781229072 shutdown_connections 2024-04-25T00:09:22.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.570+0000 7f2d95ca5640 1 -- 172.21.15.113:0/2781229072 wait complete. 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.571+0000 7f2d95ca5640 1 Processor -- start 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.571+0000 7f2d95ca5640 1 -- start start 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d95ca5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d900d06c0 0x7f2d9014a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d95ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d9013f050 0x7f2d9015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d95ca5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d901604d0 0x7f2d9015e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d95ca5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9013f430 con 0x7f2d9013f050 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d95ca5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9015d190 con 0x7f2d901604d0 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d95ca5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d90160960 con 0x7f2d900d06c0 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d9013f050 0x7f2d9015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d9013f050 0x7f2d9015e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35072/0 (socket says 172.21.15.113:35072) 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 -- 172.21.15.113:0/1530454545 learned_addr learned my addr 172.21.15.113:0/1530454545 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8f7fe640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d900d06c0 0x7f2d9014a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8ffff640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d901604d0 0x7f2d9015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 -- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d900d06c0 msgr2=0x7f2d9014a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d900d06c0 0x7f2d9014a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 -- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d901604d0 msgr2=0x7f2d9015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d901604d0 0x7f2d9015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 -- 172.21.15.113:0/1530454545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9014e7e0 con 0x7f2d9013f050 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8ffff640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d901604d0 0x7f2d9015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8f7fe640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d900d06c0 0x7f2d9014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d8effd640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d9013f050 0x7f2d9015e000 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400e8b0 tx=0x7f2d8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d77fff640 1 -- 172.21.15.113:0/1530454545 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84017070 con 0x7f2d9013f050 2024-04-25T00:09:22.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.572+0000 7f2d77fff640 1 -- 172.21.15.113:0/1530454545 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d8400c940 con 0x7f2d9013f050 2024-04-25T00:09:22.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.573+0000 7f2d77fff640 1 -- 172.21.15.113:0/1530454545 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400cd30 con 0x7f2d9013f050 2024-04-25T00:09:22.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.573+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9014b140 con 0x7f2d9013f050 2024-04-25T00:09:22.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.573+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9014b700 con 0x7f2d9013f050 2024-04-25T00:09:22.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.573+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d900691a0 con 0x7f2d9013f050 2024-04-25T00:09:22.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.574+0000 7f2d77fff640 1 -- 172.21.15.113:0/1530454545 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d8401a070 con 0x7f2d9013f050 2024-04-25T00:09:22.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.574+0000 7f2d77fff640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7c06be30 0x7f2d7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.574+0000 7f2d77fff640 1 -- 172.21.15.113:0/1530454545 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f2d8408ae40 con 0x7f2d9013f050 2024-04-25T00:09:22.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.574+0000 7f2d8f7fe640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7c06be30 0x7f2d7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.575+0000 7f2d8f7fe640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7c06be30 0x7f2d7c06e2f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007fd0 tx=0x7f2d80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.578+0000 7f2d77fff640 1 -- 172.21.15.113:0/1530454545 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d840597f0 con 0x7f2d9013f050 2024-04-25T00:09:22.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.790+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2d900d0e50 con 0x7f2d9013f050 2024-04-25T00:09:22.792 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:09:22.791+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:09:22.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.792+0000 7f2d77fff640 1 -- 172.21.15.113:0/1530454545 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f2d840590f0 con 0x7f2d9013f050 2024-04-25T00:09:22.792 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:09:22.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7c06be30 msgr2=0x7f2d7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7c06be30 0x7f2d7c06e2f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007fd0 tx=0x7f2d80009040 comp rx=0 tx=0).stop 2024-04-25T00:09:22.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d9013f050 msgr2=0x7f2d9015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d9013f050 0x7f2d9015e000 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400e8b0 tx=0x7f2d8400ed70 comp rx=0 tx=0).stop 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 shutdown_connections 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2d7c06be30 0x7f2d7c06e2f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d901604d0 0x7f2d9015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d9013f050 0x7f2d9015e000 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 --2- 172.21.15.113:0/1530454545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d900d06c0 0x7f2d9014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 >> 172.21.15.113:0/1530454545 conn(0x7f2d9006b0f0 msgr2=0x7f2d9015d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 shutdown_connections 2024-04-25T00:09:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.796+0000 7f2d95ca5640 1 -- 172.21.15.113:0/1530454545 wait complete. 2024-04-25T00:09:22.808 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:09:22.835 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:09:22.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.907+0000 7f641c3c8640 1 Processor -- start 2024-04-25T00:09:22.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.907+0000 7f641c3c8640 1 -- start start 2024-04-25T00:09:22.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641c3c8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64140b4a40 0x7f6414165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641c3c8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641415f4f0 con 0x7f641415e2d0 2024-04-25T00:09:22.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641c3c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641415f7d0 con 0x7f64140b3080 2024-04-25T00:09:22.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641c3c8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f64140cde00 con 0x7f64140b4a40 2024-04-25T00:09:22.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641a965640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64140b4a40 0x7f6414165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641a164640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f641415e2d0 0x7f641415ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36776/0 (socket says 172.21.15.113:36776) 2024-04-25T00:09:22.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641a164640 1 -- 172.21.15.113:0/78944992 learned_addr learned my addr 172.21.15.113:0/78944992 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:22.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641a965640 1 -- 172.21.15.113:0/78944992 >> v1:172.21.15.178:6789/0 conn(0x7f64140b3080 legacy=0x7f64140b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641a965640 1 -- 172.21.15.113:0/78944992 >> v1:172.21.15.113:6789/0 conn(0x7f641415e2d0 legacy=0x7f641415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.908+0000 7f641a965640 1 -- 172.21.15.113:0/78944992 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641415fab0 con 0x7f64140b4a40 2024-04-25T00:09:22.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.909+0000 7f641a965640 1 --2- 172.21.15.113:0/78944992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64140b4a40 0x7f6414165f80 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f64140cdf80 tx=0x7f640402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3ae9b898a3a74d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.909+0000 7f6419162640 1 -- 172.21.15.113:0/78944992 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404037070 con 0x7f64140b4a40 2024-04-25T00:09:22.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.909+0000 7f6419162640 1 -- 172.21.15.113:0/78944992 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6404032440 con 0x7f64140b4a40 2024-04-25T00:09:22.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.909+0000 7f6419162640 1 -- 172.21.15.113:0/78944992 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404032810 con 0x7f64140b4a40 2024-04-25T00:09:22.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.909+0000 7f641c3c8640 1 -- 172.21.15.113:0/78944992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64140b4a40 msgr2=0x7f6414165f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.909+0000 7f641c3c8640 1 --2- 172.21.15.113:0/78944992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64140b4a40 0x7f6414165f80 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f64140cdf80 tx=0x7f640402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.909+0000 7f641c3c8640 1 -- 172.21.15.113:0/78944992 shutdown_connections 2024-04-25T00:09:22.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.910+0000 7f641c3c8640 1 --2- 172.21.15.113:0/78944992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64140b4a40 0x7f6414165f80 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.910+0000 7f641c3c8640 1 -- 172.21.15.113:0/78944992 >> 172.21.15.113:0/78944992 conn(0x7f641406b0f0 msgr2=0x7f641406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.910+0000 7f641c3c8640 1 -- 172.21.15.113:0/78944992 shutdown_connections 2024-04-25T00:09:22.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.910+0000 7f641c3c8640 1 -- 172.21.15.113:0/78944992 wait complete. 2024-04-25T00:09:22.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 Processor -- start 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 -- start start 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64140b3080 0x7f641415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64140b4a40 0x7f6414157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641415e2d0 0x7f64141583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f64140cf420 con 0x7f641415e2d0 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64140cf2a0 con 0x7f64140b4a40 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.911+0000 7f641c3c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64140cf5a0 con 0x7f64140b3080 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641415e2d0 0x7f64141583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641415e2d0 0x7f64141583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35088/0 (socket says 172.21.15.113:35088) 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 -- 172.21.15.113:0/3301476528 learned_addr learned my addr 172.21.15.113:0/3301476528 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a164640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64140b3080 0x7f641415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f6419963640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64140b4a40 0x7f6414157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 -- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64140b3080 msgr2=0x7f641415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64140b3080 0x7f641415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 -- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64140b4a40 msgr2=0x7f6414157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64140b4a40 0x7f6414157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 -- 172.21.15.113:0/3301476528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64141447e0 con 0x7f641415e2d0 2024-04-25T00:09:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641a965640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641415e2d0 0x7f64141583b0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f640402a200 tx=0x7f64040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f6412ffd640 1 -- 172.21.15.113:0/3301476528 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404037040 con 0x7f641415e2d0 2024-04-25T00:09:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f6412ffd640 1 -- 172.21.15.113:0/3301476528 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6404042070 con 0x7f641415e2d0 2024-04-25T00:09:22.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64141589e0 con 0x7f641415e2d0 2024-04-25T00:09:22.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f6412ffd640 1 -- 172.21.15.113:0/3301476528 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404030bd0 con 0x7f641415e2d0 2024-04-25T00:09:22.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.912+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6414158fa0 con 0x7f641415e2d0 2024-04-25T00:09:22.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.913+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64140691a0 con 0x7f641415e2d0 2024-04-25T00:09:22.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.914+0000 7f6412ffd640 1 -- 172.21.15.113:0/3301476528 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6404047460 con 0x7f641415e2d0 2024-04-25T00:09:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.914+0000 7f6412ffd640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f640006be30 0x7f640006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.914+0000 7f6412ffd640 1 -- 172.21.15.113:0/3301476528 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5795+0+0 (secure 0 0 0) 0x7f64040abbc0 con 0x7f641415e2d0 2024-04-25T00:09:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.914+0000 7f641a164640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f640006be30 0x7f640006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.914+0000 7f641a164640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f640006be30 0x7f640006e2f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f640c0012e0 tx=0x7f640c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:22.918+0000 7f6412ffd640 1 -- 172.21.15.113:0/3301476528 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6404032e20 con 0x7f641415e2d0 2024-04-25T00:09:23.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.126+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f641406b570 con 0x7f641415e2d0 2024-04-25T00:09:23.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.797+0000 7f6412ffd640 1 -- 172.21.15.113:0/3301476528 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7f6404045360 con 0x7f641415e2d0 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f640006be30 msgr2=0x7f640006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f640006be30 0x7f640006e2f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f640c0012e0 tx=0x7f640c009040 comp rx=0 tx=0).stop 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641415e2d0 msgr2=0x7f64141583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641415e2d0 0x7f64141583b0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f640402a200 tx=0x7f64040046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 shutdown_connections 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f640006be30 0x7f640006e2f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64140b4a40 0x7f6414157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641415e2d0 0x7f64141583b0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 --2- 172.21.15.113:0/3301476528 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64140b3080 0x7f641415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 >> 172.21.15.113:0/3301476528 conn(0x7f641406b0f0 msgr2=0x7f641415d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:23.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 shutdown_connections 2024-04-25T00:09:23.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.799+0000 7f641c3c8640 1 -- 172.21.15.113:0/3301476528 wait complete. 2024-04-25T00:09:23.812 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:09:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.913+0000 7f18e222a640 1 Processor -- start 2024-04-25T00:09:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.913+0000 7f18e222a640 1 -- start start 2024-04-25T00:09:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18e222a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18dc0cc0b0 0x7f18dc15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18e222a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f18dc0b47a0 0x7f18dc154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18e222a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18dc15bcf0 con 0x7f18dc0b3f80 2024-04-25T00:09:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18e222a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f18dc06e7e0 con 0x7f18dc0cc0b0 2024-04-25T00:09:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18e222a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f18dc0b2f40 con 0x7f18dc0b47a0 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18db7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18dc0cc0b0 0x7f18dc15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18db7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18dc0cc0b0 0x7f18dc15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35092/0 (socket says 172.21.15.113:35092) 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18db7fe640 1 -- 172.21.15.113:0/805397793 learned_addr learned my addr 172.21.15.113:0/805397793 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18dbfff640 1 --2- 172.21.15.113:0/805397793 >> v2:172.21.15.178:3300/0 conn(0x7f18dc0b47a0 0x7f18dc154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18db7fe640 1 -- 172.21.15.113:0/805397793 >> v2:172.21.15.178:3300/0 conn(0x7f18dc0b47a0 msgr2=0x7f18dc154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18db7fe640 1 --2- 172.21.15.113:0/805397793 >> v2:172.21.15.178:3300/0 conn(0x7f18dc0b47a0 0x7f18dc154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18db7fe640 1 -- 172.21.15.113:0/805397793 >> v1:172.21.15.178:6789/0 conn(0x7f18dc0b3f80 legacy=0x7f18dc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18db7fe640 1 -- 172.21.15.113:0/805397793 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc0b31e0 con 0x7f18dc0cc0b0 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.914+0000 7f18dbfff640 1 --2- 172.21.15.113:0/805397793 >> v2:172.21.15.178:3300/0 conn(0x7f18dc0b47a0 0x7f18dc154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.915+0000 7f18db7fe640 1 --2- 172.21.15.113:0/805397793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0cc0b0 0x7f18dc15b7b0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0b5370 tx=0x7f18cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e2be3c42e4b1aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.915+0000 7f18da7fc640 1 -- 172.21.15.113:0/805397793 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc037070 con 0x7f18dc0cc0b0 2024-04-25T00:09:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.915+0000 7f18da7fc640 1 -- 172.21.15.113:0/805397793 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18cc032440 con 0x7f18dc0cc0b0 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.915+0000 7f18da7fc640 1 -- 172.21.15.113:0/805397793 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc032810 con 0x7f18dc0cc0b0 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.915+0000 7f18e222a640 1 -- 172.21.15.113:0/805397793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0cc0b0 msgr2=0x7f18dc15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.915+0000 7f18e222a640 1 --2- 172.21.15.113:0/805397793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0cc0b0 0x7f18dc15b7b0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0b5370 tx=0x7f18cc02a230 comp rx=0 tx=0).stop 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.916+0000 7f18e222a640 1 -- 172.21.15.113:0/805397793 shutdown_connections 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.916+0000 7f18e222a640 1 --2- 172.21.15.113:0/805397793 >> v2:172.21.15.178:3300/0 conn(0x7f18dc0b47a0 0x7f18dc154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.916+0000 7f18e222a640 1 --2- 172.21.15.113:0/805397793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0cc0b0 0x7f18dc15b7b0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.916+0000 7f18e222a640 1 -- 172.21.15.113:0/805397793 >> 172.21.15.113:0/805397793 conn(0x7f18dc0b1b30 msgr2=0x7f18dc0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.916+0000 7f18e222a640 1 -- 172.21.15.113:0/805397793 shutdown_connections 2024-04-25T00:09:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.916+0000 7f18e222a640 1 -- 172.21.15.113:0/805397793 wait complete. 2024-04-25T00:09:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.917+0000 7f18e222a640 1 Processor -- start 2024-04-25T00:09:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.917+0000 7f18e222a640 1 -- start start 2024-04-25T00:09:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc0b3f80 0x7f18dc15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0b47a0 0x7f18dc0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18dc0cc0b0 0x7f18dc145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc0b3a80 con 0x7f18dc0b47a0 2024-04-25T00:09:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc0b37a0 con 0x7f18dc0b3f80 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f18dc0b4c50 con 0x7f18dc0cc0b0 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0b47a0 0x7f18dc0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0b47a0 0x7f18dc0a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35098/0 (socket says 172.21.15.113:35098) 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 -- 172.21.15.113:0/111807642 learned_addr learned my addr 172.21.15.113:0/111807642 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18db7fe640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc0b3f80 0x7f18dc15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 -- 172.21.15.113:0/111807642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18dc0cc0b0 msgr2=0x7f18dc145920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18dc0cc0b0 0x7f18dc145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 -- 172.21.15.113:0/111807642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc0b3f80 msgr2=0x7f18dc15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc0b3f80 0x7f18dc15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 -- 172.21.15.113:0/111807642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc0a9bf0 con 0x7f18dc0b47a0 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18daffd640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0b47a0 0x7f18dc0a9480 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f18d000c460 tx=0x7f18d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18c3fff640 1 -- 172.21.15.113:0/111807642 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d00108a0 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18c3fff640 1 -- 172.21.15.113:0/111807642 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d0010a80 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18c3fff640 1 -- 172.21.15.113:0/111807642 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0011470 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc0a9ed0 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.918+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc0aa490 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.919+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc0aa980 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.920+0000 7f18c3fff640 1 -- 172.21.15.113:0/111807642 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18d0011610 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.920+0000 7f18c3fff640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18c806be30 0x7f18c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.920+0000 7f18db7fe640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18c806be30 0x7f18c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.920+0000 7f18c3fff640 1 -- 172.21.15.113:0/111807642 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5795+0+0 (secure 0 0 0) 0x7f18d008ce90 con 0x7f18dc0b47a0 2024-04-25T00:09:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.921+0000 7f18db7fe640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18c806be30 0x7f18c806e2f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f18cc034000 tx=0x7f18cc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:23.925+0000 7f18c3fff640 1 -- 172.21.15.113:0/111807642 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18d0016030 con 0x7f18dc0b47a0 2024-04-25T00:09:24.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.097+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f18dc0691a0 con 0x7f18dc0b47a0 2024-04-25T00:09:24.098 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:09:24.100 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:09:24.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.798+0000 7f18c3fff640 1 -- 172.21.15.113:0/111807642 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v172) ==== 181+0+0 (secure 0 0 0) 0x7f18d005b7e0 con 0x7f18dc0b47a0 2024-04-25T00:09:24.799 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:09:24.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.801+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18c806be30 msgr2=0x7f18c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:24.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.801+0000 7f18e222a640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18c806be30 0x7f18c806e2f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f18cc034000 tx=0x7f18cc02e000 comp rx=0 tx=0).stop 2024-04-25T00:09:24.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.801+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0b47a0 msgr2=0x7f18dc0a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:24.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.801+0000 7f18e222a640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0b47a0 0x7f18dc0a9480 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f18d000c460 tx=0x7f18d000c920 comp rx=0 tx=0).stop 2024-04-25T00:09:24.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.803+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 shutdown_connections 2024-04-25T00:09:24.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.803+0000 7f18e222a640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f18c806be30 0x7f18c806e2f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.803+0000 7f18e222a640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f18dc0cc0b0 0x7f18dc145920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.803+0000 7f18e222a640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18dc0b47a0 0x7f18dc0a9480 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.803+0000 7f18e222a640 1 --2- 172.21.15.113:0/111807642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18dc0b3f80 0x7f18dc15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.803+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 >> 172.21.15.113:0/111807642 conn(0x7f18dc0b1b30 msgr2=0x7f18dc153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:24.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.804+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 shutdown_connections 2024-04-25T00:09:24.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.804+0000 7f18e222a640 1 -- 172.21.15.113:0/111807642 wait complete. 2024-04-25T00:09:24.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:09:24.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.920+0000 7fb24adce640 1 Processor -- start 2024-04-25T00:09:24.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.921+0000 7fb24adce640 1 -- start start 2024-04-25T00:09:24.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.921+0000 7fb24adce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 0x7fb2440b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.921+0000 7fb24adce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2441626f0 con 0x7fb2440b61e0 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.921+0000 7fb24adce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2441629d0 con 0x7fb24415e510 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.921+0000 7fb24adce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb244162cb0 con 0x7fb2440b9bb0 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.921+0000 7fb24936b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb2440b61e0 0x7fb2440d2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36782/0 (socket says 172.21.15.113:36782) 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.921+0000 7fb24936b640 1 -- 172.21.15.113:0/2800308207 learned_addr learned my addr 172.21.15.113:0/2800308207 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868845781 0 0) 0x7fb2441626f0 con 0x7fb2440b61e0 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb22c003520 con 0x7fb2440b61e0 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23bfff640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 0x7fb2440b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3857510365 0 0) 0x7fb22c003520 con 0x7fb2440b61e0 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 msgr2=0x7fb2440b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 0x7fb2440b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 >> v1:172.21.15.178:6789/0 conn(0x7fb24415e510 legacy=0x7fb244161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2440d2660 con 0x7fb2440b61e0 2024-04-25T00:09:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fb23c0019f0 con 0x7fb2440b61e0 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 >> v1:172.21.15.113:6789/0 conn(0x7fb2440b61e0 legacy=0x7fb2440d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb22c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 0x7fb22c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22c004d40 0x7fb22c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb244162cb0 con 0x7fb22c0039e0 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c00baf0 con 0x7fb22c004d40 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb22c00bcf0 con 0x7fb2440b9bb0 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb248b6a640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb22c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb24936b640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22c004d40 0x7fb22c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb23bfff640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 0x7fb22c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb248b6a640 1 -- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 msgr2=0x7fb22c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb248b6a640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2440b9bb0 0x7fb22c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb248b6a640 1 -- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22c004d40 msgr2=0x7fb22c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb248b6a640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb22c004d40 0x7fb22c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.922+0000 7fb248b6a640 1 -- 172.21.15.113:0/2800308207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c003520 con 0x7fb22c0039e0 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.923+0000 7fb248b6a640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb22c003e00 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fb240001750 tx=0x7fb22c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6516a1cd4d585c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.923+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240001370 con 0x7fb22c0039e0 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.923+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2400052d0 con 0x7fb22c0039e0 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.923+0000 7fb23b7fe640 1 -- 172.21.15.113:0/2800308207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24002ab20 con 0x7fb22c0039e0 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.924+0000 7fb24adce640 1 -- 172.21.15.113:0/2800308207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 msgr2=0x7fb22c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.924+0000 7fb24adce640 1 --2- 172.21.15.113:0/2800308207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb22c003e00 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fb240001750 tx=0x7fb22c005830 comp rx=0 tx=0).stop 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.924+0000 7fb24936b640 1 -- 172.21.15.113:0/2800308207 reap_dead start 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.924+0000 7fb24adce640 1 -- 172.21.15.113:0/2800308207 shutdown_connections 2024-04-25T00:09:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.924+0000 7fb24adce640 1 -- 172.21.15.113:0/2800308207 >> 172.21.15.113:0/2800308207 conn(0x7fb24406b720 msgr2=0x7fb2440d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.924+0000 7fb24adce640 1 -- 172.21.15.113:0/2800308207 shutdown_connections 2024-04-25T00:09:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.924+0000 7fb24adce640 1 -- 172.21.15.113:0/2800308207 wait complete. 2024-04-25T00:09:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.925+0000 7fb24adce640 1 Processor -- start 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.925+0000 7fb24adce640 1 -- start start 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24adce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb2440c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24adce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb244155770 0x7fb24414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24adce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2441f4290 0x7fb2441f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24adce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2440b7540 con 0x7fb22c0039e0 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24adce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2440b73c0 con 0x7fb2441f4290 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24adce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb2440b5ca0 con 0x7fb244155770 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb2440c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb2440c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35104/0 (socket says 172.21.15.113:35104) 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 -- 172.21.15.113:0/779751305 learned_addr learned my addr 172.21.15.113:0/779751305 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24936b640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2441f4290 0x7fb2441f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb23bfff640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb244155770 0x7fb24414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 -- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb244155770 msgr2=0x7fb24414b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb244155770 0x7fb24414b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 -- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2441f4290 msgr2=0x7fb2441f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2441f4290 0x7fb2441f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 -- 172.21.15.113:0/779751305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2440c9470 con 0x7fb22c0039e0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24936b640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2441f4290 0x7fb2441f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb23bfff640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb244155770 0x7fb24414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb248b6a640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb2440c7940 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fb240008fd0 tx=0x7fb240001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb2397fa640 1 -- 172.21.15.113:0/779751305 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240037040 con 0x7fb22c0039e0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb2397fa640 1 -- 172.21.15.113:0/779751305 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb240005480 con 0x7fb22c0039e0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2440ccb20 con 0x7fb22c0039e0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.926+0000 7fb2397fa640 1 -- 172.21.15.113:0/779751305 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240007470 con 0x7fb22c0039e0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.927+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2440cd0e0 con 0x7fb22c0039e0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.927+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2440691a0 con 0x7fb22c0039e0 2024-04-25T00:09:24.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.928+0000 7fb2397fa640 1 -- 172.21.15.113:0/779751305 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb240044020 con 0x7fb22c0039e0 2024-04-25T00:09:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.928+0000 7fb2397fa640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb23006be60 0x7fb23006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.928+0000 7fb2397fa640 1 -- 172.21.15.113:0/779751305 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5357+0+0 (secure 0 0 0) 0x7fb2400aa8c0 con 0x7fb22c0039e0 2024-04-25T00:09:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.928+0000 7fb23bfff640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb23006be60 0x7fb23006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.929+0000 7fb23bfff640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb23006be60 0x7fb23006e320 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb2440cc420 tx=0x7fb234008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:24.933+0000 7fb2397fa640 1 -- 172.21.15.113:0/779751305 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb240079580 con 0x7fb22c0039e0 2024-04-25T00:09:25.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.105+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb2440d1f30 con 0x7fb22c0039e0 2024-04-25T00:09:25.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.803+0000 7fb2397fa640 1 -- 172.21.15.113:0/779751305 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) ==== 169+0+0 (secure 0 0 0) 0x7fb240078e80 con 0x7fb22c0039e0 2024-04-25T00:09:25.804 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:09:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.807+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb23006be60 msgr2=0x7fb23006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.807+0000 7fb24adce640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb23006be60 0x7fb23006e320 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb2440cc420 tx=0x7fb234008040 comp rx=0 tx=0).stop 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.807+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 msgr2=0x7fb2440c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.807+0000 7fb24adce640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb2440c7940 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fb240008fd0 tx=0x7fb240001120 comp rx=0 tx=0).stop 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.808+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 shutdown_connections 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.809+0000 7fb24adce640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb23006be60 0x7fb23006e320 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.809+0000 7fb24adce640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2441f4290 0x7fb2441f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.809+0000 7fb24adce640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb244155770 0x7fb24414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.809+0000 7fb24adce640 1 --2- 172.21.15.113:0/779751305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb22c0039e0 0x7fb2440c7940 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.809+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 >> 172.21.15.113:0/779751305 conn(0x7fb24406b720 msgr2=0x7fb244159610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.809+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 shutdown_connections 2024-04-25T00:09:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.809+0000 7fb24adce640 1 -- 172.21.15.113:0/779751305 wait complete. 2024-04-25T00:09:25.829 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:09:25.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.931+0000 7f2ba8c38640 1 Processor -- start 2024-04-25T00:09:25.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.931+0000 7f2ba8c38640 1 -- start start 2024-04-25T00:09:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba8c38640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2ba4156600 0x7f2ba415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba8c38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba40cf3f0 0x7f2ba40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba8c38640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba406bdc0 con 0x7f2ba406b800 2024-04-25T00:09:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba8c38640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba40b2280 con 0x7f2ba4156600 2024-04-25T00:09:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba8c38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba40c5700 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba259c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2ba4156600 0x7f2ba415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba1d9b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2ba406b800 0x7f2ba406d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:33912/0 (socket says 172.21.15.113:33912) 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba1d9b640 1 -- 172.21.15.113:0/3685967499 learned_addr learned my addr 172.21.15.113:0/3685967499 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.932+0000 7f2ba2d9d640 1 --2- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba40cf3f0 0x7f2ba40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.933+0000 7f2ba259c640 1 -- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba40cf3f0 msgr2=0x7f2ba40cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.933+0000 7f2ba259c640 1 --2- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba40cf3f0 0x7f2ba40cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.933+0000 7f2ba259c640 1 -- 172.21.15.113:0/3685967499 >> v1:172.21.15.178:6789/0 conn(0x7f2ba406b800 legacy=0x7f2ba406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.933+0000 7f2ba259c640 1 -- 172.21.15.113:0/3685967499 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba40c59e0 con 0x7f2ba4156600 2024-04-25T00:09:25.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.933+0000 7f2ba159a640 1 -- 172.21.15.113:0/3685967499 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1103452299 0 0) 0x7f2ba406bdc0 con 0x7f2ba406b800 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.933+0000 7f2ba259c640 1 --2- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 0x7f2ba415d0b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2b940097a0 tx=0x7f2b9402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f41e803a2c93a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba159a640 1 -- 172.21.15.113:0/3685967499 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94037070 con 0x7f2ba4156600 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba159a640 1 -- 172.21.15.113:0/3685967499 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b94032440 con 0x7f2ba4156600 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba159a640 1 -- 172.21.15.113:0/3685967499 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94032810 con 0x7f2ba4156600 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 -- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 msgr2=0x7f2ba415d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 0x7f2ba415d0b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2b940097a0 tx=0x7f2b9402a180 comp rx=0 tx=0).stop 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 -- 172.21.15.113:0/3685967499 shutdown_connections 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba40cf3f0 0x7f2ba40cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/3685967499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 0x7f2ba415d0b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 -- 172.21.15.113:0/3685967499 >> 172.21.15.113:0/3685967499 conn(0x7f2ba406fb60 msgr2=0x7f2ba406b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:25.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 -- 172.21.15.113:0/3685967499 shutdown_connections 2024-04-25T00:09:25.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.934+0000 7f2ba8c38640 1 -- 172.21.15.113:0/3685967499 wait complete. 2024-04-25T00:09:25.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.935+0000 7f2ba8c38640 1 Processor -- start 2024-04-25T00:09:25.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba8c38640 1 -- start start 2024-04-25T00:09:25.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba8c38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba406b800 0x7f2ba4152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba8c38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40cf3f0 0x7f2ba414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba8c38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 0x7f2ba40c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba8c38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba415fff0 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba8c38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41569e0 con 0x7f2ba4156600 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba8c38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba4160170 con 0x7f2ba406b800 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40cf3f0 0x7f2ba414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40cf3f0 0x7f2ba414ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35120/0 (socket says 172.21.15.113:35120) 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba1d9b640 1 -- 172.21.15.113:0/2895314735 learned_addr learned my addr 172.21.15.113:0/2895314735 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba259c640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba406b800 0x7f2ba4152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.936+0000 7f2ba2d9d640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 0x7f2ba40c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba1d9b640 1 -- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba406b800 msgr2=0x7f2ba4152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba1d9b640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba406b800 0x7f2ba4152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba1d9b640 1 -- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 msgr2=0x7f2ba40c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba1d9b640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 0x7f2ba40c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba1d9b640 1 -- 172.21.15.113:0/2895314735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba41f5ac0 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba1d9b640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40cf3f0 0x7f2ba414ecf0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c500 tx=0x7f2b9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2b92ffd640 1 -- 172.21.15.113:0/2895314735 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98018070 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2b92ffd640 1 -- 172.21.15.113:0/2895314735 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b98007580 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba41f5da0 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2b92ffd640 1 -- 172.21.15.113:0/2895314735 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98004a20 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.937+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba41f6360 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.938+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba40691a0 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.938+0000 7f2b92ffd640 1 -- 172.21.15.113:0/2895314735 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b98004bc0 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.939+0000 7f2b92ffd640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b8406be30 0x7f2b8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.939+0000 7f2b92ffd640 1 -- 172.21.15.113:0/2895314735 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4943+0+0 (secure 0 0 0) 0x7f2b98089db0 con 0x7f2ba40cf3f0 2024-04-25T00:09:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.939+0000 7f2ba259c640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b8406be30 0x7f2b8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.939+0000 7f2ba259c640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b8406be30 0x7f2b8406e2f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2b94034000 tx=0x7f2b9402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:25.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:25.943+0000 7f2b92ffd640 1 -- 172.21.15.113:0/2895314735 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b98058420 con 0x7f2ba40cf3f0 2024-04-25T00:09:26.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.115+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2ba4152650 con 0x7f2ba40cf3f0 2024-04-25T00:09:26.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.118+0000 7f2b92ffd640 1 -- 172.21.15.113:0/2895314735 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f2b9805c720 con 0x7f2ba40cf3f0 2024-04-25T00:09:26.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b8406be30 msgr2=0x7f2b8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b8406be30 0x7f2b8406e2f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2b94034000 tx=0x7f2b9402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40cf3f0 msgr2=0x7f2ba414ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40cf3f0 0x7f2ba414ecf0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c500 tx=0x7f2b9800c9c0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 shutdown_connections 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2b8406be30 0x7f2b8406e2f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ba4156600 0x7f2ba40c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba40cf3f0 0x7f2ba414ecf0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 --2- 172.21.15.113:0/2895314735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ba406b800 0x7f2ba4152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 >> 172.21.15.113:0/2895314735 conn(0x7f2ba406fb60 msgr2=0x7f2ba406df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 shutdown_connections 2024-04-25T00:09:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.120+0000 7f2ba8c38640 1 -- 172.21.15.113:0/2895314735 wait complete. 2024-04-25T00:09:26.131 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T00:09:26.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.232+0000 7efd4f831640 1 Processor -- start 2024-04-25T00:09:26.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.232+0000 7efd4f831640 1 -- start start 2024-04-25T00:09:26.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4f831640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd4815e320 0x7efd480d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4f831640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 0x7efd480b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4f831640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd480d3de0 con 0x7efd480d0b00 2024-04-25T00:09:26.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4f831640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efd480d3410 con 0x7efd4815e320 2024-04-25T00:09:26.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4f831640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd480d2800 con 0x7efd480d2290 2024-04-25T00:09:26.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4d5cd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd4815e320 0x7efd480d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4d5cd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd4815e320 0x7efd480d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35122/0 (socket says 172.21.15.113:35122) 2024-04-25T00:09:26.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.233+0000 7efd4d5cd640 1 -- 172.21.15.113:0/1778924480 learned_addr learned my addr 172.21.15.113:0/1778924480 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:26.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd4d5cd640 1 -- 172.21.15.113:0/1778924480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 msgr2=0x7efd480b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:26.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd4d5cd640 1 --2- 172.21.15.113:0/1778924480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 0x7efd480b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd4d5cd640 1 -- 172.21.15.113:0/1778924480 >> v1:172.21.15.178:6789/0 conn(0x7efd480d0b00 legacy=0x7efd480b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:26.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd4d5cd640 1 -- 172.21.15.113:0/1778924480 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd480d2ae0 con 0x7efd4815e320 2024-04-25T00:09:26.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd4d5cd640 1 --2- 172.21.15.113:0/1778924480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480d38a0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7efd44009870 tx=0x7efd4402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2606e798871597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:26.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd3ffff640 1 -- 172.21.15.113:0/1778924480 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44037070 con 0x7efd4815e320 2024-04-25T00:09:26.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd3ffff640 1 -- 172.21.15.113:0/1778924480 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd44032440 con 0x7efd4815e320 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.234+0000 7efd3ffff640 1 -- 172.21.15.113:0/1778924480 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44032810 con 0x7efd4815e320 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 -- 172.21.15.113:0/1778924480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 msgr2=0x7efd480d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 --2- 172.21.15.113:0/1778924480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480d38a0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7efd44009870 tx=0x7efd4402a2a0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 -- 172.21.15.113:0/1778924480 shutdown_connections 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 --2- 172.21.15.113:0/1778924480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 0x7efd480b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 --2- 172.21.15.113:0/1778924480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480d38a0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 -- 172.21.15.113:0/1778924480 >> 172.21.15.113:0/1778924480 conn(0x7efd480b5c20 msgr2=0x7efd480b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 -- 172.21.15.113:0/1778924480 shutdown_connections 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.235+0000 7efd4f831640 1 -- 172.21.15.113:0/1778924480 wait complete. 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.236+0000 7efd4f831640 1 Processor -- start 2024-04-25T00:09:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.236+0000 7efd4f831640 1 -- start start 2024-04-25T00:09:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4f831640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd480d0b00 0x7efd48161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4f831640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 0x7efd48154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4f831640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4f831640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd480b9fe0 con 0x7efd4815e320 2024-04-25T00:09:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4f831640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd480d30a0 con 0x7efd480d0b00 2024-04-25T00:09:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4f831640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd4815e700 con 0x7efd480d2290 2024-04-25T00:09:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35124/0 (socket says 172.21.15.113:35124) 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 -- 172.21.15.113:0/676411657 learned_addr learned my addr 172.21.15.113:0/676411657 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 -- 172.21.15.113:0/676411657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 msgr2=0x7efd48154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 0x7efd48154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 -- 172.21.15.113:0/676411657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd480d0b00 msgr2=0x7efd48161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd480d0b00 0x7efd48161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 -- 172.21.15.113:0/676411657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd48156a10 con 0x7efd4815e320 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.237+0000 7efd4ddce640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480b45c0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c780 tx=0x7efd4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.238+0000 7efd3dffb640 1 -- 172.21.15.113:0/676411657 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40004b60 con 0x7efd4815e320 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.238+0000 7efd3dffb640 1 -- 172.21.15.113:0/676411657 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd40004d00 con 0x7efd4815e320 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.238+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd48156cd0 con 0x7efd4815e320 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.238+0000 7efd3dffb640 1 -- 172.21.15.113:0/676411657 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4000aa90 con 0x7efd4815e320 2024-04-25T00:09:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.238+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd48157290 con 0x7efd4815e320 2024-04-25T00:09:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.239+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd480691a0 con 0x7efd4815e320 2024-04-25T00:09:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.239+0000 7efd3dffb640 1 -- 172.21.15.113:0/676411657 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd4000ac30 con 0x7efd4815e320 2024-04-25T00:09:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.239+0000 7efd3dffb640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd2c06be60 0x7efd2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.239+0000 7efd3dffb640 1 -- 172.21.15.113:0/676411657 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4943+0+0 (secure 0 0 0) 0x7efd4008ae40 con 0x7efd4815e320 2024-04-25T00:09:26.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.239+0000 7efd4d5cd640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd2c06be60 0x7efd2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.240+0000 7efd4d5cd640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd2c06be60 0x7efd2c06e320 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efd44034000 tx=0x7efd4402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.244+0000 7efd3dffb640 1 -- 172.21.15.113:0/676411657 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd40020070 con 0x7efd4815e320 2024-04-25T00:09:26.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.491+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) -- 0x7efd48157690 con 0x7efd4815e320 2024-04-25T00:09:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.804+0000 7efd3dffb640 1 -- 172.21.15.113:0/676411657 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v544) ==== 147+0+0 (secure 0 0 0) 0x7efd40059af0 con 0x7efd4815e320 2024-04-25T00:09:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd2c06be60 msgr2=0x7efd2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd2c06be60 0x7efd2c06e320 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efd44034000 tx=0x7efd4402c000 comp rx=0 tx=0).stop 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 msgr2=0x7efd480b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480b45c0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c780 tx=0x7efd4000cc40 comp rx=0 tx=0).stop 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 shutdown_connections 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efd2c06be60 0x7efd2c06e320 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd4815e320 0x7efd480b45c0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd480d2290 0x7efd48154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 --2- 172.21.15.113:0/676411657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd480d0b00 0x7efd48161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 >> 172.21.15.113:0/676411657 conn(0x7efd480b5c20 msgr2=0x7efd480b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 shutdown_connections 2024-04-25T00:09:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.807+0000 7efd4f831640 1 -- 172.21.15.113:0/676411657 wait complete. 2024-04-25T00:09:26.819 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T00:09:26.820 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T00:09:26.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.919+0000 7f1ebf0c3640 1 Processor -- start 2024-04-25T00:09:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.919+0000 7f1ebf0c3640 1 -- start start 2024-04-25T00:09:26.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.920+0000 7f1ebf0c3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 0x7f1eb80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.920+0000 7f1ebf0c3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1eb80d0b00 0x7f1eb80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.920+0000 7f1ebf0c3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb80d2800 con 0x7f1eb815e320 2024-04-25T00:09:26.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.920+0000 7f1ebf0c3640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb80d14d0 con 0x7f1eb80d0b00 2024-04-25T00:09:26.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.920+0000 7f1ebf0c3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb80d3410 con 0x7f1eb80d2290 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.920+0000 7f1ebce5f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1eb815e320 0x7f1eb80d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36792/0 (socket says 172.21.15.113:36792) 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.920+0000 7f1ebce5f640 1 -- 172.21.15.113:0/1032510665 learned_addr learned my addr 172.21.15.113:0/1032510665 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb7fff640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 0x7f1eb80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383969229 0 0) 0x7f1eb80d2800 con 0x7f1eb815e320 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebd660640 1 --2- 172.21.15.113:0/1032510665 >> v2:172.21.15.178:3300/0 conn(0x7f1eb80d0b00 0x7f1eb80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ea0003520 con 0x7f1eb815e320 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 234994045 0 0) 0x7f1ea0003520 con 0x7f1eb815e320 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 msgr2=0x7f1eb80b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 0x7f1eb80b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 >> v2:172.21.15.178:3300/0 conn(0x7f1eb80d0b00 msgr2=0x7f1eb80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 --2- 172.21.15.113:0/1032510665 >> v2:172.21.15.178:3300/0 conn(0x7f1eb80d0b00 0x7f1eb80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb80d2ae0 con 0x7f1eb815e320 2024-04-25T00:09:26.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f1ea8001670 con 0x7f1eb815e320 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 >> v1:172.21.15.113:6789/0 conn(0x7f1eb815e320 legacy=0x7f1eb80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea00039b0 0x7f1ea0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ea0004800 0x7f1ea0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 0x7f1ea00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80d14d0 con 0x7f1ea00039b0 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80d3410 con 0x7f1ea0004800 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea000b9c0 con 0x7f1eb80d2290 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebce5f640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea00039b0 0x7f1ea0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebd660640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 0x7f1ea00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1eb7fff640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ea0004800 0x7f1ea0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebce5f640 1 -- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 msgr2=0x7f1ea00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebce5f640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb80d2290 0x7f1ea00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebce5f640 1 -- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ea0004800 msgr2=0x7f1ea0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:26.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebce5f640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ea0004800 0x7f1ea0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.921+0000 7f1ebce5f640 1 -- 172.21.15.113:0/1032510665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea0003520 con 0x7f1ea00039b0 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.922+0000 7f1ebce5f640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea00039b0 0x7f1ea0003dd0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c210 tx=0x7f1ea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6763ce29755a86cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.922+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8007d50 con 0x7f1ea00039b0 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.922+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea8005ad0 con 0x7f1ea00039b0 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.922+0000 7f1eb77fe640 1 -- 172.21.15.113:0/1032510665 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8007070 con 0x7f1ea00039b0 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.922+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/1032510665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea00039b0 msgr2=0x7f1ea0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.922+0000 7f1ebf0c3640 1 --2- 172.21.15.113:0/1032510665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea00039b0 0x7f1ea0003dd0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c210 tx=0x7f1ea0005830 comp rx=0 tx=0).stop 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.923+0000 7f1ebd660640 1 -- 172.21.15.113:0/1032510665 reap_dead start 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.923+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/1032510665 shutdown_connections 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.923+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/1032510665 >> 172.21.15.113:0/1032510665 conn(0x7f1eb80b5c20 msgr2=0x7f1eb80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.923+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/1032510665 shutdown_connections 2024-04-25T00:09:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.923+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/1032510665 wait complete. 2024-04-25T00:09:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.924+0000 7f1ebf0c3640 1 Processor -- start 2024-04-25T00:09:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.924+0000 7f1ebf0c3640 1 -- start start 2024-04-25T00:09:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ea00039b0 0x7f1eb81554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb81f3520 0x7f1eb81614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb80b8a80 0x7f1eb80b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb815e700 con 0x7f1eb80b8a80 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80d30a0 con 0x7f1eb81f3520 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb815e880 con 0x7f1ea00039b0 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb80b8a80 0x7f1eb80b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb80b8a80 0x7f1eb80b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35138/0 (socket says 172.21.15.113:35138) 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 -- 172.21.15.113:0/2131055423 learned_addr learned my addr 172.21.15.113:0/2131055423 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebce5f640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ea00039b0 0x7f1eb81554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1eb7fff640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb81f3520 0x7f1eb81614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 -- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ea00039b0 msgr2=0x7f1eb81554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ea00039b0 0x7f1eb81554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 -- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb81f3520 msgr2=0x7f1eb81614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb81f3520 0x7f1eb81614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 -- 172.21.15.113:0/2131055423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb8161a00 con 0x7f1eb80b8a80 2024-04-25T00:09:26.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebd660640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb80b8a80 0x7f1eb80b8f00 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009e90 tx=0x7f1eb00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1eb57fa640 1 -- 172.21.15.113:0/2131055423 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0011e20 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1eb57fa640 1 -- 172.21.15.113:0/2131055423 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb001b070 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb8155c10 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1eb57fa640 1 -- 172.21.15.113:0/2131055423 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003990 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.925+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb81561d0 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.926+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb80b9440 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.927+0000 7f1eb57fa640 1 -- 172.21.15.113:0/2131055423 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1eb0005d10 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.927+0000 7f1eb57fa640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ea406be30 0x7f1ea406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.927+0000 7f1eb57fa640 1 -- 172.21.15.113:0/2131055423 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4943+0+0 (secure 0 0 0) 0x7f1eb008aed0 con 0x7f1eb80b8a80 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.927+0000 7f1ebce5f640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ea406be30 0x7f1ea406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:26.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.928+0000 7f1ebce5f640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ea406be30 0x7f1ea406e2f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8033000 tx=0x7f1ea8009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:26.932+0000 7f1eb57fa640 1 -- 172.21.15.113:0/2131055423 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1eb0025020 con 0x7f1eb80b8a80 2024-04-25T00:09:27.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.111+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1eb80691a0 con 0x7f1eb80b8a80 2024-04-25T00:09:27.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.112+0000 7f1eb57fa640 1 -- 172.21.15.113:0/2131055423 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+13080 (secure 0 0 0) 0x7f1eb0059530 con 0x7f1eb80b8a80 2024-04-25T00:09:27.113 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:27.113 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":173,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:25.799770+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/2813561857":"2024-04-26T00:09:22.791782+0000","172.21.15.178:6833/2813561857":"2024-04-26T00:09:22.791782+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ea406be30 msgr2=0x7f1ea406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ea406be30 0x7f1ea406e2f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8033000 tx=0x7f1ea8009030 comp rx=0 tx=0).stop 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb80b8a80 msgr2=0x7f1eb80b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb80b8a80 0x7f1eb80b8f00 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009e90 tx=0x7f1eb00115a0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 shutdown_connections 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ea406be30 0x7f1ea406e2f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb81f3520 0x7f1eb81614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb80b8a80 0x7f1eb80b8f00 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 --2- 172.21.15.113:0/2131055423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ea00039b0 0x7f1eb81554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 >> 172.21.15.113:0/2131055423 conn(0x7f1eb80b5c20 msgr2=0x7f1eb80b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 shutdown_connections 2024-04-25T00:09:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.114+0000 7f1ebf0c3640 1 -- 172.21.15.113:0/2131055423 wait complete. 2024-04-25T00:09:27.126 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:09:27.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:09:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.225+0000 7f061989e640 1 Processor -- start 2024-04-25T00:09:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.225+0000 7f061989e640 1 -- start start 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.225+0000 7f061989e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f061415e310 0x7f06140d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.225+0000 7f061989e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f06140b1e00 0x7f06140716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.225+0000 7f061989e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0614072040 con 0x7f0614071c20 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.225+0000 7f061989e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f061406e9a0 con 0x7f061415e310 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.225+0000 7f061989e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f061406e510 con 0x7f06140b1e00 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f0612ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f061415e310 0x7f06140d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f0612ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f061415e310 0x7f06140d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35152/0 (socket says 172.21.15.113:35152) 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f0612ffd640 1 -- 172.21.15.113:0/3399878730 learned_addr learned my addr 172.21.15.113:0/3399878730 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f06127fc640 1 --2- 172.21.15.113:0/3399878730 >> v2:172.21.15.178:3300/0 conn(0x7f06140b1e00 0x7f06140716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f0612ffd640 1 -- 172.21.15.113:0/3399878730 >> v2:172.21.15.178:3300/0 conn(0x7f06140b1e00 msgr2=0x7f06140716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f0612ffd640 1 --2- 172.21.15.113:0/3399878730 >> v2:172.21.15.178:3300/0 conn(0x7f06140b1e00 0x7f06140716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f0612ffd640 1 -- 172.21.15.113:0/3399878730 >> v1:172.21.15.178:6789/0 conn(0x7f0614071c20 legacy=0x7f0614161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f0612ffd640 1 -- 172.21.15.113:0/3399878730 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061406d900 con 0x7f061415e310 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.226+0000 7f06127fc640 1 --2- 172.21.15.113:0/3399878730 >> v2:172.21.15.178:3300/0 conn(0x7f06140b1e00 0x7f06140716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f0612ffd640 1 --2- 172.21.15.113:0/3399878730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061415e310 0x7f06140d00e0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f06140b5be0 tx=0x7f060402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedd401703c3b054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f0611ffb640 1 -- 172.21.15.113:0/3399878730 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604037070 con 0x7f061415e310 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f0611ffb640 1 -- 172.21.15.113:0/3399878730 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f060402ae20 con 0x7f061415e310 2024-04-25T00:09:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f0611ffb640 1 -- 172.21.15.113:0/3399878730 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604032660 con 0x7f061415e310 2024-04-25T00:09:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 -- 172.21.15.113:0/3399878730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061415e310 msgr2=0x7f06140d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 --2- 172.21.15.113:0/3399878730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061415e310 0x7f06140d00e0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f06140b5be0 tx=0x7f060402a110 comp rx=0 tx=0).stop 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 -- 172.21.15.113:0/3399878730 shutdown_connections 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 --2- 172.21.15.113:0/3399878730 >> v2:172.21.15.178:3300/0 conn(0x7f06140b1e00 0x7f06140716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 --2- 172.21.15.113:0/3399878730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061415e310 0x7f06140d00e0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 -- 172.21.15.113:0/3399878730 >> 172.21.15.113:0/3399878730 conn(0x7f061406fb60 msgr2=0x7f061406b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 -- 172.21.15.113:0/3399878730 shutdown_connections 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.227+0000 7f061989e640 1 -- 172.21.15.113:0/3399878730 wait complete. 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.228+0000 7f061989e640 1 Processor -- start 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f061989e640 1 -- start start 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f061989e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0614071c20 0x7f061414f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f061989e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06140b1e00 0x7f061415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f061989e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f061415e310 0x7f061415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f061989e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f061406e1a0 con 0x7f0614071c20 2024-04-25T00:09:27.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f061989e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f061406dec0 con 0x7f061415e310 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f061989e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06140ceb20 con 0x7f06140b1e00 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0614071c20 0x7f061414f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0614071c20 0x7f061414f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35158/0 (socket says 172.21.15.113:35158) 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f0612ffd640 1 -- 172.21.15.113:0/1871769412 learned_addr learned my addr 172.21.15.113:0/1871769412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f06127fc640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06140b1e00 0x7f061415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.229+0000 7f06137fe640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f061415e310 0x7f061415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f0612ffd640 1 -- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06140b1e00 msgr2=0x7f061415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f0612ffd640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06140b1e00 0x7f061415a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f0612ffd640 1 -- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f061415e310 msgr2=0x7f061415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f0612ffd640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f061415e310 0x7f061415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f0612ffd640 1 -- 172.21.15.113:0/1871769412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06141541c0 con 0x7f0614071c20 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f06127fc640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06140b1e00 0x7f061415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:27.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f06137fe640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f061415e310 0x7f061415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f0612ffd640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0614071c20 0x7f061414f040 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f0604000c00 tx=0x7f06040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f05fbfff640 1 -- 172.21.15.113:0/1871769412 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604041070 con 0x7f0614071c20 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f05fbfff640 1 -- 172.21.15.113:0/1871769412 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f060403c680 con 0x7f0614071c20 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0614154460 con 0x7f0614071c20 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f05fbfff640 1 -- 172.21.15.113:0/1871769412 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604037040 con 0x7f0614071c20 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.230+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0614154a20 con 0x7f0614071c20 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.231+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06140691a0 con 0x7f0614071c20 2024-04-25T00:09:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.231+0000 7f05fbfff640 1 -- 172.21.15.113:0/1871769412 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0604044070 con 0x7f0614071c20 2024-04-25T00:09:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.232+0000 7f05fbfff640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f060006be30 0x7f060006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.232+0000 7f05fbfff640 1 -- 172.21.15.113:0/1871769412 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4943+0+0 (secure 0 0 0) 0x7f06040ab390 con 0x7f0614071c20 2024-04-25T00:09:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.232+0000 7f06127fc640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f060006be30 0x7f060006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.232+0000 7f06127fc640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f060006be30 0x7f060006e2f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f061415b360 tx=0x7f0608008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.236+0000 7f05fbfff640 1 -- 172.21.15.113:0/1871769412 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f060407a040 con 0x7f0614071c20 2024-04-25T00:09:27.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.407+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0614158a50 con 0x7f0614071c20 2024-04-25T00:09:27.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.407+0000 7f05fbfff640 1 -- 172.21.15.113:0/1871769412 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) ==== 93+0+10 (secure 0 0 0) 0x7f060402ae20 con 0x7f0614071c20 2024-04-25T00:09:27.408 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f060006be30 msgr2=0x7f060006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f060006be30 0x7f060006e2f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f061415b360 tx=0x7f0608008040 comp rx=0 tx=0).stop 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0614071c20 msgr2=0x7f061414f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0614071c20 0x7f061414f040 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f0604000c00 tx=0x7f06040046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 shutdown_connections 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f060006be30 0x7f060006e2f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f061415e310 0x7f061415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06140b1e00 0x7f061415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 --2- 172.21.15.113:0/1871769412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0614071c20 0x7f061414f040 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 >> 172.21.15.113:0/1871769412 conn(0x7f061406fb60 msgr2=0x7f06140b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 shutdown_connections 2024-04-25T00:09:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.409+0000 7f061989e640 1 -- 172.21.15.113:0/1871769412 wait complete. 2024-04-25T00:09:27.421 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T00:09:27.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.517+0000 7f03a388c640 1 Processor -- start 2024-04-25T00:09:27.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.517+0000 7f03a388c640 1 -- start start 2024-04-25T00:09:27.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f03a388c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f039c15a6b0 0x7f039c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f03a388c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c0b2d50 con 0x7f039c072ea0 2024-04-25T00:09:27.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f03a388c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c072a70 con 0x7f039c0b3af0 2024-04-25T00:09:27.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f03a388c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f039c0b3010 con 0x7f039c15a6b0 2024-04-25T00:09:27.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f03a0e27640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f039c072ea0 0x7f039c071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38398/0 (socket says 172.21.15.113:38398) 2024-04-25T00:09:27.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f03a0e27640 1 -- 172.21.15.113:0/3200346453 learned_addr learned my addr 172.21.15.113:0/3200346453 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:27.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f03a1628640 1 --2- 172.21.15.113:0/3200346453 >> v2:172.21.15.178:3300/0 conn(0x7f039c15a6b0 0x7f039c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333152377 0 0) 0x7f039c0b2d50 con 0x7f039c072ea0 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0388004630 con 0x7f039c072ea0 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 941012903 0 0) 0x7f0388004630 con 0x7f039c072ea0 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 >> v2:172.21.15.178:3300/0 conn(0x7f039c15a6b0 msgr2=0x7f039c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 --2- 172.21.15.113:0/3200346453 >> v2:172.21.15.178:3300/0 conn(0x7f039c15a6b0 0x7f039c070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 >> v1:172.21.15.178:6789/0 conn(0x7f039c0b3af0 legacy=0x7f039c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0b32f0 con 0x7f039c072ea0 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f038c001670 con 0x7f039c072ea0 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 >> v1:172.21.15.113:6789/0 conn(0x7f039c072ea0 legacy=0x7f039c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0388004ac0 0x7f0388004ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03880054c0 0x7f0388005990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388006190 0x7f038800cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0b3010 con 0x7f0388004ac0 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f038800d0f0 con 0x7f03880054c0 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.518+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f038800d2f0 con 0x7f0388006190 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1628640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0388004ac0 0x7f0388004ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a0e27640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03880054c0 0x7f0388005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1e29640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388006190 0x7f038800cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1628640 1 -- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388006190 msgr2=0x7f038800cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1628640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388006190 0x7f038800cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1628640 1 -- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03880054c0 msgr2=0x7f0388005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1628640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03880054c0 0x7f0388005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1628640 1 -- 172.21.15.113:0/3200346453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0388004630 con 0x7f0388004ac0 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1e29640 1 -- 172.21.15.113:0/3200346453 reap_dead start 2024-04-25T00:09:27.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a0e27640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03880054c0 0x7f0388005990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.519+0000 7f03a1628640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0388004ac0 0x7f0388004ea0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f0398009ac0 tx=0x7f0388006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705fca0c001646fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398036070 con 0x7f0388004ac0 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0398031400 con 0x7f0388004ac0 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f0393fff640 1 -- 172.21.15.113:0/3200346453 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398009690 con 0x7f0388004ac0 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f03a388c640 1 -- 172.21.15.113:0/3200346453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0388004ac0 msgr2=0x7f0388004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f03a388c640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0388004ac0 0x7f0388004ea0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f0398009ac0 tx=0x7f0388006940 comp rx=0 tx=0).stop 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f03a388c640 1 -- 172.21.15.113:0/3200346453 shutdown_connections 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f03a388c640 1 --2- 172.21.15.113:0/3200346453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0388004ac0 0x7f0388004ea0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f03a388c640 1 -- 172.21.15.113:0/3200346453 >> 172.21.15.113:0/3200346453 conn(0x7f039c0b1a50 msgr2=0x7f039c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f03a388c640 1 -- 172.21.15.113:0/3200346453 shutdown_connections 2024-04-25T00:09:27.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.520+0000 7f03a388c640 1 -- 172.21.15.113:0/3200346453 wait complete. 2024-04-25T00:09:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.521+0000 7f03a388c640 1 Processor -- start 2024-04-25T00:09:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.521+0000 7f03a388c640 1 -- start start 2024-04-25T00:09:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a388c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388004ac0 0x7f039c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a388c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039c14fc90 0x7f039c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a388c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f039c146320 0x7f039c1467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a388c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f039c15aa90 con 0x7f039c14fc90 2024-04-25T00:09:27.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a388c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f039c072720 con 0x7f039c146320 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a388c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f039c15ac10 con 0x7f0388004ac0 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039c14fc90 0x7f039c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039c14fc90 0x7f039c1500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41830/0 (socket says 172.21.15.113:41830) 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 -- 172.21.15.113:0/2574282154 learned_addr learned my addr 172.21.15.113:0/2574282154 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a1e29640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f039c146320 0x7f039c1467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a1628640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388004ac0 0x7f039c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 -- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388004ac0 msgr2=0x7f039c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388004ac0 0x7f039c154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 -- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f039c146320 msgr2=0x7f039c1467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f039c146320 0x7f039c1467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a0e27640 1 -- 172.21.15.113:0/2574282154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c146ce0 con 0x7f039c14fc90 2024-04-25T00:09:27.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.522+0000 7f03a1e29640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f039c146320 0x7f039c1467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.523+0000 7f03a0e27640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039c14fc90 0x7f039c1500f0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e730 tx=0x7f038c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.523+0000 7f0391ffb640 1 -- 172.21.15.113:0/2574282154 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c00c6c0 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.523+0000 7f0391ffb640 1 -- 172.21.15.113:0/2574282154 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c018070 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.523+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c151910 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.523+0000 7f0391ffb640 1 -- 172.21.15.113:0/2574282154 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c00caa0 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.523+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c150b50 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.524+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c0691a0 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.524+0000 7f0391ffb640 1 -- 172.21.15.113:0/2574282154 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f038c00cc40 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.525+0000 7f0391ffb640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f038006be60 0x7f038006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.525+0000 7f0391ffb640 1 -- 172.21.15.113:0/2574282154 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4943+0+0 (secure 0 0 0) 0x7f038c08a810 con 0x7f039c14fc90 2024-04-25T00:09:27.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.525+0000 7f03a1628640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f038006be60 0x7f038006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.525+0000 7f03a1628640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f038006be60 0x7f038006e320 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0398033000 tx=0x7f0398008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.529+0000 7f0391ffb640 1 -- 172.21.15.113:0/2574282154 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f039c146ce0 con 0x7f039c14fc90 2024-04-25T00:09:27.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.768+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) -- 0x7f039c0b2d50 con 0x7f039c14fc90 2024-04-25T00:09:27.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.811+0000 7f0391ffb640 1 -- 172.21.15.113:0/2574282154 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v545) ==== 136+0+0 (secure 0 0 0) 0x7f038c0594c0 con 0x7f039c14fc90 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f038006be60 msgr2=0x7f038006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f038006be60 0x7f038006e320 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0398033000 tx=0x7f0398008030 comp rx=0 tx=0).stop 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039c14fc90 msgr2=0x7f039c1500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039c14fc90 0x7f039c1500f0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e730 tx=0x7f038c00ebf0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 shutdown_connections 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f038006be60 0x7f038006e320 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f039c146320 0x7f039c1467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039c14fc90 0x7f039c1500f0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 --2- 172.21.15.113:0/2574282154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0388004ac0 0x7f039c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 >> 172.21.15.113:0/2574282154 conn(0x7f039c0b1a50 msgr2=0x7f039c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:27.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 shutdown_connections 2024-04-25T00:09:27.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.813+0000 7f03a388c640 1 -- 172.21.15.113:0/2574282154 wait complete. 2024-04-25T00:09:27.825 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:09:27.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.923+0000 7fce78732640 1 Processor -- start 2024-04-25T00:09:27.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.923+0000 7fce78732640 1 -- start start 2024-04-25T00:09:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.924+0000 7fce78732640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce7015e510 0x7fce700d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.924+0000 7fce78732640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce700d2e00 con 0x7fce700b63b0 2024-04-25T00:09:27.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.924+0000 7fce78732640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce700d30e0 con 0x7fce700d28c0 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.924+0000 7fce78732640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fce700d33c0 con 0x7fce7015e510 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.924+0000 7fce76ccf640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fce700b63b0 0x7fce700b73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38402/0 (socket says 172.21.15.113:38402) 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.924+0000 7fce76ccf640 1 -- 172.21.15.113:0/2294039537 learned_addr learned my addr 172.21.15.113:0/2294039537 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce764ce640 1 --2- 172.21.15.113:0/2294039537 >> v2:172.21.15.178:3300/0 conn(0x7fce7015e510 0x7fce700d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394363252 0 0) 0x7fce700d2e00 con 0x7fce700b63b0 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce5c003520 con 0x7fce700b63b0 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 681407591 0 0) 0x7fce700d30e0 con 0x7fce700d28c0 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce700d2e00 con 0x7fce700d28c0 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2550012630 0 0) 0x7fce5c003520 con 0x7fce700b63b0 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 >> v2:172.21.15.178:3300/0 conn(0x7fce7015e510 msgr2=0x7fce700d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 --2- 172.21.15.113:0/2294039537 >> v2:172.21.15.178:3300/0 conn(0x7fce7015e510 0x7fce700d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 >> v1:172.21.15.178:6789/0 conn(0x7fce700d28c0 legacy=0x7fce700b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce700d36a0 con 0x7fce700b63b0 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fce680019f0 con 0x7fce700b63b0 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 >> v1:172.21.15.113:6789/0 conn(0x7fce700b63b0 legacy=0x7fce700b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce5c003b90 0x7fce5c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce5c004800 0x7fce5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c005210 0x7fce5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce700d33c0 con 0x7fce5c004800 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c00c140 con 0x7fce5c005210 2024-04-25T00:09:27.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c00c340 con 0x7fce5c003b90 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce75ccd640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce5c004800 0x7fce5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce76ccf640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c005210 0x7fce5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce764ce640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce5c003b90 0x7fce5c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce75ccd640 1 -- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce5c003b90 msgr2=0x7fce5c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce75ccd640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce5c003b90 0x7fce5c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce75ccd640 1 -- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c005210 msgr2=0x7fce5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce75ccd640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c005210 0x7fce5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce75ccd640 1 -- 172.21.15.113:0/2294039537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c003520 con 0x7fce5c004800 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.925+0000 7fce76ccf640 1 -- 172.21.15.113:0/2294039537 reap_dead start 2024-04-25T00:09:27.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.926+0000 7fce75ccd640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce5c004800 0x7fce5c004c80 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fce600018d0 tx=0x7fce5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f6b777cf36dba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.926+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce600014f0 con 0x7fce5c004800 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.926+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce600052d0 con 0x7fce5c004800 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.926+0000 7fce754cc640 1 -- 172.21.15.113:0/2294039537 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6002ac70 con 0x7fce5c004800 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.927+0000 7fce78732640 1 -- 172.21.15.113:0/2294039537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce5c004800 msgr2=0x7fce5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.927+0000 7fce78732640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce5c004800 0x7fce5c004c80 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fce600018d0 tx=0x7fce5c005830 comp rx=0 tx=0).stop 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.927+0000 7fce78732640 1 -- 172.21.15.113:0/2294039537 shutdown_connections 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.927+0000 7fce78732640 1 --2- 172.21.15.113:0/2294039537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce5c004800 0x7fce5c004c80 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.927+0000 7fce78732640 1 -- 172.21.15.113:0/2294039537 >> 172.21.15.113:0/2294039537 conn(0x7fce7006b720 msgr2=0x7fce700d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.927+0000 7fce78732640 1 -- 172.21.15.113:0/2294039537 shutdown_connections 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.927+0000 7fce78732640 1 -- 172.21.15.113:0/2294039537 wait complete. 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.928+0000 7fce78732640 1 Processor -- start 2024-04-25T00:09:27.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.928+0000 7fce78732640 1 -- start start 2024-04-25T00:09:27.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce78732640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c004800 0x7fce7014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce78732640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce700c7d50 0x7fce700cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce78732640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce70161440 0x7fce700cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce78732640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce700b9dd0 con 0x7fce700c7d50 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce78732640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce700d3ea0 con 0x7fce5c004800 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce78732640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce700d2380 con 0x7fce70161440 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce700c7d50 0x7fce700cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce700c7d50 0x7fce700cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41858/0 (socket says 172.21.15.113:41858) 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 -- 172.21.15.113:0/4065945049 learned_addr learned my addr 172.21.15.113:0/4065945049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce764ce640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c004800 0x7fce7014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce76ccf640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce70161440 0x7fce700cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 -- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce70161440 msgr2=0x7fce700cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce70161440 0x7fce700cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 -- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c004800 msgr2=0x7fce7014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c004800 0x7fce7014c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 -- 172.21.15.113:0/4065945049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce700cdd80 con 0x7fce700c7d50 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce76ccf640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce70161440 0x7fce700cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce764ce640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c004800 0x7fce7014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce75ccd640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce700c7d50 0x7fce700cd4a0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fce60005040 tx=0x7fce600012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.929+0000 7fce66ffd640 1 -- 172.21.15.113:0/4065945049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60037040 con 0x7fce700c7d50 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.930+0000 7fce66ffd640 1 -- 172.21.15.113:0/4065945049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce60006e50 con 0x7fce700c7d50 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.930+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce700ce060 con 0x7fce700c7d50 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.930+0000 7fce66ffd640 1 -- 172.21.15.113:0/4065945049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce600014f0 con 0x7fce700c7d50 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.930+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce70158c30 con 0x7fce700c7d50 2024-04-25T00:09:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.931+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce700691a0 con 0x7fce700c7d50 2024-04-25T00:09:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.931+0000 7fce66ffd640 1 -- 172.21.15.113:0/4065945049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce60044020 con 0x7fce700c7d50 2024-04-25T00:09:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.931+0000 7fce66ffd640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce5406be30 0x7fce5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.932+0000 7fce66ffd640 1 -- 172.21.15.113:0/4065945049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4943+0+0 (secure 0 0 0) 0x7fce600aabd0 con 0x7fce700c7d50 2024-04-25T00:09:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.932+0000 7fce764ce640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce5406be30 0x7fce5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.932+0000 7fce764ce640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce5406be30 0x7fce5406e2f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099a0 tx=0x7fce6c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:27.936+0000 7fce66ffd640 1 -- 172.21.15.113:0/4065945049 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce60079910 con 0x7fce700c7d50 2024-04-25T00:09:28.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.107+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fce700b4650 con 0x7fce700c7d50 2024-04-25T00:09:28.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.112+0000 7fce66ffd640 1 -- 172.21.15.113:0/4065945049 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fce60079210 con 0x7fce700c7d50 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.114+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce5406be30 msgr2=0x7fce5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.114+0000 7fce78732640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce5406be30 0x7fce5406e2f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099a0 tx=0x7fce6c008040 comp rx=0 tx=0).stop 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.114+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce700c7d50 msgr2=0x7fce700cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.114+0000 7fce78732640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce700c7d50 0x7fce700cd4a0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fce60005040 tx=0x7fce600012d0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 shutdown_connections 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fce5406be30 0x7fce5406e2f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce70161440 0x7fce700cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce700c7d50 0x7fce700cd4a0 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 --2- 172.21.15.113:0/4065945049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce5c004800 0x7fce7014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 >> 172.21.15.113:0/4065945049 conn(0x7fce7006b720 msgr2=0x7fce7015d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:28.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 shutdown_connections 2024-04-25T00:09:28.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.115+0000 7fce78732640 1 -- 172.21.15.113:0/4065945049 wait complete. 2024-04-25T00:09:28.126 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:09:28.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:09:28.163 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:09:28.207 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:09:28.207 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:28.207 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:28.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:09:28.223 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:09:28.223 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:28.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:09:28.265 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:28.265 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:28.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:09:28.280 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:09:28.280 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:28.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:09:28.322 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T00:09:28.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:28.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.419+0000 7f23023ee640 1 Processor -- start 2024-04-25T00:09:28.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.419+0000 7f23023ee640 1 -- start start 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.419+0000 7f23023ee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f22fc15a6b0 0x7f22fc070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.419+0000 7f23023ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22fc072ea0 0x7f22fc071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.419+0000 7f23023ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22fc0b2d50 con 0x7f22fc071780 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.419+0000 7f23023ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f22fc072a70 con 0x7f22fc072ea0 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f23023ee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f22fc0b3010 con 0x7f22fc15a6b0 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22fc072ea0 0x7f22fc071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fb7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22fc072ea0 0x7f22fc071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41870/0 (socket says 172.21.15.113:41870) 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fb7fe640 1 -- 172.21.15.113:0/2715193034 learned_addr learned my addr 172.21.15.113:0/2715193034 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:28.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fbfff640 1 --2- 172.21.15.113:0/2715193034 >> v2:172.21.15.178:3300/0 conn(0x7f22fc15a6b0 0x7f22fc070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fb7fe640 1 -- 172.21.15.113:0/2715193034 >> v2:172.21.15.178:3300/0 conn(0x7f22fc15a6b0 msgr2=0x7f22fc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fb7fe640 1 --2- 172.21.15.113:0/2715193034 >> v2:172.21.15.178:3300/0 conn(0x7f22fc15a6b0 0x7f22fc070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fb7fe640 1 -- 172.21.15.113:0/2715193034 >> v1:172.21.15.178:6789/0 conn(0x7f22fc071780 legacy=0x7f22fc15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.420+0000 7f22fb7fe640 1 -- 172.21.15.113:0/2715193034 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc0b32f0 con 0x7f22fc072ea0 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f22fb7fe640 1 --2- 172.21.15.113:0/2715193034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc072ea0 0x7f22fc071240 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f22fc062850 tx=0x7f22f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da78732ec9de677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f22faffd640 1 -- 172.21.15.113:0/2715193034 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0037070 con 0x7f22fc072ea0 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f22faffd640 1 -- 172.21.15.113:0/2715193034 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f0032440 con 0x7f22fc072ea0 2024-04-25T00:09:28.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f22faffd640 1 -- 172.21.15.113:0/2715193034 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0032810 con 0x7f22fc072ea0 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 -- 172.21.15.113:0/2715193034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc072ea0 msgr2=0x7f22fc071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 --2- 172.21.15.113:0/2715193034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc072ea0 0x7f22fc071240 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f22fc062850 tx=0x7f22f002a270 comp rx=0 tx=0).stop 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 -- 172.21.15.113:0/2715193034 shutdown_connections 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 --2- 172.21.15.113:0/2715193034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc072ea0 0x7f22fc071240 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 --2- 172.21.15.113:0/2715193034 >> v2:172.21.15.178:3300/0 conn(0x7f22fc15a6b0 0x7f22fc070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 -- 172.21.15.113:0/2715193034 >> 172.21.15.113:0/2715193034 conn(0x7f22fc0b1a50 msgr2=0x7f22fc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 -- 172.21.15.113:0/2715193034 shutdown_connections 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.421+0000 7f23023ee640 1 -- 172.21.15.113:0/2715193034 wait complete. 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.422+0000 7f23023ee640 1 Processor -- start 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f23023ee640 1 -- start start 2024-04-25T00:09:28.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f23023ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc071780 0x7f22fc14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f23023ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f22fc072ea0 0x7f22fc14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f23023ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22fc15a6b0 0x7f22fc14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f23023ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc15aa90 con 0x7f22fc071780 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f23023ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc072720 con 0x7f22fc15a6b0 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f23023ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc15ac10 con 0x7f22fc072ea0 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f22fbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc071780 0x7f22fc14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f22fbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc071780 0x7f22fc14c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41884/0 (socket says 172.21.15.113:41884) 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f22fbfff640 1 -- 172.21.15.113:0/1500672963 learned_addr learned my addr 172.21.15.113:0/1500672963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.423+0000 7f230098b640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22fc15a6b0 0x7f22fc14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22fb7fe640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f22fc072ea0 0x7f22fc14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22fbfff640 1 -- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f22fc072ea0 msgr2=0x7f22fc14d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22fbfff640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f22fc072ea0 0x7f22fc14d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22fbfff640 1 -- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22fc15a6b0 msgr2=0x7f22fc14ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22fbfff640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22fc15a6b0 0x7f22fc14ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22fbfff640 1 -- 172.21.15.113:0/1500672963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc153420 con 0x7f22fc071780 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22fbfff640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc071780 0x7f22fc14c990 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d620 tx=0x7f22ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22f8ff9640 1 -- 172.21.15.113:0/1500672963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00b620 con 0x7f22fc071780 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22f8ff9640 1 -- 172.21.15.113:0/1500672963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec019070 con 0x7f22fc071780 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22fc153700 con 0x7f22fc071780 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f22f8ff9640 1 -- 172.21.15.113:0/1500672963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00ba00 con 0x7f22fc071780 2024-04-25T00:09:28.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.424+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22fc153cc0 con 0x7f22fc071780 2024-04-25T00:09:28.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.425+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22fc0691a0 con 0x7f22fc071780 2024-04-25T00:09:28.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.425+0000 7f22f8ff9640 1 -- 172.21.15.113:0/1500672963 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22ec00bba0 con 0x7f22fc071780 2024-04-25T00:09:28.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.426+0000 7f22f8ff9640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f22e806be30 0x7f22e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.426+0000 7f22f8ff9640 1 -- 172.21.15.113:0/1500672963 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4943+0+0 (secure 0 0 0) 0x7f22ec08af70 con 0x7f22fc071780 2024-04-25T00:09:28.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.426+0000 7f22fb7fe640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f22e806be30 0x7f22e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.426+0000 7f22fb7fe640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f22e806be30 0x7f22e806e2f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f22f0034000 tx=0x7f22f002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:28.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.430+0000 7f22f8ff9640 1 -- 172.21.15.113:0/1500672963 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22ec059c20 con 0x7f22fc071780 2024-04-25T00:09:28.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.639+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22fc070d00 con 0x7f22fc071780 2024-04-25T00:09:28.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.640+0000 7f22f8ff9640 1 -- 172.21.15.113:0/1500672963 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3871 (secure 0 0 0) 0x7f22ec059520 con 0x7f22fc071780 2024-04-25T00:09:28.641 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:28.641 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/404379395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":404379395},{"type":"v1","addr":"172.21.15.178:6835","nonce":404379395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/992895335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":992895335},{"type":"v1","addr":"172.21.15.178:6834","nonce":992895335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/17230728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":17230728},{"type":"v1","addr":"172.21.15.113:6836","nonce":17230728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":10422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4170863917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4170863917},{"type":"v1","addr":"172.21.15.113:6837","nonce":4170863917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[]} 2024-04-25T00:09:28.641 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f22e806be30 msgr2=0x7f22e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f22e806be30 0x7f22e806e2f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f22f0034000 tx=0x7f22f002e000 comp rx=0 tx=0).stop 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc071780 msgr2=0x7f22fc14c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc071780 0x7f22fc14c990 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d620 tx=0x7f22ec00dae0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 shutdown_connections 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f22e806be30 0x7f22e806e2f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22fc15a6b0 0x7f22fc14ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f22fc072ea0 0x7f22fc14d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 --2- 172.21.15.113:0/1500672963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22fc071780 0x7f22fc14c990 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 >> 172.21.15.113:0/1500672963 conn(0x7f22fc0b1a50 msgr2=0x7f22fc154790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:28.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 shutdown_connections 2024-04-25T00:09:28.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.642+0000 7f23023ee640 1 -- 172.21.15.113:0/1500672963 wait complete. 2024-04-25T00:09:28.654 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:09:28.654 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:09:28.654 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:09:28.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:28.654 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:09:28.655 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T00:09:28.655 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:09:28.655 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:28.655 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:09:28.655 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:09:28.655 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:09:28.655 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:28.656 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:09:28.656 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T00:09:28.656 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:09:28.656 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:09:28.663 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:09:28.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:09:28.664 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:09:28.664 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:09:28.667 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:09:28.667 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:09:28.668 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:09:28.668 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:09:28.706 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:09:28.706 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:28.706 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:09:28.707 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:28.709 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:09:28.709 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:28.710 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:09:28.710 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:28.710 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:09:28.778 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:09:28.778 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-04-25T00:09:28.788 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:09:28.788 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-25T00:09:28.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.810+0000 7f83ad238640 1 Processor -- start 2024-04-25T00:09:28.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83ad238640 1 -- start start 2024-04-25T00:09:28.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83ad238640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f83a815e510 0x7f83a80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83ad238640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83a80d28c0 0x7f83a80b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83ad238640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a80d2e00 con 0x7f83a80b61e0 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83ad238640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f83a80b6610 con 0x7f83a80d28c0 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83ad238640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f83a80d30e0 con 0x7f83a815e510 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83a659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83a80d28c0 0x7f83a80b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83a659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83a80d28c0 0x7f83a80b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41918/0 (socket says 172.21.15.113:41918) 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83a659c640 1 -- 172.21.15.113:0/2348854298 learned_addr learned my addr 172.21.15.113:0/2348854298 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.811+0000 7f83a6d9d640 1 --2- 172.21.15.113:0/2348854298 >> v2:172.21.15.178:3300/0 conn(0x7f83a815e510 0x7f83a80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.812+0000 7f83a659c640 1 -- 172.21.15.113:0/2348854298 >> v2:172.21.15.178:3300/0 conn(0x7f83a815e510 msgr2=0x7f83a80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.812+0000 7f83a659c640 1 --2- 172.21.15.113:0/2348854298 >> v2:172.21.15.178:3300/0 conn(0x7f83a815e510 0x7f83a80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.812+0000 7f83a659c640 1 -- 172.21.15.113:0/2348854298 >> v1:172.21.15.178:6789/0 conn(0x7f83a80b61e0 legacy=0x7f83a8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.812+0000 7f83a659c640 1 -- 172.21.15.113:0/2348854298 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a80d33c0 con 0x7f83a80d28c0 2024-04-25T00:09:28.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.812+0000 7f83a659c640 1 --2- 172.21.15.113:0/2348854298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80d28c0 0x7f83a80b5ca0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f83a80d52a0 tx=0x7f839002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9298251a3dfc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.817+0000 7f83a5d9b640 1 -- 172.21.15.113:0/2348854298 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390037070 con 0x7f83a80d28c0 2024-04-25T00:09:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.817+0000 7f83a5d9b640 1 -- 172.21.15.113:0/2348854298 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8390032440 con 0x7f83a80d28c0 2024-04-25T00:09:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.817+0000 7f83a5d9b640 1 -- 172.21.15.113:0/2348854298 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390032810 con 0x7f83a80d28c0 2024-04-25T00:09:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 -- 172.21.15.113:0/2348854298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80d28c0 msgr2=0x7f83a80b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 --2- 172.21.15.113:0/2348854298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80d28c0 0x7f83a80b5ca0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f83a80d52a0 tx=0x7f839002a1e0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 -- 172.21.15.113:0/2348854298 shutdown_connections 2024-04-25T00:09:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 --2- 172.21.15.113:0/2348854298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80d28c0 0x7f83a80b5ca0 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 --2- 172.21.15.113:0/2348854298 >> v2:172.21.15.178:3300/0 conn(0x7f83a815e510 0x7f83a80d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 -- 172.21.15.113:0/2348854298 >> 172.21.15.113:0/2348854298 conn(0x7f83a806b720 msgr2=0x7f83a80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 -- 172.21.15.113:0/2348854298 shutdown_connections 2024-04-25T00:09:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.818+0000 7f83ad238640 1 -- 172.21.15.113:0/2348854298 wait complete. 2024-04-25T00:09:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.820+0000 7f83ad238640 1 Processor -- start 2024-04-25T00:09:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.820+0000 7f83ad238640 1 -- start start 2024-04-25T00:09:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83ad238640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80b61e0 0x7f83a80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83ad238640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83a80d28c0 0x7f83a815f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83ad238640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83a815e510 0x7f83a80c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83ad238640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83a80d3ea0 con 0x7f83a80b61e0 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83ad238640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83a80d3980 con 0x7f83a815e510 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83ad238640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83a80b9dd0 con 0x7f83a80d28c0 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80b61e0 0x7f83a80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80b61e0 0x7f83a80ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41922/0 (socket says 172.21.15.113:41922) 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 -- 172.21.15.113:0/2305163430 learned_addr learned my addr 172.21.15.113:0/2305163430 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a659c640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83a80d28c0 0x7f83a815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a759e640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83a815e510 0x7f83a80c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 -- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83a80d28c0 msgr2=0x7f83a815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83a80d28c0 0x7f83a815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 -- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83a815e510 msgr2=0x7f83a80c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:28.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83a815e510 0x7f83a80c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:28.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 -- 172.21.15.113:0/2305163430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a80ca3e0 con 0x7f83a80b61e0 2024-04-25T00:09:28.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.821+0000 7f83a6d9d640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80b61e0 0x7f83a80ca700 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f839800e8a0 tx=0x7f839800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:28.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.825+0000 7f839f7fe640 1 -- 172.21.15.113:0/2305163430 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398017070 con 0x7f83a80b61e0 2024-04-25T00:09:28.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.825+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a815f8a0 con 0x7f83a80b61e0 2024-04-25T00:09:28.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.825+0000 7f839f7fe640 1 -- 172.21.15.113:0/2305163430 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839800c940 con 0x7f83a80b61e0 2024-04-25T00:09:28.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.825+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a815fe60 con 0x7f83a80b61e0 2024-04-25T00:09:28.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.825+0000 7f839f7fe640 1 -- 172.21.15.113:0/2305163430 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839800cd30 con 0x7f83a80b61e0 2024-04-25T00:09:28.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.826+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a80691a0 con 0x7f83a80b61e0 2024-04-25T00:09:28.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.827+0000 7f839f7fe640 1 -- 172.21.15.113:0/2305163430 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f839801a070 con 0x7f83a80b61e0 2024-04-25T00:09:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.827+0000 7f839f7fe640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f838806be30 0x7f838806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.827+0000 7f839f7fe640 1 -- 172.21.15.113:0/2305163430 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 4943+0+0 (secure 0 0 0) 0x7f839808ada0 con 0x7f83a80b61e0 2024-04-25T00:09:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.827+0000 7f83a659c640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f838806be30 0x7f838806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:28.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.827+0000 7f83a659c640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f838806be30 0x7f838806e2f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8390034000 tx=0x7f839002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:28.831+0000 7f839f7fe640 1 -- 172.21.15.113:0/2305163430 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83a80ca3e0 con 0x7f83a80b61e0 2024-04-25T00:09:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.018+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f83a80c8af0 con 0x7f83a80b61e0 2024-04-25T00:09:29.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.838+0000 7f839f7fe640 1 -- 172.21.15.113:0/2305163430 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v178) ==== 97+0+0 (secure 0 0 0) 0x7f8398059360 con 0x7f83a80b61e0 2024-04-25T00:09:29.839 INFO:teuthology.orchestra.run.smithi113.stderr: removed all blocklist entries 2024-04-25T00:09:29.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f838806be30 msgr2=0x7f838806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f838806be30 0x7f838806e2f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8390034000 tx=0x7f839002e000 comp rx=0 tx=0).stop 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80b61e0 msgr2=0x7f83a80ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80b61e0 0x7f83a80ca700 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f839800e8a0 tx=0x7f839800ed60 comp rx=0 tx=0).stop 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 shutdown_connections 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f838806be30 0x7f838806e2f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83a815e510 0x7f83a80c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83a80d28c0 0x7f83a815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 --2- 172.21.15.113:0/2305163430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83a80b61e0 0x7f83a80ca700 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 >> 172.21.15.113:0/2305163430 conn(0x7f83a806b720 msgr2=0x7f83a80b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 shutdown_connections 2024-04-25T00:09:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.840+0000 7f83ad238640 1 -- 172.21.15.113:0/2305163430 wait complete. 2024-04-25T00:09:29.853 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:09:29.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.951+0000 7f06d0272640 1 Processor -- start 2024-04-25T00:09:29.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.951+0000 7f06d0272640 1 -- start start 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06d0272640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 0x7f06c81656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06d0272640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c8071b20 con 0x7f06c815a6b0 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06d0272640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c80b2d50 con 0x7f06c8072ea0 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06d0272640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06c8072a70 con 0x7f06c80b4b00 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06ce00e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f06c815a6b0 0x7f06c8070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38418/0 (socket says 172.21.15.113:38418) 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06ce00e640 1 -- 172.21.15.113:0/532832768 learned_addr learned my addr 172.21.15.113:0/532832768 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06ce80f640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 0x7f06c81656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954772784 0 0) 0x7f06c8071b20 con 0x7f06c815a6b0 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06b4003520 con 0x7f06c815a6b0 2024-04-25T00:09:29.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3286411353 0 0) 0x7f06b4003520 con 0x7f06c815a6b0 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 msgr2=0x7f06c81656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 0x7f06c81656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 >> v1:172.21.15.178:6789/0 conn(0x7f06c8072ea0 legacy=0x7f06c8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80b3010 con 0x7f06c815a6b0 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f06c4001670 con 0x7f06c815a6b0 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 >> v1:172.21.15.113:6789/0 conn(0x7f06c815a6b0 legacy=0x7f06c8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 0x7f06b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06b4003f60 0x7f06b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06b4004d40 0x7f06b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8072a70 con 0x7f06b4003f60 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06b400cbb0 con 0x7f06b4004d40 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.952+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06b400cdb0 con 0x7f06c80b4b00 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06cd80d640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06b4003f60 0x7f06b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06ce00e640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 0x7f06b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06ce80f640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06b4004d40 0x7f06b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06cd80d640 1 -- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 msgr2=0x7f06b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06cd80d640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 0x7f06b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06cd80d640 1 -- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06b4004d40 msgr2=0x7f06b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06cd80d640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06b4004d40 0x7f06b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06cd80d640 1 -- 172.21.15.113:0/532832768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b4003520 con 0x7f06b4003f60 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06ce00e640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06c80b4b00 0x7f06b40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06ce80f640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06b4004d40 0x7f06b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.953+0000 7f06cd80d640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06b4003f60 0x7f06b4004800 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f06b8001750 tx=0x7f06b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f23b7f6e7ada39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8001370 con 0x7f06b4003f60 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b80052d0 con 0x7f06b4003f60 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06cd00c640 1 -- 172.21.15.113:0/532832768 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b802abe0 con 0x7f06b4003f60 2024-04-25T00:09:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06d0272640 1 -- 172.21.15.113:0/532832768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06b4003f60 msgr2=0x7f06b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06d0272640 1 --2- 172.21.15.113:0/532832768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06b4003f60 0x7f06b4004800 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f06b8001750 tx=0x7f06b4005830 comp rx=0 tx=0).stop 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06ce80f640 1 -- 172.21.15.113:0/532832768 reap_dead start 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06d0272640 1 -- 172.21.15.113:0/532832768 shutdown_connections 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06d0272640 1 -- 172.21.15.113:0/532832768 >> 172.21.15.113:0/532832768 conn(0x7f06c80b1a50 msgr2=0x7f06c80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06d0272640 1 -- 172.21.15.113:0/532832768 shutdown_connections 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.954+0000 7f06d0272640 1 -- 172.21.15.113:0/532832768 wait complete. 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.955+0000 7f06d0272640 1 Processor -- start 2024-04-25T00:09:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.955+0000 7f06d0272640 1 -- start start 2024-04-25T00:09:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06d0272640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b4003f60 0x7f06c81e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06d0272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c81e1780 0x7f06c81dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06d0272640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c81e1c00 0x7f06c81dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06d0272640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8072720 con 0x7f06c81e1780 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06d0272640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06c80b35d0 con 0x7f06c81e1c00 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06d0272640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06c815aa90 con 0x7f06b4003f60 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c81e1780 0x7f06c81dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c81e1780 0x7f06c81dfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41936/0 (socket says 172.21.15.113:41936) 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 -- 172.21.15.113:0/3832524236 learned_addr learned my addr 172.21.15.113:0/3832524236 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 -- 172.21.15.113:0/3832524236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b4003f60 msgr2=0x7f06c81e2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b4003f60 0x7f06c81e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 -- 172.21.15.113:0/3832524236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c81e1c00 msgr2=0x7f06c81dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c81e1c00 0x7f06c81dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.956+0000 7f06cd80d640 1 -- 172.21.15.113:0/3832524236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c81e21b0 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.957+0000 7f06cd80d640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c81e1780 0x7f06c81dfaa0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f06b8008fd0 tx=0x7f06b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.957+0000 7f06beffd640 1 -- 172.21.15.113:0/3832524236 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8037040 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.957+0000 7f06beffd640 1 -- 172.21.15.113:0/3832524236 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b8005480 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.957+0000 7f06beffd640 1 -- 172.21.15.113:0/3832524236 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8001370 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.957+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c81f26a0 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.957+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c81f2c60 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.958+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c80691a0 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.958+0000 7f06beffd640 1 -- 172.21.15.113:0/3832524236 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06b8044020 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.958+0000 7f06beffd640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06ac06be30 0x7f06ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.959+0000 7f06beffd640 1 -- 172.21.15.113:0/3832524236 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 4857+0+0 (secure 0 0 0) 0x7f06b8007640 con 0x7f06c81e1780 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.959+0000 7f06ce00e640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06ac06be30 0x7f06ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.959+0000 7f06ce00e640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06ac06be30 0x7f06ac06e2f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f06c4001b80 tx=0x7f06c4001df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:29.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:29.963+0000 7f06beffd640 1 -- 172.21.15.113:0/3832524236 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f06b8079630 con 0x7f06c81e1780 2024-04-25T00:09:30.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.200+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f06c8150f10 con 0x7f06c81e1780 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.200+0000 7f06beffd640 1 -- 172.21.15.113:0/3832524236 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f06b8078f30 con 0x7f06c81e1780 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: "auth_dump": [ 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.a", 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmjAtcIhAAePSxw+yWB8N/vCXZE7kVdw==", 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.201 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.b", 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClma2iBLRAAV7gQLdRoMrxSsIcbMALGLw==", 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.202 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.c", 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmJK0aKRAA5co7z01cWKKuvRIZ2yCZwg==", 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.203 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mds.d", 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmFvVSNBAAuEvmcK/Kd1uxE5CMQdf22Q==", 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.0", 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBnnClmLA3wFBAAIHiR8bmDBeEZLIxbbt3Ffw==", 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.204 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.1", 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBqnClmp67LEhAAqPuihcHFq+KYAUPLveNPpA==", 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.205 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.2", 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBtnClmyCEKERAAH2S6d4aME/ZwfgGmALiL+A==", 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.206 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.3", 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBwnClmvutMDxAAfk2IDo/uEbXjsI+IwwJyAg==", 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.4", 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB0nClmw983FxAAp7st675AydbXKmej07HmSA==", 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.207 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.5", 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB3nClm8x6IFRAAnQo/CHPrPzye+sxRyrri+w==", 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.208 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.6", 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB6nClm4k92ExAA/ez08c9M6uFHvBpCqn9tXA==", 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "osd.7", 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQB9nClmWn9MERAAfhVUzr5xplRZf2V9PDWr1g==", 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow profile osd", 2024-04-25T00:09:30.209 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile osd", 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.0", 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ==", 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow r", 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.210 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.1", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClmLTv2BBAAZ30/R7K1oMPW0SSCMG0HYw==", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.2", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm2BDwARAAJRVRKFCaXdc6yYtShU+rog==", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.211 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.3", 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBknClm4JdaChAACyOA+GLrwTNwa/wcCHOtmQ==", 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow", 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow r", 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow rw", 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow rwx" 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "client.admin", 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmdw+IBxAAh7+jMQeMadrFgq8B95EP1w==", 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr": "allow *", 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow *", 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.x", 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmpX1hHxAA8uPYm4jujpa1xfzSqAg8pA==", 2024-04-25T00:09:30.213 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "entity": "mgr.y", 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "key": "AQBjnClmm0CwGxAA5NYPTugVdcoWOrUJAQy0jA==", 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": { 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": "allow *", 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "mon": "allow profile mgr", 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: "osd": "allow *" 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.214 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:30.215 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-25T00:09:30.215 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:30.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06ac06be30 msgr2=0x7f06ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06ac06be30 0x7f06ac06e2f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f06c4001b80 tx=0x7f06c4001df0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c81e1780 msgr2=0x7f06c81dfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c81e1780 0x7f06c81dfaa0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f06b8008fd0 tx=0x7f06b8001120 comp rx=0 tx=0).stop 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 shutdown_connections 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f06ac06be30 0x7f06ac06e2f0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c81e1c00 0x7f06c81dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c81e1780 0x7f06c81dfaa0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 --2- 172.21.15.113:0/3832524236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b4003f60 0x7f06c81e2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 >> 172.21.15.113:0/3832524236 conn(0x7f06c80b1a50 msgr2=0x7f06c80b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 shutdown_connections 2024-04-25T00:09:30.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.202+0000 7f06d0272640 1 -- 172.21.15.113:0/3832524236 wait complete. 2024-04-25T00:09:30.218 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:30.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.314+0000 7fa16869a640 1 Processor -- start 2024-04-25T00:09:30.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.314+0000 7fa16869a640 1 -- start start 2024-04-25T00:09:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa16869a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa160070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa16869a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa1600b3af0 0x7fa1600b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa16869a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1600b2d50 con 0x7fa160072ea0 2024-04-25T00:09:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa16869a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa160072a70 con 0x7fa1600b3af0 2024-04-25T00:09:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa16869a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1600b3010 con 0x7fa16015a6b0 2024-04-25T00:09:30.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166436640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa160070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166c37640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa1600b3af0 0x7fa1600b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166436640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa160070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:59242/0 (socket says 172.21.15.113:59242) 2024-04-25T00:09:30.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166436640 1 -- 172.21.15.113:0/4075221259 learned_addr learned my addr 172.21.15.113:0/4075221259 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:30.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166436640 1 -- 172.21.15.113:0/4075221259 >> v2:172.21.15.178:3300/0 conn(0x7fa1600b3af0 msgr2=0x7fa1600b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:30.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166436640 1 --2- 172.21.15.113:0/4075221259 >> v2:172.21.15.178:3300/0 conn(0x7fa1600b3af0 0x7fa1600b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166436640 1 -- 172.21.15.113:0/4075221259 >> v1:172.21.15.178:6789/0 conn(0x7fa160072ea0 legacy=0x7fa160071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.315+0000 7fa166436640 1 -- 172.21.15.113:0/4075221259 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1600b32f0 con 0x7fa16015a6b0 2024-04-25T00:09:30.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.316+0000 7fa166436640 1 --2- 172.21.15.113:0/4075221259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa160070d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa158009e50 tx=0x7fa15802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebc4a939c996717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.316+0000 7fa165434640 1 -- 172.21.15.113:0/4075221259 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158036070 con 0x7fa16015a6b0 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.316+0000 7fa165434640 1 -- 172.21.15.113:0/4075221259 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa158031600 con 0x7fa16015a6b0 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.316+0000 7fa165434640 1 -- 172.21.15.113:0/4075221259 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1580319d0 con 0x7fa16015a6b0 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 -- 172.21.15.113:0/4075221259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 msgr2=0x7fa160070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 --2- 172.21.15.113:0/4075221259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa160070d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa158009e50 tx=0x7fa15802a710 comp rx=0 tx=0).stop 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 -- 172.21.15.113:0/4075221259 shutdown_connections 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 --2- 172.21.15.113:0/4075221259 >> v2:172.21.15.178:3300/0 conn(0x7fa1600b3af0 0x7fa1600b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 --2- 172.21.15.113:0/4075221259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa160070d00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 -- 172.21.15.113:0/4075221259 >> 172.21.15.113:0/4075221259 conn(0x7fa1600b1a50 msgr2=0x7fa1600b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 -- 172.21.15.113:0/4075221259 shutdown_connections 2024-04-25T00:09:30.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.317+0000 7fa16869a640 1 -- 172.21.15.113:0/4075221259 wait complete. 2024-04-25T00:09:30.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.318+0000 7fa16869a640 1 Processor -- start 2024-04-25T00:09:30.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.318+0000 7fa16869a640 1 -- start start 2024-04-25T00:09:30.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa16869a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa160072ea0 0x7fa160144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa16869a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1600b3af0 0x7fa1601497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa16869a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa1601415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa16869a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa16015aa90 con 0x7fa160072ea0 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa16869a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa160072720 con 0x7fa1600b3af0 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa16869a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa16015ac10 con 0x7fa16015a6b0 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa160072ea0 0x7fa160144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa160072ea0 0x7fa160144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41950/0 (socket says 172.21.15.113:41950) 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 -- 172.21.15.113:0/2386037750 learned_addr learned my addr 172.21.15.113:0/2386037750 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166c37640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa1601415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa165c35640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1600b3af0 0x7fa1601497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 -- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 msgr2=0x7fa1601415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa1601415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 -- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1600b3af0 msgr2=0x7fa1601497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1600b3af0 0x7fa1601497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 -- 172.21.15.113:0/2386037750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa160149d30 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.319+0000 7fa166436640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa160072ea0 0x7fa160144300 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fa15802ad00 tx=0x7fa158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.320+0000 7fa156ffd640 1 -- 172.21.15.113:0/2386037750 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15803f020 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.320+0000 7fa156ffd640 1 -- 172.21.15.113:0/2386037750 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa158037040 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.320+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16014d280 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.320+0000 7fa156ffd640 1 -- 172.21.15.113:0/2386037750 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158036070 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.320+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa160148ff0 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.321+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa16014d7a0 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.321+0000 7fa156ffd640 1 -- 172.21.15.113:0/2386037750 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa158042020 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.321+0000 7fa156ffd640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14406be60 0x7fa14406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.321+0000 7fa156ffd640 1 -- 172.21.15.113:0/2386037750 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 4857+0+0 (secure 0 0 0) 0x7fa1580aaf00 con 0x7fa160072ea0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.321+0000 7fa165c35640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14406be60 0x7fa14406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.322+0000 7fa165c35640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14406be60 0x7fa14406e320 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa16014aea0 tx=0x7fa150009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.325+0000 7fa156ffd640 1 -- 172.21.15.113:0/2386037750 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa16014d7a0 con 0x7fa160072ea0 2024-04-25T00:09:30.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.499+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1600691a0 con 0x7fa160072ea0 2024-04-25T00:09:30.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.499+0000 7fa156ffd640 1 -- 172.21.15.113:0/2386037750 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) ==== 74+0+12949 (secure 0 0 0) 0x7fa158049030 con 0x7fa160072ea0 2024-04-25T00:09:30.500 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:30.500 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":178,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:29.837690+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14406be60 msgr2=0x7fa14406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14406be60 0x7fa14406e320 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa16014aea0 tx=0x7fa150009040 comp rx=0 tx=0).stop 2024-04-25T00:09:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa160072ea0 msgr2=0x7fa160144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa160072ea0 0x7fa160144300 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fa15802ad00 tx=0x7fa158004a40 comp rx=0 tx=0).stop 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 shutdown_connections 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa14406be60 0x7fa14406e320 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1600b3af0 0x7fa1601497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa16015a6b0 0x7fa1601415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 --2- 172.21.15.113:0/2386037750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa160072ea0 0x7fa160144300 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 >> 172.21.15.113:0/2386037750 conn(0x7fa1600b1a50 msgr2=0x7fa1600cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 shutdown_connections 2024-04-25T00:09:30.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.502+0000 7fa16869a640 1 -- 172.21.15.113:0/2386037750 wait complete. 2024-04-25T00:09:30.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:09:30.514 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:09:30.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.612+0000 7feca0bce640 1 Processor -- start 2024-04-25T00:09:30.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.612+0000 7feca0bce640 1 -- start start 2024-04-25T00:09:30.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.613+0000 7feca0bce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fec9c0b4a40 0x7fec9c165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.613+0000 7feca0bce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9c15f4f0 con 0x7fec9c15e2d0 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.613+0000 7feca0bce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9c15f7d0 con 0x7fec9c0b3080 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.613+0000 7feca0bce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fec9c0cde00 con 0x7fec9c0b4a40 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.613+0000 7fec9a59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fec9c15e2d0 0x7fec9c15ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38422/0 (socket says 172.21.15.113:38422) 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.613+0000 7fec9a59c640 1 -- 172.21.15.113:0/1286773018 learned_addr learned my addr 172.21.15.113:0/1286773018 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.613+0000 7fec9ad9d640 1 --2- 172.21.15.113:0/1286773018 >> v2:172.21.15.178:3300/0 conn(0x7fec9c0b4a40 0x7fec9c165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 177463616 0 0) 0x7fec9c15f4f0 con 0x7fec9c15e2d0 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec84003520 con 0x7fec9c15e2d0 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898044793 0 0) 0x7fec9c15f7d0 con 0x7fec9c0b3080 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec9c15f4f0 con 0x7fec9c0b3080 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9ad9d640 1 -- 172.21.15.113:0/1286773018 >> v1:172.21.15.178:6789/0 conn(0x7fec9c0b3080 legacy=0x7fec9c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9ad9d640 1 -- 172.21.15.113:0/1286773018 >> v1:172.21.15.113:6789/0 conn(0x7fec9c15e2d0 legacy=0x7fec9c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1392844804 0 0) 0x7fec84003520 con 0x7fec9c15e2d0 2024-04-25T00:09:30.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.614+0000 7fec9ad9d640 1 -- 172.21.15.113:0/1286773018 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c15fab0 con 0x7fec9c0b4a40 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7fec9ad9d640 1 --2- 172.21.15.113:0/1286773018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b4a40 0x7fec9c165f80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0cdf80 tx=0x7fec9402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f152206148530a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94037070 con 0x7fec9c0b4a40 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9402ac30 con 0x7fec9c0b4a40 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7fec9959a640 1 -- 172.21.15.113:0/1286773018 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0019f0 con 0x7fec9c0b4a40 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7feca0bce640 1 -- 172.21.15.113:0/1286773018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b4a40 msgr2=0x7fec9c165f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7feca0bce640 1 --2- 172.21.15.113:0/1286773018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b4a40 0x7fec9c165f80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0cdf80 tx=0x7fec9402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7feca0bce640 1 -- 172.21.15.113:0/1286773018 shutdown_connections 2024-04-25T00:09:30.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7feca0bce640 1 --2- 172.21.15.113:0/1286773018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b4a40 0x7fec9c165f80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7feca0bce640 1 -- 172.21.15.113:0/1286773018 >> 172.21.15.113:0/1286773018 conn(0x7fec9c06b0f0 msgr2=0x7fec9c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7feca0bce640 1 -- 172.21.15.113:0/1286773018 shutdown_connections 2024-04-25T00:09:30.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.615+0000 7feca0bce640 1 -- 172.21.15.113:0/1286773018 wait complete. 2024-04-25T00:09:30.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 Processor -- start 2024-04-25T00:09:30.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 -- start start 2024-04-25T00:09:30.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b3080 0x7fec9c1481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9c0b4a40 0x7fec9c14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9c15e2d0 0x7fec9c133160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c0cf420 con 0x7fec9c15e2d0 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c0cf2a0 con 0x7fec9c0b3080 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.617+0000 7feca0bce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec9c0cf5a0 con 0x7fec9c0b4a40 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9c15e2d0 0x7fec9c133160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9c15e2d0 0x7fec9c133160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41964/0 (socket says 172.21.15.113:41964) 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 -- 172.21.15.113:0/219386875 learned_addr learned my addr 172.21.15.113:0/219386875 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9a59c640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b3080 0x7fec9c1481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec99d9b640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9c0b4a40 0x7fec9c14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 -- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9c0b4a40 msgr2=0x7fec9c14b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9c0b4a40 0x7fec9c14b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 -- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b3080 msgr2=0x7fec9c1481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b3080 0x7fec9c1481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 -- 172.21.15.113:0/219386875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c14bda0 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec9ad9d640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9c15e2d0 0x7fec9c133160 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fec9402a1e0 tx=0x7fec940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec8affd640 1 -- 172.21.15.113:0/219386875 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94041020 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec8affd640 1 -- 172.21.15.113:0/219386875 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9403d620 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c14c080 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7fec8affd640 1 -- 172.21.15.113:0/219386875 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94037040 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.618+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c14c640 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.619+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9c0691a0 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.619+0000 7fec8affd640 1 -- 172.21.15.113:0/219386875 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec94044020 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.620+0000 7fec8affd640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec7c06be30 0x7fec7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.620+0000 7fec8affd640 1 -- 172.21.15.113:0/219386875 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 4857+0+0 (secure 0 0 0) 0x7fec940ab0d0 con 0x7fec9c15e2d0 2024-04-25T00:09:30.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.620+0000 7fec9a59c640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec7c06be30 0x7fec7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.620+0000 7fec9a59c640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec7c06be30 0x7fec7c06e2f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fec8c001700 tx=0x7fec8c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.624+0000 7fec8affd640 1 -- 172.21.15.113:0/219386875 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec9404b030 con 0x7fec9c15e2d0 2024-04-25T00:09:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.795+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec9c072050 con 0x7fec9c15e2d0 2024-04-25T00:09:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.796+0000 7fec8affd640 1 -- 172.21.15.113:0/219386875 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) ==== 93+0+10 (secure 0 0 0) 0x7fec94079d30 con 0x7fec9c15e2d0 2024-04-25T00:09:30.796 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:09:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec7c06be30 msgr2=0x7fec7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec7c06be30 0x7fec7c06e2f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fec8c001700 tx=0x7fec8c009040 comp rx=0 tx=0).stop 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9c15e2d0 msgr2=0x7fec9c133160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9c15e2d0 0x7fec9c133160 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fec9402a1e0 tx=0x7fec940046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 shutdown_connections 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fec7c06be30 0x7fec7c06e2f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9c15e2d0 0x7fec9c133160 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec9c0b4a40 0x7fec9c14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 --2- 172.21.15.113:0/219386875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec9c0b3080 0x7fec9c1481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 >> 172.21.15.113:0/219386875 conn(0x7fec9c06b0f0 msgr2=0x7fec9c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 shutdown_connections 2024-04-25T00:09:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.798+0000 7feca0bce640 1 -- 172.21.15.113:0/219386875 wait complete. 2024-04-25T00:09:30.809 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:09:30.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.906+0000 7fd6ae8dd640 1 Processor -- start 2024-04-25T00:09:30.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.906+0000 7fd6ae8dd640 1 -- start start 2024-04-25T00:09:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ae8dd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a8158910 0x7fd6a8162710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ae8dd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd6a815e7a0 0x7fd6a806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ae8dd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6a815ec20 0x7fd6a806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ae8dd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a8162c50 con 0x7fd6a815ec20 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ae8dd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a815f1d0 con 0x7fd6a815e7a0 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ae8dd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a815f4b0 con 0x7fd6a8158910 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6a815ec20 0x7fd6a806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6a815ec20 0x7fd6a806b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41968/0 (socket says 172.21.15.113:41968) 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 -- 172.21.15.113:0/416373936 learned_addr learned my addr 172.21.15.113:0/416373936 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6a7fff640 1 --2- 172.21.15.113:0/416373936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a8158910 0x7fd6a8162710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 -- 172.21.15.113:0/416373936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a8158910 msgr2=0x7fd6a8162710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 --2- 172.21.15.113:0/416373936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a8158910 0x7fd6a8162710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 -- 172.21.15.113:0/416373936 >> v2:172.21.15.178:3300/0 conn(0x7fd6a815e7a0 msgr2=0x7fd6a806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6a77fe640 1 --2- 172.21.15.113:0/416373936 >> v2:172.21.15.178:3300/0 conn(0x7fd6a815e7a0 0x7fd6a806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 --2- 172.21.15.113:0/416373936 >> v2:172.21.15.178:3300/0 conn(0x7fd6a815e7a0 0x7fd6a806b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.907+0000 7fd6ace7a640 1 -- 172.21.15.113:0/416373936 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a8161f10 con 0x7fd6a815ec20 2024-04-25T00:09:30.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.908+0000 7fd6ace7a640 1 --2- 172.21.15.113:0/416373936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a815ec20 0x7fd6a806b630 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8072630 tx=0x7fd6a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3e5a98fc653a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.908+0000 7fd6a6ffd640 1 -- 172.21.15.113:0/416373936 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0036070 con 0x7fd6a815ec20 2024-04-25T00:09:30.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.908+0000 7fd6a6ffd640 1 -- 172.21.15.113:0/416373936 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a0031600 con 0x7fd6a815ec20 2024-04-25T00:09:30.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.908+0000 7fd6a6ffd640 1 -- 172.21.15.113:0/416373936 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00319d0 con 0x7fd6a815ec20 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/416373936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a815ec20 msgr2=0x7fd6a806b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/416373936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a815ec20 0x7fd6a806b630 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8072630 tx=0x7fd6a002a6a0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/416373936 shutdown_connections 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/416373936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a815ec20 0x7fd6a806b630 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/416373936 >> v2:172.21.15.178:3300/0 conn(0x7fd6a815e7a0 0x7fd6a806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/416373936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a8158910 0x7fd6a8162710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/416373936 >> 172.21.15.113:0/416373936 conn(0x7fd6a80b22c0 msgr2=0x7fd6a80b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/416373936 shutdown_connections 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.909+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/416373936 wait complete. 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.910+0000 7fd6ae8dd640 1 Processor -- start 2024-04-25T00:09:30.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.910+0000 7fd6ae8dd640 1 -- start start 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8158910 0x7fd6a815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a815e7a0 0x7fd6a80d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a815ec20 0x7fd6a8142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a80b3890 con 0x7fd6a8158910 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a80b3710 con 0x7fd6a815e7a0 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a81438f0 con 0x7fd6a815ec20 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8158910 0x7fd6a815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8158910 0x7fd6a815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41982/0 (socket says 172.21.15.113:41982) 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 -- 172.21.15.113:0/2763526995 learned_addr learned my addr 172.21.15.113:0/2763526995 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a77fe640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a815e7a0 0x7fd6a80d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ace7a640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a815ec20 0x7fd6a8142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 -- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a815ec20 msgr2=0x7fd6a8142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a815ec20 0x7fd6a8142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 -- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a815e7a0 msgr2=0x7fd6a80d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a815e7a0 0x7fd6a80d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 -- 172.21.15.113:0/2763526995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a8161370 con 0x7fd6a8158910 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a77fe640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a815e7a0 0x7fd6a80d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a7fff640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8158910 0x7fd6a815c4a0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c7f0 tx=0x7fd69800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a4ff9640 1 -- 172.21.15.113:0/2763526995 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698015070 con 0x7fd6a8158910 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a4ff9640 1 -- 172.21.15.113:0/2763526995 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd698004240 con 0x7fd6a8158910 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a815cb50 con 0x7fd6a8158910 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6a4ff9640 1 -- 172.21.15.113:0/2763526995 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698007620 con 0x7fd6a8158910 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.911+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a815d110 con 0x7fd6a8158910 2024-04-25T00:09:30.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.912+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a80691a0 con 0x7fd6a8158910 2024-04-25T00:09:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.913+0000 7fd6a4ff9640 1 -- 172.21.15.113:0/2763526995 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6980077c0 con 0x7fd6a8158910 2024-04-25T00:09:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.913+0000 7fd6a4ff9640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd69406be30 0x7fd69406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.913+0000 7fd6a77fe640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd69406be30 0x7fd69406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.913+0000 7fd6a4ff9640 1 -- 172.21.15.113:0/2763526995 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 4857+0+0 (secure 0 0 0) 0x7fd69808ad30 con 0x7fd6a8158910 2024-04-25T00:09:30.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.914+0000 7fd6a77fe640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd69406be30 0x7fd69406e2f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80cfd70 tx=0x7fd69c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:30.918+0000 7fd6a4ff9640 1 -- 172.21.15.113:0/2763526995 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd698059ac0 con 0x7fd6a8158910 2024-04-25T00:09:31.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.091+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6a806f010 con 0x7fd6a8158910 2024-04-25T00:09:31.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.092+0000 7fd6a4ff9640 1 -- 172.21.15.113:0/2763526995 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v178) ==== 84+0+65 (secure 0 0 0) 0x7fd6980593c0 con 0x7fd6a8158910 2024-04-25T00:09:31.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:31.093 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-25T00:09:31.093 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-25T00:09:31.093 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-25T00:09:31.093 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-25T00:09:31.093 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:31.093 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-25T00:09:31.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd69406be30 msgr2=0x7fd69406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd69406be30 0x7fd69406e2f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80cfd70 tx=0x7fd69c008040 comp rx=0 tx=0).stop 2024-04-25T00:09:31.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8158910 msgr2=0x7fd6a815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8158910 0x7fd6a815c4a0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c7f0 tx=0x7fd69800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 shutdown_connections 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd69406be30 0x7fd69406e2f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a815e7a0 0x7fd6a80d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a815ec20 0x7fd6a8142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 --2- 172.21.15.113:0/2763526995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8158910 0x7fd6a815c4a0 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 >> 172.21.15.113:0/2763526995 conn(0x7fd6a80b22c0 msgr2=0x7fd6a80af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 shutdown_connections 2024-04-25T00:09:31.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.094+0000 7fd6ae8dd640 1 -- 172.21.15.113:0/2763526995 wait complete. 2024-04-25T00:09:31.105 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:09:31.105 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:09:31.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.207+0000 7fc819fce640 1 Processor -- start 2024-04-25T00:09:31.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.207+0000 7fc819fce640 1 -- start start 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.207+0000 7fc819fce640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc8140b9bb0 0x7fc8140b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.207+0000 7fc819fce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8141626f0 con 0x7fc81415e510 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.207+0000 7fc819fce640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8141629d0 con 0x7fc8140b61e0 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.207+0000 7fc819fce640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc814162cb0 con 0x7fc8140b9bb0 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc8137fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc81415e510 0x7fc814161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38430/0 (socket says 172.21.15.113:38430) 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc8137fe640 1 -- 172.21.15.113:0/2809463233 learned_addr learned my addr 172.21.15.113:0/2809463233 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321232140 0 0) 0x7fc8141626f0 con 0x7fc81415e510 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc800003520 con 0x7fc81415e510 2024-04-25T00:09:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc80bfff640 1 --2- 172.21.15.113:0/2809463233 >> v2:172.21.15.178:3300/0 conn(0x7fc8140b9bb0 0x7fc8140b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3478023715 0 0) 0x7fc800003520 con 0x7fc81415e510 2024-04-25T00:09:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 >> v2:172.21.15.178:3300/0 conn(0x7fc8140b9bb0 msgr2=0x7fc8140b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 --2- 172.21.15.113:0/2809463233 >> v2:172.21.15.178:3300/0 conn(0x7fc8140b9bb0 0x7fc8140b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 >> v1:172.21.15.178:6789/0 conn(0x7fc8140b61e0 legacy=0x7fc8140d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8140d2660 con 0x7fc81415e510 2024-04-25T00:09:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fc804001670 con 0x7fc81415e510 2024-04-25T00:09:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc80bfff640 1 --2- 172.21.15.113:0/2809463233 >> v2:172.21.15.178:3300/0 conn(0x7fc8140b9bb0 0x7fc8140b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 >> v1:172.21.15.113:6789/0 conn(0x7fc81415e510 legacy=0x7fc814161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8000039b0 0x7fc800003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc800004800 0x7fc800004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8000052e0 0x7fc80000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc814162cb0 con 0x7fc8000052e0 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc80000c1b0 con 0x7fc8000039b0 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc80000c3b0 con 0x7fc800004800 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc813fff640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8000052e0 0x7fc80000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc8137fe640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8000039b0 0x7fc800003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc80bfff640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc800004800 0x7fc800004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc813fff640 1 -- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc800004800 msgr2=0x7fc800004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc813fff640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc800004800 0x7fc800004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc813fff640 1 -- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8000039b0 msgr2=0x7fc800003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc813fff640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8000039b0 0x7fc800003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc813fff640 1 -- 172.21.15.113:0/2809463233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc800003520 con 0x7fc8000052e0 2024-04-25T00:09:31.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.208+0000 7fc80bfff640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc800004800 0x7fc800004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.209+0000 7fc813fff640 1 -- 172.21.15.113:0/2809463233 reap_dead start 2024-04-25T00:09:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.209+0000 7fc813fff640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8000052e0 0x7fc80000b9c0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fc80c001e20 tx=0x7fc800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2628845d43728452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.209+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c001a70 con 0x7fc8000052e0 2024-04-25T00:09:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.209+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80c02ae40 con 0x7fc8000052e0 2024-04-25T00:09:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.209+0000 7fc812ffd640 1 -- 172.21.15.113:0/2809463233 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003bb0 con 0x7fc8000052e0 2024-04-25T00:09:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.210+0000 7fc819fce640 1 -- 172.21.15.113:0/2809463233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8000052e0 msgr2=0x7fc80000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.210+0000 7fc819fce640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8000052e0 0x7fc80000b9c0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fc80c001e20 tx=0x7fc800005830 comp rx=0 tx=0).stop 2024-04-25T00:09:31.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.210+0000 7fc819fce640 1 -- 172.21.15.113:0/2809463233 shutdown_connections 2024-04-25T00:09:31.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.210+0000 7fc819fce640 1 --2- 172.21.15.113:0/2809463233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8000052e0 0x7fc80000b9c0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.210+0000 7fc819fce640 1 -- 172.21.15.113:0/2809463233 >> 172.21.15.113:0/2809463233 conn(0x7fc81406b720 msgr2=0x7fc8140d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:31.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.210+0000 7fc819fce640 1 -- 172.21.15.113:0/2809463233 shutdown_connections 2024-04-25T00:09:31.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.210+0000 7fc819fce640 1 -- 172.21.15.113:0/2809463233 wait complete. 2024-04-25T00:09:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.211+0000 7fc819fce640 1 Processor -- start 2024-04-25T00:09:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.211+0000 7fc819fce640 1 -- start start 2024-04-25T00:09:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc819fce640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8000052e0 0x7fc814161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc819fce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141544a0 0x7fc81414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc819fce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81415ff70 0x7fc8141603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc819fce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8140b7540 con 0x7fc81415ff70 2024-04-25T00:09:31.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc819fce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8140b73c0 con 0x7fc8141544a0 2024-04-25T00:09:31.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc819fce640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8140b5ca0 con 0x7fc8000052e0 2024-04-25T00:09:31.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81415ff70 0x7fc8141603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81415ff70 0x7fc8141603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42006/0 (socket says 172.21.15.113:42006) 2024-04-25T00:09:31.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 -- 172.21.15.113:0/116836358 learned_addr learned my addr 172.21.15.113:0/116836358 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:31.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc80bfff640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141544a0 0x7fc81414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc8137fe640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8000052e0 0x7fc814161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 -- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8000052e0 msgr2=0x7fc814161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8000052e0 0x7fc814161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 -- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141544a0 msgr2=0x7fc81414a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141544a0 0x7fc81414a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 -- 172.21.15.113:0/116836358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8140c7d40 con 0x7fc81415ff70 2024-04-25T00:09:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc813fff640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81415ff70 0x7fc8141603f0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005040 tx=0x7fc80c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc810ff9640 1 -- 172.21.15.113:0/116836358 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c007db0 con 0x7fc81415ff70 2024-04-25T00:09:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc810ff9640 1 -- 172.21.15.113:0/116836358 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80c041070 con 0x7fc81415ff70 2024-04-25T00:09:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8140c7fe0 con 0x7fc81415ff70 2024-04-25T00:09:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.212+0000 7fc810ff9640 1 -- 172.21.15.113:0/116836358 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c001a70 con 0x7fc81415ff70 2024-04-25T00:09:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.213+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8140c85a0 con 0x7fc81415ff70 2024-04-25T00:09:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.213+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8140691a0 con 0x7fc81415ff70 2024-04-25T00:09:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.214+0000 7fc810ff9640 1 -- 172.21.15.113:0/116836358 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc80c032660 con 0x7fc81415ff70 2024-04-25T00:09:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.214+0000 7fc810ff9640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7f406be60 0x7fc7f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.214+0000 7fc8137fe640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7f406be60 0x7fc7f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.214+0000 7fc810ff9640 1 -- 172.21.15.113:0/116836358 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 4857+0+0 (secure 0 0 0) 0x7fc80c0aa880 con 0x7fc81415ff70 2024-04-25T00:09:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.215+0000 7fc8137fe640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7f406be60 0x7fc7f406e320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc804001eb0 tx=0x7fc8040015b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.218+0000 7fc810ff9640 1 -- 172.21.15.113:0/116836358 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc80c039070 con 0x7fc81415ff70 2024-04-25T00:09:31.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.393+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc8140d1f30 con 0x7fc81415ff70 2024-04-25T00:09:31.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.843+0000 7fc810ff9640 1 -- 172.21.15.113:0/116836358 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v179) ==== 138+0+0 (secure 0 0 0) 0x7fc80c079580 con 0x7fc81415ff70 2024-04-25T00:09:31.844 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-25T00:09:31.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7f406be60 msgr2=0x7fc7f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7f406be60 0x7fc7f406e320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc804001eb0 tx=0x7fc8040015b0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81415ff70 msgr2=0x7fc8141603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81415ff70 0x7fc8141603f0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005040 tx=0x7fc80c0014a0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 shutdown_connections 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7f406be60 0x7fc7f406e320 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141544a0 0x7fc81414a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81415ff70 0x7fc8141603f0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 --2- 172.21.15.113:0/116836358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8000052e0 0x7fc814161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 >> 172.21.15.113:0/116836358 conn(0x7fc81406b720 msgr2=0x7fc81415d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 shutdown_connections 2024-04-25T00:09:31.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.847+0000 7fc819fce640 1 -- 172.21.15.113:0/116836358 wait complete. 2024-04-25T00:09:31.863 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:09:31.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.967+0000 7ff39b02c640 1 Processor -- start 2024-04-25T00:09:31.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.967+0000 7ff39b02c640 1 -- start start 2024-04-25T00:09:31.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff39b02c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff39413f050 0x7ff3940cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff39b02c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff3941606a0 0x7ff3940cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff39b02c640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3940d0970 con 0x7ff3940d04f0 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff39b02c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff39415c6a0 con 0x7ff3941606a0 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff39b02c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff394160aa0 con 0x7ff39413f050 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff3995c9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff3941606a0 0x7ff3940cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff398dc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff39413f050 0x7ff3940cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff398dc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff39413f050 0x7ff3940cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:59322/0 (socket says 172.21.15.113:59322) 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff393fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff3940d04f0 0x7ff39415ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:51804/0 (socket says 172.21.15.113:51804) 2024-04-25T00:09:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.968+0000 7ff398dc8640 1 -- 172.21.15.113:0/2305676985 learned_addr learned my addr 172.21.15.113:0/2305676985 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.969+0000 7ff3937fe640 1 -- 172.21.15.113:0/2305676985 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807366698 0 0) 0x7ff3940d0970 con 0x7ff3940d04f0 2024-04-25T00:09:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.969+0000 7ff3937fe640 1 -- 172.21.15.113:0/2305676985 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff37c003520 con 0x7ff3940d04f0 2024-04-25T00:09:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.969+0000 7ff398dc8640 1 -- 172.21.15.113:0/2305676985 >> v2:172.21.15.178:3300/0 conn(0x7ff3941606a0 msgr2=0x7ff3940cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.969+0000 7ff398dc8640 1 --2- 172.21.15.113:0/2305676985 >> v2:172.21.15.178:3300/0 conn(0x7ff3941606a0 0x7ff3940cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.969+0000 7ff398dc8640 1 -- 172.21.15.113:0/2305676985 >> v1:172.21.15.178:6789/0 conn(0x7ff3940d04f0 legacy=0x7ff39415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.969+0000 7ff398dc8640 1 -- 172.21.15.113:0/2305676985 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39415d6b0 con 0x7ff39413f050 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.969+0000 7ff3995c9640 1 --2- 172.21.15.113:0/2305676985 >> v2:172.21.15.178:3300/0 conn(0x7ff3941606a0 0x7ff3940cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff398dc8640 1 --2- 172.21.15.113:0/2305676985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff39413f050 0x7ff3940cffb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff384009870 tx=0x7ff38402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ab68fbe446bcc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff3937fe640 1 -- 172.21.15.113:0/2305676985 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384037070 con 0x7ff39413f050 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff3937fe640 1 -- 172.21.15.113:0/2305676985 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384032440 con 0x7ff39413f050 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff3937fe640 1 -- 172.21.15.113:0/2305676985 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840327f0 con 0x7ff39413f050 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 -- 172.21.15.113:0/2305676985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff39413f050 msgr2=0x7ff3940cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 --2- 172.21.15.113:0/2305676985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff39413f050 0x7ff3940cffb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff384009870 tx=0x7ff38402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 -- 172.21.15.113:0/2305676985 shutdown_connections 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 --2- 172.21.15.113:0/2305676985 >> v2:172.21.15.178:3300/0 conn(0x7ff3941606a0 0x7ff3940cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 --2- 172.21.15.113:0/2305676985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff39413f050 0x7ff3940cffb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 -- 172.21.15.113:0/2305676985 >> 172.21.15.113:0/2305676985 conn(0x7ff39406b0f0 msgr2=0x7ff39406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 -- 172.21.15.113:0/2305676985 shutdown_connections 2024-04-25T00:09:31.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.970+0000 7ff39b02c640 1 -- 172.21.15.113:0/2305676985 wait complete. 2024-04-25T00:09:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.971+0000 7ff39b02c640 1 Processor -- start 2024-04-25T00:09:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff39b02c640 1 -- start start 2024-04-25T00:09:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff39b02c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3940d04f0 0x7ff3940b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff39b02c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39413f050 0x7ff394155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff39b02c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3941606a0 0x7ff3941537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff39b02c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff39415ceb0 con 0x7ff39413f050 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff39b02c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff39415cbd0 con 0x7ff3941606a0 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff39b02c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff39415d190 con 0x7ff3940d04f0 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.972+0000 7ff393fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39413f050 0x7ff394155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39413f050 0x7ff394155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42016/0 (socket says 172.21.15.113:42016) 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 -- 172.21.15.113:0/1277679469 learned_addr learned my addr 172.21.15.113:0/1277679469 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff3995c9640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3941606a0 0x7ff3941537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff398dc8640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3940d04f0 0x7ff3940b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 -- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3940d04f0 msgr2=0x7ff3940b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3940d04f0 0x7ff3940b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 -- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3941606a0 msgr2=0x7ff3941537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3941606a0 0x7ff3941537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 -- 172.21.15.113:0/1277679469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff394154280 con 0x7ff39413f050 2024-04-25T00:09:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff3995c9640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3941606a0 0x7ff3941537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff398dc8640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3940d04f0 0x7ff3940b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff393fff640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39413f050 0x7ff394155320 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c430 tx=0x7ff38800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff3917fa640 1 -- 172.21.15.113:0/1277679469 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3880074e0 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff3917fa640 1 -- 172.21.15.113:0/1277679469 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3880047e0 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff3917fa640 1 -- 172.21.15.113:0/1277679469 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388004bd0 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff394154560 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.973+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff394154e00 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.974+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff394155000 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.975+0000 7ff3917fa640 1 -- 172.21.15.113:0/1277679469 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff38801a020 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.975+0000 7ff3917fa640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff38006be30 0x7ff38006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.975+0000 7ff398dc8640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff38006be30 0x7ff38006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.975+0000 7ff3917fa640 1 -- 172.21.15.113:0/1277679469 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 5227+0+0 (secure 0 0 0) 0x7ff38808a560 con 0x7ff39413f050 2024-04-25T00:09:31.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.976+0000 7ff398dc8640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff38006be30 0x7ff38006e2f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff384034000 tx=0x7ff38402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:31.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:31.980+0000 7ff3917fa640 1 -- 172.21.15.113:0/1277679469 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff394155000 con 0x7ff39413f050 2024-04-25T00:09:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.153+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff394154730 con 0x7ff39413f050 2024-04-25T00:09:32.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.850+0000 7ff3917fa640 1 -- 172.21.15.113:0/1277679469 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v180) ==== 170+0+0 (secure 0 0 0) 0x7ff3880591b0 con 0x7ff39413f050 2024-04-25T00:09:32.850 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-25T00:09:32.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.854+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff38006be30 msgr2=0x7ff38006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:32.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.854+0000 7ff39b02c640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff38006be30 0x7ff38006e2f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff384034000 tx=0x7ff38402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:32.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.854+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39413f050 msgr2=0x7ff394155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:32.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.854+0000 7ff39b02c640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39413f050 0x7ff394155320 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c430 tx=0x7ff38800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.857+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 shutdown_connections 2024-04-25T00:09:32.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.857+0000 7ff39b02c640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff38006be30 0x7ff38006e2f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.857+0000 7ff39b02c640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3941606a0 0x7ff3941537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.857+0000 7ff39b02c640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff39413f050 0x7ff394155320 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.857+0000 7ff39b02c640 1 --2- 172.21.15.113:0/1277679469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3940d04f0 0x7ff3940b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.857+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 >> 172.21.15.113:0/1277679469 conn(0x7ff39406b0f0 msgr2=0x7ff394151b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:32.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.864+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 shutdown_connections 2024-04-25T00:09:32.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.865+0000 7ff39b02c640 1 -- 172.21.15.113:0/1277679469 wait complete. 2024-04-25T00:09:32.883 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:09:32.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.987+0000 7f93dcf42640 1 Processor -- start 2024-04-25T00:09:32.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.987+0000 7f93dcf42640 1 -- start start 2024-04-25T00:09:32.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93dcf42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 0x7f93d80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93dcf42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f93d8071df0 0x7f93d815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93dcf42640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d806d900 con 0x7f93d80b1e00 2024-04-25T00:09:32.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93dcf42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f93d815fc40 con 0x7f93d8071df0 2024-04-25T00:09:32.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93dcf42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93d806e9a0 con 0x7f93d815e310 2024-04-25T00:09:32.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93d5d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f93d80b1e00 0x7f93d80716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38432/0 (socket says 172.21.15.113:38432) 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93d5d9b640 1 -- 172.21.15.113:0/3869733622 learned_addr learned my addr 172.21.15.113:0/3869733622 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93d659c640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 0x7f93d80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2240144860 0 0) 0x7f93d806d900 con 0x7f93d80b1e00 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93c0003520 con 0x7f93d80b1e00 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.988+0000 7f93d6d9d640 1 --2- 172.21.15.113:0/3869733622 >> v2:172.21.15.178:3300/0 conn(0x7f93d8071df0 0x7f93d815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3085811475 0 0) 0x7f93c0003520 con 0x7f93d80b1e00 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 msgr2=0x7f93d80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 0x7f93d80d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 >> v2:172.21.15.178:3300/0 conn(0x7f93d8071df0 msgr2=0x7f93d815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 --2- 172.21.15.113:0/3869733622 >> v2:172.21.15.178:3300/0 conn(0x7f93d8071df0 0x7f93d815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d806dbe0 con 0x7f93d80b1e00 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f93cc001670 con 0x7f93d80b1e00 2024-04-25T00:09:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 >> v1:172.21.15.113:6789/0 conn(0x7f93d80b1e00 legacy=0x7f93d80716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c00039b0 0x7f93c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 0x7f93c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c0004d40 0x7f93c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93d815fc40 con 0x7f93c00039b0 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93c000b9c0 con 0x7f93c0004d40 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93c000bbc0 con 0x7f93d815e310 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d659c640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c00039b0 0x7f93c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d5d9b640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 0x7f93c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d6d9d640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c0004d40 0x7f93c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d659c640 1 -- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 msgr2=0x7f93c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d659c640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93d815e310 0x7f93c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d659c640 1 -- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c0004d40 msgr2=0x7f93c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d659c640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c0004d40 0x7f93c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d659c640 1 -- 172.21.15.113:0/3869733622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c0003520 con 0x7f93c00039b0 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.989+0000 7f93d6d9d640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c0004d40 0x7f93c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.990+0000 7f93d659c640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c00039b0 0x7f93c0003dd0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009fc0 tx=0x7f93c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f57c2f47823e72a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.990+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8009c10 con 0x7f93c00039b0 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.990+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8036070 con 0x7f93c00039b0 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.990+0000 7f93d559a640 1 -- 172.21.15.113:0/3869733622 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8031880 con 0x7f93c00039b0 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.990+0000 7f93dcf42640 1 -- 172.21.15.113:0/3869733622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c00039b0 msgr2=0x7f93c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.990+0000 7f93dcf42640 1 --2- 172.21.15.113:0/3869733622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93c00039b0 0x7f93c0003dd0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009fc0 tx=0x7f93c0005830 comp rx=0 tx=0).stop 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.990+0000 7f93d6d9d640 1 -- 172.21.15.113:0/3869733622 reap_dead start 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.991+0000 7f93dcf42640 1 -- 172.21.15.113:0/3869733622 shutdown_connections 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.991+0000 7f93dcf42640 1 -- 172.21.15.113:0/3869733622 >> 172.21.15.113:0/3869733622 conn(0x7f93d806fb60 msgr2=0x7f93d806b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.991+0000 7f93dcf42640 1 -- 172.21.15.113:0/3869733622 shutdown_connections 2024-04-25T00:09:32.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.991+0000 7f93dcf42640 1 -- 172.21.15.113:0/3869733622 wait complete. 2024-04-25T00:09:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.992+0000 7f93dcf42640 1 Processor -- start 2024-04-25T00:09:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.992+0000 7f93dcf42640 1 -- start start 2024-04-25T00:09:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93c00039b0 0x7f93d8153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93d815c1c0 0x7f93d814b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d815ee60 0x7f93d815f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93d806e1a0 con 0x7f93d815c1c0 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93d806e510 con 0x7f93d815ee60 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93d80ceb20 con 0x7f93c00039b0 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93d815c1c0 0x7f93d814b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93d815c1c0 0x7f93d814b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42046/0 (socket says 172.21.15.113:42046) 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 -- 172.21.15.113:0/2658244669 learned_addr learned my addr 172.21.15.113:0/2658244669 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d659c640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93c00039b0 0x7f93d8153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d6d9d640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d815ee60 0x7f93d815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 -- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93c00039b0 msgr2=0x7f93d8153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93c00039b0 0x7f93d8153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 -- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d815ee60 msgr2=0x7f93d815f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d815ee60 0x7f93d815f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 -- 172.21.15.113:0/2658244669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d8153eb0 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93d5d9b640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93d815c1c0 0x7f93d814b530 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f93cc007310 tx=0x7f93cc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93c6ffd640 1 -- 172.21.15.113:0/2658244669 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00f9c0 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93c6ffd640 1 -- 172.21.15.113:0/2658244669 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93cc018070 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d8154190 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93c6ffd640 1 -- 172.21.15.113:0/2658244669 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00fda0 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.993+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d8154750 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.994+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d806c080 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.995+0000 7f93c6ffd640 1 -- 172.21.15.113:0/2658244669 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93cc01d460 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.995+0000 7f93c6ffd640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93b806be60 0x7f93b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.995+0000 7f93c6ffd640 1 -- 172.21.15.113:0/2658244669 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(180..180 src has 1..180) ==== 5609+0+0 (secure 0 0 0) 0x7f93cc08af00 con 0x7f93d815c1c0 2024-04-25T00:09:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.995+0000 7f93d659c640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93b806be60 0x7f93b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.995+0000 7f93d659c640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93b806be60 0x7f93b806e320 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f93c8035000 tx=0x7f93c802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:32.999+0000 7f93c6ffd640 1 -- 172.21.15.113:0/2658244669 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93cc059910 con 0x7f93d815c1c0 2024-04-25T00:09:33.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.213+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f93d81f4200 con 0x7f93d815c1c0 2024-04-25T00:09:33.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.859+0000 7f93c6ffd640 1 -- 172.21.15.113:0/2658244669 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v96) ==== 174+0+0 (secure 0 0 0) 0x7f93d81f4200 con 0x7f93d815c1c0 2024-04-25T00:09:33.860 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 26 and data pool 27 2024-04-25T00:09:33.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.866+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93b806be60 msgr2=0x7f93b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:33.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.866+0000 7f93dcf42640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93b806be60 0x7f93b806e320 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f93c8035000 tx=0x7f93c802f000 comp rx=0 tx=0).stop 2024-04-25T00:09:33.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.866+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93d815c1c0 msgr2=0x7f93d814b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:33.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.866+0000 7f93dcf42640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93d815c1c0 0x7f93d814b530 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f93cc007310 tx=0x7f93cc0077d0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.871+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 shutdown_connections 2024-04-25T00:09:33.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.871+0000 7f93dcf42640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f93b806be60 0x7f93b806e320 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.871+0000 7f93dcf42640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d815ee60 0x7f93d815f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.871+0000 7f93dcf42640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93d815c1c0 0x7f93d814b530 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.871+0000 7f93dcf42640 1 --2- 172.21.15.113:0/2658244669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93c00039b0 0x7f93d8153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.871+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 >> 172.21.15.113:0/2658244669 conn(0x7f93d806fb60 msgr2=0x7f93d80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:33.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.874+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 shutdown_connections 2024-04-25T00:09:33.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.874+0000 7f93dcf42640 1 -- 172.21.15.113:0/2658244669 wait complete. 2024-04-25T00:09:33.891 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:33.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.989+0000 7fb3488d6640 1 Processor -- start 2024-04-25T00:09:33.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.989+0000 7fb3488d6640 1 -- start start 2024-04-25T00:09:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb3488d6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb34406e6e0 0x7fb3440b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb3488d6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3440cfad0 con 0x7fb344144d60 2024-04-25T00:09:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb3488d6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3440cf230 con 0x7fb3441453c0 2024-04-25T00:09:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb3488d6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb344145c40 con 0x7fb34406e6e0 2024-04-25T00:09:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb34259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb344144d60 0x7fb34406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38444/0 (socket says 172.21.15.113:38444) 2024-04-25T00:09:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb34259c640 1 -- 172.21.15.113:0/1641474134 learned_addr learned my addr 172.21.15.113:0/1641474134 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb341d9b640 1 --2- 172.21.15.113:0/1641474134 >> v2:172.21.15.113:3300/0 conn(0x7fb34406e6e0 0x7fb3440b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.990+0000 7fb34159a640 1 -- 172.21.15.113:0/1641474134 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596537428 0 0) 0x7fb3440cfad0 con 0x7fb344144d60 2024-04-25T00:09:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb34159a640 1 -- 172.21.15.113:0/1641474134 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb32c003520 con 0x7fb344144d60 2024-04-25T00:09:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb341d9b640 1 -- 172.21.15.113:0/1641474134 >> v1:172.21.15.178:6789/0 conn(0x7fb3441453c0 legacy=0x7fb344071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb341d9b640 1 -- 172.21.15.113:0/1641474134 >> v1:172.21.15.113:6789/0 conn(0x7fb344144d60 legacy=0x7fb34406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb341d9b640 1 -- 172.21.15.113:0/1641474134 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3440cf510 con 0x7fb34406e6e0 2024-04-25T00:09:33.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb341d9b640 1 --2- 172.21.15.113:0/1641474134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb34406e6e0 0x7fb3440b5390 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fb3380097a0 tx=0x7fb33802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31b16360237d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb34159a640 1 -- 172.21.15.113:0/1641474134 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338037070 con 0x7fb34406e6e0 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb34159a640 1 -- 172.21.15.113:0/1641474134 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33802adc0 con 0x7fb34406e6e0 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.991+0000 7fb34159a640 1 -- 172.21.15.113:0/1641474134 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338032640 con 0x7fb34406e6e0 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.992+0000 7fb3488d6640 1 -- 172.21.15.113:0/1641474134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb34406e6e0 msgr2=0x7fb3440b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.992+0000 7fb3488d6640 1 --2- 172.21.15.113:0/1641474134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb34406e6e0 0x7fb3440b5390 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fb3380097a0 tx=0x7fb33802a0b0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.992+0000 7fb3488d6640 1 -- 172.21.15.113:0/1641474134 shutdown_connections 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.992+0000 7fb3488d6640 1 --2- 172.21.15.113:0/1641474134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb34406e6e0 0x7fb3440b5390 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.992+0000 7fb3488d6640 1 -- 172.21.15.113:0/1641474134 >> 172.21.15.113:0/1641474134 conn(0x7fb3440b1a50 msgr2=0x7fb3440b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.992+0000 7fb3488d6640 1 -- 172.21.15.113:0/1641474134 shutdown_connections 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.992+0000 7fb3488d6640 1 -- 172.21.15.113:0/1641474134 wait complete. 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.993+0000 7fb3488d6640 1 Processor -- start 2024-04-25T00:09:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.993+0000 7fb3488d6640 1 -- start start 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb3488d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb34406e6e0 0x7fb344159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb3488d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb344144d60 0x7fb3441586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb3488d6640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3441453c0 0x7fb3441f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb3488d6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb344145140 con 0x7fb344144d60 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb3488d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb344070e10 con 0x7fb34406e6e0 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb3488d6640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb344071a40 con 0x7fb3441453c0 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb344144d60 0x7fb3441586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb344144d60 0x7fb3441586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42062/0 (socket says 172.21.15.113:42062) 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 -- 172.21.15.113:0/3402096433 learned_addr learned my addr 172.21.15.113:0/3402096433 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb34259c640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb34406e6e0 0x7fb344159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb342d9d640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3441453c0 0x7fb3441f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 -- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3441453c0 msgr2=0x7fb3441f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3441453c0 0x7fb3441f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 -- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb34406e6e0 msgr2=0x7fb344159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb34406e6e0 0x7fb344159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb341d9b640 1 -- 172.21.15.113:0/3402096433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3441f2320 con 0x7fb344144d60 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.994+0000 7fb34259c640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb34406e6e0 0x7fb344159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.995+0000 7fb341d9b640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb344144d60 0x7fb3441586e0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fb338000c00 tx=0x7fb3380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.995+0000 7fb332ffd640 1 -- 172.21.15.113:0/3402096433 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338041070 con 0x7fb344144d60 2024-04-25T00:09:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.995+0000 7fb332ffd640 1 -- 172.21.15.113:0/3402096433 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33803c460 con 0x7fb344144d60 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.995+0000 7fb332ffd640 1 -- 172.21.15.113:0/3402096433 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338037040 con 0x7fb344144d60 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.995+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3441f2600 con 0x7fb344144d60 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.995+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb344159a70 con 0x7fb344144d60 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.996+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3440691a0 con 0x7fb344144d60 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.996+0000 7fb332ffd640 1 -- 172.21.15.113:0/3402096433 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb338044070 con 0x7fb344144d60 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.996+0000 7fb332ffd640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb32406be30 0x7fb32406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.997+0000 7fb332ffd640 1 -- 172.21.15.113:0/3402096433 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 5709+0+0 (secure 0 0 0) 0x7fb3380ab4b0 con 0x7fb344144d60 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.997+0000 7fb34259c640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb32406be30 0x7fb32406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:33.997+0000 7fb34259c640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb32406be30 0x7fb32406e2f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb334005c00 tx=0x7fb334009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.001+0000 7fb332ffd640 1 -- 172.21.15.113:0/3402096433 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb338079ef0 con 0x7fb344144d60 2024-04-25T00:09:34.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.176+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3440b27e0 con 0x7fb344144d60 2024-04-25T00:09:34.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.177+0000 7fb332ffd640 1 -- 172.21.15.113:0/3402096433 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) ==== 74+0+16744 (secure 0 0 0) 0x7fb3380797f0 con 0x7fb344144d60 2024-04-25T00:09:34.178 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:34.178 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":181,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:33.214616+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:31.412384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:32.173467+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:34.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.179+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb32406be30 msgr2=0x7fb32406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.179+0000 7fb3488d6640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb32406be30 0x7fb32406e2f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb334005c00 tx=0x7fb334009040 comp rx=0 tx=0).stop 2024-04-25T00:09:34.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.179+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb344144d60 msgr2=0x7fb3441586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.179+0000 7fb3488d6640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb344144d60 0x7fb3441586e0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fb338000c00 tx=0x7fb3380046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 shutdown_connections 2024-04-25T00:09:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb32406be30 0x7fb32406e2f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3441453c0 0x7fb3441f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb344144d60 0x7fb3441586e0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 --2- 172.21.15.113:0/3402096433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb34406e6e0 0x7fb344159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 >> 172.21.15.113:0/3402096433 conn(0x7fb3440b1a50 msgr2=0x7fb3440ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 shutdown_connections 2024-04-25T00:09:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.180+0000 7fb3488d6640 1 -- 172.21.15.113:0/3402096433 wait complete. 2024-04-25T00:09:34.191 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:34.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.287+0000 7f0f82cda640 1 Processor -- start 2024-04-25T00:09:34.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.287+0000 7f0f82cda640 1 -- start start 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f82cda640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f7c144910 0x7f0f7c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f82cda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c0b5240 0x7f0f7c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f82cda640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0f7c0b5680 0x7f0f7c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f82cda640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7c0b4510 con 0x7f0f7c144910 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f82cda640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7c144cf0 con 0x7f0f7c0b5680 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f82cda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7c0b2f30 con 0x7f0f7c0b5240 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f7c144910 0x7f0f7c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f7c144910 0x7f0f7c0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42074/0 (socket says 172.21.15.113:42074) 2024-04-25T00:09:34.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 -- 172.21.15.113:0/3272175967 learned_addr learned my addr 172.21.15.113:0/3272175967 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:34.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 -- 172.21.15.113:0/3272175967 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c0b5240 msgr2=0x7f0f7c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:34.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 --2- 172.21.15.113:0/3272175967 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c0b5240 0x7f0f7c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 -- 172.21.15.113:0/3272175967 >> v2:172.21.15.178:3300/0 conn(0x7f0f7c0b5680 msgr2=0x7f0f7c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:34.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 --2- 172.21.15.113:0/3272175967 >> v2:172.21.15.178:3300/0 conn(0x7f0f7c0b5680 0x7f0f7c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.288+0000 7f0f80a76640 1 -- 172.21.15.113:0/3272175967 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c0b3210 con 0x7f0f7c144910 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.289+0000 7f0f80a76640 1 --2- 172.21.15.113:0/3272175967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c144910 0x7f0f7c0b3fd0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c062820 tx=0x7f0f6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf52d070aeafdf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.289+0000 7f0f7b7fe640 1 -- 172.21.15.113:0/3272175967 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c037070 con 0x7f0f7c144910 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.289+0000 7f0f7b7fe640 1 -- 172.21.15.113:0/3272175967 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c032440 con 0x7f0f7c144910 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.289+0000 7f0f7b7fe640 1 -- 172.21.15.113:0/3272175967 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c032810 con 0x7f0f7c144910 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 -- 172.21.15.113:0/3272175967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c144910 msgr2=0x7f0f7c0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 --2- 172.21.15.113:0/3272175967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c144910 0x7f0f7c0b3fd0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c062820 tx=0x7f0f6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 -- 172.21.15.113:0/3272175967 shutdown_connections 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 --2- 172.21.15.113:0/3272175967 >> v2:172.21.15.178:3300/0 conn(0x7f0f7c0b5680 0x7f0f7c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 --2- 172.21.15.113:0/3272175967 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c0b5240 0x7f0f7c06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 --2- 172.21.15.113:0/3272175967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c144910 0x7f0f7c0b3fd0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 -- 172.21.15.113:0/3272175967 >> 172.21.15.113:0/3272175967 conn(0x7f0f7c0b1a50 msgr2=0x7f0f7c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 -- 172.21.15.113:0/3272175967 shutdown_connections 2024-04-25T00:09:34.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.290+0000 7f0f82cda640 1 -- 172.21.15.113:0/3272175967 wait complete. 2024-04-25T00:09:34.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.291+0000 7f0f82cda640 1 Processor -- start 2024-04-25T00:09:34.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.291+0000 7f0f82cda640 1 -- start start 2024-04-25T00:09:34.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f82cda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c0b5240 0x7f0f7c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f82cda640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f7c0b5680 0x7f0f7c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f82cda640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c144910 0x7f0f7c13f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f82cda640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c0b4da0 con 0x7f0f7c0b5240 2024-04-25T00:09:34.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f82cda640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c0b37d0 con 0x7f0f7c0b5680 2024-04-25T00:09:34.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f82cda640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7c0b4f20 con 0x7f0f7c144910 2024-04-25T00:09:34.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c0b5240 0x7f0f7c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c0b5240 0x7f0f7c149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42082/0 (socket says 172.21.15.113:42082) 2024-04-25T00:09:34.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 -- 172.21.15.113:0/1196284656 learned_addr learned my addr 172.21.15.113:0/1196284656 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f7bfff640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f7c0b5680 0x7f0f7c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f81277640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c144910 0x7f0f7c13f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 -- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c144910 msgr2=0x7f0f7c13f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c144910 0x7f0f7c13f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 -- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f7c0b5680 msgr2=0x7f0f7c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f7c0b5680 0x7f0f7c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 -- 172.21.15.113:0/1196284656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c1e55d0 con 0x7f0f7c0b5240 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f80a76640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c0b5240 0x7f0f7c149bc0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c000c00 tx=0x7f0f6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f797fa640 1 -- 172.21.15.113:0/1196284656 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c041070 con 0x7f0f7c0b5240 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f797fa640 1 -- 172.21.15.113:0/1196284656 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c03c870 con 0x7f0f7c0b5240 2024-04-25T00:09:34.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.292+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7c1e58b0 con 0x7f0f7c0b5240 2024-04-25T00:09:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.293+0000 7f0f797fa640 1 -- 172.21.15.113:0/1196284656 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c037040 con 0x7f0f7c0b5240 2024-04-25T00:09:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.293+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7c1e5e70 con 0x7f0f7c0b5240 2024-04-25T00:09:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.294+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7c1e6070 con 0x7f0f7c0b5240 2024-04-25T00:09:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.294+0000 7f0f797fa640 1 -- 172.21.15.113:0/1196284656 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f6c044070 con 0x7f0f7c0b5240 2024-04-25T00:09:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.294+0000 7f0f797fa640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f6806be30 0x7f0f6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.294+0000 7f0f797fa640 1 -- 172.21.15.113:0/1196284656 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 5709+0+0 (secure 0 0 0) 0x7f0f6c0abc00 con 0x7f0f7c0b5240 2024-04-25T00:09:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.294+0000 7f0f7bfff640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f6806be30 0x7f0f6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.295+0000 7f0f7bfff640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f6806be30 0x7f0f6806e2f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c159320 tx=0x7f0f70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.299+0000 7f0f797fa640 1 -- 172.21.15.113:0/1196284656 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f7c1e6070 con 0x7f0f7c0b5240 2024-04-25T00:09:34.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.473+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f7c0731a0 con 0x7f0f7c0b5240 2024-04-25T00:09:34.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.474+0000 7f0f797fa640 1 -- 172.21.15.113:0/1196284656 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) ==== 74+0+16744 (secure 0 0 0) 0x7f0f6c0b2080 con 0x7f0f7c0b5240 2024-04-25T00:09:34.474 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:34.475 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":181,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:33.214616+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:31.412384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:32.173467+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:34.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f6806be30 msgr2=0x7f0f6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f6806be30 0x7f0f6806e2f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c159320 tx=0x7f0f70008040 comp rx=0 tx=0).stop 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c0b5240 msgr2=0x7f0f7c149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c0b5240 0x7f0f7c149bc0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c000c00 tx=0x7f0f6c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 shutdown_connections 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f6806be30 0x7f0f6806e2f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f7c0b5680 0x7f0f7c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f7c144910 0x7f0f7c13f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 --2- 172.21.15.113:0/1196284656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f7c0b5240 0x7f0f7c149bc0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 >> 172.21.15.113:0/1196284656 conn(0x7f0f7c0b1a50 msgr2=0x7f0f7c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 shutdown_connections 2024-04-25T00:09:34.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.476+0000 7f0f82cda640 1 -- 172.21.15.113:0/1196284656 wait complete. 2024-04-25T00:09:34.488 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:09:34.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.586+0000 7fe5e59ee640 1 Processor -- start 2024-04-25T00:09:34.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.586+0000 7fe5e59ee640 1 -- start start 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5e59ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0161fd0 0x7fe5e006c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5e59ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5e0061b80 0x7fe5e0062010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5e59ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5e015ea00 con 0x7fe5e00cd4b0 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5e59ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe5e00cfc30 con 0x7fe5e0061b80 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5e59ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e0162490 con 0x7fe5e0161fd0 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5df7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5e0061b80 0x7fe5e0062010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5df7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5e0061b80 0x7fe5e0062010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42084/0 (socket says 172.21.15.113:42084) 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1170917441 learned_addr learned my addr 172.21.15.113:0/1170917441 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1170917441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0161fd0 msgr2=0x7fe5e006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5de7fc640 1 --2- 172.21.15.113:0/1170917441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0161fd0 0x7fe5e006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5df7fe640 1 --2- 172.21.15.113:0/1170917441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0161fd0 0x7fe5e006c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.587+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1170917441 >> v1:172.21.15.178:6789/0 conn(0x7fe5e00cd4b0 legacy=0x7fe5e00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:34.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.588+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1170917441 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e0162770 con 0x7fe5e0061b80 2024-04-25T00:09:34.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.588+0000 7fe5df7fe640 1 --2- 172.21.15.113:0/1170917441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0061b80 0x7fe5e0062010 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009870 tx=0x7fe5d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574a7953d79e6e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.588+0000 7fe5ddffb640 1 -- 172.21.15.113:0/1170917441 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8037070 con 0x7fe5e0061b80 2024-04-25T00:09:34.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.588+0000 7fe5ddffb640 1 -- 172.21.15.113:0/1170917441 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d8032440 con 0x7fe5e0061b80 2024-04-25T00:09:34.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.588+0000 7fe5ddffb640 1 -- 172.21.15.113:0/1170917441 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8032810 con 0x7fe5e0061b80 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1170917441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0061b80 msgr2=0x7fe5e0062010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1170917441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0061b80 0x7fe5e0062010 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009870 tx=0x7fe5d802a210 comp rx=0 tx=0).stop 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1170917441 shutdown_connections 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1170917441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0161fd0 0x7fe5e006c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1170917441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0061b80 0x7fe5e0062010 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1170917441 >> 172.21.15.113:0/1170917441 conn(0x7fe5e00d0e30 msgr2=0x7fe5e00d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1170917441 shutdown_connections 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.589+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1170917441 wait complete. 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.590+0000 7fe5e59ee640 1 Processor -- start 2024-04-25T00:09:34.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.590+0000 7fe5e59ee640 1 -- start start 2024-04-25T00:09:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5e59ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0061b80 0x7fe5e01610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5e59ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e00cd4b0 0x7fe5e014f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5e59ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0161fd0 0x7fe5e00b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5e59ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00cf1d0 con 0x7fe5e0161fd0 2024-04-25T00:09:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5e59ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0162d30 con 0x7fe5e00cd4b0 2024-04-25T00:09:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5e59ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e015eee0 con 0x7fe5e0061b80 2024-04-25T00:09:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0161fd0 0x7fe5e00b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0161fd0 0x7fe5e00b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42094/0 (socket says 172.21.15.113:42094) 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1321691273 learned_addr learned my addr 172.21.15.113:0/1321691273 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1321691273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0061b80 msgr2=0x7fe5e01610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0061b80 0x7fe5e01610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1321691273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e00cd4b0 msgr2=0x7fe5e014f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e00cd4b0 0x7fe5e014f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 -- 172.21.15.113:0/1321691273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e015fd20 con 0x7fe5e0161fd0 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.591+0000 7fe5df7fe640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0161fd0 0x7fe5e00b17e0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009870 tx=0x7fe5d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.592+0000 7fe5c7fff640 1 -- 172.21.15.113:0/1321691273 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8037040 con 0x7fe5e0161fd0 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.592+0000 7fe5c7fff640 1 -- 172.21.15.113:0/1321691273 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d8042070 con 0x7fe5e0161fd0 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.592+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e0160000 con 0x7fe5e0161fd0 2024-04-25T00:09:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.592+0000 7fe5c7fff640 1 -- 172.21.15.113:0/1321691273 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8030bd0 con 0x7fe5e0161fd0 2024-04-25T00:09:34.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.592+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01605c0 con 0x7fe5e0161fd0 2024-04-25T00:09:34.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.593+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e00691a0 con 0x7fe5e0161fd0 2024-04-25T00:09:34.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.593+0000 7fe5c7fff640 1 -- 172.21.15.113:0/1321691273 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5d8047460 con 0x7fe5e0161fd0 2024-04-25T00:09:34.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.594+0000 7fe5c7fff640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5cc06be30 0x7fe5cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.594+0000 7fe5deffd640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5cc06be30 0x7fe5cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.594+0000 7fe5c7fff640 1 -- 172.21.15.113:0/1321691273 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 5709+0+0 (secure 0 0 0) 0x7fe5d80abb10 con 0x7fe5e0161fd0 2024-04-25T00:09:34.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.594+0000 7fe5deffd640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5cc06be30 0x7fe5cc06e2f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00043c0 tx=0x7fe5d0007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.598+0000 7fe5c7fff640 1 -- 172.21.15.113:0/1321691273 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe5d8045360 con 0x7fe5e0161fd0 2024-04-25T00:09:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.807+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe5e006cee0 con 0x7fe5e0161fd0 2024-04-25T00:09:34.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.866+0000 7fe5c7fff640 1 -- 172.21.15.113:0/1321691273 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) ==== 118+0+0 (secure 0 0 0) 0x7fe5d807a4c0 con 0x7fe5e0161fd0 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5cc06be30 msgr2=0x7fe5cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5cc06be30 0x7fe5cc06e2f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00043c0 tx=0x7fe5d0007d30 comp rx=0 tx=0).stop 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0161fd0 msgr2=0x7fe5e00b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0161fd0 0x7fe5e00b17e0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009870 tx=0x7fe5d80046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 shutdown_connections 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5cc06be30 0x7fe5cc06e2f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e00cd4b0 0x7fe5e014f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0161fd0 0x7fe5e00b17e0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 --2- 172.21.15.113:0/1321691273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5e0061b80 0x7fe5e01610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 >> 172.21.15.113:0/1321691273 conn(0x7fe5e00d0e30 msgr2=0x7fe5e00d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:34.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 shutdown_connections 2024-04-25T00:09:34.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.868+0000 7fe5e59ee640 1 -- 172.21.15.113:0/1321691273 wait complete. 2024-04-25T00:09:34.879 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:09:34.880 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:09:34.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.979+0000 7fd5223ee640 1 Processor -- start 2024-04-25T00:09:34.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.979+0000 7fd5223ee640 1 -- start start 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.980+0000 7fd5223ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd51c0b3f20 0x7fd51c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.980+0000 7fd5223ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51c145ee0 con 0x7fd51c145b00 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.980+0000 7fd5223ee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51c0b58a0 con 0x7fd51c0b3ae0 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.980+0000 7fd5223ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd51c0b64b0 con 0x7fd51c0b3f20 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.980+0000 7fd51bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd51c145b00 0x7fd51c0b4e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38456/0 (socket says 172.21.15.113:38456) 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.980+0000 7fd51bfff640 1 -- 172.21.15.113:0/1949984381 learned_addr learned my addr 172.21.15.113:0/1949984381 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.980+0000 7fd52098b640 1 --2- 172.21.15.113:0/1949984381 >> v2:172.21.15.113:3300/0 conn(0x7fd51c0b3f20 0x7fd51c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd51affd640 1 -- 172.21.15.113:0/1949984381 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789195780 0 0) 0x7fd51c145ee0 con 0x7fd51c145b00 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd51affd640 1 -- 172.21.15.113:0/1949984381 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd504003520 con 0x7fd51c145b00 2024-04-25T00:09:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd52098b640 1 -- 172.21.15.113:0/1949984381 >> v1:172.21.15.178:6789/0 conn(0x7fd51c0b3ae0 legacy=0x7fd51c0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd52098b640 1 -- 172.21.15.113:0/1949984381 >> v1:172.21.15.113:6789/0 conn(0x7fd51c145b00 legacy=0x7fd51c0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd52098b640 1 -- 172.21.15.113:0/1949984381 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c0b5b80 con 0x7fd51c0b3f20 2024-04-25T00:09:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd52098b640 1 --2- 172.21.15.113:0/1949984381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c0d0fa0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fd514009d90 tx=0x7fd51402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed2de8710d67a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd51affd640 1 -- 172.21.15.113:0/1949984381 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514031600 con 0x7fd51c0b3f20 2024-04-25T00:09:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.981+0000 7fd51affd640 1 -- 172.21.15.113:0/1949984381 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5140317a0 con 0x7fd51c0b3f20 2024-04-25T00:09:34.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd51affd640 1 -- 172.21.15.113:0/1949984381 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514031b70 con 0x7fd51c0b3f20 2024-04-25T00:09:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd5223ee640 1 -- 172.21.15.113:0/1949984381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 msgr2=0x7fd51c0d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd5223ee640 1 --2- 172.21.15.113:0/1949984381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c0d0fa0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fd514009d90 tx=0x7fd51402a610 comp rx=0 tx=0).stop 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd5223ee640 1 -- 172.21.15.113:0/1949984381 shutdown_connections 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd5223ee640 1 --2- 172.21.15.113:0/1949984381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c0d0fa0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd5223ee640 1 -- 172.21.15.113:0/1949984381 >> 172.21.15.113:0/1949984381 conn(0x7fd51c0b7370 msgr2=0x7fd51c0b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd5223ee640 1 -- 172.21.15.113:0/1949984381 shutdown_connections 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.982+0000 7fd5223ee640 1 -- 172.21.15.113:0/1949984381 wait complete. 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.983+0000 7fd5223ee640 1 Processor -- start 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.983+0000 7fd5223ee640 1 -- start start 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd5223ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0b3ae0 0x7fd51c1e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd5223ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd5223ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd51c145b00 0x7fd51c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd5223ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c0b5560 con 0x7fd51c0b3f20 2024-04-25T00:09:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd5223ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c0b6140 con 0x7fd51c0b3ae0 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd5223ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c0b4370 con 0x7fd51c145b00 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42122/0 (socket says 172.21.15.113:42122) 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 -- 172.21.15.113:0/315052168 learned_addr learned my addr 172.21.15.113:0/315052168 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51bfff640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0b3ae0 0x7fd51c1e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd52098b640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd51c145b00 0x7fd51c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 -- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd51c145b00 msgr2=0x7fd51c1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd51c145b00 0x7fd51c1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 -- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0b3ae0 msgr2=0x7fd51c1e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0b3ae0 0x7fd51c1e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51b7fe640 1 -- 172.21.15.113:0/315052168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c153aa0 con 0x7fd51c0b3f20 2024-04-25T00:09:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd52098b640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd51c145b00 0x7fd51c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.984+0000 7fd51bfff640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0b3ae0 0x7fd51c1e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.985+0000 7fd51b7fe640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c152480 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b420 tx=0x7fd51000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.985+0000 7fd518ff9640 1 -- 172.21.15.113:0/315052168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510011830 con 0x7fd51c0b3f20 2024-04-25T00:09:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.985+0000 7fd518ff9640 1 -- 172.21.15.113:0/315052168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd510016070 con 0x7fd51c0b3f20 2024-04-25T00:09:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.985+0000 7fd518ff9640 1 -- 172.21.15.113:0/315052168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510011c10 con 0x7fd51c0b3f20 2024-04-25T00:09:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.985+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c153d80 con 0x7fd51c0b3f20 2024-04-25T00:09:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.985+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c154340 con 0x7fd51c0b3f20 2024-04-25T00:09:34.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.986+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c0691a0 con 0x7fd51c0b3f20 2024-04-25T00:09:34.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.986+0000 7fd518ff9640 1 -- 172.21.15.113:0/315052168 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd51001b460 con 0x7fd51c0b3f20 2024-04-25T00:09:34.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.987+0000 7fd518ff9640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be30 0x7fd50806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:34.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.987+0000 7fd51bfff640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be30 0x7fd50806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:34.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.987+0000 7fd518ff9640 1 -- 172.21.15.113:0/315052168 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7fd51008a8e0 con 0x7fd51c0b3f20 2024-04-25T00:09:34.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.987+0000 7fd51bfff640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be30 0x7fd50806e2f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005880 tx=0x7fd50c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:34.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:34.991+0000 7fd518ff9640 1 -- 172.21.15.113:0/315052168 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd510020020 con 0x7fd51c0b3f20 2024-04-25T00:09:35.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.198+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd51c06ffb0 con 0x7fd51c0b3f20 2024-04-25T00:09:35.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.873+0000 7fd518ff9640 1 -- 172.21.15.113:0/315052168 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) ==== 122+0+0 (secure 0 0 0) 0x7fd510059320 con 0x7fd51c0b3f20 2024-04-25T00:09:35.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be30 msgr2=0x7fd50806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be30 0x7fd50806e2f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005880 tx=0x7fd50c009040 comp rx=0 tx=0).stop 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 msgr2=0x7fd51c152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c152480 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b420 tx=0x7fd51000b8e0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 shutdown_connections 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd50806be30 0x7fd50806e2f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd51c145b00 0x7fd51c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0b3f20 0x7fd51c152480 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 --2- 172.21.15.113:0/315052168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0b3ae0 0x7fd51c1e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 >> 172.21.15.113:0/315052168 conn(0x7fd51c0b7370 msgr2=0x7fd51c15d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 shutdown_connections 2024-04-25T00:09:35.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.875+0000 7fd5223ee640 1 -- 172.21.15.113:0/315052168 wait complete. 2024-04-25T00:09:35.887 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:09:35.888 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:35.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.986+0000 7f7590a64640 1 Processor -- start 2024-04-25T00:09:35.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.986+0000 7f7590a64640 1 -- start start 2024-04-25T00:09:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7590a64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f758c0d2290 0x7f758c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7590a64640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f758c0d0b00 0x7f758c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7590a64640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758c0d1480 con 0x7f758c15e320 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7590a64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f758c0d3410 con 0x7f758c0d2290 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7590a64640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f758c0d2800 con 0x7f758c0d0b00 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7589d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f758c0d2290 0x7f758c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7589d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f758c0d2290 0x7f758c0b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42126/0 (socket says 172.21.15.113:42126) 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f7589d9b640 1 -- 172.21.15.113:0/2613957207 learned_addr learned my addr 172.21.15.113:0/2613957207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.987+0000 7f758ad9d640 1 --2- 172.21.15.113:0/2613957207 >> v2:172.21.15.178:3300/0 conn(0x7f758c0d0b00 0x7f758c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f7589d9b640 1 -- 172.21.15.113:0/2613957207 >> v2:172.21.15.178:3300/0 conn(0x7f758c0d0b00 msgr2=0x7f758c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f7589d9b640 1 --2- 172.21.15.113:0/2613957207 >> v2:172.21.15.178:3300/0 conn(0x7f758c0d0b00 0x7f758c0d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f7589d9b640 1 -- 172.21.15.113:0/2613957207 >> v1:172.21.15.178:6789/0 conn(0x7f758c15e320 legacy=0x7f758c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f7589d9b640 1 -- 172.21.15.113:0/2613957207 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c0d2ae0 con 0x7f758c0d2290 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f7589d9b640 1 --2- 172.21.15.113:0/2613957207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c0b9aa0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f758c0b3970 tx=0x7f758002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9358e5f895630604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f758959a640 1 -- 172.21.15.113:0/2613957207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580037070 con 0x7f758c0d2290 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f758959a640 1 -- 172.21.15.113:0/2613957207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7580032440 con 0x7f758c0d2290 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.988+0000 7f758959a640 1 -- 172.21.15.113:0/2613957207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580032810 con 0x7f758c0d2290 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 -- 172.21.15.113:0/2613957207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 msgr2=0x7f758c0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 --2- 172.21.15.113:0/2613957207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c0b9aa0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f758c0b3970 tx=0x7f758002a2c0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 -- 172.21.15.113:0/2613957207 shutdown_connections 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 --2- 172.21.15.113:0/2613957207 >> v2:172.21.15.178:3300/0 conn(0x7f758c0d0b00 0x7f758c0d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 --2- 172.21.15.113:0/2613957207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c0b9aa0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 -- 172.21.15.113:0/2613957207 >> 172.21.15.113:0/2613957207 conn(0x7f758c0b5c20 msgr2=0x7f758c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 -- 172.21.15.113:0/2613957207 shutdown_connections 2024-04-25T00:09:35.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.989+0000 7f7590a64640 1 -- 172.21.15.113:0/2613957207 wait complete. 2024-04-25T00:09:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.990+0000 7f7590a64640 1 Processor -- start 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.990+0000 7f7590a64640 1 -- start start 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7590a64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f758c0d0b00 0x7f758c157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7590a64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c1f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7590a64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758c15e320 0x7f758c0b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7590a64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f758c0b9fe0 con 0x7f758c0d2290 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7590a64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f758c0d30a0 con 0x7f758c0d0b00 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7590a64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f758c15e700 con 0x7f758c15e320 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c1f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c1f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42138/0 (socket says 172.21.15.113:42138) 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 -- 172.21.15.113:0/713927081 learned_addr learned my addr 172.21.15.113:0/713927081 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f758a59c640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f758c0d0b00 0x7f758c157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f758ad9d640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758c15e320 0x7f758c0b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 -- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758c15e320 msgr2=0x7f758c0b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758c15e320 0x7f758c0b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 -- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f758c0d0b00 msgr2=0x7f758c157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f758c0d0b00 0x7f758c157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f7589d9b640 1 -- 172.21.15.113:0/713927081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c1f7d40 con 0x7f758c0d2290 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f758a59c640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f758c0d0b00 0x7f758c157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.991+0000 7f758ad9d640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758c15e320 0x7f758c0b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.992+0000 7f7589d9b640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c1f75f0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f7580000c00 tx=0x7f75800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.992+0000 7f757affd640 1 -- 172.21.15.113:0/713927081 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580041020 con 0x7f758c0d2290 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.992+0000 7f757affd640 1 -- 172.21.15.113:0/713927081 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75800306c0 con 0x7f758c0d2290 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.992+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1f8020 con 0x7f758c0d2290 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.992+0000 7f757affd640 1 -- 172.21.15.113:0/713927081 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580037040 con 0x7f758c0d2290 2024-04-25T00:09:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.992+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1f4280 con 0x7f758c0d2290 2024-04-25T00:09:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.993+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c1f4070 con 0x7f758c0d2290 2024-04-25T00:09:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.993+0000 7f757affd640 1 -- 172.21.15.113:0/713927081 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7580044020 con 0x7f758c0d2290 2024-04-25T00:09:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.994+0000 7f757affd640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f756c06be30 0x7f756c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.994+0000 7f758a59c640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f756c06be30 0x7f756c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.994+0000 7f757affd640 1 -- 172.21.15.113:0/713927081 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f75800ab6a0 con 0x7f758c0d2290 2024-04-25T00:09:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.994+0000 7f758a59c640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f756c06be30 0x7f756c06e2f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f757c004520 tx=0x7f757c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:35.998+0000 7f757affd640 1 -- 172.21.15.113:0/713927081 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f758004b030 con 0x7f758c0d2290 2024-04-25T00:09:36.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.207+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f758c0691a0 con 0x7f758c0d2290 2024-04-25T00:09:36.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.208+0000 7f757affd640 1 -- 172.21.15.113:0/713927081 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5204 (secure 0 0 0) 0x7f758007a050 con 0x7f758c0d2290 2024-04-25T00:09:36.208 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":98,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":10794,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:35.199346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10803":{"gid":10803,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/3057967443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3057967443},{"type":"v1","addr":"172.21.15.178:6835","nonce":3057967443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10803,"qdb_cluster":[10803]},"id":13}]} 2024-04-25T00:09:36.209 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 98 2024-04-25T00:09:36.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.209+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f756c06be30 msgr2=0x7f756c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.209+0000 7f7590a64640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f756c06be30 0x7f756c06e2f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f757c004520 tx=0x7f757c009040 comp rx=0 tx=0).stop 2024-04-25T00:09:36.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 msgr2=0x7f758c1f75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c1f75f0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f7580000c00 tx=0x7f75800046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 shutdown_connections 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f756c06be30 0x7f756c06e2f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758c15e320 0x7f758c0b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c0d2290 0x7f758c1f75f0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 --2- 172.21.15.113:0/713927081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f758c0d0b00 0x7f758c157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 >> 172.21.15.113:0/713927081 conn(0x7f758c0b5c20 msgr2=0x7f758c0ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 shutdown_connections 2024-04-25T00:09:36.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.210+0000 7f7590a64640 1 -- 172.21.15.113:0/713927081 wait complete. 2024-04-25T00:09:36.221 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:36.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.318+0000 7f997e064640 1 Processor -- start 2024-04-25T00:09:36.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.318+0000 7f997e064640 1 -- start start 2024-04-25T00:09:36.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f997e064640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f9978160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f997e064640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9978074cf0 0x7f997806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f997e064640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99780d00d0 0x7f997806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f997e064640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f99780cedb0 con 0x7f9978074cf0 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f997e064640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9978160bb0 con 0x7f99780d00d0 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f997e064640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99780cf030 con 0x7f997815c010 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9978074cf0 0x7f997806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9978074cf0 0x7f997806b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42152/0 (socket says 172.21.15.113:42152) 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 -- 172.21.15.113:0/95440072 learned_addr learned my addr 172.21.15.113:0/95440072 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9977fff640 1 --2- 172.21.15.113:0/95440072 >> v2:172.21.15.178:3300/0 conn(0x7f99780d00d0 0x7f997806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f99777fe640 1 --2- 172.21.15.113:0/95440072 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f9978160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 -- 172.21.15.113:0/95440072 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 msgr2=0x7f9978160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 --2- 172.21.15.113:0/95440072 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f9978160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 -- 172.21.15.113:0/95440072 >> v2:172.21.15.178:3300/0 conn(0x7f99780d00d0 msgr2=0x7f997806b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:36.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 --2- 172.21.15.113:0/95440072 >> v2:172.21.15.178:3300/0 conn(0x7f99780d00d0 0x7f997806b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9976ffd640 1 -- 172.21.15.113:0/95440072 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780cf310 con 0x7f9978074cf0 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.319+0000 7f9977fff640 1 --2- 172.21.15.113:0/95440072 >> v2:172.21.15.178:3300/0 conn(0x7f99780d00d0 0x7f997806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.320+0000 7f9976ffd640 1 --2- 172.21.15.113:0/95440072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f997806b0f0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f99780aff20 tx=0x7f996002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c65c8fdab482d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.320+0000 7f99767fc640 1 -- 172.21.15.113:0/95440072 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960037070 con 0x7f9978074cf0 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.320+0000 7f99767fc640 1 -- 172.21.15.113:0/95440072 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960032440 con 0x7f9978074cf0 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.320+0000 7f99767fc640 1 -- 172.21.15.113:0/95440072 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960032810 con 0x7f9978074cf0 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 -- 172.21.15.113:0/95440072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 msgr2=0x7f997806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 --2- 172.21.15.113:0/95440072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f997806b0f0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f99780aff20 tx=0x7f996002a140 comp rx=0 tx=0).stop 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 -- 172.21.15.113:0/95440072 shutdown_connections 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 --2- 172.21.15.113:0/95440072 >> v2:172.21.15.178:3300/0 conn(0x7f99780d00d0 0x7f997806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 --2- 172.21.15.113:0/95440072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f997806b0f0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 --2- 172.21.15.113:0/95440072 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f9978160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 -- 172.21.15.113:0/95440072 >> 172.21.15.113:0/95440072 conn(0x7f99780b22c0 msgr2=0x7f99780b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:36.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 -- 172.21.15.113:0/95440072 shutdown_connections 2024-04-25T00:09:36.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.321+0000 7f997e064640 1 -- 172.21.15.113:0/95440072 wait complete. 2024-04-25T00:09:36.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.322+0000 7f997e064640 1 Processor -- start 2024-04-25T00:09:36.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.322+0000 7f997e064640 1 -- start start 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f99780b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99780d00d0 0x7f99781450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f997814e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f99780cfbb0 con 0x7f9978074cf0 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99780cf8d0 con 0x7f99780d00d0 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f997815c3f0 con 0x7f997815c010 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f99780b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f99780b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42162/0 (socket says 172.21.15.113:42162) 2024-04-25T00:09:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 -- 172.21.15.113:0/3860797534 learned_addr learned my addr 172.21.15.113:0/3860797534 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f9976ffd640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99780d00d0 0x7f99781450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f9977fff640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f997814e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 -- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 msgr2=0x7f997814e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f997814e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 -- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99780d00d0 msgr2=0x7f99781450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99780d00d0 0x7f99781450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 -- 172.21.15.113:0/3860797534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780ad890 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f99777fe640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f99780b47c0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f996800d620 tx=0x7f996800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f996ffff640 1 -- 172.21.15.113:0/3860797534 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996800b620 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f996ffff640 1 -- 172.21.15.113:0/3860797534 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9968019070 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f996ffff640 1 -- 172.21.15.113:0/3860797534 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996800ba00 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99780b5010 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.323+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99780b55d0 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.325+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99780691a0 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.325+0000 7f996ffff640 1 -- 172.21.15.113:0/3860797534 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f996800bba0 con 0x7f9978074cf0 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.325+0000 7f996ffff640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f995806be30 0x7f995806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.325+0000 7f996ffff640 1 -- 172.21.15.113:0/3860797534 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f996808b0b0 con 0x7f9978074cf0 2024-04-25T00:09:36.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.325+0000 7f9976ffd640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f995806be30 0x7f995806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.326+0000 7f9976ffd640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f995806be30 0x7f995806e2f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9960034000 tx=0x7f996002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.330+0000 7f996ffff640 1 -- 172.21.15.113:0/3860797534 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9968059410 con 0x7f9978074cf0 2024-04-25T00:09:36.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.503+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99780cd870 con 0x7f9978074cf0 2024-04-25T00:09:36.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.504+0000 7f996ffff640 1 -- 172.21.15.113:0/3860797534 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) ==== 74+0+16731 (secure 0 0 0) 0x7f996805d710 con 0x7f9978074cf0 2024-04-25T00:09:36.505 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:36.505 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":182,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:34.859228+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:31.412384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:32.173467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f995806be30 msgr2=0x7f995806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f995806be30 0x7f995806e2f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9960034000 tx=0x7f996002e000 comp rx=0 tx=0).stop 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 msgr2=0x7f99780b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f99780b47c0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f996800d620 tx=0x7f996800dae0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 shutdown_connections 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f995806be30 0x7f995806e2f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99780d00d0 0x7f99781450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997815c010 0x7f997814e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 --2- 172.21.15.113:0/3860797534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9978074cf0 0x7f99780b47c0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 >> 172.21.15.113:0/3860797534 conn(0x7f99780b22c0 msgr2=0x7f99780b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:36.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 shutdown_connections 2024-04-25T00:09:36.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.506+0000 7f997e064640 1 -- 172.21.15.113:0/3860797534 wait complete. 2024-04-25T00:09:36.518 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:09:36.518 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:09:36.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.614+0000 7f27b0064640 1 Processor -- start 2024-04-25T00:09:36.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.615+0000 7f27b0064640 1 -- start start 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.615+0000 7f27b0064640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27a80d2290 0x7f27a80b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.615+0000 7f27b0064640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a80d0b00 0x7f27a80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.615+0000 7f27b0064640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a80d2800 con 0x7f27a815e320 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.615+0000 7f27b0064640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f27a80d14d0 con 0x7f27a80d2290 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.615+0000 7f27b0064640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f27a80d3410 con 0x7f27a80d0b00 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.615+0000 7f27ad5ff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27a80d2290 0x7f27a80b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.616+0000 7f27ad5ff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27a80d2290 0x7f27a80b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42178/0 (socket says 172.21.15.113:42178) 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.616+0000 7f27ad5ff640 1 -- 172.21.15.113:0/1593550834 learned_addr learned my addr 172.21.15.113:0/1593550834 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:36.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.616+0000 7f27ae601640 1 --2- 172.21.15.113:0/1593550834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a80d0b00 0x7f27a80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.616+0000 7f27ad5ff640 1 -- 172.21.15.113:0/1593550834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a80d0b00 msgr2=0x7f27a80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.616+0000 7f27ad5ff640 1 --2- 172.21.15.113:0/1593550834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a80d0b00 0x7f27a80d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.616+0000 7f27ad5ff640 1 -- 172.21.15.113:0/1593550834 >> v1:172.21.15.178:6789/0 conn(0x7f27a815e320 legacy=0x7f27a80d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.616+0000 7f27ad5ff640 1 -- 172.21.15.113:0/1593550834 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a80d2ae0 con 0x7f27a80d2290 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27ad5ff640 1 --2- 172.21.15.113:0/1593550834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a80b9aa0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f279c009870 tx=0x7f279c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f872de0698d5525c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27acdfe640 1 -- 172.21.15.113:0/1593550834 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c037070 con 0x7f27a80d2290 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27acdfe640 1 -- 172.21.15.113:0/1593550834 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279c02ae50 con 0x7f27a80d2290 2024-04-25T00:09:36.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27acdfe640 1 -- 172.21.15.113:0/1593550834 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c032860 con 0x7f27a80d2290 2024-04-25T00:09:36.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27b0064640 1 -- 172.21.15.113:0/1593550834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 msgr2=0x7f27a80b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27b0064640 1 --2- 172.21.15.113:0/1593550834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a80b9aa0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f279c009870 tx=0x7f279c02a300 comp rx=0 tx=0).stop 2024-04-25T00:09:36.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27b0064640 1 -- 172.21.15.113:0/1593550834 shutdown_connections 2024-04-25T00:09:36.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27b0064640 1 --2- 172.21.15.113:0/1593550834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a80d0b00 0x7f27a80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27b0064640 1 --2- 172.21.15.113:0/1593550834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a80b9aa0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.617+0000 7f27b0064640 1 -- 172.21.15.113:0/1593550834 >> 172.21.15.113:0/1593550834 conn(0x7f27a80b5c20 msgr2=0x7f27a80b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:36.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.618+0000 7f27b0064640 1 -- 172.21.15.113:0/1593550834 shutdown_connections 2024-04-25T00:09:36.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.618+0000 7f27b0064640 1 -- 172.21.15.113:0/1593550834 wait complete. 2024-04-25T00:09:36.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 Processor -- start 2024-04-25T00:09:36.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 -- start start 2024-04-25T00:09:36.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a80d0b00 0x7f27a80c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a8159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a815e320 0x7f27a80cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f27a80b9fe0 con 0x7f27a80d2290 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27a80d30a0 con 0x7f27a80d0b00 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27b0064640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f27a815e700 con 0x7f27a815e320 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.619+0000 7f27ad5ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a8159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a8159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42184/0 (socket says 172.21.15.113:42184) 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 -- 172.21.15.113:0/2248577228 learned_addr learned my addr 172.21.15.113:0/2248577228 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ae601640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a815e320 0x7f27a80cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ade00640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a80d0b00 0x7f27a80c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 -- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a815e320 msgr2=0x7f27a80cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a815e320 0x7f27a80cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 -- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a80d0b00 msgr2=0x7f27a80c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a80d0b00 0x7f27a80c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 -- 172.21.15.113:0/2248577228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a814ada0 con 0x7f27a80d2290 2024-04-25T00:09:36.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ade00640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a80d0b00 0x7f27a80c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27ad5ff640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a8159450 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f279c000c00 tx=0x7f279c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27a67fc640 1 -- 172.21.15.113:0/2248577228 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c041070 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27a67fc640 1 -- 172.21.15.113:0/2248577228 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279c03c890 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27a67fc640 1 -- 172.21.15.113:0/2248577228 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c037040 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a814b080 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.620+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a814b640 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.621+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a80691a0 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.622+0000 7f27a67fc640 1 -- 172.21.15.113:0/2248577228 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f279c044070 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.622+0000 7f27a67fc640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f279406be30 0x7f279406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.622+0000 7f27a67fc640 1 -- 172.21.15.113:0/2248577228 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f279c0ab9c0 con 0x7f27a80d2290 2024-04-25T00:09:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.622+0000 7f27ade00640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f279406be30 0x7f279406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.623+0000 7f27ade00640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f279406be30 0x7f279406e2f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f27a00012e0 tx=0x7f27a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.626+0000 7f27a67fc640 1 -- 172.21.15.113:0/2248577228 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f279c0ade20 con 0x7f27a80d2290 2024-04-25T00:09:36.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.798+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27a80b47a0 con 0x7f279406be30 2024-04-25T00:09:36.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.803+0000 7f27a67fc640 1 -- 172.21.15.113:0/2248577228 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237266 (secure 0 0 0) 0x7f27a80b47a0 con 0x7f279406be30 2024-04-25T00:09:36.806 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:36.808 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f279406be30 msgr2=0x7f279406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f279406be30 0x7f279406e2f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f27a00012e0 tx=0x7f27a0009040 comp rx=0 tx=0).stop 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 msgr2=0x7f27a8159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a8159450 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f279c000c00 tx=0x7f279c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 shutdown_connections 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f279406be30 0x7f279406e2f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a815e320 0x7f27a80cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27a80d2290 0x7f27a8159450 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 --2- 172.21.15.113:0/2248577228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a80d0b00 0x7f27a80c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 >> 172.21.15.113:0/2248577228 conn(0x7f27a80b5c20 msgr2=0x7f27a80d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.806+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 shutdown_connections 2024-04-25T00:09:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.807+0000 7f27b0064640 1 -- 172.21.15.113:0/2248577228 wait complete. 2024-04-25T00:09:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-04-25T00:09:35.246584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":416704,"kb_used_data":18684,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754558016,"statfs":{"total":773094113280,"available":772667408384,"internally_reserved":0,"allocated":19132416,"data_stored":9021633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":407424560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000753"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":507,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-04-25T00:09:30.219059+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:09:30.219059+0000","last_peered":"2024-04-25T00:09:30.219059+0000","last_clean":"2024-04-25T00:09:30.219059+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:09:30.219059+0000","last_undegraded":"2024-04-25T00:09:30.219059+0000","last_fullsized":"2024-04-25T00:09:30.219059+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47844,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323996,"statfs":{"total":96636764160,"available":96587771904,"internally_reserved":0,"allocated":2838528,"data_stored":1571121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51424,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320416,"statfs":{"total":96636764160,"available":96584105984,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55584,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316256,"statfs":{"total":96636764160,"available":96579846144,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47276,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324564,"statfs":{"total":96636764160,"available":96588353536,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51412,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320428,"statfs":{"total":96636764160,"available":96584118272,"internally_reserved":0,"allocated":2232320,"data_stored":975626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51420,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320420,"statfs":{"total":96636764160,"available":96584110080,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59744,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312096,"statfs":{"total":96636764160,"available":96575586304,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52000,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319840,"statfs":{"total":96636764160,"available":96583516160,"internally_reserved":0,"allocated":2838528,"data_stored":1571121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:09:36.822 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:09:36.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.954+0000 7f54228fc640 1 Processor -- start 2024-04-25T00:09:36.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.955+0000 7f54228fc640 1 -- start start 2024-04-25T00:09:36.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.955+0000 7f54228fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f541c0d2290 0x7f541c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.955+0000 7f54228fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c0d0b00 0x7f541c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.955+0000 7f54228fc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f541c0d2800 con 0x7f541c15e320 2024-04-25T00:09:36.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.955+0000 7f54228fc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f541c0d14d0 con 0x7f541c0d2290 2024-04-25T00:09:36.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.955+0000 7f54228fc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f541c0d3410 con 0x7f541c0d0b00 2024-04-25T00:09:36.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f5420e99640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c0d0b00 0x7f541c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f541b7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f541c0d2290 0x7f541c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f5420e99640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c0d0b00 0x7f541c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:59480/0 (socket says 172.21.15.113:59480) 2024-04-25T00:09:36.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f5420e99640 1 -- 172.21.15.113:0/708332322 learned_addr learned my addr 172.21.15.113:0/708332322 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:36.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f541b7fe640 1 -- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c0d0b00 msgr2=0x7f541c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f541b7fe640 1 --2- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c0d0b00 0x7f541c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f541b7fe640 1 -- 172.21.15.113:0/708332322 >> v1:172.21.15.178:6789/0 conn(0x7f541c15e320 legacy=0x7f541c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f541b7fe640 1 -- 172.21.15.113:0/708332322 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c0d2ae0 con 0x7f541c0d2290 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.956+0000 7f5420e99640 1 --2- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c0d0b00 0x7f541c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.957+0000 7f541b7fe640 1 --2- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 0x7f541c0b9aa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7398660a73a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.957+0000 7f541affd640 1 -- 172.21.15.113:0/708332322 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c037070 con 0x7f541c0d2290 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.957+0000 7f541affd640 1 -- 172.21.15.113:0/708332322 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f540c032440 con 0x7f541c0d2290 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.957+0000 7f541affd640 1 -- 172.21.15.113:0/708332322 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c032810 con 0x7f541c0d2290 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 -- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 msgr2=0x7f541c0b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 --2- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 0x7f541c0b9aa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097a0 tx=0x7f540c02a210 comp rx=0 tx=0).stop 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 -- 172.21.15.113:0/708332322 shutdown_connections 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 --2- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c0d0b00 0x7f541c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 --2- 172.21.15.113:0/708332322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 0x7f541c0b9aa0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 -- 172.21.15.113:0/708332322 >> 172.21.15.113:0/708332322 conn(0x7f541c0b5c20 msgr2=0x7f541c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:36.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 -- 172.21.15.113:0/708332322 shutdown_connections 2024-04-25T00:09:36.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.958+0000 7f54228fc640 1 -- 172.21.15.113:0/708332322 wait complete. 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.959+0000 7f54228fc640 1 Processor -- start 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.959+0000 7f54228fc640 1 -- start start 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c0d0b00 0x7f541c0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 0x7f541c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c15e320 0x7f541c0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f541c0b9fe0 con 0x7f541c0d0b00 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f541c0d30a0 con 0x7f541c0d2290 2024-04-25T00:09:36.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f541c15e700 con 0x7f541c15e320 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c0d0b00 0x7f541c0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c0d0b00 0x7f541c0c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42200/0 (socket says 172.21.15.113:42200) 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 -- 172.21.15.113:0/4104901036 learned_addr learned my addr 172.21.15.113:0/4104901036 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f5420e99640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c15e320 0x7f541c0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541b7fe640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 0x7f541c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 -- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c15e320 msgr2=0x7f541c0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c15e320 0x7f541c0cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 -- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 msgr2=0x7f541c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 0x7f541c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 -- 172.21.15.113:0/4104901036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c14ada0 con 0x7f541c0d0b00 2024-04-25T00:09:36.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f541bfff640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c0d0b00 0x7f541c0c9b60 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f540400c340 tx=0x7f540400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f5418ff9640 1 -- 172.21.15.113:0/4104901036 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404017070 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f5418ff9640 1 -- 172.21.15.113:0/4104901036 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54040052d0 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c14b080 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f5418ff9640 1 -- 172.21.15.113:0/4104901036 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404003850 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.960+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c14b640 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.962+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c0691a0 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.962+0000 7f5418ff9640 1 -- 172.21.15.113:0/4104901036 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f540401a070 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.962+0000 7f5418ff9640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f53fc06be30 0x7f53fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.962+0000 7f5418ff9640 1 -- 172.21.15.113:0/4104901036 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f540408a1c0 con 0x7f541c0d0b00 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.962+0000 7f541b7fe640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f53fc06be30 0x7f53fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:36.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.963+0000 7f541b7fe640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f53fc06be30 0x7f53fc06e2f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f540c034000 tx=0x7f540c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:36.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:36.966+0000 7f5418ff9640 1 -- 172.21.15.113:0/4104901036 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5404058b70 con 0x7f541c0d0b00 2024-04-25T00:09:37.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.134+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f541c0b47a0 con 0x7f53fc06be30 2024-04-25T00:09:37.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.139+0000 7f5418ff9640 1 -- 172.21.15.113:0/4104901036 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237266 (secure 0 0 0) 0x7f541c0b47a0 con 0x7f53fc06be30 2024-04-25T00:09:37.142 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:37.143 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:09:37.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f53fc06be30 msgr2=0x7f53fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f53fc06be30 0x7f53fc06e2f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f540c034000 tx=0x7f540c02e000 comp rx=0 tx=0).stop 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c0d0b00 msgr2=0x7f541c0c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c0d0b00 0x7f541c0c9b60 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f540400c340 tx=0x7f540400c800 comp rx=0 tx=0).stop 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 shutdown_connections 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f53fc06be30 0x7f53fc06e2f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f541c0d2290 0x7f541c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f541c15e320 0x7f541c0cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 --2- 172.21.15.113:0/4104901036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541c0d0b00 0x7f541c0c9b60 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 >> 172.21.15.113:0/4104901036 conn(0x7f541c0b5c20 msgr2=0x7f541c0d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 shutdown_connections 2024-04-25T00:09:37.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.142+0000 7f54228fc640 1 -- 172.21.15.113:0/4104901036 wait complete. 2024-04-25T00:09:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-04-25T00:09:35.246584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":416704,"kb_used_data":18684,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754558016,"statfs":{"total":773094113280,"available":772667408384,"internally_reserved":0,"allocated":19132416,"data_stored":9021633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":407424560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000753"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:32.845368+0000","last_change":"2024-04-25T00:09:32.845368+0000","last_active":"2024-04-25T00:09:32.845368+0000","last_peered":"2024-04-25T00:09:32.845368+0000","last_clean":"2024-04-25T00:09:32.845368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:32.845368+0000","last_undegraded":"2024-04-25T00:09:32.845368+0000","last_fullsized":"2024-04-25T00:09:32.845368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:09:31.840517+0000","last_change":"2024-04-25T00:09:31.840517+0000","last_active":"2024-04-25T00:09:31.840517+0000","last_peered":"2024-04-25T00:09:31.840517+0000","last_clean":"2024-04-25T00:09:31.840517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:09:31.840517+0000","last_undegraded":"2024-04-25T00:09:31.840517+0000","last_fullsized":"2024-04-25T00:09:31.840517+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":507,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-04-25T00:09:30.219059+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:09:30.219059+0000","last_peered":"2024-04-25T00:09:30.219059+0000","last_clean":"2024-04-25T00:09:30.219059+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:09:30.219059+0000","last_undegraded":"2024-04-25T00:09:30.219059+0000","last_fullsized":"2024-04-25T00:09:30.219059+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47844,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323996,"statfs":{"total":96636764160,"available":96587771904,"internally_reserved":0,"allocated":2838528,"data_stored":1571121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51424,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320416,"statfs":{"total":96636764160,"available":96584105984,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55584,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316256,"statfs":{"total":96636764160,"available":96579846144,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47276,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324564,"statfs":{"total":96636764160,"available":96588353536,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51412,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320428,"statfs":{"total":96636764160,"available":96584118272,"internally_reserved":0,"allocated":2232320,"data_stored":975626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51420,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320420,"statfs":{"total":96636764160,"available":96584110080,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59744,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312096,"statfs":{"total":96636764160,"available":96575586304,"internally_reserved":0,"allocated":2244608,"data_stored":980753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52000,"kb_used_data":2772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319840,"statfs":{"total":96636764160,"available":96583516160,"internally_reserved":0,"allocated":2838528,"data_stored":1571121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:09:37.157 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:09:37.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.291+0000 7fafd6596640 1 Processor -- start 2024-04-25T00:09:37.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.291+0000 7fafd6596640 1 -- start start 2024-04-25T00:09:37.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafd6596640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafd0072ea0 0x7fafd0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafd6596640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd006dfd0 0x7fafd01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafd6596640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd00b2d50 con 0x7fafd015a6b0 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafd6596640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fafd0072a70 con 0x7fafd0072ea0 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafd6596640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fafd00b3010 con 0x7fafd006dfd0 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafcf7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafd0072ea0 0x7fafd0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafcf7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafd0072ea0 0x7fafd0071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42216/0 (socket says 172.21.15.113:42216) 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2441001234 learned_addr learned my addr 172.21.15.113:0/2441001234 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:37.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafd4b33640 1 --2- 172.21.15.113:0/2441001234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd006dfd0 0x7fafd01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2441001234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd006dfd0 msgr2=0x7fafd01656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafcf7fe640 1 --2- 172.21.15.113:0/2441001234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd006dfd0 0x7fafd01656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.292+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2441001234 >> v1:172.21.15.178:6789/0 conn(0x7fafd015a6b0 legacy=0x7fafd0070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.293+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2441001234 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd00b32f0 con 0x7fafd0072ea0 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.293+0000 7fafcf7fe640 1 --2- 172.21.15.113:0/2441001234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd0071410 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009870 tx=0x7fafc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5fccedc5907a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.293+0000 7fafceffd640 1 -- 172.21.15.113:0/2441001234 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4037070 con 0x7fafd0072ea0 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.293+0000 7fafceffd640 1 -- 172.21.15.113:0/2441001234 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc4032440 con 0x7fafd0072ea0 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.293+0000 7fafceffd640 1 -- 172.21.15.113:0/2441001234 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4032810 con 0x7fafd0072ea0 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 -- 172.21.15.113:0/2441001234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 msgr2=0x7fafd0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 --2- 172.21.15.113:0/2441001234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd0071410 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009870 tx=0x7fafc402a270 comp rx=0 tx=0).stop 2024-04-25T00:09:37.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 -- 172.21.15.113:0/2441001234 shutdown_connections 2024-04-25T00:09:37.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 --2- 172.21.15.113:0/2441001234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd006dfd0 0x7fafd01656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 --2- 172.21.15.113:0/2441001234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd0071410 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 -- 172.21.15.113:0/2441001234 >> 172.21.15.113:0/2441001234 conn(0x7fafd00b1a50 msgr2=0x7fafd00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:37.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 -- 172.21.15.113:0/2441001234 shutdown_connections 2024-04-25T00:09:37.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.294+0000 7fafd6596640 1 -- 172.21.15.113:0/2441001234 wait complete. 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.295+0000 7fafd6596640 1 Processor -- start 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.295+0000 7fafd6596640 1 -- start start 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd6596640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafd006dfd0 0x7fafd0148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd6596640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd013de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd6596640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd015a6b0 0x7fafd013e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd6596640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fafd015aa90 con 0x7fafd0072ea0 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd6596640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0072720 con 0x7fafd006dfd0 2024-04-25T00:09:37.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd6596640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fafd015ac10 con 0x7fafd015a6b0 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd013de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd013de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42232/0 (socket says 172.21.15.113:42232) 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2971758856 learned_addr learned my addr 172.21.15.113:0/2971758856 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd4b33640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd015a6b0 0x7fafd013e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcffff640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafd006dfd0 0x7fafd0148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd015a6b0 msgr2=0x7fafd013e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd015a6b0 0x7fafd013e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafd006dfd0 msgr2=0x7fafd0148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafd006dfd0 0x7fafd0148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcf7fe640 1 -- 172.21.15.113:0/2971758856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd00ce300 con 0x7fafd0072ea0 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafd4b33640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd015a6b0 0x7fafd013e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.296+0000 7fafcffff640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafd006dfd0 0x7fafd0148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.297+0000 7fafcf7fe640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd013de50 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fafc402a2b0 tx=0x7fafc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.297+0000 7fafccff9640 1 -- 172.21.15.113:0/2971758856 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4041020 con 0x7fafd0072ea0 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.297+0000 7fafccff9640 1 -- 172.21.15.113:0/2971758856 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc40306c0 con 0x7fafd0072ea0 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.297+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd00ce5e0 con 0x7fafd0072ea0 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.297+0000 7fafccff9640 1 -- 172.21.15.113:0/2971758856 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4037040 con 0x7fafd0072ea0 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.297+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd0149240 con 0x7fafd0072ea0 2024-04-25T00:09:37.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.298+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd00691a0 con 0x7fafd0072ea0 2024-04-25T00:09:37.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.298+0000 7fafccff9640 1 -- 172.21.15.113:0/2971758856 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafc4044020 con 0x7fafd0072ea0 2024-04-25T00:09:37.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.299+0000 7fafccff9640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fafbc06be60 0x7fafbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.299+0000 7fafccff9640 1 -- 172.21.15.113:0/2971758856 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7fafc40ab5d0 con 0x7fafd0072ea0 2024-04-25T00:09:37.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.299+0000 7fafcffff640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fafbc06be60 0x7fafbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.299+0000 7fafcffff640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fafbc06be60 0x7fafbc06e320 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fafd014e3e0 tx=0x7fafc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:37.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.303+0000 7fafccff9640 1 -- 172.21.15.113:0/2971758856 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fafc404b030 con 0x7fafd0072ea0 2024-04-25T00:09:37.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.473+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafd01659f0 con 0x7fafbc06be60 2024-04-25T00:09:37.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.478+0000 7fafccff9640 1 -- 172.21.15.113:0/2971758856 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7fafd01659f0 con 0x7fafbc06be60 2024-04-25T00:09:37.481 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:37.483 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fafbc06be60 msgr2=0x7fafbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fafbc06be60 0x7fafbc06e320 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fafd014e3e0 tx=0x7fafc0009040 comp rx=0 tx=0).stop 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 msgr2=0x7fafd013de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd013de50 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fafc402a2b0 tx=0x7fafc40046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 shutdown_connections 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fafbc06be60 0x7fafbc06e320 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafd015a6b0 0x7fafd013e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafd0072ea0 0x7fafd013de50 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 --2- 172.21.15.113:0/2971758856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafd006dfd0 0x7fafd0148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 >> 172.21.15.113:0/2971758856 conn(0x7fafd00b1a50 msgr2=0x7fafd00b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:37.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 shutdown_connections 2024-04-25T00:09:37.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.482+0000 7fafd6596640 1 -- 172.21.15.113:0/2971758856 wait complete. 2024-04-25T00:09:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-04-25T00:09:37.246935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":254,"num_write_kb":3476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421308,"kb_used_data":19120,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754553412,"statfs":{"total":773094113280,"available":772662693888,"internally_reserved":0,"allocated":19578880,"data_stored":9220316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.401689"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863971+0000","last_change":"2024-04-25T00:09:33.871025+0000","last_active":"2024-04-25T00:09:34.863971+0000","last_peered":"2024-04-25T00:09:34.863971+0000","last_clean":"2024-04-25T00:09:34.863971+0000","last_became_active":"2024-04-25T00:09:33.870439+0000","last_became_peered":"2024-04-25T00:09:33.870439+0000","last_unstale":"2024-04-25T00:09:34.863971+0000","last_undegraded":"2024-04-25T00:09:34.863971+0000","last_fullsized":"2024-04-25T00:09:34.863971+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:59.226155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168700+0000","last_change":"2024-04-25T00:09:33.878604+0000","last_active":"2024-04-25T00:09:35.168700+0000","last_peered":"2024-04-25T00:09:35.168700+0000","last_clean":"2024-04-25T00:09:35.168700+0000","last_became_active":"2024-04-25T00:09:33.878328+0000","last_became_peered":"2024-04-25T00:09:33.878328+0000","last_unstale":"2024-04-25T00:09:35.168700+0000","last_undegraded":"2024-04-25T00:09:35.168700+0000","last_fullsized":"2024-04-25T00:09:35.168700+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879389+0000","last_change":"2024-04-25T00:09:33.878325+0000","last_active":"2024-04-25T00:09:33.879389+0000","last_peered":"2024-04-25T00:09:33.879389+0000","last_clean":"2024-04-25T00:09:33.879389+0000","last_became_active":"2024-04-25T00:09:33.877495+0000","last_became_peered":"2024-04-25T00:09:33.877495+0000","last_unstale":"2024-04-25T00:09:33.879389+0000","last_undegraded":"2024-04-25T00:09:33.879389+0000","last_fullsized":"2024-04-25T00:09:33.879389+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:28.296941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871982+0000","last_change":"2024-04-25T00:09:33.869224+0000","last_active":"2024-04-25T00:09:34.871982+0000","last_peered":"2024-04-25T00:09:34.871982+0000","last_clean":"2024-04-25T00:09:34.871982+0000","last_became_active":"2024-04-25T00:09:33.868734+0000","last_became_peered":"2024-04-25T00:09:33.868734+0000","last_unstale":"2024-04-25T00:09:34.871982+0000","last_undegraded":"2024-04-25T00:09:34.871982+0000","last_fullsized":"2024-04-25T00:09:34.871982+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:16.109657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863334+0000","last_change":"2024-04-25T00:09:33.869714+0000","last_active":"2024-04-25T00:09:34.863334+0000","last_peered":"2024-04-25T00:09:34.863334+0000","last_clean":"2024-04-25T00:09:34.863334+0000","last_became_active":"2024-04-25T00:09:33.869435+0000","last_became_peered":"2024-04-25T00:09:33.869435+0000","last_unstale":"2024-04-25T00:09:34.863334+0000","last_undegraded":"2024-04-25T00:09:34.863334+0000","last_fullsized":"2024-04-25T00:09:34.863334+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:59.450004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863112+0000","last_change":"2024-04-25T00:09:33.866179+0000","last_active":"2024-04-25T00:09:34.863112+0000","last_peered":"2024-04-25T00:09:34.863112+0000","last_clean":"2024-04-25T00:09:34.863112+0000","last_became_active":"2024-04-25T00:09:33.865891+0000","last_became_peered":"2024-04-25T00:09:33.865891+0000","last_unstale":"2024-04-25T00:09:34.863112+0000","last_undegraded":"2024-04-25T00:09:34.863112+0000","last_fullsized":"2024-04-25T00:09:34.863112+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:00.144856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863588+0000","last_change":"2024-04-25T00:09:33.875207+0000","last_active":"2024-04-25T00:09:34.863588+0000","last_peered":"2024-04-25T00:09:34.863588+0000","last_clean":"2024-04-25T00:09:34.863588+0000","last_became_active":"2024-04-25T00:09:33.874674+0000","last_became_peered":"2024-04-25T00:09:33.874674+0000","last_unstale":"2024-04-25T00:09:34.863588+0000","last_undegraded":"2024-04-25T00:09:34.863588+0000","last_fullsized":"2024-04-25T00:09:34.863588+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:14.333375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862963+0000","last_change":"2024-04-25T00:09:33.860996+0000","last_active":"2024-04-25T00:09:34.862963+0000","last_peered":"2024-04-25T00:09:34.862963+0000","last_clean":"2024-04-25T00:09:34.862963+0000","last_became_active":"2024-04-25T00:09:33.860726+0000","last_became_peered":"2024-04-25T00:09:33.860726+0000","last_unstale":"2024-04-25T00:09:34.862963+0000","last_undegraded":"2024-04-25T00:09:34.862963+0000","last_fullsized":"2024-04-25T00:09:34.862963+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:40.927785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.169032+0000","last_change":"2024-04-25T00:09:33.877988+0000","last_active":"2024-04-25T00:09:35.169032+0000","last_peered":"2024-04-25T00:09:35.169032+0000","last_clean":"2024-04-25T00:09:35.169032+0000","last_became_active":"2024-04-25T00:09:33.877216+0000","last_became_peered":"2024-04-25T00:09:33.877216+0000","last_unstale":"2024-04-25T00:09:35.169032+0000","last_undegraded":"2024-04-25T00:09:35.169032+0000","last_fullsized":"2024-04-25T00:09:35.169032+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:41.886576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.873730+0000","last_change":"2024-04-25T00:09:33.871557+0000","last_active":"2024-04-25T00:09:33.873730+0000","last_peered":"2024-04-25T00:09:33.873730+0000","last_clean":"2024-04-25T00:09:33.873730+0000","last_became_active":"2024-04-25T00:09:33.871300+0000","last_became_peered":"2024-04-25T00:09:33.871300+0000","last_unstale":"2024-04-25T00:09:33.873730+0000","last_undegraded":"2024-04-25T00:09:33.873730+0000","last_fullsized":"2024-04-25T00:09:33.873730+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:22.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.872901+0000","last_change":"2024-04-25T00:09:33.868170+0000","last_active":"2024-04-25T00:09:33.872901+0000","last_peered":"2024-04-25T00:09:33.872901+0000","last_clean":"2024-04-25T00:09:33.872901+0000","last_became_active":"2024-04-25T00:09:33.867382+0000","last_became_peered":"2024-04-25T00:09:33.867382+0000","last_unstale":"2024-04-25T00:09:33.872901+0000","last_undegraded":"2024-04-25T00:09:33.872901+0000","last_fullsized":"2024-04-25T00:09:33.872901+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:04.488545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874365+0000","last_change":"2024-04-25T00:09:33.873865+0000","last_active":"2024-04-25T00:09:34.874365+0000","last_peered":"2024-04-25T00:09:34.874365+0000","last_clean":"2024-04-25T00:09:34.874365+0000","last_became_active":"2024-04-25T00:09:33.873531+0000","last_became_peered":"2024-04-25T00:09:33.873531+0000","last_unstale":"2024-04-25T00:09:34.874365+0000","last_undegraded":"2024-04-25T00:09:34.874365+0000","last_fullsized":"2024-04-25T00:09:34.874365+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:30.035581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874436+0000","last_change":"2024-04-25T00:09:33.867995+0000","last_active":"2024-04-25T00:09:34.874436+0000","last_peered":"2024-04-25T00:09:34.874436+0000","last_clean":"2024-04-25T00:09:34.874436+0000","last_became_active":"2024-04-25T00:09:33.867692+0000","last_became_peered":"2024-04-25T00:09:33.867692+0000","last_unstale":"2024-04-25T00:09:34.874436+0000","last_undegraded":"2024-04-25T00:09:34.874436+0000","last_fullsized":"2024-04-25T00:09:34.874436+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:33.016301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863341+0000","last_change":"2024-04-25T00:09:33.862436+0000","last_active":"2024-04-25T00:09:34.863341+0000","last_peered":"2024-04-25T00:09:34.863341+0000","last_clean":"2024-04-25T00:09:34.863341+0000","last_became_active":"2024-04-25T00:09:33.862096+0000","last_became_peered":"2024-04-25T00:09:33.862096+0000","last_unstale":"2024-04-25T00:09:34.863341+0000","last_undegraded":"2024-04-25T00:09:34.863341+0000","last_fullsized":"2024-04-25T00:09:34.863341+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:25.636303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863725+0000","last_change":"2024-04-25T00:09:33.865358+0000","last_active":"2024-04-25T00:09:34.863725+0000","last_peered":"2024-04-25T00:09:34.863725+0000","last_clean":"2024-04-25T00:09:34.863725+0000","last_became_active":"2024-04-25T00:09:33.865098+0000","last_became_peered":"2024-04-25T00:09:33.865098+0000","last_unstale":"2024-04-25T00:09:34.863725+0000","last_undegraded":"2024-04-25T00:09:34.863725+0000","last_fullsized":"2024-04-25T00:09:34.863725+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.885015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.880076+0000","last_change":"2024-04-25T00:09:33.879644+0000","last_active":"2024-04-25T00:09:33.880076+0000","last_peered":"2024-04-25T00:09:33.880076+0000","last_clean":"2024-04-25T00:09:33.880076+0000","last_became_active":"2024-04-25T00:09:33.879324+0000","last_became_peered":"2024-04-25T00:09:33.879324+0000","last_unstale":"2024-04-25T00:09:33.880076+0000","last_undegraded":"2024-04-25T00:09:33.880076+0000","last_fullsized":"2024-04-25T00:09:33.880076+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:08.003176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864131+0000","last_change":"2024-04-25T00:09:33.870900+0000","last_active":"2024-04-25T00:09:34.864131+0000","last_peered":"2024-04-25T00:09:34.864131+0000","last_clean":"2024-04-25T00:09:34.864131+0000","last_became_active":"2024-04-25T00:09:33.870115+0000","last_became_peered":"2024-04-25T00:09:33.870115+0000","last_unstale":"2024-04-25T00:09:34.864131+0000","last_undegraded":"2024-04-25T00:09:34.864131+0000","last_fullsized":"2024-04-25T00:09:34.864131+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:39.258462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.875694+0000","last_change":"2024-04-25T00:09:33.872660+0000","last_active":"2024-04-25T00:09:33.875694+0000","last_peered":"2024-04-25T00:09:33.875694+0000","last_clean":"2024-04-25T00:09:33.875694+0000","last_became_active":"2024-04-25T00:09:33.872373+0000","last_became_peered":"2024-04-25T00:09:33.872373+0000","last_unstale":"2024-04-25T00:09:33.875694+0000","last_undegraded":"2024-04-25T00:09:33.875694+0000","last_fullsized":"2024-04-25T00:09:33.875694+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:32.403959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.874556+0000","last_change":"2024-04-25T00:09:33.871847+0000","last_active":"2024-04-25T00:09:33.874556+0000","last_peered":"2024-04-25T00:09:33.874556+0000","last_clean":"2024-04-25T00:09:33.874556+0000","last_became_active":"2024-04-25T00:09:33.871112+0000","last_became_peered":"2024-04-25T00:09:33.871112+0000","last_unstale":"2024-04-25T00:09:33.874556+0000","last_undegraded":"2024-04-25T00:09:33.874556+0000","last_fullsized":"2024-04-25T00:09:33.874556+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:30.093690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863227+0000","last_change":"2024-04-25T00:09:33.869405+0000","last_active":"2024-04-25T00:09:34.863227+0000","last_peered":"2024-04-25T00:09:34.863227+0000","last_clean":"2024-04-25T00:09:34.863227+0000","last_became_active":"2024-04-25T00:09:33.869159+0000","last_became_peered":"2024-04-25T00:09:33.869159+0000","last_unstale":"2024-04-25T00:09:34.863227+0000","last_undegraded":"2024-04-25T00:09:34.863227+0000","last_fullsized":"2024-04-25T00:09:34.863227+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:30.035581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871904+0000","last_change":"2024-04-25T00:09:33.868561+0000","last_active":"2024-04-25T00:09:34.871904+0000","last_peered":"2024-04-25T00:09:34.871904+0000","last_clean":"2024-04-25T00:09:34.871904+0000","last_became_active":"2024-04-25T00:09:33.868230+0000","last_became_peered":"2024-04-25T00:09:33.868230+0000","last_unstale":"2024-04-25T00:09:34.871904+0000","last_undegraded":"2024-04-25T00:09:34.871904+0000","last_fullsized":"2024-04-25T00:09:34.871904+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.864780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863389+0000","last_change":"2024-04-25T00:09:33.870026+0000","last_active":"2024-04-25T00:09:34.863389+0000","last_peered":"2024-04-25T00:09:34.863389+0000","last_clean":"2024-04-25T00:09:34.863389+0000","last_became_active":"2024-04-25T00:09:33.869745+0000","last_became_peered":"2024-04-25T00:09:33.869745+0000","last_unstale":"2024-04-25T00:09:34.863389+0000","last_undegraded":"2024-04-25T00:09:34.863389+0000","last_fullsized":"2024-04-25T00:09:34.863389+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:31.093182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863368+0000","last_change":"2024-04-25T00:09:33.871850+0000","last_active":"2024-04-25T00:09:34.863368+0000","last_peered":"2024-04-25T00:09:34.863368+0000","last_clean":"2024-04-25T00:09:34.863368+0000","last_became_active":"2024-04-25T00:09:33.871593+0000","last_became_peered":"2024-04-25T00:09:33.871593+0000","last_unstale":"2024-04-25T00:09:34.863368+0000","last_undegraded":"2024-04-25T00:09:34.863368+0000","last_fullsized":"2024-04-25T00:09:34.863368+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:33.016301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871805+0000","last_change":"2024-04-25T00:09:33.867377+0000","last_active":"2024-04-25T00:09:34.871805+0000","last_peered":"2024-04-25T00:09:34.871805+0000","last_clean":"2024-04-25T00:09:34.871805+0000","last_became_active":"2024-04-25T00:09:33.867154+0000","last_became_peered":"2024-04-25T00:09:33.867154+0000","last_unstale":"2024-04-25T00:09:34.871805+0000","last_undegraded":"2024-04-25T00:09:34.871805+0000","last_fullsized":"2024-04-25T00:09:34.871805+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.616946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168803+0000","last_change":"2024-04-25T00:09:33.871702+0000","last_active":"2024-04-25T00:09:35.168803+0000","last_peered":"2024-04-25T00:09:35.168803+0000","last_clean":"2024-04-25T00:09:35.168803+0000","last_became_active":"2024-04-25T00:09:33.871458+0000","last_became_peered":"2024-04-25T00:09:33.871458+0000","last_unstale":"2024-04-25T00:09:35.168803+0000","last_undegraded":"2024-04-25T00:09:35.168803+0000","last_fullsized":"2024-04-25T00:09:35.168803+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:25.476623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864017+0000","last_change":"2024-04-25T00:09:33.875792+0000","last_active":"2024-04-25T00:09:34.864017+0000","last_peered":"2024-04-25T00:09:34.864017+0000","last_clean":"2024-04-25T00:09:34.864017+0000","last_became_active":"2024-04-25T00:09:33.874998+0000","last_became_peered":"2024-04-25T00:09:33.874998+0000","last_unstale":"2024-04-25T00:09:34.864017+0000","last_undegraded":"2024-04-25T00:09:34.864017+0000","last_fullsized":"2024-04-25T00:09:34.864017+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.956814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874361+0000","last_change":"2024-04-25T00:09:33.875621+0000","last_active":"2024-04-25T00:09:34.874361+0000","last_peered":"2024-04-25T00:09:34.874361+0000","last_clean":"2024-04-25T00:09:34.874361+0000","last_became_active":"2024-04-25T00:09:33.875162+0000","last_became_peered":"2024-04-25T00:09:33.875162+0000","last_unstale":"2024-04-25T00:09:34.874361+0000","last_undegraded":"2024-04-25T00:09:34.874361+0000","last_fullsized":"2024-04-25T00:09:34.874361+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:00.144856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879430+0000","last_change":"2024-04-25T00:09:33.878105+0000","last_active":"2024-04-25T00:09:33.879430+0000","last_peered":"2024-04-25T00:09:33.879430+0000","last_clean":"2024-04-25T00:09:33.879430+0000","last_became_active":"2024-04-25T00:09:33.877837+0000","last_became_peered":"2024-04-25T00:09:33.877837+0000","last_unstale":"2024-04-25T00:09:33.879430+0000","last_undegraded":"2024-04-25T00:09:33.879430+0000","last_fullsized":"2024-04-25T00:09:33.879430+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:26.774142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874230+0000","last_change":"2024-04-25T00:09:33.875264+0000","last_active":"2024-04-25T00:09:34.874230+0000","last_peered":"2024-04-25T00:09:34.874230+0000","last_clean":"2024-04-25T00:09:34.874230+0000","last_became_active":"2024-04-25T00:09:33.874940+0000","last_became_peered":"2024-04-25T00:09:33.874940+0000","last_unstale":"2024-04-25T00:09:34.874230+0000","last_undegraded":"2024-04-25T00:09:34.874230+0000","last_fullsized":"2024-04-25T00:09:34.874230+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:07.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168658+0000","last_change":"2024-04-25T00:09:33.870203+0000","last_active":"2024-04-25T00:09:35.168658+0000","last_peered":"2024-04-25T00:09:35.168658+0000","last_clean":"2024-04-25T00:09:35.168658+0000","last_became_active":"2024-04-25T00:09:33.869779+0000","last_became_peered":"2024-04-25T00:09:33.869779+0000","last_unstale":"2024-04-25T00:09:35.168658+0000","last_undegraded":"2024-04-25T00:09:35.168658+0000","last_fullsized":"2024-04-25T00:09:35.168658+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:18.123619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.870793+0000","last_change":"2024-04-25T00:09:33.868363+0000","last_active":"2024-04-25T00:09:33.870793+0000","last_peered":"2024-04-25T00:09:33.870793+0000","last_clean":"2024-04-25T00:09:33.870793+0000","last_became_active":"2024-04-25T00:09:33.868107+0000","last_became_peered":"2024-04-25T00:09:33.868107+0000","last_unstale":"2024-04-25T00:09:33.870793+0000","last_undegraded":"2024-04-25T00:09:33.870793+0000","last_fullsized":"2024-04-25T00:09:33.870793+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:08.619397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862961+0000","last_change":"2024-04-25T00:09:33.864477+0000","last_active":"2024-04-25T00:09:34.862961+0000","last_peered":"2024-04-25T00:09:34.862961+0000","last_clean":"2024-04-25T00:09:34.862961+0000","last_became_active":"2024-04-25T00:09:33.864169+0000","last_became_peered":"2024-04-25T00:09:33.864169+0000","last_unstale":"2024-04-25T00:09:34.862961+0000","last_undegraded":"2024-04-25T00:09:34.862961+0000","last_fullsized":"2024-04-25T00:09:34.862961+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.332387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168766+0000","last_change":"2024-04-25T00:09:33.874753+0000","last_active":"2024-04-25T00:09:35.168766+0000","last_peered":"2024-04-25T00:09:35.168766+0000","last_clean":"2024-04-25T00:09:35.168766+0000","last_became_active":"2024-04-25T00:09:33.874394+0000","last_became_peered":"2024-04-25T00:09:33.874394+0000","last_unstale":"2024-04-25T00:09:35.168766+0000","last_undegraded":"2024-04-25T00:09:35.168766+0000","last_fullsized":"2024-04-25T00:09:35.168766+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:28.647123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874479+0000","last_change":"2024-04-25T00:09:32.863939+0000","last_active":"2024-04-25T00:09:34.874479+0000","last_peered":"2024-04-25T00:09:34.874479+0000","last_clean":"2024-04-25T00:09:34.874479+0000","last_became_active":"2024-04-25T00:09:32.863651+0000","last_became_peered":"2024-04-25T00:09:32.863651+0000","last_unstale":"2024-04-25T00:09:34.874479+0000","last_undegraded":"2024-04-25T00:09:34.874479+0000","last_fullsized":"2024-04-25T00:09:34.874479+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:09.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863556+0000","last_change":"2024-04-25T00:09:33.867201+0000","last_active":"2024-04-25T00:09:34.863556+0000","last_peered":"2024-04-25T00:09:34.863556+0000","last_clean":"2024-04-25T00:09:34.863556+0000","last_became_active":"2024-04-25T00:09:33.866911+0000","last_became_peered":"2024-04-25T00:09:33.866911+0000","last_unstale":"2024-04-25T00:09:34.863556+0000","last_undegraded":"2024-04-25T00:09:34.863556+0000","last_fullsized":"2024-04-25T00:09:34.863556+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:54.718780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168907+0000","last_change":"2024-04-25T00:09:32.868306+0000","last_active":"2024-04-25T00:09:35.168907+0000","last_peered":"2024-04-25T00:09:35.168907+0000","last_clean":"2024-04-25T00:09:35.168907+0000","last_became_active":"2024-04-25T00:09:32.867847+0000","last_became_peered":"2024-04-25T00:09:32.867847+0000","last_unstale":"2024-04-25T00:09:35.168907+0000","last_undegraded":"2024-04-25T00:09:35.168907+0000","last_fullsized":"2024-04-25T00:09:35.168907+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:23.760066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863575+0000","last_change":"2024-04-25T00:09:33.874758+0000","last_active":"2024-04-25T00:09:34.863575+0000","last_peered":"2024-04-25T00:09:34.863575+0000","last_clean":"2024-04-25T00:09:34.863575+0000","last_became_active":"2024-04-25T00:09:33.873840+0000","last_became_peered":"2024-04-25T00:09:33.873840+0000","last_unstale":"2024-04-25T00:09:34.863575+0000","last_undegraded":"2024-04-25T00:09:34.863575+0000","last_fullsized":"2024-04-25T00:09:34.863575+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:37.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.882488+0000","last_change":"2024-04-25T00:09:32.866298+0000","last_active":"2024-04-25T00:09:33.882488+0000","last_peered":"2024-04-25T00:09:33.882488+0000","last_clean":"2024-04-25T00:09:33.882488+0000","last_became_active":"2024-04-25T00:09:32.866011+0000","last_became_peered":"2024-04-25T00:09:32.866011+0000","last_unstale":"2024-04-25T00:09:33.882488+0000","last_undegraded":"2024-04-25T00:09:33.882488+0000","last_fullsized":"2024-04-25T00:09:33.882488+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:41.242765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879218+0000","last_change":"2024-04-25T00:09:33.878077+0000","last_active":"2024-04-25T00:09:33.879218+0000","last_peered":"2024-04-25T00:09:33.879218+0000","last_clean":"2024-04-25T00:09:33.879218+0000","last_became_active":"2024-04-25T00:09:33.877808+0000","last_became_peered":"2024-04-25T00:09:33.877808+0000","last_unstale":"2024-04-25T00:09:33.879218+0000","last_undegraded":"2024-04-25T00:09:33.879218+0000","last_fullsized":"2024-04-25T00:09:33.879218+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:45.839404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872127+0000","last_change":"2024-04-25T00:09:32.870037+0000","last_active":"2024-04-25T00:09:34.872127+0000","last_peered":"2024-04-25T00:09:34.872127+0000","last_clean":"2024-04-25T00:09:34.872127+0000","last_became_active":"2024-04-25T00:09:32.869737+0000","last_became_peered":"2024-04-25T00:09:32.869737+0000","last_unstale":"2024-04-25T00:09:34.872127+0000","last_undegraded":"2024-04-25T00:09:34.872127+0000","last_fullsized":"2024-04-25T00:09:34.872127+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:16.039533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871893+0000","last_change":"2024-04-25T00:09:33.869903+0000","last_active":"2024-04-25T00:09:34.871893+0000","last_peered":"2024-04-25T00:09:34.871893+0000","last_clean":"2024-04-25T00:09:34.871893+0000","last_became_active":"2024-04-25T00:09:33.868905+0000","last_became_peered":"2024-04-25T00:09:33.868905+0000","last_unstale":"2024-04-25T00:09:34.871893+0000","last_undegraded":"2024-04-25T00:09:34.871893+0000","last_fullsized":"2024-04-25T00:09:34.871893+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:21.795335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863378+0000","last_change":"2024-04-25T00:09:32.868485+0000","last_active":"2024-04-25T00:09:34.863378+0000","last_peered":"2024-04-25T00:09:34.863378+0000","last_clean":"2024-04-25T00:09:34.863378+0000","last_became_active":"2024-04-25T00:09:32.868229+0000","last_became_peered":"2024-04-25T00:09:32.868229+0000","last_unstale":"2024-04-25T00:09:34.863378+0000","last_undegraded":"2024-04-25T00:09:34.863378+0000","last_fullsized":"2024-04-25T00:09:34.863378+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:30.649758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863102+0000","last_change":"2024-04-25T00:09:33.866550+0000","last_active":"2024-04-25T00:09:34.863102+0000","last_peered":"2024-04-25T00:09:34.863102+0000","last_clean":"2024-04-25T00:09:34.863102+0000","last_became_active":"2024-04-25T00:09:33.866017+0000","last_became_peered":"2024-04-25T00:09:33.866017+0000","last_unstale":"2024-04-25T00:09:34.863102+0000","last_undegraded":"2024-04-25T00:09:34.863102+0000","last_fullsized":"2024-04-25T00:09:34.863102+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:07.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862888+0000","last_change":"2024-04-25T00:09:32.867526+0000","last_active":"2024-04-25T00:09:34.862888+0000","last_peered":"2024-04-25T00:09:34.862888+0000","last_clean":"2024-04-25T00:09:34.862888+0000","last_became_active":"2024-04-25T00:09:32.867237+0000","last_became_peered":"2024-04-25T00:09:32.867237+0000","last_unstale":"2024-04-25T00:09:34.862888+0000","last_undegraded":"2024-04-25T00:09:34.862888+0000","last_fullsized":"2024-04-25T00:09:34.862888+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:53.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871902+0000","last_change":"2024-04-25T00:09:33.868169+0000","last_active":"2024-04-25T00:09:34.871902+0000","last_peered":"2024-04-25T00:09:34.871902+0000","last_clean":"2024-04-25T00:09:34.871902+0000","last_became_active":"2024-04-25T00:09:33.867912+0000","last_became_peered":"2024-04-25T00:09:33.867912+0000","last_unstale":"2024-04-25T00:09:34.871902+0000","last_undegraded":"2024-04-25T00:09:34.871902+0000","last_fullsized":"2024-04-25T00:09:34.871902+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:22.186105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863088+0000","last_change":"2024-04-25T00:09:32.862149+0000","last_active":"2024-04-25T00:09:34.863088+0000","last_peered":"2024-04-25T00:09:34.863088+0000","last_clean":"2024-04-25T00:09:34.863088+0000","last_became_active":"2024-04-25T00:09:32.861803+0000","last_became_peered":"2024-04-25T00:09:32.861803+0000","last_unstale":"2024-04-25T00:09:34.863088+0000","last_undegraded":"2024-04-25T00:09:34.863088+0000","last_fullsized":"2024-04-25T00:09:34.863088+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:28.007292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872261+0000","last_change":"2024-04-25T00:09:33.881209+0000","last_active":"2024-04-25T00:09:34.872261+0000","last_peered":"2024-04-25T00:09:34.872261+0000","last_clean":"2024-04-25T00:09:34.872261+0000","last_became_active":"2024-04-25T00:09:33.880746+0000","last_became_peered":"2024-04-25T00:09:33.880746+0000","last_unstale":"2024-04-25T00:09:34.872261+0000","last_undegraded":"2024-04-25T00:09:34.872261+0000","last_fullsized":"2024-04-25T00:09:34.872261+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:17.623421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863158+0000","last_change":"2024-04-25T00:09:32.867518+0000","last_active":"2024-04-25T00:09:34.863158+0000","last_peered":"2024-04-25T00:09:34.863158+0000","last_clean":"2024-04-25T00:09:34.863158+0000","last_became_active":"2024-04-25T00:09:32.867185+0000","last_became_peered":"2024-04-25T00:09:32.867185+0000","last_unstale":"2024-04-25T00:09:34.863158+0000","last_undegraded":"2024-04-25T00:09:34.863158+0000","last_fullsized":"2024-04-25T00:09:34.863158+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:17.118768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.872577+0000","last_change":"2024-04-25T00:09:33.869306+0000","last_active":"2024-04-25T00:09:33.872577+0000","last_peered":"2024-04-25T00:09:33.872577+0000","last_clean":"2024-04-25T00:09:33.872577+0000","last_became_active":"2024-04-25T00:09:33.868632+0000","last_became_peered":"2024-04-25T00:09:33.868632+0000","last_unstale":"2024-04-25T00:09:33.872577+0000","last_undegraded":"2024-04-25T00:09:33.872577+0000","last_fullsized":"2024-04-25T00:09:33.872577+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:00.532181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863652+0000","last_change":"2024-04-25T00:09:32.864351+0000","last_active":"2024-04-25T00:09:34.863652+0000","last_peered":"2024-04-25T00:09:34.863652+0000","last_clean":"2024-04-25T00:09:34.863652+0000","last_became_active":"2024-04-25T00:09:32.864085+0000","last_became_peered":"2024-04-25T00:09:32.864085+0000","last_unstale":"2024-04-25T00:09:34.863652+0000","last_undegraded":"2024-04-25T00:09:34.863652+0000","last_fullsized":"2024-04-25T00:09:34.863652+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:57.123376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872043+0000","last_change":"2024-04-25T00:09:33.881867+0000","last_active":"2024-04-25T00:09:34.872043+0000","last_peered":"2024-04-25T00:09:34.872043+0000","last_clean":"2024-04-25T00:09:34.872043+0000","last_became_active":"2024-04-25T00:09:33.881578+0000","last_became_peered":"2024-04-25T00:09:33.881578+0000","last_unstale":"2024-04-25T00:09:34.872043+0000","last_undegraded":"2024-04-25T00:09:34.872043+0000","last_fullsized":"2024-04-25T00:09:34.872043+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:44.059497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"181'3","reported_seq":15,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.882347+0000","last_change":"2024-04-25T00:09:32.866762+0000","last_active":"2024-04-25T00:09:33.882347+0000","last_peered":"2024-04-25T00:09:33.882347+0000","last_clean":"2024-04-25T00:09:33.882347+0000","last_became_active":"2024-04-25T00:09:32.866452+0000","last_became_peered":"2024-04-25T00:09:32.866452+0000","last_unstale":"2024-04-25T00:09:33.882347+0000","last_undegraded":"2024-04-25T00:09:33.882347+0000","last_fullsized":"2024-04-25T00:09:33.882347+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:49.965548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863728+0000","last_change":"2024-04-25T00:09:33.867471+0000","last_active":"2024-04-25T00:09:34.863728+0000","last_peered":"2024-04-25T00:09:34.863728+0000","last_clean":"2024-04-25T00:09:34.863728+0000","last_became_active":"2024-04-25T00:09:33.867207+0000","last_became_peered":"2024-04-25T00:09:33.867207+0000","last_unstale":"2024-04-25T00:09:34.863728+0000","last_undegraded":"2024-04-25T00:09:34.863728+0000","last_fullsized":"2024-04-25T00:09:34.863728+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:58.832182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":20,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872192+0000","last_change":"2024-04-25T00:09:32.869149+0000","last_active":"2024-04-25T00:09:34.872192+0000","last_peered":"2024-04-25T00:09:34.872192+0000","last_clean":"2024-04-25T00:09:34.872192+0000","last_became_active":"2024-04-25T00:09:32.868846+0000","last_became_peered":"2024-04-25T00:09:32.868846+0000","last_unstale":"2024-04-25T00:09:34.872192+0000","last_undegraded":"2024-04-25T00:09:34.872192+0000","last_fullsized":"2024-04-25T00:09:34.872192+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:18.348472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863360+0000","last_change":"2024-04-25T00:09:33.870433+0000","last_active":"2024-04-25T00:09:34.863360+0000","last_peered":"2024-04-25T00:09:34.863360+0000","last_clean":"2024-04-25T00:09:34.863360+0000","last_became_active":"2024-04-25T00:09:33.869736+0000","last_became_peered":"2024-04-25T00:09:33.869736+0000","last_unstale":"2024-04-25T00:09:34.863360+0000","last_undegraded":"2024-04-25T00:09:34.863360+0000","last_fullsized":"2024-04-25T00:09:34.863360+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:59.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863562+0000","last_change":"2024-04-25T00:09:32.870900+0000","last_active":"2024-04-25T00:09:34.863562+0000","last_peered":"2024-04-25T00:09:34.863562+0000","last_clean":"2024-04-25T00:09:34.863562+0000","last_became_active":"2024-04-25T00:09:32.870635+0000","last_became_peered":"2024-04-25T00:09:32.870635+0000","last_unstale":"2024-04-25T00:09:34.863562+0000","last_undegraded":"2024-04-25T00:09:34.863562+0000","last_fullsized":"2024-04-25T00:09:34.863562+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:51.613038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.876184+0000","last_change":"2024-04-25T00:09:33.874352+0000","last_active":"2024-04-25T00:09:33.876184+0000","last_peered":"2024-04-25T00:09:33.876184+0000","last_clean":"2024-04-25T00:09:33.876184+0000","last_became_active":"2024-04-25T00:09:33.874086+0000","last_became_peered":"2024-04-25T00:09:33.874086+0000","last_unstale":"2024-04-25T00:09:33.876184+0000","last_undegraded":"2024-04-25T00:09:33.876184+0000","last_fullsized":"2024-04-25T00:09:33.876184+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:21.542440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863285+0000","last_change":"2024-04-25T00:09:32.860288+0000","last_active":"2024-04-25T00:09:34.863285+0000","last_peered":"2024-04-25T00:09:34.863285+0000","last_clean":"2024-04-25T00:09:34.863285+0000","last_became_active":"2024-04-25T00:09:32.860010+0000","last_became_peered":"2024-04-25T00:09:32.860010+0000","last_unstale":"2024-04-25T00:09:34.863285+0000","last_undegraded":"2024-04-25T00:09:34.863285+0000","last_fullsized":"2024-04-25T00:09:34.863285+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:40.355571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.874431+0000","last_change":"2024-04-25T00:09:33.867592+0000","last_active":"2024-04-25T00:09:33.874431+0000","last_peered":"2024-04-25T00:09:33.874431+0000","last_clean":"2024-04-25T00:09:33.874431+0000","last_became_active":"2024-04-25T00:09:33.867328+0000","last_became_peered":"2024-04-25T00:09:33.867328+0000","last_unstale":"2024-04-25T00:09:33.874431+0000","last_undegraded":"2024-04-25T00:09:33.874431+0000","last_fullsized":"2024-04-25T00:09:33.874431+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:35.826108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.860512+0000","last_change":"2024-04-25T00:09:32.862727+0000","last_active":"2024-04-25T00:09:33.860512+0000","last_peered":"2024-04-25T00:09:33.860512+0000","last_clean":"2024-04-25T00:09:33.860512+0000","last_became_active":"2024-04-25T00:09:32.862467+0000","last_became_peered":"2024-04-25T00:09:32.862467+0000","last_unstale":"2024-04-25T00:09:33.860512+0000","last_undegraded":"2024-04-25T00:09:33.860512+0000","last_fullsized":"2024-04-25T00:09:33.860512+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:20.083614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863840+0000","last_change":"2024-04-25T00:09:33.875462+0000","last_active":"2024-04-25T00:09:34.863840+0000","last_peered":"2024-04-25T00:09:34.863840+0000","last_clean":"2024-04-25T00:09:34.863840+0000","last_became_active":"2024-04-25T00:09:33.875199+0000","last_became_peered":"2024-04-25T00:09:33.875199+0000","last_unstale":"2024-04-25T00:09:34.863840+0000","last_undegraded":"2024-04-25T00:09:34.863840+0000","last_fullsized":"2024-04-25T00:09:34.863840+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:55.059466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"181'8","reported_seq":20,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.883499+0000","last_change":"2024-04-25T00:09:32.865419+0000","last_active":"2024-04-25T00:09:33.883499+0000","last_peered":"2024-04-25T00:09:33.883499+0000","last_clean":"2024-04-25T00:09:33.883499+0000","last_became_active":"2024-04-25T00:09:32.865121+0000","last_became_peered":"2024-04-25T00:09:32.865121+0000","last_unstale":"2024-04-25T00:09:33.883499+0000","last_undegraded":"2024-04-25T00:09:33.883499+0000","last_fullsized":"2024-04-25T00:09:33.883499+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:20.150413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168820+0000","last_change":"2024-04-25T00:09:33.879136+0000","last_active":"2024-04-25T00:09:35.168820+0000","last_peered":"2024-04-25T00:09:35.168820+0000","last_clean":"2024-04-25T00:09:35.168820+0000","last_became_active":"2024-04-25T00:09:33.876729+0000","last_became_peered":"2024-04-25T00:09:33.876729+0000","last_unstale":"2024-04-25T00:09:35.168820+0000","last_undegraded":"2024-04-25T00:09:35.168820+0000","last_fullsized":"2024-04-25T00:09:35.168820+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:10.192491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863251+0000","last_change":"2024-04-25T00:09:32.872419+0000","last_active":"2024-04-25T00:09:34.863251+0000","last_peered":"2024-04-25T00:09:34.863251+0000","last_clean":"2024-04-25T00:09:34.863251+0000","last_became_active":"2024-04-25T00:09:32.872151+0000","last_became_peered":"2024-04-25T00:09:32.872151+0000","last_unstale":"2024-04-25T00:09:34.863251+0000","last_undegraded":"2024-04-25T00:09:34.863251+0000","last_fullsized":"2024-04-25T00:09:34.863251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:45.275982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168889+0000","last_change":"2024-04-25T00:09:33.871091+0000","last_active":"2024-04-25T00:09:35.168889+0000","last_peered":"2024-04-25T00:09:35.168889+0000","last_clean":"2024-04-25T00:09:35.168889+0000","last_became_active":"2024-04-25T00:09:33.870803+0000","last_became_peered":"2024-04-25T00:09:33.870803+0000","last_unstale":"2024-04-25T00:09:35.168889+0000","last_undegraded":"2024-04-25T00:09:35.168889+0000","last_fullsized":"2024-04-25T00:09:35.168889+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:12.471194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168966+0000","last_change":"2024-04-25T00:09:32.868130+0000","last_active":"2024-04-25T00:09:35.168966+0000","last_peered":"2024-04-25T00:09:35.168966+0000","last_clean":"2024-04-25T00:09:35.168966+0000","last_became_active":"2024-04-25T00:09:32.867778+0000","last_became_peered":"2024-04-25T00:09:32.867778+0000","last_unstale":"2024-04-25T00:09:35.168966+0000","last_undegraded":"2024-04-25T00:09:35.168966+0000","last_fullsized":"2024-04-25T00:09:35.168966+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:31.168558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.876812+0000","last_change":"2024-04-25T00:09:33.874649+0000","last_active":"2024-04-25T00:09:33.876812+0000","last_peered":"2024-04-25T00:09:33.876812+0000","last_clean":"2024-04-25T00:09:33.876812+0000","last_became_active":"2024-04-25T00:09:33.874370+0000","last_became_peered":"2024-04-25T00:09:33.874370+0000","last_unstale":"2024-04-25T00:09:33.876812+0000","last_undegraded":"2024-04-25T00:09:33.876812+0000","last_fullsized":"2024-04-25T00:09:33.876812+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:54.547827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.883237+0000","last_change":"2024-04-25T00:09:32.865123+0000","last_active":"2024-04-25T00:09:33.883237+0000","last_peered":"2024-04-25T00:09:33.883237+0000","last_clean":"2024-04-25T00:09:33.883237+0000","last_became_active":"2024-04-25T00:09:32.864038+0000","last_became_peered":"2024-04-25T00:09:32.864038+0000","last_unstale":"2024-04-25T00:09:33.883237+0000","last_undegraded":"2024-04-25T00:09:33.883237+0000","last_fullsized":"2024-04-25T00:09:33.883237+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:26.788441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.876230+0000","last_change":"2024-04-25T00:09:33.874638+0000","last_active":"2024-04-25T00:09:33.876230+0000","last_peered":"2024-04-25T00:09:33.876230+0000","last_clean":"2024-04-25T00:09:33.876230+0000","last_became_active":"2024-04-25T00:09:33.874363+0000","last_became_peered":"2024-04-25T00:09:33.874363+0000","last_unstale":"2024-04-25T00:09:33.876230+0000","last_undegraded":"2024-04-25T00:09:33.876230+0000","last_fullsized":"2024-04-25T00:09:33.876230+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:15.371941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.861988+0000","last_change":"2024-04-25T00:09:32.871043+0000","last_active":"2024-04-25T00:09:33.861988+0000","last_peered":"2024-04-25T00:09:33.861988+0000","last_clean":"2024-04-25T00:09:33.861988+0000","last_became_active":"2024-04-25T00:09:32.870708+0000","last_became_peered":"2024-04-25T00:09:32.870708+0000","last_unstale":"2024-04-25T00:09:33.861988+0000","last_undegraded":"2024-04-25T00:09:33.861988+0000","last_fullsized":"2024-04-25T00:09:33.861988+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:37:50.624952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879109+0000","last_change":"2024-04-25T00:09:33.878329+0000","last_active":"2024-04-25T00:09:33.879109+0000","last_peered":"2024-04-25T00:09:33.879109+0000","last_clean":"2024-04-25T00:09:33.879109+0000","last_became_active":"2024-04-25T00:09:33.878062+0000","last_became_peered":"2024-04-25T00:09:33.878062+0000","last_unstale":"2024-04-25T00:09:33.879109+0000","last_undegraded":"2024-04-25T00:09:33.879109+0000","last_fullsized":"2024-04-25T00:09:33.879109+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:17.344174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.882003+0000","last_change":"2024-04-25T00:09:32.865066+0000","last_active":"2024-04-25T00:09:33.882003+0000","last_peered":"2024-04-25T00:09:33.882003+0000","last_clean":"2024-04-25T00:09:33.882003+0000","last_became_active":"2024-04-25T00:09:32.864793+0000","last_became_peered":"2024-04-25T00:09:32.864793+0000","last_unstale":"2024-04-25T00:09:33.882003+0000","last_undegraded":"2024-04-25T00:09:33.882003+0000","last_fullsized":"2024-04-25T00:09:33.882003+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:49.165483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.878799+0000","last_change":"2024-04-25T00:09:33.877986+0000","last_active":"2024-04-25T00:09:33.878799+0000","last_peered":"2024-04-25T00:09:33.878799+0000","last_clean":"2024-04-25T00:09:33.878799+0000","last_became_active":"2024-04-25T00:09:33.877728+0000","last_became_peered":"2024-04-25T00:09:33.877728+0000","last_unstale":"2024-04-25T00:09:33.878799+0000","last_undegraded":"2024-04-25T00:09:33.878799+0000","last_fullsized":"2024-04-25T00:09:33.878799+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.356214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863732+0000","last_change":"2024-04-25T00:09:32.870566+0000","last_active":"2024-04-25T00:09:34.863732+0000","last_peered":"2024-04-25T00:09:34.863732+0000","last_clean":"2024-04-25T00:09:34.863732+0000","last_became_active":"2024-04-25T00:09:32.869700+0000","last_became_peered":"2024-04-25T00:09:32.869700+0000","last_unstale":"2024-04-25T00:09:34.863732+0000","last_undegraded":"2024-04-25T00:09:34.863732+0000","last_fullsized":"2024-04-25T00:09:34.863732+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:16.007479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168567+0000","last_change":"2024-04-25T00:09:33.878569+0000","last_active":"2024-04-25T00:09:35.168567+0000","last_peered":"2024-04-25T00:09:35.168567+0000","last_clean":"2024-04-25T00:09:35.168567+0000","last_became_active":"2024-04-25T00:09:33.878305+0000","last_became_peered":"2024-04-25T00:09:33.878305+0000","last_unstale":"2024-04-25T00:09:35.168567+0000","last_undegraded":"2024-04-25T00:09:35.168567+0000","last_fullsized":"2024-04-25T00:09:35.168567+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:37.456036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863824+0000","last_change":"2024-04-25T00:09:32.861027+0000","last_active":"2024-04-25T00:09:34.863824+0000","last_peered":"2024-04-25T00:09:34.863824+0000","last_clean":"2024-04-25T00:09:34.863824+0000","last_became_active":"2024-04-25T00:09:32.860724+0000","last_became_peered":"2024-04-25T00:09:32.860724+0000","last_unstale":"2024-04-25T00:09:34.863824+0000","last_undegraded":"2024-04-25T00:09:34.863824+0000","last_fullsized":"2024-04-25T00:09:34.863824+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:23.820088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863453+0000","last_change":"2024-04-25T00:09:33.866398+0000","last_active":"2024-04-25T00:09:34.863453+0000","last_peered":"2024-04-25T00:09:34.863453+0000","last_clean":"2024-04-25T00:09:34.863453+0000","last_became_active":"2024-04-25T00:09:33.866122+0000","last_became_peered":"2024-04-25T00:09:33.866122+0000","last_unstale":"2024-04-25T00:09:34.863453+0000","last_undegraded":"2024-04-25T00:09:34.863453+0000","last_fullsized":"2024-04-25T00:09:34.863453+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:27.159690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863217+0000","last_change":"2024-04-25T00:09:32.872369+0000","last_active":"2024-04-25T00:09:34.863217+0000","last_peered":"2024-04-25T00:09:34.863217+0000","last_clean":"2024-04-25T00:09:34.863217+0000","last_became_active":"2024-04-25T00:09:32.872083+0000","last_became_peered":"2024-04-25T00:09:32.872083+0000","last_unstale":"2024-04-25T00:09:34.863217+0000","last_undegraded":"2024-04-25T00:09:34.863217+0000","last_fullsized":"2024-04-25T00:09:34.863217+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:36.451185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874414+0000","last_change":"2024-04-25T00:09:33.874713+0000","last_active":"2024-04-25T00:09:34.874414+0000","last_peered":"2024-04-25T00:09:34.874414+0000","last_clean":"2024-04-25T00:09:34.874414+0000","last_became_active":"2024-04-25T00:09:33.874450+0000","last_became_peered":"2024-04-25T00:09:33.874450+0000","last_unstale":"2024-04-25T00:09:34.874414+0000","last_undegraded":"2024-04-25T00:09:34.874414+0000","last_fullsized":"2024-04-25T00:09:34.874414+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:59.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.883754+0000","last_change":"2024-04-25T00:09:32.864284+0000","last_active":"2024-04-25T00:09:33.883754+0000","last_peered":"2024-04-25T00:09:33.883754+0000","last_clean":"2024-04-25T00:09:33.883754+0000","last_became_active":"2024-04-25T00:09:32.863870+0000","last_became_peered":"2024-04-25T00:09:32.863870+0000","last_unstale":"2024-04-25T00:09:33.883754+0000","last_undegraded":"2024-04-25T00:09:33.883754+0000","last_fullsized":"2024-04-25T00:09:33.883754+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:23.760066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864187+0000","last_change":"2024-04-25T00:09:33.868631+0000","last_active":"2024-04-25T00:09:34.864187+0000","last_peered":"2024-04-25T00:09:34.864187+0000","last_clean":"2024-04-25T00:09:34.864187+0000","last_became_active":"2024-04-25T00:09:33.866936+0000","last_became_peered":"2024-04-25T00:09:33.866936+0000","last_unstale":"2024-04-25T00:09:34.864187+0000","last_undegraded":"2024-04-25T00:09:34.864187+0000","last_fullsized":"2024-04-25T00:09:34.864187+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:45.734090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"182'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.873583+0000","last_change":"2024-04-25T00:09:32.869135+0000","last_active":"2024-04-25T00:09:34.873583+0000","last_peered":"2024-04-25T00:09:34.873583+0000","last_clean":"2024-04-25T00:09:34.873583+0000","last_became_active":"2024-04-25T00:09:32.868833+0000","last_became_peered":"2024-04-25T00:09:32.868833+0000","last_unstale":"2024-04-25T00:09:34.873583+0000","last_undegraded":"2024-04-25T00:09:34.873583+0000","last_fullsized":"2024-04-25T00:09:34.873583+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:22.181677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168623+0000","last_change":"2024-04-25T00:09:33.869214+0000","last_active":"2024-04-25T00:09:35.168623+0000","last_peered":"2024-04-25T00:09:35.168623+0000","last_clean":"2024-04-25T00:09:35.168623+0000","last_became_active":"2024-04-25T00:09:33.868863+0000","last_became_peered":"2024-04-25T00:09:33.868863+0000","last_unstale":"2024-04-25T00:09:35.168623+0000","last_undegraded":"2024-04-25T00:09:35.168623+0000","last_fullsized":"2024-04-25T00:09:35.168623+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:31.654609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863433+0000","last_change":"2024-04-25T00:09:32.868078+0000","last_active":"2024-04-25T00:09:34.863433+0000","last_peered":"2024-04-25T00:09:34.863433+0000","last_clean":"2024-04-25T00:09:34.863433+0000","last_became_active":"2024-04-25T00:09:32.867779+0000","last_became_peered":"2024-04-25T00:09:32.867779+0000","last_unstale":"2024-04-25T00:09:34.863433+0000","last_undegraded":"2024-04-25T00:09:34.863433+0000","last_fullsized":"2024-04-25T00:09:34.863433+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:27.642272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863700+0000","last_change":"2024-04-25T00:09:33.871883+0000","last_active":"2024-04-25T00:09:34.863700+0000","last_peered":"2024-04-25T00:09:34.863700+0000","last_clean":"2024-04-25T00:09:34.863700+0000","last_became_active":"2024-04-25T00:09:33.871586+0000","last_became_peered":"2024-04-25T00:09:33.871586+0000","last_unstale":"2024-04-25T00:09:34.863700+0000","last_undegraded":"2024-04-25T00:09:34.863700+0000","last_fullsized":"2024-04-25T00:09:34.863700+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:08.604008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.862153+0000","last_change":"2024-04-25T00:09:32.863484+0000","last_active":"2024-04-25T00:09:33.862153+0000","last_peered":"2024-04-25T00:09:33.862153+0000","last_clean":"2024-04-25T00:09:33.862153+0000","last_became_active":"2024-04-25T00:09:32.863226+0000","last_became_peered":"2024-04-25T00:09:32.863226+0000","last_unstale":"2024-04-25T00:09:33.862153+0000","last_undegraded":"2024-04-25T00:09:33.862153+0000","last_fullsized":"2024-04-25T00:09:33.862153+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.948313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863681+0000","last_change":"2024-04-25T00:09:33.876228+0000","last_active":"2024-04-25T00:09:34.863681+0000","last_peered":"2024-04-25T00:09:34.863681+0000","last_clean":"2024-04-25T00:09:34.863681+0000","last_became_active":"2024-04-25T00:09:33.875900+0000","last_became_peered":"2024-04-25T00:09:33.875900+0000","last_unstale":"2024-04-25T00:09:34.863681+0000","last_undegraded":"2024-04-25T00:09:34.863681+0000","last_fullsized":"2024-04-25T00:09:34.863681+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.901658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872093+0000","last_change":"2024-04-25T00:09:32.870945+0000","last_active":"2024-04-25T00:09:34.872093+0000","last_peered":"2024-04-25T00:09:34.872093+0000","last_clean":"2024-04-25T00:09:34.872093+0000","last_became_active":"2024-04-25T00:09:32.870577+0000","last_became_peered":"2024-04-25T00:09:32.870577+0000","last_unstale":"2024-04-25T00:09:34.872093+0000","last_undegraded":"2024-04-25T00:09:34.872093+0000","last_fullsized":"2024-04-25T00:09:34.872093+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:55.965150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874401+0000","last_change":"2024-04-25T00:09:33.872463+0000","last_active":"2024-04-25T00:09:34.874401+0000","last_peered":"2024-04-25T00:09:34.874401+0000","last_clean":"2024-04-25T00:09:34.874401+0000","last_became_active":"2024-04-25T00:09:33.872181+0000","last_became_peered":"2024-04-25T00:09:33.872181+0000","last_unstale":"2024-04-25T00:09:34.874401+0000","last_undegraded":"2024-04-25T00:09:34.874401+0000","last_fullsized":"2024-04-25T00:09:34.874401+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:59.450004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863450+0000","last_change":"2024-04-25T00:09:32.872901+0000","last_active":"2024-04-25T00:09:34.863450+0000","last_peered":"2024-04-25T00:09:34.863450+0000","last_clean":"2024-04-25T00:09:34.863450+0000","last_became_active":"2024-04-25T00:09:32.872596+0000","last_became_peered":"2024-04-25T00:09:32.872596+0000","last_unstale":"2024-04-25T00:09:34.863450+0000","last_undegraded":"2024-04-25T00:09:34.863450+0000","last_fullsized":"2024-04-25T00:09:34.863450+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:24.471772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.867045+0000","last_change":"2024-04-25T00:09:33.864759+0000","last_active":"2024-04-25T00:09:33.867045+0000","last_peered":"2024-04-25T00:09:33.867045+0000","last_clean":"2024-04-25T00:09:33.867045+0000","last_became_active":"2024-04-25T00:09:33.864497+0000","last_became_peered":"2024-04-25T00:09:33.864497+0000","last_unstale":"2024-04-25T00:09:33.867045+0000","last_undegraded":"2024-04-25T00:09:33.867045+0000","last_fullsized":"2024-04-25T00:09:33.867045+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:32.173409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863516+0000","last_change":"2024-04-25T00:09:32.859351+0000","last_active":"2024-04-25T00:09:34.863516+0000","last_peered":"2024-04-25T00:09:34.863516+0000","last_clean":"2024-04-25T00:09:34.863516+0000","last_became_active":"2024-04-25T00:09:32.859068+0000","last_became_peered":"2024-04-25T00:09:32.859068+0000","last_unstale":"2024-04-25T00:09:34.863516+0000","last_undegraded":"2024-04-25T00:09:34.863516+0000","last_fullsized":"2024-04-25T00:09:34.863516+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:39.922934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168534+0000","last_change":"2024-04-25T00:09:33.872393+0000","last_active":"2024-04-25T00:09:35.168534+0000","last_peered":"2024-04-25T00:09:35.168534+0000","last_clean":"2024-04-25T00:09:35.168534+0000","last_became_active":"2024-04-25T00:09:33.872069+0000","last_became_peered":"2024-04-25T00:09:33.872069+0000","last_unstale":"2024-04-25T00:09:35.168534+0000","last_undegraded":"2024-04-25T00:09:35.168534+0000","last_fullsized":"2024-04-25T00:09:35.168534+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:54.319661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874113+0000","last_change":"2024-04-25T00:09:32.860524+0000","last_active":"2024-04-25T00:09:34.874113+0000","last_peered":"2024-04-25T00:09:34.874113+0000","last_clean":"2024-04-25T00:09:34.874113+0000","last_became_active":"2024-04-25T00:09:32.860045+0000","last_became_peered":"2024-04-25T00:09:32.860045+0000","last_unstale":"2024-04-25T00:09:34.874113+0000","last_undegraded":"2024-04-25T00:09:34.874113+0000","last_fullsized":"2024-04-25T00:09:34.874113+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:24.471772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863583+0000","last_change":"2024-04-25T00:09:33.865604+0000","last_active":"2024-04-25T00:09:34.863583+0000","last_peered":"2024-04-25T00:09:34.863583+0000","last_clean":"2024-04-25T00:09:34.863583+0000","last_became_active":"2024-04-25T00:09:33.865349+0000","last_became_peered":"2024-04-25T00:09:33.865349+0000","last_unstale":"2024-04-25T00:09:34.863583+0000","last_undegraded":"2024-04-25T00:09:34.863583+0000","last_fullsized":"2024-04-25T00:09:34.863583+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:55.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874500+0000","last_change":"2024-04-25T00:09:32.865204+0000","last_active":"2024-04-25T00:09:34.874500+0000","last_peered":"2024-04-25T00:09:34.874500+0000","last_clean":"2024-04-25T00:09:34.874500+0000","last_became_active":"2024-04-25T00:09:32.864901+0000","last_became_peered":"2024-04-25T00:09:32.864901+0000","last_unstale":"2024-04-25T00:09:34.874500+0000","last_undegraded":"2024-04-25T00:09:34.874500+0000","last_fullsized":"2024-04-25T00:09:34.874500+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:27.642272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":515,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871873+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:09:34.871873+0000","last_peered":"2024-04-25T00:09:34.871873+0000","last_clean":"2024-04-25T00:09:34.871873+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:09:34.871873+0000","last_undegraded":"2024-04-25T00:09:34.871873+0000","last_fullsized":"2024-04-25T00:09:34.871873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2899968,"data_stored":1597900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2297856,"data_stored":1006502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55640,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316200,"statfs":{"total":96636764160,"available":96579788800,"internally_reserved":0,"allocated":2297856,"data_stored":1006502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2314240,"data_stored":1007636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2293760,"data_stored":999802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51456,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320384,"statfs":{"total":96636764160,"available":96584073216,"internally_reserved":0,"allocated":2281472,"data_stored":999620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59804,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312036,"statfs":{"total":96636764160,"available":96575524864,"internally_reserved":0,"allocated":2301952,"data_stored":1005976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52060,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319780,"statfs":{"total":96636764160,"available":96583454720,"internally_reserved":0,"allocated":2891776,"data_stored":1596378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:09:37.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T00:09:37.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.628+0000 7f7ca2e72640 1 Processor -- start 2024-04-25T00:09:37.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca2e72640 1 -- start start 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca2e72640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c9c0b44c0 0x7f7c9c162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca2e72640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c9c06dfd0 0x7f7c9c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca2e72640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c14c360 con 0x7f7c9c14bf80 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca2e72640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7c9c06e940 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca2e72640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7c9c071600 con 0x7f7c9c0b44c0 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca140f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c9c06dfd0 0x7f7c9c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca0c0e640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7c9c14bf80 0x7f7c9c070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60718/0 (socket says 172.21.15.113:60718) 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.629+0000 7f7ca0c0e640 1 -- 172.21.15.113:0/1720003288 learned_addr learned my addr 172.21.15.113:0/1720003288 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.630+0000 7f7c9bfff640 1 --2- 172.21.15.113:0/1720003288 >> v2:172.21.15.178:3300/0 conn(0x7f7c9c0b44c0 0x7f7c9c162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.630+0000 7f7ca140f640 1 -- 172.21.15.113:0/1720003288 >> v2:172.21.15.178:3300/0 conn(0x7f7c9c0b44c0 msgr2=0x7f7c9c162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.630+0000 7f7ca140f640 1 --2- 172.21.15.113:0/1720003288 >> v2:172.21.15.178:3300/0 conn(0x7f7c9c0b44c0 0x7f7c9c162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.630+0000 7f7ca140f640 1 -- 172.21.15.113:0/1720003288 >> v1:172.21.15.113:6789/0 conn(0x7f7c9c14bf80 legacy=0x7f7c9c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.630+0000 7f7c9b7fe640 1 -- 172.21.15.113:0/1720003288 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025999447 0 0) 0x7f7c9c14c360 con 0x7f7c9c14bf80 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.630+0000 7f7ca140f640 1 -- 172.21.15.113:0/1720003288 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0718e0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.630+0000 7f7ca140f640 1 --2- 172.21.15.113:0/1720003288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c06e400 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0af950 tx=0x7f7c9402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d84ccc97d30bbdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7c9b7fe640 1 -- 172.21.15.113:0/1720003288 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94007740 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7c9b7fe640 1 -- 172.21.15.113:0/1720003288 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c940078e0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7c9b7fe640 1 -- 172.21.15.113:0/1720003288 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94007cb0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 -- 172.21.15.113:0/1720003288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 msgr2=0x7f7c9c06e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/1720003288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c06e400 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0af950 tx=0x7f7c9402a710 comp rx=0 tx=0).stop 2024-04-25T00:09:37.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 -- 172.21.15.113:0/1720003288 shutdown_connections 2024-04-25T00:09:37.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/1720003288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c06e400 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/1720003288 >> v2:172.21.15.178:3300/0 conn(0x7f7c9c0b44c0 0x7f7c9c162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 -- 172.21.15.113:0/1720003288 >> 172.21.15.113:0/1720003288 conn(0x7f7c9c0b1a50 msgr2=0x7f7c9c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:37.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 -- 172.21.15.113:0/1720003288 shutdown_connections 2024-04-25T00:09:37.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.631+0000 7f7ca2e72640 1 -- 172.21.15.113:0/1720003288 wait complete. 2024-04-25T00:09:37.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.632+0000 7f7ca2e72640 1 Processor -- start 2024-04-25T00:09:37.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.632+0000 7f7ca2e72640 1 -- start start 2024-04-25T00:09:37.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca2e72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c15e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca2e72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c9c0b44c0 0x7f7c9c160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca2e72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c9c14bf80 0x7f7c9c13a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca2e72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c072180 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca2e72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c071ea0 con 0x7f7c9c14bf80 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca2e72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c163230 con 0x7f7c9c0b44c0 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca0c0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca0c0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c15e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43718/0 (socket says 172.21.15.113:43718) 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca0c0e640 1 -- 172.21.15.113:0/58587688 learned_addr learned my addr 172.21.15.113:0/58587688 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7c9bfff640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c9c0b44c0 0x7f7c9c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.633+0000 7f7ca140f640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c9c14bf80 0x7f7c9c13a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca0c0e640 1 -- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c9c0b44c0 msgr2=0x7f7c9c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca0c0e640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c9c0b44c0 0x7f7c9c160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca0c0e640 1 -- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c9c14bf80 msgr2=0x7f7c9c13a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca0c0e640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c9c14bf80 0x7f7c9c13a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca0c0e640 1 -- 172.21.15.113:0/58587688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c15fe00 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7c9bfff640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c9c0b44c0 0x7f7c9c160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca0c0e640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c15e630 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c430 tx=0x7f7c9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7c997fa640 1 -- 172.21.15.113:0/58587688 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900074e0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7c997fa640 1 -- 172.21.15.113:0/58587688 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c900047e0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c13a780 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7c997fa640 1 -- 172.21.15.113:0/58587688 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90004bd0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.634+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c15f040 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.635+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c0691a0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.635+0000 7f7c997fa640 1 -- 172.21.15.113:0/58587688 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c90011070 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.636+0000 7f7c997fa640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8006be30 0x7f7c8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:37.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.636+0000 7f7c997fa640 1 -- 172.21.15.113:0/58587688 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f7c9008aaa0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.636+0000 7f7c9bfff640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8006be30 0x7f7c8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:37.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.636+0000 7f7c9bfff640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8006be30 0x7f7c8006e2f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0b3780 tx=0x7f7c88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:37.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.640+0000 7f7c997fa640 1 -- 172.21.15.113:0/58587688 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c9c13a780 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.879+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7c9c0b4ca0 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.879+0000 7f7c997fa640 1 -- 172.21.15.113:0/58587688 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1437 (secure 0 0 0) 0x7f7c9c15fe00 con 0x7f7c9c06dfd0 2024-04-25T00:09:37.880 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:{"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":702,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":182,"num_osds":8,"num_up_osds":8,"osd_up_since":1714003085,"num_in_osds":8,"osd_in_since":1714003077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":431419392,"bytes_avail":772662693888,"bytes_total":773094113280,"write_bytes_sec":2653,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":98,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"d","status":"up:active","gid":10803}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T00:09:23.244504+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T00:09:37.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8006be30 msgr2=0x7f7c8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8006be30 0x7f7c8006e2f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0b3780 tx=0x7f7c88008040 comp rx=0 tx=0).stop 2024-04-25T00:09:37.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 msgr2=0x7f7c9c15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:37.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c15e630 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c430 tx=0x7f7c9000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 shutdown_connections 2024-04-25T00:09:37.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7c8006be30 0x7f7c8006e2f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c9c14bf80 0x7f7c9c13a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c9c0b44c0 0x7f7c9c160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 --2- 172.21.15.113:0/58587688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c9c06dfd0 0x7f7c9c15e630 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:37.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 >> 172.21.15.113:0/58587688 conn(0x7f7c9c0b1a50 msgr2=0x7f7c9c14b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:37.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 shutdown_connections 2024-04-25T00:09:37.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.881+0000 7f7ca2e72640 1 -- 172.21.15.113:0/58587688 wait complete. 2024-04-25T00:09:37.893 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T00:09:37.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:09:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.991+0000 7f75cc742640 1 Processor -- start 2024-04-25T00:09:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.991+0000 7f75cc742640 1 -- start start 2024-04-25T00:09:38.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.992+0000 7f75cc742640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75c415e320 0x7f75c40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.992+0000 7f75cc742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 0x7f75c40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.992+0000 7f75cc742640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c40d3de0 con 0x7f75c40d0b00 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.992+0000 7f75cc742640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f75c40d3410 con 0x7f75c415e320 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.992+0000 7f75cc742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75c40d2800 con 0x7f75c40d2290 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.992+0000 7f75cacdf640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f75c40d0b00 0x7f75c40b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60724/0 (socket says 172.21.15.113:60724) 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.992+0000 7f75cacdf640 1 -- 172.21.15.113:0/677609855 learned_addr learned my addr 172.21.15.113:0/677609855 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c9cdd640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 0x7f75c40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752829808 0 0) 0x7f75c40d3de0 con 0x7f75c40d0b00 2024-04-25T00:09:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75b0003520 con 0x7f75c40d0b00 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75ca4de640 1 --2- 172.21.15.113:0/677609855 >> v2:172.21.15.178:3300/0 conn(0x7f75c415e320 0x7f75c40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 792130489 0 0) 0x7f75b0003520 con 0x7f75c40d0b00 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 msgr2=0x7f75c40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 0x7f75c40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 >> v2:172.21.15.178:3300/0 conn(0x7f75c415e320 msgr2=0x7f75c40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 --2- 172.21.15.113:0/677609855 >> v2:172.21.15.178:3300/0 conn(0x7f75c415e320 0x7f75c40d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c40d2ae0 con 0x7f75c40d0b00 2024-04-25T00:09:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f75c00019f0 con 0x7f75c40d0b00 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 >> v1:172.21.15.113:6789/0 conn(0x7f75c40d0b00 legacy=0x7f75c40b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75b00039b0 0x7f75b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 0x7f75b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75c40d3410 con 0x7f75b0004d40 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75c40d2800 con 0x7f75b00039b0 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75b000b9c0 con 0x7f75c40d2290 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75cacdf640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75ca4de640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75b00039b0 0x7f75b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75c9cdd640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 0x7f75b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75cacdf640 1 -- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 msgr2=0x7f75b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75cacdf640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40d2290 0x7f75b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75cacdf640 1 -- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75b00039b0 msgr2=0x7f75b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75cacdf640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75b00039b0 0x7f75b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.993+0000 7f75cacdf640 1 -- 172.21.15.113:0/677609855 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b0003520 con 0x7f75b0004d40 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.994+0000 7f75cacdf640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75b00051c0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f75c00013d0 tx=0x7f75b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc05cf949315c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.994+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000a4e0 con 0x7f75b0004d40 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.994+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c002f070 con 0x7f75b0004d40 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.994+0000 7f75c94dc640 1 -- 172.21.15.113:0/677609855 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c002bce0 con 0x7f75b0004d40 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.995+0000 7f75cc742640 1 -- 172.21.15.113:0/677609855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 msgr2=0x7f75b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.995+0000 7f75cc742640 1 --2- 172.21.15.113:0/677609855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75b00051c0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f75c00013d0 tx=0x7f75b0005830 comp rx=0 tx=0).stop 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.995+0000 7f75cacdf640 1 -- 172.21.15.113:0/677609855 reap_dead start 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.995+0000 7f75cc742640 1 -- 172.21.15.113:0/677609855 shutdown_connections 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.995+0000 7f75cc742640 1 -- 172.21.15.113:0/677609855 >> 172.21.15.113:0/677609855 conn(0x7f75c40b5c20 msgr2=0x7f75c40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.995+0000 7f75cc742640 1 -- 172.21.15.113:0/677609855 shutdown_connections 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.995+0000 7f75cc742640 1 -- 172.21.15.113:0/677609855 wait complete. 2024-04-25T00:09:38.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.996+0000 7f75cc742640 1 Processor -- start 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.996+0000 7f75cc742640 1 -- start start 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75c414e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c4159b70 0x7f75c4159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40b4900 0x7f75c40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75c415e700 con 0x7f75b0004d40 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75c40d30a0 con 0x7f75c4159b70 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75c415e880 con 0x7f75c40b4900 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75c414e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75c414e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43726/0 (socket says 172.21.15.113:43726) 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 -- 172.21.15.113:0/68243032 learned_addr learned my addr 172.21.15.113:0/68243032 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cacdf640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40b4900 0x7f75c40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75c9cdd640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c4159b70 0x7f75c4159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 -- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40b4900 msgr2=0x7f75c40b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40b4900 0x7f75c40b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 -- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c4159b70 msgr2=0x7f75c4159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c4159b70 0x7f75c4159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 -- 172.21.15.113:0/68243032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c40cb280 con 0x7f75b0004d40 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cacdf640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40b4900 0x7f75c40b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75ca4de640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75c414e3b0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f75bc011990 tx=0x7f75bc011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75baffd640 1 -- 172.21.15.113:0/68243032 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00fad0 con 0x7f75b0004d40 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75baffd640 1 -- 172.21.15.113:0/68243032 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc00fc70 con 0x7f75b0004d40 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c40cb560 con 0x7f75b0004d40 2024-04-25T00:09:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75baffd640 1 -- 172.21.15.113:0/68243032 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc014440 con 0x7f75b0004d40 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.997+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c40cbb20 con 0x7f75b0004d40 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.998+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c40691a0 con 0x7f75b0004d40 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.999+0000 7f75baffd640 1 -- 172.21.15.113:0/68243032 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75bc012040 con 0x7f75b0004d40 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.999+0000 7f75baffd640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be30 0x7f75a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.999+0000 7f75c9cdd640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be30 0x7f75a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:37.999+0000 7f75baffd640 1 -- 172.21.15.113:0/68243032 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f75bc08b560 con 0x7f75b0004d40 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.000+0000 7f75c9cdd640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be30 0x7f75a806e2f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f75c40b7db0 tx=0x7f75b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:38.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.004+0000 7f75baffd640 1 -- 172.21.15.113:0/68243032 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75bc019030 con 0x7f75b0004d40 2024-04-25T00:09:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.173+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75c40b94c0 con 0x7f75a806be30 2024-04-25T00:09:38.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.179+0000 7f75baffd640 1 -- 172.21.15.113:0/68243032 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f75c40b94c0 con 0x7f75a806be30 2024-04-25T00:09:38.182 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be30 msgr2=0x7f75a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be30 0x7f75a806e2f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f75c40b7db0 tx=0x7f75b4008040 comp rx=0 tx=0).stop 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 msgr2=0x7f75c414e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75c414e3b0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f75bc011990 tx=0x7f75bc011e50 comp rx=0 tx=0).stop 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 shutdown_connections 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f75a806be30 0x7f75a806e2f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c4159b70 0x7f75c4159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c40b4900 0x7f75c40b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 --2- 172.21.15.113:0/68243032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75b0004d40 0x7f75c414e3b0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 >> 172.21.15.113:0/68243032 conn(0x7f75c40b5c20 msgr2=0x7f75c40d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 shutdown_connections 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:38.182+0000 7f75cc742640 1 -- 172.21.15.113:0/68243032 wait complete. 2024-04-25T00:09:38.193 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-04-25T00:09:37.246935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":254,"num_write_kb":3476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421308,"kb_used_data":19120,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754553412,"statfs":{"total":773094113280,"available":772662693888,"internally_reserved":0,"allocated":19578880,"data_stored":9220316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.401689"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863971+0000","last_change":"2024-04-25T00:09:33.871025+0000","last_active":"2024-04-25T00:09:34.863971+0000","last_peered":"2024-04-25T00:09:34.863971+0000","last_clean":"2024-04-25T00:09:34.863971+0000","last_became_active":"2024-04-25T00:09:33.870439+0000","last_became_peered":"2024-04-25T00:09:33.870439+0000","last_unstale":"2024-04-25T00:09:34.863971+0000","last_undegraded":"2024-04-25T00:09:34.863971+0000","last_fullsized":"2024-04-25T00:09:34.863971+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:59.226155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168700+0000","last_change":"2024-04-25T00:09:33.878604+0000","last_active":"2024-04-25T00:09:35.168700+0000","last_peered":"2024-04-25T00:09:35.168700+0000","last_clean":"2024-04-25T00:09:35.168700+0000","last_became_active":"2024-04-25T00:09:33.878328+0000","last_became_peered":"2024-04-25T00:09:33.878328+0000","last_unstale":"2024-04-25T00:09:35.168700+0000","last_undegraded":"2024-04-25T00:09:35.168700+0000","last_fullsized":"2024-04-25T00:09:35.168700+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879389+0000","last_change":"2024-04-25T00:09:33.878325+0000","last_active":"2024-04-25T00:09:33.879389+0000","last_peered":"2024-04-25T00:09:33.879389+0000","last_clean":"2024-04-25T00:09:33.879389+0000","last_became_active":"2024-04-25T00:09:33.877495+0000","last_became_peered":"2024-04-25T00:09:33.877495+0000","last_unstale":"2024-04-25T00:09:33.879389+0000","last_undegraded":"2024-04-25T00:09:33.879389+0000","last_fullsized":"2024-04-25T00:09:33.879389+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:09:28.296941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871982+0000","last_change":"2024-04-25T00:09:33.869224+0000","last_active":"2024-04-25T00:09:34.871982+0000","last_peered":"2024-04-25T00:09:34.871982+0000","last_clean":"2024-04-25T00:09:34.871982+0000","last_became_active":"2024-04-25T00:09:33.868734+0000","last_became_peered":"2024-04-25T00:09:33.868734+0000","last_unstale":"2024-04-25T00:09:34.871982+0000","last_undegraded":"2024-04-25T00:09:34.871982+0000","last_fullsized":"2024-04-25T00:09:34.871982+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:16.109657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863334+0000","last_change":"2024-04-25T00:09:33.869714+0000","last_active":"2024-04-25T00:09:34.863334+0000","last_peered":"2024-04-25T00:09:34.863334+0000","last_clean":"2024-04-25T00:09:34.863334+0000","last_became_active":"2024-04-25T00:09:33.869435+0000","last_became_peered":"2024-04-25T00:09:33.869435+0000","last_unstale":"2024-04-25T00:09:34.863334+0000","last_undegraded":"2024-04-25T00:09:34.863334+0000","last_fullsized":"2024-04-25T00:09:34.863334+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:59.450004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863112+0000","last_change":"2024-04-25T00:09:33.866179+0000","last_active":"2024-04-25T00:09:34.863112+0000","last_peered":"2024-04-25T00:09:34.863112+0000","last_clean":"2024-04-25T00:09:34.863112+0000","last_became_active":"2024-04-25T00:09:33.865891+0000","last_became_peered":"2024-04-25T00:09:33.865891+0000","last_unstale":"2024-04-25T00:09:34.863112+0000","last_undegraded":"2024-04-25T00:09:34.863112+0000","last_fullsized":"2024-04-25T00:09:34.863112+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:00.144856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863588+0000","last_change":"2024-04-25T00:09:33.875207+0000","last_active":"2024-04-25T00:09:34.863588+0000","last_peered":"2024-04-25T00:09:34.863588+0000","last_clean":"2024-04-25T00:09:34.863588+0000","last_became_active":"2024-04-25T00:09:33.874674+0000","last_became_peered":"2024-04-25T00:09:33.874674+0000","last_unstale":"2024-04-25T00:09:34.863588+0000","last_undegraded":"2024-04-25T00:09:34.863588+0000","last_fullsized":"2024-04-25T00:09:34.863588+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:14.333375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862963+0000","last_change":"2024-04-25T00:09:33.860996+0000","last_active":"2024-04-25T00:09:34.862963+0000","last_peered":"2024-04-25T00:09:34.862963+0000","last_clean":"2024-04-25T00:09:34.862963+0000","last_became_active":"2024-04-25T00:09:33.860726+0000","last_became_peered":"2024-04-25T00:09:33.860726+0000","last_unstale":"2024-04-25T00:09:34.862963+0000","last_undegraded":"2024-04-25T00:09:34.862963+0000","last_fullsized":"2024-04-25T00:09:34.862963+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:40.927785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.169032+0000","last_change":"2024-04-25T00:09:33.877988+0000","last_active":"2024-04-25T00:09:35.169032+0000","last_peered":"2024-04-25T00:09:35.169032+0000","last_clean":"2024-04-25T00:09:35.169032+0000","last_became_active":"2024-04-25T00:09:33.877216+0000","last_became_peered":"2024-04-25T00:09:33.877216+0000","last_unstale":"2024-04-25T00:09:35.169032+0000","last_undegraded":"2024-04-25T00:09:35.169032+0000","last_fullsized":"2024-04-25T00:09:35.169032+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:41.886576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.873730+0000","last_change":"2024-04-25T00:09:33.871557+0000","last_active":"2024-04-25T00:09:33.873730+0000","last_peered":"2024-04-25T00:09:33.873730+0000","last_clean":"2024-04-25T00:09:33.873730+0000","last_became_active":"2024-04-25T00:09:33.871300+0000","last_became_peered":"2024-04-25T00:09:33.871300+0000","last_unstale":"2024-04-25T00:09:33.873730+0000","last_undegraded":"2024-04-25T00:09:33.873730+0000","last_fullsized":"2024-04-25T00:09:33.873730+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:22.733199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.872901+0000","last_change":"2024-04-25T00:09:33.868170+0000","last_active":"2024-04-25T00:09:33.872901+0000","last_peered":"2024-04-25T00:09:33.872901+0000","last_clean":"2024-04-25T00:09:33.872901+0000","last_became_active":"2024-04-25T00:09:33.867382+0000","last_became_peered":"2024-04-25T00:09:33.867382+0000","last_unstale":"2024-04-25T00:09:33.872901+0000","last_undegraded":"2024-04-25T00:09:33.872901+0000","last_fullsized":"2024-04-25T00:09:33.872901+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:04.488545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874365+0000","last_change":"2024-04-25T00:09:33.873865+0000","last_active":"2024-04-25T00:09:34.874365+0000","last_peered":"2024-04-25T00:09:34.874365+0000","last_clean":"2024-04-25T00:09:34.874365+0000","last_became_active":"2024-04-25T00:09:33.873531+0000","last_became_peered":"2024-04-25T00:09:33.873531+0000","last_unstale":"2024-04-25T00:09:34.874365+0000","last_undegraded":"2024-04-25T00:09:34.874365+0000","last_fullsized":"2024-04-25T00:09:34.874365+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:30.035581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874436+0000","last_change":"2024-04-25T00:09:33.867995+0000","last_active":"2024-04-25T00:09:34.874436+0000","last_peered":"2024-04-25T00:09:34.874436+0000","last_clean":"2024-04-25T00:09:34.874436+0000","last_became_active":"2024-04-25T00:09:33.867692+0000","last_became_peered":"2024-04-25T00:09:33.867692+0000","last_unstale":"2024-04-25T00:09:34.874436+0000","last_undegraded":"2024-04-25T00:09:34.874436+0000","last_fullsized":"2024-04-25T00:09:34.874436+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:33.016301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863341+0000","last_change":"2024-04-25T00:09:33.862436+0000","last_active":"2024-04-25T00:09:34.863341+0000","last_peered":"2024-04-25T00:09:34.863341+0000","last_clean":"2024-04-25T00:09:34.863341+0000","last_became_active":"2024-04-25T00:09:33.862096+0000","last_became_peered":"2024-04-25T00:09:33.862096+0000","last_unstale":"2024-04-25T00:09:34.863341+0000","last_undegraded":"2024-04-25T00:09:34.863341+0000","last_fullsized":"2024-04-25T00:09:34.863341+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:25.636303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863725+0000","last_change":"2024-04-25T00:09:33.865358+0000","last_active":"2024-04-25T00:09:34.863725+0000","last_peered":"2024-04-25T00:09:34.863725+0000","last_clean":"2024-04-25T00:09:34.863725+0000","last_became_active":"2024-04-25T00:09:33.865098+0000","last_became_peered":"2024-04-25T00:09:33.865098+0000","last_unstale":"2024-04-25T00:09:34.863725+0000","last_undegraded":"2024-04-25T00:09:34.863725+0000","last_fullsized":"2024-04-25T00:09:34.863725+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.885015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.880076+0000","last_change":"2024-04-25T00:09:33.879644+0000","last_active":"2024-04-25T00:09:33.880076+0000","last_peered":"2024-04-25T00:09:33.880076+0000","last_clean":"2024-04-25T00:09:33.880076+0000","last_became_active":"2024-04-25T00:09:33.879324+0000","last_became_peered":"2024-04-25T00:09:33.879324+0000","last_unstale":"2024-04-25T00:09:33.880076+0000","last_undegraded":"2024-04-25T00:09:33.880076+0000","last_fullsized":"2024-04-25T00:09:33.880076+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:08.003176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864131+0000","last_change":"2024-04-25T00:09:33.870900+0000","last_active":"2024-04-25T00:09:34.864131+0000","last_peered":"2024-04-25T00:09:34.864131+0000","last_clean":"2024-04-25T00:09:34.864131+0000","last_became_active":"2024-04-25T00:09:33.870115+0000","last_became_peered":"2024-04-25T00:09:33.870115+0000","last_unstale":"2024-04-25T00:09:34.864131+0000","last_undegraded":"2024-04-25T00:09:34.864131+0000","last_fullsized":"2024-04-25T00:09:34.864131+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:39.258462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.875694+0000","last_change":"2024-04-25T00:09:33.872660+0000","last_active":"2024-04-25T00:09:33.875694+0000","last_peered":"2024-04-25T00:09:33.875694+0000","last_clean":"2024-04-25T00:09:33.875694+0000","last_became_active":"2024-04-25T00:09:33.872373+0000","last_became_peered":"2024-04-25T00:09:33.872373+0000","last_unstale":"2024-04-25T00:09:33.875694+0000","last_undegraded":"2024-04-25T00:09:33.875694+0000","last_fullsized":"2024-04-25T00:09:33.875694+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:32.403959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.874556+0000","last_change":"2024-04-25T00:09:33.871847+0000","last_active":"2024-04-25T00:09:33.874556+0000","last_peered":"2024-04-25T00:09:33.874556+0000","last_clean":"2024-04-25T00:09:33.874556+0000","last_became_active":"2024-04-25T00:09:33.871112+0000","last_became_peered":"2024-04-25T00:09:33.871112+0000","last_unstale":"2024-04-25T00:09:33.874556+0000","last_undegraded":"2024-04-25T00:09:33.874556+0000","last_fullsized":"2024-04-25T00:09:33.874556+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:30.093690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863227+0000","last_change":"2024-04-25T00:09:33.869405+0000","last_active":"2024-04-25T00:09:34.863227+0000","last_peered":"2024-04-25T00:09:34.863227+0000","last_clean":"2024-04-25T00:09:34.863227+0000","last_became_active":"2024-04-25T00:09:33.869159+0000","last_became_peered":"2024-04-25T00:09:33.869159+0000","last_unstale":"2024-04-25T00:09:34.863227+0000","last_undegraded":"2024-04-25T00:09:34.863227+0000","last_fullsized":"2024-04-25T00:09:34.863227+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:30.035581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871904+0000","last_change":"2024-04-25T00:09:33.868561+0000","last_active":"2024-04-25T00:09:34.871904+0000","last_peered":"2024-04-25T00:09:34.871904+0000","last_clean":"2024-04-25T00:09:34.871904+0000","last_became_active":"2024-04-25T00:09:33.868230+0000","last_became_peered":"2024-04-25T00:09:33.868230+0000","last_unstale":"2024-04-25T00:09:34.871904+0000","last_undegraded":"2024-04-25T00:09:34.871904+0000","last_fullsized":"2024-04-25T00:09:34.871904+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.864780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863389+0000","last_change":"2024-04-25T00:09:33.870026+0000","last_active":"2024-04-25T00:09:34.863389+0000","last_peered":"2024-04-25T00:09:34.863389+0000","last_clean":"2024-04-25T00:09:34.863389+0000","last_became_active":"2024-04-25T00:09:33.869745+0000","last_became_peered":"2024-04-25T00:09:33.869745+0000","last_unstale":"2024-04-25T00:09:34.863389+0000","last_undegraded":"2024-04-25T00:09:34.863389+0000","last_fullsized":"2024-04-25T00:09:34.863389+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:31.093182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863368+0000","last_change":"2024-04-25T00:09:33.871850+0000","last_active":"2024-04-25T00:09:34.863368+0000","last_peered":"2024-04-25T00:09:34.863368+0000","last_clean":"2024-04-25T00:09:34.863368+0000","last_became_active":"2024-04-25T00:09:33.871593+0000","last_became_peered":"2024-04-25T00:09:33.871593+0000","last_unstale":"2024-04-25T00:09:34.863368+0000","last_undegraded":"2024-04-25T00:09:34.863368+0000","last_fullsized":"2024-04-25T00:09:34.863368+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:33.016301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871805+0000","last_change":"2024-04-25T00:09:33.867377+0000","last_active":"2024-04-25T00:09:34.871805+0000","last_peered":"2024-04-25T00:09:34.871805+0000","last_clean":"2024-04-25T00:09:34.871805+0000","last_became_active":"2024-04-25T00:09:33.867154+0000","last_became_peered":"2024-04-25T00:09:33.867154+0000","last_unstale":"2024-04-25T00:09:34.871805+0000","last_undegraded":"2024-04-25T00:09:34.871805+0000","last_fullsized":"2024-04-25T00:09:34.871805+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.616946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168803+0000","last_change":"2024-04-25T00:09:33.871702+0000","last_active":"2024-04-25T00:09:35.168803+0000","last_peered":"2024-04-25T00:09:35.168803+0000","last_clean":"2024-04-25T00:09:35.168803+0000","last_became_active":"2024-04-25T00:09:33.871458+0000","last_became_peered":"2024-04-25T00:09:33.871458+0000","last_unstale":"2024-04-25T00:09:35.168803+0000","last_undegraded":"2024-04-25T00:09:35.168803+0000","last_fullsized":"2024-04-25T00:09:35.168803+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:25.476623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864017+0000","last_change":"2024-04-25T00:09:33.875792+0000","last_active":"2024-04-25T00:09:34.864017+0000","last_peered":"2024-04-25T00:09:34.864017+0000","last_clean":"2024-04-25T00:09:34.864017+0000","last_became_active":"2024-04-25T00:09:33.874998+0000","last_became_peered":"2024-04-25T00:09:33.874998+0000","last_unstale":"2024-04-25T00:09:34.864017+0000","last_undegraded":"2024-04-25T00:09:34.864017+0000","last_fullsized":"2024-04-25T00:09:34.864017+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.956814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874361+0000","last_change":"2024-04-25T00:09:33.875621+0000","last_active":"2024-04-25T00:09:34.874361+0000","last_peered":"2024-04-25T00:09:34.874361+0000","last_clean":"2024-04-25T00:09:34.874361+0000","last_became_active":"2024-04-25T00:09:33.875162+0000","last_became_peered":"2024-04-25T00:09:33.875162+0000","last_unstale":"2024-04-25T00:09:34.874361+0000","last_undegraded":"2024-04-25T00:09:34.874361+0000","last_fullsized":"2024-04-25T00:09:34.874361+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:00.144856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879430+0000","last_change":"2024-04-25T00:09:33.878105+0000","last_active":"2024-04-25T00:09:33.879430+0000","last_peered":"2024-04-25T00:09:33.879430+0000","last_clean":"2024-04-25T00:09:33.879430+0000","last_became_active":"2024-04-25T00:09:33.877837+0000","last_became_peered":"2024-04-25T00:09:33.877837+0000","last_unstale":"2024-04-25T00:09:33.879430+0000","last_undegraded":"2024-04-25T00:09:33.879430+0000","last_fullsized":"2024-04-25T00:09:33.879430+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:26.774142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874230+0000","last_change":"2024-04-25T00:09:33.875264+0000","last_active":"2024-04-25T00:09:34.874230+0000","last_peered":"2024-04-25T00:09:34.874230+0000","last_clean":"2024-04-25T00:09:34.874230+0000","last_became_active":"2024-04-25T00:09:33.874940+0000","last_became_peered":"2024-04-25T00:09:33.874940+0000","last_unstale":"2024-04-25T00:09:34.874230+0000","last_undegraded":"2024-04-25T00:09:34.874230+0000","last_fullsized":"2024-04-25T00:09:34.874230+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:07.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168658+0000","last_change":"2024-04-25T00:09:33.870203+0000","last_active":"2024-04-25T00:09:35.168658+0000","last_peered":"2024-04-25T00:09:35.168658+0000","last_clean":"2024-04-25T00:09:35.168658+0000","last_became_active":"2024-04-25T00:09:33.869779+0000","last_became_peered":"2024-04-25T00:09:33.869779+0000","last_unstale":"2024-04-25T00:09:35.168658+0000","last_undegraded":"2024-04-25T00:09:35.168658+0000","last_fullsized":"2024-04-25T00:09:35.168658+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:18.123619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.870793+0000","last_change":"2024-04-25T00:09:33.868363+0000","last_active":"2024-04-25T00:09:33.870793+0000","last_peered":"2024-04-25T00:09:33.870793+0000","last_clean":"2024-04-25T00:09:33.870793+0000","last_became_active":"2024-04-25T00:09:33.868107+0000","last_became_peered":"2024-04-25T00:09:33.868107+0000","last_unstale":"2024-04-25T00:09:33.870793+0000","last_undegraded":"2024-04-25T00:09:33.870793+0000","last_fullsized":"2024-04-25T00:09:33.870793+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:08.619397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862961+0000","last_change":"2024-04-25T00:09:33.864477+0000","last_active":"2024-04-25T00:09:34.862961+0000","last_peered":"2024-04-25T00:09:34.862961+0000","last_clean":"2024-04-25T00:09:34.862961+0000","last_became_active":"2024-04-25T00:09:33.864169+0000","last_became_peered":"2024-04-25T00:09:33.864169+0000","last_unstale":"2024-04-25T00:09:34.862961+0000","last_undegraded":"2024-04-25T00:09:34.862961+0000","last_fullsized":"2024-04-25T00:09:34.862961+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.332387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168766+0000","last_change":"2024-04-25T00:09:33.874753+0000","last_active":"2024-04-25T00:09:35.168766+0000","last_peered":"2024-04-25T00:09:35.168766+0000","last_clean":"2024-04-25T00:09:35.168766+0000","last_became_active":"2024-04-25T00:09:33.874394+0000","last_became_peered":"2024-04-25T00:09:33.874394+0000","last_unstale":"2024-04-25T00:09:35.168766+0000","last_undegraded":"2024-04-25T00:09:35.168766+0000","last_fullsized":"2024-04-25T00:09:35.168766+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:28.647123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874479+0000","last_change":"2024-04-25T00:09:32.863939+0000","last_active":"2024-04-25T00:09:34.874479+0000","last_peered":"2024-04-25T00:09:34.874479+0000","last_clean":"2024-04-25T00:09:34.874479+0000","last_became_active":"2024-04-25T00:09:32.863651+0000","last_became_peered":"2024-04-25T00:09:32.863651+0000","last_unstale":"2024-04-25T00:09:34.874479+0000","last_undegraded":"2024-04-25T00:09:34.874479+0000","last_fullsized":"2024-04-25T00:09:34.874479+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:09.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863556+0000","last_change":"2024-04-25T00:09:33.867201+0000","last_active":"2024-04-25T00:09:34.863556+0000","last_peered":"2024-04-25T00:09:34.863556+0000","last_clean":"2024-04-25T00:09:34.863556+0000","last_became_active":"2024-04-25T00:09:33.866911+0000","last_became_peered":"2024-04-25T00:09:33.866911+0000","last_unstale":"2024-04-25T00:09:34.863556+0000","last_undegraded":"2024-04-25T00:09:34.863556+0000","last_fullsized":"2024-04-25T00:09:34.863556+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:54.718780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168907+0000","last_change":"2024-04-25T00:09:32.868306+0000","last_active":"2024-04-25T00:09:35.168907+0000","last_peered":"2024-04-25T00:09:35.168907+0000","last_clean":"2024-04-25T00:09:35.168907+0000","last_became_active":"2024-04-25T00:09:32.867847+0000","last_became_peered":"2024-04-25T00:09:32.867847+0000","last_unstale":"2024-04-25T00:09:35.168907+0000","last_undegraded":"2024-04-25T00:09:35.168907+0000","last_fullsized":"2024-04-25T00:09:35.168907+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:23.760066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863575+0000","last_change":"2024-04-25T00:09:33.874758+0000","last_active":"2024-04-25T00:09:34.863575+0000","last_peered":"2024-04-25T00:09:34.863575+0000","last_clean":"2024-04-25T00:09:34.863575+0000","last_became_active":"2024-04-25T00:09:33.873840+0000","last_became_peered":"2024-04-25T00:09:33.873840+0000","last_unstale":"2024-04-25T00:09:34.863575+0000","last_undegraded":"2024-04-25T00:09:34.863575+0000","last_fullsized":"2024-04-25T00:09:34.863575+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:37.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.882488+0000","last_change":"2024-04-25T00:09:32.866298+0000","last_active":"2024-04-25T00:09:33.882488+0000","last_peered":"2024-04-25T00:09:33.882488+0000","last_clean":"2024-04-25T00:09:33.882488+0000","last_became_active":"2024-04-25T00:09:32.866011+0000","last_became_peered":"2024-04-25T00:09:32.866011+0000","last_unstale":"2024-04-25T00:09:33.882488+0000","last_undegraded":"2024-04-25T00:09:33.882488+0000","last_fullsized":"2024-04-25T00:09:33.882488+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:41.242765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879218+0000","last_change":"2024-04-25T00:09:33.878077+0000","last_active":"2024-04-25T00:09:33.879218+0000","last_peered":"2024-04-25T00:09:33.879218+0000","last_clean":"2024-04-25T00:09:33.879218+0000","last_became_active":"2024-04-25T00:09:33.877808+0000","last_became_peered":"2024-04-25T00:09:33.877808+0000","last_unstale":"2024-04-25T00:09:33.879218+0000","last_undegraded":"2024-04-25T00:09:33.879218+0000","last_fullsized":"2024-04-25T00:09:33.879218+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:45.839404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872127+0000","last_change":"2024-04-25T00:09:32.870037+0000","last_active":"2024-04-25T00:09:34.872127+0000","last_peered":"2024-04-25T00:09:34.872127+0000","last_clean":"2024-04-25T00:09:34.872127+0000","last_became_active":"2024-04-25T00:09:32.869737+0000","last_became_peered":"2024-04-25T00:09:32.869737+0000","last_unstale":"2024-04-25T00:09:34.872127+0000","last_undegraded":"2024-04-25T00:09:34.872127+0000","last_fullsized":"2024-04-25T00:09:34.872127+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:16.039533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871893+0000","last_change":"2024-04-25T00:09:33.869903+0000","last_active":"2024-04-25T00:09:34.871893+0000","last_peered":"2024-04-25T00:09:34.871893+0000","last_clean":"2024-04-25T00:09:34.871893+0000","last_became_active":"2024-04-25T00:09:33.868905+0000","last_became_peered":"2024-04-25T00:09:33.868905+0000","last_unstale":"2024-04-25T00:09:34.871893+0000","last_undegraded":"2024-04-25T00:09:34.871893+0000","last_fullsized":"2024-04-25T00:09:34.871893+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:21.795335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863378+0000","last_change":"2024-04-25T00:09:32.868485+0000","last_active":"2024-04-25T00:09:34.863378+0000","last_peered":"2024-04-25T00:09:34.863378+0000","last_clean":"2024-04-25T00:09:34.863378+0000","last_became_active":"2024-04-25T00:09:32.868229+0000","last_became_peered":"2024-04-25T00:09:32.868229+0000","last_unstale":"2024-04-25T00:09:34.863378+0000","last_undegraded":"2024-04-25T00:09:34.863378+0000","last_fullsized":"2024-04-25T00:09:34.863378+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:30.649758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863102+0000","last_change":"2024-04-25T00:09:33.866550+0000","last_active":"2024-04-25T00:09:34.863102+0000","last_peered":"2024-04-25T00:09:34.863102+0000","last_clean":"2024-04-25T00:09:34.863102+0000","last_became_active":"2024-04-25T00:09:33.866017+0000","last_became_peered":"2024-04-25T00:09:33.866017+0000","last_unstale":"2024-04-25T00:09:34.863102+0000","last_undegraded":"2024-04-25T00:09:34.863102+0000","last_fullsized":"2024-04-25T00:09:34.863102+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:07.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862888+0000","last_change":"2024-04-25T00:09:32.867526+0000","last_active":"2024-04-25T00:09:34.862888+0000","last_peered":"2024-04-25T00:09:34.862888+0000","last_clean":"2024-04-25T00:09:34.862888+0000","last_became_active":"2024-04-25T00:09:32.867237+0000","last_became_peered":"2024-04-25T00:09:32.867237+0000","last_unstale":"2024-04-25T00:09:34.862888+0000","last_undegraded":"2024-04-25T00:09:34.862888+0000","last_fullsized":"2024-04-25T00:09:34.862888+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:53.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871902+0000","last_change":"2024-04-25T00:09:33.868169+0000","last_active":"2024-04-25T00:09:34.871902+0000","last_peered":"2024-04-25T00:09:34.871902+0000","last_clean":"2024-04-25T00:09:34.871902+0000","last_became_active":"2024-04-25T00:09:33.867912+0000","last_became_peered":"2024-04-25T00:09:33.867912+0000","last_unstale":"2024-04-25T00:09:34.871902+0000","last_undegraded":"2024-04-25T00:09:34.871902+0000","last_fullsized":"2024-04-25T00:09:34.871902+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:22.186105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863088+0000","last_change":"2024-04-25T00:09:32.862149+0000","last_active":"2024-04-25T00:09:34.863088+0000","last_peered":"2024-04-25T00:09:34.863088+0000","last_clean":"2024-04-25T00:09:34.863088+0000","last_became_active":"2024-04-25T00:09:32.861803+0000","last_became_peered":"2024-04-25T00:09:32.861803+0000","last_unstale":"2024-04-25T00:09:34.863088+0000","last_undegraded":"2024-04-25T00:09:34.863088+0000","last_fullsized":"2024-04-25T00:09:34.863088+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:28.007292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872261+0000","last_change":"2024-04-25T00:09:33.881209+0000","last_active":"2024-04-25T00:09:34.872261+0000","last_peered":"2024-04-25T00:09:34.872261+0000","last_clean":"2024-04-25T00:09:34.872261+0000","last_became_active":"2024-04-25T00:09:33.880746+0000","last_became_peered":"2024-04-25T00:09:33.880746+0000","last_unstale":"2024-04-25T00:09:34.872261+0000","last_undegraded":"2024-04-25T00:09:34.872261+0000","last_fullsized":"2024-04-25T00:09:34.872261+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:17.623421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863158+0000","last_change":"2024-04-25T00:09:32.867518+0000","last_active":"2024-04-25T00:09:34.863158+0000","last_peered":"2024-04-25T00:09:34.863158+0000","last_clean":"2024-04-25T00:09:34.863158+0000","last_became_active":"2024-04-25T00:09:32.867185+0000","last_became_peered":"2024-04-25T00:09:32.867185+0000","last_unstale":"2024-04-25T00:09:34.863158+0000","last_undegraded":"2024-04-25T00:09:34.863158+0000","last_fullsized":"2024-04-25T00:09:34.863158+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:17.118768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.872577+0000","last_change":"2024-04-25T00:09:33.869306+0000","last_active":"2024-04-25T00:09:33.872577+0000","last_peered":"2024-04-25T00:09:33.872577+0000","last_clean":"2024-04-25T00:09:33.872577+0000","last_became_active":"2024-04-25T00:09:33.868632+0000","last_became_peered":"2024-04-25T00:09:33.868632+0000","last_unstale":"2024-04-25T00:09:33.872577+0000","last_undegraded":"2024-04-25T00:09:33.872577+0000","last_fullsized":"2024-04-25T00:09:33.872577+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:00.532181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863652+0000","last_change":"2024-04-25T00:09:32.864351+0000","last_active":"2024-04-25T00:09:34.863652+0000","last_peered":"2024-04-25T00:09:34.863652+0000","last_clean":"2024-04-25T00:09:34.863652+0000","last_became_active":"2024-04-25T00:09:32.864085+0000","last_became_peered":"2024-04-25T00:09:32.864085+0000","last_unstale":"2024-04-25T00:09:34.863652+0000","last_undegraded":"2024-04-25T00:09:34.863652+0000","last_fullsized":"2024-04-25T00:09:34.863652+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:57.123376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872043+0000","last_change":"2024-04-25T00:09:33.881867+0000","last_active":"2024-04-25T00:09:34.872043+0000","last_peered":"2024-04-25T00:09:34.872043+0000","last_clean":"2024-04-25T00:09:34.872043+0000","last_became_active":"2024-04-25T00:09:33.881578+0000","last_became_peered":"2024-04-25T00:09:33.881578+0000","last_unstale":"2024-04-25T00:09:34.872043+0000","last_undegraded":"2024-04-25T00:09:34.872043+0000","last_fullsized":"2024-04-25T00:09:34.872043+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:44.059497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"181'3","reported_seq":15,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.882347+0000","last_change":"2024-04-25T00:09:32.866762+0000","last_active":"2024-04-25T00:09:33.882347+0000","last_peered":"2024-04-25T00:09:33.882347+0000","last_clean":"2024-04-25T00:09:33.882347+0000","last_became_active":"2024-04-25T00:09:32.866452+0000","last_became_peered":"2024-04-25T00:09:32.866452+0000","last_unstale":"2024-04-25T00:09:33.882347+0000","last_undegraded":"2024-04-25T00:09:33.882347+0000","last_fullsized":"2024-04-25T00:09:33.882347+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:49.965548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863728+0000","last_change":"2024-04-25T00:09:33.867471+0000","last_active":"2024-04-25T00:09:34.863728+0000","last_peered":"2024-04-25T00:09:34.863728+0000","last_clean":"2024-04-25T00:09:34.863728+0000","last_became_active":"2024-04-25T00:09:33.867207+0000","last_became_peered":"2024-04-25T00:09:33.867207+0000","last_unstale":"2024-04-25T00:09:34.863728+0000","last_undegraded":"2024-04-25T00:09:34.863728+0000","last_fullsized":"2024-04-25T00:09:34.863728+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:58.832182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":20,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872192+0000","last_change":"2024-04-25T00:09:32.869149+0000","last_active":"2024-04-25T00:09:34.872192+0000","last_peered":"2024-04-25T00:09:34.872192+0000","last_clean":"2024-04-25T00:09:34.872192+0000","last_became_active":"2024-04-25T00:09:32.868846+0000","last_became_peered":"2024-04-25T00:09:32.868846+0000","last_unstale":"2024-04-25T00:09:34.872192+0000","last_undegraded":"2024-04-25T00:09:34.872192+0000","last_fullsized":"2024-04-25T00:09:34.872192+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:18.348472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863360+0000","last_change":"2024-04-25T00:09:33.870433+0000","last_active":"2024-04-25T00:09:34.863360+0000","last_peered":"2024-04-25T00:09:34.863360+0000","last_clean":"2024-04-25T00:09:34.863360+0000","last_became_active":"2024-04-25T00:09:33.869736+0000","last_became_peered":"2024-04-25T00:09:33.869736+0000","last_unstale":"2024-04-25T00:09:34.863360+0000","last_undegraded":"2024-04-25T00:09:34.863360+0000","last_fullsized":"2024-04-25T00:09:34.863360+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:59.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863562+0000","last_change":"2024-04-25T00:09:32.870900+0000","last_active":"2024-04-25T00:09:34.863562+0000","last_peered":"2024-04-25T00:09:34.863562+0000","last_clean":"2024-04-25T00:09:34.863562+0000","last_became_active":"2024-04-25T00:09:32.870635+0000","last_became_peered":"2024-04-25T00:09:32.870635+0000","last_unstale":"2024-04-25T00:09:34.863562+0000","last_undegraded":"2024-04-25T00:09:34.863562+0000","last_fullsized":"2024-04-25T00:09:34.863562+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:51.613038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.876184+0000","last_change":"2024-04-25T00:09:33.874352+0000","last_active":"2024-04-25T00:09:33.876184+0000","last_peered":"2024-04-25T00:09:33.876184+0000","last_clean":"2024-04-25T00:09:33.876184+0000","last_became_active":"2024-04-25T00:09:33.874086+0000","last_became_peered":"2024-04-25T00:09:33.874086+0000","last_unstale":"2024-04-25T00:09:33.876184+0000","last_undegraded":"2024-04-25T00:09:33.876184+0000","last_fullsized":"2024-04-25T00:09:33.876184+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:21.542440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863285+0000","last_change":"2024-04-25T00:09:32.860288+0000","last_active":"2024-04-25T00:09:34.863285+0000","last_peered":"2024-04-25T00:09:34.863285+0000","last_clean":"2024-04-25T00:09:34.863285+0000","last_became_active":"2024-04-25T00:09:32.860010+0000","last_became_peered":"2024-04-25T00:09:32.860010+0000","last_unstale":"2024-04-25T00:09:34.863285+0000","last_undegraded":"2024-04-25T00:09:34.863285+0000","last_fullsized":"2024-04-25T00:09:34.863285+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:40.355571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.874431+0000","last_change":"2024-04-25T00:09:33.867592+0000","last_active":"2024-04-25T00:09:33.874431+0000","last_peered":"2024-04-25T00:09:33.874431+0000","last_clean":"2024-04-25T00:09:33.874431+0000","last_became_active":"2024-04-25T00:09:33.867328+0000","last_became_peered":"2024-04-25T00:09:33.867328+0000","last_unstale":"2024-04-25T00:09:33.874431+0000","last_undegraded":"2024-04-25T00:09:33.874431+0000","last_fullsized":"2024-04-25T00:09:33.874431+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:39:35.826108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.860512+0000","last_change":"2024-04-25T00:09:32.862727+0000","last_active":"2024-04-25T00:09:33.860512+0000","last_peered":"2024-04-25T00:09:33.860512+0000","last_clean":"2024-04-25T00:09:33.860512+0000","last_became_active":"2024-04-25T00:09:32.862467+0000","last_became_peered":"2024-04-25T00:09:32.862467+0000","last_unstale":"2024-04-25T00:09:33.860512+0000","last_undegraded":"2024-04-25T00:09:33.860512+0000","last_fullsized":"2024-04-25T00:09:33.860512+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:20.083614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863840+0000","last_change":"2024-04-25T00:09:33.875462+0000","last_active":"2024-04-25T00:09:34.863840+0000","last_peered":"2024-04-25T00:09:34.863840+0000","last_clean":"2024-04-25T00:09:34.863840+0000","last_became_active":"2024-04-25T00:09:33.875199+0000","last_became_peered":"2024-04-25T00:09:33.875199+0000","last_unstale":"2024-04-25T00:09:34.863840+0000","last_undegraded":"2024-04-25T00:09:34.863840+0000","last_fullsized":"2024-04-25T00:09:34.863840+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:55.059466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"181'8","reported_seq":20,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.883499+0000","last_change":"2024-04-25T00:09:32.865419+0000","last_active":"2024-04-25T00:09:33.883499+0000","last_peered":"2024-04-25T00:09:33.883499+0000","last_clean":"2024-04-25T00:09:33.883499+0000","last_became_active":"2024-04-25T00:09:32.865121+0000","last_became_peered":"2024-04-25T00:09:32.865121+0000","last_unstale":"2024-04-25T00:09:33.883499+0000","last_undegraded":"2024-04-25T00:09:33.883499+0000","last_fullsized":"2024-04-25T00:09:33.883499+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:20.150413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168820+0000","last_change":"2024-04-25T00:09:33.879136+0000","last_active":"2024-04-25T00:09:35.168820+0000","last_peered":"2024-04-25T00:09:35.168820+0000","last_clean":"2024-04-25T00:09:35.168820+0000","last_became_active":"2024-04-25T00:09:33.876729+0000","last_became_peered":"2024-04-25T00:09:33.876729+0000","last_unstale":"2024-04-25T00:09:35.168820+0000","last_undegraded":"2024-04-25T00:09:35.168820+0000","last_fullsized":"2024-04-25T00:09:35.168820+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:10.192491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863251+0000","last_change":"2024-04-25T00:09:32.872419+0000","last_active":"2024-04-25T00:09:34.863251+0000","last_peered":"2024-04-25T00:09:34.863251+0000","last_clean":"2024-04-25T00:09:34.863251+0000","last_became_active":"2024-04-25T00:09:32.872151+0000","last_became_peered":"2024-04-25T00:09:32.872151+0000","last_unstale":"2024-04-25T00:09:34.863251+0000","last_undegraded":"2024-04-25T00:09:34.863251+0000","last_fullsized":"2024-04-25T00:09:34.863251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:45.275982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168889+0000","last_change":"2024-04-25T00:09:33.871091+0000","last_active":"2024-04-25T00:09:35.168889+0000","last_peered":"2024-04-25T00:09:35.168889+0000","last_clean":"2024-04-25T00:09:35.168889+0000","last_became_active":"2024-04-25T00:09:33.870803+0000","last_became_peered":"2024-04-25T00:09:33.870803+0000","last_unstale":"2024-04-25T00:09:35.168889+0000","last_undegraded":"2024-04-25T00:09:35.168889+0000","last_fullsized":"2024-04-25T00:09:35.168889+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:12.471194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168966+0000","last_change":"2024-04-25T00:09:32.868130+0000","last_active":"2024-04-25T00:09:35.168966+0000","last_peered":"2024-04-25T00:09:35.168966+0000","last_clean":"2024-04-25T00:09:35.168966+0000","last_became_active":"2024-04-25T00:09:32.867778+0000","last_became_peered":"2024-04-25T00:09:32.867778+0000","last_unstale":"2024-04-25T00:09:35.168966+0000","last_undegraded":"2024-04-25T00:09:35.168966+0000","last_fullsized":"2024-04-25T00:09:35.168966+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:31.168558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.876812+0000","last_change":"2024-04-25T00:09:33.874649+0000","last_active":"2024-04-25T00:09:33.876812+0000","last_peered":"2024-04-25T00:09:33.876812+0000","last_clean":"2024-04-25T00:09:33.876812+0000","last_became_active":"2024-04-25T00:09:33.874370+0000","last_became_peered":"2024-04-25T00:09:33.874370+0000","last_unstale":"2024-04-25T00:09:33.876812+0000","last_undegraded":"2024-04-25T00:09:33.876812+0000","last_fullsized":"2024-04-25T00:09:33.876812+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:54.547827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.883237+0000","last_change":"2024-04-25T00:09:32.865123+0000","last_active":"2024-04-25T00:09:33.883237+0000","last_peered":"2024-04-25T00:09:33.883237+0000","last_clean":"2024-04-25T00:09:33.883237+0000","last_became_active":"2024-04-25T00:09:32.864038+0000","last_became_peered":"2024-04-25T00:09:32.864038+0000","last_unstale":"2024-04-25T00:09:33.883237+0000","last_undegraded":"2024-04-25T00:09:33.883237+0000","last_fullsized":"2024-04-25T00:09:33.883237+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:26.788441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.876230+0000","last_change":"2024-04-25T00:09:33.874638+0000","last_active":"2024-04-25T00:09:33.876230+0000","last_peered":"2024-04-25T00:09:33.876230+0000","last_clean":"2024-04-25T00:09:33.876230+0000","last_became_active":"2024-04-25T00:09:33.874363+0000","last_became_peered":"2024-04-25T00:09:33.874363+0000","last_unstale":"2024-04-25T00:09:33.876230+0000","last_undegraded":"2024-04-25T00:09:33.876230+0000","last_fullsized":"2024-04-25T00:09:33.876230+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:15.371941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.861988+0000","last_change":"2024-04-25T00:09:32.871043+0000","last_active":"2024-04-25T00:09:33.861988+0000","last_peered":"2024-04-25T00:09:33.861988+0000","last_clean":"2024-04-25T00:09:33.861988+0000","last_became_active":"2024-04-25T00:09:32.870708+0000","last_became_peered":"2024-04-25T00:09:32.870708+0000","last_unstale":"2024-04-25T00:09:33.861988+0000","last_undegraded":"2024-04-25T00:09:33.861988+0000","last_fullsized":"2024-04-25T00:09:33.861988+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:37:50.624952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.879109+0000","last_change":"2024-04-25T00:09:33.878329+0000","last_active":"2024-04-25T00:09:33.879109+0000","last_peered":"2024-04-25T00:09:33.879109+0000","last_clean":"2024-04-25T00:09:33.879109+0000","last_became_active":"2024-04-25T00:09:33.878062+0000","last_became_peered":"2024-04-25T00:09:33.878062+0000","last_unstale":"2024-04-25T00:09:33.879109+0000","last_undegraded":"2024-04-25T00:09:33.879109+0000","last_fullsized":"2024-04-25T00:09:33.879109+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:17.344174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.882003+0000","last_change":"2024-04-25T00:09:32.865066+0000","last_active":"2024-04-25T00:09:33.882003+0000","last_peered":"2024-04-25T00:09:33.882003+0000","last_clean":"2024-04-25T00:09:33.882003+0000","last_became_active":"2024-04-25T00:09:32.864793+0000","last_became_peered":"2024-04-25T00:09:32.864793+0000","last_unstale":"2024-04-25T00:09:33.882003+0000","last_undegraded":"2024-04-25T00:09:33.882003+0000","last_fullsized":"2024-04-25T00:09:33.882003+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:49.165483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.878799+0000","last_change":"2024-04-25T00:09:33.877986+0000","last_active":"2024-04-25T00:09:33.878799+0000","last_peered":"2024-04-25T00:09:33.878799+0000","last_clean":"2024-04-25T00:09:33.878799+0000","last_became_active":"2024-04-25T00:09:33.877728+0000","last_became_peered":"2024-04-25T00:09:33.877728+0000","last_unstale":"2024-04-25T00:09:33.878799+0000","last_undegraded":"2024-04-25T00:09:33.878799+0000","last_fullsized":"2024-04-25T00:09:33.878799+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.356214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863732+0000","last_change":"2024-04-25T00:09:32.870566+0000","last_active":"2024-04-25T00:09:34.863732+0000","last_peered":"2024-04-25T00:09:34.863732+0000","last_clean":"2024-04-25T00:09:34.863732+0000","last_became_active":"2024-04-25T00:09:32.869700+0000","last_became_peered":"2024-04-25T00:09:32.869700+0000","last_unstale":"2024-04-25T00:09:34.863732+0000","last_undegraded":"2024-04-25T00:09:34.863732+0000","last_fullsized":"2024-04-25T00:09:34.863732+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:16.007479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168567+0000","last_change":"2024-04-25T00:09:33.878569+0000","last_active":"2024-04-25T00:09:35.168567+0000","last_peered":"2024-04-25T00:09:35.168567+0000","last_clean":"2024-04-25T00:09:35.168567+0000","last_became_active":"2024-04-25T00:09:33.878305+0000","last_became_peered":"2024-04-25T00:09:33.878305+0000","last_unstale":"2024-04-25T00:09:35.168567+0000","last_undegraded":"2024-04-25T00:09:35.168567+0000","last_fullsized":"2024-04-25T00:09:35.168567+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:37.456036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863824+0000","last_change":"2024-04-25T00:09:32.861027+0000","last_active":"2024-04-25T00:09:34.863824+0000","last_peered":"2024-04-25T00:09:34.863824+0000","last_clean":"2024-04-25T00:09:34.863824+0000","last_became_active":"2024-04-25T00:09:32.860724+0000","last_became_peered":"2024-04-25T00:09:32.860724+0000","last_unstale":"2024-04-25T00:09:34.863824+0000","last_undegraded":"2024-04-25T00:09:34.863824+0000","last_fullsized":"2024-04-25T00:09:34.863824+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:23.820088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863453+0000","last_change":"2024-04-25T00:09:33.866398+0000","last_active":"2024-04-25T00:09:34.863453+0000","last_peered":"2024-04-25T00:09:34.863453+0000","last_clean":"2024-04-25T00:09:34.863453+0000","last_became_active":"2024-04-25T00:09:33.866122+0000","last_became_peered":"2024-04-25T00:09:33.866122+0000","last_unstale":"2024-04-25T00:09:34.863453+0000","last_undegraded":"2024-04-25T00:09:34.863453+0000","last_fullsized":"2024-04-25T00:09:34.863453+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:27.159690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863217+0000","last_change":"2024-04-25T00:09:32.872369+0000","last_active":"2024-04-25T00:09:34.863217+0000","last_peered":"2024-04-25T00:09:34.863217+0000","last_clean":"2024-04-25T00:09:34.863217+0000","last_became_active":"2024-04-25T00:09:32.872083+0000","last_became_peered":"2024-04-25T00:09:32.872083+0000","last_unstale":"2024-04-25T00:09:34.863217+0000","last_undegraded":"2024-04-25T00:09:34.863217+0000","last_fullsized":"2024-04-25T00:09:34.863217+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:36.451185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874414+0000","last_change":"2024-04-25T00:09:33.874713+0000","last_active":"2024-04-25T00:09:34.874414+0000","last_peered":"2024-04-25T00:09:34.874414+0000","last_clean":"2024-04-25T00:09:34.874414+0000","last_became_active":"2024-04-25T00:09:33.874450+0000","last_became_peered":"2024-04-25T00:09:33.874450+0000","last_unstale":"2024-04-25T00:09:34.874414+0000","last_undegraded":"2024-04-25T00:09:34.874414+0000","last_fullsized":"2024-04-25T00:09:34.874414+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:59.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.883754+0000","last_change":"2024-04-25T00:09:32.864284+0000","last_active":"2024-04-25T00:09:33.883754+0000","last_peered":"2024-04-25T00:09:33.883754+0000","last_clean":"2024-04-25T00:09:33.883754+0000","last_became_active":"2024-04-25T00:09:32.863870+0000","last_became_peered":"2024-04-25T00:09:32.863870+0000","last_unstale":"2024-04-25T00:09:33.883754+0000","last_undegraded":"2024-04-25T00:09:33.883754+0000","last_fullsized":"2024-04-25T00:09:33.883754+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:23.760066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864187+0000","last_change":"2024-04-25T00:09:33.868631+0000","last_active":"2024-04-25T00:09:34.864187+0000","last_peered":"2024-04-25T00:09:34.864187+0000","last_clean":"2024-04-25T00:09:34.864187+0000","last_became_active":"2024-04-25T00:09:33.866936+0000","last_became_peered":"2024-04-25T00:09:33.866936+0000","last_unstale":"2024-04-25T00:09:34.864187+0000","last_undegraded":"2024-04-25T00:09:34.864187+0000","last_fullsized":"2024-04-25T00:09:34.864187+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:45.734090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"182'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.873583+0000","last_change":"2024-04-25T00:09:32.869135+0000","last_active":"2024-04-25T00:09:34.873583+0000","last_peered":"2024-04-25T00:09:34.873583+0000","last_clean":"2024-04-25T00:09:34.873583+0000","last_became_active":"2024-04-25T00:09:32.868833+0000","last_became_peered":"2024-04-25T00:09:32.868833+0000","last_unstale":"2024-04-25T00:09:34.873583+0000","last_undegraded":"2024-04-25T00:09:34.873583+0000","last_fullsized":"2024-04-25T00:09:34.873583+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:22.181677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168623+0000","last_change":"2024-04-25T00:09:33.869214+0000","last_active":"2024-04-25T00:09:35.168623+0000","last_peered":"2024-04-25T00:09:35.168623+0000","last_clean":"2024-04-25T00:09:35.168623+0000","last_became_active":"2024-04-25T00:09:33.868863+0000","last_became_peered":"2024-04-25T00:09:33.868863+0000","last_unstale":"2024-04-25T00:09:35.168623+0000","last_undegraded":"2024-04-25T00:09:35.168623+0000","last_fullsized":"2024-04-25T00:09:35.168623+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:31.654609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863433+0000","last_change":"2024-04-25T00:09:32.868078+0000","last_active":"2024-04-25T00:09:34.863433+0000","last_peered":"2024-04-25T00:09:34.863433+0000","last_clean":"2024-04-25T00:09:34.863433+0000","last_became_active":"2024-04-25T00:09:32.867779+0000","last_became_peered":"2024-04-25T00:09:32.867779+0000","last_unstale":"2024-04-25T00:09:34.863433+0000","last_undegraded":"2024-04-25T00:09:34.863433+0000","last_fullsized":"2024-04-25T00:09:34.863433+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:27.642272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863700+0000","last_change":"2024-04-25T00:09:33.871883+0000","last_active":"2024-04-25T00:09:34.863700+0000","last_peered":"2024-04-25T00:09:34.863700+0000","last_clean":"2024-04-25T00:09:34.863700+0000","last_became_active":"2024-04-25T00:09:33.871586+0000","last_became_peered":"2024-04-25T00:09:33.871586+0000","last_unstale":"2024-04-25T00:09:34.863700+0000","last_undegraded":"2024-04-25T00:09:34.863700+0000","last_fullsized":"2024-04-25T00:09:34.863700+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:08.604008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.862153+0000","last_change":"2024-04-25T00:09:32.863484+0000","last_active":"2024-04-25T00:09:33.862153+0000","last_peered":"2024-04-25T00:09:33.862153+0000","last_clean":"2024-04-25T00:09:33.862153+0000","last_became_active":"2024-04-25T00:09:32.863226+0000","last_became_peered":"2024-04-25T00:09:32.863226+0000","last_unstale":"2024-04-25T00:09:33.862153+0000","last_undegraded":"2024-04-25T00:09:33.862153+0000","last_fullsized":"2024-04-25T00:09:33.862153+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.948313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863681+0000","last_change":"2024-04-25T00:09:33.876228+0000","last_active":"2024-04-25T00:09:34.863681+0000","last_peered":"2024-04-25T00:09:34.863681+0000","last_clean":"2024-04-25T00:09:34.863681+0000","last_became_active":"2024-04-25T00:09:33.875900+0000","last_became_peered":"2024-04-25T00:09:33.875900+0000","last_unstale":"2024-04-25T00:09:34.863681+0000","last_undegraded":"2024-04-25T00:09:34.863681+0000","last_fullsized":"2024-04-25T00:09:34.863681+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.901658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872093+0000","last_change":"2024-04-25T00:09:32.870945+0000","last_active":"2024-04-25T00:09:34.872093+0000","last_peered":"2024-04-25T00:09:34.872093+0000","last_clean":"2024-04-25T00:09:34.872093+0000","last_became_active":"2024-04-25T00:09:32.870577+0000","last_became_peered":"2024-04-25T00:09:32.870577+0000","last_unstale":"2024-04-25T00:09:34.872093+0000","last_undegraded":"2024-04-25T00:09:34.872093+0000","last_fullsized":"2024-04-25T00:09:34.872093+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:55.965150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874401+0000","last_change":"2024-04-25T00:09:33.872463+0000","last_active":"2024-04-25T00:09:34.874401+0000","last_peered":"2024-04-25T00:09:34.874401+0000","last_clean":"2024-04-25T00:09:34.874401+0000","last_became_active":"2024-04-25T00:09:33.872181+0000","last_became_peered":"2024-04-25T00:09:33.872181+0000","last_unstale":"2024-04-25T00:09:34.874401+0000","last_undegraded":"2024-04-25T00:09:34.874401+0000","last_fullsized":"2024-04-25T00:09:34.874401+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:59.450004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863450+0000","last_change":"2024-04-25T00:09:32.872901+0000","last_active":"2024-04-25T00:09:34.863450+0000","last_peered":"2024-04-25T00:09:34.863450+0000","last_clean":"2024-04-25T00:09:34.863450+0000","last_became_active":"2024-04-25T00:09:32.872596+0000","last_became_peered":"2024-04-25T00:09:32.872596+0000","last_unstale":"2024-04-25T00:09:34.863450+0000","last_undegraded":"2024-04-25T00:09:34.863450+0000","last_fullsized":"2024-04-25T00:09:34.863450+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:24.471772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T00:09:33.867045+0000","last_change":"2024-04-25T00:09:33.864759+0000","last_active":"2024-04-25T00:09:33.867045+0000","last_peered":"2024-04-25T00:09:33.867045+0000","last_clean":"2024-04-25T00:09:33.867045+0000","last_became_active":"2024-04-25T00:09:33.864497+0000","last_became_peered":"2024-04-25T00:09:33.864497+0000","last_unstale":"2024-04-25T00:09:33.867045+0000","last_undegraded":"2024-04-25T00:09:33.867045+0000","last_fullsized":"2024-04-25T00:09:33.867045+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:32.173409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863516+0000","last_change":"2024-04-25T00:09:32.859351+0000","last_active":"2024-04-25T00:09:34.863516+0000","last_peered":"2024-04-25T00:09:34.863516+0000","last_clean":"2024-04-25T00:09:34.863516+0000","last_became_active":"2024-04-25T00:09:32.859068+0000","last_became_peered":"2024-04-25T00:09:32.859068+0000","last_unstale":"2024-04-25T00:09:34.863516+0000","last_undegraded":"2024-04-25T00:09:34.863516+0000","last_fullsized":"2024-04-25T00:09:34.863516+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:39.922934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168534+0000","last_change":"2024-04-25T00:09:33.872393+0000","last_active":"2024-04-25T00:09:35.168534+0000","last_peered":"2024-04-25T00:09:35.168534+0000","last_clean":"2024-04-25T00:09:35.168534+0000","last_became_active":"2024-04-25T00:09:33.872069+0000","last_became_peered":"2024-04-25T00:09:33.872069+0000","last_unstale":"2024-04-25T00:09:35.168534+0000","last_undegraded":"2024-04-25T00:09:35.168534+0000","last_fullsized":"2024-04-25T00:09:35.168534+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:54.319661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874113+0000","last_change":"2024-04-25T00:09:32.860524+0000","last_active":"2024-04-25T00:09:34.874113+0000","last_peered":"2024-04-25T00:09:34.874113+0000","last_clean":"2024-04-25T00:09:34.874113+0000","last_became_active":"2024-04-25T00:09:32.860045+0000","last_became_peered":"2024-04-25T00:09:32.860045+0000","last_unstale":"2024-04-25T00:09:34.874113+0000","last_undegraded":"2024-04-25T00:09:34.874113+0000","last_fullsized":"2024-04-25T00:09:34.874113+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:24.471772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863583+0000","last_change":"2024-04-25T00:09:33.865604+0000","last_active":"2024-04-25T00:09:34.863583+0000","last_peered":"2024-04-25T00:09:34.863583+0000","last_clean":"2024-04-25T00:09:34.863583+0000","last_became_active":"2024-04-25T00:09:33.865349+0000","last_became_peered":"2024-04-25T00:09:33.865349+0000","last_unstale":"2024-04-25T00:09:34.863583+0000","last_undegraded":"2024-04-25T00:09:34.863583+0000","last_fullsized":"2024-04-25T00:09:34.863583+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:55.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874500+0000","last_change":"2024-04-25T00:09:32.865204+0000","last_active":"2024-04-25T00:09:34.874500+0000","last_peered":"2024-04-25T00:09:34.874500+0000","last_clean":"2024-04-25T00:09:34.874500+0000","last_became_active":"2024-04-25T00:09:32.864901+0000","last_became_peered":"2024-04-25T00:09:32.864901+0000","last_unstale":"2024-04-25T00:09:34.874500+0000","last_undegraded":"2024-04-25T00:09:34.874500+0000","last_fullsized":"2024-04-25T00:09:34.874500+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:27.642272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":515,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871873+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:09:34.871873+0000","last_peered":"2024-04-25T00:09:34.871873+0000","last_clean":"2024-04-25T00:09:34.871873+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:09:34.871873+0000","last_undegraded":"2024-04-25T00:09:34.871873+0000","last_fullsized":"2024-04-25T00:09:34.871873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2899968,"data_stored":1597900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2297856,"data_stored":1006502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55640,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316200,"statfs":{"total":96636764160,"available":96579788800,"internally_reserved":0,"allocated":2297856,"data_stored":1006502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2314240,"data_stored":1007636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2293760,"data_stored":999802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51456,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320384,"statfs":{"total":96636764160,"available":96584073216,"internally_reserved":0,"allocated":2281472,"data_stored":999620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59804,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312036,"statfs":{"total":96636764160,"available":96575524864,"internally_reserved":0,"allocated":2301952,"data_stored":1005976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52060,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319780,"statfs":{"total":96636764160,"available":96583454720,"internally_reserved":0,"allocated":2891776,"data_stored":1596378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:09:39.120 INFO:tasks.ceph.mds.d.smithi178.stderr:2024-04-25T00:09:39.119+0000 7f6557a9f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:09:41.198 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:09:41.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.307+0000 7f037e272640 1 Processor -- start 2024-04-25T00:09:41.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.307+0000 7f037e272640 1 -- start start 2024-04-25T00:09:41.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.308+0000 7f037e272640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0378158910 0x7f0378162710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.308+0000 7f037e272640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f037815f1d0 con 0x7f037806b800 2024-04-25T00:09:41.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.308+0000 7f037e272640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f037815f4b0 con 0x7f037815e7a0 2024-04-25T00:09:41.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.308+0000 7f037e272640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0378162c50 con 0x7f0378158910 2024-04-25T00:09:41.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.308+0000 7f0377fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0378158910 0x7f0378162710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.308+0000 7f037c80f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f037806b800 0x7f03781438f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60728/0 (socket says 172.21.15.113:60728) 2024-04-25T00:09:41.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.308+0000 7f037c80f640 1 -- 172.21.15.113:0/3109803825 learned_addr learned my addr 172.21.15.113:0/3109803825 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:41.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.309+0000 7f0377fff640 1 -- 172.21.15.113:0/3109803825 >> v1:172.21.15.178:6789/0 conn(0x7f037815e7a0 legacy=0x7f037806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:41.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.309+0000 7f0377fff640 1 -- 172.21.15.113:0/3109803825 >> v1:172.21.15.113:6789/0 conn(0x7f037806b800 legacy=0x7f03781438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.309+0000 7f0377fff640 1 -- 172.21.15.113:0/3109803825 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0378161f10 con 0x7f0378158910 2024-04-25T00:09:41.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.309+0000 7f0376ffd640 1 -- 172.21.15.113:0/3109803825 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474171592 0 0) 0x7f037815f1d0 con 0x7f037806b800 2024-04-25T00:09:41.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.309+0000 7f0377fff640 1 --2- 172.21.15.113:0/3109803825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f0378162710 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f03680097a0 tx=0x7f036802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee61f61d097ff84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f0376ffd640 1 -- 172.21.15.113:0/3109803825 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368037070 con 0x7f0378158910 2024-04-25T00:09:41.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f0376ffd640 1 -- 172.21.15.113:0/3109803825 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0368032440 con 0x7f0378158910 2024-04-25T00:09:41.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f0376ffd640 1 -- 172.21.15.113:0/3109803825 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368032810 con 0x7f0378158910 2024-04-25T00:09:41.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f037e272640 1 -- 172.21.15.113:0/3109803825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 msgr2=0x7f0378162710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f037e272640 1 --2- 172.21.15.113:0/3109803825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f0378162710 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f03680097a0 tx=0x7f036802a1e0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f037e272640 1 -- 172.21.15.113:0/3109803825 shutdown_connections 2024-04-25T00:09:41.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f037e272640 1 --2- 172.21.15.113:0/3109803825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f0378162710 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f037e272640 1 -- 172.21.15.113:0/3109803825 >> 172.21.15.113:0/3109803825 conn(0x7f03780b22c0 msgr2=0x7f03780b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:41.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f037e272640 1 -- 172.21.15.113:0/3109803825 shutdown_connections 2024-04-25T00:09:41.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.310+0000 7f037e272640 1 -- 172.21.15.113:0/3109803825 wait complete. 2024-04-25T00:09:41.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.311+0000 7f037e272640 1 Processor -- start 2024-04-25T00:09:41.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f037e272640 1 -- start start 2024-04-25T00:09:41.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f037e272640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f037806b800 0x7f0378157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f037e272640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f03780ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f037e272640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f037815e7a0 0x7f03780cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f037e272640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f03780b3890 con 0x7f0378158910 2024-04-25T00:09:41.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f037e272640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03780b3710 con 0x7f037806b800 2024-04-25T00:09:41.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f037e272640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f037815ec50 con 0x7f037815e7a0 2024-04-25T00:09:41.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f03777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f03780ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f03777fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f03780ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43746/0 (socket says 172.21.15.113:43746) 2024-04-25T00:09:41.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.312+0000 7f03777fe640 1 -- 172.21.15.113:0/3524581349 learned_addr learned my addr 172.21.15.113:0/3524581349 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:41.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f037c80f640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f037815e7a0 0x7f03780cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f0377fff640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f037806b800 0x7f0378157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f03777fe640 1 -- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f037815e7a0 msgr2=0x7f03780cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f03777fe640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f037815e7a0 0x7f03780cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f03777fe640 1 -- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f037806b800 msgr2=0x7f0378157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f03777fe640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f037806b800 0x7f0378157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f03777fe640 1 -- 172.21.15.113:0/3524581349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03780cf560 con 0x7f0378158910 2024-04-25T00:09:41.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f03777fe640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f03780ce6e0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f036c00b3f0 tx=0x7f036c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f0374ff9640 1 -- 172.21.15.113:0/3524581349 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c015020 con 0x7f0378158910 2024-04-25T00:09:41.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f0374ff9640 1 -- 172.21.15.113:0/3524581349 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f036c011830 con 0x7f0378158910 2024-04-25T00:09:41.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03780cf840 con 0x7f0378158910 2024-04-25T00:09:41.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f0374ff9640 1 -- 172.21.15.113:0/3524581349 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c011c20 con 0x7f0378158910 2024-04-25T00:09:41.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.313+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0378157a40 con 0x7f0378158910 2024-04-25T00:09:41.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.314+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03780cfc30 con 0x7f0378158910 2024-04-25T00:09:41.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.315+0000 7f0374ff9640 1 -- 172.21.15.113:0/3524581349 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f036c018020 con 0x7f0378158910 2024-04-25T00:09:41.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.315+0000 7f0374ff9640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.315+0000 7f0377fff640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.315+0000 7f0374ff9640 1 -- 172.21.15.113:0/3524581349 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f036c08aa00 con 0x7f0378158910 2024-04-25T00:09:41.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.316+0000 7f0377fff640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0368034000 tx=0x7f036802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.320+0000 7f0374ff9640 1 -- 172.21.15.113:0/3524581349 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f036c01f030 con 0x7f0378158910 2024-04-25T00:09:41.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.489+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03780691a0 con 0x7f036406be30 2024-04-25T00:09:41.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.495+0000 7f0374ff9640 1 -- 172.21.15.113:0/3524581349 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252383 (secure 0 0 0) 0x7f03780691a0 con 0x7f036406be30 2024-04-25T00:09:41.498 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:41.499 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 msgr2=0x7f036406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0368034000 tx=0x7f036802e000 comp rx=0 tx=0).stop 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 msgr2=0x7f03780ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f03780ce6e0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f036c00b3f0 tx=0x7f036c00b8b0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 shutdown_connections 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f036406be30 0x7f036406e2f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f037815e7a0 0x7f03780cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0378158910 0x7f03780ce6e0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 --2- 172.21.15.113:0/3524581349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f037806b800 0x7f0378157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.498+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 >> 172.21.15.113:0/3524581349 conn(0x7f03780b22c0 msgr2=0x7f03780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:41.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.499+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 shutdown_connections 2024-04-25T00:09:41.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.499+0000 7f037e272640 1 -- 172.21.15.113:0/3524581349 wait complete. 2024-04-25T00:09:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-04-25T00:09:41.247840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":254,"num_write_kb":3476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421324,"kb_used_data":19144,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754553396,"statfs":{"total":773094113280,"available":772662677504,"internally_reserved":0,"allocated":19603456,"data_stored":9232916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001674"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863971+0000","last_change":"2024-04-25T00:09:33.871025+0000","last_active":"2024-04-25T00:09:34.863971+0000","last_peered":"2024-04-25T00:09:34.863971+0000","last_clean":"2024-04-25T00:09:34.863971+0000","last_became_active":"2024-04-25T00:09:33.870439+0000","last_became_peered":"2024-04-25T00:09:33.870439+0000","last_unstale":"2024-04-25T00:09:34.863971+0000","last_undegraded":"2024-04-25T00:09:34.863971+0000","last_fullsized":"2024-04-25T00:09:34.863971+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:59.226155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168700+0000","last_change":"2024-04-25T00:09:33.878604+0000","last_active":"2024-04-25T00:09:35.168700+0000","last_peered":"2024-04-25T00:09:35.168700+0000","last_clean":"2024-04-25T00:09:35.168700+0000","last_became_active":"2024-04-25T00:09:33.878328+0000","last_became_peered":"2024-04-25T00:09:33.878328+0000","last_unstale":"2024-04-25T00:09:35.168700+0000","last_undegraded":"2024-04-25T00:09:35.168700+0000","last_fullsized":"2024-04-25T00:09:35.168700+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:46.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675870+0000","last_change":"2024-04-25T00:09:33.878325+0000","last_active":"2024-04-25T00:09:36.675870+0000","last_peered":"2024-04-25T00:09:36.675870+0000","last_clean":"2024-04-25T00:09:36.675870+0000","last_became_active":"2024-04-25T00:09:33.877495+0000","last_became_peered":"2024-04-25T00:09:33.877495+0000","last_unstale":"2024-04-25T00:09:36.675870+0000","last_undegraded":"2024-04-25T00:09:36.675870+0000","last_fullsized":"2024-04-25T00:09:36.675870+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:54.319661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871982+0000","last_change":"2024-04-25T00:09:33.869224+0000","last_active":"2024-04-25T00:09:34.871982+0000","last_peered":"2024-04-25T00:09:34.871982+0000","last_clean":"2024-04-25T00:09:34.871982+0000","last_became_active":"2024-04-25T00:09:33.868734+0000","last_became_peered":"2024-04-25T00:09:33.868734+0000","last_unstale":"2024-04-25T00:09:34.871982+0000","last_undegraded":"2024-04-25T00:09:34.871982+0000","last_fullsized":"2024-04-25T00:09:34.871982+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:16.109657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863334+0000","last_change":"2024-04-25T00:09:33.869714+0000","last_active":"2024-04-25T00:09:34.863334+0000","last_peered":"2024-04-25T00:09:34.863334+0000","last_clean":"2024-04-25T00:09:34.863334+0000","last_became_active":"2024-04-25T00:09:33.869435+0000","last_became_peered":"2024-04-25T00:09:33.869435+0000","last_unstale":"2024-04-25T00:09:34.863334+0000","last_undegraded":"2024-04-25T00:09:34.863334+0000","last_fullsized":"2024-04-25T00:09:34.863334+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:59.450004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863112+0000","last_change":"2024-04-25T00:09:33.866179+0000","last_active":"2024-04-25T00:09:34.863112+0000","last_peered":"2024-04-25T00:09:34.863112+0000","last_clean":"2024-04-25T00:09:34.863112+0000","last_became_active":"2024-04-25T00:09:33.865891+0000","last_became_peered":"2024-04-25T00:09:33.865891+0000","last_unstale":"2024-04-25T00:09:34.863112+0000","last_undegraded":"2024-04-25T00:09:34.863112+0000","last_fullsized":"2024-04-25T00:09:34.863112+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:00.144856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863588+0000","last_change":"2024-04-25T00:09:33.875207+0000","last_active":"2024-04-25T00:09:34.863588+0000","last_peered":"2024-04-25T00:09:34.863588+0000","last_clean":"2024-04-25T00:09:34.863588+0000","last_became_active":"2024-04-25T00:09:33.874674+0000","last_became_peered":"2024-04-25T00:09:33.874674+0000","last_unstale":"2024-04-25T00:09:34.863588+0000","last_undegraded":"2024-04-25T00:09:34.863588+0000","last_fullsized":"2024-04-25T00:09:34.863588+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:14.333375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862963+0000","last_change":"2024-04-25T00:09:33.860996+0000","last_active":"2024-04-25T00:09:34.862963+0000","last_peered":"2024-04-25T00:09:34.862963+0000","last_clean":"2024-04-25T00:09:34.862963+0000","last_became_active":"2024-04-25T00:09:33.860726+0000","last_became_peered":"2024-04-25T00:09:33.860726+0000","last_unstale":"2024-04-25T00:09:34.862963+0000","last_undegraded":"2024-04-25T00:09:34.862963+0000","last_fullsized":"2024-04-25T00:09:34.862963+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:40.927785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.169032+0000","last_change":"2024-04-25T00:09:33.877988+0000","last_active":"2024-04-25T00:09:35.169032+0000","last_peered":"2024-04-25T00:09:35.169032+0000","last_clean":"2024-04-25T00:09:35.169032+0000","last_became_active":"2024-04-25T00:09:33.877216+0000","last_became_peered":"2024-04-25T00:09:33.877216+0000","last_unstale":"2024-04-25T00:09:35.169032+0000","last_undegraded":"2024-04-25T00:09:35.169032+0000","last_fullsized":"2024-04-25T00:09:35.169032+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:21:41.886576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676560+0000","last_change":"2024-04-25T00:09:33.871557+0000","last_active":"2024-04-25T00:09:36.676560+0000","last_peered":"2024-04-25T00:09:36.676560+0000","last_clean":"2024-04-25T00:09:36.676560+0000","last_became_active":"2024-04-25T00:09:33.871300+0000","last_became_peered":"2024-04-25T00:09:33.871300+0000","last_unstale":"2024-04-25T00:09:36.676560+0000","last_undegraded":"2024-04-25T00:09:36.676560+0000","last_fullsized":"2024-04-25T00:09:36.676560+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:27.868357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676013+0000","last_change":"2024-04-25T00:09:33.868170+0000","last_active":"2024-04-25T00:09:36.676013+0000","last_peered":"2024-04-25T00:09:36.676013+0000","last_clean":"2024-04-25T00:09:36.676013+0000","last_became_active":"2024-04-25T00:09:33.867382+0000","last_became_peered":"2024-04-25T00:09:33.867382+0000","last_unstale":"2024-04-25T00:09:36.676013+0000","last_undegraded":"2024-04-25T00:09:36.676013+0000","last_fullsized":"2024-04-25T00:09:36.676013+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:18.123619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874365+0000","last_change":"2024-04-25T00:09:33.873865+0000","last_active":"2024-04-25T00:09:34.874365+0000","last_peered":"2024-04-25T00:09:34.874365+0000","last_clean":"2024-04-25T00:09:34.874365+0000","last_became_active":"2024-04-25T00:09:33.873531+0000","last_became_peered":"2024-04-25T00:09:33.873531+0000","last_unstale":"2024-04-25T00:09:34.874365+0000","last_undegraded":"2024-04-25T00:09:34.874365+0000","last_fullsized":"2024-04-25T00:09:34.874365+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:30.035581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874436+0000","last_change":"2024-04-25T00:09:33.867995+0000","last_active":"2024-04-25T00:09:34.874436+0000","last_peered":"2024-04-25T00:09:34.874436+0000","last_clean":"2024-04-25T00:09:34.874436+0000","last_became_active":"2024-04-25T00:09:33.867692+0000","last_became_peered":"2024-04-25T00:09:33.867692+0000","last_unstale":"2024-04-25T00:09:34.874436+0000","last_undegraded":"2024-04-25T00:09:34.874436+0000","last_fullsized":"2024-04-25T00:09:34.874436+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:33.016301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863341+0000","last_change":"2024-04-25T00:09:33.862436+0000","last_active":"2024-04-25T00:09:34.863341+0000","last_peered":"2024-04-25T00:09:34.863341+0000","last_clean":"2024-04-25T00:09:34.863341+0000","last_became_active":"2024-04-25T00:09:33.862096+0000","last_became_peered":"2024-04-25T00:09:33.862096+0000","last_unstale":"2024-04-25T00:09:34.863341+0000","last_undegraded":"2024-04-25T00:09:34.863341+0000","last_fullsized":"2024-04-25T00:09:34.863341+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:25.636303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863725+0000","last_change":"2024-04-25T00:09:33.865358+0000","last_active":"2024-04-25T00:09:34.863725+0000","last_peered":"2024-04-25T00:09:34.863725+0000","last_clean":"2024-04-25T00:09:34.863725+0000","last_became_active":"2024-04-25T00:09:33.865098+0000","last_became_peered":"2024-04-25T00:09:33.865098+0000","last_unstale":"2024-04-25T00:09:34.863725+0000","last_undegraded":"2024-04-25T00:09:34.863725+0000","last_fullsized":"2024-04-25T00:09:34.863725+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:08.885015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676103+0000","last_change":"2024-04-25T00:09:33.879644+0000","last_active":"2024-04-25T00:09:36.676103+0000","last_peered":"2024-04-25T00:09:36.676103+0000","last_clean":"2024-04-25T00:09:36.676103+0000","last_became_active":"2024-04-25T00:09:33.879324+0000","last_became_peered":"2024-04-25T00:09:33.879324+0000","last_unstale":"2024-04-25T00:09:36.676103+0000","last_undegraded":"2024-04-25T00:09:36.676103+0000","last_fullsized":"2024-04-25T00:09:36.676103+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:23.186528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864131+0000","last_change":"2024-04-25T00:09:33.870900+0000","last_active":"2024-04-25T00:09:34.864131+0000","last_peered":"2024-04-25T00:09:34.864131+0000","last_clean":"2024-04-25T00:09:34.864131+0000","last_became_active":"2024-04-25T00:09:33.870115+0000","last_became_peered":"2024-04-25T00:09:33.870115+0000","last_unstale":"2024-04-25T00:09:34.864131+0000","last_undegraded":"2024-04-25T00:09:34.864131+0000","last_fullsized":"2024-04-25T00:09:34.864131+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:39.258462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676436+0000","last_change":"2024-04-25T00:09:33.872660+0000","last_active":"2024-04-25T00:09:36.676436+0000","last_peered":"2024-04-25T00:09:36.676436+0000","last_clean":"2024-04-25T00:09:36.676436+0000","last_became_active":"2024-04-25T00:09:33.872373+0000","last_became_peered":"2024-04-25T00:09:33.872373+0000","last_unstale":"2024-04-25T00:09:36.676436+0000","last_undegraded":"2024-04-25T00:09:36.676436+0000","last_fullsized":"2024-04-25T00:09:36.676436+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:42.327222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676587+0000","last_change":"2024-04-25T00:09:33.871847+0000","last_active":"2024-04-25T00:09:36.676587+0000","last_peered":"2024-04-25T00:09:36.676587+0000","last_clean":"2024-04-25T00:09:36.676587+0000","last_became_active":"2024-04-25T00:09:33.871112+0000","last_became_peered":"2024-04-25T00:09:33.871112+0000","last_unstale":"2024-04-25T00:09:36.676587+0000","last_undegraded":"2024-04-25T00:09:36.676587+0000","last_fullsized":"2024-04-25T00:09:36.676587+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:24.824939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863227+0000","last_change":"2024-04-25T00:09:33.869405+0000","last_active":"2024-04-25T00:09:34.863227+0000","last_peered":"2024-04-25T00:09:34.863227+0000","last_clean":"2024-04-25T00:09:34.863227+0000","last_became_active":"2024-04-25T00:09:33.869159+0000","last_became_peered":"2024-04-25T00:09:33.869159+0000","last_unstale":"2024-04-25T00:09:34.863227+0000","last_undegraded":"2024-04-25T00:09:34.863227+0000","last_fullsized":"2024-04-25T00:09:34.863227+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:30.035581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871904+0000","last_change":"2024-04-25T00:09:33.868561+0000","last_active":"2024-04-25T00:09:34.871904+0000","last_peered":"2024-04-25T00:09:34.871904+0000","last_clean":"2024-04-25T00:09:34.871904+0000","last_became_active":"2024-04-25T00:09:33.868230+0000","last_became_peered":"2024-04-25T00:09:33.868230+0000","last_unstale":"2024-04-25T00:09:34.871904+0000","last_undegraded":"2024-04-25T00:09:34.871904+0000","last_fullsized":"2024-04-25T00:09:34.871904+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.864780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863389+0000","last_change":"2024-04-25T00:09:33.870026+0000","last_active":"2024-04-25T00:09:34.863389+0000","last_peered":"2024-04-25T00:09:34.863389+0000","last_clean":"2024-04-25T00:09:34.863389+0000","last_became_active":"2024-04-25T00:09:33.869745+0000","last_became_peered":"2024-04-25T00:09:33.869745+0000","last_unstale":"2024-04-25T00:09:34.863389+0000","last_undegraded":"2024-04-25T00:09:34.863389+0000","last_fullsized":"2024-04-25T00:09:34.863389+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:31.093182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863368+0000","last_change":"2024-04-25T00:09:33.871850+0000","last_active":"2024-04-25T00:09:34.863368+0000","last_peered":"2024-04-25T00:09:34.863368+0000","last_clean":"2024-04-25T00:09:34.863368+0000","last_became_active":"2024-04-25T00:09:33.871593+0000","last_became_peered":"2024-04-25T00:09:33.871593+0000","last_unstale":"2024-04-25T00:09:34.863368+0000","last_undegraded":"2024-04-25T00:09:34.863368+0000","last_fullsized":"2024-04-25T00:09:34.863368+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:33.016301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871805+0000","last_change":"2024-04-25T00:09:33.867377+0000","last_active":"2024-04-25T00:09:34.871805+0000","last_peered":"2024-04-25T00:09:34.871805+0000","last_clean":"2024-04-25T00:09:34.871805+0000","last_became_active":"2024-04-25T00:09:33.867154+0000","last_became_peered":"2024-04-25T00:09:33.867154+0000","last_unstale":"2024-04-25T00:09:34.871805+0000","last_undegraded":"2024-04-25T00:09:34.871805+0000","last_fullsized":"2024-04-25T00:09:34.871805+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:43.616946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168803+0000","last_change":"2024-04-25T00:09:33.871702+0000","last_active":"2024-04-25T00:09:35.168803+0000","last_peered":"2024-04-25T00:09:35.168803+0000","last_clean":"2024-04-25T00:09:35.168803+0000","last_became_active":"2024-04-25T00:09:33.871458+0000","last_became_peered":"2024-04-25T00:09:33.871458+0000","last_unstale":"2024-04-25T00:09:35.168803+0000","last_undegraded":"2024-04-25T00:09:35.168803+0000","last_fullsized":"2024-04-25T00:09:35.168803+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:25.476623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864017+0000","last_change":"2024-04-25T00:09:33.875792+0000","last_active":"2024-04-25T00:09:34.864017+0000","last_peered":"2024-04-25T00:09:34.864017+0000","last_clean":"2024-04-25T00:09:34.864017+0000","last_became_active":"2024-04-25T00:09:33.874998+0000","last_became_peered":"2024-04-25T00:09:33.874998+0000","last_unstale":"2024-04-25T00:09:34.864017+0000","last_undegraded":"2024-04-25T00:09:34.864017+0000","last_fullsized":"2024-04-25T00:09:34.864017+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.956814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874361+0000","last_change":"2024-04-25T00:09:33.875621+0000","last_active":"2024-04-25T00:09:34.874361+0000","last_peered":"2024-04-25T00:09:34.874361+0000","last_clean":"2024-04-25T00:09:34.874361+0000","last_became_active":"2024-04-25T00:09:33.875162+0000","last_became_peered":"2024-04-25T00:09:33.875162+0000","last_unstale":"2024-04-25T00:09:34.874361+0000","last_undegraded":"2024-04-25T00:09:34.874361+0000","last_fullsized":"2024-04-25T00:09:34.874361+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:00.144856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675742+0000","last_change":"2024-04-25T00:09:33.878105+0000","last_active":"2024-04-25T00:09:36.675742+0000","last_peered":"2024-04-25T00:09:36.675742+0000","last_clean":"2024-04-25T00:09:36.675742+0000","last_became_active":"2024-04-25T00:09:33.877837+0000","last_became_peered":"2024-04-25T00:09:33.877837+0000","last_unstale":"2024-04-25T00:09:36.675742+0000","last_undegraded":"2024-04-25T00:09:36.675742+0000","last_fullsized":"2024-04-25T00:09:36.675742+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:10.468326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874230+0000","last_change":"2024-04-25T00:09:33.875264+0000","last_active":"2024-04-25T00:09:34.874230+0000","last_peered":"2024-04-25T00:09:34.874230+0000","last_clean":"2024-04-25T00:09:34.874230+0000","last_became_active":"2024-04-25T00:09:33.874940+0000","last_became_peered":"2024-04-25T00:09:33.874940+0000","last_unstale":"2024-04-25T00:09:34.874230+0000","last_undegraded":"2024-04-25T00:09:34.874230+0000","last_fullsized":"2024-04-25T00:09:34.874230+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:07.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168658+0000","last_change":"2024-04-25T00:09:33.870203+0000","last_active":"2024-04-25T00:09:35.168658+0000","last_peered":"2024-04-25T00:09:35.168658+0000","last_clean":"2024-04-25T00:09:35.168658+0000","last_became_active":"2024-04-25T00:09:33.869779+0000","last_became_peered":"2024-04-25T00:09:33.869779+0000","last_unstale":"2024-04-25T00:09:35.168658+0000","last_undegraded":"2024-04-25T00:09:35.168658+0000","last_fullsized":"2024-04-25T00:09:35.168658+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:18.123619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676179+0000","last_change":"2024-04-25T00:09:33.868363+0000","last_active":"2024-04-25T00:09:36.676179+0000","last_peered":"2024-04-25T00:09:36.676179+0000","last_clean":"2024-04-25T00:09:36.676179+0000","last_became_active":"2024-04-25T00:09:33.868107+0000","last_became_peered":"2024-04-25T00:09:33.868107+0000","last_unstale":"2024-04-25T00:09:36.676179+0000","last_undegraded":"2024-04-25T00:09:36.676179+0000","last_fullsized":"2024-04-25T00:09:36.676179+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:56.970001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862961+0000","last_change":"2024-04-25T00:09:33.864477+0000","last_active":"2024-04-25T00:09:34.862961+0000","last_peered":"2024-04-25T00:09:34.862961+0000","last_clean":"2024-04-25T00:09:34.862961+0000","last_became_active":"2024-04-25T00:09:33.864169+0000","last_became_peered":"2024-04-25T00:09:33.864169+0000","last_unstale":"2024-04-25T00:09:34.862961+0000","last_undegraded":"2024-04-25T00:09:34.862961+0000","last_fullsized":"2024-04-25T00:09:34.862961+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.332387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168766+0000","last_change":"2024-04-25T00:09:33.874753+0000","last_active":"2024-04-25T00:09:35.168766+0000","last_peered":"2024-04-25T00:09:35.168766+0000","last_clean":"2024-04-25T00:09:35.168766+0000","last_became_active":"2024-04-25T00:09:33.874394+0000","last_became_peered":"2024-04-25T00:09:33.874394+0000","last_unstale":"2024-04-25T00:09:35.168766+0000","last_undegraded":"2024-04-25T00:09:35.168766+0000","last_fullsized":"2024-04-25T00:09:35.168766+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:28.647123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874479+0000","last_change":"2024-04-25T00:09:32.863939+0000","last_active":"2024-04-25T00:09:34.874479+0000","last_peered":"2024-04-25T00:09:34.874479+0000","last_clean":"2024-04-25T00:09:34.874479+0000","last_became_active":"2024-04-25T00:09:32.863651+0000","last_became_peered":"2024-04-25T00:09:32.863651+0000","last_unstale":"2024-04-25T00:09:34.874479+0000","last_undegraded":"2024-04-25T00:09:34.874479+0000","last_fullsized":"2024-04-25T00:09:34.874479+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:09.187640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863556+0000","last_change":"2024-04-25T00:09:33.867201+0000","last_active":"2024-04-25T00:09:34.863556+0000","last_peered":"2024-04-25T00:09:34.863556+0000","last_clean":"2024-04-25T00:09:34.863556+0000","last_became_active":"2024-04-25T00:09:33.866911+0000","last_became_peered":"2024-04-25T00:09:33.866911+0000","last_unstale":"2024-04-25T00:09:34.863556+0000","last_undegraded":"2024-04-25T00:09:34.863556+0000","last_fullsized":"2024-04-25T00:09:34.863556+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:54.718780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168907+0000","last_change":"2024-04-25T00:09:32.868306+0000","last_active":"2024-04-25T00:09:35.168907+0000","last_peered":"2024-04-25T00:09:35.168907+0000","last_clean":"2024-04-25T00:09:35.168907+0000","last_became_active":"2024-04-25T00:09:32.867847+0000","last_became_peered":"2024-04-25T00:09:32.867847+0000","last_unstale":"2024-04-25T00:09:35.168907+0000","last_undegraded":"2024-04-25T00:09:35.168907+0000","last_fullsized":"2024-04-25T00:09:35.168907+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:23.760066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863575+0000","last_change":"2024-04-25T00:09:33.874758+0000","last_active":"2024-04-25T00:09:34.863575+0000","last_peered":"2024-04-25T00:09:34.863575+0000","last_clean":"2024-04-25T00:09:34.863575+0000","last_became_active":"2024-04-25T00:09:33.873840+0000","last_became_peered":"2024-04-25T00:09:33.873840+0000","last_unstale":"2024-04-25T00:09:34.863575+0000","last_undegraded":"2024-04-25T00:09:34.863575+0000","last_fullsized":"2024-04-25T00:09:34.863575+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:37.375085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675968+0000","last_change":"2024-04-25T00:09:32.866298+0000","last_active":"2024-04-25T00:09:36.675968+0000","last_peered":"2024-04-25T00:09:36.675968+0000","last_clean":"2024-04-25T00:09:36.675968+0000","last_became_active":"2024-04-25T00:09:32.866011+0000","last_became_peered":"2024-04-25T00:09:32.866011+0000","last_unstale":"2024-04-25T00:09:36.675968+0000","last_undegraded":"2024-04-25T00:09:36.675968+0000","last_fullsized":"2024-04-25T00:09:36.675968+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:41.242765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676239+0000","last_change":"2024-04-25T00:09:33.878077+0000","last_active":"2024-04-25T00:09:36.676239+0000","last_peered":"2024-04-25T00:09:36.676239+0000","last_clean":"2024-04-25T00:09:36.676239+0000","last_became_active":"2024-04-25T00:09:33.877808+0000","last_became_peered":"2024-04-25T00:09:33.877808+0000","last_unstale":"2024-04-25T00:09:36.676239+0000","last_undegraded":"2024-04-25T00:09:36.676239+0000","last_fullsized":"2024-04-25T00:09:36.676239+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:17.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872127+0000","last_change":"2024-04-25T00:09:32.870037+0000","last_active":"2024-04-25T00:09:34.872127+0000","last_peered":"2024-04-25T00:09:34.872127+0000","last_clean":"2024-04-25T00:09:34.872127+0000","last_became_active":"2024-04-25T00:09:32.869737+0000","last_became_peered":"2024-04-25T00:09:32.869737+0000","last_unstale":"2024-04-25T00:09:34.872127+0000","last_undegraded":"2024-04-25T00:09:34.872127+0000","last_fullsized":"2024-04-25T00:09:34.872127+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:16.039533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871893+0000","last_change":"2024-04-25T00:09:33.869903+0000","last_active":"2024-04-25T00:09:34.871893+0000","last_peered":"2024-04-25T00:09:34.871893+0000","last_clean":"2024-04-25T00:09:34.871893+0000","last_became_active":"2024-04-25T00:09:33.868905+0000","last_became_peered":"2024-04-25T00:09:33.868905+0000","last_unstale":"2024-04-25T00:09:34.871893+0000","last_undegraded":"2024-04-25T00:09:34.871893+0000","last_fullsized":"2024-04-25T00:09:34.871893+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:21.795335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863378+0000","last_change":"2024-04-25T00:09:32.868485+0000","last_active":"2024-04-25T00:09:34.863378+0000","last_peered":"2024-04-25T00:09:34.863378+0000","last_clean":"2024-04-25T00:09:34.863378+0000","last_became_active":"2024-04-25T00:09:32.868229+0000","last_became_peered":"2024-04-25T00:09:32.868229+0000","last_unstale":"2024-04-25T00:09:34.863378+0000","last_undegraded":"2024-04-25T00:09:34.863378+0000","last_fullsized":"2024-04-25T00:09:34.863378+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:30.649758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863102+0000","last_change":"2024-04-25T00:09:33.866550+0000","last_active":"2024-04-25T00:09:34.863102+0000","last_peered":"2024-04-25T00:09:34.863102+0000","last_clean":"2024-04-25T00:09:34.863102+0000","last_became_active":"2024-04-25T00:09:33.866017+0000","last_became_peered":"2024-04-25T00:09:33.866017+0000","last_unstale":"2024-04-25T00:09:34.863102+0000","last_undegraded":"2024-04-25T00:09:34.863102+0000","last_fullsized":"2024-04-25T00:09:34.863102+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:07.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.862888+0000","last_change":"2024-04-25T00:09:32.867526+0000","last_active":"2024-04-25T00:09:34.862888+0000","last_peered":"2024-04-25T00:09:34.862888+0000","last_clean":"2024-04-25T00:09:34.862888+0000","last_became_active":"2024-04-25T00:09:32.867237+0000","last_became_peered":"2024-04-25T00:09:32.867237+0000","last_unstale":"2024-04-25T00:09:34.862888+0000","last_undegraded":"2024-04-25T00:09:34.862888+0000","last_fullsized":"2024-04-25T00:09:34.862888+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:53.314810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871902+0000","last_change":"2024-04-25T00:09:33.868169+0000","last_active":"2024-04-25T00:09:34.871902+0000","last_peered":"2024-04-25T00:09:34.871902+0000","last_clean":"2024-04-25T00:09:34.871902+0000","last_became_active":"2024-04-25T00:09:33.867912+0000","last_became_peered":"2024-04-25T00:09:33.867912+0000","last_unstale":"2024-04-25T00:09:34.871902+0000","last_undegraded":"2024-04-25T00:09:34.871902+0000","last_fullsized":"2024-04-25T00:09:34.871902+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:22.186105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863088+0000","last_change":"2024-04-25T00:09:32.862149+0000","last_active":"2024-04-25T00:09:34.863088+0000","last_peered":"2024-04-25T00:09:34.863088+0000","last_clean":"2024-04-25T00:09:34.863088+0000","last_became_active":"2024-04-25T00:09:32.861803+0000","last_became_peered":"2024-04-25T00:09:32.861803+0000","last_unstale":"2024-04-25T00:09:34.863088+0000","last_undegraded":"2024-04-25T00:09:34.863088+0000","last_fullsized":"2024-04-25T00:09:34.863088+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:28.007292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872261+0000","last_change":"2024-04-25T00:09:33.881209+0000","last_active":"2024-04-25T00:09:34.872261+0000","last_peered":"2024-04-25T00:09:34.872261+0000","last_clean":"2024-04-25T00:09:34.872261+0000","last_became_active":"2024-04-25T00:09:33.880746+0000","last_became_peered":"2024-04-25T00:09:33.880746+0000","last_unstale":"2024-04-25T00:09:34.872261+0000","last_undegraded":"2024-04-25T00:09:34.872261+0000","last_fullsized":"2024-04-25T00:09:34.872261+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:53:17.623421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863158+0000","last_change":"2024-04-25T00:09:32.867518+0000","last_active":"2024-04-25T00:09:34.863158+0000","last_peered":"2024-04-25T00:09:34.863158+0000","last_clean":"2024-04-25T00:09:34.863158+0000","last_became_active":"2024-04-25T00:09:32.867185+0000","last_became_peered":"2024-04-25T00:09:32.867185+0000","last_unstale":"2024-04-25T00:09:34.863158+0000","last_undegraded":"2024-04-25T00:09:34.863158+0000","last_fullsized":"2024-04-25T00:09:34.863158+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:17.118768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675844+0000","last_change":"2024-04-25T00:09:33.869306+0000","last_active":"2024-04-25T00:09:36.675844+0000","last_peered":"2024-04-25T00:09:36.675844+0000","last_clean":"2024-04-25T00:09:36.675844+0000","last_became_active":"2024-04-25T00:09:33.868632+0000","last_became_peered":"2024-04-25T00:09:33.868632+0000","last_unstale":"2024-04-25T00:09:36.675844+0000","last_undegraded":"2024-04-25T00:09:36.675844+0000","last_fullsized":"2024-04-25T00:09:36.675844+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:04.608188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863652+0000","last_change":"2024-04-25T00:09:32.864351+0000","last_active":"2024-04-25T00:09:34.863652+0000","last_peered":"2024-04-25T00:09:34.863652+0000","last_clean":"2024-04-25T00:09:34.863652+0000","last_became_active":"2024-04-25T00:09:32.864085+0000","last_became_peered":"2024-04-25T00:09:32.864085+0000","last_unstale":"2024-04-25T00:09:34.863652+0000","last_undegraded":"2024-04-25T00:09:34.863652+0000","last_fullsized":"2024-04-25T00:09:34.863652+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:57.123376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872043+0000","last_change":"2024-04-25T00:09:33.881867+0000","last_active":"2024-04-25T00:09:34.872043+0000","last_peered":"2024-04-25T00:09:34.872043+0000","last_clean":"2024-04-25T00:09:34.872043+0000","last_became_active":"2024-04-25T00:09:33.881578+0000","last_became_peered":"2024-04-25T00:09:33.881578+0000","last_unstale":"2024-04-25T00:09:34.872043+0000","last_undegraded":"2024-04-25T00:09:34.872043+0000","last_fullsized":"2024-04-25T00:09:34.872043+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:44.059497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"181'3","reported_seq":17,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676542+0000","last_change":"2024-04-25T00:09:32.866762+0000","last_active":"2024-04-25T00:09:36.676542+0000","last_peered":"2024-04-25T00:09:36.676542+0000","last_clean":"2024-04-25T00:09:36.676542+0000","last_became_active":"2024-04-25T00:09:32.866452+0000","last_became_peered":"2024-04-25T00:09:32.866452+0000","last_unstale":"2024-04-25T00:09:36.676542+0000","last_undegraded":"2024-04-25T00:09:36.676542+0000","last_fullsized":"2024-04-25T00:09:36.676542+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:49.965548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863728+0000","last_change":"2024-04-25T00:09:33.867471+0000","last_active":"2024-04-25T00:09:34.863728+0000","last_peered":"2024-04-25T00:09:34.863728+0000","last_clean":"2024-04-25T00:09:34.863728+0000","last_became_active":"2024-04-25T00:09:33.867207+0000","last_became_peered":"2024-04-25T00:09:33.867207+0000","last_unstale":"2024-04-25T00:09:34.863728+0000","last_undegraded":"2024-04-25T00:09:34.863728+0000","last_fullsized":"2024-04-25T00:09:34.863728+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:58.832182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":20,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872192+0000","last_change":"2024-04-25T00:09:32.869149+0000","last_active":"2024-04-25T00:09:34.872192+0000","last_peered":"2024-04-25T00:09:34.872192+0000","last_clean":"2024-04-25T00:09:34.872192+0000","last_became_active":"2024-04-25T00:09:32.868846+0000","last_became_peered":"2024-04-25T00:09:32.868846+0000","last_unstale":"2024-04-25T00:09:34.872192+0000","last_undegraded":"2024-04-25T00:09:34.872192+0000","last_fullsized":"2024-04-25T00:09:34.872192+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:18.348472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863360+0000","last_change":"2024-04-25T00:09:33.870433+0000","last_active":"2024-04-25T00:09:34.863360+0000","last_peered":"2024-04-25T00:09:34.863360+0000","last_clean":"2024-04-25T00:09:34.863360+0000","last_became_active":"2024-04-25T00:09:33.869736+0000","last_became_peered":"2024-04-25T00:09:33.869736+0000","last_unstale":"2024-04-25T00:09:34.863360+0000","last_undegraded":"2024-04-25T00:09:34.863360+0000","last_fullsized":"2024-04-25T00:09:34.863360+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:59.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863562+0000","last_change":"2024-04-25T00:09:32.870900+0000","last_active":"2024-04-25T00:09:34.863562+0000","last_peered":"2024-04-25T00:09:34.863562+0000","last_clean":"2024-04-25T00:09:34.863562+0000","last_became_active":"2024-04-25T00:09:32.870635+0000","last_became_peered":"2024-04-25T00:09:32.870635+0000","last_unstale":"2024-04-25T00:09:34.863562+0000","last_undegraded":"2024-04-25T00:09:34.863562+0000","last_fullsized":"2024-04-25T00:09:34.863562+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:24:51.613038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676256+0000","last_change":"2024-04-25T00:09:33.874352+0000","last_active":"2024-04-25T00:09:36.676256+0000","last_peered":"2024-04-25T00:09:36.676256+0000","last_clean":"2024-04-25T00:09:36.676256+0000","last_became_active":"2024-04-25T00:09:33.874086+0000","last_became_peered":"2024-04-25T00:09:33.874086+0000","last_unstale":"2024-04-25T00:09:36.676256+0000","last_undegraded":"2024-04-25T00:09:36.676256+0000","last_fullsized":"2024-04-25T00:09:36.676256+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:19.353323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863285+0000","last_change":"2024-04-25T00:09:32.860288+0000","last_active":"2024-04-25T00:09:34.863285+0000","last_peered":"2024-04-25T00:09:34.863285+0000","last_clean":"2024-04-25T00:09:34.863285+0000","last_became_active":"2024-04-25T00:09:32.860010+0000","last_became_peered":"2024-04-25T00:09:32.860010+0000","last_unstale":"2024-04-25T00:09:34.863285+0000","last_undegraded":"2024-04-25T00:09:34.863285+0000","last_fullsized":"2024-04-25T00:09:34.863285+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:40.355571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675606+0000","last_change":"2024-04-25T00:09:33.867592+0000","last_active":"2024-04-25T00:09:36.675606+0000","last_peered":"2024-04-25T00:09:36.675606+0000","last_clean":"2024-04-25T00:09:36.675606+0000","last_became_active":"2024-04-25T00:09:33.867328+0000","last_became_peered":"2024-04-25T00:09:33.867328+0000","last_unstale":"2024-04-25T00:09:36.675606+0000","last_undegraded":"2024-04-25T00:09:36.675606+0000","last_fullsized":"2024-04-25T00:09:36.675606+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:38:15.211759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676271+0000","last_change":"2024-04-25T00:09:32.862727+0000","last_active":"2024-04-25T00:09:36.676271+0000","last_peered":"2024-04-25T00:09:36.676271+0000","last_clean":"2024-04-25T00:09:36.676271+0000","last_became_active":"2024-04-25T00:09:32.862467+0000","last_became_peered":"2024-04-25T00:09:32.862467+0000","last_unstale":"2024-04-25T00:09:36.676271+0000","last_undegraded":"2024-04-25T00:09:36.676271+0000","last_fullsized":"2024-04-25T00:09:36.676271+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:20.083614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863840+0000","last_change":"2024-04-25T00:09:33.875462+0000","last_active":"2024-04-25T00:09:34.863840+0000","last_peered":"2024-04-25T00:09:34.863840+0000","last_clean":"2024-04-25T00:09:34.863840+0000","last_became_active":"2024-04-25T00:09:33.875199+0000","last_became_peered":"2024-04-25T00:09:33.875199+0000","last_unstale":"2024-04-25T00:09:34.863840+0000","last_undegraded":"2024-04-25T00:09:34.863840+0000","last_fullsized":"2024-04-25T00:09:34.863840+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:55.059466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"181'8","reported_seq":22,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675709+0000","last_change":"2024-04-25T00:09:32.865419+0000","last_active":"2024-04-25T00:09:36.675709+0000","last_peered":"2024-04-25T00:09:36.675709+0000","last_clean":"2024-04-25T00:09:36.675709+0000","last_became_active":"2024-04-25T00:09:32.865121+0000","last_became_peered":"2024-04-25T00:09:32.865121+0000","last_unstale":"2024-04-25T00:09:36.675709+0000","last_undegraded":"2024-04-25T00:09:36.675709+0000","last_fullsized":"2024-04-25T00:09:36.675709+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:20.150413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168820+0000","last_change":"2024-04-25T00:09:33.879136+0000","last_active":"2024-04-25T00:09:35.168820+0000","last_peered":"2024-04-25T00:09:35.168820+0000","last_clean":"2024-04-25T00:09:35.168820+0000","last_became_active":"2024-04-25T00:09:33.876729+0000","last_became_peered":"2024-04-25T00:09:33.876729+0000","last_unstale":"2024-04-25T00:09:35.168820+0000","last_undegraded":"2024-04-25T00:09:35.168820+0000","last_fullsized":"2024-04-25T00:09:35.168820+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:10.192491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863251+0000","last_change":"2024-04-25T00:09:32.872419+0000","last_active":"2024-04-25T00:09:34.863251+0000","last_peered":"2024-04-25T00:09:34.863251+0000","last_clean":"2024-04-25T00:09:34.863251+0000","last_became_active":"2024-04-25T00:09:32.872151+0000","last_became_peered":"2024-04-25T00:09:32.872151+0000","last_unstale":"2024-04-25T00:09:34.863251+0000","last_undegraded":"2024-04-25T00:09:34.863251+0000","last_fullsized":"2024-04-25T00:09:34.863251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:45.275982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168889+0000","last_change":"2024-04-25T00:09:33.871091+0000","last_active":"2024-04-25T00:09:35.168889+0000","last_peered":"2024-04-25T00:09:35.168889+0000","last_clean":"2024-04-25T00:09:35.168889+0000","last_became_active":"2024-04-25T00:09:33.870803+0000","last_became_peered":"2024-04-25T00:09:33.870803+0000","last_unstale":"2024-04-25T00:09:35.168889+0000","last_undegraded":"2024-04-25T00:09:35.168889+0000","last_fullsized":"2024-04-25T00:09:35.168889+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:12.471194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168966+0000","last_change":"2024-04-25T00:09:32.868130+0000","last_active":"2024-04-25T00:09:35.168966+0000","last_peered":"2024-04-25T00:09:35.168966+0000","last_clean":"2024-04-25T00:09:35.168966+0000","last_became_active":"2024-04-25T00:09:32.867778+0000","last_became_peered":"2024-04-25T00:09:32.867778+0000","last_unstale":"2024-04-25T00:09:35.168966+0000","last_undegraded":"2024-04-25T00:09:35.168966+0000","last_fullsized":"2024-04-25T00:09:35.168966+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:31.168558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675880+0000","last_change":"2024-04-25T00:09:33.874649+0000","last_active":"2024-04-25T00:09:36.675880+0000","last_peered":"2024-04-25T00:09:36.675880+0000","last_clean":"2024-04-25T00:09:36.675880+0000","last_became_active":"2024-04-25T00:09:33.874370+0000","last_became_peered":"2024-04-25T00:09:33.874370+0000","last_unstale":"2024-04-25T00:09:36.675880+0000","last_undegraded":"2024-04-25T00:09:36.675880+0000","last_fullsized":"2024-04-25T00:09:36.675880+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:37.456036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676643+0000","last_change":"2024-04-25T00:09:32.865123+0000","last_active":"2024-04-25T00:09:36.676643+0000","last_peered":"2024-04-25T00:09:36.676643+0000","last_clean":"2024-04-25T00:09:36.676643+0000","last_became_active":"2024-04-25T00:09:32.864038+0000","last_became_peered":"2024-04-25T00:09:32.864038+0000","last_unstale":"2024-04-25T00:09:36.676643+0000","last_undegraded":"2024-04-25T00:09:36.676643+0000","last_fullsized":"2024-04-25T00:09:36.676643+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:26.788441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676401+0000","last_change":"2024-04-25T00:09:33.874638+0000","last_active":"2024-04-25T00:09:36.676401+0000","last_peered":"2024-04-25T00:09:36.676401+0000","last_clean":"2024-04-25T00:09:36.676401+0000","last_became_active":"2024-04-25T00:09:33.874363+0000","last_became_peered":"2024-04-25T00:09:33.874363+0000","last_unstale":"2024-04-25T00:09:36.676401+0000","last_undegraded":"2024-04-25T00:09:36.676401+0000","last_fullsized":"2024-04-25T00:09:36.676401+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:16.742522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675815+0000","last_change":"2024-04-25T00:09:32.871043+0000","last_active":"2024-04-25T00:09:36.675815+0000","last_peered":"2024-04-25T00:09:36.675815+0000","last_clean":"2024-04-25T00:09:36.675815+0000","last_became_active":"2024-04-25T00:09:32.870708+0000","last_became_peered":"2024-04-25T00:09:32.870708+0000","last_unstale":"2024-04-25T00:09:36.675815+0000","last_undegraded":"2024-04-25T00:09:36.675815+0000","last_fullsized":"2024-04-25T00:09:36.675815+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:37:50.624952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676536+0000","last_change":"2024-04-25T00:09:33.878329+0000","last_active":"2024-04-25T00:09:36.676536+0000","last_peered":"2024-04-25T00:09:36.676536+0000","last_clean":"2024-04-25T00:09:36.676536+0000","last_became_active":"2024-04-25T00:09:33.878062+0000","last_became_peered":"2024-04-25T00:09:33.878062+0000","last_unstale":"2024-04-25T00:09:36.676536+0000","last_undegraded":"2024-04-25T00:09:36.676536+0000","last_fullsized":"2024-04-25T00:09:36.676536+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:17.012330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.676367+0000","last_change":"2024-04-25T00:09:32.865066+0000","last_active":"2024-04-25T00:09:36.676367+0000","last_peered":"2024-04-25T00:09:36.676367+0000","last_clean":"2024-04-25T00:09:36.676367+0000","last_became_active":"2024-04-25T00:09:32.864793+0000","last_became_peered":"2024-04-25T00:09:32.864793+0000","last_unstale":"2024-04-25T00:09:36.676367+0000","last_undegraded":"2024-04-25T00:09:36.676367+0000","last_fullsized":"2024-04-25T00:09:36.676367+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:49.165483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675982+0000","last_change":"2024-04-25T00:09:33.877986+0000","last_active":"2024-04-25T00:09:36.675982+0000","last_peered":"2024-04-25T00:09:36.675982+0000","last_clean":"2024-04-25T00:09:36.675982+0000","last_became_active":"2024-04-25T00:09:33.877728+0000","last_became_peered":"2024-04-25T00:09:33.877728+0000","last_unstale":"2024-04-25T00:09:36.675982+0000","last_undegraded":"2024-04-25T00:09:36.675982+0000","last_fullsized":"2024-04-25T00:09:36.675982+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:31.654609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863732+0000","last_change":"2024-04-25T00:09:32.870566+0000","last_active":"2024-04-25T00:09:34.863732+0000","last_peered":"2024-04-25T00:09:34.863732+0000","last_clean":"2024-04-25T00:09:34.863732+0000","last_became_active":"2024-04-25T00:09:32.869700+0000","last_became_peered":"2024-04-25T00:09:32.869700+0000","last_unstale":"2024-04-25T00:09:34.863732+0000","last_undegraded":"2024-04-25T00:09:34.863732+0000","last_fullsized":"2024-04-25T00:09:34.863732+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:16.007479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168567+0000","last_change":"2024-04-25T00:09:33.878569+0000","last_active":"2024-04-25T00:09:35.168567+0000","last_peered":"2024-04-25T00:09:35.168567+0000","last_clean":"2024-04-25T00:09:35.168567+0000","last_became_active":"2024-04-25T00:09:33.878305+0000","last_became_peered":"2024-04-25T00:09:33.878305+0000","last_unstale":"2024-04-25T00:09:35.168567+0000","last_undegraded":"2024-04-25T00:09:35.168567+0000","last_fullsized":"2024-04-25T00:09:35.168567+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:37.456036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863824+0000","last_change":"2024-04-25T00:09:32.861027+0000","last_active":"2024-04-25T00:09:34.863824+0000","last_peered":"2024-04-25T00:09:34.863824+0000","last_clean":"2024-04-25T00:09:34.863824+0000","last_became_active":"2024-04-25T00:09:32.860724+0000","last_became_peered":"2024-04-25T00:09:32.860724+0000","last_unstale":"2024-04-25T00:09:34.863824+0000","last_undegraded":"2024-04-25T00:09:34.863824+0000","last_fullsized":"2024-04-25T00:09:34.863824+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:23.820088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863453+0000","last_change":"2024-04-25T00:09:33.866398+0000","last_active":"2024-04-25T00:09:34.863453+0000","last_peered":"2024-04-25T00:09:34.863453+0000","last_clean":"2024-04-25T00:09:34.863453+0000","last_became_active":"2024-04-25T00:09:33.866122+0000","last_became_peered":"2024-04-25T00:09:33.866122+0000","last_unstale":"2024-04-25T00:09:34.863453+0000","last_undegraded":"2024-04-25T00:09:34.863453+0000","last_fullsized":"2024-04-25T00:09:34.863453+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:27.159690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863217+0000","last_change":"2024-04-25T00:09:32.872369+0000","last_active":"2024-04-25T00:09:34.863217+0000","last_peered":"2024-04-25T00:09:34.863217+0000","last_clean":"2024-04-25T00:09:34.863217+0000","last_became_active":"2024-04-25T00:09:32.872083+0000","last_became_peered":"2024-04-25T00:09:32.872083+0000","last_unstale":"2024-04-25T00:09:34.863217+0000","last_undegraded":"2024-04-25T00:09:34.863217+0000","last_fullsized":"2024-04-25T00:09:34.863217+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:36.451185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874414+0000","last_change":"2024-04-25T00:09:33.874713+0000","last_active":"2024-04-25T00:09:34.874414+0000","last_peered":"2024-04-25T00:09:34.874414+0000","last_clean":"2024-04-25T00:09:34.874414+0000","last_became_active":"2024-04-25T00:09:33.874450+0000","last_became_peered":"2024-04-25T00:09:33.874450+0000","last_unstale":"2024-04-25T00:09:34.874414+0000","last_undegraded":"2024-04-25T00:09:34.874414+0000","last_fullsized":"2024-04-25T00:09:34.874414+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:59.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675924+0000","last_change":"2024-04-25T00:09:32.864284+0000","last_active":"2024-04-25T00:09:36.675924+0000","last_peered":"2024-04-25T00:09:36.675924+0000","last_clean":"2024-04-25T00:09:36.675924+0000","last_became_active":"2024-04-25T00:09:32.863870+0000","last_became_peered":"2024-04-25T00:09:32.863870+0000","last_unstale":"2024-04-25T00:09:36.675924+0000","last_undegraded":"2024-04-25T00:09:36.675924+0000","last_fullsized":"2024-04-25T00:09:36.675924+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:23.760066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.864187+0000","last_change":"2024-04-25T00:09:33.868631+0000","last_active":"2024-04-25T00:09:34.864187+0000","last_peered":"2024-04-25T00:09:34.864187+0000","last_clean":"2024-04-25T00:09:34.864187+0000","last_became_active":"2024-04-25T00:09:33.866936+0000","last_became_peered":"2024-04-25T00:09:33.866936+0000","last_unstale":"2024-04-25T00:09:34.864187+0000","last_undegraded":"2024-04-25T00:09:34.864187+0000","last_fullsized":"2024-04-25T00:09:34.864187+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:45.734090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"182'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.873583+0000","last_change":"2024-04-25T00:09:32.869135+0000","last_active":"2024-04-25T00:09:34.873583+0000","last_peered":"2024-04-25T00:09:34.873583+0000","last_clean":"2024-04-25T00:09:34.873583+0000","last_became_active":"2024-04-25T00:09:32.868833+0000","last_became_peered":"2024-04-25T00:09:32.868833+0000","last_unstale":"2024-04-25T00:09:34.873583+0000","last_undegraded":"2024-04-25T00:09:34.873583+0000","last_fullsized":"2024-04-25T00:09:34.873583+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:22.181677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168623+0000","last_change":"2024-04-25T00:09:33.869214+0000","last_active":"2024-04-25T00:09:35.168623+0000","last_peered":"2024-04-25T00:09:35.168623+0000","last_clean":"2024-04-25T00:09:35.168623+0000","last_became_active":"2024-04-25T00:09:33.868863+0000","last_became_peered":"2024-04-25T00:09:33.868863+0000","last_unstale":"2024-04-25T00:09:35.168623+0000","last_undegraded":"2024-04-25T00:09:35.168623+0000","last_fullsized":"2024-04-25T00:09:35.168623+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:48:31.654609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863433+0000","last_change":"2024-04-25T00:09:32.868078+0000","last_active":"2024-04-25T00:09:34.863433+0000","last_peered":"2024-04-25T00:09:34.863433+0000","last_clean":"2024-04-25T00:09:34.863433+0000","last_became_active":"2024-04-25T00:09:32.867779+0000","last_became_peered":"2024-04-25T00:09:32.867779+0000","last_unstale":"2024-04-25T00:09:34.863433+0000","last_undegraded":"2024-04-25T00:09:34.863433+0000","last_fullsized":"2024-04-25T00:09:34.863433+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:27.642272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863700+0000","last_change":"2024-04-25T00:09:33.871883+0000","last_active":"2024-04-25T00:09:34.863700+0000","last_peered":"2024-04-25T00:09:34.863700+0000","last_clean":"2024-04-25T00:09:34.863700+0000","last_became_active":"2024-04-25T00:09:33.871586+0000","last_became_peered":"2024-04-25T00:09:33.871586+0000","last_unstale":"2024-04-25T00:09:34.863700+0000","last_undegraded":"2024-04-25T00:09:34.863700+0000","last_fullsized":"2024-04-25T00:09:34.863700+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:29:08.604008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675879+0000","last_change":"2024-04-25T00:09:32.863484+0000","last_active":"2024-04-25T00:09:36.675879+0000","last_peered":"2024-04-25T00:09:36.675879+0000","last_clean":"2024-04-25T00:09:36.675879+0000","last_became_active":"2024-04-25T00:09:32.863226+0000","last_became_peered":"2024-04-25T00:09:32.863226+0000","last_unstale":"2024-04-25T00:09:36.675879+0000","last_undegraded":"2024-04-25T00:09:36.675879+0000","last_fullsized":"2024-04-25T00:09:36.675879+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:20.948313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863681+0000","last_change":"2024-04-25T00:09:33.876228+0000","last_active":"2024-04-25T00:09:34.863681+0000","last_peered":"2024-04-25T00:09:34.863681+0000","last_clean":"2024-04-25T00:09:34.863681+0000","last_became_active":"2024-04-25T00:09:33.875900+0000","last_became_peered":"2024-04-25T00:09:33.875900+0000","last_unstale":"2024-04-25T00:09:34.863681+0000","last_undegraded":"2024-04-25T00:09:34.863681+0000","last_fullsized":"2024-04-25T00:09:34.863681+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.901658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.872093+0000","last_change":"2024-04-25T00:09:32.870945+0000","last_active":"2024-04-25T00:09:34.872093+0000","last_peered":"2024-04-25T00:09:34.872093+0000","last_clean":"2024-04-25T00:09:34.872093+0000","last_became_active":"2024-04-25T00:09:32.870577+0000","last_became_peered":"2024-04-25T00:09:32.870577+0000","last_unstale":"2024-04-25T00:09:34.872093+0000","last_undegraded":"2024-04-25T00:09:34.872093+0000","last_fullsized":"2024-04-25T00:09:34.872093+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:55.965150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874401+0000","last_change":"2024-04-25T00:09:33.872463+0000","last_active":"2024-04-25T00:09:34.874401+0000","last_peered":"2024-04-25T00:09:34.874401+0000","last_clean":"2024-04-25T00:09:34.874401+0000","last_became_active":"2024-04-25T00:09:33.872181+0000","last_became_peered":"2024-04-25T00:09:33.872181+0000","last_unstale":"2024-04-25T00:09:34.874401+0000","last_undegraded":"2024-04-25T00:09:34.874401+0000","last_fullsized":"2024-04-25T00:09:34.874401+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:59.450004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863450+0000","last_change":"2024-04-25T00:09:32.872901+0000","last_active":"2024-04-25T00:09:34.863450+0000","last_peered":"2024-04-25T00:09:34.863450+0000","last_clean":"2024-04-25T00:09:34.863450+0000","last_became_active":"2024-04-25T00:09:32.872596+0000","last_became_peered":"2024-04-25T00:09:32.872596+0000","last_unstale":"2024-04-25T00:09:34.863450+0000","last_undegraded":"2024-04-25T00:09:34.863450+0000","last_fullsized":"2024-04-25T00:09:34.863450+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:24.471772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:36.675670+0000","last_change":"2024-04-25T00:09:33.864759+0000","last_active":"2024-04-25T00:09:36.675670+0000","last_peered":"2024-04-25T00:09:36.675670+0000","last_clean":"2024-04-25T00:09:36.675670+0000","last_became_active":"2024-04-25T00:09:33.864497+0000","last_became_peered":"2024-04-25T00:09:33.864497+0000","last_unstale":"2024-04-25T00:09:36.675670+0000","last_undegraded":"2024-04-25T00:09:36.675670+0000","last_fullsized":"2024-04-25T00:09:36.675670+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:20.376300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863516+0000","last_change":"2024-04-25T00:09:32.859351+0000","last_active":"2024-04-25T00:09:34.863516+0000","last_peered":"2024-04-25T00:09:34.863516+0000","last_clean":"2024-04-25T00:09:34.863516+0000","last_became_active":"2024-04-25T00:09:32.859068+0000","last_became_peered":"2024-04-25T00:09:32.859068+0000","last_unstale":"2024-04-25T00:09:34.863516+0000","last_undegraded":"2024-04-25T00:09:34.863516+0000","last_fullsized":"2024-04-25T00:09:34.863516+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:39.922934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:35.168534+0000","last_change":"2024-04-25T00:09:33.872393+0000","last_active":"2024-04-25T00:09:35.168534+0000","last_peered":"2024-04-25T00:09:35.168534+0000","last_clean":"2024-04-25T00:09:35.168534+0000","last_became_active":"2024-04-25T00:09:33.872069+0000","last_became_peered":"2024-04-25T00:09:33.872069+0000","last_unstale":"2024-04-25T00:09:35.168534+0000","last_undegraded":"2024-04-25T00:09:35.168534+0000","last_fullsized":"2024-04-25T00:09:35.168534+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:54.319661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874113+0000","last_change":"2024-04-25T00:09:32.860524+0000","last_active":"2024-04-25T00:09:34.874113+0000","last_peered":"2024-04-25T00:09:34.874113+0000","last_clean":"2024-04-25T00:09:34.874113+0000","last_became_active":"2024-04-25T00:09:32.860045+0000","last_became_peered":"2024-04-25T00:09:32.860045+0000","last_unstale":"2024-04-25T00:09:34.874113+0000","last_undegraded":"2024-04-25T00:09:34.874113+0000","last_fullsized":"2024-04-25T00:09:34.874113+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:24.471772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.863583+0000","last_change":"2024-04-25T00:09:33.865604+0000","last_active":"2024-04-25T00:09:34.863583+0000","last_peered":"2024-04-25T00:09:34.863583+0000","last_clean":"2024-04-25T00:09:34.863583+0000","last_became_active":"2024-04-25T00:09:33.865349+0000","last_became_peered":"2024-04-25T00:09:33.865349+0000","last_unstale":"2024-04-25T00:09:34.863583+0000","last_undegraded":"2024-04-25T00:09:34.863583+0000","last_fullsized":"2024-04-25T00:09:34.863583+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:32.845368+0000","last_clean_scrub_stamp":"2024-04-25T00:09:32.845368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:55.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.874500+0000","last_change":"2024-04-25T00:09:32.865204+0000","last_active":"2024-04-25T00:09:34.874500+0000","last_peered":"2024-04-25T00:09:34.874500+0000","last_clean":"2024-04-25T00:09:34.874500+0000","last_became_active":"2024-04-25T00:09:32.864901+0000","last_became_peered":"2024-04-25T00:09:32.864901+0000","last_unstale":"2024-04-25T00:09:34.874500+0000","last_undegraded":"2024-04-25T00:09:34.874500+0000","last_fullsized":"2024-04-25T00:09:34.874500+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:31.840517+0000","last_clean_scrub_stamp":"2024-04-25T00:09:31.840517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:27.642272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"153'161","reported_seq":515,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T00:09:34.871873+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:09:34.871873+0000","last_peered":"2024-04-25T00:09:34.871873+0000","last_clean":"2024-04-25T00:09:34.871873+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:09:34.871873+0000","last_undegraded":"2024-04-25T00:09:34.871873+0000","last_fullsized":"2024-04-25T00:09:34.871873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2899968,"data_stored":1597900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2297856,"data_stored":1006502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55640,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316200,"statfs":{"total":96636764160,"available":96579788800,"internally_reserved":0,"allocated":2297856,"data_stored":1006502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2314240,"data_stored":1007636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51480,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320360,"statfs":{"total":96636764160,"available":96584048640,"internally_reserved":0,"allocated":2306048,"data_stored":1006102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51468,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320372,"statfs":{"total":96636764160,"available":96584060928,"internally_reserved":0,"allocated":2293760,"data_stored":1005920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59804,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312036,"statfs":{"total":96636764160,"available":96575524864,"internally_reserved":0,"allocated":2301952,"data_stored":1005976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52060,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319780,"statfs":{"total":96636764160,"available":96583454720,"internally_reserved":0,"allocated":2891776,"data_stored":1596378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:09:41.514 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:09:41.514 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:09:41.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.654+0000 7f7190e41640 1 Processor -- start 2024-04-25T00:09:41.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.654+0000 7f7190e41640 1 -- start start 2024-04-25T00:09:41.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f7190e41640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f7190e41640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f718c15e6b0 0x7f718c0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f7190e41640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f718c15f4f0 con 0x7f718c15e2d0 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f7190e41640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f718c0cde00 con 0x7f718c15e6b0 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f7190e41640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f718c15f7d0 con 0x7f718c0b3080 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f718c15e6b0 0x7f718c0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f7189d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f718c15e6b0 0x7f718c0b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:55916/0 (socket says 172.21.15.113:55916) 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718ad9d640 1 -- 172.21.15.113:0/2743025091 learned_addr learned my addr 172.21.15.113:0/2743025091 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718a59c640 1 --1- 172.21.15.113:0/2743025091 >> v1:172.21.15.178:6789/0 conn(0x7f718c15e2d0 0x7f718c15ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:52174/0 (socket says 172.21.15.113:52174) 2024-04-25T00:09:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718ad9d640 1 -- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 msgr2=0x7f718c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718ad9d640 1 --2- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718ad9d640 1 -- 172.21.15.113:0/2743025091 >> v1:172.21.15.178:6789/0 conn(0x7f718c15e2d0 legacy=0x7f718c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f718ad9d640 1 -- 172.21.15.113:0/2743025091 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c15fab0 con 0x7f718c15e6b0 2024-04-25T00:09:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.655+0000 7f7189d9b640 1 --2- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.656+0000 7f718ad9d640 1 --2- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e6b0 0x7f718c0b4870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7184009e50 tx=0x7f718402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d97e311474405f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.656+0000 7f718959a640 1 -- 172.21.15.113:0/2743025091 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184036070 con 0x7f718c15e6b0 2024-04-25T00:09:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.656+0000 7f718959a640 1 -- 172.21.15.113:0/2743025091 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7184031600 con 0x7f718c15e6b0 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.656+0000 7f718959a640 1 -- 172.21.15.113:0/2743025091 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71840319d0 con 0x7f718c15e6b0 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 -- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e6b0 msgr2=0x7f718c0b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 --2- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e6b0 0x7f718c0b4870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7184009e50 tx=0x7f718402a680 comp rx=0 tx=0).stop 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 -- 172.21.15.113:0/2743025091 shutdown_connections 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 --2- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e6b0 0x7f718c0b4870 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 --2- 172.21.15.113:0/2743025091 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 -- 172.21.15.113:0/2743025091 >> 172.21.15.113:0/2743025091 conn(0x7f718c06b0f0 msgr2=0x7f718c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 -- 172.21.15.113:0/2743025091 shutdown_connections 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.657+0000 7f7190e41640 1 -- 172.21.15.113:0/2743025091 wait complete. 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.658+0000 7f7190e41640 1 Processor -- start 2024-04-25T00:09:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.658+0000 7f7190e41640 1 -- start start 2024-04-25T00:09:41.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f7190e41640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f7190e41640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e2d0 0x7f718c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f7190e41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c15e6b0 0x7f718c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f7190e41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f718c0ce410 con 0x7f718c15e6b0 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f7190e41640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f718c0ce290 con 0x7f718c15e2d0 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f7190e41640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f718c0ce590 con 0x7f718c0b3080 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c15e6b0 0x7f718c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c15e6b0 0x7f718c15c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43750/0 (socket says 172.21.15.113:43750) 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 -- 172.21.15.113:0/778370043 learned_addr learned my addr 172.21.15.113:0/778370043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718a59c640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f7189d9b640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e2d0 0x7f718c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 -- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 msgr2=0x7f718c147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 -- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e2d0 msgr2=0x7f718c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e2d0 0x7f718c15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.659+0000 7f718ad9d640 1 -- 172.21.15.113:0/778370043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c15b880 con 0x7f718c15e6b0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.660+0000 7f718ad9d640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c15e6b0 0x7f718c15c7a0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f7184009f70 tx=0x7f7184004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.660+0000 7f717affd640 1 -- 172.21.15.113:0/778370043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718403f020 con 0x7f718c15e6b0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.660+0000 7f717affd640 1 -- 172.21.15.113:0/778370043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7184037040 con 0x7f718c15e6b0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.660+0000 7f717affd640 1 -- 172.21.15.113:0/778370043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184036070 con 0x7f718c15e6b0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.660+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c0ae560 con 0x7f718c15e6b0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.660+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c0ae220 con 0x7f718c15e6b0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.661+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c0691a0 con 0x7f718c15e6b0 2024-04-25T00:09:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.661+0000 7f717affd640 1 -- 172.21.15.113:0/778370043 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7184042020 con 0x7f718c15e6b0 2024-04-25T00:09:41.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.662+0000 7f717affd640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f716c06be30 0x7f716c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.662+0000 7f718a59c640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f716c06be30 0x7f716c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.662+0000 7f717affd640 1 -- 172.21.15.113:0/778370043 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f71840ab0b0 con 0x7f718c15e6b0 2024-04-25T00:09:41.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.662+0000 7f718a59c640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f716c06be30 0x7f716c06e2f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f717c007fd0 tx=0x7f717c007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.666+0000 7f717affd640 1 -- 172.21.15.113:0/778370043 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7184049030 con 0x7f718c15e6b0 2024-04-25T00:09:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.907+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f718c15d8a0 con 0x7f718c15e6b0 2024-04-25T00:09:41.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.911+0000 7f717affd640 1 -- 172.21.15.113:0/778370043 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f71840794a0 con 0x7f718c15e6b0 2024-04-25T00:09:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-25T00:09:41.911 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBjnClmT1MvOBAAL/gotuDPQBlEBzTNlEkyvQ== 2024-04-25T00:09:41.911 INFO:teuthology.orchestra.run.smithi113.stdout: caps mds = "allow" 2024-04-25T00:09:41.911 INFO:teuthology.orchestra.run.smithi113.stdout: caps mon = "allow r" 2024-04-25T00:09:41.911 INFO:teuthology.orchestra.run.smithi113.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:09:41.912 INFO:teuthology.orchestra.run.smithi113.stderr:updated caps for client.0 2024-04-25T00:09:41.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f716c06be30 msgr2=0x7f716c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f716c06be30 0x7f716c06e2f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f717c007fd0 tx=0x7f717c007e80 comp rx=0 tx=0).stop 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c15e6b0 msgr2=0x7f718c15c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c15e6b0 0x7f718c15c7a0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f7184009f70 tx=0x7f7184004a40 comp rx=0 tx=0).stop 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 shutdown_connections 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f716c06be30 0x7f716c06e2f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f718c15e2d0 0x7f718c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c15e6b0 0x7f718c15c7a0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 --2- 172.21.15.113:0/778370043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f718c0b3080 0x7f718c147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 >> 172.21.15.113:0/778370043 conn(0x7f718c06b0f0 msgr2=0x7f718c0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 shutdown_connections 2024-04-25T00:09:41.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:41.913+0000 7f7190e41640 1 -- 172.21.15.113:0/778370043 wait complete. 2024-04-25T00:09:41.924 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.020+0000 7f9e6253b640 1 Processor -- start 2024-04-25T00:09:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.020+0000 7f9e6253b640 1 -- start start 2024-04-25T00:09:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.020+0000 7f9e6253b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9e5c0b3fd0 0x7f9e5c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.020+0000 7f9e6253b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5c144cf0 con 0x7f9e5c144910 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.020+0000 7f9e6253b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5c0b2f30 con 0x7f9e5c06e510 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.020+0000 7f9e6253b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5c06e980 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9e5c0b3fd0 0x7f9e5c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9e5c144910 0x7f9e5c0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60732/0 (socket says 172.21.15.113:60732) 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5bfff640 1 -- 172.21.15.113:0/10689289 learned_addr learned my addr 172.21.15.113:0/10689289 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5b7fe640 1 -- 172.21.15.113:0/10689289 >> v1:172.21.15.178:6789/0 conn(0x7f9e5c06e510 legacy=0x7f9e5c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5b7fe640 1 -- 172.21.15.113:0/10689289 >> v1:172.21.15.113:6789/0 conn(0x7f9e5c144910 legacy=0x7f9e5c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5b7fe640 1 -- 172.21.15.113:0/10689289 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c0b3210 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5affd640 1 -- 172.21.15.113:0/10689289 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47410578 0 0) 0x7f9e5c144cf0 con 0x7f9e5c144910 2024-04-25T00:09:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.021+0000 7f9e5b7fe640 1 --2- 172.21.15.113:0/10689289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c06dfd0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0097d0 tx=0x7f9e4c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b12ffdc4c6d91ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e5affd640 1 -- 172.21.15.113:0/10689289 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c037070 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e5affd640 1 -- 172.21.15.113:0/10689289 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c032440 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e5affd640 1 -- 172.21.15.113:0/10689289 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c032810 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e6253b640 1 -- 172.21.15.113:0/10689289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 msgr2=0x7f9e5c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e6253b640 1 --2- 172.21.15.113:0/10689289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c06dfd0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0097d0 tx=0x7f9e4c02a210 comp rx=0 tx=0).stop 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e6253b640 1 -- 172.21.15.113:0/10689289 shutdown_connections 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e6253b640 1 --2- 172.21.15.113:0/10689289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c06dfd0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e6253b640 1 -- 172.21.15.113:0/10689289 >> 172.21.15.113:0/10689289 conn(0x7f9e5c0b1a50 msgr2=0x7f9e5c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e6253b640 1 -- 172.21.15.113:0/10689289 shutdown_connections 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.022+0000 7f9e6253b640 1 -- 172.21.15.113:0/10689289 wait complete. 2024-04-25T00:09:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.023+0000 7f9e6253b640 1 Processor -- start 2024-04-25T00:09:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.023+0000 7f9e6253b640 1 -- start start 2024-04-25T00:09:42.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e6253b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e5c06e510 0x7f9e5c13f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e6253b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e6253b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e5c144910 0x7f9e5c0ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e6253b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c0b4440 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e6253b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c0b37d0 con 0x7f9e5c144910 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e6253b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c0b4da0 con 0x7f9e5c06e510 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e5b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e5b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c0719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43772/0 (socket says 172.21.15.113:43772) 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.024+0000 7f9e5b7fe640 1 -- 172.21.15.113:0/220820606 learned_addr learned my addr 172.21.15.113:0/220820606 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e5b7fe640 1 -- 172.21.15.113:0/220820606 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e5c06e510 msgr2=0x7f9e5c13f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e5b7fe640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e5c06e510 0x7f9e5c13f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e5b7fe640 1 -- 172.21.15.113:0/220820606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e5c144910 msgr2=0x7f9e5c0ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e5b7fe640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e5c144910 0x7f9e5c0ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e5b7fe640 1 -- 172.21.15.113:0/220820606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c0722f0 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e5b7fe640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c0719f0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0097d0 tx=0x7f9e4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e58ff9640 1 -- 172.21.15.113:0/220820606 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c037040 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e58ff9640 1 -- 172.21.15.113:0/220820606 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c004250 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c0725d0 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e58ff9640 1 -- 172.21.15.113:0/220820606 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c03c730 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.025+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c072b90 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.026+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5c0691a0 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.026+0000 7f9e58ff9640 1 -- 172.21.15.113:0/220820606 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e4c03a070 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.027+0000 7f9e58ff9640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4806be30 0x7f9e4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.027+0000 7f9e58ff9640 1 -- 172.21.15.113:0/220820606 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f9e4c0ab780 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.027+0000 7f9e5bfff640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4806be30 0x7f9e4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.027+0000 7f9e5bfff640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4806be30 0x7f9e4806e2f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007fd0 tx=0x7f9e50007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.031+0000 7f9e58ff9640 1 -- 172.21.15.113:0/220820606 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9e4c041020 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.241+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e5c0731a0 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.242+0000 7f9e58ff9640 1 -- 172.21.15.113:0/220820606 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5204 (secure 0 0 0) 0x7f9e4c07a130 con 0x7f9e5c0b3fd0 2024-04-25T00:09:42.243 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":98,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":10794,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:35.199346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10803":{"gid":10803,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/3057967443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3057967443},{"type":"v1","addr":"172.21.15.178:6835","nonce":3057967443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10803,"qdb_cluster":[10803]},"id":13}]} 2024-04-25T00:09:42.243 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 98 2024-04-25T00:09:42.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4806be30 msgr2=0x7f9e4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4806be30 0x7f9e4806e2f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007fd0 tx=0x7f9e50007960 comp rx=0 tx=0).stop 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 msgr2=0x7f9e5c0719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c0719f0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0097d0 tx=0x7f9e4c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 shutdown_connections 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f9e4806be30 0x7f9e4806e2f0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e5c144910 0x7f9e5c0ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5c0b3fd0 0x7f9e5c0719f0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 --2- 172.21.15.113:0/220820606 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e5c06e510 0x7f9e5c13f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 >> 172.21.15.113:0/220820606 conn(0x7f9e5c0b1a50 msgr2=0x7f9e5c0cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 shutdown_connections 2024-04-25T00:09:42.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.244+0000 7f9e6253b640 1 -- 172.21.15.113:0/220820606 wait complete. 2024-04-25T00:09:42.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:09:33.215001+0000', 'modified': '2024-04-25T00:09:35.199346+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10803}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10803': {'gid': 10803, 'name': 'd', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6835/3057967443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6833', 'nonce': 3057967443}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 3057967443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10803, 'qdb_cluster': [10803]} 2024-04-25T00:09:42.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:09:42.658 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T00:09:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.758+0000 7f0fa09a0640 1 Processor -- start 2024-04-25T00:09:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.758+0000 7f0fa09a0640 1 -- start start 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.758+0000 7f0fa09a0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0f9c15e310 0x7f0f9c06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.758+0000 7f0fa09a0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f9c06bb00 0x7f0f9c0d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.758+0000 7f0fa09a0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0d1f10 0x7f0f9c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0fa09a0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9c0d1140 con 0x7f0f9c06bb00 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0fa09a0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9c0b1650 con 0x7f0f9c15e310 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0fa09a0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c15e6f0 con 0x7f0f9c0d1f10 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f9c06bb00 0x7f0f9c0d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f9c06bb00 0x7f0f9c0d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43774/0 (socket says 172.21.15.113:43774) 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 -- 172.21.15.113:0/3028444722 learned_addr learned my addr 172.21.15.113:0/3028444722 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/3028444722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0d1f10 0x7f0f9c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/3028444722 >> v2:172.21.15.178:3300/0 conn(0x7f0f9c15e310 0x7f0f9c06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 -- 172.21.15.113:0/3028444722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0d1f10 msgr2=0x7f0f9c0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 --2- 172.21.15.113:0/3028444722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0d1f10 0x7f0f9c0ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 -- 172.21.15.113:0/3028444722 >> v2:172.21.15.178:3300/0 conn(0x7f0f9c15e310 msgr2=0x7f0f9c06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 --2- 172.21.15.113:0/3028444722 >> v2:172.21.15.178:3300/0 conn(0x7f0f9c15e310 0x7f0f9c06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.759+0000 7f0f9affd640 1 -- 172.21.15.113:0/3028444722 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c06b840 con 0x7f0f9c06bb00 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0f9affd640 1 --2- 172.21.15.113:0/3028444722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c06bb00 0x7f0f9c0d19d0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c06e920 tx=0x7f0f9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93acf01eaedf6756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0f9a7fc640 1 -- 172.21.15.113:0/3028444722 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90037070 con 0x7f0f9c06bb00 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0f9a7fc640 1 -- 172.21.15.113:0/3028444722 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f90032440 con 0x7f0f9c06bb00 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0f9a7fc640 1 -- 172.21.15.113:0/3028444722 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90032810 con 0x7f0f9c06bb00 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3028444722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c06bb00 msgr2=0x7f0f9c0d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3028444722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c06bb00 0x7f0f9c0d19d0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c06e920 tx=0x7f0f9002a270 comp rx=0 tx=0).stop 2024-04-25T00:09:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3028444722 shutdown_connections 2024-04-25T00:09:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3028444722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0d1f10 0x7f0f9c0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3028444722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c06bb00 0x7f0f9c0d19d0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3028444722 >> v2:172.21.15.178:3300/0 conn(0x7f0f9c15e310 0x7f0f9c06b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3028444722 >> 172.21.15.113:0/3028444722 conn(0x7f0f9c09aa20 msgr2=0x7f0f9c09ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3028444722 shutdown_connections 2024-04-25T00:09:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.760+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3028444722 wait complete. 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 Processor -- start 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 -- start start 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f9c0d1f10 0x7f0f9c1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c15e310 0x7f0f9c09c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0b5810 0x7f0f9c160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c0d0af0 con 0x7f0f9c15e310 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c06c270 con 0x7f0f9c0d1f10 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.762+0000 7f0fa09a0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c0d0dd0 con 0x7f0f9c0b5810 2024-04-25T00:09:42.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c15e310 0x7f0f9c09c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c15e310 0x7f0f9c09c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43788/0 (socket says 172.21.15.113:43788) 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 -- 172.21.15.113:0/3494859085 learned_addr learned my addr 172.21.15.113:0/3494859085 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0b5810 0x7f0f9c160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f9c0d1f10 0x7f0f9c1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 -- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0b5810 msgr2=0x7f0f9c160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0b5810 0x7f0f9c160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 -- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f9c0d1f10 msgr2=0x7f0f9c1f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f9c0d1f10 0x7f0f9c1f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 -- 172.21.15.113:0/3494859085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c1f4b40 con 0x7f0f9c15e310 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0b5810 0x7f0f9c160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f9affd640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c15e310 0x7f0f9c09c1a0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f0f9002a2b0 tx=0x7f0f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f83fff640 1 -- 172.21.15.113:0/3494859085 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90041020 con 0x7f0f9c15e310 2024-04-25T00:09:42.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f83fff640 1 -- 172.21.15.113:0/3494859085 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f900306c0 con 0x7f0f9c15e310 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0f83fff640 1 -- 172.21.15.113:0/3494859085 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90037040 con 0x7f0f9c15e310 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c0b5dd0 con 0x7f0f9c15e310 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.763+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c0b6390 con 0x7f0f9c15e310 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.765+0000 7f0f83fff640 1 -- 172.21.15.113:0/3494859085 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f90044020 con 0x7f0f9c15e310 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.765+0000 7f0f83fff640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f8806be60 0x7f0f8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.765+0000 7f0f83fff640 1 -- 172.21.15.113:0/3494859085 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f0f900ab660 con 0x7f0f9c15e310 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.765+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f8806be60 0x7f0f8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.766+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f8806be60 0x7f0f8806e320 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006fd0 tx=0x7f0f8c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.766+0000 7f0fa49ea740 1 Processor -- start 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.766+0000 7f0fa49ea740 1 -- start start 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7677200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677740 0x5644a764c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5644a75aff00 con 0x5644a7694d60 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644a7558750 con 0x5644a7677740 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5644a75abec0 con 0x5644a7677e80 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43798/0 (socket says 172.21.15.113:43798) 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 -- 172.21.15.113:0/3241724827 learned_addr learned my addr 172.21.15.113:0/3241724827 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9affd640 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7677200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677740 0x5644a764c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 -- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 msgr2=0x5644a7677200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7677200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 -- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677740 msgr2=0x5644a764c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677740 0x5644a764c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.767+0000 7f0f9b7fe640 1 -- 172.21.15.113:0/3241724827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a72c42d0 con 0x5644a7694d60 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76476f0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00fa80 tx=0x7f0f8c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678bb841edaebb89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0f817fa640 1 -- 172.21.15.113:0/3241724827 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c01b050 con 0x5644a7694d60 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0f817fa640 1 -- 172.21.15.113:0/3241724827 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8c025070 con 0x5644a7694d60 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0f817fa640 1 -- 172.21.15.113:0/3241724827 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c014de0 con 0x5644a7694d60 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 -- 172.21.15.113:0/3241724827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 msgr2=0x5644a76476f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76476f0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00fa80 tx=0x7f0f8c012000 comp rx=0 tx=0).stop 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 -- 172.21.15.113:0/3241724827 shutdown_connections 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677740 0x5644a764c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7677200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/3241724827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76476f0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 -- 172.21.15.113:0/3241724827 >> 172.21.15.113:0/3241724827 conn(0x5644a7649960 msgr2=0x5644a7692200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 -- 172.21.15.113:0/3241724827 shutdown_connections 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.768+0000 7f0fa49ea740 1 -- 172.21.15.113:0/3241724827 wait complete. 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.769+0000 7f0fa49ea740 1 -- start start 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.769+0000 7f0fa49ea740 1 Processor -- start 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677e80 0x5644a7679820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7694d60 0x5644a7693330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a768a3c0 0x5644a7693870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5644a764bb10 con 0x5644a768a3c0 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644a756bcf0 con 0x5644a7677e80 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5644a764af00 con 0x5644a7694d60 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 10 client.0 _ll_register_callbacks cb 0x5644a756b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a768a3c0 0x5644a7693870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a768a3c0 0x5644a7693870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43800/0 (socket says 172.21.15.113:43800) 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 -- 172.21.15.113:0/2231006842 learned_addr learned my addr 172.21.15.113:0/2231006842 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9affd640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7694d60 0x5644a7693330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677e80 0x5644a7679820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 -- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7694d60 msgr2=0x5644a7693330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7694d60 0x5644a7693330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 -- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677e80 msgr2=0x5644a7679820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677e80 0x5644a7679820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 -- 172.21.15.113:0/2231006842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a738e6b0 con 0x5644a768a3c0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9affd640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7694d60 0x5644a7693330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a768a3c0 0x5644a7693870 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f0f94007b50 tx=0x7f0f940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c505d0dbed924e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/2231006842 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9401b020 con 0x5644a768a3c0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/2231006842 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f0f94007c40 con 0x5644a768a3c0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.770+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8c025070 con 0x5644a768a3c0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/2231006842 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f0f94025050 con 0x5644a768a3c0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 10 client.10953 fetch_fsmap learned FSMap version 98 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x5644a76a3390 con 0x5644a768a3c0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/2231006842 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8c025070 con 0x5644a768a3c0 2024-04-25T00:09:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/2231006842 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f0f94028020 con 0x5644a768a3c0 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 10 client.10953 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 10 client.10953 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 20 client.10953 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a76788d0 0x5644a76a2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 4 client.10953 mds_command: new command op to 10803 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 --> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644a75bd440 con 0x5644a76788d0 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a76788d0 0x5644a76a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0f7a7fc640 10 client.10953 ms_handle_connect on v2:172.21.15.178:6833/3057967443 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.771+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a76788d0 0x5644a76a2990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7217d81b2f0cea server_cookie=d5be43b1a5a6d18b in_seq=0 out_seq=0 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/2231006842 <== mds.0 v2:172.21.15.178:6833/3057967443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644a75bd440 con 0x5644a76788d0 2024-04-25T00:09:42.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0f7a7fc640 10 client.10953 handle_command_reply: tid=0 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 client.10953 shutdown 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a768a3c0 msgr2=0x5644a7693870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a768a3c0 0x5644a7693870 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f0f94007b50 tx=0x7f0f940115a0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 shutdown_connections 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a768a3c0 0x5644a7693870 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7694d60 0x5644a7693330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a76788d0 0x5644a76a2990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/2231006842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a7677e80 0x5644a7679820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0f7a7fc640 0 client.10953 ms_handle_reset on v2:172.21.15.178:6833/3057967443 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.772+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 >> 172.21.15.113:0/2231006842 conn(0x5644a7677740 msgr2=0x5644a7648430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.773+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 shutdown_connections 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.773+0000 7f0fa49ea740 1 -- 172.21.15.113:0/2231006842 wait complete. 2024-04-25T00:09:42.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.773+0000 7f0fa49ea740 20 client.10953 trim_cache size 0 max 16384 2024-04-25T00:09:42.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.790+0000 7f0fa49ea740 1 Processor -- start 2024-04-25T00:09:42.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.790+0000 7f0fa49ea740 1 -- start start 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7689670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1ff720 con 0x5644a7694d60 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1f7a80 con 0x5644a768a3c0 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c1ff2e0 con 0x5644a7677e80 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43814/0 (socket says 172.21.15.113:43814) 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.791+0000 7f0f9bfff640 1 -- 172.21.15.113:0/290128597 learned_addr learned my addr 172.21.15.113:0/290128597 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7689670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9affd640 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9bfff640 1 -- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 msgr2=0x5644a7689670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7689670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9bfff640 1 -- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 msgr2=0x5644a7646fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9bfff640 1 -- 172.21.15.113:0/290128597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a7678fc0 con 0x5644a7694d60 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7689670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474f0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f0f94016550 tx=0x7f0f94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd49c7c818b4d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f80ff9640 1 -- 172.21.15.113:0/290128597 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9402e070 con 0x5644a7694d60 2024-04-25T00:09:42.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f80ff9640 1 -- 172.21.15.113:0/290128597 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94016870 con 0x5644a7694d60 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0f80ff9640 1 -- 172.21.15.113:0/290128597 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9401b050 con 0x5644a7694d60 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0fa49ea740 1 -- 172.21.15.113:0/290128597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 msgr2=0x5644a76474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.792+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474f0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f0f94016550 tx=0x7f0f94017000 comp rx=0 tx=0).stop 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 -- 172.21.15.113:0/290128597 shutdown_connections 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474f0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/290128597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7689670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 -- 172.21.15.113:0/290128597 >> 172.21.15.113:0/290128597 conn(0x5644a7677740 msgr2=0x5644a7693110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 -- 172.21.15.113:0/290128597 shutdown_connections 2024-04-25T00:09:42.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 -- 172.21.15.113:0/290128597 wait complete. 2024-04-25T00:09:42.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 -- start start 2024-04-25T00:09:42.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.793+0000 7f0fa49ea740 1 Processor -- start 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7696050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0fa49ea740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5644a7678750 con 0x5644a7694d60 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644a7678370 con 0x5644a768a3c0 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0fa49ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5644a7678da0 con 0x5644a7677e80 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0fa49ea740 10 client.0 _ll_register_callbacks cb 0x7f0f94007680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43828/0 (socket says 172.21.15.113:43828) 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 -- 172.21.15.113:0/1080114070 learned_addr learned my addr 172.21.15.113:0/1080114070 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7696050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9affd640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 -- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 msgr2=0x5644a7696050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7696050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 -- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 msgr2=0x5644a7646f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 -- 172.21.15.113:0/1080114070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a76940c0 con 0x5644a7694d60 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9affd640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f9bfff640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474d0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f0f9402c040 tx=0x7f0f94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528e9f8258db3e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.794+0000 7f0f7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/1080114070 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9402e070 con 0x5644a7694d60 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/1080114070 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f0f94035ac0 con 0x5644a7694d60 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f94016870 con 0x5644a7694d60 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/1080114070 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f0f9401b040 con 0x5644a7694d60 2024-04-25T00:09:42.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 10 client.10959 fetch_fsmap learned FSMap version 98 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x5644a76a3a90 con 0x5644a7694d60 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/1080114070 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94016870 con 0x5644a7694d60 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/1080114070 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f0f94028040 con 0x5644a7694d60 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 10 client.10959 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 10 client.10959 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 20 client.10959 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a770c760 0x5644a7687b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 4 client.10959 mds_command: new command op to 10803 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 --> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] -- command(tid 0: {"prefix": "status"}) -- 0x5644a75b7790 con 0x5644a770c760 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.795+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a770c760 0x5644a7687b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0f7a7fc640 10 client.10959 ms_handle_connect on v2:172.21.15.178:6833/3057967443 2024-04-25T00:09:42.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0f9b7fe640 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a770c760 0x5644a7687b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32c1e6b5e51c89f9 server_cookie=fcbced1bbbf5b37a in_seq=0 out_seq=0 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0f7a7fc640 1 -- 172.21.15.113:0/1080114070 <== mds.0 v2:172.21.15.178:6833/3057967443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5644a75b7790 con 0x5644a770c760 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0f7a7fc640 10 client.10959 handle_command_reply: tid=0 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 client.10959 shutdown 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 msgr2=0x5644a76474d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474d0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f0f9402c040 tx=0x7f0f94004a40 comp rx=0 tx=0).stop 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 shutdown_connections 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:6833/3057967443,v1:172.21.15.178:6835/3057967443] conn(0x5644a770c760 0x5644a7687b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644a768a3c0 0x5644a7646f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0f7a7fc640 0 client.10959 ms_handle_reset on v2:172.21.15.178:6833/3057967443 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5644a7694d60 0x5644a76474d0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 --2- 172.21.15.113:0/1080114070 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5644a7677e80 0x5644a7696050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 >> 172.21.15.113:0/1080114070 conn(0x5644a7677740 msgr2=0x5644a7646580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 shutdown_connections 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 1 -- 172.21.15.113:0/1080114070 wait complete. 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa49ea740 20 client.10959 trim_cache size 0 max 16384 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 10803, 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:09:42.804 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:09:42.805 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 8.9327674879999996, 2024-04-25T00:09:42.805 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 98, 2024-04-25T00:09:42.805 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 182, 2024-04-25T00:09:42.805 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 182, 2024-04-25T00:09:42.805 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 14.007059803000001 2024-04-25T00:09:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:42.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f8806be60 msgr2=0x7f0f8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f8806be60 0x7f0f8806e320 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006fd0 tx=0x7f0f8c008040 comp rx=0 tx=0).stop 2024-04-25T00:09:42.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c15e310 msgr2=0x7f0f9c09c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.796+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c15e310 0x7f0f9c09c1a0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f0f9002a2b0 tx=0x7f0f900046c0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 shutdown_connections 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f0f8806be60 0x7f0f8806e320 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0b5810 0x7f0f9c160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f9c15e310 0x7f0f9c09c1a0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 --2- 172.21.15.113:0/3494859085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f9c0d1f10 0x7f0f9c1f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 >> 172.21.15.113:0/3494859085 conn(0x7f0f9c09aa20 msgr2=0x7f0f9c070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 shutdown_connections 2024-04-25T00:09:42.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.797+0000 7f0fa09a0640 1 -- 172.21.15.113:0/3494859085 wait complete. 2024-04-25T00:09:42.808 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:09:42.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.906+0000 7fcb69adf640 1 Processor -- start 2024-04-25T00:09:42.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.906+0000 7fcb69adf640 1 -- start start 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb69adf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb64144910 0x7fcb640b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb69adf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb640cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb69adf640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb64144cf0 con 0x7fcb640b5240 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb69adf640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcb640b4510 con 0x7fcb64144910 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb69adf640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcb640b2f30 con 0x7fcb640b56b0 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb637fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb64144910 0x7fcb640b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb637fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb64144910 0x7fcb640b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43830/0 (socket says 172.21.15.113:43830) 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb637fe640 1 -- 172.21.15.113:0/2191298340 learned_addr learned my addr 172.21.15.113:0/2191298340 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb63fff640 1 --2- 172.21.15.113:0/2191298340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb640cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb637fe640 1 -- 172.21.15.113:0/2191298340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 msgr2=0x7fcb640cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb637fe640 1 --2- 172.21.15.113:0/2191298340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb640cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb637fe640 1 -- 172.21.15.113:0/2191298340 >> v1:172.21.15.178:6789/0 conn(0x7fcb640b5240 legacy=0x7fcb6406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.907+0000 7fcb637fe640 1 -- 172.21.15.113:0/2191298340 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb640b3210 con 0x7fcb64144910 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb637fe640 1 --2- 172.21.15.113:0/2191298340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640b3fd0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400bb40 tx=0x7fcb5402c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7bada3b99b8ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb627fc640 1 -- 172.21.15.113:0/2191298340 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54036020 con 0x7fcb64144910 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb627fc640 1 -- 172.21.15.113:0/2191298340 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54032820 con 0x7fcb64144910 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb627fc640 1 -- 172.21.15.113:0/2191298340 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54032bf0 con 0x7fcb64144910 2024-04-25T00:09:42.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 -- 172.21.15.113:0/2191298340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 msgr2=0x7fcb640b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 --2- 172.21.15.113:0/2191298340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640b3fd0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400bb40 tx=0x7fcb5402c960 comp rx=0 tx=0).stop 2024-04-25T00:09:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 -- 172.21.15.113:0/2191298340 shutdown_connections 2024-04-25T00:09:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 --2- 172.21.15.113:0/2191298340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb640cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 --2- 172.21.15.113:0/2191298340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640b3fd0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 -- 172.21.15.113:0/2191298340 >> 172.21.15.113:0/2191298340 conn(0x7fcb640b1a50 msgr2=0x7fcb640b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 -- 172.21.15.113:0/2191298340 shutdown_connections 2024-04-25T00:09:42.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.908+0000 7fcb69adf640 1 -- 172.21.15.113:0/2191298340 wait complete. 2024-04-25T00:09:42.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.909+0000 7fcb69adf640 1 Processor -- start 2024-04-25T00:09:42.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb69adf640 1 -- start start 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb69adf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb640b5240 0x7fcb641f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb69adf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb64140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb69adf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb69adf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcb640b4da0 con 0x7fcb64144910 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb69adf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb640b37d0 con 0x7fcb640b5240 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb69adf640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcb640b4f20 con 0x7fcb640b56b0 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.910+0000 7fcb63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43838/0 (socket says 172.21.15.113:43838) 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 -- 172.21.15.113:0/1524564834 learned_addr learned my addr 172.21.15.113:0/1524564834 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb62ffd640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb64140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb637fe640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb640b5240 0x7fcb641f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 -- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 msgr2=0x7fcb64140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb64140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 -- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb640b5240 msgr2=0x7fcb641f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb640b5240 0x7fcb641f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 -- 172.21.15.113:0/1524564834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb641f8450 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb63fff640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640c6650 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00efc0 tx=0x7fcb5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb4bfff640 1 -- 172.21.15.113:0/1524564834 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c018020 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb4bfff640 1 -- 172.21.15.113:0/1524564834 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5c00cd40 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb4bfff640 1 -- 172.21.15.113:0/1524564834 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0146c0 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb640c5110 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.911+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb640c59b0 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.912+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb640691a0 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.912+0000 7fcb4bfff640 1 -- 172.21.15.113:0/1524564834 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb5c005d10 con 0x7fcb64144910 2024-04-25T00:09:42.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.913+0000 7fcb4bfff640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb5006be30 0x7fcb5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:42.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.913+0000 7fcb637fe640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb5006be30 0x7fcb5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:42.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.913+0000 7fcb4bfff640 1 -- 172.21.15.113:0/1524564834 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7fcb5c08bd50 con 0x7fcb64144910 2024-04-25T00:09:42.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.913+0000 7fcb637fe640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb5006be30 0x7fcb5006e2f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcb54037000 tx=0x7fcb54032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:42.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:42.917+0000 7fcb4bfff640 1 -- 172.21.15.113:0/1524564834 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb5c08f030 con 0x7fcb64144910 2024-04-25T00:09:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.129+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcb641417f0 con 0x7fcb64144910 2024-04-25T00:09:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.129+0000 7fcb4bfff640 1 -- 172.21.15.113:0/1524564834 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7fcb5c022030 con 0x7fcb64144910 2024-04-25T00:09:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:09:43.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb5006be30 msgr2=0x7fcb5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:43.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb5006be30 0x7fcb5006e2f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcb54037000 tx=0x7fcb54032000 comp rx=0 tx=0).stop 2024-04-25T00:09:43.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 msgr2=0x7fcb640c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640c6650 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00efc0 tx=0x7fcb5c00c490 comp rx=0 tx=0).stop 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 shutdown_connections 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fcb5006be30 0x7fcb5006e2f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb64144910 0x7fcb640c6650 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb640b56b0 0x7fcb64140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 --2- 172.21.15.113:0/1524564834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb640b5240 0x7fcb641f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 >> 172.21.15.113:0/1524564834 conn(0x7fcb640b1a50 msgr2=0x7fcb640b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 shutdown_connections 2024-04-25T00:09:43.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43.131+0000 7fcb69adf640 1 -- 172.21.15.113:0/1524564834 wait complete. 2024-04-25T00:09:43.142 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:09:43.143 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:09:43.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.143 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-25T00:09:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:09:43.158 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:09:43.158 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:09:43.158 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:43.159 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::4efb:f657:2c4d:722/64 scope link noprefixroute 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d882:32ff:fea7:771f/64 scope link 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 6e:16:d5:b9:ea:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6c16:d5ff:feb9:ea84/64 scope link 2024-04-25T00:09:43.160 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:09:43.161 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:99:bb:14:16:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:09:43.161 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5899:bbff:fe14:165a/64 scope link 2024-04-25T00:09:43.161 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:09:43.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.161 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:09:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-25T00:09:43.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.223 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-25T00:09:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-25T00:09:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:09:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:09:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:09:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:09:43.279 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:09:43.279 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-25T00:09:43.279 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:09:43.279 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:09:43.279 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:09:43.279 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-25T00:09:43.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:09:43.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:09:43.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.401 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:09:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:09:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:09:43.415 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:09:43.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.416 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-25T00:09:43.478 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:09:43.479 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:09:43.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:09:43.534 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:43.588 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T00:09:43.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.588 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:09:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:09:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:09:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:09:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:09:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:09:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T00:09:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T00:09:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-25T00:09:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-25T00:09:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4eaaf81-6120-4e3f-a50c-97aa81453142" will pass to kernel 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:09:43.683 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:09:43.685 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:09:43.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:43.748 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:09:43.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:09:43.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:09:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:client8635 2024-04-25T00:09:43.845 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:43.845 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:43.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.860 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-25T00:09:43.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:43.918 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-25T00:09:43.974 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:43.975 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:44.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:44.031 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T00:09:44.116 INFO:teuthology.orchestra.run.smithi113.stderr:6+0 records in 2024-04-25T00:09:44.116 INFO:teuthology.orchestra.run.smithi113.stderr:6+0 records out 2024-04-25T00:09:44.116 INFO:teuthology.orchestra.run.smithi113.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0303313 s, 207 MB/s 2024-04-25T00:09:44.117 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:44.117 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:44.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:44.132 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T00:09:44.216 INFO:teuthology.orchestra.run.smithi113.stderr:6+0 records in 2024-04-25T00:09:44.216 INFO:teuthology.orchestra.run.smithi113.stderr:6+0 records out 2024-04-25T00:09:44.216 INFO:teuthology.orchestra.run.smithi113.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0284608 s, 221 MB/s 2024-04-25T00:09:44.218 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:09:44.259 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:09:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-25T00:09:44.297 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-25T00:09:44.297 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:09:44.297 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-25T00:09:44.297 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-25T00:09:44.298 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:09:44.299 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.301 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.302 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-25T00:09:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.304 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-25T00:09:44.305 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-25T00:09:44.306 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: mon down mkfs grace = 300 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.307 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: mds cache max size = 1000 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-04-25T00:09:44.308 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = False 2024-04-25T00:09:44.309 INFO:teuthology.orchestra.run.smithi113.stdout: mds log max segments = 1 2024-04-25T00:09:44.309 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-04-25T00:09:44.309 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = False 2024-04-25T00:09:44.309 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:09:44.309 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.309 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.309 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-25T00:09:44.310 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-25T00:09:44.311 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-25T00:09:44.312 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.313 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-25T00:09:44.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-25T00:09:44.315 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.316 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-25T00:09:44.317 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.318 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-25T00:09:44.319 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-25T00:09:44.320 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: mds cache max size = 1000 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = False 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: mds log max segments = 1 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = False 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.321 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.323 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:09:44.364 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.402 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.403 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.404 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-25T00:09:44.405 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.406 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-04-25T00:09:44.407 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-25T00:09:44.408 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-25T00:09:44.409 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:09:44.410 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: mon down mkfs grace = 300 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:09:44.411 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-04-25T00:09:44.412 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: mds cache max size = 1000 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = False 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: mds log max segments = 1 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = False 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.413 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-25T00:09:44.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.113,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = c4eaaf81-6120-4e3f-a50c-97aa81453142 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.417 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-25T00:09:44.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-25T00:09:44.419 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:09:44.420 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-25T00:09:44.421 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:09:44.422 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:09:44.423 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-25T00:09:44.424 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: mds cache max size = 1000 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = False 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: mds log max segments = 1 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = False 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-25T00:09:44.425 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-25T00:09:44.427 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:44.427 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:44.481 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:09:44.481 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:09:44.537 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:09:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.673+0000 7fb9fc114640 1 Processor -- start 2024-04-25T00:09:44.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.673+0000 7fb9fc114640 1 -- start start 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9fc114640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f415e0b0 0x7fb9f415f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9fc114640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9f40ca870 0x7fb9f40b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9fc114640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9f40cdd80 0x7fb9f40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9fc114640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb9f415f7a0 con 0x7fb9f40ca870 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9fc114640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb9f40cfb30 con 0x7fb9f40cdd80 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9fc114640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f40cfdf0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9f96af640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9f40ca870 0x7fb9f40b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9f96af640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9f40ca870 0x7fb9f40b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43852/0 (socket says 172.21.15.113:43852) 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.674+0000 7fb9f96af640 1 -- 172.21.15.113:0/2968863195 learned_addr learned my addr 172.21.15.113:0/2968863195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9f96af640 1 -- 172.21.15.113:0/2968863195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f415e0b0 msgr2=0x7fb9f415f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9fa6b1640 1 --2- 172.21.15.113:0/2968863195 >> v2:172.21.15.178:3300/0 conn(0x7fb9f40cdd80 0x7fb9f40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9f9eb0640 1 --2- 172.21.15.113:0/2968863195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f415e0b0 0x7fb9f415f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9f96af640 1 --2- 172.21.15.113:0/2968863195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f415e0b0 0x7fb9f415f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9f96af640 1 -- 172.21.15.113:0/2968863195 >> v2:172.21.15.178:3300/0 conn(0x7fb9f40cdd80 msgr2=0x7fb9f40b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9f96af640 1 --2- 172.21.15.113:0/2968863195 >> v2:172.21.15.178:3300/0 conn(0x7fb9f40cdd80 0x7fb9f40b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9f96af640 1 -- 172.21.15.113:0/2968863195 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f40cc570 con 0x7fb9f40ca870 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9fa6b1640 1 --2- 172.21.15.113:0/2968863195 >> v2:172.21.15.178:3300/0 conn(0x7fb9f40cdd80 0x7fb9f40b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.675+0000 7fb9f96af640 1 --2- 172.21.15.113:0/2968863195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f40ca870 0x7fb9f40b1c30 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40c3c90 tx=0x7fb9e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c77f4629ce5793 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9f8eae640 1 -- 172.21.15.113:0/2968863195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4037070 con 0x7fb9f40ca870 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9f8eae640 1 -- 172.21.15.113:0/2968863195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e4032440 con 0x7fb9f40ca870 2024-04-25T00:09:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9f8eae640 1 -- 172.21.15.113:0/2968863195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4032810 con 0x7fb9f40ca870 2024-04-25T00:09:44.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 -- 172.21.15.113:0/2968863195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f40ca870 msgr2=0x7fb9f40b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 --2- 172.21.15.113:0/2968863195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f40ca870 0x7fb9f40b1c30 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40c3c90 tx=0x7fb9e402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 -- 172.21.15.113:0/2968863195 shutdown_connections 2024-04-25T00:09:44.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 --2- 172.21.15.113:0/2968863195 >> v2:172.21.15.178:3300/0 conn(0x7fb9f40cdd80 0x7fb9f40b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 --2- 172.21.15.113:0/2968863195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f40ca870 0x7fb9f40b1c30 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 --2- 172.21.15.113:0/2968863195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f415e0b0 0x7fb9f415f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 -- 172.21.15.113:0/2968863195 >> 172.21.15.113:0/2968863195 conn(0x7fb9f40ccc30 msgr2=0x7fb9f40cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 -- 172.21.15.113:0/2968863195 shutdown_connections 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.676+0000 7fb9fc114640 1 -- 172.21.15.113:0/2968863195 wait complete. 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.677+0000 7fb9fc114640 1 Processor -- start 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.677+0000 7fb9fc114640 1 -- start start 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fc114640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f40ca870 0x7fb9f40c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fc114640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9f40cdd80 0x7fb9f415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fc114640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f415e0b0 0x7fb9f409d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fc114640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f40cc350 con 0x7fb9f415e0b0 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fc114640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f40cc1d0 con 0x7fb9f40cdd80 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fc114640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f415e490 con 0x7fb9f40ca870 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f415e0b0 0x7fb9f409d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f415e0b0 0x7fb9f409d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43866/0 (socket says 172.21.15.113:43866) 2024-04-25T00:09:44.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 -- 172.21.15.113:0/1978402832 learned_addr learned my addr 172.21.15.113:0/1978402832 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9f96af640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9f40cdd80 0x7fb9f415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9f9eb0640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f40ca870 0x7fb9f40c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 -- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f40ca870 msgr2=0x7fb9f40c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f40ca870 0x7fb9f40c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 -- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9f40cdd80 msgr2=0x7fb9f415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9f40cdd80 0x7fb9f415b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9fa6b1640 1 -- 172.21.15.113:0/1978402832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f409d8d0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.678+0000 7fb9f96af640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9f40cdd80 0x7fb9f415b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.679+0000 7fb9fa6b1640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f415e0b0 0x7fb9f409d390 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c590 tx=0x7fb9ec00ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.679+0000 7fb9ea7fc640 1 -- 172.21.15.113:0/1978402832 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec015070 con 0x7fb9f415e0b0 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.679+0000 7fb9ea7fc640 1 -- 172.21.15.113:0/1978402832 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ec011800 con 0x7fb9f415e0b0 2024-04-25T00:09:44.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.679+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f409c2f0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.679+0000 7fb9ea7fc640 1 -- 172.21.15.113:0/1978402832 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec011bf0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.679+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f409c8b0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.680+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f40691a0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.680+0000 7fb9ea7fc640 1 -- 172.21.15.113:0/1978402832 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9ec018070 con 0x7fb9f415e0b0 2024-04-25T00:09:44.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.681+0000 7fb9ea7fc640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9d806be60 0x7fb9d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.681+0000 7fb9f9eb0640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9d806be60 0x7fb9d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.681+0000 7fb9ea7fc640 1 -- 172.21.15.113:0/1978402832 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7fb9ec08ac10 con 0x7fb9f415e0b0 2024-04-25T00:09:44.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.681+0000 7fb9f9eb0640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9d806be60 0x7fb9d806e320 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0008ef0 tx=0x7fb9f0008440 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:44.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.685+0000 7fb9ea7fc640 1 -- 172.21.15.113:0/1978402832 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9ec0595c0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.859+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9f406dd30 con 0x7fb9f415e0b0 2024-04-25T00:09:44.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.860+0000 7fb9ea7fc640 1 -- 172.21.15.113:0/1978402832 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) ==== 74+0+16731 (secure 0 0 0) 0x7fb9ec058ec0 con 0x7fb9f415e0b0 2024-04-25T00:09:44.861 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:44.861 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":182,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:34.859228+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:31.412384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:32.173467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9d806be60 msgr2=0x7fb9d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9d806be60 0x7fb9d806e320 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0008ef0 tx=0x7fb9f0008440 comp rx=0 tx=0).stop 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f415e0b0 msgr2=0x7fb9f409d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f415e0b0 0x7fb9f409d390 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c590 tx=0x7fb9ec00ca50 comp rx=0 tx=0).stop 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 shutdown_connections 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb9d806be60 0x7fb9d806e320 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9f40cdd80 0x7fb9f415b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f415e0b0 0x7fb9f409d390 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 --2- 172.21.15.113:0/1978402832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9f40ca870 0x7fb9f40c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 >> 172.21.15.113:0/1978402832 conn(0x7fb9f40ccc30 msgr2=0x7fb9f415d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 shutdown_connections 2024-04-25T00:09:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.862+0000 7fb9fc114640 1 -- 172.21.15.113:0/1978402832 wait complete. 2024-04-25T00:09:44.874 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:09:31.412384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:09:32.173467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:09:44.874 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:09:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.974+0000 7f946803f640 1 Processor -- start 2024-04-25T00:09:44.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.974+0000 7f946803f640 1 -- start start 2024-04-25T00:09:44.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.975+0000 7f946803f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f946015e510 0x7f94600d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.975+0000 7f946803f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 0x7f94600b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.975+0000 7f946803f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94600d2e00 con 0x7f94600b61e0 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.975+0000 7f946803f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f94600d30e0 con 0x7f946015e510 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.975+0000 7f946803f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94600d33c0 con 0x7f94600d28c0 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.975+0000 7f94665dc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f94600b61e0 0x7f9460071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60746/0 (socket says 172.21.15.113:60746) 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.975+0000 7f94665dc640 1 -- 172.21.15.113:0/605538133 learned_addr learned my addr 172.21.15.113:0/605538133 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574109932 0 0) 0x7f94600d2e00 con 0x7f94600b61e0 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9448003520 con 0x7f94600b61e0 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9465ddb640 1 --2- 172.21.15.113:0/605538133 >> v2:172.21.15.178:3300/0 conn(0x7f946015e510 0x7f94600d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94655da640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 0x7f94600b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 758910599 0 0) 0x7f9448003520 con 0x7f94600b61e0 2024-04-25T00:09:44.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 msgr2=0x7f94600b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 0x7f94600b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 >> v2:172.21.15.178:3300/0 conn(0x7f946015e510 msgr2=0x7f94600d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 --2- 172.21.15.113:0/605538133 >> v2:172.21.15.178:3300/0 conn(0x7f946015e510 0x7f94600d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94600d36a0 con 0x7f94600b61e0 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f9458001670 con 0x7f94600b61e0 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 >> v1:172.21.15.113:6789/0 conn(0x7f94600b61e0 legacy=0x7f9460071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 0x7f94480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9448003fe0 0x7f9448004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9448004d40 0x7f94480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94600d30e0 con 0x7f9448004d40 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94600d33c0 con 0x7f9448003fe0 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9448014390 con 0x7f94600d28c0 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94665dc640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9448004d40 0x7f94480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f9465ddb640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 0x7f94480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94655da640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9448003fe0 0x7f9448004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94665dc640 1 -- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 msgr2=0x7f94480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94665dc640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94600d28c0 0x7f94480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94665dc640 1 -- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9448003fe0 msgr2=0x7f9448004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94665dc640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9448003fe0 0x7f9448004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.976+0000 7f94665dc640 1 -- 172.21.15.113:0/605538133 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9448003520 con 0x7f9448004d40 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.977+0000 7f94665dc640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9448004d40 0x7f94480051c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f945800c290 tx=0x7f9448005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd81aac621e2c70d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.977+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458007d50 con 0x7f9448004d40 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.977+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9458005af0 con 0x7f9448004d40 2024-04-25T00:09:44.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.977+0000 7f9464dd9640 1 -- 172.21.15.113:0/605538133 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458007070 con 0x7f9448004d40 2024-04-25T00:09:44.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.978+0000 7f946803f640 1 -- 172.21.15.113:0/605538133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9448004d40 msgr2=0x7f94480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.978+0000 7f946803f640 1 --2- 172.21.15.113:0/605538133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9448004d40 0x7f94480051c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f945800c290 tx=0x7f9448005830 comp rx=0 tx=0).stop 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.978+0000 7f94665dc640 1 -- 172.21.15.113:0/605538133 reap_dead start 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.978+0000 7f946803f640 1 -- 172.21.15.113:0/605538133 shutdown_connections 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.978+0000 7f946803f640 1 -- 172.21.15.113:0/605538133 >> 172.21.15.113:0/605538133 conn(0x7f946006b720 msgr2=0x7f94600d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.978+0000 7f946803f640 1 -- 172.21.15.113:0/605538133 shutdown_connections 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.978+0000 7f946803f640 1 -- 172.21.15.113:0/605538133 wait complete. 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.979+0000 7f946803f640 1 Processor -- start 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.979+0000 7f946803f640 1 -- start start 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f946803f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9460150610 0x7f94600b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f946803f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94600b78c0 0x7f94601efc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f946803f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94601f0150 0x7f946015e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f946803f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94600b9dd0 con 0x7f94601f0150 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f946803f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94600d3ea0 con 0x7f94600b78c0 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f946803f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94600b83d0 con 0x7f9460150610 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94601f0150 0x7f946015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94601f0150 0x7f946015e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43872/0 (socket says 172.21.15.113:43872) 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 -- 172.21.15.113:0/1091714918 learned_addr learned my addr 172.21.15.113:0/1091714918 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f9465ddb640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9460150610 0x7f94600b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94655da640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94600b78c0 0x7f94601efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 -- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9460150610 msgr2=0x7f94600b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9460150610 0x7f94600b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 -- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94600b78c0 msgr2=0x7f94601efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94600b78c0 0x7f94601efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 -- 172.21.15.113:0/1091714918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601f0c30 con 0x7f94601f0150 2024-04-25T00:09:44.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f94665dc640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94601f0150 0x7f946015e980 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f94600d2fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f945e7fc640 1 -- 172.21.15.113:0/1091714918 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458007d50 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.980+0000 7f945e7fc640 1 -- 172.21.15.113:0/1091714918 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9458044070 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.981+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601f0f10 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.981+0000 7f945e7fc640 1 -- 172.21.15.113:0/1091714918 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458001c80 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.981+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601f05d0 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.982+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94600691a0 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.982+0000 7f945e7fc640 1 -- 172.21.15.113:0/1091714918 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f945803b040 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.982+0000 7f945e7fc640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f944c06be30 0x7f944c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.982+0000 7f945e7fc640 1 -- 172.21.15.113:0/1091714918 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f94580aaec0 con 0x7f94601f0150 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.982+0000 7f9465ddb640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f944c06be30 0x7f944c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.983+0000 7f9465ddb640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f944c06be30 0x7f944c06e2f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9450003b80 tx=0x7f9450009760 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:44.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:44.987+0000 7f945e7fc640 1 -- 172.21.15.113:0/1091714918 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f94580797e0 con 0x7f94601f0150 2024-04-25T00:09:45.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.160+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f94600b7ca0 con 0x7f94601f0150 2024-04-25T00:09:45.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.161+0000 7f945e7fc640 1 -- 172.21.15.113:0/1091714918 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) ==== 93+0+10 (secure 0 0 0) 0x7f94580790e0 con 0x7f94601f0150 2024-04-25T00:09:45.161 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:09:45.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f944c06be30 msgr2=0x7f944c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f944c06be30 0x7f944c06e2f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9450003b80 tx=0x7f9450009760 comp rx=0 tx=0).stop 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94601f0150 msgr2=0x7f946015e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94601f0150 0x7f946015e980 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f94600d2fd0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 shutdown_connections 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f944c06be30 0x7f944c06e2f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94600b78c0 0x7f94601efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94601f0150 0x7f946015e980 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 --2- 172.21.15.113:0/1091714918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9460150610 0x7f94600b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 >> 172.21.15.113:0/1091714918 conn(0x7f946006b720 msgr2=0x7f94600b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 shutdown_connections 2024-04-25T00:09:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.163+0000 7f946803f640 1 -- 172.21.15.113:0/1091714918 wait complete. 2024-04-25T00:09:45.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:09:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.275+0000 7fe5d7a7b640 1 Processor -- start 2024-04-25T00:09:45.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.275+0000 7fe5d7a7b640 1 -- start start 2024-04-25T00:09:45.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.276+0000 7fe5d7a7b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5d0161ff0 0x7fe5d0162480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.276+0000 7fe5d7a7b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d006bb40 0x7fe5d006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.276+0000 7fe5d7a7b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5d00d2ce0 con 0x7fe5d0158a00 2024-04-25T00:09:45.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.276+0000 7fe5d7a7b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe5d006b0f0 con 0x7fe5d0161ff0 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.276+0000 7fe5d7a7b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d006b6b0 con 0x7fe5d006bb40 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d6018640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d006bb40 0x7fe5d006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5016640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5d0161ff0 0x7fe5d0162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5016640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5d0161ff0 0x7fe5d0162480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.113:56050/0 (socket says 172.21.15.113:56050) 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5817640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe5d0158a00 0x7fe5d00d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:52204/0 (socket says 172.21.15.113:52204) 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5016640 1 -- 172.21.15.113:0/3901805361 learned_addr learned my addr 172.21.15.113:0/3901805361 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5016640 1 -- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d006bb40 msgr2=0x7fe5d006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5016640 1 --2- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d006bb40 0x7fe5d006bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5016640 1 -- 172.21.15.113:0/3901805361 >> v1:172.21.15.178:6789/0 conn(0x7fe5d0158a00 legacy=0x7fe5d00d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d5016640 1 -- 172.21.15.113:0/3901805361 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d00d0170 con 0x7fe5d0161ff0 2024-04-25T00:09:45.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.277+0000 7fe5d6018640 1 --2- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d006bb40 0x7fe5d006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d5016640 1 --2- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0161ff0 0x7fe5d0162480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009870 tx=0x7fe5c002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a45c08f59b32ed3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d4815640 1 -- 172.21.15.113:0/3901805361 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0037070 con 0x7fe5d0161ff0 2024-04-25T00:09:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d4815640 1 -- 172.21.15.113:0/3901805361 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c0032440 con 0x7fe5d0161ff0 2024-04-25T00:09:45.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d4815640 1 -- 172.21.15.113:0/3901805361 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0032810 con 0x7fe5d0161ff0 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0161ff0 msgr2=0x7fe5d0162480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0161ff0 0x7fe5d0162480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009870 tx=0x7fe5c002a270 comp rx=0 tx=0).stop 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/3901805361 shutdown_connections 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d006bb40 0x7fe5d006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/3901805361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0161ff0 0x7fe5d0162480 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.278+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/3901805361 >> 172.21.15.113:0/3901805361 conn(0x7fe5d00b3d80 msgr2=0x7fe5d00b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.279+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/3901805361 shutdown_connections 2024-04-25T00:09:45.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.279+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/3901805361 wait complete. 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 Processor -- start 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 -- start start 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d006bb40 0x7fe5d00ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0158a00 0x7fe5d0141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d0161ff0 0x7fe5d01f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d00cfe40 con 0x7fe5d006bb40 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d006b340 con 0x7fe5d0158a00 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d7a7b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d0158de0 con 0x7fe5d0161ff0 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d5817640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d006bb40 0x7fe5d00ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d5817640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d006bb40 0x7fe5d00ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43874/0 (socket says 172.21.15.113:43874) 2024-04-25T00:09:45.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d5817640 1 -- 172.21.15.113:0/1302734674 learned_addr learned my addr 172.21.15.113:0/1302734674 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.280+0000 7fe5d5016640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0158a00 0x7fe5d0141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d6018640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d0161ff0 0x7fe5d01f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d5817640 1 -- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d0161ff0 msgr2=0x7fe5d01f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d5817640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d0161ff0 0x7fe5d01f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d5817640 1 -- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0158a00 msgr2=0x7fe5d0141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d5817640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0158a00 0x7fe5d0141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d5817640 1 -- 172.21.15.113:0/1302734674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d00caa50 con 0x7fe5d006bb40 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d5817640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d006bb40 0x7fe5d00ca390 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c430 tx=0x7fe5cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5c67fc640 1 -- 172.21.15.113:0/1302734674 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0074e0 con 0x7fe5d006bb40 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5c67fc640 1 -- 172.21.15.113:0/1302734674 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5cc007680 con 0x7fe5d006bb40 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5c67fc640 1 -- 172.21.15.113:0/1302734674 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc004a00 con 0x7fe5d006bb40 2024-04-25T00:09:45.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d00cad30 con 0x7fe5d006bb40 2024-04-25T00:09:45.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.281+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d00cb2f0 con 0x7fe5d006bb40 2024-04-25T00:09:45.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.282+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d00691a0 con 0x7fe5d006bb40 2024-04-25T00:09:45.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.282+0000 7fe5c67fc640 1 -- 172.21.15.113:0/1302734674 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5cc01a020 con 0x7fe5d006bb40 2024-04-25T00:09:45.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.283+0000 7fe5c67fc640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5b406be30 0x7fe5b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.283+0000 7fe5c67fc640 1 -- 172.21.15.113:0/1302734674 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7fe5cc08a610 con 0x7fe5d006bb40 2024-04-25T00:09:45.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.283+0000 7fe5d5016640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5b406be30 0x7fe5b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.283+0000 7fe5d5016640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5b406be30 0x7fe5b406e2f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0034000 tx=0x7fe5c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.287+0000 7fe5c67fc640 1 -- 172.21.15.113:0/1302734674 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe5cc058eb0 con 0x7fe5d006bb40 2024-04-25T00:09:45.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.460+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5d015e560 con 0x7fe5d006bb40 2024-04-25T00:09:45.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.460+0000 7fe5c67fc640 1 -- 172.21.15.113:0/1302734674 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v182) ==== 104+0+11 (secure 0 0 0) 0x7fe5cc0587b0 con 0x7fe5d006bb40 2024-04-25T00:09:45.461 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:09:45.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5b406be30 msgr2=0x7fe5b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5b406be30 0x7fe5b406e2f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0034000 tx=0x7fe5c002e000 comp rx=0 tx=0).stop 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d006bb40 msgr2=0x7fe5d00ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d006bb40 0x7fe5d00ca390 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c430 tx=0x7fe5cc00c830 comp rx=0 tx=0).stop 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 shutdown_connections 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe5b406be30 0x7fe5b406e2f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5d0158a00 0x7fe5d0141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5d0161ff0 0x7fe5d01f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 --2- 172.21.15.113:0/1302734674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d006bb40 0x7fe5d00ca390 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 >> 172.21.15.113:0/1302734674 conn(0x7fe5d00b3d80 msgr2=0x7fe5d00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 shutdown_connections 2024-04-25T00:09:45.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.462+0000 7fe5d7a7b640 1 -- 172.21.15.113:0/1302734674 wait complete. 2024-04-25T00:09:45.473 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:09:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.572+0000 7fc052a15640 1 Processor -- start 2024-04-25T00:09:45.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.572+0000 7fc052a15640 1 -- start start 2024-04-25T00:09:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc052a15640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc04c15e510 0x7fc04c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc052a15640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c0d28c0 0x7fc04c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc052a15640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04c0d2e00 con 0x7fc04c0b61e0 2024-04-25T00:09:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc052a15640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc04c0d30e0 con 0x7fc04c15e510 2024-04-25T00:09:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc052a15640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c0d33c0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc04bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc04c15e510 0x7fc04c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc050fb2640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc04c0b61e0 0x7fc04c071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60754/0 (socket says 172.21.15.113:60754) 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.573+0000 7fc050fb2640 1 -- 172.21.15.113:0/1289879746 learned_addr learned my addr 172.21.15.113:0/1289879746 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.574+0000 7fc04b7fe640 1 --2- 172.21.15.113:0/1289879746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c0d28c0 0x7fc04c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.574+0000 7fc04bfff640 1 -- 172.21.15.113:0/1289879746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c0d28c0 msgr2=0x7fc04c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.574+0000 7fc04bfff640 1 --2- 172.21.15.113:0/1289879746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c0d28c0 0x7fc04c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.574+0000 7fc04bfff640 1 -- 172.21.15.113:0/1289879746 >> v1:172.21.15.113:6789/0 conn(0x7fc04c0b61e0 legacy=0x7fc04c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.574+0000 7fc04bfff640 1 -- 172.21.15.113:0/1289879746 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c0d36a0 con 0x7fc04c15e510 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.574+0000 7fc04affd640 1 -- 172.21.15.113:0/1289879746 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293238624 0 0) 0x7fc04c0d2e00 con 0x7fc04c0b61e0 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.574+0000 7fc04bfff640 1 --2- 172.21.15.113:0/1289879746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c15e510 0x7fc04c0d2380 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0097a0 tx=0x7fc03c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab8a2f394c30821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc04affd640 1 -- 172.21.15.113:0/1289879746 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c037070 con 0x7fc04c15e510 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc04affd640 1 -- 172.21.15.113:0/1289879746 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03c02ae50 con 0x7fc04c15e510 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc04affd640 1 -- 172.21.15.113:0/1289879746 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c032660 con 0x7fc04c15e510 2024-04-25T00:09:45.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 -- 172.21.15.113:0/1289879746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c15e510 msgr2=0x7fc04c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 --2- 172.21.15.113:0/1289879746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c15e510 0x7fc04c0d2380 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0097a0 tx=0x7fc03c02a100 comp rx=0 tx=0).stop 2024-04-25T00:09:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 -- 172.21.15.113:0/1289879746 shutdown_connections 2024-04-25T00:09:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 --2- 172.21.15.113:0/1289879746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c0d28c0 0x7fc04c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 --2- 172.21.15.113:0/1289879746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c15e510 0x7fc04c0d2380 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 -- 172.21.15.113:0/1289879746 >> 172.21.15.113:0/1289879746 conn(0x7fc04c06b720 msgr2=0x7fc04c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 -- 172.21.15.113:0/1289879746 shutdown_connections 2024-04-25T00:09:45.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.575+0000 7fc052a15640 1 -- 172.21.15.113:0/1289879746 wait complete. 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.576+0000 7fc052a15640 1 Processor -- start 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc052a15640 1 -- start start 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc052a15640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04c0b61e0 0x7fc04c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc052a15640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c0d28c0 0x7fc04c0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc052a15640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c15e510 0x7fc04c14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc052a15640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c0b9dd0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc052a15640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c0d3ea0 con 0x7fc04c0b61e0 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc052a15640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c0b83d0 con 0x7fc04c15e510 2024-04-25T00:09:45.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc04b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c0d28c0 0x7fc04c0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc04b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c0d28c0 0x7fc04c0ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43894/0 (socket says 172.21.15.113:43894) 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.577+0000 7fc04b7fe640 1 -- 172.21.15.113:0/528943338 learned_addr learned my addr 172.21.15.113:0/528943338 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc04b7fe640 1 -- 172.21.15.113:0/528943338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c15e510 msgr2=0x7fc04c14a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc04b7fe640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c15e510 0x7fc04c14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc04b7fe640 1 -- 172.21.15.113:0/528943338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04c0b61e0 msgr2=0x7fc04c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc04b7fe640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04c0b61e0 0x7fc04c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc04b7fe640 1 -- 172.21.15.113:0/528943338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c0cb9f0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc04b7fe640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c0d28c0 0x7fc04c0ca720 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fc04000e9e0 tx=0x7fc04000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc048ff9640 1 -- 172.21.15.113:0/528943338 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040017070 con 0x7fc04c0d28c0 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc048ff9640 1 -- 172.21.15.113:0/528943338 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc04000c9d0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c0cbcd0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc048ff9640 1 -- 172.21.15.113:0/528943338 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04000cdc0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.578+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c0cc290 con 0x7fc04c0d28c0 2024-04-25T00:09:45.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.579+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c0691a0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.579+0000 7fc048ff9640 1 -- 172.21.15.113:0/528943338 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc04001a070 con 0x7fc04c0d28c0 2024-04-25T00:09:45.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.580+0000 7fc048ff9640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc03806be30 0x7fc03806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.580+0000 7fc048ff9640 1 -- 172.21.15.113:0/528943338 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7fc04008ae00 con 0x7fc04c0d28c0 2024-04-25T00:09:45.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.580+0000 7fc04bfff640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc03806be30 0x7fc03806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.580+0000 7fc04bfff640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc03806be30 0x7fc03806e2f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc03c034000 tx=0x7fc03c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.584+0000 7fc048ff9640 1 -- 172.21.15.113:0/528943338 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc040059840 con 0x7fc04c0d28c0 2024-04-25T00:09:45.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.756+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc04c0b9af0 con 0x7fc04c0d28c0 2024-04-25T00:09:45.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.756+0000 7fc048ff9640 1 -- 172.21.15.113:0/528943338 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v182) ==== 100+0+11 (secure 0 0 0) 0x7fc040059140 con 0x7fc04c0d28c0 2024-04-25T00:09:45.757 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc03806be30 msgr2=0x7fc03806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc03806be30 0x7fc03806e2f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc03c034000 tx=0x7fc03c02c000 comp rx=0 tx=0).stop 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c0d28c0 msgr2=0x7fc04c0ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c0d28c0 0x7fc04c0ca720 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fc04000e9e0 tx=0x7fc04000eea0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 shutdown_connections 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc03806be30 0x7fc03806e2f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc04c15e510 0x7fc04c14a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc04c0d28c0 0x7fc04c0ca720 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 --2- 172.21.15.113:0/528943338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc04c0b61e0 0x7fc04c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 >> 172.21.15.113:0/528943338 conn(0x7fc04c06b720 msgr2=0x7fc04c0b8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 shutdown_connections 2024-04-25T00:09:45.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.758+0000 7fc052a15640 1 -- 172.21.15.113:0/528943338 wait complete. 2024-04-25T00:09:45.769 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:09:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.865+0000 7f6412dd5640 1 Processor -- start 2024-04-25T00:09:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.865+0000 7f6412dd5640 1 -- start start 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6412dd5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15ba50 0x7f640c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6412dd5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f640c06cea0 0x7f640c06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6412dd5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c071170 con 0x7f640c15b670 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6412dd5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f640c06d810 con 0x7f640c06cea0 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6412dd5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f640c071450 con 0x7f640c15ba50 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6411372640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f640c06cea0 0x7f640c06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6411372640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f640c06cea0 0x7f640c06d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43908/0 (socket says 172.21.15.113:43908) 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.866+0000 7f6411372640 1 -- 172.21.15.113:0/3218693089 learned_addr learned my addr 172.21.15.113:0/3218693089 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.867+0000 7f640bfff640 1 --2- 172.21.15.113:0/3218693089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15ba50 0x7f640c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.867+0000 7f6411372640 1 -- 172.21.15.113:0/3218693089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15ba50 msgr2=0x7f640c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.867+0000 7f6411372640 1 --2- 172.21.15.113:0/3218693089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15ba50 0x7f640c15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.867+0000 7f6411372640 1 -- 172.21.15.113:0/3218693089 >> v1:172.21.15.178:6789/0 conn(0x7f640c15b670 legacy=0x7f640c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.867+0000 7f6411372640 1 -- 172.21.15.113:0/3218693089 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c071730 con 0x7f640c06cea0 2024-04-25T00:09:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.867+0000 7f6411372640 1 --2- 172.21.15.113:0/3218693089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c06d2d0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f6404009e50 tx=0x7f640402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eeb209941b09c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f640b7fe640 1 -- 172.21.15.113:0/3218693089 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404036070 con 0x7f640c06cea0 2024-04-25T00:09:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f640b7fe640 1 -- 172.21.15.113:0/3218693089 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6404031600 con 0x7f640c06cea0 2024-04-25T00:09:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f640b7fe640 1 -- 172.21.15.113:0/3218693089 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040319d0 con 0x7f640c06cea0 2024-04-25T00:09:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 -- 172.21.15.113:0/3218693089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 msgr2=0x7f640c06d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 --2- 172.21.15.113:0/3218693089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c06d2d0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f6404009e50 tx=0x7f640402a5f0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 -- 172.21.15.113:0/3218693089 shutdown_connections 2024-04-25T00:09:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 --2- 172.21.15.113:0/3218693089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15ba50 0x7f640c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 --2- 172.21.15.113:0/3218693089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c06d2d0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 -- 172.21.15.113:0/3218693089 >> 172.21.15.113:0/3218693089 conn(0x7f640c06b0f0 msgr2=0x7f640c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 -- 172.21.15.113:0/3218693089 shutdown_connections 2024-04-25T00:09:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.868+0000 7f6412dd5640 1 -- 172.21.15.113:0/3218693089 wait complete. 2024-04-25T00:09:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.869+0000 7f6412dd5640 1 Processor -- start 2024-04-25T00:09:45.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.869+0000 7f6412dd5640 1 -- start start 2024-04-25T00:09:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6412dd5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6412dd5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15b670 0x7f640c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6412dd5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640c15ba50 0x7f640c0ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6412dd5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f640c0cca90 con 0x7f640c06cea0 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6412dd5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f640c0cc910 con 0x7f640c15ba50 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6412dd5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f640c0d0080 con 0x7f640c15b670 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6410b71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6410b71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43912/0 (socket says 172.21.15.113:43912) 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.870+0000 7f6410b71640 1 -- 172.21.15.113:0/486817217 learned_addr learned my addr 172.21.15.113:0/486817217 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6410b71640 1 -- 172.21.15.113:0/486817217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15b670 msgr2=0x7f640c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6410b71640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15b670 0x7f640c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6410b71640 1 -- 172.21.15.113:0/486817217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640c15ba50 msgr2=0x7f640c0ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6411372640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640c15ba50 0x7f640c0ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6410b71640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640c15ba50 0x7f640c0ab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6410b71640 1 -- 172.21.15.113:0/486817217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1543b0 con 0x7f640c06cea0 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6410b71640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c153e70 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c400 tx=0x7f63fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f64097fa640 1 -- 172.21.15.113:0/486817217 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0052d0 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f64097fa640 1 -- 172.21.15.113:0/486817217 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63fc005490 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f64097fa640 1 -- 172.21.15.113:0/486817217 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc007260 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c0cbc90 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.871+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1525c0 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.872+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c0d0420 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.872+0000 7f64097fa640 1 -- 172.21.15.113:0/486817217 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63fc007400 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.873+0000 7f64097fa640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63f806be30 0x7f63f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.873+0000 7f64097fa640 1 -- 172.21.15.113:0/486817217 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f63fc08a3f0 con 0x7f640c06cea0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.873+0000 7f640bfff640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63f806be30 0x7f63f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.873+0000 7f640bfff640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63f806be30 0x7f63f806e2f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6400006fd0 tx=0x7f6400008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:45.877+0000 7f64097fa640 1 -- 172.21.15.113:0/486817217 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f640c0d0420 con 0x7f640c06cea0 2024-04-25T00:09:46.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.064+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f640c06ba00 con 0x7f640c06cea0 2024-04-25T00:09:46.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.064+0000 7f64097fa640 1 -- 172.21.15.113:0/486817217 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v182) ==== 80+0+0 (secure 0 0 0) 0x7f640c06ba00 con 0x7f640c06cea0 2024-04-25T00:09:46.065 INFO:teuthology.orchestra.run.smithi113.stderr:listed 0 entries 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63f806be30 msgr2=0x7f63f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63f806be30 0x7f63f806e2f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6400006fd0 tx=0x7f6400008040 comp rx=0 tx=0).stop 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 msgr2=0x7f640c153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c153e70 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c400 tx=0x7f63fc00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 shutdown_connections 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63f806be30 0x7f63f806e2f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640c15ba50 0x7f640c0ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640c15b670 0x7f640c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 --2- 172.21.15.113:0/486817217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f640c06cea0 0x7f640c153e70 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 >> 172.21.15.113:0/486817217 conn(0x7f640c06b0f0 msgr2=0x7f640c15a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:46.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.066+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 shutdown_connections 2024-04-25T00:09:46.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.067+0000 7f6412dd5640 1 -- 172.21.15.113:0/486817217 wait complete. 2024-04-25T00:09:46.078 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:09:46.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:46.078 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:09:46.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:09:46.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:46.136 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:09:46.155 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:46.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.291+0000 7f3ee5a66640 1 Processor -- start 2024-04-25T00:09:46.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.291+0000 7f3ee5a66640 1 -- start start 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3ee5a66640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee0144910 0x7f3ee00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3ee5a66640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3ee00b5240 0x7f3ee006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3ee5a66640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee0144cf0 con 0x7f3ee00b5680 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3ee5a66640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3ee00b4510 con 0x7f3ee00b5240 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3ee5a66640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee00b2f30 con 0x7f3ee0144910 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3edf7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee0144910 0x7f3ee00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3edeffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3ee00b5240 0x7f3ee006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3edffff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3ee00b5680 0x7f3ee00cd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.113:52230/0 (socket says 172.21.15.113:52230) 2024-04-25T00:09:46.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.292+0000 7f3edffff640 1 -- 172.21.15.113:0/3630981271 learned_addr learned my addr 172.21.15.113:0/3630981271 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:46.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.293+0000 7f3edf7fe640 1 -- 172.21.15.113:0/3630981271 >> v2:172.21.15.178:3300/0 conn(0x7f3ee00b5240 msgr2=0x7f3ee006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:46.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.293+0000 7f3edf7fe640 1 --2- 172.21.15.113:0/3630981271 >> v2:172.21.15.178:3300/0 conn(0x7f3ee00b5240 0x7f3ee006dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.293+0000 7f3edf7fe640 1 -- 172.21.15.113:0/3630981271 >> v1:172.21.15.178:6789/0 conn(0x7f3ee00b5680 legacy=0x7f3ee00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.293+0000 7f3edf7fe640 1 -- 172.21.15.113:0/3630981271 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee00b3210 con 0x7f3ee0144910 2024-04-25T00:09:46.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.293+0000 7f3edeffd640 1 --2- 172.21.15.113:0/3630981271 >> v2:172.21.15.178:3300/0 conn(0x7f3ee00b5240 0x7f3ee006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:46.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.293+0000 7f3edf7fe640 1 --2- 172.21.15.113:0/3630981271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee0144910 0x7f3ee00b3fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00097a0 tx=0x7f3ed002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4e3655686a2803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ede7fc640 1 -- 172.21.15.113:0/3630981271 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0037070 con 0x7f3ee0144910 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ede7fc640 1 -- 172.21.15.113:0/3630981271 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed0032440 con 0x7f3ee0144910 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ede7fc640 1 -- 172.21.15.113:0/3630981271 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0032810 con 0x7f3ee0144910 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 -- 172.21.15.113:0/3630981271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee0144910 msgr2=0x7f3ee00b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/3630981271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee0144910 0x7f3ee00b3fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00097a0 tx=0x7f3ed002a180 comp rx=0 tx=0).stop 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 -- 172.21.15.113:0/3630981271 shutdown_connections 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/3630981271 >> v2:172.21.15.178:3300/0 conn(0x7f3ee00b5240 0x7f3ee006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/3630981271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee0144910 0x7f3ee00b3fd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 -- 172.21.15.113:0/3630981271 >> 172.21.15.113:0/3630981271 conn(0x7f3ee00b1a50 msgr2=0x7f3ee00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 -- 172.21.15.113:0/3630981271 shutdown_connections 2024-04-25T00:09:46.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.294+0000 7f3ee5a66640 1 -- 172.21.15.113:0/3630981271 wait complete. 2024-04-25T00:09:46.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.295+0000 7f3ee5a66640 1 Processor -- start 2024-04-25T00:09:46.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.296+0000 7f3ee5a66640 1 -- start start 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.296+0000 7f3ee5a66640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee00b5240 0x7f3ee0158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.296+0000 7f3ee5a66640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee00b5680 0x7f3ee0142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.296+0000 7f3ee5a66640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee0144910 0x7f3ee00ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.296+0000 7f3ee5a66640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee00b4da0 con 0x7f3ee0144910 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.296+0000 7f3ee5a66640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee00b37d0 con 0x7f3ee00b5680 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.296+0000 7f3ee5a66640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee00b4f20 con 0x7f3ee00b5240 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee0144910 0x7f3ee00ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee0144910 0x7f3ee00ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43914/0 (socket says 172.21.15.113:43914) 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 -- 172.21.15.113:0/2390672604 learned_addr learned my addr 172.21.15.113:0/2390672604 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edf7fe640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee00b5240 0x7f3ee0158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edeffd640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee00b5680 0x7f3ee0142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 -- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee00b5240 msgr2=0x7f3ee0158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee00b5240 0x7f3ee0158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 -- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee00b5680 msgr2=0x7f3ee0142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee00b5680 0x7f3ee0142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 -- 172.21.15.113:0/2390672604 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee015a7d0 con 0x7f3ee0144910 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3edffff640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee0144910 0x7f3ee00ad890 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800cad0 tx=0x7f3ed800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3ec7fff640 1 -- 172.21.15.113:0/2390672604 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8017070 con 0x7f3ee0144910 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3ec7fff640 1 -- 172.21.15.113:0/2390672604 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed800abd0 con 0x7f3ee0144910 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3ec7fff640 1 -- 172.21.15.113:0/2390672604 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8012400 con 0x7f3ee0144910 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee015aab0 con 0x7f3ee0144910 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.297+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee015b070 con 0x7f3ee0144910 2024-04-25T00:09:46.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.298+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee015b270 con 0x7f3ee0144910 2024-04-25T00:09:46.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.299+0000 7f3ec7fff640 1 -- 172.21.15.113:0/2390672604 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ed801a070 con 0x7f3ee0144910 2024-04-25T00:09:46.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.299+0000 7f3ec7fff640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.299+0000 7f3ec7fff640 1 -- 172.21.15.113:0/2390672604 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f3ed808a9d0 con 0x7f3ee0144910 2024-04-25T00:09:46.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.299+0000 7f3edf7fe640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.300+0000 7f3edf7fe640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0034000 tx=0x7f3ed002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:46.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.303+0000 7f3ec7fff640 1 -- 172.21.15.113:0/2390672604 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ee015b270 con 0x7f3ee0144910 2024-04-25T00:09:46.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.511+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ee00691a0 con 0x7f3ee0144910 2024-04-25T00:09:46.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.512+0000 7f3ec7fff640 1 -- 172.21.15.113:0/2390672604 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5204 (secure 0 0 0) 0x7f3ed8058d30 con 0x7f3ee0144910 2024-04-25T00:09:46.513 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:46.513 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":98,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":10794,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:35.199346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10803":{"gid":10803,"name":"d","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6835/3057967443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":3057967443},{"type":"v1","addr":"172.21.15.178:6835","nonce":3057967443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10803,"qdb_cluster":[10803]},"id":13}]} 2024-04-25T00:09:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 98 2024-04-25T00:09:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ecc06be30 msgr2=0x7f3ecc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0034000 tx=0x7f3ed002e000 comp rx=0 tx=0).stop 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee0144910 msgr2=0x7f3ee00ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee0144910 0x7f3ee00ad890 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800cad0 tx=0x7f3ed800cf90 comp rx=0 tx=0).stop 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 shutdown_connections 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f3ecc06be30 0x7f3ecc06e2f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ee00b5680 0x7f3ee0142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee0144910 0x7f3ee00ad890 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 --2- 172.21.15.113:0/2390672604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ee00b5240 0x7f3ee0158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 >> 172.21.15.113:0/2390672604 conn(0x7f3ee00b1a50 msgr2=0x7f3ee0072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 shutdown_connections 2024-04-25T00:09:46.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.514+0000 7f3ee5a66640 1 -- 172.21.15.113:0/2390672604 wait complete. 2024-04-25T00:09:46.526 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T00:09:46.657 INFO:teuthology.orchestra.run.smithi178.stdout:{"message":"","return_code":0} 2024-04-25T00:09:46.658 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T00:09:46.658 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:09:46.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.755+0000 7f44d8fd5640 1 Processor -- start 2024-04-25T00:09:46.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.755+0000 7f44d8fd5640 1 -- start start 2024-04-25T00:09:46.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d8fd5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f44d415e2d0 0x7f44d415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d8fd5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d415f4f0 con 0x7f44d40b3080 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d8fd5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d415f7d0 con 0x7f44d40b4a40 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d8fd5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f44d40cde00 con 0x7f44d415e2d0 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44cbfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f44d40b3080 0x7f44d40b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60770/0 (socket says 172.21.15.113:60770) 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44cbfff640 1 -- 172.21.15.113:0/1051360089 learned_addr learned my addr 172.21.15.113:0/1051360089 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d259c640 1 --2- 172.21.15.113:0/1051360089 >> v2:172.21.15.178:3300/0 conn(0x7f44d415e2d0 0x7f44d415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d1d9b640 1 -- 172.21.15.113:0/1051360089 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239600029 0 0) 0x7f44d415f4f0 con 0x7f44d40b3080 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d1d9b640 1 -- 172.21.15.113:0/1051360089 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44c0003520 con 0x7f44d40b3080 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d1d9b640 1 -- 172.21.15.113:0/1051360089 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 159889660 0 0) 0x7f44c0003520 con 0x7f44d40b3080 2024-04-25T00:09:46.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d259c640 1 -- 172.21.15.113:0/1051360089 >> v1:172.21.15.178:6789/0 conn(0x7f44d40b4a40 legacy=0x7f44d40ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d259c640 1 -- 172.21.15.113:0/1051360089 >> v1:172.21.15.113:6789/0 conn(0x7f44d40b3080 legacy=0x7f44d40b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.756+0000 7f44d259c640 1 -- 172.21.15.113:0/1051360089 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d415fab0 con 0x7f44d415e2d0 2024-04-25T00:09:46.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.757+0000 7f44d259c640 1 --2- 172.21.15.113:0/1051360089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d415e2d0 0x7f44d415ebf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f44d40cdf80 tx=0x7f44c402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1faccb0c93dabae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:46.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.757+0000 7f44d1d9b640 1 -- 172.21.15.113:0/1051360089 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4037070 con 0x7f44d415e2d0 2024-04-25T00:09:46.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.757+0000 7f44d1d9b640 1 -- 172.21.15.113:0/1051360089 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c402ac30 con 0x7f44d415e2d0 2024-04-25T00:09:46.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.757+0000 7f44d1d9b640 1 -- 172.21.15.113:0/1051360089 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc001670 con 0x7f44d415e2d0 2024-04-25T00:09:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.758+0000 7f44d8fd5640 1 -- 172.21.15.113:0/1051360089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d415e2d0 msgr2=0x7f44d415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.758+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/1051360089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d415e2d0 0x7f44d415ebf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f44d40cdf80 tx=0x7f44c402a1a0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.758+0000 7f44d8fd5640 1 -- 172.21.15.113:0/1051360089 shutdown_connections 2024-04-25T00:09:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.758+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/1051360089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d415e2d0 0x7f44d415ebf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.758+0000 7f44d8fd5640 1 -- 172.21.15.113:0/1051360089 >> 172.21.15.113:0/1051360089 conn(0x7f44d406b0f0 msgr2=0x7f44d406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.758+0000 7f44d8fd5640 1 -- 172.21.15.113:0/1051360089 shutdown_connections 2024-04-25T00:09:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.758+0000 7f44d8fd5640 1 -- 172.21.15.113:0/1051360089 wait complete. 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.759+0000 7f44d8fd5640 1 Processor -- start 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.759+0000 7f44d8fd5640 1 -- start start 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d8fd5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d40b3080 0x7f44d414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d8fd5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44d40b4a40 0x7f44d4147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d8fd5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d415e2d0 0x7f44d40ab590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d8fd5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f44d415f2b0 con 0x7f44d40b4a40 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d8fd5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44d415f130 con 0x7f44d40b3080 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d8fd5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44d415e6b0 con 0x7f44d415e2d0 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44d40b4a40 0x7f44d4147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44d40b4a40 0x7f44d4147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43916/0 (socket says 172.21.15.113:43916) 2024-04-25T00:09:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 -- 172.21.15.113:0/2701989442 learned_addr learned my addr 172.21.15.113:0/2701989442 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d259c640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d40b3080 0x7f44d414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d2d9d640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d415e2d0 0x7f44d40ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 -- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d415e2d0 msgr2=0x7f44d40ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d415e2d0 0x7f44d40ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 -- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d40b3080 msgr2=0x7f44d414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d40b3080 0x7f44d414a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 -- 172.21.15.113:0/2701989442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d40ae2e0 con 0x7f44d40b4a40 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44d2d9d640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d415e2d0 0x7f44d40ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44cbfff640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44d40b4a40 0x7f44d4147230 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c750 tx=0x7f44bc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.760+0000 7f44caffd640 1 -- 172.21.15.113:0/2701989442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc017070 con 0x7f44d40b4a40 2024-04-25T00:09:46.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.761+0000 7f44caffd640 1 -- 172.21.15.113:0/2701989442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc005ad0 con 0x7f44d40b4a40 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.761+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d414ab20 con 0x7f44d40b4a40 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.761+0000 7f44caffd640 1 -- 172.21.15.113:0/2701989442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc007070 con 0x7f44d40b4a40 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.761+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41482e0 con 0x7f44d40b4a40 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.762+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d40691a0 con 0x7f44d40b4a40 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.762+0000 7f44caffd640 1 -- 172.21.15.113:0/2701989442 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44bc01a070 con 0x7f44d40b4a40 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.762+0000 7f44caffd640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44b406be30 0x7f44b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.762+0000 7f44caffd640 1 -- 172.21.15.113:0/2701989442 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f44bc08a3a0 con 0x7f44d40b4a40 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.762+0000 7f44d259c640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44b406be30 0x7f44b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.763+0000 7f44d259c640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44b406be30 0x7f44b406e2f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f44c4034000 tx=0x7f44c402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.767+0000 7f44caffd640 1 -- 172.21.15.113:0/2701989442 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f44d414ab20 con 0x7f44d40b4a40 2024-04-25T00:09:46.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.978+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f44d4159740 con 0x7f44d40b4a40 2024-04-25T00:09:46.979 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:09:46.979+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:09:46.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.980+0000 7f44caffd640 1 -- 172.21.15.113:0/2701989442 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7f44d40ae2e0 con 0x7f44d40b4a40 2024-04-25T00:09:46.980 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:09:46.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.983+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44b406be30 msgr2=0x7f44b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.983+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44b406be30 0x7f44b406e2f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f44c4034000 tx=0x7f44c402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:46.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.983+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44d40b4a40 msgr2=0x7f44d4147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:46.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.983+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44d40b4a40 0x7f44d4147230 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c750 tx=0x7f44bc00cc10 comp rx=0 tx=0).stop 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 shutdown_connections 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f44b406be30 0x7f44b406e2f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d415e2d0 0x7f44d40ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44d40b4a40 0x7f44d4147230 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 --2- 172.21.15.113:0/2701989442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d40b3080 0x7f44d414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 >> 172.21.15.113:0/2701989442 conn(0x7f44d406b0f0 msgr2=0x7f44d40b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 shutdown_connections 2024-04-25T00:09:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:46.984+0000 7f44d8fd5640 1 -- 172.21.15.113:0/2701989442 wait complete. 2024-04-25T00:09:46.985 INFO:tasks.ceph.mds.d.smithi178.stderr: -4300> 2024-04-25T00:09:39.119+0000 7f6557a9f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:09:46.995 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-25T00:09:47.014 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-25T00:09:47.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.038+0000 7fae91d40f40 1 Processor -- start 2024-04-25T00:09:47.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.038+0000 7fae91d40f40 1 -- start start 2024-04-25T00:09:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae91d40f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d2350 0x55a48b35f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae91d40f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d5850 0x55a48b2d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae91d40f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b3f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae91d40f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a48b2d17e0 con 0x55a48b345f90 2024-04-25T00:09:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae91d40f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a48b231640 con 0x55a48b2d2350 2024-04-25T00:09:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae91d40f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a48b2d0c30 con 0x55a48b2d5850 2024-04-25T00:09:47.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b3f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b3f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43932/0 (socket says 172.21.15.113:43932) 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 -- 172.21.15.113:0/3686929497 learned_addr learned my addr 172.21.15.113:0/3686929497 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 -- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d5850 msgr2=0x55a48b2d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae8f843640 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d2350 0x55a48b35f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae8f042640 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d5850 0x55a48b2d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d5850 0x55a48b2d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 -- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d2350 msgr2=0x55a48b35f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d2350 0x55a48b35f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 -- 172.21.15.113:0/3686929497 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48b3428a0 con 0x55a48b345f90 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae8f843640 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d2350 0x55a48b35f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae8f042640 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d5850 0x55a48b2d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae90044640 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b3f1c50 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fae88055040 tx=0x7fae88051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e35f1ba964b542b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae8e040640 1 -- 172.21.15.113:0/3686929497 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88053800 con 0x55a48b345f90 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae8e040640 1 -- 172.21.15.113:0/3686929497 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae880539a0 con 0x55a48b345f90 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.039+0000 7fae8e040640 1 -- 172.21.15.113:0/3686929497 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88053d90 con 0x55a48b345f90 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 -- 172.21.15.113:0/3686929497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 msgr2=0x55a48b3f1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b3f1c50 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fae88055040 tx=0x7fae88051000 comp rx=0 tx=0).stop 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 -- 172.21.15.113:0/3686929497 shutdown_connections 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b3f1c50 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d5850 0x55a48b2d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3686929497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d2350 0x55a48b35f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 -- 172.21.15.113:0/3686929497 >> 172.21.15.113:0/3686929497 conn(0x55a48b2bbd10 msgr2=0x55a48b2d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 -- 172.21.15.113:0/3686929497 shutdown_connections 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 -- 172.21.15.113:0/3686929497 wait complete. 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 Processor -- start 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.040+0000 7fae91d40f40 1 -- start start 2024-04-25T00:09:47.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae91d40f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d2350 0x55a48b33e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae91d40f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d5850 0x55a48b33d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae91d40f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b340150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae91d40f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a48b342cb0 con 0x55a48b345f90 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae91d40f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a48b33ec90 con 0x55a48b2d5850 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae91d40f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a48b3425c0 con 0x55a48b2d2350 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b340150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b340150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43940/0 (socket says 172.21.15.113:43940) 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 -- 172.21.15.113:0/3009422689 learned_addr learned my addr 172.21.15.113:0/3009422689 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 -- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d2350 msgr2=0x55a48b33e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae8f042640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d5850 0x55a48b33d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae8f843640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d2350 0x55a48b33e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d2350 0x55a48b33e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 -- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d5850 msgr2=0x55a48b33d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d5850 0x55a48b33d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae90044640 1 -- 172.21.15.113:0/3009422689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48b3e7700 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae8f042640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a48b2d5850 0x55a48b33d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.041+0000 7fae8f843640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a48b2d2350 0x55a48b33e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.042+0000 7fae90044640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b340150 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7fae88025d20 tx=0x7fae88002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.042+0000 7fae8d03e640 1 -- 172.21.15.113:0/3009422689 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88053b50 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.042+0000 7fae8d03e640 1 -- 172.21.15.113:0/3009422689 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae88074070 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.042+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a48b345790 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.042+0000 7fae8d03e640 1 -- 172.21.15.113:0/3009422689 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88067430 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.042+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48b3e6d90 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.043+0000 7fae8d03e640 1 -- 172.21.15.113:0/3009422689 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae880675d0 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.043+0000 7fae8d03e640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae7006a810 0x7fae7006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.043+0000 7fae8d03e640 1 -- 172.21.15.113:0/3009422689 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fae880cfcd0 con 0x55a48b345f90 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.043+0000 7fae8f843640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae7006a810 0x7fae7006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55a48b3e93e0 0x55a48b35d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 --> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] -- osd_op(unknown.0.0:1 26.10 26:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e183) -- 0x55a48b3e7c00 con 0x55a48b3e93e0 2024-04-25T00:09:47.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae8f042640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55a48b3e93e0 0x55a48b35d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae8f843640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae7006a810 0x7fae7006ccd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fae8000d360 tx=0x7fae80008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae8f042640 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55a48b3e93e0 0x55a48b35d5e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae8f042640 1 -- 172.21.15.113:0/3009422689 <== osd.2 v2:172.21.15.113:6816/3097656860 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=588b] v0'0 uv10 ondisk = 0) ==== 164+0+588 (crc 0 0 0) 0x7fae780097f0 con 0x55a48b3e93e0 2024-04-25T00:09:47.049 INFO:teuthology.orchestra.run.smithi113.stderr:Writing to - 2024-04-25T00:09:47.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55a48b3e93e0 msgr2=0x55a48b35d5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55a48b3e93e0 0x55a48b35d5e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae7006a810 msgr2=0x7fae7006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae7006a810 0x7fae7006ccd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fae8000d360 tx=0x7fae80008040 comp rx=0 tx=0).stop 2024-04-25T00:09:47.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 msgr2=0x55a48b340150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 --2- 172.21.15.113:0/3009422689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a48b345f90 0x55a48b340150 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7fae88025d20 tx=0x7fae88002fb0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae90044640 1 -- 172.21.15.113:0/3009422689 reap_dead start 2024-04-25T00:09:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 shutdown_connections 2024-04-25T00:09:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.044+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 >> 172.21.15.113:0/3009422689 conn(0x55a48b2bbd10 msgr2=0x55a48b2d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.045+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 shutdown_connections 2024-04-25T00:09:47.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.045+0000 7fae91d40f40 1 -- 172.21.15.113:0/3009422689 wait complete. 2024-04-25T00:09:47.052 INFO:teuthology.orchestra.run.smithi113.stdout:i"�H�)f�{'���$@@`��������H�)f�{'H�)f|�(8`H�)f�{'8`H�)f�{'��������H�)f|�����security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� 2024-04-25T00:09:47.052 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-25T00:09:47.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 Processor -- start 2024-04-25T00:09:47.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 -- start start 2024-04-25T00:09:47.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 0x55f3af4c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 0x55f3af4c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f3af4c67e0 con 0x55f3af53af90 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f3af426640 con 0x55f3af4ca850 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdd000bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f3af4c5c30 con 0x55f3af4c7350 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdcd62a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdcd62a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43954/0 (socket says 172.21.15.113:43954) 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.133+0000 7efdcd62a640 1 -- 172.21.15.113:0/30144637 learned_addr learned my addr 172.21.15.113:0/30144637 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcce29640 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 0x55f3af4c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcc628640 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 0x55f3af4c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcd62a640 1 -- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 msgr2=0x55f3af4c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcd62a640 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 0x55f3af4c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcd62a640 1 -- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 msgr2=0x55f3af4c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcd62a640 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 0x55f3af4c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcd62a640 1 -- 172.21.15.113:0/30144637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3af4c31f0 con 0x55f3af53af90 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcd62a640 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c3900 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00a870 tx=0x7efdbc005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76ece3d799483bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcb626640 1 -- 172.21.15.113:0/30144637 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc017020 con 0x55f3af53af90 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcb626640 1 -- 172.21.15.113:0/30144637 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdbc00d9c0 con 0x55f3af53af90 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdcb626640 1 -- 172.21.15.113:0/30144637 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00dd70 con 0x55f3af53af90 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 -- 172.21.15.113:0/30144637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 msgr2=0x55f3af4c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c3900 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00a870 tx=0x7efdbc005d80 comp rx=0 tx=0).stop 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 -- 172.21.15.113:0/30144637 shutdown_connections 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 0x55f3af4c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c3900 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 --2- 172.21.15.113:0/30144637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 0x55f3af4c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 -- 172.21.15.113:0/30144637 >> 172.21.15.113:0/30144637 conn(0x55f3af4b0d10 msgr2=0x55f3af4ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.134+0000 7efdd000bf40 1 -- 172.21.15.113:0/30144637 shutdown_connections 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.135+0000 7efdd000bf40 1 -- 172.21.15.113:0/30144637 wait complete. 2024-04-25T00:09:47.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.135+0000 7efdd000bf40 1 Processor -- start 2024-04-25T00:09:47.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.135+0000 7efdd000bf40 1 -- start start 2024-04-25T00:09:47.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdd000bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 0x55f3af553a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdd000bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 0x55f3af4c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdd000bf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdd000bf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f3af537e60 con 0x55f3af53af90 2024-04-25T00:09:47.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdd000bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f3af537c10 con 0x55f3af4ca850 2024-04-25T00:09:47.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdd000bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f3af53dfe0 con 0x55f3af4c7350 2024-04-25T00:09:47.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43966/0 (socket says 172.21.15.113:43966) 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 -- 172.21.15.113:0/1395027477 learned_addr learned my addr 172.21.15.113:0/1395027477 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcce29640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 0x55f3af553a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcc628640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 0x55f3af4c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 -- 172.21.15.113:0/1395027477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 msgr2=0x55f3af553a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f3af4c7350 0x55f3af553a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 -- 172.21.15.113:0/1395027477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 msgr2=0x55f3af4c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3af4ca850 0x55f3af4c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 -- 172.21.15.113:0/1395027477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3af5e54f0 con 0x55f3af53af90 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdcd62a640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c8900 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7efdbc016040 tx=0x7efdbc0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdca624640 1 -- 172.21.15.113:0/1395027477 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc012c10 con 0x55f3af53af90 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdca624640 1 -- 172.21.15.113:0/1395027477 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdbc012dd0 con 0x55f3af53af90 2024-04-25T00:09:47.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f3af5e57d0 con 0x55f3af53af90 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.136+0000 7efdca624640 1 -- 172.21.15.113:0/1395027477 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0276c0 con 0x55f3af53af90 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.137+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3af5e5d90 con 0x55f3af53af90 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.138+0000 7efdca624640 1 -- 172.21.15.113:0/1395027477 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdbc01a020 con 0x55f3af53af90 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.138+0000 7efdca624640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdb806a810 0x7efdb806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.138+0000 7efdca624640 1 -- 172.21.15.113:0/1395027477 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7efdbc08bd50 con 0x55f3af53af90 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.138+0000 7efdcce29640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdb806a810 0x7efdb806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.138+0000 7efdd000bf40 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55f3af47ffb0 0x55f3af551d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.138+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 --> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] -- osd_op(unknown.0.0:1 26.10 26:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183) -- 0x55f3af5522b0 con 0x55f3af47ffb0 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.138+0000 7efdcc628640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55f3af47ffb0 0x55f3af551d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.139+0000 7efdcce29640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdb806a810 0x7efdb806ccd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efdc4057000 tx=0x7efdc4052000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.139+0000 7efdcc628640 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55f3af47ffb0 0x55f3af551d70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdcc628640 1 -- 172.21.15.113:0/1395027477 <== osd.2 v2:172.21.15.113:6816/3097656860 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v183'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efdc00107a0 con 0x55f3af47ffb0 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55f3af47ffb0 msgr2=0x55f3af551d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x55f3af47ffb0 0x55f3af551d70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdb806a810 msgr2=0x7efdb806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7efdb806a810 0x7efdb806ccd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efdc4057000 tx=0x7efdc4052000 comp rx=0 tx=0).stop 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 msgr2=0x55f3af4c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 --2- 172.21.15.113:0/1395027477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3af53af90 0x55f3af4c8900 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7efdbc016040 tx=0x7efdbc0033a0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdcd62a640 1 -- 172.21.15.113:0/1395027477 reap_dead start 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 shutdown_connections 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 >> 172.21.15.113:0/1395027477 conn(0x55f3af4b0d10 msgr2=0x55f3af4c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 shutdown_connections 2024-04-25T00:09:47.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.140+0000 7efdd000bf40 1 -- 172.21.15.113:0/1395027477 wait complete. 2024-04-25T00:09:47.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-25T00:09:47.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.284+0000 7f83d8dee640 1 Processor -- start 2024-04-25T00:09:47.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.284+0000 7f83d8dee640 1 -- start start 2024-04-25T00:09:47.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d8dee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d415e2d0 0x7f83d415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d8dee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f83d40b4a40 0x7f83d40ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d8dee640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d415f4f0 con 0x7f83d40b3080 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d8dee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40cde00 con 0x7f83d415e2d0 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d8dee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f83d415f7d0 con 0x7f83d40b4a40 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d415e2d0 0x7f83d415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83d415e2d0 0x7f83d415ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43970/0 (socket says 172.21.15.113:43970) 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d259c640 1 -- 172.21.15.113:0/1551758807 learned_addr learned my addr 172.21.15.113:0/1551758807 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d2d9d640 1 --2- 172.21.15.113:0/1551758807 >> v2:172.21.15.178:3300/0 conn(0x7f83d40b4a40 0x7f83d40ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d259c640 1 -- 172.21.15.113:0/1551758807 >> v2:172.21.15.178:3300/0 conn(0x7f83d40b4a40 msgr2=0x7f83d40ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d259c640 1 --2- 172.21.15.113:0/1551758807 >> v2:172.21.15.178:3300/0 conn(0x7f83d40b4a40 0x7f83d40ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d259c640 1 -- 172.21.15.113:0/1551758807 >> v1:172.21.15.178:6789/0 conn(0x7f83d40b3080 legacy=0x7f83d40b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.285+0000 7f83d259c640 1 -- 172.21.15.113:0/1551758807 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d415fab0 con 0x7f83d415e2d0 2024-04-25T00:09:47.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.286+0000 7f83d259c640 1 --2- 172.21.15.113:0/1551758807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d415e2d0 0x7f83d415ebf0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f83d40d0f20 tx=0x7f83c402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc9eebea58b32e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.286+0000 7f83d159a640 1 -- 172.21.15.113:0/1551758807 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4036070 con 0x7f83d415e2d0 2024-04-25T00:09:47.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.286+0000 7f83d159a640 1 -- 172.21.15.113:0/1551758807 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c4031600 con 0x7f83d415e2d0 2024-04-25T00:09:47.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.286+0000 7f83d159a640 1 -- 172.21.15.113:0/1551758807 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40319d0 con 0x7f83d415e2d0 2024-04-25T00:09:47.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 -- 172.21.15.113:0/1551758807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d415e2d0 msgr2=0x7f83d415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 --2- 172.21.15.113:0/1551758807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d415e2d0 0x7f83d415ebf0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f83d40d0f20 tx=0x7f83c402a6f0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 -- 172.21.15.113:0/1551758807 shutdown_connections 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 --2- 172.21.15.113:0/1551758807 >> v2:172.21.15.178:3300/0 conn(0x7f83d40b4a40 0x7f83d40ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 --2- 172.21.15.113:0/1551758807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d415e2d0 0x7f83d415ebf0 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 -- 172.21.15.113:0/1551758807 >> 172.21.15.113:0/1551758807 conn(0x7f83d406b0f0 msgr2=0x7f83d406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 -- 172.21.15.113:0/1551758807 shutdown_connections 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.287+0000 7f83d8dee640 1 -- 172.21.15.113:0/1551758807 wait complete. 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.288+0000 7f83d8dee640 1 Processor -- start 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.288+0000 7f83d8dee640 1 -- start start 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d8dee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d40b3080 0x7f83d4155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d8dee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b4a40 0x7f83d4152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d8dee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d415e2d0 0x7f83d41508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d8dee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83d415f2b0 con 0x7f83d40b4a40 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d8dee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83d415f130 con 0x7f83d415e2d0 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d8dee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83d415e6b0 con 0x7f83d40b3080 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b4a40 0x7f83d4152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b4a40 0x7f83d4152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43980/0 (socket says 172.21.15.113:43980) 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 -- 172.21.15.113:0/2287927836 learned_addr learned my addr 172.21.15.113:0/2287927836 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d259c640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d40b3080 0x7f83d4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d2d9d640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d415e2d0 0x7f83d41508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 -- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d40b3080 msgr2=0x7f83d4155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d40b3080 0x7f83d4155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 -- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d415e2d0 msgr2=0x7f83d41508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d415e2d0 0x7f83d41508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 -- 172.21.15.113:0/2287927836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d4153390 con 0x7f83d40b4a40 2024-04-25T00:09:47.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d2d9d640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d415e2d0 0x7f83d41508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:47.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d259c640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d40b3080 0x7f83d4155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:47.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83d1d9b640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b4a40 0x7f83d4152850 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c430 tx=0x7f83c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.289+0000 7f83c2ffd640 1 -- 172.21.15.113:0/2287927836 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8016070 con 0x7f83d40b4a40 2024-04-25T00:09:47.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.290+0000 7f83c2ffd640 1 -- 172.21.15.113:0/2287927836 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c80108a0 con 0x7f83d40b4a40 2024-04-25T00:09:47.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.290+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d4153670 con 0x7f83d40b4a40 2024-04-25T00:09:47.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.290+0000 7f83c2ffd640 1 -- 172.21.15.113:0/2287927836 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8010cd0 con 0x7f83d40b4a40 2024-04-25T00:09:47.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.290+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d4155c90 con 0x7f83d40b4a40 2024-04-25T00:09:47.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.291+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d40691a0 con 0x7f83d40b4a40 2024-04-25T00:09:47.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.291+0000 7f83c2ffd640 1 -- 172.21.15.113:0/2287927836 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83c8019070 con 0x7f83d40b4a40 2024-04-25T00:09:47.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.291+0000 7f83c2ffd640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b406be30 0x7f83b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.292+0000 7f83d259c640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b406be30 0x7f83b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.292+0000 7f83c2ffd640 1 -- 172.21.15.113:0/2287927836 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f83c808ca20 con 0x7f83d40b4a40 2024-04-25T00:09:47.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.292+0000 7f83d259c640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b406be30 0x7f83b406e2f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f83c4033000 tx=0x7f83c4009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.296+0000 7f83c2ffd640 1 -- 172.21.15.113:0/2287927836 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83c805b400 con 0x7f83d40b4a40 2024-04-25T00:09:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.469+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) -- 0x7f83d4072050 con 0x7f83d40b4a40 2024-04-25T00:09:47.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.472+0000 7f83c2ffd640 1 -- 172.21.15.113:0/2287927836 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v33)=0 v33) ==== 140+0+0 (secure 0 0 0) 0x7f83c805ad00 con 0x7f83d40b4a40 2024-04-25T00:09:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b406be30 msgr2=0x7f83b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b406be30 0x7f83b406e2f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f83c4033000 tx=0x7f83c4009000 comp rx=0 tx=0).stop 2024-04-25T00:09:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b4a40 msgr2=0x7f83d4152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b4a40 0x7f83d4152850 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c430 tx=0x7f83c800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 shutdown_connections 2024-04-25T00:09:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f83b406be30 0x7f83b406e2f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d415e2d0 0x7f83d41508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d40b4a40 0x7f83d4152850 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 --2- 172.21.15.113:0/2287927836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d40b3080 0x7f83d4155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 >> 172.21.15.113:0/2287927836 conn(0x7f83d406b0f0 msgr2=0x7f83d40b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 shutdown_connections 2024-04-25T00:09:47.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.475+0000 7f83d8dee640 1 -- 172.21.15.113:0/2287927836 wait complete. 2024-04-25T00:09:47.486 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T00:09:47.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.584+0000 7f852ad14640 1 Processor -- start 2024-04-25T00:09:47.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.584+0000 7f852ad14640 1 -- start start 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f852ad14640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85240b5ca0 0x7f85240b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f852ad14640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85240b73c0 0x7f85240b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f852ad14640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85240d2e00 con 0x7f852415e510 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f852ad14640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f85240b6640 con 0x7f85240b5ca0 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f852ad14640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f85240d30e0 con 0x7f85240b73c0 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f8523fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85240b5ca0 0x7f85240b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f8523fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85240b5ca0 0x7f85240b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41024/0 (socket says 172.21.15.113:41024) 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.585+0000 7f8523fff640 1 -- 172.21.15.113:0/290613952 learned_addr learned my addr 172.21.15.113:0/290613952 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.586+0000 7f85292b1640 1 --2- 172.21.15.113:0/290613952 >> v2:172.21.15.178:3300/0 conn(0x7f85240b73c0 0x7f85240b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.586+0000 7f8523fff640 1 -- 172.21.15.113:0/290613952 >> v2:172.21.15.178:3300/0 conn(0x7f85240b73c0 msgr2=0x7f85240b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.586+0000 7f8523fff640 1 --2- 172.21.15.113:0/290613952 >> v2:172.21.15.178:3300/0 conn(0x7f85240b73c0 0x7f85240b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.586+0000 7f8523fff640 1 -- 172.21.15.113:0/290613952 >> v1:172.21.15.178:6789/0 conn(0x7f852415e510 legacy=0x7f85240d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:47.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.586+0000 7f8523fff640 1 -- 172.21.15.113:0/290613952 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85240d33c0 con 0x7f85240b5ca0 2024-04-25T00:09:47.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.586+0000 7f85292b1640 1 --2- 172.21.15.113:0/290613952 >> v2:172.21.15.178:3300/0 conn(0x7f85240b73c0 0x7f85240b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:47.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.586+0000 7f8523fff640 1 --2- 172.21.15.113:0/290613952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f85240b6100 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f85240d1d80 tx=0x7f851802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bded7f056ea95819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f85237fe640 1 -- 172.21.15.113:0/290613952 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518037070 con 0x7f85240b5ca0 2024-04-25T00:09:47.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f85237fe640 1 -- 172.21.15.113:0/290613952 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8518032440 con 0x7f85240b5ca0 2024-04-25T00:09:47.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f85237fe640 1 -- 172.21.15.113:0/290613952 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518032810 con 0x7f85240b5ca0 2024-04-25T00:09:47.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 -- 172.21.15.113:0/290613952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 msgr2=0x7f85240b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 --2- 172.21.15.113:0/290613952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f85240b6100 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f85240d1d80 tx=0x7f851802a1a0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 -- 172.21.15.113:0/290613952 shutdown_connections 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 --2- 172.21.15.113:0/290613952 >> v2:172.21.15.178:3300/0 conn(0x7f85240b73c0 0x7f85240b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 --2- 172.21.15.113:0/290613952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f85240b6100 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 -- 172.21.15.113:0/290613952 >> 172.21.15.113:0/290613952 conn(0x7f852406b720 msgr2=0x7f85240d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 -- 172.21.15.113:0/290613952 shutdown_connections 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.587+0000 7f852ad14640 1 -- 172.21.15.113:0/290613952 wait complete. 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.588+0000 7f852ad14640 1 Processor -- start 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.588+0000 7f852ad14640 1 -- start start 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f852ad14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f8524154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f852ad14640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85240b73c0 0x7f85240b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f852ad14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f852415e510 0x7f85240b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f852ad14640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85240d3ea0 con 0x7f85240b5ca0 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f852ad14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85240d3980 con 0x7f852415e510 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f852ad14640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85240d2a90 con 0x7f85240b73c0 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f8528ab0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f8524154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f8528ab0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f8524154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41032/0 (socket says 172.21.15.113:41032) 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f8528ab0640 1 -- 172.21.15.113:0/2174247164 learned_addr learned my addr 172.21.15.113:0/2174247164 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f85292b1640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f852415e510 0x7f85240b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.589+0000 7f8523fff640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85240b73c0 0x7f85240b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f8528ab0640 1 -- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85240b73c0 msgr2=0x7f85240b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f8528ab0640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85240b73c0 0x7f85240b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f8528ab0640 1 -- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f852415e510 msgr2=0x7f85240b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f8528ab0640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f852415e510 0x7f85240b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f8528ab0640 1 -- 172.21.15.113:0/2174247164 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85240c8d60 con 0x7f85240b5ca0 2024-04-25T00:09:47.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f8528ab0640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f8524154f30 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f851400c430 tx=0x7f851400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f85217fa640 1 -- 172.21.15.113:0/2174247164 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514011790 con 0x7f85240b5ca0 2024-04-25T00:09:47.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f85217fa640 1 -- 172.21.15.113:0/2174247164 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8514017070 con 0x7f85240b5ca0 2024-04-25T00:09:47.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85240c9320 con 0x7f85240b5ca0 2024-04-25T00:09:47.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f85217fa640 1 -- 172.21.15.113:0/2174247164 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514011b70 con 0x7f85240b5ca0 2024-04-25T00:09:47.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.590+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8524155470 con 0x7f85240b5ca0 2024-04-25T00:09:47.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.591+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85240691a0 con 0x7f85240b5ca0 2024-04-25T00:09:47.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.591+0000 7f85217fa640 1 -- 172.21.15.113:0/2174247164 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8514011d10 con 0x7f85240b5ca0 2024-04-25T00:09:47.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.592+0000 7f85217fa640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f851006be30 0x7f851006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:47.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.592+0000 7f8523fff640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f851006be30 0x7f851006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:47.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.592+0000 7f85217fa640 1 -- 172.21.15.113:0/2174247164 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f851408ab60 con 0x7f85240b5ca0 2024-04-25T00:09:47.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.592+0000 7f8523fff640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f851006be30 0x7f851006e2f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8518034000 tx=0x7f851802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:47.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.596+0000 7f85217fa640 1 -- 172.21.15.113:0/2174247164 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8514058ef0 con 0x7f85240b5ca0 2024-04-25T00:09:47.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.803+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f85240cec00 con 0x7f85240b5ca0 2024-04-25T00:09:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.984+0000 7f85217fa640 1 -- 172.21.15.113:0/2174247164 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v100) ==== 162+0+0 (secure 0 0 0) 0x7f851405d1f0 con 0x7f85240b5ca0 2024-04-25T00:09:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f851006be30 msgr2=0x7f851006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f851006be30 0x7f851006e2f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8518034000 tx=0x7f851802e000 comp rx=0 tx=0).stop 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 msgr2=0x7f8524154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f8524154f30 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f851400c430 tx=0x7f851400c8f0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 shutdown_connections 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f851006be30 0x7f851006e2f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f852415e510 0x7f85240b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85240b73c0 0x7f85240b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 --2- 172.21.15.113:0/2174247164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85240b5ca0 0x7f8524154f30 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 >> 172.21.15.113:0/2174247164 conn(0x7f852406b720 msgr2=0x7f85240b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 shutdown_connections 2024-04-25T00:09:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:47.986+0000 7f852ad14640 1 -- 172.21.15.113:0/2174247164 wait complete. 2024-04-25T00:09:48.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.096+0000 7f5807e2a640 1 Processor -- start 2024-04-25T00:09:48.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.096+0000 7f5807e2a640 1 -- start start 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.097+0000 7f5807e2a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58000b6810 0x7f5800168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.097+0000 7f5807e2a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580015e6a0 con 0x7f580015e2c0 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.097+0000 7f5807e2a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58000d2c80 con 0x7f58000b9c90 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.097+0000 7f5807e2a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f58000d2f60 con 0x7f58000b6810 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.097+0000 7f58063c7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58000b6810 0x7f5800168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.097+0000 7f5805bc6640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f580015e2c0 0x7f58000d24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46340/0 (socket says 172.21.15.113:46340) 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.097+0000 7f5805bc6640 1 -- 172.21.15.113:0/3550904525 learned_addr learned my addr 172.21.15.113:0/3550904525 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f58063c7640 1 -- 172.21.15.113:0/3550904525 >> v1:172.21.15.178:6789/0 conn(0x7f58000b9c90 legacy=0x7f58000b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f58063c7640 1 -- 172.21.15.113:0/3550904525 >> v1:172.21.15.113:6789/0 conn(0x7f580015e2c0 legacy=0x7f58000d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f5804bc4640 1 -- 172.21.15.113:0/3550904525 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133088723 0 0) 0x7f580015e6a0 con 0x7f580015e2c0 2024-04-25T00:09:48.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f58063c7640 1 -- 172.21.15.113:0/3550904525 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58000d3240 con 0x7f58000b6810 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f58063c7640 1 --2- 172.21.15.113:0/3550904525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b6810 0x7f5800168e20 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48256dd002047eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f5804bc4640 1 -- 172.21.15.113:0/3550904525 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8031600 con 0x7f58000b6810 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f5804bc4640 1 -- 172.21.15.113:0/3550904525 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f80317a0 con 0x7f58000b6810 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.098+0000 7f5804bc4640 1 -- 172.21.15.113:0/3550904525 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8031b70 con 0x7f58000b6810 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.099+0000 7f5807e2a640 1 -- 172.21.15.113:0/3550904525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b6810 msgr2=0x7f5800168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.099+0000 7f5807e2a640 1 --2- 172.21.15.113:0/3550904525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b6810 0x7f5800168e20 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f802a610 comp rx=0 tx=0).stop 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.099+0000 7f5807e2a640 1 -- 172.21.15.113:0/3550904525 shutdown_connections 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.099+0000 7f5807e2a640 1 --2- 172.21.15.113:0/3550904525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b6810 0x7f5800168e20 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.099+0000 7f5807e2a640 1 -- 172.21.15.113:0/3550904525 >> 172.21.15.113:0/3550904525 conn(0x7f580006b510 msgr2=0x7f580006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.099+0000 7f5807e2a640 1 -- 172.21.15.113:0/3550904525 shutdown_connections 2024-04-25T00:09:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.099+0000 7f5807e2a640 1 -- 172.21.15.113:0/3550904525 wait complete. 2024-04-25T00:09:48.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.100+0000 7f5807e2a640 1 Processor -- start 2024-04-25T00:09:48.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.100+0000 7f5807e2a640 1 -- start start 2024-04-25T00:09:48.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f5807e2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58000b6810 0x7f5800159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f5807e2a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b9c90 0x7f58000d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f5807e2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f580015e2c0 0x7f580015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f5807e2a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58000b8210 con 0x7f58000b9c90 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f5807e2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58000d3d20 con 0x7f580015e2c0 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f5807e2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58000b8390 con 0x7f58000b6810 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b9c90 0x7f58000d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b9c90 0x7f58000d42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41048/0 (socket says 172.21.15.113:41048) 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 -- 172.21.15.113:0/2973766372 learned_addr learned my addr 172.21.15.113:0/2973766372 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58063c7640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f580015e2c0 0x7f580015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 -- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58000b6810 msgr2=0x7f5800159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f5805bc6640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58000b6810 0x7f5800159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58000b6810 0x7f5800159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 -- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f580015e2c0 msgr2=0x7f580015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f580015e2c0 0x7f580015b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58053c5640 1 -- 172.21.15.113:0/2973766372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001f7670 con 0x7f58000b9c90 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.101+0000 7f58063c7640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f580015e2c0 0x7f580015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.102+0000 7f58053c5640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b9c90 0x7f58000d42d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c430 tx=0x7f57f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.102+0000 7f57f67fc640 1 -- 172.21.15.113:0/2973766372 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0011790 con 0x7f58000b9c90 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.102+0000 7f57f67fc640 1 -- 172.21.15.113:0/2973766372 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0017070 con 0x7f58000b9c90 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.102+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001f7970 con 0x7f58000b9c90 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.102+0000 7f57f67fc640 1 -- 172.21.15.113:0/2973766372 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0011bb0 con 0x7f58000b9c90 2024-04-25T00:09:48.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.102+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001f81b0 con 0x7f58000b9c90 2024-04-25T00:09:48.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.103+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5800072540 con 0x7f58000b9c90 2024-04-25T00:09:48.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.103+0000 7f57f67fc640 1 -- 172.21.15.113:0/2973766372 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57f001c460 con 0x7f58000b9c90 2024-04-25T00:09:48.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.103+0000 7f57f67fc640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f57e406be60 0x7f57e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.104+0000 7f57f67fc640 1 -- 172.21.15.113:0/2973766372 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f57f008ad90 con 0x7f58000b9c90 2024-04-25T00:09:48.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.104+0000 7f5805bc6640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f57e406be60 0x7f57e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.104+0000 7f5805bc6640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f57e406be60 0x7f57e406e320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000f60 tx=0x7f57fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.108+0000 7f57f67fc640 1 -- 172.21.15.113:0/2973766372 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5800072540 con 0x7f58000b9c90 2024-04-25T00:09:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.317+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58000d21b0 con 0x7f58000b9c90 2024-04-25T00:09:48.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.318+0000 7f57f67fc640 1 -- 172.21.15.113:0/2973766372 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5200 (secure 0 0 0) 0x7f58000d21b0 con 0x7f58000b9c90 2024-04-25T00:09:48.318 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:48.319 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":101,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:47.987127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-04-25T00:09:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 101 2024-04-25T00:09:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f57e406be60 msgr2=0x7f57e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f57e406be60 0x7f57e406e320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000f60 tx=0x7f57fc009040 comp rx=0 tx=0).stop 2024-04-25T00:09:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b9c90 msgr2=0x7f58000d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b9c90 0x7f58000d42d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c430 tx=0x7f57f000c8f0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 shutdown_connections 2024-04-25T00:09:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f57e406be60 0x7f57e406e320 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f580015e2c0 0x7f580015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58000b9c90 0x7f58000d42d0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 --2- 172.21.15.113:0/2973766372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58000b6810 0x7f5800159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 >> 172.21.15.113:0/2973766372 conn(0x7f580006b510 msgr2=0x7f58000d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 shutdown_connections 2024-04-25T00:09:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:48.320+0000 7f5807e2a640 1 -- 172.21.15.113:0/2973766372 wait complete. 2024-04-25T00:09:48.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:09:33.215001+0000', 'modified': '2024-04-25T00:09:47.987127+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 183, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10794': {'gid': 10794, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.113:6836/2086115851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 2086115851}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 2086115851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T00:09:48.734 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10794:up:replay 2024-04-25T00:09:49.736 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:49.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.845+0000 7f0223cbd640 1 Processor -- start 2024-04-25T00:09:49.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.846+0000 7f0223cbd640 1 -- start start 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.846+0000 7f0223cbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 0x7f021c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.846+0000 7f0223cbd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f021c0d3de0 con 0x7f021c0d2290 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.846+0000 7f0223cbd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f021c0d2800 con 0x7f021c0d0b00 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.846+0000 7f0223cbd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f021c0d3410 con 0x7f021c15e320 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0221258640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f021c0d2290 0x7f021c0b9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46348/0 (socket says 172.21.15.113:46348) 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0221258640 1 -- 172.21.15.113:0/2029621719 learned_addr learned my addr 172.21.15.113:0/2029621719 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0221a59640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 0x7f021c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724879329 0 0) 0x7f021c0d3de0 con 0x7f021c0d2290 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0208003520 con 0x7f021c0d2290 2024-04-25T00:09:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926942769 0 0) 0x7f021c0d2800 con 0x7f021c0d0b00 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f021c0d3de0 con 0x7f021c0d0b00 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2246474585 0 0) 0x7f0208003520 con 0x7f021c0d2290 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 msgr2=0x7f021c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 0x7f021c0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 >> v1:172.21.15.178:6789/0 conn(0x7f021c0d0b00 legacy=0x7f021c0b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c0d2ae0 con 0x7f021c0d2290 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f020c001670 con 0x7f021c0d2290 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 >> v1:172.21.15.113:6789/0 conn(0x7f021c0d2290 legacy=0x7f021c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 0x7f0208003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02080040d0 0x7f0208004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0208004d40 0x7f02080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f021c0d3410 con 0x7f02080040d0 2024-04-25T00:09:49.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f020800baa0 con 0x7f0208004d40 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f020800bca0 con 0x7f021c15e320 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0221258640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02080040d0 0x7f0208004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f0221a59640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 0x7f0208003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.847+0000 7f022225a640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0208004d40 0x7f02080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0221258640 1 -- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 msgr2=0x7f0208003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0221258640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c15e320 0x7f0208003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0221258640 1 -- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0208004d40 msgr2=0x7f02080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0221258640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0208004d40 0x7f02080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0221258640 1 -- 172.21.15.113:0/2029621719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0208003520 con 0x7f02080040d0 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f022225a640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0208004d40 0x7f02080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0221258640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02080040d0 0x7f0208004800 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f020c00c190 tx=0x7f0208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32530c0f0b2de05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c007ce0 con 0x7f02080040d0 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020c005ad0 con 0x7f02080040d0 2024-04-25T00:09:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.848+0000 7f0220a57640 1 -- 172.21.15.113:0/2029621719 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c007070 con 0x7f02080040d0 2024-04-25T00:09:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.849+0000 7f0223cbd640 1 -- 172.21.15.113:0/2029621719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02080040d0 msgr2=0x7f0208004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.849+0000 7f0223cbd640 1 --2- 172.21.15.113:0/2029621719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02080040d0 0x7f0208004800 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f020c00c190 tx=0x7f0208005830 comp rx=0 tx=0).stop 2024-04-25T00:09:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.849+0000 7f022225a640 1 -- 172.21.15.113:0/2029621719 reap_dead start 2024-04-25T00:09:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.849+0000 7f0223cbd640 1 -- 172.21.15.113:0/2029621719 shutdown_connections 2024-04-25T00:09:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.849+0000 7f0223cbd640 1 -- 172.21.15.113:0/2029621719 >> 172.21.15.113:0/2029621719 conn(0x7f021c0b5c20 msgr2=0x7f021c0b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.849+0000 7f0223cbd640 1 -- 172.21.15.113:0/2029621719 shutdown_connections 2024-04-25T00:09:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.849+0000 7f0223cbd640 1 -- 172.21.15.113:0/2029621719 wait complete. 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.850+0000 7f0223cbd640 1 Processor -- start 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.850+0000 7f0223cbd640 1 -- start start 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0223cbd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02080040d0 0x7f021c0b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0223cbd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c153610 0x7f021c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0223cbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c0b0670 0x7f021c0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0223cbd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f021c15e700 con 0x7f021c153610 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0223cbd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f021c0d30a0 con 0x7f02080040d0 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0223cbd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f021c15e880 con 0x7f021c0b0670 2024-04-25T00:09:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c153610 0x7f021c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c153610 0x7f021c0b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41072/0 (socket says 172.21.15.113:41072) 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 -- 172.21.15.113:0/827640759 learned_addr learned my addr 172.21.15.113:0/827640759 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f022225a640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c0b0670 0x7f021c0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221a59640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02080040d0 0x7f021c0b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 -- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c0b0670 msgr2=0x7f021c0ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c0b0670 0x7f021c0ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 -- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02080040d0 msgr2=0x7f021c0b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02080040d0 0x7f021c0b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.851+0000 7f0221258640 1 -- 172.21.15.113:0/827640759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c0b2260 con 0x7f021c153610 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.852+0000 7f0221258640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c153610 0x7f021c0b0130 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f020c00c190 tx=0x7f020c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.852+0000 7f02127fc640 1 -- 172.21.15.113:0/827640759 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c001670 con 0x7f021c153610 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.852+0000 7f02127fc640 1 -- 172.21.15.113:0/827640759 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020c042070 con 0x7f021c153610 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.852+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c0b0c80 con 0x7f021c153610 2024-04-25T00:09:49.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.852+0000 7f02127fc640 1 -- 172.21.15.113:0/827640759 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c0072d0 con 0x7f021c153610 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.852+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c0b1240 con 0x7f021c153610 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.853+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c0691a0 con 0x7f021c153610 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.853+0000 7f02127fc640 1 -- 172.21.15.113:0/827640759 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f020c007470 con 0x7f021c153610 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.854+0000 7f02127fc640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f020006be30 0x7f020006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.854+0000 7f0221a59640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f020006be30 0x7f020006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.854+0000 7f02127fc640 1 -- 172.21.15.113:0/827640759 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f020c0aaea0 con 0x7f021c153610 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.854+0000 7f0221a59640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f020006be30 0x7f020006e2f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0218014430 tx=0x7f021800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:49.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:49.858+0000 7f02127fc640 1 -- 172.21.15.113:0/827640759 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f020c005aa0 con 0x7f021c153610 2024-04-25T00:09:50.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.070+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f021c0b47f0 con 0x7f021c153610 2024-04-25T00:09:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.071+0000 7f02127fc640 1 -- 172.21.15.113:0/827640759 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5200 (secure 0 0 0) 0x7f020c0791b0 con 0x7f021c153610 2024-04-25T00:09:50.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:50.071 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":103,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:48.992268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-04-25T00:09:50.072 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 103 2024-04-25T00:09:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f020006be30 msgr2=0x7f020006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f020006be30 0x7f020006e2f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0218014430 tx=0x7f021800b040 comp rx=0 tx=0).stop 2024-04-25T00:09:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c153610 msgr2=0x7f021c0b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c153610 0x7f021c0b0130 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f020c00c190 tx=0x7f020c0019c0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 shutdown_connections 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f020006be30 0x7f020006e2f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f021c0b0670 0x7f021c0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c153610 0x7f021c0b0130 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 --2- 172.21.15.113:0/827640759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02080040d0 0x7f021c0b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 >> 172.21.15.113:0/827640759 conn(0x7f021c0b5c20 msgr2=0x7f021c0b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 shutdown_connections 2024-04-25T00:09:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:50.073+0000 7f0223cbd640 1 -- 172.21.15.113:0/827640759 wait complete. 2024-04-25T00:09:50.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:09:33.215001+0000', 'modified': '2024-04-25T00:09:48.992268+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 183, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10794': {'gid': 10794, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.113:6836/2086115851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 2086115851}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 2086115851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T00:09:50.487 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10794:up:rejoin 2024-04-25T00:09:51.488 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:09:51.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.596+0000 7f30f4864640 1 Processor -- start 2024-04-25T00:09:51.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.596+0000 7f30f4864640 1 -- start start 2024-04-25T00:09:51.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.597+0000 7f30f4864640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f30f006bc50 0x7f30f006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:51.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.597+0000 7f30f4864640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f0161fe0 0x7f30f006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:51.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.597+0000 7f30f4864640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f01624e0 con 0x7f30f006b0f0 2024-04-25T00:09:51.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.597+0000 7f30f4864640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f30f01627c0 con 0x7f30f006bc50 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.597+0000 7f30f4864640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f30f0162aa0 con 0x7f30f0161fe0 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30ee59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f30f006bc50 0x7f30f006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30eed9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f30f006b0f0 0x7f30f00709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46352/0 (socket says 172.21.15.113:46352) 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30eed9d640 1 -- 172.21.15.113:0/593676174 learned_addr learned my addr 172.21.15.113:0/593676174 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30edd9b640 1 --2- 172.21.15.113:0/593676174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f0161fe0 0x7f30f006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30ee59c640 1 -- 172.21.15.113:0/593676174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f0161fe0 msgr2=0x7f30f006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30ee59c640 1 --2- 172.21.15.113:0/593676174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f0161fe0 0x7f30f006c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30ed59a640 1 -- 172.21.15.113:0/593676174 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858490258 0 0) 0x7f30f01624e0 con 0x7f30f006b0f0 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30ee59c640 1 -- 172.21.15.113:0/593676174 >> v1:172.21.15.113:6789/0 conn(0x7f30f006b0f0 legacy=0x7f30f00709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.598+0000 7f30ee59c640 1 -- 172.21.15.113:0/593676174 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f0162d80 con 0x7f30f006bc50 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30ee59c640 1 --2- 172.21.15.113:0/593676174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f006bc50 0x7f30f006cc70 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009d90 tx=0x7f30e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e33673a08e3776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30ed59a640 1 -- 172.21.15.113:0/593676174 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0007740 con 0x7f30f006bc50 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30ed59a640 1 -- 172.21.15.113:0/593676174 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e00078e0 con 0x7f30f006bc50 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30ed59a640 1 -- 172.21.15.113:0/593676174 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0007cb0 con 0x7f30f006bc50 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 -- 172.21.15.113:0/593676174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f006bc50 msgr2=0x7f30f006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 --2- 172.21.15.113:0/593676174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f006bc50 0x7f30f006cc70 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009d90 tx=0x7f30e002a660 comp rx=0 tx=0).stop 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 -- 172.21.15.113:0/593676174 shutdown_connections 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 --2- 172.21.15.113:0/593676174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f0161fe0 0x7f30f006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 --2- 172.21.15.113:0/593676174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f006bc50 0x7f30f006cc70 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 -- 172.21.15.113:0/593676174 >> 172.21.15.113:0/593676174 conn(0x7f30f00b3a70 msgr2=0x7f30f00b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 -- 172.21.15.113:0/593676174 shutdown_connections 2024-04-25T00:09:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.599+0000 7f30f4864640 1 -- 172.21.15.113:0/593676174 wait complete. 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.601+0000 7f30f4864640 1 Processor -- start 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.601+0000 7f30f4864640 1 -- start start 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.601+0000 7f30f4864640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f006b0f0 0x7f30f015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.601+0000 7f30f4864640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f006bc50 0x7f30f00c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30f4864640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f0161fe0 0x7f30f00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30f4864640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00710b0 con 0x7f30f0161fe0 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30f4864640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0071940 con 0x7f30f006b0f0 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30f4864640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f30f0071230 con 0x7f30f006bc50 2024-04-25T00:09:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f0161fe0 0x7f30f00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f0161fe0 0x7f30f00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41090/0 (socket says 172.21.15.113:41090) 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 -- 172.21.15.113:0/2469465489 learned_addr learned my addr 172.21.15.113:0/2469465489 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30ee59c640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f006b0f0 0x7f30f015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30edd9b640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f006bc50 0x7f30f00c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 -- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f006bc50 msgr2=0x7f30f00c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f006bc50 0x7f30f00c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 -- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f006b0f0 msgr2=0x7f30f015b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f006b0f0 0x7f30f015b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 -- 172.21.15.113:0/2469465489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f00c9a60 con 0x7f30f0161fe0 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30ee59c640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f006b0f0 0x7f30f015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30eed9d640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f0161fe0 0x7f30f00cd3f0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c410 tx=0x7f30e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30deffd640 1 -- 172.21.15.113:0/2469465489 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80074e0 con 0x7f30f0161fe0 2024-04-25T00:09:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30deffd640 1 -- 172.21.15.113:0/2469465489 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e80047e0 con 0x7f30f0161fe0 2024-04-25T00:09:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f00c9d40 con 0x7f30f0161fe0 2024-04-25T00:09:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30deffd640 1 -- 172.21.15.113:0/2469465489 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8004bd0 con 0x7f30f0161fe0 2024-04-25T00:09:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.602+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f00ca300 con 0x7f30f0161fe0 2024-04-25T00:09:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.603+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f00691a0 con 0x7f30f0161fe0 2024-04-25T00:09:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.604+0000 7f30deffd640 1 -- 172.21.15.113:0/2469465489 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30e8011070 con 0x7f30f0161fe0 2024-04-25T00:09:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.604+0000 7f30deffd640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f30d006be30 0x7f30d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.604+0000 7f30deffd640 1 -- 172.21.15.113:0/2469465489 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f30e808aa30 con 0x7f30f0161fe0 2024-04-25T00:09:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.604+0000 7f30ee59c640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f30d006be30 0x7f30d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.605+0000 7f30ee59c640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f30d006be30 0x7f30d006e2f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f30e0034000 tx=0x7f30e0033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.608+0000 7f30deffd640 1 -- 172.21.15.113:0/2469465489 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30e801a2b0 con 0x7f30f0161fe0 2024-04-25T00:09:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.817+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30f00714d0 con 0x7f30f0161fe0 2024-04-25T00:09:51.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.817+0000 7f30deffd640 1 -- 172.21.15.113:0/2469465489 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7f30e8059410 con 0x7f30f0161fe0 2024-04-25T00:09:51.818 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:09:51.818 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:50.995654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10794,"qdb_cluster":[10794]},"id":13}]} 2024-04-25T00:09:51.818 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f30d006be30 msgr2=0x7f30d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f30d006be30 0x7f30d006e2f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f30e0034000 tx=0x7f30e0033040 comp rx=0 tx=0).stop 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f0161fe0 msgr2=0x7f30f00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f0161fe0 0x7f30f00cd3f0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c410 tx=0x7f30e800c8d0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 shutdown_connections 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f30d006be30 0x7f30d006e2f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30f0161fe0 0x7f30f00cd3f0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30f006bc50 0x7f30f00c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 --2- 172.21.15.113:0/2469465489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f006b0f0 0x7f30f015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 >> 172.21.15.113:0/2469465489 conn(0x7f30f00b3a70 msgr2=0x7f30f00709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 shutdown_connections 2024-04-25T00:09:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:51.819+0000 7f30f4864640 1 -- 172.21.15.113:0/2469465489 wait complete. 2024-04-25T00:09:52.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:09:33.215001+0000', 'modified': '2024-04-25T00:09:50.995654+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 183, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10794': {'gid': 10794, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.113:6836/2086115851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 2086115851}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 2086115851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10794, 'qdb_cluster': [10794]} 2024-04-25T00:09:52.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:09:52.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:09:52.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.331+0000 7f94bc89e640 1 Processor -- start 2024-04-25T00:09:52.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.331+0000 7f94bc89e640 1 -- start start 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.331+0000 7f94bc89e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f94b815e2d0 0x7f94b815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94bc89e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f94b80b3080 0x7f94b80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94bc89e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b815f4f0 con 0x7f94b815e6b0 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94bc89e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f94b80cde00 con 0x7f94b80b3080 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94bc89e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f94b815f7d0 con 0x7f94b815e2d0 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94add9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f94b80b3080 0x7f94b80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94add9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f94b80b3080 0x7f94b80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41092/0 (socket says 172.21.15.113:41092) 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94add9b640 1 -- 172.21.15.113:0/2990801137 learned_addr learned my addr 172.21.15.113:0/2990801137 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94b659c640 1 --2- 172.21.15.113:0/2990801137 >> v2:172.21.15.178:3300/0 conn(0x7f94b815e2d0 0x7f94b815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94add9b640 1 -- 172.21.15.113:0/2990801137 >> v2:172.21.15.178:3300/0 conn(0x7f94b815e2d0 msgr2=0x7f94b815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94add9b640 1 --2- 172.21.15.113:0/2990801137 >> v2:172.21.15.178:3300/0 conn(0x7f94b815e2d0 0x7f94b815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94add9b640 1 -- 172.21.15.113:0/2990801137 >> v1:172.21.15.178:6789/0 conn(0x7f94b815e6b0 legacy=0x7f94b8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94add9b640 1 -- 172.21.15.113:0/2990801137 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b815fab0 con 0x7f94b80b3080 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.332+0000 7f94b659c640 1 --2- 172.21.15.113:0/2990801137 >> v2:172.21.15.178:3300/0 conn(0x7f94b815e2d0 0x7f94b815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94add9b640 1 --2- 172.21.15.113:0/2990801137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b80b3080 0x7f94b80b4330 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f94b80b2d40 tx=0x7f94a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b886b3a044e6d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94b5d9b640 1 -- 172.21.15.113:0/2990801137 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0037070 con 0x7f94b80b3080 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94b5d9b640 1 -- 172.21.15.113:0/2990801137 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a0032440 con 0x7f94b80b3080 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94b5d9b640 1 -- 172.21.15.113:0/2990801137 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0032810 con 0x7f94b80b3080 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 -- 172.21.15.113:0/2990801137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b80b3080 msgr2=0x7f94b80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 --2- 172.21.15.113:0/2990801137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b80b3080 0x7f94b80b4330 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f94b80b2d40 tx=0x7f94a002a230 comp rx=0 tx=0).stop 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 -- 172.21.15.113:0/2990801137 shutdown_connections 2024-04-25T00:09:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 --2- 172.21.15.113:0/2990801137 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b80b3080 0x7f94b80b4330 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 --2- 172.21.15.113:0/2990801137 >> v2:172.21.15.178:3300/0 conn(0x7f94b815e2d0 0x7f94b815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 -- 172.21.15.113:0/2990801137 >> 172.21.15.113:0/2990801137 conn(0x7f94b806b0f0 msgr2=0x7f94b806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 -- 172.21.15.113:0/2990801137 shutdown_connections 2024-04-25T00:09:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.333+0000 7f94bc89e640 1 -- 172.21.15.113:0/2990801137 wait complete. 2024-04-25T00:09:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.335+0000 7f94bc89e640 1 Processor -- start 2024-04-25T00:09:52.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.335+0000 7f94bc89e640 1 -- start start 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94b80b3080 0x7f94b8143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94b815e2d0 0x7f94b815b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b815e6b0 0x7f94b8153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94b815f2b0 con 0x7f94b815e6b0 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94b815f130 con 0x7f94b815e2d0 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80cf2a0 con 0x7f94b80b3080 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b815e6b0 0x7f94b8153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b815e6b0 0x7f94b8153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41104/0 (socket says 172.21.15.113:41104) 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3887724795 learned_addr learned my addr 172.21.15.113:0/3887724795 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b659c640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94b80b3080 0x7f94b8143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94add9b640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94b815e2d0 0x7f94b815b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94b80b3080 msgr2=0x7f94b8143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94b80b3080 0x7f94b8143ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94b815e2d0 msgr2=0x7f94b815b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94b815e2d0 0x7f94b815b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3887724795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b8144020 con 0x7f94b815e6b0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b815e6b0 0x7f94b8153050 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f94b000b970 tx=0x7f94b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94af7fe640 1 -- 172.21.15.113:0/3887724795 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0019070 con 0x7f94b815e6b0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94af7fe640 1 -- 172.21.15.113:0/3887724795 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b000cb40 con 0x7f94b815e6b0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b815ba30 con 0x7f94b815e6b0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94af7fe640 1 -- 172.21.15.113:0/3887724795 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0014440 con 0x7f94b815e6b0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.336+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b8156100 con 0x7f94b815e6b0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.338+0000 7f94af7fe640 1 -- 172.21.15.113:0/3887724795 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94b001c070 con 0x7f94b815e6b0 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.338+0000 7f94af7fe640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949806be30 0x7f949806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.338+0000 7f94af7fe640 1 -- 172.21.15.113:0/3887724795 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f94b008aaa0 con 0x7f94b815e6b0 2024-04-25T00:09:52.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.338+0000 7f94b659c640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949806be30 0x7f949806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.339+0000 7f94b659c640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949806be30 0x7f949806e2f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f94a800ab60 tx=0x7f94a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.339+0000 7f94bf5af740 1 Processor -- start 2024-04-25T00:09:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.339+0000 7f94bf5af740 1 -- start start 2024-04-25T00:09:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b90f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9125c80 0x5630b9125000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125540 0x5630b90fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94bf5af740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5630b905df00 con 0x5630b9125540 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5630b9006750 con 0x5630b9125c80 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5630b9059ec0 con 0x5630b9142b60 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94b6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125540 0x5630b90fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94b6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125540 0x5630b90fa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41120/0 (socket says 172.21.15.113:41120) 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3069052491 learned_addr learned my addr 172.21.15.113:0/3069052491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3069052491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 msgr2=0x5630b90f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b90f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3069052491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9125c80 msgr2=0x5630b9125000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94add9b640 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9125c80 0x5630b9125000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.340+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9125c80 0x5630b9125000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94b6d9d640 1 -- 172.21.15.113:0/3069052491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b8d722d0 con 0x5630b9125540 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125540 0x5630b90fa350 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f94b0088f10 tx=0x7f94b0055140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7284ebff1f6c2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f9497fff640 1 -- 172.21.15.113:0/3069052491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00570a0 con 0x5630b9125540 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f9497fff640 1 -- 172.21.15.113:0/3069052491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b006e070 con 0x5630b9125540 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f9497fff640 1 -- 172.21.15.113:0/3069052491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0061440 con 0x5630b9125540 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94bf5af740 1 -- 172.21.15.113:0/3069052491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125540 msgr2=0x5630b90fa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125540 0x5630b90fa350 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f94b0088f10 tx=0x7f94b0055140 comp rx=0 tx=0).stop 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94bf5af740 1 -- 172.21.15.113:0/3069052491 shutdown_connections 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9125c80 0x5630b9125000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125540 0x5630b90fa350 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3069052491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b90f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.341+0000 7f94bf5af740 1 -- 172.21.15.113:0/3069052491 >> 172.21.15.113:0/3069052491 conn(0x5630b90f7760 msgr2=0x5630b9140000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.342+0000 7f94bf5af740 1 -- 172.21.15.113:0/3069052491 shutdown_connections 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.342+0000 7f94bf5af740 1 -- 172.21.15.113:0/3069052491 wait complete. 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.342+0000 7f94bf5af740 1 -- start start 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.342+0000 7f94bf5af740 1 Processor -- start 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b913e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9142b60 0x5630b9151230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b9141a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94bf5af740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5630b90f9910 con 0x5630b9142b60 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5630b9019cf0 con 0x5630b91415c0 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5630b90f8d00 con 0x5630b9125c80 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94bf5af740 10 client.0 _ll_register_callbacks cb 0x5630b90196e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9142b60 0x5630b9151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9142b60 0x5630b9151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41134/0 (socket says 172.21.15.113:41134) 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 -- 172.21.15.113:0/3687336389 learned_addr learned my addr 172.21.15.113:0/3687336389 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 -- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 msgr2=0x5630b913e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b9141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94b659c640 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b913e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b913e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 -- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 msgr2=0x5630b9141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b9141a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.343+0000 7f94add9b640 1 -- 172.21.15.113:0/3687336389 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b8e3c6b0 con 0x5630b9142b60 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94add9b640 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9142b60 0x5630b9151230 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f94a0035040 tx=0x7f94a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d5a872b0f5f0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94957fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94957fa640 1 -- 172.21.15.113:0/3687336389 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0043070 con 0x5630b9142b60 2024-04-25T00:09:52.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94957fa640 1 -- 172.21.15.113:0/3687336389 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f94a003c440 con 0x5630b9142b60 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b006e070 con 0x5630b9142b60 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94957fa640 1 -- 172.21.15.113:0/3687336389 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f94a004e050 con 0x5630b9142b60 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 10 client.11076 fetch_fsmap learned FSMap version 104 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5630b9127f40 con 0x5630b9142b60 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94957fa640 1 -- 172.21.15.113:0/3687336389 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b006e070 con 0x5630b9142b60 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94957fa640 1 -- 172.21.15.113:0/3687336389 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f94a0051020 con 0x5630b9142b60 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 10 client.11076 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 10 client.11076 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 20 client.11076 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b915c660 0x5630b915ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 4 client.11076 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:52.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b906b440 con 0x5630b915c660 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.344+0000 7f94b659c640 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b915c660 0x5630b915ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.345+0000 7f94957fa640 10 client.11076 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.345+0000 7f94b659c640 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b915c660 0x5630b915ea50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce4170a91bbb6a8 server_cookie=f6d7198b6c439428 in_seq=0 out_seq=0 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.345+0000 7f94957fa640 1 -- 172.21.15.113:0/3687336389 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630b906b440 con 0x5630b915c660 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.345+0000 7f94957fa640 10 client.11076 handle_command_reply: tid=0 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.345+0000 7f94bf5af740 1 client.11076 shutdown 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.345+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9142b60 msgr2=0x5630b9151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.345+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9142b60 0x5630b9151230 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f94a0035040 tx=0x7f94a0004730 comp rx=0 tx=0).stop 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 shutdown_connections 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b915c660 0x5630b915ea50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b9141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9142b60 0x5630b9151230 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94957fa640 0 client.11076 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3687336389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b913e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 >> 172.21.15.113:0/3687336389 conn(0x5630b9125540 msgr2=0x5630b90f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 shutdown_connections 2024-04-25T00:09:52.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 1 -- 172.21.15.113:0/3687336389 wait complete. 2024-04-25T00:09:52.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.346+0000 7f94bf5af740 20 client.11076 trim_cache size 0 max 16384 2024-04-25T00:09:52.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 Processor -- start 2024-04-25T00:09:52.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 -- start start 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125c80 0x5630b9128960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b913e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b913e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94b81fd900 con 0x5630b9125c80 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94b8156ca0 con 0x5630b91415c0 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94b81572f0 con 0x5630b9142b60 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94b659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125c80 0x5630b9128960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94b659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125c80 0x5630b9128960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41146/0 (socket says 172.21.15.113:41146) 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.364+0000 7f94b659c640 1 -- 172.21.15.113:0/1830983128 learned_addr learned my addr 172.21.15.113:0/1830983128 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b913e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94add9b640 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b913e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b659c640 1 -- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 msgr2=0x5630b913e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b659c640 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b913e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b659c640 1 -- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 msgr2=0x5630b913e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b659c640 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b913e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b659c640 1 -- 172.21.15.113:0/1830983128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b9150900 con 0x5630b9125c80 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b913e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94add9b640 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b913e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94b659c640 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125c80 0x5630b9128960 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f94a800ffb0 tx=0x7f94a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5085a837abb85fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94977fe640 1 -- 172.21.15.113:0/1830983128 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a802b070 con 0x5630b9125c80 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94977fe640 1 -- 172.21.15.113:0/1830983128 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a8025e50 con 0x5630b9125c80 2024-04-25T00:09:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94977fe640 1 -- 172.21.15.113:0/1830983128 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a801b070 con 0x5630b9125c80 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94bf5af740 1 -- 172.21.15.113:0/1830983128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125c80 msgr2=0x5630b9128960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94bf5af740 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125c80 0x5630b9128960 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f94a800ffb0 tx=0x7f94a8004730 comp rx=0 tx=0).stop 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94bf5af740 1 -- 172.21.15.113:0/1830983128 shutdown_connections 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94bf5af740 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b91415c0 0x5630b913e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94bf5af740 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9142b60 0x5630b913e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94bf5af740 1 --2- 172.21.15.113:0/1830983128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b9125c80 0x5630b9128960 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.365+0000 7f94bf5af740 1 -- 172.21.15.113:0/1830983128 >> 172.21.15.113:0/1830983128 conn(0x5630b9125540 msgr2=0x5630b91500c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.366+0000 7f94bf5af740 1 -- 172.21.15.113:0/1830983128 shutdown_connections 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.366+0000 7f94bf5af740 1 -- 172.21.15.113:0/1830983128 wait complete. 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.366+0000 7f94bf5af740 1 -- start start 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.366+0000 7f94bf5af740 1 Processor -- start 2024-04-25T00:09:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b91bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b91415c0 0x5630b91ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94bf5af740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9142b60 0x5630b913fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94bf5af740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5630b9137730 con 0x5630b91415c0 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5630b9126230 con 0x5630b9142b60 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94bf5af740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5630b9137d80 con 0x5630b9125c80 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94bf5af740 10 client.0 _ll_register_callbacks cb 0x7f94a0032840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b91415c0 0x5630b91ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b91415c0 0x5630b91ba4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41150/0 (socket says 172.21.15.113:41150) 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 -- 172.21.15.113:0/3983365531 learned_addr learned my addr 172.21.15.113:0/3983365531 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9142b60 0x5630b913fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94b659c640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b91bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 -- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 msgr2=0x5630b91bb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b91bb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 -- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9142b60 msgr2=0x5630b913fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9142b60 0x5630b913fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94add9b640 1 -- 172.21.15.113:0/3983365531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b90f4e20 con 0x5630b91415c0 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94b6d9d640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9142b60 0x5630b913fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.367+0000 7f94b659c640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b91bb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94add9b640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b91415c0 0x5630b91ba4c0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x5630b9136c10 tx=0x7f94a0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cceb9d524da01409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94957fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94957fa640 1 -- 172.21.15.113:0/3983365531 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0043040 con 0x5630b91415c0 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94957fa640 1 -- 172.21.15.113:0/3983365531 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f94a0053b00 con 0x5630b91415c0 2024-04-25T00:09:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a8025e50 con 0x5630b91415c0 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94957fa640 1 -- 172.21.15.113:0/3983365531 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f94a0037040 con 0x5630b91415c0 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 10 client.11082 fetch_fsmap learned FSMap version 104 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5630b913e4a0 con 0x5630b91415c0 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94957fa640 1 -- 172.21.15.113:0/3983365531 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a8025e50 con 0x5630b91415c0 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94957fa640 1 -- 172.21.15.113:0/3983365531 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f94a0051050 con 0x5630b91415c0 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 10 client.11082 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 10 client.11082 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 20 client.11082 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b9151070 0x5630b914ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 4 client.11082 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "status"}) -- 0x5630b9065790 con 0x5630b9151070 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.368+0000 7f94b659c640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b9151070 0x5630b914ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94b659c640 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b9151070 0x5630b914ce80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4484b8bbba6db8 server_cookie=e2ac9fea5bbeb249 in_seq=0 out_seq=0 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94957fa640 10 client.11082 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94957fa640 1 -- 172.21.15.113:0/3983365531 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5630b9065790 con 0x5630b9151070 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94957fa640 10 client.11082 handle_command_reply: tid=0 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 client.11082 shutdown 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b91415c0 msgr2=0x5630b91ba4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b91415c0 0x5630b91ba4c0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x5630b9136c10 tx=0x7f94a0009240 comp rx=0 tx=0).stop 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 shutdown_connections 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5630b9151070 0x5630b914ce80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630b9142b60 0x5630b913fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630b91415c0 0x5630b91ba4c0 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94957fa640 0 client.11082 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 --2- 172.21.15.113:0/3983365531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5630b9125c80 0x5630b91bb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 >> 172.21.15.113:0/3983365531 conn(0x5630b9125540 msgr2=0x5630b90f6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 shutdown_connections 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 1 -- 172.21.15.113:0/3983365531 wait complete. 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.369+0000 7f94bf5af740 20 client.11082 trim_cache size 0 max 16384 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 10794, 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:09:52.378 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:09:52.379 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:09:52.379 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 4.381386966, 2024-04-25T00:09:52.379 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 104, 2024-04-25T00:09:52.379 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 183, 2024-04-25T00:09:52.379 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 183, 2024-04-25T00:09:52.379 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 23.590156262000001 2024-04-25T00:09:52.379 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949806be30 msgr2=0x7f949806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949806be30 0x7f949806e2f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f94a800ab60 tx=0x7f94a8008040 comp rx=0 tx=0).stop 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b815e6b0 msgr2=0x7f94b8153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b815e6b0 0x7f94b8153050 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f94b000b970 tx=0x7f94b000be30 comp rx=0 tx=0).stop 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 shutdown_connections 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f949806be30 0x7f949806e2f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94b815e2d0 0x7f94b815b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94b815e6b0 0x7f94b8153050 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 --2- 172.21.15.113:0/3887724795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94b80b3080 0x7f94b8143ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 >> 172.21.15.113:0/3887724795 conn(0x7f94b806b0f0 msgr2=0x7f94b80b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 shutdown_connections 2024-04-25T00:09:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.370+0000 7f94bc89e640 1 -- 172.21.15.113:0/3887724795 wait complete. 2024-04-25T00:09:52.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:52.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub start / recursive,repair 2024-04-25T00:09:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.481+0000 7f32b31dc640 1 Processor -- start 2024-04-25T00:09:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.481+0000 7f32b31dc640 1 -- start start 2024-04-25T00:09:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b31dc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f32ac0cc0b0 0x7f32ac154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b31dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f32ac0b4b80 0x7f32ac154d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b31dc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ac0b2f40 con 0x7f32ac0b47a0 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b31dc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f32ac0b31e0 con 0x7f32ac0b4b80 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b31dc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f32ac0b34c0 con 0x7f32ac0cc0b0 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b1779640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f32ac0b4b80 0x7f32ac154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b0f78640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f32ac0b47a0 0x7f32ac15b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46358/0 (socket says 172.21.15.113:46358) 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b0f78640 1 -- 172.21.15.113:0/3547059582 learned_addr learned my addr 172.21.15.113:0/3547059582 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b1779640 1 -- 172.21.15.113:0/3547059582 >> v2:172.21.15.178:3300/0 conn(0x7f32ac0cc0b0 msgr2=0x7f32ac154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32a3fff640 1 --2- 172.21.15.113:0/3547059582 >> v2:172.21.15.178:3300/0 conn(0x7f32ac0cc0b0 0x7f32ac154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b1779640 1 --2- 172.21.15.113:0/3547059582 >> v2:172.21.15.178:3300/0 conn(0x7f32ac0cc0b0 0x7f32ac154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32a37fe640 1 -- 172.21.15.113:0/3547059582 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132046707 0 0) 0x7f32ac0b2f40 con 0x7f32ac0b47a0 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b1779640 1 -- 172.21.15.113:0/3547059582 >> v1:172.21.15.113:6789/0 conn(0x7f32ac0b47a0 legacy=0x7f32ac15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32b1779640 1 -- 172.21.15.113:0/3547059582 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac0b37a0 con 0x7f32ac0b4b80 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.482+0000 7f32a3fff640 1 --2- 172.21.15.113:0/3547059582 >> v2:172.21.15.178:3300/0 conn(0x7f32ac0cc0b0 0x7f32ac154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.483+0000 7f32b1779640 1 --2- 172.21.15.113:0/3547059582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac154d70 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b2e30 tx=0x7f32a402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97112b4bf420ce78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.483+0000 7f32a37fe640 1 -- 172.21.15.113:0/3547059582 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4007740 con 0x7f32ac0b4b80 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.483+0000 7f32a37fe640 1 -- 172.21.15.113:0/3547059582 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a40078e0 con 0x7f32ac0b4b80 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.483+0000 7f32a37fe640 1 -- 172.21.15.113:0/3547059582 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4007cb0 con 0x7f32ac0b4b80 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.483+0000 7f32b31dc640 1 -- 172.21.15.113:0/3547059582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 msgr2=0x7f32ac154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.483+0000 7f32b31dc640 1 --2- 172.21.15.113:0/3547059582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac154d70 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b2e30 tx=0x7f32a402a680 comp rx=0 tx=0).stop 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.484+0000 7f32b31dc640 1 -- 172.21.15.113:0/3547059582 shutdown_connections 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.484+0000 7f32b31dc640 1 --2- 172.21.15.113:0/3547059582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac154d70 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.484+0000 7f32b31dc640 1 --2- 172.21.15.113:0/3547059582 >> v2:172.21.15.178:3300/0 conn(0x7f32ac0cc0b0 0x7f32ac154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.484+0000 7f32b31dc640 1 -- 172.21.15.113:0/3547059582 >> 172.21.15.113:0/3547059582 conn(0x7f32ac0b1b30 msgr2=0x7f32ac0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.484+0000 7f32b31dc640 1 -- 172.21.15.113:0/3547059582 shutdown_connections 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.484+0000 7f32b31dc640 1 -- 172.21.15.113:0/3547059582 wait complete. 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.485+0000 7f32b31dc640 1 Processor -- start 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.485+0000 7f32b31dc640 1 -- start start 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32ac0b47a0 0x7f32ac15e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32ac0cc0b0 0x7f32ac0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0b4100 con 0x7f32ac0b4b80 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0b3f80 con 0x7f32ac0b47a0 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac0b4280 con 0x7f32ac0cc0b0 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac13fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41178/0 (socket says 172.21.15.113:41178) 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 -- 172.21.15.113:0/2948429011 learned_addr learned my addr 172.21.15.113:0/2948429011 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b0f78640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32ac0b47a0 0x7f32ac15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b1779640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32ac0cc0b0 0x7f32ac0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 -- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32ac0cc0b0 msgr2=0x7f32ac0b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32ac0cc0b0 0x7f32ac0b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 -- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32ac0b47a0 msgr2=0x7f32ac15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32ac0b47a0 0x7f32ac15e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 -- 172.21.15.113:0/2948429011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac15d6c0 con 0x7f32ac0b4b80 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a3fff640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac13fac0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c7f0 tx=0x7f329c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a17fa640 1 -- 172.21.15.113:0/2948429011 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c015070 con 0x7f32ac0b4b80 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a17fa640 1 -- 172.21.15.113:0/2948429011 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c007820 con 0x7f32ac0b4b80 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac140000 con 0x7f32ac0b4b80 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32a17fa640 1 -- 172.21.15.113:0/2948429011 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c007bd0 con 0x7f32ac0b4b80 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.486+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac15d960 con 0x7f32ac0b4b80 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.488+0000 7f32a17fa640 1 -- 172.21.15.113:0/2948429011 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f329c018070 con 0x7f32ac0b4b80 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.488+0000 7f32a17fa640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f329806be60 0x7f329806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.488+0000 7f32a17fa640 1 -- 172.21.15.113:0/2948429011 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f329c08ae00 con 0x7f32ac0b4b80 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.488+0000 7f32b0f78640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f329806be60 0x7f329806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.489+0000 7f32b0f78640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f329806be60 0x7f329806e320 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000f60 tx=0x7f32a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.489+0000 7f32b5fc6740 1 Processor -- start 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.489+0000 7f32b5fc6740 1 -- start start 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69be280 0x55add69c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69e8d40 0x55add69e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55add69212c0 con 0x55add69e8d40 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55add68c9910 con 0x55add69fa950 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55add691d280 con 0x55add69be280 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69e8d40 0x55add69e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69e8d40 0x55add69e91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41186/0 (socket says 172.21.15.113:41186) 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 -- 172.21.15.113:0/1317513993 learned_addr learned my addr 172.21.15.113:0/1317513993 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b0f78640 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69be280 0x55add69c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32a3fff640 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 -- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69be280 msgr2=0x55add69c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69be280 0x55add69c03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 -- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 msgr2=0x55add69be660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69be660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 -- 172.21.15.113:0/1317513993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add68dafb0 con 0x55add69e8d40 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32b1779640 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69e8d40 0x55add69e91c0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009e50 tx=0x7f32a402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f578ce39c67718b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32927fc640 1 -- 172.21.15.113:0/1317513993 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4003df0 con 0x55add69e8d40 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32927fc640 1 -- 172.21.15.113:0/1317513993 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a4041070 con 0x55add69e8d40 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.490+0000 7f32927fc640 1 -- 172.21.15.113:0/1317513993 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4004260 con 0x55add69e8d40 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1317513993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69e8d40 msgr2=0x55add69e91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69e8d40 0x55add69e91c0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009e50 tx=0x7f32a402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1317513993 shutdown_connections 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69e8d40 0x55add69e91c0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1317513993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69be280 0x55add69c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1317513993 >> 172.21.15.113:0/1317513993 conn(0x55add6985950 msgr2=0x55add6a03d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1317513993 shutdown_connections 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.491+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1317513993 wait complete. 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.492+0000 7f32b5fc6740 1 -- start start 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.492+0000 7f32b5fc6740 1 Processor -- start 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add69e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a14560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69fb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55add68dce90 con 0x55add69be280 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55add69e6890 con 0x55add69fa950 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55add6a14aa0 con 0x55add69e8d40 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 10 client.0 _ll_register_callbacks cb 0x55add6918970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:52.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add69e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add69e5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41192/0 (socket says 172.21.15.113:41192) 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 -- 172.21.15.113:0/1333966442 learned_addr learned my addr 172.21.15.113:0/1333966442 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32a3fff640 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a14560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b1779640 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 -- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 msgr2=0x55add6a14560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a14560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 -- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 msgr2=0x55add69fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69fb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 -- 172.21.15.113:0/1333966442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add66fd7e0 con 0x55add69be280 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b0f78640 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add69e5b40 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f32a8005e30 tx=0x7f32a8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9adf417b034a92f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f328b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:52.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f328b7fe640 1 -- 172.21.15.113:0/1333966442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8007560 con 0x55add69be280 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f328b7fe640 1 -- 172.21.15.113:0/1333966442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f32a8021ae0 con 0x55add69be280 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.493+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a4041070 con 0x55add69be280 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f328b7fe640 1 -- 172.21.15.113:0/1333966442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f32a8007700 con 0x55add69be280 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 10 client.11097 fetch_fsmap learned FSMap version 104 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55add675df70 con 0x55add69be280 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f328b7fe640 1 -- 172.21.15.113:0/1333966442 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a4041070 con 0x55add69be280 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f328b7fe640 1 -- 172.21.15.113:0/1333966442 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f32a8021ce0 con 0x55add69be280 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 10 client.11097 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 10 client.11097 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 20 client.11097 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add69fbd80 0x55add69fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 4 client.11097 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55add68f55a0 con 0x55add69fbd80 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32a3fff640 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add69fbd80 0x55add69fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f328b7fe640 10 client.11097 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.494+0000 7f32a3fff640 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add69fbd80 0x55add69fe1b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f244ab51ab8804b3 server_cookie=2f43b8dbd12cb760 in_seq=0 out_seq=0 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f328b7fe640 1 -- 172.21.15.113:0/1333966442 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55add68f55a0 con 0x55add69fbd80 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f328b7fe640 10 client.11097 handle_command_reply: tid=0 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 client.11097 shutdown 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 msgr2=0x55add69e5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add69e5b40 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f32a8005e30 tx=0x7f32a8000ed0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 shutdown_connections 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add69fbd80 0x55add69fe1b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a14560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f328b7fe640 0 client.11097 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/1333966442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add69e5b40 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 >> 172.21.15.113:0/1333966442 conn(0x55add6985950 msgr2=0x55add6a070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 shutdown_connections 2024-04-25T00:09:52.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 1 -- 172.21.15.113:0/1333966442 wait complete. 2024-04-25T00:09:52.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.495+0000 7f32b5fc6740 20 client.11097 trim_cache size 0 max 16384 2024-04-25T00:09:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.512+0000 7f32b5fc6740 1 Processor -- start 2024-04-25T00:09:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.512+0000 7f32b5fc6740 1 -- start start 2024-04-25T00:09:52.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a07070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a78160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add6a786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac2007c0 con 0x55add69be280 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0cec70 con 0x55add69fa950 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac0cdfd0 con 0x55add69e8d40 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a07070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a07070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41198/0 (socket says 172.21.15.113:41198) 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 -- 172.21.15.113:0/3636895543 learned_addr learned my addr 172.21.15.113:0/3636895543 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 -- 172.21.15.113:0/3636895543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 msgr2=0x55add6a78160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 --2- 172.21.15.113:0/3636895543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a78160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 -- 172.21.15.113:0/3636895543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 msgr2=0x55add6a786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 --2- 172.21.15.113:0/3636895543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add6a786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.513+0000 7f32b0f78640 1 -- 172.21.15.113:0/3636895543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add65f5170 con 0x55add69be280 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b0f78640 1 --2- 172.21.15.113:0/3636895543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a07070 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f32a800e720 tx=0x7f32a801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c910637801a6876b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f3291ffb640 1 -- 172.21.15.113:0/3636895543 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8030040 con 0x55add69be280 2024-04-25T00:09:52.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f3291ffb640 1 -- 172.21.15.113:0/3636895543 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a80014a0 con 0x55add69be280 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f3291ffb640 1 -- 172.21.15.113:0/3636895543 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8022630 con 0x55add69be280 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 -- 172.21.15.113:0/3636895543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 msgr2=0x55add6a07070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/3636895543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a07070 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f32a800e720 tx=0x7f32a801e000 comp rx=0 tx=0).stop 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 -- 172.21.15.113:0/3636895543 shutdown_connections 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/3636895543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add6a786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/3636895543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a78160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/3636895543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a07070 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 -- 172.21.15.113:0/3636895543 >> 172.21.15.113:0/3636895543 conn(0x55add6985950 msgr2=0x55add69ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 -- 172.21.15.113:0/3636895543 shutdown_connections 2024-04-25T00:09:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.514+0000 7f32b5fc6740 1 -- 172.21.15.113:0/3636895543 wait complete. 2024-04-25T00:09:52.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.515+0000 7f32b5fc6740 1 -- start start 2024-04-25T00:09:52.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.515+0000 7f32b5fc6740 1 Processor -- start 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a7d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a7ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b5fc6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69fe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55add69b83e0 con 0x55add69be280 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0ce620 con 0x55add69fa950 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b5fc6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55add6a04bc0 con 0x55add69e8d40 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b5fc6740 10 client.0 _ll_register_callbacks cb 0x7f32a8001d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:52.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a7d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a7d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41202/0 (socket says 172.21.15.113:41202) 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 -- 172.21.15.113:0/2773216055 learned_addr learned my addr 172.21.15.113:0/2773216055 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 -- 172.21.15.113:0/2773216055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 msgr2=0x55add6a7ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a7ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 -- 172.21.15.113:0/2773216055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 msgr2=0x55add69fe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69fe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 -- 172.21.15.113:0/2773216055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add6a77b20 con 0x55add69be280 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f32b0f78640 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a7d8b0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001140 tx=0x7f32a8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9711e4de534c6a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.516+0000 7f328b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f328b7fe640 1 -- 172.21.15.113:0/2773216055 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8030040 con 0x55add69be280 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f328b7fe640 1 -- 172.21.15.113:0/2773216055 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f32a8023d90 con 0x55add69be280 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a80014a0 con 0x55add69be280 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f328b7fe640 1 -- 172.21.15.113:0/2773216055 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f32a8034430 con 0x55add69be280 2024-04-25T00:09:52.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 10 client.11103 fetch_fsmap learned FSMap version 104 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55add69ffa70 con 0x55add69be280 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f328b7fe640 1 -- 172.21.15.113:0/2773216055 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a80014a0 con 0x55add69be280 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f328b7fe640 1 -- 172.21.15.113:0/2773216055 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f32a803b050 con 0x55add69be280 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 10 client.11103 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 10 client.11103 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 20 client.11103 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add6a1de30 0x55add69fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 4 client.11103 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) -- 0x55add692e800 con 0x55add6a1de30 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32a3fff640 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add6a1de30 0x55add69fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f328b7fe640 10 client.11103 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.517+0000 7f32a3fff640 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add6a1de30 0x55add69fef30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d193d68ebb8a99cd server_cookie=1905311e63a329a2 in_seq=0 out_seq=0 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f328b7fe640 1 -- 172.21.15.113:0/2773216055 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55add692e800 con 0x55add6a1de30 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f328b7fe640 10 client.11103 handle_command_reply: tid=0 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 client.11103 shutdown 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 msgr2=0x55add6a7d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a7d8b0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001140 tx=0x7f32a8013000 comp rx=0 tx=0).stop 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 shutdown_connections 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55add6a1de30 0x55add69fef30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55add69fa950 0x55add69fe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55add69e8d40 0x55add6a7ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f328b7fe640 0 client.11103 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 --2- 172.21.15.113:0/2773216055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55add69be280 0x55add6a7d8b0 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 >> 172.21.15.113:0/2773216055 conn(0x55add6985950 msgr2=0x55add6a07390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 shutdown_connections 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 1 -- 172.21.15.113:0/2773216055 wait complete. 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.518+0000 7f32b5fc6740 20 client.11103 trim_cache size 0 max 16384 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "56c5ba42-328b-408e-b051-3ee1a5f81a6a", 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:09:52.525 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f329806be60 msgr2=0x7f329806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f329806be60 0x7f329806e320 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000f60 tx=0x7f32a8009040 comp rx=0 tx=0).stop 2024-04-25T00:09:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 msgr2=0x7f32ac13fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac13fac0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c7f0 tx=0x7f329c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 shutdown_connections 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f329806be60 0x7f329806e320 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32ac0cc0b0 0x7f32ac0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32ac0b4b80 0x7f32ac13fac0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 --2- 172.21.15.113:0/2948429011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32ac0b47a0 0x7f32ac15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 >> 172.21.15.113:0/2948429011 conn(0x7f32ac0b1b30 msgr2=0x7f32ac154600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 shutdown_connections 2024-04-25T00:09:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.519+0000 7f32b31dc640 1 -- 172.21.15.113:0/2948429011 wait complete. 2024-04-25T00:09:52.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:09:52.531 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T00:09:52.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.628+0000 7f45d4612640 1 Processor -- start 2024-04-25T00:09:52.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.628+0000 7f45d4612640 1 -- start start 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d4612640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc071db0 0x7f45cc15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d4612640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45cc06cea0 0x7f45cc0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d4612640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45cc071580 con 0x7f45cc0719d0 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d4612640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc141620 con 0x7f45cc06cea0 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d4612640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc141fc0 con 0x7f45cc071db0 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d2baf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45cc06cea0 0x7f45cc0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d2baf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45cc06cea0 0x7f45cc0cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41214/0 (socket says 172.21.15.113:41214) 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d23ae640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f45cc0719d0 0x7f45cc1418b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46370/0 (socket says 172.21.15.113:46370) 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d2baf640 1 -- 172.21.15.113:0/736615374 learned_addr learned my addr 172.21.15.113:0/736615374 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d1bad640 1 --2- 172.21.15.113:0/736615374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc071db0 0x7f45cc15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d2baf640 1 -- 172.21.15.113:0/736615374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc071db0 msgr2=0x7f45cc15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d2baf640 1 --2- 172.21.15.113:0/736615374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc071db0 0x7f45cc15f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d2baf640 1 -- 172.21.15.113:0/736615374 >> v1:172.21.15.113:6789/0 conn(0x7f45cc0719d0 legacy=0x7f45cc1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d13ac640 1 -- 172.21.15.113:0/736615374 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4242407643 0 0) 0x7f45c8001110 con 0x7f45cc0719d0 2024-04-25T00:09:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d2baf640 1 -- 172.21.15.113:0/736615374 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc06d3f0 con 0x7f45cc06cea0 2024-04-25T00:09:52.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.629+0000 7f45d1bad640 1 --2- 172.21.15.113:0/736615374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc071db0 0x7f45cc15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.630+0000 7f45d2baf640 1 --2- 172.21.15.113:0/736615374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc0cf020 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009e50 tx=0x7f45c402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe05378a98f205e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.630+0000 7f45d13ac640 1 -- 172.21.15.113:0/736615374 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4036070 con 0x7f45cc06cea0 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.630+0000 7f45d13ac640 1 -- 172.21.15.113:0/736615374 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4031600 con 0x7f45cc06cea0 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.630+0000 7f45d13ac640 1 -- 172.21.15.113:0/736615374 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40319d0 con 0x7f45cc06cea0 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 -- 172.21.15.113:0/736615374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 msgr2=0x7f45cc0cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 --2- 172.21.15.113:0/736615374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc0cf020 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009e50 tx=0x7f45c402a7a0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 -- 172.21.15.113:0/736615374 shutdown_connections 2024-04-25T00:09:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 --2- 172.21.15.113:0/736615374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc0cf020 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 --2- 172.21.15.113:0/736615374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc071db0 0x7f45cc15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 -- 172.21.15.113:0/736615374 >> 172.21.15.113:0/736615374 conn(0x7f45cc06b0f0 msgr2=0x7f45cc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 -- 172.21.15.113:0/736615374 shutdown_connections 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.631+0000 7f45d4612640 1 -- 172.21.15.113:0/736615374 wait complete. 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.632+0000 7f45d4612640 1 Processor -- start 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.632+0000 7f45d4612640 1 -- start start 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.632+0000 7f45d4612640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc1e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d4612640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc0719d0 0x7f45cc15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d4612640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc071db0 0x7f45cc140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d4612640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc1439e0 con 0x7f45cc06cea0 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d4612640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc06d9b0 con 0x7f45cc071db0 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d4612640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc06dc90 con 0x7f45cc0719d0 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc1e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc1e8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41220/0 (socket says 172.21.15.113:41220) 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 -- 172.21.15.113:0/3159921195 learned_addr learned my addr 172.21.15.113:0/3159921195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 -- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc0719d0 msgr2=0x7f45cc15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d2baf640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc071db0 0x7f45cc140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d1bad640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc0719d0 0x7f45cc15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc0719d0 0x7f45cc15a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 -- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc071db0 msgr2=0x7f45cc140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc071db0 0x7f45cc140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 -- 172.21.15.113:0/3159921195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc15abd0 con 0x7f45cc06cea0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d23ae640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc1e8b60 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c780 tx=0x7f45c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45c2ffd640 1 -- 172.21.15.113:0/3159921195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8018070 con 0x7f45cc06cea0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45c2ffd640 1 -- 172.21.15.113:0/3159921195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c80072d0 con 0x7f45cc06cea0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc15aeb0 con 0x7f45cc06cea0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45c2ffd640 1 -- 172.21.15.113:0/3159921195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80047e0 con 0x7f45cc06cea0 2024-04-25T00:09:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.633+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc15b470 con 0x7f45cc06cea0 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.635+0000 7f45c2ffd640 1 -- 172.21.15.113:0/3159921195 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45c801b070 con 0x7f45cc06cea0 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.635+0000 7f45c2ffd640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45b006be30 0x7f45b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.635+0000 7f45c2ffd640 1 -- 172.21.15.113:0/3159921195 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f45c808a260 con 0x7f45cc06cea0 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.636+0000 7f45d1bad640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45b006be30 0x7f45b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.636+0000 7f45d1bad640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45b006be30 0x7f45b006e2f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f45cc1556d0 tx=0x7f45bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.636+0000 7f45d73f6740 1 Processor -- start 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d73f6740 1 -- start start 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b154d60 0x55787b155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b172b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b12aa40 0x55787b170cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d73f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55787b08e6b0 con 0x55787b12aa40 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55787b036880 con 0x55787b154d60 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55787b08a670 con 0x55787b172600 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d2baf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b12aa40 0x55787b170cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d2baf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b12aa40 0x55787b170cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41236/0 (socket says 172.21.15.113:41236) 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.637+0000 7f45d2baf640 1 -- 172.21.15.113:0/1837593367 learned_addr learned my addr 172.21.15.113:0/1837593367 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d23ae640 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b154d60 0x55787b155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d1bad640 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b172b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d2baf640 1 -- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 msgr2=0x55787b172b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d2baf640 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b172b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d2baf640 1 -- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b154d60 msgr2=0x55787b155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d2baf640 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b154d60 0x55787b155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d2baf640 1 -- 172.21.15.113:0/1837593367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787ae6c7d0 con 0x55787b12aa40 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d23ae640 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b154d60 0x55787b155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d1bad640 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b172b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45d2baf640 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b12aa40 0x55787b170cc0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009f70 tx=0x7f45c402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386634771a64cc8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45affff640 1 -- 172.21.15.113:0/1837593367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4036040 con 0x55787b12aa40 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45affff640 1 -- 172.21.15.113:0/1837593367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4042070 con 0x55787b12aa40 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.638+0000 7f45affff640 1 -- 172.21.15.113:0/1837593367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4009510 con 0x55787b12aa40 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 -- 172.21.15.113:0/1837593367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b12aa40 msgr2=0x55787b170cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b12aa40 0x55787b170cc0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009f70 tx=0x7f45c402e000 comp rx=0 tx=0).stop 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 -- 172.21.15.113:0/1837593367 shutdown_connections 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b12aa40 0x55787b170cc0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b172b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1837593367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b154d60 0x55787b155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 -- 172.21.15.113:0/1837593367 >> 172.21.15.113:0/1837593367 conn(0x55787b127600 msgr2=0x55787b16fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 -- 172.21.15.113:0/1837593367 shutdown_connections 2024-04-25T00:09:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 -- 172.21.15.113:0/1837593367 wait complete. 2024-04-25T00:09:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 -- start start 2024-04-25T00:09:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.639+0000 7f45d73f6740 1 Processor -- start 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b154d60 0x55787b156da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b172600 0x55787b16a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b16a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d73f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55787b12b7e0 con 0x55787b172600 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55787b049f50 con 0x55787b182f40 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55787b125530 con 0x55787b154d60 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d73f6740 10 client.0 _ll_register_callbacks cb 0x55787b085000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d1bad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b172600 0x55787b16a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d1bad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b172600 0x55787b16a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41240/0 (socket says 172.21.15.113:41240) 2024-04-25T00:09:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d1bad640 1 -- 172.21.15.113:0/1340631729 learned_addr learned my addr 172.21.15.113:0/1340631729 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d23ae640 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b154d60 0x55787b156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.640+0000 7f45d2baf640 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b16a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d1bad640 1 -- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b154d60 msgr2=0x55787b156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d1bad640 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b154d60 0x55787b156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d1bad640 1 -- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 msgr2=0x55787b16a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d1bad640 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b16a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d1bad640 1 -- 172.21.15.113:0/1340631729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787ada23f0 con 0x55787b172600 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d1bad640 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b172600 0x55787b16a100 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0721f0 tx=0x7f45bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c0a38a07603310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45ad7fa640 1 -- 172.21.15.113:0/1340631729 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00f040 con 0x55787b172600 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45ad7fa640 1 -- 172.21.15.113:0/1340631729 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f45bc00edf0 con 0x55787b172600 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c4042070 con 0x55787b172600 2024-04-25T00:09:52.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45ad7fa640 1 -- 172.21.15.113:0/1340631729 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f45bc02d050 con 0x55787b172600 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 10 client.11118 fetch_fsmap learned FSMap version 104 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55787ad620f0 con 0x55787b172600 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45ad7fa640 1 -- 172.21.15.113:0/1340631729 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4042070 con 0x55787b172600 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45ad7fa640 1 -- 172.21.15.113:0/1340631729 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f45bc030020 con 0x55787b172600 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 10 client.11118 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 10 client.11118 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 20 client.11118 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b183890 0x55787b183cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 4 client.11118 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55787b09bbf0 con 0x55787b183890 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.641+0000 7f45d23ae640 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b183890 0x55787b183cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.642+0000 7f45ad7fa640 10 client.11118 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.642+0000 7f45d23ae640 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b183890 0x55787b183cd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52987141e206de66 server_cookie=9b0394ad24202d34 in_seq=0 out_seq=0 2024-04-25T00:09:52.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.642+0000 7f45ad7fa640 1 -- 172.21.15.113:0/1340631729 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55787b09bbf0 con 0x55787b183890 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.642+0000 7f45ad7fa640 10 client.11118 handle_command_reply: tid=0 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.642+0000 7f45d73f6740 1 client.11118 shutdown 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.642+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b172600 msgr2=0x55787b16a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.642+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b172600 0x55787b16a100 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0721f0 tx=0x7f45bc004730 comp rx=0 tx=0).stop 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 shutdown_connections 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b183890 0x55787b183cd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b16a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45ad7fa640 0 client.11118 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b172600 0x55787b16a100 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 --2- 172.21.15.113:0/1340631729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b154d60 0x55787b156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 >> 172.21.15.113:0/1340631729 conn(0x55787b12aa40 msgr2=0x55787b173980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 shutdown_connections 2024-04-25T00:09:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 1 -- 172.21.15.113:0/1340631729 wait complete. 2024-04-25T00:09:52.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.643+0000 7f45d73f6740 20 client.11118 trim_cache size 0 max 16384 2024-04-25T00:09:52.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.660+0000 7f45d73f6740 1 Processor -- start 2024-04-25T00:09:52.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.660+0000 7f45d73f6740 1 -- start start 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b154d60 0x55787b16a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b12ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b12b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d73f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc2007d0 con 0x55787b154d60 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc1ff830 con 0x55787b182f40 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc1fec30 con 0x55787b172600 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b154d60 0x55787b16a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b154d60 0x55787b16a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41254/0 (socket says 172.21.15.113:41254) 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 -- 172.21.15.113:0/2152417582 learned_addr learned my addr 172.21.15.113:0/2152417582 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d2baf640 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b12b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d1bad640 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b12ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 -- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 msgr2=0x55787b12ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b12ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 -- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 msgr2=0x55787b12b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b12b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 -- 172.21.15.113:0/2152417582 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787b16a980 con 0x55787b154d60 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d2baf640 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b12b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d1bad640 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b12ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45d23ae640 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b154d60 0x55787b16a440 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f45c8054910 tx=0x7f45c805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413a2a369e6b60c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45af7fe640 1 -- 172.21.15.113:0/2152417582 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8070070 con 0x55787b154d60 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45af7fe640 1 -- 172.21.15.113:0/2152417582 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c8061090 con 0x55787b154d60 2024-04-25T00:09:52.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.661+0000 7f45af7fe640 1 -- 172.21.15.113:0/2152417582 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80567b0 con 0x55787b154d60 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 -- 172.21.15.113:0/2152417582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b154d60 msgr2=0x55787b16a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b154d60 0x55787b16a440 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f45c8054910 tx=0x7f45c805b040 comp rx=0 tx=0).stop 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 -- 172.21.15.113:0/2152417582 shutdown_connections 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b182f40 0x55787b12b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b172600 0x55787b12ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 --2- 172.21.15.113:0/2152417582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b154d60 0x55787b16a440 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 -- 172.21.15.113:0/2152417582 >> 172.21.15.113:0/2152417582 conn(0x55787b12aa40 msgr2=0x55787b193af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 -- 172.21.15.113:0/2152417582 shutdown_connections 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 -- 172.21.15.113:0/2152417582 wait complete. 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 -- start start 2024-04-25T00:09:52.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.662+0000 7f45d73f6740 1 Processor -- start 2024-04-25T00:09:52.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b172600 0x55787b1670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b182f40 0x55787b1e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d73f6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b186220 0x55787b129d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d73f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55787b16b710 con 0x55787b182f40 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc1ff1e0 con 0x55787b172600 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d73f6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55787b1547a0 con 0x55787b186220 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d73f6740 10 client.0 _ll_register_callbacks cb 0x7f45bc009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d1bad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b182f40 0x55787b1e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d1bad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b182f40 0x55787b1e9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41260/0 (socket says 172.21.15.113:41260) 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.663+0000 7f45d1bad640 1 -- 172.21.15.113:0/501879409 learned_addr learned my addr 172.21.15.113:0/501879409 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d2baf640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b186220 0x55787b129d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d23ae640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b172600 0x55787b1670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d1bad640 1 -- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b186220 msgr2=0x55787b129d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d1bad640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b186220 0x55787b129d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d1bad640 1 -- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b172600 msgr2=0x55787b1670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d1bad640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b172600 0x55787b1670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d1bad640 1 -- 172.21.15.113:0/501879409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787b189e70 con 0x55787b182f40 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d2baf640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b186220 0x55787b129d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d1bad640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b182f40 0x55787b1e9cd0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006dc0 tx=0x7f45bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68aeb1c7a77d41a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45ad7fa640 1 -- 172.21.15.113:0/501879409 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc013c30 con 0x55787b182f40 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45ad7fa640 1 -- 172.21.15.113:0/501879409 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f45bc013dd0 con 0x55787b182f40 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c8061090 con 0x55787b182f40 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45ad7fa640 1 -- 172.21.15.113:0/501879409 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f45bc039400 con 0x55787b182f40 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 10 client.11124 fetch_fsmap learned FSMap version 104 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55787b1866d0 con 0x55787b182f40 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45ad7fa640 1 -- 172.21.15.113:0/501879409 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c8061090 con 0x55787b182f40 2024-04-25T00:09:52.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45ad7fa640 1 -- 172.21.15.113:0/501879409 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f45bc030040 con 0x55787b182f40 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 10 client.11124 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 10 client.11124 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 20 client.11124 populate_metadata read hostname 'smithi113' 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b12a270 0x55787b17f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 4 client.11124 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.664+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55787b095f40 con 0x55787b12a270 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d23ae640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b12a270 0x55787b17f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45ad7fa640 10 client.11124 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d23ae640 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b12a270 0x55787b17f850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5878c52b637e90 server_cookie=ed9815f98138cc78 in_seq=0 out_seq=0 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45ad7fa640 1 -- 172.21.15.113:0/501879409 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55787b095f40 con 0x55787b12a270 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45ad7fa640 10 client.11124 handle_command_reply: tid=0 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 client.11124 shutdown 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b182f40 msgr2=0x55787b1e9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b182f40 0x55787b1e9cd0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006dc0 tx=0x7f45bc013000 comp rx=0 tx=0).stop 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 shutdown_connections 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x55787b12a270 0x55787b17f850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55787b186220 0x55787b129d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55787b182f40 0x55787b1e9cd0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 --2- 172.21.15.113:0/501879409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55787b172600 0x55787b1670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45ad7fa640 0 client.11124 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 >> 172.21.15.113:0/501879409 conn(0x55787b12aa40 msgr2=0x55787b169730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 shutdown_connections 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 1 -- 172.21.15.113:0/501879409 wait complete. 2024-04-25T00:09:52.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.665+0000 7f45d73f6740 20 client.11124 trim_cache size 0 max 16384 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: "56c5ba42-328b-408e-b051-3ee1a5f81a6a": { 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "56c5ba42-328b-408e-b051-3ee1a5f81a6a", 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,repair" 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:09:52.674 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:09:52.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45b006be30 msgr2=0x7f45b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45b006be30 0x7f45b006e2f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f45cc1556d0 tx=0x7f45bc008040 comp rx=0 tx=0).stop 2024-04-25T00:09:52.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 msgr2=0x7f45cc1e8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc1e8b60 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c780 tx=0x7f45c800cc40 comp rx=0 tx=0).stop 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 shutdown_connections 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f45b006be30 0x7f45b006e2f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc071db0 0x7f45cc140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45cc0719d0 0x7f45cc15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 --2- 172.21.15.113:0/3159921195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45cc06cea0 0x7f45cc1e8b60 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 >> 172.21.15.113:0/3159921195 conn(0x7f45cc06b0f0 msgr2=0x7f45cc15c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 shutdown_connections 2024-04-25T00:09:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:09:52.666+0000 7f45d4612640 1 -- 172.21.15.113:0/3159921195 wait complete. 2024-04-25T00:09:52.677 INFO:tasks.cephfs.filesystem:scrub status for tag:56c5ba42-328b-408e-b051-3ee1a5f81a6a - {'path': '/', 'tag': '56c5ba42-328b-408e-b051-3ee1a5f81a6a', 'options': 'recursive,repair'} 2024-04-25T00:09:54.119 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-04-25T00:09:54.119+0000 7f5bf2e55640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:09:54.188 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:09:54.191 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:22.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:10:22.679 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T00:10:22.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.790+0000 7fc4f562c640 1 Processor -- start 2024-04-25T00:10:22.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.790+0000 7fc4f562c640 1 -- start start 2024-04-25T00:10:22.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4f562c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc4f013f050 0x7fc4f00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4f562c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4f01606a0 0x7fc4f00cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4f562c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f015d6b0 con 0x7fc4f00d04f0 2024-04-25T00:10:22.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4f562c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f00d09a0 con 0x7fc4f01606a0 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4f562c640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f015c6a0 con 0x7fc4f013f050 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4ef7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4f01606a0 0x7fc4f00cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4ee7fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc4f00d04f0 0x7fc4f015ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35092/0 (socket says 172.21.15.113:35092) 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.791+0000 7fc4ee7fc640 1 -- 172.21.15.113:0/3290716345 learned_addr learned my addr 172.21.15.113:0/3290716345 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/3290716345 >> v2:172.21.15.178:3300/0 conn(0x7fc4f013f050 0x7fc4f00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/3290716345 >> v2:172.21.15.178:3300/0 conn(0x7fc4f013f050 msgr2=0x7fc4f00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/3290716345 >> v2:172.21.15.178:3300/0 conn(0x7fc4f013f050 0x7fc4f00cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/3290716345 >> v1:172.21.15.113:6789/0 conn(0x7fc4f00d04f0 legacy=0x7fc4f015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4edffb640 1 -- 172.21.15.113:0/3290716345 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1628920530 0 0) 0x7fc4f015d6b0 con 0x7fc4f00d04f0 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/3290716345 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f015c8f0 con 0x7fc4f01606a0 2024-04-25T00:10:22.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/3290716345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f01606a0 0x7fc4f00cea60 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006f550 tx=0x7fc4e802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acec98ab8e86a2a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4edffb640 1 -- 172.21.15.113:0/3290716345 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8037070 con 0x7fc4f01606a0 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.792+0000 7fc4edffb640 1 -- 172.21.15.113:0/3290716345 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8032440 con 0x7fc4f01606a0 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4edffb640 1 -- 172.21.15.113:0/3290716345 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e802ae60 con 0x7fc4f01606a0 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 -- 172.21.15.113:0/3290716345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f01606a0 msgr2=0x7fc4f00cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3290716345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f01606a0 0x7fc4f00cea60 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006f550 tx=0x7fc4e802a110 comp rx=0 tx=0).stop 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 -- 172.21.15.113:0/3290716345 shutdown_connections 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3290716345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f01606a0 0x7fc4f00cea60 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3290716345 >> v2:172.21.15.178:3300/0 conn(0x7fc4f013f050 0x7fc4f00cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 -- 172.21.15.113:0/3290716345 >> 172.21.15.113:0/3290716345 conn(0x7fc4f006b0f0 msgr2=0x7fc4f006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 -- 172.21.15.113:0/3290716345 shutdown_connections 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.793+0000 7fc4f562c640 1 -- 172.21.15.113:0/3290716345 wait complete. 2024-04-25T00:10:22.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.794+0000 7fc4f562c640 1 Processor -- start 2024-04-25T00:10:22.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4f562c640 1 -- start start 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4f562c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f00d04f0 0x7fc4f00b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4f562c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f013f050 0x7fc4f01f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4f562c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f01606a0 0x7fc4f014d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4f562c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f015d190 con 0x7fc4f013f050 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4f562c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f015ceb0 con 0x7fc4f00d04f0 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4f562c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f013f430 con 0x7fc4f01606a0 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4ee7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f013f050 0x7fc4f01f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4ee7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f013f050 0x7fc4f01f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41910/0 (socket says 172.21.15.113:41910) 2024-04-25T00:10:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.795+0000 7fc4ee7fc640 1 -- 172.21.15.113:0/3668857616 learned_addr learned my addr 172.21.15.113:0/3668857616 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f00d04f0 0x7fc4f00b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f01606a0 0x7fc4f014d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4ee7fc640 1 -- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f01606a0 msgr2=0x7fc4f014d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f01606a0 0x7fc4f014d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4ee7fc640 1 -- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f00d04f0 msgr2=0x7fc4f00b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f00d04f0 0x7fc4f00b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4ee7fc640 1 -- 172.21.15.113:0/3668857616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00b0d00 con 0x7fc4f013f050 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f013f050 0x7fc4f01f3700 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c4a0 tx=0x7fc4e400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4d77fe640 1 -- 172.21.15.113:0/3668857616 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4018070 con 0x7fc4f013f050 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4d77fe640 1 -- 172.21.15.113:0/3668857616 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e40074e0 con 0x7fc4f013f050 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f0154d40 con 0x7fc4f013f050 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4d77fe640 1 -- 172.21.15.113:0/3668857616 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4004a20 con 0x7fc4f013f050 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.796+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f00b4ef0 con 0x7fc4f013f050 2024-04-25T00:10:22.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.798+0000 7fc4d77fe640 1 -- 172.21.15.113:0/3668857616 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4e401b070 con 0x7fc4f013f050 2024-04-25T00:10:22.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.798+0000 7fc4d77fe640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4dc06be60 0x7fc4dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.798+0000 7fc4d77fe640 1 -- 172.21.15.113:0/3668857616 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fc4e408a070 con 0x7fc4f013f050 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.798+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4dc06be60 0x7fc4dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.799+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4dc06be60 0x7fc4dc06e320 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0154a40 tx=0x7fc4e00098d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.799+0000 7fc4f8410740 1 Processor -- start 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.799+0000 7fc4f8410740 1 -- start start 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 0x556662d1a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d37600 0x556662d37b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662cefa40 0x556662d35cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4f8410740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556662c536b0 con 0x556662cefa40 2024-04-25T00:10:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556662bfb880 con 0x556662d19d60 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556662c4f670 con 0x556662d37600 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662cefa40 0x556662d35cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662cefa40 0x556662d35cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41916/0 (socket says 172.21.15.113:41916) 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2998384308 learned_addr learned my addr 172.21.15.113:0/2998384308 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d37600 0x556662d37b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 0x556662d1a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d37600 msgr2=0x556662d37b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d37600 0x556662d37b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 msgr2=0x556662d1a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 0x556662d1a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2998384308 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556662a317d0 con 0x556662cefa40 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.800+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d37600 0x556662d37b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662cefa40 0x556662d35cc0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802a150 tx=0x7fc4e802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b022e044c2fb349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4d4ff9640 1 -- 172.21.15.113:0/2998384308 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8043070 con 0x556662cefa40 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4d4ff9640 1 -- 172.21.15.113:0/2998384308 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8032800 con 0x556662cefa40 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4d4ff9640 1 -- 172.21.15.113:0/2998384308 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8037040 con 0x556662cefa40 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4f8410740 1 -- 172.21.15.113:0/2998384308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662cefa40 msgr2=0x556662d35cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662cefa40 0x556662d35cc0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802a150 tx=0x7fc4e802c000 comp rx=0 tx=0).stop 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4f8410740 1 -- 172.21.15.113:0/2998384308 shutdown_connections 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662cefa40 0x556662d35cc0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d37600 0x556662d37b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2998384308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 0x556662d1a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.801+0000 7fc4f8410740 1 -- 172.21.15.113:0/2998384308 >> 172.21.15.113:0/2998384308 conn(0x556662cec600 msgr2=0x556662d34b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.802+0000 7fc4f8410740 1 -- 172.21.15.113:0/2998384308 shutdown_connections 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.802+0000 7fc4f8410740 1 -- 172.21.15.113:0/2998384308 wait complete. 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.802+0000 7fc4f8410740 1 -- start start 2024-04-25T00:10:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.802+0000 7fc4f8410740 1 Processor -- start 2024-04-25T00:10:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d1d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d35ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4f8410740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556662cf07e0 con 0x556662d0f840 2024-04-25T00:10:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556662c0ef50 con 0x556662d37600 2024-04-25T00:10:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556662cea530 con 0x556662d19d60 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4f8410740 10 client.0 _ll_register_callbacks cb 0x556662c4a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d35ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d35ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41928/0 (socket says 172.21.15.113:41928) 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/1239613061 learned_addr learned my addr 172.21.15.113:0/1239613061 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d1d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 msgr2=0x556662d1d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d1d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 msgr2=0x556662d355a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d355a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/1239613061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566629673f0 con 0x556662d0f840 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d355a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.803+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d1d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d35ae0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802a700 tx=0x7fc4e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdba29c79b46f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4cdffb640 1 -- 172.21.15.113:0/1239613061 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e804c060 con 0x556662d0f840 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4cdffb640 1 -- 172.21.15.113:0/1239613061 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fc4e8032dc0 con 0x556662d0f840 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e8032800 con 0x556662d0f840 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4cdffb640 1 -- 172.21.15.113:0/1239613061 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc4e8037040 con 0x556662d0f840 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 10 client.11139 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5566629270f0 con 0x556662d0f840 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4cdffb640 1 -- 172.21.15.113:0/1239613061 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8032800 con 0x556662d0f840 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4cdffb640 1 -- 172.21.15.113:0/1239613061 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fc4e8055080 con 0x556662d0f840 2024-04-25T00:10:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 10 client.11139 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 10 client.11139 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 20 client.11139 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d18570 0x556662d18980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 4 client.11139 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556662c60bf0 con 0x556662d18570 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.804+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d18570 0x556662d18980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4cdffb640 10 client.11139 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d18570 0x556662d18980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13543ae5b0b5370 server_cookie=6181f9eb32772b31 in_seq=0 out_seq=0 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4cdffb640 1 -- 172.21.15.113:0/1239613061 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556662c60bf0 con 0x556662d18570 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4cdffb640 10 client.11139 handle_command_reply: tid=0 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 client.11139 shutdown 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 msgr2=0x556662d35ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d35ae0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802a700 tx=0x7fc4e8004730 comp rx=0 tx=0).stop 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 shutdown_connections 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d18570 0x556662d18980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d355a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d35ae0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4cdffb640 0 client.11139 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 --2- 172.21.15.113:0/1239613061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d1d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 >> 172.21.15.113:0/1239613061 conn(0x556662cefa40 msgr2=0x556662ced910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 shutdown_connections 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.805+0000 7fc4f8410740 1 -- 172.21.15.113:0/1239613061 wait complete. 2024-04-25T00:10:22.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.806+0000 7fc4f8410740 20 client.11139 trim_cache size 0 max 16384 2024-04-25T00:10:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.823+0000 7fc4f8410740 1 Processor -- start 2024-04-25T00:10:22.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.823+0000 7fc4f8410740 1 -- start start 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d0f840 0x556662d18980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 0x556662d35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d37600 0x556662d35aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4f8410740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0154100 con 0x556662d37600 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f013ba30 con 0x556662d19d60 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f0156060 con 0x556662d0f840 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d37600 0x556662d35aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d37600 0x556662d35aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41938/0 (socket says 172.21.15.113:41938) 2024-04-25T00:10:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2963306389 learned_addr learned my addr 172.21.15.113:0/2963306389 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2963306389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d0f840 msgr2=0x556662d18980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/2963306389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d0f840 0x556662d18980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2963306389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 msgr2=0x556662d35560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/2963306389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 0x556662d35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 -- 172.21.15.113:0/2963306389 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556662d1aa20 con 0x556662d37600 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4ef7fe640 1 --2- 172.21.15.113:0/2963306389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d37600 0x556662d35aa0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802a900 tx=0x7fc4e8034070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a1e22d8f8b5ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4cffff640 1 -- 172.21.15.113:0/2963306389 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8043040 con 0x556662d37600 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4cffff640 1 -- 172.21.15.113:0/2963306389 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8032bc0 con 0x556662d37600 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.824+0000 7fc4cffff640 1 -- 172.21.15.113:0/2963306389 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8003660 con 0x556662d37600 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 -- 172.21.15.113:0/2963306389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d37600 msgr2=0x556662d35aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2963306389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d37600 0x556662d35aa0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802a900 tx=0x7fc4e8034070 comp rx=0 tx=0).stop 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 -- 172.21.15.113:0/2963306389 shutdown_connections 2024-04-25T00:10:22.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2963306389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d19d60 0x556662d35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2963306389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d37600 0x556662d35aa0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2963306389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d0f840 0x556662d18980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 -- 172.21.15.113:0/2963306389 >> 172.21.15.113:0/2963306389 conn(0x556662cefa40 msgr2=0x556662cf0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 -- 172.21.15.113:0/2963306389 shutdown_connections 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 -- 172.21.15.113:0/2963306389 wait complete. 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.825+0000 7fc4f8410740 1 -- start start 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 1 Processor -- start 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d56e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d0de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556662d12c30 con 0x556662d0f840 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f013b3e0 con 0x556662d37600 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556662d1a420 con 0x556662d19d60 2024-04-25T00:10:22.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4f8410740 10 client.0 _ll_register_callbacks cb 0x7fc4e8009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4eeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4eeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41954/0 (socket says 172.21.15.113:41954) 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.826+0000 7fc4eeffd640 1 -- 172.21.15.113:0/2741815758 learned_addr learned my addr 172.21.15.113:0/2741815758 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4eeffd640 1 -- 172.21.15.113:0/2741815758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 msgr2=0x556662d56e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d56e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4eeffd640 1 -- 172.21.15.113:0/2741815758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 msgr2=0x556662d0de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d0de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4eeffd640 1 -- 172.21.15.113:0/2741815758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556662d575b0 con 0x556662d0f840 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4eeffd640 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d568e0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x556662d1c140 tx=0x7fc4e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c87577ea0f182e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4cdffb640 1 -- 172.21.15.113:0/2741815758 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0015870 con 0x556662d0f840 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4cdffb640 1 -- 172.21.15.113:0/2741815758 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fc4e0015a10 con 0x556662d0f840 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e8032bc0 con 0x556662d0f840 2024-04-25T00:10:22.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4cdffb640 1 -- 172.21.15.113:0/2741815758 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc4e0034050 con 0x556662d0f840 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 10 client.11145 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x556662d0ec20 con 0x556662d0f840 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4cdffb640 1 -- 172.21.15.113:0/2741815758 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8032bc0 con 0x556662d0f840 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4cdffb640 1 -- 172.21.15.113:0/2741815758 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fc4e0037020 con 0x556662d0f840 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 10 client.11145 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 10 client.11145 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 20 client.11145 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d38600 0x556662d0e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 4 client.11145 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.827+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556662c5af40 con 0x556662d38600 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d38600 0x556662d0e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4ee7fc640 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d38600 0x556662d0e380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b14f0f72b5dc27b server_cookie=7d8c533f992b9987 in_seq=0 out_seq=0 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4cdffb640 10 client.11145 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4cdffb640 1 -- 172.21.15.113:0/2741815758 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556662c5af40 con 0x556662d38600 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4cdffb640 10 client.11145 handle_command_reply: tid=0 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 client.11145 shutdown 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 msgr2=0x556662d568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d568e0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x556662d1c140 tx=0x7fc4e0004a40 comp rx=0 tx=0).stop 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 shutdown_connections 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x556662d38600 0x556662d0e380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556662d37600 0x556662d0de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556662d19d60 0x556662d56e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4cdffb640 0 client.11145 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 --2- 172.21.15.113:0/2741815758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556662d0f840 0x556662d568e0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 >> 172.21.15.113:0/2741815758 conn(0x556662cefa40 msgr2=0x556662ce9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 shutdown_connections 2024-04-25T00:10:22.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 1 -- 172.21.15.113:0/2741815758 wait complete. 2024-04-25T00:10:22.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.828+0000 7fc4f8410740 20 client.11145 trim_cache size 0 max 16384 2024-04-25T00:10:22.840 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:10:22.840 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:10:22.840 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:10:22.840 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4dc06be60 msgr2=0x7fc4dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4dc06be60 0x7fc4dc06e320 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0154a40 tx=0x7fc4e00098d0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f013f050 msgr2=0x7fc4f01f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f013f050 0x7fc4f01f3700 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c4a0 tx=0x7fc4e400c960 comp rx=0 tx=0).stop 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 shutdown_connections 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc4dc06be60 0x7fc4dc06e320 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4f01606a0 0x7fc4f014d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f013f050 0x7fc4f01f3700 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 --2- 172.21.15.113:0/3668857616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f00d04f0 0x7fc4f00b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 >> 172.21.15.113:0/3668857616 conn(0x7fc4f006b0f0 msgr2=0x7fc4f013c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 shutdown_connections 2024-04-25T00:10:22.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.829+0000 7fc4f562c640 1 -- 172.21.15.113:0/3668857616 wait complete. 2024-04-25T00:10:22.843 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:10:22.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:10:22.843 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub start / recursive,repair 2024-04-25T00:10:22.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.944+0000 7fb636c64640 1 Processor -- start 2024-04-25T00:10:22.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.944+0000 7fb636c64640 1 -- start start 2024-04-25T00:10:22.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb636c64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63015e2f0 0x7fb6300ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb636c64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb63006e7e0 0x7fb6300b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb636c64640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb630072a80 con 0x7fb6300b2740 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb636c64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb63015e6d0 con 0x7fb63006e7e0 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb636c64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb630072d20 con 0x7fb63015e2f0 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb635201640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb63006e7e0 0x7fb6300b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb635201640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb63006e7e0 0x7fb6300b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41970/0 (socket says 172.21.15.113:41970) 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb635201640 1 -- 172.21.15.113:0/2213804380 learned_addr learned my addr 172.21.15.113:0/2213804380 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb634a00640 1 --2- 172.21.15.113:0/2213804380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63015e2f0 0x7fb6300ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb635201640 1 -- 172.21.15.113:0/2213804380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63015e2f0 msgr2=0x7fb6300ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb635201640 1 --2- 172.21.15.113:0/2213804380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63015e2f0 0x7fb6300ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb635201640 1 -- 172.21.15.113:0/2213804380 >> v1:172.21.15.178:6789/0 conn(0x7fb6300b2740 legacy=0x7fb63006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.945+0000 7fb635201640 1 -- 172.21.15.113:0/2213804380 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6300ad890 con 0x7fb63006e7e0 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.946+0000 7fb635201640 1 --2- 172.21.15.113:0/2213804380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63006e7e0 0x7fb6300b3d20 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fb628009e50 tx=0x7fb62802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5530f7e2af3a160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.946+0000 7fb62f7fe640 1 -- 172.21.15.113:0/2213804380 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628036070 con 0x7fb63006e7e0 2024-04-25T00:10:22.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.946+0000 7fb62f7fe640 1 -- 172.21.15.113:0/2213804380 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb628031600 con 0x7fb63006e7e0 2024-04-25T00:10:22.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.946+0000 7fb62f7fe640 1 -- 172.21.15.113:0/2213804380 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6280319d0 con 0x7fb63006e7e0 2024-04-25T00:10:22.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 -- 172.21.15.113:0/2213804380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63006e7e0 msgr2=0x7fb6300b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 --2- 172.21.15.113:0/2213804380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63006e7e0 0x7fb6300b3d20 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fb628009e50 tx=0x7fb62802a680 comp rx=0 tx=0).stop 2024-04-25T00:10:22.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 -- 172.21.15.113:0/2213804380 shutdown_connections 2024-04-25T00:10:22.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 --2- 172.21.15.113:0/2213804380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63006e7e0 0x7fb6300b3d20 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 --2- 172.21.15.113:0/2213804380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63015e2f0 0x7fb6300ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 -- 172.21.15.113:0/2213804380 >> 172.21.15.113:0/2213804380 conn(0x7fb6300b1b30 msgr2=0x7fb6300b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 -- 172.21.15.113:0/2213804380 shutdown_connections 2024-04-25T00:10:22.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.947+0000 7fb636c64640 1 -- 172.21.15.113:0/2213804380 wait complete. 2024-04-25T00:10:22.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.948+0000 7fb636c64640 1 Processor -- start 2024-04-25T00:10:22.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.948+0000 7fb636c64640 1 -- start start 2024-04-25T00:10:22.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb636c64640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63006e7e0 0x7fb6301f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb636c64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6300b2740 0x7fb6301456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb636c64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63015e2f0 0x7fb6301512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb636c64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6300ae130 con 0x7fb63015e2f0 2024-04-25T00:10:22.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb636c64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6300ade50 con 0x7fb6300b2740 2024-04-25T00:10:22.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb636c64640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6300aed70 con 0x7fb63006e7e0 2024-04-25T00:10:22.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63015e2f0 0x7fb6301512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63015e2f0 0x7fb6301512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41974/0 (socket says 172.21.15.113:41974) 2024-04-25T00:10:22.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 -- 172.21.15.113:0/3349773440 learned_addr learned my addr 172.21.15.113:0/3349773440 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb634a00640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63006e7e0 0x7fb6301f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb62ffff640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6300b2740 0x7fb6301456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 -- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63006e7e0 msgr2=0x7fb6301f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63006e7e0 0x7fb6301f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 -- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6300b2740 msgr2=0x7fb6301456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6300b2740 0x7fb6301456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 -- 172.21.15.113:0/3349773440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb630146130 con 0x7fb63015e2f0 2024-04-25T00:10:22.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb634a00640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63006e7e0 0x7fb6301f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:22.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.949+0000 7fb635201640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63015e2f0 0x7fb6301512a0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fb628034040 tx=0x7fb628004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.950+0000 7fb62d7fa640 1 -- 172.21.15.113:0/3349773440 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628036050 con 0x7fb63015e2f0 2024-04-25T00:10:22.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.950+0000 7fb62d7fa640 1 -- 172.21.15.113:0/3349773440 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb628042070 con 0x7fb63015e2f0 2024-04-25T00:10:22.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.950+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6301f4830 con 0x7fb63015e2f0 2024-04-25T00:10:22.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.950+0000 7fb62d7fa640 1 -- 172.21.15.113:0/3349773440 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628009510 con 0x7fb63015e2f0 2024-04-25T00:10:22.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.950+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6301f4df0 con 0x7fb63015e2f0 2024-04-25T00:10:22.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.951+0000 7fb62d7fa640 1 -- 172.21.15.113:0/3349773440 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb628007bd0 con 0x7fb63015e2f0 2024-04-25T00:10:22.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.952+0000 7fb62d7fa640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb61406be30 0x7fb61406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.952+0000 7fb634a00640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb61406be30 0x7fb61406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.952+0000 7fb62d7fa640 1 -- 172.21.15.113:0/3349773440 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fb6280abce0 con 0x7fb63015e2f0 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.952+0000 7fb634a00640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb61406be30 0x7fb61406e2f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fb6300b00d0 tx=0x7fb624008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 Processor -- start 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 -- start start 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db4583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db456660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db4811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5607db3b92c0 con 0x5607db480d40 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5607db361910 con 0x5607db492950 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5607db3b5280 con 0x5607db456280 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb635201640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db4811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb635201640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db4811c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41984/0 (socket says 172.21.15.113:41984) 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.953+0000 7fb635201640 1 -- 172.21.15.113:0/3544595781 learned_addr learned my addr 172.21.15.113:0/3544595781 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb635201640 1 -- 172.21.15.113:0/3544595781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 msgr2=0x5607db4583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb635201640 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db4583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb635201640 1 -- 172.21.15.113:0/3544595781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 msgr2=0x5607db456660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb62ffff640 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db456660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb635201640 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db456660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb635201640 1 -- 172.21.15.113:0/3544595781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607db372fb0 con 0x5607db480d40 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb635201640 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db4811c0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fb6280aa1d0 tx=0x7fb628079a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc51ccc4eb716204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb61a7fc640 1 -- 172.21.15.113:0/3544595781 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6280a8070 con 0x5607db480d40 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb61a7fc640 1 -- 172.21.15.113:0/3544595781 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb628037040 con 0x5607db480d40 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.954+0000 7fb61a7fc640 1 -- 172.21.15.113:0/3544595781 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62807a630 con 0x5607db480d40 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 -- 172.21.15.113:0/3544595781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 msgr2=0x5607db4811c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db4811c0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fb6280aa1d0 tx=0x7fb628079a70 comp rx=0 tx=0).stop 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 -- 172.21.15.113:0/3544595781 shutdown_connections 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db456660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db4811c0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 --2- 172.21.15.113:0/3544595781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db4583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 -- 172.21.15.113:0/3544595781 >> 172.21.15.113:0/3544595781 conn(0x5607db41d950 msgr2=0x5607db49bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 -- 172.21.15.113:0/3544595781 shutdown_connections 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 -- 172.21.15.113:0/3544595781 wait complete. 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 -- start start 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.955+0000 7fb639971740 1 Processor -- start 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db481fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb639971740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db46f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db46f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb639971740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5607db3b92c0 con 0x5607db480d40 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5607db374e90 con 0x5607db492950 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5607db483000 con 0x5607db456280 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb639971740 10 client.0 _ll_register_callbacks cb 0x5607db3b0970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb62ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db46f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb62ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db46f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41990/0 (socket says 172.21.15.113:41990) 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb62ffff640 1 -- 172.21.15.113:0/2333018879 learned_addr learned my addr 172.21.15.113:0/2333018879 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb635201640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db46f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.956+0000 7fb634a00640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db481fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb62ffff640 1 -- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 msgr2=0x5607db481fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb62ffff640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db481fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb62ffff640 1 -- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 msgr2=0x5607db46f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb62ffff640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db46f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb62ffff640 1 -- 172.21.15.113:0/2333018879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607db1957e0 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb635201640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db46f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb62ffff640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db46f0a0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fb61c003af0 tx=0x7fb61c0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cbb4119e075b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb60f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2333018879 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c001b30 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2333018879 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fb61c001cd0 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb628037040 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2333018879 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fb61c021050 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 10 client.11157 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5607db1f5f70 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2333018879 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb628037040 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2333018879 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fb61c024020 con 0x5607db480d40 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 10 client.11157 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 10 client.11157 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 20 client.11157 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db4b61b0 0x5607db4b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 4 client.11157 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607db38d5a0 con 0x5607db4b61b0 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.957+0000 7fb634a00640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db4b61b0 0x5607db4b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb60f7fe640 10 client.11157 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb634a00640 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db4b61b0 0x5607db4b85a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd45736b403bfb1 server_cookie=474a27f70de4421f in_seq=0 out_seq=0 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2333018879 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607db38d5a0 con 0x5607db4b61b0 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb60f7fe640 10 client.11157 handle_command_reply: tid=0 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 client.11157 shutdown 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 msgr2=0x5607db46f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db46f0a0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fb61c003af0 tx=0x7fb61c0055d0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 shutdown_connections 2024-04-25T00:10:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db4b61b0 0x5607db4b85a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db46f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db480d40 0x5607db46f0a0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb60f7fe640 0 client.11157 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 --2- 172.21.15.113:0/2333018879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db481fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.958+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 >> 172.21.15.113:0/2333018879 conn(0x5607db41d950 msgr2=0x5607db49f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.959+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 shutdown_connections 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.959+0000 7fb639971740 1 -- 172.21.15.113:0/2333018879 wait complete. 2024-04-25T00:10:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.959+0000 7fb639971740 20 client.11157 trim_cache size 0 max 16384 2024-04-25T00:10:22.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.976+0000 7fb639971740 1 Processor -- start 2024-04-25T00:10:22.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.976+0000 7fb639971740 1 -- start start 2024-04-25T00:10:22.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb639971740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db456280 0x5607db499bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db480d40 0x5607db481360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db4818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb639971740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb630200950 con 0x5607db456280 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ff9c0 con 0x5607db492950 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6301f79e0 con 0x5607db480d40 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db456280 0x5607db499bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db456280 0x5607db499bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42004/0 (socket says 172.21.15.113:42004) 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 -- 172.21.15.113:0/3097168986 learned_addr learned my addr 172.21.15.113:0/3097168986 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb635201640 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db4818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb62ffff640 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db480d40 0x5607db481360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 -- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db480d40 msgr2=0x5607db481360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db480d40 0x5607db481360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 -- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 msgr2=0x5607db4818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db4818a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 -- 172.21.15.113:0/3097168986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607db08d170 con 0x5607db456280 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb634a00640 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db456280 0x5607db499bb0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fb62400f5b0 tx=0x7fb624012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9c0c7e75841be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb619ffb640 1 -- 172.21.15.113:0/3097168986 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624026220 con 0x5607db456280 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb619ffb640 1 -- 172.21.15.113:0/3097168986 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62402c070 con 0x5607db456280 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.977+0000 7fb619ffb640 1 -- 172.21.15.113:0/3097168986 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240265c0 con 0x5607db456280 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 -- 172.21.15.113:0/3097168986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db456280 msgr2=0x5607db499bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db456280 0x5607db499bb0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fb62400f5b0 tx=0x7fb624012000 comp rx=0 tx=0).stop 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 -- 172.21.15.113:0/3097168986 shutdown_connections 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db492950 0x5607db4818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db480d40 0x5607db481360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 --2- 172.21.15.113:0/3097168986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db456280 0x5607db499bb0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 -- 172.21.15.113:0/3097168986 >> 172.21.15.113:0/3097168986 conn(0x5607db41d950 msgr2=0x5607db484970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 -- 172.21.15.113:0/3097168986 shutdown_connections 2024-04-25T00:10:22.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 -- 172.21.15.113:0/3097168986 wait complete. 2024-04-25T00:10:22.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.978+0000 7fb639971740 1 -- start start 2024-04-25T00:10:22.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 1 Processor -- start 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db515c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db480d40 0x5607db4fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db492950 0x5607db481360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5607db46f820 con 0x5607db492950 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301f8030 con 0x5607db480d40 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5607db49dd70 con 0x5607db456280 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb639971740 10 client.0 _ll_register_callbacks cb 0x7fb61c005090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb635201640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db492950 0x5607db481360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.979+0000 7fb635201640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db492950 0x5607db481360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42018/0 (socket says 172.21.15.113:42018) 2024-04-25T00:10:22.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb635201640 1 -- 172.21.15.113:0/2201949042 learned_addr learned my addr 172.21.15.113:0/2201949042 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb635201640 1 -- 172.21.15.113:0/2201949042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 msgr2=0x5607db515c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb635201640 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db515c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb635201640 1 -- 172.21.15.113:0/2201949042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db480d40 msgr2=0x5607db4fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb62ffff640 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db480d40 0x5607db4fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb635201640 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db480d40 0x5607db4fde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb635201640 1 -- 172.21.15.113:0/2201949042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607db4be1a0 con 0x5607db492950 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb635201640 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db492950 0x5607db481360 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fb628007fd0 tx=0x7fb628009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c4d7abad43b7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb60f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2201949042 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6280a8070 con 0x5607db492950 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2201949042 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fb628079580 con 0x5607db492950 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62402c070 con 0x5607db492950 2024-04-25T00:10:22.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2201949042 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fb62807a630 con 0x5607db492950 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 10 client.11163 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5607db492370 con 0x5607db492950 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2201949042 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62402c070 con 0x5607db492950 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2201949042 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fb6280a3b10 con 0x5607db492950 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 10 client.11163 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 10 client.11163 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 20 client.11163 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db483be0 0x5607db4818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 4 client.11163 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.980+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) -- 0x5607db3c6800 con 0x5607db483be0 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.981+0000 7fb634a00640 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db483be0 0x5607db4818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:22.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.981+0000 7fb60f7fe640 10 client.11163 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.981+0000 7fb634a00640 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db483be0 0x5607db4818a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4fb212f32955b01 server_cookie=e994184ecd42f3eb in_seq=0 out_seq=0 2024-04-25T00:10:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.981+0000 7fb60f7fe640 1 -- 172.21.15.113:0/2201949042 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5607db3c6800 con 0x5607db483be0 2024-04-25T00:10:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.981+0000 7fb60f7fe640 10 client.11163 handle_command_reply: tid=0 2024-04-25T00:10:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.981+0000 7fb639971740 1 client.11163 shutdown 2024-04-25T00:10:22.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db492950 msgr2=0x5607db481360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db492950 0x5607db481360 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fb628007fd0 tx=0x7fb628009810 comp rx=0 tx=0).stop 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 shutdown_connections 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x5607db483be0 0x5607db4818a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607db480d40 0x5607db4fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5607db492950 0x5607db481360 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb60f7fe640 0 client.11163 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 --2- 172.21.15.113:0/2201949042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5607db456280 0x5607db515c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 >> 172.21.15.113:0/2201949042 conn(0x5607db41d950 msgr2=0x5607db49e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 shutdown_connections 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 1 -- 172.21.15.113:0/2201949042 wait complete. 2024-04-25T00:10:22.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb639971740 20 client.11163 trim_cache size 0 max 16384 2024-04-25T00:10:22.991 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:10:22.991 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-25T00:10:22.991 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "896b5c83-1514-48b6-84b1-c00637b54da6", 2024-04-25T00:10:22.991 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-25T00:10:22.991 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:10:22.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb61406be30 msgr2=0x7fb61406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb61406be30 0x7fb61406e2f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fb6300b00d0 tx=0x7fb624008040 comp rx=0 tx=0).stop 2024-04-25T00:10:22.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63015e2f0 msgr2=0x7fb6301512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63015e2f0 0x7fb6301512a0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fb628034040 tx=0x7fb628004a40 comp rx=0 tx=0).stop 2024-04-25T00:10:22.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 shutdown_connections 2024-04-25T00:10:22.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb61406be30 0x7fb61406e2f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6300b2740 0x7fb6301456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63015e2f0 0x7fb6301512a0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 --2- 172.21.15.113:0/3349773440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb63006e7e0 0x7fb6301f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.982+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 >> 172.21.15.113:0/3349773440 conn(0x7fb6300b1b30 msgr2=0x7fb6300b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.983+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 shutdown_connections 2024-04-25T00:10:22.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:22.983+0000 7fb636c64640 1 -- 172.21.15.113:0/3349773440 wait complete. 2024-04-25T00:10:22.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:10:22.994 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T00:10:23.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.094+0000 7f6f2649a640 1 Processor -- start 2024-04-25T00:10:23.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.094+0000 7f6f2649a640 1 -- start start 2024-04-25T00:10:23.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f2649a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f200afa40 0x7f6f200cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f2649a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6f2015f230 0x7f6f2015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f2649a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f20158f60 con 0x7f6f200afe20 2024-04-25T00:10:23.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f2649a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6f20071ac0 con 0x7f6f200afa40 2024-04-25T00:10:23.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f2649a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6f2015fa50 con 0x7f6f2015f230 2024-04-25T00:10:23.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f200afa40 0x7f6f200cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6f200afe20 0x7f6f2015eb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35100/0 (socket says 172.21.15.113:35100) 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1f7fe640 1 -- 172.21.15.113:0/1848921119 learned_addr learned my addr 172.21.15.113:0/1848921119 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f24a37640 1 --2- 172.21.15.113:0/1848921119 >> v2:172.21.15.178:3300/0 conn(0x7f6f2015f230 0x7f6f2015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1ffff640 1 -- 172.21.15.113:0/1848921119 >> v2:172.21.15.178:3300/0 conn(0x7f6f2015f230 msgr2=0x7f6f2015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/1848921119 >> v2:172.21.15.178:3300/0 conn(0x7f6f2015f230 0x7f6f2015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1ffff640 1 -- 172.21.15.113:0/1848921119 >> v1:172.21.15.113:6789/0 conn(0x7f6f200afe20 legacy=0x7f6f2015eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1ffff640 1 -- 172.21.15.113:0/1848921119 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2015f770 con 0x7f6f200afa40 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.095+0000 7f6f1effd640 1 -- 172.21.15.113:0/1848921119 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606053624 0 0) 0x7f6f20158f60 con 0x7f6f200afe20 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/1848921119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f200afa40 0x7f6f200cd720 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f6f200711e0 tx=0x7f6f1002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa9193d5ccf871e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f1effd640 1 -- 172.21.15.113:0/1848921119 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10037070 con 0x7f6f200afa40 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f1effd640 1 -- 172.21.15.113:0/1848921119 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f10032440 con 0x7f6f200afa40 2024-04-25T00:10:23.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f1effd640 1 -- 172.21.15.113:0/1848921119 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1002ae60 con 0x7f6f200afa40 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f2649a640 1 -- 172.21.15.113:0/1848921119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f200afa40 msgr2=0x7f6f200cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f2649a640 1 --2- 172.21.15.113:0/1848921119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f200afa40 0x7f6f200cd720 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f6f200711e0 tx=0x7f6f1002a110 comp rx=0 tx=0).stop 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f2649a640 1 -- 172.21.15.113:0/1848921119 shutdown_connections 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f2649a640 1 --2- 172.21.15.113:0/1848921119 >> v2:172.21.15.178:3300/0 conn(0x7f6f2015f230 0x7f6f2015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f2649a640 1 --2- 172.21.15.113:0/1848921119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f200afa40 0x7f6f200cd720 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.096+0000 7f6f2649a640 1 -- 172.21.15.113:0/1848921119 >> 172.21.15.113:0/1848921119 conn(0x7f6f200b57e0 msgr2=0x7f6f20072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.097+0000 7f6f2649a640 1 -- 172.21.15.113:0/1848921119 shutdown_connections 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.097+0000 7f6f2649a640 1 -- 172.21.15.113:0/1848921119 wait complete. 2024-04-25T00:10:23.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.098+0000 7f6f2649a640 1 Processor -- start 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.098+0000 7f6f2649a640 1 -- start start 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f200afa40 0x7f6f20156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f200afe20 0x7f6f200d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2015f230 0x7f6f2006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6f200ccf20 con 0x7f6f2015f230 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f200cd290 con 0x7f6f200afa40 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6f200b02a0 con 0x7f6f200afe20 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2015f230 0x7f6f2006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2015f230 0x7f6f2006d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42040/0 (socket says 172.21.15.113:42040) 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 -- 172.21.15.113:0/3803489942 learned_addr learned my addr 172.21.15.113:0/3803489942 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f200afa40 0x7f6f20156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f200afe20 0x7f6f200d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 -- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f200afe20 msgr2=0x7f6f200d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f200afe20 0x7f6f200d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 -- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f200afa40 msgr2=0x7f6f20156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f200afa40 0x7f6f20156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 -- 172.21.15.113:0/3803489942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f20156b60 con 0x7f6f2015f230 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f24a37640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2015f230 0x7f6f2006d940 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f6f18007d80 tx=0x7f6f1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f1cff9640 1 -- 172.21.15.113:0/3803489942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18014420 con 0x7f6f2015f230 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f1cff9640 1 -- 172.21.15.113:0/3803489942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18019070 con 0x7f6f2015f230 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f1cff9640 1 -- 172.21.15.113:0/3803489942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18014800 con 0x7f6f2015f230 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f200ae570 con 0x7f6f2015f230 2024-04-25T00:10:23.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.099+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f200aeb30 con 0x7f6f2015f230 2024-04-25T00:10:23.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.101+0000 7f6f1cff9640 1 -- 172.21.15.113:0/3803489942 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f180149a0 con 0x7f6f2015f230 2024-04-25T00:10:23.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.101+0000 7f6f1cff9640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f0c06be30 0x7f6f0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.101+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f0c06be30 0x7f6f0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.101+0000 7f6f1cff9640 1 -- 172.21.15.113:0/3803489942 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f6f1808b770 con 0x7f6f2015f230 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.102+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f0c06be30 0x7f6f0c06e2f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6f10034000 tx=0x7f6f1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.102+0000 7f6f291a3740 1 Processor -- start 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.102+0000 7f6f291a3740 1 -- start start 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae43d60 0x559e6ae44140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 0x559e6ae61b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae19a40 0x559e6ae5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f291a3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559e6ad7d6b0 con 0x559e6ae19a40 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559e6ad25880 con 0x559e6ae43d60 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559e6ad79670 con 0x559e6ae61600 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae19a40 0x559e6ae5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae19a40 0x559e6ae5fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42044/0 (socket says 172.21.15.113:42044) 2024-04-25T00:10:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 -- 172.21.15.113:0/4284545964 learned_addr learned my addr 172.21.15.113:0/4284545964 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 0x559e6ae61b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae43d60 0x559e6ae44140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 -- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 msgr2=0x559e6ae61b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 0x559e6ae61b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 -- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae43d60 msgr2=0x559e6ae44140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae43d60 0x559e6ae44140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 -- 172.21.15.113:0/4284545964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6ab5b7d0 con 0x559e6ae19a40 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f24a37640 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae19a40 0x559e6ae5fcc0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c040 tx=0x7f6f18059390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67ad60adb0cecac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.103+0000 7f6f05ffb640 1 -- 172.21.15.113:0/4284545964 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1805ee60 con 0x559e6ae19a40 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f05ffb640 1 -- 172.21.15.113:0/4284545964 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18057c20 con 0x559e6ae19a40 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f05ffb640 1 -- 172.21.15.113:0/4284545964 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f180600c0 con 0x559e6ae19a40 2024-04-25T00:10:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 -- 172.21.15.113:0/4284545964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae19a40 msgr2=0x559e6ae5fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae19a40 0x559e6ae5fcc0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c040 tx=0x7f6f18059390 comp rx=0 tx=0).stop 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 -- 172.21.15.113:0/4284545964 shutdown_connections 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae19a40 0x559e6ae5fcc0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 0x559e6ae61b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 --2- 172.21.15.113:0/4284545964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae43d60 0x559e6ae44140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 -- 172.21.15.113:0/4284545964 >> 172.21.15.113:0/4284545964 conn(0x559e6ae16600 msgr2=0x559e6ae5eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 -- 172.21.15.113:0/4284545964 shutdown_connections 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.104+0000 7f6f291a3740 1 -- 172.21.15.113:0/4284545964 wait complete. 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.105+0000 7f6f291a3740 1 -- start start 2024-04-25T00:10:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.105+0000 7f6f291a3740 1 Processor -- start 2024-04-25T00:10:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae43d60 0x559e6ae3c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae61600 0x559e6ae5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559e6ae1a7e0 con 0x559e6ae61600 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559e6ad38f50 con 0x559e6ae5dbd0 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559e6ae14530 con 0x559e6ae43d60 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 10 client.0 _ll_register_callbacks cb 0x559e6ad74000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae61600 0x559e6ae5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae61600 0x559e6ae5d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42056/0 (socket says 172.21.15.113:42056) 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 -- 172.21.15.113:0/2311484361 learned_addr learned my addr 172.21.15.113:0/2311484361 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae43d60 0x559e6ae3c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f24a37640 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 -- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae43d60 msgr2=0x559e6ae3c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae43d60 0x559e6ae3c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 -- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 msgr2=0x559e6ae728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 -- 172.21.15.113:0/2311484361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6aa913f0 con 0x559e6ae61600 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae61600 0x559e6ae5d690 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f6f140011e0 tx=0x7f6f14007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c8e45d98ef3a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6efeffd640 1 -- 172.21.15.113:0/2311484361 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14001c80 con 0x559e6ae61600 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6efeffd640 1 -- 172.21.15.113:0/2311484361 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f6f14014400 con 0x559e6ae61600 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.106+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f18057c20 con 0x559e6ae61600 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6efeffd640 1 -- 172.21.15.113:0/2311484361 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f6f14021050 con 0x559e6ae61600 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 10 client.11178 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x559e6aa510f0 con 0x559e6ae61600 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6efeffd640 1 -- 172.21.15.113:0/2311484361 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18057c20 con 0x559e6ae61600 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6efeffd640 1 -- 172.21.15.113:0/2311484361 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f6f14024020 con 0x559e6ae61600 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 10 client.11178 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 10 client.11178 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 20 client.11178 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae13260 0x559e6ae13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 4 client.11178 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6ad8abf0 con 0x559e6ae13260 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae13260 0x559e6ae13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae13260 0x559e6ae13660 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29258db81c9cce7 server_cookie=d36b4e93863d9a7d in_seq=0 out_seq=0 2024-04-25T00:10:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.107+0000 7f6efeffd640 10 client.11178 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6efeffd640 1 -- 172.21.15.113:0/2311484361 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e6ad8abf0 con 0x559e6ae13260 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6efeffd640 10 client.11178 handle_command_reply: tid=0 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 client.11178 shutdown 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae61600 msgr2=0x559e6ae5d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae61600 0x559e6ae5d690 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f6f140011e0 tx=0x7f6f14007780 comp rx=0 tx=0).stop 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 shutdown_connections 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae13260 0x559e6ae13660 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae61600 0x559e6ae5d690 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2311484361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae43d60 0x559e6ae3c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6efeffd640 0 client.11178 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 >> 172.21.15.113:0/2311484361 conn(0x559e6ae19a40 msgr2=0x559e6ae620f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 shutdown_connections 2024-04-25T00:10:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 1 -- 172.21.15.113:0/2311484361 wait complete. 2024-04-25T00:10:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.108+0000 7f6f291a3740 20 client.11178 trim_cache size 0 max 16384 2024-04-25T00:10:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.125+0000 7f6f291a3740 1 Processor -- start 2024-04-25T00:10:23.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.125+0000 7f6f291a3740 1 -- start start 2024-04-25T00:10:23.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae13640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 0x559e6ae70d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f291a3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6f202008b0 con 0x559e6ae43d60 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201ff990 con 0x559e6ae5dbd0 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6f200ade60 con 0x559e6ae61600 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42068/0 (socket says 172.21.15.113:42068) 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 -- 172.21.15.113:0/17406786 learned_addr learned my addr 172.21.15.113:0/17406786 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 -- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 msgr2=0x559e6ae70d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 0x559e6ae70d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae13640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 -- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 msgr2=0x559e6ae13640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae13640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1ffff640 1 -- 172.21.15.113:0/17406786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6ae441e0 con 0x559e6ae43d60 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.126+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae13640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e240 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035040 tx=0x7f6f10008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4578b2acdb9f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f057fa640 1 -- 172.21.15.113:0/17406786 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10057070 con 0x559e6ae43d60 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f057fa640 1 -- 172.21.15.113:0/17406786 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f10032c50 con 0x559e6ae43d60 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f057fa640 1 -- 172.21.15.113:0/17406786 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10037040 con 0x559e6ae43d60 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 -- 172.21.15.113:0/17406786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 msgr2=0x559e6ae6e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e240 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035040 tx=0x7f6f10008030 comp rx=0 tx=0).stop 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 -- 172.21.15.113:0/17406786 shutdown_connections 2024-04-25T00:10:23.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae5dbd0 0x559e6ae13640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae61600 0x559e6ae70d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 --2- 172.21.15.113:0/17406786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e240 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 -- 172.21.15.113:0/17406786 >> 172.21.15.113:0/17406786 conn(0x559e6ae19a40 msgr2=0x559e6ae3c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 -- 172.21.15.113:0/17406786 shutdown_connections 2024-04-25T00:10:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.127+0000 7f6f291a3740 1 -- 172.21.15.113:0/17406786 wait complete. 2024-04-25T00:10:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.128+0000 7f6f291a3740 1 -- start start 2024-04-25T00:10:23.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.128+0000 7f6f291a3740 1 Processor -- start 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae5dbd0 0x559e6ae70d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae61600 0x559e6ae712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559e6ae43930 con 0x559e6ae43d60 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559e6ad03500 con 0x559e6ae61600 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559e6ae1afb0 con 0x559e6ae5dbd0 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 10 client.0 _ll_register_callbacks cb 0x7f6f14007df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:23.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42076/0 (socket says 172.21.15.113:42076) 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 -- 172.21.15.113:0/2758721917 learned_addr learned my addr 172.21.15.113:0/2758721917 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae5dbd0 0x559e6ae70d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f24a37640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae61600 0x559e6ae712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 -- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae5dbd0 msgr2=0x559e6ae70d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae5dbd0 0x559e6ae70d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 -- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae61600 msgr2=0x559e6ae712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae61600 0x559e6ae712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 -- 172.21.15.113:0/2758721917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6ae56a20 con 0x559e6ae43d60 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f24a37640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae61600 0x559e6ae712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f1ffff640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e2c0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035040 tx=0x7f6f10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e289d08f8193727d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6efeffd640 1 -- 172.21.15.113:0/2758721917 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10057030 con 0x559e6ae43d60 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6efeffd640 1 -- 172.21.15.113:0/2758721917 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f6f1005eb20 con 0x559e6ae43d60 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f10032c50 con 0x559e6ae43d60 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6efeffd640 1 -- 172.21.15.113:0/2758721917 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f6f10037040 con 0x559e6ae43d60 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 10 client.11184 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.129+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x559e6ae5ef80 con 0x559e6ae43d60 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6efeffd640 1 -- 172.21.15.113:0/2758721917 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f10032c50 con 0x559e6ae43d60 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6efeffd640 1 -- 172.21.15.113:0/2758721917 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f6f1005e020 con 0x559e6ae43d60 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 10 client.11184 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 10 client.11184 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 20 client.11184 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae562b0 0x559e6ae5dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 4 client.11184 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e6ae578e0 con 0x559e6ae562b0 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae562b0 0x559e6ae5dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6efeffd640 10 client.11184 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f1f7fe640 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae562b0 0x559e6ae5dfb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d0034377cf91a8 server_cookie=cd87711ab80aa478 in_seq=0 out_seq=0 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6efeffd640 1 -- 172.21.15.113:0/2758721917 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x559e6ae578e0 con 0x559e6ae562b0 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6efeffd640 10 client.11184 handle_command_reply: tid=0 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 client.11184 shutdown 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 msgr2=0x559e6ae6e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e2c0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035040 tx=0x7f6f10004730 comp rx=0 tx=0).stop 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 shutdown_connections 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x559e6ae562b0 0x559e6ae5dfb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e6ae61600 0x559e6ae712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559e6ae5dbd0 0x559e6ae70d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6efeffd640 0 client.11184 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 --2- 172.21.15.113:0/2758721917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e6ae43d60 0x559e6ae6e2c0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.130+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 >> 172.21.15.113:0/2758721917 conn(0x559e6ae19a40 msgr2=0x559e6ae5d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 shutdown_connections 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f291a3740 1 -- 172.21.15.113:0/2758721917 wait complete. 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f291a3740 20 client.11184 trim_cache size 0 max 16384 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:10:23.139 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout: "896b5c83-1514-48b6-84b1-c00637b54da6": { 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "896b5c83-1514-48b6-84b1-c00637b54da6", 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,repair" 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-25T00:10:23.140 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:10:23.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f0c06be30 msgr2=0x7f6f0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f0c06be30 0x7f6f0c06e2f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6f10034000 tx=0x7f6f1002e000 comp rx=0 tx=0).stop 2024-04-25T00:10:23.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2015f230 msgr2=0x7f6f2006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:23.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2015f230 0x7f6f2006d940 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f6f18007d80 tx=0x7f6f1800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 shutdown_connections 2024-04-25T00:10:23.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6f0c06be30 0x7f6f0c06e2f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2015f230 0x7f6f2006d940 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f200afe20 0x7f6f200d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 --2- 172.21.15.113:0/3803489942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f200afa40 0x7f6f20156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:23.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 >> 172.21.15.113:0/3803489942 conn(0x7f6f200b57e0 msgr2=0x7f6f201585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:23.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 shutdown_connections 2024-04-25T00:10:23.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:23.131+0000 7f6f2649a640 1 -- 172.21.15.113:0/3803489942 wait complete. 2024-04-25T00:10:23.143 INFO:tasks.cephfs.filesystem:scrub status for tag:896b5c83-1514-48b6-84b1-c00637b54da6 - {'path': '/', 'tag': '896b5c83-1514-48b6-84b1-c00637b54da6', 'options': 'recursive,repair'} 2024-04-25T00:10:24.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:24.339 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:53.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:10:53.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T00:10:53.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.253+0000 7fa2ce3a5640 1 Processor -- start 2024-04-25T00:10:53.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.253+0000 7fa2ce3a5640 1 -- start start 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.253+0000 7fa2ce3a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2c80b5ca0 0x7fa2c80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.253+0000 7fa2ce3a5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa2c80b73c0 0x7fa2c80b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.253+0000 7fa2ce3a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c80d2e00 con 0x7fa2c815e510 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.253+0000 7fa2ce3a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c80b6640 con 0x7fa2c80b5ca0 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.253+0000 7fa2ce3a5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c80d30e0 con 0x7fa2c80b73c0 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2c80b5ca0 0x7fa2c80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa2c815e510 0x7fa2c80d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34394/0 (socket says 172.21.15.113:34394) 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c7fff640 1 -- 172.21.15.113:0/733456409 learned_addr learned my addr 172.21.15.113:0/733456409 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2cc942640 1 --2- 172.21.15.113:0/733456409 >> v2:172.21.15.178:3300/0 conn(0x7fa2c80b73c0 0x7fa2c80b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c77fe640 1 -- 172.21.15.113:0/733456409 >> v2:172.21.15.178:3300/0 conn(0x7fa2c80b73c0 msgr2=0x7fa2c80b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/733456409 >> v2:172.21.15.178:3300/0 conn(0x7fa2c80b73c0 0x7fa2c80b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c77fe640 1 -- 172.21.15.113:0/733456409 >> v1:172.21.15.113:6789/0 conn(0x7fa2c815e510 legacy=0x7fa2c80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c77fe640 1 -- 172.21.15.113:0/733456409 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c80d33c0 con 0x7fa2c80b5ca0 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2c6ffd640 1 -- 172.21.15.113:0/733456409 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362714813 0 0) 0x7fa2c80d2e00 con 0x7fa2c815e510 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.254+0000 7fa2cc942640 1 --2- 172.21.15.113:0/733456409 >> v2:172.21.15.178:3300/0 conn(0x7fa2c80b73c0 0x7fa2c80b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/733456409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b5ca0 0x7fa2c80b6100 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80d1d80 tx=0x7fa2bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52b24c1524a1966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2c6ffd640 1 -- 172.21.15.113:0/733456409 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc037070 con 0x7fa2c80b5ca0 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2c6ffd640 1 -- 172.21.15.113:0/733456409 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc032440 con 0x7fa2c80b5ca0 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2c6ffd640 1 -- 172.21.15.113:0/733456409 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc02ae60 con 0x7fa2c80b5ca0 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/733456409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b5ca0 msgr2=0x7fa2c80b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/733456409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b5ca0 0x7fa2c80b6100 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80d1d80 tx=0x7fa2bc02a110 comp rx=0 tx=0).stop 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/733456409 shutdown_connections 2024-04-25T00:10:53.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/733456409 >> v2:172.21.15.178:3300/0 conn(0x7fa2c80b73c0 0x7fa2c80b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/733456409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b5ca0 0x7fa2c80b6100 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/733456409 >> 172.21.15.113:0/733456409 conn(0x7fa2c806b720 msgr2=0x7fa2c80d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/733456409 shutdown_connections 2024-04-25T00:10:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.255+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/733456409 wait complete. 2024-04-25T00:10:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.256+0000 7fa2ce3a5640 1 Processor -- start 2024-04-25T00:10:53.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2ce3a5640 1 -- start start 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2ce3a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c80b5ca0 0x7fa2c81f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2ce3a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b73c0 0x7fa2c8161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2ce3a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2c815e510 0x7fa2c80ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2ce3a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c80d3ea0 con 0x7fa2c80b73c0 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2ce3a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c80d3980 con 0x7fa2c80b5ca0 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2ce3a5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c80d2a90 con 0x7fa2c815e510 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.257+0000 7fa2c77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b73c0 0x7fa2c8161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b73c0 0x7fa2c8161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47946/0 (socket says 172.21.15.113:47946) 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 -- 172.21.15.113:0/1465451104 learned_addr learned my addr 172.21.15.113:0/1465451104 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:53.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 -- 172.21.15.113:0/1465451104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2c815e510 msgr2=0x7fa2c80ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2c815e510 0x7fa2c80ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 -- 172.21.15.113:0/1465451104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c80b5ca0 msgr2=0x7fa2c81f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c80b5ca0 0x7fa2c81f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 -- 172.21.15.113:0/1465451104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c80c9f80 con 0x7fa2c80b73c0 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b73c0 0x7fa2c8161010 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c4ff9640 1 -- 172.21.15.113:0/1465451104 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc037040 con 0x7fa2c80b73c0 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c4ff9640 1 -- 172.21.15.113:0/1465451104 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc042070 con 0x7fa2c80b73c0 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2c4ff9640 1 -- 172.21.15.113:0/1465451104 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc030a50 con 0x7fa2c80b73c0 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c80ca260 con 0x7fa2c80b73c0 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.258+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c80ca820 con 0x7fa2c80b73c0 2024-04-25T00:10:53.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.260+0000 7fa2c4ff9640 1 -- 172.21.15.113:0/1465451104 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2bc030bf0 con 0x7fa2c80b73c0 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.260+0000 7fa2c4ff9640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2b406be30 0x7fa2b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.260+0000 7fa2c4ff9640 1 -- 172.21.15.113:0/1465451104 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fa2bc0ab9c0 con 0x7fa2c80b73c0 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.260+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2b406be30 0x7fa2b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.261+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2b406be30 0x7fa2b406e2f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8007fd0 tx=0x7fa2b8007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.261+0000 7fa2d118f740 1 Processor -- start 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.261+0000 7fa2d118f740 1 -- start start 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2a2a40 0x557eeb2e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2d118f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557eeb2066b0 con 0x557eeb2a2a40 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557eeb1ae880 con 0x557eeb2ccd60 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557eeb202670 con 0x557eeb2ea600 2024-04-25T00:10:53.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2a2a40 0x557eeb2e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2a2a40 0x557eeb2e8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47960/0 (socket says 172.21.15.113:47960) 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 -- 172.21.15.113:0/2355517129 learned_addr learned my addr 172.21.15.113:0/2355517129 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 -- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 msgr2=0x557eeb2eab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2eab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 -- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 msgr2=0x557eeb2cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 -- 172.21.15.113:0/2355517129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eeafe47d0 con 0x557eeb2a2a40 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2cc942640 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2a2a40 0x557eeb2e8cc0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0008e30 tx=0x7fa2c000d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b8a0a22b91ffd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2adffb640 1 -- 172.21.15.113:0/2355517129 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000f800 con 0x557eeb2a2a40 2024-04-25T00:10:53.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2adffb640 1 -- 172.21.15.113:0/2355517129 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c000f9a0 con 0x557eeb2a2a40 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.262+0000 7fa2adffb640 1 -- 172.21.15.113:0/2355517129 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000fd70 con 0x557eeb2a2a40 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 -- 172.21.15.113:0/2355517129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2a2a40 msgr2=0x557eeb2e8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2a2a40 0x557eeb2e8cc0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0008e30 tx=0x7fa2c000d6e0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 -- 172.21.15.113:0/2355517129 shutdown_connections 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2a2a40 0x557eeb2e8cc0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 --2- 172.21.15.113:0/2355517129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 -- 172.21.15.113:0/2355517129 >> 172.21.15.113:0/2355517129 conn(0x557eeb29f600 msgr2=0x557eeb2e7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 -- 172.21.15.113:0/2355517129 shutdown_connections 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.263+0000 7fa2d118f740 1 -- 172.21.15.113:0/2355517129 wait complete. 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.264+0000 7fa2d118f740 1 -- start start 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.264+0000 7fa2d118f740 1 Processor -- start 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.264+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.264+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2ea600 0x557eeb2c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.264+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2b5070 0x557eeb2cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.264+0000 7fa2d118f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557eeb2a37e0 con 0x557eeb2ea600 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557eeb1c1f50 con 0x557eeb2ccd60 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557eeb29d530 con 0x557eeb2b5070 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2d118f740 10 client.0 _ll_register_callbacks cb 0x557eeb1fd000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2ea600 0x557eeb2c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2ea600 0x557eeb2c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47966/0 (socket says 172.21.15.113:47966) 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 -- 172.21.15.113:0/3517388011 learned_addr learned my addr 172.21.15.113:0/3517388011 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2cc942640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2b5070 0x557eeb2cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 -- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2b5070 msgr2=0x557eeb2cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2b5070 0x557eeb2cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 -- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 msgr2=0x557eeb2c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 -- 172.21.15.113:0/3517388011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eeaf1a3f0 con 0x557eeb2ea600 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2cc942640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2b5070 0x557eeb2cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2ea600 0x557eeb2c9110 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009250 tx=0x7fa2bc079630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3c9420ff5377f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/3517388011 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc07a070 con 0x557eeb2ea600 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/3517388011 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fa2bc082360 con 0x557eeb2ea600 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.265+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c000f9a0 con 0x557eeb2ea600 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/3517388011 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fa2bc098050 con 0x557eeb2ea600 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 10 client.11199 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x557eeaeda0f0 con 0x557eeb2ea600 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/3517388011 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c000f9a0 con 0x557eeb2ea600 2024-04-25T00:10:53.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/3517388011 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fa2bc03a030 con 0x557eeb2ea600 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 10 client.11199 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 10 client.11199 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 20 client.11199 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2b7af0 0x557eeb2b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 4 client.11199 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eeb213bf0 con 0x557eeb2b7af0 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2b7af0 0x557eeb2b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2b7af0 0x557eeb2b7ef0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c2f575f5f657eab server_cookie=40a10fb8199afc14 in_seq=0 out_seq=0 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.266+0000 7fa2a6ffd640 10 client.11199 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/3517388011 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eeb213bf0 con 0x557eeb2b7af0 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2a6ffd640 10 client.11199 handle_command_reply: tid=0 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 client.11199 shutdown 2024-04-25T00:10:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2ea600 msgr2=0x557eeb2c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2ea600 0x557eeb2c9110 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009250 tx=0x7fa2bc079630 comp rx=0 tx=0).stop 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 shutdown_connections 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2b7af0 0x557eeb2b7ef0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2b5070 0x557eeb2cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2ea600 0x557eeb2c9110 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2a6ffd640 0 client.11199 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3517388011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 >> 172.21.15.113:0/3517388011 conn(0x557eeb2a2a40 msgr2=0x557eeb2eb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 shutdown_connections 2024-04-25T00:10:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 1 -- 172.21.15.113:0/3517388011 wait complete. 2024-04-25T00:10:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.267+0000 7fa2d118f740 20 client.11199 trim_cache size 0 max 16384 2024-04-25T00:10:53.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.285+0000 7fa2d118f740 1 Processor -- start 2024-04-25T00:10:53.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.285+0000 7fa2d118f740 1 -- start start 2024-04-25T00:10:53.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb2ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2d118f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c82007c0 con 0x557eeb2b5070 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c81ff930 con 0x557eeb2ccd60 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c815c070 con 0x557eeb2ea600 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb2ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb2ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47970/0 (socket says 172.21.15.113:47970) 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 -- 172.21.15.113:0/3775550206 learned_addr learned my addr 172.21.15.113:0/3775550206 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2cc942640 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 -- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 msgr2=0x557eeb2c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 -- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 msgr2=0x557eeb2b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 -- 172.21.15.113:0/3775550206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eeb2cefa0 con 0x557eeb2b5070 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2cc942640 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb2ce690 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80011e0 tx=0x7fa2b8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cfe07a61f952d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2ad7fa640 1 -- 172.21.15.113:0/3775550206 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8014ab0 con 0x557eeb2b5070 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2ad7fa640 1 -- 172.21.15.113:0/3775550206 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b8014c50 con 0x557eeb2b5070 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.286+0000 7fa2ad7fa640 1 -- 172.21.15.113:0/3775550206 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8026120 con 0x557eeb2b5070 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 -- 172.21.15.113:0/3775550206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 msgr2=0x557eeb2ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb2ce690 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80011e0 tx=0x7fa2b8010000 comp rx=0 tx=0).stop 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 -- 172.21.15.113:0/3775550206 shutdown_connections 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb2c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 --2- 172.21.15.113:0/3775550206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb2ce690 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 -- 172.21.15.113:0/3775550206 >> 172.21.15.113:0/3775550206 conn(0x557eeb2a2a40 msgr2=0x557eeb2df640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 -- 172.21.15.113:0/3775550206 shutdown_connections 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 -- 172.21.15.113:0/3775550206 wait complete. 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.287+0000 7fa2d118f740 1 -- start start 2024-04-25T00:10:53.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 1 Processor -- start 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb361dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb301540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557eeb2cede0 con 0x557eeb2b5070 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c815f9f0 con 0x557eeb2ccd60 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557eeb297f00 con 0x557eeb2ea600 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2d118f740 10 client.0 _ll_register_callbacks cb 0x7fa2bc02a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb361dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb361dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47982/0 (socket says 172.21.15.113:47982) 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.288+0000 7fa2c7fff640 1 -- 172.21.15.113:0/1768047982 learned_addr learned my addr 172.21.15.113:0/1768047982 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2cc942640 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb301540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2c7fff640 1 -- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 msgr2=0x557eeb301540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb301540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2c7fff640 1 -- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 msgr2=0x557eeb2dd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2dd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2c7fff640 1 -- 172.21.15.113:0/1768047982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eeb2de3c0 con 0x557eeb2b5070 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2c7fff640 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb361dd0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8000f60 tx=0x7fa2b80010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15564712514f5a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/1768047982 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8014e00 con 0x557eeb2b5070 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/1768047982 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fa2b8031b30 con 0x557eeb2b5070 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2b8014c50 con 0x557eeb2b5070 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/1768047982 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fa2b8038050 con 0x557eeb2b5070 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 10 client.11205 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x557eeb362330 con 0x557eeb2b5070 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/1768047982 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b8014c50 con 0x557eeb2b5070 2024-04-25T00:10:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/1768047982 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fa2b803b020 con 0x557eeb2b5070 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 10 client.11205 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 10 client.11205 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 20 client.11205 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2df5b0 0x557eeb301a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 4 client.11205 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.289+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557eeb20df40 con 0x557eeb2df5b0 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2df5b0 0x557eeb301a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2a6ffd640 10 client.11205 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2c77fe640 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2df5b0 0x557eeb301a80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2327f003995dd2df server_cookie=ade2d5fad7140b1c in_seq=0 out_seq=0 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2a6ffd640 1 -- 172.21.15.113:0/1768047982 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557eeb20df40 con 0x557eeb2df5b0 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2a6ffd640 10 client.11205 handle_command_reply: tid=0 2024-04-25T00:10:53.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 client.11205 shutdown 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 msgr2=0x557eeb361dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb361dd0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8000f60 tx=0x7fa2b80010f0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 shutdown_connections 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x557eeb2df5b0 0x557eeb301a80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb2ccd60 0x557eeb2dd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb2ea600 0x557eeb301540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2a6ffd640 0 client.11205 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 --2- 172.21.15.113:0/1768047982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557eeb2b5070 0x557eeb361dd0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 >> 172.21.15.113:0/1768047982 conn(0x557eeb2a2a40 msgr2=0x557eeb2e9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 shutdown_connections 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 1 -- 172.21.15.113:0/1768047982 wait complete. 2024-04-25T00:10:53.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.290+0000 7fa2d118f740 20 client.11205 trim_cache size 0 max 16384 2024-04-25T00:10:53.300 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:10:53.300 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-25T00:10:53.300 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-25T00:10:53.300 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:10:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2b406be30 msgr2=0x7fa2b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2b406be30 0x7fa2b406e2f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8007fd0 tx=0x7fa2b8007920 comp rx=0 tx=0).stop 2024-04-25T00:10:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b73c0 msgr2=0x7fa2c8161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b73c0 0x7fa2c8161010 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 shutdown_connections 2024-04-25T00:10:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa2b406be30 0x7fa2b406e2f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2c815e510 0x7fa2c80ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2c80b73c0 0x7fa2c8161010 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 --2- 172.21.15.113:0/1465451104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c80b5ca0 0x7fa2c81f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 >> 172.21.15.113:0/1465451104 conn(0x7fa2c806b720 msgr2=0x7fa2c80b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:53.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 shutdown_connections 2024-04-25T00:10:53.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.291+0000 7fa2ce3a5640 1 -- 172.21.15.113:0/1465451104 wait complete. 2024-04-25T00:10:53.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:10:53.707 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:10:53.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.805+0000 7fc7c2e72640 1 Processor -- start 2024-04-25T00:10:53.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.805+0000 7fc7c2e72640 1 -- start start 2024-04-25T00:10:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c2e72640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7bc1453c0 0x7fc7bc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c2e72640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7bc0cfad0 con 0x7fc7bc144d60 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c2e72640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7bc0cf230 con 0x7fc7bc06e6e0 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c2e72640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc7bc145c40 con 0x7fc7bc1453c0 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c140f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7bc1453c0 0x7fc7bc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c140f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7bc1453c0 0x7fc7bc0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47986/0 (socket says 172.21.15.113:47986) 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c140f640 1 -- 172.21.15.113:0/2139881158 learned_addr learned my addr 172.21.15.113:0/2139881158 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c0c0e640 1 --1- 172.21.15.113:0/2139881158 >> v1:172.21.15.113:6789/0 conn(0x7fc7bc144d60 0x7fc7bc06dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34402/0 (socket says 172.21.15.113:34402) 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c140f640 1 -- 172.21.15.113:0/2139881158 >> v1:172.21.15.178:6789/0 conn(0x7fc7bc06e6e0 legacy=0x7fc7bc071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c140f640 1 -- 172.21.15.113:0/2139881158 >> v1:172.21.15.113:6789/0 conn(0x7fc7bc144d60 legacy=0x7fc7bc06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7c140f640 1 -- 172.21.15.113:0/2139881158 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc0cf510 con 0x7fc7bc1453c0 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.806+0000 7fc7bb7fe640 1 -- 172.21.15.113:0/2139881158 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1150613931 0 0) 0x7fc7bc0cfad0 con 0x7fc7bc144d60 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.807+0000 7fc7c140f640 1 --2- 172.21.15.113:0/2139881158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc0b5390 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009e50 tx=0x7fc7b402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5d301d6c9525f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.807+0000 7fc7bb7fe640 1 -- 172.21.15.113:0/2139881158 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4036070 con 0x7fc7bc1453c0 2024-04-25T00:10:53.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.807+0000 7fc7bb7fe640 1 -- 172.21.15.113:0/2139881158 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b4031600 con 0x7fc7bc1453c0 2024-04-25T00:10:53.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.807+0000 7fc7bb7fe640 1 -- 172.21.15.113:0/2139881158 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40319d0 con 0x7fc7bc1453c0 2024-04-25T00:10:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.808+0000 7fc7c2e72640 1 -- 172.21.15.113:0/2139881158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc1453c0 msgr2=0x7fc7bc0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.808+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/2139881158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc0b5390 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009e50 tx=0x7fc7b402a640 comp rx=0 tx=0).stop 2024-04-25T00:10:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.808+0000 7fc7c2e72640 1 -- 172.21.15.113:0/2139881158 shutdown_connections 2024-04-25T00:10:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.808+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/2139881158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc0b5390 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.808+0000 7fc7c2e72640 1 -- 172.21.15.113:0/2139881158 >> 172.21.15.113:0/2139881158 conn(0x7fc7bc0b1a50 msgr2=0x7fc7bc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.808+0000 7fc7c2e72640 1 -- 172.21.15.113:0/2139881158 shutdown_connections 2024-04-25T00:10:53.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.808+0000 7fc7c2e72640 1 -- 172.21.15.113:0/2139881158 wait complete. 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.809+0000 7fc7c2e72640 1 Processor -- start 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.809+0000 7fc7c2e72640 1 -- start start 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c2e72640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7bc06e6e0 0x7fc7bc157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c2e72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc144d60 0x7fc7bc143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c2e72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c2e72640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc145140 con 0x7fc7bc144d60 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c2e72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc070e10 con 0x7fc7bc1453c0 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c2e72640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc7bc071a40 con 0x7fc7bc06e6e0 2024-04-25T00:10:53.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc144d60 0x7fc7bc143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc144d60 0x7fc7bc143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47996/0 (socket says 172.21.15.113:47996) 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 -- 172.21.15.113:0/3399154886 learned_addr learned my addr 172.21.15.113:0/3399154886 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c140f640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c0c0e640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7bc06e6e0 0x7fc7bc157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 -- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7bc06e6e0 msgr2=0x7fc7bc157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7bc06e6e0 0x7fc7bc157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 -- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7bc1453c0 msgr2=0x7fc7bc06d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc06d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:53.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 -- 172.21.15.113:0/3399154886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc1562a0 con 0x7fc7bc144d60 2024-04-25T00:10:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7c140f640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc06d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.810+0000 7fc7bbfff640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc144d60 0x7fc7bc143ef0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c400 tx=0x7fc7b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.811+0000 7fc7b97fa640 1 -- 172.21.15.113:0/3399154886 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0011790 con 0x7fc7bc144d60 2024-04-25T00:10:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.811+0000 7fc7b97fa640 1 -- 172.21.15.113:0/3399154886 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0017070 con 0x7fc7bc144d60 2024-04-25T00:10:53.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.811+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc156580 con 0x7fc7bc144d60 2024-04-25T00:10:53.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.811+0000 7fc7b97fa640 1 -- 172.21.15.113:0/3399154886 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0011bb0 con 0x7fc7bc144d60 2024-04-25T00:10:53.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.811+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc156b40 con 0x7fc7bc144d60 2024-04-25T00:10:53.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.812+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc0691a0 con 0x7fc7bc144d60 2024-04-25T00:10:53.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.812+0000 7fc7b97fa640 1 -- 172.21.15.113:0/3399154886 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7b001c460 con 0x7fc7bc144d60 2024-04-25T00:10:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.812+0000 7fc7b97fa640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7a806be30 0x7fc7a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.813+0000 7fc7b97fa640 1 -- 172.21.15.113:0/3399154886 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fc7b008ac70 con 0x7fc7bc144d60 2024-04-25T00:10:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.813+0000 7fc7c0c0e640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7a806be30 0x7fc7a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.813+0000 7fc7c0c0e640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7a806be30 0x7fc7a806e2f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac000f60 tx=0x7fc7ac007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:53.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:53.817+0000 7fc7b97fa640 1 -- 172.21.15.113:0/3399154886 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc7b0059650 con 0x7fc7bc144d60 2024-04-25T00:10:54.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.029+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7bc140fe0 con 0x7fc7bc144d60 2024-04-25T00:10:54.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.029+0000 7fc7b97fa640 1 -- 172.21.15.113:0/3399154886 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7fc7b0058f50 con 0x7fc7bc144d60 2024-04-25T00:10:54.030 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:54.030 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:50.995654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10794,"qdb_cluster":[10794]},"id":13}]} 2024-04-25T00:10:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7a806be30 msgr2=0x7fc7a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7a806be30 0x7fc7a806e2f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac000f60 tx=0x7fc7ac007b50 comp rx=0 tx=0).stop 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc144d60 msgr2=0x7fc7bc143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc144d60 0x7fc7bc143ef0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c400 tx=0x7fc7b000c8c0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 shutdown_connections 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc7a806be30 0x7fc7a806e2f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7bc1453c0 0x7fc7bc06d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7bc144d60 0x7fc7bc143ef0 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 --2- 172.21.15.113:0/3399154886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7bc06e6e0 0x7fc7bc157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 >> 172.21.15.113:0/3399154886 conn(0x7fc7bc0b1a50 msgr2=0x7fc7bc071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 shutdown_connections 2024-04-25T00:10:54.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.031+0000 7fc7c2e72640 1 -- 172.21.15.113:0/3399154886 wait complete. 2024-04-25T00:10:54.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:09:33.215001+0000', 'modified': '2024-04-25T00:09:50.995654+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 183, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10794': {'gid': 10794, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.113:6836/2086115851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 2086115851}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 2086115851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10794, 'qdb_cluster': [10794]} 2024-04-25T00:10:54.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:10:54.043 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:10:54.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.140+0000 7f5a735dc640 1 Processor -- start 2024-04-25T00:10:54.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.140+0000 7f5a735dc640 1 -- start start 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a735dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a6c0b1e00 0x7f5a6c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a735dc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6c06d900 con 0x7f5a6c071df0 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a735dc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6c06dbe0 con 0x7f5a6c15e310 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a735dc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6c06e9a0 con 0x7f5a6c0b1e00 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a71b79640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a6c0b1e00 0x7f5a6c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a70b77640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5a6c071df0 0x7f5a6c15e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34412/0 (socket says 172.21.15.113:34412) 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a70b77640 1 -- 172.21.15.113:0/29585974 learned_addr learned my addr 172.21.15.113:0/29585974 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a71b79640 1 -- 172.21.15.113:0/29585974 >> v1:172.21.15.178:6789/0 conn(0x7f5a6c15e310 legacy=0x7f5a6c0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a71b79640 1 -- 172.21.15.113:0/29585974 >> v1:172.21.15.113:6789/0 conn(0x7f5a6c071df0 legacy=0x7f5a6c15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a63fff640 1 -- 172.21.15.113:0/29585974 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 163079587 0 0) 0x7f5a6c06d900 con 0x7f5a6c071df0 2024-04-25T00:10:54.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.141+0000 7f5a71b79640 1 -- 172.21.15.113:0/29585974 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c06dec0 con 0x7f5a6c0b1e00 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.142+0000 7f5a71b79640 1 --2- 172.21.15.113:0/29585974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c0d00e0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009d90 tx=0x7f5a6402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a033aca7c04c506f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.142+0000 7f5a63fff640 1 -- 172.21.15.113:0/29585974 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64031600 con 0x7f5a6c0b1e00 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.142+0000 7f5a63fff640 1 -- 172.21.15.113:0/29585974 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a640317a0 con 0x7f5a6c0b1e00 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.142+0000 7f5a63fff640 1 -- 172.21.15.113:0/29585974 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64031b70 con 0x7f5a6c0b1e00 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.143+0000 7f5a735dc640 1 -- 172.21.15.113:0/29585974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0b1e00 msgr2=0x7f5a6c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.143+0000 7f5a735dc640 1 --2- 172.21.15.113:0/29585974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c0d00e0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009d90 tx=0x7f5a6402a5d0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.143+0000 7f5a735dc640 1 -- 172.21.15.113:0/29585974 shutdown_connections 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.143+0000 7f5a735dc640 1 --2- 172.21.15.113:0/29585974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c0d00e0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.143+0000 7f5a735dc640 1 -- 172.21.15.113:0/29585974 >> 172.21.15.113:0/29585974 conn(0x7f5a6c06fb60 msgr2=0x7f5a6c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.143+0000 7f5a735dc640 1 -- 172.21.15.113:0/29585974 shutdown_connections 2024-04-25T00:10:54.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.143+0000 7f5a735dc640 1 -- 172.21.15.113:0/29585974 wait complete. 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.144+0000 7f5a735dc640 1 Processor -- start 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.144+0000 7f5a735dc640 1 -- start start 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a735dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c071df0 0x7f5a6c148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a735dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a735dc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a6c15e310 0x7f5a6c147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a735dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c0ceb20 con 0x7f5a6c071df0 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a735dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c06e510 con 0x7f5a6c0b1e00 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a735dc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c0ceca0 con 0x7f5a6c15e310 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c071df0 0x7f5a6c148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c071df0 0x7f5a6c148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48016/0 (socket says 172.21.15.113:48016) 2024-04-25T00:10:54.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 -- 172.21.15.113:0/1323442591 learned_addr learned my addr 172.21.15.113:0/1323442591 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a70b77640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71b79640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a6c15e310 0x7f5a6c147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 -- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a6c15e310 msgr2=0x7f5a6c147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a6c15e310 0x7f5a6c147a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 -- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a6c0b1e00 msgr2=0x7f5a6c146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 -- 172.21.15.113:0/1323442591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c146750 con 0x7f5a6c071df0 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a70b77640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a71378640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c071df0 0x7f5a6c148e80 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c400 tx=0x7f5a6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a61ffb640 1 -- 172.21.15.113:0/1323442591 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68017070 con 0x7f5a6c071df0 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a61ffb640 1 -- 172.21.15.113:0/1323442591 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a680052d0 con 0x7f5a6c071df0 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c1469d0 con 0x7f5a6c071df0 2024-04-25T00:10:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.145+0000 7f5a61ffb640 1 -- 172.21.15.113:0/1323442591 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68003830 con 0x7f5a6c071df0 2024-04-25T00:10:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.146+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c147270 con 0x7f5a6c071df0 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.147+0000 7f5a61ffb640 1 -- 172.21.15.113:0/1323442591 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a6801a070 con 0x7f5a6c071df0 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.147+0000 7f5a61ffb640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a5006be30 0x7f5a5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.148+0000 7f5a61ffb640 1 -- 172.21.15.113:0/1323442591 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f5a6808a200 con 0x7f5a6c071df0 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.148+0000 7f5a70b77640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a5006be30 0x7f5a5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.148+0000 7f5a70b77640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a5006be30 0x7f5a5006e2f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0ce150 tx=0x7f5a5c007c50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.148+0000 7f5a763c8740 1 Processor -- start 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a763c8740 1 -- start start 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120b914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120bc1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1540 0x564120b96350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a763c8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564120af9f00 con 0x564120bdeb60 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564120aa2750 con 0x564120bc1c80 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564120af5ec0 con 0x564120bc1540 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a71378640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120b914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a71378640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120b914f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48022/0 (socket says 172.21.15.113:48022) 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.149+0000 7f5a71378640 1 -- 172.21.15.113:0/1695805742 learned_addr learned my addr 172.21.15.113:0/1695805742 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a71378640 1 -- 172.21.15.113:0/1695805742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1540 msgr2=0x564120b96350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a71378640 1 --2- 172.21.15.113:0/1695805742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1540 0x564120b96350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a71378640 1 -- 172.21.15.113:0/1695805742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 msgr2=0x564120bc1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a71378640 1 --2- 172.21.15.113:0/1695805742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120bc1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a71378640 1 -- 172.21.15.113:0/1695805742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412080e2d0 con 0x564120bdeb60 2024-04-25T00:10:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a71378640 1 --2- 172.21.15.113:0/1695805742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120b914f0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f5a68000fb0 tx=0x7f5a68058b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb4f6c0c699033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a56ffd640 1 -- 172.21.15.113:0/1695805742 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68061030 con 0x564120bdeb60 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a56ffd640 1 -- 172.21.15.113:0/1695805742 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a68056750 con 0x564120bdeb60 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a56ffd640 1 -- 172.21.15.113:0/1695805742 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a680578a0 con 0x564120bdeb60 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 -- 172.21.15.113:0/1695805742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 msgr2=0x564120b914f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1695805742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120b914f0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f5a68000fb0 tx=0x7f5a68058b50 comp rx=0 tx=0).stop 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 -- 172.21.15.113:0/1695805742 shutdown_connections 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1695805742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120bc1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1695805742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1540 0x564120b96350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1695805742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120b914f0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 -- 172.21.15.113:0/1695805742 >> 172.21.15.113:0/1695805742 conn(0x564120b93760 msgr2=0x564120bdc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 -- 172.21.15.113:0/1695805742 shutdown_connections 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.150+0000 7f5a763c8740 1 -- 172.21.15.113:0/1695805742 wait complete. 2024-04-25T00:10:54.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.151+0000 7f5a763c8740 1 -- start start 2024-04-25T00:10:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.151+0000 7f5a763c8740 1 Processor -- start 2024-04-25T00:10:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120bd4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120bc4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bd5070 0x564120bc4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a763c8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564120b95910 con 0x564120bdeb60 2024-04-25T00:10:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564120ab5cf0 con 0x564120bc1c80 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564120b94d00 con 0x564120bd5070 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a763c8740 10 client.0 _ll_register_callbacks cb 0x564120ab56e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120bc4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120bc4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48038/0 (socket says 172.21.15.113:48038) 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 -- 172.21.15.113:0/776369508 learned_addr learned my addr 172.21.15.113:0/776369508 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 -- 172.21.15.113:0/776369508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bd5070 msgr2=0x564120bc4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bd5070 0x564120bc4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 -- 172.21.15.113:0/776369508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 msgr2=0x564120bd4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a71378640 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120bd4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120bd4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 -- 172.21.15.113:0/776369508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641208d86b0 con 0x564120bdeb60 2024-04-25T00:10:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a70b77640 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120bc4630 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c001c80 tx=0x7f5a5c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd7afb34b42b722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.152+0000 7f5a4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a4bfff640 1 -- 172.21.15.113:0/776369508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c015110 con 0x564120bdeb60 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a4bfff640 1 -- 172.21.15.113:0/776369508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f5a5c023b30 con 0x564120bdeb60 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a68056750 con 0x564120bdeb60 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a4bfff640 1 -- 172.21.15.113:0/776369508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f5a5c023d30 con 0x564120bdeb60 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 10 client.11229 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x564120bd3740 con 0x564120bdeb60 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a4bfff640 1 -- 172.21.15.113:0/776369508 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a68056750 con 0x564120bdeb60 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a4bfff640 1 -- 172.21.15.113:0/776369508 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f5a5c02a050 con 0x564120bdeb60 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 10 client.11229 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 10 client.11229 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:10:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 20 client.11229 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120bc2430 0x564120bf0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 4 client.11229 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564120b07440 con 0x564120bc2430 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a71378640 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120bc2430 0x564120bf0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a4bfff640 10 client.11229 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.153+0000 7f5a71378640 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120bc2430 0x564120bf0210 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff90b466ec42164 server_cookie=29bb5ec371a81d1e in_seq=0 out_seq=0 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a4bfff640 1 -- 172.21.15.113:0/776369508 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564120b07440 con 0x564120bc2430 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a4bfff640 10 client.11229 handle_command_reply: tid=0 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 client.11229 shutdown 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 msgr2=0x564120bc4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120bc4630 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c001c80 tx=0x7f5a5c000ed0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 shutdown_connections 2024-04-25T00:10:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120bc2430 0x564120bf0210 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bd5070 0x564120bc4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a4bfff640 0 client.11229 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bdeb60 0x564120bc4630 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 --2- 172.21.15.113:0/776369508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120bd4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 >> 172.21.15.113:0/776369508 conn(0x564120bc1540 msgr2=0x564120bde120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 shutdown_connections 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 1 -- 172.21.15.113:0/776369508 wait complete. 2024-04-25T00:10:54.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.154+0000 7f5a763c8740 20 client.11229 trim_cache size 0 max 16384 2024-04-25T00:10:54.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.172+0000 7f5a763c8740 1 Processor -- start 2024-04-25T00:10:54.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.172+0000 7f5a763c8740 1 -- start start 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1c80 0x564120b91230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bdab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bdeb60 0x564120bdb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a763c8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c15bb70 con 0x564120bd5070 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c15aed0 con 0x564120bdeb60 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c15b520 con 0x564120bc1c80 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bdab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bdab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48040/0 (socket says 172.21.15.113:48040) 2024-04-25T00:10:54.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 -- 172.21.15.113:0/1553766238 learned_addr learned my addr 172.21.15.113:0/1553766238 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 -- 172.21.15.113:0/1553766238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1c80 msgr2=0x564120b91230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 --2- 172.21.15.113:0/1553766238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1c80 0x564120b91230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 -- 172.21.15.113:0/1553766238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bdeb60 msgr2=0x564120bdb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 --2- 172.21.15.113:0/1553766238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bdeb60 0x564120bdb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 -- 172.21.15.113:0/1553766238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564120be0470 con 0x564120bd5070 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.173+0000 7f5a70b77640 1 --2- 172.21.15.113:0/1553766238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bdab60 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c001700 tx=0x7f5a5c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4393526debc52810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a567fc640 1 -- 172.21.15.113:0/1553766238 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c015110 con 0x564120bd5070 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a567fc640 1 -- 172.21.15.113:0/1553766238 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c02c470 con 0x564120bd5070 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a567fc640 1 -- 172.21.15.113:0/1553766238 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c02c820 con 0x564120bd5070 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 -- 172.21.15.113:0/1553766238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 msgr2=0x564120bdab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1553766238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bdab60 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c001700 tx=0x7f5a5c01d000 comp rx=0 tx=0).stop 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 -- 172.21.15.113:0/1553766238 shutdown_connections 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1553766238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bdeb60 0x564120bdb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1553766238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bdab60 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 --2- 172.21.15.113:0/1553766238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bc1c80 0x564120b91230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 -- 172.21.15.113:0/1553766238 >> 172.21.15.113:0/1553766238 conn(0x564120bc1540 msgr2=0x564120bebe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 -- 172.21.15.113:0/1553766238 shutdown_connections 2024-04-25T00:10:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.174+0000 7f5a763c8740 1 -- 172.21.15.113:0/1553766238 wait complete. 2024-04-25T00:10:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.175+0000 7f5a763c8740 1 -- start start 2024-04-25T00:10:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.175+0000 7f5a763c8740 1 Processor -- start 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120c56790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bd1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bdeb60 0x564120bd5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564120bc22b0 con 0x564120bd5070 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564120bc2f50 con 0x564120bc1c80 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564120bc2840 con 0x564120bdeb60 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 10 client.0 _ll_register_callbacks cb 0x7f5a5c001900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bd1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bd1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48052/0 (socket says 172.21.15.113:48052) 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 -- 172.21.15.113:0/2842627248 learned_addr learned my addr 172.21.15.113:0/2842627248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a71b79640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bdeb60 0x564120bd5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a71378640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120c56790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 -- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bdeb60 msgr2=0x564120bd5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bdeb60 0x564120bd5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 -- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 msgr2=0x564120c56790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120c56790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 -- 172.21.15.113:0/2842627248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564120bc4690 con 0x564120bd5070 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a71b79640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bdeb60 0x564120bd5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a71378640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120c56790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:54.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a70b77640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bd1990 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c014000 tx=0x7f5a5c009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb43285b5f4b213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a4bfff640 1 -- 172.21.15.113:0/2842627248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c03a020 con 0x564120bd5070 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a4bfff640 1 -- 172.21.15.113:0/2842627248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f5a5c02ca30 con 0x564120bd5070 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a5c02c470 con 0x564120bd5070 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a4bfff640 1 -- 172.21.15.113:0/2842627248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f5a5c02cc30 con 0x564120bd5070 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 10 client.11235 fetch_fsmap learned FSMap version 104 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.176+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x564120c570a0 con 0x564120bd5070 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a4bfff640 1 -- 172.21.15.113:0/2842627248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c02c470 con 0x564120bd5070 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a4bfff640 1 -- 172.21.15.113:0/2842627248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f5a5c02a070 con 0x564120bd5070 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 10 client.11235 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 10 client.11235 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 20 client.11235 populate_metadata read hostname 'smithi113' 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120c5aa00 0x564120c5ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 4 client.11235 mds_command: new command op to 10794 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 --> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] -- command(tid 0: {"prefix": "status"}) -- 0x564120b01790 con 0x564120c5aa00 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a71378640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120c5aa00 0x564120c5ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a4bfff640 10 client.11235 ms_handle_connect on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a71378640 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120c5aa00 0x564120c5ade0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4afecd90110ec61 server_cookie=117f43dfae8550b4 in_seq=0 out_seq=0 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a4bfff640 1 -- 172.21.15.113:0/2842627248 <== mds.0 v2:172.21.15.113:6835/2086115851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x564120b01790 con 0x564120c5aa00 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a4bfff640 10 client.11235 handle_command_reply: tid=0 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 client.11235 shutdown 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 msgr2=0x564120bd1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bd1990 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c014000 tx=0x7f5a5c009030 comp rx=0 tx=0).stop 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 shutdown_connections 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:6835/2086115851,v1:172.21.15.113:6836/2086115851] conn(0x564120c5aa00 0x564120c5ade0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564120bdeb60 0x564120bd5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564120bd5070 0x564120bd1990 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a4bfff640 0 client.11235 ms_handle_reset on v2:172.21.15.113:6835/2086115851 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 --2- 172.21.15.113:0/2842627248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564120bc1c80 0x564120c56790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.177+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 >> 172.21.15.113:0/2842627248 conn(0x564120bc1540 msgr2=0x564120b96e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 shutdown_connections 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a763c8740 1 -- 172.21.15.113:0/2842627248 wait complete. 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a763c8740 20 client.11235 trim_cache size 0 max 16384 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "c4eaaf81-6120-4e3f-a50c-97aa81453142", 2024-04-25T00:10:54.185 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 10794, 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 66.189852060000007, 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 104, 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 183, 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 183, 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 85.398622771999996 2024-04-25T00:10:54.186 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:10:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a5006be30 msgr2=0x7f5a5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a5006be30 0x7f5a5006e2f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0ce150 tx=0x7f5a5c007c50 comp rx=0 tx=0).stop 2024-04-25T00:10:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c071df0 msgr2=0x7f5a6c148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c071df0 0x7f5a6c148e80 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c400 tx=0x7f5a6800c8c0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 shutdown_connections 2024-04-25T00:10:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5a5006be30 0x7f5a5006e2f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a6c0b1e00 0x7f5a6c146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a6c15e310 0x7f5a6c147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 --2- 172.21.15.113:0/1323442591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c071df0 0x7f5a6c148e80 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 >> 172.21.15.113:0/1323442591 conn(0x7f5a6c06fb60 msgr2=0x7f5a6c0b50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 shutdown_connections 2024-04-25T00:10:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.178+0000 7f5a735dc640 1 -- 172.21.15.113:0/1323442591 wait complete. 2024-04-25T00:10:54.189 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:10:54.189 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:10:54.235 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:10:54.284 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:10:54.284 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:10:54.285 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:10:54.285 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:10:54.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:10:54.300 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:10:54.300 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:10:54.300 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:10:54.318 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:10:54.318 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:10:54.319 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:10:54.319 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:10:54.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:10:54.355 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:10:54.355 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:10:54.355 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:10:54.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:10:54.375 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:10:54.474 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:54.477 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:10:54.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.493+0000 7ff4d8da5640 1 Processor -- start 2024-04-25T00:10:54.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.493+0000 7ff4d8da5640 1 -- start start 2024-04-25T00:10:54.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4d8da5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4d40ce370 0x7ff4d40ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4d8da5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d40c46d0 con 0x7ff4d415e310 2024-04-25T00:10:54.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4d8da5640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d406b190 con 0x7ff4d40ce9a0 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4d8da5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d415e6f0 con 0x7ff4d40ce370 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4cbfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4d40ce370 0x7ff4d40ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4d259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff4d415e310 0x7ff4d406bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34414/0 (socket says 172.21.15.113:34414) 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4d259c640 1 -- 172.21.15.113:0/3134186638 learned_addr learned my addr 172.21.15.113:0/3134186638 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4cbfff640 1 -- 172.21.15.113:0/3134186638 >> v1:172.21.15.178:6789/0 conn(0x7ff4d40ce9a0 legacy=0x7ff4d4061b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4cbfff640 1 -- 172.21.15.113:0/3134186638 >> v1:172.21.15.113:6789/0 conn(0x7ff4d415e310 legacy=0x7ff4d406bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.494+0000 7ff4cbfff640 1 -- 172.21.15.113:0/3134186638 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d406b470 con 0x7ff4d40ce370 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.495+0000 7ff4cbfff640 1 --2- 172.21.15.113:0/3134186638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d40ce370 0x7ff4d40ccf70 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0097a0 tx=0x7ff4bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d579729f5ac62c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.495+0000 7ff4d1d9b640 1 -- 172.21.15.113:0/3134186638 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc037070 con 0x7ff4d40ce370 2024-04-25T00:10:54.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.495+0000 7ff4d1d9b640 1 -- 172.21.15.113:0/3134186638 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4bc032440 con 0x7ff4d40ce370 2024-04-25T00:10:54.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.495+0000 7ff4d1d9b640 1 -- 172.21.15.113:0/3134186638 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc032810 con 0x7ff4d40ce370 2024-04-25T00:10:54.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.495+0000 7ff4d8da5640 1 -- 172.21.15.113:0/3134186638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d40ce370 msgr2=0x7ff4d40ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.496+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/3134186638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d40ce370 0x7ff4d40ccf70 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0097a0 tx=0x7ff4bc02a230 comp rx=0 tx=0).stop 2024-04-25T00:10:54.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.496+0000 7ff4d8da5640 1 -- 172.21.15.113:0/3134186638 shutdown_connections 2024-04-25T00:10:54.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.496+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/3134186638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d40ce370 0x7ff4d40ccf70 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.496+0000 7ff4d8da5640 1 -- 172.21.15.113:0/3134186638 >> 172.21.15.113:0/3134186638 conn(0x7ff4d40cd850 msgr2=0x7ff4d40cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.496+0000 7ff4d8da5640 1 -- 172.21.15.113:0/3134186638 shutdown_connections 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.496+0000 7ff4d8da5640 1 -- 172.21.15.113:0/3134186638 wait complete. 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.497+0000 7ff4d8da5640 1 Processor -- start 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.497+0000 7ff4d8da5640 1 -- start start 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d8da5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4d40ce370 0x7ff4d4158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d8da5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4d40ce9a0 0x7ff4d40c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d8da5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d415e310 0x7ff4d40c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d8da5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40cd600 con 0x7ff4d415e310 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d8da5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d406bac0 con 0x7ff4d40ce370 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d8da5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d406c660 con 0x7ff4d40ce9a0 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d415e310 0x7ff4d40c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d415e310 0x7ff4d40c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48062/0 (socket says 172.21.15.113:48062) 2024-04-25T00:10:54.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 -- 172.21.15.113:0/817150585 learned_addr learned my addr 172.21.15.113:0/817150585 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d259c640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4d40ce370 0x7ff4d4158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4cbfff640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4d40ce9a0 0x7ff4d40c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 -- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4d40ce9a0 msgr2=0x7ff4d40c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4d40ce9a0 0x7ff4d40c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 -- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4d40ce370 msgr2=0x7ff4d4158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4d40ce370 0x7ff4d4158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 -- 172.21.15.113:0/817150585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d415f650 con 0x7ff4d415e310 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4cbfff640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4d40ce9a0 0x7ff4d40c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d2d9d640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d415e310 0x7ff4d40c6a50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c7a0 tx=0x7ff4cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4caffd640 1 -- 172.21.15.113:0/817150585 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc003b70 con 0x7ff4d415e310 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4caffd640 1 -- 172.21.15.113:0/817150585 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc003d30 con 0x7ff4d415e310 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d415f930 con 0x7ff4d415e310 2024-04-25T00:10:54.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.498+0000 7ff4caffd640 1 -- 172.21.15.113:0/817150585 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc00aa90 con 0x7ff4d415e310 2024-04-25T00:10:54.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.499+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d415fef0 con 0x7ff4d415e310 2024-04-25T00:10:54.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.500+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d4068220 con 0x7ff4d415e310 2024-04-25T00:10:54.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.500+0000 7ff4caffd640 1 -- 172.21.15.113:0/817150585 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4cc00ac30 con 0x7ff4d415e310 2024-04-25T00:10:54.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.500+0000 7ff4caffd640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff4b406be60 0x7ff4b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.500+0000 7ff4d259c640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff4b406be60 0x7ff4b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.500+0000 7ff4caffd640 1 -- 172.21.15.113:0/817150585 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7ff4cc08ad40 con 0x7ff4d415e310 2024-04-25T00:10:54.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.501+0000 7ff4d259c640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff4b406be60 0x7ff4b406e320 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005880 tx=0x7ff4c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.505+0000 7ff4caffd640 1 -- 172.21.15.113:0/817150585 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4cc059690 con 0x7ff4d415e310 2024-04-25T00:10:54.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.718+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4d40628f0 con 0x7ff4d415e310 2024-04-25T00:10:54.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.719+0000 7ff4caffd640 1 -- 172.21.15.113:0/817150585 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7ff4cc058f90 con 0x7ff4d415e310 2024-04-25T00:10:54.720 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:54.720 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:50.995654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10794,"qdb_cluster":[10794]},"id":13}]} 2024-04-25T00:10:54.720 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff4b406be60 msgr2=0x7ff4b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff4b406be60 0x7ff4b406e320 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005880 tx=0x7ff4c4009040 comp rx=0 tx=0).stop 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d415e310 msgr2=0x7ff4d40c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d415e310 0x7ff4d40c6a50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c7a0 tx=0x7ff4cc00cc60 comp rx=0 tx=0).stop 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 shutdown_connections 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff4b406be60 0x7ff4b406e320 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4d415e310 0x7ff4d40c6a50 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4d40ce9a0 0x7ff4d40c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 --2- 172.21.15.113:0/817150585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4d40ce370 0x7ff4d4158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 >> 172.21.15.113:0/817150585 conn(0x7ff4d40cd850 msgr2=0x7ff4d40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 shutdown_connections 2024-04-25T00:10:54.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.721+0000 7ff4d8da5640 1 -- 172.21.15.113:0/817150585 wait complete. 2024-04-25T00:10:54.733 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:54.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.828+0000 7f7a8bbc8640 1 Processor -- start 2024-04-25T00:10:54.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.828+0000 7f7a8bbc8640 1 -- start start 2024-04-25T00:10:54.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.829+0000 7f7a8bbc8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a840b3fd0 0x7f7a8406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.829+0000 7f7a8bbc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.829+0000 7f7a8bbc8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a84144cf0 con 0x7f7a84144910 2024-04-25T00:10:54.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.829+0000 7f7a8bbc8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7a840b2f30 con 0x7f7a840b3fd0 2024-04-25T00:10:54.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.829+0000 7f7a8bbc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a840b3210 con 0x7f7a8406e510 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a8a165640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a8a165640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:50026/0 (socket says 172.21.15.113:50026) 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a8a165640 1 -- 172.21.15.113:0/3582316244 learned_addr learned my addr 172.21.15.113:0/3582316244 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a89163640 1 --2- 172.21.15.113:0/3582316244 >> v2:172.21.15.178:3300/0 conn(0x7f7a840b3fd0 0x7f7a8406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a8a165640 1 -- 172.21.15.113:0/3582316244 >> v2:172.21.15.178:3300/0 conn(0x7f7a840b3fd0 msgr2=0x7f7a8406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a8a165640 1 --2- 172.21.15.113:0/3582316244 >> v2:172.21.15.178:3300/0 conn(0x7f7a840b3fd0 0x7f7a8406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a8a165640 1 -- 172.21.15.113:0/3582316244 >> v1:172.21.15.178:6789/0 conn(0x7f7a84144910 legacy=0x7f7a840b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a8a165640 1 -- 172.21.15.113:0/3582316244 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a840b34f0 con 0x7f7a8406e510 2024-04-25T00:10:54.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a88962640 1 -- 172.21.15.113:0/3582316244 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325852950 0 0) 0x7f7a84144cf0 con 0x7f7a84144910 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.830+0000 7f7a89163640 1 --2- 172.21.15.113:0/3582316244 >> v2:172.21.15.178:3300/0 conn(0x7f7a840b3fd0 0x7f7a8406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a8a165640 1 --2- 172.21.15.113:0/3582316244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840cd500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098a0 tx=0x7f7a7c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae88cd9d74c5c83f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a88962640 1 -- 172.21.15.113:0/3582316244 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c037070 con 0x7f7a8406e510 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a88962640 1 -- 172.21.15.113:0/3582316244 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c032440 con 0x7f7a8406e510 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a88962640 1 -- 172.21.15.113:0/3582316244 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c032810 con 0x7f7a8406e510 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3582316244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 msgr2=0x7f7a840cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3582316244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840cd500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098a0 tx=0x7f7a7c02a290 comp rx=0 tx=0).stop 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3582316244 shutdown_connections 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3582316244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840cd500 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3582316244 >> v2:172.21.15.178:3300/0 conn(0x7f7a840b3fd0 0x7f7a8406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.831+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3582316244 >> 172.21.15.113:0/3582316244 conn(0x7f7a840b1a50 msgr2=0x7f7a840b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.832+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3582316244 shutdown_connections 2024-04-25T00:10:54.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.832+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3582316244 wait complete. 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 Processor -- start 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 -- start start 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a840b3fd0 0x7f7a84133b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a84144910 0x7f7a840ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a840b4da0 con 0x7f7a84144910 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a840b4440 con 0x7f7a840b3fd0 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8bbc8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a840b4f20 con 0x7f7a8406e510 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.833+0000 7f7a8a165640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a84144910 0x7f7a840ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a84144910 0x7f7a840ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48070/0 (socket says 172.21.15.113:48070) 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 -- 172.21.15.113:0/3141519493 learned_addr learned my addr 172.21.15.113:0/3141519493 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a89964640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a89163640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a840b3fd0 0x7f7a84133b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 -- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 msgr2=0x7f7a840c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 -- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a840b3fd0 msgr2=0x7f7a84133b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a840b3fd0 0x7f7a84133b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 -- 172.21.15.113:0/3141519493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a840ca480 con 0x7f7a84144910 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a89964640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a89163640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a840b3fd0 0x7f7a84133b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8a165640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a84144910 0x7f7a840ae6f0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c035040 tx=0x7f7a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a7a7fc640 1 -- 172.21.15.113:0/3141519493 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c037040 con 0x7f7a84144910 2024-04-25T00:10:54.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a7a7fc640 1 -- 172.21.15.113:0/3141519493 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c004230 con 0x7f7a84144910 2024-04-25T00:10:54.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a841368a0 con 0x7f7a84144910 2024-04-25T00:10:54.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a7a7fc640 1 -- 172.21.15.113:0/3141519493 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c003610 con 0x7f7a84144910 2024-04-25T00:10:54.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.834+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a840caa00 con 0x7f7a84144910 2024-04-25T00:10:54.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.835+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a840b6070 con 0x7f7a84144910 2024-04-25T00:10:54.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.836+0000 7f7a7a7fc640 1 -- 172.21.15.113:0/3141519493 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a7c0037b0 con 0x7f7a84144910 2024-04-25T00:10:54.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.836+0000 7f7a7a7fc640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a6806be30 0x7f7a6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:54.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.836+0000 7f7a7a7fc640 1 -- 172.21.15.113:0/3141519493 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f7a7c0aba80 con 0x7f7a84144910 2024-04-25T00:10:54.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.836+0000 7f7a89964640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a6806be30 0x7f7a6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:54.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.837+0000 7f7a89964640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a6806be30 0x7f7a6806e2f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7a80007fd0 tx=0x7f7a80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:54.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:54.841+0000 7f7a7a7fc640 1 -- 172.21.15.113:0/3141519493 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a840b6070 con 0x7f7a84144910 2024-04-25T00:10:55.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.014+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a84143f50 con 0x7f7a84144910 2024-04-25T00:10:55.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.015+0000 7f7a7a7fc640 1 -- 172.21.15.113:0/3141519493 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) ==== 74+0+16862 (secure 0 0 0) 0x7f7a7c07a4f0 con 0x7f7a84144910 2024-04-25T00:10:55.016 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:55.016 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":183,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:46.979993+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:31.412384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:32.173467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/3057967443":"2024-04-26T00:09:46.979970+0000","172.21.15.178:6833/3057967443":"2024-04-26T00:09:46.979970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:10:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a6806be30 msgr2=0x7f7a6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a6806be30 0x7f7a6806e2f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7a80007fd0 tx=0x7f7a80009040 comp rx=0 tx=0).stop 2024-04-25T00:10:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a84144910 msgr2=0x7f7a840ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a84144910 0x7f7a840ae6f0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c035040 tx=0x7f7a7c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 shutdown_connections 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7a6806be30 0x7f7a6806e2f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a840b3fd0 0x7f7a84133b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a84144910 0x7f7a840ae6f0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.017+0000 7f7a8bbc8640 1 --2- 172.21.15.113:0/3141519493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a8406e510 0x7f7a840c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.018+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 >> 172.21.15.113:0/3141519493 conn(0x7f7a840b1a50 msgr2=0x7f7a840b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.018+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 shutdown_connections 2024-04-25T00:10:55.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.018+0000 7f7a8bbc8640 1 -- 172.21.15.113:0/3141519493 wait complete. 2024-04-25T00:10:55.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:58:07.141026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:09:31.412384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:09:32.173467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:10:55.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:10:55.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.128+0000 7f2c6ef9e640 1 Processor -- start 2024-04-25T00:10:55.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6ef9e640 1 -- start start 2024-04-25T00:10:55.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6ef9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c6815e310 0x7f2c680b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6ef9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c680b3030 0x7f2c6806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6ef9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c680b1f80 con 0x7f2c6806be40 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6ef9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2c6806c2b0 con 0x7f2c680b3030 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6ef9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c680b2b90 con 0x7f2c6815e310 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6d79b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c680b3030 0x7f2c6806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6e79d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2c6806be40 0x7f2c680ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34430/0 (socket says 172.21.15.113:34430) 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.129+0000 7f2c6e79d640 1 -- 172.21.15.113:0/2867520124 learned_addr learned my addr 172.21.15.113:0/2867520124 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.130+0000 7f2c6df9c640 1 --2- 172.21.15.113:0/2867520124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c6815e310 0x7f2c680b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.130+0000 7f2c6d79b640 1 -- 172.21.15.113:0/2867520124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c6815e310 msgr2=0x7f2c680b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.130+0000 7f2c6d79b640 1 --2- 172.21.15.113:0/2867520124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c6815e310 0x7f2c680b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.130+0000 7f2c6d79b640 1 -- 172.21.15.113:0/2867520124 >> v1:172.21.15.113:6789/0 conn(0x7f2c6806be40 legacy=0x7f2c680ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.130+0000 7f2c6cf9a640 1 -- 172.21.15.113:0/2867520124 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189596006 0 0) 0x7f2c680b1f80 con 0x7f2c6806be40 2024-04-25T00:10:55.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.130+0000 7f2c6d79b640 1 -- 172.21.15.113:0/2867520124 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c680b2260 con 0x7f2c680b3030 2024-04-25T00:10:55.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.130+0000 7f2c6d79b640 1 --2- 172.21.15.113:0/2867520124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b3030 0x7f2c6806b900 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f2c580097a0 tx=0x7f2c5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac10a3d440c3470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6cf9a640 1 -- 172.21.15.113:0/2867520124 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58037070 con 0x7f2c680b3030 2024-04-25T00:10:55.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6cf9a640 1 -- 172.21.15.113:0/2867520124 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c58032440 con 0x7f2c680b3030 2024-04-25T00:10:55.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6cf9a640 1 -- 172.21.15.113:0/2867520124 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58032810 con 0x7f2c680b3030 2024-04-25T00:10:55.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/2867520124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b3030 msgr2=0x7f2c6806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/2867520124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b3030 0x7f2c6806b900 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f2c580097a0 tx=0x7f2c5802a180 comp rx=0 tx=0).stop 2024-04-25T00:10:55.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/2867520124 shutdown_connections 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/2867520124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b3030 0x7f2c6806b900 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/2867520124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c6815e310 0x7f2c680b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/2867520124 >> 172.21.15.113:0/2867520124 conn(0x7f2c6809aa20 msgr2=0x7f2c6809ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/2867520124 shutdown_connections 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.131+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/2867520124 wait complete. 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.132+0000 7f2c6ef9e640 1 Processor -- start 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.132+0000 7f2c6ef9e640 1 -- start start 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6ef9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c6806be40 0x7f2c6814a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6ef9e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c680b3030 0x7f2c68150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6ef9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c6815e310 0x7f2c680ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6ef9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6815e6f0 con 0x7f2c6815e310 2024-04-25T00:10:55.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6ef9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c680b2820 con 0x7f2c6806be40 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6ef9e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c680b1bc0 con 0x7f2c680b3030 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6e79d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c6815e310 0x7f2c680ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6e79d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c6815e310 0x7f2c680ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48082/0 (socket says 172.21.15.113:48082) 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6e79d640 1 -- 172.21.15.113:0/3392512681 learned_addr learned my addr 172.21.15.113:0/3392512681 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.133+0000 7f2c6e79d640 1 -- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c680b3030 msgr2=0x7f2c68150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6df9c640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c6806be40 0x7f2c6814a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6d79b640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c680b3030 0x7f2c68150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6e79d640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c680b3030 0x7f2c68150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6e79d640 1 -- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c6806be40 msgr2=0x7f2c6814a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6e79d640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c6806be40 0x7f2c6814a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6e79d640 1 -- 172.21.15.113:0/3392512681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c680c7960 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6e79d640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c6815e310 0x7f2c680ca3e0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c450 tx=0x7f2c6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c5e7fc640 1 -- 172.21.15.113:0/3392512681 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60018070 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c5e7fc640 1 -- 172.21.15.113:0/3392512681 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c60007500 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c680c7c00 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c5e7fc640 1 -- 172.21.15.113:0/3392512681 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60004a20 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.134+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c680c7ea0 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.135+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c68068220 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.135+0000 7f2c5e7fc640 1 -- 172.21.15.113:0/3392512681 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c60004bc0 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.136+0000 7f2c5e7fc640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c4c06be60 0x7f2c4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.136+0000 7f2c5e7fc640 1 -- 172.21.15.113:0/3392512681 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f2c6008a0f0 con 0x7f2c6815e310 2024-04-25T00:10:55.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.136+0000 7f2c6df9c640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c4c06be60 0x7f2c4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.136+0000 7f2c6df9c640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c4c06be60 0x7f2c4c06e320 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008e50 tx=0x7f2c640083a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.140+0000 7f2c5e7fc640 1 -- 172.21.15.113:0/3392512681 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c600589c0 con 0x7f2c6815e310 2024-04-25T00:10:55.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.312+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c680cf5c0 con 0x7f2c6815e310 2024-04-25T00:10:55.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.312+0000 7f2c5e7fc640 1 -- 172.21.15.113:0/3392512681 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) ==== 93+0+10 (secure 0 0 0) 0x7f2c600582c0 con 0x7f2c6815e310 2024-04-25T00:10:55.313 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c4c06be60 msgr2=0x7f2c4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c4c06be60 0x7f2c4c06e320 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008e50 tx=0x7f2c640083a0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c6815e310 msgr2=0x7f2c680ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c6815e310 0x7f2c680ca3e0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c450 tx=0x7f2c6000c910 comp rx=0 tx=0).stop 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 shutdown_connections 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f2c4c06be60 0x7f2c4c06e320 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c6815e310 0x7f2c680ca3e0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c680b3030 0x7f2c68150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 --2- 172.21.15.113:0/3392512681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c6806be40 0x7f2c6814a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 >> 172.21.15.113:0/3392512681 conn(0x7f2c6809aa20 msgr2=0x7f2c680af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 shutdown_connections 2024-04-25T00:10:55.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.314+0000 7f2c6ef9e640 1 -- 172.21.15.113:0/3392512681 wait complete. 2024-04-25T00:10:55.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:10:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.424+0000 7f17bf241640 1 Processor -- start 2024-04-25T00:10:55.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.424+0000 7f17bf241640 1 -- start start 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.424+0000 7f17bf241640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 0x7f17b80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.424+0000 7f17bf241640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f17b815e6b0 0x7f17b80b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bf241640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17b815f4f0 con 0x7f17b815e2d0 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bf241640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f17b80cde00 con 0x7f17b815e6b0 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bf241640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17b815f7d0 con 0x7f17b80b3080 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bcfdd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f17b815e2d0 0x7f17b815ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34432/0 (socket says 172.21.15.113:34432) 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bcfdd640 1 -- 172.21.15.113:0/2770787453 learned_addr learned my addr 172.21.15.113:0/2770787453 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b7fff640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 0x7f17b80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bd7de640 1 --2- 172.21.15.113:0/2770787453 >> v2:172.21.15.178:3300/0 conn(0x7f17b815e6b0 0x7f17b80b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510065985 0 0) 0x7f17b815f4f0 con 0x7f17b815e2d0 2024-04-25T00:10:55.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17a4003520 con 0x7f17b815e2d0 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2756333321 0 0) 0x7f17a4003520 con 0x7f17b815e2d0 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 msgr2=0x7f17b80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 0x7f17b80b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 >> v2:172.21.15.178:3300/0 conn(0x7f17b815e6b0 msgr2=0x7f17b80b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 --2- 172.21.15.113:0/2770787453 >> v2:172.21.15.178:3300/0 conn(0x7f17b815e6b0 0x7f17b80b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b815fab0 con 0x7f17b815e2d0 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f17b00019f0 con 0x7f17b815e2d0 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 >> v1:172.21.15.113:6789/0 conn(0x7f17b815e2d0 legacy=0x7f17b815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a40039b0 0x7f17a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 0x7f17a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a4004d40 0x7f17a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f17b80cde00 con 0x7f17a4004d40 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17b815f7d0 con 0x7f17a40039b0 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17a400b9c0 con 0x7f17b80b3080 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bd7de640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a4004d40 0x7f17a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17bcfdd640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a40039b0 0x7f17a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.425+0000 7f17b7fff640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 0x7f17a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17bd7de640 1 -- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 msgr2=0x7f17a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17bd7de640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80b3080 0x7f17a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17bd7de640 1 -- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a40039b0 msgr2=0x7f17a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17bd7de640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a40039b0 0x7f17a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17bd7de640 1 -- 172.21.15.113:0/2770787453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a4003520 con 0x7f17a4004d40 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17bd7de640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a4004d40 0x7f17a40051c0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009ac0 tx=0x7f17a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a49580e8174ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0036070 con 0x7f17a4004d40 2024-04-25T00:10:55.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a0031400 con 0x7f17a4004d40 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.426+0000 7f17b77fe640 1 -- 172.21.15.113:0/2770787453 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0009690 con 0x7f17a4004d40 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.427+0000 7f17bf241640 1 -- 172.21.15.113:0/2770787453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a4004d40 msgr2=0x7f17a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.427+0000 7f17bf241640 1 --2- 172.21.15.113:0/2770787453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a4004d40 0x7f17a40051c0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009ac0 tx=0x7f17a4005830 comp rx=0 tx=0).stop 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.427+0000 7f17bd7de640 1 -- 172.21.15.113:0/2770787453 reap_dead start 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.427+0000 7f17bf241640 1 -- 172.21.15.113:0/2770787453 shutdown_connections 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.427+0000 7f17bf241640 1 -- 172.21.15.113:0/2770787453 >> 172.21.15.113:0/2770787453 conn(0x7f17b806b0f0 msgr2=0x7f17b806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.427+0000 7f17bf241640 1 -- 172.21.15.113:0/2770787453 shutdown_connections 2024-04-25T00:10:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.427+0000 7f17bf241640 1 -- 172.21.15.113:0/2770787453 wait complete. 2024-04-25T00:10:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.428+0000 7f17bf241640 1 Processor -- start 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.428+0000 7f17bf241640 1 -- start start 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bf241640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b81540f0 0x7f17b813f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bf241640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17b81544d0 0x7f17b813fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bf241640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b81515f0 0x7f17b8151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bf241640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f17b80b44b0 con 0x7f17b81544d0 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bf241640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17b80b4330 con 0x7f17b81540f0 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bf241640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17b80b4630 con 0x7f17b81515f0 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17b81544d0 0x7f17b813fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17b81544d0 0x7f17b813fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48094/0 (socket says 172.21.15.113:48094) 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 -- 172.21.15.113:0/1340864211 learned_addr learned my addr 172.21.15.113:0/1340864211 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bcfdd640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b81540f0 0x7f17b813f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17bd7de640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b81515f0 0x7f17b8151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 -- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b81515f0 msgr2=0x7f17b8151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b81515f0 0x7f17b8151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 -- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b81540f0 msgr2=0x7f17b813f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b81540f0 0x7f17b813f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 -- 172.21.15.113:0/1340864211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b8152300 con 0x7f17b81544d0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.429+0000 7f17b7fff640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17b81544d0 0x7f17b813fdc0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f17a8011910 tx=0x7f17a8011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.430+0000 7f17b57fa640 1 -- 172.21.15.113:0/1340864211 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8019070 con 0x7f17b81544d0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.430+0000 7f17b57fa640 1 -- 172.21.15.113:0/1340864211 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a8004240 con 0x7f17b81544d0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.430+0000 7f17b57fa640 1 -- 172.21.15.113:0/1340864211 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800f7c0 con 0x7f17b81544d0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.430+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b81525e0 con 0x7f17b81544d0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.430+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b8152ba0 con 0x7f17b81544d0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.431+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b80691a0 con 0x7f17b81544d0 2024-04-25T00:10:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.431+0000 7f17b57fa640 1 -- 172.21.15.113:0/1340864211 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17a801c070 con 0x7f17b81544d0 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.432+0000 7f17b57fa640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f179806be60 0x7f179806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.432+0000 7f17bcfdd640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f179806be60 0x7f179806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.432+0000 7f17b57fa640 1 -- 172.21.15.113:0/1340864211 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f17a808b280 con 0x7f17b81544d0 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.432+0000 7f17bcfdd640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f179806be60 0x7f179806e320 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004bc0 tx=0x7f17b0006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.436+0000 7f17b57fa640 1 -- 172.21.15.113:0/1340864211 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f17a8059bd0 con 0x7f17b81544d0 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.608+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f17b8072680 con 0x7f17b81544d0 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.609+0000 7f17b57fa640 1 -- 172.21.15.113:0/1340864211 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) ==== 104+0+11 (secure 0 0 0) 0x7f17a80594d0 con 0x7f17b81544d0 2024-04-25T00:10:55.877 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f179806be60 msgr2=0x7f179806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f179806be60 0x7f179806e320 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004bc0 tx=0x7f17b0006040 comp rx=0 tx=0).stop 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17b81544d0 msgr2=0x7f17b813fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17b81544d0 0x7f17b813fdc0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f17a8011910 tx=0x7f17a8011d10 comp rx=0 tx=0).stop 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 shutdown_connections 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f179806be60 0x7f179806e320 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b81515f0 0x7f17b8151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17b81544d0 0x7f17b813fdc0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 --2- 172.21.15.113:0/1340864211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b81540f0 0x7f17b813f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 >> 172.21.15.113:0/1340864211 conn(0x7f17b806b0f0 msgr2=0x7f17b80b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 shutdown_connections 2024-04-25T00:10:55.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.611+0000 7f17bf241640 1 -- 172.21.15.113:0/1340864211 wait complete. 2024-04-25T00:10:55.880 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:10:55.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.980+0000 7fd32a0fc640 1 Processor -- start 2024-04-25T00:10:55.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.980+0000 7fd32a0fc640 1 -- start start 2024-04-25T00:10:55.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd32a0fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd32415e510 0x7fd3240d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd32a0fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3240d28c0 0x7fd3240b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd32a0fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd32a0fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd3240b6650 con 0x7fd3240d28c0 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd32a0fc640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3240d2e00 con 0x7fd32415e510 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd32a0fc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240d30e0 con 0x7fd3240b61e0 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd322ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3240d28c0 0x7fd3240b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd322ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3240d28c0 0x7fd3240b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48100/0 (socket says 172.21.15.113:48100) 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.981+0000 7fd322ffd640 1 -- 172.21.15.113:0/4110703928 learned_addr learned my addr 172.21.15.113:0/4110703928 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd322ffd640 1 -- 172.21.15.113:0/4110703928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 msgr2=0x7fd324071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd3237fe640 1 --2- 172.21.15.113:0/4110703928 >> v2:172.21.15.178:3300/0 conn(0x7fd32415e510 0x7fd3240d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd323fff640 1 --2- 172.21.15.113:0/4110703928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd322ffd640 1 --2- 172.21.15.113:0/4110703928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd322ffd640 1 -- 172.21.15.113:0/4110703928 >> v2:172.21.15.178:3300/0 conn(0x7fd32415e510 msgr2=0x7fd3240d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd322ffd640 1 --2- 172.21.15.113:0/4110703928 >> v2:172.21.15.178:3300/0 conn(0x7fd32415e510 0x7fd3240d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd322ffd640 1 -- 172.21.15.113:0/4110703928 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240d33c0 con 0x7fd3240d28c0 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd322ffd640 1 --2- 172.21.15.113:0/4110703928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3240d28c0 0x7fd3240b5ca0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fd3240d52a0 tx=0x7fd31802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157972b35acd5a11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd3227fc640 1 -- 172.21.15.113:0/4110703928 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318037070 con 0x7fd3240d28c0 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd3227fc640 1 -- 172.21.15.113:0/4110703928 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd318032440 con 0x7fd3240d28c0 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.982+0000 7fd3227fc640 1 -- 172.21.15.113:0/4110703928 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318032810 con 0x7fd3240d28c0 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 -- 172.21.15.113:0/4110703928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3240d28c0 msgr2=0x7fd3240b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/4110703928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3240d28c0 0x7fd3240b5ca0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fd3240d52a0 tx=0x7fd31802a140 comp rx=0 tx=0).stop 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 -- 172.21.15.113:0/4110703928 shutdown_connections 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/4110703928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/4110703928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3240d28c0 0x7fd3240b5ca0 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/4110703928 >> v2:172.21.15.178:3300/0 conn(0x7fd32415e510 0x7fd3240d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 -- 172.21.15.113:0/4110703928 >> 172.21.15.113:0/4110703928 conn(0x7fd32406b720 msgr2=0x7fd3240d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 -- 172.21.15.113:0/4110703928 shutdown_connections 2024-04-25T00:10:55.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.983+0000 7fd32a0fc640 1 -- 172.21.15.113:0/4110703928 wait complete. 2024-04-25T00:10:55.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.984+0000 7fd32a0fc640 1 Processor -- start 2024-04-25T00:10:55.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.984+0000 7fd32a0fc640 1 -- start start 2024-04-25T00:10:55.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd32a0fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd32a0fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd32415e510 0x7fd3241f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd32a0fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32415c480 0x7fd3241f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd32a0fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3240d3ea0 con 0x7fd32415c480 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd32a0fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3240d3980 con 0x7fd32415e510 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd32a0fc640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240b9dd0 con 0x7fd3240b61e0 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32415c480 0x7fd3241f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32415c480 0x7fd3241f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48104/0 (socket says 172.21.15.113:48104) 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 -- 172.21.15.113:0/1402672171 learned_addr learned my addr 172.21.15.113:0/1402672171 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd3237fe640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd322ffd640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd32415e510 0x7fd3241f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 -- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 msgr2=0x7fd324160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 -- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd32415e510 msgr2=0x7fd3241f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd32415e510 0x7fd3241f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.985+0000 7fd323fff640 1 -- 172.21.15.113:0/1402672171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3241617c0 con 0x7fd32415c480 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.986+0000 7fd323fff640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32415c480 0x7fd3241f7bb0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00cbb0 tx=0x7fd31c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.986+0000 7fd30bfff640 1 -- 172.21.15.113:0/1402672171 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c015070 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.986+0000 7fd30bfff640 1 -- 172.21.15.113:0/1402672171 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c00abd0 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.986+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32415b9e0 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.986+0000 7fd30bfff640 1 -- 172.21.15.113:0/1402672171 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00d040 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.986+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32415bc80 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.987+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3240691a0 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.987+0000 7fd30bfff640 1 -- 172.21.15.113:0/1402672171 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd31c018070 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.988+0000 7fd30bfff640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd31006be30 0x7fd31006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.988+0000 7fd30bfff640 1 -- 172.21.15.113:0/1402672171 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fd31c08add0 con 0x7fd32415c480 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.988+0000 7fd3237fe640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd31006be30 0x7fd31006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.988+0000 7fd3237fe640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd31006be30 0x7fd31006e2f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd31400aa20 tx=0x7fd314008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:55.992+0000 7fd30bfff640 1 -- 172.21.15.113:0/1402672171 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd31c0590d0 con 0x7fd32415c480 2024-04-25T00:10:56.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.164+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd3240d13b0 con 0x7fd32415c480 2024-04-25T00:10:56.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.165+0000 7fd30bfff640 1 -- 172.21.15.113:0/1402672171 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) ==== 100+0+11 (secure 0 0 0) 0x7fd31c05d3d0 con 0x7fd32415c480 2024-04-25T00:10:56.165 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-25T00:10:56.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.166+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd31006be30 msgr2=0x7fd31006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.166+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd31006be30 0x7fd31006e2f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd31400aa20 tx=0x7fd314008040 comp rx=0 tx=0).stop 2024-04-25T00:10:56.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32415c480 msgr2=0x7fd3241f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32415c480 0x7fd3241f7bb0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00cbb0 tx=0x7fd31c00a4c0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 shutdown_connections 2024-04-25T00:10:56.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd31006be30 0x7fd31006e2f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd32415e510 0x7fd3241f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32415c480 0x7fd3241f7bb0 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 --2- 172.21.15.113:0/1402672171 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3240b61e0 0x7fd324160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 >> 172.21.15.113:0/1402672171 conn(0x7fd32406b720 msgr2=0x7fd324159590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:56.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 shutdown_connections 2024-04-25T00:10:56.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.167+0000 7fd32a0fc640 1 -- 172.21.15.113:0/1402672171 wait complete. 2024-04-25T00:10:56.178 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:10:56.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.274+0000 7fe81ca38640 1 Processor -- start 2024-04-25T00:10:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.274+0000 7fe81ca38640 1 -- start start 2024-04-25T00:10:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe81ca38640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe818072ea0 0x7fe818071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe81ca38640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe8180b3af0 0x7fe8180b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe81ca38640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8180b2d50 con 0x7fe81815a6b0 2024-04-25T00:10:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe81ca38640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe818072a70 con 0x7fe818072ea0 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe81ca38640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe8180b3010 con 0x7fe8180b3af0 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe80ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe818072ea0 0x7fe818071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe81659c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe81815a6b0 0x7fe818070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34440/0 (socket says 172.21.15.113:34440) 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe81659c640 1 -- 172.21.15.113:0/2547217801 learned_addr learned my addr 172.21.15.113:0/2547217801 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe815d9b640 1 -- 172.21.15.113:0/2547217801 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948816666 0 0) 0x7fe8180b2d50 con 0x7fe81815a6b0 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe815d9b640 1 -- 172.21.15.113:0/2547217801 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe804005730 con 0x7fe81815a6b0 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe816d9d640 1 --2- 172.21.15.113:0/2547217801 >> v2:172.21.15.178:3300/0 conn(0x7fe8180b3af0 0x7fe8180b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe80ffff640 1 -- 172.21.15.113:0/2547217801 >> v2:172.21.15.178:3300/0 conn(0x7fe8180b3af0 msgr2=0x7fe8180b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe80ffff640 1 --2- 172.21.15.113:0/2547217801 >> v2:172.21.15.178:3300/0 conn(0x7fe8180b3af0 0x7fe8180b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe80ffff640 1 -- 172.21.15.113:0/2547217801 >> v1:172.21.15.113:6789/0 conn(0x7fe81815a6b0 legacy=0x7fe818070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe80ffff640 1 -- 172.21.15.113:0/2547217801 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8180b32f0 con 0x7fe818072ea0 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.275+0000 7fe816d9d640 1 --2- 172.21.15.113:0/2547217801 >> v2:172.21.15.178:3300/0 conn(0x7fe8180b3af0 0x7fe8180b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.276+0000 7fe80ffff640 1 --2- 172.21.15.113:0/2547217801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe818072ea0 0x7fe818071410 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fe8180cc5a0 tx=0x7fe80002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7132484173bd951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.276+0000 7fe815d9b640 1 -- 172.21.15.113:0/2547217801 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800037070 con 0x7fe818072ea0 2024-04-25T00:10:56.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.276+0000 7fe815d9b640 1 -- 172.21.15.113:0/2547217801 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe800032440 con 0x7fe818072ea0 2024-04-25T00:10:56.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.276+0000 7fe815d9b640 1 -- 172.21.15.113:0/2547217801 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800032810 con 0x7fe818072ea0 2024-04-25T00:10:56.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 -- 172.21.15.113:0/2547217801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe818072ea0 msgr2=0x7fe818071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 --2- 172.21.15.113:0/2547217801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe818072ea0 0x7fe818071410 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fe8180cc5a0 tx=0x7fe80002a1a0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 -- 172.21.15.113:0/2547217801 shutdown_connections 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 --2- 172.21.15.113:0/2547217801 >> v2:172.21.15.178:3300/0 conn(0x7fe8180b3af0 0x7fe8180b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 --2- 172.21.15.113:0/2547217801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe818072ea0 0x7fe818071410 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 -- 172.21.15.113:0/2547217801 >> 172.21.15.113:0/2547217801 conn(0x7fe8180b1a50 msgr2=0x7fe8180b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 -- 172.21.15.113:0/2547217801 shutdown_connections 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.277+0000 7fe81ca38640 1 -- 172.21.15.113:0/2547217801 wait complete. 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.278+0000 7fe81ca38640 1 Processor -- start 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.278+0000 7fe81ca38640 1 -- start start 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe818072ea0 0x7fe818141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8180b3af0 0x7fe81814f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81815a6b0 0x7fe8181f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe81815aa90 con 0x7fe81815a6b0 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe818072720 con 0x7fe818072ea0 2024-04-25T00:10:56.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe81815ac10 con 0x7fe8180b3af0 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81815a6b0 0x7fe8181f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81815a6b0 0x7fe8181f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48118/0 (socket says 172.21.15.113:48118) 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 -- 172.21.15.113:0/1485837707 learned_addr learned my addr 172.21.15.113:0/1485837707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe80ffff640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8180b3af0 0x7fe81814f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81659c640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe818072ea0 0x7fe818141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 -- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8180b3af0 msgr2=0x7fe81814f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8180b3af0 0x7fe81814f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 -- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe818072ea0 msgr2=0x7fe818141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe818072ea0 0x7fe818141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 -- 172.21.15.113:0/1485837707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81814f930 con 0x7fe81815a6b0 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81659c640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe818072ea0 0x7fe818141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe816d9d640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81815a6b0 0x7fe8181f4250 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ce40 tx=0x7fe81000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe80effd640 1 -- 172.21.15.113:0/1485837707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81001a070 con 0x7fe81815a6b0 2024-04-25T00:10:56.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe80effd640 1 -- 172.21.15.113:0/1485837707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81000d840 con 0x7fe81815a6b0 2024-04-25T00:10:56.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81814fc10 con 0x7fe81815a6b0 2024-04-25T00:10:56.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe80effd640 1 -- 172.21.15.113:0/1485837707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81000dc30 con 0x7fe81815a6b0 2024-04-25T00:10:56.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.279+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181501d0 con 0x7fe81815a6b0 2024-04-25T00:10:56.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.280+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8180691a0 con 0x7fe81815a6b0 2024-04-25T00:10:56.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.281+0000 7fe80effd640 1 -- 172.21.15.113:0/1485837707 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe81001d070 con 0x7fe81815a6b0 2024-04-25T00:10:56.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.281+0000 7fe80effd640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7f806be30 0x7fe7f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.281+0000 7fe80effd640 1 -- 172.21.15.113:0/1485837707 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fe81008b600 con 0x7fe81815a6b0 2024-04-25T00:10:56.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.281+0000 7fe81659c640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7f806be30 0x7fe7f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.282+0000 7fe81659c640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7f806be30 0x7fe7f806e2f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe808005880 tx=0x7fe808009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.286+0000 7fe80effd640 1 -- 172.21.15.113:0/1485837707 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe810059fe0 con 0x7fe81815a6b0 2024-04-25T00:10:56.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.494+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8180b2900 con 0x7fe81815a6b0 2024-04-25T00:10:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.494+0000 7fe80effd640 1 -- 172.21.15.113:0/1485837707 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7fe81005ee20 con 0x7fe81815a6b0 2024-04-25T00:10:56.495 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:56.495 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:50.995654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10794,"qdb_cluster":[10794]},"id":13}]} 2024-04-25T00:10:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-04-25T00:10:56.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7f806be30 msgr2=0x7fe7f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7f806be30 0x7fe7f806e2f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe808005880 tx=0x7fe808009040 comp rx=0 tx=0).stop 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81815a6b0 msgr2=0x7fe8181f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81815a6b0 0x7fe8181f4250 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ce40 tx=0x7fe81000a710 comp rx=0 tx=0).stop 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 shutdown_connections 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe7f806be30 0x7fe7f806e2f0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81815a6b0 0x7fe8181f4250 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8180b3af0 0x7fe81814f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 --2- 172.21.15.113:0/1485837707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe818072ea0 0x7fe818141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 >> 172.21.15.113:0/1485837707 conn(0x7fe8180b1a50 msgr2=0x7fe818154720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 shutdown_connections 2024-04-25T00:10:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.496+0000 7fe81ca38640 1 -- 172.21.15.113:0/1485837707 wait complete. 2024-04-25T00:10:56.508 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:56.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.604+0000 7f974c9b7640 1 Processor -- start 2024-04-25T00:10:56.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.604+0000 7f974c9b7640 1 -- start start 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974c9b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f97480b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974c9b7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97480b5240 0x7f974806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974c9b7640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97480b5680 0x7f974806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974c9b7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f97480b4510 con 0x7f97480b5240 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974c9b7640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9748144cf0 con 0x7f97480b5680 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974c9b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97480b2f30 con 0x7f9748144910 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97480b5240 0x7f974806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97480b5240 0x7f974806dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48120/0 (socket says 172.21.15.113:48120) 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 -- 172.21.15.113:0/3773221163 learned_addr learned my addr 172.21.15.113:0/3773221163 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974659c640 1 --2- 172.21.15.113:0/3773221163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f97480b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9746d9d640 1 --2- 172.21.15.113:0/3773221163 >> v2:172.21.15.178:3300/0 conn(0x7f97480b5680 0x7f974806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 -- 172.21.15.113:0/3773221163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 msgr2=0x7f97480b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 --2- 172.21.15.113:0/3773221163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f97480b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 -- 172.21.15.113:0/3773221163 >> v2:172.21.15.178:3300/0 conn(0x7f97480b5680 msgr2=0x7f974806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 --2- 172.21.15.113:0/3773221163 >> v2:172.21.15.178:3300/0 conn(0x7f97480b5680 0x7f974806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f9745d9b640 1 -- 172.21.15.113:0/3773221163 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97480b3210 con 0x7f97480b5240 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.605+0000 7f974659c640 1 --2- 172.21.15.113:0/3773221163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f97480b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.606+0000 7f9745d9b640 1 --2- 172.21.15.113:0/3773221163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f974806dfd0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f9748062820 tx=0x7f973c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6903d983536c9cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.606+0000 7f974559a640 1 -- 172.21.15.113:0/3773221163 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c036070 con 0x7f97480b5240 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.606+0000 7f974559a640 1 -- 172.21.15.113:0/3773221163 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c031600 con 0x7f97480b5240 2024-04-25T00:10:56.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.606+0000 7f974559a640 1 -- 172.21.15.113:0/3773221163 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0319d0 con 0x7f97480b5240 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 -- 172.21.15.113:0/3773221163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 msgr2=0x7f974806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 --2- 172.21.15.113:0/3773221163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f974806dfd0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f9748062820 tx=0x7f973c02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 -- 172.21.15.113:0/3773221163 shutdown_connections 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 --2- 172.21.15.113:0/3773221163 >> v2:172.21.15.178:3300/0 conn(0x7f97480b5680 0x7f974806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 --2- 172.21.15.113:0/3773221163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f974806dfd0 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 --2- 172.21.15.113:0/3773221163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f97480b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 -- 172.21.15.113:0/3773221163 >> 172.21.15.113:0/3773221163 conn(0x7f97480b1a50 msgr2=0x7f97480b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:56.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 -- 172.21.15.113:0/3773221163 shutdown_connections 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.607+0000 7f974c9b7640 1 -- 172.21.15.113:0/3773221163 wait complete. 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.608+0000 7f974c9b7640 1 Processor -- start 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.608+0000 7f974c9b7640 1 -- start start 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f97481347d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97480b5680 0x7f9748158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f974813d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97480b4da0 con 0x7f97480b5240 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97480b37d0 con 0x7f97480b5680 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97480b4f20 con 0x7f9748144910 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f97481347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f97481347d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48128/0 (socket says 172.21.15.113:48128) 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 -- 172.21.15.113:0/531831107 learned_addr learned my addr 172.21.15.113:0/531831107 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:56.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f9746d9d640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f974813d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f9745d9b640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97480b5680 0x7f9748158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 -- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 msgr2=0x7f974813d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f974813d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 -- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97480b5680 msgr2=0x7f9748158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97480b5680 0x7f9748158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 -- 172.21.15.113:0/531831107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9748133730 con 0x7f97480b5240 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f9745d9b640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97480b5680 0x7f9748158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f9746d9d640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f974813d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974659c640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f97481347d0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f974000c7f0 tx=0x7f974000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f9736ffd640 1 -- 172.21.15.113:0/531831107 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740015070 con 0x7f97480b5240 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f9736ffd640 1 -- 172.21.15.113:0/531831107 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9740007a30 con 0x7f97480b5240 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9748133a10 con 0x7f97480b5240 2024-04-25T00:10:56.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f9736ffd640 1 -- 172.21.15.113:0/531831107 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740007e00 con 0x7f97480b5240 2024-04-25T00:10:56.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.609+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97481342b0 con 0x7f97480b5240 2024-04-25T00:10:56.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.611+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97480691a0 con 0x7f97480b5240 2024-04-25T00:10:56.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.611+0000 7f9736ffd640 1 -- 172.21.15.113:0/531831107 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9740018070 con 0x7f97480b5240 2024-04-25T00:10:56.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.611+0000 7f9736ffd640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f972806be30 0x7f972806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.611+0000 7f9736ffd640 1 -- 172.21.15.113:0/531831107 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f974008aeb0 con 0x7f97480b5240 2024-04-25T00:10:56.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.611+0000 7f9745d9b640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f972806be30 0x7f972806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.612+0000 7f9745d9b640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f972806be30 0x7f972806e2f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f973c033000 tx=0x7f973c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.616+0000 7f9736ffd640 1 -- 172.21.15.113:0/531831107 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9740059800 con 0x7f97480b5240 2024-04-25T00:10:56.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.789+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f974815e090 con 0x7f97480b5240 2024-04-25T00:10:56.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.790+0000 7f9736ffd640 1 -- 172.21.15.113:0/531831107 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) ==== 74+0+16862 (secure 0 0 0) 0x7f9740059100 con 0x7f97480b5240 2024-04-25T00:10:56.791 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:56.791 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":183,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:46.979993+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:31.412384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:32.173467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/3057967443":"2024-04-26T00:09:46.979970+0000","172.21.15.178:6833/3057967443":"2024-04-26T00:09:46.979970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:10:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f972806be30 msgr2=0x7f972806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f972806be30 0x7f972806e2f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f973c033000 tx=0x7f973c009000 comp rx=0 tx=0).stop 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 msgr2=0x7f97481347d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f97481347d0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f974000c7f0 tx=0x7f974000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 shutdown_connections 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f972806be30 0x7f972806e2f0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97480b5680 0x7f9748158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9748144910 0x7f974813d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 --2- 172.21.15.113:0/531831107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97480b5240 0x7f97481347d0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 >> 172.21.15.113:0/531831107 conn(0x7f97480b1a50 msgr2=0x7f974806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 shutdown_connections 2024-04-25T00:10:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.792+0000 7f974c9b7640 1 -- 172.21.15.113:0/531831107 wait complete. 2024-04-25T00:10:56.804 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:10:56.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:10:56.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.900+0000 7fd366b22640 1 Processor -- start 2024-04-25T00:10:56.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.900+0000 7fd366b22640 1 -- start start 2024-04-25T00:10:56.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd366b22640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd36006e6e0 0x7fd3600b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd366b22640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3601453c0 0x7fd360071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd366b22640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3600cfad0 con 0x7fd360144d60 2024-04-25T00:10:56.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd366b22640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd360145c40 con 0x7fd3601453c0 2024-04-25T00:10:56.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd366b22640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3600cf230 con 0x7fd36006e6e0 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd3650bf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3601453c0 0x7fd360071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd3650bf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3601453c0 0x7fd360071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48138/0 (socket says 172.21.15.113:48138) 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd3650bf640 1 -- 172.21.15.113:0/955202243 learned_addr learned my addr 172.21.15.113:0/955202243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd35ffff640 1 --2- 172.21.15.113:0/955202243 >> v2:172.21.15.178:3300/0 conn(0x7fd36006e6e0 0x7fd3600b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd3650bf640 1 -- 172.21.15.113:0/955202243 >> v2:172.21.15.178:3300/0 conn(0x7fd36006e6e0 msgr2=0x7fd3600b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd3650bf640 1 --2- 172.21.15.113:0/955202243 >> v2:172.21.15.178:3300/0 conn(0x7fd36006e6e0 0x7fd3600b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd3650bf640 1 -- 172.21.15.113:0/955202243 >> v1:172.21.15.178:6789/0 conn(0x7fd360144d60 legacy=0x7fd36006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.901+0000 7fd3650bf640 1 -- 172.21.15.113:0/955202243 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3600cf510 con 0x7fd3601453c0 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd3650bf640 1 --2- 172.21.15.113:0/955202243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd360071330 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fd360075250 tx=0x7fd35802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea85abe031f38b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd35f7fe640 1 -- 172.21.15.113:0/955202243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358036070 con 0x7fd3601453c0 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd35f7fe640 1 -- 172.21.15.113:0/955202243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd358031600 con 0x7fd3601453c0 2024-04-25T00:10:56.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd35f7fe640 1 -- 172.21.15.113:0/955202243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580319d0 con 0x7fd3601453c0 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd366b22640 1 -- 172.21.15.113:0/955202243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 msgr2=0x7fd360071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd366b22640 1 --2- 172.21.15.113:0/955202243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd360071330 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fd360075250 tx=0x7fd35802a5d0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd366b22640 1 -- 172.21.15.113:0/955202243 shutdown_connections 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd366b22640 1 --2- 172.21.15.113:0/955202243 >> v2:172.21.15.178:3300/0 conn(0x7fd36006e6e0 0x7fd3600b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd366b22640 1 --2- 172.21.15.113:0/955202243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd360071330 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.902+0000 7fd366b22640 1 -- 172.21.15.113:0/955202243 >> 172.21.15.113:0/955202243 conn(0x7fd3600b1a50 msgr2=0x7fd3600b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.903+0000 7fd366b22640 1 -- 172.21.15.113:0/955202243 shutdown_connections 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.903+0000 7fd366b22640 1 -- 172.21.15.113:0/955202243 wait complete. 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 Processor -- start 2024-04-25T00:10:56.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 -- start start 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd36006e6e0 0x7fd360131300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd360144d60 0x7fd360159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd3601f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd360145140 con 0x7fd3601453c0 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd360070e10 con 0x7fd36006e6e0 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.904+0000 7fd366b22640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd360071870 con 0x7fd360144d60 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd3601f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd3601f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48152/0 (socket says 172.21.15.113:48152) 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 -- 172.21.15.113:0/1729597959 learned_addr learned my addr 172.21.15.113:0/1729597959 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd35ffff640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd360144d60 0x7fd360159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3648be640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd36006e6e0 0x7fd360131300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 -- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd360144d60 msgr2=0x7fd360159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd360144d60 0x7fd360159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 -- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd36006e6e0 msgr2=0x7fd360131300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd36006e6e0 0x7fd360131300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 -- 172.21.15.113:0/1729597959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36013caa0 con 0x7fd3601453c0 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd3650bf640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd3601f78a0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fd358009f40 tx=0x7fd358004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd35d7fa640 1 -- 172.21.15.113:0/1729597959 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358036050 con 0x7fd3601453c0 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd35d7fa640 1 -- 172.21.15.113:0/1729597959 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd358042070 con 0x7fd3601453c0 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd35d7fa640 1 -- 172.21.15.113:0/1729597959 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358009510 con 0x7fd3601453c0 2024-04-25T00:10:56.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601f86f0 con 0x7fd3601453c0 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.905+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd360130820 con 0x7fd3601453c0 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.906+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3600691a0 con 0x7fd3601453c0 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.907+0000 7fd35d7fa640 1 -- 172.21.15.113:0/1729597959 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd358007bf0 con 0x7fd3601453c0 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.907+0000 7fd35d7fa640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.907+0000 7fd35d7fa640 1 -- 172.21.15.113:0/1729597959 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7fd3580abc40 con 0x7fd3601453c0 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.907+0000 7fd3648be640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.907+0000 7fd3648be640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd34406be30 0x7fd34406e2f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd354004ea0 tx=0x7fd354009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:56.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:56.911+0000 7fd35d7fa640 1 -- 172.21.15.113:0/1729597959 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd35807a620 con 0x7fd3601453c0 2024-04-25T00:10:57.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.120+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36015c1c0 con 0x7fd3601453c0 2024-04-25T00:10:57.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.121+0000 7fd35d7fa640 1 -- 172.21.15.113:0/1729597959 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7fd358079f20 con 0x7fd3601453c0 2024-04-25T00:10:57.121 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:57.122 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:50.995654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10794,"qdb_cluster":[10794]},"id":13}]} 2024-04-25T00:10:57.122 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd34406be30 msgr2=0x7fd34406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd34406be30 0x7fd34406e2f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd354004ea0 tx=0x7fd354009040 comp rx=0 tx=0).stop 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 msgr2=0x7fd3601f78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd3601f78a0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fd358009f40 tx=0x7fd358004a40 comp rx=0 tx=0).stop 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 shutdown_connections 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3601453c0 0x7fd3601f78a0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd360144d60 0x7fd360159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 --2- 172.21.15.113:0/1729597959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd36006e6e0 0x7fd360131300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 >> 172.21.15.113:0/1729597959 conn(0x7fd3600b1a50 msgr2=0x7fd3600b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 shutdown_connections 2024-04-25T00:10:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.123+0000 7fd366b22640 1 -- 172.21.15.113:0/1729597959 wait complete. 2024-04-25T00:10:57.135 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:10:57.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.231+0000 7f3442310640 1 Processor -- start 2024-04-25T00:10:57.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.231+0000 7f3442310640 1 -- start start 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f3442310640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 0x7f343c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f3442310640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f343c144cf0 con 0x7f343c144910 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f3442310640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f343c0b2f30 con 0x7f343c06e510 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f3442310640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f343c0b3210 con 0x7f343c0b3fd0 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343bfff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f343c144910 0x7f343c0b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34442/0 (socket says 172.21.15.113:34442) 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343bfff640 1 -- 172.21.15.113:0/2502567115 learned_addr learned my addr 172.21.15.113:0/2502567115 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342553712 0 0) 0x7f343c144cf0 con 0x7f343c144910 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343b7fe640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 0x7f343c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3424003520 con 0x7f343c144910 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4145148441 0 0) 0x7f3424003520 con 0x7f343c144910 2024-04-25T00:10:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 msgr2=0x7f343c06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 0x7f343c06dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 >> v1:172.21.15.178:6789/0 conn(0x7f343c06e510 legacy=0x7f343c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c0b34f0 con 0x7f343c144910 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f342c001670 con 0x7f343c144910 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.232+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 >> v1:172.21.15.113:6789/0 conn(0x7f343c144910 legacy=0x7f343c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343affd640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 0x7f34240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343affd640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3424003f20 0x7f3424004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343affd640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3424004d40 0x7f34240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f343c0b3210 con 0x7f3424003f20 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f342400ba70 con 0x7f3424004d40 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f342400de80 con 0x7f343c0b3fd0 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343b7fe640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3424003f20 0x7f3424004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343bfff640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 0x7f34240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f34408ad640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3424004d40 0x7f34240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343b7fe640 1 -- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 msgr2=0x7f34240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343b7fe640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 0x7f34240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343b7fe640 1 -- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3424004d40 msgr2=0x7f34240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343b7fe640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3424004d40 0x7f34240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343b7fe640 1 -- 172.21.15.113:0/2502567115 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3424003520 con 0x7f3424003f20 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.233+0000 7f343bfff640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f343c0b3fd0 0x7f34240039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f343b7fe640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3424003f20 0x7f3424004800 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f3430009ac0 tx=0x7f3424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6190cd1cccd28c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430036070 con 0x7f3424003f20 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3430031400 con 0x7f3424003f20 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f343affd640 1 -- 172.21.15.113:0/2502567115 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430009690 con 0x7f3424003f20 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f3442310640 1 -- 172.21.15.113:0/2502567115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3424003f20 msgr2=0x7f3424004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f3442310640 1 --2- 172.21.15.113:0/2502567115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3424003f20 0x7f3424004800 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f3430009ac0 tx=0x7f3424005830 comp rx=0 tx=0).stop 2024-04-25T00:10:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f34408ad640 1 -- 172.21.15.113:0/2502567115 reap_dead start 2024-04-25T00:10:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f3442310640 1 -- 172.21.15.113:0/2502567115 shutdown_connections 2024-04-25T00:10:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f3442310640 1 -- 172.21.15.113:0/2502567115 >> 172.21.15.113:0/2502567115 conn(0x7f343c0b1a50 msgr2=0x7f343c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f3442310640 1 -- 172.21.15.113:0/2502567115 shutdown_connections 2024-04-25T00:10:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.234+0000 7f3442310640 1 -- 172.21.15.113:0/2502567115 wait complete. 2024-04-25T00:10:57.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.235+0000 7f3442310640 1 Processor -- start 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f3442310640 1 -- start start 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f3442310640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3424003f20 0x7f343c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f3442310640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c14f420 0x7f343c13add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f3442310640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f343c14e690 0x7f343c14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f3442310640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f343c0b4da0 con 0x7f343c14e690 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f3442310640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f343c0b4410 con 0x7f343c14f420 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f3442310640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f343c0b4f20 con 0x7f3424003f20 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f34408ad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f343c14e690 0x7f343c14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f34408ad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f343c14e690 0x7f343c14eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48174/0 (socket says 172.21.15.113:48174) 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.236+0000 7f34408ad640 1 -- 172.21.15.113:0/3577268092 learned_addr learned my addr 172.21.15.113:0/3577268092 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f34408ad640 1 -- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3424003f20 msgr2=0x7f343c0ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f343b7fe640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c14f420 0x7f343c13add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f343bfff640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3424003f20 0x7f343c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f34408ad640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3424003f20 0x7f343c0ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f34408ad640 1 -- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c14f420 msgr2=0x7f343c13add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f34408ad640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c14f420 0x7f343c13add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f34408ad640 1 -- 172.21.15.113:0/3577268092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c0ce000 con 0x7f343c14e690 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f343b7fe640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c14f420 0x7f343c13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f343bfff640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3424003f20 0x7f343c0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f34408ad640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f343c14e690 0x7f343c14eb40 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f343c06ed40 tx=0x7f343400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f3438ff9640 1 -- 172.21.15.113:0/3577268092 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434019070 con 0x7f343c14e690 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f3438ff9640 1 -- 172.21.15.113:0/3577268092 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3434006e20 con 0x7f343c14e690 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f3438ff9640 1 -- 172.21.15.113:0/3577268092 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434003b90 con 0x7f343c14e690 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c13bcb0 con 0x7f343c14e690 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.237+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c13c270 con 0x7f343c14e690 2024-04-25T00:10:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.238+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c0691a0 con 0x7f343c14e690 2024-04-25T00:10:57.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.238+0000 7f3438ff9640 1 -- 172.21.15.113:0/3577268092 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f343401c070 con 0x7f343c14e690 2024-04-25T00:10:57.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.239+0000 7f3438ff9640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f342806be30 0x7f342806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.239+0000 7f3438ff9640 1 -- 172.21.15.113:0/3577268092 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f343408a2e0 con 0x7f343c14e690 2024-04-25T00:10:57.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.239+0000 7f343bfff640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f342806be30 0x7f342806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.239+0000 7f343bfff640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f342806be30 0x7f342806e2f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f342c0058b0 tx=0x7f342c000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.243+0000 7f3438ff9640 1 -- 172.21.15.113:0/3577268092 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3434058cc0 con 0x7f343c14e690 2024-04-25T00:10:57.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.452+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f343c0731a0 con 0x7f343c14e690 2024-04-25T00:10:57.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.452+0000 7f3438ff9640 1 -- 172.21.15.113:0/3577268092 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7f34340585c0 con 0x7f343c14e690 2024-04-25T00:10:57.453 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:57.453 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:09:33.215001+0000","modified":"2024-04-25T00:09:50.995654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10794":{"gid":10794,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6836/2086115851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2086115851},{"type":"v1","addr":"172.21.15.113:6836","nonce":2086115851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10794,"qdb_cluster":[10794]},"id":13}]} 2024-04-25T00:10:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-04-25T00:10:57.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f342806be30 msgr2=0x7f342806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f342806be30 0x7f342806e2f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f342c0058b0 tx=0x7f342c000ed0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f343c14e690 msgr2=0x7f343c14eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f343c14e690 0x7f343c14eb40 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f343c06ed40 tx=0x7f343400fd40 comp rx=0 tx=0).stop 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 shutdown_connections 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f342806be30 0x7f342806e2f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f343c14f420 0x7f343c13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.454+0000 7f3442310640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f343c14e690 0x7f343c14eb40 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.455+0000 7f3442310640 1 --2- 172.21.15.113:0/3577268092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3424003f20 0x7f343c0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.455+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 >> 172.21.15.113:0/3577268092 conn(0x7f343c0b1a50 msgr2=0x7f343c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:57.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.455+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 shutdown_connections 2024-04-25T00:10:57.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.455+0000 7f3442310640 1 -- 172.21.15.113:0/3577268092 wait complete. 2024-04-25T00:10:57.466 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:10:57.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.562+0000 7f4ceaabd640 1 Processor -- start 2024-04-25T00:10:57.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.562+0000 7f4ceaabd640 1 -- start start 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.562+0000 7f4ceaabd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ce406e6e0 0x7f4ce40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.562+0000 7f4ceaabd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce41453c0 0x7f4ce4071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.562+0000 7f4ceaabd640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce40cfad0 con 0x7f4ce4144d60 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.562+0000 7f4ceaabd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce4145c40 con 0x7f4ce406e6e0 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.562+0000 7f4ceaabd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce40cf230 con 0x7f4ce41453c0 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ce406e6e0 0x7f4ce40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ce406e6e0 0x7f4ce40b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50508/0 (socket says 172.21.15.113:50508) 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 -- 172.21.15.113:0/2225332157 learned_addr learned my addr 172.21.15.113:0/2225332157 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce905a640 1 --2- 172.21.15.113:0/2225332157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce41453c0 0x7f4ce4071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 -- 172.21.15.113:0/2225332157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce41453c0 msgr2=0x7f4ce4071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 --2- 172.21.15.113:0/2225332157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce41453c0 0x7f4ce4071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 -- 172.21.15.113:0/2225332157 >> v1:172.21.15.178:6789/0 conn(0x7f4ce4144d60 legacy=0x7f4ce406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 -- 172.21.15.113:0/2225332157 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce40cf510 con 0x7f4ce406e6e0 2024-04-25T00:10:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.563+0000 7f4ce3fff640 1 --2- 172.21.15.113:0/2225332157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce406e6e0 0x7f4ce40b5390 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009870 tx=0x7f4cd402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebce42aeb5c4633 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ce37fe640 1 -- 172.21.15.113:0/2225332157 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4037070 con 0x7f4ce406e6e0 2024-04-25T00:10:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ce37fe640 1 -- 172.21.15.113:0/2225332157 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd402ae10 con 0x7f4ce406e6e0 2024-04-25T00:10:57.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ce37fe640 1 -- 172.21.15.113:0/2225332157 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4032660 con 0x7f4ce406e6e0 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 -- 172.21.15.113:0/2225332157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce406e6e0 msgr2=0x7f4ce40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/2225332157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce406e6e0 0x7f4ce40b5390 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009870 tx=0x7f4cd402a100 comp rx=0 tx=0).stop 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 -- 172.21.15.113:0/2225332157 shutdown_connections 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/2225332157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce41453c0 0x7f4ce4071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/2225332157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce406e6e0 0x7f4ce40b5390 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 -- 172.21.15.113:0/2225332157 >> 172.21.15.113:0/2225332157 conn(0x7f4ce40b1a50 msgr2=0x7f4ce40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 -- 172.21.15.113:0/2225332157 shutdown_connections 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.564+0000 7f4ceaabd640 1 -- 172.21.15.113:0/2225332157 wait complete. 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.565+0000 7f4ceaabd640 1 Processor -- start 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.565+0000 7f4ceaabd640 1 -- start start 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ceaabd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce406e6e0 0x7f4ce41ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ceaabd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce4144d60 0x7f4ce40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ceaabd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce41453c0 0x7f4ce40b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ceaabd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4145140 con 0x7f4ce4144d60 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ceaabd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4070e10 con 0x7f4ce41453c0 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ceaabd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce4071870 con 0x7f4ce406e6e0 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ce3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce4144d60 0x7f4ce40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ce3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce4144d60 0x7f4ce40b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50520/0 (socket says 172.21.15.113:50520) 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ce3fff640 1 -- 172.21.15.113:0/459856359 learned_addr learned my addr 172.21.15.113:0/459856359 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ce905a640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce41453c0 0x7f4ce40b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.566+0000 7f4ce8859640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce406e6e0 0x7f4ce41ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce3fff640 1 -- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce406e6e0 msgr2=0x7f4ce41ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce3fff640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce406e6e0 0x7f4ce41ef1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce3fff640 1 -- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce41453c0 msgr2=0x7f4ce40b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce3fff640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce41453c0 0x7f4ce40b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce3fff640 1 -- 172.21.15.113:0/459856359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce41ef700 con 0x7f4ce4144d60 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce3fff640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce4144d60 0x7f4ce40b05f0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009870 tx=0x7f4cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce17fa640 1 -- 172.21.15.113:0/459856359 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4041070 con 0x7f4ce4144d60 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce17fa640 1 -- 172.21.15.113:0/459856359 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd403c670 con 0x7f4ce4144d60 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce41f3db0 con 0x7f4ce4144d60 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ce17fa640 1 -- 172.21.15.113:0/459856359 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4037040 con 0x7f4ce4144d60 2024-04-25T00:10:57.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.567+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce41f4370 con 0x7f4ce4144d60 2024-04-25T00:10:57.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.568+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce40691a0 con 0x7f4ce4144d60 2024-04-25T00:10:57.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.568+0000 7f4ce17fa640 1 -- 172.21.15.113:0/459856359 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4cd4044070 con 0x7f4ce4144d60 2024-04-25T00:10:57.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.569+0000 7f4ce17fa640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc406be30 0x7f4cc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.569+0000 7f4ce17fa640 1 -- 172.21.15.113:0/459856359 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f4cd40ab580 con 0x7f4ce4144d60 2024-04-25T00:10:57.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.569+0000 7f4ce8859640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc406be30 0x7f4cc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.569+0000 7f4ce8859640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc406be30 0x7f4cc406e2f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc005660 tx=0x7f4cdc003d90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.573+0000 7f4ce17fa640 1 -- 172.21.15.113:0/459856359 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4cd4079980 con 0x7f4ce4144d60 2024-04-25T00:10:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.745+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ce40cd5a0 con 0x7f4ce4144d60 2024-04-25T00:10:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.746+0000 7f4ce17fa640 1 -- 172.21.15.113:0/459856359 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) ==== 74+0+16862 (secure 0 0 0) 0x7f4cd407dc80 con 0x7f4ce4144d60 2024-04-25T00:10:57.747 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:10:57.747 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":183,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:09:46.979993+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:09:31.412384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T00:09:32.173467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/3057967443":"2024-04-26T00:09:46.979970+0000","172.21.15.178:6833/3057967443":"2024-04-26T00:09:46.979970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:10:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc406be30 msgr2=0x7f4cc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc406be30 0x7f4cc406e2f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc005660 tx=0x7f4cdc003d90 comp rx=0 tx=0).stop 2024-04-25T00:10:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce4144d60 msgr2=0x7f4ce40b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce4144d60 0x7f4ce40b05f0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009870 tx=0x7f4cd40046c0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 shutdown_connections 2024-04-25T00:10:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f4cc406be30 0x7f4cc406e2f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce41453c0 0x7f4ce40b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce4144d60 0x7f4ce40b05f0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 --2- 172.21.15.113:0/459856359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ce406e6e0 0x7f4ce41ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 >> 172.21.15.113:0/459856359 conn(0x7f4ce40b1a50 msgr2=0x7f4ce40afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:57.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 shutdown_connections 2024-04-25T00:10:57.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.748+0000 7f4ceaabd640 1 -- 172.21.15.113:0/459856359 wait complete. 2024-04-25T00:10:57.759 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:10:57.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.856+0000 7f7d07d42640 1 Processor -- start 2024-04-25T00:10:57.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.856+0000 7f7d07d42640 1 -- start start 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d07d42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e2d0 0x7f7d0015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d07d42640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d000b3080 0x7f7d000b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d07d42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7d0015e6b0 0x7f7d000b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d07d42640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7d000cde00 con 0x7f7d000b3080 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d07d42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0015f4f0 con 0x7f7d0015e6b0 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d07d42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0015f7d0 con 0x7f7d0015e2d0 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d052dd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d000b3080 0x7f7d000b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d052dd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d000b3080 0x7f7d000b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50522/0 (socket says 172.21.15.113:50522) 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d052dd640 1 -- 172.21.15.113:0/121214784 learned_addr learned my addr 172.21.15.113:0/121214784 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d05ade640 1 --2- 172.21.15.113:0/121214784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e2d0 0x7f7d0015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.857+0000 7f7d062df640 1 --2- 172.21.15.113:0/121214784 >> v2:172.21.15.178:3300/0 conn(0x7f7d0015e6b0 0x7f7d000b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d052dd640 1 -- 172.21.15.113:0/121214784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e2d0 msgr2=0x7f7d0015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d052dd640 1 --2- 172.21.15.113:0/121214784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e2d0 0x7f7d0015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d052dd640 1 -- 172.21.15.113:0/121214784 >> v2:172.21.15.178:3300/0 conn(0x7f7d0015e6b0 msgr2=0x7f7d000b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d052dd640 1 --2- 172.21.15.113:0/121214784 >> v2:172.21.15.178:3300/0 conn(0x7f7d0015e6b0 0x7f7d000b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d052dd640 1 -- 172.21.15.113:0/121214784 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0015fab0 con 0x7f7d000b3080 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d052dd640 1 --2- 172.21.15.113:0/121214784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d000b3080 0x7f7d000b4330 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f7d0006f500 tx=0x7f7cf402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d17396b4b3bdb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d04adc640 1 -- 172.21.15.113:0/121214784 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4037070 con 0x7f7d000b3080 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d04adc640 1 -- 172.21.15.113:0/121214784 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf4032440 con 0x7f7d000b3080 2024-04-25T00:10:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.858+0000 7f7d04adc640 1 -- 172.21.15.113:0/121214784 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4032810 con 0x7f7d000b3080 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 -- 172.21.15.113:0/121214784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d000b3080 msgr2=0x7f7d000b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 --2- 172.21.15.113:0/121214784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d000b3080 0x7f7d000b4330 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f7d0006f500 tx=0x7f7cf402a140 comp rx=0 tx=0).stop 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 -- 172.21.15.113:0/121214784 shutdown_connections 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 --2- 172.21.15.113:0/121214784 >> v2:172.21.15.178:3300/0 conn(0x7f7d0015e6b0 0x7f7d000b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 --2- 172.21.15.113:0/121214784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d000b3080 0x7f7d000b4330 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 --2- 172.21.15.113:0/121214784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e2d0 0x7f7d0015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 -- 172.21.15.113:0/121214784 >> 172.21.15.113:0/121214784 conn(0x7f7d0006b0f0 msgr2=0x7f7d0006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 -- 172.21.15.113:0/121214784 shutdown_connections 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.859+0000 7f7d07d42640 1 -- 172.21.15.113:0/121214784 wait complete. 2024-04-25T00:10:57.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.860+0000 7f7d07d42640 1 Processor -- start 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.860+0000 7f7d07d42640 1 -- start start 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d07d42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d000b3080 0x7f7d001531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d07d42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d0015e2d0 0x7f7d00153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d07d42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e6b0 0x7f7d00148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d07d42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0015f2b0 con 0x7f7d0015e2d0 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d07d42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0015f130 con 0x7f7d000b3080 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d07d42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d00071040 con 0x7f7d0015e6b0 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d0015e2d0 0x7f7d00153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d0015e2d0 0x7f7d00153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50528/0 (socket says 172.21.15.113:50528) 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 -- 172.21.15.113:0/2498631667 learned_addr learned my addr 172.21.15.113:0/2498631667 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d062df640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e6b0 0x7f7d00148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d05ade640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d000b3080 0x7f7d001531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 -- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e6b0 msgr2=0x7f7d00148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e6b0 0x7f7d00148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 -- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d000b3080 msgr2=0x7f7d001531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d000b3080 0x7f7d001531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 -- 172.21.15.113:0/2498631667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d00143f00 con 0x7f7d0015e2d0 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d062df640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e6b0 0x7f7d00148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d05ade640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d000b3080 0x7f7d001531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.861+0000 7f7d052dd640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d0015e2d0 0x7f7d00153720 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402a180 tx=0x7f7cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.862+0000 7f7cfe7fc640 1 -- 172.21.15.113:0/2498631667 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4041020 con 0x7f7d0015e2d0 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.862+0000 7f7cfe7fc640 1 -- 172.21.15.113:0/2498631667 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf40306c0 con 0x7f7d0015e2d0 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.862+0000 7f7cfe7fc640 1 -- 172.21.15.113:0/2498631667 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4037040 con 0x7f7d0015e2d0 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.862+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d00147d80 con 0x7f7d0015e2d0 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.862+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d001542f0 con 0x7f7d0015e2d0 2024-04-25T00:10:57.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.863+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d000691a0 con 0x7f7d0015e2d0 2024-04-25T00:10:57.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.863+0000 7f7cfe7fc640 1 -- 172.21.15.113:0/2498631667 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cf4044020 con 0x7f7d0015e2d0 2024-04-25T00:10:57.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.863+0000 7f7cfe7fc640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cec06be60 0x7f7cec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:57.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.864+0000 7f7d05ade640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cec06be60 0x7f7cec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:57.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.864+0000 7f7cfe7fc640 1 -- 172.21.15.113:0/2498631667 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5795+0+0 (secure 0 0 0) 0x7f7cf40ab6c0 con 0x7f7d0015e2d0 2024-04-25T00:10:57.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.864+0000 7f7d05ade640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cec06be60 0x7f7cec06e320 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009b20 tx=0x7f7cf00039d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:57.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:57.868+0000 7f7cfe7fc640 1 -- 172.21.15.113:0/2498631667 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7cf404b030 con 0x7f7d0015e2d0 2024-04-25T00:10:58.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.077+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7d0015d540 con 0x7f7d0015e2d0 2024-04-25T00:10:58.077 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-04-25T00:10:58.077+0000 7f15dab1c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:10:58.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.078+0000 7f7cfe7fc640 1 -- 172.21.15.113:0/2498631667 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) ==== 161+0+0 (secure 0 0 0) 0x7f7d00143f00 con 0x7f7d0015e2d0 2024-04-25T00:10:58.079 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:10:58.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.081+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cec06be60 msgr2=0x7f7cec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:58.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.081+0000 7f7d07d42640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cec06be60 0x7f7cec06e320 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009b20 tx=0x7f7cf00039d0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.081+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d0015e2d0 msgr2=0x7f7d00153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:58.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.081+0000 7f7d07d42640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d0015e2d0 0x7f7d00153720 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402a180 tx=0x7f7cf40046c0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 shutdown_connections 2024-04-25T00:10:58.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7cec06be60 0x7f7cec06e320 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d0015e6b0 0x7f7d00148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d0015e2d0 0x7f7d00153720 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 --2- 172.21.15.113:0/2498631667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d000b3080 0x7f7d001531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 >> 172.21.15.113:0/2498631667 conn(0x7f7d0006b0f0 msgr2=0x7f7d000afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:58.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 shutdown_connections 2024-04-25T00:10:58.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.082+0000 7f7d07d42640 1 -- 172.21.15.113:0/2498631667 wait complete. 2024-04-25T00:10:58.087 INFO:tasks.ceph.mds.a.smithi113.stderr: -2381> 2024-04-25T00:09:54.119+0000 7f5bf2e55640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:10:58.098 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:10:58.114 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-04-25T00:10:58.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.200+0000 7f535d400640 1 Processor -- start 2024-04-25T00:10:58.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.200+0000 7f535d400640 1 -- start start 2024-04-25T00:10:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.200+0000 7f535d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f53580b5af0 0x7f53580b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.200+0000 7f535d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.200+0000 7f535d400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53580d2c90 con 0x7f535815e2d0 2024-04-25T00:10:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.200+0000 7f535d400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f53580d38a0 con 0x7f53580b5af0 2024-04-25T00:10:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.200+0000 7f535d400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53580b6470 con 0x7f53580b7280 2024-04-25T00:10:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f534ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f53580b5af0 0x7f53580b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f534ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f53580b5af0 0x7f53580b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50540/0 (socket says 172.21.15.113:50540) 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f534ffff640 1 -- 172.21.15.113:0/539005549 learned_addr learned my addr 172.21.15.113:0/539005549 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f53577fe640 1 --2- 172.21.15.113:0/539005549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f534ffff640 1 -- 172.21.15.113:0/539005549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 msgr2=0x7f53580b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f534ffff640 1 --2- 172.21.15.113:0/539005549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f534ffff640 1 -- 172.21.15.113:0/539005549 >> v1:172.21.15.178:6789/0 conn(0x7f535815e2d0 legacy=0x7f53580d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.201+0000 7f534ffff640 1 -- 172.21.15.113:0/539005549 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53580d2f70 con 0x7f53580b5af0 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f534ffff640 1 --2- 172.21.15.113:0/539005549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f53580b5f30 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f5340009870 tx=0x7f534002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b28f4584ee04898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f53567fc640 1 -- 172.21.15.113:0/539005549 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340037070 con 0x7f53580b5af0 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f53567fc640 1 -- 172.21.15.113:0/539005549 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5340032440 con 0x7f53580b5af0 2024-04-25T00:10:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f53567fc640 1 -- 172.21.15.113:0/539005549 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340032810 con 0x7f53580b5af0 2024-04-25T00:10:58.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 -- 172.21.15.113:0/539005549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 msgr2=0x7f53580b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:58.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 --2- 172.21.15.113:0/539005549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f53580b5f30 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f5340009870 tx=0x7f534002a2a0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 -- 172.21.15.113:0/539005549 shutdown_connections 2024-04-25T00:10:58.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 --2- 172.21.15.113:0/539005549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 --2- 172.21.15.113:0/539005549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f53580b5f30 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 -- 172.21.15.113:0/539005549 >> 172.21.15.113:0/539005549 conn(0x7f5358071d40 msgr2=0x7f53580d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:58.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 -- 172.21.15.113:0/539005549 shutdown_connections 2024-04-25T00:10:58.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.202+0000 7f535d400640 1 -- 172.21.15.113:0/539005549 wait complete. 2024-04-25T00:10:58.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 Processor -- start 2024-04-25T00:10:58.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 -- start start 2024-04-25T00:10:58.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f5358151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:58.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:58.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f535815e2d0 0x7f53580cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:58.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f53580d3d30 con 0x7f53580b5af0 2024-04-25T00:10:58.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53580d3530 con 0x7f535815e2d0 2024-04-25T00:10:58.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f535d400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53580d2950 con 0x7f53580b7280 2024-04-25T00:10:58.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f5356ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f5358151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:58.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f5356ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f5358151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50554/0 (socket says 172.21.15.113:50554) 2024-04-25T00:10:58.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.204+0000 7f5356ffd640 1 -- 172.21.15.113:0/2063737259 learned_addr learned my addr 172.21.15.113:0/2063737259 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:58.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f53577fe640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f535815e2d0 0x7f53580cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:58.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f534ffff640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:58.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f5356ffd640 1 -- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 msgr2=0x7f53580ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:58.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f5356ffd640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f5356ffd640 1 -- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f535815e2d0 msgr2=0x7f53580cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:58.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f5356ffd640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f535815e2d0 0x7f53580cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:58.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f5356ffd640 1 -- 172.21.15.113:0/2063737259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53580c9740 con 0x7f53580b5af0 2024-04-25T00:10:58.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f534ffff640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:58.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f5356ffd640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f5358151120 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f534800b3f0 tx=0x7f534800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:58.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f534f7fe640 1 -- 172.21.15.113:0/2063737259 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348015020 con 0x7f53580b5af0 2024-04-25T00:10:58.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f534f7fe640 1 -- 172.21.15.113:0/2063737259 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5348011830 con 0x7f53580b5af0 2024-04-25T00:10:58.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53580c9a20 con 0x7f53580b5af0 2024-04-25T00:10:58.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f534f7fe640 1 -- 172.21.15.113:0/2063737259 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348011c20 con 0x7f53580b5af0 2024-04-25T00:10:58.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.205+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53581531c0 con 0x7f53580b5af0 2024-04-25T00:10:58.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.206+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5358062b60 con 0x7f53580b5af0 2024-04-25T00:10:58.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.207+0000 7f534f7fe640 1 -- 172.21.15.113:0/2063737259 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5348018020 con 0x7f53580b5af0 2024-04-25T00:10:58.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.207+0000 7f534f7fe640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f533806be30 0x7f533806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:58.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.207+0000 7f534ffff640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f533806be30 0x7f533806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:58.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.207+0000 7f534f7fe640 1 -- 172.21.15.113:0/2063737259 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5881+0+0 (secure 0 0 0) 0x7f534808ab70 con 0x7f53580b5af0 2024-04-25T00:10:58.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.207+0000 7f534ffff640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f533806be30 0x7f533806e2f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5340034000 tx=0x7f534002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:58.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.211+0000 7f534f7fe640 1 -- 172.21.15.113:0/2063737259 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f534801f030 con 0x7f53580b5af0 2024-04-25T00:10:58.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:58.421+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f53580b8cd0 con 0x7f53580b5af0 2024-04-25T00:10:59.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.084+0000 7f534f7fe640 1 -- 172.21.15.113:0/2063737259 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v106) ==== 104+0+0 (secure 0 0 0) 0x7f53480593e0 con 0x7f53580b5af0 2024-04-25T00:10:59.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f533806be30 msgr2=0x7f533806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f533806be30 0x7f533806e2f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5340034000 tx=0x7f534002e000 comp rx=0 tx=0).stop 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 msgr2=0x7f5358151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f5358151120 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f534800b3f0 tx=0x7f534800b8b0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 shutdown_connections 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f533806be30 0x7f533806e2f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f535815e2d0 0x7f53580cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53580b7280 0x7f53580ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 --2- 172.21.15.113:0/2063737259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53580b5af0 0x7f5358151120 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 >> 172.21.15.113:0/2063737259 conn(0x7f5358071d40 msgr2=0x7f53580ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:59.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 shutdown_connections 2024-04-25T00:10:59.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.087+0000 7f535d400640 1 -- 172.21.15.113:0/2063737259 wait complete. 2024-04-25T00:10:59.102 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:10:59.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.248+0000 7f477a89e640 1 Processor -- start 2024-04-25T00:10:59.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.248+0000 7f477a89e640 1 -- start start 2024-04-25T00:10:59.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f477a89e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4774145890 0x7f47740ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f477a89e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f477a89e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f477406e610 0x7f477415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f477a89e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f47740cf180 con 0x7f4774145890 2024-04-25T00:10:59.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f477a89e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4774145c70 con 0x7f477406e610 2024-04-25T00:10:59.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f477a89e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f477406ea60 con 0x7f47740ced20 2024-04-25T00:10:59.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f4773fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4774145890 0x7f47740ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f4773fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4774145890 0x7f47740ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50556/0 (socket says 172.21.15.113:50556) 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.249+0000 7f4773fff640 1 -- 172.21.15.113:0/3508832434 learned_addr learned my addr 172.21.15.113:0/3508832434 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4773fff640 1 -- 172.21.15.113:0/3508832434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 msgr2=0x7f477406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4778e3b640 1 --2- 172.21.15.113:0/3508832434 >> v2:172.21.15.178:3300/0 conn(0x7f477406e610 0x7f477415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f47737fe640 1 --2- 172.21.15.113:0/3508832434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4773fff640 1 --2- 172.21.15.113:0/3508832434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4773fff640 1 -- 172.21.15.113:0/3508832434 >> v2:172.21.15.178:3300/0 conn(0x7f477406e610 msgr2=0x7f477415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4773fff640 1 --2- 172.21.15.113:0/3508832434 >> v2:172.21.15.178:3300/0 conn(0x7f477406e610 0x7f477415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4773fff640 1 -- 172.21.15.113:0/3508832434 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47740cf3a0 con 0x7f4774145890 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4778e3b640 1 --2- 172.21.15.113:0/3508832434 >> v2:172.21.15.178:3300/0 conn(0x7f477406e610 0x7f477415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f47737fe640 1 --2- 172.21.15.113:0/3508832434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:59.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4773fff640 1 --2- 172.21.15.113:0/3508832434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f47740ce7e0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f4774145ed0 tx=0x7f476402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495a5153ebbad01f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4772ffd640 1 -- 172.21.15.113:0/3508832434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764037070 con 0x7f4774145890 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.250+0000 7f4772ffd640 1 -- 172.21.15.113:0/3508832434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4764032440 con 0x7f4774145890 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f4772ffd640 1 -- 172.21.15.113:0/3508832434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764032810 con 0x7f4774145890 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 -- 172.21.15.113:0/3508832434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 msgr2=0x7f47740ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 --2- 172.21.15.113:0/3508832434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f47740ce7e0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f4774145ed0 tx=0x7f476402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 -- 172.21.15.113:0/3508832434 shutdown_connections 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 --2- 172.21.15.113:0/3508832434 >> v2:172.21.15.178:3300/0 conn(0x7f477406e610 0x7f477415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 --2- 172.21.15.113:0/3508832434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 --2- 172.21.15.113:0/3508832434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f47740ce7e0 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 -- 172.21.15.113:0/3508832434 >> 172.21.15.113:0/3508832434 conn(0x7f47740b1b30 msgr2=0x7f47740b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 -- 172.21.15.113:0/3508832434 shutdown_connections 2024-04-25T00:10:59.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.251+0000 7f477a89e640 1 -- 172.21.15.113:0/3508832434 wait complete. 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.252+0000 7f477a89e640 1 Processor -- start 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.252+0000 7f477a89e640 1 -- start start 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f477a89e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477406e610 0x7f477415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f477a89e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f477a89e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f477412fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f477a89e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4774070f10 con 0x7f4774145890 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f477a89e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47740cf960 con 0x7f477406e610 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f477a89e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47740b53b0 con 0x7f47740ced20 2024-04-25T00:10:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f4778e3b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f477412fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f4778e3b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f477412fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50564/0 (socket says 172.21.15.113:50564) 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f4778e3b640 1 -- 172.21.15.113:0/3289333792 learned_addr learned my addr 172.21.15.113:0/3289333792 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f47737fe640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.253+0000 7f4773fff640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477406e610 0x7f477415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4778e3b640 1 -- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 msgr2=0x7f477415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4778e3b640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4778e3b640 1 -- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477406e610 msgr2=0x7f477415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4778e3b640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477406e610 0x7f477415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4778e3b640 1 -- 172.21.15.113:0/3289333792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477415c6f0 con 0x7f4774145890 2024-04-25T00:10:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4773fff640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477406e610 0x7f477415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4778e3b640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f477412fec0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f476c00cef0 tx=0x7f476c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4770ff9640 1 -- 172.21.15.113:0/3289333792 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c01a070 con 0x7f4774145890 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4770ff9640 1 -- 172.21.15.113:0/3289333792 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476c00d860 con 0x7f4774145890 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477415bd80 con 0x7f4774145890 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f4770ff9640 1 -- 172.21.15.113:0/3289333792 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c00dc50 con 0x7f4774145890 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.254+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477415cc40 con 0x7f4774145890 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.255+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47740691a0 con 0x7f4774145890 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.255+0000 7f4770ff9640 1 -- 172.21.15.113:0/3289333792 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f476c01d070 con 0x7f4774145890 2024-04-25T00:10:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.256+0000 7f4770ff9640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f476006be30 0x7f476006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.256+0000 7f4770ff9640 1 -- 172.21.15.113:0/3289333792 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5881+0+0 (secure 0 0 0) 0x7f476c08b690 con 0x7f4774145890 2024-04-25T00:10:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.256+0000 7f4773fff640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f476006be30 0x7f476006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.256+0000 7f4773fff640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f476006be30 0x7f476006e2f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4764034000 tx=0x7f476402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.260+0000 7f4770ff9640 1 -- 172.21.15.113:0/3289333792 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f477415bd80 con 0x7f4774145890 2024-04-25T00:10:59.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:10:59.433+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f47740cc0c0 con 0x7f4774145890 2024-04-25T00:11:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.093+0000 7f4770ff9640 1 -- 172.21.15.113:0/3289333792 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v185) ==== 181+0+0 (secure 0 0 0) 0x7f476c059a90 con 0x7f4774145890 2024-04-25T00:11:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f476006be30 msgr2=0x7f476006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f476006be30 0x7f476006e2f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4764034000 tx=0x7f476402e000 comp rx=0 tx=0).stop 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 msgr2=0x7f477412fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f477412fec0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f476c00cef0 tx=0x7f476c00a710 comp rx=0 tx=0).stop 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 shutdown_connections 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f476006be30 0x7f476006e2f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4774145890 0x7f477412fec0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47740ced20 0x7f477415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 --2- 172.21.15.113:0/3289333792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477406e610 0x7f477415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.095+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 >> 172.21.15.113:0/3289333792 conn(0x7f47740b1b30 msgr2=0x7f4774144a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.097+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 shutdown_connections 2024-04-25T00:11:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.097+0000 7f477a89e640 1 -- 172.21.15.113:0/3289333792 wait complete. 2024-04-25T00:11:00.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:11:00.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.217+0000 7f829bc12640 1 Processor -- start 2024-04-25T00:11:00.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.217+0000 7f829bc12640 1 -- start start 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f829bc12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829415e1a0 0x7f8294162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f829bc12640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f829415e580 0x7f82940716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f829bc12640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f829406f240 0x7f8294071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f829bc12640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8294162da0 con 0x7f829415e580 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f829bc12640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f829415ea50 con 0x7f829406f240 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f829bc12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f829415ed30 con 0x7f829415e1a0 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82991ad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f829415e580 0x7f82940716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82991ad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f829415e580 0x7f82940716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50578/0 (socket says 172.21.15.113:50578) 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82991ad640 1 -- 172.21.15.113:0/105366398 learned_addr learned my addr 172.21.15.113:0/105366398 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82999ae640 1 --2- 172.21.15.113:0/105366398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829415e1a0 0x7f8294162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f829a1af640 1 --2- 172.21.15.113:0/105366398 >> v2:172.21.15.178:3300/0 conn(0x7f829406f240 0x7f8294071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82991ad640 1 -- 172.21.15.113:0/105366398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829415e1a0 msgr2=0x7f8294162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82991ad640 1 --2- 172.21.15.113:0/105366398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829415e1a0 0x7f8294162860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82991ad640 1 -- 172.21.15.113:0/105366398 >> v2:172.21.15.178:3300/0 conn(0x7f829406f240 msgr2=0x7f8294071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.218+0000 7f82991ad640 1 --2- 172.21.15.113:0/105366398 >> v2:172.21.15.178:3300/0 conn(0x7f829406f240 0x7f8294071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.219+0000 7f82991ad640 1 -- 172.21.15.113:0/105366398 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829415f010 con 0x7f829415e580 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.219+0000 7f82991ad640 1 --2- 172.21.15.113:0/105366398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f82940716a0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f8294072690 tx=0x7f828402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902117129861823b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.219+0000 7f82989ac640 1 -- 172.21.15.113:0/105366398 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284037070 con 0x7f829415e580 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.219+0000 7f82989ac640 1 -- 172.21.15.113:0/105366398 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8284032440 con 0x7f829415e580 2024-04-25T00:11:00.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.219+0000 7f82989ac640 1 -- 172.21.15.113:0/105366398 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284032810 con 0x7f829415e580 2024-04-25T00:11:00.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 -- 172.21.15.113:0/105366398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 msgr2=0x7f82940716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 --2- 172.21.15.113:0/105366398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f82940716a0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f8294072690 tx=0x7f828402a1e0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 -- 172.21.15.113:0/105366398 shutdown_connections 2024-04-25T00:11:00.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 --2- 172.21.15.113:0/105366398 >> v2:172.21.15.178:3300/0 conn(0x7f829406f240 0x7f8294071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 --2- 172.21.15.113:0/105366398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f82940716a0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 --2- 172.21.15.113:0/105366398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829415e1a0 0x7f8294162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 -- 172.21.15.113:0/105366398 >> 172.21.15.113:0/105366398 conn(0x7f829406fb20 msgr2=0x7f829406b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 -- 172.21.15.113:0/105366398 shutdown_connections 2024-04-25T00:11:00.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.220+0000 7f829bc12640 1 -- 172.21.15.113:0/105366398 wait complete. 2024-04-25T00:11:00.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.221+0000 7f829bc12640 1 Processor -- start 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.221+0000 7f829bc12640 1 -- start start 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829bc12640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829406f240 0x7f82941e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829bc12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f829415e1a0 0x7f8294152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829bc12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f8294159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829bc12640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f829406d790 con 0x7f829415e580 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829bc12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f829415f5d0 con 0x7f829415e1a0 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829bc12640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f829406d910 con 0x7f829406f240 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f8294159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f8294159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50594/0 (socket says 172.21.15.113:50594) 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 -- 172.21.15.113:0/3431913936 learned_addr learned my addr 172.21.15.113:0/3431913936 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f82999ae640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829406f240 0x7f82941e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f82991ad640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f829415e1a0 0x7f8294152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 -- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829406f240 msgr2=0x7f82941e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829406f240 0x7f82941e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 -- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f829415e1a0 msgr2=0x7f8294152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f829415e1a0 0x7f8294152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 -- 172.21.15.113:0/3431913936 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8294159960 con 0x7f829415e580 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f82999ae640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829406f240 0x7f82941e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f82991ad640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f829415e1a0 0x7f8294152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f829a1af640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f8294159420 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f828c00d830 tx=0x7f828c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f828a7fc640 1 -- 172.21.15.113:0/3431913936 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c00b6c0 con 0x7f829415e580 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.222+0000 7f828a7fc640 1 -- 172.21.15.113:0/3431913936 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c019070 con 0x7f829415e580 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.223+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82941e92c0 con 0x7f829415e580 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.223+0000 7f828a7fc640 1 -- 172.21.15.113:0/3431913936 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c00bac0 con 0x7f829415e580 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.223+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82941e9880 con 0x7f829415e580 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.223+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82940691a0 con 0x7f829415e580 2024-04-25T00:11:00.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.224+0000 7f828a7fc640 1 -- 172.21.15.113:0/3431913936 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f828c01e460 con 0x7f829415e580 2024-04-25T00:11:00.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.224+0000 7f828a7fc640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f827806be30 0x7f827806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:00.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.224+0000 7f828a7fc640 1 -- 172.21.15.113:0/3431913936 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5443+0+0 (secure 0 0 0) 0x7f828c08b400 con 0x7f829415e580 2024-04-25T00:11:00.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.224+0000 7f82999ae640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f827806be30 0x7f827806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:00.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.225+0000 7f82999ae640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f827806be30 0x7f827806e2f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8290009f70 tx=0x7f829000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:00.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.229+0000 7f828a7fc640 1 -- 172.21.15.113:0/3431913936 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f828c059eb0 con 0x7f829415e580 2024-04-25T00:11:00.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:00.400+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8294160cd0 con 0x7f829415e580 2024-04-25T00:11:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.094+0000 7f828a7fc640 1 -- 172.21.15.113:0/3431913936 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v186) ==== 169+0+0 (secure 0 0 0) 0x7f828c0597b0 con 0x7f829415e580 2024-04-25T00:11:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-25T00:11:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.100+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f827806be30 msgr2=0x7f827806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.100+0000 7f829bc12640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f827806be30 0x7f827806e2f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8290009f70 tx=0x7f829000b040 comp rx=0 tx=0).stop 2024-04-25T00:11:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.100+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 msgr2=0x7f8294159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.100+0000 7f829bc12640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f8294159420 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f828c00d830 tx=0x7f828c00dcf0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.101+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 shutdown_connections 2024-04-25T00:11:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.101+0000 7f829bc12640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f827806be30 0x7f827806e2f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.101+0000 7f829bc12640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f829415e1a0 0x7f8294152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.101+0000 7f829bc12640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f829415e580 0x7f8294159420 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.101+0000 7f829bc12640 1 --2- 172.21.15.113:0/3431913936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f829406f240 0x7f82941e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.101+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 >> 172.21.15.113:0/3431913936 conn(0x7f829406fb20 msgr2=0x7f82941614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:01.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.104+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 shutdown_connections 2024-04-25T00:11:01.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.105+0000 7f829bc12640 1 -- 172.21.15.113:0/3431913936 wait complete. 2024-04-25T00:11:01.118 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:11:01.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.218+0000 7f1f13064640 1 Processor -- start 2024-04-25T00:11:01.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.218+0000 7f1f13064640 1 -- start start 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f13064640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f0c15e2d0 0x7f1f0c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f13064640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c15f4f0 con 0x7f1f0c0b4a40 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f13064640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c15f7d0 con 0x7f1f0c0b3080 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f13064640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1f0c0cde00 con 0x7f1f0c15e2d0 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f11601640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1f0c0b4a40 0x7f1f0c0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40908/0 (socket says 172.21.15.113:40908) 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f11601640 1 -- 172.21.15.113:0/3705988597 learned_addr learned my addr 172.21.15.113:0/3705988597 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f10e00640 1 --2- 172.21.15.113:0/3705988597 >> v2:172.21.15.178:3300/0 conn(0x7f1f0c15e2d0 0x7f1f0c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.219+0000 7f1f0b7fe640 1 -- 172.21.15.113:0/3705988597 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198876181 0 0) 0x7f1f0c15f4f0 con 0x7f1f0c0b4a40 2024-04-25T00:11:01.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.220+0000 7f1f0b7fe640 1 -- 172.21.15.113:0/3705988597 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1efc003520 con 0x7f1f0c0b4a40 2024-04-25T00:11:01.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.220+0000 7f1f10e00640 1 -- 172.21.15.113:0/3705988597 >> v1:172.21.15.178:6789/0 conn(0x7f1f0c0b3080 legacy=0x7f1f0c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.220+0000 7f1f10e00640 1 -- 172.21.15.113:0/3705988597 >> v1:172.21.15.113:6789/0 conn(0x7f1f0c0b4a40 legacy=0x7f1f0c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.220+0000 7f1f10e00640 1 -- 172.21.15.113:0/3705988597 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c15fab0 con 0x7f1f0c15e2d0 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.220+0000 7f1f10e00640 1 --2- 172.21.15.113:0/3705988597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c15e2d0 0x7f1f0c15ebf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0cdf80 tx=0x7f1f0002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65699c00db5d09e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f0b7fe640 1 -- 172.21.15.113:0/3705988597 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00007740 con 0x7f1f0c15e2d0 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f0b7fe640 1 -- 172.21.15.113:0/3705988597 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f000078e0 con 0x7f1f0c15e2d0 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f0b7fe640 1 -- 172.21.15.113:0/3705988597 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00007c90 con 0x7f1f0c15e2d0 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f13064640 1 -- 172.21.15.113:0/3705988597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c15e2d0 msgr2=0x7f1f0c15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f13064640 1 --2- 172.21.15.113:0/3705988597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c15e2d0 0x7f1f0c15ebf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0cdf80 tx=0x7f1f0002a6f0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f13064640 1 -- 172.21.15.113:0/3705988597 shutdown_connections 2024-04-25T00:11:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f13064640 1 --2- 172.21.15.113:0/3705988597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c15e2d0 0x7f1f0c15ebf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f13064640 1 -- 172.21.15.113:0/3705988597 >> 172.21.15.113:0/3705988597 conn(0x7f1f0c06b0f0 msgr2=0x7f1f0c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:01.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f13064640 1 -- 172.21.15.113:0/3705988597 shutdown_connections 2024-04-25T00:11:01.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.221+0000 7f1f13064640 1 -- 172.21.15.113:0/3705988597 wait complete. 2024-04-25T00:11:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.222+0000 7f1f13064640 1 Processor -- start 2024-04-25T00:11:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.222+0000 7f1f13064640 1 -- start start 2024-04-25T00:11:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f13064640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c0b3080 0x7f1f0c13e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f13064640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c0b4a40 0x7f1f0c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f13064640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f0c15e2d0 0x7f1f0c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f13064640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c15f2b0 con 0x7f1f0c0b3080 2024-04-25T00:11:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f13064640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c15f130 con 0x7f1f0c0b4a40 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f13064640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c15e6b0 con 0x7f1f0c15e2d0 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c0b3080 0x7f1f0c13e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c0b3080 0x7f1f0c13e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50604/0 (socket says 172.21.15.113:50604) 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.223+0000 7f1f10e00640 1 -- 172.21.15.113:0/214733420 learned_addr learned my addr 172.21.15.113:0/214733420 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f11601640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f0c15e2d0 0x7f1f0c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f0bfff640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c0b4a40 0x7f1f0c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f10e00640 1 -- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f0c15e2d0 msgr2=0x7f1f0c0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f10e00640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f0c15e2d0 0x7f1f0c0b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f10e00640 1 -- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c0b4a40 msgr2=0x7f1f0c0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f10e00640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c0b4a40 0x7f1f0c0b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f10e00640 1 -- 172.21.15.113:0/214733420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c140880 con 0x7f1f0c0b3080 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f0bfff640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c0b4a40 0x7f1f0c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f10e00640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c0b3080 0x7f1f0c13e1a0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009e80 tx=0x7f1f00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f097fa640 1 -- 172.21.15.113:0/214733420 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00009450 con 0x7f1f0c0b3080 2024-04-25T00:11:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f097fa640 1 -- 172.21.15.113:0/214733420 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f00004240 con 0x7f1f0c0b3080 2024-04-25T00:11:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c13f7e0 con 0x7f1f0c0b3080 2024-04-25T00:11:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f097fa640 1 -- 172.21.15.113:0/214733420 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000045f0 con 0x7f1f0c0b3080 2024-04-25T00:11:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.224+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c13fda0 con 0x7f1f0c0b3080 2024-04-25T00:11:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.225+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c0691a0 con 0x7f1f0c0b3080 2024-04-25T00:11:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.225+0000 7f1f097fa640 1 -- 172.21.15.113:0/214733420 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f0003b410 con 0x7f1f0c0b3080 2024-04-25T00:11:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.226+0000 7f1f097fa640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ef006be30 0x7f1ef006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.226+0000 7f1f097fa640 1 -- 172.21.15.113:0/214733420 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f1f000ac210 con 0x7f1f0c0b3080 2024-04-25T00:11:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.226+0000 7f1f0bfff640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ef006be30 0x7f1ef006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.226+0000 7f1f0bfff640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ef006be30 0x7f1ef006e2f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1f43f0 tx=0x7f1ef8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.230+0000 7f1f097fa640 1 -- 172.21.15.113:0/214733420 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1f0003f030 con 0x7f1f0c0b3080 2024-04-25T00:11:01.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.409+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1f0c0af750 con 0x7f1f0c0b3080 2024-04-25T00:11:01.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.413+0000 7f1f097fa640 1 -- 172.21.15.113:0/214733420 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7f1f0007aef0 con 0x7f1f0c0b3080 2024-04-25T00:11:01.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ef006be30 msgr2=0x7f1ef006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ef006be30 0x7f1ef006e2f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1f43f0 tx=0x7f1ef8009040 comp rx=0 tx=0).stop 2024-04-25T00:11:01.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c0b3080 msgr2=0x7f1f0c13e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c0b3080 0x7f1f0c13e1a0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009e80 tx=0x7f1f00004a40 comp rx=0 tx=0).stop 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 shutdown_connections 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f1ef006be30 0x7f1ef006e2f0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c0b4a40 0x7f1f0c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f0c15e2d0 0x7f1f0c0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 --2- 172.21.15.113:0/214733420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c0b3080 0x7f1f0c13e1a0 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 >> 172.21.15.113:0/214733420 conn(0x7f1f0c06b0f0 msgr2=0x7f1f0c0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 shutdown_connections 2024-04-25T00:11:01.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.415+0000 7f1f13064640 1 -- 172.21.15.113:0/214733420 wait complete. 2024-04-25T00:11:01.426 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-25T00:11:01.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.524+0000 7fae3adee640 1 Processor -- start 2024-04-25T00:11:01.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.524+0000 7fae3adee640 1 -- start start 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae3adee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae340d06c0 0x7fae3415ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae3adee640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fae341604d0 0x7fae340cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae3adee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae3415d6b0 con 0x7fae3413f050 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae3adee640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fae3415c6a0 con 0x7fae341604d0 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae3adee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae3415c8f0 con 0x7fae340d06c0 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae38b8a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fae3413f050 0x7fae340cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40912/0 (socket says 172.21.15.113:40912) 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae38b8a640 1 -- 172.21.15.113:0/2333263166 learned_addr learned my addr 172.21.15.113:0/2333263166 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452049012 0 0) 0x7fae3415d6b0 con 0x7fae3413f050 2024-04-25T00:11:01.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae24003520 con 0x7fae3413f050 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae3938b640 1 --2- 172.21.15.113:0/2333263166 >> v2:172.21.15.178:3300/0 conn(0x7fae341604d0 0x7fae340cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae33fff640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae340d06c0 0x7fae3415ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3354856295 0 0) 0x7fae24003520 con 0x7fae3413f050 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae340d06c0 msgr2=0x7fae3415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae340d06c0 0x7fae3415ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 >> v2:172.21.15.178:3300/0 conn(0x7fae341604d0 msgr2=0x7fae340cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 --2- 172.21.15.113:0/2333263166 >> v2:172.21.15.178:3300/0 conn(0x7fae341604d0 0x7fae340cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3415cbd0 con 0x7fae3413f050 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fae28001670 con 0x7fae3413f050 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae3938b640 1 --2- 172.21.15.113:0/2333263166 >> v2:172.21.15.178:3300/0 conn(0x7fae341604d0 0x7fae340cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 >> v1:172.21.15.113:6789/0 conn(0x7fae3413f050 legacy=0x7fae340cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae240039b0 0x7fae24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae24004800 0x7fae24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae240051c0 0x7fae2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fae3415c6a0 con 0x7fae24004800 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae3415c8f0 con 0x7fae240039b0 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae2400c090 con 0x7fae240051c0 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.525+0000 7fae33fff640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae340d06c0 0x7fae3415ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae33fff640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae24004800 0x7fae24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae38b8a640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae240039b0 0x7fae24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae3938b640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae240051c0 0x7fae2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae33fff640 1 -- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae240051c0 msgr2=0x7fae2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae33fff640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae240051c0 0x7fae2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae33fff640 1 -- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae240039b0 msgr2=0x7fae24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae33fff640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae240039b0 0x7fae24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae33fff640 1 -- 172.21.15.113:0/2333263166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae24003520 con 0x7fae24004800 2024-04-25T00:11:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.526+0000 7fae33fff640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae24004800 0x7fae24004c80 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fae340afc30 tx=0x7fae24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a060227f912a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20037070 con 0x7fae24004800 2024-04-25T00:11:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2002aae0 con 0x7fae24004800 2024-04-25T00:11:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae337fe640 1 -- 172.21.15.113:0/2333263166 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000f040 con 0x7fae24004800 2024-04-25T00:11:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae3adee640 1 -- 172.21.15.113:0/2333263166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae24004800 msgr2=0x7fae24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae3adee640 1 --2- 172.21.15.113:0/2333263166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae24004800 0x7fae24004c80 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fae340afc30 tx=0x7fae24005830 comp rx=0 tx=0).stop 2024-04-25T00:11:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae3938b640 1 -- 172.21.15.113:0/2333263166 reap_dead start 2024-04-25T00:11:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae3adee640 1 -- 172.21.15.113:0/2333263166 shutdown_connections 2024-04-25T00:11:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae3adee640 1 -- 172.21.15.113:0/2333263166 >> 172.21.15.113:0/2333263166 conn(0x7fae3406b0f0 msgr2=0x7fae3406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae3adee640 1 -- 172.21.15.113:0/2333263166 shutdown_connections 2024-04-25T00:11:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.527+0000 7fae3adee640 1 -- 172.21.15.113:0/2333263166 wait complete. 2024-04-25T00:11:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.528+0000 7fae3adee640 1 Processor -- start 2024-04-25T00:11:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.528+0000 7fae3adee640 1 -- start start 2024-04-25T00:11:01.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae3adee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae24004800 0x7fae341f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae3adee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3414a9d0 0x7fae340b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae3adee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae340b2550 0x7fae3415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae3adee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fae3413f430 con 0x7fae3414a9d0 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae3adee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae3415d190 con 0x7fae340b2550 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae3adee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae341608d0 con 0x7fae24004800 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae33fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3414a9d0 0x7fae340b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae33fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3414a9d0 0x7fae340b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50608/0 (socket says 172.21.15.113:50608) 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.529+0000 7fae33fff640 1 -- 172.21.15.113:0/2673514045 learned_addr learned my addr 172.21.15.113:0/2673514045 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae33fff640 1 -- 172.21.15.113:0/2673514045 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae24004800 msgr2=0x7fae341f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae33fff640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae24004800 0x7fae341f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae33fff640 1 -- 172.21.15.113:0/2673514045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae340b2550 msgr2=0x7fae3415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae33fff640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae340b2550 0x7fae3415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae33fff640 1 -- 172.21.15.113:0/2673514045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae340b2a10 con 0x7fae3414a9d0 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae33fff640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3414a9d0 0x7fae340b2010 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fae340d0b10 tx=0x7fae20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae317fa640 1 -- 172.21.15.113:0/2673514045 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20040020 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae317fa640 1 -- 172.21.15.113:0/2673514045 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae20038040 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae34071040 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae317fa640 1 -- 172.21.15.113:0/2673514045 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20037070 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.530+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae34071600 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.531+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae340691a0 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.531+0000 7fae317fa640 1 -- 172.21.15.113:0/2673514045 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae20043020 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.532+0000 7fae317fa640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae1806be30 0x7fae1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.532+0000 7fae317fa640 1 -- 172.21.15.113:0/2673514045 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fae200aaa20 con 0x7fae3414a9d0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.532+0000 7fae38b8a640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae1806be30 0x7fae1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.532+0000 7fae38b8a640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae1806be30 0x7fae1806e2f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fae28001490 tx=0x7fae28007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.536+0000 7fae317fa640 1 -- 172.21.15.113:0/2673514045 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fae2004a030 con 0x7fae3414a9d0 2024-04-25T00:11:01.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.709+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) -- 0x7fae34156b30 con 0x7fae3414a9d0 2024-04-25T00:11:01.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.712+0000 7fae317fa640 1 -- 172.21.15.113:0/2673514045 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fae20079700 con 0x7fae3414a9d0 2024-04-25T00:11:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.714+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae1806be30 msgr2=0x7fae1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.714+0000 7fae3adee640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae1806be30 0x7fae1806e2f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fae28001490 tx=0x7fae28007c10 comp rx=0 tx=0).stop 2024-04-25T00:11:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.714+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3414a9d0 msgr2=0x7fae340b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.714+0000 7fae3adee640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3414a9d0 0x7fae340b2010 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fae340d0b10 tx=0x7fae20004730 comp rx=0 tx=0).stop 2024-04-25T00:11:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.714+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 shutdown_connections 2024-04-25T00:11:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.715+0000 7fae3adee640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fae1806be30 0x7fae1806e2f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.715+0000 7fae3adee640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae340b2550 0x7fae3415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.715+0000 7fae3adee640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3414a9d0 0x7fae340b2010 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.715+0000 7fae3adee640 1 --2- 172.21.15.113:0/2673514045 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae24004800 0x7fae341f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.715+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 >> 172.21.15.113:0/2673514045 conn(0x7fae3406b0f0 msgr2=0x7fae34151b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.715+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 shutdown_connections 2024-04-25T00:11:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.715+0000 7fae3adee640 1 -- 172.21.15.113:0/2673514045 wait complete. 2024-04-25T00:11:01.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T00:11:01.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.822+0000 7f7eabbf4640 1 Processor -- start 2024-04-25T00:11:01.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.822+0000 7f7eabbf4640 1 -- start start 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7eabbf4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7ea40d2290 0x7f7ea40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7eabbf4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ea40d0b00 0x7f7ea40d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7eabbf4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea40d1480 con 0x7f7ea415e320 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7eabbf4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea40d3410 con 0x7f7ea40d0b00 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7eabbf4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea40d2800 con 0x7f7ea40d2290 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7eaa191640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ea40d0b00 0x7f7ea40d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7ea9990640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7ea415e320 0x7f7ea40d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40920/0 (socket says 172.21.15.113:40920) 2024-04-25T00:11:01.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.823+0000 7f7ea9990640 1 -- 172.21.15.113:0/2867613318 learned_addr learned my addr 172.21.15.113:0/2867613318 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:01.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7ea918f640 1 --2- 172.21.15.113:0/2867613318 >> v2:172.21.15.178:3300/0 conn(0x7f7ea40d2290 0x7f7ea40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7eaa191640 1 -- 172.21.15.113:0/2867613318 >> v2:172.21.15.178:3300/0 conn(0x7f7ea40d2290 msgr2=0x7f7ea40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:01.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7eaa191640 1 --2- 172.21.15.113:0/2867613318 >> v2:172.21.15.178:3300/0 conn(0x7f7ea40d2290 0x7f7ea40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7eaa191640 1 -- 172.21.15.113:0/2867613318 >> v1:172.21.15.113:6789/0 conn(0x7f7ea415e320 legacy=0x7f7ea40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7eaa191640 1 -- 172.21.15.113:0/2867613318 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40d2ae0 con 0x7f7ea40d0b00 2024-04-25T00:11:01.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7ea918f640 1 --2- 172.21.15.113:0/2867613318 >> v2:172.21.15.178:3300/0 conn(0x7f7ea40d2290 0x7f7ea40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:01.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7eaa191640 1 --2- 172.21.15.113:0/2867613318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea40d0f40 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4072810 tx=0x7f7e9c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf9322f50ba4155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7ea898e640 1 -- 172.21.15.113:0/2867613318 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c037070 con 0x7f7ea40d0b00 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.824+0000 7f7ea898e640 1 -- 172.21.15.113:0/2867613318 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c032440 con 0x7f7ea40d0b00 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7ea898e640 1 -- 172.21.15.113:0/2867613318 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c032810 con 0x7f7ea40d0b00 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2867613318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 msgr2=0x7f7ea40d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2867613318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea40d0f40 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4072810 tx=0x7f7e9c02a230 comp rx=0 tx=0).stop 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2867613318 shutdown_connections 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2867613318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea40d0f40 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2867613318 >> v2:172.21.15.178:3300/0 conn(0x7f7ea40d2290 0x7f7ea40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2867613318 >> 172.21.15.113:0/2867613318 conn(0x7f7ea40b5c20 msgr2=0x7f7ea40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2867613318 shutdown_connections 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.825+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2867613318 wait complete. 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.826+0000 7f7eabbf4640 1 Processor -- start 2024-04-25T00:11:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.826+0000 7f7eabbf4640 1 -- start start 2024-04-25T00:11:01.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7eabbf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea41443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7eabbf4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ea40d2290 0x7f7ea4146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7eabbf4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea415e320 0x7f7ea41542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7eabbf4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40b9fe0 con 0x7f7ea40d0b00 2024-04-25T00:11:01.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7eabbf4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40d30a0 con 0x7f7ea415e320 2024-04-25T00:11:01.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7eabbf4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea415e700 con 0x7f7ea40d2290 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea41443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea41443d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50618/0 (socket says 172.21.15.113:50618) 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 -- 172.21.15.113:0/2122729869 learned_addr learned my addr 172.21.15.113:0/2122729869 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7eaa191640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea415e320 0x7f7ea41542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 -- 172.21.15.113:0/2122729869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ea40d2290 msgr2=0x7f7ea4146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ea40d2290 0x7f7ea4146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 -- 172.21.15.113:0/2122729869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea415e320 msgr2=0x7f7ea41542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea415e320 0x7f7ea41542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 -- 172.21.15.113:0/2122729869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea415eea0 con 0x7f7ea40d0b00 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.827+0000 7f7ea9990640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea41443d0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000cb10 tx=0x7f7ea000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.828+0000 7f7e9a7fc640 1 -- 172.21.15.113:0/2122729869 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0017070 con 0x7f7ea40d0b00 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.828+0000 7f7e9a7fc640 1 -- 172.21.15.113:0/2122729869 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea000ac70 con 0x7f7ea40d0b00 2024-04-25T00:11:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.828+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea415f180 con 0x7f7ea40d0b00 2024-04-25T00:11:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.828+0000 7f7e9a7fc640 1 -- 172.21.15.113:0/2122729869 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0012620 con 0x7f7ea40d0b00 2024-04-25T00:11:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.828+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea40cc6b0 con 0x7f7ea40d0b00 2024-04-25T00:11:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.829+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea40691a0 con 0x7f7ea40d0b00 2024-04-25T00:11:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.829+0000 7f7e9a7fc640 1 -- 172.21.15.113:0/2122729869 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ea001a070 con 0x7f7ea40d0b00 2024-04-25T00:11:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.829+0000 7f7e9a7fc640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7e8806be30 0x7f7e8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.830+0000 7f7e9a7fc640 1 -- 172.21.15.113:0/2122729869 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f7ea008aa40 con 0x7f7ea40d0b00 2024-04-25T00:11:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.830+0000 7f7ea918f640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7e8806be30 0x7f7e8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.830+0000 7f7ea918f640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7e8806be30 0x7f7e8806e2f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7e94006fd0 tx=0x7f7e94008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:01.834+0000 7f7e9a7fc640 1 -- 172.21.15.113:0/2122729869 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ea0059690 con 0x7f7ea40d0b00 2024-04-25T00:11:02.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.074+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) -- 0x7f7ea40b41f0 con 0x7f7ea40d0b00 2024-04-25T00:11:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.104+0000 7f7e9a7fc640 1 -- 172.21.15.113:0/2122729869 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v608) ==== 133+0+0 (secure 0 0 0) 0x7f7ea008ae20 con 0x7f7ea40d0b00 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7e8806be30 msgr2=0x7f7e8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7e8806be30 0x7f7e8806e2f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7e94006fd0 tx=0x7f7e94008040 comp rx=0 tx=0).stop 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 msgr2=0x7f7ea41443d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea41443d0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000cb10 tx=0x7f7ea000a060 comp rx=0 tx=0).stop 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 shutdown_connections 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7e8806be30 0x7f7e8806e2f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ea415e320 0x7f7ea41542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ea40d2290 0x7f7ea4146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 --2- 172.21.15.113:0/2122729869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ea40d0b00 0x7f7ea41443d0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 >> 172.21.15.113:0/2122729869 conn(0x7f7ea40b5c20 msgr2=0x7f7ea40b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 shutdown_connections 2024-04-25T00:11:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:02.106+0000 7f7eabbf4640 1 -- 172.21.15.113:0/2122729869 wait complete. 2024-04-25T00:11:02.117 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T00:11:02.118 INFO:tasks.cephfs_test_runner: 2024-04-25T00:11:02.118 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T00:11:02.118 INFO:tasks.cephfs_test_runner:Ran 12 tests in 734.819s 2024-04-25T00:11:02.118 INFO:tasks.cephfs_test_runner: 2024-04-25T00:11:02.118 INFO:tasks.cephfs_test_runner:OK 2024-04-25T00:11:02.118 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T00:11:02.138 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T00:11:02.150 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T00:11:02.150 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:11:02.150 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:02.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:11:02.166 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-25T00:11:02.166 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:02.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15967 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T00:11:02.222 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:11:02.222 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:02.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:11:02.242 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-25T00:11:02.242 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:11:02.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056086,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T00:11:02.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.299 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:11:02.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:11:02.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:11:02.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.315 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:11:02.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.378 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:11:02.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.443 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-04-25T00:11:02.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.554 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-25T00:11:02.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.569 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-04-25T00:11:02.636 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-04-25T00:11:02.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:11:02.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.638 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-25T00:11:02.655 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:11:02.655 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:11:02.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.656 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:11:02.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.724 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:11:02.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.791 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-25T00:11:02.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.882 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-25T00:11:02.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:11:02.897 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-25T00:11:02.964 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2024-04-25T00:11:02.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:11:02.966 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T00:11:03.010 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:11:03.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:03.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.109+0000 7ff4b92da640 1 Processor -- start 2024-04-25T00:11:03.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.109+0000 7ff4b92da640 1 -- start start 2024-04-25T00:11:03.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b92da640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4b415e320 0x7ff4b40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b92da640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d2290 0x7ff4b40b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b92da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4b40d0b00 0x7ff4b40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b92da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b40d14d0 con 0x7ff4b40d0b00 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b92da640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b40d3410 con 0x7ff4b415e320 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b92da640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b40d3de0 con 0x7ff4b40d2290 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4b40d0b00 0x7ff4b40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4b40d0b00 0x7ff4b40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50634/0 (socket says 172.21.15.113:50634) 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 -- 172.21.15.113:0/38689803 learned_addr learned my addr 172.21.15.113:0/38689803 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b2ffd640 1 --2- 172.21.15.113:0/38689803 >> v2:172.21.15.178:3300/0 conn(0x7ff4b415e320 0x7ff4b40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b27fc640 1 --2- 172.21.15.113:0/38689803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d2290 0x7ff4b40b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 -- 172.21.15.113:0/38689803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d2290 msgr2=0x7ff4b40b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 --2- 172.21.15.113:0/38689803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d2290 0x7ff4b40b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 -- 172.21.15.113:0/38689803 >> v2:172.21.15.178:3300/0 conn(0x7ff4b415e320 msgr2=0x7ff4b40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 --2- 172.21.15.113:0/38689803 >> v2:172.21.15.178:3300/0 conn(0x7ff4b415e320 0x7ff4b40d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.110+0000 7ff4b37fe640 1 -- 172.21.15.113:0/38689803 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b40d2800 con 0x7ff4b40d0b00 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.111+0000 7ff4b37fe640 1 --2- 172.21.15.113:0/38689803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b40d0b00 0x7ff4b40d0f90 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40b6d00 tx=0x7ff4a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef0a7943751e03a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.111+0000 7ff4b1ffb640 1 -- 172.21.15.113:0/38689803 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4037070 con 0x7ff4b40d0b00 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.111+0000 7ff4b1ffb640 1 -- 172.21.15.113:0/38689803 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a4032440 con 0x7ff4b40d0b00 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.111+0000 7ff4b1ffb640 1 -- 172.21.15.113:0/38689803 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4032810 con 0x7ff4b40d0b00 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 -- 172.21.15.113:0/38689803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b40d0b00 msgr2=0x7ff4b40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 --2- 172.21.15.113:0/38689803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b40d0b00 0x7ff4b40d0f90 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40b6d00 tx=0x7ff4a402a270 comp rx=0 tx=0).stop 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 -- 172.21.15.113:0/38689803 shutdown_connections 2024-04-25T00:11:03.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 --2- 172.21.15.113:0/38689803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b40d0b00 0x7ff4b40d0f90 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 --2- 172.21.15.113:0/38689803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d2290 0x7ff4b40b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 --2- 172.21.15.113:0/38689803 >> v2:172.21.15.178:3300/0 conn(0x7ff4b415e320 0x7ff4b40d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 -- 172.21.15.113:0/38689803 >> 172.21.15.113:0/38689803 conn(0x7ff4b40b5c20 msgr2=0x7ff4b40b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 -- 172.21.15.113:0/38689803 shutdown_connections 2024-04-25T00:11:03.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.112+0000 7ff4b92da640 1 -- 172.21.15.113:0/38689803 wait complete. 2024-04-25T00:11:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.113+0000 7ff4b92da640 1 Processor -- start 2024-04-25T00:11:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.113+0000 7ff4b92da640 1 -- start start 2024-04-25T00:11:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b92da640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d0b00 0x7ff4b41533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b92da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40d2290 0x7ff4b41613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b92da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b415e320 0x7ff4b4155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b92da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b40d30a0 con 0x7ff4b415e320 2024-04-25T00:11:03.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b92da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b40d2dc0 con 0x7ff4b40d2290 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b92da640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b40b9fe0 con 0x7ff4b40d0b00 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b415e320 0x7ff4b4155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b415e320 0x7ff4b4155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50638/0 (socket says 172.21.15.113:50638) 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 -- 172.21.15.113:0/2150897580 learned_addr learned my addr 172.21.15.113:0/2150897580 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b27fc640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40d2290 0x7ff4b41613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b2ffd640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d0b00 0x7ff4b41533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 -- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d0b00 msgr2=0x7ff4b41533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d0b00 0x7ff4b41533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 -- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40d2290 msgr2=0x7ff4b41613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40d2290 0x7ff4b41613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 -- 172.21.15.113:0/2150897580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b41574b0 con 0x7ff4b415e320 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.114+0000 7ff4b37fe640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b415e320 0x7ff4b4155c90 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40090b0 tx=0x7ff4a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.115+0000 7ff4ab7fe640 1 -- 172.21.15.113:0/2150897580 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4037040 con 0x7ff4b415e320 2024-04-25T00:11:03.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.115+0000 7ff4ab7fe640 1 -- 172.21.15.113:0/2150897580 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a4042070 con 0x7ff4b415e320 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.115+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b4156c10 con 0x7ff4b415e320 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.115+0000 7ff4ab7fe640 1 -- 172.21.15.113:0/2150897580 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4030920 con 0x7ff4b415e320 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.115+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41571d0 con 0x7ff4b415e320 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.116+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b40691a0 con 0x7ff4b415e320 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.116+0000 7ff4ab7fe640 1 -- 172.21.15.113:0/2150897580 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4a4030ac0 con 0x7ff4b415e320 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.116+0000 7ff4ab7fe640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff49406be30 0x7ff49406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.117+0000 7ff4ab7fe640 1 -- 172.21.15.113:0/2150897580 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7ff4a40ab3c0 con 0x7ff4b415e320 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.117+0000 7ff4b2ffd640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff49406be30 0x7ff49406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.117+0000 7ff4b2ffd640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff49406be30 0x7ff49406e2f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4068600 tx=0x7ff4ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.121+0000 7ff4ab7fe640 1 -- 172.21.15.113:0/2150897580 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4a40799c0 con 0x7ff4b415e320 2024-04-25T00:11:03.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.292+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4b4153900 con 0x7ff49406be30 2024-04-25T00:11:03.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.293+0000 7ff4ab7fe640 1 -- 172.21.15.113:0/2150897580 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13454 (secure 0 0 0) 0x7ff4b4153900 con 0x7ff49406be30 2024-04-25T00:11:03.293 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:11:03.294 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:11:03.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff49406be30 msgr2=0x7ff49406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff49406be30 0x7ff49406e2f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4068600 tx=0x7ff4ac008040 comp rx=0 tx=0).stop 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b415e320 msgr2=0x7ff4b4155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b415e320 0x7ff4b4155c90 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40090b0 tx=0x7ff4a40046c0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 shutdown_connections 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff49406be30 0x7ff49406e2f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40d2290 0x7ff4b41613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b415e320 0x7ff4b4155c90 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 --2- 172.21.15.113:0/2150897580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40d0b00 0x7ff4b41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 >> 172.21.15.113:0/2150897580 conn(0x7ff4b40b5c20 msgr2=0x7ff4b40b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 shutdown_connections 2024-04-25T00:11:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.295+0000 7ff4b92da640 1 -- 172.21.15.113:0/2150897580 wait complete. 2024-04-25T00:11:03.306 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":568,"stamp":"2024-04-25T00:11:03.267910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":450324,"kb_used_data":44004,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754524396,"statfs":{"total":773094113280,"available":772632981504,"internally_reserved":0,"allocated":45060096,"data_stored":34569256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001046"},"pg_stats":[{"pgid":"1.0","version":"153'161","reported_seq":524,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T00:11:01.098059+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:11:01.098059+0000","last_peered":"2024-04-25T00:11:01.098059+0000","last_clean":"2024-04-25T00:11:01.098059+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:11:01.098059+0000","last_undegraded":"2024-04-25T00:11:01.098059+0000","last_fullsized":"2024-04-25T00:11:01.098059+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54168,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317672,"statfs":{"total":96636764160,"available":96581296128,"internally_reserved":0,"allocated":5058560,"data_stored":3746324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55672,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316168,"statfs":{"total":96636764160,"available":96579756032,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63792,"kb_used_data":14564,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308048,"statfs":{"total":96636764160,"available":96571441152,"internally_reserved":0,"allocated":14913536,"data_stored":13601262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53556,"kb_used_data":4328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318284,"statfs":{"total":96636764160,"available":96581922816,"internally_reserved":0,"allocated":4431872,"data_stored":3121409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":6376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":6529024,"data_stored":5218561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56184,"kb_used_data":6956,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315656,"statfs":{"total":96636764160,"available":96579231744,"internally_reserved":0,"allocated":7122944,"data_stored":5808929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:03.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:03.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.403+0000 7fa54d39e640 1 Processor -- start 2024-04-25T00:11:03.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.403+0000 7fa54d39e640 1 -- start start 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa54d39e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54815e4f0 0x7fa54815f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa54d39e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5480b6470 0x7fa5480d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa54d39e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa54815f8b0 con 0x7fa5480cf2b0 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa54d39e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa548161ff0 con 0x7fa5480b6470 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa54d39e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa5481622d0 con 0x7fa54815e4f0 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa5477fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5480b6470 0x7fa5480d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa5467fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa5480cf2b0 0x7fa5480b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40924/0 (socket says 172.21.15.113:40924) 2024-04-25T00:11:03.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa5467fc640 1 -- 172.21.15.113:0/338259690 learned_addr learned my addr 172.21.15.113:0/338259690 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa546ffd640 1 --2- 172.21.15.113:0/338259690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54815e4f0 0x7fa54815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa5477fe640 1 -- 172.21.15.113:0/338259690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54815e4f0 msgr2=0x7fa54815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa5477fe640 1 --2- 172.21.15.113:0/338259690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54815e4f0 0x7fa54815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa5477fe640 1 -- 172.21.15.113:0/338259690 >> v1:172.21.15.113:6789/0 conn(0x7fa5480cf2b0 legacy=0x7fa5480b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa5477fe640 1 -- 172.21.15.113:0/338259690 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5481625b0 con 0x7fa5480b6470 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.404+0000 7fa545ffb640 1 -- 172.21.15.113:0/338259690 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331548125 0 0) 0x7fa53c001110 con 0x7fa5480cf2b0 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.405+0000 7fa5477fe640 1 --2- 172.21.15.113:0/338259690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5480b6470 0x7fa5480d2760 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fa540009d90 tx=0x7fa54002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87eb97d458cdb63e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.405+0000 7fa545ffb640 1 -- 172.21.15.113:0/338259690 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540007740 con 0x7fa5480b6470 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.405+0000 7fa545ffb640 1 -- 172.21.15.113:0/338259690 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5400078e0 con 0x7fa5480b6470 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.405+0000 7fa545ffb640 1 -- 172.21.15.113:0/338259690 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540007cb0 con 0x7fa5480b6470 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.405+0000 7fa54d39e640 1 -- 172.21.15.113:0/338259690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5480b6470 msgr2=0x7fa5480d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.405+0000 7fa54d39e640 1 --2- 172.21.15.113:0/338259690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5480b6470 0x7fa5480d2760 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fa540009d90 tx=0x7fa54002a530 comp rx=0 tx=0).stop 2024-04-25T00:11:03.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.406+0000 7fa54d39e640 1 -- 172.21.15.113:0/338259690 shutdown_connections 2024-04-25T00:11:03.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.406+0000 7fa54d39e640 1 --2- 172.21.15.113:0/338259690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5480b6470 0x7fa5480d2760 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.406+0000 7fa54d39e640 1 --2- 172.21.15.113:0/338259690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54815e4f0 0x7fa54815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.406+0000 7fa54d39e640 1 -- 172.21.15.113:0/338259690 >> 172.21.15.113:0/338259690 conn(0x7fa54806b720 msgr2=0x7fa5480d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.406+0000 7fa54d39e640 1 -- 172.21.15.113:0/338259690 shutdown_connections 2024-04-25T00:11:03.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.406+0000 7fa54d39e640 1 -- 172.21.15.113:0/338259690 wait complete. 2024-04-25T00:11:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.407+0000 7fa54d39e640 1 Processor -- start 2024-04-25T00:11:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.407+0000 7fa54d39e640 1 -- start start 2024-04-25T00:11:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5480b6470 0x7fa548157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5480cf2b0 0x7fa5480c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54815e4f0 0x7fa54815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa5480d2e20 con 0x7fa54815e4f0 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5480d2ca0 con 0x7fa5480b6470 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa5480d2fa0 con 0x7fa5480cf2b0 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54815e4f0 0x7fa54815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54815e4f0 0x7fa54815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50654/0 (socket says 172.21.15.113:50654) 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 -- 172.21.15.113:0/3204021640 learned_addr learned my addr 172.21.15.113:0/3204021640 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa546ffd640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5480b6470 0x7fa548157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5467fc640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5480cf2b0 0x7fa5480c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 -- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5480cf2b0 msgr2=0x7fa5480c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5480cf2b0 0x7fa5480c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 -- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5480b6470 msgr2=0x7fa548157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5480b6470 0x7fa548157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 -- 172.21.15.113:0/3204021640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5480ca670 con 0x7fa54815e4f0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa5477fe640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54815e4f0 0x7fa54815aa10 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fa540009eb0 tx=0x7fa540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa52f7fe640 1 -- 172.21.15.113:0/3204021640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540040070 con 0x7fa54815e4f0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa52f7fe640 1 -- 172.21.15.113:0/3204021640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa540031bc0 con 0x7fa54815e4f0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5480ca950 con 0x7fa54815e4f0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa52f7fe640 1 -- 172.21.15.113:0/3204021640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540009450 con 0x7fa54815e4f0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.408+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5480caf10 con 0x7fa54815e4f0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.409+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5480691a0 con 0x7fa54815e4f0 2024-04-25T00:11:03.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.410+0000 7fa52f7fe640 1 -- 172.21.15.113:0/3204021640 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa540039040 con 0x7fa54815e4f0 2024-04-25T00:11:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.410+0000 7fa52f7fe640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa53406be30 0x7fa53406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.410+0000 7fa546ffd640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa53406be30 0x7fa53406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.410+0000 7fa52f7fe640 1 -- 172.21.15.113:0/3204021640 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fa5400ac250 con 0x7fa54815e4f0 2024-04-25T00:11:03.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.411+0000 7fa546ffd640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa53406be30 0x7fa53406e2f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa538009aa0 tx=0x7fa538009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.415+0000 7fa52f7fe640 1 -- 172.21.15.113:0/3204021640 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa54007a8e0 con 0x7fa54815e4f0 2024-04-25T00:11:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.582+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5480d1ff0 con 0x7fa53406be30 2024-04-25T00:11:03.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.583+0000 7fa52f7fe640 1 -- 172.21.15.113:0/3204021640 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13454 (secure 0 0 0) 0x7fa5480d1ff0 con 0x7fa53406be30 2024-04-25T00:11:03.587 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa53406be30 msgr2=0x7fa53406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa53406be30 0x7fa53406e2f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa538009aa0 tx=0x7fa538009190 comp rx=0 tx=0).stop 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54815e4f0 msgr2=0x7fa54815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54815e4f0 0x7fa54815aa10 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fa540009eb0 tx=0x7fa540004a40 comp rx=0 tx=0).stop 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 shutdown_connections 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fa53406be30 0x7fa53406e2f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54815e4f0 0x7fa54815aa10 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5480cf2b0 0x7fa5480c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 --2- 172.21.15.113:0/3204021640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5480b6470 0x7fa548157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 >> 172.21.15.113:0/3204021640 conn(0x7fa54806b720 msgr2=0x7fa54815fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 shutdown_connections 2024-04-25T00:11:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.585+0000 7fa54d39e640 1 -- 172.21.15.113:0/3204021640 wait complete. 2024-04-25T00:11:03.596 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":568,"stamp":"2024-04-25T00:11:03.267910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":450324,"kb_used_data":44004,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754524396,"statfs":{"total":773094113280,"available":772632981504,"internally_reserved":0,"allocated":45060096,"data_stored":34569256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001046"},"pg_stats":[{"pgid":"1.0","version":"153'161","reported_seq":524,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T00:11:01.098059+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:11:01.098059+0000","last_peered":"2024-04-25T00:11:01.098059+0000","last_clean":"2024-04-25T00:11:01.098059+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:11:01.098059+0000","last_undegraded":"2024-04-25T00:11:01.098059+0000","last_fullsized":"2024-04-25T00:11:01.098059+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54168,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317672,"statfs":{"total":96636764160,"available":96581296128,"internally_reserved":0,"allocated":5058560,"data_stored":3746324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55672,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316168,"statfs":{"total":96636764160,"available":96579756032,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63792,"kb_used_data":14564,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308048,"statfs":{"total":96636764160,"available":96571441152,"internally_reserved":0,"allocated":14913536,"data_stored":13601262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53556,"kb_used_data":4328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318284,"statfs":{"total":96636764160,"available":96581922816,"internally_reserved":0,"allocated":4431872,"data_stored":3121409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":6376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":6529024,"data_stored":5218561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56184,"kb_used_data":6956,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315656,"statfs":{"total":96636764160,"available":96579231744,"internally_reserved":0,"allocated":7122944,"data_stored":5808929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:03.597 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:11:03.597 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:03.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.693+0000 7f5e819c8640 1 Processor -- start 2024-04-25T00:11:03.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.693+0000 7f5e819c8640 1 -- start start 2024-04-25T00:11:03.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e819c8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e7c15a6b0 0x7f5e7c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e819c8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e7c0b2d50 con 0x7f5e7c072ea0 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e819c8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e7c072a70 con 0x7f5e7c0b3af0 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e819c8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5e7c0b3010 con 0x7f5e7c15a6b0 2024-04-25T00:11:03.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e7affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e7c15a6b0 0x7f5e7c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e7a7fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5e7c072ea0 0x7f5e7c071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40936/0 (socket says 172.21.15.113:40936) 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e7a7fc640 1 -- 172.21.15.113:0/1007774023 learned_addr learned my addr 172.21.15.113:0/1007774023 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e7affd640 1 -- 172.21.15.113:0/1007774023 >> v1:172.21.15.178:6789/0 conn(0x7f5e7c0b3af0 legacy=0x7f5e7c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e7affd640 1 -- 172.21.15.113:0/1007774023 >> v1:172.21.15.113:6789/0 conn(0x7f5e7c072ea0 legacy=0x7f5e7c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e7affd640 1 -- 172.21.15.113:0/1007774023 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c0b32f0 con 0x7f5e7c15a6b0 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.694+0000 7f5e79ffb640 1 -- 172.21.15.113:0/1007774023 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2161713812 0 0) 0x7f5e7c0b2d50 con 0x7f5e7c072ea0 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.695+0000 7f5e7affd640 1 --2- 172.21.15.113:0/1007774023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c15a6b0 0x7f5e7c070d00 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009d90 tx=0x7f5e6c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2147d01cde41632c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.695+0000 7f5e79ffb640 1 -- 172.21.15.113:0/1007774023 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c007740 con 0x7f5e7c15a6b0 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.695+0000 7f5e79ffb640 1 -- 172.21.15.113:0/1007774023 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e6c0078e0 con 0x7f5e7c15a6b0 2024-04-25T00:11:03.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.695+0000 7f5e79ffb640 1 -- 172.21.15.113:0/1007774023 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c007cb0 con 0x7f5e7c15a6b0 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.696+0000 7f5e819c8640 1 -- 172.21.15.113:0/1007774023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c15a6b0 msgr2=0x7f5e7c070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.696+0000 7f5e819c8640 1 --2- 172.21.15.113:0/1007774023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c15a6b0 0x7f5e7c070d00 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009d90 tx=0x7f5e6c02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.696+0000 7f5e819c8640 1 -- 172.21.15.113:0/1007774023 shutdown_connections 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.696+0000 7f5e819c8640 1 --2- 172.21.15.113:0/1007774023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c15a6b0 0x7f5e7c070d00 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.696+0000 7f5e819c8640 1 -- 172.21.15.113:0/1007774023 >> 172.21.15.113:0/1007774023 conn(0x7f5e7c0b1a50 msgr2=0x7f5e7c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.696+0000 7f5e819c8640 1 -- 172.21.15.113:0/1007774023 shutdown_connections 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.696+0000 7f5e819c8640 1 -- 172.21.15.113:0/1007774023 wait complete. 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.697+0000 7f5e819c8640 1 Processor -- start 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.697+0000 7f5e819c8640 1 -- start start 2024-04-25T00:11:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e819c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c072ea0 0x7f5e7c1477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e819c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7c0b3af0 0x7f5e7c14b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e819c8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e7c15a6b0 0x7f5e7c1439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e819c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c15aa90 con 0x7f5e7c072ea0 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e819c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c072720 con 0x7f5e7c0b3af0 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e819c8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c15ac10 con 0x7f5e7c15a6b0 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c072ea0 0x7f5e7c1477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c072ea0 0x7f5e7c1477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50678/0 (socket says 172.21.15.113:50678) 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 -- 172.21.15.113:0/3945977973 learned_addr learned my addr 172.21.15.113:0/3945977973 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 -- 172.21.15.113:0/3945977973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e7c15a6b0 msgr2=0x7f5e7c1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e7c15a6b0 0x7f5e7c1439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7a7fc640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7c0b3af0 0x7f5e7c14b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 -- 172.21.15.113:0/3945977973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7c0b3af0 msgr2=0x7f5e7c14b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7c0b3af0 0x7f5e7c14b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 -- 172.21.15.113:0/3945977973 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c15beb0 con 0x7f5e7c072ea0 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e7affd640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c072ea0 0x7f5e7c1477c0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009eb0 tx=0x7f5e6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e63fff640 1 -- 172.21.15.113:0/3945977973 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c040070 con 0x7f5e7c072ea0 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e63fff640 1 -- 172.21.15.113:0/3945977973 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e6c031c40 con 0x7f5e7c072ea0 2024-04-25T00:11:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e63fff640 1 -- 172.21.15.113:0/3945977973 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c03c420 con 0x7f5e7c072ea0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.698+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c15c120 con 0x7f5e7c072ea0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.699+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c15c6e0 con 0x7f5e7c072ea0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.700+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7c147d60 con 0x7f5e7c072ea0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.700+0000 7f5e63fff640 1 -- 172.21.15.113:0/3945977973 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e6c039040 con 0x7f5e7c072ea0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.700+0000 7f5e63fff640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e6806be30 0x7f5e6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.700+0000 7f5e63fff640 1 -- 172.21.15.113:0/3945977973 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f5e6c0ac280 con 0x7f5e7c072ea0 2024-04-25T00:11:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.700+0000 7f5e7a7fc640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e6806be30 0x7f5e6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.701+0000 7f5e7a7fc640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e6806be30 0x7f5e6806e2f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007fd0 tx=0x7f5e70009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.705+0000 7f5e63fff640 1 -- 172.21.15.113:0/3945977973 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e7c147d60 con 0x7f5e7c072ea0 2024-04-25T00:11:03.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.873+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e7c0691a0 con 0x7f5e6806be30 2024-04-25T00:11:03.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.874+0000 7f5e63fff640 1 -- 172.21.15.113:0/3945977973 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13454 (secure 0 0 0) 0x7f5e7c0691a0 con 0x7f5e6806be30 2024-04-25T00:11:03.875 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:11:03.875 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:11:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e6806be30 msgr2=0x7f5e6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e6806be30 0x7f5e6806e2f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007fd0 tx=0x7f5e70009040 comp rx=0 tx=0).stop 2024-04-25T00:11:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c072ea0 msgr2=0x7f5e7c1477c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c072ea0 0x7f5e7c1477c0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009eb0 tx=0x7f5e6c004a40 comp rx=0 tx=0).stop 2024-04-25T00:11:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 shutdown_connections 2024-04-25T00:11:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5e6806be30 0x7f5e6806e2f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e7c0b3af0 0x7f5e7c14b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e7c15a6b0 0x7f5e7c1439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 --2- 172.21.15.113:0/3945977973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e7c072ea0 0x7f5e7c1477c0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 >> 172.21.15.113:0/3945977973 conn(0x7f5e7c0b1a50 msgr2=0x7f5e7c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 shutdown_connections 2024-04-25T00:11:03.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.876+0000 7f5e819c8640 1 -- 172.21.15.113:0/3945977973 wait complete. 2024-04-25T00:11:03.887 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":568,"stamp":"2024-04-25T00:11:03.267910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":450324,"kb_used_data":44004,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754524396,"statfs":{"total":773094113280,"available":772632981504,"internally_reserved":0,"allocated":45060096,"data_stored":34569256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001046"},"pg_stats":[{"pgid":"1.0","version":"153'161","reported_seq":524,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T00:11:01.098059+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:11:01.098059+0000","last_peered":"2024-04-25T00:11:01.098059+0000","last_clean":"2024-04-25T00:11:01.098059+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:11:01.098059+0000","last_undegraded":"2024-04-25T00:11:01.098059+0000","last_fullsized":"2024-04-25T00:11:01.098059+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54168,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317672,"statfs":{"total":96636764160,"available":96581296128,"internally_reserved":0,"allocated":5058560,"data_stored":3746324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55672,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316168,"statfs":{"total":96636764160,"available":96579756032,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51512,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320328,"statfs":{"total":96636764160,"available":96584015872,"internally_reserved":0,"allocated":2334720,"data_stored":1024257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63792,"kb_used_data":14564,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308048,"statfs":{"total":96636764160,"available":96571441152,"internally_reserved":0,"allocated":14913536,"data_stored":13601262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53556,"kb_used_data":4328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318284,"statfs":{"total":96636764160,"available":96581922816,"internally_reserved":0,"allocated":4431872,"data_stored":3121409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":6376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":6529024,"data_stored":5218561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56184,"kb_used_data":6956,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315656,"statfs":{"total":96636764160,"available":96579231744,"internally_reserved":0,"allocated":7122944,"data_stored":5808929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:03.888 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:11:03.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.983+0000 7f92e6aba640 1 Processor -- start 2024-04-25T00:11:03.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.983+0000 7f92e6aba640 1 -- start start 2024-04-25T00:11:03.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e6aba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e015e240 0x7f92e01627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e6aba640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92e006e6e0 0x7f92e00b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e6aba640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e0162d20 con 0x7f92e00b5620 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e6aba640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f92e0072d90 con 0x7f92e006e6e0 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e6aba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92e00b3c20 con 0x7f92e015e240 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e5057640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92e006e6e0 0x7f92e00b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92dffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f92e00b5620 0x7f92e006dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40942/0 (socket says 172.21.15.113:40942) 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e5057640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92e006e6e0 0x7f92e00b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50688/0 (socket says 172.21.15.113:50688) 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92dffff640 1 -- 172.21.15.113:0/2765636670 learned_addr learned my addr 172.21.15.113:0/2765636670 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e4856640 1 --2- 172.21.15.113:0/2765636670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e015e240 0x7f92e01627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e5057640 1 -- 172.21.15.113:0/2765636670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e015e240 msgr2=0x7f92e01627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:03.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e5057640 1 --2- 172.21.15.113:0/2765636670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e015e240 0x7f92e01627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92e5057640 1 -- 172.21.15.113:0/2765636670 >> v1:172.21.15.113:6789/0 conn(0x7f92e00b5620 legacy=0x7f92e006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.984+0000 7f92df7fe640 1 -- 172.21.15.113:0/2765636670 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060071739 0 0) 0x7f92e0162d20 con 0x7f92e00b5620 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.985+0000 7f92e5057640 1 -- 172.21.15.113:0/2765636670 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e015e620 con 0x7f92e006e6e0 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.985+0000 7f92e4856640 1 --2- 172.21.15.113:0/2765636670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e015e240 0x7f92e01627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.985+0000 7f92e5057640 1 --2- 172.21.15.113:0/2765636670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e006e6e0 0x7f92e00b4120 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009d90 tx=0x7f92d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d04f8908ed8155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.985+0000 7f92df7fe640 1 -- 172.21.15.113:0/2765636670 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8007740 con 0x7f92e006e6e0 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.985+0000 7f92df7fe640 1 -- 172.21.15.113:0/2765636670 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d80078e0 con 0x7f92e006e6e0 2024-04-25T00:11:03.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.985+0000 7f92df7fe640 1 -- 172.21.15.113:0/2765636670 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8007cb0 con 0x7f92e006e6e0 2024-04-25T00:11:03.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 -- 172.21.15.113:0/2765636670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e006e6e0 msgr2=0x7f92e00b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 --2- 172.21.15.113:0/2765636670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e006e6e0 0x7f92e00b4120 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009d90 tx=0x7f92d802a660 comp rx=0 tx=0).stop 2024-04-25T00:11:03.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 -- 172.21.15.113:0/2765636670 shutdown_connections 2024-04-25T00:11:03.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 --2- 172.21.15.113:0/2765636670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e006e6e0 0x7f92e00b4120 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 --2- 172.21.15.113:0/2765636670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e015e240 0x7f92e01627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 -- 172.21.15.113:0/2765636670 >> 172.21.15.113:0/2765636670 conn(0x7f92e00b1a50 msgr2=0x7f92e00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:03.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 -- 172.21.15.113:0/2765636670 shutdown_connections 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.986+0000 7f92e6aba640 1 -- 172.21.15.113:0/2765636670 wait complete. 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.987+0000 7f92e6aba640 1 Processor -- start 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.987+0000 7f92e6aba640 1 -- start start 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e6aba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e006e6e0 0x7f92e015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e6aba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e00b5620 0x7f92e0141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e6aba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e015e240 0x7f92e0158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e6aba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00727d0 con 0x7f92e015e240 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e6aba640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00724f0 con 0x7f92e006e6e0 2024-04-25T00:11:03.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e6aba640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92e0072ab0 con 0x7f92e00b5620 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e015e240 0x7f92e0158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e015e240 0x7f92e0158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50696/0 (socket says 172.21.15.113:50696) 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 -- 172.21.15.113:0/3909518746 learned_addr learned my addr 172.21.15.113:0/3909518746 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e4856640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e006e6e0 0x7f92e015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92dffff640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e00b5620 0x7f92e0141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 -- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e00b5620 msgr2=0x7f92e0141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e00b5620 0x7f92e0141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 -- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e006e6e0 msgr2=0x7f92e015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e006e6e0 0x7f92e015fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 -- 172.21.15.113:0/3909518746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e01e8e90 con 0x7f92e015e240 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e4856640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e006e6e0 0x7f92e015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92dffff640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e00b5620 0x7f92e0141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:03.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92e5057640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e015e240 0x7f92e0158ac0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009d90 tx=0x7f92d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.988+0000 7f92dd7fa640 1 -- 172.21.15.113:0/3909518746 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8040070 con 0x7f92e015e240 2024-04-25T00:11:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.989+0000 7f92dd7fa640 1 -- 172.21.15.113:0/3909518746 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d8031bc0 con 0x7f92e015e240 2024-04-25T00:11:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.989+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e01e9170 con 0x7f92e015e240 2024-04-25T00:11:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.989+0000 7f92dd7fa640 1 -- 172.21.15.113:0/3909518746 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8009450 con 0x7f92e015e240 2024-04-25T00:11:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.989+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e014cf30 con 0x7f92e015e240 2024-04-25T00:11:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.990+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e00691a0 con 0x7f92e015e240 2024-04-25T00:11:03.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.990+0000 7f92dd7fa640 1 -- 172.21.15.113:0/3909518746 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92d8039040 con 0x7f92e015e240 2024-04-25T00:11:03.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.991+0000 7f92dd7fa640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92cc06be60 0x7f92cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:03.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.991+0000 7f92e4856640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92cc06be60 0x7f92cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:03.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.991+0000 7f92dd7fa640 1 -- 172.21.15.113:0/3909518746 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f92d80abd70 con 0x7f92e015e240 2024-04-25T00:11:03.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.991+0000 7f92e4856640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92cc06be60 0x7f92cc06e320 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f92d0006fd0 tx=0x7f92d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:03.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:03.995+0000 7f92dd7fa640 1 -- 172.21.15.113:0/3909518746 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f92d807a940 con 0x7f92e015e240 2024-04-25T00:11:04.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.170+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92e0161b40 con 0x7f92e015e240 2024-04-25T00:11:04.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.171+0000 7f92dd7fa640 1 -- 172.21.15.113:0/3909518746 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) ==== 74+0+13212 (secure 0 0 0) 0x7f92d807a240 con 0x7f92e015e240 2024-04-25T00:11:04.172 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:11:04.172 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":186,"fsid":"c4eaaf81-6120-4e3f-a50c-97aa81453142","created":"2024-04-24T23:57:54.996580+0000","modified":"2024-04-25T00:11:01.093941+0000","last_up_change":"2024-04-24T23:58:05.468169+0000","last_in_change":"2024-04-24T23:57:57.101501+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:58:07.141026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69805b28-aeb7-471f-b7ac-7181bae022b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6801","nonce":1851994484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6803","nonce":1851994484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6807","nonce":1851994484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1851994484},{"type":"v1","addr":"172.21.15.113:6805","nonce":1851994484}]},"public_addr":"172.21.15.113:6801/1851994484","cluster_addr":"172.21.15.113:6803/1851994484","heartbeat_back_addr":"172.21.15.113:6807/1851994484","heartbeat_front_addr":"172.21.15.113:6805/1851994484","state":["exists","up"]},{"osd":1,"uuid":"57db595e-9575-4f51-b81c-078f498b956a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6809","nonce":3114085930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6811","nonce":3114085930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6815","nonce":3114085930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3114085930},{"type":"v1","addr":"172.21.15.113:6813","nonce":3114085930}]},"public_addr":"172.21.15.113:6809/3114085930","cluster_addr":"172.21.15.113:6811/3114085930","heartbeat_back_addr":"172.21.15.113:6815/3114085930","heartbeat_front_addr":"172.21.15.113:6813/3114085930","state":["exists","up"]},{"osd":2,"uuid":"d1cd1ff3-77ed-4c8a-a6fd-7f490d7c7535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6817","nonce":3097656860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6819","nonce":3097656860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6823","nonce":3097656860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3097656860},{"type":"v1","addr":"172.21.15.113:6821","nonce":3097656860}]},"public_addr":"172.21.15.113:6817/3097656860","cluster_addr":"172.21.15.113:6819/3097656860","heartbeat_back_addr":"172.21.15.113:6823/3097656860","heartbeat_front_addr":"172.21.15.113:6821/3097656860","state":["exists","up"]},{"osd":3,"uuid":"a878c827-2212-4027-84a3-c426552c4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6825","nonce":1975200730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6827","nonce":1975200730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6831","nonce":1975200730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1975200730},{"type":"v1","addr":"172.21.15.113:6829","nonce":1975200730}]},"public_addr":"172.21.15.113:6825/1975200730","cluster_addr":"172.21.15.113:6827/1975200730","heartbeat_back_addr":"172.21.15.113:6831/1975200730","heartbeat_front_addr":"172.21.15.113:6829/1975200730","state":["exists","up"]},{"osd":4,"uuid":"4c8d4c12-d33b-44fa-8393-a4628959cccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6809","nonce":146317928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6811","nonce":146317928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6815","nonce":146317928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":146317928},{"type":"v1","addr":"172.21.15.178:6813","nonce":146317928}]},"public_addr":"172.21.15.178:6809/146317928","cluster_addr":"172.21.15.178:6811/146317928","heartbeat_back_addr":"172.21.15.178:6815/146317928","heartbeat_front_addr":"172.21.15.178:6813/146317928","state":["exists","up"]},{"osd":5,"uuid":"4df70f87-c9a1-4614-928d-b71a7a4fc6a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6817","nonce":2661350487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6819","nonce":2661350487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6823","nonce":2661350487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2661350487},{"type":"v1","addr":"172.21.15.178:6821","nonce":2661350487}]},"public_addr":"172.21.15.178:6817/2661350487","cluster_addr":"172.21.15.178:6819/2661350487","heartbeat_back_addr":"172.21.15.178:6823/2661350487","heartbeat_front_addr":"172.21.15.178:6821/2661350487","state":["exists","up"]},{"osd":6,"uuid":"dc93ad16-c47e-4936-b4e4-1ee139eff53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6801","nonce":1646944139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6803","nonce":1646944139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6807","nonce":1646944139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1646944139},{"type":"v1","addr":"172.21.15.178:6805","nonce":1646944139}]},"public_addr":"172.21.15.178:6801/1646944139","cluster_addr":"172.21.15.178:6803/1646944139","heartbeat_back_addr":"172.21.15.178:6807/1646944139","heartbeat_front_addr":"172.21.15.178:6805/1646944139","state":["exists","up"]},{"osd":7,"uuid":"c237b717-8504-49fd-a49b-d9a5ac74c922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6825","nonce":3779386153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6827","nonce":3779386153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6831","nonce":3779386153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3779386153},{"type":"v1","addr":"172.21.15.178:6829","nonce":3779386153}]},"public_addr":"172.21.15.178:6825/3779386153","cluster_addr":"172.21.15.178:6827/3779386153","heartbeat_back_addr":"172.21.15.178:6831/3779386153","heartbeat_front_addr":"172.21.15.178:6829/3779386153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.284124+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.273281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.271017+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.325078+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.257446+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.225747+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.302096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:02.275028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6836/2086115851":"2024-04-26T00:10:58.078225+0000","172.21.15.113:6835/2086115851":"2024-04-26T00:10:58.078225+0000","172.21.15.178:6835/3057967443":"2024-04-26T00:09:46.979970+0000","172.21.15.178:6833/3057967443":"2024-04-26T00:09:46.979970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92cc06be60 msgr2=0x7f92cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92cc06be60 0x7f92cc06e320 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f92d0006fd0 tx=0x7f92d0008040 comp rx=0 tx=0).stop 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e015e240 msgr2=0x7f92e0158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e015e240 0x7f92e0158ac0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009d90 tx=0x7f92d8004a40 comp rx=0 tx=0).stop 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 shutdown_connections 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f92cc06be60 0x7f92cc06e320 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92e015e240 0x7f92e0158ac0 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e00b5620 0x7f92e0141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 --2- 172.21.15.113:0/3909518746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e006e6e0 0x7f92e015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 >> 172.21.15.113:0/3909518746 conn(0x7f92e00b1a50 msgr2=0x7f92e015bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 shutdown_connections 2024-04-25T00:11:04.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:04.173+0000 7f92e6aba640 1 -- 172.21.15.113:0/3909518746 wait complete. 2024-04-25T00:11:05.185 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T00:11:05.186 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:05.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.297+0000 7f571dd09640 1 Processor -- start 2024-04-25T00:11:05.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.297+0000 7f571dd09640 1 -- start start 2024-04-25T00:11:05.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f571dd09640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5718144a80 0x7f57180b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f571dd09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571806e7e0 0x7f57180cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f571dd09640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5718072f70 con 0x7f57180b3ff0 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f571dd09640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f57180b2fd0 con 0x7f5718144a80 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f571dd09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57180b32b0 con 0x7f571806e7e0 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f5717fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571806e7e0 0x7f57180cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f5717fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571806e7e0 0x7f57180cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:48034/0 (socket says 172.21.15.113:48034) 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f5717fff640 1 -- 172.21.15.113:0/847032296 learned_addr learned my addr 172.21.15.113:0/847032296 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f57177fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5718144a80 0x7f57180b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f57167fc640 1 -- 172.21.15.113:0/847032296 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425681449 0 0) 0x7f5718072f70 con 0x7f57180b3ff0 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f57167fc640 1 -- 172.21.15.113:0/847032296 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5700003520 con 0x7f57180b3ff0 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f5717fff640 1 -- 172.21.15.113:0/847032296 >> v2:172.21.15.178:3300/0 conn(0x7f5718144a80 msgr2=0x7f57180b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:05.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f5717fff640 1 --2- 172.21.15.113:0/847032296 >> v2:172.21.15.178:3300/0 conn(0x7f5718144a80 0x7f57180b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f5717fff640 1 -- 172.21.15.113:0/847032296 >> v1:172.21.15.178:6789/0 conn(0x7f57180b3ff0 legacy=0x7f571806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f5717fff640 1 -- 172.21.15.113:0/847032296 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180b3590 con 0x7f571806e7e0 2024-04-25T00:11:05.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.298+0000 7f57177fe640 1 --2- 172.21.15.113:0/847032296 >> v2:172.21.15.178:3300/0 conn(0x7f5718144a80 0x7f57180b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:05.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.299+0000 7f5717fff640 1 --2- 172.21.15.113:0/847032296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571806e7e0 0x7f57180cd630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5710009e50 tx=0x7f571002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f568d1219354660c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.299+0000 7f57167fc640 1 -- 172.21.15.113:0/847032296 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710036070 con 0x7f571806e7e0 2024-04-25T00:11:05.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.299+0000 7f57167fc640 1 -- 172.21.15.113:0/847032296 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5710031600 con 0x7f571806e7e0 2024-04-25T00:11:05.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.299+0000 7f57167fc640 1 -- 172.21.15.113:0/847032296 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100319d0 con 0x7f571806e7e0 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 -- 172.21.15.113:0/847032296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571806e7e0 msgr2=0x7f57180cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 --2- 172.21.15.113:0/847032296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571806e7e0 0x7f57180cd630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5710009e50 tx=0x7f571002a680 comp rx=0 tx=0).stop 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 -- 172.21.15.113:0/847032296 shutdown_connections 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 --2- 172.21.15.113:0/847032296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571806e7e0 0x7f57180cd630 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 --2- 172.21.15.113:0/847032296 >> v2:172.21.15.178:3300/0 conn(0x7f5718144a80 0x7f57180b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 -- 172.21.15.113:0/847032296 >> 172.21.15.113:0/847032296 conn(0x7f57180b1b30 msgr2=0x7f57180b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 -- 172.21.15.113:0/847032296 shutdown_connections 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.300+0000 7f571dd09640 1 -- 172.21.15.113:0/847032296 wait complete. 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.301+0000 7f571dd09640 1 Processor -- start 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.301+0000 7f571dd09640 1 -- start start 2024-04-25T00:11:05.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f571dd09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571806e7e0 0x7f57180af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f571dd09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57180b3ff0 0x7f571815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f571dd09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5718144a80 0x7f57180b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f571dd09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f57180cdf00 con 0x7f5718144a80 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f571dd09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57180cdd80 con 0x7f571806e7e0 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f571dd09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5718072200 con 0x7f57180b3ff0 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5718144a80 0x7f57180b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5718144a80 0x7f57180b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50700/0 (socket says 172.21.15.113:50700) 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 -- 172.21.15.113:0/2471006668 learned_addr learned my addr 172.21.15.113:0/2471006668 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f57177fe640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571806e7e0 0x7f57180af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5716ffd640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57180b3ff0 0x7f571815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 -- 172.21.15.113:0/2471006668 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57180b3ff0 msgr2=0x7f571815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57180b3ff0 0x7f571815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 -- 172.21.15.113:0/2471006668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571806e7e0 msgr2=0x7f57180af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571806e7e0 0x7f57180af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f5717fff640 1 -- 172.21.15.113:0/2471006668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571815aa40 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.302+0000 7f57177fe640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571806e7e0 0x7f57180af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.303+0000 7f5717fff640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5718144a80 0x7f57180b1020 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f5710009e50 tx=0x7f5710004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.303+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710036050 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.303+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5710042070 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.303+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57180aff80 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.303+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710009510 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.303+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57180b0540 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.304+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57180691a0 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.304+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5710007bf0 con 0x7f5718144a80 2024-04-25T00:11:05.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f56fffff640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570406be60 0x7f570406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f57100ab980 con 0x7f5718144a80 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f57177fe640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570406be60 0x7f570406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f56fffff640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7f5704071dc0 0x7f57040741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 --> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f571004ae80 con 0x7f5704071dc0 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7f57100b02c0 con 0x7f5718144a80 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f5716ffd640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7f5704071dc0 0x7f57040741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f57177fe640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570406be60 0x7f570406e320 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5708009af0 tx=0x7f5708008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.305+0000 7f5716ffd640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7f5704071dc0 0x7f57040741e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.306+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== osd.0 v2:172.21.15.113:6800/1851994484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f571004ae80 con 0x7f5704071dc0 2024-04-25T00:11:05.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.319+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 --> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5718071d40 con 0x7f5704071dc0 2024-04-25T00:11:05.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.319+0000 7f56fffff640 1 -- 172.21.15.113:0/2471006668 <== osd.0 v2:172.21.15.113:6800/1851994484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5718071d40 con 0x7f5704071dc0 2024-04-25T00:11:05.320 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:05.320 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:05.320 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7f5704071dc0 msgr2=0x7f57040741e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6800/1851994484,v1:172.21.15.113:6801/1851994484] conn(0x7f5704071dc0 0x7f57040741e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570406be60 msgr2=0x7f570406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570406be60 0x7f570406e320 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5708009af0 tx=0x7f5708008040 comp rx=0 tx=0).stop 2024-04-25T00:11:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5718144a80 msgr2=0x7f57180b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 --2- 172.21.15.113:0/2471006668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5718144a80 0x7f57180b1020 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f5710009e50 tx=0x7f5710004a40 comp rx=0 tx=0).stop 2024-04-25T00:11:05.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f5717fff640 1 -- 172.21.15.113:0/2471006668 reap_dead start 2024-04-25T00:11:05.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 shutdown_connections 2024-04-25T00:11:05.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 >> 172.21.15.113:0/2471006668 conn(0x7f57180b1b30 msgr2=0x7f57180cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 shutdown_connections 2024-04-25T00:11:05.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.320+0000 7f571dd09640 1 -- 172.21.15.113:0/2471006668 wait complete. 2024-04-25T00:11:05.331 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T00:11:05.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.429+0000 7f544e215640 1 Processor -- start 2024-04-25T00:11:05.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.429+0000 7f544e215640 1 -- start start 2024-04-25T00:11:05.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.429+0000 7f544e215640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5448071db0 0x7f544815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.429+0000 7f544e215640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f544806cea0 0x7f54480cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f544e215640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5448071580 con 0x7f54480719d0 2024-04-25T00:11:05.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f544e215640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5448141620 con 0x7f544806cea0 2024-04-25T00:11:05.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f544e215640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5448141fc0 con 0x7f5448071db0 2024-04-25T00:11:05.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5447fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f544806cea0 0x7f54480cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5447fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f544806cea0 0x7f54480cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50716/0 (socket says 172.21.15.113:50716) 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5447fff640 1 -- 172.21.15.113:0/3971218203 learned_addr learned my addr 172.21.15.113:0/3971218203 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5446ffd640 1 --2- 172.21.15.113:0/3971218203 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5448071db0 0x7f544815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5447fff640 1 -- 172.21.15.113:0/3971218203 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5448071db0 msgr2=0x7f544815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5447fff640 1 --2- 172.21.15.113:0/3971218203 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5448071db0 0x7f544815f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5447fff640 1 -- 172.21.15.113:0/3971218203 >> v1:172.21.15.178:6789/0 conn(0x7f54480719d0 legacy=0x7f54481418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.430+0000 7f5447fff640 1 -- 172.21.15.113:0/3971218203 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544806d3f0 con 0x7f544806cea0 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f5447fff640 1 --2- 172.21.15.113:0/3971218203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544806cea0 0x7f54480cf020 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e50 tx=0x7f544002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4de1d36d12b1226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f54467fc640 1 -- 172.21.15.113:0/3971218203 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440036070 con 0x7f544806cea0 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f54467fc640 1 -- 172.21.15.113:0/3971218203 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5440031600 con 0x7f544806cea0 2024-04-25T00:11:05.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f54467fc640 1 -- 172.21.15.113:0/3971218203 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54400319d0 con 0x7f544806cea0 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 -- 172.21.15.113:0/3971218203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544806cea0 msgr2=0x7f54480cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 --2- 172.21.15.113:0/3971218203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544806cea0 0x7f54480cf020 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e50 tx=0x7f544002a750 comp rx=0 tx=0).stop 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 -- 172.21.15.113:0/3971218203 shutdown_connections 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 --2- 172.21.15.113:0/3971218203 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5448071db0 0x7f544815f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 --2- 172.21.15.113:0/3971218203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f544806cea0 0x7f54480cf020 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 -- 172.21.15.113:0/3971218203 >> 172.21.15.113:0/3971218203 conn(0x7f544806b0f0 msgr2=0x7f544806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 -- 172.21.15.113:0/3971218203 shutdown_connections 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.431+0000 7f544e215640 1 -- 172.21.15.113:0/3971218203 wait complete. 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.432+0000 7f544e215640 1 Processor -- start 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f544e215640 1 -- start start 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f544e215640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544806cea0 0x7f544815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f544e215640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54480719d0 0x7f544815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f544e215640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5448071db0 0x7f5448132980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f544e215640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54481439e0 con 0x7f5448071db0 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f544e215640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f544806d9b0 con 0x7f544806cea0 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f544e215640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f544806dc90 con 0x7f54480719d0 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f5447fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5448071db0 0x7f5448132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f5447fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5448071db0 0x7f5448132980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50728/0 (socket says 172.21.15.113:50728) 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.433+0000 7f5447fff640 1 -- 172.21.15.113:0/1063774530 learned_addr learned my addr 172.21.15.113:0/1063774530 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f54477fe640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544806cea0 0x7f544815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f5446ffd640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54480719d0 0x7f544815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f5447fff640 1 -- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54480719d0 msgr2=0x7f544815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f5447fff640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54480719d0 0x7f544815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f5447fff640 1 -- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544806cea0 msgr2=0x7f544815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f5447fff640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544806cea0 0x7f544815b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f5447fff640 1 -- 172.21.15.113:0/1063774530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5448130a90 con 0x7f5448071db0 2024-04-25T00:11:05.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f5447fff640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5448071db0 0x7f5448132980 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e50 tx=0x7f5440004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f542ffff640 1 -- 172.21.15.113:0/1063774530 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440036050 con 0x7f5448071db0 2024-04-25T00:11:05.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f542ffff640 1 -- 172.21.15.113:0/1063774530 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5440042070 con 0x7f5448071db0 2024-04-25T00:11:05.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5448130d70 con 0x7f5448071db0 2024-04-25T00:11:05.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f542ffff640 1 -- 172.21.15.113:0/1063774530 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440009510 con 0x7f5448071db0 2024-04-25T00:11:05.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.434+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5448131330 con 0x7f5448071db0 2024-04-25T00:11:05.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.435+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54480691a0 con 0x7f5448071db0 2024-04-25T00:11:05.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.435+0000 7f542ffff640 1 -- 172.21.15.113:0/1063774530 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5440007bd0 con 0x7f5448071db0 2024-04-25T00:11:05.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.436+0000 7f542ffff640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f543406be30 0x7f543406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.436+0000 7f542ffff640 1 -- 172.21.15.113:0/1063774530 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f54400abb30 con 0x7f5448071db0 2024-04-25T00:11:05.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.436+0000 7f54477fe640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f543406be30 0x7f543406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.436+0000 7f54477fe640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f543406be30 0x7f543406e2f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5438004ea0 tx=0x7f5438009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.440+0000 7f542ffff640 1 -- 172.21.15.113:0/1063774530 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5448071580 con 0x7f5448071db0 2024-04-25T00:11:05.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.612+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f5448131570 con 0x7f543406be30 2024-04-25T00:11:05.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.612+0000 7f542ffff640 1 -- 172.21.15.113:0/1063774530 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5438005420 con 0x7f543406be30 2024-04-25T00:11:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T00:11:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f543406be30 msgr2=0x7f543406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f543406be30 0x7f543406e2f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5438004ea0 tx=0x7f5438009040 comp rx=0 tx=0).stop 2024-04-25T00:11:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5448071db0 msgr2=0x7f5448132980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5448071db0 0x7f5448132980 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e50 tx=0x7f5440004a40 comp rx=0 tx=0).stop 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 shutdown_connections 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f543406be30 0x7f543406e2f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5448071db0 0x7f5448132980 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54480719d0 0x7f544815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 --2- 172.21.15.113:0/1063774530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f544806cea0 0x7f544815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.614+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 >> 172.21.15.113:0/1063774530 conn(0x7f544806b0f0 msgr2=0x7f54480ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.615+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 shutdown_connections 2024-04-25T00:11:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.615+0000 7f544e215640 1 -- 172.21.15.113:0/1063774530 wait complete. 2024-04-25T00:11:05.626 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T00:11:05.626 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:05.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.726+0000 7f509bf42640 1 Processor -- start 2024-04-25T00:11:05.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.726+0000 7f509bf42640 1 -- start start 2024-04-25T00:11:05.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.726+0000 7f509bf42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5094144a80 0x7f50940b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509bf42640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f509406e7e0 0x7f50940cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509bf42640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5094072f70 con 0x7f50940b3ff0 2024-04-25T00:11:05.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509bf42640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f50940b2fd0 con 0x7f509406e7e0 2024-04-25T00:11:05.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509bf42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f50940b32b0 con 0x7f5094144a80 2024-04-25T00:11:05.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509a4df640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f509406e7e0 0x7f50940cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509a4df640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f509406e7e0 0x7f50940cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50736/0 (socket says 172.21.15.113:50736) 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509a4df640 1 -- 172.21.15.113:0/4134421554 learned_addr learned my addr 172.21.15.113:0/4134421554 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f5099cde640 1 --2- 172.21.15.113:0/4134421554 >> v2:172.21.15.178:3300/0 conn(0x7f5094144a80 0x7f50940b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509a4df640 1 -- 172.21.15.113:0/4134421554 >> v2:172.21.15.178:3300/0 conn(0x7f5094144a80 msgr2=0x7f50940b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509a4df640 1 --2- 172.21.15.113:0/4134421554 >> v2:172.21.15.178:3300/0 conn(0x7f5094144a80 0x7f50940b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509a4df640 1 -- 172.21.15.113:0/4134421554 >> v1:172.21.15.178:6789/0 conn(0x7f50940b3ff0 legacy=0x7f509406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f509a4df640 1 -- 172.21.15.113:0/4134421554 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50940b3590 con 0x7f509406e7e0 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.727+0000 7f5099cde640 1 --2- 172.21.15.113:0/4134421554 >> v2:172.21.15.178:3300/0 conn(0x7f5094144a80 0x7f50940b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509a4df640 1 --2- 172.21.15.113:0/4134421554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50940cd630 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f509406bf70 tx=0x7f508c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6438e9ad1808c20e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f5098cdc640 1 -- 172.21.15.113:0/4134421554 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c036070 con 0x7f509406e7e0 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f5098cdc640 1 -- 172.21.15.113:0/4134421554 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c031600 con 0x7f509406e7e0 2024-04-25T00:11:05.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f5098cdc640 1 -- 172.21.15.113:0/4134421554 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c0319d0 con 0x7f509406e7e0 2024-04-25T00:11:05.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509bf42640 1 -- 172.21.15.113:0/4134421554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 msgr2=0x7f50940cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509bf42640 1 --2- 172.21.15.113:0/4134421554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50940cd630 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f509406bf70 tx=0x7f508c02a6a0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509bf42640 1 -- 172.21.15.113:0/4134421554 shutdown_connections 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509bf42640 1 --2- 172.21.15.113:0/4134421554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50940cd630 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509bf42640 1 --2- 172.21.15.113:0/4134421554 >> v2:172.21.15.178:3300/0 conn(0x7f5094144a80 0x7f50940b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509bf42640 1 -- 172.21.15.113:0/4134421554 >> 172.21.15.113:0/4134421554 conn(0x7f50940b1b30 msgr2=0x7f50940b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.728+0000 7f509bf42640 1 -- 172.21.15.113:0/4134421554 shutdown_connections 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.729+0000 7f509bf42640 1 -- 172.21.15.113:0/4134421554 wait complete. 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 Processor -- start 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 -- start start 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50941f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50940b3ff0 0x7f509413a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5094144a80 0x7f50941431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50940cdf00 con 0x7f509406e7e0 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50940cdd80 con 0x7f5094144a80 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f509bf42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5094072200 con 0x7f50940b3ff0 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f5099cde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50941f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f5099cde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50941f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50752/0 (socket says 172.21.15.113:50752) 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.730+0000 7f5099cde640 1 -- 172.21.15.113:0/142120764 learned_addr learned my addr 172.21.15.113:0/142120764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f509a4df640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5094144a80 0x7f50941431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f50994dd640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50940b3ff0 0x7f509413a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f5099cde640 1 -- 172.21.15.113:0/142120764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50940b3ff0 msgr2=0x7f509413a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f5099cde640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50940b3ff0 0x7f509413a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f5099cde640 1 -- 172.21.15.113:0/142120764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5094144a80 msgr2=0x7f50941431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f5099cde640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5094144a80 0x7f50941431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f5099cde640 1 -- 172.21.15.113:0/142120764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509413ac90 con 0x7f509406e7e0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f50994dd640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50940b3ff0 0x7f509413a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f5099cde640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50941f5f60 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f509000d7d0 tx=0x7f509000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090004030 con 0x7f509406e7e0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50900041d0 con 0x7f509406e7e0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50941e3ae0 con 0x7f509406e7e0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50900096a0 con 0x7f509406e7e0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.731+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509415bcd0 con 0x7f509406e7e0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.732+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50941f64a0 con 0x7f509406e7e0 2024-04-25T00:11:05.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5090009840 con 0x7f509406e7e0 2024-04-25T00:11:05.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f508a7fc640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f507806be30 0x7f507806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f50994dd640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f507806be30 0x7f507806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f509008adc0 con 0x7f509406e7e0 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f508a7fc640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f5078071d90 0x7f50780741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 --> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f509001de50 con 0x7f5078071d90 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7f509008d4c0 con 0x7f509406e7e0 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f509a4df640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f5078071d90 0x7f50780741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.733+0000 7f50994dd640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f507806be30 0x7f507806e2f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f50941f54b0 tx=0x7f5084009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.734+0000 7f509a4df640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f5078071d90 0x7f50780741b0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.734+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== osd.1 v2:172.21.15.113:6808/3114085930 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f509001de50 con 0x7f5078071d90 2024-04-25T00:11:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.747+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 --> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f50940691a0 con 0x7f5078071d90 2024-04-25T00:11:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f508a7fc640 1 -- 172.21.15.113:0/142120764 <== osd.1 v2:172.21.15.113:6808/3114085930 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f50940691a0 con 0x7f5078071d90 2024-04-25T00:11:05.748 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:05.748 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:05.748 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f5078071d90 msgr2=0x7f50780741b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6808/3114085930,v1:172.21.15.113:6809/3114085930] conn(0x7f5078071d90 0x7f50780741b0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f507806be30 msgr2=0x7f507806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f507806be30 0x7f507806e2f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f50941f54b0 tx=0x7f5084009040 comp rx=0 tx=0).stop 2024-04-25T00:11:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 msgr2=0x7f50941f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 --2- 172.21.15.113:0/142120764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f509406e7e0 0x7f50941f5f60 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f509000d7d0 tx=0x7f509000dc90 comp rx=0 tx=0).stop 2024-04-25T00:11:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509a4df640 1 -- 172.21.15.113:0/142120764 reap_dead start 2024-04-25T00:11:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 shutdown_connections 2024-04-25T00:11:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 >> 172.21.15.113:0/142120764 conn(0x7f50940b1b30 msgr2=0x7f50940b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 shutdown_connections 2024-04-25T00:11:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.748+0000 7f509bf42640 1 -- 172.21.15.113:0/142120764 wait complete. 2024-04-25T00:11:05.759 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T00:11:05.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.856+0000 7f6a779c4640 1 Processor -- start 2024-04-25T00:11:05.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.856+0000 7f6a779c4640 1 -- start start 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a779c4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a70158a00 0x7f6a700d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a779c4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a700d0170 con 0x7f6a70162700 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a779c4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a700d0450 con 0x7f6a700d2b10 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a779c4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6a7006b0f0 con 0x7f6a70158a00 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a75760640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a70158a00 0x7f6a700d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a75f61640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6a70162700 0x7f6a7006bb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40944/0 (socket says 172.21.15.113:40944) 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a75f61640 1 -- 172.21.15.113:0/619855453 learned_addr learned my addr 172.21.15.113:0/619855453 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a75760640 1 -- 172.21.15.113:0/619855453 >> v1:172.21.15.178:6789/0 conn(0x7f6a700d2b10 legacy=0x7f6a70161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a75760640 1 -- 172.21.15.113:0/619855453 >> v1:172.21.15.113:6789/0 conn(0x7f6a70162700 legacy=0x7f6a7006bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a75760640 1 -- 172.21.15.113:0/619855453 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7006b340 con 0x7f6a70158a00 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.857+0000 7f6a67fff640 1 -- 172.21.15.113:0/619855453 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975409111 0 0) 0x7f6a700d0170 con 0x7f6a70162700 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a75760640 1 --2- 172.21.15.113:0/619855453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a70158a00 0x7f6a700d25d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0097a0 tx=0x7f6a6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6b3874b2e608f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a67fff640 1 -- 172.21.15.113:0/619855453 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c037070 con 0x7f6a70158a00 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a67fff640 1 -- 172.21.15.113:0/619855453 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6c032440 con 0x7f6a70158a00 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a67fff640 1 -- 172.21.15.113:0/619855453 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c032810 con 0x7f6a70158a00 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a779c4640 1 -- 172.21.15.113:0/619855453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a70158a00 msgr2=0x7f6a700d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a779c4640 1 --2- 172.21.15.113:0/619855453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a70158a00 0x7f6a700d25d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0097a0 tx=0x7f6a6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a779c4640 1 -- 172.21.15.113:0/619855453 shutdown_connections 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a779c4640 1 --2- 172.21.15.113:0/619855453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a70158a00 0x7f6a700d25d0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.858+0000 7f6a779c4640 1 -- 172.21.15.113:0/619855453 >> 172.21.15.113:0/619855453 conn(0x7f6a700b3d80 msgr2=0x7f6a700b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.859+0000 7f6a779c4640 1 -- 172.21.15.113:0/619855453 shutdown_connections 2024-04-25T00:11:05.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.859+0000 7f6a779c4640 1 -- 172.21.15.113:0/619855453 wait complete. 2024-04-25T00:11:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 Processor -- start 2024-04-25T00:11:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 -- start start 2024-04-25T00:11:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a700d2b10 0x7f6a70156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a70158a00 0x7f6a700b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a70162700 0x7f6a700b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70158de0 con 0x7f6a700d2b10 2024-04-25T00:11:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7006b6b0 con 0x7f6a70158a00 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a779c4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7006c250 con 0x7f6a70162700 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a75760640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a700d2b10 0x7f6a70156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a75760640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a700d2b10 0x7f6a70156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50766/0 (socket says 172.21.15.113:50766) 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.860+0000 7f6a75760640 1 -- 172.21.15.113:0/1625454999 learned_addr learned my addr 172.21.15.113:0/1625454999 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a75760640 1 -- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a70162700 msgr2=0x7f6a700b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a75760640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a70162700 0x7f6a700b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a75760640 1 -- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a70158a00 msgr2=0x7f6a700b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a74f5f640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a70158a00 0x7f6a700b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a75760640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a70158a00 0x7f6a700b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a75760640 1 -- 172.21.15.113:0/1625454999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a700b5870 con 0x7f6a700d2b10 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a74f5f640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a70158a00 0x7f6a700b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a75760640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a700d2b10 0x7f6a70156b20 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0097a0 tx=0x7f6a6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a65ffb640 1 -- 172.21.15.113:0/1625454999 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c041070 con 0x7f6a700d2b10 2024-04-25T00:11:05.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a65ffb640 1 -- 172.21.15.113:0/1625454999 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6c03c870 con 0x7f6a700d2b10 2024-04-25T00:11:05.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7015efd0 con 0x7f6a700d2b10 2024-04-25T00:11:05.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a65ffb640 1 -- 172.21.15.113:0/1625454999 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c037040 con 0x7f6a700d2b10 2024-04-25T00:11:05.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.861+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a700b5af0 con 0x7f6a700d2b10 2024-04-25T00:11:05.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.862+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a700691a0 con 0x7f6a700d2b10 2024-04-25T00:11:05.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.863+0000 7f6a65ffb640 1 -- 172.21.15.113:0/1625454999 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a6c044070 con 0x7f6a700d2b10 2024-04-25T00:11:05.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.863+0000 7f6a65ffb640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a5406be30 0x7f6a5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.863+0000 7f6a74f5f640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a5406be30 0x7f6a5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.863+0000 7f6a65ffb640 1 -- 172.21.15.113:0/1625454999 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f6a6c0aba80 con 0x7f6a700d2b10 2024-04-25T00:11:05.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.864+0000 7f6a74f5f640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a5406be30 0x7f6a5406e2f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6a701607a0 tx=0x7f6a60009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:05.867+0000 7f6a65ffb640 1 -- 172.21.15.113:0/1625454999 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a700b5870 con 0x7f6a700d2b10 2024-04-25T00:11:06.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.038+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f6a70157c90 con 0x7f6a5406be30 2024-04-25T00:11:06.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.038+0000 7f6a65ffb640 1 -- 172.21.15.113:0/1625454999 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6a70157c90 con 0x7f6a5406be30 2024-04-25T00:11:06.039 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a5406be30 msgr2=0x7f6a5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a5406be30 0x7f6a5406e2f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6a701607a0 tx=0x7f6a60009040 comp rx=0 tx=0).stop 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a700d2b10 msgr2=0x7f6a70156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a700d2b10 0x7f6a70156b20 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0097a0 tx=0x7f6a6c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 shutdown_connections 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a5406be30 0x7f6a5406e2f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a70158a00 0x7f6a700b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a70162700 0x7f6a700b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 --2- 172.21.15.113:0/1625454999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a700d2b10 0x7f6a70156b20 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.040+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 >> 172.21.15.113:0/1625454999 conn(0x7f6a700b3d80 msgr2=0x7f6a700b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.041+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 shutdown_connections 2024-04-25T00:11:06.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.041+0000 7f6a779c4640 1 -- 172.21.15.113:0/1625454999 wait complete. 2024-04-25T00:11:06.052 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T00:11:06.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.153+0000 7fb50a07d640 1 Processor -- start 2024-04-25T00:11:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.153+0000 7fb50a07d640 1 -- start start 2024-04-25T00:11:06.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb50a07d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb50415e510 0x7fb504161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb50a07d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5040b63b0 0x7fb5040d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb50a07d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5041626f0 con 0x7fb5040b9bb0 2024-04-25T00:11:06.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb50a07d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb5041629d0 con 0x7fb5040b63b0 2024-04-25T00:11:06.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb50a07d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb504162cb0 con 0x7fb50415e510 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb503fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5040b63b0 0x7fb5040d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb502ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb5040b9bb0 0x7fb5040b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40948/0 (socket says 172.21.15.113:40948) 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.154+0000 7fb502ffd640 1 -- 172.21.15.113:0/1866704733 learned_addr learned my addr 172.21.15.113:0/1866704733 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.155+0000 7fb5037fe640 1 --2- 172.21.15.113:0/1866704733 >> v2:172.21.15.178:3300/0 conn(0x7fb50415e510 0x7fb504161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.155+0000 7fb503fff640 1 -- 172.21.15.113:0/1866704733 >> v2:172.21.15.178:3300/0 conn(0x7fb50415e510 msgr2=0x7fb504161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.155+0000 7fb503fff640 1 --2- 172.21.15.113:0/1866704733 >> v2:172.21.15.178:3300/0 conn(0x7fb50415e510 0x7fb504161ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.155+0000 7fb503fff640 1 -- 172.21.15.113:0/1866704733 >> v1:172.21.15.113:6789/0 conn(0x7fb5040b9bb0 legacy=0x7fb5040b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.155+0000 7fb503fff640 1 -- 172.21.15.113:0/1866704733 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5040d2660 con 0x7fb5040b63b0 2024-04-25T00:11:06.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.155+0000 7fb503fff640 1 --2- 172.21.15.113:0/1866704733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5040d2e60 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fb5040727e0 tx=0x7fb4fc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a641f3574e9893e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.155+0000 7fb5027fc640 1 -- 172.21.15.113:0/1866704733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc007740 con 0x7fb5040b63b0 2024-04-25T00:11:06.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb5027fc640 1 -- 172.21.15.113:0/1866704733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc0078e0 con 0x7fb5040b63b0 2024-04-25T00:11:06.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb5027fc640 1 -- 172.21.15.113:0/1866704733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc007cb0 con 0x7fb5040b63b0 2024-04-25T00:11:06.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 -- 172.21.15.113:0/1866704733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 msgr2=0x7fb5040d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 --2- 172.21.15.113:0/1866704733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5040d2e60 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fb5040727e0 tx=0x7fb4fc02a5f0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 -- 172.21.15.113:0/1866704733 shutdown_connections 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 --2- 172.21.15.113:0/1866704733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5040d2e60 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 --2- 172.21.15.113:0/1866704733 >> v2:172.21.15.178:3300/0 conn(0x7fb50415e510 0x7fb504161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 -- 172.21.15.113:0/1866704733 >> 172.21.15.113:0/1866704733 conn(0x7fb50406b720 msgr2=0x7fb5040d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 -- 172.21.15.113:0/1866704733 shutdown_connections 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.156+0000 7fb50a07d640 1 -- 172.21.15.113:0/1866704733 wait complete. 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.157+0000 7fb50a07d640 1 Processor -- start 2024-04-25T00:11:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.157+0000 7fb50a07d640 1 -- start start 2024-04-25T00:11:06.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb50a07d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5041f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb50a07d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040b9bb0 0x7fb5041587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb50a07d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb50415e510 0x7fb504141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb50a07d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040b73c0 con 0x7fb5040b63b0 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb50a07d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041623e0 con 0x7fb50415e510 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb50a07d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb5040b7540 con 0x7fb5040b9bb0 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5041f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5041f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50772/0 (socket says 172.21.15.113:50772) 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 -- 172.21.15.113:0/1477096255 learned_addr learned my addr 172.21.15.113:0/1477096255 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb503fff640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb50415e510 0x7fb504141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 -- 172.21.15.113:0/1477096255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040b9bb0 msgr2=0x7fb5041587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040b9bb0 0x7fb5041587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 -- 172.21.15.113:0/1477096255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb50415e510 msgr2=0x7fb504141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb50415e510 0x7fb504141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.158+0000 7fb5037fe640 1 -- 172.21.15.113:0/1477096255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb504142c80 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.159+0000 7fb503fff640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb50415e510 0x7fb504141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.159+0000 7fb5037fe640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5041f4fa0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400d800 tx=0x7fb4f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.159+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400b6c0 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.159+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f4019070 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.159+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041429a0 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.159+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400ba80 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.159+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041601c0 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.160+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb50415fcf0 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.160+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4f400bc20 con 0x7fb5040b63b0 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb4ebfff640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4f006be30 0x7fb4f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb502ffd640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4f006be30 0x7fb4f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fb4f408baf0 con 0x7fb5040b63b0 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb4ebfff640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7fb4f0071d90 0x7fb4f00741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 --> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4f0074860 con 0x7fb4f0071d90 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7fb4f408bf80 con 0x7fb5040b63b0 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb503fff640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7fb4f0071d90 0x7fb4f00741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb502ffd640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4f006be30 0x7fb4f006e2f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb5041f4e70 tx=0x7fb4f8007a40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.161+0000 7fb503fff640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7fb4f0071d90 0x7fb4f00741b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.162+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== osd.2 v2:172.21.15.113:6816/3097656860 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4f0074860 con 0x7fb4f0071d90 2024-04-25T00:11:06.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.175+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 --> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb5040691a0 con 0x7fb4f0071d90 2024-04-25T00:11:06.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.175+0000 7fb4ebfff640 1 -- 172.21.15.113:0/1477096255 <== osd.2 v2:172.21.15.113:6816/3097656860 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb5040691a0 con 0x7fb4f0071d90 2024-04-25T00:11:06.176 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:06.176 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:06.176 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.175+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7fb4f0071d90 msgr2=0x7fb4f00741b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.175+0000 7fb50a07d640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6816/3097656860,v1:172.21.15.113:6817/3097656860] conn(0x7fb4f0071d90 0x7fb4f00741b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.175+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4f006be30 msgr2=0x7fb4f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.175+0000 7fb50a07d640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fb4f006be30 0x7fb4f006e2f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb5041f4e70 tx=0x7fb4f8007a40 comp rx=0 tx=0).stop 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.176+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 msgr2=0x7fb5041f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.176+0000 7fb50a07d640 1 --2- 172.21.15.113:0/1477096255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5040b63b0 0x7fb5041f4fa0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400d800 tx=0x7fb4f400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.176+0000 7fb503fff640 1 -- 172.21.15.113:0/1477096255 reap_dead start 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.176+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 shutdown_connections 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.176+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 >> 172.21.15.113:0/1477096255 conn(0x7fb50406b720 msgr2=0x7fb504156900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.176+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 shutdown_connections 2024-04-25T00:11:06.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.176+0000 7fb50a07d640 1 -- 172.21.15.113:0/1477096255 wait complete. 2024-04-25T00:11:06.186 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T00:11:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.284+0000 7fd7118b3640 1 Processor -- start 2024-04-25T00:11:06.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.284+0000 7fd7118b3640 1 -- start start 2024-04-25T00:11:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd7118b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd7118b3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd70c06f180 0x7fd70c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd7118b3640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd70c15cb40 con 0x7fd70c15c050 2024-04-25T00:11:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd7118b3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd70c160900 con 0x7fd70c06f180 2024-04-25T00:11:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd7118b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd70c06d870 con 0x7fd70c074c00 2024-04-25T00:11:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd70c06f180 0x7fd70c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd70c06f180 0x7fd70c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50776/0 (socket says 172.21.15.113:50776) 2024-04-25T00:11:06.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70b7fe640 1 -- 172.21.15.113:0/283476662 learned_addr learned my addr 172.21.15.113:0/283476662 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:06.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70a7fc640 1 --2- 172.21.15.113:0/283476662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70b7fe640 1 -- 172.21.15.113:0/283476662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 msgr2=0x7fd70c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:06.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70b7fe640 1 --2- 172.21.15.113:0/283476662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70b7fe640 1 -- 172.21.15.113:0/283476662 >> v1:172.21.15.178:6789/0 conn(0x7fd70c15c050 legacy=0x7fd70c15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.285+0000 7fd70b7fe640 1 -- 172.21.15.113:0/283476662 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c06db50 con 0x7fd70c06f180 2024-04-25T00:11:06.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.286+0000 7fd70b7fe640 1 --2- 172.21.15.113:0/283476662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c0cf930 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fd704009e50 tx=0x7fd70402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea84f755d527657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.286+0000 7fd709ffb640 1 -- 172.21.15.113:0/283476662 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd704036070 con 0x7fd70c06f180 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.286+0000 7fd709ffb640 1 -- 172.21.15.113:0/283476662 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd704031600 con 0x7fd70c06f180 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.286+0000 7fd709ffb640 1 -- 172.21.15.113:0/283476662 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7040319d0 con 0x7fd70c06f180 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 -- 172.21.15.113:0/283476662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 msgr2=0x7fd70c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 --2- 172.21.15.113:0/283476662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c0cf930 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fd704009e50 tx=0x7fd70402a710 comp rx=0 tx=0).stop 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 -- 172.21.15.113:0/283476662 shutdown_connections 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 --2- 172.21.15.113:0/283476662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 --2- 172.21.15.113:0/283476662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c0cf930 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 -- 172.21.15.113:0/283476662 >> 172.21.15.113:0/283476662 conn(0x7fd70c06fb60 msgr2=0x7fd70c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 -- 172.21.15.113:0/283476662 shutdown_connections 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.287+0000 7fd7118b3640 1 -- 172.21.15.113:0/283476662 wait complete. 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.288+0000 7fd7118b3640 1 Processor -- start 2024-04-25T00:11:06.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.288+0000 7fd7118b3640 1 -- start start 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd7118b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c14bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd7118b3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd7118b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd70c15c050 0x7fd70c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd7118b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0c5880 con 0x7fd70c06f180 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd7118b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0c5700 con 0x7fd70c15c050 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd7118b3640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd70c06e070 con 0x7fd70c074c00 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c14bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c14bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50792/0 (socket says 172.21.15.113:50792) 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 -- 172.21.15.113:0/1604719874 learned_addr learned my addr 172.21.15.113:0/1604719874 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70b7fe640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd70c15c050 0x7fd70c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70a7fc640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 -- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 msgr2=0x7fd70c0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 -- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd70c15c050 msgr2=0x7fd70c0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd70c15c050 0x7fd70c0cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 -- 172.21.15.113:0/1604719874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c0c8c30 con 0x7fd70c06f180 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.289+0000 7fd70affd640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c14bff0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c3c0 tx=0x7fd6fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.290+0000 7fd6f3fff640 1 -- 172.21.15.113:0/1604719874 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc0052d0 con 0x7fd70c06f180 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.290+0000 7fd6f3fff640 1 -- 172.21.15.113:0/1604719874 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6fc005490 con 0x7fd70c06f180 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.290+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c0c9440 con 0x7fd70c06f180 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.290+0000 7fd6f3fff640 1 -- 172.21.15.113:0/1604719874 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc007260 con 0x7fd70c06f180 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.290+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c14c530 con 0x7fd70c06f180 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.291+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c0691a0 con 0x7fd70c06f180 2024-04-25T00:11:06.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.291+0000 7fd6f3fff640 1 -- 172.21.15.113:0/1604719874 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6fc007400 con 0x7fd70c06f180 2024-04-25T00:11:06.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.292+0000 7fd6f3fff640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd6f806be30 0x7fd6f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.292+0000 7fd6f3fff640 1 -- 172.21.15.113:0/1604719874 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fd6fc089fc0 con 0x7fd70c06f180 2024-04-25T00:11:06.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.292+0000 7fd70a7fc640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd6f806be30 0x7fd6f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.292+0000 7fd70a7fc640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd6f806be30 0x7fd6f806e2f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd70c15aad0 tx=0x7fd700009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.296+0000 7fd6f3fff640 1 -- 172.21.15.113:0/1604719874 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd6fc0585c0 con 0x7fd70c06f180 2024-04-25T00:11:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.467+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fd70c06e8d0 con 0x7fd6f806be30 2024-04-25T00:11:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.467+0000 7fd6f3fff640 1 -- 172.21.15.113:0/1604719874 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd70c06e8d0 con 0x7fd6f806be30 2024-04-25T00:11:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T00:11:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd6f806be30 msgr2=0x7fd6f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd6f806be30 0x7fd6f806e2f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd70c15aad0 tx=0x7fd700009190 comp rx=0 tx=0).stop 2024-04-25T00:11:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 msgr2=0x7fd70c14bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c14bff0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c3c0 tx=0x7fd6fc00c880 comp rx=0 tx=0).stop 2024-04-25T00:11:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 shutdown_connections 2024-04-25T00:11:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fd6f806be30 0x7fd6f806e2f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd70c15c050 0x7fd70c0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd70c074c00 0x7fd70c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 --2- 172.21.15.113:0/1604719874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c06f180 0x7fd70c14bff0 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 >> 172.21.15.113:0/1604719874 conn(0x7fd70c06fb60 msgr2=0x7fd70c15dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 shutdown_connections 2024-04-25T00:11:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.469+0000 7fd7118b3640 1 -- 172.21.15.113:0/1604719874 wait complete. 2024-04-25T00:11:06.481 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T00:11:06.481 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:06.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.582+0000 7f641e1a0640 1 Processor -- start 2024-04-25T00:11:06.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.582+0000 7f641e1a0640 1 -- start start 2024-04-25T00:11:06.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641e1a0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6418161bc0 0x7f64180ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641e1a0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64180cf8c0 0x7f64180d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641e1a0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6418161fa0 con 0x7f64180d0c90 2024-04-25T00:11:06.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641e1a0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f64180b6550 con 0x7f64180cf8c0 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641e1a0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64180d3e20 con 0x7f6418161bc0 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641c99d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64180cf8c0 0x7f64180d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641d99f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f64180d0c90 0x7f64180d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40950/0 (socket says 172.21.15.113:40950) 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641d99f640 1 -- 172.21.15.113:0/1220442344 learned_addr learned my addr 172.21.15.113:0/1220442344 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641d19e640 1 --2- 172.21.15.113:0/1220442344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6418161bc0 0x7f64180ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641c99d640 1 -- 172.21.15.113:0/1220442344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6418161bc0 msgr2=0x7f64180ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641c99d640 1 --2- 172.21.15.113:0/1220442344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6418161bc0 0x7f64180ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641c99d640 1 -- 172.21.15.113:0/1220442344 >> v1:172.21.15.113:6789/0 conn(0x7f64180d0c90 legacy=0x7f64180d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641c99d640 1 -- 172.21.15.113:0/1220442344 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64180d1120 con 0x7f64180cf8c0 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f641d19e640 1 --2- 172.21.15.113:0/1220442344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6418161bc0 0x7f64180ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.583+0000 7f640ffff640 1 -- 172.21.15.113:0/1220442344 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659163696 0 0) 0x7f6418161fa0 con 0x7f64180d0c90 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.584+0000 7f641c99d640 1 --2- 172.21.15.113:0/1220442344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180cf8c0 0x7f64180d0750 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f64080097a0 tx=0x7f640802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da4b5d8baabb052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.584+0000 7f640ffff640 1 -- 172.21.15.113:0/1220442344 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408037070 con 0x7f64180cf8c0 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.584+0000 7f640ffff640 1 -- 172.21.15.113:0/1220442344 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640802adb0 con 0x7f64180cf8c0 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.584+0000 7f640ffff640 1 -- 172.21.15.113:0/1220442344 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408032660 con 0x7f64180cf8c0 2024-04-25T00:11:06.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 -- 172.21.15.113:0/1220442344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180cf8c0 msgr2=0x7f64180d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 --2- 172.21.15.113:0/1220442344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180cf8c0 0x7f64180d0750 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f64080097a0 tx=0x7f640802a060 comp rx=0 tx=0).stop 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 -- 172.21.15.113:0/1220442344 shutdown_connections 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 --2- 172.21.15.113:0/1220442344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180cf8c0 0x7f64180d0750 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 --2- 172.21.15.113:0/1220442344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6418161bc0 0x7f64180ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 -- 172.21.15.113:0/1220442344 >> 172.21.15.113:0/1220442344 conn(0x7f641806b2d0 msgr2=0x7f641806b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 -- 172.21.15.113:0/1220442344 shutdown_connections 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.585+0000 7f641e1a0640 1 -- 172.21.15.113:0/1220442344 wait complete. 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.586+0000 7f641e1a0640 1 Processor -- start 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.586+0000 7f641e1a0640 1 -- start start 2024-04-25T00:11:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641e1a0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64180cf8c0 0x7f64181575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641e1a0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180d0c90 0x7f6418159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641e1a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6418161bc0 0x7f6418160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641e1a0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f64180b5f00 con 0x7f64180d0c90 2024-04-25T00:11:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641e1a0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64180b5c20 con 0x7f6418161bc0 2024-04-25T00:11:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641e1a0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64180b61e0 con 0x7f64180cf8c0 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180d0c90 0x7f6418159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180d0c90 0x7f6418159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50808/0 (socket says 172.21.15.113:50808) 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 -- 172.21.15.113:0/2774045534 learned_addr learned my addr 172.21.15.113:0/2774045534 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641d19e640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64180cf8c0 0x7f64181575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 -- 172.21.15.113:0/2774045534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64180cf8c0 msgr2=0x7f64181575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64180cf8c0 0x7f64181575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641d99f640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6418161bc0 0x7f6418160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 -- 172.21.15.113:0/2774045534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6418161bc0 msgr2=0x7f6418160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6418161bc0 0x7f6418160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 -- 172.21.15.113:0/2774045534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64181597b0 con 0x7f64180d0c90 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641c99d640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180d0c90 0x7f6418159b40 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f6408035040 tx=0x7f64080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408041070 con 0x7f64180d0c90 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640803c670 con 0x7f64180d0c90 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408037040 con 0x7f64180d0c90 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.587+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64181581f0 con 0x7f64180d0c90 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.588+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64181587b0 con 0x7f64180d0c90 2024-04-25T00:11:06.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.588+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6418157b10 con 0x7f64180d0c90 2024-04-25T00:11:06.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.589+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f640803a070 con 0x7f64180d0c90 2024-04-25T00:11:06.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.589+0000 7f640dffb640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63fc06be30 0x7f63fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.589+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f64080aafa0 con 0x7f64180d0c90 2024-04-25T00:11:06.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.589+0000 7f641d19e640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63fc06be30 0x7f63fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.589+0000 7f640dffb640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f63fc071d90 0x7f63fc0741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.589+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 --> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6408008eb0 con 0x7f63fc071d90 2024-04-25T00:11:06.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.590+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7f64080b2050 con 0x7f64180d0c90 2024-04-25T00:11:06.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.590+0000 7f641d99f640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f63fc071d90 0x7f63fc0741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.590+0000 7f641d19e640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63fc06be30 0x7f63fc06e2f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6414008850 tx=0x7f6414008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.590+0000 7f641d99f640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f63fc071d90 0x7f63fc0741b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.591+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== osd.3 v2:172.21.15.113:6824/1975200730 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6408008eb0 con 0x7f63fc071d90 2024-04-25T00:11:06.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.603+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 --> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f64180691a0 con 0x7f63fc071d90 2024-04-25T00:11:06.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.603+0000 7f640dffb640 1 -- 172.21.15.113:0/2774045534 <== osd.3 v2:172.21.15.113:6824/1975200730 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f64180691a0 con 0x7f63fc071d90 2024-04-25T00:11:06.605 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:06.605 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:06.605 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f63fc071d90 msgr2=0x7f63fc0741b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6824/1975200730,v1:172.21.15.113:6825/1975200730] conn(0x7f63fc071d90 0x7f63fc0741b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63fc06be30 msgr2=0x7f63fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f63fc06be30 0x7f63fc06e2f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6414008850 tx=0x7f6414008040 comp rx=0 tx=0).stop 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180d0c90 msgr2=0x7f6418159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 --2- 172.21.15.113:0/2774045534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64180d0c90 0x7f6418159b40 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f6408035040 tx=0x7f64080046c0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641d99f640 1 -- 172.21.15.113:0/2774045534 reap_dead start 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 shutdown_connections 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 >> 172.21.15.113:0/2774045534 conn(0x7f641806b2d0 msgr2=0x7f64180b9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 shutdown_connections 2024-04-25T00:11:06.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.604+0000 7f641e1a0640 1 -- 172.21.15.113:0/2774045534 wait complete. 2024-04-25T00:11:06.615 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T00:11:06.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.711+0000 7fc61102c640 1 Processor -- start 2024-04-25T00:11:06.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.711+0000 7fc61102c640 1 -- start start 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc61102c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc60c13f050 0x7fc60c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc61102c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c1606a0 0x7fc60c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc61102c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc60c15d6b0 con 0x7fc60c0d04f0 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc61102c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc60c0d09a0 con 0x7fc60c13f050 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc61102c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc60c15c6a0 con 0x7fc60c1606a0 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc60ad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc60c13f050 0x7fc60c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc60a59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc60c0d04f0 0x7fc60c15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40958/0 (socket says 172.21.15.113:40958) 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc60ad9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc60c13f050 0x7fc60c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50818/0 (socket says 172.21.15.113:50818) 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.712+0000 7fc60a59c640 1 -- 172.21.15.113:0/1680444760 learned_addr learned my addr 172.21.15.113:0/1680444760 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc60b59e640 1 --2- 172.21.15.113:0/1680444760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c1606a0 0x7fc60c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc60ad9d640 1 -- 172.21.15.113:0/1680444760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c1606a0 msgr2=0x7fc60c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc60ad9d640 1 --2- 172.21.15.113:0/1680444760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c1606a0 0x7fc60c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc60ad9d640 1 -- 172.21.15.113:0/1680444760 >> v1:172.21.15.113:6789/0 conn(0x7fc60c0d04f0 legacy=0x7fc60c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc609d9b640 1 -- 172.21.15.113:0/1680444760 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366126857 0 0) 0x7fc60c15d6b0 con 0x7fc60c0d04f0 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc60ad9d640 1 -- 172.21.15.113:0/1680444760 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c15c8f0 con 0x7fc60c13f050 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc60ad9d640 1 --2- 172.21.15.113:0/1680444760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c13f050 0x7fc60c0cffb0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc6040097a0 tx=0x7fc60402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3ad05dd5a1bc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc609d9b640 1 -- 172.21.15.113:0/1680444760 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604037070 con 0x7fc60c13f050 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.713+0000 7fc609d9b640 1 -- 172.21.15.113:0/1680444760 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60402ae50 con 0x7fc60c13f050 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc609d9b640 1 -- 172.21.15.113:0/1680444760 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604032660 con 0x7fc60c13f050 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 -- 172.21.15.113:0/1680444760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c13f050 msgr2=0x7fc60c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 --2- 172.21.15.113:0/1680444760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c13f050 0x7fc60c0cffb0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc6040097a0 tx=0x7fc60402a100 comp rx=0 tx=0).stop 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 -- 172.21.15.113:0/1680444760 shutdown_connections 2024-04-25T00:11:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 --2- 172.21.15.113:0/1680444760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c1606a0 0x7fc60c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 --2- 172.21.15.113:0/1680444760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c13f050 0x7fc60c0cffb0 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 -- 172.21.15.113:0/1680444760 >> 172.21.15.113:0/1680444760 conn(0x7fc60c06b0f0 msgr2=0x7fc60c06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 -- 172.21.15.113:0/1680444760 shutdown_connections 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.714+0000 7fc61102c640 1 -- 172.21.15.113:0/1680444760 wait complete. 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.715+0000 7fc61102c640 1 Processor -- start 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.715+0000 7fc61102c640 1 -- start start 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc61102c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c0d04f0 0x7fc60c13ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc61102c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc60c13f050 0x7fc60c143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc61102c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c1606a0 0x7fc60c143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc61102c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c15d190 con 0x7fc60c1606a0 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc61102c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c15ceb0 con 0x7fc60c13f050 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc61102c640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc60c13f430 con 0x7fc60c0d04f0 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc60b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c1606a0 0x7fc60c143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc60b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c1606a0 0x7fc60c143d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50822/0 (socket says 172.21.15.113:50822) 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc60b59e640 1 -- 172.21.15.113:0/3195211071 learned_addr learned my addr 172.21.15.113:0/3195211071 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc60a59c640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc60c13f050 0x7fc60c143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.716+0000 7fc60ad9d640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c0d04f0 0x7fc60c13ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc60b59e640 1 -- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c0d04f0 msgr2=0x7fc60c13ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc60b59e640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c0d04f0 0x7fc60c13ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc60b59e640 1 -- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc60c13f050 msgr2=0x7fc60c143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc60b59e640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc60c13f050 0x7fc60c143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc60b59e640 1 -- 172.21.15.113:0/3195211071 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c1e5cd0 con 0x7fc60c1606a0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc60b59e640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c1606a0 0x7fc60c143d70 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00d5d0 tx=0x7fc5fc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc5f37fe640 1 -- 172.21.15.113:0/3195211071 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0125f0 con 0x7fc60c1606a0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc5f37fe640 1 -- 172.21.15.113:0/3195211071 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5fc019070 con 0x7fc60c1606a0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c1443a0 con 0x7fc60c1606a0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc5f37fe640 1 -- 172.21.15.113:0/3195211071 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0129f0 con 0x7fc60c1606a0 2024-04-25T00:11:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.717+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c144960 con 0x7fc60c1606a0 2024-04-25T00:11:06.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.718+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c0691a0 con 0x7fc60c1606a0 2024-04-25T00:11:06.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.718+0000 7fc5f37fe640 1 -- 172.21.15.113:0/3195211071 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5fc012b90 con 0x7fc60c1606a0 2024-04-25T00:11:06.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.719+0000 7fc5f37fe640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc5f806be30 0x7fc5f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.719+0000 7fc60ad9d640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc5f806be30 0x7fc5f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.719+0000 7fc5f37fe640 1 -- 172.21.15.113:0/3195211071 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fc5fc08ae00 con 0x7fc60c1606a0 2024-04-25T00:11:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.719+0000 7fc60ad9d640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc5f806be30 0x7fc5f806e2f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc604034000 tx=0x7fc60402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.723+0000 7fc5f37fe640 1 -- 172.21.15.113:0/3195211071 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc5fc059410 con 0x7fc60c1606a0 2024-04-25T00:11:06.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.895+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fc60c0cfd90 con 0x7fc5f806be30 2024-04-25T00:11:06.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.895+0000 7fc5f37fe640 1 -- 172.21.15.113:0/3195211071 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc60c0cfd90 con 0x7fc5f806be30 2024-04-25T00:11:06.896 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T00:11:06.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc5f806be30 msgr2=0x7fc5f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc5f806be30 0x7fc5f806e2f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc604034000 tx=0x7fc60402e000 comp rx=0 tx=0).stop 2024-04-25T00:11:06.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c1606a0 msgr2=0x7fc60c143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:06.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c1606a0 0x7fc60c143d70 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00d5d0 tx=0x7fc5fc00da90 comp rx=0 tx=0).stop 2024-04-25T00:11:06.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 shutdown_connections 2024-04-25T00:11:06.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fc5f806be30 0x7fc5f806e2f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc60c13f050 0x7fc60c143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc60c1606a0 0x7fc60c143d70 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 --2- 172.21.15.113:0/3195211071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc60c0d04f0 0x7fc60c13ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:06.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 >> 172.21.15.113:0/3195211071 conn(0x7fc60c06b0f0 msgr2=0x7fc60c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:06.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 shutdown_connections 2024-04-25T00:11:06.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:06.897+0000 7fc61102c640 1 -- 172.21.15.113:0/3195211071 wait complete. 2024-04-25T00:11:06.908 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T00:11:06.909 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.009+0000 7faae4d7e640 1 Processor -- start 2024-04-25T00:11:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.009+0000 7faae4d7e640 1 -- start start 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faae4d7e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faae0145890 0x7faae00ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faae4d7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae00ced20 0x7faae006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faae4d7e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae00cf3a0 con 0x7faae006e610 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faae4d7e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faae00cf180 con 0x7faae0145890 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faae4d7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faae0145c70 con 0x7faae00ced20 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faaddd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae00ced20 0x7faae006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faade59c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faae0145890 0x7faae00ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faaddd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae00ced20 0x7faae006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.113:48186/0 (socket says 172.21.15.113:48186) 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.010+0000 7faaddd9b640 1 -- 172.21.15.113:0/1772930256 learned_addr learned my addr 172.21.15.113:0/1772930256 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.011+0000 7faaddd9b640 1 -- 172.21.15.113:0/1772930256 >> v2:172.21.15.178:3300/0 conn(0x7faae0145890 msgr2=0x7faae00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:07.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.011+0000 7faaddd9b640 1 --2- 172.21.15.113:0/1772930256 >> v2:172.21.15.178:3300/0 conn(0x7faae0145890 0x7faae00ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.011+0000 7faaddd9b640 1 -- 172.21.15.113:0/1772930256 >> v1:172.21.15.178:6789/0 conn(0x7faae006e610 legacy=0x7faae015fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.011+0000 7faaddd9b640 1 -- 172.21.15.113:0/1772930256 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae00cf680 con 0x7faae00ced20 2024-04-25T00:11:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.011+0000 7faaddd9b640 1 --2- 172.21.15.113:0/1772930256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae00ced20 0x7faae006e0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e298333be239fc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faadd59a640 1 -- 172.21.15.113:0/1772930256 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4037070 con 0x7faae00ced20 2024-04-25T00:11:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faadd59a640 1 -- 172.21.15.113:0/1772930256 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad4032440 con 0x7faae00ced20 2024-04-25T00:11:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faadd59a640 1 -- 172.21.15.113:0/1772930256 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4032810 con 0x7faae00ced20 2024-04-25T00:11:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 -- 172.21.15.113:0/1772930256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae00ced20 msgr2=0x7faae006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 --2- 172.21.15.113:0/1772930256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae00ced20 0x7faae006e0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad402a270 comp rx=0 tx=0).stop 2024-04-25T00:11:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 -- 172.21.15.113:0/1772930256 shutdown_connections 2024-04-25T00:11:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 --2- 172.21.15.113:0/1772930256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae00ced20 0x7faae006e0d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 --2- 172.21.15.113:0/1772930256 >> v2:172.21.15.178:3300/0 conn(0x7faae0145890 0x7faae00ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 -- 172.21.15.113:0/1772930256 >> 172.21.15.113:0/1772930256 conn(0x7faae00b1b30 msgr2=0x7faae00b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 -- 172.21.15.113:0/1772930256 shutdown_connections 2024-04-25T00:11:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.012+0000 7faae4d7e640 1 -- 172.21.15.113:0/1772930256 wait complete. 2024-04-25T00:11:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 Processor -- start 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 -- start start 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faae006e610 0x7faae015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faae00ced20 0x7faae013e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae0145890 0x7faae013e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faae0070f10 con 0x7faae006e610 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faae006ea60 con 0x7faae00ced20 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faae4d7e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faae00b53b0 con 0x7faae0145890 2024-04-25T00:11:07.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faade59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faae006e610 0x7faae015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faade59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faae006e610 0x7faae015b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50834/0 (socket says 172.21.15.113:50834) 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.014+0000 7faade59c640 1 -- 172.21.15.113:0/2479975608 learned_addr learned my addr 172.21.15.113:0/2479975608 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faaddd9b640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faae00ced20 0x7faae013e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faaded9d640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae0145890 0x7faae013e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faade59c640 1 -- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae0145890 msgr2=0x7faae013e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faade59c640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faae0145890 0x7faae013e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faade59c640 1 -- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faae00ced20 msgr2=0x7faae013e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faade59c640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faae00ced20 0x7faae013e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faade59c640 1 -- 172.21.15.113:0/2479975608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae01f28b0 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faade59c640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faae006e610 0x7faae015b840 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7faad000efc0 tx=0x7faad000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad0018020 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad0014480 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad0014850 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae01f52f0 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.015+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae01f58b0 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.016+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faae01f5100 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.016+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faad0005d10 con 0x7faae006e610 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaceffd640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faac006be30 0x7faac006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7faad008bd90 con 0x7faae006e610 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaddd9b640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faac006be30 0x7faac006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaceffd640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7faac0071d90 0x7faac00741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 --> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faad0012e50 con 0x7faac0071d90 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7faad008c220 con 0x7faae006e610 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaded9d640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7faac0071d90 0x7faac00741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.017+0000 7faaddd9b640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faac006be30 0x7faac006e2f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7faad4034000 tx=0x7faad402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.018+0000 7faaded9d640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7faac0071d90 0x7faac00741b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.018+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== osd.4 v2:172.21.15.178:6808/146317928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faad0012e50 con 0x7faac0071d90 2024-04-25T00:11:07.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.031+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 --> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7faae00691a0 con 0x7faac0071d90 2024-04-25T00:11:07.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faaceffd640 1 -- 172.21.15.113:0/2479975608 <== osd.4 v2:172.21.15.178:6808/146317928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7faae00691a0 con 0x7faac0071d90 2024-04-25T00:11:07.033 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:07.033 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:07.033 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7faac0071d90 msgr2=0x7faac00741b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faae4d7e640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.178:6808/146317928,v1:172.21.15.178:6809/146317928] conn(0x7faac0071d90 0x7faac00741b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faac006be30 msgr2=0x7faac006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faae4d7e640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7faac006be30 0x7faac006e2f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7faad4034000 tx=0x7faad402e000 comp rx=0 tx=0).stop 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faae006e610 msgr2=0x7faae015b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faae4d7e640 1 --2- 172.21.15.113:0/2479975608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faae006e610 0x7faae015b840 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7faad000efc0 tx=0x7faad000c490 comp rx=0 tx=0).stop 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.032+0000 7faaded9d640 1 -- 172.21.15.113:0/2479975608 reap_dead start 2024-04-25T00:11:07.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.033+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 shutdown_connections 2024-04-25T00:11:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.033+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 >> 172.21.15.113:0/2479975608 conn(0x7faae00b1b30 msgr2=0x7faae0143e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.033+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 shutdown_connections 2024-04-25T00:11:07.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.033+0000 7faae4d7e640 1 -- 172.21.15.113:0/2479975608 wait complete. 2024-04-25T00:11:07.043 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T00:11:07.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.141+0000 7f57236ee640 1 Processor -- start 2024-04-25T00:11:07.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f57236ee640 1 -- start start 2024-04-25T00:11:07.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f57236ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f57236ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f571c0b63b0 0x7f571c0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f57236ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f571c0d2e00 con 0x7f571c0d28c0 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f57236ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f571c0d30e0 con 0x7f571c0b63b0 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f57236ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f571c0d33c0 con 0x7f571c15e510 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f5721c8b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f571c0b63b0 0x7f571c0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f5720c89640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f571c0d28c0 0x7f571c0b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40962/0 (socket says 172.21.15.113:40962) 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.142+0000 7f5720c89640 1 -- 172.21.15.113:0/3444216463 learned_addr learned my addr 172.21.15.113:0/3444216463 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f5713fff640 1 -- 172.21.15.113:0/3444216463 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162017825 0 0) 0x7f571c0d2e00 con 0x7f571c0d28c0 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f5713fff640 1 -- 172.21.15.113:0/3444216463 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5708003520 con 0x7f571c0d28c0 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f572148a640 1 --2- 172.21.15.113:0/3444216463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f5721c8b640 1 -- 172.21.15.113:0/3444216463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 msgr2=0x7f571c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f5721c8b640 1 --2- 172.21.15.113:0/3444216463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f5721c8b640 1 -- 172.21.15.113:0/3444216463 >> v1:172.21.15.113:6789/0 conn(0x7f571c0d28c0 legacy=0x7f571c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f5721c8b640 1 -- 172.21.15.113:0/3444216463 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c0d36a0 con 0x7f571c0b63b0 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f572148a640 1 --2- 172.21.15.113:0/3444216463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.143+0000 7f5721c8b640 1 --2- 172.21.15.113:0/3444216463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c0b73c0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f57140097a0 tx=0x7f571402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d45edce3ee8cf77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f5713fff640 1 -- 172.21.15.113:0/3444216463 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714037070 con 0x7f571c0b63b0 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f5713fff640 1 -- 172.21.15.113:0/3444216463 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571402ae50 con 0x7f571c0b63b0 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f5713fff640 1 -- 172.21.15.113:0/3444216463 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714032660 con 0x7f571c0b63b0 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 -- 172.21.15.113:0/3444216463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 msgr2=0x7f571c0b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 --2- 172.21.15.113:0/3444216463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c0b73c0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f57140097a0 tx=0x7f571402a100 comp rx=0 tx=0).stop 2024-04-25T00:11:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 -- 172.21.15.113:0/3444216463 shutdown_connections 2024-04-25T00:11:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 --2- 172.21.15.113:0/3444216463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c0b73c0 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 --2- 172.21.15.113:0/3444216463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 -- 172.21.15.113:0/3444216463 >> 172.21.15.113:0/3444216463 conn(0x7f571c06b720 msgr2=0x7f571c0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 -- 172.21.15.113:0/3444216463 shutdown_connections 2024-04-25T00:11:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.144+0000 7f57236ee640 1 -- 172.21.15.113:0/3444216463 wait complete. 2024-04-25T00:11:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.145+0000 7f57236ee640 1 Processor -- start 2024-04-25T00:11:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.145+0000 7f57236ee640 1 -- start start 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f57236ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f57236ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571c0d28c0 0x7f571c0ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f57236ee640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f57236ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0b9dd0 con 0x7f571c0b63b0 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f57236ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0d3ea0 con 0x7f571c0d28c0 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f57236ee640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f571c0b7900 con 0x7f571c15e510 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c14c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50848/0 (socket says 172.21.15.113:50848) 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 -- 172.21.15.113:0/311100774 learned_addr learned my addr 172.21.15.113:0/311100774 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f5720c89640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571c0d28c0 0x7f571c0ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f5721c8b640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 -- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 msgr2=0x7f571c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 -- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571c0d28c0 msgr2=0x7f571c0ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571c0d28c0 0x7f571c0ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f572148a640 1 -- 172.21.15.113:0/311100774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c1586c0 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f5721c8b640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.146+0000 7f5720c89640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571c0d28c0 0x7f571c0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.147+0000 7f572148a640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c14c6b0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f571800ce00 tx=0x7f571800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.147+0000 7f5711ffb640 1 -- 172.21.15.113:0/311100774 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571800f890 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.147+0000 7f5711ffb640 1 -- 172.21.15.113:0/311100774 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571800fa30 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.147+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571c1589a0 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.147+0000 7f5711ffb640 1 -- 172.21.15.113:0/311100774 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571800fe00 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.147+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571c0c7e80 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.148+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571c0691a0 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.148+0000 7f5711ffb640 1 -- 172.21.15.113:0/311100774 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5718018400 con 0x7f571c0b63b0 2024-04-25T00:11:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.149+0000 7f5711ffb640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570006be30 0x7f570006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.149+0000 7f5711ffb640 1 -- 172.21.15.113:0/311100774 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f571808b9e0 con 0x7f571c0b63b0 2024-04-25T00:11:07.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.149+0000 7f5720c89640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570006be30 0x7f570006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.149+0000 7f5720c89640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570006be30 0x7f570006e2f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f571c0ceb70 tx=0x7f570c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.153+0000 7f5711ffb640 1 -- 172.21.15.113:0/311100774 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5718014030 con 0x7f571c0b63b0 2024-04-25T00:11:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.324+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f571c0b6c30 con 0x7f570006be30 2024-04-25T00:11:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.324+0000 7f5711ffb640 1 -- 172.21.15.113:0/311100774 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f571c0b6c30 con 0x7f570006be30 2024-04-25T00:11:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T00:11:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570006be30 msgr2=0x7f570006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570006be30 0x7f570006e2f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f571c0ceb70 tx=0x7f570c009040 comp rx=0 tx=0).stop 2024-04-25T00:11:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 msgr2=0x7f571c14c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c14c6b0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f571800ce00 tx=0x7f571800a650 comp rx=0 tx=0).stop 2024-04-25T00:11:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 shutdown_connections 2024-04-25T00:11:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f570006be30 0x7f570006e2f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571c0d28c0 0x7f571c0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f571c15e510 0x7f571c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.326+0000 7f57236ee640 1 --2- 172.21.15.113:0/311100774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c0b63b0 0x7f571c14c6b0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.327+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 >> 172.21.15.113:0/311100774 conn(0x7f571c06b720 msgr2=0x7f571c0b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.327+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 shutdown_connections 2024-04-25T00:11:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.327+0000 7f57236ee640 1 -- 172.21.15.113:0/311100774 wait complete. 2024-04-25T00:11:07.338 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T00:11:07.338 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:07.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.437+0000 7f3069f42640 1 Processor -- start 2024-04-25T00:11:07.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.437+0000 7f3069f42640 1 -- start start 2024-04-25T00:11:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f3069f42640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f30640b9bb0 0x7f30640b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f3069f42640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f30640b61e0 0x7f30640d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f3069f42640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30641626f0 con 0x7f306415e510 2024-04-25T00:11:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f3069f42640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f30641629d0 con 0x7f30640b61e0 2024-04-25T00:11:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f3069f42640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3064162cb0 con 0x7f30640b9bb0 2024-04-25T00:11:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f30637fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f306415e510 0x7f3064161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40978/0 (socket says 172.21.15.113:40978) 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f30637fe640 1 -- 172.21.15.113:0/1025894483 learned_addr learned my addr 172.21.15.113:0/1025894483 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.438+0000 7f3063fff640 1 --2- 172.21.15.113:0/1025894483 >> v2:172.21.15.113:3300/0 conn(0x7f30640b61e0 0x7f30640d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f30627fc640 1 -- 172.21.15.113:0/1025894483 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867151420 0 0) 0x7f30641626f0 con 0x7f306415e510 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f30627fc640 1 -- 172.21.15.113:0/1025894483 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f304c003520 con 0x7f306415e510 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f3062ffd640 1 --2- 172.21.15.113:0/1025894483 >> v2:172.21.15.178:3300/0 conn(0x7f30640b9bb0 0x7f30640b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f3063fff640 1 -- 172.21.15.113:0/1025894483 >> v2:172.21.15.178:3300/0 conn(0x7f30640b9bb0 msgr2=0x7f30640b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f3063fff640 1 --2- 172.21.15.113:0/1025894483 >> v2:172.21.15.178:3300/0 conn(0x7f30640b9bb0 0x7f30640b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f3063fff640 1 -- 172.21.15.113:0/1025894483 >> v1:172.21.15.113:6789/0 conn(0x7f306415e510 legacy=0x7f3064161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f3063fff640 1 -- 172.21.15.113:0/1025894483 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30640d2660 con 0x7f30640b61e0 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f3062ffd640 1 --2- 172.21.15.113:0/1025894483 >> v2:172.21.15.178:3300/0 conn(0x7f30640b9bb0 0x7f30640b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.439+0000 7f3063fff640 1 --2- 172.21.15.113:0/1025894483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30640d2e60 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f30640b3c50 tx=0x7f305c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4922ce40f39c3368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f30627fc640 1 -- 172.21.15.113:0/1025894483 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c007740 con 0x7f30640b61e0 2024-04-25T00:11:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f30627fc640 1 -- 172.21.15.113:0/1025894483 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305c0078e0 con 0x7f30640b61e0 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f30627fc640 1 -- 172.21.15.113:0/1025894483 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c007cb0 con 0x7f30640b61e0 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 -- 172.21.15.113:0/1025894483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 msgr2=0x7f30640d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 --2- 172.21.15.113:0/1025894483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30640d2e60 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f30640b3c50 tx=0x7f305c02a5f0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 -- 172.21.15.113:0/1025894483 shutdown_connections 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 --2- 172.21.15.113:0/1025894483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30640d2e60 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 --2- 172.21.15.113:0/1025894483 >> v2:172.21.15.178:3300/0 conn(0x7f30640b9bb0 0x7f30640b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 -- 172.21.15.113:0/1025894483 >> 172.21.15.113:0/1025894483 conn(0x7f306406b720 msgr2=0x7f30640d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 -- 172.21.15.113:0/1025894483 shutdown_connections 2024-04-25T00:11:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.440+0000 7f3069f42640 1 -- 172.21.15.113:0/1025894483 wait complete. 2024-04-25T00:11:07.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.441+0000 7f3069f42640 1 Processor -- start 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.441+0000 7f3069f42640 1 -- start start 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3069f42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30641439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3069f42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30640b9bb0 0x7f30640cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3069f42640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f306415e510 0x7f30640cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3069f42640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f30640b7540 con 0x7f30640b61e0 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3069f42640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30640b73c0 con 0x7f30640b9bb0 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3069f42640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f30640d33a0 con 0x7f306415e510 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f30637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30641439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f30637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30641439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50860/0 (socket says 172.21.15.113:50860) 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f30637fe640 1 -- 172.21.15.113:0/2359023835 learned_addr learned my addr 172.21.15.113:0/2359023835 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3062ffd640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30640b9bb0 0x7f30640cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f30637fe640 1 -- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f306415e510 msgr2=0x7f30640cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.442+0000 7f3063fff640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f306415e510 0x7f30640cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f30637fe640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f306415e510 0x7f30640cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f30637fe640 1 -- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30640b9bb0 msgr2=0x7f30640cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f30637fe640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30640b9bb0 0x7f30640cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f30637fe640 1 -- 172.21.15.113:0/2359023835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30640cb830 con 0x7f30640b61e0 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f3062ffd640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30640b9bb0 0x7f30640cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f30637fe640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30641439f0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f305400c390 tx=0x7f305400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054017070 con 0x7f30640b61e0 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3054005ce0 con 0x7f30640b61e0 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3064144180 con 0x7f30640b61e0 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540072f0 con 0x7f30640b61e0 2024-04-25T00:11:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.443+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30640cd140 con 0x7f30640b61e0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.444+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3064143f30 con 0x7f30640b61e0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.444+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f305401a070 con 0x7f30640b61e0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f304bfff640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f305006be30 0x7f305006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f3054089f30 con 0x7f30640b61e0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f3062ffd640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f305006be30 0x7f305006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f304bfff640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f3050071d90 0x7f30500741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 --> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3054015e70 con 0x7f3050071d90 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7f305408c5d0 con 0x7f30640b61e0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f3062ffd640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f305006be30 0x7f305006e2f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3064158bb0 tx=0x7f3058008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.445+0000 7f3063fff640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f3050071d90 0x7f30500741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.446+0000 7f3063fff640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f3050071d90 0x7f30500741b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.446+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== osd.5 v2:172.21.15.178:6816/2661350487 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3054015e70 con 0x7f3050071d90 2024-04-25T00:11:07.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.459+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 --> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f30640691a0 con 0x7f3050071d90 2024-04-25T00:11:07.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f304bfff640 1 -- 172.21.15.113:0/2359023835 <== osd.5 v2:172.21.15.178:6816/2661350487 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f30640691a0 con 0x7f3050071d90 2024-04-25T00:11:07.461 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:07.461 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:07.461 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:07.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f3050071d90 msgr2=0x7f30500741b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.178:6816/2661350487,v1:172.21.15.178:6817/2661350487] conn(0x7f3050071d90 0x7f30500741b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f305006be30 msgr2=0x7f305006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f305006be30 0x7f305006e2f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3064158bb0 tx=0x7f3058008040 comp rx=0 tx=0).stop 2024-04-25T00:11:07.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 msgr2=0x7f30641439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 --2- 172.21.15.113:0/2359023835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30640b61e0 0x7f30641439f0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f305400c390 tx=0x7f305400c850 comp rx=0 tx=0).stop 2024-04-25T00:11:07.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3063fff640 1 -- 172.21.15.113:0/2359023835 reap_dead start 2024-04-25T00:11:07.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 shutdown_connections 2024-04-25T00:11:07.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.460+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 >> 172.21.15.113:0/2359023835 conn(0x7f306406b720 msgr2=0x7f30640d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.461+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 shutdown_connections 2024-04-25T00:11:07.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.461+0000 7f3069f42640 1 -- 172.21.15.113:0/2359023835 wait complete. 2024-04-25T00:11:07.472 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T00:11:07.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.569+0000 7f949eed2640 1 Processor -- start 2024-04-25T00:11:07.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.569+0000 7f949eed2640 1 -- start start 2024-04-25T00:11:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949eed2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f949815e2d0 0x7f949815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949eed2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f949815f4f0 con 0x7f94980b4a40 2024-04-25T00:11:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949eed2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f949815f7d0 con 0x7f94980b3080 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949eed2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94980cde00 con 0x7f949815e2d0 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949d46f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f94980b4a40 0x7f94980ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48944/0 (socket says 172.21.15.113:48944) 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949d46f640 1 -- 172.21.15.113:0/1098997318 learned_addr learned my addr 172.21.15.113:0/1098997318 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949cc6e640 1 --2- 172.21.15.113:0/1098997318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f949815e2d0 0x7f949815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2500265260 0 0) 0x7f949815f4f0 con 0x7f94980b4a40 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9488003520 con 0x7f94980b4a40 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2037247558 0 0) 0x7f949815f7d0 con 0x7f94980b3080 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f949815f4f0 con 0x7f94980b3080 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949cc6e640 1 -- 172.21.15.113:0/1098997318 >> v1:172.21.15.178:6789/0 conn(0x7f94980b3080 legacy=0x7f94980b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949cc6e640 1 -- 172.21.15.113:0/1098997318 >> v1:172.21.15.113:6789/0 conn(0x7f94980b4a40 legacy=0x7f94980ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 890586426 0 0) 0x7f9488003520 con 0x7f94980b4a40 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.570+0000 7f949cc6e640 1 -- 172.21.15.113:0/1098997318 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949815fab0 con 0x7f949815e2d0 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.571+0000 7f949cc6e640 1 --2- 172.21.15.113:0/1098997318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f949815e2d0 0x7f949815ebf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94980cdf80 tx=0x7f948c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b1ebcd931b05fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.571+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c037070 con 0x7f949815e2d0 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.571+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948c02ac10 con 0x7f949815e2d0 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.571+0000 7f94977fe640 1 -- 172.21.15.113:0/1098997318 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94900019f0 con 0x7f949815e2d0 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.572+0000 7f949eed2640 1 -- 172.21.15.113:0/1098997318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f949815e2d0 msgr2=0x7f949815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.572+0000 7f949eed2640 1 --2- 172.21.15.113:0/1098997318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f949815e2d0 0x7f949815ebf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94980cdf80 tx=0x7f948c02a180 comp rx=0 tx=0).stop 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.572+0000 7f949eed2640 1 -- 172.21.15.113:0/1098997318 shutdown_connections 2024-04-25T00:11:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.572+0000 7f949eed2640 1 --2- 172.21.15.113:0/1098997318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f949815e2d0 0x7f949815ebf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.572+0000 7f949eed2640 1 -- 172.21.15.113:0/1098997318 >> 172.21.15.113:0/1098997318 conn(0x7f949806b0f0 msgr2=0x7f949806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.572+0000 7f949eed2640 1 -- 172.21.15.113:0/1098997318 shutdown_connections 2024-04-25T00:11:07.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.572+0000 7f949eed2640 1 -- 172.21.15.113:0/1098997318 wait complete. 2024-04-25T00:11:07.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.573+0000 7f949eed2640 1 Processor -- start 2024-04-25T00:11:07.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.573+0000 7f949eed2640 1 -- start start 2024-04-25T00:11:07.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949eed2640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94980b3080 0x7f949814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949eed2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94980b4a40 0x7f949815be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949eed2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f949815e2d0 0x7f94980d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949eed2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f949815f2b0 con 0x7f94980b4a40 2024-04-25T00:11:07.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949eed2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f949815f130 con 0x7f949815e2d0 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949eed2640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f949815e6b0 con 0x7f94980b3080 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94980b4a40 0x7f949815be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94980b4a40 0x7f949815be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39152/0 (socket says 172.21.15.113:39152) 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 -- 172.21.15.113:0/1628693467 learned_addr learned my addr 172.21.15.113:0/1628693467 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949cc6e640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94980b3080 0x7f949814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f949d46f640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f949815e2d0 0x7f94980d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 -- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94980b3080 msgr2=0x7f949814f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94980b3080 0x7f949814f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 -- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f949815e2d0 msgr2=0x7f94980d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f949815e2d0 0x7f94980d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 -- 172.21.15.113:0/1628693467 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94981f0b00 con 0x7f94980b4a40 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f9497fff640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94980b4a40 0x7f949815be60 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f948400e730 tx=0x7f948400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.574+0000 7f94957fa640 1 -- 172.21.15.113:0/1628693467 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94840052d0 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.575+0000 7f94957fa640 1 -- 172.21.15.113:0/1628693467 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9484005470 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.575+0000 7f94957fa640 1 -- 172.21.15.113:0/1628693467 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948400cc60 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.575+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94981f0de0 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.575+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94981f13a0 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.576+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94980691a0 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.576+0000 7f94957fa640 1 -- 172.21.15.113:0/1628693467 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94840035f0 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.577+0000 7f94957fa640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.577+0000 7f949cc6e640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.577+0000 7f94957fa640 1 -- 172.21.15.113:0/1628693467 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f948408cf20 con 0x7f94980b4a40 2024-04-25T00:11:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.577+0000 7f949cc6e640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f948c034000 tx=0x7f948c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.581+0000 7f94957fa640 1 -- 172.21.15.113:0/1628693467 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9484016020 con 0x7f94980b4a40 2024-04-25T00:11:07.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.753+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f9498072050 con 0x7f947c06be30 2024-04-25T00:11:07.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.754+0000 7f94957fa640 1 -- 172.21.15.113:0/1628693467 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9498072050 con 0x7f947c06be30 2024-04-25T00:11:07.754 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T00:11:07.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 msgr2=0x7f947c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f948c034000 tx=0x7f948c02e000 comp rx=0 tx=0).stop 2024-04-25T00:11:07.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94980b4a40 msgr2=0x7f949815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94980b4a40 0x7f949815be60 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f948400e730 tx=0x7f948400ebf0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 shutdown_connections 2024-04-25T00:11:07.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f947c06be30 0x7f947c06e2f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f949815e2d0 0x7f94980d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94980b4a40 0x7f949815be60 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 --2- 172.21.15.113:0/1628693467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94980b3080 0x7f949814f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 >> 172.21.15.113:0/1628693467 conn(0x7f949806b0f0 msgr2=0x7f94980afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 shutdown_connections 2024-04-25T00:11:07.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.756+0000 7f949eed2640 1 -- 172.21.15.113:0/1628693467 wait complete. 2024-04-25T00:11:07.767 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T00:11:07.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:07.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.867+0000 7f5d77f61640 1 Processor -- start 2024-04-25T00:11:07.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.867+0000 7f5d77f61640 1 -- start start 2024-04-25T00:11:07.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d77f61640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5d70144a80 0x7f5d700b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d77f61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d700b3ff0 0x7f5d7006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d77f61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d7006e610 0x7f5d700cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d77f61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5d70072f70 con 0x7f5d7006e610 2024-04-25T00:11:07.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d77f61640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5d700b2fd0 con 0x7f5d70144a80 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d77f61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d700b32b0 con 0x7f5d700b3ff0 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d7006e610 0x7f5d700cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d7006e610 0x7f5d700cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39162/0 (socket says 172.21.15.113:39162) 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 -- 172.21.15.113:0/752462576 learned_addr learned my addr 172.21.15.113:0/752462576 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d75cfd640 1 --2- 172.21.15.113:0/752462576 >> v2:172.21.15.178:3300/0 conn(0x7f5d70144a80 0x7f5d700b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d754fc640 1 --2- 172.21.15.113:0/752462576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d700b3ff0 0x7f5d7006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 -- 172.21.15.113:0/752462576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d700b3ff0 msgr2=0x7f5d7006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 --2- 172.21.15.113:0/752462576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d700b3ff0 0x7f5d7006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 -- 172.21.15.113:0/752462576 >> v2:172.21.15.178:3300/0 conn(0x7f5d70144a80 msgr2=0x7f5d700b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 --2- 172.21.15.113:0/752462576 >> v2:172.21.15.178:3300/0 conn(0x7f5d70144a80 0x7f5d700b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d764fe640 1 -- 172.21.15.113:0/752462576 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d700b3590 con 0x7f5d7006e610 2024-04-25T00:11:07.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.868+0000 7f5d754fc640 1 --2- 172.21.15.113:0/752462576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d700b3ff0 0x7f5d7006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:07.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d764fe640 1 --2- 172.21.15.113:0/752462576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7006e610 0x7f5d700cd630 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f5d7006bf70 tx=0x7f5d6802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ef29b00778d8f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d74cfb640 1 -- 172.21.15.113:0/752462576 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68037070 con 0x7f5d7006e610 2024-04-25T00:11:07.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d74cfb640 1 -- 172.21.15.113:0/752462576 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d68032440 con 0x7f5d7006e610 2024-04-25T00:11:07.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d74cfb640 1 -- 172.21.15.113:0/752462576 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68032810 con 0x7f5d7006e610 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d77f61640 1 -- 172.21.15.113:0/752462576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7006e610 msgr2=0x7f5d700cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d77f61640 1 --2- 172.21.15.113:0/752462576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7006e610 0x7f5d700cd630 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f5d7006bf70 tx=0x7f5d6802a140 comp rx=0 tx=0).stop 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d77f61640 1 -- 172.21.15.113:0/752462576 shutdown_connections 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d77f61640 1 --2- 172.21.15.113:0/752462576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7006e610 0x7f5d700cd630 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d77f61640 1 --2- 172.21.15.113:0/752462576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d700b3ff0 0x7f5d7006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d77f61640 1 --2- 172.21.15.113:0/752462576 >> v2:172.21.15.178:3300/0 conn(0x7f5d70144a80 0x7f5d700b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.869+0000 7f5d77f61640 1 -- 172.21.15.113:0/752462576 >> 172.21.15.113:0/752462576 conn(0x7f5d700b1b30 msgr2=0x7f5d700b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.870+0000 7f5d77f61640 1 -- 172.21.15.113:0/752462576 shutdown_connections 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.870+0000 7f5d77f61640 1 -- 172.21.15.113:0/752462576 wait complete. 2024-04-25T00:11:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 Processor -- start 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 -- start start 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d7006e610 0x7f5d70155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d700b3ff0 0x7f5d70156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d70144a80 0x7f5d70140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5d700cdf00 con 0x7f5d700b3ff0 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d700cdd80 con 0x7f5d70144a80 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d77f61640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d70072200 con 0x7f5d7006e610 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d754fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d700b3ff0 0x7f5d70156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d754fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d700b3ff0 0x7f5d70156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39164/0 (socket says 172.21.15.113:39164) 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.871+0000 7f5d754fc640 1 -- 172.21.15.113:0/1432315842 learned_addr learned my addr 172.21.15.113:0/1432315842 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d764fe640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d70144a80 0x7f5d70140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d75cfd640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d7006e610 0x7f5d70155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d754fc640 1 -- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d7006e610 msgr2=0x7f5d70155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d754fc640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d7006e610 0x7f5d70155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d754fc640 1 -- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d70144a80 msgr2=0x7f5d70140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d754fc640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d70144a80 0x7f5d70140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d754fc640 1 -- 172.21.15.113:0/1432315842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7015d890 con 0x7f5d700b3ff0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d75cfd640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d7006e610 0x7f5d70155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d764fe640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d70144a80 0x7f5d70140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d754fc640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d700b3ff0 0x7f5d70156e20 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f5d600106b0 tx=0x7f5d60010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60004230 con 0x7f5d700b3ff0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d600043f0 con 0x7f5d700b3ff0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7015db70 con 0x7f5d700b3ff0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60009920 con 0x7f5d700b3ff0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.872+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7015e130 con 0x7f5d700b3ff0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.873+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d700691a0 con 0x7f5d700b3ff0 2024-04-25T00:11:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d60009ac0 con 0x7f5d700b3ff0 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d667fc640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f5d6008cb50 con 0x7f5d700b3ff0 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d75cfd640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d667fc640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f5d54071d90 0x7f5d540741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 --> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d60013e50 con 0x7f5d54071d90 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7f5d600912c0 con 0x7f5d700b3ff0 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d75cfd640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d5406be30 0x7f5d5406e2f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5d7015e750 tx=0x7f5d6c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.874+0000 7f5d764fe640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f5d54071d90 0x7f5d540741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:07.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.875+0000 7f5d764fe640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f5d54071d90 0x7f5d540741b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:07.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.876+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== osd.6 v2:172.21.15.178:6800/1646944139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d60013e50 con 0x7f5d54071d90 2024-04-25T00:11:07.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.888+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 --> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5d700b29a0 con 0x7f5d54071d90 2024-04-25T00:11:07.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.889+0000 7f5d667fc640 1 -- 172.21.15.113:0/1432315842 <== osd.6 v2:172.21.15.178:6800/1646944139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5d700b29a0 con 0x7f5d54071d90 2024-04-25T00:11:07.890 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:07.890 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.889+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f5d54071d90 msgr2=0x7f5d540741b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.178:6800/1646944139,v1:172.21.15.178:6801/1646944139] conn(0x7f5d54071d90 0x7f5d540741b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d5406be30 msgr2=0x7f5d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f5d5406be30 0x7f5d5406e2f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5d7015e750 tx=0x7f5d6c00b040 comp rx=0 tx=0).stop 2024-04-25T00:11:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d700b3ff0 msgr2=0x7f5d70156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 --2- 172.21.15.113:0/1432315842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d700b3ff0 0x7f5d70156e20 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f5d600106b0 tx=0x7f5d60010b70 comp rx=0 tx=0).stop 2024-04-25T00:11:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d764fe640 1 -- 172.21.15.113:0/1432315842 reap_dead start 2024-04-25T00:11:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 shutdown_connections 2024-04-25T00:11:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 >> 172.21.15.113:0/1432315842 conn(0x7f5d700b1b30 msgr2=0x7f5d700af810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 shutdown_connections 2024-04-25T00:11:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.890+0000 7f5d77f61640 1 -- 172.21.15.113:0/1432315842 wait complete. 2024-04-25T00:11:07.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T00:11:07.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.997+0000 7fbb67a2a640 1 Processor -- start 2024-04-25T00:11:07.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.998+0000 7fbb67a2a640 1 -- start start 2024-04-25T00:11:08.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.998+0000 7fbb67a2a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbb600b5ca0 0x7fbb600b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.998+0000 7fbb67a2a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600d2e00 con 0x7fbb600b73c0 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.998+0000 7fbb67a2a640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600d30e0 con 0x7fbb6015e510 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.998+0000 7fbb67a2a640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbb600b6640 con 0x7fbb600b5ca0 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb65fc7640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbb600b73c0 0x7fbb600b77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48960/0 (socket says 172.21.15.113:48960) 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb65fc7640 1 -- 172.21.15.113:0/2694981922 learned_addr learned my addr 172.21.15.113:0/2694981922 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb64fc5640 1 --2- 172.21.15.113:0/2694981922 >> v2:172.21.15.178:3300/0 conn(0x7fbb600b5ca0 0x7fbb600b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193614542 0 0) 0x7fbb600d2e00 con 0x7fbb600b73c0 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb4c003520 con 0x7fbb600b73c0 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3175350098 0 0) 0x7fbb4c003520 con 0x7fbb600b73c0 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 >> v2:172.21.15.178:3300/0 conn(0x7fbb600b5ca0 msgr2=0x7fbb600b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 --2- 172.21.15.113:0/2694981922 >> v2:172.21.15.178:3300/0 conn(0x7fbb600b5ca0 0x7fbb600b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 >> v1:172.21.15.178:6789/0 conn(0x7fbb6015e510 legacy=0x7fbb600d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb600d33c0 con 0x7fbb600b73c0 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fbb580019f0 con 0x7fbb600b73c0 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb64fc5640 1 --2- 172.21.15.113:0/2694981922 >> v2:172.21.15.178:3300/0 conn(0x7fbb600b5ca0 0x7fbb600b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 >> v1:172.21.15.113:6789/0 conn(0x7fbb600b73c0 legacy=0x7fbb600b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0039b0 0x7fbb4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb4c004800 0x7fbb4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb4c0051c0 0x7fbb4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c00c130 con 0x7fbb4c0051c0 2024-04-25T00:11:08.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c00c2e0 con 0x7fbb4c004800 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c00c4e0 con 0x7fbb4c0039b0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb65fc7640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb4c0051c0 0x7fbb4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb64fc5640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb4c004800 0x7fbb4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb657c6640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0039b0 0x7fbb4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb65fc7640 1 -- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0039b0 msgr2=0x7fbb4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb65fc7640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0039b0 0x7fbb4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb65fc7640 1 -- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb4c004800 msgr2=0x7fbb4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:07.999+0000 7fbb65fc7640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb4c004800 0x7fbb4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.000+0000 7fbb65fc7640 1 -- 172.21.15.113:0/2694981922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c003520 con 0x7fbb4c0051c0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.000+0000 7fbb65fc7640 1 -- 172.21.15.113:0/2694981922 reap_dead start 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.000+0000 7fbb65fc7640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb4c0051c0 0x7fbb4c00b9c0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800a860 tx=0x7fbb4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e7708b6d814757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.000+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800a480 con 0x7fbb4c0051c0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.000+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5802f070 con 0x7fbb4c0051c0 2024-04-25T00:11:08.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.000+0000 7fbb57fff640 1 -- 172.21.15.113:0/2694981922 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5802bca0 con 0x7fbb4c0051c0 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.001+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2694981922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb4c0051c0 msgr2=0x7fbb4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.001+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb4c0051c0 0x7fbb4c00b9c0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800a860 tx=0x7fbb4c005830 comp rx=0 tx=0).stop 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.001+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2694981922 shutdown_connections 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.001+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2694981922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb4c0051c0 0x7fbb4c00b9c0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.001+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2694981922 >> 172.21.15.113:0/2694981922 conn(0x7fbb6006b720 msgr2=0x7fbb600d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.001+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2694981922 shutdown_connections 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.001+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2694981922 wait complete. 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.002+0000 7fbb67a2a640 1 Processor -- start 2024-04-25T00:11:08.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.002+0000 7fbb67a2a640 1 -- start start 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb67a2a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0051c0 0x7fbb600b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb67a2a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb60156680 0x7fbb600c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb67a2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb60156ae0 0x7fbb600c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb67a2a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600d3ea0 con 0x7fbb60156680 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb67a2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600d3980 con 0x7fbb60156ae0 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb67a2a640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbb600b9dd0 con 0x7fbb4c0051c0 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb60156680 0x7fbb600c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb60156680 0x7fbb600c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39194/0 (socket says 172.21.15.113:39194) 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 -- 172.21.15.113:0/2514577436 learned_addr learned my addr 172.21.15.113:0/2514577436 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:08.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb657c6640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0051c0 0x7fbb600b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb65fc7640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb60156ae0 0x7fbb600c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 -- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0051c0 msgr2=0x7fbb600b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0051c0 0x7fbb600b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 -- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb60156ae0 msgr2=0x7fbb600c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb60156ae0 0x7fbb600c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 -- 172.21.15.113:0/2514577436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb601f7640 con 0x7fbb60156680 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb65fc7640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb60156ae0 0x7fbb600c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:08.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb64fc5640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb60156680 0x7fbb600c9970 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c011a90 tx=0x7fbb5c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb55ffb640 1 -- 172.21.15.113:0/2514577436 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00fc10 con 0x7fbb60156680 2024-04-25T00:11:08.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb55ffb640 1 -- 172.21.15.113:0/2514577436 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c00fdb0 con 0x7fbb60156680 2024-04-25T00:11:08.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb55ffb640 1 -- 172.21.15.113:0/2514577436 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c004260 con 0x7fbb60156680 2024-04-25T00:11:08.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.003+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb601f7920 con 0x7fbb60156680 2024-04-25T00:11:08.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.004+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601f7ee0 con 0x7fbb60156680 2024-04-25T00:11:08.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.005+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb600691a0 con 0x7fbb60156680 2024-04-25T00:11:08.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.005+0000 7fbb55ffb640 1 -- 172.21.15.113:0/2514577436 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb5c012040 con 0x7fbb60156680 2024-04-25T00:11:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.005+0000 7fbb55ffb640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb4406be60 0x7fbb4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.006+0000 7fbb55ffb640 1 -- 172.21.15.113:0/2514577436 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fbb5c08b0e0 con 0x7fbb60156680 2024-04-25T00:11:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.006+0000 7fbb657c6640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb4406be60 0x7fbb4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.006+0000 7fbb657c6640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb4406be60 0x7fbb4406e320 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000c00 tx=0x7fbb50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.010+0000 7fbb55ffb640 1 -- 172.21.15.113:0/2514577436 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb5c019030 con 0x7fbb60156680 2024-04-25T00:11:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.186+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fbb60155a00 con 0x7fbb4406be60 2024-04-25T00:11:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.186+0000 7fbb55ffb640 1 -- 172.21.15.113:0/2514577436 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbb60155a00 con 0x7fbb4406be60 2024-04-25T00:11:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T00:11:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb4406be60 msgr2=0x7fbb4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb4406be60 0x7fbb4406e320 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000c00 tx=0x7fbb50009040 comp rx=0 tx=0).stop 2024-04-25T00:11:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb60156680 msgr2=0x7fbb600c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb60156680 0x7fbb600c9970 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c011a90 tx=0x7fbb5c011f50 comp rx=0 tx=0).stop 2024-04-25T00:11:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 shutdown_connections 2024-04-25T00:11:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fbb4406be60 0x7fbb4406e320 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb60156ae0 0x7fbb600c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb60156680 0x7fbb600c9970 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 --2- 172.21.15.113:0/2514577436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb4c0051c0 0x7fbb600b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 >> 172.21.15.113:0/2514577436 conn(0x7fbb6006b720 msgr2=0x7fbb600b7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 shutdown_connections 2024-04-25T00:11:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.188+0000 7fbb67a2a640 1 -- 172.21.15.113:0/2514577436 wait complete. 2024-04-25T00:11:08.200 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T00:11:08.200 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.301+0000 7fca85431640 1 Processor -- start 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.301+0000 7fca85431640 1 -- start start 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca85431640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca8015e510 0x7fca80161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca85431640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca800b9bb0 0x7fca800b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca85431640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca801626f0 con 0x7fca800b61e0 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca85431640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fca801629d0 con 0x7fca800b9bb0 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca85431640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fca80162cb0 con 0x7fca8015e510 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca7e7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca800b9bb0 0x7fca800b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca7e7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca800b9bb0 0x7fca800b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39200/0 (socket says 172.21.15.113:39200) 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca7e7fc640 1 -- 172.21.15.113:0/1501040188 learned_addr learned my addr 172.21.15.113:0/1501040188 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca7e7fc640 1 -- 172.21.15.113:0/1501040188 >> v2:172.21.15.178:3300/0 conn(0x7fca8015e510 msgr2=0x7fca80161ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca7e7fc640 1 --2- 172.21.15.113:0/1501040188 >> v2:172.21.15.178:3300/0 conn(0x7fca8015e510 0x7fca80161ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.302+0000 7fca7e7fc640 1 -- 172.21.15.113:0/1501040188 >> v1:172.21.15.178:6789/0 conn(0x7fca800b61e0 legacy=0x7fca80071710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.303+0000 7fca7e7fc640 1 -- 172.21.15.113:0/1501040188 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca800d2660 con 0x7fca800b9bb0 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.303+0000 7fca7e7fc640 1 --2- 172.21.15.113:0/1501040188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca800b9bb0 0x7fca800b5ca0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fca800727e0 tx=0x7fca7402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d75bcef03cd36f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.303+0000 7fca7dffb640 1 -- 172.21.15.113:0/1501040188 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74037070 con 0x7fca800b9bb0 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.303+0000 7fca7dffb640 1 -- 172.21.15.113:0/1501040188 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74032440 con 0x7fca800b9bb0 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.303+0000 7fca7dffb640 1 -- 172.21.15.113:0/1501040188 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74032810 con 0x7fca800b9bb0 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 -- 172.21.15.113:0/1501040188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca800b9bb0 msgr2=0x7fca800b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 --2- 172.21.15.113:0/1501040188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca800b9bb0 0x7fca800b5ca0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fca800727e0 tx=0x7fca7402a1c0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 -- 172.21.15.113:0/1501040188 shutdown_connections 2024-04-25T00:11:08.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 --2- 172.21.15.113:0/1501040188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca800b9bb0 0x7fca800b5ca0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 --2- 172.21.15.113:0/1501040188 >> v2:172.21.15.178:3300/0 conn(0x7fca8015e510 0x7fca80161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 -- 172.21.15.113:0/1501040188 >> 172.21.15.113:0/1501040188 conn(0x7fca8006b720 msgr2=0x7fca800d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 -- 172.21.15.113:0/1501040188 shutdown_connections 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.304+0000 7fca85431640 1 -- 172.21.15.113:0/1501040188 wait complete. 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.305+0000 7fca85431640 1 Processor -- start 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.305+0000 7fca85431640 1 -- start start 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca85431640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca800b61e0 0x7fca80148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca85431640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca800b9bb0 0x7fca800c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca85431640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8015e510 0x7fca801e9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca85431640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fca800d3e70 con 0x7fca8015e510 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca85431640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca801623e0 con 0x7fca800b61e0 2024-04-25T00:11:08.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca85431640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca800b73c0 con 0x7fca800b9bb0 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8015e510 0x7fca801e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8015e510 0x7fca801e9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39212/0 (socket says 172.21.15.113:39212) 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 -- 172.21.15.113:0/2798082596 learned_addr learned my addr 172.21.15.113:0/2798082596 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7e7fc640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca800b9bb0 0x7fca800c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7effd640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca800b61e0 0x7fca80148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 -- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca800b9bb0 msgr2=0x7fca800c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca800b9bb0 0x7fca800c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 -- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca800b61e0 msgr2=0x7fca80148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca800b61e0 0x7fca80148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.306+0000 7fca7f7fe640 1 -- 172.21.15.113:0/2798082596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8015c750 con 0x7fca8015e510 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.307+0000 7fca7f7fe640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8015e510 0x7fca801e9160 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b490 tx=0x7fca7800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.307+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78015020 con 0x7fca8015e510 2024-04-25T00:11:08.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.307+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78011850 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.307+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca801e8960 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.307+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78011c40 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.307+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca80149590 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.308+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca801493b0 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.308+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca78018020 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca677fe640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca6c06be30 0x7fca6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fca7808a9a0 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca7effd640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca6c06be30 0x7fca6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca677fe640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7fca6c071d90 0x7fca6c0741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 --> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca78004450 con 0x7fca6c071d90 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=186) ==== 24+0+0 (secure 0 0 0) 0x7fca7808f2c0 con 0x7fca8015e510 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca7effd640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca6c06be30 0x7fca6c06e2f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fca8014a4a0 tx=0x7fca700094f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.309+0000 7fca7e7fc640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7fca6c071d90 0x7fca6c0741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.310+0000 7fca7e7fc640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7fca6c071d90 0x7fca6c0741b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.310+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== osd.7 v2:172.21.15.178:6824/3779386153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca78004450 con 0x7fca6c071d90 2024-04-25T00:11:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.323+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 --> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fca800691a0 con 0x7fca6c071d90 2024-04-25T00:11:08.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca677fe640 1 -- 172.21.15.113:0/2798082596 <== osd.7 v2:172.21.15.178:6824/3779386153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fca800691a0 con 0x7fca6c071d90 2024-04-25T00:11:08.324 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-25T00:11:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:11:08.324 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-25T00:11:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7fca6c071d90 msgr2=0x7fca6c0741b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.178:6824/3779386153,v1:172.21.15.178:6825/3779386153] conn(0x7fca6c071d90 0x7fca6c0741b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca6c06be30 msgr2=0x7fca6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fca6c06be30 0x7fca6c06e2f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fca8014a4a0 tx=0x7fca700094f0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8015e510 msgr2=0x7fca801e9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 --2- 172.21.15.113:0/2798082596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8015e510 0x7fca801e9160 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b490 tx=0x7fca7800b950 comp rx=0 tx=0).stop 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca7f7fe640 1 -- 172.21.15.113:0/2798082596 reap_dead start 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 shutdown_connections 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 >> 172.21.15.113:0/2798082596 conn(0x7fca8006b720 msgr2=0x7fca800d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 shutdown_connections 2024-04-25T00:11:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.324+0000 7fca85431640 1 -- 172.21.15.113:0/2798082596 wait complete. 2024-04-25T00:11:08.335 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T00:11:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.432+0000 7ff907858640 1 Processor -- start 2024-04-25T00:11:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.432+0000 7ff907858640 1 -- start start 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff907858640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff90015e2d0 0x7ff90015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff907858640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff90015f4f0 con 0x7ff9000b3080 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff907858640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff90015f7d0 con 0x7ff9000b4a40 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff907858640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff9000cde00 con 0x7ff90015e2d0 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff9055f4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff90015e2d0 0x7ff90015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff904df3640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff9000b3080 0x7ff9000b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48972/0 (socket says 172.21.15.113:48972) 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff904df3640 1 -- 172.21.15.113:0/425506348 learned_addr learned my addr 172.21.15.113:0/425506348 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff9055f4640 1 -- 172.21.15.113:0/425506348 >> v1:172.21.15.178:6789/0 conn(0x7ff9000b4a40 legacy=0x7ff9000ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff9055f4640 1 -- 172.21.15.113:0/425506348 >> v1:172.21.15.113:6789/0 conn(0x7ff9000b3080 legacy=0x7ff9000b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff9055f4640 1 -- 172.21.15.113:0/425506348 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90015fab0 con 0x7ff90015e2d0 2024-04-25T00:11:08.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.433+0000 7ff8f7fff640 1 -- 172.21.15.113:0/425506348 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985136300 0 0) 0x7ff90015f4f0 con 0x7ff9000b3080 2024-04-25T00:11:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.434+0000 7ff9055f4640 1 --2- 172.21.15.113:0/425506348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff90015ebf0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7ff9000cdf80 tx=0x7ff8fc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847f3f0cea0df290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.434+0000 7ff8f7fff640 1 -- 172.21.15.113:0/425506348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc037070 con 0x7ff90015e2d0 2024-04-25T00:11:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.434+0000 7ff8f7fff640 1 -- 172.21.15.113:0/425506348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc032440 con 0x7ff90015e2d0 2024-04-25T00:11:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.434+0000 7ff8f7fff640 1 -- 172.21.15.113:0/425506348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc032810 con 0x7ff90015e2d0 2024-04-25T00:11:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.435+0000 7ff907858640 1 -- 172.21.15.113:0/425506348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 msgr2=0x7ff90015ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.435+0000 7ff907858640 1 --2- 172.21.15.113:0/425506348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff90015ebf0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7ff9000cdf80 tx=0x7ff8fc02a1c0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.435+0000 7ff907858640 1 -- 172.21.15.113:0/425506348 shutdown_connections 2024-04-25T00:11:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.435+0000 7ff907858640 1 --2- 172.21.15.113:0/425506348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff90015ebf0 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.435+0000 7ff907858640 1 -- 172.21.15.113:0/425506348 >> 172.21.15.113:0/425506348 conn(0x7ff90006b0f0 msgr2=0x7ff90006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.435+0000 7ff907858640 1 -- 172.21.15.113:0/425506348 shutdown_connections 2024-04-25T00:11:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.435+0000 7ff907858640 1 -- 172.21.15.113:0/425506348 wait complete. 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 Processor -- start 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 -- start start 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9000b3080 0x7ff900148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9000b4a40 0x7ff9000ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff900154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff90015f2b0 con 0x7ff90015e2d0 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff90015f130 con 0x7ff9000b4a40 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.436+0000 7ff907858640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff90015e6b0 con 0x7ff9000b3080 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff900154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff900154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39224/0 (socket says 172.21.15.113:39224) 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 -- 172.21.15.113:0/759570865 learned_addr learned my addr 172.21.15.113:0/759570865 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff9055f4640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9000b3080 0x7ff900148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff904df3640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9000b4a40 0x7ff9000ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 -- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9000b3080 msgr2=0x7ff900148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9000b3080 0x7ff900148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 -- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9000b4a40 msgr2=0x7ff9000ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9000b4a40 0x7ff9000ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 -- 172.21.15.113:0/759570865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff900146530 con 0x7ff90015e2d0 2024-04-25T00:11:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff904df3640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9000b4a40 0x7ff9000ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff905df5640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff900154af0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c730 tx=0x7ff8f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff8f5ffb640 1 -- 172.21.15.113:0/759570865 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8016020 con 0x7ff90015e2d0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff8f5ffb640 1 -- 172.21.15.113:0/759570865 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f800aa50 con 0x7ff90015e2d0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff900156d40 con 0x7ff90015e2d0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff8f5ffb640 1 -- 172.21.15.113:0/759570865 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800ce40 con 0x7ff90015e2d0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.437+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff900157300 con 0x7ff90015e2d0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.438+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9000691a0 con 0x7ff90015e2d0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.439+0000 7ff8f5ffb640 1 -- 172.21.15.113:0/759570865 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8f8019020 con 0x7ff90015e2d0 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.439+0000 7ff8f5ffb640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8e406be30 0x7ff8e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.439+0000 7ff8f5ffb640 1 -- 172.21.15.113:0/759570865 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7ff8f808aba0 con 0x7ff90015e2d0 2024-04-25T00:11:08.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.439+0000 7ff9055f4640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8e406be30 0x7ff8e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.440+0000 7ff9055f4640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8e406be30 0x7ff8e406e2f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc034000 tx=0x7ff8fc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.443+0000 7ff8f5ffb640 1 -- 172.21.15.113:0/759570865 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8f8020030 con 0x7ff90015e2d0 2024-04-25T00:11:08.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.619+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7ff9000af6f0 con 0x7ff8e406be30 2024-04-25T00:11:08.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.619+0000 7ff8f5ffb640 1 -- 172.21.15.113:0/759570865 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff9000af6f0 con 0x7ff8e406be30 2024-04-25T00:11:08.620 INFO:teuthology.orchestra.run.smithi113.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T00:11:08.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8e406be30 msgr2=0x7ff8e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8e406be30 0x7ff8e406e2f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc034000 tx=0x7ff8fc02e000 comp rx=0 tx=0).stop 2024-04-25T00:11:08.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 msgr2=0x7ff900154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff900154af0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c730 tx=0x7ff8f800cbf0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 shutdown_connections 2024-04-25T00:11:08.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7ff8e406be30 0x7ff8e406e2f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9000b4a40 0x7ff9000ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90015e2d0 0x7ff900154af0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 --2- 172.21.15.113:0/759570865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9000b3080 0x7ff900148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 >> 172.21.15.113:0/759570865 conn(0x7ff90006b0f0 msgr2=0x7ff9000afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 shutdown_connections 2024-04-25T00:11:08.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.622+0000 7ff907858640 1 -- 172.21.15.113:0/759570865 wait complete. 2024-04-25T00:11:08.633 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.728+0000 7f8018570640 1 Processor -- start 2024-04-25T00:11:08.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.728+0000 7f8018570640 1 -- start start 2024-04-25T00:11:08.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.729+0000 7f8018570640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 0x7f80100d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.729+0000 7f8018570640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100d3de0 con 0x7f80100d0b00 2024-04-25T00:11:08.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.729+0000 7f8018570640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100d2800 con 0x7f80100d2290 2024-04-25T00:11:08.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.729+0000 7f8018570640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f80100d3410 con 0x7f801015e320 2024-04-25T00:11:08.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.729+0000 7f8016b0d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f80100d0b00 0x7f80100b7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48974/0 (socket says 172.21.15.113:48974) 2024-04-25T00:11:08.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.729+0000 7f8016b0d640 1 -- 172.21.15.113:0/4276893463 learned_addr learned my addr 172.21.15.113:0/4276893463 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1097141072 0 0) 0x7f80100d3de0 con 0x7f80100d0b00 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ffc003520 con 0x7f80100d0b00 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801630c640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 0x7f80100d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3695262426 0 0) 0x7f7ffc003520 con 0x7f80100d0b00 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 msgr2=0x7f80100d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 0x7f80100d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 >> v1:172.21.15.178:6789/0 conn(0x7f80100d2290 legacy=0x7f80100b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80100d2ae0 con 0x7f80100d0b00 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7f8008001670 con 0x7f80100d0b00 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 >> v1:172.21.15.113:6789/0 conn(0x7f80100d0b00 legacy=0x7f80100b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ffc0039e0 0x7f7ffc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc004800 0x7f7ffc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 0x7f7ffc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f80100d3410 con 0x7f7ffc0039e0 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc00bb80 con 0x7f7ffc004800 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc00bd80 con 0x7f801015e320 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801630c640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ffc0039e0 0x7f7ffc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f8015b0b640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc004800 0x7f7ffc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f8016b0d640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 0x7f7ffc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801630c640 1 -- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 msgr2=0x7f7ffc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801630c640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f801015e320 0x7f7ffc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801630c640 1 -- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc004800 msgr2=0x7f7ffc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801630c640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc004800 0x7f7ffc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.730+0000 7f801630c640 1 -- 172.21.15.113:0/4276893463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc003520 con 0x7f7ffc0039e0 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.731+0000 7f801630c640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ffc0039e0 0x7f7ffc003e00 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f800c007360 tx=0x7f7ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e2ef9c9c37fd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.731+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c009be0 con 0x7f7ffc0039e0 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.731+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c036600 con 0x7f7ffc0039e0 2024-04-25T00:11:08.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.731+0000 7f801530a640 1 -- 172.21.15.113:0/4276893463 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c0369d0 con 0x7f7ffc0039e0 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.732+0000 7f8018570640 1 -- 172.21.15.113:0/4276893463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ffc0039e0 msgr2=0x7f7ffc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.732+0000 7f8018570640 1 --2- 172.21.15.113:0/4276893463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ffc0039e0 0x7f7ffc003e00 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f800c007360 tx=0x7f7ffc005830 comp rx=0 tx=0).stop 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.732+0000 7f8016b0d640 1 -- 172.21.15.113:0/4276893463 reap_dead start 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.732+0000 7f8018570640 1 -- 172.21.15.113:0/4276893463 shutdown_connections 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.732+0000 7f8018570640 1 -- 172.21.15.113:0/4276893463 >> 172.21.15.113:0/4276893463 conn(0x7f80100b5c20 msgr2=0x7f80100b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.732+0000 7f8018570640 1 -- 172.21.15.113:0/4276893463 shutdown_connections 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.732+0000 7f8018570640 1 -- 172.21.15.113:0/4276893463 wait complete. 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.733+0000 7f8018570640 1 Processor -- start 2024-04-25T00:11:08.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.733+0000 7f8018570640 1 -- start start 2024-04-25T00:11:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc0039e0 0x7f80101f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80100b2430 0x7f8010158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80100d0f90 0x7f80100d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f801015e700 con 0x7f80100b2430 2024-04-25T00:11:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80100d30a0 con 0x7f7ffc0039e0 2024-04-25T00:11:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f801015e880 con 0x7f80100d0f90 2024-04-25T00:11:08.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80100b2430 0x7f8010158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80100b2430 0x7f8010158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39244/0 (socket says 172.21.15.113:39244) 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 -- 172.21.15.113:0/4288883520 learned_addr learned my addr 172.21.15.113:0/4288883520 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8016b0d640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80100d0f90 0x7f80100d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 -- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80100d0f90 msgr2=0x7f80100d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80100d0f90 0x7f80100d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f801630c640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc0039e0 0x7f80101f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 -- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc0039e0 msgr2=0x7f80101f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc0039e0 0x7f80101f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 -- 172.21.15.113:0/4288883520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80101f4860 con 0x7f80100b2430 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8015b0b640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80100b2430 0x7f8010158aa0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f800000e580 tx=0x7f800000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8006ffd640 1 -- 172.21.15.113:0/4288883520 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000052e0 con 0x7f80100b2430 2024-04-25T00:11:08.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8006ffd640 1 -- 172.21.15.113:0/4288883520 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000018070 con 0x7f80100b2430 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80101f4b40 con 0x7f80100b2430 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8006ffd640 1 -- 172.21.15.113:0/4288883520 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000003810 con 0x7f80100b2430 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.734+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80101f5100 con 0x7f80100b2430 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.735+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80100d1950 con 0x7f80100b2430 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.736+0000 7f8006ffd640 1 -- 172.21.15.113:0/4288883520 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80000039b0 con 0x7f80100b2430 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.736+0000 7f8006ffd640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ff406be30 0x7f7ff406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.736+0000 7f801630c640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ff406be30 0x7f7ff406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.736+0000 7f8006ffd640 1 -- 172.21.15.113:0/4288883520 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f800008a8a0 con 0x7f80100b2430 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.737+0000 7f801630c640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ff406be30 0x7f7ff406e2f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f800c035000 tx=0x7f800c038040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.740+0000 7f8006ffd640 1 -- 172.21.15.113:0/4288883520 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8000059500 con 0x7f80100b2430 2024-04-25T00:11:08.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.911+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80100ca5c0 con 0x7f7ff406be30 2024-04-25T00:11:08.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.911+0000 7f8006ffd640 1 -- 172.21.15.113:0/4288883520 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13452 (secure 0 0 0) 0x7f80100ca5c0 con 0x7f7ff406be30 2024-04-25T00:11:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:11:08.912 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:11:08.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ff406be30 msgr2=0x7f7ff406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ff406be30 0x7f7ff406e2f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f800c035000 tx=0x7f800c038040 comp rx=0 tx=0).stop 2024-04-25T00:11:08.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80100b2430 msgr2=0x7f8010158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80100b2430 0x7f8010158aa0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f800000e580 tx=0x7f800000ea40 comp rx=0 tx=0).stop 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 shutdown_connections 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f7ff406be30 0x7f7ff406e2f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f80100d0f90 0x7f80100d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80100b2430 0x7f8010158aa0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 --2- 172.21.15.113:0/4288883520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ffc0039e0 0x7f80101f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 >> 172.21.15.113:0/4288883520 conn(0x7f80100b5c20 msgr2=0x7f801015d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 shutdown_connections 2024-04-25T00:11:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:08.913+0000 7f8018570640 1 -- 172.21.15.113:0/4288883520 wait complete. 2024-04-25T00:11:08.925 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":570,"stamp":"2024-04-25T00:11:07.268496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":425872,"kb_used_data":19528,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754548848,"statfs":{"total":773094113280,"available":772658020352,"internally_reserved":0,"allocated":19996672,"data_stored":9420000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001172"},"pg_stats":[{"pgid":"1.0","version":"153'161","reported_seq":524,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T00:11:01.098059+0000","last_change":"2024-04-24T23:58:08.160904+0000","last_active":"2024-04-25T00:11:01.098059+0000","last_peered":"2024-04-25T00:11:01.098059+0000","last_clean":"2024-04-25T00:11:01.098059+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:11:01.098059+0000","last_undegraded":"2024-04-25T00:11:01.098059+0000","last_fullsized":"2024-04-25T00:11:01.098059+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:58:07.143374+0000","last_clean_scrub_stamp":"2024-04-24T23:58:07.143374+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:28.051028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52108,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319732,"statfs":{"total":96636764160,"available":96583405568,"internally_reserved":0,"allocated":2945024,"data_stored":1620276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51532,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55692,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316148,"statfs":{"total":96636764160,"available":96579735552,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51524,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320316,"statfs":{"total":96636764160,"available":96584003584,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51528,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320312,"statfs":{"total":96636764160,"available":96583999488,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51528,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320312,"statfs":{"total":96636764160,"available":96583999488,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575006,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52108,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319732,"statfs":{"total":96636764160,"available":96583405568,"internally_reserved":0,"allocated":2945024,"data_stored":1620276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:08.927 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-24T23:58:07.143374+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=23, tm_min=58, tm_sec=7, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:11:08.927 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T00:11:24.608 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:24.611 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:28.928 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:11:29.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.038+0000 7f9298ca5640 1 Processor -- start 2024-04-25T00:11:29.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.038+0000 7f9298ca5640 1 -- start start 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f9298ca5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f929406e6e0 0x7f92940b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f9298ca5640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f92941453c0 0x7f9294071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f9298ca5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92940cfad0 con 0x7f9294144d60 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f9298ca5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9294145c40 con 0x7f929406e6e0 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f9298ca5640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f92940cf230 con 0x7f92941453c0 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f928bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f929406e6e0 0x7f92940b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f929259c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9294144d60 0x7f929406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48902/0 (socket says 172.21.15.113:48902) 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f929259c640 1 -- 172.21.15.113:0/1700413118 learned_addr learned my addr 172.21.15.113:0/1700413118 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f9292d9d640 1 --2- 172.21.15.113:0/1700413118 >> v2:172.21.15.178:3300/0 conn(0x7f92941453c0 0x7f9294071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f928bfff640 1 -- 172.21.15.113:0/1700413118 >> v2:172.21.15.178:3300/0 conn(0x7f92941453c0 msgr2=0x7f9294071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f928bfff640 1 --2- 172.21.15.113:0/1700413118 >> v2:172.21.15.178:3300/0 conn(0x7f92941453c0 0x7f9294071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f928bfff640 1 -- 172.21.15.113:0/1700413118 >> v1:172.21.15.113:6789/0 conn(0x7f9294144d60 legacy=0x7f929406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f9291d9b640 1 -- 172.21.15.113:0/1700413118 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830116694 0 0) 0x7f92940cfad0 con 0x7f9294144d60 2024-04-25T00:11:29.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.039+0000 7f928bfff640 1 -- 172.21.15.113:0/1700413118 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92940cf510 con 0x7f929406e6e0 2024-04-25T00:11:29.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.040+0000 7f928bfff640 1 --2- 172.21.15.113:0/1700413118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929406e6e0 0x7f92940b5390 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f92940733d0 tx=0x7f927c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5a74adf1dc8b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.040+0000 7f9291d9b640 1 -- 172.21.15.113:0/1700413118 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c037070 con 0x7f929406e6e0 2024-04-25T00:11:29.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.040+0000 7f9291d9b640 1 -- 172.21.15.113:0/1700413118 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927c02add0 con 0x7f929406e6e0 2024-04-25T00:11:29.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.040+0000 7f9291d9b640 1 -- 172.21.15.113:0/1700413118 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c032660 con 0x7f929406e6e0 2024-04-25T00:11:29.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 -- 172.21.15.113:0/1700413118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929406e6e0 msgr2=0x7f92940b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1700413118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929406e6e0 0x7f92940b5390 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f92940733d0 tx=0x7f927c02a080 comp rx=0 tx=0).stop 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 -- 172.21.15.113:0/1700413118 shutdown_connections 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1700413118 >> v2:172.21.15.178:3300/0 conn(0x7f92941453c0 0x7f9294071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1700413118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f929406e6e0 0x7f92940b5390 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 -- 172.21.15.113:0/1700413118 >> 172.21.15.113:0/1700413118 conn(0x7f92940b1a50 msgr2=0x7f92940b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 -- 172.21.15.113:0/1700413118 shutdown_connections 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.041+0000 7f9298ca5640 1 -- 172.21.15.113:0/1700413118 wait complete. 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.042+0000 7f9298ca5640 1 Processor -- start 2024-04-25T00:11:29.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.042+0000 7f9298ca5640 1 -- start start 2024-04-25T00:11:29.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9298ca5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406e6e0 0x7f92941f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9298ca5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9294144d60 0x7f92941596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9298ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92941453c0 0x7f929413e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9298ca5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9294145140 con 0x7f92941453c0 2024-04-25T00:11:29.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9298ca5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9294070e10 con 0x7f9294144d60 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9298ca5640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9294071870 con 0x7f929406e6e0 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92941453c0 0x7f929413e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92941453c0 0x7f929413e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37398/0 (socket says 172.21.15.113:37398) 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 -- 172.21.15.113:0/1196086592 learned_addr learned my addr 172.21.15.113:0/1196086592 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f928bfff640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9294144d60 0x7f92941596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f929259c640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406e6e0 0x7f92941f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 -- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406e6e0 msgr2=0x7f92941f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406e6e0 0x7f92941f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 -- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9294144d60 msgr2=0x7f92941596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9294144d60 0x7f92941596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f9292d9d640 1 -- 172.21.15.113:0/1196086592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9294159c20 con 0x7f92941453c0 2024-04-25T00:11:29.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.043+0000 7f929259c640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406e6e0 0x7f92941f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.044+0000 7f9292d9d640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92941453c0 0x7f929413e6d0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f928c007cf0 tx=0x7f928c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.044+0000 7f928affd640 1 -- 172.21.15.113:0/1196086592 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c014420 con 0x7f92941453c0 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.044+0000 7f928affd640 1 -- 172.21.15.113:0/1196086592 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c019070 con 0x7f92941453c0 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.044+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9294159ea0 con 0x7f92941453c0 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.044+0000 7f928affd640 1 -- 172.21.15.113:0/1196086592 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c014820 con 0x7f92941453c0 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.044+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92941f7a70 con 0x7f92941453c0 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.045+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92941f75c0 con 0x7f92941453c0 2024-04-25T00:11:29.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.045+0000 7f928affd640 1 -- 172.21.15.113:0/1196086592 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f928c0149c0 con 0x7f92941453c0 2024-04-25T00:11:29.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.045+0000 7f928affd640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.046+0000 7f928affd640 1 -- 172.21.15.113:0/1196086592 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f928c08b220 con 0x7f92941453c0 2024-04-25T00:11:29.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.046+0000 7f929259c640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.046+0000 7f929259c640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9284000f60 tx=0x7f9284007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.050+0000 7f928affd640 1 -- 172.21.15.113:0/1196086592 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9294159c20 con 0x7f92941453c0 2024-04-25T00:11:29.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.219+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 --> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92940691a0 con 0x7f927406be30 2024-04-25T00:11:29.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.219+0000 7f928affd640 1 -- 172.21.15.113:0/1196086592 <== mgr.4113 v2:172.21.15.113:6832/3117017300 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13385 (secure 0 0 0) 0x7f92940691a0 con 0x7f927406be30 2024-04-25T00:11:29.220 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:11:29.220 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-25T00:11:29.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 msgr2=0x7f927406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9284000f60 tx=0x7f9284007c10 comp rx=0 tx=0).stop 2024-04-25T00:11:29.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92941453c0 msgr2=0x7f929413e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92941453c0 0x7f929413e6d0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f928c007cf0 tx=0x7f928c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 shutdown_connections 2024-04-25T00:11:29.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f927406be30 0x7f927406e2f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9294144d60 0x7f92941596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92941453c0 0x7f929413e6d0 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 --2- 172.21.15.113:0/1196086592 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406e6e0 0x7f92941f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 >> 172.21.15.113:0/1196086592 conn(0x7f92940b1a50 msgr2=0x7f92940b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.221+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 shutdown_connections 2024-04-25T00:11:29.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.222+0000 7f9298ca5640 1 -- 172.21.15.113:0/1196086592 wait complete. 2024-04-25T00:11:29.233 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":580,"stamp":"2024-04-25T00:11:27.271994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":425856,"kb_used_data":19528,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754548864,"statfs":{"total":773094113280,"available":772658036736,"internally_reserved":0,"allocated":19996672,"data_stored":9420000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002166"},"pg_stats":[{"pgid":"1.0","version":"186'163","reported_seq":535,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-04-25T00:11:08.833737+0000","last_change":"2024-04-25T00:11:08.833775+0000","last_active":"2024-04-25T00:11:08.833737+0000","last_peered":"2024-04-25T00:11:08.833737+0000","last_clean":"2024-04-25T00:11:08.833737+0000","last_became_active":"2024-04-24T23:58:08.159860+0000","last_became_peered":"2024-04-24T23:58:08.159860+0000","last_unstale":"2024-04-25T00:11:08.833737+0000","last_undegraded":"2024-04-25T00:11:08.833737+0000","last_fullsized":"2024-04-25T00:11:08.833737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"186'163","last_scrub_stamp":"2024-04-25T00:11:08.833654+0000","last_deep_scrub":"186'163","last_deep_scrub_stamp":"2024-04-25T00:11:08.833654+0000","last_clean_scrub_stamp":"2024-04-25T00:11:08.833654+0000","objects_scrubbed":2,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:53.085491+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":209,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52100,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319740,"statfs":{"total":96636764160,"available":96583413760,"internally_reserved":0,"allocated":2945024,"data_stored":1620276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51532,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55692,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316148,"statfs":{"total":96636764160,"available":96579735552,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51524,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320316,"statfs":{"total":96636764160,"available":96584003584,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51528,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320312,"statfs":{"total":96636764160,"available":96583999488,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51528,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320312,"statfs":{"total":96636764160,"available":96583999488,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1029908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52100,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319740,"statfs":{"total":96636764160,"available":96583413760,"internally_reserved":0,"allocated":2945024,"data_stored":1620276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:11:29.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T00:11:29.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.331+0000 7fe85b97e640 1 Processor -- start 2024-04-25T00:11:29.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.331+0000 7fe85b97e640 1 -- start start 2024-04-25T00:11:29.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85b97e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 0x7fe85406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85b97e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe854061b80 0x7fe854062010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85b97e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe85415ea00 con 0x7fe8540cd4b0 2024-04-25T00:11:29.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85b97e640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe8540cfc30 con 0x7fe854061b80 2024-04-25T00:11:29.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85b97e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe854162490 con 0x7fe854161fd0 2024-04-25T00:11:29.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85971a640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe8540cd4b0 0x7fe8540cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48916/0 (socket says 172.21.15.113:48916) 2024-04-25T00:11:29.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85971a640 1 -- 172.21.15.113:0/1476199460 learned_addr learned my addr 172.21.15.113:0/1476199460 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-25T00:11:29.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe859f1b640 1 --2- 172.21.15.113:0/1476199460 >> v2:172.21.15.178:3300/0 conn(0x7fe854061b80 0x7fe854062010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe858f19640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 0x7fe85406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821227968 0 0) 0x7fe85415ea00 con 0x7fe8540cd4b0 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe840003520 con 0x7fe8540cd4b0 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2244281262 0 0) 0x7fe840003520 con 0x7fe8540cd4b0 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 msgr2=0x7fe85406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 0x7fe85406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 >> v2:172.21.15.178:3300/0 conn(0x7fe854061b80 msgr2=0x7fe854062010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 --2- 172.21.15.113:0/1476199460 >> v2:172.21.15.178:3300/0 conn(0x7fe854061b80 0x7fe854062010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe854162770 con 0x7fe8540cd4b0 2024-04-25T00:11:29.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3426632699 0 0) 0x7fe850001670 con 0x7fe8540cd4b0 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 >> v1:172.21.15.113:6789/0 conn(0x7fe8540cd4b0 legacy=0x7fe8540cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8400039b0 0x7fe840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 0x7fe840004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe840004d40 0x7fe8400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540cfc30 con 0x7fe8400039b0 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe854162490 con 0x7fe840004d40 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe84000b9c0 con 0x7fe854161fd0 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe85971a640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8400039b0 0x7fe840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.332+0000 7fe858f19640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 0x7fe840004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe859f1b640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe840004d40 0x7fe8400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe85971a640 1 -- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 msgr2=0x7fe840004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe85971a640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe854161fd0 0x7fe840004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe85971a640 1 -- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe840004d40 msgr2=0x7fe8400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe85971a640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe840004d40 0x7fe8400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe85971a640 1 -- 172.21.15.113:0/1476199460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe840003520 con 0x7fe8400039b0 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe859f1b640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe840004d40 0x7fe8400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe85971a640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8400039b0 0x7fe840003dd0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c290 tx=0x7fe840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca0c3824a879d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850007d50 con 0x7fe8400039b0 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe850005af0 con 0x7fe8400039b0 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.333+0000 7fe847fff640 1 -- 172.21.15.113:0/1476199460 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850007070 con 0x7fe8400039b0 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.334+0000 7fe85b97e640 1 -- 172.21.15.113:0/1476199460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8400039b0 msgr2=0x7fe840003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.334+0000 7fe85b97e640 1 --2- 172.21.15.113:0/1476199460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8400039b0 0x7fe840003dd0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c290 tx=0x7fe840005830 comp rx=0 tx=0).stop 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.334+0000 7fe859f1b640 1 -- 172.21.15.113:0/1476199460 reap_dead start 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.334+0000 7fe85b97e640 1 -- 172.21.15.113:0/1476199460 shutdown_connections 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.334+0000 7fe85b97e640 1 -- 172.21.15.113:0/1476199460 >> 172.21.15.113:0/1476199460 conn(0x7fe8540d0e30 msgr2=0x7fe8540d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.334+0000 7fe85b97e640 1 -- 172.21.15.113:0/1476199460 shutdown_connections 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.334+0000 7fe85b97e640 1 -- 172.21.15.113:0/1476199460 wait complete. 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.335+0000 7fe85b97e640 1 Processor -- start 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.335+0000 7fe85b97e640 1 -- start start 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe85b97e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8400039b0 0x7fe8540c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe85b97e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8540c6230 0x7fe85415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe85b97e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85415cb10 0x7fe8540b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe85b97e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540cf1d0 con 0x7fe85415cb10 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe85b97e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe854162d30 con 0x7fe8540c6230 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe85b97e640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe85406c210 con 0x7fe8400039b0 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85415cb10 0x7fe8540b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85415cb10 0x7fe8540b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37422/0 (socket says 172.21.15.113:37422) 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 -- 172.21.15.113:0/2761181914 learned_addr learned my addr 172.21.15.113:0/2761181914 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe858f19640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8540c6230 0x7fe85415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe85971a640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8400039b0 0x7fe8540c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 -- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8400039b0 msgr2=0x7fe8540c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8400039b0 0x7fe8540c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 -- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8540c6230 msgr2=0x7fe85415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8540c6230 0x7fe85415c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 -- 172.21.15.113:0/2761181914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85415cf90 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.336+0000 7fe859f1b640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85415cb10 0x7fe8540b4c40 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fe8540627d0 tx=0x7fe84c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.337+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c010d80 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.337+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84c003990 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.337+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85415b810 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.337+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c003dc0 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.337+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85415bdd0 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.338+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8540691a0 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.338+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe84c005d10 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.339+0000 7fe845ffb640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe83806be60 0x7fe83806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.339+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7fe84c08b240 con 0x7fe85415cb10 2024-04-25T00:11:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.339+0000 7fe85971a640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe83806be60 0x7fe83806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.339+0000 7fe85971a640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe83806be60 0x7fe83806e320 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe850033000 tx=0x7fe850009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.343+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe84c025020 con 0x7fe85415cb10 2024-04-25T00:11:29.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.514+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fe8540c6610 con 0x7fe85415cb10 2024-04-25T00:11:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.518+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v36)=0 v36) ==== 123+0+0 (secure 0 0 0) 0x7fe84c059e90 con 0x7fe85415cb10 2024-04-25T00:11:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.519+0000 7fe845ffb640 1 -- 172.21.15.113:0/2761181914 <== mon.0 v2:172.21.15.113:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe84c059790 con 0x7fe85415cb10 2024-04-25T00:11:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe83806be60 msgr2=0x7fe83806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe83806be60 0x7fe83806e320 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe850033000 tx=0x7fe850009030 comp rx=0 tx=0).stop 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85415cb10 msgr2=0x7fe8540b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85415cb10 0x7fe8540b4c40 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fe8540627d0 tx=0x7fe84c010590 comp rx=0 tx=0).stop 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 shutdown_connections 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7fe83806be60 0x7fe83806e320 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8540c6230 0x7fe85415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85415cb10 0x7fe8540b4c40 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 --2- 172.21.15.113:0/2761181914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe8400039b0 0x7fe8540c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 >> 172.21.15.113:0/2761181914 conn(0x7fe8540d0e30 msgr2=0x7fe8540cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 shutdown_connections 2024-04-25T00:11:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.520+0000 7fe85b97e640 1 -- 172.21.15.113:0/2761181914 wait complete. 2024-04-25T00:11:29.532 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:11:29.532 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:11:29.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.628+0000 7f6a49493640 1 Processor -- start 2024-04-25T00:11:29.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.628+0000 7f6a49493640 1 -- start start 2024-04-25T00:11:29.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a49493640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4415e320 0x7f6a440d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a49493640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6a440d0b00 0x7f6a440d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a49493640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a440d3de0 con 0x7f6a440d2290 2024-04-25T00:11:29.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a49493640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6a440d14b0 con 0x7f6a4415e320 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a49493640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6a440d3410 con 0x7f6a440d0b00 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a42ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4415e320 0x7f6a440d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a42ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4415e320 0x7f6a440d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37436/0 (socket says 172.21.15.113:37436) 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.629+0000 7f6a42ffd640 1 -- 172.21.15.113:0/3273059749 learned_addr learned my addr 172.21.15.113:0/3273059749 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.630+0000 7f6a42ffd640 1 -- 172.21.15.113:0/3273059749 >> v2:172.21.15.178:3300/0 conn(0x7f6a440d0b00 msgr2=0x7f6a440d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.630+0000 7f6a42ffd640 1 --2- 172.21.15.113:0/3273059749 >> v2:172.21.15.178:3300/0 conn(0x7f6a440d0b00 0x7f6a440d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.630+0000 7f6a42ffd640 1 -- 172.21.15.113:0/3273059749 >> v1:172.21.15.178:6789/0 conn(0x7f6a440d2290 legacy=0x7f6a440b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.630+0000 7f6a42ffd640 1 -- 172.21.15.113:0/3273059749 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a440d2800 con 0x7f6a4415e320 2024-04-25T00:11:29.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.630+0000 7f6a42ffd640 1 --2- 172.21.15.113:0/3273059749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a440d38a0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f6a440d2030 tx=0x7f6a3402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1085e487caa5f9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.630+0000 7f6a41ffb640 1 -- 172.21.15.113:0/3273059749 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34036070 con 0x7f6a4415e320 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a41ffb640 1 -- 172.21.15.113:0/3273059749 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6a34031600 con 0x7f6a4415e320 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a41ffb640 1 -- 172.21.15.113:0/3273059749 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a340319f0 con 0x7f6a4415e320 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 -- 172.21.15.113:0/3273059749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 msgr2=0x7f6a440d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 --2- 172.21.15.113:0/3273059749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a440d38a0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f6a440d2030 tx=0x7f6a3402a780 comp rx=0 tx=0).stop 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 -- 172.21.15.113:0/3273059749 shutdown_connections 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 --2- 172.21.15.113:0/3273059749 >> v2:172.21.15.178:3300/0 conn(0x7f6a440d0b00 0x7f6a440d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 --2- 172.21.15.113:0/3273059749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a440d38a0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 -- 172.21.15.113:0/3273059749 >> 172.21.15.113:0/3273059749 conn(0x7f6a440b5c20 msgr2=0x7f6a440b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 -- 172.21.15.113:0/3273059749 shutdown_connections 2024-04-25T00:11:29.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.631+0000 7f6a49493640 1 -- 172.21.15.113:0/3273059749 wait complete. 2024-04-25T00:11:29.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.632+0000 7f6a49493640 1 Processor -- start 2024-04-25T00:11:29.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.632+0000 7f6a49493640 1 -- start start 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a49493640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a440d0b00 0x7f6a441537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a49493640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a440d2290 0x7f6a4415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a49493640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a441568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a49493640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440d30a0 con 0x7f6a4415e320 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a49493640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440d2dc0 con 0x7f6a440d2290 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a49493640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4415e700 con 0x7f6a440d0b00 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a441568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a441568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37446/0 (socket says 172.21.15.113:37446) 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a437fe640 1 -- 172.21.15.113:0/3641120677 learned_addr learned my addr 172.21.15.113:0/3641120677 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a427fc640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a440d2290 0x7f6a4415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.633+0000 7f6a42ffd640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a440d0b00 0x7f6a441537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a437fe640 1 -- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a440d0b00 msgr2=0x7f6a441537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a437fe640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a440d0b00 0x7f6a441537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a437fe640 1 -- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a440d2290 msgr2=0x7f6a4415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a437fe640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a440d2290 0x7f6a4415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a437fe640 1 -- 172.21.15.113:0/3641120677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4415aab0 con 0x7f6a4415e320 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a437fe640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a441568a0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009fd0 tx=0x7f6a3c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a2b7fe640 1 -- 172.21.15.113:0/3641120677 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c011830 con 0x7f6a4415e320 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a2b7fe640 1 -- 172.21.15.113:0/3641120677 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6a3c016070 con 0x7f6a4415e320 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4415f980 con 0x7f6a4415e320 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a2b7fe640 1 -- 172.21.15.113:0/3641120677 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c011c10 con 0x7f6a4415e320 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.634+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4415ff40 con 0x7f6a4415e320 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.635+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a44160140 con 0x7f6a4415e320 2024-04-25T00:11:29.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.635+0000 7f6a2b7fe640 1 -- 172.21.15.113:0/3641120677 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a3c01b460 con 0x7f6a4415e320 2024-04-25T00:11:29.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.636+0000 7f6a2b7fe640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a3006be30 0x7f6a3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:29.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.636+0000 7f6a42ffd640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a3006be30 0x7f6a3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:29.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.636+0000 7f6a2b7fe640 1 -- 172.21.15.113:0/3641120677 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5029+0+0 (secure 0 0 0) 0x7f6a3c08a970 con 0x7f6a4415e320 2024-04-25T00:11:29.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.636+0000 7f6a42ffd640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a3006be30 0x7f6a3006e2f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6a34033000 tx=0x7f6a34009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.640+0000 7f6a2b7fe640 1 -- 172.21.15.113:0/3641120677 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a44160140 con 0x7f6a4415e320 2024-04-25T00:11:29.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.846+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a440691a0 con 0x7f6a4415e320 2024-04-25T00:11:29.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.847+0000 7f6a2b7fe640 1 -- 172.21.15.113:0/3641120677 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+3886 (secure 0 0 0) 0x7f6a3c059530 con 0x7f6a4415e320 2024-04-25T00:11:29.847 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-25T00:11:29.847 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3095815035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3095815035},{"type":"v1","addr":"172.21.15.178:6834","nonce":3095815035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2901681233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":2901681233},{"type":"v1","addr":"172.21.15.178:6835","nonce":2901681233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":10791,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2286114318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2286114318},{"type":"v1","addr":"172.21.15.113:6837","nonce":2286114318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":11313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2624734670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2624734670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2624734670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[]} 2024-04-25T00:11:29.848 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-04-25T00:11:29.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a3006be30 msgr2=0x7f6a3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a3006be30 0x7f6a3006e2f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6a34033000 tx=0x7f6a34009000 comp rx=0 tx=0).stop 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 msgr2=0x7f6a441568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a441568a0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009fd0 tx=0x7f6a3c00b8e0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 shutdown_connections 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:6832/3117017300,v1:172.21.15.113:6833/3117017300] conn(0x7f6a3006be30 0x7f6a3006e2f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a440d2290 0x7f6a4415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4415e320 0x7f6a441568a0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 --2- 172.21.15.113:0/3641120677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a440d0b00 0x7f6a441537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 >> 172.21.15.113:0/3641120677 conn(0x7f6a440b5c20 msgr2=0x7f6a440b7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 shutdown_connections 2024-04-25T00:11:29.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-25T00:11:29.849+0000 7f6a49493640 1 -- 172.21.15.113:0/3641120677 wait complete. 2024-04-25T00:11:29.860 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T00:11:29.860 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T00:11:29.860 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:11:29.860 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:29.869 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:11:29.869 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:11:29.869 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:29.918 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:11:29.918 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:11:29.918 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:29.930 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:11:29.930 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:11:29.930 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:29.982 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:11:29.982 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T00:11:29.983 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T00:11:29.983 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.016 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T00:11:30.016 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T00:11:30.016 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.084 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T00:11:30.084 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T00:11:30.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.152 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T00:11:30.152 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T00:11:30.152 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.219 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T00:11:30.219 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T00:11:30.219 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.256 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T00:11:30.256 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T00:11:30.256 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.327 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T00:11:30.327 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T00:11:30.328 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.400 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T00:11:30.400 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T00:11:30.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.476 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T00:11:30.476 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T00:11:30.476 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T00:11:30.476 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.501 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T00:11:30.502 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T00:11:30.502 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.545 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T00:11:30.545 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T00:11:30.546 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T00:11:30.546 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.557 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T00:11:30.557 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T00:11:30.557 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.597 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T00:11:30.598 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T00:11:30.598 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:11:30.651 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T00:11:30.651 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T00:11:30.651 DEBUG:teuthology.orchestra.run.smithi113:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T00:11:30.682 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:11:30.683 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T00:11:30.929 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:11:30.930 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T00:11:31.029 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:11:31.029 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T00:11:31.102 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:11:31.103 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T00:11:31.169 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-25T00:11:31.169 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T00:11:31.304 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-25T00:11:31.304 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T00:11:31.382 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-25T00:11:31.382 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T00:11:31.425 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-25T00:11:31.425 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T00:11:31.525 INFO:tasks.ceph:Archiving mon data... 2024-04-25T00:11:31.526 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/data/mon.a.tgz 2024-04-25T00:11:31.526 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-04-25T00:11:31.541 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.9cWmiUbeRD 2024-04-25T00:11:31.541 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.9cWmiUbeRD 2024-04-25T00:11:31.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.9cWmiUbeRD 2024-04-25T00:11:31.783 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.9cWmiUbeRD is 464KB 2024-04-25T00:11:31.881 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.9cWmiUbeRD 2024-04-25T00:11:31.895 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/data/mon.b.tgz 2024-04-25T00:11:31.895 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-04-25T00:11:31.913 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.grUaAD9Qgu 2024-04-25T00:11:31.913 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.grUaAD9Qgu 2024-04-25T00:11:32.079 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.grUaAD9Qgu 2024-04-25T00:11:32.161 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.grUaAD9Qgu is 466KB 2024-04-25T00:11:32.260 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.grUaAD9Qgu 2024-04-25T00:11:32.279 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/data/mon.c.tgz 2024-04-25T00:11:32.280 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-04-25T00:11:32.332 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.WwN7uuEW2U 2024-04-25T00:11:32.333 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WwN7uuEW2U 2024-04-25T00:11:32.479 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.WwN7uuEW2U 2024-04-25T00:11:32.560 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.WwN7uuEW2U is 466KB 2024-04-25T00:11:32.657 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.WwN7uuEW2U 2024-04-25T00:11:32.671 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T00:11:32.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:11:32.674 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:11:32.754 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:11:32.790 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:11:32.791 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T00:11:32.793 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/remote/smithi113/crash 2024-04-25T00:11:32.794 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:11:32.820 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/remote/smithi178/crash 2024-04-25T00:11:32.821 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:11:32.854 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T00:11:32.854 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:32.865 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:32.945 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:32.970 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:11:32.995 INFO:tasks.ceph:Compressing logs... 2024-04-25T00:11:32.995 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:11:33.013 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:11:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.40775.log 2024-04-25T00:11:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.3.40775.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40775.log.gz 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40374.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34736.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.1.40374.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40374.log.gz 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34736.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34736.log.gz 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34816.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34776.log 2024-04-25T00:11:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34649.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34816.log.gz 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34776.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34776.log.gz 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.40779.log 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.3.40779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40779.log.gz 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34696.log 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34696.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34696.log.gz 2024-04-25T00:11:33.110 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40379.log 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34565.log 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.1.40379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40379.log.gz 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34525.log 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34565.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34565.log.gz 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34525.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34525.log.gz 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34605.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T00:11:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T00:11:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T00:11:33.403 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46692.log 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78666.log 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40898.log 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77221.log 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46692.log.gz 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60425.log 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77221.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.78666.log.gz 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40898.log.gz 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59113.log 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77221.log.gz 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83426.log 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59113.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60425.log.gz 2024-04-25T00:11:33.404 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106163.log 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83426.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59113.log.gz 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103504.log 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95224.log 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83426.log.gz 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106163.log.gz 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95476.log 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103504.log: /var/log/ceph/ceph-client.admin.95224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99186.log 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103504.log.gz 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97933.log 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95224.log.gz 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95476.log.gz 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81952.log 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99186.log.gz 2024-04-25T00:11:33.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94962.log 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97933.log.gz 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87662.log 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103462.log 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.81952.log.gz 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87662.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94962.log.gz 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99526.log 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87662.log.gz 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40330.log 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99526.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103462.log.gz 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95705.log 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40330.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99526.log.gz 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96424.log 2024-04-25T00:11:33.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95705.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40330.log.gz 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40856.log 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105004.log 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95705.log.gz 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8%/var/log/ceph/ceph-client.admin.40856.log: -- replaced with /var/log/ceph/ceph-client.admin.96424.log.gz 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100732.log 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40856.log.gz 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87998.log 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63405.log 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105004.log.gz 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100732.log.gz 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73133.log 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87998.log.gz 2024-04-25T00:11:33.407 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78920.log 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94878.log 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63405.log.gz 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.78920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57568.log 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73133.log.gz 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78920.log.gz 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58181.log 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61646.log 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94878.log.gz 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40372.log 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92234.log 89.1% 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58181.log.gz 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57568.log.gz 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40372.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.61646.log.gz -- 2024-04-25T00:11:33.408 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.61987.log 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92234.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40372.log.gz 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59874.log 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61987.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92234.log.gz 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99484.log 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76707.log 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61987.log.gz 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101236.log 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59874.log.gz 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1%/var/log/ceph/ceph-client.admin.76707.log: -- replaced with /var/log/ceph/ceph-client.admin.99484.log.gz 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100350.log 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101236.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76707.log.gz 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88889.log 2024-04-25T00:11:33.409 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100350.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101236.log.gz 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57267.log 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63573.log 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88889.log.gz 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63363.log 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.100350.log.gz 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53707.log 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63363.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63573.log.gz 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41948.log 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57267.log.gz 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63363.log.gz 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100554.log 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53707.log: /var/log/ceph/ceph-client.admin.41948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98339.log 2024-04-25T00:11:33.410 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100554.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41948.log.gz -- 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.64166.log 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53707.log.gz 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100554.log.gz 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44950.log 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93847.log 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98339.log.gz 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64166.log.gz 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83170.log 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44950.log.gz 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.93847.log: -- /var/log/ceph/ceph-client.admin.100470.log 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82577.log 2024-04-25T00:11:33.411 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93847.log.gz 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98887.log 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83170.log.gz 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82577.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.100470.log.gz 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55827.log 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98887.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82577.log.gz 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81511.log 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98887.log.gz 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44992.log 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55827.log.gz 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81511.log.gz 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82962.log 2024-04-25T00:11:33.412 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44992.log.gz 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72304.log 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92715.log 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82962.log.gz 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41454.log 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92715.log: gzip -5 89.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.72304.log.gz -- 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.62891.log 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92715.log.gz 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41454.log.gz 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43855.log 2024-04-25T00:11:33.413 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62891.log.gz 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.41114.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.105449.log 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91684.log 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83256.log 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43855.log.gz 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91684.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.105449.log.gz 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44692.log 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55007.log 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91684.log.gz 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43435.log 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83256.log.gz 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44692.log.gz 2024-04-25T00:11:33.414 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62849.log 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55007.log.gz 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102951.log 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53773.log 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43435.log.gz 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62849.log.gz 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.102951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88339.log 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102951.log.gz 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78624.log 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47032.log 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104734.log 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88339.log.gz 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53773.log.gz 2024-04-25T00:11:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47032.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.78624.log.gz 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78354.log 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105617.log 88.4% 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47032.log.gz 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78354.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104734.log.gz 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55432.log 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78354.log.gz 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.105617.log: /var/log/ceph/ceph-client.admin.42422.log 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57966.log 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105617.log.gz 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42422.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55432.log.gz 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91202.log 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57966.log: gzip -5 --verbose 89.1% -- -- replaced with /var/log/ceph/ceph-client.admin.42422.log.gz 2024-04-25T00:11:33.416 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.65107.log 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65299.log 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.65107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102357.log 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57966.log.gz 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.65299.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65107.log.gz 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.91202.log.gz 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65299.log.gz 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105407.log 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105173.log 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59561.log 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102357.log.gz 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105407.log.gz 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59519.log 2024-04-25T00:11:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43348.log 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105173.log.gz 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95004.log 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59561.log.gz 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8%/var/log/ceph/ceph-client.admin.43348.log: -- replaced with /var/log/ceph/ceph-client.admin.59519.log.gz 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105995.log 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96888.log 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43348.log.gz 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95004.log.gz 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103911.log 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105995.log.gz 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83637.log 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92757.log 2024-04-25T00:11:33.418 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96888.log.gz 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101152.log 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92757.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83637.log.gz 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99820.log 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.103911.log.gz 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81301.log 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92757.log.gz 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99820.log: gzip -5 --verbose 88.1% -- /var/log/ceph/ceph-client.admin.106205.log 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101152.log.gz 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64873.log 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99820.log.gz 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81301.log.gz 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54610.log 2024-04-25T00:11:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88423.log 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106205.log.gz 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54610.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64873.log.gz 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92360.log 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46597.log 87.7% 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54610.log.gz 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88423.log.gz 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77053.log 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46597.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.92360.log.gz -- 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.44428.log 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46597.log.gz 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77053.log.gz/var/log/ceph/ceph-client.admin.44428.log: 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86116.log 2024-04-25T00:11:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43519.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98549.log 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86116.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62576.log 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98549.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86116.log.gz 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.62576.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.98549.log.gz -5 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62576.log.gz 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74448.log 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91070.log 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.74448.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40107.log 2024-04-25T00:11:33.421 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74448.log.gz 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82787.log 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.91070.log.gz 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40107.log.gz 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34805.log 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73858.log 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82787.log: /var/log/ceph/ceph-client.admin.34805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53869.log 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82787.log.gz 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73858.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34805.log.gz 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78582.log 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98035.log 2024-04-25T00:11:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53869.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73858.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41121.log 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98035.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78582.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45364.log 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53869.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54965.log 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98035.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41121.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4%/var/log/ceph/ceph-client.admin.54965.log: -- replaced with /var/log/ceph/ceph-client.admin.45364.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72514.log 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93522.log 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54965.log.gz 2024-04-25T00:11:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72514.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.83595.log 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93522.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72514.log.gz 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82120.log 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93522.log.gz 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93973.log 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87410.log 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83595.log.gz 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82120.log.gz 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83829.log 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87410.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.59695.log 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93973.log.gz 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83829.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87410.log.gz 2024-04-25T00:11:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78308.log 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91478.log 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83829.log.gz 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78308.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59695.log.gz 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40418.log 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91478.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78308.log.gz 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81690.log 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40418.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91478.log.gz 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86501.log 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40418.log.gz 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39857.log 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77179.log 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81690.log.gz 2024-04-25T00:11:33.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39857.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77179.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39857.log.gz 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46776.log 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.86501.log.gz 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72828.log 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77179.log.gz 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105743.log 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46776.log.gz/var/log/ceph/ceph-client.admin.72828.log: 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42284.log 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105743.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.72828.log.gz 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41453.log 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78540.log 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105743.log.gz 2024-04-25T00:11:33.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60383.log 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41453.log.gz 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34924.log 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42284.log.gz 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78540.log.gz 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57634.log 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60383.log: /var/log/ceph/ceph-client.admin.34924.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60383.log.gz 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63783.log 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86633.log 89.0% 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34924.log.gz 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57634.log.gz 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87914.log 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63783.log.gz 2024-04-25T00:11:33.427 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46734.log 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87914.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86633.log.gz 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100512.log 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63279.log 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87914.log.gz 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100512.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46734.log.gz 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74194.log 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100512.log.gz 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100638.log 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.74194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103259.log 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63279.log.gz 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47182.log 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100638.log.gz 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103259.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74194.log.gz 2024-04-25T00:11:33.428 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92528.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71604.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47182.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103259.log.gz 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82745.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47182.log.gz 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92528.log.gz 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.71604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81469.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73644.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81469.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82745.log.gz 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.71604.log.gz 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82264.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81774.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73644.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81469.log.gz 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83553.log 2024-04-25T00:11:33.429 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73644.log.gz 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41906.log 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82264.log.gz 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81774.log.gz 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81169.log 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86759.log 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83553.log.gz 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81169.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41906.log.gz 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82348.log 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106319.log 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82348.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.81169.log.gz 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86759.log.gz 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45574.log 2024-04-25T00:11:33.430 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104506.log 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82348.log.gz 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106319.log: /var/log/ceph/ceph-client.admin.45574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55308.log 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106319.log.gz 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104506.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45574.log.gz 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61185.log 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55308.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104506.log.gz 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55308.log.gz 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55887.log 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86980.log 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55887.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61185.log.gz 2024-04-25T00:11:33.431 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91268.log 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55887.log.gz 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58524.log 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86980.log.gz 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57436.log 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60975.log 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91268.log.gz 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103301.log 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60975.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58524.log.gz 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54138.log 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57436.log.gz 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60975.log.gz/var/log/ceph/ceph-client.admin.103301.log: 2024-04-25T00:11:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63962.log 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103301.log.gz 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56933.log 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63962.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54138.log.gz 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58458.log 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80917.log 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63962.log.gz 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58458.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56933.log.gz 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96103.log 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.80917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83086.log 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96103.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58458.log.gz 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72202.log 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80917.log.gz 2024-04-25T00:11:33.433 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83086.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96103.log.gz 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54054.log 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83086.log.gz 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72202.log.gz 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55781.log 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82535.log 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55781.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54054.log.gz 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98171.log 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55781.log.gz 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44776.log 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82535.log.gz 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74152.log 2024-04-25T00:11:33.434 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98171.log.gz 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47074.log 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44776.log: /var/log/ceph/ceph-client.admin.74152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76623.log 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44776.log.gz 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74152.log.gz 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94269.log 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40730.log 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76623.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47074.log.gz 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.94269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103845.log 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76623.log.gz 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.94269.log.gz 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104386.log 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:11:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103845.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40730.log.gz 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61772.log 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83721.log 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61772.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.103845.log.gz 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76581.log 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.104386.log.gz 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61772.log.gz 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93104.log 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83721.log.gz 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88093.log 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93104.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.76581.log.gz 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45196.log 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100158.log 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93104.log.gz 2024-04-25T00:11:33.436 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88093.log.gz 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74452.log 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45196.log.gz 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.100158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78173.log 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.100158.log.gz 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.74452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39761.log 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74452.log.gz 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45118.log 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78173.log.gz 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62572.log 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45118.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39761.log.gz 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60849.log 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45118.log.gz 2024-04-25T00:11:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.62572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95663.log 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62572.log.gz 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78456.log 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63447.log 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60849.log.gz 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78456.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95663.log.gz 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55929.log 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78456.log.gz 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39938.log 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55929.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63447.log.gz 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63489.log 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39938.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55929.log.gz 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87326.log 2024-04-25T00:11:33.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63489.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39938.log.gz 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102315.log 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42158.log 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63489.log.gz 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63699.log 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87326.log.gz 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42158.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102315.log.gz 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83469.log 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42158.log.gz 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64208.log 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42032.log 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63699.log.gz 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83469.log.gz 2024-04-25T00:11:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98255.log 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64208.log.gz 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102399.log 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42032.log: /var/log/ceph/ceph-client.admin.98255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61101.log 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42032.log.gz 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102399.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98255.log.gz 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103715.log 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102399.log.gz 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.61101.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45616.log 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56805.log 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61101.log.gz 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45616.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103715.log.gz 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82661.log 2024-04-25T00:11:33.440 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45616.log.gz 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62975.log 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103757.log 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56805.log.gz 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62975.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82661.log.gz 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54694.log 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62975.log.gz 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103662.log 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73942.log 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103757.log.gz 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103662.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54694.log.gz 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102525.log 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103662.log.gz 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87494.log 2024-04-25T00:11:33.441 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44065.log 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73942.log.gz 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87494.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102525.log.gz 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103385.log 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87494.log.gz 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94351.log 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63321.log 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44065.log.gz 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94351.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103385.log.gz 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103803.log 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56397.log 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.94351.log.gz 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103803.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63321.log.gz 2024-04-25T00:11:33.442 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65041.log 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56995.log 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103803.log.gz 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.56397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56397.log.gz 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.65041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95518.log 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97412.log 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65041.log.gz 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40520.log 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99568.log 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97412.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95518.log.gz 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56995.log.gz 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71821.log 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97412.log.gz 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40520.log.gz 2024-04-25T00:11:33.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92402.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.71821.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99568.log.gz 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41120.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.3.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71821.log.gz 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92402.log.gz 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78924.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81085.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.78924.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41120.log.gz 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78924.log.gz 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82390.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81085.log: /var/log/ceph/ceph-client.admin.82390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82078.log 2024-04-25T00:11:33.444 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81085.log.gz 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62318.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82390.log.gz 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42200.log 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82078.log.gz 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78732.log 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91972.log 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42200.log.gz 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41822.log 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78732.log.gz 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63195.log 2024-04-25T00:11:33.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91972.log: /var/log/ceph/ceph-client.admin.41822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92318.log 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91972.log.gz 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63195.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41822.log.gz 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87536.log 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63195.log.gz 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87106.log 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92318.log: /var/log/ceph/ceph-client.admin.87536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99970.log 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92318.log.gz 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87536.log.gz 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59029.log 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87106.log.gz 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93392.log 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99970.log: /var/log/ceph/ceph-client.admin.59029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76312.log 2024-04-25T00:11:33.446 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99970.log.gz 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58276.log 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59029.log.gz 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93392.log.gz 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105575.log 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98129.log 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58276.log: /var/log/ceph/ceph-client.admin.105575.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.76312.log.gz 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94015.log 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58276.log.gz 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98129.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105575.log.gz 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96846.log 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98129.log.gz 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59207.log 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62807.log 2024-04-25T00:11:33.447 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96846.log: /var/log/ceph/ceph-client.admin.59207.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.94015.log.gz 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72244.log 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96846.log.gz 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56611.log 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62807.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59207.log.gz 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72244.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62807.log.gzgzip 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76927.log 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64380.log 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1%/var/log/ceph/ceph-client.admin.56611.log: -- replaced with /var/log/ceph/ceph-client.admin.72244.log.gz 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96468.log 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76927.log.gz 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56611.log.gz 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97891.log 2024-04-25T00:11:33.448 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73379.log 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64380.log.gz 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96468.log.gz 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59071.log 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73379.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.97891.log.gz --verbose 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr: -- /var/log/ceph/ceph-client.admin.82180.log 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59071.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73379.log.gz 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45929.log 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59071.log.gz 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56205.log 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82180.log.gz 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58402.log 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45929.log: /var/log/ceph/ceph-client.admin.56205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41117.log 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45929.log.gz 2024-04-25T00:11:33.449 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58402.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56205.log.gz 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59958.log 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91624.log 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41117.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58402.log.gz 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55516.log 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41117.log.gz 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59958.log.gz 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98507.log 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91624.log: /var/log/ceph/ceph-client.admin.55516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95098.log 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91624.log.gz 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98507.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55516.log.gz 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59393.log 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98507.log.gz 2024-04-25T00:11:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55390.log 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95098.log.gz 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77835.log 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59393.log: /var/log/ceph/ceph-client.admin.55390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105869.log 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59393.log.gz 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77835.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55390.log.gz 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104344.log 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77835.log.gz 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65377.log 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105869.log: /var/log/ceph/ceph-client.admin.104344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105869.log.gz 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.65377.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104344.log.gz 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72785.log 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65377.log.gz 2024-04-25T00:11:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39980.log 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46469.log: /var/log/ceph/ceph-client.admin.72785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41118.log 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39980.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72785.log.gz 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45280.log 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39980.log.gz 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61251.log 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41118.log: /var/log/ceph/ceph-client.admin.45280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54568.log 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41118.log.gz 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61251.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45280.log.gz 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55558.log 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61251.log.gz 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104776.log 2024-04-25T00:11:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54568.log: /var/log/ceph/ceph-client.admin.55558.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54568.log.gz 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104776.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55558.log.gz 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83384.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72472.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.c.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.104776.log.gz 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88974.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6%/var/log/ceph/ceph-client.admin.72472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95350.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83384.log.gz 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34544.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72472.log.gz 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88974.log.gz 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58945.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93350.log 2024-04-25T00:11:33.453 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95350.log.gz 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58945.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.34544.log.gz /var/log/ceph/ceph-client.admin.98381.log 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93350.log: gzip -5 87.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.58945.log.gz /var/log/ceph/ceph-client.admin.81343.log 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93350.log.gz 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55739.log 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81343.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98381.log.gz 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98633.log 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81343.log.gz 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88931.log 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55739.log.gz 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-04-25T00:11:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88931.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98633.log.gz 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56013.log 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88931.log.gz 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54736.log 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56013.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101423.log 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54736.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56013.log.gz 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93745.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54736.log.gz 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72870.log 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93745.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101423.log.gz 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43813.log 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72870.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93745.log.gz 2024-04-25T00:11:33.455 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94675.log 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72870.log.gz 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60681.log 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94675.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43813.log.gz 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41451.log 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60681.log: gzip -5 --verbose -- 91.0% /var/log/ceph/ceph-client.admin.102105.log 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94675.log.gz 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41451.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60681.log.gz 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104818.log 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41451.log.gz 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55228.log 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96636.log 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102105.log.gz 2024-04-25T00:11:33.456 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8%/var/log/ceph/ceph-client.admin.55228.log: -- replaced with /var/log/ceph/ceph-client.admin.104818.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.47370.log 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45742.log 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55228.log.gz 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96636.log.gz 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.47370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102483.log 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47370.log.gz 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45742.log: /var/log/ceph/ceph-client.admin.71940.log 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102483.log: gzip -5 --verbose -- 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45742.log.gz 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.89353.log 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102483.log.gz 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.71940.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71940.log.gz 2024-04-25T00:11:33.457 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.89353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92883.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89353.log.gz 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91136.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92883.log.gz 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41458.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46990.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40772.log: 87.8% 91.4% -- replaced with /var/log/ceph/ceph-client.admin.91136.log.gz 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41458.log.gz 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83679.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.46990.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54484.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76969.log 2024-04-25T00:11:33.458 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46990.log.gz 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4%/var/log/ceph/ceph-client.admin.54484.log: -- replaced with /var/log/ceph/ceph-client.admin.83679.log.gz 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85971.log 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101296.log 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54484.log.gz 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.85971.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76969.log.gz 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85971.log.gz 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86285.log 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5/var/log/ceph/ceph-client.admin.101296.log: --verbose -- /var/log/ceph/ceph-client.admin.92662.log 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86285.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72108.log 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101296.log.gz 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104608.log 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.86285.log.gz 2024-04-25T00:11:33.459 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72108.log: /var/log/ceph/ceph-client.admin.92662.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60137.log 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72108.log.gz 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92662.log.gz 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44023.log 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104608.log.gz/var/log/ceph/ceph-client.admin.60137.log: 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56097.log 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3%/var/log/ceph/ceph-client.admin.44023.log: -- replaced with /var/log/ceph/ceph-client.admin.60137.log.gz 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88173.log 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44023.log.gz 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58764.log 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99862.log 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88173.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56097.log.gz 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58764.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88173.log.gz 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101849.log 2024-04-25T00:11:33.460 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96804.log 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99862.log.gz 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58764.log.gz 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr:gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101849.log.gz 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80875.log 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73602.log 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96804.log.gz 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.80875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86938.log 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80875.log.gz 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73602.log: gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73602.log.gz 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102609.log 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94741.log 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86938.log.gz/var/log/ceph/ceph-client.admin.102609.log: 2024-04-25T00:11:33.461 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102609.log.gz 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54358.log 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94741.log.gz 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64789.log 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81994.log 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54358.log.gz 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92056.log 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81994.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64789.log.gz 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43645.log 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81994.log.gz 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105131.log 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43645.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92056.log.gz 2024-04-25T00:11:33.462 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43645.log.gz 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95182.log 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54923.log 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105131.log: /var/log/ceph/ceph-client.admin.95182.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105131.log.gz 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95182.log.gz 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83300.log 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74026.log 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54923.log.gz 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88604.log 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83300.log.gz 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.74026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55971.log 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74026.log.gz 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45238.log 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55971.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88604.log.gz 2024-04-25T00:11:33.463 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76843.log 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55971.log.gz 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93203.log 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76843.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45238.log.gz 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76843.log.gz 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86801.log 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55646.log 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86801.log: /var/log/ceph/ceph-client.admin.93203.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93203.log.gz 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86801.log.gz 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87788.log 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78043.log 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55646.log.gz 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91310.log 2024-04-25T00:11:33.464 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87788.log.gz 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96010.log 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7%/var/log/ceph/ceph-client.admin.91310.log: -- replaced with /var/log/ceph/ceph-client.admin.78043.log.gz 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58722.log 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91310.log.gz/var/log/ceph/ceph-client.admin.96010.log: 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62216.log 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96010.log.gz 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82493.log 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62216.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58722.log.gz 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62216.log.gz 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100774.log 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102189.log 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82493.log.gz 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102441.log 2024-04-25T00:11:33.465 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102189.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100774.log.gz 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102189.log.gz 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81553.log 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5/var/log/ceph/ceph-client.admin.102441.log: --verbose -- /var/log/ceph/ceph-client.admin.40646.log 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102441.log.gz 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62671.log 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40646.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81553.log.gz 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43771.log 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40646.log.gz 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96678.log 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43771.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.62671.log.gz 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43771.log.gz 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87578.log 2024-04-25T00:11:33.466 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92799.log 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96678.log.gz/var/log/ceph/ceph-client.admin.87578.log: 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82440.log 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87578.log.gz/var/log/ceph/ceph-client.admin.92799.log: 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73732.log 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92799.log.gz 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106247.log 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73732.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82440.log.gz 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73732.log.gz 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63153.log 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81858.log 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106247.log.gz 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44860.log 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81858.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63153.log.gz 2024-04-25T00:11:33.467 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64431.log 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81858.log.gz 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99904.log 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64431.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44860.log.gz 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64431.log.gz 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41446.log 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59832.log 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41446.log: 91.4% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41446.log.gz 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99904.log.gz 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91835.log 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88777.log 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59832.log.gz 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88562.log 2024-04-25T00:11:33.468 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88777.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91835.log.gz 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87620.log 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88777.log.gz 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58355.log 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87620.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88562.log.gz 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87620.log.gz 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45826.log 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.58355.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45826.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45826.log.gz 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.58355.log.gz 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100858.log 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80959.log 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56677.log 2024-04-25T00:11:33.469 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.80959.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100858.log.gz 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40562.log 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80959.log.gz 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93308.log 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40562.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56677.log.gz 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40562.log.gz 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57314.log 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78001.log 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93308.log.gz/var/log/ceph/ceph-client.admin.57314.log: 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76444.log 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78001.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57314.log.gz 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46343.log 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78001.log.gz 2024-04-25T00:11:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78396.log 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46343.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76444.log.gz 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46343.log.gz 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40235.log 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97618.log 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40235.log: /var/log/ceph/ceph-client.admin.78396.log: 89.5% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78396.log.gz 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40235.log.gz 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98591.log 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63237.log 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98591.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.97618.log.gz 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98591.log.gz 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98759.log 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74110.log 2024-04-25T00:11:33.471 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63237.log.gz 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89270.log 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.74110.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98759.log.gz 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99778.log 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74110.log.gz 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.89270.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99270.log 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.0.89270.log.gz/var/log/ceph/ceph-client.admin.99778.log: 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88297.log 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99778.log.gz 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100816.log 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88297.log: 88.7% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99270.log.gz 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88297.log.gz 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41457.log 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89395.log 2024-04-25T00:11:33.472 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100816.log.gz 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72996.log 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.89395.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41457.log.gz 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55049.log 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89395.log.gz 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96720.log 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55049.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72996.log.gz 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55049.log.gz 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62276.log 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.96720.log: -- /var/log/ceph/ceph-client.admin.45322.log 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96720.log.gz 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87284.log 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62276.log.gz 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57078.log 2024-04-25T00:11:33.473 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8%/var/log/ceph/ceph-client.admin.87284.log: -- replaced with /var/log/ceph/ceph-client.admin.45322.log.gz 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40604.log 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87284.log.gz/var/log/ceph/ceph-client.admin.57078.log: 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106359.log 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57078.log.gz 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106359.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40604.log.gz 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85929.log 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106359.log.gz 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64250.log 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.85929.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85929.log.gz 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77576.log 2024-04-25T00:11:33.474 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46511.log 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64250.log.gz 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43729.log 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77576.log.gz 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101026.log 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46511.log.gz/var/log/ceph/ceph-client.admin.43729.log: 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64336.log 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101026.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98929.log 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101026.log.gz 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99312.log 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98929.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64336.log.gz 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98929.log.gz 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44317.log 2024-04-25T00:11:33.475 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92925.log 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99312.log.gz 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54786.log 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92925.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44317.log.gz 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96996.log 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92925.log.gz 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63825.log 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96996.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54786.log.gz 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96996.log.gz 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60299.log 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63825.log: -- /var/log/ceph/ceph-client.admin.104860.log 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63825.log.gz 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41456.log 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104860.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60299.log.gz 2024-04-25T00:11:33.476 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93805.log 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104860.log.gz 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96212.log 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93805.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41456.log.gz 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93805.log.gz 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63741.log 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96212.log.gz/var/log/ceph/ceph-mon.a.log: 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46259.log 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63741.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63741.log.gz 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95434.log 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46259.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46259.log.gz 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59645.log 2024-04-25T00:11:33.477 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95434.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95434.log.gz 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77263.log 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59645.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59645.log.gz 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80833.log 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77263.log.gz 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104128.log 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.80833.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80833.log.gz 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86031.log 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104128.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.104128.log.gz 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40191.log 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86031.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86031.log.gz 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82222.log 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40191.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40191.log.gz 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97188.log 2024-04-25T00:11:33.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82222.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82222.log.gz 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71538.log 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.97188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97188.log.gz 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43207.log 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.71538.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.71538.log.gz 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57676.log 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.2.43207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43207.log.gz 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82703.log 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57676.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.57676.log.gz 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101508.log 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82703.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82703.log.gz 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98717.log 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101508.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101508.log.gz 2024-04-25T00:11:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93931.log 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98717.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98717.log.gz 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60807.log 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93931.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93931.log.gz 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60807.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60807.log.gz 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101338.log 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45971.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44149.log 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101338.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101338.log.gz 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99610.log 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44149.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44149.log.gz 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64621.log 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99610.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99610.log.gz 2024-04-25T00:11:33.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64621.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64621.log.gz 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97468.log 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46174.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94459.log 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97468.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.97468.log.gz 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41738.log 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94459.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.94459.log.gz 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101891.log 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41738.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41738.log.gz 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89274.log 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101891.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101891.log.gz 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73295.log 2024-04-25T00:11:33.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.89274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89274.log.gz 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92967.log 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73295.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73295.log.gz 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80767.log 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92967.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92967.log.gz 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92486.log 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.80767.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.80767.log.gz 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46864.log 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92486.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92486.log.gz 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73686.log 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46864.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46864.log.gz 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105215.log 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73686.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73686.log.gz 2024-04-25T00:11:33.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97975.log 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105215.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105215.log.gz 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71670.log 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97975.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97975.log.gz 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93266.log 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.71670.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71670.log.gz 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73900.log 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93266.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93266.log.gz 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103564.log 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73900.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73900.log.gz 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81595.log 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103564.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.103564.log.gz 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101110.log 2024-04-25T00:11:33.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81595.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81595.log.gz 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105533.log 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101110.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101110.log.gz 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92841.log 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105533.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105533.log.gz 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64915.log 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92841.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92841.log.gz 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45076.log 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64915.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64915.log.gz 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44734.log 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45076.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45076.log.gz 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104278.log 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44734.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44734.log.gz 2024-04-25T00:11:33.484 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74260.log 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104278.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.104278.log.gz 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86073.log 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.74260.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74260.log.gz 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86073.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86073.log.gz 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92276.log 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41864.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93657.log 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92276.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92276.log.gz 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72598.log 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93657.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93657.log.gz 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60597.log 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72598.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72598.log.gz 2024-04-25T00:11:33.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95266.log 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60597.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60597.log.gz 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86417.log 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95266.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.95266.log.gz 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43939.log 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86417.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86417.log.gz 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98213.log 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43939.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43939.log.gz 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95831.log 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98213.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98213.log.gz 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47116.log 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95831.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95831.log.gz 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101725.log 2024-04-25T00:11:33.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47116.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47116.log.gz 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84021.log 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101725.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101725.log.gz 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53935.log 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.84021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84021.log.gz 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77137.log 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53935.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53935.log.gz 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72430.log 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77137.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77137.log.gz 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92444.log 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72430.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72430.log.gz 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101550.log 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92444.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92444.log.gz 2024-04-25T00:11:33.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64705.log 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101550.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101550.log.gz 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72346.log 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64705.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64705.log.gz 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73816.log 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72346.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72346.log.gz 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93889.log 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73816.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73816.log.gz 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81816.log 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93889.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93889.log.gz 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72066.log 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81816.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81816.log.gz 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55697.log 2024-04-25T00:11:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72066.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72066.log.gz 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87452.log 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55697.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55697.log.gz 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103343.log 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87452.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87452.log.gz 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55602.log 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103343.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103343.log.gz 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83763.log 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55602.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55602.log.gz 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76378.log 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83763.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.83763.log.gz 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101933.log 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76378.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.76378.log.gz 2024-04-25T00:11:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89016.log 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101933.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101933.log.gz 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42765.log 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.89016.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.89016.log.gz 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65295.log 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.42765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42765.log.gz 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106037.log 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.65295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65295.log.gz 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77959.log 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106037.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106037.log.gz 2024-04-25T00:11:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53632.log 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77959.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77959.log.gz 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94920.log 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53632.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53632.log.gz 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61017.log 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94920.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94920.log.gz 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64831.log 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61017.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61017.log.gz 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64747.log 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64831.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64831.log.gz 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64579.log 2024-04-25T00:11:33.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64747.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64747.log.gz 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100984.log 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64579.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64579.log.gz 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103527.log 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100984.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100984.log.gz 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62384.log 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103527.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.103527.log.gz 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101194.log 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62384.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62384.log.gz 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44275.log 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101194.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101194.log.gz 2024-04-25T00:11:33.492 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105953.log 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44275.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44275.log.gz 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95789.log 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105953.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105953.log.gz 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91742.log 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95789.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95789.log.gz 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73774.log 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91742.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91742.log.gz 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60639.log 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73774.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73774.log.gz 2024-04-25T00:11:33.493 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101380.log 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60639.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60639.log.gz 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44233.log 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101380.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101380.log.gz 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81385.log 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44233.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44233.log.gz 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101645.log 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73984.log 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81385.log: /var/log/ceph/ceph-client.admin.101645.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81385.log.gz 2024-04-25T00:11:33.494 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101645.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72743.log 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73984.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73984.log.gz 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86459.log 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40023.log 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72743.log: /var/log/ceph/ceph-client.admin.86459.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72743.log.gz 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86459.log.gz 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76180.log 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40023.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40023.log.gz 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102947.log 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46051.log 2024-04-25T00:11:33.495 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76180.log: /var/log/ceph/ceph-client.0.102947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102947.log.gz 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76180.log.gz 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57370.log 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83342.log 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46051.log: /var/log/ceph/ceph-client.admin.57370.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46051.log.gz 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61730.log 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0%/var/log/ceph/ceph-client.admin.83342.log: -- replaced with /var/log/ceph/ceph-client.admin.57370.log.gz 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83342.log.gz 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77450.log 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60555.log 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61730.log: /var/log/ceph/ceph-client.admin.77450.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61730.log.gz 2024-04-25T00:11:33.496 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.77450.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.87368.log 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60555.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60555.log.gz 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93573.log 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93615.log 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87368.log: /var/log/ceph/ceph-client.admin.93573.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87368.log.gz 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63069.log 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93573.log.gz 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93615.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93615.log.gz 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54222.log 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2024-04-25T00:11:33.497 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63069.log: /var/log/ceph/ceph-client.admin.54222.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63069.log.gz 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54222.log.gz 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61143.log 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43561.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98465.log 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54180.log 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61143.log: /var/log/ceph/ceph-client.admin.98465.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61143.log.gz 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98465.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40065.log 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.498 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54180.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54180.log.gz 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93434.log 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87872.log 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40065.log: /var/log/ceph/ceph-client.admin.93434.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40065.log.gz 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93434.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78085.log 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87872.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87872.log.gz 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95308.log 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64999.log 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78085.log: /var/log/ceph/ceph-client.admin.95308.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78085.log.gz 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95308.log.gz 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101975.log 2024-04-25T00:11:33.499 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64999.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64999.log.gz 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102651.log 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56055.log 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101975.log: /var/log/ceph/ceph-client.admin.102651.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101975.log.gz 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102651.log.gz 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82036.log 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56055.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56055.log.gz 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86675.log 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54400.log 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82036.log: /var/log/ceph/ceph-client.admin.86675.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82036.log.gz 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86675.log.gz 2024-04-25T00:11:33.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42116.log 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59351.log 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54400.log: /var/log/ceph/ceph-client.admin.42116.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54400.log.gz 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42116.log.gz 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106079.log 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81043.log 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106079.log: /var/log/ceph/ceph-client.admin.59351.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59351.log.gz 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106079.log.gz 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71982.log 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61604.log 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81043.log: /var/log/ceph/ceph-client.admin.71982.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81043.log.gz 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94081.log 2024-04-25T00:11:33.501 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71982.log.gz 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77492.log 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61604.log: /var/log/ceph/ceph-client.admin.94081.log: 88.9% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61604.log.gz 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94081.log.gz 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78224.log 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.77492.log: -- /var/log/ceph/ceph-client.admin.61688.log 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77492.log.gz 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101466.log 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61688.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78224.log.gz 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61688.log.gz 2024-04-25T00:11:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58098.log 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99104.log 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101466.log.gz 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88692.log 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99104.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58098.log.gz 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99104.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99398.log 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.88692.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60467.log 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88692.log.gz 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97184.log 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60467.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99398.log.gz 2024-04-25T00:11:33.503 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60467.log.gz 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87064.log 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.97184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97184.log.gz 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77755.log 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87064.log.gz 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72954.log 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77755.log.gz 2024-04-25T00:11:33.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47374.log 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45532.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72954.log.gz 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77011.log 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62933.log 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77011.log: /var/log/ceph/ceph-client.0.47374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47374.log.gz 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77011.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63867.log 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46641.log 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63867.log: /var/log/ceph/ceph-client.admin.62933.log: 88.0% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62933.log.gz 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63867.log.gz 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72556.log 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41113.log 2024-04-25T00:11:33.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46641.log: /var/log/ceph/ceph-client.admin.72556.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46641.log.gz 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72556.log.gz 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91930.log 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58656.log 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41113.log: /var/log/ceph/ceph-client.admin.91930.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41113.log.gz 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91930.log.gz 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104902.log 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96762.log 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104902.log: /var/log/ceph/ceph-client.admin.58656.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104902.log.gz 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61059.log 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3%/var/log/ceph/ceph-client.admin.96762.log: -- replaced with /var/log/ceph/ceph-client.admin.58656.log.gz 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.96762.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.86843.log 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64533.log 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61059.log: /var/log/ceph/ceph-client.admin.86843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102019.log 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61059.log.gz 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64533.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86843.log.gz 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64533.log.gz 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56867.log 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100900.log 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102019.log.gz 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102759.log 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.56867.log.gz 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43393.log 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102759.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100900.log.gz 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82619.log 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102759.log.gz 2024-04-25T00:11:33.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56549.log 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz/var/log/ceph/ceph-client.admin.82619.log: 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86717.log 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82619.log.gz/var/log/ceph/ceph-client.admin.56549.log: 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56549.log.gz 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73421.log 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58590.log 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86717.log: /var/log/ceph/ceph-client.admin.73421.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86717.log.gz 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73421.log.gz 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61443.log 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58590.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58590.log.gz 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105911.log 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.61443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61443.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72024.log 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.508 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93703.log 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72024.log: /var/log/ceph/ceph-client.admin.105911.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72024.log.gz 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105911.log.gz 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88646.log 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93703.log: gzip 87.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.45886.log 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93703.log.gz 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84017.log 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45886.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88646.log.gz 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45886.log.gz 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41780.log 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94417.log 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.84017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84017.log.gz 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41696.log 2024-04-25T00:11:33.509 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41780.log.gz 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94417.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28464.log 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41696.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94417.log.gz 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45784.log 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41696.log.gz 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.tmp-client.admin.28464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97389.log 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45784.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28464.log.gz 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97534.log 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45784.log.gz 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88734.log 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97534.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97389.log.gz 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102063.log 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.97534.log.gz 2024-04-25T00:11:33.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59309.log 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88734.log.gz 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43203.log 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59309.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102063.log.gz 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59309.log.gz 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105785.log 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.2.43203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43203.log.gz 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87242.log 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46553.log 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105785.log: /var/log/ceph/ceph-client.admin.87242.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105785.log.gz 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87242.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96510.log 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100284.log 2024-04-25T00:11:33.511 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46553.log: /var/log/ceph/ceph-client.admin.96510.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96510.log.gz 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46553.log.gz 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43687.log 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76749.log 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100284.log: /var/log/ceph/ceph-client.admin.43687.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43687.log.gz 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.100284.log.gz 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76486.log 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103620.log 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76749.log.gz 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96338.log 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103620.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76486.log.gz 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103620.log.gz 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87022.log 2024-04-25T00:11:33.512 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85731.log 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96338.log.gz/var/log/ceph/ceph-client.admin.87022.log: 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45700.log 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87022.log.gz/var/log/ceph/ceph-client.admin.85731.log: 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.85731.log.gz 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57502.log 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104650.log 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2%/var/log/ceph/ceph-client.admin.57502.log: -- replaced with /var/log/ceph/ceph-client.admin.45700.log.gz 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95915.log 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104650.log: 89.3% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57502.log.gz 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104650.log.gz 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77314.log 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98423.log 2024-04-25T00:11:33.513 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95915.log: /var/log/ceph/ceph-client.admin.77314.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95915.log.gz 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77314.log.gz 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40279.log 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91436.log 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98423.log: /var/log/ceph/ceph-client.admin.40279.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40279.log.gz 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.98423.log.gz 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73337.log 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43897.log 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91436.log.gz 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59267.log 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43897.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73337.log.gz 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43897.log.gz 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86201.log 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.59267.log: -- /var/log/ceph/ceph-client.admin.83212.log 2024-04-25T00:11:33.514 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59267.log.gz 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85797.log 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83212.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86201.log.gz 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83212.log.gz 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54316.log 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.85797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102693.log 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54316.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.85797.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54316.log.gz 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58987.log 2024-04-25T00:11:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91394.log 2024-04-25T00:11:33.516 INFO:teuthology.orchestra.run.smithi178.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T00:11:33.547 INFO:teuthology.orchestra.run.smithi178.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T00:11:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102693.log: /var/log/ceph/ceph-client.admin.58987.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58987.log.gz 2024-04-25T00:11:34.064 INFO:teuthology.orchestra.run.smithi113.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.102693.log.gz 2024-04-25T00:11:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55091.log 2024-04-25T00:11:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104944.log 2024-04-25T00:11:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91394.log: /var/log/ceph/ceph-client.admin.55091.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91394.log.gz 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55091.log.gz 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61985.log 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98845.log 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61985.log: /var/log/ceph/ceph-client.admin.104944.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104944.log.gz 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61985.log.gz 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104062.log 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105827.log 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98845.log: /var/log/ceph/ceph-client.admin.104062.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98845.log.gz 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59790.log 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9%/var/log/ceph/ceph-client.admin.105827.log: -- replaced with /var/log/ceph/ceph-client.admin.104062.log.gz 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40149.log 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59790.log: -- replaced with /var/log/ceph/ceph-client.admin.105827.log.gz 2024-04-25T00:11:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56139.log 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59790.log.gz 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54652.log 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56139.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40149.log.gz 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80635.log 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56139.log.gz 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98297.log 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6%/var/log/ceph/ceph-client.admin.80635.log: -- replaced with /var/log/ceph/ceph-client.admin.54652.log.gz 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64663.log 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.80635.log.gz 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77618.log 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64663.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98297.log.gz 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:gzip 89.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.100596.log 2024-04-25T00:11:34.066 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64663.log.gz 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95873.log 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100596.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77618.log.gz 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100596.log.gz 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62154.log 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81127.log 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95873.log.gz 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.62154.log: gzip 88.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.72388.log 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62154.log.gz 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102273.log 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81127.log: /var/log/ceph/ceph-client.admin.72388.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72388.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81127.log.gz 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99736.log 2024-04-25T00:11:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72640.log 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102273.log: /var/log/ceph/ceph-client.admin.99736.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102273.log.gz 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99736.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105089.log 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63111.log 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72640.log: /var/log/ceph/ceph-client.admin.105089.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105089.log.gz 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72640.log.gz 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81235.log 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41654.log 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63111.log.gz 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104194.log 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41654.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.81235.log.gz 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41654.log.gz 2024-04-25T00:11:34.068 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99354.log 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.104194.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.87704.log 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99354.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.104194.log.gz 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99354.log.gz 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78129.log 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61986.log 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87704.log.gz/var/log/ceph/ceph-client.admin.78129.log: 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106121.log 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61986.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78129.log.gz 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99652.log 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61986.log.gz 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99024.log 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99652.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106121.log.gz 2024-04-25T00:11:34.069 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99652.log.gz 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93478.log 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98803.log 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99024.log: /var/log/ceph/ceph-client.admin.93478.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99024.log.gz 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93478.log.gz 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88845.log 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60511.log 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98803.log: /var/log/ceph/ceph-client.admin.88845.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98803.log.gz 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88845.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104566.log 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60511.log: /var/log/ceph/ceph-client.admin.104566.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104566.log.gz 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60511.log.gz 2024-04-25T00:11:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102567.log 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72690.log 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82306.log 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72690.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102567.log.gz 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72690.log.gz 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60891.log 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83128.log 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60891.log: /var/log/ceph/ceph-client.admin.82306.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82306.log.gz 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60891.log.gz 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59603.log 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45490.log 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83128.log: /var/log/ceph/ceph-client.admin.59603.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83128.log.gz 2024-04-25T00:11:34.071 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59603.log.gz 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54526.log 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86243.log 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45490.log: /var/log/ceph/ceph-client.admin.54526.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45490.log.gz 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97701.log 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54526.log.gz 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60765.log 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97701.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86243.log.gz 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97701.log.gz 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94836.log 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87956.log 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60765.log: /var/log/ceph/ceph-client.admin.94836.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60765.log.gz 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87200.log 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94836.log.gz 2024-04-25T00:11:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45034.log 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87200.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87956.log.gz 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87200.log.gz 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105047.log 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54442.log 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45034.log: /var/log/ceph/ceph-client.admin.105047.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45034.log.gz 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105047.log.gz 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77366.log 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94273.log 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54442.log: /var/log/ceph/ceph-client.admin.77366.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54442.log.gz 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98675.log 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77366.log.gz 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.94273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96254.log 2024-04-25T00:11:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.98675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.94273.log.gz 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57188.log 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98675.log.gz 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77917.log 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57188.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96254.log.gz 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57188.log.gz 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64292.log 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77917.log.gz 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72912.log 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4%/var/log/ceph/ceph-client.admin.43603.log: -- replaced with /var/log/ceph/ceph-client.admin.64292.log.gz 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85863.log 2024-04-25T00:11:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.72912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57742.log 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72912.log.gz 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.85863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92014.log 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57742.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57742.log.gz 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.85863.log.gz 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91540.log 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59916.log 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92014.log.gz 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41119.log 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59916.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91540.log.gz 2024-04-25T00:11:34.075 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46427.log 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59916.log.gz 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41119.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41119.log.gz 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95747.log 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55474.log 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46427.log: /var/log/ceph/ceph-client.admin.95747.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46427.log.gz 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41122.log 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95747.log.gz 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86351.log 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41122.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55474.log.gz 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41122.log.gz 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73213.log 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95578.log 2024-04-25T00:11:34.076 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86351.log: /var/log/ceph/ceph-client.admin.73213.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73213.log.gz 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.86351.log.gz 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76665.log 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94609.log 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95578.log.gz 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76665.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94609.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76665.log.gz 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83511.log 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.94609.log.gz 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89082.log 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83511.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83511.log.gz 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61849.log 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.89082.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89082.log.gz 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59748.log 2024-04-25T00:11:34.077 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61849.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61849.log.gz 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56739.log 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59748.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59748.log.gz 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64124.log 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56739.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56739.log.gz 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54839.log 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64124.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64124.log.gz 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88511.log 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54839.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54839.log.gz 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96594.log 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88511.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88511.log.gz 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96594.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96594.log.gz 2024-04-25T00:11:34.078 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77095.log 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41990.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60042.log 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77095.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77095.log.gz 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96930.log 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60042.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60042.log.gz 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95621.log 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96930.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.96930.log.gz 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86567.log 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95621.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95621.log.gz 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103217.log 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86567.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.86567.log.gz 2024-04-25T00:11:34.079 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61988.log 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103217.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103217.log.gz 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73463.log 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61988.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61988.log.gz 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73038.log 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73463.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73463.log.gz 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82882.log 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73038.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73038.log.gz 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.82882.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82882.log.gz 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73551.log 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45658.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42242.log 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73551.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73551.log.gz 2024-04-25T00:11:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59477.log 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42242.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42242.log.gz 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86159.log 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59477.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59477.log.gz 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61439.log 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.86159.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86159.log.gz 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83044.log 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.61439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61439.log.gz 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45406.log 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.83044.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83044.log.gz 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61872.log 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45406.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45406.log.gz 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91352.log 2024-04-25T00:11:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.61872.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.61872.log.gz 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73507.log 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91352.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91352.log.gz 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81427.log 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.73507.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73507.log.gz 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95392.log 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81427.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81427.log.gz 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63657.log 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95392.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95392.log.gz 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92570.log 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63657.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63657.log.gz 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63615.log 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92570.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92570.log.gz 2024-04-25T00:11:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88255.log 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63615.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63615.log.gz 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91793.log 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88255.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88255.log.gz 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88381.log 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91793.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91793.log.gz 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63531.log 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88381.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88381.log.gz 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106401.log 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.63531.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63531.log.gz 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87746.log 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.106401.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106401.log.gz 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54096.log 2024-04-25T00:11:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87746.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87746.log.gz 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76885.log 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54096.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54096.log.gz 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76246.log 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76885.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76885.log.gz 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.76246.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.76246.log.gz 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60723.log 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43981.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105659.log 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60723.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60723.log.gz 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59435.log 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105659.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105659.log.gz 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58830.log 2024-04-25T00:11:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.59435.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59435.log.gz 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40688.log 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58830.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58830.log.gz 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60000.log 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40688.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40688.log.gz 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64957.log 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60000.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60000.log.gz 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101068.log 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64957.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64957.log.gz 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40814.log 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101068.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101068.log.gz 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44470.log 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40814.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40814.log.gz 2024-04-25T00:11:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77660.log 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44470.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44470.log.gz 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55133.log 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77660.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77660.log.gz 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46906.log 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55133.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55133.log.gz 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100942.log 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46906.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46906.log.gz 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99228.log 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.100942.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100942.log.gz 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99442.log 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99228.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99228.log.gz 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87830.log 2024-04-25T00:11:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99442.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99442.log.gz 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81732.log 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.87830.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87830.log.gz 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103977.log 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81732.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81732.log.gz 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96380.log 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.103977.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.103977.log.gz 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99694.log 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96380.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96380.log.gz 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40460.log 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.99694.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99694.log.gz 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40460.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40460.log.gz 2024-04-25T00:11:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78498.log 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47452.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80701.log 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78498.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78498.log.gz 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93009.log 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.80701.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.80701.log.gz 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71755.log 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.93009.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93009.log.gz 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60341.log 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.71755.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71755.log.gz 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56393.log 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60341.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60341.log.gz 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77534.log 2024-04-25T00:11:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.56393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56393.log.gz 2024-04-25T00:11:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91582.log 2024-04-25T00:11:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77534.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77534.log.gz 2024-04-25T00:11:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102147.log 2024-04-25T00:11:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.91582.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91582.log.gz 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101807.log 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102147.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102147.log.gz 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96296.log 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.101807.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101807.log.gz 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77408.log 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96296.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96296.log.gz 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46301.log 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.77408.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77408.log.gz 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46301.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46301.log.gz 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40940.log 2024-04-25T00:11:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44554.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100162.log 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40940.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40940.log.gz 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42898.log 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.100162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.100162.log.gz 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46948.log 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42898.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42898.log.gz 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104692.log 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46948.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46948.log.gz 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57838.log 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.104692.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104692.log.gz 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88467.log 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57838.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57838.log.gz 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92192.log 2024-04-25T00:11:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88467.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88467.log.gz 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81001.log 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92192.log.gz 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57900.log 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.81001.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81001.log.gz 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105491.log 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57900.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57900.log.gz 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94525.log 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105491.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105491.log.gz 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.94525.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.94525.log.gz 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97849.log 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45448.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58032.log 2024-04-25T00:11:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97849.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97849.log.gz 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105701.log 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58032.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58032.log.gz 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64491.log 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.105701.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105701.log.gz 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102231.log 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64491.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64491.log.gz 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64042.log 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.102231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102231.log.gz 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60217.log 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.64042.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64042.log.gz 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96552.log 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60217.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60217.log.gz 2024-04-25T00:11:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95140.log 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.96552.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96552.log.gz 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78266.log 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.95140.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95140.log.gz 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92612.log 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.78266.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78266.log.gz 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97724.log 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92612.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92612.log.gz 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54881.log 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.97724.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.97724.log.gz 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74068.log 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54881.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54881.log.gz 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42760.log 2024-04-25T00:11:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.74068.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74068.log.gz 2024-04-25T00:11:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46822.log 2024-04-25T00:11:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.42760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42760.log.gz 2024-04-25T00:11:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92098.log 2024-04-25T00:11:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46822.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46822.log.gz 2024-04-25T00:11:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60933.log 2024-04-25T00:11:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.92098.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92098.log.gz 2024-04-25T00:11:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.60933.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60933.log.gz 2024-04-25T00:11:34.141 INFO:teuthology.orchestra.run.smithi113.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T00:11:34.729 INFO:teuthology.orchestra.run.smithi178.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T00:11:34.965 INFO:teuthology.orchestra.run.smithi178.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T00:11:35.081 INFO:teuthology.orchestra.run.smithi113.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T00:11:35.142 INFO:teuthology.orchestra.run.smithi178.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T00:11:35.213 INFO:teuthology.orchestra.run.smithi113.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T00:11:35.394 INFO:teuthology.orchestra.run.smithi113.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T00:11:35.506 INFO:teuthology.orchestra.run.smithi178.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T00:11:35.598 INFO:teuthology.orchestra.run.smithi113.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T00:11:35.855 INFO:teuthology.orchestra.run.smithi178.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T00:11:35.856 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-04-25T00:11:35.856 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m2.804s 2024-04-25T00:11:35.856 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m11.419s 2024-04-25T00:11:35.856 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.386s 2024-04-25T00:11:55.424 INFO:teuthology.orchestra.run.smithi113.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T00:11:59.630 INFO:teuthology.orchestra.run.smithi113.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T00:11:59.632 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-25T00:11:59.632 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m26.608s 2024-04-25T00:11:59.632 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m56.877s 2024-04-25T00:11:59.633 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m2.367s 2024-04-25T00:11:59.633 INFO:tasks.ceph:Archiving logs... 2024-04-25T00:11:59.633 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/remote/smithi113/log 2024-04-25T00:11:59.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:12:09.878 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/remote/smithi178/log 2024-04-25T00:12:09.879 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:12:11.339 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:12:11.349 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:12:11.349 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:12:11.352 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:12:11.439 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:12:11.454 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:12:11.455 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:12:11.457 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:12:11.471 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:12:11.471 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:12:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:12:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-04-25T00:12:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:^- hv01.front.sepia.ceph.com 2 6 377 12 +876us[ +876us] +/- 590ms 2024-04-25T00:12:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -6811ns[+2107ns] +/- 62ms 2024-04-25T00:12:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +35us[ +44us] +/- 84ms 2024-04-25T00:12:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:12:11.477 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:12:11.477 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-25T00:12:11.477 INFO:teuthology.orchestra.run.smithi113.stdout:^- hv01.front.sepia.ceph.com 2 6 377 16 +885us[ +899us] +/- 590ms 2024-04-25T00:12:11.477 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 +4740ns[ +19us] +/- 61ms 2024-04-25T00:12:11.477 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +18us[ +18us] +/- 84ms 2024-04-25T00:12:11.477 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:12:11.477 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:12:11.499 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:12:11.499 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:12:11.511 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:12:11.610 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:12:11.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:12:11.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:12:11.701 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:12:11.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:12:11.727 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:12:11.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:12:11.782 INFO:teuthology.task.internal:Duration was 1200.386207 seconds 2024-04-25T00:12:11.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:12:11.794 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:12:11.795 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:12:11.797 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:12:11.831 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:12:11.833 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:12:12.128 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:12:12.128 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-04-25T00:12:12.129 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:12:12.147 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-04-25T00:12:12.147 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:12:12.187 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:12:12.187 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:12:12.190 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:12:12.252 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:12:12.252 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:12:12.255 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:12:12.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:12:12.449 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:12:12.450 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:12:12.473 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:12:12.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:12:12.527 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:12:12.530 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:12:12.549 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-04-25T00:12:12.559 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-04-25T00:12:12.572 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:12:12.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:12:12.617 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:12:12.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:12:12.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:12:12.674 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:12:12.675 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/remote/smithi113 2024-04-25T00:12:12.675 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:12:12.709 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672227/remote/smithi178 2024-04-25T00:12:12.710 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:12:12.742 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:12:12.742 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:12:12.745 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:12:12.791 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:12:12.801 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:12:12.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:12:12.888 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:12:12.888 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:12:12.890 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:12:12.902 INFO:teuthology.orchestra.run.smithi113.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:12 /home/ubuntu/cephtest 2024-04-25T00:12:12.904 INFO:teuthology.orchestra.run.smithi178.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:12 /home/ubuntu/cephtest 2024-04-25T00:12:12.906 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:12:12.915 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:12:12.943 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/scrub} duration: 1200.3862073421478 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:12:12.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:12:13.022 INFO:teuthology.run:pass